DPChallenge: A Digital Photography Contest You are not logged in. (log in or register
 

DPChallenge Forums >> Challenge Results >> Should low voters have to justify their marks???
Pages:  
Showing posts 76 - 89 of 89, (reverse)
AuthorThread
10/09/2004 11:14:08 PM · #76
Originally posted by zeuszen:

Originally posted by mavrik:

...I'm not bothering to vote then unless I'm on VC...


Lol. Come on, be a sport. ;-)


Ok, I'm basketball.

:)

M
10/09/2004 11:30:30 PM · #77
I like this approach as long as the voter can see his/her karma score and as long as it shows up in a member's profile for all to see.

Originally posted by EddyG:

Originally posted by SDW65:

It would be very hard for DPC to do anything that would improve the voting system that we already have.

In my opinion, I'm not sure this is necessarily true. Although I've brought this up in the past (read my posts in this thread and this thread), I'll mention it briefly again for those that are new. Other sites address the issue of "unreasonable votes" through the concept of "voter karma".

Basically, it is an automated system that determines how much your vote will count towards the final results. (Right now on DPC, every vote counts the same, unless the "vote scrubber" determines you are voting too many pictures with 1's or 2's in which case all your votes in that challenge are ignored.) If you vote reasonably, you karma is "high" and your votes count like they do today. But if your voting habits show that you vote unreasonably, your votes count less and less (until you start to vote reasonably again).

Whenever this concept is brought up, people cry "No way! That system sucks! If I want to vote a picture a 1 I should be able to!" without realizing how the whole system works. You need to understand deviation, karma, etc. and how the various aspects of participating on the site affect those things. For example, your karma would also increase when you vote on more than the required minimum: you'd get some amount for voting on 50%, a little more for 75% and even more for voting on 100% of the challenge entries. Same thing with comments.

And when you vote, just because you gave a ribbon-winner a 1 does not mean that next time your votes would count for less, as long as you voted reasonably on the majority of the other images. The karma "ding" on that image would be offset by the karma calculations on the other images in the challenge. The system that Worth1000 uses to determine whether your vote is reasonable is simply based on your vote being within +/- 3 of the final score. So if the ribbon winner scored a 7.000, you could have ranked that picture anywhere from a 10 to a 4 and you would be considered to have voted reasonably -- plenty of latitude to express your "personal" feelings about the photo. It obviously must have some redeeming qualities (which voters should learn to recognize) to have won a ribbon. Likewise, if the last-place image had a 4.000, you could vote it anywhere from 7 to 1 and be considered to be voting reasonably on that image. So if someone had their friends sign up and give that image a 10 and all the rest a 1, their scores would have much less of impact on the final results because of the "unreasonable" natures of their votes.

Quoting from the Worth1000 FAQ:

"This is not a system of punishment and reward, nor is it a judgement of your personal tastes. It's merely a way to measure your ability to discern actual image quality, and allows plenty of room for your individual tastes to come into play without damaging your karma. It maximizes the effects that consistantly fair and observant voters have on an entry's average, and minimizes the effects that 'quirky' or downright dishonest voters can have. Trust us. It works."

I don't want to re-hash all of the specifics again here; you can read the above-referenced threads for more information and previous reaction.

On an unrelated note: I am all for having scores hidden until the results are posted, and have been a proponent of that for a long time. I do think seeing your current score influences "self-serving" voters.
10/09/2004 11:52:34 PM · #78
Originally posted by Digital Quixote:

I like this approach as long as the voter can see his/her karma score and as long as it shows up in a member's profile for all to see.

Originally posted by EddyG:

Originally posted by SDW65:

It would be very hard for DPC to do anything that would improve the voting system that we already have.

In my opinion, I'm not sure this is necessarily true. Although I've brought this up in the past (read my posts in this thread and this thread), I'll mention it briefly again for those that are new. Other sites address the issue of "unreasonable votes" through the concept of "voter karma".

Basically, it is an automated system that determines how much your vote will count towards the final results. (Right now on DPC, every vote counts the same, unless the "vote scrubber" determines you are voting too many pictures with 1's or 2's in which case all your votes in that challenge are ignored.) If you vote reasonably, you karma is "high" and your votes count like they do today. But if your voting habits show that you vote unreasonably, your votes count less and less (until you start to vote reasonably again).

Whenever this concept is brought up, people cry "No way! That system sucks! If I want to vote a picture a 1 I should be able to!" without realizing how the whole system works. You need to understand deviation, karma, etc. and how the various aspects of participating on the site affect those things. For example, your karma would also increase when you vote on more than the required minimum: you'd get some amount for voting on 50%, a little more for 75% and even more for voting on 100% of the challenge entries. Same thing with comments.

And when you vote, just because you gave a ribbon-winner a 1 does not mean that next time your votes would count for less, as long as you voted reasonably on the majority of the other images. The karma "ding" on that image would be offset by the karma calculations on the other images in the challenge. The system that Worth1000 uses to determine whether your vote is reasonable is simply based on your vote being within +/- 3 of the final score. So if the ribbon winner scored a 7.000, you could have ranked that picture anywhere from a 10 to a 4 and you would be considered to have voted reasonably -- plenty of latitude to express your "personal" feelings about the photo. It obviously must have some redeeming qualities (which voters should learn to recognize) to have won a ribbon. Likewise, if the last-place image had a 4.000, you could vote it anywhere from 7 to 1 and be considered to be voting reasonably on that image. So if someone had their friends sign up and give that image a 10 and all the rest a 1, their scores would have much less of impact on the final results because of the "unreasonable" natures of their votes.

Quoting from the Worth1000 FAQ:

"This is not a system of punishment and reward, nor is it a judgement of your personal tastes. It's merely a way to measure your ability to discern actual image quality, and allows plenty of room for your individual tastes to come into play without damaging your karma. It maximizes the effects that consistantly fair and observant voters have on an entry's average, and minimizes the effects that 'quirky' or downright dishonest voters can have. Trust us. It works."

I don't want to re-hash all of the specifics again here; you can read the above-referenced threads for more information and previous reaction.

On an unrelated note: I am all for having scores hidden until the results are posted, and have been a proponent of that for a long time. I do think seeing your current score influences "self-serving" voters.


I like this approach as well. Good idea it should be brought up to the admin's
10/10/2004 05:19:59 AM · #79
weights could be a solution.

like this:

your weight = the sum of your votes to the 10 best divided by the sum of average votes they get

or something like it
10/10/2004 05:42:56 AM · #80
I think the voting format we have now is fine. 1 - 10 averaged works and always has. One single vote has little weight whether it's a 1 or a 10 but as group of voters we end up with a consensus.

As for a 1 given to the winner of the masters challenge. I think a 1 could have been given legitimately. If you look at the photograph in question you will see a line around the castle where the burning of the sky doesn't quite meet the castle. Secondly there are people who don't like the over burnt sky effect. I can see how someone can look at the photograph and see how good the original must have been and in their opinion thought it was spoilt by the editting and gave it a 1.

The 1s were given legitimately or not but the photograph won. Doesn't that mean the system is working?
10/10/2004 08:50:43 AM · #81
Originally posted by EddyG:


Quoting from the Worth1000 FAQ:

"This is not a system of punishment and reward, nor is it a judgement of your personal tastes. It's merely a way to measure your ability to discern actual image quality,"


The flaw in this approach is right here - it assumes that the majority are somehow excellent at discerning actual image quality. All schemes like this actually do is evaluate how far your opinion lies from the standard group think, and diminishes your impact the more you disagree with the site norm over time.

There is nothing in this that is about actual image quality, just about how average your opinion is, compared to everyone else. One thing DPC really does not need is more homogeneity in the voting process. In fact, probably for growth, the site needs to move to a way that promotes new ideas, more creative styles etc, rather than a scheme that would further re-enforce the current dpc stylistic modes.
10/10/2004 11:34:03 AM · #82
Originally posted by timj351:

Originally posted by coolhar:

In my scheme for voting I make an effort to decide which are the best and the worst entries in every challenge, and therefore, give at least one 1 and one 10 almost every time. Anyone want to psychoanaylze that?


I don't agree with this at all. It has always been my understanding that we are supposed to simply judge each photo individually on it's own merits, period. The order of the photos will be determined electronically and that is not for us to try to determine. I don't think one score should be at all related to another score and that goes for subject matter too if you are truly trying to vote objectively. Sure, individual biasis are going to creep in (some people really dislike puppy pictures) but you should, at least, be trying to overcome them and be attempting to vote for the quality of the image as apposed to your personal feelings of the subject matter.

T


timj- I don't see what it is that you disagree with. I do try to judge each entry on it's own merits, to vote objectively. The random order in which they appear has no bearing on my vote. I don't think there is anything in what I said that indicates I let my vote on one image influence my vote on another image. Nor is there anything to indicate that I am not trying to suppress my personal biases.

I just want to use the whole 1 thru 10 voting scale in each challenge. I have seen forum posts where people say that they never, or very rarely, give a vote of 10. That strikes me as elitist, sort of like saying "I'm so knowledgable about digital photography that none of these pictures are up to my standard of excellence". Others have said that they reserve their 10's for the truely outstanding images that only come along once a year, or so. Fine, but what if I miss that once a year special image because I'm on vacation, or just didn't get to vote on all the entries in a large challenge? My interpretation is that the voting scale is 1 thru 10 for each of the challenges, not for all the photos you will ever see online in your lifetime. Will anyone dispute that there is a best image in every challenge? You may not agree with the voters choice as to which is the best one, but do you disagree that each voter has the right to choose which they think is the best? and also the worst? A case could be made that it is the voters who do not award a 10 and a 1 in every challenge are the voters who are skewing the results.

Message edited by author 2004-10-10 11:37:44.
10/10/2004 01:27:31 PM · #83
Originally posted by Gordon:

Originally posted by EddyG:


Quoting from the Worth1000 FAQ:

"This is not a system of punishment and reward, nor is it a judgement of your personal tastes. It's merely a way to measure your ability to discern actual image quality,"


The flaw in this approach is right here - it assumes that the majority are somehow excellent at discerning actual image quality. All schemes like this actually do is evaluate how far your opinion lies from the standard group think, and diminishes your impact the more you disagree with the site norm over time.

There is nothing in this that is about actual image quality, just about how average your opinion is, compared to everyone else. One thing DPC really does not need is more homogeneity in the voting process. In fact, probably for growth, the site needs to move to a way that promotes new ideas, more creative styles etc, rather than a scheme that would further re-enforce the current dpc stylistic modes.


This is the very crux of the matter, and I dearly hope that its gets sufficient notice and consideration to bear on the discussion.
10/10/2004 01:32:59 PM · #84
Perhaps a karma system could be developed which weights most heavily those whose votes are consistently between 1 and 2 standard deviations from the final average vote -- those with a broad perspective but not on the fringe.

Or we could leave it like it is but discuss it incessantly : )
10/10/2004 01:58:55 PM · #85
Originally posted by GeneralE:

Perhaps a karma system could be developed which weights most heavily those whose votes are consistently between 1 and 2 standard deviations from the final average vote -- those with a broad perspective but not on the fringe.

Or we could leave it like it is but discuss it incessantly : )


I'd prefer the 'incessant discussion' approach, which you, likely, posed here in jest, mainly because of its inclusive engagement of all kinds of views and experiences as opposed to implementing the artifice of a 'scheme' to skew the skewers. While the 'scheme' may well accomplish what its was set out to do via its broad sweep, I question the benefit of rendering all disparate votes invalid or diminishing their relative weight.

I don't mind suffering a lil chaos, if the struggle goes some place that doesn't have to be eternally reviewed and revised.

Message edited by author 2004-10-10 16:17:28.
10/10/2004 02:46:34 PM · #86
Originally posted by GeneralE:

Perhaps a karma system could be developed which weights most heavily those whose votes are consistently between 1 and 2 standard deviations from the final average vote -- those with a broad perspective but not on the fringe.

Or we could leave it like it is but discuss it incessantly : )


I think this kind of system would work even though some do not. The reasoning behind this is we have high voters and we have low voters and thatâs ok as long as they vote on 100% of the entries in the challenge. People have the right to vote the way they wish, set their own scale of voting, but they have to be consistent and vote 100% if they donât then it will cause incessant problem.

Message edited by author 2004-10-10 14:48:11.
10/10/2004 03:20:16 PM · #87
MY LAST POST ON VOTING, I HOPE.
Here is the problem when voters donât use the complete scale of 1 to 10. Or if they are a low voter or high voter donât vote on 100% of the entries.
Lets say you have two Great photographs and they are competing for the blue ribbon. The first picture we will call picture âAâ and the second picture we will call picture âBâ.

Picture âAâ stats:
Votes 250
Average 7.2500

Picture âBâ Stats:
Votes 250
Average 7.2300

A low voter sees picture âAâ and thinks it is great and votes it a 4 because she/he has their voting system set up so that a 4 is consider a good picture. But does not vote on 100% of the challenge and over look Picture âBâ. Now lets look at the averages.

Picture âAâ stats:
Votes 251
Average 7.2370

Picture âBâ Stats:
Votes 250
Average 7.2300

Even though the low voter gave what she/he thinks to be a good score on the picture, the average has gone down and the margin has tightened.
Now lets look and see what happens when we introduce a high voter to picture âBâ at the same time.
A high votes sees picture âBâ and thinks it is a great picture and votes it 9. But does not vote on 100% of the challenge and over look picture âAâ. Lets see the out come.

Picture âAâ stats:
Votes 251
Average 7.2370

Picture âBâ Stats:
Votes 251
Average 7.3506

In my conclusion I think voters can vote how they wish after all it is there interpretation of the photograph. Some may vote low, some may vote high, and some may use the complete â1 to 10â scale. BUT if we donât vote on 100% of the entries some great photographs will suffer. I donât have to worry about this with my photographs because Iâm not to the level of most of the photographers here. But for the ones that are great photographers deserve a fair vote. And the only way to do this with our current system is vote 100% of the entries.
10/10/2004 04:50:20 PM · #88
Originally posted by coolhar:


timj- I don't see what it is that you disagree with. I do try to judge each entry on it's own merits, to vote objectively. The random order in which they appear has no bearing on my vote. I don't think there is anything in what I said that indicates I let my vote on one image influence my vote on another image. Nor is there anything to indicate that I am not trying to suppress my personal biases.

I just want to use the whole 1 thru 10 voting scale in each challenge. I have seen forum posts where people say that they never, or very rarely, give a vote of 10. That strikes me as elitist, sort of like saying "I'm so knowledgable about digital photography that none of these pictures are up to my standard of excellence". Others have said that they reserve their 10's for the truely outstanding images that only come along once a year, or so. Fine, but what if I miss that once a year special image because I'm on vacation, or just didn't get to vote on all the entries in a large challenge? My interpretation is that the voting scale is 1 thru 10 for each of the challenges, not for all the photos you will ever see online in your lifetime. Will anyone dispute that there is a best image in every challenge? You may not agree with the voters choice as to which is the best one, but do you disagree that each voter has the right to choose which they think is the best? and also the worst? A case could be made that it is the voters who do not award a 10 and a 1 in every challenge are the voters who are skewing the results.


Coolhar, you make an interesting argument for your voting method and even though I now understand your point better I still disagree with it. I'm accepting the fact that these challenges are to be considered at a very high level of quality (afterall, these are open challenges that involve professionals as well as amateurs from all over the world) and I use this to create my standard of quality that I apply to all of the challenges and I vote accordingly. Since I use this standard of quality I have no need to impose a requirement of creating a numbering range of 1 to 10 for all of the challenges. I do try to be reasonable in my standards so that I will still award several tens but often I will not give out any ones. In fact, I try not to pay any attention as to whether I have evenly spread out my votes because I don't think that is relevant. As I stated previously these voting decisions are based solely on an individual basis according to my own set of standards. My standards are not going to change when the quality of the photos in a particular challenge is lower or higher than usual.

This is just my belief in how the system is supposed to work but obviously this is going to vary between members and even with a great diversity in voting methods I still think it works just fine. I find this discussion very interesting but, in the end, I don't think anything is broke.

T
10/10/2004 05:22:14 PM · #89
Originally posted by zeuszen:

Originally posted by GeneralE:

... Or we could leave it like it is but discuss it incessantly : )


I'd prefer the 'incessant discussion' approach, which you, likely, posed here in jest ...

Actually no, in this case it is my preferred option for much the same reasons as you list. Just trying to be realistic about the consequences of that choice ...

Message edited by author 2004-10-10 17:22:32.
Pages:  
Current Server Time: 07/28/2025 04:14:22 AM

Please log in or register to post to the forums.


Home - Challenges - Community - League - Photos - Cameras - Lenses - Learn - Help - Terms of Use - Privacy - Top ^
DPChallenge, and website content and design, Copyright © 2001-2025 Challenging Technologies, LLC.
All digital photo copyrights belong to the photographers and may not be used without permission.
Current Server Time: 07/28/2025 04:14:22 AM EDT.