| Author | Thread |
|
|
09/14/2005 11:04:37 AM · #1 |
I would like to see Avg (Members) and Avg (Reg Users) added to the voting stats on a photo. In my opinion, this would be insightful in that it would give me an idea how these two segments of voters reacted to my photos in the open challenges. If necessary, we could drop stats Avg (camera) and Avg (no camera) which aren't very helpful to me.
Currently the stats look like this:
Place: XXX out of XXX
Avg (all users): X.XXXX
Avg (commenters): X.XXXX
Avg (camera): X.XXXX
Avg (no camera): X.XXXX
Views since voting: XXX
Views during voting: XXX
Votes: XXX
Comments: XXX
Favorites: XX (view)
This is how they might look:
Place: XXX out of XXX
Avg (all users): X.XXXX
Avg (commenters): X.XXXX
Avg (members): X.XXXX
Avg (reg users): X.XXXX
Views since voting: XXX
Views during voting: XXX
Votes: XXX
Comments: XXX
Favorites: XX (view)
Would anyone else find this helpful? |
|
|
|
09/14/2005 11:06:37 AM · #2 |
I think Male Vs Female Vs No-gender would also be interesting. Age brackets too.
|
|
|
|
09/14/2005 11:09:43 AM · #3 |
I'd be curious to see Pro versus NonPro, or categorize it by $$$ earned in photography in the previous calendar year, in ranges, 0-$100, $100-1000, $1000-$10000, $10000+., the latter number really reflecting "pro".
|
|
|
|
09/14/2005 11:20:56 AM · #4 |
Originally posted by nshapiro:
I'd be curious to see Pro versus NonPro, or categorize it by $$$ earned in photography in the previous calendar year, in ranges, 0-$100, $100-1000, $1000-$10000, $10000+., the latter number really reflecting "pro". |
On a similar note, according to the voter's Avg Score Received at the time of voting, if they've entered a pre-set number of challenges to steady-out the number.
|
|
|
|
09/14/2005 11:31:00 AM · #5 |
Before any of those other suggested statistics, I personally feel the most revealing one would be
Avg (vested interest): X.XXXX
Avg (just a voter): X.XXXX
to shed more light on the "self-serving voter" problem. I can't think of any other competition where the actual participants can directly affect the final score of other competitors. As an analogy, if the Miss America pageant was scored like DPC, the girls would vote on all the other girls (along with the non-participant judges), and be able to see their current score, before having to score the other contestants.
The problem is made worse because participants can see their current score during the voting process... which results in the (potentially subconscious) behavior of "I'm only getting a 5.7 so I'm not going to vote anybody else's picture higher than a 5" syndrome in an attempt to increase their own final ranking. I've long been a proponent of hiding participant's scores and comments until they click an "I've finished voting, lock my votes" button. (They could, of course, continue to add comments after clicking said button.)
And for those that don't think "self-serving voting" is a problem, I have lies, damn lies and more damn lies, erh, I mean statistics, to back it up. When I was on the SC, I analyzed a metric boatload of historical data and came up with this chart:
Which shows that, at the time I did the analysis, participants (those with a vested interested) voted lower than those without a vested interest on 94% of the challenges. (And the 6% where this didn't happen was largely in the very early days of the site, where challenges had a couple dozen entries.) And the difference is not all that miniscule. On average, if vested-interest votes were removed, final scores would have been higher by over a quarter point (i.e., a 5.75 entry would have ended over a 6) with the trend showing that difference increasing.
Message edited by author 2005-09-14 11:53:57. |
|
|
|
09/14/2005 12:35:55 PM · #6 |
Eddy, your math tends to confirm what we have all suspected all along. Good job on the math, by the way!
As long as vested interest voters vote a range of scores, and as long as they give the best scores to the best photos, their bias won't disrupt the law of averages and the best photos will still average to the top.
Among vested interest voters, did you find evidence that the best photos were given poorer scores than poorer photos? This would manifest itself if voters w/o a vested interest tended to give a photo 8's, 9's and 10's, say, and vested interest voters gave the same photos 1's, 2's, and 3's. Or did vested interest voters score in a range of 1-5 fairly, and voters w/o a vested interest score over a range of 1-10 ro 5-10, or ...?
I am still interested in the Member & Registered User stats. Without trying to diminish anyone, I believe that members on average and in the aggregate are more experienced and more knowledgeable photographers and editors. I believe registered users on average and in the aggregate are newer to photography and photoediting. While I am sure there are exceptions, I would find the voting stats of the two segments of voters insightful.
Message edited by author 2005-09-14 12:49:59. |
|
|
|
09/14/2005 12:44:59 PM · #7 |
As long as the results were relatively consistent, it doesn't matter all that much.
I think a lot of other factors contribute as much (or more) to photos not getting their "true" score:
ΓΆ€ΒΆ lack of standardized voting guidelines
ΓΆ€ΒΆ ability to "cherry-pick" entries to vote on, instead of true randomization
ΓΆ€ΒΆ low quorum of votes needed to qualify (20%)
ΓΆ€ΒΆ variation in monitor quality/calibration
ΓΆ€ΒΆ (list your factor here) |
|
|
|
09/14/2005 01:29:54 PM · #8 |
Originally posted by EddyG:
Before any of those other suggested statistics, I personally feel the most revealing one would be
Avg (vested interest): X.XXXX
Avg (just a voter): X.XXXX
to shed more light on the "self-serving voter" problem. I can't think of any other competition where the actual participants can directly affect the final score of other competitors. As an analogy, if the Miss America pageant was scored like DPC, the girls would vote on all the other girls (along with the non-participant judges), and be able to see their current score, before having to score the other contestants.
The problem is made worse because participants can see their current score during the voting process... which results in the (potentially subconscious) behavior of "I'm only getting a 5.7 so I'm not going to vote anybody else's picture higher than a 5" syndrome in an attempt to increase their own final ranking. I've long been a proponent of hiding participant's scores and comments until they click an "I've finished voting, lock my votes" button. (They could, of course, continue to add comments after clicking said button.)
And for those that don't think "self-serving voting" is a problem, I have lies, damn lies and more damn lies, erh, I mean statistics, to back it up. When I was on the SC, I analyzed a metric boatload of historical data and came up with this chart:
Which shows that, at the time I did the analysis, participants (those with a vested interested) voted lower than those without a vested interest on 94% of the challenges. (And the 6% where this didn't happen was largely in the very early days of the site, where challenges had a couple dozen entries.) And the difference is not all that miniscule. On average, if vested-interest votes were removed, final scores would have been higher by over a quarter point (i.e., a 5.75 entry would have ended over a 6) with the trend showing that difference increasing. |
Vestered interest is an interesting stat but this can also have a lot of factors that cause these statistics.For example challenges I have tried to enter but could not get the right image I have always voted higher, my reasoning is if I could not meet the challenge then therefore the challenge was too difficult and entries deserve a higher vote range because of the difficulty of the challenge.
A classic example of this was the nude self portriat.
|
|
|
|
09/14/2005 01:57:09 PM · #9 |
Originally posted by keegbow:
Vestered interest is an interesting stat but this can also have a lot of factors that cause these statistics.For example challenges I have tried to enter but could not get the right image I have always voted higher, my reasoning is if I could not meet the challenge then therefore the challenge was too difficult and entries deserve a higher vote range because of the difficulty of the challenge.
A classic example of this was the nude self portriat. |
Voting in this challenge was probably quite skewed, regardless ... |
|
|
|
09/14/2005 02:27:33 PM · #10 |
Originally posted by GeneralE:
Originally posted by keegbow:
Vestered interest is an interesting stat but this can also have a lot of factors that cause these statistics.For example challenges I have tried to enter but could not get the right image I have always voted higher, my reasoning is if I could not meet the challenge then therefore the challenge was too difficult and entries deserve a higher vote range because of the difficulty of the challenge.
A classic example of this was the nude self portriat. |
Voting in this challenge was probably quite skewed, regardless ... |
I was thinking the same thing for some reason... =o |
|
|
|
09/14/2005 10:20:56 PM · #11 |
I know that I would make too much of the graph. Especially without knowing more about the data.
First - most of the decline happened in the first 20% or so of the values - then there is a hill up to about the 50% of entries point. I don't see any trend at all in the last half of the entries.
Second - when did the member only challenges start on the graph? Clearly the vote level of the member only vs the open challenges show a different scale. Does this effect the graph and if it does what is the effect.
Third - how has the proportion of voters who enter vs does who do not enter a challenge changed over time?
Fourth - with the increased number of entries - is the voting on only the better images that at least some belive happens effecting the graph.
I could go on....
Larry C
A statistician who views all numbers with suspicion and view statistic with even more suspicion. |
|
|
|
09/14/2005 11:02:14 PM · #12 |
Originally posted by EddyG:
Before any of those other suggested statistics, I personally feel the most revealing one would be
Avg (vested interest): X.XXXX
Avg (just a voter): X.XXXX |
I have a confession - I'm human.
When I've entered, and I think I have a strong entry, my votes are heavily skewed. If I'm scoring a 4.x, any image that's not as good as mine in my biased opinon (which might happen to be a large percentage) automatically gets a 4 or less. If I'm scoring a 5.x, I happily hand out 5s to average entries.
When I dont enter challenges because I couldnt find a good idea, I vote with an average of atleast 6.
Originally posted by EddyG:
I've long been a proponent of hiding participant's scores and comments until they click an "I've finished voting, lock my votes" button. (They could, of course, continue to add comments after clicking said button.) |
This is the most sensible solution to the problem above. I cant think of good reasons to oppose this. Except for the same old - 'nothing is broken, dont fix it. I hate any change whatsoever.'
Another good idea was to divide the 2 open challenges into 'enter one, vote in the other'.
Message edited by author 2005-09-14 23:03:07.
|
|
Home -
Challenges -
Community -
League -
Photos -
Cameras -
Lenses -
Learn -
Help -
Terms of Use -
Privacy -
Top ^
DPChallenge, and website content and design, Copyright © 2001-2025 Challenging Technologies, LLC.
All digital photo copyrights belong to the photographers and may not be used without permission.
Current Server Time: 07/20/2025 04:26:15 PM EDT.
