| Author | Thread |
|
|
02/06/2011 08:30:33 PM · #26 |
Originally posted by SDW:
This is what turned 2006-2007 around! |
ooh!! Those are the stats I want to see. How do the average/high/low votes compare on the DPL months vs the non-DPL months.
|
|
|
|
02/06/2011 08:37:56 PM · #27 |
Originally posted by PGerst:
So, a few points here... |
Hi Paul,
Here are a few answers:
1) On weighted average, I wanted to be sensitive to "averaging averages", i.e. a simple average of the average of an image with 300 votes and an image with only 100 votes isn't completely accurate. Here̢۪s the concept:
Link
If you (or anyone else) would like to discuss it more I̢۪d be glad to take it to the private messages. For the record, the difference between weighted average and average are pretty minor for the discussion purposes here.
2) Interesting point of the improvements of cameras ~2006; I̢۪m open to any theories folks have on the shift because it appears to be statistically significant.
3) It̢۪s been a while since I̢۪ve used any multivariate techniques. As I said I̢۪m not a statistician :-). I̢۪d be open to discussing it though.
4) Your model sounds interesting. In a later section I̢۪m going to talk about predicting the finishing position of an image based on a known / expected final average but what you propose sounds like it will go to a much higher level. Voting is so subjective I can't imagine how it could be done but I̢۪d love to hear your theory.
5) Robert expressed my motivation well. I really do think the voting process is fundamentally fair. It̢۪s not perfect but it has a good balance of ease of use and open participation which ensures continued voting and value to determine ribbon winners, progress of a user over time, etc. I think that the fear of Trolls and unfair voting practices is overstated. I really detest the persecution of people who vote outside the norm. I feel that they are entitled to their own style of voting and usually are balanced by others equally outside the norm in another direction. These discussions won̢۪t prove the fairness conclusively, but I hope to use some data to show why I think the process is fair and in some cases just to share some interesting statistics found in the DPC history data.
6) The database contains the following data for 246,760 images submitted to 1314 challenges between 2002 and 2010:
Challenge ID
Challenge Title
Challenge Rules
Challenge Type
Submission Dates
Image Name
User Name
Average, (Final Score)
Average, Commenters
Average, Participants
Average, Non Participants
# of Votes
Finishing Position
And some fields calculated from the data above, i.e. Weighted Average
|
|
|
|
02/06/2011 08:40:48 PM · #28 |
Originally posted by SDW:
This is what I believe turned 2006-2007 around! I may be wrong but participation increased during the time of the WPL (now called the DPL). I don't know if the FSM or the implementation of the monthly free studies have increased the scores from 2007 to present; but I believe on or both has contributed. |
Another good theory! Is there specific data available of what challenges have been in DPL versus those that were not? |
|
|
|
02/06/2011 08:41:50 PM · #29 |
I guess high-level curiosity is the only way to look at it. Statistics are good, but special care should be given about drawing conclusions from them. Stats are very good at pointing out "hey, there is something out of the norm here worth looking at", but rarely on their own show which attribute is 'at fault'. I can't tell you how many times I've analyzed people's data for which points "A" and "B" are causing the wrong conclusion, but actually are the two 'correct' points.
The one thing I have seen, although not surprising, is the change in the average over time. This should be expected as the site develops. In fact, how people vote is influenced by many things: software enhancements, cost of hardware, participation in forums, side challenges, DPL, etc.
Voting participation is high enough so the CLT is satisfied (see my previous posts on overplaying the average). In the end, the total votes within a challenge determines who gets a blue, red, yellow ribbon.
However, decoupled completely from the 'average score' is the question: "Would this blue ribbon winner from ages ago, win today". With the advancements I mentioned above, probably not.
Which leads to what I think a problem statement is: "How do I win a blue". The answer is not in the analysis of the averages, but in the analysis of images themselves.
Originally posted by Bear_Music:
1. As far as I know, Del's felt for a long time that there's been a lot of irrational nonsense tossed around regarding voting, so-called trolls, effect of low votes on scores, etc etc, so he decided to see if he could drill down to the facts and draw valid conclusions. I think that's his motivation, I guess it qualifies as high-level curiosity?
|
|
|
|
|
02/06/2011 08:51:04 PM · #30 |
Hmm...not entirely true. If I have time later, I'll send you a PM to discuss more. But, since its posted here, you need to keep in mind the central limit theorem. Once an image receives 80 votes, its mean and distribution have well been characterized. Direct comparison of an image with 100 votes to one with 300 votes is valid.
Originally posted by DJWoodward:
1) On weighted average, I wanted to be sensitive to "averaging averages", i.e. a simple average of the average of an image with 300 votes and an image with only 100 votes isn't completely accurate. Here̢۪s the concept:
Link
If you (or anyone else) would like to discuss it more I̢۪d be glad to take it to the private messages. For the record, the difference between weighted average and average are pretty minor for the discussion purposes here.
|
|
|
|
|
02/06/2011 09:30:25 PM · #31 |
| In some respects I'm a little surprised that the average vote cast hasn't increased evn more over time. Ribbon winning shots from the early years are probably just as (subjectively) appealing as they are now and garner similar average votes. However as we get further down the tree I believe that today's technology is 'rescuing' shots that couldn't have been saved several years ago. Ever heavily cropped an image from a 2MP point and shoot, because you didn't compose the shot well enough in the first place? Looks nasty compared to doing the same thing with an image from a 12MP DSLR. Maybe voters have lifted their expectations in line with improvements in technology. But I think it's probably more related to our 10 point Likert scale scoring system, as previously discussed. |
|
|
|
02/06/2011 11:13:23 PM · #32 |
Originally posted by PGerst:
However, decoupled completely from the 'average score' is the question: "Would this blue ribbon winner from ages ago, win today". With the advancements I mentioned above, probably not. |
The only way to test this would be to completely hide the results of a previous challenge, and post all the entries for a revote, and then compare the finishes according to the two groups of voters. |
|
|
|
02/07/2011 12:56:09 AM · #33 |
Heh...I know what you are saying, but I was referring to a different scenario. I question what would happen if an old challenge was posted today and the blue ribbon winner submitted as is, and all other entries were new, how would it place? Chances are, with exceptions, that photo would place much lower. This isn't a bad thing, just pointing out that times change, thus questioning the usefulness of some stats.
Originally posted by GeneralE:
The only way to test this would be to completely hide the results of a previous challenge, and post all the entries for a revote, and then compare the finishes according to the two groups of voters. |
|
|
|
|
02/07/2011 01:50:59 AM · #34 |
great work!! many compliments...
this mean that my 5.8683 average is a little more than the DPC average... well, i start my new week better! ;-) Thank you!!! |
|
|
|
02/07/2011 03:55:11 AM · #35 |
Originally posted by GeneralE:
Originally posted by PGerst:
However, decoupled completely from the 'average score' is the question: "Would this blue ribbon winner from ages ago, win today". With the advancements I mentioned above, probably not. |
The only way to test this would be to completely hide the results of a previous challenge, and post all the entries for a revote, and then compare the finishes according to the two groups of voters. |
Now that would be an interesting exercise! |
|
|
|
02/07/2011 04:39:59 AM · #36 |
Originally posted by GiorgioBaruffi:
great work!! many compliments...
this mean that my 5.8683 average is a little more than the DPC average... well, i start my new week better! ;-) Thank you!!! |
wow my average 7.0296 is more than the Dpc average... Well I gave out more 8!
|
|
|
|
02/07/2011 05:47:59 AM · #37 |
Originally posted by PGerst:
Direct comparison of an image with 100 votes to one with 300 votes is valid.
|
You are ignoring the law of large numbers
Message edited by author 2011-02-07 05:48:16. |
|
|
|
02/07/2011 07:30:13 AM · #38 |
Not sure what you are trying to get at. An image with 100 versus 300 votes have enough to statistically represent the mean and stdev.
Originally posted by h2:
Originally posted by PGerst:
Direct comparison of an image with 100 votes to one with 300 votes is valid.
|
You are ignoring the law of large numbers |
|
|
|
|
02/07/2011 07:55:04 AM · #39 |
Originally posted by PGerst:
Not sure what you are trying to get at. An image with 100 versus 300 votes have enough to statistically represent the mean and stdev.
Originally posted by h2:
Originally posted by PGerst:
Direct comparison of an image with 100 votes to one with 300 votes is valid.
|
You are ignoring the law of large numbers |
|
The stats of the image with 300 votes are three times as accurate as the other one; each single vote weighs only a third - any "troll vote", be it negative or positive, has a smaller effect the more votes there have been given |
|
|
|
02/07/2011 11:22:38 AM · #40 |
That statement is not correct for many reasons. If you want, PM me, and I'll explain. But it is NOT more accurate because of the number of votes. Start by examining the distribution and it should become more apparent.
Originally posted by h2:
The stats of the image with 300 votes are three times as accurate as the other one; each single vote weighs only a third - any "troll vote", be it negative or positive, has a smaller effect the more votes there have been given |
|
|
|
|
02/10/2011 09:54:42 AM · #41 |
|
Home -
Challenges -
Community -
League -
Photos -
Cameras -
Lenses -
Learn -
Help -
Terms of Use -
Privacy -
Top ^
DPChallenge, and website content and design, Copyright © 2001-2025 Challenging Technologies, LLC.
All digital photo copyrights belong to the photographers and may not be used without permission.
Current Server Time: 07/23/2025 11:22:13 AM EDT.
