DPChallenge: A Digital Photography Contest You are not logged in. (log in or register
 

DPChallenge Forums >> Web Site Suggestions >> Alternative vote calculation (warn:contains math)
Pages:  
Showing posts 1 - 13 of 13, (reverse)
AuthorThread
09/02/2007 03:14:51 AM · #1
Something to throw out there for a discussion:

How about weighing everyone's vote with the amount of votes cast in the challenge?

If all people were to vote on the same number of photos in a particular challenge, this new style of calculating average vote received would not change.
If, however, you had people that only voted on the required 20% of the photos on one, and people that voted on 80+% on the other side, I would recommend giving those voters that voted on more images higher weight.

Notice that this has nothing to do with voter's history, their overall # of/average vote given/received, their "DPC age" nor status.
My intent is neither an attempt to 'reward' voters that vote on all images. I often cannot fulfill that requirement.

It is simply a measure of relative value of the photos. Reading threads on voting here at DPC, such as the one where people stipulate that lower scores in a free study come from the fact that people 'sort' their votes and group them into low/medium/high groups, and then give the ratings from certain ranges to these groups.

Devil's advocate asking: if many voters vote only on 20% of the photos in a given challenge, and these 20% are randomly picked, isn't it a wash in the end?

Answer: No. Randomization does not help, in fact it augments the inequality of votes cast. E.g. user 1 casts votes to 20% of images, and her average vote cast to these 20% is 3.5. User 2 comes and casts his votes to other 20% of images, with an average vote of 6.5.

If both of these voters were to judge 100% of the images, I would be ready to accept the full value of their vote, even if they gave me a 1, knowing that slightly better photo than mine got a 3.

However, if I got a 3, and a slightly worse photo than mine got a 5, then there is a problem.

My proposed scheme would then value votes from voters that had the ability to compare more images over the votes of those that only passed their judgment on a fraction of the images.

I promised some math, so here we go:



Say an image has currently n votes and average vote of p. Voter v comes and votes the photo a solid 6. That voter has voted on 80% of the images in this challenge so far.

The average vote received (currently) after this n+1st vote is calculated as:

ave = sum[i=1 to n+1](Vi) / (n+1)

where Vi is the vote cast by ith voter.

My new formula would look something like this:

ave = sum[i=1..n+1](Wi * Vi)

where weight Wi = Pi/(n+1)

and Pi is the ith voter's percentage of images voted in the challenge.



Pros: - (based on the premise that users' votes are based on sorting images) more fair vote result
- less score fluctuations (statistically) over the challenge course
Cons: more calculations (although fast)
inability to claim that you know if you got a 2 or a 9 from someone.

If you think that this is a good idea/bad idea/wash, I'd like to hear about it.
09/02/2007 04:57:27 AM · #2
If I have understood this correctly (and I have yet to have coffee so no guarantees) then this normalises a voter's scoring patterns across all of the pictures that they voted for as well as weight their scores according to the number of images that they have voted on in a challenge.

I guess my only query here is that this could not really happen in real time, could it? What I mean is that as the voter is voting then the percentage of votes she is casting will change as their voting continues. Also, if folk elect to vote in batch mode then there is a need to recalculate their old votes as they apply there new ones. While I do not have a problem with a form of retrospective calculation, where all of the votes are recalculated once voting has closed (is this done anyhow?) there may be some who object as their "real time" scores may be significantly different from their weighted scores.

Personally, I like this approach.
09/02/2007 05:49:01 AM · #3
I disagree with this.

You're making the assumption that because someone didn't vote on the entire challenge that their vote is somehow less meaningful because they didn't see every entry.

Their opinion regarding the photos that they voted on is no more/less important just because they did/n't see the competitors.
09/02/2007 05:50:33 AM · #4
Sounds interesting. While you're at it you should also factor in length of time voters spent viewing each entry. For example, reduce the weight from votes that are cast very quickly (i.e. 0-2 seconds or so).
09/02/2007 05:57:33 AM · #5
Though I didn't try to understand the math, I got the idea and I think it is a good one. I don't care if the vote is recalculated at the end of the challenge, I imagine most scores will only change by small amounts. Maybe the effect could be tested with a finished challenge to see if there is any effect or how larger it is.
09/02/2007 07:15:07 AM · #6
Your solution is interesting and would work (subject to sorting out the real time issue raised by obsidian). However, I think that you are perhaps trying to fix a problem that does not exist.

With over 100 voters per image, the range of high av. scorers, low av. scorers, and medium av. scorers, is overwhelmingly likely to be balanced.

If there were regular examples of images being disadvantaged by voting patterns, then maybe this would be something to look at. However, it has only happened four times according to my profile of near misses.


09/02/2007 07:55:52 AM · #7
I don't think this is necessary. The photos are distributed randomly, and everyone should get about the same number of 20% voters. I don't think it would make much diffenece in the scores, and would discourage some voters.
09/02/2007 09:18:19 AM · #8
I understand the statistical argument, but I think the argument relies on the false assumption that the voter will always score relative to the other images he has seen in that particular challenge.

While it will vary from person to person, I think that people vote both on an absolute scale of what they perceive to be good photography, and, partially I admit, on the quality of other images in the challenge. Unless there was some way of measuring that balance for each individual voter, bearing in mind that it will change from day to day, your proposed system seems redundant.

I don't see why we can't just leave people alone and stop worrying so much about the scores. They may not be completely just, but they are at the very least equally and impartially unfair. Voting is something that takes up a lot of time and frankly can be incredibly tedious. I'm just thankful that anybody can be bothered to vote at all, especially given the complete lack of any gratefulness in the forums.

edit: Voting isn't always tedious :-)

Message edited by author 2007-09-02 09:19:51.
09/02/2007 11:11:29 AM · #9
Means all votes would need to be recalculated at rollover ... since that's the only time we know the total votes cast and can do the math accurately. Might make rollover more complex and take longer.
09/02/2007 11:14:55 AM · #10
No need to change the way voting works imo. Other issues worth spending time on.
09/02/2007 11:36:21 AM · #11
This scheme does not address the fact that some voters 'highgrade' their vote by selecting specific images via the thumbnails, and vote only on those images whose thumbnail shows promise. We all know that the thumbnails are not representative of image quality or merit, so why not abolish the thumbnail?
09/02/2007 06:06:20 PM · #12
OK, a couple of points.

We already do recognize the effect of low number of votes, but we have a binary weight for it. If you vote for less than 20%, your weight is 0, and if you vote for more than 20%, then your weight is 1.

Hence any argument that this is something we should not have is hardly valid since we already do something like this. I only propose that we have a different weight distribution than a step function that goes from 0 to 1 at 20%.

This is just an attempt to filter (for the lack of better word) the transfer function - we could even eliminate the 20% limit if we implement this. So you decide to only vote on one image out of 200, great, your vote will count with 0.005 weight.

Also, we do not have to have a linear scale, i.e. 1% of votes to represent 1% weight. We can make it ramp up faster, or have it start slow, (very small between 0 and 10% of images) ramp up faster at ~ 20% (to keep similarity with the current design) and then continue to steadily grow from 50% to the end.

I think that 20% is/was good threshold for having your votes count at all. I'm just recommending that your votes should count more if you vote for more photos.

For the argument about statistically everyone gets different 20% - I already addressed that one in my devil's advocate assumption in my original post.

Yes, this does not solve all problems. It would be interesting to see the effect if such algorithm was to run on previous challenges, to see if it has any merit. (D&L would have to do it as we do not have data publicly about who voted on what percentage of images in any given challenge).

Another benefit would be that you could then vote on less than 20% of the photos, and still have your vote count some.
09/02/2007 06:07:36 PM · #13
Originally posted by Dr.Confuser:

Means all votes would need to be recalculated at rollover ... since that's the only time we know the total votes cast and can do the math accurately. Might make rollover more complex and take longer.


See my previous post. The exact calculation is run today. The only thing that changes is the transfer function - from binary (0 or 1) depending on the voter's percentage, it would be a number between 0 and 1...

Computer is doing it, so it would take not a second longer.
Pages:  
Current Server Time: 03/29/2024 06:01:57 AM

Please log in or register to post to the forums.


Home - Challenges - Community - League - Photos - Cameras - Lenses - Learn - Prints! - Help - Terms of Use - Privacy - Top ^
DPChallenge, and website content and design, Copyright © 2001-2024 Challenging Technologies, LLC.
All digital photo copyrights belong to the photographers and may not be used without permission.
Current Server Time: 03/29/2024 06:01:57 AM EDT.