DPChallenge: A Digital Photography Contest You are not logged in. (log in or register
 

DPChallenge Forums >> Rant >> A More Precise Voting Method
Pages:  
Showing posts 1 - 25 of 48, (reverse)
AuthorThread
05/16/2015 07:59:04 AM · #1
About 90% of the votes fall between 4 and 7. That means that there is not much of a differentiation in the voting between pictures within a whole number vote. In other words a picture that one might vote a 5.0 would be considered equally as good as a score of 5.5 when we vote individually and yet we would have to vote both pictures a 5. In the final group voting there can be a very large difference between the placements of a picture averaging a 5 and one averaging a 5.5.

My suggestion, therefore, is rather than a whole number choice of scoring that there be a slider which allows the voter greater discrimination in his voting score. we could then vote a picture a score of anywhere in the range. I think it would be more fair and be more interesting for the voter. Has this been considered before? What do you think?
05/16/2015 10:32:56 AM · #2
Originally posted by lnede:

About 90% of the votes fall between 4 and 7. That means that there is not much of a differentiation in the voting between pictures within a whole number vote. In other words a picture that one might vote a 5.0 would be considered equally as good as a score of 5.5 when we vote individually and yet we would have to vote both pictures a 5. In the final group voting there can be a very large difference between the placements of a picture averaging a 5 and one averaging a 5.5.

My suggestion, therefore, is rather than a whole number choice of scoring that there be a slider which allows the voter greater discrimination in his voting score. we could then vote a picture a score of anywhere in the range. I think it would be more fair and be more interesting for the voter. Has this been considered before? What do you think?


the top pics win in any case, does middle placement even matter?
05/16/2015 03:01:22 PM · #3
Yes. It's not a matter of winning our losing. It's a method of fairly grading the quality of the picture which is a learning experience for the voter and the entrant. Both can benefit in seeing where the picture actually ends up and being able to compare to those that were considered better pictures. Thank you for your response. It appears that you would not be in favor of this concept.
05/16/2015 05:42:50 PM · #4
I'm a proponent of using the entire range of the 10 whole numbers. The thought of more increments is intriguing but I doubt it would have much affect on defining placement though. Averages aren't in whole numbers like the raw scores. They're already posted out to four decimal places. Although interesting, I don't think your proposal would make voting any more "fair". At least the current whole number system forces us to vote below or above the mathematical average on each image.
05/16/2015 05:53:32 PM · #5
Mr. Woodward - I'm so glad you popped into this discussion.

As ' . substr('//www.dpchallenge.com/images/user_icon/21.gif', strrpos('//www.dpchallenge.com/images/user_icon/21.gif', '/') + 1) . ' lnede pointed out, it seems folks are loath to use one through three and eight through ten with any regularity. So...

What would be the effect of REDUCING the scoring range to FIVE whole numbers? One through five, with five being the highest and one the lowest.

In your opinion, would that force (encourage) folks to use the entire range of scoring options instead of just the middle of the ten whole numbers and produce more of a bell-curve spread which, I think, ' . substr('//www.dpchallenge.com/images/user_icon/21.gif', strrpos('//www.dpchallenge.com/images/user_icon/21.gif', '/') + 1) . ' lnede is proposing working towards.

I'm a total dodo at math, so please keep it simple. :-) Thanx!



05/16/2015 07:45:51 PM · #6
Originally posted by Zita:

What would be the effect of REDUCING the scoring range to FIVE whole numbers? One through five, with five being the highest and one the lowest.

People would vote predominately from 3-5 ...

When this (the original proposal) has been considered in the past, it was usually framed as expanding the voting range to 1-100, and most people thought it was hard enough deciding if something was a 5 or 6, without having to stress over whether a picture was a 56 or 58 ... a "more precise" voting range implies that people are going to spend more time looking at a photo and carefully considering it's "value" (voting-wise) -- I have serious doubt that this would happen.
05/16/2015 08:10:19 PM · #7
Ok - thanks, Mr. Marcus. Your knowledge of the history is helpful.

It somewhat saddens me that the implication might be that folks are not giving careful consideration to the 'value' of an image when assigning it a score. That is why I wondered if tightening - rather than expanding - the voting scale might encourage more thoughtful assignment of a score. But, you are probably right that folks would then restrict their votes to between three and five. Oh well...

Again - thanks for the insights.

05/16/2015 08:41:52 PM · #8
Originally posted by Zita:

Your knowledge of the history is helpful.

That proposition may be subject to considerable debate, but you're welcome anyway ... :-)

Mostly I subscribe to the "if it ain't broke don't fix it" approach, and something has to be pretty broken to be worth fixing, especially when it involves reworking complex proprietary code. That voting patterns, and the same complaints and counter-arguments about the voting (and commenting) have remained remarkable consistent for over ten years suggests to me that it's not broken enough to "fix" ...

' . substr('//images.dpchallenge.com/images_portfolio/0-4999/1031/120/Copyrighted_Image_Reuse_Prohibited_1132631.jpg', strrpos('//images.dpchallenge.com/images_portfolio/0-4999/1031/120/Copyrighted_Image_Reuse_Prohibited_1132631.jpg', '/') + 1) . '
05/16/2015 08:46:48 PM · #9
I like the idea of a voting range of 1 to 100. We should try it on a few challenges and see how it goes. Then have a discussion afterwards.
05/16/2015 08:53:44 PM · #10
Originally posted by lnede:

I like the idea of a voting range of 1 to 100. We should try it on a few challenges and see how it goes. Then have a discussion afterwards.

That's not possible. It's a major coding issue that has ripple effects. To name just ONE, where do these new-scored images fir into the overall fabric when we look at top-50 lifetime etc?
05/16/2015 08:58:44 PM · #11
Originally posted by lnede:

I like the idea of a voting range of 1 to 100. We should try it on a few challenges and see how it goes. Then have a discussion afterwards.

Before messing with site code, why don't you (and anyone else interested), go ahead perform the experiment and try it yourselves, and see how it actually feels in practice.

Put your 1-100 score in the comment box, and then round to the nearest "tenth" if you want to cast an official DPC vote. You should be checking whether it takes longer (and how the extra time is spent), and how "different" the results might between the two systems.
05/16/2015 09:45:40 PM · #12
I'm not sure doing it by myself would accomplish anything. It would take a beta test of many people to get a fair overall opinion and evaluation of the results.
I understand that it would be a revamping of the present system and might change the overall statistics which may not be worth it for what it is.
05/16/2015 10:09:04 PM · #13
Originally posted by Zita:

Mr. Woodward - I'm so glad you popped into this discussion.

As ' . substr('//www.dpchallenge.com/images/user_icon/21.gif', strrpos('//www.dpchallenge.com/images/user_icon/21.gif', '/') + 1) . ' lnede pointed out, it seems folks are loath to use one through three and eight through ten with any regularity. So...

What would be the effect of REDUCING the scoring range to FIVE whole numbers? One through five, with five being the highest and one the lowest.

In your opinion, would that force (encourage) folks to use the entire range of scoring options instead of just the middle of the ten whole numbers and produce more of a bell-curve spread which, I think, ' . substr('//www.dpchallenge.com/images/user_icon/21.gif', strrpos('//www.dpchallenge.com/images/user_icon/21.gif', '/') + 1) . ' lnede is proposing working towards.

I'm a total dodo at math, so please keep it simple. :-) Thanx!


I'd have to agree with the General on this. There is some psychology to the use of a Likert scale, i.e. 1-5 or 1-10. Studies have shown that the extreme ends are seldom used. So most votes would fall between 2 and 4. I think I mentioned this in one of the first voting stats posts in 2011. You can get to them from my profile.
05/16/2015 10:12:54 PM · #14
Originally posted by Zita:

Mr. Woodward - I'm so glad you popped into this discussion.

As ' . substr('//www.dpchallenge.com/images/user_icon/21.gif', strrpos('//www.dpchallenge.com/images/user_icon/21.gif', '/') + 1) . ' lnede pointed out, it seems folks are loath to use one through three and eight through ten with any regularity. So...

What would be the effect of REDUCING the scoring range to FIVE whole numbers? One through five, with five being the highest and one the lowest.

In your opinion, would that force (encourage) folks to use the entire range of scoring options instead of just the middle of the ten whole numbers and produce more of a bell-curve spread which, I think, ' . substr('//www.dpchallenge.com/images/user_icon/21.gif', strrpos('//www.dpchallenge.com/images/user_icon/21.gif', '/') + 1) . ' lnede is proposing working towards.

I'm a total dodo at math, so please keep it simple. :-) Thanx!


By the way, the total voting curve is already very much a classic bell shaped distribution ;-)
05/16/2015 10:19:57 PM · #15
I have three images in voting. Number of votes for the three, in succession is 55, 53 and 21.
Do you think having a different voting scale will increase the number of voters?
If not, whether we vote from 1-5, 1-10 or 1-100 is really a moot point.
What we need is a method w/reasoning that will increase the number of voters, especially those who vote the whole challenge100%.
05/16/2015 10:38:50 PM · #16
Originally posted by PennyStreet:

I have three images in voting. Number of votes for the three, in succession is 55, 53 and 21.
Do you think having a different voting scale will increase the number of voters?
If not, whether we vote from 1-5, 1-10 or 1-100 is really a moot point.
What we need is a method w/reasoning that will increase the number of voters, especially those who vote the whole challenge100%.


Can't agree more with the desire for more voters. More votes lead to a even more stabilized averages. I truly hope that DPC voting isn't a reason for lack of participation though. I prompted discussions on voting back in 2011 because I believed the voting was fundamentally fair and I still believe that today.

Many people, most better statisticians than me, have joined into the discussions in support of the system. There seems to be two camps. One is data driven and supportive of the voting system. The other seems to have a more emotional approach. Scores don't feel right to them and they want to see voting improvements but they offer little data to support their position.

We need methods to increase participation but I dont think changing the voting system will do that


Message edited by author 2015-05-16 22:46:55.
05/17/2015 01:23:45 AM · #17
Originally posted by Bear_Music:

Originally posted by lnede:

I like the idea of a voting range of 1 to 100. We should try it on a few challenges and see how it goes. Then have a discussion afterwards.

That's not possible. It's a major coding issue that has ripple effects. To name just ONE, where do these new-scored images fir into the overall fabric when we look at top-50 lifetime etc?


Invoke an automated /10 operation at the rollover point for each score?

Score 1-100, then move the decimal point at the end. A different meaning to dpchallenge!
05/17/2015 01:26:54 AM · #18
Originally posted by Paul:

Originally posted by Bear_Music:

Originally posted by lnede:

I like the idea of a voting range of 1 to 100. We should try it on a few challenges and see how it goes. Then have a discussion afterwards.

That's not possible. It's a major coding issue that has ripple effects. To name just ONE, where do these new-scored images fir into the overall fabric when we look at top-50 lifetime etc?


Invoke an automated /10 operation at the rollover point for each score?

Score 1-100, then move the decimal point at the end. A different meaning to dpchallenge!


However, equivalence would only come from a 10-100 scale and even then I suspect great use of 9x scores rather 100 scores would have the effect of disadvantaging new finishing scores in relation to high legacy scores.
05/18/2015 10:05:27 AM · #19
Interesting topic. When I first considered the 1-100 idea, my thought was that it would complicate things and potentially reduce voting.

However, perhaps we could *encourage* voting with a variation of this idea.
I personally like the breakdown page that creates a chart (if you will) to show the voter what scores he/she has given out during the voting stage of the challenge. At that point, I often feel obligated to broaden my spectrum of scores to differentiate images that i feel should be higher / lower than others (in comparison to eachother). In other words I will look at three images I have given an "8" and break them up into 8 - 9 - and 10 in order of my preference.

However, since there are obviously more that 10 images in each challenge, it would be impossible to separate all images in this way with just 1-10 (hence this thread topic) ...

So MAYBE, when a voter has scored ALL images in a challenge, they can be given the option to further tinker with the scores to allow for a "favorite" image to be followed by all the other images in an order of preference. This could be a GUI where we click/drag images above / below eachother ... yeah maybe a programing challenge, but how hard could it be? ;) If an image is moved between an 8 and a 9, it is scored 8.5. If an image is moved between that image and a 9, it is then given an 8.75 and so-on.

If there is anything this site needs IMHO, is something fresh .. the increase in image size has gone over well and the 5-images-on-the front page seems to be well-received. Voting seems to be the one area that could use improvement and some sort of motivator. Personally, I have not voted enough and I have been trying to vote more lately.

Originally posted by PennyStreet:


What we need is a method w/reasoning that will increase the number of voters, especially those who vote the whole challenge100%.


Message edited by author 2015-05-18 10:31:51.
05/18/2015 12:17:07 PM · #20
When scoring art, numbers are necessarily emotional signifiers and not units of measure, because there is nothing to measure, no criteria have been defined.

To end this confusion once and for all, we simply need to replace numbers with other signifiers.

1 shall be a chicken
2 shall be a moss-covered rock
3 shall be an ocean wave reflecting moonlight
4 shall be moosen
5 shall be a traffic light, auburn in all directions
6 shall be planetary motion as described by Ptolemy
7 shall be a turnstile in an abandoned underground railway
8 shall be a mother's cough
9 shall be a skyscraper
10 shall be a flattened aluminum can

I can provide simple schematic illustrations of all of the above signifiers, which voters can click on. "Averages" will consist of amalgams of these images, for example if one person votes "chicken" and another votes "moosen", the "average" vote will be a chicken with many large antlers. Any combinations of the above, in any proportion, can be calculate and drawn. The resulting picture is your score.

This should resolve any concerns about precision or, more importantly, accuracy.
05/18/2015 01:05:08 PM · #21
And assign random images to the front page I assume ? =D

Originally posted by posthumous:


I can provide simple schematic illustrations of all of the above signifiers, which voters can click on. "Averages" will consist of amalgams of these images, for example if one person votes "chicken" and another votes "moosen", the "average" vote will be a chicken with many large antlers. Any combinations of the above, in any proportion, can be calculate and drawn. The resulting picture is your score.

This should resolve any concerns about precision or, more importantly, accuracy.
05/18/2015 01:36:39 PM · #22
I can provide simple schematic illustrations of all of the above signifiers, which voters can click on. "Averages" will consist of amalgams of these images, for example if one person votes "chicken" and another votes "moosen", the "average" vote will be a chicken with many large antlers. Any combinations of the above, in any proportion, can be calculate and drawn. The resulting picture is your score.

This should resolve any concerns about precision or, more importantly, accuracy. [/quote]

I like it. Did you also invent Rock, Paper, Scissors or perhaps the Big Bang Theory version of Scissors cuts Paper, Paper covers Rock, Rock crushes Lizard, Lizard poisons Spock, Spock smashes Scissors, Scissors decapitates Lizard, Lizard eats Paper Paper disproves Spock, Spock vaporizes Rock (and as it always has) Rock crushes scissors
05/18/2015 01:54:37 PM · #23
Maybe each month could have an EXTRA challenge and only those who have voted on 100% on 3 challenges during the prior month could participate. Once again, some programming involved but hey ...

Originally posted by DJWoodward:

Can't agree more with the desire for more voters. More votes lead to a even more stabilized averages. I truly hope that DPC voting isn't a reason for lack of participation though. I prompted discussions on voting back in 2011 because I believed the voting was fundamentally fair and I still believe that today.
05/18/2015 03:06:55 PM · #24
Originally posted by tate:

Maybe each month could have an EXTRA challenge and only those who have voted on 100% on 3 challenges during the prior month could participate. Once again, some programming involved but hey ...

Trouble with THAT is, it eliminates those of us who CAN'T vote on all the images because we have seen some of them. Which definitely impacts SC, since we're asked to evaluate images for legality in advance of rollover with some frequency. Then there's those of us who have spouses or children entering challenges, and those of us who help noobies with processing hints and those of us who help others choose their "best image" for a challenge etc etc. How many will be left who can do 100%? :-)
05/18/2015 03:11:06 PM · #25
If we all spent as much time voting as complaining about thoughtfully (and endlessly) discussing voting the number of complaints comments about voting would be markedly reduced.
Pages:  
Current Server Time: 12/06/2019 02:15:16 PM

Please log in or register to post to the forums.


Home - Challenges - Community - League - Photos - Cameras - Lenses - Learn - Prints! - Help - Terms of Use - Privacy - Top ^
DPChallenge, and website content and design, Copyright © 2001-2019 Challenging Technologies, LLC.
All digital photo copyrights belong to the photographers and may not be used without permission.
Proudly hosted by Sargasso Networks. Current Server Time: 12/06/2019 02:15:16 PM EST.