DPChallenge: A Digital Photography Contest You are not logged in. (log in or register) 
 

DPChallenge Forums >> Challenge Suggestions >> A voting experiment...
Pages:  
Showing posts 1 - 19 of 19, (reverse)
AuthorThread
 11/19/2005 04:55:38 AM · #1
I have seen this argument many times over the last three years or so. People with low scoring shots supposidly will vote lower on others shots. I propose an experiment. How about a challenge that is blind. I don't care what the challenge is. The caveat is that no one can see what their score is untill the voting period is over. Then all the mathematically anal people can compare scoring curves of openly scored challenges with this closed scoring test challenge...

Don't kick me it's just an idea...

Message edited by author 2005-11-19 04:57:10.
 11/19/2005 06:55:00 AM · #2
The 'Update' button and the ability to see a current score at any time is a feature sold as a member 'perk'. I know because I was just at the subscription page yesterday renewing my membership. One of the reasons, among many yes, for shelling out the $25. I, for one, would like to keep mine. ;^)
 11/19/2005 07:17:25 AM · #3
For me, over half the fun is the update button. It's an interesting idea, but I'm trying to imagine not knowing what my score is and I'm getting the jitters. :)
 11/19/2005 07:19:14 AM · #4
I would die without my update button.
 11/19/2005 07:33:21 AM · #5
Originally posted by TooCool:
I have seen this argument many times over the last three years or so. People with low scoring shots supposidly will vote lower on others shots. I propose an experiment. How about a challenge that is blind. I don't care what the challenge is. The caveat is that no one can see what their score is untill the voting period is over. Then all the mathematically anal people can compare scoring curves of openly scored challenges with this closed scoring test challenge...

Don't kick me it's just an idea...


I good experiment but it would be lot easier to implement if for at least one week in the open challenges you could only vote on the challenge you did not enter.
 11/19/2005 08:35:50 AM · #6
Originally posted by TooCool:
I have seen this argument many times over the last three years or so. People with low scoring shots supposidly will vote lower on others shots. I propose an experiment. How about a challenge that is blind. I don't care what the challenge is. The caveat is that no one can see what their score is untill the voting period is over. Then all the mathematically anal people can compare scoring curves of openly scored challenges with this closed scoring test challenge...

Don't kick me it's just an idea...


I like this idea and have suggest it before just to see what happens. I think you'l notice a huge difference in voting but not necessarily from the people with update addiction.
 11/19/2005 09:42:30 PM · #7
I think it's a great idea. I also like the one where you vote in the one you've not entered.

Without my compulsion to hit the update button, I might actually clean my house!!! ;~O
 11/19/2005 10:02:18 PM · #8
Originally posted by keegbow:
I good experiment but it would be lot easier to implement if for at least one week in the open challenges you could only vote on the challenge you did not enter.

Prohibiting people from voting in a challenge they have entered would likely result in a significantly lower number of votes being cast in that challenge, and therefore have a negative effect on the accuracy of the results. Doing that at the same time as the experiment proposed by TooCool would only serve to confuse to the point of not being able to reach a valid conclusion on either proposal. If we experiment with either of these proposals we should do it separately.

Just my two cents.
 11/19/2005 10:34:09 PM · #9
i am actually the other way around. I begrudge good photos my vote when my own photo is doing well. Then i realise that i am doing that and give them the vote they deserve.
 11/19/2005 10:50:27 PM · #10
My 2 cents concerning the voting. Bottom line is that a good shot is a good shot and even with a couple votes bashing these photos they still win. Here's my experiment... Go back into the archives and find somewhere that someone has won a ribbon that does not have a great shot. I believe that you will not find very many, if any. This site has many different styles that are displayed and you can see it with the submissions and the voting. For example last weeks winner; personally I did not like it so (not my style) I voted a 4. This did not stop it from winning. My personal photos are getting comments like (check this out). Great photo, Nice idea, my favorite of the competition, heres where it gets cool - would be better in focus, looks like a snapshot, composition too centered would be better if cropped to place subject off center. My question is how can a photo get so wide of a range of comments? Well it is because of all the different styles here.

One guy actually likes to set up elaborate props to take his photos and believes that anyone that is out walking around with his camera all week looking for the perfect photo that will fit the competition has not earned the ribbon. He believes that this kind of photo did not work hard enough to earn his best vote, hense his term snapshot. He cannot realize how hard it was to walk around all week with your camera taking several hunderd photos just to find one for this week submission.

Bottom line is that even though we get low votes and do not ribbon we get a lot of feedback, sometimes this is a better prize.

11/19/2005 11:26:04 PM · #11
TooCool, your experiment would work just as well if we restricted the voting in the exclusive open challenge to the one that we did NOT enter.
That way, there is no reason to try and vote anybody lower.
 11/19/2005 11:30:32 PM · #12
Originally posted by Beetle:
TooCool, your experiment would work just as well if we restricted the voting in the exclusive open challenge to the one that we did NOT enter.
That way, there is no reason to try and vote anybody lower.


I would have to agree but it would be cool to do one ware you have no idea what your score is.

Message edited by author 2005-11-20 12:20:12.
 11/20/2005 12:19:56 AM · #13
OK - So you do this experiment. How/who says what the results were? How can any valid point be made, positive or negative?

In the last ten challenges the ranges in voting are:
Highest score 6.511 -> 7.751
Lowest score 2.079 -> 4.444

Fairly wide ranges there. So again, what does this experiment tell you? I say it wouldn't show any kind of measurable results - period.

Each challenge is unique. The number of entries, the challenge theme, heck - maybe even recently completed challenges can impact results. Example - want to enter a soda can in the next challenge? ;^)

This idea is fun to think about, stir people up, etc... but not one that I hope goes any farther than this thread. JMO.
 11/20/2005 03:33:30 PM · #14
Originally posted by Beetle:
TooCool, your experiment would work just as well if we restricted the voting in the exclusive open challenge to the one that we did NOT enter.
That way, there is no reason to try and vote anybody lower.


If you run the two experiments -- TooCool's blind challenge where no one knows how their entry is faring, and the other one where you can only vote in the one you have not entered -- at the same time on the same challenge, or pair of Open challenges, you won't be able to tell anything from the results. Suppose the scores where, on average, a full point higher than in previous challenges. Would you conclude that to be because people didn't know how their shot was being scored, or because they coldn't vote in the challenge they had entered? They need to be separated if we are to learn anything from them.

Being banned from voting on a challenge you have entered is an idea that has been discussed a lot in the past but never tried. My guess is that if this were tried there would be two distinct results that we could reach agreement upon. First, the total number of votes would go down, maybe by as much as half. That's a bad result, and probably the main reason why this experiment has never been tried. Second, the scores would likely go up, by how much is anybody's guess. That is not a bad result in and of itself. But those scores would not be comparable to past challenges. And they would skew all the stats in the database if they were averaged in with previous results. We probably don't want to do that to the data that has been accumulating for almost 3 years now.

I never heard the idea of a blind challenge put forward until this past week. It's an interesting concept. My prediction is that the number of votes would go down but only by a small amount. And the score might rise but not by much. But I'm only speculating here, I may be way off target. If it were up to me, I would like to see the comments hidden until voting is finished as well as scores in any such experiment. One possible outcome that I forsee as a positive would be to drastically reduce the amount of silly and whiny forum posts. That might encourage us to put more emphasis on teaching/learning photography.
11/20/2005 03:49:07 PM · #15
Originally posted by coolhar:


If you run the two experiments -- TooCool's blind challenge where no one knows how their entry is faring, and the other one where you can only vote in the one you have not entered -- at the same time on the same challenge, or pair of Open challenges, you won't be able to tell anything from the results. Suppose the scores where, on average, a full point higher than in previous challenges. Would you conclude that to be because people didn't know how their shot was being scored, or because they coldn't vote in the challenge they had entered? They need to be separated if we are to learn anything from them.


It's actually even harder to learn anything. By declaring the experiment, we would not know if the results were caused by the experiment itself. Too many people would know "this is the blinded vote / restricted vote challenge, maybe I should vote higher"...

Good scientific studies are very hard to carry out without introducing possible bias.
 11/20/2005 04:11:55 PM · #16
Originally posted by Jutilda:
I think it's a great idea. I also like the one where you vote in the one you've not entered.

Without my compulsion to hit the update button, I might actually clean my house!!! ;~O


or my yard...
maybe simply having a rule where I am not allowed near photoshop or my camera for a few weeks would yield even better results..(please don't anyone mention this to my wife)..
oh and yes I like the idea...
 11/20/2005 04:12:52 PM · #17
While it seems that the most successful photos on this site strive for mass appeal, no method of voting is going to result with a winner recieving a perfect 10. Voter's tastes and photographer's niches are too widely varied.

Do voters vote based on how thier photo is doing? Yes, more than likely. Most submitters more than likely believe they have a good photo and want to see it do well. It's human nature to judge others based on what you have done.

The problem with blind voting would be this. Voters will make a guess as to what thier photo will score. Thier percieved score could vary wildly from thier actual score. However, the voters would continue to vote as if thier percieved score is accurate. Based on the quality of the submissions, this has a very good chance of lowering the overall averages, not raising them as desired.

The psychology of mass voting is very complex, especially when those voting are also being voted on.

I do have one suggestion though to help eliminate the "vote lower than my score" syndrome. Allow voters to vote on thier own photos. More than likely they will give themselves a 10, but this will give them the confidence that they have at least that one 10 and allow them to vote for the rest of the photos more accurately.
 11/20/2005 04:50:26 PM · #18
It would seem far easier to get agreement to try this by preventing a person from seeing their score until after they have finished voting. After which the update addiction could continue unabated.

It was mentioned that changing the voting method would destroy relevance to past stats -- I have to disagree here. Comparing voting results from two seperate challenges (each with their own theme) is like taking a poll in a mall on two different weeks; one asking who likes apples and the other who likes oranges.

The only stat that is comparable from one challenge to the next is the 'percentile ranking' -- and that stat would remain valid with any of the voting changes suggested here.

David
 11/21/2005 02:09:08 PM · #19
Originally posted by Britannica:
... It was mentioned that changing the voting method would destroy relevance to past stats -- I have to disagree here. Comparing voting results from two seperate challenges (each with their own theme) is like taking a poll in a mall on two different weeks; one asking who likes apples and the other who likes oranges.

The only stat that is comparable from one challenge to the next is the 'percentile ranking' -- and that stat would remain valid with any of the voting changes suggested here.

David

Comparing results from one challenge to another tends toward apples and oranges. But comparing a challenge result to what one has done over the course of many challenges in the past is more valid. And comparing a group of recent challenge results (at either the individual or community level) to another group from the past, or to the entire past record is even more valid.

Percentile is a very valuable tool, but it wouldn't help if you wanted to compare the averaged results of an entire challenge to the past record.

Preventing a person from seeing their own score until they have finished voting has been discussed in the past. A few members of SC were pretty adamantly against it. And some legitimate problems were raised but not answered. It may deserve further discussion but my fear would be that the total number of votes cast per challenge would suffer.
Pages:  
Current Server Time: 08/12/2025 08:55:02 AM

Please log in or register to post to the forums.


Home - Challenges - Community - League - Photos - Cameras - Lenses - Learn - Help - Terms of Use - Privacy - Top ^
DPChallenge, and website content and design, Copyright © 2001-2025 Challenging Technologies, LLC.
All digital photo copyrights belong to the photographers and may not be used without permission.
Current Server Time: 08/12/2025 08:55:02 AM EDT.