| Author | Thread |
|
|
03/03/2004 10:34:46 AM · #1 |
I realized last night (when I opted not to participate in "silence") that for some reason I'm not as happy when there is a large number of submissions. This is irrational thinking. I "crunched" the numbers this morning and it is true that larger submission challenges yield lower overall scores, but that could mean one of 2 things:
1 - More bad photos are being submitted, or
2 - People tend to vote lower when they have 300+ images to vote on
If it's number 1, then good for me. If it's number 2, then hopefully my image comes up early in voting before voter get burned out.
Anyone else put any thought into this, or am I just a wacko? |
|
|
|
03/03/2004 10:48:48 AM · #2 |
I too get overwhelmed by large number of submissions. And I tend to go through them faster and leave fewer comments.
|
|
|
|
03/03/2004 10:55:40 AM · #3 |
Originally posted by Rtwo:
I too get overwhelmed by large number of submissions. And I tend to go through them faster and leave fewer comments. |
I think most are guilty of this...though some might not like to admit it. Also to add to your point, I think there is often a bias against images which have similar subject matter, in any given challenge.
|
|
|
|
03/03/2004 11:11:16 AM · #4 |
I know what you mean about the large number of submissions being somewhat overwhelming. Especially when the 400 mark is breached.
I had an idea about an alternate voting system, but it is only an idea and I am not actually proposing the change - it is just an idea that I would like to hear feedback on.
What if for certain challenges (like the monthly free study challenges for example, that will in general be a large draw) there was a two stage voting system?
In the first stage, you would vote (yes or no, not a 1-10 value) for the top 100 photos in the challenge (or some other finite number). You could still comment on any submission, whether you choose it for advancement or not.
In the second stage, those 100 photos are voted on using the regular voting system in place for the ultimate prize of ribbons.
There are several downsides to this of course - those that do not make the cut for round two will not have a score like the final 100 will have. You could let them know how many times they were selected for advancement, even if they miss the cut, but it is not exactly the same.
Also, voting would out of necessity take longer since it would have to go through two rounds.
The big benefits as I see them, however, are simpler voting during round 1 - simply shoose your 100 favs, and not have to dwell on 1-10 scores. Then the second round is a manageable number for more in-depth consideration.
You would also have two prize systems - it would be considered an achievement to simply make the cut, even if you don't come close to winning.
This would also allow for growth in challenge submissions - I could reasonably pick a top 100 out of 600 submissions, but that number would be very daunting to try to pin 600 1-10 scores on. I know you only have to vote on 20% of them, but I always feel unfair if I do that. And this would keep the voting quantity uniform instead of some folks voting partially, and some voting on all images.
Anyway, just some random thoughts. I know it is not really practical considering the complexity it would add. Blast away!
Rich.
|
|
|
|
03/03/2004 11:20:02 AM · #5 |
Good ideas here, Rich. Among the other downsides you mention, however, is the fact that one would need to do a first pass through ALL the submissions before being able to reasonably rank any in the top 100...
-len |
|
|
|
03/03/2004 11:26:58 AM · #6 |
Originally posted by lenkphotos:
Good ideas here, Rich. Among the other downsides you mention, however, is the fact that one would need to do a first pass through ALL the submissions before being able to reasonably rank any in the top 100...
-len |
True - but that is the downside of the 20% rule to me as well - I feel like I have to look at all of them to be fair about which 20% to vote on.
|
|
|
|
03/03/2004 11:54:02 AM · #7 |
I too tend to leave much fewer comments when you have 300 plus photos to go through and rate
Secondly I think we need a post again on rating photos. Set up parameters for giving a certain number. For instance, composition, creativity, meeting the challenge, focus, lighting, etc.
It seems to me that some people start with 5 and then start taking down from there. |
|
|
|
03/03/2004 11:58:07 AM · #8 |
i think the voting scheme set up now is nice. I like the broad range of numbers to choose from (1-10) and that we can comment.
I commented more than usual this time, and i find it helps me dertermine what i will vote the image. when thinking about it for 20 seconds insated of 5, i vote better i think. HOwever i am also finding myself basing more importance on creativity, or perhaps AS MUCH on creativity as on technical skills. if it has great technical skills but i feel no creativity it will be given the same mark as a dark blurry image WITH lots of creativity. (which most people usually vote very low)
Message edited by author 2004-03-03 11:58:31.
|
|
|
|
03/03/2004 12:06:49 PM · #9 |
Originally posted by richterrell:
I feel like I have to look at all of them to be fair about which 20% to vote on. |
Not so if you start out with the intention of voting on all of them.
|
|
|
|
03/03/2004 12:23:52 PM · #10 |
Originally posted by coolhar:
Originally posted by richterrell:
I feel like I have to look at all of them to be fair about which 20% to vote on. |
Not so if you start out with the intention of voting on all of them. |
I'm not sure I follow - that is the original dilemma for me. I do vote on them all for the few challenges I have voted on. But large numbers like 400 make for quite a time investment to vote on them all. Of course I will do it, but it is still pretty daunting. |
|
|
|
03/03/2004 12:30:20 PM · #11 |
Originally posted by richterrell:
True - but that is the downside of the 20% rule to me as well - I feel like I have to look at all of them to be fair about which 20% to vote on. |
You can vote on 20% and still be fair. the 80% you might not vote on are not biased against, they simply don't get your vote. Your one vote among hundreds is not very significant. Among the 80% will be images that get voted on by others who do not vote on your 20%. by the law of averages and as long as by and large everyone is voting systematically according to their own system, there should be no bias involved in only voting for 20%. Voting fo them all is preferable but I find myself wanting to take time to look at and comment on those images at are good and/or creative and/or demonstrate real effort and thought even though they might be technically poor. |
|
|
|
03/03/2004 12:33:59 PM · #12 |
Dont know if low scores in large submissions are because of many bad pictures or if people tend to vote lower when they have to scroll through many shots.
I´ve made the decision not to try to scroll through all images. In the conflict challenges I votet for just over 20% of all images and made a coment on most of the images I voted for. Since we are suposed to get random samples of images then I think everybody should try to look carefully at each picture that pop up on their screen and add coment if they can. Not try to vote all images unless they have all the time in the world. |
|
|
|
03/03/2004 12:34:48 PM · #13 |
Originally posted by andywightman:
Originally posted by richterrell:
True - but that is the downside of the 20% rule to me as well - I feel like I have to look at all of them to be fair about which 20% to vote on. |
You can vote on 20% and still be fair. the 80% you might not vote on are not biased against, they simply don't get your vote. Your one vote among hundreds is not very significant. Among the 80% will be images that get voted on by others who do not vote on your 20%. by the law of averages and as long as by and large everyone is voting systematically according to their own system, there should be no bias involved in only voting for 20%. Voting fo them all is preferable but I find myself wanting to take time to look at and comment on those images at are good and/or creative and/or demonstrate real effort and thought even though they might be technically poor. |
I see what you mean, but I stil feel it is unfair, even with my one vote. If in my 20% that I vote on I give someone a 10, and the other 80% has some 10s in it that did not benefit from having their score raised, then I think they DO miss out. So I feel like I need to look at all of the photos, and if I am going that far, I go ahead and cast votes for all of them, and leave comments where I can.
I get what you are saying though - I know I don't HAVE to look at all of them to put in an unbiased 20% vote, but I still personally feel that I need to.
Rich. |
|
|
|
03/03/2004 12:52:02 PM · #14 |
Because of the way the voting page is set up, when voting, alot of people simply sift through the thumbnail views and only vote on those that appeal to them. When they hit 20% they quit. I don't think this is hardly fair to the other 80% that don't catch their eye in the thumbs. Don't get me wrong, with so many entries, the 20% rules is a great option, but I think it ought to have some randomness built into it. I think there should be an option button on the page which would link you to a randomly selected 20% of entries. This is not an issue for those who vote as the images are displayed or those that vote on all entries.
|
|
|
|
03/03/2004 12:54:14 PM · #15 |
Yep. I feel the need to too.
Remember though for every shot you don't vote on that you would have given a 10 to, there will be others that you would have given a 1/2/3/4/5 to. They benefit from your NOT voting. So, unless there is systematic bias in individual voting (like someone who picks 20% and gives them all ones in which case there is bias albeit small) the law of averages should ensure an accurate reflection of opinionsin the final tally. |
|
|
|
03/03/2004 01:03:52 PM · #16 |
Originally posted by andywightman:
Yep. I feel the need to too.
Remember though for every shot you don't vote on that you would have given a 10 to, there will be others that you would have given a 1/2/3/4/5 to. They benefit from your NOT voting. So, unless there is systematic bias in individual voting (like someone who picks 20% and gives them all ones in which case there is bias albeit small) the law of averages should ensure an accurate reflection of opinionsin the final tally. |
Maybe if the admins envoked a 20% maximum vote for everyone. They could setup some kind of 20% template unique to each user so there are only certain images you could vote and comment on. I think more thought would go into voting and commenting again, like back when there were less participating here.
|
|
|
|
03/03/2004 01:14:50 PM · #17 |
Originally posted by richterrell:
Originally posted by lenkphotos:
Good ideas here, Rich. Among the other downsides you mention, however, is the fact that one would need to do a first pass through ALL the submissions before being able to reasonably rank any in the top 100...
-len |
True - but that is the downside of the 20% rule to me as well - I feel like I have to look at all of them to be fair about which 20% to vote on. |
The idea of presenting the images randomly to each voter is to get about the same number of votes for each. I just start with the first thumbnail and vote on as many as I can. If you start scanning the thumbnails and then picking and choosing which to vote on you are both subverting the randomizing effect, and you are conducting a "preliminary elimination round" (as suggested earlier) based on the THUMBNAILS only. This also defeats the purpose of requiring you to see the full-size image before voting, since you are actually rejecting up to 80% of the imgages based on the thumbnail view alone.
Personally, I'm thinking maybe nobody should see the thumbnails until they're done voting; the link to vote in a challenge could just go right to the first (random) picture. This would make the voting a lot fairer in my opinion. |
|
|
|
03/03/2004 01:22:27 PM · #18 |
| Maybe when there is a Large amount of submissions you get just a portion to vote on and the next person gets another portion, lets say for instance you get the first 100 and the next person would get the next 100 and so on...Just a thought |
|
|
|
03/03/2004 01:25:01 PM · #19 |
Originally posted by GeneralE:
Originally posted by richterrell:
Originally posted by lenkphotos:
Good ideas here, Rich. Among the other downsides you mention, however, is the fact that one would need to do a first pass through ALL the submissions before being able to reasonably rank any in the top 100...
-len |
True - but that is the downside of the 20% rule to me as well - I feel like I have to look at all of them to be fair about which 20% to vote on. |
The idea of presenting the images randomly to each voter is to get about the same number of votes for each. I just start with the first thumbnail and vote on as many as I can. If you start scanning the thumbnails and then picking and choosing which to vote on you are both subverting the randomizing effect, and you are conducting a "preliminary elimination round" (as suggested earlier) based on the THUMBNAILS only. This also defeats the purpose of requiring you to see the full-size image before voting, since you are actually rejecting up to 80% of the imgages based on the thumbnail view alone.
Personally, I'm thinking maybe nobody should see the thumbnails until they're done voting; the link to vote in a challenge could just go right to the first (random) picture. This would make the voting a lot fairer in my opinion. |
All the more reason for having a 20% vote across the board with a selection of random images unique to each voter.
|
|
|
|
03/03/2004 01:39:30 PM · #20 |
I don't know how it would be possible, but I do think there needs to be some kind of filtering system in place. Being expected to vote on 500-ish images a week is far too burdensome to get any meaningful results. And, unfortunately, there are always a significant number of images that have no business being submitted in the first place.
I do like the idea about not being able to see the thumbnails. Make it more like the Critique Club where you don't get to choose what you're voting on until you're stuck looking at it. |
|
|
|
03/03/2004 04:53:34 PM · #21 |
I like the part about not seeing the thumbnails. I find that I do pick and choose when I feel like I don't have the time to vote on all the pictures. I know that throws the scores off, but what can I say.
Maybe, if we could choose the number of entries we feel we can vote on, in groups of 50, no one would feel like it is such a chore. Or have everyone get a random group of 50, and then to get more, they will have to vote on all of the first group. |
|
Home -
Challenges -
Community -
League -
Photos -
Cameras -
Lenses -
Learn -
Prints! -
Help -
Terms of Use -
Privacy -
Top ^
DPChallenge, and website content and design, Copyright © 2001-2024 Challenging Technologies, LLC.
All digital photo copyrights belong to the photographers and may not be used without permission.
Current Server Time: 04/23/2024 08:10:31 PM EDT.
