| Author | Thread |
|
|
04/15/2004 10:12:58 AM · #1 |
I have read in many threads that scores tend to be very low and, very strange, tends to start higer, then decrease in the next hours/days, and I personally have suffered it in my first challenge, so I have been thinking about it: How can the time make the score be lower than initial ones?
I think I have the answer and the cure.
What I think is happening (and had happened to me too), is that: You sned your pic, mentally score it as a 6 or more (for instance), then the votation starts, and, if you are one of the first in voting, do it honestly: The more you like, the more you score.
The problem comes when, after some hours you come back to score the other half of the pics, you see your score: 4.5 so far! You get angry, and then you start to score: This one is NOT better than mine, so I CAN'T give it more than what I am getting, so here you have just a 4... but if this pic had appeared among the first scored ones, I would had give it a 7 or 8, but now, it pays my fustration.
So, after the hour pass, the scoring go down, so people that come to score are even more angry, so votes are even lower... it is a "lowering-war" that goes worst with the time.
OK, may be you don't agree with this analisys, but the "cure" is good never mind witch is the real problem.
I propose that you have to vote first, at least 50% of pics or similar, then "close" your voting sesion (no more scoring is possible from that point), and only then you can see how good you are scoring.
Some details I have thought about implementation of this method:
You have to score 20% of pics for your votes to be counted (this is an actual rule), but also you need to reach 50% to be able to close your votation and see your personal scoring. This could make people tend to vote all pics instead of only the "good" ones. This rule is very personal and may be is not interesting for web site.
After the moment you close your votation, you can review your comments on other pics, but not change your scorings at all.
That's all, this is my proposal, but finally I want to point something: In any democracy, you can not know how the scorings go until votations are closed, as knowing partial result will chanche the voting intentions of those that still are deciding what to vote. Why not apply this here? It could make this GREAT site work better. |
|
|
|
04/15/2004 10:21:40 AM · #2 |
Fantastic idea, I have a modification to this I'd like to propose.
Members can see their score whenever they like... that's part of what they pay for.
Non-members have to vote on all pictures in a challenge before they see their score during a challenge. This will motivate non-members to give to the community and will push up the number of votes and comments during non-member challenges which at times has been a bit low recently. This will also prevent people from scoring others too low in frustration.
Bob |
|
|
|
04/15/2004 10:24:03 AM · #3 |
Originally posted by BobsterLobster:
Fantastic idea, I have a modification to this I'd like to propose.
Members can see their score whenever they like... that's part of what they pay for.
Non-members have to vote on all pictures in a challenge before they see their score during a challenge. This will motivate non-members to give to the community and will push up the number of votes and comments during non-member challenges which at times has been a bit low recently. This will also prevent people from scoring others too low in frustration.
Bob |
...or encourage a bunch of thoughtless 5's just so they can see their own score. |
|
|
|
04/15/2004 10:24:44 AM · #4 |
My scoring approach is to score all the pictures at one go. Then over the week, I go back and bump the scores up or down for the pictures at a given level. Over the week, pictures I was too harsh with tend to rise. Those I was too generous with tend to fall. By the end of the week, I am quite sure the scores represent my considered view of the photos.
Your approach would make my scoring less thoughtful, and in the end less reflective of my view about relative merits.
In the end I am not sure what problem you are trying to solve. If the scores tend to creep up or down, so what? The rules of math guarantee they will change over the week. The intrinsic quality of the photo will suggest the direction of the change.
Just my point of view. I am sure there will be others.
Message edited by author 2004-04-15 10:27:19. |
|
|
|
04/15/2004 10:28:39 AM · #5 |
| Hmm, you all have a point... shame, because I like the idea in principle. How about using algorithms to make sure voting is fair (ie, weed out random scoring, or scoring where the score is the same each time), and allow scores to change once every picture has been voted on. I'm sure no-one would be bothered to take a point off every photo because their own is doing badly. |
|
|
|
04/15/2004 10:33:15 AM · #6 |
I've been a strong advocate of hiding scores until they've finished voting for a long time (Unfortunately, I can't find any references except this thread and this thread). I do think "self-serving voting" (what bitfarmer describes) is a problem.
My idea is to hide scores/comments until users have clicked an "I'm finished voting" button (which wouldn't be available at all until you've reached the required 20% minimum). At that point, users are free to go back and leave comments, their votes would be "locked", and their score and comments would be displayed on the front page as they are now.
This would likely encourage more people to vote as well, since if they want to see their score before the results are posted, they'd have to vote, instead of just "taking advantage" of the site and submitting without returning the courtesy of voting.
Message edited by author 2004-04-15 10:38:02. |
|
|
|
04/15/2004 10:33:30 AM · #7 |
| I've also had shots start low and rise gradually throughout the week. There really are no fixed patterns, except there usually will be no significant changes after 75-100 votes are cast. |
|
|
|
04/15/2004 10:35:57 AM · #8 |
| I like our current system. |
|
|
|
04/15/2004 10:45:13 AM · #9 |
@ GeneralE - Agreed
@ EddyG - the only problem with that (as far as the site is concerned) is that part of the DPC Addiction is the relentless clicking of the stats button. Taking that away would discourage what I'm guessing is 50% of the reason people become members (and pay their dues) - the 'update score' button.
I agree with you, though, that it can be an issue. then i remember what the point of the site is (competing to win the right to have your name displayed beside a series of blue pixels), and I stop worrying about it :)
P-ness |
|
|
|
04/15/2004 10:46:16 AM · #10 |
Originally posted by BobsterLobster:
How about using algorithms to make sure voting is fair (ie, weed out random scoring, or scoring where the score is the same each time |
I've also proposed in the past that DPC consider implementing a "voting karma" system like is used at other sites (i.e., Worth1000).
Basically, "the level of karma you have is automatically determined by your personal voting habits. It drops when you deliberately give an image what you don't think it deserves. It raises when you rate all the images in a contest fairly."
Your karma also increases for voting on more images in a contest, which encourages more voting participation. The DPC implementation could also offer karma bonuses for commenting.
If you read the details, your vote just has to be within +/-3 of the final weighted score for your karma to increase. So if the final score is 7.0, then anyone who voted within 4 - 10 is deemed to be voting fairly and their karma goes up (until it reaches some maximum). Anybody who voted below 4 would have their voting karma reduced -- and the higher the deviation the more the karma reduction (so somebody who voted a 1 is dinged more than somebody who voted a 3).
That also means that average images (those that score in the middle of the pack, near 5.5) have a very wide scoring region -- from 2.5 to 8.5. Which is plenty of room for "individual voter expression". In those cases, users who voted 1, 2 (dishonest voters) or 9, 10 (friend of the photographer/multiple accounts) would have karma reductions, everyone else would increase their voting karma.
Quoting: "this is not a system of punishment and reward, nor is it a judgement of your personal tastes. It's merely a way to measure your ability to discern actual image quality, and allows plenty of room for your individual tastes to come into play without damaging your karma. It maximizes the effects that consistantly fair and observant voters have on an entry's average, and minimizes the effects that 'quirky' or downright dishonest voters can have.
Trust us. It works."
Message edited by author 2004-04-15 10:51:04. |
|
|
|
04/15/2004 10:50:03 AM · #11 |
Originally posted by EddyG:
Basically, "the level of karma you have is automatically determined by your personal voting habits. It drops when you deliberately give an image what you don't think it deserves. It raises when you rate all the images in a contest fairly."
Your karma also increases for voting on more images in a contest, which encourages more voting participation.
If you read the details, your vote just has to be within +/-3 of the final weighted score for your karma to increase. So if the final score is 7.0, then anyone who voted within 4 - 10 is deemed to be voting fairly and their karma goes up (until it reaches some maximum). Anybody who voted below 4 would have their voting karma reduced -- and the higher the deviation the more the karma reduction (so somebody who voted a 1 is dinged more than somebody who voted a 3).
That also means that average images (those that score in the middle of the pack, near 5.5) have a very wide scoring region -- from 2.5 to 8.5. Which is plenty of room for "individual voter expression". In those cases, users who voted 1, 2 (dishonest voters) or 9, 10 (friend of the photographer) would have karma reductions, everyone else would increase their voting karma.
Quoting: "this is not a system of punishment and reward, nor is it a judgement of your personal tastes. It's merely a way to measure your ability to discern actual image quality, and allows plenty of room for your individual tastes to come into play without damaging your karma. It maximizes the effects that consistantly fair and observant voters have on an entry's average, and minimizes the effects that 'quirky' or downright dishonest voters can have.
Trust us. It works." |
There are times when I vote something very low because it doesn't meet the challenge IMO. However, it might get quite a high score. This would bother me. |
|
|
|
04/15/2004 10:51:07 AM · #12 |
| Could it just do my voting for me? If it knows whether or not I gave it what I thought it deserved, it might as well free up some time for me and just place the vote itself. |
|
|
|
04/15/2004 10:52:27 AM · #13 |
Originally posted by EddyG:
Basically, "the level of karma you have is automatically determined by your personal voting habits. It drops when you deliberately give an image what you don't think it deserves. It raises when you rate all the images in a contest fairly." *snipped*
Trust us. It works." |
Baaaaaa. (that's my best sheep impersonation :) ) |
|
|
|
04/15/2004 10:53:57 AM · #14 |
Originally posted by BobsterLobster:
There are times when I vote something very low because it doesn't meet the challenge IMO. However, it might get quite a high score. This would bother me. |
"Disagreeing" with a single image would not have any effect your voting karma. It would be more than offset by all the shots where you fell within the "scoring region". |
|
|
|
04/15/2004 11:03:53 AM · #15 |
Simplest of all... I'd like NOT to see my score until after the challenge ends.
You can't see anyone else's score til it's over can you? I think that would really solve the problem completely! I know everyone wants to see their score, but it does sway your vote... so, I opt out of seeing my own score! Anyone agree? If so, is this change something the DPChallengers will make for us?
Arie
|
|
|
|
04/15/2004 11:04:47 AM · #16 |
Well, I didn't realized the problem about "voting fast to see my scoring"... it can be a drawback, but allowing puntual changes after closing you votation could be a partial cure as mentioned before.
Anyway, actual system is also OK with me: What is good/bad for one is good/bad for all, so it is 100% fair. I was just wondering if it could be done better.
EddyG: You proposal is quite similar with mine if not 100% equal: you close "your" voting when you want, just when you are done, like in your idea. |
|
|
|
04/15/2004 11:04:53 AM · #17 |
Arielle: Go to your Preferences and uncheck the "My Scores" box and you're all set ( now just convince everybody else to do that!!)
Message edited by author 2004-04-15 11:06:23. |
|
|
|
04/15/2004 11:07:13 AM · #18 |
| Another possibility: If you enter a challenge, you can't vote on it, just add comments. Maybe that will promote more comments. |
|
|
|
04/15/2004 11:47:03 AM · #19 |
Originally posted by EddyG:
My idea is to hide scores/comments until users have clicked an "I'm finished voting" button (which wouldn't be available at all until you've reached the required 20% minimum). At that point, users are free to go back and leave comments, their votes would be "locked", and their score and comments would be displayed on the front page as they are now. |
Can't go for that. People would be closing off their voting as soon as they got to 20% so they could see their own score. It's guaranteed that the overall total number of votes would decrease significantly. In no time at all voting on only 20% would become the accepted norm.
|
|
|
|
04/15/2004 11:53:27 AM · #20 |
Statistics say that the scoring on this site is amazingly conformal and consistent. I am constantly impressed with the curves and the shifts that produce a winner or loser.
I would guess that the unhappiness stems from an unwillingness to have a couple of ones in the curve. If this is the case, make the voting from 3 to 10 or add 'fairy' points via software. Either way you are still going to see scores at the low end of whatever rage you use if (in someone's opinion) the photo sucks.
I don't think there is a problem. If there is, then it needs to be more defined than 'my scores drop'. Everybody's drops, it's consistent, deal with it. Or get a grant to study it~
|
|
|
|
04/15/2004 11:53:38 AM · #21 |
| I vote on no scores showing until the end of the week when we know the winners, etc. I find I get addicted to looking at my score anyway, when I really can't do anything about it and I should be out looking for more photos to take for the next round. I like seeing the comments because they tell me what I can do better/what I'm doing right. I can deal with only seeing the comments until the end of the week. I'd like to see this with both member and non-member challenges. |
|
|
|
04/15/2004 11:53:55 AM · #22 |
Personally, I don't understand the "revenge" or "that photo's not better than mine!" scoring mentality. If one feels that one is being cheated score-wise it does nothing to help the problem by repeating the same mistake. I actually tend to adjust my scores upward later on.. very rarely do I think I've scored something too high. I'm usually overly-critical at the beginning, then, later on, having absorbed the quality of the shots as a whole, I get a better feel for what stands out as the best of the challenge and give those photographs higher marks.
I also like being able to see my score, and I think forcing people to vote would just make scoring worse. I'd rather have fewer voters than a bunch of bitter ones nitpicking out of spite. |
|
|
|
04/15/2004 11:57:57 AM · #23 |
Actually, I grade on a curve, so I adjust scores later to make sure I gave what I thought was the best of the lot the highest scores, and move things around. I think a lot of people do that.
But it gets me thinking about people who vote a lot of low scores to inflate their own status: what if the site automatically "curved/normalized" your scores at the end--so the ones you gave the highest score got the top mark and the rest followed the distribution? Then everyone would be voting via the same scale, and everyone would be giving a good distribution of marks. |
|
|
|
04/15/2004 11:58:02 AM · #24 |
Originally posted by nshapiro:
Another possibility: If you enter a challenge, you can't vote on it, just add comments. Maybe that will promote more comments. |
Thats seems like a good idea, but the problem would be that non-members can't vote on member challenges. So if most of the members entered the member challenge there is no one left to vote. |
|
|
|
04/15/2004 01:13:33 PM · #25 |
There are other ways to score things...fujimugs is a similar site, and there instead of giving 1-10, you pick favorites whwn you go thru the pics to 'vote' and comment. At then end, you have a page of favorites. You arrange those pics from you most favorite to the least. Your top 10 pics, in order, get points (10 for a first place, 9 for second, etc). Pic with highest points wins. You can see on your entry (afterwards) how many votes you got for any of the 10 places (like here we see how may 1s,2s,3s etc we got).
You still get comments.
You don't have to actually vote on 400 pics. only click on your favorites and rank them at the end.
ranking instead of voting hard numbers is interesting in concept..
no one gets or gives 1s, 2s, etc so no one gets upset.
you still place overall by your points.
Both systems work. I find the other one better in some ways, this one better in others. Fewer off topic pics there for some reason. Smaller site yes, but you'd think that gratuitous cat/flower/sunset shots would be entered at a similar ratio. I tend to agree with the top 10 there more than the top 10 here too..not sure why that is either. Makes me think that system is better. Until i get a ribbon here..the this one will be better LOL
|
|
