Decoding Art's Weighted Voting System

Nikon AF-S Nikkor 17-35mm f/2.8D IF-ED

09/07/2026 08:36:20 PM · #1

Buckle up, this is lengthy. Anyone who's entered challenges in the past several months has no doubt received a category breakout with or without weighted values attached to the categories from me. This has been something of an experiment that really goes back to the Paleochallenge Era of 2005 when we voted with stone chisels. The idea is that by voting separately in each of five categories and averaging that into the final score and putting the category scores in the comment, the photographer would get a much better idea of what I thought of the image (in the context of the challenge, ruleset, etc.) - it was easier than coming up with a comment each time and I felt it had more value to the participants than clicking a score and not leaving a comment. To be clear, THAT was the alternative with the exception of me occasionally leaving a "great job!" or more likely, some light-hearted, humorous quip. Sure, I could leave an honest, detailed critique, but ain't nobody got time for dat.

Anyway, I used to use a javascript app (originally developed by a fellow Neanderthal back in those caveman days) to calculate the score and build the comment that I would paste in before clicking the actual vote number on the voting page. Since AI, I rebuilt the tool into a Chrome extension that I used for about a month or two and then updated it to incorporate weighted values for each category. It allows me to insert a comment for each category, so I can get detailed if I want/have time to. Here's a screenshot (not a real entry):

Ok, so what's the point of this post? Well, I have been anxious to get some feedback (good or bad) about my process and whether anyone found it helpful or not (beyond the

✅) and today I got a message from

Fredster, letting me know that he used Gemini AI to inquire about my scoring and he sent me the response thread below. For the most part, AI gets it, except for a couple details like "his weights are always the same" - I keep them the same per challenge at a minimum, but I have tweaked them a couple times. I'll talk about that at the end of this LONG post and add my conclusions and notes.

Decoding Art's Weighted Voting System

Originally posted by Fredster:

I've been trying to figure out how Art scores entries. Here's a recent comment on one of mine:

Composition: 8 [w:2]
Technical: 10 [w:2]
Creativity: 8 [w:4]
Appeal: 7 [w:4]
Challenge: N/A (Free Study)
Overall Weighted Average Score: 8
Note: Weighted category voting.

AI: Here's what's actually going on.

1. How the math works

It's a weighted category voting system. Instead of treating every category as equally important, he assigns a weight [w:x] to each one, so some aspects of your photo affect the final score much more than others.

The category breakdown:

Composition — score 8, weight 2 — impact value 16
Technical — score 10, weight 2 — impact value 20
Creativity — score 8, weight 4 — impact value 32
Appeal — score 7, weight 4 — impact value 28
Challenge — N/A on a Free Study, so weight 0 and no effect on the math

The formula:

Add up the impact values, then divide by the sum of the weights.

Total points: 16 + 20 + 32 + 28 = 96
Total weights: 2 + 2 + 4 + 4 = 12
Weighted average: 96 / 12 = 8

2. What that says about his judging style

Creativity and Appeal are king. Creativity [w:4] and Appeal [w:4] carry double the weight of Technical [w:2] and Composition [w:2].

The "flawless technical" trap. Even a perfect 10/10 on technical execution only nudges the score up, because it's a low-weight category.

Where to focus. To pull a 9 or 10 out of him, the leverage is entirely in Creativity and Appeal. A single point lost there drags the average down much faster than a point lost anywhere else.

3. Are the weights always the same?

The four core weights are consistent. Only the Challenge category moves.

The fixed pillars — on every normal entry:
Composition is always [w:2]
Technical is always [w:2]
Creativity is always [w:4]
Appeal is always [w:4]

Because those four never change, Creativity and Appeal always dictate two-thirds of the baseline score.

The one variable — Challenge:

Free Study entries: no theme constraints, so he marks Challenge N/A (effectively [w:0]) and divides by 12.
Standard challenges: on regular contests like Markets & Bazaars or Strata, he rates how well you hit the prompt, usually at [w:4]. Divisor becomes 16.
Thematic exceptions: occasionally — Summer in the Big City, Cliff — he's scaled theme strictness down to [w:3], making the divisor 15.

Examples this is drawn from:
Lines And Curves Take Flight by Refocused — Comp 7 [w:2], Tech 8 [w:2], Crea 8 [w:4], App 9 [w:4], Chal 8 [w:4], Overall 8
Summer in the Big City by markwiley — Comp 8 [w:2], Tech 10 [w:2], Crea 7 [w:4], App 9 [w:4], Chal 9 [w:3], Overall 8

4. Why bother with weights at all?

He could just add the four scores and divide by four — but that would change how he judges entirely.

Without weights: 8 + 10 + 8 + 7, divided by 4, gives 8.25. In that version, a perfect 10 in Technical counts exactly as much as a 7 in Appeal.

With weights: by making Creativity and Appeal [w:4] against Technical and Composition [w:2], he's saying outright that he cares twice as much about imagination and emotional impact as he does about camera settings and framing.

A comparison. Take two photos:

Photo A (technically perfect, low creativity) — Composition 10, Technical 10, Creativity 5, Appeal 5. Regular average 7.5. Art's weighted score 6.6.

Photo B (technically flawed, high creativity) — Composition 6, Technical 6, Creativity 10, Appeal 10. Regular average 8.0. Art's weighted score 8.6.

The weights are his way of programming personal taste into a formula, so a photo with soul beats one that's mechanically perfect but boring.

5. So what does he actually think of my entry?

Reading between the lines, roughly: "This is an incredibly well-made, flawless shot that lacks a bit of a creative wow factor to make it unforgettable."

He's impressed by the craft. A 10/10 in Technical means focus, exposure, lighting and clarity were perfect — no amateur sloppiness anywhere. An 8 in Composition means the framing, lines and balance were strong too.

He found it safe. The 8 in Creativity and 7 in Appeal say the execution was stellar but the concept felt familiar. It didn't hit him emotionally or surprise him.

The weight penalty. Judged on technical execution alone, the score would've been higher. Because the image didn't grab him, the formula dragged it back to an 8.

Short version: he thinks the technical work is excellent and wants to see bigger creative risks.

__________________________________________________________________________

AI's conclusions like "This is an incredibly well-made, flawless shot that lacks a bit of a creative wow factor to make it unforgettable." are pretty much what I am hoping someone interprets from my numbers.

Ok, here's what I've learned and have yet to figure out:

- My time spent voting vs. just clicking the score is longer, but surprisingly not as long as I thought it would be - maybe 2x. I tend to study the photo a bit more in the context of the categories as I click the score in each one. Then Calculate/Compile, Copy to Clipboard, paste into comment field on the voting page, click the indicated score, then click Clear for the next image.

- I think the weights could just be limited to 1-3 or something. No category is 10x more important than any other. Also, keep in mind that the weights assigned are MY OWN opinion and I might only reassign them based on the challenge.

- Even with weighting, the overall score is rarely that much different than just averaging the categories, but - full disclosure - if I don't quite agree with the final score, I may rescore one or two of the categories until I get the final number that I feel reflects what I think the image is worth (in that challenge).

- My scoring average tends to be a full point higher than when I don't use the tool. Not sure how to interpret that and it sort of bothers me. Frankly, I have a hard time scoring below a 5 in any specific category and I think I should expand my use of the 1-10 range - even if I don't use the breakout tool. It would be interesting to see what challenge results would look like using a 1-100 scale.

- I don't use the tool on every challenge - about 80% maybe.

What I have been anxious to find out is how much value, if any, people are getting from my scoring and feedback method. I feel like it always comes down to what the voter values most and because we are all different, the combination of categories (fixed, but not necessarily finalized) and weighted values, the tool could be beneficial to everyone.

The current categories are:
-Composition
-Technical
-Creativity
-Appeal (considering changing to Originality)
-Challenge (relevance)

I am considering adding one or two open categories that could be filled in on a per challenge or even per entry basis that would allow me to reward things like Humor or something.

Any and all suggestions, criticisms, etc. are welcome.

ETA: Links to the Chrome extension, instructions, user guide here: https://artroflmao.com/dpc/voting_notes/

Message edited by author 2026-09-07 20:40:24.

tanguera

Nikon D750

09/07/2026 09:31:25 PM · #2

Personally, I love your system, and I actually appreciate it even more now that I know which categories you give most weight to: creativity and appeal. I know everything is subjective, and what you might find uninspiring might knock someone else's socks off. But I find your method appealing.

I do think an added category for humor, or x factor, or unexpected, could be added when needed.

roz

Olympus OM-D E-M1 Mark III

OM System 90mm f/3.5 Macro IS PRO M.Zuiko Digital ED

09/07/2026 09:36:09 PM · #3

i did wade through most of your post . .
and was following along quite nicely till you got into the details . like fixed pillars and weights etc ..
sorry about saying wading through .. it was a bit too technical for me ..

however .. i love your voting style .. and love that you say you spend more time on each photo ..
you are putting your money where your mouth is .. which i find refreshing in regard to the voting ..
many ppl hide behind anonymity especially when giving relatively low votes .. and i know that subject has been done to death ..

anyway .. thanks for spending the time .. i'm sure its hugely appreciated by most of us here ..xx..

GeneralE

Canon PowerShot S3 IS

Canon 12x IS USM (S3 IS built-in Lens) -- 36-432mm (35mm EQ)

09/07/2026 10:07:19 PM · #4

I think "creativity" and "originality" are too close (choose one) -- the current "appeal" category is fine and more closely reflects your personal reaction.

Message edited by author 2026-09-07 22:07:52.

kanaj

Canon EOS M200

Canon EF-M 55-200mm f/4.5-6.3 IS STM

09/08/2026 01:44:44 AM · #5

Personally, I applaud the effort, Ken. I have found it to be a valuable data point in my own ingestion of others' feedback here, and from the first when you began using this method I have been wondering whether you experienced the effort as more or less cumbersome overall. I'm glad to hear that you seem to find it mostly a positive experience on your end.

I do think it's telling that you find that your average score is higher for having used a more methodological approach and spending, on average, a bit more time considering each before casting the vote. I doubt that would be unique to you, but I do (as a statistical realist) suspect that there are also those of us who would tend to vote lower on average if we spent more time with each individual image before casting a vote. In other words, if we were all truly "objective" (in whatever sense that can even be relevant with regards to something as intrinsically subjective as art), I think we'd invariably find that the statistical bell curve would increasingly center over the 5 vote across the board. My sense right now is that the greater the number of users active on the site at any given time, the lower the bell curve sits in the overall range of votes, and the fewer the overall number of active voters, the higher it sits. My working sociological theory of this revolves around the social investment of the average voter when the number of voters is fewer: I suspect that most of us tend to vote more charitably when the (perhaps unacknowledged) psychological end goal is to feel like we're part of a supportive community rather than jurists on an art panel--and that when the average number of participants active on the site is higher, what accompanies that phenomenon is the presence of a greater concentration of individuals who are simply less psychologically invested in the outcome of the enterprise to begin with, and so they tend to vote with less psychological precognition in the first place. This is rooted in a theory of social participation, of course--namely, that those who are most consistently active on this site are photographers whose primary value exchange for the effort involved in consistent participation is the social reward of belonging and mutual support rather than, say, secondary goals of learning new techniques, obtaining "objective" feedback, or the aesthetic satisfaction of seeing a variety of novel images for inspiration and enjoyment. All of those are, of course, held in different proportion to our individual personalities; I only mean that my suspicion is that the average voter when there are, say, 30ish votes cast on a challenge is that better than 15 of those voters, if polled, would probably rank social belonging and mutual encouragement as the highest value they extract from their participation in this platform--because those who have alternative primary goals tend to self-exempt over time and pursue their goals via alternative means and use this platform more sporadically.

It's just a theory. But since this is a long post, it felt like a place to air that theory in good company. :-) But no matter how wildly accurate or inaccurate it might be, my sense is that no matter how "objective" any of us try to be with our voting, the actual vote cast--whether high or low--frequently reflects less on how technically accomplished a particular image is and more on how well we "connect with" or "respond" to the perceived artistic expression (or whatever it represents on a conscious or subconscious level to us). It might be easier to tack that down if there was more liberal commenting, but honestly, I think many of us struggle to cogently articulate why we do or don't "like" a particular thing in photography or digital imagery. To that end, I think your weighting system is probably pretty fair as a representation of how art generally works on us--whether we're shopping for a piece of waiting room art or designing a new kids room or curating a gallery for a fine art exhibit. If anything, I suspect that codifying the more subjective side of artistic response to an image has the impact of restraining its impact rather than amplifying it, even when the weight assigned to it is higher. As in, you're probably less likely to let it disproportionately affect your final vote--which may inform why your average vote is higher, too.

I'd be interested in your own read on the psychology behind feeling compelled to "tweak" the numbers until you arrive at that number you feel an image "should" receive when the algorithmic calculation doesn't produce it. To me, it sounds a little like an (entirely understandable) admission that sometimes even in your own schema there are times when an image connects with you (or possibly utterly fails to connect with you) in a way that you can't quite delimit by the numbers, as it were?

Either way, I suspect it makes a lot of intuitive sense to most of us here. :-)

09/08/2026 02:39:53 AM · #6

Originally posted by kanaj:

Personally...yada yada yada...

Jeez, I can't be expected to read all that! 😜

Kidding. I read it and appreciate it and I agree with most if not all of your conclusions or theories, etc. I'll respond in a little more detail tomorrow.

09/08/2026 02:48:55 AM · #7

Originally posted by GeneralE:

I think "creativity" and "originality" are too close (choose one) -- the current "appeal" category is fine and more closely reflects your personal reaction.

Agreed. I think that is what i thought i wrote or meant to write.

I like the idea of "Originality" over Creativity because some images have been produced very creatively, but often are regurgitate concepts. I'm sure im guilty of that and this voting does help me put more thought (and sometimes even effort) into my entries - much like how some people claim that commenting on others' images helps improve their own photography.