Camera test images question - DPChallenge Forums

Canon 12x IS USM (S3 IS built-in Lens) -- 36-432mm (35mm EQ)

12/08/2008 12:41:28 AM · #1

I have a question about testing procedures for two cameras. I am referring to these two images from dpreview.com:

Canon Rebel 450D test image

Nikon D90 test image

The source page is here.

The exif data from those two images say they were captured with not the exact same settings, e.g. ISO settings differ (100 vs 200), also the exposure times differ (1/25 sec vs 1/30 sec).

My understanding was that when comparing two cameras, one must capture the same scene with identical settings. Or am I wrong?

GeneralE

Canon PowerShot S3 IS

12/08/2008 12:47:49 AM · #2

Not all cameras have the same settings available -- they are probably using the closest available.

Also, if there was a significant time gap between tests, they may have changed their "standard" settings.

Why not sumbit a query dirctly to dpreview with this question, and get a real answer?

Canon 12x IS USM (S3 IS built-in Lens) -- 36-432mm (35mm EQ)

12/08/2008 01:01:55 AM · #3

I had already asked them. Waiting for an answer. I just wanted to confirm on DPC if I misunderstood the test procedure.

Also, I checked that both cameras have ISO 200 available at least, so the Canon one could definitely be set at that.

This particular test is very interesting, because the results suggest the two perfom almost equally. When in fact Nikon could have performed better if the Canon was set to ISO 200 too... probably getting more noise in the shadows.

GeneralE

Canon PowerShot S3 IS

12/08/2008 01:19:06 AM · #4

They might be set at the "lowest available" ISO value, which would still be a fair comparison.

Canon 12x IS USM (S3 IS built-in Lens) -- 36-432mm (35mm EQ)

12/08/2008 01:26:50 AM · #5

Originally posted by GeneralE:

They might be set at the "lowest available" ISO value, which would still be a fair comparison.

Uhm.. may be not. Sorry but thats just my opinion. The settings must be identical as long as it is possible to make them so. That one is a detailed technical review, and so one would expect that. Especially since a difference in ISO settings could cause difference in noise levels. No? Unless the ISO scale implementations are not required to be standardized.

I will wait for their confirmation and share it as I hear back.

GeneralE

Canon PowerShot S3 IS

12/08/2008 01:32:01 AM · #6

If I was evaluating cameras for noise, I'd for sure want to know what the best available is. It makes no difference if Camera A is worse than B at 200, if camera A is better than B at 100 and B doesn't have 100 available; A would then have the least noise available.

Of course, it depends on what you want to do to determine which is "best" -- in this case, A would be best for the studio product shooter with static subjects and lots of light, and B for the sports/photo-journalist who needs faster shutter speeds.

12/08/2008 01:49:03 AM · #7

Originally posted by GeneralE:

Agreed. ETA: I re-read it. So if B doesnt have 100ISO available, I would test them both for the setting that IS available. Thus, at 100ISO, they cant be compared and perhaps B is the better one by default. However, we wont know if B is still better/worse at 200ISO since we didnt test that. I, for one, would not conclude that A is better than B at 200ISO, or even vice versae, if I didnt test that. So I understand what you are trying to say.. but that test is missing in the dpreview review to give a fair conclusion.

Originally posted by GeneralE:

Of course, it depends on what you want to do to determine which is "best" -- in this case, A would be best for the studio product shooter with static subjects and lots of light, and B for the sports/photo-journalist who needs faster shutter speeds.

Makes sense again.

Its just that I, as a reader, was expecting the test images to be under identical conditions for me to be reading about their comparison at all. I just could not think of comparing two objects if they are not evaluated under the same conditions.

(((I guess I am talking too many numbers here... going to get some sleep now:-)))

Message edited by author 2008-12-08 01:56:25.