Designing an image quality experiment...

Nikon AF-S DX Zoom Nikkor 18-135mm f/3.5-5.6G IF-ED

08/09/2004 01:37:04 PM · #1

I'm calling the collective wisdom of the experts here... I'm trying to design some image quality experiments to help me decide on whether or not I want to work with the dcraw utility for converting NEF images, or just stick with high quality JPEGs. I've read all the philosophies and theories, but I want to put images on my monitor that make the decision clear. In addition to dcraw converstion vs. JPEG, I will also compare to Nikon's converter (Picture Project / Nikon Capture) just for interest. I hope to post the results in an article (somewhere?) to help others as the information is spotty at best.

What makes this effort a bit less obvious is that I'm using Linux as my digital darkroom platform. The implication is that I don't currently employ a fully 24 bit workflow. While Cinepaint is technically available (but ugly), I use GIMP 2.0, so am limited to 8 bit JPEGs. I'd like to illustrate the types of photos which may (or may not) benefit from an 8 bit RAW workflow (similar to PS Elements). There are some maniplulations available to me in 24 bit (PPM format) like gamma, white balance and brightness, but anything else (sharpen, etc) is in 8 bit mode.

Can anyone suggest types of images to include, ideas for experiments, best way to present the data / results, etc? I've got a few I've tried and thus far am showing some shots are improved by a RAW workflow, and others were better under JPEG. This seemed complex enough that I shouldn't rely on my feeble mind when there are som many more experienced than myself!

Thanks in advance!

joebok

Fujifilm X100F

08/09/2004 01:51:55 PM · #2

This page, //www.imaging-resource.com/TIPS/TESTS/TESTS.HTM, has the most exhaustive explanation of good image tests that I've seen - in particular the "davebox" is designed to show a number of issues common in digital cameras. These tests are for comparing cameras, not so much what you are after but there might be some good ideas there for you.

08/09/2004 02:37:56 PM · #3

I have the exact same setup as you, cg. D70, Linux, Gimp 2.0, dcraw. And I waver between raw and JPEG.

I think the biggest difference isn't in sharpness, because fine JPEG has a great amount of detail. But I recently found that quality is much improved if I turn off the sharpening in-camera completely. I just chose Optimize->Custom->Sharpening->None with all other custom settings left at defaults. This was any noise isn't exaggerated by the sharpening and JPEG compression. I use NeatImage (works great under Wine!) to clean up images much of the time, and apply unsharp mask in postprocessing when I can tweak it carefully.

The biggest raw advantage seems to be in high contrast or badly exposed shots. The 12 bits of resolution lets you pull a lot more detail out of the extreme dark areas. I've had good success using dcraw to convert from NEF to 16-bit TIFF, then use NeatImage on it, then open it in Cinepaint, tweak levels, curves, colors in Cinepaint. Doing this part of the processing in Cinepaint seems most important when you need lots of adjustments, because each time you do a curves, levels, or whatever the pixel values are getting rounded off and a little detail is lost, so better to do this part in 16 bit than 8 bit. Then I export from Cinepaint to a 8-bit PNG, so I can open it in Gimp 2.0 (which has a much improved JPEG save dialog from Cinepaint) and do the rest of my work here because Gimp 2.0 simply kicks it. Except for that pesky missing 16 bit color. Tempted to get into the code myself and hack around a bit, but I understand it's a bit a large job, to say the least.

Anyway, I usually just shoot fine JPEG -> use NeatImage save to 16 bit TIFF -> use GIMP2 to process -> final image. Only when I think I'll need maximum color and brightness detail and high contrast have I shot raw.

Oh, one other advantage of RAW on the Nikon D70 you might not have noticed: RAW SHOOTS A HELL OF A LOT FASTER THAN JPEG. It's weird. I think you only get like 1.5 or 2 fps in JPEG (basic, norm, fine, doesn't matter) but 3 fps in RAW. Try continuous drive, the raw performance is stunning but will devour a 1 gig card in scant seconds. So when I need to capture some critical action, raw is often my choice for that reason alone. At my friend's wedding ceremony last month, I switched to raw for my shots of the kiss, so I wouldn't miss that perfect moment. I got about 10 shots of the kiss this way.

Besides taking up a lot more space, I lose access to the EXIF data by using RAW. I'm thinking making a program to extract the EXIF from the NEF and merge it with the output TIFF from dcraw. I wish EXIF could be stored in PNGs.

Hope you post your findings here if you do do some tests on raw/JPEG.

08/09/2004 02:41:50 PM · #4

Originally posted by joebok:

This is definitely going to a good resource... By incorperating some of their ideas I'll have a much more rounded group of tests. The trick is I need to take pictures of things / scenarios which bring out the best and worst of digital imaging. For example, images with really wide tonal ranges, tricky lighting, extreme detail, etc.

The basic idea I have is to set up the tripod and take one shot in RAW, and one in JPEG, then compare them (RAW unedited, RAW Edited, JPEG unedited (since JPEG unedited is really pre-processed by the camera) in some way that helps to show strengths / weaknesses of each image.

I know we have some serious imaging techies on DPC... Any one else have ideas / links as far as good examples?

08/09/2004 03:00:57 PM · #5

Originally posted by skylen:

Good point - I need to turn off the sharpening. Completely forgot about that, and it would be dumb to compare a sharpened JPEG to a RAW file. Thanks for the reminder!

I've messed with NeatImage under WINE, but didn't have much luck. Had a lot of crashes and the fonts came through really goofy. I upgraded to FC2, so perhaps with the updated WINE it'll be improved. I've found that other than really noisy images I can get equivalent resuts playing with GIMP's selective guassian blur filter. Slow, but it keeps me in a single program which is more efficient overall.

I haven't had a shot where I needed to rescue the exposure since I've been working on this experiment, and thus far my shots have been against using a direct dcraw conversion with all defaults. I found generally better contrast, but what really blew me away was how much better the color balance was in RAW. I'm guessing that when custom settings moved me into sRGB-I (portrait mode) it compressed the color range poorly for natural settings (which are supposed to be in sRGB-II or landscape mode). I've since switched my custom settings to Adobe RGB rather than sRGB as it is supposed to preserve the widest spectrum.

I envy your patience for inserting cinepaint into your workflow. I have trouble with the idea of moving through so many different programs as it takes so much time. I don't mind spending time, but it's tough to justify moving from a 1.x gimp that supports 24bit then into a 2.x GIMP that supports 8bit for final tweaks. GIMP 2 is a pretty sweet program, so I'm trying to keep everything else before it as a CLI utility I can easily batch (like ImageMagic, dcraw, etc.). I might have to reload Cinepaint and try adding a few experiments to show its strengths and weaknesses.

As far as your EXIF data, I'd suggest you try metacam. Works flawlessly on NEF and JPEG and can display the advanced / proprietary tags that most stand-alone EXIF viewers miss.

I'll definitely post something here when I have something useful... Not sure if this will be the final place as I don't know if enough of the user population would find it useful, but I'll place a reference here at a minimum. Thanks for the information, and I'm glad to hear someone else out there is making use of Linux and GIMP!

08/10/2004 03:59:17 AM · #6

Originally posted by cghubbell:

I've messed with NeatImage under WINE, but didn't have much luck. Had a lot of crashes and the fonts came through really goofy.

Make sure you're not trying to use the Demo version of Neat Image, it has problems with Wine (most importantly, the menu bar doesn't work at all). The Demo version is v4.0, while the full versions are v4.25. Also using the newest Wine version might help, I'm using a July 2004 build.

Originally posted by cghubbell:

I found generally better contrast, but what really blew me away was how much better the color balance was in RAW. I'm guessing that when custom settings moved me into sRGB-I (portrait mode) it compressed the color range poorly for natural settings (which are supposed to be in sRGB-II or landscape mode). I've since switched my custom settings to Adobe RGB rather than sRGB as it is supposed to preserve the widest spectrum.

Does the color model (sRGB or Adobe RGB etc.) set on the camera affect RAW images? I thought if might only affect JPEGs but I'm not sure.

Originally posted by cghubbell:

I envy your patience for inserting cinepaint into your workflow. I have trouble with the idea of moving through so many different programs as it takes so much time.

Well, I only use Cinepaint when I'm really spending a lot of effort on a specific image to get the best color. Really rarely, but it seems to be useful on occasion. I agree, most everything can be done satisfactorily in Gimp2 though most of the time.

Originally posted by cghubbell:

As far as your EXIF data, I'd suggest you try metacam. Works flawlessly on NEF and JPEG and can display the advanced / proprietary tags that most stand-alone EXIF viewers miss.

Cool, I will check it out.

Cheers,
skylen

jonpink

Panasonic Lumix DMC-LX2

08/10/2004 05:22:31 AM · #7

lots of people / web sites seem to go to so much trouble obtaining technical data about shrpness resolution etc.

When I wanted to know those types of things, I simply took a few pictures - printed them out on A3 and looked at them.

After all that's how you really judge a photographs quality isn't it?

08/10/2004 09:24:48 AM · #8

Originally posted by jonpink:

It would seem so, but in fact it's not so simple... For one thing I find that some types of images come through the JPEG stream looking more attractive than those that come through the RAW converter. Of course, I'm sure that there are some additional means of analyzing images which would show non-obvious flaws or advantages. I have a beautiful image of a lilly which came through the JPEG looking great, but not so great in the RAW. I then have an image of clethra (summersweet) which was vastly improved by the RAW process.

So, I was oringinally hoping it would be that simple, but after trying the "look and see" approach I found that in 5 images I had not been able to conclusively tell whether RAW was always better, sometimes better, or could be better if I knew how to use it more proficiently.

jonpink

Panasonic Lumix DMC-LX2

08/10/2004 09:31:10 AM · #9

Ah I see ;D

Well for fixing pictures I find RAW much better. As for the overall results I don't think there is much in it.

As you said if you can't tell much difference, then ther isn't much difference, and if that's the case then shoot in JPEG as it's quicker and more portable.

I will only shoot in RAW if it's a large paid job. I have just under 200 images on istockpro, and had to convert all those from RAW to Jpeg which was a big pain and very very time consuming. So now I shoot in JPEG in case i want to upload more.

Message edited by author 2004-08-10 09:34:48.

08/10/2004 09:41:26 AM · #10

Originally posted by jonpink:

My guess is that I'll end up shooting JPEG for generic photography but use RAW when I know I have a print-worthy subject. I think of JPEG like a Polariod instant image and RAW like shooting film. The RAW is a negative and will always need developing. Both approaches have their merits.

What I really want to do is illustrate the scenarios where it is advantageous to shoot RAW. I have some images from my initial comparison which had no real image clean up involved - just run it through the converter and load it into GIMP. And some looked better (although color shifted) in the camera JPEGs while others looked better from the RAW converter. If my work is successful it will make it obvious when to use RAW even if image clean-up isn't necessary.

Message edited by author 2004-08-10 09:42:01.

Canon EF 100mm f/2.8 USM Macro

08/10/2004 10:16:32 AM · #11

Originally posted by cghubbell:

And some looked better (although color shifted) in the camera JPEGs

This sounds like poor raw conversion software; I can't think of any reason why you shouldn't be able to "develop" as good an image as the camera-generated JPEG from the corresponding raw file.

FWIW, I always shoot in raw, and thanks to Capture One, don't find it to be an "extra burden" at all. If I had the camera set properly, then I can convert hundreds of raw files to JPEGs using the default ("as shot") settings with minimal effort. But when necessary, or when I want to try and tweak an image to see if I can make it better, I appreciate the extra benefits shooting in raw affords.

Message edited by author 2004-08-10 10:21:57.

Spork99

OM System OM-1

08/10/2004 10:30:35 AM · #12

Originally posted by EddyG:

Originally posted by cghubbell:
And some looked better (although color shifted) in the camera JPEGs

This sounds like poor raw conversion software; I can't think of any reason why you shouldn't be able to "develop" as good an image as the camera-generated JPEG from the corresponding raw file.

FWIW, I always shoot in raw, and thanks to Capture One, don't find it to be an "extra burden" at all. If I had the camera set properly, then I can convert hundreds of raw files to JPEGs using the default ("as shot") settings with minimal effort. But when necessary, or when I want to try and tweak an image to see if I can make it better, I appreciate the extra benefits shooting in raw affords.

I second that Eddy.

You can also tweak one image and apply those setting to all the other selected RAW files for conversion.

08/10/2004 10:31:47 AM · #13

Originally posted by EddyG:

Actually, the RAW image was closer to the true color than the Camera's JPEG. It's very possible that the auto white balance (or something else?) made a poor choice - there's many variables which could have made this happen. Dcraw is the same code base used by Adobe for their Photoshop RAW converter, so I'm quite certain that it's not the converter's quality which is the problem. In fact, all of the commercial programs in the list below use dcraw's code base in their converters:

* Adobe Photoshop
* Bibble by Eric Hyman
* BreezeBrowser by Chris Breeze
* Conceiva Lightbox
* cPicture by JÃ¼rgen Eidt
* dcRAW-X by Bryan Chang
* GraphicConverter by Thorsten Lemke
* IrfanView by Irfan Skiljan
* IRIS image processor for astronomers
* Lightbox by Josh Anon
* Photo Companion by Jeff Moore
* PhotoReviewer by Ben Haller
* PolyView by Polybytes
* PowerShovel-II by Luc Minnebo
* RawDrop by Frank Siegert
* SilverFast DCPro by LaserSoft Imaging
* VueScan by Ed Hamrick

I can do batch conversion quite simply and efficiently using dcraw in Linux. Just need to get more familiar with the settings to make the most of it. I agree that you should be able to develop as good an image as JPEG. BUT, I would imagine that in the rigth circumstances RAW can do better then JPEG. I'm working on illustrating what those scenarios are so that I don't HAVE to shoot in RAW when it's not beneficial. That would save me considerable storage space, and editing time over a few weeks of shooting. I always save my originals, so any time I shoot RAW I end up storing a (large) NEF and a (large) JPEG.

Just trying to tune my workflow in as scientific a manner as can be done... Maybe I'll get more proficient with the RAW converter and go 100% RAW. We'll see...

08/10/2004 10:51:14 AM · #14

Originally posted by cghubbell:

In fact, all of the commercial programs in the list below use dcraw's code base in their converters:

But your list is missing the most important raw converter: Nikon Capture (or whatever it is called). Everything you listed is "third party", and rather than reinventing the wheel, they use'd Dave Coffin's excellent public domain code as a "baseline" for at least part of their raw strategy. There is nothing wrong with that, except that Dave doesn't have any inherent knowledge of how JPEGs are created in the camera -- which the Nikon software engineers would.

Similarly, for Canon, it would be Canon's Digital Photo Professional software.

And notice that Capture One is not on that list, as they have developed their own raw conversion stuff from scratch, which is considered to be among the best out there.

While there may be instances where shooting in raw would offer minimal benefit over shooting in JPEG (i.e., you got everything correct about the exposure as shot), personally I'd rather not worry about switching between raw and JPEG and risk taking a great shot that is underexposed or overexposed or improperly white balanced because I thought I had everything "right" and was in JPEG mode.

Of course the alternative is to shoot in raw + JPEG (I'm assuming the D70 has that option), and simply throw away the raw file after you've looked at the JPEGs and are satisfied with the results. That way you get the best of both worlds. Me, I rather get a few more raw photos on my memory cards by not storing a full-size JPEG. =]

08/10/2004 02:13:30 PM · #15

Originally posted by EddyG:

Originally posted by cghubbell:
In fact, all of the commercial programs in the list below use dcraw's code base in their converters:

But your list is missing the most important raw converter: Nikon Capture (or whatever it is called). Everything you listed is "third party", and rather than reinventing the wheel, they use'd Dave Coffin's excellent public domain code as a "baseline" for at least part of their raw strategy. There is nothing wrong with that, except that Dave doesn't have any inherent knowledge of how JPEGs are created in the camera -- which the Nikon software engineers would.

Similarly, for Canon, it would be Canon's Digital Photo Professional software.

And notice that Capture One is not on that list, as they have developed their own raw conversion stuff from scratch, which is considered to be among the best out there.

While there may be instances where shooting in raw would offer minimal benefit over shooting in JPEG (i.e., you got everything correct about the exposure as shot), personally I'd rather not worry about switching between raw and JPEG and risk taking a great shot that is underexposed or overexposed or improperly white balanced because I thought I had everything "right" and was in JPEG mode.

Of course the alternative is to shoot in raw + JPEG (I'm assuming the D70 has that option), and simply throw away the raw file after you've looked at the JPEGs and are satisfied with the results. That way you get the best of both worlds. Me, I rather get a few more raw photos on my memory cards by not storing a full-size JPEG. =]

I agree with you... Although I have to say that I haven't heard too many complaints about Photoshop's RAW import. I was just trying to express that while it may indeed be different than C1, Nikon Capture, etc, (all of which I agree are top-notch) dcraw is an excellent quality product. In fact, there is a review here that suggests dcraw does a superior job to Canon's and C1. I'm reserving judgement, but it is an interesting link.

As I've stated, my goal is to write an article which includes very clear examples of in what scenarios it is beneficial to shoot RAW when using dcraw (vs the traditional and expensive solutions), and how an efficient workflow would look.

Another point of interest is illustrating any shortcomings of the GIMP editing being limited to 8-bit - in that circumstance can the RAW workflow still be demonstrated as superior? There's no end to the text examples out there telling what and why, but I haven't actually seen any well illustrated articles displaying this workflow and actually demonstrating examples of situations where use of RAW conversion using dcraw+gimp improved the resulting image. We all know GIMP is great for JPEG editing, but what about taking it to the next level? Not as much has been written in that space.

There's a lot to be said for NOT spending $500 on Photoshop and another $100 on Nikon Capture (not that I disagree with their quality and benefits!). For many of us that money would be better spent on glass, or trips to scenic destinations. In fact, as far as I can see, GIMP 2.0 is much more capable than PS Elements, and $99 less.

The assumption is usually that you need a Windows or Mac workstation and photoshop to have professional results. I'd like to create a professional-grade analysis (or as close to it as I'm capable) of an alternative as I believe that in most cases, a Linux workstation with GIMP+dcraw can yield similar results. And yep, there are color management solutions available for both printers and displays in GIMP under Linux.

Would John Shaw go this route? Of course not - no argument :) But there's a lot of folks who sell their work either through DPC prints, or local craft shows, etc. Who are "semi-pro." Their work requires a higher quality workflow than basic JPEG editing, but doesn't justify an extensive investment in PS and plugins. I think in this case, saving significant amounts of $$ can make a big difference as long as the final quality is not negatively inpacted. It's not for everyone, but I think that with recent (huge) advances in Linux desktops, the gap is narrowing and worth a quality analysis.

08/10/2004 02:38:50 PM · #16

Originally posted by cghubbell:

Although I have to say that I haven't heard too many complaints about Photoshop's RAW import. I was just trying to express that while it may indeed be different than C1, Nikon Capture, etc, (all of which I agree are top-notch) dcraw is an excellent quality product. In fact, there is a review here that suggests dcraw does a superior job to Canon's and C1. I'm reserving judgement, but it is an interesting link.

I'm sure Photoshop's raw plug-in is quite capable. (I personally prefer the workflow of C1.) BTW, I went to that link, and saw a review comparing it to FVU (Canon's old "File Viewer Utility"), which has since been replaced by DPP (Digital Photo Professional). It would be interesting if the results were still so different. Also, I couldn't find a comparison to C1; it said "coming soon..." when I clicked it.

I guess I didn't realize that you were concentrating on an analysis/workflow that involved "free" OS/tools (i.e., Linux, The Gimp, dcraw, etc.) I'll be anxious to see what you find out!

08/10/2004 02:46:10 PM · #17

Originally posted by EddyG:

Of course the alternative is to shoot in raw + JPEG (I'm assuming the D70 has that option), and simply throw away the raw file after you've looked at the JPEGs and are satisfied with the results. That way you get the best of both worlds. Me, I rather get a few more raw photos on my memory cards by not storing a full-size JPEG. =]

Even if you were up for the wasted space of the JPEG, the D70 only has a RAW + Basic JPEG option, with no choice for the JPEG quality. :-( I swear I'm gonna hack that firmware like the Digital Rebel dude did and put in those teensy annoying misfeatures.

08/10/2004 02:50:35 PM · #18

EddyG, I was curious, do you use 16-bit color throughout your editing process? It only makes sense if you're working from RAW, right?

Even working from JPEGs, I would think that working with that 8-bit image, each color adjustment would lose much less detail in 16-bit than 8-bit color. My thinking is that if there are only 256 levels for each color component, then tweaking levels, then curves, then saturation etc. will have to round off to the nearest of those 256 levels each time you use one of those tools, progressively losing color precision. Perhaps this doesn't matter, practically speaking, though.

I might just shoot RAW all the time if it will help me get such fabulously vibrant and smooth images as EddyG's...

Canon PowerShot Pro 90 IS

08/10/2004 02:59:23 PM · #19

Originally posted by EddyG:

I guess I didn't realize that you were concentrating on an analysis/workflow that involved "free" OS/tools (i.e., Linux, The Gimp, dcraw, etc.) I'll be anxious to see what you find out!

Yep. Since I have a Win2k laptop with Nikon software on it I'll probably include something from that workflow just for comparitive curiosity but my real objective is to carefully explore a RAW workflow for Linux. Practically speaking, this would also apply to Windows/Mac since dcraw is written in ansi-c and can compile on almost anything. GIMP also runs nicely on Windows, so even though I'm focused on Linux this results should be interesting to almost anyone.

It'll take me some time to get this together but I'll be sure to post when I have something worth reviewing.

dwoolridge

08/10/2004 05:11:45 PM · #20

Originally posted by skylen:

There are still inherent advantages to incorporating RAW into 8-bit workflows. ACR (and I imagine C1 has similarities) does most of its work in a very wide gamut while working on the linear data (i.e. before gamma correction is applied). I don't recall which settings are applied post-gamma correction. Besides the advantages of WB and EV control, you will get consistently better results if you apply global changes (brightness, contrast, shadows, etc.) within ACR. This is especially true if you are going directly to 8-bit. With not very much work, you can even create standard color filter settings, although B&W conversions are best left until after RAW conversion.

Your output device will all too likely require 8-bit input, so unless you are certain you need 16-bit support (very heavy non-global editing - i.e. what can't be done in ACR), this is a good step towards getting your feet wet with a RAW workflow. 16-bit files can be very large (my latest project uses 16-bit tiffs whose files are over 2G in size - compressed!).

With dcraw you can achieve the same thing, by doing a linear conversion and using an editor which supports editing in a linear working space (this naturally excludes the GIMP). Was it in another thread someone mentioned a not-yet-stable release of a GUI RAW conversion plugin for the GIMP? It would be useful to know if the settings it allows are done before gamma correction or not.

coolhar

08/10/2004 11:20:42 PM · #21

Looking forward to the results of your experiment. A high quality all open source workflow is a very attractive idea.

08/11/2004 08:01:27 AM · #22

Originally posted by dwoolridge:

With dcraw you can achieve the same thing, by doing a linear conversion and using an editor which supports editing in a linear working space (this naturally excludes the GIMP). Was it in another thread someone mentioned a not-yet-stable release of a GUI RAW conversion plugin for the GIMP? It would be useful to know if the settings it allows are done before gamma correction or not.

The information on the GIMP plugin (which I would call very stable - works like a charm) is here. I don't see anything about how the conversion works. The main dcraw page is here. Again, I see no clear reference to how the conversion happens.

What is the linear decoding you mentioned? The options I have are listed below, but I'm not sure which you are referring to. My normal conversion is using camera WB, 24 bit PPM, and then I mess with brightness and gamma. I'm also not sure about the blue and red multipliers... Why is there no green? Would they be used for removing color casts, or are they for something different? I just ignore them at this point.

Raw Photo Decoder "dcraw" v5.88
by Dave Coffin, dcoffin a cybercom o net

Usage: dcraw [options] file1 file2 ...

Valid options:
-i Identify files but don't decode them
-c Write to standard output
-v Print verbose messages while decoding
-f Interpolate RGBG as four colors
-d Document Mode (no color, no interpolation)
-q Quick, low-quality color interpolation
-h Half-size color image (3x faster than -q)
-g Set gamma (0.6 by default, only for 24-bpp output)
-b Set brightness (1.0 by default)
-a Use automatic white balance
-w Use camera white balance, if possible
-r Set red multiplier (daylight = 1.0)
-l Set blue multiplier (daylight = 1.0)
-2 Write 24-bpp PPM (default)
-3 Write 48-bpp PSD (Adobe Photoshop)
-4 Write 48-bpp PPM