Question about RAW data - DPChallenge Forums

Sony SAL-18250 18-250mm f/3.5-6.3 High Magnification Zoom Lens

01/02/2006 10:20:15 AM · #1

My camera takes ~4mb pictures. In RAW it takes ~13mb pictures. Of the extra 9mb of data, how much of it is actually "more" data than the jpg. In other words, some of it just has to be the same data taking up more space (I imagine) - so how much extra do you get by switching to raw?

lkn4truth

Sony Alpha a77

01/02/2006 10:27:43 AM · #2

The way I understand it (and I'm not the expert) is that the jpeg format is compressed and takes averages of colors to make things easier to compress. For example, imagine every pixel in your picture is a set color with it's own color, intensity, etc. The JPEG sees an area of black and says there are a bunch of "mostly black" pixels here. Let's average out the color and call this whole batch "black." In RAW however it says "this pixel is black...this pixel next to it is blackish gray, this pixel next to it is black again...the pixel next to that is blackish gray with a bit of white, etc). RAW truly captures the color, intensity etc of EVERY pixel while JPEG averages things a bit making JPEG a smaller file since it doesn't have to record EVERY pixel. Now the experts can correct me and you'll really start to understand.

01/02/2006 10:36:08 AM · #3

The short answer: You get a LOT more picture data with RAW.

Yes, JPEG is compressed, but it doesn't end there. With Raw you are using a 12 bit or better file, compared to JPEG's 8 bit structure. With that, you recieve higher exposure latitudes, more gradation of color and grey scales, etc.

Think of RAW as a negative ... with it, you have more control of the final product in post-processing. Thus, we can compare JPEG to slide film, where everything has to be nearly perfect from the camera.

Edit: Note, that with RAW you won't get any sharpening, saturation, or contrast adjustments by your camera. All that will have to be done in Post-process.

Message edited by author 2006-01-02 10:41:33.

01/02/2006 10:36:15 AM · #4

Remember that the 5D captures much more than 8 bits per channel, it's more like 12 bits of usable data. By shooting JPEG, you're therefore squashing this extra data into the 8-bit space, or throwing it away. The 5D has, in fact, about 1 stop of dynamic range beyond where it appears to "clip" in JPEG mode. By exposing to the right and pulling back exposure a little in conversion, you can really exploit the maximum DR the camera is capable of.
The JPEG compression algorithm is very efficient, but do remember that it is "lossy." A JPEG file always loses something, however little, upon compressing/decompressing. This allows the JPEG to be much smaller than the corresponding RAW file, which is compressed, but using a non-lossy algorithm.
Another benefit to RAW is that you worry much less about the minutiae of white balance, since it can be effectively tweaked, or change dramatically for that matter, in conversion with no penalty. Sure, you can effectively change the WB of a JPEG file, warming or cooling it using curves, but your output file's quality will suffer from this if you need to shift significantly.
yet another benefit relates to using the much more powerful processor in your 'pooter to do bayer interpolation, sharpening, CA & vignetting correction, exposure adjustments, etc., all during RAW conversion. Yoiur camera has limited computing power, and must perform all this work within a very short time, so you should expect to be able to get superior results in RAW conversion vs. letting the cam's conversion engine apply the same in-camera processing to every shot.

eslaydog

Canon EOS-5D

01/02/2006 10:54:08 AM · #5

Kirbic,

What is exposing to the right? Does that mean erring by overexposure or staying on the underexposed side?

Thanks,
E

Canon EF 17-40mm f/4.0L USM

01/02/2006 10:59:25 AM · #6

Originally posted by eslaydog:

What is exposing to the right? Does that mean erring by overexposure or staying on the underexposed side?

Exposing to the right would be overexposing.

soup

Canon EOS-30D

01/02/2006 11:01:38 AM · #7

pushing the histogram to the right. slight over exposure. the histogram isn't 100% accurate. there is data in the clipped areas that is recoverable in post processing by sliding the exposure compensation to the left slightly.

Azrifel

Fujifilm X100VI

01/02/2006 11:08:51 AM · #8

Originally posted by fotomann_forever:

Originally posted by eslaydog:
What is exposing to the right? Does that mean erring by overexposure or staying on the underexposed side?

Exposing to the right would be overexposing.

No, Expose to the right article.

Sony 24-70mm F4 Vario-Tessar T* FE OSS

01/02/2006 11:15:29 AM · #9

Originally posted by Azrifel:

Originally posted by fotomann_forever:

Originally posted by eslaydog:
What is exposing to the right? Does that mean erring by overexposure or staying on the underexposed side?

Exposing to the right would be overexposing.

No, Expose to the right article.

OK, maybe I should have rephrased that a little... it would be towards the overexposing side. Not that you want to blow your highlights, but want to move the histogram more to the right.

To quote the article "Now of course when you look at the RAW file in your favourite RAW processing software, like Camera RAW, the image will likely appear to be too light."

That is overexposing.

Message edited by author 2006-01-02 11:17:30.

Bear_Music

Samsung Galaxy S24 Ultra

01/02/2006 11:28:57 AM · #10

Originally posted by lkn4truth:

You're certainly on the right track. JPG is a little like black magic, there's a lot to it below the surface. The way I find useful for visualizing it is this:

When you "map" a bitmapped image, every pixel has to have values appended to it, color values, tonal values, whatnot. When a program "reads" the map it extracts the value for each pixel and displays the whole on the screen. Now, the values take up a lot of space; themap, in its "written form", says pixel 1 = xxx,yyy,zzz,aa,bb, pixel 2 = xxx,yyy,zzz,aa,bb, etc etc. Lot of description going on.

But in a 10 Mp image there are 10 million pixels, and by no means are all of these unique. So in the above case, instead of "completely describing" pixel 2, the map can just refer back to the already descibed pixel 1, essentially telling the program "same as 1" instead of "xxx,yyy,zzz,aa,bb". To follow this to its logical conclusion, imagine shooting an absolutely featureless, pure black wall; the RAW file will be as large as any RAW file of anything, more or less, but a jpg file would be TINY because it only has to map to a single reference pixel.

Differences in degrees of compression in jpg basically are telling the software how much lenience it is allowed, how much slop, in assigning "same as" values. The more you compress, the more similar values will be recorded as identical values, and the more the blocky pixelation you will get.

This is a layman's description, but I find it a simple way of visualizing what's going on.

Robt.

Man_Called_Horse

Canon EOS-300D Rebel

01/02/2006 11:39:46 AM · #11

All about RAW

01/02/2006 12:04:05 PM · #12

Originally posted by Bear_Music:

To follow this to its logical conclusion, imagine shooting an absolutely featureless, pure black wall; the RAW file will be as large as any RAW file of anything, more or less, but a jpg file would be TINY because it only has to map to a single reference pixel.
Robt.

I just tried that ... shot at 1/4000 sec with lens cap on and got an 8 megapixel Jpeg file that is only 187kb compared to a 12MB RAW. Talking about compression, huh?

Message edited by author 2006-01-02 12:07:13.

Nikon AF Zoom-Nikkor 28-80mm f/3.5-5.6

01/02/2006 12:13:24 PM · #13

For those who don't mind a quite a bit of reading, here is a good article on the advantages of RAW. I can't vouch for all the information there, I admit I have not analyzed it great detail but what I've seen is accurate and goes to great depth, for those inclined to dig in. It's also written in a very readable form.

Givemeashot

Nikon D200

01/02/2006 12:25:01 PM · #14

I was told to shot a wedding in "Raw" is this a good thing because i have noticed that my memory card will hold like 50 pics or a little more??

01/02/2006 12:29:32 PM · #15

Originally posted by Givemeashot:

I was told to shot a wedding in "Raw" is this a good thing because i have noticed that my memory card will hold like 50 pics or a little more??

A RAW workflow will definitely increase both the need for memory card space & hard drive space, as well as processing power on the PC. I would not recommend shooting any paid gig in RAW unless you...

1.) Are confident you've developed a viable RAW workflow that will accommodate the number of shots you'll take
2.) Have addressed all the additional hardware and software needs (see number 1)

You can assume that a RAW workflow, with retained RAW files as your archived originals, will at least double space requirements.

Message edited by author 2006-01-02 12:31:01.

eslaydog

Canon EOS-5D

Nikon AF Zoom-Nikkor 28-80mm f/3.5-5.6

01/02/2006 12:34:11 PM · #16

Originally posted by kirbic:

Now that was a great article. I'm sold!

Givemeashot

Nikon D200

01/02/2006 12:35:13 PM · #17

I usually do all my shooting in Fine.... comes out pretty crip to me.

Azrifel

Fujifilm X100VI

01/02/2006 12:45:51 PM · #18

For a wedding with all those details and chances of moirÃ© I'd definitely use raw. And I recommend the Nikon Capture Software for the conversion.

01/02/2006 12:46:47 PM · #19

The luminous Landscape article referenced by Azrifel was what prodded me, in the Fall of 2003, to really think about how to get the most dynamic range out of my 10D. The information presented made perfect sense from a technical perspective, but would it translate to real-world differences? I quickly found that employing the suggested techniqes really did give me improved dynamic range, shadow detail, and shadow noise levels in my JPEG images; I was an instant convert.
In the Fall of 2004 I converted to a RAW workflow, and again experimented, realizing that some additional gains in dynamic range were possible by exposing just past the "clip point," as determined by the camera histogram, and recovering it in RAW conversion. Again, I was pleased to find that it made a perceptible difference on the 10D, maybe 1/2 stop. There was a potential penalty, though, since the 10D only had a luminance histogram. Because of this, there was uncertainty as to where each channel's histogram REALLY was. That fact, added to the uncertainty imposed by exposing outside the (JPEG-based) camera histogram, meant that I'd occasionally irrecoverably blow a channel on a saturated subject if not very careful.
Fast forward again to the Fall of 2005, and with my transition to the 5D, I now have an RGB histogram, which gives much more confidence in "exposing to the right." This histogram is still based on in-camera JPEG conversion using the active camera tone curves, though, so there is still a need to guess how far past the clip point can be recovered. With the 5D it is possible to recover a full stop of "overexposure" though, so it's more forgiving than the 10D.
One note, all Canon users should be aware that the Canon RAW converters, e.g. DPP, do NOT make use of the information beyond the JPEG clip point; they simply throw it away! This is because there is some small chance that this information could be innacurate. Other converters, such as ACR, RSE/RSP, C1, etc., do make good use of this information, however.

01/02/2006 01:13:44 PM · #20

Some ways to get around the JPEG problems:

1. Turn off camera Auto-Contrast settings
2. Turn off camera Auto-Saturation settings
3. Turn off camera Sharpening settings.

Noted you're still going to be working with an 8 bit jpeg in post processing. But, if memory and write times are keeping you from using RAW, this will help a lot.

Message edited by author 2006-01-02 13:18:46.

doctornick

Apple iPhone 13 Pro