A pixel is a pixel - right? - DPChallenge Forums

Nikon AF-S DX Zoom-Nikkor 18-55mm f/3.5-5.6G VR

06/13/2007 11:48:22 PM · #26

whoa guys someone is gonna bust a pocket protector up in here :-)

RainMotorsports

Nikon D60

06/13/2007 11:49:24 PM · #27

Originally posted by fotomann_forever:

whoa guys someone is gonna bust a pocket protector up in here :-)

Specify big endian or little endian and I will start speaking in binary with the 255 visible characters available to me lol.

Canon EF 24-70mm f/2.8L USM

06/13/2007 11:51:30 PM · #28

Yeah, RAW files are compressed without loss, but are still generally pretty much the same size.

This doesn't really translate to pixels though...

One way to look at it is this: RAW files are information from the camera. JPG's are pictures.

This is why RAW files are so hard to view (speaking of compatibility) and edit.

It is true that each pixel only contains 1 color in a RAW, but the information is fully expanded.

In a JPG, information is compressed to make room for the 3 color channels in each pixel. Same number of pixels. But in a RAW, each pixel has one channel, where in JPG, each pixel has three. this is what RAW conversion is all about.

On the other hand, this is comparing an information file with a picture file. To compare more accurately to see where the extra information can go, try TIFF's or DNG or PSD files in 16 bit color spaces. You can work on these images just the same as JPG, but they are much, much, much larger.

To give some oversimplified examples:

RAW = 8 megapixels with 16 bit depth, one channel per pixel
JPG = 8 megapixels with 8 bit depth, 3 channels per pixel = 24 bits per pixel, then compressed
TIFF = 8 megapixels with 16 bit depth, 3 channels per pixel = 48 bits per pixel, with much less compression.

smurfguy

Canon EOS-5D Mark II

06/13/2007 11:53:10 PM · #29

Originally posted by fotomann_forever:

whoa guys someone is gonna bust a pocket protector up in here :-)

LOL - well, say what you want. I'm fairly proud of my JPEG review in 150 words or less. =)

GeneralE

Canon PowerShot S3 IS

Canon 12x IS USM (S3 IS built-in Lens) -- 36-432mm (35mm EQ)

06/14/2007 12:15:05 AM · #30

Originally posted by Brad:

funny about the little pixel.

Even with sharpening, which is none other than enhancing contrast between a light and a dark pixel, when it's all said & done, that darned itty bitty pixel is still but one color or hex value.

When compressed, those hex values are being mathematically re-encoded to take up fewer bytes. The efficiency with which the compression works is dependent on both the total number of colors described in the image, and the number of transitions from one color to another (level of detail).

Take any image, save a copy as a JPEG, then run the USM filter on it and re-save with the Same JPEG settings, and I bet the sharpened file will be bigger.

Brad

Olympus OM-D E-M1

Olympus 40-150mm f/2.8 PRO M.Zuiko

06/14/2007 12:25:02 AM · #31

funny that little pixel.

I can clone it, but it still stays a pixel.
I can compress it, and yet remains faithfully, a pixel.
I can erase it, yet still remains as part of the team.
I can change it and now has to take on a new identity.
I can let it carry more information, or less.

But it still remains, that funny little pixel.

Sigma 15-30mm f/3.5-4.5 EX Aspherical DG DF for Minolta

06/14/2007 01:35:35 AM · #32

MrEd

Sony DSLR-A100

06/14/2007 01:55:18 AM · #33

Originally posted by Brad:

I need to stop coming on here before bed.....

Olympus 40-150mm f/2.8 PRO M.Zuiko

06/14/2007 10:02:32 PM · #34

Originally posted by MrEd:

I need to stop coming on here before bed.....

Amen

Brad

Olympus OM-D E-M1

06/14/2007 10:08:32 PM · #35

Originally posted by GeneralE:

Take any image, save a copy as a JPEG, then run the USM filter on it and re-save with the Same JPEG settings, and I bet the sharpened file will be bigger.

That would be the "assembly" instructions in the file that added on the extra Kb? After all, each pixel is only a square of one shade of color (or lack thereof).

eamurdock

Pentax KP

Pentax FA 77mm f/1.8 Limited AF

06/14/2007 10:29:40 PM · #36

Ok, I was holding off, but here goes: super basic file compression 101:

Imagine instead of 16.7 million colors, you had 2, black and white. You could create an image file where each pixel is coded with 1 bit; 0 = white, 1 = black. So uncompressed, an 8X8 pixel image requires 64 bits of information, whether its:

11111111
11111111
11111111
11111111
00000000
00000000
00000000
00000000

or

10101010
01010101
10101010
01010101
10101010
01010101
10101010
01010101

You could store these files as a 64 bit string with a quick code to tell the computer where the line breaks are.

BUT: we could also represent the first one as:

1(32)0(32)

where '32' is the number of 1's in a row. So if we assign 8 bits to the count portion, suddenly we can store this file with only 18 bits, rather than 64.

BUT: the same representation of the second file would look like

1(1)0(1)1(1)... etc

and would actually require (much) more space to hold than even the uncompressed file.

This is an incredibly simple compression method called "run-length encoding;" it's not very efficient, but it shows how the nature of the data can lead to dramatically different compression ratios. It's also a lossless compression; that is, the original data can be recovered 100% from the compressed version. It's also a terrible method for images, since you'll only get significant compression from large blocks of identical colors (and there are 17 million colors possible in a 24 bit jpeg).

JPEG compression is different in that it actually throws out data based on information about how the human brain interprets images; you can read about it here: //www.prepressure.com/techno/compressionjpeg.htm. But again, the more even an image is, the higher the compression is.

The key thing to remember is that while a pixel is indeed a pixel, the computer need not STORE it as a pixel; it can store it based, at least in part, on its relationship to neighboring pixels. So a picture of a blank (or mostly blank) sky will require FAR less space than a picture of with a lot of texture.

MaryO

Nikon Z8

Nikon NIKKOR Z 28-400mm f/4-8

06/14/2007 10:42:09 PM · #37

Originally posted by Brad:

Originally posted by GeneralE:

Take any image, save a copy as a JPEG, then run the USM filter on it and re-save with the Same JPEG settings, and I bet the sharpened file will be bigger.

That would be the "assembly" instructions in the file that added on the extra Kb? After all, each pixel is only a square of one shade of color (or lack thereof).

An entertaining paper entitled A Pixel Is NOT A Little Square, A Pixel Is NOT A Little Square, A Pixel Is NOT A Little Square!

GeneralE

Canon PowerShot S3 IS

Canon 12x IS USM (S3 IS built-in Lens) -- 36-432mm (35mm EQ)

06/14/2007 10:45:40 PM · #38

The general way that compression works is that it creates a table of abbreviations for patterns which repeat in the image. In the following example, I will consider each pixel to have only 100% of one of the three colors -- in reality, it can be any of about 16.7 million colors (in 8-bit RGB mode).

Here is one row of pixels in an image:

R R B G B R G B R G

This takes 10 characters (3 bytes each) to store on disk. The compression software starts looking for patterns and assigning codes to the various groupings. For example, the two repeated R's can be substituted with the single character %, so now the image is stored as

% B G B R G B R G

and takes 9 characters to store on disk. Next, look at the repeated G B R sequences, and replace those with the single character $, so now you have

% B $ $ G

and takes 5 characters, or half the original amount of space (50% compression). This is also a "lossless" compression scheme -- all of the information will be restored when the file is decompressed. JPEG uses stages of "lossy" compression which actually discard some of the data, so what is restored is only a close approximation of the original.

Images with large areas of smooth/solid color will compress efficiently because there are fewer colors to describe overall, and fewer transitions between colors (detail) -- a file with a row of pixels like

B B B B B B B B B G G R R B B B B B B B B B G G R R

Might compress to something like

9B % 9B %

Let's say it was describing a blue block 10 pixels high with those green and red stripes, you could have something like

({9B % 9B %}x10rows)

These are all illustrative examples, and don't (knowingly) represent any actual compression algorithm ...

So, again ... the efficiency of compression you get compared to the actual pixel dimensions will decrease the more colors and detail present in the image.

If you want reasonable compression which is lossless, save your final composite file as a TIFF with the LZW option enabled -- this uses essentially the same method as the ZIP archive format, and restores 100% of the data upon decompression. You can always then save the TIFF as a JPEG for output or posting or whatever.

Sigma 18-200mm f/3.5-6.3 DC OS for Canon

06/14/2007 11:53:18 PM · #39

Originally posted by Brad:

Brad. Be careful of lumping JPEG definitions with pixel definitions.

A JPEG is a set of instructions for assembling a group of pixels.

The more complex the instructions are, the larger the JPEG becomes.

The greater the range of variance is and the greater the difference is, the more complex the instructions become.

It therefore follows that:
A) The more noise you have, the more complex the instructions become because noise increases the differences between adjacent pixels.

and

B) the sharper the image you have, the more complex the instructions become because increased detail means both increased range of variance throughout the image and increased differences where there is detail.

Mary, what a great paper! It also has some fantastic information to illustrate some aspects of how printers calculate DPI as well as why a Bayer sensor might suffer in image quality compared to a Full Color sensor (ie foveon/hasselblad) and the principles of how bayer sensors get around this.

I saved that one and it's going into my Tutorial pack that I share with other photogs that I meet that are interested around here.

magnus

Canon EOS-30D

06/14/2007 11:58:58 PM · #40

Originally posted by eschelar:

Now you got me curious for more. So If RAW only has one channel, then that must mean the Bayer pattern has not been interpolated in raw data? So that would explain why you typically get better quality from doing the raw to jpeg conversion in the computer rather than in the camera. The camera has very little time to process each image because it needs to be ready to capture the next one. So it probably can't use as fancy an interpolation algorithm as the computer, where taking many seconds to convert one raw image is fine. Hey, yet again, I learn something geeky on DPC!

BTW, just today, Kodak came out with an alternative to the Bayer pattern, which they claim will revolutionize high-ISO photography.

Pentax FA 77mm f/1.8 Limited AF

06/15/2007 12:01:35 AM · #41

The sad thing is, I keep coming back to read this stuff. It's confirmed, I am a geek.

eamurdock

Pentax KP

06/15/2007 12:03:04 AM · #42

Originally posted by eschelar:

A JPEG is a set of instructions for assembling a group of pixels.

Very well said, and deserves a re-read.

Mary - that is an interesting paper. My experience comes in working with raster datasets in geospatial data; for instance digital elevation models (DEM's). A 30m DEM has a 30 meter pixel size (or spacing); for the purposes of display everything within that 'little square' is assumed to be homogeneous; but suppose we're resampling from within it? will a sample from the very edge of a pixel be given the same value as a sample from the far side of said pixel? The answer is that an interpolation scheme is used between the pixel and it's neighbors.

Brad

Olympus OM-D E-M1

Olympus 40-150mm f/2.8 PRO M.Zuiko

06/15/2007 12:08:22 AM · #43

Originally posted by eschelar:

Brad. Be careful of lumping JPEG definitions with pixel definitions.

A JPEG is a set of instructions for assembling a group of pixels.

Actually that's what I replied to GeneralE above.

Darn. Now I forgot what the original the question was.

msdoubletrouble

Canon EOS-20D

Canon EF 17-40mm f/4.0L USM

06/15/2007 12:21:34 AM · #44

only men ;-)

Nikon AF-S DX Zoom-Nikkor 18-55mm f/3.5-5.6G VR

06/15/2007 12:23:45 AM · #45

Originally posted by msdoubletrouble:

only men ;-)

Read every other line and it makes more sense. :-)

RainMotorsports

Nikon D60

06/15/2007 12:23:49 AM · #46

Originally posted by msdoubletrouble:

only men ;-)

And the women of the Joint Picture Experts Group - JPEG

msdoubletrouble

Canon EOS-20D

Canon EF 17-40mm f/4.0L USM

06/15/2007 12:26:18 AM · #47

lol...had to chuckle on the "pocket protector" comment you made... somewhere in all that some partners shoulda said, "just shut up and kiss me..lol"

mary I'm impressed you hung with em..go girl!

Nikon NIKKOR Z 28-400mm f/4-8

06/15/2007 01:32:12 AM · #48

Pshaw, maryO? that girl? She the shazniggit!

She da bom, yo!

something like that anyhow...

eamurdock, I was merely restating a few things earlier posted in point form as I saw relevance to brad's post. I think you knew that though...

Magnus. I checked the info on the new filter layout. I think it won't fly.

Why? it reduces color accuracy, so it won't really help anything... Further, it doesn't actually address the source of the problem that is digital noise, so is little more than a patch. it will effectively boost sensitivity only by decreasing light thrown out.

Of course, it is true that color microfilters on photosites reduce the overall intake of light by 2/3 on the basis that even with a full white pixel, it is only actually gathering 1/3 of the light because much is thrown away by the color filter - by definition. As such, the efficiency of the sensor is reduced a fair bit.

However, the way to counteract this is to increase the amount of light entering the camera, and you effectively counteract this problem with no reduction in the quality of result (if aperture is used to increase light intake), and only minor reduction in quality (if shutter speed is used to increase light intake). To counteract the loss of light to color filters, you need to increase light intake by a factor of 3 (the inverse of 1/3) and this is 1 1/3 stops. not really a huge deal.

The example on DPReview.com is rather exaggerated IMHO as this is likely a result from an older generation sensor (kodak hasn't been keeping up as quickly as Sony and Canon IMHO).

even without noise reduction, shooting in RAW, I don't have anywhere near as much noise in my images at ISO 1250 as the example shown.

ISO 1250. Properly exposed. With noise reduction.

Perhaps they are talking about minicam sensors which are loaded with WAY too many photosites which are fighting over too little light thanks to itty bitty lenses.

this is one instance where it would help, but it will only be effective once they start reducing the number of output pixels. Kind of the reverse of the Foveon method (which de-interpolates, then re-interpolates in order to attain rough equivalent pixel counts to Bayer style image sensors).

For example, they could combine the color information with the adjacent luminance pixel with some clever averaging algorithms to triple the gamut over the standard Bayer method.

What is really important to realize is that the 3x boost in sensitivity would only affect the LUMINANCE pixels, but this might be enough information to apply heavy NR to the CHROMA pixels and recombine effectively at near full information accuracy.

The end result though would be a way to get back noise levels similar to cameras a few years ago which had much lower pixel counts, but with newer NR reduction algo's.

It's a move forward and that's a fact. Just probably not needed if you don't have tooooo many pixels already.

MaryO

Nikon Z8

06/15/2007 08:18:21 AM · #49

Originally posted by eschelar:

Pshaw, maryO? that girl? She the shazniggit!

She da bom, yo!

something like that anyhow...

Gosh, um ... thanks. I think.