How Can Digicams Work At All?

Mr Mars' Camera Resolution Diatribe Page 4 - How Can They Work At All?

Well I guess it works... sort of...

How Well Do They Actually Work?

Obviously digital cameras DO work! I myself am addicted to them, otherwise I wouldn't be writing this. But do they work as well as we seem to think? The short answer is NO! The long answer is a bit more complicated...

Compacts: With a few exceptions at the high end, today's compacts produce rubbish images. Even their owners will apologise before showing their pics saying: ".. it's only an 8MP." or "... sorry it's not an SLR." or some such. The fact is that the Manufacturers have dumbed down this segment of the market. They are no longer trying to make compacts that take good pics, (that went out years ago), they are trying to sell cheap cameras to the ignorant. They deliberately use too-small sensors with too many pixels and second rate lenses. These cameras are nowhere near the theoretical standard they ought to be. In expert hands they can take a half decent shot in full sun but that's about it. Manufacturers and Retailers want it this way, so that when people complain about the image quality they can say: "Oh, these are just beginners' cameras, what you need is an SLR." and direct them to the most profitable segment of the market. This is disingenuous as optimally designed compacts are capable of the same image quality in real-world photography as SLRs. Too bad they don't make them!

SLRs: As I have found to my cost, the IQ from SLRs varies greatly from manufacturer to manufacturer. Since I am talking about the theoretical aspects of cameras here I will restrict my remarks to the best of SLRs. I have an APS-C CCD SLR which I use often and I love it. With the right lens it takes great shots in almost any situation. However, it does not take great shots in standard house light at night. It cannot break the laws of physics and even with an f/2.8 lens it runs into the wall at this point. However, when you consider that I should need a theoretical ambient light level of full sun to achieve perfection¹ that's a pretty impressive achievement!

Yes, my SLR simply can't take a 1/60 sec well-exposed shot in house light, but it will take such a shot in office light at ISO 1600. The quality of such a shot is very acceptable, however, there IS noticeable noise. This is Poisson noise. As I decrease the ISO rating the noise decreases until at ISO 100 it can't be seen at all. But at ISO 100 I can't shoot inside!² This is in line with my previous theoretical limitations. Ye cannae change the laws of Physics!

You can argue that you don't mind a bit of noise in the shadows, or that you can't see any difference between ISO 100 and 200, or that ISO 400 is great on your camera, and I am right with you! But the fact remains, to achieve perfection, you need full sun. Whether you actually need perfection is another matter entirely.

This leads to the most important question of all from the point of view of the Manufacturers: "What level of quality does the market consider acceptable?". And it is here that we find the real answer to the mystery of how modern cameras appear to defy the laws of Physics. The fact is, that the camera you own is WAY short of the specifications of theoretical perfection that I have laid down in the preceding pages. They get away with it by feeding you a raft of lies and half-truths, by covering up the flaws with some pretty nifty image processing and most of all: by not giving you perfection to compare your camera to, so that you are not aware of how far from that mark it is. Yes, the greatest asset to modern photography is human tolerance of technical imperfection and the determination to fill in flaws with one's imagination!

So How Is It That I Can Take Shots Indoors?

My specification for perfection requires an afferent 256 million photons per pixel per exposure, yet an APS SLR indoors in suburban lighting gets only around 160,000, (1,600 times less). Bear in mind that this is the number of photons that are on course to hit that pixel before they actually hit the lens. The number that actually strike the photo-detector is FAR less than this: For an APS SLR with an f/2.8 lens there is an 8 stop loss, so only around 600 photons are actually counted in the brightest pixels. How can a dynamic range of 600 in the linear domain give a acceptable 14 bit image?

There are two issues here with their corresponding two techniques that are used to give the illusion of achieving the impossible:

Dynamic Range: First of all the Manufacturer's stated dynamic range of 14 bits is nothing of the sort. A look at the true raw output will quickly convince you that the maximum dynamic level is around 6,000, NOT the 16,000 you might have imagined. To convert the 600 count to 6,000 they simply amplify it in the analog domain before the count is read out with the ADC. This is known as the Gain Stage. Of course you can't get something for nothing, and though the maximum level is greater the resolution is no different. A gain of 10 simply gives you a parade of values in steps of 10 with nothing in between. You don't see this though because small random analog errors and noises blur these partitions, giving the appearance of fine resolution. This is illusory. There is no useful dynamic detail there, only noise.
Poisson Aliasing: It is Poisson Aliasing that demands 1 million photons at the hot pixel to achieve ±1 SD partition confidence limits. By reducing the dynamic range by 1,600 you have GREATLY increased the aliasing. This is taken care off by Noise Filtering which blurs fine detail to obtain smooth colour. The extent of this is truly horrendous. It is a testament to the imagination of humans that we put up with it. To give you a clear idea of the extent of the trouble: perfection is 1,000 levels in the square domain with ±1 SD partition confidence limits, the 625 levels in the living room example give just 25 levels in the square domain with the same confidence limits. In the final JPG image the clean 25 levels are expanded to 256, smearing dynamic detail all over the place along the way.

The Gain Stage

What happens in the silicon hardware of your camera is that:

each pixel stores a charge proportional to the number of photons that strike it,
when the exposure is complete this bucket of excited electrons is passed to the analog processing stage,
this signal is amplified if necessary (this is the Gain Stage),
the processed charge is then converted to an integer using an ADC, generally 12 or 14 bits of resolution,
the resultant numbers from each pixel are assembled into a Raw format,
if only Raw format is desired then it is stored along with all the EXIF camera parameters, proprietary and public, for later processing,
if JPG is required then demosaicing is performed to convert the image to raster format,
the raster is then further processed to filter noise, adjust white balance, sharpen it up etc
finally the image is gamma corrected, compressed and stored.

When you attempt to take a shot that you don't have enough photons for, say, indoors at night, you turn up the magic ISO control and this turns up the gain. Voila! It appears that you have enough photons even when you don't. However there is no such thing as a free lunch, and what happens is that you have simply reduced the size of your category partitions, so that they no longer give you the 68% confidence that your pixel value is not aliased. Intensity levels become poorly discriminated, aliasing becomes serious and you get noise. The more you turn up the magic gain control the worse the noise. You can try to remove it or you can live with it but you can't get the image you would if you had more photons.

In the expanded square domain the ±1 SD confidence width basically becomes the square root of the gain. For example a gain of 25 changes the ±1 partition confidence width to a width of 5 partitions thereby giving a 68% chance that your value will be aliased by no more than ±5. Only a minor portion of that 68% will not be aliased, so it is very unlikely that any given pixel will be accurate.

To illustrate the above example I have calculated the theoretical distributions for an 8 bit system with a gain of 25 using square compression and here are the results:

Partition bounds 8 bit, gain 50, top end

Partition bounds 8 bit, gain 50, low end

Partition discrimination for 8 bit dynamic range, square compression, gain of 25

Note that I have not shown successive partitions, I assure you they overlap greatly. What I have done is to show the distributions for means 5 partitions apart at max, mid and min intensity levels. Such a setup gives the same square compression confidence limits shown on the previous page. Or it does if you ignore the intervening partitions. This confirms that the width to maintain the 1 SD confidence limit is multiplied by the square root of the gain, in this case 5.

Also note that there is considerable overlap in the bottom 3 levels shown, just as there is in a non-gained setup.

Remember that there are actually 4 other curves between each of these shown. Where once 2 SDs spanned 1 partition it now spans 5. There are 2½ partitions either way within the ±1 SD limit. Since each partition represents 1 unit in an 8 bit intensity scale we can say that the theoretical SD for this setup is 2.5.

Why Are The Darks Noisier Than The Lights?

As for the greater figure for the low end, (double the mid), this is because of overlap in the bottom 4 levels which normally isn't significant because it is only 4 levels out of 256, but thanks to turning the gain up to 25 it is now 20 out of 256 levels. Significant!

Here at last, as promised, is the real reason for noise in the shadows. Turning up the Gain exacerbates the Poisson Noise. Higher intensities have their Poisson Aliasing problems washed away by square or gamma compression. The lowest intensities are affected by Poisson Noise at unity gain but there are so few you don't notice. It is only when you combine high Gain with low intensities that you get a problem, and that is ISO 1600+ in the shadows.

Noise Reduction & Sharpening

Noise Reduction is one of the two great tools used by the industry to cover up the enormous disparity between perfection and what actually comes out of the sensor. The other is Sharpening. It is especially noticeable with compacts, since they are so poorly designed. When you see a compact offering ISO 1600 and greater you should realise this is simply pushing the tool beyond what it can really do. The unprocessed image will be appalling and it will be cleaned up with heavy NR processing in camera.

To be fair, 2D noise filtering technology has come a long way in recent years and given the right sort of image the results can be startling! In particular, areas without fine detail come up a treat, such as buildings and motor cars. However, ye cannae break the laws of maths, and images containing fine detail such as human hair and skin pores will be smudged. Don't listen to manufacturers who talk about "new advances in noise reduction", you can't put back what you didn't have to start with! Just ask Harry Nyquist.

Even SLR cameras have NR applied in camera, although not if you shoot RAW, which is just one reason why you should always shoot RAW. Strangely, Sony are the exception here and you are stuck with their damn NR even if you do shoot RAW. This is one good reason not to use Sony.

NR is a useful tool in small amounts, as is sharpening, which makes a wonderful difference to the eye so long as you don't over do it. Exactly how much is just right varies from shot to shot and from person to person. It is best to keep it to a minimum in order to preserve detail. Most experienced photographers will tell you that they prefer detail with noise over a smooth blur.

The Manufacturers Lie

It so happens that all the major manufacturers of cameras today are Japanese, and there is as much chance of them admitting to imperfections and limitations in their products as there is of you walking on the moon. These people mislead and misrepresent their products on a daily basis and the odd outright lie is par for the course. This certainly doesn't mean that everything they say is untrue but you should take their utterances with a healthy sprinkling of salt. Even worse is true of most retail salesmen who have no idea what they are talking about and possess the morality of a jackal.

Cameras for sale! Do they know or care? I think not.

One of the classic lies used by Manufacturers is that "Thanks to advances in sensor technology and our proprietary noise filtering software, we have been able to increase the resolution with no loss in image quality!" VERY occasionally there is a real improvement in sensor efficiency, the great majority of the time however, this is a flat out lie!

Here is another: "Utilising our unique next generation technology we can now offer class leading ISO 12,600 on our latest compact!" Well, yes, they may offer that speed, but when you see the image quality you realise that you can't use it, the IQ is unacceptable at any size.

I can't let this section finish without making a special mention of Fuji: It is my assessment that Fuji make a living in the hyper-competitive world of digicams by making cheap, 2nd rate products, then use some gimmick to misrepresent them as class-leading and charge accordingly, but leaving a margin so that their price seems a bargain. They love to talk up their "Super CCD" technology which uses a diamond pattern instead of a square and has piss-farting little extra sensors at the corners. Supposedly this gives amazing improvements in sensitivity... This is a LIE. As I have shown in the course of this analysis the only thing that matters is pixel pitch. Little corner sensors aren't going to do Jack Shit and the wasted space means that the end result is actually LESS sensitive not more. If you need proof, just use one of their crap products and look at the horrible NR-smeared results for yourself. Their latest trick is their "EXR" technology, which is just pixel binning in camera. You can do pixel binning on your computer simply by resizing in your favourite editor. "Get Fucked Fuji!" You are the snake oil merchants of the digital camera world.

Designing To Minimise Poisson Aliasing

You don't need to understand Poisson Aliasing to build or operate a camera. Thanks to the Lovely Fluke the necessary square compression is achieved when gamma is applied. For this reason it has been overlooked, but this doesn't mean its effects are not apparent. You can't remove the laws of Physics simply by ignoring them and Poisson Aliasing has its fingerprints all over every image in the world, like it or not.

Designing a camera to minimise the effects of the Poisson monster means working in the square domain and keeping the dynamic range in that domain as close to 1,000 as possible. 1,000 levels in the square domain equates to 1 million in the linear domain so you need pixels with a greater full well capacity than are currently available, and this is the area that manufacturers ought to concentrate on. In full sun, at base ISO, we often have more speed than we need, so here is an opportunity to get a lot more photons into the image. The perfection I have outlined in this diatribe is achievable in full sun so let's work towards it!

Manufacturers should also rethink their approach to Compacts. They should apply the Martian Categorisation Scheme and put a constant f/1.4 zoom lens on them. The image quality would then equal and perhaps surpass, that of the best SLRs.

To Sum Up

Due to the quantum nature of light, Poisson aliasing and human visual perception we need a minimum of 1 million photons per pixel, at the sensor, per exposure, at the brightest part of an image to achieve a photo that looks perfect to a human being at full zoom without the use of image processing. These are innate restrictions caused by the laws of physics and the biology of humans; they cannot be changed.

For a large sensor camera such as an SLR, due to aperture constraints and inefficiencies inherent in camera hardware most of this light is lost. Under normal photography less than 1/256th of afferent light at the lens is actually used to make the image. The maximum theoretical improvement that could be obtained through technological advance here is 2 stops. Smaller sensor devices don't have the same inherent aperture constraints and can thus be more efficient.

Thanks to the equation for Depth of Field, a well designed Compact will give the same image quality as a well designed SLR in normal³ photography. Such compacts are not currently made.

Combining the effect of hardware inefficiencies, the physics of light and the characteristic of human perception gives us a theoretical minimum requirement for an APS camera of 256 million photons per pixel, per exposure, at the brightest part of the image to completely satisfy a human being. This equates to full sun.

Shooting a real-world scene in less than full sun will result in an image that becomes progressively more noisy as the light level drops. This does not mean that these images are not useful, far from it! People are tolerant of a moderate amount of noise and small amounts will not even be noticed.

Image processing techniques such as noise filtering and edge sharpening can significantly improve the look of an image as perceived by a human. Information is lost by this process however and it can only be used so far.

Manufacturers use every trick in the book to get you to buy their products and they misrepresent and lie on a daily basis. They cannot be trusted and their words must always be read with skepticism. Most, (but not all), of their claimed technological advances are useless. Often they use heavy image processing to cover up the hideous short-comings of their poorly designed, 2nd rate compacts. Don't buy a camera that can't shoot true RAW.

By following my guidelines for minimising Poisson Aliasing, Manufacturers have the opportunity to make a real improvement in image quality, both in large sensor and small sensor cameras.

A well designed digicam using the best technology can record real-world photos of wonderful quality, although normal house light shots will always be noisy. The trick to getting it right is to use the correct resolution/sensor size trade off and the optimal aperture lens. I have set these out in the Martian Categorisation Scheme which comes later.

Here is the spreadsheet for this page's calculations.

Where's The Proof?

Well it's a fair call and I know that many hard-headed pests out there lack the imagination to accept anything that appears to go against accepted wisdom, even when that wisdom is a lie. For these people and also for the merely curious I have gone to considerable trouble to prove what I have said, using real output from real cameras. Check it out on the stage of our journey.

See previous section
At least, not without flash.
That is: where the DoF is around 1/3rd of the scene height.