Camera Physics Proof

Mr Mars' Camera Resolution Diatribe Page 5 - Real World Proof

The basis of proof - the true raw data from a real camera. Obtained with TrueRaw.

Procedure

In order to verify my theoretical analysis of the Physics of Cameras with respect to the issue of resolution I had perforce, to obtain sets of data from the ADCs of a range of real world cameras showing particular test images. In particular I needed areas of constant colour so that I could subject the variation to statistical analysis. I found a set of Raw images on the internet that provided areas of constant black, grey and white. These provided the raw data that I needed to verify my results, but how could I get at that data?

There are a number of difficulties in getting at the raw data from a Raw file. Firstly, Raw decoders are designed to output images, not text formatted numbers. Secondly, Raw decoders process the data, so what you see is NOT the true raw data. In particular, they demosaic the image which completely alters the data, making it useless for the purposes of statistical analysis.

I solved both these problems by modifying Dave Coffin's DCRAW to create my TrueRaw program. This outputs the true raw data from any camera Raw file in tab delimited, text formatted, decimal numbers. No filtering, no interpolation, no white balance, just the raw data from the sensor. The output text file can be examined in a programmers text editor or read directly into a spreadsheet. You can see a screenshot of the output above.

I located patches of even white, grey and black in the text output file for each camera and copied them across to a spreadsheet. My version of XL only handles a maximum of 30 elements in its statistical analysis so that is the sample size I used. Ideally one would use considerably more but this is enough for my purposes. I am only interested in 1 significant figure since some of my parameters are a little rough.

You can download my spreadsheet containing my proof here.

Analysis of the relative behaviour of the D300s at ISO 200

I shall limit my discussion of the evidence presented in my spreadsheet to the D300s. It's true that its behaviour fits my analysis most closely but the other cameras also fit well enough so that any of them constitute proof. Obviously I'm not going to go step by step through the analysis for each camera, so I have chosen this one for the blackboard treatment. Once you've followed the analysis for this camera it is a simple matter for you to look at the others for yourself.

Spreadsheet screenshot of D300s at ISO 200

Screenshot of my spreadsheet showing statistical analysis of data from the D300s at ISO 200

Here we see some of the data samples for this camera at its base ISO and their concomitant statistical parameters. The linear data is not particularly relevant here except to observe that the white level or maximum recorded intensity is less than 6,000, despite the Manufacturer's ostensible dynamic range of 16,000. Once again, NEVER trust what the Manufacturers say!

The data from the square domain is the stuff that matters.

Note that the calculated gain is 1.0. This proves that ISO 200 is the unity gain setting for this sensor in this camera.
Note that sigma (the standard deviation) averages out at 0.5 for white. This is precisely as expected from my analysis, where square compressed partitions for unity gain Poisson variables are 2x0.5 standard deviations in width. The grey and black results are a little higher for this camera but not for the other cameras.
Note that the maximum intensity in the square domain is only 72. To map this into an 8 bit JPG you will need to multiply the white value by almost 4. Doing so will spread your 68% confidence limits of ±½ to ±2. This is equivalent to a gain of 16 which shows how far even this acclaimed camera is from perfection at base ISO.
Note that in theory the standard deviation for a unity gain Poisson variable is equal to the square root of the mean (µ). The ratio of these two quantities here is very nearly 1.0 for white, demonstrating that this sensor is exhibiting proper Poisson behaviour. The grey and black ratios are a little higher on this camera for some reason. Other cameras have a more constant ratio across intensity levels but it is not 1.0 because their base ISO gain is not 1.0.

Analysis of the absolute behaviour of the D300s at ISO 200

So much for the relative stuff, the next objective is to tie my estimates of the hardware limits of modern cameras down to some real-world absolute values.

The main difficulty here is to obtain the light flux at which the sample images were taken. Since I didn't take them myself and the photographers didn't publish this data it is necessary to deduce it from other shots taken with the same camera at the same ISO and aperture. We can guess from a sunny outdoor shot roughly what the light flux was, allowing for latitude. We can then use the ratio of exposures to calculate the light flux used for the sample images.

Once we know the light flux for the sample images we can use the sensor parameters, the hardware limits, the aperture and exposure to calculate the number of photons hitting a white pixel.

The number of photons should be roughly the same as raw value obtained at the ADC, so we can easily compare my theoretical results with those obtained from real cameras.

Screenshot of my spreadsheet deriving absolute values of the D300s at ISO 200

Here we see how I have applied this approach to calculating the theoretical white values for this camera.

Note that I used two different calibration images to guess the light flux. I admit that the approach is a little rough but an average of the two should give something within the ballpark.
I'm not going to give the formulas here, you can find them in the spreadsheet.
Note that the total loss in stops comes from my page on hardware limits.

What matters here is that the real value for white (that is, its maximum intensity green channel) was 5,173. The theoretical value using my analysis was in the ballpark of 7,088 and 4,536. A very good agreement!

Analysis of the behaviour at high ISO

There's no need to go through a lot of talk, here is the screenshot:

Spreadsheet screenshot of D300s at ISO 3200

Screenshot of my spreadsheet showing statistical analysis of data from the D300s at ISO 3200

Here we see the same parameters as we did earlier, except this time they show the behaviour of the D300s at ISO 3200.

Note that the calculated gain is just over 16. Very much as expected considering the ISO rating is 16 times the previous.
Note that the standard deviation averages out at 2.0 for white. This is precisely as expected for a gain of 16.
Note that the maximum intensity in the square domain is only 75. The same remarks apply as earlier.
Note that the ratio of the standard deviation to the square root of the mean is very constant at 4.0. This is as expected for a gain of 16 and once again demonstrates that this system is exhibiting proper Poisson behaviour

Conclusion

The above results demonstrate a VERY close conformation between those predicted by my theories and those obtained from experiment. I realise that the procedure I followed won't meet the scientific rigour demanded by the most punctilious reader but it's good enough for me, and will stay that way until someone disproves me.

I invite you to study the spreadsheet for yourself and look at each camera in turn. Apart from verifying my analysis you will be able to spot a few interesting things: for example, observe the behaviour of the Pentax Kx: its base ISO gain is only 0.3! Either their sensor is doing the impossible or they are doing a Sony and doing noise filtering at the hardware level. Cross Pentax off your list, they have joined the ranks of the cheaters!

To those who want to perform these same experiments with greater rigour I say: "Be my guest!". Feel free to use TrueRaw and even my spreadsheet if you wish, I will be interested to check your results.

Anyway, now that I have proved that I know what I'm talking about, we can move on to the next stage of our journey and finally bring the talk back to that which was promised, namely: a discussion on the central and critical issue of resolution. I begin the final leg with a little round up of the current scene.