Bayer CFA reconstruction

Bayer Color Filter Array, demosaicing examples.

On this page you can find a few examples of the effect that Bayer CFA demosaicing has on image resolution. The evaluations on this page focus mainly on the differences between the RGB channel and Luminance resolutions. Some basic understanding of Bayer CFAs is assumed.

Bi-tonal

Sinusoidal

For this evaluation I used a 100x100 pixel star target with 60 full cycles of modulation from black (0) to white (255). In the examples below, there is one bi-tonal Black and White version with anti-aliased edges (top left) and a sinusoidal version (top right). At the right you can see what a small crop of two bigger versions looks like.

Both versions are also used in a 50% contrast reduced version (bottom left and bottom right), modulating between 63 and 192. These 4 targets were used as a single RGB composite original target which was subjected to several processing steps, as outlined below.

This is the original target

GRGB "Bayerized" target

"Demosaiced" target

"Demosaiced" Luminance

You can see that even the original target will show some aliasing, especially when the edges are sharp (despite a bit of anti-aliasing), or when detail is fine. This is partly due to the interaction of fine detail with the monitor grid, and it is also due to the attempt to display features smaller than 1 pixel. Also, because the Bayerized version only has 25% Red, 25% Blue and 50% Green pixels, the color interpolation can fail on the finest details of the final full color reconstruction. This will normally show as False Color fringing which, in this critical case of a gray background, may stand out clearly. With a noisy multi-colored pattern, it wouldn't show that clearly.
Demosaiced Luminance resolution is slightly (6.4%) reduced (mostly in the diagonal direction), but still very close to the original resolution.

The solution against color fringing is to slightly pre-blur the highest spatial frequencies, allowing some of the missing pixel (luminance and color) information to be also registered in the neighboring pixels. In Digicams this is usually achieved with a so-called Anti-Aliasing filter. It is usually composed of several very thin layers (with different rotation) of a bi-refringent material such as Lithium Niobate, placed just in front of the sensor.
For this evaluation, I have used a simple Gaussian blur (so my procedure can be independently repeated) to mimick the behavior of such an expensive filter. Different types of filter may be more (or less) efficient, and can also be tailored for a particular type of image content.

So, now the same procedure but with a Gaussian blur with 0.2 radius applied to the target first. The rest of the processing is the same as in the first example.

Original target, 0.2 blur

GRGB "Bayerized" target

"Demosaiced" target

Other than some reduction of the aliasing in the original target, it doesn't have much effect on the color fringing.

We'll do the same but with a Gaussian blur with 0.3 radius applied to the target first. The rest of the processing is the same as in the first example.

Original target, 0.3 blur

GRGB "Bayerized" target

"Demosaiced" target

As you can see, most of the color fringing has disappeared, but at the expense of some sharpness. Some of the sharpness can be restored by applying a small radius UnSharp Mask sharpening, as will be shown later.

To get rid of the most significant bits of color fringing, we could desaturate such areas, but here I've done it with a Gaussian blur with 0.4 radius applied to the target first. The rest of the processing is the same as in the first example.

Original target, 0.4 blur

GRGB "Bayerized" target

"Demosaiced" target

This all shows that if a signal is not properly pre-filtered, a Bayer CFA will produce color aliasing or Moiré. The example above uses a very critical test target to simulate a real life scene. There was no lens to prefilter the image, it is therefore a brutal, but realistic test.

Now, we may have reduced aliasing artifacts, but how much resolution did we lose by pre-filtering? Let's compare the before and after results (limited to the two more successful ones).

Original target, 2x zoom (Nearest Neighbor)

"Demosaiced" target, Gaussian blur (radius 0.3), USM

Original target, 2x zoom (Nearest Neighbor)

"Demosaiced" target, Gaussian blur (radius 0.4), USM

If you have left Javascript enabled in your browser, then you can switch here between the same images, superimposed on each other for a more accurate comparison.
It may be interesting to note, that any detail within this target's central blur diameter of 38.2 pixels (76.4 pixels in the zoomed-in version) is smaller than 1 pixel wide. Therefore, such fine detail can only have a partial pixel contribution, together with parts of its neighbor detail. Any detail within this target's diameter of 76.4 pixels (152.8 pixels in the zoomed-in version) will be subject to aliasing, as (amongst others) is witnessed by the tendency for apparent hyperbolic direction changes of the radial patterns.
R, G and B resolution is virtually identical to Luminance resolution with this target, but will decrease for monochromatic colors or low luminance contrast.

So even while the pre-filtered images may look soft near the highest spatial frequencies visible, there is still more detail than is beneficial for an artifact free image. Even the image with a pre-blur of 0.4 radius, still has (reduced contrast) Luminance detail (after Bayer reconstruction) down to a minimum diameter of 38.2 pixel diameter (31.1 on the original target), so 81% of the resolution but without color Moiré. To eliminate all traces of aliasing, a stronger anti-aliasing filter is needed, but that will cost additional resolution.

As a final note, one should realize that the target I used, presents an ideal case of weighted amounts of Luminance signal in all pixels, which makes reconstruction easier. It is possible to fabricate a worst case target with only red and blue with very little luminance variation.

However, there are two reasons why that is not a very useful exercise (but I have done it nevertheless).
First, real world scenes are very unlikely to have fine detail existing of spectrally pure adjacent colors (with equal luminance) from the opposite ends of the spectrum.
Second, the choice of reconstruction algorithm becomes very (!) important, because common reconstruction algorithms favor Luminance information over color information, just like the human eye does. An optimized reconstruction algorithm for red and blue only (sacrificing the visually more important Luminance), would have half the color resolution of a more general reconstruction algorithm. This is due to the under-sampling of those two color bands, but again that all has little bearing on a visually optimal reconstruction, because Luminance resolution of the human eye is 10 times better than color resolution.

Concluding we can say that Luminance resolution only suffers a little from a Bayer CFA reconstruction and that the algorithm chosen is important. However, in order to suppress aliasing it is required to pre-filter the highest spatial frequencies with a low-pass filter. That is irrespective of using a Bayer CFA or not, but the aliasing of a Bayerized image results in color aliasing which looks worse than luminance aliasing. This low-pass filtering will result in reduced contrast, and thus visible resolution, of the finest detail. Low Luminance contrast will therefore lose visible resolution a bit faster, although working with higher bit depth images will allow to restore more fine detail with appropriate sharpening.

Top of page

You can reach me at this e-mail address with questions, remarks and suggestions about this page.

Designed by Bart van der Wolf	Latest revision: 30-Jan-2004
Copyright © 2004, Bart van der Wolf