Beyond the MTF

[ Home ] [ Gallery ] [ Features ]

Beyond the MTF

Lens Performance in the Real World

The mind can store an estimated 100 trillion bits of  information--compared with which a computer's mere billions are  virtually amnesiac. -Sharon Begley

The Carl Zeiss G-series lenses are a joy to use and often surprise us in their performance, but the Performance Data Sheet included with each Carl Zeiss lens can inform us about only a portion of the optical characteristics. The MTF responses presented on those data sheets, as discussed in a previous article on this site (Lens Performance), are a laboratory determination of how well a lens transmits a flat, black and white test image. Although the MTF is reproducible and quantifiable, MTFs can mislead us in many ways. The deception of the MTF is due to the multiple types of information omitted from the charts. Some of these omissions are inherent to MTF measurements and others are simply a matter of the incomplete reporting endemic to published MTF data. I'll discuss the latter case first.

MTF Reporting: Dodging and Burning the Data

Although an MTF chart can represent the response of a lens at a wide range of focus distances, the variation of lens performance across this range is not typically reported. Neither Carl Zeiss nor the Photodo web site (www.photodo.com) routinely report MTF performance at a distance other than infinity. Results at infinity, however, can be markedly different than results at the closer focusing distances at which most photographs are taken. This is the case, for example, with the 60mm C f/2.8 Makro Planar for Contax SLRs, an exception in which Carl Zeiss does report an MTF at a close focus distance. The Makro Planar's performance at infinity is fine, but it really doesn't start to shine until used at a focus corresponding to a 1:10 reproduction ratio. Aside from the special case of a macro lens, a 90mm portrait lens, for example, would also be better served by good performance in the near range compared to infinity. Using the internally consistent measurements from Photodo to compare lenses across brands, therefore, must be approached with some caution. In the case of the Carl Zeiss MTF measurements, the Performance Data Sheets sometimes include a verbal indication of a lens's close focus performance, but just as often make no mention of it.

Aside from the general lack of published MTF response at multiple focus distances (reproduction ratios), the response across a complete range of apertures is not commonly reported in published MTFs. Most published data consider response wide open and two or three stops down. This is reasonable since those are common working apertures, but can be misleading in assessment of performance at the smallest apertures. Consider a 45mm f/2 lens that can stop down to f/16. Beyond f/8, most such lenses will suffer visually from image degradation (under critical inspection) due to diffraction of light by the edges of the tiny aperture opening. The degree to which this degradation occurs as you stop down below f/8, however, can be greater than the theoretical diffraction limit. For those who like to shoot images with great depth of field, this small aperture image degradation is an important lens characteristic that is not generally addressed in MTF reporting. Lacking this MTF information, the best way to know if the lens suits your needs in this respect is to take photographs with it.

In addition to a lack of assessment across focus distances and apertures, MTFs mislead in other ways. If you look at a typical MTF chart, the response will degrade towards the edges of the image area. This is due to the facts that an MTF response is measured by imaging a flat test target and that most lenses (even a "Planar") do exhibit some field curvature (the G Hologon 16mm f/8 excepted!). The degradation of the MTF near the image edge, therefore, will be a function of both the actual performance of the optics and the amount that the lens is out-of-focus due to field curvature towards the edge. Most people recognize that objects in real photographs are not flat and so the edge performance must not be too strictly interpreted when reading an MTF chart. The problem is that given the way MTFs are measured, we have no easy way of accurately comparing the edge performance of two lenses unless we correct for the effects of field curvature. Field curvature data are provided in the Carl Zeiss Performance Data Sheets and are inconsistently provided on the Photodo site. However, in neither case is it simple or obvious how to apply a field curvature correction to the MTF measurement based on those data.

A final problem with MTF reporting is not so much an error of omission but of interpretation. This problem arises because it's difficult to look at two MTF curves side by side and fully comprehend the magnitude of difference between them. In fact Carl Zeiss notes in the 7th issue of Camera Lens News that "we often encounter cases where the curves are misinterpreted by enthusiasts quite grossly." In order to quantify the differences among MTFs (in a dubious attempt to prevent misinterpretation), sites such as Photodo compute a score based on performance at a couple of apertures and distances from the image center. This score also weights the performance based on the response at 10, 20, and 40 line pairs/millimeter, giving much more importance to the response at 10 and 20 (lens contrast) than to 40 (lens resolution). This is not a problem found in the Carl Zeiss MTF data sheets since no such scoring is performed, but it is a potentially misleading aspect of other magazine and web site lens tests that rely on one number to provide summary judgment. An example of how the overall score of a lens could be misleading is in the choice of a lens for use in available light photography. The Photodo score gives equal weight to performance in the center of the image and performance away from the center. If you tend to place your subject near the center then this weighting may be inappropriate since you will prefer a lens with the highest center contrast and resolution, allowing for a small sacrifice in performance farther out. Some f/1.4 lenses are optimized this way when used wide open, but they would perform poorly on the Photodo score. Only by studying the MTF charts themselves can a useful comparison be attempted.

Beyond the MTF and Into the Real World

Although an MTF can provide useful information about contrast and resolution when reported and interpreted adequately, there are three important aspects of lens performance about which the MTF is silent: color response, flare resistance, and quality of out-of-focus image. Since lens manufacturers are now using computers to optimize for MTF performance, it is these other lens characteristics that often mark the distinction between a very good lens and a superb lens of modern design. Since no standardized tests are commonly reported to measure these other important lens characteristics, however, some people dismiss them as poor cousins to the impressive tables of numbers that an MTF report contains. They can, however, be quantified, and they provide the basis for some photographer's statements about the look of a lens family and its subjective qualities. And in the case of Carl Zeiss lenses I think that they are qualities that set these lenses apart from many others.

The color response of a lens is not assessed by an MTF because the MTF test target is black and white. Certainly a significant chromatic aberration will show up as a loss of contrast due to a bleeding of boundaries between white and black, but beyond such egregious problems the MTF informs us little about the tonal response of lenses. In fact some MTF assessments are conducted only at a narrow frequency of light, and so even the small amount of information that they might provide about color response is lost. The tonal response can be assessed, however, both by observation and measurement. In the case of Carl Zeiss, the measured tonal response is quite flat across the spectrum. In practice, the lenses produce a wonderful gradation and separation of tones, but how can this observation be verified in comparison to other lenses? Is it just the biased eye of Contax users that creates this impression?

In order to determine whether the impression of excellent tonal gradation is reality or fantasy, I compared a Contax G 45 f/2 Planar with a Leica M 50 f/2 Summicron. I was looking for tonal reproduction and so didn't use flat test charts, a tripod, or an optical bench. I loaded both cameras with fresh film (Kodak 100S) from the same emulsion batch and took photos of identical street scenes with each camera in Palo Alto, California on a sunny day. The shutter speeds and apertures were more than adequate to assure good performance from both lenses. I had the films developed together at the same processor, and I compared the results using a very expensive research grade microscope (a Leica, actually). What I saw surprised me [note 1]. When looking at the details of flowers painted on a street-side mural, the Contax lens exhibited an obviously greater separation of hues (both green from green and red from red) than did the Leica lens. Hues portrayed by the Leica lens were distinct, but they were much more distinct (i.e. different in hue) in the Contax lens. This translated into an overall difference in the look of photographs of identical scenes with the Planar and Summicron. The words to describe that difference in look may seem subjective (e.g. the Leica was analytical and precise while the Contax was expressive and vibrant), but the fundamental reason for such characterizations is observable and could be quantified if someone created a more thorough test method than my microscope observations. Whether the tonal reproduction of the Leica or Contax is preferable is a personal choice, but this lens characteristic may have more to do with our lens preference as photographers than does a table of MTF results that differ only slightly among well-made prime lenses.

A second lens characteristic that is off the MTF charts is resistance to flare. To be perfectly accurate, the MTF does account for flare under the lighting conditions in which the test is conducted, but most of us don't photograph under such well-controlled lighting. In the real world, point sources of light (not the least of which is the sun) can hit the lens from a variety of angles. Some lenses resist internal reflections and veiling flare from these point sources quite well while others that may be almost indistinguishable on an MTF are poor or mediocre at resisting this flare. In my own experience, the Nikkors that I own are mediocre at resisting flare and internal reflections, the Leica M lenses are very good, and the Contax lenses are excellent. In the day of shooting with Leica and Contax described above, the Summicron flared out in one scene where the Contax showed no ill effects. A good, quantifiable test of flare response would be easy to devise and informative for photographers. In the absence of such tests we must rely on experience to make such judgments.

A final lens characteristic that is not measured by MTF testing is the appearance of out-of-focus areas. In the aesthetically sensitive Japanese language this appearance is called bokeh, and it is prized in a lens. This is why Japanese lens manufacturers such as Canon and Nikon have produced portrait lenses that allow defocus control. These lenses have a separate ring that adjusts the degree and quality of out of focus blur. To some this is an absurd or at least irrelevant consideration, but I find it an important aspect of lens performance. I once had a Nikkor AF 50 f/1.4 that produced such a jarring and distracting bokeh that I would never use it at its widest apertures. The Contax G lenses, as we know, are at the other end of the spectrum. They produce a beautiful and soft out-of-focus blur that greatly enhances photographs when used creatively as part of the composition.

As with tonal response and flare resistance, bokeh can easily be quantified by someone who invests the effort. Although such quantification is largely unnecessary since the exact quality of bokeh in a lens is an aesthetic preference, the appearance of an out-of-focus point of light can give us more concrete insight into lens bokeh than a simple examination of photographs. For lenses with a pleasing (to my eye at least) bokeh, an out-of-focus point of light will generally be an evenly illuminated circular blur. For other forms of bokeh the point will be brighter at the edges, brighter in the middle, or will consist of two slightly overlapping images (that may each exhibit center or edge brightness). I haven't had a chance to test this point-of-light phenomenon in the Contax, Leica, or Nikon lenses that I own, and I leave it to the reader to investigate whether it is true of the Contax G lenses.

Conclusion

The human eye and brain are able to observe far more about photographic lens performance than any single lens test yet devised. The lens performance that we see in a photograph is a combination of a variety of lens characteristics. Some of these can be quantified by MTF tests, some can be concretely observed but are not commonly measured on an optical bench, and still others may not have yet been articulated or may result from interactions between a particular lens and film rather than be a property of the optics alone. In order to know a lens we must use it and look with an informed eye at the results. The MTF is a tool that is useful within its constraints, but we should strive to articulate our observations that extend beyond the MTF and allow us to fully assess the real world performance of the lenses that we use. Carl Zeiss lenses excel in real world use and have a character that is different from lenses of other makes. To try to put this into words and demonstrable observations is not only useful for discussion, but may, through the effort, make us better and more fulfilled photographers.

Copyright 2000 Charles E. Dunlap

[Note 1] The sharpness of the lenses was identical. Although I tried to discern a difference, the fast shutter speeds and mid-range apertures allowed both lenses to perform with equal excellence in this regard.