DxOMark Introduces Lens Testing and Scoring: Cue "My Lens is Better Than Your Lens" Internet Arguments

I have just been informed that the DxOMark site is now doing lens and sensor tests as well. This means you can see how well a lens does on a particular camera. Here's a partial screenshot of how it looks when set to rank at "Peak Score":

While the lens tests has some good bits of information in there, I am not liking how DxO decides to give a "Peak Score" and then rank lenses according to that score. DxO's definition of "Peak Score" is the best possible rating a lens has at a certain focus distance, aperture (and for zooms) zoom setting. The number is derived from a number of tests that include resolution, chromatic aberration, distortion, and so on.

The problem with this approach, is that many are going to just look at the one single number, determine that is a good lens, and buy it without looking at the full report of how the lens performs throughout the aperture and zoom ranges. Not going to be great if you buy a 70-200mm f/2.8 and realise it performs best at 110mm at f/5.6, and not so good at 200mm.

Granted, modern lens design means a lens will not perform so unevenly that it is brilliant at one setting but totally terrible at another, but it does mean one could end up buying a lens that is sub-optimal for the intended purpose. I do urge everyone who is checking out the DxO tests to go beyond the single number of "Peak Score", and as always, please do your own lens testing whenever possible!