Thursday, September 14, 2017

Does a Mantis Shrimp have a Real Color Space?

The small mantis shrimp (a stomatopod crustacean) has a mean right hook--able to accelerate its “punch” to a speed of 50 mph in a few milliseconds, smashing prey and glass fish tanks. It also has an amazing visual system, with 12 or more different receptor cells covering 300 to 720 nm.


Mantis shrimp from the front.[https://en.wikipedia.org/wiki/Mantis_shrimp]

A fairly recent article by H H Thoen et al. in Science [1] describes two hypotheses for the color vision of a particular mantis shrimp (one of more than 500 known species). The first hypothesis is that the color vision is “like ours” in making opponent-pair comparisons between receptor types that allow good discrimination of wavelength.  The other hypothesis is that the mantis shrimp processes each kind of receptor input (spectral band) separately, a method that would give poor wavelength discrimination but might have other advantages.  It turns out that, despite having many more photoreceptor types than we do, the mantis shrimp has poorer wavelength discrimination. Therefore, Thoen et al conclude that the mantis shrimp has no color space at all, but recognizes reflectances by comparing inputs to each kind of receptor separately.  The idea is that color discrimination (e.g., by humans) is facilitated by a ratio (comparison) between spectral bands at the same point in the visual field, whereas the mantis shrimp performs ratios of inputs at different spatial locations to each spectral band separately, and thereby performs reflectance recognition at 12 spectral bins. The authors claim that the mantis-shrimp spectral sensitivity curves are narrow enough so that within-band ratios derived from them exhibit illuminant invariance (color constancy) and allow the mantis shrimp to accurately recognize a reflectance in 12 bins. But the spectral sensitivity curves of the mantis shrimp, shown in Fig. 1A of the same article, tend to have bandwidths of about 100 nm, similar to our own receptor sensitivities. The poor wavelength discrimination of the mantis shrimp seems to be experimental fact, and that would imply a reduced inter-band comparison.  However, that does not mean color constancy by within-band ratios must be enhanced in this animal. 

Another of these authors’ ideas seems to have more traction. If the bands act separately, the neural processing may be accelerated to match the mantis shrimp’s top-speed lifestyle.  The quickly passing world could be processed by a kind of “push-broom sensor” architecture, whereby the 12 kinds of receptors are arranged in one spatial direction, replicates of the arrangement occupy the perpendicular spatial direction, and motion in the first spatial direction accumulates spatial details in a time-encoded form.  Such a design is common for the push-broom sensors that we use in our remote sensing apparatus based on the same principle.

Yet another idea from Thoen et al. is also worth mention: These authors seem to believe that a true color space requires inter-band comparisons, and that such comparisons impose a processing overhead that may not be acceptable to a simple if strongly aggressive creature such as the mantis shrimp.  This rationale bears comparison with the idea, briefly explored by Mark Fairchild ([2], [3]), that even humans don’t need a color space at all, and that what we call color can be expressed with a small number of one-dimensional scales. By this reasoning, color space is a construct of theory, and not intrinsic to visual information. 

Such discussion will inevitably lead to a philosophical and definitional problem.  What, after all, comprises a color space, versus “not-a-space”? Mark Fairchild required a space to have a metric, but I think that requirement could be waived, as could Thoen’s band-comparison requirement.  To me a space is just a representation that allows important features to be salient.  It’s hard to visualize a structure in a 12-dimensional space (such as that of the mantis shrimp investigated by Thoen, et al.), if that space is represented in conventional rectangular coordinates.  But there’s an alternative picture, in which the 12 coordinate axes are lined up parallel with each other and evenly spaced in a plane. Each spectrum is a 12-component object in the space, and shows up as a point on each of the parallel one-dimensional axes.  The original 12D point is represented as an open polygon, with vertices being the component values along the consecutive axes, and line segments connecting the consecutive vertices. Such a structure allows you to see 12-dimensional structures in two dimensions. For example, all the points on a line in the 12-dimensional rectangular space generate a set of 11 intersection points that characterize the line.  The use of parallel coordinates to represent high-dimensional data was invented by Alfred Inselberg more than 30 years ago [4].  Maybe our champion mantis shrimp is using such a representation to track prey, detect mates, and fool anthropocentric color researchers. 

Representative sample of parallel coordinates [By Yug - Own work, CC BY-SA 4.0, https://commons.wikimedia.org/w/index.php?curid=37631153]

References:

[1]. H H Thoen, M J How, T-H Chiou, and J Marshall. A different form of color vision in mantis shrimp. Science Vol. 343, 24 Jan 2014, 411- 413.

[2] M D Fairchild. Is there really such a thing as color space? Foundations of uni-dimensional appearance spaces. ISCC/IS&T/SID Special Topics Meeting Revisiting Color Spaces, San Jose CA (2011), 21-22.

[3] M D Fairchild and R L Heckaman. Deriving appearance scales. IS&T Color and Imaging Conference 20 (2012), 281-287.

[4] A Inselberg. The plane with parallel coordinates. Visual Computer 1 (4): 69-91 (1985).

Michael H. Brill
Datacolor

Friday, May 19, 2017

Recollections of a Vanishing Art

In this column, Hugh Fairman, whom we know for his deep expertise in color and the paint industry, shares some special recollections…

There was a time, within the span of my participation in the color field, when it was common for colorists to practice visual identification of colorants by their curve shapes. With the advent of computer control of color, the capability to identify colorants by shape has become mostly a lost art. I can remember when persons such as Henry Hemmendinger and Hugh Davidson might be holding quite fascinating conversations on the subject at a wine and cheese hour of an ISCC meeting with people like Jackie Welker, who was an absolute devotee of visual identification of colorants. She would go so far as to prepare a colored specimen with an off-beat color combi- nation in advance of a meeting and bring its spectrophotometric curve to the meeting. Then she would go around the room asking others “What is it?”. Your expected answer was to take the form, “Molybdate orange, phthalo blue and rutile.” If you didn’t have at least one modifier for each color name, you were rated an amateur.

It may be of interest to look back at the kind of thing that was considered in identifying colorants. The item best remembered by me is that of the difference between the curve shapes of the rutile and the anatase crystal forms of titanium dioxide. Rutile has a deep absorption trough in the low wave region between 360 (or lower) and 420 nm. Anatase has none, and its curve is thus level with the rest of the high reflectance curve through all of this region. This was of importance in those days, because anatase had an additional property that it chalked upon exterior exposure so effectively that a hard rain would wash it clean. It was thus the preferred white pigment for outside exposure of straight white colors, while rutile was used indoors and in tinted colors outdoors. Thus, an immediate and definitive distinction identifying each crystal positively was useful. Times have changed. Today vehicles have improved for outdoor endurance, and the same material that is used for white is tinted to colors in the store. Anatase is no longer used. In fact, I am told it is no longer made in the United States. It can, however, still be obtained from China and India.
Another telltale curve anomaly positively identifies phthalo blue and distinguishes it from cobalt blue. Cobalt blue can be made to be identical in color with phthalo blue and it is a lot less expensive, but phthalo blue is excellent for exterior exposure. When it is considered that it is an organic pigment, it is nothing short of phenomenal for exterior exposure properties. Cobalt blue is quite impermanent in outdoor exposures, being highly sensitive to acid rain, under whose influence it bleaches readily. Here then is the reason that we might care to identify each of these curves from each other. Phthalo blue has a pronounced secondary reflectance maximum in the 680 to 720 nm region. Cobalt blue completely lacks this hump. Virtually no other portion of the two curves are different but this region, but identification by this difference is definitive.
Kraft paper is a word in the paper industry for that kind of brown paper we know is often used to make what we call ‘brown paper bags’. Kraft paper is widely used also to make the outside surfaces of corrugated cardboard boxes, and many other uses. Kraft paper has a distinctive signature spectrophotometric curve. It is almost a straight line from low to high on a plot of reflectance on a wavelength scale ascending from left to right.  Human hair and human skin have much the same property, that of the straight-line curve. This is different from colors made with red and yellow oxide whose curves are mildly spectrally selective while the same colors made with red and yellow organics are very spectrally selective. This leaves us with three distinct sets of colorations from straight line to mildly twisting selectivity to strong selectivity, and one can look at the curve of any one of these tans, or beiges, and know exactly what its pigmentation is.
I close with one additional thought. Two of the three scenarios I have presented may only be well detected by spectrophotometers with a spectral range of 360 to 750 nm, or more. Think about that when you are about to deprecate the regions outside of 400 to 700 nm as having little weight in human vision.

Hugh S. Fairman
Editor’s note: When I tried to learn this art in short courses, the instructors tended to emphasize absorption maxima, which could show up as reflectance minima (if you were lucky) or as reflectance points of inflection (for more challenging mixtures of colorants). It was not an easy art to learn! [MHB]