Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 117
The Photoreceptor Mosaic
as an Image Sampling Device
JOH N I. YEI,I,OTT
The fact that vision begins with a spatial sampling of the retinal image
by discrete photoreceptors was first recognized in the 1850s, when Heinrich
Muller combined psychophysics and anatomy to show that the light-sensitive
elements of the retina must be the rods and cones (Brindley, 1970~. That
recognition touched off a debate about the perceptual implications of "mo-
saic vision," in particular its implications for spatial resolution. Apparently
Muller himself initially argued that for two points to be seen as separate it
was sufficient for them simply to stimulate two different cones (L.N. Thibos,
personal communication, 1988~. But his rival, Bergmann (1858), pointed
out that in that case a continuous line would be indistinguishable from a
row of dots! ~ Bergmann, and to Helmholtz (1860), it was obvious that for
two points to be resolved there must be a third unstimulated cone between
two stimulated ones. And on that basis Helmholtz was satisfied that the
limits of visual acuity were consistent with the anatomical dimensions of the
receptor mosaic: at his own resolution limit for gratings, 60 cycles/degree,
alternate light and dark bars would stimulate alternate rows of cones.
Once this point was settled, the visual consequences of receptor sam-
pling seem to have attracted surprisingly little attention for quite a long
time. But in this century the general topic of signal sampling has as-
sumed great practical importance, and mathematicians and engineers have
developed sophisticated tools for analyzing the information transmission
properties of arbitrary sampling schemes. Recently visual scientists have
begun to apply these tools to the photoreceptor mosaic to analyze its prop-
erties as an image sampling device. This paper reviews some of what has
been learned from this analysis.
For an engineer setting out to build an image sampling device, the
natural starting point is Shannon's (1949) sampling theorem. I will start by
117
OCR for page 118
118
JOHN I. YELLOIT
reviewing the assumptions and consequences of that theorem and then will
describe the image sampling parameters of the cones and point out how they
violate Shannon's assumptions. Work by anatomists and psychophysicists
over the past decade has shown that the cone mosaic is quite unlike the
kind of sampling device envisioned by Shannon from his perspective its
construction seems much too sloppy. I think this suggests that the receptor
mosaic is not really designed with Shannon's theorem in mind (so to speak),
but instead follows a different blueprint based on some other approach to
recovering continuous images from discrete samples. Part of this paper
deals with image recovery algorithms that do not require the kind of
precise architecture assumed by Shannon's theorem and therefore seem
better suited to the kind of image sampling found in the primate retina.
SHANNON'S THEOREM
We are concerned with the sampling of two-dimensional signals-
retinal images but to illustrate the basic ideas of sampling theory it is
convenient to begin with one-dimensional signals, such as temporal wave-
forms (see Figure 1~. Shannon (1949) originally stated his theorem for
such signals: "If a function fate contains no frequencies higher than W
cycles/sec it is completely determined by giving its ordinates at a series of
points spaced 1/2W seconds apart."
In other words, no information is lost by discrete sampling, provided
we sample periodically (as in Figure 1A) and the highest frequency in the
signal is at most one-half the sampling rate. That frequency cutoff, half
the sampling rate, is called the Nyquist limit of the sampling array. The
Nyquist limit is a key concept in a moment we will see what it means to
violate it.
"Completely determined" in the theorem means that the continuous
input signal can be exactly reconstructed from its sample values. Shannon
showed that this can be achieved by convolving the sample values with a
function of the form sin 2 or Wt~ ~ Wt the so-called sine function. I will
refer to this process as sine interpolation. The sine function arises here
because it produces perfect low-pass filtering in the frequency domain: sine
interpolation kills all frequencies above the Nyquist limit and passes all
lower frequencies intact. So another way to describe the reconstruction
process is to say that the input signal is recovered by low-pass filtering of
the sample values.
~ visualize the process of image reconstruction by sine interpolation,
one can imagine that each sample point creates a rippling point-spread
function whose value equals the signal at that point and zero at all the
other sample points, as illustrated in Figure 1B. If we add up all these
point-spread functions, we get the original signal.
OCR for page 119
c
B
PHOTORECEPTOR MOSAIC AS AN WAGE SAPLING DEVICE 119
R
- it 1.l'~:'i'l!'''~
_~.
~~ ~°
L: :.t~tt/:
>~`
\'4 ~0
.~: ~ ~
FIGURE 1 Shannon's sampling theorem in one dimension. NOTE: (A) Continuous signal
sampled at equally spaced points. (B) Sinc interpolation functions around the sample
points. (For clarity only one function is shown as a continuous curve.) (C3 Postsampling
reconstn~ction of a sinusoid at half the Nyquist frequency. The input signal is shown as a
continuous curve; its reconstruction is shown as a dotted curve. (D) Aliasing: the input
signal is a sinusoid whose frequency is 1.5 times the Nyquist limit of the sampling array.
Sinc interpolation yields a reconstruction (dotted curve) whose frequency is one-half the
Nyquist limit.
Shannon's theorem deals strictly with bandlimited signals and infinite
sampling arrays. In the real world, of course, we never encounter either
one. (In principle, a band-limited signal must have infinite duration.) So it
is natural to wonder how well sine interpolation works for finite-duration
signals sampled by finite sets of points. Figure 1C shows a sinusoidal signal
at half the Nyquist frequency that has been sampled at 11 points and
reconstructed by sine interpolation. The results here are essentially perfect,
OCR for page 120
120
JOHN I. YEl l OlT
and that is quite characteristic: Shannon's reconstruction algorithm works
very well on a local scale for signals below the Nyquist limit.
Figure ID shows what happens when the input signal exceeds the
Nyquist limit. Here a sinusoid at 1.5 times the Nyquist frequency has been
sampled and reconstructed by low-pass filtering. The result is a sinusoid
whose frequency is half the Nyquist limit. This is an example of aliasing
distortion: an input frequency above the Nyquist limit creates a spurious
low frequency in the postsampling reconstruction. Here it is obvious from
the figure why this occurs: the supra-Nyquist sinusoid and its sub-Nyquist
alias create exactly the same sample values, so low-pass filtering of those
values must produce the same result in both cases. The same would be
true for any operation we performed on the samples.
Shannon's theorem generalizes readily to two dimensions in other
words, to image sampling (e.g., Goodman, 1968~. There it assumes that
the sampling array is a perfect lattice and tells us that any image can
be reconstructed from its sample values provided it contains no spatial
frequencies higher than a cutoff a Nyquist limit that depend on the
spacing of the sample points. Roughly speaking, that limit again is half
the sampling rate. And here again the input signal is reconstructed by
low-pass filtering. In this case that filtering is accomplished by convolving
the sample values with a two-dimensional analog of the sine function a
Mexican sombrero with multiple ripples in its brim. And just as in one
dimension, if we sample a spatial frequency higher than the Nyquist limit,
we get sample values identical to those-of some lower-frequency alias, when
we reconstruct by low-pass filtering it is the alias that survives.
The only new wrinkle in two dimensions is that the alias generally has a
different orientation than the input as well as a different frequency. Figure
2 illustrates this by showing a grating seen through a hexagonal lattice of
holes whose Nyquist limit is less than the grating frequency. A point to
note for future reference is the perfect regularity of the alias pattern.
Our concern is with violations of Shannon's assumptions by the cone
mosaic. 1b understand the effects of those violations, one needs a general
idea of the proof of the theorem, especially of the role played in that proof
by the Fourier spectrum of the sampling array. Figure 3 illustrates the key
points. Panel A represents a two-dimensional array of sample points a
forest of delta functions. Image sampling is a matter of multiplying the
image times such an array, and in the spectral domain that means we are
convolving the spectrum of the image (Figure 3B) with that of the array.
When the sampling array is a perfect lattice, its spectrum is also a lattice,
and the spacing of the spectral lattice points is inversely proportional to the
spacing of the sample points (i.e., it is proportional to the sampling rate).
Convolution creates multiple replicas of the image spectrum, one centered
at each point of the array spectrum (Figure 3C). When the sampling rate
OCR for page 121
PHOTORECEPTOR MOSAIC AS AN EDGE SAPLING DEICE 121
FIGURE 2 Aliasing in two dimensions. NOTE: A hexagonal array (seen alone on the
left) samples a grating (seen alone on the right) whose frequency exceeds its Nyquist limit.
After sampling (middle) the grating appears to have a lower frequency and a different
orientation. (gibe figure is panted in reverse contrast.)
is sufficiently high these replicas do not overlap, and the one centered on
zero frequency is a perfect copy of the image spectrum. So we can isolate
it by low-pass filtering and recover the image by Fourier inversion. But if
the sampling rate is too low, the replicas do overlap, and no clean copy can
be isolated by spatial filtering.
So the critical assumption of Shannon's theorem is really the spatial
regularity of the sampling arrays, equivalently, the lattice-like nature of
its spectrum. It is this regularity that creates the discrete spectral replicas
needed for perfect image reconstruction by low-pass filtering.
SAMPLING PARAMETERS OF THE FOVEAL CONE MOSAIC
Now we turn to the parameters of retinal image sampling by the cones.
I will deal first with the fovea and then with the rest of the retina. Our
concerns are (1) the spatial frequency bandwidth of the image, (2) the
Nyquist limit implied by cone density, and (3) the spatial regularity of the
mosaic.
OCR for page 122
122
JOHN I. YELLOIT
A
-
G(J`. fy)
it- A--''--
,~ -1 >~ , >_.
~ ,
~ ~ ,y
\ ,'1)
/ /~-
\~--' ~ I ~ ~ 'A
7-~--T
,
FIGURE 3 Key concepts in the proof of Shannon's theorem for two-dimensional sampling.
NOTE: (A) Two-dimensional lattice of sample points represented as delta functions. Here
the lattice is rectangular, with By spacings MY. The Fourier spectrum of this lattice is
another lattice with reciprocal spacings 1/X, 1/Y. (B) G(f=, fly) is the spectrum of some
two-dimensional signal grays. (C) The spectrum of g(~;y) after sampling consists of multiple
replicas of G. one centered at each point in the spectrum of the sampling array. SOURCE:
Adapted from Goodman (1968~.
OCR for page 123
PHOTORECEPTOR MOSAIC AS AN IMAGE SAMPLING DEVICE
123
For the fovea, Campbell and Gubisch's (1966) line-spread measure-
ments indicate that the spatial bandwidth of normal retinal images is on
the order of 60 cycles/degree.
Osterberg's (1935) classic measurements of cone density in a single
human eye implied a Nyquist limit for the foveal center of almost exactly
60 cycles/degree. Recently, Curcio and her colleagues examined several
eyes and found foveal Nyquist limits ranging from 50 cycles all the way up
to 85 (Curcio et al., 1987~! But their average value is 65 cycles, so we can
still say that the Nyquist limit of the foveal cones is on the order of 60
cycles/degree.
Now if the foveal cones formed a perfect spatial lattice, Shannon's
theorem would allow us to say that in principle no information is lost there
by receptor sampling. Figure 4 shows that foveal cones in the primate retina
can achieve a high degree of spatial regularity over distances on the order
of one-tenth of a degree. The top panel shows the center-point positions
of roughly 100 cones in the center of the fovea of a macaque monkey,
carefully measured by Hirsch and Hylton (1984~. The cones were sectioned
at the inner-segment level, near their optical entrance aperture (Miller and
Bernard, 1983), so the picture accurately represents the effective sampling
regularity of the mosaic.
The bottom panel of Figure 4 shows the Fourier spectrum of this array
of cones (Ahumada and Yellott, 1985~. We see that the spectrum is quite
lattice-like out to very high frequencies. So over small regions containing
a 100 or so cones, the lattice assumption of Shannon's theorem seems to
be well satisfied by the foveal receptor mosaic.
But this high degree o~ sparest Tory us nor ma~n~a~neu over larger
distances. Figure 5 (panels C and D) shows the aliasing patterns created
when a 1-de" section of monkey foveal cones is made to sample gratings at
frequencies above its nominal Nyquist limit (Williams, 1985~. The section
itself (shown in Figure SA) is the same one from which Hirsch and Hylton
selected the 0.1-de" patch of cones whose positions are shown in Figure
4. We see that there is very conspicuous aliasing here, but it does not
have the long-range periodic character that would be produced by a perfect
lattice (cf. Figure 2~. And when the normal 60-cycle bandwidth of the eye
is bypassed by interferometry, as Williams (1985) has done, the aliasing
patterns one perceives have the same ragged quality as those we see here.
So both anatomy and psychophysics indicate that spatial regularity in the
human foveal cone mosaic is preserved only over distances on the order of
one-tenth of a degree.
In the fovea then the spatial bandwidth of the retinal image matches
the Nyquist limit implied by overall cone density, and the cone mosaic
is locally regular enough to create narrowband aliasing of super-Nyquist
. . . ~
OCR for page 124
124
JOHN I. YELLOIT
.
·
.
.
. ~ .
At.
By.:
.6 -
:~
As.
:~
.
.
:~;
:.
~i.s`:
I.
i :~
At.
-
~o ~
. ~
UP
i:
I.
s ..
6:
Hirsch & Hylton Data
Fourier Amplitude Transform
· ·
.
Hirsch & Hylton Data
Cone Positions
is. A '
-I Us
-
_
-
:.
id..
. :
.
..
as,
.>
*s
.
FIGURE 4 Spatial regularity of the central foveal receptor lattice. NOTE: Top: dots
mark the positions (centerpoints) of cone inner segments in the central fovea of a monkey
(Macaca fasiculans) as measured lay Hirsch and Hylton (1984~. The mean interpoint distance
is 3 ,um. Section width corresponds to about 0.1 deg on the human retina. Bottom: Fourier
amplitude spectrum of the point array on the left.
OCR for page 125
PHOTORECEPTOR MOSAIC AS AN IM4GE SAMPLING DEVICE
B
125
c
tD'
FIGURE 5 Gratings sampled By the cones in a monkey fovea. NOI,E: (A) Cone inner-
segment positions (at the external limiting membrane) in the fovea of Macaca fasiculans.
Section width corresponds to 1 deg on the human retina. (B) Foveal cones sampling at 40
pycles/degree (i.e., sub-Nyquist) square wave. (C) 80 pycles/degree. (D) 110 pyclesldegree.
SOURCE: Adapted from Williams (1985~.
gratings. But the mosaic overall certainly does not have the kind of perfect
lattice structure assumed by Shannon's theorem.
IMAGE RECOVERY FROM IRREGULAR SAMPLES:
YEN'S THEOREM
Does this lack of spatial regularity imply that foveal receptor sampling
necessarily loses information? In principle, the answer is no—provided the
actual positions of the sample points are known. Perhaps the best-known
mathematical result in this connection is Yen's theorem (1956, Theorem A,
which provides an explicit reconstruction algorithm for the case in which
any finite number of sample points have been arbitrarily displaced away
from their lattice positions. Yen's theorem assumes that the input signal
contains no frequencies higher than the nominal Nyquist limit (i.e., the
Nyquist limit implied by the average sampling rate).
OCR for page 126
126
R
B
D
JOHN L YELLOIT
~ ~ C
t! ~
Hi. ./
I I I I ~ 1 1 1 1 1~
11 ~ '1
i: "A i.,,,,,.t , ~
r
_
ttttit
9754 t
let I ' ~
,It I ', t I
~. 1,
t I t '. ' I I ~ I ~ , t I
Jew
tt~tt I
flit i.
1
1` t I t 11 , . ~
FIGURE 6 Yen's theorem in one dimension. NOTE: (A) Continuous signal sampled at
irregularly spaced points. (B) Yen interpolation functions for sample points 3, 6, and 9.
(The curve for point 3 is dotted for visibility.) (C) Yen reconstruction of a sinusoid at
one-half the nominal Nyquist frequency. (D) Yen reconstruction of a sinusoid at 1.5 times
the Nyquist frequency.
OCR for page 127
PHOTORECEPTOR MOSAIC AS AN IMAGE SAMPLING DEVICE
127
Yen's sampling theorem, like Shannon's, assumes an infinite number of
sample points and strictly bandlimited signals, so we need assurance that his
algorithm actually works under realistic conditions. Figure 6 demonstrates
that it does and also illustrates the mechanics of Yen reconstruction. Panel
A shows 11 irregularly spaced sample points. Like Shannon reconstruction,
Yen reconstruction is based on interpolation functions centered at the
sample points. But instead of using the same sine function for every
point, Yen's algorithm requires each point to have its own idiosyncratic
interpolation function one that depends on the distances between that
point and all the others. Panel B shows the interpolation functions for
three of the points from panel ~ One can see that Yen interpolation
functions look like distorted sine functions drawn by a careless draftsman:
they are often distinctly asymmetric, and no two are exactly alike.
Like Shannon's algorithm, Yen reconstruction consists of weighting
each sample point's interpolation function by the sample value (i.e., the
input signal value) at that point and then summing all the interpolation
functions. Figure 6C shows the results for a sinusoid whose frequency
is one-half the nominal Nyquist limit. The results are as good as the
comparable Shannon reconstruction shown in Figure 1C (i.e., essentially
perfect).
Figure 6D shows what happens when Yen's algorithm is applied to a si-
nusoid whose frequency exceeds the nominal Nyquist limit. Here the results
are quite unlike those produced by regularly spaced sample points (e.g.,
in Figure 1D): instead of aliasing creating a perfect low-frequency (sub-
Nyquist) sinusoid, the reconstructed signal looks like a sinusoid corrupted
by low-frequency noise. This is a general feature of Yen reconstruction:
as the spacing of the sample points becomes more irregular, the less the
aliases of supra-Nyquist sinusoids look like perfect low-frequency sinusoids
and the more they look like broadband noise. This property of Yen re-
construction is at least qualitatively consistent with what one sees when
supra-Nyquist gratings are imaged on the fovea by interferometry.
In the vision literature Yen's algorithm has sometimes been treated as
one that could not be readily implemented by a visual system, and this has
been used as an argument that irregular image sampling must have very
deleterious consequences for vision (French et al., 1977~. This idea may
have been prompted in part by the fact that in print the analytic expressions
for Yen interpolation functions look horribly complicated. But in reality
the theorem is based on a simple idea, and the computations involved are
all straightforward linear operations.
The idea is that if we had sampled an appropriately bandlimited signal
at regularly spaced points, we could have used sine interpolation to find its
values at any other set of points. So given the signal values at the points
where we actually did sample, we can interpolate backward to find the
OCR for page 128
128
JOHN I. YELL07T
values that must have occurred at the lattice points and then use them to
reconstruct the entire signal by Shannon's method. This process turns out
to be equivalent to erecting a tailor-made interpolation function around
each of the actual sample points a function that is a weighted sum of
sine functions. The only trick is to find the proper weights, and these can
be readily calculated (by a matrix inversion) if the actual positions of the
sample points are known.
It is important to note that these weights need to be computed only
once for any given sampling array. Thereafter, Yen reconstruction is com-
putationally just as simple as Shannon reconstruction. So Yen's algorithm
seems quite compatible with a visual system whose sampling elements do
not change position from moment to moment, like blades of grass in the
wind, but that form a stable irregular mosaic that lasts a lifetime.
EXTRAFOVEAL IMAGE SAMPLING
Yen's theorem allows us to say that if the visual system can learn the
actual positions of its foveal cones, then despite spatial disorder in the
cone mosaic, receptor sampling of normal foveal images need not impose
any loss of information. The critical point is that in the fovea the average
sampling rate matches the spatial frequency bandwidth imposed by the
optics of the eye.
Outside the fovea, however, there is a significant mismatch: the cones
undersample the retinal image. Osterberg (1935) found that cone density
decreased very rapidly with eccentricity in the single retina he examined,
and the recent work of Curcio et al. (1987) indicates that Osterberg's
density versus eccentricity curve is generally valid across individuals for
eccentricities beyond about 1 deg. Thus, in a typical human retina the
nominal Nyquist limit of the cones at 4-de" eccentricity has dropped from
its foveal value of 60 cycles/degree to about 20. But the available evidence
indicates that the spatial bandwidth of the retinal image has not decreased
by anything like the same amount. In fact the line-spread measurements
of Jennings and Charman (1981) suggest that at 4 deg the bandwidth is
still around 60 cycles/degree, so in that region the cones undersample the
retinal image by a factor of 3!
That estimate may be somewhat extreme, because increased retinal
thickness outside the fovea complicates the problem of estimating the true
spatial bandwidth of the image at the level of the receptors. But there
seems to be little doubt that image bandwidth decreases more slowly than
cone density, so that the retinal image is significantly undersampled outside
the fovea. Recent psychophysical results support this conclusion: Still and
Thibos (1987) found that at 20 deg in the periphery the gratings on a CRT
screen can be discriminated from uniform fields up to 22 cycles/degree,
OCR for page 129
PHOTORECEPTOR MOSAIC AS AN IMAGE SAMPLING DEVICE
129
while the nominal Nyquist limit at that eccentricity is about 10 cycles. In
other words, the retinal image bandwidth at 20 deg must be at least twice
the local Nyquist limit of the cones.
Unlike cone density, which decreases continuously with retinal ec-
centricity, the spatial regularity of the cones declines only over the first
two to three degrees and then appears to reach a stable state that pre-
vails throughout the rest of the retina. This fact is easy to appreciate
by examining the Fourier spectra of small sections of cones from differ-
ent eccentricities (Yellott, 1983a) or from comparable statistical analyses
(Hirsch and Miller, 1987~. Figure 7 shows, on the left, the cone positions
in a small section of monkey retina at 2.5~eg eccentricity (from Hirsch
and Miller, 1987) and, on the right, the Fourier spectrum of this cone
array. Clearly this spectrum looks nothing like the perfect lattice of deltas
required by Shannon's theorem. Instead it contains a single delta at the
origin, surrounded by a circular island of empty space that ends abruptly
in a sea of noise. This kind of "desert island" spectrum characterizes local
sections of the cone mosaic at all eccentricities beyond about 2.5 deg. As
cone density decreases beyond that point, the radius of the desert island
decreases, but it always has a value that is approximately twice the nominal
Nyquist frequency implied by the local density. [A desert island spectrum
indicates that the extrafoveal cones are packed essentially at random but
are subject to a constraint on the minimum cone-cone distance (Yellott,
1983b).]
Figure 8 illustrates the spectral consequences of desert island sampling
for a sinusoidal grating. The top represents the spectrum of the sampling
array and the spectrum of the grating and simply reminds us that the spec-
trum of the grating after sampling will be the convolution of the two. On
the bottom we see the postsampling spectrum for cases in which the grating
frequency falls below or above the nominal Nyquist limit. Three points
can be observed here: (1) For sub-Nyquist frequencies the postsampling
spectrum contains deltas that fall in a relatively noise-free region of fre-
quency space. In other words, sub-Nyquist images do not mask themselves.
(2) For frequencies above the Nyquist limit there is no concentration of
aliased energy at any single low frequency. Instead the aliased energy is
scattered out into broadband noise at all orientations. Consequently, alias-
ing here will not take the form of periodic moire patterns but instead will
look like broadband noise. (3) The postsampling spectrum retains a strong
concentration of spectral power at the original input frequency, even when
that frequency falls above the Nyquist limit. This means that an irregular
sampling array offers the potential for vision beyond the Nyquist frequency.
Figure 9 illustrates the last two points using an actual array of ex-
trafoveal cones. At the top the cones are shown sampling horizontal and
vertical gratings at 1.25 times the Nyquist limit. Below are the power
OCR for page 130
9'
*
* * ~
2.53
130
a
e
(Deg.) 2 7R
JOHN I. YELLOIT
Section
Spectrum
Hirsch & Miller Data Bar = 38 cvales/dec
P.
r .~
FIGURE 7 Spatial disorder in the extrafoveal cone mosaic. NOISE: Left side: dots mark
the positions of extrafoveal cone inner segments in a monkey retina at Z5 deg eccentricity
(as measured By Hirsch and Miller, 1987~. Right side: Fourier spectrum of the cone array
(computed by A.J. Ahumada). The horizontal bar across the desert island portion of the
spectn~m marks a distance equal to twice the Nyquist frequency implied by local cone
density.
spectra of the sampled images. We see that there is no hint of the kind of
narrowband aliasing (i.e., moire patterns) produced by the foveal cones (cf.
Figure 5~. Aliasing here takes the form of broadband noise. And we see
in the postsampling spectra that there is easily enough energy concentrated
at the input frequencies to allow us to determine which spectrum is which.
Of course for the visual system to make such a discrimination, it
must have orientation-sensitive mechanisms tuned to spatial frequencies
that exceed the local Nyquist limits of its cones. Apparently our visual
system does have such mechanisms, because Williams and Coletta (1987)
have recently shown that observers can discriminate grating orientation
at frequencies up to 1.5 times the nominal Nyquist limit out to lO~eg
eccentricity. (At higher frequencies grating orientation can no longer be
OCR for page 131
PHOTORECEPTOR MOSAIC AS AN IMAGE SAMPLING DEVICE
-
R
~<~'2 ~
itch
I_
,
~ o i'
-K -~i +~t
OR
131
1 1 1
-id o F~
FIGURE 8 Schematic illustration of the effects of sampling a grating with a point array
whose spectrum has the desert island form found in the primate extrafoveal cone mosaic.
NOTE: Top left: idealized desert island spectrum whose noise-free island has radius R
(]V2 = nominal Nyquist frequency). Top right: spectrum of a sinusoid with frequency F.
Bottom left: spectrum of the sampled grating when F < RL2. Here the grating energy at
~rF escapes masking by the sampling noise, all of which falls outside the Nyquist region.
Bottom right: postsampling spectrum when F > R/2. Aliased energy in the form of
broadband noise is widely scattered over the Nyquist region, with no concentration at any
single sub-Nyquist frequency, and sharp spikes remain at the input frequency points OF.
identified, but spatial contrast can still be detected in the form of broadband
noise.)
How can the visual system make effective use of the information pro-
vided by a highly disordered receptor mosaic that undersamples the retinal
image by a factor of 2 or more? Yen's theorem provides no guidance here,
but recently Chen and Allebach (1987) showed that for bandlimited images
undersampled by irregular point arrays, the least-squares reconstruction al-
gorithm under quite general assumptions is very similar to Yen's. Like Yen
reconstruction, Chen-Allebach reconstruction is based on tailor-made in-
terpolation functions for each sample point (quite like those in Figure 6B),
OCR for page 132
32
JOlIN I. YELLOlT
FIGURE 9 Visibility of spatial frequencies beyond the Nyquist limit in extrafoveal retina.
NOTE: Top: one degree section of extrafoveal rhesus monkey cones (3.8~eg eccentricity
sampling horizontal and vertical gratings whose frequent is 1.25 times the nominal Nyquist
frequency of the cone array. Bottom: Fourier power spectra (optical transforms) of the
sampled gratings. SOURCE: Adapted from Williams and Coletta (1987~.
and the construction of those functions requires knowledge of the actual
sampling position. But here again, once the interpolation functions have
been created, they are good for life, and the reconstruction of any given
input image is computationally just as simple as it would be for spatially
regular sampling below the Nyquist limit. It is too early to say how well the
Chen-Allebach algorithm will mesh with the facts of extrafoveal vision, but
I think it is a promising direction for exploration. J. Ahumada and I are
currently studying mechanisms by which higher visual centers could learn
OCR for page 133
PHOTORECEPTOR MOSAIC AS AN IMAGE SAMPLING DEVICE
133
the receptor positions and compute the appropriate interpolation functions
(Ahumada and Yellott, 1988).
ACKNOWLEDGMENTS
I thank L.N. Thibos for sharing information on the work of Bergmann
(1858), M. D'Zmura for providing his English translation of Bergmann's
paper, D.R. Williams for permission to reproduce Figures 5 and 9, and J.
Ahumada for his long-term collaboration. This research was supported in
part by the National Aeronautics and Space Administration under joint
research interchange NCA2-5.
REFERENCES
Ahumada, A.J., and J.I. Yellott
1985 A model for foveal photoreceptor placing. Investigative Ophthalmology and
Visual Science 26 (Suppl.): 11 (abstract).
1988 A connectionist model for learning receptor positions. Investigative Ophthal-
mology and Visual Science 29 (Suppl.~:58 (abstract).
Bergmann, C.
1858 Anatomisches und physiologisches uber die netzhaut des auges. Zeitschnit
fir rationelle Medicine 2:83-108.
Brindley, G.S.
1970 Physiology of the Retina and Visual Pathway. Baltimore, Md.: Williams &
Wilkins.
Campbell, F.W., and R.W. Gubisch
1966 Optical quality of the human eye. Joumal of Physiology (London) 186:55
578.
Chen, D.S., and J.P. Allebach
1987 Analysis of error in reconstruction of two-dimensional signals from irregularly
spaced points. IEEE Transactions on Acoustics, Speech, and Signal Processing
35: 173-180.
Curcio, C.A., K.R. Sloan, O. Packer, A.E. Hendrickson, and R. E. Kalina
1987 Distribution of cones in human and monkey retina: individual variability
and radial asymmetry. Science 236:579-582.
French, AS., A.W. Snyder, and D.G. Stavenga
1977 Image degradation by an irregular retinal mosaic. Biological Cybemetics
27:229-233.
Goodman, J.W.
1968 Introduction to Fourier Optics. New York: McGraw-Hill.
Helmholtz, H.
1860 Handbuch den Physiologichen Optik, vol. II. (English edition republished by
Dover Publications, New York, 1962~.
Hirsch, J., and R. Hylton
1984 Quality of the primate photoreceptor lattice and limits of spatial vision.
Vision Research 24:347-356.
OCR for page 134
134
JOHN L YELLOlT
Hirsch, J., and W.H. Miller
1987 Does cone positional disorder limit near-fovea! acuity? Joumal of the Optical
Society of America A 2:1481-1942.
Jennings, J.NM., and W.N. Chalman
1981 Off-axis image quality in the human eye. Vision Research 21:445-455.
Miller, W.H., and G.D. Bernard
1983 Averaging over the foveal receptor aperture curtails aliasing. Vision Research
23:1365-1369.
Osterberg, G.
1935 Topology of the layer of rods and cones in the human retina. Acta
OpAthalmologica 6(Suppl.~:1-103.
Shannon, C.E.
1949 Communication in the presence of noise. Proceedings of the IRE 37:1(}21.
Still, D.L., and L^N. Thibos
1987 Detection of peripheral aliasing for gratings seen in the Newtonian view.
Joumal of the Optical Society America A. 4:P79-P80 (abstract).
Williams, D.R.
1985 Aliasing in human foveal vision. Vision Research 24: 195-205.
Williams, D.R. and N.J. Coletta
1987 Cone spacing and the visual resolution limit. Joumal of the Optical Society
of America A 4:1514-1523.
Yellott, J.I.
1983a Spectral consequences of photoreceptor sampling in the rhesus retina.
Science 221:382-385.
1983b Nonhomogeneous Poisson disks model the photoreceptor mosaic. Investigative
Ophthalmology and Visual Science 24 (Suppl.~:147 (abstract).
Yen, JOLT
1956 On the nonuniform sampling of bandwidth limited signals. IRE liens.
Circuit Theory CI`-3:251-257.
Representative terms from entire chapter:
image sampling
