G. Sperling
Workshop, July 7-11, 1997

Overall Topic:  Early visual processing: Selected Methods and Theories 

  1. Spatial Channels:  Immediately after the visual receptors, the rods and
cones, the visual system segregates information according to scale, so that
information within a spatial neighborhood is normally is represented
concurrently by neurons with overlapping receptive fields of enormously
different sizes.  Ultimately, after more than a half-dozen stages of
processing, information from these various channels is again combined.

  The questions asked are:

    Why does the visual system have channels?  How do they operate?
   
    
  2. Sandwich Models:   Contemporary theories of visual processing can be
characterized as sandwich models, in which stages of linear processing and 
nonlinear processing alternate.  Some methods for isolating and measuring
layers of the sandwich will discussed, primarily in the context of models
for visual motion and texture.

 -----------

   The critical computational element of channels is that they are,
approximately, spatial bandpass filters.  Therefore, the course will begin
with a review of linear systems theory, covering such items as superposition,
convolution, filters, Fourier analysis.  As this probably is familiar
to most of the participants, the lecture will concentrate on a graphical
approach that has been found useful in communicating these concepts to
undergraduates and in developing good intuitions, and on good tools for
understanding the two-dimensional transformations commonly used in image
processing and assumed to occur in early vision.  

   If this subsection is successful, students will be able to look at
images and functions and make quick, accurate estimations of their Fourier
transform, and to look at a pair of related images A,A' (or functions) and
estimate the transform that mapped A to A'.

   How these simplest linear models and methods are usefully applied in the
psychophysical investigation of visual spatial channels will be illustrated
in selected experiments.

 -----------


   Among the nonlinear layers in sandwich models are gain-control,
rectification (halfwave and fullwave), detection (e.g., correlation), these
three occurring repeatedly, followed by decision processes (e.g., ideal
detectors and Signal Detection Theory).  These various nonlinear processes
will be illustrated with examples from visual motion and texture perception
(which are formally quite similar).  There will be an abbreviated and
selective presentation of first- second- and third-order motion models
and their texture analogs.    A full development could easily occupy the
whole unit; some of the selected topics are: Reichardt and motion energy
models, driftbalanced and microbalanced stimuli, how to discriminate
fullwave versus halfwave rectification, texture grabbers, phase methods
for discriminating between single and multiple mechanisms, and possibly
more.

   If time permits, we will consider histogram contrast analysis (a technique
currently being exploited to characterize nonlinear processes in texture
perception but applicable more generally) and wavelet theory (an alternative
to Fourier decomposition that creates a hierarchical representation of an
image utilizing both scale and location).
   
  An attempt will be made to coordinate some of the materials in these
lectures with those of Donald Laming.


 -----------

SELECTED REFERENCES ( * indicates handout at first class meeting)

*Sperling, G. (1989) Three stages and two systems of visual processing.
Spatial Vision, 183-207.
  (Overview of early visual processing and channels)

*Sperling, G. (1964).  Linear theory and the psychophysics of flicker.
Documenta Ophthalmologica, 18, 3-15.  
  Very elementary tutorial to linear theory in the context of flicker expts

van Santen, J. P. H., and G. Sperling.
Temporal covariance model of human motion perception.
Journal of the Optical Society of America A: Optics and Image Science, 1,
451-473.
  (Statement of Reichardt model for human motion-direction detection)

Adelson, E. H. and Bergen, J. K. (1985).
Spatio-temporal energy models for the perception of apparent motion.
Journal of the Optical Society of America A: Optics and Image Science, 2,
284-299.
  (Excellent tutorial introduction to motion stimuli and motion energy model)

Chubb, C., and G. Sperling. (1988).  Drift-balanced random stimuli:
A general basis for studying non-Fourier motion perception.
Journal of the Optical Society of America A: Optics and Image Science, 5,
1986-2006.
   (Defines second-order motion)

Lu, Zhong-Lin and Sperling, G.. (1995).
The functional architecture of human visual motion perception.
Vision Research, 35, 2697-2722.
   (The demonstration of three motion systems)

Sperling, G., and B. A. Dosher.  Strategy and optimization in human
information processing.  In K. Boff, L. Kaufman, and J. Thomas (Eds.),
Handbook of Perception and Performance. Vol. 1.  New York, NY: Wiley, 1986.
   (Lots of useful info. Cited here for demonstration of the equivalence of
   ROC (receiver operating characteristic) and SDT (signal detection theory)

Parish, D. H. and G. Sperling. (1991). Object spatial frequencies, retinal
spatial frequencies, noise, and the efficiency of letter discrimination.
Vision Research, 31, 1399-1415.
   (Source for bandwidth manipulation and for ideal detectors of computer
   presented, bandwidth filtered images) 

Geisler, Wilson S. (1989).  Sequential ideal-observer analysis of visual
discriminations.  Psychological Review, 96, 267-314.
   (Application of ideal detector theory to successive stages of visual
   processing)