I have very broad interests in visual neuroscience, ranging from photoreceptor response non-linearities, to the use of heuristics based on stored knowledge to accomplish perceptual tasks. At present, I am most involved in the following projects:
Judging poses, sizes and shapes of objects accurately is necessary for organisms and machines to operate successfully in the world. Retinal images of 3D objects are mapped by the rules of projective geometry, and preserve the invariants of that geometry. Since Plato, it has been debated whether geometry is innate to the human brain, and Poincare and Einstein thought it worth examining whether formal geometry arises from experience with the world. A photograph seen from the camera position, can form the same retinal projection as the physical 3D scene, but retinal projections of sizes and shapes are distorted in oblique viewing, so picture perception can provide critical tests of models of scene inference. In a series of papers we have shown that observers use internalized projective geometry to perceive sizes, shapes and poses in 3D scenes and their pictures. There are two ways we are expanding this geometry based work. The first is to understand action perception with dynamically deforming objects, and the second is to understand how sensory information interacts with stored geometric knowledge in the cortex.
Colors provide critical information about object and material identity and history, but they are completely created by our brains from the wavelengths in a scene. We have started to unravel the perceptual and neural mechanisms that assign colors to lights, transparencies and variegated surfaces, by combining human psychophysical experiments, computational models, and fMRI-guided microelectrode recordings from inferior-temporal (IT) macaque cortex. We showed that a limited set of flashed colors could be decoded from individual IT cells that are narrowly tuned. By measuring the complete 3-D color tuning of IT cells, we are testing if winner-take-all decoding, based only on the most responsive IT cell, performs as well as optimal Bayesian decoding of the complete population response. We will then proceed to the case where a colored filter or spotlight overlays a patterned surface, and observers perceive separate surface and overlay colors, thus requiring a more complicated decoding scheme. By identifying IT cells that signal transparency, we can test whether the winner-take-all decoding scheme still works. In addition we are looking at how decoding is altered by spatial context in natural images of faces and other objects. This work will elucidate the advantages of specialized narrowly tuned neurons generating high-dimensional representations in parallel, an important theoretical issue in neuroscience.
My previous clinical research contributions have been retinal, but now I am interested in understanding cortical processes related to visual function.
Significant distortions in orientation processing have been demonstrated in amblyopia, and these are likely to be more significant to the patient than some loss of visual acuity, and more revealing about the underlying neurological problems. I am interested in causes and implications: Are perceived orientation distortions caused by orientation under-sampling in cortical ocular dominance columns from the amblyopic eye? Is this a general effect of ocular dominance column shrinkage? Is the effect greater for the ON system? Can early corrections for strabismus or anisometropia alleviate the distortions? Do orientation processing deficits hamper perception of symmetry and 3D shape from texture and contour cues in real life? Can amblyopes be trained to use monocular projective geometry back-transforms to overcome this handicap?
How can humans perceive subtle differences in surface lightness when photoreceptor responses severely compress the luminance range of visual signals for efficient transmission? Using illusions, experiments and simulations, we have shown that cortical lateral interactions expand the range of perceived lightness in faces and other objects. Many retinal disorders are characterized by elevated contrast thresholds. I am interested in possible cortical compensations for these deficits in real world tasks.
- Steps towards neural decoding of colors, Current Opinion in Behavioral Sciences, 30(Array): 169-177, 2019
- Connectomic Identification and Three-Dimensional Color Tuning of S-OFF Midget Ganglion Cells in the Primate Retina., The Journal of neuroscience : the official journal of the Society for Neuroscience, 39(40): 7893-7909, 2019
- Geometrical structure of perceptual color space: Mental representations and adaptation invariance., Journal of vision, 19(12): 1, 2019
- Rotational-symmetry in a 3D scene and its 2D image., Journal of mathematical psychology, 87(Array): 108-125, 2018
- Picture perception reveals mental geometry of 3D scene inferences., Proceedings of the National Academy of Sciences of the United States of America, 115(30): 7807-7812, 2018
- Nonselective Wiring Accounts for Red-Green Opponency in Midget Ganglion Cells of the Primate Retina., The Journal of neuroscience : the official journal of the Society for Neuroscience, 38(6): 1520-1540, 2018
- Functional implications of orientation maps in primary visual cortex., Nature communications, 7(Array): 13529, 2016
- Evolution of neural computations: Mantis shrimp and human color decoding., i-Perception, 5(6): 492-6, 2014
- Eye movements and the neural basis of context effects on visual sensitivity., The Journal of neuroscience : the official journal of the Society for Neuroscience, 34(24): 8119-29, 2014
- Neuronal and perceptual differences in the temporal processing of darks and lights., Neuron, 82(1): 224-34, 2014
- Neuronal nonlinearity explains greater visual spatial resolution for darks than lights., Proceedings of the National Academy of Sciences of the United States of America, 111(8): 3170-5, 2014
- Frequency-based heuristics for material perception., Journal of vision, 13(14): Array, 2013
- Neural locus of color afterimages., Current biology : CB, 22(3): 220-4, 2012
- Visual inferences of material changes: color as clue and distraction., Wiley interdisciplinary reviews. Cognitive science, 2(6): 686-700, 2011
- Discerning nonrigid 3D shapes from motion cues., Proceedings of the National Academy of Sciences of the United States of America, 108(4): 1663-8, 2011