Merriam, E. P., Gardner, J. L., Movshon, J. A., and Heeger, D. J. (2013) Modulation of visual responses by gaze direction in human visual cortex. Journal of Neuroscience 33: 9879-9889 DOI Abstract
To locate visual objects, the brain combines information about retinal location and direction of gaze. Studies in monkeys have demonstrated that eye position modulates the gain of visual signals with “gain fields,” so that single neurons represent both retinotopic location and eye position. We wished to know whether eye position and retinotopic stimulus location are both represented in human visual cortex. Using functional magnetic resonance imaging, we measured separately for each of several different gaze positions cortical responses to stimuli that varied periodically in retinal locus. Visually evoked responses were periodic following the periodic retinotopic stimulation. Only the response amplitudes depended on eye position; response phases were indistinguishable across eye positions. We used multivoxel pattern analysis to decode eye position from the spatial pattern of response amplitudes. The decoder reliably discriminated eye position in five of the early visual cortical areas by taking advantage of a spatially heterogeneous eye position-dependent modulation of cortical activity. We conclude that responses in retinotopically organized visual cortical areas are modulated by gain fields qualitatively similar to those previously observed neurophysiologically.
pdf
Costagli, M., Ueno, K., Sun, P., Gardner, J. L., Wan, X., Ricciardi, E., Pietrini, P., Tanaka, K., and Cheng, K. (2012) Functional signalers of changes in visual stimuli: Cortical responses to increments and decrements in motion coherence. Cerebral Cortex Advance access:1-9 DOI Abstract
How does our brain detect changes in a natural scene? While changes by increments of specific visual attributes, such as contrast or motion coherence, can be signaled by an increase in neuronal activity in early visual areas, like the primary visual cortex (V1) or the human middle temporal complex (hMT+), respectively, the mechanisms for signaling changes resulting from decrements in a stimulus attribute are largely unknown. We have discovered opposing patterns of cortical responses to changes in motion coherence: unlike areas hMT+, V3A and parieto-occipital complex (V6+) that respond to changes in the level of motion coherence monotonically, human areas V4 (hV4), V3B, and ventral occipital always respond positively to both transient increments and decrements. This pattern of responding always positively to stimulus changes can emerge in the presence of either coherence-selective neuron populations, or neurons that are not tuned to particular coherences but adapt to a particular coherence level in a stimulus-selective manner. Our findings provide evidence that these areas possess physiological properties suited for signaling increments and decrements in a stimulus and may form a part of cortical vigilance system for detecting salient changes in the environment.
pdf
Suzuki, S., Harasawa, N., Ueno, K., Gardner, J. L., Ichinohe, N., Haruno, M., Cheng, K., and Nakahara H. (2012) Learning to simulate others' decisions. Neuron 74:1125-37 DOI SI Abstract
A fundamental challenge in social cognition is how humans learn another person's values to predict their decision-making behavior. This form of learning is often assumed to require simulation of the other by direct recruitment of one's own valuation process to model the other's process. However, the cognitive and neural mechanism of simulation learning is not known. Using behavior, modeling, and fMRI, we show that simulation involves two learning signals in a hierarchical arrangement. A simulated-other's reward prediction error processed in ventromedial prefrontal cortex mediated simulation by direct recruitment, being identical for valuation of the self and simulated-other. However, direct recruitment was insufficient for learning, and also required observation of the other's choices to generate a simulated-other's action prediction error encoded in dorsomedial/dorsolateral prefrontal cortex. These findings show that simulation uses a core prefrontal circuit for modeling the other's valuation to generate prediction and an adjunct circuit for tracking behavioral variation to refine prediction.
pdf
Sun, P., Gardner, J. L., Costagli, M., Ueno, K., Waggoner, R. A., Tanaka, K., and Cheng K. (2012) Demonstration of tuning to stimulus orientation in the human visual cortex: A high-resolution fMRI study with a novel continuous and periodic stimulation paradigm. Cerebral Cortex Advance access:1-12 DOI SI Abstract
Cells in the animal early visual cortex are sensitive to contour orientations and form repeated structures known as orientation columns. At the behavioral level, there exist 2 well-known global biases in orientation perception (oblique effect and radial bias) in both animals and humans. However, their neural bases are still under debate. To unveil how these behavioral biases are achieved in the early visual cortex, we conducted high-resolution functional magnetic resonance imaging experiments with a novel continuous and periodic stimulation paradigm. By inserting resting recovery periods between successive stimulation periods and introducing a pair of orthogonal stimulation conditions that differed by 90° continuously, we focused on analyzing a blood oxygenation level-dependent response modulated by the change in stimulus orientation and reliably extracted orientation preferences of single voxels. We found that there are more voxels preferring horizontal and vertical orientations, a physiological substrate underlying the oblique effect, and that these over-representations of horizontal and vertical orientations are prevalent in the cortical regions near the horizontal- and vertical-meridian representations, a phenomenon related to the radial bias. Behaviorally, we also confirmed that there exists perceptual superiority for horizontal and vertical orientations around horizontal and vertical meridians, respectively. Our results, thus, refined the neural mechanisms of these 2 global biases in orientation perception.
pdf
Pestilli, F., Carrasco, M., Heeger, D. J. and Gardner, J. L. (2011) Attentional enhancement via selection and pooling of early sensory responses in human visual cortex. Neuron 72:832-46 DOI SI
<Preview by John T. Serences> Abstract
To characterize the computational processes by which attention improves behavioral performance, we measured activity in visual cortex with functional magnetic resonance imaging as humans performed a contrast-discrimination task with focal and distributed attention. Focal attention yielded robust improvements in behavioral performance that were accompanied by increases in cortical responses. Using a quantitative analysis, we determined that if performance were limited only by the sensitivity of the measured sensory signals, the improvements in behavioral performance would have corresponded to an unrealistically large (approximately 400%) reduction in response variability. Instead, behavioral performance was well characterized by a pooling and selection process for which the largest sensory responses, those most strongly modulated by attention, dominated the perceptual decision. This characterization predicts that high contrast distracters that evoke large sensory responses should have a negative impact on behavioral performance. We tested and confirmed this prediction. We conclude that attention enhanced behavioral performance predominantly by enabling efficient selection of the behaviorally relevant sensory signals.pdf
Liu, T., Hospadaruk, L., Zhu, D., and Gardner, J. L. (2011) Feature-specific attentional priority signals in human cortex. Journal of Neuroscience 31:4484-95 DOI Abstract
Human can flexibly attend to a variety of stimulus dimensions, including spatial location and various features such as color and direction of motion. While the locus of spatial attention has been hypothesized to be represented by priority maps encoded in several dorsal frontal and parietal areas, it is unknown how the brain represents attended features. Here we examined the distribution and organization of neural signals related to deployment of feature-based attention. Subjects viewed a compound stimulus containing two superimposed motion directions (or colors), and were instructed to perform an attention-demanding task on one of the directions (or colors). We found elevated and sustained fMRI response for the attention task compared to a neutral condition, without reliable differences in overall response amplitude between attending to different features. However, using multi-voxel pattern analysis, we were able to decode the attended feature in both early visual areas (V1 to hMT+) and frontal and parietal areas (e.g., IPS1-4 and FEF) that are commonly associated with spatial attention. Furthermore, analysis of the classifier weight maps showed that attending to motion and color evoked different patterns of activity, suggesting different neuronal subpopulations in these regions are recruited for attending to different feature dimensions. Thus, our finding suggests that rather than a purely spatial representation of priority, frontal and parietal cortical areas also contain multiplexed signals related to the priority of different non-spatial features.
pdf
Offen S, Gardner, J. L., Schluppeck D and Heeger, D.J. (2010) Differential roles for frontal eye fields (FEFs) and intraparietal sulcus (IPS) in visual working memory and visual attention. Journal of Vision 10:1-14 DOI Abstract
Cortical activity was measured with functional magnetic resonance imaging to probe the involvement of the superior precentral sulcus (including putative human frontal eye fields, FEFs) and the intraparietal sulcus (IPS) in visual short-term memory and visual attention. In two experimental tasks, human subjects viewed two visual stimuli separated by a variable delay period. The tasks placed differential demands on short-term memory and attention, but the stimuli were visually identical until after the delay period. An earlier study (S. Offen, D. Schluppeck, & D. J. Heeger, 2009) had found a dissociation in early visual cortex that suggested different computational mechanisms underlying the two processes. In contrast, the results reported here show that the patterns of activation in prefrontal and parietal cortex were different from one another but were similar for the two tasks. In particular, the FEF showed evidence for sustained delay period activity for both the working memory and the attention task, while the IPS did not show evidence for sustained delay period activity for either task. The results imply differential roles for the FEF and IPS in these tasks; the results also suggest that feedback of sustained activity from frontal cortex to visual cortex might be gated by task demands.
pdf
Dinstein I, Gardner, J. L., Jazayeri, M and Heeger, D.J. (2008) Executed and observed movements have different distributed representations in human aIPS. Journal of Neuroscience 28:11231-11239 DOI Abstract
How similar are the representations of executed and observed hand movements in the human brain? We used functional magnetic resonance imaging (fMRI) and multivariate pattern classification analysis to compare spatial distributions of cortical activity in response to several observed and executed movements. Subjects played the rock-paper-scissors game against a videotaped opponent, freely choosing their movement on each trial and observing the opponent's hand movement after a short delay. The identities of executed movements were correctly classified from fMRI responses in several areas of motor cortex, observed movements were classified from responses in visual cortex, and both observed and executed movements were classified from responses in either left or right anterior intraparietal sulcus (aIPS). We interpret above chance classification as evidence for reproducible, distributed patterns of cortical activity that were unique for execution and/or observation of each movement. Responses in aIPS enabled accurate classification of movement identity within each modality (visual or motor), but did not enable accurate classification across modalities (i.e., decoding observed movements from a classifier trained on executed movements and vice versa). These results support theories regarding the central role of aIPS in the perception and execution of movements. However, the spatial pattern of activity for a particular observed movement was distinctly different from that for the same movement when executed, suggesting that observed and executed movements are mostly represented by distinctly different subpopulations of neurons in aIPS.
pdf
Gardner, J. L. , Merriam, E. P., Movshon, J. A., and Heeger, D.J. (2008) Maps of visual space in human occipital cortex are retinotopic, not spatiotopic. Journal of Neuroscience 28:3988-3999 DOI This Week in the Journal Abstract
We experience the visual world as phenomenally invariant to eye position, but almost all cortical maps of visual space in monkeys use a retinotopic reference frame, that is, the cortical representation of a point in the visual world is different across eye positions. It was recently reported that human cortical area MT (unlike monkey MT) represents stimuli in a reference frame linked to the position of stimuli in space, a “spatiotopic” reference frame. We used visuotopic mapping with blood oxygen level-dependent functional magnetic resonance imaging signals to define 12 human visual cortical areas, and then determined whether the reference frame in each area was spatiotopic or retinotopic. We found that all 12 areas, including MT, represented stimuli in a retinotopic reference frame. Although there were patches of cortex in and around these visual areas that were ostensibly spatiotopic, none of these patches exhibited reliable stimulus-evoked responses. We conclude that the early, visuotopically organized visual cortical areas in the human brain (like their counterparts in the monkey brain) represent stimuli in a retinotopic reference frame.
pdf
Sun, P., Ueno K., Waggoner, R. A., Gardner, J. L. , Tanaka, K., and Cheng K. (2007) A temporal frequency-dependent functional architecture in human V1 revealed by high-resolution fmri. Nature Neuroscience 10:1404-1406 DOI Abstract
Although cortical neurons with similar functional properties often cluster together in a columnar organization, only ocular dominance columns, the columnar structure representing segregated anatomical input (from one of the two eyes), have been found in human primary visual cortex (V1). It has yet to be shown whether other columnar organizations that arise only from differential responses to stimulus properties also exist in human V1. Using high-resolution functional magnetic resonance imaging, we have found such a functional architecture containing domains that respond preferentially to either low or high temporal frequency.
pdf
Gardner, J. L. , Sun, P., Waggoner, R. A., Ueno K., Tanaka, K., and Cheng K. (2005) Contrast adaptation and representation in human early visual cortex. Neuron 47:607-620 DOI <Preview by Geoffrey M. Boynton> Abstact
The human visual system can distinguish variations in image contrast over a much larger range than measurements of the static relationship between contrast and response in visual cortex would suggest. This discrepancy may be explained if adaptation serves to re-center contrast response functions around the ambient contrast, yet experiments on humans have yet to report such an effect. By using event-related fMRI and a data-driven analysis approach, we found that contrast response functions in V1, V2, and V3 shift to approximately center on the adapting contrast. Furthermore, we discovered that, unlike earlier areas, human V4 (hV4) responds positively to contrast changes, whether increments or decrements, suggesting that hV4 does not faithfully represent contrast, but instead responds to salient changes. These findings suggest that the visual system discounts slow uninformative changes in contrast with adaptation, yet remains exquisitely sensitive to changes that may signal important events in the environment.
pdf
Gardner, J. L. , Tokiyama, S., and Lisberger, S. G. (2004) A population decoding framework for motion aftereffects on smooth pursuit eye movements. Journal of Neuroscience 24:9035-9048 DOI Abstract
Both perceptual and motor systems must decode visual information from the distributed activity of large populations of cortical neurons. We have sought a common framework for understanding decoding strategies for visually guided movement and perception by asking whether the strong motion aftereffects seen in the perceptual domain lead to similar expressions in motor output. We found that motion adaptation indeed has strong sequelae in the direction and speed of smooth pursuit eye movements. After adaptation with a stimulus that moves in a given direction for 7 sec, the direction of pursuit is repelled from the direction of pursuit targets that move within 90 degrees of the adapting direction. The speed of pursuit decreases for targets that move at the direction and speed of the adapting stimulus and is repelled from the adapting speed in the sense that the decrease either becomes greater or smaller (eventually turning to an increase) when tracking targets move slower or faster than the adapting speed. The effects of adaptation are spatially specific and fixed to the retinal location of the adapting stimulus. The magnitude of adaptation of pursuit speed and direction is uncorrelated, suggesting that the two parameters are decoded independently. Computer simulation of motion adaptation in the middle temporal visual area (MT) shows that vector-averaging decoding of the population response in MT can account for the effects of adaptation on the direction of pursuit. Our results suggest a unified framework for thinking, in terms of population decoding, about motion adaptation for both perception and action.
pdf
Churchland, A. K., Gardner, J. L. , Chou, I. H., Priebe, N. J., and Lisberger, S. G. (2003) Directional anisotropies reveal a functional segregation of visual motion processing for perception and action. Neuron 37:1001-1011 DOI Abstract
Human exhibits an anisotropy in direction perception: discrimination is superior when motion is around horizontal or vertical rather than diagonal axes. In contrast to the consistent directional anisotropy in perception, we found only small idiosyncratic anisotropies in smooth pursuit eye movements, a motor action requiring accurate discrimination of visual motion direction. Both pursuit and perceptual direction discrimination rely on signals from the middle temporal visual area (MT), yet analysis of multiple measures of MT neuronal responses in the macaque failed to provide evidence of a directional anisotropy. We conclude that MT represents different motion directions uniformly, and subsequent processing creates a directional anisotropy in pathways unique to perception. Our data support the hypothesis that, at least for visual motion, perception and action are guided by inputs from separate sensory streams. The directional anisotropy of perception appears to originate after the two streams have segregated and downstream from area MT.
pdf
Gardner, J. L. , and Lisberger, S. G. (2002) Serial linkage of target selection for orienting and tracking eye movements. Nature Neuroscience 5:892-899 DOI <News and Views by Michael N. Shadlen> Abstract
Many natural actions require the coordination of two different kinds of movements. How are targets chosen under these circumstances: do central commands instruct different movement systems in parallel, or does the execution of one movement activate a serial chain that automatically chooses targets for the other movement? We examined a natural eye tracking action that consists of orienting saccades and tracking smooth pursuit eye movements, and found strong physiological evidence for a serial strategy. Monkeys chose freely between two identical spots that appeared at different sites in the visual field and moved in orthogonal directions. If a saccade was evoked to one of the moving targets by microstimulation in either the frontal eye field (FEF) or the superior colliculus (SC), then the same target was automatically chosen for pursuit. Our results imply that the neural signals responsible for saccade execution can also act as an internal command of target choice for other movement systems.
pdf
Gardner, J. L. , and Lisberger, S. G. (2001) Linked target selection for saccadic and smooth pursuit eye movements. Journal of Neuroscience 21(6):2075-2084 link Abstract
In natural situations, motor activity must often choose a single target when multiple distractors are present. The present paper asks how primate smooth pursuit eye movements choose targets, by analysis of a natural target-selection task. Monkeys tracked two targets that started 1.5 degrees eccentric and moved in different directions (up, right, down, and left) toward the position of fixation. As expected from previous results, the smooth pursuit before the first saccade reflected a vector average of the responses to the two target motions individually. However, post-saccadic smooth eye velocity showed enhancement that was spatially selective for the motion at the endpoint of the saccade. If the saccade endpoint was close to one of the two targets, creating a targeting saccade, then pursuit was selectively enhanced for the visual motion of that target and suppressed for the other target. If the endpoint landed between the two targets, creating an averaging saccade, then post-saccadic smooth eye velocity also reflected a vector average of the two target motions. Saccades with latencies >200 msec were almost always targeting saccades. However, pursuit did not transition from vector-averaging to target-selecting until the occurrence of a saccade, even when saccade latencies were >300 msec. Thus, our data demonstrate that post-saccadic enhancement of pursuit is spatially selective and that noncued target selection for pursuit is time-locked to the occurrence of a saccade. This raises the possibility that the motor commands for saccades play a causal role, not only in enhancing visuomotor transmission for pursuit but also in choosing a target for pursuit.
pdf
Gardner, J. L. , Anzai, A., Ohzawa. I., and Freeman, R. D. (1999) Linear and nonlinear contributions to orientation tuning of simple cells in the cat's striate cortex. Visual Neuroscience 16:1115-1121 DOI Abstract
Orientation selectivity is one of the most conspicuous receptive-field (RF) properties that distinguishes neurons in the striate cortex from those in the lateral geniculate nucleus (LGN). It has been suggested that orientation selectivity arises from an elongated array of feedforward LGN inputs (Hubel & Wiesel, 1962). Others have argued that cortical mechanisms underlie orientation selectivity (e.g. Sillito, 1975; Somers et al., 1995). However, isolation of each mechanism is experimentally difficult and no single study has analyzed both processes simultaneously to address their relative roles. An alternative approach, which we have employed in this study, is to examine the relative contributions of linear and nonlinear mechanisms in sharpening orientation tuning. Since the input stage of simple cells is remarkably linear, the nonlinear contribution can be attributed solely to cortical factors. Therefore, if the nonlinear component is substantial compared to the linear contribution, it can be concluded that cortical factors play a prominent role in sharpening orientation tuning. To obtain the linear contribution, we first measure RF profiles of simple cells in the cat's striate cortex using a binary m-sequence noise stimulus. Then, based on linear spatial summation of the RF profile, we obtain a predicted orientation-tuning curve, which represents the linear contribution. The nonlinear contribution is estimated as the difference between the predicted tuning curve and that measured with drifting sinusoidal gratings. We find that measured tuning curves are generally more sharply tuned for orientation than predicted curves, which indicates that the linear mechanism is not enough to account for the sharpness of orientation-tuning. Therefore, cortical factors must play an important role in sharpening orientation tuning of simple cells. We also examine the relationship of RF shape (subregion aspect ratio) and size (subregion length and width) to orientation-tuning halfwidth. As expected, predicted tuning halfwidths are found to depend strongly on both subregion length and subregion aspect ratio. However, we find that measured tuning halfwidths show only a weak correlation with subregion aspect ratio, and no significant correlation with RF length and width. These results suggest that cortical mechanisms not only serve to sharpen orientation tuning, but also serve to make orientation tuning less dependent on the size and shape of the RF. This ensures that orientation is represented equally well regardless of RF size and shape.
pdf