Representation of Visual Location and Orientation
This project uses cognitive neuropsychological methods and studies of normal adults and children to explore how the brain represents two very basic properties of visual stimuli: location and orientation.
When the brain is damaged or fails to develop normally, basic perceptual and cognitive abilities may malfunction. The resulting deficits, although unfortunate or even tragic for those who suffer from them, offer unique windows into the structure and functioning of the normal mind. My colleagues and I have carried out an extensive study of a college student, AH, who has a remarkable deficit in visual perception. When AH looks at an object, she sees it clearly and identifies it readily; yet she is often dramatically mistaken about where the object is or how it is oriented. For example, she may reach out to grasp on object that she sees on her left, only to miss it completely because it is actually on her right; or she may see an arrow pointing up when it is really pointing down. Despite her perceptual impairment, AH functions surprisingly well in daily and professional life, in part because of sophisticated adaptations to the impairment. Results from the study of AH support a number of interesting conclusions about how we perceive the world. In Visual Reflections (McCloskey, 2009) I develop conclusions about how the brain codes the locations and orientations of objects, how the brain’s visual system is organized, and what levels of the system contribute to conscious awareness.
Perceiving the orientation of objects is important for interacting with the world, yet little is known about the mental representation or processing of object orientation information. The tendency of humans and other species to confuse mirror images provides a potential clue. However, the appropriate characterization of this phenomenon is not entirely clear, in part because the stimuli used in most previous studies were not adequate for distinguishing various forms of mirror-image and non-mirror-image error. In a recent study Emma Gregory and I explored the nature of mirror-image confusion and what the phenomenon can reveal about object orientation representations. We conducted several experiments in which adult participants reported the orientations of pictures (Gregory & McCloskey, 2010). In all of the experiments mirror-reflection errors were more frequent than other orientation errors. However, whereas mirror-image confusion has previously been described as a tendency to confuse stimuli that are related by reflection across an extrinsic (usually vertical) axis, the vast majority of mirror-image errors in our experiments were reflections across an object axis. This finding calls into question several hypotheses proposed to explain mirror-image confusion. We proposed a coordinate-system orientation representation (COR) hypothesis that can account for the results (McCloskey, Valtonen, & Sherman, 2006). COR assumes that orientation representations map an object-centered reference frame onto a reference frame extrinsic to the object, with this mapping specified by several parameters. According to COR, mirror-image confusions and other orientation errors arise from failures in representing or processing specific parameters. Considered in light of COR, our results suggest that orientation representations are compositional, and that object-centered reference frames play a central role in orientation representation.
In other studies my colleagues and I have explored, and are exploring, object orientation representations in young children, and in individuals with acquired and developmental deficits in visual-spatial cognition.
In addition, we are employing new stimulus categories as well as neuroimaging methods in order to shed further light on orientation representation. Projects in this line of research include:
1) Representing Scene Orientation: Testing the Scope of the COR Hypothesis.
Objects are not the only phenomena for which orientation information must be represented. Is the COR hypothesis a domain-general description of the representation of orientation, or is it category-specific, describing the orientation representation for objects but not for other categories of stimuli? To address this question, we have designed a set of experiments using scenes (aerial views in particular) as stimuli. On the assumption that a path across a scene defines the scene’s primary axis, the COR hypothesis allows for a description of scene orientation in the same terms as objects; using these stimuli in the same experimental designs used for objects provides an opportunity to test whether COR captures the representation of orientation in this new domain. Preliminary studies with aerial views have revealed error patterns for scenes that are importantly similar to those found with objects: notably, confusion between “path-axis” reflections is prominent, whereas reflections about an external axis are rare. However, there are also differences between the error patterns for scenes and objects that are worthy of study in their own right. Further experiments continue to probe these similarities and differences to shed light on how orientation information is represented.
2) Neural Representation of Orientation Information.
Object-specialized cortical regions within the lateral occipital complex (LOC) respond to objects in a way that is sensitive to their position (Sayres & Grill-Spector, 2008) and shape (Kim et al, 2009). Similar properties have been found in scene-selective regions (Dilks et al., 2011). However, the particular transformations in position and location that these regions are sensitive to is still very much under investigation. The COR hypothesis provides one possible guide for delineating and interpreting the spatial transformations that elicit responses in these areas. For instance, our behavioral findings suggest that object-axis reflections are more confusable than other transformations. Does this mirror-image confusion at the behavioral level have a neural correlate, such that certain object or scene-selective regions fail to distinguish between object-axis reflections? Might different reflection relationships (extrinsic-axis vs. object-axis) be encoded in different neural regions? To investigate these questions, we are employing fMRI techniques to understand the neural basis of object and scene perception and identification. In so doing we hope to shed light not only on the representation of orientation information, but also on the exact functions of the brain regions under investigation.
Dilks, D. D., Julian, J. B., Kubilius, J., Spelke, E. S., & Kanwisher, N. (2011). Mirror-image sensitivity and invariance in object and scene processing pathways. Journal of Neuroscience, 31, 11305–11312.
Kim, J. G., Biederman, I., Lescroart, M. D., & Hayworth, K. J. (2009). Adaptation to objects in the lateral occipital complex (LOC): shape or semantics? Vision Research, 49, 2297–2305.
Sayres, R., Grill-Spector, K. (2008). Relating retinotopic and object-selective responses in human lateral occipital cortex. Journal of Neurophysiology, 100, 249-267.
McCloskey, M. (2009). Visual reflections: A perceptual deficit and its implications. New York: Oxford University Press.
Gregory, E., & McCloskey, M. (2010). Mirror-image confusions: Implications for representation and processing of object orientation. Cognition, 116, 110-129.
Valtonen, J., Dilks, D. D., & McCloskey, M. (2008). Cognitive representation of orientation: A case study. Cortex, 44, 1171-1181.
McCloskey, M., Valtonen, J., & Sherman, J. (2006). Representing orientation: A coordinate-system hypothesis, and evidence from developmental deficits. Cognitive Neuropsychology, 23, 680-713.