Bruno A. Olshausen, Ph.D.

Project

Computational Theories of Perception and Scene Analysis

The problem of scene analysis in vision concerns the process by which we build representations of surfaces, objects, and their material properties from the raw sensory input provided by the retina. It is an act we perform with relative ease, yet describing how this actually occurs in concrete, computational terms has proven elusive. There is also very little understood about the neural basis of scene analysis. Probably intermediate-level areas in the visual cortex such as V2 and V4 are involved, but how exactly they contribute and what computations they perform is not at all clear. Making progress requires not only new experimental approaches, but also the construction of new theoretical frameworks for interpreting findings and guiding experiments.
In recent years, a number of investigators have proposed that generative models could provide an appropriate theoretical framework for the scene analysis problem. The idea is that the cortex contains an internal model of the environment, and that it represents the causes of images (surfaces, objects, motion, etc.) in terms of this model. There is substantial evidence in support of this idea coming from psychophysics (e. g., the work of Nakayama, Bregman, Kersten, and others), but very little is known in concrete terms about how this could be accomplished by neural circuits in the brain. Over the past ten years I, in addition to other investigators, have been exploring the process of learning and inference in simple generative models that are adapted to the statistics of natural scenes. When cast in neural terms, these models can account for many of the response properties of neurons at lower levels of the visual cortex in mammals (i. e., the receptive fields of so-called "simple cells"). I am now exploring how to extend these models to intermediate levels of vision. The goal is to account not only for the psychophysics, but also to develop neural models that produce concrete, testable hypotheses regarding the nature of representation in intermediate-level areas such as V2 and V4.

Recommended Reading

Olshausen, B. A. and D. J. Field. 2005. "How close are we to understanding V1?" Neural Computation 17: 1665-1699.
Rao, R. P. N., B. A. Olshausen, and M. Lewicki, eds. 2002. Probabilistic models of the brain: perception and neural function. Cambridge, Mass: MIT Press.
Olshausen, B. A. and D. J. Field. 1996. "Emergence of simple-cell receptive field properties by learning a sparse code for natural images." Nature 381: 607-609.

Colloquium, 19.03.2009

The Puzzle of Perception

Every waking moment we see, hear, feel, smell and taste the world around us. It is a process that begins as voltage fluctuations in sensory receptor cells and ultimately, through the processing of millions of neurons in the brain, leads to a unified perceptual experience of the environment. How this is accomplished is one of the deepest mysteries of modern science. In fact, for all practical purposes it can be regarded a miracle. Over the past half century, beginning with the early movements in cybernetics and artificial intelligence and up to the modern day field of machine learning, scientists and engineers have sought to emulate the sensory intelligence of biological systems. However, even building systems that can achieve the navigational abilities of a bee or the pattern recognition abilities of a jumping spider - let alone birds, fish or mammals - still lies far beyond our grasp. The problem is not due to lack of computational resources such as storage capacity or speed. Rather, we are missing something fundamental - we do not understand the underlying principles of perception. From the biological side, while there has been dramatic progress in our understanding of the molecular and cellular mechanisms that govern signal processing and transmission in neurons, these findings have provided little true insight about how perception works. Rather than feeling enlightened, we now find ourselves swimming in a sea of data, an overwhelming table of facts and figures. In many ways then, the field is "pre-Newton" - it is data-rich but theory-poor.

In this talk I will attempt to explain what we know and don't know about the neural processes involved in perception, as well as the state of the art in our ability to emulate perceptual abilities in machines. I will focus especially on the magnitude of the explanatory gap before us, and its implications for the scientific process. How does science - which prides itself on the ability to explain - approach a problem of such magnitude, and which may ultimately exceed the capacity of our own intelligence to comprehend? Scientists by their nature are eager to test hypotheses, or to "tell a story" about how a given set of facts or findings fit together and explain perceptual phenomena. But most of these hypotheses and stories are far too simple minded, and ultimately they turn out to be wrong. Worse yet, they can be misleading and stifling because they encourage one to design experiments and examine data through a narrow lens. I shall argue here that, given the state of our knowledge in neuroscience, we are more often better served by taking an exploratory approach that casts a wide net and seeks to reveal and characterize interesting phenomena rather than carefully designing experiments to test a specific set of hypotheses.

Publications from the Fellows' Library

Olshausen, Bruno A. (Lausanne, 2014)

Scene analysis in the natural environment

Olshausen, Bruno A. (Cambridge, Mass., 2013)

Perception as an inference problem