“throw” their voice. Your brain does all of the work for them.
Take the McGurk effect as another example: when the sound of a syllable (
ba
) is synchronized with a video of lip movements mouthing a different syllable (
ga
), it produces the powerful illusion that you are hearing yet a third syllable (
da
). This results from the dense interconnectivity and loopiness in the brain, which allows voice and lip-movement cues to become combined at an early processing stage. 40
Vision usually dominates over hearing, but a counter example is the illusoryflash effect: when a flashed spot is accompanied by two beeps, it appears to flash twice. 41 This is related to another phenomenon called “auditory driving,” in which the apparent rate of a flickering light is driven faster or slower by an accompanying beeping sound presented at a different rate. 42 Simple illusions like these serve as powerful clues into neural circuitry, telling us that the visual and auditory systems are densely tied in with each other, trying to relate a unified story of events in the world. The assembly line model of vision in introductory textbooks isn’t just misleading, it’s dead wrong.
* * *
So what is the advantage of a loopy brain? First, it permits an organism to transcend stimulus–response behavior, and instead confers the ability to make predictions ahead of actual sensory input. Think about trying to catch a fly ball. If you were merely an assembly line device, you couldn’t do it: there’d be a delay of hundreds of milliseconds from the time light strikes your retina until you could execute a motor command. Your hand would always be reaching for a place where the ball
used
to be. We’re able to catch baseballs only because we have deeply hardwiredinternal models of physics. 43 These internal models generateexpectations about when and where the ball will land given the effects of gravitational acceleration. 44 The parameters of the predictive internal models are trained by lifelong exposure in normal, Earth-bound experience. This way, our brains do not work solely from the latest sensory data, but instead construct predictions about where the ball is about to be.
This is a specific example of the broader concept of internal models of the outside world. The brain internally simulates what will happen if you were to perform some action under specific conditions. Internal models not only play a role in motor acts (such as catching or dodging) but also underlieconscious
perception
. As early as the 1940s, thinkers began to toy with the idea that perception works not by building up bits of captured data, but instead by matching
expectations
to incoming sensory data. 45
As strange as it sounds, this framework was inspired by the observation that our expectations influence what we see. Don’t believe it? Try to discern what’s in the figure on the following page. If your brain doesn’t have a prior expectation about what the blobs mean, you simply see blobs. There has to be a match between your expectations and the incoming data for you to “see” anything.
A demonstration of the role of expectation in perception. These blobs generally have no meaning to a viewer initially, and only after a hint does the image make sense. (Don’t worry if they still look like blobs to you; a hint comes later in the chapter.) From Ahissar and Hochstein, 2004.
One of the earliest examples of this framework came from the neuroscientistDonald MacKay, who in 1956 proposed that thevisual cortex is fundamentally a machine whose job is to generate a model of the world. 46 He suggested that the primary visual cortex constructs an internal model that allows it to anticipate the datastreaming up from the retina (see the appendix for an anatomical guide). The cortex sends itspredictions to the thalamus, which reports on the
difference
between what comes in through the eyes and what was already anticipated. The thalamus sends back to the
Take the McGurk effect as another example: when the sound of a syllable (
ba
) is synchronized with a video of lip movements mouthing a different syllable (
ga
), it produces the powerful illusion that you are hearing yet a third syllable (
da
). This results from the dense interconnectivity and loopiness in the brain, which allows voice and lip-movement cues to become combined at an early processing stage. 40
Vision usually dominates over hearing, but a counter example is the illusoryflash effect: when a flashed spot is accompanied by two beeps, it appears to flash twice. 41 This is related to another phenomenon called “auditory driving,” in which the apparent rate of a flickering light is driven faster or slower by an accompanying beeping sound presented at a different rate. 42 Simple illusions like these serve as powerful clues into neural circuitry, telling us that the visual and auditory systems are densely tied in with each other, trying to relate a unified story of events in the world. The assembly line model of vision in introductory textbooks isn’t just misleading, it’s dead wrong.
* * *
So what is the advantage of a loopy brain? First, it permits an organism to transcend stimulus–response behavior, and instead confers the ability to make predictions ahead of actual sensory input. Think about trying to catch a fly ball. If you were merely an assembly line device, you couldn’t do it: there’d be a delay of hundreds of milliseconds from the time light strikes your retina until you could execute a motor command. Your hand would always be reaching for a place where the ball
used
to be. We’re able to catch baseballs only because we have deeply hardwiredinternal models of physics. 43 These internal models generateexpectations about when and where the ball will land given the effects of gravitational acceleration. 44 The parameters of the predictive internal models are trained by lifelong exposure in normal, Earth-bound experience. This way, our brains do not work solely from the latest sensory data, but instead construct predictions about where the ball is about to be.
This is a specific example of the broader concept of internal models of the outside world. The brain internally simulates what will happen if you were to perform some action under specific conditions. Internal models not only play a role in motor acts (such as catching or dodging) but also underlieconscious
perception
. As early as the 1940s, thinkers began to toy with the idea that perception works not by building up bits of captured data, but instead by matching
expectations
to incoming sensory data. 45
As strange as it sounds, this framework was inspired by the observation that our expectations influence what we see. Don’t believe it? Try to discern what’s in the figure on the following page. If your brain doesn’t have a prior expectation about what the blobs mean, you simply see blobs. There has to be a match between your expectations and the incoming data for you to “see” anything.
A demonstration of the role of expectation in perception. These blobs generally have no meaning to a viewer initially, and only after a hint does the image make sense. (Don’t worry if they still look like blobs to you; a hint comes later in the chapter.) From Ahissar and Hochstein, 2004.
One of the earliest examples of this framework came from the neuroscientistDonald MacKay, who in 1956 proposed that thevisual cortex is fundamentally a machine whose job is to generate a model of the world. 46 He suggested that the primary visual cortex constructs an internal model that allows it to anticipate the datastreaming up from the retina (see the appendix for an anatomical guide). The cortex sends itspredictions to the thalamus, which reports on the
difference
between what comes in through the eyes and what was already anticipated. The thalamus sends back to the
Similar Books
