Refers to a number of techniques that result in images that show which areas of the brain are active.
-One of these techniques, positron emission tomography (PET), was introduced in the mid-1970s. A Person is injected with a low dose of a radioactive tracer. The tracer enters the bloodstream and indicates the volume of blood flow. The basic principle behind the PET scan is that the parts of the brain that are active will require more "fuel" from the blood than other, less active parts of the brain. Thus, changes in the activity of the brain are accompanied by changes in blood flow.
Functional Magnetic Resonance Imaging (fMRI). Like PET, fMRI is based on the measurement of blood flow. Because hemoglobin, which carries oxygen in the blood, contains an iron molecule and therefore has magnetic properties, presenting a magnetic field to the brain causes the hemoglobin molecules to line up like tiny magnets. fMRI indicates the presence of brain activity because the hemoglobin molecules in areas of high brain activity lose some of the oxygen they are transporting. This makes the hemoglobin more magnetic, so these molecules respond more strongly to the magnetic field. The fMRI apparatus determines the relative activity of various areas of the brain by detecting changes in the magnetic response of the hemoglobin that occurs when a person perceives a stimulus or engages in a specific behavior. Because fMRI doesn't require radioactive tracers and because it is more precise, this technique has become the main method for localizing brain activity in humans.
Hubel and Wiesel (1965) carried out a series of experiments in which they recorded from neurons they encountered as they lowered electrodes into the cortex. When they positioned an electrode perpendicular to the surface of a cat's cortex, they found that every neuron they encountered had its receptive field at about the same location on the retina.
-Figure 4.6a, shows four neurons along the electrode track, and Figure 4.6b, which shows that these neurons' receptive fields are all located at about the same place on the retina. From this result, Hubel and Wiesel concluded that the striate cortex is organized into location columns that are perpendicular to the surface of the cortex, so that all of the neurons within a location column have their receptive fields at the same location on the retina.
As Hubel and Wiesel lowered their electrodes along perpendicular tracks, they noted not only that the neurons along this track had receptive fields with the same location on the retina, but that these neurons all preferred stimuli with the same orientation. Thus, all cells encountered along the electrode track at A in Figure 4.7 fired the most to horizontal lines, whereas all those along electrode track B fired the most to lines oriented at about 45 degrees. Based on this result, Hubel and Wiesel concluded that the cortex is organized into orientation columns, with each column containing cells that respond best to a particular orientation.
-Hubel and Wiesel also showed that adjacent orientation columns have cells with slightly different preferred orientations. When they moved an electrode through the cortex obliquely (not perpendicular to the surface), so that the electrode cut across orientation columns, they found that the neurons' preferred orientations changed in an orderly fashion, so a column of cells that respond best to 90 degrees is right next to the column of cells that respond best to 85 degrees (Figure 4.8). Hubel and Wiesel also found that as they moved their electrode 1 millimeter across the cortex, their electrode passed through orientation columns that represented the entire range of orientations. Interestingly enough, this 1-mm dimension is the size of one location column.
Determining how the millions of neurons in the cortex respond when we look at a scene such as the one in Figure 4.10a is an ambitious undertaking. We will simplify the task by focusing on one small part of the scene—the tree trunk in Figure 4.10b. We focus specifically on the part of the trunk shown passing through the three circles, A, B, and C.
-Figure 4.11a shows how the image of this part of the tree trunk is imaged on the retina. Each of the circles represents the area served by a location column. Figure 4.11b shows the location columns in the cortex. Remember that each of these location columns contains a complete set of orientation columns (Figure 4.9). This means that the vertical tree trunk will activate the 90-degree orientation columns in each location column, as indicated by the orange areas in each column.
Thus, the continuous tree trunk is represented by the firing of neurons in a number of separated columns in the cortex. Although it may be a bit surprising that the tree is represented by separate columns in the cortex, it simply confirms a property of our perceptual system that we mentioned earlier: The cortical representation of a stimulus does not have to resemble the stimulus; it just has to contain information that represents the stimulus. The representation of the tree in the visual cortex is contained in the firings of neurons in separate cortical columns. At some point in the cortex, the information in these separated columns must be combined to create our perception of the tree.
Before leaving our description of how objects are represented by feature detectors, let's return to our scene (Figure 4.12). Each circle or ellipse in the scene represents an area that sends information to one location column. Working together, these columns cover the entire visual field, an effect called tiling. Just as a wall can be covered by adjacent tiles, the visual field is served by adjacent (and often overlapping) location columns (Nassi & Callaway, 2009). (Does this sound familiar? Remember the "football analogy" for optic nerve fiber receptive fields in Chapter 3, in which each spectator was observing a small area of the field. In that example, the spectators were tiling the football field.)
The idea that any scene is represented by activity in many location columns means that a scene is represented in the striate cortex by an amazingly complex pattern of firing. Just imagine the process we described for the three small areas
on the tree trunk multiplied by hundreds or thousands. Of course, this representation in the striate cortex is only the first step in representing the tree. As we will now see, signals from the striate cortex travel to a number of other places in the cortex for further processing.
Ungerleider and Mishkin (1982) used a technique called ablation (also called lesioning). Ablation refers to the destruction or removal of tissue in the nervous system.
Ungerleider and Mishkin presented monkeys with two tasks: (1) an object discrimination problem and (2) a landmark discrimination problem. In the object discrimination problem, a monkey was shown one object, such as a rectangular solid, and was then presented with a two-choice task like the one shown in Figure 4.13a, which included the "target" object (the rectangular solid) and another stimulus, such as the triangular solid. If the monkey pushed aside the target object, it received the food reward that was hidden in a well under the object. The landmark discrimination problem is shown in Figure 4.13b. Here, the monkey's task was to remove the cover of the food well that was closest to the tall cylinder.
In the ablation part of the experiment, part of the temporal lobe was removed in some monkeys. After ablation, behavioral testing showed that the object discrimination problem was very difficult for monkeys with their temporal lobes removed. This result indicates that the pathway that reaches the temporal lobes is responsible for determining an object's identity. Ungerleider and Mishkin therefore called the pathway leading from the striate cortex to the temporal lobe the what pathway.
-Other monkeys had their parietal lobes removed, and they had difficulty solving the landmark discrimination problem. This result indicates that the pathway that leads to the parietal lobe is responsible for determining an object's location. Ungerleider and Mishkin therefore called the pathway leading from the striate cortex to the parietal lobe the where pathway
The discovery of two pathways in the cortex—one for identifying objects (what) and one for locating objects (where)—led some researchers to look back at the retina and LGN. Using the techniques of both recording from neurons and ablation, they found that properties of the ventral and dorsal streams are established by two different types of ganglion cells in the retina, which transmit signals to different layers of the LGN. Thus, the cortical ventral and dorsal streams can actually be traced back to the retina and LGN.
Although there is good evidence that the ventral and dorsal pathways serve different functions, it is important to note that (1) the pathways are not totally separated, but have connections between them; and (2) signals flow not only "up" the pathway toward the parietal and temporal lobes, but "back" as well. It makes sense that there would be communication between the pathways because in our everyday behavior we need to both identify and locate objects, and we routinely coordinate these two activities every time we identify something (for example, a pen) and take action with regard to it (picking up the pen and writing with it). Thus, there are two distinct pathways, but some information is shared between them. The "backward" flow of information, called feedback, provides information from higher centers that can influence the signals flowing into the system. This feedback is one of the mechanisms behind top-down processing, introduced in Chapter 1.
One of the basic principles of neuropsychology is that we can understand the effects of brain damage by determining double dissociations, which involve two people: In one person, dam- age to one area of the brain causes function A to be absent while function B is present; in the other person, damage to another area of the brain causes function B to be absent while function A is present.
-Ungerleider and Mishkin's monkeys provide an example of a double dissociation. The monkey with damage to the temporal lobe was unable to discriminate objects (function A) but had the ability to solve the landmark problem (function B). The monkey with damage to the parietal lobe was unable to solve the landmark problem (function B) but was able to discriminate objects (function A). These two findings, taken together, are an example of a double dissociation. The fact that object discrimination and the landmark task can be disrupted separately and in opposite ways means that these two functions operate independently of one another.
An example of a double dissociation in humans is provided by two hypothetical patients. Alice, who has suffered damage to her temporal lobe, has difficulty naming objects but has no trouble in- dicating where they are located (Table 4.1a). Bert, who has parietal lobe damage, has the opposite problem—he can identify objects but can't tell exactly where they are located (Table 4.1b). The cases of Alice and Bert, taken together, represent a double dis- sociation and enable us to conclude that recognizing objects and locating objects operate independently of each other.
The method of determining dissociations was used by Milner and Goodale (1995) to study D.F., a 34-year-old woman who suffered damage to her ventral pathway from carbon monoxide poisoning caused by a gas leak in her home.
-One result of the brain damage was that D.F. was not able to match the orientation of a card held in her hand to different orientations of a slot. This is shown in the left circle in Figure 4.16a. Each line in the circle indicates the orientation to which D.F. adjusted the card. Perfect matching performance would be indicated by a vertical line for each trial, but D.F.'s responses are widely scattered. The right circle shows the accurate performance of the normal controls.
Because D.F. had trouble orienting a card to match the orientation of the slot, it would seem reasonable that she would also have trouble placing the card through the slot, because to do this she would have to turn the card so that it was lined up with the slot. But when D.F. was asked to "mail" the card through the slot, she could do it! Even though D.F. could not turn the card to match the slot's orientation, once she started moving the card toward the slot, she was able to rotate it to match the orientation of the slot (Figure 4.16b). Thus, D.F. performed poorly in the static orientation-matching task but did well as soon as action was involved (Murphy et al., 1996). Milner and Goodale interpreted D.F.'s behavior as showing that there is one mechanism for judging orientation and another for coordinating vision and action. These results for D.F. demonstrate a double dissociation when compared with other patients whose symptoms are the opposite of D.F.'s, and such people do, in fact, exist. These people can judge visual orientation, but they can't accom- plish the task that combines vision and action. As we would expect, whereas D.F.'s ventral stream is damaged, these other people have damage to their dorsal streams.
Based on these results, Milner and Goodale suggested that the ventral pathway should still be called the what pathway, as Ungerleider and Mishkin suggested, but that a better description of the dorsal pathway would be the how pathway, or the action pathway, because it determines how a person carries out an action. As sometimes occurs in science, not everyone uses the same terms. Thus, some researchers call the dorsal stream the where pathway, and some call it the how or action pathway.
Cases like that of D.F., in which one stream is damaged, reveal the existence of these two streams. But what about people without damaged brains?
Figure 4.17a shows the stimulus used by Tzvi Ganel and coworkers (2008) in an experiment designed to demonstrate a separation of perception and action in non-brain-damaged subjects. This stimulus creates a visual illusion: Line 1 is actually longer than line 2 (see Figure 4.17b), but line 2 appears longer.
-Presented subjects with two tasks: (1) a length estimation task in which they were asked to indicate how they perceived the lines' length by spreading their thumb and index finger, as shown in Figure 4.17c; and (2) a grasping task in which they were asked to reach toward the lines and grasp each line by its ends. Sensors on the subjects' fingers measured the separation between the fingers as the subjects grasped the lines. These two tasks were chosen because they depend on different processing streams. The length estimation task involves the ventral or what stream. The grasping task involves the dorsal or how stream.
The results of this experiment, shown in Figure 4.17d, indicate that in the length estimation task, subjects judged line 1 (the longer line) as looking shorter than line 2, but in the grasping task, they separated their fingers farther apart for line 1. Thus, the illusion works for perception (the length estimation task), but not for action (the grasping task). These results support the idea that perception and action are served by different mechanisms. An idea that originated with observations of patients with brain damage is supported by the performance of observers without brain damage.
Edmund Rolls and Martin Tovee (1995) measured the response of neurons in the monkey's inferotemporal (IT) cortex. When they presented pictures of faces and pictures of nonface stimuli (mostly landscapes and food), they found many neurons that responded best to faces.
Figure 4.19 shows the results for a neuron that responded briskly to faces but hardly at all to other types of stimuli.
-What is particularly significant about "face neurons" is that there are areas in the monkey temporal lobe that are particularly rich in these neurons. Doris Tsao and coworkers (2006) presented 96 images of faces, bodies, fruits, gadgets, hands, and scrambled patterns to two monkeys while recording from cortical neurons inside this face area. They classified neurons as "face selective" if they responded at least twice as strongly to faces as to nonfaces. Using this criterion, they found that 97 percent of the cells were face selective. The high level of face selectivity within this area is illustrated in Figure 4.20, which shows the average response for both monkeys to each of the 96 objects. The response to the 16 faces, on the left, is far greater than the response to any of the other objects.
We have seen how neural processing that involves the mechanisms of convergence, excitation, and inhibition can create neurons that respond best to small spots of light. The same mechanisms are presumably involved in creating neurons that respond to more complex stimuli. Of course, the neural circuits involved in creating a "face-detecting" neuron must be extremely complex. However, the potential for this complexity is there, because there are a hundred billion (1011) neurons and each neuron in the cortex receives inputs from an average of 1,000 other neurons. Based on these numbers, it has been estimated that there are several hundred trillion synaptic connections between neurons in the brain. When we consider the vast complexity of the neural interconnections that must be involved in creating a neuron that responds best to faces, it is easy to agree with the description of the brain by William James, professor of psychology at Harvard and author of one of the first psychology textbooks (James, 1890/1981), as "the most mysterious thing in the universe." Brain imaging has been used to identify areas of the human brain that contain neurons that respond best to faces, and others that respond best to pictures of scenes and human bodies. In one of these experiments, Nancy Kanwisher and coworkers (1997) used fMRI to determine brain activity in response to pictures of faces and other objects, such as scrambled faces, household objects, houses, and hands. When they subtracted the response to the other objects from the response to the faces, Kanwisher and coworkers found that activity remained in an area they called the fusiform face area (FFA), which is located in the fusiform gyrus on the underside of the brain directly below the IT cortex (see Figure 3.35b). This area is roughly equivalent to the face areas in the temporal cortex of the monkey. Kanwisher's results, plus the results of many other experiments, have shown that the FFA is specialized to respond to faces.
-Additional evidence of an area specialized for the perception of faces is that damage to the temporal lobe causes prosopagnosia—difficulty recognizing the faces of familiar people. Even very familiar faces are affected, so people with prosopagnosia may not be able to recognize close friends or family members—or even their own reflection in the mirror— although they can easily identify such people as soon as they hear them speak.
In addition to the FFA, which contains neurons that are activated by faces, two other specialized areas in the temporal cortex have been identified. The parahippocampal place area (PPA) is activated by pictures depicting indoor and outdoor scenes. The other specialized area, the extrastriate body area (EBA), is activated by pictures of bodies and parts of bodies (but not by faces).
The existence of neurons that are specialized to respond to faces, places, and bodies brings us closer to being able to explain how perception is based on the firing of neurons. It is likely that our perception of faces, landmarks, and people's bodies depends on specifically tuned neurons in areas such as the FFA, PPA, and EBA. But it is also important to recognize that even though stimuli like faces and buildings activate specific areas of the brain, these stimuli also activate other areas of the brain as well. This is illustrated in Figure 4.22, which shows the results of an fMRI experiment on humans.
Figure 4.22a shows that pictures of houses, faces, and chairs cause maximum activation in three separate areas in the IT cortex. However, each type of stimulus also causes substantial activity within the other areas, as shown in the three panels limited to just these areas. Thus, objects such as faces may cause a large focus of activity in an area specialized for faces, such as the FFA, but they also cause additional activity that is distributed over a wide area of the cortex.
We can summarize what we know about organization in the visual system by noting that the visual system is organized both spatially and functionally. The spatial map is retinotopic, which means that points on the LGN or cortex correspond to specific points on the retina or in a scene. But spatial organization becomes weaker as we move to higher cortical areas, because in areas such as IT cortex, neurons have very large receptive fields that extend over large areas of the retina and visual field. Most of the face neurons respond when the face is imaged on the fovea, which makes sense, because when we want to identify a face we usually look directly at it.
The visual system is organized functionally, with different streams for what and where/how and with specific cortical areas that are rich in neurons that respond to specific types of stimuli such as faces, places, and bodies. It is no coincidence that the stimuli that have specific areas in the brain are ones we see all the time (faces and bodies) and that are important for helping us find our way through the environment (place neurons).
From our story so far, it might be tempting to say that the IT cortex, with its neurons for faces and other complex objects, is "the end of the line" for processing information about objects. As we will now see, this conclusion may be partially correct, but signals from the IT cortex also continue on to other structures that may be involved in remembering objects.
Some of the signals leaving the IT cortex reach structures in the medial temporal lobe (MTL), such as the parahippocampal cortex, the entorhinal cortex, and the hippocampus (Figure 4.23a). These MTL structures are extremely important for memory. The classic demonstration of the importance of one of the structures in the MTL, the hippocampus, is the case of H.M., who had his hippocampus on both sides of his brain removed in an attempt to eliminate epileptic seizures that had not responded to other treatments.
The operation eliminated H.M.'s seizures, but it also eliminated his ability to store experiences in his memory. Thus, when H.M. experienced something, such as a visit from his doctor, he was unable to remember the experience, so the next time the doctor appeared, H.M. had no memory of having seen him. H.M.'s unfortunate situation occurred because in 1953, the surgeons did not realize that the hippocampus is crucial for the formation of long-term memories. Once they realized the devastating effects of removing the hippocampus on both sides of the brain, H.M.'s operation was never repeated.
The connection between the hippocampus and vision was demonstrated in experiments by R. Quian Quiroga and coworkers. These experiments showed that there are neurons in the hippocampus that respond to faces of specific people, like Steve Carell (see Figure 3.40), and also to specific structures such as the Eiffel Tower or the Sydney Opera House.
Quiroga recorded from eight patients with epilepsy who, in preparation for surgery, had electrodes implanted in their hippocampus or other areas in the medial temporal lobe to help localize precisely where their seizures originated. Patients saw a number of different views of specific individuals and objects plus pictures of other things, such as faces, buildings, and animals. Not surprisingly, a number of neurons responded to some of these stimuli. What was surprising, however, was that some neurons responded to a number of different views of just one person or building or to a number of ways of representing that person or building. For example, one neuron responded to all pictures of the actress Jennifer Aniston, but did not respond to faces of other famous people, non-famous people, landmarks, animals, or other objects. As we noted in Chapter 3, another neuron responded to pictures of actor Steve Carell. Still another neuron responded to photographs of Halle Berry, to drawings of her, to pictures of her dressed as Catwoman from Batman, and also to the words "Halle Berry" (Figure 4.23b).
According to Quiroga, the MTL—and especially the hippocampus—is not responsible for recognizing objects. Patient H.M. for example, who had no hippocampus, could still recognize objects. He just couldn't remember them later. Thus, just because a hippocampus neuron responds to a visual stimulus doesn't mean it is responsible for seeing. What it is responsible for is remembering.
The possible role of these neurons in memory is supported by the way they respond to many different views of the stimulus, different modes of depiction, and even words signifying the stimulus. These neurons are not responding to visual features of the pictures, but to concepts—"Jennifer Aniston," "Halle Berry," "Sydney Opera House"—that the stimuli represent. Thus, the fact that the neuron that responded to Jennifer Aniston also responded to Lisa Kudrow was not a coincidence, because both appeared on the Friends TV series. The response of these MTL neurons to visual stimuli appears to depend, therefore, on a particular person's past experiences.
We have seen that the brain is organized based on function, with neurons specialized to respond to faces, places, and bodies located in specific areas of the brain. Consider the role of experience in creating these specialized neurons. There is evidence, that some perceptual capacities, such as the ability to perceive movement, light-dark contrasts, faces, depth, tastes, and smells, are present at or near birth, although not at adult levels. Other capacities, such as color perception, depth that can be seen with one eye, and visual attention, emerge slightly later, also not at adult levels. Over time, these capacities improve—some rapidly, such as visual acuity, which reaches near adult levels by 9 months of age, and some over a longer time, such as recognizing faces, which continues developing into adolescence.
-What causes this improvement over time? Biological maturation is clearly involved, as we saw when we described the connection between improvement of visual acuity and the development of the rod and cone receptors. On a longer time scale, there is evidence that some aspects of face recognition depend on the emergence of the fusiform face area (FFA), which is not fully developed until adolescence.
In addition to biological maturation, experience in perceiving the environment also plays a role in perceptual development. One line of evidence supporting the role of experience is the research on experience-dependent plasticity that we described in Chapter 3. Blakemore and Cooper's experiments, in which they reared kittens in striped tubes, showed that these kittens' visual systems were shaped by the environment in which they were raised, so kittens reared seeing only vertical stripes had neurons that responded only to vertical or near vertical orientations.
Humans aren't usually reared in deprived environments, but we do grow up in an environment in which many features occur regularly, and these repeating features of the environment can influence how our visual system develops and, therefore, how we perceive. One example of this, which we described in Chapter 1, is the finding that people perceive horizontal and vertical orientations more easily than other orientations, called the oblique effect. There is evidence that the oblique effect occurs because there are more cortical neurons that respond to horizontal and vertical orientations, and it is no coincidence that horizontals and verticals occur more frequently in the environment than slanted orientations.
The fact that experience with the environment can shape the nervous system is the basis of the expertise hypothesis, which proposes that our proficiency in perceiving certain things can be explained by changes in the brain caused by long exposure, practice, or training. Isabel Gauthier and coworkers (1999) demonstrated an expertise effect by using fMRI to determine the level of activity in the fusiform face area (FFA) in response to faces and to objects called Greebles—families of computer-generated "beings" that all have the same basic configuration but differ in the shapes of their parts (Figure 4.25a). Initially, the observers were shown both human faces and Greebles. The results for this part of the experiment, shown by the left pair of bars in Figure 4.25b, indicate that the FFA neurons responded poorly to the Greebles but well to the faces.
The participants were then trained in "Greeble recognition" for 7 hours over a 4-day period. After the training sessions, participants had become "Greeble experts," as indicated by their ability to rapidly identify many different Greebles by the names they had learned during the training. The right pair of bars in Figure 4.25b shows how becoming a Greeble expert affected the neural response in the participants' FFA. After the training, the FFA neurons responded about as well to Greebles as to faces.
This result shows that the FFA area of the cortex responds not just to faces but to other complex objects as well, and that the objects that the neurons respond to can be established by experience with those objects. In fact, Gauthier has also shown that neurons in the FFA of people who are experts in recognizing cars or birds respond well not only to human faces but to cars (for the car experts) and to birds (for the bird experts). Recently, another study showed that viewing the positions of chess pieces on a chess board causes a larger activation of the FFA in chess experts than in non-experts. Results such as these have led many researchers to suggest that that the reason the FFA responds well to faces is because we are all "face experts."
It is important to note that although there is good evidence that experience can influence the types of stimuli to which a neuron responds, the role of experience in establishing the FFA as a module for faces is controversial. Some researchers agree with Gauthier that experience is important for establishing the FFA as a module for faces; others argue that the FFA's role as a face area does not depend on experience.
Whatever the outcome of this ongoing debate about the FFA, there is no question that properties of neurons are influenced by our experience with stimuli in the environment. This experience, which "tunes" our perceptual system to respond best to what is usually present in the environment, is likely to play a role in determining the improvements in perception that occur from infancy into adulthood.