Is what you think you see really what you see?
A few years ago, University of California, Berkeley researchers managed to do something that had never before been accomplished, except perhaps in science fiction. Using a technology called functional Magnetic Resonance Imaging, or fMRI, to pick up brain activity and a computer modeling program to analyze the patterns of neural impulses, they essentially were able to peer inside people’s brains and view what they were seeing at a particular moment. When the experimental subjects watched a video image of an elephant, an airplane, or actor Steve Martin as Inspector Clouseau in the remake of “The Pink Panther,” the researchers could see a murky, but usually recognizable, reconstruction of the image on their
own screen. “This a major leap toward reconstructing internal imagery,” explained UC Berkeley neuroscientist Jack Gallant, a coauthor of the study published in Current Biology in 2011. "We are opening a window into the movies of our minds."
Amazing as the scientists’ breakthrough was, it was nowhere near as difficult as feats that your own brain routinely pulls off every moment of your waking existence.
Though we have five different senses, almost a third of your brain is devoted to vision. That’s because sight plays a particularly crucial role in helping you to make sense of the reality around us and perform the myriad activities of our daily lives. Your eyes and your brain work together to make sense of a vast, continual stream of visual information. To cope with that demand, the human brain has developed various tricks and shortcuts that enable us to quickly do everything from recognize other people’s faces, to the position
and relative size of the objects around us. Conversely, though, the brain’s reliance on those gimmicks makes it vulnerable to being fooled by optical illusions and magicians’ slight-of-hand tricks. Scientists even have utilized such deceptions to demonstrate how the brain’s visual processing of reality works, and what are its built-in limitations.
Making the Movie Inside Your Head
You rely heavily upon what you see to make sense of reality, and to help you to figure out what to do from moment to moment, whether you’re trying to thread a needle or find where you left your car keys. The task of constructing that visual map of your environment falls to the visual-processing centers of your brain, and it’s a really, really tough job. Researchers at the University of Pennsylvania’s School of Medicine estimate that each of your retinas are capable of transmitting 10 million bits of information every second, which makes it comparable to the Ethernet connection on a typical computer network (10 million to 100 million bits).
That cascade of data is transmitted along a pair of optic nerves, which become intertwined and switch sides along the way to your brain’s visual processing area. (That means that the left half of your field of vision actually is perceived by the
right hemisphere of your brain, and vice-versa.) As that information makes its way to the main visual processing areas in the rear of your brain, the primary and secondary visual cortexes, it passes through other parts of the brain, which also make use of some of the data. In the middle of the brain, for example, part of your hypothalamus taps into some of the visual data about whether the environment around you is dark or brightly lit. It uses that information to adjust to the cycle of day and night, and to synchronize biological rhythms that help you to sleep and to awaken in the morning. Meanwhile, another area of the brain, the pulvinar nucleus in the thalamus, also grabs a little bit of data, which it uses to help stabilize your view of the world as you move, sort of like the anti-shake feature on a fancy video camera.
But it’s the main processing areas, the primary and secondary visual cortexes in the back of the brain, which sift through most of the visual data, making sense of the sea of color,
shape and movement around you. They organize the info and send it along one of two main relays back to the parts of the brain that need it. The dorsal visual pathway, which goes to the brain’s parietal lobe, provides data that helps you to figure out the spatial relationships between our bodies and objects in the environment, so that you can interact with them. If you’re walking across your office, it’s what helps to keep you from banging into the walls or tripping over your
coworker’s trash basket. The ventral visual pathway, in contrast, which goes to the brain’s temporal lobe, contains data that helps you to figure out the identity of objects. It’s what helps you to perceive that the thing on the counter of the break room is a delicious-looking cherry Danish, rather than a stapler.
How the brain organizes, retains, and utilizes all that visual data is even more, well, mind-boggling. In another recent UC Berkeley study, led by neuroscientist Alexander Huth, researchers showed subjects hundreds of hours of video footage of everyday objects and scenes, all the while scanning their brains to see what activity occurred. They discovered that visual information apparently is stored in myriad different locations across the expanse of your brain, where objects are grouped into a tree of nested categories, based upon how they relate to other objects—whether an object is more or less “animal-like” or “man-made,” for
example. Sometimes, an object can be associated with more than one category.
Reality Is Actually a Lot of Guesswork
That mental filing system is critical, because your brain is constantly making assumptions to help you figure out the world around you. It makes educated guesses about everything from the relative location and size of objects in a room, to where the walls end and the floor and the ceiling begin.
A lot of these guesses are based upon cues. For example, you’ve seen an ordinary straight-backed chair countless times, and you have a pretty good idea of how big such a chair usually is. So if you see a person sitting in such a chair, you can judge that person’s size and how far away he or she is, as well. That is, unless perception researchers are trying
to fool you with a specially-designed piece of furniture called a Beuchet chair (not pictured). The latter is actually two separate pieces of furniture. A seat, which is placed a distance from the viewer, and a pair of unattached legs, which are positioned closer. When you view them from a certain angle, your brain’s visual processing, which is accustomed to seeing ordinary chairs, stitches together the two objects into one. But because the seat is far away, a
normal-sized adult who sits in it will look the size of a child, because you mistake their size in your retinal image of them for their actual size.
Another similar trick is the Ames room, which to your eyes looks like a normal room with rectangular walls, windows, and floor tiles. In fact, all of those objects are trapezoids, with two sides that are parallel and two that are not. But because you’re used to making judgments based upon rooms that are regular, your sense of perspective is tricked. A person can walk across an Ames room and seem to change in size drastically. Or two people who are the same height will appear drastically different in proportion to one another.
Seeing Things That Aren’t There
Your brain’s job it to make sense of reality, and it takes that job very seriously. The visual control-room of your brain is so
determined to make a decision about what it is that you’re seeing, in fact, that it will do so, even if it doesn’t have that much good visual information to work with. Again, your brain relies upon educated guesses. Perception researchers can use that tendency to trick your mind into seeing things they want you to see.
A prime example is something called the hollow mask
illusion. Take a Halloween mask and set it so that the concave, or hollowed-out, side of the mask is turned toward you. Then, from a distance, take a look at the mask. You’ll see the same face that you would see if the mask were turned toward you. This happens, even though you’re aware that you’re actually looking at the mask from the concave angle.
"Our top-down processing holds memories, like stock models," Danai Dima, a researcher at Hanover Medical University in Germany, explained to Wired magazine in 2009. "All the models in our head have a face coming out, so whenever we see a face, of course if has to come out."
(Except, that is, if you’re a person with schizophrenia, a mental disorder characterized by hallucinations, delusions, and poor planning. As Dima and colleagues discovered, schizophrenics actually see a hollowed-out object, but their brains don’t connect it with a face.)
The reason that your brain is so eager to see a face in a hollowed-out mask is because of pareidolia—humans' innate tendency to read significance into random or vague stimuli, not just visual but auditory as well. You experience pareidolia when you look at a cloud and think that it looks like an animal or a plant, but the most common manifestation seems to be seeing human faces in objects—whether they are hollowed-out masks, rock formations, or a piece of French toast. The famous astronomer and author Carl Sagan, in his 1995 book "The Demon-Haunted World," hypothesized that this tendency evolved from primitive humans' need to recognize other humans' faces in order to survive.
What You’re Seeing Depends On Where You’re Looking
Back in 1940, surrealist painter Salvador Dali created a painting, Slave Market with the Disappearing Bust of Voltaire. Scan the background, and two women will
mysteriously morph into the face of the 18th century French writer. It’s a particularly trippy example of something called a hybrid image, which can change in character, depending upon how you look at it.
In the 1990s, neuroscientists Aude Oliva and Philippe Schyns at the University of Glasgow began to use such images to study how the brain sees the same object differently at different resolutions—that is, now the fine close-up details of a picture can tell a completely different story than the rougher lines that are visible from farther away.
In the example used in the “Seeing Is Believing” episode, you’re shown a picture that, when you look at it from close up, resembles the famous theoretical physicist Albert Einstein. Step back a bit, so that the image is smaller, and it suddenly appears to morph into an image of glamorous movie star Marilyn Monroe. Which is it? Actually, the image is a hybrid, a blend of both people’s faces. But subjects who
look at it don’t realize this, because at different distances, your eyes zero in on different bits of visual information. From close up, your eyes are great at recognizing fine details—in this case, the wrinkles on Einstein’s face and his bushy moustache. But as you get further away from the image, that ability fades. Instead, your eyes work with what they can perceive—the broader lines and shapes of the image. Thus, the homely scientist is transformed into a goddess of the silver screen.
- The Devil
- 1The Devil
Don't let your visual perception fall prey.
In the “Seeing is Believing” episode of Brain Games, you learn that you rely heavily upon sight to function in the world, and that your brain uses a number of different shortcuts to make sense of the daunting amount of visual information that it’s continually bombarded with. But those perceptual shortcuts also make you vulnerable to being fooled—particularly when you sit down at a poker table, and go up against someone who’s spent years figuring out how to exploit those weaknesses to fleece you. “Although the technical work of card cheats and conjurors differs greatly... the underlying principles are the same,” writes Allan Zola Kronzek, author of 52 Ways to Cheat at Poker. “The deceiver shows the false and hides the real.”
But don’t worry. Kronzek describes some of the cheaters’ most common tricks, and how to look out for them.
1. The Jog Shuffle:
Shuffling is supposed to make the order of the cards unpredictable, which is something that card cheats don’t want. Thus, a wily grifter is likely to be a master of the jog shuffle, in which he actually is protecting the top cards and replacing them on the top of the stack. He does this by drawing off some of the cards and then leaving one card slightly protruding as he puts the bottom cards on top. He then subtly shuffles only the cards that are now on top, leaving the ones beneath the jogged card undisturbed and still in their original order. As he repeats the shuffle, he slips out those undisturbed cards and slides them on top. (The second part actually is more complicated than that, but you get the picture.) One way to spot this maneuver is to watch for a protruding card.
2. The Fake Cut:
According to Kronzek, card cheats who’ve positioned known cards in a deck sometimes try to trick an opponent into not cutting it, by appearing to cut it themselves. One venerable technique for doing this is the “jump,” first documented by French magician Jean Eugène Robert-Houdin in 1861, which exploits your brain’s tendency to tell you that you’ve seen what you expect to see, rather than what actually happened. A cheat will pick up a packet, or handful, of cards from the deck and then set it to the left of the remaining cards, as if he’s then going to follow the standard practice of putting the remaining cards on top. Instead, though, when he picks up the bottom packet with his right hand, he subtly transfers it to his left hand, and then reaches with his right for what originally was the top packet, which he places back on top. To disrupt the card shark, simply ask to make the cut yourself.
3. The Lay Stack:
In this trick, a card cheat will drop out early in a hand and put his cards face down on the table. He’s secretly memorized the card sequence—A-6-K-4-4, for example—and intends to use subterfuge to position the cards on the top of the deck on the next deal. After a fake shuffle and cut, he calls for five-card stud. The result: he knows the value of every face-down card on the table. To thwart this subterfuge, watch carefully what players do with their cards when they fold.