The integration of cinema and science is at the foundation of Vanderbilt University professor Daniel Levin’s research. Levin, a professor of psychology and human development at Peabody College of education and human development, teaches Film and Psychology in addition to other classes that examine how cognitive science and the arts collide.
Research in Levin’s lab focuses on the interface between concepts and visual perception. He and his students have explored how people perceive the sequence of natural visual events and how they represent space while viewing films.
The group’s discoveries resulted in newly published research in the journal Psychology of Aesthetics, Creativity, and the Arts. Levin recently spoke to MyVU about his new research article, “Perceiving versus scrutinizing: Viewers do not default to awareness of small spatiotemporal inconsistencies in movie edits,” and the unexpected findings that may change perceptions about how people view movie scenes.
What did you explore in your latest research?
At some level, the question is simple: How do you edit a movie scene where you cut from one shot that shows the beginning of an action to a second shot that shows the end of that same action? Do you start the second shot exactly where the first left off? Or maybe you should overlap a little, allowing the second shot to repeat a few frames of the action that was already shown in the first shot. Or maybe you skip forward just a bit—allowing the second shot to start a few frames after the first left off.
The interesting thing about this question is that everyone disagrees about its answer. If you ask film editors, some will say that an exact match is best, some say that a small overlap is best and some say that skipping ahead is best. Even the scientists disagree. One study found that viewers seemed to like a small jump forward best. More recently, another study found the opposite. Viewers seemed to choose a three-frame overlap as the best one. The researchers who observed this preference hypothesized that the brain needs a moment to “catch up” on its processing after the disruption of an edit. So, repeating a little of the action would allow for that additional time and produce the “smoothest” edit because the action is just where the delayed brain expects it to be.
Does this need to “catch up” mean that movie edits are an “unnatural” disruption to our normal process of seeing? What did you think about this idea?
To be honest, we were skeptical. First, people blink all the time, producing disruptions that are in some ways like movie edits, and the world sure doesn’t wait around for a moment while we recover from blinking. Second, there is plenty of research demonstrating that our perceptual system can extrapolate forward when something disappears. These extrapolations could easily compensate for a moment of perceptual disconnect caused by a movie edit.
I was discussing this with the students in my film and psychology class, and we decided that some of the small incidental features of the movies used to demonstrate the overlap preference caused the result. Take away those little features, and you’d eliminate the result—viewers would prefer the straight cut.
We made some movies without the incidental features, set up the experiment and had a bunch of participants watch our movies.
Then we got the results and realized our experiment was a complete failure.
There wasn’t a preference for an overlap, there wasn’t a preference for a straight cut. There wasn’t even a preference for skipping frames. There wasn’t preference for anything. The graph relating how “smooth” viewers thought the edit was to the amount of overlap or skipped frames was dead flat from 12 frames overlap to skipping 12 frames.
My first thought was that there must be some mistake—there MUST be some effect of altering the temporal match between the beginning of an action and the end of that same action. So, one of the students, Josh Little, and I remade the movies and reran the experiment for Josh’s summer project. Same thing: nothing.
Even after two tries, we still didn’t really believe that all those different movies looked the same, so we ran the whole thing again. We also decided to ask more questions: How smooth did the movie look? How fast did it appear to go? Was there anything missing? Was there a gap or overlap in the edit? The results: nothing, nothing, nothing and nothing.
So, although previous researchers could not agree whether overlaps or skipped frames look better, the one thing they did agree on was that a few frames here or there were critical to making a good match-action edit. We went way beyond that, and nobody saw the difference.
Why do you believe no one saw the difference in the film edits? And why is your new finding significant?
One big difference between the previous experiments and ours is that in the previous experiments, participants were able to compare different amounts of overlap in the same brief film over and over. They tried one amount of overlap, then another, repeatedly scrutinizing the differences. In contrast, our participants just watched the movie once and told us what they experienced. At first, we had no clue that would matter so much, but it sure did. When we started repeating the movies and explicitly asking participants to compare and scrutinize them, we started finding they had more specific expectations. But when people just watched the movies once . . . nothing.
We did not set out to document how flexible viewers are when matching the beginnings and ends of actions, so we had to change our minds about how people perceive the actions of highly familiar things such as people. We had thought that the brain intensively processes movements, predicts their trajectories and raises an alert if something does not match. Instead, the brain seems flexible about the details of actions so long as their goals remain consistent, and so long as things stay in the same approximate space. One interesting possibility here is that you may think you are in continuous contact with your perceptual world, but perhaps this sense of continuity is less a matter of what you see . . . and more a question of how your thoughts interact with what you see.
How would you like to build on these findings?
We are doing several projects to follow up on this. In one project we are trying to use more sensitive measures to test for effects of action mismatches. In collaboration with Dan Simons and some of his students at the University of Illinois, we are using a game engine to create highly controlled rendered movies to test whether mismatches might remain unnoticed, but increase the degree to which viewers are aware of visual property changes (such as the color of a shirt an actor is wearing).
We have also documented situations where backwards actions go unnoticed. For example, we showed participants movies where someone is seen using a screwdriver *before* they are seen grabbing it. We have shown that participants have an incredibly difficult time detecting this reversal even though they can clearly see the individual shots.
Finally, we have been doing some work exploring the neuroscience of cinematic event perception. In an upcoming article that was a collaboration between our lab and Bruce McCandliss’ lab at Stanford University, we report the results of an experiment where we made our own movies to show as many events involving math and “theory of mind” (e.g., inferences about the beliefs, desires and goals of characters) as possible. We have demonstrated that there are interesting brain areas where the mechanics of visual attention and the processing of these meaningful domains seem to converge to guide viewers’ thoughts in what we call an “arena” of action. That is, some of the attentional systems in the brain might be well characterized as functioning to track active and passive “characters” in spaces where they cooperate, compete and have beliefs about each other. This is potentially interesting because filmmakers have a lot of insights about how to organize this arena, and the science of attention might benefit from knowing more about this visual-cognitive art form.
How can this work inform our understanding of everyday, real-world perception?
Lots of important issues arise from this research. First, what are the consequences of paying attention to something? For example, if we know someone has looked at something, what follows from that in normal real-world circumstances? Will they actually think about the thing they looked at? Will they remember it? Will they continue thinking about that thing over time?
Researchers have long known that these are different questions, and they have even developed some great behavioral tools and neural markers of these different processes. Now the big job is to figure out the answers to these questions in specific useful circumstances. I think our work with movies provides important information about how people perceive meaningful events as observers. In such circumstances, attention can have surprisingly limited effects on thinking unless viewers have a specific functional goal to think about information. Knowing this, we can focus research on how those goals drive processing away from a very limited processing of visual information to processing that is deeper and that produces more durable results.