Floraine Berthouzoz was an intern when she created GapStop, a technology that helps create the perfect sound bite.
Any filmmaker who’s worked on a documentary or testimonial video knows: you can’t always choose your subjects. You can make them look good with the right lighting and camera shot. You can dress them up and put make-up on them, but you’re at their mercy when it comes to getting the perfect sound bite.
Sometimes your subject wanders off topic. Sometimes they stutter. Sometimes they make too frequent use of umms, uhs and you knows. Sometimes they give you the perfect sound bite—but spread across multiple sentences during the course of an interview.
And sometimes, even the best editor can’t piece together the right parts of the story without making heavy, distracting use of jump cuts or cross-fades.
That’s where Floraine Berthouzoz, a research scientist in Adobe’s Creative Technologies Group, comes in. As part of the Sneaks program at Adobe MAX 2014, she demonstrated an amazing technology called GapStop that allows film editors working in Adobe Premiere Pro to create jump cuts so natural and seamless there appears to be no cut in the footage at all.
Gap Stop’s algorithm is impressive. It scans available footage of an interview subject, selects existing frames of the subject’s face, and assembles them into a natural-looking morph cut that bridges the subject’s facial expressions and movements across the gap in footage.
“The idea is that the editor cuts something and they immediately create a morph cut, like the same way you would drop a cross-fade transition into the video, you could drop a morph cut transition into the video,” Floraine explains.
Since it uses frames from elsewhere in the footage, the result is a seamless transition that looks as natural as a pause in between words. “That gives you a transition that is more visually coherent,” she says.
The idea for Gap Stop originates from Floraine’s time as an Intern at Adobe and her collaboration with Senior Research Scientist Wil Li. It’s also inspired by her work as a PhD student in computer science, minoring in journalism at University of California, Berkeley. “I had to edit a lot of videos. Interview footage, but also other footage. And so I got really interested in how we could make the whole process easier.”
Adobe, she says, is a great place to explore those sorts of questions. “I think it’s a fantastic place, especially for people who are really interested in visual computing. Getting so much freedom in the research group, allows us to be really innovative.”
Floraine hopes to apply that innovation to make GapStop even more versatile. Today, it works well with talking head video shot against static backgrounds, but she sees potential for outdoor and moving shots to be edited in similar ways.
“I think it will give [editors] a lot more freedom in their editing choices,” she says, “It will allow them to cut videos, not exactly wherever they want, but pretty much, wherever they want and still obtain high quality results … without compromising the quality of the final edit, which is especially important for interview footage, any kind of talking head videos that you might have.”
Original post published on the Adobe Conversations blog.
