I was catching up on mail after Thanksgiving vacation and came across a cool article by Paul Marks writing in New Scientist magazine about plot-deconstructing software that could let you make your own episodes of favorite shows.
LOST the plot watching Homeland or Game of Thrones? Wondering when a strange character you’d never seen before on Doctor Who was introduced? You’re not alone: the tremendous choice of programmes on offer on today’s multichannel TV services can make it hard to keep up. But help is at hand, thanks to scene-analysis software that can compile a video sequence summarizing any chosen plot line or character’s appearances in a TV series. Choose a scene, for instance, and the software will assemble a personalized video episode based around it. And in a move screenwriters will doubtless detest, it can also help fans compile customized episodes starring only their favorite actors.
Called StoryVisualizer – or StoViz – the PC-based technology has been made possible by the increasing amount of content that can be stored on digital video recorders, says lead developer Hervé Bredin of the Computer Sciences Laboratory for Mechanics and Engineering Sciences (LIMSI) in Orsay, France. TiVo and Sky Plus, for example, can store up to a week of continuous broadcast TV plus all of the previous episodes in various series. “We think it’s time users had search and browsing tools to use within these digital video collections,” Bredin says.
He worked with a team at the Toulouse Institute for Computer Science Research to make that possible, exploiting the brawn of today’s graphics processing chips. The task required more than just good hardware, though. Identifying key plot points and tracing character arcs are not easy, even for a human. In fact what makes a show compelling is often the way it weaves these elements together in sophisticated ways.
So StoViz’s first task involved “de-interlacing” those themes into individual threads. To find a particular storyline, it used image analysis to seek actors’ faces and background scenery used in certain scenarios, in addition to analysing the audio for key words associated with that story. The software then assembles a group of scenes that its video and audio algorithms have decided are semantically similar, and therefore hopefully represents the same plot line. In the same way, if you are interested in a certain actor, you can choose their face and only their scenes will be compiled into a summary.
The researchers successfully tested StoViz on three TV series with very different formats: the sitcom Malcolm in the Middle, the fantasy drama Game of Thrones and the legal lunacy of Ally McBeal. Now all that remains is to integrate the technology into existing entertainment systems. There are a number of possibilities. The system could easily be activated via a screen menu, says Bredin, and perhaps in the not-too-distant future you might simply say the name of your favorite character to a Siri-like voice interface (see “Coming soon… talking to your TV”).
Marek Barwinski, a vision engineer with imaging-app maker Cortexica in London, says that the “fairly restricted number of characters” in most TV series means the processing won’t be too demanding. “But I wonder if in large video collections it could keep up with soap characters as they age,” he says. The device will be useful in some situations, such as “extreme catch-up situations” after a long holiday, says Simon Parnall of pay-TV systems maker NDS of Staines, UK. But he questions its overall usefulness. “I frankly wonder about the character of a viewer who would wish to use this when up to date with a series,” he says.
I agree with Parnall’s quote about anyone using this who is already up to date with a series, but I was late to Breaking Bad and this tool would have been helpful (not to mention cool). Like so many things, it’s probably not a stand-alone world-changer, but it certainly could be a second-screen extension or a feature on newer smart TVs.
# # #
We build strategies and everything that goes with them.
Some of the largest organizations in the world, including many in the mortgage and finance industries, trust us with the most important aspects of their business. From defining clients’ brands and identities to developing ongoing campaigns in a variety of media, we provide the communications and measurement tools to move them forward. Applying our experience and dedication to the media and the message, bloomfield knoble handles every detail of our clients’ strategic marketing initiatives.