Comcast Technology Solutions is taking aim at operators, programmers and streaming services with VideoAI, a new software-as-a-service product that enables those partners to analyze their video feeds – down to the audio and closed captioning – and build rich metadata that can be used to fuel an array of new applications.

One initial use case for VideoAI is something called "segmentation," which helps content partners build ad breaks with commercials that are more targeted and relevant to the viewer, identify the intro or end of a TV episode to help facilitate series binge-viewing, automate the creation of individual chapters within a show or movie, or quickly generate a highlight reel of a live sporting event. Those apps are driven by the use of a broad base of AI-assisted detectors that, for example, can recognize an actor's face or identify text or audio within the video asset itself.

Figure 1:

VideoAI's workflow
(Source: Comcast Technology Solutions)

"Customers can adapt the AI to automatically analyze their video assets with key on-screen moments, like transitions, hard-cuts or even audio. There are even certain sounds we can tag," said Bart Spriester, VP and GM of the content and streaming providers suite at Comcast Technology Solutions (CTS), a unit of Comcast that provides a mix of video, IP transit and ad-tech services and serves as the syndication arm for Comcast's X1 platform.

While that sort of detection can be done manually, VideoAI strives to automate the process of metadata coding using "trained algorithms" that can reduce the operational costs for content and service provider partners, Spriester adds.

Using a software-as-a-service model, VideoAI is designed to decode the various elements of video (including the accompanying audio and closed captioning data), send it along to the platform's AI-powered detectors, and then shuttle it over to the system's pre-trained algorithms.

"All of this generates what I'd call enhanced metadata for applications to act on and make decisions," Spriester explained.

Comcast developed VideoAI internally. However, aspects of the system's underlying technology can be traced back to the company's 2017 acquisition of Watchwith, a tech company that specialized in deep metadata that, for example, describes what's occurring within the scene of a TV show or movie, and the ability to add in-program interactivity such as poling and voting.

VideoAI has already been deployed at Comcast, NBCUniversal and UK-based Sky. The platform's pre-trained algorithms have been developed based on "millions and millions of hours of Comcast, NBCU and Sky content," Spriester said.

CTS has not announced any commercial deployments for VideoAI outside of the realm of Comcast, but Spriester said the company has already received "pretty strong interest from several customers." CTS, he adds, has already put together some proofs-of-concept with operators and content providers that are sizing ways to use VideoAI to support their streaming operations.

Related posts:

— Jeff Baumgartner, Senior Editor, Light Reading