Multi-Search of Video Segments Indexed by Time-Aligned Annotations of Video Content

This paper describes an approach to the design and implementation of a video content management system that enables the efficient indexing and storing, and search and browsing of video segments defined by annotation of video content attributes. The annotation values represent the durations of independently indexed and potentially overlapping attributes of the video content, such as camera motion, shot distance and face count, and associated information, such as spoken words in the soundtrack. These values define segments of video of any granularity form entire programs down to arbitrarily short segments. Boolean searches on the multiple indexed criteria can be performed, and temporal intervals corresponding to video segments can be computed that reflect the union and intersection of search results meeting these Boolean conditions. We propose a relational database-based model for storing indexed attributes which is flexible and because it does not index or store any fixed temporal segmentation of the video. Rather annotation values can be stored in any way appropriate for an open-ended set of annotation methods for an open-ended set of annotation methods representing attributes of video content. The system is part of a project to exploit a set of automatic annotation methods for the visual attributes of video content, and to develop a component of a complete studio for broadcasting digital HDTV (High Definition Television).