View source for Video Content Structuring

== Basic Description ==

With the advances in storage devices, networks and compression techniques, more and more video data has become available to ordinary users. But the efficient access to an unstructured video is a challenging task due to the huge number of frames. Video structuring is proposed aiming at tackling this difficulty and it becomes a necessary step in many video processing techniques, such as video annotation (Wang et al., 2009) and video presentation (Wang et al., 2012). For example, videos can be partitioned into shots and then one or more keyframes can be selected from each shot for indexing. From 2001, video shot detection is established as an evaluation task in TREC Video Retrieval Evaluation (TRECVID) benchmark (http://www-nlpir.nist.gov/projects/trecvid/) by the National Institute of Standard and Technology (NIST). Shots can be grouped into scenes, such that the video content can be structured at higher level. For several video genres, such as news and movies, “story” is also widely applied as the unit for content organization. Shots can also be further segmented into subshots in order to facilitate finer implementations. From each shot or subshot, one or more keyframes can be extracted to represent its content. Therefore, as illustrated in Fig. 1, generally a video can be structured in a hierarchical form as “videos->stories->scenes -> shots->subshots->keyframes”. The definitions of these terms are as follows:

*''Shot'': a shot is an uninterrupted clip recorded by a single camera. It is a physical entity which often forms the building block of video content.
*''Scene'': a scene is defined as a collection of semantically related and temporally adjacent shots, depicting and conveying a high-level concept. A scene usually comprises a series of consecutive shots that are recorded in the same location.
*''Story'': a story is referred to as a clip that captures a continuous action or a series of events and it may be composed of several scenes and shots. Note that the stories lines are usually only clear for rigidly structured videos. Currently, most story identification methods are developed for news videos. Therefore, here only “news story” is considered. A definition of a news story in TRECVID is “a segment of a news broadcast with a coherent news focus which contains at least two independent, declarative clauses”.
*''Subshot'': subshot is a segment within a shot that corresponds to a unique camera motion. A shot can be divided into one or more consecutive subshots according to the movement of the camera.
*''Keyframe'': a keyframe is the frame which best represents the content of a shot or a subshot. According to variation of the content, one or more keyframes can be extracted from each shot or subshot. Keyframes can be used as the entries of the video data for manipulations, such as indexing and browsing.

Therefore, video content structuring may involve the following five techniques: shot detection, scene grouping, story identification, subshot segmentation, and keyframe extraction.

[[Image:VideoContentStructuring_Fig1.jpg|thumb|600px|center|F1|Hierarchical decomposition and representation of video content]]