Automatic Keyframe Selection over TVC database

Resource Type Date
Results 2013-06-10


Our proposed solution for the problem of video summarization through an automatic selection of a single representative keyframe has been evaluated in a dataset from the TVC broadcasting company which consists of 50 videos. Two domain were selected:

  • Morning Show. This domain contained videos of a controlled environment, a TV studio, with a high repetition of similar shots between different assets and the clips were from 22 to 80 minutes. This domain has been split in two categories: Interview and Discussion. 10 videos for each of these categories were included.
  • News. This domain was much more challenging as its videos were much more diverse from both visually and semantically, but they were shorter (from 1 to 4 minutes long).  This domain has been split in three categories: Economy, International and Politics. 10 videos for each of these categories were also included.

The experiments over this dataset were designed to compare four different strategies for the selection of a representative keyframe:

  • Semi-manual: The existing solution at the broadcasting company requires the documentalist to choose among three frames from the video, which have been extracted after an arbitrary time-based sampling.
  • Intra-clip: The keyframe is automatically selected following the intra-clip technique presented in Section III-A of the conference paper.
  • Inter-clip: The keyframe is automatically selected following the inter-clip technique presented in Section III-B of the coference paper.
  • Random: The keyframe is automatically selected after a random sampling of the video clip.

Both dataset and results are available here. There are 10 videos for each category (5 categories from 2 different domains) and each video directory is structured as follows:

  • input keyframes: The images resulting from a uniform downsampling process of the video frames.
  • input video: Video file cannot be upload due to copyright problems. However, the metadata file associated to it has been included.
  • input inter - retrieved videos: Representative keyframes from the videos retrieved by textual similarity.
  • results: Representative keyframe and XML file with the ranking scores obtained for each of the four techniques mentioned above. Ranking scores are only given for intra-clip and inter-clip approaches since they compute it to select the representative keyframe.

Next we illustrate the structure of the data available in the following scheme:

Scheme of the experimental results for our automatic keyframe selection system

People involved

Xavier Giró Associate Professor
Veronica Vilaplana Associate Professor
Carles Ventura PhD Candidate

Related Projects