MuViPro - Multicamera Video Processing

Type Start End
National Jan 2011 Aug 2014
Responsible URL
Josep R. Casas

Reference

Multicamera Video Processing exploiting scene information applied to Sports Events, Visual Interaction and 3DTV. Ref. TEC2010-18094, Spanish Ministerio de Ciencia e Innovación, now Ministerio de Economía y Competitividad

Description

The major goal of the current project is to investigate the extension of video processing tools to more generic multicamera settings, and a wider range of video processing applications. This goal is stated through the following objectives:

  • Extension of existing and development of new video processing algorithms for visual analysis and representation considering the multiview and segmented views for multicamera setups and the available knowledge for controlled scenarios. In particular, low-level analysis algorithms including foreground detection and tracking, visual matching and 3D scene representation, high-level analysis algorithms such as human body analysis and the analysis of objects, text, faces and events, and video coding algorithms in multicamera settings for 3D and stereoscopic video signals
  • Focus new target applications such as sport events, 3DTV/Free view-point TV (FTV), and visual interaction where multi-camera setups and ‘a priori‘ knowledge of the scenario can be exploited for analysis and representation tasks. These three applications offer adequate and demanding scenarios to extend the tools developped for controlled environments to a wider set of multiple camera setups and scenarios.

The specific objectives for the extension of video processing algorithms are as follows

  • Low-level analysis algorithms: to improve the performance of available tools for foreground detection and tracking, visual matching and 3D scene representation considering a wide range of camera setups (including segmented and multiview multicamera setups) and study their extension to less controlled environments.
  • High-level analysis algorithms and video coding: to extend the tools for human body analysis and the analysis of objects, text, faces and events in the scene to the requirements of the new application scenarios and to exploit the particular setup of multicamera capture.

The specific objectives for the newly targeted applications have the common goal of evaluating the tools and proving their interest:

  • Sports events: definition of a processing strategy exploiting the knowledge of the scenario and the multicamera setup for the visual analysis and coding of sports footage
  • Visual interaction: increase the robustness of gesture detection for visual interaction either in multiview camera settings or in limited multicamera settings for office or home applications
  • 3DTV/FTV: algorithms for stereoscopic video analysis and coding will be extended to data from a new audiovisual production laboratory, which will be built in 2010 in the Signal Theory and Communications Department to foster research and technology transfer in 3DTV video and Free viewpoint TV applications

Publications

Calderero F, Marqués F. Image Analysis and Understanding Based on Information Theoretical Region Merging Approaches for Segmentation and Cooperative Fusion. In: Handbook of Research on Computational Intelligence for Engineering, Science, and Business. Handbook of Research on Computational Intelligence for Engineering, Science, and Business. IGI Global; 2012. pp. 75-121.
Suau X, Alcoverro M, López-Méndez A, Ruiz-Hidalgo J, Casas J. INTAIRACT: Joint Hand Gesture and Fingertip Classification for Touchless Interaction. In: Computer Vision – ECCV 2012. Vol. 7585. Computer Vision – ECCV 2012. Heidelberg: Springer; 2012. pp. 602-606. (214.4 KB)
Carcel E, Martos M, Giró-i-Nieto X, Marqués F. Rich Internet Application for Semi-automatic Annotation of Semantic Shots on Keyframes. In: Computational Intelligence for Multimedia Understanding. Vol. 7242. Computational Intelligence for Multimedia Understanding. Pisa, Italy: Springer-Verlag; 2012. pp. 172-182. (6.93 MB)
Navarro S, López-Méndez A, Alcoverro M, Casas J. Multi-view Body Tracking with a Detector-Driven Hierarchical Particle Filter. In: Perales F, Fisher R, Moeslund T Lecture Notes in Computer Science: Articulated Motion and Deformable Objects. Vol. 7378. Lecture Notes in Computer Science: Articulated Motion and Deformable Objects. Berlin / Heidelberg: Springer ; 2012. pp. 82-91.
Salvador J. Surface Reconstruction for Multi-View Video Casas J. 2011 . (4.74 MB)
Suau X, Casas J, Ruiz-Hidalgo J. Real-time head and hand tracking based on 2.5D data. In: ICME - 2011 IEEE International Conference on Multimedia and Expo. ICME - 2011 IEEE International Conference on Multimedia and Expo. ; 2011. pp. 1–6. (1.59 MB)
Salvador J, Casas J. A compact 3D representation for multi-view video. In: 2011 International Conference on 3D Imaging. 2011 International Conference on 3D Imaging. ; 2011. pp. 1–8. (4.14 MB)
Palou G, Salembier P. Occlusion-based depth ordering on monocular images with binary partition tree. In: IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2011. IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2011. Prague, Czech Republic; 2011. pp. 1093–1096. (444.04 KB)
López-Méndez A, Alcoverro M, Pardàs M, Casas J. Real-time upper body tracking with online initialization using a range sensor. In: 2011 IEEE International Conference on Computer VIsion Workshops (ICCV Workshops). 2011 IEEE International Conference on Computer VIsion Workshops (ICCV Workshops). ; 2011. pp. 391–398.
López-Méndez A, Alcoverro M, Pardàs M, Casas J. Approximate partitioning of observations in hierarchical particle filter body tracking. In: 2011 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops. 2011 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops. ; 2011. pp. 19–24.

Pages