MuViPro - Multicamera Video Processing

The major goal of the current project is to investigate the extension of video processing tools to more generic multicamera settings, and a wider range of video processing applications. This goal is stated through the following objectives:

• Extension of existing and development of new video processing algorithms for visual analysis and representation considering the multiview and segmented views for multicamera setups and the available knowledge for controlled scenarios. In particular, low-level analysis algorithms including foreground detection and tracking, visual matching and 3D scene representation, high-level analysis algorithms such as human body analysis and the analysis of objects, text, faces and events, and video coding algorithms in multicamera settings for 3D and stereoscopic video signals
• Focus new target applications such as sport events, 3DTV/Free view-point TV (FTV), and visual interaction where multi-camera setups and ‘a priori‘ knowledge of the scenario can be exploited for analysis and representation tasks. These three applications offer adequate and demanding scenarios to extend the tools developped for controlled environments to a wider set of multiple camera setups and scenarios.

The specific objectives for the extension of video processing algorithms are as follows

• Low-level analysis algorithms: to improve the performance of available tools for foreground detection and tracking, visual matching and 3D scene representation considering a wide range of camera setups (including segmented and multiview multicamera setups) and study their extension to less controlled environments.
• High-level analysis algorithms and video coding: to extend the tools for human body analysis and the analysis of objects, text, faces and events in the scene to the requirements of the new application scenarios and to exploit the particular setup of multicamera capture.

The specific objectives for the newly targeted applications have the common goal of evaluating the tools and proving their interest:

• Sports events: definition of a processing strategy exploiting the knowledge of the scenario and the multicamera setup for the visual analysis and coding of sports footage
• Visual interaction: increase the robustness of gesture detection for visual interaction either in multiview camera settings or in limited multicamera settings for office or home applications
• 3DTV/FTV: algorithms for stereoscopic video analysis and coding will be extended to data from a new audiovisual production laboratory, which will be built in 2010 in the Signal Theory and Communications Department to foster research and technology transfer in 3DTV video and Free viewpoint TV applications