FascinatE - Format-Agnostic SCript-based INterAcTive Experience

Type Start End
European Feb 2010 Jul 2013
Responsible URL
Javier Ruiz Hidalgo http://www.fascinate-project.eu/

Reference

Format-Agnostic SCript-based INterAcTive Experience. 
Ref.: IP 248138, EU 7th Framework Programme Call: FP7-ICP-2009-1.5

Description

FascinatE is a €9.5m EU funded project involving a group of partners from across Europe. FascinatE is looking at broadcasting live events to give the viewer a more interactive experience no matter what device they are using the view the broadcast.

The FascinatE project will develop a system to allow end-users to interactively view and navigate around an ultra-high resolution video panorama showing a live event, with the accompanying audio automatically changing to match the selected view. The output will be adapted to their particular kind of device, covering anything from a mobile handset to an immersive panoramic display. At the production side, this requires the development of new audio and video capture systems, and scripting systems to control the shot framing options presented to the viewer. Intelligent networks with processing components will be needed to repurpose the content to suit different device types and framing selections, and user terminals supporting innovative interaction methods will be needed to allow viewers to control and display the content.

To implement a system to achieve these aims requires a paradigm shift in video production, towards capturing a format-agnostic representation of the whole scene from a given viewpoint, rather than the view selected by a cameraman based on assumptions about the viewer’s screen size and interests. It should be noted that we are not proposing to capture a 3D representation of the scene, nor to support true ‘free viewpoint’ rendering. Such systems, in the experience of the project partners, will be incapable of presenting really high-quality images for the foreseeable future, so we are concentrating on an approach that will deliver true high-quality images, whilst allowing a significant degree of interactivity, and which is practical within the time frame of the project. 

By format-agnostic we mean that the resolution, field-of-view, aspect ratio, frame rate and colour depth of the captured image are chosen based on the requirements of the particular production, rather than being tailored to a particular delivery format. Indeed, there will be no single camera with a given set of parameters covering the scene; rather, different parts of the scene will be captured with different types of camera, clustered around one or more viewpoints.

The FascinatE Project consortium is made up from 11 partners from 11 institutions from across Europe:

 

Publications

Alcoverro M. Stochastic optimization and interactive machine learning for human motion analysis Pardàs M, Casas J. Signal Theory and Communications. 2014 . (17.16 MB)
Barkhuus L, Zoric G, Engström A, Ruiz-Hidalgo J, Verzijp N. New interaction modes for rich panoramic live video experiences. Behaviour & Information Technology. 2014 ;33(8).
Suau X, Alcoverro M, López-Méndez A, Ruiz-Hidalgo J, Casas J. Real-time Fingertip Localization Conditioned on Hand Gesture Classification. Image and Vision Computing. 2014 ;32(8):522 - 532. (2.66 MB)
Schreer O, Macq J, Niamut O, Ruiz-Hidalgo J, Shirley B, Thallinger G, Thomas G. Media Production, Delivery and Interaction for Platform Independent Systems. Wiley; 2014.
Niamut O, Kaiser R, Kienast G, Kochale A, Spille J, Schreer O, Ruiz-Hidalgo J, Macq J, Shirley B. Towards A Format-agnostic Approach for Production, Delivery and Rendering of Immersive Media. In: ACM Multimedia Systems. ACM Multimedia Systems. Oslo, Norway; 2013. (2.24 MB)
Alcoverro M, Suau X, Morros JR, López-Méndez A, Gil-Moreno A, Ruiz-Hidalgo J, Casas J. Gesture Control Interface for immersive panoramic displays. Multimedia Tools and Applications. 2013 :1-27.
Kochale A, Ruiz-Hidalgo J, Borsum M. Gesture controlled interactive rendering in a panoramic scene. In: European Interactive TV Conference, EuroITV. European Interactive TV Conference, EuroITV. Como, Italy; 2013. (466.78 KB)
Zoric G, Engström A, Barkhuus L, Ruiz-Hidalgo J, Kochale A. Gesture Interaction with Rich TV Content in the Social Setting. In: Exploring and Enhancing the User Experience for Television, Workshop of ACM SIGCHI Conference on Human Factors in Computing Systems, CHI'13. Exploring and Enhancing the User Experience for Television, Workshop of ACM SIGCHI Conference on Human Factors in Computing Systems, CHI'13. Paris, France; 2013. (4.47 MB)
Suau X. Human body analysis using depth data Casas J, Ruiz-Hidalgo J. 2013 . (10.67 MB)
Suau X, Ruiz-Hidalgo J, Casas J. Detecting End-Effectors on 2.5D data using Geometric Deformable Models: Application to Human Pose Estimation. Computer Vision and Image Understanding (CVIU). 2013 ;117(3). (1.44 MB)

Pages

Demos and Resources