The proposal addresses the problem of creating a new annotation framework of 3M (multimodal, multimedia, multilingual) data with different levels of annotation. In this regard it is compliant with the general subject of D2K topic, as described in the call, to extract new knowledge from large amounts of heterogeneous, unstructured data, in order to elaborate models which, in turn, will be used to process other data: the annotation is essential to improve the modeling used in the automatic systems, but also to acquire new insights from the human and social sciences to whom these annotated data and tools will be made available. The data we want to process consists of videos from different countries and languages, most of them available on the web; this point is also mentioned in the call (“multilingual and multimedia data”). This proposal proposes a new way to envision the multimodal annotation process, and to deliver, at the end of the project, a pilot framework, but also tools and data; the core aspect of this annotation process is a shared format between the different annotation levels, which will be used by different manual and automatic annotation tools and instruments. In this regard, we address the point “6. Generic methodologies, tools and formats to ease the exchange of data and models” of the call, by developing a new methodology, tools, instruments and formats.
At a more general level, the proposal clearly wants to challenge the actual annotation framework, in order to enable annotation of very large amounts of 3M data. This in turn will enable significant breakthroughs in the social sciences and computer sciences which share this same object of study. This ambitious project which is multidisciplinary in the sense that it leads to the set up of a common experimental framework for social sciences and computer sciences, is in line with the general CHIST-ERA principles as expressed in the 2011 call.
(174.87 KB)