Large Scale Video Tagging with Knowledge Bases

Type Start End
National Jun 2017 May 2020
Responsible URL
Xavier Giró-i-Nieto Industrial Doctorates

Reference

2017-DI-011

Description

In the recent years, video sharing in social media from different video recording devices has resulted in a exponential growth of videos on the Internet. Such video data is continuously increasing with daily recordings related to a wide number of topics. In this context, video understanding has become a critical problem to address. Video search and indexation benefits from the use of keyword tags related to the video content, but most of the shared video content does not contain these tags. Although the use of deep learning has become essential for image analysis in several areas, video domain is still a relatively unexplored field for these type of methods. On the other hand knowledge graphs as Freebase or WordNet store high quantities of information about the word and relations that can be used to disambiguate concepts and relate them through contextual information In this research project we search to explore and improve the understanding of video content through the use of automatic tagging models based on Machine Learning and Deep Learning techniques, improved by the use of knowledge bases. 

Publications

Fernàndez D, Varas D, Espadaler J, Ferreira J, Woodward A, Rodríguez D, Giró-i-Nieto X, Riveiro JCarlos, Bou E. ViTS: Video Tagging System from Massive Web Multimedia Collections. In: ICCV 2017 Workshop on Web-scale Vision and Social Media . ICCV 2017 Workshop on Web-scale Vision and Social Media . Venice, Italy; 2017. (1.18 MB)
Fernàndez D, Woodward A, Campos V, Jou B, Giró-i-Nieto X, Chang S-F. More cat than cute? Interpretable Prediction of Adjective-Noun Pairs. In: ACM Multimedia 2017 Workshop on Multimodal Understanding of Social, Affective and Subjective Attributes. ACM Multimedia 2017 Workshop on Multimodal Understanding of Social, Affective and Subjective Attributes. Mountain View, CA (USA): ACM SIGMM; 2017. (9.62 MB)

Collaborators

Xavier Giró Associate Professor xavier.giro@upc.edu