(10.38 MB) Abstract
Student: Dèlia Fernàndez
Advisors: Víctor Campos (UPC), Brendan Jou (Columbia University), Xavier Giró-i-Nieto (UPC) and Shih-Fu Chang (Columbia University)
Grade: A+ (10.0/10.0) - Best Master Thesis award (Class 2016)
One of the main problems in visual affective computing is overcoming the affective gap between low-level visual features and the emotional content of the image. One rising method to capture visual affection is through the use of Adjective-Noun Pairs (ANP), a mid-level affect representation. This thesis addresses two challenges related to ANPs: representing ANPs in a structured ontology and improving ANP detectability. The first part develops two techniques to exploit relations between adjectives and nouns for automatic ANP clustering. The second part introduces and analyzes a novel deep neural network for ANP prediction. Based on the hypothesis of a different contribution of the adjective and the noun depending of the ANP, the novel network fuses the feature representations of adjectives and nouns from two independently trained convolutional neural networks.
