Duarte A, Camli G, Torres J, Giró-i-Nieto X. Towards Speech to Sign Language Translation. In ECCV 2018 Workshop on Shortcomings in Vision and Language. 2018.  (142.48 KB)

Abstract

Sign Language (SL) is the primary means of communication for a majority of the hearing-impaired community. Current computational approaches in this research area have focused specifically on Sign Language Recognition (SLR) and Sign Language Translation (from SL to text) (SLT). However, the reverse problem of translating from spoken language to sign language has so far been unexplored. The goal of our ongoing project is to provide to people with hearing disabilities the audio tracks from online videos, by automatically generating a video-based speech to sign language translation. In this paper, we will point out the shortcomings that limit the advances of this research area and propose first steps towards this end.