Abstract

Sign Language is the primary means of communication for the majority of the Deaf and hard-of-hearing communities. Current computational approaches in this general research area have focused specifically on Sign Language Recognition and Sign Language Translation (from Sign Language to text). However, the reverse problem of translating from spoken language to sign language has so far been not widely explored.

The goal of this dissertation research is to explore the Sign Language Translation (from spoken language to Sign Language) venue and make the audio track content from online videos available to Deaf and hard-of-hearing people by automatically generating a video-based speech to sign language translation. 

Towards that end, we propose different approaches to explore the problem as well as the first Continuous American Sign Language Translation Dataset, a novel dataset with sign language videos associated with the speech and transcription of videos from the instructional videos.