Abstract for Matt Huenerfauth's talk

A linguistically motivated model for speed and pausing in animations of American Sign Language
Matt Huenerfauth (Queens College & Graduate Center, CUNY)
September 9, 2008 (Tuesday)
6:30 PM - 8:00 PM; Room 7102, CUNY Graduate Center

A majority of deaf 18-year-olds in the United States have an English reading level below that of a typical 10-year-old student, and so machine translation (MT) software that could translate English text into American Sign Language (ASL) animations could significantly improve these individuals' access to information, communication, and services. Instead of presenting these individuals with English text on computer screens, information could be presented in the form of animations of a virtual human character performing ASL.

An important part of English-to-ASL MT software is the "generation" component, which is responsible for planning and scripting the movements of the virtual character's arms and body to perform a grammatically correct and understandable ASL sentence. This talk will discuss recent research in which results in the psycholinguistics literature on the speed and timing of ASL have been used to design software to calculate realistic timing of the movements in ASL animations.

We have built algorithms to calculate the time-duration of signs and the location/length of pauses during an ASL animation. To determine whether our software can improve the quality of ASL animations, we conducted a study in which native ASL signers evaluated the ASL animations processed by our algorithms. We have found that: (1) adding linguistically motivated pauses and variations in sign-durations improved signers' performance on a comprehension task and (2) these animations were rated as more understandable by ASL signers.