Abstract:
Hearing loss is an invisible disability, hampering communication and alienating the disabled person from the mainstream. Hearing impairment poses a serious challenge to access information that are in the form of audio/video on the internet. The assistive tool presented is envisioned as a visual aid in the form of lip movement sequence, for the hearing impaired, to enable access to videos. Lip reading allows access to speech through visual reading of the movement of the lips, face and tongue in the absence of audible sound, making use of the information associated with the context, the knowledge of the language, and also the residual hearing of the person. Our tool essentially produces lip movement sequences for a selected video, such that it is in time synchronization with the speech of the video. The lip movement sequence is then superimposed onto and displayed along with the original video, we believe that the video would set the context. For better and easier understanding, care is taken that the synthesized lip movements are near natural. Conventionally, access to the audio or speech, in a video, by the hearing impaired is provided by means of either text subtitles or sign language gestures by an interpreter. Manual generation of text subtitles is a laborious process especially in the script of the spoken language and assumes fair levels of literacy in the user, which may not be the case. Presence of a human interpreter at all times is again an impractical solution. The proposed tool would be beneficial, especially in situations where such aids are not readily available or generating such aids is difficult.
Some demo videos: