TIPS

Tabletop Interactive Play System: A computer-mediated communication technology to mitigate language deprivation for DHH children.


Problem Statement

With more than 90% of Deaf and Hard of Hearing children have hearing parents.[1] They’re often at risk of language deprivation due to hearing parents’ limited knowledge of sign language. [2] Parental language input in early childhood plays a critical role for lifelong neurocognitive and social development.


System Goals

  • Offers an immersive sign language environment

  • Emphasizes supporting parent autonomy

  • Offers hearing parents context-responsive ASL recommendations

  • Minimizes obtrusion for face-to-face interaction


AI-driven interaction

AI-driven interaction

American Sign Language


Final presentation Slide

TIPS final presentation

[1] Mitchell, Ross E., and Michaela Karchmer. "Chasing the mythical ten percent: Parental hearing status of deaf and hard of hearing students in the United States." Sign language studies 4.2 (2004): 138-163.

[2] Hsu, Wei-Ning, et al. "Hubert: Self-supervised speech representation learning by masked prediction of hidden units." IEEE/ACM Transactions on Audio, Speech, and Language Processing 29 (2021): 3451-3460.