MeBook

MEBOOK is an interactive game to improve social greeting behaviors, which are often lacking or absent among children with autism. MEBOOK has two components. The first component is a social narrative combined with computed-generated self modeling. The social narrative is an animated story about the main character meeting and greeting different cartoon characters in a clinic. The self modeling is achieved by first replacing the main character’s face with an image of the subject, and then animating the subject to match the narration. An age and gender-appropriate voice is used for the narration.

The second component of MEBOOK is a positive reinforcement game (Figure 2). A live video of the subject is captured by the Microsoft Kinect camera and shown on the left side of the screen. Different animated characters come in from the right to greet the subject. If the subject performs the appropriate greeting behavior (for example, say hi and waving hand), verbal and visual positive feedback is given in the form of the character verbally praising the subject (for example, “Good Job” is spoken and written) as well as confetti falling down. The greeting behavior is recognized based on a classifier on the movement of the detected skeleton.

A human subject study was conducted from June 2014-Aug. 2014. It was a multiple baseline single subject study with three male subjects, ages from 7 to 12, all with difficulties in proper social greeting behaviors. The study consisted of 20 sessions. Each session consisted of a 20-minute intervention (with or without MEBOOK depending on the phase of the study), sandwiched by two 10-minute data collection periods during which the subject could freely interact with age-appropriate typical peers.

Selected Publications:

• Uzuegbunam, N. M., W.-H. Wong, S.-C. Cheung, L. A. Ruble. 2018. MeBook: combining evidence-based autism intervention with camera-based multimedia systems. IEEE Transactions on Learning Technologies. DOI: 10.1109/TLT.2017.2772255.
• Uzuegbunam, N. M., W.-H. Wong, S.-C. Cheung, L. A. Ruble. 2015. MEBook: Kinect-based self-modeling intervention for children with autism. In IEEE International Conference on Multimedia Expo (ICME 2015), 20 June – July 3, 2015, Torino, Italy.
• Uzuegbunam, N. M., W.-H. Wong, S.-C. Cheung, L. A. Ruble. 2015. Mebook – a First-Person Social Narrative Game. In International Meeting for Autism Research (IMFAR), May 13-16, 2015.
• Ukaobah, N., J. Shen and S.-C. Cheung. 2013. MEBOOK: a novel device using video self-modeling to enhance literacy among children with ASD. In International Meeting for Autism Research (IMFAR), May 2-4, 2013.