CoWriting Kazakh: Learning a New Script with a Robot

This research is situated in a specialized context offering a rarely occurring opportunity for  research questions and technical objectives about language acquisition. The Kazakh language transition from Cyrillic to Latin alphabet in Kazakhstan raises challenges to teach the whole population to write and read in the new script. We propose an unique interdisciplinary approach by integrating innovative solutions from robotics, computer vision fields and pedagogical strategies from education, linguistics and cognitive sciences that will assist various demographic groups in this challenging endeavor. 

This research is in collaboration with Dr. Wafa Johal from UNSW, Thibault Asselborn and Dr. Pierre Dillenbourg from EPFL, Dr. Anna CohenMiller from NU Graduate School of Education, and Dr. Danna Summers from Baishev Aktobe University. 

Its funded by the NU Collaborative Research Project (2020-2022) and by the Swiss State Secretariat for Education, Research and Innovation (SERI) as parts of the CIS Region Seeding Grants (2019-2020).

• Handwriting Recognition of Cyrillic scripts: we implemented a real-time recognition of children's and adults' handwritten letters in Cyrillic and convert them to Latin letters. The handwriting recognition was developed based upon our collection of the first-ever Cyrillic-MNIST dataset. Apart from automatically converting the spellings of words for the user, the system is intended to motivate children to learn and memorize the new alphabet. 

• CoWriting Kazakh HRI system: this scenario is designed to assist primary school children in learning a new script and its associated handwriting. The system was deployed in a series of experiments where children increase their knowledge of the new script when practicing handwriting with a robot. 

• The transferability of handwriting skills: Do handwriting skills transfer when a child writes in two different scripts, such as the Latin and Cyrillic alphabets? Are our measures of handwriting skills intrinsically bound to one alphabet or will a child who faces handwriting difficulties in one script experience similar difficulties in the other script? To answer these questions, 190 children from grades 1 to 4 were asked to copy a small text using both the Cyrillic and Latin alphabets on a digital tablet. This unique situation created a quasi-experimental situation that allowed us to measure the influence of the number of years spent practicing Cyrillic on the quality of handwriting in the Latin alphabet. The results showed that some of the differences between the two scripts were constant across all grades. These differences thus reflect the intrinsic differences in the handwriting dynamics between the two alphabets. For instance, several features related to the pen pressure on the tablet are quite different. Other features, however, revealed decreasing differences between the two scripts across grades. While found that the quality of Cyrillic writing increased from grades 1 to 4, due to increased practice, we also found that the quality of Latin increased as well, despite the fact that all of the pupils had the same absence of experience in writing in Latin. We can therefore interpret this improvement in Latin script as an indicator of the transfer of fine motor control skills from Cyrillic to Latin. This result is especially surprising, as one could instead hypothesize a negative transfer, i.e., that the finger controls automated for one alphabet would interfere with those required by  the other alphabet. 

Publications on CoWriting Kazakh project