Round table
The gap between MT for spoken and MT for signed languages:
data and technology challenges
MT for spoken languages (SpL) still mainly concerns the written modality (vs the oral one). Signed languages (SL) don't have a written modality, nor do SL have an equivalent to the SpL International Phonetic Alphabet (IPA). Indeed, even if certain graphical systems (e.g. HamNoSys, SignWriting) can represent part of the phonetics of SL, they are not designed or suited to represent all the phenomena specific to SL, such as the linguistic use of the signing space (which is in 3D), the refinement of what is expressed by the non-manual components, and the complex spatio-temporal structures such as the depicting units. Even less can the use of glosses be considered as a transcription of SL content.
Translation between a spoken / written and a signed language therefore consists of translating between different modalities (written, oral and gestural) of two different linguistic systems (spoken and signed). This makes things much more complicated.
Added to this is the very small size of the corpus available for training the MT models.
A large proportion of current studies on MT for SL use glosses as a "written representation" of SL, which makes it possible to use current MT architectures designed for translations between two written languages. But in this case, the specific features of the SL are not taken into account, and the SL content loses its integrity.
Other studies attempt end-to-end approaches, but these are much more data-hungry.
In this round-table would like to open a discussion:
on data: Depending on the nature of the corpus, the specific features of SL are more or less represented, from interpreted content to SL content produced by deaf people not in lab conditions.
on technology: The vast majority of researchers in the field are hearing people for whom SL is not their first language, who are not necessarily familiar with the SL linguistic system, and who design systems through the prism of their knowledge of their own language, which has a huge bias, the first being to consider more or less implicitly that SL can be considered as sequences of lexical units.
Moderator: Mathias Müller (UZH)
Ask your question here:
https://dory.app/events/SAAZDsEFXb9TgAfE5gKr/ae56a1aa_at4ssl-round-table
