SEECAT - Speech & Eye-Tracking Enabled CAT
summer 2013 The Bridge (CRITT + danCAST) plans to conduct an
implementation workshop for computer assisted translation (CAT), in
which a translator reads a source text on a computer screen and speaks
out the translation in the target language, a process called sight translation.
This sight translation process is supported by an Automatic Speech
Recognition (ASR) and a Machine Translation (MT) system, which
transcribe the spoken speech signal into the target text and which
assist the translator with partial translation proposals, predictions
and completions on the computer monitor. An eye-tracking device follows
the translators gaze path on the screen, detects where he or she faces
translation problems and triggers reactive assistance.
The project will extend the CASMACAT workbench,
transforming it to a Speech and Eye-tracking Enabled Computer-Assisted
Translation (SEECAT) platform which will be experimentally implemented
Objective: Use speech input as a post-editing tool
for language translators in order to enhance their efficiency. Use
eyetracker to synchronize reading and speaking with the MT output, for
positioning of input cursor.
Success Metric: Demonstrate increase in translation throughput using speech input for post-editing over a system without speech input.
Why: Currently, human post-editors use keystrokes
to improve the quality of the translation output. The project is to
investigate the efficiency impact if they were to correct the
translation output using speech input.
: In the framework of the current CASMACAT workbench
integrating an automatic speech recognizer (ASR) that would accept
spoken translations as input from human translators/post-editors to
improve the quality output generated by a machine translation system.
The speech recognition system would be partly constrained by the gaze
data and output of a machine translation system, but will also be
flexible enough to accept broader language. Strategies for balancing
these approaches will be investigated.
Technology and Languages: The Sphinx and AT&T
ASR system (Watson) will be trained for Danish and Hindi and the Moses
open source decoder will be used for English > Danish and English
> Hindi translation. More language combinations (e.g. Bengali >
English) are investigated.
The SEECAT summer project is divided into 3 workshops, of each approximately 3 weeks:
- May 21 to June 7: introduction ASR, MT, Eye-tracking, GUI technology;
- June 8 to June 30: work in smaller groups on sub-projects;
- July 1 to July 21: integration of sub-projects into a SEECAT prototype.
The introductory workshop (May 21 to June 7) will take place at Copenhagen Business School, Dalgas Have 15, room 2Ø091.
We'll have lectures and practical sessions in the morning and in the
afternoons. According to the interests of each participant, we will
define small tasks and sub-groups to work on a focussed sub-project and
discuss its relation to the SEECAT project on a regular basis.
For the group work (June 8 to June 30) we have rented a spacious summer house in Nykøbing, Falster. The third integration workshop will, again, take place at Copenhagen Business School.
Directions to get to the Marielyst Strand summerhouse:
1) TRAIN from Copenhagen:
Departure Copenhagen Central Station
Arrival Nykøbing Falster Station
Travel time: aprox. 1h 50 min – 1h 24 min depending on train.
Price: approx. 166,00 DKK
2) BUS from Nykøbing to summerhouse
Departure Nykøbing Falster Rutebilstation
Bus no. 741 (direction Bøtøskoven)
Get off at stop Stovby Ringvej
Travel time: approx. 25 min
Price: 2 zones with clipcard / approx. 24.00 kr (with a train ticket it's 12 kr)
3) WALK to summerhouse
Address is Blommestien 17
The programme for the SEECAT course can be found here (updated as of May 15, 2013).
Presentations and hand-outs will be made available in this section:
- May 21:
- Statistical Machine Translation - Overview (Jakob Elming): (PDF)
- CasMaCat meets SEECAT (Ragnar Bonk)
- CasMaCat GUI: upload - replay - list (Ragnar Bonk & Mercedes García): (PDF)
- May 22:
- Introduction to Post-editing / On-line workbenches (Bartolomé Mesa-Lao) : (PDF)
- Fundamentals of ASR 1 (Richard Rose): (PDF)
- Language modelling: when, why, how (Jeevanthi Liyanapathirana): (PDF)
- May 23:
- Interactive MT (Philipp Koehn): (PDF)
- Fundamentals of ASR 2 (Richard Rose): (PDF)
- May 24:
- Some thoughts about the conceptual/procedural distinction in translation (Fabio Alves): (PDF)
Employing speech recognition software in human translation
(Barbara Dragsted): (PDF)
- Psycholinguistics meets translation studies (Laura W. Balling): (PDF)
- June 3
- Web Technologies in CASMACAT (Vicent Alabau): (PDF)
- June 4
- Interactive machine translation (Vicent Alabau): (PDF)
- June 5
- Multimodal post-editing (Vicent Alabau): (PDF)
- July 9
- ETAP-3 Linguistic Processor: an NLP Implementation of the Meaning/Text Theory (Leonid Iomdin): (PDF)
Lecturers and presenters:
- Alexandra Birch (UEDIN, UK)
- Andreas Søeborg Kirkedal (CBS, Denmark)
- Anusuya Mallavalli Andanigowda (Mysore, India)
- Arnt Lykke Jakobsen (CBS, Denmark)
- Barbara Dragsted (CBS, Denmark)
- Bartolomé Mesa-Lao (CBS, Denmark)
- Fabio Alves (UFMG, Brazil)
- Jakob Elming (KU, Denmark)
- Jeevanthi Liyanapathirana (CBS, Denmark)
- Joris Driesen (UEDIN, UK)
- Leonid Iomdin (IPPI, Moskwa)
- Mercedes García Martínez (CBS, Denmark)
- Michael Carl (CBS, Denmark)
- Pascual Martínez (Tokyo, Japan)
- Peter Juel Henrichsen (CBS, Denmark)
- Philipp Koehn (UEDIN, UK)
- Richard Rose (McGill, Canada)
- Ragnar Bonk (CBS, Denmark)
- Silvia Hansen-Schirra (Germersheim, Germany)
- Srinivas Bangalore (AT&T, USA)
- Syam Agrawal (KIIT, India)
- Titus von Malsberg (Potsdam, Germany)
- Ulrich Germann (UEDIN, UK)
- Vicent Alabau (UPV, Spain)
V. Alabau, L. Rodríguez-Ruiz, A. Sanchis, P. Martínez-Gómez, F.
Casacuberta (2011). "Multimodal Interactive Machine Translation Using
Speech Recognition". Proceedings of the 13th International Conference on Multimodal Interaction. pp. 129-136. 2011.
J. Brousseau, C. Drouin, G. Foster, P. Isabelle, R. Kuhn, Y.
Normandin, P. Plamondon. "French Speech Recognition in an Automatic
Dictation System for Translators: the TransTalk Project". Proceedings of Eurospeech 1995, pp. 193-196. 1995.
P.F. Brown, S.F. Chen, S.A. Della Pietra, V.J. Della Pietra, A.S.
Kehler, R.L. Mercer. "Automatic speech recognition in machine-aided
translation". Computer Speech & Language 8(3):177-187. 1994.
B. Dragsted, M. Mees, I Hansen. "Speaking your translation: students’ first encounter with speech recognition technology", Translation & Interpreting Vol 3, No 1. 2011.
M. Dymetman, J. Brousseau, G. Foster, P. Isabelle, Y. Normandin, P.
Plamondon. "Towards an Automatic Dictation System for Translators: the
TransTalk Project". Proceedings of the 3rd International Conference on Spoken Language Processing, ICSLP, pp. 691-694. 1994.
J. Elming and Bonk, "The CASMACAT workbench: a tool for investigating the integration of technology in translation", In Proceedings of the AMTA 2012 Workshop on Post-Editing Technology and Practice (WPTP2012). October 2012, San Diego, California, USA.
S. Khadivi, H. Ney. "Integration of Speech Recognition and Machine Translation in Computer-Assisted Translation". IEEE Transactions on Audio, Speech, and Language Processing, 16(8): 1551-1564. 2008.
S. Khadivi, A. Zolnay, H. Ney. "Automatic Text Dictation in Computer-Assisted Translation". Proceedings of the European Conference on Speech Communication and Technology, Interspeech, pp. 2265-2268, Lisbon, Portugal, September 2005.
S. Khadivi, R. Zens, H. Ney. "Integration of Speech to Computer-Assisted Translation Using Finite-State Automata". Proceedings
of joint conference of the International Committee on Computational
Linguistics and the Association for Computational Linguistics. 467-474. 2006.
P. Koehn. "Aiding Human Translators. Seminar in The Center For Language and Speech". Processing at the Johns Hopkins University. January 29th. 2013.
Y. Ludovik , R. Zacharski. "MT and Topic-Based Techniques to Enhance Speech Recognition Systems for Professional Translators". Proceedings of the 18th conference on Computational linguistics, 2:1061-1065. 2000.
D. Ortiz-Martínez, Germán Sanchis-Trilles, Francisco Casacuberta,
Vicent Alabau, Enrique Vidal, José-Miguel Benedí, et al. "The CASMACAT
Project: The Next Generation Translator’s Workbench". Proceedings of iberSPEECH 2012. 2012.
A. Reddy, R. Rose, A. Désilets. "Integration of ASR and Machine Translation Models in a Document Translation Task". Proceedings of International Conference on Spoken Language Processing, pp. 2457-2460. 2007.
A. Reddy, R. C. Rose. "Towards domain independence in machine aided human translation". Proceedings of InterSpeech, 9th Annual Conference of the International Speech Communication Association, pp. 2358-2361. 2008.
A. Reddy, R. C. Rose. "Integration of Statistical Models for
Dictation of Document Translations in a Machine Aided Human Translation
Task". IEEE Transactions on Audio, Speech, and Language Processing, 18(8): 2015-2027. 2010.
L. Rodríguez, A. Reddy, R. Rose. "Efficient integration of
translation and speech models in dictation based machine aided human
translation". Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing. pp. 4949 - 4952. 2012.
E. Vidal, F. Casacuberta, L. Rodríguez, J. Civera, C. D.
Martínez-Hinarejos. "Computer-Assisted Translation Using Speech
Recognition". IEEE Transactions on Audio, Speech, and Language Processing, 14(3):941-951. 2006.
Some of these papers can be retrieved from this link.
- Franz Pöchhacker, Arnt Lykke Jakobsen, Inger M. Mees (eds). Interpreting Studies and Beyond. A Tribute to Miriam Shlesinger. CSL 35, 2007
- Susanne Göpferich, Arnt Lykke Jakobsen (eds). Looking at eyes. CSL 36, 2008
- Susanne Göpferich, Arnt Lykke Jakobsen, Inger M. Mees (eds). Behind the Mind. CSL 37, 2010
- Inger M. Mees, Fabio Alves, Susanne Göpferich (eds). Methodology, Technology and Innovation in Translation Process Research. CSL 38, 2010
- Inger Mees, Susanne Göpferich, Fabio Alves (eds). New Approaches in Translation Proces Research. CSL 39, 2010
- B. Sharp, M. Zock, M. Carl, A.L. Jakobsen (eds). Human-Machine Interaction in Translation. CSL 41, 2011
The Danish Agency for Science, Technology and Innovation and