Call for Papers

With the recent advances in the machine translation era and the high quality translations obtained by neural MT systems, we observe human translators and MT systems changing their roles. Instead of using the MT outputs as the raw material to start the translation, human translators now just need to perform the very last touches on the automatic translations and send them to the end-users.

The increased trust in MT quality, however, requires a more careful monitoring of MT systems in the production line in order to spot errors at the end of the translation pipeline and to fix them, either automatically or manually.

In this pipeline, Quality, Cost, and Delivery speed are the three main factors. We ultimately want to preserve translation quality while increasing translation speed and keeping the final cost of translation in different scenarios under control. To this end, quality estimation and automatic post-editing solutions play important roles. The goal of quality estimation is to evaluate a translation system’s quality without access to the reference translations (Blatz et al., 2004; Specia et al., 2009). This has many potential uses: informing the end user about the reliability of translated content; deciding if a translation is ready for publishing or if it requires human post-editing; and highlighting the words that need to be changed. Quality estimation systems are particularly appealing for crowd-sourced and professional translation services due to their potential to dramatically reduce post-editing times and to save labor costs (Specia, 2011). The increasing interest in this problem from an industry angle comes as no surprise (Federico et al., 2014; de Souza et al., 2015; Kozlova et al., 2016; Martins et al., 2016, Martins et al., 2017; Wang et al., 2018). Recently, it has also started to attract attention in the direct publishing scenario, mostly from e-commerce companies (Ueffing, 2018; Wang et al., 2018).

Automatic post-editing, on the other hand, aims to automatically correct the output of machine translation (Simard et al., 2007, Junczys-Dowmunt and Grundkiewicz, 2017, Junczys-Dowmunt and Grundkiewicz, 2018). Given the high quality translations obtained by neural MT systems, the key question is if quality estimation and automatic post-editing are still the thing!

The workshop of “Human-aided Translation” builds upon the workshop of “First Workshop on Translation Quality Estimation and Automatic Post-Editing”, a successful and well-attended workshop recently held with AMTA 2018. It will bring together academic and industry researchers, as well as practitioners interested in the tasks of quality estimation (word, sentence, or document level) and automatic post-editing, both from a research perspective and with the goal of applying these systems in industry settings for routing, for improving translation quality, or for making human post-editors more efficient. In this edition, we will give special emphasis to neural-based solutions for quality estimation and automatic post-editing tools and their integration with neural machine translation systems.

Submissions

We invite the submission of extended abstracts related to the topics of the workshop. The authors of the accepted submissions will be invited for contribution talks in the workshop. The abstracts should be no longer than two pages, including references. Topics of the workshop include but are not limited to:

• Research, review, and position papers on document-level, sentence-level, or word-level Quality Estimation of neural MTs
• Research, review, and position papers on Automatic Post-Editing for neural MTs
• Research, review, and position papers on Interactive neural MTs
• Corpora curation technologies for developing Quality Estimation datasets
• User studies showing the impact of Quality Estimation tools in translator productivity
• Automatic metrics for translation fluency and adequacy
• Industrial experiences of adopting Quality Estimation for neural MTs
• Industrial experiences of adopting Automatic Post-Editing for neural MTs

Submissions should be formatted according to the ACL template (http://www.acl2019.org/medias/340-acl2019-latex.zip).

The extended abstracts should be submitted via EasyChair system: https://easychair.org/conferences/?conf=hat19. Abstracts will be reviewed for relevance and quality. Accepted submissions will be posted online, and offered oral presentations.

Important dates

• Extended submission deadline: June 7 (Previous date: May 31)
• Notification date: June 17
• Camera-ready submission: July 25
• Workshop day: August 19

Program

• 8.50-9.00 Welcome
• 9.00-9.45 Invited talk1: Lucia Specia (Imperial College London and University of Sheffield)

Quality Estimation and Automatic Post-editing in the Neural Machine Translation Era (slides)

• 9.45-10.30 Invited talk 2: Markus Freitag (Google)

APE at scale and its Implications on MT Evaluation Biases

• 10.30-11.00 Coffee break

• 11.00-11.45 Invited talk 3: Jiayi Wang (Alibaba)

Quality Estimation Technology and its Applications in E-Commerce Machine Translation

• 11.45-12.30 Invited talk 4: Fabio Kepler (Unbabel)

Quality Estimation in Practice: from Implementation to State-of-the-Art (slides)

• 12.30-14.00 Lunch break

• 14.00-14.45 Invited talk 5: Marco Turchi (FBK)

Quality estimation in support of automatic post-editing (slides)

• 14.45-15.00 Contributed talk. Tsz Kin Lam (Heidelberg University)

MT Quality Estimation for e-Commerce: Automatically Assessing the Quality of Machine Translated Titles (slides)

• 15.00-15.30 Coffee break

• 15.30-16.15 Invited talk 6: Dimitar Shterionov (Adapt Center)

Neural Quality Estimation as a Bridge for Human-Computer Translation Symbiosis (slides)

• 16.15-17.30 Panel discussion

Alon Lavie (Unbabel), Jiayi Wang (Alibaba), Markus Freitag (Google), Carlos Teixeira (IOTA), Christian Federmann (Microsoft Research)

• 17.30 Adjourn

Confirmed invited speakers

Christian Federmann (Microsoft)

Markus Freitag (Google)

Fabio Kepler (Unbabel)

Marco Turchi (FBK)

Dimitar Shterionov (ADAPT Centre)

Lucia Specia (Imperial College London and University fo Sheffield)

Jiayi Wang (Alibaba)

Organizers

Maxim Khalilov (Unbabel): maxim@unbabel.com

M. Amin Farajian (Unbabel): amin@unbabel.com

André Martins (Unbabel): andre.martins@unbabel.com

References

John Blatz, Erin Fitzgerald, George Foster, Simona Gandrabur, Cyril Goutte, Alex Kulesza, Alberto Sanchis, and Nicola Ueffing. Confidence estimation for machine translation. In Proceedings of the 20th International Conference on Computational Linguistics (COLING), Geneva, Switzerland, August, 2004

Lucia Specia, Marco Turchi, Nicola Cancedda, Marc Dymetman, and Nello Cristianini. Estimating the sentence-level quality of machine translation systems. In Proceedings of 13th Annual Conference of the European Association for Machine Translation (EAMT), 2009

Lucia Specia. Exploiting objective annotations for measuring translation post-editing effort. In Proceedings of 15th International Conference of the European Association for Machine Translation (EAMT), 2011

Jiayi Wang, Kai Fan, Bo Li, Fengming Zhou, Boxing Chen, Yangbin Shi, Luo Si. Alibaba Submission for WMT18 Quality Estimation Task. In Proceedings of the Third Conference on Machine Translation (WMT), 2018

Marcello Federico, Nicola Bertoldi, Mauro Cettolo, Matteo Negri, Marco Turchi, Marco Trombetti, Alessandro Cattelan, Antonio Farina, Domenico Lupinetti, Andrea Martines, Alberto Massidda, Holger Schwenk, Loïc Barrault, Frederic Blain, Philipp Koehn, Christian Buck, Ulrich Germann. The matecat tool. In Proceedings of COLING 2014

José GC de Souza, Matteo Negri, Elisa Ricci, Marco Turchi. Online multitask learning for machine translation quality estimation. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics (ACL), 2015

Anna Kozlova, Mariya Shmatova, and Anton Frolov. YSDA Participation in the WMT16 Quality Estimation Shared Task. In Proceedings of the First Conference on Machine Translation, 2016

André FT Martins, Ramón Astudillo, Chris Hokamp, Fabio Kepler. Unbabel's participation in the WMT16 word-level translation quality estimation shared task. In Proceedings of the First Conference on Machine Translation. 2016

André F.T. Martins, Marcin Junczys-Dowmunt, Fabio N. Kepler, Ramón Astudillo, Chris Hokamp, Roman Grundkiewicz. Pushing the Limits of Translation Quality Estimation. Transactions of the Association for Computational Linguistics, vol. 5, pp. 205–218, 2017

M. Simard, C. Goutte, and P. Isabelle. Statistical Phrase-based Post-editing. In Proceedings of the Human Language Technology Conference and the North American Chapter of the Association for Computational Linguistics (HLT/NAACL), 2007

Marcin Junczys-Dowmunt, Roman Grundkiewicz. An Exploration of Neural Sequence-to-Sequence Architectures for Automatic Post-Editing. In Proceedings of the Eighth International Joint Conference on Natural Language Processing, IJCNLP, 2017

Marcin Junczys-Dowmunt, Roman Grundkiewicz. MS-UEdin Submission to the WMT2018 APE Shared Task: Dual-Source Transformer for Automatic Post-Editing. In Proceedings of the Third Conference on Machine Translation (WMT), Volume 2: Shared Task Papers, pages 822–826

Nicola Ueffing. Automatic Post- Editing and Machine Translation Quality Estimation at eBay. In Proceedings for AMTA 2018 Workshop: Translation Quality Estimation and Automatic Post-Editing