News

• 2024.09: A paper is accepted at EMNLP 2024 @ Miami🇺🇸

• 2024.07: A paper is accepted at ECAI 2024 @ Santiago de Compostela🇪🇸

• 2023.12: A paper is accepted at ACL 2024 @ Bangkok🇹🇭

• 2024.02: Two papers are accepted at LREC-COLING 2024 @ Torino🇮🇹

• 2024.01: Two papers are accepted at EACL 2024 Findings @ Malta🇲🇹

• 2023.12: A paper is accepted at AAAI 2024 @ Vancouver🇨🇦

• 2023.10: A paper is accepted at EMNLP 2023 @ Singapore🇸🇬

• 2023.09: A paper is accepted at IJCNLP-AACL 2023 @Bali🇮🇩

• 2023.07: I started working as a postdoctoral researcher at Timothy Baldwin's group at MBZUAI@Abu Dhabi🇦🇪

• 2023.05: A paper is accepted at BEA 2023 @Toronto🇨🇦

• 2023.01: A paper is accepted at EACL 2023 @Dubrovnik🇭🇷

• 2022.10: A paper is accepted at Findings of EMNLP 2022 @Abu Dhabi🇦🇪

• 2022.09: Visiting researcher at Carnegie Mellon University, NeuLab (Advisor: Prof. Graham Neubig), Pittsburgh🇺🇸

• 2022.08: Two papers are accepted at COLING 2022 @Gyeongju🇰🇷

• 2022.06: A paper is accepted at IEEE Access

• 2022.05: A paper is accepted at NAACL SRW 2022 @Seattle🇺🇸

• 2022.04: A paper is accepted at NAACL 2022 @Seattle🇺🇸

• 2022.04: A paper is accepted at  LREC 2022 @Marseille🇫🇷

• 2022.02: Two papers are accepted at ACL 2022 @Dublin🇮🇪

Professional Experience

• 2023.07 - Present: Postdoctoral researcher @ MBZUAI, Timothy Baldwin's group, Abu dhabi🇦🇪

• 2023.07 - Present: Research Fellow @ Tokyo Institute of Technology, Okazaki Lab, Tokyo🇯🇵

• 2022.12 - Present: Collaborator @ Carnegie Mellon University, NeuLab (Advisor: Prof. Graham Neubig), Pittsburgh🇺🇸

• 2019.03 - Present: Collaborator @ University of Liverpool, UoL NLP Group, Liverpool🇬🇧

• 2021.04 - 2023.05: Postdoctoral Researcher @ Tokyo Institute of Technology, Okazaki Lab, Tokyo🇯🇵

• 2021.04 - 2023.03: Visiting Researcher @ Tokyo Metropolitan University, Komachi Lab, Tokyo🇯🇵

• 2022.09 - 2022.11: Visiting Researcher @ Carnegie Mellon University, NeuLab (Advisor: Prof. Graham Neubig), Pittsburgh🇺🇸

• 2019.11 - 2019.12: Visiting Researcher @ Tohoku University, Natural Language Processing Lab, Sendai🇯🇵

• 2019.08 - 2019.09: Visiting Researcher @ Tohoku University, Natural Language Processing Lab, Sendai🇯🇵

• 2018.10 - 2019.02: Visiting Researcher @ UoL NLP Group (Advisor: Prof. Danushka Bollegala), University of Liverpool, Liverpool🇬🇧

• 2018.08 - 2021.03 : NLP Research Assistant @ RIKEN AIP, Natural Language Understanding Team (Team Leader: Prof. Kentaro Inui), Tokyo🇯🇵

• 2018.07: Participate in Deep Learning Camp (24/562) for a month, Jeju🇰🇷

• 2017.06 - 2017.12: Part time job as NLP Engineer @ Best Teacher, Inc., Tokyo🇯🇵

• 2016.10 - 2018.03: Joint-research with TOYOTA Motor Corporation., Tokyo🇯🇵

• 2016.08 - 2016.09: Research internship @ TOYOTA InfoTechnology Center Co., LTD., Tokyo🇯🇵

Publications

2024

• Ryuto Koike, Masahiro Kaneko, Naoaki Okazaki. How You Prompt Matters! Even Task-Oriented Constraints in Instructions Affect LLM-Generated Text Detection. The 2024 Conference on Empirical Methods in Natural Language Processing (Findings: EMNLP). [arXiv]

• Rem Hida, Masahiro Kaneko, Naoaki Okazaki. Social Bias Evaluation for Large Language Models Requires Prompt Variations. arXiv. [arXiv]

• Mengsay Loem, Masahiro Kaneko, Naoaki Okazaki. SAIE Framework: Support Alone Isn't Enough - Advancing LLM Training with Adversarial Remarks. The 27th European Conference on Artificial Intelligence (ECAI). [arXiv]

• Masanari Ohi, Masahiro Kaneko, Ryuto Koike, Mengsay Loem, Naoaki Okazaki. Likelihood-based Mitigation of Evaluation Bias in Large Language Models. The 62nd Annual Meeting of the Association for Computational Linguistics (Findings: ACL). [arXiv]

• Masahiro Kaneko, Youmi Ma, Yuki Wata, Naoaki Okazaki. Sampling-based Pseudo-Likelihood for Membership Inference Attacks. arXiv. [arXiv]  [Code]

• Masahiro Kaneko, Timothy Baldwin. A Little Leak Will Sink a Great Ship: Survey of Transparency for Large Language Models from Start to Finish. arXiv. [arXiv]

• Masahiro Kaneko, Danushka Bollegala, Timothy Baldwin. Eagle 🦅: Ethical Dataset Given from Real Interactions. arXiv. [arXiv] [data]

• Masahiro Kaneko, Naoaki Okazaki. Controlled Generation with Prompt Insertion for Natural Language Explanations in Grammatical Error Correction. The 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING). (Short paper). [arXiv] [paper] [data]

• Panatchakorn Anantaprayoon, Masahiro Kaneko, Naoaki Okazaki. Evaluating Gender Bias of Pre-trained Language Models in Natural Language Inference by Considering All Labels. The 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING). (Long paper). [arXiv] [paper] [data]

• Masahiro Kaneko, Danushka Bollegala, Naoaki Okazaki, Timothy Baldwin. Evaluating Gender Bias in Large Language Models via Chain-of-Thought Prompting. arXiv. [arXiv]

• Masahiro Kaneko, Graham Neubig, Naoaki Okazaki. Solving NLP Problems through Human-System Collaboration: A Discussion-based Approach. The 18th Conference of the European Chapter of the Association for Computational Linguistics (Findings: EACL). (Long paper). [arXiv] [data]

• Daisuke Oba, Masahiro Kaneko, Danushka Bollegala. In-Contextual Bias Suppression for Large Language Models. The 18th Conference of the European Chapter of the Association for Computational Linguistics (Findings: EACL). (Long paper). [arXiv] [paper]

• Masahiro Kaneko, Danushka Bollegala, Timothy Baldwin. The Gaps between Pre-train and Downstream Settings in Bias Evaluation and Debiasing. arXiv. [arXiv]

• Ryuto Koike, Masahiro Kaneko, Naoaki Okazaki. OUTFOX: LLM-generated Essay Detection through In-context Learning with Adversarially Generated Examples. Proceedings of the 38th AAAI Conference on Artificial Intelligence (AAAI). (Acceptance rate: 21.3%) [arXiv]

2023

• Masahiro Kaneko, Naoaki Okazaki. Reducing Sequence Length by Predicting Edit Operations with Large Language Models. The 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP). (Long paper, Acceptance rate: 23.3%) [arXiv] [paper]

• Masahiro Kaneko, Danushka Bollegala, Naoaki Okazaki. The Impact of Debiasing on the Performance of Language Models in Downstream Tasks is Underestimated. The 13th International Joint Conference on Natural Language Processing and the 3rd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (IJCNLP-AACL). (Short paper, Acceptance rate: 19.1%). [arXiv] [paper] [slide]

• Mengsay Loem, Masahiro Kaneko, Sho Takase, Naoaki Okazaki. Exploring Effectiveness of GPT-3 in Grammatical Error Correction: A Study on Performance and Controllability in Prompt-Based Methods. In Proceedings of the 18th Workshop on Innovative Use of NLP for Building Educational Applications (BEA). (Long paper) [arXiv] [paper]

• Masahiro Kaneko, Danushka Bollegala, Naoaki Okazaki. Comparing Intrinsic Gender Bias Evaluation Measures without using Human Annotated Examples. The 17th Conference of the European Chapter of the Association for Computational Linguistics (EACL). (Short paper). [arXiv] [paper]

2022

• Masahiro Kaneko, Danushka Bollegala, Naoaki Okazaki. Gender Bias in Meta-Embeddings. Findings of the Association for Computational Linguistics: The 2022 Conference on Empirical Methods in Natural Language Processing (Findings: EMNLP ). [arXiv] [paper]

• Hiroyuki Deguchi, Kenji Imamura, Masahiro Kaneko, Yuto Nishida, Yusuke Sakai, Justin Vasselli, Huy Hien Vu, Taro Watanabe. NAIST-NICT-TIT WMT22 General MT Task Submission. Proceedings of the Seventh Conference on Machine Translation (WMT). [Paper]

• Masahiro Kaneko, Danushka Bollegala, Naoaki Okazaki. Debiasing isn't enough! - On the Effectiveness of Debiasing MLMs and their Social Biases in Downstream Tasks. The 29th International Conference on Computational Linguistics (COLING). (Long paper, Acceptance rate: 33.4%) [arXiv] [paper]

• Koki Maeda, Masahiro Kaneko, Naoaki Okazaki. IMPARA: Impact-based Metric for GEC using Parallel Data. The 29th International Conference on Computational Linguistics (COLING). (Long paper, Acceptance rate: 33.4%) [paper]

• Masahiro Kaneko, Aizhan Imankulova, Danushka Bollegala, Naoaki Okazaki. Gender Bias in Masked Language Models for Multiple Languages. In Proceedings of the 2022 Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL). (Long paper, Acceptance rate: 26%) [arXiv] [paper] [code]

• Mengsay Loem, Sho Takase, Masahiro Kaneko, Naoaki Okazaki. ExtraPhrase: Efficient Data Augmentation for Abstractive Summarization. In Proceedings of the 2022 Annual Conference of the North American Chapter of the Association for Computational Linguistics: Student Research Workshop (NAACL SRW). [arXiv]

• Tosho Hirasawa, Masahiro Kaneko, Aizhan Imankulova, Mamoru Komachi. Pre-trained Word Embedding and Language Model Improve Multimodal Machine Translation: A Case Study in Multi30K. IEEE Access, 2022. [paper]

• Yujin Takahashi, Masahiro Kaneko, Masato Mita, Mamoru Komachi. Proficiency Matters Quality Estimation in Grammatical Error Correction. Proceedings of the 13th Language Resources and Evaluation Conference (LREC). [arXiv]

• Masahiro Kaneko, Sho Takase, Ayana Niwa, Naoaki Okazaki. Interpretability for Language Learners Using Example-Based Grammatical Error Correction. In Proceedings of the 60th Annual Conference of the Association for Computational Linguistics (ACL). (Long paper, Acceptance rate: 20.75%) [arXiv] [paper] [code]

• Yi Zhou, Masahiro Kaneko, Danushka Bollegala. Sense Embeddings are also Biased -- Evaluating Social Biases in Static and Contextualised Sense Embeddings. In Proceedings of the 60th Annual Conference of the Association for Computational Linguistics (ACL). (Long paper, Acceptance rate: 20.75%) [arXiv] [paper] [code]

• Masahiro Kaneko and Danushka Bollegala. Unmasking the Mask -- Evaluating Social Biases in Masked Language Models. Proceedings of the 36th AAAI Conference on Artificial Intelligence (AAAI). (Acceptance rate: 15%) [arXiv] [paper] [code]

• Mengsay Loem, Sho Takase, Masahiro Kaneko, Naoaki Okazaki. Are Neighbors Enough? Multi-Head Neural n-gram can be Alternative to Self-attention. arXiv. [arXiv]

2021

• Raj Dabre, Aizhan Imankulova, Masahiro Kaneko. Studying The Impact Of Document-level Context On Simultaneous Neural Machine Translation. Proceedings of the 18th Biennial Machine Translation Summit (MT Summit). [paper]

• Aomi Koyama, Kengo Hotate, Masahiro Kaneko and Mamoru Komachi. Comparison of Grammatical Error Correction Using Back-Translation Models. 2021 Annual Conference of the North American Chapter of the Association for Computational Linguistics: Student Research Workshop (NAACL SRW) (Acceptance rate: 44%) [arXiv] [paper]

• Seiichiro Kondo, Kengo Hotate, Tosho Hirasawa, Masahiro Kaneko and Mamoru Komachi. Sentence Concatenation Approach to Data Augmentation for Neural Machine Translation. 2021 Annual Conference of the North American Chapter of the Association for Computational Linguistics: Student Research Workshop (NAACL SRW) (Acceptance rate: 44%) [paper]

• Masahiro Kaneko and Danushka Bollegala. Debiasing Pre-trained Contextualised Embeddings. The 16th Conference of the European Chapter of the Association for Computational Linguistics (EACL). (Long paper, Acceptance rate: 27%) [arXiv] [paper] [code] [poster]

• Masahiro Kaneko and Danushka Bollegala. Dictionary-based Debiasing of Pre-trained Word Embeddings. The 16th Conference of the European Chapter of the Association for Computational Linguistics (EACL). (Long paper, Acceptance rate: 27%) [arXiv] [paper] [code] [poster]

• Raj Dabre, Aizhan Imankulova, Masahiro Kaneko and Abhisek Chakrabarty. Simultaneous Multi-Pivot Neural Machine Translation. arXiv. [arXiv]

2020

• Masahiro Kaneko and Danushka Bollegala. Autoencoding Improves Pre-trained Word Embeddings. The 28th International Conference on Computational Linguistics (COLING). (Short paper, Acceptance rate: 26.2%) [arXiv] [paper] [bib] [slide]

• Ikumi Yamashita, Satoru Katsumata, Masahiro Kaneko, Aizhan Imankulova and Mamoru Komachi. Cross-lingual Transfer Learning for Grammatical Error Correction. The 28th International Conference on Computational Linguistics (COLING). (Long paper, Acceptance rate: 35.3%) [paper]

• Kengo Hotate, Masahiro Kaneko and Mamoru Komachi. Generating Diverse Corrections with Local Beam Search for Grammatical Error Correction. The 28th International Conference on Computational Linguistics (COLING). (Short paper, Acceptance rate: 26.2%) [paper]

• Ryoma Yoshimura, Masahiro Kaneko, Tomoyuki Kajiwara and Mamoru Komachi. SOME: Reference-less Sub-Metrics Optimized for Manual Evaluations of Grammatical Error Correction. The 28th International Conference on Computational Linguistics (COLING). (Short paper, Acceptance rate: 26.2%) [paper]

• Aizhan Imankulova, Masahiro Kaneko, Tosho Hirasawa and Mamoru Komachi. Towards Multimodal Simultaneous Neural Machine Translation. The Fifth Conference in Machine Translation (WMT). (Acceptance rate: 32.7%) [arXiv] [paper] [code]

• Masato Mita, Shun Kiyono, Masahiro Kaneko, Jun Suzuki and Kentaro Inui. A Self-Refinement Strategy for Noise Reduction in Grammatical Error Correction. Findings of the Association for Computational Linguistics: The 2020 Conference on Empirical Methods in Natural Language Processing (Findings: EMNLP ) [paper]

• Zizheng Zhang, Tosho Hirasawa, Wei Houjing, Masahiro Kaneko and Mamoru Komachi. Translation of New Named Entities from English to Chinese. In Proceedings of the 7th Workshop on Asian Translation (WAT). [paper]

• Masahiro Kaneko, Masato Mita, Shun Kiyono, Jun Suzuki and Kentaro Inui. Encoder-Decoder Models Can Benefit from Pre-trained Masked Language Models in Grammatical Error Correction. In Proceedings of the 58th Annual Conference of the Association for Computational Linguistics (ACL). (Short paper, Acceptance rate: 17.6%) [arXiv] [paper] [bib] [slide] [code]

• Hiroto Tamura, Tosho Hirasawa, Masahiro Kaneko and Mamoru Komachi. TMU Japanese-English Multimodal Machine Translation System for WAT 2020. In Proceedings of the 7th Workshop on Asian Translation (WAT): Japanese-English Multimodal Machine Translation track. 

• Masahiro Kaneko, Aizhan Imankulova, Tosho Hirasawa and Mamoru Komachi. English-to-Japanese Diverse Translation by Combining Forward and Backward Outputs. The 4th Workshop on Neural Generation and Translation (WNGT): Simultaneous Translation And Paraphrase for Language Education (STAPLE) English-to-Japanese track [paper] [bib]

2019

• Masahiro Kaneko and Danushka Bollegala. Gender-preserving Debiasing for Pre-trained Word Embeddings. In Proceedings of the 57th Annual Conference of the Association for Computational Linguistics (ACL). (Long paper, Acceptance rate: 25.7%) [arXiv] [paper] [slide] [bib] [code]

• Kengo Hotate, Masahiro Kaneko, Satoru Katsumata and Mamoru Komachi. Controlling Grammatical Error Correction Using Word Edit Rate. In Proceedings of the 57th Annual Conference of the Association for Computational Linguistics: Student Research Workshop (ACL SRW). [paper] [bib]

• Mio Arai, Masahiro Kaneko and Mamoru Komachi. Grammatical-Error-Aware Incorrect Example Retrieval System for Learners of Japanese as a Second Language. In Proceedings of the 14th Workshop on Innovative Use of NLP for Building Educational Applications (BEA). [paper]

• Masahiro Kaneko and Mamoru Komachi. Multi-Head Multi-Layer Attention to Deep Language Representations for Grammatical Error Detection. In 20th International Conference on Computational Linguistics and Intelligent Text Processing (CICLing). [arXiv] [poster]

• Masato Mita, Tomoya Mizumoto, Masahiro Kaneko, Ryo Nagata and Kentaro Inui. Cross-Corpora Evaluation and Analysis of Grammatical Error Correction Models — Is Single-Corpus Evaluation Enough? 2019 Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL). (Short paper, Acceptance rate: 21.3%) [pdf]

• Masahiro Kaneko, Mamoru Komachi. Multi-Head Multi-Layer Attention to Deep Language Representations for Grammatical Error Detection. Computacion y Sistemas. Vol. 23, No. 3, pp. 883-891. September, 2019. [paper]

• Aizhan Imankulova, Masahiro Kaneko and Mamoru Komachi. Japanese-Russian TMU Neural Machine Translation System using Multilingual Model for WAT 2019. The 6th Workshop on Asian Translation (WAT): News Commentary task.

• Masahiro Kaneko, Kengo Hotate, Satoru Katsumata and Mamoru Komachi. TMU Transformer System Using BERT for Re-ranking at BEA 2019 Grammatical Error Correction on Restricted Track. In Proceedings of the 14th Workshop on Innovative Use of NLP for Building Educational Applications (BEA): Shared Task on Grammatical Error Correction. [paper] [poster] [bib]

2018

• Masahiro Kaneko, Tomoyuki Kajiwara and Mamoru Komachi. TMU System for SLAM-2018. In Proceedings of The 13th Workshop on Innovative Use of NLP for Building Educational Applications (BEA): Shared Task on Second Language Acquisition Modeling. June 2018. [paper] [poster] [bib] [code]

2017

• Masahiro Kaneko, Yuya Sakaizawa and Mamoru Komachi. Grammatical Error Detection Using Error- and Grammaticality-Specific Word Embeddings. In Proceedings of the 8th International Joint Conference on Natural Language Processing (IJCNLP). (Long paper, Acceptance rate: 31%) [paper] [slide] [bib] [code]

Talks

• Gender-preserving Debiasing for Pre-trained Word Embeddings. Academia Meets Amazon. Oct 02, 2019. 

• Gender-preserving Debiasing for Pre-trained Word Embeddings. GeBNLP2019. Aug 02, 2019. [poster]

• Document-level re-ranking for NMT. TFCON2018. Jeju, Korea. July 2018.

Education

• 2018.04 - 2021.03: Ph.D., Tokyo Metropolitan University, Computer Science

• 2016.04 - 2018.03: Master, Tokyo Metropolitan University, Computer Science.

• 2012.04 - 2016.03: Bachelor, Kitami Institute of Technology, Engineering.

Fellowship

• 2021.04 - 2024.03: Japan Society for the Promotion of Science Research Fellowship for Young Scientists-PD (8/35, Decline)

• 2019.04 - 2021.03: Japan Society for the Promotion of Science Research Fellowship for Young Scientists-DC2 (107/624)

• 2018: Tokyo Metropolitan University Dispatch International Student Economic Support

Awards

• 2022.03: Best Award (9/386), The 28rd Annual Meeting of the Association for Natural Language Processing@Japan

• 2021.08: Recommended Doctoral Thesis, Grammatical and Semantic Biases in Representation Learning from Raw Datasets. Information Processing Society of Japan.

• 2017.03: Young Researcher Award (4/301), The 23rd Annual Meeting of the Association for Natural Language Processing@Japan

• 2017.01: Hackathon Technology and Audience Award, First PwC's Technology@Japan

Reviewer

• Area Chair

  • 2024: *SEM, EMNLP, COLING

  • 2023: EMNLP

• Peer

  • 2024: ECAI, NAACL, ACL, AAAI, KDD

  • 2023: ACL, BEA, AAAI

  • 2022: AAAI, ARR, COLING, EMNLP

  • 2021: NAACL, ACL, EMNLP, ARR

  • 2020: ACL, EMNLP

  • 2019: PACLIC 

• Secondary

  • 2020: ACL, CIKM, EMNLP

  • 2019: NAACL, *SEM, ACL, ACL demo, BEA

  • 2018: NAACL, ACL demo, EMNLP demo

  • 2017: NLPTEA

Activities

• 2021.03 - 2023.03: Journal of Natural Language Processing, Editorial Committee

I was a member of 2 academic activities in Japan. Details are here.