CSCI-644: Natural Language Dialogue Systems - Spring 2024

Readings

Based on the readings, each student should prepare at least one question and optionally additional comments about each one of the required readings. So if for example there are three required to read papers there should be at least three questions overall (one for each paper) and optionally additional comments for each paper. These could be questions about aspects of the research that were interesting or unclear to you, or comments you have on the methodology or results in the work, or implications of the work, or how it might be applied to other work. Students should post their questions on the course Piazza site in the topic for that week's readings by 11:59 pm the day prior to that week's lecture, and come prepared to discuss their questions in class. When you post questions/comments on Piazza use the "Readings" tag, include in the title of the message the week that the question/comment refers to, and mention in the body of the message the paper that the question/comment refers to.

Week 1: Traum &  Georgila - Oveview, basic principles, genres, different types of dialogue, example dialogue systems, topics to be covered (questions can be posted on Piazza for extra credit before the last class)

Optional

• David Traum, "Computational Approaches to Dialogue" in The Routledge Handbook of Language and Dialogue Edited by Edda Weigand, Routledge, 2017, pp. 143-161. Pre-release version

• David Traum Socially Interactive Agent Dialogue, Chapter 15 of The Handbook on Socially Interactive Agents (Volume 2) 2022. preprint Preprint

• Chatbots & Dialogue Systems Chaptert 15 of Speech and Language Processing. Daniel Jurafsky & James H. Martin, Draft of January 2023.

Week 2: Traum - Overview of Dialogue Structure (turn-taking, initiative, relations, dialogue acts, intentional structure), introduction to  organizing principles for dialogue management

Optional

Week 3: Georgila - Deep learning approaches to dialogue (questions should be posted on Piazza by January 23, 11:59 pm)

Required

• Donghoon Ham, Jeong-Gwan Lee, Youngsoo Jang, and Kee-Eung Kim. 2020. End-to-End Neural Pipeline for Goal-Oriented Dialogue Systems using GPT-2. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 583–592, Online. 

• Jiwei Li, Michel Galley, Chris Brockett, Georgios Spithourakis, Jianfeng Gao, and Bill Dolan. 2016. A Persona-Based Neural Conversation Model. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 994–1003, Berlin, Germany. 

• Daniel Adiwardana, Minh-Thang Luong, David R So, Jamie Hall, Noah Fiedel, Romal Thoppilan, Zi Yang, Apoorv Kulshreshtha, Gaurav Nemade, Yifeng Lu, and Quoc V. Le. 2020. Towards a human-like open-domain chatbot. arXiv preprint arXiv:2001.09977.

• Yizhe Zhang, Siqi Sun, Michel Galley, Yen-Chun Chen, Chris Brockett, Xiang Gao, Jianfeng Gao, Jingjing Liu, and Bill Dolan. 2020. DIALOGPT: Large-Scale Generative Pre-training for Conversational Response Generation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 270–278, Online. 

Optional

• Iulian V. Serban, Alessandro Sordoni, Yoshua Bengio, Aaron Courville, and Joelle Pineau. 2016. Building end-to-end dialogue systems using generative hierarchical neural network models. In Proceedings of the AAAI Conference on Artificial Intelligence.

• Antoine Bordes, Y-Lan Boureau, and Jason Weston. 2017. Learning end-to-end goal-oriented dialogue. In Proceedings of the International Conference on Learning Representations (ICLR).

• Alessandro Sordoni, Michel Galley, Michael Auli, Chris Brockett, Yangfeng Ji, Margaret Mitchell, Jian-Yun Nie, Jianfeng Gao, and Bill Dolan. 2015. A Neural Network Approach to Context-Sensitive Generation of Conversational Responses. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 196–205, Denver, Colorado. 

• Ryan Lowe, Nissan Pow, Iulian Serban, and Joelle Pineau. 2015. The Ubuntu Dialogue Corpus: A Large Dataset for Research in Unstructured Multi-Turn Dialogue Systems. In Proceedings of the 16th Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 285–294, Prague, Czech Republic. 

• Miaoran Li, Baolin Peng, Michel Galley, Jianfeng Gao, and Zhu (Drew) Zhang. 2023. Enhancing Task Bot Engagement with Synthesized Open-Domain Dialog. In Proceedings of the 24th Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 496–508, Prague, Czechia. 

• Ethan A. Chi, Ashwin Paranjape, Abigail See, Caleb Chiam, Trenton Chang, Kathleen Kenealy, Swee Kiat Lim, Amelia Hardy, Chetanya Rastogi, Haojun Li, Alexander Iyabor, Yutong He, Hari Sowrirajan, Peng Qi, Kaushik Ram Sadagopan, Nguyet Minh Phu, Dilara Soylu, Jillian Tang, Avanika Narayan, Giovanni Campagna, and Christopher Manning. 2022. Neural Generation Meets Real People: Building a Social, Informative Open-Domain Dialogue Agent. In Proceedings of the 23rd Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 376–395, Edinburgh, UK. 

• Stephen Roller, Emily Dinan, Naman Goyal, Da Ju, Mary Williamson, Yinhan Liu, Jing Xu, Myle Ott, Eric Michael Smith, Y-Lan Boureau, and Jason Weston. 2021. Recipes for Building an Open-Domain Chatbot. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 300–325, Online. 

• Baolin Peng, Chunyuan Li, Jinchao Li, Shahin Shayandeh, Lars Liden, and Jianfeng Gao. 2021. Soloist: Building Task Bots at Scale with Transfer Learning and Machine Teaching. Transactions of the Association for Computational Linguistics, 9:807–824. 

Week 4: Traum - Information State, Plan and Logic-based approaches to Dialogue  (questions should be posted on Piazza by January 30, 11:59 pm)

Required

• David Traum and Staffan Larsson, The Information State Approach to Dialogue Management in Current and New Directions in Discourse and Dialogue, Ed. Jan van Kuppevelt and Ronnie Smith, Kluwer, 2003, pp 325-354.

• Perrault and Allen A plan-based analysis of indirect speech acts. Computational Linguistics, 6:167-183, 1980

• Rich, C.; Sidner, C.L.; Lesh, N.B., "COLLAGEN: Applying Collaborative Discourse Theory to Human-Computer Interaction", Artificial Intelligence Magazine, Winter 2001 (Vol 22, Issue 4, pps 15-25)

• Sadek, M. D., Bretier, P., & Panaget, F. (1997). ARTIMIS: Natural dialogue meets rational agency. IJCAI (2), 1030, 1035.

Optional

• Rich, C.; Sidner, C.L., "COLLAGEN: A Collaboration Manager for Software Interface Agents", An International Journal: User Modeling and User-Adapted Interaction, Vol. 8, Issue 3/4, pps 315-350, 1998

• The TRAINS Project James F. Allen et al. Journal of Experimental and Theoretical AI, 1995.

• Colin Matheson, Massimo Poesio, and David Traum, Modelling Grounding and Discourse Obligations Using Update Rules, in Proceedings of the 1st Annual Meeting of the North American Association for Computational Linguistics (NAACL2000), May 2000.

• David Traum and Jeff Rickel, Embodied Agents for Multi-party Dialogue in Immersive Virtual World in proceedings of the first International Joint Conference on Autonomous Agents and Multi-agent Systems (AAMAS 2002), pp. 766-773, July 2002.

• Smith, D.R. Hipp, and A.W. Biermann. An Architecture for Voice Dialog Systems Based on Prolog-Style Theorem Proving. Computational Linguistics 21:3, 1995.

• J. Bos and T. Oka. 2002. An Inference-based Approach to Dialogue System Design. In COLING 2002. Proceedings of the 19th International Conference on Computational Linguistics, pages 113–119, Taipei.

Week 5: Traum - Identity in Dialogue Systems: Role-play Dialogue Systems, Systems  Representing Real People, Storytelling in Dialogue (questions should be posted on Piazza by February 6, 11:59 pm)

Required

• Kathryn J. Collins and David Traum,Towards a multi-dimensional taxonomy of stories in dialogue in proceedings of the Language Resources and Evaluation Conference (LREC) pp. 118-124, 2016.

• Sarah Fillwock and David Traum, Identification of Personal Information Shared in Chat-Oriented Dialogue in Proceedings of the LREC 2018 conference.

• Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston. 2018. Personalizing Dialogue Agents: I have a dog, do you have pets too?. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2204–2213, Melbourne, Australia. Association for Computational Linguistics.

• David Traum, Andrew Jones, Kia Hays, Heather Maio, Oleg Alexander, Ron Artstein, Paul Debevec, Alesia Gainer, Kallirroi Georgila, Kathleen Haase, Karen Jungblut, Anton Leuski, Stephen Smith, and William Swartout. New Dimensions in Testimony: Digitally Preserving a Holocaust Survivor's Interactive Storytelling. In Proceedings of the 8th International Conference on Interactive Digital Storytelling (ICIDS), Copenhagen, Denmark. Lecture Notes in Computer Science, Vol. 9445, pp. 269-281, Springer International Publishing Switzerland, 2015. Best Paper Award

Optional

• Ruth Aylett. Interactive Narrative and Story-telling Chapter 26 in of The Handbook on Socially Interactive Agents (Volume 2) 2022. 

• Setareh Nasihati Gilani, Kraig Sheetz, Gale Lucas, and David Traum, What Kind of Stories Should a Virtual Human Swap? in proceedings of IVA 2016 conference, 2016.

• David DeVault, Ron Artstein, Grace Benn, Teresa Dey, Ed Fast, Alesia Gainer, Kallirroi Georgila, Jon Gratch, Arno Hartholt, Margaux Lhommet, Gale Lucas, Stacy Marsella, Fabrizio Morbini, Angela Nazarian, Stefan Scherer, Giota Stratou, Apar Suri, David Traum, Rachel Wood, Yuyu Xu, Albert Rizzo, and Louis-Philippe Morency. SimSensei kiosk: A virtual human interviewer for healthcare Decision Support. In  Proceedings of the 13th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2014), pages 1061–1068, Paris, May 2014.

• Mateas, Michael, and Andrew Stern. "Structuring content in the Façade interactive drama architecture." Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment. Vol. 1. No. 1. 2005.

• Chris Callison-Burch, Gaurav Singh Tomar, Lara Martin, Daphne Ippolito, Suma Bailis, and David Reitter. 2022. Dungeons and Dragons as a Dialog Challenge for Artificial Intelligence. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 9379–9393, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.

• Yunfan Shao, Linyang Li, Junqi Dai, and Xipeng Qiu. 2023. Character-LLM: A Trainable Agent for Role-Playing. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 13153–13187, Singapore. Association for Computational Linguistics.

• Ryuichiro Higashinaka, Masahiro Mizukami, Hidetoshi Kawabata, Emi Yamaguchi, Noritake Adachi, and Junji Tomita. 2018. Role play-based question-answering by real users for building chatbots with consistent personalities. In Proceedings of the 19th Annual SIGdial Meeting on Discourse and Dialogue, pages 264–272, Melbourne, Australia. Association for Computational Linguistics.

Week 6: Georgila - Reinforcement learning and simulated users for dialogue management (Part 1) (questions should be posted on Piazza by February 13, 11:59 pm)

Required

• Jason D. Williams and Steve Young. Scaling POMDPs for spoken dialog management. IEEE Transactions on Audio, Speech, and Language Processing, 15(7):2116-2129, 2007.

• Kallirroi Georgila, James Henderson, and Oliver Lemon. User Simulation for Spoken Dialogue Systems: Learning and Evaluation. Proceedings of the Annual Conference of the International Speech Communication Association (INTERSPEECH), pp. 1065-1068, Pittsburgh, USA, 2006.

• Oliver Lemon, Kallirroi Georgila, and James Henderson. Evaluating Effectiveness and Portability of Reinforcement Learned Dialogue Strategies with Real Users: The TALK TownInfo Evaluation. Proceedings of the IEEE/ACL Workshop on Spoken Language Technology (SLT), pp. 178-181, Aruba, 2006.

• Jost Schatzmann and Steve Young. The Hidden Agenda User Simulation Model. IEEE Transactions on Audio, Speech, and Language Processing, 17(4):733-747, 2009.

Optional

• James Henderson, Oliver Lemon, and Kallirroi Georgila. Hybrid Reinforcement/Supervised Learning of Dialogue Policies from Fixed Datasets. Computational Linguistics, 34(4):487-511, MIT Press, 2008.

• Jason D. Williams and Steve Young. Partially observable Markov decision processes for spoken dialog systems. Computer Speech and Language, 21:393-422, 2007

• Milica Gasic and Steve Young. Gaussian processes for POMDP-based dialogue manager optimization. IEEE Transactions on Audio, Speech, and Language Processing, 22(1):28-40, 2014.

• Jost Schatzmann, Kallirroi Georgila, and Steve Young. Quantitative Evaluation of User Simulation Techniques for Spoken Dialogue Systems. Proceedings of the 6th Annual SIGdial Meeting on Discourse and Dialogue (SIGDIAL), pp. 45-54, Lisbon, Portugal, 2005.

• Kallirroi Georgila, James Henderson, and Oliver Lemon. Learning User Simulations for Information State Update Dialogue Systems. Proceedings of the Annual Conference of the International Speech Communication Association (INTERSPEECH), pp. 893-896, Lisbon, Portugal, 2005.

• Ramesh Manuvinakurike, David DeVault, and Kallirroi Georgila. Using Reinforcement Learning to Model Incrementality in a Fast-Paced Dialogue Game. Proceedings of the 18th Annual SIGdial Meeting on Discourse and Dialogue (SIGDIAL), pp. 331-341, Saarbruecken, Germany, 2017.

• Alexandros Papangelis and Kallirroi Georgila. Reinforcement Learning of Multi-Issue Negotiation Dialogue Policies. Proceedings of the 16th Annual SIGdial Meeting on Discourse and Dialogue (SIGDIAL), pp. 154-158, Prague, Czech Republic, 2015.

• Kallirroi Georgila, Maria Wolters, and Johanna D. Moore. Simulating the Behaviour of Older versus Younger Users when Interacting with Spoken Dialogue Systems. Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics - Human Language Technologies (ACL-HLT), Short Papers, pp. 49-52, Columbus, Ohio, USA, 2008.

Week 7: Georgila - Reinforcement learning and simulated users for dialogue management (Part 2) (questions should be posted on Piazza by February 20, 11:59 pm)

Required

• Florian L. Kreyssig, Inigo Casanueva, Pawel Budzianowski, and Milica Gasic. Neural User Simulator for Corpus-based Policy Optimisation for Spoken Dialogue Systems. SIGDIAL 2018.

• Layla El Asri, Jing He, and Kaheer Suleman. A Sequence-to-Sequence Model for User Simulation in Spoken Dialogue Systems. Interspeech 2016.

• Pei-Hao Su, Paweł Budzianowski, Stefan Ultes, Milica Gasic, and Steve Young. Sample-efficient Actor-Critic Reinforcement Learning with Supervised Data for Dialogue Management. SIGDIAL 2017.

• Baolin Peng, Xiujun Li, Jianfeng Gao, Jingjing Liu, and Kam-Fai Wong. Deep Dyna-Q: Integrating Planning for Task-Completion Dialogue Policy Learning. ACL 2018.

Optional

• Lu Chen, Zhi Chen, Bowen Tan, Sishan Long, Milica Gasic, and Kai Yu. AgentGraph: Towards Universal Dialogue Management with Structured Deep Reinforcement Learning. IEEE/ACM Transactions on Audio, Speech, and Language Processing 2019.

• Heriberto Cuayahuitl, Simon Keizer, and Oliver Lemon. Strategic Dialogue Management via Deep Reinforcement Learning. NIPS Workshop on Deep Reinforcement Learning 2015.

• Pararth Shah, Dilek Hakkani-Tur, and Larry Heck. Interactive reinforcement learning for task-oriented dialogue management.

• Baolin Peng, Xiujun Li, Lihong Li, Jianfeng Gao, Asli Celikyilmaz, Sungjin Lee, and Kam-Fai Wong. Composite Task-Completion Dialogue Policy Learning via Hierarchical Deep Reinforcement Learning. EMNLP 2017.

Week 8: Traum - Multimodal Dialogue, Identity and Story in Dialogue (questions should be posted on Piazza by February 27, 11:59 pm. Note: the first 3 required readings are carried over from week 5 - it's not necessary to send in more questions for these if you already sent them then)

Required

• Kathryn J. Collins and David Traum, Towards a multi-dimensional taxonomy of stories in dialogue in proceedings of the Language Resources and Evaluation Conference (LREC) pp. 118-124, 2016.

• Sarah Fillwock and David Traum, Identification of Personal Information Shared in Chat-Oriented Dialogue in Proceedings of the LREC 2018 conference.

• Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston. 2018. Personalizing Dialogue Agents: I have a dog, do you have pets too?. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2204–2213, Melbourne, Australia. Association for Computational Linguistics.

• David Traum and L. P. Morency, Integration of Visual Perception in Dialogue Understanding for Virtual Humans in Multi-Party interaction in proceedings of AAMAS International Workshop on Interacting with ECAs as Virtual Characters, May 2010.

• Stephanie M. Lukin, Felix Gervits, Cory Hayes, Pooja Moolchandani, Anton Leuski, John Rogers, Carlos Sanchez Amaro, Matthew Marge, Clare Voss, David Traum ScoutBot: A Dialogue System for Collaborative Navigation Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics-System Demonstrations, pages 93–98 Melbourne, Australia, July 15 - 20, 2018.

Optional (see also week 5 optional readings)

• Clark, H. H., and Brennan, S. A. (1991). Grounding in communication. In L.B. Resnick, J.M. Levine, & S.D. Teasley (Eds.). Perspectives on socially shared cognition . Washington: APA Books.

• Nakano, Y. I., Reinstein, G., Stocky, T., & Cassell, J. (2003, July). Towards a model of face-to-face grounding. In Proceedings of the 41st annual meeting of the Association for Computational Linguistics (pp. 553-561).

• Dillenbourg, Pierre, and David Traum. "Sharing solutions: Persistence and grounding in multimodal collaborative problem solving." The Journal of the Learning Sciences 15.1 (2006): 121-151.

• Belpaeme, T., Baxter, P., Read, R., Wood, R., Cuayáhuitl, H., Kiefer, B., Racioppa, S., Kruijff-Korbayová, I., Athanasopoulos, G., Enescu, V. and Looije, R., 2013. Multimodal child-robot interaction: Building social bonds. Journal of Human-Robot Interaction, 1(2), pp.33-53.

• Robots That Learn to Communicate: A Developmental Approach to Personally and Physically Situated Human-Robot Conversations N. Iwahashi, K. Sugiura, R. Taguchi, T. Nagai, and T. Taniguchi, In Proc. The 2010 AAAI Fall Symposium on Dialog with Robots, November 11-13, 2010, Arlington, Virginia, USA, pp. 38-43.

• Where to look: a study of human-robot engagement Candace L. Sidner, Cory D. Kidd, Christopher Lee, Neal Lesh, in Proc. of IUI 2004.

Week 9: Core - Dialogue Systems for Education (questions should be posted on Piazza by March 5, 11:59 pm)

Required

• Kallirroi Georgila, Mark G. Core, Benjamin D. Nye, Shamya Karumbaiah, Daniel Auerbach, and Maya Ram. Using Reinforcement Learning to Optimize the Policies of an Intelligent Tutoring System for Interpersonal Skills Training. Proceedings of the 18th International Conference on Autonomous Agents and Multiagent Systems (AAMAS), 2019.

• Benjamin D. Nye, Dillon Mee, and Mark G. Core. Generative Large Language Models for Dialog-Based Tutoring: An Early Consideration of Opportunities and Concerns. AIED Workshop on Empowering Education with LLMs – the Next-Gen Interface and Content Generation, 2023.

• Benjamin D. Nye, Rushit Sanghrajka, Vinit Bodhwani, Martin Acob, Daniel Budziwojski, Kayla Carr, Larry Kirschner, and William R. Swartout. OpenTutor: Designing a Rapid-Authored Tutor that Learns as you Grade. The International FLAIRS Conference Proceedings, 2021.

Optional

• Min Chi, Kurt VanLehn, Diane Litman, and Pamela Jordan. An Evaluation of Pedagogical Tutorial Tactics for a Natural Language Tutoring System: A Reinforcement Learning Approach. International Journal of Artificial Intelligence in Education, 21(2):83-113, 2011.

• Diane Litman, Heather Friedberg, and Kate Forbes-Riley. Prosodic Cues to Disengagement and Uncertainty in Physics Tutorial Dialogues. Proceedings of the 13th Annual Conference of the International Speech Communication Association (Interspeech), 2012.

• Mark G. Core, Johanna D. Moore, and Claus Zinn. (2003). The Role of Initiative in Tutorial Dialogue. Proceedings of the 10th Conference of the European Chapter of the Association for Computational Linguistics (EACL), 2003.

• Diane Litman, Johanna Moore, Myroslava O. Dzikovska, and Elaine Farrow. Using Natural Language Processing to Analyze Tutorial Dialogue Corpora Across Domains and Modalities. Proceedings of the 14th International Conference on Artificial Intelligence in Education (AIED), 2010.

• Myroslava O. Dzikovska, Johanna D. Moore, Natalie Steinhauser, Gwendolyn Campbell, Elaine Farrow, and Charles B. Callaway. BEETLE II: a system for tutoring and computational linguistics experimentation. Proceedings of the ACL System Demonstrations, 2010.

Week 10: Georgila - Speech recognition and speech synthesis for dialogue 

Speech synthesis for dialogue (Xuan Shi & Daniel Yang)

(questions should be posted on Piazza by March 19, 11:59 pm)

Required

• Kun Wei, Yike Zhang, Sining Sun, Lei Xie, and Long Ma. Conversational speech recognition by learning conversation-level characteristics. ICASSP 2022.

• Suyoun Kim and Florian Metze. Dialog-context aware end-to-end speech recognition. SLT 2018.

• Kentaro Mitsui, Tianyu Zhao, Kei Sawada, Yukiya Hono, Yoshihiko Nankaku, and Keiichi Tokuda. End-to-end text-to-speech based on latent representation of speaking styles using spontaneous dialogue. Interspeech 2022.

• Eva Szekely, Gustav Eje Henter, Jonas Beskow, and Joakin Gustafson. Spontaneous conversational speech synthesis from found data. Interspeech 2019.

Optional

• Wayne Xiong, Jasha Droppo, Xuedong Huang, Frank Seide, Mike Seltzer, Andreas Stolcke, Dong Yu, and Geoffrey Zweig. Toward human parity in conversational speech recognition. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 25(12):2410-2423, 2017.

• Wayne Xiong, Lingfeng Wu, Jun Zhang, and Andreas Stolcke. Session-level Language Modeling for Conversational Speech. EMNLP 2018.

• Kallirroi Georgila, Anton Leuski, Volodymyr Yanov, and David Traum. Evaluation of Off-the-shelf Speech Recognizers Across Diverse Dialogue Domains. LREC 2020.

• Jian Cong, Shan Yang, Na Hu, Guangzhi Li, Lei Xie, and Dan Su. Controllable context-aware conversational speech synthesis. Interspeech 2021.

• Johannah O’Mahony, Catherine Lai, and Simon King. Synthesising turn-taking cues using natural conversational data. Speech Synthesis Workshop 2023.

• Elijah Gutierrez, Pilar Oplustil-Gallegos, and Catherine Lai. Location, location: Enhancing the evaluation of text-to-speech synthesis using the rapid prosody transcription paradigm. Speech Synthesis Workshop 2019.

• Zhifeng Kong, Arushi Goel, Rohan Badlani, Wei Ping, Rafael Walle, and Bryan Catanzaro. Audio Flamingo: A novel audio language model with few-shot learning and dialogue abilities. arXiv:2402.01831 2024. - part of the student topic presentation

• Heeseung Kim, Soonshin Seo, Kyeongseok Jeong, Ohsung Kwon, Jungwhan Kim, Jaehong Lee, Eunwoo Song, Myungwoo Oh, Sungroh Yoon, and Kang Min Yoo. Unified speech-text pre-training for spoken dialog modeling. arXiv:2402.05706 2024. - part of the student topic presentation

Week 11: Traum - Data Collection and Evaluation

Multi-party dialogue (Khoi Pham) 

Non-cooperative dialogue systems (negotiation, deception) (Ritvik Nimmagadda)

 (questions should be posted on Piazza by March 26, 11:59 pm)

Required

• Ai, H., Raux, A., Bohus, D., Eskenazi, M., and Litman, D. (2007). Comparing spoken dialog corpora collected with recruited subjects versus real users.Proceedings of the 8th SIGDial Workshop on Discourse and Dialogue (SIGdial 2007).

• Chia-Wei Liu, Ryan Lowe, Iulian V. Serban, Michael Noseworthy, Laurent Charlin, Joelle Pineau, How NOT To Evaluate Your Dialogue System: An Empirical Study of Unsupervised Evaluation Metrics for Dialogue Response Generation Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 2122–2132, Austin, Texas, November 1-5, 2016.

• David Traum Issues in multi-party dialogues, in Advances in Agent Communication Ed. F. Dignum, Springer-Verlag LNAI 2922 pp 201-211, 2004. - part of the student topic presentation

• D. R. Traum, W. Swartout, J Gratch, S Marsella, A Virtual Human Dialogue Model for Non-team Interaction, in Recent Trends in Discourse and DialogueSpringer, Laila Dybkjaer and Wolfgang Minker, Eds, pp. 45--67, 2008. - part of the student topic presentation

Optional

Evaluation

• Mehri, Shikib, and Maxine Eskenazi. "Unsupervised Evaluation of Interactive Dialog with DialoGPT." Proceedings of the 21th Annual Meeting of the Special Interest Group on Discourse and Dialogue. 2020.

• Ron Artstein. Inter-annotator agreement In Handbook of Linguistic Annotation, edited by Nancy Ide and James Pustejovsky, pages 297–313. Springer, Dordrecht, 2017.

• Marilyn A. Walker, Candace Kamm and Diane J. Litman. Towards Developing General Models of Usability with PARADISE.  Natural Language Engineering 2000.

• Hone, K. S., and Graham, R. (2000). Towards a tool for the subjective assesment of speech system interfaces (SASSI). Nat. Lang. Eng. 6(3/4), pp. 287–303.

• Cross-Site Evaluation in DARPA Communicator: The June 2000 Data Collection Submitted to  Computer Speech and Language , 2002.

• Bohlin P, Bos J, Larsson S, Lewin I, Matheson C, Milward D. Survey of existing interactive systems. Deliverable D1.3, TRINDI Project, LE4-8314; 1999.

• Sudeep Gandhe and David Traum Evaluation Understudy for Dialogue Coherence Models In proceedings of The 9th SIGdial Workshop on Discourse and Dialogue (SIGdial 2008), June, 2008.

• Sebastian Moller Assessment and Evaluation of Speech-Based Interactive Systems: From Manual Annotation to Automatic Usability Evaluation Chapter 15 of Speech Technology, Fang Chen, ed., Springer, 2010.

Multiparty

• Mahajan, Khyati, and Samira Shaikh. "On the need for thoughtful data collection for multi-party dialogue: A survey of available corpora and collection methods."Proceedings of the 22nd annual meeting of the special interest group on discourse and dialogue. 2021. - part of the student topic presentation

• Bohus, D, Horvitz, E. (2009) Models for Multiparty Engagement in Open-World Dialog, in Proceedings of SIGdial '09, London, UK SIGdial'09 best paper award

• Jia-Chen Gu, Chongyang Tao, and Zhen-Hua Ling. Who says what to whom: A survey of multi- party conversations. In Lud De Raedt, editor, Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence, IJCAI-22, pages 5486–5493. International Joint Confer- ences on Artificial Intelligence Organization, 7 2022b. doi: 10.24963/ijcai.2022/768. URL https://doi.org/10.24963/ijcai.2022/768. Survey Track.

Non-Cooperative Dialogue

• Guhe, M. and A. Lascarides Trading in a Multiplayer Board Game: Towards an Analysis of Non-Cooperative Dialogue, Proceedings of Cognitive Science, Tokyo. 2012. - part of the student topic presentation

• Anthony Sicilia, Tristan Maidment, Pat Healy, Malihe Alikhani; Modeling Non-Cooperative Dialogue: Theoretical and Empirical Insights. Transactions of the Association for Computational Linguistics 2022; 10 1084–1102. doi: https://urldefense.com/v3/__https://doi.org/10.1162/tacl_a_00507__;!!LIr3w8kk_Xxm!rDKvCMIrkulqYllshlJWK7zc6VKzZ8AtcE7XgwH9VIMGYPL25FahL1ibtSbFDVmH4-ovbNufmrS3s8Ax$

• Georgila K. and Traum D. Reinforcement Learning of Argumentation Dialogue Policies in Negotiation. In Proceedings of the 12th Annual Conference of the International Speech Communication Association (INTERSPEECH), Florence, Italy, 2011.

• Lewis, Mike, et al. "Deal or No Deal? End-to-End Learning of Negotiation Dialogues." Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. 2017.

Week 12: Georgila - Natural language understanding and dialogue state tracking, natural language generation

Natural language generation for dialogue (Hossein Entezari Zarch) 

Dialogue act recognition (Hirona Arai)

Dialogue systems for language learning (Eric Boxer)

 (questions should be posted on Piazza by April 2, 11:59 pm)

Required

• Piotr Zelasko, Raghavendra Pappagari, and Najim Dehak. What Helps Transformers Recognize Conversational Structure? Importance of Context, Punctuation, and Labels in Dialog Act Recognition. Transactions of the Association for Computational Linguistics, 9:1163-1179, 2021. - part of the student topic presentation

• Baolin Peng, Chenguang Zhu, Chunyuan Li, Xiujun Li, Jinchao Li, Michael Zeng, and Jianfeng Gao. Few-shot Natural Language Generation for Task-Oriented Dialog. Findings of the Association for Computational Linguistics: EMNLP, 2020. - part of the student topic presentation

• Li, Kuo-Chen, Maiga Chang, and Kuan-Hsing Wu. "Developing a task-based dialogue system for English language learning." Education Sciences 10.11 (2020): 306. - part of the student topic presentation

• Ramesh Manuvinakurike, Trung Bui, Walter Chang, and Kallirroi Georgila. Conversational Image Editing: Incremental Intent Identification in a New Dialogue Task. Proceedings of the 19th Annual SIGdial Meeting on Discourse and Dialogue (SIGDIAL), 2018. – Best paper award

Optional

Natural language understanding, dialogue act recognition, dialogue state tracking, natural language generation (some papers include multiple topics)

• Chandrakant Bothe, Cornelius Weber, Sven Magg, and Stefan Wermter. A context-based approach for dialogue act recognition using simple recurrent neural networks. Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC), 2018. - part of the student topic presentation

• Viet-Trung Dang, Tianyu Zhao, Sei Ueno, Hirofumi Inaguma, and Tatsuya Kawahara. End-to-end speech-to-dialog-act recognition. Proceedings of Interspeech, 2020.

• Pierre Colombo, Emile Chapuis, Matteo Manica, Emmanuel Vignon, Giovanna Varni, and Chloe Clavel. Guiding attention in sequence-to-sequence act prediction models for dialogue. Proceedings of AAAI, 2020.

• Yang Liu, Kun Han, Zhao Tan, and Yun Lei. Using context information for dialog act classification in DNN framework. Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), 2017.

• Daniel Ortega and Ngoc Thang Vu. Neural-based context representation learning for dialog act classification. Proceedings of the 18th Annual SIGdial Meeting on Discourse and Dialogue (SIGDIAL), 2017.

• Ali Ahmadvand, Jason Ingyu Choi, and Eugene Agichtein. Contextual Dialogue Act Classification for Open-Domain Conversational Agents. Proceedings of SIGIR, 2019.

• Arshit Gupta, Peng Zhang, Garima Lalwani, and Mona Diab. CASA-NLU: Context-Aware Self-Attentive Natural Language Understanding for Task-Oriented Chatbots. Proceedings of the Conference on Empirical Methods in Natural Language Processing and the International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), 2019. 

• Chien-Sheng Wu, Steven C.H. Hoi, Richard Socher, and Caiming Xiong. 2020. TOD-BERT: Pre-trained Natural Language Understanding for Task-Oriented Dialogue. Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), 2020.

• Jinyu Guo, Kai Shuang, Jijie Li, Zihan Wang, and Yixuan Liu. Beyond the Granularity: Multi-Perspective Dialogue Collaborative Selection for Dialogue State Tracking. Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), 2022. 

• Yixuan Su, Lei Shu, Elman Mansimov, Arshit Gupta, Deng Cai, Yi-An Lai, and Yi Zhang. Multi-Task Pre-Training for Plug-and-Play Task-Oriented Dialogue System. Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), 2022.

• Lu Chen, Boer Lv, Chi Wang, Su Zhu, Bowen Tan, and Kai Yu. Schema-Guided Multi-Domain Dialogue State Tracking with Graph Attention Neural Networks. Proceedings of AAAI, 2020. 

• Chenguang Zhu, Michael Zeng, and Xuedong Huang. Multi-task Learning for Natural Language Generation in Task-Oriented Dialogue. Proceedings of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), 2019. - part of the student topic presentation

• Mihir Kale and Abhinav Rastogi. Template Guided Text Generation for Task-Oriented Dialogue. Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), 2020. 

Language learning

• Bibauw, Serge, Thomas François, and Piet Desmet. "Dialogue systems for language learning: Chatbots and beyond." The Routledge handbook of second language acquisition and technology. Routledge, 2022. 121-135. - part of the student topic presentation

• Brixey, Jacqueline, and David Traum. "Masheli: A Choctaw-English Bilingual Chatbot." Conversational Dialogue Systems for the Next Decade (2021): 41-50.

• Jose Lopes, Olov Engwall, Gabriel Skantze, A First Visit to the Robot Language Cafe in proceedings of SLaTE 2017 Workshop.

• Lewis, W. (2010). Serious use of a serious game for language learning. International Journal of Artificial Intelligence in Education, 20(2), 175-195.

• Raux, Antoine / Eskenazi, Maxine  Using task-oriented spoken dialogue systems for language learning: potential, practical applications and challenges, In ICALL-2004. - part of the student topic presentation

• Alistair Knott and Peter Vlugter. 2008. Multi-agent human-machine dialogue: issues in dialogue management and referring expression semantics. Artif. Intell., 172(2-3):69–102.

• Manny Rayner, Claudia Baur, Cathy Chua, Nikos Tsourakis Supervised Learning of Response Grammars in a Spoken Call System SLaTE 2015 Workshop.

• Hazel Morton, Nancie Gunson, and Mervyn Jack, Interactive Language Learning through Speech-Enabled Virtual Scenarios Advances in Human-Computer Interaction, vol. 2012, Article ID 389523, 14 pages, 2012.

• Wang N., Johnson W.L. (2008) The Politeness Effect in an Intelligent Foreign Language Tutoring System. In: Woolf B.P., Aïmeur E., Nkambou R., Lajoie S. (eds) Intelligent Tutoring Systems. ITS 2008. Lecture Notes in Computer Science, vol 5091. Springer, Berlin, Heidelberg.

• Veronika Timpe-Laughlin, Keelan Evanini, Ashley Green, Ian Blood, Judit Dombi and Vikram Ramanarayanan (2017). Designing interactive, automated dialogues for L2 pragmatics learning, in proceedings of: 21st Workshop on the Semantics and Pragmatics of Dialogue (SemDial 2017 - SaarDial), Saarbrucken, Germany, Aug 2017.

Week 13

Empathetic dialogue systems with reinforcement learning (Ala Tak & Alireza Ziabari) 

Storytelling in dialogue (Ankur Chemburkar & Siraj Sandhu)

Culture-specific dialogue systems (Yubo Zhang)

Chat dialogue (Philipp Eibl & Taiwei Shi)

 (questions should be posted on Piazza by April 9, 11:59 pm)

Required

• Ashish Sharma, Inna W. Lin, Adam S. Miner, David C. Atkins, and Tim Althoff. Towards Facilitating Empathic Conversations in Online Mental Health Support: A Reinforcement Learning Approach. Web Conference 2021. - part of the student topic presentation

• Siqi Bao, Huang He, Fan Wang, Hua Wu, Haifeng Wang, Wenquan Wu, Zhen Guo, Zhibin Liu, and Xinchao Xu. PLATO-2: Towards Building an Open-Domain Chatbot via Curriculum Learning. ACL Findings 2021. - part of the student topic presentation

• Kathryn J. Collins and David Traum. Towards a multi-dimensional taxonomy of stories in dialogue. In Proceedings of the Language Resources and Evaluation Conference (LREC) pp. 118-124, 2016. - part of the student topic presentation 

• Qiu, L., Zhao, Y., Li, J., Lu, P., Peng, B., Gao, J., & Zhu, S. C. (2022, June). Valuenet: A new dataset for human value driven dialogue system. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 36, No. 10, pp. 11183-11191). - part of the student topic presentation

Optional

Empathetic dialogue systems with reinforcement learning

• Jinfeng Zhou, Zhuang Chen, Bo Wang, and Minlie Huang. Facilitating Multi-turn Emotional Support Conversation with Positive Emotion Elicitation: A Reinforcement Learning Approach. ACL 2023.

• Lei Shen and Yang Feng. CDL: Curriculum dual learning for emotion-controllable response generation. ACL 2020.

• Jia Li, Xiao Sun, Xing Wei, Changliang Li, and Jianhua Tao. Reinforcement Learning Based Emotional Editing Constraint Conversation Generation. arXiv 2019.

• Xiao Sun, Jia Li, Xing Wei, Changliang Li, and Jianhua Tao. Emotional editing constraint conversation content generation based on reinforcement learning. Information Fusion 2020.

Chat dialogue

• Ethan A. Chi, Ashwin Paranjape, Abigail See, Caleb Chiam, Trenton Chang, Kathleen Kenealy, Swee Kiat Lim, Amelia Hardy, Chetanya Rastogi, Haojun Li, Alexander Iyabor, Yutong He, Hari Sowrirajan, Peng Qi, Kaushik Ram Sadagopan, Nguyet Minh Phu, Dilara Soylu, Jillian Tang, Avanika Narayan, Giovanni Campagna, and Christopher Manning. 2022. Neural Generation Meets Real People: Building a Social, Informative Open-Domain Dialogue Agent. In Proceedings of the 23rd Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 376–395, Edinburgh, UK. 

• Stephen Roller, Emily Dinan, Naman Goyal, Da Ju, Mary Williamson, Yinhan Liu, Jing Xu, Myle Ott, Eric Michael Smith, Y-Lan Boureau, and Jason Weston. 2021. Recipes for Building an Open-Domain Chatbot. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 300–325, Online.

• Bin Sun, Yitong Li, Fei Mi, Weichao Wang, Yiwei Li, and Kan Li. Towards Diverse, Relevant and Coherent Open-Domain Dialogue Generation via Hybrid Latent Variables. AAAI 2023.

• Omar Shaikh, Kristina Gligoric, Ashna Khetan, Matthias Gerstgrasser, Diyi Yang, and Dan Jurafsky. Grounding gaps in language model generations. NAACL 2024.

Storytelling in dialogue

• Ruth Aylett. Interactive Narrative and Story-telling Chapter 26 in of The Handbook on Socially Interactive Agents (Volume 2) 2022.

• Setareh Nasihati Gilani, Kraig Sheetz, Gale Lucas, and David Traum, What Kind of Stories Should a Virtual Human Swap? in proceedings of IVA 2016 conference, 2016.

• Mateas, Michael, and Andrew Stern. "Structuring content in the Façade interactive drama architecture." Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment. Vol. 1. No. 1. 2005.

• Eric Nichols, Leo Gao, and Randy Gomez. 2020. Collaborative Storytelling with Large-scale Neural Language Models. In Proceedings of the 13th ACM SIGGRAPH Conference on Motion, Interaction and Games (MIG '20). Association for Computing Machinery, New York, NY, USA, Article 17, 1–10.

• E. Nichols, L. Gao, Y. Vasylkiv and R. Gomez, "Collaborative Storytelling with Social Robots," 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Prague, Czech Republic, 2021, pp. 1903-1910.

• E. Nichols, D. Szapiro, Y. Vasylkiv and R. Gomez, "I Can’t Believe That Happened! : Exploring Expressivity in Collaborative Storytelling with the Tabletop Robot Haru," 2022 31st IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), Napoli, Italy, 2022, pp. 59-59.

Culture-specific dialogue systems

• Haolan Zhan, Zhuang Li, Yufei Wang, Linhao Luo, Tao Feng, Xiaoxi Kang, Yuncheng Hua, Lizhen Qu, Lay-Ki Soon, Suraj Sharma, Ingrid Zukerman, Zhaleh Semnani-Azad, and Gholamreza Haffari. 2023. SocialDial: A Benchmark for Socially-Aware Dialogue Systems. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '23). Association for Computing Machinery, New York, NY, USA, 2712–2722. 

• Lugrin, B., Rehm, M. (2021). Culture for Socially Interactive Agents. In B. Lugrin, C. Pelachaud, D. Traum (Eds.), Handbook on Socially Interactive Agents – 20 Years of Research on Embodied Conversational Agents, Intelligent Virtual Agents, and Social Robotics, Volume 1: Methods, Behavior, Cognition (pp. 463-493). ACM.

• Miehle, Juliana, Nicolas Wagner, Wolfgang Minker, and Stefan Ultes. "Culture-Aware Dialogue Management for Conversational Assistants." Conversational Dialogue Systems for the Next Decade (2021): 103-115.

• Elnaz Nouri, Kallirroi Georgila, David Traum. Culture-specific models of negotiation for virtual characters: multi-attribute decision-making based on culture-specific values AI & Society, 32:1, pp. 51--63, February 2017.

• Elnaz Nouri, Kallirroi Georgila, and David Traum. A Cultural Decision-Making Model for Negotiation Based on Inverse Reinforcement Learning. In Proceedings of the 34th Annual Meeting of the Cognitive Science Society (CogSci), pp. 2097-2102, Sapporo, Japan, 2012.

• Kallirroi Georgila and David Traum. Learning Culture-Specific Dialogue Models from Non-Culture Specific Data. In Proceedings of Universal Access in Human-Computer Interaction, HCI International, Orlando, Florida, USA. Lecture Notes in Computer Science, Vol. 6766, pp. 440-449, Springer Berlin Heidelberg, 2011.

• Kallirroi Georgila and David Traum. Reinforcement Learning of Argumentation Dialogue Policies in Negotiation. In Proceedings of the 12th Annual Conference of the International Speech Communication Association (INTERSPEECH), pp. 2073-2076, Florence, Italy, 2011. 

• David Herrera, David Novick, Dusan Jan, David Traum, Dialog behaviors across culture and group size, Proceedings of HCI International 2011, July 11-14, 2011, Orlando, FL, Lecture Notes in Computer Science, 2011, Volume 6766/2011, 450-459.

• David Traum,  Models of Culture for Virtual Human Conversation, in Human Computer Interaction International (HCII), July 2009, San Diego.

• Solomon, S., van Lent, M., Core, M., Carpenter, P., & Rosenberg, M. (2008, April). A language for modeling cultural norms, biases and stereotypes for human behavior models. In Proceedings of the Seventeenth Conference on Behavior Representation in Modeling and Simulation (BRIMS).

• D Jan, D Herrera, B Martinovski, D Novick and D Traum A computational Model of Culture-specific Conversational Behavior in proceedings of Intelligent Virtual Agents Conference, pp. 45--56, September, 2007. 

Week 14

Adversarial attacks on LLMs (Mingzhe Wu) 

How dialogue state tracking and dialogue summarization can complement each other (Joseph Wang)

 (questions should be posted on Piazza by April 16, 11:59 pm)

Required

• Jamin Shin, Hangyeol Yu, Hyeongdon Moon, Andrea Madotto, and Juneyoung Park. Dialogue Summaries as Dialogue States (DS2), Template-Guided Summarization for Few-shot Dialogue State Tracking. In Findings of the Association for Computational Linguistics: ACL 2022, pages 3824–3846, Dublin, Ireland. - part of the student topic presentation 

• Kai Greshake, Sahar Abdelnabi, Shailesh Mishra, Christoph Endres, Thorsten Holz, and Mario Fritz. Not what you’ve signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection. arXiv 2023. - part of the student topic presentation

Optional

• Lulu Zhao, Fujia Zheng, Keqing He, Weihao Zeng, Yuejie Lei, Huixing Jiang, Wei Wu, Weiran Xu, Jun Guo, and Fanyu Meng. TODSum: Task-Oriented Dialogue Summarization with State Tracking. arXiv 2021. - part of the student topic presentation

• Bowen Liu, Boao Xiao, Xutong Jiang, Siyuan Cen, Xin He, and Wanchun Dou. Adversarial attacks on large language model-based system and mitigating strategies: A case study on ChatGPT. Security and Communication Networks 2023.