Program

June 11, 2021. All times are PDT, the official time zone of the conference

Click here to join us!

The 2nd Workshop on Data Science with Human-in-the-loop: Language Advances (DaSH-LA)

8:00 – 8:10am: Workshop introduction

Lucian Popa, IBM Research - Almaden

8:10 – 9:00am: Keynote talk 1 (Joyce Chai)

Title: Task Learning through Language Communication in Embodied Agents

Abstract: Language communication plays an important role in human learning and knowledge acquisition. With the emergence of a new generation of embodied AI agents, empowering these agents to learn directly from human partners becomes increasingly important. In this talk, I will give a brief introduction to action learning by following natural language instructions. I will talk about some work done in my lab on interactive task learning where humans can teach agents new tasks through natural language communication and action demonstration. I will highlight the role of physical causality in language grounding to perception and action and discuss cooperative principles in communication grounding between humans and agents.

Session Chair: Yunyao Li, IBM Research - Almaden

9:00 – 10:00am: 1st Session Regular papers (Support for Text Analytics with Human in the Loop / 5 papers)

Krati Saxena, Tushita Singh, Ashwini Patil, Sagar Sunkle and Vinay Kulkarni. Leveraging Wikipedia Navigational Templates for Curating Domain-Specific Fuzzy Conceptual Bases
Verena Weber, Enrico Piovano and Melanie Bradford. It is better to Verify: Semi-Supervised Learning with a human in the loop for large-scale NLU models
Natraj Raman, Sameena Shah, Tucker Balch and Manuela Veloso. ViziTex: Interactive Visual Sense-Making of Text Corpora
Saman Enayati, Ziyu Yang, Slobodan Vucetic and Benjamin Lu. A Visualization Approach for Rapid Labeling of Clinical Notes for Smoking Status Extraction
Saurav Sahay, Eda Okur, Nagib Hakim and Lama Nachman. Semi-supervised Interactive Intent Labeling

Session Chair: Slobodan Vucetic, Temple University

10:00 – 10:10am: Break

10:10 – 10:40am: Invited papers (2 paper highlights from recent conferences focused on Human in the loop NLP)

Jan-Christoph Klie, Richard Eckart de Castilho and Iryna Gurevych. Human-In-The-Loop Entity Linking for Low Resource Domains
Soya Park. Bridging Multi-disciplinary Collaboration Challenges in ML Development via Domain Knowledge Elicitation

Session Chair: Slobodan Vucetic, Temple University

10:40 – 11:40am: 2nd Session Regular papers (Human in the Loop for NLP Tasks/ 5 papers)

Thomas Lippincott and Ben Van Durme. Active learning and negative evidence for language identification
Peter Griggs, Cagatay Demiralp and Sajjadur Rahman. Towards integrated, interactive, and extensible text data analytics with Leam
Karthik Muthuraman, Frederick Reiss, Hong Xu, Bryan Cutler and Zachary Eichenberger. Data Cleaning Tools for Token Classification Tasks
Tatiana Tsygankova, Francesca Marini, Stephen Mayhew and Dan Roth. Building Low-Resource NER Models Using Non-Speaker Annotations
Kayla Duskin, Shivam Sharma, Ji Young Yun, Emily Saldanha and Dustin Arendt. Evaluating and Explaining Natural Language Generation with GenX

Session Chair: Lucian Popa, IBM Research - Almaden

11:40am – 1:10pm: Lunch break + Poster Session

1:10 – 2pm Keynote Talk 2 (Daniel Weld)

Title: Optimizing Human-AI Teams

Abstract: While some AI systems operate in an autonomous mode, many AI systems are deployed in an advisory capacity where a human (eg, loan-officer, judge or doctor) is responsible for the final decision. Should such collaborative AI system be trained in the same way as autonomous AIs, e.g., by solely optimizing their accuracy? We answer "No" – instead arguing for a human-centered AI (HCAI) approach, that makes fundamentally different demands: complementary performance instead of just accuracy, explanations that increase appropriate reliance instead of mere trust, and controls for accepting user feedback. We discuss several novel methods for achieving these goals, present the results of user studies which challenge current directions in explanation generation, and sketch directions for future research.

Session Chair: Eduard Dragut, Temple University

2 – 2:50pm 3rd Session Regular papers (Human in the Loop Tools / 4 papers)

Dustin Arendt, Zhuanyi Shaw, Prasha Shrestha, Ellyn Ayton, Maria Glenski and Svitlana Volkova. CrossCheck: Rapid, Reproducible, and Interpretable Model Evaluation
Raymond Finzel, Esha Singh, Martin Michalowski, Maria Gini and Serguei Pakhomov. Everyday Living Artificial Intelligence Hub
William Lane, Mat Bettinson and Steven Bird. A Computational Model for Interactive Transcription
Rebecca Iglesias-Flores, Megha Mishra, Ajay Patel, Akanksha Malhotra, Reno Kriz, Martha Palmer and Chris Callison-Burch. TopGuNN: Fast NLP Training Data Augmentation using Large Corpora