Invited Speakers


Modern Reinforcement Learning and the Atari 2600

Marc Bellemare, DeepMind

Abstract: Reinforcement learning sprung into the spotlight a few years ago with the development of a super-human Atari 2600 agent embodied in the Deep Q-Network architecture. This talk will offer a retrospective on AI research using the Arcade Learning Environment, the research platform that supported this success, from its inception at a small workshop in Barbados to current fast-paced developments.

Bio: Marc G. Bellemare received his Ph.D. from the University of Alberta, where he led the design of the highly-successful Arcade Learning Environment. He joined then-DeepMind Technologies to help build the first deep reinforcement learning agents. His interests include information theory, online learning, probabilistic modelling, and reinforcement learning. He is currently a senior research scientist at DeepMind.

Thinking Outside the Box: AI Models of Curiosity

Mary Lou Maher, UNC Charlotte

Abstract: Curiosity is a desire to learn or know something. With the increasing use of AI to support design and creativity there arises the potential to go beyond goal based reasoning and deliberately trigger the user’s curiosity. Psychological accounts of curiosity refer to curiosity as a trait and curiosity as a state. In this presentation, the concept of curiosity as state is adopted with a focus on AI models of surprise to stimulate curiosity and ultimately goal reformulation. Cognitive studies of human creative design suggest that problem decomposition, goal formulation and solution search do not happen discretely and sequentially, but iteratively interact as designers re-interpret, re-formulate and solve problems. One documented trigger of this iterative reformulation is unexpected discovery – the ability to surprise oneself with intermediate external representations. This presentation describes computational models of surprise as a basis for stimulating curiosity and goal reformulation. The models have been applied in the context of intrinsically motivated reinforcement learning for agents in a virtual world, designing surprising recipes to encourage the user to be more curious about food, and unexpected topic co-occurrence in text documents to guide the user to think outside their information bubble.

Bio: Mary Lou Maher is Professor and Chair of Software and Information Systems at UNC Charlotte. Dr. Maher’s research interests include computational creativity, design cognition and computing, and CS education. She is Director of the Center for Education Innovation in the College of Computing and Informatics at UNCC and is Principle Investigator on NSF funded projects on cognitive models of curiosity, crowdsourcing design for citizen science, and transforming CS education with the Connected Learner.

Additional keynote details coming soon...