Invited Speakers

John AndersonJohn R. Anderson
R. K. Mellon University Professor of Psychology and Computer Science
Carnegie Mellon University
 
TITLE
Discovering the Structure of Mathematical Problem Solving

ABSTRACT 
It is possible to combine multivariate pattern analysis (MVPA) and hidden Markov models (HMM) to discover the major phases that students go through in solving complex problems. I will illustrate this methodology by applying it to the learning of graphical isomorphs of algebra problems. We discovered a sequence of 5 major phases that students went through: An Orient Phase where they identified the problem to be solved, an Encode Phase where they encoded the needed information, a Compute Phase where they performed the necessary arithmetic calculations, a Transform Phase where they performed any mathematical transformations, and a Respond Phase where they generated the answer. The duration of the Compute and Transform Phases were they only ones that distinguished different problem types. Increased duration in these two phases is also associated with making errors. Looking at learning, 2 features distinguished the problems on which participants came to understand a new problem type. First, the duration of late phases of the problem solution increased. Second, there was increased activation in the rostrolateral prefrontal cortex (RLPFC) and angular gyrus (AG), regions associated with metacognition. We think this indicates the importance of reflection to successful learning.

SHORT BIO
John Anderson received his B.A. from the University of British Columbia in 1968 and his Ph.D. from Stanford University 1972. He has been at Carnegie Mellon University since 1978 where he is the Richard King Mellon Professor of Psychology and Computer Science. He has been served as president of the Cognitive Science Society, and has been elected to the American Academy of Arts and Sciences, the National Academy of Sciences, and the American Philosophical Society. He is the current editor of Psychological Review. He has received numerous scientific awards including the American Psychological Association’s Distinguished Scientific Career Award, the David E. Rumelhart Prize for Contributions to the Formal Analysis of Human Cognition, the inaugural Dr A.H. Heineken Prize for Cognitive Science, and the Benjamin Franklin Medal in Computer and Cognitive Science. He is known for developing ACT-R, which is the most widely used cognitive architecture in cognitive science. Anderson was also an early leader in research on intelligent tutoring systems. Computer systems based on his cognitive tutors teach currently mathematics to over 500,000 children in American schools. He has published a number of books including Human Associative Memory (1973 with Gordon Bower), Language, Memory, and Thought (1976), The Architecture of Cognition (1983), The Adaptive Character of Thought (1990), Rules of the Mind (1993), and The Atomic Components of Thought (1998 with Christian Lebiere), and How Can the Human Mind Occur in the Physical Universe? (2007). His current research interest is focused on combining cognitive modeling and brain imaging to understand the processes of mathematical learning.


Valerie J. Shute
Professor of Ed Psych and Learning Systems
Florida State University

TITLE
Stealth Assessment in Games: Why, What, and How

ABSTRACT
You can discover more about a person in an hour of play than in a year of conversation (Plato). For the past 6-7 years, I have been examining ways to leverage good video games to assess and support important student competencies, especially those that are not optimally measured by traditional assessment formats. The term "stealth assessment" refers to the process of embedding assessments deeply and invisibly into the gaming environment. Though this approach produces ample real-time data on a player's interactions within the game environment and preserves player engagement, a primary challenge for using stealth assessment in games is taking this stream of data and making valid inferences about players' competencies that can be examined at various points in time (to see growth), and also at various grain sizes (for diagnostic purposes). In this talk, I will present recent work related to creating and embedding three stealth assessments--for creativity, conscientiousness, and qualitative physics understanding--into Newton's Playground, a game we developed that emphasizes non-linear gameplay and puzzle-solving in a 2D physics simulation environment. I will begin by framing the topic in terms of why this type of research is sorely needed in education, then generally describe the stealth assessment approach, and finally provide some concrete examples of how to do it and how well it works regarding validity issues, learning, and enjoyment from a recent research study. 


SHORT BIO
Valerie Shute is the Mack & Effie Campbell Tyner Endowed Professor in Education in the Department of Educational Psychology and Learning Systems at Florida State University. Before coming to FSU in 2007, she was a principal research scientist at Educational Testing Service where she was involved with basic and applied research projects related to assessment, cognitive diagnosis, and learning from advanced instructional systems. Her general research interests hover around the design, development, and evaluation of advanced systems to support learning--particularly related to 21st century competencies. An example of current research involves using immersive games with stealth assessment to support learning—of cognitive and non-cognitive knowledge, skills, and dispositions. Her research has resulted in numerous grants, journal articles, chapters in edited books, a patent, and several recent books such as Innovative assessment for the 21st century: Supporting educational needs (Shute & Becker, 2010) and Measuring and supporting learning in games: Stealth assessment (Shute & Ventura, 2013).


Ryan Shaun Joazeiro de Baker
Teachers College Columbia University

TITLE
EDM in a Complex and Changing World

ABSTRACT
We've started to answer the questions of what we can model through EDM, and we're getting better and better at modeling each year. We publish papers that present solid numbers under reasonably stringent cross-validation, and we find that our models don't just agree with training labels, but can predict future performance and engagement as well. We're making progress as a field in figuring out how to use these models to drive and support intervention, although there's a whole lot more to learn. But when and where can we trust our models? One of the greatest powers of EDM models is that we can use them outside the contexts in which they were originally developed, but how can we trust that we're doing so wisely and safely? Theory from machine learning and statistics can be used to study generalizability, and we know empirically that models developed with explicit attention to generalizability and construct validity are more likely to generalize and to be valid. But our conceptions and characterizations of population and context remain insufficient to fully answer the question of whether a model will be valid where will apply it. What's worse, the world is constantly changing; the model that works today may not work tomorrow, if the context changes in important ways, and we don't know yet which changes matter. In this talk, I will illustrate these issues by discussing our work to develop models that generalize across urban, rural, and suburban settings in the United States, and to study model generalizability internationally. I will discuss work from other groups that starts to think more carefully about characterizing context and population in a concrete and precise fashion; where this work is successful, and where it remains incomplete. By considering these issues more thoroughly, we can become increasingly confident in the applicability, validity, and usefulness of our models for broad and general use, a necessity for using EDM in a complex and changing world.

SHORT BIO
Ryan Shaun Joazeiro de Baker is the Julius and Rosa Sachs Distinguished Lecturer at Teachers College, Columbia University. He earned his Ph.D. in Human-Computer Interaction from Carnegie Mellon University. Baker was previously Assistant Professor of Psychology and the Learning Sciences at Worcester Polytechnic Institute, and he served as the first Technical Director of the Pittsburgh Science of Learning Center DataShop, the largest public repository for data on the interaction between learners and educational software. He is currently serving as the founding President of the International Educational Data Mining Society, and as Associate Editor of the Journal of Educational Data Mining. His research combines educational data mining and quantitative field observation methods in order to better understand how students respond to educational software, and how these responses impact their learning. He studies these issues within intelligent tutors, simulations, multi-user virtual environments, and educational games.