John R. Anderson
Carnegie Mellon University
Discovering the Structure of Mathematical Problem Solving
ABSTRACTIt is possible to combine multivariate pattern analysis (MVPA) and hidden Markov models (HMM) to discover the major phases that students go through in solving complex problems. I will illustrate this methodology by applying it to the learning of graphical isomorphs of algebra problems. We discovered a sequence of 5 major phases that students went through: An Orient Phase where they identified the problem to be solved, an Encode Phase where they encoded the needed information, a Compute Phase where they performed the necessary arithmetic calculations, a Transform Phase where they performed any mathematical transformations, and a Respond Phase where they generated the answer. The duration of the Compute and Transform Phases were they only ones that distinguished different problem types. Increased duration in these two phases is also associated with making errors. Looking at learning, 2 features distinguished the problems on which participants came to understand a new problem type. First, the duration of late phases of the problem solution increased. Second, there was increased activation in the rostrolateral prefrontal cortex (RLPFC) and angular gyrus (AG), regions associated with metacognition. We think this indicates the importance of reflection to successful learning.
John Anderson received his B.A. from the University of British Columbia in 1968 and his Ph.D. from Stanford University 1972. He has been at Carnegie Mellon University since 1978 where he is the Richard King Mellon Professor of Psychology and Computer Science. He has been served as president of the Cognitive Science Society, and has been elected to the American Academy of Arts and Sciences, the National Academy of Sciences, and the American Philosophical Society. He is the current editor of Psychological Review. He has received numerous scientific awards including the American Psychological Association’s Distinguished Scientific Career Award, the David E. Rumelhart Prize for Contributions to the Formal Analysis of Human Cognition, the inaugural Dr A.H. Heineken Prize for Cognitive Science, and the Benjamin Franklin Medal in Computer and Cognitive Science. He is known for developing ACT-R, which is the most widely used cognitive architecture in cognitive science. Anderson was also an early leader in research on intelligent tutoring systems. Computer systems based on his cognitive tutors teach currently mathematics to over 500,000 children in American schools. He has published a number of books including Human Associative Memory (1973 with Gordon Bower), Language, Memory, and Thought (1976), The Architecture of Cognition (1983), The Adaptive Character of Thought (1990), Rules of the Mind (1993), and The Atomic Components of Thought (1998 with Christian Lebiere), and How Can the Human Mind Occur in the Physical Universe? (2007). His current research interest is focused on combining cognitive modeling and brain imaging to understand the processes of mathematical learning.
Professor of Ed Psych and Learning Systems
Florida State University
Valerie Shute is the Mack & Effie Campbell Tyner Endowed Professor in Education in the Department of Educational Psychology and Learning Systems at Florida State University. Before coming to FSU in 2007, she was a principal research scientist at Educational Testing Service where she was involved with basic and applied research projects related to assessment, cognitive diagnosis, and learning from advanced instructional systems. Her general research interests hover around the design, development, and evaluation of advanced systems to support learning--particularly related to 21st century competencies. An example of current research involves using immersive games with stealth assessment to support learning—of cognitive and non-cognitive knowledge, skills, and dispositions. Her research has resulted in numerous grants, journal articles, chapters in edited books, a patent, and several recent books such as Innovative assessment for the 21st century: Supporting educational needs (Shute & Becker, 2010) and Measuring and supporting learning in games: Stealth assessment (Shute & Ventura, 2013).
Ryan Shaun Joazeiro de BakerTeachers College Columbia University
EDM in a Complex and Changing World
We've started to answer the questions of what we can model through EDM, and we're getting better and better at modeling each year. We publish papers that present solid numbers under reasonably stringent cross-validation, and we find that our models don't just agree with training labels, but can predict future performance and engagement as well. We're making progress as a field in figuring out how to use these models to drive and support intervention, although there's a whole lot more to learn. But when and where can we trust our models? One of the greatest powers of EDM models is that we can use them outside the contexts in which they were originally developed, but how can we trust that we're doing so wisely and safely? Theory from machine learning and statistics can be used to study generalizability, and we know empirically that models developed with explicit attention to generalizability and construct validity are more likely to generalize and to be valid. But our conceptions and characterizations of population and context remain insufficient to fully answer the question of whether a model will be valid where will apply it. What's worse, the world is constantly changing; the model that works today may not work tomorrow, if the context changes in important ways, and we don't know yet which changes matter. In this talk, I will illustrate these issues by discussing our work to develop models that generalize across urban, rural, and suburban settings in the United States, and to study model generalizability internationally. I will discuss work from other groups that starts to think more carefully about characterizing context and population in a concrete and precise fashion; where this work is successful, and where it remains incomplete. By considering these issues more thoroughly, we can become increasingly confident in the applicability, validity, and usefulness of our models for broad and general use, a necessity for using EDM in a complex and changing world.
Ryan Shaun Joazeiro de Baker is the Julius and Rosa Sachs Distinguished Lecturer at Teachers College, Columbia University. He earned his Ph.D. in Human-Computer Interaction from Carnegie Mellon University. Baker was previously Assistant Professor of Psychology and the Learning Sciences at Worcester Polytechnic Institute, and he served as the first Technical Director of the Pittsburgh Science of Learning Center DataShop, the largest public repository for data on the interaction between learners and educational software. He is currently serving as the founding President of the International Educational Data Mining Society, and as Associate Editor of the Journal of Educational Data Mining. His research combines educational data mining and quantitative field observation methods in order to better understand how students respond to educational software, and how these responses impact their learning. He studies these issues within intelligent tutors, simulations, multi-user virtual environments, and educational games.