Home & Education
Meta-Learning
Our research, which spans early-education learning science through professional development, clearly show that the skills we traditionally value are not only doomed to be outdated by technological advances, but historically have played only a secondary role in individual success.
We know how kids develop strong perspective-taking, deep endogenous motivation, and a belief that their hard work will pay off. Instead of focusing education almost solely on traditional skills and knowledges, it must instead focus on meta-learning, the collection of attributes most strongly related to positive life outcomes: health, happiness, and meaning. We explore both the human and technological side of this transformation via research in the learning sciences, behavioral economics, and machine learning-based augmented intelligence.
Read more in the Socos White Paper: Aligning Learning with Life Outcomes through Naturalistic Assessment
Muse provides family-focused development of children’s life outcomes for long term community impact. We help parents support their child’s cognitive, social, and emotional growth by offering detailed, research-supported recommendations that foster each child’s unique long-term development. Our individually targeted, daily messages draw from decades of peer-reviewed research on life outcomes and child development alongside our own analysis of over 100 million working professionals. These predominately play-based activities are designed to improve life outcomes based on each child's specific meta-learning strengths and needs (e.g., GRIT, mindset, working memory, executive control, self-efficacy, and more). And the entire system can run via SMS or within an app.
Traditional academic measures are only weakly predictive, or not at all, when “deeper” qualities of a learner are taken into account. Those qualities, which we call meta-learning, should be the principal focus of formal and informal development, at home and in the school (and, we believe, even in the workplace). Kids absolutely need to learn specific academic skills, but the foundation is the meta-learner. In fact, when we orient our academic ambitions on long-term (life) outcomes, we see that transient skill learning doesn’t affect those outcomes. For example, Stanford economist Raj Chetty found the teachers with the biggest long-term impact on their students actual produced students that tended to slightly underperform on standardized assessment. Similarly, the role-modeling of parents, not their more directive behaviors, are substantially more predictive of long-term outcomes. We believe play and natural learning experiences promote meta-learning and should be to the core of both growth and assessment. AI — in this case augmented intelligence — can support parents and teachers in these unstructured learning environments.
The goal of Muse is not simply to improve the life outcomes of individual children, but to have a community-wide impact. At Socos, we combined the work of economists like James Heckmann and Raj Chetty with intervention studies into 35 different meta-learning constructs to model the hypothetical impact of meta-learning interventions had they started 25 years ago. We had to explicitly take into account factors such as socioeconomic demographics and regional differences to reflect the differential impact of interventions in the research literature. In the model, we assumed a fixed annual cost based on the under-12 population of the US. We found nearly immediate savings on the cost of education and healthcare, which continued to produce returns regardless of market fluctuation. Eventually the empirically reported productivity increases being to accelerate the returns. Overall, these changes would results in $1.3-1.8 trillion (+10%) added to the US economy each year. Even more startling, applying the same model to South Africa suggested annual economic gains of 60%, and 110% in India. Both the New America Foundation and McKinsey have also modeled the impact of improved educational equality, and their results, $1 trillion and $2-4 trillion, respectively, strongly support our more detailed analysis. If we built our education systems on these research-driven lessons, we’d change not only our children’s lives, but also the entire economy.
You can learn more about Muse here.
Past Projects
Kindersight: Assessing the Linguistic Environment of Kindergarteners
Interventions in early childhood are of great interest as they can have significant impacts down the road (Karoly, Kilburn, et al., 2006). However, interest in improving early childhood learning across school and home settings is colliding with movements to increase standardized testing at younger ages. While testing proponents are rightly concerned about measuring children’s learning, tests carry many problems. Tests are valid only for the population and purpose for which they were designed, eliminating cultural bias from tests is extremely difficult, and tests are often designed as sequestered experiences stripped from authentic contexts. Standardization narrows the range for what is considered acceptable progress regardless of developmental variation, and testing is intrusive, displacing instruction which might yield better learning.
Working with researchers at University of Texas at Austin, Socos is building an alternative method for assessing young children’s linguistic and metacognitive development in richer detail and more naturalistic contexts. The new system extends our existing assessment algorithm for adult learners to deliver rapid, actionable feedback for parents, teachers, and caregivers based on the broad range of learning experiences already taking place in the classroom and at home. In academic studies, Socos has successfully predicted final grades by analyzing unstructured student text in online discussion forums, which also yielded preliminary topic maps that could be used to trace different trajectories in student thinking.
Analysis of young children’s linguistic experiences from audio recordings have demonstrated the feasibility of automatically tracking word exposure and adult- child conversational turns. We are in the process of deploying similar technology throughout kindergarteners’ learning environment in conjunction with location data and analyze them with our continuous passive assessment algorithms. We are producing a map of the conceptual space of young learners with which we can explore the predictive value of the individual language experience, including self-generated as well as child-to-child and adult-to-child speech in each language. Combined with student artifacts and information about classroom activities, these data sources would serve as multiple inputs illuminating students’ knowledge and skill that can then be connected with externally validated assessment outcomes. By distinguishing between what students generate and what students experience, the predictive system further elucidates school and home interventions with the greatest potential to boost learning.
The implications of this work would be quite far-reaching: The ability to assess students’ knowledge from existing learning experiences removes the burden of testing and enables teachers to focus on instruction with the greatest learning impact. Evaluating the benefits of different experiences at school and at home would better inform teachers’ and caregivers’ choices of interventions, in addition to facilitating more productive collaboration between them. It could clarify the relative effectiveness of parental contributions in reinforcing school lessons (repeating the same language) or elaborating upon them (adding new language). It exemplifies a minimally-intrusive technological innovation that would advance what instructors know about their teaching, rather than immediately requiring drastic changes in practice.
Beyond simply increasing and enriching children’s vocabularies, supporting their linguistic development in real-world contexts directly advances their skill in using language to monitor and guide their own learning. Our system augments the critical support of caregivers through brief messages informing and guiding their interactions with the children. Our ultimate goal is not only to provide daily, personalized interventions to decrease the word gap, but to drive the development of broader lifelong meta-learning skills.
College Learners: Innovative Competency-Based Online College
This project continues to build on our interest in continuous passive assessment as an innovative method for real-time assessment of unstructured student work. We validate our assessment algorithms based on their ability to predict concrete outcomes such as persistence and progression, as well as mapping them to established program goals and competencies. Linking this competency map to employers’ needs can further help gauge progress in students’ career trajectories, via an adaptive guide for leveraging student strengths. By offering more timely and specific feedback based on automated analyses of student work and interactions, such a system can more effectively guide self-assessments, instructional coaching, and peer learning arrangements.
The optimal feedback is derived from the data of student-coach interactions (phone and email). By examining the data of student-student interaction, student-coach discussion, and student-reviewer feedback, Socos will be able to make predictions both about individual student’s future performance and provide constructive feedback to coaches and reviewers about how students may improve in school and life outcomes.
We will be using this data combined with comprehensive student profiles to evaluate their engagement with the school. Socos will be able to offer coaches warnings of disengagement and recommendations of known interventions.
Online Students: Online Student Discussion Predictive of Grades
Using only student discussion data from introductory courses in biology and economics, both probabilistic latent semantic analysis (pLSA) and hierarchical latent Dirichlet allocation (hLDA) produced significantly better than chance predictions which improved with additional data collected over the duration of the course. Results indicate that topic modeling of student-generated text may provide a useful source of formative assessment to support learning and instruction.
Our goals in this research were to identify facilitation strategies that encourage productive discussion, and to explore text mining techniques that can help discover meaningful patterns in the discussions more efficiently at scale. Based on a close reading of selected discussion threads from online undergraduate science classes, we observed a variety of facilitation strategies associated with discussion quality. These observations informed our selection of a larger dataset of discussion threads to analyze via text mining techniques. Using latent semantic analysis to produce topic models of the content of the discussions, we constructed visualizations of the topical and temporal development of those discussions among students and faculty.
These visualizations revealed patterns that appeared to correspond with specific facilitation styles and with the extent to which discussions remained focused on particular topics. From a case study focusing on six of these discussions, we documented distinct patterns in the types of facilitation strategies employed and the character of the discussions that followed. In our conclusion, we discuss potential applications of these analytical techniques for helping students, faculty, and faculty developers become more aware of their participation and influence in online discussions, thereby improving their value as a learning environment.