What is the motivation for the Inspecting Foundations Project?

Math education researchers frequently make recommendations to teachers, schools, and policymakers, particularly about instructional strategies and approaches to mathematics teaching. These recommendations are often grounded in research, but how strong is the underlying evidence for links to student outcomes? Inspecting Foundations is motivated by a desire to better align our confidence in claims with the evidence that supports them. Previous experiences examining influential citations in mathematics education (Otten et al., 2017), along with conversations about the questionable quality of evidence behind the "Science of Math" (Otten et al., 2025), have reinforced the importance of critically examining our own field's research. We believe math education scholars should scrutinize the evidence supporting our own recommendations just as carefully as we examine evidence from other disciplines.

Another motivation is the distinction between Potential Impact and Expected Impact (Otten, under review, and see figure below). An instructional strategy or educational reform might have considerable potential to improve student learning under favorable conditions, but its expected impact also depends on whether it can be adopted, implemented, and sustained in ordinary educational settings. Inspecting Foundations seeks to examine not only evidence that particular approaches can be effective, but also evidence about the likelihood that they will reach classrooms, influence students' experiences, and produce meaningful outcomes at a broader scale. This broader perspective can help us develop more realistic expectations about educational improvement.

What will be the outcomes of the Inspecting Foundations project?

Broadly, the intended outcome is clarity of claims and evidence within the math education scholarly community and also increased levels of trust from the practitioner community that seeks expert guidance on teaching and learning. More tangibly, outcomes will be an inspection report for each claim that includes...

1. Background and initial perspectives: An explanation of the claim, how it appears in mathematics education, what our initial survey reveals about educators' perspectives, and the inspection team's initial assessment of the claim.

2. Citation inspection: An examination of the sources commonly cited to support the claim, including whether those sources actually provide the support attributed to them and how closely their findings align with the claim.

3. Evidence inspection: An examination of the broader research evidence, including relevant studies that may not be commonly cited, with attention to the strength, consistency, and limitations of the available evidence.

4. Conclusions and confidence assessment: A summary of how confident we believe educators should be in the claim, including how our assessment changed during the inspection, whether the claim should be revised or qualified, and what future research would be valuable.

Each report will remain open to revision as new evidence emerges or previously overlooked research is brought to our attention. Our goal is not to deliver final verdicts, but to model how confidence in educational claims can—and should—change in response to evidence.

Is this a meta-analysis focused on effect sizes?

No, the inspection reports are not the same as a meta-analysis. This project is focused on a reasonable attempt at categorically assessing the evidence for a given claim (strong, moderate, weak, indeterminate) based on citation checking and a targeted search for relevant evidence. Studies with reported effect sizes can certainly be included in the cited studies or can come up during the evidence search, but we will not attempt to determine a "true" effect size. And many other forms of evidence will be considered, as well.

Instead of a meta-analysis, we will look at whatever evidence we can find and then attempt to calibrate our confidence in the core claim based on that evidence. Anything we share in the inspection report (e.g., a confidence rating in the link to student outcomes, or a revised claim based on what can be established empirically) will be tentative and always open to revision if more evidence is found, either studies we missed in our search or new studies that are published.

Is this the same as the What Works Clearinghouse efforts to evaluate the quality of evidence?

No. Although both WWC and the Inspecting Foundations project attend to evidence in math education, we take a substantially different approach.

The WWC (IES, 2010) uses pre-defined standards to identify the studies that provide "rigorous" evidence about the effectiveness of educational programs and practices. These standards emphasize particular research designs that can support conclusions about cause and effect but there are substantial concerns with this approach (Schoenfeld, 2006). Inspecting Foundations takes a more inclusive approach. We begin not with methodological thresholds but with the influential claims about mathematics teaching and learning, then dig down to examine the evidence underlying those claims. Rather than excluding studies based on their research design, we consider a wide range of methods and sources of evidence, asking how well each contributes to understanding the claim or calibrating our confidence in the claim.

For example, imagine that several small studies examine how mathematics teachers use a particular instructional strategy in their everyday classrooms. These studies might include classroom observations, teacher interviews, and samples of student work, but no experimental comparison groups. Such studies would generally not qualify for WWC effectiveness reviews, but they could provide meaningful evidence for Inspecting Foundations. They might reveal how teachers implement the strategy, how students respond, and under what circumstances it appears promising.  In fact, realistic implementations (with some sort of measure of student outcomes), though not "controlled," may shed particular light on the feasibility of the teaching strategy in ordinary circumstances rather than just evidence of the strategy's impact in more idealized conditions.

Our goal is to recognize what different kinds of evidence can—and cannot—tell us, relative to a given claim, rather than treating certain research designs as the only legitimate source of knowledge.

Is this the same as the Solid Findings work from Europe in the 2010s?

No, but it's similar in several ways. Inspecting Foundations shares important goals with the Solid Findings initiative of the European Mathematical Society (see Education Committee of the European Mathematics Society, 2011; Bosch et al., 2017). Both efforts seek to clarify what research in mathematics education tells us and how well the available evidence supports particular claims. However, Solid Findings focused specifically on identifying findings that were supported by especially strong research evidence. Inspecting Foundations begins with a wider range of influential claims, without assuming in advance how well supported they are. Our inspections may conclude that the evidence for a claim is strong, possibly affirming it as a "solid finding," but we may also find the evidence to be moderate, weak, or insufficient. We believe that identifying gaps and weaknesses in the evidence is just as valuable as identifying well-established findings.

Is this the same as the Basic Research discussion group?

Inspecting Foundations shares important goals with the ongoing Basic Research conversations organized by Dor Abrahamson, Irene Biza, Daniel Chazan, and colleagues. Their initiative encourages math education researchers to reflect on the fundamental knowledge our field has developed or seeks to develop, as well as the theories, methods, and research priorities that can help advance that knowledge. Inspecting Foundations contributes to this broader effort by examining what our field currently claims to know and how well those claims are supported by evidence. And both endeavors hinge on the definitions of certain constructs from math education literature.

But it is our understanding that the Basic Research conversations foreground constructs from basic research and is oriented toward mobilizing those constructs to shape mathematics curricula. Inspecting Foundations, on the other hand, foregrounds the practical realm. Even though we are inspecting research, we do so in order to calibrate our confidence and also clarify our recommendations so that we can be a more trusted voice for teachers and school leaders.

How does Inspecting Foundations relate to the "Science of Math"?

The "Science of Math" is an effort initiated by a group of scholars, primarily associated with special education and psychology disciplines, seeking to improve mathematics instruction by emphasizing scientific research. Their website identifies what they call "misconceptions" about math teaching and attempts to establish "truths" based on some citations from their disciplinary research literature. Inspecting Foundations shares their commitment to evidence-informed practice and their willingness to question common beliefs about teaching. The "Science of Math" initiative, however, is not representative of the entirety of scientific research on mathematics education, nor does its website constitute a comprehensive synthesis of that research. As discussed in a Math Ed Podcast episode, questions can be raised about whether particular citations adequately support the claims being made and how the supporting studies were selected. Inspecting Foundations aims to address these concerns by examining claims (within mathematics education literature but from sites like the Science of Math) through two complementary processes: checking the citations attached to a claim and also checking for evidence across mathematics education and related disciplines, without privileging a particular research tradition or instructional approach.

Another distinction concerns the endpoint of the process. Whereas the Science of Math website identifies several purported "misconceptions" and "truths" about mathematics teaching and learning, with an air of certainty, Inspecting Foundations instead treats the validity of claims as an open question, with the evidence subsequently shaping our degrees of confidence. The inspection reports may also identify gaps in the evidence and, in all cases, the confidence levels and conclusions will remain open to revision in light of new evidence.

References

Bosch, M., Dreyfus, T., Primi, C., & Shiel, G. (2017). Solid findings in mathematics education: What are they and what are they good for? CERME 10. 

Education Committee of the European Mathematics Society. (2011). "Solid Findings" in mathematics education. Newsletter of the European Mathematical Society, 81, 46–48. https://euromathsoc.org/committee-education/reports 

Institute of Education Sciences. (2010). What Works Clearinghouse. U.S. Department of Education. http://ies.ed.gov/ncee/wwc/ 

Otten, S., Powell, S., & Lambert, R. (2025, April 12). Discussing the Science of Math. Math Ed Podcast. https://www.podomatic.com/podcasts/mathed/episodes/2025-04-12T17_38_17-07_00

Otten, S., Webel, C., & de Araujo, Z. (2017). Inspecting the foundations of claims about cognitive demand and student learning: A citation analysis of Stein and Lane (1996). Journal of Mathematical Behavior, 45(1), 111–120. https://doi.org/10.1016/j.jmathb.2016.12.008

Schoenfeld, A. H. (2006). What doesn't work: The challenge and failure of the What Works Clearinghouse to conduct meaningful reviews of studies of mathematics curricula. Educational Researcher, 35(2). https://doi.org/10.3102/0013189X035002013