Donaldson is one of the leading figures in the area of theory-driven evaluation ...this book [is] an important book for anyone interested in theory-driven evaluation ...[it] would also seem to be a strong contender for courses on evaluation and health. -- Melvin M. Mark, Ph.D., Pennsylvania State University
Donaldson’s book makes an important contribution to the advancement of theory-driven evaluation, and toward improving the evaluation profession … It allows readers to not only understand how and when to use theory-driven evaluations, but also how to perform them. -- Huey T. Chen, Ph.D., The University of Alabama at Birmingham
[T]his book... makes a significant contribution to the field by taking the next step in clarifying how this evaluation approach works in practice. -- David Fetterman, Ph.D., School of Medicine and Education, Stanford University
Strengths are its writing style, level of detail, and its practical application to real life projects. -- Thomas Chapel, M.A., M.B.A. Center for Disease Control, Atlanta, GA
This book has multiple audiences and numerous potential uses. It can be used in graduate courses focused on student and faculty development and with cohorts and other groups where acclimation to the academy and relational learning are critical. ... The teachings of this book could serve faculty and administrators at all levels of experience. A final audience is mentoring scholars and researchers for whom this contemporary compendium of ideas and practices can further inform professions. -- Education Review
American Journal of Evaluation, Book Reviews, June 2008
With Program Theory-Driven Evaluation Science: Strategies and Applications, Stewart Donaldson has firmly established himself as one of the leading theory-driven evaluation scholars and practitioners. While emphasizing the theory, methodology, and practice of the program theory-driven evaluation approach, he decisively gives greater attention to the latter of these throughout the book. This is particularly noteworthy, given that much of the literature related to this and similar approaches is often at such a “stratospheric level of abstraction” (Weiss, 1997, p. 502, as cited in Donaldson, 2007, p. 8) as to be of little, or limited, value to students and practitioners. In response, Donaldson has endeavored to strike a much-needed balance between the theory, methodology, and practice of program theory-driven evaluation, with the express intent of making the approach accessible to those seeking the “know-how” to conduct such evaluations.
The book consists of 13 chapters that are organized into three parts. Part 1, “Foundations and Strategies for Program Theory-Driven Evaluation Science,” consists of three chapters in which Donaldson discusses the emergence of program theory-driven science, strategies for developing program impact theory, and tactics for formulating, prioritizing, and answering evaluation questions. Early on, Donaldson describes the main procedural elements involved in conducting program theory-driven evaluation: developing program impact theory, formulating and prioritizing evaluation questions, and answering evaluation questions.
Like several others, Donaldson advocates using two techniques when developing program impact theory. First, he suggests working with key stakeholders to develop a common, shared understanding of how a program is presumed, or intends, to prevent or ameliorate a particular problem or meet a particular need. Second, he stresses using social science theory and prior research (“if they exist,” p. 10) to assess the feasibility of proposed relationships between a program and its desired outcomes. This process is also intended to facilitate formulating and prioritizing the relevant evaluation questions as well as strategies for answering them. Combined, these three components serve as the foundational basis of the case studies presented in Part 2, “Applications of Program Theory-Driven Evaluation Science.” In this part, Donaldson offers six chapters that describe and deconstruct case studies of actual program theory-driven evaluations in practice, most related to work and health programming. These cases are all evaluations of statewide programs (some multisite) of varying size and complexity that were implemented in California. The final two chapters in this part of the book present the hypothetical case of the Bunche–Da Vinci Learning Partnership Academy (previously published in Alkin & Christie’s  Theorists’ Models in Action) and Donaldson’s proposal for evaluating the hypothetical program using the program theory-driven evaluation three-step approach.
In Part 3, “Lessons Learned and Future Directions,” Donaldson turns his attention to some of the pervasive issues that emerged from the case examples presented in prior chapters, as well as those facing evaluation theory and practice in general. Here, he focuses on matters such as what these lessons might suggest for improved practice, including the methodological and practical constraints (e.g., time, resource, and feasibility) often faced by evaluators when questions about “what constitutes credible evidence?” arise (and they are nearly always bound to occur) and the potential versatility of program theory driven evaluation. In fact, Donaldson suggests that, unlike some approaches, the program theory-driven evaluation approach can defensibly handle the task of investigating the mediating causal mechanisms through which programs operate as well as those that moderate these relationships and that this approach is not as method dependent (i.e., that it is method neutral) as many of the more technically demanding analytic techniques (e.g., HLM, LCM, and SEM). He concludes with his vision for future directions in program theory-driven evaluation and the evaluation discipline in general, which includes using evaluation theory in practice and developing an evidence-based evaluation practice, among many other things.
The strengths of this book are many, and in my view, its strongest features include Donaldson’s clear, concise, and nontechnical articulation of the approach’s general principles and procedures; that he has aptly conveyed the approach’s relevance and application to real-world cases; and, perhaps most important, that he has succeeded in making the approach accessible to those seeking the know-how to conduct program theory-driven evaluations. Most striking, and a stance that I would like to see taken more frequently, is Donaldson’s forthrightness and integrity regarding such matters as when, why, and how breakdowns occur in the approach and the authentic challenges sometimes faced when conducting these types of evaluation (e.g., problems that often arise in highly politicized environments, seeking consensus where none exists). His discussion of these issues provides insights about how the program theory driven evaluation approach can be improved, or at the very least, how these problems can be potentially reduced, avoided, or in some cases eliminated entirely. Although much of what is presented in this book is an extension or modification of the work of evaluation luminaries such as Chen, Freeman, Lipsey, Mark, Pawson, Rogers, Rossi, Tilley, Weiss, and many others, it is also unique in that it provides in depth, real-world case examples that apply the program theory-driven evaluation principles and procedures, including the aforementioned breakdowns, challenges, and lessons learned. These case examples form the core of the book and are truly exceptional in that they allow the reader to follow the step-by-step process of Donaldson’s approach and the contingencies that inform his reasoning, logic, and decision making in the course of conducting these evaluations.
Despite the book’s noteworthy features, and these are numerous, I do continue to be baffled by Donaldson’s need to affix the term science to the approach described in the book. Should we then conclude that other approaches are “unscientific” and therefore inferior? Perhaps this decision was driven by the ongoing debates about evidence-based evaluation methodologies and efforts to deter the controversy surrounding the issue of “scientific” methodology in federally funded research and evaluation (Coryn, Hattie, Scriven, & Hartmann, 2007; Coryn & Scriven, 2008; Julnes & Rog, 2007). Arguments about terminology aside, I find again and again the same questions and concerns arising when considering the relative merits and value of applying theory-driven evaluation approaches in practice. First, it is not always clear whether the priority is evaluating the program theory or the program itself (Coryn, 2005). Second, why expend such great efforts to develop and verify a program theory if it is not used evaluatively? In many of the case studies presented in Part 2, the evaluation questions formulated were primarily descriptive (e.g., “Whom are the program sites serving?”) and in some instances unrelated to the program impact model entirely. Relatedly, program theory is sometimes only tangentially connected to the critical evaluation questions or what stakeholders really need to know (Davidson, 2007), and this point itself only serves to reinforce and confirm my concerns (e.g., alternative, “atheoretical” approaches can just as easily respond to the key issues, such as “Does the program work?” and “Are the results attributable to the program worth the costs it took to attain them?”). In all fairness, Donaldson does make it quite clear in Program Theory-Driven Evaluation Science that program impact models are not intended as (only) measurement or causal models (and that they should be parsimonious) but rather that they are reasonable approximations of how programs are intended to produce results and, ultimately, that these models can serve as useful tools or frameworks for contemplating how a particular evaluation in a particular context is best conceptualized and designed.
It is, or arguably should be, our obligation as scholars and practitioners not only to critique existing principles and procedures but also to improve upon them and develop new ones. Here, Donaldson has succeeded admirably. His insight, vast experience, forthrightness, integrity, and sheer determination and hard work, not to mention the clarity of writing, has resulted in significant payoffs in terms of giving the theory-driven approach greater transparency, accessibility, and intelligibility to a wide audience, including students and practitioners as well as funders and clients of evaluation. Moreover, and despite the relatively minor concerns just outlined, Donaldson has clarified and in some cases resolved many of the enduring challenges inherent in applying theory-driven evaluation that, in my humble opinion, has long plagued this approach to program evaluation. In addition, he has gone to great lengths to begin the process of moving evaluation away from its long-standing tradition of prescriptive theories of practice (i.e., the so-called atheoretical nature of evaluation practice) toward a more empirically driven, evidence-based form of evaluative inquiry derived from systematically generated knowledge. The bottom line is that the six case studies alone make this book a valuable, useful, and, I suspect, effective teaching tool and practitioner resource, and I strongly, and without hesitation, recommend this book. — Chris L. S. Coryn
Alkin, M. C., & Christie, C. A. (Eds.). (2005). Theorists’ models in action (New Directions for Evaluation, No. 106). San Francisco: Jossey-Bass.
Coryn, C. L. S. (2005). Book review: Practical program evaluation: Assessing and improving planning, implementation, and effectiveness. American Journal of Evaluation, 26(3), 405-407.
Coryn, C. L. S., Hattie, J. A., Scriven, M., & Hartmann, D. J. (2007). Models and mechanisms for evaluating government-funded research: An international comparison. American Journal of Evaluation, 28(4), 437-457.
Coryn, C. L. S., & Scriven, M. (Eds.). (2008). Reforming the evaluation of research (New Directions for Evaluation). San Francisco: Jossey-Bass.
Davidson, E. J. (2007). Unlearning some of our social scientist habits. Journal of MultiDisciplinary Evaluation, 4(8), iii-vi.
Julnes, G., & Rog, D. L. (Eds.). (2007). Informing federal policies on evaluation
methodology: Building the evidencebase for method choice in government sponsored evaluation (New Directions for Evaluation, No. 113). San