This book contains a wealth of ideas that are useful for teachers, scholars, and students...It is very informative, thought-provoking, and highly recommended reading.
—Evaluation and Program Planning
American Journal of Evaluation, Book Review, June 2006
The illustration on the cover of Evaluating Social Programs and Problems: Visions for the New Millennium gives the reader a clear idea about the contents of the book. Perched on the top of the earth is a figure staring straight ahead through a telescope. The figure is looking forward, across miles and into the future.
I like to imagine the conversation between Donaldson and Scriven in the halls at Claremont Graduate University that gave seed to the project that culminated in this book. Perhaps they were talking about updating or developing new course offerings. Perhaps they were talking about their evaluation students and the types of work they would be doing following graduation. I’d bet a cup of coffee that their conversation went something like this: We know where we’ve been. We know where we are. But do we know where we’re going? What will be the face of evaluation in the new millennium? Let’s ask a few folks what they think.
And so they came up with the idea of a symposium to explore visions for evaluation in the new millennium. In the dead of winter (February 2001), more than 300 participants, intrigued by the notion of thinking futuristically, traveled to the Claremont Colleges in southern California to attend the Stauffer Symposium on Applied Psychology. This book is the product of that symposium.
Thinking about the future is a reasonable thing to do. Individuals do it, organizations do it, and so should disciplines. In his children’s book Oh, the Places You’ll Go, Dr. Seuss (1990) provided some sage advice about the process of growing up. Seuss wrote this book when he was in his mid-80s, old enough to know a thing or two. He was writing to the individual reader, but Donaldson and Scriven would, I believe, agree with me that Seuss’s perspectives apply to the field of evaluation as well. The book begins,
You have brains in your head.
You have feet in your shoes.
You can steer yourself
any direction you choose.
Evaluating Social Programs and Problems tackles the knotty question of the future of evaluation. Which direction(s) will the discipline have chosen? How will it be practiced? Will the discipline and its practices be integrative or fractured?
The book is organized into three main parts. Part 1 consists of Donaldson and Scriven’s introduction, titled “Diverse Visions for Evaluation in the New Millennium: Should We Integrate or Embrace Diversity?” The introduction foreshadows spicy debates that are presented in subsequent chapters. It provides titillating hints to the reader that vestiges of divisiveness left over from the infamous paradigm wars of the 1980s and 1990s linger still. The introduction made me want to read on.
Part 2, “Visions for Evaluating Social Programs and Problems,” consists of six chapters representing different perspectives on the future of evaluation. It provides for the reader a smorgasbord of evaluative approaches, in which eager supporters argue that their approaches should occupy key places in the future of evaluation. Here, readers learn about evaluation as a transdiscipline (Michael Scriven), results-oriented management (Joseph Wholey), empowerment evaluation (David Fetterman), fourth-generation evaluation (Yvonna Lincoln), inclusive evaluation (Donna Mertens), and theory-driven evaluation (Stewart Donaldson). Although there is some overlap in the characteristics of these approaches (e.g., several use participatory and community-oriented approaches to facilitate self-evaluation and to address power imbalances), the perspectives are mostly unique. As a practitioner, I appreciate writing that has clear application for the work that I do in the field. I picked up useful tidbits for my work throughout these chapters, although, as I have found in the past, I struggled with some of the ideas and vocabulary in Lincoln’s chapter.
Scriven elaborates on his views of evaluation as a “transdiscipline,” that is, a discipline that also provides tools for other disciplines. One of my favorite quotations in the book encapsulates this perspective: “The new millennium schools of social science will divide themselves into progressive, evaluation enriched schools, and into conservative, evaluation-impaired schools” (p. 6). I read this as a call to social science departments everywhere to start including a space within their ranks for someone with evaluation expertise. Definitely a good idea.
As noted by the editors with considerable disappointment, social experimentation is not discussed in the book. Tom Cook was one of the presenters and reactors at the symposium, but because of technical difficulties, his talk was not audio taped, and as a result, a written transcript of his comments could not be developed. According to the editors’ brief summary of Cook’s presentation, he expressed strong opinions about the visions presented by the other presenters and argued that social experimentation reigns as the premiere paradigm. Because I grew up (as an evaluator) as a Cook and Campbell devotee, the omission of a chapter devoted to this topic was a disappointment. I wish that the editors had been able to generate a chapter using an alternative method.
Part 3 is titled “Reactions and Alternative Visions.” This title didn’t quite fit for me, because several chapters in this part (i.e., those considered alternative visions) could just as easily have been in part 2. In part 3, we learn about theory-driven evaluation (William Crano), the need for diverse evaluators in diverse communities (Edith Thomas), and the challenge of culturally competent evaluation (Bianca Guzmán). Both Thomas and Guzmán write about the critical need for evaluators of color, as well as the need for strong competencies in serving racially, culturally, and linguistically diverse communities.
The book ends with a unifying perspective provided by Mel Mark. Mark embraces Scriven’s concept of evaluation as a transdiscipline. He also asserts that the task of evaluation in the new millennium will be the “ongoing work on establishing the types of evaluation approaches best suited to different conditions and audiences” (p. 184).
Mark notes, however, that during the conference, he caught strong echoes of the old, familiar paradigm wars and worries that in this new millennium, the resumption of that unproductive discord may be imminent. He senses an uneasy peace at the heart of our hard-earned cease-fire and points out the divisive tendency of some evaluators to overadvocate for one approach and denigrate others. Mark sees this as a problem and states, “What I fear is that some evaluators are increasingly inclined to disparage other approaches on moral grounds, ignoring the variety of niches where different kinds of evaluation may quite appropriately fit” (p. 190). Mark’s critiques are insightful, pithy, and well intentioned. For example,
Mark concludes his chapter, and the book, with a collection of hopeful recommendations for moving toward an integrated and integrative field of evaluation. He calls for increased efforts at community building within our discipline and urges greater respect for the diversity of evaluation approaches.
I liked this book a lot. It got me thinking about the future of our discipline and about our professional behavior toward our colleagues. One thing I appreciated was the book’s reasonable length (as well as its font size, which was quite comfortable for my midcentury eyes). The book’s most significant shortcoming, I believe, is its lack of balance. There were other perspectives, or visions, I would have liked to have seen discussed. But overall, the editors have assembled a valuable resource. Their intent was to get the reader thinking futuristically about evaluation, not to provide an in-depth analysis of where we are now. This is a book you can read during your commute to work on the train or in the La-Z-Boy on a Sunday afternoon. The writing is engaging and accessible. The ideas are provocative. You’ll be glad you read it.
—Jana Kay Slater, Center for Research on Adolescent Health and Development, Public Health Institute, 227 Luz Place, Davis, CA 95616; e-mail: firstname.lastname@example.org.
Dr. Seuss. (1990). Oh, the places you’ll go. New York: Random House.