Looking Collaboratively at Student Work: An Essential Toolkit Published: 1996 by Kathleen Cushman

Looking closely together at student work can unveil a treasure trove of insights to guide school communities as they reflect on their purpose, assess their progress, and plan strategies for reaching all children better. It's scary work, though, and respectful protocols can help.

The New York Times Science pages recently told the story of the heart surgeons in Maine, New Hampshire, and Vermont-there are only 23 in all-who agreed in 1993 to observe each other regularly in the operating room and share their know-how, insights, and approaches.

In the two years after their nine-month-long project, the death rate among their patients fell by an astonishing 25 percent. Merely by emphasizing teamwork and communication instead of functioning like solitary craftsmen, the study showed, all the doctors brought about major changes in their individual and institutional practices.

For teachers who, like heart surgeons, have traditionally worked as isolated professionals, the experiment holds a powerful lesson. If their goal is to lower the "death rate" of young minds and see them thrive, many educators now emphatically believe, they can do it better together than by working alone.

Like doctors making hospital rounds, architects gathered for a charrette, or lawyers examining clues to build a case, teachers in Essential schools have begun purposefully probing the rich evidence that lies immediately at hand in every school, searching for what it can yield about how students best learn.

They bring to the table their students' writing, math problem-solving, science projects, artwork, and whatever other evidence they can gather-in written notes or audio or video form-of what kids are producing every day.

Instead of disappearing into the bookbag or the wastebasket, these artifacts become a valuable mirror of how the school's practice does or does not reflect its intentions. Unlike a standardized test, their evidence speaks directly and revealingly of what teachers and students actually do and learn. Like a compass reading, it can then translate into informed action: changed perceptions of students; revised curricula and teaching strategies; new goals and a sense of direction for a faculty.

Rather than first focusing on the work's quality, these processes often ask teachers to suspend judgment and describe its qualities-bringing multiple perspectives to bear on what makes students tick and how a school can better reach them.

At first that multiplicity may complicate rather than simplify things, observes Joe McDonald, who has worked with the Coalition of Essential Schools on developing new methods to look at student work. But such discussion does not require consensus. Indeed, when people who come at school change with very different beliefs and assumptions meet to look at student work, their mutual understanding often deepens. Using diplomatic protocols that make communication feel "safe," they often find common ground and can move more surely toward creating the conditions in which teachers and students might do better throughout the system.

Across the country and abroad, school reformers have recognized the pressing need to place actual student work formally and respectfully at the center of both public and private conversations about school. From California to Vermont, from the Coalition of Essential Schools to its many working partners and friends in schools, teaching centers, and universities, people are trying out new tools for making change through that most radical of activities: unarmed discussion.

Though these tools differ, all share a focus on bringing together people across the school community -teachers, parents, students, and outside visitors-to look at work. All aim to learn something that will then affect future teaching and learning, not just the individual student whose work they examine. And all provide a formal structure, or "protocol," that, while often uncomfortable at first, surfaces and values different points of view.

In his work with the Coalition, David Allen has compared such protocols to putting on a play, "though the dialogue," he notes, "is mainly improvisational." Yet just as theatrical styles that range from classical to "method" can all work magic on the mind and soul, effective protocols have their styles and purposes too.

Some fall on the more evaluative end of the spectrum, aiming to analyze and thus improve teaching strategies and curriculum. Others rely more on close description to heighten teachers' understanding of individual children and hence affect their practice. Some look at a moment in time and extend its meaning outward; others take an accumulated body of evidence and draw new meanings from its larger picture.

"All these years we have been looking at students' work in order to see whether they have done what we told them to do," says Maine educator Marylyn Wentworth drily. Now a new set of purposes suggests itself, larger than what a red pencil can accomplish. And which process one chooses from the interesting array that has sprung up must grow from how well it suits one's purpose.

"Tuning" the Work Upward

In California, for instance, two groups of students from different houses in the same school used the CES-born "tuning protocol" (see sidebar, page 4) to present project work to each other, critiquing the work in front of an audience of school administrators. In this structured, facilitated discussion model, participants give both "warm," supportive feedback and "cool," more critical feedback to the presenters, who then reflect on it together without interruption, "tuning" their craft much as a musician might tune an instrument to its peak effectiveness.

"The students asked each other hard questions about grading, about standards, about the objectives of the projects and the coaching students received," says Joel Kammer, a school coach who teaches at Piner High School in Santa Rosa and who has written about California's use of the protocol in David Allen's forthcoming book for Teachers College Press. "It helped build community and a sense of common purpose and shared responsibility. And it produced substantial and useful information about student work and the possibilities for improvement."

The tuning protocol is widely used within and between Essential schools-as a means of developing more effective exhibitions and assessments, as a way of developing common standards, and as a means to gather and reflect on ideas for revising classroom practice. Kammer describes, for example, how the staff of a restructuring school uses the same steps to get feedback from a panel of teachers from another school. And he tells how a "critical friends group" of faculty members meets regularly in Piner's library to scrutinize student and teacher work, ask questions, and suggest improvements.

In most instances, the protocol's ritual of presentation and response works to mitigate the defensiveness people typically feel when they present work for public critique. Participants take turns in timed segments and eschew direct response. Even the placement of chairs contributes to the purpose of this technique: neither to argue nor to reach agreement, but to gain the benefits of each other's diverse perspectives.

Nonetheless, Joe McDonald observes, bringing private work into the public eye constitutes a "culturally wrenching act" in most schools.

"Some people would never do this unless there was a clear structure to protect them," says Bill Munro-Leighton, who teaches at Brown School in Louisville. And Ceronne Berkeley of Boston's Center for Collaborative Education recalls that at first she reacted to the protocol with "very real fear that I would be criticized as a teacher."

The tuning protocol can prove especially useful in loaded situations where poor communication is a problem, school people say. Typically, its users pose an important question they hope to answer by a close look at actual work-whether the school needs a new policy on spelling, for example, or how to incorporate writing across the curriculum.

In California, the state Restructuring Initiative has taken the ritual one step further, using it to reflect on change at the system level. At an annual symposium, analysis teams from restructuring schools now go through its "California protocol" to discuss before a reflective audience the "critical questions" they have identified as a result of earlier sessions examining student work.

That state's wide-scale use of both the tuning protocol and its own "meta-protocol" reflects a decision to ground systemic decisions firmly in student work, says Juli Quinn, who works with many Essential schools in Los Angeles. "Otherwise the kinds of things the system does will not be related to what students and teachers need," she observes.

The tuning protocol has proved equally useful in giving shape to conversations among parents and community members about the content and quality of student work. In New York City, University Heights High School asks parents to use it at students' "roundtable exhibitions." And both University Heights and Central Park East Secondary School use it to obtain feedback from outside visitors on their graduation requirements and academic programs.

As these examples show, people tend to use the tuning protocol and its relatives as a concrete way to hold up their practice against some standard, or even to work out what their own standards look like. In that process, however, unexpected new meanings often arise.

"When we look as individual teachers at student work we often see it through the narrow lens of the assignment," says Daniel Baron of the Harmony School and Education Center in Bloomington, Indiana. "But a group looking at it together makes a new meaning, focused not on evaluation but on the much bigger question of what we can learn. Doing this builds a profound sense of community."

That sense of shared interest in the big picture of student learning also shows up in another method of looking at student work: the "Language Record" or "Learning Record" first developed for British primary schools and now used in California, New York, and elsewhere at many levels. This far-reaching description of a child's growing language skills across the curriculum draws the family, the student, and all the student's teachers into observing and discussing his or her learning over time.

Assessing the System

The Learning Record stands out among portfolio assessment techniques because it regards the student's entire life experience as relevant-honoring, for instance, a bilingual child's fluency in another language as a demonstration of literacy and communicativeness. But it holds promise at the systemic level, too, as a reliable and valid portfolio-based picture of a program's effectiveness. Teachers across a system use the same scale to rate students' growing fluency over time, resolving variations among their scoring through a sampling and "moderation" process.

Reflecting on the "authenticity" of a student's learning tasks is still another way to frame a collaborative look at student work in a way that has usefulness both to the teacher and to the system. Fred Newmann at the University of Wisconsin has devised a set of criteria that prompts teachers to think through their work with that quality in mind. His standards emphasize not only higher order thinking skills and application to real life but also the central content and processes of the academic disciplines. Though they can be used to "score" classroom instruction, assessment tasks, and student performances, they make an even more useful filter as schools use student work to prompt long-range plans for raising the level of teaching and learning.

At Harvard University, Dennie Palmer Wolf leads the Performance Assessment Collaboratives in Education (PACE) project in another effort to mine portfolio assessment for what it can reveal about the bigger picture of teaching and learning. Funded by the Rockefeller Foundation and the Annie E. Casey Foundation, teachers from urban middle schools in Fort Worth, Pittsburgh, Rochester, San Diego, and San Francisco have gathered for close looks at portfolios-not only to chart a picture of students' growth over time, but also to raise their schools. consciousness about what opportunities students get to continuously use their minds in more demanding ways.

This kind of protocol can also yield important information, Wolf observes, on how well curriculum connects across the grades. Even in more privileged school communities, she notes, and especially in middle schools, "we often could not find in portfolios aprogression of opportunities from simple to demanding between the sixth and the eighth grade. And within any one year kids are typically writing the same thing over and over again in the same context. The topic may change, but not the demand level."

Public discussions about opportunity to learn, Wolf says, typically rely on "easy countables" such as property-tax dollars, numbers of books in the library, or staff education levels. "We need to think more about the conditions-the real quality-that make learning possible," she urges. "If schools want to become accountable, not just "be held. accountable, we must make that visible and discussable."

Seeing the Student Anew

The protocols described thus far fall on the more evaluative end of the spectrum of ways to look at student work. Toward the other end of the continuum lie methods that deliberately steer away from evaluation of student work and toward close description instead.

When a group agrees to withhold judgment and simply describe what each member sees in a piece of student work-absent its context or any other introductory information- a new sense of wonder can emerge about the way the particular student engages with learning, writes Harvard University's Steve Seidel. The "collaborative assessment conference" he developed with colleagues at Project Zero sets the stage for teachers to open themselves to the interests, passions, and direction that reveal themselves subtly in their students. work, and to unearth new means of reaching them.

Describing work closely without leaping to judgment proves extremely difficult in a culture used to a "thumbs up, thumbs down" critical style, Seidel observes. Yet as each participant offers what she notices to the group, the student's work yields new insights-not only about its own complexity, but also about the subjectivity of teaching in general. Because this structure demands questions more than seeking answers, Seidel says, teachers find it both frustrating and exciting.

The Descriptive Review of a Child, developed by Patricia Carini at the Prospect Center for Education and Research in Bennington, Vermont, also emphasizes discovering the whole child and also frames that inquiry around a question the teacher brings to a group of peers (sometimes including the child's parents). But for its "text" it takes a wide range of observed characteristics: physical presence, relationships, disposition, and interests as well as formal learning behaviors. As one by one participants amplify the teacher's description, and as the group then questions and comments on the observations, the child becomes increasingly "visible," writes Rhoda Kanevsky, a teacher in Philadelphia who has used the process for many years. "The child emerges as a unique person who is trying to make sense of the world," she says; and all participants gain new insights into the complex business of teaching and learning.

At New York's Central Park East Elementary School I, teachers gather regularly for such Descriptive Reviews of children, says principal Jane Andrias. But they also use that process for reflecting on curriculum and practice. Last year, for instance, the staff began an inquiry into the school's homework practices, using the descriptive protocol to surface harmonies and discrepancies among different teachers. practices and perceptions. "We began to see change right away," says Andrias. "Now we'll continue to meet this way and reflect on where we're going."

At Pasadena High School, Christelle Estrada says, her "critical friends group" found the purely observational approach to looking at student work so powerful that it has also begun using a similar protocol for peer review among teachers. And at Harmony School, Daniel Baron has devised a "constructivist" descriptive protocol in which students describe the qualities in their own "best work" from any time or place, then look for those same qualities in work they do for assignments.

Perhaps the most sweeping of all descriptive review protocols made its stage debut last January, when one full day's ordinary work collected from a sample of students in a small Minnesota school district came under the lens for collective discussion.

Such a "vertical slice" affords a unique cross-sectional look at the evidence, says Joe McDonald, and can yield powerfully authentic answers for schools struggling to look honestly at the need for change. Already several groups have chewed on the Minnesota slice, and several Coalition member schools are taking a comparable approach to exploring the issues they face.

The Minnesota slice came to the table in heavy brown cardboard boxes, but it might as easily have appeared on a small computer disk, David Niguidula points out. "The conversation grows more comprehensive and rich," he notes, "when you have access to different media-say, a video clip of a student doing peer tutoring-and when you can include people even if they're not all in the same room at once."

The Croton-Harmon, New York school district is one of six pilot sites where "digital portfolios" have provided a way to look at student work over time-work organized on disk according to the school's own goals. Indeed, notes Croton's former superintendent, Sherry King, coming together to design and discuss the digital portfolios brought the entire district together in a coherent way around student work, and helped articulate a common vision to link teaching and learning at every level.

How to Do It, and Why

It's not easy, of course, for a group to suspend judgment as it regards student work together; and a recent Atlas Communities paper (complete with its own protocol) lays out some guidelines for those who try it, no matter which method they select. Not unlike the norms for a good text-based Socratic seminar, these suggestions focus on sticking to the evidence, on understanding where other perspectives arise from, and on identifying patterns that emerge as discussion continues.

In another forthcoming guide from the Atlas Seminar, David Allen, Tina Blythe, and Barbara Powell have useful advice for those who would gather around student work; and they describe ways that several Atlas school communities have made up their own collaborative protocols to suit specific purposes. Indeed, logistical questions-who should participate, when and where to gather, who will facilitate the discussion and how, and which format fits the situation best-loom large in a context already charged with anxiety about exposing one's professional work.

Despite the benefits of hearing from multiple perspectives, for example, many who have used these protocols caution teachers to practice them in a safe environment before trying out broader forums. Joel Kammer writes of witnessing how a politically motivated audience tainted the atmosphere of trust when teachers presented work before an adversarial school board.

Yet some of the most useful feedback in protocols, teachers say, comes when they involve students and parents. "Our conversations took a huge leap forward when students joined the adult audiences at our senior exhibitions," says Allison Rowe at New Hampshire's Souhegan High School. "The protocols taught us to conduct that discourse without saying hurtful things."

No matter how powerful the experience of looking together at work, warns Nancy Mohr, who used several of these protocols for years as principal at University Heights, the challenge of bringing the insights it yields back into daily practice may prove daunting. "It is essential," she says, "to establish a process for taking what we learn from the examination of student work and using it in classrooms." That may involve teaming this work with other forms of professional development that are imbedded in practice, like peer coaching or critical friends groups.

And although it takes time and effort, few who have tried it would give up this simple and powerful practice. "The more I looked, the more I saw," observed Brad Stam of San Francisco's James Lick Middle School teacher after his first collaborative assessment conference.

"It affects everything you do," another teacher observed. "Once you routinely look at the work, you can begin moving from just ideas into your daily practice and planning."

Once begun, that cycle-reflecting together on direct evidence, drawing out its meaning, then folding what we learn back into the daily work-may prove the very engine of school change in the critical years ahead. "I used to think student work was between student and teacher," says Jon Appleby, who teaches at Maine's Noble High School. "Now I think all work should be as public, and as shared, as possible." When a teacher can say that, things have begun to move.

.

The Collaborative Assessment Conference

Developed in 1988 by Steve Seidel and his colleagues at Harvard University's Project Zero, the Collaborative Assessment Conference asks teachers to look together at pieces of student work and discuss, quite literally, what they see in the work. Through observing and describing the work, participants practice "looking more and seeing more" of what is in the work. The protocol is based on the notion that students are often working on problems or exploring interests that extend beyond the parameters of the assignment. To see the student's work fully, a teacher may have to look beyond those parameters and, with the help of colleagues less familiar with the child and the assignment, mine the piece for new insights.

Initially intended for use in middle and high schools, the Collaborative Assessment Conference is often used in elementary schools as well. Conferences can focus on all types of student work, though they tend to work best with open-ended assignments (as opposed to worksheets). The process takes anywhere from 45 minutes to an hour and a quarter, using these steps:

1. Getting started. The group chooses a facilitator to keep it focused. Then the presenting teacher gives out copies of the selected work or displays it so all can see it. At this point she says nothing about the work, its context, or the student. The participants read or observe the work in silence, making notes if they like.
2. Describing the work (5-10 minutes). The facilitator asks, "What do you see?" Participants respond without making judgments about the quality of the work or their personal preferences. If judgments emerge, the facilitator asks the speaker to describe the evidence on which the judgment is based.
3. Raising questions (5-10 minutes). The facilitator asks, "What questions does this work raise for you?" Group members ask any questions about the work, the child, the assignment, the circumstances of the work, and so forth that have come up for them during the previous steps of the conference. The presenting teacher makes notes, but does not yet respond.
4. Speculating about what the student is working on (5-10 minutes). The facilitator asks, "What do you think the child is working on?" Based on their reading or observation of the work, participants offer their ideas.
5. Hearing from the presenting teacher (10 minutes). At the facilitator's invitation, the presenting teacher provides her perspective on the work and what she sees in it, responding to the questions raised and adding any other relevant information. She also comments on any unexpected things that she heard in the group's responses and questions.
6. Discussing implications for teaching and learning (10 minutes). The group and the presenting teacher together discuss their thoughts about their own teaching, children's learning, or ways to support this student.
7. Reflecting on the conference (5-10 minutes). Putting the student work aside, the group reflects together on how they experienced the conference itself.