Introduction

As I write this, the sixth annual conference on Pattern Languages of Program Design (PLoP) is just around the corner at Allerton Park, Illinois. It is one of several PLoP conferences that brighten the calendar during the year: EuroPLoP at Kloster Irsee, Germany; ChiliPLoP in Arizona, with plans for a KoalaPLoP near Melbourne, Australia. One sees many of the same faces at these conferences as one sees at OOPSLA, TOOLS, or ECOOP. But the similarity ends there.

The mainstream and academic conferences are built on a legacy of novelty. Papers must extol a new idea. It is important that papers cite relevant prior art, and that they distinguish themselves from this prior art. In general, claims of better/cheaper/faster must either be formally proven or substantiated through reduction to practice. There is an anonymous peer review process, and there is usually a single review cycle.

PLoP conferences are different. Papers should describe how to use a long-standing, proven idea. It is still important to cite prior art to the degree it can help readers solve their problems. It is more important to cite where the technique was profitably used than to locate publications where related work has been published. And the peer review process is engaging, intense, and supportive.

Why the differences? The two communities have different value systems and goals. Conferences like ECOOP reward advances in the state of the art and provide a forum for sharing these advances with the community. The PLoP conferences reward people who take the time to document successful practices. The peer reviews in the PLoP community focus not only on the quality of the idea in a work, but pay close attention to the quality of presentation as well. To help encourage people to contribute to the growing body of pattern literature, the pattern community strives to protect authors' dignity and to provide a safe review environment.

The PLoP community peer review culture is therefore much different than that for traditional academic conferences. The details vary from conference to conference, but the overall process goes like this:

1. A call for papers is issued for a particular PLoP conference.

2. An author submits a work in response to the call for papers. A shepherd works with the author, offering feedback to improve the paper and improve the chances that the program committee will accept the paper.

3. The author and shepherd work on several iterations of the paper.

4. The paper goes before a program committee that decides to turn back the paper for further work or to accept it for presentation at the conference. What "presentation at the conference" means is that authors bring their works to a Writers' Workshop, where the author is given the benefit of additional peer feedback in a supportive, constructive setting [Coplien1997].

5. The author participates in a Writers' Workshop at the conference.

6. The author incorporates feedback from the workshop, perhaps in conjunction with a shepherd.

7. The author re-submits the work to the conference, and it is published in post-conference proceedings, usually as a technical report for a sponsoring university or corporation (for example, [EuroPLoP1998]).

8. The author may submit the work to be considered for publication in a collected volume. In the early days of the pattern community, the works of one or more PLoPs were aggregated into a post-conference proceedings of sorts (such as [PLoPD1], which collected works from the first PLoP). More recently, these books have been decoupled from the conference process (such as the forthcoming volume 4 of the PLoPD series, which comprises works from several recent PLoP and EuroPLoP conferences).

Writing Workshops

There was another step in the process at this year's EuroPLoP, building on a similar but much smaller effort last year. In addition to the standard Writers' Workshops where the author receives constructive feedback in a group setting, the conference featured writing workshops where authors could interact directly with shepherds to refine their works. (Yes, the terms are confusing, so pay close attention where they are used in this article. Maybe we'll fix the terminology problem some day; let me know if you have any suggestions.)

The writing workshop program this year was put together by Christa Schwanninger of Siemens in Munich. Before the conference, the program committee had allocated 6 papers (one of which had two authors) to the writing workshops. Christa first selected the shepherds for the activity and then asked shepherds to select papers of interest to them, one shepherd for each paper.

I was one of the shepherds for the writing workshop this year. My first meeting with my "sheep" was at the Kaufbeuren train station; by coincidence, we had both arrived at the same time and were waiting to hail a cab for Irsee. The opening dinner was that evening, and the conference formally kicked off the next morning. The opening ceremonies include a sample Writers' Workshop for the sake of those who had never participated in one before.

Later that day, all of the sheep and shepherds met together to make face-to-face acquaintance. We had planned to get together as a group for lunch, but the logistics made that difficult. At 5:00 that afternoon, the sheep and shepherds formally convened for their first session.

As we would do for the rest of the week, we met in the conference computer room where authors could edit their works-in-progress. Most sheep had arrived early to make sure they could load their papers onto the machines. The sheep and shepherds chatted briefly, and most sheep started making changes to their works right away. Some stayed after the meeting or returned after dinner to continue working.

I was skeptical whether the writing workshops would do the right thing, especially since the authors had already benefited from working with their shepherds (a different group of shepherds than those at the writing workshops) before the conference. But I found that the shepherding had been uneven. Some shepherds had deferred largely to the authors' inclinations, sending only a few lines of E-mail to the authors during the entire shepherding process, while others had provided more direction. It's possible that with stronger shepherding, some of the papers in the writing workshops would easily have been ready for the main conference program of Writers' Workshops.That opened my eyes a bit to the value of the activity. And maybe some things just work better face-to-face.

The authors--all of them--worked unbelievably hard on their papers. Would they have worked as hard had this forum not have been provided for them? It's hard to say, but I think the environment, the facilities, and the support available to them incented and inspired them to move forward.

Last, I watched the papers improve immensely. On one hand, this points to a partial failing in the earlier shepherding process. What we accomplished in the writing workshops in a few contact hours certainly could have been achieved over the weeks and weeks of shepherding time. On the other hand, for whatever reason, all of these authors had some great patterns in them--and the writing workshop seemed to be the tool that could bring those patterns to light.

The next step would be to assess the writing workshop papers for readiness for presentation in one of the mainstream Writers' Workshops. I think all of the shepherds got caught up in the enthusiasm of helping their sheep, and most were probably ready to recommend that their sheep's patterns go the next step. But they were also circumspect enough to know that they might be getting caught up in the moment, and most shepherds asked another shepherd, or another member of the pattern community, to have a look at their sheep's paper and offer a corroborating opinion. In the end, we sent two of the papers on for reviews in Writers' Workshops, and those works will be published in the proceedings. Two of the authors who were working on another paper announced that they had decided on their own to put more work into the paper before subjecting it to a workshop -- this in spite of the fact that at least some of the shepherds felt their paper was ready. One other author had to leave the conference early, so whether that paper could be admitted to a Writers' Workshop became a moot point.

We convened the writing workshop one last time in a nicely appointed conference room next to the computer room. We held a Writers's Workshop, outside the main conference track, for the remaining two papers. They were enjoyable workshops with a good balance of positive feedback and suggestions for improvement.

One might suppose that there are dangers associated with the stigma of having one's work relegated to a writing workshop. I asked some of the authors about this, but they had no such feelings: it had been a positive experience for them. They would be back next year, too. There was no stigma, but a wide-ranging aura of support. I talked to Norm Kerth about this afterwards; he hadn't been at the conference, but had heard various reports about the writing workshop from afar. "This goes to show that the community recognizes itself as one whose goal it is to help its members," he noted.

I ended being completely convinced of the value of this activity. Some of the papers that came out of the writing workshops were, in my opinion, better than the majority of the patterns that had been accepted for the mainstream Writers' Workshops. Resources permitting, maybe we should open up writing workshops for anyone who wants to register for them. I know I'd sign up for one if the opportunity presented itself!

After the experience had shattered my skepticism, I asked Christa what she had done to make it work. EuroPLoP had conducted writing workshops in 1998 as well, though it was a much smaller activity, and I think it was less formally integrated into the program. I asked Christa how she had planned this so it worked out so well: the schedule went smoothly, and by any measure, it was a success. "The only thing I planned," she noted, "was the selection of the shepherds: people who have experience, and who can work with and want to work with other people who don't have this experience." Neil Harrison, who participated in last year's writing workshops at EuroPLoP, also said, "Getting the right people is key." The rest took care of itself. It was a wonderfully emergent result; maybe it's a pattern.

Shepherding

The shepherding process, as outlined here, goes all the way back to the first PLoP conference in 1994. I remember it as being intense and important in the early days. My sense is that it's lost some of its lustre and visibility in more recent conferences. There is a more visible program committee process--something that was absent from the first two PLoPs--and I fear it has perhaps displaced the emphasis on shepherding.

Others share my concern, among them Neil Harrison, who has been working to capture the patterns of shepherding based on successful shepherding efforts over the years, with an eye to helping people avoid the pitfalls that result in less successful shepherding efforts.

"Shepherding is important for four reasons," says Neil. "First, it's a vehicle to convey expert knowledge about pattern writing from the experts to the novices. Second, any additional set of eyes is always beneficial when you're writing, and it can be both a comfort and motivation knowing that someone else will be looking over your shoulder. Of course, it improves the quality of the works: we've seen this again and again. Last, it strengthens the pattern community as a structure that builds relationships, encourages communication, and provides an outlet for our concerns about the human side of what we do."

Shepherding is valuable not only in its traditional role to make papers as good as they can be for a Writers' Workshop, but as a set of techniques well-suited to activities like the EuroPLoP writing workshops. It doesn't replace the writers' workshop, but supplements it; the human dynamics are completely different. Think of shepherding as a quality assurance step. Deming notes that quality decreases with an increase in the number of quality engineers. Why? Each thinks that the other will catch the problems, so each one becomes less attentive than they would be if the onus of quality fell on their shoulders alone. A good shepherd is committed; there is no one else to pick up the slack in the shepherding phase, and the author knows that, and the shepherd knows that the author knows that! Writers' Workshops, on the other hand, engender a social dynamic where one person's thoughts can build on those of other reviewers, and that takes the review discussions into areas of consideration that might not come to mind for a single reviewer. Neil adds, "There is also the difference that writers' workshops are a one-shot, impersonal evaluation of the pattern, while shepherding is iterative and highly personal. In one case, you judge the pattern as the world will judge it; in the other case, you can get inside the author's head to see what is really going on." Both review forums add value.

Neil has assembled some patterns of shepherding. Here, I relate some of the patlets for those patterns, patlets that were presented in an informal Pattern Shepherding session at PLoP last August. Some of them are about the shepherding process itself:

Three Iterations

The amount of work required to shepherd any work is an unknown quantity a priori.

Therefore:

Get started early, and plan on about three iterations. Pace yourself to be able to complete three iterations.

Half a Loaf

In our zeal to be the best shepherd, we often dump a boatload of comments on the poor author.

Therefore:

Give small sets of comments more often; give half a loaf rather than the whole loaf. Start with the highest priority problems.

And many of them are about the makeup of the patterns themselves:

Forces Define Problem

Problem statements are often weak. In many cases, the problem statement presupposes the solution.

Therefore:

Use the forces to help crystallize the problem. Help the author to iterate between the forces and the problem statement.

Big Picture

It is hard to know where to start with a pattern.

Therefore:

Get the big picture by reading the problem and solution together first. They should be able to pretty much stand on their own.

That's just a patlet-level taste of the patterns Neil is developing, patterns he will present at PLoP '99. I hope to see these patterns appear on a public web site and in a suitable publication in the near future. They can help provide consistency across the individual shepherding experiences for PLoP authors and bolster the standards of excellence to which our community aspires.

Epilogue

Several years ago I was program chair of OOPSLA '96, a more conventional academic conference with a traditional refereeing process and program committee. As is true with most conferences, the committee found it had papers on its hands that contained the germ of a good idea, an idea that had not yet been fleshed out effectively. Someone put the idea on the table to shepherd these papers; they would be conditionally accepted, subject to revisions. It wasn't a new practice, but had seen limited application in past OOPSLAs. And it is a common practice in professional journal editing. But the number of papers we shepherded grew at OOPSLA '96. At OOPSLA '97, the shepherding effort grew to about a handful of papers, a sizable fraction of the papers accepted for the technical program. To this day I still remember the incredible effort that Adele Goldberg put into helping her sheep. I think she spent more time writing the shepherding comments than some authors do writing conference papers.

I don't know if OOPSLA is still doing this, or if the shepherding practice grew at OOPSLA because it had been found successful in the pattern community. I do believe it's a good thing, and brings more dignity and value to the processes we use to admit new works to a community's body of literature. If you have an opportunity to participate in a program committee, please give it a try. Not only will you see the conference papers vastly improve, but you will have program committee members who will feel good for having made a tangible contribution to the works, and a more grateful author community.

Signposts

Many thanks to Christa and to Neil for input and comments on this article; the ideas are all theirs!

Christa and I will be running Writers' Workshops at OOPSLA (with Richard Gabriel and Honna Segel) the first week of November. See you there.

References

• [Coplien1997] Coplien, James. The Column Without a Name: Writers' Workshops. C++ Report 9(4), April 1997, 51-60.

• [EuroPLoP1998] Coldewey, Jens, and Paul Dyson, eds. Proceedings of the Third European Conference on Pattern Languages of Programing 1998, Kloster Irsee, Germany. Universitätsverlag Konstanz Technical Report, July, 1998.

• [PLoPD1] Coplien, James, and Douglas Schmidt, eds. Pattern Languages of Program Design. Reading, MA: Addison-Wesley, 1996.