The AEA Learning Circle Fellows

American Evaluation Association

Closing the Circle Reflection

In November of 2008, AEA contracted with Margaret Riel to work with a group of 14 selected Learning Circles Fellows. The goal was to help them learn how to use the learning circle model in their work. We met for a 2-day workshop at the AEA annual meeting (led by Dr Riel and Dr. Purrington) and then Dr. Riel continued to work and meet online with the learning circle fellows for the next six months. After that period, the initial learning circles remained “on-call” as a support group as the AEA Learning Circle Fellows experimented with facilitating learning circles in their own practice. At the completion of the first phase of learning circles, as a closing activity, the fellows were asked to reflect on the process. All but two of the evaluators contributed a circle reflection which I have used to create this summary.

Reflections on Planning to Facilitate Learning Circles

The reflections indicated both strong positive outcomes and the identification of a number of issues and challenges that were faced over the time we worked in circles. Some of them might have been avoided by minor adjustments to the process, others suggest ideas for substantial shifts in the approach to helping evaluations learn to facilitate learning circles. The topics that were central in the reflections provide the structure.

Circle Interactions:

The AEA evaluators were an exceptional group of researchers to work with. They were located around the world and work at many different types of institutions with varied approaches to evaluation research. The most common expression in most of the circle reflections was how much they enjoyed meeting and working with each other. In general, we were successful in creating a level of trust and respect that is essential for collaboration in learning circles. Here are some of the comments that indicate that the groundwork for learning circles appeared to have been established:

My own participation in the Learning Circle to plan a project was richly rewarding, as I learned much about evaluation practice and the unique concerns of my colleagues in their diverse environments of Chicago, Los Angeles, New Mexico, and Viet Nam!

The relationship aspects of this opportunity meant the most...both within and across circles. Perhaps, as I wrote earlier, the diversity of our group allowed me to think differently about how I might do my own Learning Circles in the future.

In general, this learning circle experience has been meaningful, fantastic, and well worth it to me. I'd like to thank you all who initiated, contributed, shared, and turned this into a reality.

So, I'll stop my ramblings here and end with my extreme appreciation for each of you, my gratitude for being included in the process and my willingness to see this as a cycle of learning and not as a failure. I plan to continue to nurture the seeds planted within me and I know that valuable results will continue to generate from what I have learned.

The focus on shared learning, equally from the process and the content, is an important and compelling aspect of the overall experience. Working collaboratively within an online community, using internet tools, offers the promise of reaching across barriers that limit our face to face world to less diversity of thought, background, experience, etc. The potential to connect the learning to a practical activity that produces some product holds the potential of embedding the learning opportunities into our practice as evaluators, one of the hallmarks of good professional development.

I particularly learned a lot from reading the LC plans of others, and from seeing how others created their Google sites….What worked well for me was the flexibility in creating our own LCs, having Margaret and others serve as resources, and knowing that the process was allowed to evolve. And what I valued most about the experience was having thought-provoking conversations, sharing laughter, and networking with a great group of people.

As for the November 2008 learning circle, I appreciate the follow on work done by the small group after our initial training. I don't think as many of us would have been as motivated to design and implement our circles if we didn't have this sort of support, sharing, and accountability. And I will stress accountability, as it was the expectation that each member would actually run a learning circle after attending the initial training. As for my small group, each of our circles had different motivations, different participants, and different leadership styles. Getting to see how it all worked up close with three other colleagues helped me understand the myriad ways in which learning circles could occur.

I really enjoyed having a way to connect with a community of evaluators even though my project didn't get off the ground. I work in a corporate environment where I am the only evaluation professional, so having the opportunity to connect with other evaluators and to hear about their work was the highlight of my LC experience this time around. ...I enjoyed the time we spent together very much.

And the issue of developing the relationships in the circle also extended to the experience of creating learning circles in new contexts:

By setting up a Learning Circle that set aside time and space to do this work and to value collaborative deliberation of ideas as they morph into publishable articles was both invigorating and challenging. We all matured in the process, came to recognize common and diverse interests. We came to see strengths and challenges for each person and to work better as a team of colleagues. We also wrote. As a result, we are presenting more of our work and we have two manuscripts in review by refereed journals and others in progress. Today, we continue to include writing and thinking together as part of our regular meeting schedule. The staff still refer to this as “learning circle days” however, the culture has spread beyond set times and spaces and I notice more (and deeper) collaboration across projects and report writing as well as academic efforts. While my staff is publishing more, my schedule still seems to conspire against me and I often take second seat or third as author as a way of maximizing graduate student experience and also managing my too-busy schedule.

I think it's interesting to note that one of our largest clients picked up on the notion of learning circles and, in fact, embraced the idea for internal professional development specifically focusing their efforts to co-produce publishable pieces as part of their work product. This is suggested and highly valued by their funder, so it is a useful tool for them and us too have been successful in generating work under review.

Technological Issues and Concerns

Not everyone was clear at the onset that this learning circle would involve the use of technology and the member of the group had vastly different experiences with and ways of approaching technology. Often learning circles are part of a large community, and the technical structure for communication is established as part of the larger group interaction. Since there was no shared technical structure available for this project, we chose to experiment with some tools that were relatively new—Google sites and embedded Google Groups for discussions. We also made use of Voice threads, Skype, and Etherpad. For some participants, learning how to use freely available tools to build and sustain a learning community was a critical part of the learning.

From the start, I knew that my project would need to involve asynchronous online discussion. However, participation in the Learning Circles Fellows project showed me that initial face-to-face relationships are essential in establishing the trust that is critical to successful collaboration. Learning of the need for this mix of “high tech” and “high touch” (ala Dave Moursund) was one of my unexpected outcomes from the Learning Circles Fellows.

I did appreciate using Google Groups, Meeting Wizard, and SKYPE and will use these tools in my own individual LC.

The idea of learning in a group via internet techno is not new to Vietnam but learning in an organized way like our learning circle is new to Vietnam. In the case of Vietnam, it helps level out the distance, that is, provided that learners have a strong commitment to participate.

However, for many others, it was frustrating that so much time was lost in getting technology to work and to develop online spaces which were not as useful in their research and evaluation contexts. Also because the tools were new to everyone, there was a level of experimentation in setting up the technology and supporting each person as they set up what they needed on their computers. The majority of negative comments hinged on aspects of technology:

The emphasis on technology and web-based collaboration was not particularly helpful to me personally… It sometimes “got in the way” of effective and efficient communication, rather than being a tool to enhance it. I think for the less technologically literate, it would be good to select a few tools and learn them well

The exploration of two many tools caused frustration as it was not clear where discussions or work was taking place.

There were too many changes in where meetings are posted and where we posted feedback. A small thing, but incredibly frustrating. There should be one place, and it should remain constant.

Cost-free technology such as Skype and other internet tools and services is amazing but not always effective especially during peak times or internet traffic. Therefore, an alternative should be considered during the preparation phase.

There is a kind of anti-technology culture in the communities I work in.

And for some, freely available does not provide the level of security that is needed.

Use of open tools like Google sites was not permissible in some contexts which require higher levels of security

We built a Google site for a project and some of our partners were forbidden to use it at their workplaces. So security issues need to be addressed. We need collaborative web-based environments that are fully accessible to all partners and LC participants. Free is not enough.

Time Commitments and Constraints

In all professional learning, the issue of time and commitment poses a challenge. Rarely does a professional have enough time for their own growth and learning within the press of the workday. Building learning circle participation into a part of the workday rather than something extra to be done leads to higher levels of success. However, in this case, all of the learning circle participants were interacting outside of the job-related work. This made it more difficult to work together and the rate of participation more impressive. However, most participants mentioned time as a factor that challenged their learning:

One of the things that I think got in the way for many of us, most assuredly me is the lack of time to commit. Best intentions offset by the tyranny of the urgent (as one of my favorite colleagues phrases it!)

Evaluators' time is not their own. That makes any timeline for his/her own development very difficult to follow.

Differing time zones and personal and professional priorities made working synchronously challenging to say the least.

It was very challenging for me professionally to make our collaboration work given the challenges of different time zones and the high demands of our respective work environments.

I had little time and flexibility for the intensity of effort that I needed to invest in this process.

The truth is when presented with competing priority tasks, my understanding of the learning circle was never solid enough to get it moved up the list to the top. I found my work on the learning circle was more in my head than out on the website here or on any other medium. When confronted with my personal limit of human resources potential, the LC always lost (without a clone, one can only do so much...).

On the other hand, the timing of this experience worked better for others. They were in a position of leadership and this collaborative, distributed leadership model fit well into their workplace needs.

Participation in AEA Learning Circles Fellows appeared to be the perfect professional development opportunity for me at just the right time. I had just started in a new job where I would be focused on program evaluation full-time. My new responsibilities would include facilitating collaborative teams – sometimes virtual – to address program evaluation projects from within as well as external to the Munroe-Meyer Institute (MMI) for which I work.

I am looking forward to doing this process myself - starting in just a few weeks. I am glad that I made it a part of my normal work process, as that made it more real and brought the pressures and timelines of my everyday workload. In general, the more people can do this process in the ongoing standard business of their lives the better I would say.

Circle Size

First, in any learning circle effort, the coordinator has to gauge the time, interest, and motivation of the participants. The larger the group the larger the investment of time as each person brings work to the group, the smaller the group the less diversity of perspectives and higher possibility of low participation. The ideal size is four to six active participants.

Given the process of setting up these circles was an experiment across different constituents, and it was the first time for a group like this, it was difficult to predict the level of participation. As it worked out, four of the fourteen Fellows rarely participated after the initial workshop. During the workshop, some of them expressed concern over the focus on the use of technology. They were clear that in their work, an online learning circle would not work. But they were also less tolerant of time lost to technology problems. In final reflections, one of these fellows mentioned shifts in work duties and frustration with technical difficulties as the two reasons for his lower level of participation. With hindsight, it would have been better to have formed two circles of seven each, to achieve the optimal circle interactions.

The Task

The learning circle tasks in these circles operated at a meta-circle as the task was for the group to help each of the circle participants craft their own learning circle plan. This would have been great if the Learning Circle Fellows began with a strong understanding of Learning Circles and perhaps some experience in having participated in one. Fortunately, many of the Fellows brought career experiences that help them understand the process as they worked through planning for the second phases. Those evaluators with less experience found the task of understanding learning circles more difficult. Also, many of the evaluators were very reluctant to share work in progress. Thinking together with incomplete conceptual artifacts was difficult for many. There is an issue of a norm of privacy and some level of concern in sharing work in progress. Two of the evaluators suggested another way to have structured the time. They suggested a three phrase structure, in phase one, we could have participated in short learning circles around evaluation topics, then Phase two would be learning circles to plan for how to use learning circles. The implementation of the circle would have been phase three.

Maybe we should have done a learning circle together on an evaluation topic and learned more experientially through doing. I am compelled more and more by the power of action. I certainly felt somewhat stuck in the intellectual side where our discussions and descriptions and readings seemed quite clear to me, but when it came to action, I felt much less clear or confident.

I would like to have participated as a member of an LC focused on a specific professional topic before I set out to facilitate a circle on my own. I think the "meta" process of a learning circle about learning circles wasn't effective for me because I have such limited experience working this way. It's more than just the format or technology, it really represents a fundamentally different way of thinking about how I work and what I produce. Personally, I experienced a lot of anxiety about letting my work go public in the early, unfinished stages. I would value more experience to adapt to this learning style. I see tremendous potential for learning to work this way.

Trying to manage a new language (Learning Circles), a fairly new concept (collective PD across many folks and many places), we certainly were unclear.

Phase Two: Implementing Evaluation Learning Circles

Given issues of technology and time constraints, the fact that more than half of the evaluators had some level of experimentation with starting their own circles was impressive. For some, the stumbling block was the initial recruitment of people. They didn’t have a team of people who were driven to develop their skills and knowledge and they didn’t have enough of a sense of the process to feel comfortable in recruiting them.

I’m not sure, frankly, that I had enough of a feel for what a learning circle could be to be a good salesperson for it when it came time for me to recruit my own circle.

For me, what was the most challenging about this process was finding the right group. Learning circles require active participation, and trying to motivate a group of adults that doesn't have the interest or the time to "learn something new", as good for them as it should be(!) is sort of doomed.

Partners need to be ready, and not many of my partners are. So I had to put some of my most ambitious goals on hold.

As was expected, the timeline for experimenting with learning circles would be different for each person. There was no effort to have everyone start their experience with facilitating a learning circle at the same time. One person started during phase one, with others starting at all different times during the second phase.

I am looking forward to doing this process myself - starting in just a few weeks. I am glad that I made it a part of my normal work process, as that made it more real and brought the pressures and timelines of my everyday work load. In general the more people can do this process in the ongoing standard business of their lives the better I would say.

In contrast to the others, recruiting participants for my LC was done very differently from how others did it. Much of my summer was spent in conversations with others in the community about who the best person from the non-profit organization would be to participate in the LC, how these individuals felt about the time commitment involved in participating in the LC, how they and their organization would benefit, what the potential products might be, etc. In a sense, the preparation phase for my own LC was much longer that the preparation phase I underwent to participate in the larger LC.

Mostly I have used what I have learned to enhance the way I work with my clients, to bring the flavor of learning circles into how I conduct evaluation. That is a work in progress that may influence what I ultimately choose to do with the LC, since my topic was still a bit diffuse.

And there are those who have not, as yet, facilitated a learning circle but have found value from the process of thinking about how to support others in collaborative work through distributed leadership.

I think Margaret's teaching brought me further along as an evaluator and helped me greatly as I transition into the kind of leadership positions that senior evaluators are expected to take. In the end, that is the outcome that AEA should focus on. The experiment with technology per se was telling, but it is not the whole story of the LC experiment.

The joining together of the ambiguousness of expectations, technological murkiness, and reticence of my target audience all lead to not fully implementing a learning circle model. As you know, in the end, I didn't end up doing an online learning circle. I have used some of the skills and discussions in bringing together both evaluators and community-based organizations.

There was a lot of learning that I did that did not end up in the final product. For example, the whole LC concept was new to me and I was able to incorporate that into some group leading I do with colleagues and students. It has been incredibly beneficial and will be for many years to come.

I am extremely grateful for the opportunity to be exposed to the Learning Circles concept as well as online collaborative technology. Both were new to me and have opened the doors to some exciting possibilities in future professional work.

My own reflection is to see this as a successful experience with some reservations. I see it as a success as I think that all of the participants have some sense of what it means to be in a learning circle. The issue of balancing individual ownership and group investment in the circle is not easy and the nature of the task affects the balancing act. I think that this was clear both from the nature of our shared task (creating plans) and the tasks that were chosen for learning circles in phase two.

Work with the learning circle fellows has helped us be more explicit about the model and the ways that it changes as it is employed in different contexts. As a direct result of this experience, we have taken the time to develop this website. The purpose of this site is to pull together the community of people who are using learning circles with learners of different ages and for different purposes. By creating a shared space, hopefully, we can continue to discuss and develop the features of collaborative learning with distributed leadership that are the defining characteristics of learning circles.