Initial Prototype

• Color-coded display of student data for attendance, administrative referrals, and academics to show if a student's data is trending positively, negatively, or remaining neutral

• Ability to schedule circles that will appear on participating student's data so trends can be compared to the implementation of the circle

• Ability to share a student's data with other support specialists in the school if more interventions are deemed necessary for a student

• The initial prototype testing sessions consisted of two rounds. was conducted on 10/28/21 in Peabody Hall on the University of North Carolina Campus, Chapel Hill, North Carolina. Two laptops were used to observe CircleKeeper being used simultaneously. There were two testing rounds in the first session. The first round had a group of 3 MEITE students, and the second round used a group of 4 MEITE students. Each round lasted for roughly 25 minutes..

The testers were briefed on restorative circles prior to to the start of testing. The facilitator, David Russo, read the testers the short background information on testing circles as well as the background of the conflict that the testers would be using CircleKeeper. Each tester was polled to gauge if they had any experience in education, or with restorative justice circles as the target learners had experience in both

Testing Round 1

Tester #1 - Experience in elementary school, familiar but no experience using circles

Tester #2 - Experience in high school, familiar but no experience using circles

Tester #3 - Experience working in higher education setting, no formal teaching experience, no experience in circles

Testing Round 2

Tester #4 - no formal teaching experience, no experience in circles

Tester #5 - no formal teaching experience, no experience in circles

Tester #6 - no formal teaching experience, no experience in circles

Tester #7 - no formal teaching experience, no experience in circles

*The background information was sent to the testers via email the day before the 1st testing session. It was read to them beforehand. Each tester was provided a copy of the task.

Background Info:

Restorative circles are a tool that teachers, administrators, and restorative justice county officials are being trained to implement into schools to practice relationship building skills to strengthen relationships and connections within communities. Ultimately, WCPSS is implementing circles to reduce, prevent and improve harmful behavior within schools.

Restorative circles work by having participants and a facilitator sit in a circle. Norms are established, and a talking piece is shared, which allows one participant to speak at a time. The facilitator asks meaningful and thought-provoking questions to the participants that allow them to share thoughts and insights either to get to know each other better, or to get to the root of an issue.

Task:

1) You are a 7^th grade science teacher at Wake Forest Middle School. You completed a circle with 3 students in your period 2 class who were involved in a bullying incident 2 weeks ago and you want to know if the circle was successful. Tanya Sandler and Brittany Ballard were making fun of Carla Henderson's new haircut during group work. You checked in with Carla once since the circle and she says that she is "fine," but you want to be sure that this incident isn’t continuing to affect the girls. Use the Circle Keeper program to determine the success of your circle. Share your thoughts and findings.

• Can teachers determine positive or negative trends using student data following the implementation of circles?

• Can teachers evaluate data points and trends in student data to determine if the implementation of a restorative circle was successful?

• Can teachers identify students who need further interventions provided trends in student data?

Questions:

Before:

• Do you have experience teaching? If yes, please explain.

• When during the school day do you believe this program will be best utilized?

• What questions do you have right now, before using the program?

During:

• What trends do you see in Carla, Brittany and Tanya’s data?

• What data is associated with circles? How do you think it correlates?

• How do you think this relates to the implementation of circles?

• What would/ did you do next with these conclusions?

• 
  

After:

• How easy was it to follow a meaningful sequence through the program? Did you get frustrated at all using Circle Keeper? What would you change about the sequence?

• What do you see as being the main features of the prototype?

• Are there any parts of the prototype that should be further developed, changed, or gotten rid of?

• What do you think the overall learning goal is?

• How do you see this program being used in education?

Data was collected for the first testing session through two means:

• Tester actions and statements were video recorded by one observer, Jackie Russo

• Important remarks and comments made by testers, especially those pertaining to our "during and after" interview questions were recorded by our second observer, Alex Conte

Data was viewed and analyzed by the team together, watching snippets of the recorded video andreading the tester comments, feedback and suggestions closely to determine which points were made that would have the biggest

The testers found our graphs confusing. Many of them expressed some sort of indecision while making observations because of their inability to decipher or understand parts of the graphs. Therefore, we decided to make a number of changes to the graphs in our second prototype. The testers also noted that it would be helpful if the line indicating a Restorative Circle was clickable and directed the user to a page for notes

Most testers suggested our second prototype should include more qualitative data for teachers to analyze. Some of their suggestions were to include a teacher comment section for teachers to talk amongst each-other, and more data surrounding the students who participated in the Restorative Circles. They also suggested that there should be ways of sharing a teachers notes with other teachers. Due to this feedback, we updated our notes feature within the app.

Most testers emphasized that teacher interaction be maximized on the platform. Some suggestions included creating a platform for teachers to discuss individual students and a live notification system to provide updates on student behavior. In our second prototype we will modify our take actions slide to allow for more interaction.

Below is a list of the main revisions that were made to the Initial Prototype based on feedback from our initial testers.