Final Evaluation from Gatekeeper

To conclude the project, my gatekeeper/external advisor — the Director of the Hamden Public Library — evaluated the project. Since she was away on vacation at the time, I sent her a copy of the evaluation questions via email. She completed the form and returned it with her responses. The questions primarily focused on the areas of behavior and results, which correspond to Levels 3 and 4 of Kirkpatrick’s Model of Evaluation.

In her response, it is evident that she was very pleased with how the project turned out. Some of the lines that stood out to me as I was reading her responses are:

"Ms. Vigonte’s work took us beyond just a map with a key. She created an interactive online discovery tool that will allow visitors to reserve a room, if needed. Likewise, it gives the visitors a real time to tool to use on a smart phone. Again, this work is beyond our wildest dreams."

"...[this work is] beyond anything a professional consultant would offer us."

"Each staff member that has used the tutorial finds it extremely user-friendly, easy to follow, informative, and intuitive."

"I would be misrepresenting my expectations regarding results if I did not tell you that Ms. Vigonte delivered wayfinding materials much beyond my hopes."

"Ms. Vigonte did superlative work...An often skeptical and very tired staff have embraced her work, calling it superlative."

"It gives us the opportunity to give volunteers meaningful tools to serve purposefully."

"It will give our visitors powerful information to navigate our offerings and to better understand our answers as we help them find resources."

She also shared that they plan to establish a Greeter’s Desk in late August or early September. The map will serve as the primary tool that volunteer greeters can use to help patrons navigate the library more confidently and independently. It felt incredibly rewarding to hear that my work is going to be a part of that. 

Moreover, she mentioned that they plan to create a Spanish version of the map, and with the help of the online course developed for library staff, this task would be attainable. This reminds me of the importance of openly licensing our work. As seen in this instance, because the wayfinding map is openly licensed, they are able to make remixes and adaptations to suit the specific needs of their community. This extends the reach of the map and allows it to be of help to a greater number of people.

I am very happy with the evaluation I received from my gatekeeper. It means a lot to me that everyone — library patrons, staff, and administration — were satisfied with the outcome of the project. It’s fulfilling to know that the work I poured my heart into is making a difference in their organization. 

Through this evaluation from my gatekeeper, I realized how important it is to evaluate not only how the learners reacted and learned from the instruction delivered, but also whether there is a change in their behavior and what results were achieved for the organization as a whole. This provides a whole picture of how effective the instructional material developed truly is. 

As described in the narration of events, the problem that came up was that my gatekeeper went on vacation, so we weren’t able to conduct a face-to-face interview for the evaluation. Nonetheless, this was resolved by using a sort of asynchronous interview wherein I sent her the evaluation questions via email. 

Moreover, another hiccup was that the rollout of the map to the public was delayed. It was originally planned for this July but was moved to late August or September. Nonetheless, she reassured that: "The delay is by no means a reflection of the quality of the work...but is directed by the start of a new volunteer-based service, Library Greeter." Nonetheless, this didn't turn into a major problem because a pilot implementation had already been conducted, which sufficed to evaluate the outcome of the project.

As made evident in this final evaluation from the gatekeeper, one of the major strengths of this project lies in the thorough evaluation process that considered multiple levels of impact: reaction, learning, behavior, and results. 

The weakness, on the other hand, is mostly related to time. One is that due to scheduling conflicts, we weren’t able to conduct a face-to-face interview, which limited the opportunity for real-time dialogue that might have further enriched the feedback. Another is that, since the project follows a timeline based on the university calendar, we could not wait for the full implementation before conducting the final evaluation. This limitation may have prevented my gatekeeper from gathering deeper insights about the sustained behavior change among the participants and the long-term results of the project. Nevertheless, this is the reality of our work as educators. There will always be constraints related to time, and most of the time, we just have to make the best out of the circumstances handed to us.

Based on this experience, I would recommend to my future self to anticipate contingencies when drafting the project timeline. Always be prepared with alternative strategies and plan Bs, because in real-world practice, the circumstances are not always ideal, as many external factors can impact the timeline of an instructional design project. In the end, what matters is that we are ready to adapt and make thoughtful adjustments to see the project through.