Building Evaluation Capacity in Graduate Training Programs
Facilitating a community to support program evaluation planning and implementation across the biomedical sciences
WELCOME!
This project comprises a series of three facilitated, half-day workshops that are linked with intervening activities to foster skill development and reflection on program structure, methods, and goals. The workshops will occur at 4 week intervals and are designed to guide the development and deployment of tools to support assessment and evaluation.
Structure: The three day-long workshops will be preceded by a project planning phase that will include a needs assessment that will allow the team to more effectively target the participants based on their skill level and interest. Each workshop will include a mix of conceptual and hands-on work, and feature talks by leaders in various aspects program design, management, and evaluation. Though the workshop series will finish in June 2021, the program leaders will use this website and other existing infrastructure and networks to continue to facilitate communication and collaboration among participants for the coming year and beyond.
This program is funded by a supplement awarded to GM007628 “Training in Pharmacological Sciences” at Vanderbilt University to support the development of a program of study to build capacity and competence in graduate program evaluation.
Workshop I: Program design with quality improvement in mind
Summary: participants re-conceptualize their programs as “outcome generators” instead of training implementations and begin to model and map their own programs, identifying the connections between intended outcomes and discrete training practices or program elements.
Guest Speaker: Dr. Gundula Bosch, PhD.
Johns Hopkins R3 Center for Innovation in Science Education
(See Workshop 1 page for details)
Workshop II: Mapping indicators with instruments and methods
Summary: In this second session, you will be working in teams to create a full-scale evaluation matrix, connecting all indicators associated with training activities and desired outcomes with the tools to measure them and methods to analyze them.
Description: This workshop will build on the work that you completed in Workshop 1 and practicum session. You and your team will make short presentations summarizing your work from the first workshop and practicum session, including any model development or other tools for your program, and we will begin reviewing and implementing instruments for evaluation. At the close of the second workshop, your team will be equipped to create a full evaluation plan, and continue the work in a second practicum intersession.
Guest Speaker: Dr. Betty Onyura
University of Toronto, Program Evaluation
(See workshop 2 page for additional details)
Workshop III: Improvement, broader impacts, and reporting
Summary: In the third session we will focus on reviewing your draft evaluation plans to date, and discuss interpretation of results, framing communications about your program, and the challenges specific to evaluating mentoring.
Description: You and your team will make short presentations summarizing your work from the second workshop and practicum session, including any instrument development or other tools for your program. We will help guide you in using your newly acquired skills and tools to examine both the strengths and the opportunities for improvement within your program strategies. You will also explore additional uses of evaluation results for communicating outcomes, possible broader impacts, and sharing their successes and challenges with their communities of practice. Guest speakers will discuss the opportunities and challenges in the evaluation of mentoring. At the close of the third workshop, you should have a substantial draft of a full evaluation plan for your program.
Guest Speaker: Dr. Suzanne Ortega
President, Council of Graduate Schools
Guest Speaker: Dr. Jenna Rogers
Research Director, Center for the Improvement of Mentored Experiences in Research
(See workshop 3 page for details)