Pilot Committees

Overview of Committees

The Pilot launched in fall 2020 as an effort to improve the training of surgeons. On the heels of network-wide data analyses, a group of researchers and surgery educators at the University of Michigan, Stanford University, the University of Kentucky, the University of Wisconsin, and Boston Children's Hospital convened to discuss the possibility of using the lessons learned from various analyses to drive changes in trainees' development of operative performance skills. These researchers and educators formed a steering committee (SC) and identified a driving problem: "Graduating surgeons may not be—and do not perceive themselves to be—ready for independent practice." Using this problem, members of the SC conducted multiple virtual meetings to engage in a root-cause analysis that was then translated into a working theory for improvement that guided the pilot's development and testing work.

The pilot has grown from these initial efforts into multiple committees working to advance a key driver of improvement:

• Trainee engagement

• Aims and measurement

• Teaching and learning

The SC plays a key role coordinating the pilot, helping to share insights across committees, and organizing the overall evaluation plan.

Engagement Committee (EG)

In order to assess a trainee's ability, multiple SIMPL OR observations are needed. Yet, for trainees, the simple act of requesting a SIMPL evaluation can be affected by multiple factors. To better understand factors affecting how and why trainees request feedback, we used an A3 Problem Solving approach. Using this process, members of the EG identified 5 countermeasures based on a robust understanding of the problem.

1. To support faculty and trainees in remembering to request feedback through SIMPL OR, we will develop "leaderboards" on the total number of requested evaluations as well as documented competencies every two-weeks (i.e., every 14 days).

2. To help incentivize both trainees and faculty in requesting SIMPL OR evaluations, we will develop reports that illustrate how well trainees are developing compared to standards and peers.

3. To help both trainees and faculty integrate SIMPL OR into their work routines, we will develop a "readiness to use SIMPL OR checklist."

4. To support faculty in providing more and more beneficial feedback, we will provide a "do's and don'ts" document for providing dictated feedback using SIMPL OR.

5. To better integrate SIMPL OR into the OR, we will develop a guide to prepare for and reflect on a SIMPL OR evaluation.

Aim and Measurement Committee (A&M)

The A&M committee is charged with two tasks: identifying index procedures and making sense of scoring models applied to SIMPL OR data. Three specialities have identified index procedures for a specific PGY. In addition to selecting index procedures, the committee investigated the usefulness and interpretability of various measurement models, i.e., statistical models used to generate an overall score, that is central to quantifying the pilot's overall aim.