Pilot Committees
Overview of Committees
The Pilot launched in fall 2020 as an effort to improve the training of surgeons. On the heels of network-wide data analyses, a group of researchers and surgery educators at the University of Michigan, Stanford University, the University of Kentucky, the University of Wisconsin, and Boston Children's Hospital convened to discuss the possibility of using the lessons learned from various analyses to drive changes in trainees' development of operative performance skills. These researchers and educators formed a steering committee (SC) and identified a driving problem: "Graduating surgeons may not be—and do not perceive themselves to be—ready for independent practice." Using this problem, members of the SC conducted multiple virtual meetings to engage in a root-cause analysis that was then translated into a working theory for improvement that guided the pilot's development and testing work.
The pilot has grown from these initial efforts into multiple committees working to advance a key driver of improvement:
Trainee engagement
Aims and measurement
Teaching and learning
The SC plays a key role coordinating the pilot, helping to share insights across committees, and organizing the overall evaluation plan.
Root Cause Analysis
Working Theory
Engagement Committee (EG)
In order to assess a trainee's ability, multiple SIMPL OR observations are needed. Yet, for trainees, the simple act of requesting a SIMPL evaluation can be affected by multiple factors. To better understand factors affecting how and why trainees request feedback, we used an A3 Problem Solving approach. Using this process, members of the EG identified 5 countermeasures based on a robust understanding of the problem.
To support faculty and trainees in remembering to request feedback through SIMPL OR, we will develop "leaderboards" on the total number of requested evaluations as well as documented competencies every two-weeks (i.e., every 14 days).
To help incentivize both trainees and faculty in requesting SIMPL OR evaluations, we will develop reports that illustrate how well trainees are developing compared to standards and peers.
To help both trainees and faculty integrate SIMPL OR into their work routines, we will develop a "readiness to use SIMPL OR checklist."
To support faculty in providing more and more beneficial feedback, we will provide a "do's and don'ts" document for providing dictated feedback using SIMPL OR.
To better integrate SIMPL OR into the OR, we will develop a guide to prepare for and reflect on a SIMPL OR evaluation.
A3 Problem Solving
Aim and Measurement Committee (A&M)
The A&M committee is charged with two tasks: identifying index procedures and making sense of scoring models applied to SIMPL OR data. Three specialities have identified index procedures for a specific PGY. In addition to selecting index procedures, the committee investigated the usefulness and interpretability of various measurement models, i.e., statistical models used to generate an overall score, that is central to quantifying the pilot's overall aim.
Teaching and Learning Committee (T&L)
The T&L Committee has worked to identify specific changes that attending physicians can enact with trainees. To identify strategies, the T&L members first brainstormed a series of procedure-agnostic strategies, i.e., strategies that are not specific to learning a given procedure. After generating a list of of potential strategies, the committee arrayed each strategy along two dimensions: "doability" and "impact." After much discussion, the idea of a preoperative briefing approach based on the "BID Model" was identified as a powerful instructional approach that could be readily implemented. The basic idea is to provide slightly more structure to a SIMPL OR evaluation by having a trainee reach out to an attending surgeon about his or her learning goals. Combined with a checklist to support SIMPL OR use and guidelines for providing dictated feedback, the T&L committee in combination with the engagement committee have identified a powerful, coherent set of changes to enact.
Doable / Impact Matrix
