Evaluation of Resident Performance
The General Surgery Training Program is committed to comprehensive, regular and timely evaluation of the educational and professional performance of surgical residents. This section presents the goals, components, and processes of the resident evaluation system.
The Goals of the Evaluation System are to:
1. Promote professionalism and the maintenance of a life-long portfolio of career accomplishments
2. Provide information on resident progress and performance in order to:
Make informed decisions on resident promotion
Provide data to specialty boards for certification
Write letters of recommendations
Identify performance deficits and thereby improve performance
Identify program strengths and weaknesses and target areas for modification in the training curriculum or program structure
General Expectations
Residents are expected to achieve at high standards of performance. Further, we consider residents to be adult learners, and as such, responsible for self-directed, proactive learning throughout their training. We expect them to monitor their own progress, and to consciously work to acquire the habits of mind, the professional attitudes and demeanor, as well as the knowledge and skills of a consummate surgeon. We expect residents to:
1. Develop a personal program of self-study and professional growth with guidance from faculty advisors.
2. Participate in safe, effective and compassionate patient care under supervision, commensurate with their level of responsibility.
3. Participate fully in the education activities of the program and, as required, assume responsibility for teaching and supervising other residents and students.
4. Participate in institutional programs and activities involving the medical staff; and adhere to established practices, procedures, and policies of the institutions.
5. Serve on institutional committees and councils, especially those that relate to education and patient care review activities.
6. Annually, compose elements of an electronic portfolio and present this portfolio to members of the Surgical Education Council (SEC) and Program Director for review.
Components of the Evaluation System
The evaluation system is based on the ACGME core competencies as presented previously:
Patient Care
Technical Skills
Medical Knowledge
Practice-Based Learning + Improvement
Interpersonal Skills + Communication
Professionalism
Systems-Based Practice
The Department has developed evaluation tools that provide the program, as well as the residents, with information pertinent to these areas.
American Board of Surgery In-Training Exam (ABSITE) scores
Faculty clinical evaluations, for each rotation assignment
Phone app “SIMPL” evaluation tool
Medical student evaluations of resident teaching in skills lab and on rotation
Operative logs
Tuesday conference attendance
OSATS (Objective Structured Assessment of Technical Skills) (PGY 1 + 2 only)
Mock oral exams (PGY 4 + 5 only)
Certification in the Fundamentals of Laparoscopic Skills by chief year
Presentations at M+M and Grand Rounds
Lecture materials, teaching presentations, or curricula
Participation in professional development courses, conferences
Membership in professional organizations
List of papers, abstracts, posters, presentations
SIMPL Phone app evaluation Tool
SIMPL: System for Improving and Measuring Procedural Learning is a smart-phone based system that makes it feasible to evaluate residents after every procedure they perform. The SIMPL app, used by residents and faculty, is designed from the ground up for efficiency and ease of use while still providing meaningful evaluation data. Each evaluation consists of 3 questions plus the option to dictate formative feedback. Aggregated evaluation data is accessible to faculty who can use it to see how much experience a resident has with any given procedure or with a group of procedures. This knowledge can, in turn, serve to inform a conversation with the resident prior to each operation so as to better individualize both teaching and supervision in the operating room. SIMPL is a smart phone based assessment system that utilizes the four-level Zwisch scale to assess the level of autonomy achieved by a resident in performing a surgical procedure. After an evaluation is created by any participant in an operation, a notification is immediately sent to the phone of other participants asking them to complete an evaluation for the procedure they just performed. Completion of the entire assessment requires only a few seconds and does not disrupt busy surgical workflow. Evaluations can also be completed while the user is offline (e.g. in an elevator) and will be uploaded after the phone again has an internet connection.
Biannual Review Process
Members of the Clinical Competency Committee (CCC) meet twice a year to review resident progress. For the first review in January, they review data from the previous six months. The second review takes place in May and data from the previous six months is reviewed. After reviewing key evaluation elements, they provide milestone scores for the sixteen surgery-specific milestones as determined by the Surgery Review Committee (RC) of the ACGME. Residents then meet individually with their assigned advisors to review this feedback. Adviser Assignments
1. PGY 1 residents: PGY-1 Director
2. PGY 2 + 3 residents: PGY-2/3 Mentor
3. Laboratory residents: Laboratory Mentor
4. PGY 4 + 5 residents: Program Director
Annual Promotion
There is no single criterion for successful promotion and no single criterion for academic probation or dismissal. Rather, the CCC looks at the totality of the resident’s progress and becomes concerned only if there is a pattern of “red flags.” In reviewing the mid-year or end-of-year milestones, the CCC becomes concerned if:
Faculty evaluations fall consistently below average for any ACGME competencies.
Professional indiscretions are identified by faculty, nurses, students, or peers.
ABSITE scores (Total Test Percentile Score) fall below the 30th percentile.
ABSITE Performance Standards
Because of the centrality of medical knowledge as a foundation for surgical competency, the Department has adopted a clear set of expectations regarding performance on the ABSITE. The Department desires to “leave no resident behind,” and requires remediation if performance falls bellows the 30th percentile. This cut-point has been shown to be predictive of performance on the ABS written boards (Qualifying Exam). The ABSITE performance expectations are as follows:
Scoring below the 30th percentile (Total Test Score) on the ABSITE for the first time places a resident at risk for academic probation. The at-risk status is removed if the resident’s subsequent year’s performance exceeds the 30th percentile.
Scoring two times in a row below the 30th percentile results in automatic academic probation. The probationary status is removed from the resident’s file if the subsequent year’s performance exceeds the 30th percentile.
Scoring below the 30th percentile for a second time in three or more years, after one or more years of improved performance, places the resident back in the at-risk status for academic probation.
Scoring three times below the 30th percentile (either sequentially or intermittently), despite attempts at remediation, and in combination with poor performance in other areas, places a resident at risk for dismissal from the program. The ABSITE score will never constitute the sole criterion for promotion decisions or dismissal, however it has been shown to be a valid predictor of performance on the written ABS board exam, and as such will be taken very seriously in determining satisfactory progress towards graduation.
Residents at-risk for academic probation, and those on probation due to their performance on the ABSITE, are required to participate in remediation as directed by the Department. Residents who are at risk or on academic probation during any year they are scheduled for the research lab will have their moonlighting privileges significantly reduced or removed. Privileges will be regained if performance on the subsequent year’s ABSITE exceeds the 30th percentile. Review Decisions
The Clinical Competency Committee concludes its discussion of each resident with one of the following recommendations:
1. Advancement with statement of exemplary performance and any areas to develop. 2. Advancement with statement of deficiencies to be improved. 3. Advancement with notification of one-year probation and statement of deficiencies to be improved. 4. No advancement with one-year probation and discussion of alternative career choices. 5. Unsatisfactory performance and dismissal from program.
Appeal Process
A resident may exercise the right to appeal any decision regarding plans for non-renewal of contract or dismissal from the program. This process is outlined in the resident contract as well as the resident manual. The Department of Surgery, General Surgery Residency Training Program in such case will carefully follow the University of Minnesota, School of Medicine appeal guidelines.
Overview of Expectations, by Resident Level
The first two years of residency training are designed to produce a surgical trainee who can thoroughly evaluate patients for elective and emergency operations, and who can competently manage nonsurgical illness (e.g., acute pancreatitis, trauma, portal hypertension, acute renal failure, acute respiratory failure, sepsis syndrome) by applying a broad knowledge of basic and applied physiology. The PGY-1 and PGY-2 resident is exposed to a wide number of surgical subspecialties to promote an understanding of wound healing and management (burns, plastics), general and cardiac critical care, fracture care and hand evaluation (trauma, plastics), care of neurological trauma and emergencies (neurosurgery), urology, and gynecology.
• The PGY-1 resident will be expected to do a complete preoperative evaluation and postoperative care plan, using his or her mastery of physical examination skills, fluid and electrolyte management, nutrition, wound healing, microbiology, and metabolic response to stress. During each rotation, the resident will accomplish these goals through structured lecture material, clinical experience, and self-motivated learning. By the end of the first year of training, the resident should be competent in performing bedside procedures (central venous catheterization, pulmonary artery catheterization, tube thoracostomy, pericardiocentesis, and fine-needle aspiration), advanced cardiac life support, and advanced trauma life support protocols. PGY-1 residents are tested on many of these skills in the SimPORTAL.
• The PGY-2 resident will build on a foundation of knowledge in physiology, anatomy, microbiology, and clinical patient care to care for increasingly complex general surgery, cardiovascular, transplantation, and critical care patients. Skills will include: opening and closing the abdomen, obtaining abdominal access for laparoscopy, and performing open hernia repair, appendectomy, and gastrointestinal anastomoses.
• The PGY-3 resident will focus on the general surgery services. Advanced surgical skills will be developed. The resident will independently evaluate surgical consults and emergency room patients, and prepare patients for elective surgery. He or she will be expected to independently formulate differential diagnoses and develop diagnostic evaluations for complex surgical problems. Surgical skills will include increasingly complex operations (e.g., thyroidectomy, tracheostomy, small and large bowel resection, laparoscopic cholecystectomy, splenectomy, Nissen fundoplication), performed with appropriate assistance. In addition, the PGY-3 resident will develop increasing responsibility in the education of more junior residents and medical students, through both formal and informal teaching sessions.
• During the laboratory years, the resident will be devoted to surgical research, with careful mentoring. He or she will develop the skills necessary to independently generate and test hypotheses, apply for and obtain extramural research funding, and critically evaluate published literature. For many of our residents, this experience has a profound impact on the direction of their future research careers.
• The PGY-4, or senior, resident will perform core surgery procedures with the assistance of staff. He or she will have an understanding of the medical and surgical literature beyond textbooks and review articles, as well as advanced knowledge of available diagnostic tests and nonsurgical therapeutic alternatives. Beyond this, the resident will develop the skills and confidence to function independently in evaluating and caring for general surgery patients and their families. The PGY-4 resident will function with the highest degree of professionalism and integrity in all interactions with all levels of hospital staff. By the end of the fourth year, the resident will have developed all the skills necessary to function as a chief resident.
• The PGY-5, or chief, resident will develop an ability to function in accordance with independent surgical practice. He or she will become facile in the conduct of all aspects of surgical care, will coordinate the service, and will serve as the first-line leader and role model for residents and medical students. The chief resident will build on the skills formalized during the fourth year, gaining additional independence and experience in patient evaluation, decision-making, operative conduct, and post- operative and follow-up care. The chief resident will conduct complex multistep procedures (e.g., esophago- gastrectomy, abdominoperineal resection, low anterior resection, and pancreatico-duodenectomy) with minimal direction and intervention by the supervising surgeon. The chief resident will be able to anticipate and treat the complications of general surgery procedures and synthesize an understanding of these complications through morbidity and mortality conference presentations. The chief resident will be able to speak publicly in an authoritative, articulate manner, serving as a repository of knowledge for more junior residents.
BACK TO: