Evaluations and Outcomes Assessment

Evaluation Process

Feedback and evaluation is ongoing and provides information regarding aspects of performance, knowledge and understanding. The program expects there is a spirit of openness, continuous learning and self-reflection. 

Feedback to Fellows

Faculty will observe, evaluate and frequently provide feedback on fellow performance during each rotation or similar educational assignment.   Feedback is provided to fellows after each clinical rotation by the inpatient attending(s), in person and via New-Innovations and semiannually by the program director (in person), faculty mentor (in person and via New-Innovations), and continuity clinic attending(s) (in person and via New-Innovations).

Fellows receive feedback from faculty members after each of their Clinical Conference presentations (via a web-based evaluation system).

Fellows receive feedback from the Scholarship Oversight Committee with each meeting, both in person and in a written summary prepared by the committee chair.

Fellows receive feedback from the Clinical Competency Committee semi-annually.

It is an ACGME requirement that the evaluations of a fellow's performance must be accessible for review by the fellow. All fellows have access to their evaluations in New-Innovations. There is also a private Google folder for each fellow where reviews, letters, Time Away reporting for the academic year,  ITE Scores and SOC committee reporting documentation are available.

Milestones

The Infectious Disease Milestones are the ACGME-generated metrics used to assess current fellows in ACGME accredited programs. These competencies are part of the goals and objectives of educational experiences. Evaluation of these milestone skills are assessed by faculty during inpatient, outpatient and research rotations throughout fellowship. The actual milestone scores for each fellow are generated after analyzing the cumulative evaluations for each fellow by the Clinical Competency Committee and making recommendation to the program director. Final assessments are at the discretion of the program director. A summary of the milestones follows.

New-Innovations/Residency Management Suite (RMS)

The University of Minnesota Medical School is part of a consortium of metro-area teaching health systems that use New-Innovations for GME. As part of an incoming fellow’s onboarding process, they will be required to use various checklists to meet institutional and participating site requirements. New-Innovations also serves as a repository and “source of truth” for each fellow’s training record. All fellows will receive New-Innovations training during their first-year orientation. Of particular importance to fellows is the evaluations module that is used; fellows are instructed how to use the evaluations menu, including finding open evaluations, completing evaluations in a timely manner and looking up competed evaluations on themselves. Question regarding New-Innovations can be addressed to the program coordinator or email rmshelp@umn.edu. 

Qualtrics

The University of Minnesota uses Qualtrics for an number of evaluations. At times, fellows will be asked to complete surveys and evaluations, generated by the GME Office and the program. 

Evaluations of Faculty and Educational Assignments

Timely and informative evaluations of faculty and rotations are an important component of gaging the success of the program and targeting areas of improvement. Fellows are required to complete any open evaluation in New-Innovations on rotations and continuity clinic. Fellows are also sent email links to evaluate didactic sessions at the University (weekly ID Intercity Clinical Case Conference and Tuesday morning Didactic lectures). At the end of each clinical rotation, fellows are also asked to log into a GME-sponsored Qualtrics site: "Trainee Evaluation of Individual Faculty" and evaluate a minimum of three faculty listed at the training site. Providing feedback on faculty also helps build a repository of documentation that is used when an individual goes up for promotion. 

Program evaluations are collected and reported anonymously and in aggregate form to sites and individual faculty. 

Scholarship Oversite Committee (SOC)

The Infectious Diseases Fellowship Program has instituted a Scholarship Oversight Committee (SOC) to provide consistent, systematic, and centralized oversight of fellows' academic progress. The specific goals of the SOC are as follows:

• to document for the program and the training grant the fellow's academic progress, including any need for redirection or

• enhancement

• to provide an outside perspective for the fellow and mentor regarding the fellow's academic progress

• to provide additional guidance to the fellow

• and mentor regarding desirable or necessary future professional development efforts

• to identify any problematic aspects of the fellow-mentor relationship, and to report these to relevant parties

• to improve faculty cohesiveness and communication

The SOC membership includes: 

• Current research mentors of adult ID fellows and any pediatric ID fellows who have played a roll in research with the program

• Faculty who are supportive of research in the Department and

• Adult ID fellowship Program Director (SOC chair)

Fellows to be overseen by the SOC include all adult ID fellows and any fellows/post-docs on the "Infectious Diseases Training in Clinical Investigation" training grant.

The SOC meets semiannually to review each trainee's progress. First-year fellows will have their first SOC scheduled in January of their first year. It provides a written report of each review (with evaluative comments and suggestions) to the fellow, the fellow's research mentor and career mentor, the fellowship program director and, for training grant-supported fellows, the PI of the training grant. Fellows submit written materials in advance of the semiannual SOC meetings, according to a structured outline. They then appear before the committee individually (30-45 min.) to present their report in person and to answer questions from committee members. When a fellow appears before the SOC, the fellow's research mentor is invited to attend the meeting as a (nonparticipating) observer. He/she joins the post-appearance discussion after the fellow leaves the room. Immediate feedback to the fellow is provided after this discussion; a written report is then generated and provided to the fellow and becomes part of the training record.

During a typical 2-year,

3-year, or 4-year fellowship, fellows meet with the SOC 4, 6, or 8 times, i.e. usually each Fall and Spring. The final meeting is used for a fellow’s assessment of the process and for benchmarking. 

The SOC does not replace fellow's individual Research Committees with respect to their role in shaping the content and structure of the fellow's research project. Fellows still need to identify content experts to assist them and their mentors in the design and conduct of their individual research projects. The SOC is designed to address the larger picture of overall professional development to which the fellow's research project contributes, but not to help design and direct the specific research project per se.

Clinical Competency Committee

The Clinical Competency Committee (CCC) meets semi-annually (January and June each year) to discuss and synthesize the clinical and educational evaluations and feedback of each fellow and provide comprehensive recommendation to the program director on the status of each fellow in the program. The CCC is chaired by Dr. Muth Narayan (Mpls-VA ID Education Director) and members consist of faculty at each clinical site (UMMC, HCMC, and VAMC) who have robust clinical exposure to the fellows. Current members include the following:

Dimitri Drekonja (MVAMC)

Muthu Narayan (MVAMC)

Meghan Shaughnessy (HCMC)

Rebecca Zadroga (HCMC)

Mahsa Abassi (UMMC)

Karam Obeid (UMMC)

Evaluations and reports generated through New-Innovations, Qualtrics and the SOC are synthesized along with additional input from faculty/staff with first-hand information on each fellow. This feedback and milestone recommendations are communicated to the Program Director. The Program Director takes this information under consideration and makes a final decision on the status of each fellow. The Program Director then conducts a face-to-face semi-annual review with each fellow. Milestone scores are uploaded to the ACGME website.

Confidential Reporting

This program strives to provide a learning and working environment in which fellows have the opportunity to raise concerns and provide feedback in a confidential manner as appropriate, without fear of intimidation or retaliation. Fellows are protected in the following ways.

• All ID teaching faculty are evaluated by fellows through the Graduate Medical Education Office in a confidential and anonymous manner.

• All ID rotations are evaluated by fellows through the program’s New-Innovations portal. All information is de-identified and reported in an aggregate manner, to protect the confidentiality of each fellow. Only the program director and program coordinator can identify evaluators. All evaluations of fellows by teaching faculty are handled in the same manner.

There are a number of ways for a fellow to report a confidential matter. Within the program, fellows are encouraged to contact the program director directly by calling, texting or using private email leuck002@umn.edu.

If a fellow is not comfortable contacting the program director, they may contact Carolyn Hildebrandt, hilde308@umn.edu, the Organizational Development Manager in the UMN Office of Graduate Medical Education. Carolyn provides free coaching and is a source for fellows to go to for any confidential matter. Dr Susan Culican, the Vice Dean of Graduate Medical Education may also be contacted through her private email, culican@umn.edu. U Report may also be utilized to report suspected misconduct.

Annual Evaluation of Program Goals and Objectives

The Program Evaluation Committee meets at least on an annual basis and reviews the goals of the program and provides input for improvement. Program leadership takes into consideration input from faculty and fellows from both ACGME surveys and ongoing discussions that take place throughout the year.

Please also see the GME Evaluation Policy.

Fellows as Teachers

Fellows are an essential part of the teaching of medical students and residents. It is critical that any fellow who supervises or teaches medical students and/or residents be familiar with the educational objectives of such rotations and be prepared for their roles in teaching and providing feedback.

ID Fellow Small Group Teaching 

ID fellows are encouraged to participate in small group teaching sessions for medical students. This is a great opportunity for obtaining experience in medical education and will be a valuable addition to a CV. If on inpatient service, the fellow may be excused for the 1-2 hours for the lectures. It is recommended to not schedule the lectures during outpatient clinic time. Fellows will need to plan ahead for this, and put a hold on their clinic spots during the lecture time at least 60 days in advance if giving a lecture during clinic time. If <60 days and no patients are scheduled, it is ok to request clinic spots to be held. If patients are scheduled and <60 days, fellows will not be able to accommodate the lecture.

LCME Requirements - 

The syllabus for MED 7521 Infectious Diseases Clerkship, which is offered at the three rotation sites of the fellowship program is available here. 

Resident ID Rotations

Residents rotate at each of the three rotations sites as well. Hennepin has their own IM Residency program, while residents at the VA and UMMC are both from the University program. Fellows are to familiarize themselves with the resident expectations.

• Details of the University rotation can be found here.

• Details of the VA rotation can be found here.

• Details of the Hennepin rotation can be found here.