CBD FAQs

• • A competence committee needs to use human judgement to assess competence, based on their review of not only the entrustment scores but also the  breadth of contexts encountered vs expected, narrative comments, consistency of scores and other program assessments (e.g. OSCEs etc.) to ensure there is evidence of a pattern of competence.

  • As far as the EPA Entrustment scores go, the goal is to accept only “I didn’t need to be there (in theory)” but we realize in the early implementation, we are all still learning how to use these assessments and so we need to read the narratives to determine if the description is in keeping with competence.  E.g. Score 4 with comment “They did a great job and I didn’t add anything to the care of the patient” might be considered by the CC as evidence of competence, whereas a score of 5 with comments “The resident tells me I have to put a 5” might not be considered by the CC as evidence of competence.

  • We ask that programs please not put in place blanket policies linking the achievement of competence in their EPAs to particular numeric scores, as determination of competence requires human judgement by the competence committee through comprehensive review.

  • An analogy would be that we don’t use numeric values alone to determine discharge of a patient (e.g. Hemoglobin = 100 so patient ready for discharge).  We have to assess other aspects of context and wellness (Is the patient still actively bleeding, what is the trend, what about living situation or co-existing issues?).

Click here to view the 4 vs 5 Entrustment Score descriptor that is featured in CBME.med

• The principles behind CBME suggest that when a resident is deemed competent in their EPAs they are meant to continue to work toward mastery/expertise level in those skills as part of their life long learning expectations. PGME recommends ongoing completion of EPA observations at 3 attempts per week even if the residents have been signed off on their required EPAs before the end of their training.  This is meant to encourage ongoing teaching moments and to document feedback for the purpose of ensuring competence is maintained and/or that residents are working toward excellence/mastery.   

• Regularly re-educate and re-engage residents and encourage them to share strategies for improving completion rates – this message needs to be repeated regularly

• Remind faculty that doing EPA observations in the moment right after an encounter is less time consuming than doing an ITER later (encourage residents to ask for them then)

• Request a standing ~5 minute slot in regular meetings attended by your faculty (rounds, business meetings, etc) to present EPA observation tips/reminders or other CBD topics to keep CBD on faculty members’ radar

• Add to your existing newsletter a section highlighting top EPA observers

• Consider a rewards / acknowledgement system such as EPA ‘faculty and resident of the month’ for number of completed EPA observations; some programs even had success with friendly competition between mini-teams of supervisors/residents

• Program directors/leads may review quarterly EPA observation reports to offer targeted faculty development or reminders based on faculty completion rates

• Remind faculty that their EPA observation completion and other CBD roles can be recognized through their Annual Report process 

• Discuss with your department whether EPA observation completion numbers could be incorporated into calculation of clinical stipends

• • On Dash, you can filter the view to show only those observations that were at particular levels on the entrustment scale by clicking that part of the scale.  When you then scroll back up to the context histograms, they will then only show those observations at that level. For example, you can click “I didn’t need to be there (in theory)” and then scroll up and see the context histograms.  The context sections have text indicating Royal College guidelines (e.g. at least 3 Direct observations), and you can then see on the graph if there where indeed 3 observations at that level. You will still need to review comments and possibly other levels (e.g. “I needed to be there just in case”) to see if they actually represented competence.

• • As a national requirement, every EPA observation form in CBD includes the question: "Do you have any concerns regarding this learner's professionalism or patient safety?" Unlike the FoMD's professionalism button, clicking 'yes' to this question in CBME.med does not trigger the professionalism process/review with the FoMD. It simply notifies the learner's program director and/or administrator that a concern was raised.

  • §  By noting professionalism concerns, the program's competence committee can see each entry in the learner's portfolio and determine whether it is a 'one-off oops' or a pattern of behavior showing up across a variety of preceptors or settings. Examples of concerns that might be noted include the following: punctuality, returning pages, professional language or clothing, etc.

Generally, the principle is that the resident should have competence in the task they are observing. It is the resident’s responsibility to enter a comment to explain how/why this peer observer might be competent to assess this EPA as the CC members will not know resident observers, particularly if they are from a different program.

•  Yes. Every time the Competence Committee meets, there will be learner status decisions (e.g. progressing as expected…) and there may be additional recommendations around promotion or a learning plans etc.  These recommendations must be taken to the RPC for ratification after each competence committee.

• All decisions must also be communicated to the resident after RPC ratification.

Review the following steps to make a decision:

• The resident needs to make an administrative application long in advance of the exam (about a year). 

• The program needs to make an attestation that the resident is on track to be ready for the exam long in advance of the exam (about 6-8 months).

• The Royal College expects that the program will review the resident in the 2-3 months prior to the exam for readiness and that the program will notify the Royal College if the resident is no longer on track to be ready for the exam.  As part of that, I would recommend ensuring the program has a CC meeting and RPC for ratification about 3 months prior to the exam.

• If things change with the resident's progress, the program can alert the Royal College even up to the night before the exam (not advised) that the resident is no longer ready for the exam.

Programs may need to make the judgement of whether a resident can be put forth for their RC exam in advance of their official stage change to Core and the program will need to project how likely they think it will be for the resident to be ready to go to the exam.  This is the same as what happened pre-CBD. If a resident turns out not to be as ready as the program thought as it gets closer to the exam the program can withdraw the resident from the exam as can happen now in pre-CBD programs.

PGME supports programs wishing to start CBD with later-than-launch residents.  This is at the discretion of those programs. Note that there are no accreditation or resident notification policies that would be breached if later-year residents move to CBD; expectations must be clear for these residents, and they are not permitted to ‘skip’ a stage. They will need to catch up on any earlier EPAs with the exception of late year residents where you have created a document to show training equivalency.  Programs are welcome to use the existing platform and resources, but we are not able to customize for hybrid applications.

• • • All Royal College CBD residency programs at the U of A must be on CBME.med and DASH.med.  However, electronic platform decisions for Family Medicine programs and the related CCFP enhanced skills programs are at the discretion of the U of A Family Medicine program leadership. Thus, Family Medicine approval is required prior to any exploration of the feasibility of a transition to the CBME.med and DASH.med platforms for CCFP program.

These will be handled through essentially the same process that existed for traditional programs.  Resident data and review will be done on a case-by-case basis and must be done through PGME. For CBD residents coming into your program, training and portfolios will need to be reviewed just as for traditional residents training and assessments would be reviewed.

• • At this time, PGME has decided to direct resources toward optimizing the current system, and so we will not be supporting the creation of new program-specific forms for other assessments.

  • Some programs are implementing additional assessment forms in One45, and after completion these are being uploaded as PDF in DASH.

• • Please contact our PGME tech learning analyst to ensure that the revisions have been submitted to for upload to the platform.

  • Next, review your EPA revisions and your existing curriculum map to determine whether your mapping needs to be revised to incorporate the changes. Adobe Pro has the ability to compare PDF documents and highlight changes, if needed.

  • It is important to communicate these changes to your residents, faculty, and off-service rotations so that they know which EPAs are affected.

Yes.  We have developed a readiness checklist and the Royal College also has a helpful implementation planner. 

Required steps include:

• Curriculum mapping: programs must either share their maps at the implementation meetings or directly with PGME (see curriculum mapping presentation tips below)

• Communication to off-service rotations regarding EPA assignments and expectations

• Meet with PGME at least 6 months prior to launch for any requests to remove off-service rotations

• Form a Competence Committee and develop terms of reference based on PGME doc

• Identify Academic Advisor(s)

• Coordinate training for Competence Committee members

• Provide PGME resident and preceptor/observer lists, short EPA names and form review/edits for online platform upload

• Host resident orientation to CBD (What is CBD, Why CBD, Online System training) and ensure communication to residents about EPA mapping and expectations

• • Examples from 2018/19 and 2019/20 launch programs

  • Royal College resources

  • Generally the Program Director or CBD Lead will walk through a brief summary of their program structure (e.g. we are a 5 year program but 1st part is mostly SF) and stages (we have about X number of EPAs and Y estimated duration of stages).  Then they show their spreadsheet on the screen. The things to look for are the following:

    • • 1. Have you identified required vs optional EPA assignments? This helps safeguard against multiple rotations all selecting the same choices out of a possible group of EPAs and nobody selecting the remaining choices.  For example if several rotations are offered 12 EPAs that they could choose and all complete assessments the 1st 8 and none do the last 4 then you have a problem. By allocating responsibility to rotation 1 for the 1st four and the rest are optional, the next 4 to rotation 2 and the rest are optional and the last 4 to rotation 3 and the rest are optional then you have less worry about some being left out.  We do understand that often the same EPAs can be achieved in many rotations. The trick is to have a system that ensure all are assessed without gaps.

        2. Is the volume of EPA expectations vs rotation duration reasonable (e.g. requiring completion of 12 EPAs on a 4 week block may not be if it takes about 3 tries to achieve success) and balance across rotations

        3. Ensure there are no EPAs without learning experiences and if there are learning experiences without EPAs that there are plans for other assessments.

• In most circumstances, it is ok for a rotation coordinator to be an Academic Advisor.  However, if the rotation coordinator has a major role in making final decisions on resident progress (e.g. for higher year traditional residents who may be participating in some aspects of CBD while maintaining their traditional assessment structure), there is potential for a conflict of interest in that on one hand they are supporting/coaching/championing resident success as an Academic Advisor, and on the other hand acting as a gatekeeper for a final progress decision.  This is why it is not advisable for CC chairs to be Academic Advisors and that a CC member would recuse themselves for decisions about their own advisees. If the rotation coordinator is only contributing to one part of a resident’s assessment/progress decision, then it may be manageable, but it requires some explicit conversation.

• For all live CBD programs, it is mandatory that there is a process to provide each individual resident with regular Coaching Over TIme based upon their portfolio assessment data.  We strongly encourage that this be accomplished using Academic Advisors: each resident should have an Academic Advisor assigned with whom they meet on a quarterly basis to review their assessment data and discuss strategies for ongoing learning and improvement (or more frequently if needed). (Please see the Academic Advisor Role Description for details on the function of this role and the Coaching Over Time module for how to implement longitudinal coaching.) 

If any program is unable to implement academic advisors, they should request a meeting with the PGME CBD implementation team to present a proposal for an alternative plan for providing regular Coaching Over Time to their residents (email pged@ualberta.ca to request). 

EPA completions will be automatically uploaded to the ARO system.  Other activities related to CBD can be entered in your annual report using this guide 

• You may claim Section 1 MOC credits for attending accredited (1 credit/hr) or non-accredited (0.5 credit/hour, max 50 per cycle) faculty development sessions related to your role or CBD topics

• You may claim Section 2 MOC credits if you engage in reading about how best to do your role as an AA/CC/RPC; or if you create a learning plan or revise/create a curriculum as part of your CBD role

• You may claim Section 3 MOC credits if you get feedback on your performance as an AA/CC/RPC member: the time you spend reviewing this feedback and reflecting on it can count for section 3 credits (3 credits per hour spent, or 1 credit per 20min).  This is easiest if the CC is doing mock cases or discussing its handling of a situation.  For an AA the CC could give feedback about how they performed their primary review of resident assessment data.

• Unfortunately, the regular work of a CC/AA/RPC generally cannot be claimed for MOC credits unless there is some kind of development/adaptation of a curriculum/learning product or reflection on feedback.

• Certain Royal College Online Modules (eg. “Coaching to Competence,” “Case Scenarios: Mock Competence Committees,” or “Entrustability Scales: WBA Rating Anchors to Trust”) may also be completed for MOC credits.

In brief, this may happen but only rarely due to the following considerations:

• In its implementation of CBME nationally, the Royal College has introduced CBD as a hybrid curriculum where progression is based on demonstration of competence without being time-free since residents also play a role in contributing to healthcare delivery and therefore predictable scheduling is needed.

• Furthermore, residents should only be considered for accelerated training when they are clearly on a steeper trajectory / learning curve (as in the cases of residents with prior training in the field, for example). There is a difference between a resident whose development of competence overall is clearly accelerated vs a resident whose developmental learning curve is similar to the usual rate but has just completed lots of EPA observations to have them done early. In other words, residents who are simply able to collect and document EPAs more quickly without being clearly ahead of their expected trajectory or cohort in terms of competence, independence, and maturity should not have their training program duration truncated.

• In addition, EPAs are only one part of a resident’s program of assessment. Each program will have defined a set of requirements of training that residents will have to complete to ensure they are competent.

• Residents who complete program expectations and EPA requirements early should continue seeking EPA observations to receive and document feedback / coaching towards mastery of skills (see FAQ#2 above).

• For further information please see; RC guidance doc on the role of time in CBD  

Appealing assessment decisions occur in two steps: there is an informal appeal and a formal appeal.  The informal appeal is directed to the program director or program designate – the resident discusses their concerns and reasons for appeal. If the informal appeal discussions do not resolve the matter, then the resident may submit a formal appeal to the Faculty Academic Appeals Committee (FAAC). For the formal appeal process, the appeal is submitted to the Vice Dean, Education. The FAAC does not have any jurisdiction to hear a formal appeal unless the informal appeal process has been exhausted. There needs to be ground for an appeal, and that is outlined in the PGME Academic Appeals Policy. Residents cannot appeal Competence Committee decisions about individual EPA achievement, but they may contest any decision ratified by the RPC (stage progression, promotion, readiness for exam or practice) if there are accepted grounds for appeal. 

Although a major strength of competence committees is the diversity and combined expertise of its members, the use of group-based decision making means that they are susceptible to conflicts arising in interpreting learner data.  As Chan, Oswald, Hauer et al. (2021) describe, these conflicts can stem from conflicting/discrepant data about the learner, conflicts between CC members, or conflicts of interest within CC member roles.  Refer to the article by Chan, Oswald, Hauer et al. 2021 for strategies for (a) diagnosing sources of conflict in your CC, and (b) mitigating and addressing conflict with CC processes and development.

*Common conflicts of interest in roles that should be avoided wherever possible (or have a mitigation strategy if there are personnel limitations): 

• A learner’s Academic Advisor or member engaged in that learner’s remediation / major learning plans should recuse themselves from voting on decisions regarding that learner

• The program director should be a non-voting member on the CC

• Leaders who have positions in hiring / privileging trainees, or deciding on future training opportunities should not be on the CC wherever possible

Besides EPA observations, the CC should have access to any/all other assessment data that would be part of the program’s expectations for assessment, e.g. ITERs, OSCEs, formal presentation assessments, written exam scores, scholarly project reviews etc.

• • • • If the program requires these assessments then they should be part of the resident’s comprehensive review by the CC.

      • Examples of things that might not be included could be personal self-reflections meant for the resident’s eyes only.

This is not advisable due to the risk of conflict of interest between the role of assessor and coach/mentor.  We recommend that academic advisors recuse themselves from the decision making part of the CC meeting for a resident that they coach and we would not want the CC Chair to have to recuse themselves from part of a CC meeting.

In addition to resident status (progressing as expected, not progressing as expected, failing to progress, progress accelerated, etc.) and promotion decisions, the competence committee is expected to review EPAs for individual residents at each meeting in order to determine if they can be signed off as “competent” in that EPA.  This allows residents to be notified so that they can focus their subsequent feedback requests on outstanding areas.

Competence Committee meetings should be recorded through our online forms system (user guides available) rather than through traditional minutes. We strongly recommend using the comment section on these forms to provide brief justifications of decisions; more details may be needed for contentious decisions, but this can still be recorded in these comment boxes. In particular, if you are progressing a resident who has not met the national assessment guidelines from your specialty committee, the justification should be documented in the associated comment boxes. Please do not maintain separate minutes with information about individual resident discussions or progress decisions. Programs may wish to maintain separate minutes for content that is not related to individual residents, such as general committee decisions regarding group processes or general EPA or curriculum mapping concerns. For example, if there is an EPA that you have flagged as needing review by the specialty committee, this would be a good place to document that information. Ensure that all content is neutral in language as both online and separate minutes could be “FOIPP-able”.

While the national assessment guidelines are meant to be treated as guidelines and there is some flexibility, there should be justification documented if you choose to recommend progressing residents who have less evidence of competence than outlined in the guideline.  However, it would be more difficult to justify these CC decisions to sign off on particular EPAs or to recommend to progress residents if the deficiencies are in multiple domains.  External accreditation reviewers are being trained to watch for CCs progressing residents with deficiencies in multiple domains and so this is a particularly risky practice. We have also seen on several occasion situations where residents who are progressed with not having met all expectations outlined in the guidelines then promptly significantly reduced their completion rates for EPA observations.   In these cases, residents appear to have interpreted these actions by the CC as messaging that they don't need to do their EPA observations and can still be promoted.   

Primary reviewer prepares the CC prep form, and the program can decide who they want as the primary reviewer (e.g. Academic advisor, CC member, other). Academic advisors who are primary reviewers can join to present their primary reviews at the meetings.  Once the CC is established in its processes and calibrated as to expectations, primary reviewers may submit a written review for residents that appear to be progressing as expected. 

In competence committee meetings, each resident must have their file reviewed in detail by a faculty member who presents their summary and key evidence to the rest of the committee: this person is known as the ‘Primary Reviewer.’ 

An Academic Advisor is a longitudinal coach who reviews a resident’s assessment portfolio in detail on a regular basis and engages in individualized Coaching Over Time conversations with the resident based on the assessment data to guide their learning and help them keep on track.  

The primary reviewer role may be assigned to an academic advisor or to a competence committee member.  Because Primary Reviewers and Academic Advisors both have to conduct an in-depth review of resident portfolio data, programs may have a single faculty member fulfilling both these roles, however, in those cases this member should recuse themselves from voting on summative decisions for their assigned advisees to avoid a conflict of interest (the Academic Advisors should retain a role as primarily advocates for their resident’s success and not also gatekeeper for promotion decisions).  

Each specialty program that has launched (or is within ~6 months of launching) in CBD has a Pathway to Competence document that is created by their specialty committee as part of the document suite provided by the Royal College.  This document collates the CanMEDS framework and maps out how it integrates into CBD components:  all CanMEDS roles, key competencies, enabling competencies are listed and mapped to appropriate stages of training and EPAs that assess those competencies.  This document can allow you to quickly see which CanMEDS competencies may require additional learning experiences and assessments since they are not all covered by dedicated EPAs. This will help with your curriculum mapping.

• Regularly re-educate and re-engage residents and encourage them to share strategies for improving completion rates – this message needs to be repeated regularly

• Remind faculty that doing EPA observations in the moment right after an encounter is less time consuming than doing an ITER later (encourage residents to ask for them then)

• Request a standing ~5 minute slot in regular meetings attended by your faculty (rounds, business meetings, etc) to present EPA observation tips/reminders or other CBD topics to keep CBD on faculty members’ radar

• Add to your existing newsletter a section highlighting top EPA observers

• Consider a rewards / acknowledgement system such as EPA ‘faculty and resident of the month’ for number of completed EPA observations; some programs even had success with friendly competition between mini-teams of supervisors/residents

• Program directors/leads may review quarterly EPA observation reports to offer targeted faculty development or reminders based on faculty completion rates

• Remind faculty that their EPA observation completion and other CBD roles can be recognized through their Annual Report process 

• Discuss with your department whether EPA observation completion numbers could be incorporated into calculation of clinical stipends