The simulation lab is a clean, but limited-dimensional representation of emergency medicine (EM) clinical practice. EM educators inevitably select which elements of the emergency department’s reality can be replicated and reproduced for learners. Simulation attempts to recreate “clinical chaos” in a manner that is sufficiently structured to enable deliberate practice. Yet, the necessary design constraints and educational trade-offs mean that the dynamic, unpredictable nature of emergency care, the ever-present distractions, and the semantic richness of real clinical environments cannot be fully preserved. The ordered state of the simulation lab does not fully represent the disordered state of the emergency department. A metaphor can be found in the second law of thermodynamics, where entropy unravels sophisticated design, spreading it into distributed, less structured forms.
The challenges of simulation educational design are not unique. If anything, they reflect a broader truth about educational innovation: an idea, elegant in the abstract, is inevitably transformed as it is translated into practice. The pipeline from design to implementation becomes vulnerable when complex concepts are simplified, reinterpreted, or reshaped by local context, time pressures, competing priorities, resource constraints, and human systems [
1]. One of the clearest examples of this implementation challenge in EM education is found in the contrast between the intended design and lived experience of competency-based medical education (CBME), particularly in Canada.
CBME AND CANADIAN EM
In 2018, all Canadian EM specialist training migrated to a CBME model, as part of the ongoing transformation of a national postgraduate medical education system [
2]. CBME is a form of outcomes-based education, organized around the abilities expected of graduates, tailoring learning and assessment towards individual acquisition of these abilities [
3]. The sophisticated and organized design of the new education model was three years in the making, informed by theory, pilot research, and thousands of hours of expert debate and contributions. Yet, within the Canadian context, defining and articulating the theories and processes of CBME is a disordered conversation among education researchers, curriculum designers, administrators and front-line teachers [
4]. Conflicting learning theory and different definitions are the hallmark of the conversation among leaders engaged in the delivery of CBME. Entropy is at play. One explanation, perhaps, is that CBME is misinterpreted as a single intervention. Unlike the discrete and bounded articulation of educational innovations, such as the multiple mini-interview or team-based learning, CBME is better understood as an educational bundle. It is analogous to the clinical care bundles routinely delivered in EM (e.g., sepsis management, stroke pathways, etc.). In such bundles, multiple coordinated components are meant to work together to achieve better outcomes than any one element alone.
DEFINING CBME
The International CBME Collaborators articulated five core components: an outcomes (competency) framework, sequenced progression, tailored learning experiences, competency-focused instruction, and programmatic assessment [
5]. Yet in practice, CBME has often degraded to one or two visible artifacts. In many settings, CBME has been represented primarily as programmatic assessment, or more narrowly still as entrustable professional activities (EPAs). Over time, much of the original complexity and organization disperses, such that CBME increasingly becomes synonymous with “EPA completion.” Implementation entropy has created a reframing of CBME as EPA. This is problematic across several dimensions.
THE CHALLENGES OF EPAs
First, within the Canadian system, EPAs have increasingly become a single instrument of programmatic assessment, something that was never the intent. The shift toward direct observation tools is, in many respects, a welcome departure from prior systems that leaned heavily on indirect and surrogate markers of performance (such as end-of-unit written reports or summative written knowledge tests). However, direct observation is neither universally efficient nor effective, and its utility depends on what is being assessed. For example, written assessments remain efficient and effective tests of knowledge—a foundational element necessary for clinical practice. Conversely, observations of context-specific competencies can produce questionable data when the assessor is poorly positioned to observe the behavior in question. For example, patient advocacy may be more directly observed by patients, families, and interprofessional colleagues than by attending physicians [
6], yet the data stream in many EPA systems implicitly privileges the physician observer.
Second, in the Canadian system, EPAs are broad, spanning multiple CanMEDS Roles. While they are intentionally focused on observable clinical behaviors, this design can obscure the detailed competencies within the CanMEDS framework that inform complex physician performance. A downstream risk is that an emerging generation of teachers and learners, trained within an EPA-dominant ecosystem, have diminishing familiarity with the nuanced educational theory that has shaped the last two decades of practice in medical education [
7]. When the system’s daily currency becomes “EPA achieved,” the deeper conceptual scaffolding of the CanMEDS framework may gradually fade from understanding. What does the specialty of EM become if reduced to the five EPAs endorsed by the International Federation for Emergency Medicine [
8]: “patient resuscitation,” “adult and/or pediatric non-resuscitation emergency care,” “managing multiple emergency patients,” “maintain quality emergency care,” and “teaching emergency staff”?
Third, the original hope for EPAs included fostering a growth mindset through low-stake, coaching-oriented assessment moments. With implementation the opposite has become apparent. Learners frequently report a performance mindset when observed by faculty during an EPA assessment. Rather than actively seeking coaching that targets their developmental needs, residents try to manipulate the system so that EPAs are assessed only when the individual is confident of both success and only positive, validating feedback [
9]. Instead of functioning as one data point within a rich program of assessment, the EPA can become a micro-summative event, an all-or-none threshold phenomenon that encourages strategic performance rather than authentic learning.
Fourth, scope creep has occurred as EPAs have migrated into the simulation lab. This is, on its face, a non sequitur: an activity assessed in an intentionally artificial context does not require the same entrustment logic as emergency department–based clinical practice. Yet, simulation is increasingly the environment for EPA completion to ensure that the necessary completion thresholds are achieved [
10]. The ideal of authentic, work-based, direct observation assessment has experienced implementation entropy. As a result, simulation-based assessments become the surrogate when the original CBME design is too clunky to efficiently or effectively integrate. The drift in EPA adoption is also visible beyond EM residency education into EM education fellowships [
11]. Is there appropriate functional task alignment in the adoption of EPAs to assess domains such as critical appraisal or scholarship? A one-size-fits-all assessment approach was never the intent of programmatic assessment; it is a violation of the principles of CBME. Programmatic assessment endorses a variety of instruments, mapped to the necessary assessment goals, to ensure reliable sampling across the entire outcomes framework to produce a summative judgment. But, implementation entropy has turned CBME into programmatic assessment into EPAs.
Finally, the promise of EPAs was reinforced by enthusiasm for entrustment scales. Following earlier attempts to move assessment anchors from generic labels to behaviorally informed descriptors [
12], there was hope that centuries of apprenticeship traditions in medicine could be captured in a five-point scale ranging from “I had to do it” to “I didn’t need to be there” (taken from the perspective of the assessor). However, there is very limited validity evidence supporting the interpretation and use of entrustment scales in many contexts, including EM [
13]. The entrustment scale does not seem to be working as designed.
The challenges of EPAs are not isolated to Canada. While the first two points may be unique to Canada, it is easy to see how these challenges could translate to other contexts. However, the final three points speak to EPAs in general, and are a concern of CBME implementation globally. As educational jurisdictions move towards adopting CBME, the challenges above should be scrutinized.
THE PROMISE OF EPAs
Of course, polarization of the value of EPAs should be resisted. Important benefits have emerged during the Canadian transition to CBME. There has been a notable increase in the amount and quality of feedback conversations as a function of EPA implementation [
14,
15]. EPAs have also unmasked structural problems, including gender bias, within the education system, providing data to advocate for curricular change [
16]. More broadly, programmatic assessment has catalyzed improvements in assessment decisions and application of learning analytics to track the developmental progression of learners [
17]. These gains matter, and they should be acknowledged as part of a balanced appraisal. CBME did not arise in a vacuum. It was a response to an education system that required renewed attention to learners, patients, and society [
18]. Despite implementation entropy, improvements to postgraduate medical education have been realized. The current Canadian EM residency system has advanced past where it was situated before 2018. However, it is not as far as the innovators had designed.
INNOVATION IS NOT ENOUGH: THE IMPORTANCE OF IMPLEMENTATION
The larger lesson may be this: as an EM education community we have a natural proclivity for innovation and creativity. However, we must be more deliberate about the distance between design and implementation. Educational ideas are not implemented in isolation from context. They require robust, practical, sequenced, flexible execution strategies. They require realistic resourcing, recognizing that additional, new processes consume faculty time and may add administrative costs. They require effective communication between all educational and clinical stakeholders. Finally, they require careful program evaluation at the right scale, where pilot projects inform the adaptation of larger implementation efforts. In the absence of this, entropy takes over.
More than two decades after EPAs were introduced to Canadian educators [
19] and nearly one decade after the transformation of the Canadian EM residency training system to a CBME model, one wonders whether CBME implementation in Canada will be a case study of implementation vulnerability for future researchers. It may stand as a complex and compelling design that was not consistently paired with sufficiently robust and sufficiently practical implementation and evaluation strategies to address the entropy within the system.
NOTES
-
Conflicts of interest
The author has no conflicts of interest to declare.
-
Funding
The author received no financial support for this study.
-
Data availability
Data sharing is not applicable as no new data were created or analyzed in this study.
REFERENCES
- 1. Price DW, Wagner DP, Krane NK, et al. What are the implications of implementation science for medical education? Med Educ Online 2015;20:27003.
- 2. Sherbino J, Bandiera G, Doyle K, et al. The competency-based medical education evolution of Canadian emergency medicine specialist training. CJEM 2020;22:95-102.
- 3. Frank JR, Snell LS, Cate OT, et al. Competency-based medical education: theory to practice. Med Teach 2010;32:638-45.
- 4. Sherbino J, Regehr G, Dore K, Ginsburg S. Tensions in describing competency-based medical education: a study of Canadian key opinion leaders. Adv Health Sci Educ Theory Pract 2021;26:1277-89.
- 5. Van Melle E, Frank JR, Holmboe ES, Dagnone D, Stockley D, Sherbino J. A core components framework for evaluating implementation of competency-based medical education programs. Acad Med 2019;94:1002-9.
- 6. McDonald M, Lavelle C, Wen M, Sherbino J, Hulme J. The state of health advocacy training in postgraduate medical education: a scoping review. Med Educ 2019;53:1209-20.
- 7. LoGiudice AB, Sibbald M, Monteiro S, et al. Intrinsic or invisible? An audit of CanMEDS Roles in entrustable professional activities. Acad Med 2022;97:1213-8.
- 8. Teng D, Venkataraman A, Singer A, et al. Entrustable professional activities for emergency medicine specialists. Int J Emerg Med 2025;18:248.
- 9. LaDonna KA, Hatala R, Lingard L, Voyer S, Watling C. Staging a performance: learners' perceptions about direct observation during residency. Med Educ 2017;51:498-510.
- 10. Sahi N, Humphrey-Murto S, Brennan EE, O'Brien M, Hall AK. Current use of simulation for EPA assessment in emergency medicine. CJEM 2024;26:179-87.
- 11. Villa S, Caretta-Weyer H, Yarris LM, et al. Development of entrustable professional activities for emergency medicine medical education fellowships: a modified Delphi study. AEM Educ Train 2024;8:e10944.
- 12. Crossley J, Johnson G, Booth J, Wade W. Good questions, good answers: construct alignment improves the performance of workplace-based assessment scales. Med Educ 2011;45:560-9.
- 13. Spencer M, Sherbino J, Hatala R. Examining the validity argument for the Ottawa Surgical Competency Operating Room Evaluation (OSCORE): a systematic review and narrative synthesis. Adv Health Sci Educ Theory Pract 2022;27:659-89.
- 14. Martin L, Sibbald M, Brandt Vegas D, Russell D, Govaerts M. The impact of entrustment assessments on feedback and learning: trainee perspectives. Med Educ 2020;54:328-36.
- 15. Yilmaz Y, Jurado Nunez A, Ariaeinejad A, Lee M, Sherbino J, Chan TM. Harnessing natural language processing to support decisions around workplace-based assessment: machine learning study of competency-based medical education. JMIR Med Educ 2022;8:e30537.
- 16. Dayal A, O'Connor DM, Qadri U, Arora VM. Comparison of male vs female resident milestone evaluations by faculty during emergency medicine residency training. JAMA Intern Med 2017;177:651-7.
- 17. Frank JR, Karpinski J, Sherbino J, et al. Competence by design: a transformational national model of time-variable competency-based postgraduate medical education. Perspect Med Educ 2024;13:201-23.
- 18. Holmboe ES, Sherbino J, Englander R, Snell L, Frank JR. A call to action: the controversy of and rationale for competency-based medical education. Med Teach 2017;39:574-81.
- 19. ten Cate O. Entrustability of professional activities and competency-based training. Med Educ 2005;39:1176-7.
