BACKGROUND
Competency-based medical education (CBME) has been increasingly adopted worldwide since the late 20th century as the standard framework for graduate medical education (GME) [
1]. Unlike basic medical education, GME is inherently workplace-based, making it difficult to teach and assess individual competencies in isolation. Consequently, the concept of entrustable professional activities (EPAs) was introduced. EPAs are defined as tasks that can be entrusted to trainees for unsupervised execution once they have achieved sufficient competence [
2]. This framework aims to integrate multiple competencies into comprehensive clinical tasks that can be directly observed and assessed in real-world practice [
2,
3]. Globally, emergency medicine (EM) training programs have been at the forefront of this transition. Canada implemented its national CBME model, “Competence by Design” (CBD), in 2018 [
4], whereas the United States has been transitioning from the Milestones Project toward an EPA-based framework [
5]. In Korea, medical schools follow international standards and undergo regular accreditation by the Korean Institute of Medical Education and Evaluation, which is recognized by the World Federation for Medical Education. In contrast, the residency training system has remained largely unchanged for decades. Despite rapid changes in the healthcare environment, systematic improvements in training outcomes, assessment methods, and accreditation standards for residency education have lagged behind. More recently, however, interest in CBME has increased within the Korean medical community. The internal medicine department at a major hospital and the Korean Neuropsychiatric Association were among the first groups to reform residency training programs based on CBME principles [
6,
7]. Subsequently, the Korean Academy of Medical Sciences, with support from the Ministry of Health and Welfare (MOHW) and the Korean Hospital Association, funded the “Research on the Development and Systematization of the Training Curriculum for Residency” project in 2020. This initiative led to the development of standardized curricula for specialties including family medicine, anesthesiology, and radiology [
8–
10].
THE KOREAN SOCIETY OF EMERGENCY MEDICINE CBME PROJECT
In alignment with this national initiative, the Korean Society of Emergency Medicine (KSEM) established a task force in 2021 to develop a CBME-based residency training model. The task force identified core competencies for EM specialists and developed an EM competency framework that included EPAs, assessment tools and guides, feedback strategies, and a programmatic assessment framework. In December 2021, KSEM established 10 EPAs corresponding to 47 subcompetencies, together with assessment tools and faculty guidelines (
Table 1). Following pilot implementation of workplace-based assessments (WBAs) for selected EPAs at a teaching hospital, the perceptions of both residents and supervisors were analyzed to refine the model [
11]. By March 2023, the working group had further developed feedback strategies, operational frameworks, and an electronic portfolio (e-portfolio) system [
12]. The task force has since evolved into the formal “Quality Improvement in Residency Training Committee,” which continues to conduct research and educational activities to facilitate implementation of the CBME model.
THE NEED FOR CONSENSUS ON REQUIRED TRAINING EXPERIENCES
Of the 10 EPAs that were developed, EPAs 5–10 primarily involve nontechnical skills that are cultivated longitudinally throughout residency training alongside routine clinical practice. In contrast, EPAs 1–4 are directly grounded in clinical care. Valid assessment of these EPAs requires consensus regarding the specific clinical experience required during residency training. Given the variability in training environments across hospitals in Korea, the standardization of core clinical experience is essential to ensure adequate and consistent training. To address these needs, the working group conducted modified Delphi surveys to define the required training experience (RTE) for competency-based training and to identify the essential emergency procedures required for EPA 4.
THE DELPHI CONSENSUS PROCESS
Consensus regarding RTE and essential emergency procedures was achieved through two rounds of modified Delphi surveys. The committee consisted of 30 EM specialists from various training hospitals; after excluding the 2 study authors, surveys were distributed to 28 expert panel members. The numbers of respondents and survey periods are summarized in
Table 2 and
Suppl. 1. Panelists were asked to rate the validity of the listed RTE and essential procedures using a 5-point Likert scale (1, strongly disagree; 5, strongly agree). Consensus was evaluated using the content validity ratio (CVR), as described by Lawshe [
13]. Agreement for each item was defined as a rating of 4 (agree) or 5 (strongly agree). Notably, no responses were missing for any item across either Delphi round. Therefore, the CVR for each item was calculated on the basis of complete item-level responses from all respondents in each round. Accordingly, the minimum acceptable CVR value was adjusted according to the number of experts who participated in each Delphi round without missing responses (
Table 2) [
13]. The formula used to calculate the CVR was as follows:
DEFINING RTE
For the RTE, the authors focused on quantifiable and minimal training elements, including external rotations (e.g., other departments and emergency medical services), specific areas within the emergency department (ED), and completion of educational courses or workshops. General adult EM experience was excluded because it is intrinsic to all residency programs. Initial items were derived from Canadian curricula (Canadian Association of Emergency Physicians, McMaster University, and University of Ottawa) and existing KSEM residency regulations [
4,
14–
16].
The Delphi process resulted in consensus regarding the following minimal educational elements for EM residency training (
Suppl. 2,
Table 3):
(1) EPA 1 (Triage): A dedicated rotation period was not considered necessary; however, participation in a patient-acuity assessment program, such as the Korean Triage and Acuity Scale provider course, was considered essential.
(2) EPA 2 (Patient care): The panel agreed that clinical rotations in a pediatric ED or pediatric ward, trauma center, trauma intensive care unit (ICU), or trauma surgery department were necessary. Although consensus was not reached regarding the specific stage of residency training (junior vs. senior), there was strong agreement regarding the need for educational programs in pediatric care, thoracic and abdominal radiology, adult echocardiography, and electrocardiography.
(3) EPA 3 (Resuscitation and critical care): Clinical rotations in a resuscitation unit, critical care area, or emergency ICU were considered essential, particularly during the senior residency stage. In addition, educational programs addressing adult and pediatric resuscitation/critical care and trauma management were strongly recommended.
(4) EPA 4 (Procedures): Consensus supported the need for educational programs focused on airway management, pediatric procedures, and wound management.
DEFINING ESSENTIAL EMERGENCY PROCEDURES
Although EPA 4 encompasses procedural skills, the RTE Delphi survey addressed only educational programs. To implement direct observation of procedural skills, it was necessary to define the specific procedures that required evaluation. The initial list of 50 procedures was compiled from multiple sources, including the current resident e-portfolio contents (33 items), KSEM assessment requirements (10 items), credit-granting educational courses (4 domains), MOHW-recommended procedures (23 items), and the 14 procedures originally included in EPA 4 (
Suppl. 3) [
16–
19]. During the Delphi process, items with low CVR values were removed, and panel suggestions were incorporated. These modifications included separating pediatric procedural sedation from adult procedural sedation and distinguishing rapid sequence intubation from standard endotracheal intubation. The final consensus established 26 essential emergency procedures, including critical skills such as mechanical ventilation management, cardiopulmonary resuscitation, multiple forms of catheter insertion, and point-of-care ultrasonography (
Table 4).
DISCUSSION AND IMPLICATIONS
These two Delphi studies established the RTE and essential emergency procedures needed to operationalize the KSEM CBME model. The resulting output represents a minimal set of standard educational elements for KSEM residency training. The findings also reveal meaningful contrasts with international models. The Canadian CBD model is structured according to “stages of training” rather than residency year, with training experience and procedures aligned with these stages [
4]. It also distinguishes between “required” experience and “recommended” experience. In the present study, advanced areas such as the helicopter emergency medical services and hyperbaric oxygen therapy were considered valuable but were not classified as universally essential because training hospitals in Korea differ substantially in their current capabilities. These areas could therefore be considered as recommended experience in the future. Similarly, the European Society for Emergency Medicine’s European Training Requirements includes prehospital settings as a necessary training experience, reflecting differences in the scope of practice of European EM physicians [
20]. Although European requirements do not explicitly specify every required clinical context for individual residents, they recommend courses similar to those identified in the present study, such as Advanced Life Support and Major Trauma Management. Because RTE and essential emergency procedures are inherently linked to the local healthcare system and the scope of practice of its specialists, these Delphi surveys are significant in that they produced a curriculum consensus tailored to the Korean context.
Several limitations should be considered. The expert panel was drawn from a committee rather than from a probabilistic national sample, and response rates declined modestly across rounds. Therefore, the recommended sets should be viewed as a minimal set of standard educational elements that will require refinement through iterative implementation, stakeholder feedback, particularly from residents, and periodic re-evaluation as training environments and national policies evolve. Next steps include embedding these RTE and essential procedures within the KSEM programmatic assessment framework, including EPA-aligned WBAs, e-portfolio use, and competence committee review, and evaluating their downstream effects on resident learning, entrustment decisions, and patient safety outcomes.
Conclusions
These two modified Delphi consensus studies established a consensus-derived minimal set of RTE for EPAs 1–4 and a core list of 26 essential emergency procedures for EPA 4. This initiative represents a foundational step toward practical implementation of the KSEM CBME model. By defining clinically feasible RTE and essential emergency procedures through expert consensus, KSEM has created a blueprint for standardization. Moving forward, sustained discussion and coordinated effort will be needed to refine the standardized curriculum and assessment system based on these findings and to ensure successful implementation across residency programs.
NOTES
-
Ethics statement
Prior to the survey, participants were fully informed about the study’s purpose, the voluntary nature of their participation, and the confidentiality of their responses. Completion and submission of the questionnaire was considered as implied consent to participate.
-
Author contributions
Conceptualization: all authors; Data curation: all authors; Investigation: all authors; Methodology: all authors; Writing–original draft: all authors; Writing–review & editing: all authors. All authors read and approved the final manuscript.
-
Conflicts of interest
The authors have no conflicts of interest to declare.
-
Funding
The authors received no financial support for this study.
-
Data availability
Data analyzed in this study are available from the corresponding author upon reasonable request.
Supplementary materials
Suppl. 2.
Items for round 1 Delphi survey to evaluate the validity of minimal set of educational elements for emergency medicine residency training.
ceem-26-003-Suppl-2.pdf
Suppl. 3.
Items for round 1 Delphi survey to evaluate the validity of the essential emergency procedures.
ceem-26-003-Suppl-3.pdf
Table 1.EPAs for Korean emergency physicians
Table 1.
|
Detail |
|
EPA 1 |
Triage emergency patients according to their clinical situation and severity |
|
EPA 2 |
Perform history taking and physical examination and formulate a differential diagnosis and care plan |
|
EPA 3 |
Perform resuscitation and critical care for critically ill patients |
|
EPA 4 |
Perform emergency procedures for diagnosis and treatment |
|
EPA 5 |
Communicate the emergency care process and results with patients, guardians, and healthcare teams |
|
EPA 6 |
Share information and collaborate with other healthcare providers for continuity of care |
|
EPA 7 |
Provide clinical education related to emergency care |
|
EPA 8 |
Conduct evidence-based practice drawing upon the latest scientific evidence |
|
EPA 9 |
Manage legal and ethical situations frequently encountered in emergency care |
|
EPA 10 |
Participate in patient safety and quality improvement activities related to emergency care |
Table 2.Number of experts and survey periods in each round of the Delphi survey
Table 2.
|
Required training experience |
Essential emergency procedures |
|
No. of invited experts |
28 |
28 |
|
Round 1 |
|
|
|
No. of respondents |
20 |
20 |
|
Survey period |
Jul 17–Aug 2, 2024 |
Oct 22–Dec 17, 2024 |
|
Minimum value of CVR |
0.42 |
0.42 |
|
Round 2 |
|
|
|
No. of respondents |
17 |
18 |
|
Survey period |
Sep 2–23, 2024 |
Jan 3–17, 2025 |
|
Minimum value of CVR |
0.49 |
0.49 |
Table 3.The minimal set of educational elements required for emergency medicine residency training, as derived from the Delphi survey
Table 3.
|
Educational element |
Agreement range (%) |
Agreement ratinga)
|
CVR |
|
EPA 1. Triage emergency patients according to their clinical situation and severity |
|
|
|
|
Education program on emergency patient severity assessment |
70.5 |
4 (4–4.5) |
0.53 |
|
EPA 2. Perform history taking and physical examination and formulate differential diagnosis and care plan |
|
|
|
|
Clinical rotation at a pediatric ED or pediatric ward |
94.1 |
4.5 (4–5) |
0.76 |
|
Clinical rotation at a trauma center, trauma ICU, or OR |
82.3 |
4 (4–5) |
0.65 |
|
Education program on pediatric care |
76.4 |
4 (4–5) |
0.53 |
|
Education program on thoracic radiology |
82.3 |
4 (4–5) |
0.65 |
|
Education program on abdominal radiology |
94.1 |
4 (4–5) |
0.65 |
|
Education program on adult echocardiography |
94.1 |
5 (4–5) |
0.76 |
|
Education program on electrocardiography |
94.1 |
5 (4–5) |
0.76 |
|
EPA 3. Perform resuscitation and critical care for critically ill patients |
|
|
|
|
Clinical rotation in a resuscitation/critical care area within the ED or emergency ICU |
94.1 |
5 (4–5) |
0.76 |
|
Clinical rotation during senior resident stage |
94.1 |
5 (4–5) |
0.88 |
|
Education program on adult resuscitation/critical care |
94.1 |
5 (4–5) |
0.76 |
|
Education program on pediatric resuscitation/critical care |
76.4 |
5 (4–5) |
0.53 |
|
Education program on trauma resuscitation/critical care |
76.4 |
5 (4–5) |
0.53 |
|
EPA 4. Perform emergency procedures for diagnosis and treatment |
|
|
|
|
Education program on airway management |
94.1 |
5 (5–5) |
0.88 |
|
Education program on pediatric procedures |
88.2 |
4 (4–5) |
0.76 |
|
Education program on wound management |
82.3 |
5 (4–5) |
0.65 |
Table 4.Essential procedures adopted after the Delphi survey
Table 4.
|
Essential procedure |
Agreement range (%) |
Agreement ratinga)
|
CVR |
|
Simple wound suture |
85.0 |
5 (5–5) |
0.70 |
|
Burn management |
95.0 |
5 (4–5) |
0.90 |
|
Procedural sedation |
|
|
|
|
Adult |
100 |
5 (5–5) |
1.00 |
|
Pediatric |
94.4 |
5 (4.25–5) |
0.89 |
|
Local anesthesia |
88.9 |
4 (4–5) |
0.78 |
|
Endotracheal intubation |
100 |
5 (5–5) |
1.00 |
|
Rapid sequence intubation |
100 |
5 (5–5) |
1.00 |
|
Mechanical ventilation management |
100 |
5 (5–5) |
1.00 |
|
Noninvasive positive pressure ventilation |
80.0 |
4.5 (4–5) |
0.60 |
|
Cricothyrotomy (surgical, needle) |
95.0 |
5 (5–5) |
0.90 |
|
Thoracentesis |
100 |
5 (4–5) |
1.00 |
|
Closed thoracostomy |
100 |
5 (5–5) |
1.00 |
|
Cardiopulmonary resuscitation and defibrillation |
100 |
5 (5–5) |
1.00 |
|
Transcutaneous pacing |
100 |
5 (4.75–5) |
1.00 |
|
Pericardiocentesis |
80.0 |
4.5 (4–5) |
0.60 |
|
Emergency transthoracic echocardiography |
95.0 |
5 (4–5) |
0.90 |
|
Arterial catheter insertion |
95.0 |
5 (4.75–5) |
0.90 |
|
Central venous catheter insertion |
100 |
5 (5–5) |
1.00 |
|
Intraosseous injection |
90.0 |
5 (4–5) |
0.80 |
|
Fracture immobilization |
85.0 |
5 (4–5) |
0.70 |
|
Joint dislocation reduction |
95.0 |
5 (4–5) |
0.90 |
|
Emergency abdominal ultrasonography |
80.0 |
5 (4.75–5) |
0.80 |
|
Focused assessment with sonography for trauma |
100 |
5 (5–5) |
1.00 |
|
Paracentesis |
80.0 |
4.5 (4–5) |
0.60 |
|
Esophageal variceal hemostatic tube insertion |
90.0 |
4.5 (4–5) |
0.80 |
|
Anterior epistaxis nasal packing |
72.2 |
4.5 (3.75–5) |
0.50 |
REFERENCES
- 1. Ten Cate O. Competency-based postgraduate medical education: past, present and future. GMS J Med Educ 2017;34:Doc69.
- 2. Ten Cate O. Entrustability of professional activities and competency-based training. Med Educ 2005;39:1176-7.
- 3. Pangaro L, ten Cate O. Frameworks for learner assessment in medicine: AMEE guide No. 78. Med Teach 2013;35:e1197-210.
- 4. Sherbino J, Bandiera G, Doyle K, et al. The competency-based medical education evolution of Canadian emergency medicine specialist training. CJEM 2020;22:95-102.
- 5. Society for Academic Emergency Medicine (SAEM). CBME in emergency medicine [Internet]. SAEM; [cited 2025 Dec 24] Available from: https://www.saem.org/education/saem-online-academic-resources/saem-competency-based-medical-education-primer/cbme-in-emergency-medicine
- 6. Yoon CH, Myung SJ, Park WB. Implementing competency-based medical education in internal medicine residency training program: the process and impact on residents' satisfaction. J Korean Med Sci 2019;34:e201.
- 7. Kyeon YG, Kim J, Shim S, Sohn I, Seo J, Lee K. Competency-based psychiatry residency training program development in South Korea. Korean Med Educ Rev 2018;20:51-9.
- 8. Park YC. Past, present, and future of family medicine residency training: transition to the competency-based training. Korean J Fam Pract 2022;12:298-303.
- 9. Kim KW, Choe WJ, Kim JH. Application of competency-based education in the Korean anesthesiology residency program and survey analysis. Korean J Anesthesiol 2023;76:135-42.
- 10. Kim YM, Choi MH, Lee JH, et al. Development of entrustable professional activity, core competencies, and guidelines in the 2021 Radiology Competency Education Project. Taehan Yongsang Uihakhoe Chi 2022;83:284-92.
- 11. Park KH, Oh YT, Park JO. Entrustable professional activity-aligned workplace-based assessments in the emergency department: perceptions of emergency medicine residents and assessors. Clin Exp Emerg Med 2025;12:391-9.
- 12. The Korean Society of Emergency Medicine (KSEM). [Research on the development and systematization of the training curriculum for residency]. KSEM; 2023 [cited 2025 Dec 24]. Available from: https://www.emergency.or.kr/bbs/data/2936
- 13. Lawshe CH. A quantitative approach to content validity. Personal Psychol 1975;28:563-75.
- 14. Department of Medicine, Faculty of Health Sciences, McMaster University. Emergency medicine: residency program: residency program at McMaster [Internet]. McMaster University; [cited 2025 Dec 24]. Available from: https://medicine.healthsci.mcmaster.ca/divisions/emergency-medicine/emergency-medicine-education/residency-program/
- 15. Department of Emergency Medicine, Faculty of Medicine, University of Ottawa. RCPSC (Royal College of Physicians and Surgeons of Canada) [Internet]. University of Ottawa; [cited 2025 Dec 24]. Available from: https://www.uottawa.ca/faculty-medicine/emergency/residency-education/residency-programs/rcpsc
- 16. The Korean Society of Emergency Medicine (KSEM). [The assessment of residency training conditions] [Internet]. KSEM; 2024 [cited 2025 Dec 24]. Available from: https://emergency.or.kr/bbs/notice/2702
- 17. The Korean Society of Emergency Medicine (KSEM). [Resident’s e-portfolio] [Internet]. KSEM; [cited 2025 Dec 24]. Available from: https://record.emergency.or.kr/
- 18. The Korean Society of Emergency Medicine (KSEM). [List of credits by curriculum for emergency medicine residency program 2024] [Internet]. KSEM; 2024 [cited 2025 Dec 24]. Available from: https://www.emergency.or.kr/bbs/edu-grade/2661
- 19. Ministry of Health and Welfare. [Annual curriculum for residency training] [Internet]. Korean Law Information Center, Ministry of Government Legislation; 2025 [cited 2025 Dec 24]. Available from: https://www.law.go.kr/행정규칙/전공의의연차별수련교과과정/(2025-18,20250123)
- 20. European Society for Emergency Medicine (EUSEM). European curriculum of emergency medicine [Internet]. EUSEM; [cited 2025 Dec 24]. Available from: https://eusem.org/education/learning-resources/european-curriculum-of-emergency-medicine/
