Objective
Workplace-based assessment (WBA) plays a crucial role in assessing entrustable professional activities (EPAs) in the competency-based medical education era. This pilot study explored the perceptions of residents and assessors of two WBAs for three Korean Society of Emergency Medicine EPAs.
Methods
Eight emergency medicine (EM) residents underwent WBAs, with mini-clinical evaluation exercises (mini-CEX) conducted by nine EM faculty members and multisource feedback (MSF) provided by two internal medicine faculty members and four emergency room nurses, for a total of 69 assessments. We conducted an anonymous online survey to gather feedback on experiences, perceptions, and recommendations for improving WBA, such as mini-CEX and MSF, with responses scored on a 5-point Likert scale.
Results
Of the 23 initial participants, 15 (65.2%) responded, including 5 residents and 10 assessors. EM faculty viewed mini-CEX favorably, noting its strong integration of supervision and effectiveness in assessing resident performance. EM residents reported comfort issues during assessments, preferring immediate feedback and multiple assessors. MSF was generally perceived positively but showed discrepancies in the utilities of rating scales and feedback types, indicating potential areas for improvement.
Conclusion
Two WBAs for three Korean Society of Emergency Medicine EPAs were found to be feasible and acceptable in the context of Korean EM residency training. However, perceptions varied between assessors and residents, necessitating clear communication about WBA objectives and processes. Our findings are useful for shaping future EPA-based training programs, balancing traditional and WBA methods, and enhancing feedback quality.

Objective
This study aimed to develop and validate MEDIVAL (Medical Documentation Validation), a progressive chain-of-thought (CoT) evaluation framework for automated assessment of large language model (LLM)-generated emergency department documentation, designed to align with expert clinical judgment in acute care settings. Methods We designed a three-tier evaluation framework incorporating persona-based, error-enhanced, and insight-integrated strategies. The framework was tested across four LLMs (GPT-4o, GPT-4.1, Claude-3.5, Claude-3.7) on 33 emergency department records reviewed by four expert emergency physicians. Each model applied the three CoT strategies across five criteria: appropriateness, accuracy, structure/format, conciseness, and clinical validity. Model outputs were compared with expert ratings using Spearman correlation coefficients. Differences were analyzed with the Friedman test and Wilcoxon signed rank test with Bonferroni correction. Reproducibility was assessed through intraclass correlation coefficient (ICC) analysis. Results All models demonstrated stronger alignment with expert ratings as CoT complexity increased, with Claude-3.7 (r=0.712, P<0.001) and GPT-4o (r=0.702, P<0.001) showing the highest correlations under the insight-integrated strategy. GPT-4.1 showed the greatest relative improvement (43.3% increase, r=0.457 to r=0.655, P<0.001). Significant overall differences were observed across strategies (χ2 (2)=48.39, P<0.001), though the error-enhanced and insight-integrated approaches differed only modestly yet significantly (P=0.002). High reproducibility was confirmed (ICC >0.919), with Claude-3.5 achieving the most consistent results (ICC, 0.997–0.998). Conclusion MEDIVAL demonstrates that progressive CoT strategies systematically improve automated evaluation of emergency department documentation while maintaining excellent reproducibility. This framework offers a viable prescreening tool to reduce expert workload and support reliable artificial intelligence integration into emergency medicine workflows.