The study facility was a urban teaching hospital with an annual ED census of 65,000 visits per year. The study was conducted in 2008 when the facility used a 64-channel multi-detector CT for cardiac imaging. At the beginning of the study, the cardiac CT imaging had been actively used for more than a year in the study ED for evaluation of patients with acute chest pain.

A heterogeneous group of emergency medicine (EM) residents including two postgraduate year (PGY)-2 residents, two PGY-3 residents, and two PGY-4 residents participated in this study. As cardiac CT imaging had been actively used in the study ED for more than a year, they had basic knowledge about how to read a cardiac CT image. At the beginning of the study, they had undergone a one-hour training session (didactic, image review) provided by a radiologist specialized in cardiovascular imaging. The participating EPs were not blinded to the clinical information of enrolled patients which included initial examination findings, ECG and biomarker levels.

Patients presenting with acute chest pain who were determined to require a cardiac CT imaging after initial evaluation that included both ECG and cardiac biomarker tests were prospectively enrolled by the participating EPs unless the following exclusion criteria were met (1) age less than 18 years, (2) pregnancy, (3) contraindication to iodinated contrast or β-blocking agents, (4) atrial fibrillation or markedly irregular rhythm, (5) renal insufficiency (creatinine >1.4 mg/dL), (6) high risk features or previous coronary artery bypass grafting or recent (within 6 months) percutaneous coronary intervention. High risk features included (1) ST-segment elevation, new onset LBBB or pathologic Q-wave in more than two consecutive leads, (2) positive cardiac biomarkers, (3) typical chest pain with ischemic ST-T change (ST-segment depression >1 mm or T-wave inversion >3 mm in more than two consecutive leads that is not proven to be old), (4) any clinical feature of decompensated heart failure, (5) recurrent ventricular arrhythmia or high-degree atrioventricular block, (6) ongoing or recurrent angina at rest or minimal effort, (7) history of recent revascularization within 6 months, and (8) previously documented high-degree stenosis without history of revascularization. Patients who developed any of the high risk features after cardiac CT imaging were not excluded. Patients with suboptimal image result were also not excluded.

All enrolled patients were provided with the usual standard care which was consistent with 2005 American Heart Association guidelines for acute coronary syndrome [8]. For heart rate control, 100 mg of metoprolol was administered orally to patients with resting heart rate above 70 bpm. If further rate control was indicated, intravenous esmolol bolus injection was administered just before the test. Participating EPs read cardiac CT images right after the imaging process. Relevant clinical information and the CT findings observed by the participating physicians were recorded on a study registry form. Although the participating physicians were allowed to make clinical decisions based on their own readings, it was recommended that they wait for a formal report from the radiology department before making any new significant decision. Patients without significant coronary and/or non-coronary lesions were discharged from the ED if no other conflicting clinical findings existed. Upon discharge, a follow up visit to cardiology was arranged.

Cardiac CT findings reported by EPs included the presence and severity of coronary artery stenosis and plaque calcification and other miscellaneous/extra-cardiac findings. The evaluation of stenosis and plaque calcification was done separately for each anatomic unit of coronary arteries (left main artery [LM], left anterior descending artery [LAD], left circumflex artery [LCx], right coronary artery [RCA]) using visual estimation. If there were multiple coronary stenosis/plaques in an arterial unit, the most significant lesion in the unit was sought and analyzed. The severity of stenosis was classified as one of three levels: (1) no stenosis, (2) clinically non-significant stenosis (stenosis less than 50%), and (3) significant stenosis (50% or more). The severity of plaque calcification was measured for the same lesion for plaque characterization that might provide additional prognostic information [9]. The severity of plaque calcification had four levels which were (1) no plaque, (2) calcified plaque (calcified area occupies 50% or more), (3) mixed plaque (less than 50%), and (4) non-calcified plaque. Further distinction between fibrous and lipid-rich plaques was not done because of their wide overlap [10-12]. Radiologists’ interpretation of the same cardiac CT images was reformatted to make direct comparison with the EPs’ interpretation. To assess the longterm (1 year) progress of patients discharged from EDs, the occurrences of major adverse cardiac events (MACE) within a year after discharge from the ED was assessed by patient chart review if there were regular follow-up visits during the first year. Otherwise, a structured telephone interview was performed by an assistant researcher.

The interrater agreement between EPs and radiologists were assessed with Cohen’s kappa and Gwet’s AC1. The kappa coefficient is commonly used to evaluate the interrater agreement of categorical variables between two observers. A kappa value of 1 represents a perfect agreement, whereas a kappa value of 0 represents agreement equal to that of chance alone. A negative value implies that agreement is worse than chance alone. Landis and Koch [13] characterized values <0 as indicating no agreement and 0–0.20 as slight, 0.21–0.40 as fair, 0.41–0.60 as moderate, 0.61– 0.80 as substantial, and 0.81–1 as almost perfect agreement. But the reliability of kappa statistics has been questioned as it is affected by the rater’s classification and risk prevalence of the subjects [14-16]. Gwet [17,18] introduced an alternative “AC1” which is more stable agreement coefficient. The AC1 statistic can be interpreted similarly [17]. We calculated and reported both of the coefficients in addition to the proportionate agreement. The data were analyzed using IBM SPSS ver. 19.0 (IBM Co., Armonk, NY, USA) and Agreestat (Advanced Analytics, Gaithersburg, MD, USA; http://www.agreestat.com).