INTRODUCTION
Stroke is a leading cause of disability and death worldwide and is clinically described as a neurological deficit resulting from an infarction in the central nervous system (brain, spinal cord, and retinal cell death) that was caused by ischemia or hemorrhage (ischemic or hemorrhagic stroke, respectively) [1]. Approximately 795,000 individuals in the United States experience a stroke each year; of these, 87% are ischemic strokes, and 185,000 are recurrent strokes [2]. In Korea, approximately 105,000 people experience a new or recurrent stroke annually, and 76.3% of those are ischemic strokes [3]. In 2019, the Global Burden of Diseases, Injuries, and Risk Factors Study showed that stroke was the second most common cause of disability-adjusted life years in the 50 to 74 and >75 years age groups [4]. With its high prevalence and tremendous burden on society, prevention, diagnosis, treatment, and rehabilitation of stroke survivors are essential parts of the public health agenda [5,6].
Stroke is a time-sensitive emergency, and emergency medical services (EMS) transport up to 70% of patients with stroke [7,8]. To achieve optimal outcomes in patients with stroke, it is crucial to minimize the interval from symptom onset to definitive treatment to restore blood flow to the stroke-affected tissue [9,10]. The EMS is the first point of contact in the prehospital phase of the stroke chain [11]. Therefore, rapid EMS activation and ambulance transport are recommended for patients with suspected stroke [12,13]. As a continuous process to reduce prehospital and in-hospital delays for acute stroke, this recommendation carries the advantage of allowing stroke screening and identification to be performed by the EMS providers even before hospital arrival [14].
Many prehospital stroke screening tools have been developed to support the rapid and accurate recognition of stroke by EMS providers during their first contact. In fact, the use of prehospital stroke screening tools by EMS providers during their initial triage of patients with symptoms of stroke has been recommended internationally, with a positive screening result indicating a high suspicion of stroke that calls for urgent specialized assessments [15,16].
Sometimes, performing a prehospital stroke screening test (PHSST) is impossible because the patient suspected of having an acute stroke is uncooperative or medically unstable in the field. At present, the consensus under the Korean EMS policy is that uncooperative prehospital patients suspected to have acute stroke who cannot undergo a PHSST are considered to have a high stroke risk. However, studies of uncooperative prehospital patients suspected to have acute stroke are lacking, and prehospital evaluation performance is uncertain.
In this study, we investigated the final hospital diagnoses and characteristics of uncooperative prehospital patients suspected to have acute stroke who were transported by the EMS and unable to undergo a PHSST. Our secondary objective was to evaluate the scenario-based real-world performance of the current Korean EMS stroke screening policy for uncooperative prehospital patients suspected to have acute stroke.
METHODS
Ethics statements
This study was approved by the Institutional Review Board of Jeju National University Hospital (No. 2021-07-013). Informed consent was waived due to the retrospective nature of the study.
Study design
This retrospective cross-sectional observational study was conducted at a single academic hospital from January 2015 to December 2019 to investigate the final hospital diagnoses and characteristics of uncooperative prehospital patients suspected to have acute stroke who were transported by the EMS and unable to undergo a PHSST.
Study setting
This retrospective observational study was conducted at the single academic hospital possessing the only regional comprehensive stroke center (CSC) on Jeju Island, which has a population of approximately 670,000 citizens and an area of 1,833.2 km2. The prehospital EMS system on Jeju Island is a government-operated fire-based system with a mostly single-tiered intermediate service level. It comprises a single centralized dispatch center, 29 ambulances, and approximately 130 EMS providers. It possesses six emergency medical institutions (receiving facilities) and one CSC, which provides in-hospital services for acute stroke patients on Jeju Island.
The Korean EMS protocol for patients with suspected acute stroke follows the national EMS standards, which were based on the American Heart Association and the American Stroke Association recommendations for triage, treatment, transport, and documentation during the study period [17]. Under this protocol, the prehospital criteria for suspicion of acute stroke are non-traumatic patients older than 15 years with any of the following structured chief complaint codes: headache, dizziness, altered mental status, seizure, convulsion, syncope, motor weakness, sensory change, or other findings indicative of acute stroke. If a patient meets the prehospital criteria for a suspected acute stroke, EMS providers collect important information (apparent onset, last normal time, and first abnormal time [FAT]) and determine the Cincinnati Prehospital Stroke Scale (CPSS) as the PHSST. If the result of the CPSS is positive, the patient is classified as an EMS-assessed acute stroke case. All EMS-assessed acute stroke cases must be transported to the nearest CSC after prehospital notification.
If it is not possible to perform CPSS in a patient who is uncooperative or medically unstable, the patient is classified as an EMS-assessed acute stroke case.
Data sources
Using EMS run sheets, EMS stroke registries, and hospital medical records from January 2015 to December 2019 as the data sources, we created a merged database for EMS-suspected acute stroke patients by manually linking the prehospital and hospital-phase data for each patient. The EMS stroke registry, which was developed by the National Fire Agency in 2012, has been used for patients with suspected acute stroke on Jeju Island since 2015. In this registry system, the EMS provider is required to record the EMS run sheets, which contain basic but comprehensive information for all patients who are transferred by EMS ambulance, with additional mandatory records for all patients who meet the prehospital criteria for suspicion of acute stroke. To merge individual patient records from the EMS run sheets, EMS stroke registry, and hospital medical records, we reviewed the common identifiers (EMS call time, emergency department [ED] arrival time, sex, and age) and the context.
Study population
Only adult patients who met the prehospital criteria for suspicion of acute stroke, were transported by the EMS to our ED and were registered in the EMS stroke registry between January 2015 and December 2019 were eligible for this study. Within that eligible population, patients for whom the EMS-assessed acute stroke case status or final clinical outcomes could not be determined from the EMS stroke registry or hospital record were excluded from the final analysis. The analyzed population contained participants aged 18 years or older at the time of the incident without other exclusion criteria and was divided into cooperative and uncooperative groups.
Variables and measurements
We compiled and categorized the demographic and clinical information of the participants at both the prehospital and hospital phases: age, sex, chief complaint on EMS arrival (dizziness, altered mental status, seizure/convulsion, loss of consciousness, motor weakness, headache, dysarthria, facial palsy, and other), FAT (clear or unclear), activity at onset (work, sleeping, daily activities, and other), medical comorbidities (yes or no), prehospital mental status (alert, verbal, painful, and unresponsive), blood glucose test (performed or not), results of CPSS test (positive or negative), level of EMS provider (nurse, emergency medical technician [EMT]-intermediate, EMT-basic), ED disposition (discharge, admission, interhospital transfer, or death), and hospital diagnosis codes.
Main variables for diagnostic testing
The EMS-assessed stroke recognition is a binary item determined by the results of the CPSS, which evaluates the presence of facial palsy, asymmetric arm weakness, and speech abnormalities. If any of the three items were marked positive, the EMS-assessed stroke recognition was positive.
The true diagnosis of the participants was considered the final hospital diagnosis. Therefore, we categorized three dichotomous indicators of acute stroke using the final hospital diagnosis ICD-10 codes: hemorrhagic stroke (ICD-10 diagnosis codes, I60.0–I62.9), ischemic stroke (ICD-10 diagnosis codes, I63.0–I63.9), and all strokes (ICD-10 diagnosis codes, I60.0–I64) [19].
We then evaluated the alternative diagnoses of patients in the uncooperative group whose final diagnoses did not contain ICD-10 codes for acute stroke. Two researchers independently reviewed the medical records and confirmed all final hospital diagnoses. The validated alternative diagnoses of the nonstroke patients in the uncooperative group were summarized based on a portion of their ICD codes (a single letter followed by the two digits that precede the period).
Statistical analysis
Descriptive statistics are presented as frequencies and percentages for categorical variables and means with standard deviations for continuous variables. Descriptive analyses between the cooperative and uncooperative groups were used to compare baseline demographic and clinical characteristics variables using Student t-test, chi-square test, or Fisher exact test as appropriate for the distribution. The performance (sensitivity, specificity, positive and negative likelihood ratios, positive predictive value [PPV], and negative predictive value [NPV]) of the EMS-assessed stroke recognition was evaluated for each of three scenarios using the final hospital diagnosis as the gold standard.
The first scenario used a conservative approach that calculated performance statistics only in patients who underwent the PHSST and excluded the uncooperative group. The second scenario used a real-world approach that considered all uncooperative patients as positive for EMS-assessed stroke recognition in calculating the performance statistics. The third scenario used a contrapositive approach, wherein all uncooperative patients were considered negative for EMS-assessed stroke recognition. We calculated the PPV and NPV with a stroke prevalence of 3% and compared the EMS stroke recognition performance in all three scenarios using the area under the receiver operating characteristic curve (AUC) [3].
To examine the prehospital factors associated with false-positive results in the uncooperative group, we performed univariate and multivariate logistic regressions adjusted for age and sex. All statistical analyses were performed using Stata ver. 17.0 (Stata Corp), with a two-tailed test and a statistical significance level of <0.05.
RESULTS
Study flow
Among the 41,002 EMS-transferred patients who visited our ED during the study period, 3,205 met the prehospital criteria for suspicion of acute stroke and were registered in the EMS stroke registry. Of that eligible population, 369 patients were excluded for the following reasons with overlapping cases: 55 were younger than 18 years or their age was undetermined, 117 were undetermined EMS-assessed acute stroke cases from the EMS stroke registry, 203 had undetermined final clinical outcomes in the hospital records, and 174 had EMS-to-hospital data linkage errors. The final analyzed population of 2,836 participants contained 486 uncooperative patients (17.1%) and 2,350 cooperative patients (82.9%) (Fig. 1).
Baseline demographics
The demographic and clinical characteristics of the uncooperative and cooperative groups are shown in Table 1. Of the 2,836 EMS-transported adult patients who met the prehospital criteria for suspicion of acute stroke, 570 (20.1%) had a confirmed final diagnosis of ischemic or hemorrhagic stroke; only the final diagnosis of hemorrhagic stroke was significantly higher in the uncooperative group than in the cooperative group (10.5% vs. 5.3%, P<0.001). The mean age of the participants in the uncooperative group was significantly higher (69.8±17.5 years vs. 63.7±17.5 years, P<0.001). However, the proportions of female patients and medical comorbidities, except diabetes and stroke, were similar in the two groups. Clinical characteristics, such as the chief complaint at EMS arrival, FAT, and ED disposition, differed between the groups. The chief complaint of altered mental status (78.0% vs. 14.7%) was the most common complaint in the uncooperative group, whereas dizziness (43.6% vs. 1.9%) was the most common complaint in the cooperative group. In the uncooperative group, the FAT was more often unclear (37.9% vs. 18.7%, P<0.001), and symptoms were more likely to occur during sleep (21.6% vs. 14.5%, P<0.001).
Accuracy analysis for each of the three scenarios
The sensitivity, specificity, positive and negative likelihood ratios, PPV, NPV, and AUCs for each of the three scenarios are summarized in Table 2. Among the 2,350 patients in the cooperative group (who underwent the CPSS), 552 (23.5%) were positive for one of the three CPSS items (presence of facial palsy, asymmetric arm weakness, and speech abnormalities) in the first scenario (conservative approach). The overall diagnostic performance statistics for the EMS-assessed stroke cases were as follows: 79.5% sensitivity (95% confidence interval [CI], 75.5%–83.1%), 90.2% specificity (95% CI, 88.8%–91.6%), and 0.849 AUC (95% CI, 0.829–0.868).
Among the 2,836 participants in the second (real-world approach) and third (contrapositive approach) scenarios, 1,038 (36.6%) and 552 participants (19.5%), respectively, were deemed to be positive EMS-assessed stroke cases. The sensitivity and specificity of EMS-assessed stroke cases were 83.3% (95% CI, 80.0%–86.3%) and 75.2% (95% CI, 73.3%–76.9%), respectively, in the second scenario and 64.6% (95% CI, 60.5%–68.5%) and 91.9% (95% CI, 90.7%–93.0%), respectively, in the third scenario. No significant difference was evident in the AUC between the second and third scenarios (0.792 [95% CI, 0.775–0.810] vs. 0.782 [95% CI, 0.762–0.803], P>0.05).
Prehospital factors associated with false-positive results for acute stroke in the uncooperative group
The prehospital factors that were associated with false-positive results for acute stroke in the uncooperative group are summarized in Table 3. Multivariate logistic regression revealed that seizure at EMS arrival (adjusted odds ratio [aOR], 3.27; 95% CI, 1.13–9.44), malignancy as a comorbidity (aOR, 4.36; 95% CI, 1.31–14.50), clear onset of FAT (aOR, 2.33; 95% CI, 1.50–3.61), and systolic blood pressure (SBP) <90 mmHg (aOR, 15.89; 95% CI, 2.16–117.07) were significantly associated with increased false-positive results for acute stroke after adjustment for age and sex. In contrast, weakness of the upper extremities (aOR, 0.10; 95% CI, 0.02–0.49) upon EMS arrival was associated with decreased false-positive instances. The remaining demographic and clinical factors were not significantly associated with false-positive results for acute stroke.
Alternative diagnoses for nonstroke patients in the uncooperative group
Of the 486 EMS-transferred uncooperative prehospital patients suspected to have acute stroke, 378 (77.8%) were identified as false positives. The alternative diagnoses, which were classified clinically using the final hospital ICD diagnosis codes, are summarized in Table 4. The commonly documented main categories of alternative diagnoses for the false-positive patients were nondiagnosis-classified symptoms and signs (n=85, 22.5%); infectious diseases of a particular system (n=66, 17.5%); mental and behavioral disorders (n=52, 13.8%); diabetes mellitus and metabolic complications (n=35, 9.3%); injury, poisoning, and other effects of external causes (n=31, 8.2%); episodic and paroxysmal disorders of the nervous system (n=27, 7.1%); diseases of the liver and biliary tract (n=26, 6.9%); malignant neoplasms (n=20, 5.3%); acute and chronic kidney failure (n=15, 4.0%); diseases of the circulatory system (n=11, 2.9%); and respiratory noninfectious diseases (n=10, 2.6%). The five most common alternative diagnoses were convulsions (n=47, 12.4%); infectious diseases of the respiratory system, including upper respiratory infection, influenza, and pneumonia (n=31, 8.2%); dementia, delirium, and other mental disorders (n=29, 7.7%); diabetes mellitus with hypoglycemia (n=27, 7.1%); and nonclassified shock (n=25, 6.6%).
DISCUSSION
Patients suspected of acute stroke can be medically unstable and uncooperative, and early stroke recognition in such patients is challenging in the prehospital setting [20,21]. However, little is known about prehospital evaluation of uncooperative patients suspected of acute stroke. Furthermore, the real-world performance of the current Korean EMS stroke screening policy for uncooperative prehospital patients suspected of acute stroke remains unclear. Therefore, we performed this study to clarify the final hospital diagnoses and scenario-based real-world performance of the current Korean EMS stroke screening policy for uncooperative prehospital patients suspected of acute stroke.
Although the EMS transports up to 70% of stroke patients, the proportion of uncooperative stroke patients is not well established [7,8]. In the present study, 17.1% of the patients in the uncooperative group met the prehospital criteria for suspected acute stroke. Although our data do not specify the exact proportion of uncooperative patients, previous studies have shown that 8% to 25% of stroke patients have altered mental status, which is similar to our result [22]. We found several significant intergroup differences in baseline demographic and clinical characteristics, including patient age, chief complaint at EMS arrival, FAT, activity at onset, presence of blood glucose testing, and ED disposition. In particular, inspection of the chief complaint at EMS arrival revealed that altered mental status (78.0%) was the most common presentation in the uncooperative group, followed by seizures (10.3%) and loss of consciousness (3.7%). In a study that investigated patient characteristics that affect prehospital identification of stroke by the EMS, altered mental status was associated with a 6.5-fold higher risk that EMS providers would miss a diagnosis of stroke because the PHSST could not be performed in those patients. However, no intergroup difference was observed in our data with regard to the prevalence of a final diagnosis of ischemic or hemorrhagic stroke. These findings suggest that the number of stroke patients in the uncooperative group cannot be ignored, and that a novel stroke screening approach is needed to replace the current conventional stroke screening methods [20,22–24].
We also evaluated the real-world performance of the current Korean EMS stroke screening policy using different scenarios for uncooperative prehospital patients suspected of acute stroke. EMS providers in Korea use the CPSS as the primary PHSST for stroke identification. A recent systematic study evaluated the diagnostic performance of clinical tools for stroke identification and reported that the CPSS distinguished between acute stroke and stroke mimics with 83% sensitivity, 69% specificity, 50% PPV, and 91% NPV [15,16]. In our results of CPSS diagnostic performance in the first scenario (conservative approach), its sensitivity was 79.5% (95% CI, 75.5%–83.1%), which was in the same range as in a previous review, whereas its 90.2% specificity (95% CI, 88.8%–91.6%) and 99.3% NPV (95% CI, 99.2%–99.4%) in our study were higher than those reported in previous reviews. On the other hand, the 20.1% PPV (95% CI, 17.9%–22.6%) was less than half of that reported previously. It is plausible that these results could be related to selection of the study samples. Unlike other studies, which included only cooperative patients from a convenience sample drawn from prehospital patients subjectively suspected of acute stroke by EMS providers, our study population was systematically recruited in accordance with the national EMS stroke protocol, which is based on structured chief complaint codes and mandatory standardized records [21–29]. Therefore, our study population is inclusive and consistent irrespective of the subjective suspicions of EMS providers.
Compared with the first scenario (conservative approach), the sensitivity increased slightly, and the specificity decreased markedly in the second scenario (real-world approach). In contrast, the sensitivity decreased, and the specificity increased slightly in the third scenario (contrapositive approach). In a population with a 3% prevalence of stroke, overcalls (false positives) are expected to increase by 16,199 persons, whereas missed strokes (false negatives) would decrease by 516 persons per 100,000 population in the second scenario compared with the third scenario (Supplementary Table 1) [3]. These findings provide quantitative evidence for predicting the overestimation (false positive) and underestimation (false negative) effects of changing the EMS stroke screening policy for uncooperative patients. Therefore, although there is currently no consensus on optimizing the prehospital stroke assessment policy for uncooperative patients, it is necessary to compare the EMS burden of policy options based on likely overestimation or underestimation.
Most of the currently available prehospital screening tools are designed to assess the most common symptoms of acute stroke [15,16,30]. Therefore, they all provide prehospital responders a good ability to recognize positive acute stroke cases; however, their ability to exclude stroke mimics is not good and offers only modest diagnostic accuracy [16]. Therefore, this pattern of prehospital diagnosis might lead to overestimation of acute stroke and could overburden special EMS facilities.
The diagnostic performance for distinguishing between acute stroke and stroke mimics is crucial for uncooperative patients because it is very difficult to evaluate most of the prehospital screening items in this population. This situation emphasizes the need for an alternative approach to exclude stroke-mimicking conditions in uncooperative patients. One scoring system (FABS) was developed to identify stroke mimics in patients with suspected acute stroke [16,31,32]. This scoring system is calculated based on six variables, with one point for each variable present (absence of facial droop, negative history of atrial fibrillation, age <50 years, systolic blood pressure <150 mmHg at presentation, history of seizures, and isolated sensory symptoms without weakness at presentation). A FABS score of ≥3 demonstrated the best overall diagnostic performance, with 90% sensitivity (95% CI, 86%–93%), 91% specificity (95% CI, 88%–93%), 87% PPV (95% CI, 83%–91%), and 93% NPV (95% CI, 90%–95%) [31].
Similar to the characteristics of the FABS scoring system, our study identified five prehospital factors that were associated with stroke mimics in the uncooperative group. Seizure at presentation, medical history of malignancy, clear onset of FAT, SBP <90 mmHg, and absence of motor weakness of the upper extremities were significantly associated with stroke mimics in our logistic regression model after adjustment for age and sex.
Furthermore, we summarized the alternative diagnoses and the five most common stroke mimics in the uncooperative group. Together, our findings provide important insights into the potential parameters needed to facilitate triage of uncooperative patients suspected of acute stroke in the prehospital setting and the ED.
Novel advanced technologies could be another option for optimal prehospital stroke triage, even in uncooperative patients [33–35]. In 2019, a systematic literature review outlined the potential of noninvasive sensor technology for prehospital stroke diagnosis and provided information on 10 noninvasive external sensor devices based on seven technologies (accelerometers, electroencephalography, microwaves, near-infrared, radiofrequency, transcranial Doppler ultrasound, and volumetric impedance phase-shift spectroscopy) [33]. However, further studies are required to verify the feasibility and validation of those prehospital stroke screening systems.
Several limitations of our study should be considered when interpreting the results. First, the principal limitation of our study is the difference in design of the study sample from previous studies, which we chose to minimize the subjective suspicions of EMS providers; in our sample, all participants who matched the chief complaint codes for prehospital patients suspected of acute stroke were used as denominators. Thus, our diagnostic performance, especially PPV, could differ from that of other studies. Second, the EMS transport-to-hospital data linkage verification was limited, and 174 EMS-transported records were not linked to hospital data. The overall match rate for unique one to one EMS transport-to-ED visits was approximately 95%. Third, our retrospective study used the ICD-10 diagnostic codes from the EMR to establish a final diagnosis, which was affected by the completeness and accuracy of the EMR. Fourth, the study population was obtained from a small subset of EMS operations in only one province, limiting the generalizability of these results to other regions.
In conclusion, the final diagnosis of acute stroke in the uncooperative group did not differ significantly from that in the cooperative group. Given the changes in overestimation (false positive) and underestimation (false negative) according to the EMS stroke screening policy for the uncooperative group, further research is needed to develop a novel stroke screening approach that is customized for evaluation of uncooperative patients suspected of acute stroke.
PDF Links
PubReader
ePub Link
XML Download
Full text via DOI
Download Citation 
Supplement
Print 
