Abstract
-
Objective
This study investigated the hospital diagnoses and characteristics of uncooperative prehospital patients suspected of acute stroke who could not undergo a prehospital stroke screening test (PHSST).
-
Methods
This retrospective observational study was conducted at a single academic hospital with a regional stroke center. We analyzed three scenario-based prehospital stroke screening performances using the final hospital diagnoses: (1) a conservative approach only in patients who underwent the PHSST, (2) a real-world approach that considered all uncooperative patients as screening positive, and (3) a contrapositive approach that all uncooperative patients were considered as negative.
-
Results
Of the 2,836 emergency medical services (EMS)-transported adult patients who met the prehospital criteria for suspicion of acute stroke, 486 (17.1%) were uncooperative, and 570 (20.1%) had a confirmed final diagnosis of acute stroke. The diagnosis in the uncooperative group did not differ from that in the cooperative group (22.0% vs. 19.7%, P=0.246). The diagnostic performances of the PHSST in the conservative approach were as follows: 79.5% sensitivity (95% confidence interval [CI], 75.5%–83.1%), 90.2% specificity (95% CI, 88.8%–91.6%), and 0.849 area under the receiver operating characteristic curve (AUC; 95% CI, 0.829–0.868). The sensitivity and specificity were 83.3% (95% CI, 80.0%–86.3%) and 75.2% (95% CI, 73.3%–76.9%), respectively, in the real-world approach and 64.6% (95% CI, 60.5%–68.5%) and 91.9% (95% CI, 90.7%–93.0%), respectively, in the contrapositive approach. No significant difference was evident in the AUC between the real-world approach and the contrapositive approach (0.792 [95% CI, 0.775–0.810] vs. 0.782 [95% CI, 0.762–0.803], P>0.05).
-
Conclusion
We found overestimation (false positive) and underestimation (false negative) in the uncooperative group depending on the scenario-based EMS stroke screening policy for uncooperative prehospital patients suspected of acute stroke.
-
Keywords: Emergency medical services; Stroke; Early diagnosis; Sensitivity and specificity
Capsule Summary
What is already known
Patients suspected of acute stroke can be medically unstable and uncooperative, and early stroke recognition in such patients is challenging in the prehospital setting. However, little is known about the prehospital evaluations of uncooperative patients suspected of acute stroke.
What is new in the current study
The final diagnosis of acute stroke in the uncooperative group did not differ significantly from that in the cooperative group (22.0% vs. 19.7%, P=0.246). We provide quantitative evidence of overestimation (false positive) and underestimation (false negative) in the uncooperative group depending on the emergency medical services stroke screening policy. In addition, prehospital factors (seizure at presentation, medical history of malignancy, clear onset of first abnormal time, systolic blood pressure <90 mmHg, and absence of motor weakness of the upper extremities) were significantly associated with stroke mimics in the uncooperative group.
INTRODUCTION
Stroke is a leading cause of disability and death worldwide and is clinically described as a neurological deficit resulting from an infarction in the central nervous system (brain, spinal cord, and retinal cell death) that was caused by ischemia or hemorrhage (ischemic or hemorrhagic stroke, respectively) [
1]. Approximately 795,000 individuals in the United States experience a stroke each year; of these, 87% are ischemic strokes, and 185,000 are recurrent strokes [
2]. In Korea, approximately 105,000 people experience a new or recurrent stroke annually, and 76.3% of those are ischemic strokes [
3]. In 2019, the Global Burden of Diseases, Injuries, and Risk Factors Study showed that stroke was the second most common cause of disability-adjusted life years in the 50 to 74 and >75 years age groups [
4]. With its high prevalence and tremendous burden on society, prevention, diagnosis, treatment, and rehabilitation of stroke survivors are essential parts of the public health agenda [
5,
6].
Stroke is a time-sensitive emergency, and emergency medical services (EMS) transport up to 70% of patients with stroke [
7,
8]. To achieve optimal outcomes in patients with stroke, it is crucial to minimize the interval from symptom onset to definitive treatment to restore blood flow to the stroke-affected tissue [
9,
10]. The EMS is the first point of contact in the prehospital phase of the stroke chain [
11]. Therefore, rapid EMS activation and ambulance transport are recommended for patients with suspected stroke [
12,
13]. As a continuous process to reduce prehospital and in-hospital delays for acute stroke, this recommendation carries the advantage of allowing stroke screening and identification to be performed by the EMS providers even before hospital arrival [
14].
Many prehospital stroke screening tools have been developed to support the rapid and accurate recognition of stroke by EMS providers during their first contact. In fact, the use of prehospital stroke screening tools by EMS providers during their initial triage of patients with symptoms of stroke has been recommended internationally, with a positive screening result indicating a high suspicion of stroke that calls for urgent specialized assessments [
15,
16].
Sometimes, performing a prehospital stroke screening test (PHSST) is impossible because the patient suspected of having an acute stroke is uncooperative or medically unstable in the field. At present, the consensus under the Korean EMS policy is that uncooperative prehospital patients suspected to have acute stroke who cannot undergo a PHSST are considered to have a high stroke risk. However, studies of uncooperative prehospital patients suspected to have acute stroke are lacking, and prehospital evaluation performance is uncertain.
In this study, we investigated the final hospital diagnoses and characteristics of uncooperative prehospital patients suspected to have acute stroke who were transported by the EMS and unable to undergo a PHSST. Our secondary objective was to evaluate the scenario-based real-world performance of the current Korean EMS stroke screening policy for uncooperative prehospital patients suspected to have acute stroke.
METHODS
Ethics statements
This study was approved by the Institutional Review Board of Jeju National University Hospital (No. 2021-07-013). Informed consent was waived due to the retrospective nature of the study.
Study design
This retrospective cross-sectional observational study was conducted at a single academic hospital from January 2015 to December 2019 to investigate the final hospital diagnoses and characteristics of uncooperative prehospital patients suspected to have acute stroke who were transported by the EMS and unable to undergo a PHSST.
Study setting
This retrospective observational study was conducted at the single academic hospital possessing the only regional comprehensive stroke center (CSC) on Jeju Island, which has a population of approximately 670,000 citizens and an area of 1,833.2 km2. The prehospital EMS system on Jeju Island is a government-operated fire-based system with a mostly single-tiered intermediate service level. It comprises a single centralized dispatch center, 29 ambulances, and approximately 130 EMS providers. It possesses six emergency medical institutions (receiving facilities) and one CSC, which provides in-hospital services for acute stroke patients on Jeju Island.
The Korean EMS protocol for patients with suspected acute stroke follows the national EMS standards, which were based on the American Heart Association and the American Stroke Association recommendations for triage, treatment, transport, and documentation during the study period [
17]. Under this protocol, the prehospital criteria for suspicion of acute stroke are non-traumatic patients older than 15 years with any of the following structured chief complaint codes: headache, dizziness, altered mental status, seizure, convulsion, syncope, motor weakness, sensory change, or other findings indicative of acute stroke. If a patient meets the prehospital criteria for a suspected acute stroke, EMS providers collect important information (apparent onset, last normal time, and first abnormal time [FAT]) and determine the Cincinnati Prehospital Stroke Scale (CPSS) as the PHSST. If the result of the CPSS is positive, the patient is classified as an EMS-assessed acute stroke case. All EMS-assessed acute stroke cases must be transported to the nearest CSC after prehospital notification.
If it is not possible to perform CPSS in a patient who is uncooperative or medically unstable, the patient is classified as an EMS-assessed acute stroke case.
Data sources
Using EMS run sheets, EMS stroke registries, and hospital medical records from January 2015 to December 2019 as the data sources, we created a merged database for EMS-suspected acute stroke patients by manually linking the prehospital and hospital-phase data for each patient. The EMS stroke registry, which was developed by the National Fire Agency in 2012, has been used for patients with suspected acute stroke on Jeju Island since 2015. In this registry system, the EMS provider is required to record the EMS run sheets, which contain basic but comprehensive information for all patients who are transferred by EMS ambulance, with additional mandatory records for all patients who meet the prehospital criteria for suspicion of acute stroke. To merge individual patient records from the EMS run sheets, EMS stroke registry, and hospital medical records, we reviewed the common identifiers (EMS call time, emergency department [ED] arrival time, sex, and age) and the context.
Study population
Only adult patients who met the prehospital criteria for suspicion of acute stroke, were transported by the EMS to our ED and were registered in the EMS stroke registry between January 2015 and December 2019 were eligible for this study. Within that eligible population, patients for whom the EMS-assessed acute stroke case status or final clinical outcomes could not be determined from the EMS stroke registry or hospital record were excluded from the final analysis. The analyzed population contained participants aged 18 years or older at the time of the incident without other exclusion criteria and was divided into cooperative and uncooperative groups.
Variables and measurements
We compiled and categorized the demographic and clinical information of the participants at both the prehospital and hospital phases: age, sex, chief complaint on EMS arrival (dizziness, altered mental status, seizure/convulsion, loss of consciousness, motor weakness, headache, dysarthria, facial palsy, and other), FAT (clear or unclear), activity at onset (work, sleeping, daily activities, and other), medical comorbidities (yes or no), prehospital mental status (alert, verbal, painful, and unresponsive), blood glucose test (performed or not), results of CPSS test (positive or negative), level of EMS provider (nurse, emergency medical technician [EMT]-intermediate, EMT-basic), ED disposition (discharge, admission, interhospital transfer, or death), and hospital diagnosis codes.
The hospital diagnoses for the participants were obtained from the electronic medical records (EMR) and were based on the World Health Organization’s International Classification of Diseases 10th Revision (ICD-10) diagnosis codes [
18,
19].
The EMS-assessed stroke recognition is a binary item determined by the results of the CPSS, which evaluates the presence of facial palsy, asymmetric arm weakness, and speech abnormalities. If any of the three items were marked positive, the EMS-assessed stroke recognition was positive.
The true diagnosis of the participants was considered the final hospital diagnosis. Therefore, we categorized three dichotomous indicators of acute stroke using the final hospital diagnosis ICD-10 codes: hemorrhagic stroke (ICD-10 diagnosis codes, I60.0–I62.9), ischemic stroke (ICD-10 diagnosis codes, I63.0–I63.9), and all strokes (ICD-10 diagnosis codes, I60.0–I64) [
19].
We then evaluated the alternative diagnoses of patients in the uncooperative group whose final diagnoses did not contain ICD-10 codes for acute stroke. Two researchers independently reviewed the medical records and confirmed all final hospital diagnoses. The validated alternative diagnoses of the nonstroke patients in the uncooperative group were summarized based on a portion of their ICD codes (a single letter followed by the two digits that precede the period).
Statistical analysis
Descriptive statistics are presented as frequencies and percentages for categorical variables and means with standard deviations for continuous variables. Descriptive analyses between the cooperative and uncooperative groups were used to compare baseline demographic and clinical characteristics variables using Student t-test, chi-square test, or Fisher exact test as appropriate for the distribution. The performance (sensitivity, specificity, positive and negative likelihood ratios, positive predictive value [PPV], and negative predictive value [NPV]) of the EMS-assessed stroke recognition was evaluated for each of three scenarios using the final hospital diagnosis as the gold standard.
The first scenario used a conservative approach that calculated performance statistics only in patients who underwent the PHSST and excluded the uncooperative group. The second scenario used a real-world approach that considered all uncooperative patients as positive for EMS-assessed stroke recognition in calculating the performance statistics. The third scenario used a contrapositive approach, wherein all uncooperative patients were considered negative for EMS-assessed stroke recognition. We calculated the PPV and NPV with a stroke prevalence of 3% and compared the EMS stroke recognition performance in all three scenarios using the area under the receiver operating characteristic curve (AUC) [
3].
To examine the prehospital factors associated with false-positive results in the uncooperative group, we performed univariate and multivariate logistic regressions adjusted for age and sex. All statistical analyses were performed using Stata ver. 17.0 (Stata Corp), with a two-tailed test and a statistical significance level of <0.05.
RESULTS
Study flow
Among the 41,002 EMS-transferred patients who visited our ED during the study period, 3,205 met the prehospital criteria for suspicion of acute stroke and were registered in the EMS stroke registry. Of that eligible population, 369 patients were excluded for the following reasons with overlapping cases: 55 were younger than 18 years or their age was undetermined, 117 were undetermined EMS-assessed acute stroke cases from the EMS stroke registry, 203 had undetermined final clinical outcomes in the hospital records, and 174 had EMS-to-hospital data linkage errors. The final analyzed population of 2,836 participants contained 486 uncooperative patients (17.1%) and 2,350 cooperative patients (82.9%) (
Fig. 1).
The demographic and clinical characteristics of the uncooperative and cooperative groups are shown in
Table 1. Of the 2,836 EMS-transported adult patients who met the prehospital criteria for suspicion of acute stroke, 570 (20.1%) had a confirmed final diagnosis of ischemic or hemorrhagic stroke; only the final diagnosis of hemorrhagic stroke was significantly higher in the uncooperative group than in the cooperative group (10.5% vs. 5.3%, P<0.001). The mean age of the participants in the uncooperative group was significantly higher (69.8±17.5 years vs. 63.7±17.5 years, P<0.001). However, the proportions of female patients and medical comorbidities, except diabetes and stroke, were similar in the two groups. Clinical characteristics, such as the chief complaint at EMS arrival, FAT, and ED disposition, differed between the groups. The chief complaint of altered mental status (78.0% vs. 14.7%) was the most common complaint in the uncooperative group, whereas dizziness (43.6% vs. 1.9%) was the most common complaint in the cooperative group. In the uncooperative group, the FAT was more often unclear (37.9% vs. 18.7%, P<0.001), and symptoms were more likely to occur during sleep (21.6% vs. 14.5%, P<0.001).
The sensitivity, specificity, positive and negative likelihood ratios, PPV, NPV, and AUCs for each of the three scenarios are summarized in
Table 2. Among the 2,350 patients in the cooperative group (who underwent the CPSS), 552 (23.5%) were positive for one of the three CPSS items (presence of facial palsy, asymmetric arm weakness, and speech abnormalities) in the first scenario (conservative approach). The overall diagnostic performance statistics for the EMS-assessed stroke cases were as follows: 79.5% sensitivity (95% confidence interval [CI], 75.5%–83.1%), 90.2% specificity (95% CI, 88.8%–91.6%), and 0.849 AUC (95% CI, 0.829–0.868).
Among the 2,836 participants in the second (real-world approach) and third (contrapositive approach) scenarios, 1,038 (36.6%) and 552 participants (19.5%), respectively, were deemed to be positive EMS-assessed stroke cases. The sensitivity and specificity of EMS-assessed stroke cases were 83.3% (95% CI, 80.0%–86.3%) and 75.2% (95% CI, 73.3%–76.9%), respectively, in the second scenario and 64.6% (95% CI, 60.5%–68.5%) and 91.9% (95% CI, 90.7%–93.0%), respectively, in the third scenario. No significant difference was evident in the AUC between the second and third scenarios (0.792 [95% CI, 0.775–0.810] vs. 0.782 [95% CI, 0.762–0.803], P>0.05).
Prehospital factors associated with false-positive results for acute stroke in the uncooperative group
The prehospital factors that were associated with false-positive results for acute stroke in the uncooperative group are summarized in
Table 3. Multivariate logistic regression revealed that seizure at EMS arrival (adjusted odds ratio [aOR], 3.27; 95% CI, 1.13–9.44), malignancy as a comorbidity (aOR, 4.36; 95% CI, 1.31–14.50), clear onset of FAT (aOR, 2.33; 95% CI, 1.50–3.61), and systolic blood pressure (SBP) <90 mmHg (aOR, 15.89; 95% CI, 2.16–117.07) were significantly associated with increased false-positive results for acute stroke after adjustment for age and sex. In contrast, weakness of the upper extremities (aOR, 0.10; 95% CI, 0.02–0.49) upon EMS arrival was associated with decreased false-positive instances. The remaining demographic and clinical factors were not significantly associated with false-positive results for acute stroke.
Of the 486 EMS-transferred uncooperative prehospital patients suspected to have acute stroke, 378 (77.8%) were identified as false positives. The alternative diagnoses, which were classified clinically using the final hospital ICD diagnosis codes, are summarized in
Table 4. The commonly documented main categories of alternative diagnoses for the false-positive patients were nondiagnosis-classified symptoms and signs (n=85, 22.5%); infectious diseases of a particular system (n=66, 17.5%); mental and behavioral disorders (n=52, 13.8%); diabetes mellitus and metabolic complications (n=35, 9.3%); injury, poisoning, and other effects of external causes (n=31, 8.2%); episodic and paroxysmal disorders of the nervous system (n=27, 7.1%); diseases of the liver and biliary tract (n=26, 6.9%); malignant neoplasms (n=20, 5.3%); acute and chronic kidney failure (n=15, 4.0%); diseases of the circulatory system (n=11, 2.9%); and respiratory noninfectious diseases (n=10, 2.6%). The five most common alternative diagnoses were convulsions (n=47, 12.4%); infectious diseases of the respiratory system, including upper respiratory infection, influenza, and pneumonia (n=31, 8.2%); dementia, delirium, and other mental disorders (n=29, 7.7%); diabetes mellitus with hypoglycemia (n=27, 7.1%); and nonclassified shock (n=25, 6.6%).
DISCUSSION
Patients suspected of acute stroke can be medically unstable and uncooperative, and early stroke recognition in such patients is challenging in the prehospital setting [
20,
21]. However, little is known about prehospital evaluation of uncooperative patients suspected of acute stroke. Furthermore, the real-world performance of the current Korean EMS stroke screening policy for uncooperative prehospital patients suspected of acute stroke remains unclear. Therefore, we performed this study to clarify the final hospital diagnoses and scenario-based real-world performance of the current Korean EMS stroke screening policy for uncooperative prehospital patients suspected of acute stroke.
Although the EMS transports up to 70% of stroke patients, the proportion of uncooperative stroke patients is not well established [
7,
8]. In the present study, 17.1% of the patients in the uncooperative group met the prehospital criteria for suspected acute stroke. Although our data do not specify the exact proportion of uncooperative patients, previous studies have shown that 8% to 25% of stroke patients have altered mental status, which is similar to our result [
22]. We found several significant intergroup differences in baseline demographic and clinical characteristics, including patient age, chief complaint at EMS arrival, FAT, activity at onset, presence of blood glucose testing, and ED disposition. In particular, inspection of the chief complaint at EMS arrival revealed that altered mental status (78.0%) was the most common presentation in the uncooperative group, followed by seizures (10.3%) and loss of consciousness (3.7%). In a study that investigated patient characteristics that affect prehospital identification of stroke by the EMS, altered mental status was associated with a 6.5-fold higher risk that EMS providers would miss a diagnosis of stroke because the PHSST could not be performed in those patients. However, no intergroup difference was observed in our data with regard to the prevalence of a final diagnosis of ischemic or hemorrhagic stroke. These findings suggest that the number of stroke patients in the uncooperative group cannot be ignored, and that a novel stroke screening approach is needed to replace the current conventional stroke screening methods [
20,
22–
24].
We also evaluated the real-world performance of the current Korean EMS stroke screening policy using different scenarios for uncooperative prehospital patients suspected of acute stroke. EMS providers in Korea use the CPSS as the primary PHSST for stroke identification. A recent systematic study evaluated the diagnostic performance of clinical tools for stroke identification and reported that the CPSS distinguished between acute stroke and stroke mimics with 83% sensitivity, 69% specificity, 50% PPV, and 91% NPV [
15,
16]. In our results of CPSS diagnostic performance in the first scenario (conservative approach), its sensitivity was 79.5% (95% CI, 75.5%–83.1%), which was in the same range as in a previous review, whereas its 90.2% specificity (95% CI, 88.8%–91.6%) and 99.3% NPV (95% CI, 99.2%–99.4%) in our study were higher than those reported in previous reviews. On the other hand, the 20.1% PPV (95% CI, 17.9%–22.6%) was less than half of that reported previously. It is plausible that these results could be related to selection of the study samples. Unlike other studies, which included only cooperative patients from a convenience sample drawn from prehospital patients subjectively suspected of acute stroke by EMS providers, our study population was systematically recruited in accordance with the national EMS stroke protocol, which is based on structured chief complaint codes and mandatory standardized records [
21–
29]. Therefore, our study population is inclusive and consistent irrespective of the subjective suspicions of EMS providers.
Compared with the first scenario (conservative approach), the sensitivity increased slightly, and the specificity decreased markedly in the second scenario (real-world approach). In contrast, the sensitivity decreased, and the specificity increased slightly in the third scenario (contrapositive approach). In a population with a 3% prevalence of stroke, overcalls (false positives) are expected to increase by 16,199 persons, whereas missed strokes (false negatives) would decrease by 516 persons per 100,000 population in the second scenario compared with the third scenario (
Supplementary Table 1) [
3]. These findings provide quantitative evidence for predicting the overestimation (false positive) and underestimation (false negative) effects of changing the EMS stroke screening policy for uncooperative patients. Therefore, although there is currently no consensus on optimizing the prehospital stroke assessment policy for uncooperative patients, it is necessary to compare the EMS burden of policy options based on likely overestimation or underestimation.
Most of the currently available prehospital screening tools are designed to assess the most common symptoms of acute stroke [
15,
16,
30]. Therefore, they all provide prehospital responders a good ability to recognize positive acute stroke cases; however, their ability to exclude stroke mimics is not good and offers only modest diagnostic accuracy [
16]. Therefore, this pattern of prehospital diagnosis might lead to overestimation of acute stroke and could overburden special EMS facilities.
The diagnostic performance for distinguishing between acute stroke and stroke mimics is crucial for uncooperative patients because it is very difficult to evaluate most of the prehospital screening items in this population. This situation emphasizes the need for an alternative approach to exclude stroke-mimicking conditions in uncooperative patients. One scoring system (FABS) was developed to identify stroke mimics in patients with suspected acute stroke [
16,
31,
32]. This scoring system is calculated based on six variables, with one point for each variable present (absence of facial droop, negative history of atrial fibrillation, age <50 years, systolic blood pressure <150 mmHg at presentation, history of seizures, and isolated sensory symptoms without weakness at presentation). A FABS score of ≥3 demonstrated the best overall diagnostic performance, with 90% sensitivity (95% CI, 86%–93%), 91% specificity (95% CI, 88%–93%), 87% PPV (95% CI, 83%–91%), and 93% NPV (95% CI, 90%–95%) [
31].
Similar to the characteristics of the FABS scoring system, our study identified five prehospital factors that were associated with stroke mimics in the uncooperative group. Seizure at presentation, medical history of malignancy, clear onset of FAT, SBP <90 mmHg, and absence of motor weakness of the upper extremities were significantly associated with stroke mimics in our logistic regression model after adjustment for age and sex.
Furthermore, we summarized the alternative diagnoses and the five most common stroke mimics in the uncooperative group. Together, our findings provide important insights into the potential parameters needed to facilitate triage of uncooperative patients suspected of acute stroke in the prehospital setting and the ED.
Novel advanced technologies could be another option for optimal prehospital stroke triage, even in uncooperative patients [
33–
35]. In 2019, a systematic literature review outlined the potential of noninvasive sensor technology for prehospital stroke diagnosis and provided information on 10 noninvasive external sensor devices based on seven technologies (accelerometers, electroencephalography, microwaves, near-infrared, radiofrequency, transcranial Doppler ultrasound, and volumetric impedance phase-shift spectroscopy) [
33]. However, further studies are required to verify the feasibility and validation of those prehospital stroke screening systems.
Several limitations of our study should be considered when interpreting the results. First, the principal limitation of our study is the difference in design of the study sample from previous studies, which we chose to minimize the subjective suspicions of EMS providers; in our sample, all participants who matched the chief complaint codes for prehospital patients suspected of acute stroke were used as denominators. Thus, our diagnostic performance, especially PPV, could differ from that of other studies. Second, the EMS transport-to-hospital data linkage verification was limited, and 174 EMS-transported records were not linked to hospital data. The overall match rate for unique one to one EMS transport-to-ED visits was approximately 95%. Third, our retrospective study used the ICD-10 diagnostic codes from the EMR to establish a final diagnosis, which was affected by the completeness and accuracy of the EMR. Fourth, the study population was obtained from a small subset of EMS operations in only one province, limiting the generalizability of these results to other regions.
In conclusion, the final diagnosis of acute stroke in the uncooperative group did not differ significantly from that in the cooperative group. Given the changes in overestimation (false positive) and underestimation (false negative) according to the EMS stroke screening policy for the uncooperative group, further research is needed to develop a novel stroke screening approach that is customized for evaluation of uncooperative patients suspected of acute stroke.
NOTES
-
ETHICS STATEMENTS
This study was approved by the Institutional Review Board of Jeju National University Hospital (No. 2021-07-013). Informed consent was waived due to the retrospective nature of the study.
-
CONFLICT OF INTEREST
No potential conflict of interest relevant to this article was reported.
-
FUNDING
None.
-
AUTHOR CONTRIBUTIONS
Conceptualization: SWS, WJK; Data curation: SWS, CHK; Formal analysis: SWS, JHK; Investigation: HH, CBP, JHB; Methodology: SWS, JHK, , SHL; Supervision: SWS, YJK, CHK; Validation: JHB, SKL, SYK; Visualization: SWS, CBP; Writing–original draft: SH, HH; Writing–review & editing: WJK, JHK, SKL, SYK, SHL. All authors read and approved the final manuscript.
SUPPLEMENTARY MATERIAL
Supplementary Table 1.
Overestimation or underestimation effects of the emergency medical services stroke screening policy for the uncooperative group
ceem-22-372-supplementary-Table-1.pdf
Fig. 1.Flow diagram of patient enrollment. EMS, emergency medical services; ED, emergency department; CPSS, Cincinnati Prehospital Stroke Scale. a)Inconsistent total due to overlapping cases.
Table 1.Baseline characteristics of EMS-transferred patients with suspected acute stroke
Table 1.
|
Characteristic |
Total (n = 2,836) |
Uncooperative (n = 486) |
Cooperative (n = 2,350) |
P-value |
|
Age (yr) |
64.8 ± 17.7 |
69.8 ± 17.5 |
63.7 ± 17.5 |
< 0.001 |
|
< 45 |
402 (14.2) |
48 (9.9) |
354 (15.1) |
0.003 |
|
Female sex |
1,381 (48.7) |
250 (51.4) |
1,131 (48.1) |
0.184 |
|
Chief complaint at EMS arrival |
|
|
|
< 0.001 |
|
Dizziness |
1,034 (36.5) |
9 (1.9) |
1,025 (43.6) |
|
|
Altered mental status |
725 (25.6) |
379 (78.0) |
346 (14.7) |
|
|
Seizure |
273 (9.6) |
50 (10.3) |
223 (9.5) |
|
|
Loss of consciousness |
173 (6.1) |
18 (3.7) |
155 (6.6) |
|
|
Weakness |
|
|
|
|
|
Upper extremity |
254 (9.0) |
8 (1.7) |
246 (10.5) |
|
|
Lower extremity |
60 (2.1) |
2 (0.4) |
58 (2.5) |
|
|
Headache |
107 (3.8) |
5 (1.0) |
102 (4.3) |
|
|
Dysarthria |
102 (3.6) |
6 (1.2) |
96 (4.1) |
|
|
Facial palsy |
60 (2.1) |
2 (0.4) |
58 (2.5) |
|
|
Other |
47 (1.7) |
7 (1.4) |
40 (1.7) |
|
|
First abnormal time |
|
|
|
< 0.001 |
|
Clear |
2,212 (78.0) |
302 (62.1) |
1,910 (81.3) |
|
|
Unclear |
624 (22.0) |
184 (37.9) |
440 (18.7) |
|
|
Activity at onset |
|
|
|
< 0.001 |
|
On duty |
85 (3.0) |
11 (2.3) |
74 (3.1) |
|
|
Sleeping |
446 (15.7) |
105 (21.6) |
341 (14.5) |
|
|
Daily activity |
2,100 (74.1) |
329 (67.7) |
1,771 (75.4) |
|
|
Other activity |
205 (7.2) |
41 (8.4) |
164 (7.0) |
|
|
Medical history |
|
|
|
|
|
Hypertension |
1,089 (38.4) |
174 (35.8) |
915 (38.9) |
0.196 |
|
Diabetes |
591 (20.8) |
118 (24.3) |
473 (20.1) |
0.040 |
|
Stroke |
407 (14.4) |
94 (19.3) |
313 (13.3) |
0.001 |
|
Cardiovascular disease |
298 (10.5) |
40 (8.2) |
258 (11.0) |
0.072 |
|
Malignancy |
200 (7.1) |
43 (8.9) |
157 (6.7) |
0.089 |
|
Blood glucose test (yes) |
1,793 (63.2) |
382 (78.6) |
1,411 (60.0) |
< 0.001 |
|
CPSS (positive) |
|
|
|
|
|
Facial palsy |
296 (12.6) |
NA |
296 (12.6) |
NA |
|
Arm weakness |
472 (20.1) |
NA |
472 (20.1) |
NA |
|
Dysarthria |
417 (17.7) |
NA |
417 (17.7) |
NA |
|
Level of the EMS crew |
|
|
|
0.964 |
|
Nurse |
959 (33.8) |
163 (33.5) |
796 (33.8) |
|
|
EMT-intermediate |
1,833 (64.6) |
316 (65.0) |
1,517 (64.6) |
|
|
EMT-basic |
44 (1.6) |
7 (1.4) |
37 (1.6) |
|
|
ED disposition |
|
|
|
< 0.001 |
|
Discharge |
1,536 (54.2) |
116 (23.9) |
1,420 (60.4) |
|
|
Admission |
1,235 (43.5) |
342 (70.4) |
893 (38.0) |
|
|
Interhospital transfer |
50 (1.8) |
20 (4.1) |
30 (1.3) |
|
|
Death |
15 (0.5) |
8 (1.6) |
7 (0.3) |
|
|
Final diagnosis (stroke)a)
|
570 (20.1) |
107 (22.0) |
463 (19.7) |
0.246 |
|
Ischemic stroke |
402 (14.2) |
59 (12.1) |
343 (14.6) |
0.158 |
|
Hemorrhagic stroke |
176 (6.2) |
51 (10.5) |
125 (5.3) |
< 0.001 |
Table 2.Performance analysis of EMS-assessed stroke recognition for three scenarios
Table 2.
|
Parameter |
Scenario 1 (conservative approach)
|
Scenario 2 (real-world approach)
|
Scenario 3 (contrapositive approach)
|
|
Positive |
Negative |
Total |
Positive |
Negative |
Total |
Positive |
Negative |
Total |
|
Stroke |
368 |
95 |
463 |
475 |
95 |
570 |
368 |
202 |
570 |
|
Stroke mimic |
184 |
1,703 |
1,887 |
563 |
1,703 |
2,266 |
184 |
2,082 |
2,266 |
|
Total |
552 |
1,798 |
2,350 |
1,038 |
1,798 |
2,836 |
552 |
2,284 |
2,836 |
|
Test characteristic |
|
|
|
|
|
Sensitivity (%) |
79.5 (75.5–83.1) |
83.3 (80.0–86.3) |
64.6 (60.5–68.5) |
|
|
Specificity (%) |
90.2 (88.8–91.6) |
75.2 (73.3–76.9) |
91.9 (90.7–93.0) |
|
|
Positive likelihood ratio |
8.15 (7.05–9.42) |
3.35 (3.09–3.64) |
7.95 (6.83–9.25) |
|
|
Negative likelihood ratio |
0.227 (0.190–0.272) |
0.222 (0.184–0.267) |
0.386 (0.345–0.431) |
|
|
Positive predictive value (%) |
20.1 (17.9–22.6) |
9.4 (8.7–10.1) |
19.7 (17.4–22.2) |
|
|
Negative predictive value (%) |
99.3 (99.2–99.4) |
99.3 (99.2–99.4) |
98.8 (98.7–98.9) |
|
|
AUC |
0.849 (0.829–0.868) |
0.792 (0.775–0.810) |
0.782 (0.762–0.803) |
Table 3.Prehospital factors associated with stroke mimic in the uncooperative group
Table 3.
|
False-positive acute stroke |
Odds ratio (95% CI) |
Adjusted odds ratioa) (95% CI) |
|
Age (yr) |
0.99 (0.98–1.00) |
- |
|
Male sex |
1.05 (0.68–1.61) |
- |
|
Chief complaint at EMS arrival |
|
|
|
Dizziness |
2.29 (0.28–18.48) |
2.16 (0.27–17.61) |
|
Altered mental status |
1.10 (0.66–1.84) |
1.18 (0.70–1.98) |
|
Seizure |
3.56 (1.25–10.12) |
3.27 (1.13–9.44) |
|
Loss of consciousness |
0.73 (0.25–2.08) |
0.74 (0.26–2.12) |
|
Weakness |
|
|
|
Upper extremity |
0.09 (0.02–0.45) |
0.10 (0.02–0.49) |
|
Lower extremity |
0.28 (0.02–4.52) |
0.29 (0.02–4.67) |
|
Headache |
0.18 (0.03–1.12) |
0.17 (0.03–1.05) |
|
Dysarthria |
0.28 (0.06–1.39) |
0.26 (0.05–1.34) |
|
Clear first abnormal time |
2.26 (1.46–3.50) |
2.33 (1.40–3.61) |
|
Activity at onset |
0.88 (0.61–1.26) |
0.88 (0.61–1.28) |
|
Medical history |
|
|
|
Hypertension |
0.68 (0.44–1.05) |
0.72 (0.46–1.14) |
|
Diabetes |
1.52 (0.89–2.60) |
1.60 (0.93–2.76) |
|
Stroke |
0.78 (0.46–1.32) |
0.83 (0.49–1.42) |
|
Cardiovascular disease |
0.72 (0.35–1.50) |
0.79 (0.38–1.65) |
|
Malignancy |
4.09 (1.24–13.50) |
4.36 (1.31–14.50) |
|
Prehospital mental statusb)
|
1.11 (0.60–2.05) |
1.09 (0.59–2.02) |
|
Prehospital vitals |
|
|
|
Systolic blood pressure < 90 mmHg |
14.28 (1.95–104.86) |
15.89 (2.16–117.07) |
|
Body temperature > 38°C |
0.87 (0.38–2.00) |
0.86 (0.37–1.96) |
|
Blood glucose test check (yes) |
0.58 (0.33–1.04) |
0.57 (0.32–1.02) |
|
Prehospital provider level |
0.84 (0.55–1.31) |
0.85 (0.55–1.31) |
Table 4.Alternative clinical diagnoses (based on the final hospital ICD diagnosis code) for nonstroke patients (false positive) in the uncooperative group
Table 4.
|
Alternative diagnoses of nonstroke (false positive) patients in the uncooperative group |
Value (n=378) |
|
Symptoms and signs (not elsewhere classified) |
85 (22.5) |
|
Convulsions (not elsewhere classified, R56) |
47 (12.4) |
|
Shock (not elsewhere classified, R58) |
25 (6.6) |
|
Syncope and collapse (R55) |
13 (3.4) |
|
Infectious diseases of a particular system |
66 (17.5) |
|
URI, influenza, and pneumonia (J00–J06, J09–J18) |
31 (8.2) |
|
Acute pyelonephritis and urinary tract infection (N10, N39) |
16 (4.2) |
|
Other sepsis (A41) |
12 (3.2) |
|
Bacterial meningitis and encephalitis (G00, G04) |
4 (1.1) |
|
Infectious gastroenteritis and colitis (A09) |
2 (0.5) |
|
Other local infections of skin and subcutaneous tissue (L08) |
1 (0.3) |
|
Mental and behavioral disorders |
52 (13.8) |
|
Dementia, delirium, and other mental disorders (F01–F09) |
29 (7.7) |
|
Alcohol, sedative, hypnotic, or anxiolytic-related disorders (F10–F19) |
18 (4.8) |
|
Mood disorders and nonpsychotic mental disorders (F30–F48) |
5 (1.3) |
|
Diabetes mellitus and metabolic complications (E10–E13) |
35 (9.3) |
|
Diabetes mellitus with hypoglycemia |
27 (7.1) |
|
Hyperglycemic-related metabolic complications |
8 (2.1) |
|
Injury, poisoning, and other effects of external causes |
31 (8.2) |
|
Injuries to the head (S00–S09) |
16 (4.2) |
|
Poisoning and toxic effects of substances (T36–T65) |
11 (2.9) |
|
Heat-related conditions (T67–T69) |
4 (1.1) |
|
Episodic and paroxysmal disorders of the nervous systems |
27 (7.1) |
|
Epilepsy and recurrent seizures (G40) |
13 (3.4) |
|
Transient cerebral ischemic attacks and related syndromes (G45) |
12 (3.2) |
|
Migraine and other headache syndromes (G43, G44) |
2 (0.5) |
|
Diseases of the liver and biliary tract |
26 (6.9) |
|
Diseases of liver (K70–K77) |
23 (6.1) |
|
Other diseases of the biliary tract (K83) |
3 (0.8) |
|
Malignant neoplasms |
20 (5.3) |
|
Malignant neoplasms of particular systems (C00–C72) |
16 (4.2) |
|
Secondary malignant neoplasms (C76–C80) |
4 (1.1) |
|
Acute kidney failure and chronic kidney disease (N17–N19) |
15 (4.0) |
|
Diseases of the circulatory system |
11 (2.9) |
|
Cardiac arrest and dysrhythmia (I44–I49) |
5 (1.3) |
|
Heart failure (I50) |
3 (0.8) |
|
Aortic aneurysm and dissection (I71) |
2 (0.5) |
|
Acute myocardial infarction (I21) |
1 (0.3) |
|
Respiratory noninfectious diseases |
10 (2.6) |
|
Chronic lower respiratory diseases (J40–J47) |
5 (1.3) |
|
Other diseases of the pleura (J90–J94) |
5 (1.3) |
REFERENCES
- 1. Sacco RL, Kasner SE, Broderick JP, et al. An updated definition of stroke for the 21st century: a statement for healthcare professionals from the American Heart Association/American Stroke Association. Stroke 2013;44:2064-89.
- 2. Tsao CW, Aday AW, Almarzooq ZI, et al. Heart disease and stroke statistics: 2022 update: a report from the American Heart Association. Circulation 2022;145:e153-639.
- 3. Kim JY, Kang K, Kang J, et al. Executive summary of stroke statistics in Korea 2018: a report from the Epidemiology Research Council of the Korean Stroke Society. J Stroke 2019;21:42-59.
- 4. GBD 2019 Stroke Collaborators. Global, regional, and national burden of stroke and its risk factors, 1990-2019: a systematic analysis for the Global Burden of Disease Study 2019. Lancet Neurol 2021;20:795-820.
- 5. Ziaeian B, Xu H, Matsouaka RA, et al. US surveillance of acute ischemic stroke patient characteristics, care quality, and outcomes for 2019. Stroke 2022;53:3386-93.
- 6. Church G, Ali A, Smith CL, Broom D, Sage K. Examining clinical practice guidelines for exercise and physical activity as part of rehabilitation for people with stroke: a systematic review. Int J Environ Res Public Health 2022;19:1707.
- 7. Mochari-Greenberger H, Xian Y, Hellkamp AS, et al. Racial/ethnic and sex differences in emergency medical services transport among hospitalized US stroke patients: analysis of the national Get With the Guidelines-Stroke registry. J Am Heart Assoc 2015;4:e002099.
- 8. Appelros P, Jonsson F, Asberg S, et al. Trends in stroke treatment and outcome between 1995 and 2010: observations from Riks-Stroke, the Swedish stroke register. Cerebrovasc Dis 2014;37:22-9.
- 9. Boulanger JM, Lindsay MP, Gubitz G, et al. Canadian stroke best practice recommendations for acute stroke management: prehospital, emergency department, and acute inpatient stroke care, 6th edition, update 2018. Int J Stroke 2018;13:949-84.
- 10. Uwishema O, Berjaoui C, Correia IF, et al. Current management of acute ischemic stroke in Africa: a review of the literature. Eur J Neurol 2022;29:3460-5.
- 11. Puolakka T, Strbian D, Harve H, Kuisma M, Lindsberg PJ. Prehospital phase of the stroke chain of survival: a prospective observational study. J Am Heart Assoc 2016;5:e002808.
- 12. Ramos A, Guerrero WR, Perez de la Ossa N. Prehospital stroke triage. Neurology 2021;97(20 Suppl 2):S25-33.
- 13. Shkirkova K, Schuberg S, Balouzian E, et al. Paramedic global impression of change during prehospital evaluation and transport for acute stroke. Stroke 2020;51:784-91.
- 14. Larsen K, Jaeger HS, Hov MR, et al. Streamlining acute stroke care by introducing National Institutes of Health Stroke Scale in the emergency medical services: a prospective cohort study. Stroke 2022;53:2050-7.
- 15. Zhelev Z, Walker G, Henschke N, Fridhandler J, Yip S. Prehospital stroke scales as screening tools for early identification of stroke and transient ischemic attack. Cochrane Database Syst Rev 2019;4:CD011427.
- 16. Antipova D, Eadie L, Macaden A, Wilson P. Diagnostic accuracy of clinical tools for assessment of acute stroke: a systematic review. BMC Emerg Med 2019;19:49.
- 17. Powers WJ, Rabinstein AA, Ackerson T, et al. Guidelines for the early management of patients with acute ischemic stroke: 2019 update to the 2018 guidelines for the early management of acute ischemic stroke: a guideline for healthcare professionals from the American Heart Association/American Stroke Association. Stroke 2019;50:e344-418.
- 18. Ramalle-Gomara E, Ruiz E, Serrano M, Bartulos M, Gonzalez MA, Matute B. Validity of discharge diagnoses in the surveillance of stroke. Neuroepidemiology 2013;41:185-8.
- 19. Uysal S. ICD-10-CM diagnosis coding for neuropsychological assessment. Arch Clin Neuropsychol 2019;34:721-30.
- 20. Ghadimi N, Hanifi N, Dinmohammadi M. Factors affecting pre-hospital and in-hospital delays in treatment of ischemic stroke: a prospective cohort study. Arch Acad Emerg Med 2021;9:e52.
- 21. Sharma M, Helzner E, Sinert R, Levine SR, Brandler ES. Patient characteristics affecting stroke identification by emergency medical service providers in Brooklyn, New York. Intern Emerg Med 2016;11:229-36.
- 22. Jones SP, Bray JE, Gibson JM, et al. Characteristics of patients who had a stroke not initially identified during emergency prehospital assessment: a systematic review. Emerg Med J 2021;38:387-93.
- 23. Oostema JA, Konen J, Chassee T, Nasiri M, Reeves MJ. Clinical predictors of accurate prehospital stroke recognition. Stroke 2015;46:1513-7.
- 24. Andersson E, Bohlin L, Herlitz J, Sundler AJ, Fekete Z, Andersson Hagiwara M. Prehospital identification of patients with a final hospital diagnosis of stroke. Prehosp Disaster Med 2018;33:63-70.
- 25. Leibson CL, Naessens JM, Brown RD, Whisnant JP. Accuracy of hospital discharge abstracts for identifying stroke. Stroke 1994;25:2348-55.
- 26. McMullan JT, Katz B, Broderick J, Schmit P, Sucharew H, Adeoye O. Prospective prehospital evaluation of the Cincinnati Stroke Triage Assessment Tool. Prehosp Emerg Care 2017;21:481-8.
- 27. Wolf ME, Chatzikonstantinou A, Gruttner J, et al. Is it acute stroke or not?: a prospective observational study from a multidisciplinary emergency department. Eur Neurol 2016;75:170-7.
- 28. Brandler ES, Sharma M, McCullough F, et al. Prehospital stroke identification: factors associated with diagnostic accuracy. J Stroke Cerebrovasc Dis 2015;24:2161-6.
- 29. Asimos AW, Ward S, Brice JH, Rosamond WD, Goldstein LB, Studnek J. Out-of-hospital stroke screen accuracy in a state with an emergency medical services protocol for routing patients to acute stroke centers. Ann Emerg Med 2014;64:509-15.
- 30. Aroor S, Singh R, Goldstein LB. BE-FAST (balance, eyes, face, arm, speech, time): reducing the proportion of strokes missed using the FAST mnemonic. Stroke 2017;48:479-81.
- 31. Goyal N, Tsivgoulis G, Male S, et al. FABS: an intuitive tool for screening of stroke mimics in the emergency department. Stroke 2016;47:2216-20.
- 32. Tu TM, Tan GZ, Saffari SE, et al. External validation of stroke mimic prediction scales in the emergency department. BMC Neurol 2020;20:269.
- 33. Walsh KB. Non-invasive sensor technology for prehospital stroke diagnosis: current status and future directions. Int J Stroke 2019;14:592-602.
- 34. Shuaib A, Amlani S, Kalashyan H, et al. Mobile stroke unit triage of patients with a suspected stroke: a novel solution to reducing suspected stroke admissions in busy emergency departments. BMJ Innov 2018;4:54-9.
- 35. Rogers H, Chalil Madathil K, Joseph A, et al. Task, usability, and error analyses of ambulance-based telemedicine for stroke care. IISE Trans Healthc Syst Eng 2021;11:192-208.
, Sung Wook Song2
