As one of the leading causes of accidental death, foreign body airway obstruction (FBAO) is recognized as a major public health problem worldwide [1,2]. Severe airway obstruction induces rapid progression of hypoxia, and longer obstruction time represents a significant risk factor for death or vegetative state [2]. Early recognition of the problem and removal of the foreign body by a witness of the emergency at the accident scene is critical to prevent loss of life and is associated with better neurological outcomes [3,4]. Although severe FBAO is commonly witnessed by other people [5], bystanders rarely attempt first aid to remove foreign bodies [2,6]. This emphasizes the importance of the wide distribution of adequate knowledge and skills of first aid for FBAO in the community.

While instructor-led first aid courses constitute the best practice for acquiring and maintaining life-saving competencies, considering the limited availability of such training [7] and the progressive digitalization of society, internet-based self-learning represents an important option for first aid education. Internet-based self-learning can be used to supplement instructor-led training or may serve as a better-than-nothing solution for people who cannot attend a live course. Despite the potential utility, including the opportunity to reach a huge number of learners, online resources may not adhere to relevant guidelines on first aid and may potentially contain mistakes that represent the risk of harm [8,9]. In fact, little is known about the quality of open information about first aid for FBAO on the internet. Accordingly, our study was carried out to assess the accuracy of online instruction in first aid for FBAO. Also, it was assumed that novel artificial intelligence (AI)-powered chatbots may perform quality checks automatically. Consequently, our secondary aim was to test the performance of two cutting-edge chatbots in this regard.

In October 2023, Google was prompted with the phrase “what to do choking” on a desktop computer with a search location set to London, UK. The search results ordered by Google according to relevance were reviewed consecutively, and links to the first 35 websites that provided textual instructions on first aid for FBAO were collected for analysis. Manual evaluation of the instructions in terms of completeness and correctness was performed independently by the authors (physicians with experience in clinical practice and education in emergency medicine) using the pertinent 10-item section of the European Resuscitation Council Research NET checklist for quality appraisal of educational resources on adult basic life support [10]. Each checklist item was rated as “true (completely satisfied, 2 points),” “partially true (satisfied in part, 1 point),” or “not true (not satisfied or absent, 0 points).” Results of evaluations were compared, and any discrepancies between the reviewers’ ratings were resolved through discussion. Based on evaluation, a sum score was calculated for each website. For automated evaluation, inquiries were submitted to Bard (Google LLC) and Bing (Microsoft Corp) AI chatbots using the predeveloped prompt (Supplementary Material 1) to evaluate website content based on the same checklist and rating criteria. Results of the manual and the chatbot-run evaluations were compared.

Of the 35 studied websites, 15 (42.9%) were owned by commercial entities, 10 (28.6%) by charities or other nonprofit entities, six (17.1%) by government organizations, and four (11.4%) by academic organizations. Based on the manual evaluation, the sum score varied from 3 to 16 of the possible 20 points (median, 11; interquartile range [IQR], 8–13). Essential advice on first aid, including instructions to immediately call for help in cases of severe FBAO, to encourage coughing when the victim is conscious and able to cough, and to start cardiopulmonary resuscitation in victims that are unresponsive or breathing abnormally, was commonly missing or incomplete (Dataset [11]) (Table 1). Description of back blows and abdominal thrusts on many occasions contained insufficient details on how to correctly perform the maneuvers. Most websites omitted guideline-recommended instructions [12] to avoid blind finger sweeps and to have victims examined by a qualified healthcare practitioner following successful application of abdominal thrusts or chest compressions. Further, some websites contained guideline-discordant and potentially harmful advice—namely, to perform abdominal thrusts on unresponsive victims or to bring victims with FBAO to the emergency department instead of calling for emergency medical services (Dataset [11]). Among the websites with relatively low scores (≤10 points), there were reputable and well-trusted sources, such as the British Red Cross and WebMD, which are consistently highly ranked by search engines and usually presented among the top search results.

Evaluation of chatbot performance in terms of their ability to assess the quality of website content showed that these AI tools were generally inaccurate (Dataset [11]) (Table 1). Bard persistently assigned the highest possible rating for all checklist items (median, 20; IQR, 20–20). In contrast, Bing’s rating (median, 14; IQR, 10.3–15.8; P=0.029) was variable but much higher than that of manual assessment, with which it was not correlated. Further, Bing failed to produce any rating for three websites. Overall, in their current iteration, these chatbots cannot be recommended for automated quality assessment for emergency choking first aid. The AI chatbots represent a sort of “black box” technology, and it is not clear how these tools generated specific scores for website content. Further research with the participation of experts in AI is necessary to expose the reasons behind the chatbots’ variable and inaccurate assessments, as well as to better understand future implications for the use of AI chatbots in automated quality appraisal of public digital resources on first aid.

In summary, this explorative study confirms the poor quality of instructions in first aid for FBAO presented on highly ranked, openly available websites. This finding suggests the ongoing dissemination of inaccurate, incomplete, and occasionally harmful advice among a vast number of people around the globe. While true that the owners of pertinent websites should comprehend their responsibilities when publishing content on first aid online and guarantee its full compliance with the recommended best practices, it also stands that relevant healthcare stakeholders, including the World Health Organization and the International Liaison Committee on Resuscitation, are supposed to recognize misinformation on first aid as a public health problem and strive toward development of uniform procedures for infodemiological surveillance and quality assurance of such information. Emphasizing the issue of misinformation as part of the international consensus on first aid and the creation of a corresponding expert task force could be reasonable first steps to raise awareness of the problem and to potentiate pertinent research.