BACKGROUND: Snakebite envenoming is a neglected tropical disease that kills an estimated 81,000 to 138,000 people and disables another 400,000 globally every year. The World Health Organization aims to halve this burden by 2030. To achieve this ambitious goal, we need to close the data gap in snake ecology and snakebite epidemiology and give healthcare providers up-to-date knowledge and access to better diagnostic tools. An essential first step is to improve the capacity to identify biting snakes taxonomically. The existence of AI-based identification tools for other animals offers an innovative opportunity to apply machine learning to snake identification and snakebite envenoming, a life-threatening situation. METHODOLOGY: We developed an AI model based on Vision Transformer, a recent neural network architecture, and a comprehensive snake photo dataset of 386,006 training photos covering 198 venomous and 574 non-venomous snake species from 188 countries. We gathered photos from online biodiversity platforms (iNaturalist and HerpMapper) and a photo-sharing site (Flickr). PRINCIPAL FINDINGS: The model macro-averaged F1 score, which reflects the species-wise performance as averaging performance for each species, is 92.2%. The accuracy on a species and genus level is 96.0% and 99.0%, respectively. The average accuracy per country is 94.2%. The model accurately classifies selected venomous and non-venomous lookalike species from Southeast Asia and sub-Saharan Africa. CONCLUSIONS: To our knowledge, this model's taxonomic and geographic coverage and performance are unprecedented. This model could provide high-speed and low-cost snake identification to support snakebite victims and healthcare providers in low-resource settings, as well as zoologists, conservationists, and nature lovers from across the world.

Department of Biological Sciences Florida Gulf Coast University Fort Myers Florida United States of America

Department of Community Health and Medicine Faculty of Medicine University of Geneva Geneva Switzerland

Department of Cybernetics Faculty of Applied Sciences University of West Bohemia Pilsen Czechia

Division of Tropical and Humanitarian Medicine Geneva University Hospitals and University of Geneva Geneva Switzerland

Institute of Global Health Department of Community Health and Medicine Faculty of Medicine University of Geneva Geneva Switzerland

Médecins Sans Frontières Doctors Without Borders Geneva Switzerland

PiVa AI s r o Plzeň Czechia

Zobrazit více v PubMed

Williams DJ, Faiz MA, Abela-Ridder B, Ainsworth S, Bulfone TC, Nickerson AD, et al.. Strategy for a globally coordinated response to a priority neglected tropical disease: Snakebite envenoming. PLoS Negl Trop Dis. 2019;13(2):e0007059. doi: 10.1371/journal.pntd.0007059 PubMed DOI PMC

Williams HF, Layfield HJ, Vallance T, Patel K, Bicknell AB, Trim SA, et al.. The urgent need to develop novel strategies for the diagnosis and treatment of snakebites. Toxins. 2019;11(6):363. doi: 10.3390/toxins11060363 PubMed DOI PMC

Laxme RS, Khochare S, de Souza HF, Ahuja B, Suranse V, Martin G, et al.. Beyond the ‘big four’: Venom profiling of the medically important yet neglected Indian snakes reveals disturbing antivenom deficiencies. PLoS Negl Trop Dis. 2019;13(12):e0007899. doi: 10.1371/journal.pntd.0007899 PubMed DOI PMC

Uetz P, Freed P, Hošek J. The Reptile Database. 2020. [cited 2022 February 02]. Available from: http://reptile-database.reptarium.cz.

Ariaratnam CA, Sheriff MHR, Arambepola C, Theakston RDG, Warrell DA. Syndromic approach to treatment of snake bite in Sri Lanka based on results of a prospective national hospital-based survey of patients envenomed by identified snakes. Am J Tropl Med Hyg. 2009;81(4):725–31. doi: 10.4269/ajtmh.2009.09-0225 PubMed DOI

Williams DJ, Habib AG, Warrell DA. Clinical studies of the effectiveness and safety of antivenoms. Toxicon. 2018;150:1–10. doi: 10.1016/j.toxicon.2018.05.001 PubMed DOI

Ruiz de Castañeda R, Durso AM, Ray N, Fernández JL, Williams DJ, Alcoba G, et al.. Snakebite and snake identification: empowering neglected communities and health-care providers with AI. The Lancet Digital Health. 2019;1(5):e202–e3. doi: 10.1016/S2589-7500(19)30086-X PubMed DOI

Ralph R, Sharma SK, Faiz MA, Ribeiro I, Rijal S, Chappuis F, et al.. The timing is right to end snakebite deaths in South Asia. BMJ. 2019;364. doi: 10.1136/bmj.k5317 PubMed DOI PMC

Bolon I, Durso AM, Botero Mesa S, Ray N, Alcoba G, Chappuis F, et al.. Identifying the snake: First scoping review on practices of communities and healthcare providers confronted with snakebite across the world. PLoS ONE. 2020;15(3):e0229989. doi: 10.1371/journal.pone.0229989 PubMed DOI PMC

Knudsen C, Jürgensen JA, Føns S, Haack AM, Friis RU, Dam SH, et al.. Snakebite Envenoming Diagnosis and Diagnostics. Front Immunol. 2021;12:1268. doi: 10.3389/fimmu.2021.661457 PubMed DOI PMC

Sharma SK, Alirol E, Ghimire A, Shrestha S, Jha R, Parajuli SB, et al.. Acute severe anaphylaxis in Nepali patients with neurotoxic snakebite envenoming treated with the VINS polyvalent antivenom. J Trop Med. 2019;2019. doi: 10.1155/2019/2689171 PubMed DOI PMC

Inthanomchanh V, Reyer JA, Blessmen J, Phrasisombath K, Yamamoto E, Hamajima N. Assessment of knowledge about snakebite management amongst healthcare providers in the provincial and two district hospitals in Savannakhet Province, Lao PDR. Nagoya J Med Sci. 2017;79(3):299. doi: 10.18999/nagjms.79.3.299 PubMed DOI PMC

Michael GC, Grema BA, Aliyu I, Alhaji MA, Lawal TO, Ibrahim H, et al.. Knowledge of venomous snakes, snakebite first aid, treatment, and prevention among clinicians in northern Nigeria: a cross-sectional multicentre study. Trans R Soc Trop Med Hyg. 2018;112(2):47–56. doi: 10.1093/trstmh/try028 PubMed DOI

WHO. Regional Office for South-East Asia, Guidelines for the management of snakebite 2nd edition 2016. [cited 2022 February 02]. Available from: https://apps.who.int/iris/handle/10665/249547.

Weinstein BG. A computer vision for animal ecology. J Anim. Ecol. 2018;87(3):533–45. doi: 10.1111/1365-2656.12780 PubMed DOI

Amir A, Zahri NAH, Yaakob N, Ahmad RB. Image classification for snake species using machine learning techniques. International Conference on Computational Intelligence in Information System; 2016: Springer; Cham; 2016. (pp. 52–59)

Patel A, Cheung L, Khatod N, Matijosaitiene I, Arteaga A, Gilkey JW. Revealing the unknown: real-time recognition of Galápagos snake species using deep learning. Animals. 2020;10(5):806. doi: 10.3390/ani10050806 PubMed DOI PMC

Rajabizadeh M, Rezghi M. A comparative study on image-based snake identification using machine learning. Sci Rep. 2021;11(1):1–16. https://www.nature.com/articles/s41598-021-96031-1 PubMed PMC

Geneviève LD, Ray N, Chappuis F, Alcoba G, Mondardini MR, Bolon I, et al.. Participatory approaches and open data on venomous snakes: A neglected opportunity in the global snakebite crisis? PLoS Negl Trop Dis. 2018;12(3):e0006162. doi: 10.1371/journal.pntd.0006162 PubMed DOI PMC

Durso AM, Ruiz de Castañeda R, Montalcini C, Mondardini MR, Fernandez-Marques JL, Grey F, et al.. Citizen science and online data: Opportunities and challenges for snake ecology and action against snakebite. Toxicon: X. 2021:100071. doi: 10.1016/j.toxcx.2021.100071 PubMed DOI PMC

Krause J, Sapp B, Howard A, Zhou H, Toshev A, Duerig T, et al.. The unreasonable effectiveness of noisy data for fine-grained recognition. European Conference on Computer Vision; 2016: Springer. https://link.springer.com/chapter/10.1007/978-3-319-46487-9_19 DOI

Sun C, Shrivastava A, Singh S, Gupta A. Revisiting unreasonable effectiveness of data in deep learning era. Proceedings of the IEEE international conference on computer vision; 2017. https://openaccess.thecvf.com/content_ICCV_2017/papers/Sun_Revisiting_Unreasonable_Effectiveness_ICCV_2017_paper.pdf

Joly A, Goëau H, Kahl S, Picek L, Lorieul T, Cole E, et al.. Overview of lifeclef 2021: An evaluation of machine-learning based species identification and species distribution prediction. International Conference of the Cross-Language Evaluation Forum for European Languages; 2021: Springer.

WHO. Snakebite Information and Data Platform 2021. [cited 2022 February 02]. Available from: https://www.who.int/teams/control-of-neglected-tropical-diseases/snakebite-envenoming/snakebite-information-and-data-platform/overview#tab=tab_1.

Roll U, Feldman A, Novosolov M, Allison A, Bauer AM, Bernard R, et al.. The global distribution of tetrapods reveals a need for targeted reptile conservation. Nat Ecol Evol. 2017;1(11):1677–82. doi: 10.1038/s41559-017-0332-2 PubMed DOI

South A. rworldmap: A New R package for Mapping Global Data. R J. 2011; Vol. 3(1): 35–43.

Hierink F, Bolon I, Durso AM, Ruiz de Castañeda R, Zambrana-Torrelio C, Eskew EA, et al.. Forty-four years of global trade in CITES-listed snakes: Trends and implications for conservation and public health. Biol Conserv. 2020;248:108601. 10.1016/j.biocon.2020.108601 DOI

Warrick BJ, Boyer LV, Seifert SA. Non-native (exotic) snake envenomations in the US, 2005–2011. Toxins. 2014;6(10):2899–911. doi: 10.3390/toxins6102899 PubMed DOI PMC

Kraus F. Alien reptiles and amphibians: a scientific compendium and analysis: Springer Science & Business Media; 2008.

Dosovitskiy A, Beyer L, Kolesnikov A, Weissenborn D, Zhai X, Unterthiner T, et al.. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:201011929. 2020. [cited 2022 February 02]. Available from: https://arxiv.org/abs/2010.11929.

Lin T-Y, Goyal P, Girshick R, He K, Dollár P. Focal loss for dense object detection. Proceedings of the IEEE international conference on computer vision; 2017. https://openaccess.thecvf.com/content_ICCV_2017/papers/Lin_Focal_Loss_for_ICCV_2017_paper.pdf PubMed

Smith LN, Topin N. Super-convergence: Very fast training of neural networks using large learning rates. Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications; 2019: International Society for Optics and Photonics.

Buslaev A, Iglovikov VI, Khvedchenya E, Parinov A, Druzhinin M, Kalinin AA. Albumentations: fast and flexible image augmentations. Information. 2020;11(2):125.

Durso AM, Moorthy GK, Mohanty SP, Bolon I, Salathé M, Ruiz De Castañeda R. Supervised learning computer vision benchmark for snake species identification from photographs: Implications for herpetology and global health. Frontiers in Artificial Intelligence. 2021;4:17. doi: 10.3389/frai.2021.582110 PubMed DOI PMC

Farooq H, Uetz P. Identifying Australian snakes by color patterns. Vertebr Zool. 2020;70(3):473–82.

Bustard HR. Defensive display behavior in the bandy-bandy, Vermicella annulata (Serpentes: Elapidae). Herpetologica. 1969;25(4):319–20.

Durso AM, Bolon I, Kleinhesselink A, Mondardini M, Fernandez-Marquez J, Gutsche-Jones F, et al.. Crowdsourcing snake identification with online communities of professional herpetologists and avocational snake enthusiasts. R Soc Open Sci. 2021;8(1):201273. doi: 10.1098/rsos.201273 PubMed DOI PMC

Rathnayaka RN, Ranathunga PN, Kularatne S. Paediatric cases of Ceylon krait (Bungarus ceylonicus) bites and some similar looking non-venomous snakebites in Sri Lanka: Misidentification and antivenom administration. Toxicon. 2021;198:143–50. doi: 10.1016/j.toxicon.2021.04.019 PubMed DOI

Pandey DP, Pandey GS, Devkota K, Goode M. Public perceptions of snakes and snakebite management: implications for conservation and human health in southern Nepal. J Ethnobiol Ethnomed. 2016;12(1):1–25. doi: 10.1186/s13002-016-0092-0 PubMed DOI PMC

Picek L, Durso A, Ruiz de Castañeda R, Bolon I. Overview of SnakeCLEF 2021: Automatic snake species identification with country-level focus. Working Notes of CLEF. 2021.

Picek L, Šulc M, Matas J, Heilmann-Clausen J, Jeppesen TS, Lind E. Automatic Fungi Recognition: Deep Learning Meets Mycology. Sensors. 2022;22(2):633. doi: 10.3390/s22020633 PubMed DOI PMC

Maduwage K, Karunathilake P, Gutiérrez JM. Web-based snake identification service: A successful model of snake identification in Sri Lanka. Toxicon. 2022;205:24–30. doi: 10.1016/j.toxicon.2021.11.007 PubMed DOI

Ruiz de Castañeda R, Grey F, Williams DJ. Citizen surveys could map snakebite risks. Nature. 2019;571(7766):478–. 10.1038/d41586-019-02247-7 PubMed DOI

We are social 2021. [cited 2022 February 02]. Available from: https://wearesocial.com/uk/blog/2021/01/digital-2021-uk/.

Bolon I, Finat M, Herrera M, Nickerson A, Grace D, Schütte S, et al.. Snakebite in domestic animals: First global scoping review. Prev Vet Med. 2019; 170: 104729. doi: 10.1016/j.prevetmed.2019.104729 PubMed DOI