BACKGROUND: Our purpose is to assess epidemiological agent-based models-or ABMs-of the SARS-CoV-2 pandemic methodologically. The rapid spread of the outbreak requires fast-paced decision-making regarding mitigation measures. However, the evidence for the efficacy of non-pharmaceutical interventions such as imposed social distancing and school or workplace closures is scarce: few observational studies use quasi-experimental research designs, and conducting randomized controlled trials seems infeasible. Additionally, evidence from the previous coronavirus outbreaks of SARS and MERS lacks external validity, given the significant differences in contagiousness of those pathogens relative to SARS-CoV-2. To address the pressing policy questions that have emerged as a result of COVID-19, epidemiologists have produced numerous models that range from simple compartmental models to highly advanced agent-based models. These models have been criticized for involving simplifications and lacking empirical support for their assumptions. METHODS: To address these voices and methodologically appraise epidemiological ABMs, we consider AceMod (the model of the COVID-19 epidemic in Australia) as a case study of the modelling practice. RESULTS: Our example shows that, although epidemiological ABMs involve simplifications of various sorts, the key characteristics of social interactions and the spread of SARS-CoV-2 are represented sufficiently accurately. This is the case because these modellers treat empirical results as inputs for constructing modelling assumptions and rules that the agents follow; and they use calibration to assert the adequacy to benchmark variables. CONCLUSIONS: Given this, we claim that the best epidemiological ABMs are models of actual mechanisms and deliver both mechanistic and difference-making evidence. Consequently, they may also adequately describe the effects of possible interventions. Finally, we discuss the limitations of ABMs and put forward policy recommendations.

Department of Philosophy and Religious Studies Faculty of Arts Charles University Prague Prague Czech Republic

Institute of Philosophy Jagiellonian University Kraków Poland

Interdisciplinary Centre for Ethics Jagiellonian University Kraków Poland

See more in PubMed

Wilder‐Smith A, Freedman DO. Isolation, quarantine, social distancing and community containment: pivotal role for old‐style public health measures in the novel coronavirus (2019‐nCoV) outbreak. J Travel Med. 2020;27(2):taaa020. 10.1093/jtm/taaa020. PubMed DOI PMC

Lewis D. Is the coronavirus airborne? Experts can't agree. Nature. 2020;580(7802):175‐175. 10.1038/d41586-020-00974-w. PubMed DOI

Li C, Ji F, Wang L, et al. Asymptomatic and human‐to‐human transmission of SARS‐CoV‐2 in a 2‐family cluster, Xuzhou, China. Emerg Infect Dis. 2020;26(7):1626‐1628. 10.3201/eid2607.200718. PubMed DOI PMC

Bai Y, Yao L, Wei T, et al. Presumed asymptomatic carrier transmission of COVID‐19. JAMA. 2020;323(14):1406. 10.1001/jama.2020.2565. PubMed DOI PMC

Pan A, Liu L, Wang C, et al. Association of Public Health Interventions with the epidemiology of the COVID‐19 outbreak in Wuhan, China. JAMA. 2020;323:1915. 10.1001/jama.2020.6130. PubMed DOI PMC

Railsback SF, Grimm V. Agent‐Based and Individual‐Based Modeling: A Practical Introduction. Princeton, NJ: Princeton University Press; 2012.

Wilensky U, Rand W. An Introduction to Agent‐Based Modeling: Modeling Natural, Social, and Engineered Complex Systems with NetLogo. Cambridge, MA: MIT Press; 2015.

Eisinger D, Thulke H‐H. Spatial pattern formation facilitates eradication of infectious diseases. J Appl Ecol. 2008;45(2):415‐423. 10.1111/j.1365-2664.2007.01439.x. PubMed DOI PMC

Squazzoni F, Polhill JG, Edmonds B, et al. Computational models that matter during a global pandemic outbreak: a call to action. J Artif Soc Soc Simul. 2020;23(2):10. 10.18564/jasss.4298. DOI

Sridhar D, Majumder MS. Modelling the pandemic. BMJ. 2020;369:m1567. 10.1136/BMJ.M1567. PubMed DOI

Ferguson NM, Laydon D, Nedjati‐Gilani G, et al. Report 9: impact of non‐pharmaceutical interventions (NPIs) to reduce COVID19 mortality and healthcare demand. 2020. https://www.imperial.ac.uk/media/imperial-college/medicine/mrc-gida/2020-03-16-COVID19-Report-9.pdf. PubMed PMC

OCEBM Levels of Evidence Working Group . The Oxford 2009 Levels of Evidence. Oxford: Oxford Centre for Evidence‐Based Medicine; 2009. https://www.cebm.net/2009/06/oxford-centre-evidence-based-medicine-levels-evidence-march-2009/.

National Institute for Health and Care Excellence . Developing NICE Guidelines: the Manual. London: National Institute for Health and Care Excellence; 2015. https://www.nice.org.uk/media/default/about/what-we-do/our-programmes/developing-nice-guidelines-the-manual.pdf. PubMed

Rocca E. The judgements that evidence‐based medicine adopts. J Eval Clin Pract. 2018;24(5):1184‐1190. 10.1111/jep.12994. PubMed DOI

Worrall J. Evidence: philosophy of science meets medicine. J Eval Clin Pract. 2010;16(2):356‐362. 10.1111/j.1365-2753.2010.01400.x. PubMed DOI

Williamson J. EBM+: increasing the systematic use of mechanistic evidence. Oral Presentations. Vol 24. London: BMJ Publishing Group; 2019:A13‐A14. 10.1136/bmjebm-2019-EBMLive.25. DOI

Parkkinen V‐P, Wallmann C, Wilde M, et al. Evaluating Evidence of Mechanisms in Medicine: Principles and Procedures. Berlin: Springer; 2018. PubMed

Clarke B, Gillies D, Illari P, Russo F, Williamson J. The evidence that evidence‐based medicine omits. Prev Med (Baltim). 2013;57(6):745‐747. 10.1016/J.YPMED.2012.10.020. PubMed DOI

Russo F, Williamson J. Interpreting causality in the health sciences. Int Stud Philos Sci. 2007;21(2):157‐170. 10.1080/02698590701498084. DOI

Marshall BDL, Galea S. Formalizing the role of agent‐based modeling in causal inference and epidemiology. Am J Epidemiol. 2015;181(2):92‐99. 10.1093/aje/kwu274. PubMed DOI PMC

Hernan MA. Invited commentary: agent‐based models for causal inference—reweighting data and theory in epidemiology. Am J Epidemiol. 2015;181(2):103‐105. 10.1093/aje/kwu272. PubMed DOI PMC

Clarke B, Gillies D, Illari P, Russo F, Williamson J. Mechanisms and the evidence hierarchy. Topoi. 2014;33(2):339‐360. 10.1007/s11245-013-9220-9. DOI

Yang Z, Zeng Z, Wang K, et al. Modified SEIR and AI prediction of the epidemics trend of COVID‐19 in China under public health interventions. J Thorac Dis. 2020;12(3):165‐174. 10.21037/jtd.2020.02.64. PubMed DOI PMC

Neher RA, Dyrdak R, Druelle V, Hodcroft EB, Albert J. Potential impact of seasonal forcing on a SARS‐CoV‐2 pandemic. Swiss Med Wkly. 2020;150(1112):w20224. 10.4414/smw.2020.20224. PubMed DOI

Giordano G, Blanchini F, Bruno R, et al. A SIDARTHE Model of COVID‐19 Epidemic in Italy. 2020. http://arxiv.org/abs/2003.09861. Accessed May 16, 2020

Acuna‐Zegarra MA, Comas‐Garcia A, Hernandez‐Vargas E, Santana‐Cibrian M, Velasco‐Hernandez JX. The SARS‐CoV‐2 Epidemic Outbreak: a Review of Plausible Scenarios of Containment and Mitigation for Mexico. medRxiv. 2020;2020:20046276. 10.1101/2020.03.28.20046276. DOI

Kissler SM, Tedijanto C, Lipsitch M, Grad Y. Social distancing strategies for curbing the COVID‐19 epidemic. medRxiv. 2020;2020:20041079. 10.1101/2020.03.22.20041079. DOI

Peng L, Yang W, Zhang D, et al. Epidemic Analysis of COVID‐19 in China by Dynamical Modeling. 2020. http://arxiv.org/abs/2002.06563. Accessed May 16, 2020.

Fu X, Ying Q, Zeng T, Long T, Wang Y. Simulating and forecasting the cumulative confirmed cases of SARS‐CoV‐2 in China by Boltzmann function‐based regression analyses. J Infect. 2020;80(5):578‐606. 10.1016/J.JINF.2020.02.019. PubMed DOI PMC

Tobías A. Evaluation of the lockdowns for the SARS‐CoV‐2 epidemic in Italy and Spain after one month follow up. Sci Total Environ. 2020;725:138539. 10.1016/J.SCITOTENV.2020.138539. PubMed DOI PMC

Chang SL, Harding N, Zachreson C, et al. Modelling Transmission and Control of the COVID‐19 Pandemic in Australia. 2020. http://arxiv.org/abs/2003.10218. Accessed May 16, 2020. PubMed PMC

Adam D. Special report: the simulations driving the world's response to COVID‐19. Nature. 2020;580(7803):316‐318. 10.1038/d41586-020-01003-6. PubMed DOI

Cliff OM, Harding N, Piraveenan M, Erten EY, Gambhir M, Prokopenko M. Investigating spatiotemporal dynamics and synchrony of influenza epidemics in Australia: an agent‐based modelling approach. Simul Model Pract Theory. 2018;87:412‐431. 10.1016/J.SIMPAT.2018.07.005. DOI

Cauchemez S, Bhattarai A, Marchbanks TL, et al. Role of social networks in shaping disease transmission during a community outbreak of 2009 H1N1 pandemic influenza. Proc Natl Acad Sci. 2011;108(7):2825‐2830. 10.1073/PNAS.1008895108. PubMed DOI PMC

Lai A, Bergna A, Acciarri C, Galli M, Zehender G. Early phylogenetic estimate of the effective reproduction number of SARS‐CoV‐2. J Med Virol. 2020;92(6):675‐679. 10.1002/jmv.25723. PubMed DOI PMC

Liu Y, Eggo RM, Kucharski AJ. Secondary attack rate and superspreading events for SARS‐CoV‐2. Lancet. 2020;395(10227):e47. 10.1016/S0140-6736(20)30462-1. PubMed DOI PMC

Illari PM, Williamson J. What is a mechanism? Thinking about mechanisms across the sciences. Eur J Philos Sci. 2012;2(1):119‐135. 10.1007/s13194-011-0038-2. DOI

Glennan S. The New Mechanical Philosophy. Oxford: Oxford University Press; 2017.

Glennan S, Illari PM. Varieties of mechanisms. In: Glennan S, Illari PM, eds. The Routledge Handbook of Mechanisms and Mechanical Philosophy. London: Routledge; 2018:91‐103.

Kelly MP, Kelly RS, Russo F. The integration of social, behavioral, and biological mechanisms in models of pathogenesis. Perspect Biol Med. 2014;57(3):308‐328. 10.1353/pbm.2014.0026. PubMed DOI

Portides D. Idealization and abstraction in scientific modeling. Synthese. 2018;1‐23. 10.1007/s11229-018-01919-7. PubMed DOI

Frieden TR, Lee CT. Identifying and interrupting Superspreading events—implications for control of severe acute respiratory syndrome coronavirus 2. Emerg Infect Dis. 2020;26(6):1059‐1066. 10.3201/eid2606.200495. PubMed DOI PMC

Lloyd‐Smith JO, Schreiber SJ, Kopp PE, Getz WM. Superspreading and the effect of individual variation on disease emergence. Nature. 2005;438(7066):355‐359. 10.1038/nature04153. PubMed DOI PMC

Wong G, Liu W, Liu Y, Zhou B, Bi Y, Gao GF. MERS, SARS, and Ebola: the role of super‐spreaders in infectious disease. Cell Host Microbe. 2015;18(4):398‐401. 10.1016/J.CHOM.2015.09.013. PubMed DOI PMC

Everhart JE, Kruszon‐Moran D, Perez‐Perez GI, Tralka TS, McQuillan G. Seroprevalence and ethnic differences in helicobacter pylori infection among adults in the United States. J Infect Dis. 2000;181(4):1359‐1363. 10.1086/315384. PubMed DOI

Lazarus R, Vercelli D, Palmer LJ, et al. Single nucleotide polymorphisms in innate immunity genes: abundant variation and potential role in complex human disease. Immunol Rev. 2002;190(1):9‐25. 10.1034/j.1600-065X.2002.19002.x. PubMed DOI

Delgado JC, Baena A, Thim S, Goldfeld AE. Ethnic‐specific genetic associations with pulmonary tuberculosis. J Infect Dis. 2002;186(10):1463‐1468. 10.1086/344891. PubMed DOI

Redelings MD, Sorvillo F, Mascola L. Increase in Clostridium difficile‐related mortality rates, United States, 1999–2004. Emerg Infect Dis. 2007;13(9):1417‐1419. 10.3201/eid1309.061116. PubMed DOI PMC

Smith KGC, Clatworthy MR. FcγRIIB in autoimmunity and infection: evolutionary and therapeutic implications. Nat Rev Immunol. 2010;10(5):328‐343. 10.1038/nri2762. PubMed DOI PMC

Zou L, Ruan F, Huang M, et al. SARS‐CoV‐2 viral load in upper respiratory specimens of infected patients. N Engl J Med. 2020;382(12):1177‐1179. 10.1056/NEJMc2001737. PubMed DOI PMC

Andersen H. Mechanisms: what are they evidence for in evidence‐based medicine? J Eval Clin Pract. 2012;18(5):992‐999. 10.1111/j.1365-2753.2012.01906.x. PubMed DOI

Bankes S. Exploratory modeling for policy analysis. Oper Res. 1993;41(3):435‐449. 10.1287/opre.41.3.435. DOI

Wu J, Dhingra R, Gambhir M, Remais JV. Sensitivity analysis of infectious disease models: methods, advances and their application. J R Soc Interface. 2013;10(86):20121018. 10.1098/rsif.2012.1018. PubMed DOI PMC

Weisberg M. Robustness analysis. Philos Sci. 2006;73(5):730‐742. 10.1086/518628. DOI

Machamer P, Darden L, Craver CF. Thinking about mechanisms. Philos Sci. 2000;67(1):1‐25. 10.1086/392759. DOI

Giere RN. How models are used to represent reality. Philos Sci. 2004;71(5):742‐752. 10.1086/425063. DOI

Giere RN. An agent‐based conception of models and scientific representation. Synthese. 2010;172(2):269‐281. 10.1007/s11229-009-9506-z. DOI

Glennan S. Modeling mechanisms. Stud Hist Philos Sci Part C Stud Hist Philos Biol Biomed Sci. 2005;36(2):443‐464. 10.1016/J.SHPSC.2005.03.011. PubMed DOI

Maziarz M. The Philosophy of Causality in Economics: Causal Inferences and Policy Proposals. London: Routledge; 2020.

Eubank S, Eckstrand I, Lewis B, et al. Impact of non‐pharmaceutical interventions (NPIs) to reduce COVID‐19 mortality and healthcare demand. Bull Math Biol. 2020;82(4):52. 10.1007/s11538-020-00726-x. PubMed DOI PMC

Worldometers.info . Coronavirus. https://www.worldometers.info/coronavirus/country/australia/. Accessed June 25, 2020.

Pannett R. Australia restricts travelers from mainland China as virus impact spreads. Wall Street Journal. https://www.wsj.com/articles/australia-s-qantas-suspends-china-flights-as-virus-impact-spreads-11580536238. 2020, February 1.

Worthington B, Snape J. Australia blocks arrival of all non‐citizens, non‐residents in expanded coronavirus travel ban. ABC News. https://www.abc.net.au/news/2020-03-19/coronavirus-non-resident-travel-ban-australia/12071640. 2020, March 19.

Australian Prime Minister . Update on coronavirus measures. https://www.pm.gov.au/media/update-coronavirus-measures-220320. 2020, March 22.

Knaus C, Wahlquist C, Remeikis A. PM Announces Pubs, Clubs and Cinemas to Close, Schools Stay Open in Stage One Measures—as it Happened. The Guardian. https://www.theguardian.com/world/live/2020/mar/22/coronavirus-updates-live-australia-nsw-victoria-qld-tasmania-cases-government-stimulus-latest-update-news. 2020, March 22.

National Cabinet . National Cabinet Statement. https://www.pm.gov.au/media/national-cabinet-statement. 2020, March 29.

Karp P, Davey M. Why Australia Is Not Shutting Schools to Help Control the Spread of Coronavirus. The Guardian. https://www.theguardian.com/world/2020/mar/16/why-australia-is-not-shutting-schools-to-help-control-the-spread-of-coronavirus. 2020, March 16.

Victoria State Government . Coronavirus update for Victoria. https://www.dhhs.vic.gov.au/coronavirus-update-victoria-22-june-2020. 2020, June 22.

Dignum F, Dignum V, Davidsson P, et al. Analysing the combined health, social and economic impacts of the corovanvirus pandemic using agent‐based social simulation. 2020. http://arxiv.org/abs/2004.12809. Accessed May 16, 2020. PubMed PMC

Bodenstein M, Corsetti G, Guerrieri L. Social distancing and supply disruptions in a pandemic. Financ Econ Discuss Ser. 2020;2020(031):1–52. 10.17016/feds.2020.031. PubMed DOI PMC

Broadbent A, Smart B. Why a one‐size‐fits‐all approach to COVID‐19 could have lethal consequences. 2020. https://theconversation.com/why-a-one-size-fits-all-approach-to-covid-19-could-have-lethal-consequences-134252.

Anjum RL, Copeland S, Rocca E. Medical scientists and philosophers worldwide appeal to EBM to expand the notion of ‘evidence’. BMJ Evidence‐Based Med. 2020;25(1):6‐8. 10.1136/BMJEBM-2018-111092. PubMed DOI

Sackett DL, Straus SE, Richardson WS, Rosenberg W, Haynes RB. Evidence‐Based Medicine. How to Practice and Teach EBM. 2nd ed. London: Churchill Livingstone; 2000.

Assessing the quality of evidence from epidemiological agent-based models for the COVID-19 pandemic