Cyber situational awareness has been proven to be of value in forming a comprehensive understanding of threats and vulnerabilities within organisations, as the degree of exposure is governed by the prevailing levels of cyber-hygiene and established processes. A more accurate assessment of the security provision informs on the most vulnerable environments that necessitate more diligent management. The rapid proliferation in the automation of cyber-attacks is reducing the gap between information and operational technologies and the need to review the current levels of robustness against new sophisticated cyber-attacks, trends, technologies and mitigation countermeasures has become pressing. A deeper characterisation is also the basis with which to predict future vulnerabilities in turn guiding the most appropriate deployment technologies. Thus, refreshing established practices and the scope of the training to support the decision making of users and operators. The foundation of the training provision is the use of Cyber-Ranges (CRs) and Test-Beds (TBs), platforms/tools that help inculcate a deeper understanding of the evolution of an attack and the methodology to deploy the most impactful countermeasures to arrest breaches. In this paper, an evaluation of documented CR and TB platforms is evaluated. CRs and TBs are segmented by type, technology, threat scenarios, applications and the scope of attainable training. To enrich the analysis of documented CR and TB research and cap the study, a taxonomy is developed to provide a broader comprehension of the future of CRs and TBs. The taxonomy elaborates on the CRs/TBs dimensions, as well as, highlighting a diminishing differentiation between application areas.

Department of Computer Science Cardiff School of Technologies Cardiff Metropolitan University Cardiff CF5 2YB UK

Department of Computer Science Faculty of Electrical Engineering Czech Technical University Prague 166 36 Prague 6 Czech Republic

Department of Electronic and Electrical Engineering University of Strathclyde Glasgow G1 1XQ UK

Department of Informatics and Telecommunications University of Peloponnese 221 00 Tripoli Greece

Division of Cybersecurity Abertay University Dundee DD1 1HG UK

Naval Academy Research Institute Arts et Métiers Institute of Technology Rue du Poulmic 29160 Lanvéoc France

The Center for Security Studies GR 101 77 Athens Greece

Zobrazit více v PubMed

Denning D.E. Stuxnet: What has changed? Future Internet. 2012;4:672–687. doi: 10.3390/fi4030672. DOI

Lallie H.S., Shepherd L.A., Nurse J.R.C., Erola A., Epiphaniou G., Maple C., Bellekens X. Cyber Security in the Age of COVID-19: A Timeline and Analysis of Cyber-Crime and Cyber-Attacks during the Pandemic. arXiv. 2020cs.CR/2006.11929 PubMed PMC

Leyden J. Stuxnet’a Game Changer for Malware Defence’. The Register; London, UK: 2010.

Mitchell R. After Stuxnet: The new rules of cyberwar. Computerworld. 2012;5:1.

Benson P. Computer Virus Stuxnet a ‘Game Changer’, DHS Official Tells Senate. [(accessed on 11 December 2020)]; Available online: http://edition.cnn.com/2010/TECH/web/11/17/stuxnet.virus/index.html.

Aldawood H., Skinner G. Educating and raising awareness on cyber security social engineering: A literature review; Proceedings of the 2018 IEEE International Conference on Teaching, Assessment, and Learning for Engineering (TALE); Wollongong, NSW, Australia. 4–7 December 2018; pp. 62–68.

Zwilling M., Klien G., Lesjak D., Wiechetek Ł., Cetin F., Basim H.N. Cyber Security Awareness, Knowledge and Behavior: A Comparative Study. J. Comput. Inf. Syst. 2020:1–16. doi: 10.1080/08874417.2020.1712269. DOI

Newhouse W., Keith S., Scribner B., Witte G. National initiative for cybersecurity education (NICE) cybersecurity workforce framework. NIST Spec. Publ. 2017;800:181.

Giuliano V., Formicola V. ICSrange: A Simulation-based Cyber Range Platform for Industrial Control Systems. arXiv. 20191909.01910

Yonemura K., Sato J., Komura R., Matsuoka M. Practical security education on combination of OT and ICT using gamfication method; Proceedings of the 2018 IEEE Global Engineering Education Conference (EDUCON 2018); Tenerife, Spain. 17–20 April 2018; pp. 746–750.

Van der Velde D., Henze M., Kathmann P., Wassermann E., Andres M., Bracht D., Ernst R., Hallak G., Klaer B., Linnartz P., et al. Methods for Actors in the Electric Power System to Prevent, Detect and React to ICT Attacks and Failures. arXiv. 20202003.06185

Vozikis D., Darra E., Kuusk T., Kavallieros D., Reintam A., Bellekens X. On the Importance of Cyber-Security Training for Multi-Vector Energy Distribution System Operators; Proceedings of the 15th International Conference on Availability, Reliability and Security, online; online. 25–28 August 2020; DOI

Okoli C., Schabram K. A Guide to Conducting a Systematic Literature Review of Information Systems Research. SSRN Electron. J. 2010 doi: 10.2139/ssrn.1954824. DOI

Okoli C. A Guide to Conducting a Standalone Systematic Literature Review. Commun. Assoc. Inf. Syst. 2015;37:43. doi: 10.17705/1CAIS.03743. DOI

Bures M., Klima M., Rechtberger V., Bellekens X., Tachtatzis C., Atkinson R., Ahmed B.S. Interoperability and Integration Testing Methods for IoT Systems: A Systematic Mapping Study. In: de Boer F., Cerone A., editors. Software Engineering and Formal Methods. Springer International Publishing; Cham, Switzerland: 2020. pp. 93–112.

Davis J., Magrath S. A Survey of Cyber Ranges and Testbeds. [(accessed on 11 December 2020)]; Available online: https://apps.dtic.mil/dtic/tr/fulltext/u2/a594524.pdf.

Ranka J. National Cyber Range. [(accessed on 11 December 2020)]; Available online: https://apps.dtic.mil/dtic/tr/fulltext/u2/a551864.pdf.

Leblanc S.P., Partington A., Chapman I.M., Bernier M. An Overview of Cyber Attack and Computer Network Operations Simulation; Proceedings of the 2011 Military Modeling & Simulation Symposium; Boston, MA, USA. 3–7 April 2011; pp. 92–100.

Priyadarshini I. Ph.D. Thesis. University of Delaware; Newark, DE, USA: 2018. Features and Architecture of the Modern Cyber Range: A Qualitative Analysis and Survey.

Yamin M.M., Katt B., Gkioulos V. Cyber Ranges and Security Testbeds: Scenarios, Functions, Tools and Architecture. Comput. Secur. 2020;88:101636. doi: 10.1016/j.cose.2019.101636. DOI

Radziwill N.M. Virginia cyber range. Softw. Qual. Prof. 2017;19:46.

Sommestad T. Experimentation on Operational Cyber Security in CRATE. [(accessed on 11 December 2020)]; Available online: http://www.sommestad.com/teodor/Filer/Sommestad%20-%202015%20-%20Experimentation%20on%20operational%20cyber%20security%20in%20CRATE.pdf.

Pernik P. Improving Cyber Security: NATO and the EU. [(accessed on 11 December 2020)]; Available online: https://icds.ee/wp-content/uploads/2010/02/Piret_Pernik_-_Improving_Cyber_Security.pdf.

Ferguson B., Tall A., Olsen D. National cyber range overview; Proceedings of the 2014 IEEE Military Communications Conference; Baltimore, MD, USA. 6–8 October 2014; pp. 123–128.

Mudge R.S., Lingley S. Cyber and Air Joint Effects Demonstration (CAAJED) [(accessed on 11 December 2020)]; Available online: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.460.476&rep=rep1&type=pdf.

Meitzler W.D., Ouderkirk S.J., Hughes C.O. Security Assessment Simulation Toolkit (SAST) Final Report. Pacific Northwest National Lab.; Richland, WA, USA: 2009. Technical report.

Varshney M., Pickett K., Bagrodia R. A live-virtual-constructive (LVC) framework for cyber operations test, evaluation and training; Proceedings of the 2011-MILCOM 2011 Military Communications Conference; Baltimore, MD, USA. 7–10 November 2011; pp. 1387–1392.

Chi S.D., Park J.S., Lee J.S. International Workshop on Information Security Applications. Springer; Berlin/Heidelberg, Germany: 2003. A role of DEVS simulation for information assurance; pp. 27–41.

Liljenstam M., Liu J., Nicol D., Yuan Y., Yan G., Grier C. Rinse: The real-time immersive network simulation environment for network security exercises; Proceedings of the Workshop on Principles of Advanced and Distributed Simulation (PADS’05); Monterey, CA, USA. 1–3 June 2005; pp. 119–128.

Brown B., Cutts A., McGrath D., Nicol D.M., Smith T.P., Tofel B. Simulation of cyber attacks with applications in homeland defense training; Proceedings of the Sensors, and Command, Control, Communications, and Intelligence (C3I) Technologies for Homeland Defense and Law Enforcement II; Orlando, FL, USA. 21–25 April 2003; pp. 63–71.

Kuhl M.E., Sudit M., Kistner J., Costantini K. Cyber attack modeling and simulation for network security analysis; Proceedings of the 2007 Winter Simulation Conference; Washington, DC, USA. 9–12 December 2007; pp. 1180–1188.

Zhou M., Lang S.D. A Frequency-Based Approach to Intrusion Detection. [(accessed on 11 December 2020)]; Available online: https://d1wqtxts1xzle7.cloudfront.net/56628641/dosdos.pdf?1527000071=&response-content-disposition=inline%3B+filename%3DA_Frequency_Based_Approach_to_Intrusion.pdf&Expires=1607693224&Signature=BsARTy9SFcwtnh3em1-GAbLEdtohfiBZqLiuH5ODCyL5hife4Ga95~urso2IqlZxQnaAN~GepEHt8ovbOD9-XdAXMfLvKQ6PCBdFX5fhWYE2ZOGWfgcBjEy16pMT9egqXlZkSr4e-raPjDQy6GxeQRTTe4Z~tlQTCtdI6dn6GOGQssPh3lv5bwwbi6CnPBScFCv823Gwfbkk2rPp5WyLGRMKFaXlqRdzmFrqYKdUpsTeRK-bNXBjViTrD6hlbA0iFl3ELVh431TX1OC1elbrdZWaQL8sZPmi6YmM5XNH2SJ5RyqawQ1HrHjA9-Jhs3kMs5rqRw9l857KsGg9G4Whwg__&Key-Pair-Id=APKAJLOHF5GGSLRBV4ZA.

Rossey L.M., Cunningham R.K., Fried D.J., Rabek J.C., Lippmann R.P., Haines J.W., Zissman M.A. LARIAT: Lincoln adaptable real-time information assurance testbed; Proceedings of the IEEE Aerospace Conference; Big Sky, MT, USA. 9–16 March 2002; p. 6.

Pederson P., Lee D., Shu G., Chen D., Liu Z., Li N., Sang L. Virtual Cyber-Security Testing Capability for Large Scale Distributed Information Infrastructure Protection; Proceedings of the 2008 IEEE Conference on Technologies for Homeland Security; Waltham, MA, USA. 12–13 May 2008; pp. 372–377.

Anderson R.S. Cyber Security and Resilient Systems. Idaho National Laboratory; Idaho Falls, ID, USA: 2009. Technical report.

Siaterlis C., Garcia A.P., Genge B. On the use of Emulab testbeds for scientifically rigorous experiments. IEEE Commun. Surv. Tutor. 2012;15:929–942. doi: 10.1109/SURV.2012.0601112.00185. DOI

Mayo J., Minnich R., Rudish D., Armstrong R. Approaches for Scalable Modeling and Emulation of Cyber Systems: LDRD Final Report. [(accessed on 11 December 2020)]; Available online: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.463.1793&rep=rep1&type=pdf.

Thomas J., Meunier P., Eugster P., Vitek J. Mandatory access control for experiments with malware; Proceedings of the 10th Annual Information Security Symposium; West Lafayette, IN, USA. 9–10 April 2009.

Brueckner S., Guaspari D., Adelstein F., Weeks J. Automated computer forensics training in a virtualized environment. Digit. Investig. 2008;5:S105–S111. doi: 10.1016/j.diin.2008.05.009. DOI

Tsai P.W., Yang C.S. Testbed@ TWISC: A network security experiment platform. Int. J. Commun. Syst. 2018;31:e3446. doi: 10.1002/dac.3446. DOI

Herold N., Wachs M., Dorfhuber M., Rudolf C., Liebald S., Carle G. IFIP International Conference on Autonomous Infrastructure, Management and Security. Springer; Cham, Switzerland: 2017. Achieving reproducible network environments with INSALATA; pp. 30–44.

Chadha R., Bowen T., Chiang C.Y.J., Gottlieb Y.M., Poylisher A., Sapello A., Serban C., Sugrim S., Walther G., Marvel L.M., et al. Cybervan: A cyber security virtual assured network testbed; Proceedings of the MILCOM 2016—2016 IEEE Military Communications Conference; Baltimore, MD, USA. 1–3 November 2016; pp. 1125–1130.

Gunathilaka P., Mashima D., Chen B. Softgrid: A software-based smart grid testbed for evaluating substation cybersecurity solutions; Proceedings of the 2nd ACM Workshop on Cyber-Physical Systems Security and Privacy; Vienna, Austria. 28 October 2016; pp. 113–124.

Haines J.W., Rossey L.M., Lippmann R.P., Cunningham R.K. Extending the darpa off-line intrusion detection evaluations; Proceedings of the DARPA Information Survivability Conference and Exposition II. DISCEX’01; Anaheim, CA, USA. 12–16 June 2001; pp. 35–45.

Hahn A., Kregel B., Govindarasu M., Fitzpatrick J., Adnan R., Sridhar S., Higdon M. Development of the PowerCyber SCADA security testbed; Proceedings of the Sixth Annual Workshop on Cyber Security and Information Intelligence Research; Oak Ridge, TN, USA. 21–23 April 2010; pp. 1–4.

Hindy H., Brosset D., Bayne E., Seeam A., Bellekens X. Improving SIEM for Critical SCADA Water Infrastructures Using Machine Learning. In: Katsikas S.K., Cuppens F., Cuppens N., Lambrinoudakis C., Antón A., Gritzalis S., Mylopoulos J., Kalloniatis C., editors. Computer Security. Springer International Publishing; Cham, Switzerland: 2019. pp. 3–19.

Tam K., Jones K. Cyber-SHIP: Developing Next Generation Maritime Cyber Research Capabilities. [(accessed on 11 December 2020)]; Available online: https://pearl.plymouth.ac.uk/handle/10026.1/14949.

Hildebrand E., Flinterman R., Mulder J., Smit A. Clusus: A Cyber Range for Network Attack Simulations. [(accessed on 11 December 2020)]; Available online: http://resolver.tudelft.nl/uuid:2588ee53-0249-476f-aa53-54d7a368dc8e.

Hallaq B., Nicholson A., Smith R., Maglaras L., Janicke H., Jones K. Cyber Security and Threats: Concepts, Methodologies, Tools, and Applications. IGI Global; Hershey, PA, USA: 2018. CYRAN: A hybrid cyber range for testing security on ICS/SCADA systems; pp. 622–637.

Ghaleb A., Zhioua S., Almulhem A. SCADA-SST: A SCADA security testbed; Proceedings of the 2016 World Congress on Industrial Control Systems Security (WCICSS); London, UK. 12–14 December 2016; pp. 1–6.

Čeleda P., Čegan J., Vykopal J., Tovarňák D. Kypo—A platform for cyber defence exercises. [(accessed on 11 December 2020)]; Available online: https://is.muni.cz/repo/1319597/2015-NATO-MSG-133-kypo-platform-cyber-defence-exercises-paper.pdf.

Vykopal J., Ošlejšek R., Čeleda P., Vizvary M., Tovarňák D. Kypo Cyber Range: Design and Use Cases. [(accessed on 11 December 2020)]; Available online: https://repozitar.cz/repo/26486/

Almalawi A., Tari Z., Khalil I., Fahad A. SCADAVT-A framework for SCADA security testbed based on virtualization technology; Proceedings of the 38th Annual IEEE Conference on Local Computer Networks; Sydney, Australia. 21–24 October 2013; pp. 639–646.

Benzel T., Braden R., Kim D., Neuman C., Joseph A., Sklower K., Ostrenga R., Schwab S. Experience with deter: A testbed for security research; Proceedings of the 2nd International Conference on Testbeds and Research Infrastructures for the Development of Networks and Communities, TRIDENTCOM; Barcelona, Spain. 1–3 March 2006.

Richmond M. ViSe: A virtual security testbed. [(accessed on 11 December 2020)]; Available online: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.90.2205&rep=rep1&type=pdf.

Marrocco D. Ph.D. Thesis. Politecnico di Torino; Torino, Italy: 2018. Design and Deployment of a Virtual Environment to Emulate a Scada Network within Cyber Ranges.

Ten C.W., Liu C.C., Govindarasu M. Vulnerability assessment of cybersecurity for SCADA systems using attack trees; Proceedings of the 2007 IEEE Power Engineering Society General Meeting; Tampa, FL, USA. 24–28 June 2007; pp. 1–8.

Russo E., Costa G., Armando A. Scenario design and validation for next generation cyber ranges; Proceedings of the 2018 IEEE 17th International Symposium on Network Computing and Applications (NCA); Cambridge, MA, USA. 1–3 November 2018; pp. 1–4.

Binz T., Breitenbücher U., Kopp O., Leymann F. Advanced Web Services. Springer; New York, NY, USA: 2014. TOSCA: Portable automated deployment and management of cloud applications; pp. 527–549.

Russo E., Costa G., Armando A. Building Next Generation Cyber Ranges with CRACK. Comput. Secur. 2020;95:101837. doi: 10.1016/j.cose.2020.101837. DOI

De Leon D.C., Goes C.E., Haney M.A., Krings A.W. ADLES: Specifying, deploying, and sharing hands-on cyber-exercises. Comput. Secur. 2018;74:12–40. doi: 10.1016/j.cose.2017.12.007. DOI

Cohen F. Simulating cyber attacks, defences, and consequences. Comput. Secur. 1999;18:479–518. doi: 10.1016/S0167-4048(99)80115-1. DOI

Park J.S., Lee J.S., Kim H.K., Jeong J.R., Yeom D.B., Chi S.D. Secusim: A tool for the cyber-attack simulation; Proceedings of the International Conference on Information and Communications Security; Xi’an, China. 13–16 November 2001; pp. 471–475.

Lynn W.F., III Defending a new domain-the Pentagon’s cyberstrategy. Foreign Aff. 2010;89:97.

Hammad E., Ezeme M., Farraj A. Implementation and development of an offline co-simulation testbed for studies of power systems cyber security and control verification. Int. J. Electr. Power Energy Syst. 2019;104:817–826. doi: 10.1016/j.ijepes.2018.07.058. DOI

Poudel S., Ni Z., Malla N. Real-time cyber physical system testbed for power system security and control. Int. J. Electr. Power Energy Syst. 2017;90:124–133. doi: 10.1016/j.ijepes.2017.01.016. DOI

Waraga O.A., Bettayeb M., Nasir Q., Talib M.A. Design and implementation of automated IoT security testbed. Comput. Secur. 2020;88:101648. doi: 10.1016/j.cose.2019.101648. DOI

Kim Y., Nam J., Park T., Scott-Hayward S., Shin S. SODA: A software-defined security framework for IoT environments. Comput. Netw. 2019;163:106889. doi: 10.1016/j.comnet.2019.106889. DOI

Lee S., Lee S., Yoo H., Kwon S., Shon T. Design and implementation of cybersecurity testbed for industrial IoT systems. J. Supercomput. 2018;74:4506–4520. doi: 10.1007/s11227-017-2219-z. DOI

Wang Y., Nguyen T.L., Xu Y., Shi D. Distributed control of heterogeneous energy storage systems in islanded microgrids: Finite-time approach and cyber-physical implementation. Int. J. Electr. Power Energy Syst. 2020;119:105898. doi: 10.1016/j.ijepes.2020.105898. DOI

De La Torre G., Rad P., Choo K.K.R. Implementation of deep packet inspection in smart grids and industrial Internet of Things: Challenges and opportunities. J. Netw. Comput. Appl. 2019;135:32–46. doi: 10.1016/j.jnca.2019.02.022. DOI

Kumar A., Lim T.J. A Secure Contained Testbed for Analyzing IoT Botnets; Proceedings of the International Conference on Testbeds and Research Infrastructures; Shanghai, China. 1–3 December 2018; pp. 124–137.

Alves T., Das R., Werth A., Morris T. Virtualization of SCADA testbeds for cybersecurity research: A modular approach. Comput. Secur. 2018;77:531–546. doi: 10.1016/j.cose.2018.05.002. DOI

Siracusano G., Salsano S., Ventre P.L., Detti A., Rashed O., Blefari-Melazzi N. A framework for experimenting ICN over SDN solutions using physical and virtual testbeds. Comput. Netw. 2018;134:245–259. doi: 10.1016/j.comnet.2018.01.026. DOI

Papadopoulos G.Z., Gallais A., Schreiner G., Jou E., Noel T. Thorough IoT testbed characterization: From proof-of-concept to repeatable experimentations. Comput. Netw. 2017;119:86–101. doi: 10.1016/j.comnet.2017.03.012. DOI

Liu X.F., Shahriar M.R., Al Sunny S.N., Leu M.C., Hu L. Cyber-physical manufacturing cloud: Architecture, virtualization, communication, and testbed. J. Manuf. Syst. 2017;43:352–364. doi: 10.1016/j.jmsy.2017.04.004. DOI

Bernieri G., Miciolino E.E., Pascucci F., Setola R. Monitoring system reaction in cyber-physical testbed under cyber-attacks. Comput. Electr. Eng. 2017;59:86–98. doi: 10.1016/j.compeleceng.2017.02.010. DOI

Flauzac O., Gonzalez C., Nolot F. Developing a distributed software defined networking testbed for IoT. Procedia Comput. Sci. 2016;83:680–684. doi: 10.1016/j.procs.2016.04.151. DOI

Ashok A., Sridhar S., McKinnon A.D., Wang P., Govindarasu M. Testbed-based performance evaluation of attack resilient control for AGC; Proceedings of the 2016 Resilience Week (RWS); Chicago, IL, USA. 16–18 August 2016; pp. 125–129.

Deshmukh P.P., Patterson C.D., Baumann W.T. A hands-on modular laboratory environment to foster learning in control system security; Proceedings of the 2016 IEEE Frontiers in Education Conference (FIE); Erie, PA, USA. 12–15 October 2016; pp. 1–9.

Adhikari U., Morris T., Pan S. WAMS cyber-physical test bed for power system, cybersecurity study, and data mining. IEEE Trans. Smart Grid. 2016;8:2744–2753. doi: 10.1109/TSG.2016.2537210. DOI

Ashok A., Wang P., Brown M., Govindarasu M. Experimental evaluation of cyber attacks on automatic generation control using a CPS security testbed; Proceedings of the 2015 IEEE Power & Energy Society General Meeting; Denver, CO, USA. 26–30 July 2015; pp. 1–5.

Bauer S.W. What Is Classical Education? [(accessed on 11 December 2020)]; Available online: http://www.bluegrasschristianacademy.org/wp-content/uploads/2015/06/Classical-Education-_-The-Well-Trained-Mind.pdf.

Brynielsson J., Franke U., Varga S. Combatting Cybercrime and Cyberterrorism. Springer; Cham, Switzerland: 2016. Cyber situational awareness testing; pp. 209–233.

Boopathi K., Sreejith S., Bithin A. Learning cyber security through gamification. Indian J. Sci. Technol. 2015;8:642–649. doi: 10.17485/ijst/2015/v8i7/67760. DOI

Adams M., Makramalla M. Cybersecurity skills training: An attacker-centric gamified approach. Technol. Innov. Manag. Rev. 2015;5 doi: 10.22215/timreview/861. DOI

Bellekens X., Jayasekara G., Hindy H., Bures M., Brosset D., Tachtatzis C., Atkinson R. From Cyber-Security Deception to Manipulation and Gratification Through Gamification. In: Moallem A., editor. HCI for Cybersecurity, Privacy and Trust. Springer International Publishing; Cham, Switerzerland: 2019. pp. 99–114.

Sadeh-Koniecpol N., Wescoe K., Brubaker J., Hong J. Mock Attack Cybersecurity Training System and Methods. US9558677. US Patent. 2017 Jan 31;

Toth P., Klein P. A role-based model for federal information technology/cyber security training. NIST Spec. Publ. 2013;800:1–152.

Glumich S.M., Kropa B.A. Defex: Hands-On Cyber Defense Exercise for Undergraduate Students. [(accessed on 11 December 2020)]; Available online: https://apps.dtic.mil/dtic/tr/fulltext/u2/a560055.pdf.

Conklin A. The use of a collegiate cyber defense competition in information security education; Proceedings of the 2nd Annual Conference on Information Security Curriculum Development; Kennesaw, GA, USA. 22–23 September 2005; pp. 16–18.

Dodge R., Ragsdale D.J. Organized cyber defense competitions; Proceedings of the IEEE International Conference on Advanced Learning Technologies; Joensuu, Finland. 30 August–1 September 2004; pp. 768–770.

Augustine T., Dodge R.C. Cyber Defense Exercise: Meeting Learning Objectives Thru Competition. [(accessed on 11 December 2020)];2006 Available online: https://tinyurl.com/y254qtj3.

Mattson J.A. Cyber defense exercise: A service provider model; Proceedings of the Fifth World Conference on Information Security Education; West Point, NY, USA. 19–21 June 2007; pp. 81–86.

Patriciu V.V., Furtuna A.C. Guide for Designing Cyber Security Exercises. [(accessed on 11 December 2020)]; Available online: http://www.wseas.us/e-library/conferences/2009/tenerife/EACT-ISP/EACT-ISP-28.pdf.

Braje T.M. Advanced Tools for Cyber Ranges. MIT Lincoln Laboratory; Lexington, KY, USA: 2016. Technical report.

Talbot J., Pikula P., Sweetmore C., Rowe S., Hindy H., Tachtatzis C., Atkinson R., Bellekens X. A Security Perspective on Unikernels; Proceedings of the 2020 International Conference on Cyber Security and Protection of Digital Services (Cyber Security); Dublin, Ireland. 15–19 June 2020; pp. 1–7.

Pham C.D. On Automatic Cyber Range Instantiation for Facilitating Security Training. [(accessed on 11 December 2020)]; Available online: https://dspace.jaist.ac.jp/dspace/bitstream/10119/14161/3/paper.pdf.

Eckroth J., Chen K., Gatewood H., Belna B. Alpaca: Building Dynamic Cyber Ranges with Procedurally-Generated Vulnerability Lattices; Proceedings of the 2019 ACM Southeast Conference; Kennesaw, GA, USA. 18–20 April 2019; pp. 78–85.

Fok C., Petz A., Stovall D., Paine N., Julien C., Vishwanath S. Pharos: A Testbed for Mobile Cyber-Physical Systems. [(accessed on 11 December 2020)]; Available online: https://www.cs.odu.edu/~nadeem/classes/cs795-CPS-S13/papers/eval-001.pdf.

Deckard G.M. Cybertropolis: Breaking the paradigm of cyber-ranges and testbeds; Proceedings of the 2018 IEEE International Symposium on Technologies for Homeland Security (HST); Woburn, MA, USA. 23–24 October 2018; pp. 1–4.

Tranoris C., Denazis S., Guardalben L., Pereira J., Sargento S. Enabling Cyber-Physical Systems for 5G networking: A case study on the Automotive Vertical domain; Proceedings of the 2018 IEEE/ACM 4th International Workshop on Software Engineering for Smart Cyber-Physical Systems (SEsCPS); Gothenburg, Sweden. 27 May–3 June 2018; pp. 37–40.

Urquhart C., Bellekens X., Tachtatzis C., Atkinson R., Hindy H., Seeam A. Cyber-security internals of a skoda octavia vRS: A hands on approach. IEEE Access. 2019;7:146057–146069. doi: 10.1109/ACCESS.2019.2943837. DOI

Mitra R.N., Agrawal D.P. 5G mobile technology: A survey. ICT Express. 2015;1:132–137. doi: 10.1016/j.icte.2016.01.003. DOI

West D.M. How 5G technology enables the health internet of things. Brook. Cent. Technol. Innov. 2016;3:1–20.

Letaief K.B., Chen W., Shi Y., Zhang J., Zhang Y.J.A. The roadmap to 6G: AI empowered wireless networks. IEEE Commun. Mag. 2019;57:84–90. doi: 10.1109/MCOM.2019.1900271. DOI

Bhardwaj A., Krishna C.R. A Container-Based Technique to Improve Virtual Machine Migration in Cloud Computing. IETE J. Res. 2019:1–16. doi: 10.1080/03772063.2019.1605848. DOI

Lovas R., Kardos P., Gyöngyösi A.Z., Bottyán Z. Weather model fine-tuning with software container-based simulation platform. IDOJÁRÁS/Q. J. Hung. Meteorol. Serv. 2019;123:165–181. doi: 10.28974/idojaras.2019.2.3. DOI

Mucci D., Blumbergs B. TED: A Container Based Tool to Perform Security Risk Assessment for ELF Binaries. [(accessed on 11 December 2020)]; Available online: https://www.scitepress.org/Papers/2019/73716/73716.pdf.

Kyriakou A., Sklavos N. Container-based honeypot deployment for the analysis of malicious activity; Proceedings of the 2018 Global Information Infrastructure and Networking Symposium (GIIS); Thessaloniki, Greece. 23–25 October 2018; pp. 1–4.

Delicato F.C., Al-Anbuky A., Kevin I., Wang K. Smart Cyber–Physical Systems: Toward Pervasive Intelligence Systems. Future Gener. Comput. Syst. 2020;107:1134–1139. doi: 10.1016/j.future.2019.06.031. DOI

Kaloudi N., Li J. The ai-based cyber threat landscape: A survey. ACM Comput. Surv. (CSUR) 2020;53:1–34. doi: 10.1145/3372823. DOI

Brundage M., Avin S., Clark J., Toner H., Eckersley P., Garfinkel B., Dafoe A., Scharre P., Zeitzoff T., Filar B., et al. The malicious use of artificial intelligence: Forecasting, prevention, and mitigation. arXiv. 20181802.07228

Bécue A., Fourastier Y., Praça I., Savarit A., Baron C., Gradussofs B., Pouille E., Thomas C. CyberFactory# 1—Securing the industry 4.0 with cyber-ranges and digital twins; Proceedings of the 2018 14th IEEE International Workshop on Factory Communication Systems (WFCS); Imperia, Italy. 13–15 June 2018; pp. 1–4.

Reys N. Smart Cities and Cyber Threats, ControlRisks. [(accessed on 11 December 2020)]; Available online: https://www.controlrisks.com/-/media/corporate/files/our-thinking/insights/smart-cities-and-cyber-threats/smart-cities-article.pdf.

Baig Z.A., Szewczyk P., Valli C., Rabadia P., Hannay P., Chernyshev M., Johnstone M., Kerai P., Ibrahim A., Sansurooah K., et al. Future challenges for smart cities: Cyber-security and digital forensics. Digit. Investig. 2017;22:3–13. doi: 10.1016/j.diin.2017.06.015. DOI

Vitunskaite M., He Y., Brandstetter T., Janicke H. Smart cities and cyber security: Are we there yet? A comparative study on the role of standards, third party risk management and security ownership. Comput. Secur. 2019;83:313–331. doi: 10.1016/j.cose.2019.02.009. DOI

Mylrea M. Singapore’s Smart City: Securing It from Emerging Cyber Threats. [(accessed on 11 December 2020)]; Available online: https://dr.ntu.edu.sg/handle/10356/80353.

Srivastava S., Bisht A., Narayan N. Safety and security in smart cities using artificial intelligence—A review; Proceedings of the 2017 7th International Conference on Cloud Computing, Data Science & Engineering-Confluence; Noida, India. 12–13 January 2017; pp. 130–133.

AlDairi A. Lo’ai Tawalbeh Cyber security attacks on smart cities and associated mobile technologies. Procedia Comput. Sci. 2017;109:1086–1091. doi: 10.1016/j.procs.2017.05.391. DOI

Cerrudo C. Hacking Smart Cities. [(accessed on 11 December 2020)]; Available online: https://docs.huihoo.com/rsaconference/usa-2015/hta-t10-hacking-smart-cities.pdf.

Alibasic A., Al Junaibi R., Aung Z., Woon W.L., Omar M.A. Cybersecurity for smart cities: A brief review; Proceedings of the International Workshop on Data Analytics for Renewable Energy Integration; Riva del Garda, Italy. 23 September 2016; pp. 22–30.

Braun T., Fung B.C., Iqbal F., Shah B. Security and privacy challenges in smart cities. Sustain. Cities Soc. 2018;39:499–507. doi: 10.1016/j.scs.2018.02.039. DOI

Wang P., Ali A., Kelly W. Data security and threat modeling for smart city infrastructure; Proceedings of the 2015 International Conference on Cyber Security of Smart Cities, Industrial Control System and Communications, (SSIC); Shanghai, China. 5–7 August 2015; pp. 1–6.

Farahat I., Tolba A., Elhoseny M., Eladrosy W. Security in Smart Cities: Models, Applications, and Challenges. Springer; Cham, Switzerland: 2019. Data security and challenges in smart cities; pp. 117–142.

Vattapparamban E., Güvenç İ., Yurekli A.İ., Akkaya K., Uluağaç S. Drones for smart cities: Issues in cybersecurity, privacy, and public safety; Proceedings of the 2016 International Wireless Communications and Mobile Computing Conference (IWCMC); Paphos, Cyprus. 5–9 September 2016; pp. 216–221.

Li Z., Shahidehpour M. Deployment of cybersecurity for managing traffic efficiency and safety in smart cities. Electr. J. 2017;30:52–61. doi: 10.1016/j.tej.2017.04.003. DOI

Hands-on cybersecurity training behavior data for process mining