Evaluation of safety performance remains central to any safety and risk management. Currently, there are very few support tools and methods which allow for quantitative approach in this domain. One of the successful methods available to this end is the Aerospace Performance Factor (APF). The method is based on hierarchical clustering of taxonomy-based safety performance indicators, using simple and intelligible formula to compute the overall safety performance signal. The work presented in this study deals with one of the APF shortcomings, namely the absence of nonlinear relations among the performance indicators to capture more accurately the risk in the assessed system. It proposes an addition of new decision criteria behind the APF method as part of the application of Analytical Hierarchy Process (AHP), namely the impact of respective performance indicator on other indicators, regardless of their hierarchical structure. This addition leads to relative changes of performance indicators significance, where those with the highest potential for nonlinear interactions among the entire set of performance indicators are emphasized and the change in their weight ultimately leads to changes in the overall APF signal. The study results indicate that the extended APF signal is refined in terms of extremes and it draws more accurate picture about the actual safety performance, eventually supporting better identification of deviations from its acceptable values. The study was experimentally carried out in the aviation with data from the European Central Repository (ECR) originating from United Kingdom during the years 2013-2015 and verified further on data sets from Finland and Denmark.

Faculty of Transportation Sciences Czech Technical University Prague Czech Republic

See more in PubMed

Castilho, D. S. (2019). Active STPA: Integration of hazard analysis into a safety management system framework. (Doctoral dissertation). Massachusetts Institute of Technology.

De Jong, H. H., & van Es, G. W. H. (2006). A framework of indicators for the potential influence of ANS on air traffic safety in Europe. (Technical Report). National Aerospace Laboratory (NLR). https://www.eurocontrol.int/sites/default/files/field_tabs/content/documents/single-sky/pru/publications/other/framework-safety.pdf

Dekker, S. (2011). Drift into failure: From hunting broken components to understanding complex systems. Ashgate.

Dekker, S. (2015). Safety differently: Human factors for a new era. CRC Press.

Di Gravio, G., Patriarca, R., Mancini, M., & Costantino, F. (2015). Overall safety performance of air traffic management system: Forecasting and monitoring. Safety Science, 72, 351-362.

Di Gravio, G., Mancini, M., Patriarca, R., & Costantino, F. (2015). Overall safety performance of the air traffic management system: Indicators and analysis. Journal of Air Transport Management, 44-45, 65-69.

Di Gravio, G., Patriarca, R., Mancini, M., & Costantino, F. (2016). Overall safety performance of the air traffic management system: The Italian ANSP's experience on APF. Research in Transportation Business & Management, 20, 3-12.

Di Gravio, G., Patriarca, R., Costantino, F., & Sikora, I. (2017). Safety Assessment for an ATM system change: A Methodology for the ANSPs. Promet - Traffic - Traffico, 29(1), 99.

EASA. (2014). Annex to ED Decision 2014/035/R: Acceptable means of compliance and guidance material for the implementation and measurement of Safety (Key) Performance indicators (S(KP)Is). https://www.easa.europa.eu/sites/default/files/dfu/Annex%20to%20ED%20Decision%202014-035-R.pdf

EUROCONTROL. (1999). ESARR 2 guidance to ATM safety regulators: Severity classification scheme for safety occurrences in ATM. https://www.eurocontrol.int/sites/default/files/2019-05/eam2-gui2-e1.0.pdf

EUROCONTROL. (2004). Harmonisation of European incident definitions initiative for ATM (HEIDI). https://www.eurocontrol.int/sites/default/files/2019-06/src-heidi-taxonomy.xls

EUROCONTROL. (2009). The aerospace performance factor (APF): Developing the EUROCONTROL ESARR 2 APF. (Technical Report). European Organization for the Safety of Air Navigation (EUROCONTROL).

European Commission. (2007). Commission Regulation (EC) No 1321/2007, laying down implementing rules for the integration into a central repository of information on civil aviation occurrences exchanged in accordance with Directive 2003/42/EC of the European Parliament and of the Council. Official Journal of the European Union, OJ L, 294, 3-4. https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:32007R1321&from=EN

Hollnagel, E. (2012). FRAM, The functional resonance analysis method: Modelling complex socio-technical systems. Ashgate.

Hollnagel, E. (2014). Safety-I and Safety-II: the past and future of safety management. Ashgate.

Hollnagel, E. (2017). Safety-II in practice: DEVELOPING the resilience potentials. Taylor and Francis.

ICAO. (2016). Annex 19: Safety management. International Civil Aviation Organization.

ICAO. (2018). Safety management manual (SMM). International Civil Aviation Organization.

Joint Research Centre. (2012). ECCAIRS 5 reporting system: Browser, Grapher, Data manager user manual [Computer software manual].

Kaliyamurthi, K. P. (2017). A comparison of strengths and weaknesses for analytical hierarchy process. International Journal of Pure and Applied Mathematics, 116 (8), 29-33.

Kovacova, M., Licu, A., & Lintner, T. (2018). Aerospace performance factor and its potential advances. MATEC Web of Conferences, 236, 01007.

Lalis, A., Socha, V., Kraus, J., Nagy, I., & Licu, A. (2018). Conditional and unconditional safety performance forecasts for aviation predictive risk management. In 2018 IEEE aerospace conference (pp. 1-8). Institute of Electrical and Electronics Engineers.

Leveson, N. (2012). Engineering a safer world: Systems thinking applied to safety. The MIT Press.

Leveson, N. (2019). CAST Handbook: How to learn more from incidents and accidents. http://sunnyday.mit.edu/CAST-Handbook.pdf

Leveson, N. (2020). Safety III: a systems approach to safety and resilience. http://sunnyday.mit.edu/safety-3.pdf

Lintner, T., Smith, S., Lieu, A., Cioponea, R., Stewart, S., Majumdar, A., & Dupuy, M. D. (2009). The measurement of system-wide safety performance in aviation: Three case studies in the development of the Aerospace Performance Factor (APF). Proceedings of the Flight Safety Foundation Annual International Air Safety Seminar, International Federation of Airworthiness International Conference, and the International Air Transport Association, 2, 1060-1104.

Neubauer, K., & Lintner, T. (2010). The APF: Using the aerospace performance factor to measure safety performance. In Proceedings of the Corporate Aviation Safety Seminar (p. 319-372).

Neubauer, K., & Stewart, S. (2010). The Aerospace Performance Factor: How the APF is changing the way aviation views safety information. (Technical Report). Futron Corporation.

Parmenter, D. (2015). Key performance indicators: Developing, implementing, and using winning KPIs. Wiley.

Patriarca, R., Di Gravio, G., Mancini, M., & Costantino, F. (2016). Change management in the ATM system: Integrating information in the preliminary system safety assessment. International Journal of Applied Decision Sciences, 9(2), 121.

Roelen, A. L. C., Verstraeten, J., Save, L., & Aghdassi, N. (2014). Framework safety performance indicators. (Technical Report). ASCOS - Aviation Safety and Certification of new Operations and Systems. https://www.ascos-project.eu/downloads/ascos_wp2_nlr_d2.1_version-1.5.pdf

Saaty, T. (1995). Decision making for leaders: The analytic hierarchy process for decisions in a complex world. RWS Publications.

Saaty, T. (2008). Decision making with the analytic hierarchy process. International Journal of Services Sciences, 1(1), 83.

Smith, S. D., & Smurthwaite, S. (2009). The Aerospace Performance Factor: Utilization of the analytical hierarchy process to develop a balanced performance and safety indicator of the national airspace system for the Federal Aviation Administration. In Proceedings of the 2009 International Symposium on the Analytic Hierarchy Process (ISAHP), July 29-August1, University of Pittsburgh Pittsburgh, PA.

Verstraeten, J., Roelen, A. L. C., & Speijker, L. J. P. (2014) Safety performance indicators for system of organizations in aviation. (Technical Report). ASCOS - Aviation Safety and Certification of new Operations and Systems. https://www.ascos-project.eu/downloads/ascos_paper_verstraeten.pdf

Vlad, M., & Pleter, O. T. (2015). Aerospace performance factor optimization. Review of the Air Force Academy, 13(3), 101-106.