Skin sensitization potential or potency is an important end point in the safety assessment of new chemicals and new chemical mixtures. Formerly, animal experiments such as the local lymph node assay (LLNA) were the main form of assessment. Today, however, the focus lies on the development of nonanimal testing approaches (i.e., in vitro and in chemico assays) and computational models. In this work, we investigate, based on publicly available LLNA data, the ability of aggregated, Mondrian conformal prediction classifiers to differentiate between non- sensitizing and sensitizing compounds as well as between two levels of skin sensitization potential (weak to moderate sensitizers, and strong to extreme sensitizers). The advantage of the conformal prediction framework over other modeling approaches is that it assigns compounds to activity classes only if a defined minimum level of confidence is reached for the individual predictions. This eliminates the need for applicability domain criteria that often are arbitrary in their nature and less flexible. Our new binary classifier, named Skin Doctor CP, differentiates nonsensitizers from sensitizers with a higher reliability-to-efficiency ratio than the corresponding nonconformal prediction workflow that we presented earlier. When tested on a set of 257 compounds at the significance levels of 0.10 and 0.30, the model reached an efficiency of 0.49 and 0.92, and an accuracy of 0.83 and 0.75, respectively. In addition, we developed a ternary classification workflow to differentiate nonsensitizers, weak to moderate sensitizers, and strong to extreme sensitizers. Although this model achieved satisfactory overall performance (accuracies of 0.90 and 0.73, and efficiencies of 0.42 and 0.90, at significance levels 0.10 and 0.30, respectively), it did not obtain satisfying class-wise results (at a significance level of 0.30, the validities obtained for nonsensitizers, weak to moderate sensitizers, and strong to extreme sensitizers were 0.70, 0.58, and 0.63, respectively). We argue that the model is, in consequence, unable to reliably identify strong to extreme sensitizers and suggest that other ternary models derived from the currently accessible LLNA data might suffer from the same problem. Skin Doctor CP is available via a public web service at https://nerdd.zbh.uni-hamburg.de/skinDoctorII/.

Center for Bioinformatics Department of Informatics Universität Hamburg 20146 Hamburg Germany

Department of Computer and Systems Sciences Stockholm University SE 16407 Kista Sweden

Department of Informatics and Chemistry University of Chemistry and Technology Prague 16628 Prague Czech Republic

Department of Pharmaceutical Biosciences Uppsala University SE 75124 Uppsala Sweden

Department of Pharmaceutical Chemistry University of Vienna 1090 Vienna Austria

Front End Innovation Beiersdorf AG 22529 Hamburg Germany

HITeC e 5 22527 Hamburg Germany

MTM Research Centre School of Science and Technology Örebro University SE 70182 Örebro Sweden

Zobrazit více v PubMed

Kimber I.; Basketter D. A.; Gerberick G. F.; Ryan C. A.; Dearman R. J. (2011) Chemical Allergy: Translating Biology into Hazard Characterization. Toxicol. Sci. 120 (Suppl 1), S238–S268. 10.1093/toxsci/kfq346. PubMed DOI

Lushniak B. D. (2004) Occupational Contact Dermatitis. Dermatol. Ther. 17, 272–277. 10.1111/j.1396-0296.2004.04032.x. PubMed DOI

Thyssen J. P.; Linneberg A.; Menné T.; Johansen J. D. (2007) The Epidemiology of Contact Allergy in the General Population – Prevalence and Main Findings. Contact Dermatitis 57, 287–299. 10.1111/j.1600-0536.2007.01220.x. PubMed DOI

Felter S.; Kern P.; Ryan C. (2018) Allergic Contact Dermatitis: Adequacy of the Default 10X Assessment Factor for Human Variability to Protect Infants and Children. Regul. Toxicol. Pharmacol. 99, 116–121. 10.1016/j.yrtph.2018.09.011. PubMed DOI

OECD . (2010) OECD Guidelines for the Testing of Chemicals, Section 4 Test No. 429: Skin Sensitisation Local Lymph Node Assay: Local Lymph Node Assay, OECD Publishing.

Anderson S. E.; Siegel P. D.; Meade B. J. (2011) The LLNA: A Brief Review of Recent Advances and Limitations. J. Allergy 2011, 424203–424213. 10.1155/2011/424203. PubMed DOI PMC

Gerberick G. F.; House R. V.; Fletcher E. R.; Ryan C. A. (1992) Examination of the Local Lymph Node Assay for Use in Contact Sensitization Risk Assessment. Fundam. Appl. Toxicol. 19, 438–445. 10.1016/0272-0590(92)90183-I. PubMed DOI

Leenaars C. H. C.; Kouwenaar C.; Stafleu F. R.; Bleich A.; Ritskes-Hoitinga M.; De Vries R. B. M.; Meijboom F. L. B. (2019) Animal to Human Translation: A Systematic Scoping Review of Reported Concordance Rates. J. Transl. Med. 17, 223.10.1186/s12967-019-1976-2. PubMed DOI PMC

Hoffmann S.; Kleinstreuer N.; Alépée N.; Allen D.; Api A. M.; Ashikaga T.; Clouet E.; Cluzel M.; Desprez B.; Gellatly N.; Goebel C.; Kern P. S.; Klaric M.; Kühnl J.; Lalko J. F.; Martinozzi-Teissier S.; Mewes K.; Miyazawa M.; Parakhia R.; van Vliet E.; Zang Q.; Petersohn D. (2018) Non-Animal Methods to Predict Skin Sensitization (I): The Cosmetics Europe Database. Crit. Rev. Toxicol. 48, 344–358. 10.1080/10408444.2018.1429385. PubMed DOI

Mehling A.; Eriksson T.; Eltze T.; Kolle S.; Ramirez T.; Teubner W.; van Ravenzwaay B.; Landsiedel R. (2012) Non-Animal Test Methods for Predicting Skin Sensitization Potentials. Arch. Toxicol. 86, 1273–1295. 10.1007/s00204-012-0867-6. PubMed DOI

Reisinger K.; Hoffmann S.; Alépée N.; Ashikaga T.; Barroso J.; Elcombe C.; Gellatly N.; Galbiati V.; Gibbs S.; Groux H.; Hibatallah J.; Keller D.; Kern P.; Klaric M.; Kolle S.; Kuehnl J.; Lambrechts N.; Lindstedt M.; Millet M.; Martinozzi-Teissier S.; Natsch A.; Petersohn D.; Pike I.; Sakaguchi H.; Schepky A.; Tailhardat M.; Templier M.; van Vliet E.; Maxwell G. (2015) Systematic Evaluation of Non-Animal Test Methods for Skin Sensitisation Safety Assessment. Toxicol. In Vitro 29, 259–270. 10.1016/j.tiv.2014.10.018. PubMed DOI

Ezendam J.; Braakhuis H. M.; Vandebriel R. J. (2016) State of the Art in Non-Animal Approaches for Skin Sensitization Testing: From Individual Test Methods towards Testing Strategies. Arch. Toxicol. 90, 2861–2883. 10.1007/s00204-016-1842-4. PubMed DOI

Thyssen J. P.; Giménez-Arnau E.; Lepoittevin J.-P.; Menné T.; Boman A.; Schnuch A. (2012) The Critical Review of Methodologies and Approaches to Assess the Inherent Skin Sensitization Potential (skin Allergies) of Chemicals. Contact Dermatitis 66 (Suppl 1), 11–24. 10.1111/j.1600-0536.2011.02004_2.x. PubMed DOI

Wilm A.; Kühnl J.; Kirchmair J. (2018) Computational Approaches for Skin Sensitization Prediction. Crit. Rev. Toxicol. 48, 738–760. 10.1080/10408444.2018.1528207. PubMed DOI

ECHA (European Chemicals Agency). (2017) The use of alternatives to testing on animals for the REACH regulation, third report under article 117(3) of the REACH regulation, ECHA. https://echa.europa.eu/documents/10162/13639/alternatives_test_animals_2017_en.pdf (accessed Jul 10, 2019).

Jowsey I. R.; Basketter D. A.; Westmoreland C.; Kimber I. (2006) A Future Approach to Measuring Relative Skin Sensitising Potency: A Proposal. J. Appl. Toxicol. 26, 341–350. 10.1002/jat.1146. PubMed DOI

Safford R. J.; Api A. M.; Roberts D. W.; Lalko J. F. (2015) Extension of the Dermal Sensitisation Threshold (DST) Approach to Incorporate Chemicals Classified as Reactive. Regul. Toxicol. Pharmacol. 72, 694–701. 10.1016/j.yrtph.2015.04.020. PubMed DOI

OECD . (2004) OECD Principles for the Validation, for Regulatory Purposes, of (Quantitative) Structure-Activity Relationship Models, OECD. https://www.oecd.org/chemicalsafety/risk-assessment/37849783.pdf.

Netzeva T. I.; Worth A.; Aldenberg T.; Benigni R.; Cronin M. T. D.; Gramatica P.; Jaworska J. S.; Kahn S.; Klopman G.; Marchant C. A.; Myatt G.; Nikolova-Jeliazkova N.; Patlewicz G. Y.; Perkins R.; Roberts D.; Schultz T.; Stanton D. W.; van de Sandt J. J. M.; Tong W.; Veith G.; Yang C. (2005) Current Status of Methods for Defining the Applicability Domain of (quantitative) Structure-Activity Relationships. The Report and Recommendations of ECVAM Workshop 52. ATLA, Altern. Lab. Anim. 33, 155–173. 10.1177/026119290503300209. PubMed DOI

Carrió P.; Pinto M.; Ecker G.; Sanz F.; Pastor M. (2014) Applicability Domain ANalysis (ADAN): A Robust Method for Assessing the Reliability of Drug Property Predictions. J. Chem. Inf. Model. 54, 1500–1511. 10.1021/ci500172z. PubMed DOI

Klingspohn W.; Mathea M.; Ter Laak A.; Heinrich N.; Baumann K. (2017) Efficiency of Different Measures for Defining the Applicability Domain of Classification Models. J. Cheminf. 9, 44–61. 10.1186/s13321-017-0230-2. PubMed DOI PMC

Vovk V., Gammerman A., and Shafer G. (2005) Algorithmic Learning in a Random World, Springer Science & Business Media.

Norinder U.; Carlsson L.; Boyer S.; Eklund M. (2015) Introducing Conformal Prediction in Predictive Modeling for Regulatory Purposes. A Transparent and Flexible Alternative to Applicability Domain Determination. Regul. Toxicol. Pharmacol. 71, 279–284. 10.1016/j.yrtph.2014.12.021. PubMed DOI

Norinder U.; Rybacka A.; Andersson P. L. (2016) Conformal Prediction to Define Applicability Domain – A Case Study on Predicting ER and AR Binding. SAR and QSAR in Environmental Research 27, 303–316. 10.1080/1062936X.2016.1172665. PubMed DOI

Cortés-Ciriano I., and Bender A. (2020) Concepts and applications of conformal prediction in computational drug discovery. ArXiv. https://arxiv.org/pdf/1908.03569.pdf (accessed 03-17-2020).

Svensson F.; Afzal A. M.; Norinder U.; Bender A. (2018) Maximizing Gain in High-Throughput Screening Using Conformal Prediction. J. Cheminf. 10, 7.10.1186/s13321-018-0260-4. PubMed DOI PMC

Norinder U.; Svensson F. (2019) Multitask Modeling with Confidence Using Matrix Factorization and Conformal Prediction. J. Chem. Inf. Model. 59, 1598–1604. 10.1021/acs.jcim.9b00027. PubMed DOI

Norinder U.; Ahlberg E.; Carlsson L. (2019) Predicting Ames Mutagenicity Using Conformal Prediction in the Ames/QSAR International Challenge Project. Mutagenesis 34, 33–40. 10.1093/mutage/gey038. PubMed DOI

Carlsson L., Eklund M., and Norinder U. (2014) Aggregated Conformal Prediction. In Artificial Intelligence Applications and Innovations, Springer, pp 231–240.

Alves V. M.; Capuzzi S. J.; Braga R. C.; Borba J. V. B.; Silva A. C.; Luechtefeld T.; Hartung T.; Andrade C. H.; Muratov E. N.; Tropsha A. (2018) A Perspective and a New Integrated Computational Strategy for Skin Sensitization Assessment. ACS Sustainable Chem. Eng. 6, 2845–2859. 10.1021/acssuschemeng.7b04220. DOI

Di P.; Yin Y.; Jiang C.; Cai Y.; Li W.; Tang Y.; Liu G. (2019) Prediction of the Skin Sensitising Potential and Potency of Compounds via Mechanism-Based Binary and Ternary Classification Models. Toxicol. In Vitro 59, 204–214. 10.1016/j.tiv.2019.01.004. PubMed DOI

Wilm A.; Stork C.; Bauer C.; Schepky A.; Kühnl J.; Kirchmair J. (2019) Skin Doctor: Machine Learning Models for Skin Sensitization Prediction That Provide Estimates and Indicators of Prediction Reliability. Int. J. Mol. Sci. 20, 4833–4856. 10.3390/ijms20194833. PubMed DOI PMC

Laggner C. (2005) SMARTS Patterns for Functional Group Classification, Inte:Ligand Software-Entwicklungs und Consulting GmbH. https://github.com/openbabel/openbabel/blob/master/data/SMARTS_InteLigand.txt (accessed 10-02-2020).

Landrum G. (2019) RDKit, GitHub. http://www.rdkit.org (accessed 04-26-2019).

(2019) scikit-learn: machine learning in Python — scikit-learn 0.21.0 documentation, scikit. https://scikit-learn.org/stable/ (accessed 05-10-2019).

Matthews B. W. (1975) Comparison of the Predicted and Observed Secondary Structure of T4 Phage Lysozyme. Biochim. Biophys. Acta, Protein Struct. 405, 442–451. 10.1016/0005-2795(75)90109-9. PubMed DOI

Linusson H.; Norinder U.; Boström H.; Johansson U.; Löfström T. (2017) On the calibration of aggregated conformal predictors. Proc. Mach Learn Res. 60, 1–20.

Williams R. V.; Amberg A.; Brigo A.; Coquin L.; Giddings A.; Glowienke S.; Greene N.; Jolly R.; Kemper R.; O’Leary-Steele C.; Parenty A.; Spirkl H.-P.; Stalford S. A.; Weiner S. K.; Wichard J. (2016) It’s Difficult, but Important, to Make Negative Predictions. Regul. Toxicol. Pharmacol. 76, 79–86. 10.1016/j.yrtph.2016.01.008. PubMed DOI