BACKGROUND: Estimation of the risk of malignancy in pulmonary nodules detected by CT is central in clinical management. The use of artificial intelligence (AI) offers an opportunity to improve risk prediction. Here we compare the performance of an AI algorithm, the lung cancer prediction convolutional neural network (LCP-CNN), with that of the Brock University model, recommended in UK guidelines. METHODS: A dataset of incidentally detected pulmonary nodules measuring 5-15 mm was collected retrospectively from three UK hospitals for use in a validation study. Ground truth diagnosis for each nodule was based on histology (required for any cancer), resolution, stability or (for pulmonary lymph nodes only) expert opinion. There were 1397 nodules in 1187 patients, of which 234 nodules in 229 (19.3%) patients were cancer. Model discrimination and performance statistics at predefined score thresholds were compared between the Brock model and the LCP-CNN. RESULTS: The area under the curve for LCP-CNN was 89.6% (95% CI 87.6 to 91.5), compared with 86.8% (95% CI 84.3 to 89.1) for the Brock model (p≤0.005). Using the LCP-CNN, we found that 24.5% of nodules scored below the lowest cancer nodule score, compared with 10.9% using the Brock score. Using the predefined thresholds, we found that the LCP-CNN gave one false negative (0.4% of cancers), whereas the Brock model gave six (2.5%), while specificity statistics were similar between the two models. CONCLUSION: The LCP-CNN score has better discrimination and allows a larger proportion of benign nodules to be identified without missing cancers than the Brock model. This has the potential to substantially reduce the proportion of surveillance CT scans required and thus save significant resources.

Motol Teaching Hospital 5 Prague Czech Republic

Optellum Ltd Oxford UK

Oxford University Hospitals NHS Foundation Trust Oxford Oxfordshire UK

Radiology Churchill Hospital Oxford UK

Radiology Leeds Teaching Hospitals NHS Trust Leeds UK

Radiology Nottingham University Hospitals NHS Trust Nottingham UK

Respiratory Medicine Glenfield General Hospital Leicester UK

Respiratory Medicine Leeds Teaching Hospitals NHS Trust Leeds UK

Respiratory Medicine Nottingham University Hospitals City Campus Nottingham UK

See more in PubMed

Callister MEJ, Baldwin DR, Akram AR, et al. . British thoracic Society guidelines for the investigation and management of pulmonary nodules. Thorax 2015;70(Suppl 2):ii1–54. 10.1136/thoraxjnl-2015-207168 PubMed DOI

MacMahon H, Naidich DP, Goo JM, et al. . Guidelines for management of incidental pulmonary nodules detected on CT images: from the Fleischner Society 2017. Radiology 2017;284:228–43. 10.1148/radiol.2017161659 PubMed DOI

Rami-Porta R, Bolejack V, Crowley J, et al. . The IASLC lung cancer staging project: proposals for the revisions of the T descriptors in the forthcoming eighth edition of the TNM classification for lung cancer. J Thorac Oncol 2015;10:990–1003. 10.1097/JTO.0000000000000559 PubMed DOI

McWilliams A, Tammemagi MC, Mayo JR, et al. . Probability of cancer in pulmonary nodules detected on first screening CT. N Engl J Med 2013;369:910–9. 10.1056/NEJMoa1214726 PubMed DOI PMC

van Riel SJ, Ciompi F, Jacobs C, et al. . Malignancy risk estimation of screen-detected nodules at baseline CT: comparison of the PanCan model, Lung-RADS and NCCN guidelines. Eur Radiol 2017;27:4019–29. 10.1007/s00330-017-4767-2 PubMed DOI PMC

Winkler Wille MM, van Riel SJ, Saghir Z, et al. . Predictive accuracy of the PanCan lung cancer risk prediction model -external validation based on CT from the Danish lung cancer screening trial. Eur Radiol 2015;25:3093–9. 10.1007/s00330-015-3689-0 PubMed DOI

Al-Ameri A, Malhotra P, Thygesen H, et al. . Risk of malignancy in pulmonary nodules: a validation study of four prediction models. Lung Cancer 2015;89:27–30. 10.1016/j.lungcan.2015.03.018 PubMed DOI

Nair A, Baldwin DR, Field JK, et al. . Measurement methods and algorithms for the management of solid nodules. J Thorac Imaging 2012;27:230–9. 10.1097/RTI.0b013e31824f83e1 PubMed DOI

Revel M-P, Bissery A, Bienvenu M, et al. . Are two-dimensional CT measurements of small noncalcified pulmonary nodules reliable? Radiology 2004;231:453–8. 10.1148/radiol.2312030167 PubMed DOI

LeCun Y, Bengio Y, Hinton G. Deep learning. Nature 2015;521:436–44. 10.1038/nature14539 PubMed DOI

Huang G, Liu Z, Lvd M. , eds Densely connected Convolutional networks. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017. , 21-26 July 2017.

Aberle DR, Adams AM, Berg CD, et al. . Reduced lung-cancer mortality with low-dose computed tomographic screening. N Engl J Med 2011;365:395–409. 10.1056/NEJMoa1102873 PubMed DOI PMC

Oke JL, Pickup LC, Declerck J, et al. . Development and validation of clinical prediction models to risk stratify patients presenting with small pulmonary nodules: a research protocol. Diagn Progn Res 2018;2:22 10.1186/s41512-018-0044-3 PubMed DOI PMC

Efron BTR. An introduction to the bootstrap. New York: Chapman and Hall, 1993.

Horeweg N, van Rosmalen J, Heuvelmans MA, et al. . Lung cancer probability in patients with CT-detected pulmonary nodules: a prespecified analysis of data from the Nelson trial of low-dose CT screening. Lancet Oncol 2014;15:1332–41. 10.1016/S1470-2045(14)70389-4 PubMed DOI