https://orcid.org/0000-0003-3486-8740
Figures
Abstract
Background
Radiomics involves the extraction of quantitative information from annotated Computed-Tomography (CT) images, and has been used to predict outcomes in Head and Neck Squamous Cell Carcinoma (HNSCC). Subjecting combined Radiomics and Clinical features to Machine Learning (ML) could offer better predictions of clinical outcomes. This study is a comparative performance analysis of ML models with Clinical, Radiomics, and Clinico-Radiomic datasets for predicting four outcomes of HNSCC treated with Curative Radiation Therapy (RT): Distant Metastases, Locoregional Recurrence, New Primary, and Residual Disease.
Methodology
The study used retrospective data of 311 HNSCC patients treated with radiotherapy between 2013–2018 at our centre. Binary prediction models were developed for the four outcomes with Clinical-only, Clinico-Radiomic, and Radiomics-only datasets, using three different ML classification algorithms namely, Random Forest (RF), Kernel Support Vector Machine (KSVM), and XGBoost. The best-performing ML algorithms of the three dataset groups was then compared.
Results
The Clinico-Radiomic dataset using KSVM classifier provided the best prediction. Predicted mean testing accuracy for Distant Metastases, Locoregional Recurrence, New Primary, and Residual Disease was 97%, 72%, 99%, and 96%, respectively. The mean area under the receiver operating curve (AUC) was calculated and displayed for all the models using three dataset groups.
Citation: Gangil T, Sharan K, Rao BD, Palanisamy K, Chakrabarti B, Kadavigere R (2022) Utility of adding Radiomics to clinical features in predicting the outcomes of radiotherapy for head and neck cancer using machine learning. PLoS ONE 17(12): e0277168. https://doi.org/10.1371/journal.pone.0277168
Editor: Panagiotis Balermpas, University Hospital Zurich, SWITZERLAND
Received: June 12, 2022; Accepted: October 24, 2022; Published: December 15, 2022
Copyright: © 2022 Gangil et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: The data could not be shared because it is an Intellectual property of Manipal Academy of Higher Education, Manipal, India and Philips Healthcare, Bangalore. They may have future commercial interest using the dataset. The details of the person who can be contacted regarding the same is as follows: Name: Mr. Manjunatha Maiya Designation: Senior Project Manager, Philips, Bangalore email: manjunatha.maiya@philips.com. The authors had no special access privileges to the data others would not have.
Funding: The study was funded by Manipal Academy of Higher Education and Philips, India. The role of the funding agency was in designing the study.
Competing interests: The authors have declared that no competing interests exist.
List of abbreviations: HNSCC, Squamous Cell Head and Neck Cancer; RT, Radiation Therapy; RF, Random Fores; KSVM, Kernel Support Vector Machine; AUC, Area under the Receiver operating curve; SFFS, Sequential Forward Floating selection; ML, Machine Learning; ROC, Receiver operating curve; CNN, Convolution Neural Network; CT, Computed Tomography; AI, Artificial Intelligence; PET, Positron Emission Tomography; MRI, Magnetic Resonance Imaging
1. Introduction
Head and Neck Squamous Cell Carcinoma (HNSCC) refers to a constellation of cancers in the Head and Neck region and is an important contributor to cancer-related morbidity and mortality. HNSCC is a significant health issue in India with an annual incidence of 77,000 new cases contributing to one-third of all cancers diagnosed. Treatment outcomes of HNSCC can be pretty variable, and there is scope for improvement in predicting treatment outcomes to improve survival, reduce toxicity, and further the understanding of HNSCC.
Development in the field of Artificial Intelligence (AI) has enabled the analysis of large volume data, and Machine Learning (ML) tools are being increasingly utilized in diverse fields within medicine. They are used for studying HNSCC [1]. Radiomics is an emerging method used to obtain additional information from imaging. Radiomics involves the conversion of medical images into quantitative data, thereby potentially enabling healthcare personnel in better diagnostic, therapeutic and prognostic decision-making [2]. Statistical, shape-based, histogram, and texture-based features are extracted from the Regions of Interest (ROI) for analysis and clinical correlation [3, 4]. Studies have shown that Radiomics facilitates diagnosis, treatment planning, and predicting outcomes. The development of AI, principally ML and deep learning algorithms has boosted the potential of the typically large-volume quantitative Radiomics data [5].
This research evaluates the additional benefit of Radiomics from annotated diagnostic CT images over clinical data in predicting the outcomes of HNSCC patients treated with radiotherapy. This study is an extension of our previous study [6]. It presents the change in the performance of individual models to predict Distant and Locoregional Recurrence, New Primary and Residual Disease after adding Radiomics data to the previously built models using only clinical data.
2. Methodology
This study developed and compared the analytics model to predict the clinical outcomes of HNSCC. The clinical information was retrieved from the hospital medical records, and Radiomics information was obtained from diagnostic contrast CT images. The study included HNSCC patients treated with curative intent radiotherapy at our centre. Though the minimum sample size was calculated to be 256 (S1B in S1 Appendix), we included all eligible patients treated between 2013–2018 [7]. Eligible cases included those treated with curative intent radiotherapy (either as definitive or as an adjuvant to surgery), completed their prescribed treatment, and had a minimum follow-up of three months post-treatment in addition to having the CT images necessary for the extraction of Radiomics. Clearance was obtained from the Institutional Ethics Committee before collecting the data from patient records, and the study was registered with the Clinical Trials Registry of India (CTRI Number: CTRI/2018/04/013517). The details of the dataset are presented in Fig 1.
2.1. CT image acquisition
All CT scans were acquired on Incisive CT 128-slice, or Big bore Philips CT-16 slice (Philips Healthcare, Netherlands) multidetector CT scanner. The standard protocol was followed for image acquisition. The scanning parameters were namely, region- skull base down to the thoracic inlet; energy, 120 kV; milliampere-seconds, 250 mAs; pitch, 0.993; detector collimation, 0.625 mm; rotation time, 0.75s; matrix, 512×512; section thickness, 3 mm; and field-of-view, 250 mm. Only contrast-enhanced CT scans were taken for Radiomics analysis. Non-ionic iodinated contrast medium was used (Ultravist 300, Bayer Schering Pharma, Berlin, Germany, 50ml @ 4 mL/sec). Scans were obtained 70 seconds after intravenous contrast administration.
2.2. Radiomics data extraction
The radiation oncologist annotated the diagnostic CT images separately for the primary tumour region and lymph node/s. Radiomics features were thereafter extracted from these annotated regions. An example of annotation of tumour region and lymph node/s using open source 3D Slicer software, version 4.10.0, is shown in Fig 2.
Illustration of original contrast CT-slices in axial view and its respective annotated slice using 3D slicer; a) Tumour region b) Lymph nodes region.
Pyradiomics toolbox [8], provided as an extension in the 3D-Slicer tool, was used to extract Radiomics features from the gross tumour and lymph nodes. The extracted features were structured in columns as inputs for each model. The data constituted of quantitative features from medical images using various data characterization algorithms, including quantitative information on Region of Interest (ROI) for statistical-based, transform-based, model-based, and shape-based features [3]. The Radiomics dataset included:
- Shape-based features: 14
- Gray-level Dependence matrix features: 14
- Gray-level Cooccurence matrix features: 24
- First-order statistics features:18
- Gray-level run length matrix features:16
- Gray-level size zone matrix: 16
- Neighbouring gray-tone difference matrix features: 5
2.3. Workflow of the study
2.3.1 Structuring of the data.
The clinical factors and Radiomics data were structured in separate non-mutually exclusive columns. The Radiomics dataset was placed alongside the clinical features [6] in separate columns, making a total of 602 columns (388 Clinical and 214 Radiomics features) and 311 rows (samples).
2.3.2 Data pre-processing.
The raw collected data must be pre-processed to make it suitable for ML algorithms. The collected data consisted of missing values, necessitating imputed corrections beforehand. We used iterative imputation techniques [6] to impute the missing values. To handle class-imbalance in the dataset, minority oversampling techniques such as Randomoversampler, Synthetic Minority Oversampling Technique (SMOTE), BorderlineSMOTE, Support Vector Machine SMOTE (SVM SMOTE), and Adaptive Synthetic (ADASYN) were used [6]. To build each model, all five minority oversampling techniques were used; the results from the best-performing technique are reported.
After performing minority oversampling on the original dataset, the synthetic samples were separated from the original samples. The original samples were divided into two parts: majority and minority, based on class-labels. In our case, patients without any recurrence constituted the majority (class-label 0). The training-testing split was then performed on samples from majority-class using 70:30 ratio. To build the ML models, the training dataset was assembled in such a way that it would contain train-split and synthetic samples, and testing dataset would unconditionally include all original minority samples along with test-split.
Thereafter, the training dataset was scaled to zero mean and unit standard deviation using standard scalar function. Due to curse-of-dimensionality resulting from a large number of variables in relation to sample size, feature selection algorithms, namely Sequential Forward Floating selection (SFFS) and Boruta algorithms, were implemented on the dataset. The best k-features based on the highest accuracy (hyperparameter of SFFS) scores were chosen to build ML models.
2.3.3 Fitting model on the training dataset.
Predictive ML models using Radiomics alone and combined Clinical-Radiomics data were then built to predict four clinical outcomes: Distant Recurrence, Locoregional Recurrence, Residual Disease, and development of New Primary. RF, KSVM and XGBoost algorithms were used for fitting onto the training dataset. The selected features were made to fit on the dataset, and the mean training accuracy was calculated for ten iterations.
2.3.4 Evaluation using test-split.
The performance of the designed classifier was evaluated using the testing dataset. Each value of the performance metrics [9] was reported as the mean of ten iterations.
2.3.5 Visualising ROC plots and performance comparison.
Stratified k-fold cross-validation was performed for ten folds, and its plots were visualized for each model to ensure consistency of the performance reported throughout the dataset.
The best-performing ML model for each group- Clinical-only, Radiomics-only, and combined Clinical-Radiomics datasets- was identified based on accuracy, macro and weighted F-1 score, and area under the Receiver Operating Characteristic (ROC) curve (Fig 3). The representative ML model for each of the three dataset groups was then compared to determine if Radiomics could supplant or replace clinical data (S1C in S1 Appendix).
3. Results
A total of 311 patients with HNSCC were included in the study. The mean/median follow-up duration of the cohort was 18 months (3–85.1 months). The details of the dataset are summarized in Table 1. The Radiomics Quality score (RQS) was calculated to be 45% (S1A in S1 Appendix) [10]. The best-performing ML algorithm with each of the three datasets is compared and reported below.
The SFFS algorithm, and utilizing accuracy score for selecting k_features, performed better than Boruta for all three datasets in the optimal feature selection. Also, KSVM performed better than RF and XGBoost while implementing the ML algorithm on the datasets.
It was observed that the model developed using Radiomics alone performed the poorest for all four clinical outcomes. In contrast, the Clinico-Radiomics dataset provided the best predictive performance. The persistence of residual disease on completion of treatment was the most challenging clinical outcome to predict, and the Radiomics-alone dataset performed poorly in this regard.
With Clinico-Radiomic data, the mean training and testing accuracy were calculated to be 100% and 97% for Distant Metastases, 100% and 72% for Locoregional Recurrence, 100% and 99% for development of New Primary, and 100% and 96% for persistent Residual Disease. Sensitivity and specificity were calculated to be 96% and 98% for Distant Metastases, 100% and 82% for Locoregional Recurrence, 94% and 100% for New Primary, and 94% and 100% for Residual Disease, respectively. Mean ROC was calculated to be 99%, 94%, 99%, and 98% for Distant Metastases, Locoregional Recurrence, New Primary, and Residual Disease, respectively. Further, the performance of the individual class of the designed binary classifier model was measured for training and testing datasets using a weighted F1-score for imbalanced classes. It was calculated to be typically over 90%, except for Locoregional Recurrence, where the score was only 70% (Tables 2–5), despite evidence of overfitting. In order to reduce overfitting here, we attempted reiterating after changing learning-rate hyperparameters of SFSS and also of KSVM and RF, with no significant improvement.
The column of the model utilizing only clinical data from our previous study [6] is included for comparison.
The column of the model utilizing only clinical data from our previous study [6] is included for comparison.
The column of model utilizing only clinical data from our previous study [6] is included for comparison.
The column of the model utilizing only clinical data from our previous study [6] is included for comparison.
ROC curves were obtained to visualize the results. Ten-split stratified k-fold cross-validation was performed using the best combination of ML workflow identified to ensure consistency of reported ROC values (Fig 4).
The column of the model utilizing only clinical data from our previous study [6] is included for comparison.
The ROC value using only the clinical dataset was 0.99 (±0.00), 0.96 (±0.02), 0.99 (±0.00) and 0.98 (±0.03), respectively, for Distance Recurrence, Locoregional Recurrence, New Primary, and Residual Disease. Similarly, using the Clinico-Radiomics dataset, the ROC values were 0.99 (±0.00), 0.98 (±0.01), 0.99 (±0.00) and 0.95 (±0.05), respectively. Using only Radiomics, the ROC values were the lowest, with values of 0.74 (±0.10), 0.85 (±0.07), 0.82 (±0.06) and 0.73 (±0.16).
4. Discussion
Accurate prediction of eventual outcomes carries great importance in medicine, where outcomes can be pretty variable across patients, scenarios, and treatments. Radiomics has an unexplored potential in clinical prediction and has been utilized in various degrees in radiotherapy for HNSCC, including tumour segmentation, prognostication, and monitoring of changes in normal tissues following radiotherapy [11]. Recent developments in AI have exponentially enhanced the ability to analyze large volume data, further increasing the potential of Radiomics. However, it has already been reasoned that it would be best to use Radiomics along with additional information, including clinical features, rather than in isolation to maximize the potential of predictive modeling [12]. Therefore, this study was conducted assuming that Radiomics when added to Clinico-Pathological data would improve the performance of ML models in predicting outcomes of HNSCC treated with RT.
Clinico-Radiomic dataset performed the best in all the four outcome predictions referred to in this study. Despite the models working on clinical data alone are already providing excellent predictive abilities in our study, adding Radiomics further improved the predictive ability.
Radiomics have been studied for a myriad of applications across all disciplines involved in HNSCC management, including molecular characterization [13], image segmentation [14], pre-surgical decision making [15], prognostication, improving treatment quality and efficiency, etc. However, most prognostication studies on Radiomics in HNSCC have utilized PET-CT scans and MRIs to a lesser extent, given the wealth of biological and chemical data captured in these images. In contrast, Radiomics has been used on diagnostic CT scans because these are the most widely available scans used to evaluate the extent and planning of treatment. CT scan Radiomics have been studied to a lesser extent with conflicting conclusions. For example, Ger et al., concluded that neither CT nor PET was independently capable of predicting survival in HNSCC [16], while Cozzi et al., were able to successfully predict survival and local control in a retrospective series of 110 patients with HNSCC treated with radiotherapy [17].
However, fewer studies have attempted to combine Radiomics with AI to enhance its potential. For example, a survey by Diamant et al., was able to successfully predict distant metastases in HNSCC by applying CNN to Radiomics data [18]. A systematic review by Giraud et al., summarizes some possible applications in HNSCC [19]. More recent reviews have looked explicitly at this integrated application for predicting disease outcomes and treatment toxicities [20, 21]. Despite promising reports, they all agree on the need for prospective validation studies prior to clinical translation.
The ML algorithms gave the best performance when fitted onto the dataset under optimal hypermeter settings that were obtained using grid search [22]. Also, the feature selection method SFFS [23] automatically selected k_best features from an exhaustive list of variables so that features with the best accuracy scores are chosen for model-building, thereby preventing the curse of dimensionality [24].
While it provides proof-of-concept of integrated Clinical-Radiomic predictive ML models, this study has several limitations. For one, it is a single-center study that relied on retrospective data with small sample size and a considerably limited follow-up duration. Second, it used only contrast CT images, which can add a degree of heterogeneity to the collected Radiomics data. Radiomics naturally demands high-quality, standardized imaging for optimum performance [25]. Factors such as motion artifacts, time-lapse between contrast administration and image acquisition, image processing algorithms, etc., have been shown to affect data. The study also does not work to determining the mechanisms underlying the possible correlation between radiological findings and outcomes. Moreover, as with most other research [26] on Radiomics and AI, the findings need multicentric protocol-constrained imaging and prospective validation prior to routine use. Developing healthcare data management systems with the ability to recognize, classify and segregate data in real-time could greatly expedite the implementation of such AI applications in the clinic.
5. Conclusion
Radiomics features supplemented with clinical features predicted the clinical outcomes of HNSCC treated with radiotherapy to a high degree of accuracy. In contrast, using only Radiomics data as input offered suboptimal performance. For all the models, KSVM performed better than either RF or XGBoost. Weighted average training and testing F1-scores were equally good with clinical and Clinico-Radiomic datasets but were poor with only the Radiomics dataset. We conclude that only Radiomics data could be insufficient in predicting the HNSCC outcomes, while Clinico-Radiomic data can provide predictions of clinical utility. With prospective validation, such predictive models can be of great utility for clinical exploitation.
Acknowledgments
The authors would like to acknowledge Mr. Manjunatha Maiya, Project Manager, Philips Research India, Bangalore, Karnataka, Mr. Prasad R V, Program Manager Philips Research India, Bangalore, Karnataka, and Mr. Shrinidhi G. C., Chief Physicist, Dept. of Radiotherapy, KMC, Manipal for their help and support in the conduct of this research.
References
- 1. Forghani R, Chatterjee A, Reinhold C, Pérez-Lara A, Romero-Sanchez G, Ueno Y, et al. Head and neck squamous cell carcinoma: prediction of cervical lymph node metastasis by dual-energy CT texture analysis with machine learning. Eur Radiol. 2019 Nov;29(11):6172–6181. Epub 2019 Apr 12. pmid:30980127.
- 2. Tsougos I, Vamvakas A, Kappas C, Fezoulidis I, Vassiou K. Application of Radiomics and Decision Support Systems for Breast MR Differential Diagnosis. Comput Math Methods Med. 2018 Sep 23;2018:7417126. pmid:30344618; PMCID: PMC6174735.
- 3. Mayerhoefer ME, Materka A, Langs G, Häggström I, Szczypiński P, Gibbs P, et al. Introduction to Radiomics. J Nucl Med. 2020 Apr;61(4):488–495. Epub 2020 Feb 14. pmid:32060219; PMCID: PMC9374044.
- 4.
Zwanenburg A, Leger S, Vallières M, Löck S. Image biomarker standardisation initiative. arXiv:161207003 [Internet]. 2019;1. Available from: https://arxiv.org/abs/1612.07003
- 5. Peng Z, Wang Y, Wang Y, Jiang S, Fan R, Zhang H, et al. Application of radiomics and machine learning in head and neck cancers. Int J Biol Sci. 2021 Jan 1;17(2):475–486. pmid:33613106; PMCID: PMC7893590.
- 6. Gangil T, Shahabuddin AB, Dinesh Rao B, Palanisamy K, Chakrabarti B, Sharan K. Predicting clinical outcomes of radiotherapy for head and neck squamous cell carcinoma patients using machine learning algorithms. J Big Data [Internet]. 2022;9(1):25. Available from:
- 7. Schoenfeld DA. Sample-Size Formula for the Proportional-Hazards Regression Model. Biometrics [Internet]. 1983 Oct 9;39(2):499–503. Available from: http://www.jstor.org/stable/2531021 pmid:6354290
- 8. Van Griethuysen JJM, Fedorov A, Parmar C, Hosny A, Aucoin N, Narayan V, et al. Computational Radiomics System to Decode the Radiographic Phenotype. Cancer Res. 2017 Nov 1;77(21):e104–e107. pmid:29092951; PMCID: PMC5672828.
- 9. Ferri C, Hernández-Orallo J, Modroiu R. An experimental comparison of performance measures for classification. Pattern Recognit Lett [Internet]. 2009;30(1):27–38. Available from: https://www.sciencedirect.com/science/article/pii/S0167865508002687
- 10. Lambin P, Leijenaar RTH, Deist TM, Peerlings J, de Jong EEC, van Timmeren J, et al. Radiomics: the bridge between medical imaging and personalized medicine. Nat Rev Clin Oncol. 2017 Dec;14(12):749–762. Epub 2017 Oct 4. pmid:28975929.
- 11. Wong AJ, Kanwar A, Mohamed AS, Fuller CD. Radiomics in head and neck cancer: from exploration to application. Transl Cancer Res. 2016 Aug;5(4):371–382. pmid:30627523; PMCID: PMC6322843.
- 12. Holzinger A, Haibe-Kains B, Jurisica I. Why imaging data alone is not enough: AI-based integration of imaging, omics, and clinical data. Eur J Nucl Med Mol Imaging. 2019 Dec;46(13):2722–2730. Epub 2019 Jun 15. pmid:31203421.
- 13. Huang C, Cintra M, Brennan K, Zhou M, Colevas AD, Fischbein N, et al. Development and validation of radiomic signatures of head and neck squamous cell carcinoma molecular features and subtypes. EBioMedicine. 2019 Jul;45:70–80. Epub 2019 Jun 27. pmid:31255659; PMCID: PMC6642281.
- 14. Cardenas CE, Yang J, Anderson BM, Court LE, Brock KB. Advances in Auto-Segmentation. Semin Radiat Oncol [Internet]. 2019;29(3):185–97. Available from: https://www.sciencedirect.com/science/article/pii/S1053429619300104 pmid:31027636
- 15. Wu W, Ye J, Wang Q, Luo J, Xu S. CT-Based Radiomics Signature for the Preoperative Discrimination Between Head and Neck Squamous Cell Carcinoma Grades. Front Oncol [Internet]. 2019;9. Available from: https://www.frontiersin.org/articles/10.3389/fonc.2019.00821 pmid:31544063
- 16. Ger RB, Zhou S, Elgohari B, Elhalawani H, Mackin DM, Meier JG, et al. Radiomics features of the primary tumor fail to improve prediction of overall survival in large cohorts of CT- and PET-imaged head and neck cancer patients. PLoS One [Internet]. 2019;14(9):1–13. Available from: pmid:31536526
- 17. Cozzi L, Franzese C, Fogliata A, Franceschini D, Navarria P, Tomatis S, et al. Predicting survival and local control after radiochemotherapy in locally advanced head and neck cancer by means of computed tomography based radiomics. Strahlenther Onkol. 2019 Sep;195(9):805–818. English. Epub 2019 Jun 20. pmid:31222468.
- 18. Diamant A, Chatterjee A, Vallières M, Shenouda G, Seuntjens J. Deep learning in head & neck cancer outcome prediction. Sci Rep [Internet]. 2019;9(1):2764. Available from: https://doi.org/10.1038/s41598-019-39206-1
- 19. Giraud P, Giraud P, Gasnier A, El Ayachy R, Kreps S, Foy JP, et al. Radiomics and Machine Learning for Radiotherapy in Head and Neck Cancers. Front Oncol. 2019 Mar 27;9:174. pmid:30972291; PMCID: PMC6445892.
- 20. Chinnery T, Arifin A, Tay KY, Leung A, Nichols AC, Palma DA, et al. Utilizing Artificial Intelligence for Head and Neck Cancer Outcomes Prediction From Imaging. Can Assoc Radiol J. 2021 Feb;72(1):73–85. Epub 2020 Jul 31. pmid:32735452.
- 21. Carbonara R, Bonomo P, Di Rito A, Didonna V, Gregucci F, Ciliberti MP, et al. Investigation of Radiation-Induced Toxicity in Head and Neck Cancer Patients through Radiomics and Machine Learning: A Systematic Review. Ahmad N, editor. J Oncol [Internet]. 2021;2021:5566508. Available from: pmid:34211551
- 22. Belete DM, Huchaiah MD. Grid search in hyperparameter optimization of machine learning models for prediction of HIV/AIDS test results. Int J Comput Appl [Internet]. 2022 Sep 2;44(9):875–86. Available from: https://doi.org/10.1080/1206212X.2021.1974663
- 23. Pudil P, Novovičová J, Kittler J. Floating search methods in feature selection. Pattern Recognit Lett. 1994;15(11):1119–25.
- 24.
Verleysen M, François D. The Curse of Dimensionality in Data Mining and Time Series Prediction BT—Computational Intelligence and Bioinspired Systems. In: Cabestany J, Prieto A, Sandoval F, editors. Berlin, Heidelberg: Springer Berlin Heidelberg; 2005. p. 758–70.
- 25. Papadimitroulas P, Brocki L, Christopher Chung N, Marchadour W, Vermet F, Gaubert L, et al. Artificial intelligence: Deep learning in oncological radiomics and challenges of interpretability and data harmonization. Phys Medica [Internet]. 2021;83:108–21. Available from: https://www.sciencedirect.com/science/article/pii/S1120179721001253 pmid:33765601
- 26. Zhai TT, van Dijk LV, Huang BT, Lin ZX, Ribeiro CO, Brouwer CL, et al. Improving the prediction of overall survival for head and neck cancer patients using image biomarkers in combination with clinical parameters. Radiother Oncol. 2017 Aug;124(2):256–262. Epub 2017 Jul 29. pmid:28764926.