https://orcid.org/0000-0003-4879-675X
Figures
Abstract
Background
Globally there are over 1,400 cases of pneumonia per 100,000 children every year, where children in South Asia and Sub-Saharan Africa are disproportionately affected. Some of the cases develop poor treatment outcome (treatment failure or antibiotic change or staying longer in the hospital or death), while others develop good outcome during interventions. Although clinical decision-making is a key aspect of the interventions, there are limited tools such as risk scores to assist the clinical judgment in low-income settings. This study aimed to validate a prediction model and develop risk scores for poor outcomes of severe community-acquired pneumonia (SCAP).
Methods
A cohort study was conducted among 539 under-five children hospitalized with SCAP. Data analysis was done using R version 4.0.5 software. A multivariable analysis was done. We developed a simplified risk score to facilitate clinical utility. Model performance was evaluated using the area under the receiver operating characteristic curve (AUC) and calibration plot. Bootstrapping was used to validate all accuracy measures. A decision curve analysis was used to evaluate the clinical and public health utility of our model.
Results
The incidence of poor outcomes of pneumonia was 151(28%) (95%CI: 24.2%-31.8%). Vaccination status, fever, pallor, unable to breastfeed, impaired consciousness, CBC abnormal, entered ICU, and vomiting remained in the reduced model. The AUC of the original model was 0.927, 95% (CI (0.90, 0.96), whereas the risk score model produced prediction accuracy of an AUC of 0.89 (95%CI: 0.853–0.922. Our decision curve analysis for the model provides a higher net benefit across ranges of threshold probabilities.
Conclusions
Our model has excellent discrimination and calibration performance. Similarly, the risk score model has excellent discrimination and calibration ability with an insignificant loss of accuracy from the original. The models can have the potential to improve care and treatment outcomes in the clinical settings.
Citation: Anteneh ZA, Arega HE, Mihretie KM (2023) Validation of risk prediction for outcomes of severe community-acquired pneumonia among under-five children in Amhara region, Northwest Ethiopia. PLoS ONE 18(2): e0281209. https://doi.org/10.1371/journal.pone.0281209
Editor: Yasir Bin Nisar, World Health Organization, SWITZERLAND
Received: February 23, 2022; Accepted: January 10, 2023; Published: February 15, 2023
Copyright: © 2023 Anteneh et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the manuscript and its Supporting Information files.
Funding: The author(s) received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Pneumonia is a form of an acute respiratory tract infection that causes inflammation of the pulmonary parenchyma (alveoli and bronchioles). It is an acute and lower respiratory tract infection caused by a number of infectious agents, including viruses, bacteria, and fungi, and the most severe form of pneumonia is due to bacterial causes [1, 2]. For the purpose of an intervention, pneumonia is commonly classified into community acquire, hospital acquire, aspiration, and ventilator-associated pneumonia; it is also often classified based on the area of the lung it affects [3]. Pneumonia affects all age groups; however, children are severely and disproportionately affected than any other age group. This could be associated with younger people might have a weakened immunity due to malnutrition and other diseases [4, 5].
Community acquire pneumonia (CAP) in children can be caused by bacteria, viruses, or mixed infections. Pieces of evidence showed that the etiology of CAP in hospitalized children accounted for 33% viral, 19% bacterial, and 33% mixed infections [6, 7].
Pneumonia is the single largest infectious cause of death in children, in 2019 pneumonia claimed the lives of over 800,000 children less than 5 years of age, which implies nearly, 2,200 children die daily. This accounts for 14% of all deaths of children less than five years old but 22% of all deaths in children aged 1 to 5 [1, 8]. Globally, there are over 1,400 cases of pneumonia per 100,000 children, implying 1 pneumonia case per 71 children every year, where children in South Asia and Africa are disproportionately affected [9].
Evidence showed that the prognosis of pneumonia in children is good; most cases of viral pneumonia resolve without treatment; common bacterial pathogens and atypical organisms respond to antimicrobial therapy. Long-term alteration of pulmonary function is rare [10, 11]. Although the definition for severe community acquired pneumonia (SCAP) isn’t uniform; based on WHO guideline for pneumonia control strategy for resource limited settings; severe community acquired pneumonia is defined as children who had chest indrawing with or without fast breathing or children who had any general danger signs such as: inability to breast feed/eat, vomiting everything, grunting, lower chest endowing, altered mental status) that requires referral to the closest higher-level health facility for a high intensity monitoring and treatment [12–14].
A child diagnosed with severe community-acquired pneumonia (SCAP) can be treated with parenteral or oral roots of administration of medicines [15]. Those children diagnosed with SCAP should be treated with parenteral antibiotics at the health facility level [2, 16]. Although patients placed on a protocol-driven pneumonia clinical pathway are more likely to have favorable outcomes, there is a chance that the outcomes of the clinical care could be poor because of malnutrition, immunosuppression, prematurity, co-existing illness, and others [7, 17–19]. In light of the clinical care, it is very important to determine the prognosis and predict the outcomes of CAP to provide an individual case-based intervention for patients with higher risks of poor outcomes of SCAP. Therefore, we developed a prediction model and risk scores to assist the clinical judgment made by the health care workers.
Methods and materials
An institutional based prospective cohort study was conducted among children less than 5 years of age hospitalized with severe community-acquired pneumonia in Amhara Regional State, Northwest Ethiopia. The study was conducted from February to May/ 2020 in public referral hospitals (Felege-Hiwot Comprehensive Specialized Hospital (FHCSH), Tibebe Ghion Specialized Hospital (TGSH), and Debre-Markos Comprehensive Specialized Hospital (DMCSH). FHCSH and TGSH are located in Bahir Dar city, the capital of Amhara Regional State at a distance of 565kms from Addis Ababa capital city of Ethiopia. Both are governmental Hospitals providing inpatient services for children in the pediatric ward and have an emergency outpatient department (EOPD) which can admit for 24-72hours. And DMCSH is Referral Hospital in East Gojam Zone in Debre Markos city. Debre Markos city is the capital of East Gojam located at a distance of 185kms away from Addis Ababa. Similarly, it is a governmental hospital providing both inpatient and outpatient health care services in the region.
The study domain for our study is children from 2–59 months of age diagnosed with severe community-acquired pneumonia based on World Health Organization diagnosing (WHO) criteria of Pneumonia Severity Index (PSI) at the selected hospitals admitted to pediatrics emergency outpatient department or pediatrics wards. In the hospitals, for the purpose of intervention and classifications of pneumonia, SCAP is defined as children who reported chest indrawing with or without fast breathing or children who had any general danger signs such as: inability to breast feed/eat, vomiting everything, grunting, lower chest endowing, altered mental status [13, 15]. Children diagnosed with SCAP but died before the start of medication, and children diagnosed with SCAP at admission, but later whose diagnosis changed at any time during the follow-up were excluded from our study.
The sample size was determined based on the minimum standard of 10 events per candidate predictor considered, thus the formula used was N = (n × 10)/I, where N is the sample size, n is the number of candidate predictor variables, and I is the estimated event rate in the population [20]. We have 18 potential prognostic determinants for severe community-acquired pneumonia, and the prevalence of one of the components for poor outcomes of severe community-acquired pneumonia was 35% [21]. Therefore, the calculated sample size was 514, and adding 10% refusal for participation rate, the total sample size required was 565 children. As a sampling procedure, we allocated the sample size to the three hospitals above based on their previous monthly admission rates taken from DHIS reports. So, 203 cases from FHCSH, 147 cases from TGSH, and 215 cases from DMCSH were selected consecutively.
Outcome measurement
Poor outcomes of severe community-acquired pneumonia said to be occurred if treatment failure or antibiotic change or staying longer in the hospital or death occurred during follow-up days, otherwise, good outcome.
Operational definitions for some terminologies
Treatment failure: treatment failure operationalized as persistence of features of severe pneumonia after initiation of antimicrobial therapy or worsening clinical condition within 48–72 hours of the commencement of the antibiotics [22, 23].
Antibiotic change: is a shift from one drug to the other (first line to second line) after taking at least two doses &, A longer stay in the hospital: staying for more than five days in the hospital regarded as longer [7].
Death: If a child has died during the hospitalization days and confirmed by health professionals.
Complete Blood Count (CBC)—It is a composite variable. Laboratory findings in an increased in the level of platelets, white bloods, neutrophils, or presence of anemia categorized (abnormal), otherwise, the finding is normal.
Prognostic determinants
Sociodemographic factors (age, sex, immunization status, delayed presentation (> 5 days), exclusive breastfeeding, and complementary feeding), clinical at presentation (breathlessness, fever, pallor, cyanosis, grunting, and vomiting), co-morbidities (Human Immune Virus (HIV), diarrhea in the last 2 weeks, cough in the last 2 weeks, stunting and wasting), environmental conditions (cooking place, fuels used for cooking, smokers in the house, family size).
A structured questionnaire was developed based on the literature available on the subject [21, 23–26], and a medical chart used to collect data for patients in the hospitals. Six data collectors and two supervisors were selected. Senior health officers and nurses who trained on Integrated Management of Neonate and Child Illness (IMNCI) were involved in the data collection. Training for one day was provided to the data collectors and supervisors on objectives of the study, steps, and approaches for interviewing, to take accurate and proper measurements of weight, height/length and calculate height for age (H/A), weight for height (W/H) and how to take correct vital signs and physical examinations. The data were collected in phases, the first data were taken on admission date as baseline information, and the second was on the third day about the clinical prognosis and antibiotics taken. It is concerned with the clinical status of the patient, either stable, same, or worsen. The final data were collected on the discharge time. If the child was not discharged, the final data were collected on the sixth to the eighth day, date of discharge, discharging conditions (improved same, none improved or death), antibiotics are taken, and treatment failure and other important information were registered. If the final diagnosis was other than severe community-acquired pneumonia, the case was excluded.
Data processing and analysis
Data were collected using EPI DATA, version 3.02; and were exported to R statistical programming language version 4.0.5 for analysis. Descriptive statistics such as mean, standard deviation (SD), median, interquartile range (IQR), and percentages were performed and presented in tables.
Bivariate logistic regression analysis was performed to show unadjusted associations between each prognostic factor and poor outcomes of severe community-acquired pneumonia and to select potential candidate prognostic determinants for multivariate prediction modeling. Variables with a p-value less than 0.25 in the bivariate regression were retained for multivariate analysis. We performed multicollinearity tests between each determinant using variance inflation factor (VIF), we used VIF>10 as a cut-off point to exclude a variable for multivariate modeling [27].
A backward stepwise logistic regression analysis was performed to come up with the final reduced model. The regression coefficients with their 95% confidence levels and p-values were reported for the models. Model accuracy was checked using the area under the ROC curve (discrimination) and calibration plot (calibration) respectively. Interpretation of discrimination ability of a model; an AUC of 0.5 is worthless, and AUC from 0.80-.90 and 0.90–1.0 is good and excellent model performance respectively [28, 29].
To internally validate our model, a bootstrapping technique was used. For model derivation, we had drawn 1000 random bootstrap samples with replacement. Similar to the original model, the regression coefficients and AUC of the validated model were reported and compared with the original model. We used a decision curve analysis (DCA) to evaluate the clinical and public health impact of the model of standardized net benefit across a range of threshold probabilities (0 to 1) [30].
To develop a simplified and easily applicable prediction score for the outcomes of severe community-acquired pneumonia, each regression coefficient in the validated model was divided by the smallest coefficient and rounded to the nearest integer. We determined the total score for each individual by assigning the points for each variable present and adding them up. For simple interpretation in a clinical setting, we categorized the total risk score into two based on the Youden index (optimal cut-off point). Then, patients were categorized into high-risk or low-risk groups based on the summation of individualized risk scores. Since, a simplification of a risk score might cause loss of information, which might result in some loss in prognostic accuracy; we created a model for the risk score to compare its accuracy with the original one. So, ROC was plotted and an AUC with its 95% confidence level was computed to evaluate the discriminatory power of the scoring system.
Ethics approval and consent to participate
Ethical clearance was obtained from the Ethical Review Committee of Bahir Dar University. Letters were written from Bahir Dar University to each study hospital for a corporation. Permission letters were obtained from each hospital administrative office to conduct the study on the ground. The study participant’s information sheet was attached to the front page of each questionnaire; before proceeding to the data collection process, the caregiver/family of each patient was asked for participation and well-informed. Verbal informed consent was received from the caregiver/family of the patient following explaining the purpose of the study.
Results
Baseline demographic and clinical characteristics of under-five with pneumonia
Regarding to the nature of our dataset, there is no missing data. We aimed for 565 children to be included in our study; however, 539 children were included in this study, giving a response rate of 95%. In this study, 270(50.1%) were females. The median age of the children was 17 months (IQR: 34–8), 200(37.1%) were in the ranges of 2 to 11 months age. One hundred forty two (26.3%), and 65 (12.1%) of children had cyanosis and paleness upon admission. This study also indicated that 239(44.3%) had vomiting, 243(45.1%) were unable to breastfeed, 138 (25.6%) had impaired consciousness, 107 (19.9%) were wasted, and 101(18.7%) were stunted, 83(15.4%) were admitted to ICU.The mean body temperature was 37.6 (±1.0 SD), and the median HCT was 33.2(IQR: 37.6–33.2) and. The median respiratory rate was 56(IQR: 62–48) (Table 1).
The incidence of poor outcomes and prognostic determinants of pneumonia among under-five children
The findings of this study revealed that the proportion of treatment failure, switching treatment, staying longer in the hospital, and death rates were 51(33.6%), 34(22.7%, 61(40.7%) and 5(3%) respectively. However, the overall incidence of poor outcomes of pneumonia was 151(28%) (95%CI: 24.2%-31.8%). We conducted a bivariate logistic regression analysis to identify prognostic determinants of poor outcomes of Pneumonia. Several prognostic determinants were entered in the bivariate modeling to select candidate predictors for the multivariate model. Accordingly, 14 candidate predictors were eligible for further regression analysis (Table 2).
A predictive model for poor outcomes of severe community-acquired pneumonia in children
We entered 14 candidate prognostic determinants from the bivariate model, into original multivariate model. Prognostic determinants such as vaccination status, fever, pallor, unable to breastfeed, impaired consciousness, CBC positive, entered ICU, and vomiting remained in the reduced mode. To this end, we developed a prediction model, and risk scores based on the results of the reduced model (Table 3).
The area under the receiver operating characteristics curve (AUC) of the original model was 0.926(95% confidence interval 0.897 to 0.952) (Fig 1A). The model fitness test had a p-value of 0.176; the calibration curve is nearly 45 degrees, showing that there is no difference between predicted and the observed probabilities (Fig 1B).
Receiver operating characteristics curve for prediction model of the original model (a), Calibration belt for the original model: predicted probability versus observed probability poor outcomes of SCAP in under-five children (b).
Based on the default 0.5 cut off probability, the original model has accuracy (ACC) of 0.899 (10% misclassification rate), sensitivity(S) 0.742, specificity (SP) 0.961, positive predictive value (PV+) 0.882, and negative predictive value (PV-) 0.905. However, based on the optimal cut of point (Youden index) cut off point (S1 Fig) 0.339 probability, the model has accuracy (ACC) of 0.898, sensitivity(S) 0.801,95%CI (0.729, 0.862), specificity (SP) 0.935,95%CI(0.906, 0.958), positive predictive value (PV+) 0.828,95%CI (0.763, 0.882), and negative predictive value (PV-) 0.923, 95%CI (0.889, 0.950). The density plot of the original multivariate model indicated that 28% of the study subjects were with poor outcomes of SCAP (positive cases). The graph with red one represents children with low risk of SCAP, and the blue one was children at high risk of poor outcomes of SCAP. The plot showed some overlap in the model, revealing, it isn’t 100% perfect (S2 Fig).
To avoid over-interpretation and minimize too optimistic results from the original model; we used a bootstrapping technique using mrs package to validate our model. This study used 1000 bootstrap samples with replacement; the corrected AUC was 0.913, 95%CI (0.899, 0.956) and the optimism coefficient for the validated model was 0.0135. The β coefficients from the bootstrapped model produced marginally the same results as the original β coefficients. The calibration plot for the validated model is shown (Fig 2); indicates a very good agreement between predicted and observed probabilities; very slightly that the apparent curve seems to outperform the bias-corrected curve between 0.2 and 0.7 predicted probabilities. Therefore, given the limited optimism, and excellent calibration, the model might perform well in a new sample.
To predict an individual estimated risk of poor outcomes of pneumonia based on linear predictors of validated regression coefficients as: the probability of poor outcomes of pneumonia = 1/1+exp-(-15.64+0.69*fully vaccinated + 0.37*fever+ 2.27*pallor (yes) + 0.68*unable to breastfeed+ 1.84*impaired consciousness + 0.79*CBC positive + 2.04*admitted to ICU + 1.14*Vomiting (yes).
Regarding the decision to use our model, the decision curve outperforms the default strategies (referring all and none) across the entire range of threshold probabilities. This implies that our model has the highest clinical and public health importance. Therefore, decisions made using the model such as safely discharging children with some medications or keeping children for more intensive care in the hospitals has a higher net benefit (Fig 3).
Risk classification using a simplified risk score
For practical utility, we developed a simplified risk score from the validated model. The risk score produced prediction accuracy of an AUC of 0.89 (95%CI: 0.853–0.922); which is nearly a comparable prediction accuracy with the original model, a curve with reddish-purple color (See Fig 4). This reveals that the probability of a randomly selected child with poor outcomes of pneumonia will receive a higher risk score than a randomly selected child without poor outcomes of pneumonia is 89%.
In this study the maximum total risk score is 27; for simple interpretation in the clinical settings, we categorized risk scores into less than nine points (< 9) (low-risk group), and greater or equal to nine points (≥ 9) points (high-risk group) based on Youden index (optimal cut-off point) which corresponds to the probability of 0.333 in the model. Therefore, a child can have a minimum and maximum risk score of 0 and 27 respectively. Out of the total 151(28%) cases of poor outcomes of SCAP, 47(11.5%) were in the low-risk group, and 104(81%) were in the high-risk group (Table 4).
As to the risk score model performance, it had accuracy of 0.861, 95%CI (0.828, 0.889); indicating that the misclassification rate for the risk score model is less than 14%. The model has a Sensitivity 0.715, 95%CI (0.636, 0.786), specificity 0.918, 95%CI (0.886, 0.943), PPV 0.771, 95%CI (0.701, 0.831), NPV 0.892, 95%CI (0.852, 0.925).
Discussions
To halt the loss of millions of young lives from preventable causes including pneumonia, the World Health Organization and UNICEF set an integrated Global Action Plan for the Prevention and Control of Pneumonia and Diarrhea (GAPPD). The plan had targeted less than three child pneumonia deaths per 1,000 live births by 2025. The plan brings together critical services and interventions to create healthy environments, encourages activities that save children from diseases in order to ensure every child has the access to recognized and appropriate preventive and clinical interventions [31]. Similarly, the Sustainable Development Goal (SDG) targeted less than 25 under-five child death per 1,000 live births by 2030 [32].
As part of these targets, accurate risk-stratification tools and models to guide clinical decision-making in the health care settings are very essential. Therefore, we developed a prediction model and risk scores for poor outcomes of SCAP for under-five children. A total of 539 under-five children admitted with SCAP in the hospitals were studied, and the incidence of poor outcomes of SCAP was 28%. We predicted poor outcomes of SCAP by using prognostic determinants that remained in our reduced model (vaccination status, fever, Pallor, unable to breastfeed, impaired consciousness, vomiting on admission, CBC positive, and being admitted to ICU).
The findings of this study revealed that our model has produced discrimination performance of AUC 0.927(95% confidence interval 0.90 to 0.96) with calibration a p-value of 0.132; the calibration curve is almost in 45 degrees, showing a magnificent agreement between predicted and the observed probabilities (Fig 1B). To make internal validation of our study, a bootstrapping technique was used to minimize too optimistic results in the original model. We used 1000 bootstrap samples with replacement; the adjusted AUC was 0.913, 95%CI (0.899, 0.956) and optimism for the validated model was 0.0135. The bootstrapped model has produced marginally the same β coefficients as the original model; and the calibration plot for the validated model is shown (Fig 2); indicates fits well over the ranges of predicted probabilities. Therefore, given the limited optimism, and excellent calibration, the model might perform well in a new sample in the future. To this end, our risk score model has magnificent discrimination ability, with an AUC of 0.89 (95%CI: 0.85–0.92).
To our knowledge, there are no prediction models for poor outcomes of SCAP for children presented with severe community-acquired pneumonia in children less than five in low and middle-income countries; however, several prognostic models were available to predict mortality in children hospitalized with SCAP.
A study conducted by Dana W Flanders and his colleagues, the area under the ROC curve for the Pneumonia Severity Index (PSI) model to predict in-hospital mortality was 0.847, with a calibration p-value of < 0.001 [26]. A prediction study on mortality in children with Respiratory Illness in western Kenya using a modified Respiratory Index of Severity in Children (RISC); the model resulted in an AUC of 0.854 with optimism coefficient of 0.002 [33]. Similarly, an area under the ROC curve of another Pneumonia Severity Index (PSI) study to predict mortality was 0.84 with a p-value < 0.0001 [34]. The findings of these studies showed that the discrimination performance of tools to identify children at greater risk of death from SCAP is very good. The accuracy of our model is slightly higher than a study conducted in Gambia, where the aim of this study was to predict the mortality due to SCAP in children, and the model had an AUC of 0.88 (95% confidence interval: 0.84, 0.91), sensitivity was 0.78 and specificity was 0.77 [35].
However, our model performance is much higher than a study conducted in Malawi to predict in hospital mortality in children from pneumonia, two models were used for external validations; the Respiratory Index of Severity in Children (RISC) and the modified RISC (mRISC) scores in child pneumonia. The models produced discrimination performance of 0.72, and 0.79 respectively [36]. Showing that, the models have produced good discrimination ability among children with pneumonia.
In a study conducted in the US to predict severe outcomes of SCAP, the model had an accuracy of AUC 0.79 (0.77–0.81). This model is slightly different from ours as it strived to predict severe outcomes of SCAP using mechanical ventilation, shock, or death, while ours is aimed at poor outcomes of SCAP following admission using treatment failure, antibiotic change, prolonged hospital stay, and death during hospitalization. In addition, the earlier study used radiologic infiltrate patterns, and microbiologic data in addition to physical and clinical presentations [37].
Biomarker studies conducted to assist clinical decision-making on community-acquired pneumonia were revealed that inflammatory biomarkers including C-reactive protein (CRP), procalcitonin (PCT), cytokines [38–40], and cardiovascular biomarkers such as N-terminal B-type natriuretic peptide (NT-proBNP), proarginin-vasopressin (copeptin), and D-dimer can predict mortality and disease severity in CAP [41–44]. However, these mechanisms are less likely to be implemented and to be used in routine patient care practice in low and middle-income countries.
Most of the available prediction tools and risk scores for SCAP were conducted to predict mortality and identify low-risk patients that are suitable for ambulatory management and admission to the hospitals. However, identification of patients at high risk for poor outcomes by the existing scores remains suboptimal. Hence, it is essential to conduct a repeated evaluation of hospitalized patients to account for the risk of poor outcomes in the course of the illness. So, our prognostic determinants that predicted SCAP were vaccination status, fever, Pallor, unable to breastfeed, impaired consciousness, vomiting on admission, CBC positive, and being admitted to ICU. These variables are easy to be collected on a bedside that might be used to predict the risk of poor outcomes of a given individual patient for a prompt decision to complement the clinical judgment by health care workers.
Strength and limitations
We developed a risk stratification tool by using easily obtainable prognostic determinants to assist clinical decision making that can be utilized in low and middle income settings, where the availability of imaging and lab tests are limited. However, our findings should be used with great caution because this study should be externally validated before use. The prognostic outcome isn’t adjusted for interventions provided.
Conclusions
We developed well-calibrated predicted models that have an excellent discrimination performance for poor outcomes of SCAP among children admitted to the hospitals. Our model has 8 prognostic determinants that can be easily collected from each patient following admission. These variables are easy to be obtained from the diagnostic workup to predict the outcomes of the patients with SCAP. Therefore, our risk score tool could be used in the clinical care settings to identify children at low risk for poor outcomes that could be safely discharged with some medications, and those with a high risk of poor outcomes that require intensive management. External validation is a prerequisite and highly recommended before the implementation of our prediction tool. After external validation, the implementation of our prediction score model can facilitate patient management decisions by offering individualized risk estimates that can be utilized with clinical judgment to enhance the recovery of children with SCAP.
Supporting information
S1 Checklist. TRIPOD checklist: Prediction model development.
https://doi.org/10.1371/journal.pone.0281209.s001
(DOCX)
S1 Fig. Optimal cut off value in ROC curve (Youden index) for the original model.
The black dot on the curve represents cut off the predicted probability (0.339), and sensitivity and specificity values in bracket.
https://doi.org/10.1371/journal.pone.0281209.s002
(TIF)
S2 Fig. Density plot of original model (children with low risk of poor outcomes of SCAP, red in color versus children with high risk of poor outcomes SCAP blue in color).
https://doi.org/10.1371/journal.pone.0281209.s003
(TIF)
S1 Dataset. Full dataset for validation of risk prediction for outcomes of severe community-acquired pneumonia among under-five children in Amhara region, Northwest Ethiopia.
https://doi.org/10.1371/journal.pone.0281209.s004
(SAV)
Acknowledgments
We acknowledge Bahir Dar University for offering us ethical approval to commence our study and the authors are grateful to the FHCSH, TGSH, and DMCSH hospitals that granted permission to conduct this study, data collectors, and supervisors of the study. Finally, our special gratitude goes to the mothers/caregivers of children with CAP who participated in our study.
References
- 1.
World Health Organisation. Pneumonia. Geneva [updated 11 November 2021]; cited 2021 12 January]. Available from: https://www.who.int/news-room/fact-sheets/detail/pneumonia.
- 2.
World Health Organisation. Revised WHO classification and treatment of pneumonia in children at health facilities: evidence summaries. Geneva 2014.
- 3. InformedHealth.org. Pneumonia: Overview. Cologne, Germany: Institute for Quality and Efficiency in Health Care (IQWiG)2018 [updated 2021]; cited 2022 12 January 2022]. Available from: https://www.ncbi.nlm.nih.gov/books/NBK525774/.
- 4. Chisti MJT M. La Vincente S. Graham S. M. Duke T. Pneumonia in severely malnourished children in developing countries—mortality risk, aetiology and validity of WHO clinical signs: a systematic review. Tropical medicine & international health: TM & IH. 2009;14(10):1173–89. Epub 2009/09/24. pmid:19772545.
- 5. Sanjay Sethi. Pneumonia in Immunocompromised People. 2020 [cited January, 2022 13]. Available from: https://www.msdmanuals.com/home/lung-and-airway-disorders/pneumonia/pneumonia-in-immunocompromised-people.
- 6. Cevey-Macherel MG-L A. Gervaix A., et al. Etiology of community-acquired pneumonia in hospitalized children based on WHO clinical guidelines. European journal of pediatrics. 2009;168(12):1429–36. Epub 2009/02/25. pmid:19238436; PubMed Central PMCID: PMC7087130.
- 7. Jakhar SK, Pandey M, Shah D, Ramachandran VG, Saha R, Gupta N, et al. Etiology and Risk Factors Determining Poor Outcome of Severe Pneumonia in Under-Five Children. Indian journal of pediatrics. 2018;85(1):20–4. Epub 2017/10/14. pmid:29027126.
- 8.
UNICEF. Child Health coverage database: pneumonia. [updated Jully 2021; cited 2022 January 13]. Available from: https://data.unicef.org/topic/child-health/pneumonia/.
- 9.
UNICEF. Monitoring the situation of children and women. Pneumonia [cited 2022 September 14]. Available from: https://data.unicef.org/topic/child-health/pneumonia/.
- 10. Chang AB, Ooi MH, Perera D, Grimwood K. Improving the Diagnosis, Management, and Outcomes of Children with Pneumonia: Where are the Gaps? Front Pediatr. 2013;1:29-. pmid:24400275.
- 11.
Muhammad Waseem. Pediatric Pneumonia [updated Jun 05, 2020; cited 2022 January 13]. Available from: https://emedicine.medscape.com/article/967822-overview#a6.
- 12.
World Heath Organisation. Pocket book of hospital care for children: Second edition. Guidelines for the management of common childhood illnesses: World Health Organization; 2013 [cited 2022 17 June]. Available from: https://www.who.int/publications/i/item/978-92-4-154837-3.
- 13. Principi N, Esposito S. Management of severe community-acquired pneumonia of children in developing and developed countries. 2011;66(9):815–22. pmid:20965930
- 14.
World Health O. WHO recommendations on the management of diarrhoea and pneumonia in HIV-infected infants and children: integrated management of childhood illness (IMCI). Geneva: World Health Organization; 2010.
- 15.
World Health Organization. Pocket Book of Hospital Care for Children: Guidelines for the Management of Common Childhood Illnesses Geneva2013. 2nd edition:[Available from: https://www.ncbi.nlm.nih.gov/books/NBK154448/.
- 16.
Organization WH. Handbook: IMCI integrated management of childhood illness. Geneva: World Health Organization; 2005.
- 17. Grimwood K, Chang AB. Long-term effects of pneumonia in young children. Pneumonia. 2015;6(1):101–14. pmid:31641584
- 18. Moschovis PP, Addo-Yobo EO, Banajeh S, Chisaka N, Christiani DC, Hayden D, et al. Stunting is associated with poor outcomes in childhood pneumonia. Tropical medicine & international health: TM & IH. 2015;20(10):1320–8. Epub 2015/06/18. pmid:26083963; PubMed Central PMCID: PMC4729453.
- 19. Nguyen MTN, Saito N, Wagatsuma Y. The effect of comorbidities for the prognosis of community-acquired pneumonia: an epidemiologic study using a hospital surveillance in Japan. BMC Research Notes. 2019;12(1):817. pmid:31856910
- 20. Peduzzi P, Concato J. Kemper E. Holford T. R. Feinstein A. R. A simulation study of the number of events per variable in logistic regression analysis. Journal of clinical epidemiology. 1996;49(12):1373–9. Epub 1996/12/01. pmid:8970487.
- 21. Webb C, Ngama M, Ngatia A, Shebbe M, Morpeth S, Mwarumba S, et al. Treatment failure among Kenyan children with severe pneumonia—a cohort study. Pediatr Infect Dis J. 2012;31(9):e152–e7. pmid:22692700.
- 22. Menendez R, Torres A. Treatment failure in community-acquired pneumonia. Chest. 2007;132(4):1348–55. Epub 2007/10/16. pmid:17934120.
- 23. Harris MC J. Coote N. Fletcher P. Harnden A. McKean M. Thomson A. British Thoracic Society guidelines for the management of community acquired pneumonia in children: update 2011. Thorax. 2011;66 Suppl 2:ii1–23. Epub 2011/10/19. pmid:21903691.
- 24. Aujesky D, Fine MJ. The Pneumonia Severity Index: A Decade after the Initial Derivation and Validation. Clinical Infectious Diseases. 2008;47(Supplement_3):S133–S9. pmid:18986279
- 25. Chalmers JD, Singanayagam A, Akram AR, Mandal P, Short PM, Choudhury G, et al. Severity assessment tools for predicting mortality in hospitalised patients with community-acquired pneumonia. Systematic review and meta-analysis. Thorax. 2010;65(10):878. pmid:20729231
- 26. Flanders WD, Tucker G, Krishnadasan A, Martin D, Honig E, McClellan WM. Validation of the pneumonia severity index. Importance of study-specific recalibration. J Gen Intern Med. 1999;14(6):333–40. pmid:10354252.
- 27. James GWD, Hastie T, Tibshirani R. An Introduction to Statistical Learning: With Applications in R. 2017 ed. ed. s, editor: Springer; 2013.
- 28.
DW Hosmer SL. Area under the ROC curve. Applied Logistic Regression. New York: John Wiley and Sons; 2002. p. 160–4.
- 29.
Tape TG. The Area Under an ROC Curve. Interpreting Diagnostic Tests. [cited 2022 25 January]. Available from: http://gim.unmc.edu/dxtests/Default.htm.
- 30. Vickers AJ, Elkin EB. Decision curve analysis: a novel method for evaluating prediction models. Med Decis Making. 2006;26(6):565–74. pmid:17099194.
- 31. Shamim Qazi SA, Rachel MacLean, et al. Ending preventable child deaths from pneumonia and diarrhoea by 2025. Development of the integrated Global Action Plan for the Prevention and Control of Pneumonia and Diarrhoea. Arch Dis Child. 2015;100(Suppl 1):s23–s8. pmid:25613963
- 32. World Health Organisation. Sustainable Development Goals (SDGsTarget 3.2) World Health Organisation; 2020 [cited 2022 17 January]. Available from: https://www.who.int/health-topics/sustainable-development-goals#tab=tab_3.
- 33. Emukule GO, McMorrow M, Ulloa C, Khagayi S, Njuguna HN, Burton D, et al. Predicting mortality among hospitalized children with respiratory illness in Western Kenya, 2009–2012. PloS one. 2014;9(3):e92968. Epub 2014/03/29. PubMed Central PMCID: PMC3965502. pmid:24667695
- 34. Fine MJ, Hanusa B. H., Lave J. R., et al. Comparison of a disease-specific and a generic severity of illness measure for patients with community-acquired pneumonia. J Gen Intern Med. 1995;10(7):359–68. Epub 1995/07/01. pmid:7472683.
- 35. Alexander Jarde DJJ, Mackenzie Grant A. Development and validation of a model for the prediction of mortality in children under five years with clinical pneumonia in rural Gambia. 2021.
- 36. Hooli S, Colbourn T, Lufesi N, Costello A, Nambiar B, Thammasitboon S, et al. Predicting Hospitalised Paediatric Pneumonia Mortality Risk: An External Validation of RISC and mRISC, and Local Tool Development (RISC-Malawi) from Malawi. PloS one. 2016;11(12):e0168126. Epub 2016/12/29. pmid:28030608; PubMed Central PMCID: PMC5193399.
- 37. Williams DJZ Y. Grijalva C. G., et al. Predicting Severe Pneumonia Outcomes in Children. Pediatrics. 2016;138(4). Epub 2016/10/01. pmid:27688362; PubMed Central PMCID: PMC5051209 conflicts of interest to disclose.
- 38. Nastasijević Borovac DRP T. Pejčić T., et al. Role of D-dimer in predicting mortality in patients with community-acquired pneumonia. Medicinski glasnik: official publication of the Medical Association of Zenica-Doboj Canton, Bosnia and Herzegovina. 2014;11(1):37–43. Epub 2014/02/06. pmid:24496339.
- 39. Mohamed GBS, Madeha Abdellah Abdelhakeem, Abdelhakeem Abdelmohsen, et al Predictive value of copeptin as a severity marker of community-acquired pneumonia. Electron Physician. 2017;9(7):4880–5. pmid:28894549.
- 40. Li JY H. Zhao L. B-type natriuretic peptide in predicting the severity of community-acquired pneumonia. World journal of emergency medicine. 2015;6(2):131–6. Epub 2015/06/10. pmid:26056544; PubMed Central PMCID: PMC4458473.
- 41. Huang DTW, Lisa A. Kellum , John A., et al. Risk prediction with procalcitonin and clinical rules in community-acquired pneumonia. Ann Emerg Med. 2008;52(1):48–58.e2. Epub 2008/03/17. pmid:18342993.
- 42. Menéndez RM R. Reyes S., et al. Biomarkers improve mortality prediction by prognostic scales in community-acquired pneumonia. Thorax. 2009;64(7):587–91. Epub 2009/01/10. pmid:19131448.
- 43. Principi NE, Susanna . Biomarkers in Pediatric Community-Acquired Pneumonia. Int J Mol Sci. 2017;18(2):447. pmid:28218726.
- 44. Wu JJ Y. U. Hailong Li, et al. Evaluation and significance of C-reactive protein in the clinical diagnosis of severe pneumonia. Exp Ther Med. 2015;10(1):175–80. Epub 2015/05/12. pmid:26170931.