http://orcid.org/0000-0001-7227-8261
Figures
Abstract
Translational Statistics proposes to promote the use of Statistics within research and improve the communication of statistical findings in an accurate and accessible manner to diverse audiences. When statistical models become more complex, it becomes harder to evaluate the role of explanatory variables on the response. For example, the interpretation and communication of the effect of predictors in regression models where interactions or smoothing splines are included can be challenging. Informative graphical representations of statistical models play a critical translational role; static nomograms are one such useful tool to visualise statistical models. In this paper, we propose the use of dynamic nomogram as a translational tool which can accommodate models of increased complexity. In theory, all models appearing in the literature could be accompanied by the corresponding dynamic nomogram to translate models in an informative manner. The R package presented will facilitate this communication for a variety of linear and non-linear models.
Citation: Jalali A, Alvarez-Iglesias A, Roshan D, Newell J (2019) Visualising statistical models using dynamic nomograms. PLoS ONE 14(11): e0225253. https://doi.org/10.1371/journal.pone.0225253
Editor: Raffaella Calabrese, University of Edinburgh, UNITED KINGDOM
Received: March 12, 2019; Accepted: October 31, 2019; Published: November 15, 2019
Copyright: © 2019 Jalali et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: Dataset are freely available in the Comprehensive R Archive Network (CRAN) Repository. The titanic dataset is in the ‘PASWR’ package (https://github.com/cran/PASWR/raw/master/data/titanic3.rda), the crabs dataset is in the ‘glm2’ package (https://github.com/cran/glm2/raw/master/data/crabs.rda), the ragweed data is in the ‘SemiPar’ package (https://github.com/cran/SemiPar/raw/master/data/ragweed.txt.gz) and the lung dataset is in the ‘survival’ package (https://github.com/cran/survival/raw/master/data/lung.rda). Also, the dynamic nomogram application for Titanic example is freely available on https://dynnom.shinyapps.io/titanic/.
Funding: A. Jalali received funding through the Health Research Board (https://www.hrb.ie/funding/) via HRB Clinical Research Facility, Galway (http://www.nuigalway.ie/hrb_crfg/). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Translational Medicine, within biomedical and public health research domains, is defined as the convergence of basic and clinical research with the aim to transfer knowledge on the benefits and risks of therapies. The concept of ‘Translational Statistics’ was proposed [1] to facilitate the integration of biostatistics within clinical research in order to enhance communication of statistical research findings in an accurate and accessible manner to diverse audiences (e.g. policy makers, patients and the media).
The use of appropriate graphics is central to all areas of statistical research. For example, the graphical representation of data is necessary to provide a way of assessing at least parts of any assumed statistical model before engaging in formal analysis and to aid in the presentation and understanding of results and conclusions. Nomograms have been used to visualise statistical models, playing the role of a graphical ‘predict’ function, facilitating the calculation of a point estimate of the response variable for a particular set of values of the explanatory variables. In this paper, we propose the use of dynamic nomograms as a visualisation and translational tool to further aid the communication of the results of a statistical analysis to a non-statistical audience. We introduce the R package DynNom for generating dynamic nomograms (as Shiny objects) for a variety of linear and non-linear models. In theory, most models presented in the literature could have an accompanying web address to direct the reader to the corresponding dynamic nomogram allowing them to ‘interact’ with the model to gain insight into the effect of each explanatory variable on the primary response of interest.
Nomograms
The field of nomography was invented by the French mathematician Maurice d’Ocagne in 1880 to provide engineers with fast graphical calculation tools for complicated formulas to a practical level of precision [2, 3]. An early use of nomograms is attributed to [4], where they were employed in mathematics for calculating elementary arithmetic, quadratic equations and trigonometrical functions. One of the first nomograms in statistics was used to calculate the coefficient of variation, interval estimates for a mean and the comparison of two means and variances [5]. Other examples of nomograms in this domain are the chi-square test nomogram (Fig 1), Altman’s nomogram to calculate sample size or power [6] and Fagan’s nomogram [7] for the applications of Bayes’s theorem. Fagan’s nomogram, widely used in the context of diagnostic tests, was recently reproduced as the Held’s nomogram for the calculation of post-test probabilities [8, 9].
To use the nomogram draw a straight line from the value of the chi-square test statistic through the required degrees to read off the corresponding p-value.
Nomograms have been widely used in applications in science, engineering, the military, clinical research and epidemiology; see [4, 11]. The use of nomograms in clinical decision making includes prediction of prostate cancer severity [12, 13], and the Partin nomogram (known as Partin Table) [14] which has been used mainly as a prognostic tool over the last two decades for the prediction of pathological stage in prostate cancer.
More recently, nomograms have been used to visualise statistical models. A nomogram generated from a model plays the role of a graphical ‘predict’ function facilitating the calculation of a point estimate of the response variable for a particular set of values of the explanatory variables [15, 16]. It consists of rulers for each predictor (X) along with two rulers to convert the total points to the desired risk scale. The length of the ith predictor ruler provides a visual representation of the relative effect sizes β) of each explanatory variable in the model. It is defined in the Eq 1, which is calculated as a scaled version of the proportion of the predictor’s contribution divided by the maximum predictor contribution:
(1)
Software is available to create nomograms for statistical models in SAS [17], Stata [18], Python [19] and as online tools for constructing simple JAVA-based interactive nomograms [20] as well as the rms [21] and hdnom [22] packages in R.
The rms package in R [21] includes the nomogram function to generate nomograms from a fitted statistical model. For example, the code below creates a nomogram from a logistic regression model used to model the relationships between age, gender and passenger class on surviving the Titanic disaster (data are included in the PASWR package [23]).
R> data(titanic3)
R> t.data <- datadist(titanic3)
R> options(datadist = “t.data”)
R> fit <- lrm(survived ~ age + pclass + sex, data = titanic3)
R> plot(nomogram(fit, fun = function(x)plogis(x)))
The resulting nomogram is given in Fig 2. The horizontal line at the top labelled ‘points’ allows the effect size of each variable to be assessed, including direct visualisation of the potential variation in the effect. The points measure the contribution of the values/levels of the explanatory variable of interest on the ‘linear predictor’ scale. The total points are then mapped from the ‘linear predictor’ to obtain the ‘predicted value’, which in this case is a probability. It is clear from the graph on the top that all three explanatory variables make similar contributions to the linear predictor, over the spread of value/levels that each can take. For example, in the main effects model presented the effect of gender on the probability of survival is comparable to a change in the age of 70 years. Such an insight is less evident from the estimated coefficients. The model represented at the bottom in Fig 2 incorporates higher order interactions and suggests that the effect of age is most prominent in 2nd class males. As an example, the superimposed dashed lines correspond to a 40-year-old male passenger with a 1st class ticket. The estimated probability of survival is calculated as 0.41 based on the main effects model and 0.35 when all two-way interactions are included.
A logistic regression model containing age, sex and passenger class as main effects is displayed on top while a model including all their two-way interactions is shown at the bottom.
More recent developments in this area include the coloured-based nomogram in the VRPM package [24] proposed by Belle and Caster in 2015 [25]. Fig 3 represents a colour-based nomogram for the main effect model fitted to the Titanic data. The survival probability is calculated by summing up the contribution from each covariate/factor when matched to the colour legend. The effect of gender relative to age is again evident. The accuracy of any prediction depends on the users’ ability to match colours accordingly.
When modelling a binary outcome (e.g. using logistic regression), modelling the log odds is mathematically attractive, summarising a treatment effect as an odds ratio may be misleading [26, 27]. A nomogram is helpful as the predicted response is given as a probability.
It has been argued [28] that a summary quoting the underlying probabilities is more informative than one based on ratios of odds or probabilities. Odds and probability are often (mistakenly) used interchangeably where large and small odds ratios are interpreted as meaningful without reference to the underline probability of the event of interest. Similarly, in a logistic regression model the importance of an explanatory variable is often assessed (incorrectly) by the magnitude of the regression coefficient while ignoring the scale of the variable (e.g. hours, minutes, days) or by the p-value while ignoring the sample size. Given that the response variable of interest is a binary outcome, the effect of each variable on the response may be more easily translated if represented on a probabilistic scale.
Nomograms achieve this and provide an attractive and useful graphical summary of a logistic regression model with the ability to make predictions as probabilities. Static Nomograms can become cumbersome to use as models become more complex (e.g. higher order interactions) or the inclusion of smoothers [29], and when visualising uncertainty (e.g. interval estimates). One solution is to create dynamic nomograms as interactive applications to visualise statistical models such that effect sizes can be assessed graphically, by visualising the predicted value and corresponding confidence interval automatically.
Dynamic nomograms
The emergence of the rpanel [30] and Shiny packages [31] for R allowed the creation of interactive, user-friendly graphical interfaces to be displayed either in a standalone window or delivered as a webpage.
The DynNom package [32] is built to generate dynamic nomograms as a Shiny application for a variety of statistical models to allow a reader to interact with the model in a user-friendly manner. The package can be install from the CRAN repository using the following command:
R> install.packages(“DynNom”)
For example, the R code below will create a dynamic nomogram, using a simple function, for a logistic regression model of the Titanic data.
R> fit <- glm(survived ~ (age + pclass + sex) ^ 3, titanic3, family = “binomial”)
R> DynNom(fit)
The dynamic nomogram displays sliders for covariates (bounded by their observed ranges) and drop-down boxes for factors. The predict function maps the appropriate inverse link function (η = g(μ)) from the generalised linear model object to generate predicted values (on the scale of the response variable) and corresponding 95% confidence intervals (see Fig 4). The predicted value and corresponding confidence interval are plotted and presented in the tooltip labels and the ‘Numerical Summary’ tab. Further, a formatted model output summary is displayed in the ‘Model Summary’ tab.
The plot represents survival probability (with 95% confidence interval) of 30-year-old females with different ticket classes. The actual explanatory values and their corresponding predictions are given in the ‘Numerical Summary’ tab in S1 Fig, and the ‘Model Summary’ tab is provided in S2 Fig.
Predicted quantities are accompanied by measures of uncertainty [33] in the form of confidence intervals. The confidence interval for the mean response will quantify the uncertainty around the estimation of the mean response for a given set of explanatory variables. In generalised linear model models, for example, an approximate 100(1 − α)% confidence interval for the mean response for a given new observation () is calculated as:
(2)
where x is a vector of observed predictors,
is the linear equation estimates,
indicates the corresponding critical value and Σ denotes the covariance matrix. The confidence interval is estimated in the linear form; however, for a clear interpretation, it is best to apply the inverse link transformation g−1(.) (see Table 1) to the confidence limits in formula (2) [34].
Two main functions of the package are DynNom and DNbuilder functions. The former extracts the model class from the fitted model to build a dynamic nomogram as a shiny app which computes and displays the point and interval estimate using Eq 2 while the latter generates the files necessary to deploy the app.
The DynNom package supports model objects created by the lm, glm and coxph functions in R. It also supports ols, Glm, lrm and cph models in the rms package which allow the inclusion of smoothing splines as well as gam functions from mgcv [35] and gam packages [36]. When applied to a Cox proportional hazards model object, the estimated survival function will be displayed with alpha-blending (i.e. thickness of the line reflex the number of subjects over time) to represent the underlying uncertainty of the estimates. Table 2 summarises all the model objects supported by DynNom which cover a wide variety of the most common models fitted depending on the type of response variable of interest.
Examples
For illustration, dynamic nomograms for a Poisson response, a linear regression incorporating smoothing splines and a Cox proportional hazards model for time-to-event data are presented in the following sections. Examples involving Gamma regression and generalized additive model objects are not presented for brevity as the same procedure applies.
Modelling a count response
The ‘crabs’ dataset [37] (available in the glm2 package [38]) is used as an example of visualising a Poisson regression model. This dataset comprises 173 female horseshoe crabs, each of which had at least one male crab attached to it in their nest. This study aims to investigate factors affecting the number of male partners in addition to the female’s primary partner (called satellites). The explanatory variables include the width of the female (Width) and a binary factor indicating whether the female colouring is dark or not (Dark). A Poisson regression model has been used previously [37] to model the relationship between the ‘Width’ and ‘Dark’ on the number of partners. The summary of the fitted Poisson model is given in Table 3, where the effect of Width and Dark are identified as significant (at the 5% level).
The accompanying dynamic nomogram (Fig 5) can be created as follows:
R> fit2 <- glm(Satellites ~ Width + Dark, family = poisson(log), data = crabs)
R> DynNom(fit2)
The plot displays the predicted number of additional male partners of female crabs of 24, 28 or 32 centimetres width of either dark or light colour. The actual explanatory values/levels and predictions are given in the ‘Numerical Summary’ tab in S3 Fig.
The estimates displayed in Fig 5 are for the predicted (mean) response for scenarios covering the two levels of crab colour and three values of Width. The joint effects of ‘Width’ and ‘Dark’ (and corresponding uncertainty estimates) on the response are visually evident which would be more difficult to glean from the model summary.
Modelling a continuous response with a smoothing spline
Interpreting the role explanatory variables play can be difficult in a model when the functional form of an explanatory is modelled as a smooth function. For example, the ‘ragweed’ dataset, introduced in [39] is an example where a regression model can be used for a continuous response incorporating a smoothing spline (data is available in the SemiPar package [40]). The aim is to forecast the ragweed pollen level as a function of the temperature, rain, wind speed forecast for the following day and pollen season day. The corresponding boxplot and the scatterplot (Fig 6) illustrate the marginal relationships between the explanatory variables and the (transformed) response, the square root of ragweed level.
The nonlinear relationship between the day in pollen season and the mean response is modelled using a restricted cubic spline function, with temperature, and wind speed as (linear) predictors and rain presence as a factor. The following code fits the model using the ols function in the rms package (Table 4) and creates the dynamic nomogram shown in Fig 7.
R> ragweed$sqrtragweed <- sqrt(ragweed$ragweed)
R> dd <- datadist(ragweed)
R> options(datadist = “dd”)
R> fit3 <- ols(sqrtragweed ~ rain + temperature + wind.speed + rcs(day.in.seas, 5), data = ragweed)
R> DynNom(fit3)
The ′, ″, ‴, ‴′ represent different knot effects in the restricted cubic splines specified for pollen season day.
The plot displays the predicted response for ‘day in season’ set at 1, 11, 21, 31, 41, 51, 61, 71, 81 and 91 respectively while fixing all other explanatory variables at the mean/mode as appropriate. The actual explanatory values/levels and predictions are given in the ‘Numerical Summary’ tab in S4 Fig.
The need for a non-linear function for the day in season variable appears apparent where the effect increases for up to 25 days in the season and then decreases from that time point onwards. Interpreting the effect of this covariate using the model summary alone, while adjusting for rain, temperature and wind speed, is less clear.
The effect of ‘days in season’ is arguably more accessible using a dynamic nomogram (Fig 7) by choosing values of this covariate ranging from low to high and setting the remaining explanatory variables at their mode (for factors) and means (for covariates).
Modelling a time-to-event response
Classical graphical summaries for time-to-event data include plots of the estimated survival or hazard functions. When modelling the effect of covariates and factors on time-to-event data, the Cox proportional hazards model is a popular choice as the underlying survival distribution does not need to be specified. The interpretation of the effect of explanatory variables on the ‘risk’ of the event of interest is often made using hazard ratios or by using a plot of the estimated survival function conditioning on the set of explanatory variables of interest. For example, the lung cancer dataset in the survival package [41] includes the time to death for 228 advanced lung cancer patients where gender, age, weight loss and daily activity performance scores (such as ECOG) were recorded as potentially useful explanatory variables. The code needed to fit a Cox proportional hazards model and the corresponding dynamic nomogram to investigate the effect of these explanatory variables on time to death is displayed below.
R> model <- coxph(Surv(time, status) ~ age + wt.loss + ph.ecog + sex, data = lung)
R> DynNom(model)
The summary of the fitted Cox proportional hazards model is given in Table 5, where the effect of each explanatory variable is given on the log scale and as a hazard ratio.
The dynamic nomogram (Fig 8) presents the estimated survival function for a given set of values of the explanatory values which avoid the use of hazard ratios and interpretation of effects that act multiplicatively. Rather than clutter the graph by including the corresponding interval estimates for each survival function, the degree of uncertainty in the estimate is highlighted graphically by adjusting the colour transparency (alpha-blending) to represent the number of subjects at risk over time. If required the predicted survival at any given follow-up time point can be generated using the ‘Predicted Survival’ tab.
The Kaplan-Meier plot displays survival curves correspond to 55 and 60 years old males/females (ECOG and weight loss set at their mean). The predicted survival time (with 95% confidence interval) at 250 days is given in the ‘Predicted survival’ tab (S5 Fig) and the actual explanatory values/levels and predictions are given in the ‘Numerical Summary’ tab (S6 Fig).
Publishing dynamic nomograms
In order to make a dynamic nomogram accessible to a wider audience additional functions (i.e. DNbuilder family functions) are provided to generate the files necessary (i.e. ui.r, server.r and global.r) to publish the dynamic nomogram as illustrated below:
R> library(PASWR)
R> library(DynNom)
R> data(titanic3)
R> fit <- glm(survived ~ (age + pclass + sex) ^ 3, titanic3, family = “binomial”)
R> DNbuilder(fit)
The corresponding files can then be uploaded to a server running Shiny (e.g. https://www.shinyapps.io/) which facilitates the deployment of Shiny applications. For example, the dynamic nomogram built for the analysis of the Titanic data described above is published at https://dynnom.shinyapps.io/titanic/. Examples of the use of the DynNom package (and the DNbuilder function in particular) to generate linked dynamic nomograms to statistical models published in research articles includes [42–47].
All models presented in the literature could in theory and practice have an accompanying web address to direct the reader to the corresponding dynamic nomogram allowing them to interact with the model and identify the exact nature of the model fitted. The ‘Model Summary’ tab provides the (native) model summary from R allowing an examination of the precise model fitted and accompanying model summaries (e.g. R2, AIC etc.), which makes it clear as to the actual model (and its parametrisation) fitted.
Conclusion
As inferential statistical methods become more computational, the models arising are increasingly complex and are often hard to interpret and translate, in particular for non-statisticians. Dynamic nomograms are one such translational tool to help address this issue. They provide a visual representation of a statistical model where the importance of each explanatory variable, in terms of their effect size, becomes more evident through the assessment of the change in the predicted value. This is particularly useful for models containing modifiable risk factors.
The ability to accompany a model summary in a journal with a link to the corresponding dynamic nomogram has the potential to make models more translatable, and analyses more transparent in terms of fulfilling the recommendations of making all research understandable and reproducible.
Supporting information
S1 Fig. The ‘Numerical Summary’ tab of the dynamic nomogram in Fig 4.
https://doi.org/10.1371/journal.pone.0225253.s001
(TIF)
S2 Fig. The ‘Model Summary’ tab of the dynamic nomogram in Fig 4.
https://doi.org/10.1371/journal.pone.0225253.s002
(TIF)
S3 Fig. The ‘Numerical Summary’ tab of the dynamic nomogram in Fig 5.
https://doi.org/10.1371/journal.pone.0225253.s003
(TIF)
S4 Fig. The ‘Numerical Summary’ tab of the dynamic nomogram in Fig 7.
https://doi.org/10.1371/journal.pone.0225253.s004
(TIF)
S5 Fig. The ‘Predicted survival’ tab of the dynamic nomogram in Fig 8.
https://doi.org/10.1371/journal.pone.0225253.s005
(TIF)
S6 Fig. The ‘Numerical Summary’ tab of the dynamic nomogram in Fig 8.
https://doi.org/10.1371/journal.pone.0225253.s006
(TIF)
Acknowledgments
The authors gratefully acknowledge the many helpful contributions made by Professor John Hinde and Professor Martin O’Donnell.
References
- 1.
Newell J, Jalali A, Alvarez-Iglesias A, O’Donnell M, Hinde J. Translational Statistics and Dynamic Nomograms. In: 34th Conference on Applied Statistics in Ireland; 2014. p. 73–74.
- 2.
Allcock HJ, Jones JR, Michel J. The nomogram: The theory and practical construction of computation charts. Pitman; 1950.
- 3.
Evesham HA. The history and development of nomography. Docent Press; 2010.
- 4.
Brodetsky S. A First Course in Nomography. Bell’s mathematical series. G. Bell and sons, Limited; 1925.
- 5. Broadbent S. Some Uses of the Nomogram in Statistics. Journal of the Royal Statistical Society Series C (Applied Statistics). 1954;3(1):33–43.
- 6. Altman DG. Statistics and ethics in medical research: III How large a sample? British Medical Journal. 1980;281:1336–1338. pmid:7437789
- 7. Fagan T. nomogram for Bayes theorem. New England Journal of Medicine. 1975;293(5):257. pmid:1143310
- 8. Held L. A nomogram for P values. BMC Medical Research Methodology. 2010;10(1):21. pmid:20233437
- 9. Safari S, Baratloo A, Elfil M, Negida A. Evidence Based Emergency Medicine; Part 4: Pre-test and Post-test Probabilities and Fagan’s nomogram. Emergency. 2016;4(1):48–51. pmid:26862553
- 10. Boyd WC. A Nomogram for Chi-Square. Journal of the American Statistical Association. 1965;60(309):344–346.
- 11.
Možina M, Demšar J, Kattan M, Zupan B. Nomograms for visualization of naive Bayesian classifier. In: European Conference on Principles of Data Mining and Knowledge Discovery. Springer; 2004. p. 337–348.
- 12. Kattan MW. Nomograms are superior to staging and risk grouping systems for identifying high-risk patients: preoperative application in prostate cancer. Current Opinion in Urology. 2003;13(2):111–116. pmid:12584470
- 13. Steyerberg E, Roobol M, Kattan M, Van der Kwast T, De Koning H, Schröder F. Prediction of Indolent Prostate Cancer: Validation and Updating of a Prognostic Nomogram. The Journal of Urology. 2007;177(1):107–112. pmid:17162015
- 14. Partin A, Yoo J, Carter H, Pearson J, Chan D, Epstein J, et al. The use of prostate specific antigen, clinical stage and Gleason score to predict pathological stage in men with localized prostate cancer. The Journal of urology. 1993;150(1):110–114. pmid:7685418
- 15.
Levens AS. Nomography. Fearon Publishers, Lear Siegler, Incorporated; 1937.
- 16.
Banks J. Nomograms. In: Encyclopedia of statistical sciences. Wiley Online Library; 2004.
- 17.
Yang D. Build Prognostic Nomograms for Risk Assessment Using SAS®. In: Proceedings of SAS Global Forum; 2013.
- 18. Zlotnik A, Abraira V, et al. A general-purpose nomogram generator for predictive logistic regression models. Stata Journal. 2015;15(2):537–546.
- 19.
Doerfler R. Creating Nomograms with the PyNomo Software. 2009;.
- 20.
Jones TB. An online tool for creating custom, interactive nomograms; 2009. Available from: www.ece.rochester.edu/~jones/NomoDevel/nomogram.
- 21.
Harrell Jr FE. rms: Regression Modeling Strategies; 2017. Available from: https://CRAN.R-project.org/package=rms.
- 22. Xiao N, Xu QS, Li MZ. hdnom: Building Nomograms for Penalized Cox Models with High-Dimensional Survival Data. bioRxiv. 2016;.
- 23.
Arnholt AT. PASWR: PROBABILITY and STATISTICS WITH R; 2012. Available from: https://CRAN.R-project.org/package=PASWR.
- 24.
Van Belle V. VRPM: Visualizing Risk Prediction Models; 2017. Available from: https://CRAN.R-project.org/package=VRPM.
- 25. Van Belle V, Van Calster B. Visualizing risk prediction models. PloS one. 2015;10(7):e0132614. pmid:26176945
- 26. Agrawal D. Inappropriate Interpretation of the Odds Ratio: Oddly Not That Uncommon. Pediatrics. 2005;116(6):1612–1613. pmid:16322199
- 27. Rico-Villademoros F. On the Interpretation of Odds Ratios. The Clinical journal of pain. 2012;28(5):462. pmid:22569219
- 28. Davies HTO, Crombie IK, Tavakoli M. When can odds ratios mislead? BMJ. 1998;316(7136):989–991. pmid:9550961
- 29.
Harrell F. Regression modeling strategies: with applications to linear models, logistic and ordinal regression, and survival analysis. Springer; 2015.
- 30. Bowman A, Crawford E, Alexander G, Bowman R. rpanel: Simple Interactive Controls for R Functions Using the tcltk Package. Journal of Statistical Software, Articles. 2007;17(9):1–18.
- 31.
Chang W, Cheng J, Allaire J, Xie Y, McPherson J. shiny: Web Application Framework for R; 2017. Available from: https://CRAN.R-project.org/package=shiny.
- 32.
Jalali A, Roshan D, Alvarez-Iglesias A, Newell J. DynNom: Visualising Statistical Models using Dynamic Nomograms; 2019. Available from: https://CRAN.R-project.org/package=DynNom.
- 33.
McCullagh P, Nelder JA. Generalized Linear Models, Second Edition. Chapman & Hall/CRC Monographs on Statistics & Applied Probability. Taylor & Francis; 1989.
- 34.
Madsen H, Thyregod P. Introduction to General and Generalized Linear Models. Chapman & Hall/CRC Texts in Statistical Science. CRC Press; 2010.
- 35. Wood SN. Fast stable restricted maximum likelihood and marginal likelihood estimation of semiparametric generalized linear models. Journal of the Royal Statistical Society (B). 2011;73(1):3–36.
- 36.
Hastie TJ. gam: Generalized Additive Models; 2018. Available from: https://CRAN.R-project.org/package=gam.
- 37.
Agresti A, Kateri M. Categorical data analysis. Springer; 2011.
- 38. Marschner IC. glm2: Fitting generalized linear models with convergence problems. The R Journal. 2011;3 (2) 12–15.
- 39.
Ruppert D, Wand MP, Carroll RJ. Semiparametric regression. 12. Cambridge university press; 2003.
- 40.
Wand M. SemiPar: Semiparametic Regression; 2018. Available from: https://CRAN.R-project.org/package=SemiPar.
- 41.
Therneau TM. A Package for Survival Analysis in S; 2015. Available from: https://CRAN.R-project.org/package=survival.
- 42. Li W, Xie B, Qiu S, Huang X, Chen J, Wang X, et al. Non-lab and semi-lab algorithms for screening undiagnosed diabetes: A cross-sectional study. EBioMedicine. 2018;35:307–316.
- 43. Kim H, Cutter GR, George B, Chen Y. Understanding and preventing loss to follow-up: experiences from the spinal cord injury model systems. Topics in spinal cord injury rehabilitation. 2018;24(2):97–109. pmid:29706754
- 44. Goltz DE, Ryan SP, Hopkins TJ, Howell CB, Attarian DE, Bolognesi MP, et al. A novel risk calculator predicts 90-day readmission following total joint arthroplasty. JBJS. 2019;101(6), 547–556.
- 45. Chen QY, Zhong Q, Zhou JF, Qiu XT, Dang XY, Cai LS, et al. Development and External Validation of Web-Based Models to Predict the Prognosis of Remnant Gastric Cancer after Surgery: A Multicenter Study. Journal of oncology. 2019.
- 46. Bellam BL, Samanta J, Gupta P, Kumar P, Sharma V, Dhaka N, et al. Predictors of outcome of percutaneous catheter drainage in patients with acute pancreatitis having acute fluid collection and development of a predictive model. Pancreatology. 2019. pmid:31204261
- 47. Andersen MØ, Fritzell P, Eiskjaer SP, Lagerbäck T, Hägg O, Nordvall D, et al. Surgical Treatment of Degenerative Disk Disease in Three Scandinavian Countries: An International Register Study Based on Three Merged National Spine Registers. Global Spine Journal. 2019;2192568219838535.