Response to the Letter to the Editor: “Examining Bias and Reporting in Oral Health Prediction Modeling Studies”

M. Du, D. Haag, J. Lynch, M. Mittinty

Research output: Contribution to journalLetterpeer-review

Abstract

In our paper (Du et al. 2020), we rated each included study following PROBAST (Prediction Model Risk of Bias Assessment Tool) criteria (Moons et al. 2019), which we believe need to be improved. We have re-visited the study by Kuboniwa et al. (2016), and we agree that the authors have avoided using univariate analysis (related to PROBAST 4.5) to select predictors. Instead, they have used variable importance score as a selection criterion based on orthogonal projections to latent structures (OPLS) regression analysis (as we had described in Table 1). PROBAST 4.8 relates to the possibility of a model’s overfitting/underfitting; this could happen when there is lack of internal validation during the final modelling analysis. In the study by Kuboniwa et al., though the authors selected predictors using OPLS with 7-fold cross-validation (as they have mentioned in the letter), they developed the final prediction model (the combination of the metabolites) using logistic regression without information on internal validation (e.g., cross-validation) (as they had stated on page 1,383). Moreover, in terms of measuring models’ prediction performance (PROBAST 4.7), we found the authors did not evaluate the models’ calibration. According to PROBAST, a study could not be rated as low risk in the analysis domain if any of the signaling questions were answered “no.” Nevertheless, we did not challenge the reliability of this study or the model’s predictive ability.
Original languageEnglish
Article number1307
Number of pages1
JournalJournal of Dental Research
Volume99
Issue number11
Early online date7 Jul 2020
DOIs
Publication statusPublished - Oct 2020
Externally publishedYes

Keywords

  • clinical prediction
  • forecasting
  • model development
  • PROBAST
  • systematic error
  • TRIPOD

Fingerprint

Dive into the research topics of 'Response to the Letter to the Editor: “Examining Bias and Reporting in Oral Health Prediction Modeling Studies”'. Together they form a unique fingerprint.

Cite this