TY - JOUR
T1 - Pitfalls When Using Area Under the Curve to Evaluate Item Content for Early Screening Tests for Autism
AU - Lucas, Carmen
AU - Brewer, Neil
AU - Young, Robyn
PY - 2022/6
Y1 - 2022/6
N2 - Evaluations of early screening tests for autism commonly rely on receiver operating characteristic (ROC) analysis and comparisons of area under the curve (AUC). Whether AUC differs significantly from chance or between test items is not always assessed. Two recent and independent evaluations of the Brief Autism Detection in Early Childhood (BADEC) constructed a short-form by selecting the five items with the highest AUC values, leading to inconsistencies regarding appropriate item content (Nah et al., 2018; Nevill et al., 2019). Using significance testing to compare AUC values for each test item from each dataset, we demonstrate which items justify inclusion in the BADEC, which items can be ruled out, and highlight key factors influencing AUC significance testing outcomes.
AB - Evaluations of early screening tests for autism commonly rely on receiver operating characteristic (ROC) analysis and comparisons of area under the curve (AUC). Whether AUC differs significantly from chance or between test items is not always assessed. Two recent and independent evaluations of the Brief Autism Detection in Early Childhood (BADEC) constructed a short-form by selecting the five items with the highest AUC values, leading to inconsistencies regarding appropriate item content (Nah et al., 2018; Nevill et al., 2019). Using significance testing to compare AUC values for each test item from each dataset, we demonstrate which items justify inclusion in the BADEC, which items can be ruled out, and highlight key factors influencing AUC significance testing outcomes.
KW - Autism screening
KW - ROC analysis
KW - AUC comparisons
KW - ADEC
KW - BADEC
UR - http://www.scopus.com/inward/record.url?scp=85124368844&partnerID=8YFLogxK
UR - http://purl.org/au-research/grants/arc/DP190100162
U2 - 10.1177/07342829211067128
DO - 10.1177/07342829211067128
M3 - Article
VL - 40
SP - 430
EP - 437
JO - Journal of Psychoeducational Assessment
JF - Journal of Psychoeducational Assessment
SN - 0734-2829
IS - 3
ER -