TY - JOUR
T1 - A collaborative comparison of objective structured clinical examination (OSCE) standard setting methods at Australian medical schools
AU - Malau-Aduli, Bunmi
AU - Teague, Peta-Ann
AU - D'Souza, Karen
AU - Heal, Clare
AU - Turner, Richard
AU - Garne, David
AU - van der Vleuten, Cees
PY - 2017
Y1 - 2017
N2 - Background: A key issue underpinning the usefulness of the OSCE assessment to medical education is standard setting, but the majority of standard-setting methods remain challenging for performance assessment because they produce varying passing marks. Several studies have compared standard-setting methods; however, most of these studies are limited by their experimental scope, or use data on examinee performance at a single OSCE station or from a single medical school. This collaborative study between 10 Australian medical schools investigated the effect of standard-setting methods on OSCE cut scores and failure rates. Methods: This research used 5256 examinee scores from seven shared OSCE stations to calculate cut scores and failure rates using two different compromise standard-setting methods, namely the Borderline Regression and Cohen’s methods. Results: The results of this study indicate that Cohen’s method yields similar outcomes to the Borderline Regression method, particularly for large examinee cohort sizes. However, with lower examinee numbers on a station, the Borderline Regression method resulted in higher cut scores and larger difference margins in the failure rates. Conclusion: Cohen’s method yields similar outcomes as the Borderline Regression method and its application for benchmarking purposes and in resource-limited settings is justifiable, particularly with large examinee numbers.
AB - Background: A key issue underpinning the usefulness of the OSCE assessment to medical education is standard setting, but the majority of standard-setting methods remain challenging for performance assessment because they produce varying passing marks. Several studies have compared standard-setting methods; however, most of these studies are limited by their experimental scope, or use data on examinee performance at a single OSCE station or from a single medical school. This collaborative study between 10 Australian medical schools investigated the effect of standard-setting methods on OSCE cut scores and failure rates. Methods: This research used 5256 examinee scores from seven shared OSCE stations to calculate cut scores and failure rates using two different compromise standard-setting methods, namely the Borderline Regression and Cohen’s methods. Results: The results of this study indicate that Cohen’s method yields similar outcomes to the Borderline Regression method, particularly for large examinee cohort sizes. However, with lower examinee numbers on a station, the Borderline Regression method resulted in higher cut scores and larger difference margins in the failure rates. Conclusion: Cohen’s method yields similar outcomes as the Borderline Regression method and its application for benchmarking purposes and in resource-limited settings is justifiable, particularly with large examinee numbers.
UR - http://www.scopus.com/inward/record.url?scp=85029908570&partnerID=8YFLogxK
U2 - 10.1080/0142159X.2017.1372565
DO - 10.1080/0142159X.2017.1372565
M3 - Article
SN - 0142-159X
VL - 39
SP - 1261
EP - 1267
JO - Medical Teacher
JF - Medical Teacher
IS - 12
ER -