A One-Day Dental Faculty Workshop in Writing Multiple-Choice Questions: An Impact Evaluation

Eiad AlFaris, N Naeem, Farhana Irfan, Riaz Qureshi, H Saad, R Al Sadhan, H Abdulghani, Cees van der Vleuten

    Research output: Contribution to journalArticlepeer-review

    9 Citations (Scopus)

    Abstract

    Long training workshops on the writing of exam questions have been shown to be effective; however, the effectiveness of short workshops needs to be demonstrated. The aim of this study was to evaluate the impact of a one-day, seven-hour faculty development workshop at the College of Dentistry, King Saud University, Saudi Arabia, on the quality of multiple-choice questions (MCQs). Kirkpatrick's four-level evaluation model was used. Participants' satisfaction (Kirkpatrick's Level 1) was evaluated with a post-workshop questionnaire. A quasi-experimental, randomized separate sample, pretest-posttest design was used to assess the learning effect (Kirkpatrick's Level 2). To evaluate transfer of learning to practice (Kirkpatrick's Level 3), MCQs created by ten faculty members as a result of the training were assessed. To assess Kirkpatrick's Level 4 regarding institutional change, interviews with three key leaders of the school were conducted, coded, and analyzed. A total of 72 course directors were invited to and attended some part of the workshop; all 52 who attended the entire workshop completed the satisfaction form; and 22 of the 36 participants in the experimental group completed the posttest. The results showed that all 52 participants were highly satisfied with the workshop, and significant positive changes were found in the faculty members' knowledge and the quality of their MCQs with effect sizes of 0.7 and 0.28, respectively. At the institutional level, the interviews demonstrated positive structural changes in the school's assessment system. Overall, this one-day item-writing faculty workshop resulted in positive changes at all four of Kirkpatrick's levels; these effects suggest that even a short training session can improve a dental school's assessment of its students.

    Original languageEnglish
    Pages (from-to)1305-1313
    Number of pages9
    JournalJournal of Dental Education
    Volume79
    Issue number11
    DOIs
    Publication statusPublished - 1 Nov 2015

    Keywords

    • assessment
    • dental education
    • dental faculty
    • faculty development
    • Kirkpatrick's framework
    • multiple-choice questions

    Fingerprint Dive into the research topics of 'A One-Day Dental Faculty Workshop in Writing Multiple-Choice Questions: An Impact Evaluation'. Together they form a unique fingerprint.

    Cite this