A little bias goes a long way: The effects of feedback on the strategic regulation of accuracy on formula-scored tests

Michelle Arnold, Philip Higham, Beatriz Martin-Luengo

    Research output: Contribution to journalArticlepeer-review

    13 Citations (Scopus)

    Abstract

    Under formula-scoring rules for multiple-choice exams, a penalty is applied to incorrect responses to reduce noise in the observed score. To avoid the penalty individuals are allowed to "pass," and therefore they must be able to strategically regulate the accuracy of their reporting by deciding which and how many questions to answer. To investigate the effect of bias within this framework, Higham (2007) introduced bias profiles, which show the score obtained under formula scoring (corrected score) as a function of the omission rate. Bias profiles estimate the optimal number of questions that should be answered to maximize the corrected score (i.e., optimal bias). Our initial research showed that individuals tend to be too conservative when setting reporting criteria, "omitting" too many answers. The present three experiments introduced a feedback manipulation whereby participants were informed of the optimal omission rate after completing a test and asked to alter their reporting decisions accordingly. This feedback and concomitant alteration of reporting decisions led to improved corrected scores on true/false (Experiment 1), 2-alternative tests (Experiments 2), and 4-alternative tests (Experiment 3). Importantly, corrected scores at optimal bias also were higher than at forced-report for both true/false and 2-alternative tests. Furthermore, in Experiment 3, feedback based on one test improved scores on a second test, and participants were more likely to perform optimally on a third test without feedback. These effects suggest that optimal-bias feedback may have long-term effects and generalize to new tests.

    Original languageEnglish
    Pages (from-to)383-402
    Number of pages20
    JournalJournal of Experimental Psychology: Applied
    Volume19
    Issue number4
    DOIs
    Publication statusPublished - Dec 2013

    Keywords

    • Bias profiles
    • Formula scoring
    • Metacognition
    • Strategic regulation of accuracy
    • Type-2 signal detection theory

    Fingerprint Dive into the research topics of 'A little bias goes a long way: The effects of feedback on the strategic regulation of accuracy on formula-scored tests'. Together they form a unique fingerprint.

    Cite this