A Systematic Review of Machine Learning for Assessment and Feedback of Treatment Fidelity

Asghar Ahmadi, Michael Noetel, Melissa Schellekens, Philip Parker, Devan Antczak, Mark Beauchamp, Theresa Dicke, Carmel Diezmann, Anthony Maeder, Nikos Ntoumanis, Alexander Yeung, Chris Lonsdale

Research output: Contribution to journalReview articlepeer-review

5 Citations (Scopus)
138 Downloads (Pure)

Abstract

Many psychological treatments have been shown to be cost-effective and efficacious, as long as they are implemented faithfully. Assessing fidelity and providing feedback is expensive and time-consuming. Machine learning has been used to assess treatment fidelity, but the reliability and generalisability is unclear. We collated and critiqued all implementations of machine learning to assess the verbal behaviour of all helping professionals, with particular emphasis on treatment fidelity for therapists. We conducted searches using nine electronic databases for automated approaches of coding verbal behaviour in therapy and similar contexts. We completed screening, extraction, and quality assessment in duplicate. Fifty-two studies met our inclusion criteria (65.3% in psychotherapy). Automated coding methods performed better than chance, and some methods showed near human-level performance; performance tended to be better with larger data sets, a smaller number of codes, conceptually simple codes, and when predicting session-level ratings than utterance-level ones. Few studies adhered to best-practice machine learning guidelines. Machine learning demonstrated promising results, particularly where there are large, annotated datasets and a modest number of concrete features to code. These methods are novel, cost-effective, scalable ways of assessing fidelity and providing therapists with individualised, prompt, and objective feedback.

Original languageEnglish
Pages (from-to)139-153
Number of pages15
JournalPsychosocial Intervention
Volume30
Issue number3
DOIs
Publication statusPublished - Sept 2021

Keywords

  • Clinical supervision
  • Feedback
  • Machine learning
  • Treatment fidelity
  • Treatment integrity

Fingerprint

Dive into the research topics of 'A Systematic Review of Machine Learning for Assessment and Feedback of Treatment Fidelity'. Together they form a unique fingerprint.

Cite this