An assessment of program evaluation methods and quality in Australian prevention agencies

Joanna Schwarzman, Tracy Nau, Adrian Bauman, Belinda J. Gabbe, Chris Rissel, Trevor Shilton, Ben J. Smith

Research output: Contribution to journalArticlepeer-review

1 Citation (Scopus)

Abstract

Issue addressed: This study aimed to examine evaluation methods and quality in Australian health promotion agencies and the factors associated with this. The evidence base for prevention strategies is limited, with the evidence generated through program evaluation by health promotion and disease prevention agencies lacking rigour. Despite the need to improve the quality of evaluation, there is limited evidence of what influences evaluation quality in the prevention field. Methods: Data were collected using the Evaluation Practice Analysis Survey and an audit and appraisal of evaluation reports. Descriptive analysis was used to examine evaluation characteristics and multivariable regression was used to explore the association between evaluation and organisational attributes and evaluation quality. Results: In total, 392 evaluation reports were reviewed from 78 government and non-government agencies. Process evaluation was conducted most frequently, followed by impact evaluation. Overall evaluation quality was low (median 24.5%). In multivariable regression analysis, only two factors were associated with evaluation quality: health promotion budget (ratio of geometric means 1.53 [95% CI 1.02-2.29]); and, conducting statewide or national prevention programs (1.38 [95% CI 1.05-1.82]). Conclusions: The findings show that the potential to improve evaluation quality is greatest in smaller organisations that deliver health promotion at a local or regional scale. So what?: By improving the rigour of existing evaluation, there is opportunity to build the evidence base for prevention strategies, which highlights the importance of embedding the enablers of program learning and evidence generation within health promotion and prevention organisations.

Original languageEnglish
Pages (from-to)456-467
Number of pages12
JournalHealth Promotion Journal of Australia
Volume31
Issue number3
Early online date13 Aug 2019
DOIs
Publication statusPublished - 1 Sep 2020
Externally publishedYes

Keywords

  • evidence-based practice
  • government
  • health equity
  • health promotion
  • non-government organisations
  • primary prevention
  • program evaluation

Fingerprint Dive into the research topics of 'An assessment of program evaluation methods and quality in Australian prevention agencies'. Together they form a unique fingerprint.

Cite this