TY - JOUR
T1 - Applying pragmatic approaches to complex program evaluation
T2 - A case study of implementation of the New South Wales Get Healthy at Work program
AU - Crane, Melanie
AU - Bauman, Adrian
AU - Lloyd, Beverley
AU - McGill, Bronwyn
AU - Rissel, Chris
AU - Grunseit, Anne
PY - 2019/9
Y1 - 2019/9
N2 - Issue addressed: Complex health promotion programs, which can have multilevels of implementation and multi-components with nonlinear causal pathways, present many evaluation challenges. Traditional evaluation methods often fail to account for the complexity inherent in assessing these programs. In real-world settings, evaluations of complex programs are often beset by additional constraints of limited budgets and short timeframes. Determining whether a complex program is successful and how a program worked requires evaluators of complex programs to adopt a level of pragmatism. Methods: This paper describes a pragmatic evaluation approach used to evaluate the Get Healthy at Work workplace health promotion program, implemented in New South Wales, Australia. Using the program as a case study, we describe some key principles for applying a pragmatic evaluation approach and use these principles to develop an appropriate evaluation strategy. Results: The evaluation includes multiple research methods to assess program outputs and implementation; and identify emergent program impacts, within constrained resources. The evaluation was guided by epistemological flexibility, methodological comprehensiveness and operational practicality. Conclusion: Health promotion programs, such as state-wide obesity prevention programs, require appropriate evaluation methods which address their inherent complexity amidst the real-world evaluation constraints, and focuses on the essential evaluation needs. So what: The main complex program evaluation principles are applicable to other multilevel health promotion programs, challenged by methodological and practical or political constraints.
AB - Issue addressed: Complex health promotion programs, which can have multilevels of implementation and multi-components with nonlinear causal pathways, present many evaluation challenges. Traditional evaluation methods often fail to account for the complexity inherent in assessing these programs. In real-world settings, evaluations of complex programs are often beset by additional constraints of limited budgets and short timeframes. Determining whether a complex program is successful and how a program worked requires evaluators of complex programs to adopt a level of pragmatism. Methods: This paper describes a pragmatic evaluation approach used to evaluate the Get Healthy at Work workplace health promotion program, implemented in New South Wales, Australia. Using the program as a case study, we describe some key principles for applying a pragmatic evaluation approach and use these principles to develop an appropriate evaluation strategy. Results: The evaluation includes multiple research methods to assess program outputs and implementation; and identify emergent program impacts, within constrained resources. The evaluation was guided by epistemological flexibility, methodological comprehensiveness and operational practicality. Conclusion: Health promotion programs, such as state-wide obesity prevention programs, require appropriate evaluation methods which address their inherent complexity amidst the real-world evaluation constraints, and focuses on the essential evaluation needs. So what: The main complex program evaluation principles are applicable to other multilevel health promotion programs, challenged by methodological and practical or political constraints.
KW - evaluation methods
KW - health promotion theory
KW - obesity
KW - program evaluation
KW - workplaces
UR - http://www.scopus.com/inward/record.url?scp=85063579281&partnerID=8YFLogxK
UR - http://purl.org/au-research/grants/NHMRC/9100001
U2 - 10.1002/hpja.239
DO - 10.1002/hpja.239
M3 - Article
C2 - 30860630
AN - SCOPUS:85063579281
SN - 1036-1073
VL - 30
SP - 422
EP - 432
JO - Health Promotion Journal of Australia
JF - Health Promotion Journal of Australia
IS - 3
ER -