Early economic evaluation to develop and manage health services: experience of an Action Learning Set within an Australian Local Hospital Network

Context: The management of demand for acute care and the flow of patients through and around hospitals is a challenge for health systems internationally. To address such challenges, reactive and implicit decision-making has historically led to cost-effective options being overlooked or displaced. This is partly due to difficulties instituting Health Technology Assessment (HTA) or Program Budgeting Marginal Analysis (PBMA) processes within local services' planning, commissioning, and improvement. Embedded within a Local Hospital Network (LHN), we adapt economic evaluation methods to the dynamic context of an Action Learning Set, inform the development of out-of-hospital services for older adults who would otherwise present to the Emergency Department, and iteratively update internal business cases.

Method: Several complementary methods were employed to estimate potential service recipient and broader system effects: (1) virtual whiteboard mapping of the intervention logic and value proposition from a system perspective; (2) algorithmic data mining of routinely-collected data to profile existing care pathways as a status quo comparator; (3) a structured elicitation exercise to capture stakeholder interpretations of available evidence and expectations of intervention effects, including a ‘think aloud’ component fed back to stakeholders that enabled them to update their initial expectations; (4) exploratory decision-analytic modelling and headroom analyses using untimed decision-trees and Monte Carlo simulation, based on the outputs from (1), (2), and (3); and (5) Interrupted Time Series (ITS) analyses during early implementation, to estimate potential counterfactuals and update the decision-analyses. The information generated was fed back to clinical and executive leaders as their business case was developed to justify funding and quality improvement proposals.

Results: The logic model helped engender a shared understanding of the intervention and evaluation. Existing care analyses profiled patient pathways for a population of 10,828 people presenting to the ED, of which 0.79 (n=8,569) were admitted. Modelling based on pooled elicited expectations indicated the new service would add +540 bed days but with a net saving of -14,030 bed days across the LHN. Uncertainties and disagreement, particularly around admission rates and inpatient lengths of stay for those who continue to present to the ED, affected potential cost-effectiveness. However, given the expected costs and historical prices for a bed day, the intervention was expected to meet the minimum required bed day savings in 63% of simulated scenarios. Early ITS results indicated intervention patients spent an average of +3.8 more days at home in the first 30 days following an encounter with the system and resulted in 3,236 fewer ED presentations, with bed day savings exceeding elicited/modelled expectations.

Discussion: HTA colleagues, including economists, have largely ignored technology development and often focus solely on products or policies. Experiences with PBMA suggest it is mismatched to the complex organisational and political dynamics of providing care. This study provides insights into the expected value of a novel service intervention for older adults, compiled during its design and early implementation. More importantly, we demonstrate how economic evidence can be proactively used in an iterative “search for efficiency”, as part of a collaborative approach to managing acute care demand and patient flow. Ongoing formative evaluations will continue to inform service development, including retargeting the intervention and modifying resource inputs to increase the likelihood of realizing its expected value. Further methodological work is required to estimate relevant spill-over effects across the system and define acceptable, value-based prices for service outcomes.