Markov decision process model for optimisation of patient flow

Australian hospitals are facing the challenge of increased demand in ED and inpatient services. Congestion and poor quality of care for some patients may occur as a consequence of such challenging situations. Modelling patient flow is a way to capture a small number of salient characteristics of a hospital operation permitting us to investigate the causes and effects of congestion, and identify optimal improvement policies. The data used in this project are drawn from an extensive Patient Journey Database of a major South Australia hospital, Flinders Medical Centre (FMC). This database tracks journeys of patients through the hospital system, from arrival to discharge. This allows for very detailed analysis and modelling of these journeys. Analysis of historial bed occupancy data indicates that there are remarkably regular patterns in patient occupancy, on both daily and weekly time scales. Based on this observation, we develop a Markov decision process (MDP) model over a weekly time horizon to analyse FMC's bed occupancy patterns with a goal of proposing policies that lead to reduced incidence of congestion. A simple, four state, approach is used in categorising the occupancy level, and daily and weekly probability transition matrices are derived from midnight census data over a three year period. We then introduce a reward function that attempts to balance the benefit of maintaining sufficient buffer in the occupancy of wards versus the cost of diverting patients elsewhere. An optimal policy from the MDP model was found using the dynamic programming approach. Employing an MDP approach in these problems will also, potentially, allow hospital staff the ability to forecast possible onset of very high occupancy levels, based on the hospital's current situation. Such a “warning signal” may enable hospital managers to be proactive in their strategies to reduce the magnitude and impacts of congestion episodes. Although, to managers, the underlying mathematics may appear relatively complex, it is possible to embed an MDP model inside a software package so that the model is available in a more user-friendly form, allowing it to be easily run on-the-fly so that staff can react to changes in the hospital in real time. The issue of validation of “optimal” policies suggested by an MDP model is addressed in three, separate, ways. Firstly, the steady state probabilities resulting from such policies are examined to see if they reduce the long-run frequency of highest occupancy levels. Secondly, consultation with FMC experts, an integral part of this study, has provided a degree of qualitative validation. Thirdly, synthetic data generated by a discrete event simulation system developed in another component of the larger project is used to provide surrogate statistical testing environment for the performance of MDP's policies. Simulated experiments also serve the purpose of convincing hospital's management before piloting policies in real setting. Future work includes refinement of both the MDP and the simulation model and establishment of a formal framework for cross-validating MDP and simulation modelling results.