Markov decision process model for optimisation of patient flow

Anthony Clissold, Jerzy Filar, Shaowen Qin, Dale Ward

    Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

    1 Citation (Scopus)

    Abstract

    Australian hospitals are facing the challenge of increased demand in ED and inpatient services. Congestion and poor quality of care for some patients may occur as a consequence of such challenging situations. Modelling patient flow is a way to capture a small number of salient characteristics of a hospital operation permitting us to investigate the causes and effects of congestion, and identify optimal improvement policies. The data used in this project are drawn from an extensive Patient Journey Database of a major South Australia hospital, Flinders Medical Centre (FMC). This database tracks journeys of patients through the hospital system, from arrival to discharge. This allows for very detailed analysis and modelling of these journeys. Analysis of historial bed occupancy data indicates that there are remarkably regular patterns in patient occupancy, on both daily and weekly time scales. Based on this observation, we develop a Markov decision process (MDP) model over a weekly time horizon to analyse FMC's bed occupancy patterns with a goal of proposing policies that lead to reduced incidence of congestion. A simple, four state, approach is used in categorising the occupancy level, and daily and weekly probability transition matrices are derived from midnight census data over a three year period. We then introduce a reward function that attempts to balance the benefit of maintaining sufficient buffer in the occupancy of wards versus the cost of diverting patients elsewhere. An optimal policy from the MDP model was found using the dynamic programming approach. Employing an MDP approach in these problems will also, potentially, allow hospital staff the ability to forecast possible onset of very high occupancy levels, based on the hospital's current situation. Such a “warning signal” may enable hospital managers to be proactive in their strategies to reduce the magnitude and impacts of congestion episodes. Although, to managers, the underlying mathematics may appear relatively complex, it is possible to embed an MDP model inside a software package so that the model is available in a more user-friendly form, allowing it to be easily run on-the-fly so that staff can react to changes in the hospital in real time. The issue of validation of “optimal” policies suggested by an MDP model is addressed in three, separate, ways. Firstly, the steady state probabilities resulting from such policies are examined to see if they reduce the long-run frequency of highest occupancy levels. Secondly, consultation with FMC experts, an integral part of this study, has provided a degree of qualitative validation. Thirdly, synthetic data generated by a discrete event simulation system developed in another component of the larger project is used to provide surrogate statistical testing environment for the performance of MDP's policies. Simulated experiments also serve the purpose of convincing hospital's management before piloting policies in real setting. Future work includes refinement of both the MDP and the simulation model and establishment of a formal framework for cross-validating MDP and simulation modelling results.

    Original languageEnglish
    Title of host publicationProceedings - 21st International Congress on Modelling and Simulation, MODSIM 2015
    Subtitle of host publication21st International Congress on Modelling and Simulation, MODSIM2015
    EditorsTony Weber, Malcolm McPhee, Robert Anderssen
    PublisherModelling and Simulation Society of Australia and New Zealand Inc. (MSSANZ)
    Pages1752-1758
    Number of pages7
    ISBN (Electronic)9780987214355
    DOIs
    Publication statusPublished - 2015
    Event21st International Congress on Modelling and Simulation: Partnering with Industry and the Community for Innovation and Impact through Modelling, MODSIM 2015 - Held jointly with the 23rd National Conference of the Australian Society for Operations Research and the DSTO led Defence Operations Research Symposium, DORS 2015: Partnering with industry and the community for innovation and impact through modelling - Gold Coast Convention and Exhibition Centre, Broadbeach, Australia
    Duration: 29 Nov 20154 Dec 2015
    Conference number: 21st
    https://www.mssanz.org.au/modsim2015/ (Conference link)

    Publication series

    NameProceedings - 21st International Congress on Modelling and Simulation, MODSIM 2015

    Conference

    Conference21st International Congress on Modelling and Simulation: Partnering with Industry and the Community for Innovation and Impact through Modelling, MODSIM 2015 - Held jointly with the 23rd National Conference of the Australian Society for Operations Research and the DSTO led Defence Operations Research Symposium, DORS 2015
    Abbreviated titleMODSIM2015
    CountryAustralia
    CityBroadbeach
    Period29/11/154/12/15
    Internet address

    Keywords

    • Hospital patient flow
    • Markov decision process
    • Process optimisation

    Fingerprint Dive into the research topics of 'Markov decision process model for optimisation of patient flow'. Together they form a unique fingerprint.

    Cite this