Patient Scheduling with Approximate Dynamic Programming for Optimization of Health Care Services

Model that prescribes the optimal appointment date for a patient at the moment this patient makes his request at the outpatient clinic is developed. We categorized patients into two. The first category is concerned with patients with a maximum recommended waiting time. For these types of patient, the sooner these patients are scheduled the better and when the maximum recommended waiting time is exceeded, extra costs are incurred. The other category is characterized by a specific appointment time. The closer the scheduled appointment time is to the specific appointment time, the lower the costs. The objective is to minimize the long-run expected average cost. We modelled the scheduling process as a Markov Decision Process (MDP). we then apply the Bellman Error Minimization (BEM) method as an Approximate Dynamic Programming technique in order to derive an estimate of the optimal value function of our MDP of which the optimal policy (appointment date) can be derived. To determine the set of representative states, which is an element of the BEM method, we use the k-means algorithm. We test several approximation functions and find an approximation function that outperforms all other functions in the scheduling process over four, six, and eight working days. The Approximation Function B gives the near optimal appointment date for patients when appointments are requested. In general, it holds that the higher the arrival rate of patients at the outpatient clinic, the better our BEM method performs. But if the arrival rate reaches a certain value the load of the system becomes that high that it does not matter what policy is applied, since many patients have to be rejected.

Markov Decision Process, Approximate Dynamic Programming, Arrival rate, Policy Improvement

1. Bailey, N.T.J. (1952). A study of queues and appointment systems in hospital out-patient departments, with special reference to waiting-times. Journal of the Royal Statistical Society. Series B (Methodological)14(2) 185 ̶ 199. [Google Scholar] [Crossref]

2. Caryirli, T. and Veral, E. (2003). Outpatient scheduling in health care: a review of literature. Production and operations management, 12(4):519 ̶ 549. [Google Scholar] [Crossref]

3. Caryirli, T., Yang, K. K. and Quek, S. A. (2012). A universal appointment rule in the presence of no-shows and walk-ins. Production and Operations Management, 21(4):682 ̶ 697. [Google Scholar] [Crossref]

4. Charlton, E. (2020). 28 million elective surgeries may be cancelled worldwide: how non COVID-19 medical care is suffering. World Economic Forum. Accessed 27 January 2021 https://www.weforum.org/agenda/2020/05/covid-19-elective-surgery cancellation-cancer-pandemic. [Google Scholar] [Crossref]

5. COVIDSurg Collaborative. (2020). Elective surgery cancellations due to the covid-19 pandemic: global predictive modelling to inform surgical recovery plans. British Journal of Surgery 107(11) 1440 ̶ 1449. [Google Scholar] [Crossref]

6. Deo, S., Iravani, S., Jiang, T., Smilowitz, K. and Samuelson, S. (2013). Improving health outcomes through better capacity allocation in a community-based chronic care model. Operations Research 61(6) 1277 ̶ 1294. [Google Scholar] [Crossref]

7. Diamant, A., Milner, J. and Quereshy, F. (2018). Dynamic patient scheduling for multi-appointment health care programs. Production and Operations Management 27(1) 58 ̶ 79. [Google Scholar] [Crossref]

8. Erdelyi, A. and Topaloglu, H. (2010). Approximate dynamic programming for dynamic capacity allocation with multiple priority levels. IIE Transactions, 43(2):129 ̶ 142. [Google Scholar] [Crossref]

9. Erdogan, S. A., Gose, A. and Denton, B. T. (2015). Online appointment sequencing and scheduling. IIE Transactions, 47(11):1267 ̶ 1286. [Google Scholar] [Crossref]

10. Feldman, J., Liu, N., Topaloglu, H. and Ziya, S. (2014). Appointment scheduling under patient preference and no-show behavior. Operations Research, 62(4):794 ̶ 811. [Google Scholar] [Crossref]

11. Fries B. and Marathe, V. (1991). Determination of Optimal Variable-Sized Multiple-Block Appointment Systems, Operation Research, 29(2), 324 ̶ 345. [Google Scholar] [Crossref]

12. Grais, R. F., Dubray, C., Gerstl, S., Guthmann, J. P., Djibo, A., Nargaye, K. D., et al. (2007). Unacceptably high mortality related to measles epidemics in Niger, Nigeria, and Chad. PLoS Med; 4:e16 [Google Scholar] [Crossref]

13. Gupta, D. and Denton, B. (2008). Appointment scheduling in health care: Challenges and opportunities. IIE transactions, 40(9):800 ̶ 819. [Google Scholar] [Crossref]

14. Gupta D. and Wang, L. (2008). Revenue management for a primary-care clinic in the presence of patient choice. Operations Research, 56(3):576 ̶ 592. [Google Scholar] [Crossref]

15. Heaney, D. J., Howie, J. G. and Porter, A. M. (1991). Factors Influencing Waiting Times and Consultation Times in General Practice, British Journal of General Practice, 41, 315 ̶ 319. [Google Scholar] [Crossref]

16. Kaandorp, G. C. and Koole, G. (2007). Optimal outpatient appointment scheduling. HealthCare Management Science, 10(3):217 ̶ 229. [Google Scholar] [Crossref]

17. Keller T. F., Laughhunn. D. J. (1973). An Application of Queuing Theory to a Congestion Problem in an Outpatient Clinic. Decision Sciences, 4, 379 ̶ 394. [Google Scholar] [Crossref]

18. Klassen, K. J. and Rohleder, T. R. (1996). Scheduling Outpatient Appointments in a Dynamic Environment, JJournal of Operations Management, 14, 2, 83 ̶ 101. [Google Scholar] [Crossref]

19. Lindley, D. V. (1952). The Theory of Queues with a Single Server, Proceedings Cambridge Philosophy Society, 48, 277 ̶ 289. [Google Scholar] [Crossref]

20. Liu, N., Ziya, S. and Kulkarni, V. G. (2010). Dynamic scheduling of outpatient appointments under patient no-shows and cancellations. Manufacturing & Service Operations Management, 12(2):347 ̶ 364. [Google Scholar] [Crossref]

21. Minerva Strategies, Addressing health challenges in Nigeria. Accessed 21 January2021,https://www.minervastrategies.com/blog/addressing-health-challenges-in-nigeria/. [Google Scholar] [Crossref]

22. Murray, M. and Berwick, D.M. (2003) Advanced access: reducing waiting and delays in primary care. Journal of the American Medical Association, 289, 1035–1040. [Google Scholar] [Crossref]

23. Onwujekwe, O., Onoka, C., Uguru, N., Nnenna, T., Uzochukwu, B., Eze, S., et al. Preferences for benefit packages for community-based health insurance: An exploratory study in Nigeria. BMC Health Services Research. 2010 10:162. [Google Scholar] [Crossref]

24. Panaviwat, C., Lohasiriwat, H. and Tharmmaphornphilas, W. (2014). Designing an appointment system for an outpatient department. Materials Science and Engineering [Google Scholar] [Crossref]

25. Patrick, J., Puterman, M. L. and Queyranne, M. (2008). Dynamic multi priority patient scheduling for a diagnostic resource. Operations research, 56(6):1507 ̶ 1525.48Bibliography. [Google Scholar] [Crossref]

26. Puterman, M. L. (2005). Markov decision processes: discrete stochastic dynamic programming. John Wiley & Sons. [Google Scholar] [Crossref]

27. Roubos, D. (2010). The application of approximate dynamic programming techniques. PhD thesis, VU Amsterdam, the Netherlands. [Google Scholar] [Crossref]

28. Swisher, J. R., Jacobson, S. H., Jun, J. B. and Balci, O. (2001). Modeling and Analyzing a Physician Clinic Environment Using Discrete-Event (Visual) Simulation, Computers & Operations Research, 28, 2, 105 ̶ 125. [Google Scholar] [Crossref]

29. Tijms, H. C. (2003). A first course in stochastic models. John Wiley & Sons. [Google Scholar] [Crossref]

30. Truong, V.A. (2015). Optimal advance scheduling. Management Science 61(7) 1584 ̶ 1597. [Google Scholar] [Crossref]

31. Wang, S., Liu, N. and Wan, G. (2020). Managing appointment-based services in the presence of walk-in customers. Management Science 66(2) 667 ̶ 686. [Google Scholar] [Crossref]

32. Wolfswinkel, E. (2017). Scheduling patients at the outpatient clinic. Master thesis, VU Amsterdam, the Netherlands. [Google Scholar] [Crossref]

33. Zacharias, C., Liu, N. and Begen, M. A. (2020). Dynamic Inter-day and Intra-day Scheduling [Google Scholar] [Crossref]

34. Zhou, M., Loke, G. G., Bandi, C., Liau, Z. Q. and Wang, W. (2020). Intraday Scheduling with Patient Re-entries and Variability in Behaviours. Analytics and operations forth coming. [Google Scholar] [Crossref]

• The Net Relative Run-Ratio Method (NRRR), a Foolproof Technique to Replace the Net Run Rate (NRR) Method in Evaluating the Authority of Match-Wins
• Statistical Role of CB-SEM Vs PLS-SEM in the Field of Social Science
• Predictive Modelling and Statistical Analysis of Housing Prices in Lagos State, Nigeria
• Collocational Patterns of Guru in American Business vs. Spiritual Discourse
• A Comparative Analysis of Heuristic and Dynamic Algorithms for Route Optimization in Johor’s Delivery Hubs