Index Polices for Patient Scheduling and ATM Replenishment Public Deposited

Downloadable Content

Download PDF
Last Modified
  • March 21, 2019
  • Zhang, Yu
    • Affiliation: College of Arts and Sciences, Department of Statistics and Operations Research
  • Markov Decision Processes (MDP) are one of the most commonly used stochastic models to solve sequential decision making problems. The optimal solution to many real-world problems cannot be achieved due to the curse of dimensionality. It is common to use a heuristic policy called the index policy, which is obtained by applying one-step policy improvement to a simple initial policy. The index policy performs close to the optimal policy and is easily implementable, which makes it attractive to use in practice. In this dissertation, we first introduce the background information on MDP and index policies in Chapter 1. We then study their applications in two problems: the appointment scheduling problem with patient preferences, and the automated teller machine (ATM) replenishment problem. In Chapter 2, we build an MDP model to design appointment scheduling policies in the presence of patient preferences. We model the patient preferences by assuming that each patient has a set of appointment days that are equally acceptable to the patient. We consider a service provider which receives the appointment-booking requests and makes an appointment decision one at a time. The objective is to minimize the long-run average cost while responding to the patients' booking requests based on their preferences. We propose the index policy and show it performs close to the optimal policy in the two-day horizon and outperforms other benchmarks in the multi-day horizon. In Chapter 3, we build an MDP model to design ATM replenishment schedules, while balancing the cost of replenishments and the cost of stock-outs. We propose a method to establish a relationship between the service level and the cost of a stock-out. We also assume that the replenishment cost is a sub-modular function of the set of ATMs that are replenished together. We derive the index policy, prove it has the same structural properties as the optimal policy, and show it performs close to the optimal policy when there are two or three ATMs. When there are a large number of ATMs, we show the index policy outperforms a benchmark policy through a simulation study and a real-world data-set.
Date of publication
Resource type
Rights statement
  • In Copyright
  • Argon, Nilay
  • Tran-Dinh, Quoc
  • Ziya, Serhan
  • Kulkarni, Vidyadhar
  • Swaminathan, Jayashankar M.
  • Doctor of Philosophy
Degree granting institution
  • University of North Carolina at Chapel Hill Graduate School
Graduation year
  • 2016

This work has no parents.