INTERNATIONAL JOURNAL OF RESEARCH AND INNOVATION IN APPLIED SCIENCE (IJRIAS)  
ISSN No. 2454-6194 | DOI: 10.51584/IJRIAS |Volume X Issue XI November 2025  
Patient Scheduling with Approximate Dynamic Programming for  
Optimization of Health Care Services  
Ajala, O. H., Adebola, F. B  
Department of Statistics, Federal University of Technology, Akure, Nigeria  
DOI: https://dx.doi.org/10.51584/IJRIAS.2025.101100095  
Received: 22 October 2025; Accepted: 08 November 2025; Published: 19 December 2025  
ABSTRACT  
Model that prescribes the optimal appointment date for a patient at the moment this patient makes his request at  
the outpatient clinic is developed. We categorized patients into two. The first category is concerned with patients  
with a maximum recommended waiting time. For these types of patient, the sooner these patients are scheduled  
the better and when the maximum recommended waiting time is exceeded, extra costs are incurred. The other  
category is characterized by a specific appointment time. The closer the scheduled appointment time is to the  
specific appointment time, the lower the costs. The objective is to minimize the long-run expected average cost.  
We modelled the scheduling process as a Markov Decision Process (MDP). we then apply the Bellman Error  
Minimization (BEM) method as an Approximate Dynamic Programming technique in order to derive an estimate  
of the optimal value function of our MDP of which the optimal policy (appointment date) can be derived. To  
determine the set of representative states, which is an element of the BEM method, we use the k-means algorithm.  
We test several approximation functions and find an approximation function that outperforms all other functions  
in the scheduling process over four, six, and eight working days. The Approximation Function B gives the near  
optimal appointment date for patients when appointments are requested. In general, it holds that the higher the  
arrival rate of patients at the outpatient clinic, the better our BEM method performs. But if the arrival rate reaches  
a certain value the load of the system becomes that high that it does not matter what policy is applied, since  
many patients have to be rejected.  
Keywords: Markov Decision Process, Approximate Dynamic Programming, Arrival rate, Policy Improvement  
INTRODUCTION  
Long waiting line is a major challenge in the health care system. Appointment scheduling systems are faced with  
the challenge of ensuring efficiency of health care services as well as timely access to health care services (Gupta  
and Denton, 2008). Being able to access services on time plays a major role in realizing good medical outcomes.  
Difficulty in the accessing health care services e.g. being in the waiting line for a long time can lead to  
complications medically. For instance, a certain health condition may be in its early stage and could be treated  
easily if the patient gets access to the necessary health care service on time, failure to get such service on time  
would lead to the patient having to wait much longer which may result in an aggravation of the patient medical  
condition. More critically is that situations like this could eventually result to the death of the patient.  
Customer’s satisfaction is also an important point to note, as no one enjoys having to wait for a long time in  
order to get a particular service. A better organization of the health care system would therefore yield more  
customer satisfaction. Optimizing appointment system in the health care facility would reduce waiting lines  
(direct and indirect) in the system.  
In optimizing appointment system, we would relatively help to reduce idleness and overtime with specialist. The  
effect of overtime for specialist could be more substantial as it could lead to inefficient rendering of services to  
patient which is so undesirable in a delicate field like medicine. A good appointment system is also a good tool  
for eliminating tardiness (laziness) amongst specialist.  
Page 1012  
www.rsisinternational.org  
INTERNATIONAL JOURNAL OF RESEARCH AND INNOVATION IN APPLIED SCIENCE (IJRIAS)  
ISSN No. 2454-6194 | DOI: 10.51584/IJRIAS |Volume X Issue XI November 2025  
The overall goal of a well-designed appointment system is to facilitate efficiency and effectiveness in the  
delivery of health services for all patients in the health care facility. It facilitates smooth work flow, eliminates  
long waiting lines and allows patient’s preference when requested.  
The global pandemic has had a lot of impact on the need for a good appointment system. To lower its spread,  
protocols has been set in place such as social distancing. Social distancing cannot be underemphasized in the  
health sector. It is the environment that needs the most of its implementation. An appointment system presents  
a way to ensure this. Appointment systems can be used to ensure a maximum number of patients are in the health  
facility at the same time.  
The need for a less crowded health facility has always been a pressing issue-though ignored. Appointment system  
plays a vital role in decongesting clinics and hospitals, as they ensure short wait time and an optimized use of  
health facilities.  
Covid19 has affected a lot of things which include the regular operation of many healthcare service providers.  
Many services have been halted while some have been shifted, leaving patient in wait for a new appointment  
date (Charlton, 2020; COVIDSurg Collaborative, 2020).  
In order to save many lives, an effective and efficient appointment system cannot be overemphasized.  
In other to evaluate the optimization in the health facility, researchers make use of various performance measure.  
Cost-based measures are the performance measure many literatures adopted  
Liu et al. (2010) proposed dynamic policies for appointment system considering no-shows and cancellations and  
discovered that the heuristic policies performed better than other policies. Feldman et al. (2014) advanced Liu  
et al. (2010) and developed a model that considers that a patient may have preference, thereby choosing one of  
the days offered to them by the clinic or leaving without an appointment. Feldman et al. (2014) also paid attention  
to cancellations and no-shows with an objective to maximize the net profit per day.  
Trung (2015) considered a canonical model of dynamic scheduling without considering patients’ preference for  
a specialist and derived an algorithm which assures efficient computation of the policy. Wang et al. (2020)  
developed an optimization model to optimize the appointment system while paying attention to potential walk-  
ins. The study reveals that we cannot consider walk-ins as a reward for no-shows from patients. Diamant et al.  
(2018) looked into how health care schedule patients for multi stage programs such as elective surgery. It was  
observed that high rates of no shows has an effect on the system such as treatment delays. The problem was  
formulated as a Markov decision process and solved using approximate dynamic programming techniques.  
Patrick et al. (2008) were interested in a scheduling multi-priority patient dynamically. They formulated the  
problem as a Markov Decision Process (MDP) and transformed to a linear program in order to solve. However,  
it wasn’t solvable due to a large state space. Approximate Dynamic Programming (ADP) was then used to derive  
an approximate linear program which was then solvable. Erdelyi and Topaloglu (2010) also made use of  
Approximate Dynamic Programming to solve an allocation problem.  
Appointment process as a Markov Decision Process  
We consider two categories of patient. The first category is characterized by waiting time limit. For example, a  
patient that requests for an appointment and wants it scheduled latest five days ahead. The second category is  
characterized by appointment request for a specific date e.g. a patient that needs an appointment with the  
specialist on Saturday. Emergency patient is a type that would fall under the first category while usual patient  
(that sees the specialist regularly) falls under the second category.  
{
}
Mathematically, we take patient to be of type (푥, 푦); 푥 ∈ {1, 2}, 푦 ∈ 1, … , 푌 ; 푥 denote the category while 푦  
ꢀ
denote the type under each category.  
We define several variable for our MDP model  
Page 1013  
www.rsisinternational.org  
INTERNATIONAL JOURNAL OF RESEARCH AND INNOVATION IN APPLIED SCIENCE (IJRIAS)  
ISSN No. 2454-6194 | DOI: 10.51584/IJRIAS |Volume X Issue XI November 2025  
푷풙풚: probability that a patient of type (푥, 푦) will enter the system during a time interval, 푃  
ꢀꢁ  
> 0  
풎풙풚: when x = 1, i.e 푚1ꢁ, this denote the waiting time limit for a patient of type y and when x = 2, i.e 푚2ꢁ, it  
would denote the specific time for a patient of type y. we express 푚ꢀꢁin working days, so 푚ꢀꢁ ∈ {1,2, … }  
풛풙풚: service time required during scheduling for a patient of type (x, y); 푧ꢀꢁ is expressed in sections(blocks), e.g.  
if we have appointment time for a day for two(2) hours(e.g. 10 a.m to 12 a.m); we divide the whole time into  
sections. For instance, a section can be of 5 minutes; this means the whole appointment time would be divided  
into 24 sections. A patient that requires 10 minutes would require two (2) sections.  
풒ퟏ풚: extra cost for not scheduling a patient of type (1, y) within the requested limit. 푞1ꢁ > 0.  
휶풙풚: rejection cost for a patient of type (x, y), 훼ꢀꢁ > 0. The penalty cost of rejecting a patient of type (x, y)  
reflects when a patient is not scheduled in the n-days appointment system, but instead appointed to N+1 days,  
which is not on the present schedule. We would make 훼ꢀꢁ high enough to ensure that the planner would not  
consider pushing patients to a new appointment schedule for certain reasons such as maybe the patient require a  
long service time. Patient should only be rejected when there is no opportunity on the present schedule.  
Formulation of the components of our MDP  
State space  
A good appointment system should have a stipulated time horizon it would span for. We define that time horizon  
as N which is expressed in working days. n = 1 represent the next working day, n = 2 as the next two working  
days and so on. In our model, we are interested in scheduling patients on days of appointment and not the time  
in that particular day.  
For every day, we have a fixed amount of capacity available denoted as B. B is expressed in the number of  
available sections. Service time is expressed in this regard.  
To absorb arrival into the system, we divide the day into intervals, these intervals allow only one event to occur  
in them, which is either a patient comes in or not. We determine the number of intervals mathematically.  
Since we know that arrival during one day follows a Poisson process with parameter 휆. Dividing the day into D  
interval, that means we say arrival in each of the interval, 푑 occur according to a poisson process with parameter  
ꢂ
퐷
The probability that more than one arrival would happen in an interval is as small as less than 0.05 ≡ 5%.  
mathematically,  
푃(푎푟푟푖푣푎푙 > 1) < 0.05  
1 − (푃(푎푟푟푖푣푎푙 = 0) + 푃(푎푟푟푖푣푎푙 = 1)) < 0.05  
0
1
ꢂ
퐷
ꢂ
퐷
휆
ꢃ
휆
ꢃ
( ) 푒−  
( ) 푒−  
1 −  
+
< 0.05  
0!  
ꢂ
1!  
1. 푒−  
퐷
휆
1
ꢂ
퐷
1 −  
+ ( ) 푒− < 0.05  
1
ꢃ
ꢂ
퐷
휆
1 − [1 +  
] 푒− < 0.05  
ꢃ
Page 1014  
www.rsisinternational.org  
INTERNATIONAL JOURNAL OF RESEARCH AND INNOVATION IN APPLIED SCIENCE (IJRIAS)  
ISSN No. 2454-6194 | DOI: 10.51584/IJRIAS |Volume X Issue XI November 2025  
ꢂ
퐷
휆
1 − 0.05 < [1 +  
] 푒−  
ꢃ
ꢂ
퐷
휆
0.95 < [1 +  
ꢃ
] 푒−  
ꢄ
ꢂ
[1 + 퐷] 푒− ≥ 0.95  
(1)  
ꢅ
Therefore to determine D, we solve equation (1) using an arrival rate, 휆  
Action space  
The scheduler accepts or rejects a patient. By rejecting a patient, he/she can assign the patient to a date outside  
the N-day schedule. i.e N + 1  
( )  
= 푎⃗ = (푎1, 푎2, … , 푎푁; 푎푁+1  
퐴
)
⃗
ꢆ,ꢇ  
푎푛 is the action of rejecting or accepting the patient n working days ahead; 푎푛 ∈ (0,1), ꢈ ∈ ꢉ  
푎푁+1 is the action of accepting the patient in a new plan or totally reject the patient from the system 푎푁+1  
∈
(0,1)  
For this model, to ensure we do not go above the available capacity on a particular day  
푖푛 + 푧ꢀꢁ푎푛 ≤ 퐵, ꢈ ∈ ꢉ  
And also 푁+1 푎푛 = 1(patient can only be scheduled on at most one day)  
∑
푛=1  
Cost function  
The cost function would be formulated based on the two category of patient. For the first category, we defined  
already 푚1ꢁ as the waiting time limit for a patient of type y, 푞1ꢁ as the penalty cost for not scheduling such  
patient within the requested limit and 훼1ꢁ as the rejection cost. Cost function is written as  
∑ (ꢈ − 1 + 푞1ꢁ(ꢈ − 푚1ꢁ)+)푎푛 + 훼1ꢁ푎푁+1  
( )  
퐶1ꢁ 푎⃗ =  
[
]
푛 ∈푁  
For the second category, we define 푚2ꢁ as the specific time request by the patient of type y and 훼2ꢁ as the  
rejection cost  
∑ (푚2ꢁ − ꢈ)2푎푛 + 훼2ꢁ푎푁+1  
( )  
퐶2ꢁ 푎⃗ =  
[
]
푛 ∈푁  
Transition probabilities  
Transition occurs in the system in time. The system shifts from one state to another in time depending on 푑 (time  
interval). After an event occur(which could either be an arrival or no arrival), 푑 moves to the next time interval.  
If 푑 < ꢃ, it moves to the next, however if 푑 = ꢃ, this means it is the end of the day. A new day comes in and  
the previous day disappears from the plan. This continues till the end of N-time horizon planned for the  
appointment system. The planner then makes a new N-day plan and starts the same process  
When 푑 < ꢃ and we have an arrival, the transition is from  
(
)
푖1, 푖2, … , 푖푁; 푑 → (푖1 + 푧ꢀꢁ푎1, … , 푖푁 + 푧ꢀꢁ푎푁; 푑 + 1)  
Page 1015  
www.rsisinternational.org  
INTERNATIONAL JOURNAL OF RESEARCH AND INNOVATION IN APPLIED SCIENCE (IJRIAS)  
ISSN No. 2454-6194 | DOI: 10.51584/IJRIAS |Volume X Issue XI November 2025  
If no arrival occurs  
(
)
푖1, 푖2, … , 푖푁; 푑 → (푖1, 푖2, … , 푖푁; 푑 + 1)  
whereas if 푑 = ꢃ which signals the end of the day, If there is an arrival, it can only be put in the next day  
(
)
푖1, 푖2, … , 푖푁; 푑 → (푖2 + 푧ꢀꢁ푎2, … , 푖푁 + 푧ꢀꢁ푎푁, 0; 1)  
For no arrival  
(
)
푖1, 푖2, … , 푖푁; 푑 → (푖2, 푖3, … , 푖푁, 0; 1)  
Having stated the basics of our model, we formulate our optimality equation  
V(ꢊ⃗, 푑) + g =  
(
)
(
)
(
)
핝{  
ꢇ<퐷  
} [휏 ∑ 푃  
ꢀꢁ  
min {퐶ꢀꢁ 푎⃗ + 푉 푖0 + 푧ꢀꢁ푎0, … , 푖푁 + 푧ꢀꢁ푎푁; 푑 + 1 } + 휏 (1 − ∑ 푃 ) 푉 ꢊ⃗, 푑 + 1  
ꢀꢁ  
⃗⃗  
ꢋ ∈ ꢌꢍ⃗,ꢎ  
ꢀ,ꢁ  
ꢀ,ꢁ  
(
) (  
)
+ 1 − 휏 푉 ꢊ⃗, 푑 ]  
( )  
min {퐶ꢀꢁ 푎⃗ + 푉 푖1 + 푧ꢀꢁ푎1, … , 푖푁 + 푧ꢀꢁ푎푁; 0; 1 }  
(
)
+핝{  
ꢇ=퐷  
} [휏 ∑ 푃  
ꢀꢁ  
⃗⃗  
ꢋ ∈ ꢌꢍ⃗,ꢎ  
ꢀ,ꢁ  
(
)
(
) (  
)
+ 휏 (1 − ∑ 푃 ) 푉 푖1, 푖2, 푖3, … , 푖푁; 0; 1 + 1 − 휏 푉 ꢊ⃗, 푑 ]  
ꢀꢁ  
ꢀ,ꢁ  
(2)  
3 Applying BEM to the appointment process  
From equation (2), we have our Bellman error given as  
⃗
⃗
)
(
)
(
D ꢊ⃗, 푑, 푡 = −g − ℧ ꢊ⃗, 푑, 푡  
⃗
( )  
min {퐶ꢀꢁ 푎⃗ + ℧(푖0 + 푧ꢀꢁ푎0, … , 푖푁 + 푧ꢀꢁ푎푁; 푑 + 1; 푡)}  
+핝{ꢇ<퐷} [휏 ∑ 푃  
ꢀꢁ  
⃗⃗  
ꢋ ∈ ꢌꢍ⃗,ꢎ  
ꢀ,ꢁ  
⃗
⃗
)
(
)
(
)
(
+ 휏 (1 − ∑ 푃 ) ℧ ꢊ⃗, 푑 + 1, 푡 + 1 − 휏 ℧ ꢊ⃗, 푑, 푡 ]  
ꢀꢁ  
ꢀ,ꢁ  
⃗
( )  
min {퐶ꢀꢁ 푎⃗ + ℧(푖1 + 푧ꢀꢁ푎1, … , 푖푁 + 푧ꢀꢁ푎푁; 0; 1; 푡)}  
+핝{ꢇ=퐷} [휏 ∑ 푃  
ꢀꢁ  
⃗⃗  
ꢋ ∈ ꢌꢍ⃗,ꢎ  
ꢀ,ꢁ  
⃗
⃗
)
(
)
(
)
(
+ 휏 (1 − ∑ 푃 ) ℧ 푖1, 푖2, 푖3, … , 푖푁; 0; 1; 푡 + 1 − 휏 ℧ ꢊ⃗, 푑, 푡 ]  
ꢀꢁ  
ꢀ,ꢁ  
(3)  
In order to apply this method, we need to determine all the components needed to for its application, which are  
Page 1016  
www.rsisinternational.org  
INTERNATIONAL JOURNAL OF RESEARCH AND INNOVATION IN APPLIED SCIENCE (IJRIAS)  
ISSN No. 2454-6194 | DOI: 10.51584/IJRIAS |Volume X Issue XI November 2025  
1.  
2.  
3.  
4.  
An initial policy;  
The long-run expected average cost for the initial policy, g;  
The set of representative states, Ì ⊂ 퐼  
The weights, 푤(푖), 푖 ∈ Ì  
Initial policy for the BEM method  
We choose a greedy policy which allows patients of type 푥 = 1 to be scheduled as soon as possible while patients  
of type 푥 = 2 would be given appointments as closely as possible to their requested date. Rejection under this  
policy would only happen when the capacity is exhausted for any particular day.  
Long run expected average cost  
We use simulation to determine the long-run expected average cost, g, belonging to a certain policy. The  
simulation is performed as follows: starting with the first day, this day is cut into D intervals. In each interval  
d ∈ {1, … , D} there is one patient arrival of type (푥, 푦) with probability 푝ꢀꢁ or no patient arrival with probability  
∑
ꢀ,ꢁ 푃  
ꢀꢁ  
1 −  
If d ≠ D and there is a patient arrival, this patient is scheduled according to  
⃗
( )  
( )  
min {퐶ꢀꢁ 푎⃗ + ℧(푖1 + 푧ꢀꢁ푎1, … , 푖푁 + 푧ꢀꢁ푎푁; 푑 + 1; 푡)} and the corresponding costs, 퐶ꢀꢁ 푎⃗ , are incurred.  
⃗⃗  
ꢋ ∈ ꢌꢍ⃗,ꢎ  
After the patient is scheduled we move to the next interval. If no arrival occurs, we move directly to the next  
interval. If  
d = D and  
there  
is a  
patient arrival,  
this patient is scheduled according  
( )  
to  
⃗
( )  
min {퐶ꢀꢁ 푎⃗ + ℧(푖2 + 푧ꢀꢁ푎2, … , 푖푁 + 푧ꢀꢁ푎푁; 0; 1; 푡)} and the corresponding costs, 퐶ꢀꢁ 푎⃗ , are incurred.  
⃗⃗  
ꢋ ∈ ꢌꢍ⃗,ꢎ  
After the patient is scheduled we move to the next day and shift the schedule. The first day disappears from the  
schedule, the second day becomes the first day, the third day becomes the second day and so on, and finally a  
new empty day enters the schedule and we start with d = 1. If no arrival occurs, we move directly to the new  
day and shift the schedule. We let the simulation run over ꢈ days. At the end of the simulation we can obtain g  
by dividing the total costs incurred by the length of the simulation, ꢈ ∗ D. Note that the initial policy can be  
⃗⃗  
⃗
(
)
achieved by setting the parameter vector 푡 to zero. In this case ℧ ꢊ⃗, 푑, 0 equals zero, ∀ ꢊ⃗, 푑 and hence, the actions  
to choose only depends on the cost function. To make sure we only reject patients if there is no sufficient capacity  
available in any day, the rejection costs must be chosen higher than the highest costs that can be obtained when  
patients are scheduled.  
Set of representative states and the corresponding weight vector  
The set Ì should contain the most important states in the state space, while w should represent the importance of  
the states in Ì. The choice of the set of representative states could be to include only the states that have a high  
probability of being visited.  
Step 1 we get a list of sampled state S.  
Step 2 K-means clustering From this list, we want to cluster these states into K clusters. As clustering technique,  
we use Elkan algorithm. The algorithm returns the clusters and the cluster centre.  
Step 3 Determine the set of representative states and the corresponding weight vector For each cluster centre we  
want to find the state with the shortest Euclidean distance as the most representative state of this cluster. Hence,  
after this step, we have a set of K representative states, which will be our Ì and the number of states in each  
cluster will determine our weight vector w.  
Besides the patient-specific parameters, our model has a few other parameters that need to be determined. These  
parameters are: N, the number of working days which covers the scheduling process; B, the fixed amount of  
capacity available on any day; λ, the rate for patient arrivals and from which parameter D can be determined,  
Page 1017  
www.rsisinternational.org  
INTERNATIONAL JOURNAL OF RESEARCH AND INNOVATION IN APPLIED SCIENCE (IJRIAS)  
ISSN No. 2454-6194 | DOI: 10.51584/IJRIAS |Volume X Issue XI November 2025  
and 휏 needed for the data transformation to overcome the problem of aperiodicity. We set B to 20 blocks and τ  
to 0.9.  
Table 1: Values for different parameters  
Parameters  
Values  
D
S
{ 31, 33, 36, 38, 40}  
{1000000, 2000000, 3000000}  
{50, 100, 150}  
K
Table 2: Different types of basis functions  
Number  
0
Basic function  
3
푁
푑 + ∑ ∑ 푖푛ꢏ  
ꢏ=1 푛=1  
푁−1  
1
2
∑ 푖푛 ∗ 푖푛+1  
푛=1  
푁−2  
∑ 푖푛 ∗ 푖푛+1 ∗ 푖푛+2  
푛=1  
푑2  
3
4
푁
∑ 푖푛 ∗ 푑  
푛=1  
푁
5
6
7
∑ 푖푛2 ∗ 푑  
푛=1  
푁−1  
∑ 푖푛 ∗ 푖푛+1 ∗ 푑  
푛=1  
푁−2  
∑ 푖푛 ∗ 푖푛+1 ∗ 푖푛+2 ∗ 푑  
푛=1  
Each approximation function we use contains basis function 0(Roubos, 2010). The other basis functions contain  
several cross terms between different parts of the state space. If we refer to approximation function 046, then  
this approximation function consist of the basis functions: 0, 4 and 6. for example If N = 3 and we refer to  
2
3
approximation function 046, then ℧ ꢊ⃗, 푑, 푡 = 푑푡1 + 푖1푡2 + 푖2푡3 + 푖3푡4 + 푖1 푡5 + 푖22푡6 + 푖32푡7 + 푖1 푡8 +  
⃗
(
)
푖23푡9 + 푖3 푡10 + 푖1푑푡11 + 푖2푑푡12 + 푖3푑푡13 + 푖1푖2푑푡14 + 푖2푖3푑푡15  
3
Page 1018  
www.rsisinternational.org  
INTERNATIONAL JOURNAL OF RESEARCH AND INNOVATION IN APPLIED SCIENCE (IJRIAS)  
ISSN No. 2454-6194 | DOI: 10.51584/IJRIAS |Volume X Issue XI November 2025  
Bottom-up Approach  
We start with the following approximation functions: {0, 01, 02, 03, 04, 05, 06, 07}. From here we use a so-  
called bottom up approach. We take the functions that show the best improvements overall and then add the  
other remaining functions one at a time. For instance, if function 05 performs best, then we make the following  
new combinations: {051, 052, 053, 054, 056, 057}. This is repeated until no further improvement occurs. At the  
end we have a set of approximation functions which shows in general the best improvements for a scheduling  
process over four working days. To test if these set of approximation functions also perform well for a larger  
scheduling process we expand our model to N = 6 and 8. The parameters S and K are set to the value that in  
general performs best in the model with N = 4.  
Data Analysis on Four, Six and Eight working days  
Four working days  
For each combination of the parameters in Table 3 we apply the BEM method. We can make 5∗3∗3 = 45  
combinations, for each combination we apply the BEM method with 8 different approximation functions. Each  
time we apply the BEM method, we compare 푔 obtained from our initial policy with 푔 obtained after the one-  
step policy improvement and compute the improvement that is made. We refer to this as the improvement of the  
BEM method.  
Table 3: Median and average of the improvement for each approximation function.  
Function  
Mean (%)  
7.26  
Median (%)  
6.37  
0
01  
02  
03  
04  
05  
06  
07  
-3.51  
-3.80  
6.70  
-2.60  
-2.69  
5.74  
7.00  
5.70  
6.41  
5.26  
6.57  
5.30  
7.16  
5.75  
we remove the value of 휆 = 11, 12. It seems plausible logically that a low 휆 would mean a low load on the  
system, thereby making the initial policy a good policy. We use the BEM method for a reduced number of D.  
we have 3∗3∗3 = 27 combinations remaining. Table 4 shows for the remaining approximation functions the  
average and median of the improvement of the BEM method over 27 combinations. As can be seen functions 07  
give the best results with an average improvement of 7.16%. we start with our bottom up approach with all of  
the functions over the 27 combinations.  
Table 4: Median and average of the improvement for each approximation function for a reduced number of D.  
Function  
Mean (%)  
6.06  
Median (%)  
6.25  
0
01  
6.17  
4.62  
Page 1019  
www.rsisinternational.org  
INTERNATIONAL JOURNAL OF RESEARCH AND INNOVATION IN APPLIED SCIENCE (IJRIAS)  
ISSN No. 2454-6194 | DOI: 10.51584/IJRIAS |Volume X Issue XI November 2025  
02  
03  
04  
05  
06  
07  
6.28  
5.85  
6.01  
4.95  
4.67  
3.91  
5.36  
6.45  
5.25  
5.94  
5.47  
7.16  
Table 5 shows the results from the first step of the bottom up approach. For each approximation function the  
average and median of the improvement of the BEM method are given. For function 04 it holds that the mean  
increases slightly from 5.25% to 6.20% when function 1 is added. Adding one of the other functions does not  
improve the average or median. For function 05, an improvement is made when function 3 is added. We also  
have an improvement made with function 06 when function 1 or 3 is added. Therefore, in our second step of the  
bottom up approach we start with 041, 053, 061 and 063.  
Table 5: Median and average of the improvement for each approximation function after one step.  
Function  
021  
Mean (%)  
4.85  
Median (%)  
4.04  
023  
5.12  
3.68  
024  
5.08  
3.95  
025  
4.85  
4.02  
026  
4.78  
3.88  
027  
5.44  
4.07  
(a) Function 01  
Function  
012  
(b) Function 02  
Median (%)  
3.56  
Mean (%)  
4.66  
013  
4.99  
3.26  
014  
4.24  
3.24  
015  
5.22  
3.07  
016  
4.94  
3.20  
017  
4.30  
2.95  
Function  
041  
Mean (%)  
6.20  
Median (%)  
6.11  
Page 1020  
www.rsisinternational.org  
INTERNATIONAL JOURNAL OF RESEARCH AND INNOVATION IN APPLIED SCIENCE (IJRIAS)  
ISSN No. 2454-6194 | DOI: 10.51584/IJRIAS |Volume X Issue XI November 2025  
042  
043  
045  
046  
047  
4.89  
3.75  
1.55  
3.75  
4.16  
3.84  
2.37  
4.59  
5.14  
4.72  
(c) Function 03 (d) Function 04  
Function  
031  
Mean (%)  
5.06  
Median (%)  
4.12  
032  
034  
035  
036  
037  
5.17  
4.00  
4.66  
3.71  
5.53  
3.76  
5.19  
4.12  
5.19  
4.24  
Function  
061  
Mean (%)  
5.67  
Median (%)  
4.04  
062  
5.32  
4.22  
063  
9.21  
5.74  
064  
5.21  
4.16  
065  
5.24  
3.58  
067  
4.86  
3.91  
(e) Function 05  
(f) Function 06  
Function  
051  
Mean (%)  
4.95  
Median (%)  
4.12  
052  
5.32  
4.13  
053  
8.29  
6.48  
054  
5.49  
3.77  
056  
-4.89  
-3.79  
Page 1021  
www.rsisinternational.org  
INTERNATIONAL JOURNAL OF RESEARCH AND INNOVATION IN APPLIED SCIENCE (IJRIAS)  
ISSN No. 2454-6194 | DOI: 10.51584/IJRIAS |Volume X Issue XI November 2025  
057  
4.66  
3.68  
(g) Function 07  
Function  
071  
Mean (%)  
5.00  
Median (%)  
3.87  
072  
5.04  
3.67  
073  
1.03  
0.95  
074  
5.09  
3.86  
075  
5.10  
4.04  
076  
5.02  
3.54  
Table 6 shows the results of the second step of the bottom up approach. As can be seen, no further improvement  
is obtained.  
Table 6: Median and average of the improvement for the different functions after two steps.  
Function  
0531  
Mean (%)  
7.80  
Median (%)  
5.79  
0532  
4.63  
3.42  
0534  
1.45  
1.43  
0536  
0.78  
1.00  
0537  
5.09  
3.77  
(a) Function 041  
Function  
0412  
(b) Function 053  
Mean (%)  
5.37  
Median (%)  
3.84  
0413  
5.16  
3.73  
0415  
5.77  
4.03  
0416  
5.34  
3.87  
0417  
5.05  
3.76  
(c) Function 061  
Function  
(d) Function 063  
Mean (%)  
1.00  
Median (%)  
1.00  
0612  
Page 1022  
www.rsisinternational.org  
INTERNATIONAL JOURNAL OF RESEARCH AND INNOVATION IN APPLIED SCIENCE (IJRIAS)  
ISSN No. 2454-6194 | DOI: 10.51584/IJRIAS |Volume X Issue XI November 2025  
0613  
0614  
0615  
0617  
-4.04  
-2.13  
1.00  
1.00  
1.00  
1.00  
1.00  
1.00  
Function  
0631  
Mean (%)  
5.09  
Median (%)  
3.95  
0632  
4.77  
3.79  
0634  
5.34  
3.26  
0635  
5.05  
3.78  
0637  
5.55  
4.18  
Function 041, 053, 061 and 063 are the function that gives the best improvements during the one-step policy  
improvement, based on the median and average for the scheduling process over four working days. Therefore,  
we apply these functions to the scheduling process over six, and eight working days. Since function 07 also  
performs very good for the scheduling process over four working days, we also apply these functions to the  
scheduling process over six, and eight working days. To simplify the figures in the following sections, we create  
a translation table, see Table 7. From here, if we write about function A, we actually mean function 07.  
Table 7: Translation table for the different functions.  
New Function Name  
Old Function Name  
A
B
C
D
E
07  
041  
053  
061  
063  
Figure 1 shows for each 휆 the average improvement of the BEM method by each of the approximation function.  
By improvement, we refer to the improvement made when we compare 푔 obtained from our initial policy with  
푔 obtained after the one-step policy improvement. We see that if 휆 ≤ 12, the average improvement for each  
function increases as 휆 increases. When 휆 > 12 the average improvement for each function seem to decrease as  
휆 decreases. The lower 휆, the lower the load of the system which infer the better our initial policy performs and  
hence, less improvement is possible. Whereas on the other hand, the higher 휆, the higher the load of the system  
which infer the worse our initial policy performs and hence, the more important our one-step policy  
improvement. But if 휆 reaches a certain value, the load of the system becomes that high that it does not matter  
what policy is applied, since it will be imperative to reject many patients.  
Page 1023  
www.rsisinternational.org  
INTERNATIONAL JOURNAL OF RESEARCH AND INNOVATION IN APPLIED SCIENCE (IJRIAS)  
ISSN No. 2454-6194 | DOI: 10.51584/IJRIAS |Volume X Issue XI November 2025  
Figure 1: Average improvement by 휆 for different functions for the scheduling process over four working days  
Six, Eight working days  
We apply the BEM method over six and eight working days. The results of the approximation functions {A, B,  
C, D, E} of the scheduling process over six, eight days are given. The parameters S and K needed for the BEM  
method are fixed to 1000000 and 50 respectively. Figure 2 shows for each 휆 the average improvement of the  
BEM method by the different functions for the scheduling process over six working days. It shows more or less  
the same pattern as the results of the scheduling process over four working days. We see that if 휆 ≤ 12, the  
average improvement for each function increases as 휆 increases. When 휆 > 12 the average improvement for each  
function seem to decrease as 휆 decreases.  
Figure 2: Average improvement by 휆 for different functions for the scheduling process over six working days  
Figure 3 shows for each 휆 the average improvement of the BEM method by the different functions for the  
scheduling process over 8 working days. Function {A, B, E} shows the same pattern as the results for four and  
six working days. For function {A, B, E}, the threshold is at 휆 = 12. Function C shows a different pattern than  
have been seen before. It has its highest result at 휆 = 11 and continues with a decline thereafter. At 휆 = 11 the  
average improvement gives 4.19% which apparently shows the importance of the one-step policy with this load  
of the system. Function D also shows a similar pattern to Function C. It has its highest result at 휆 = 11. The  
average improvement then decreases to 2.76% After which the average improvement is increased at 휆 = 14  
Page 1024  
www.rsisinternational.org  
INTERNATIONAL JOURNAL OF RESEARCH AND INNOVATION IN APPLIED SCIENCE (IJRIAS)  
ISSN No. 2454-6194 | DOI: 10.51584/IJRIAS |Volume X Issue XI November 2025  
Figure 3: Average improvement by 휆 for different functions for the scheduling process over eight working days  
Table 8: Average of the improvement by functions for four, six and eight working days.  
Function  
ꢉ = 4  
Avg (%)  
5.571572  
5.628243  
5.408628  
5.660669  
4.576536  
ꢉ = 6  
ꢉ = 8  
Avg (%)  
2.105545  
2.123410  
2.344520  
2.446660  
1.874466  
Avg (%)  
A
B
C
D
E
3.601719  
3.889145  
3.225292  
3.529022  
3.183158  
Table 8 shows for the scheduling process over four, six and eight working days for each function the average of  
the improvement of the BEM method relative to the initial policy. Overall, function B give the overall best  
improvement over four, six and eight working.  
CONCLUSION  
With the goal to develop a model that prescribes the optimal appointment date for a patient at the moment this  
patient makes his request. From all combinations of the set of basis functions, the following combination  
outperforms all other combinations:  
3
푁
푁
푁−1  
푑 + ∑ ∑ 푖푛ꢏ + ∑ 푖푛 ∗ 푑 + ∑ 푖푛 ∗ 푖푛+1  
ꢏ=1 푛=1  
푛=1  
푛=1  
The overall average improvement of the Approximation Function B compared to the initial policy over four, six  
and eight working days is 11.640798%. In general, it holds that the lower λ, the lower the low load of the system,  
the better our initial policy performs and hence, less improvement is obtained. The higher the load of the system,  
the worse our initial policy performs and the more important is our one-step policy improvement. But if λ reaches  
a certain value the load of the system becomes that high that it does not matter what policy is applied, since  
Page 1025  
www.rsisinternational.org  
INTERNATIONAL JOURNAL OF RESEARCH AND INNOVATION IN APPLIED SCIENCE (IJRIAS)  
ISSN No. 2454-6194 | DOI: 10.51584/IJRIAS |Volume X Issue XI November 2025  
many patients have to be rejected. we have been able to make use of Approximate Dynamic Programming to  
solve problems that arises in appointment system in the health care facility by developing a model that prescribes  
the optimal appointment date for a patient at the moment this patient makes his request.  
REFERENCES  
1. Bailey, N.T.J. (1952). A study of queues and appointment systems in hospital out-patient departments,  
with special reference to waiting-times. Journal of the Royal Statistical Society. Series B  
(Methodological)14(2) 185 ̶ 199.  
2. Caryirli, T. and Veral, E. (2003). Outpatient scheduling in health care: a review of literature. Production  
and operations management, 12(4):519 ̶ 549.  
3. Caryirli, T., Yang, K. K. and Quek, S. A. (2012). A universal appointment rule in the presence of no-  
shows and walk-ins. Production and Operations Management, 21(4):682 ̶ 697.  
4. Charlton, E. (2020). 28 million elective surgeries may be cancelled worldwide: how non COVID-19  
medical  
care  
is  
suffering.  
World  
Economic  
Forum.  
Accessed  
27  
January  
2021  
https://www.weforum.org/agenda/2020/05/covid-19-elective-surgery cancellation-cancer-pandemic.  
5. COVIDSurg Collaborative. (2020). Elective surgery cancellations due to the covid-19 pandemic: global  
predictive modelling to inform surgical recovery plans. British Journal of Surgery 107(11) 1440 ̶ 1449.  
6. Deo, S., Iravani, S., Jiang, T., Smilowitz, K. and Samuelson, S. (2013). Improving health outcomes  
through better capacity allocation in a community-based chronic care model. Operations Research 61(6)  
1277 ̶ 1294.  
7. Diamant, A., Milner, J. and Quereshy, F. (2018). Dynamic patient scheduling for multi-appointment  
health care programs. Production and Operations Management 27(1) 58 ̶ 79.  
8. Erdelyi, A. and Topaloglu, H. (2010). Approximate dynamic programming for dynamic capacity  
allocation with multiple priority levels. IIE Transactions, 43(2):129 ̶ 142.  
9. Erdogan, S. A., Gose, A. and Denton, B. T. (2015). Online appointment sequencing and scheduling. IIE  
Transactions, 47(11):1267 ̶ 1286.  
10. Feldman, J., Liu, N., Topaloglu, H. and Ziya, S. (2014). Appointment scheduling under patient preference  
and no-show behavior. Operations Research, 62(4):794 ̶ 811.  
11. Fries B. and Marathe, V. (1991). Determination of Optimal Variable-Sized Multiple-Block Appointment  
Systems, Operation Research, 29(2), 324 ̶ 345.  
12. Grais, R. F., Dubray, C., Gerstl, S., Guthmann, J. P., Djibo, A., Nargaye, K. D., et al. (2007).  
Unacceptably high mortality related to measles epidemics in Niger, Nigeria, and Chad. PLoS Med; 4:e16  
13. Gupta, D. and Denton, B. (2008). Appointment scheduling in health care: Challenges and opportunities.  
IIE transactions, 40(9):800 ̶ 819.  
14. Gupta D. and Wang, L. (2008). Revenue management for a primary-care clinic in the presence of patient  
choice. Operations Research, 56(3):576 ̶ 592.  
15. Heaney, D. J., Howie, J. G. and Porter, A. M. (1991). Factors Influencing Waiting Times and  
Consultation Times in General Practice, British Journal of General Practice, 41, 315 ̶ 319.  
16. Kaandorp, G. C. and Koole, G. (2007). Optimal outpatient appointment scheduling. HealthCare  
Management Science, 10(3):217 ̶ 229.  
17. Keller T. F., Laughhunn. D. J. (1973). An Application of Queuing Theory to a Congestion Problem in  
an Outpatient Clinic. Decision Sciences, 4, 379 ̶ 394.  
18. Klassen, K. J. and Rohleder, T. R. (1996). Scheduling Outpatient Appointments in a Dynamic  
Environment, JJournal of Operations Management, 14, 2, 83 ̶ 101.  
19. Lindley, D. V. (1952). The Theory of Queues with a Single Server, Proceedings Cambridge Philosophy  
Society, 48, 277 ̶ 289.  
20. Liu, N., Ziya, S. and Kulkarni, V. G. (2010). Dynamic scheduling of outpatient appointments under  
patient no-shows and cancellations. Manufacturing & Service Operations Management, 12(2):347 ̶ 364.  
21. Minerva  
Strategies,  
Addressing  
health  
challenges  
in  
Nigeria.  
Accessed  
21  
January2021,https://www.minervastrategies.com/blog/addressing-health-challenges-in-nigeria/.  
22. Murray, M. and Berwick, D.M. (2003) Advanced access: reducing waiting and delays in primary care.  
Journal of the American Medical Association, 289, 1035–1040.  
Page 1026  
www.rsisinternational.org  
INTERNATIONAL JOURNAL OF RESEARCH AND INNOVATION IN APPLIED SCIENCE (IJRIAS)  
ISSN No. 2454-6194 | DOI: 10.51584/IJRIAS |Volume X Issue XI November 2025  
23. Onwujekwe, O., Onoka, C., Uguru, N., Nnenna, T., Uzochukwu, B., Eze, S., et al. Preferences for benefit  
packages for community-based health insurance: An exploratory study in Nigeria. BMC Health Services  
Research. 2010 10:162.  
24. Panaviwat, C., Lohasiriwat, H. and Tharmmaphornphilas, W. (2014). Designing an appointment system  
for an outpatient department. Materials Science and Engineering  
25. Patrick, J., Puterman, M. L. and Queyranne, M. (2008). Dynamic multi priority patient scheduling for a  
diagnostic resource. Operations research, 56(6):1507 ̶ 1525.48Bibliography.  
26. Puterman, M. L. (2005). Markov decision processes: discrete stochastic dynamic programming. John  
Wiley & Sons.  
27. Roubos, D. (2010). The application of approximate dynamic programming techniques. PhD thesis, VU  
Amsterdam, the Netherlands.  
28. Swisher, J. R., Jacobson, S. H., Jun, J. B. and Balci, O. (2001). Modeling and Analyzing a Physician  
Clinic Environment Using Discrete-Event (Visual) Simulation, Computers & Operations Research, 28,  
2, 105 ̶ 125.  
29. Tijms, H. C. (2003). A first course in stochastic models. John Wiley & Sons.  
30. Truong, V.A. (2015). Optimal advance scheduling. Management Science 61(7) 1584 ̶ 1597.  
31. Wang, S., Liu, N. and Wan, G. (2020). Managing appointment-based services in the presence of walk-  
in customers. Management Science 66(2) 667 ̶ 686.  
32. Wolfswinkel, E. (2017). Scheduling patients at the outpatient clinic. Master thesis, VU Amsterdam, the  
Netherlands.  
33. Zacharias, C., Liu, N. and Begen, M. A. (2020). Dynamic Inter-day and Intra-day Scheduling  
34. Zhou, M., Loke, G. G., Bandi, C., Liau, Z. Q. and Wang, W. (2020). Intraday Scheduling with Patient  
Re-entries and Variability in Behaviours. Analytics and operations forth coming.  
Page 1027  
www.rsisinternational.org