Fluent Essays

Title: Anticipate rush hour times and places in high demand for Uber drivers Research Area: Data mining  technical content:  Using machine learning algorithms to make predictions  the purpose of this research paper:  after the research paper, I will develop a model that makes High Demand and rush hour time prediction to implement solutions in advance.  ======================= Research paper outlines: I- Title page and abstract  II- Introduction (should include the following):  a. Motivation/scope (1-2 pages).  b. Problem statement (< 3 pages)and significance for IS.  c. Importance of the problem, big picture.  d. Objectives: 3 to 5 sentence summary of overall goals of the project.  III- Literature review (20-30 pages) organized from the general to specific, subheadings reflecting internal organization, and may include: theoretical basis, related research, and similar research methods.  IV- Discussion and anticipated applications (5-10 pages)  V- Conclusion and Future Work (2-3 pages)  VI- References (20-30 references)  you can find an example in the attached file. the similarity check should be 15\% or below. Predicting No-Shows for Dental Appointments Using Machine Learning Techniques Student Name: Bassil Makanati Student ID: 441106223 Course: IS595 Advisor Name: Dr. Yazeed Alabdulkarim 2 Abstract Medical no-show is defined as not showing up or missing a patient’s booked appointment for routine visit, dental clinic, radiology imaging, etc. It is a common problem with rates reaching up to 50\% worldwide. In this paper, we shed light on the factors that contribute to this issue and its negative impact on the patient, medical practitioner and most importantly the draining costs on the healthcare system. Over many years, solutions have been suggested and implemented. We review related work and existing techniques to address this problem in this phase of the project. In the second phase, we will develop a model based on machine learning techniques such as; logistic regression, neural network, and gradient boosting, to predict no-shows. The data used in this study was obtained from a dental clinic in Saudi Arabia in 2019. The assessment of the best model will be evaluated by Area Under the ROC Curve (AUC) and F1-score. 3 Contents Abstract ....................................................................................................................... 2 1. Introduction ............................................................................................................. 4 2. Literature Review ..................................................................................................... 6 2.1 Related Work ...................................................................................................... 6 2.1.1 Used Features ............................................................................................... 8 2.1.2 Used Models .............................................................................................. 10 2.1.3 No-show Mitigation Strategies ..................................................................... 14 2.1.3.1 Reminders ........................................................................................... 14 2.1.3.2 Overbooking........................................................................................ 14 2.1.3.3 Financial Penalties ............................................................................... 15 2.2 Background of Main Machine Learning Techniques .............................................. 15 2.2.1 Supervised Learning ................................................................................... 16 2.2.1.1 Random Forest .................................................................................... 17 2.2.1.2 Logistic Regression .............................................................................. 18 2.2.1.3 Neural Network ................................................................................... 19 2.2.1.4 Gradient Boosting ................................................................................ 22 2.3 Background about Metrics .................................................................................. 23 2.3.1 Confusion Matrix ........................................................................................ 25 2.3.2 F1-Score .................................................................................................... 27 2.3.3 AUC (Area Under the ROC Curve) .................................................................. 28 3. Data Analysis ........................................................................................................ 30 4. Methodology ......................................................................................................... 33 4.1 Preprocessing.................................................................................................... 33 4.2 Feature Engineering ........................................................................................... 34 4.3 Model Selection ................................................................................................ 34 4.4 Evaluation ......................................................................................................... 34 5. Conclusion and Future Work ................................................................................... 35 References ................................................................................................................. 36 4 1. Introduction No-show is a major problem in the healthcare outpatient system, which refers to patients missing their booked appointments. Patient no-shows are common, and their rates may reach up to 50\% [1]. No-show rates vary across different medical specialties [2]. For instance, no-show rates for Urology (6.5\%), oncology (6.9\%), whereas surgery was (16.6\%), optometry (23.8\%) and dietary (27.23\%) [3]. It is roughly estimated that the patient no-show costs the US healthcare system an average of 150 billion a year [4]. No-show has many reasons and contributing factors, most but not all, are patient related. Factors include inconvenience of the timing to the patient, inability to get to the appointment due to transportation issues, inability to leave a job at a certain time, or forgetfulness. Other causes are failure of the health system to make the outpatient visit a pleasant one, due to longer waiting time, poor patient management, etc. Consequences of this problem spread widely, affecting all involved parties. Patients are perhaps the most affected by no-shows, whether it is the patient that has missed the appointment herself or those that were not booked due to unavailability of appointments. When a patient misses her appointment, she loses the chance to be treated. This may impact the patients health and lead to complications that would warrant costly and bothersome medical interventions, such as the need for visiting the emergency room, or performing a surgery. So no-show delays treatment which may allow for complications of a health issue to arise, lengthen patient recovery, and increase rates of morbidity and mortality. Not only that, but it also increases the number of healthcare professionals that need to be involved along with the consumed materials, adding up to the increased costs. 5 Outpatient clinics are implementing basic strategies in hopes of altering the no-show rates to the better, however not all of them have proven to be effective. For example, sending reminders to booked patients did not improve the no-show rate [5]. Blind overbooking is not optimal as well, as the expected no-show rates are not computed on a per patient basis. This would be problematic if implemented aggressively, as it would crowd the clinic and over utilize its resources and increase the wait time for patients as well as leading to dissatisfaction and over working the medical practitioners. On the other hand, if overbooking is conservative, resources would be wasted, and the clinic is yet again under booked [2]. Financial penalties are another method for reducing no- show rates, to those who missed appointments. Studies have shown that this may help reduce the occurrence of no-show [5] [6]. This paper utilizes various machine learning techniques, such as neural network, random forest, and gradient boosting classifiers, along with logistic regression to develop a predictive model for no-shows. This could help foresee the number of possible patients that would miss their appointments. This solution would help in dealing with the issue beforehand, giving time to think ahead and overbook patients, which allows for less time wasting for the available patients and the health care practitioners. It also prompts the responsible parties to send texts or emails to the patients that are likely going to miss the appointment which might remind them of it to make certain arrangements to their schedules, increasing the likelihood of showing up. Several papers in the literature [7], [8], [9], [3], [10], [11] have developed predictive models for patient no-shows. As no-show rates differ by specialties, this paper focuses on dental appointments. Dental appointments are longer than regular medical appointments due to the complexity of dental procedures. The average length of 6 primary care appointments is 17.4 minutes while the average for dental appointments is 48.7 minutes, as reported by the American Dental Association [12]. Dental appointments are almost three times the duration of primary care appointments, which implies a more significant impact of dental no-shows. The economic effect of dental patients missing their appointments was estimated to cost the health ministry in the United Kingdom around 300 million pounds per year [13]. The resources employed by the dental clinic remain unutilized while the dentist waits for the patient to show up and increases the wait time of the next booked patient [14]. The effect on the patient is of course, delaying the diagnosis of the, which in turn delays treatment and makes room for complications to arise, naturally increasing time and cost spent on this particular patient [15]. On the other hand, studies have shown that compliant patients to dental appointments especially, had less incidence of complications like bleeding, periodontitis, and incidence of tooth loss [16]. 2. Literature Review 2.1 Related Work No-show behavior is an age-old problem, and it exists in several domains [17], [18], [19], such as healthcare, airlines, etc. Early papers in the literature [20], [21] tried to determine the significant predictors of no-show behavior. For example, previous no- shows and age. They also evaluated the effectiveness of various interventions, such as sending reminders, to mitigate for no-show behavior. For predicting no-shows, early work used statistics and data mining techniques, such as decision trees [20] and association rules [22]. These techniques are used to discover patterns and properties of the data. They are unable to effectively capture and predict complex behavior, such as no-shows, that involve several factors. Furthermore, these techniques are not able to adapt with changing behavior and new data effectively. 7 The early 2010s have seen a proliferation of studies to predict no-shows using various machine learning techniques, including random forests [8], [9], [10], [11], logistic regression [23], [6], [1], [24], [25], [7], [8], [9], [11], neural networks [1], [10], gradient boosting [8], naïve bayes [1], [8], support vector machine [8], [9], hoeffding trees [3] and others. Logistic regression is easy to use and implement in excel for example. Other papers used different models and techniques in combinations like logistic regression, artificial neural network and naïve bayes classifier to compare results and what yields the best score. The different models were chosen based on the type of data. When interpreting the evaluation results the type of dataset must be taken into consideration as results would vary depending on the dataset. Most papers used AUC-ROC (Area Under the Curve of Receiver Characteristic Operator) as an evaluation metric and the best scores ranged from 0.6 [11] to 0.95 [24]. As no-show behavior varies by demographics and clinic specialties, several papers have contributed by studying data from different populations and specialties. These papers collected data from the United States [23], [6], [1], [24], [4], [8], Brazil [5], [7], [9], [11], Italy [25], Columbia [10], and Saudi Arabia [3]. Data included various specialties; such as adult primary care [23], [1], [4], [5], [7], [8], [10], pediatric primary care [26], mental health, rehabilitation, surgery [23], radiology [6], Cardiology, Dermatology, Urology [24], diagnostic procedure clinic [25] and others. Some papers used different techniques to capture features or select predictive models. For example, a paper used a Markov model to capture patients’ no-show history based on the last 10 appointments [23]. This model gives more weight to recent appointments, rather than averaging previous appointments and treating them the same. Others in Brazil used Akaike Information Criteria (AIC) selected the best model, with a minimum number of considered features to reduce complexity and improve performance [5], [7]. 8 Some introduced AutoEncoder (AE) and Factor Analysis of Mixed Data (FAMD) as possible options for dimensionality reduction in the no-show prediction [9]. While other studies designed a decision support system to cluster patients into three-level groups of targeted interventions based on their no-show probabilities [10]. The first group (group A) involved 30\% of the patients, and no extra action will be taken for them due to financial and operational limitations. As for the second group (B), which pertains 40\% of the patients, low costing technology was used as an intervention like SMS reminders. For the third and final group (C), containing the rest of the 30\% of the patients, individualized interventions such as educational programs to improve attendance numbers [10]. 2.1.1 Used Features On analyzing the different factors that were considered important or predictive of no- show, different studies have shown different results. Some predicted the likelihood of missing future appointments by the past appointment history. Patients who previously missed appointments were more likely to miss future ones [23], [6], [1], [24], [26], [7], [9], [3]. This perhaps was the most important and predictive of no-shows in almost all previous studies. Lead time was also an important factor, which is defined as the time between scheduling the appointment and the date of the appointment. The shorter the lead time, the more likely the appointment was attended [23], [1], [26], [7], [10], [11]. Longer lead times increase no-show odds. Patients who are scheduled for appointments with lead times between 8 and 10 days are more likely to show up to their booked appointments [10]. Therefore, appointments scheduled on the same day had higher rates for attendance [9]. The day of the week is also a significant factor; the majority of papers observed that the best attendance levels were seen during the weekend [23], [6], [1], [26], [7], [9], [10], 9 [11]. In some papers, this was the most important of the predictive features [1]. In one paper, it was observed that 92\% of the patients kept their appointments on Sunday, this indicator decreases to 69\% on Fridays [6], [1], [24], [4], [8], [9]. The timing of the appointment was also relevant in some studies [6], [1], [24], [8]. Type of patient was a factor; whether the patient was new to the clinic or had regular follow ups was also observed to have an effect on showing up [25]. Reminding patients of appointments and confirmation obviously reduced the rates of not showing up [25]. Being able to confirm appointments is of course dependent on the ability to reach the patients by any means [24]. Naturally, patients with no cell phones were less likely to be reminded and therefore more likely to miss appointments than those with cell phones. The time from the call that scheduled the appointment to the actual date was significant [4]. Reasons for why the appointment was booked are contributory [8]. As an acute urgent issue was logically less likely for no-show, opposite was the behavioral health visit which had the highest rate of missed appointments [1]. On the other hand, chronic issues or complaints that have been resolved that mandated the patient’s attendance, if resolved, were likely that the patient would not attend [6]. The duration or length of the appointment mattered, the shorter and more efficient the better the attendance was [24]. Having multiple appointments per day actually reduced the probability of no-show by 38\% [23]. Patients scheduled with their own PCP (primary care provider) were less likely to miss the appointment since they are familiar with the physician and usually follow with them for many years or even decades [1], [24], [4]. So, patient-provider relationships were contributors to no-show [26]. In one study, interestingly, the name of the medical 10 provider was observed to play a role [8], as well as the provider’s type and race mattered [1]. The type of insurance played a major role in some studies [6], [24], [4] and financial status or income level [6], [24], [26], [25], [8] along with the number of family members or dependents [26], [8]. There was also the demographics like the age factor; especially ages between 20 to 40 years [25], [7], [10], the older the patient, the more likely he/she were adherent to attendance, but it was a less significant factor in other studies [23], [6]. Male gender contributed to more no-show cases [23], [6], [24], [4], [26], [25], [10], [11]. Race interestingly was a factor [24], [4], [8]. Marital status had an impact as well, having a spouse was associated with lower probability of missed appointments [23], [4], [8]. Underserved population and distance from the hospital, the farther it was, the less likely the patient showed up [6], [24], [4], [26], [8]. Interestingly, history of anxiety was also a factor for no-show [6], [24] or other issues or comorbidities like smoking [1] and substance abuse [24]. The weather on that day naturally affected the patient’s ability to attend [6], [4], [8], seasons like spring or summer were more likely to be missed than winter [1], [11]. While other predictors are shared across specialties (e.g., copays due, number of previous no-shows, number of previous appointments) others are also unique to the specialty (patient sex, appointment length) [24]. 2.1.2 Used Models This section covers used predictive models in the literature and compares their results, see Table 1. Most papers developed logistic regression models either alone or in combination with other models, to predict outpatient appointment absence. While others used multiple different types of models and compared their results. 11 Some papers used logistic regression as a modeling technique as its coefficient can be interpreted with ease and because the model could be implemented in an excel routine [23]. It is a commonly used algorithm for binary classification problems [9]. Regression models focused on the modality specific patient subsets demonstrated better predictive ability [6]. Another paper used regularized logistic regression that resembles logistic regression except it “shrinks” b-coefficients towards zero to produce a more stable risk estimate and avoid over-fitting [24]. One paper used to compare multiple different predictive modeling methodologies, like logistic regression, naïve bayes classifier, and artificial neural networks [1]. The latter was used as a complex machine learning algorithm that was needed to process the large number of features and observations such as multilayer perceptron [1]. As for categorical data, naïve bayes classifier is best used. Similarly, a paper compares between nine machine learning algorithms performances that has included; adaptive boosting (AdaBoost), logistic regression, naïve bayes, support vector machine, stochastic gradient descent, decision tree classifier, extra trees classifier (Extremely Randomized Trees), random forest classifier, and eXtreme gradient boosting (XGBoost). The most successful algorithm is defined as one with high recall [8]. It was observed that adaptive boosting has the most advantage between all algorithms. Again, a paper used multiple different models such as; logistic regression, random forest, k-nearest neighbors, support vector classifier, and stochastic gradient descent algorithms were implemented and evaluated. Most real-life datasets cannot be separated, which is where the support vector classifier comes in handy. The stochastic gradient descent speeds up the training of the support vector machine [9]. In this paper, 12 different combinations of methods were used as the best results were obtained by this method. For example, five different configurations of logistic regression yielded the best outcome according to the evaluation scheme. The random forest algorithm did best when used with FAMD with 10 components and SMOTEENN (Synthetic Minority Over Sampling technique by Edited Nearest Neighbor). The support vector classifier received the best score on five different settings, one of the balancing techniques and four of them used AE. Six various settings obtained the best results in the case of SGD (stochastic gradient descent), four of these configurations applied AE and the remaining two used no dimensionality reduction. Since the training of the k-nearest-neighbors algorithm when applied on all features was time consuming, it was only used with dimensionality reduction. The results show that according to the evaluation scheme, SGD and SVM had the highest execution in terms of G-mean, F1-score, and AUC ROC. Although logistic regression had the same G-mean and AUC ROC scores, it did not do as well in terms of F1-score by both SGD and SVM [9]. A different paper used Bayesian belief network (BBN); which is most commonly applied in medical research as it skillfully handles new information embedded in networks [26]. Another paper used JRip because it is characterized by compact size models, which helps understanding. Hoeffding trees are chosen because of their ability to manage very large datasets in a short duration of time and with less costs, along with their ability to adapt to the changes in concepts leading to no-shows [3]. In the learning stage of a paper, three algorithms were utilized in order to adapt and alter the models; logistic regression, random forest, and decision tree. When deciding on the best performing models, the hyper parameters of the algorithms were optimized by 10-fold cross- validation. The optimal model was evaluated in test data using the AUC [11]. 13 Used Model Result Logistic Regression [23], [6], [1], [24], [4], [25], [7], [8], [9], [11] Range between 0.60 and 0.95 Random Forest Classifier [4], [8], [9], [10], [11] Rang between 0.60 and 0.763 Naïve Bayes Classifier [1], [8] Rang between 0.71 and 0.86 Artificial Neural Network [1], [4], [10] Range between 0.77 and 0.774 Gradient Boosting Classifier [4] Adaptive Boosting (AdaBoost) Classifier [8] eXtreme Gradient Boosting (XGBoost) [8] Stochastic Gradient Descent Classifier [8], [9] Rang between 0.71 and 0.763 Stacking Classifier [4] 0.846 Bayesian Belief Network (BBN) [26], [5] Range between 0.691 and 0.828 Decision Tree Classifier [8], [11] Extra Trees Classifier (Extremely Randomized Trees) [8] Rang between 0.60 and 0.763 Support Vector Classifier [8], [9] Rang between 0.71 and 0.763 K-Nearest Neighbors [9] 0.713 Hoeffding Trees [3] 0.861 JRip Classifier [3] 0.776 Elastic Net (EN) [26] 0.691 Table 1. Demonstrates the used models with their results measured by AUC. 14 2.1.3 No-show Mitigation Strategies 2.1.3.1 Reminders Knowing the attributing factors or characteristics for no-show, helps predict the likelihood of missing the appointment, which can give time ahead to come up with solutions that may reduce that. An example for that is appointment schedulers contact patients ahead of time by calls or text messages or emails to remind them of their booked appointments, which supposedly should reduce the likelihood of no-show. This would actually help the schedulers to reduce no-show rates, by reminding the patient or by giving opportunity ahead of time to book other patients in the event that the call or email was not responded to [23], [9]. It was observed that this strategy has limited effect on reducing the no-show rate along with patient education [26]. This was contradicted in another study that has shown that contacting patients to remind them of their appointments reduced the no-show rates [5]. 2.1.3.2 Overbooking Another solution to compensate for no-show, is overbooking [6], [4]. This does reduce the number of schedule gaps or free time wasted until a patient shows up. Most hospitals consider the patients to have the same no-show probability (i.e., homogeneous) and overbook by a flat overbooking percentage [4], however some hospitals may consider heterogeneity, as in different factors affecting no-show. Paper [27] was the first to focus on the use of overbooking to minimize the negative consequence of no-shows for homogeneous patients. Paper [28] proposed a stochastic overbooking model and a myopic scheduling policy by dividing patients into groups depending on their no-show probability and scheduling patients sequentially as and when the call arrives. Paper [29] suggested an overbooking policy for a set of heterogeneous patients with the goal of maximizing expected profit and provided 15 managerial insights. Paper [30] suggested an overbooking policy by considering the likelihood of patient no-shows. On the other hand, few studies adjust the appointment interval using revised mean and standard deviation of consultation times to overcome or adjust for no-shows [4]. As discussed earlier, this solution is not ideal due to the unpredictability of no-show rates; where if it were overbooked due to low number of no-show, resources would be exhausted along with the staff and increase in the patient wait time. Whereas if no-show rates were higher than anticipated, resources would be wasted [2]. 2.1.3.3 Financial Penalties Financial penalties to those who missed appointments may help reduce that issue rate occurrence [6], [5]. Reservation fees at the time of booking have also been found to be an effective deterrent [9]. Disincentives to alter patient behavior, like moving no-show patients to the end of the waiting list or having to mandatory prepay a portion of the cost on booking the appointment [5]. 2.2 Background of Main Machine Learning Techniques Machine learning utilization gives the ability to handle and predict the pattern of data more efficiently. For example, we may sometimes not extract information and interpret the pattern from the data. In this case, we apply the machine learning techniques to extract the accurate data. The primary goal of ML is to gain an understanding from the datasets [31]. There are two types of learning; supervised learning and unsupervised learning. This section will focus on random forest, logistic regression, neural network, and gradient boosting. As the most commonly used machine learning techniques which produce the best results as indicated in Table 1. 16 2.2.1 Supervised Learning The supervised learning technique is the type of technique that needs external assistance. In this supervised learning, the input is mainly split into test and train datasets. The training dataset must be classified or predicted because it has the output variable Y. Every algorithm in machine learning applies that pattern to testing data for classification or prediction and may learn different patterns from the training dataset [31]. ▪ Regression In this regression technique, there is an identification of several functions to find out the correlation between variables. There are two variables, one is the independent variable and the other is the dependent variable. The independent variable should be continuous for better results. There can be only one dependent variable and multiple independent variables. Regression is used mostly to analyze the relationship between variables [32]. ▪ Classification The technique of classification is a systematic approach that is used to build the models of classification for testing and training datasets. There are different models of classification, such as naïve bayes, decision tree, SVM, etc. The classification divides the sample data into target classes. It mainly does predictions for every target class. For instance, based on the attendance, the patient can be categorized as show or no-show patients. It is mainly used to categorize class categories. There are different classification methods; the multi-level classification and binary classification. The binary classification can be categorized into 2 classes as ‘small’ and big while multilevel has targets, for instance, low, medium, and high [32]. 17 2.2.1.1 Random Forest Random forest is a principal class of classification that utilizes the classifiers of L tree- structured bases. It is categorized as dependent and identical random distributed vectors. Random forest makes a decision tree by randomly selecting data from the dataset. For instance, each decision tree is made by a random sampling of a training data subset or feature subset [33]. It is also used to build a strong classifier using an assembly of weaker ones, yielding high scores for problems [9]. The features in random forest are randomly selected in each split of the decision tree. This correlation between decision trees is mainly reduced by random selection of features. It also results in higher efficiency and results in prediction …