Fatigue Detection Using Raspberry Pi 3

Driver drowsiness is a primary cause of several highway calamities leads to severe physical injuries, loss of money, and loss of human life. The implementation of driver drowsiness detection in real-time will aid in avoiding major accidents. The system is designed for four-wheelers wherein the driver’s fatigue or drowsiness is detected and alerts the person. The proposed method will use 5-megapixel Raspbian camera that captures driver’s face and eyes and processes the images to detect driver’s fatigue. On the detection of drowsiness, the programmed system cautions the driver through an alarm to ensure vigilance. The proposed method constitutes of various stages to determine wakefulness of the driver. According to this output, the warning message is generated. Haar Cascade Classifiers is used to detect the blink duration of the driver and Eye Aspect Ratio (EAR) is calculated. Finally, the alert message along with car plate number is sent to the concerned person mobile with help of Ubidots cloud service and Twilio API. For this Raspberry Pi 3 with Raspbian (Linux Based) Operating System is used.


Introduction
In recent years, driving has become an important part of our dayto-day life, especially in urban areas sleepiness-related accidents are occurring infrequent [1]. Road accidents are apparently a global hazard in our country. Based on the survey of the National Crime Records Bureau (NCRB) about 1, 35, 000 traffic-related demise happen every year in India. These factors lead to the development of Intelligent Transportation System (ITS). If the accident caused by abnormalities of the driver, it can be prevented by placing abnormality detecting system within the vehicle [2]. Tiredness can be caused by several phenomena like psychosocial factors, health factors, and physiological factors [3]. Based on the survey of National Highway Traffic Safety Administration of the United States of America (USA), police studied around 1, 00, 000 crashes occurred due to the drowsiness of the driver, it causes major losses like 71,000 wounds, $12.5 billion budgetary mishap and 1,550 passings [4]. The issues can be overcome by implementing several methods which include Support Vector Machines (SVM), fuzzy-based system, neuro-fuzzy approach and Artificial Neural Networks (ANN) [5] for detecting the drowsiness. It is difficult to correctly say about an exact number of sleep-related accidents, but traffic research shows up to 20% of accidents happen due to fatigue of the driver [6]. Drowsiness detection can be carried out by two techniques. The first technique is intrusive and second is nonintrusive. The intrusive technique involves computation of mind wave monitoring, heart-beat rate etc. Non-invasive techniques are appropriate to find facial appearance for tiredness detection. Mouth gaping and Eye closure are the wellknown symptoms of the drowsiness detection [9]. The nonintrusive technique involves head pose, eye blinking rate, yawn detection, eye closure, etc., [8]. Another non-invasive way to detect fatigue can be divided into three scenarios: visual cues, physiological measurements, driving performance. Physiological and visual cues involve direct computation, whereas driving performance involves indirect computation [10]. It is suitable for the real-time application, because of no need for sensing electrodes. Automatic eye detection in video sequence involves several methods like motion estimation in the eye. The eye region is evaluated by sparse tracking, adaptive thresholding or by frame-toframe intensity variation and from optical flow [16]. The driver fatigue is detected using computer vision [6]. The driver fatigue is determined by the frequency of eye blinking and head tilting and the drowsiness level is calculated by Raspberry Pi3 along with a Pi camera [8]. The experiment is carried out on ten volunteers under different lighting conditions and they obtain face and eye detection rate up to 99.59% [7]. Real-time video-based vision processing method is recommended to estimate fatigue of the driver and a buzzer is used to alert the driver [11]. Relative Areal Ratio (RAR) is used to identify facial landmarks and to calculate six landmark points between each eye [12]. Driver fatigue is monitored by serially interfacing Raspberry Pi board loaded with raspbian OS to Arduino Uno, and 8-megapixels digital USB camera [13].

Methodology
The main aim is to detect drowsiness of driver, it can be done in different ways like detecting facial expression of the driver and measuring Eye Aspect Ratio (EAR). Blinking pattern is different for each and every individual. The pattern gets varied in terms of squeezing degree of eye, blink duration and speed of closing and opening the eye [16]. The proposed method involved with the following methodologies such as Haar Cascade Classifiers, Shape Predictor_68_facial landmark detection, Eye Aspect Ratio (EAR), Ubidots cloud service and Twilio API.

Haar Cascade Classifiers
In Haar Cascade Classifiers, a lot of similar and dissimilar images are trained in order to detect fatigue of the driver. OpenCV is a learning-based method, packed with a detector as well as a trainer. For training, a separate database is maintained for face and eye with several positive and negative images having eye closed and opened conditions and different set facial images [15]. In 2013, Patil et al suggested a vision-based drowsiness is carried out using Support Vector Machine and Haar Cascade Classifiers [14].

Shape predictor_68_Facial Landmark detection and Eye Aspect Ratio (EAR)
In order to predict the face and eye region in the live video stream, shape predictor is used. Fig.1 shows the sleepiness which is measured by calculating the eye aspect ratio (Euclidean distance between the eyes are calculated), the arguments are passed to the predefined dataset and facial landmark detection is carried out. For every video sequence, the eye landmarks are located. The aspect ratio between width and height of the eye is calibrated.

EAR =
Where p1,…,p6 are the two-dimensional landmark location, represented in Fig.2. The EAR is mostly stable when an eye is open and is getting close to zero while the eye is not in open state. If the person viewing the camera continuously, the Eye Aspect Ratio (EAR) is found to be normal and it reaches low value when he/she closing the eye for a longer time. When the lower value is reached, then drowsiness is detected. In 2012, Ubidots elaborated on connected software and hardware solutions to remotely control, automate processes for healthcare clients and monitor. Twilio is a cloud communication platform as a service (PaaS), it allows a software developer to programmatically send and receive text message using its web service APIs, make and receive phone calls. The paper introduces alerting process when the driver feels sleepy using cloud server and mobile API to send a message and at the same time providing an alarm signal to the driver.

Programming Algorithm
Raspberry Pi 3 is effective with Python IDLE. It involves programming in software with extensions of OpenCV computer vision installed. The entire algorithm for drowsiness detection is shown with the help of a flowchart shown in Fig 2. The program will start to run and it can be terminated using command line interfaces in Raspberry Pi 3 or the system is turned off. In order to initiate the program execution, it will import the following libraries like numpy, OpenCV, play sound, argparse, dlib, distance, timer, client, ApiClient, and picamera

Result and Discussion
The driver drowsiness can be measured using Eye Aspect Ratio(EAR). The ratio of the eye can vary for each and every person. Fig.4. The following case is tested for ten different set of people with two conditions. One is calculated for eye-opening condition and another one for eye closing condition. Fig.3. Eye closing rate is measured after every 0.5 seconds and if the value crosses already existed threshold value, then the raspberry pi 3 receives the alert signal from alarm connected to the GPIO pins of Pi 3 board. Fig 5.When the person closing his eyes for more than fixed threshold range then the alert signal is generated to wake up the driver from sleepy state and also through the cloud service the alert message is sent to the owner of the car along with the car plate number.

Conclusion
Driver drowsiness detection is designed mainly to keep the driver awake while driving to avoid the accident due to sleepiness. The alert signal is generated from embedded device to awake driver from sleepy state. The Pi along with Raspbian camera is used to calculate the drowsiness of the driver in real time. Fatigue is measured by detecting Eye and face using Haar Cascade Classifier, especially facial landmarks is detected using shape-predictor and Eye Aspect Ratio (EAR) by calculating the Euclidean distance between the eyes. Accurate eye detection and faces in every frame will help to calculate drowsiness level. Frequent detection of eye blinking and head tilting is measured properly and it helps to indicates drowsiness. When he/she reaches maximum threshold the driver will be alarmed by a loud warning that will wake up the driver from the sleep state. In future, the implementation can be carried out in a bright room with consistent light, for different lighting conditions and also for the persons with dark skin can be considered.