Multimodal Human Eye Blink Recognition Using Z-score Based Thresholding and Weighted Features.
DOI:
https://doi.org/10.9781/ijimai.2021.11.002Keywords:
Eye Blink, Multimodal, Z Score Threshold, Weighted FeaturesAbstract
A novel real-time multimodal eye blink detection method using an amalgam of five unique weighted features extracted from the circle boundary formed from the eye landmarks is proposed. The five features, namely (Vertical Head Positioning, Orientation Factor, Proportional Ratio, Area of Intersection, and Upper Eyelid Radius), provide imperative gen (z score threshold) accurately predicting the eye status and thus the blinking status. An accurate and precise algorithm employing the five weighted features is proposed to predict eye status (open/close). One state-of-the-art dataset ZJU (eye-blink), is used to measure the performance of the method. Precision, recall, F1-score, and ROC curve measure the proposed method performance qualitatively and quantitatively. Increased accuracy (of around 97.2%) and precision (97.4%) are obtained compared to other existing unimodal approaches. The efficiency of the proposed method is shown to outperform the state-of-the-art methods.
Downloads
References
T. Soukupova, “Eye Blink Detection using Facial Landmarks,” Diploma Thesis, Department of Cybernetics, Faculty of Electrical Engineering, Czech Technical University, Prague, 2016.
Puneet Singh Lamba and Deepali Virmani, “Information Retrieval from Emotions and Eye Blinks with Help of Sensor Nodes,” International Journal of Electrical and Computer Engineering (IJECE), vol. 8, no.4, pp. 2433-2411, 2018.
Puneet Singh Lamba and Deepali Virmani, “Reckoning number of eye blinks using eye facet correlation for exigency detection,” Journal of Intelligent & Fuzzy Systems, vol. 35, no. 5, pp. 5279-5286, 2018.
Stephen Milborrow and Fred Nicolls, “Locating Facial Features with an Extended Active Shape Model,” Computer Vision – ECCV 2008, Lecture Notes in Computer Science, Springer, 5305 (2008), pp. 504-513.
T. Drutarovsky and A. Fogelton, “Eye blink detection using variance of motion vectors,” In: Computer Vision - ECCV Workshops, 2014.
F. Song, X. Tan, X. Liu and S. Chen, “Eyes closeness detection from still images with multi-scale histograms of principal oriented gradients,” Pattern Recognition, The Journal of the Pattern Recognition Society, vol. 47, pp. 2825-2838, 2014.
T. Soukupova and J. Cech, “Real-Time Eye Blink Detection using Facial Landmarks,” 21st Computer vision Winter Workshop, Luka Cehovin, RokMandeljc, VitomirStruc (eds.) RimskeToplice, Slovenia, February 3-5, 2016.
W.H. Lee, E.C. Lee and K.E. Park, “Blink detection robust to various facial poses,” Journal of Neuroscience Methods, vol. 193, no. 2, pp. 356-372, 2010.
N. Alsaeedi, and D. Wloka, “Real-Time Eyeblink Detector and Eye State Classifier for Virtual Reality (VR) Headsets (Head-Mounted Displays, HMDs),” Sensors, vol. 19, no. 5, 2019.
R. Sanyal and K. Chakrabarty, “Two Stream Deep Convolutional Neural Network for Eye State Recognition and Blink Detection,” 2019 3rd International Conference on Electronics, Materials Engineering & NanoTechnology (IEMENTech), Kolkata, India, 2019, pp. 1-8.
C. Zhu and C. Huang, “Adaptive Gabor algorithm for face posture and its application in blink detection,” 2019 IEEE 3rd Advanced Information Management, Communicates, Electronic and Automation Control Conference (IMCEC), Chongqing, China, 2019, pp. 1871-1875.
Muhammad Tayab Khan, Hafeez Anwar, Farman Ullah, et al., “Smart Real-Time Video Surveillance Platform for Drowsiness Detection Based on Eyelid Closure,” Wireless Communications and Mobile Computing, 2019.
A. Asthana, S. Zafeoriou, S. Cheng, and M. Pantic, “Incremental face alignment in the wild,” in Conference on Computer Vision and Pattern Recognition, 2014.
Gianluca Donato, Marian Stewart Bartlett, Joseph C. Hager, Paul Ekman, and Terrence J. Sejnowski, “Classifying Facial Actions,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 21, no. 10, pp.974-989, 1999.
Ying-li, Takeo Kanade, and Jeffrey, “Recognizing Action Units for Facial Expression Analysis,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 23, no. 2, pp. 1-19, 2001.
Choi, I., Han, S., & Kim, D. (2011, July), “Eye detection and eye blink detection using adaboost learning and grouping,” In 2011 Proceedings of 20th International Conference on Computer Communications and Networks (ICCCN), 2011, pp. 1-4. IEEE.
Singh, H., & Singh, J., “Real-time eye blink and wink detection for object selection in HCI systems,” Journal on Multimodal User Interfaces, vol. 12, no. 1, pp. 55-65, 2018.
G. Pan, L. Sun, Z. Wu, and S. Lao, “Eyeblink-based anti-spoofing in face recognition from a generic webcamera,”, In ICCV, 2007.
L. Tan, K. Yu, F. Ming, X. Cheng & G. Srivastava, “Secure and Resilient Artificial Intelligence of Things: a HoneyNet Approach for Threat Detection and Situational Awareness,” IEEE Consumer Electronics Magazine, 2021.
F. M. Sukno, S.-K. Pavani, C. Butakoff and A.F. Frangi, “Automatic assessment of eye blinking patterns through statistical shape models,” In ICVS, 2009.
K. Arai, R. Mardiyanto, “Real Time Blinking Detection Based on Gabor Filter,” International Journal of Human Computer Interaction, vol. 1, no. 3, pp. 33-45, 2010.
Lamba, P.S., Virmani, D. & Castillo, O., “Multimodal human eye blink recognition method using feature level fusion for exigency detection,” Soft Comput vol. 24, pp. 16829–16845, 2020, https://doi.org/10.1007/s00500-020-04979-5
Usakli, A. B., Gurkan, S., Aloise, F., Vecchiato, G., & Babiloni, F., “On the use of electrooculogram for efficient human computer interfaces,” Computational intelligence and neuroscience, 2010.
Pander, T., Przybyla, T., & Czabanski, R. (2008, May), “An application of detection function for the eye blinking detection,” In 2008 Conference on Human System Interactions, 2008, pp. 287-291. IEEE.
Królak, A., & Strumiłło, P., “Eyeblink detection system for human–computer interaction,” Universal Access in the Information Society, vol. 11, no. 4, pp. 409-419, 2012.
Kraichan, C., & Pumrin, S. (2014, May), “Face and eye tracking for controlling computer functions,” In 2014 11th International Conference on Electrical Engineering/Electronics, Computer, Telecommunications and Information Technology (ECTI-CON), 2014, pp. 1-6. IEEE.
MacKenzie, I. S., & Ashtiani, B., “BlinkWrite: efficient text entry using eye blinks,” Universal Access in the Information Society, vol. 10, no. 1, pp. 69-80, 2011.
Arai, K., & Mardiyanto, R. (2011, April) “Eye-based HCI with full specification of mouse and keyboard using pupil knowledge in the gaze estimation,” In 2011 Eighth International Conference on Information Technology: New Generations, 2011, pp. 423-428. IEEE.
Khilari, R. (2010, December),” Iris tracking and blink detection for humancomputer interaction using a low resolution webcam,” In Proceedings of the Seventh Indian Conference on Computer Vision, Graphics and Image Processing, 2010, pp. 456-463.
Grauman, K., Betke, M., Lombardi, J., Gips, J., & Bradski, G. R., “Communication via eye blinks and eyebrow raises: Video-based human-computer interfaces,” Universal Access in the Information Society, vol. 2, no. 4, pp. 359-373, 2003.
Siriluck W, Kamolphiwong S, Kamolphiwong T, “Blink and click,” In: Proceedings of the 1st international convention on rehabilitation engineering and assistive technology: in conjunction with 1st Tan Tock Seng Hospital neurorehabilitation meeting, 2007, pp. 43–46.
Pimplaskar, D., Nagmode, M. S., & Borkar, A., “Real time eye blinking detection and tracking using OpenCV,” Technology, vol. 13, no. 14, pp. 15, 2015.
Missimer, E., & Betke, M. (2010, June), “Blink and wink detection for mouse pointer control,” In Proceedings of the 3rd International Conference on PErvasive Technologies Related to Assistive Environments, 2010, pp. 1-8.
Su, M., Yeh, C., Lin, S., Wang, P., & Hou, S. (2008, July), “An implementation of an eye-blink-based communication aid for people with severe disabilities,” In 2008 International Conference on Audio, Language and Image Processing, 2008, pp. 351-356. IEEE.
Venkataramanan, S., Prabhat, P., Choudhury, S. R., Nemade, H. B., & Sahambi, J. S. (2005, January),” Biomedical instrumentation based on electrooculogram (EOG) signal processing and application to a hospital alarm system,” In Proceedings of 2005 International Conference on Intelligent Sensing and Information Processing, 2005, pp. 535-540. IEEE.
Kumar, D., & Poole, E. (2002, October), “Classification of EOG for human computer interface,” In Proceedings of the Second Joint 24th Annual Conference and the Annual Fall Meeting of the Biomedical Engineering Society] [Engineering in Medicine and Biology, Vol. 1, pp. 64-67. IEEE.
Al-gawwam, S., & Benaissa, M., “Robust eye blink detection based on eye landmarks and Savitzky–Golay filtering,” Information, vol. 9, no. 4, pp. 93, 2018.
Han, Y. J., Kim, W., & Park, J. S., “Efficient eye-blinking detection on smartphones: A hybrid approach based on deep learning,” Mobile Information Systems, 2018.
M. Chau and M. Betke, “Real time eye tracking and blink detection with USB cameras,” Technical Report 2005-12, Boston University Computer Science, May 2005.
D. Torricelli, M. Goffredo, S. Conforto and M. Schmid, “An adaptive blink detector to initialize and update a view- based remote eye gaze tracking system in a natural scenario,” Pattern Recogn Lett, vol. 30, no. 12, pp. 1144-1150, 2009.
Trobo, I. P., Díaz, V. G., Espada, J. P., Crespo, R. G., & Moreno-Ger, P., “Rapid modeling of human-defined AI behavior patterns in games,” Journal of Ambient Intelligence and Humanized Computing, vol. 10, no. 7, pp. 2683-2692, 2019.
Khari, M., Kumar, P., Burgos, D., & Crespo, R. G., “Optimized test suites for automated testing using different optimization techniques,” Soft Computing, vol. 22, no. 24, pp. 8341-8352, 2018.
García-Díaz, V., Espada, J. P., Crespo, R. G., G-Bustelo, B. C. P., & Lovelle, J. M. C., “An approach to improve the accuracy of probabilistic classifiers for decision support systems in sentiment analysis,” Applied Soft Computing, vol. 67, pp. 822-833, 2018.
Zhang, K., Zhang, Z., Li, Z., & Qiao, Y., “Joint face detection and alignment using multitask cascaded convolutional networks,” IEEE Signal Processing Letters, vol. 23, no. 10, pp. 1499-1503, 2016.
Harish, B. S., Maheshan, M. S., & Nagadarshan, N., “A Convolution Neural Network Engine for Sclera Recognition,” International Journal of Interactive Multimedia and Artificial Intelligence, vol. 6, no. 1, pp. 78-83, 2020. http://doi.org/10.9781/ijimai.2019.03.006
Verma, K. K., Singh, B. M., Mandoria, H. L., & Chauhan, P., “Two-Stage Human Activity Recognition Using 2D-ConvNet,” International Journal of Interactive Multimedia and Artificial Intelligence, vol. 6, no. 2, 2020. http://doi.org/10.9781/ijimai.2020.04.002
Downloads
Published
-
Abstract211
-
PDF40
Submit Article
Information
