Face and Facial Expressions Recognition System for Blind People Using ResNet50 Architecture and CNN
Article Sidebar
Main Article Content
Abstract
Many blind individuals have difficulties in recognizing people’s facial expression which may impact their social interaction. With the recognition, the blind individuals can accurately interpret and respond to the emotions. There is a lack in the existing application with the combination of face and facial expressions recognition. The blind individuals have to rely on multiple applications to accomplish the same task, making it difficult and time-consuming for them to use. The paper aims to recognize faces and facial expressions for blind individuals and provides feedback in real-time. Three face detection algorithms of Haar Cascade Classifier, Dlib, and RetinaFace are compared. Dlib is chosen to process with Histogram of Oriented Gradients (HOG) and Support Vector Machine (SVM). It loads the pre-trained model, computes the HOG features, slide the window scanning at different scales, classify the windows using the SVM classifier, generate bounding boxes, and applying non-maximum suppression. ResNet50 architecture is employed to recognize face and Convolutional Neural Networks (CNN) is applied to recognize facial expression. The training accuracy is 70% and validation accuracy is 60%.
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
All articles published in JIWE are licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0) License. Readers are allowed to
- Share — copy and redistribute the material in any medium or format under the following conditions:
- Attribution — You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use;
- NonCommercial — You may not use the material for commercial purposes;
- NoDerivatives — If you remix, transform, or build upon the material, you may not distribute the modified material.
References
Islam, B., Mahmud, F., & Hossain, A. (2018, December). High performance facial expression recognition system using facial region segmentation, fusion of HOG & LBP features and multiclass SVM. In 2018 10th International Conference on Electrical and Computer Engineering (ICECE) (pp. 42-45). IEEE.
Joseph, J. L., & Mathew, S. P. (2021, September). Facial expression recognition for the blind using deep learning. In 2021 IEEE 4th international conference on computing, power and communication technologies (GUCON) (pp. 1-5). IEEE.
Lin, S. Y., Tseng, Y. W., Wu, C. R., Kung, Y. C., Chen, Y. Z., & Wu, C. M. (2019, December). A continuous facial expression recognition model based on deep learning method. In 2019 International Symposium on Intelligent Signal Processing and Communication Systems (ISPACS) (pp. 1-2). IEEE.
Liu, D., Zhang, H., & Zhou, P. (2021, January). Video-based facial expression recognition using graph convolutional networks. In 2020 25th International Conference on Pattern Recognition (ICPR) (pp. 607-614). IEEE.
Supta, S. R., Sahriar, M. R., Rashed, M. G., Das, D., & Yasmin, R. (2020, December). An Effective Facial Expression Recognition System. In 2020 IEEE International Women in Engineering (WIE) Conference on Electrical and Computer Engineering (WIECON-ECE) (pp. 66-69). IEEE.
Xu, X., Ruan, Z., & Yang, L. (2020, July). Facial expression recognition based on graph neural network. In 2020 IEEE 5th International Conference on Image, Vision and Computing (ICIVC) (pp. 211-214). IEEE.
Ullah, S., Jan, A., & Khan, G. M. (2021, October). Facial expression recognition using machine learning techniques. In 2021 International Conference on Engineering and Emerging Technologies (ICEET) (pp. 1-6). IEEE.
Zou, J., Cao, X., Zhang, S., & Ge, B. (2019, April). A facial expression recognition based on improved convolutional neural network. In 2019 IEEE International Conference of Intelligent Applied Systems on Engineering (ICIASE) (pp. 301-304). IEEE.
Wang, D., Yu, H., Wang, D., & Li, G. (2020, April). Face recognition system based on CNN. In 2020 International Conference on Computer Information and Big Data Applications (CIBDA) (pp. 470-473). IEEE.
Chinimilli, B. T., Anjali, T., Kotturi, A., Kaipu, V. R., & Mandapati, J. V. (2020, June). Face recognition based attendance system using Haar cascade and local binary pattern histogram algorithm. In 2020 4th international conference on trends in electronics and informatics (ICOEI)(48184) (pp. 701-704). IEEE.
Fu, L., & Shao, X. (2020, January). Reseach and implementation of face detection, tracking and recognition based on video. In 2020 International Conference on Intelligent Transportation, Big Data & Smart City (ICITBS) (pp. 914-917). IEEE.
Sarwar, M. G., Dey, A., & Das, A. (2021, April). Developing a LBPH-based face recognition system for visually impaired people. In 2021 1st International Conference on Artificial Intelligence and Data Analytics (CAIDA) (pp. 286-289). IEEE.
Naik, A. U., & Guinde, N. (2020, August). LBPH algorithm for frontal and side profile face recognition on GPU. In 2020 Third International Conference on Smart Systems and Inventive Technology (ICSSIT) (pp. 776-779). IEEE.
Flores, R. R., & Dominguez, M. J. (2021, December). Real time automatic face recognition system using LBPH technique. In 2021 IEEE CHILEAN Conference on Electrical, Electronics Engineering, Information and Communication Technologies (CHILECON) (pp. 1-6). IEEE.
Wang, X., Liu, L., & Yang, S. (2021, December). An Improved Face Recognition Algorithm based on Fisherface and Machine Learning. In ICMLCA 2021; 2nd International Conference on Machine Learning and Computer Application (pp. 1-4). VDE.
Rosnelly, R., Simanjuntak, M. S., Sitepu, A. C., Azhari, M., & Kosasi, S. (2020, October). Face recognition using eigenface algorithm on laptop camera. In 2020 8th International Conference on Cyber and IT Service Management (CITSM) (pp. 1-4). IEEE.
Mohite, P., Vaibhav, K., & Annapurani, P. K. (2022, May). Thermal and Visual Face Recognition using Eigenfaces and Transfer Learning. In 2022 International Conference on Applied Artificial Intelligence and Computing (ICAAIC) (pp. 656-662). IEEE.
Winarno, E., Al Amin, I. H., Februariyanti, H., Adi, P. W., Hadikurniawati, W., & Anwar, M. T. (2019, December). Attendance system based on face recognition system using cnn-pca method and real-time camera. In 2019 International Seminar on Research of Information Technology and Intelligent Systems (ISRITI) (pp. 301-304). IEEE.;.;
Lim, Y., Ng, K. W., Naveen, P., & Haw, S. C. (2022). Emotion Recognition by Facial Expression and Voice: Review and Analysis. Journal of Informatics and Web Engineering, (pp. 45-54). https://doi.org/10.33093/jiwe.2022.1.2.4
Xin, M., Ang, L. W., & Palaniappan, S. (2023). A Data Augmented Method for Plant Disease Leaf Image Recognition based on Enhanced GAN Model Network. Journal of Informatics and Web Engineering, (pp. 1-12). https://doi.org/10.33093/jiwe.2023.2.1.1