Artificial Intelligence-Based Facial Expression Recognition for Identifying Customer satisfaction on Products
Article Sidebar
Main Article Content
Abstract
Facial Expression Recognition (FER) for Identifying Customer Satisfaction on Products is one of the most powerful and challenging research tasks in social communication. Artificial intelligence (AI)-based emotion recognition harnesses the collective strength of machine learning, deep learning, and computer vision to decipher the subtleties of human emotions. By intricately analyzing facial expression, including the nuanced movements of the mouth, eyes, and eyebrows. Recent innovations have driven notable progress in face detection and recognition that enhance performance and reliability. This study focuses on leveraging AI-based facial expression recognition to identify customer satisfaction with products. The objective of this research is to develop a robust and accurate facial expression recognition system capable of analyzing customer emotions and determining their satisfaction levels based on their facial expressions. The proposed study used a hybrid convolutional neural network (CNN) and deep neural networks (DNN) model to extract meaningful features from facial images and classify them into different emotional states. The trained model is to be evaluated using a separate test dataset to measure its performance in accurately recognizing customer emotions and assessing satisfaction levels. The evaluation metrics include accuracy, precision, recall, and F1-score. The proposed experiment achieved excellent result with a real-time image-based dataset.
Manuscript received:7 Mar 2025 | Revised: 22 May 2025 | Accepted: 11 Jun 2025 | Published: 30 Jul 2025
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
References
R.K. Kanna, B.S. Panigrahi, S.K. Sahoo, A.R. Reddy, Y. Manchala and N.K. Swain, "CNN Based Face Emotion Recognition System for Healthcare Application," EAI Endorsed Transactions on Pervasive Health and Technology, vol. 10, 2024.
DOI: https://doi.org/10.4108/eetpht.10.5458
Z. Li and H. Jia, "The Application and Survey of Emotion Recognition Technology in Digital Guide System," ISPRS Journal of Photogrammetry and Remote Sensing, vol. 59, pp. 24-49, 2024.
DOI: https://doi.org/10.1117/12.3034930
G. Guanghui and Z. Xiaoping, "Multi-Modal Emotion Recognition by Fusing Correlation Features of Speech-Visual," IEEE Signal Processing Letters, vol. 28, no. 3, pp. 533-537, 2021.
DOI: https://doi.org/10.1109/LSP.2021.3055755
Y. Liu and G. Fu, "Emotion Recognition by Deeply Learned Multi-Channel Textual and EEG Features," Future Generation Computer Systems, vol. 119, pp. 1-6, 2021.
DOI: https://doi.org/10.1016/j.future.2021.01.010
H. Zhou, "The Application of Artificial Intelligence-Based Multimodal Emotion Analysis," Proceedings of the 2024 International Conference on Artificial Intelligence and Communication, pp. 1-6, 2024.
DOI: https://doi.org/10.2991/978-94-6463-512-6
D.N. Amali, A.R. Barakbah, A.R.A. Besari and D. Agata, "Semantic Video Recommendation System Based on Viewers’ Impression from Emotion Detection," International Journal on Robotics, Automation and Sciences, vol. 5, no. 2, pp. 176–183, 2022.
DOI: https://doi.org/10.1109/kcic.2018.8628592
R. Venkatesan, S. Shirly, M. Selvarathi and T.J. Jebaseeli, "Human Emotion Detection Using DeepFace and Artificial Intelligence," Engineering Proceedings, vol. 37, no. 1, 2023.
DOI: https://doi.org/10.3390/engproc2023059037
X. Lu, "Deep Learning-Based Emotion Recognition and Visualization of Figural Representation," Frontiers in Psychology, vol. 12, 2022.
DOI: https://doi.org/10.3389/fpsyg.2021.818833
Y. Narayan, M.A. Tripathi, P.P. Singh, D.A. Vidhate, R. Singh and M.M.S. Rao, "Development of AI Model to Analyze Customer Behavior by Decision Making System," IEEE Access, vol. 10, 2023.
DOI: https://doi.org/10.1109/iccams60113.2023.10525748
N. Rane, "Enhancing Customer Loyalty Through Artificial Intelligence (AI), Internet of Things (IoT), and Big Data Technologies," SSRN Electronic Journal, 2023.
DOI: https://doi.org/10.2139/ssrn.4616051
A. Dessai and H. Virani, "Multimodal and Multidomain Feature Fusion for Emotion Classification," Science, vol. 6, no. 1, p. 10, 2024.
DOI: https://doi.org/10.3390/sci6010010
K.A. Mamun, R.A. Nabid, S.I. Pranto, et al., "Smart Reception: An AI-Driven Bangla Language Receptionist System," Engineering Applications of Artificial Intelligence, vol. 136, 2024.
DOI: https://doi.org/10.1016/j.engappai.2024.108923
Y. Li, Z. Zhong, F. Zhang and X. Zhao, "AI-Based Human-Computer Interaction in Consumer Behavior Analysis," Frontiers in Psychology, vol. 13, 2022.
DOI: https://doi.org/10.3389/fpsyg.2022.784311
R. Venkatesan, S. Shirly, M. Selvarathi and T.J. Jebaseeli, "Human Emotion Detection Using DeepFace," Engineering Proceedings, vol. 37, 2023.
DOI: https://doi.org/10.3390/engproc2023059037
N. Mishra, A.S. Kumar, L. Jenefa, et al., "AI and ML in Business Applications," Cutting-Edge Technologies for Business Sectors, 2025.
DOI: https://doi.org/10.4018/979-8-3693-9586-8
W.T. Meshach, S. Hemajothi and E.M. Anita, "Real-Time Facial Expression Recognition Using SVM," Journal of Ambient Intelligence and Humanized Computing, vol. 12, no. 6, pp. 6355–6365, 2020.
DOI: https://doi.org/10.1007/s12652-020-02221-6
G. Morshed, H. Ujir and I. Hipiny, "Customer's Spontaneous Facial Expression Recognition," Indonesian Journal of Electrical Engineering and Computer Science, vol. 22, no. 3, pp. 1436–1445, 2021.
DOI: https://doi.org/10.11591/ijeecs.v22.i3.pp1436-1445
A.O.R. Vistorte, A. Deroncele-Acosta, J.L.M. Ayala, et al., "AI for Assessing Emotions in Learning Environments," Frontiers in Psychology, vol. 15, 2024.
DOI: https://doi.org/10.3389/fpsyg.2024.1387089
Y. Khaireddin and Z. Chen, "Facial Emotion Recognition on FER2013," arXiv, 2021.
DOI: https://doi.org/10.48550/arxiv.2105.03588
B. Wang and Y. Shi, "Expression Dynamic Capture and 3D Animation Generation Method Based on Deep Learning," Neural Computing and Applications, 2022.
DOI: https://doi.org/10.1007/s00521-022-07644-0
A. Chargui, N.E. Bourabah and K. Beggari, "Context-Based Emotion Recognition Using Deep Learning Model," University of Ouargla Institutional Repository, 2024.
DOI: https://doi.org/10.2991/978-94-6463-512-6
S.R. Chanthati, "Product Color Variation Management with AI," Authorea, 2024.
DOI: https://doi.org/10.22541/au.172114369.98060132/v1
A.T. Rosário and J.A. Casaca, "AI in Relationship Marketing Strategies," Advances in Marketing, pp. 1–34, 2024.
DOI: https://doi.org/10.4018/979-8-3693-5340-0.ch001
M.S. Bhuiyan, "AI-Enhanced Personalization in Customer Experiences," Journal of Computer Science and Technology Studies, vol. 6, no. 1, pp. 162–169, 2024.
DOI: https://doi.org/10.32996/jcsts.2024.6.1.17
A.S.A. Hans and S. Rao, "CNN-LSTM for Facial Emotion Detection in Videos," International Journal of Advances in Signal and Image Sciences, vol. 7, no. 1, pp. 11–20, 2021.
DOI: https://doi.org/10.29284/ijasis.7.1.2021.11-20
S. Singh, J.L. Gambill, M. Attalla, et al., "AI-Enabled Diagnostic Tools for Neuropsychiatric Disorders," Cureus, 2024.
DOI: https://doi.org/10.7759/cureus.71651
L.E. Arenas-Deseano, J.M. Ramirez-Cortes, J. Rangel-Magdaleno and I. Cruz-Vega, "Real-Time Emotion Classification Using CNN," IEEE Access, vol. 12, pp. 139988–139997, 2024.
DOI: https://doi.org/10.1109/ACCESS.2024.3450633
T. Kattenborn, J. Leitloff, F. Schiefer and S. Hinz, "Review on CNNs in Vegetation Remote Sensing," ISPRS Journal of Photogrammetry and Remote Sensing, vol. 173, pp. 24–49, 2021.
DOI: https://doi.org/10.1016/j.isprsjprs.2020.12.010
M.J. Ali, M.R.V. Vyshnavi, K.V.S.H. Kumar, et al., "AWS-Driven AI Chatbots for Restaurant Ordering," IEEE Conference Proceedings, pp. 1312–1318, 2024.
DOI: https://doi.org/10.1109/icses63445.2024.10762974
A.O.R. Vistorte, A. Deroncele-Acosta, J.L.M. Ayala, et al., "Integrating AI to Assess Emotions in Learning Environments," Frontiers in Psychology, vol. 15, 2024.
DOI: https://doi.org/10.3389/fpsyg.2024.1387089
G.Z. Karimova, "Personality-Grounded Framework for AI-Based Product Design," International Journal of Human-Computer Interaction, vol. 40, no. 7, pp. 1689–1701, 2022.
DOI: https://doi.org/10.1080/10447318.2022.2150744
R. Jamali, A. Generosi, J.Y. Villafan, et al., "Facial Expression Recognition for Juror Attention Measurement," Sensors, vol. 24, no. 7, p. 2298, 2024.
DOI: https://doi.org/10.3390/s24072298
J. Miao and W. Zhu, "Precision-Recall Curve Classification Trees," Evolutionary Intelligence, vol. 15, no. 3, pp. 1545–1569, 2021.
DOI: https://doi.org/10.1007/s12065-021-00565-2
N.G.Z. Khan and N.I. Ullah, "Monkeypox Classification Using Pre-Trained Models," Journal of Innovative Image Processing, vol. 5, no. 2, pp. 192–213, 2023.
DOI: https://doi.org/10.36548/jiip.2023.2.009
G.Z. Khan, I.A. Shah, N. Farhatullah, et al., "Deep Learning Model for Lung Cancer Diagnosis," IEEE Access, vol. 10, 2023.
DOI: https://doi.org/10.1109/icomet57998.2023.10099357
N. Ahmad, S. Hao, T. Liu, et al., "Data-Driven Control for Nonlinear Batch Processes," ISA Transactions, vol. 146, pp. 308–318, 2024.
DOI: https://doi.org/10.1016/j.isatra.2023.12.044
G.Z. Khan, I.A. Shah, N. Farhatullah, et al., "Malaria Detection Using Transfer Learning," IEEE Conference Proceedings, vol. 10, 2023.
DOI: https://doi.org/10.1109/icomet57998.2023.10099260
R. Zaman, I.A. Shah, N. Ullah and G.Z. Khan, "Breast Cancer Detection from Histopathological Images," Applied Sciences, vol. 40, pp. 313–325, 2023.
DOI: https://doi.org/10.3390/asec2023-16598
N. Ahmad, S. Hao and T. Liu, "ESO-Based Iterative Learning Control," Asian Control Conference Proceedings, pp. 1–8, 2022.
DOI: https://doi.org/10.23919/ascc56756.2022.9828355
G.Z. Khan, N.I. Ulhaq, N.I. Adil, et al., "Privacy-Preserving Customer Churn Prediction," VFAST Transactions on Software Engineering, vol. 11, no. 3, pp. 73–80, 2023.
DOI: https://doi.org/10.21015/vtse.v11i3.1642
N. Ahmad, S. Hao, T. Liu, et al., "Data-Driven Set-Point Learning Control," IEEE DDCLS Proceedings, pp. 2082–2088, 2024.
DOI: https://doi.org/10.1109/ddcls61622.2024.10606659
S.H. Badshah, N. Farhatullah, G.Z. Khan, et al., "Brain Tumor Classification Using ML Algorithms," Lecture Notes on Data Engineering, pp. 757–775, 2023.
DOI: https://doi.org/10.1007/978-3-031-24475-9_62
K.C. Chin, G.C. Lee, H.R. Yon, et al., "Development of an IoT-based Wireless Controlled Power Adapter," International Journal on Robotics, Automation and Sciences, vol. 7, no. 1, pp. 7–18, 2025.
DOI: https://doi.org/10.33093/ijoras.2025.7.1.2
K.B. Gan, Y.E. Teoh, "An Edge Convolution Neural Network Model for Plant Health Classification Using Camera," International Journal on Robotics, Automation and Sciences, vol. 7, no. 1, pp. 1–6, 2025.