Facial Expression for emotion recognition using Infrared Imaging

Abstract

Facial expression recognition (FER) plays a crucial role in human-computer interaction, with applications ranging from healthcare to security systems. This study proposes a robust FER system leveraging thermal imaging for emotion recognition. Utilizing the Kaggle dataset "Chia theo nguoi_KTFEV2-7 emotions," our approach focuses exclusively on thermal facial images, comprising 2,538 labeled samples spanning seven emotional categories: happiness, sadness, anger, fear, surprise, disgust, and neutrality. The VGG-16 convolutional neural network architecture, combined with a Support Vector Machine (SVM) classifier, is employed to identify emotions. The model's performance is evaluated using accuracy, precision, recall, and F1-score. Experimental results demonstrate the system's effectiveness, with classification accuracies of up to 94% for neutral expressions and 93% for fear. Despite challenges in detecting surprise (52% accuracy), the approach proves promising for FER applications in thermal imaging, highlighting its potential for emotion analysis in non-ideal visual conditions.

Introduction: Facial expression recognition (FER) has become an integral aspect of human-computer interaction, with diverse applicatins ranging from healthcare and security systems to virtual reality and emotional intelligence analysis. By analyzing facial expressions, machines can interpret human emotions, enhancing their ability to respond appropriately in real-time scenarios.

Objectives: propose a robust FER system utilizing thermal images to classify facial expressions into seven distinct emotions: happiness, sadness, anger, fear, surprise, disgust, and neutrality.

Methods: The proposed model  (as shown in figure 1) for facial expression recognition utilizes the VGG-16 architecture, a widely recognized convolutional neural network (CNN) designed for image classification tasks. VGG-16 is known for its simplicity, depth, and effectiveness in extracting hierarchical image features. Below, the architecture's components and their contributions are detailed

Results: The proposed model is trained with data as shown in table no. 1 and tested with 200 images. Figure 2 shows feature map generation for anger and happy emotion.

Conclusions: The model achieved an accuracy of 90.52%, which indicates its capability to correctly classify emotions. The precision of 95.59% which indicates the model's ability to identify true positive instances of emotion without being misled by false positives. And a recall of 93.44% says the effectiveness in capturing true positive emotions.