 |
The proposed multi-view method, MV-FER, is depicted in Figure 5.
MV-FER utilizes three complementary views: original facial images and horizontally flipped augmented images as dual input branches, while AU-guided heatmaps—derived from facial landmarks and FACS theory—to guide attention toward expression-critical regions including the eyes, brows, and mouth areas. The framework employs a unified training paradigm ensuring consistent representation learning across both input views while aligning learned attention with expression-relevant facial regions. This integration of multi-view learning and attention alignment enables our proposed method, named MV-FER, to achieve enhanced facial expression recognition performance.
During training, the pair (x_1, x_2) is fed into a shared backbone network and Classifier head to produce class predictions and feature representations, while x_3 is used for attention alignment.
During the validation and inference phases, only the original view of each image in the test set is employed. Each image is fed into the trained backbone network to generate class logits, which are subsequently passed through a softmax layer to obtain a probability distribution over the seven expression categories: surprise, fear, disgust, happiness, sadness, anger, and neutral. The final prediction is determined by selecting the class with the highest probability.
|
IEEE/ICACT20260151 Slide.09
[Big Slide]