Emerged human-like facial expression representation in a deep convolutional neural network

Abstract

Face identity and expression play critical roles in social communication. Recent research found that the deep convolutional neural networks (DCNNs) trained to recognize facial identities spontaneously learn features that support facial expression recognition, and vice versa, suggesting an integrated representation of facial identity and expression. In the present study, we found that the expression-selective units spontaneously emerged in a VGG-Face trained for facial identity recognition and tuned to distinct basic expressions. Importantly, they exhibited typical hallmarks of human expression perception, i.e., the facial expression confusion effect and categorical perception effect. We then investigated whether the emergence of expression-selective units is attributed to either face-specific experience or domain-general processing, by carrying out the same analysis on a VGG-16 trained for object classification and an untrained VGG-Face without any visual experience, both of them having the identical architecture with the pretrained VGG-Face. Although Similar expression-selective units were found in both DCNNs, they did not exhibit reliable human-like characteristics of facial expression perception. Taken together, our computational findings revealed the necessity of domain-specific visual experience of face identity for the development of facial expression perception, highlighting the contribution of nurture to form human-like facial expression perception. Beyond the weak equivalence between human and DCNNS at the input-output behavior, emerging simulated algorithms between models and humans could be established through domain-specific experience.