Integrating face and voice in person perception

doi:10.1007/978-1-4614-3585-3

Paper

Integrating face and voice in person perception

Published Dec 31, 2007 · S. Campanella, P. Belin

Trends in Cognitive Sciences

Q1 SJR score

452

Citations

21

Influential Citations

Full text Semantic Scholar

Abstract

Integration of information from face and voice plays a central role in our social interactions. It has been mostly studied in the context of audiovisual speech perception: integration of affective or identity information has received comparatively little scientific attention. Here, we review behavioural and neuroimaging studies of face-voice integration in the context of person perception. Clear evidence for interference between facial and vocal information has been observed during affect recognition or identity processing. Integration effects on cerebral activity are apparent both at the level of heteromodal cortical regions of convergence, particularly bilateral posterior superior temporal sulcus (pSTS), and at 'unimodal' levels of sensory processing. Whether the latter reflects feedback mechanisms or direct crosstalk between auditory and visual cortices is as yet unclear.

Highly Cited

Study Snapshot

Key takeawayFace-voice integration plays a crucial role in person perception, with interference observed during affect recognition and identity processing, and integration effects on cerebral activity at both heteromodal and unimodal levels.

PopulationOlder adults (50-71 years)

Sample size24

MethodsObservational

OutcomesBody Mass Index projections

ResultsSocial networks mitigate obesity in older groups.

Sign up to use Study Snapshot

Consensus is limited without an account. Create an account or sign in to get more searches and use the Study Snapshot.

Create an account

Paper

Integrating face and voice in person perception

Sign up to use Study Snapshot

References

Citations

Hear Your Face: Face-based voice conversion with F0 estimation

Our face-based voice conversion framework, which utilizes the average fundamental frequency of the target speaker, provides superior speech generation quality and aligns facial features with voice characteristics.

The dissociating effects of fear and disgust on multisensory integration in autism: evidence from evoked potentials

Autism adolescents show only limited multisensory integration for fear, but this deficit can be compensated for at later processing stages through attention-orienting effects of fear and increased attentional effort.

Neuronal Correlates of Empathy: A Systematic Review of Event-Related Potentials Studies in Perceptual Tasks

Early perceptual stages of empathy, such as N170 and augmented N400, are crucial for understanding others' emotions and inferring their emotions.

Smiling in the Face and Voice of Avatars and Robots: Evidence for a ‘Smiling McGurk Effect’

The 'emotional McGurk effect' suggests that face transmits valence information, while voice transmits arousal, with both visual and audio information needed for emotional expression in artificial agents.

Enhancing Emotion Recognition through Federated Learning: A Multimodal Approach with Convolutional Neural Networks

Federated learning with CNNs on edge devices improves audiovisual emotion recognition accuracy by 2% and preserves local client privacy.

Violations of physical and psychological expectations in the human adult brain

Adult brains process violations of physical and psychological expectations through domain-specific and domain-general computations, rather than purely visual ones.