Description: Research of gender effects in the field of automatic speech emotion recognition (SER) has been subject to research in the past. Still, however, it is somewhat unclear to which degree speaker gender influences SER. Although we will prove it to be wrong for our SER model, usually it is assumed that a "gender-dependent" emotion recognizer performs better than a "gender-independent" one by using gender-dependent emotional speech training data. In this paper, we use a state-of-the-art emotion recognizer model to investigate the effects of speaker genders in training and in test. The gender-specific SER model performs well under matched-gender test conditions, as expected. Training with approximate half the amount of the training data from male and female speakers jointly, is about as good as separate training using gender-specific data and test in matched conditions with in total the same amount of training data. This however, only holds for an average over male and female speakers: Interestingly, we show that female voices' emotions are recognized better on a mixed-gender SER model, than on a female SER model, indicating that female speakers express emotions in a wider variety than male speakers?

Description:

Description: While most pattern recognition approaches are designed and trained with clearly defined reference labels, there are a few new applications working with ambiguous ones. Since the linear discriminant analysis (LDA) is one of the most utilized methods in pattern recognition to reduce the dimensionality of feature vectors, typically increasing the robustness of the features, we propose in this work a modification of the LDA in order to be able to handle ambiguous reference labels in a soft-decision way. In the field of social signal processing (here: emotion recognition) we demonstrate that using a soft accuracy measure evaluating the classifier’s confidence output by means of a soft-labeled emotional speech database really provides a degree of similarity to (naturally ambiguous) human votes. The adaptation of our classifier to such soft accuracy measure takes place by a retraining w.r.t. the human vote distribution. Applying this soft accuracy measure to emotion recognition with ambiguous reference labels both retraining the classifier and using the new soft LDA method leads to around 22% relative increase of accuracy.