Beschreibung:

Beschreibung: Performance of automatic speech recognition (ASR) systems can significantly be improved by integrating further sources of information such as additional modalities, or acoustic channels, or acoustic models. Given the arising problem of information fusion, striking parallels to problems in digital communications are exhibited, where the discovery of the turbo codes by Berrou et al. was a groundbreaking innovation. In this paper, we show ways how to successfully apply the turbo principle to the domain of ASR and thereby provide solutions to the abovementioned information fusion problem. The contribution of our work is fourfold: First, we review the turbo decoding forward-backward algorithm (FBA), giving detailed insights into turbo ASR, and providing a new interpretation and formulation of the so-called extrinsic information being passed between the recognizers. Second, we present a real-time capable turbodecoding Viterbi algorithm suitable for practical information fusion and recognition tasks. Then we present simulation results for a multimodal example of information fusion. Finally we prove the suitability of both our turbo FBA and turbo Viterbi algorithm also for a single-channel multi-model recognition task obtained by using two acoustic feature extraction methods. On a small vocabulary task (challenging, since spelling is included), our proposed turbo ASR approach outperforms even the best reference system on average over all SNR conditions and investigated noise types by a relative word error rate (WER) reduction of 22.4% (audio-visual task) and 18.2% (audio-only task), respectively.

Beschreibung: While most pattern recognition approaches are designed and trained with clearly defined reference labels, there are a few new applications working with ambiguous ones. Since the linear discriminant analysis (LDA) is one of the most utilized methods in pattern recognition to reduce the dimensionality of feature vectors, typically increasing the robustness of the features, we propose in this work a modification of the LDA in order to be able to handle ambiguous reference labels in a soft-decision way. In the field of social signal processing (here: emotion recognition) we demonstrate that using a soft accuracy measure evaluating the classifier’s confidence output by means of a soft-labeled emotional speech database really provides a degree of similarity to (naturally ambiguous) human votes. The adaptation of our classifier to such soft accuracy measure takes place by a retraining w.r.t. the human vote distribution. Applying this soft accuracy measure to emotion recognition with ambiguous reference labels both retraining the classifier and using the new soft LDA method leads to around 22% relative increase of accuracy.