UDK 519.87
ROBUST AND RELIABLE TECHNIQUES FOR SPEECH-BASED EMOTION RECOGNITION
C. Yu. Brester* [1], O. E. Semenkina [1], M. Yu. Sidorov [2]
[1] Siberian State Aerospace University named after academician M. F. Reshetnev 31, Krasnoyarsky Rabochy Av., Krasnoyarsk, 660014, Russian Federation [2]Ulm University 43, Albert-Einstein-Allee, Ulm, 89081, Germany Е-mail: christina.bre@yandex.ru
One of the crucial challenges related to the spacecraft control is the monitoring of the mental state of crew members as well as operators of the flight control centre. In most cases, visual information is not sufficient, because spacemen are trained to cope with feelings and not to express emotions explicitly. In order to identify the genuine mental state of a crew member, it is reasonable to engage the acoustic characteristics obtained from speech signals presenting voice commands during the spacecraft control and interpersonal communication. Human emotion recognition implies flexible algorithmic techniques satisfying the requirements of reliability and fast operation in real time. In this paper we consider the heuristic feature selection procedure based on the self-adaptive multi-objective genetic algorithm that allows the number of acoustic characteristics involved in the recognition process to be reduced. The effectiveness of this approach and its robustness property are revealed in experiments with various classification models. The usage of this procedure leads to a reduction of the feature space dimension by a factor of two (from 384 to approximately 180 attributes), which means decreasing the time resources spent by the recognition algorithm. Moreover, it is proposed to implement some algorithmic schemes based on collective decision making by the set of classifiers (Multilayer Perceptron, Support Vector Machine, Linear Logistic Regression) that permits the improvement of the recognition quality (by up to 10% relative improvement). The developed algorithmic schemes provide a guaranteed level of effectiveness and might be used as a reliable alternative to the random choice of a classification model. Due to the robustness property the heuristic feature selection procedure is successfully applied on the data pre-processing stage, and then the approaches realizing the collective decision making schemes are used.
emotion recognition, adaptive multi-objective genetic algorithm, classifier, collective decision making.
References
  1. Sidorov M., Ultes S., Schmitt A. Emotions are a personal thing: Towards speaker-adaptive emotion recognition. ICASSP, 2014, p. 4803–4807.
  2. Eyben F., Wöllmer M., and Schuller B. Opensmile: the munich versatile and fast opensource audio feature extractor. Proceedings of the international conference on Multimedia, 2010. ACM, p. 1459–1462.
  3. Boersma P. Praat, a system for doing phonetics by computer. Glot international, 2002, no. 5(9/10), p. 341–345.
  4. Sidorov M., Brester C., Minker W., Semenkin E. Speech-Based Emotion Recognition: Feature Selection by Self-Adaptive Multi-Criteria Genetic Algorithm. LREC, 2014, p. 3481–3485.
  5. Brester C., Semenkin E., Sidorov M., Minker W. Self-adaptive multi-objective genetic algorithms for feature selection. Proceedings of International Conference on Engineering and Applied Sciences Optimization  (OPT-i’14), 2014, p. 1838–1846.
  6. Kohavi R., John G. H. Wrappers for feature subset selection. Artificial Intelligence, 97, 1997, p. 273–324.
  7. Venkatadri M., Srinivasa Rao K. A multiobjective genetic algorithm for feature selection in data mining. International Journal of Computer Science and Information Technologies, 2010, vol. 1, no. 5, p. 443–448.
  8. Brester C., Sidorov M., Semenkin E. Acoustic Emotion Recognition: Two Ways of Features Selection Based on Self-Adaptive Multi-Objective Genetic Algorithm. Proceedings of the International Conference on Informatics in Control, Automation and Robotics (ICINCO), 2014, p. 851–855.
  9. Zitzler E., Thiele L. Multiobjective evolutionary algorithms: A comparative case study and the strength pareto approach. Evolutionary Computation, IEEE Transactions on, 1999, vol. 3, no. 4, p. 257–271.
  10. Sergienko R., Semenkin E. Competitive Cooperation for Strategy Adaptation in Coevolutionary Genetic Algorithm for Constrained Optimization. IEEE World Congress on Computational Intelligence (WCCI'2010), Barcelona, Spain, 2010, p. 1626–1631.
  11. Daridi F., Kharma N., and Salik, J. Parameterless genetic algorithms: review and innovation. IEEE Canadian Review, 2004, no. (47), p. 19–23.
  12. Burkhardt F., Paeschke A., Rolfes M., Sendlmeier W. F., and Weiss B. A database of german emotional speech. In Interspeech, 2005, p. 1517–1520.
  13. Grimm M., Kroschel K., and Narayanan S. The vera am mittag German audio-visual emotional speech database. In Multimedia and Expo, IEEE International Conference on, IEEE, 2008, p. 865–868.
  14. Mori H., Satake T., Nakamura M., and Kasuya H. Constructing a spoken dialogue corpus for studying paralinguistic information in expressive conversation and analyzing its statistical/acoustic characteristics. Speech Communication, 2011, 53 p.
  15. Hall M., Frank E., Holmes G., Pfahringer B., Reutemann P., Witten I. H. The WEKA Data Mining Software: An Update, SIGKDD Explorations, 2009, vol. 11, Iss. 1.
  16. Goutte C., Gaussier E. A probabilistic interpretation of precision, recall and F-score, with implication for evaluation. ECIR'05 Proceedings of the 27th European conference on Advances in Information Retrieval Research, 2005, p. 345–359.
  17. Popov E. A., Semenkina M. E., Lipinskiy L. V. [Decision making with intelligent information technology ensemble]. Vestnik SibGAU. 2012, no. 5 (45), p. 95–99 (In Russ).

Brester Christina Yurievna – junior research fellow, Siberian State Aerospace University named after academician M. F. Reshetnev. Е-mail: christina.bre@yandex.ru

Semenkina Olga Ernestovna – Dr. Sc., professor, professor of the Higher Mathematics Department, Siberian State Aerospace University named after academician M. F. Reshetnev. Е-mail: semenkina.olga@mail.ru

Sidorov Maxim Yurievich – research fellow, University of Ulm. Е-mail: maxim.sidorov@uni-ulm.de