Научная электронная библиотека ULRICH'S Periodicals Directory

Сибирский государственный университет
науки и технологий имени академика М.Ф. Решетнева

Сибирский аэрокосмический журнал
ISSN 2712-8970 (Print) ISSN 2782-5760 (on-line)

Сведения об образовательной организации

UDK 004.93
HYBRID ALGORITHM FOR CONVOLUTIONAL NEURAL NETWORK LEARNING
I. A. Ivanov, E. A. Sopov
Reshetnev Siberian State Aerospace University 31, Krasnoyarsky Rabochy Av., Krasnoyarsk, 660037, Russian Federation
The problems of image pattern recognition are nowadays solved in many areas – beginning with satellite analysis of the Earth surface all the way to human face analysis in the human-machine interaction systems. One of the most successful algorithms of image analysis and recognition is the convolutional neural network. The back propagation algorithm used for training of these networks belongs to the class of gradient algorithms; therefore it often converges to local minima. In this paper we propose a hybrid algorithm for convolutional network training, which is aimed at searching for global optimum, and consisting of two stages. At the first stage a genetic algorithm is used to search for the subdomain of the search space that includes the global optimum, whereas at the second stage a back propagation algorithm is used to find the actual global optimum. Genetic algorithm includes an internal procedure that maintains diversity of the solutions population, which allows examining the search space more thoroughly and finding various successful convolutional network configurations. The solution found by the genetic algorithm is used at the second stage as an initial approximation of network weights. After that the network is trained using a back propagation algorithm. The developed hybrid algorithm has been tested on the emotion recognition problem; it has been compared to a traditional back propagation algorithm. A comparison was made on the classification accuracy, as well as the F-measure for the emotion recognition problem in two formulations – speaker-dependent classification, and speakerindependent classification. Hybrid algorithm has yielded a better performance according to both criteria, and in both problem formulations compared to the standard algorithm used for convolutional neural network training.
Keywords: convolutional neural network, genetic algorithm, back propagation algorithm, human-machine interaction.
References

1. LeCun Y., Jackel L. D., Bottou L., Brunot A., Cortes C., Denker J. S., Drucker H., Guyon I., Muller U. A., Sackinger E., Simard P., Vapnik V. Comparison of learning algorithms for handwritten digit recognition. International Conference on Artificial Neural Networks. 1995, P. 53–60.

2. Semenkina M. E., Semenkin E. S., Popov E. A. [Effectiveness study of the self-configuring genetic algorithm for selecting an effective variant of spacecraft control system]. Programmnye produkty i sistemy. 2013, No. 4, P. 208–213 (In Russ.).

3. Ivanov I. A., Sopov E. A. [Effectiveness study of the self-configuring of co-evolutionary algorithm for solving complex multi-objective optimization problems]. Sistemy upravleniya i informatsionnye tekhnologii. 2013, Vol. 51, No. 1.1, P. 141–145 (In Russ.).

4. Eiben A. E., Raue P.-E., Ruttkay Z. Genetic algorithms with multi-parent recombination. Proceedings of the International Conference on Evolutionary Computation, The Third Conference on Parallel Problem Solving from Nature. 1994, P. 78–87.

5. Ting C-K. On the Mean Convergence Time of Multi-parent Genetic Algorithms Without Selection. Advances in Artificial Life. 2005, P. 403–412.

6. Akbari Z. A multilevel evolutionary algorithm for optimizing numerical functions. International Journal of Industrial Engineering Computations 2. 2010, P. 419–430.

7. Panfilov I. A., Bazanova E. P., Sopov E. A. [Effectiveness study of the genetic optimization algorithm with alternative solutions representation]. Vestnik SibGAU. 2013, No. 4(50), P. 68–71 (In Russ.).

8. Sopov E. A., Semenkin E. S. [Effectiveness study of the modified probabilistic genetic algorithm on deceptive “trap” functions]. Sistemy upravleniya i informatsionnye tekhnologii. 2011, No. 3 (45), P. 90–95 (In Russ.).

9. Akhmedova Sh. A., Semenkin E. S. [Cooperative bionic algorithm of unconditional optimization]. Programmnye produkty i sistemy. 2013, No. 4 (104), P. 133–137 (In Russ.).

10. Michalewicz Z. Genetic Algorithms + Data Structures = Evolution Programs, 3rd Edition. Springer-Verlag. 1996.

11. Ciresan D. C., Meier U., Masci J., Gambardella L. M., Schmidhuber J. Flexible, high performance convolutional neural networks for image classification. Proceedings of the International Joint Conference on Artificial Intelligence. 2011, Vol. 22 (1), P. 1237–1242.

12. Haq S., Jackson P. J. B. Speaker-dependent audiovisual emotion recognition. Proceedings Int. Conf. on Auditory-Visual Speech Processing, 2009, P. 53–58.

13. Sidorov M., Ultes S., Schmitt A. Emotions are a personal thing: Toward speaker-adaptive emotion recognition. IEEE International Conference on Acoustics, Speech and Signal Processing. 2014, P. 4803–4807.

14. Schuller B., Muller R., Lang M., Rigoll G. Speaker independent emotion recognition by early fusion of acoustic and linguistic features within ensembles. Proc. INTERSPEECH. 2005, P. 805–808.

15. Brester C., Sidorov M., Semenkin E. Speechbased emotion recognition: Application of collective decision making concepts. Proceedings of the 2nd International Conference on Computer Science and Artificial Intelligence. 2014, P. 216–220.


Ivanov Ilia Andreievich – postgraduate student, Reshetnev Siberian State Aerospace University. E-mail:

ilyaiv92@gmail.com.

Sopov Evgenii Alexandrovich – Cand. Sc., docent, docent of Department of System analysis and operations

research, Reshetnev Siberian State Aerospace University. E-mail: evgenysopov@gmail.com.