Research of neural network models of gesture recognition in the presence of negative factors

Cover Page

Cite item

Full Text

Open Access Open Access
Restricted Access Access granted
Restricted Access Subscription or Fee Access

Abstract

The problem of automatic recognition of gesture images for computer vision systems is considered. The process of preparing the initial data, creating a training and test dataset is described. A custom dataset has been created, as well as integration and preparation of external data. Research of popular neural network methods of sign language recognition has been conducted and an assessment of their effectiveness in the presence of negative factors has been obtained. Recommendations are given to improve the quality of gesture recognition.

About the authors

T. I. Buldakova

Bauman Moscow State Technical University

Author for correspondence.
Email: buldakova@bmstu.ru

Dr. Sc., Professor

Russian Federation, Moscow

V. A. Gordeev

Bauman Moscow State Technical University

Email: gordeevva@student.bmstu.ru

Master’s Student

Russian Federation, Moscow

References

  1. Valdivieso L., Vàsconez J. P., Barona L., Benalcàzar M. E. Recognition of Hand Gestures Based on EMG Signals with Deep and Double-Deep Q-Networks, Sensors, 2023, vol. 23(8), p. 3905, DOI: 1O.339O/S23O839O5.
  2. Ma’asum F. F. M., Sulaiman S., Saparon A. An overview of hand gestures recognition system techniques, IOP Conference Series: Materials Science and Engineering, IOP Publishing, Bristol, UK, 2015, vol. 99, p. 012012, doi: 10.1088/1757-899X/99/1/012012.
  3. Katasev A. S., Tukhbatullin T. I. Sign language recognition using a convolutional neural network, Vestnik Tekhnologicheskogo universiteta, 2023, vol. 26, no. 4, pp. 53—57, doi: 10.55421/1998-7072_2023_26_4_53 (in Russian).
  4. Grif M. G., Lakiya R., Prikhodko A. L., Bakeev M. A., Rajalakshmi E. Recognition of Russian and Indian sign languages based on machine learning, Sistemy analiza i obrabotki dannykh, 2021, no. 3(83), pp. 53—74, doi: 10.17212/2782-2001-2021-3-53-74 (in Russian).
  5. Yasen M., Jusoh S. S. A systematic review on hand gesture recognition techniques, challenges and applications, PeerJ Computer Science, 2019, no. 5, doi: 10.7717/peerj-cs.218.
  6. Cheok M. J., Omar Z., Jaward M. H. A review of hand gesture and sign language recognition techniques, International Journal of Machine Learning and Cybernetics, 2019, vol. 10, pp. 131—153, DOI: 1O.1OO7/S13O42-017-O7O5-5.
  7. Ryumin D. A., Kagirov I. A., Aksenov A. A., Karpov A. A. Analytical review of models and methods of automatic recognition of gestures and sign languages, Informatsionno-upravlyayushchie sistemy, 2021, no. 6 (115), pp. 10—20, doi: 10.31799/1684-88532021-6-10-20 (in Russian).
  8. Kurbanova K. Sh. Research of stages, types of modeling and methods of gesture recognition, Informacionnye tehnologii, 2024, vol. 30, no. 2, pp. 85—90, doi: 10.17587/it.30.85-90 (in Russian).
  9. Buldakova T. I., Suyatinov S. I. Biological Principles of Integration Information at Big Data Processing, International Russian Automation Conference (RusAutoCon), Sochi, Russia, 2019, p. 8867710, DOI: 1O.11O9/RUSAUTOCON.2O19.886771O.
  10. Kuntsevich A. A., Kulik G. V., Zhitnik M. E. Machine learning technologies and image capture for sign language recognition, Big Data and Advanced Analytics, 2020, no. 6-2, pp. 308—310 (in Russian).
  11. Vishnevskaya Yu. A., Buldakova T. I. Application of a synergetic model for character recognition, Yuzhno-Uralskaya molodezhnaya shkola po matematicheskomu modelirovaniyu: Sbornik trudov IV vserossiyskoy studencheskoy nauchno-prakticheskoy konferentsii, Chelyabinsk, SUSU Publishing Center, 2021, pp. 58—62 (in Russian).
  12. Liu W., Anguelov D., Erhan D., Szegedy C., Reed S., Fu C.-Y., Berg A. SSD: Single Shot MultiBox Detector, Leibe B., Matas J., Sebe N., Welling M. (eds), Computer Vision — ECCV 2016. Lecture Notes in Computer Science, 2016, vol. 9905, pp. 21—37, Springer, Cham, doi: 10.1007/978-3-319-46448-0_2.
  13. Ghosh A., Sufian A., Sultana F., Chakrabarti A., De D. Fundamental Concepts of Convolutional Neural Network, Balas V., Kumar R., Srivastava R. (eds) Recent Trends and Advances in Artificial Intelligence and Internet of Things. Intelligent Systems Reference Library, 2020, vol. 172, Springer, Cham., doi: 10.1007/978-3-030-32644-9_36.
  14. Tammina S. Transfer learning using VGG-16 with Deep Convolutional Neural Network for Classifying Images, International Journal of Scientific and Research Publications (IJSRP), 2019, vol. 9, iss. 10, doi: 10.29322/IJSRP.9.10.2019.p9420.
  15. Theckedath D., Sedamkar R. R. Detecting Affect States Using VGG16, ResNet50 and SE-ResNet50 Networks, SN Computer Science, 2020, vol. 1, no. 79, doi: 10.1007/s42979-020-0114-9.

Supplementary files

Supplementary Files
Action
1. JATS XML
Download

Copyright (c) 2025 Informacionnye Tehnologii


СМИ зарегистрировано Федеральной службой по надзору в сфере связи, информационных технологий и массовых коммуникаций (Роскомнадзор).
Регистрационный номер и дата принятия решения о регистрации СМИ: серия ПИ № 77 - 15565 от 02 июня 2003 г.