Algorithm for identifying abnormal actions

Capa

Citar

Texto integral

Acesso aberto Acesso aberto
Acesso é fechado Acesso está concedido
Acesso é fechado Acesso é pago ou somente para assinantes

Resumo

The study is devoted to the problem of recognition of human activity recognition and the definition of normal and abnormal behavior (activity) depending on the action scene. Automated detection of abnormal activity using computer vision technologies and rapid response makes it possible to improve the work of rapid response services, thereby saving human lives or stopping offenses. The paper presents a comprehensive review of methods for recognizing human activity and detecting abnormal human activity based on deep learning. Various classifications of abnormal activity are investigated, and then deep learning methods and neural network architectures used to detect abnormal activity are discussed and analyzed. Based on the comparative analysis of various approaches, an algorithm for recognizing human activity has been proposed and a neural network has been developed that determines violent and nonviolent actions with an accuracy of 92,22% in 150 epochs.

Texto integral

Acesso é fechado

Sobre autores

Namir Khadi

MIREA – Russian Technological University

Autor responsável pela correspondência
Email: hadi@mirea.ru
ORCID ID: 0009-0000-7122-5942
Código SPIN: 4079-2513
Researcher ID: LEM-0157-2024

assistant lecturer, Department of Computer and Information Security

Rússia, Moscow

Dmitry Andryushenkov

MIREA – Russian Technological University

Email: andryushenkov@mirea.ru
ORCID ID: 0009-0004-5927-9795
Código SPIN: 4113-2967

assistant lecturer, Department of Computer and Information Security

Rússia, Moscow

Alexander Chesalin

MIREA – Russian Technological University

Email: chesalin@mirea.ru
ORCID ID: 0000-0002-1154-6151
Código SPIN: 4334-5520
Scopus Author ID: 57210931888
Researcher ID: D-8080-2019

Cand. Sci. (Eng.), Head, Department of Computer and Information Security

Rússia, Moscow

Bibliografia

  1. Aberkane S., Elarbi M. Deep reinforcement learning for real-world anomaly detection in surveillance videos. URL: https://sci-hub.ru/10.1109/ispa48434.2019.8966795 (data of accesses: 28.01.2024).
  2. Agahian S., Negin F., Köse С. An efficient human action recognition framework with pose-based spatiotemporal features. URL: https://www.sciencedirect.com/science/article/pii/S2215098618312345?via%3Dihub (data of accesses: 02.02.2024).
  3. Al-Dhamari A., Sudirman R., Mahmood N.H. Transfer deep learning along with binary support vector machine for abnormal behavior detection. URL: https://www.researchgate.net/publication/340145184_Transfer_Deep_Learning_Along_With_Binary_Support_Vector_Machine_for_Abnormal_Behavior_Detection#fullTextFileContent (data of accesses: 24.01.2024).
  4. Amrani H., Micucci D., Paolo N. Unsupervised deep learning-based clustering for human activity recognition. URL: https://www.researchgate.net/publication/365337299_Unsupervised_Deep_Learning-based_clustering_for_Human_Activity_Recognition#fullTextFileContent (data of accesses: 27.01.2024).
  5. Contardo P., Tomassini S., Falcionelli N. et al. Combining a mobile deep neural network and a recurrent layer for violence detection in videos. URL: https://ceur-ws.org/Vol-3402/paper05.pdf (data of accesses: 25.01.2024).
  6. Duong H.-T., Le V.-T., Hoang V. Deep learning-based anomaly detection in video surveillance: A survey. URL: https://typeset.io/papers/deep-learning-based-anomaly-detection-in-video-surveillance-29l7zb9s (data of accesses: 26.01.2024).
  7. Elesawy M., Hussein M., El Massih M.A. Real life violence situations dataset. URL: https://www.kaggle.com/datasets/mohamedmustafa/real-life-violence-situations-dataset (data of accesses: 03.02.2024).
  8. Gu T., Dolan-Gavitt B., Garg S. BadNets: Identifying vulnerabilities in the machine learning model supply chain. URL: https://arxiv.org/abs/1708.06733 (data of accesses: 03.02.2024).
  9. Haq Qazi E.U., Zia T., Almorjan A. Deep learning-based digital image forgery detection system. URL: https://www.researchgate.net/publication/359153551_Deep_Learning-Based_Digital_Image_Forgery_Detection_System (data of accesses: 03.02.2024).
  10. Jayaswal R., Dixit M. Framework for anomaly classification using deep transfer learning approach. URL: https://iieta.org/journals/ria/paper/10.18280/ria.350309 (data of accesses: 24.01.2024).
  11. Jia C., Yi W., Wu Y. et al. Abnormal activity capture from passenger flow of elevator based on unsupervised learning and fine-grained multi-label recognition. URL: https://arxiv.org/abs/2006.15873 (data of accesses: 28.01.2024).
  12. Jia J.-G., Zhou Y.-F., Hao X.-W. et al. Two-stream temporal convolutional networks for skeleton-based human action recognition. URL: https://sci-hub.ru/10.1007/s11390-020-0405-6 (data of accesses: 30.01.2024).
  13. Lathifah N., Lin H.-I. A brief review on behavior recognition based on key points of human skeleton and eye gaze to prevent human error. In: Proceedings of the 2022 13th Asian Control Conference (ASCC). Jeju Island, Republic of Korea, 2022. Pp. 1396–1403.
  14. Lin C.-B., Dong Z., Kuan W.-K., Huang Y.-F. A framework for fall detection based on openpose skeleton and LSTM/GRU models. URL: https://www.researchgate.net/publication/348142284_A_Framework_for_Fall_Detection_Based_on_OpenPose_Skeleton_and_LSTMGRU_Models (data of accesses: 29.01.2024).
  15. Lin F.-C., Ngo H.-H., Dow C.-R. et al. Student behavior recognition system. for the classroom environment based on skeleton pose estimation and person detection. URL: https://www.researchgate.net/publication/353746430_Student_Behavior_Recognition_System_for_the_Classroom_Environment_Based_on_Skeleton_Pose_Estimation_and_Person_Detection (data of accesses: 30.01.2024).
  16. Maqsood R., Bajwa UI., Saleem G. et al. Anomaly recognition from surveillance videos using 3D convolutional neural networks. URL: https://arxiv.org/pdf/2101.01073 (data of accesses: 03.02.2024).
  17. Naik A., Gopalakrishna M. Deep-violence: Individual person violent activity detection in Video. URL: https://www.researchgate.net/publication/349393071_Deep-violence_individual_person_violent_activity_detection_in_video (data of accesses: 29.01.2024).
  18. Nauman M.A., Shoaib M. Identification of anomalous behavioral patterns in crowd scenes. URL: https://www.techscience.com/cmc/v71n1/45453/html (data of accesses: 29.01.2024).
  19. Pang G., Shen C., Cao L., Hengel A. Deep learning for anomaly detection: A review. URL: https://ink.library.smu.edu.sg/cgi/viewcontent.cgi?article=8019&context=sis_research (data of accesses: 26.01.2024).
  20. Pawar K., Attar V. Deep learning approaches for video-based anomalous activity detection. URL: https://www.sci-hub.ru/10.1007/s11280-018-0582-1 (data of accesses: 26.01.2024).
  21. Pimentel T., Monteiro M., Veloso A., Ziviani N. Deep active learning for anomaly detection. URL: https://sci-hub.ru/10.1109/ijcnn48605.2020.9206769 (data of accesses: 27.01.2024).
  22. Simonyan K., Zisserman A. Two-stream convolutional networks for action recognition in videos. URL: https://arxiv.org/pdf/1406.2199 (data of accesses: 28.01.2024).
  23. Sultani W., Chen C., Shah M. Real-world anomaly detection in surveillance videos. URL: https://paperswithcode.com/paper/real-world-anomaly-detection-in-surveillance (data of accesses: 03.02.2024).
  24. Tomar S., Sharma A.K., Tina, Gupta K. Pose based activity recognition using supervised machine learning algorithms. URL: https://www.ijert.org/research/pose-based-activity-recognition-using-supervised-machine-learning-algorithms-IJERTV10IS120084.pdf (data of accesses: 27.01.2024).
  25. Tran D., Bourdev L., Fergus R. et al. Learning spatiotemporal features with 3D convolutional networks. URL: https://arxiv.org/pdf/1412.0767 (data of accesses: 28.01.2024).
  26. Tran D.A., Fischer P., Smajic A., So Y. Real-time object detection for autonomous driving using deep learning. URL: https://www.researchgate.net/publication/350090136_Real-time_Object_Detection_for_Autonomous_Driving_using_Deep_Learning (data of accesses: 26.01.2024).
  27. Vrskova R., Hudec R., Kamencay P., Sykora P. A new approach for abnormal human activities recognition based on ConvLSTM architecture. URL: https://www.researchgate.net/publication/360159604_A_New_Approach_for_Abnormal_Human_Activities_Recognition_Based_on_ConvLSTM_Architecture (data of accesses: 28.01.2024).
  28. Zhang F., Bazarevsky V., Vakunov A. et al. Mediapipe hands: On-device real-time hand tracking. URL: https://arxiv.org/pdf/2006.10214.pdf (data of accesses: 01.02.2024).
  29. Zhao Y., Deng B., Shen Ch. et al. Spatio-temporal autoencoder for video anomaly detection. URL: https://sci-hub.ru/10.1145/3123266.3123451 (data of accesses: 28.01.2024).

Arquivos suplementares

Arquivos suplementares
Ação
1. JATS XML
Baixar
2. Fig. 1. Supervised learning [24]

Baixar (899KB)
Metadados
3. Fig. 2. Semi-supervised learning on GAN example

Baixar (794KB)
Metadados
4. Fig. 3. Unsupervised learning [4]

Baixar (1MB)
Metadados
5. Fig. 4. Transfer learning on example of anomaly detection действий [3]

Baixar (544KB)
Metadados
6. Fig. 5. Deep active learning [21]: re – external reward; ri – intrinsic reward; s – observation; a – action

Baixar (377KB)
Metadados
7. Fig. 6. Deep reinforcement learning [1]

Baixar (1MB)
Metadados
8. Fig. 7. Example of deep hybrid models-based on anomaly detection [3] (below – abnormal crowd activity)

Baixar (1MB)
Metadados
9. Fig. 8. Two-stream network architecture [22]

Baixar (174KB)
Metadados
10. Fig. 9. 3D-convolution operation for anomaly detection [16]

Baixar (1MB)
Metadados
11. Fig. 10. ConvLSTM architecture [27]

Baixar (505KB)
Metadados
12. Fig. 11. The scheme of the algorithm using OpenPose [14]

Baixar (1MB)
Metadados
13. Fig. 12. Original image

Baixar (704KB)
Metadados
14. Fig. 13. Output image with labels

Baixar (750KB)
Metadados
15. Fig. 14. A skeleton obtained from an image in 3D space

Baixar (340KB)
Metadados
16. Fig. 15. Examples of different anomalies

Baixar (2MB)
Metadados
17. Fig. 16. Resnet50v2 architecture [9]

Baixar (997KB)
Metadados
18. Fig. 17. The algorithm for anomaly detection

Baixar (1MB)
Metadados
19. Fig. 18. Loss and accuracy graphs on the training and validation sample

Baixar (357KB)
Metadados
20. Fig. 19. Confusion matrix

Baixar (264KB)
Metadados
21. Fig. 20. Classification report

Baixar (231KB)
Metadados
22. Fig. 21. Detection of violent acts

Baixar (2MB)
Metadados
23. Fig. 22. Detection of nonviolent acts

Baixar (2MB)
Metadados