A methodology for evaluating the training effectiveness of quadrocopter operators based on digital behavioral trace analysis and latent action patterns

Cover Page

Cite item

Full Text

Open Access Open Access
Restricted Access Access granted
Restricted Access Subscription or Fee Access

Abstract

The paper proposes and substantiates a new methodology for evaluating the effectiveness of quadcopter operator training Digital Behavioral Trace Analysis, based on end-to-end analysis of the digital trace of small managerial actions. Unlike traditional approaches that rely on objective performance metrics, psychological questionnaires or expensive physiomonitoring, Digital Behavioral Trace Analysis captures high-frequency control parameters: reaction time to a command, frequency of stick micro-corrections, statistics of deviations from the optimal trajectory and additional attributes. To test the methodology, a quadrocopter flight simulator was used, in which 10 operators performed five runs of a standard scenario "takeoff — maneuver — landing". Using automated logging, data were collected for four key attributes, after which the mean values for each participant underwent clustering using the K-means method. The experimental results showed a clear separation of the participants into two groups: "confident" operators with low reaction times and infrequent micro-corrections and uncertain participants with elongated reaction times and high frequency of minor corrections. Identifying latent patterns of behavior without involving subjective assessments or sophisticated equipment provides new opportunities for adaptive simulators and predictive monitoring systems. In particular, DBTA can serve as a basis for automatic adjustment of the complexity of training tasks in real time, timely detection of operator fatigue or overload, and more objective selection of candidates for the position of UAV pilot. In the future, the methodology is planned to be extended to include additional attributes: camera operation, trajectory analysis, interaction with the user interface, and application of supervised models for predicting readiness to perform complex flight scenarios.

Full Text

Restricted Access

About the authors

M. Y. Serebryakov

Saint-Petersburg State Marine Technical University,

Author for correspondence.
Email: dop3fun@gmail.com

Postgraduate Student, Senior Lecturer

Russian Federation, Saint-Petersburg

References

  1. Khan M. S. I., Ohlemüller R., Maloney R. F., Seddon P. J. Monitoring Dynamic Braided River Habitats: Applicability and Efficacy of Aerial Photogrammetry from Manned Aircraft versus Unmanned Aerial Systems, Drones, 2021, vol. 5, no. 2, art. 39.
  2. Somerville A., Lynar T., Joiner K., Wild G. Use of Simulation for Pre-Training of Drone Pilots, Drones, 2024, vol. 8, no. 11, art. 640.
  3. Agarwal S., Hervas-Martin E., Byrne J., Dunne A., Luis Espinosa-Aranda J., Rijlaarsdam D. An Evaluation of Low-Cost Vision Processors for Efficient Star Identification, Sensors, 2020, vol. 20, no. 21, art. 6250.
  4. Doroftei D., De Cubber G., Lo Bue S., De Smet Н. Quantitative Assessment of Drone Pilot Performance, Drones, 2024, vol. 8, no. 9, art. 482.
  5. Bauer R. M., Bilder R. M. Automated neuropsychological testing, APA handbook of neuropsychology: Neuroscience and neuromethods, 2023, vol. 2, pp. 233—258.
  6. Sorrel M. A., Olea J., Abad F. J., de la Torre J., Aguado D., Lievens F. Validity and Reliability of Situational Judgement Test Scores: А New Approach Based on Cognitive Diagnosis Models, Organizational Research Methods, 2016, vol. 19, no. 3, pp. 506—532.
  7. Bolton M. L., Biltekoff E., Humphrey L. The Mathematical Meaninglessness of the NASA Task Load Index: А Level of Measurement Analysis, IEEE Transactions on Human-Machine Systems, 2023, vol. 53, no. 3, pp. 590—599.
  8. Valenzano A., Moscatelli F., Messina A., Monda V., Orsitto R., Zezza G., Fiorentino G., Salerno M., Triggiani AI., Viggiano A., Mollica M. P., Carotenuto M., Monda M., Cibelli G., Messina G. Stress Profile in Remotely Piloted Aircraft Crewmembers During 2 h Operating Mission, Frontiers in Physiology, 2018, vol. 7, no. 9, art. 461.
  9. Ayaz H., Shewokis P. A., Bunce S., Izzetoglu K., Willems B., Onaral В. Optical brain monitoring for operator training and mental workload assessment, Neuroimage, 2012, vol. 59, no. 1, pp. 36—47.
  10. Maciejewska M., Galant-Gołębiewska M., Łodygowski T. Psychophysical State Aspect during UAV Operations, Applied Sciences. 2024, vol. 14, no. 1, art. 150.
  11. Alonso F., Faus M., Riera J. V., Fernandez-Marin M., Useche S. A. Effectiveness of Driving Simulators for Drivers’ Training: А Systematic Review, Applied Sciences, 2023, vol. 13, no. 9, art. 5266.
  12. Koç D., Seçkin A. Ç., Satı Z. E. Evaluation of Participant Success in Gamified Drone Training Simulator Using Brain Signals and Key Logs. Brain Sciences. 2021, vol. 11, no. 8, art. 1024.
  13. Papamitsiou Z., Economides А. A. Learning Analytics and Educational Data Mining in Practice: А Systematic Literature Review of Empirical Evidence, Educational Technology & Society, 2014, vol. 17, no. 4, pp. 49—64.
  14. Shukla D., Alim А. Big Data Analytics Approach using Indexing and Ranking for Excellence in Higher Education, International Journal of Computer Applications, 2018, vol. 180, no. 35, pp. 8—22.

Supplementary files

Supplementary Files
Action
1. JATS XML
2. Fig. 1. The DBTA conceptual model

Download (395KB)
3. Fig. 2. Clusterization of operators by averaged features

Download (184KB)

Copyright (c) 2026 Informacionnye Tehnologii



СМИ зарегистрировано Федеральной службой по надзору в сфере связи, информационных технологий и массовых коммуникаций (Роскомнадзор).
Регистрационный номер и дата принятия решения о регистрации СМИ: серия ПИ № 77 - 15565 от 02 июня 2003 г.