Using a neural network to diagnose the fatigue of a serviceman-operator by his speech
- Authors: Yakovlev A.V.1, Matytsin V.O.1, Matytsina S.V.1
-
Affiliations:
- The S.M.Kirov Military Medical Academy of the Ministry of Defense of the Russian Federation
- Issue: Vol 344, No 9 (2023)
- Pages: 56-63
- Section: Hygiene and physiology of military labor
- URL: https://journals.eco-vector.com/0026-9050/article/view/630190
- DOI: https://doi.org/10.52424/00269050_2023_344_9_56
- ID: 630190
Cite item
Open Access
Access granted
Abstract
The work of a serviceman of an operator profile is characterized by an intense strain of attention with the development of fatigue, which can lead to an increase in the number of errors. Therefore, an urgent task is to conduct non-invasive monitoring of the operator’s activity without interfering with his work. The development of an adequate tool for diagnosing a person’s fatigue through his speech will allow him to control his functional state in the process of work. To simulate fatigue, a cardiorespiratory test with physical activity was used in 9 male volunteers aged 21–24 years. An audio recording of the volunteers’ voices was carried out initially and after the test was completed; as a speech load, the volunteers read out a set of standard phonetically representative texts (commands, verse, prose). Thus, a data bank of Russian-language audio recordings of speech was collected, containing information about the state of fatigue caused by physical activity. The databank made it possible to create a training sample for the neural network, because of which the ability to recognize the state of physical fatigue by the respondent’s voice was achieved with a result of at least 65%.
Full Text
About the authors
A. V. Yakovlev
The S.M.Kirov Military Medical Academy of the Ministry of Defense of the Russian Federation
Email: matitsin@list.ru
кандидат технических наук, доцент
Russian Federation, St. PetersburgV. O. Matytsin
The S.M.Kirov Military Medical Academy of the Ministry of Defense of the Russian Federation
Author for correspondence.
Email: matitsin@list.ru
кандидат медицинских наук
Russian Federation, St. PetersburgS. V. Matytsina
The S.M.Kirov Military Medical Academy of the Ministry of Defense of the Russian Federation
Email: matitsin@list.ru
Russian Federation, St. Petersburg
References
- ГОСТ 16600-72 Передача речи по трактам радиотелефонной связи. Требования… [Электронный ресурс]. URL: http://docs.cntd.ru/document/gost-16600-72 (дата обращения: 31.10.2022).
- Матыцин В.О., Велюга В.А., Найденова К.А., Пархоменко В.А. Распознавание утомления человека на основе анализа его речи с помощью нейросетевых технологий // Вестн. Юж.-Урал. гос. ун-та. Серия: Вычислит. математ. и информат. – 2023. – Т. 12, № 1. – С. 46–60.
- Медведев В.И. Физиология трудовой деятельности. Основы современной физиологии. – СПб: Наука, 1993. – 528 c.
- Сысоев В.Н., Апчел В.Я., Корнилова А.А. и др. Оценка динамики функционального состояния организма операторов в течение рабочей смены // Вестн. Рос. воен.-мед. акад. – 2012. – № 4 (40). – С. 169–172.
- Яковлев А.В. Использование многослойных сетей-автоэнкодеров для распознавания усталости человека на основе речевых данных / Обработка, передача и защита информации в компьютерных системах: Сб. докл. Второй междунар. науч. конф. Санкт-Петербург, 2022. – СПб, 2022. – С. 87–94.
- Яковлев А., Матыцин В., Велюга В., Найденова К. Опыт распознавания состояния утомления с использованием нейронных сетей. – СПб, 2021. – 15 с.
- Baykaner Kh.R., Huckvale M., Whiteley I., Andreeva S. Predicting Fatigue and Psychophysiological Test Performance from Speech for Safety-Critical Environments // Frontiers in Bioengineering and Biotechnology. – 2015. – Vol. 3. – Р. 124.
- Cohen A.S., Renshaw T.L., Mitchell K.R., Kim Yu. A psychometric investigation of «macroscopic» speech measures for clinical and psychological science // Behavior Research Methods. 2016. – Vol. 48, N 2. – Р. 475–486.
- Eyben F., Scherer K.R., Schuller B.W. et al. The Geneva Minimalistic Acoustic Parameter Set (GeMAPS) for Voice Research and Affective Computing // IEEE Transactions on Affective Computing. – 2016. – Vol. 7, N 2. – Р. 190–202.
- Fan X., Sheng W.Yi., Cheng X.G., Zhi X.Z. An event-related potential objective evaluation study of mental fatigue based on 2-back task // J. of Biomedical Engineering. – 2018. – Vol. 35, N 6. – Р. 837–844.
- Hidalgo-Gadea G., Kreuder A., Krajewski J., Vorstius C. Towards better microsleep predictions in fatigued drivers: Exploring benefits of personality traits and IQ // Ergonomics. – 2021. – Vol. 64, N 6. – Р. 1–43.
- Levitan S.I., An G., Ma M. et al. Combining acoustic-prosodic, lexical, and phonotactic features for automatic deception detection / Interspeech-2016. – P. 2006–2010.
- Martin V.P., Rouas J.-L., Philip P. Dеtection de la somnolence dans la voix: nouveaux marqueurs et nouvelles stratеgies // TAL Traitement Automatique des Langues. – 2021. – Vol. 61. – Р. 67–90.
- Parada-Cabaleiro E., Costantini G., Batliner A. et al. DEMoS: an Italian emotional speech corpus: Elicitation methods, machine learning, and perception // Language Resources and Evaluation. – 2019. – Vol. 54. – Р. 341–383.
- Remacle A., Garnier M., Gerber S. et al. Vocal Change Patterns During a Teaching Day: Inter- and Intra-subject Variability // J. of Voice: Official Journal of the Voice Foundation. – 2018. – Vol. 32, N 1. – Р. 57–63.
- Solberg G., Robstad B., Skjonsberg O.H., Borchsenius F. Respiratory gas exchange indices for estimating the anaerobic threshold // J. of Sports Science & Medicine. – 2005. – Vol. 4, N 1. – Р. 29–36.
- Yamada Y., Kobayashi M. Detecting mental fatigue from eye-tracking data gathered while watching video: Evaluation in younger and older adults // Artificial Intelligence in Medicine. – 2018. – Vol. 91. – Р. 39–48.
Supplementary files
СМИ зарегистрировано Федеральной службой по надзору в сфере связи, информационных технологий и массовых коммуникаций (Роскомнадзор).