Informacionnye TehnologiiInformacionnye TehnologiiИнформационные технологии1684-6400New Technologies Publishing House70601710.17587/it.32.171-184Modeling and optimizationМоделирование и оптимизацияResearch ArticleValidation of social agent training: synthesis of reinforcement learning and evolutionary optimization methodsАпробация обучения социальных агентов: синтез обучения с подкреплением и эволюционных методов оптимизацииChernikovA. V.ЧерниковА. В.
Russian Federation

Postgraduate Student, Senior Lecturer

аспирант, ст. преподаватель

aleksandrchernikov98@gmail.comMoscow State University of Technology "STANKIN"Московский государственный технологический университет "СТАНКИН"110420263241711841104202611042026Copyright ©; 2026, Informacionnye TehnologiiCopyright ©; 2026, Информационные технологии2026Informacionnye TehnologiiИнформационные технологииhttps://journals.eco-vector.com/1684-6400/article/view/706017

The article proposes a time series forecasting model designed for unstable and partially observable environments. Unlike traditional approaches, the developed FELAR architecture combines local agent learning with reward adjustment based on collective characteristics (trust, reputation, influence), alongside global evolutionary adaptation of strategies. The proposed model operates in a distributed multi-agent environment, enabling both local adaptive behavior and global strategy evolution. A set of experiments on publicly available time series datasets (urban traffic, transformer temperatures, electricity consumption) confirms the model’s high forecasting accuracy and robustness to concept drift. The article details the agent architecture, algorithmic loop, experimental setup, and computational efficiency of the approach. The paper highlights key advantages of the approach, including robustness to concept drift, real-time adaptability, and low computational overhead.

Предлагается модель прогнозирования временных рядов в условиях нестабильной и частично наблюдаемой среды. В отличие от классических решений разработанная архитектура FELAR сочетает локальное обучение агента с корректировкой награды на основе коллективных характеристик (доверие, репутация, влияние) и глобальной эволюционной адаптацией стратегий. Модель реализована в виде распределенной мультиагентной системы с возможностью локального адаптивного поведения и глобального обновления стратегий. Проведен эксперимент на открытых наборах временных рядов (дорожный трафик, температура трансформаторов, прогнозирование энергопотребления), где модель демонстрирует высокую точность прогнозирования и устойчивость к концептуальному дрейфу. Описана архитектура агентов, алгоритмический цикл, параметры эксперимента и вычислительная эффективность предложенного подхода. Обсуждаются преимущества подхода, включая устойчивость к дрейфу данных, возможность работы в режиме реального времени и низкие вычислительные затраты.

adaptive learningreinforcement learningevolutionary algorithmsfuzzy logicmulti-agent systemsadaptationconcept drifttime series forecastingQ-learningpopulation-based methodsdistributed controlадаптивное обучениеобучение с подкреплениемэволюционные алгоритмынечеткая логикамультиагентные системыадаптацияконцептуальный дрейфпрогноз временных рядовQ-обучениепопуляционные методыраспределенное управлениеYakovleva E. A. et al. The Role of Artificial Intelligence Technologies in the Digital Transformation of the Economy, Questions of Innovative Economy, 2023, vol. 13, no. 2, pp. 707—726.Яковлева Е. А. и др. Роль технологий искусственного интеллекта в цифровой трансформации экономики // Вопросы инновационной экономики. 2023. Т. 13, № 2. С. 707—726.Chalam Y. Y. Adaptive control systems: Techniques and applications, London, UK, Routledge, 2017.Chalam Y. Y. Adaptive control systems: Techniques and applications. London, UK: Routledge, 2017.Sewak M., Sahay S. K., Rathore H. Policy-approximation based deep reinforcement learning techniques: an overview, Information and Communication Technology for Competitive Strategies (ICTCS 2020) ICT: Applications and Social Interfaces, 2021, pp. 493—507.Sewak M., Sahay S. K., Rathore H. Policy-approximation based deep reinforcement learning techniques: an overview // Information and Communication Technology for Competitive Strategies (ICTCS 2020) ICT: Applications and Social Interfaces. 2021. P. 493—507.Yang X. S. Nature-inspired optimization algorithms: Challenges and open problems, Journal of Computational Science, 2020, vol. 46, p. 101104.Yang X. S. Nature-inspired optimization algorithms: Challenges and open problems // Journal of Computational Science. 2020. Vol. 46. P. 101104.Hansen N., Wang X., Su H. Temporal difference learning for model predictive control, arXiv preprint arXiv:2203.04955, 2022.Hansen N., Wang X., Su H. Temporal difference learning for model predictive control // arXiv preprint arXiv:2203.04955. 2022.Gladkov L., Kureichik V., Kureichik V. Genetic algorithms, Moscow, Publishing house Liters "Physical and Mathematical Literature", 2022.Гладков Л., Курейчик В., Курейчик В. Генетические алгоритмы. Москва: Litres "Физико-математическая литература", 2022.Sakharov M. K., Karpenko A. P. Memetic algorithms for solving the problem of global nonlinear optimization. Review, Mechanical Engineering and Computer Technologies, 2015, no. 12, pp. 119—142.Сахаров М. К., Карпенко А. П. Меметические алгоритмы для решения задачи глобальной нелинейной оптимизации. Обзор // Машиностроение и компьютерные технологии. 2015. № . 12. С. 119—142.Karpenko A. P. Modern algorithms for search engine optimization, Algorithms inspired by nature, Moscow, Bauman Moscow State Technical University, 2017.Карпенко А. П. Современные алгоритмы поисковой оптимизации. Алгоритмы, вдохновленные природой. М.: МГТУ им. Баумана, 2017.Li S. E. Deep reinforcement learning //Reinforcement learning for sequential decision and optimal control, Singapore, Springer Nature Singapore, 2023, pp. 365-402.Li S. E. Deep reinforcement learning //Reinforcement learning for sequential decision and optimal control. Singapore: Springer Nature Singapore, 2023. P. 365—402.Gao Z. et al. Application of deep q-network in portfolio management, 2020 5th IEEE International Conference on Big Data Analytics (ICBDA), IEEE, 2020, pp. 268—275.Gao Z. et al. Application of deep q-network in portfolio management // 2020 5th IEEE International Conference on Big Data Analytics (ICBDA). IEEE, 2020. P. 268—275.Ceron J. S. O., Castro P. S. Revisiting rainbow: Promoting more insightful and inclusive deep reinforcement learning research, International Conference on Machine Learning, PMLR, 2021, pp. 1373—1383.Ceron J. S. O., Castro P. S. Revisiting rainbow: Promoting more insightful and inclusive deep reinforcement learning research // International Conference on Machine Learning. PMLR, 2021. P. 1373—1383.Yu E. et al. Online boosting adaptive learning under concept drift for multistream classification, Proceedings of the AAAI Conference on Artificial Intelligence, 2024, vol. 38, no. 15, pp. 16522—16530.Yu E. et al. Online boosting adaptive learning under concept drift for multistream classification // Proceedings of the AAAI Conference on Artificial Intelligence. 2024. Vol. 38, N. 15. P. 16522—16530.Saleh A. et al. Trust-aware routing mechanism through an edge node for IoT-enabled sensor networks, Sensors, 2022, vol. 22, no. 20, p. 7820.Saleh A. et al. Trust-aware routing mechanism through an edge node for IoT-enabled sensor networks // Sensors. 2022. Vol. 22, N. 20. P. 7820.Zadeh L. A. Fuzzy logic //Granular, fuzzy, and soft computing, New York, NY, Springer US, 2023, pp. 19—49.Zadeh L. A. Fuzzy logic //Granular, fuzzy, and soft computing. New York, NY: Springer US, 2023. P. 19—49.Zhong X. et al. Hybrid path planning based on safe A* algorithm and adaptive window approach for mobile robot in large-scale dynamic environment, Journal of Intelligent & Robotic Systems, 2020, vol. 99, no. 1, pp. 65—77.Zhong X. et al. Hybrid path planning based on safe A* algorithm and adaptive window approach for mobile robot in large-scale dynamic environment //J ournal of Intelligent & Robotic Systems. 2020. Vol. 99, N. 1. P. 65—77.Bessonov A. V. Symbolic specification and analysis of software models of hybrid systems: diss., Novosib. state tech. univ, 2016.Бессонов А. В. Символьная спецификация и анализ программных моделей гибридных систем: дис. Новосиб. гос. техн. ун-т, 2016.Gladkov L. A., Gladkova N. V. New approaches to building systems for analysis and knowledge extraction based on hybrid methods, Bulletin of the Southern Federal University. Technical sciences, 2010, vol. 108, no. 7, pp. 146—153.Гладков Л. А., Гладкова Н. В. Новые подходы к построению систем анализа и извлечения знаний на основе гибридных методов // Известия Южного федерального университета. Технические науки. 2010. Т. 108, № . 7. С. 146—153.Oroojlooy A., Hajinezhad D. A review of cooperative multi-agent deep reinforcement learning, Applied Intelligence, 2023, vol. 53, no. 11, pp. 13677—13722.Oroojlooy A., Hajinezhad D. A review of cooperative multi-agent deep reinforcement learning // Applied Intelligence. 2023. Vol. 53, N. 11. P. 13677—13722.Malyszko M. Fuzzy logic in selection of maritime search and rescue units, Applied Sciences, 2021, vol. 12, no. 1, p. 21.Malyszko M. Fuzzy logic in selection of maritime search and rescue units // Applied Sciences. 2021. Vol. 12, N. 1. P. 21.Willemsen F. J. et al. A methodology for comparing optimization algorithms for auto-tuning, Future Generation Computer Systems, 2024, vol. 159 pp. 489—504.Willemsen F. J. et al. A methodology for comparing optimization algorithms for auto-tuning // Future Generation Computer Systems. 2024. Vol. 159. P. 489—504.Ponnambalam S. G., Janardhanan M. N., Rishwaraj G. Trust-based decision-making framework for multiagent system, Soft Computing, 2021, vol. 25 no. 11, pp. 7559—7575.Ponnambalam S. G., Janardhanan M. N., Rishwaraj G. Trust-based decision-making framework for multiagent system // Soft Computing. 2021. Vol. 25, N. 11. P. 7559—7575.Zubaroğlu A., Atalay V. Data stream clustering: a review, Artificial Intelligence Review, 2021, vol. 54, no 2, pp. 1201—1236.Zubaroğlu A., Atalay V. Data stream clustering: a review // Artificial Intelligence Review. 2021. Vol. 54, N. 2. P. 1201—1236.Li F. et al. Dynamic graph convolutional recurrent network for traffic prediction: Benchmark and solution, ACM Transactions on Knowledge Discovery from Data, 2023, vol. 17, no. 1, pp. 1—21.Li F. et al. Dynamic graph convolutional recurrent network for traffic prediction: Benchmark and solution // ACM Transactions on Knowledge Discovery from Data. 2023. Vol. 17, N. 1. P. 1—21.Zhao Y. et al. TranDRL: a transformer-driven deep reinforcement learning enabled prescriptive maintenance framework, IEEE Internet of Things Journal, 2024, pp. 35432—35444.Zhao Y. et al. TranDRL: a transformer-driven deep reinforcement learning enabled prescriptive maintenance framework // IEEE Internet of Things Journal. 2024. P. 35432—35444.Goncalves C. et al. Dataset of an energy community’s generation and consumption with appliance allocation, Data in Brief, 2022, vol. 45, p. 108590.Goncalves C. et al. Dataset of an energy community’s generation and consumption with appliance allocation // Data in Brief. 2022. Vol. 45. P. 108590.Wang C., Wei H. Dynamic Memory Derived Graph learning for Spatio-Temporal Metro Flow Prediction, 2024 IEEE International Conference on Big Data (BigData), IEEE, 2024, pp. 2924—2931.Wang C., Wei H. Dynamic Memory Derived Graph learning for Spatio-Temporal Metro Flow Prediction // 2024 IEEE International Conference on Big Data (BigData). IEEE, 2024. P. 2924—2931.Zhu S., Zhang Y., Feng Y. GW-net: An efficient grad-CAM consistency neural network with weakening of random erasing features for semi-supervised person re-identification, Image and Vision Computing, 2023, vol. 137, p. 104790.Zhu S., Zhang Y., Feng Y. GW-net: An efficient grad-CAM consistency neural network with weakening of random erasing features for semi-supervised person re-identification // Image and Vision Computing. 2023. Vol. 137. P. 104790.Plizzari C., Cannici M., Matteucci M. Spatial temporal transformer network for skeleton-based action recognition, Pattern recognition. ICPR international workshops and challenges: virtual event, January 10—15, 2021, Proceedings, Part III, Springer International Publishing, 2021, pp. 694—701.Plizzari C., Cannici M., Matteucci M. Spatial temporal transformer network for skeleton-based action recognition // Pattern recognition. ICPR international workshops and challenges: virtual event, January 10—15, 2021. Proceedings, Part III. Springer International Publishing, 2021. P. 694—701.Yoon J., Zame W. R., van der Schaar M. Multi-directional recurrent neural networks: A novel method for estimating missing data, Time series workshop in international conference on machine learning, 2017, pp. 1—5.Yoon J., Zame W. R., van der Schaar M. Multi-directional recurrent neural networks: A novel method for estimating missing data // Time series workshop in international conference on machine learning. 2017. P. 1—5.Du W., Côté D., Liu Y. Saits: Self-attention-based imputation for time series, Expert Systems with Applications, 2023, vol. 219, p. 119619.Du W., Côté D., Liu Y. Saits: Self-attention-based imputation for time series // Expert Systems with Applications. 2023. Vol. 219. P. 119619.Li Y. et al. Deep learning based on Transformer architecture for power system short-term voltage stability assessment with class imbalance, Renewable and Sustainable Energy Reviews, 2024, vol. 189, p. 113913.Li Y. et al. Deep learning based on Transformer architecture for power system short-term voltage stability assessment with class imbalance //Renewable and Sustainable Energy Reviews. 2024. Vol. 189. P. 113913.Gupta M. et al. Concurrent imputation and prediction on EHR data using bi-directional GANs: Bi-GANs for EHR imputation and prediction, Proceedings of the 12th ACM International Conference on Bioinformatics, Computational Biology, and Health Informatics, 2021, pp. 1—9.Gupta M. et al. Concurrent imputation and prediction on EHR data using bi-directional GANs: Bi-GANs for EHR imputation and prediction // Proceedings of the 12th ACM International Conference on Bioinformatics, Computational Biology, and Health Informatics. 2021. P. 1—9.Ashman M. et al. Sparse gaussian process variational autoencoders, arXiv preprint arXiv:2010.10177, 2020.Ashman M. et al. Sparse gaussian process variational autoencoders // arXiv preprint arXiv:2010.10177. 2020.Zhang J., Zheng Y., Qi D. Deep spatio-temporal residual networks for citywide crowd flows prediction, Proceedings of the AAAI conference on artificial intelligence, 2017, vol. 31, no. 1.Zhang J., Zheng Y., Qi D. Deep spatio-temporal residual networks for citywide crowd flows prediction // Proceedings of the AAAI conference on artificial intelligence. 2017. Vol. 31, N. 1.Li Y. et al. Diffusion convolutional recurrent neural network: Data-driven traffic forecasting, arXiv preprint arXiv:1707.01926, 2017.Li Y. et al. Diffusion convolutional recurrent neural network: Data-driven traffic forecasting // arXiv preprint arXiv:1707.01926. 2017.Zeng A. et al. Are transformers effective for time series forecasting?, Proceedings of the AAAI conference on artificial intelligence, 2023, vol. 37, no. 9, pp. 11121—11128.Zeng A. et al. Are transformers effective for time series forecasting? // Proceedings of the AAAI conference on artificial intelligence. 2023. Vol. 37, N. 9. P. 11121—11128.Amalou I., Mouhni N., Abdali A. Multivariate time series prediction by RNN architectures for energy consumption forecasting, Energy Reports, 2022, vol. 8, pp. 1084—1091.Amalou I., Mouhni N., Abdali A. Multivariate time series prediction by RNN architectures for energy consumption forecasting // Energy Reports. 2022. Vol. 8. P. 1084—1091.Khalil M. et al. Machine Learning, Deep Learning and Statistical Analysis for forecasting building energy consumption—A systematic review, Engineering Applications of Artificial Intelligence, 2022, vol. 115, p. 105287.Khalil M. et al. Machine Learning, Deep Learning and Statistical Analysis for forecasting building energy consumption— A systematic review // Engineering Applications of Artificial Intelligence. 2022. Vol. 115. P. 105287.Zhou H. et al. Informer: Beyond efficient transformer for long sequence time-series forecasting, Proceedings of the AAAI conference on artificial intelligence, 2021, vol. 35, no. 12, pp. 11106—11115.Zhou H. et al. Informer: Beyond efficient transformer for long sequence time-series forecasting // Proceedings of the AAAI conference on artificial intelligence. 2021. Vol. 35, N. 12. P. 11106—11115.Dissanayake B. et al. A comparison of ARIMAX, VAR and LSTM on multivariate short-term traffic volume forecasting, Conference of open innovations association, FRUCT, FRUCT Oy, 2021, no. 28, pp. 564—570.Dissanayake B. et al. A comparison of ARIMAX, VAR and LSTM on multivariate short-term traffic volume forecasting // Conference of open innovations association, FRUCT. FRUCT Oy, 2021. N. 28. P. 564—570.Assad U. et al. Smart grid, demand response and optimization: A critical review of computational methods, Energies, 2022, vol. 15, no. 6, p. 2003.Assad U. et al. Smart grid, demand response and optimization: A critical review of computational methods // Energies. 2022. Vol. 15, N. 6. P. 2003.Pham Q. V. et al. Deep learning for intelligent demand response and smart grids: A comprehensive survey, arXiv preprint arXiv:2101.08013, 2021.Pham Q. V. et al. Deep learning for intelligent demand response and smart grids: A comprehensive survey // arXiv preprint arXiv:2101.08013. 2021.Aziz H. et al. Task allocation using a team of robots, Current Robotics Reports, 2022, vol. 3, no. 4, pp. 227—238.Aziz H. et al. Task allocation using a team of robots // Current Robotics Reports. 2022. Vol. 3, N. 4. P. 227—238.Sharma S., Kumar V. A comprehensive review on multi-objective optimization techniques: Past, present and future, Archives of Computational Methods in Engineering, 2022, vol. 29, no. 7, pp. 5605—5633.Sharma S., Kumar V. A comprehensive review on multi-objective optimization techniques: Past, present and future // Archives of Computational Methods in Engineering. 2022. Vol. 29, N. 7. P. 5605—5633.Cano A., Krawczyk B. Kappa updated ensemble for drifting data stream mining, Machine Learning, 2020, vol. 109, no. 1, pp. 175—218.Cano A., Krawczyk B. Kappa updated ensemble for drifting data stream mining // Machine Learning. 2020. Vol. 109, N. 1. P. 175—218.Wu J. et al. Toward human-in-the-loop AI: Enhancing deep reinforcement learning via real-time human guidance for autonomous driving, Engineering, 2023, no. 21, pp. 75—91.Wu J. et al. Toward human-in-the-loop AI: Enhancing deep reinforcement learning via real-time human guidance for autonomous driving // Engineering. 2023. N. 21, P. 75—91.