Informacionnye TehnologiiInformacionnye TehnologiiИнформационные технологии1684-6400New Technologies Publishing House70234010.17587/it.32.20-27Modeling and optimizationМоделирование и оптимизацияResearch ArticleMethodology for constructing benchmarks for assessing the efficiency of feature selection methods when constructing regressionМетодика построения бенчмарков для оценки эффективности методов отбора признаков при решении задач регрессииCheremuhinA. D.ЧеремухинА. Д.
Russian Federation

Cand. Econ. Sc., Associate Professor

канд. экон. наук, доц.

ngieu.cheremuhin@yandex.ruLyaminA. S.ЛяминА. С.
Russian Federation

Cand. Econ. Sc., Associate Professor

канд. экон. наук, доц.

a.s.lyamin@gmail.comNizhny Novgorod State Engineering and Economics UniversityНижегородский государственный инженерно-экономический университет1501202632120270802202608022026Copyright ©; 2026, Informacionnye TehnologiiCopyright ©; 2026, Информационные технологии2026Informacionnye TehnologiiИнформационные технологииhttps://journals.eco-vector.com/1684-6400/article/view/702340

The problem of constructing benchmarks for assessing the efficiency of feature selection algorithms used in regression problems is considered. The developed benchmark includes synthetic and real dataframes that differ in various data parameters, such as dimensionality, correlation between features, noise level, and the ratio between numerical and categorical variables. А generation method using various distributions and control parameters is proposed for synthetic data. А computational experiment using a feature selection method based on the partial least squares method was conducted, demonstrating the efficiency of the proposed benchmark for objective comparison of algorithms. The ways of further development of the benchmark are outlined, including expanding the set of parameters and using new data types.

Рассматривается проблема построения бенчмарков для оценки эффективности алгоритмов отбора признаков, применяемых в задачах регрессии. Разработанный бенчмарк включает синтетические и реальные датафреймы, отличающиеся разнообразными параметрами данных, такими как размерность, корреляция между признаками, уровень шума и соотношение между числовыми и категориальными переменными. Для синтетических данных предложен метод генерации с использованием различных распределений и контрольных параметров. В отличие от традиционных бенчмарков разработанная методика строится не на использовании устоявшихся библиотек, а на оригинальном подходе к генерации и отбору данных, ориентированном на редкие и трудноформализуемые свойства реальных задач. Проведен вычислительный эксперимент с использованием метода отбора признаков, основанного на методе частичных наименьших квадратов, продемонстрировавший эффективность предложенного бенчмарка для объективного сравнения алгоритмов. Намечены пути дальнейшего развития бенчмарка, включая расширение набора параметров и использование новых типов данных.

feature selectionbenchmarkregression problemssynthetic datafeature correlationfiltering methodswrapper methodsbuilt-in methodsотбор признаковбенчмаркзадачи регрессиисинтетические данныекорреляция признаковметоды фильтрацииметоды оберткивстроенные методыZhou Y., Cheng G., Jiang S., Da M. Building an efficient intrusion detection system based on feature selection and ensemble classifier, Computer networks, 2020, vol. 174, p. 107247, DOI: 10.1016/j.comnet.2020.107247Zhou Y. et al. Building an efficient intrusion detection system based on feature selection and ensemble classifier // Computer networks. 2020. Vol. 174. P. 107247. DOI: 10.1016/j.comnet.2020.107247Zargari S., Voorhis D. Feature Selection in the Corrected KDD-dataset, 2012 Third International Conference on Emerging Intelligent Data and Web Technologies, IEEE, 2012, pp. 174—180, DOI: 10.1109/EIDWT.2012.10Zargari S., Voorhis D. Feature Selection in the Corrected KDD-dataset // 2012 Third International Conference on Emerging Intelligent Data and Web Technologies. IEEE, 2012. P. 174—180. DOI: 10.1109/EIDWT.2012.10Urbanowicz R. J., Olson R. S., Schmitt P., Meeker M., Moore J. H. Benchmarking relief-based feature selection methods for bioinformatics data mining, Journal of biomedical informatics, 2018, vol. 85, pp. 168—188, DOI:10.1016/j.jbi.2018.07.015Urbanowicz R. J. et al. Benchmarking relief-based feature selection methods for bioinformatics data mining //Journal of biomedical informatics. 2018. Vol. 85. P. 168—188. DOI: 10.1016/j.jbi.2018.07.015Zhang Y., Mistry K., Lim C. P., Neoh S. C. Binary differential evolution with self-learning for multi-objective feature selection, Information Sciences, 2020, vol. 507, pp. 67—85, DOI: 10.1016/j.ins.2019.08.040Zhang Y. et al. Binary differential evolution with self-learning for multi-objective feature selection //Information Sciences. 2020. Vol. 507. P. 67—-85. DOI: 10.1016/j.ins.2019.08.040Visalakshi S., Radha V. А literature review of feature selection techniques and applications: Review of feature selection in data mining, 2014 IEEE international conference on computational intelligence and computing research, IEEE, 2014, pp. 1—6, DOI: 10.1109/ICCIC.2014.7238499Visalakshi S., Radha V. А literature review of feature selection techniques and applications: Review of feature selection in data mining // 2014 IEEE international conference on computational intelligence and computing research. IEEE, 2014. P. 1—6. DOI: 10.1109/ICCIC.2014.7238499Singh R., Kumar H., Singla R. K. Analysis of feature selection techniques for network traffic dataset, 2013 international conference on machine intelligence and research advancement, IEEE, 2013, pp. 42—46, DOI: 10.1109/ICMIRA.2013.15Singh R., Kumar H., Singla R. K. Analysis of feature selection techniques for network traffic dataset //2013 international conference on machine intelligence and research advancement. IEEE, 2013. P. 42—46. DOI: 10.1109/ICMIRA.2013.15Hoffmann F., Bertram T., Mikut R., Reischl M., Nelles O. Benchmarking in classification and regression, Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 2019, vol. 9, no. 5, p. e1318, DOI: 10.1002/widm.1318Hoffmann F. et al. Benchmarking in classification and regression //Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery. 2019. Vol. 9. N. 5. P. e1318. DOI: 10.1002/widm.1318Bommert A., Sun X., Bischl B., Rahnenführer J., Lang M. Benchmark for filter methods for feature selection in high-dimensional classification data, Computational Statistics & Data Analysis, 2020, vol. 143, p. 106839, DOI: 10.1016/j.csda.2019.106839Bommert A. et al. Benchmark for filter methods for feature selection in high-dimensional classification data //Computational Statistics & Data Analysis. 2020. Vol. 143. P. 106839. DOI: 10.1016/j.csda.2019.106839Hopf K., Reifenrath S. Filter Methods for Feature Selection in Supervised Machine Learning Applications--Review and Benchmark, arXiv preprint arXiv:2111.12140, 2021, DOI: 10.48550/arXiv.2111.12140Hopf K., Reifenrath S. Filter Methods for Feature Selection in Supervised Machine Learning Applications--Review and Benchmark // arXiv preprint arXiv:2111.12140. 2021. DOI: 10.48550/arXiv.2111.12140Overschie J. G. S., Alsahaf A., Azzopardi G. fseval: a benchmarking framework for feature selection and feature ranking algorithms, Journal of Open Source Software, 2022, vol. 7, no. 79, p. 4611, DOI: 10.21105/joss.04611Overschie J. G. S., Alsahaf A., Azzopardi G. fseval: a benchmarking framework for feature selection and feature ranking algorithms // Journal of Open Source Software. 2022. Vol. 7, N. 79. P. 4611. DOI: 10.21105/joss.04611El-Kenawy E. S. M., Eid M. M., Saber M., Ibrahim А. MbGWO-SFS: Modified binary grey wolf optimizer based on stochastic fractal search for feature selection, IEEE Access, 2020, vol. 8, pp. 107635—107649, DOI: 10.1109/ACCESS.2020.3001151El-Kenawy E. S. M. et al. MbGWO-SFS: Modified binary grey wolf optimizer based on stochastic fractal search for feature selection //IEEE Access. 2020. Vol. 8. P. 107635—107649. DOI: 10.1109/ACCESS.2020.3001151Alhussan A. A. et al. А binary waterwheel plant optimization algorithm for feature selection, IEEE Access, 2023, vol. 11, pp. 94227—94251, DOI: 10.1109/ACCESS.2023.3312022Alhussan A. A. et al. А binary waterwheel plant optimization algorithm for feature selection // IEEE Access. 2023. Vol. 11. P. 94227—94251. DOI: 10.1109/ACCESS.2023.3312022Li Y., Mansmann U., Du S., Hornung R. Benchmark study of feature selection strategies for multi-omics data, BMC bioinformatics, 2022, vol. 23, no. 1, p. 412, DOI: 10.1186/s12859-022-04962-xLi Y. et al. Benchmark study of feature selection strategies for multi-omics data //BMC bioinformatics. 2022. Vol. 23, N. 1. P. 412. DOI: 10.1186/s12859-022-04962-xOreski D., Oreski S., Klicek В. Effects of dataset characteristics on the performance of feature selection techniques, Applied Soft Computing, 2017, vol. 52, pp. 109—119, DOI: 10.1016/j.asoc.2016.12.023Oreski D., Oreski S., Klicek В. Effects of dataset characteristics on the performance of feature selection techniques //Applied Soft Computing. 2017. Vol. 52. P. 109—119. DOI: 10.1016/j.asoc.2016.12.023Parmezan A. R. S., Lee H. D., Spolaôr N., Wu F. C. Automatic recommendation of feature selection algorithms based on dataset characteristics, Expert Systems with Applications, 2021, vol. 185, p. 115589, DOI: 10.1016/j.eswa.2021.115589Parmezan A. R. S. et al. Automatic recommendation of feature selection algorithms based on dataset characteristics //Expert Systems with Applications. 2021. Vol. 185. P. 115589. DOI: 10.1016/j.eswa.2021.115589Oliveira A. L. I., Braga P. L., Lima R. M. F., Cornélio M. L. GA-based method for feature selection and parameters optimization for machine learning regression applied to software effort estimation, Information and Software Technology, 2010, vol. 52, no. 11, pp. 1155—1166, DOI: 10.1016/j.infsof.2010.05.009Oliveira A. L. I. et al. GA-based method for feature selection and parameters optimization for machine learning regression applied to software effort estimation // information and Software Technology. 2010. Vol. 52, N. 11. P. 1155—1166. DOI: 10.1016/j.infsof.2010.05.009Agrawal P., Abutarboush H. F., Ganesh T., Mohamed А. W. Metaheuristic algorithms on feature selection: А survey of one decade of research (2009-2019), IEEE Access, 2021, vol. 9, pp. 26766—26791, DOI: 10.1109/ACCESS.2021.3056407Agrawal P. et al. Metaheuristic algorithms on feature selection: А survey of one decade of research (2009-2019) // Ieee Access. 2021. Vol. 9. P. 26766—26791. DOI: 10.1109/ACCESS.2021.3056407Pearson K. Mathematical contributions to the theory of evolution, X: Supplement to a memoir on skew variation, Philosophical Transactions of the Royal Society A., 1901, no. 197, pp. 287—299, DOI: 10.1098/rsta.1901.0023Pearson K. Mathematical contributions to the theory of evolution, X: Supplement to a memoir on skew variation // Philosophical Transactions of the Royal Society A. 1901. N. 197. P. 287—299. DOI: 10.1098/rsta.1901.0023Endres D. M., Schindelin J. E. A new metric for probability distributions, IEEE Trans. Inf. Theory, 2003, vol. 49, no. 7, pp. 1858—1860, DOI: 10.1109/TIT.2003.813506Endres D. M., Schindelin J. E. A new metric for probability distributions // IEEE Trans. Inf. Theory. 2003. Vol. 49, N. 7. P. 1858—1860. DOI: 10.1109/TIT.2003.813506Ye K. Q. Orthogonal column Latin hypercubes and their application in computer experiments, Journal of the American Statistical Association, 1998, vol. 93, no. 444, pp. 1430—1439, DOI: 10.1080/01621459.1998.10473803Ye K. Q. Orthogonal column Latin hypercubes and their application in computer experiments // Journal of the American Statistical Association. 1998. N. 93 (444). P. 1430—1439. DOI: 10.1080/01621459.1998.10473803Mehmood T., Sæbø S., Liland K. H. Comparison of variable selection methods in partial least squares regression, Journal of Chemometrics, 2020, vol. 34, no. 6, p. e3226, DOI: 10.1002/cem.3226Mehmood T., Sæbø S., Liland K. H. Comparison of variable selection methods in partial least squares regression // Journal of Chemometrics. 2020. Vol. 34, N. 6. P. e3226. DOI: 10.1002/cem.3226Jia P., Wang Y., Du Z., Zhao X., Wang Y., Chen B., Wang W., Guo H., Tang R. Erase: Benchmarking feature selection methods for deep recommender systems, Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2024, pp. 5194—5205. DOI: 10.1145/3637528.3671571Jia P. et al. Erase: Benchmarking feature selection methods for deep recommender systems // Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 2024. P. 5194—5205. DOI: 10.1145/3637528.3671571Karagiannaki K., Panousopoulou A., Tsakalides P. А benchmark study on feature selection for human activity recognition, Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct, 2016, pp. 105—108, DOI: 10.1145/2968219.2971421Karagiannaki K., Panousopoulou A., Tsakalides P. А benchmark study on feature selection for human activity recognition // Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct. 2016. P. 105—108. DOI: 10.1145/2968219.2971421Generalov I. G. Cyclical influence of factors on the strategic development of grain production, Vestnik NGIEI, 2024, no. 8 (159), pp. 74—83 (in Russian), DOI: 10.24412/2227-9407-2024-8-74-83Генералов И. Г. Циклическое влияние факторов на стратегическое развитие производства зерна // Вестник НГИЭИ. 2024. № 8(159). С. 74—83. DOI: 10.24412/2227-9407-2024-8-74-83Kafiev I. R., Romanov P. S., Romanov I. P. On the issue of automation of the process of sorting porcini mushrooms using neural networks, Vestnik NGIEI, 2024, no. 4 (155), pp. 34—49 (in Russian), DOI: 10.24412/2227-9407-2024-4-34-49Кафиев И. Р., Романов П. С., Романов И. П. К вопросу автоматизации процесса сортировки белых грибов с использованием нейронных сетей // Вестник НГИЭИ. 2024. № 4 (155). С. 34—49. DOI: 10.24412/2227-9407-2024-4-34-49