Computational nanotechnologyComputational nanotechnologyComputational nanotechnologyComputational nanotechnologyComputational nanotechnologyComputational nanotechnology2313-223X2587-9693YUR-VAK62663110.33693/2313-223X-2023-10-4-46-55SYSTEM ANALYSIS, INFORMATION MANAGEMENT AND PROCESSING, STATISTICSСИСТЕМНЫЙ АНАЛИЗ, УПРАВЛЕНИЕ И ОБРАБОТКА ИНФОРМАЦИИ, СТАТИСТИКАResearch ArticleAn Overview of Existing Methods for Automatic Generation of Test Tasks in Natural LanguageОбзор существующих методов автоматической генерации тестовых заданий на естественном языкеhttps://orcid.org/0000-0003-3845-3972MaslovaMaria A.МасловаМария Александровна
Russian Federation

senior teacher at the Department of Computer Science and Programming Technology

старший преподаватель кафедры информатики и технологии программирования

miss.mari.m@inbox.ruVolzhsky Polytechnic Institute (branch) of Volgograd State Technical UniversityВолжский политехнический институт (филиал) Волгоградского государственного технического университета1212202310446550702202407022024Copyright ©; 2023, Yur-VAKCopyright ©; 2023, Юр-ВАК2023Yur-VAKЮр-ВАКhttps://journals.eco-vector.com/2313-223X/about/editorialPolicieshttps://journals.eco-vector.com/2313-223X/article/view/626631

Recently, in the field of education, much attention has been paid to the use of multiple choice questions as a tool for assessing knowledge. The development of test tasks requires a lot of time and is highly labor intensive. It is difficult to perform such a task manually, so many researchers offer various ways and approaches to automate the creation of test tasks in natural language. In this paper, we present an overview of scientific achievements in the field of automatic question generation, which examines the classification of question generation systems by dividing them into five groups: machine learning-based methods, neural network-based, tree-based, rule-based or template-based and hybrid methods.

В последнее время в сфере образования большое внимание уделяется использованию вопросов с несколькими вариантами ответов в качестве инструмента оценки знаний. Разработка тестовых заданий требует затрат большого количества времени и имеет высокую трудоемкость. Вручную такую задачу сложно выполнить, поэтому многие исследователи предлагают различные способы и подходы для автоматизации создания тестовых заданий на естественном языке. В данной работе представляется обзор научных достижений в области автоматической генерации вопросов, в котором рассматривается классификация систем генерации вопросов путем разделения их на пять групп: методы, основанные на машинном обучении, основанные на нейронных сетях, основанные на деревьях, основанные на правилах или шаблона и гибридные методы.

automatic generation of test tasksautomatic generation of test questionsnatural language processingnatural language generationавтоматическая генерация тестовых заданийавтоматическая генерация тестовых вопросовобработка естественного языкагенерация естественного языкаAgarwal R., Negi V., Kalra A., Mittal A. Deep learning and linguistic feature based automatic multiple choice question generation from text. In: International Conference on Distributed Computing and Intelligent Technology. 2022. Vol. 13145. Pp. 260–264. URL: https://doi.org/10.1007/978-3-030-94876-4_18Aithal S.G., Rao A.B., Singh S. Automatic question-answer pairs generation and question similarity mechanism in question answering system. Applied Intelligence. 2021. No. 51. Pp. 8484–8497. URL: https://doi.org/10.1007/s10489-021-02348-9Aithal S.G., Rao A.B., Singh S. Automatic question-answer pairs generation and question similarity mechanism in question answering system // Applied Intelligence. 2021. No. 51. Pp. 8484–8497. URL: https://doi.org/10.1007/s10489-021-02348-9Azevedo P., Leite B., Cardoso H.L. et al. Exploring NLP and information extraction to jointly address question generation and answering. In: International Conference on Artificial Intelligence Applications and Innovations. 2020. Vol. 584. Pp. 396–407. URL: https://doi.org/10.1007/978-3-030-49186-4_33Chen Y., Wu L., Zaki M.J. Reinforcement learning based graph-to-sequence model for natural question generation. In: International Conference on Learning Representations. 2019. URL: https://arxiv.org/abs/1908.04942Chomphooyod P., Suchato A., Tuaycharoen N., Punyabukkana Pr. English grammar multiple-choice question generation using Text-to-Text Transfer Transformer. Computers and Education: Artificial Intelligence. 2023. Vol. 5. URL: https://doi.org/10.1016/j.caeai.2023.100158.Chomphooyod P., Suchato A., Tuaycharoen N., Punyabukkana Pr. English grammar multiple-choice question generation using Text-to-Text Transfer Transformer // Computers and Education: Artificial Intelligence. 2023. Vol. 5. URL: https://doi.org/10.1016/j.caeai.2023.100158.Conejo R., del Campo-Ávila J., Barros B. First steps towards automatic question generation and assessment of LL(1) grammars. In: International Conference on Artificial Intelligence in Education. Posters and Late Breaking Results, Workshops and Tutorials, Industry and Innovation Tracks, Practitioners’ and Doctoral Consortium. 2022. Vol. 13356. Pp. 271–275. URL: https://doi.org/10.1007/978-3-031-11647-6_50Das B., Majumder M., Phadikar S., Ahmed Sk.A. Automatic generation of fill-in-the-blank question with corpus-based distractors for E-assessment to enhance learning. Computer Applications in Engineering Education. 2019. No. 27. Pp. 1485–1495.Das B., Majumder M., Phadikar S., Ahmed Sk.A. Automatic generation of fill-in-the-blank question with corpus-based distractors for E-assessment to enhance learning // Computer Applications in Engineering Education. 2019. No. 27. Pp. 1485–1495.Das B., Majumder M., Phadikar S., Sekh A.A. Multiple-choice question generation with auto-generated distractors for computer-assisted educational assessment. Multimedia Tools and Applications. 2021. No. 80. Pp. 31907–31925. URL: https://doi.org/10.1007/s11042-021-11222-2Das B., Majumder M., Phadikar S., Sekh A.A. Multiple-choice question generation with auto-generated distractors for computer-assisted educational assessment // Multimedia Tools and Applications. 2021. No. 80. Pp. 31907–31925. URL: https://doi.org/10.1007/s11042-021-11222-2Fung Y., Kwok J.C., Lee L. et al. Automatic question generation system for English reading comprehension. ICTE: Technology in Education. Innovations for Online Teaching and Learning. 2020. Pp. 136–146.Fung Y., Kwok J.C., Lee L. et al. Automatic question generation system for English reading comprehension // ICTE: Technology in Education. Innovations for Online Teaching and Learning. 2020. Pp. 136–146.Guan M., Mondal S.K., Dai H.N., Bao H. Reinforcement learning-driven deep question generation with rich semantics. Information Processing and Management. 2023. Vol. 60. No. 2. URL: https://doi.org/10.1016/j.ipm.2022.103232Guan M., Mondal S.K., Dai H.N., Bao H. Reinforcement learning-driven deep question generation with rich semantics // Information Processing and Management. 2023. Vol. 60. No. 2. URL: https://doi.org/10.1016/j.ipm.2022.103232Kim Ya., Lee Hw., Shin J., Jung K. Improving neural question generation using answer separation. In: The Thirty-Third AAAI Conference on Artificial Intelligence. 2019. Pp. 6602–6609.Kumar A.P., Nayak A., K M.S. et al. A Novel Framework for the Generation of Multiple Choice Question Stems Using Semantic and Machine-Learning Techniques. International Journal of Artificial Intelligence in Education. 2023. URL: https://doi.org/10.1007/s40593-023-00333-6Kumar A.P., Nayak A., K M.S. et al. A Novel Framework for the Generation of Multiple Choice Question Stems Using Semantic and Machine-Learning Techniques // International Journal of Artificial Intelligence in Education. 2023. URL: https://doi.org/10.1007/s40593-023-00333-6Kusuma S.F., Siahaan D.O., Fatichah Ch. Automatic question generation with various difficulty levels based on knowledge ontology using a query template. Knowledge-Based Systems. 2022. Vol. 249. URL: https://doi.org/10.1016/j.knosys.2022.108906Kusuma S.F., Siahaan D.O., Fatichah Ch. Automatic question generation with various difficulty levels based on knowledge ontology using a query template // Knowledge-Based Systems. 2022. Vol. 249. URL: https://doi.org/10.1016/j.knosys.2022.108906Leo J., Kurdi G., Matentzoglu N. et al. Ontology-based generation of medical, multi-term MCQ. International Journal of Artificial Intelligence in Education. 2019. No. 29. Pp. 145–188.Leo J., Kurdi G., Matentzoglu N. et al. Ontology-based generation of medical, multi-term MCQ // International Journal of Artificial Intelligence in Education. 2019. No. 29. Pp. 145–188.Li W. A semantics-based approach to automatic generation of test questions and automatic verification of user answers in the intelligent tutoring systems. Open Semantic Technologies for Intelligent Systems Design. 2022. No. 6. Pp. 381–394.Li W. A semantics-based approach to automatic generation of test questions and automatic verification of user answers in the intelligent tutoring systems // Открытые семантические технологии проектирования интеллектуальных систем. 2022. № 6. С. 381–394.Li W., Grakova N., Qian L. Ontological approach to automating the processes of question generation and knowledge control in intelligent learning systems. Open Semantic Technologies for Designing Intelligent Systems. 2020. No. 4. Pp. 215–224.Li W., Grakova N., Qian L. Ontological approach to automating the processes of question generation and knowledge control in intelligent learning systems // Открытые семантические технологии проектирования интеллектуальных систем. 2020. № 4. С. 215–224.Murugan Sh., Ramakrishnan B.S. Automatic morpheme-based distractors generation for fill-in-the-blank questions using listwise learning-to-rank method for agglutinative language. Engineering Science and Technology. 2022. Vol. 26. URL: https://doi.org/10.1016/j.jestch.2021.04.012Murugan Sh., Ramakrishnan B.S. Automatic morpheme-based distractors generation for fill-in-the-blank questions using listwise learning-to-rank method for agglutinative language // Engineering Science and Technology. 2022. Vol. 26. URL: https://doi.org/10.1016/j.jestch.2021.04.012Riza L.S., Firdaus Y., Sukamto R.A., Samah W.Kh.A.F.A. Automatic generation of short-answer questions in reading comprehension using NLP and KNN. Multimedia Tools and Applications. 2023. No. 82. Pp. 41913–41940. URL: https://doi.org/10.1007/s11042-023-15191-6Riza L.S., Firdaus Y., Sukamto R.A., Samah W.Kh.A.F.A. Automatic generation of short-answer questions in reading comprehension using NLP and KNN // Multimedia Tools and Applications. 2023. No. 82. Pp. 41913–41940. URL: https://doi.org/10.1007/s11042-023-15191-6Rodriguez-Torrealba R., Garcia-Lopez E., Garcia-Cabot A. End-to-end generation of multiple-choice questions using text-to-text transfer transformer models. Expert Systems with Applications. 2022. Vol. 208. URL: https://doi.org/10.1016/j.eswa.2022.118258Rodriguez-Torrealba R., Garcia-Lopez E., Garcia-Cabot A. End-to-end generation of multiple-choice questions using text-to-text transfer transformer models // Expert Systems with Applications. 2022. Vol. 208. URL: https://doi.org/10.1016/j.eswa.2022.118258Shuai P., Li L., Liu S., Shen J. QDG: A unified model for automatic question-distractor pairs generation. Applied Intelligence. 2023. No. 53. Pp. 8275–8285. URL: https://doi.org/10.1007/s10489-022-03894-6Shuai P., Li L., Liu S., Shen J. QDG: A unified model for automatic question-distractor pairs generation // Applied Intelligence. 2023. No. 53. Pp. 8275–8285. URL: https://doi.org/10.1007/s10489-022-03894-6Steuer T., Filighera A., Rensing Ch. Remember the Facts? Investigating Answer-Aware Neural Question Generation for Text Comprehension. In: International Conference on Artificial Intelligence in Education. 2020. Vol. 12163. URL: https://doi.org/10.1007/978-3-030-52237-7_AWang H.C., Maslim M., Kan C.H. A question–answer generation system for an asynchronous distance learning platform. Education and Information Technologies. 2023. No. 28. Pp. 12059–12088. URL: https://doi.org/10.1007/s10639-023-11675-yWang H.C., Maslim M., Kan C.H. A question–answer generation system for an asynchronous distance learning platform // Education and Information Technologies. 2023. No. 28. Pp. 12059–12088. URL: https://doi.org/10.1007/s10639-023-11675-yWang Y., Rong W., Zhang J. et al. Multi-turn dialogue-oriented pretrained question generation model. Complex & Intelligent Systems. 2020. No. 6. Pp. 493–505. URL: https://doi.org/10.1007/s40747-020-00147-2Wang Y., Rong W., Zhang J. et al. Multi-turn dialogue-oriented pretrained question generation model // Complex & Intelligent Systems. 2020. No. 6. Pp. 493–505. URL: https://doi.org/10.1007/s40747-020-00147-2Astrashab V.V., Kalugina M.A., Klebanov D.A. et al. Neural network for generating questions to the text. In: International scientific practice. conf. “BIG DATA and Advanced Analytics. BIG DATA and high-level analysis”. 2020. No. 6. Pp. 363–369. (In Rus.)Астрашаб В.В., Калугина М.А., Клебанов Д.А. и др. Нейронная сеть для генерации вопросов к тексту // Междунар. науч.-практ. конф. «BIG DATA and Advanced Analytics. BIG DATA и анализ высокого уровня». 2020. № 6. С. 363–369.Desyatirikov F.A. A systematic approach to the organization of automatic testing of students of musical educational institutions. In: XXV International Scientific and Educational-Practical Conference. 2021. Vol. 3. Pp. 452–458.Десятириков Ф.А. Системный подход к организации автоматического тестирования студентов музыкальных учебных заведений // XXV Междунар. науч. и учебно-практ. конф. 2021. Т. 3. С. 452–458.Lee V. An ontological approach to automatic question generation in intelligent learning systems. Reports of BGUIR. 2020. Vol. 18. No. 5. Pp. 44–52. (In Rus.)Ли В. Онтологический подход к автоматической генерации вопросов в интеллектуальных обучающих системах // Доклады БГУИР. 2020. Т. 18. № 5. С. 44–52.