Informacionnye TehnologiiInformacionnye TehnologiiИнформационные технологии1684-6400New Technologies Publishing House70219310.17587/it.31.80-87Neural network technologiesНейросетевые технологииReview ArticleOn neural network methods of image reconstruction and super-resolutionО методах восстановления и увеличения разрешения изображений на основе генеративно-состязательных нейронных сетейRubinovK. A.РубиновК. А.
Russian Federation

аспирант

RubinovKA@mpei.ruNational Research University "Moscow Power Engineering Institute"Национальный исследовательский университет "МЭИ"1502202531280870402202604022026Copyright ©; 2025, Informacionnye TehnologiiCopyright ©; 2025, Информационные технологии2025Informacionnye TehnologiiИнформационные технологииhttps://journals.eco-vector.com/1684-6400/article/view/702193

The methods for solving the problems of image inpainting and image super-resolution by means of image generation with neural networks are considered. Generative and adversarial neural networks are created and trained to solve them. It is shown that, in a wide range, the recovery quality almost does not depend on the fraction of damaged pixels, that adding residual blocks does not lead to its improvement, and that the generative adversarial network for resolution increase gives better results than the bicubic interpolation.

Рассмотрены методы решения задач восстановления изображения и увеличения разрешения изображения с помощью генерации изображений генеративно-состязательными нейронными сетями. Показано, что в широком диапазоне качество восстановления практически не зависит от доли поврежденных пикселей, добавление остаточных блоков не приводит к его улучшению, генеративо-состязательная сеть для увеличения разрешения дает лучшие результаты, чем бикубическая интерполяция.

neural networksgenerative adversarial networksimage inpaintingsuper-resolutionнейронные сетигенеративно-состязательные сетивосстановление изображенийповышение разрешенияKarras T., Laine S., Aila T. A Style-Based Generator Architecture for Generative Adversarial Networks, arxiv, 2018, available at: https://arxiv.org/abs/1812.04948.Karras T., Laine S., Aila T. A Style-Based Generator Architecture for Generative Adversarial Networks // arxiv. 2018. URL: https://arxiv.org/abs/1812.04948.He K., Zhang X., Ren S., Sun J. Deep Residual Learning for Image Recognition, arxiv, 2015, available at: https://arxiv.org/abs/1512.03385.He K., Zhang X., Ren S., Sun J. Deep Residual Learning for Image Recognition // arxiv. 2015. URL: https://arxiv.org/abs/1512.03385.Foster D. Generative Deep Learning, Sebastopol, California, USA, O’Reilly Media, 2019, 286 p.Foster D. Generative Deep Learning. Sebastopol, California, USA: O’Reilly Media, 2019. 286 с.Langr J., Bok V. GANs in Action: Deep learning with Generative Adversarial Networks, Shelter Island, New York, USA, Manning Publications, 2019, 278 p.Langr J., Bok V. GANs in Action: Deep learning with Generative Adversarial Networks. Shelter Island, New York, USA: Manning Publications, 2019. 278 с.Goodfellow I. J., Pouget-Abadie J., Mirza M., Xu B., Warde-Farley D., Ozair S., Courville A., Bengio Y. Generative Adversarial Networks, arxiv, 2014, available at: https://arxiv.org/abs/1406.2661.Goodfellow I. J., Pouget-Abadie J., Mirza M., Xu B., Warde-Farley D., Ozair S., Courville A., Bengio Y. Generative Adversarial Networks // arxiv. 2014. URL: https://arxiv.org/abs/1406.2661.Radford A., Metz L., Chintala S. Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks, arxiv, 2016, available at: https://arxiv.org/abs/1511.06434.Radford A., Metz L., Chintala S. Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks // arxiv. 2016. URL: https://arxiv.org/abs/1511.06434.Ioffe S., Szeged C. Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift, arxiv, 2015, available at: URL: https://arxiv.org/abs/1502.03167.Ioffe S., Szeged C. Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift // arxiv. 2015. URL: https://arxiv.org/abs/1502.03167.Yeh R. A., Chen C., Yian Lim T., Schwing A. G., Hasegawa-Johnson M., Do M. N. Semantic Image Inpainting with Deep Generative Models, arxiv, 2016, available at: https://arxiv.org/abs/1607.07539.Yeh R. A., Chen C., Yian Lim T., Schwing A. G., Hasegawa-Johnson M., Do M. N. Semantic Image Inpainting with Deep Generative Models // arxiv. 2016. URL: https://arxiv. org/abs/1607.07539.Ledig C., Theis L., Huszar F., Caballero J., Cunningham A., Acosta A., Aitken A., Tejani A., Totz J., Wang Z., Sh W. Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network, arxiv, 2016, available at: https://arxiv.org/abs/1609.04802.Ledig C., Theis L., Huszar F., Caballero J., Cunningham A., Acosta A., Aitken A., Tejani A., Totz J., Wang Z., Sh W. Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network // arxiv. 2016. URL: https://arxiv.org/abs/1609.04802.Lim B., Son S., Kim H., Nah S., Mu Lee K. Enhanced Deep Residual Networks for Single Image Super-Resolution, arxiv, 2017, available at: https://arxiv.org/abs/1707.02921.Lim B., Son S., Kim H., Nah S., Mu Lee K. Enhanced Deep Residual Networks for Single Image Super-Resolution // arxiv. 2017. URL: https://arxiv.org/abs/1707.02921.Simonyan K., Zisserman A. Very Deep Convolutional Networks for Large-Scale Image Recognition, arxiv, 2015, available at: https://arxiv.org/abs/1409.1556.Simonyan K., Zisserman A. Very Deep Convolutional Networks for Large-Scale Image Recognition // arxiv. 2015. URL: https://arxiv.org/abs/1409.1556.Huang R., Zhang S., Li T., He R. (2017). Beyond Face Rotation: Global and Local Perception GAN for Photorealistic and Identity Preserving Frontal View Synthesis, arxiv, 2017, available at: https://arxiv.org/abs/1704.04086.Huang R., Zhang S., Li T., He R. Beyond Face Rotation: Global and Local Perception GAN for Photorealistic and Identity Preserving Frontal View Synthesis // arxiv. 2017. URL: https:// arxiv.org/abs/1704.04086.Celebrities Images, available at: https://www.kaggle.com/greg115/celebrities-100k.Celebrities Images. URL: https://www.kaggle.com/greg115/celebrities-100k.Bulat A., Tzimiropoulos G. How far are we from solving the 2D & 3D Face Alignment problem? (and adataset of 230,000 3D facial landmarks), arxiv, 2017, available at: https://arxiv.org/abs/1703.07332.Bulat A., Tzimiropoulos G. How far are we from solving the 2D & 3D Face Alignment problem? (and adataset of 230,000 3D facial landmarks) // arxiv. 2017. URL: https://arxiv.org/abs/1703.07332.Aitken A., Ledig C., Theis L., Caballero J., Wang Z., Shi W. Checkerboard artifact free sub-pixel convolution: A note on sub-pixel convolution, resize convolution and convolution resize, arxiv, 2017, available at: https://arxiv.org/abs/1707.02937.Aitken A., Ledig C., Theis L., Caballero J., Wang Z., Shi W. Checkerboard artifact free sub-pixel convolution: A note on sub-pixel convolution, resize convolution and convolution resize // arxiv. 2017. URL: https://arxiv.org/abs/1707.02937.Antoniou A., Storkey A., Edwards H. Data Augmentation Generative Adversarial Networks, arxiv, 2017, available at: https://arxiv.org/abs/1711.04340.Antoniou A., Storkey A., Edwards H. Data Augmentation Generative Adversarial Networks // arxiv. 2017. URL: https:// arxiv.org/abs/1711.04340.Reed S., Akata Z., Yan X., Logeswaran L., Schiele B., Lee H. Generative Adversarial Text to Image Synthesis, arxiv, 2016, available at: https://arxiv.org/abs/1605.05396.Reed S., Akata Z., Yan X., Logeswaran L., Schiele B., Lee H. Generative Adversarial Text to Image Synthesis // arxiv. 2016. URL: https://arxiv.org/abs/1605.05396.Zhu J. Y., Park T., Isola P., Efros A. A. Unpaired Imageto-Image Translation using Cycle-Consistent Adversarial Networks, arxiv, 2017, available at: https://arxiv.org/abs/1703.10593.Zhu J. Y., Park T., Isola P., Efros A. A. Unpaired Imageto-Image Translation using Cycle-Consistent Adversarial Networks // arxiv. 2017. URL: https://arxiv.org/abs/1703.10593.Kim T., Cha M., Kim H., Lee J. K., Kim J. Learning to Discover Cross-Domain Relations with Generative Adversarial Networks, arxiv, 2017, available at: https://arxiv.org/abs/1703.05192.Kim T., Cha M., Kim H., Lee J. K., Kim J. Learning to Discover Cross-Domain Relations with Generative Adversarial Networks // arxiv. 2017. URL: https://arxiv.org/abs/1703.05192.Yi Z., Zhang H., Tan P., Gong M. DualGAN: Unsupervised Dual Learning for Image-to-Image Translation, arxiv, 2017, available at: https://arxiv.org/abs/1704.02510.Yi Z., Zhang H., Tan P., Gong M. DualGAN: Unsupervised Dual Learning for Image-to-Image Translation // arxiv. 2017. URL: https://arxiv.org/abs/1704.02510.Agustsson E., Timofte R. NTIRE 2017 Challenge on Single Image Super-Resolution: Dataset and Study, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2017, pp. 126—135.Agustsson E., Timofte R. NTIRE 2017 Challenge on Single Image Super-Resolution: Dataset and Study // 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). 2017. P. 126—135.Timofte R., Agustsson E., Gool L. V., Yang M. H., Zhang L., Lim B. and et al. NTIRE 2017 Challenge on Single Image Super-Resolution: Methods and Results, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2017, pp. 114—125.Timofte R., Agustsson E., Gool L. V., Yang M. H., Zhang L., Lim B. et al. NTIRE 2017 Challenge on Single Image Super-Resolution: Methods and Results // 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). 2017. P. 114—125.