Email this article 
Application of integer tables for quantisation of activation functions of neural networks
- Authors: Vasilev А.А.1, Kapitanov А.I.2
-
Affiliations:
- LLC "Alphachip"
- SPINTech Institute, National Research University "MIET"
- Issue: Vol 31, No 7 (2025)
- Pages: 364-369
- Section: Neural network technologies
- Published: 15.07.2025
- URL: https://journals.eco-vector.com/1684-6400/article/view/702282
- DOI: https://doi.org/10.17587/it.31.364-369
- ID: 702282
Cite item
Open Access
Access granted
Subscription or Fee Access
Abstract
The paper considers the problem of efficient hardware implementation of nonlinear activation functions of neural networks under low-bit computing conditions. Standard activations, such as sigmoid and hyperbolic tangent, require resource-intensive floating-point operations, which limits their use on microcontrollers, FPGAs and other peripheral platforms. As a solution, an approach based on precomputed integer substitution tables (LUTs) is proposed to reduce computational complexity and power consumption. Using the example of the SiLU activation function widely used in popular object detection networks (e.g., YOLO), the quantisation procedure is demonstrated, the principles of constructing and using LUTs are formulated, and a practical algorithm for computing activations using them is described.
About the authors
А. А. Vasilev
LLC "Alphachip"
Author for correspondence.
Email: artvasilev@alphachip.ru
Middle Engineer
Russian Federation, MoscowА. I. Kapitanov
SPINTech Institute, National Research University "MIET"
Email: andrey@kapdx.ru
Associate Professor
Russian Federation, MoscowReferences
- Romanov A. Y., Stempkovsky A. L., Lariushkin I. V., Novoselov G. E., Solovyev R. A., Starykh V. A., Romanova I. I., Telpukhov D. V., Mkrtchan I. A. Analysis of posit and bfloat arithmetic of real numbers for machine learning // IEEE Access. 2021. N. 9. P. 82318—82324.
- Zhao X., Wang Y., Cai X., Liu C., Zhang L. Linear symmetric quantization of neural networks for low-precision integer hardware //International Conference on Learning Representations. 2020, April.
- Krishnamoorthi R. Quantizing deep convolutional networks for efficient inference: A whitepaper // arXiv preprint arXiv:1806.08342. 2018.
- Solovyev R. A., Kalinin A. A., Kustov A. G., Telpukhov D. V., Ruhlov V. S. FPGA implementation of convolutional neural networks with fixed-point calculations // CoRR. 2018. [Online]. Available: https://openreview.net/forum?id=ktBJlHahoT.
- Solovyev R. A., Telpukhov D. V., Romanova I. I., Kustov A. G., Mkrtchan I. A. Real-time Object Detection with FPGA Using CenterNet // 2021 IEEE Conference of Russian Young Researchers in Electrical and Electronic Engineering (ElConRus). 2021, January. P. 2029—2034.
- Nagel M., Fournarakis M., Amjad R. A., Bondarenko Y., Van Baalen M., Blankevoort T. A white paper on neural network quantization // arXiv preprint arXiv:2106.08295. 2021.
- Yang J., Shen X., Xing J., Tian X., Li H., Deng B., Huang J., Hua X. S. Quantization networks //Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2019. P. 7308—7316.
- Gholami A., Kim S., Dong Z., Yao Z., Mahoney M. W., Keutzer K. A survey of quantization methods for efficient neural network inference // Low-power computer vision. Chapman and Hall/CRC. 2022. P. 291—326.
- Hubara I., Courbariaux M., Soudry D., El-Yaniv R., Bengio Y. Quantized neural networks: Training neural networks with low precision weights and activations // Journal of machine learning research. 2018. Vol. 18, N. 187. P. 1—30.
- Hagiescu A., Langhammer M., Pasca B., Colangelo P., Thong J., Ilkhani N. Bfloat MLP training accelerator for FPGAs // 2019 International Conference on ReConFigurable Computing and FPGAs (ReConFig). 2019, December. P. 1—5.
- Zhai H. Z., Du J. W., Ai Y. H., Hu T. J. Edge Deployment of Deep Networks for Visual Object Detection: A Review // IEEE Sensors Journal. 2024. [Online]. Available: https://ieeexplore.ieee. org/abstract/document/10786287
Supplementary files
