Systems and Means of Informatics
2020, Volume 30, Issue 4, pp 159-167
PARABOLIC INTEGRODIFFERENTIAL SPLINES AS ACTIVATION FUNCTIONS TO INCREASE THE EFFICIENCY OF INFORMATION PROCESSING BY NEURAL NETWORKS
Abstract
The paper considers the method to increase the efficiency of information processing by neural networks by using the parabolic integrodifferential splines (ID-splines) developed by the author as an activation function (AF) for neurons. If the coefficients of parabolic ID-splines along with the weights of the neurons are the trainable parameters of the neural network, then the AF in the form of a parabolic ID spline changes in the learning process to minimize the error function. This increases the accuracy of the results of the neural network calculations and accelerates its training and operation. The prospects for modifying neural networks with known architectures (such as ResNet) by introducing ID-spline as AF are analyzed. Apparently, such an approach can improve the quality of functioning of some popular neural networks. It is concluded that parabolic ID splines as AF can increase the efficiency of artificial intelligence technologies in such tasks as decision making, computer games development, approximating and predicting data (in the financial and social spheres, in science, etc.), classification of information, processing of images and videos, application of computer vision, processing of texts, speech, and music, etc.
[+] References (6)
- Zatsarinnyy, A. A., E.V. Kiselev, S.V. Kozlov, and K. K. Kolin. 2018. Informa- tsionnoe prostranstvo tsifrovoy ekonomiki Rossii. Kontseptual'nye osnovy i problemy formirovaniya [Information space of the digital economy of Russia. Conceptual foundations and problems of formation]. Moscow: FRC CSC RAS. 236 p.
- Kireev, V.I., and T. K. Biryukova. 2014. Integrodifferentsial'nyy metod obrabotki informatsii i ego primenenie v chislennom analize [Integrodifferential method of in-formation processing and its application in numerical analysis]. Moscow: IPI RAN. 267 p.
- Stechkin, S.B., and Yu.N. Subbotin. 1976. Splayny v vychislitel'noy matematike [Splines in computational mathematics]. Moscow: Nauka. 248 p.
- Campolucci, P., F. Capperelli, S. Guarnieri, F. Piazza, and A. Uncini. 1996. Neural networks with adaptive spline activation function. 8th Mediterranean Electrotechnical Conference on Industrial Applications in Power Systems, Computer Science and Telecommunications Proceedings. IEEE. 1442-1445.
- Mayer, H. A., and R. Schwaiger. 2001. Evolution of cubic spline activation functions for artificial neural networks. Progress in artificial intelligence, knowledge extraction, multi-agent systems, logic programming and constraint solving. Eds. P. Brazdil and
A. Jorge. Lecture notes in computer science ser. Springer. 2258:63-73.
- Scardapane, S., M. Scarpiniti, D. Comminiello, and A. Uncini. 2017. Learning activation functions from data using cubic spline interpolation. Neural advances in processing nonlinear dynamics signals: Learning activation functions from data using cubic spline interpolation. Eds. A. Esposito, M. Faundez-Zanuy, F. Morabito, and E. Pasero. Smart innovation, systems and technologies ser. Springer. 102:73-83.
[+] About this article
Title
PARABOLIC INTEGRODIFFERENTIAL SPLINES AS ACTIVATION FUNCTIONS TO INCREASE THE EFFICIENCY OF INFORMATION PROCESSING BY NEURAL NETWORKS
Journal
Systems and Means of Informatics
Volume 30, Issue 4, pp 159-167
Cover Date
2020-12-10
DOI
10.14357/08696527200415
Print ISSN
0869-6527
Publisher
Institute of Informatics Problems, Russian Academy of Sciences
Additional Links
Key words
àrtificial intelligence; deep learning; neural network; activation function; spline interpolation; integrodifferential spline; parabolic spline
Authors
T. K. Biryukova 
Author Affiliations
Institute of Informatics Problems, Federal Research Center "Computer Science
and Control", Russian Academy of Sciences, 44-2 Vavilov Str., Moscow 119333, Russian Federation
|