Vestn. YuUrGU. Ser. Vych. Matem. Inform., 2017, Volume 6, Issue 3, Pages 28–59
Computer Science, Engineering and Control
An overview of methods for deep learning in neural networks
A. V. Sozykinab
a N.N. Krasovskii Institute of Mathematics and Mechanics (S. Kovalevskaya str. 16, Yekaterinburg, 620990 Russia)
b Ural Federal University (Mira str. 19, Yekaterinburg, 620002 Russia)
At present, deep learning is becoming one of the most popular approach to creation of the artificial intelligences systems such as speech recognition, natural language processing, computer vision and so on. Thepaper presents a historical overview of deep learning in neural networks. The model of the artificial neural networkis described as well as the learning algorithms for neural networks including the error backpropagation algorithm, which is used to train deep neural networks. The development of neural networks architectures is presentedincluding neocognitron, autoencoders, convolutional neural networks, restricted Boltzmann machine, deep beliefnetworks, long short-term memory, gated recurrent neural networks, and residual networks. Training deep neuralnetworks with many hidden layers is impeded by the vanishing gradient problem. The paper describes theapproaches to solve this problem that provide the ability to train neural networks with more than hundred layers.An overview of popular deep learning libraries is presented. Nowadays, for computer vision tasks convolutionalneural networks are utilized, while for sequence processing, including natural language processing, recurrentnetworks are preferred solution, primarily long short-term memory networks and gated recurrent neural networks.
deep learning, neural networks, machine learning.
PDF file (896 kB)
A. V. Sozykin, “An overview of methods for deep learning in neural networks”, Vestn. YuUrGU. Ser. Vych. Matem. Inform., 6:3 (2017), 28–59
Citation in format AMSBIB
\paper An overview of methods for deep learning in neural networks
\jour Vestn. YuUrGU. Ser. Vych. Matem. Inform.
Citing articles on Google Scholar:
Related articles on Google Scholar:
|Number of views:|