Vestnik Yuzhno-Ural'skogo Gosudarstvennogo Universiteta. Seriya "Vychislitelnaya Matematika i Informatika"
RUS  ENG    JOURNALS   PEOPLE   ORGANISATIONS   CONFERENCES   SEMINARS   VIDEO LIBRARY   PACKAGE AMSBIB  
General information
Latest issue
Archive

Search papers
Search references

RSS
Latest issue
Current issues
Archive issues
What is RSS



Vestn. YuUrGU. Ser. Vych. Matem. Inform.:
Year:
Volume:
Issue:
Page:
Find






Personal entry:
Login:
Password:
Save password
Enter
Forgotten password?
Register


Vestn. YuUrGU. Ser. Vych. Matem. Inform., 2017, Volume 6, Issue 3, Pages 28–59 (Mi vyurv170)  

Computer Science, Engineering and Control

An overview of methods for deep learning in neural networks

A. V. Sozykinab

a N.N. Krasovskii Institute of Mathematics and Mechanics (S. Kovalevskaya str. 16, Yekaterinburg, 620990 Russia)
b Ural Federal University (Mira str. 19, Yekaterinburg, 620002 Russia)

Abstract: At present, deep learning is becoming one of the most popular approach to creation of the artificial intelligences systems such as speech recognition, natural language processing, computer vision and so on. Thepaper presents a historical overview of deep learning in neural networks. The model of the artificial neural networkis described as well as the learning algorithms for neural networks including the error backpropagation algorithm, which is used to train deep neural networks. The development of neural networks architectures is presentedincluding neocognitron, autoencoders, convolutional neural networks, restricted Boltzmann machine, deep beliefnetworks, long short-term memory, gated recurrent neural networks, and residual networks. Training deep neuralnetworks with many hidden layers is impeded by the vanishing gradient problem. The paper describes theapproaches to solve this problem that provide the ability to train neural networks with more than hundred layers.An overview of popular deep learning libraries is presented. Nowadays, for computer vision tasks convolutionalneural networks are utilized, while for sequence processing, including natural language processing, recurrentnetworks are preferred solution, primarily long short-term memory networks and gated recurrent neural networks.

Keywords: deep learning, neural networks, machine learning.

Funding Agency Grant Number
Ural Branch of the Russian Academy of Sciences 15-7-1-8


DOI: https://doi.org/10.14529/cmse170303

Full text: PDF file (896 kB)
References: PDF file   HTML file

UDC: 004.85
Received: 12.04.2017

Citation: A. V. Sozykin, “An overview of methods for deep learning in neural networks”, Vestn. YuUrGU. Ser. Vych. Matem. Inform., 6:3 (2017), 28–59

Citation in format AMSBIB
\Bibitem{Soz17}
\by A.~V.~Sozykin
\paper An overview of methods for deep learning in neural networks
\jour Vestn. YuUrGU. Ser. Vych. Matem. Inform.
\yr 2017
\vol 6
\issue 3
\pages 28--59
\mathnet{http://mi.mathnet.ru/vyurv170}
\crossref{https://doi.org/10.14529/cmse170303}
\elib{https://elibrary.ru/item.asp?id=30016527}


Linking options:
  • http://mi.mathnet.ru/eng/vyurv170
  • http://mi.mathnet.ru/eng/vyurv/v6/i3/p28

    SHARE: VKontakte.ru FaceBook Twitter Mail.ru Livejournal Memori.ru


    Citing articles on Google Scholar: Russian citations, English citations
    Related articles on Google Scholar: Russian articles, English articles
  • Vestnik Yuzhno-Ural'skogo Gosudarstvennogo Universiteta. Seriya "Vychislitelnaya Matematika i Informatika"
    Number of views:
    This page:1157
    Full text:1521
    References:7

     
    Contact us:
     Terms of Use  Registration to the website  Logotypes © Steklov Mathematical Institute RAS, 2021