Distributed systems for machine learning

Machine learning is a rapidly developing area of research. Many machine learning and data science applications face very large datasets. These datasets are hard to process on a single computer or this processing will be very time consuming. Using only a subsample of a dataset typically leads to the worse quality of model’s predictions. Distributed computational systems are used to solve this problem. The most popular approaches for developing software of such systems include the following computational models: Map/Reduce, Spark, graph computational models and parameter server architecture. Current paper is a review of such systems with analysis of their advantages and disadvantages regarding to machine learning applications. Systems for training artificial neural networks are discussed separately.