Compensation for nonlinear distortions in optical communication systems using perturbation theory and multiparameter optimization

Nonlinear signal distortions are one of the main reasons limiting the throughput and length of modern fiber-optic communication lines. One of the developed approaches to nonlinear distortion compensation is based on the application of perturbation theory methods to the nonlinear Schrödinger equation, and allows one to obtain a relationship between transmitted and received symbols. Gradient methods that minimize the standard deviation between symbols are usually used to find the perturbation coefficients. However, the main parameter characterizing the data transmission quality is the bit error rate. We propose a modification of this approach in the form of a two-stage scheme for calculating the perturbation coefficients. At the first stage, the coefficients are calculated using the least squares method by minimizing the standard deviation, and at the second stage, the obtained solution is used as an initial approximation to minimize the error coefficient using the particle swarm method. In a numerical experiment, using the algorithm for compensating for received signal distortions based on the proposed scheme, a 0.9 dB gain in the signal-to-noise ratio is obtained for a multi-span line 20 × 100 km long and a 16QAM signal with a channel rate of 267 Gbit/s. An improvement in the accuracy of the algorithm (compared to a single-stage scheme) is shown, estimates of the computational complexity of the algorithm are obtained, and the relationship between its complexity and accuracy is presented.