On the effectiveness of using visual transformers in detecting abnormalities of histopathological images

Breast cancer is one of the most common and dangerous types of cancer in women. Modern approaches to its early detection and treatment increasingly use methods of artificial intelligence and deep learning. In this paper we investigate breast histopathological images using transformer neural networks and deep neural networks. Vision Transformer (ViT), Data Efficient Image Transformer (DeiT) models were used as transformer models. ResNet50, VGG16, DenseNet201, EfficientNet-B7, Xception were chosen as models of deep convolutional neural networks. All models were pre-trained on the ImageNet1k image set and then further trained on a set of 4356 histopathological images. The results of computer experiments showed that the EfficientNet-B7 model outperformed the other models, achieving an accuracy rate of 95.84%. In order to improve the performance of histopathological image classification models, a knowledge distillation method was applied in this work, which allowed us to obtain a new deep neural network model DeiT_B_dist for high-precision classification of histopathological images and breast cancer detection. Its accuracy rate was 98.14% and outperformed the other models and is comparable to the results of other researchers.