A Review of Methodologies for Conducting Subjective Comparisons in Crowdsourcing

The report will examine methods for conducting subjective comparisons with an emphasis on the uncontrolled environment of the assessor in crowdsourcing, and will discuss existing approaches to vote aggregation and ways to optimize the required number of participants in experiments.
Link for the online translation: https://telemost.yandex.ru/j/63389006341822.