Abstract
Subjective video quality assessment (VQA) is the most reliable way to get accurate quality scores, providing first-hand data on the research of quality of experience (QoE). However, traditional subjective VQA has the disadvantage of being cumbersome and time-consuming. In this paper, we propose an efficient subjective VQA framework based on hybrid information and active learning, named HA-SVQA. Built on the principle of active learning for data annotation, HA-SVQA allows iterative assessments on the most valuable or informative videos, which are selected based on hybrid information from the subject's prior decisions and the objective quality predictions. By eliminating the redundant (or less valuable) videos to be assessed, HA-SVQA can speed up the process of subjective VQA. Concretely, our framework starts with a few quality-known videos to initialize dual-regression models. It then uses a scoring-difference stratified sampling strategy to iteratively select the next group of videos to be assessed, which contain high quality uncertainty. The newly scored videos are used to continually update every part of our framework. By this way, subjective VQA can be stopped early while meeting the acceptable goal of a full-time subjective study. We conducted simulation experiments on three different datasets: LIVE, LIVE VQC, and an underwater video quality dataset. The results show that HA-SVQA can effectively speed up the process of subjective VQA and reduce about 1/3 of the human workload or time cost in the presence of data redundancy. In order to investigate the effectiveness of HA-SVQA in more depth, we conducted a field experiment with deep-sea videos. We found that HA-SVQA is still effective in reducing about 1/3 of the overall human workload, which is consistent with the conclusion of simulation experiments. Finally, we discussed some of the factors that potentially affect the QoE modelling and the subjective VQA.