Hou, J., Ding, H., Lin, W., Liu, W., & Fang, Y. (2022). Distilling Knowledge from Object Classification to Aesthetics Assessment. IEEE Transactions on Circuits and Systems for Video Technology, 1–1. https://doi.org/10.1109/tcsvt.2022.3186307
Abstract:
In this work, we point out that the major dilemma
of image aesthetics assessment (IAA) comes from the abstract
nature of aesthetic labels. That is, a vast variety of distinct
contents can correspond to the same aesthetic label. On the
one hand, during inference, the IAA model is required to relate
various distinct contents to the same aesthetic label. On the other
hand, when training, it would be hard for the IAA model to learn
to distinguish different contents merely with the supervision from
aesthetic labels, since aesthetic labels are not directly related to
any specific content. To deal with this dilemma, we propose to
distill knowledge on semantic patterns for a vast variety of image
contents from multiple pre-trained object classification (POC)
models to an IAA model. Expecting the combination of multiple
POC models can provide sufficient knowledge on various image
contents, the IAA model can easier learn to relate various distinct
contents to a limited number of aesthetic labels. By supervising
an end-to-end single-backbone IAA model with the distilled
knowledge, the performance of the IAA model is significantly
improved by 4.8% in SRCC compared to the version trained
only with ground-truth aesthetic labels. On specific categories of
images, the SRCC improvement brought by the proposed method
can achieve up to 7.2%. Peer comparison also shows that our
method outperforms 10 previous IAA methods.
License type:
Publisher Copyright
Funding Info:
This research / project is supported by the Ministry of Education - Tier-1 Fund MOE2021
Grant Reference no. : RG14/21