Huang, Z., Zhu, H., Sun, Y., Choi, D., Tan, C., Lim, J.-H. (2021). A Diagnostic Study Of Visual Question Answering With Analogical Reasoning. 2021 IEEE International Conference on Image Processing (ICIP). https://doi.org/10.1109/icip42928.2021.9506539
Abstract:
The deep learning community has made rapid progress in
low-level visual perception tasks such as object localization,
detection and segmentation. However, for tasks such as Visual Question Answering (VQA) and visual language grounding that require high-level reasoning abilities, huge gaps still
exist between artificial systems and human intelligence. In
this work, we perform a diagnostic study on recent popular
VQA in terms of analogical reasoning. We term it as Analogical VQA, where a system needs to reason on a group of images to find analogical relations among them in order to correctly answer a natural language question. To study the task
in depth, we propose an initial diagnostic synthetic dataset
CLEVR-Analogy, which tests a range of analogical reasoning abilities (e.g. reasoning on object attributes, spatial relationships, existence, and arithmetic analogies). We benchmark various recent state-of-the-art methods on our dataset
and compare the results against human performance, and discover that existing systems fall shorts when facing analogical reasoning involving spatial relationships. The dataset and
code will be publicly available to facilitate future research.
License type:
Publisher Copyright
Funding Info:
This research / project is supported by the A*STAR - AME Programmatic Fund
Grant Reference no. : A18A2b0046