SegEQA: Video Segmentation Based Visual Attention for Embodied Question Answering

Page view(s)
26
Checked on Apr 08, 2024
SegEQA: Video Segmentation Based Visual Attention for Embodied Question Answering
Title:
SegEQA: Video Segmentation Based Visual Attention for Embodied Question Answering
Journal Title:
ICCV 2019
DOI:
Publication Date:
03 November 2019
Citation:
Abstract:
Embodied Question Answering (EQA) is a newly defined research area where an agent is required to answer the user's questions by exploring the real world environment. It has attracted increasing research interests due to its broad applications in automatic driving system, in-home robots, and personal assistants. Most of the existing methods perform poorly in terms of answering and navigation accuracy due to the absence of local details and vulnerability to the ambiguity caused by complicated vision conditions. To tackle these problems, we propose a segmentation based visual attention mechanism for Embodied Question Answering. Firstly, We extract the local semantic features by introducing a novel high-speed video segmentation framework. Then by the guide of extracted semantic features, a bottom-up visual attention mechanism is proposed for the Visual Question Answering (VQA) sub-task. Further, a feature fusion strategy is proposed to guide the training of the navigator without much additional computational cost. The ablation experiments show that our method boosts the performance of VQA module by 4.2% (68.99% vs 64.73%) and leads to 3.6% (48.59% vs 44.98%) overall improvement in EQA accuracy.
License type:
PublisherCopyrights
Funding Info:
Description:
ISBN:

Files uploaded:
File Size Format Action
There are no attached files.