An End-to-End Network for Generating Social Relationship Graphs

An End-to-End Network for Generating Social Relationship Graphs
Title:
An End-to-End Network for Generating Social Relationship Graphs
Other Titles:
2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Publication Date:
15 June 2019
Citation:
A. Goel, K. T. Ma and C. Tan, "An End-To-End Network for Generating Social Relationship Graphs," 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, 2019, pp. 11178-11187, doi: 10.1109/CVPR.2019.01144.
Abstract:
Socially-intelligent agents are of growing interest in artificial intelligence. To this end, we need systems that can understand social relationships in diverse social contexts. Inferring the social context in a given visual scene not only involves recognizing objects, but also demands a more in-depth understanding of the relationships and attributes of the people involved. To achieve this, one computational approach for representing human relationships and attributes is to use an explicit knowledge graph, which allows for high-level reasoning. We introduce a novel end-to-end-trainable neural network that is capable of generating a Social Relationship Graph - a structured, unified representation of social relationships and attributes - from a given input image. Our Social Relationship Graph Generation Network (SRG-GN) is the first to use memory cells like Gated Recurrent Units (GRUs) to iteratively update the social relationship states in a graph using scene and attribute context. The neural network exploits the recurrent connections among the GRUs to implement message passing between nodes and edges in the graph, and results in significant improvement over previous methods for social relationship recognition.
License type:
PublisherCopyrights
Funding Info:
This work was supported by NRF grant no. NRF2015-NRFISF001-2541 (KTM and CT) and A*STAR SERC SSF grant no. A1718g0048 (AG and KTM).
Description:
© 2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
ISSN:
2575-7075
1063-6919
ISBN:
978-1-7281-3293-8
978-1-7281-3294-5
Files uploaded: