Wang, Y., Wu, A., Yang, M., Min, Y., Zhu, Y., & Deng, C. (2025). Reasoning Mamba: Hypergraph-Guided Region Relation Calculating for Weakly Supervised Affordance Grounding. In (Editor), 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). https://doi.org/10.1109/cvpr52734.2025.02572
Abstract:
This paper pays attention to Weakly Supervised Affordance Grounding (WSAG) task that aims to train model to identify affordance regions using human-object interaction images and egocentric images without the need for costly pixel-level annotations. Most existing methods usually consider the affordance regions to be isolated and directly employ class activation maps to conduct localization, ignoring the relationships with other object components and weakening the performance. For example, a cup’s handle is combined with its body to achieve the pouring ability. Obviously, capturing the region relationships is beneficial for improving the localization accuracy of affordance regions. To this end, we first explore exploiting hypergraph to discover these relations and propose a Reasoning Mamba (R-Mamba) framework. We first extract feature embeddings from exocentric and egocentric images to construct the hypergraphs consisting of multiple vertices and hyperedges, which capture the in-context local region relationships between different visual components. Subsequently, we design a Hypergraph-Guided State Space (HSS) block to reorganize these local relationships from the global perspective. By this mechanism, the model could leverage the captured relationships to improve the localization accuracy of affordance regions. Extensive experiments and visualization analyses demonstrate the superiority of our method.
License type:
Publisher Copyright
Funding Info:
This research / project is supported by the National Natural Science Foundation of China - NA
Grant Reference no. : 62132016, 62472333
This research / project is supported by the Ministry of Education of China - Joint Fund of Ministry of Education of China
Grant Reference no. : 8091B022149, 8091B02072404
This research / project is supported by the Department of Science and Technology of Shaanxi Province - Natural Science Basic Research Program of Shaanxi
Grant Reference no. : 2020JC-23