Zhou, H., He, T., Ong, Y.-S., Cong, G., & Chen, Q. (2024). Differentiable Clustering for Graph Attention. IEEE Transactions on Knowledge and Data Engineering, 36(8), 3751–3764. https://doi.org/10.1109/tkde.2024.3363703
Abstract:
Graph clusters (or communities) represent important graph structural information. In this paper, we present Differentiable Clustering for graph ATtention (DCAT). To the best of our knowledge, DCAT is the first solution that incorporates graph clustering into graph attention networks (GAT) to learn cluster-aware attention scores for semi-supervised learning tasks. In DCAT, we propose a novel approach to formulating graph clustering as an auxiliary differentiable objective based on modularity maximization, which can be optimized together with the learning objective of GAT for a semi-supervised task. Specifically, we propose a solution to relaxing modularity maximization from a discrete optimization problem to a differentiable objective with theoretical guarantee so that we can learn cluster-aware attention scores by jointly learning from graph clustering and a semi-supervised learning task. To address the computational challenge, we further propose to reformulate the constraint introduced by the clustering objective into a new form. Our analysis shows that DCAT allocates higher attention scores to nodes within the same cluster, allowing them to have a higher influence in node representation learning, and thus DCAT will generate better node representations for downstream applications. The experimental results on commonly used datasets show that DCAT outperforms popular and state-of-the-art graph neural networks.
License type:
Publisher Copyright
Funding Info:
This work was supported in part by A*STAR under the RIE2020 Industry Alignment Fund – Industry Collaboration Projects (IAF-ICP) Funding Initiative, as well as cash and in-kind contribution from Singapore Telecommunications Limited (Singtel), through Singtel Cognitive and Artificial Intelligence Lab for Enterprises (SCALE@NTU), and in part by A*STAR I&E GAP Funding under Grant I23D1AG080.