Point 4D Transformer Networks for Spatio-Temporal Modeling in Point Cloud Videos

Page view(s)
12
Checked on Dec 05, 2024
Point 4D Transformer Networks for Spatio-Temporal Modeling in Point Cloud Videos
Title:
Point 4D Transformer Networks for Spatio-Temporal Modeling in Point Cloud Videos
Journal Title:
2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Keywords:
Publication Date:
02 November 2021
Citation:
Fan, H., Yang, Y., & Kankanhalli, M. (2021, June). Point 4D Transformer Networks for Spatio-Temporal Modeling in Point Cloud Videos. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). https://doi.org/10.1109/cvpr46437.2021.01398
Abstract:
Point cloud videos exhibit irregularities and lack of order along the spatial dimension where points emerge inconsistently across different frames. To capture the dynamics in point cloud videos, point tracking is usually employed. However, as points may flow in and out across frames, computing accurate point trajectories is extremely difficult. Moreover, tracking usually relies on point colors and thus may fail to handle colorless point clouds. In this paper, to avoid point tracking, we propose a novel Point 4D Transformer (P4Transformer) network to model raw point cloud videos. Specifically, P4Transformer consists of (i) a point 4D convolution to embed the spatio-temporal local structures presented in a point cloud video and (ii) a transformer to capture the appearance and motion information across the entire video by performing self-attention on the embedded local features. In this fashion, related or similar local areas are merged with attention weight rather than by explicit tracking. Extensive experiments, including 3D action recognition and 4D semantic segmentation, on four benchmarks demonstrate the effectiveness of our P4Transformer for point cloud video modeling.
License type:
Publisher Copyright
Funding Info:
This research / project is supported by the Agency for Science, Technology and Research, Singapore - AME Programmatic Funding Scheme
Grant Reference no. : A18A2b0046
Description:
© 2021 IEEE.  Personal use of this material is permitted.  Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
ISSN:
10.1109/CVPR46437.2021.01398
Files uploaded: