On-the-fly Point Feature Representation for Point Clouds Analysis

On-the-fly Point Feature Representation for Point Clouds Analysis

Proceedings of the 32nd ACM International Conference on Multimedia

10.1145/3664647.3680700

http://dx.doi.org/10.1145/3664647.3680700

Jiangyi Wang, Zhongyao Cheng, Na Zhao, Jun Cheng, Xulei Yang

28 October 2024

Wang, J., Cheng, Z., Zhao, N., Cheng, J., & Yang, X. (2024). On-the-fly Point Feature Representation for Point Clouds Analysis. Proceedings of the 32nd ACM International Conference on Multimedia, 9204–9213. https://doi.org/10.1145/3664647.3680700

Point cloud analysis is challenging due to its unique characteristics of unorderness, sparsity and irregularity. Prior works attempt to capture local relationships by convolution operations or attention mechanisms, exploiting geometric information from coordinates implicitly. These methods, however, are insufficient to describe the explicit local geometry, e.g., curvature and orientation. In this paper, we propose On-the-fly Point Feature Representation (OPFR), which captures abundant geometric information explicitly through Curve Feature Generator module. This is inspired by Point Feature Histogram (PFH) from computer vision community. However, the utilization of vanilla PFH encounters great difficulties when applied to large datasets and dense point clouds, as it demands considerable time for feature generation. In contrast, we introduce the Local Reference Constructor module, which approximates the local coordinate systems based on triangle sets. Owing to this, our OPFR only requires extra 1.56ms for inference (65× faster than vanilla PFH) and 0.012M more parameters, and it can serve as a versatile plug-and-play module for various backbones, particularly MLP-based and Transformer-based backbones examined in this study. Additionally, we introduce the novel Hierarchical Sampling module aimed at enhancing the quality of triangle sets, thereby ensuring robustness of the obtained geometric features. Our proposed method improves overall accuracy (OA) on ModelNet40 from 90.7% to 94.5% (+3.8%) for classification, and OA on S3DIS Area-5 from 86.4% to 90.0% (+3.6%) for semantic segmentation, respectively, building upon PointNet++ backbone. When integrated with Point Transformer backbone, we achieve state-of-the-art results on both tasks: 94.8% OA on ModelNet40 and 91.7% OA on S3DIS Area-5.

Attribution-NonCommercial 4.0 International (CC BY-NC 4.0)

This research / project is supported by the Agency for Science, Technology and Research (A*STAR) - Manufacturing, Trade, and Connectivity Programmatic Funds
Grant Reference no. : M23L7b0021

© 2024 Copyright held by the owner/author(s). Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.

https://oar.a-star.edu.sg/communities-collections/articles/21364

9798400706868

Institute for Infocomm Research