Dense Supervision Propagation for Weakly Supervised Semantic Segmentation on 3D Point Clouds

Dense Supervision Propagation for Weakly Supervised Semantic Segmentation on 3D Point Clouds

IEEE Transactions on Circuits and Systems for Video Technology

10.1109/TCSVT.2023.3336323

http://dx.doi.org/10.1109/tcsvt.2023.3336323

Jiacheng Wei, Guosheng Lin, Kim-Hui Yap, Fayao Liu, Tzu-Yi Hung

Electrical and Electronic Engineering, Media Technology

23 November 2023

Wei, J., Lin, G., Yap, K.-H., Liu, F., & Hung, T.-Y. (2024). Dense Supervision Propagation for Weakly Supervised Semantic Segmentation on 3D Point Clouds. IEEE Transactions on Circuits and Systems for Video Technology, 1–1. https://doi.org/10.1109/tcsvt.2023.3336323

Semantic segmentation on 3D point clouds is an important task for 3D scene understanding. While dense labeling on 3D data is expensive and time-consuming, only a few works address weakly supervised semantic point cloud segmentation methods to relieve the labeling cost by learning from simpler and cheaper labels. Meanwhile, there are still huge performance gaps between existing weakly supervised methods and state- of-the-art fully supervised methods. In this paper, we propose Dense Supervision Propagation (DSP) to train a semantic point cloud segmentation network with only a small portion of points being labeled. We argue that we can better utilize the limited supervision information as we densely propagate the supervision signal from the labeled points to other points within and across the input samples. Specifically, we propose a cross-sample feature reallocating module to transfer similar features and therefore re-route the gradients across two samples with common classes and an intra-sample feature redistribution module to propagate supervision signals on unlabeled points across and within point cloud samples. We conduct extensive experiments on public datasets S3DIS and ScanNet. Our weakly supervised method with only 10% and 1% of labels can produce competitive results with the fully supervised counterpart.

Publisher Copyright

This research / project is supported by the Agency for Science, Tech- nology and Research (A*STAR) - MTC Programmatic Funds
Grant Reference no. : M23L7b0021

This research / project is supported by the Ministry of Education - AcRF Tier 2 grant
Grant Reference no. : MOE-T2EP20220-0007

This research / project is supported by the Ministry of Education - AcRF Tier 1 grant
Grant Reference no. : RG14/22

This research / project is supported by the A*STAR - Career Development Award (CDA)
Grant Reference no. : 202D8243

© 2023 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.

https://oar.a-star.edu.sg/communities-collections/articles/19701

1051-8215
1558-2205

Institute for Infocomm Research