Tang, Z., Chen, G., Wang, R., Miao, Z., Dai, M., Ma, Y., & Liao, X. (2023). CD3IS: cross dimensional 3D instance segmentation network for production workshop. Journal of Intelligent Manufacturing. https://doi.org/10.1007/s10845-023-02200-6
Three-dimensional (3D) instance segmentation, as an effective method for scene object recognition, can effectively enhance workshop intelligence. However, the existing 3D instance segmentation network is difficult to apply in workshop scenes due to the large number of 3D instance segmentation labels and 3D information acquisition devices required. In this paper, a monocular 3D instance segmentation network is proposed, which achieves satisfactory results while relying on a monocular RGB camera without 3D instance segmentation labels. The proposed method has two stages. In the first stage, the double-snake multitask network is proposed to solve the problem of a lack of 3D information acquisition devices. It simultaneously performs depth estimation and instance segmentation and uses the features obtained from depth estimation to guide the instance segmentation task. In the second stage, an adaptive point cloud filtering algorithm that performs adaptive point cloud noise filtering on multi-scale objects based on two-dimensional prior information is proposed to solve the problem of a lack of 3D labels. In addition, color information is introduced into the filtering process to further improve filtering accuracy. Experiments on the Cityscapes and SOP datasets demonstrate the competitive performance of the proposed method. When the IoU threshold is set to 0.35, the mean average precision (mAP) is 50.41. Our approach is deployed in an actual production workshop to verify its feasibility.
This work was supported in part by the Sichuan Province Foreign and Overseas High-end Talent Introduction Program (under Grant 22RCYJ0024), in part by the Sichuan Province Science and Technology Support Program (under Grant 2022YFG0198), and in part by the Sichuan International Hong Kong, Macao, and Taiwan Science and Technology Innovation Cooperation Program (under Grant 2023YFH0029)