Song,CY., Wei, JC., Li, RB., Liu, Lin, GS. 3D Pose Transfer with Correspondence Learning and Mesh Refinement. NeurIPS 2021
Abstract:
3D pose transfer is one of the most challenging 3D generation tasks. It aims to transfer the pose of a source mesh to a target mesh and keep the identity (e.g., body shape) of the target mesh. Some previous works require key point annotations to build reliable correspondence between the source and target meshes, while other methods do not consider any shape correspondence between sources and targets, which leads to limited generation quality. In this work, we propose a correspondence-refinement network to help the 3D pose transfer for both human and animal meshes. The correspondence between source and target meshes is first established by solving an optimal transport problem. Then, we warp the source mesh according to the dense correspondence and obtain a coarse warped mesh. The warped mesh will be better refined with our proposed Elastic Instance Normalization, which is a conditional normalization layer and can help to generate highquality meshes. Extensive experimental results show that the proposed architecture can effectively transfer the poses from source to target meshes and produce better results with satisfied visual performance than state-of-the-art methods. Our code and data are available at https://github.com/ChaoyueSong/3d-corenet.
License type:
Publisher Copyright
Funding Info:
This research / project is supported by the A*STAR - IAF-PP - RIE2020 Industry Alignment Fund
Grant Reference no. : I1901E0052
This research / project is supported by the National Research Foundation, Singapore - AI Singapore Programme
Grant Reference no. : AISG-RP-2018-003
This research / project is supported by the Ministry of Education - MOE Tier-1
Grant Reference no. : RG28/18 (S)
This research / project is supported by the Ministry of Education - MOE Tier-1
Grant Reference no. : RG95/20