Diverse and Stable 2D Diffusion Guided Text to 3D Generation with Noise Recalibration

Page view(s)
75
Checked on Sep 07, 2024
Diverse and Stable 2D Diffusion Guided Text to 3D Generation with Noise Recalibration
Title:
Diverse and Stable 2D Diffusion Guided Text to 3D Generation with Noise Recalibration
Journal Title:
Proceedings of the AAAI Conference on Artificial Intelligence
Publication Date:
25 March 2024
Citation:
Yang, X., Liu, F., Xu, Y., Su, H., Wu, Q., & Lin, G. (2024). Diverse and Stable 2D Diffusion Guided Text to 3D Generation with Noise Recalibration. Proceedings of the AAAI Conference on Artificial Intelligence, 38(7), 6549–6557. https://doi.org/10.1609/aaai.v38i7.28476
Abstract:
In recent years, following the success of text guided image generation, text guided 3D generation has gained increasing attention among researchers. Dreamfusion is a notable approach that enhances generation quality by utilizing 2D text guided diffusion models and introducing SDS loss, a technique for distilling 2D diffusion model information to train 3D models. However, the SDS loss has two major limitations that hinder its effectiveness. Firstly, when given a text prompt, the SDS loss struggles to produce diverse content. Secondly, during training, SDS loss may cause the generated content to overfit and collapse, limiting the model's ability to learn intricate texture details. To overcome these challenges, we propose a novel approach called Noise Recalibration algorithm. By incorporating this technique, we can generate 3D content with significantly greater diversity and stunning details. Our approach offers a promising solution to the limitations of SDS loss.
License type:
Publisher Copyright
Funding Info:
This research / project is supported by the Agency for Science, Tech- nology and Research (A*STAR) - MTC Programmatic Funds
Grant Reference no. : M23L7b0021

This research / project is supported by the OPPO - research grant
Grant Reference no. : NA
Description:
ISSN:
2374-3468
2159-5399
Files uploaded:

File Size Format Action
xiaofeng-aaai-camera.pdf 1.60 MB PDF Open