Wang, Y., Shen, S., & Lim, B. Y. (2023). RePrompt: Automatic Prompt Editing to Refine AI-Generative Art Towards Precise Expressions. Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3544548.3581402
Abstract:
Generative AI models have shown impressive ability to produce images with text prompts, which could benefit creativity in visual art creation and self-expression. However, it is unclear how precisely the generated images express contexts and emotions from the input texts. We explored the emotional expressiveness of AI-generated images and developed RePrompt, an automatic method to refine text prompts toward precise expression of the generated images. Inspired by crowdsourced editing strategies, we curated intuitive text features, such as the number and concreteness of nouns, and trained a proxy model to analyze the feature effects on the AI-generated image. With model explanations of the proxy model, we curated a rubric to adjust text prompts to optimize image generation for precise emotion expression. We conducted simulation and user studies, which showed that RePrompt significantly improves the emotional expressiveness of AI-generated images, especially for negative emotions.
License type:
Publisher Copyright
Funding Info:
This research / project is supported by the Ministry of Education (MOE) - Academic Research Fund Tier 2
Grant Reference no. : T2EP20121-004
This research / project is supported by the National University of Singapore - iHealthtech Smart Sensors and Artificial Intelligence (AI) for Health grant
Grant Reference no. : NA