UniFL: Improve Stable Diffusion via Unified Feedback Learning
The paper proposes UniFL, a unified framework that leverages feedback learning to enhance the visual quality, aesthetic appeal, and inference speed of diffusion models like Stable Diffusion, aiming to address these limitations.
Method Overview
UniFL (Unified Feedback Learning) is a unified, effective generalizable solution that uses feedback learning to enhance diffusion models. UniFL consists of three key components:
1. Perceptual Feedback Learning (PeFL): This leverages existing perceptual models to provide feedback signals for improving visual generation quality. It incorporates image content as an additional condition along with the text prompt.
2. Decoupled Feedback Learning: Instead of using a single reward model for aesthetic preferences, UniFL decouples the aesthetic concept into distinct dimensions like color, layout, lighting, and detail. Separate reward models are trained for each dimension using annotated data. An active prompt selection strategy is also used to mitigate over-optimization during fine-tuning.
3. Adversarial Feedback Learning: This combines feedback learning with an adversarial objective to accelerate inference. The diffusion model acts as the generator, optimized to increase the reward score. The reward model acts as a discriminator, distinguishing between preferred and unpreferred samples. This enables optimization across all denoising steps, facilitating faster inference.
Results
UniFL demonstrates consistent improvements across various diffusion models, such as SD1.5 and SDXL.
User studies confirm that UniFL outperforms other methods in terms of image quality and aesthetic appeal. Additionally, UniFL achieves significant acceleration, surpassing existing methods like LCM and SDXL Turbo in 4-step inference scenarios.
Conclusion
UniFL provides a unified framework for enhancing diffusion models, effectively addressing limitations in visual quality, aesthetics, and inference speed. For more information please consult the full paper or the project page.
Congrats to the authors for their work!
Zhang, Jiacheng, et al. “UniFL: Improve Stable Diffusion via Unified Feedback Learning.” arXiv preprint arXiv:2404.05595, 2024