Text-to-Image Generation: A Comprehensive Review of Evaluation Metrics and Baselines

In this article, researchers propose a new method called Check, Locate, Rectify (CLR) to improve the accuracy of text-to-image generation models. These models are trained on large datasets of text-image pairs, but they often struggle to accurately interpret the layout instructions in the text. CLR addresses this challenge by using a training-free approach that leverages the inherent structure of the text data to calibrate the image generation process.
The key idea behind CLR is to assign superlative boxes to the objects described in the text, and then use these boxes to guide the image generation process. For example, if the text says "A dog to the left of a cat," CLR would assign the superlative box to the dog and the relative box to the cat. This allows the model to generate an image that is consistent with the layout instructions in the text.
To improve the accuracy of the image generation process, CLR also uses a semantic parsing technique to identify the objects mentioned in the text. This involves assigning a semantic meaning to each word in the text, which helps the model to better understand the context and content of the text.
The CLR system is evaluated on a benchmark dataset of 203 prompts, and the results show that it outperforms existing text-to-image generation models. The researchers also demonstrate that their approach can be used to generate images that are consistent with different superlative relations, such as "the crown on top of the lion."
In summary, CLR is a training-free layout calibration system that improves the accuracy of text-to-image generation models by leveraging the inherent structure of the text data. By assigning superlative boxes and using semantic parsing, CLR enables the model to generate images that are consistent with the layout instructions in the text, resulting in more realistic and accurate image generation.