Text-image composed retrieval is a powerful technology that enables us to search for images based on textual descriptions. However, this process can be challenging due to the inherent limitations of natural language and visual representations. To address these challenges, researchers have proposed various models that can robustly retrieve images based on textual queries. In this article, we will discuss the criteria used to evaluate the robustness of these models.
Criteria for Robustness
There are twofold criteria for evaluating the robustness of text-image composed retrieval models: natural corruption and textual understanding.
1. Natural Corruption
Natural corruption refers to the degradation of images or text due to various factors such as noise, blur, weather, and digital following. To evaluate the robustness of models under natural corruption, we use 15 standard image corruptions and 7 text corruptions, categorized into character-level and word-level. We assess the performance of models in retrieving the target visual content through dense continuous images guided by sparse discrete text.
2. Textual Understanding
Textual understanding refers to the ability of models to reason between textual and visual modalities consistently. To evaluate this criterion, we use a combination of character-level and word-level textual corruptions. We analyze the performance of models in retrieving images based on textual descriptions that have been modified with specific keywords or gallery sets.
Analysis
Our analysis shows that the performance of text-image composed retrieval models degrades significantly under natural corruption, particularly in the presence of image noise and blur. However, some models demonstrate remarkable robustness against textual understanding, consistently retrieving the target visual content even when the textual description has been modified significantly.
Conclusion
In conclusion, evaluating the robustness of text-image composed retrieval models is crucial to ensure their effectiveness in real-life scenarios. Our analysis highlights the importance of considering both natural corruption and textual understanding when assessing the performance of these models. By developing more robust models that can handle diverse types of corruptions, we can improve the accuracy and reliability of text-image composed retrieval systems.
