Enhancing Vision-Based People Recognition with Mixed Precision Training and Robust Image Descriptors

In the field of computer vision, image descriptors play a crucial role in identifying and categorizing images. However, creating robust image descriptors that can withstand changes in appearance or viewpoint is a significant challenge. This article provides an overview of various techniques used to form image descriptors that are resilient to these changes.
Early attempts involved handcrafting features to identify key points in an image. The Scale-Invariant Feature Transform (SIFT) method uses a Difference Of Gaussians blob detector to locate key points and describes them using a histogram of Oriented Gradients (HOG). SIFT descriptions are then speeded up in the Suppressed Uniformity and Detail (SURF) method by utilizing integral images.
To further improve efficiency, BRIEF (Binary Robust Independent Elementary Features) is introduced as a lightweight keypoint descriptor that produces binary features for fast similarity search. BRIEF aggregates features with Bag Of Visual Words (BOVW), Fisher Vectors, or Vector of Locally Aggregated Descriptors (VLAD) techniques to form a robust codebook.
While both BOVW and VLAD cluster keypoint features to form a codebook, BOVW computes its descriptor using a histogram of code frequencies, while VLAD sums the residuals between features and their corresponding codes.
Training these models involves contrastive learning with the triplet loss function, which compares the similarity between an anchor image and positive images that depict the same location to negative images that do not. The Adam optimizer is used with a learning rate of 1e-4, and training stops when validation accuracy does not increase for three epochs.
In terms of backbones, the descriptor size D determines the retrieval latency τr, which significantly affects the total VPR system latency τtotal, including encoding and retrieval latency. A jump in descriptor size from 512 to 4096 increases VPR system latency by 63% and memory use by 8 times, highlighting the importance of small descriptors for efficient VPR.
In summary, this article provides a comprehensive overview of techniques used to form image descriptors that are robust to appearance and viewpoint changes. By utilizing integral images, histograms, and other techniques, these descriptor methods can significantly improve efficiency while maintaining accuracy. Understanding the trade-offs between descriptor size and resource utilization is crucial for developing efficient VPR systems.