Out-of-Distribution Detection Methods: A Comprehensive Survey

Out-of-distribution (OOD) detection is a crucial task in machine learning, as it helps identify when a model is fed data that is significantly different from the training data. In this article, we review various score functions and pretext tasks used for OOD detection, including probability-based, logit-based, and feature-based methods.
Probability-based methods, such as maximum softmax probabilities and minimum KL-divergence, rely on the difference between the model’s confidence in the prediction and the true label. Logit-based methods, like logsumexp and the logit function, use the difference between the model’s predictions and the true labels to calculate a score. Feature-based methods, such as norm of the residual and Mahalanobis distance, focus on the similarity between the feature distributions of the test data and the training data.
The article analyzes the performance of these methods using various datasets, including CIFAR-10, ImageNet, and MSP. It also compares the performance of different score functions and pretext tasks, showing that some methods are more effective than others depending on the dataset and the specific task.
One key finding is that even simple score functions can perform equally well as more complex ones when applied to well-designed ID features. This underscores the importance of carefully designing the ID feature representation rather than relying solely on the choice of score function.
The article also discusses some challenges in OOD detection, such as the need to balance accuracy and FPR (false positive rate), and the potential for overfitting when using complex score functions.
In summary, this article provides a comprehensive review of various score functions and pretext tasks used for out-of-distribution detection, highlighting their strengths and weaknesses, and demonstrating the importance of carefully designing ID features to achieve optimal performance.