Improving Vulnerability Detection with Duplicate Sample Handling and Benign Code Fragment Identification

In the field of machine learning, understanding how models make predictions is crucial for developers to trust their work. However, current methods for explaining these predictions lack accuracy and relevance, leading to confusion and inefficiency. This paper presents two perspectives on evaluating explanation methods: Descriptive Accuracy and Relevancy.
Descriptive Accuracy
Imagine you’re trying to understand how a recipe works by reading its instructions. If the recipe doesn’t accurately describe how it prepares the dish, you might end up with a confusing mess. Similarly, Descriptive Accuracy in machine learning evaluates if explanation methods correctly capture the behavior of models. The paper proposes using fidelity to measure this accuracy, which is like checking if the explanation method’s "ingredients" (important features) match the model’s actual behavior.
Relevancy
Now imagine you have a recipe book with recipes from different cuisines. Some might be relevant for your current meal, while others might not. Relevancy in machine learning evaluates if explanation methods provide valuable information for specific tasks. The paper proposes using LC (Lexical-Functional Complexity) to measure relevancy, which is like checking if the explanation method’s "ingredients" are relevant for your current dish.
To evaluate these metrics, the paper introduces a wide net approach that generates many potential instances with modified features and calculates their prediction scores. This approach ensures that both accuracy and relevance are considered in the evaluation process.
In conclusion, this paper provides a comprehensive framework for evaluating explanation methods in machine learning. By considering both descriptive accuracy and relevancy, developers can trust these methods more and build better models.