In this article, we present a novel approach for addressing dataset imbalance in machine learning by generating synthetic samples that balance the distribution of classes. Our method is based on the concept of entropy-based active learning, which selects informative samples from the minority class and leverages them to generate synthetic data that balances the distribution. The key insight is that by adding synthetic samples to the original dataset, we can overcome the class imbalance challenge without sacrificing accuracy. We demonstrate the effectiveness of our approach through extensive experiments on several datasets and show that it significantly improves the performance of machine learning algorithms. Our strategy is summarized in Algorithm 2, which consists of two main steps: active learning to select informative samples, and generating synthetic data to balance the distribution. By repeating these steps until the desired balance is achieved, we can create a balanced dataset that performs better than the original imbalanced dataset.
Section 1: Introduction
Dataset imbalance is a common problem in machine learning, where the classes are not evenly represented in the dataset. This can lead to biased models that perform poorly on minority classes, resulting in lower accuracy and unfairness. To address this challenge, we propose a novel approach based on entropy-based active learning, which generates synthetic samples that balance the distribution of classes. By adding these synthetic samples to the original dataset, we can create a balanced dataset that performs better than the original imbalanced dataset.
Section 2: Algorithm Description
Our strategy consists of two main steps: active learning and generating synthetic data. In the first step, we choose an active learning method AL(ยท) and find a subset of informative samples S by leveraging entropy-based active learning. These informative samples are selected based on their high entropy, which means they are likely to provide valuable information for balancing the distribution. In the second step, we generate synthetic data to balance the distribution of classes. For each random sample xc in S and belonging to minority class c, we randomly sample a small radius r and find a synthetic sample that lies on the sphere centered at xc and maximizes the posterior ratio in Equation 11. The process is repeated until the informative set S is balanced, and the remaining region is also balanced. The final output of the algorithm is a balanced dataset D’.
Section 3: Time Complexity Analysis
The time complexity of our algorithm depends on the choice of active learning method and the size of the dataset. For example, if we use the uniform random sampling method as the active learning method, the time complexity will be O(n), where n is the number of samples in the dataset. However, if we use a more sophisticated active learning method such as Uncertainty Sampling, the time complexity can be reduced to O(log n). In general, our algorithm has a polynomial time complexity that grows slowly with the size of the dataset, making it feasible for large-scale datasets.
Conclusion
In this article, we proposed a novel approach for addressing dataset imbalance by generating synthetic samples that balance the distribution of classes. Our method is based on entropy-based active learning and can significantly improve the performance of machine learning algorithms on imbalanced datasets. By adding synthetic samples to the original dataset, we can create a balanced dataset that performs better than the original imbalanced dataset without sacrificing accuracy. We demonstrated the effectiveness of our approach through extensive experiments on several datasets and showed that it outperforms other state-of-the-art methods for imbalanced data. Our strategy provides a practical solution for real-world applications where dataset imbalance is a common challenge.
