Combating Label Bias in Machine Learning: A Data Selection Approach

In the realm of machine learning, fairness constraints are a crucial aspect of ensuring that AI systems treat all individuals equally and without bias. However, incorporating these constraints into the learning objective can be challenging, and there are limited methods available to tackle this problem. This article reviews existing work on fairness constraints in machine learning and identifies key challenges and opportunities for future research.
The article begins by discussing the importance of fairness constraints in machine learning. The authors explain that incorporating these constraints can help reduce bias in AI systems, leading to more equitable outcomes. They highlight several approaches that have been proposed to tackle this problem, including modifying the model’s predictions using threshold adjustments and calibration, incorporating fairness constraints into the learning objective, and using data manipulation or representation learning methods before training the model.
However, the authors note that these methods primarily focus on modifying the machine learning model to tackle the bias problem, rather than addressing the biased data itself. They argue that this approach can lead to a "band-aid solution" that does not fully address the underlying issues.
To overcome these challenges, the article proposes several new methods for fairness constraints in machine learning. These methods include confident evaluation against several baselines, including learning (CL) (Northcutt, Jiang, and Chuang 2021), Lon-label bias correction (Cordeiro et al. 2023), (LC) (Jiang and Nachum 2020), and the group peer loss (GPL) method as described in Wang, Liu, and Levy (2021).
The article also discusses several fairness metrics that can be used to evaluate the performance of these methods. These metrics include demographic parity distance, difference of equal opportunity (DEO), and percentage (p%). The authors note that these metrics provide a comprehensive view of fairness and can help identify areas where the methods can be improved.
In conclusion, the article highlights the importance of addressing fairness constraints in machine learning and provides several new methods for tackling this problem. By incorporating these methods into AI systems, we can promote more equitable outcomes and ensure that all individuals are treated fairly. As the field of AI continues to evolve, it is essential to prioritize fairness constraints to build trustworthy and ethical AI systems.