Efficient Fairness-Aware Compression of Deep Neural Networks

In recent years, there has been a growing interest in compressing deep neural networks (DNNs) to improve their efficiency while maintaining their performance. One popular approach is to prune away redundant or unnecessary neurons and connections within the network. However, this process can also introduce biases into the model, particularly if not done carefully.
Researchers have found that when they prune DNNs, the removed neurons and connections tend to disproportionately affect certain groups of people, such as those from different demographics or with varying backgrounds. This means that the compressed model can perform worse on certain groups of people than others, leading to unfairness and potential legal issues.
To address this issue, researchers have proposed various methods to improve fairness in compressed DNNs. These include optimizing pruning techniques to minimize bias, using fairness-aware regularization methods, and inserting fairness considerations into the pruning process itself. Researchers have also explored different approaches to prune DNNs, such as structured pruning, which involves removing specific structures within the network, and filter-wise pruning, which involves removing filters or feature maps from convolutional layers.
One important finding in this area of research is that fairness considerations should be integrated into the pruning process to ensure that compressed models are both efficient and fair. This can involve using Pareto-based frameworks to insert fairness considerations into pruning processes, as well as developing specific criteria to determine whether bias intensification is likely to occur post-pruning.
Overall, the article provides a comprehensive overview of the challenges and opportunities in compressing deep neural networks while ensuring fairness and avoiding biases. By understanding these issues, researchers can develop more efficient and fair DNNs that can be deployed in real-world applications.