Unlearning in Machine Learning: A Survey

In the world of deep learning, it’s important to understand how models can memorize training data. This can be a problem when the training data contains private information, as it can lead to unauthorized access or exploitation of that data. The authors of this article explore the concept of "membership inference," which is the ability to infer whether a particular data point was used to train a deep learning model. They discuss two approaches to membership inference: white-box and black-box methods.
White-box methods are like a locked box that only the owner knows the combination to. These methods use all the information available about the model, including its architecture and parameters, to make inferences about the data. Black-box methods, on the other hand, are like a mysterious box with no visible clues about what’s inside. These methods use only the output of the model to make inferences about the data.
The authors propose Bayes optimal strategies for membership inference, which involve using both white-box and black-box methods depending on the situation. They show that these strategies can significantly improve the accuracy of membership inference while also reducing the risk of making incorrect inferences.
In summary, this article explores the challenges of dealing with privacy-sensitive data in deep learning models and proposes a new approach to membership inference that balances accuracy and security. By using both white-box and black-box methods, the authors demonstrate that it’s possible to make more informed decisions about data privacy while also protecting sensitive information.