Pruning Methods for Neural Networks: A Comparative Analysis

In this article, we explore a new method called M-Sup that can help improve the accuracy of graph neural networks (GNNs) in various tasks. M-Sup is based on the idea of finding the best "lottery tickets" among a set of candidate weights for a GNN model. These lottery tickets are chosen such that they have high accuracy while using fewer parameters than the original model.
To understand how M-Sup works, let’s first consider what is meant by "accuracy" in the context of GNNs. Accuracy refers to how well a model can predict the correct labels for a given dataset. In the case of GNNs, accuracy is often measured on a test set that has not been used during training.
Now, imagine you are playing a game where you have to choose a lottery ticket with a random combination of numbers. The ticket might win you a prize if the numbers match those drawn from a pool. In the same way, M-Sup chooses the best lottery tickets (i.e., weights) for a GNN model to improve its accuracy while using fewer parameters.
M-Sup achieves this by first defining a set of sparsity values, called K, which determine how many of the original weights are kept and how many are pruned. The method then selects the best lottery tickets from among these sparse weights based on their accuracy.
The authors of the article demonstrate that M-Sup can find better graph lottery tickets than another similar method called S-Sup, while achieving comparable accuracy as baseline models that use dense-weight learning (DWL). They also show that M-Sup is more effective in certain tasks, such as node-level tasks on small datasets and graph-level tasks on large datasets.
In summary, M-Sup is a method for improving the accuracy of GNNs by finding the best "lottery tickets" among a set of candidate weights. By pruning unnecessary weights and selecting the most accurate ones, M-Sup can help GNNs achieve better performance while using fewer parameters. This approach has important implications for practical applications of GNNs, where computational resources may be limited.