Ablation Study Reveals Importance of Including All Graph Components for Text Classification

In this research paper, the authors explore the use of graph neural networks (GNNs) for text classification tasks. They examine how GNNs can capture global information about the vocabulary of a language and achieve better performance compared to traditional models. The authors conduct an ablation study to determine which components of the heterogeneous graph contribute the most to the task performances, and they find that including all graph components results in the best performances. They also analyze the impact of different features on the text classification tasks and conclude that GNNs can benefit from additional information such as part of speech (POS) tags, named entities, and transformer-based word/sentence embeddings.
The authors explain that traditional machine learning models, such as multi-layer perceptron (MLP), convolutional neural network (CNN), and recurrent neural network (RNN), capture contextual information within a sentence but lack the ability to capture global information about the vocabulary of a language. In contrast, GNNs can capture both local and global information by representing text as a graph. The authors demonstrate that augmenting the graph with additional information can further improve the performance of the model.
The authors also analyze the performances of different models in previous years’ shared tasks on digital text forensics and stylometry, including author profiling shared tasks focused on antisocial behavior detection on Twitter. They find that GNNs have gained interest in the NLP field for text classification in recent years and that most participants have used traditional machine learning approaches with various features such as n-grams, TF-IDF, lexicons, word embeddings, sentence embeddings, etc. However, some participants have created deep learning models such as MLP, CNN, and RNNs.
In summary, the authors of this study demonstrate that GNNs can achieve better performance than traditional models in text classification tasks by capturing both local and global information about the vocabulary of a language. They also show that augmenting the graph with additional information can further improve the performance of the model. These findings suggest that GNNs are a promising approach for text classification tasks, particularly in the field of digital text forensics and stylometry.