In the field of neural networks, there is a common issue where new classes are not well represented in the network’s performance. This problem has been studied extensively, and various methods have been proposed to address it. However, these methods often overlook the abundant semantic information present in base classes, which can significantly improve the network’s performance on new classes. In this study, we aim to take a small step towards filling this gap by leveraging the semantic information from base classes to improve the performance of new classes.
Methods
To achieve this goal, we propose a novel method called TEEN (Training with Empirical Evidence and Neural Networks). Our approach utilizes the empirical observation of the similarity between base and new classes to align the prototypes of new classes with their corresponding well-represented semantic similar classes. We experiment with different hyperparameters and demonstrate that our method outperforms existing methods in terms of accuracy on both all classes and new classes.
Results
Our results on miniImageNet show a significant improvement in performance compared to existing methods. We also observe that the similarity-based weight, which aligns new class prototypes with well-represented semantic similar base class prototypes, is crucial for achieving good performance. Removing this weight results in a substantial drop in accuracy, confirming the importance of utilizing semantic information from base classes.
Discussion
Our findings demonstrate that leveraging the abundant semantic information present in base classes can significantly improve the performance of new classes. By aligning new class prototypes with well-represented semantic similar base class prototypes, we can effectively utilize the semantic information from base classes to improve the network’s performance on new classes. This approach has important implications for improving the performance of neural networks on new classes, and it highlights the importance of considering the semantic structure of the data when training neural networks.
Conclusion
In conclusion, our study demonstrates that utilizing the abundant semantic information present in base classes can significantly improve the performance of new classes. By leveraging this information, we can effectively align new class prototypes with well-represented semantic similar base class prototypes, leading to improved accuracy on both all classes and new classes. Our proposed method, TEEN, offers a novel approach to filling the gap in neural network performance on new classes and highlights the importance of considering the semantic structure of the data when training neural networks.
