Efficient Transferability Estimation for Regression Tasks with True Labels

In this paper, the authors propose a new approach to transfer learning called simple transferability estimation for regression tasks. They aim to provide a simple and accurate method for estimating the transferability of a model to a new task without requiring extensive computational resources or complex theoretical derivations. The proposed method is based on the idea that the transferability of a model can be estimated by comparing the performance of the model on the original training data with its performance on a set of auxiliary tasks.
The authors provide theoretical bounds on the generalization gap between the original training data and the auxiliary tasks, which shows the relationship between the transferability of the model and the generalization gap. They also conduct an experiment to investigate the usefulness of their bounds in practice by comparing the generalization gap with the absolute value of their transferability score. The results show that the transferability scores dominate the generalization gap in practice, and there is no significant correlation between the generalization gap and the actual transferred MSE.
The authors also provide additional results for Section 6.1, which shows the ratios between the absolute value of transferability score and the generalization gap for different settings of the linear regression model. The results indicate that the complexity term in their bounds may have little effects for transferability estimation, while the transferability score term has a strong effect.
Overall, the authors demonstrate that their proposed method can provide a simple and accurate way to estimate the transferability of a model to a new task without requiring extensive computational resources or complex theoretical derivations. The results show that the proposed method is effective in practice and provides a promising approach for transfer learning in regression tasks.