Evaluating Unsupervised Machine Translation with BPE-Encoded Data

In the field of machine translation, obtaining paired training data can be a significant challenge, especially for low-resource languages. To address this issue, unsupervised machine translation (UMT) was proposed, which aligns languages without parallel data using vocabulary-level statistics or abstract language representations.
The article begins by explaining that traditional UMT approaches rely on self-supervised learning and statistical methods, such as back-translation and synchronized training. However, these methods can be computationally expensive and may not always produce optimal results.
To overcome these limitations, the authors introduce a novel approach called quick back-translation synced (QBT-Synced). This method combines the strengths of back-translation and synchronized training while reducing computational complexity.
The QBT-Synced algorithm works by first preparing all monolingual data for English, French, German, and Romanian up to 2017. The dataset contains sentence counts of 190M, 62M, and 270M for the English, French, and German languages, respectively.
The authors then encode the data using Byte-Pair Encoding (BPE) with a dictionary of 60K sub-words provided by Conneau and Lample (2019). Each language pair, such as English and German, is treated as a single encoder-decoder Transformer model.
The key innovation of QBT-Synced lies in its ability to truncate longer sentences to have the same length as shorter ones. This ensures that the model can learn to translate both equally well.
The authors evaluate their proposed method on WMT News Crawl datasets and compare it with other state-of-the-art unsupervised machine translation methods, such as dual learning (He et al., 2016) and non-autoregressive machine translation (Vu et al., 2020). The results show that QBT-Synced outperforms these methods in terms of translation quality.
In conclusion, the article provides a novel approach to unsupervised machine translation by leveraging vocabulary-level statistics. The proposed QBT-Synced algorithm demonstrates improved performance compared to existing methods and has the potential to significantly improve the efficiency and accuracy of machine translation systems in low-resource languages.
Metaphor: Imagine trying to build a tower without using any blueprints or guidelines. Just like how unsupervised machine translation requires aligning languages without parallel data, building the tower requires stacking blocks on top of each other without knowing their exact size or position. By leveraging vocabulary-level statistics or abstract language representations, UMT is like using a blueprint to ensure that the tower is built correctly and efficiently.