David Ifeoluwa Adelani, Dana Ruiter, Jesujoba Oluwadara Alabi, Damilola Adebonojo, Adesina Ayeni, Mofetoluwa Adeyemi, Ayodele Awokoya, C. España-Bonet

AfricaNLP

Paper

Abstract

Massively multilingual machine translation (MT) has shown impressive capabilities, including zero and few-shot translation between low-resource language pairs. However, these models are often evaluated on high-resource languages with the assumption that they generalize to low-resource ones. The difficulty of evaluating MT models on low-resource pairs is often due the lack of standardized evaluation datasets. In this paper, we present MENYO-20k, the first multi-domain parallel corpus for the low-resource Yor ` ub ´ a–English ( yo – en ) language pair with standardized train-test splits for benchmarking. We provide several neural MT (NMT) benchmarks on this dataset and compare to the performance of popular pre-trained (massively multilingual) MT models, showing that, in almost all cases, our simple benchmarks outperform the pre-trained MT models. A major gain of BLEU +9 . 9 and +8 . 6 ( en 2 yo ) is achieved in comparison to Facebook’s M2M-100 and Google multilingual NMT respectively when we use MENYO-20k to fine-tune generic models.