Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Separating Grains from the Chaff: Using Data Filtering to Improve Multilingual Translation for Low-Resourced African Languages
Ahmadu Bello University, Zaria, Nigeria; HausaNLP.
University of the Witwatersrand, South Africa.
Saarland University, Germany.
TUM, Germany; Mila - Quebec AI Institute.
Show others and affiliations
2022 (English)In: Proceedings of the Seventh Conference on Machine Translation (WMT) / [ed] Philipp Koehn, Loïc Barrault, Ondřej Bojar, Fethi Bougares, Rajen Chatterjee, Marta R. Costa-jussà, Christian Federmann, Mark Fishel, Alexander Fraser, Markus Freitag, Yvette Graham, Roman Grundkiewicz, Paco Guzman, Barry Haddow, Matthias Huck, Antonio Jimeno Yepes, Tom Kocmi, André Martins, Makoto Morishita, Christof Monz, Masaaki Nagata, Toshiaki Nakazawa, Matteo Negri, Aurélie Névéol, Mariana Neves, Martin Popel, Marco Turchi, Marcos Zampieri, Association for Computational Linguistics , 2022, p. 1001-1014Conference paper, Published paper (Refereed)
Abstract [en]

We participated in the WMT 2022 Large-Scale Machine Translation Evaluation for the African Languages Shared Task. This work de-scribes our approach, which is based on filtering the given noisy data using a sentence-pair classifier that was built by fine-tuning a pre-trained language model. To train the classifier, we obtain positive samples (i.e. high-quality parallel sentences) from a gold-standard curated dataset and extract negative samples (i.e.low-quality parallel sentences) from automatically aligned parallel data by choosing sentences with low alignment scores. Our final machine translation model was then trained on filtered data, instead of the entire noisy dataset. We empirically validate our approach by evaluating on two common datasets and show that data filtering generally improves overall translation quality, in some cases even significantly.

Place, publisher, year, edition, pages
Association for Computational Linguistics , 2022. p. 1001-1014
Series
Workshop on Statistical Machine Translation, ISSN 2768-0983
National Category
Language Technology (Computational Linguistics)
Research subject
Machine Learning
Identifiers
URN: urn:nbn:se:ltu:diva-98272Scopus ID: 2-s2.0-85160510996ISBN: 978-1-959429-29-6 (print)OAI: oai:DiVA.org:ltu-98272DiVA, id: diva2:1766629
Conference
Seventh Conference on Machine Translation, (WMT 2022), December 7-8, 2022, Abu Dhabi, United Arab Emirates
Available from: 2023-06-13 Created: 2023-06-13 Last updated: 2023-06-13Bibliographically approved

Open Access in DiVA

No full text in DiVA

Scopus

Authority records

Adewumi, Oluwatosin

Search in DiVA

By author/editor
Adewumi, Oluwatosin
By organisation
Embedded Internet Systems Lab
Language Technology (Computational Linguistics)

Search outside of DiVA

GoogleGoogle Scholar

isbn
urn-nbn

Altmetric score

isbn
urn-nbn
Total: 167 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf