AfriNames: Most ASR models "butcher" African Names
Authors:
Tobi Olatunji,
Tejumade Afonja,
Bonaventure F. P. Dossou,
Atnafu Lambebo Tonja,
Chris Chinenye Emezue,
Amina Mardiyyah Rufai,
Sahib Singh
Abstract:
Useful conversational agents must accurately capture named entities to minimize error for downstream tasks, for example, asking a voice assistant to play a track from a certain artist, initiating navigation to a specific location, or documenting a laboratory result for a patient. However, where named entities such as ``Ukachukwu`` (Igbo), ``Lakicia`` (Swahili), or ``Ingabire`` (Rwandan) are spoken…
▽ More
Useful conversational agents must accurately capture named entities to minimize error for downstream tasks, for example, asking a voice assistant to play a track from a certain artist, initiating navigation to a specific location, or documenting a laboratory result for a patient. However, where named entities such as ``Ukachukwu`` (Igbo), ``Lakicia`` (Swahili), or ``Ingabire`` (Rwandan) are spoken, automatic speech recognition (ASR) models' performance degrades significantly, propagating errors to downstream systems. We model this problem as a distribution shift and demonstrate that such model bias can be mitigated through multilingual pre-training, intelligent data augmentation strategies to increase the representation of African-named entities, and fine-tuning multilingual ASR models on multiple African accents. The resulting fine-tuned models show an 81.5\% relative WER improvement compared with the baseline on samples with African-named entities.
△ Less
Submitted 2 June, 2023; v1 submitted 31 May, 2023;
originally announced June 2023.
AfriWOZ: Corpus for Exploiting Cross-Lingual Transferability for Generation of Dialogues in Low-Resource, African Languages
Authors:
Tosin Adewumi,
Mofetoluwa Adeyemi,
Aremu Anuoluwapo,
Bukola Peters,
Happy Buzaaba,
Oyerinde Samuel,
Amina Mardiyyah Rufai,
Benjamin Ajibade,
Tajudeen Gwadabe,
Mory Moussou Koulibaly Traore,
Tunde Ajayi,
Shamsuddeen Muhammad,
Ahmed Baruwa,
Paul Owoicho,
Tolulope Ogunremi,
Phylis Ngigi,
Orevaoghene Ahia,
Ruqayya Nasir,
Foteini Liwicki,
Marcus Liwicki
Abstract:
Dialogue generation is an important NLP task fraught with many challenges. The challenges become more daunting for low-resource African languages. To enable the creation of dialogue agents for African languages, we contribute the first high-quality dialogue datasets for 6 African languages: Swahili, Wolof, Hausa, Nigerian Pidgin English, Kinyarwanda & Yorùbá. These datasets consist of 1,500 turns…
▽ More
Dialogue generation is an important NLP task fraught with many challenges. The challenges become more daunting for low-resource African languages. To enable the creation of dialogue agents for African languages, we contribute the first high-quality dialogue datasets for 6 African languages: Swahili, Wolof, Hausa, Nigerian Pidgin English, Kinyarwanda & Yorùbá. These datasets consist of 1,500 turns each, which we translate from a portion of the English multi-domain MultiWOZ dataset. Subsequently, we investigate & analyze the effectiveness of modelling through transfer learning by utilziing state-of-the-art (SoTA) deep monolingual models: DialoGPT and BlenderBot. We compare the models with a simple seq2seq baseline using perplexity. Besides this, we conduct human evaluation of single-turn conversations by using majority votes and measure inter-annotator agreement (IAA). We find that the hypothesis that deep monolingual models learn some abstractions that generalize across languages holds. We observe human-like conversations, to different degrees, in 5 out of the 6 languages. The language with the most transferable properties is the Nigerian Pidgin English, with a human-likeness score of 78.1%, of which 34.4% are unanimous. We freely provide the datasets and host the model checkpoints/demos on the HuggingFace hub for public access.
△ Less
Submitted 19 May, 2022; v1 submitted 17 April, 2022;
originally announced April 2022.
