Togo has asked its citizens to contribute texts, recordings and translations to “develop AI models that address the linguistic diversity of the country”, said AI for Developing Countries Forum.
More than just a “regional experiment”, it is a “blueprint for how developing nations can use technology to assert their cultural sovereignty and economic relevance in the digital age”.
Africa has around 2,000 languages but very few are represented comprehensively in AI models, said academics on The Conversation. Failing to be a part of the tech revolution not only “marginalises millions of people” but also “increases the technology divide”.
98% of African languages ‘unsupported’
AI’s inability to handle most African languages comes down to the “gap between the number of real-world speakers and a language’s digital footprint,” said African Business.
According to research published in the Proceedings of Machine Learning Research, across six large language models (LLMs), eight small language models and six specialised small language models, only 41 African languages were represented. Only four – Amharic, Swahili, Afrikaans and Malagasy – were “always handled”. “Over 98% of African languages went unsupported.”
According to OpenAI, the number of ChatGPT users in Africa is “growing faster than in any other region, said The Economist. Yet, despite the surge in popularity, even the best LLMs “falter in African languages, typically scoring 10-30 percentage points lower on benchmarking tests than they do in English”. The models mainly struggle with numerous dialects and “code-switching”, which is when speakers use different languages at the same time.
“The root problem is lack of data.” Much of the training for “smaller” languages is taken from Bible verses. High-quality audio fit for speech recognition is also “scarce”, so researchers are “recording how people talk” to build the data supply.
It’s not just a lack of text and speech that is the problem, said the academics on The Conversation, it’s also the infrastructure that supports it. “Basic tools” like dictionaries, terminologies and glossaries are missing. Then there are inadequacies in African language keyboards, fonts, spell-checkers and tokenisers – which “break text into smaller pieces so a language model can understand it”. All of these impact orthographic variation – how the same words are spelled differently across regions – tone marking and “rich dialect diversity”.
‘Global North cultural assumptions’
The texts that dominate LLMs “reflect Global North cultural assumptions, about individualism, competition and about globalisation”, said University World News. By definition, therefore, this is a continuation of the “long history of epistemic marginalisation” of the Global South.
The lack of African languages in LLMs has become a “major concern for the continent’s policymakers and tech leaders”, said Semafor. The World Bank estimates that Africa has only 0.6% of global data-centre capacity, despite having 18% of the world’s population.
Africa’s “main link” to the AI revolution has been through minerals such as cobalt and copper, but the “bigger prize” is through productivity, or being able to harness the end product of AI. Currently, however, there isn’t the infrastructure or energy to achieve this, meaning that AI is “unlikely to add meaningfully to growth”.
African AI companies need to break beyond being “suppliers to the AI revolution or just consumers of finished AI products”. Their continent needs to be the place where the “revolution delivers its biggest gains”.
Building AI models from scratch is “expensive” and uses “limited resources”. By adopting and adapting existing models, the continent’s outputs can evolve more quickly. And Togo’s initiative suggests that the future of AI in Africa “may hinge less on who builds the biggest model than on who makes the technology useful to the most people”.