Case Study: MyAIFactChecker
Executive Summary
Executive Summary
Journalism in Sub-Saharan Africa faces many challenges. Most newsrooms have little money and few resources to work with. Journalists often worry about being censored or punished for their work. The many languages spoken in the region make it hard to fact-check stories across all communities. Old technology and a lack of modern tools make it difficult for journalists to do their job well. At the same time, false information spreads quickly on social media. All these problems make it hard, but very important, to check facts quickly and correctly. This situation shows why new solutions that fit the region’s needs are so important.
This case study examines the development and implementation of “MyAIFactChecker,” an AI-powered tool designed to enhance fact-checking capabilities in the region. Launched by Fact Check Africa, a Brain Builders Youth Development Initiative (BBYDI) initiative, this tool represents a significant step towards combating misinformation and strengthening journalistic integrity in Sub-Saharan Africa. Abideen Olasupo, the Global Director of BBYDI, said the initiative is a product of months of dedicated efforts and research with support from BECERA and the U.S. State Department.
Ideation and Initial Development
In 2023, BBYDI identified the need for an innovative solution to address these challenges. Alamin Musa Magaga, the product development lead, was brought on board to spearhead the technical development.
As Magaga recalls: “I was contacted by the global director of Brain Builders Youth Development Initiative, BBYDI, Olasupo Abideen, to facilitate the development of an AI solution for fact-checking.”
The project was initially conceived in anticipation of the U.S. West African Tech Challenge, where BBYDI aimed to present strategies to combat disinformation in West Africa.
Data Sources and AI Integration
MyAIFactChecker combines advanced search APIs, fact-checking databases, and a curated list of credible sources. Magaga explains the process:
“When the user inputs the claim, the first thing the tool will do is utilise the advanced search API, which will then go through a lot of credible resources and fact-checking databases and then it will check information related to that specific claim. After fetching the top sources, it’ll then utilise Generative AI to provide the response for the claim”
The system incorporates the OpenAI GPT-4 and LlaMA 3 model for efficient analysis and responses.
Recognising that many top news sources weren’t locally relevant, the team took steps to include reputable local sources. This approach ensured that the fact-checking process was grounded in locally relevant and trusted sources, enhancing the tool’s effectiveness in the West African context.
Pilot Implementation and Accuracy Benchmarking
The team at MyAIFactChecker places a high priority on accuracy, recognising its critical importance in journalism and fact-checking. Their approach to ensuring accuracy has evolved.
Initially, the team took a conservative approach by limiting the tool to only verifying information that had already been fact-checked by reputable sources. Magaga explains this early strategy:
“We decided to focus only on already fact-checked data, limiting our AI system to be able to fully respond only to questions that had been fact-checked by the organisation [Fact Check Africa].”
This cautious approach ensured high accuracy but limited the tool’s scope. More importantly, it was rooted in their primary goal, as Magaga emphatically states: “We had to optimise for correctness. It had limited functionalities, but our goal was to build a tool to curb misinformation, and not contribute to fake news, therefore, we had to push for 100% accuracy.”
As the project progressed, they expanded their capabilities while maintaining this strong focus on accuracy. Magaga describes their ongoing efforts: “We needed to monitor the tool for hallucinations because this is very common with generative AI. We are working on a monitoring system that helps us monitor the hallucination rate and manage it.”
This evolution from a system based solely on pre-verified information to a more comprehensive tool that can handle a wider range of queries, while still prioritising accuracy, demonstrates the team’s dedication to creating a reliable fact-checking resource. Their ongoing work on managing and mitigating AI hallucinations shows a nuanced understanding of the challenges posed by generative AI in the context of fact-checking.
Testing and Launch
The development, testing, and launch of MyAIFactChecker was an iterative process that involved continuous feedback and improvement. The initial development of MyAIFactChecker was a focused effort. The first four to five months were intensive and involved the development of the core functionality that was used in the U.S. West African Tech Challenge.
Following the initial development, the team extensively tested and refined the tool. Magaga describes the process:
“After the competition, we started working on the original version. The UI/UX team had to design the user interface, and then we started developing, after which people tested it and gave their feedback. Then, continuous iterations until the product was ready for launch.”
The team placed a strong emphasis on user feedback, both during development and after launch. Magaga explains: “We had a lot of feedback, much of which we couldn’t incorporate.” This highlights the challenges of balancing user requests with technical feasibility and project scope.
MyAIFactChecker was launched in April 2024, but this wasn’t the end of the development process. The team actively sought feedback even during launch, wanting to know what problems users had and how to fix them.
This continuous improvement and user engagement approach has contributed to the tool’s adoption and effectiveness.
Challenges and Solutions
Resource Constraints
Building an AI-powered tool like MyAIFactChecker is capital-intensive, especially for a small media team. The team faced significant resource constraints but employed various strategies to overcome these challenges.
Before securing external support, the team focused on finding cost-effective solutions. Magaga explains their initial approach:
“We looked at a lot of alternatives and found something cheaper or even free, sometimes even better than the paid solutions or APIs that exist. We understood our capacity and had to focus on that.”
This strategy of seeking out affordable or free alternatives demonstrates the team’s resourcefulness in the face of financial constraints.
As the project progressed, the team successfully leveraged startup support programs to access necessary resources. Magaga shares:
“We applied for the Microsoft Startup Founders program, and they gave us access to different tools, which are part of the tools we are using, and now we’re also part of the Google Ads rate tool startup program, which also gives us at least $100,000 Google credit for different tools and resources”
These programs provided crucial support, allowing the team to access advanced tools and services that might otherwise have been out of reach.
The team had to make strategic decisions about which features to implement based on available resources, with a strong preference for existing libraries and open-source solutions instead of building from scratch.
This approach of carefully weighing the costs and benefits of different tools and features allowed the team to maximise their limited resources. The team’s approach to resource management evolved. The team’s efforts to secure resources and manage costs have contributed to the project’s sustainability.
Infrastructure Challenges
Power outages are a common issue in the region. Magaga describes a workaround employed in the very early stages of his career: “Power is a problem because when you wake up every day, you have to think of the best place to go, sometimes we’d have power issues in our area, and I’ll have to be at the office overnight and sleep there.”
Data Limitations
The team faced challenges in building predictive models due to limited datasets. Magaga explains: “We wanted to build a simple machine learning model for detecting and predicting fake news, but we understand how limited this can be because the sources play a big role in trying to determine whether a piece of information is accurate or not. That is very problematic in the real-world aspect of Nigeria, where we have limited data sets.”
To address this, they created a database to store fact-checked information and sources, building a foundation for future machine learning models. “But we have a database to store all the information that’s being fact-checked and their sources. This means we now have sources for a data set, and then we can build machine learning models for predicting fake news, and other research purposes”, he says.
Impact and Usage
Since its launch in early 2024, MyAIFactChecker has processed over 10,000 queries, especially during the August 2024 Nigerian protests. They have also trained over 12,000 students in universities on fact-checking Information.
Ethical Considerations
The team prioritises accuracy to avoid contributing to misinformation. Magaga emphasises: “We had to optimise for correctness. The tool incorporates generative AI to analyse claims and generate responses based on the gathered information.”
Lessons Learned
1. Prioritise Accuracy: In journalism, and as an organisation that tackles fake news, maintaining 100% accuracy is crucial.
2. Leverage Existing Tools: Instead of building everything from scratch, Magaga advises the utilisation and integration of existing APIs and libraries when possible, to come up with a solution that matches the organisation’s needs.
3. Continuous Learning: Stay updated with the latest AI tools and techniques. Magaga notes: “Each and every day, a lot of tools are coming to make this much easier, but it can also make the solutions you have implemented obsolete. Then you have to remove all the time and effort you spent, and just replace it with something better. But you just have to find a way to keep yourself updated for a more effective experience.”
4. Localisation is Key: Customising sources and content for specific regions enhances the tool’s relevance and effectiveness. The team created a custom list of top sources for Nigeria and other West African countries.
5. Feedback Loop:
Incorporate user feedback and continuous monitoring. Magaga mentions: “We had a lot of feedback, much of which we couldn’t incorporate. But when we get feedback and people tell us all the things they want us to implement, they’ll tell us what to remove and what to add. This was a hard decision, especially when you have a complete platform. But we take it all into account for the next version. We implement some of the changes, but we don’t do what people suggest all the time.” When it comes to resistance, it’s the ability to say no and manage expectations, especially from non-technical team members. “We know the consequences and what it’ll look like.”
“Whatever you build, people have to test it. From my perspective, it’s about the expectations. Are we able to build something that’ll satisfy what the journalists want? The recommendations and feedback are quite important. When we released the first version, people tested it, we got feedback, and had to go back to the drawing table, and do it all over again.”
“It’s what the customer wants, and you as the solution provider, have to do whatever it takes to satisfy their needs. Even during the launch, we had people provide feedback because we wanted to understand the position of the user. From the user, we can understand what is wrong with the program and how we can improve it. We also have a feedback box on the app. Feedback is very important, and we have it recurring.”
Conclusion
MyAIFactChecker represents a significant step forward in combating misinformation in Sub-Saharan Africa. By leveraging AI technologies and focusing on local needs, it offers a promising solution to the challenges faced by journalists in the region.
Magaga’s final thoughts encapsulate the project’s vision: “Artificial Intelligence is the future, the choice is now just up to the journalists, whether they’re going to use it or not. It’s not just fact-checking, it’s the storytelling, animation, video and image editing. It’s just part of the future and it has now started changing different aspects of journalism. It’s going to be there, things will be different and it’s going to change almost everything. People just have to find a way to be able to adapt and learn how to use it.”
This case study highlights the importance of innovation, adaptability, and collaboration in developing AI solutions for journalism in Sub-Saharan Africa. As the project continues to evolve, it has the potential to significantly impact the landscape of fact-checking and journalism in the region.
Thanks to Aisha Bello for support with research and copy.
First published on Medium by Stephanie S.I. Ohumu on January 30, 2025.
Exported from Medium on April 22, 2025.