How Did an AI Hallucination Lead to the Publication of a Fabricated Government Draft Policy in South Africa?

In a notable incident concerning AI-assisted governance, South African Minister Solly Malatsi revealed that a draft national AI policy containing fabricated academic citations was published in the official Government Gazette and later withdrawn. The citations were the direct result of an artificial intelligence tool generating entirely fictitious references, a phenomenon known as an AI hallucination.

The Draft South Africa National Artificial Intelligence Policy was published for public comment on April 10, 2026, following Cabinet approval on March 25, 2026. It was withdrawn on April 27, 2026, after at least 6 of its 67 academic citations were confirmed as fabrications, referencing journals, authors, and sources that simply do not exist. The minister was direct in his assessment: this was not a technical glitch but a failure of oversight.

The event underscores the critical vulnerabilities introduced when generative AI is integrated into official administrative workflows without adequate human verification. Because the AI-generated citations appeared structurally plausible and matched the formal tone of academic referencing, they bypassed internal review processes and were released to the public as part of a legitimate draft policy.

The Mechanics of the Incident

The publication of the fabricated citations was the result of two compounding factors: technological error and a breakdown in human verification.

• AI Hallucination: Generative AI models predict the next most likely word based on their training data. When asked to draft or summarize a document, the AI can confidently generate text that looks structurally and linguistically correct but is entirely invented. In this instance, the AI tool fabricated academic citations rather than drawing from real, verifiable sources.
• Lack of Human Verification: AI tools are designed to assist, not replace, human judgment. The fabricated citations advanced to the Government Gazette because personnel involved in the drafting and approval pipeline failed to cross-reference the AI output against actual published academic sources. The professional formatting and authoritative appearance of the hallucinated references masked their factual inaccuracy.

Risks in AI-Assisted Governance

The South African incident highlights several serious risks associated with using AI in public administration and policy development.

• Erosion of Public Trust: The publication of a policy document containing fabricated sources undermines the credibility of government institutions and the reliability of official record-keeping systems like the Government Gazette.
• Legal and Regulatory Confusion: If a policy built on fictitious academic foundations is perceived as legitimate, it can cause confusion among citizens, businesses, and legal professionals who rely on official publications for compliance and planning.
• Operational Disruption: Retracting a published government document requires administrative resources, public corrections, and internal audits, diverting time and attention away from actual governance work.

Preventing Future Occurrences

To safely leverage AI in drafting sensitive or official documents, organizations and government bodies must implement strict operational safeguards.

• Mandatory Source Verification: Every citation, reference, and claim generated by an AI must be independently verified against primary source material before approval. This means confirming that cited authors, journals, and publications actually exist.
• Strict Review Pipelines: Implementing multi-tiered approval processes ensures that no single point of failure allows unverified AI content to reach publication.
• AI Literacy Training: Personnel using AI tools must be trained to recognize the limitations of the technology, specifically the tendency of generative models to produce plausible but entirely false information with apparent confidence.

Summary

The withdrawal of South Africa’s Draft National AI Policy from the Government Gazette serves as a clear case study in the risks of AI-assisted governance. At least 6 of the document’s 67 academic citations were confirmed fabrications, most likely generated by an AI drafting tool and never verified by a human reviewer. While AI tools offer real efficiency gains in drafting and administrative tasks, their capacity to confidently produce false information requires rigorous human oversight, strict verification protocols, and a genuine understanding of AI limitations to prevent serious institutional errors.