Fake Citations Generated by AI Are Quietly Shaping Australian Policy Debates

Imagine a doctor prescribing treatment based on a clinical trial that was never conducted. That is roughly what is happening inside Australia's parliament right now, except the patient is public policy and the treatment is legislation.

Guardian Australia has revealed that at least 39 submissions to political inquiries contain what appear to be AI-generated hallucinations: fabricated studies, misattributed research, and citations pointing to sources that simply do not exist. The submissions span the political spectrum, from individual citizens to organised advocacy groups, and touch on issues as sensitive as family violence, housing inequality, and climate policy.

For anyone unfamiliar with how these AI systems work, a hallucination is what happens when a large language model like ChatGPT or Claude generates text that sounds authoritative but has no basis in reality. Think of it less like lying and more like a very confident guess. These systems do not know facts the way a person does. They predict the next likely word based on patterns in their training data, with a bit of randomness thrown in for good measure. When the topic is obscure or the data thin, the guessing gets riskier, and the model can invent a study, an author, or a statistic that looks entirely plausible.

This is not a glitch that better engineering will eventually fix. As OpenAI and outside researchers have acknowledged, hallucinations are a structural feature of how these models operate, not a bug waiting to be patched out. They can be reduced through more training data or web-search integration, but they cannot be eliminated entirely.

That inherent unpredictability is now colliding with one of democracy's most trusted mechanisms: the public inquiry.

How the Errors Slip Through

Christian Downie, a professor at the Australian National University's school of regulation and global governance, warns that the consequence could be lawmakers "making decisions based on evidence that doesn't exist." That is not a hypothetical. One submission to an inquiry into family violence and suicide included a hallucinated reference wrongly attributed to Divna Haslam, a University of Queensland clinical psychologist. It misstated her actual research findings.

Haslam described the fake citation as "scary" precisely because it looked so convincing. Google's AI summary tool compounded the problem, summarising the fabricated reference as though it were a genuine paper. That is the disturbing feedback loop investigators uncovered: a hallucinated reference gets published in a government submission, search engines and AI crawlers index it, and then AI summary tools present it back to the public as legitimate scholarship, sometimes even citing the original submission as their source.

The organisation behind that particular submission, Drilldown Reports, told Guardian Australia it had identified the errors after the fact but missed the deadline to correct them. A spokesperson insisted the mistake was human error in uploading the wrong document, not a failure of AI itself. Haslam sees it differently. "It's very frustrating to have invested time, money, effort, expertise in rigorous research," she said, "then to see something that's inaccurate and inappropriately attributed anyway is really concerning."

Other cases follow a similar pattern. A submission to a housing inequity inquiry cited nonexistent work attributed to University of Sydney planning professor Nicole Gurran. She noted that citations exist precisely to make research contestable and verifiable, and that even an accidental "collage" of a correct claim stitched to a broken evidence chain undermines that entire system.

Perhaps the most striking example involved journalist and academic Margaret Simons. A submission to a Senate inquiry into climate misinformation, filed by a group called the National Rational Energy Network, claimed the Guardian was "left-leaning, activist-oriented" and attributed part of that claim to a supposed academic article by Simons. She never wrote it. The fabricated citation was so convincingly formatted that Simons, who sits on the board of the Guardian's owner, briefly second-guessed her own memory. "Even though I know I didn't write this," she said, "I did have that moment of self-questioning." The network did not respond to requests for comment.

To find these cases, journalists built a program that extracted every reference from submissions made to the current parliament and cross-checked them against academic databases including CrossRef and Google Scholar, verifying digital object identifiers where available. Documents where 20 percent or more of citations could not be matched were then checked manually, with many verified directly against the original authors. More than 100 papers also contained ChatGPT's automatically generated link tags, a telltale sign the platform had been used somewhere in the drafting process.

This method almost certainly understates the true scale of the problem, since it can only catch AI-generated text that includes fabricated citations. Submissions using AI to generate uncited claims would slip through entirely.

The parallels to the private sector are already visible. Last year, consultancy giant Deloitte issued a partial refund to the federal government after a $440,000 report was found to contain fake references, including a fabricated court citation. If a firm with Deloitte's resources can let that slip through, the risk for individual submitters working without dedicated fact-checking teams is considerably higher.

Why It Matters

The stakes here go beyond any single flawed submission. Downie frames the deeper danger in institutional terms: fake material embedded in government documents or court judgments risks eroding "the public's trust and confidence in the types of institutions that underpin our democracy." Haslam draws the line even more sharply when it comes to family violence policy specifically. "We just can't risk that," she says.

Senate guidance already warns submitters that AI use carries risks to accuracy and that responsibility for correctness sits with whoever submits the document, not with the software they used. But guidance alone has not stopped the problem from growing. Downie argues that the inquiry process must stay open to public participation, yet new guidelines may be needed to actively "encourage truthfulness" among submitters relying on AI tools.

Google, for its part, says its AI Overviews function "like traditional Search," surfacing web pages that match a query's terms rather than independently verifying them. OpenAI recommends treating ChatGPT outputs as a first draft rather than a finished source, urging users to verify quotes, data and references before relying on them. Both responses place the burden of verification back on the human user, precisely the safeguard that appears to be failing across dozens of Australian parliamentary submissions right now.