Avoid the “false claims” of AI hallucinations when litigating FCA matters

While commentators continue to predict that AI will fundamentally alter the legal profession, and in many cases greatly reduce or eliminate the need for lawyers, real world examples continue to demonstrate that AI is far from perfect. AI outputs can be prone to “hallucinations,” which are assertions that sound confidently true, but are actually false, fictitious, or misleading. In the legal sphere, such hallucinations may manifest as made-up caselaw, fake statutes, or incorrect legal assertions.

Documented instances of AI hallucinations in court submissions are on the rise. As of September 19, 2025, one researcher has identified 244 court opinions that note fabricated or incorrect material submitted due to AI, and one study identified 22 cases between June 30 and August 1, 2025, alone where fictitious cases were identified in courts.

False Claims Act (FCA) litigation – cases in which either the government or an individual suing in the government's name (a “relator”) seeks civil penalties and forfeitures from defendants who defrauded the government¹ – is no exception. In one recent FCA matter, Smith v. Athena Construction Group, Inc., an attorney withdrew from representing a relator after submitting an opposition brief that defendants alleged contained non-existent caselaw, fake quotations, and incorrect legal assertions. The withdrawal motion explained that the attorney had utilized Grammarly and Lexis+, and that co-counsel, who had filed on the attorney's behalf, had not cite checked the brief because there was “no reason to know or suspect that the case citations were inaccurate or false.”

Likewise, in another recent FCA case, Khourey v. Intermountain Health Care Inc., defendants accused the relators' expert witness of submitting reports and other documents with fake citations and information. The expert witness admitted during a deposition to using ChatGPT to assist with his research, and the relators ultimately withdrew the expert report and testimony in question.²

These two recent qui tam actions highlight the potential pitfalls of AI usage in FCA litigation – both by attorneys and experts retained for a matter. These cases also demonstrate that parties litigating FCA matters must keep an eye out for hallucinations in their own filings, as well as those submitted to the Court by other parties.

Attorneys should be sure to check all citations and assertions of any filing before it goes out the door, regardless of whether they have a reason to believe a colleague relied on AI in creating any part of the material. A failure to do so could constitute violations of Federal Rule of Civil Procedure Rule 11 or the Professional Rules of Responsibility, as outlined in Formal Opinion 512 issued by the American Bar Association in June 2024,³ as well as result in sanctions from the Court or other discipline. Attorneys should also closely examine opposing parties' briefs to determine if any cited authorities or purported legal conclusions do not exist, and if so, bring it to the attention of the court.

FCA litigators should additionally check whether the court they are operating in has any standing orders or local rules regarding AI usage, as federal judges are increasingly adopting such orders to combat issues with AI hallucination.

Similarly, to avoid submitting expert reports that may inadvertently rely on false information, FCA litigants should consider requiring clauses in expert engagement letters that mandate the expert disclose any AI use.

In addition to hallucination risks, experts' use of AI raises credibility and reliability concerns if it is unclear how exactly an AI tool operates and if it can yield consistent, replicable results. Accordingly, if an expert does use AI, he or she must be able to understand and explain how an AI tool generated its result, ensure that the results can be recreated, and confirm its findings to ensure no fake or misleading sources of information were relied upon or cited. Transparency and verification are of paramount importance.

AI is not going anywhere anytime soon. It has the potential to be a useful tool in the FCA litigation space, but litigants must be on the lookout for these potential pitfalls to avoid inadvertently making legal “false claims” of their own.

Authored by Allison Caplis, Rachel Stuckey, and Paget Barranco.