On 13 August, Musick Peeler & Garrett admitted that motions filed on behalf of State Farm contained citations to nonexistent cases and fabricated legal quotes. The errors were caught by plaintiff's attorneys, not by the firm's own quality controls. For UK regulated firms—particularly in insurance, accountancy and financial services—this matters acutely. The SRA Code of Conduct requires solicitors to act with competence and integrity. The FCA Consumer Duty PS22/9 holds firms accountable for the accuracy of advice and documentation. Filing court papers with hallucinated citations is not a minor slip. It is a breach of professional duty, a reputational catastrophe, and—potentially—grounds for sanctions under the EU AI Act and the ICO's AI governance framework.
What State Farm's case reveals is a widespread misunderstanding of what large language models actually do. Generic tools like Microsoft Copilot or unvetted third-party integrations (including some marketed directly to law firms) generate plausible-sounding text. They do not verify. They do not check. They confabulate with confidence. The legal tech market has spent two years selling AI as a research accelerator. What the market has not emphasised is that every output requires human expert verification before it leaves the firm. Some vendors—Harvey, Legora, and Luminance—have built domain-specific models trained on verified legal databases, which reduces (but does not eliminate) hallucination risk. But even these require gate-keeping. State Farm's failure was not that they used AI. It was that they trusted it.
Trovix's approach to AI governance rests on a principle: outputs are only trusted outputs if they are auditable and verified by the person responsible under regulation. That is why Trovix Audit exists—to create a compliance record showing which AI tools touched which documents, what they generated, and who signed off on it. For insurance firms processing claims, underwriting memos, or regulatory correspondence, that trail is non-negotiable. For accountancy practices generating tax positions or financial commentaries, it is critical. For legal teams, it is survival. You cannot say to the SRA that 'the AI did it' any more than you can blame a paralegal for not checking their own work. The responsibility sits with you. The governance must sit with you too.
If you are a mid-market firm in the UK right now, here is what you should do on Monday morning. First, audit what AI tools your teams are actually using—not what you have approved, but what they have installed. Second, impose a mandatory verification layer: no AI-generated legal research, regulatory analysis, or client-facing document goes out without a named individual reviewing it for accuracy and flagging it as verified. Third, document that process in your AI governance dashboard so you have an audit trail if regulators ask. Fourth, train staff that AI is a first-draft tool, not a finished product. This is not about banning AI. It is about using it responsibly under the rules you operate under.
Source: Law360