The New Statesman has confirmed what should alarm every regulated firm in the UK: large language models have written portions of acts of parliament, and AI systems analysed departmental spending bids in the June 2025 Spending Review, influencing billions in public allocation across housing, schools and hospitals. This is not theoretical. Government officials have admitted it. The FCA, PRA, SRA, FRC and ICO are now operating within a legislative and regulatory framework that may have been drafted or shaped by systems with known failure modes: hallucination, bias amplification, prompt injection vulnerabilities and training data corruption. If your firm is regulated under frameworks that were written or influenced by unaudited LLMs, the question is not whether you should worry about AI governance. The question is how you survive in a system where your regulator may not know what rules actually govern you.

This story is the logical endpoint of an industry-wide pattern. For four years, AI vendors have sold 'efficiency' and 'scale' to regulated firms without building corresponding governance infrastructure. Products like Harvey and Luminance have found genuine use in legal document review and contract analysis—narrow, bounded tasks where output can be validated by a human expert. But those same LLM architectures are now running in government departments where no validation is possible. The result: rules are made by systems nobody understands, allocated using logic nobody can audit, enforced by regulators who themselves may not know whether their guidance came from a person or a prompt. This is not innovation. It is governance by accident.

Trovix's position is blunt: AI in regulated environments must be auditable, traceable and constrained to tasks where failure is recoverable. That means rejecting the generalist LLM approach for most compliance and regulatory work. When a lawyer uses a RAG system like Trovix Aria (/solutions/trovix-aria) to search case law or a claims handler uses Trovix Sift (/solutions/trovix-sift) to extract policy data, the system is anchored to source documents that can be verified. When a firm uses Trovix Audit (/solutions/trovix-audit) to track how AI decisions were made, every choice is logged and explainable. This is not as flashy as ChatGPT with a legal wrapper. It is also not the approach the government took. The difference matters now, because you cannot build a sustainable practice on unauditable AI—and your regulators, whether they know it or not, are now trying to do exactly that.

If you are a mid-market law firm, insurer, accountancy practice or financial services business: assume that at least some of the rules you operate under were drafted or influenced by AI systems you have never seen. Your regulators have no audit trail. Your compliance team cannot call the FCA and ask 'which bits of COBS 2.1R did a language model write?' Instead, build AI governance that is the inverse of government practice. Every AI system your firm uses must produce an audit trail. Every high-stakes decision—underwriting, legal advice, tax position, claims reserve—must be explainable to a human expert and a regulator. Do not wait for the government to fix this. That era has passed. Treat unauditable AI as a compliance risk equivalent to a rogue trader or a data breach.

Source: New Statesman