The New Statesman's reporting that large language models have composed text now embedded in UK acts of parliament is not a curiosity—it is a material governance problem for every regulated firm in the UK. The government has spent £476m on AI consultancy since 2022 and deployed LLMs to analyse spending bids in the June 2025 Spending Review. When the bodies that write regulation and regulatory guidance begin to use unauditable AI systems to do so, the downstream effect is clear: firms must now comply with rules they may not be able to fully understand or challenge. The FCA's Consumer Duty (PS22/9), the SRA Code, and PRA SS1/23 all demand that firms demonstrate evidential control over their compliance obligations. If the regulation itself was drafted by a system no one can fully interrogate, that chain of custody is broken.

This is part of a broader pattern. The UK government has adopted AI tooling faster than it has built governance frameworks around AI tooling. We have seen this before: early adopters of Generative AI in financial services and legal practice deployed ChatGPT and Claude on client data without proper data handling protocols, only discovering the gaps when the ICO started investigating. We are now watching the same pattern repeat at the policy level. Regulators and government departments are deploying large language models—tools designed for plausible text generation, not legal precision—to write the very rules that firms must live by. The EU AI Act treats AI used in legislative processes as high-risk. The UK has not yet caught up to that thinking.

Trovix's view is straightforward: using closed, proprietary LLMs to draft legislation or regulatory guidance is incompatible with the transparency and auditability that the regulated sector now expects and the law increasingly demands. Systems like GPT-4, Claude 3, or even Harvey (which some law firms use for contract analysis) can hallucinate, can embed training-data biases, and crucially, cannot be independently audited in real time. When a law firm uses Trovix Aria to assist fee-earners with regulatory interpretation, we build RAG systems that point back to source material—you always know where the answer came from. We do not ask an LLM to invent regulatory truth. The government's approach inverts that. It asks an LLM to invent legislative text. The difference is not academic.

If you are a general counsel at a mid-market law firm, insurer, or accountancy practice, here is what you need to do now. First, assume that regulation you receive from government or regulators over the next 12 months may have been drafted with AI assistance, and you will not be told. Second, use Trovix Watch to monitor regulatory change with the assumption that you need to understand not just the content but the provenance. Third, when new guidance lands, do not assume it is internally consistent—ask your compliance team to test it for logical contradictions, which are a signature of LLM-generated text. Fourth, document your own interpretation of the rule in writing and keep that alongside the rule itself. If a regulator later challenges your compliance posture, you will need to show that you applied good faith, reasonable interpretation to an instruction set that may itself have been generated by a black box. The responsibility is now yours to impose the rigour the government did not.

Source: New Statesman