The FCA's statement on Monday is unambiguous: regulated firms are deploying AI systems, then retrofitting governance frameworks around them. Some are using consumer-facing chatbots like ChatGPT and Gemini without any formal risk assessment first. Others are monitoring systems inconsistently as the underlying models update. This is not edge-case misadventure. This is how mid-market law firms, insurers and financial advisory practices are currently rolling out AI. The regulator has now flagged it as a material breach waiting to happen. Under FCA Consumer Duty PS22/9, you are required to understand material risks to consumers before they materialise. Generic AI tools deployed without governance frameworks create exactly those risks—and saying 'we use ChatGPT' is not a governance framework.

What the FCA is describing is a systemic failure in how the industry has approached AI adoption. The problem is not ChatGPT or Gemini themselves. The problem is that those tools were designed for consumer convenience, not for regulated financial advice. Yet firms have grafted them directly into client-facing workflows without the compliance architecture that FCA SS1/23 and PRA guidelines now expect. This pattern—rushing deployment, adding controls later—echoes earlier failures in algorithmic decision-making and outsourced operations. It reveals an industry that still treats AI as a consumer technology to be adopted, rather than a regulated tool to be governed. The FCA's alarm is not new regulation. It is enforcement notice in paragraph form.

Trovix's position on this is straightforward: governance must precede deployment, not follow it. That means mapping your intended use case against your regulatory obligations before you put any AI system into production. It means understanding what decisions the AI will influence, who will review them, and how you will monitor for drift when vendors push model updates. It means documenting your risk appetite and testing your monitoring against actual transaction patterns—not generic threat matrices. Systems like Harvey and Luminance have built governance checkpoints into their architecture from the start; they require you to define use cases, data sources and human oversight before they deploy. That is expensive, but it is defensible. The alternative—retrofitting governance onto ChatGPT or Claude instances already in use—leaves you exposed to the exact regulatory observation the FCA just made public. Trovix Watch was designed to help mid-market firms track these regulatory shifts in real time so you are not caught reactive.

If you are a mid-market law firm, accountancy practice, insurer or financial services business, here is what to do this week: audit every AI tool currently in use by your staff. How many are generic consumer products? For each one, ask: what decision does this influence, who checks the output, and how do we evidence that checking? If you cannot answer those questions with documentation, that tool should not be in production. Second, before deploying any new AI—whether that is Trovix Aria for research, Sift for document intelligence, or Brief for intake automation—insist that governance design happens before day one. Require your vendors to map their systems to your regulatory obligations. Third, commit to ongoing monitoring. The FCA specifically called out firms that fail to adapt monitoring as systems change. Your vendor's model updates do not pause your compliance obligation. Build that into your vendor management and your budget.

Source: City AM