The FCA is sounding an alarm that should worry every regulated firm using AI. Last week it warned that too many companies are letting generic platforms like ChatGPT and Gemini drift into advisory territory, with inadequate governance bolted on afterwards. The regulator is right. This isn't a problem confined to fintech startups operating outside the perimeter. It's happening inside regulated firms — law practices, insurers, accountancies and wealth managers — where fee-earners are using LLMs as pseudo-advisors without proper validation, audit trails or consumer-facing risk disclosure. Under FCA Consumer Duty PS22/9, this is a breach. The SRA Code section 2 makes it worse for legal practices. And when the EU AI Act enforcement kicks in properly next year, retrofitted governance won't save you.

What the FCA has identified is a widespread assumption that AI governance is a compliance checkbox you complete after the system goes live. It isn't. The firms doing this are treating LLMs like they treat email — plug it in, use it, sort out the rules later. That worked in 2020. It doesn't work now. The real pattern here is that generic AI products like ChatGPT and Gemini are designed for productivity, not liability management. They don't produce the audit trails that FCA Handbook section DISP or PRA SS1/23 demand. They don't flag conflicts of interest. They don't track who advised whom or why. They hallucinate. And they do all of this silently. Mid-market regulated firms are now caught between wanting the efficiency gains those tools promise and the governance reality their regulators demand.

Here's Trovix's honest take: you cannot retrofit AI governance. Governance must be designed in from the moment you decide to deploy AI, not added afterwards. That means three things. First, you need to know exactly what your AI system is actually doing — which means controlled, domain-specific systems that work within your firm's knowledge, not generic LLMs that roam across the public internet. Systems like Trovix Aria are built for legal and financial teams with built-in audit trails and knowledge boundaries precisely because generic tools don't have them. Second, you need real-time monitoring that flags drift or unexpected behaviour. Third, you need human sign-off gates for any decision that touches client risk. Compare this to how some vendors sell 'AI co-pilots' that simply wrap ChatGPT with a dashboard. They're selling you the same compliance problem with better branding.

If you are a mid-market law firm, insurer or accountancy practice, here is what you should do this week. One: audit your AI use right now. Find every instance where your teams are using ChatGPT, Copilot, Claude or Gemini for tasks that touch client advice, underwriting decisions or recommendations. Two: classify which of those uses actually require governance (hint: most of them do). Three: understand that your existing data governance framework probably doesn't extend to AI systems — you need to extend it. Four: if you are building or planning to build internal AI systems, start with governance architecture, not feature roadmaps. The FCA is not going to ease up on this. Trovix Watch tracks FCA notices and guidance updates specifically for this reason — because the regulatory expectation is hardening, not softening.

Source: City AM