We read courtesy as competence. AI writes courteously by default, which means the warmest reply your business sends might also be the wrongest — and nobody will question it until it costs something.

Why do we trust a well-written wrong answer?

Because for most of human history, effort was the signal. A carefully written letter meant someone had spent time. Time meant care. Care meant they'd probably checked. Courtesy was expensive, so it carried information.

It isn't expensive any more. Fluency is now free, and it decoupled from everything it used to indicate — but the instinct that reads it as competence hasn't updated. This is the quiet risk in automating customer contact, and it's not really a technology problem. It's that the tell we've always used to spot a careless reply has stopped working.

What does this look like in practice?

Someone asks whether your service covers a particular thing. The automated reply is warm, structured, apologises for the ambiguity, explains the position clearly and offers a next step. It reads like your best work.

It's also wrong — the position it explained isn't your position.

Nobody queries it. Not the customer, because it sounded authoritative. Not you, because you skimmed it and it looked right, and looking right is exactly what it's optimised for. The error surfaces three weeks later when someone holds you to a commitment you never made, and by then it's in writing, in your voice, from your address.

Compare that with the same error delivered badly — a curt half-sentence with a typo. Someone would have questioned it immediately. The bad version fails safe. The polished version fails silently. That inversion is the entire problem, and it's why "the output is good quality" is a weaker reassurance than it sounds.

Where should the scepticism go?

Not everywhere — that's exhausting and you'd stop within a week. Put it where wrongness is expensive:

Anything stating a fact about your business. Prices, timelines, what's included, what isn't. These are commitments dressed as information.

Anything with a number. A plausible figure in a tidy sentence gets read past. This is where fluency does the most damage per word.

Anything a customer could reasonably act on. If someone might book a flight, cancel a supplier, or turn down another quote because of your reply, that reply needs a human read.

Anything in a complaint thread. The emotional register of automated apology is disturbingly good, which makes it worse — it can de-escalate a situation while committing you to something you'd never have offered.

None of this means the automation isn't worth running. It means the review time belongs in the cost, which is what scoring a tool by minutes back per pound is for — a system that saves an hour and needs twenty minutes of checking still wins, but only if you counted the twenty minutes.

The labelling line from Friday is the practical countermeasure: force the output to declare which claims are confident and which are inferred, so the polish stops being uniform and the seams come back.

Isn't this an argument against automation?

No — it's an argument for automating the right layer.

Automate retrieval: here's the document, here's the booking link, here's what we published about that. Retrieval is checkable, and being wrong about it is obvious.

Don't automate commitment: what we'll do, by when, for how much. Commitment is a judgement, and judgement is the thing you're actually paid for. That's the boundary Tuesday's escalation rule exists to enforce, and it's the same line running through this whole week.

Here's the checkable part, and it's a good one to hold on to: from 2 August 2026 — today — EU AI Act transparency obligations apply, including telling people when they're dealing with an AI system. Machine-readable marking of AI-generated content follows for existing systems from 2 December 2026, with a simplified route for smaller businesses. (Not legal advice — check your own obligations.)

That requirement is usually read as compliance friction. Read it the other way: the disclosure line is what stops courtesy being mistaken for a person having checked. It reintroduces the signal that fluency destroyed. Telling someone "this reply is automated" makes them read it with appropriate scepticism — which is exactly what you want, because the alternative is trust you haven't earned on that particular sentence.

The thing worth remembering

Your automated systems will be the politest part of your business. Make sure they're also the most bounded. A system that says less, and says honestly when it doesn't know, is worth more than one that says everything beautifully — and it's the only version that's still working for you in six months.

For the checks we run before letting anything answer in our name, the free 5-min read is here: 10 AI Tools That Save 10 Hours a Week.

Related Decodes

• Speed Is Cheap Now. Trust Is the Premium. — the market argument beneath this one.
• August on Autopilot: The 5-Check Trust Test — the week's checklist, in full.
• Your Week Leaks 10 Hours — where the time actually goes.

TL;DR: We read courtesy as competence because effort used to be expensive — but fluency is free now, so a polished wrong answer fails silently where a clumsy one would have been questioned. Put scepticism where wrongness costs: claims about your business, numbers, anything a customer might act on, and complaint threads. Automate retrieval, never commitment — and note that from 2 August 2026 EU AI Act transparency duties require telling people they're dealing with an AI system.