Owners on the forums say AI tools often do most of a job and hand back a last piece that takes as long as doing it themselves. That last step is what this research tested: if a product leaves the final step to you, it is writing you a draft, whatever the marketing calls it.
People who build and buy these agents describe the same gap:
- A builder who sets up automations for small companies says a client’s supplier-order agent looks like it runs on its own, but it gets stuck about twice a week and he fixes it before the client notices.
- An owner was pitched a $2,000 a month chatbot. What he needed was 5,000 badly labeled supplier part numbers entered correctly into his inventory system. He estimated that paying someone a few hundred dollars a month to type them in would cost less and produce fewer mistakes.
These are individual accounts, not measurements. Three more places products stop short:
In support, products lose track of the customer. Someone who writes in by chat, then emails, then calls is treated as three different people and has to explain the problem again each time, because the product links the conversation to the chat window rather than to the customer.
Voice agents get numbers wrong, and the advertised accuracy figure hides it. A team running voice agents for real customers reports that an advertised 95 percent accuracy at understanding speech concealed the failure that matters. Roughly half the time somebody spoke a bank code into the phone, at least one character came back wrong. Numbers said out loud were the worst of it, because “double five”, “fifty five” and “five five” were each heard as something different. Taking down a phone number, an address or a card number is most of what a receptionist is for, so getting numbers wrong matters far more than the overall accuracy figure suggests.
In reporting, a simple agent lasted longer than an ambitious one. A builder made a weekly client report agent that gathered the figures, explained them and proposed what to do next. It looked good in a demonstration and did not last in real use. When its reasoning came out wrong, nobody could see at what point it had gone astray, and when it came out right, it still did not get trusted. The version still running puts the figures into the same headings every week and writes one flat sentence under each saying which way the number moved.