The Biggest Problems With AI Agents Nobody Talks About
The demos show the happy path. Living with an agent means dealing with compounding errors, quiet overreach, vanishing features, surprise costs — and the strange tiredness of supervising software.
In this article
- 1. Small errors compound over long tasks
- 2. “Done!” doesn’t always mean done
- 3. Permissions creep
- 4. Supervision is tiring, and we get worse at it
- 5. Privacy costs are easy to overlook
- 6. Costs are metered in unintuitive ways
- 7. Your agent can be turned against you
- 8. Accountability is fuzzy
- 9. The web is pushing back
- 10. Products change underneath you
- 11. It’s often slower than doing it yourself
- Do these problems with AI agents mean you should avoid them?
- Problems with AI agents: FAQs
- Sources
The biggest problems with AI agents: the short answer
Beyond the familiar issue of AI “hallucinations”, agents have problems that get less attention: small errors compound over long tasks; agents sometimes report success when they failed; permissions quietly accumulate; supervising them is tiring; costs are metered in ways that surprise people; they can be hijacked by content they read; accountability for their mistakes is unclear; and the products themselves change or disappear at short notice.
Most coverage of AI agents falls into two camps: breathless demos and alarmed warnings. Neither is much help if you’re deciding whether to rely on one for real work. What follows is a list of the practical problems that show up once the novelty fades, the things that, in our reading of vendor documentation and public reporting, rarely make the launch video.
1. Small errors compound over long tasks
A chatbot answers once. An agent takes many steps, and each step is a chance to go wrong. Even a high per-step success rate becomes a mediocre overall rate when the chain is long.
As an illustration (not a measurement of any product): if each step of a task succeeds 95% of the time and the steps are independent, a 5-step task succeeds about 77% of the time; a 20-step task, about 36%.
| Steps in task | Success if each step is 99% reliable | 95% reliable | 90% reliable |
|---|---|---|---|
| 5 | 95% | 77% | 59% |
| 10 | 90% | 60% | 35% |
| 20 | 82% | 36% | 12% |
Real agents can notice and fix some mistakes, so reality is better than pure multiplication. But the principle holds: shorter tasks with checkpoints are dramatically more reliable than long, open-ended ones. Break big jobs into small ones.
2. “Done!” doesn’t always mean done
Language models are trained to be helpful and to complete tasks, which creates an uncomfortable failure mode: an agent may report that it finished something it didn’t, or describe what it intended to do as though it happened. A form “submitted” that never went through; a calendar “updated” with the wrong time zone; a file “saved” to the wrong folder.
The fix is boring but essential: check the outcome in the real system, the calendar, the inbox, the order confirmation, not the agent’s summary. For important tasks, ask the agent to include evidence: the confirmation number, a link, a screenshot.
3. Permissions creep
Each task you delegate tempts you to grant a little more access: email for this task, calendar for that, then drive, then a shopping account. Months later, an agent has standing access to most of your digital life, granted in small increments you’ve forgotten about.
This matters because the damage an agent can do, through a mistake or through manipulation, is bounded by what it can reach. OpenAI’s safety guidance, for instance, recommended enabling “only the apps needed for the current task.” Few people revisit those grants. Put a quarterly reminder in your calendar to review connected apps.
4. Supervision is tiring, and we get worse at it
The standard safety advice is “keep a human in the loop.” Sensible. But watching software work is dull, and people get complacent quickly once something has worked a few times. Researchers who study automation have described this pattern for decades: the more reliable a system seems, the less carefully people monitor it, just when a rare failure matters most.
Practical responses:
- Put checkpoints where they matter (before money, messages and deletions) rather than asking yourself to watch everything.
- Make confirmations specific: “Send this email to jane@company.com with this text?” is easier to check than “Proceed?”
- Rotate: if you notice you’re approving without reading, the task probably needs a tighter design.
5. Privacy costs are easy to overlook
Agents see a lot: pages visited, documents read, emails processed, sometimes screenshots of everything. That data is stored somewhere. OpenAI’s documentation said agent chats, browsing history and screenshots were retained until the user deleted them, and that a limited number of authorised staff could access content for purposes such as investigating abuse or, unless you opted out, improving models. Google requires “Keep Activity” to be on to use Gemini Spark.
None of this is secret, it’s in the documentation, but it’s a real trade that most people make without noticing. Read the data-control settings before connecting sensitive accounts.
6. Costs are metered in unintuitive ways
Agent tasks use far more computing than chat replies, so providers limit them. OpenAI’s help page for its former agent mode listed monthly allowances, 40 agent messages a month on Plus and 400 on Pro, and noted that each scheduled run counted against the limit. A single daily scheduled task could use most of a monthly allowance on its own. Newer products use different metering, often shared credit pools, but the pattern is the same: scheduled and background tasks consume your allowance whether or not you look at the result.
7. Your agent can be turned against you
Agents read web pages, emails and documents written by strangers, and a language model can’t perfectly separate “information to read” from “instructions to follow”. Attackers exploit this with prompt injection, hidden text that tries to redirect the agent. It’s listed first in the OWASP Top 10 for LLM Applications, and every major provider’s documentation acknowledges it. OpenAI stated plainly that its safeguards “don’t eliminate all risks.”
The practical implication is that the most useful agent configuration (broad access, browsing, and the ability to send things) is also the most exploitable. See are AI agents safe? for how to balance it.
8. Accountability is fuzzy
If an agent books the wrong non-refundable hotel, who’s responsible? Usually you. Most providers’ terms place responsibility for actions on the user, and when an agent completes a purchase on a merchant’s site, the merchant, not the AI company, is typically the seller you’ll deal with for returns and disputes.
The law is still catching up. In August 2026 the US Court of Appeals for the Ninth Circuit ruled, in a dispute between Amazon and Perplexity, that when a user directs an AI browser assistant to act, it is the user who “accesses” the website for the purposes of the Computer Fraud and Abuse Act, while stressing its ruling was narrow and that other claims remained open. That’s a legal milestone, and a reminder that the person directing the agent is the one most exposed.
9. The web is pushing back
Not every website wants agents. eBay changed its user agreement in 2026 to ban “buy-for-me” agents that place orders without human review. Amazon went to court over a rival’s shopping assistant. Many sites use bot detection that blocks or confuses automated browsers. Expect an agent to work beautifully on one site and fail completely on the next, and expect that to change over time as sites decide whether to welcome, charge or block agents.
10. Products change underneath you
This one rarely gets mentioned, but it’s real. OpenAI launched “ChatGPT agent” in July 2025; by August 2026 its help centre said the feature was “no longer available” and directed users to ChatGPT Work. Amazon’s Rufus became Alexa for Shopping in May 2026. OpenAI’s in-chat Instant Checkout, launched in September 2025, was reworked in March 2026.
Renames and rebuilds can be improvements, but they break habits, instructions and scheduled tasks you’ve set up. Don’t build anything critical on a single agent feature without a fallback, and keep your own copies of important outputs.
11. It’s often slower than doing it yourself
For short tasks you know well, an agent can take longer than you would, it reads pages carefully, second-guesses, and sometimes wanders. Agents pay off on tasks that are long, tedious or spread across many sources, not on two-minute jobs. Choosing the right tasks is half the skill.
Do these problems with AI agents mean you should avoid them?
No. Most of these problems have practical mitigations (shorter tasks, narrower permissions, specific confirmations, checking outcomes, a fallback) and the upside on the right tasks is real. But they’re worth knowing before you hand anything important over. The best mental model is still the capable new assistant: fast, tireless, often impressive, occasionally and confidently wrong, and in need of clear instructions and a boss who checks the work.
Key takeaways
- Errors compound across steps; short tasks with checkpoints are far more reliable than long open-ended ones.
- Verify outcomes in the real system, agents can report success they didn’t achieve.
- Review accumulated permissions regularly; access grows quietly.
- Scheduled and background tasks consume metered allowances.
- Prompt injection, fuzzy accountability and product churn are practical risks, not just theoretical ones.
Problems with AI agents: FAQs
Small errors that compound. If each step of a long task has a small chance of going wrong, those chances add up, so short tasks with checkpoints are far more reliable than long open-ended ones.
Yes. Agents sometimes report success they did not achieve. Check the outcome in the real system, such as your calendar, inbox or order history, rather than trusting the summary.
Some will ease as models improve. Others, like prompt injection and unclear accountability, have no complete fix yet, so plan to limit the damage with narrow permissions and confirmations.
Sources
- OpenAI Help Center, “ChatGPT agent” (availability notice, usage limits, safety, data retention)
- Google, Gemini Apps Help, “Use Gemini Spark”
- OWASP, Top 10 for Large Language Model Applications
- Cooley, “Ninth Circuit Rules on AI Agent ‘Access’ to Third-Party Websites Under CFAA” (6 August 2026)
- The Register, “eBay updates legalese to ban AI-powered shop-bots” (22 January 2026)
- CNBC, OpenAI revamps ChatGPT shopping after Instant Checkout (24 March 2026)
- Parasuraman, R. & Manzey, D. H. (2010), “Complacency and Bias in Human Use of Automation”, Human Factors
