AI agents are getting smarter, but memory is still the mess. When an agent has to plan, act, check results, and keep going for hours or days, the hard part is...
AI Agents Need Better Information for Long-Horizon Work
AI agents are getting smarter, but memory is still the mess. When an agent has to plan, act, check results, and keep going for hours or days, the hard part is not just reasoning — it is feeding the model the right facts at the right time. I’m Ermenildo V Castro, and after years of building systems at Google, Microsoft, Amazon, and Zulily, I can tell you this much: clever code fails fast when the context is thin.
Key Takeaways- Long-horizon agents need more than prompts. They need good retrieval, clean state, and fresh context.
- The real problem is not raw model power. It is information quality, timing, and structure.
- Memory, tools, and task state must work together, or the agent wanders.
- Simple systems beat fancy ones when the job is messy and long.
- Good agent design is also a matter of stewardship: use data carefully, reduce waste, and serve people well.
What Are AI Agents and Why Does Information Supply Matter?
AI agents are software systems that can reason, plan, use tools, and take steps on their own. They are not just chatbots with a bigger hat. They can inspect files, call APIs, search the web, write code, and keep working across many steps. That sounds neat. It is also where things break.
The deeper the task, the more the agent needs context. A short request like “summarize this PDF” is easy. A long task like “monitor a customer issue, inspect logs, ask the right service, compare results, retry safely, and report back” is a different beast. The model must remember goals, constraints, past actions, and new facts. If any part goes stale, the whole thing slips.
I saw this pattern again and again. At Amazon, systems that looked smart in demos often fell apart once real data, edge cases, and noisy logs entered the room. Why? Because the model guessed when it should have known. At Microsoft, I watched teams spend too much time polishing prompts and too little time shaping the state that fed the prompts. That is backwards. Prompting matters, sure, but prompt quality without proper context is like giving a pilot a good speech and no radar.
The best mental model is simple. An agent is a worker. Workers need tools, notes, and a clear desk. They also need to know what changed. The same holds for AI agents. If you want reliable long-horizon work, you must treat information as a first-class system, not a side detail.
For a broader look at agent design, OpenAI’s work on agents and tools is worth reading: OpenAI function calling. Anthropic also explains tool use and structured thinking well: Anthropic tool use. And if you care about retrieval systems, Google’s own material on retrieval-augmented generation is a solid anchor: Google Cloud RAG guidance.
Core Benefits of Supplying Agents with Better Information
Better information changes everything. No magic. Just fewer mistakes and less nonsense.
- Higher accuracy: When an agent pulls from current facts, it makes fewer dumb calls. I have seen teams cut error rates just by tightening the source set and removing stale documents.
- Longer task survival: Agents working over many steps need memory that holds up. If the task is six hours long, yesterday’s assumptions can sink it.
- Less retry spam: A well-fed agent does not keep poking the wrong API or re-reading the same bad note. That saves time and money.
- Cleaner handoffs: Good state makes it easier for one agent to pass work to another. The next worker should not need a detective kit.
- Better human trust: People trust systems that can explain what they used and why. A clean trail beats fuzzy “trust me” behavior.
Frankly, this is where a lot of teams get lazy. They chase benchmark scores and ignore the lived mess of production. But real users do not care that your demo scored high on a blog post test. They care whether the system handled the case without inventing facts.
When I worked on large-scale systems, I learned that “more data” is not the same as “better data.” You can drown a model in junk and still get a bad answer. The trick is selectivity. Feed the agent the pieces that matter, not the whole attic.
There is also a moral side to this. Good system design respects people’s time and effort. It does not waste compute, and it does not pretend certainty where there is none. That kind of care matters. Work should serve the common good, not just a demo slide.
If you want the retrieval side explained in practical terms, this guide on retrieval-augmented generation best practices helps. If you are building workflow-heavy systems, the lessons from agent tool use patterns are useful too.
How to Keep AI Agents Supplied with the Right Information
This is where theory gets shoved out the door. Good.
- Define the task boundary first
Start with the exact job. What must the agent finish? What counts as done? What is off-limits? If you do not pin this down, the agent will chew on the wrong things. I’ve seen teams ask for “autonomy” when they really wanted a checklist. - Store state separately from prompts
Do not cram everything into one giant prompt. That is sloppy. Keep task state, tool outputs, user preferences, and history in structured storage. Use JSON, tables, or a small state machine. The prompt should point to the facts, not carry the whole truck. - Use retrieval, not memory theater
People love talking about “memory.” Most of it is smoke. Real memory should be searchable and current. Use embeddings, keyword search, metadata filters, and ranking. Then give the agent only what fits the step it is on. - Refresh context at decision points
A long task should pause and ask, “What changed?” That means new logs, new documents, new API results, new user input. I learned this the hard way on systems where a ten-minute-old assumption caused a stupid cascade. - Separate stable facts from noisy facts
Stable facts are things like policies, account rules, and known constraints. Noisy facts are logs, live events, and one-off messages. Mixing them together is asking for trouble. Keep them in different buckets. - Add a verification layer
Every serious agent needs a check. Compare answers against tools, source docs, or a second pass. This is not paranoia. It is basic discipline. As the old wisdom goes, measure twice, cut once. - Log what the agent saw
If the system fails, you need a trail. What context was loaded? Which source won? Which tool output was used? Without that, you are guessing in the dark. - Keep humans in the loop for risky moves
Full autonomy is not always wise. In healthcare, finance, identity, or legal work, human review is not a crutch. It is responsibility. People have dignity, and systems should respect that.
I like to think of this as tending a workshop. A carpenter does not work well with broken saws, missing nails, and no bench. An agent is no different. Give it structure, and it can work. Give it a junk pile, and it will improvise badly.
For a practical example of how tools fit into this process, see OpenAI’s function calling overview. For agent orchestration ideas, many teams also study workflow patterns from real-world systems, not just toy demos.
AI Agents vs. Traditional Automation
| Feature | AI Agents | Traditional Automation |
|---|
| Reasoning | Can adapt to new inputs and changing goals | Follows fixed rules only |
| Long-horizon work | Can continue across many steps if context is managed well | Good at repeatable tasks, weak at flexible tasks |
| Information needs | Needs fresh retrieval, memory, and state management | Needs predefined inputs and scripts |
| Error handling | Can inspect context and choose next steps | Usually stops or fails on unexpected cases |
| Flexibility | High, but harder to control | Lower, but simpler to predict |
| Maintenance | Needs monitoring, evaluation, and guardrails | Easier to keep stable if the process stays fixed |
The biggest competitor to agents is not another model. It is plain old automation. Why? Because automation is cheap, clear, and boring in a good way. It works when the path is known. Agents shine when the path changes, but only if the information supply stays clean.
Most teams compare agents to chatbots. That is too small. The real comparison is against scripts, workflows, rule engines, and human operators. If a task is stable, a script may beat an agent every time. If the task is messy, changing, and full of exceptions, the agent can win — but only with disciplined context design.
I learned this in e-commerce systems at Zulily. Some flows were perfect for rules. Others needed judgment, branching, and fresh signals. The lesson was simple: don’t use a forklift to carry a spoon. The right tool matters.
Common Mistakes to Avoid
Here’s the kicker. Most bad agent systems fail for boring reasons.
- Stuffing too much into the prompt: People think a longer prompt means a smarter agent. Nope. It often means confusion, token waste, and slower runs.
- Using stale memory: Old facts stick around like bad leftovers. If you never refresh context, the agent makes confident mistakes.
- Skipping source ranking: Not every document deserves equal weight. Without ranking, the agent may grab the loudest lie in the room.
- Ignoring tool quality: If your tools return junk, your agent will too. A weak API, messy schema, or vague error message drags the whole system down.
- No eval set: You need test cases that mimic real tasks. Benchmarks are fine, but they are not your actual users.
- Too much autonomy too soon: Letting an agent act without guardrails is how you buy a headache. Start narrow, then widen the leash.
I am skeptical of the “just add more context” advice. That line gets repeated a lot because it sounds smart. In practice, it is lazy. More text is not better if it is the wrong text. The agent needs the right slice, at the right moment, in the right form.
Another mistake is forgetting that people matter. Systems should help workers do honest labor better, not bury them in cleanup. Good design respects time, clarity, and fairness. That is not soft talk. It is good engineering.
I have seen teams chase flashy demos while ignoring evals, logging, and source control for prompts. That is backwards. Build the boring stuff first. The boring stuff keeps the lights on.
If you want more on structured control and tool use, revisit Anthropic’s tool-use guidance and Google Cloud’s RAG overview. The details matter.
Frequently Asked Questions
What makes long-horizon AI agents hard to build?
Long tasks need memory, fresh context, tool access, and good error handling. The longer the task runs, the more chances there are for stale facts and bad assumptions to creep in.
Is prompt engineering enough for AI agents?
No. Prompting helps, but it is only one piece. You also need retrieval, task state, tool design, evaluation, and logging.
Should every AI agent have memory?
Not in the same way. Some tasks need short-lived context, while others need persistent memory. Use the lightest version that still does the job.
When should a human review an agent’s output?
Any time the task affects money, safety, rights, or important decisions. That is just basic care and responsibility.
Final Thought
The future of agents will not be won by the loudest model demo. It will be won by teams that respect information, structure, and the people who rely on the system. I’ve spent enough years in big tech to know that fancy reasoning is nice, but clean inputs, honest checks, and good judgment do the real work. Ernie Castro here, still stubborn about the basics: if you feed an agent junk, do not act shocked when it gives you junk back.
The better path is plain. Keep state tidy. Refresh context often. Use tools that tell the truth. Leave room for human judgment where it matters. That is how you build agents that can work for hours without drifting off course. And if your system serves people well, with care and fairness, that is not just solid engineering. It is the right thing to do.