Developers looking inside any production-grade AI agent built today may notice that the ‘simple-prompt response loop’ is dead. This is because agents now use multi-step graphs with classifiers, reviewers, security vetoes, and human approvals instead of single prompt-response loops. Techzine Global explains what’s happening here.
Things change a lot once developers put a classifier (an algorithm that sorts input into categories) in front of a specialist agent, then add a reviewer and a security check with the power to veto. Once we pause for human approval, it’s no longer a loop; it’s a graph.
Tobie Morgan Hitchcock, co-founder and CEO, SurrealDB says that the interesting question now is what happens underneath that graph.
“We’ve been so busy graphing agents, we forgot to properly architect their memory, which still sits in raw text chunks in vector stores. Most agents assemble context by cobbling together a range of data – from vector similarity searches to raw documents – which is messy and inefficient. What’s missing is structured memory, rather than crude context stitching,” he explains.
Execution versus context
The question we need to ask next is… what happens when software engineering teams graph what an agent does, but not what an agent knows? To understand where current AI architectures reach their limits, there are two concepts to unpick: the execution graph and the context graph.
Morgan Hitchcock notes that the execution graph “determines what an agent does next”, so that means where it can run and branch.
“It manages several operational processes including state transitions, retry loops, and fallback paths. The context graph represents what an agent knows. This is where data originates, when it was true (and whether it remains true over time), and how other data is related. It also governs whether an agent has permission to access and act on that information,” he said.
Similarity is not the same as relationships
“When developers hit this wall, the default patch is to throw re-rankers or complex prompt-chains at the problem,” said Morgan Hitchcock. “We fetch fifty results from each, feed them into a re-ranker, and hope a downstream LLM figures out what the query planner should have known in the first place. This isn’t a credible architecture. Putting the massive token bill aside, there’s a more fundamental problem – you can’t clean up bad retrieval after the fact.”
Why GraphRAG isn’t enough
GraphRAG is thought to be a genuine step in the right direction. The SurrealDB CEO says that “Microsoft’s own documentation is clear” that vector-only retrieval is less effective when answering questions that depend on complex relationships between fragmented data streams. GraphRAG connects these strands of information across large volumes of data to answer questions that are difficult to answer using traditional keyword or vector search.
“GraphRAG is useful, but has its limitations. Indexing chunks of text into an external graph, or extracting entities on the fly, treats the graph as a secondary retrieval technique, rather than a reliable source of truth. In production, AI agents don’t just need better retrieval; they need a unified memory layer that solves four persistent engineering problems,” underlined Morgan Hitchcock.
Techniques available to software engineers here include the following:
Entity resolution. Teams cannot throw an LLM at transcripts and extract hundreds of unlinked triples. If one turn calls a field home_city and another city_of_residence, we have a vocabulary problem. If the same customer becomes five different nodes, every traversal starts lying. The context graph must participate in its own construction by reusing established entities.
Temporal knowledge. Facts are impermanent. What was true yesterday may be invalid today. A true context graph tracks tri-temporal state, capturing when a fact was originally recorded in the system, when it was first asserted, and the period during which it actually held true in the world.
Provenance and permissions. Trustworthy enterprise agents must be auditable. If an auditor asks why an agent made a decision, ‘the model said so’ is not a defensible answer when the fact ends up in a regulatory filing. You need row and graph-level access permissions enforced before the model ever sees the data, paired with a complete audit trail of fact sources.
Agent write-back. Memory cannot be read-only. Every serious agent reads, thinks, and writes. As agents execute tool calls and observe outcomes, they must write structured state back into their memory layer. If write-back lands in unstructured tables or logs, context degrades over time.
If an agent can’t tell us what it knew, where it got the information, and whether it had permission to use it, that agent can’t be trusted to operate in enterprise workflows.
One traversal instead of three searches
“The answer is to run similarity inside the traversal, rather than alongside it. Traditional search over multiple vectors creates over-fetch. So, essentially, you pay for the same fact three times. Rather than searching a broad corpus and hoping the relevant information is returned, the query establishes the neighbourhood, then ranks within it,” noted Morgan Hitchcock.
A single traversal limits the candidate set before ranking, which reduces token usage while improving accuracy.
He implores technology practitioners to “stop seeing databases as passive bit-stores, sitting at the end of an API call” – and instead start building data layers that reflect the multi-dimensional nature of context. Instead of making developers patch together five different engines, the data layer needs to step directly into the agent’s reasoning layer as a single engine that natively understands how entities and time intersect.
Morgan Hitchcock concluded, saying that the future of agent engineering isn’t just about making the workflow a graph; it’s about ensuring the knowledge the workflow operates on is also a graph. Until the data layer catches up with our execution frameworks, AI agents will remain brilliant orchestrators balancing on shaky foundations.