Software has agents. Agents take the form of individual sections of code tasked with specific dedicated and typically relatively defined functions. Not a new technology principle by any means, agent-based software architectures have come to the fore in the containerised world of cloud computing and its current need to serve the still-burgeoning world of artificial intelligence. Agentive architectures are recognised as a type of co-pilot function in various development scenarios, this is because they automate tasks and transform workflows. As a good example, GitHub Copilot is helping developers automate code generation and testing. How then should we consider working with this strain of technology in the near and immediate future?
Creating these agents for AI isn’t your run-of-the-mill engineering project says Luis Ceze in his role CEO of OctoAI. This is because, unlike generic AI models, agents need to be carefully customised and integrated into specific workflows to be useful. This requires development teams to pick up new skills, focusing on how to assemble and orchestrate agents that meet the unique needs of different industries. For instance, it’s crucial to understand the differences between single-agent and multi-agent systems and what it takes to deploy them successfully.
Rewriting problem-solving rules
“AI agents are essentially language model-powered entities built to plan and execute goals over multiple steps,” explains Ceze. “They can work solo, or they can team up with other agents to tackle complex problems. Each agent usually has its own ‘persona’ and is equipped with various tools to get tasks done, whether they’re flying solo or working as part of a team. Some agents even have a memory feature, letting them save and recall information beyond just their current interactions.”
What we can say today is that a software agent is the brain behind every AI system, orchestrating all decision-making processes. This command centre defines what goals to chase and decides which tools to pull from the toolbox to get there. An agent also selects the right planning modules to handle different situations, making sure it can adapt to whatever challenges come its way. Perhaps the best part is the fact that this decision-making process gets sharper over time, thanks to the agent’s ability to learn from past interactions.
Memory: A repository of experience
“Memory is where the magic of learning from experience happens,” clarifies Ceze. “It’s what lets AI agents store and retrieve information, guiding their future actions. There are two types of memory at play here: short-term and long-term. Short-term memory handles what’s happening right now, allowing the agent to react in real-time. Long-term memory, on the other hand, keeps a log of extended interactions, giving the agent a historical perspective that makes tackling complex tasks a breeze. By drawing on these memories, AI agents can continuously fine-tune their strategies, getting better and better over time.”
Tools: The execution arsenal
In developer-speak, we can carry this story forward and think of tools as the agent’s secret weapons, allowing it to execute tasks with precision. Software tools and tooling (in this sense of the word) can take the form of anything from executable workflows and third-party APIs, to highly specialised services designed for specific needs. For example, an AI agent might use context-aware answer generators for customer service, code interpreters for software development, or weather APIs for logistics planning. With these tools in hand, AI agents are equipped to operate across various domains, bringing a high level of precision and agility to whatever they do.
According to OctoAI’s Ceze, “The way we design and deploy AI agents is absolutely critical to unlocking their full potential. The architecture you choose can make all the difference in how effectively these agents tackle tasks, solve problems, and work alongside human users. There are two main types of AI architectures: single-agent and multi-agent systems – and each comes with its own set of perks and challenges.”
Single-agent architectures are like the ‘lone wolves’ of the AI world – it’s a term that Ceze favours and it is (arguably) pretty illustrative of the way they work. Here, we have one AI agent, armed with a language model, handling everything i.e. reasoning, planning and executing tasks. The beauty here is in its simplicity i.e. there’s just one brain to coordinate. But with that simplicity comes a downside: the lack of collaboration. Without input from other agents, the system leans heavily on human feedback to stay on course. This works well for straightforward tasks but might struggle in more complex, ever-changing environments.
“On the flip side, multi-agent architectures are all about teamwork. Here, you’ve got two or more AI agents joining forces, each contributing to the task at hand. These systems can be thought of as a symphony, with each agent playing its part to create a harmonious performance. The complexity of these systems can vary, often depending on whether they’re organized vertically or horizontally, with each structure dictating how agents interact,” detailed Ceze.
Vertical architectures
In vertical architectures, the OctoAI team explain what’s going on and note that one agent steps up as the conductor, directing the other agents toward a common goal. This hierarchical structure ensures there’s a clear chain of command and a solid division of labor. Depending on the design, reporting agents might communicate directly with the lead agent or participate in broader discussions. This (they argue) is perfect for tasks that require strong leadership and a clear structure, ensuring everything runs smoothly and efficiently.
“Horizontal architectures, on the other hand, throw hierarchy out the window. Here, all agents are on equal footing, engaging in open dialogues and sharing ideas and responsibilities. This structure shines in tasks that need creativity, feedback, and collaboration. By allowing agents to freely volunteer for tasks and interact openly, horizontal architectures create a space where collective intelligence can thrive – much like a think tank where every voice is heard,” said Ceze, in exact terms..
Side note, but still important.
Exciting as the possibilities for agentive workflows are, there are important considerations for infrastructure. A suite of models executing tool calls in concert will most certainly be more compute-intensive than a single LLM operating alone. This increased inference volume will put a premium on model efficiency.
The bottom line
“The architecture we choose shapes not only how these agents operate today but also how they will evolve in the future. Whether working as lone operators or part of a dynamic team, the design and deployment of AI systems will determine their impact and effectiveness. Understanding and mastering these architectures is key to leveraging the full potential of AI in solving complex challenges,” concluded the OctoAI CEO.
OctoAI held a private briefing for a small group of European and North American press and analysis with the CEO, who clearly comes from a hard core software engineering background, leading all the explanation, clarification and discussion.