In the previous article, we established that agentic architecture comes from the attempt to replicate the successes in human experience of having multiple people working together toward an end goal rather than working alone.
Specifically, an LLM can provide remarkable responses to prompts, but there is a potential strength in numbers. There is a potential strength from both working in parallel (width) and passing along the fruits of one’s labors sequentially (depth).
Agentic architecture, therefore, has a horizontal and vertical shape that constitutes its organization. Like human organizations, it is both an art and science to form an ideal organization. In an agentic context, the ideal organization is one that finds the proper width and depth to solve a problem with optimal performance and cost efficiency. Secondarily, predictability is also a potential tradeoff.
In addition to the width-depth ratio, there are two levels of considerations when designing an agent organization:
Human vs. agent determination — Should humans or agents be trusted with optimizing for cost and performance? And if the answer is both, then what is the precise division of responsibilities?
Timing — Should agent organizations structure at runtime, or should they be structured upfront? And if the answer is both, then what does it look like to stitch this together?
In the last article, we looked at these considerations with the structure/shape of an agent organization in mind, noting that Claude Code tends toward trusting agents to dynamically optimize the width-depth ratio at run time to attain cost-performance optimization, whereas LangChain’s approach tends toward trusting humans to determine the ideal width-depth ratio upfront. Nevertheless, as we shall see, it is possible customize whichever tool is the starting point to match the ideal response to the two levels of tradeoffs.
In this article, we will continue the theme of weighing these two levels of tradeoffs for cost-performance optimization, but with an eye toward the internal workflow of an agent organization (and not just it’s external structure).
When considering the optimal internal workflow, it is important to disassociate from a particular tool. While we could dive immediately into the architecture of, say, Claude Code and then extract the principles found therein, I think it is beneficial to first reflect on internal workflows on their own. When we do this, we can continue to uncover the parallels with human organizations that might clue us in to the breadth of factors to consider, as well as the time-tested solutions that could equally apply to agent organizations. If we immediately start with a tool and its architecture, we may box ourselves in before we really understand the problem and potential sources for solutions. So let’s continue our illustration of using military organizations as an analogy.
You’re Napoleon. A young, ambitious Napoleon. And you’re heavily outnumbered.
You need to consider a solution that will give you an edge against the enemy. What do you do?
Napoleon’s innovative solution was to divide his military forces into self-autonomous corps. Whereas traditional armies had to all fight together in order to adhere to the chain of command, these corps could fight on their own, with marshals making autonomous decisions. We might say that they were “freed up.” Truly, they were free to roam outside of the vicinity of Napoleon, and the marshal could execute a strategy without referring to his superior. Napoleon was still the commander in chief, and he’d want to know where his corps were, communicating with them as needed given what was developing across the overall battle. Yet, his trust in a shared doctrine of tactics with his marshals allowed for the corps to move independently and act with devastating speed. And so, Napoleon gained the upper hand.
But was Napoleon needed? I’m not asking for an evaluation of the life and campaigns of Napoleon as to their historical impact. I mean, quite simply, what would it look like if there was a team of marshals without a commander in chief? Well, there would be no one who saw the full chess board at once, as it were. No one would be able to dynamically adjust while seeing the full battle unfold. If it would be strategically optimal for one corps needed to retreat, reenforce another corps, etc., no one would know. Optimal adjustments wouldn’t happen. The advantageous speed of the autonomously deployed would be canceled out by the lack of adaptability mid-battle.
Pausing here, we can extract some principles that carry over to an agent organization. Napoleon’s corps system introduced greater width in military tactics. However, that width needed to be a principle. Without an individual to see the full landscape of the battle and make the ultimate strategic decisions, the advantage of the width collapses. Hence, even in agent organizations, the immediate width needs a principle, commonly called an orchestrator.
This is exactly where the two levels of tradeoffs from the introduction resurface. Who is the Napoleon? Is it an overseeing human or an agent? And should that choice be settled upfront, or determined dynamically as the work unfolds?
Let’s bracket this thought and continue with our illustration to discuss communication within an organization.
Because Napoleon had trust in his marshals, they could be sent to greater distances and make decisions without constant approval. Should Napoleon want to communicate with his marshals, he’d need to send someone into the distance to relay the message. In effect, this technical limitation reinforced that the marshals operated freely. Napoleon’s strategic thinking was the high-level context of the battle; the marshal’s strategic thinking was for his particular role within it. Napoleon’s general contextual deliberation did not cross into the particular contextual deliberation of the marshal, and vice versa. If the marshal heard from Napoleon, it was because something important needed to be conveyed given the big picture. This allowed the marshal to adjust without the marshal hearing all of the deliberation leading up to that adjustment. Moreover, sideways communication between marshals would be feasible, but it required dispatching a rider across the field; hence, it wouldn’t be as important as the big-picture communication from Napoleon.
Now, imagine Napoleon and his marshals had access to walkie talkies. The technology now allows for Napoleon to involve his marshals in the deliberation before commands are given and vice versa. The same now applies for sideways communication. Is this an advantage?
When we think about agentic organizations, we will assume a walkie talkie scenario, for there is no technical restraint on agents communicating with each other. It’s important to use a pre-walkie talkie scenario to highlight that there are models of more limited communication, and that there may be some advantages in this approach. An agent organization, like modern human organizations, requires a noise-insight tradeoff. Meaning, consideration has to be given to what’s really advantageous to communicate, since every message attended to divides attention from the task at hand.
Here too, the two levels of tradeoffs from the introduction resurface. Does the agent or overseeing human optimize for the noise-insight tradeoff, and how much of that should be determined upfront versus determined dynamically?
To optimize the noise-insight tradeoff, it’s important to keep in mind what each agent is responsible for. Napoleon had a goal: win the battle. The marshals, therefore, inherited an objective; that is, to accomplish it so as to help win the battle. To execute the objective, the marshals needed to carry out strategic tasks. Presumably, the marshals typically had their strategic tasks determined upfront with Napoleon’s collaboration. However, perhaps Napoleon needed to modify the objective entirely, or just the strategic tasks in particular, based upon how the entire battle was playing out. And the marshals may have needed to make these modifications based on the developments happening before them. Hence, communication and coordination go hand-in-hand. We can determine what is insightful communication, rather than noise, by determining whether it helps achieve strategic coordination.
Given this, organizations will have a hierarchy of tasks. There are the concerns relevant to Napoleon's scope of the big picture, the end goal itself. In addition, there are particular tasks relevant to the marshal’s scope of their objective. Certainly, a marshal’s insight that might impact Napoleon’s end goal is worth communicating, but Napoleon doesn’t need to hear the entirety of the marshal’s “how I came to this determination.” Likewise, any time Napoleon’s high-level insight could impact a marshal’s objective, then communication is worthwhile. The muddier question is whether/when Napoleon should coordinate at the level of a task within a marshal’s objective. Moreover, should the marshal ever suggest changes to the delegated objective? Even muddier, should “What do you think about this?” ever precede explicit coordination? However these considerations are handled, as we shall discuss in the future, it is useful to see coordination as a separate thing despite informing the majority of communication.
The technological constraints in Napoleon’s time lessened the complexity found in resolving these considerations. A Napoleon with walkie talkies would have complicated things. Organizations composed of agents are even more complicated. Coordination, like orchestration and communication, raises the same two questions. Should a human or an agent decide when task-level coordination is warranted, and is that call made once, upfront, or continuously, as the situation unfolds?
Let’s imagine Napoleon did have walkie talkies. Would maximizing the possible directions of coordination be helpful? Initially, it might sound quite useful that a marshal provides insights laterally to an adjacent marshal. However, some questions emerge. How would an adjacent marshal know that it has insight that is truly unique and not something also seen by Napoleon? Even a marshal acting on excellent instincts can unknowingly produce a mixed signal, simply for lack of knowing what Napoleon has already seen. In addition, if the big-picture strategy adapts dynamically, then there is a real risk of sideways coordination accidentally undermining it. The marshals would have to check with Napoleon each time, which could cross a line from strategy to noise. Moreover, any retrospective analysis of what went wrong would require untangling a complex web. Marshals initiating the double-checking of objectives would also increase the noise, and even more so if the double-checking happens at the level of tasks. Likewise, Napoleon getting involved at the level of a marshal’s tasks increases noise.
There are several concerns in this account worth separating. For one, there is the potential noise of multi-directional coordination in general. Second, Napoleon could communicate across a marshal’s objective-scope and bleed into the task-scope. We can call this scope-crossing. Arguably, Napoleon should not command his marshals at the task-scope but only objective-scope. A third but related failure is scope-depth-crossing. Napoleon may remain at objective-scope, yet over-engage within it, potentially leading to unhelpful micromanagement. Both scope-crossing and scope-depth-crossing are noise problems, and may be solved through better strategic discipline.
There is a fourth concern, however, that not about noise per se. Marshals coordinating sideways could reach a conclusion that is entirely coherent among themselves and still undermine the chessboard-scope of Napoleon. Hence, it’s not just about maintaining effective coordination; it’s also about maintaining a unity of command. Coordination, however optimized via strategic discipline, cannot supply what only the orchestrator’s big-picture vision can.
Unity of command inherits the same human-or-agent question already posed for the orchestrator, since the two roles are one and the same. But agent organizations raise another question. Can a human supervisor step outside the hierarchy altogether, overriding at the marshal’s scope, or the orchestrator’s, on demand, rather than occupying one designated seat? This is appealing, but it risks reintroducing the unity of command problem it was meant to solve. A second source of authoritative instruction over the same work, this time human rather than agentic. The remaining open question is timing: should such overrides, along with scope-crossing, scope-depth-crossing, and uncoordinated peer consensus, be prevented upfront through structural constraint, or caught and corrected dynamically, by whoever holds command, after the fact?
To summarize, this article has sought to develop the contours of agent organization, reasoning from historical analogies before introducing specific agentic architectures. If the agentic architecture got introduced first, I fear the implicit why behind certain architectures would be missed, making it harder to evaluate various architectures. Specifically, it has moved beyond the tradeoffs regarding the optimal structure of an agent organization, toward the dynamic interplay within it: orchestration, communication, coordination, and unity of command. In the next article, we will look at the considerations around specialization with the optimal agent structure.