From Single LLM Calls to Multi-Agent Intelligence

Over the past few years, large language models have evolved from simple conversational systems into the foundation of increasingly complex AI applications. In the early stage of the LLM revolution, most applications followed a straightforward pattern: a user submitted a prompt, the model generated an answer, and the interaction ended. The main challenge was improving the capability of the model itself. Researchers focused on increasing model size, improving training strategies, and designing better prompting methods because the assumption was that a stronger model could solve increasingly difficult problems.

However, as LLMs started entering more complex scenarios, this assumption became less sufficient. Many real-world tasks are not difficult because they require more language generation. They are difficult because they require different types of reasoning processes working together. A complicated task may require planning before execution, collecting information from different sources, evaluating multiple possibilities, checking whether a conclusion is reliable, and correcting mistakes during the process. Expecting one model to perform all these functions simultaneously is similar to asking one person to independently perform the roles of a researcher, analyst, engineer, manager, and reviewer. Even if that person is highly capable, the structure itself creates limitations.

This limitation has motivated the development of multi-agent systems. Instead of treating an LLM as a single problem-solving unit, multi-agent systems organize multiple AI agents into a collaborative structure. Each agent can have a different responsibility, and the final solution emerges from communication among these specialized components. A planning agent may decide how to approach a task, research agents may collect relevant information, reasoning agents may analyze different aspects of the problem, and critic agents may evaluate whether the intermediate results are trustworthy. The goal is not simply to add more models, but to create an organized system where different capabilities complement each other.

This idea has become one of the most active research directions in the current AI ecosystem. Frameworks for agent orchestration, autonomous research systems, coding agents, and collaborative AI assistants are all based on a similar assumption: intelligence may not come only from making one model larger, but also from designing better interactions between multiple intelligent components.

However, once AI systems become collections of interacting agents, a new challenge appears. The system is no longer just a model. It becomes a network. The way agents communicate, the order in which they interact, and the structure connecting them become important design decisions. This communication structure is usually described as the coordination topology.

Topology determines how information moves inside the system. A simple system may connect agents in a fixed sequence, where one agent completes a task and passes the result to the next agent. A more complex system may allow several agents to work in parallel under the control of a coordinator. An even more connected system may allow every agent to communicate directly with many other agents. These different structures may influence not only the quality of reasoning but also the behavior of the entire AI system.

While many studies on multi-agent systems ask whether collaboration improves intelligence, we should ask a different question: when agents collaborate, what kind of traffic patterns do they create?

This question becomes increasingly important because future AI systems will not only receive requests from humans. They will also generate internal requests among agents. A single user instruction may trigger many hidden interactions: agents calling other agents, requesting additional reasoning, checking intermediate outputs, and repeating certain processes. From the user’s perspective, there may be only one question and one answer. From the infrastructure perspective, there may be dozens of model calls happening behind the scenes.

Therefore, the future of AI system design requires a broader view. We cannot only ask whether an agent architecture is intelligent. We also need to understand whether it is efficient, scalable, and stable when deployed in real environments.

This perspective also connects with recent research on automatic agent topology generation. These systems attempt to discover better communication structures for multi-agent systems instead of relying on manually designed workflows. However, designing a good agent topology should not only consider answer quality or token efficiency. The topology also determines how requests are generated and distributed across the underlying infrastructure. A structure that improves reasoning but creates unstable traffic patterns may not be the best solution for real-world deployment.

Traffic modeling provides an important piece of this larger puzzle. It suggests that the architecture of intelligence and the architecture of workload are deeply connected. When AI agents start communicating, their conversations become part of the system engineering problem.

The Hidden Traffic Problem Behind Agent Collaboration

This article examines the experiments reported in Towards Traffic Modelling of Multi-Agent Systems: The Role of Coordination Topology, published at the ACM SIGCOMM Workshop on Networks for AI Computing (NAIC ‘26). The authors also released their AgenTraffic measurement framework, including the workflow, monitoring stack, and analysis pipeline.

The rapid growth of multi-agent systems introduces a fundamental shift in how we think about AI workloads. Traditional AI services are usually designed around the assumption that requests come from users. A person sends a question, waits for a response, and then decides what to do next. The timing and frequency of these requests are largely controlled by human behavior. Although users can create high traffic volumes, the requests are usually independent from each other.

Agent-based systems behave differently because the workload is generated by software entities that follow internal reasoning processes. When one agent completes a task, it may automatically trigger another agent. When a coordinator divides a problem into smaller parts, multiple agents may begin working simultaneously. When a reviewer detects uncertainty, it may request additional analysis. The traffic pattern is therefore not simply a reflection of user activity. It is a reflection of the internal organization of the AI system.

This is the central idea behind what the study describes as agent traffic. The researchers investigate how coordination topology influences the arrival pattern of LLM requests. Instead of looking only at the total number of requests, they examine the timing relationship between requests. The key measurement is inter-arrival time, which describes the time interval between two consecutive requests reaching the model-serving system.

The reason this measurement matters is that average request volume does not fully describe system pressure. Two systems may produce the same number of requests within an hour but behave completely differently from the perspective of infrastructure. One system may distribute requests evenly, allowing servers to process them smoothly. Another system may generate many requests at almost the same moment, creating temporary overload and increasing latency. For large-scale LLM services, these differences are extremely important because model inference depends heavily on GPU resources, scheduling strategies, and batching mechanisms.

The researchers investigate three representative coordination topologies: sequential, star, and full-mesh structures. A sequential topology represents a pipeline-style workflow. Agents communicate in a fixed order, where each stage depends on the completion of the previous stage. This structure is simple and predictable. Because the next request is usually triggered after the previous step finishes, the traffic pattern tends to be smoother.

A star topology introduces a central coordinator. One main agent receives the task, divides the work, and sends requests to multiple worker agents. This design improves parallelism because several agents can work at the same time. However, the same mechanism that improves efficiency can also create traffic bursts. When the central agent activates multiple workers, many requests may appear within a short period.

A full-mesh topology allows agents to communicate more freely. Each agent can exchange information directly with other agents instead of depending on a central controller. This structure can support richer collaboration, but it also increases communication complexity. More possible connections mean more possible interactions, and more interactions can lead to more model calls.

The important observation is that these topology choices are not only architectural decisions for improving reasoning. They also act as mechanisms that shape workload behavior. The way intelligence is organized directly influences the way computation resources are consumed.

This finding has important implications for future autonomous AI systems. If millions of agents are deployed in practical environments, their communication structures will determine not only how well they solve problems but also how efficiently AI infrastructure can support them.

A future agent architecture search system should therefore consider more than traditional optimization objectives. Current approaches often focus on accuracy, token consumption, or the number of agents. But traffic behavior introduces another dimension. The best topology may be the one that achieves a balance between intelligence, cost, latency, and workload stability.

How Agent Traffic Can Be Measured and Modelled

Once we accept that a multi-agent system can generate its own workload, the next question is how to measure that workload in a useful way. Simply counting how many model calls happen is not enough. Imagine two agent systems that each generate thirty LLM requests during a task. In the first system, those requests arrive one after another over two minutes. In the second, twelve requests arrive almost at the same time, followed by a long quiet period, and then another burst appears. Both systems have the same total number of calls, but they are very different from the point of view of an inference server.

The first workload is relatively easy to schedule because the requests are spread out. The second may suddenly increase queue length, GPU concurrency, memory pressure, or time to first token even though its average request rate looks perfectly reasonable. This is why inter-arrival time, usually shortened to IAT, becomes such a useful measurement. IAT is simply the wall-clock gap between two consecutive LLM requests. If one request arrives at time t_i and the next arrives at t_(i+1), then the inter-arrival time is IAT_i = t_(i+1) - t_i.

This simple quantity reveals something that an average request count cannot: whether requests are spread smoothly through time or concentrated into bursts. In sequential workflows, this gap is strongly connected to model latency because one agent usually waits for the previous call to finish before the next one begins. In parallel workflows, however, several long-running model calls can have an IAT close to zero because they are dispatched almost simultaneously. The workflow therefore leaves a visible timing signature in the request stream.

To observe this properly, the experimental environment keeps the surrounding infrastructure fixed while changing the coordination structure. The agent workflow is built with AgentVerse, four subagents are used in every run, and all agents share the same inference backend. The backend uses Llama-3.2-3B through an asynchronous vLLM engine, which is important because the system can genuinely accept several requests at once and schedule them for GPU batching rather than artificially forcing everything into a serial queue. The three coordination modes then differ mainly in how discussion calls are dispatched. Under sequential coordination, agents are called one at a time in a fixed order. Under star coordination, one solver produces an initial proposal and the remaining agents are launched simultaneously as reviewers. Under full-mesh coordination, all directed peer-to-peer message pairs are submitted concurrently during a discussion round. Each topology is repeated 500 times under the same task source, agent implementation, inference backend, and monitoring environment, producing thousands of individual request gaps rather than a handful of anecdotal examples.

Another useful feature of this setup is that it does not treat the application layer as the whole story. Measurements are collected across the agent workflow, the LLM serving layer, containers, and the network. At the application level, this includes request latency, time to first token, token counts, and inter-arrival time. At the inference layer, it includes request rate, token throughput, in-flight calls, and concurrency peaks. At the network level, it includes TCP byte rates and connection events. These measurements share timestamps and agent identifiers, making it possible to see whether a change that begins in the coordination logic also shows up lower in the stack. It does. Shorter IATs and larger burst fractions are accompanied by higher concurrency and token throughput, and these changes are also visible in network traffic rates. That matters because it shows that agent topology is not an abstract graph property that disappears once the workflow reaches the server. Its effects propagate all the way down into the infrastructure actually carrying and processing the requests.

Why Poisson Traffic Falls Short

The next challenge is describing the shape of those inter-arrival times. A natural starting point is the exponential distribution, because it represents the classic memoryless arrival process associated with a Poisson model. Its density is f(x) = lambda * e^(-lambda*x) for x >= 0. The important idea is not really the equation itself but the assumption behind it. A memoryless process says that the probability of the next arrival does not depend on how long we have already waited. If requests were appearing independently with a stable average rate, an exponential model would be a reasonable first approximation. But coordinated agents clearly have internal state. A reviewer cannot start before some other event tells it to start. Parallel workers may be launched together. A later reasoning step may depend on the completion time of earlier generations. That makes the exponential model a useful null hypothesis precisely because rejecting it tells us that these workloads have stronger structure than a simple random arrival process.

A more flexible candidate is the Weibull distribution. One common form is f(x) = (k/theta)(x/theta)^(k-1)e^(-(x/theta)^k), where k controls the shape and theta controls the time scale. The exponential distribution is effectively a special case when the shape parameter is one, so Weibull gives us more freedom to represent timing behavior in which the effective arrival tendency changes with elapsed time. If k is larger than one, the shape becomes more concentrated than an exponential. If k is below one, it can produce a heavier right side and a stronger concentration near short intervals. This makes Weibull much more adaptable than a pure Poisson assumption without requiring an extremely complicated model.

The third candidate is the log-normal distribution. A positive variable X is log-normal when log(X) follows a normal distribution. Its density can be written as f(x) = exp(-((ln(x)-mu)^2)/(2*sigma^2)) / (x*sigma*sqrt(2*pi)). Again, the intuition matters more than memorizing the formula. A log-normal distribution naturally appears when the final outcome is influenced by several multiplicative factors rather than one simple additive mechanism. LLM request timing fits this idea surprisingly well. The delay between meaningful agent calls can depend on prompt length, accumulated context, the number of generated tokens, batching behavior, and other serving decisions. These factors interact rather than contributing a fixed number of milliseconds each. As context grows, for example, processing cost can influence generation time, which then affects when dependent agents are able to make their own requests. That kind of compound timing behavior makes a skewed distribution with a longer right tail much more plausible than a symmetric bell curve or a purely memoryless process. The measured reasoning-phase gaps are consistent with exactly this picture: exponential is strongly rejected, while log-normal provides the best overall parametric fit across the three coordination modes.

Why Fan-Out Creates Bimodal Traffic

There is one important complication, though. Star and full-mesh traffic should not really be described by a single distribution over their complete set of inter-arrival times. Their traffic contains two different mechanisms mixed together. The first is fan-out. When a coordinator launches several reviewers or multiple peers send messages at the beginning of a discussion round, requests can be separated by only a few milliseconds. The second is the reasoning phase, where the interval is determined by actual model generation and accumulated context. These two mechanisms create two visible modes. In practical terms, there is a cluster of almost-simultaneous requests and another cluster representing the much slower pace of reasoning. A threshold of 50 milliseconds is used to separate these behaviors because it lies around the visible trough between the two modes: IATs at or below 50 ms represent fan-out, while gaps above 50 ms are treated as reasoning-phase arrivals. Sequential execution does not contain the same structural fan-out mechanism and is approximately unimodal. Star and full mesh are therefore better understood as mixture processes rather than as one clean probability distribution.

This distinction is critical because otherwise the statistical analysis would answer the wrong question. Fitting one exponential, Weibull, or log-normal curve across the entire star workload would force a single model to explain both near-zero dispatch events and multi-second reasoning intervals, even though those events come from different causes. By isolating IAT values greater than 50 ms, we can ask a more meaningful question: once the initial fan-out burst has happened, what distribution best describes the timing of agent reasoning? That is where the three candidate models can be compared properly. Maximum-likelihood fitting is used to estimate their parameters, and two measures help evaluate how well they fit: AIC and the Kolmogorov-Smirnov statistic. AIC rewards a model for fitting the observed data well but also penalizes unnecessary complexity, so lower values are better. The KS statistic measures the largest difference between the empirical cumulative distribution and the fitted distribution; again, smaller is better. With thousands of samples, formal KS hypothesis tests become extremely sensitive and eventually reject even small imperfections, so the size of the KS statistic becomes more informative than simply looking at whether a p-value crosses a conventional threshold.

What Changes When the Topology Changes

Sequential: Smooth and Predictable

The most striking result is how dramatically the timing behavior changes even though the underlying model, agent framework, and task source stay fixed. Sequential coordination behaves roughly the way we might expect from a pipeline. During the discussion stage, it averages about 14 LLM requests per repetition, produces a mean inter-arrival time of 4.83 seconds and a median of 4.49 seconds, and never produces an IAT below 50 milliseconds in the reported discussion-stage metrics. Mean LLM concurrency is essentially one and peak concurrency is exactly one. In other words, the topology effectively serializes the workload. One model call happens, another follows, and there is no systematic mechanism creating simultaneous requests. This is the easiest of the three workloads to understand from an infrastructure perspective because average rate and instantaneous load are relatively close to each other.

Star: Similar Average, Sharper Bursts

The star structure looks completely different even though its mean IAT initially seems surprisingly similar. Its discussion-stage mean IAT is 4.87 seconds, almost identical to the sequential value of 4.83 seconds. If we looked only at the mean, we might conclude that almost nothing changed. The median reveals the problem immediately: it falls from 4.49 seconds to only 0.40 milliseconds. More than half of the discussion-stage gaps—54.9 percent—are below 50 milliseconds. Peak LLM concurrency rises from one to four, while mean concurrency increases from 0.97 to 1.62. At the same time, the 95th-percentile LLM latency rises from 10.35 seconds to 23.94 seconds and median time to first token rises from 125 to 227 milliseconds. The system therefore contains both very short arrival gaps and much longer model-serving delays. The average IAT hides this because the two behaviors partially cancel each other numerically. A huge number of almost-zero intervals sit alongside slower reasoning gaps, producing a mean that looks ordinary even though the underlying distribution is nothing like the sequential case.

This is a useful lesson for anyone building real multi-agent infrastructure: averages can become actively misleading once the workflow introduces parallelism. Suppose an engineer looks only at a dashboard showing about 0.2 calls per second for both sequential and star coordination. They might reasonably assume that the two workloads require similar provisioning. Yet the star system contains synchronized fan-out events that temporarily create four concurrent requests and push token throughput much higher. The average request rate does not capture when those requests arrive. If enough user tasks happen to enter their fan-out phases at similar times, the backend can see a much more serious concurrency spike than average-rate capacity planning would predict. This is why burst fraction, IAT percentiles, and concurrency peaks become important operational metrics alongside simple requests per second.

Full Mesh: Denser Traffic, Shorter Runs

Full mesh introduces another kind of behavior. It generates almost 30 LLM requests per repetition, more than twice the sequential count, and its discussion call rate rises to 1.366 calls per second, roughly 6.5 times the sequential rate. Mean IAT drops to 1.09 seconds and median IAT drops to 236 milliseconds, while 38.2 percent of arrivals occur within 50 milliseconds. Peak concurrency reaches five and mean concurrency rises to 2.94. Prompt-token throughput climbs to 1,052 tokens per second and completion-token throughput reaches 364 tokens per second, compared with 237 and 110 tokens per second in the sequential case. Yet the full-mesh discussion stage is actually much shorter overall: about 31.6 seconds compared with 81.1 seconds for sequential execution. The system is doing far more work at once, compressing a much denser communication pattern into a shorter period.

That result is especially interesting because it shows why “more traffic” and “slower system” are not the same thing. Parallelism can increase instantaneous load while reducing end-to-end duration. Full mesh generates much denser request traffic, but individual calls contain fewer tokens on average—about 1.10k compared with 1.80k under sequential coordination—and that reduces individual call latency. At the same time, higher concurrency allows much more completion-token generation to happen per second. In the measured configuration, this produces a shorter discussion phase despite a much higher call rate. This should not be interpreted as evidence that full mesh is universally faster, because the backend was deliberately provisioned to avoid saturation. The effective concurrency limit was eight requests, so the measured peaks of five remained within the serving capacity. In a heavily loaded production backend, the same increase in parallelism could instead produce queueing and contention. The numbers therefore illustrate the behavior of a controlled environment, not a universal guarantee about topology performance.

The Arrival Model Matters

The bimodal shape of star and full-mesh traffic is probably the most important conceptual result because it tells us that agent-generated traffic has phases. In star coordination, the first mode comes from the solver dispatching several reviewers almost simultaneously. In full mesh, it comes from agents launching peer messages concurrently near the beginning of a discussion round. These actions create the near-zero gaps. Once the fan-out has happened, agents begin generating outputs and the system enters a second timing regime governed by reasoning and model-serving latency. The traffic is therefore not simply “bursty” in the vague sense that some requests happen close together. The burst is structurally created by the coordination protocol. If the topology contains fan-out, the traffic distribution contains a fan-out component. This gives us a much stronger explanation than merely saying that workload varies over time.

When the fan-out component is removed and only reasoning-phase intervals above 50 milliseconds are fitted, the log-normal model performs best by AIC across all three topologies. Sequential traffic has an exponential AIC of 31,604, Weibull at 23,165, and log-normal at 21,782. Star gives 22,173 for exponential, 19,131 for Weibull, and 18,377 for log-normal. Full mesh gives 22,078, 21,760, and 20,590 respectively. In every case, the lowest AIC belongs to log-normal. The KS statistic tells a slightly more nuanced story. Log-normal is clearly strongest for sequential traffic, with a KS value of 0.027 compared with 0.091 for Weibull and 0.396 for exponential. It is also best for star, with 0.077 compared with 0.104 and 0.359. Full mesh is the exception: Weibull achieves a slightly smaller KS statistic, 0.068 versus 0.089 for log-normal, although log-normal still has a substantially better AIC. This suggests that Weibull matches the central peak of the full-mesh distribution a little more closely, while log-normal gives a better overall description once the entire distribution and model fit are taken into account.

The exponential model performs poorly enough that this is more than a small statistical preference. In subsampling checks, exponential is rejected extremely often even with much smaller samples. For sequential and star traffic, it is rejected in 100 percent of random samples containing only 50 observations, while full mesh rejects it in 51 percent of those small samples. At sample size 100, the full-mesh rejection rate rises to 80 percent, and at 200 observations it reaches 98 percent. Log-normal, by comparison, is rejected much less frequently in these smaller-sample checks. This supports the idea that reasoning-phase arrivals are not memoryless. The interval before the next request is being shaped by what the workflow is doing, by how long the model is generating, by accumulated context, and by serving behavior. Treating it as a Poisson arrival process throws away precisely the structure that makes agentic workloads different.

Design Implications for Multi-Agent Infrastructure

There is also a broader systems insight hiding inside these numbers. The topology changes application-level timing, and those changes continue downward into the inference and network layers. As coordination becomes more parallel, concurrency and token throughput increase. TCP byte rates change as well. This means topology should be thought of as part of the workload generator itself. It does not merely decide who receives which message; it decides when computation is requested. A sequential graph effectively inserts waits between calls. A star graph introduces synchronized dispatch. A full-mesh graph creates dense peer communication and a much higher request rate. The graph is therefore acting almost like a scheduling policy.

That observation becomes especially important when we think about automatically generated agent architectures. If a system is allowed to choose its own number of agents and communication links, optimizing only task accuracy and token usage is incomplete. Imagine two candidate topologies that achieve the same answer quality and consume roughly the same number of tokens. One executes mostly sequentially and keeps backend concurrency near one. The other finishes faster by launching many agents together but regularly produces sharp bursts. Which topology is better? There is no universal answer. If the system is running on dedicated GPUs with spare capacity, the parallel topology may be preferable because it reduces task latency. If thousands of users share the same inference cluster, its burstiness may create queueing that affects everyone else. A topology designer therefore needs to know not only how much computation a graph uses but also when that computation arrives.

This leads toward a more realistic idea of efficiency. For a multi-agent system, cost cannot be summarized by agent count or token count alone. The temporal shape of computation matters. Ten requests launched one after another and ten requests launched in the same 20-millisecond window represent very different operational demands. A future topology optimizer could therefore treat burst fraction, peak concurrency, p95 inter-arrival time, model latency, or even expected queueing delay as explicit optimization signals. Instead of asking only, “Can this graph solve the task with fewer agents?”, it could ask, “Can this graph solve the task without creating unnecessary synchronized load?” That would move agent architecture design closer to the concerns that already matter in distributed systems, including concurrency control, admission control, resource scheduling, load balancing, tail latency, and capacity planning.

At the same time, the measured values should not be treated as universal constants. The setup uses one task family, one agent framework, one model, one host, and one serving configuration. A larger model could change generation latency dramatically. A different batching strategy could alter the cost of concurrent requests. Tool calls, browser actions, retrieval systems, and remote APIs could introduce entirely new timing components. Real production systems would also contain multiple user tasks overlapping in time, which could mix workflow-generated bursts with external demand. The useful lesson is therefore not that a star topology always has a particular median IAT or that full mesh always generates a particular number of tokens per second. The stronger lesson is that coordination structure creates measurable timing behavior, and that behavior survives beyond the logical agent layer into the infrastructure that executes it.

Toward Traffic-Aware Agent Orchestration

This changes how we should think about the next generation of agent systems. Today, an agent graph is often treated as a reasoning architecture: agents are connected in whatever way seems most likely to improve the final answer. In a mature deployment environment, the graph may need to be treated simultaneously as a reasoning architecture and a workload specification. Every new edge potentially changes not only information flow but also request timing. Every parallel branch potentially changes latency and concurrency. Every additional discussion round potentially changes context size, token throughput, and network activity. Once these relationships are taken seriously, coordination topology stops being a small implementation detail. It becomes one of the main interfaces between AI intelligence and systems engineering.

The practical consequence is that future multi-agent platforms may need to become traffic-aware by design. Rather than constructing a graph and only discovering its serving behavior after deployment, a system could estimate or simulate the workload while designing the graph itself. A topology generator might predict expected token usage, concurrency, latency, and burstiness before execution. If several graph structures are likely to solve the same task, it could choose among them according to the current state of the infrastructure. During quiet periods, it might use a more parallel graph to minimize response time. During peak load, it could switch to a more sequential or sparse topology to reduce concurrency pressure. The coordination graph would then become adaptive not only to the intellectual difficulty of the task but also to the condition of the backend serving system.

That possibility points to a larger shift in how autonomous AI will probably be engineered. The earliest generation of LLM applications mostly treated model serving and application logic as separate concerns: developers designed a prompt or workflow, and infrastructure engineers tried to serve it efficiently. Multi-agent systems weaken that separation because application logic directly determines request creation. A planner that decides to recruit three additional agents has just made an infrastructure decision, whether or not it realizes it. A topology generator that adds dense peer communication has just created a concurrency pattern. As agent systems become more autonomous, some form of systems awareness may therefore need to become part of their decision-making process. The most capable architecture may not be the one that communicates the most. It may be the one that knows when communication is actually worth the load it creates.

Sources