Towards Collaborative Intelligence: Propagating Intentions and Reasoning for Multi-Agent Coordination with Large Language Models

We formulate a decentralized, partially observable Markov game for multi-agent collaboration. Each agent $i$ maintains a private intention $\mathcal{I}_{i}$ encoded as a tuple $\mathcal{I}_{i}=(\gamma_{i},\Sigma_{i},\pi_{i},\delta_{i})$, where $\gamma_{i}$ is the current goal, $\Sigma_{i}=\{\sigma_{i1},\sigma_{i2},\ldots\}$ is a set of related sub-goals, $\pi_{i}(\sigma)$ is a probability distribution over possible next sub-goals, and $\delta_{i}(\sigma)$ is the desired teammate assignment for sub-goal $\sigma$.

Intentions are propagated through a communication channel $f_{\Lambda}$ parameterized by $\Lambda$. For a received message $m_{ij}$ from agent $j$, agent $i$ infers a belief over teammate $j$’s intention $b_{i}(\mathcal{I}_{j}|m_{ij})=f_{\Lambda}(m_{ij})$, where $\Lambda$ is a recurrent neural network. The channel $f_{\theta}$ is trained in an end-to-end manner to maximize the coordination reward function $R_{c}$. This propagates relevant sub-task dependencies to enhance common grounding on collaborative goals.

At each time-step $t$, the LLM witll processinputs comprising the agent’s state $s_{t}$, the intention $\mathcal{I}_{t}$, and the feedback $f_{1:t}$.

The execution agent policies, denoted by $\pi_{\xi}(a_{i}|s_{i},\mathcal{I}_{i})$, are parameterized by $\xi$ and conditioned on the agent’s state $s_{i}$ and intention $\mathcal{I}_{i}$. Emergent coordination patterns are encoded in a summary statistic $c_{t}$ and passed to upstream modules to guide planning and grounding adjustments. For example, frequent miscoordination on sub-goal $\sigma$ indicates the necessity to re-plan $\sigma$ dependencies in $\mathcal{I}$.

This bidirectional feedback aligns low-level execution with high-level comprehension strategies. In addition to the downstream propagation of intents, execution layers provide bidirectional feedback signals $\psi(t)$ to upstream modules $\psi(t)=\Phi(h^{\text{exec}}_{t})$:

where $\Phi(\cdot)$ aggregates agent encodings to summarize emergent coordination, and $\phi_{i}(\cdot)$ encodes the observation $o_{i}$ for agent $i$.

Execution agents generate feedback $f_{t}$ to guide upstream LLM modules through: $f_{t}=g_{\theta}(\tau_{1:t})$, where $g_{\theta}$ processes the action-observation history $\tau_{1:t}$. These signals include coordination errors $\mathcal{E}_{t}$ which indicate misalignment of sub-tasks; grounding uncertainty $\mathcal{U}_{t}$, measured as entropy over grounded symbol embeddings; and re-planning triggers $\mathcal{R}_{t}$, which flag the need for sub-task reordering. These signals can reflect inconsistencies between sub-task objectives, the ambiguity of symbols in different contexts, and the need to adjust previous sub-task sequencing.

The coordination feedback signals $c_{t}$ and interpretability signals $\mathcal{E}_{t},\mathcal{U}_{t},\mathcal{R}_{t}$ play a crucial role in enabling the LLM agents to adapt and learn collectively. By incorporating these signals into the training process, the agents can adjust their strategies and policies to better align with the emerging coordination patterns and requirements of the collaborative tasks.

The collective learning process can be formalized as an optimization problem, where the goal is to minimize the following objective function $\mathcal{L}(\eta,\gamma,\zeta,\xi)=\mathbb{E}_{s_{t},\mathcal{I}_{t},f_{1:t}}\left[\alpha\mathcal{U}_{t}+\beta\mathcal{E}_{t}-\mathcal{R}\right]+\Omega(\eta,\gamma,\zeta,\xi)$. Here, $\alpha$ and $\beta$ are weighting factors that balance the contributions of the grounding uncertainty $\mathcal{U}_{t}$ and coordination errors $\mathcal{E}_{t}$, respectively. The team reward $\mathcal{R}$ is maximized to encourage collaborative behavior. The term $\Omega(\eta,\gamma,\zeta,\xi)$ represents regularization terms or constraints on the model parameters to ensure stable and robust learning.

The objective function $\mathcal{L}$ is defined over the current state $s_{t}$, the interpretability signals $\mathcal{I}_{t}=\{\mathcal{E}_{t},\mathcal{U}_{t},\mathcal{R}_{t}\}$, and the trajectory of feedback signals $f_{1:t}=\{c_{1},\mathcal{I}_{1},\ldots,c_{t},\mathcal{I}_{t}\}$ up to the current time step $t$. The expectation $\mathbb{E}_{s_{t},\mathcal{I}_{t},f_{1:t}}[\cdot]$ is taken over the distribution of states, interpretability signals, and feedback signal trajectories encountered during training.

To assess the performance of our models, we conducted evaluations using two large-scale real-world datasets: the traffic flow prediction (TFP) dataset and the web-based activities dataset.

TFP dataset comprises 100,000 traffic scenarios, each accompanied by corresponding flow outcomes. Each example is detailed with descriptions of road conditions, vehicle count, weather, and traffic control measures, and is classified as traffic flow: smooth, congested, or jammed. The raw data was sourced from traffic cameras, incident reports, and simulations, and underwent preprocessing to normalize entities and eliminate duplicates.

Web activities dataset contains over 500,000 examples of structured web interactions such as booking flights, scheduling appointments, and making reservations. Each activity follows a template with multiple steps like searching, selecting, filling forms, and confirming. User utterances and system responses were extracted to form the input-output pairs across 150 domains, originating from real anonymized interactions with chatbots, virtual assistants, and website frontends.

To handle the computational demands of training our framework with LLMs, we employ 8 Nvidia A800-80G GPUs Chen et al. (2024) under the DeepSpeed Aminabadi et al. (2022) training framework, which can effectively accommodate the extensive parameter spaces and activations required by our framework’s LLM components and multi-agent architecture Rasley et al. (2020).

For the TFP dataset, we classified the examples into four difficulty levels: “Easy”, “Medium”, “Hard”, and “Hell”. The “Easy” level comprises small grid networks with low, stable vehicle arrival rates. The “Medium” level includes larger grids with variable arrival rates. “Hard” tasks feature large, irregular networks with highly dynamic arrival rates and complex intersection configurations. The “Hell” level introduces challenges such as partially observable states, changing road conditions, and fully decentralized environments.

For the web activities dataset, we divided the tasks into “Easy”, “Medium”, “Hard”, and “All” levels. “Easy” tasks required basic single-click or short phrase interactions. “Medium” involved complex multi-page sequences like form submissions. “Hard” tasks demanded significant reasoning through ambiguous, dense websites. The “All” level combined tasks across the full difficulty spectrum.

The dataset was divided into 80% for training, 10% for validation, and 10% for testing, with examples shuffled. These large-scale datasets offer a challenging and naturalistic benchmark to evaluate our multi-agent framework on complex, real-world prediction and interaction tasks.

Table 1 displays the principal experimental results of our ReMALIS framework in comparison with various single-agent baselines and contemporary methods using the web activities dataset. We evaluated the models across four levels of task difficulty: “Easy”, “Medium”, “Hard”, and “All”.

The results from our comparative analysis indicate that ReMALIS (7B), equipped with a 7B parameter LLM backbone, significantly outperforms competing methods. On the comprehensive “All” difficulty level, which aggregates tasks across a range of complexities, ReMALIS achieved a notable score of 55.37%, surpassing the second-highest scoring method, LUMOS, which scored 51.49%. Additionally, ReMALIS (7B) also excelled against AUTOACT, which utilizes a larger 13B parameter model, by achieving a score that is over 3 percentage points higher at 52.87%. These findings highlight the efficacy of ReMALIS’s parameter-efficient design and its advanced multi-agent collaborative training approach, which allow it to outperform larger single-agent LLMs significantly.

Notably, ReMALIS (7B) also exceeded the performance of GPT-3.5 (Turbo), a substantially larger foundation model, across all difficulty levels. On “Hard” tasks, ReMALIS’s 21.42% surpassed GPT-3.5’s 17.36% by over 4 points. This indicates that ReMALIS’s coordination mechanisms transform relatively modest LLMs into highly capable collaborative agents.

Despite their larger sizes, single-agent approaches like GPT-3.5 CoT, ReAct, and AgentLM significantly underperformed. Notably, even the advanced single-agent method LUMOS (13B) could not rival the performance of ReMALIS (7B). The superiority of ReMALIS, attributed to its specialized multi-agent design and novel features such as intention propagation, bidirectional feedback, and recursive reasoning, was particularly evident. On complex “Hard” tasks that required extensive reasoning, ReMALIS achieved a notable performance of 21.42%, surpassing LUMOS by over 2 percentage points, thus highlighting the benefits of its multi-agent architecture and collaborative learning mechanisms.

The exceptional performance of our proposed ReMALIS framework on the Traffic Flow Prediction (TFP) dataset can also be attributed to its innovative design and the effective integration of advanced techniques. On the "Easy" difficulty level, ReMALIS achieved an impressive accuracy of 89.15%, outperforming the second-best method, AUTOACT, by a substantial margin of 1.26%. In the "Medium" category, ReMALIS secured an accuracy of 77.62%, surpassing AUTOACT’s 76.29% by 1.33%. Even in the most challenging "Hard" and "Hell" levels, ReMALIS maintained its lead with accuracies of 64.53% and 55.37%, respectively, outperforming the next best methods, DGN (62.34%) and LToS (54.81%), by 2.19% and 0.56%.

1)The Impact on Improving Multi-Agent Coordination Accuracy We conduct ablation studies to evaluate the impact of each component within the ReMALIS framework. The observations can be found in Table 2. Excluding intention propagation results in a decrease in accuracy by over 6% across both datasets, highlighting difficulties in achieving common grounding among agents without shared local beliefs This highlights the importance of intention sharing for emergent team behaviors.

The absence of bidirectional coordination channels leads to a 4.37% decline in performance across various metrics, illustrating the importance of execution-level signals in shaping planning and grounding strategies. Without feedback coordination, agents become less responsive to new scenarios that require re-planning.

Substituting recursive reasoning with convolutional and recurrent neural networks reduces contextual inference accuracy by 5.86%. Non-recursive agents display short-sighted behavior compared to the holistic reasoning enabled by recursive transformer modeling. This emphasizes that recursive architectures are vital for complex temporal dependencies.

2)The Impact on Improving Multi-Agent Coordination Capability As presented in Table 3, on aligned sub-task percentage, the proposed Basic Propagation, Selective Propagation, and Full Intention Sharing methods consistently outperform baseline models like REACT and AgentLM across varying difficulty levels (“easy”, “medium”, and “hard”). For example, Full Intention Sharing achieves alignment of 91%, 71%, and 62% across these levels, respectively. These results are substantially higher compared to scenarios with no communication (31%, 23%, and 17%).

Similarly, coordination time metrics exhibit major efficiency gains from intention propagation. On “Hard” tasks, Full Intention Sharing reduces coordination time to 521 ms, 57% faster than the 1198 ms for No Communication. As task complexity increases from easy to hard, the coordination time savings compared to baselines grows from 138 ms to 677 ms. This reveals that intention sharing mitigates growing coordination delays for difficult scenarios.

The highlighted propagation mechanisms also demonstrate clear incremental performance improvements over increasingly selective information sharing. As agents propagate more precise intentions to relevant teammates, both sub-task alignment and coordination efficiency improve. Moving from Basic to Selective to Full sharing provides gains on top of gains.