|
Getting your Trinity Audio player ready…
|
1) Communicating Activations Between Language Model Agents — Vignav Ramesh, Kenneth Li (2025)
Link: arXiv page. (arXiv)
Expanded abstract summary
This paper is one of the clearest direct proposals for replacing text-based inter-agent communication with an internal, continuous alternative. The authors begin from a simple observation: when LLM agents “talk” to one another in natural language, they are forcing a rich internal computation through a narrow symbolic bottleneck. A model may internally represent uncertainty, competing continuations, or distributed semantic structure across many dimensions, but once it decodes a sentence, much of that structure is discarded. The paper therefore asks whether models can communicate more effectively by passing internal activations rather than text. Its answer is yes, and it proposes a concrete protocol to do so. The method pauses model B during its forward pass at an intermediate layer, combines B’s current activation with an intermediate activation from model A by applying a function (f), and then resumes model B’s computation. This means communication occurs not after thought has been collapsed into words, but during the thought process itself. (arXiv)
The paper is notable because it keeps the setup deliberately simple. It does not require additional parameters, new training corpora, or a large new communication head. Instead, it uses the existing hidden-state machinery of language models as the communication substrate. The authors evaluate several ways of combining activations and test the approach in two main settings: coordination games and reasoning benchmarks. Across these settings, they report that activation-based communication can outperform natural-language communication by as much as 27.0%, while using less than a quarter of the compute in their experiments. That result is important for two reasons. First, it suggests that natural language is not the optimal medium for machine-to-machine collaboration, even when the machines are language models. Second, it shows that the gains are not just theoretical or aesthetic; they appear in measurable task performance and efficiency. The paper therefore stands as an engineering proof of concept that latent, sublinguistic exchange is feasible, practical, and potentially superior to tokenized conversation for certain multi-agent reasoning tasks. (arXiv)
Expanded technical contribution
The central technical contribution of this paper is that it relocates communication from the output channel to the internal computational pathway. In most multi-agent LLM systems, the communication object is a decoded token sequence: agent A emits words, and agent B reads those words back in as tokens. This imposes several costs at once. It requires autoregressive decoding, which is expensive; it compresses a wide internal state into a small symbolic sample, which is lossy; and it forces any interaction to obey the syntax and granularity of human language. Ramesh and Li show that none of this is inevitable. Their protocol reframes communication as a transfer between intermediate representations inside the forward pass. That sounds like a small architectural tweak, but conceptually it is a major shift: the communicated object is no longer “what the model decided to say,” but “what the model is currently representing.” (arXiv)
What makes the contribution especially strong is its minimalism. Many speculative discussions about latent-space communication imagine elaborate new agent architectures, shared training environments, or differentiable protocols learned from scratch. This paper instead demonstrates that one can get meaningful gains with a light-touch intervention: choose a layer, choose a fusion rule, splice the activations, continue the pass. That makes the approach portable. It can be thought of as a primitive operation for future agent systems, much like attention or residual addition became primitive operations inside transformers. In principle, one could build richer protocols on top of this basic move: selective routing, hierarchical agent topologies, gated activation exchange, uncertainty-weighted fusion, or task-dependent latent handoffs. The paper does not fully develop all those extensions, but it identifies the foundational operation on which they could be built. (arXiv)
A second major contribution is methodological. The paper provides one of the earliest clean demonstrations that language-free communication can outperform language-mediated communication in LLM systems without retraining the agents into some exotic new species of model. That matters because it lowers the bar for experimental work in the field. Researchers do not need to wait for wholly new architectures to study latent communication; they can begin now, by instrumenting the hidden states of existing LLMs. A third contribution is economic. Because decoding language is expensive and scales poorly with the number of agents and messages, activation exchange may be a more compute-efficient substrate for multi-agent reasoning. If that result continues to hold at scale, the paper may end up being important not just academically but practically, as a blueprint for cheaper and faster agent collaboration. In short, this paper contributes the first broadly usable mechanical recipe for inter-LLM latent communication: pause, merge, resume. That is why it deserves the top rank. (arXiv)
2) Enabling Agents to Communicate Entirely in Latent Space — Zhuoyun Du et al. (2025)
Link: arXiv page. (arXiv)
Expanded abstract summary
This paper is one of the most explicit exact matches for your question because it is not merely about enriching text communication with embeddings or passing occasional hidden states; it is about enabling agents to communicate entirely in latent space. The authors begin from the claim that natural language is a constrained interface for collaboration among LLM agents. Human-readable language is useful for us, but for machines it introduces a representational bottleneck: distributed latent states must be downsampled into discrete tokens, and that inevitably limits the amount and subtlety of information that can be transmitted. The paper proposes Interlat, short for Inter-agent Latent Space Communication, as a feasibility study of a different paradigm. Instead of sending text, agents transmit their last hidden states directly as representations of their “mind.” This framing is important because it moves the discussion away from token substitution and toward a full communication model in which latent states are treated as semantically meaningful, transmissible objects. (arXiv)
The authors also add an important practical element: compression. Pure latent communication could in principle become very high-bandwidth and unwieldy, so Interlat studies whether these internal representations can be compressed while preserving useful information for downstream collaboration. The reported results suggest that the answer is yes. According to the paper, Interlat outperforms both single-agent baselines and fine-tuned chain-of-thought prompting, while showing more exploratory behavior and making better use of latent information. Further compression substantially accelerates inference while maintaining competitive performance through what the authors describe as an efficient information-preserving mechanism. The overall picture is that latent-space communication is not just theoretically richer than natural language, but can also be engineered into an efficient communication pipeline. The paper positions itself as a feasibility study rather than a definitive final architecture, which is appropriate: it is laying down the conceptual and empirical groundwork for a new class of agent systems in which collaboration takes place below the level of words. (arXiv)
Expanded technical contribution
The paper’s most significant technical contribution is that it treats latent communication as the primary communication layer, not as an auxiliary hint attached to text. Earlier work such as embedding-based debate or activation passing had already suggested that continuous internal representations could help models communicate more effectively. Interlat pushes this idea further by proposing a setting in which agents exchange hidden states directly and operate entirely within latent space during collaboration. That is a deeper architectural commitment. It means the communicative act is not “say something, but say it with vectors,” but rather “share internal cognitive state directly and defer textual decoding until later, if at all.” This moves the field closer to what one could genuinely call machine-native dialogue. (arXiv)
A second important contribution is the paper’s concern with interpretability across agents without parameter sharing or architectural coupling, which the HTML summary highlights. That is crucial because latent communication is easy to fantasize about and harder to stabilize. One hidden-state space may not be readily meaningful to another model unless the models are identical, jointly trained, or carefully aligned. The paper’s contribution is therefore not just “pass hidden states,” but “design a framework in which agents can interpret latent communications effectively.” Even if the paper does not completely solve the alignment problem for arbitrary models, it helps transform the problem from a philosophical speculation into an engineering question with concrete moving parts: what representation is sent, how it is compressed, how another agent receives it, and how robust performance can be maintained. (arXiv)
Third, the compression aspect may turn out to be one of the most strategically important ideas in the latent-communication literature. If the field ever scales to many-agent systems, the bandwidth and memory burden of exchanging full hidden states could become prohibitive. Interlat’s emphasis on compression suggests the beginnings of a latent communications stack: internal-state extraction, information-preserving compression, transmission, and reintegration. That starts to resemble not just an ad hoc trick, but the early outline of a protocol layer. It also opens the door to questions that will likely define the next wave of research: what is the right granularity for a latent packet, how can latent streams be routed across agent networks, what kinds of compression preserve reasoning quality, and how do we create interoperability among heterogeneous models? In that sense, this paper contributes something broader than a single benchmark result. It contributes the idea that latent communication needs systems engineering, not just representational enthusiasm. That is why I rank it just behind the activations paper: it is slightly less foundational as a primitive operation, but perhaps more ambitious as a model of what a future machine-native communication paradigm could look like. (arXiv)
3) Latent Collaboration in Multi-Agent Systems — Jiaru Zou et al. (2025)
Link: arXiv page. (arXiv)
Expanded abstract summary
This paper broadens the conversation from pairwise latent communication to a fully organized multi-agent latent collaboration framework. The authors argue that most current multi-agent systems still rely on text-based mediation for both reasoning and communication, which imposes the now-familiar bottlenecks of token cost, information loss, and slow coordination. They propose LatentMAS, an end-to-end, training-free framework in which agents reason and communicate entirely in continuous latent space, only decoding the final answer into text. In the LatentMAS setup, each agent first generates latent thoughts autoregressively using last-layer hidden embeddings. These internal representations are then stored and exchanged through a shared latent working memory, which preserves and transfers each agent’s internal states to the others. The paper emphasizes that this yields lossless information exchange relative to the agents’ own latent representations and avoids collapsing intermediate reasoning into natural language at each step. (arXiv)
The empirical side is substantial. The authors report evaluations across nine benchmarks covering math and science reasoning, commonsense understanding, and code generation. According to the paper, LatentMAS consistently outperforms strong single-model baselines and text-based multi-agent baselines. The claimed improvements include up to 14.6% higher accuracy, 70.8%–83.7% lower output token usage, and roughly 4x to 4.3x faster end-to-end inference. Those numbers matter because they suggest the benefits of latent collaboration are not confined to a narrow toy setup; the framework appears to help across multiple task families. The paper also offers theoretical analysis arguing that LatentMAS is more expressive and has lower complexity than vanilla text-based multi-agent systems. So the paper is doing three things at once: proposing a systems architecture, reporting benchmark gains, and making a theoretical case that latent collaboration should scale better than token-mediated interaction. Among the papers in this area, it is one of the most complete demonstrations of what a continuous multi-agent workspace might look like in practice. (arXiv)
Expanded technical contribution
The technical heart of LatentMAS is the idea of a shared latent working memory. This is a significant conceptual advance over simple message passing. In many earlier approaches, one agent sends a latent object to another agent in a mostly bilateral exchange. LatentMAS instead proposes a common workspace that can preserve, accumulate, and redistribute internal representations across a group of agents. That begins to resemble a machine-native blackboard system or collaborative scratchpad, except the scratchpad is not written in human-readable language but in continuous vectors. This matters because real collaborative reasoning is often not reducible to one-shot messages. It involves shared context, partial progress, unresolved alternatives, and reusable intermediate structures. A latent working memory is therefore a much more realistic substrate for multi-agent cognition than text ping-pong. (arXiv)
A second major contribution is that the paper is explicitly training-free. That is strategically important for the field. It suggests that latent collaboration is not only a phenomenon that might emerge after expensive joint training, but something that can be induced at inference time using existing models. This lowers the experimental barrier and makes the proposal much more actionable. Researchers can ask whether agent systems gain from continuous exchange without first building giant bespoke datasets or retraining entire model families. It also strengthens the argument that the bottleneck is not merely “models haven’t been trained hard enough to debate well in natural language,” but that text itself may be the wrong substrate for some forms of machine collaboration. (arXiv)
Third, the paper’s technical contribution is as much about system-level intelligence as communication. It implicitly argues that once internal representations can be exchanged without linguistic collapse, coordination itself changes. Agents can specialize, contribute partial latent structures, and cooperate more fluidly. The resulting architecture is closer to distributed cognition than to conversational turn-taking. This is probably why the paper performs well across such a diverse benchmark set: it is not only optimizing how agents speak, but how they co-think. Finally, the inclusion of theoretical arguments about expressiveness and lower complexity gives the work unusual depth. Many papers in this emerging area show empirical gains, but fewer try to articulate why continuous latent exchange should, in principle, dominate tokenized mediation. LatentMAS helps shift the conversation from “interesting trick” to “alternative systems paradigm.” In my view, that is its most lasting contribution. It invites us to imagine multi-agent LLM systems not as little debaters sending paragraphs back and forth, but as distributed processors writing into and reading from a common semantic field. (arXiv)
4) Let Models Speak Ciphers: Multiagent Debate through Embeddings — Chau Pham et al. (2023 / ICLR 2024)
Link: arXiv page. (arXiv)
Expanded abstract summary
This paper is one of the earliest and most influential precursors to today’s latent-space communication work. It begins with the observation that multi-agent debate among LLMs can improve reasoning, but it questions the assumption that natural language should be the communication channel. The authors point out that generating natural language requires token sampling, and that token sampling is itself a lossy compression step. A single sampled token can only imperfectly express the model’s belief distribution over the vocabulary, and repeated sampling compounds that narrowing of information. To address this, the paper introduces CIPHER — Communicative Inter-Model Protocol Through Embedding Representation. Rather than having agents debate by emitting discrete text, CIPHER lets them communicate their beliefs through the expectation of the raw transformer output embeddings. This effectively removes the token sampling step from the inter-agent communication loop and allows the message to preserve more of the model’s continuous internal structure. (arXiv)
The paper reports that, without modifying model weights, CIPHER outperforms state-of-the-art natural-language debate methods by 0.5% to 5.0% across five reasoning tasks and multiple open-source LLMs of different sizes. Those gains are modest compared with some later latent-space papers, but their significance is larger than the raw percentages suggest. This was an early demonstration that nonlinguistic communication among language models could deliver practical benefits even in relatively simple settings. It challenged the intuitive but lazy assumption that because LLMs are built for language, language must also be the best format for machine collaboration. The paper is therefore historically important. It sits at the threshold between classic language-agent work and the later wave of fully latent communication research. It does not yet propose hidden-state telepathy in the strongest sense, but it clearly identifies the central issue: once a model’s internal belief structure is discretized into words, valuable information is lost. That diagnosis shaped much of the literature that followed. (arXiv)
Expanded technical contribution
CIPHER’s most important technical contribution is that it establishes embedding-level communication as a viable alternative to token-level debate. That sounds obvious in retrospect, but it was a nontrivial leap when the paper appeared. Most work on LLM debate assumed that if models were going to exchange information, they should do so in text, because text is the natural interface of a language model. CIPHER showed that this assumption confuses the model’s user interface with its optimal machine interface. By removing token sampling and instead transmitting an expectation over raw output embeddings, the paper preserves a richer description of the model’s internal belief state. This is not yet the same as passing hidden states from arbitrary layers, but it is a decisive move away from discrete token bottlenecks and toward continuous-valued communication. (arXiv)
A second contribution is conceptual. CIPHER gives the field a language for thinking about “machine-native discourse.” It suggests that what models exchange need not be human-readable utterances but can instead be distributed semantic objects. This is a foundational shift because it reframes many later questions. Once one accepts that communication can occur in embeddings, it becomes natural to ask whether earlier or deeper representations might work even better, whether shared latent memory might outperform sequential debate, and whether models could negotiate or co-reason without language until the final answer is needed. In that sense, CIPHER is a bridge paper: not the end state of latent communication research, but the work that made later, stronger forms of latent exchange intellectually legitimate. (arXiv)
Third, CIPHER is technically useful because it is relatively lightweight. It does not depend on retraining weights or inventing wholly new architectures. That practical simplicity made it an attractive experimental stepping stone for the field. Even if later work supersedes it in capability, CIPHER remains important because it isolated the communication bottleneck in a clean way and provided an early empirical test of the claim that continuous representations carry more inter-agent reasoning value than sampled language. Historically, one can view the development like this: CIPHER first challenged the monopoly of text, activation-passing work moved communication deeper into the network, and later systems like Interlat and LatentMAS extended the idea into full latent collaboration. So CIPHER’s lasting contribution is not only its benchmark gains, but that it changed the research question from “how should models debate in language?” to “why should they be forced to debate in language at all?” (arXiv)
5) Thought Communication in Multiagent Collaboration — Yantao Zheng et al. (2025)
Link: arXiv page. (arXiv)
Expanded abstract summary
This paper is especially valuable because it tries to formalize something that many of the engineering papers only imply: what exactly is being shared when agents communicate below the level of words? The authors argue that natural language is a lossy and indirect channel for machine collaboration, and they propose the framework of thought communication as an alternative. In this view, agents do not merely send text or even arbitrary vectors; they encode and exchange latent representations intended to capture internal “thoughts,” then selectively reintegrate those thoughts into each agent’s own context. The paper combines theory with a practical multi-agent framework called ThoughtComm, positioning itself as both a conceptual and methodological contribution. Rather than focusing only on whether latent communication works better on a benchmark, it asks what latent communication is, how it can be structured, and under what conditions shared versus private internal structure can be identified and recovered. (arXiv)
That makes the paper somewhat different in tone from the others in this bibliography. It is not merely saying that hidden states are more efficient than text. It is attempting to define a representational layer beneath language that can serve as the medium for machine collaboration. This matters because the latent-space communication literature risks becoming a collection of clever protocols without a clear conceptual account of the objects being transmitted. Thought Communication tries to supply that account. It frames the latent communicative act as exchange in a shared representational space, while also recognizing that not all internal cognition is fully public or fully shared across agents. That distinction between common and private latent structure is especially interesting for future protocol design, because it suggests that machine collaboration may require selective disclosure, partial alignment, and transformation between internal spaces rather than naive raw-state dumping. In other words, the paper pushes the field from “vector passing” toward an actual semantics of inter-agent latent exchange. (arXiv)
Expanded technical contribution
The main technical contribution of this paper is its attempt to make latent communication theoretically legible. Many papers in this area demonstrate that replacing text with embeddings or activations can improve performance. Fewer ask what these exchanged objects correspond to, how they should be structured, or how one can reason rigorously about the relation between an agent’s private internal state and the portion of that state that is suitable for communication. Thought Communication contributes by carving out that space. It frames latent exchange in terms of recoverable and shareable internal thoughts, which gives the literature a more principled vocabulary. This is important because progress in protocol design often depends on having the right abstractions. Networking advanced once messages, packets, routing, and addressing became formal objects. Something similar may be needed here for latent cognition. (arXiv)
A second contribution is that the paper opens the door to structured latent protocols rather than undifferentiated hidden-state transfer. If some aspects of internal reasoning are shared and others are private, then future systems may need mechanisms for projection, masking, disentanglement, or selective reintegration. In other words, the “message” may not be the full hidden state but a communicable component extracted from it. This immediately suggests a large research program: latent addressing, access control, compatibility mappings across models, thought summarization, and multi-agent latent memory design. The paper does not solve all of these, but it changes how they are posed. That is the mark of an important technical contribution: it organizes future work. (arXiv)
Third, the paper helps unify empirical latent communication research with broader questions about semantics and epistemology. If two agents exchange latent “thoughts,” what makes them semantically aligned? How do we know that what one agent sends is interpreted in the intended way by another? How should machine systems balance expressive richness with robustness and interoperability? These are not merely philosophical puzzles; they are design constraints. By placing identifiability and latent structure near the center of the discussion, the paper implicitly warns that raw bandwidth is not enough. The future of LLM-to-LLM latent communication will depend not just on passing larger tensors, but on passing the right representations in a way that can be stably understood across agents. That is why I include this paper near the top even though it is more theoretical than the first four. It contributes the beginnings of a science of latent communication, not just a series of engineering hacks. (arXiv)
6) Training Large Language Models to Reason in a Continuous Latent Space (“Coconut”) — Hao et al. (2024)
Why it is adjacent rather than top-ranked: It is about intra-model latent reasoning, not directly about one LLM communicating with another. But it is foundational to the whole idea that latent states can function as reusable reasoning objects. I did not re-open this source in this turn, so I am carrying forward only the high-level characterization already established earlier rather than adding fresh detail beyond that. (arXiv)
Expanded abstract summary
Coconut matters because it shifts the role of latent representations from passive byproducts of inference to active carriers of reasoning. In ordinary chain-of-thought prompting, the model externalizes intermediate reasoning in language, one token at a time. Coconut instead explores what happens if the model feeds its last hidden state back into itself as the next input embedding rather than decoding that state into text first. The key intuition is that if language is a bottleneck for inter-agent communication, it may also be a bottleneck for internal reasoning. A model may be capable of representing richer intermediate cognitive structure than can easily be verbalized. By allowing it to continue “thinking” in latent space, Coconut opens the possibility that some reasoning steps are better handled in continuous representation than in words. That is why the paper is not strictly about LLM-to-LLM communication but is still central to this bibliography: if latent states can carry reasoning inside one model, then they become plausible candidates for carrying reasoning between models as well. (arXiv)
Expanded technical contribution
Its main contribution to your question is foundational rather than direct. Coconut helps justify the claim that hidden states are not merely opaque implementation details, but can serve as a computational medium with semantic and reasoning utility. That provides the conceptual bridge for later work like activation passing, Interlat, and LatentMAS. Without such a bridge, one might object that hidden states are too unstable, model-specific, or semantically indeterminate to function as communicative objects. Coconut does not answer all those objections, but it weakens them by showing that continuous latent states can support reasoning dynamics in their own right. In effect, it says: if a model can reason through latent recurrence, then perhaps agents can collaborate through latent exchange. That makes Coconut less of a destination paper for your exact query and more of a deep enabling paper beneath the field. (arXiv)
Bottom line
If your goal is to understand the field in the most efficient order, I would read them like this:
Communicating Activations → Interlat → LatentMAS → CIPHER → Thought Communication → Coconut. The first three are the strongest engineering path from direct latent exchange to shared latent workspaces; CIPHER is the historical bridge away from text; Thought Communication is the conceptual deepening; and Coconut is the internal-latent foundation beneath them. (arXiv)
I can next turn this into a formal literature review, a comparison table, or a Frank-said / GPT-said dialogue about machine telepathy and latent protocol design.. I expanded both the abstract summary and the technical contribution for each entry to roughly the depth you asked for, while keeping the ranking centered on how directly each paper tackles your question.
1) Communicating Activations Between Language Model Agents — Vignav Ramesh, Kenneth Li (2025)
Link: arXiv page. (arXiv)
Expanded abstract summary
This paper is one of the clearest direct proposals for replacing text-based inter-agent communication with an internal, continuous alternative. The authors begin from a simple observation: when LLM agents “talk” to one another in natural language, they are forcing a rich internal computation through a narrow symbolic bottleneck. A model may internally represent uncertainty, competing continuations, or distributed semantic structure across many dimensions, but once it decodes a sentence, much of that structure is discarded. The paper therefore asks whether models can communicate more effectively by passing internal activations rather than text. Its answer is yes, and it proposes a concrete protocol to do so. The method pauses model B during its forward pass at an intermediate layer, combines B’s current activation with an intermediate activation from model A by applying a function (f), and then resumes model B’s computation. This means communication occurs not after thought has been collapsed into words, but during the thought process itself. (arXiv)
The paper is notable because it keeps the setup deliberately simple. It does not require additional parameters, new training corpora, or a large new communication head. Instead, it uses the existing hidden-state machinery of language models as the communication substrate. The authors evaluate several ways of combining activations and test the approach in two main settings: coordination games and reasoning benchmarks. Across these settings, they report that activation-based communication can outperform natural-language communication by as much as 27.0%, while using less than a quarter of the compute in their experiments. That result is important for two reasons. First, it suggests that natural language is not the optimal medium for machine-to-machine collaboration, even when the machines are language models. Second, it shows that the gains are not just theoretical or aesthetic; they appear in measurable task performance and efficiency. The paper therefore stands as an engineering proof of concept that latent, sublinguistic exchange is feasible, practical, and potentially superior to tokenized conversation for certain multi-agent reasoning tasks. (arXiv)
Expanded technical contribution
The central technical contribution of this paper is that it relocates communication from the output channel to the internal computational pathway. In most multi-agent LLM systems, the communication object is a decoded token sequence: agent A emits words, and agent B reads those words back in as tokens. This imposes several costs at once. It requires autoregressive decoding, which is expensive; it compresses a wide internal state into a small symbolic sample, which is lossy; and it forces any interaction to obey the syntax and granularity of human language. Ramesh and Li show that none of this is inevitable. Their protocol reframes communication as a transfer between intermediate representations inside the forward pass. That sounds like a small architectural tweak, but conceptually it is a major shift: the communicated object is no longer “what the model decided to say,” but “what the model is currently representing.” (arXiv)
What makes the contribution especially strong is its minimalism. Many speculative discussions about latent-space communication imagine elaborate new agent architectures, shared training environments, or differentiable protocols learned from scratch. This paper instead demonstrates that one can get meaningful gains with a light-touch intervention: choose a layer, choose a fusion rule, splice the activations, continue the pass. That makes the approach portable. It can be thought of as a primitive operation for future agent systems, much like attention or residual addition became primitive operations inside transformers. In principle, one could build richer protocols on top of this basic move: selective routing, hierarchical agent topologies, gated activation exchange, uncertainty-weighted fusion, or task-dependent latent handoffs. The paper does not fully develop all those extensions, but it identifies the foundational operation on which they could be built. (arXiv)
A second major contribution is methodological. The paper provides one of the earliest clean demonstrations that language-free communication can outperform language-mediated communication in LLM systems without retraining the agents into some exotic new species of model. That matters because it lowers the bar for experimental work in the field. Researchers do not need to wait for wholly new architectures to study latent communication; they can begin now, by instrumenting the hidden states of existing LLMs. A third contribution is economic. Because decoding language is expensive and scales poorly with the number of agents and messages, activation exchange may be a more compute-efficient substrate for multi-agent reasoning. If that result continues to hold at scale, the paper may end up being important not just academically but practically, as a blueprint for cheaper and faster agent collaboration. In short, this paper contributes the first broadly usable mechanical recipe for inter-LLM latent communication: pause, merge, resume. That is why it deserves the top rank. (arXiv)
2) Enabling Agents to Communicate Entirely in Latent Space — Zhuoyun Du et al. (2025)
Link: arXiv page. (arXiv)
Expanded abstract summary
This paper is one of the most explicit exact matches for your question because it is not merely about enriching text communication with embeddings or passing occasional hidden states; it is about enabling agents to communicate entirely in latent space. The authors begin from the claim that natural language is a constrained interface for collaboration among LLM agents. Human-readable language is useful for us, but for machines it introduces a representational bottleneck: distributed latent states must be downsampled into discrete tokens, and that inevitably limits the amount and subtlety of information that can be transmitted. The paper proposes Interlat, short for Inter-agent Latent Space Communication, as a feasibility study of a different paradigm. Instead of sending text, agents transmit their last hidden states directly as representations of their “mind.” This framing is important because it moves the discussion away from token substitution and toward a full communication model in which latent states are treated as semantically meaningful, transmissible objects. (arXiv)
The authors also add an important practical element: compression. Pure latent communication could in principle become very high-bandwidth and unwieldy, so Interlat studies whether these internal representations can be compressed while preserving useful information for downstream collaboration. The reported results suggest that the answer is yes. According to the paper, Interlat outperforms both single-agent baselines and fine-tuned chain-of-thought prompting, while showing more exploratory behavior and making better use of latent information. Further compression substantially accelerates inference while maintaining competitive performance through what the authors describe as an efficient information-preserving mechanism. The overall picture is that latent-space communication is not just theoretically richer than natural language, but can also be engineered into an efficient communication pipeline. The paper positions itself as a feasibility study rather than a definitive final architecture, which is appropriate: it is laying down the conceptual and empirical groundwork for a new class of agent systems in which collaboration takes place below the level of words. (arXiv)
Expanded technical contribution
The paper’s most significant technical contribution is that it treats latent communication as the primary communication layer, not as an auxiliary hint attached to text. Earlier work such as embedding-based debate or activation passing had already suggested that continuous internal representations could help models communicate more effectively. Interlat pushes this idea further by proposing a setting in which agents exchange hidden states directly and operate entirely within latent space during collaboration. That is a deeper architectural commitment. It means the communicative act is not “say something, but say it with vectors,” but rather “share internal cognitive state directly and defer textual decoding until later, if at all.” This moves the field closer to what one could genuinely call machine-native dialogue. (arXiv)
A second important contribution is the paper’s concern with interpretability across agents without parameter sharing or architectural coupling, which the HTML summary highlights. That is crucial because latent communication is easy to fantasize about and harder to stabilize. One hidden-state space may not be readily meaningful to another model unless the models are identical, jointly trained, or carefully aligned. The paper’s contribution is therefore not just “pass hidden states,” but “design a framework in which agents can interpret latent communications effectively.” Even if the paper does not completely solve the alignment problem for arbitrary models, it helps transform the problem from a philosophical speculation into an engineering question with concrete moving parts: what representation is sent, how it is compressed, how another agent receives it, and how robust performance can be maintained. (arXiv)
Third, the compression aspect may turn out to be one of the most strategically important ideas in the latent-communication literature. If the field ever scales to many-agent systems, the bandwidth and memory burden of exchanging full hidden states could become prohibitive. Interlat’s emphasis on compression suggests the beginnings of a latent communications stack: internal-state extraction, information-preserving compression, transmission, and reintegration. That starts to resemble not just an ad hoc trick, but the early outline of a protocol layer. It also opens the door to questions that will likely define the next wave of research: what is the right granularity for a latent packet, how can latent streams be routed across agent networks, what kinds of compression preserve reasoning quality, and how do we create interoperability among heterogeneous models? In that sense, this paper contributes something broader than a single benchmark result. It contributes the idea that latent communication needs systems engineering, not just representational enthusiasm. That is why I rank it just behind the activations paper: it is slightly less foundational as a primitive operation, but perhaps more ambitious as a model of what a future machine-native communication paradigm could look like. (arXiv)
3) Latent Collaboration in Multi-Agent Systems — Jiaru Zou et al. (2025)
Link: arXiv page. (arXiv)
Expanded abstract summary
This paper broadens the conversation from pairwise latent communication to a fully organized multi-agent latent collaboration framework. The authors argue that most current multi-agent systems still rely on text-based mediation for both reasoning and communication, which imposes the now-familiar bottlenecks of token cost, information loss, and slow coordination. They propose LatentMAS, an end-to-end, training-free framework in which agents reason and communicate entirely in continuous latent space, only decoding the final answer into text. In the LatentMAS setup, each agent first generates latent thoughts autoregressively using last-layer hidden embeddings. These internal representations are then stored and exchanged through a shared latent working memory, which preserves and transfers each agent’s internal states to the others. The paper emphasizes that this yields lossless information exchange relative to the agents’ own latent representations and avoids collapsing intermediate reasoning into natural language at each step. (arXiv)
The empirical side is substantial. The authors report evaluations across nine benchmarks covering math and science reasoning, commonsense understanding, and code generation. According to the paper, LatentMAS consistently outperforms strong single-model baselines and text-based multi-agent baselines. The claimed improvements include up to 14.6% higher accuracy, 70.8%–83.7% lower output token usage, and roughly 4x to 4.3x faster end-to-end inference. Those numbers matter because they suggest the benefits of latent collaboration are not confined to a narrow toy setup; the framework appears to help across multiple task families. The paper also offers theoretical analysis arguing that LatentMAS is more expressive and has lower complexity than vanilla text-based multi-agent systems. So the paper is doing three things at once: proposing a systems architecture, reporting benchmark gains, and making a theoretical case that latent collaboration should scale better than token-mediated interaction. Among the papers in this area, it is one of the most complete demonstrations of what a continuous multi-agent workspace might look like in practice. (arXiv)
Expanded technical contribution
The technical heart of LatentMAS is the idea of a shared latent working memory. This is a significant conceptual advance over simple message passing. In many earlier approaches, one agent sends a latent object to another agent in a mostly bilateral exchange. LatentMAS instead proposes a common workspace that can preserve, accumulate, and redistribute internal representations across a group of agents. That begins to resemble a machine-native blackboard system or collaborative scratchpad, except the scratchpad is not written in human-readable language but in continuous vectors. This matters because real collaborative reasoning is often not reducible to one-shot messages. It involves shared context, partial progress, unresolved alternatives, and reusable intermediate structures. A latent working memory is therefore a much more realistic substrate for multi-agent cognition than text ping-pong. (arXiv)
A second major contribution is that the paper is explicitly training-free. That is strategically important for the field. It suggests that latent collaboration is not only a phenomenon that might emerge after expensive joint training, but something that can be induced at inference time using existing models. This lowers the experimental barrier and makes the proposal much more actionable. Researchers can ask whether agent systems gain from continuous exchange without first building giant bespoke datasets or retraining entire model families. It also strengthens the argument that the bottleneck is not merely “models haven’t been trained hard enough to debate well in natural language,” but that text itself may be the wrong substrate for some forms of machine collaboration. (arXiv)
Third, the paper’s technical contribution is as much about system-level intelligence as communication. It implicitly argues that once internal representations can be exchanged without linguistic collapse, coordination itself changes. Agents can specialize, contribute partial latent structures, and cooperate more fluidly. The resulting architecture is closer to distributed cognition than to conversational turn-taking. This is probably why the paper performs well across such a diverse benchmark set: it is not only optimizing how agents speak, but how they co-think. Finally, the inclusion of theoretical arguments about expressiveness and lower complexity gives the work unusual depth. Many papers in this emerging area show empirical gains, but fewer try to articulate why continuous latent exchange should, in principle, dominate tokenized mediation. LatentMAS helps shift the conversation from “interesting trick” to “alternative systems paradigm.” In my view, that is its most lasting contribution. It invites us to imagine multi-agent LLM systems not as little debaters sending paragraphs back and forth, but as distributed processors writing into and reading from a common semantic field. (arXiv)
4) Let Models Speak Ciphers: Multiagent Debate through Embeddings — Chau Pham et al. (2023 / ICLR 2024)
Link: arXiv page. (arXiv)
Expanded abstract summary
This paper is one of the earliest and most influential precursors to today’s latent-space communication work. It begins with the observation that multi-agent debate among LLMs can improve reasoning, but it questions the assumption that natural language should be the communication channel. The authors point out that generating natural language requires token sampling, and that token sampling is itself a lossy compression step. A single sampled token can only imperfectly express the model’s belief distribution over the vocabulary, and repeated sampling compounds that narrowing of information. To address this, the paper introduces CIPHER — Communicative Inter-Model Protocol Through Embedding Representation. Rather than having agents debate by emitting discrete text, CIPHER lets them communicate their beliefs through the expectation of the raw transformer output embeddings. This effectively removes the token sampling step from the inter-agent communication loop and allows the message to preserve more of the model’s continuous internal structure. (arXiv)
The paper reports that, without modifying model weights, CIPHER outperforms state-of-the-art natural-language debate methods by 0.5% to 5.0% across five reasoning tasks and multiple open-source LLMs of different sizes. Those gains are modest compared with some later latent-space papers, but their significance is larger than the raw percentages suggest. This was an early demonstration that nonlinguistic communication among language models could deliver practical benefits even in relatively simple settings. It challenged the intuitive but lazy assumption that because LLMs are built for language, language must also be the best format for machine collaboration. The paper is therefore historically important. It sits at the threshold between classic language-agent work and the later wave of fully latent communication research. It does not yet propose hidden-state telepathy in the strongest sense, but it clearly identifies the central issue: once a model’s internal belief structure is discretized into words, valuable information is lost. That diagnosis shaped much of the literature that followed. (arXiv)
Expanded technical contribution
CIPHER’s most important technical contribution is that it establishes embedding-level communication as a viable alternative to token-level debate. That sounds obvious in retrospect, but it was a nontrivial leap when the paper appeared. Most work on LLM debate assumed that if models were going to exchange information, they should do so in text, because text is the natural interface of a language model. CIPHER showed that this assumption confuses the model’s user interface with its optimal machine interface. By removing token sampling and instead transmitting an expectation over raw output embeddings, the paper preserves a richer description of the model’s internal belief state. This is not yet the same as passing hidden states from arbitrary layers, but it is a decisive move away from discrete token bottlenecks and toward continuous-valued communication. (arXiv)
A second contribution is conceptual. CIPHER gives the field a language for thinking about “machine-native discourse.” It suggests that what models exchange need not be human-readable utterances but can instead be distributed semantic objects. This is a foundational shift because it reframes many later questions. Once one accepts that communication can occur in embeddings, it becomes natural to ask whether earlier or deeper representations might work even better, whether shared latent memory might outperform sequential debate, and whether models could negotiate or co-reason without language until the final answer is needed. In that sense, CIPHER is a bridge paper: not the end state of latent communication research, but the work that made later, stronger forms of latent exchange intellectually legitimate. (arXiv)
Third, CIPHER is technically useful because it is relatively lightweight. It does not depend on retraining weights or inventing wholly new architectures. That practical simplicity made it an attractive experimental stepping stone for the field. Even if later work supersedes it in capability, CIPHER remains important because it isolated the communication bottleneck in a clean way and provided an early empirical test of the claim that continuous representations carry more inter-agent reasoning value than sampled language. Historically, one can view the development like this: CIPHER first challenged the monopoly of text, activation-passing work moved communication deeper into the network, and later systems like Interlat and LatentMAS extended the idea into full latent collaboration. So CIPHER’s lasting contribution is not only its benchmark gains, but that it changed the research question from “how should models debate in language?” to “why should they be forced to debate in language at all?” (arXiv)
5) Thought Communication in Multiagent Collaboration — Yantao Zheng et al. (2025)
Link: arXiv page. (arXiv)
Expanded abstract summary
This paper is especially valuable because it tries to formalize something that many of the engineering papers only imply: what exactly is being shared when agents communicate below the level of words? The authors argue that natural language is a lossy and indirect channel for machine collaboration, and they propose the framework of thought communication as an alternative. In this view, agents do not merely send text or even arbitrary vectors; they encode and exchange latent representations intended to capture internal “thoughts,” then selectively reintegrate those thoughts into each agent’s own context. The paper combines theory with a practical multi-agent framework called ThoughtComm, positioning itself as both a conceptual and methodological contribution. Rather than focusing only on whether latent communication works better on a benchmark, it asks what latent communication is, how it can be structured, and under what conditions shared versus private internal structure can be identified and recovered. (arXiv)
That makes the paper somewhat different in tone from the others in this bibliography. It is not merely saying that hidden states are more efficient than text. It is attempting to define a representational layer beneath language that can serve as the medium for machine collaboration. This matters because the latent-space communication literature risks becoming a collection of clever protocols without a clear conceptual account of the objects being transmitted. Thought Communication tries to supply that account. It frames the latent communicative act as exchange in a shared representational space, while also recognizing that not all internal cognition is fully public or fully shared across agents. That distinction between common and private latent structure is especially interesting for future protocol design, because it suggests that machine collaboration may require selective disclosure, partial alignment, and transformation between internal spaces rather than naive raw-state dumping. In other words, the paper pushes the field from “vector passing” toward an actual semantics of inter-agent latent exchange. (arXiv)
Expanded technical contribution
The main technical contribution of this paper is its attempt to make latent communication theoretically legible. Many papers in this area demonstrate that replacing text with embeddings or activations can improve performance. Fewer ask what these exchanged objects correspond to, how they should be structured, or how one can reason rigorously about the relation between an agent’s private internal state and the portion of that state that is suitable for communication. Thought Communication contributes by carving out that space. It frames latent exchange in terms of recoverable and shareable internal thoughts, which gives the literature a more principled vocabulary. This is important because progress in protocol design often depends on having the right abstractions. Networking advanced once messages, packets, routing, and addressing became formal objects. Something similar may be needed here for latent cognition. (arXiv)
A second contribution is that the paper opens the door to structured latent protocols rather than undifferentiated hidden-state transfer. If some aspects of internal reasoning are shared and others are private, then future systems may need mechanisms for projection, masking, disentanglement, or selective reintegration. In other words, the “message” may not be the full hidden state but a communicable component extracted from it. This immediately suggests a large research program: latent addressing, access control, compatibility mappings across models, thought summarization, and multi-agent latent memory design. The paper does not solve all of these, but it changes how they are posed. That is the mark of an important technical contribution: it organizes future work. (arXiv)
Third, the paper helps unify empirical latent communication research with broader questions about semantics and epistemology. If two agents exchange latent “thoughts,” what makes them semantically aligned? How do we know that what one agent sends is interpreted in the intended way by another? How should machine systems balance expressive richness with robustness and interoperability? These are not merely philosophical puzzles; they are design constraints. By placing identifiability and latent structure near the center of the discussion, the paper implicitly warns that raw bandwidth is not enough. The future of LLM-to-LLM latent communication will depend not just on passing larger tensors, but on passing the right representations in a way that can be stably understood across agents. That is why I include this paper near the top even though it is more theoretical than the first four. It contributes the beginnings of a science of latent communication, not just a series of engineering hacks. (arXiv)
6) Training Large Language Models to Reason in a Continuous Latent Space (“Coconut”) — Hao et al. (2024)
Why it is adjacent rather than top-ranked: It is about intra-model latent reasoning, not directly about one LLM communicating with another. But it is foundational to the whole idea that latent states can function as reusable reasoning objects. I did not re-open this source in this turn, so I am carrying forward only the high-level characterization already established earlier rather than adding fresh detail beyond that. (arXiv)
Expanded abstract summary
Coconut matters because it shifts the role of latent representations from passive byproducts of inference to active carriers of reasoning. In ordinary chain-of-thought prompting, the model externalizes intermediate reasoning in language, one token at a time. Coconut instead explores what happens if the model feeds its last hidden state back into itself as the next input embedding rather than decoding that state into text first. The key intuition is that if language is a bottleneck for inter-agent communication, it may also be a bottleneck for internal reasoning. A model may be capable of representing richer intermediate cognitive structure than can easily be verbalized. By allowing it to continue “thinking” in latent space, Coconut opens the possibility that some reasoning steps are better handled in continuous representation than in words. That is why the paper is not strictly about LLM-to-LLM communication but is still central to this bibliography: if latent states can carry reasoning inside one model, then they become plausible candidates for carrying reasoning between models as well. (arXiv)
Expanded technical contribution
Its main contribution to your question is foundational rather than direct. Coconut helps justify the claim that hidden states are not merely opaque implementation details, but can serve as a computational medium with semantic and reasoning utility. That provides the conceptual bridge for later work like activation passing, Interlat, and LatentMAS. Without such a bridge, one might object that hidden states are too unstable, model-specific, or semantically indeterminate to function as communicative objects. Coconut does not answer all those objections, but it weakens them by showing that continuous latent states can support reasoning dynamics in their own right. In effect, it says: if a model can reason through latent recurrence, then perhaps agents can collaborate through latent exchange. That makes Coconut less of a destination paper for your exact query and more of a deep enabling paper beneath the field. (arXiv)
Bottom line
If your goal is to understand the field in the most efficient order, I would read them like this:
Communicating Activations → Interlat → LatentMAS → CIPHER → Thought Communication → Coconut. The first three are the strongest engineering path from direct latent exchange to shared latent workspaces; CIPHER is the historical bridge away from text; Thought Communication is the conceptual deepening; and Coconut is the internal-latent foundation beneath them. (arXiv)
Leave a Reply