Researchers develop shared latent space for LLM collaboration

Global: Researchers develop shared latent space for LLM collaboration

New preprint outlines a high‑bandwidth communication channel for large language models

On January 4, 2026, a team of five authors—including Lucio M. Dery and Zohar Yahav—submitted a paper to arXiv describing a method that aligns the key‑value (k‑v) caches of multiple large language models (LLMs) to create a shared representation space. The work aims to enable direct, internal‑state communication between models without modifying their pre‑trained parameters, thereby facilitating more efficient multi‑model problem solving.

Aligning k‑v caches to form a shared latent space

The proposed approach introduces adapters that translate each model’s internal cache into and out of a common space. By synchronizing these caches, the authors claim the system provides a high‑bandwidth channel that surpasses traditional text‑based exchanges, allowing models to access each other’s intermediate computations directly.

Adapter‑based translation preserves original model weights

Crucially, the method does not alter the underlying weights of the participating LLMs. Instead, lightweight adapter modules are trained to map the model‑specific cache representations to the shared space and back, preserving the integrity of the original models while adding a collaborative layer.

Experimental validation with Gemma‑2 models

In a series of experiments using Gemma‑2 variants, the researchers reported that the shared‑space configuration not only enabled seamless inter‑model dialogue but also yielded measurable performance gains on benchmark tasks. The paper notes that the experiments were conducted on a dataset of unspecified size, and the PDF of the submission is 1,222 KB.

Direct transfer of learned skills between models

The authors further demonstrate that the shared latent space can convey learned artifacts such as soft prompts from one model to another. This capability suggests that knowledge acquired by one instance can be reused by a different model without retraining, potentially reducing computational overhead in multi‑model pipelines.

Implications for future AI system design

By providing a mechanism for LLMs to share internal states, the study contributes to a growing body of research on collaborative AI architectures. The authors position their work as a step toward systems where multiple models cooperate fluidly, opening avenues for more complex problem solving that leverages the strengths of diverse model families.This report is based on information from arXiv, licensed under Academic Preprint / Open Access. Based on the abstract of the research paper. Full text available via ArXiv.