The math that connects two AI models
When you send a long conversation to an artificial intelligence model, the system spends an enormous amount of processing power just to "read" the entire context before generating the first word. This process, called prefill, is the most expensive phase of inference — and it is entirely wasted whenever the same conversation is routed to a different model from the same manufacturer mid-stream. Imagine starting a question in a small, inexpensive model, and when the question becomes complex, the system transfers the conversation to a larger one. Until now, the large model had to read everything from scratch, as if the conversation had never happened. This is exactly the problem that a group of Nvidia researchers identified and solved with a surprisingly simple solution.
The paper, titled "Cross-Model KV Cache Transfer in LLM Families: A Closed-Form Linear Mapping for Prefill Reuse" and submitted to arXiv on August 4, 2026, by Taekyung Heo, Rasoul Shafipour, and Nvidia colleagues, shows that it is possible to transfer the attention cache (the so-called KV cache) from one model to another within the same family. Instead of re-reading the entire context, the receiving model uses a direct linear formula to "translate" the original model's internal notes into the format it understands. The result: up to 25 times faster than full reprocessing, with 73% to 98% retention of original accuracy on four of six tested pairs.
Why the cache is lost on model switching
To understand the innovation, it helps first to understand what happens during prefill. Every language model generates, for each processed token, a pair of vectors: the key and the value. These pairs make up the KV cache — essentially the notes the model takes while reading your prompt. When the prompt is sent to the same model again, these notes can be reused, which is why providers like Anthropic and OpenAI offer 90% discounts on input costs for cached prompts.
The problem is that these notes are shaped by that model's specific weights. A small model and a large model, even belonging to the same family and trained on similar data, produce notes in different representation spaces. It's as if two people from different families had completely incompatible note-taking systems, even though they understand the same language. When a conversation is routed from one model to another — an increasingly common practice in production systems that perform cost and quality cascading — the cache is discarded and the new model has to start over.
The Nvidia researchers observed that this premise, which seemed unquestionable, was wrong. They discovered that there is a real linear relationship between the KV caches of different sizes within the same family of models.
The discovery: linear structure in cross-model caches
The team first tested something basic: is there any correlation between the notes of a small model and those of a larger one? Using Qwen3 14B and 32B, they applied a simple linear regression — the most basic type of mathematical formula — to map the keys from one layer of the small model to the corresponding keys of the large model. The result was revealing: a single layer of the small model already explained 56% of the variance in the larger model's keys and 32% of its values.
This is not a trivial number. In complex neural representation problems, predicting more than half the variance of a larger model with a simple linear formula from a smaller model indicates that the two share a deep internal structure. It's not irrelevant noise — it's signal.
The analysis was extended with greedy forward selection of layers: instead of using just one source layer, the team iteratively added the layers that most increased mapping accuracy. For the Qwen3 14B → 32B pair, a single layer captured only 66% of maximum possible accuracy for keys (with all layers), and 42% for values. The biggest gain came from moving to four layers, and accuracy stabilized around six layers.
The mapper design: three engineering pieces
From this discovery, the researchers built the cross-model cache mapper in three main components. The first is ridge regression (linear regression with regularization) per attention layer and head. Each target layer receives its own independent linear mapping — there are no shared parameters between different heads nor between keys and values. Regularization with lambda of 0.01 is used instead of pure least squares to avoid numerical problems when the feature dimension reaches tens of thousands.
The second component is source layer selection by target layer. Each layer in the receiving model decides which layers in the supplying model most predict its representation, and concatenates the KV caches from those layers as input. This is the single largest contributor among the mapper's three components — the paper's ablation shows that reducing from 8 to 1 source layer would cause a dramatic performance drop.
The third component is the content-space mapping, which separates positional rotation (RoPE) from semantic content. RoPE (Rotary Positional Encoding) applies a position-dependent rotation to keys — which means the same word has a different vector depending on its position in the sentence. By removing this rotation before fitting the mapper, the weights become valid regardless of context length. The receiving model reapplies its own RoPE after the transformation. This allows the same mapper to be used for prompts up to 32,000 tokens without recalibration.
Practical results: what works and what doesn't
The experiments covered six model pairs from three families (Qwen3, Llama 3.1, Ministral 3), all with matched-head configuration (matched-KV). The results split into two clear tiers.
At the top tier, four pairs maintained between 73% and 98% of the target model's accuracy across five benchmarks: ARC-Challenge, HellaSwag, WinoGrande, MMLU, and GSM8K, plus two additional checks. The most notable case was the Qwen3 14B → 32B, which retained 97.6% on ARC-Challenge, 96.3% on HellaSwag, 101% on WinoGrande, 97.6% on MMLU, and 98.5% on GSM8K — numbers that, on some benchmarks, slightly surpassed the target model's standalone accuracy. The cache conversion speed from small to large was 2.7 to 25 times faster than re-prefill. In the reverse direction, large to small, the acceleration was 3 to 7 times.
At the lower tier, two pairs fell significantly: Ministral 3B → 14B with 44.2% retention, and Ministral 8B → 14B with 41.6%. The crucial difference between the groups was that both poor pairs both targeted the same 14B Ministral 3 model, but with a significantly larger scale jump than the successful pairs. The paper did not conclude exactly why this specific configuration fails, but suggested that error concentration in attention-sensitive subspaces may be the determining factor — not the magnitude of the error, but where it lands in the representation space.
Here comes an interesting finding: when the researchers replaced linear regression with a nonlinear MLP (multi-layer perceptron), HellaSwag retention jumped from 14.7% to 92.3% on the 3B→14B pair, and from 58.7% to 95.5% on the 8B→14B pair. The trade-off is direct: you lose the "zero-training" property, but you regain accuracy. The MLP works by redistributing residual error away from attention-sensitive subspaces, which are the most critical for generation quality.
Implications for the AI ecosystem
This work touches on two simultaneous vectors in the AI industry. The first is the growing cost of inference in long-duration agentic sessions. As systems like LangMem, Mem0, and Zep evolve to give agents cross-turn memory capability, the amount of accumulated context increases — and with it, the cost of prefill. The second is multi-model orchestration, increasingly common in production: a small model for simple tasks, a large one for complex reasoning, and dynamic routing between them.
Nvidia's linear mapper has a one-time setup cost of 47 to 87 minutes on an 8-H100 GPU machine, per model pair. The total mapper size ranges from 4 to 12 GB per pair. Once installed, it completely eliminates the need for re-prefill on model swaps — and works without any neural training step, meaning any engineering team can apply it without needing training infrastructure.
The paper is clear about what has not yet been tested: mismatched head configurations, other text domains (the calibration set uses only FineWeb-Edu, educational web text), non-standard architectures (models that combine attention layers with state-space layers, such as Nvidia's own newer Nemotron models), and the fact that there is a minor benchmark leakage when selecting the optimal k parameter. But what is already validated is enough to change the cost calculation in production systems.
The question remains whether the ability to transfer knowledge between models within the same family will become a standard component of the inference stack — and what that means for manufacturers whose strategy depends on cache fragmentation as a competitive advantage.
Sources: arXiv, Few-Shot Academy, AI Daily Post
✓ Independent sources cross-checked and verified before publishing
