FrontierReportCore2026~18 min theory

DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression

Split the transformer so input is cheap and output is smart, then squeeze the KV cache to 890 bytes per token: the smallest model in a new architecture family.

Source

Where this sits

DeepSeek-V4: Towards Highly Efficient Million-Token Context IntelligenceDeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression

V4 made million-token context affordable on paper; V4.1-Flash is the release that turns cache compression into the organizing principle of a whole architecture. It is the first model of what DeepSeek calls a new family, sized small so the same structure can scale up later - with a larger sibling, V4.1-Pro, announced but not yet shipped at release time.

Splits the stack into a causal encoder and decoder so prefill activates 8B parameters per token and decode 16B, making input-heavy agent workloads cheap.
Introduces CSA2 with three static layer modes (Full, Reindex, Reuse) that share main KV and indexer keys across layers and reuse top-k indices, cutting indexer work.
Reduces the global KV cache to 890 bytes per token with FP4 caching - roughly a quarter of V4-Flash's - and, with sliding-window-attention (SWA) bounded replay, cuts the persistent footprint to about an eighth.
Adds native visual understanding, a 196B-parameter Engram conditional memory, and DSpark speculative decoding in one release.
Exposes reasoning effort as a continuous setting from 1 to 100 and cuts API prices, positioning a Flash-tier model against the previous flagship.

What it is

DeepSeek-V4.1-Flash, released September 10, 2026, is the smallest model of DeepSeek's new architecture family: a multimodal MoE with 552B backbone parameters, a one-million-token context, and native image plus text input. Its core idea is asymmetric compute. A Causal Encoder-Decoder stack (40 layers: a 20-layer causal encoder followed by a 20-layer decoder) projects the decoder's global KV cache from the encoder's final hidden states, so prefill activates only 8B parameters per token while decode activates 16B. On top of that, Compressed Sparse Attention 2 assigns each attention layer a static mode - Full, Reindex, or Reuse - to share KV and indexer state across layers, and a hierarchical sparse indexer bounds deeper indexing cost. With FP4 KV caching, the global cache comes to 890 bytes per token, about a quarter of V4-Flash's, and SWA bounded replay cuts the persistent cache footprint to about an eighth. The model was trained from scratch on 45T multimodal tokens and exposes a continuous reasoning-effort setting. It replaces V4-Flash in the API; the model card and technical report are open, and the weights are MIT licensed.

1Theory from first principles

The KV cache is the bill

In agentic workloads the input side dominates: long documents, long tool traces, long conversations where every turn re-reads the history. Two costs follow. Compute is paid at prefill for every input token, and memory is paid to store the KV cache for the whole context so decoding can read it later. On top of that, providers charge cache-hit input tokens much less than cache misses, so a small cache footprint is not just a GPU memory saving - it is the difference between a cached and an uncached price tier. V4.1-Flash attacks all of it at once: make prefill cheap, make the cache tiny, and make the expensive decode pass read as little as possible.

Asymmetric compute: a causal encoder and decoder

The Causal Encoder-Decoder (CED) split is the structural move. The first 20 layers form a causal encoder that processes the input; the last 20 form the decoder that generates. The decoder's global KV cache is projected from the encoder's final hidden states rather than derived from each decoder layer's own hidden states. Because prefilling is input-heavy and decoding is output-heavy, the split lets the model spend fewer parameters per token on the prefill (8B activated) and more on the decode (16B activated). For an agent that ingests a hundred thousand tokens and emits a short tool call, that asymmetry is exactly the right shape: the half of the model that touches most tokens is the cheap half.

CSA2: layer modes and a hierarchical indexer

V4.1 keeps the compressed-sparse-attention idea and makes it a per-layer design. Each attention layer is statically assigned one of three modes: Full maintains and writes the main KV and indexer state; Reindex recomputes sparse indices against shared state; Reuse simply reuses the top-k indices computed by an earlier layer. Sharing KV and indexer keys across layers means the cache and the indexer traffic are amortized over more computation. Then a hierarchical sparse indexer narrows the search: the first Full-mode layer constructs a candidate pool, and later indexing layers only score tokens inside that pool, which bounds the indexing cost independently of context length. In a design where the indexer is the remaining quadratic term, bounding it structurally is the important part.

SWA bounded replay: do not store what you can rebuild

Sliding-window attention normally creates a storage problem: to serve the next request you must either persist the window KV states to SSD or recompute them. V4.1-Flash chooses recomputation with a bound - SWA bounded replay reconstructs missing sliding-window KV states by replaying only the most recent n_win tokens. Because the window is small and the replay is bounded, the persistent footprint drops to roughly an eighth of V4-Flash's. The general lesson is worth more than the number: if a piece of state is cheap to reconstruct deterministically from a bounded suffix, it does not belong in durable storage at all.

Bytes per cached token

bytes_per_token = (layers * stored_entries_per_layer * dim_kept * bits_per_value) / (8 * compression)

The numerator counts how much must be stored per token; the denominator includes the compression and precision choices. V4.1-Flash drives the result down three ways: sharing KV across layers (fewer stored copies), compressing the sequence dimension (fewer entries), and storing what remains in FP4 with a small per-channel scale (fewer bits). The model card reports the combined result as 890 bytes per token in the global cache, roughly one quarter of V4-Flash. Note what the formula does not include: attention weights, the 196B-parameter Engram memory, and the rest of the fixed model footprint, so the ratio is a statement about the cache, not about total deployment memory.

Assigning per-layer attention modes

python

def assign_modes(num_layers, every_full=4):
    """Static schedule: one Full layer writes shared KV and indexer state,
    Reindex layers recompute top-k indices, Reuse layers inherit them."""
    modes = []
    for layer in range(num_layers):
        if layer % every_full == 0:
            modes.append("full")        # writes KV + indexer keys, builds pool
        elif layer % every_full == 1:
            modes.append("reindex")     # scores against shared state, new top-k
        else:
            modes.append("reuse")       # borrows the last top-k indices
    return modes

# print(assign_modes(20))
# ['full', 'reindex', 'reuse', 'reuse', 'full', 'reindex', 'reuse', ...]
# Mode sharing is what amortizes the cache and indexer traffic: not every layer
# pays to store keys or to run the indexer from scratch.

The schedule here is illustrative; the model card fixes the actual per-layer assignment statically and does not expose it as a runtime dial.
Reuse layers inherit indices, so they cannot recover if the shared candidate pool missed a token - that risk is the price of the saving.
A hierarchical indexer bounds the deeper layers' search to the pool built by the first Full layer, which is where the context-independence of indexer cost comes from.

Multimodal from the start, plus memory and drafting

V4.1-Flash is not a text model with a vision adapter bolted on. A vision encoder (DeepSeek-ViT, trained from scratch with 2D-RoPE and 3x3 pixel-unshuffle downsampling) and a two-layer MLP projector turn images into visual embeddings that are processed jointly with text from the beginning of language-model pretraining. The card also lists two unusual components: Engram, a 196B-parameter conditional memory sparsely accessed by token lookup, and DSpark, a speculative decoding scheme with semi-autoregressive draft generation and confidence-scheduled verification. Both are statements about where the remaining costs are - knowledge that can be looked up rather than recomputed, and cheap draft tokens to hide decode latency.

Reasoning effort as a serving dial

The model card exposes a continuously controllable reasoning effort from 1 to 100, with the API surfacing it through effort settings, and all reported instruct numbers use the maximum. This is the V3.1 mode switch evolved into a numeric cost dial: instead of two templates, the caller chooses how much deliberation to buy per request, and the model was trained to behave sensibly across the range. The pretraining story behind it: 45T multimodal tokens, sparse attention trained at 64K sequence length, and context extended to 1M at the 34T-token mark - long context is in the pretraining curriculum, not patched on in post-training.

KV cache per token across the DeepSeek generations: V4.1-Flash reports 890 bytes per token in the global cache, roughly a quarter of V4-Flash and, on the card's own arithmetic, over 400x below the first-generation V1 baseline.

2Inside the paper

Problem the report sets out to solve

Agentic workloads are input-heavy: they read long contexts, call tools, and repeat, so prefill compute and KV cache memory dominate cost. The report asks what an architecture looks like when cache compression is the primary objective rather than a side optimization, and whether a Flash-tier model built that way can beat the previous flagship on the work agents actually do.

Key idea to look for while reading

Four mechanisms and their accounting. The CED split (architecture section), CSA2's three layer modes and hierarchical indexer, SWA bounded replay, and FP4 KV caching. Then the pre-training section for the 45T/34T/64K schedule, and the post-training section, which is deliberately plain: standard SFT, RL, and on-policy distillation, with the real work in synthetic agent-task data at scale.

Evidence the report musters

Reported cache numbers: 890 bytes per token global KV, about a quarter of V4-Flash and roughly an eighth of its persistent footprint after bounded replay. Activation asymmetry: 8B parameters per token at prefill, 16B at decode. Training: 45T multimodal tokens from scratch, sparse attention at 64K, 1M context extension at 34T. Base-model rows show MMLU-Pro 74.1, HumanEval 79.4, GSM8K 93.0, DocVQA 95.6, MMMU-Pro 56.5. Instruct rows include Codeforces 3471, Terminal-Bench 2.1 90.6, DeepSWE v1.1 74.2, GPQA Diamond 90.9, and HLE with tools 63.9. The release positions it ahead of V4-Pro on several agentic measures while phasing the older flagship toward routing to Flash.

Limits to keep in view

All benchmark numbers are vendor reported; as of launch, independent evaluators had not published measurements of the model. The 'ahead of V4-Pro' claim is benchmark-specific: on GPQA Diamond, V4.1-Flash scores 90.9 against V4-Pro's 92.4, and frontier closed models report higher still on the newest terminal-bench tiers. One benchmark number, NL2Repo-Bench, differs between the changelog (65.4) and the model card (64.0) and was not reconciled. The cache reductions are cache reductions: total serving memory also includes weights and the 196B-parameter Engram, so a quarter of the KV cache is not a quarter of the deployment. The naming is also odd - a point release that is a new base model - and V4.1-Pro, the sibling the release implicitly promises, had no published date, weights, or card at release time.

Practitioner takeaway

Optimize the half of the stack that touches the most tokens. If prefill dominates your traffic, asymmetric activation is a bigger lever than a smaller model. Share state across layers and bound recomputation, because persistent cache is a storage and price-tier problem, not just a memory problem. And read vendor efficiency ratios carefully: ask per token, per layer, or per deployment, and check which denominator the claim uses.

3Implementation check

0 of 6 questions answered

1.In the Causal Encoder-Decoder design, where does the decoder's global KV cache come from?
2.Which statement correctly describes CSA2's three layer modes?
3.What problem does SWA bounded replay solve?
4.The 890 bytes per token figure measures what?
5.Which honest caveat about V4.1-Flash's benchmark table is supported by the sources?
6.A team serves long-document agent tasks where inputs are huge and outputs are short tool calls. Why does V4.1-Flash's asymmetric activation fit?

4Build it yourself

Cache one extreme per block, raw only when needed

Intermediate3-4 hours

Build the cache trade V4.1-Flash describes. Every block stores one summary value and its position; the effort knob upgrades the top blocks to raw values. A clue-routed expert compares the observed extremes, and the sweep reports accuracy against values kept per token for 0, 1, and 2 raw blocks.

Milestones

Generate sequences holding one +0.9 and one -0.9 token, plus a clue that names the hidden reading direction with 20% error.
Compute the full-cache ceiling with the clue-routed expert and report it at 1.0 values per token.
Implement decode(seq, clue, raw): summaries for every block, full values for the top raw blocks by salience, and 0.5 when an extreme is hidden inside a summary.
Sweep raw = 0, 1, 2 and record accuracy and values kept per token.
Identify the knee and explain why the second raw block is redundant here: the ambiguous block is always the top-salience block.

Starter code

python · editable

import random
random.seed(35)
# EXPECTED before TODOs: full cache 0.782 acc at 1.00 values/token; then
# "unimplemented: TODO: decode".
# EXPECTED after TODOs: raw 0 0.708 at 0.25, raw 1 0.782 at 0.50, raw 2
# 0.782 at 0.75: one raw block reaches full-cache quality, the second adds none.
N, L, B, FLIP = 400, 16, 4, 0.2
def make_example(rng):
    seq = [rng.uniform(-0.45, 0.45) for _ in range(L)]
    pos, neg = rng.sample(range(L), 2)
    seq[pos], seq[neg] = 0.9, -0.9
    domain = rng.randrange(2)
    clue = domain if rng.random() > FLIP else 1 - domain
    return seq, clue, int((pos < neg) == (domain == 0))
def blocks_of(seq): return [seq[b * B:(b + 1) * B] for b in range(L // B)]
def decode(seq, clue, raw):
    """TODO: return (prediction, values_kept). Every block keeps a summary:
    its extreme value and the position it sits at. The top raw blocks by
    salience (max |value|) keep all their values. Compare the observed +0.9
    and -0.9 positions; return 0.5 when one is hidden inside a summary.
    Reverse the comparison when clue is 1."""
    raise NotImplementedError("TODO: decode")
rule = lambda seq: int(seq.index(0.9) < seq.index(-0.9))
DATA = [make_example(random) for _ in range(N)]
def main():
    print("full-cache acc", round(sum((rule(s) if c == 0 else 1 - rule(s)) == y for s, c, y in DATA) / N, 3))
    try:
        for raw in (0, 1, 2):
            rows = [decode(s, c, raw) for s, c, y in DATA]
            hits = sum(1.0 if p == y else 0.5 if p == 0.5 else 0.0 for (p, _), (_, _, y) in zip(rows, DATA)) / N
            print("raw blocks", raw, "acc", round(hits, 3), "values/token", sum(v for _, v in rows) / (N * L))
    except NotImplementedError as exc:
        print("unimplemented:", exc)
main()

Runs locally in your browser.

Success criteria

The sweep prints accuracy and values per token for raw = 0, 1, 2 and exits 0.
raw = 1 matches the full-cache accuracy while keeping at most half the values per token.
The write-up names the knee and gives the ambiguity argument for why the second raw block adds nothing.

Stretch goals

Round summaries to 4-bit levels and measure accuracy lost per half byte saved.
Add a second extreme pair so the task genuinely needs two raw blocks and the knee moves.
Make the cache adaptive: keep a block raw only when the summary margin is ambiguous.

Problems

5Keep going

previousDeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence

Where this sits

What it is

1·Theory from first principles

The KV cache is the bill

Asymmetric compute: a causal encoder and decoder

CSA2: layer modes and a hierarchical indexer

SWA bounded replay: do not store what you can rebuild

Multimodal from the start, plus memory and drafting

Reasoning effort as a serving dial

2·Inside the paper

Problem the report sets out to solve

Key idea to look for while reading

Evidence the report musters

Limits to keep in view

Practitioner takeaway

3·Implementation check

4·Build it yourself

Milestones

Success criteria

Stretch goals

Problems

5·Keep going

Problems

Articles

1Theory from first principles

2Inside the paper

3Implementation check

4Build it yourself

5Keep going