Boa

Linear-Time Context Compression with State-Space Encoding

  • Marc Serra Ortega
  • Andrey Barsky
  • Carlos Boned Riera
  • Tomás Ockier
  • Dimosthenis Karatzas

Computer Vision Center, Universitat Autònoma de Barcelona

Boa is a context compressor for a frozen language model. A Mamba-2 encoder reads the context in one linear-time pass and keeps a short sequence of continuous representations, which the model reads in place of the text.

This figure needs the page to be served over HTTP.

The same question, answered two ways by the same frozen LLM. Above, the LLM reads every context token. Below, Boa reads the context once and hands the LLM a much shorter input, so the answer can start sooner.

What context compression does

A language model normally reads every token of its context, and that work grows with the length of the context. Context compression replaces the text with a shorter sequence of continuous vectors, often called soft tokens, which the unchanged model reads instead. Boa keeps one vector for every r tokens, counting back from the final token, so a context of L tokens becomes ⌈L / r⌉ soft tokens.

This figure needs the page to be served over HTTP.

Change the compression ratio to see how many soft tokens remain. The final token is always kept, because in a causal model it has seen the whole context.

Time and memory

Compression pays off for a single query only if producing the compressed context costs less than reading the full one. Boa’s encoder runs in linear time, so its own cost stays small as the context grows, whereas Transformer-based compressors first make a long pass of their own. Choose a context length to compare Boa with the same LLM reading the full context.

This figure needs the page to be served over HTTP.

Time to answer is the end-to-end latency: compressing the context, reading it and generating the answer. The time bars and the chart use logarithmic scales; the memory bars use a linear one.

How Boa works

Boa has two trained parts, a Mamba-2 encoder and a projection, and one frozen part, the target LLM, which is never modified. Rather than entering at the input, the soft tokens are projected into keys and values in the LLM’s attention cache at every layer.

This figure needs the page to be served over HTTP.

  1. Mamba-2 reads the context once, in linear time.
  2. A temporal stride keeps every r-th hidden state.
  3. Per-layer K and V projections turn them into a key-value cache.
  4. The frozen LLM attends to that cache instead of the text.
  5. Training matches its next-token distribution to the full-context teacher (KL).
The encoder reads the context in one pass, every r-th hidden state is kept, and per-layer key and value projections turn those states into a KV cache for the frozen LLM. Training matches the next-token distribution of this student to that of the same LLM reading the full context.

Results

On the in-domain MRQA question-answering datasets, Boa scores above Activation Beacon and SAC at every compression ratio tested, and loses little as the ratio grows. Out of domain the three methods are much closer, and which one leads depends on the ratio.

This figure needs the page to be served over HTTP.

Normalised F1 on the in-domain MRQA datasets: 0 is the LLM without any context and 100 is the same LLM reading the full context. Hover, focus or tap a point to read its value.

Average exact match and F1

Averages over the datasets in each split, with a frozen Qwen3 decoder. The best compressed score in each column is in bold.

In domain
Out of domain

Limitations

Verbatim span extraction is weaker.
Relation Extraction asks the model to copy exact spans of text. This is the setting in which attention-based Transformers exceed state-space models, and here Activation Beacon leads Boa at every compression ratio.
Long-context question answering trails Activation Beacon.
At 32K tokens and 8× and 16× compression, Boa averages 1.3 to 1.4 F1 below Activation Beacon, and the gap is concentrated in question answering: on QASPER Boa trails by 5.4 to 6.4 F1. On summarisation the two are close: Boa leads on GovReport by 1.9 to 4.0 F1 and trails on BookSum by 0.8 to 1.4. We attribute the gap to a weaker long-context prior, and offer this as a hypothesis rather than a measured effect. The two encoder backbones start from very different pretraining. SAC and Activation Beacon encode the context with Qwen3 itself, which was pretrained on roughly 36 trillion tokens and includes a dedicated long-context stage with sequences of up to 32K tokens. Boa's encoder starts from a Mamba-2 checkpoint pretrained on roughly 300 billion tokens, about a hundred times fewer, in sequences of at most 2K tokens; our own compression pretraining adds 10 billion tokens in sequences of up to 10K. A 32K-token document is therefore well beyond anything the Mamba-2 encoder saw before fine-tuning, while it is within what Qwen3 was trained on, and Activation Beacon also works directly on full-attention windows across the extended context. A state-space encoder pretrained at a comparable scale and context length is the natural next step.

Citation

BibTeX
@article{serraortega2026boa,
  title   = {Linear-Time Context Compression with State-Space Encoding},
  author  = {Serra Ortega, Marc and Barsky, Andrey
             and Boned Riera, Carlos and Ockier, Tom{\'a}s
             and Karatzas, Dimosthenis},
  journal = {arXiv preprint arXiv:2610.33006},
  year    = {2026},
  url     = {https://arxiv.org/abs/2610.33006}
}