DeepSeek OCR - More than OCR

DeepSeek OCR’s headline idea isn’t better document reading—it’s a new way to compress long text into far fewer “vision tokens,” then decode it back with high accuracy. The practical payoff is memory: large language models struggle with very long contexts because text costs roughly one token per word, so pushing from millions toward 10 million+ tokens quickly becomes expensive. DeepSeek’s approach aims to store millions of text tokens as an image-like representation that can fit into a much smaller context window.

The method centers on “context optimal compression,” treating vision as a compression medium for text rather than just a way to interpret pictures. Instead of converting an image to tokens and then treating those tokens as ordinary visual features, the system is trained so that a small number of vision tokens can be decoded back into a much larger amount of text. The transcript cites results such as using 100 vision tokens to recover about 1,000 text tokens with roughly 97% accuracy, and still retaining around 60% accuracy at 20× compression (50 vision tokens for 1,000 text tokens). Even though the paper is framed around OCR, the underlying claim is that this could become a general-purpose memory compression mechanism for long-context AI.

A key engineering piece is a two-stage “deep encoder” designed to avoid the token explosion that typical vision encoders can cause. Stage one uses a SAM model (about 18 million parameters) to focus attention at high resolution—effectively deciding what details matter—then a CNN compresses the image representation by 16×. Stage two feeds the compressed representation into a CLIP-style model to use global attention and produce an efficient set of vision tokens. The system also supports multiple “zoom levels,” outputting different token counts depending on mode: 64 tokens in a tiny mode, 100 in a small mode, 256 in a base mode, up to about 1,800 tokens in a larger mode.

The transcript contrasts this with a more conventional tokenization approach for long documents, which might require around 6,000 text tokens for a given document and heavy compute/memory in the language model. DeepSeek’s encoder can achieve the same task under 800 vision tokens, with reported performance that can be better despite the compression.

Benchmarks reported in the OCR setting suggest accuracy stays above 95% as long as compression remains around 10×. The work is still presented as proof-of-concept for the compression idea: it demonstrates the concept on OCR tasks, but scaling to extreme ratios (for example, replacing hundreds of thousands of vision tokens for millions of text tokens) remains an open question. Still, the implications are clear—rendering older conversation history into image-like token bundles could let systems keep 10–20 million text-token equivalents in context using dramatically fewer tokens, enabling longer in-context learning and retrieval without the usual cost explosion.

Alongside the model, the transcript notes a decoder described as “DeepSeek 3B” with about 570 million active parameters, and points to released code on GitHub and a model on Hugging Face. It also situates the work among other OCR-focused systems such as Nanets OCR2 and Paddle OCR VL, but positions DeepSeek’s contribution as a broader shift in how text can be stored and retrieved inside transformer pipelines—using vision-token compression rather than raw text tokens.