onllm.dev
DeepSeek just dropped something that's generating serious buzz in the local LLM community — and despite the name, it's not just another OCR model. Developers and researchers are dissecting the innovations, and the consensus is clear: this could change how we think about LLM inputs entirely.
What's the Big Deal?
DeepSeek-OCR introduces Contexts Optical Compression (COC), a technique that achieves 10-20x reduction in input tokens by treating text as images. Instead of optimizing the transformer's attention mechanism, they shrink the input sequence length from the start.
The result? A single A100 can process 200,000+ pages per day with around 2,500 tokens/sec throughput.
The Architecture
The system uses a 380M parameter Deep Encoder with three cascaded levels:
- SAM-based Sensor — Handles high-resolution input (4,096 patches) using window attention for local details
- 16x Compressor — A conv network that downsizes 4,096 patches → 256 visual tokens
- CLIP Layer — Global attention for semantic understanding at the compressed scale
The decoder is a DeepSeek3B-MoE model that handles specialized OCR subtasks.
Resolution Modes
- Tiny: 512×512 (64 vision tokens)
- Small: 640×640 (100 vision tokens)
- Base: 1024×1024 (256 vision tokens)
- Large: 1280×1280 (400 vision tokens)
- Gundam: Dynamic resolution for complex docs
Real-World Use Cases
The community is already putting DeepSeek-OCR to work:
📄 Document Processing Pipelines
- Invoices, contracts, and scanned PDFs → structured Markdown/JSON
- Integration with n8n, Airflow, and custom ETL jobs
- Compliance audits and knowledge base ingestion
📊 Complex Data Extraction
- Tables (including cross-page and multi-column layouts)
- Charts and figures with
Parse the figure.prompt - Mathematical formulas and equations
- Handwritten notes (with caveats — still challenging)
🌍 Multilingual Support
- 90+ languages out of the box
- Fine-tuning works well — Unsloth showed 88% improvement on Persian text after just 60 training steps
🔍 RAG & Document Q&A
- Feed compressed visual tokens directly into LLMs
- Potentially eliminates traditional "parsing" for many retrieval use cases
- LlamaIndex is already exploring integration
🏢 Enterprise "AI Memory"
- Load entire company wikis, reports, and emails into compressed visual context
- A cluster of 20 servers (160 A100s) can handle 33 million pages/day
⚠️ Community Tip: Pair DeepSeek-OCR with validation workflows. Like all LLM-based OCR, it can hallucinate — add JSON schema validation and spot
# Ollama (v0.13.0+)
ollama run deepseek-ocr "/path/to/image\n<|grounding|>Convert the document to markdown."Why This Matters
Andrej Karpathy called it more than just a good OCR model — the real innovation is using visual encoding as efficient memory for LLMs. This could be the blueprint for how future models handle long-context processing without the quadratic cost explosion.
On OmniDocBench, the compressed Gundam mode (under 800 tokens) beat competitors using nearly 7,000 tokens.
Links
- GitHub — 20k+ stars, MIT licensed
- Hugging Face
- Paper (arXiv)
Developers are already experimenting with integrating this into ETL pipelines, RAG systems, and document processing workflows. Worth watching.
Build with us at onllm.dev — where local AI comes to life.