We benchmarked every small model we could find on five tasks: rewrite this properly, extract this JSON, summarize this paragraph, classify this ticket, draft this reply.
Phi-4 (3.8B): the best small reasoning model. Outscores models 10x its size on math and logic.
Gemma 3n (2B-4B): the on-device champion. Runs at usable speeds on a phone.
Qwen3-4B: the multilingual workhorse. Handles Chinese, Japanese, Korean natively.
SmolLM3 (3B): the honest little model. Follows instructions literally, rarely goes rogue.
Llama 3.2 3B: the safe default. Everybody knows how to fine-tune it; the ecosystem is enormous.
Mistral 7B v0.3: the OG small model. Still standing, still solid.
Where they shine: bounded, well-defined tasks with explicit output formats. Where they don't: open-ended reasoning, multi-hop analysis, nuanced judgment. Use a small model the way you'd use a calculator, not a consultant.