Skip to content

Search stories, models, companies…Subscribe

Topic · Concept

Small models

Small models are language models designed to be cheap and fast enough to run at high volume or on local hardware such as laptops and phones. Labs ship them as "mini," "Flash," "Haiku" or "Lite" tiers, and they are often the best choice when a task does not need a frontier model.

24 stories · updated Oct 16

Follow via RSS Get Small models alerts

Key facts

Checked Oct 8

Example on-device model: phi-3-mini, 3.8B parameters, "small enough to be deployed on a phone" (Apr 22, 2024)
Knowledge distillation (canonical paper): Hinton, Vinyals, Dean, "Distilling the Knowledge in a Neural Network," Mar 9, 2015
Example commercial small tier: Claude Haiku 5.5: 1M context; API price "from $0.10 / from $0.50" per MTok
Example commercial small tier: Gemini 3.5 Flash-Lite and 3.1 Flash-Lite listed as stable

Latest on Small models

Open source · ExplainerJanas runs LLMs larger than RAM by streaming experts from SSD: how it works and what it needsJanas, a GPL-licensed engine written in C, runs mixture-of-experts models larger than RAM by streaming experts from SSD and keeping the busiest ones in memory. Its developer reports 24 tokens per second for an 80-billion-parameter model on a 32 GB laptop. Here's how it works and what the numbers do and don't show.Oct 16·6 min read

Models · NewsKrea 3 and EmbeddingGemma 2 speech-to-text: two smaller AI releases worth knowingKrea 3, launched October 15, 2026, is an image model built for editing at scale at $0.03 to $0.06 an image. Separately, a developer turned Google's open EmbeddingGemma 2 into an on-device speech-to-text app. What each is, what's open, and what it costs.Oct 16·6 min read

Open source · NewsNvidia Nemotron 3.5 Super VL and Bolt: open weights under OpenMDW, and what the Opus 5 comparison really showsNvidia released Nemotron 3.5 Super VL, a 120B-parameter (12B active) hybrid Mamba-Transformer model with a 1M-token context, and Nemotron 3.5 Bolt 74B-A9B on October 15, 2026, with a 4 GB Bolt E4B due October 23. The license is permissive; the headline wins over Claude Opus 5 and GPT-6.1 Sol come from customized versions on narrow tasks.Oct 16·6 min read

Open source · NewsFLUX 3 Image Klein: Black Forest Labs' 5.6B open-weight image model, and what 'free under $5M' really coversBlack Forest Labs released FLUX 3 Image Klein on October 15, 2026: a 5.6B-parameter open-weight model for image generation and editing with native 4K output. Its FLUX Kommunity License lets companies under $5 million in revenue use it commercially, but only to generate images, not to distribute or host the model.Oct 16·5 min read

Open source · NewsLiquid AI LFM2.5-24B-A2B: a 24B open-weight MoE with 2B active parameters, its license and what it needs to runLiquid AI released LFM2.5-24B-A2B on October 13, 2026: a 24-billion-parameter mixture-of-experts reasoning model with about 2.45B active parameters and a 131K context. The weights are free to download, but Liquid's license excludes commercial use by companies with $10 million or more in annual revenue.Oct 14·5 min read

Research · AnalysisCan AI models self-replicate on IoT devices? What Anthropic's Frontier Red Team study actually showsAnthropic's Frontier Red Team says small open-weight AI models may be months away from being able to self-replicate on consumer and IoT devices. What it tested, what the 8-of-9 and 8-of-8 results mean, and what the study does not show.Oct 14·8 min read

Models · NewsClaude Haiku 5.5: price, 1M context and how it compares with Haiku 4.5Anthropic released Claude Haiku 5.5 on October 7, 2026. It costs $0.10/$0.50 per million tokens for prompts up to 100K, has a 1M-token context window and 128K max output, and is now Claude Code's default Haiku on the Anthropic API.Oct 13·5 min read

Models · ExplainerDecision models explained: Microsoft-Decision-1, OpenAI's Decisions API and when to use themMicrosoft-Decision-1 and OpenAI's Decisions API don't write text. They score a fixed set of answers and return probabilities your code can act on. How decision models work, what they cost, and where they fit: classification, routing and verification.Oct 13·7 min read

Models · AnalysisDo AI labs really ship 'mini' models first? What the release record since 2024 showsAcross 32 small-vs-flagship pairings from six labs, the small model came first only five times. Here's what really drives the timing, and the price war at the bottom.Oct 9·9 min read

Industry · CompareFrontier APIs vs open models: what they really costHosted open models are cheap, but the frontier labs' small models now list even lower. Self-hosting pays off only at very high, steady volume. Verified prices and the arithmetic.Oct 9·Checked Oct 8

Nothing in this format on this page — try another filter or the next page.