Why small models are winning on-device
Four-bit quantization plus better distillation is making 3B models genuinely useful in production.
The headline numbers keep coming from frontier labs, but the deployments that actually ship are small. Three billion parameters, four bits, and a task-specific fine-tune is now good enough for classification, extraction and short-form generation.
Continue reading with a subscription
This piece is part of the subscriber archive. Subscribe to the daily brief for full access — one email, no card, cancel any time.
Already a subscriber?
More in this section
Synthetic data is hitting a wall nobody wants to name
Model collapse is real, but the more immediate problem is boring: attribution and licensing.
🔒The open-weights licence split is becoming permanent
Two families of licence, two incompatible expectations, and a lot of downstream confusion.
🔒The inference cost curve is flattening, and that changes plans
Price per token has stopped halving every few months. Budgets built on that assumption need revisiting.