The open-weights evaluation gap is narrowing
The best open models are no longer a generation behind on the tasks that matter to most teams.
On reasoning benchmarks the gap is still visible. On extraction, classification and short-form writing, it is close enough to be a procurement question.
The bigger difference is tooling: evaluation harnesses, guardrails and deployment support favour the closed providers.
That is a support gap, not a model gap, and it is cheaper to close.
#open weights#accelerators#models
More in this section
🔒aiSubscriber
Why small models are winning on-device
Four-bit quantization plus better distillation is making 3B models genuinely useful in production.
ai
Synthetic data is hitting a wall nobody wants to name
Model collapse is real, but the more immediate problem is boring: attribution and licensing.
🔒aiSubscriber
The open-weights licence split is becoming permanent
Two families of licence, two incompatible expectations, and a lot of downstream confusion.