Synthetic data is hitting a wall nobody wants to name
Model collapse is real, but the more immediate problem is boring: attribution and licensing.
The technical discussion focuses on distribution drift. The practical blocker is contracts: nobody can say cleanly where a synthetic corpus came from.
Two labs have quietly paused releases over this. Neither will say so on record.
The fix being trialled is provenance metadata attached at generation time — a chain of custody for data.
#datasets#licensing#training
More in this section
🔒aiSubscriber
Why small models are winning on-device
Four-bit quantization plus better distillation is making 3B models genuinely useful in production.
🔒aiSubscriber
The open-weights licence split is becoming permanent
Two families of licence, two incompatible expectations, and a lot of downstream confusion.
🔒aiSubscriber
The inference cost curve is flattening, and that changes plans
Price per token has stopped halving every few months. Budgets built on that assumption need revisiting.