Every lab is quietly running on the same fumes: human-written internet text is finite, and the good stuff is already scraped. The answer, for both pre-training and post-training, is synthetic data, content generated by models instead of people. It used to be an embarrassment, a sign your data was incestuous. Now it's load-bearing infrastructure. Here's what it actually is, where it works, and where it bites.
Why real data ran out
Estimates of high-quality human text on the public web vary, but the consensus across Epoch AI and others is that we'll exhaust the usable stock within a few years at current training-run sizes. Meanwhile compute keeps growing. That mismatch, a compute surplus bolted onto a data deficit, forced the field's hand. If you can't get more signal from the internet, you generate your own. Llama's later releases, Phi's textbook-style data, DeepSeek's training mixture, all lean heavily on generated corpora. Nobody serious trains frontier models on raw scraped text alone anymore.
There's a licensing angle too. Scraped text comes with lawsuits attached, and provenance is murky. Data you generate yourself is clean-ish: you control the license, the lineage, and the revocation story. For enterprises fine-tuning on proprietary material, synthetic expansion of internal corpora is often the only legally comfortable path to scale.
The legal picture is genuinely murky and moving. Several jurisdictions are still litigating whether training on copyrighted text is fair use or reproduction, and the safest corporate posture is trending toward "generate your own and document the pipeline." Synthetic data doesn't eliminate IP risk (teachers ingest the same web), but it gives you an auditable, revokable lineage that is much easier to defend to a board than "we scraped Common Crawl and hoped."
The main flavors of synthetic data
Not all synthetic data is the same thing wearing a lab coat. Roughly four categories dominate:
- Distilled teacher data: a strong model generates answers or reasoning traces, a smaller model trains on them. This is how most small open models get their capability. Quality is gated by the teacher and by whatever filter you apply.
- Rejection sampling: sample many candidate solutions, keep the verified-correct ones (tests pass, answer checks), discard the rest. The backbone of RLVR-adjacent pipelines and reasoning distillation.
- Self-instruct and persona data: bootstrapping instruction-following from a seed set of human prompts, expanded by a model. Good for breadth, dangerous for homogenization.
- Environment and agent traces: simulated tool use, coding sessions, web browsing logs generated by agents. Growing fast as agentic training scales.
- Synthetic preference pairs: model-generated responses ranked by a rubric or judge, used to train reward models without millions of human labels. Cheap, fast, and quietly responsible for a lot of post-training quality.
Where it works brilliantly
Domains with cheap verifiers are the sweet spot. Math and code can be checked by execution, so correctness filtering is mechanical. Synthetic reasoning traces, filtered for right answers, produce models that punch far above their parameter count, as Phi and Qwen distilled models have shown. Synthetic data also wins for long-tail coverage: rare languages, obscure APIs, niche formats, where human data is thin but a decent model can generate passable examples. And for eval-adjacent uses, generating adversarial or edge-case test items synthetically is now standard practice.
It also shines for privacy. A hospital can't email its patient notes to a labeling vendor, but it can train a model on local synthetic records that preserve statistical shape without carrying identifiable information, subject to the usual caveats and audits. Same trick powers financial KYC simulations, telco logs, and industrial sensor narratives. When real data can't move, synthetic data can commute.
Where it goes wrong
The failure mode everyone worries about is model collapse: training on your own output until the tails of the distribution vanish and the model regresses to a bland mean. Pure, unfiltered self-training over generations does degrade, and the early panic papers were directionally right. In practice, labs mitigate this by anchoring to real data, filtering hard, and never letting synthetic fractions go to 100%. Collapse is a manageable disease, not an inevitability, but it's real if your pipeline is lazy.
Subtler problems show up first: diversity loss (everything sounds like the teacher), style lock-in (synthetic data over-represents confident, verbose, listicle prose), and benchmark contamination (synthetic eval items leaking into training sets, inflating scores that mean nothing). There's also the question of baked-in bias amplification: any skews in the teacher model get re-encoded and re-amplified with no human in the loop to catch them.
A quick reality check on quality bars
Cheap synthetic data is free to generate and worthless. The useful stuff costs real money: teacher inference at scale, verification compute, human audits on flagged samples. A rough mental benchmark for teams: if you haven't spent meaningfully on filtering, you haven't made synthetic data, you've made autocomplete soup. The labs doing this well talk about their filtering pipeline before their generation pipeline, and you should too. Ask any vendor how much of their synthetic mixture survives verification; the ratio tells you everything about their bar for quality.
Also worth saying: synthetic data is an amplifier of whatever process you already have. If your data governance is sloppy, synthetic volume makes the mess bigger. If it's disciplined, a skeleton crew can service a training run that would otherwise take a small army of annotators. The technology doesn't fix your process; it audits it brutally and passes the bill on.
How the good pipelines actually work
The winning recipes share a shape: generate broadly, verify aggressively, mix with real data, and measure constantly. A strong teacher produces candidate generations. Automated checks (unit tests, answer matching, format validators, sometimes a second model as judge) filter the pile. The survivors get mixed with high-quality human data at some ratio, often somewhere between a third and two-thirds synthetic depending on the stage, and the mixture gets validated on held-out real benchmarks. If evals stagnate while perplexity improves, your synthetic data is teaching fluency, not capability. That's the canary.
Note the asymmetry: synthetic data is cheap to generate and expensive to validate. The bottleneck isn't GPU time for generation, it's trust infrastructure for filtering. Teams that treat validation as an afterthought ship confident, fluent, wrong models.
Diversity engineering deserves its own paragraph. Good pipelines don't just sample from the teacher on temperature 0.7 and call it a day; they prompt across personas, seed topics, difficulty tiers, and adversarial framings, then dedupe aggressively with embedding clustering. They also track mixture entropy over training, and they cap repetition of any single synthetic template. The teams that win treat data like agriculture, with crop rotation and soil health, not like strip mining.
A field guide for a small team
If you're a six-person company wondering whether any of this applies to you, the honest answer is partially, yes, with discipline. Start by auditing your existing corpus: how much of it is licensed, how much is internal, how much is scraped and murky. That map is the prerequisite for everything else, because once you know where the gaps are, you know where synthetic data earns its keep. Usually the answer is long-tail coverage: rare formats, niche APIs, internal jargon, multilingual edge cases your human data never captured.
Operationally, the minimum viable pipeline is boring: pick one teacher model you trust, generate candidate examples for the weak slices, filter with whatever check you can automate (schema validation, unit tests, a rubric-scored second pass), mix the survivors with real data at a modest ratio, and evaluate on held-out real data before and after. Keep the mixture ratios, pass rates and failure examples in a shared doc. That doc becomes your provenance story when a customer or auditor asks what the model was trained on, and "here are the verifier pass rates and the audit trail" lands very differently than silence.
One opinion from the trenches: resist the temptation to synthetic-generate your eval set from the same teacher you trained on. That's grading your kid's homework with the teacher's own answer key; everyone scores beautifully and nobody learns anything. Keep evals human-anchored, even if that means fifty painful, hand-written cases. And when someone proposes skipping verification "just for this batch," remember that unverified synthetic data doesn't just waste the slot in the training mix, it actively teaches fluent wrongness. The filter is the product. The generation is just raw material.
What's next
Expect synthetic data to keep eating more of the pre-training budget, and almost all of the post-training budget, since RL itself runs on model-generated rollouts. Expect better provenance tooling, too; regulators and buyers both want to know what a model was trained on, and "we don't know, it generated itself" isn't a satisfying answer. One contrarian note: the biggest labs are quietly paling at how homogeneous the public synthetic supply has become. When everyone's teacher models distill into one another, the data ecosystem converges to a narrow accent. We'll likely see a backlash toward premium human data collected for one purpose, creative writing with real voice, messy human judgment, genuine expertise, precisely because it can't be faked at scale. The open questions are fidelity of simulated environments for agent training and whether any mixture escapes the ceiling set by the teacher. Synthetic is the floor of modern training now. The ceiling still belongs to distinctive human signal, awkward and expensive as it is. For now, the honest summary is this: synthetic data doesn't replace human knowledge. It amplifies and redistributes it, and the amplification works best where verification is cheap and humility about limits is high.
Do the napkin math: one million verified reasoning traces at a few cents of teacher inference each is a very doable budget for a mid-size lab, and it buys a training signal no amount of scraped forum posts contains. The moat in post-training is now less "do you have GPUs" and more "do you have a generator-verifier loop tuned to your domain." That loop is the modern equivalent of a well-organized library, and building it is an engineering discipline, not a secret.
One last thought on collapse research: nearly all the doom scenarios assume naive, unfiltered self-consumption. Real pipelines keep humans and verifiers in the loop precisely to break that cycle. Treat scary collapse headlines the way you treat actuarial tables, as a warning about a failure regime you can engineer around, not a prophecy.
If you remember nothing else from this piece: audit your synthetic fractions, keep real data anchors, and measure on real benchmarks, not on the teacher's own outputs. The models you can trust are the ones whose training lineage you can actually describe out loud.
And budget for ownership: someone on the team has to be the synthetic-data steward, tracking mixture ratios, verifier pass rates and audit findings like a safety officer tracks incidents. The moment nobody owns it, drift wins by default.