Synthetic Data for Fine-Tuning: Generate, Filter and Avoid Model Collapse

What you need to know

Synthetic data has become one of the most powerful and most misused tools in the fine-tuning toolkit. Used well, it lets a small team in Bengaluru or Bristol bootstrap a training set for a niche domain in an afternoon, widening coverage around a handful of real examples and unlocking a model that would otherwise need months of human annotation. Used carelessly, it does something subtler and more dangerous: it produces a dataset that looks plausible, passes a casual eyeball check, and quietly drags your model toward the bland, low-variance distribution that recent research calls model collapse. The gap between those two outcomes is not luck. It is a pipeline.

This guide builds that pipeline end to end. It covers when synthetic data is genuinely the right move and when you should be collecting real data instead; the generation methods that matter — Self-Instruct and Evol-Instruct style instruction generation from a strong teacher, and the generation of preference pairs for DPO and ORPO using a judge model; the quality-filtering funnel that is the heart of any serious effort, taking a large raw set and reducing it to a smaller, high-quality one; and the collapse-avoidance rules you must obey or your model degrades irreversibly. It closes with the open-source tooling — Distilabel and Argilla — and a licensing caution that is a real commercial risk, not a footnote.

The throughline is honesty about distribution. Real data carries the long tail — the rare phrasings, the awkward edge cases, the genuine diversity of how people actually write and ask. Synthetic generation, left unchecked, regresses toward the mean of whatever produced it. Everything in this playbook exists to keep that tail alive: anchoring to real data, mixing multiple teachers, scoring for difficulty and diversity, and capping how far synthetic data is allowed to dominate. Get that right and synthetic data is a multiplier. Get it wrong and it is a slow poison.

When synthetic data is the right move

The first decision is not how to generate, but whether to generate at all. Synthetic data earns its place in three clear situations, and is the wrong call in a fourth that teams reach for far too readily.

The strongest case is scarce labelled data. If you are fine-tuning a model to extract clauses from Indian rental agreements or to triage NHS appointment requests, and you have two hundred hand-labelled examples rather than the twenty thousand you would like, synthetic generation can expand that seed into a usable training set by paraphrasing, varying and recombining the patterns the seed demonstrates. The second case is domain or format adaptation: a public instruction dataset will not teach a model your house JSON schema, your brand voice or your regulatory phrasing, and generating examples in exactly that shape closes the gap cheaply. The third is bootstrapping preference pairs — before any real users have rated your model's answers, you cannot run DPO or ORPO on real feedback, so a judge model that ranks candidate responses gives you a starting preference dataset to align against.

The case against synthetic data is just as important. When you have access to plentiful, representative real data — production logs, labelled support tickets, expert annotations, real user transcripts — collect and clean that first. Real data carries the diversity and the rare events that synthetic generation tends to flatten, and no amount of clever prompting recovers a tail that was never sampled. The deeper question of whether you should be fine-tuning at all, rather than reaching for retrieval or a better prompt, is worth settling before you build any pipeline; our decision ladder for whether to fine-tune walks through that choice, and if the answer is yes, the eval-driven LoRA and QLoRA recipe is where the synthetic set you build here gets put to work.

Generation methods that matter

Once you have decided to generate, the question is how. Three families of method cover the great majority of supervised and preference fine-tuning work, and they compose rather than compete.

Self-Instruct and Evol-Instruct

The original Self-Instruct approach starts from a small pool of human-written seed instructions and few-shot prompts a strong teacher model to generate new instructions, then generates the inputs and outputs for each, filtering invalid or near-duplicate items before use. It is the canonical way to turn a handful of seeds into thousands of instruction-response pairs. Its known weakness is diversity: few-shot prompting tends to produce instructions that resemble the demonstrations, so coverage narrows unless you intervene.

Evol-Instruct, introduced with WizardLM, addresses exactly that. Rather than generating flat variations, it iteratively rewrites seed instructions into progressively more complex ones, deepening the reasoning required, adding constraints, or broadening the concepts involved. The result is a dataset with a genuine spread of difficulty rather than a cluster of easy, similar tasks — which matters, because a model learns more from instructions that stretch it than from a thousand restatements of the same easy ask. In practice many teams combine the two: Self-Instruct to widen the breadth of topics, Evol-Instruct to deepen the complexity within each.

The single biggest lever on generation quality is seed and persona diversity. If every instruction is generated from the same three seeds and the same neutral persona, the output clusters. Vary the seeds across your real domains, and prompt the teacher to adopt different personas — a cautious compliance officer, a hurried support agent, a curious student — so the generated instructions span the range of real users. Widening coverage at the seed stage is far cheaper than trying to recover diversity through filtering later.

Generating preference pairs for DPO and ORPO

Preference fine-tuning needs pairs: for a given prompt, a chosen response and a rejected one. When you have no real human ratings yet, you synthesise them. The standard recipe samples several candidate responses for each prompt — ideally from more than one model, to widen the spread — then uses a separate judge model to score them across dimensions such as helpfulness, correctness and instruction-following. The highest-scoring response becomes the chosen example and a lower-scoring one the rejected, forming a preference pair. This is the pattern behind widely used preference datasets such as UltraFeedback, which samples completions from many models per prompt and annotates them with an LLM judge. Once you have the pairs, the alignment method itself — DPO, ORPO or a margin variant — is a separate decision covered in our guide to aligning an LLM with DPO and ORPO.

The quality-filtering funnel

This is the heart of the piece. A raw generation is never your training set — it is the input to a funnel that removes a different failure mode at each stage and emerges far smaller and far better. The cardinal rule of ordering is cheap filters first, expensive scoring last: never pay an LLM judge to evaluate a line you could have dropped with a deduplication hash or a length check. The table below sets out a sensible default funnel and what each stage is for.

Two stages deserve a closer word. Semantic deduplication goes beyond exact matching by embedding each example and dropping any whose nearest neighbour exceeds a similarity threshold, because a generator that paraphrases the same idea forty ways produces forty rows that teach one thing. IFD scoring — Instruction-Following Difficulty, introduced in the Cherry LLM work — compares the model's loss in producing a response with and without the instruction; a useful example is one where the instruction genuinely helps, and scoring lets you keep the examples that carry real teaching signal while discarding the trivial and the unteachable. Notably, the IFD signal is consistent enough across model sizes that a small model can score data for a large one, which makes this stage far cheaper than it sounds.

From a large raw set to a small high-quality one

The funnel's whole purpose is reduction, and the numbers are dramatic. The table below is an illustrative pass starting from a hundred thousand raw generations — the exact figures depend entirely on your generator and thresholds, but the shape is representative of what disciplined filtering produces.

All figures are illustrative and chosen to show the shape of the funnel, not measured results; your own retention depends on the teacher model, the thresholds you set and how noisy the raw generation is. The headline point holds across setups: a serious funnel keeps a small fraction of what it ingests, and the surviving fraction is what makes the difference between a model that improves and one that collapses.

Model collapse: the failure you must design against

Model collapse is the degenerative process that occurs when models are trained on data produced by other models, recursively, across generations. Shumailov and colleagues demonstrated in Nature in 2024 that when each new model is trained on the previous model's output and the original real data is replaced rather than retained, the models degrade: the tails of the distribution thin out first, rare events disappear, variance shrinks, and successive generations converge toward a narrow, homogeneous distribution that no longer resembles reality. The damage is cumulative and, once the tail is gone, effectively irreversible within that lineage — you cannot generate back the diversity you discarded.

The mechanism matters because it tells you what to do about it. Collapse is driven by recursive substitution and lost diversity, so the mitigations all push in the opposite direction: keep real data in the mix, widen the synthetic distribution, and check that you have actually done so. The most reassuring finding from the follow-up literature is that collapse is not inevitable — research through 2024 and 2025 shows that when you accumulate real data alongside synthetic rather than replacing it, models stay stable across generations, and some theoretical work even derives an optimal weighting on real data to best resist collapse. The table and callout below distil the practical rules.

The code: generate, then filter

Two runnable snippets make the pipeline concrete. The first generates instruction-response pairs from a teacher model in the spirit of a Distilabel generation step; the second is the filtering core — semantic deduplication followed by an LLM-judge threshold. Code stays in US English, as is conventional, and both are deliberately simple so the logic is clear rather than clever.

Generation: instruction-response pairs from a teacher

import json
from openai import OpenAI

client = OpenAI()   # or an Anthropic / open-weight client behind the same shape

# Seed instructions drawn from REAL examples in your domain.
# Variety in the seeds is the cheapest path to a diverse generated set.
seeds = [
    "Extract the notice period from this Indian rental agreement clause.",
    "Summarize this NHS appointment request into a structured triage note.",
    "Rewrite this support reply in our brand's plain, reassuring voice.",
]

# Personas widen coverage so the output doesn't cluster on one register.
personas = ["a cautious compliance officer", "a hurried support agent", "a curious student"]

def generate_pair(seed: str, persona: str) -> dict:
    prompt = (
        f"You are {persona}. Write ONE new instruction in the same spirit as the "
        f"example below, then a high-quality response to your own instruction. "
        f"Return strict JSON with keys 'instruction' and 'response'.\n\n"
        f"Example instruction: {seed}"
    )
    resp = client.chat.completions.create(
        model="gpt-strong-teacher",         # a strong teacher tier
        messages=[{"role": "user", "content": prompt}],
        temperature=0.9,                     # higher temp = more diverse generations
        response_format={"type": "json_object"},
    )
    return json.loads(resp.choices[0].message.content)

raw = []
for seed in seeds:
    for persona in personas:
        for _ in range(50):                  # sample many per (seed, persona) cell
            try:
                raw.append(generate_pair(seed, persona))
            except (json.JSONDecodeError, KeyError):
                continue                     # drop malformed generations early

print(f"generated {len(raw)} raw pairs")
# 'raw' now feeds the filtering funnel below — it is NOT a training set yet.

Filtering: semantic dedup plus a judge-score threshold

import numpy as np
from openai import OpenAI

client = OpenAI()

def embed(texts: list[str]) -> np.ndarray:
    out = client.embeddings.create(model="text-embedding-3-small", input=texts)
    vecs = np.array([d.embedding for d in out.data])
    # L2-normalize so dot product == cosine similarity
    return vecs / np.linalg.norm(vecs, axis=1, keepdims=True)

def semantic_dedup(pairs: list[dict], threshold: float = 0.92) -> list[dict]:
    """Greedy near-duplicate removal on instruction embeddings."""
    vecs = embed([p["instruction"] for p in pairs])
    keep, kept_vecs = [], []
    for i, p in enumerate(pairs):
        v = vecs[i]
        if kept_vecs and max(float(v @ kv) for kv in kept_vecs) >= threshold:
            continue                          # too close to something we already kept
        keep.append(p)
        kept_vecs.append(v)
    return keep

def judge_score(pair: dict) -> int:
    """LLM-as-a-judge: score 1-10 on correctness, helpfulness, adherence."""
    rubric = (
        "Score the response from 1 (poor) to 10 (excellent) on correctness, "
        "helpfulness and how well it follows the instruction. "
        "Reply with ONLY the integer."
    )
    resp = client.chat.completions.create(
        model="gpt-judge",                    # a capable, separate judge model
        messages=[
            {"role": "system", "content": rubric},
            {"role": "user", "content":
                f"INSTRUCTION:\n{pair['instruction']}\n\nRESPONSE:\n{pair['response']}"},
        ],
        temperature=0,                        # deterministic scoring
    )
    try:
        return int(resp.choices[0].message.content.strip())
    except ValueError:
        return 0                              # unparseable score = reject

# Funnel: cheap dedup first, expensive judge last, threshold the score.
deduped = semantic_dedup(raw, threshold=0.92)
clean = [p for p in deduped if judge_score(p) >= 8]

print(f"raw={len(raw)} deduped={len(deduped)} final={len(clean)}")
# 'clean' is the high-quality set. Mix it with REAL data before training.

Three things in those snippets are load-bearing. In generation, the temperature=0.9 and the persona loop exist to widen the distribution at the source, which is far cheaper than recovering diversity later. In filtering, the semantic_dedup runs before the judge so you never pay to score a near-duplicate, and the embeddings are L2-normalised so a dot product is a clean cosine similarity. And the judge runs at temperature=0 so scores are reproducible — a judge that wanders run to run gives you a moving threshold. The closing comment is the rule that ties the whole article together: clean is mixed with real data before training, never used alone.

Tooling and the three lines of defence

You do not have to build all of this from raw API calls. As of June 2026, the most common open-source starting point is Distilabel, a framework from the Argilla team for building synthetic-data and AI-feedback pipelines as a directed acyclic graph of steps — generation, instruction evolution, LLM-as-a-judge scoring — that chain together so the output of one step feeds the next. It ships tasks that map directly onto the methods above, which means a Self-Instruct or preference-pair pipeline is configuration rather than bespoke code. Its companion, Argilla, is the human-in-the-loop curation layer: you load a sample of your filtered set into Argilla and people accept, reject or correct examples through a review interface.

The right way to think about these tools is as three lines of defence, applied in order of cost and breadth. The first line is rule-based filtering — dedup, length, format, language — which is cheap, deterministic and catches the bulk of obvious junk. The second is the LLM judge, which catches the plausible-but-wrong examples that rules cannot see. The third, and the point of Argilla, is human sampling review: you do not inspect every example, you inspect a representative sample to catch the systematic failures the automated stages let through — a subtle bias in the generator, a recurring factual error, a tone that drifts from your brand. Sampling review is how you find the failure mode you did not anticipate, and it is the cheapest insurance against shipping a flawed dataset at scale. If your fine-tuning is really a distillation from a larger teacher into a smaller student, the same funnel applies; our guide to teacher-student distillation in production covers that path, and for the reinforcement route, the GRPO recipe for small models shows where synthetic prompts feed RL rather than supervised tuning.

The licensing trap nobody warns you about

There is a commercial risk that sits underneath all of this and that engineers routinely overlook in the rush to ship: using a commercial model's outputs to train a competing model may violate that provider's terms of use. Several major providers include explicit anti-competitive clauses prohibiting the use of their service or outputs to develop a model that competes with theirs, and these clauses have been at the centre of high-profile disputes through 2025 and 2026. This is not a theoretical concern. If your synthetic-data pipeline points at a commercial API and the resulting model competes with that provider, you may be in breach of a contract you agreed to when you signed up.

Conclusion and next steps

Synthetic data is neither a miracle nor a trap — it is a tool that rewards discipline and punishes shortcuts. The disciplined version is a pipeline: decide honestly whether you need synthetic data or just better real data; generate from strong teachers with diverse seeds and personas, using Self-Instruct and Evol-Instruct for instructions and a judge model for preference pairs; push everything through a filtering funnel that removes duplicates, malformed rows, wrong-language lines, trivial examples and low-quality answers, in that cost order; and above all, anchor every training run to real data, mix multiple teachers, check the diversity of what you generated, and cap how far synthetic data is allowed to dominate, so you never collapse the model you are trying to improve.

Start small and measurable. Take one fine-tuning task where labelled data is scarce, build a few hundred real seeds, generate against them, run the funnel, mix the survivors with the real anchor, and compare evals against a real-data-only baseline. If the synthetic-augmented model wins on your held-out set, you have a repeatable pipeline and a result worth showing. If you are the engineer who built a synthetic-data pipeline that improved a model without collapsing it — and who can explain the funnel and the collapse mitigations — that is exactly the shipped, measurable work the people hiring in AI want to see, and a Verified Builder profile on AI Tech Connect is where you put it in front of them.

Sources

• Shumailov et al. — AI models collapse when trained on recursively generated data (Nature, 2024)
• Wang et al. — Self-Instruct: Aligning Language Models with Self-Generated Instructions (arXiv 2212.10560)
• Xu et al. — WizardLM / Evol-Instruct: Empowering LLMs to Follow Complex Instructions (arXiv 2304.12244)
• Li et al. — From Quantity to Quality: IFD-based Self-Guided Data Selection (arXiv 2308.12032, Cherry LLM)
• Gerstgrasser et al. — Is Model Collapse Inevitable? Accumulating Real and Synthetic Data (arXiv 2404.01413)
• Distilabel — synthetic-data and AI-feedback pipeline framework (Argilla)
• UltraFeedback — LLM-judge-annotated preference dataset
• OpenAI — Terms of Use (anti-competitive use clause)