Porting a Prompt Suite Across Vendors: Claude to GPT to Gemini Without Regressions

Why a prompt that works does not travel

A prompt that has been in production for a year is not a piece of text. It is a negotiated settlement between what you wanted and how one specific model behaves: every clause in it exists because somebody observed a failure and added a sentence to stop it. Move that prompt to a different model and roughly half those clauses now solve a problem the new model never had, while a set of problems you never encountered are entirely unaddressed.

Put more precisely: a production prompt encodes an implicit contract with one model's instruction-following calibration. How strictly it obeys a negative instruction. How verbose it is when you say nothing about length. How much weight it gives a constraint buried on line 40 of a long system prompt versus the same constraint restated beside the user's message. What it does when the schema you supplied and the prose you wrote disagree. None of that is written down in your prompt, and all of it is a dependency you never pinned and never had a test for.

Hence the shape of the naive migration. Someone swaps the model identifier, tries five inputs they remember, sees five sensible answers and ships it. Three weeks later the support queue holds a class of complaint nobody can reproduce, the JSON parse-failure rate has gone from 0.1% to 3%, and a downstream service is silently retrying because a tool call now arrives without an optional field the old model always populated.

There are three good reasons to do this work. Cost: traffic has grown to the point where a per-token difference is worth an engineering month. Capability: you need something the incumbent does not do well — a context length, a modality, a latency profile, a reasoning behaviour. Resilience and residency: you want a second provider warm for outages and rate-limit ceilings, or you need inference to happen in a particular jurisdiction, which for a product serving both India and the UK is a routing question before it is a model question. There is one bad reason, and it is the most common: a comparison table on a launch post, measured on tasks that are not yours with prompts that are not yours.

The reframe that makes this tractable is to stop thinking of it as prompt editing and start thinking of it as a database migration. You would not point production at a new database because it benchmarked faster. You would write the migration, run it against a copy, diff the results, shadow the traffic, cut over a slice, and keep the old one available. That is exactly the shape of this job.

Four classes of incompatibility

Cross-vendor breakage falls into four classes, and they are wildly unequal in how much trouble they cause relative to how much attention they get. The prose gets almost all the attention. The tool contracts cause almost all the incidents.

On message and role structure, the question is where your instruction lives in the request and how much authority that position confers. Some APIs take a system instruction as a dedicated top-level parameter; some accept it as a message with a distinguished role; some expose it under a different name again. The lazy port glues the system text onto the front of the first user message, which works everywhere and quietly discards whatever weighting the dedicated field carried. It also interacts with a second variable: how well each model holds a constraint stated once at the top of a 3,000-token instruction across a twelve-turn conversation — the failure that never appears in a single-turn test. The structure described in the guide to system prompt design for production agents makes this mapping mechanical rather than interpretive.

Tool contracts are where the real bugs live. The parameter schemas are usually close to portable because JSON Schema sits underneath the major dialects, but the envelope differs, the supported subset differs, and the semantics differ in ways that do not surface as errors. Whether the model can emit several tool calls in one turn is a common one — a suite built against a model that emitted one call per turn will happily accept a parallel-calling model right up until two calls mutate the same record. How a model signals that no tool is required is another: one returns plain text, another an empty call list, a third a call to whatever tool looks vaguely relevant.

On structured output, the distinction that matters is enforcement level, and most teams do not know which one they were relying on. Asking politely for JSON in the prompt is one thing; supplying a schema the decoder actually constrains generation against is another; a dedicated structured-output mode with strict guarantees is a third. A prompt tuned under strict enforcement carries almost no defensive scaffolding because it never needed any, and porting it to a configuration where the schema is advisory degrades it into a "usually valid JSON" prompt with a small, steady, extremely annoying error rate. The guides to structured-output prompting and reliable JSON with constrained decoding cover the mechanics.

On reasoning and verbosity, half your prompt may exist to suppress a default. A prompt that has spent a year learning to say "answer in at most three sentences, do not restate the question, do not add caveats" is calibrated against a chatty model; give those instructions to a terse one and the answers become uselessly short. The reverse is worse: no length guidance at all, ported to a model that defaults to a long structured answer with headings, blows through your UI, your token budget and possibly your timeout. Where thinking budgets are exposed as a parameter, they are a further axis to re-tune rather than copy — see the guide to prompting reasoning models and setting thinking budgets.

Do not start with the prompts. Start with the golden set

The strongest predictor of whether a migration goes well is whether the team built an evaluation set before they started rewriting. Teams that rewrite first spend the project arguing about taste. Teams that build the set first spend it reading numbers.

A golden set here means, per prompt, 100 to 300 real production inputs with either an accepted output or a rubric describing an acceptable one. Real inputs, not invented ones — what your users actually send is stranger, longer and more badly formed than anything you would write by hand, and the strangeness is where models differ. Below about a hundred cases you cannot separate a genuine regression from noise on a category that appears twice; much above three hundred and the run gets slow enough that people stop running it, which is worse than a smaller set used on every change.

Composition matters more than size, and the most valuable thing you can do is stratify to include the tail. Sample by input length so the longest decile is represented. Include malformed and adversarial inputs — the truncated paste, the customer who typed their entire ticket in the subject line, the injection attempt. Include the multilingual tail honestly: a product serving both India and the UK fields English alongside Hindi, Tamil, Bengali, Marathi and plenty of code-mixed text, and cross-vendor performance on Indic scripts is uneven in ways an English-only sample never reveals. And name your high-value flows explicitly, so they can be reported separately.

Then freeze it and version it. The set lives in your repository next to the prompt specs, changes only through a reviewed commit, and carries a version identifier that every evaluation run records; a golden set that quietly acquires cases mid-migration tells a story you cannot audit afterwards. Where outputs are open-ended enough that exact matching is meaningless, a rubric-driven judge is the workable answer — the guide to LLM-as-a-judge rubrics, bias and calibration covers how to keep it honest, and the judge should run on a model that is not one of the candidates.

Finally, the step teams skip: score the incumbent first. Run the entire golden set against the model you are already using, on the prompts you are already shipping, and record the result as the baseline with its date and version. Without that number, "the new model is worse" is an opinion — and, more painfully, you will usually discover that the system you already ship scores well short of the 100% everyone quietly assumed it was getting. If you are already running evals in CI for prompt and agent regression testing, the migration becomes a new column in an existing report rather than a new programme of work.

The migration harness

With a golden set in hand, the tooling is worth building properly, because you will use it on every model change for as long as the product lives. It has two halves: a way to author a prompt once and compile it for each vendor, and a runner that scores every case against every candidate and reports the differences per case.

The first half is a provider-abstraction layer. The point is not elegance; it is that copy-pasting a prompt into three vendor-shaped request builders guarantees the three copies diverge. Author the prompt spec once — system text, tool schemas, output schema, generation parameters — and let an adapter render it into each vendor's request shape.

# prompt_spec.py — author once, compile per vendor.
# Request shapes below reflect the public APIs as of August 2026;
# re-check each vendor's current reference before you rely on them.

from dataclasses import dataclass, field
from typing import Any


@dataclass(frozen=True)
class PromptSpec:
    key: str                              # e.g. "ticket_triage"
    version: str                          # bump on every edit
    system: str                           # the durable instruction
    tools: list[dict[str, Any]] = field(default_factory=list)
    output_schema: dict[str, Any] | None = None
    max_tokens: int = 1024
    temperature: float = 0.0


class AnthropicAdapter:
    name = "anthropic"

    def render(self, spec: PromptSpec, user_text: str) -> dict[str, Any]:
        req: dict[str, Any] = {
            "model": self.model,
            "system": spec.system,          # dedicated top-level field
            "messages": [{"role": "user", "content": user_text}],
            "max_tokens": spec.max_tokens,
            "temperature": spec.temperature,
        }
        if spec.tools:
            req["tools"] = [
                {
                    "name": t["name"],
                    "description": t["description"],
                    "input_schema": t["parameters"],
                }
                for t in spec.tools
            ]
        return req

    def __init__(self, model: str):
        self.model = model


class OpenAIAdapter:
    name = "openai"

    def __init__(self, model: str):
        self.model = model

    def render(self, spec: PromptSpec, user_text: str) -> dict[str, Any]:
        req: dict[str, Any] = {
            "model": self.model,
            "messages": [                    # system travels as a message
                {"role": "system", "content": spec.system},
                {"role": "user", "content": user_text},
            ],
            "max_completion_tokens": spec.max_tokens,  # renamed from max_tokens
            "temperature": spec.temperature,
        }
        if spec.tools:
            req["tools"] = [
                {"type": "function", "function": t} for t in spec.tools
            ]
        if spec.output_schema:
            req["response_format"] = {
                "type": "json_schema",
                "json_schema": {
                    "name": spec.key,
                    "schema": spec.output_schema,
                    "strict": True,
                },
            }
        return req


class GeminiAdapter:
    name = "gemini"

    def __init__(self, model: str):
        self.model = model

    def render(self, spec: PromptSpec, user_text: str) -> dict[str, Any]:
        req: dict[str, Any] = {
            # self.model goes in the URL path, not the request body
            "systemInstruction": {"parts": [{"text": spec.system}]},
            "contents": [
                {"role": "user", "parts": [{"text": user_text}]}
            ],
            "generationConfig": {
                "temperature": spec.temperature,
                "maxOutputTokens": spec.max_tokens,
            },
        }
        if spec.tools:
            req["tools"] = [{"functionDeclarations": spec.tools}]
        if spec.output_schema:
            req["generationConfig"]["responseMimeType"] = "application/json"
            req["generationConfig"]["responseSchema"] = spec.output_schema
        return req

An LLM gateway gives you a chunk of this for free — LiteLLM, OpenRouter and Portkey all normalise authentication, retries, streaming and usage accounting behind one client, and the comparison in the guide to LLM gateways is worth reading before you write your own adapter layer. But be clear about what you are buying. A gateway normalises the transport; it does not normalise the behaviour, and the behaviour is what breaks. One client forwarding one request shape to three providers still leaves you with three models weighting your instruction differently, three default verbosities and three tool-calling temperaments. It makes the migration cheap to run, not safe to ship.

The second half is the runner. Its one non-negotiable feature is that it reports per-case regressions and not merely aggregates.

# harness.py — run the golden set against every candidate,
# then report per-case regressions, not just the mean.

import json, statistics
from dataclasses import dataclass


@dataclass
class Case:
    case_id: str
    user_text: str
    expected: dict          # accepted output, or rubric for a judge
    tier: str = "standard"  # "critical" for named high-value flows


def run_suite(spec, cases, adapters, invoke, score) -> dict:
    """invoke(adapter, request) -> raw response; score(case, response) -> 0..1"""
    results: dict[str, dict[str, float]] = {}
    for adapter in adapters:
        per_case = {}
        for case in cases:
            request = adapter.render(spec, case.user_text)
            try:
                response = invoke(adapter, request)
                per_case[case.case_id] = score(case, response)
            except Exception as exc:            # an error is a zero, not a gap
                per_case[case.case_id] = 0.0
                print(f"[{adapter.name}] {case.case_id} failed: {exc}")
        results[adapter.name] = per_case
    return results


def regression_report(spec, results, cases, baseline, candidate, tolerance=0.05):
    by_id = {c.case_id: c for c in cases}
    base, cand = results[baseline], results[candidate]

    regressions = []
    for case_id, base_score in base.items():
        delta = cand[case_id] - base_score
        critical = by_id[case_id].tier == "critical"
        # Critical flows get zero tolerance; everything else gets a band.
        if delta < (0.0 if critical else -tolerance):
            regressions.append(
                {"case_id": case_id, "tier": by_id[case_id].tier,
                 "baseline": base_score, "candidate": cand[case_id],
                 "delta": round(delta, 3)}
            )

    report = {
        "spec": f"{spec.key}@{spec.version}",
        "baseline_mean": round(statistics.mean(base.values()), 4),
        "candidate_mean": round(statistics.mean(cand.values()), 4),
        "regressed_cases": len(regressions),
        "critical_regressions": sum(
            1 for r in regressions if r["tier"] == "critical"
        ),
        "detail": sorted(regressions, key=lambda r: r["delta"]),
    }
    # Per-case first; the mean is only a floor. No critical flow may move at
    # all, no ordinary case may fall past the tolerance band, and the
    # aggregate may not drop below the incumbent's baseline.
    report["ship"] = (
        report["critical_regressions"] == 0
        and report["regressed_cases"] == 0
        and report["candidate_mean"] >= report["baseline_mean"]
    )
    print(json.dumps(report, indent=2))
    return report

The reason the gate is written that way is worth stating plainly: a migration that improves the mean by two points while breaking four of your ten largest customers' flows is a failed migration, and every summary statistic you own will report it as a success. Aggregate scores are for tracking; per-case deltas are for deciding.

Rewriting, in the right order

Only now do you touch the prompts, and the order of operations is not the intuitive one. Most people start with the prose, because it is the part they can read. Start at the other end.

First, port the tool schemas and output contracts. This work is mechanical, it is where the silent failures live, and it is testable without any judgement at all. Write a fixture test per tool that asserts on the emitted call — name, arguments, types — rather than on the surrounding prose, and run those before you evaluate anything else. Decide what happens when several calls arrive in one turn, confirm what a no-tool-needed turn looks like on the new provider, and add validation and repair at the boundary if you are moving down a rung of output enforcement.

Second, port the generation parameters. Temperature, token limits and any thinking controls are not comparable across vendors and should be re-derived rather than copied. A token cap tuned to a terse model will truncate a verbose one mid-sentence, which produces invalid JSON rather than a short answer.

Third, and last, rewrite the prose — by symptom, never by taste. You do not change a sentence because you think it reads better; you change it because the harness showed you a specific failure, and you can name which failure the change is meant to fix.

Every row in that table is a hypothesis, and the harness is what tests it — a rewrite that fixes the symptom on three inputs you looked at and regresses eleven you did not is an ordinary outcome. And after several rounds by hand, prompt optimisation is better treated as a search problem than a craft one; the guide to programmatic prompt optimisation with DSPy is the next step, because a golden set and a scoring function are already everything such an optimiser needs.

Keep every prompt version under the same review discipline as code, with the spec version recorded in each evaluation run so a score can always be traced to the exact text that produced it. The workflow in the guide to prompt management, versioning and the staging eval loop is the substrate this exercise runs on; a migration without it degenerates into a folder of files named v2_final_actually.txt.

A second provider is a resilience feature, not just a migration

Once the harness exists and both adapters pass, there is a decision that is easy to make badly: whether to decommission the old provider. The instinct is to pick a winner and delete the loser, because two of anything is more to maintain. Sometimes that is right. Often it is not.

Keeping a second adapter warm buys three things. Provider outages happen, they are rarely correlated across vendors unless the vendors share upstream infrastructure, and an incident that takes your product down for four hours because one upstream had a bad afternoon is avoidable when a tested alternative is one configuration flag away. Rate-limit ceilings are the second: a traffic spike — a campaign, a press mention, a festival-season surge for an Indian consumer product — can push you into throttling on one provider while capacity sits unused on another, and the failover patterns in the guide to building a resilient LLM gateway with failover and rate-limit handling turn that from an incident into a graph nobody looked at. Third, increasingly the deciding factor, is residency: the same product deployed for Indian and UK customers may face different constraints on where inference may happen, and provider coverage of Mumbai and London regions is not uniform. The routing architecture in the guide to data residency for AI apps under DPDP and GDPR is the same spine that carries a second provider.

State the cost honestly, though, because dual-provider setups are sold to engineering leadership as free and are not. Every prompt now has two golden-set runs in CI rather than one. Every behavioural quirk is maintained in two places, and the two drift unless someone owns keeping them level. Every prompt edit is two rewrites and two reviews. The answer is to scope it: keep dual adapters for the flows that genuinely need failover — checkout, authentication-adjacent journeys, anything with a contractual availability commitment or a residency requirement — and run a single provider everywhere else with a documented, tested degraded mode. "All flows, both providers, always" is a decision most teams regret within two quarters.

Rolling it out

The harness tells you about the traffic you sampled. Production tells you about the traffic you did not. The rollout sequence closes that gap without betting the product on it.

Shadow first. Send live requests to the new provider in parallel with the incumbent, serve only the incumbent's output, and log both. This is the highest-value step, because it exercises the genuine input distribution — including the inputs nobody thought to put in the golden set — at zero user risk. Compare offline, look at the cases where the two providers disagree most, and feed the interesting ones back into the golden set.

Then canary. Route a small slice of real traffic to the new provider with an automatic rollback trigger wired to your guard metrics — parse-failure rate, tool-error rate, p95 latency, cost per request, and whatever quality proxy you trust online. Automatic means automatic: a rollback that needs a human to notice a dashboard at two in the morning is not a rollback. Then ramp in steps, holding at each level long enough to see a full daily cycle, because your traffic mix at 10:00 IST is not your mix at 22:00 BST. The deploy mechanics are in the guide to shadow and canary deploys for LLM model upgrades, and reading the online result properly is its own discipline, set out in the guide to A/B testing LLM features with online experiments.

What you keep afterwards

When the ramp finishes, the temptation is to close the project and move on. Do not: the artefacts you built are worth more than the migration that prompted them, and they are cheap to keep and expensive to rebuild.

You now have a versioned prompt spec decoupled from any single vendor: instruction, tool schemas and output contract authored once and compiled per provider, so the next evaluation is a new adapter rather than a new project. You have a golden set per prompt, stratified, frozen and versioned — the only durable record of what your system is supposed to do, and the item on this list most likely to be quietly abandoned. You have a harness that runs on every model change, and the important word is every: a same-vendor minor version bump is this problem in miniature, more dangerous precisely because nobody schedules a review for it. And you have a documented behaviour delta — which prompts needed rewriting, what symptom each rewrite addressed, and what you decided not to fix.

The harness pays for itself the next time any model changes under you, and one will. Providers deprecate versions, adjust defaults, and ship changes that are improvements on average and regressions for your particular contract. A team with a golden set and a runner treats that as a Tuesday. A team without one treats it as an incident, usually discovered by a customer.

It is also one of the more legible pieces of work an engineer can point at. Plenty of people can say they have used three model providers. Very few can show a migration harness, a stratified golden set and a regression table with a per-case ship gate on it — and that combination demonstrates the judgement hardest to interview for: that the mean is not the metric, that the tail is where the risk lives, and that a change you cannot measure is a change you cannot ship. If you have built one, at a Bengaluru fintech or a Manchester health-tech, it belongs on your Builder profile as a named project rather than a line in a skills list.