Serverless GPUs Compared: Modal vs RunPod vs Baseten vs Replicate

What you need to know

Serverless GPU platforms occupy the tier between the two options every AI team already understands. Below them sit managed model APIs — you buy tokens from a model somebody else runs and you cannot change. Above them sits self-hosted infrastructure — your own GPU nodes, your own serving stack, your own autoscaler, and a bill that arrives whether anyone used the product or not. The serverless tier splits the difference: you bring your own model and container, the platform brings the GPUs, spins your workload up when a request arrives, bills by the second or minute, and scales it back to zero when traffic goes quiet.

That middle tier wins in three situations. First, spiky or unpredictable traffic — a product that does 40 requests a minute at Indian lunchtime and UK early evening but nearly nothing at 4am, where an always-on GPU spends most of the day idle. Second, custom models — a fine-tune, an unusual pipeline, or open weights configured your way, which rules out the stock API tier entirely. Third, anything batch-shaped: transcription backlogs, nightly embedding jobs, evaluation sweeps, where a five-minute cold start costs nothing and scale-to-zero saves everything. Conversely, steady high-volume traffic that keeps GPUs busy around the clock erodes the serverless premium quickly — at that point the economics swing towards running your own autoscaled serving stack on Kubernetes — and if all you need is tokens from a frontier model, a managed API is usually the right call and none of this article applies.

• Four platforms dominate the conversation in mid-2026: Modal (developer experience), RunPod (price), Baseten (production SLOs and enterprise), Replicate (model marketplace, now joining Cloudflare).
• Cold starts are the defining engineering problem of the category — snapshotting, cache-preloading and warm pools are where the platforms genuinely differ.
• Billing granularity differs more than headline prices: per-second on Modal, RunPod and Replicate; per-minute on Baseten; per-output on many Replicate public models.
• List prices span roughly 2x for the same card. As of July 2026, an 80GB A100 runs from about $2.50/hr (Modal) to $5.04/hr (Replicate).
• The category is consolidating: Cloudflare agreed to acquire Replicate in November 2025, Nvidia backed Baseten's $300M round in January 2026, and Modal became a unicorn in September 2025 before reportedly reaching a $4.65bn valuation by May 2026.

The four platforms, briefly profiled

Modal: the Python-native one

Modal's pitch is that deploying to the cloud should feel like writing ordinary Python: decorate a function, declare its image and GPU in code, and modal deploy turns it into an autoscaling cloud endpoint. That developer experience carried the company to an $87M Series B in September 2025, led by Lux Capital at a $1.1B valuation — and the company has since raised again: our May 2026 coverage reported a $355M Series C at a $4.65bn valuation. Its most distinctive engineering asset is snapshotting: Modal's memory snapshots capture a container's initialised state — imports done, libraries loaded — and restore it directly on later boots, which the documentation says makes initialisation-heavy functions start 3–10x faster; GPU memory snapshots, which additionally capture GPU state, are available in alpha. Billing is per-second across CPU, memory and GPU, with $30/month of free compute credits on the starter plan — enough to evaluate seriously before spending anything.

RunPod: the price-led one

RunPod competes hardest on the number at the bottom of the invoice. As of July 2026 its on-demand pods price an H100 SXM at $2.99/hr and an 80GB A100 at $1.49/hr — among the lowest published rates for that hardware anywhere — while its serverless tier adds autoscaling flex workers billed per second. Its cold-start answer is FlashBoot, a caching layer that pre-positions container images and model weights on host machines: RunPod states that 48% of its serverless cold starts complete in under 200ms. Two further points matter for budgeting: RunPod charges nothing for data ingress or egress, and it operates GPUs across roughly 31 global regions spanning the US, Canada, Europe and Asia-Pacific. The trade-off is that you do more assembly yourself — it is closer to cheap, flexible GPU plumbing than a batteries-included ML platform.

Baseten: the production-grade one

Baseten aims at teams whose inference is already load-bearing revenue infrastructure. Models are packaged with Truss, its open-source framework, and served on dedicated deployments with autoscaling, canarying and observability built in. Investors have priced that positioning aggressively: Baseten raised $300M in January 2026 at a $5B valuation, a round led by IVP and CapitalG with participation from Nvidia. Under the hood, its multi-cloud capacity management layer schedules workloads across 20+ clouds and regions, with region- and provider-locking for residency requirements, and self-hosted and hybrid modes that run the whole stack inside your own cloud. You pay for the polish: per-minute billing at $6.50/hr-equivalent for an H100 and $4.00/hr for an 80GB A100 as of July 2026 — the highest H100 list price of the four.

Replicate: the marketplace one, now joining Cloudflare

Replicate made its name as the place you run someone else's model in one API call — a catalogue of over 50,000 community and official models behind a uniform prediction API, with private deployments for your own weights billed per second of compute. Its 2026 story is ownership: Cloudflare announced an agreement to acquire Replicate on 17 November 2025, terms undisclosed, saying the catalogue will become available to Workers AI users and that Replicate's expertise will power custom-model support on Cloudflare's platform. Replicate's own announcement stresses continuity — existing APIs and workflows keep working, with Cloudflare's network underneath. On price it is the dearest per-second option for private workloads ($5.04/hr-equivalent for an 80GB A100, $5.49/hr for an H100, July 2026), but many public models bill per output or per token instead, which can be cheaper than any per-second arithmetic for occasional use.

Cold starts: the problem that defines the category

Scale-to-zero is the whole economic point of serverless GPUs, and cold starts are its price. When a request arrives and no warm worker exists, the platform must find a free GPU, pull or restore your container, initialise the CUDA runtime, and load model weights into GPU memory before a single token is generated. Do that naively with a multi-gigabyte image and a 14GB checkpoint pulled over the network and the first user waits half a minute or more — which, for an interactive product, is indistinguishable from an outage. Every platform in this comparison is, at heart, a different set of engineering answers to that one problem.

The first answer is snapshotting and cache-preloading. Modal's memory snapshots skip re-executing initialisation by restoring a saved image of the container's memory after imports and model setup — 3–10x faster starts for initialisation-heavy functions, per its documentation — with alpha-stage GPU snapshots extending that to GPU memory state. RunPod's FlashBoot attacks the same problem from the storage side, keeping container images and weights cached on or near host machines so a flex worker doesn't pull 20GB from a registry on every boot; the 48%-under-200ms figure is the visible result, though by implication the other half of cold starts take longer, and worst cases on large images still stretch to tens of seconds. Baseten and Replicate both do weight-caching and image optimisation behind the scenes, with less public instrumentation.

The second answer is paying for warmth. Every platform lets you hold a minimum number of workers warm — RunPod calls them active workers and discounts them relative to flex rates, Baseten and Replicate let you set minimum replica counts on deployments, and Modal supports warm containers via configuration. A single warm A100 worker is roughly $60–120 a day at July 2026 list prices depending on platform, so a warm pool is an SLO expense you should attribute to the endpoints that need it, not a default you apply everywhere. The third answer is discipline you own: slim base images, weights staged on the platform's fast local storage rather than pulled from Hugging Face on boot, and lazy-loading anything not needed for the first request. No platform feature rescues a 30GB container built on carelessness.

Here is what the deploy surface actually looks like on Modal — a minimal, current-API example (verified against modal.com documentation, July 2026) with snapshotting enabled:

# deploy.py -- minimal Modal service (API as of July 2026)
import modal

image = (
    modal.Image.debian_slim()
    .pip_install("fastapi[standard]", "transformers", "torch")
)

app = modal.App("summariser", image=image)

@app.function(
    gpu="A100",
    enable_memory_snapshot=True,  # restore initialised state on later boots
)
@modal.fastapi_endpoint(method="POST")
def generate(payload: dict):
    # Module-level / first-call initialisation is captured by the snapshot,
    # so later cold starts skip straight past model loading.
    return {"summary": run_model(payload["text"])}

# Ship it:
#   modal deploy deploy.py

RunPod's equivalent is a queue-consuming handler rather than a decorated function — you define a worker, package it in your own Docker image, and the platform feeds it jobs:

# handler.py -- RunPod serverless worker
import runpod

MODEL = load_model()  # module scope: runs once per worker, not per request

def handler(event):
    prompt = event["input"]["prompt"]
    return {"output": MODEL.generate(prompt)}

runpod.serverless.start({"handler": handler})

The difference in feel is the difference in philosophy: Modal abstracts the container away until you ask for it; RunPod hands you the container and stays out of your way.

How the billing models compare

Headline hourly rates get the attention, but the billing model — what starts the meter, what granularity it ticks at, and what counts as billable — moves real bills more than the rate card does. Three models are in play. Per-second compute billing (Modal, RunPod, Replicate private deployments) is the purest: the meter runs while your container does. Per-minute billing (Baseten) is coarser, which is irrelevant for long-running deployments and mildly unfavourable for very short, spiky tasks. Per-request or per-output billing (many Replicate public models) transfers all utilisation risk to the platform — you pay the same for an image whether the GPU behind it was busy or idle — and is often the cheapest way to consume a popular open model occasionally.

Watch the edges of the meter, too. Replicate's private-model pricing explicitly bills setup time and idle time while instances are online (fast-booting fine-tunes excepted), RunPod bills a configurable idle window after each request on flex workers, and any keep-warm floor is idle billing you volunteered for. All prices below are list prices as of July 2026 and will drift — treat them as a snapshot, not scripture.

Two readings of that table are worth resisting. Baseten is not "60% more expensive than Modal" — its rate buys dedicated capacity management, region-locking and an enterprise support surface that the cheaper platforms charge you in engineering time instead. And RunPod's pod pricing being cheaper than its serverless pricing is not an anomaly: you are paying the serverless premium for the autoscaler, the queue and FlashBoot. If your utilisation is high enough that the premium annoys you, that is the signal to re-run the cost maths on serving your own stack.

What three common workloads actually cost

Rate cards abstract away the question that matters: what does one unit of your work cost? The table below estimates three canonical tasks on comparable 80GB A100 hardware, using each platform's July 2026 list price. These are estimates, not benchmarks — the stated assumptions are: a single request on a warm worker (no cold start, no batching, no keep-warm amortisation); Whisper large-class transcription at roughly 20x real time; an SDXL-class 1024px image in about 5 seconds; and a 7B-parameter LLM generating 1,000 tokens at about 90 tokens/second single-stream. Your model, quantisation and batch depth will move every number, in either direction.

Three honest caveats. First, at these per-task magnitudes, cold starts and idle windows can easily exceed the task itself — a 30-second cold start ahead of a 5-second image multiplies its effective cost several times over, which is why the previous section matters more than this one. Second, Baseten's per-minute granularity slightly penalises sub-minute one-off tasks but washes out entirely under steady traffic. Third, for popular public models on Replicate, per-output pricing frequently beats this per-second arithmetic — always check the model's own pricing line before assuming. Batching changes everything again: a continuously-batched 7B endpoint serving many concurrent streams can cut effective per-request cost by an order of magnitude, exactly as it does when self-hosting with vLLM.

Regions and data residency for India and UK teams

For a category built on abstracting infrastructure away, serverless GPU platforms make you think surprisingly hard about where the silicon physically sits. The picture as of July 2026, platform by platform, from most to least controllable.

Baseten has the strongest residency story of the four. Its multi-cloud capacity management layer supports region-locking and provider-locking per deployment, and its self-hosted and hybrid modes run inference entirely inside your own cloud account, with Baseten stating that inputs and outputs never touch its premises in self-hosted mode. For Indian teams in RBI-regulated financial services, or UK teams whose data-protection impact assessments frown on ambiguous processing locations, that is often the difference between usable and not. Modal exposes explicit region selection — broad us, eu and ap groups with narrower selectors beneath them, including ap-south — and its documentation explicitly positions region="eu" as a GDPR data-residency control. Note the documented trade-off: the narrower the region, the smaller the capacity pool and the worse your cold starts and availability. RunPod publishes a map of roughly 31 regions across the US, Canada, Europe and Asia-Pacific, with a broad EU footprint; region pinning is per-endpoint, but confirm current region-level GPU availability in the console rather than assuming a specific city. Replicate offers no customer-facing region selection as of July 2026 — you should assume US-centric processing unless its documentation tells you otherwise for a given deployment.

Latency follows geography. From London, EU-hosted GPUs are a round trip of a few tens of milliseconds — imperceptible next to LLM generation time. From Mumbai, the nearest well-stocked GPU regions are often Southeast or East Asian rather than Indian, adding very roughly 50–100ms, and US-only processing adds on the order of 200ms each way — irrelevant for batch transcription, noticeable at the start of every streamed chat response, and material for voice. On the regulatory side, India's DPDP Act broadly permits cross-border transfers except to government-restricted jurisdictions, but sectoral overlays (RBI localisation expectations chief among them) can be stricter, and UK GDPR requires recognised safeguards — adequacy, the UK-US data bridge, or contractual clauses — for transfers out of the UK. None of this forbids using these platforms; all of it obliges you to know, and document, where inference runs.

The Cloudflare angle deserves its own sentence: the acquisition points Replicate towards inference running across Cloudflare's global edge network, which — if delivered — would invert Replicate's position from the least geographically flexible platform of the four to the most distributed. As of July 2026 that is direction of travel, not a shipped, controllable feature; plan on what exists today.

The decision matrix

No single winner exists, which is the honest conclusion of every real platform comparison. But the mapping from situation to shortlist is fairly stable:

Where to go from here

Treat the choice as reversible and the evaluation as cheap, because both are true. All four platforms let you deploy a containerised model in an afternoon, and Modal's free monthly credits mean a serious trial can cost nothing at all. A sensible sequence for a small team in Bengaluru or Bristol: pick the two platforms your situation maps to in the matrix above; deploy the same model and container to both; replay a day of real traffic at each; and compare cold-start rate, P95 latency from your users' geography, and the invoice — not the rate card. Keep your container slim and your weights cached wherever you land, because that discipline transfers between platforms even when nothing else does.

And revisit the decision on a calendar, not on vibes. Prices in this piece are date-stamped July 2026 for a reason: this market reprices constantly, Cloudflare's absorption of Replicate is likely to reshape one corner of it, and the moment your traffic flattens into steady 24/7 utilisation, the right answer may stop being serverless at all. If you have shipped production inference on any of these platforms — or migrated between them and kept the receipts — that is exactly the kind of concrete, load-bearing work worth showing to the people hiring for it.