Durable Agent Execution: Checkpoint, Resume, Never Replay

The failure that looks like a success

Picture a settlement agent. It serves a subscription business with customers in Bengaluru and Birmingham, and once a month it works through a list of accounts. For each account it reads the usage, decides the amount, calls the payment provider, writes an invoice row, and emails a receipt. It is a perfectly ordinary agent: a handful of nodes, a model call in the middle to handle the awkward proration cases, four tools, one loop.

At 02:14 the host running it is replaced. A node pool scales in, or a deploy rolls, or the kernel decides it has had enough. The process disappears somewhere after the payment call has gone out and before the invoice row has been written. Nobody notices, because the system is built for this: there is a checkpointer, the thread has a saved state, and the supervisor restarts the run.

The run resumes. It resumes from the last committed checkpoint, which was taken before the payment node ran, because the process died before that node's output was committed. So the payment node executes again. It calls the provider again. The provider, having no reason to think otherwise, takes the money again. The invoice is written, the receipt is emailed, the run completes, and every dashboard in the building turns green. The alert you get three days later comes from a customer, not from your monitoring.

This is the gap at the centre of the subject. Checkpointing is a restart mechanism. Durable execution is a correctness property. The two are routinely conflated, including by engineers who would never conflate them in a database context, and the conflation is expensive precisely because the failure mode presents as a success.

LangGraph's own documentation is unusually honest about this, which is worth crediting. It states plainly that on resume, later graph work can re-execute; that nondeterministic operations and side effects should be wrapped in tasks or nodes; and that mutating operations still need idempotency. That last clause is the whole article. The framework gives you resumability. It cannot give you exactly-once effects, because exactly-once effects are not a property of your orchestration layer — they are a property of the contract between your code and every external system it touches.

Three guarantees that are not the same guarantee

Teams talk about durability in a single word, which is why the conversation goes badly. There are three distinct properties, stacked, and each is strictly stronger than the one below it.

Checkpointing means the run's state is persisted at defined boundaries, so that a new process can pick the run up from a known-good point rather than from the beginning. In LangGraph terms this is the checkpointer: a store that captures the graph state after each superstep. Checkpoints are what make memory, fault recovery, state history, time travel and human-in-the-loop possible — five capabilities that all come from the same primitive, which is part of why it is so tempting to believe the primitive does more than it does. What checkpointing guarantees is that the run does not lose its place. It guarantees nothing about what the run already did to the outside world.

At-least-once execution means every step will run to completion at least once, with retries, until it succeeds or the run gives up. This is what you actually get from a checkpointer plus a supervisor that restarts failed runs. It is a genuinely useful guarantee — it is the difference between a job that finishes and a job that silently stops halfway — and it is the guarantee that most agent stacks provide by default as of September 2026. It is also the guarantee that charges the customer twice.

Exactly-once effect means that no matter how many times a step is attempted, the observable consequence in the external system happens once. Note the careful wording. Not exactly-once delivery, which is impossible, but exactly-once effect, which is achievable and is the only thing your finance team actually cares about.

Why delivery cannot be exactly-once

The impossibility is not a limitation of any particular framework, and it is worth being precise because engineers who have not met it before tend to assume a library will solve it. Your agent sends a charge request to a payment provider. The request goes out. Then the connection drops. You have no response.

You now know exactly one thing: you do not know whether the charge happened. The failure might have occurred before the provider received the request, after it processed the request but before the acknowledgement was written, or anywhere in the acknowledgement's journey back to you. These three worlds are indistinguishable from where you are standing, and no amount of local logging distinguishes them, because your local log can only record what you sent and what you received.

Given that, you have exactly two choices. Send again, and risk a duplicate — at-least-once. Do not send again, and risk the charge never happening — at-most-once. There is no third option that keeps the semantics of a single send. Every system that claims exactly-once has, underneath the claim, chosen at-least-once and then made the duplicate harmless.

That is the move. You cannot control how many times the message arrives, so you control what the receiver does with the second one. The caller attaches a stable key. The receiver recognises the key, declines to repeat the work, and returns the original result as though it had just done it. Delivery stays at-least-once; the effect becomes once. Everything in the rest of this guide is an elaboration of that single idea.

Classify your side effects before you write any code

The work is not "make everything idempotent". Most of your graph does not need protecting, and treating it as though it does produces a system nobody wants to maintain. The work is to classify each effect and then apply the treatment that class deserves. There are four classes, and the boundaries are sharp enough to use as a checklist.

Class	Examples	What a replay does	Treatment
Pure compute	Parsing, formatting, scoring, prompt assembly, deterministic routing decisions	Produces the same answer. Costs a little CPU.	Nothing. Let it re-run. Do not add machinery here.
Idempotent write	Upsert by primary key, setting a status field to a fixed value, writing a file at a fixed path	Converges to the same state. No harm done.	Nothing, provided the write is genuinely a set and not an increment. Check this claim; teams get it wrong.
Non-idempotent external call	Card charge, UPI collect request, ticket creation, outbound webhook, counter increment, queue publish	Fires twice. Two charges, two tickets, a counter that is now wrong.	Idempotency key plus a result store. This is the main body of the work.
Irreversible action	Email or SMS to a customer, file deletion, an executed trade, a public post, a KYC submission	Fires twice and cannot be undone by any means.	Move the trigger behind a keyed gate, and prefer to make it the very last step. Compensation does not exist here.

Two observations about this table that change how people build. The first is that the classification is about the effect, not the tool. The same HTTP client is class one when it does a GET and class three when it does a POST. Classify call sites, not libraries.

The second is that class four is a design constraint, not an engineering problem. If your agent sends a customer-facing email in the middle of a ten-node graph, you have accepted that a mid-run failure may double it, because there is no key you can attach to a human's inbox that makes the second message unhappen. The fix is structural: buffer the intent, and send at the end, once, from a single node that does nothing else. The same instinct that keeps harmful actions inside a bounded blast radius applies here — irreversible things go last, alone, and behind a gate.

A useful exercise, and one that takes an hour: open your graph definition, list every node, and write the class next to it. In a typical production agent of fifteen nodes, ten are class one, two or three are class two, one or two are class three, and there is usually exactly one class four that nobody had thought about. That last one is your incident.

Idempotency keys in practice

An idempotency key is a string that identifies a logical operation, not an attempt at it. Every property of a good key follows from that sentence. It must be identical across attempts, distinct across different logical operations, derivable without any coordination, and computable before the operation runs.

The derivation that satisfies all four is a hash over values that are fixed at the moment the run reaches that point: the thread or run identifier, the node identifier, an occurrence counter for that node within the run, and a canonical serialisation of the attempt-invariant inputs. Nothing generated inside the node may participate. No timestamps, no fresh UUIDs, no random draws, no unpinned model output.

import hashlib
import json
from dataclasses import dataclass
from typing import Any, Callable, Optional


def idempotency_key(
    thread_id: str,
    node_id: str,
    occurrence: int,
    payload: dict[str, Any],
) -> str:
    """Stable across attempts, distinct across logical operations.

    Every input must be attempt-invariant. If you cannot explain why a
    value will be byte-identical on the replay, it does not belong here.
    """
    canonical = json.dumps(payload, sort_keys=True, separators=(",", ":"))
    material = f"{thread_id}|{node_id}|{occurrence}|{canonical}"
    digest = hashlib.sha256(material.encode("utf-8")).hexdigest()
    return f"{node_id}-{digest[:32]}"


@dataclass
class Outcome:
    key: str
    status: str          # "in_flight" | "succeeded" | "failed"
    result: Optional[dict[str, Any]]


class ResultStore:
    """Durable record of intent and outcome, keyed by idempotency key.

    Back this with the same Postgres instance as your checkpointer so a
    single transaction can cover both when you need it to.
    """

    def __init__(self, conn):
        self._conn = conn

    def claim(self, key: str) -> Optional[Outcome]:
        """Record intent. Returns None if this attempt owns the work,
        or the existing Outcome if someone already claimed the key."""
        with self._conn.transaction():
            row = self._conn.fetchone(
                "SELECT key, status, result FROM effect_log "
                "WHERE key = %s FOR UPDATE",
                (key,),
            )
            if row is not None:
                return Outcome(row["key"], row["status"], row["result"])
            self._conn.execute(
                "INSERT INTO effect_log (key, status, result) "
                "VALUES (%s, 'in_flight', NULL)",
                (key,),
            )
            return None

    def complete(self, key: str, result: dict[str, Any]) -> None:
        self._conn.execute(
            "UPDATE effect_log SET status = 'succeeded', result = %s "
            "WHERE key = %s",
            (json.dumps(result), key),
        )

    def release(self, key: str) -> None:
        """Failed before the effect could have landed — allow a clean retry."""
        self._conn.execute("DELETE FROM effect_log WHERE key = %s", (key,))

Three details in that code carry the weight. Intent is recorded before the effect, not after — a log written after a successful call tells you nothing about the call that died mid-flight. The claim is taken under a row lock, so two workers resuming the same thread cannot both believe they own the work. And release is deliberately narrow: it is only safe to call when you know the effect could not have landed, which in practice means a connection error raised before the request was transmitted, or an explicit rejection from the provider. Anything ambiguous stays in_flight and is resolved by reconciliation, never by deletion.

Now the wrapper that a node actually uses.

class TransportNotSent(Exception):
    """Raised by the client only when the request provably never left us —
    DNS failure, connection refused, TLS handshake rejected. Anything that
    might have reached the provider must NOT raise this."""


class EffectAlreadyInFlight(Exception):
    """Prior attempt may have landed. Do not retry blindly."""


def durable_effect(
    store: ResultStore,
    thread_id: str,
    node_id: str,
    occurrence: int,
    payload: dict[str, Any],
    perform: Callable[[str, dict[str, Any]], dict[str, Any]],
) -> dict[str, Any]:
    """Run `perform` at most once per logical operation.

    `perform` receives the key and must forward it to the downstream
    system as its own idempotency header. Local deduplication alone is
    not enough: the process can die between the call and the write.
    """
    key = idempotency_key(thread_id, node_id, occurrence, payload)

    existing = store.claim(key)
    if existing is not None:
        if existing.status == "succeeded":
            return existing.result          # replay: return, do not redo
        if existing.status == "in_flight":
            raise EffectAlreadyInFlight(key)
        store.release(key)
        existing = store.claim(key)

    try:
        result = perform(key, payload)
    except TransportNotSent:
        store.release(key)                   # provably never reached them
        raise
    store.complete(key, result)
    return result


# --- the node itself -------------------------------------------------

def charge_customer(state: dict) -> dict:
    payload = {
        "account_id": state["account_id"],
        "amount_minor": state["amount_minor"],   # decided upstream, checkpointed
        "currency": state["currency"],           # "INR" or "GBP"
    }
    result = durable_effect(
        store=STORE,
        thread_id=state["thread_id"],
        node_id="charge_customer",
        occurrence=state["billing_cycle"],       # stable per cycle, not a counter in RAM
        payload=payload,
        perform=lambda key, p: payments.charge(**p, idempotency_key=key),
    )
    return {"charge": result}

Read the replay path carefully, because it is the point of the whole exercise. On the second attempt, claim finds a succeeded row and the function returns the stored result. The provider is never called. The node's output is byte-identical to the first attempt, so everything downstream of it behaves as though the failure never happened. The graph re-executed; the effect did not.

Note also that the amount is taken from checkpointed state rather than recomputed. If the amount were derived inside the node from a model call, the replay would produce a different payload, a different key, and a second charge. Anything that feeds the key must be pinned upstream and committed. This is the same discipline that makes agent state survive summarisation — see the guide on keeping an agent's rules alive through compaction for the general case, and the deeper treatment of races and duplicate work in concurrency bugs in multi-agent systems.

The in_flight branch deserves its own paragraph. Raising rather than retrying feels unhelpful the first time you hit it, and it is the correct behaviour. An in_flight row means a previous attempt reached the point of no return and we do not know how it ended. Retrying risks a duplicate; treating it as failed risks losing a real payment. The resolution is to query the provider by the key — every serious payments API in India and the UK supports a lookup by idempotency key or reference — and to write the answer into the store. That reconciliation belongs in a separate, human-supervised path, not in the agent loop.

Saga compensation: when you cannot deduplicate, be reversible

Idempotency keys assume a cooperative receiver. Plenty of systems are not cooperative. Internal services written before anyone thought about retries, partner APIs that accept no such header, legacy endpoints in exactly the corners of a business where the money moves. For those, the fallback is the Saga pattern: pair every forward action with a compensating action that semantically undoes it, and if a later step fails, wind the completed steps back in reverse order.

LangGraph 1.2.0, released on 11 May 2026, added Saga compensation as a first-class feature alongside graceful shutdown and per-node timeouts. Whether you use that implementation or write your own, the shape is the same, and the shape is what matters.

from typing import Protocol


class Step(Protocol):
    name: str
    def forward(self, state: dict) -> dict: ...
    def compensate(self, state: dict, forward_result: dict) -> None: ...


class ReserveInventory:
    name = "reserve_inventory"

    def forward(self, state: dict) -> dict:
        return inventory.reserve(
            sku=state["sku"],
            qty=state["qty"],
            ref=f"{state['thread_id']}:reserve",
        )

    def compensate(self, state: dict, forward_result: dict) -> None:
        # Keyed on the reservation id, so releasing twice is harmless.
        inventory.release(reservation_id=forward_result["reservation_id"])


def run_saga(steps: list[Step], state: dict, store: ResultStore) -> dict:
    completed: list[tuple[Step, dict]] = []
    try:
        for i, step in enumerate(steps):
            result = durable_effect(
                store, state["thread_id"], step.name, i,
                payload=state, perform=lambda k, s: step.forward(s),
            )
            completed.append((step, result))
            state = {**state, step.name: result}
        return state
    except Exception:
        for step, result in reversed(completed):
            key = idempotency_key(
                state["thread_id"], f"compensate:{step.name}", 0, result,
            )
            if store.claim(key) is None:          # compensate at most once
                step.compensate(state, result)
                store.complete(key, {"compensated": True})
        raise

Two things are easy to miss here. The compensations are themselves run through the result store, because a crash during the unwind is just as likely as a crash during the forward pass, and a compensation that runs twice is a bug of exactly the same family as a forward action that runs twice — releasing the same inventory twice can free stock you no longer hold. And the unwind runs in reverse order, because the steps have dependencies and undoing them out of order can leave a state that neither the forward nor the backward path expects.

Choosing a checkpointer

The choice is narrower than the discussion around it suggests. As of September 2026, LangGraph's guidance is that SQLite suits a single server and PostgresSaver is the option for multiple instances. That is the decision, and the interesting part is what changes when you cross the line.

Concern	SQLite, single server	Postgres, multiple instances
Who can resume a thread	One process, by construction	Any worker — so you must decide who is allowed to
Concurrent resume	Not possible	Possible and likely; needs a lease or a row lock per thread
Effect log co-location	Same file; a single transaction covers state and effects	Same database if you put it there; otherwise two systems to reconcile
Survives host loss	Only if the volume does	Yes, subject to your database's own durability settings
Data residency	Wherever that one host is	Pin the database region explicitly — Mumbai or London, decided, not inherited
Good fit for	Development, single-tenant tools, batch jobs on one box	Anything with an autoscaler, a rolling deploy, or an uptime commitment

The row that costs teams the most is concurrent resume. On a single server the checkpointer is a private notebook. On several instances it is shared coordination state, and two workers can both decide a thread is orphaned and both resume it. Every idempotency key you derived now has to hold not only across sequential retries but across simultaneous attempts, which is exactly why the claim above takes a row lock rather than doing a read followed by an insert. A read-then-insert has a window, and distributed systems find windows.

The residency row matters more in this market than in most. A team in Chennai serving UK customers and a team in London serving Indian ones face the same question from opposite directions, and the checkpointer is a genuine data store — it holds conversation state, tool arguments and whatever your agent has read. Decide its region deliberately and write the decision down; do not let it be inherited from whichever region your cluster happened to default to. The broader trade-offs of who hosts what are laid out in the managed runtime versus own harness decision ladder.

Third-party durable-execution runtimes are a legitimate third option. Temporal ships a LangGraph plugin, and there are others; they take on retry bookkeeping, timers and replay determinism as a platform concern rather than something you hand-roll. This guide takes no position on whether any of them is right for you, and makes no performance claims for them. What is worth saying is that adopting one does not delete the work in this article. A durable runtime enforces replay discipline inside its own boundary; the payment provider on the other side of the boundary still needs your key.

Graceful shutdown and deployments

Most mid-run kills are not mysterious infrastructure failures. They are deploys. You ship four times a day, each ship rolls the pods, and every agent mid-flight at that moment is a candidate for the double-charge story above. Reducing the number of ungraceful terminations is therefore the cheapest reliability work available to you, and it is mostly configuration.

LangGraph 1.2.0 added graceful shutdown on 11 May 2026 with sensible semantics. On a shutdown signal, the agent finishes its current superstep, saves a resumable checkpoint, and raises GraphDrained. The critical detail is that GraphDrained represents a paused state, not a failure. It means the run stopped at a clean boundary with its state committed, and it is waiting to be resumed by whoever picks it up next.

Treat that distinction as an operational rule. A GraphDrained outcome must not page anyone, must not increment your error rate, and must not trigger the retry path that exists for genuine crashes. It should increment a drain counter and enqueue a resume. Teams that route it into the generic exception handler get two bad outcomes at once: alert fatigue during every deploy, and a retry path exercised so often that nobody looks at it.

The same release added per-node timeouts, which solve the adjacent problem. Without them a single hung tool call blocks the whole run, and when the orchestrator eventually gives up it does so from outside the graph, with no checkpoint and no idea which effects landed. A per-node timeout turns that into a bounded, in-graph failure that you can handle deliberately. Set them per node rather than globally: a model call and a database read have nothing in common, and a single global timeout is always wrong for one of them.

A drain-aware deployment sequence looks like this, and the numbers are yours to choose based on your own node durations rather than copied from anyone:

1. Stop admitting new runs to the instance being replaced. Nothing else should happen until this has taken effect.
2. Send the shutdown signal. In-flight runs finish their current superstep and checkpoint.
3. Set the grace period to comfortably exceed your longest per-node timeout. If the platform kills the process before the superstep completes, you have a graceful shutdown that is not graceful.
4. Let the supervisor resume drained threads on the new instances. Because they were drained at a boundary, resumption is ordinary work and not recovery.
5. Alert only on runs that neither completed nor drained. That set should be small, and every member of it deserves a look.

Long-running work deserves a note. If your agent runs for hours rather than seconds, draining cleanly on every deploy is harder and the design shifts towards treating each run as a resumable job in its own right — the territory covered in background agents and durable execution for long-running jobs. The instinct transfers directly from infrastructure work many teams already do on preemptible capacity, where losing the machine is routine rather than exceptional.

Testing durability: the kill-and-resume drill

Durability is a claim about what happens when things go wrong, so a test suite that never makes things go wrong tests nothing. The drill is simple, mechanical, and the highest-value thing in this guide.

Run the agent against fake external systems that count invocations by key. Kill the process at a chosen point. Restart from the checkpointer. Assert that the run completed and that every external call happened exactly once. Then repeat with the kill point moved, until you have covered every boundary in the graph.

import subprocess
import pytest


class CountingPayments:
    """Stand-in provider that enforces real idempotency semantics."""

    def __init__(self):
        self.calls: list[dict] = []
        self.by_key: dict[str, dict] = {}

    def charge(self, *, idempotency_key, **payload):
        self.calls.append({"key": idempotency_key, **payload})
        if idempotency_key in self.by_key:
            return self.by_key[idempotency_key]          # replayed, not repeated
        result = {"charge_id": f"ch_{len(self.by_key) + 1}", "status": "captured"}
        self.by_key[idempotency_key] = result
        return result

    def distinct_charges(self) -> int:
        return len(self.by_key)


KILL_POINTS = [
    "after:decide_amount",
    "before:charge_customer",
    "during:charge_customer",      # request sent, process dies before response
    "after:charge_customer",
    "before:write_invoice",
    "before:send_receipt",
]


@pytest.mark.parametrize("kill_point", KILL_POINTS)
def test_exactly_once_effect_across_a_kill(kill_point, tmp_path):
    thread_id = f"drill-{kill_point.replace(':', '-')}"

    # Attempt 1: run under a supervisor that SIGKILLs at `kill_point`.
    first = subprocess.run(
        ["python", "-m", "agent.run",
         "--thread", thread_id,
         "--checkpoint-db", str(tmp_path / "ckpt.sqlite"),
         "--effect-log", str(tmp_path / "effects.sqlite"),
         "--kill-at", kill_point],
        capture_output=True,
    )
    assert first.returncode != 0, "the drill must actually kill the process"

    # Attempt 2: same thread id, no kill switch. Resumes from the checkpoint.
    second = subprocess.run(
        ["python", "-m", "agent.run",
         "--thread", thread_id,
         "--checkpoint-db", str(tmp_path / "ckpt.sqlite"),
         "--effect-log", str(tmp_path / "effects.sqlite")],
        capture_output=True,
    )
    assert second.returncode == 0

    payments = load_recorded_provider(tmp_path)
    assert payments.distinct_charges() == 1, "customer charged more than once"
    assert count_invoices(thread_id) == 1
    assert count_receipts(thread_id) == 1

    # The provider may legitimately be CALLED twice — once per attempt —
    # so long as both calls carry the same key and only one charge exists.
    keys = {c["key"] for c in payments.calls}
    assert len(keys) == 1, f"key changed across attempts: {keys}"

The final assertion is the one that earns its place. It fails loudly for the most common bug in this whole area: a key that is recomputed on the replay from something that has changed, so the downstream system sees two different logical operations and obliges. A test that only counted charges against a naive fake would pass while the production system double-charged, because the naive fake deduplicates nothing.

SIGKILL rather than an exception matters too. An exception unwinds the stack and runs your finally blocks, which is the polite failure you already handle. SIGKILL runs nothing. That is the failure that actually happens when a node is reclaimed, and it is the one worth testing. Once the parameterised drill is green, extend it into randomised chaos: kill at a random millisecond offset, a few hundred times, in a nightly job. Fixed kill points prove the boundaries you thought of are safe. Random kills find the ones you did not. This sits naturally alongside the fault-injection techniques in the guide on testing agent reliability with pass^k and perturbation.

Observability: proving exactly-once after the fact

When someone asks in six months whether a particular customer was charged twice on a particular night, you need an answer from data, not from an argument about the design. That requires deciding now what to record.

Log a row at effect-claim time, before the call goes out, carrying the idempotency key, thread id, node id, occurrence, attempt number, and a hash of the payload. Log a second row at resolution, carrying the key, the outcome, the downstream identifier the provider returned, and whether the result was freshly computed or served from the store. That last field — call it replayed: true|false — is the single most useful thing in your logs, because it turns the invisible question into a query.

Put the idempotency key on the trace span too, as an attribute, so a distributed trace and the effect log join on it. With that in place the proofs become mechanical:

• Was anything charged twice? Group the effect log by key and count fresh, non-replayed successes. Any key with more than one is an incident, and the query is cheap enough to run on a schedule rather than after a complaint.
• Are keys stable? Group by thread and node, and count distinct keys. More than one distinct key for the same logical operation means a nondeterministic input has crept into the derivation, usually via a refactor.
• How much work is replay? The ratio of replayed to fresh results tells you how often runs are being resumed, which is a deploy-hygiene metric more than an agent metric.
• What is stuck? Anything in_flight beyond a threshold needs reconciliation against the provider. This should be a dashboard with a name and an owner, not a query someone remembers to run.

Keep drain events separate from failures in your metrics, for the reason given earlier: a deploy that drains forty runs is a healthy deploy, and if that shows up as forty errors, the number stops meaning anything. The general instrumentation approach — spans, attributes, cost attribution — is covered in the guide on instrumenting agents with OpenTelemetry; the only addition here is that the idempotency key is a first-class attribute and not a debug string.

Common pitfalls

These are the ones that recur, roughly in order of how often they bite.

Keying on something generated inside the node

A uuid4(), a datetime.now(), a random sample, or an unpinned model output. On the replay, the value changes, the key changes, and the downstream system sees a new request. This defeats the entire mechanism while looking completely correct in review, because the code plainly does attach an idempotency key. Ask of every input to the key: why will this be byte-identical on the replay? If there is no good answer, it does not belong in the key.

Treating the checkpoint as the source of truth for external state

The checkpoint records what your graph believed. It does not record what the payment provider did. If the process died between the provider committing the charge and your node returning, the checkpoint says no charge and the provider says charge. The checkpoint is wrong, and it will stay wrong until something reconciles it. External state is owned by the external system; your job is to hold a key that lets you ask.

Compensations that are not themselves idempotent

The unwind path gets a fraction of the review attention the forward path gets, and it runs under exactly the same crash conditions. A compensation that releases inventory twice, or issues two refunds, is the original bug wearing a different hat. Run compensations through the same result store, key them on the forward result, and test the unwind path with the same kill drill.

Assuming retries only happen sequentially

Once you move to a shared checkpointer, two workers can resume the same thread at the same moment. A read-then-insert on the effect log has a window between the two statements, and both workers can pass through it. Take a lock, or use a unique constraint and treat the violation as the signal that someone else owns the work.

Putting the irreversible action in the middle

An email, an SMS, a public post or an executed trade in node six of twelve means every failure in nodes seven through twelve risks doubling it on the resume. Move it to the end, alone, behind a keyed gate. This is free at design time and impossible to retrofit cheaply.

Wrapping everything

The opposite failure, and a real one. Putting a result-store round trip in front of pure computation buys nothing and costs a database call per node, plus a codebase where the genuinely dangerous calls no longer stand out. Wrap class three and class four. Leave class one and class two alone. The classification table exists so you can be selective with a clear conscience.

Where this leaves you

Durable execution is not a feature you switch on. It is a property you establish per effect, and the framework's job is to give you the boundaries at which to establish it. LangGraph gives you good ones: checkpoints at every superstep, graceful shutdown that drains to a paused state rather than a failure, per-node timeouts that keep a hung call from taking the run with it, and Saga compensation for the systems that will not cooperate with a key. What it cannot give you — what nothing can give you — is a guarantee about a payment provider it has never heard of.

If you are starting from a working agent with a checkpointer and nothing else, the order of work is fixed by return on effort. Classify every node into the four classes; the exercise takes an hour and usually finds one class four nobody had noticed. Wrap the class three and class four calls with a keyed result store, recording intent before the effect. Write the kill-and-resume drill for every boundary, and make it part of CI rather than a thing someone ran once. Make drain events a first-class, non-alerting outcome so deploys stop generating noise. Then add the replayed-versus-fresh metric, so the question "did we ever charge anyone twice" has an answer that is a query.

None of it is exotic. All of it is the difference between an agent that recovers and an agent that appears to.

Sources

• LangGraph durable execution and checkpointing documentation, including the caveat that later graph work can re-execute on resume — langchain-ai.github.io/langgraph
• LangGraph 1.2.0 release, 11 May 2026: graceful shutdown, per-node timeouts and Saga compensation — github.com/langchain-ai/langgraph/releases
• Temporal's LangGraph plugin, as an example of a third-party durable-execution runtime — temporal.io