ADR-0008: Sagas — a small orchestrator over the existing effect primitives¶

• Status: Accepted (2026-10-25; phase 0 and phase 1 built)
• Date: 2026-10-25
• Deciders: Sandloom maintainers (open questions at the end)
• Complements: ADR-0005 (the effect model; this ADR adds one matrix row and no new dedupe record), ADR-0007 (kernel layers; the placement below), ADR-0003 (billing operation intents, the hand-rolled saga this generalises later, not now)
• Research: docs/design/sagas-research-2026-10-25.md (internal research, retained in the original private repository) (inventory, survey with sources, option analysis, worked example)

Context¶

Cross-module writes in Sandloom are sagas by construction. A command contract runs in the provider's own Unit of Work and is "NOT atomic with the caller's transaction — design it as a saga step" (AGENTS.md). There is no distributed transaction, and there never will be (ADR-0005 §1). The framework already has every primitive a saga step needs:

• the outbox and its relay: a durable intent written in the same transaction as the business change;
• durable consumers and durable_inbox: exactly-once effect per event;
• command contracts with the durable command log: exactly-once effect at the provider, keyed by an idempotency_key, local and RPC alike. A concurrent repeat waits for the in-flight call and never runs twice;
• the task queue, the beat lease and projections.

What it lacks is the part that turns those primitives into a business process:

• a place where "where is this process now?" is recorded;
• a rule that derives step operation keys;
• forward and backward recovery;
• deadlines;
• a way for an operator to list, retry, compensate or park a stuck instance.

Today each flow writes this by hand. Billing's operation intents (ADR-0003 G1) and their reconciler are a complete hand-rolled saga, about 1,330 lines of service code plus tests. docs/ROADMAP.md lists "Saga / process-manager" as a capability gap, and review C2 #13 proposes sandloom.core.sagas. Without a shared shape, the next flows will each invent their own:

• org onboarding;
• user erasure (review C5 N5);
• org deletion → subscription cancel.

Decision drivers¶

1. Reuse ADR-0005. ADR-0005 forbids a new dedupe store or post-commit variant unless an amendment justifies one.
2. Local equals remote. Monolith and split deployments must produce the same outcome.
3. Only deterministic checks. Every check is structural and runs at startup or registration. None is a heuristic lint.
4. Module isolation. A saga talks to other modules only through command contracts and events, referenced by string.
5. Fits ADR-0007. The saga runtime has a place in the layer map, and it adds no ignore_imports.
6. Pragmatic. Reuse the existing infrastructure and keep orchestration off the request hot path. This decision preceded the initial public alpha.

Considered options¶

(A) stays the right tool for fan-out reactions without compensation (erasure, notifications). (C) stays possible as a plugin through the existing experimental workflow capability contract. Neither gives the default platform a saga with central state, deadlines and operator control at zero infrastructure cost.

Decision¶

Adopt (B): sandloom.core.sagas is a small orchestrator. A saga is a named, versioned, linear sequence of steps, with backward recovery (compensation) before its pivot and forward recovery (retry) after it. All the effect guarantees come from primitives that already exist. The orchestrator only adds the instance state machine, which is version-guarded state, not a dedupe record. Phase 0 also ships the (A) guidance. (C) is documented as a plugin path.

1. Definition (registration-time checks)¶

A module declares sagas in setup with core.register_saga(definition). A definition has:

• a name, unique per module (<module>.<name>);
• a version;
• an input model, which is persisted in the owner's database and never crosses RPC, so it may live in the module;
• an overall deadline;
• an ordered list of steps.

Each step is one of the following:

A step also has a kind. The three kinds follow Garcia-Molina & Salem and Richardson:

• compensatable: may be undone;
• pivot: the go/no-go point, at most one;
• retriable: after the pivot, must eventually succeed.

register_saga refuses the following, deterministically:

• duplicate step names;
• a compensatable step without a compensation;
• a compensation on a pivot or retriable step;
• a step before the pivot that is not compensatable, or a step after it that is not retriable;
• more than one pivot;
• a call= that is not a command contract of a registered local provider.

For a remote provider, the bound contract's describe document is checked by the existing contract-compat check. Registering a saga adds a ContractRequirement(name, methods=(...)) for every step to the module's consumes. So a missing provider is a ContractBindingError at startup, as for any other consumed contract.

2. State: two core-chain tables in the owning module's database¶

saga_instances and saga_steps join the core chain. That chain already holds the shared infra tables that every hosting database receives (outbox, audit, inbox, command log). The rows live in the database that hosts the owning module, the same rule as its outbox. saga_type is prefixed <module>., so sandloom-migrate extract-module copies a module's instances with it, the way it copies command-log records.

• saga_instances has these columns: id, saga_type, definition_version, saga_id (the business id, UNIQUE (saga_type, saga_id)), tenant_id, status, step_index, direction (forward / backward), attempt, next_attempt_at, deadline_at, input (JSON), results (JSON, step name → result), last_error, version, created_at, updated_at. The status is one of running, compensating, completed, compensated, failed or parked.
• saga_steps holds one row per (instance, step, direction), updated in place: status, attempts, first_attempt_at, last_attempt_at, last_error. It exists for sandloom sagas show and for metrics. It is not a dedupe record.

These tables are not TenantScoped. Like the outbox, they are kernel infra. The runtime binds the instance's tenant_id while it runs a step, so command calls carry the signed tenant context. input and results must hold ids, not PII; the same rule applies to event payloads.

Row-level security. Under tenant_rls (and therefore under PRODUCTION_PROFILE=multi_tenant_strict) both tables are migrated with enable_tenant_rls(op, "<table>", system_access=True). The runtime always touches an instance with its tenant_id bound, so the app role sees exactly the rows of the instance it drives. Cross-tenant reads and writes — the timer sweep, sandloom sagas list without --tenant, and every operator mutation — go through the kernel's audited system channel (system_session(actor=, reason=)), never through unscoped(). The strict profile's startup check covers both tables like any other tenant table.

3. Operation keys¶

The runtime injects every step's idempotency argument, using the contract's declared idempotency_key parameter:

saga:<saga_type>:<saga_id>:<step>:do      # forward
saga:<saga_type>:<saga_id>:<step>:undo    # compensation

Arguments are recomputed on every attempt from the persisted input and results. The command log's argument digest therefore catches a non-deterministic args callable: the same key with other arguments returns 409 ContractIdempotencyConflict. The runtime parks the instance with nondeterministic_args. No new mechanism is needed, because the existing record already enforces it.

4. Start: transactional with the business change¶

await start_saga(ORG_SETUP, saga_id=org.id, input=...), imported from sandloom.core.di, runs inside the caller's UoW. It inserts the instance row and publishes <module>.saga_advance.v1 through the outbox in the same transaction. Starting the same saga_id again is a create-or-get on the UNIQUE key. If the input differs, the call is a conflict (409); if it matches, it returns the existing instance. Outside a UoW, start_saga raises UnitOfWorkRequiredError, like publish_event.

5. Driving: an outbox-fed consumer plus a timer sweep¶

• Advance: each module that registers a saga gets a kernel-installed consumer <module>:sagas@advance on its private channel <module>.saga_advance.v1. It runs with dead_letter=False, because a saga must never be silently parked in a DLQ, and on the owning database. In a split deployment, only the owning module's service consumes it.

An advance event is a nudge, not a command: its payload is {instance_id, expected_version}. The handler locks the row and does nothing unless the row is active, due, and still at expected_version. A duplicate, a late or a reordered nudge is therefore a no-op by the state check alone. The consumer is registered without the durable inbox (durable=False, an existing consumer option, not a new mode): an inbox claim would protect nothing the row lock and version do not already protect, and would add an INSERT per step and a growing table. - Timer sweep: a kernel loop in every process that hosts a saga-owning module, guarded by a fenced cache lease (the beat_lease mechanism, one holder per database). It is not tied to the outbox relay: the relay is off by default outside prod, and dev must drive sagas exactly as prod does. Each tick selects active instances whose next_attempt_at or deadline_at is due, FOR UPDATE SKIP LOCKED LIMIT n, and publishes a nudge for each one. It also stamps swept_at so it re-drives at most once per interval (SAGA_SWEEP_INTERVAL_SECONDS, an ops knob, default 5 s).

Durable timers are therefore a column on the instance row, with no new store. Celery countdown/ETA is rejected for timers, because those messages sit in worker memory and depend on broker visibility timeouts.

6. The step protocol (one UoW per step)¶

advance(nudge):                                     # consumer UoW, owner DB
  row ← SELECT … FOR UPDATE SKIP LOCKED             # none → another worker owns it; ack
  if row terminal or row.version ≠ nudge.expected_version
     or row.next_attempt_at > now: ack              # duplicate / stale / early: no-op
  if row.deadline_at ≤ now: → timeout transition (§7), commit
  bind_tenant(row.tenant_id)
  outcome ← run current step (command call with its key | local fn in this UoW)
  apply transition (§7): results, step_index ± 1, attempt, next_attempt_at,
                         status, version + 1; upsert saga_steps
  publish <module>.saga_advance.v1 (if more to do)   # outbox, same transaction
  commit                                             # pipeline of ADR-0005 §3

The instance row lock is held across one step call. That call is bounded by SERVICE_RPC_TIMEOUT_SECONDS (1 s default) plus one transport retry, or by the local contract pool's lock_timeout. A command runs on the contract pool or in another process, never on this connection. The provider never touches saga tables, so the lock cannot deadlock with the step. Holding the lock serializes everything that can move an instance: concurrent deliveries, the timer, an arriving reply event, and operator commands.

Why one transaction per step, not two. A "lease, commit, call, commit" variant would release the connection during the call, but it needs a lease with an expiry on the row, a reclaim rule for expired leases, and it reopens exactly the races the lock closes (timer vs reply vs operator). The hold is bounded (≈ 1–2 s worst case, one connection per in-flight step, steps run off the request path), so the MVP keeps the single transaction and measures it: the benchmarks suite gets a saga rung, and docs/operate/operator-reference.md gets sizing guidance (consumer concurrency × one connection). The two-transaction variant is reconsidered only if that measurement shows pool pressure.

How each failure is handled (each maps onto ADR-0005 §4/§4a):

7. Recovery rules¶

Classifying a step outcome. The class is decided by an explicit, closed table in sandloom.core.sagas.outcomes, never by the first digit of a status code. An unknown status is transient (retrying is the safe default: the command log makes a repeat harmless, and the deadline bounds it).

A module may narrow this per step (step(..., refuse_on={409}) for a provider whose domain conflict is a real refusal); it can never widen transient to refused for 408/425/429/5xx.

Resolve before compensating. A step whose last attempt was ambiguous (transient error or timeout) is not compensated blindly. The runtime first re-issues the forward call with the same key. The durable command log either replays the committed outcome, makes the call wait for the in-flight one, or runs the step now. The step is then known to be applied, or definitively refused, and only an applied step is compensated.

This closes the "compensation overtakes a late forward" hole. Seata calls it the empty-rollback / suspension problem of TCC. Sandloom closes it with the existing command log, without a provider-side tombstone. Compensations must still be idempotent and must accept "nothing to undo".

Timeout racing completion. Deadlines, replies and step outcomes all move the instance under the same row lock with a version bump. Whichever commits first wins, and the loser sees the new state and does nothing. A reply that arrives after a timeout has started compensation is recorded in saga_steps. The awaited step is then treated as applied, so its compensation runs. This is the phase-2 await_event case.

Parking sets status = parked with a reason. It increments sandloom_saga_parked_total{saga_type, reason} (alert on it) and writes an audit row (core.saga.parked, required=True). A parked instance never moves without an operator. "Retry forever" is not the default for business compensations, because an operator must see a compensation that cannot succeed. Kernel security effects stay on dead_letter=False consumers, as today.

8. Operations¶

The mutating commands are dry runs unless given --apply --reason. In prod they also need --confirm-production, the same convention as sandloom security …. Every mutation is audited (required=True).

Retry after the dedupe window. The window check (§9) bounds automatic retries, but an operator can retry a parked instance days later. By then the command-log record of the current step may have expired, and a blind re-call would run the step a second time. So saga_steps.first_attempt_at is compared with CONTRACT_COMMAND_DEDUPE_TTL_SECONDS (minus a safety margin of one sweep interval plus the RPC timeout):

• inside the window: retry re-drives as usual; the command log replays or waits for whatever already happened;
• outside the window, and the step's outcome is unknown (last attempt transient or ambiguous): retry refuses with exit code 1 and a hint. The operator must first establish the outcome with the provider (its own records, sandloom sagas show prints the step's operation key to search for) and record it with sandloom sagas resolve --step S --as applied|skipped. resolve therefore ships in the MVP, not in phase 2. After that the instance continues from a known state;
• outside the window, but the outcome is known (the last attempt was a definitive refusal, or the step was never attempted): retry proceeds — a re-call cannot double an effect that never happened.

The same rule applies to an instance un-parked from unknown_definition after a deploy: the runtime refuses to re-call a step whose record may have expired and parks it with outcome_unknown_after_ttl instead.

Metrics:

• sandloom_saga_instances{saga_type,status}, a gauge from a bounded query on the sweep tick;
• sandloom_saga_step_duration_seconds;
• sandloom_saga_step_attempts_total{outcome};
• sandloom_saga_parked_total.

/diagnostics/core lists the registered definitions and versions. sandloom doctor warns about parked instances and instances older than their deadline. An owner-console page comes later.

9. Effect-model amendment (ADR-0005 §4, one new row; no new record)¶

The instance row answers "where is the process". It does not answer "did this effect already run" for any effect outside its own transaction. That question is always answered by the command log. Local steps are protected by the transition commit itself, which is ADR-0005 §6.1's "state on the effect's own row". The saga runtime therefore needs no new dedupe store.

New guarantee window (sandloom.core.guarantee_windows, deterministic): saga_horizon_beyond_command_dedupe. For every command step of every registered definition, the longest span from the step's first attempt to its last possible retry or resolve must fit the dedupe TTL:

(step deadline, else saga deadline) + compensation_deadline
    ≤ CONTRACT_COMMAND_DEDUPE_TTL_SECONDS

Otherwise a late retry or resolve could re-execute a step whose record expired. Waiting in an await_event step holds no command key open, so it does not count. Production refuses this with GuaranteeWindowError; other environments warn. It can be waived through SANDLOOM_ACCEPTED_RISKS (ADR-0006), but only as a recorded risk.

10. Placement in the kernel layers (ADR-0007)¶

sagas is added to the kernel-layers contract next to module_api. Because module_api must not import sagas, the acyclic-siblings contract holds.

11. Versioning in-flight instances¶

An instance is pinned to definition_version. A new version is a new definition registered alongside the old one, for as long as old instances run. sandloom sagas list --type T --version V shows them, and a later release removes the old one. If the advance finds an instance whose (type, version) is not registered, it parks the instance with unknown_definition. The consumer never crashes. sandloom doctor and the startup task report such instances.

Editing a registered version in place is not detected at runtime. Instead, step names, kinds and called contracts are pinned per version in the module's committed contracts.snapshot.json, under a sagas key. A stale snapshot fails the suite, the same mechanism as contract snapshots, and it is deterministic.

12. What sagas do not provide¶

• Isolation (ACD, no I). Intermediate states are visible. Modules apply Richardson's countermeasures explicitly. The most common one is a semantic lock, a status column such as provisioning that routes refuse until the saga completes. The others are commutative updates, reread value (version re-check, ADR-0003 G1), by value and version file. The framework documents them and does not automate them.
• Parallel steps, nested or child sagas, branching, loops, human tasks. A definition is linear. Branching belongs in a step's arguments or in two sagas.
• Ordering across sagas, and any atomicity with external systems. A PSP call is a local step that keys its own idempotency on the derived key, as billing intents do today.

13. Testing¶

The saga_harness fixture (§14) drives the real protocol synchronously, with fault injection (fail_step, crash_after, deliver_twice), an injectable clock and the assert_compensations_idempotent / assert_resolves_before_compensating checks. The kernel's own suite adds a fault test for every row of the §6 and §7 tables and for the post-TTL retry rule (§8).

A split-conformance script case runs the worked example in the monolith and in split deployments, and requires the same terminal state.

14. Developer experience¶

A saga is only worth having if writing, testing and debugging one is easier than hand-rolling it. The DX rules below are part of the decision, not polish.

Authoring — typed, declarative, named functions.

# sandloom/<module>/sagas.py — the one place a module's sagas live
from datetime import timedelta

from sandloom.core.module_api import Saga, SagaContext, local_step, step

from .schemas.sagas import EnrollInput, Enrollment


def charge_args(c: SagaContext[EnrollInput]) -> dict[str, object]:
    return {"customer_id": c.input.customer_id, "amount": c.input.price}


def refund_args(c: SagaContext[EnrollInput]) -> dict[str, object]:
    return {"customer_id": c.input.customer_id, "amount": c.input.price,
            "of_operation_key": c.key("charge")}


ENROLL = Saga[EnrollInput](
    "enroll", version=1, input=EnrollInput,
    deadline=timedelta(minutes=30),
    steps=[
        step("charge", call="billing.credits.v1.spend", args=charge_args,
             compensate="billing.credits.v1.reverse", compensate_args=refund_args),
        local_step("grant_access", kind="pivot", apply=Enrollment.activate),
    ],
)

• Saga[InputT] and SagaContext[InputT] are generic, so c.input is the module's own pydantic model with IDE completion and mypy checking. c.result("charge", SpendResult) returns a step's decoded result typed.
• Arguments are named module-level functions, not lambdas: they show up in tracebacks and coverage, can be unit-tested, and the docs and the scaffold use them. (Lambdas work; the guide recommends against them.)
• Keyword-only builders with defaults: kind defaults to compensatable before the pivot and retriable after it, so a typical step names only call, args and its compensation.
• call= stays a string ("<contract>.<method>"), like every cross-module reference in Sandloom — modules never import each other.

Fail at startup with a fix, never at step 3 in production. Every registration error names the saga, the step and the remedy, in the same fix: style as capability errors:

SagaDefinitionError: tenancy.org_setup step "assign_owner_role":
  billing.credits.v1.grant has no parameter "tennant_id" (did you mean "tenant_id"?)
  fix: return the provider's parameter names from grant_args()

Checked deterministically at registration: the structure rules of §1; the called method exists and is a command (command=True, with an idempotency_key); and the parameter names the args function returns match the method's signature. The names are read from the provider's describe document — the local provider at registration, a remote one from the module's committed contract snapshot (and again at /ready by the existing compat check). The check calls each args function once with a synthetic SagaContext built from the input model's schema; a function that cannot run on synthetic input opts out per step (check_args=False) and is then covered only by tests.

Starting from an endpoint — one recommended HTTP shape. A saga is asynchronous, so the route that starts one answers 202 Accepted with the instance's status URL; saga_status() gives the read side:

@router.post("/enrollments", status_code=202,
             dependencies=[Depends(operation_class("W"))])
async def enroll(body: EnrollIn, db: DbSession, user=Depends(get_current_user_id),
                 _=Depends(idempotency_guard(scope="enroll", durable=True))):
    handle = await start_saga(ENROLL, saga_id=body.enrollment_id, input=...)
    return {"status": handle.status, "status_url": f"/enrollments/{handle.saga_id}"}

start_saga returns a SagaHandle(saga_id, status, created); a repeat with the same input returns the existing handle (created=False), so the route needs no special idempotency code of its own beyond the guard.

Scaffold. uv run sandloom add saga <module> <name> generates, in the module's own package:

• sagas.py with a two-step definition and its argument functions;
• schemas/sagas.py with the input model;
• the register_saga line in setup;
• tests/test_saga_<name>.py with the happy path, a refusal before the pivot, a crash after the provider committed, a timeout that resolves before compensating, and assert_compensations_idempotent — all green on the generated code;
• a commented 202 route.

sandloom new apps get the same command. The generated code passes sandloom check and the operation-class test.

Testing — synchronous, deterministic, no sleeps. The saga_harness fixture (from the sandloom.core.testing plugin) drives the real protocol in the test's process:

async def test_refund_when_access_cannot_be_granted(saga_harness):
    saga_harness.fail_step("grant_access", error="refused")
    run = await saga_harness.run(ENROLL, saga_id=..., input=...)
    assert run.status == "compensated"
    assert run.calls == ["charge:do", "grant_access:do", "charge:undo"]
    run.assert_keys_derived()            # every call carried its derived key

• run() executes the steps synchronously through the real advance path (row lock, transition, nudges) and returns a SagaRun with the status, the ordered calls, the keys, the persisted results and the step table;
• fail_step(name, times=n, error="transient"|"refused"|"timeout"), crash_after(step) (the provider committed, our transition did not), deliver_twice(), and clock.advance(timedelta) for deadlines and backoff;
• assert_compensations_idempotent(definition) and assert_resolves_before_compensating(definition) are one-liners a module's suite is expected to contain (the scaffold writes them);
• providers are the real ones when co-loaded, or fakes registered with saga_harness.fake_contract("billing.credits.v1", spend=..., reverse=...) that still go through the command-log dedupe, so key behaviour is real.

Seeing what a saga did. sandloom sagas show <type> <id> prints a timeline, not a table dump:

tenancy.org_setup  4f1c…  status=parked (compensation_transient)  v1  tenant=9a2e…
  started 10:02:11 by POST /orgs  trace=7b3d…
  ✓ grant_trial_credits  do    10:02:11  1 attempt   42 ms  key saga:…:grant_trial_credits:do
  ✗ assign_owner_role    do    10:02:12  refused 404 user_inactive
  ↺ grant_trial_credits  undo  10:02:12 → 11:02:12  14 attempts  last: 503 billing unavailable
  why parked: the compensation kept failing past compensation_deadline (1h)
  next:       fix billing, then `sandloom sagas retry tenancy.org_setup 4f1c… --apply --reason …`

• every park reason has a fixed, documented remediation that show prints;
• each step runs in its own trace span, linked to the trace that started the saga (the start's trace_id is stored on the instance), and every log line of a step carries saga_type, saga_id, step, direction and attempt;
• sandloom sagas list defaults to the actionable view (parked, overdue, compensating) with ages; --all shows everything;
• phase 2 adds the same timeline as an owner-console page.

Local dev behaves like prod. The timer sweep runs in every process hosting a saga module (§5), the advance consumer runs on the dev bus, and no "inline mode" exists that would skip the real path. make_headers + client + saga_harness are enough for any test.

Docs and agent rules. docs/modules/between/sagas.md opens with a decision tree (one module → plain UoW; fan-out without undo → choreography; several modules with undo → saga), then the countermeasure catalogue and recipes. AGENTS.md gets a short block of the rules an author (human or AI) must follow: sagas live in sagas.py; args are pure named functions of input and results; every compensatable step names a compensation command that accepts "nothing to undo"; retriable steps never refuse; a semantic lock guards user-visible intermediate states; a definition change is a new version.

Consequences¶

• Cross-module business processes get one shape:
• central state;
• derived keys;
• deadlines;
• parking;
• an operator CLI.

There are no new infrastructure, dedupe stores or post-commit variants. - There is no cost on requests that do not start a saga. Starting one adds an instance INSERT and an outbox row to the caller's flush. Each step costs 4 statements in its own UoW (lock, upsert step, update instance, outbox insert — no inbox claim, §5), plus the step's command, which already pays its command-log claim and update. The step also pays one Streams hop. - The runtime holds a pooled connection and an instance row lock for one step call, about 1–2 s worst case (§6 explains why that is kept). Sizing guidance goes into docs/operate/operator-reference.md; a saga rung joins the benchmarks suite. - Module authors must write compensations that are idempotent and accept "nothing to undo". They must also make retriable steps unable to refuse. The structure checks and the test seam make the gap visible, not impossible. - register_saga adds consumed-contract requirements, so a module owning a saga does not boot without its providers, locally or through bindings. This is the intended, explicit failure (test_module_boots_alone pins it).

Phased plan¶

Amendments (implementation of phases 0 and 1, 2026-10-25)¶

The phase-1 build follows the decision above. Where it had to differ, the reason is recorded here; this section supersedes the text it names.

• A1 — statuses (§2). An instance is running, compensating, completed, compensated or parked. failed is not produced: every outcome that is neither a success nor a finished compensation parks, so an operator sees it (§7). A compensated saga is the business "failure".
• A2 — instance columns (§2). Besides the listed columns the row carries resolving (the current step must be re-issued before compensating), last_outcome and step_first_attempt_at (the post-TTL guard reads them without an extra saga_steps read — the transition stays at four statements), step_deadline_at / compensation_deadline_at, applied (the forward steps whose effect exists; compensation walks only those, so a step resolved as skipped is never undone), park_reason / parked_from (the direction retry returns to), trace_id / started_by (§14) and swept_at (§5). saga_steps also carries saga_type and tenant_id (extraction filter, RLS), the step's operation_key, outcome and last_duration_ms.
• A3 — the nudge carries the tenant (§5). {instance_id, expected_version, tenant_id}: under tenant_rls the consumer must bind the instance's tenant BEFORE the row lock, or RLS hides the row. The value authorizes nothing — a wrong tenant only makes the row invisible (a no-op).
• A4 — deadline before "early" (§6). The advance checks the deadline first, then next_attempt_at > now: a deadline must fire while a retry is backing off. A transient failure publishes no nudge; the sweep re-drives the instance when its backoff (or a deadline) is due, so retry latency is max(backoff, SAGA_SWEEP_INTERVAL_SECONDS).
• A5 — the sweep's transactions (§5). The claim (FOR UPDATE SKIP LOCKED + the swept_at stamp) and the nudges are two transactions: the system session is not a Unit of Work, so the nudges go through the outbox of an ordinary UoW right after (a lost nudge is re-driven next interval). An idle tick on a tenant_rls database rolls its system session back, so it leaves no sandloom.system_db.session audit row (only its log lines). The sweep also refreshes the sandloom_saga_instances gauge about once a minute and purges terminal instances older than SAGA_RETENTION_DAYS about once an hour (parked instances are kept). Without a cache that supports cas the sweep runs unleased (safe, only redundant). The lease is sandloom.core.beat_lease.FencedLease, the beat lease's state machine made async and shared.
• A6 — the system channel is injected (§2, §10). sandloom.core.sagas sits in the contracts & RPC layer and may not import a concrete driver (test_concrete_drivers_imported_only_in_composition_layer), so create_app passes sandloom.core.drivers.system_db's system_session to install_sagas(core, system_channel=...). Startup refuses a saga database with tenant_rls and no system channel. The core migration enables system_access only when the sandloom_system role exists (an RLS deployment creates it first).
• A7 — remote providers need a snapshot (§1, §14). The runtime injects a command's idempotency argument by name, so it must know the name for a remote provider at startup: a call to a contract that is neither local nor covered by a snapshot (contracts.snapshot.json in the provider's package or CONTRACT_SNAPSHOT_PATHS) is a SagaDefinitionError with a fix, not a deferred check.
• A8 — classification details (§7). A ServiceNotFound that is not the 501 deployment gap is misconfigured. A local step refuses by raising an exception whose http_status is in the refused set; any other exception is transient and logged with its traceback. An args function that raises parks the instance as misconfigured (code saga_args_failed): a deterministic failure never heals by retrying.
• A9 — operator semantics (§8). retry also grants a fresh window (deadline_at for a forward instance, compensation_deadline_at for a backward one) — otherwise an instance parked by a deadline would park again on the next tick; the post-TTL rule still guards the re-call. resolve applies to a parked instance's current step: forward applied → the next step; forward skipped → the next step without the effect (never compensated); for a compensation (or the ambiguous step of a compensating instance) applied / skipped record the outcome and the saga continues from there. resolve continues the instance at once (no separate retry).
• A10 — unknown versions. Besides parking at advance time, a non-critical startup task logs the (type, version) of active instances that no registered definition covers, and sandloom doctor warns about them.
• A11 — per-version snapshot (§11) is phase 2, as the phased plan says; phase 1 does not pin step names under a sagas key yet.
• A12 — test seams (§14). saga_harness.drive() acts as the timer sweep: with no nudge pending it advances its clock to the instance's next due time (jitter fixed at 0.5). crash_before(step) joins crash_after; the two one-liners take input= and tenant_id= (a real provider needs real input and, when tenant-scoped, a tenant).
• A13 — the worked example is a new app, examples/academy (payments, notifications, enrollment); its saga reserves the seat in a local compensatable step first, so a refused charge cancels the seat.

Measurements (Consequences)¶

benchmarks/ladder/saga_rung.py, statements pinned by benchmarks/tests/test_saga_rung.py; wall times from one run on a shared dev machine (load ≈ 3–4, n = 1000), indicative only:

The "4 statements per step" of the Consequences holds; the fifth is the existing outbox mark-delivered of the nudge. The single-transaction step (§6, question 6) stays: the hold is one step call, measured in milliseconds.

Alternatives rejected¶

• Holding the transaction across the whole saga, or 2PC/XA: forbidden by ADR-0005 and by the split deployment model.
• Event-sourced saga state (replaying a step journal, as Temporal, Azure Durable Functions and Restate do): it imposes determinism rules on module code and a journal store, which is a new dedupe and step store, for no gain over a version-guarded row with persisted results.
• Timers as delayed outbox rows (deliver_at, review C2 #12): viable, and still useful for plain delayed events. For sagas the deadline belongs on the instance row, because a cancelled timer must not need a delete.
• Celery ETA/countdown for timers: these messages are held in worker memory and are subject to broker visibility timeouts and redelivery quirks.
• A DLQ for stuck sagas: a parked status on the instance is visible, queryable and audited, while a DLQ entry is not.

Worked example placement¶

The research document's org-setup example needs four new commands in the bundled modules, an org_owner platform role in identity (tenancy's membership already records the owner), and makes tenancy consume billing and identity — which would break test_module_boots_alone for a demo. Granting trial credits on org creation is also a plain fan-out that choreography handles (billing reacts to tenancy.organization_created.v1).

So the MVP's worked example lives in an example app (a course-enrollment flow in the shape of §14: charge → grant access as the pivot → notify as a retriable step), with its own modules and contracts, and runs in the split conformance matrix. The bundled modules gain saga-ready commands only when a real flow needs them.

Open questions (owner) — with the recommended answers¶

1. Who owns org onboarding? Recommended: nobody yet. Trial credits are choreography; a real onboarding saga, when one appears, belongs to a separate module started by tenancy.organization_created.v1, so tenancy stays free of billing and identity.
2. Timer sweep in the relay leader or its own loop? Decided in §5: its own lease-guarded loop in every process hosting a saga module, so dev and prod drive sagas the same way.
3. Default compensation_deadline? 1 h, as an ops setting (SAGA_COMPENSATION_DEADLINE_SECONDS), overridable per definition; the window check covers it.
4. Durable inbox on the advance consumer? Decided in §5: no. Advances are versioned nudges; the row lock and version are the guard.
5. Billing's intent request path? Stays hand-rolled permanently (its pivot is the request's own commit and its result is a bearer secret); only the reconciler is a phase-3 candidate.
6. One transaction per step? Decided in §6: yes, measured by a benchmark rung; the two-transaction variant only if the measurement demands it.
7. When to build phase 1? Decided: build now (owner, 2026-10-25). Phase 0 and the phase-1 MVP ship together; the worked example is the course-enrollment flow the Tutor product needs (payment → course access → notifications).

Related reading¶

• Add a saga
• Saga concepts