ADR-0004: Internal RPC request authentication with per-pair HMAC keys¶

• Status: Accepted (draft — awaiting the maintainers' sign-off). Amended 2026-10-25: the pair MAC is the only internal-RPC authentication; RPC_AUTH_MODE and RPC_RESPONSE_BINDING are removed — setting either fails startup unless it is set to its inert value (see "Amendment (2026-10-25)"). The sections below that compare modes describe the migration that the amendment completed
• Date: 2026-10-21
• Deciders: Sandloom maintainers
• Supersedes: — (extends the service-identity model of #57 and the keyed response binding of docs/SERVICE_RPC.md → "Response binding")

Context¶

In RPC_AUTH_MODE=service_identity every internal RPC call (and every describe / readiness probe) is signed by the caller with its own Ed25519 key (sandloom.core.rpc_identity); the provider verifies the signature with the caller's public key from RPC_SERVICE_KEYS and authorizes the caller against RPC_ALLOWED_CALLS. The native-acceleration research (docs/design/native-acceleration-research-2026-10-21.md, R2, branch claude/confident-sammet-a122dc) measured the cost:

A Rust verifier (ed25519-dalek) would only buy 2.1× at the price of a native crypto dependency. Every service already holds a dedicated static X25519 key for the keyed response binding (RPC_DH_PRIVATE_KEY, public halves in RPC_SERVICE_DH_KEYS), and both ends of every pair can already derive a shared per-pair secret. The 2026-10-21 RPC review also found:

1. No audience. The Ed25519 signature covers method, target, body digest and context, but not the provider. A request captured on the wire can be replayed to another service that serves the same contract (two replicas sets, a staging/prod pair sharing keys, a contract hosted by two services) within the 300 s context window.
2. Identity failures surface as 401/403 to end users. rpc_identity_* and rpc_caller_forbidden were not in the caller's "unavailable" code table, so a provider's 401/403 became a ContractCallError with the same status — a deployment misconfiguration answered the end user "unauthorized".
3. RPC_RESPONSE_BINDING=strict accepts alg=token bindings without a pinned service_id (token deployments only; see "Follow-ups").

Decision¶

Add a request authenticator that is a MAC under a per-pair, directional, HKDF-derived key from the services' existing X25519 keys, behind two new RPC_AUTH_MODE values, and make it the target mode of split deployments:

The command signature is the mitigation of key-compromise impersonation (see "Key-compromise impersonation" below): the pair key is symmetric, so whoever holds a provider's X25519 private key can compute it for every caller. Effects therefore also need a proof only the caller can produce; reads — the hot path — stay MAC-only.

Implementation: sandloom.core.rpc_pair_mac (key schedule, canonical encoding, MAC, header, replay guard, configuration checks); wiring in sandloom.core.rpc. All primitives come from cryptography (X25519, HKDF) and the stdlib (hmac, hashlib); there is no custom crypto code.

Key schedule¶

For a caller C (X25519 key c, kid ck) and a provider P (key p, kid pk), both sides compute

Z      = X25519(c, P_pub) = X25519(p, C_pub)          # cryptography's exchange
         reject: exchange error (low-order / non-canonical point) or Z == 0^32
K_req  = HKDF-SHA256(IKM = Z,
                     salt = "sandloom-rpc-pair-mac/v1",
                     info = LP("sandloom-rpc-request-key", "1", "caller->provider",
                               C, ck, P, pk),
                     L = 32)

LP(...) is the canonical length-prefixed encoding below. The identities are in role order (caller first, provider second) with an explicit direction label, which is canonical: both sides know which role each id plays, and K_req(C→P) ≠ K_req(P→C) (the reverse direction is a different key — a provider can never turn a request it received into one it sends). Keys per pair and direction:

Distinct salts and infos make the keys independent HKDF outputs of the same Z: neither key ever MACs the other direction's messages, and no X25519 key is used for anything but key agreement (the Ed25519 signing key stays separate; startup refuses a DH key whose 32 bytes equal the signing seed). No third "response" key is introduced: the provider→caller direction is already served, domain-separated, by K_resp. Derived keys are cached as prepared HMAC contexts keyed by (own private PEM, peer public PEM, info); the info names both kids, so a rotated key is a new cache entry and the cache is bounded (cleared when full) and cleared explicitly by clear_key_cache().

Request MAC¶

Headers (one call attempt):

X-SANDLOOM-RPC-Nonce:   32 lowercase hex = 128 random bits (os.urandom), fresh per attempt
X-SANDLOOM-RPC-Context: base64url(JSON {tenant_id, idk, iat[, aud, svc, kid, dhk]})
X-SANDLOOM-RPC-Auth:    v=1;caller=<C>;ck=<ck>;aud=<P>;pk=<pk>;mac=<base64url(32-byte tag)>

tag = HMAC-SHA256(K_req(C→P),
        LP("sandloom-rpc-request-mac", "1", C, ck, P, pk,
           HTTP method, target "<contract>/<method>" (GET: "<contract>"),
           nonce, raw context bytes, SHA-256(raw body)))
LP(x1..xn) = for each xi: uint32_be(len(xi)) || xi        (fixed arity n)

• Unambiguous. A fixed number of fields, each with a 4-byte length prefix: the encoding is injective (it has a decoder; the property tests check decode(encode(x)) == x and that no two distinct inputs collide).
• Covers everything the provider acts on. The provider authorizes the caller id from the header (the one the key was derived from), serves the target from the URL, binds tenant / idempotency key / issue time from the context bytes, and executes the body — all inside the tag. The context is covered as the exact bytes the provider parses (it is parsed only after the tag verified), so no JSON canonicalisation is needed; duplicate JSON keys are refused anyway. Future context fields (actor, deadline) are covered automatically.
• Audience by construction. P and pk are in the key and in the MAC input; the provider requires aud == RPC_SERVICE_ID and pk ∈ its own kids. A request for P never verifies at Q (Q derives K_req(C→Q) ≠ K_req(C→P)).
• The response covers the request MAC. The keyed response binding's MAC input includes the request's X-SANDLOOM-RPC-Auth (with the signature headers), so a pair-keyed response verifies only for the exact request it answers — never for another request re-labelled with its nonce.
• Order (S5 kept). The provider reads the raw body under the body cap, verifies the tag over the raw bytes, and only then decodes the context and the MessagePack body. Comparison is hmac.compare_digest on the full 32-byte tag; a tag that does not decode to exactly 32 bytes (truncated, padded, non-base64url) is rejected before any comparison.

Replay¶

• Freshness. |now − iat| ≤ RPC_AUTH_MAX_SKEW_SECONDS (default 60 s, 5…300) in the MAC modes; iat is integer seconds from the caller's clock. The rule is symmetric: a caller up to the skew ahead or behind is accepted; beyond it the call fails rpc_auth_expired (→ 503 at the caller; alert on sandloom_rpc_auth_rejected_total{reason="expired"} — a drifting node). The Ed25519 path keeps its 300 s window.
• Nonce replay guard. After the tag verified (only authenticated callers can insert entries), the provider records (caller, nonce) until the request could no longer pass the freshness check on its own clock (iat + skew − now + 2 s, at most 2·skew + 2). A repeat inside that window is 401 rpc_auth_replayed. The nonce guard runs only AFTER the caller is authorized (RPC_ALLOWED_CALLS): a key holder without an allowlist entry cannot fill it.
• Layer 1, always: a per-process bounded table (RPC_REPLAY_CACHE_MAX_ENTRIES, default 200 000 ≈ 2·60 s at 1 600 calls/s per process, ≈ 40 MB worst case), of which one caller may hold at most RPC_REPLAY_CACHE_PER_CALLER_MAX_ENTRIES (default 100 000 = half the table). Sizing: a nonce lives ≈ skew + 2 s (worst case 2·skew + 2), so a caller needs at least its peak calls/s per provider process × (2·skew + 2) — the default carries ≈ 830 calls/s per caller per process at skew = 60; alert on sandloom_rpc_auth_rejected_total{reason="caller_quota"}: an allowed peer flooding it (e.g. iat = now + skew at full rate) is refused itself (503, reason caller_quota), the other callers are not. When the table is full of live entries the provider fails closed (503 rpc_replay_unavailable) instead of evicting a live nonce; it is never rebuilt on reconfiguration.
• Layer 2, RPC_REPLAY_GUARD=commands (default) or all: an atomic SET NX with that TTL on the security cache (SECURITY_CACHE_INSTANCE, shared by every replica and worker) for command calls (commands) or every call (all); local disables it. The shared key is a SHA-256 of LP(provider, caller, nonce) and lives a constant 3·skew s — not derived from one replica's clock: a request is accepted by replicas over at most 2·skew + d of real time, d being the clock difference between two replicas, and the deployment rule (the same one callers meet) is d ≤ skew — a deployment requirement (deploy/README.md), watched by sandloom_rpc_auth_clock_skew_total{caller} (an accepted request off by more than half the window; one log line a minute per caller). A cache outage fails closed in production (503 rpc_replay_unavailable, the caller retries) and open outside it (logged, counted; a startup warning says so). RPC_REPLAY_GUARD=local is refused in production.
• Multi-replica limit. With local (or for reads under commands) a replay that lands on another worker process or replica within the window is not detected. That is accepted for reads (no effect; an on-path party can read the response anyway without TLS) and is why commands default to the shared layer — on top of the durable command dedupe (contract_command_log), which already turns a replayed command with the same idempotency key into a replay of its stored result.
• Transport retry. The proxy's single retry uses a new nonce and a new tag, so it is never mistaken for a replay.

Downgrade¶

• A provider in service_mac accepts only X-SANDLOOM-RPC-Auth. A request with only an Ed25519 signature gets 401 rpc_auth_required (a caller still on service_identity: an operator error worth naming); a request with only X-Internal-Token, or with no credential at all, keeps the hidden 404 of service_identity. Ed25519 signatures are never verified in this mode.
• In service_mac_dual a present X-SANDLOOM-RPC-Auth is the only credential checked: a bad MAC is 401 even when a valid Ed25519 signature rides along (no fallback an attacker could steer). A dual caller marks the context it MACs ("mac": 1, inside the signed context); a MAC-mode provider refuses such a context on the Ed25519 path (401 rpc_auth_required), so stripping the MAC cannot skip the nonce guard. Only a service_identity caller's request (no flag) uses the Ed25519 path — its context carries aud when the binding is pinned (see below). A service_identity provider (mid-rollout) ignores the flag.
• A request carrying any of X-SANDLOOM-RPC-Auth, X-SANDLOOM-RPC-Nonce, X-SANDLOOM-RPC-Context, the signature headers more than once is refused (401) and its response is never bound: the verifier and the response binder can never read two different nonces.
• The caller never falls back: in a MAC mode a call whose pair key cannot be derived (binding without service_id, provider without published X25519 key, low-order key) fails 503 rpc_auth_unavailable; startup refuses that configuration (below), so this only happens on a live misconfiguration.
• Unknown header versions (v≠1), unknown or duplicated header parameters, and unknown caller / provider kids are 401 rpc_auth_invalid.
• RPC_ALLOWED_CALLS and deny-by-default are unchanged; the caller id the allowlist is consulted with is the verified caller of the MAC header.

Key-compromise impersonation (KCI) and the command signature¶

Static-static ECDH is symmetric: X25519(c, P) = X25519(p, C). Whoever holds a provider P's X25519 private key — a leaked secret, a stolen config, a retired key kept in RPC_DH_EXTRA_PRIVATE_KEYS; code execution is not needed — derives K_req(C→P) for every caller C and can present requests to P as the most privileged caller (RPC_ALLOWED_CALLS on P is then moot). With Ed25519, the same leak gave only P's outbound rights. A provider's X25519 key is therefore an "all inbound rights" secret, on top of being its callers'-response key.

Mitigation (decided):

• RPC_COMMANDS_REQUIRE_SIGNATURE (auto = on in prod; on; all; off, refused under multi_tenant_strict): a command call (any method that is not a declared read, and every not-yet-described method) additionally carries the caller's Ed25519 signature over the same context, target and body (X-SANDLOOM-RPC-Service-Signature, the service_identity proof). The signed context carries the attempt's nonce (n; the caller re-signs per transport attempt), so the signature binds the nonce the MAC and the replay guard use: a captured signed command cannot be re-MACed under a fresh nonce by a holder of the provider key. The provider verifies it with the caller's PUBLIC key (RPC_SERVICE_KEYS) after the MAC and the allowlist and requires the signer to be the MAC's caller and n to be the request's nonce (401 rpc_auth_signature_required / rpc_auth_invalid). The provider never holds a caller's Ed25519 private key, so a leaked provider X25519 key can forge reads at most, never effects. A caller in service_mac signs every call that may write whenever it holds RPC_PRIVATE_KEY; startup refuses a calling service without one while the requirement is on.
• all: every call is signed and checked — reads become KCI-resistant too, at the Ed25519 price on every call (the ≈ 126 µs this ADR removes).
• Residual risk (documented, accepted in the default): with P's X25519 key an attacker can issue reads to P as any caller, naming any tenant — i.e. read whatever P's read contracts expose. Treat a provider's X25519 key like its database credentials; rotate it at once on suspicion; deployments that need KCI-resistant reads set all.
• Retired keys widen the window: a kid kept in RPC_DH_EXTRA_PRIVATE_KEYS but no longer published is reported at startup (warning; an error under multi_tenant_strict). The rotation overlap is bounded (below).

Alternatives considered for KCI: an interactive authenticated key exchange with ephemeral keys on both sides (Noise KK/IK style) resists KCI for every call but needs per-connection handshakes and session state across a pooled HTTP client — a protocol of its own, not a MAC; per-caller provider keys only shrink the blast radius per leaked key, they do not remove it (all of a provider's keys live in the same secret store); signing every call is the all option. Signatures on effects only keep the read path cheap and make the worst case of a provider-key leak "reads", which the decision accepts.

Ed25519 audience (closes review finding 1 for the migration window)¶

In service_identity, dual and service_mac_dual a caller whose binding pins the provider (service_id) adds aud to its signed context; a provider that finds aud ≠ its own id answers 401. Old providers ignore the extra field, old callers send none (accepted) — so it is rolling-upgrade safe.

Key rotation¶

• RPC_SERVICE_DH_KEYS[svc] may list several kids. Select the addressed kid explicitly with RPC_SERVICE_DH_CURRENT_KIDS={"<svc>": "<kid>"}; JSON object insertion order is not a rotation protocol. Existing configurations without an explicit selection use the lexicographically greatest published kid, which is deterministic under JSON reordering but does not infer which arbitrary kid is newest. Set the explicit map before rotating.
• A provider accepts requests addressed to its current kid (RPC_DH_KEY_ID) and to any kid in RPC_DH_EXTRA_PRIVATE_KEYS ({"<kid>": "<PEM>"}, SECRETS_DIR file rpc_dh_extra_private_keys).
• Rotating provider P (no failed call at any step): 1) pin the existing kid1 in RPC_SERVICE_DH_CURRENT_KIDS[P], generate kid2, add its private key to P's RPC_DH_EXTRA_PRIVATE_KEYS; 2) publish kid2 in RPC_SERVICE_DH_KEYS[P] everywhere, then set RPC_SERVICE_DH_CURRENT_KIDS[P]=kid2 (callers start addressing kid2; P accepts both); 3) make kid2 P's RPC_DH_PRIVATE_KEY/RPC_DH_KEY_ID and move kid1 to the extras; 4) remove kid1 from RPC_SERVICE_DH_KEYS everywhere; 5) after RPC_AUTH_MAX_SKEW_SECONDS, drop kid1 from P. A caller's own rotation needs only steps 2–5 (its kid travels in the header).
• Bounded overlap. Steps 1–5 take one rolling deploy each; the whole rotation should complete within 24 h, and step 5 must follow step 4 within one deploy: a kid still in RPC_DH_EXTRA_PRIVATE_KEYS after it left RPC_SERVICE_DH_KEYS keeps deriving every caller's key towards P (KCI, above) and is reported at every start (multi_tenant_strict: refused). sandloom keys generate --purpose rpc-dh --service <id> emits one X25519 pair.
• Startup checks (Settings validation, every MAC mode): own X25519 key present and its published half matches; every extra key parses, has a distinct kid, and (if published) matches; every binding without its own internal_token is pinned (service_id) to a service with published X25519 keys and each pair key derives (rejects low-order / all-zero points at startup); every explicit current kid is published and a selected local kid has its private key installed; no two services publish the same X25519 public key; the X25519 private key is not the Ed25519 signing seed; every published key is an X25519 key (the existing parse). service_mac_dual additionally needs the Ed25519 identity of service_identity.

Errors¶

• Rejection log lines are rate-limited (one per reason and caller per minute); the counter keeps the exact rate.
• Provider: 401 rpc_auth_invalid for every cryptographic / parse failure (malformed header, unknown kid, wrong audience, wrong key, bad tag, unknown version) — an unauthenticated party learns nothing about which check failed; 401 rpc_auth_expired / rpc_auth_replayed are only reachable after the tag verified; 401 rpc_auth_required (downgrade); 403 rpc_caller_forbidden; 503 rpc_replay_unavailable. The detailed reason is a fixed vocabulary in sandloom_rpc_auth_rejected_total{reason} and a log line with ids and kids only — never a key, tag, or derived secret.
• Caller: every auth code (rpc_auth_*, rpc_identity_*, rpc_caller_forbidden, rpc_replay_unavailable) maps to 503 RemoteServiceUnavailable with its code (review finding 2); a bare 401/403 without an envelope already did.

Threat model¶

Assets: the right to invoke contract methods of a provider as a given caller service (and to name a tenant in its context); integrity of arguments and results. Out of scope: confidentiality (TLS/mTLS), compromise of the provider process itself (it runs the contract), denial of service by an on-path party.

What changes for the worse, stated plainly:

1. No third-party non-repudiation. An Ed25519 signature proves to anyone holding C's public key that C produced the request; a MAC proves it only to P (both C and P can compute it). Nothing in Sandloom uses requests as evidence for a third party (audit records are written by the provider about an authenticated caller), so this property is not relied upon.
2. A provider can forge requests from a caller — to itself only. P can compute K_req(C→P), so P could fabricate "C called me". P already executes whatever it likes; it cannot use the key towards any other service.
3. The X25519 key becomes a signing-grade secret (rows "Compromise of a static X25519 private key" above): before, its leak allowed forging responses of that service; now it also allows forging that service's read requests to every peer and — for a provider — any caller's read requests to it (KCI). Rotation is per service and needs no other service to change keys.
4. Key-compromise impersonation of reads (section above): accepted in the default; commands are protected by the caller's signature, and RPC_COMMANDS_REQUIRE_SIGNATURE=all removes it at the Ed25519 price.

What improves:

1. Audience binding by construction (review finding 1 closed in service_mac; in the Ed25519 modes for pinned bindings).
2. Replay protection beyond a time window (nonce guard).
3. Cost. Per read call ≈ 126 µs (sign + verify) → ≈ 11 µs on both sides together, replay guard and strict context decode included; a command keeps the Ed25519 cost (+11 % for the MAC) — see "Measurements".
4. Error mapping (review finding 2 closed in every mode).

Measurements¶

benchmarks/native/bench_rpc_auth.py — one call's authentication work per side, through the real Sandloom functions (context built outside the op in both modes, ~100-byte msgpack body). Wall clock, CPython 3.12.10, arm64, cases interleaved round-robin, n = 5000 × 9 reps, median (min); benchmarks/native/results/rpc_auth_wall_2026-10-21.txt:

Where the remaining µs go (cProfile): the strict context decode (duplicate- key check, 2.6 µs), header regex, key-cache lookups; the HMAC itself is ≈ 1 µs (the pair's fixed MAC-input prefix is absorbed into the prepared HMAC context once per key). The shared nonce layer adds one security-cache SET NX round trip for commands only (RPC_REPLAY_GUARD=commands), which already pay a database transaction.

Cachegrind (instruction counts — the stand's evidence, insensitive to the shared machine's load): CG_K=1000 benchmarks/native/cg_micro.sh bench_rpc_auth.py, CPython 3.13.16 in the ladder's valgrind image, Ir/op = (Ir(K) − Ir(0)) / K; benchmarks/native/results/cg_rpc_auth.json:

Reads — the hot path (membership, claims, facts) — get the full saving. A command pays the Ed25519 price plus the MAC (+118 k Ir, ≈ +15 µs wall: mac.cmd.caller 47 µs / mac.cmd.provider 127 µs on a loaded machine, results/rpc_auth_wall_2026-10-21.txt), on top of the database transaction every command already runs — the price of "a leaked provider key never forges an effect". RPC_COMMANDS_REQUIRE_SIGNATURE=all makes every call cost the command column.

The provider's Ed25519 verify was ≈ 28 % of the provider side of an RPC call (R3t.srv 2.51 M Ir in the research stand); a MAC-only read leaves ≈ 7 %.

Consequences¶

• Split deployments move to RPC_AUTH_MODE=service_mac. Calling services keep their Ed25519 key (RPC_PRIVATE_KEY, public halves in RPC_SERVICE_KEYS) — it now signs only their commands. A pure provider needs only its X25519 key. Migration from service_identity: deploy this version everywhere (still service_identity; pinned callers now send aud) → every service holds its X25519 key, every binding pinned → service_mac_dual everywhere → service_mac everywhere. The first step is a hard precondition: a MAC-mode caller's keyed responses cover X-SANDLOOM-RPC-Auth, which a provider of an older version does not know (its keyed responses would fail mac_mismatch). The shipped microservices profile and the split conformance matrix run service_mac.
• RPC_RESPONSE_BINDING is unchanged and still covers responses (removed 2026-10-25: responses are always keyed — see the amendment).
• The monolith is unaffected (contract calls are direct Python calls).

Alternatives considered¶

• Rust Ed25519 verifier (ed25519-dalek): 2.1× wall, keeps non-repudiation, adds a native crypto dependency and keeps ~40 µs/call — rejected (research R2).
• mTLS only: authenticates the peer, but not the request at the application layer (a sidecar or gateway terminating TLS erases it) and gives no per-contract authorization input — complementary, not a replacement.
• One shared HMAC key (the shared_token model): any holder can impersonate every service — the property service_identity was introduced to remove.
• Ephemeral DH per connection / Noise: forward secrecy for a protocol that does not encrypt; much more code and state for no gain here.

Security review¶

An adversarial review (a separate agent whose only brief was to break the design and the code, 2026-10-21) produced eight findings; five were confirmed with proof-of-concept tests. All are fixed, each with a regression test (test_rpc_pair_mac.py::test_r1…r8), and the original PoCs now fail.

Claims the review could not break: forgery without K_req (swapped ids, reversed direction, response key as request key, self-pairs, a compromised third service); injectivity of the encoding and the single canonical header form; nothing decoded before the tag verifies; audience binding (also the Ed25519 aud); downgrade refusal in service_mac and no fallback once a MAC is present; low-order / all-zero points refused at startup and verification; cache keys follow key rotation; replay-guard window arithmetic; 503 mapping at the caller.

Remaining, accepted: the endpoint answers 401 rpc_auth_invalid (not the hidden 404) to a malformed X-SANDLOOM-RPC-Auth, revealing the MAC mode to a party that can already reach the internal network; reads replayed to a different replica / worker within the window are served under RPC_REPLAY_GUARD=commands (no effect, and the response is bound to the original request, so it cannot be substituted for another call).

Second review (independent, 2026-10-21)¶

A second adversarial review by a reviewer who had not seen the design found no exploitable cryptographic or verification flaw, and five items, all fixed (tests in test_rpc_pair_mac.py, section "second review"):

Third review (re-review of the KCI fix, 2026-10-21)¶

Verdict "safe to merge": the KCI mitigation holds (no bypass through a missing / duplicated signature, a foreign signer, method-kind confusion or a mode switch). Three low findings, fixed:

Follow-ups¶

• RPC_RESPONSE_BINDING=strict still accepts alg=token response bindings for token-authenticated calls (shared_token, dual, a binding with its own internal_token) — inherent to the token model; drop it once dual is retired. Done 2026-10-25 (amendment): the token modes and alg=token bindings are gone.

Amendment (2026-10-25)¶

Sandloom carries no backward compatibility before its first release, so the migration path above is removed and the target state is the only state:

• RPC_AUTH_MODE (shared_token, dual, service_identity, service_mac_dual, service_mac) and RPC_RESPONSE_BINDING (permissive, strict) are removed settings (sandloom.core.config.tiers.REMOVED): startup refuses them unless set to their inert value (service_mac / strict).
• The pair MAC is the only internal-RPC request authentication (key from RPC_DH_PRIVATE_KEY / RPC_SERVICE_DH_KEYS, audience-bound, nonce replay guard RPC_REPLAY_GUARD, window RPC_AUTH_MAX_SKEW_SECONDS), plus the caller's Ed25519 signature on command calls (RPC_PRIVATE_KEY / RPC_KEY_ID / RPC_SERVICE_KEYS; RPC_COMMANDS_REQUIRE_SIGNATURE: auto = on in prod, off in dev/test; all signs reads too) and the deny-by-default allowlist RPC_ALLOWED_CALLS. Ed25519-only and token requests are refused; INTERNAL_API_TOKEN guards only the ops endpoints and never authenticates /_sandloom/rpc (nor does INTERNAL_ENDPOINTS_OPEN_IN_DEV open it).
• Every SERVICE_BINDINGS entry must pin "service_id"; a process with bindings must hold its own RPC_DH_PRIVATE_KEY (startup refuses otherwise); a provider without one refuses every RPC call; a monolith (no bindings) needs no RPC keys.
• Responses are always keyed (v2 binding, X25519 pair key, pinned provider): unbound, unkeyed (v1) and alg=token replies are never accepted. An unkeyed error answer — the provider could not authenticate the request — surfaces at the caller as 503 RemoteServiceUnavailable carrying the provider's error code.

Related reading¶

• Service RPC guide
• Security model