Skip to content
Menu

Documentation

Migrations

Sandloom keeps schema changes as Alembic chains, split the way the code is split. One core chain, owned by the framework, creates the shared infrastructure tables (outbox, audit, consumer inbox, command log, HTTP idempotency log) in every database that hosts modules. Every module carries its own chain in its migrations/ directory, for its own tables, applied to exactly one database. A module owns its schema the way it owns its code, so it can move to its own database or service without untangling a shared history. You write a revision with sandloom migrate new <module> "<message>", which also renders row-level security for each new tenant table, and apply everything with sandloom migrate upgrade-all --apply.

What you get by default / what you declare

You get by default You declare
A migrations/ directory and a baseline revision rendered from the generated model by sandloom add module your models in models/
migrations/env.py as the one line run_module_migrations(__file__): package, table prefix and version table derived by convention nothing
An autogenerated revision in the module's own chain, with enable_tenant_rls / disable_tenant_rls for each new TenantScoped table sandloom migrate new <module> "<message>", then review the file
The chain applied to the module's database, with the core chain applied to every hosting database CapabilityRequirement("database", instance="x") or migrations_database= when not default
A version table per chain, alembic_version_<chain key>, so chains coexist in one database migrations_chain= only when two modules would share a key
A per-database advisory lock, so concurrent runs serialise nothing

How to use it

Two kinds of chain

Chain Location Contents Applied to
Core the framework's chain (source migrations/core/versions/, shipped in the wheel as sandloom/core/migrations/core_chain) shared infrastructure tables only every hosting database
Module <module package>/migrations/ that module's own tables (prefixed <module>_) the one database that hosts the module's tables

An app generated with sandloom new has no project-level Alembic tree: the migrator uses the framework's packaged core chain, and every module, bundled or yours, carries its own chain. Never add module tables to the core chain, and never add infrastructure tables to a module chain.

Writing a revision

sandloom migrate new tasks "add due date"            # autogenerate + RLS for new tenant tables
sandloom migrate revision tasks -m "backfill" # an empty revision to write by hand

sandloom migrate new diffs the module's models against its database, which must be at the chain head (sandloom test provisions it). For each new TenantScoped table the revision calls enable_tenant_rls(op, "<table>") (and disable_tenant_rls in downgrade); system_access=True is rendered only for a model that sets __system_access__ = True. A MoneyLedger model gets its unique operation-key constraint. Review the file before committing it: autogenerate cannot see renames or data moves. --database picks the database when the module has several; --rev-id sets an explicit id.

See Write a migration for the steps and Migrations and upgrades for upgrade workflows.

Applying migrations

Use the framework migrator, not raw Alembic, so every chain lands on the right database:

sandloom migrate upgrade-all --apply                 # core chain -> every hosting database, each module chain -> its database
sandloom migrate upgrade-module tasks --apply        # one module's chain
sandloom migrate upgrade-core --database analytics --apply
sandloom migrate plan                                # what would run (dry run)
sandloom migrate databases                           # configured database instances

Every command is a dry run without --apply. upgrade-all exits non-zero on failure, which is what the deploy profiles' one-shot migrate service relies on before the app and worker start. Module chains run in a deterministic order that honours depends_on=; a cycle or a missing dependency of a required module is fatal. A concurrent run waits on the per-database lock at most --lock-timeout seconds (MIGRATION_LOCK_TIMEOUT_SECONDS, default 300) and logs the holder.

The migrator resolves the module set through the composition root (SANDLOOM_COMPOSITION_ROOT), so it migrates exactly the modules the app loads.

Which database hosts a module's chain

A module's chain is applied to exactly one database:

  1. define_module(..., migrations_database="x") when set (always wins);
  2. otherwise the module's single required database capability requirement (CapabilityRequirement("database", instance="x"));
  3. no requirement: default; several including default: default (with a warning); several without default: an error asking you to set migrations_database.

Optional (required=False) database requirements never host the chain.

The chain key (the version-table suffix) is the last segment of the module name (sandloom_billing becomes billing) unless migrations_chain= sets it. Two loaded modules with the same key fail the run instead of silently sharing a chain.

Moving a module's data

Moving a module's existing data out of a shared database into its own is a separate, offline step: sandloom migrate extract-module (dry run by default; fence, copy, verify, cut over). See the split-deploy runbook in deploy/README.md and Deployment.

Dev and repair

sandloom migrate ensure-outbox --all-databases --apply

ensure-outbox creates the shared core tables without version tracking. It is a development and repair convenience, refused in production without --force; production uses upgrade-core.

What breaks it

Code When
SANDLOOM-T002 a table or index the models declare is not created by its owning chain (the suite builds tables from the models, production from the chains)
SANDLOOM-T006 planned: sandloom test will fail when the models differ from the chain head
SANDLOOM-C014 a per-tenant table that is not TenantScoped, so no RLS is rendered for it
MigrationTargetError a module whose chain database cannot be resolved (rule 3 above)
Guarantees, precisely
  • Ownership (G-MIGRATE-01, 03, 05). Each module owns a disjoint chain with its own version table; together the chains create every ORM table and reproduce the schema; two modules resolving to one chain key fail; the core chain creates only infrastructure tables.
  • Placement (G-MIGRATE-02). A module chain lands on exactly one database by the rules above.
  • Runs (G-MIGRATE-04, 09, 10, 11). Advisory lock per database; composition-root module set; non-zero exit on failure; deterministic, dependency-honouring order; explicit sync driver URLs.
  • Repair and extraction (G-MIGRATE-06 to 08). ensure-outbox is refused in production without --force; extract-module is a dry run by default, fenced, crash-resumable and verified.
  • Commands (C-MIGRATE-01 to 08).

See the guarantee ledger.

Design rationale