Skip to content
Menu

Documentation

ADR-0001: Framework distribution & packaging

  • Status: Accepted
  • Date: 2026-07-01
  • Deciders: Sandloom maintainers
  • Supersedes: —

Historical design

This ADR records the original distribution design. The current four-package layout is documented in Packages, and the release procedure in Publishing. Package names and layout examples below describe the design at the time of this decision.

Context

At the time of this decision, Sandloom used a single repository (backend/) that built one wheel (sandloom-framework) containing the kernel and the bundled business modules (identity, tenancy, billing) and all infrastructure drivers (SQLAlchemy, Redis, Celery, SMTP, filesystem).

We need a distribution model that answers, coherently:

  • How does a consumer obtain Sandloom and build a SaaS on it?
  • How are security patches to cross-cutting platform code (auth, billing webhooks) delivered without forcing every consumer to fork?
  • Where is the boundary between "the framework" and "the consumer's product"?
  • How are independent components versioned, made compatible, and published?

The current single-wheel layout conflates three very different kinds of code, and two distribution signals already contradict each other: a pip-installable sandloom-framework package (library model) and an sandloom-scaffold / sandloom new project generator (template model).

Verification of current boundaries (from the codebase)

A boundary audit of sandloom.core confirmed the kernel is cleanly separable at runtime, with a few leaks to remediate before the split:

  • No real code import of identity / tenancy / billing from core — only strings, comments, config defaults, and tests.
  • sandloom.shared → sandloom.core.di is one-directional (acyclic). Apparent core → shared hits are only in scaffold template strings (generated code) and integration tests under core/tests/.
  • SQLAlchemy / Redis / Celery are confined to drivers/* (formerly infra/drivers/*, ADR-0007) and the relational surface (orm, di, sql_alchemy/, projections/, migrations/). The paradigm-agnostic kernel is SDK-free.
  • Decisive fact for the infra boundary: the relational paradigm is not behind a swappable contract — UoW, outbox_store, audit_store, durable_inbox, tenant scoping, projections and migrations are direct SQLAlchemy code inside the kernel. import sandloom.core.di cannot load without SQLAlchemy. The settings validator (_db_dsn_ok) further mandates postgresql+asyncpg://. The framework is therefore Postgres-relational by construction today; an in-memory/SQLite relational default would be a lie.

The enforced public contract (from architecture_rules.yml, checked by tools/checks/check_architecture.py as a strict whitelist) is exactly six modules: sandloom.core.facade, sandloom.core.di, sandloom.core.module_api, sandloom.core.descriptors, sandloom.core.orm, sandloom.core.projections. The prose list in AGENTS.md (which also names settings, idempotency, exceptions, security, lifecycle, runtime as a stable surface) is broader than the enforced contract and is corrected to match.

Decision

1. Consumption model — three tiers

  • Kernel = library. sandloom-framework is a versioned pip package, read-only to consumers, upgraded by version bump.
  • Platform modules = versioned packages. identity / tenancy / billing ship as separate versioned packages, not scaffold — so security-sensitive code (JWT, RBAC, 2FA, SSO, Stripe webhooks, credits ledger) is patchable centrally by a version bump rather than a per-consumer fork.
  • Domain modules = scaffold. The consumer's own product modules are generated by sandloom new and owned by the consumer.

2. Package graph (monorepo + uv workspace, N wheels, independent versions)

  • sandloom-framework — kernel plus the relational spine (hard deps: fastapi, uvicorn, pydantic, pydantic-settings, sqlalchemy, alembic, asyncpg, orjson, jsonschema, prometheus-client, pyjwt, passlib /argon2). Bundles zero-dependency infra defaults in-tree: in-memory cache, in-memory event bus, filesystem object store, log notifier. Ships no main:app.
  • sandloom-shared — thin layer above the kernel (depends on sandloom-framework); the second public surface (sandloom.shared.*).
  • sandloom-identity, sandloom-tenancy, sandloom-billing — platform packages.
  • sandloom-infra-redis, sandloom-infra-celery, sandloom-infra-smtp, sandloom-infra-s3 — opt-in backend plugins registered via the sandloom.infra entry-point group.
  • sandloom-platform — thin meta-package ("batteries"): depends on sandloom-framework[postgres] + sandloom-identity + sandloom-tenancy + sandloom-billing; carries create_app(), main:app, and the default module list.
  • Consumer domain modules — scaffolded, consumer-owned.

The relational backend server is swappable only by DSN within asyncpg; Redis / Celery / SMTP / S3 are the genuinely swappable infrastructure and become opt-in packages. The kernel boots on Postgres + in-memory everything else.

3. Compatibility contract

  • SemVer is governed by the enforced public_api (the six modules above) plus sandloom.shared.*, plus behavioural/wire contracts (event channel names *.vN, capability contracts) which are versioned separately from the import API.
  • A CI gate snapshots the public surface (e.g. griffe) and fails the build when it changes incompatibly without a version bump. The existing check_architecture.py only stops consumers from crossing the line; this gate stops the line from moving silently.
  • 0.x policy: lockstep-minor / independent-patch. All workspace packages move together on a minor (one minor = one compatible set); security/bug fixes patch independently. Satellites pin sandloom-framework ~=0.N.

4. Assembly

sandloom-platform is the assembly point. sandloom.main:app and the default module list move out of the kernel into sandloom-platform. Consumers either depend on sandloom-platform (batteries) or on bare sandloom-framework (--minimal) and call create_app(modules=[...]) in their own owned app.py.

5. Scaffold mechanics (sandloom new)

  • sandloom new <project> generates a consumer-owned app project depending on sandloom-platform ~=0.N by default (--minimal → bare sandloom-framework). It emits app.py (create_app(modules=[...])), pyproject.toml (compatible-release pins) plus a committed uv.lock for reproducibility, .env.example, Alembic wiring, a consumer-owned Dockerfile, compose, and a CI skeleton.
  • sandloom new module <name> generates a domain-module skeleton inside the project (owned code).
  • Templates ship inside the sandloom-framework CLI and emit code that imports only the public surface.

6. Distribution channels

  • Canonical channel: public PyPI wheels, namespace sandloom-*.
  • Publish functional distributions, with matching metadata and source archives; an empty placeholder release is not a distribution strategy.
  • Source development: use editable local dependencies. Portable deployments install published packages or a complete matching wheel set.
  • Trusted Publishing (PyPI OIDC) authenticates package uploads without long-lived API tokens in CI. The checked-in workflow is build-only; enabling uploads requires the publisher configuration and explicit workflow changes described in the current publishing guide.
  • Docker → GHCR is a secondary track for the reference sandloom-platform app only (quickstart / self-host), not a framework consumption channel. There is no FROM sandloom base image; the scaffold emits a consumer-owned Dockerfile that installs the pinned packages.

7. Release engineering

  • Automated, per package: python-semantic-release, scope/path-aware conventional commits (feat(identity): …, fix(core): …). The bot computes the version and changelog from the commits touching that package and pushes a per-package tag sandloom-<pkg>-vX.Y.Z; CI builds and publishes only the changed package.
  • The griffe public-surface snapshot and the lockstep-minor gate remain on top as a safety net: a feat: that breaks the public surface without the corresponding bump still fails CI. Automation derives versions; the gates stop it from lying.
  • Per-package CHANGELOG (Keep a Changelog); the root changelog becomes a framework-wide aggregator.

Consequences

Positive

  • Security patches to platform modules ship as version bumps, never forks.
  • A clear, machine-enforced framework/product boundary.
  • Lightweight kernel footprint (no Redis/Celery/SMTP/S3 unless opted in).
  • Atomic cross-cutting refactors in one monorepo PR, with independent release cadence per package.

Negative / costs

  • Repository restructure into packages/*, each with its own pyproject.toml.
  • More CI machinery (per-package build/publish matrix, griffe gate, lockstep gate).
  • Architecture tests must enforce package boundaries, not only intra-tree imports.

Boundary remediations required before the split

  1. Remove the satellite names baked into the kernel: settings.py defaults sandloom_modules = ["sandloom_identity", "sandloom_tenancy", "sandloom_billing"] and module_required = ["sandloom_identity"] move to sandloom-platform / scaffold.
  2. Move core/tests/* that import identity / tenancy / billing / shared into a workspace-level tests/integration package (they cannot ship inside the kernel wheel's suite).
  3. Keep only zero-dependency drivers in the kernel; move Redis / Celery / SMTP / S3 drivers into sandloom-infra-*.
  4. Move main:app + composition into sandloom-platform.
  5. Dependency hygiene: pytest, factory-boy, pytest-xdist, pytest-* are currently in the kernel's runtime dependencies — move them to [dev] so consumers of sandloom-framework do not pull the test stack at runtime.
  6. Narrow the capability docs honestly: "database is a freely swappable paradigm" is untrue while only the SQLAlchemy/Postgres driver exists and the PG-only validator stands. SQLite-for-tests is roadmap, not a v0.1.0 promise.

Implementation status

All six remediations are done, and the packaging split has landed (source of truth stays in src/sandloom/**; each packages/<name> builds a distributable wheel via hatchling force-include):

  • 1 (satellite names out of the kernel) and #3 (lean kernel — Redis/Celery are

    optional extras sandloom-framework[redis|celery|infra], with thin sandloom-infra-redis / sandloom-infra-celery meta-packages) — done in PR #18.

  • 2 (integration tests moved to tests/integration), #4 (main:app in

    sandloom-platform), #5 (test deps → [dev]), and #9 (packages CI build/compose + lean-import gate) — done in PRs #17 and earlier.
  • See packages/README.md for the current tiers and build hook. The original implementation follow-up notes are archived separately from the maintained source.

Distribution & release machinery (this ADR's §3/§5/§6/§7):

  • §3 versioning — a public-API surface gate (tools/packaging/check_public_api.py
  • committed public_api_baseline.json) fails the build on a backward-incompatible change to the enforced public surface; wired into the Architecture Check workflow.
  • §5 scaffold — sandloom new depends on sandloom-platform in batteries mode and bare sandloom-framework for --minimal/--no-batteries (matching the tiers).
  • §6/§7 distribution & release — per-package changelogs, a per-package tag scheme sandloom-<pkg>-vX.Y.Z, and a Trusted-Publishing (OIDC, no-secrets) release workflow that fails closed until PyPI pending publishers exist. Operator steps for claiming the PyPI names and configuring pending publishers are in docs/contributing/publishing.md — these are the only steps that must be done by a human on pypi.org.
  • Intentionally deferred: the sandloom-infra-smtp meta-package (SMTP driver is stdlib-only), the sandloom-infra-s3 meta-package (no S3 driver exists yet), a SQLite-for-tests relational driver (roadmap), and python-semantic-release full automation (optional polish).

Update 2026-10-28: four distributions. The eight packages above became four, and the module packages became ecosystem modules shipped the way a third party ships one. The text above records the earlier layout.

  • sandloom is the framework: the kernel, sandloom.sdk, sandloom.shared (framework DTOs, guards, authz), the CLI and the composition root sandloom.main. Its extras are redis, celery, standard (Redis + Celery + the three modules, the former sandloom-platform), fast, check, testing and sentry. sandloom-framework, sandloom-shared, sandloom-platform, sandloom-infra-redis and sandloom-infra-celery are removed outright before the initial public alpha (no shims).
  • sandloom-identity, sandloom-tenancy and sandloom-billing ship the top-level import packages sandloom_identity, sandloom_tenancy and sandloom_billing (source in src/sandloom_<module>/), each with one sandloom.modules entry point whose name is the module name. They depend on sandloom>=0.7,<0.8 and on no other module; versions are independent and start at 0.7.0.
  • sandloom.main:app composes every installed entry point when SANDLOOM_MODULES is unset; the kernel names no module. A requested module name with two providers fails startup (SANDLOOM-B020).
  • sandloom.shared is unchanged and ships whole inside the sandloom distribution (contract DTOs, the standard modules' generated stubs and event schemas, authz, guards): every caller process imports it (SANDLOOM-C012).
  • Dependencies are ranges (>=tested,<next breaking release); exact versions live only in uv.lock.
  • tools/packaging/check_package_distributions.py (CI: .github/workflows/packages.yml) builds the four wheels and sdists, checks import layering across distributions, installs sandloom alone to run every CLI subcommand, and installs sandloom[standard] to compose the modules.
  • sandloom new depends on sandloom[redis,celery]>=0.7,<0.8 plus one sandloom-<module>>=0.7,<0.8 per --with module. Tags are sandloom-vX.Y.Z and sandloom-<module>-vX.Y.Z.

Alternatives considered

  • Single wheel with extras (sandloom-framework[identity,billing], one version for everything). Rejected: kills independent security patching of platform modules — the core reason for the tiering.
  • Polyrepo (one git repo per package). Rejected for 0.x: cross-cutting changes become N-PR synchronization with no offsetting benefit yet.
  • Contracts-only kernel (relational/Postgres fully extracted into sandloom-infra-sqlalchemy, kernel depends only on capability contracts). Rejected: the relational paradigm is structurally in the kernel (UoW/outbox/audit/ migrations are SQLAlchemy code, and import sandloom.core.di requires it). Making it "optional" would be dishonest and require a large refactor for an abstraction the framework does not actually want.
  • Manual versioning (bump-my-version + per-package tags, human decides the number). Rejected in favour of automation; the griffe + lockstep gates are retained regardless as the forcing function.