# Release packaging

This page explains how Bookclerk is packaged: what ships in a platform
release, how the pinned Cloudflare `workerd` binary is kept current, the
plugin author tools, and how to build the archives yourself.

Platform installs, crates.io plugin installs and CI all use the same
artifact layout (see [Plugin registry](/docs/plugins/registry/index.md)).

## What pulls from crates.io?

| Artifact | crates.io role | Installed automatically? |
| --- | --- | --- |
| **Host** (`bookclerk`, `bookclerkd`) | Not on crates.io today | No: an OS installer or GitHub Release |
| **Platform plugins** (`sqlite`, `local`, `embedder`) | May publish metadata crates later | **Bundled** in the platform install (`cargo package-platform`) |
| **Storefront plugins** (Audible, Libro, …) | A discovery index, when published | **Opt-in**, through `bookclerk plugins install` (which records a library install) |

crates.io is a **catalog**, not a runtime dependency resolver. Nothing is
compiled from crates.io on the operator's machine.

- `bookclerk plugins search` works today.
- `bookclerk plugins install` resolves a **prebuilt HTTPS archive** from
  the package's artifact URLs, verifies it, and records it as an installed
  version. Each host then fetches the archive for its own target, verifies
  its payload digest, and materializes it (see
  [Installing plugins](/docs/server/plugins/index.md#installing-plugins)).

## What a platform release contains

A platform release ships, so a fresh install works without storefronts
or a Rust toolchain:

- the hosts;
- `bookclerk-jail`, `bookclerk-workerd` and `bookclerk-media-worker`;
- the pinned Cloudflare `workerd` binary;
- the platform plugins (sqlite, local and embedder) under
  `plugins/<plugin-key-fs-id>/`, recorded in `install-ledger.json`.

On startup the host installs those bundled plugins from the `plugins/`
folder beside its executable into `$BOOKCLERK_FILES_DIR/plugins/`, and
stamps its own ledger. So any files folder works with a release, including
a container volume (see [Running Bookclerk → Docker](/docs/server/operations/index.md#docker)).

`cargo build-app --platform` and `cargo ensure-workerd` fetch the pinned
`workerd` into `target/<profile>/`. The download is tried up to four times,
waiting 2, 4 and 8 seconds between tries, when the connection drops, a try
takes longer than five minutes, or the server answers 5xx, 408 or 429. Any
other answer, a redirect loop, or a digest that does not match the pin fails
at once. Reference Echo examples are never
packaged.

### Cloudflare `workerd` pin

The release tag and per-architecture sha256 digests live in
[`crates/bookclerk-workerd/workerd-pin.json`](https://github.com/Bookclerk/bookclerk/blob/main/crates/bookclerk-workerd/workerd-pin.json).
The Rust `pin.rs` and the SDK stub copies are generated from it by
`scripts/sync-workerd-pin.py`.

> [!IMPORTANT]
> Don't float "latest" on every `cargo dev`. Hosts and packages must share
> one audited binary.

A daily GitHub Actions job (`.github/workflows/workerd-pin-bump.yml`)
proposes bumps:

1. `scripts/bump-workerd-pin.py` picks the newest Cloudflare `workerd`
   release that is at least **7 days** old. That is the same supply-chain
   cooldown as Dependabot's `cooldown.default-days: 7` in
   `.github/dependabot.yml`. It writes `workerd-pin.json`, then
   regenerates `pin.rs` and the SDK stubs.
2. The job installs `capnproto` (needed to compile `schema/plugin.capnp`),
   rebuilds `bookclerk-workerd`, runs `cargo ensure-workerd`, then runs
   `scripts/test-workerd-echo.sh` against the Echo workerd example guests.
3. It opens `deps/workerd-pin` with the automation App token and turns on
   auto-merge, so the merge queue lands it once its required checks pass
   (see [Automation App](/docs/contributing/automation/index.md#automation-app)).

To check by hand, without writing anything:

```sh
python3 scripts/bump-workerd-pin.py --check --cooldown-days 7
python3 scripts/sync-workerd-pin.py --check
```

#### The Pyodide bundle

The `embedder` plugin loads the ONNX Runtime shared library from beside
its own executable. The library is the upstream build, pinned by version
and SHA-256 for each platform in
`crates/bookclerk-onnxruntime/onnxruntime-pin.json`:

- `cargo ensure-onnxruntime` downloads it into
  `target/onnxruntime-<version>/` and checks the digest. Staging,
  `cargo install-platform` and `cargo package-platform` do the same and
  copy it into the plugin's folder, so platform packages carry it.
- Upstream publishes no build for Intel macOS. There the plugin is still
  packaged, reports itself unhealthy and serves no model.
- The library and the plugin need the C++ runtime (`libstdc++` on Linux),
  which the Docker image and every supported distribution already have.

See [Embedding model packs](/docs/internals/model-packs/index.md).

Python isolates load the Pyodide runtime bundle when they start. workerd
would download it on its own connection, but the jail gives workerd no
network, so the bundle is pinned too: `pyodide_bundle` in
`workerd-pin.json` names the file and its sha256.

- `cargo ensure-workerd` (and every caller of `ensure_workerd`) downloads it
  from `pyodide-capnp-bin.edgeworker.net` into `pyodide-bundles/` beside the
  binary and checks the digest. Platform packages carry the same directory.
- The launcher passes that directory to workerd with
  `--pyodide-bundle-disk-cache-dir`, and the gateway jail may only read it.
- A Python plugin refuses to start, naming the missing path, when the bundle
  is not there.

A pin bump keeps the current bundle. When a new workerd release wants another
one, the staged Python plugin checks fail and the plugin's error names the
bundle URL workerd asked for. Download that file, put its name and sha256
under `pyodide_bundle`, and run `python3 scripts/sync-workerd-pin.py --write`.

### Plugin author tools (`check` / `fmt` / `sync-embed` / `package` / `smoke`)

Each language SDK bakes in the same authoring CLI, so there is no
multi-architecture tool binary to ship. Commands and flags match across
languages, and each SDK is **self-contained** for its workerd guest
language (embedding and runtime settings):

| Launcher | Example |
| --- | --- |
| Rust | `cargo plugin -- check .` / `cargo plugin -- smoke .` / `cargo plugin -- test .` / `cargo plugin -- package --out dist .` |
| TypeScript | `npx bookclerk-plugin check .` / `npx bookclerk-plugin smoke .` |
| Python | `python -m bookclerk_plugin_sdk check .` / `python -m bookclerk_plugin_sdk smoke .` |

`smoke`:

1. Downloads the pinned Cloudflare `workerd` binary. TypeScript and Python
   don't need the Rust `bookclerk-workerd` launcher.
2. Materializes Cap'n Proto and the bridge, including a per-isolate
   `BRIDGE_TOKEN`.
3. POSTs `describe` (plus the role's `health` route) with
   `Authorization: Bearer …`.

`test` (Rust CLI only) runs the plugin's own suite once per database
capability profile; see
[Database limits](/docs/plugins/database-limits/index.md#the-suite-under-every-profile).

The Rust CLI lives in the **`bookclerk-plugin-tools`** crate (`cargo plugin`
runs its `bookclerk-plugin` binary), and uses the `bookclerk-workerd`
library for `smoke`. Guest plugins depend only on `bookclerk-plugin-sdk`,
so they never pull in workerd download dependencies.
`scripts/sync-workerd-pin.py` keeps the pin and bridge copies in sync
(`--check` in CI).

**Workerd packaging.** Authors import the language package
(`@bookclerk/plugin-sdk/workerd`, `bookclerk_plugin_sdk.workerd`), and
`bookclerk-workerd` injects those modules into the isolate.

- The Python SDK also requires, and writes,
  `compatibility_flags = ["python_workers", "disable_python_external_sdk"]`
  for `.py` mains.
- The optional `sync-embed` vendors SDK sources for offline trees.

The shared contracts are
[`plugin.capnp`](https://github.com/Bookclerk/bookclerk/blob/main/crates/bookclerk-plugin-abi/schema/plugin.capnp),
[`plugin-toml.json`](https://github.com/Bookclerk/bookclerk/blob/main/crates/bookclerk-plugin-abi/schema/plugin-toml.json),
and the fixtures under
[`crates/bookclerk-plugin-abi/fixtures/tools/`](https://github.com/Bookclerk/bookclerk/tree/main/crates/bookclerk-plugin-abi/fixtures/tools/).

## Local packaging (current OS/arch)

Host bundles need `ui/dist`. `cargo package-hosts` rebuilds it when the
SPA sources are stale, or run `cd ui && npm ci && npm run build`.

```sh
cargo package-plugins      # product plugin archives (no examples) → target/dist/plugins/
cargo package-hosts        # hosts + jail + media-worker + bookclerk-workerd + workerd → target/dist/
cargo package-platform     # hosts + helpers + pinned workerd + sqlite + local → target/dist/
```

Archive names follow the [Plugin registry](/docs/plugins/registry/index.md):

```text
bookclerk-plugin-source-audible-0.1.0-linux-x64-gnu.tar.gz
bookclerk-platform-0.1.0-linux-x64-gnu.tar.gz
```

Each output folder includes `SHA256SUMS*` files for verification.

## Container images

[`.github/workflows/images-publish.yml`](https://github.com/Bookclerk/bookclerk/blob/main/.github/workflows/images-publish.yml)
publishes three images to `ghcr.io/<owner>/`, where `<owner>` is the
repository's owner. A fork publishes to its own namespace with its own
`GITHUB_TOKEN`.

| Image | Built from | Architectures |
| --- | --- | --- |
| `bookclerk` | `packaging/docker/Dockerfile` | `amd64`, `arm64` |
| `bookclerk-dev` | `.devcontainer/Dockerfile` | `amd64`, `arm64` |
| `bookclerk-demo` | `packaging/demo/Dockerfile` | `amd64` |

The `bookclerk-dev` image carries inline cache metadata, so the Dev
Container's `build.cacheFrom` reuses its layers. The demo image builds on
the production builder and runtime, all three in one build from
[`packaging/demo/docker-bake.hcl`](https://github.com/Bookclerk/bookclerk/blob/main/packaging/demo/docker-bake.hcl).
That build links with thin LTO and parallel codegen units. Shipped images
keep `Cargo.toml`'s fat LTO: `packaging/docker/Dockerfile` takes the
settings as build arguments that default to it. The demo's sample-data
seed links the host's library crates, so it builds with exactly the
package set `cargo package-platform` built (`cargo build-app --platform
--print`), where those crates are already compiled. The storefronts build
in a separate Cargo invocation, because building them beside the seed
would change those crates' features and recompile them. The demo image's
last stage seeds the demo library at build time
(`packaging/demo/bake-seed.sh`): it runs the seed and `bookclerkd` with
network access to the storefronts' public catalogs, downloads the covers
a new session shows first into the cover cache, then strips every
per-container secret, so building the demo image needs outbound network
(see [`tools/demo/README.md`](https://github.com/Bookclerk/bookclerk/blob/main/tools/demo/README.md)).
The hosted demo's deploy pulls the
published demo image without signing in, so the package must be public.
When the image for the deployed commit exists and its attestation
verifies, the deploy uses it instead of building it again. Otherwise a
push or manual deploy builds the image itself, and a deploy started by a
publish that pushed no demo image stops (see
[The demo deploy](/docs/contributing/automation/index.md#the-demo-deploy)).

Each architecture builds on a native runner and is checked before it is
pushed, then the architectures are joined into one index:

- The production image boots twice through `packaging/docker/smoke.sh`.
- The Dev Container's tools run as the dev user.

### Tags

- **`sha-<12 hex>`**: built from that `main` commit, and never moved. The
  per-architecture `sha-<12 hex>-<arch>` tags are the index's inputs.
- **`edge`**: the newest build from `main`.
- **`<version>`, `<major>.<minor>`, `<major>`, `latest`**: added when a
  `v<major>.<minor>.<patch>` release is published, by retagging that
  commit's images (built then if missing). Leading zeros and suffixes are
  refused by `scripts/image_release_tags.py`.

### When it runs

The workflow runs:

- nightly when `main` has moved (an image whose `sha-` tag exists is not
  rebuilt);
- on a published release whose commit is on `main` (see
  [release tags](#release-tags));
- on `main` pushes that change the Dev Container's inputs;
- by hand (a run on any branch but `main` is a dry run).

A pull request that changes the workflow,
[`.github/actions/image-build`](https://github.com/Bookclerk/bookclerk/blob/main/.github/actions/image-build/action.yml)
or `scripts/image_release_tags.py` builds and checks every image with a read-only token, and pushes nothing.
Both paths run the same build steps from that action.

### Provenance and permissions

Every published index gets a GitHub build-provenance attestation. It is
signed through Sigstore with the workflow's OIDC identity and stored in the
registry. To check one, replace `<owner>` with the repository's owner:

```sh
gh attestation verify oci://ghcr.io/<owner>/bookclerk:edge --owner <owner>
```

If the attest job fails, re-running the workflow attests the published
index without rebuilding it.

Only the push jobs hold `packages: write`. Only the attest job holds
`id-token: write` and `attestations: write`.

### Making the packages public

The first publish creates each package as **private**, so `docker pull`
without signing in is refused. An organization owner makes each package
public once. A package appears only after its first publish: pushes to
`main` publish only `bookclerk-dev`, so `bookclerk` and `bookclerk-demo`
first appear after the nightly run or a manual run (see
[When it runs](#when-it-runs)).

1. Open `https://github.com/orgs/<owner>/packages/container/<image>/settings`
   (for example `https://github.com/orgs/bookclerk/packages/container/bookclerk/settings`).
2. Under **Danger Zone**, choose **Change visibility**, then **Public**.
3. Type the package name to confirm.
4. Repeat for `bookclerk-dev` and `bookclerk-demo`.

> [!NOTE]
> The organization setting that lets members create public packages
> (**Settings** > **Packages** > **Package creation**) only decides which
> visibilities new packages may get. It does not change packages that
> already exist.

> [!WARNING]
> GitHub does not let a public package go back to private. Later publishes
> keep the package's visibility, so this step is needed only once.

## Desktop packages

The desktop app's unsigned packages (see
[Packages](/docs/contributing/ui/index.md#packages)) are built by
[`.github/workflows/desktop-packages.yml`](https://github.com/Bookclerk/bookclerk/blob/main/.github/workflows/desktop-packages.yml):
a `.deb` and an AppImage on Linux, an NSIS installer on Windows, and a
`.dmg` on macOS. Each is a full release build on its own runner.

The images workflow calls it, so it runs on the images' schedule (see
[When it runs](#when-it-runs)):

- nightly, and by hand, when `main` has moved since the last publish;
- for a published release, whose packages it attaches to the release.

A pull request runs it only when it changes the workflow or the packaging
configuration (`tauri.packaging*.json`, `crates/bookclerk-dev/src/package.rs`).
CI's [desktop job](/docs/contributing/ci/index.md#desktop-job) covers the window's code on every pull
request. A change that breaks how the packaged app finds its runtime shows
up in the next nightly build.

Every build uploads the packages as workflow artifacts, kept for 14 days.
For a release, the `publish` job attaches them to the GitHub release with a
build-provenance attestation each. To check a downloaded package, replace
`<owner>` with the repository's owner:

```sh
gh attestation verify <downloaded package> --owner <owner>
```

Only that job holds `contents: write`, `id-token: write` and
`attestations: write`.

The package builds keep no Rust cache. Three release caches would not fit
beside CI's caches in the repository's 10 GB, and a nightly build has no
one waiting on it.

## Release tags

Publishing a release starts the image publish, the docs publish, the
website deploy and the Audiobookshelf integration test, each for the
commit the release's `v*` tag names. Those runs use that commit's own copy
of each workflow, so whoever can create or move a `v*` tag decides what
gets built, signed and published as a release. Three layers keep that to
reviewed code:

1. **Rulesets.** Only repository administrators may create a `v*` tag,
   and nobody may move or delete one. The automation App can write
   contents, but it can't tag a release. To set them up, run this once
   as an administrator, with your own GitHub CLI login:

   ```bash
   bash scripts/setup-release-rulesets.sh Bookclerk/bookclerk
   ```

   It creates, or updates in place, the **Release tags: creators** and
   **Release tags: history** rulesets. An administrator still publishes
   releases from the web UI as before, since that creates the tag.
2. **Immutable releases.** Turn on **Settings → General → Releases →
   Enable release immutability**, so a published release's tag and assets
   can't change afterwards.
3. **The commit must be on `main`.** The image and docs publishes start
   with [`.github/actions/release-on-main`](https://github.com/Bookclerk/bookclerk/blob/main/.github/actions/release-on-main/action.yml),
   which fails unless the default branch contains the release's commit.
   The website deploy builds its code from the default branch and takes
   only the release's `docs/` pages from the tag.

> [!NOTE]
> The check in step 3 runs from the tagged commit's own workflow, so it
> catches a tag put on the wrong commit by mistake, not a forged one: a
> commit written to drop the check would skip it. The rulesets are the
> control that stops that.

## GitHub Actions (signing sketch — not a contract)

> [!NOTE]
> The workflow below is an **illustrative sketch** for publisher-side
> packaging. Digest pinning, notarization and Authenticode requirements
> are set by follow-up install and trust work, not frozen by this page.

Per-OS matrix builds should run the same cargo aliases, then optionally
sign or notarize:

```yaml
# .github/workflows/release.yml (sketch — subject to change)
jobs:
  package:
    strategy:
      matrix:
        include:
          - os: ubuntu-latest
            target: x86_64-unknown-linux-gnu
          - os: macos-latest
            target: aarch64-apple-darwin
          - os: windows-latest
            target: x86_64-pc-windows-msvc
    runs-on: ${{ matrix.os }}
    steps:
      - uses: actions/checkout@v7
      - uses: dtolnay/rust-toolchain@stable
      - uses: actions/setup-node@v7
        with:
          node-version-file: .tool-versions
      - run: cd ui && npm ci && npm run build
      - run: cargo package-platform --out dist
      - run: cargo package-plugins --out dist/plugins
      # Signing (examples — pick per OS):
      # - Linux: cosign sign-blob / minisign
      # - macOS: codesign + notarytool (after package-platform)
      # - Windows: Authenticode (signtool)
      - uses: actions/upload-artifact@v4
        with:
          name: bookclerk-${{ matrix.target }}
          path: dist/
```

Publish the release assets to GitHub Releases (or S3 or R2), and set
`artifact_base_url` in each plugin crate's `[package.metadata.bookclerk]`
before `cargo publish` on crates.io.

## Next steps

- [`crates/bookclerk-dev/README.md`](https://github.com/Bookclerk/bookclerk/blob/main/crates/bookclerk-dev/README.md):
  the development workflow aliases.
- [Plugin registry](/docs/plugins/registry/index.md): the crates.io taxonomy.
- [Repository automation](/docs/contributing/automation/index.md): the jobs that build and bump
  these.
