Release packaging
This page explains how Bookclerk is packaged: what ships in a platform
release, how the pinned Cloudflare workerd binary is kept current, the
plugin author tools, and how to build the archives yourself.
Platform installs, crates.io plugin installs and CI all use the same artifact layout (see Plugin registry).
| Artifact | crates.io role | Installed automatically? |
|---|---|---|
Host (bookclerk, bookclerkd) |
Not on crates.io today | No: an OS installer or GitHub Release |
Platform plugins (sqlite, local, embedder) |
May publish metadata crates later | Bundled in the platform install (cargo package-platform) |
| Storefront plugins (Audible, Libro, …) | A discovery index, when published | Opt-in, through bookclerk plugins install (which records a library install) |
crates.io is a catalog, not a runtime dependency resolver. Nothing is compiled from crates.io on the operator's machine.
bookclerk plugins searchworks today.bookclerk plugins installresolves a prebuilt HTTPS archive from the package's artifact URLs, verifies it, and records it as an installed version. Each host then fetches the archive for its own target, verifies its payload digest, and materializes it (see Installing plugins).
A platform release ships, so a fresh install works without storefronts or a Rust toolchain:
- the hosts;
bookclerk-jail,bookclerk-workerdandbookclerk-media-worker;- the pinned Cloudflare
workerdbinary; - the platform plugins (sqlite, local and embedder) under
plugins/<plugin-key-fs-id>/, recorded ininstall-ledger.json.
On startup the host installs those bundled plugins from the plugins/
folder beside its executable into $BOOKCLERK_FILES_DIR/plugins/, and
stamps its own ledger. So any files folder works with a release, including
a container volume (see Running Bookclerk → Docker).
cargo build-app --platform and cargo ensure-workerd fetch the pinned
workerd into target/<profile>/. The download is tried up to four times,
waiting 2, 4 and 8 seconds between tries, when the connection drops, a try
takes longer than five minutes, or the server answers 5xx, 408 or 429. Any
other answer, a redirect loop, or a digest that does not match the pin fails
at once. Reference Echo examples are never
packaged.
The release tag and per-architecture sha256 digests live in
crates/bookclerk-workerd/workerd-pin.json.
The Rust pin.rs and the SDK stub copies are generated from it by
scripts/sync-workerd-pin.py.
Important
Don't float "latest" on every cargo dev. Hosts and packages must share
one audited binary.
A daily GitHub Actions job (.github/workflows/workerd-pin-bump.yml)
proposes bumps:
scripts/bump-workerd-pin.pypicks the newest Cloudflareworkerdrelease that is at least 7 days old. That is the same supply-chain cooldown as Dependabot'scooldown.default-days: 7in.github/dependabot.yml. It writesworkerd-pin.json, then regeneratespin.rsand the SDK stubs.- The job installs
capnproto(needed to compileschema/plugin.capnp), rebuildsbookclerk-workerd, runscargo ensure-workerd, then runsscripts/test-workerd-echo.shagainst the Echo workerd example guests. - It opens
deps/workerd-pinwith the automation App token and turns on auto-merge, so the merge queue lands it once its required checks pass (see Automation App).
To check by hand, without writing anything:
python3 scripts/bump-workerd-pin.py --check --cooldown-days 7
python3 scripts/sync-workerd-pin.py --check
The embedder plugin loads the ONNX Runtime shared library from beside
its own executable. The library is the upstream build, pinned by version
and SHA-256 for each platform in
crates/bookclerk-onnxruntime/onnxruntime-pin.json:
cargo ensure-onnxruntimedownloads it intotarget/onnxruntime-<version>/and checks the digest. Staging,cargo install-platformandcargo package-platformdo the same and copy it into the plugin's folder, so platform packages carry it.- Upstream publishes no build for Intel macOS. There the plugin is still packaged, reports itself unhealthy and serves no model.
- The library and the plugin need the C++ runtime (
libstdc++on Linux), which the Docker image and every supported distribution already have.
Python isolates load the Pyodide runtime bundle when they start. workerd
would download it on its own connection, but the jail gives workerd no
network, so the bundle is pinned too: pyodide_bundle in
workerd-pin.json names the file and its sha256.
cargo ensure-workerd(and every caller ofensure_workerd) downloads it frompyodide-capnp-bin.edgeworker.netintopyodide-bundles/beside the binary and checks the digest. Platform packages carry the same directory.- The launcher passes that directory to workerd with
--pyodide-bundle-disk-cache-dir, and the gateway jail may only read it. - A Python plugin refuses to start, naming the missing path, when the bundle is not there.
A pin bump keeps the current bundle. When a new workerd release wants another
one, the staged Python plugin checks fail and the plugin's error names the
bundle URL workerd asked for. Download that file, put its name and sha256
under pyodide_bundle, and run python3 scripts/sync-workerd-pin.py --write.
Each language SDK bakes in the same authoring CLI, so there is no multi-architecture tool binary to ship. Commands and flags match across languages, and each SDK is self-contained for its workerd guest language (embedding and runtime settings):
| Launcher | Example |
|---|---|
| Rust | cargo plugin -- check . / cargo plugin -- smoke . / cargo plugin -- test . / cargo plugin -- package --out dist . |
| TypeScript | npx bookclerk-plugin check . / npx bookclerk-plugin smoke . |
| Python | python -m bookclerk_plugin_sdk check . / python -m bookclerk_plugin_sdk smoke . |
smoke:
- Downloads the pinned Cloudflare
workerdbinary. TypeScript and Python don't need the Rustbookclerk-workerdlauncher. - Materializes Cap'n Proto and the bridge, including a per-isolate
BRIDGE_TOKEN. - POSTs
describe(plus the role'shealthroute) withAuthorization: Bearer ….
test (Rust CLI only) runs the plugin's own suite once per database
capability profile; see
Database limits.
The Rust CLI lives in the bookclerk-plugin-tools crate (cargo plugin
runs its bookclerk-plugin binary), and uses the bookclerk-workerd
library for smoke. Guest plugins depend only on bookclerk-plugin-sdk,
so they never pull in workerd download dependencies.
scripts/sync-workerd-pin.py keeps the pin and bridge copies in sync
(--check in CI).
Workerd packaging. Authors import the language package
(@bookclerk/plugin-sdk/workerd, bookclerk_plugin_sdk.workerd), and
bookclerk-workerd injects those modules into the isolate.
- The Python SDK also requires, and writes,
compatibility_flags = ["python_workers", "disable_python_external_sdk"]for.pymains. - The optional
sync-embedvendors SDK sources for offline trees.
The shared contracts are
plugin.capnp,
plugin-toml.json,
and the fixtures under
crates/bookclerk-plugin-abi/fixtures/tools/.
Host bundles need ui/dist. cargo package-hosts rebuilds it when the
SPA sources are stale, or run cd ui && npm ci && npm run build.
cargo package-plugins # product plugin archives (no examples) → target/dist/plugins/
cargo package-hosts # hosts + jail + media-worker + bookclerk-workerd + workerd → target/dist/
cargo package-platform # hosts + helpers + pinned workerd + sqlite + local → target/dist/
Archive names follow the Plugin registry:
bookclerk-plugin-source-audible-0.1.0-linux-x64-gnu.tar.gz
bookclerk-platform-0.1.0-linux-x64-gnu.tar.gz
Each output folder includes SHA256SUMS* files for verification.
.github/workflows/images-publish.yml
publishes three images to ghcr.io/<owner>/, where <owner> is the
repository's owner. A fork publishes to its own namespace with its own
GITHUB_TOKEN.
| Image | Built from | Architectures |
|---|---|---|
bookclerk |
packaging/docker/Dockerfile |
amd64, arm64 |
bookclerk-dev |
.devcontainer/Dockerfile |
amd64, arm64 |
bookclerk-demo |
packaging/demo/Dockerfile |
amd64 |
The bookclerk-dev image carries inline cache metadata, so the Dev
Container's build.cacheFrom reuses its layers. The demo image builds on
the production builder and runtime, all three in one build from
packaging/demo/docker-bake.hcl.
That build links with thin LTO and parallel codegen units. Shipped images
keep Cargo.toml's fat LTO: packaging/docker/Dockerfile takes the
settings as build arguments that default to it. The demo's sample-data
seed links the host's library crates, so it builds with exactly the
package set cargo package-platform built (cargo build-app --platform --print), where those crates are already compiled. The storefronts build
in a separate Cargo invocation, because building them beside the seed
would change those crates' features and recompile them. The demo image's
last stage seeds the demo library at build time
(packaging/demo/bake-seed.sh): it runs the seed and bookclerkd with
network access to the storefronts' public catalogs, downloads the covers
a new session shows first into the cover cache, then strips every
per-container secret, so building the demo image needs outbound network
(see tools/demo/README.md).
The hosted demo's deploy pulls the
published demo image without signing in, so the package must be public.
When the image for the deployed commit exists and its attestation
verifies, the deploy uses it instead of building it again. Otherwise a
push or manual deploy builds the image itself, and a deploy started by a
publish that pushed no demo image stops (see
The demo deploy).
Each architecture builds on a native runner and is checked before it is pushed, then the architectures are joined into one index:
- The production image boots twice through
packaging/docker/smoke.sh. - The Dev Container's tools run as the dev user.
sha-<12 hex>: built from thatmaincommit, and never moved. The per-architecturesha-<12 hex>-<arch>tags are the index's inputs.edge: the newest build frommain.<version>,<major>.<minor>,<major>,latest: added when av<major>.<minor>.<patch>release is published, by retagging that commit's images (built then if missing). Leading zeros and suffixes are refused byscripts/image_release_tags.py.
The workflow runs:
- nightly when
mainhas moved (an image whosesha-tag exists is not rebuilt); - on a published release whose commit is on
main(see release tags); - on
mainpushes that change the Dev Container's inputs; - by hand (a run on any branch but
mainis a dry run).
A pull request that changes the workflow,
.github/actions/image-build
or scripts/image_release_tags.py builds and checks every image with a read-only token, and pushes nothing.
Both paths run the same build steps from that action.
Every published index gets a GitHub build-provenance attestation. It is
signed through Sigstore with the workflow's OIDC identity and stored in the
registry. To check one, replace <owner> with the repository's owner:
gh attestation verify oci://ghcr.io/<owner>/bookclerk:edge --owner <owner>
If the attest job fails, re-running the workflow attests the published index without rebuilding it.
Only the push jobs hold packages: write. Only the attest job holds
id-token: write and attestations: write.
The first publish creates each package as private, so docker pull
without signing in is refused. An organization owner makes each package
public once. A package appears only after its first publish: pushes to
main publish only bookclerk-dev, so bookclerk and bookclerk-demo
first appear after the nightly run or a manual run (see
When it runs).
- Open
https://github.com/orgs/<owner>/packages/container/<image>/settings(for examplehttps://github.com/orgs/bookclerk/packages/container/bookclerk/settings). - Under Danger Zone, choose Change visibility, then Public.
- Type the package name to confirm.
- Repeat for
bookclerk-devandbookclerk-demo.
Note
The organization setting that lets members create public packages (Settings > Packages > Package creation) only decides which visibilities new packages may get. It does not change packages that already exist.
Warning
GitHub does not let a public package go back to private. Later publishes keep the package's visibility, so this step is needed only once.
The desktop app's unsigned packages (see
Packages) are built by
.github/workflows/desktop-packages.yml:
a .deb and an AppImage on Linux, an NSIS installer on Windows, and a
.dmg on macOS. Each is a full release build on its own runner.
The images workflow calls it, so it runs on the images' schedule (see When it runs):
- nightly, and by hand, when
mainhas moved since the last publish; - for a published release, whose packages it attaches to the release.
A pull request runs it only when it changes the workflow or the packaging
configuration (tauri.packaging*.json, crates/bookclerk-dev/src/package.rs).
CI's desktop job covers the window's code on every pull
request. A change that breaks how the packaged app finds its runtime shows
up in the next nightly build.
Every build uploads the packages as workflow artifacts, kept for 14 days.
For a release, the publish job attaches them to the GitHub release with a
build-provenance attestation each. To check a downloaded package, replace
<owner> with the repository's owner:
gh attestation verify <downloaded package> --owner <owner>
Only that job holds contents: write, id-token: write and
attestations: write.
The package builds keep no Rust cache. Three release caches would not fit beside CI's caches in the repository's 10 GB, and a nightly build has no one waiting on it.
Publishing a release starts the image publish, the docs publish, the
website deploy and the Audiobookshelf integration test, each for the
commit the release's v* tag names. Those runs use that commit's own copy
of each workflow, so whoever can create or move a v* tag decides what
gets built, signed and published as a release. Three layers keep that to
reviewed code:
-
Rulesets. Only repository administrators may create a
v*tag, and nobody may move or delete one. The automation App can write contents, but it can't tag a release. To set them up, run this once as an administrator, with your own GitHub CLI login:bash scripts/setup-release-rulesets.sh Bookclerk/bookclerkIt creates, or updates in place, the Release tags: creators and Release tags: history rulesets. An administrator still publishes releases from the web UI as before, since that creates the tag.
-
Immutable releases. Turn on Settings → General → Releases → Enable release immutability, so a published release's tag and assets can't change afterwards.
-
The commit must be on
main. The image and docs publishes start with.github/actions/release-on-main, which fails unless the default branch contains the release's commit. The website deploy builds its code from the default branch and takes only the release'sdocs/pages from the tag.
Note
The check in step 3 runs from the tagged commit's own workflow, so it catches a tag put on the wrong commit by mistake, not a forged one: a commit written to drop the check would skip it. The rulesets are the control that stops that.
Note
The workflow below is an illustrative sketch for publisher-side packaging. Digest pinning, notarization and Authenticode requirements are set by follow-up install and trust work, not frozen by this page.
Per-OS matrix builds should run the same cargo aliases, then optionally sign or notarize:
# .github/workflows/release.yml (sketch — subject to change)
jobs:
package:
strategy:
matrix:
include:
- os: ubuntu-latest
target: x86_64-unknown-linux-gnu
- os: macos-latest
target: aarch64-apple-darwin
- os: windows-latest
target: x86_64-pc-windows-msvc
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v7
- uses: dtolnay/rust-toolchain@stable
- uses: actions/setup-node@v7
with:
node-version-file: .tool-versions
- run: cd ui && npm ci && npm run build
- run: cargo package-platform --out dist
- run: cargo package-plugins --out dist/plugins
# Signing (examples — pick per OS):
# - Linux: cosign sign-blob / minisign
# - macOS: codesign + notarytool (after package-platform)
# - Windows: Authenticode (signtool)
- uses: actions/upload-artifact@v4
with:
name: bookclerk-${{ matrix.target }}
path: dist/
Publish the release assets to GitHub Releases (or S3 or R2), and set
artifact_base_url in each plugin crate's [package.metadata.bookclerk]
before cargo publish on crates.io.
crates/bookclerk-dev/README.md: the development workflow aliases.- Plugin registry: the crates.io taxonomy.
- Repository automation: the jobs that build and bump these.
Source: docs/contributing/packaging.md at main · View as Markdown