Dynamic plugins
This shelf is for people writing Bookclerk plugins. This page gives the overview: what a plugin is, how it runs, which SDK to use, and how to try one locally.
Bookclerk is built around pluggable sources, destinations and integrations.
- First-party guest implementations live under
crates/bookclerk-plugins/{platform,optional}/, one package per plugin. - The host runtime is
bookclerk-plugin-host: discovery, spawn, jail, consent and Workers RPC. It is distinct from the runtime install tree$BOOKCLERK_FILES_DIR/plugins/.
Every plugin, first-party or not, runs as a separate guest process
through bookclerk-workerd. There is no in-process fallback.
- plugin.toml: package layout, identity, runtime, entrypoints and settings.
- Capabilities and consent: what a plugin may ask for, and how the operator approves it.
- Entrypoints, events and the ABI: the Workers RPC surface, events, KV, queues and every capability's methods.
- Plugin registry: crate naming, release assets and the planned registry.
- SQL contract: what a database adapter must accept.
- Working within database limits: the caps a
[[databases]]binding meets, howall()streams large results, anditerate()andchunk(). - Plugin SDKs: the TypeScript, Python and Rust guides, each followed by the reference generated from its code.
Operators install, approve and place plugins as described in Managing plugins. How the host confines every guest is in The guest jail.
Authors write a Workers-shaped module:
- The default export extends
BookclerkEntrypointand implements the triggersplugin.tomldeclares:event(batch)for[[events.consumers]];job(job)for[triggers] jobs;scheduled(controller)for[triggers] crons;databaseMigrations(binding)for[[databases]];- plus an optional
describe()refinement.
- Every named entrypoint in
entrypoints = [...]is an exported class with the matching name, extending its base:Storefront,Storage,RemoteLibrary,DatabaseAdapter,Cli,Oidc,Metadata,Covers,MetadataFiles,NotifierorAi. - Native guests implement the Rust
PluginWorker(describe/open(invocation, bindings) -> Entrypoints) and callserve. On native guests,serveis the stdin/stdout Cap'n Proto runner forapi_version = 3.
The trusted adapter isolate installs the granted bindings on env for
each invocation: CONFIG, SECRETS, EVENTS, WORK_FS and named
databases. It never exposes HTTP endpoints, PIDs, credentials or Cap'n
Proto to authors.
- Native guests behind workerd are reached as typed Cap'n Proto by the launcher, never through an isolate binding.
- Byte
Sourceis the job input opener. Storefronts are the separately namedstorefrontentrypoint. - JSON is allowed only for plugin-specific extensible config
(
schemaVersionplusmediaType/schemaId, and a bounded payload).
Each language has its own guide and generated reference under
Plugin SDKs: TypeScript
(@bookclerk/plugin-sdk), Python
(bookclerk-plugin-sdk) and Rust
(bookclerk-plugin-sdk). That page also explains the two runtimes
plugin.toml can name, workerd and native.
Important
Third-party authors should depend on the SDK only, not on
bookclerk-plugin-host, bookclerk-library, bookclerk-source,
bookclerk-storefront or bookclerk-audible-catalog (AGPL crates that
first-party storefront guests share).
The product ABI is object-capability Workers RPC at
api_version = 3: capability entrypoints and triggers, transferred
ReadableStream and Cap'n Proto byte sources, and no public handleId,
writeChunk or base64 media.
- Native guests serve the Bookclerk Cap'n Proto schema on stdio. Workerd
guests keep isolate
RpcTargetstubs. - Each HTTP or RPC request is invocation-scoped. The trusted adapter opens the guest, invokes the method, and disposes the stubs before the request completes.
- Survival across suspend is checkpoint data plus stable binding
identifiers. It is never an in-memory
RpcTarget, an adapter map id, a PID or an open connection.
The authoritative artifacts are:
- the Cap'n Proto
schema/plugin.capnp(append-only ordinals; unknown union members fail closed or return a typedunsupported); - the TypeScript
packages/plugin-sdk/src/plugin.ts.
The decision records are ADR: Plugin capabilities v3 (the v3 surface) and ADR: Workers RPC and workerd (the jail and the workerd front door).
- Plugin host: Crate
bookclerk-plugin-host. Loads guests, and never ships storefront logic by default. - External guest: Any jailed subprocess plugin (platform, product, example or third-party).
- Platform guest: Bundled in the installer:
sqlite,localandembedderonly (plugins/platform/). - Optional plugin: Bookclerk-maintained optional guests (storefronts, ABS, s3, d1, postgres under
plugins/optional/). - Product plugin: Another name for optional first-party guests (packaged with
cargo package-plugins). - Reference example: Echo samples under
examples/. For CI andcargo dev --examplesonly, never packaged. - Third-party plugin: Outside this monorepo, with the same jail and Workers RPC ABI.
- Plugin package: A Rust crate under
crates/bookclerk-plugins/, or a workerd archive (plugin.toml+modules/). - In-process fallback: None. Storefronts, destinations and database adapters run as staged external guests, and physical lowering (
openSession,dropUnit, engine connect) stays in the adapter. - Verified artifact: Installed bytes that match the install receipt (
manifest_sha256+payload_root_sha256) for a provenance-qualified PluginKey. BookclerkEntrypoint: The workerd default-export base (event(batch)/job(job)/databaseMigrations(binding)/ optionaldescribe()/shutdown()), which extendsWorkerEntrypointin TypeScript. Named entrypoints are exported*Entrypointsubclasses, and Rust guests implementPluginWorker(describe/open(invocation, bindings) -> Entrypoints) +serveinstead.
A plugin's alias is its plugin.toml id. Aliases are for humans and
CLI ergonomics (bookclerk plugins info postgres). Durable ownership
(grants, sessions, databases, jobs, state directories, the install ledger)
uses the PluginKey.
Aliases are unique within one host plugin namespace
($BOOKCLERK_FILES_DIR), across kinds (source, integration, output,
database). Each alias maps to zero or one installed PluginKey.
- Duplicate aliases from different PluginKeys on the same host are
invalid installation state. Discovery fails closed, and install rejects
the second package, including with
--replace. - Separate Bookclerk hosts with their own files directories may install the same alias and PluginKey independently, even when they share a library database. Aliases are not globally unique across an HA library.
The grammar is strict and non-lossy: lowercase [a-z0-9_]{2,32}, with no
leading or trailing _ and no __ (the same rule as the crates.io {id}
segment). Invalid characters are rejected at manifest load and install,
never rewritten, so values such as a/b and a_b can't collide after
sanitization.
An installed PluginKey's alias can't change across an ordinary install,
update or replace. A package can't keep the same PluginKey while changing
id = "foo" to id = "foo2", because accounts keep source as the
display alias and CLI targeting resolves it.
Warning
An alias is not trust. A third-party package that is the only
installed sqlite still receives no platform secrets, no SQLite library
path and no first-party jail allowances.
cargo dev builds a lean platform set (daemon, helpers and platform
guests), installs the platform guests into $BOOKCLERK_FILES_DIR/plugins/,
and runs bookclerkd. It restarts rather than reloading in place.
Optional storefronts are opt-in:
export BOOKCLERK_FILES_DIR=./BookclerkFiles # optional; cargo-dev default
cargo dev # lean platform + daemon
cargo dev --optional # + optional storefronts
cargo dev --examples # + reference Echo examples
cargo dev-cli -- version # lean CLI path
Finer-grained aliases use the same dispatcher (see
crates/bookclerk-dev/README.md):
cargo build-app --platform # full default-members + platform guests
cargo build-app --optional --examples # optional + Cargo examples
cargo install-platform # sqlite + local → FILES_DIR/plugins
cargo stage-plugins --optional # optional → target/plugin-artifacts
cargo stage-plugins --examples --skip-build
cargo test-staged # describe/health/migrations conformance smoke (bookclerk-plugin-e2e)
- Add
--releaseto any alias for release builds. - Override the staging folder with
BOOKCLERK_PLUGIN_ARTIFACTS. - Pass host arguments after
--(for examplecargo dev -- --help).
First-party storefronts are always staged external guests. There is no in-process host feature.
The staged suite lives in crates/bookclerk-plugin-e2e, so ordinary
bookclerk-plugin-host tests never need a staged installation.
- Set
BOOKCLERK_STAGED_PLUGINS=libro,echo_workerd_tsto check a subset. Platform guests are always checked. - A new first-party plugin joins the crate's id list (
PLATFORM_PLUGINS,OPTIONAL_PLUGINSorEXAMPLE_PLUGINS). A unit test fails while aplugin.tomlundercrates/bookclerk-plugins/orexamples/is missing from its list, because a run that selects an unlisted id stops with "unknown plugin id". - Set
BOOKCLERK_REQUIRE_STAGED_PLUGINS=1to fail, instead of skipping, when staging roots are missing.
The same crate measures how long each staged guest takes to start, which sets the budget for starting asleep plugins on demand. The measurement is an ignored test, so run it explicitly against a staged installation:
BOOKCLERK_PLUGIN_ARTIFACTS="$PWD/target/plugin-artifacts" \
cargo test --release -p bookclerk-plugin-e2e --test cold_start -- --ignored --nocapture
It spawns every guest several times through the production path and
reports, per plugin, family and runtime, the ready time (the spawn, which
waits for the guest's first describe), a warm describe round trip and
the first health call.
- Set
BOOKCLERK_COLD_START_RUNSto the spawns per guest (default 5; the first is the coldest). - Set
BOOKCLERK_COLD_START_OUTto a folder to writecold-start.mdandcold-start.jsonthere. - Set
BOOKCLERK_COLD_START_CONTEXTto one line describing the machine, such as its CPU limit, for the report.
The Echo examples each have their own plugin id:
| Path | Runtime |
|---|---|
examples/plugins-echo-native-rust |
native Rust (api_version = 3) |
examples/plugins-echo-native-node |
workerd JS (id echo_native_node) |
examples/plugins-echo-native-python |
workerd Python (id echo_native_python) |
examples/plugins-echo-workerd-ts |
workerd TypeScript |
examples/plugins-echo-workerd-python |
workerd Python Workers |
examples/plugins-echo-workerd-rust |
workerd Rust/Wasm |
examples/plugins-echo-workerd-fetch |
workerd outbound *.example.com + logo |
To build the TypeScript example:
cd packages/plugin-sdk && npm ci && npm run build
cd ../../examples/plugins-echo-workerd-ts && npm ci && npm run typecheck
-
Stage the native Echo into the host plugin namespace. Don't copy it into a legacy
plugins/echo/alias folder.cargo stage-plugins --examplesThe installer places the guest at
plugins/<PluginKey fs-id>/(pk-*) and records host-owned ledger identity. Aliases such asechostay presentation handles for the CLI and config.For a workerd Echo, install
plugin.tomlandmodules/from any ofplugins-echo-workerd-ts/,plugins-echo-workerd-python/,plugins-echo-workerd-rust/orplugins-echo-workerd-fetch/(outbound*.example.comand an embedded logo). The host spawnsbookclerk-workerd, not a SEA binary. -
Approve, deploy and configure it, then call it:
$ bookclerk plugins approve echo --yes $ bookclerk plugins list $ bookclerk plugins deploy echo # deployment desired = present on this host $ bookclerk plugins config set echo greeting hello $ bookclerk integrations status echo enabled=true ok=true echo plugin ready $ bookclerk plugins echo ping --message hello pong: hello -
The TypeScript, Python and native Rust Echo examples (
echo_workerd_ts,echo_workerd_pythonandecho_native_rust) also keep anotestable on their own[[databases]]binding, a worked example of the database helpers:$ bookclerk plugins echo_workerd_ts notes-add --body "remember" --count 3 stored 3; 3 in total $ bookclerk plugins echo_workerd_ts notes-list 1: remember 2: remember 3: rememberRun their tests under every database profile with
cargo plugin -- test examples/plugins-echo-workerd-ts,cargo plugin -- test examples/plugins-echo-workerd-pythonorcargo plugin -- test examples/plugins-echo-native-rust.
What you ship depends on the runtime:
- Native: a folder (or archive) with
plugin.tomland the binary for the target OS and architecture. - Workerd or script:
plugin.tomlandmodules/, with no per-OS binary. The operator's Bookclerk install already includesbookclerk-workerd, the pinned Cloudflareworkerdbinary andbookclerk-jail.
Operators then:
- Install the package, with
bookclerk plugins install,POST /api/config/plugins, or a package folder underplugin-packages/. - Approve its capabilities.
- Deploy it with
bookclerk plugins deploy.
A development tree unpacked under plugins/ (or a
BOOKCLERK_PLUGIN_DIRS root) still runs as a host-local plugin. No
rebuild of Bookclerk is needed while api_version matches.
For crates.io naming, release-asset conventions, and installing
without Rust (the planned bookclerk plugins search|install and a
dashboard browser), see Plugin registry.
Audible, Libro.fm, Chirp, GraphicAudio and Audiobookshelf ship as
external plugins under crates/bookclerk-plugins/.
bookclerk-plugin-hostloads them throughload_sourcesandload_integrationsas staged guests, the same path third-party plugins use. The CLI and daemon call only those host helpers, never store crates by name.- Discovery skips a PluginKey that is already registered. A clashing display alias from a different PluginKey on the same host is invalid (fail closed).
The Audiobookshelf integration, the Open Library and Audnexus metadata
providers, the covers fetcher and the metadata_files reader are
first-party workerd TypeScript guests in the same tree
(crates/bookclerk-plugins/optional/integration-audiobookshelf,
metadata-openlibrary, metadata-audnexus, covers and
metadata-files). They have no Cargo package:
- Their source is in
src/.npm run buildcompiles it with TypeScript into themodules/that ships, and both are committed. CI'sworkerd_guestscheck rebuildsmodules/and fails when it differs from the committed copy. - The parsing and request planning sit apart from the network call, so the
unit tests (
npm test, Node's test runner) drive the same code the guest runs. The entrypoint module is tested with the SDK's isolate module replaced, because the real one only loads insideworkerd. - A guest with a plugin database (Audiobookshelf's
LISTENING_STATE) tests it under everybookclerk-db-testkitprofile through the SDK'screateTestDatabase, so its tests needBOOKCLERK_DB_TESTKITset to the built kit (or the kit onPATH). - The root
Cargo.tomlexcludes each of them from the plugin-tree member globs by name.
CI builds and stages plugin artifacts with cargo stage-plugins, as local
development does. Artifacts are not published to crates.io or GitHub
Releases yet.
cargo stage-plugins # build + stage to target/plugin-artifacts
cargo dev # stage + run bookclerkd
BOOKCLERK_PLUGIN_ARTIFACTS=/tmp/bc-plugins cargo stage-plugins # another staging folder
- plugin.toml: start your plugin's manifest.
- Capabilities and consent: ask for what your plugin needs.
- Entrypoints, events and the ABI: the methods your plugin implements.
- Stores reference and Integrations: the first-party plugins.
Source: docs/plugins/README.md at main · View as Markdown