ADR: Workers RPC plugin ABI + per-jail embedded workerd
- Status: Accepted (partially superseded)
- Date: 2026-08-09
- Updated: 2026-09-25 (native-behind-workerd is two host-spawned sibling jails joined by inherited links; nesting is removed because Seatbelt and AppContainer cannot re-apply / cross-connect)
- Superseded in part by:
plugin-capabilities-v3.md(2026-09-10) for productapi_version = 3— capability entrypoints, typed Cap'n payloads,env.EVENTS/ outbox replace role factories andHOST.notify. Isolation, jail, network consent, workerd pin, and distribution-tier decisions below remain in force.
Bookclerk needs a multi-language plugin model where third-party authors ship portable JS/TS (or Python) modules without per-OS binaries, while native Rust guests remain available for DRM and heavy dependencies. The host↔plugin contract must be identical across runtimes.
- Product
api_version = 2. The ABI is an object-capability Workers RPC: role-specific classes (BookclerkPlugin,Destination,Source,JobHandler,ContentSource,Integration,Database), transferable byte streams, and invocation-scoped stub lifetime (create, invoke, dispose in one request). RPC carries bounded values and stream/stub capabilities. Cap'n Proto field/union ordinals are append-only; unknown members fail closed or return typedunsupported.describe()advertisesapiVersionandsupportedRoles; the verified install receipt (provenance + content hashes) is the host allowlist, not a publisher-signed manifest. Manifestapi_versionanddescribe().apiVersionmust matchPRODUCT_API_VERSION. Optional facilities use namedrpcFeatures. - Workerd is the control-plane front door; native jail is a backend.
- Control plane: invocation, policy, binding, lifecycle, and outcome always pass through the workerd entrypoint (or a generated backend-proxy entrypoint the first-party wrapper owns).
- Typed passthrough: for a native guest the
bookclerk-workerdlauncher forwards everyEntrypointsfamily to the guest as typed Cap'n Proto without entering JavaScript memory. Observable backpressure, cancellation, checksum, and error semantics stay identical to the isolate path. - Cap'n Proto is the launcher↔native protocol. Direct native Cap'n Proto to the host is a host-selected compatibility fallback — the plugin cannot request it to bypass policy.
- Bindings: authors see frozen
BookclerkContext.bindings(HTTP/STORAGE/SECRETS/OAUTH) and optionalctx.native. The trusted adapter sees privateAdapterEnv.PLUGIN_DESCRIBE(the manifest projection) and answersdescribe/openpolicy for native guests. Do not freezeenv.NATIVE_PLUGIN. The host executor owns the process tree and both sandboxes; it launchesbookclerk-workerdand the verified native guest as sibling jails joined by host-created inherited links. The launcher never nests a jail and never chooses the backend path. Plugin-controlled input cannot choose the executable or weaken the sandbox. - Native: guests serve
plugin.capnpviacapnp-rpc(serve). They do not speak newline JSON as the product ABI. Native DRM plugins do not implement Cloudflare’s privateworker-interface.capnp; the host maps both adapters onto the same Rust traits.
- v1 newline JSON is removed. Guests speak Cap'n Proto
api_version = 2(or workerd Workers RPC mapped onto the same contract). Scalar JSON remains only as a versioned escape hatch for plugin-specific config, not as a transport envelope. - Greenfield (no
protocolkey). No dual-stack product ABI with legacy JSON-RPC stdio framing. - Isolation: instances are keyed by
(plugin_id, account_id). Different accounts / security principals never execute concurrently in the same isolate. One isolate per invocation is allowed for higher-risk work. Per-invocation grants are scoped, expiring, revocable, and operation-limited; they are revoked on completion, cancellation, fence loss, suspension, or disconnect. Within one plugin instance, capability delegation is allowed (Workers RPC stubs are transferable). The OS jail remains mandatory: workerd is not a hardened sandbox. Pin advances via a daily CI job with a 7-day publish cooldown (same supply-chain posture as Dependabot); see packaging.md. Local embed only — not Cloudflare cloud execution. No JS-less stdio shim. - Network:
- Workerd: operator approves
capabilities.network.domains. Isolate egress enforces the allowlist. Every redirect hop is checked, not only the initial host. Resolved IPs are checked against private / local / metadata ranges; DNS rebinding and Host/SNI mismatch are rejected (full IP/SNI enforcement in the native broker is follow-up — this ADR must not freeze initial-host-only or “native = coarse unrestricted”). - Brokered HTTP uses the same domain grants as isolate fetch. Raw TCP, UDP, and listen are distinct capabilities. Jail default-deny remains. Native outbound is not permanently coarse-unrestricted.
- The OS jail for every workerd guest is
NetPolicy::LocalIpc, whatever the grant says: the host↔isolate RPC bridge is a Unix socket and the isolate's egress leaves through the launcher's relay as tunnels the unsandboxed host checks and dials, so neitherbookclerk-workerdnorworkerdneeds an IP socket (Linux seccomp refusesAF_INET,AF_INET6andAF_PACKET; macOS denies all network but Unix sockets; Windows keeps only in-container loopback for the bridge). The grant is enforced twice: by the egress worker inside the isolate, and by the host's tunnel check outside the jail, which holds even if the isolate is escaped. A denied grant gets no host link and ablockedglobalOutbound. Until per-port fetch grants exist, the host admits a fetch host on any port. This replaces the earlierNetPolicy::OutboundListenjail, which left outboundconnectopen at the OS layer. The generated workerd config exposes a single listen socket (rpc→ bridge); compatibility flags (the Python workers pair) do not add sockets.
- Workerd: operator approves
- Consent UX: CLI and web UI prompt before enable; grants persisted;
capability widening re-prompts. The same covering grant is enforced at every
external spawn and at privileged delivery (
config/secrets/work_fs/oauth). Native outbound shows an explicit warning. compatibility_datenewer than bundled workerd: warn, still load, at the pin date. workerd refuses a date pastsupported-compatibility-date.txtin the binary (a compatibility date selects breaking runtime changes). The author date stays inplugin.toml. The isolate runs at the applied pin date. Host surfaces such as events and jobs do not yet choose behavior from that date; when they do, later behavior changes ship with an enable date in a release.[workerd].limits: local workerd does not Cap'n Proto-enforcecpuMs/subRequests. Bookclerk clampscpu_ms/subrequests(defaults and hard caps), injectssubrequestsinto egress policy JSON, and the egress bridge counts hops per egress invocation (one pluginfetch()+ redirect chain → 429) — Cloudflare-style per-invocation budgeting, not isolate-lifetime module state. Cross-fetchaggregation within a host RPC is deferred until that invocation unit is defined. Effectivecpu_msis logged at isolate start; OS-jail CPU enforcement is tracked separately.
- Manifests declare
runtime = "workerd" | "native"and capability sections. - Catalog script archives ship
plugin.toml+modules/only. - Native publishers still build per-arch when they need native capabilities.
- CI must fail on ABI schema drift vs generated SDK outputs.
- Every byte a guest can send the host crosses a fuzzed boundary per
Untrusted-input fuzzing: Cap'n Proto replies
the granted HTTP channel and the mux frames (
mux_peer) have targets today; CONNECT handshakes are still to add (docs/contributing/fuzzing.md). - CI requires the pinned
workerdbinary for the workerd contract suite; the job fails if the workerd adapter cannot start (BOOKCLERK_SKIP_WORKERD=1is local native-only).
| Tier | Contents |
|---|---|
| Platform | Hosts + jail + workerd + media-worker + sqlite + local |
| Product | Optional first-party guests (storefronts, ABS, s3, d1, postgres) |
| Reference examples | Echo under examples/ — CI / cargo dev --examples only; never packaged |
Every guest is an external (jailed) subprocess, including platform sqlite/local.
- Operator-side
cargo/rustc/npmto install plugins. - In-process Wasm/
cdylibloading in the host. - Marketing workerd alone as the security boundary (jail remains mandatory).
- Cloudflare-hosted Dynamic Workers for operator plugin execution.
- HMR for
cargo dev(restart-based only).
The native-behind-workerd executor is now the production path: the host
spawns every plugin through bookclerk-workerd (SpawnPlan in
bookclerk-plugin-host), runtime = "native" only selects the backend
behind the isolate, and a missing bookclerk-workerd / pinned workerd is
a hard spawn error in every [plugins].isolation mode. Direct host↔native
Cap'n Proto survives only as SpawnTransport::DirectNativeDiagnostic for
tests and diagnostics; no product binary selects it.
Native-behind-workerd guests are a sibling NetPolicy::Deny jail the
host starts next to the gateway, not a child of bookclerk-workerd.
native_guest.rs is gone. IPC is host-created links delivered as
fd:<n> / handle:<n> (BOOKCLERK_GATEWAY_GUEST_RPC,
BOOKCLERK_SOCKET_PROXY). The production CONNECT mux is served by the
unsandboxed host so granted loopback dials do not depend on AppContainer
loopback (no CheckNetIsolation exemption). Unjailed conformance may still
pass BOOKCLERK_GATEWAY_PROXY into bookclerk-workerd. Unix links are
socketpairs. Windows links are two unidirectional pipes
(BOOKCLERK_GATEWAY_GUEST_RPC read half,
BOOKCLERK_GATEWAY_GUEST_RPC_WRITE write half, guest proxy write half
BOOKCLERK_SOCKET_PROXY_WRITE) so a concurrent read cannot lock the write.
The guest has no
ambient AF_INET and no named endpoint. The gateway jail is
LocalIpc: its host↔isolate RPC bridge is a Unix socket. Gateway TMPDIR is a host-owned session-<nonce>/ directory;
guest TMPDIR is its own scratch.
A confined process cannot apply a second OS jail on macOS or Windows:
- macOS Seatbelt.
sandbox_initfrom inside an existing sandbox returns EPERM. Apple DTS and Seatbelt notes state a process cannot re-apply or tighten a profile. Nestedsandbox-execfails even when the outer profile is(allow default). Profile tuning cannot fix this. - Windows AppContainer. Pipes created inside a container must use
\\.\pipe\LOCAL\, and pipes cannot be used between different AppContainers (Win10 1709+). Creating a nested container from inside one also fails. Host↔container loopback stays blocked; same-container loopback (launcher →workerd) is the E3 contract. - Linux Landlock domains stack and seccomp Deny only bans
socket(AF_INET/AF_INET6/AF_PACKET), which is why the nested topology passed there and hid the other two OSes.
The replacement applies each jail once from the unsandboxed host and
reproduces the intersection of the old two layers (not a copy of the
inner deny_spec_with). Windows Isolation::Off still requires
bookclerk-jail.exe beside the host so handle handoff can run with
Enforcement::Disabled.
Still deferred: OAuth/listen broker beyond the existing callback proxy, container executor, and VPS benchmarks.
Instances are keyed by (plugin_id, account_id). Shared-isolate concurrent
principals are not a proven isolation boundary (stubs are transferable).
When Bookclerk defines a Cloudflare-comparable invocation unit (host RPC ≈
one Worker invocation) for cpu_ms / subrequest aggregation, plan to evolve
lifecycle and tenancy as follows — no implementation in this ADR revision:
- Invocation-style budgeting — treat a host RPC (or an explicit job
envelope) as the CF invocation boundary so isolate
cpu_msand subrequests aggregate like Workers, not as isolate-lifetime module state. - Per-account instantiation — for account-bearing kinds (especially
source/ OAuth storefronts), prefer a jail (and workerd isolate) keyed by(plugin_id, account_id)so linked users do not share process memory orplugin-state/<PluginKey fs-id>/data. Resource caps (extraProcesses, memory, CPU) stay per instance. - Lifecycle by kind — split long-lived vs one-shot:
- Long-lived: platform DB/output and other shared infrastructure guests that amortize connection state.
- One-shot / short-lived: high-trust-boundary or bursty work (and, eventually, many storefront account sessions) closer to Workers request/isolate churn.
- Until then, keep host hygiene (prefer one account per privileged RPC) and document that enabling a shared storefront guest trusts it with every account the host sends it.
Source: docs/adr/plugin-workers-rpc-workerd.md at main · View as Markdown