# ADR: Workers RPC plugin ABI + per-jail embedded workerd

- **Status:** Accepted (partially superseded)
- **Date:** 2026-08-09
- **Updated:** 2026-09-25 (native-behind-workerd is two host-spawned sibling
  jails joined by inherited links; nesting is removed because Seatbelt and
  AppContainer cannot re-apply / cross-connect)
- **Superseded in part by:** [`plugin-capabilities-v3.md`](/docs/adr/plugin-capabilities-v3/index.md)
  (2026-09-10) for product `api_version = 3` — capability entrypoints, typed
  Cap'n payloads, `env.EVENTS` / outbox replace role factories and
  `HOST.notify`. Isolation, jail, network consent, workerd pin, and
  distribution-tier decisions below remain in force.

## Context

Bookclerk needs a multi-language plugin model where third-party authors ship
portable JS/TS (or Python) modules without per-OS binaries, while native Rust
guests remain available for DRM and heavy dependencies. The host↔plugin
contract must be **identical** across runtimes.

## Decision

1. **Product `api_version = 2`.** The ABI is an object-capability Workers RPC:
   role-specific classes (`BookclerkPlugin`, `Destination`, `Source`,
   `JobHandler`, `ContentSource`, `Integration`, `Database`), transferable byte
   streams, and **invocation-scoped** stub lifetime (create, invoke, dispose in
   one request). RPC carries bounded values and **stream/stub capabilities**.
   Cap'n Proto field/union ordinals are append-only; unknown members fail
   closed or return typed `unsupported`. `describe()` advertises `apiVersion`
   and `supportedRoles`; the verified install receipt (provenance + content
   hashes) is the host allowlist, not a publisher-signed manifest. Manifest
   `api_version` and `describe().apiVersion` must match
   `PRODUCT_API_VERSION`. Optional facilities use named `rpcFeatures`.
2. **Workerd is the control-plane front door; native jail is a backend.**
   - **Control plane:** invocation, policy, binding, lifecycle, and outcome
     always pass through the workerd entrypoint (or a generated backend-proxy
     entrypoint the first-party wrapper owns).
   - **Typed passthrough:** for a native guest the `bookclerk-workerd`
     launcher forwards every `Entrypoints` family to the guest as typed Cap'n
     Proto **without entering JavaScript** memory. Observable backpressure,
     cancellation, checksum, and error semantics stay identical to the isolate
     path.
   - **Cap'n Proto** is the launcher↔native protocol. Direct native Cap'n
     Proto to the host is a **host-selected** compatibility fallback — the
     plugin cannot request it to bypass policy.
   - **Bindings:** authors see frozen `BookclerkContext.bindings` (`HTTP` /
     `STORAGE` / `SECRETS` / `OAUTH`) and optional `ctx.native`. The trusted
     adapter sees private `AdapterEnv.PLUGIN_DESCRIBE` (the manifest
     projection) and answers `describe` / `open` policy for native guests. Do
     **not** freeze `env.NATIVE_PLUGIN`. The host executor owns the process
     tree and both sandboxes; it launches `bookclerk-workerd` and the verified
     native guest as **sibling** jails joined by host-created inherited links.
     The launcher never nests a jail and never chooses the backend path.
     Plugin-controlled input cannot choose the executable or weaken the
     sandbox.
   - **Native:** guests serve [`plugin.capnp`](https://github.com/Bookclerk/bookclerk/blob/main/crates/bookclerk-plugin-abi/schema/plugin.capnp)
     via `capnp-rpc` (`serve`). They do **not** speak newline JSON as the
     product ABI. Native DRM plugins do not implement Cloudflare’s private
     `worker-interface.capnp`; the host maps both adapters onto the same Rust
     traits.
3. **v1 newline JSON is removed.** Guests speak Cap'n Proto `api_version = 2`
   (or workerd Workers RPC mapped onto the same contract). Scalar JSON remains
   only as a versioned escape hatch for plugin-specific config, not as a
   transport envelope.
4. **Greenfield (no `protocol` key).** No dual-stack product ABI with legacy
   JSON-RPC stdio framing.
5. **Isolation:** instances are keyed by `(plugin_id, account_id)`. Different
   accounts / security principals **never execute concurrently in the same
   isolate**. One isolate per invocation is allowed for higher-risk work.
   Per-invocation grants are scoped, expiring, revocable, and operation-limited;
   they are revoked on completion, cancellation, fence loss, suspension, or
   disconnect. Within one plugin instance, capability delegation is allowed
   (Workers RPC stubs are transferable). The OS jail remains mandatory:
   workerd is [not a hardened sandbox](https://github.com/cloudflare/workerd#warning-workerd-is-not-a-hardened-sandbox).
   Pin advances via a daily CI job with a **7-day publish cooldown**
   (same supply-chain posture as Dependabot); see
   [packaging.md](/docs/contributing/packaging/index.md#cloudflare-workerd-pin).
   Local embed only — not Cloudflare cloud execution. No JS-less stdio shim.
6. **Network:**
   - **Workerd:** operator approves `capabilities.network.domains`. Isolate
     egress enforces the allowlist. **Every redirect hop is checked**, not
     only the initial host. Resolved IPs are checked against private / local /
     metadata ranges; DNS rebinding and Host/SNI mismatch are rejected
     (full IP/SNI enforcement in the native broker is follow-up — this ADR
     must not freeze initial-host-only or “native = coarse unrestricted”).
   - **Brokered HTTP** uses the same domain grants as isolate fetch. **Raw
     TCP, UDP, and listen are distinct capabilities.** Jail default-deny
     remains. Native outbound is not permanently coarse-unrestricted.
   - The OS jail for every workerd guest is `NetPolicy::LocalIpc`, whatever
     the grant says: the host↔isolate RPC bridge is a Unix socket and the
     isolate's egress leaves through the launcher's relay as tunnels the
     unsandboxed host checks and dials, so neither `bookclerk-workerd` nor
     `workerd` needs an IP socket (Linux seccomp refuses `AF_INET`,
     `AF_INET6` and `AF_PACKET`; macOS denies all network but Unix sockets;
     Windows keeps only in-container loopback for the bridge). The grant is
     enforced twice: by the egress worker inside the isolate, and by the
     host's tunnel check outside the jail, which holds even if the isolate
     is escaped. A denied grant gets no host link and a `blocked`
     `globalOutbound`. Until per-port fetch grants exist, the host admits a
     fetch host on any port. This replaces the earlier
     `NetPolicy::OutboundListen` jail, which left outbound `connect` open at
     the OS layer. The generated workerd config exposes a single listen socket
     (`rpc` → bridge); compatibility flags (the Python workers pair) do not
     add sockets.
7. **Consent UX:** CLI and web UI prompt before enable; grants persisted;
   capability widening re-prompts. The same covering grant is enforced at every
   external spawn and at privileged delivery (`config` / `secrets` / `work_fs` /
   `oauth`). Native outbound shows an explicit warning.
8. **`compatibility_date` newer than bundled workerd:** warn, still load, at the
   pin date. workerd refuses a date past `supported-compatibility-date.txt` in
   the binary (a compatibility date selects breaking runtime changes). The
   author date stays in `plugin.toml`. The isolate runs at the applied pin
   date. Host surfaces such as events and jobs do not yet choose behavior from
   that date; when they do, later behavior changes ship with an enable date
   in a release.
9. **`[workerd].limits`:** local workerd does **not** Cap'n Proto-enforce
   `cpuMs` / `subRequests`. Bookclerk clamps `cpu_ms` / `subrequests` (defaults
   and hard caps), injects `subrequests` into egress policy JSON, and the
   egress bridge counts hops **per egress invocation** (one plugin `fetch()` +
   redirect chain → 429) — Cloudflare-style per-invocation budgeting, not
   isolate-lifetime module state. Cross-`fetch` aggregation within a host RPC
   is deferred until that invocation unit is defined. Effective `cpu_ms` is
   logged at isolate start; OS-jail CPU enforcement is tracked separately.

## Consequences

- Manifests declare `runtime = "workerd" | "native"` and capability sections.
- Catalog script archives ship `plugin.toml` + `modules/` only.
- Native publishers still build per-arch when they need native capabilities.
- CI must fail on ABI schema drift vs generated SDK outputs.
- Every byte a guest can send the host crosses a fuzzed boundary per
  [Untrusted-input fuzzing](/docs/adr/untrusted-input-fuzzing/index.md): Cap'n Proto replies
  the granted HTTP channel and the mux frames (`mux_peer`) have targets
  today; CONNECT handshakes are still to add
  ([`docs/contributing/fuzzing.md`](/docs/contributing/fuzzing/index.md)).
- CI requires the pinned `workerd` binary for the workerd contract suite; the job
  fails if the workerd adapter cannot start (`BOOKCLERK_SKIP_WORKERD=1` is
  local native-only).

### Distribution tiers

| Tier | Contents |
| --- | --- |
| **Platform** | Hosts + jail + workerd + media-worker + `sqlite` + `local` |
| **Product** | Optional first-party guests (storefronts, ABS, s3, d1, postgres) |
| **Reference examples** | Echo under `examples/` — CI / `cargo dev --examples` only; never packaged |

Every guest is an **external** (jailed) subprocess, including platform sqlite/local.

## Non-goals

- Operator-side `cargo` / `rustc` / `npm` to install plugins.
- In-process Wasm/`cdylib` loading in the host.
- Marketing workerd alone as the security boundary (jail remains mandatory).
- Cloudflare-hosted Dynamic Workers for operator plugin execution.
- HMR for `cargo dev` (restart-based only).

## Follow-ups (deferred)

The native-behind-workerd executor is now the production path: the host
spawns every plugin through `bookclerk-workerd` (`SpawnPlan` in
`bookclerk-plugin-host`), `runtime = "native"` only selects the backend
behind the isolate, and a missing `bookclerk-workerd` / pinned `workerd` is
a hard spawn error in every `[plugins].isolation` mode. Direct host↔native
Cap'n Proto survives only as `SpawnTransport::DirectNativeDiagnostic` for
tests and diagnostics; no product binary selects it.

Native-behind-workerd guests are a **sibling** `NetPolicy::Deny` jail the
host starts next to the gateway, not a child of `bookclerk-workerd`.
`native_guest.rs` is gone. IPC is host-created links delivered as
`fd:<n>` / `handle:<n>` (`BOOKCLERK_GATEWAY_GUEST_RPC`,
`BOOKCLERK_SOCKET_PROXY`). The production CONNECT mux is served by the
unsandboxed host so granted loopback dials do not depend on AppContainer
loopback (no CheckNetIsolation exemption). Unjailed conformance may still
pass `BOOKCLERK_GATEWAY_PROXY` into `bookclerk-workerd`. Unix links are
socketpairs. Windows links are two unidirectional pipes
(`BOOKCLERK_GATEWAY_GUEST_RPC` read half,
`BOOKCLERK_GATEWAY_GUEST_RPC_WRITE` write half, guest proxy write half
`BOOKCLERK_SOCKET_PROXY_WRITE`) so a concurrent read cannot lock the write.
The guest has no
ambient `AF_INET` and no named endpoint. The gateway jail is
`LocalIpc`: its host↔isolate RPC bridge is a Unix socket. Gateway `TMPDIR` is a host-owned `session-<nonce>/` directory;
guest `TMPDIR` is its own scratch.

### Why nesting was removed

A confined process cannot apply a second OS jail on macOS or Windows:

- **macOS Seatbelt.** `sandbox_init` from inside an existing sandbox returns
  EPERM. Apple DTS and Seatbelt notes state a process cannot re-apply or
  tighten a profile. Nested `sandbox-exec` fails even when the outer
  profile is `(allow default)`. Profile tuning cannot fix this.
- **Windows AppContainer.** Pipes created inside a container must use
  `\\.\pipe\LOCAL\`, and pipes cannot be used between different
  AppContainers (Win10 1709+). Creating a nested container from inside one
  also fails. Host↔container loopback stays blocked; same-container
  loopback (launcher → `workerd`) is the E3 contract.
- **Linux** Landlock domains stack and seccomp Deny only bans
  `socket(AF_INET/AF_INET6/AF_PACKET)`, which is why the nested topology
  passed there and hid the other two OSes.

The replacement applies each jail once from the unsandboxed host and
reproduces the **intersection** of the old two layers (not a copy of the
inner `deny_spec_with`). Windows `Isolation::Off` still requires
`bookclerk-jail.exe` beside the host so handle handoff can run with
`Enforcement::Disabled`.

Still deferred: OAuth/listen broker beyond the existing callback proxy,
container executor, and VPS benchmarks.

Instances are keyed by `(plugin_id, account_id)`. Shared-isolate concurrent
principals are not a proven isolation boundary (stubs are transferable).

When Bookclerk defines a Cloudflare-comparable **invocation unit** (host RPC ≈
one Worker invocation) for `cpu_ms` / subrequest aggregation, plan to evolve
lifecycle and tenancy as follows — **no implementation in this ADR revision**:

1. **Invocation-style budgeting** — treat a host RPC (or an explicit job
   envelope) as the CF invocation boundary so isolate `cpu_ms` and subrequests
   aggregate like Workers, not as isolate-lifetime module state.
2. **Per-account instantiation** — for account-bearing kinds (especially
   `source` / OAuth storefronts), prefer a jail (and workerd isolate) keyed by
   `(plugin_id, account_id)` so linked users do not share process memory or
   `plugin-state/<PluginKey fs-id>/data`. Resource caps (`extraProcesses`, memory, CPU) stay
   **per instance**.
3. **Lifecycle by kind** — split long-lived vs one-shot:
   - **Long-lived:** platform DB/output and other shared infrastructure guests
     that amortize connection state.
   - **One-shot / short-lived:** high-trust-boundary or bursty work (and,
     eventually, many storefront account sessions) closer to Workers
     request/isolate churn.
4. Until then, keep host hygiene (prefer one account per privileged RPC) and
   document that enabling a shared storefront guest trusts it with every
   account the host sends it.
