Windows AppContainer confinement notes
This page records enforcement details and measured assumptions for the Windows spawn path. Green CI is necessary but not sufficient — each claim below ties to code or an explicit limitation.
Implemented in bookclerk-sandbox (windows_launch.rs), not rappct 0.13.3:
- Default:
CREATE_SUSPENDED→ Job limits (KILL_ON_JOB_CLOSE, memory / active-process / optional CPU) →AssignProcessToJobObject→ResumeThreadso no guest instruction runs before Job membership. - Optional: set
BOOKCLERK_AC_USE_JOB_LIST=1to tryPROC_THREAD_ATTRIBUTE_JOB_LISTfirst (with AppContainer caps + handle list). On hosts where that returnsERROR_INVALID_HANDLE(seen on GitHubwindows-latest), CreateProcess retries the suspended path automatically. - On any failure after
CreateProcessW,TerminateProcessthe child and close process, thread, pipe, and Job handles.
Test hook: BOOKCLERK_TEST_FAIL_JOB_ASSIGN=1 forces fail-closed teardown.
The host never nests an AppContainer. Microsoft documents that pipes created
inside a container must use \\.\pipe\LOCAL\ and that pipes cannot be used
between different AppContainers (Win10 1709+). Creating a second container
from inside one also fails. So bookclerk-workerd cannot spawn or name-connect
the native backend.
Instead the unsandboxed host:
- Creates two duplex links (guest stdio = two unidirectional pipes with the launcher end overlapped; proxy = one duplex overlapped pipe). Names are random. The DACL grants only the creating user, SYSTEM, and Administrators — not Everyone, Anonymous, or All Application Packages. Each pipe is the first instance, rejects remote clients, and is connected before the handle is delivered. The AppContainer guest uses that inherited handle; it does not open the pipe by name.
- Spawns two
bookclerk-jail.exeprocesses (gatewayLocalIpc, guestNetPolicy::Deny). Isolation::Off still goes through the jail withEnforcement::Disabledso handle handoff has a single-threaded parent. DuplicateHandles each link end into the matching jail, then writes one bounded JSON handoff line on that jail's stdin (BOOKCLERK_JAIL_HANDOFF=1). The jail marks the duplicates inheritable, puts them onPROC_THREAD_ATTRIBUTE_HANDLE_LIST, and exportshandle:<n>env. The list contains only those handed-off handles. Unrelated process handles stay off it. The host never marks a handle inheritable itself. A numeric handle value is not a session identity: the guest must write a per-session challenge on the proxy before the host serves mux frames.- Assigns both jail processes to one session Job (
KILL_ON_JOB_CLOSEplus the grant's aggregate memory / active-process / CPU limits). Each jail's own Job still nests under it.
The guest never opens a named pipe for RPC or SOCKET_PROXY. OAuth callback
pipes stay host-created and ACLed to the guest Package SID. Isolation::Required
fails closed if either profile or the session Job cannot be created.
Permanent tests: E2 (inherited pipe + ambient TCP denied), E2b (two AppContainers on one inherited duplex), E3 (same-container loopback for the workerd bridge), E5 (closing the session Job kills the tree).
GetAppContainerFolderPath is authoritative when the path is under Known Folder
LocalAppData \Packages\. Documented layout is
%LOCALAPPDATA%\Packages\<moniker>\AC; measured on Windows CI the API
returns %LOCALAPPDATA%\Packages\<package-SID>. Bookclerk then ensures the
\AC child exists and uses that as cwd / LOCALAPPDATA (TEMP/TMP →
AC\Temp). Fail closed on API failure or paths outside Packages — Bookclerk
does not synthesize a Packages path when the API fails.
Named mutex Local\bookclerk-dacl-tx (120s timeout, fail closed) around every
DACL RMW, plus an in-process mutex. The same pair serializes
CreateAppContainerProfile and DeleteAppContainerProfile: those Win32 calls
are not thread-safe, and four overlapping session launches plus each jail's
attach otherwise sit inside the API together. The lock is released before
the host waits for jail-ready, so the jail can attach while the host is
waiting. Ancestor traverse ACEs are written with
SetKernelObjectSecurity so inheritable ACEs already on a broad parent
(%TEMP%, a build directory) are not propagated to every child while that
mutex is held. Leaf directory grants still use inheritable ACEs.
Revoke does not invalidate already-open handles.
Capability SIDs and same-host loopback are different knobs.
internetClientServer / privateNetworkClientServer
authorize remote inbound (Internet / private LAN), not host↔AppContainer
localhost IPC. Same-machine host→guest TCP on loopback requires a
CheckNetIsolation LoopbackExempt
escape hatch that Microsoft documents as dev-only — and measured for
Bookclerk’s unpackaged CreateAppContainerProfile guests, even an active
-a/-is session did not restore host→guest connect.
There is no AppContainer capability SID for “talk to localhost.” Product code must not call CheckNetIsolation or ship a LoopbackExempt dependency.
For Bookclerk: the host owns the browser TCP listener and proxies bytes to the
guest over IPC (callback_proxy + callback_ipc on login.start). The Windows
named pipe is ACLed to that guest’s Package SID (GRGW) with a Low mandatory
label (S:(ML;;NW;;;LW)). See plugins.md (Interactive listeners).
| Plugins | Media | |
|---|---|---|
| Job memory (cumulative) | 512 MiB | 2 GiB |
| Active processes | overhead + extra (direct-native 1+2=3, isolate 2+2=4, native-behind gateway 2, guest 1+extra, outer session Job 5+extra) | 64 |
| CPU rate | 80% of one core hard cap on the outer session Job (sibling inner specs send "off" so the label default is not applied; standalone Jobs still set a percent) |
uncapped |
| Stderr proxy budget | 1 MiB | 16 MiB |
| data/tmp growth (plan + side-pass) | 512 MiB each | n/a |
| RPC timeout | kill + quarantine | n/a (stdio job) |
Defaults come from the jail label (plugin:… vs media-…) and grant/host
ceilings. A Spec may set memory_bytes / active_processes /
cpu_rate_percent explicitly. Omitted or null CPU is unspecified and takes the
label default; a number is a percent; "off" disables the rate and is not
filled. Memory uses Job Object job-wide commit charge
(JOB_OBJECT_LIMIT_JOB_MEMORY), aligned with Linux cgroup memory.max (main +
children). Limits are best-effort Job Object + host policy, not a hard
multi-tenant quota.
Cross-platform: Linux applies the same Spec fields via cgroup v2 into an
exclusive child cgroup when the hierarchy allows it (never onto a shared
parent slice, and never a leaf that already exists). The leaf stays inside
the process's current cgroup. Bookclerk does not place it on an ancestor.
When that cgroup has member processes, the host process moves into
bookclerk-host and the leaf is created beside it, still under the same
parent, so that parent's limits and cgroup.kill still cover the session.
pids.max counts threads.
It is the infrastructure thread budget plus extraProcesses, not the Windows
outer baseline (5 + extra). bookclerk-workerd pins its Tokio worker and
blocking-thread counts so that budget does not grow with CPU count. If a leaf
cannot be created, the
resources layer is reported not-applicable, process-group kill is the fallback
(it does not cover setsid), and Required still rests on FS/net. When the
leaf exists, teardown is cgroup.kill and a wait for populated 0. That
owns the leaf, including a setsid descendant. A missing cgroup.kill is
unsupported; Bookclerk does not signal pids from a membership snapshot. It is
a different path from process-group kill. macOS
Seatbelt cannot enforce memory/CPU/pids — Bookclerk reports that layer as not
applicable and does not fake enforcement.
The host keeps a journal of each session's package SID and the paths that
session's spec grants: inheritable leaf ACEs, no-inherit ancestor traverse, and
the profile folder plus Temp. After the session Job is closed and the
siblings have been reaped, on graceful exit and on failed startup, the host
deletes the AppContainer profiles and then revokes that journal under
Local\bookclerk-dacl-tx. The session directory is removed only when that
revoke returns success.
DeleteAppContainerProfile does not strip package-SID ACEs, and the profile
moniker (bc.<stem>.<hex>) is not the SID. Directory removal is the signal
that those ACEs are gone, and only on that success path. Before the jail
grants those ACEs, the host locks acl-journals/<session>.lock and writes the
complete journal to acl-journals/<session>.json in the plugin-state directory,
beside the session directory. The gateway jail can write only the session
directory, so it cannot rewrite those paths and SIDs or delete the record to
skip recovery. If that write fails, the spawn fails and no grants are applied.
The lock stays until the journal is dropped, so a concurrent session plan does
not treat this live directory as abandoned. Every non-empty journal write,
including the rewrite after a failed revoke, syncs the temp file and the
rename (sync_all, and MOVEFILE_WRITE_THROUGH on Windows). An empty journal
is deleted rather than renamed. On Windows the directory gets a protected
DACL for Administrators, SYSTEM, and the owner, so a parent inheritable write
ACE does not cover it. Revoke tries every entry and keeps only the ones that
failed. Open and try_lock of <session>.lock, and deletion of an orphan
lock, both hold acl-journals/.sweep.lock for that short section so one
process cannot unlink a lock another process has opened but not locked yet.
The sweep then holds <session>.lock until revoke and directory removal
finish, and deletes that lock only after success. A later sweep also deletes
a .lock whose .json is gone. If revoke later fails or stops partway, the
host replaces the file with the entries that failed and leaves the directory
in place. A failed replacement does not remove the earlier file. Host startup
retries every plugin-state/*/acl-journals/session-*.json, and the next
native-behind-workerd plan of that plugin retries again. A journal is accepted
only when every path is a local absolute path and it names at most two
AppContainer package SIDs with exactly seven RIDs after S-1-15-2-. That
excludes S-1-15-2-1 and S-1-15-2-2. A UNC share or a device path, including
a UNC local destination root, fails that check and the error names the path;
map a drive letter or use the S3 destination. A record that cannot be parsed
or fails that check is renamed to session-*.json.rejected and logged once.
Nothing revokes the grants in that file. The session directory stays. Sweeps
skip the renamed file. The log says other recovery can continue and that an
operator must read the rejected file and remove those ACEs by hand
(icacls <path> /remove *S-1-15-2-…). If revoke of the in-memory grants
failed in the same drop, those grants are written to a new session-*.json
so the next sweep can retry them.
bookclerk plugins uninstall --purge-state revokes that plugin's journal first
and leaves plugin-state in place if a readable record remains, if a live
session holds the lock, or if a .json.rejected file remains.
--discard-acl-journals (with --purge-state) deletes rejected records with
plugin state. It does not revoke the package-SID ACEs those records name.
After discard those ACEs stay on the host paths and the only list of them is
gone. Read the rejected file and remove the ACEs by hand
(icacls <path> /remove *S-1-15-2-…) before passing the flag. A file inside the session
directory is not a journal. Revoke is
idempotent, treats a missing path as
success, and removes only that session's SID. The jail may revoke as well when
its process exits normally. Job kill skips that Drop, so the host journal is
the owner that still runs. Ancestor traverse stays a SetKernelObjectSecurity
write on that directory alone.
Bookclerk uses regular AppContainer (capability SIDs + path ACLs + Job). Less Privileged AppContainer (LPAC) ambient restrictions are not applied.
Sandboxing ≠ publisher authentication. SHA-256 verifies artifact integrity, not publisher identity. Install UX should surface registry/publisher/network (and callback-transport if ever shipped) / signature before unattended install; unsigned installs need explicit approval. See plugins.md.
rappct 0.13.3 assigns the Job after the guest may already run and does not
resume suspended threads or reliably terminate on mid-launch failure. Bookclerk
bypasses launch_in_container_with_io for production launches and keeps rappct
for profile/SID/capability helpers only. File/track an upstream issue on
cpjet64/rappct describing:
CreateProcessthenAssignProcessToJobObjectrace with a runnable primary thread- no
ResumeThreadwhensuspended: true - failure after create does not reliably
TerminateProcess
Source: docs/internals/windows-appcontainer.md at main · View as Markdown