Media worker pool

Development These docs follow the development version and change with every merge.

Last updated View as Markdown

This page explains the media worker: the confined helper process that decodes, encodes and packages audio, re-encodes cover images, and reads the tags embedded in media files. It covers how to install and configure it, and what to check when media work fails.

What it does

Every decode, encode and packaging step runs in a short-lived bookclerk-media-worker process, confined to the paths its job declared. On Linux and macOS the worker confines itself; on Windows bookclerk-jail confines it at spawn.

Acquire feeds audio from outside sources through C libraries: LAME (mp3lame-sys), FDK-AAC (fdk-aac-sys), libopus (opusic-sys), the FFmpeg decoders in release builds (bookclerk-libav) and Symphonia's native dependencies. Before the pool, those parsers ran on tokio's blocking threads, inside the process that holds the master encryption key and an open library.db. A parser bug there would have disclosed the key, not just failed a book.

Moving the work out gives three properties:

  • The codecs can't reach key material. The jail allows the job's declared inputs and one output folder. master.key, library.db and the rest of $HOME are not allowed.
  • A crash costs one book. A decoder segfault becomes a failed job, not a dead daemon. The LAME bug fixed in mp3.rs (writing past its output buffer) used to take the daemon down with it.
  • You choose the concurrency. media.workers caps how many codec jobs run at once, instead of however many threads tokio's blocking pool grows under load.

Uploaded archives

An uploaded ZIP or tar archive is expanded by the extract_archive job into an empty folder the host made. The walk refuses links, devices, encrypted entries, nested archives and names that would leave the folder, and counts the bytes it writes against caps on entries, total size, entry size and compression ratio. The host checks the worker's report against the same caps and the files on disk before it moves anything. See Uploads.

What audio it reads

The worker recognises a file's container and codec from its bytes, never from its name, so a file named .mp3 that holds an MP4 is read as an MP4. It reads these files:

Container Codecs
MP4 (.m4a, .m4b) AAC, ALAC, FLAC, Opus, and E-AC-3 and AC-3 in release builds
MPEG audio (.mp3, .mp2) MP3, MP2, MP1
ADTS (.aac) AAC
FLAC FLAC
Ogg (.opus, .ogg, .oga) Opus, Vorbis
WAV, AIFF, CAF PCM, IMA ADPCM
ASF (.wma), release builds WMA 1, WMA 2, WMA 9 Pro, WMA 9 Lossless
Raw Dolby (.ac3, .eac3), release builds AC-3, E-AC-3
AMR (.amr, .awb, .3gp), release builds AMR-NB, AMR-WB

Every AAC profile decodes through FDK: AAC-LC, HE-AAC, HE-AAC v2 and xHE-AAC (USAC). An HE-AAC book is reported and converted at the rate it plays at, not the half rate an MP4 header states for it. Opus decodes through libopus at 48 kHz, mono or stereo; a surround Opus stream (more than one stream in the file) fails to open. AC-4 isn't read by any build.

The rows marked release builds decode through FFmpeg's decoders. Every release archive and the Docker image carry them; a plain cargo build doesn't (see FFmpeg decoders).

Surround is folded to stereo by speaker position: left speakers go left, right speakers go right, centre and rear-centre speakers go to both sides at -3 dB, and the LFE channel is left out. Channels with no position go to both sides equally. A file whose sample rate changes partway through fails to convert rather than losing the audio after the change.

The worker writes AAC-LC M4B, MP3 and Ogg Opus. With output.keep_codec_in_m4b on, an M4B also carries Opus (Opus with dOps), FLAC (fLaC with dfLa), ALAC, E-AC-3 or AC-4 audio copied unchanged from the source, with the Opus pre-skip as an edit list (see Keeping unusual audio in M4B). It writes tags, covers and chapters into MP4 and MP3 files, and into FLAC, Ogg Opus and Ogg Vorbis files as Vorbis comments: TITLE, ALBUM, ARTIST, ALBUMARTIST, COMPOSER (the narrator), DATE, GENRE, PUBLISHER, DESCRIPTION, LANGUAGE, SERIES, SERIES-PART, ASIN, ISBN, chapters as CHAPTERnnn and CHAPTERnnnNAME (up to 999), and the cover as a front-cover picture. Opus output is encoded at 48 kHz: audio at another rate is converted with a windowed-sinc resampler first. Every M4B it writes is faststart: the sample table comes before the audio, so a player can start the book from its first bytes. The sample table keeps 1 MiB of room for tags, a cover and chapters, so adding them later never moves the audio.

When an output re-encodes or repackages audio, the acquire takes another store's copy from a library only if the worker reads its kind of file and the tag reader names its codec. That means AAC or ALAC in MP4, MP3, FLAC, Opus or Vorbis in Ogg, and in release builds E-AC-3 in MP4. WAV, AIFF, CAF, ADTS, WMA, raw AC-3 and AMR files decode, but a copy in one of them isn't taken, because its codec isn't recorded. The lists the acquire checks are in bookclerk_media::capabilities. The daemon serves the same report at GET /api/status/media, and bookclerk media capabilities prints it:

$ bookclerk media capabilities
decodes: aac, alac, flac, mp3, opus, vorbis
reads:   audio/mp4, audio/mpeg, audio/flac, audio/aac, audio/wav, audio/aiff, audio/x-caf, audio/ogg, audio/opus
encodes: m4b, mp3, opus (advanced)

A release build on Linux or macOS also lists eac3 under decodes, and audio/x-ms-wma, audio/ac3, audio/eac3, audio/amr, audio/amr-wb and audio/3gpp under reads.

An encoding marked advanced is one fewer players open than M4B and MP3, so settings offer it under their advanced options.

Tagging any other container, such as WAV, fails with a message naming it, and the acquire stores the file without the new tags. Files kept as delivered (output.format = "none") are not decoded at all, so any format can be stored that way.

Cover images

Cover art is downloaded from any public website, so Bookclerk never shows or stores the downloaded bytes. One long-lived bookclerk-media-worker covers process turns each download into the JPEGs Bookclerk serves:

  1. It checks the file signature and refuses anything but JPEG, PNG and WebP (no SVG, GIF or HTML).
  2. It decodes with pure-Rust codecs under limits: at most 10 MiB in, 5000 px a side and 128 MiB of decode memory.
  3. It resizes with a Lanczos filter and writes three 4:4:4 JPEGs: 128 px and 360 px (quality 85) and the full image up to 2400 px (quality 90, progressive).

Unlike a media job, a cover needs no file, so this process is confined to no path and no network before it reads its first image, and then serves covers until the daemon stops. It decodes on media.workers threads, covers don't wait for a job slot, and a cover that panics the decoder fails alone. A cover that takes longer than 30 seconds is refused and the process is stopped; when the process stops or dies, the next cover starts a new one. With media.isolation = "off" covers are processed in the daemon, like media jobs.

A page of covers fits a one-core node: the budget test in bookclerk-cover processes a 20-cover page, one large cover among them, one after another and fails when an optimized build takes longer than 2 seconds. Run it with cargo test --release -p bookclerk-cover --test budget -- --nocapture to see the time on your hardware.

Embedded tags

Files already on storage carry their own metadata: MP4 ilst items and chapter tracks, ID3 tags in MP3s, and Vorbis comments in FLAC and Ogg files. Anyone who can put a file on storage controls those bytes, so they are read by one long-lived bookclerk-media-worker tags process, never by the daemon. For each file it reports:

  • the text entries (title, album, artist, composer, genre, freeform items such as com.pilabor.tone:AUDIBLE_ASIN and TXXX:ASIN, and so on), as the file holds them;
  • the chapters, from the QuickTime chapter track, Nero chpl, ID3 CHAP frames or Vorbis CHAPTERnnn comments;
  • the running time, and whether the container states it or it was estimated;
  • the audio encoding (AAC, ALAC, FLAC, MP3, Opus, Vorbis, E-AC-3 or AC-4), from the audio track's sample description in an MP4 file or the first packet of an Ogg stream;
  • where the cover picture lies in the file, its format and its SHA-256, without the picture itself;
  • a SHA-256 digest of the metadata region (versioned), which changes when a tag is edited and not when the audio is.

The worker opens no file. It asks the daemon for the byte ranges it needs, and the daemon fetches them from wherever the file lives, so a file on S3 is read with ranged GETs, just like a local one. Only box and frame headers and the metadata region are fetched, never the audio: the sample tables of a long book's audio track are skipped by size. Files are read in pages, up to 16 at once, and don't wait for a job slot.

Like the cover process, the tag process is confined to no path and no network before it reads its first frame, and serves files until the daemon stops. The daemon treats it as compromised: it fetches only ranges inside the file, caps each file at 128 rounds and 16 MiB, holds at most 64 MiB of fetched bytes at once, stops the process when a round takes longer than 30 seconds or it asks for more than that, and checks every record against the reader's limits before using it. A file the reader can't parse fails alone. With media.isolation = "off" tags are read in the daemon.

The reader lives in bookclerk-tags and is fuzzed by the tags_read and tags_wire targets (see Fuzzing).

Audio for the desktop player

The desktop window's player can't stream audio to its webview, so the window decodes it natively instead (see Playback). The decoding happens in a bookclerk-media-worker pcm process, one for each track being played, never in the window's process. The worker:

  1. Reads the track's size and the starting position from the host.
  2. Asks for the byte ranges it needs, 256 KiB at a time. The host reads them through the play session's stream route, so every block passes the same checks a browser's request does.
  3. Seeks to the start, reports the sample rate, channel count (mono or stereo), actual start and running time, then sends 16-bit samples until the track ends.

Like the tag process, it is confined to no path and no network before it reads its first frame. The host treats it as compromised. It answers only ranges inside the track, and stops the process when the format or a range request takes longer than 30 seconds or the process asks for more than the track's size plus 64 MiB in all. It checks every frame against the wire's limits and the order the protocol allows. Seeking or closing the track stops the process.

Playback never decodes in-process: with media.isolation = "off" the desktop player refuses to play. The frames are fuzzed by the media_pcm_wire target (see Fuzzing).

What a stored file's row says

Every audio file an acquire writes is probed once it is on its destination, through the same tag process, and the result is kept on the file's row in the database (edition_files):

Value Meaning
Media type audio/mp4, audio/mpeg or audio/flac, from the container the reader found, not from the file name
Bitrate The file's size over its running time, in kbps: an average that includes the cover and tags. Empty when the running time is unknown
Lossless Whether the audio track's encoding is lossless (FLAC, or ALAC in an MP4 file)
Source store The store the copy was downloaded from, such as audible or libro
Copy Which of the library's copies the file belongs to, by format and quality (mp3 and 64k-mono); see Keeping several copies
Codec The audio encoding: aac, alac, flac, mp3, opus, vorbis, eac3 or ac4
AAC profile For AAC, the profile the decoder configuration names: lc, he, he_v2 or xhe. HE-AAC that is signalled only inside the audio frames reads as lc
Channels and sample rate As the stream's headers state them; for HE-AAC, the rate after band replication
Duration The running time a player sees, in milliseconds: an MP4's movie duration (which edit lists shape), an Opus stream's samples past its pre-skip, an MP3's frame count. Empty when the file states none, never 0

Nothing is decoded to find these values. The codec, profile, channels, sample rate and duration are stored only for files an acquire on this node probed; an upload's row carries the sender's media type, bitrate, lossless flag and store, and the rest stays empty.

The row answers "what copy do we hold?" without opening the file again, so comparing two copies of one edition, or telling another install what is on offer, is a database read.

A probe never decides whether a file is kept. When the tag process is missing and media.isolation is required, or the reader cannot parse the file, the file is stored as before and its row stays unprobed.

A host or a branch that acquires a book probes it the same way before uploading, and sends the bitrate, lossless flag and source store beside the file's media type. The library checks them with the manifest (a bitrate up to 20,000 kbps, a plain store id) and stores them when the upload commits, so an upload that is aborted or never finishes describes nothing. The values are the sender's word about its own file. They come from a node the library already trusts with the title, and they are only used to choose between two copies of the same edition.

Note

Files stored before probes were recorded have unprobed rows. They are not probed again by this change.

Chapters an acquire records

An acquire writes chapters into the files and also records, in the database, every chapter list it computed for the edition: the markers the store returned (or one chapter per delivered part when it returned none), the layout a metadata provider answered, and the provider's chapters after the worker's align_chapters job snapped them to the spoken titles. Each is a claim from its own source, and the library chooses the list the edition plays (see Chapters). A failed record is logged and never fails the acquire, because the files already hold their chapters.

Install the worker

Build the worker and ship it with the hosts, beside the plugin jail's launcher (found the same way):

cargo build --release -p bookclerk-cli -p bookclerkd -p bookclerk-media-worker -p bookclerk-jail

Add --features bookclerk-media-worker/libav for the FFmpeg decoders, as cargo build-app --release and cargo package-platform do.

The Docker image ships it in the read-only bundle under /opt/bookclerk, next to bookclerk and bookclerkd (see Running Bookclerk → Docker).

The pool looks for the worker in this order:

  1. media.worker_bin in config.toml
  2. BOOKCLERK_MEDIA_WORKER
  3. bookclerk-media-worker beside the running executable

Each candidate must exist, so a stale configured path fails loudly instead of quietly encoding unconfined.

FFmpeg decoders

WMA, AC-3, E-AC-3 and AMR decode through FFmpeg, built from source by the bookclerk-libav crate behind the worker's libav feature:

  • The build script downloads one pinned FFmpeg release from ffmpeg.org, checks its SHA-256, and builds only those decoders, their demuxers (ASF, raw AC-3/E-AC-3, raw AMR and MP4/3GP) and the AC-3 parser. It builds no encoders, filters, protocols, devices, hardware decoders or programs, and none of FFmpeg's GPL or nonfree parts.
  • On Linux and macOS it needs curl, tar with xz support, a POSIX shell and make.
  • On Windows it builds with MSVC (x64 and arm64): FFmpeg's configure runs under MSYS2's shell, which needs its make package (pacman -S make). MSYS2 is found at C:\msys64, or at MSYS2_ROOT when that is set.
  • FFmpeg decodes single-threaded inside the worker's confinement, like every other codec. It never opens a file or a network connection: the worker hands it the bytes.
  • The worker tries Symphonia first. A file that starts like ASF, raw AMR or a raw AC-3 stream goes straight to FFmpeg, and so does a file Symphonia can't read or whose audio it has no decoder for, such as AMR in a 3GP file.

The release builds of the repository's cargo aliases turn the feature on by themselves: cargo build-app --release, cargo package-platform and cargo install-platform --release. A plain cargo build --release needs the --features flag shown in Install the worker. Tests run with it too, so the first test build downloads FFmpeg and takes about half a minute longer (a few minutes on Windows).

Configure it

[media]
workers = 0             # 0 derives one per core, capped at 8; 1 when memory.max <= 1 GiB or cpu.max is one core
isolation = "required"  # required | best-effort | off
# worker_bin = "/usr/local/bin/bookclerk-media-worker"

Environment variables win over TOML, as everywhere else: BOOKCLERK_MEDIA_WORKERS, BOOKCLERK_MEDIA_ISOLATION and BOOKCLERK_MEDIA_WORKER.

Isolation modes

isolation decides what happens when a confinement layer doesn't engage:

  • required (default): refuse the job. A host that can't sandbox shouldn't decode untrusted audio next to the master key.
  • best-effort: apply what the platform supports and log the rest. For kernels without Landlock, where you accept unconfined codecs.
  • off: run codecs in-process on a blocking thread, with no jail and no child process. For development only.

Important

best-effort only covers layers the platform can't enforce. A worker binary that was never installed is a packaging error, and both required and best-effort refuse the job rather than fall back to in-process work. Set isolation = "off" only if unconfined codecs are really what you want.

The pool decides at startup, not at the first acquire. A host that can't honor its mode says so in the startup log, instead of failing every book later.

Changing settings while running

[media] reloads without a restart. A change builds a new pool, sends new jobs to it, and lets the old one drain. Nothing is interrupted, and nothing waits.

  • Jobs already running keep the isolation they started with. A worker confines itself at spawn and can't change afterwards, so lowering isolation never loosens a running job, and raising it never confines one after the fact.
  • While the old pool drains, total codec concurrency can briefly reach its in-flight jobs plus the new limit. That only shrinks, because the old pool admits no new jobs.

Note

If you lower media.workers to relieve memory pressure, the new limit applies to new jobs at once, but jobs already encoding run to the end.

Troubleshooting

refusing to run <job> unconfined: media isolation is required but unavailable

Either the worker binary wasn't found (a configured path doesn't exist, or nothing sits beside the host) or the platform has no way to confine itself. The startup log says which, naming the folder it searched or the backend that fell short.

  • The worker is missing. Install it beside the host binary, or point media.worker_bin at it. best-effort doesn't help here; only off runs without the worker.
  • The platform can't confine. If you accept unconfined codecs, drop to best-effort.

media worker (<job>) failed: worker exited with <status> before replying

The worker died during the job. Its stderr goes to the host, so the confinement summary and any codec output are in the daemon log just above.

media worker (<job>) failed: worker reported success but exited with <status>

The worker wrote a success reply and then died anyway, which it has no legitimate way to do. It is usually the OOM killer, or a signal arriving between the reply and the exit.

The job fails instead of being recorded, because a killed process can't vouch for the file it just wrote. Check the host's memory pressure, and lower media.workers if encoders are killed under load.

A job can't find an input that is clearly on disk

The path wasn't in read_paths() for that kind of job, so the jail denied it. A new input for a job must be added there too. The worker checks declared inputs before it runs the codec, so the error names the path instead of a bare No such file from inside a decoder.

Everything is slow, with idle cores

media.workers is the cap. Acquire batches are also run one at a time, by the job queue's network class (concurrency 1 by default) and the daemon's work_lock. The pool limits codec work within one acquire, not across acquires. See Jobs.

How the boundary works

One job is one process. Workers are never reused: confining a file system can't be undone and applies to the whole process, so a long-lived worker would need a jail wide enough for every job it might get. Media jobs take seconds to minutes, so a few milliseconds to spawn a process is nothing next to a tight per-job allowlist.

  1. The host writes a JSON MediaJob to the worker's stdin, and closes it.
  2. The worker builds its policy from the job and confines itself. On Windows, bookclerk-jail applied that policy at spawn instead.
  3. It runs the codec and writes a JSON reply to stdout.

Nothing configures the jail apart from the request, so a job can never get more than it declared.

By platform

Platform Backend File system Syscalls Network Memory, CPU, processes
Linux Landlock and seccomp-bpf Allowlist, ABI-probed Deny list No IP sockets cgroup v2, only when the spec sets limits (the media spec doesn't)
macOS Seatbelt (sandbox_init) Deny-by-default SBPL profile Not applicable Denied in the profile Not supported (file system and network only)
Windows AppContainer At spawn only Not applicable No network capabilities Job Object (media defaults: 2 GiB, 64 processes, CPU uncapped)

macOS has no seccomp. Seatbelt gates classes of operations, not syscall numbers, so the syscall layer reports "not applicable" rather than a gap. The (deny default) profile already refuses exec and anything not listed.

Windows can't confine a running process; AppContainer isolation is granted at CreateProcess. So on Windows the pool launches the worker through bookclerk-jail, the launcher plugins use:

  • The launcher applies a per-job AppContainer profile and an ACL allowlist built from the job's read and write paths. Bookclerk never changes DACLs under protected, OS-managed folders.
  • The worker joins a kill-on-close Job Object before it runs, with higher memory and process limits than plugins get.
  • The worker skips confining itself and relies on that outer jail.
  • Missing allowlist paths fail closed in the launcher, before the worker starts.

Warning

Ship bookclerk-jail beside the worker. With the default isolation = "required", a missing jail refuses media work instead of decoding unconfined.

What each job may touch

Job Reads Writes
encode_mp3 The input file The output's folder
remux_trimmed The input file The output's folder
fixup The input file and cover art The output's folder
package_m4b Every part The output's folder
align_chapters The input file Nothing
covers Nothing (images arrive on stdin) Nothing
tags Nothing (file ranges arrive on stdin) Nothing
pcm Nothing (track ranges arrive on stdin) Nothing
extract_archive The uploaded archive The empty folder it expands into

Write grants are folders, not single files, because codecs stage output in temporary files beside the destination. Packaging can't know the final sample table until it has read every part, so it first spills the joined AAC data to a scratch file. That file goes in the output folder, not $TMPDIR (which the jail doesn't grant), and is deleted when the job ends.

A clean environment

The worker starts with a scrubbed environment. It gets its job over stdin and needs nothing from Bookclerk's configuration, so BOOKCLERK_* variables, operator tokens and cloud credentials are dropped at spawn. It keeps only locale, time zone, RUST_BACKTRACE, and the Windows variables the loader and AppContainer need (such as SystemRoot, PATH and LOCALAPPDATA). Otherwise a compromised codec could read the host's settings from its own environment.

A symbolic link is granted at its resolved target, because that is the inode the kernel checks. Declaring a link grants whatever it points at. The host builds every job path from its own cache and output roots, so nothing untrusted chooses them. A destination plugin that builds output paths from user input should canonicalize them first.

What this jail doesn't cover

DRM decryption stays in the store plugin

DRM decryption also parses ISO-BMFF from outside sources, but it doesn't run in a media worker. Audible's Adrm and Widevine CENC decryption run on a blocking thread inside whichever process hosts the store plugin.

Two reasons keep it there:

  • Keys. Decryption needs each title's content key, and the host is built never to hold one: fetch_title decrypts inside the store plugin and returns plaintext paths. Decrypting in the pool would mean writing content keys to a worker's stdin, trading a parser boundary for a key-handling one.
  • Distribution. The decryption code lives in a plugin that can be left out of a build or package, which some regions may require of anyone shipping the core binaries. Moving it into bookclerk-media would remove that option.

Brand trimming follows the same line

Where Audible's brand intro and outro sit is arithmetic on chapter_info. bookclerk_mp4::brand_trim_range computes the window (bookclerk-mp4 holds it so the Audible guest need not link bookclerk-media), and the plugin applies it during the decryption pass it already makes. So the branding never reaches the disk, and no second copy of the file is made. Chapter splitting works on clear media, so it is an ordinary remux_trimmed job in the pool.

The shared MP4 code

The ISO-BMFF code underneath is shared, without moving that line. bookclerk-mp4 owns the parser, the sample table and the progressive remuxer for both callers. What differs is the SampleTransform each passes in:

  • bookclerk-media passes CopySamples, which passes data through untouched.
  • The Audible plugin passes a decryptor that holds the content key and CENC metadata. The remuxer never sees either.

No cipher, key or license code links into the shared crate, so leaving the plugin out of a build still leaves out every part of decryption. See shipping without a store, a build both hosts support and CI checks.

Sharing also made both muxers faster:

  • Both used to clone the whole sample table to trim it, once in the plugin and once in the host. The shared remuxer selects by index, so a trim no longer allocates a second table.
  • Both used to copy samples one syscall at a time: a seek, a read and a write per AAC frame of a few hundred bytes, a couple of million times per book. SampleReader buffers reads and tracks position, so samples in track order hit the buffer instead of seeking, and the writers are buffered to match.

Measured with cargo run --release -p bookclerk-mp4 --example bench_remux, a copy remux of a 3-hour book went from about 700 ms to 400 ms. The same buffering covers the packaging muxer and the plugin's fragmented-DASH path.

The plugin jail covers decryption

Confining the store plugin's process covers the decryption parser, and everything else a store does. First-party stores ship only as plugin binaries, so one jail reaches Audible, Libro.fm, Chirp and GraphicAudio together. Hosts always run them as external plugin processes, never inside bookclerkd, so decryption stays out of the host. With the default plugins.isolation = "required", a store whose jail cannot be applied is not loaded at all. See The guest jail.

Next steps

Source: docs/server/media.md at main · View as Markdown