Media worker pool
This page explains the media worker: the confined helper process that decodes, encodes and packages audio, re-encodes cover images, and reads the tags embedded in media files. It covers how to install and configure it, and what to check when media work fails.
Every decode, encode and packaging step runs in a short-lived
bookclerk-media-worker process, confined to the paths its job declared.
On Linux and macOS the worker confines itself; on Windows bookclerk-jail
confines it at spawn.
Acquire feeds audio from outside sources through C libraries: LAME
(mp3lame-sys), FDK-AAC (fdk-aac-sys), libopus (opusic-sys), the
FFmpeg decoders in release builds (bookclerk-libav) and Symphonia's
native dependencies. Before the pool, those parsers ran on tokio's blocking
threads, inside the process that holds the master encryption key and an
open library.db. A parser bug there would have disclosed the key, not
just failed a book.
Moving the work out gives three properties:
- The codecs can't reach key material. The jail allows the job's
declared inputs and one output folder.
master.key,library.dband the rest of$HOMEare not allowed. - A crash costs one book. A decoder segfault becomes a failed job,
not a dead daemon. The LAME bug fixed in
mp3.rs(writing past its output buffer) used to take the daemon down with it. - You choose the concurrency.
media.workerscaps how many codec jobs run at once, instead of however many threads tokio's blocking pool grows under load.
An uploaded ZIP or tar archive is expanded by the extract_archive job
into an empty folder the host made. The walk refuses links, devices,
encrypted entries, nested archives and names that would leave the
folder, and counts the bytes it writes against caps on entries, total
size, entry size and compression ratio. The host checks the worker's
report against the same caps and the files on disk before it moves
anything. See Uploads.
The worker recognises a file's container and codec from its bytes, never
from its name, so a file named .mp3 that holds an MP4 is read as an MP4.
It reads these files:
| Container | Codecs |
|---|---|
MP4 (.m4a, .m4b) |
AAC, ALAC, FLAC, Opus, and E-AC-3 and AC-3 in release builds |
MPEG audio (.mp3, .mp2) |
MP3, MP2, MP1 |
ADTS (.aac) |
AAC |
| FLAC | FLAC |
Ogg (.opus, .ogg, .oga) |
Opus, Vorbis |
| WAV, AIFF, CAF | PCM, IMA ADPCM |
ASF (.wma), release builds |
WMA 1, WMA 2, WMA 9 Pro, WMA 9 Lossless |
Raw Dolby (.ac3, .eac3), release builds |
AC-3, E-AC-3 |
AMR (.amr, .awb, .3gp), release builds |
AMR-NB, AMR-WB |
Every AAC profile decodes through FDK: AAC-LC, HE-AAC, HE-AAC v2 and xHE-AAC (USAC). An HE-AAC book is reported and converted at the rate it plays at, not the half rate an MP4 header states for it. Opus decodes through libopus at 48 kHz, mono or stereo; a surround Opus stream (more than one stream in the file) fails to open. AC-4 isn't read by any build.
The rows marked release builds decode through FFmpeg's decoders. Every
release archive and the Docker image carry them; a plain cargo build
doesn't (see FFmpeg decoders).
Surround is folded to stereo by speaker position: left speakers go left, right speakers go right, centre and rear-centre speakers go to both sides at -3 dB, and the LFE channel is left out. Channels with no position go to both sides equally. A file whose sample rate changes partway through fails to convert rather than losing the audio after the change.
The worker writes AAC-LC M4B, MP3 and Ogg Opus. With
output.keep_codec_in_m4b on, an M4B also carries Opus (Opus with
dOps), FLAC (fLaC with dfLa), ALAC, E-AC-3 or AC-4 audio copied
unchanged from the source, with the Opus pre-skip as an edit list (see
Keeping unusual audio in M4B).
It writes tags, covers
and chapters into MP4 and MP3 files, and into FLAC, Ogg Opus and Ogg
Vorbis files as Vorbis comments: TITLE, ALBUM, ARTIST,
ALBUMARTIST, COMPOSER (the narrator), DATE, GENRE, PUBLISHER,
DESCRIPTION, LANGUAGE, SERIES, SERIES-PART, ASIN, ISBN,
chapters as CHAPTERnnn and CHAPTERnnnNAME (up to 999), and the cover
as a front-cover picture. Opus output is encoded at 48 kHz: audio at
another rate is converted with a windowed-sinc resampler first. Every
M4B it writes is faststart: the
sample table comes before the audio, so a player can start the book from
its first bytes. The sample table keeps 1 MiB of room for tags, a
cover and chapters, so adding them later never moves the audio.
When an output re-encodes or repackages audio, the acquire takes another
store's copy from a library only if the worker reads its kind of file and
the tag reader names its codec. That means AAC or ALAC in MP4, MP3,
FLAC, Opus or Vorbis in Ogg, and in release builds E-AC-3 in MP4. WAV,
AIFF, CAF, ADTS, WMA, raw AC-3 and AMR files decode, but a copy in one of
them isn't taken, because its codec isn't recorded.
The lists the acquire checks are in bookclerk_media::capabilities. The
daemon serves the same report at GET /api/status/media, and
bookclerk media capabilities prints it:
$ bookclerk media capabilities
decodes: aac, alac, flac, mp3, opus, vorbis
reads: audio/mp4, audio/mpeg, audio/flac, audio/aac, audio/wav, audio/aiff, audio/x-caf, audio/ogg, audio/opus
encodes: m4b, mp3, opus (advanced)
A release build on Linux or macOS also lists eac3 under decodes, and
audio/x-ms-wma, audio/ac3, audio/eac3, audio/amr, audio/amr-wb
and audio/3gpp under reads.
An encoding marked advanced is one fewer players open than M4B and MP3, so settings offer it under their advanced options.
Tagging any other container, such as WAV, fails with a message naming
it, and the acquire stores the file without the new tags. Files kept as
delivered (output.format = "none") are not decoded at all, so any format
can be stored that way.
Cover art is downloaded from any public website, so Bookclerk never shows
or stores the downloaded bytes. One long-lived bookclerk-media-worker covers process turns each download into the JPEGs Bookclerk serves:
- It checks the file signature and refuses anything but JPEG, PNG and WebP (no SVG, GIF or HTML).
- It decodes with pure-Rust codecs under limits: at most 10 MiB in, 5000 px a side and 128 MiB of decode memory.
- It resizes with a Lanczos filter and writes three 4:4:4 JPEGs: 128 px and 360 px (quality 85) and the full image up to 2400 px (quality 90, progressive).
Unlike a media job, a cover needs no file, so this process is confined to
no path and no network before it reads its first image, and then serves
covers until the daemon stops. It decodes on media.workers threads,
covers don't wait for a job slot, and a cover that panics the decoder
fails alone. A cover that takes longer than 30 seconds is refused and the
process is stopped; when the process stops or dies, the next cover starts
a new one. With media.isolation = "off"
covers are processed in the daemon, like media jobs.
A page of covers fits a one-core node: the budget test in
bookclerk-cover processes a 20-cover page, one large cover among them,
one after another and fails when an optimized build takes longer than
2 seconds. Run it with
cargo test --release -p bookclerk-cover --test budget -- --nocapture
to see the time on your hardware.
Files already on storage carry their own metadata: MP4 ilst items and
chapter tracks, ID3 tags in MP3s, and Vorbis comments in FLAC and Ogg
files.
Anyone who can put a file on storage controls those bytes, so they are
read by one long-lived bookclerk-media-worker tags process, never by the
daemon. For each file it reports:
- the text entries (title, album, artist, composer, genre, freeform items
such as
com.pilabor.tone:AUDIBLE_ASINandTXXX:ASIN, and so on), as the file holds them; - the chapters, from the QuickTime chapter track, Nero
chpl, ID3CHAPframes or VorbisCHAPTERnnncomments; - the running time, and whether the container states it or it was estimated;
- the audio encoding (AAC, ALAC, FLAC, MP3, Opus, Vorbis, E-AC-3 or AC-4), from the audio track's sample description in an MP4 file or the first packet of an Ogg stream;
- where the cover picture lies in the file, its format and its SHA-256, without the picture itself;
- a SHA-256 digest of the metadata region (versioned), which changes when a tag is edited and not when the audio is.
The worker opens no file. It asks the daemon for the byte ranges it needs,
and the daemon fetches them from wherever the file lives, so a file on S3
is read with ranged GETs, just like a local one. Only box and frame
headers and the metadata region are fetched, never the audio: the sample
tables of a long book's audio track are skipped by size. Files are read in
pages, up to 16 at once, and don't wait for a job slot.
Like the cover process, the tag process is confined to no path and no
network before it reads its first frame, and serves files until the daemon
stops. The daemon treats it as compromised: it fetches only ranges inside
the file, caps each file at 128 rounds and 16 MiB, holds at most 64 MiB of
fetched bytes at once, stops the process when a round takes longer than
30 seconds or it asks for more than that, and checks every record against
the reader's limits before using it. A file the reader can't parse fails
alone. With media.isolation = "off" tags are read in the daemon.
The reader lives in bookclerk-tags and is fuzzed by the tags_read and
tags_wire targets (see Fuzzing).
The desktop window's player can't stream audio to its webview, so the
window decodes it natively instead (see
Playback). The decoding happens in a
bookclerk-media-worker pcm process, one for each track being played,
never in the window's process. The worker:
- Reads the track's size and the starting position from the host.
- Asks for the byte ranges it needs, 256 KiB at a time. The host reads them through the play session's stream route, so every block passes the same checks a browser's request does.
- Seeks to the start, reports the sample rate, channel count (mono or stereo), actual start and running time, then sends 16-bit samples until the track ends.
Like the tag process, it is confined to no path and no network before it reads its first frame. The host treats it as compromised. It answers only ranges inside the track, and stops the process when the format or a range request takes longer than 30 seconds or the process asks for more than the track's size plus 64 MiB in all. It checks every frame against the wire's limits and the order the protocol allows. Seeking or closing the track stops the process.
Playback never decodes in-process: with media.isolation = "off" the
desktop player refuses to play. The frames are fuzzed by the
media_pcm_wire target (see Fuzzing).
Every audio file an acquire writes is probed once it is on its
destination, through the same tag process, and the result is kept on the
file's row in the database (edition_files):
| Value | Meaning |
|---|---|
| Media type | audio/mp4, audio/mpeg or audio/flac, from the container the reader found, not from the file name |
| Bitrate | The file's size over its running time, in kbps: an average that includes the cover and tags. Empty when the running time is unknown |
| Lossless | Whether the audio track's encoding is lossless (FLAC, or ALAC in an MP4 file) |
| Source store | The store the copy was downloaded from, such as audible or libro |
| Copy | Which of the library's copies the file belongs to, by format and quality (mp3 and 64k-mono); see Keeping several copies |
| Codec | The audio encoding: aac, alac, flac, mp3, opus, vorbis, eac3 or ac4 |
| AAC profile | For AAC, the profile the decoder configuration names: lc, he, he_v2 or xhe. HE-AAC that is signalled only inside the audio frames reads as lc |
| Channels and sample rate | As the stream's headers state them; for HE-AAC, the rate after band replication |
| Duration | The running time a player sees, in milliseconds: an MP4's movie duration (which edit lists shape), an Opus stream's samples past its pre-skip, an MP3's frame count. Empty when the file states none, never 0 |
Nothing is decoded to find these values. The codec, profile, channels, sample rate and duration are stored only for files an acquire on this node probed; an upload's row carries the sender's media type, bitrate, lossless flag and store, and the rest stays empty.
The row answers "what copy do we hold?" without opening the file again, so comparing two copies of one edition, or telling another install what is on offer, is a database read.
A probe never decides whether a file is kept. When the tag process is
missing and media.isolation is required, or the reader cannot parse
the file, the file is stored as before and its row stays unprobed.
A host or a branch that acquires a book probes it the same way before uploading, and sends the bitrate, lossless flag and source store beside the file's media type. The library checks them with the manifest (a bitrate up to 20,000 kbps, a plain store id) and stores them when the upload commits, so an upload that is aborted or never finishes describes nothing. The values are the sender's word about its own file. They come from a node the library already trusts with the title, and they are only used to choose between two copies of the same edition.
Note
Files stored before probes were recorded have unprobed rows. They are not probed again by this change.
An acquire writes chapters into the files and also records, in the
database, every chapter list it computed for the edition: the markers the
store returned (or one chapter per delivered part when it returned none),
the layout a metadata provider answered, and the provider's chapters after
the worker's align_chapters job snapped them to the spoken titles. Each
is a claim from its own source, and the library chooses the list the
edition plays (see Chapters). A failed record is
logged and never fails the acquire, because the files already hold their
chapters.
Build the worker and ship it with the hosts, beside the plugin jail's launcher (found the same way):
cargo build --release -p bookclerk-cli -p bookclerkd -p bookclerk-media-worker -p bookclerk-jail
Add --features bookclerk-media-worker/libav for the FFmpeg decoders, as
cargo build-app --release and cargo package-platform do.
The Docker image ships it in the read-only bundle under /opt/bookclerk,
next to bookclerk and bookclerkd (see
Running Bookclerk → Docker).
The pool looks for the worker in this order:
media.worker_bininconfig.tomlBOOKCLERK_MEDIA_WORKERbookclerk-media-workerbeside the running executable
Each candidate must exist, so a stale configured path fails loudly instead of quietly encoding unconfined.
WMA, AC-3, E-AC-3 and AMR decode through FFmpeg, built from source by the
bookclerk-libav crate behind the worker's libav feature:
- The build script downloads one pinned FFmpeg release from
ffmpeg.org, checks its SHA-256, and builds only those decoders, their demuxers (ASF, raw AC-3/E-AC-3, raw AMR and MP4/3GP) and the AC-3 parser. It builds no encoders, filters, protocols, devices, hardware decoders or programs, and none of FFmpeg's GPL or nonfree parts. - On Linux and macOS it needs
curl,tarwith xz support, a POSIX shell andmake. - On Windows it builds with MSVC (x64 and arm64): FFmpeg's
configureruns under MSYS2's shell, which needs itsmakepackage (pacman -S make). MSYS2 is found atC:\msys64, or atMSYS2_ROOTwhen that is set. - FFmpeg decodes single-threaded inside the worker's confinement, like every other codec. It never opens a file or a network connection: the worker hands it the bytes.
- The worker tries Symphonia first. A file that starts like ASF, raw AMR or a raw AC-3 stream goes straight to FFmpeg, and so does a file Symphonia can't read or whose audio it has no decoder for, such as AMR in a 3GP file.
The release builds of the repository's cargo aliases turn the feature on
by themselves: cargo build-app --release, cargo package-platform and
cargo install-platform --release. A plain cargo build --release needs
the --features flag shown in Install the worker.
Tests run with it too, so the first test build downloads FFmpeg and takes
about half a minute longer (a few minutes on Windows).
[media]
workers = 0 # 0 derives one per core, capped at 8; 1 when memory.max <= 1 GiB or cpu.max is one core
isolation = "required" # required | best-effort | off
# worker_bin = "/usr/local/bin/bookclerk-media-worker"
Environment variables win over TOML, as everywhere else:
BOOKCLERK_MEDIA_WORKERS, BOOKCLERK_MEDIA_ISOLATION and
BOOKCLERK_MEDIA_WORKER.
isolation decides what happens when a confinement layer doesn't
engage:
required(default): refuse the job. A host that can't sandbox shouldn't decode untrusted audio next to the master key.best-effort: apply what the platform supports and log the rest. For kernels without Landlock, where you accept unconfined codecs.off: run codecs in-process on a blocking thread, with no jail and no child process. For development only.
Important
best-effort only covers layers the platform can't enforce. A
worker binary that was never installed is a packaging error, and both
required and best-effort refuse the job rather than fall back to
in-process work. Set isolation = "off" only if unconfined codecs are
really what you want.
The pool decides at startup, not at the first acquire. A host that can't honor its mode says so in the startup log, instead of failing every book later.
[media] reloads without a restart. A change builds a new pool, sends
new jobs to it, and lets the old one drain. Nothing is interrupted, and
nothing waits.
- Jobs already running keep the isolation they started with. A worker
confines itself at spawn and can't change afterwards, so lowering
isolationnever loosens a running job, and raising it never confines one after the fact. - While the old pool drains, total codec concurrency can briefly reach its in-flight jobs plus the new limit. That only shrinks, because the old pool admits no new jobs.
Note
If you lower media.workers to relieve memory pressure, the new limit
applies to new jobs at once, but jobs already encoding run to the end.
Either the worker binary wasn't found (a configured path doesn't exist, or nothing sits beside the host) or the platform has no way to confine itself. The startup log says which, naming the folder it searched or the backend that fell short.
- The worker is missing. Install it beside the host binary, or point
media.worker_binat it.best-effortdoesn't help here; onlyoffruns without the worker. - The platform can't confine. If you accept unconfined codecs, drop to
best-effort.
The worker died during the job. Its stderr goes to the host, so the confinement summary and any codec output are in the daemon log just above.
The worker wrote a success reply and then died anyway, which it has no legitimate way to do. It is usually the OOM killer, or a signal arriving between the reply and the exit.
The job fails instead of being recorded, because a killed process can't
vouch for the file it just wrote. Check the host's memory pressure, and
lower media.workers if encoders are killed under load.
The path wasn't in read_paths() for that kind of job, so the jail
denied it. A new input for a job must be added there too. The worker
checks declared inputs before it runs the codec, so the error names the
path instead of a bare No such file from inside a decoder.
media.workers is the cap. Acquire batches are also run one at a
time, by the job queue's network class (concurrency 1 by default) and
the daemon's work_lock. The pool limits codec work within one acquire,
not across acquires. See Jobs.
One job is one process. Workers are never reused: confining a file system can't be undone and applies to the whole process, so a long-lived worker would need a jail wide enough for every job it might get. Media jobs take seconds to minutes, so a few milliseconds to spawn a process is nothing next to a tight per-job allowlist.
- The host writes a JSON
MediaJobto the worker's stdin, and closes it. - The worker builds its policy from the job and confines itself. On
Windows,
bookclerk-jailapplied that policy at spawn instead. - It runs the codec and writes a JSON reply to stdout.
Nothing configures the jail apart from the request, so a job can never get more than it declared.
| Platform | Backend | File system | Syscalls | Network | Memory, CPU, processes |
|---|---|---|---|---|---|
| Linux | Landlock and seccomp-bpf | Allowlist, ABI-probed | Deny list | No IP sockets | cgroup v2, only when the spec sets limits (the media spec doesn't) |
| macOS | Seatbelt (sandbox_init) |
Deny-by-default SBPL profile | Not applicable | Denied in the profile | Not supported (file system and network only) |
| Windows | AppContainer | At spawn only | Not applicable | No network capabilities | Job Object (media defaults: 2 GiB, 64 processes, CPU uncapped) |
macOS has no seccomp. Seatbelt gates classes of operations, not
syscall numbers, so the syscall layer reports "not applicable" rather
than a gap. The (deny default) profile already refuses exec and
anything not listed.
Windows can't confine a running process; AppContainer isolation is
granted at CreateProcess. So on Windows the pool launches the worker
through bookclerk-jail, the launcher plugins use:
- The launcher applies a per-job AppContainer profile and an ACL allowlist built from the job's read and write paths. Bookclerk never changes DACLs under protected, OS-managed folders.
- The worker joins a kill-on-close Job Object before it runs, with higher memory and process limits than plugins get.
- The worker skips confining itself and relies on that outer jail.
- Missing allowlist paths fail closed in the launcher, before the worker starts.
Warning
Ship bookclerk-jail beside the worker. With the default
isolation = "required", a missing jail refuses media work instead of
decoding unconfined.
| Job | Reads | Writes |
|---|---|---|
encode_mp3 |
The input file | The output's folder |
remux_trimmed |
The input file | The output's folder |
fixup |
The input file and cover art | The output's folder |
package_m4b |
Every part | The output's folder |
align_chapters |
The input file | Nothing |
covers |
Nothing (images arrive on stdin) | Nothing |
tags |
Nothing (file ranges arrive on stdin) | Nothing |
pcm |
Nothing (track ranges arrive on stdin) | Nothing |
extract_archive |
The uploaded archive | The empty folder it expands into |
Write grants are folders, not single files, because codecs stage output
in temporary files beside the destination. Packaging can't know the final
sample table until it has read every part, so it first spills the joined
AAC data to a scratch file. That file goes in the output folder, not
$TMPDIR (which the jail doesn't grant), and is deleted when the job
ends.
The worker starts with a scrubbed environment. It gets its job over stdin
and needs nothing from Bookclerk's configuration, so BOOKCLERK_*
variables, operator tokens and cloud credentials are dropped at spawn.
It keeps only locale, time zone, RUST_BACKTRACE, and the Windows
variables the loader and AppContainer need (such as SystemRoot, PATH
and LOCALAPPDATA). Otherwise a compromised codec could read the host's settings
from its own environment.
A symbolic link is granted at its resolved target, because that is the inode the kernel checks. Declaring a link grants whatever it points at. The host builds every job path from its own cache and output roots, so nothing untrusted chooses them. A destination plugin that builds output paths from user input should canonicalize them first.
DRM decryption also parses ISO-BMFF from outside sources, but it doesn't run in a media worker. Audible's Adrm and Widevine CENC decryption run on a blocking thread inside whichever process hosts the store plugin.
Two reasons keep it there:
- Keys. Decryption needs each title's content key, and the host is
built never to hold one:
fetch_titledecrypts inside the store plugin and returns plaintext paths. Decrypting in the pool would mean writing content keys to a worker's stdin, trading a parser boundary for a key-handling one. - Distribution. The decryption code lives in a plugin that can be left
out of a build or package, which some regions may require of anyone
shipping the core binaries. Moving it into
bookclerk-mediawould remove that option.
Where Audible's brand intro and outro sit is arithmetic on
chapter_info. bookclerk_mp4::brand_trim_range computes the window
(bookclerk-mp4 holds it so the Audible guest need not link
bookclerk-media), and the plugin applies it during the decryption pass it
already makes. So the branding never reaches the disk, and no second copy
of the file is made. Chapter splitting works on clear media, so it is an ordinary
remux_trimmed job in the pool.
The ISO-BMFF code underneath is shared, without moving that line.
bookclerk-mp4 owns the parser, the sample table and the progressive
remuxer for both callers. What differs is the SampleTransform each
passes in:
bookclerk-mediapassesCopySamples, which passes data through untouched.- The Audible plugin passes a decryptor that holds the content key and CENC metadata. The remuxer never sees either.
No cipher, key or license code links into the shared crate, so leaving the plugin out of a build still leaves out every part of decryption. See shipping without a store, a build both hosts support and CI checks.
Sharing also made both muxers faster:
- Both used to clone the whole sample table to trim it, once in the plugin and once in the host. The shared remuxer selects by index, so a trim no longer allocates a second table.
- Both used to copy samples one syscall at a time: a
seek, areadand awriteper AAC frame of a few hundred bytes, a couple of million times per book.SampleReaderbuffers reads and tracks position, so samples in track order hit the buffer instead of seeking, and the writers are buffered to match.
Measured with cargo run --release -p bookclerk-mp4 --example bench_remux,
a copy remux of a 3-hour book went from about 700 ms to 400 ms. The same
buffering covers the packaging muxer and the plugin's fragmented-DASH
path.
Confining the store plugin's process covers the decryption parser, and
everything else a store does. First-party stores ship only as plugin
binaries, so one jail reaches Audible, Libro.fm, Chirp and GraphicAudio
together. Hosts always run them as external plugin processes, never
inside bookclerkd, so decryption stays out of the host. With the default
plugins.isolation = "required", a store whose jail cannot be applied is
not loaded at all. See The guest jail.
- Architecture: where the pool sits in the acquire pipeline.
- The guest jail: the other confined tier, for plugin processes. It lives longer, grants more, and is applied by a launcher.
- Configuration: every
config.tomlsetting. - Running Bookclerk: running
bookclerkd.
Source: docs/server/media.md at main · View as Markdown