bookclerk_acquire
Acquire orchestration: license → download → decrypt → metadata → storage.
Host-facing pipeline for fetching a storefront title, packaging audio, and
writing it to one or more output destinations. DRM decrypt happens inside
content-source plugins; this crate never sees license keys. See
docs/server/media.md for encode/remux details.
| Item | Kind | Summary |
|---|---|---|
LibraryCopies |
trait | Where an acquire asks for the copy a library holds of a title. |
LibraryCopy |
trait | A copy of one edition a library offered this install. |
CoverImages |
trait | Turns a cover URL a metadata provider named into an image the acquire can embed, without the acquire itself requesting the URL. |
AcquireDestination |
struct | One configured output destination for an acquire run. |
AcquireDestinations |
struct | Ordered set of acquire destinations built from the deployed destination plugins. |
DestinationKeys |
struct | The destination an acquire request names files for, which translates its host keys. |
LibraryCopyFile |
struct | One file of a library's copy, as the library listed it. |
LibraryCopyListing |
struct | The title an acquire wants, as its store lists it. |
LibraryCopySource |
struct | The libraries an acquire asks for a copy before it fetches from the store. The default asks none, so every title comes from its store. |
MatchStorageOptions |
struct | Options for match_storage_to_library. |
MatchStorageSummary |
struct | Summary of a storage→library match run. |
NamingContext |
struct | Metadata available to naming templates. |
AcquireMetadata |
struct | The deployed metadata providers an acquire asks for the plain-audio overlay (catalog tags, cover and chapter layout of an enriched ASIN), and what turns the overlay's cover URL into an image. |
AcquireRequest |
struct | Request to acquire a single title. |
AcquireResult |
struct | Result after a successful acquire. |
CopyOutcome |
struct | What one destination ended up with in one acquire. |
PlannedTitleNames |
struct | The names an acquire of one title writes on one destination, as the owner plans them, to bound a host's upload of that title. |
ReconcileOptions |
struct | Options for reconcile_library. |
ReconcileSummary |
struct | Summary of a reconcile run. |
StorageIndex |
struct | Index of storage object keys, keyed by ASIN found in the path. |
ScanCheckpoint |
struct | Versioned scan progress stored in the job payload checkpoint. |
AcquireError |
enum | Failures from the acquire / convert / reconcile pipeline. |
CopyResult |
enum | Whether a destination holds an acquire's audio. |
from_store |
function | True when the library downloaded its copy from store, the store this install's title belongs to. A copy whose store the library does not know is never taken, and one from another store only replaces the install's own download when it is the better copy. |
match_storage_to_library |
function | List audio in storage, probe metadata (no body download), match to library. |
match_storage_using_index |
function | Match library rows against an existing scan index. |
audio_basename |
function | Basename of the acquired audio file (for .cue FILE lines). |
chapter_storage_key |
function | Storage key for a split chapter file. |
chapter_storage_key_with_folder |
function | Chapter storage key with a separate folder naming context (podcast parent folder). |
default_storage_key |
function | Build a relative storage key for a acquired title using the default NamingProfile (Audiobookshelf). |
resolve_templates |
function | Resolve optional template overrides against a naming profile. |
sidecar_key |
function | Replace the audio file extension on a storage key with a sidecar suffix. |
storage_key |
function | Build a storage key from optional classic Libation folder/file templates. |
storage_key_with_contexts |
function | Build a storage key using separate folder and file naming contexts. |
storage_key_with_rules |
function | Like storage_key but applies classic ReplacementCharacters. |
swap_audio_extension |
function | Replace the audio file extension on a storage key (e.g. .m4b → .mp3). |
acquire_book |
function | Run the acquire pipeline for one book. |
acquire_book_indexed |
function | Acquire with an optional pre-built StorageIndex (avoids re-listing storage when liberating many titles). On success, newly written keys are inserted into the index so later books in the same batch can match them. |
acquire_pdf_only |
function | Download and store companion PDF only (classic acquire --pdf). |
call_number_of |
function | The short Bookclerk id of the edition account holding holding_id (holdings.id) sells, for <call number> (bookclerk_library::call_number). |
convert_copy |
function | Adds copy to the edition of the title req names, made from a copy the library already stores, on every destination whose filter takes that copy. |
planned_storage_key |
function | Compute the storage key that would be used (for dry-run / set-status). |
planned_storage_key_for |
function | Planned audio storage key for ext, honoring folder/file templates and save_podcasts_to_parent_folder: the host key the templates build, translated by the request's destination (AcquireRequest::destination_keys). |
planned_storage_key_with_rules |
function | The host key planned_storage_key_for translates, built with an explicit replacement-rule set and not translated (used by reconcile to probe wildcard patterns across sanitization profiles). |
planned_title_folders |
function | The folders an acquire of req writes the title's files under on a destination with req.options: the folder of its planned audio key, which its sidecars, chapter files and parts share. When podcast episodes are saved to the parent's folder, the episode's own folder is included too, since a library without the parent's row names it so. An empty string is the destination's root (a template with no folder). |
planned_title_names |
function | The title's planned names on the destination req.options describe: PlannedTitleNames. |
share_acquired_sibling |
function | Marks book acquired at the files of its edition, when another book of the edition acquired it (edition_acquisitions) and every destination holds the copy, so the edition is not fetched again. The edition's companion PDF is shared too when that book has one. |
extract_asins_from_key |
function | Extract Audible-like ASINs and ISBN-13 tokens from a storage key / file path. |
find_existing_for_request |
function | Same as find_existing_for_title but for a acquire request before DB status is Acquired. |
find_existing_for_title |
function | Find an existing acquired file for a title (planned path or ASIN-in-path). |
reconcile_library |
function | Scan storage and update library acquire status / storage_key for matches. |
scan_storage |
function | Builds or resumes a durable identity/object index. |
Result |
type alias | Result alias for AcquireError. |
LibraryCopyError |
type alias | Why a library's copy could not be asked for, downloaded or used. The acquire logs it and fetches from the store instead. |
NAME_WILDCARD |
constant | In a PlannedTitleNames pattern: one or more characters of a file name (no /), where the source's part or chapter title goes; the marker the owner's upload scope matches (bookclerk_library::control_plane::TitleNames). |
NUMBER_WILDCARD |
constant | In a PlannedTitleNames pattern: one or more ASCII digits, where a part or chapter number goes (the owner's upload scope matches it too). |
pub trait LibraryCopies: Send + Sync {
fn copy_of<'life0, 'life1, 'async_trait>(
&'life0 self,
listing: LibraryCopyListing<'life1>,
) -> Pin<Box<dyn Future<Output = Option<Box<dyn LibraryCopy>>> + Send + 'async_trait>>
where
Self: 'async_trait,
'life0: 'async_trait,
'life1: 'async_trait,;
}
Where an acquire asks for the copy a library holds of a title.
fn copy_of<'life0, 'life1, 'async_trait>(
&'life0 self,
listing: LibraryCopyListing<'life1>,
) -> Pin<Box<dyn Future<Output = Option<Box<dyn LibraryCopy>>> + Send + 'async_trait>>
where
Self: 'async_trait,
'life0: 'async_trait,
'life1: 'async_trait,
The copy a library this install joined offers of listing; None
when no library holds one this install's person may take, when
this install does not keep copies, or when no library answers.
A copy from the listing's own store (from_store) is taken
instead of the store download; a copy from another store only
replaces the store download when it is the better one. An
implementation that asks several libraries therefore keeps asking
past a copy from another store and returns it only when no library
has one from the listing's own, and never returns a copy whose
store the library does not know.
pub trait LibraryCopy: Send + Sync {
fn source_store(&self) -> Option<&str>;
fn probe(&self) -> FileProbe;
fn files(&self) -> &[LibraryCopyFile];
fn fetch_file<'life0, 'life1, 'async_trait>(
&'life0 self,
index: usize,
dest: &'life1 Path,
) -> Pin<Box<dyn Future<Output = Result<(), LibraryCopyError>> + Send + 'async_trait>>
where
Self: 'async_trait,
'life0: 'async_trait,
'life1: 'async_trait,;
fn taken<'life0, 'async_trait>(
&'life0 self,
) -> Pin<Box<dyn Future<Output = Result<(), LibraryCopyError>> + Send + 'async_trait>>
where
Self: 'async_trait,
'life0: 'async_trait,;
}
A copy of one edition a library offered this install.
fn source_store(&self) -> Option<&str>
The store the library downloaded the copy from, when it knows.
fn probe(&self) -> FileProbe
What the library's probe found in the copy's primary audio: its
media type, average bitrate and whether it is lossless. The acquire
compares it with its own store download when the copy came from
another store (bookclerk_library::FileProbe::is_better_than),
and probes the copy itself before it believes the answer.
fn files(&self) -> &[LibraryCopyFile]
The copy's files, the primary audio first; a file's place is its download index.
fn fetch_file<'life0, 'life1, 'async_trait>(
&'life0 self,
index: usize,
dest: &'life1 Path,
) -> Pin<Box<dyn Future<Output = Result<(), LibraryCopyError>> + Send + 'async_trait>>
where
Self: 'async_trait,
'life0: 'async_trait,
'life1: 'async_trait,
Downloads file index to dest, which must end up holding exactly
the file's listed size. dest is a path nothing else writes to, so
an implementation may resume what it already holds.
Returns an error when the library refuses or the transfer fails.
fn taken<'life0, 'async_trait>(
&'life0 self,
) -> Pin<Box<dyn Future<Output = Result<(), LibraryCopyError>> + Send + 'async_trait>>
where
Self: 'async_trait,
'life0: 'async_trait,
Called once the acquire holds the copy's files and before it stores any of them, so the implementation can record where the book comes from (a branch never shares a book back to the library it took it from). Recording first means audio a failed store step leaves on a destination is already known as the library's. It may run again for the same title, so it must be safe to repeat.
Returns an error when the record cannot be written; the acquire then does not use the copy.
pub trait CoverImages: Send + Sync {
fn full_cover<'life0, 'life1, 'async_trait>(
&'life0 self,
url: &'life1 str,
) -> Pin<Box<dyn Future<Output = Option<Vec<u8>>> + Send + 'async_trait>>
where
Self: 'async_trait,
'life0: 'async_trait,
'life1: 'async_trait,;
}
Turns a cover URL a metadata provider named into an image the acquire can embed, without the acquire itself requesting the URL.
The daemon implements this with its cover service: a covers plugin
downloads the image and the confined media worker re-encodes it, so what
reaches the tagger is a JPEG the host produced.
fn full_cover<'life0, 'life1, 'async_trait>(
&'life0 self,
url: &'life1 str,
) -> Pin<Box<dyn Future<Output = Option<Vec<u8>>> + Send + 'async_trait>>
where
Self: 'async_trait,
'life0: 'async_trait,
'life1: 'async_trait,
The full-size JPEG for url, or None when it cannot be fetched or
is not an image.
pub struct AcquireDestination {
pub id: DestinationId,
pub backend: Box<dyn StorageBackend>,
pub options: DownloadOptions,
pub filter: DestinationFilter,
}
One configured output destination for an acquire run.
Fields
id: The destination's id: its storage plugin instance id.backend: Constructed storage backend for this destination.options: Source download options (quality, chapter prefs, …).filter: Which books the destination takes; no rules take every book.
pub struct AcquireDestinations {
pub items: Vec<AcquireDestination>,
pub primary: DestinationId,
pub multi_destination: MultiDestinationMode,
}
Ordered set of acquire destinations built from the deployed destination plugins.
Fields
items: Configured acquire destinations in evaluation order.primary: The primary destination, whose keys the library records as the title's storage key.multi_destination: Whether multiple destinations are enabled for this run.
pub fn len(&self) -> usize
Number of destinations.
pub fn is_empty(&self) -> bool
Returns true when no destinations are deployed.
pub fn primary_destination(&self) -> &AcquireDestination
Returns the primary destination used for library storage keys.
pub fn destination(&self, id: &DestinationId) -> Option<&AcquireDestination>
Returns the destination id, if configured.
pub fn set_filters(&mut self, filters: &BTreeMap<String, DestinationFilter>)
Replaces every destination's filter with the one stored for it now.
A long acquire run calls this before each book's target set is
chosen, so a filter edited mid-run applies to the books that start
afterwards. A destination absent from filters takes every book.
filters- Stored filters by plugin instance id.
pub fn for_book(&self, facts: &PlacementFacts) -> Result<AcquireDestinations>
The destinations that take the book facts describes: the target
set an acquire of it writes to, fixed before the acquire starts.
The primary stays the primary when it takes the book; otherwise the first destination that does becomes it.
Returns AcquireError::NoDestinationAccepts when no destination's
filter takes the book.
pub async fn reachable(
&self,
) -> Result<(AcquireDestinations, Vec<(DestinationId, String)>)>
Splits the destinations into the ones that answer a listing now and the ones that do not.
An acquire run lists every destination before it starts. One that is offline is left out of the run rather than failing it: the titles go to the destinations that answer, and the missing copies are written by a later run once it is back. The primary stays the primary when it answers; otherwise the first destination that does becomes it.
The destinations that answered, and each one that did not with its listing error.
Returns AcquireError::NoDestinationReachable when none answers.
pub fn copy_plan(&self) -> Vec<(OutputCopy, AcquireDestinations)>
The copies of each book this acquire makes and the destinations
that receive each, in order: the main output's copy first, then
output.copies (see bookclerk_source::DownloadOptions::copies_to_make,
read from the primary destination's options). A destination
receives a copy when its filter's copy rules take it
(DestinationFilter::accepts_copy). A copy no destination takes
is left out.
Each destination's options package the copy
(bookclerk_source::DownloadOptions::for_copy). The main
output's copy keeps the plain names; every other copy goes under a
folder of its own at the top, named after the copy's label, then the
destination's usual folder template (mp3 64k-mono/Title [ID]). Two
copies so never share a folder or a key, on one destination or
across several, and a file at the plain names is always the main
copy's. The label is a short folder of its own, so path-length
limits shorten the book's folders before it. Each copy's primary
destination is the set's primary when it receives the copy, else the
first that does.
pub fn added_copy_plan(&self, copy: &OutputCopy) -> Option<AcquireDestinations>
The destinations that receive copy when it is added to a book
already stored (a convert, see crate::convert_copy): each whose
filter's copy rules take it, with options that package it under a
folder of its own named after the copy, as every copy but the main
output's is. None when no destination takes the copy.
The options carry copy as the library's only extra copy, so an
MP3 or Opus copy is always encoded at its own quality.
pub fn listing_backend(&self) -> Result<Box<dyn StorageBackend>>
Clone destination backends into a listing/match storage handle.
Reuses the backends acquire already built instead of spawning the destination guests a second time.
Returns an error when the operation fails.
pub fn into_listing_backend(self) -> Result<Box<dyn StorageBackend>>
Consumes this set and returns a listing backend over the primary destination.
Returns an error when the operation fails.
pub struct DestinationKeys {
/* private fields */
}
The destination an acquire request names files for, which translates its host keys.
The default stands for no destination in particular: host keys are
kept as written. crate::AcquireRequests made for one destination
carry that destination's backend.
pub fn of(backend: &dyn StorageBackend) -> Self
Keys translated by backend.
pub async fn translate(&self, host_key: String) -> String
The key the destination stores host_key under.
A key that is not a valid host key (a reconcile pattern, an empty name) is returned as given. When the destination cannot translate it (its guest is down), the host key is returned and a warning logged: the write that follows then either stores it as written or fails and is retried like any other failed copy.
Implements: Clone, Debug, Default
pub struct LibraryCopyFile {
pub name: String,
pub role: FileRole,
pub size: u64,
}
One file of a library's copy, as the library listed it.
Fields
name: The file's path below the title's folder on the library. Only its extension is read, to tell what kind of file it is; nothing is ever written to this path.role: What the file is.size: Exact size in bytes.
Implements: Clone, Debug, Eq, PartialEq
pub struct LibraryCopyListing<'a> {
pub store: &'a str,
pub plugin_key: &'a str,
pub account_id: &'a str,
pub marketplace: &'a str,
pub product_id: &'a str,
}
The title an acquire wants, as its store lists it.
Fields
store: Store id as this install's books name it (the store's alias, such asaudible).plugin_key: Key of the plugin the title belongs to on this install.account_id: Store account on this install that owns the title.marketplace: Lowercase marketplace; empty for a store without marketplaces.product_id: The store's product id.
Implements: Clone, Copy, Debug, Eq, PartialEq
pub struct LibraryCopySource {
/* private fields */
}
The libraries an acquire asks for a copy before it fetches from the store. The default asks none, so every title comes from its store.
pub fn new(copies: Arc<dyn LibraryCopies>) -> Self
Asks copies for each title before its store.
pub fn read_by(self, media: Arc<MediaPool>) -> Self
The same source, reading downloaded files with media instead of
the process pool.
Implements: Clone, Debug, Default
pub struct MatchStorageOptions {
pub account: Option<String>,
pub clear_missing: bool,
pub asins: Vec<String>,
pub only_mark_found: bool,
pub only_clear_missing: bool,
pub fix_layout: bool,
pub download: DownloadOptions,
}
Options for match_storage_to_library.
Fields
account: Optional account id filter;Nonemeans all accounts.clear_missing: Clear Acquired when no matching audio is found.asins: Limit to these ASINs / ids (empty = all).only_mark_found: When true, only mark found files as Acquired (do not clear missing).only_clear_missing: When true, only clear Acquired rows with no matching file.fix_layout: Relocate matched audio (and sidecars) onto the configured template layout.download: When true, treat unmatched titles as download candidates.
Implements: Clone, Debug, Default
pub struct MatchStorageSummary {
pub matched: u32,
pub relocated: u32,
pub cleared: u32,
pub unchanged: u32,
pub unmatched_files: u32,
}
Summary of a storage→library match run.
Fields
matched: Titles matched to existing storage objects.relocated: Titles whose storage_key was updated to a newly matched object.cleared: Titles cleared from Acquired because no matching file was found.unchanged: Titles already consistent with storage.unmatched_files: Storage keys that could not be matched to a library row.
Implements: Clone, Debug, Default
pub struct NamingContext {
pub asin: String,
pub title: String,
pub subtitle: Option<String>,
pub authors: Option<String>,
pub narrators: Option<String>,
pub series: Option<String>,
pub series_index: Option<String>,
pub series_asin: Option<String>,
pub year_published: Option<i32>,
pub account_id: Option<String>,
pub account_nickname: Option<String>,
pub locale: Option<String>,
pub language: Option<String>,
pub publisher: Option<String>,
pub categories: Option<String>,
pub length_minutes: Option<i64>,
pub bitrate: Option<i64>,
pub samplerate: Option<i64>,
pub channels: Option<i64>,
pub codec: Option<String>,
pub is_abridged: bool,
pub content_kind: Option<String>,
pub call_number: Option<String>,
pub chapter_number: Option<u32>,
pub chapter_count: Option<u32>,
pub chapter_title: Option<String>,
}
Metadata available to naming templates.
Fields
asin: Primary product id (Audible ASIN, ISBN, …) for this job.title: Display title used for naming and metadata.subtitle: Optional subtitle substituted into naming templates.authors: Author list for naming templates and tags.narrators: Narrator list for naming templates and tags.series: Series name for naming templates, when present.series_index: Series position string for naming templates, when present.series_asin: Podcast parent / series ASIN forSavePodcastsToParentFolder.year_published: Publication year for<year>(frompublished_atwhen known).account_id: Store account id that owns this title.account_nickname: Operator-facing account label for<account nickname>templates.locale: Marketplace / locale code for naming templates.language: Content language code for naming templates.publisher: Publisher name for naming templates, when known.categories: Category / genre path list for naming templates.length_minutes: Runtime in minutes for naming templates, when known.bitrate: Audio bitrate label substituted into naming templates, when known.samplerate: Audio sample rate in Hz for naming templates, when known.channels: Channel count for naming templates, when known.codec: Audio codec label for naming templates, when known.is_abridged: Whether the edition is abridged (template boolean).content_kind: Title kind (book,episode, …) for naming templates.call_number: The edition's short Bookclerk id for<call number>;Nonefor a title with no edition, which then names by its product id.chapter_number: Chapter-specific: 1-based index.chapter_count: Total chapter count when naming a split chapter file.chapter_title: Chapter title when naming a split chapter file.
Implements: Clone, Debug, Default
pub struct AcquireMetadata {
/* private fields */
}
The deployed metadata providers an acquire asks for the plain-audio overlay (catalog tags, cover and chapter layout of an enriched ASIN), and what turns the overlay's cover URL into an image.
pub fn new(providers: Arc<dyn MetadataProviders>) -> Self
Wraps the providers an acquire should ask.
providers- Usually the daemon's or CLI's integration registry.
pub fn with_covers(self, covers: Arc<dyn CoverImages>) -> Self
The same providers, with covers fetching the overlay's cover.
pub fn providers(&self) -> &dyn MetadataProviders
The wrapped providers.
Implements: Clone, Debug, Default
pub struct AcquireRequest {
pub asin: String,
pub holding_id: Option<String>,
pub source: String,
pub account_id: String,
pub title: String,
pub authors: Option<String>,
pub narrators: Option<String>,
pub series: Option<String>,
pub series_index: Option<String>,
pub options: DownloadOptions,
pub files_dir: PathBuf,
pub cache_dir: PathBuf,
pub force: bool,
pub write_destinations: Option<Vec<DestinationId>>,
pub placement: Option<PlacementFacts>,
pub job_id: Option<String>,
pub temp_quota_bytes: Option<u64>,
pub call_number: Option<String>,
pub parent_call_number: Option<String>,
pub metadata: AcquireMetadata,
pub library_copies: LibraryCopySource,
pub destination_keys: DestinationKeys,
}
Request to acquire a single title.
Fields
asin: Download product id (Audible ASIN / Libro ISBN).holding_id: The account holding (holdings.id) the title is acquired for, when known: acquire state is written under it. Without one the state is written to the titleasinnames onaccount_id.source: Which store owns this title (plugin id:audible,libro, …).account_id: Store account id that owns this title.title: Display title used for naming and metadata.authors: Author list for naming templates and tags.narrators: Narrator list for naming templates and tags.series: Series name for naming templates, when present.series_index: Series position string for naming templates, when present.options: Source download options (quality, chapter prefs, …).files_dir: Bookclerk files directory root (BOOKCLERK_FILES_DIR).cache_dir: Scratch directory for encrypted + decrypted temps.force: When true, download even if matching media already exists in storage.write_destinations: When set, only write prepared audio to these destinations (output.multi_destination = refetch_missing).placement: What the destination filters know about the book. When set, the acquire writes only to the destinations whose filter takes it (AcquireDestinations::for_book) and fails before fetching when none does.Nonewrites to every destination given: a host acquiring for an owner gets destinations the owner already chose.job_id: Durable daemon job id that owns this acquire's scratch directory.temp_quota_bytes: Refuse a new scratch dir when registered temps already exceed this.call_number: The<call number>to name the title by instead of its edition inlibrary: a host acquiring for a shared node names the title as the owner does, since its scratch library's edition ids are its own.parent_call_number: Likewise for the title's podcast parent, whose folder an episode may be saved in.metadata: Metadata providers asked for the plain-audio catalog and chapter overlay. The default has none, so plain audio keeps its store tags.library_copies: The libraries asked for the copy they already hold of the title before it is fetched from its store. The default asks none.destination_keys: The destination this request names files for, which translates the host keys its naming templates build. The default keeps host keys; the per-destination requests set it for each destination.
Implements: Clone, Debug
pub struct AcquireResult {
pub asin: String,
pub storage_key: String,
pub written_keys: Vec<String>,
pub matched_existing: bool,
pub from_library: bool,
pub store_fetches: u32,
pub copies: Vec<CopyOutcome>,
}
Result after a successful acquire.
Fields
asin: Primary product id (Audible ASIN, ISBN, …) for this job.storage_key: Object-storage key written for the primary audio artifact.written_keys: All object keys written during this acquire (audio + sidecars).matched_existing: True when an existing file was matched and no download ran.from_library: True when the audio was taken from the copy a library this install joined already held, instead of fetched from the store.store_fetches: How many times this acquire downloaded the title (or its companion PDF) from the store: 0 for a copy matched in storage or taken from a joined library, 1 for a store download (even one a better copy from a joined library replaced afterwards), 2 when that library copy then failed packaging and the store was asked again. Download pacing counts each against the account (bookclerk_library::pacing).copies: Each destination this acquire wrote or found the audio on, and the ones whose writes failed; recorded as the edition's copies (edition_copies) once the acquire returns. Empty when nothing was written or checked per destination (an edition already stored everywhere, or a companion PDF).
Implements: Clone, Debug, Deserialize<'de>, Serialize
pub struct CopyOutcome {
pub destination: DestinationId,
pub result: CopyResult,
}
What one destination ended up with in one acquire.
Fields
destination: The destination's id.result: Whether the destination now holds the audio.
Implements: Clone, Debug, Deserialize<'de>, Eq, PartialEq, Serialize
pub struct PlannedTitleNames {
pub folders: Vec<String>,
pub own_folder: bool,
pub audio_stems: Vec<String>,
pub part_patterns: Vec<String>,
}
The names an acquire of one title writes on one destination, as the owner plans them, to bound a host's upload of that title.
Fields
folders: The folders the title's files go under (planned_title_folders).own_folder: True when the folder template has a component of the title's own (its id, title or subtitle), so no other title's files go there and any name under it is the title's. False for a folder several titles share (<author>), the destination's root, or a podcast parent's folder: there only the names below are the title's.audio_stems: The planned audio key of a single-file acquire, without its extension (one per folder variant): the title's audio is this stem with any audio extension, and its sidecars this stem, a.and a suffix.part_patterns: Part and chapter keys without their extension, withNAME_WILDCARDandNUMBER_WILDCARDwhere the source's part or chapter title and number go. Those titles are not known before the acquire runs, so they are matched rather than listed; a template with no part or chapter field yields a literal name.
Implements: Clone, Debug, Eq, PartialEq
pub struct ReconcileOptions {
pub account: Option<String>,
pub clear_missing: bool,
pub asins: Vec<String>,
pub only_mark_found: bool,
pub only_clear_missing: bool,
pub download: DownloadOptions,
pub fence: Option<JobFence>,
pub checkpoint: Option<JobCheckpoint>,
}
Options for reconcile_library.
Fields
account: Optional account id filter;Nonemeans all accounts.clear_missing: When true, books marked Acquired whose file is missing become NotAcquired.asins: Limit to these ASINs (empty = all).only_mark_found: When true, only mark found files as Acquired (do not clear missing).only_clear_missing: When true, only clear Acquired rows with no matching file.download: Naming prefs (templates, podcast parent folder) for planned-path matching.fence: Lease fence when this reconcile is a phase of a durable job.checkpoint: Checkpoint to resume, whenfenceis set.
Implements: Clone, Debug, Default
pub struct ReconcileSummary {
pub matched: u32,
pub cleared: u32,
pub unchanged: u32,
}
Summary of a reconcile run.
Fields
matched: Titles matched to existing storage objects.cleared: Titles cleared from Acquired because no matching file was found.unchanged: Titles already consistent with storage.
Implements: Clone, Debug, Default
pub struct StorageIndex {
/* private fields */
}
Index of storage object keys, keyed by ASIN found in the path.
pub async fn from_storage(storage: &dyn StorageBackend) -> Result<Self>
Empty overlay.
This does not list storage. Call crate::scan_storage
when identity matching needs a durable index. Exact keys are resolved
with StorageBackend::exists at decision time.
Always returns Ok. The storage argument is accepted so older call
sites keep compiling while they move to crate::scan_storage.
pub fn with_scan(scan_id: impl Into<String>) -> Self
Overlay plus a durable scan id.
pub fn scan_id(&self) -> Option<&str>
Durable scan generation, when present.
pub fn insert_key(&mut self, key: String)
Insert a storage key into the index (test helper / incremental).
pub fn contains_key(&self, key: &str) -> bool
Returns true when key is present in the index.
pub fn keys_matching_pattern(&self, pattern: &str) -> Vec<String>
Overlay keys matching a reconcile wildcard pattern, best media rank first.
pub fn find_key_matching_pattern(&self, pattern: &str) -> Option<&str>
First key matching a reconcile wildcard pattern, preferring better media types.
pub fn best_key_for_asin(&self, asin: &str) -> Option<&str>
Best matching storage key for an ASIN / ISBN token, if any.
Implements: Clone, Debug, Default
pub struct ScanCheckpoint {
pub v: u32,
pub op: String,
pub instance_id: String,
pub namespace: String,
pub generation: String,
pub phase: String,
pub cursor: Option<String>,
pub node_local: bool,
pub placement: Option<String>,
}
Versioned scan progress stored in the job payload checkpoint.
Fields
v: Schema version. Unknown versions fail closed.op: Alwaysstorage_scan.instance_id:StorageBackend::instance_idthis cursor belongs to.namespace: List prefix (empty scans the namespace).generation:storage_scan_rows.scan_id.phase:listwhile paging,applyafter the scan committed.cursor: Backend list cursor.Nonestarts at the first page.node_local: True when the list index or scratch is node-local.placement: Host placement for node-local bytes. Absent means the scan was recorded as portable. A mismatch restarts instead of adopting another node's rows.
Implements: Clone, Debug, Deserialize<'de>, Eq, PartialEq, Serialize
pub enum AcquireError {
Source(SourceError),
Storage(StorageError),
Media(MediaError),
Library(LibraryError),
Io(Error),
NoDestinationAccepts,
NoDestinationReachable { errors: String },
Other(Error),
}
Failures from the acquire / convert / reconcile pipeline.
Variants
Source: Failure originating in a content-source plugin.Storage: Failure originating in an object-storage backend.Media: Failure originating in media encode/remux/fixup.Library: Failure originating in the library database layer.Io: Filesystem or other I/O failure.NoDestinationAccepts: No deployed destination's filter takes the book, so the acquire stops before it fetches anything.NoDestinationReachable: No deployed destination answered a listing, so an acquire run has nowhere to write.errors: Each destination with its listing error,;-separated.
Other: Catch-all for otherwise unclassified failures.
Implements: Debug, Display, Error, From<Error>, From<LibraryError>, From<MediaError>, From<SourceError>, From<StorageError>
pub enum CopyResult {
Stored { key: String },
Failed { error: String },
}
Whether a destination holds an acquire's audio.
Variants
Stored: The destination holds the complete files.key: The edition's primary audio key there.
Failed: The destination's writes failed; the copy is retried later.error: The last write's error.
Implements: Clone, Debug, Deserialize<'de>, Eq, PartialEq, Serialize
pub fn from_store(copy_store: Option<&str>, store: &str) -> bool
True when the library downloaded its copy from store, the store this
install's title belongs to. A copy whose store the library does not
know is never taken, and one from another store only replaces the
install's own download when it is the better copy.
An implementation of LibraryCopies that asks several libraries uses
this to pass over a copy from another store and ask the next library.
pub async fn match_storage_to_library(
library: &LibraryStore,
storage: &dyn StorageBackend,
options: MatchStorageOptions,
) -> Result<MatchStorageSummary>
List audio in storage, probe metadata (no body download), match to library.
Returns an error when the operation fails.
pub async fn match_storage_using_index(
library: &LibraryStore,
storage: &dyn StorageBackend,
options: MatchStorageOptions,
index: &StorageIndex,
) -> Result<MatchStorageSummary>
Match library rows against an existing scan index.
Does not delete the generation. The caller finishes it when the job completes.
Returns an error when the operation fails.
pub fn audio_basename(storage_key: &str) -> String
Basename of the acquired audio file (for .cue FILE lines).
pub fn chapter_storage_key(
ctx: &NamingContext,
folder_template: Option<&str>,
chapter_file_template: Option<&str>,
replacement_rules: &[ReplacementRule],
chapter_number: usize,
chapter_title: &str,
ext: &str,
) -> String
Storage key for a split chapter file.
pub fn chapter_storage_key_with_folder(
folder_ctx: &NamingContext,
file_ctx: &NamingContext,
folder_template: Option<&str>,
chapter_file_template: Option<&str>,
replacement_rules: &[ReplacementRule],
chapter_number: usize,
chapter_title: &str,
ext: &str,
path_limits: PathLimits,
) -> String
Chapter storage key with a separate folder naming context (podcast parent folder).
pub fn default_storage_key(
authors: Option<&str>,
title: &str,
asin: &str,
ext: &str,
) -> String
Build a relative storage key for a acquired title using the default
NamingProfile (Audiobookshelf).
Uses POSIX separator rules for creation. Reconcile probes this layout with wildcard sanitization so historical Windows/S3 keys still match.
pub fn resolve_templates(
profile: NamingProfile,
folder_template: Option<&str>,
file_template: Option<&str>,
chapter_file_template: Option<&str>,
) -> ResolvedNamingTemplates
Resolve optional template overrides against a naming profile.
pub fn sidecar_key(audio_storage_key: &str, suffix: &str) -> String
Replace the audio file extension on a storage key with a sidecar suffix.
pub fn storage_key(
ctx: &NamingContext,
folder_template: Option<&str>,
file_template: Option<&str>,
ext: &str,
) -> String
Build a storage key from optional classic Libation folder/file templates.
pub fn storage_key_with_contexts(
folder_ctx: &NamingContext,
file_ctx: &NamingContext,
folder_template: Option<&str>,
file_template: Option<&str>,
ext: &str,
replacement_rules: &[ReplacementRule],
path_limits: PathLimits,
) -> String
Build a storage key using separate folder and file naming contexts.
Used when SavePodcastsToParentFolder evaluates the folder template against
the podcast parent while the file template still uses the episode.
pub fn storage_key_with_rules(
ctx: &NamingContext,
folder_template: Option<&str>,
file_template: Option<&str>,
ext: &str,
replacement_rules: &[ReplacementRule],
) -> String
Like storage_key but applies classic ReplacementCharacters.
pub fn swap_audio_extension(key: &str, new_ext: &str) -> String
Replace the audio file extension on a storage key (e.g. .m4b → .mp3).
pub async fn acquire_book(
library: &LibraryStore,
destinations: &AcquireDestinations,
req: AcquireRequest,
source: &dyn ContentSource,
) -> Result<AcquireResult>
Run the acquire pipeline for one book.
library- Library store for status updates.source- Content-source plugin used to fetch the title.destinations- Output backends to write packaged audio into.req- Title identity, account, and download options.
AcquireResult describing written keys and match status.
Returns AcquireError when fetch, package, storage, or library updates fail.
pub async fn acquire_book_indexed(
library: &LibraryStore,
destinations: &AcquireDestinations,
req: AcquireRequest,
index: Option<&mut StorageIndex>,
source: &dyn ContentSource,
) -> Result<AcquireResult>
Acquire with an optional pre-built StorageIndex (avoids re-listing storage
when liberating many titles). On success, newly written keys are inserted into
the index so later books in the same batch can match them.
Fetch always goes through ContentSource::fetch_title (Plain only;
Audible decrypts Adrm/CENC inside the plugin before returning).
Every key the run writes on a destination (audio, chapter or part
files, sidecars, the companion PDF) is recorded for the book's edition
in the library's edition_files, with its role, just before it is
written, so a host upload of another edition is never granted it.
A book whose edition another book has already acquired (the same edition bought on two accounts, say) is marked acquired at that book's key without fetching it again.
Returns an error when the operation fails.
pub async fn acquire_pdf_only(
library: &LibraryStore,
destinations: &AcquireDestinations,
req: &AcquireRequest,
source: &dyn ContentSource,
) -> Result<AcquireResult>
Download and store companion PDF only (classic acquire --pdf).
Fetches via ContentSource::fetch_title; requires PlainFetch.pdf_url, then HTTP GETs that URL.
Each PDF key written is recorded for the book's edition in edition_files.
Returns an error when the operation fails.
pub async fn call_number_of(
library: &LibraryStore,
holding_id: &str,
) -> Result<Option<String>>
The short Bookclerk id of the edition account holding holding_id
(holdings.id) sells, for <call number>
(bookclerk_library::call_number).
None when no account holding has that id.
Returns an error when the holding cannot be read.
pub async fn convert_copy(
library: &LibraryStore,
destinations: &AcquireDestinations,
req: AcquireRequest,
copy: &OutputCopy,
) -> Result<AcquireResult>
Adds copy to the edition of the title req names, made from a copy
the library already stores, on every destination whose filter takes
that copy.
The source is the best stored copy the destinations hold: one kept as
delivered or as an M4B before an encoded one, then the main output's
copy, then the primary destination's. Its files are downloaded into
scratch and packaged like a store download, so tags, chapters, cover
and naming follow the copy's options. The new files are recorded as
copy's on the edition (edition_files); the title's storage key and
its main copy are left as they are. No store is asked.
library- Library store holding the edition's recorded files.destinations- Every reachable destination, with its current filter.req- The title, as an acquire of it would name it;forcemakes the copy again when the edition already has it.copy- The copy to add.
What was written: matched_existing is true when the edition already
had the copy and force was off.
Returns AcquireError::Other for a copy that is not a new encoding
(original), the library's main output, a title that has no library
row or is not stored, or an edition with no stored audio on a reachable
destination; AcquireError::NoDestinationAccepts when no destination
takes the book or the copy; and the download, packaging or storage error
otherwise.
pub async fn planned_storage_key(library: &LibraryStore, req: &AcquireRequest) -> String
Compute the storage key that would be used (for dry-run / set-status).
Uses the library row (when present) so podcast episodes honor
save_podcasts_to_parent_folder the same way as a real acquire.
pub async fn planned_storage_key_for(
library: &LibraryStore,
req: &AcquireRequest,
ext: &str,
) -> String
Planned audio storage key for ext, honoring folder/file templates and
save_podcasts_to_parent_folder: the host key the templates build,
translated by the request's destination
(AcquireRequest::destination_keys).
pub async fn planned_storage_key_with_rules(
library: &LibraryStore,
req: &AcquireRequest,
ext: &str,
replacement_rules: &[ReplacementRule],
) -> String
The host key planned_storage_key_for translates, built with an
explicit replacement-rule set and not translated (used by reconcile to
probe wildcard patterns across sanitization profiles).
pub async fn planned_title_folders(
library: &LibraryStore,
req: &AcquireRequest,
) -> Vec<String>
The folders an acquire of req writes the title's files under on a
destination with req.options: the folder of its planned audio key,
which its sidecars, chapter files and parts share. When podcast
episodes are saved to the parent's folder, the episode's own folder is
included too, since a library without the parent's row names it so. An
empty string is the destination's root (a template with no folder).
pub async fn planned_title_names(
library: &LibraryStore,
req: &AcquireRequest,
) -> PlannedTitleNames
The title's planned names on the destination req.options describe:
PlannedTitleNames.
pub async fn share_acquired_sibling(
library: &LibraryStore,
destinations: &[(&str, &dyn StorageBackend)],
title: &AcquireTitle,
) -> Result<Option<String>>
Marks book acquired at the files of its edition, when another book
of the edition acquired it (edition_acquisitions) and every
destination holds the copy, so the edition is not fetched again. The edition's companion PDF is shared too
when that book has one.
A destination holds the copy when the sibling's primary key there is the edition's own (unrecorded, as storage matching adopts it, or recorded for this edition, never another's: a listing that moved editions leaves its old files with the old one) and every audio key recorded for the edition there (parts, chapters) is present.
destinations- Each destination's id and backend.book- The book to mark acquired.
The shared primary key; None when the book has no edition yet, or no
other book of its edition is acquired with a copy every destination
holds (a destination deployed since, another node's layout, or a file
removed since). The acquire then plans as usual and syncs or fetches
what is missing.
Returns an error when a library lookup or write, or a backend probe, fails.
pub fn extract_asins_from_key(key: &str) -> Vec<String>
Extract Audible-like ASINs and ISBN-13 tokens from a storage key / file path.
Matches:
- filename stem
B00EXAMPLE.m4b - bracket form
Title [B00EXAMPLE].m4b(classic Libation) - path segment containing a standalone ASIN token
- ISBN-13 shaped tokens (13 digits, optional hyphens)
pub async fn find_existing_for_request(
index: &StorageIndex,
storage: &dyn StorageBackend,
library: &LibraryStore,
req: &AcquireRequest,
) -> Result<Option<String>>
Same as find_existing_for_title but for a acquire request before DB status is Acquired.
Matching strategy:
- Exact planned path under the creation replacement rules
- Same templates with sanitizable characters as wildcards (cross OS/backend)
- Template path without podcast-parent rewrite (exact, then wildcard)
- ASIN token found anywhere in a storage key
Returns a storage or database error from the lookup. A completed miss is
Ok(None).
pub async fn find_existing_for_title(
index: &StorageIndex,
storage: &dyn StorageBackend,
library: &LibraryStore,
title: &AcquireTitle,
download: &DownloadOptions,
) -> Result<Option<String>>
Find an existing acquired file for a title (planned path or ASIN-in-path).
Honors configured folder/file templates and save_podcasts_to_parent_folder
via the same path planner as acquire.
Returns a storage or database error from the lookup. A completed miss is
Ok(None) and is not an error.
pub async fn reconcile_library(
library: &LibraryStore,
storage: &dyn StorageBackend,
options: ReconcileOptions,
) -> Result<ReconcileSummary>
Scan storage and update library acquire status / storage_key for matches.
Returns an error when the operation fails.
pub async fn scan_storage(
library: &LibraryStore,
storage: &dyn StorageBackend,
fence: Option<&JobFence>,
prior: Option<&JobCheckpoint>,
probe_metadata: bool,
) -> Result<StorageIndex>
Builds or resumes a durable identity/object index.
Returns AcquireError when listing, the database, or a checkpoint write fails,
or when prior names a different storage instance.
pub type Result<T> = Result<T, AcquireError>;
Result alias for AcquireError.
pub type LibraryCopyError = Box<dyn Error + Send + Sync>;
Why a library's copy could not be asked for, downloaded or used. The acquire logs it and fetches from the store instead.
pub const NAME_WILDCARD: char = bookclerk_library::control_plane::UPLOAD_NAME_WILDCARD;
In a PlannedTitleNames pattern: one or more characters of a file
name (no /), where the source's part or chapter title goes; the
marker the owner's upload scope matches
(bookclerk_library::control_plane::TitleNames).
pub const NUMBER_WILDCARD: char = bookclerk_library::control_plane::UPLOAD_NUMBER_WILDCARD;
In a PlannedTitleNames pattern: one or more ASCII digits, where a
part or chapter number goes (the owner's upload scope matches it too).
Generated from crates/bookclerk-acquire at main · View as Markdown