# Repository automation

This page explains the automation App and the workflows around the main CI
run: dependency updates, security advisories, docs publishing, the website
checks and the Cloudflare Worker deploys.

The website checks also run on pull requests that change the site. The
planner, its jobs and `CI Gate` are in [Continuous integration](/docs/contributing/ci/index.md).

## Dependabot auto-merge

`.github/workflows/dependabot-auto-merge.yml` runs
[`fastify/github-action-merge-dependabot`](https://github.com/fastify/github-action-merge-dependabot)
when a Dependabot pull request opens or updates. It approves the pull
request and turns on GitHub auto-merge (squash) **for that pull request
only**.

GitHub then merges it, or adds it to the merge queue when one is required,
once `CI Gate` and `PR scan / osv-scan` pass. Failing pull requests stay
open for a human.

The repository-level **Allow auto-merge** setting only makes the
per-pull-request "Enable auto-merge" button and API available. It never
merges a pull request on its own, so human pull requests merge exactly as
before unless someone opts one in.

The action must run while checks are still pending, because GitHub rejects
enabling auto-merge on a pull request that is already mergeable. To retry a
Dependabot pull request (for example, after adding the secrets), comment
`@dependabot rebase`.

### What Dependabot updates

Version updates (`.github/dependabot.yml`, weekly, with a 7-day cooldown)
cover every lockfile of a shipped artifact or its build:

- Cargo at the workspace root;
- npm in `ui/`, `packages/plugin-sdk`, `tools/diagnostics-collector`,
  `tools/demo` and `tools/site`;
- uv in `packages/plugin-sdk-python`;
- GitHub Actions;
- the digest-pinned base images of `packaging/docker/Dockerfile`.

Tool versions that several workflows, images and scripts share live in
[`.tool-versions`](/docs/contributing/ci/index.md#tool-versions) instead. Dependabot leaves the uv
image and Node.js majors alone; the
[tool versions bump](/docs/contributing/ci/index.md#tool-versions) moves uv, cargo-nextest and Rust, and a
Node.js major moves by hand. Dependabot still updates the `node` image's
digest within the pinned major.

Packages that must share one version are grouped so they move in one pull
request: `capnp`, `capnp-rpc`, `capnpc` and `capnp-futures` in Cargo, and
`vitest` with its `@vitest/*` plugins in each Worker under `tools/`
(`@vitest/coverage-v8` accepts only the vitest release it shipped with, so
bumping either one alone fails `npm ci`).

`examples/` lockfiles are left out, because they are reference plugins and
never packaged. The full OSV scan still reports them.

The Python SDK has no runtime dependencies. Its `uv.lock` pins the `dev`
and `docs` extras that the `python_sdk` check and
`generate-api-docs.sh --python` install, so OSV scans them, and a runtime
dependency added later is scanned from the start.

- The install is `uv sync --locked --no-install-project`, which fails when
  the lockfile is out of date with `pyproject.toml`.
- The SDK is imported from `src/` rather than built, so its unlocked build
  backend never runs.
- [Docs publishing](#docs-publishing) reads the SDK's source with the
  standard library and installs nothing.

After changing `pyproject.toml` dependencies, run this in
`packages/plugin-sdk-python`:

```sh
uv lock
```

### Policy knobs

The knobs live in the workflow's `with:` block:

- **`target: any`** merges every update type. Tighten it to `minor` or
  `patch` to leave larger bumps for manual review.
- **`exclude: 'crate-a,crate-b'`** skips auto-merge for pull requests
  touching those packages.

### Setup

1. In **Settings → General → Pull Requests**, enable **Allow auto-merge**.
2. Create a GitHub App with repository permissions **Contents: read and
   write**, **Pull requests: read and write** and **Workflows: read and
   write** (GitHub refuses to let an App turn on auto-merge for a pull
   request that edits `.github/workflows/`), and no webhook. Install
   it on this repository only. Transferring the repository doesn't
   transfer the App: if the organization should own it, transfer the App
   separately.
3. Add two **Dependabot** secrets (**Settings → Secrets and variables →
   Dependabot**), because Dependabot-triggered runs don't receive Actions
   secrets:
   - `DEPENDABOT_MERGE_APP_CLIENT_ID`: the App's client ID.
   - `DEPENDABOT_MERGE_APP_KEY`: a private key generated for the App.

The App token matters. Merges and queue entries attributed to
`GITHUB_TOKEN` don't start new workflow runs, so `push` CI on `main`, the
OSV SARIF upload and the `merge_group` checks would be skipped. The App can
also approve pull requests without the "Allow GitHub Actions to create and
approve pull requests" setting.

### Before the merge queue exists

While the repository is still owned by a user, `Protect Main` keeps
"require branches to be up to date" on. Auto-merge then waits on a
Dependabot pull request that has fallen behind `main` until it is rebased.
Either:

- turn that option off in the `Protect Main` required-status-checks rule,
  so green Dependabot pull requests merge as soon as their checks pass.
  Dependabot rebases pull requests that actually conflict, and `push` CI on
  `main` is the safety net. This also relaxes the rule for human pull
  requests until the queue replaces it; or
- leave it on, and comment `@dependabot rebase` on pull requests that fall
  behind.

## Automation App

Scheduled workflows that open pull requests or file issues use the same
GitHub App as [Dependabot auto-merge](#dependabot-auto-merge), not
`GITHUB_TOKEN`:

- The organization doesn't allow GitHub Actions to create or approve pull
  requests, so `GITHUB_TOKEN` can't open them.
- Pushes and pull requests made with `GITHUB_TOKEN` don't start workflow
  runs, so their required checks would never report. App-token pushes
  start CI like any other contributor's.

| Workflow | Writes |
| --- | --- |
| `workerd-pin-bump.yml` | The `deps/workerd-pin` branch, pull request and auto-merge |
| `tool-versions-bump.yml` | The `deps/tool-versions` branch, pull request and auto-merge ([tool versions](/docs/contributing/ci/index.md#tool-versions)) |
| `coverage-floors.yml` | The `ci/coverage-floors` branch, pull request and auto-merge ([coverage](/docs/contributing/ci/index.md#coverage)) |
| `librofm-apk-probe.yml` | An `api-drift` issue; the `api-drift/librofm` branch and pull request |
| `graphicaudio-api-probe.yml` | An `api-drift` issue; the `api-drift/graphicaudio` branch and pull request |
| `chirp-api-probe.yml` | An `api-drift` issue; the `api-drift/chirp` branch and pull request |
| `osv-remediate.yml` | Reads OSV code-scanning alerts; the `deps/osv-remediation` branch, pull request and auto-merge |
| `cflite-batch.yml` | `fuzz-crash` issues for oracle failures; draft security advisories for crashes ([triage](/docs/contributing/fuzzing/index.md#fuzz-crash-triage)) |
| `osv-advisories.yml` | Reads OSV code-scanning and Dependabot alerts; draft security advisories ([verdicts](#osv-advisories)) |
| `docs-publish.yml` | Commits and release tags on the `bookclerk-docs` repository ([docs publishing](#docs-publishing)) |
| `merge-queue-requeue.yml` | Auto-merge on a pull request the queue removed only because a job never got a runner ([re-enqueue](/docs/contributing/ci/index.md#merge-queue-re-enqueue)) |
| `cla.yml` | Signature commits on the `cla-signatures` branch, the only writer its ruleset allows; the check's pull request comment and `Bookclerk CLA` check run ([CLA check](/docs/contributing/licensing/index.md#the-cla-check)) |
| `maintainer-review.yml` | The `Maintainer review` check run and the `touches-ci-or-build` label ([maintainer review](#maintainer-review)) |
| `signed-commits.yml` | The `Signed commits` check run and the pull request comment on how to sign ([signed commits](#signed-commits)) |
| `visual-baselines.yml` | Pending sets and `approved/*` tags on the `bookclerk-visual-baselines` repository; the review comment and the approved lock on a pull request's branch; reads `visual-reviewers` membership ([visual baselines](/docs/contributing/ui/visual-testing/index.md#baselines)) |

The probes share `.github/actions/api-drift-report`:

- One open issue per source, found by a hidden
  `<!-- api-drift:<source> -->` marker and updated in place, with a comment
  per new detection.
- When the probe's fixer produced verified changes, a draft pull request on
  a fixed branch that closes the issue. A maintainer marks it ready after
  CI and review.

Each workflow's own `GITHUB_TOKEN` stays read-only. Each job mints the App
token only when it has something to write, scoped to the permissions that
step needs. Commits are made through the API (`sign-commits`), so they are
signed and attributed to the App.

### Setup

1. Give the App these repository permissions, then accept the new
   permissions on its installation (an installation keeps its old
   permissions until they are accepted):
   - **Contents: read and write**, to push branches.
   - **Pull requests: read and write**, to open, update and auto-merge
     pull requests.
   - **Workflows: read and write**, only for
     `dependabot-auto-merge.yml`. GitHub refuses to let an App turn on
     auto-merge for a pull request that edits `.github/workflows/`, and
     Dependabot's GitHub Actions updates bump the action pins there. No
     other workflow requests it.
   - **Checks: read and write**, for the `Bookclerk CLA` check run
     (`cla.yml`), which GitHub lists under the App's name and the logo
     set in the App's **Display information**.
   - **Commit statuses: read and write**, so `cla.yml` can update a
     `Bookclerk CLA` commit status left on a commit from before the check
     became a check run.
   - **Issues: read and write**, to file and update drift issues and add
     labels.
   - **Code scanning alerts: read**, because `osv-remediate.yml` and
     `osv-advisories.yml` read the OSV alerts on `main`.
   - **Dependabot alerts: read**, for `osv-advisories.yml`.
   - **Repository security advisories: read and write**, because
     `cflite-batch.yml` and `osv-advisories.yml` file draft advisories and
     deduplicate against them.
   - **Metadata: read** (always granted).
   - **Variables: read and write**, as an organization or repository
     permission, or both. `diagnostics-collector-deploy.yml` stores the
     workers.dev collector URL when `DOMAIN` is unset, organization first
     and repository as the fallback. That token can't be narrowed, because
     the token API has no Variables permission to request, so the job mints
     it only for the one write.

   > [!NOTE]
   > Grant and accept a permission before merging a workflow change that
   > requests it. A job that asks for a permission the installation lacks
   > fails to mint its token, so `cla.yml`, for one, would report no
   > **Bookclerk CLA** check at all and block every pull request.

   Install the App on `bookclerk-docs` as well, with
   the same Contents permission, for `docs-publish.yml`.
2. Add two **Actions** secrets (**Settings → Secrets and variables →
   Actions**). Scheduled runs don't receive the Dependabot copies.
   - `AUTOMATION_APP_CLIENT_ID`: the App's client ID.
   - `AUTOMATION_APP_PRIVATE_KEY`: a private key generated for the App.
3. For `osv-remediate.yml`, add one more Actions secret,
   `OSV_REMEDIATION_BUNDLE_KEY`: a random passphrase (for example, from
   `openssl rand -base64 32`). It only encrypts the bump while CI tests it,
   so rotating it affects nothing but a run already in progress.
4. For `cflite-batch.yml`, add `FUZZ_CRASH_BUNDLE_KEY`, generated the same
   way and kept by maintainers. It seals crash bundles that are read after
   the run. Without it, a run that finds a crash fails before uploading
   anything.

`osv-advisories.yml` seals its verdict report with `FUZZ_CRASH_BUNDLE_KEY`
too, and its intermediate evidence with `OSV_REMEDIATION_BUNDLE_KEY`.

## Maintainer review

Anyone can open a pull request, and a fork's pull request runs its own
copy of the CI workflows, both on the pull request and in the merge queue.
So a green `CI Gate` on an outside contribution only proves that its own
workflows passed. The `Maintainer review` check makes sure someone with
write access has read the change before it can merge:

- A pull request passes when its author has write access to the
  repository, or is Dependabot or the automation App.
- Any other author's pull request passes only once someone with write
  access, other than the author, approves its **current head commit**, and
  no such reviewer's latest review asks for changes. Every push needs a
  fresh approval.
- For those pull requests, the check's page lists the changed files that
  run in CI or at build time, or have no text diff, and the pull request
  gets the `touches-ci-or-build` label. The list covers:
  - workflows and actions, scripts and Cargo configuration (`.github/`,
    `scripts/`, `.cargo/`);
  - the Dev Container, packaging, the lint tools and fuzz corpora;
  - build scripts, manifests and lockfiles anywhere (`build.rs`,
    `Cargo.toml`, `Cargo.lock`, `package.json`, `uv.lock` and the like);
  - toolchain and container files (`.tool-versions`,
    `rust-toolchain.toml`, `Dockerfile`);
  - binary files, and diffs too large to show.

  Read those files in full before approving.
- The check belongs to the head commit, and passes while any open pull
  request on that commit passes: a maintainer who opened or approved one of
  them vouched for exactly that commit, so a second pull request on it
  can't block it.
- Merge groups pass: every pull request in one already passed the check to
  enter the queue.

The check needs the approval to be on the head commit, so it never trusts
an approval given before a later push. When it doesn't update after you
approve, press **Re-run** on the check.

`maintainer-review-trigger.yml` runs on `pull_request` and
`pull_request_review` with no token permissions and no secrets.
`maintainer-review.yml` then runs from the default branch on
`workflow_run`, reads the pull request, its reviews and its changed files
through the API (`scripts/maintainer_review.py`), and posts the check as
the automation App. It never checks out or runs pull request code. A
first-time contributor's trigger run waits for a maintainer to approve
workflow runs, and until then no check arrives and the pull request stays
blocked.

### Setup

1. The App's **Checks** and **Pull requests** permissions (see
   [Automation App](#automation-app)) cover the check run and the label.
2. Once `maintainer-review.yml` is on `main`, add `Maintainer review` to
   the required status checks of the default branch's ruleset, expected
   from the automation App, like `Bookclerk CLA`.
3. Open pull requests whose head commit was pushed before the check
   existed have no `Maintainer review` check, so they wait for it forever
   and there is nothing to **Re-run**. Push to them, or submit any review
   (a comment review is enough): the review starts the trigger workflow
   without re-running CI.

## Signed commits

The default branch's ruleset accepts only commits that GitHub shows as
**Verified**. A pull request with an unsigned commit still passes CI and
can be approved, but GitHub never lets it into the merge queue, and
nothing on the pull request says why. The `Signed commits` check says why:

- It fails while any of the pull request's commits isn't verified, whoever
  wrote it. The ruleset exempts nobody, so neither does the check. It
  judges only pull requests into the default branch, the one the ruleset
  protects.
- It keeps one comment on such a pull request, posted as the automation
  App and edited in place on every push. The comment lists each unverified
  commit with GitHub's reason (not signed, a key that isn't a signing key,
  an unverified email, and so on), then the steps to set up SSH signing,
  re-sign the commits without changing them (a rebase from where the branch
  left the default branch), and force-push.
- Once every commit is verified, the check passes and the comment says so.
  A pull request whose commits were always verified gets no comment.
- The check belongs to the head commit and passes while any open pull
  request on that commit passes; the ruleset still judges each pull
  request's own commits. Merge groups pass.

When it doesn't update after GitHub finishes verifying a signature, press
**Re-run** on the check.

`signed-commits.yml` runs from the default branch on `workflow_run` when
the unprivileged `Maintainer review trigger` workflow completes for a pull
request event (opened, pushed to, reopened), so one trigger job serves both
checks. It reads the pull request's commits and comments through the API
(`scripts/signed_commits.py`) and never checks out or runs pull request
code. Like the maintainer review check, it waits for a maintainer to
approve a first-time contributor's workflow runs.

### Setup

1. The App's **Checks** and **Pull requests** permissions (see
   [Automation App](#automation-app)) cover the check run and the comment.
2. Optionally, once `signed-commits.yml` is on `main`, add `Signed commits`
   to the required status checks of the default branch's ruleset, expected
   from the automation App. The ruleset's **Require signed commits** rule
   already blocks the merge, so requiring the check only makes the reason
   show among the failed checks.

## OSV remediation

`osv-remediate.yml` runs after every successful full OSV scan of `main`
(push, schedule or manual run). It also runs on demand from `main`,
without a preceding scan. It reads the open OSV-Scanner
alerts in **Security → Code scanning** and looks each advisory up on
api.osv.dev, with its aliases.

A request to api.osv.dev or the GitHub API that times out, drops its
connection or gets a 408, 429 or 5xx answer is tried up to four times,
2, 4 and 8 seconds apart, or longer when a `Retry-After` header asks for
up to a minute. A request that still fails, and any other error, stops
the run with an error message.

When OSV doesn't know an alert's rule id, the other advisory ids in the
alert's rule and message text stand in for it. That happens, for example,
with a new CVE that OSV-Scanner chose to name a group of advisories.

`scripts/osv-remediate.py` then tries lockfile-only updates until no locked
version of the package is inside the advisory's affected ranges:

- **Cargo**: `cargo update --package <name>@<locked> --precise <fix>`,
  then the newest compatible release.
- **npm**: `npm update <name> --package-lock-only --ignore-scripts`.
- **uv**: `uv lock --no-build --upgrade-package <name>==<fix>`, then the
  newest allowed release.

`<name>` is the package, `<locked>` its locked version and `<fix>` the
smallest fixed version above it.

An attempt that doesn't clear the advisory is rolled back. Manifests are
never edited, so a fix that needs a new major version, or a change
upstream, stays an open alert for a maintainer.

A bump is published only once it is known to pass:

1. **`bump`** commits the bump locally and seals it, with the pull request
   body, using `OSV_REMEDIATION_BUNDLE_KEY`
   (`scripts/osv-remediation-bundle.sh`). Anyone can download the
   artifacts of a public repository, so the bump never travels in the
   clear.
2. **`verify`** calls `ci.yml` with `private-head: true`. Every CI job
   opens the sealed bundle right after checkout
   (`.github/actions/osv-remediation-head`) and tests the bump with the
   full suite. As in the merge queue, caches aren't saved, cargo output is
   quiet, and the first failed job cancels the run.
3. **`publish`** runs only when that `CI Gate` passed. It pushes
   `deps/osv-remediation` and opens the pull request ready for review,
   skipping the draft stage because the suite has already passed. It turns
   on auto-merge, so the merge queue lands it as soon as the pull request's
   required checks pass.

A bump that fails verification publishes nothing. The run fails, and the
alerts stay in **Security → Code scanning** for a maintainer. A run with
nothing to bump publishes nothing either.

While a remediation pull request is open, later runs stop at the `pending`
job and change nothing. Rebuilding `deps/osv-remediation` on every push to
`main` would restart the pull request's checks and drop it from the merge
queue, and the queue already tests it against the latest `main`.

- Once it merges or is closed, the next full scan of `main` starts a fresh
  run.
- To replace an open remediation pull request that failed its checks,
  close it and run the workflow manually.

Scan results aren't public, so the run keeps them private:

- Its log prints counts only. Command output is discarded, because a
  failed attempt would name a vulnerable package.
- The pull request lists only the bumps it applied, with their advisory
  ids.

That hides which advisories the project is acting on until the fix is
ready to land. It can't hide the advisories themselves: `main`'s lockfiles
are public, so anyone can scan them.

Dependabot doesn't cover this. It raises alerts and security updates only
for advisories GitHub has reviewed, and RustSec advisories often stay
unreviewed. Its version updates wait out the cooldown and never touch
transitive-only crates.

> [!NOTE]
> Pull requests the App opens aren't authored by a person. Rulesets that
> require approval for such changes still need a maintainer's approval
> before the merge queue takes them. Auto-merge stays on while it waits.

## OSV advisories

`osv-advisories.yml` runs after every successful full OSV scan of `main`,
next to the remediation, and on demand from `main`. It answers a different question: does an open dependency
alert affect something Bookclerk ships?

A draft repository security advisory is filed only when it does (see the
[ADR](/docs/adr/security-advisory-automation/index.md)). Drafts are private to
maintainers, and are filed before the first release too, which proves the
path. Nothing becomes public until a maintainer publishes.

### Verdicts

[`shipped-artifacts.toml`](https://github.com/Bookclerk/bookclerk/blob/main/shipped-artifacts.toml) declares the
shipped artifacts, and which lockfile entries end up in each. Every
lockfile in the repository is either listed there or named as not shipped;
`scripts/tests/test_osv_advisories.py` fails on an unclassified one.

`scripts/osv-advisories.py` takes each open OSV-Scanner alert and
Dependabot alert on `main`, merged per lockfile when their ids or aliases
overlap. It gives every artifact of that lockfile a verdict:

| Lockfile | Artifact |
| --- | --- |
| `Cargo.lock` | Host binaries, platform and optional guests |
| `Cargo.lock` | The Rust plugin SDK |
| `ui/package-lock.json` | The web UI, served by `bookclerkd` |
| `packages/plugin-sdk/package-lock.json` | `@bookclerk/plugin-sdk` |
| `tools/diagnostics-collector/package-lock.json` | The diagnostics collector Worker |
| `tools/demo/package-lock.json` | The hosted demo |
| `packages/plugin-sdk-python/uv.lock` | `bookclerk-plugin-sdk` (PyPI) |
| Anything else | None |

An artifact is affected when:

- **Host binaries and guests**: the crate is in the shipped closure of the
  shipped crates in `cargo metadata`. When the upstream advisory names
  vulnerable functions, one of them must also be in the DWARF of the
  fat-LTO release build of the shipped binaries (the method of
  OSV-Scanner's Rust call analysis).
- **The Rust plugin SDK**: the crate is in the SDK's shipped closure. It is
  a library, so there is no binary to analyze.
- **The web UI**: a production dependency has code in the production
  bundle (`vite build --sourcemap`, then the source maps). If the bundle
  can't be built, any production dependency counts.
- **`@bookclerk/plugin-sdk`**: a production dependency (not `dev` in the
  lockfile).
- **The diagnostics collector**: a production dependency. It has none
  today.
- **The Python SDK**: never by a dependency, because the SDK has no runtime
  dependencies. Both the test and the workflow fail if
  `[project].dependencies` or a non-dev extra gains one before its closure
  is resolved.
- **Anything else**: never. Examples, fuzzing, scripts and dev dependencies
  stay an alert plus the remediation bump.

The shipped Rust closure follows normal edges on every target platform. It
excludes dev and build dependencies and proc macros, except `*-src` build
dependencies, which compile vendored C into the binary. Their own
dependencies, such as `cc`, are build tooling and are excluded.

A test keeps the UI from importing package files as `?url` or `?raw`
assets, or as wasm, which the source maps wouldn't show.

### Call analysis

The call-analysis build takes hours. So it runs only when an advisory with
function data for a shipped crate has no advisory yet, or only an open
draft (which the analysis then refreshes). Even then, it runs only on the
weekly scheduled scan or a manual run; the `call-analysis` input forces
it.

- On other runs, such an advisory waits as "awaiting call analysis".
- Runs queue one at a time, and a newer run replaces one still waiting. So
  a scheduled run queued behind another can give way to a push's run. Its
  call analysis then waits a week, or a manual run with `call-analysis`
  does it now.
- An open draft whose advisory the analysis (or the web bundle) later shows
  unaffected is left as it is. It is reported as "open draft, now not
  affected" for a maintainer to close.

The web UI bundle is built only when an open advisory is on
`ui/package-lock.json`.

A build that fails, reports no executables, or leaves a binary with too
few functions or no `main` in its DWARF fails closed: the advisory is filed
on the closure alone. An advisory a maintainer judges unaffected is closed
as a draft (it isn't filed again) or ignored in `osv-scanner.toml`.

### What a draft carries

- **Upstream ids**, each as a hidden
  `<!-- osv-advisory: <id> <ecosystem>/<package> -->` marker, so it is
  never filed twice and a draft a maintainer closed isn't filed again.
- **The locked and first fixed versions.**
- **Severity**: a CVSS 3.1 vector from the GHSA record or the RustSec
  entry, else the highest GHSA severity. Failing both, low for an
  informational notice such as "unmaintained", and medium otherwise.
- **CWE ids**: CWE-1395 for every draft, the GHSA records' CWE ids, and
  CWEs mapped from RustSec categories and informational kinds, which
  RustSec records carry instead.
- **The verdict and its evidence**, saying so when reachability is
  unproven.
- **A proposed range** up to the highest release tag, for a maintainer to
  confirm. For the TypeScript and Python SDKs, it runs up to the version in
  their manifest.

The affected package's ecosystem is the one `shipped-artifacts.toml`
declares (`other` for packages on no public registry). The exception is an
artifact with a `crate_manifest`, which is filed as `rust` once that
crate's `publish` allows crates.io. Markers keep the declared ecosystem.

### Later runs

A later run that finds the same advisory never files a second draft. While
the first is still a draft, the run:

- rewrites only the automated part of its description, between the
  `<!-- bookclerk-auto: begin … -->` and `<!-- bookclerk-auto: end -->`
  comments (new evidence, fixed versions, aliases, the newest release);
- adds markers for new aliases;
- fills an affected range that is still unset, once a release exists.

Text a maintainer adds outside that part is left alone, and so are the
severity, CWE ids, summary and any range a maintainer set. Nothing is
rewritten when only the "Last seen" line would change. Closed and
published advisories are never touched.

One run files at most ten drafts and updates at most ten. A draft GitHub
refuses is reported, and fails the run without holding back the others.

Before the first release, no draft proposes a range, because nothing is
published yet. The draft says so, and a maintainer either closes it once
the fix is on `main`, or sets the range when a release ships the code.

### Jobs and the verdict report

Three jobs keep secrets away from third-party build code:

1. **`fetch`** mints a read-only App token (code scanning, Dependabot and
   repository advisories) for one step. That step reads the alerts, their
   records on api.osv.dev, the newest release and the advisories on
   record, then revokes the token. The job then computes the closures from
   the lockfiles and `cargo metadata` (which runs no build script), and
   decides on call analysis. Its output leaves the job sealed with
   `OSV_REMEDIATION_BUNDLE_KEY`.
2. **`analyze`** has no secrets and no token. It builds the web UI
   (`npm ci --ignore-scripts`) and, when planned, the shipped binaries. Its
   output names no advisory (the bundled packages and a truncated SHA-256
   of every function in each binary), so it travels unsealed.
3. **`file`** builds nothing. It mints an advisory-write token, files the
   drafts and writes the verdict report.

The verdict report records every decision:

- one row per advisory and shipped artifact, with its verdict, packages
  and outcome (filed, already on record, capped, failed);
- the advisories on lockfiles that don't ship;
- the evidence behind each verdict.

It never leaves the job in the clear. It is sealed with
`FUZZ_CRASH_BUNDLE_KEY` (the passphrase maintainers keep) and uploaded as
the `osv-advisories-report` artifact of that workflow run, kept 30 days.

To read it:

1. Download the sealed file from the run page (**Actions → OSV advisories
   → the run → Artifacts**), or with the GitHub CLI:

   ```sh
   gh run download <run-id> -n osv-advisories-report
   ```

   Replace `<run-id>` with the number in the run page's URL.

2. Open it with the passphrase:

   ```sh
   BUNDLE_KEY=<passphrase> scripts/osv-remediation-bundle.sh open sealed.bin <dir>
   ```

   Replace `<passphrase>` with the bundle key maintainers keep, and `<dir>`
   with a new folder to open the report into. `sealed.bin` is the file the
   download writes.

3. Read `<dir>/report.md`.

Logs print counts only. A new shipped artifact, or a new lockfile, goes
into `shipped-artifacts.toml` in the pull request that adds it. Cutting the
fix release automatically is a documented future goal, not built yet (see
the ADR).

## Docs publishing

`docs-publish.yml` keeps the separate public repository
[`Bookclerk/bookclerk-docs`](https://github.com/Bookclerk/bookclerk-docs)
in step with this one, as Markdown that the website and agents read.

### Main and releases

- **Its `main` mirrors this repository's `main`.** Every push to `main`
  builds the tree and commits it there, with a `Source-Commit: <sha>`
  trailer naming the commit it was built from. A build identical to the
  docs `main` adds no commit.
- **Bursts publish the newest commit only.** Runs on `main` share one
  concurrency group that cancels the older run. A run whose commit is no
  longer the tip of `main` when it reaches the publish step skips it.
- **A published release tags the matching docs commit** (the one whose
  trailer names the release's commit) with the same tag name. When no
  `main` commit matches because a burst skipped it, the release's build is
  committed beside `main`, parented on its head without moving it, and
  that commit is tagged. An existing tag is never moved.
- **A release's commit must be on `main`.** The build refuses a release
  whose tag names a commit the default branch doesn't contain (see
  [release tags](/docs/contributing/packaging/index.md#release-tags)).

### The tree

The tree is `docs/` as it is, with the generated references written into
the shelves their readers use. Only markdownlint's directive comments
(`<!-- markdownlint-disable-line … -->`,
`<!-- markdownlint-disable-next-line … -->`, `<!-- markdownlint-disable … -->`
and the rest) are left out of its pages' prose, since they matter to the
`lint` check alone; code spans and fenced code keep them.

Each reference has a folder, and `generated.json` at the root of the tree
lists them: the folder, its kind, its title, what it is generated from,
whether its landing page is hand-written, and every page it wrote. The
site reads the manifest rather than guessing from paths, so a reference
can move without a change to the Worker.

| Folder | Reference | Generated from | Landing page |
| --- | --- | --- | --- |
| `plugins/sdk/typescript/` | TypeScript plugin SDK | `packages/plugin-sdk` | hand-written guide |
| `plugins/sdk/python/` | Python plugin SDK | `packages/plugin-sdk-python` | hand-written guide |
| `plugins/sdk/rust/` | Rust plugin SDK | `crates/bookclerk-plugin-sdk` | hand-written guide |
| `server/api/` | Web API | `crates/bookclerkd/openapi.json` | generated index |
| `internals/crates/` | Rust crates | every workspace library | generated index |
| `contributing/ui/` | Web UI library | `ui/src/lib` | hand-written guide |

A reference with a hand-written landing page keeps that page's guide and
appends the generated part under its **Reference** heading: the list of
module pages, then the package or crate root. The module pages sit beside
it (`plugins/sdk/python/workerd.md`, `plugins/sdk/rust/net.md`); a module
with children is a folder with its own `README.md`. The build fails when
the landing page is missing, or when a generated page would land on a
hand-written one.

- **TypeScript**: TypeDoc's Markdown theme for `@bookclerk/plugin-sdk` and
  `ui/src/lib` (`typedoc-plugin-markdown`, a dev dependency of both). One
  page per module, without source links or breadcrumbs. The SDK's root
  module is named `plugin-sdk` (`@module` in `src/index.ts`) rather than
  TypeDoc's default `index`; the build folds it into the landing page and
  writes the other modules (`abi`, `workerd`, `sparse-workerd`, …) as
  sibling pages, so no single page carries the whole SDK.
- **Python**: `scripts/python_api_markdown.py` reads the SDK with `ast` (it
  imports nothing), and renders each public module's `__all__`, with
  Google-style sections as lists and cross-references as links. Packages
  are folders, modules are pages.
- **Rust**: `scripts/rustdoc_markdown.py` renders rustdoc JSON for every
  workspace library from a pinned nightly, one page per public module.
  Items re-exported from private modules are documented where they are
  re-exported. Signatures are rebuilt from the JSON, doc headings sit under
  their item, and intra-doc links come from rustdoc's own link table.
  `bookclerk-plugin-sdk` lands in the Rust SDK guide's folder; every other
  crate is `internals/crates/<crate>/README.md` with its modules beside
  it, and `internals/crates/README.md` groups them (plugin interface,
  plugins, host internals, development).
- **Web API**: `scripts/openapi_markdown.py` renders `bookclerkd`'s
  committed OpenAPI document (`crates/bookclerkd/openapi.json`) into
  `server/api/`. It writes
  an index with a quick-start request, the credentials and every route, one page per route group (with each
  route's access rule, parameters, body, responses, an example request
  as a `curl` command, a PowerShell `Invoke-RestMethod` call, Python
  (`requests`), JavaScript (`fetch`) and Rust (`reqwest`), and an
  example response for each status: the example the route states for
  that status (an error status shows the body that route really
  returns, the message the handler writes or the one the error
  middleware writes for the status), otherwise one built from the
  schemas, or a sentence saying why there is no JSON body; the request
  sends only required fields, since optional ones can be alternatives
  that must not be sent together), and a page of
  schemas. Parameters, body fields and responses are lists, one item per
  property (`` `name` (path) *type*, **required** — description ``), with
  an inline object's fields nested under it. A request body that names a
  schema lists that schema's fields in place, and a response whose body
  is an object, or an array of objects, lists its fields too, whatever
  its status or media type, so a route reads without a trip to the
  schemas page; when a status offers several media types, the fields
  and the example come from the JSON one.
  - The website lays the route group pages out like an OpenAPI viewer
    (`tools/site/api-reference.ts`): a method badge beside each path, the
    examples in a second column when the reading column is wide enough,
    and the usual "On this page" rail. The rail lists the routes nested
    by path, relative to the start every route of the page shares
    (`/login` on the Auth page, whose headings keep the full path), with
    the shared start of each deeper path written once above the routes
    under it. Every method badge is the same width so the paths line up,
    and the rail is sized to fit the longest path on one line.
  - The property lists become property rows (`tools/site/api-reference.ts`
    `layoutProperties`): name, location, type and a required mark on one
    line, the description under it, a nested object's fields folded with
    their count, and each response's body fields folded under its status
    mark. A row that folds is the fold's summary, so a tap anywhere on it
    opens the fields; every fold is closed on load and the page's script
    keeps at most one response's open at a time.
    Any other list, and the tables of pages generated before this format,
    pass through unchanged.
  - The example responses become one panel per status with a colored
    status mark, each holding the example body (JSON, or a text body in
    its own language; a binary body gets a sentence) and then, after a
    `Schema:` line, the body's schema: its JSON Schema for a JSON body
    (component references expanded in place), or a sentence naming the
    media type and the shape for any other body whose schema is a string,
    such as "A `text/plain` body: a string." (a non-JSON body with another
    schema keeps the JSON Schema block); the page's script shows the
    statuses as tabs in one card
    with an Example/Schema switch between the two. The example request's
    samples become one panel per language the same way, and the page's
    script shows them as a language picker. Picking a language or a view
    picks it for every route on the page and is kept in the browser's
    local storage, a display preference the privacy page lists; a page
    restored from the back/forward cache reads the choice again.
  - Like every generated page, each one ends with a Source on GitHub link
    to what it is generated from: `crates/bookclerkd/openapi.json` here.
  - Under **Web API** the sidebar lists every route group and the schemas
    page, not their routes; those are in the rail. The index, titled **Web API** like
    its sidebar entry and breadcrumb, puts its quick start beside the
    introduction and shows each route group as a card of badged routes.
  - The Markdown itself stays plain, so the pages read the same on GitHub.

Every relative link to a file in the tree must resolve, and links in a
generated page must stay inside the tree, or the build fails. The build
also refuses a page named `index.md` or in a folder named `index` (the
site serves a folder's `README.md` at the folder's URL and `<page>/index.md`
is the page's Markdown copy), and a page beside a folder of the same name
(make it that folder's `README.md`).

The Rust reference puts a module's docs at a different depth than its
source file, so a doc comment in a library crate must not link a file
relatively (a link target such as `../../docs/adr/x.md`). Write the path in a code span or
link an absolute URL instead; intra-doc links to Rust items are fine. The
build checks this before it starts, and so do the plan job's
`test_docs_markdown.py` tests on every pull request, because the build
itself runs only on `main`.

### Last updated dates

`last-updated.json` at the root holds the instant each page of `docs/`
last changed, in UTC to the second (`YYYY-MM-DDTHH:MM:SSZ`). The website
shows it as "Last updated", in the reader's own time zone.

- It is the author date of the newest commit that changed the page's
  content. A pure move (a rename git sees as unchanged) is followed back to
  the old path rather than counted. Author dates survive history rewrites
  that reset commit dates.
- The workflow checks out the full history and passes
  `--require-history`, so a shallow checkout fails the build. A local build
  from a shallow clone leaves the file out, with a warning.
- The build dates only hand-written pages. `docs-publish.py` adds the
  pages `generated.json` lists, since only it can compare them with the
  docs repository. A page whose content is unchanged there keeps its
  previous date, and a new or changed one gets the publish time. A
  generated page shows that date and names its source.
- Files and entries published before instants were written hold a date
  alone (`YYYY-MM-DD`, UTC). The site and the next publish still read them,
  and the site shows such a date as it is.

### Jobs and safety

- **`build`** has no secrets and a read-only token. It runs
  `scripts/docs-markdown.py` and uploads the tree as an artifact.
- **`publish`** checks out only `scripts/docs-publish.py` from the default
  branch. It mints the [Automation App](#automation-app) token for
  `bookclerk-docs` alone (contents: write), and writes one commit through
  the Git Data API, so GitHub signs it as the App. Pages over 32 KiB are
  uploaded as blobs first, so each create-tree call stays small. GitHub
  sometimes answers a create-tree call with HTTP 422 "Sorry, your request
  timed out" or a 5xx; the script retries those up to four times, waiting
  5, 10 and 20 seconds.

A manual run publishes only from `main`. The publish script refuses
redirects, so the token is only sent to `api.github.com`.

`docs-markdown.py` clears its output before each build. So it refuses:

- a scratch root (`--scratch-root`, default `target/docs-markdown/`) that
  isn't strictly inside the repository's `target/` or the system temp
  folder;
- an output folder that isn't strictly inside the scratch root;
- an existing output folder that lacks the marker file it writes;
- the filesystem root, the home folder, the repository root, `/mnt` and
  `/tmp`, as either the output or the scratch root.

### Build the tree locally

The tree lands in `target/docs-markdown/out/`:

```sh
rustup toolchain install nightly-2025-11-15 --profile minimal
python3 scripts/docs-markdown.py
```

The nightly is pinned because rustdoc's JSON format changes between
nightlies, and `rustdoc_markdown.py` reads format 56.

- Bump `RUST_NIGHTLY` and `RUST_JSON_FORMAT` in `docs-markdown.py` and the
  workflow together, and adjust the renderer for any change in the format.
- The nightly may predate the workspace's `rust-version`, so the build
  passes `--ignore-rust-version`. It only produces documentation.

## Website checks

`site_audit` (`tools/site/audit/audit.ts`) builds the site, serves it
through the real Worker under `wrangler dev`, and audits it with the
runner's Chrome.

The homepage's catalog sheets come from
`tools/site/src/catalog-records.json`, which the build and the Worker bundle
but never publish. Its `series` entries share names/universe context
and contain ordered `books`; unrelated novels live under `standalones`.
On each homepage request the Worker selects four distinct groups with equal
probability, then one uniformly selected book per group, and writes those
four sheets into the page. Visitors get no extra request, and neither the
full catalog nor its source links reach the page. Shared
universes, including the Cosmere, count as one group; unrelated standalones
count individually. Longer series cannot dominate a stack. The build-time
stack in `index.html`, used only if the Worker is bypassed, also uses four
different groups.
Metadata includes source editions, verification dates, complete narrator
credits, integer runtimes in minutes and optional series position/universe.
A card names a store (Audible, Libro.fm or Chirp) only when that store sells
the same edition; each non-Audible store a record names needs its product
page under `listings`, and the Audible page is `sourceUrl`.
Tests check series coverage, metadata integrity, store listings, escaping and distinct
group sampling independent of series size; the browser audit checks group
diversity and every authored card for text/stamp overlap.
The complete stack drops and stamps
once it fits below the header. Cards remain hidden until that entrance;
unsupported animation, reduced motion and very short views retain static
artwork. The page never captures or repositions ordinary scrolling.
After the drop, wheel input over an actual sheet optionally cycles the
front record behind the stack and stamps the next record, but only once a
mouse or pen has rested on the stack for 450 ms. Wheel events that arrive
sooner restart that delay, so scrolling past the stack stays ordinary
scrolling. A sideways touch
or mouse drag does the same on release. Native vertical panning and pinch
zoom remain available. Independent gestures can cycle every record;
residual wheel momentum cannot start another turn.
Site tests run the shipped controller with an independent bounds oracle
and cover the hover delay, local wheel targeting, repeated order changes, drag displacement,
taps, vertical swipes, multi-touch, lost capture, timeout, resize, navigation
and reduced motion. The browser audit also exercises real wheel events and
Chromium touch input, including native vertical scrolling. Cards keep a fixed
aspect ratio and centered, fitted artwork areas. The Worker-chosen records remain
visible without JavaScript. Page transitions and FAQ expansion use CSS
with ordinary navigation and native disclosures as fallbacks.

The pre-release notice measures its full contents and collapses to the
badge and detail link when it would occupy more than two lines, regardless
of viewport width. It rechecks after resizing and web-font loading; mobile
CSS provides the compact fallback when scripts are unavailable.
Marketing pages share the docs header's sticky behavior. Their section
links account for the measured header height so headings remain visible
when navigation wraps.

Docs pages are read from this checkout's `docs/` through a local server
(`tools/site/audit/serve.ts` and `entry.ts`), not from the docs repository.
So the audit covers the docs as the pull request has them, and needs no
network. A docs key the local server fails renders the docs-unavailable
page. The [visual checks](/docs/contributing/ui/visual-testing/index.md#website-and-collector-pages)
screenshot the site through the same server.

| Pages | Lighthouse (mobile and desktop) |
| --- | --- |
| `/`, `/download`, `/docs/guide/getting-started`, `/docs/contributing/ci` | Performance ≥ 90, accessibility 100, best practices ≥ 95, SEO 100, CLS ≤ 0.05 |
| 404 and docs-unavailable | Accessibility 100 |

The 404 and docs-unavailable pages' SEO and best-practice misses, such as
the status code and `noindex`, are intended.

- **Retries.** A run that misses only on performance or CLS is repeated, up
  to three runs, and the run with the fewest misses kept. Lab scores vary on
  shared runners, and Chrome sometimes records a load with no navigation
  start (`NO_NAVSTART`), more often on pages whose
  `Cross-Origin-Opener-Policy` moves the load to a new browsing context
  group; that run has no performance score or CLS.
- **Accessibility.** axe-core then checks every page in light and dark
  color schemes, at 1350 and 390 px, against WCAG 2.0 to 2.2 A/AA and its
  best-practice rules. Any violation fails the check.
- **Results.** The results table goes to the job summary. A failed audit
  uploads the Lighthouse reports as the `site-audit` artifact.
- **Chrome starts.** Chrome is started once more when its first start
  never reaches the debugging port (busy runners). The tail of each failed
  start's `chrome-err.log` goes to the job log, and a copy
  (`chrome-err-<attempt>.log`) sits beside the reports when the audit
  fails.

To run it locally:

1. In `tools/site`, run `npm run build`.
2. In `tools/site/audit`, run `npm ci && node audit.ts`. `--chrome <path>`
   or `CHROME_PATH` picks the browser.

## Cloudflare Worker deploys

Three workflows deploy Workers with `wrangler deploy`, sharing the
repository secrets `CLOUDFLARE_API_TOKEN` and `CLOUDFLARE_ACCOUNT_ID`:

| Workflow | Worker | Custom domain |
| --- | --- | --- |
| `site-deploy.yml` | `bookclerk-site` | `<DOMAIN>` |
| `diagnostics-collector-deploy.yml` | `bookclerk-diagnostics` | `diagnostics.<DOMAIN>` |
| `demo-deploy.yml` | `bookclerk-demo` | `demo.<DOMAIN>` |

`DOMAIN` is the organization variable (default `bookclerk.app` for the
site). With `DOMAIN` unset, the diagnostics collector and the demo are
served on the account's workers.dev subdomain instead (see
[Diagnostics](/docs/server/diagnostics/index.md) and
[`tools/demo/README.md`](https://github.com/Bookclerk/bookclerk/blob/main/tools/demo/README.md)).

After each deploy, a step reads the startup time Cloudflare measured from
wrangler's output and writes it to the run summary, with a warning past
500 ms (Cloudflare's limit is 1000 ms). See
[Worker startup time](/docs/contributing/ci/index.md#worker-startup-time).

### The demo deploy

The demo image is the same one `images-publish.yml` publishes as
`ghcr.io/<owner>/bookclerk-demo`, so the deploy reuses it rather than
building it twice:

1. When the published image for the deployed commit
   (`bookclerk-demo:sha-<12 hex>`) exists, the deploy checks its build
   provenance with `gh attestation verify`. The attestation must name
   `images-publish.yml` in this repository, the deployed commit, and a
   GitHub-hosted runner.
2. A verified image is deployed by digest: `wrangler deploy` builds a
   one-line Dockerfile on it and pushes that to Cloudflare's container
   registry, which can't pull from GHCR itself. This takes a few minutes.
3. Without one (a push that changes the demo, a fork that publishes no
   images), or when verification fails, the deploy builds the demo image
   from the deployed commit with `packaging/demo/docker-bake.hcl`, the build
   `images-publish.yml` runs, and deploys it the same way. This takes about
   15 minutes.

It runs when the demo changes on `main`, when a release is published, after
every `Publish images` run on `main` (the nightly publish, which is how crate
and plugin changes reach the demo), and on demand.

- A run after a publish or a release first asks the live demo's
  `/_demo/health` which commit it runs. It stops when it is the same one
  and an earlier run deployed that commit and passed its smoke check. Each
  run is named `Deploy demo <commit>` after the commit it deploys, since a
  run started by a publish has `main`'s newest commit as its own.
- A run after a publish deploys the commit that publish built. It stops
  when the live demo already runs a newer commit, so the demo never goes
  back, and when the publish pushed no demo image. It fails when the
  registry can't be read, rather than skipping the deploy.
- The Worker reports its version before migrations and smoke checks, so a
  half-finished deploy is retried.
- A repository without the two Cloudflare secrets gets a notice instead of
  a failure.

#### The daily seed refresh

The demo image carries a pre-seeded library whose store records and Discover
shelves are cached for a day. A container that boots from an older template
fetches them again, so its first pages are slow. The run after each nightly
publish keeps the seed current without moving the demo to newer code:

1. When the live demo already runs the published commit (the run would
   otherwise stop), it checks whether a run in the last 12 hours baked a
   seed: a push (which builds its image), a run after a publish (which
   deploys the image that publish just built) or another refresh. A release
   run doesn't count, since it can deploy an image built days earlier.
2. If none did, it verifies that commit's published demo image as above,
   and stops when there is none. A refresh never builds the release
   binaries.
3. It re-bakes only the seed on top of that image with
   `packaging/demo/Dockerfile.refresh` and the commit's own
   `packaging/demo/bake-seed.sh`, then deploys the result with that commit's
   Worker, as any other deploy.

The refresh rides on the nightly publish rather than a schedule of its own,
because GitHub starts scheduled runs hours late: a separate schedule would
often fire before the nightly deploy and bake a seed that deploy replaces.
A manual run with the **refresh** input checked re-bakes the live commit's
seed at once, without the 12-hour check. Its run is named
`Refresh demo seed`, so it never counts as a deploy of a commit.

### API token permissions

The API token is scoped to the Workers by name, so it needs these for each
Worker the workflows deploy:

- **Workers, with the Editor role**, scoped to *Individual Workers* and
  naming the Worker (`bookclerk-site`, `bookclerk-diagnostics`,
  `bookclerk-demo`). This is the current Workers permission. The legacy
  *Workers Scripts: Edit* maps to Editor on all Workers, which is broader
  than the deploys need.
- **Zone → Workers Routes → Edit** (Write in the API reference) on the
  `<DOMAIN>` zone, for the custom domain.

`bookclerk-diagnostics` also needs:

- **Account → Workers KV Storage → Edit**, for the deploy that first
  creates its `TRIAGE_BUDGET` namespace (the binding has no id, so wrangler
  creates it). Deploying bindings needs only Editor on the Worker, so this
  can be removed once the namespace exists.
- **Account → D1 → Edit**. The first deploy creates the dashboard's
  `DASHBOARD_DB` database (no id in the binding), and every deploy applies
  its migrations with `wrangler d1 migrations apply`, so this one stays.

`bookclerk-demo` also needs:

- **Account → D1 → Edit**, for the same reasons (its `DB` database).
- **Account → Containers → Edit**, since every deploy pushes the image and
  updates the container application.
- **Account → Turnstile → Edit**, since every deploy finds or creates the
  `bookclerk-demo` widget, keeps its hostnames current and reads its keys
  (`tools/demo/scripts/worker-secrets.ts`).

### First deploy of a new Worker

Editor can't create a Worker; creating one takes the **Admin** role on the
Workers product. So the first deploy of a new Worker needs one of these:

1. Create the Worker in the Cloudflare dashboard first, under its exact
   name (an empty "Hello World" Worker is fine, because the first deploy
   replaces it). Then add it to the token's Individual Workers scope.
2. Or temporarily give the token the Workers Admin role, run the first
   deploy, then remove the role again and add the new Worker to the
   Individual Workers scope.

See
[Workers roles and permissions](https://developers.cloudflare.com/workers/authorization/workers/).

After that, the two base permissions are enough, plus the per-Worker
permissions that every deploy uses: D1 for `bookclerk-diagnostics` and
`bookclerk-demo`, and Containers and Turnstile for the demo.

> [!WARNING]
> Renaming a Worker in `wrangler.toml` creates a new Worker. That repeats
> this step and leaves the old one behind, so keep the names as they are.

## IndexNow

`indexnow.yml` tells search engines which pages of the website are new or
changed, through [IndexNow](https://www.indexnow.org). Bing, Yandex,
Seznam, Naver and Yep share what it receives, so they need not wait to
recrawl the site.

It runs after each `site-deploy` and `docs-publish` run, daily, and on
demand. A run after `docs-publish` first waits ten minutes, until the site's
edge cache shows the published docs.

1. It reads every URL that the live `/sitemap.xml` index lists, and gives
   each one a fingerprint: its `<lastmod>`, or a SHA-256 of the page as
   Markdown for pages without one.
2. It compares them with the snapshot that the previous run kept as the
   `indexnow-snapshot` artifact. URLs that are new, changed or no longer
   listed are submitted, each at most once per UTC day. A change made after
   that day's submission waits for the next day's run. A dated page
   submitted on the day it is dated is submitted once more on a later day,
   since its date cannot show a second change that day.
3. Only once IndexNow accepts the URLs does it save the new snapshot, so a
   failed run is retried in full.

With no snapshot (the first run, or 90 days without an accepted run) only
pages whose sitemap `<lastmod>` is today or one of the 7 days before are
submitted, and every listed URL becomes the snapshot the next run compares
against. Search engines learn the older and undated pages from the sitemaps.

> [!NOTE]
> The snapshot is an artifact rather than an Actions cache entry. CI's
> build caches fill the repository's cache quota and push small entries out
> within minutes, so a cached snapshot was often missing and every URL was
> submitted again.

- **Setup.** Set the repository or organization variable `INDEXNOW_KEY` to
  8 to 128 letters, digits or `-`, for example `uuidgen | tr -d -`. The key
  is public by design: `site-deploy.yml` serves it as
  `https://<DOMAIN>/<key>.txt`, and IndexNow fetches that file to check that
  a submission came from the site. Deploy the site once, then run the
  workflow by hand.
- **Forks.** Without the variable the job is skipped, and the site has no
  key file.
- **Skipped runs.** A run warns and submits nothing when the site cannot
  be reached, the key file is missing, a section sitemap cannot be read, or
  `/sitemap.xml` is the build's fallback (the docs dates could not be read).
  An undated page that cannot be fetched keeps its previous fingerprint,
  with a warning. A refusal from IndexNow fails the run.
- **Locally.** In `tools/site`, run
  `INDEXNOW_KEY=<key> INDEXNOW_DRY_RUN=1 node indexnow-run.ts in.json out.json`
  to list the URLs that would be submitted, with no key file needed.

> [!NOTE]
> Cloudflare's Crawler Hints also sends IndexNow signals, but only for
> responses that miss the zone's cache. The docs pages are rendered by the
> Worker and skip that cache, so Crawler Hints covers the static files only.

## Next steps

- [Continuous integration](/docs/contributing/ci/index.md): the checks these workflows build on.
- [Fuzzing](/docs/contributing/fuzzing/index.md): the crash triage that shares the App and the
  bundle keys.
- [Release packaging](/docs/contributing/packaging/index.md): the workerd pin bump the App
  proposes.
