fix(engine)+docs: env overrides config for targeted runs; add full REFERENCE.md

- Engine: same targeted-run fix as apiscp-kopia (env BACKUP_SITES/SYSTEM/PATHS/
  DATABASES now override the sourced config, so `run --site` scopes correctly
  instead of backing up every site).
- docs/REFERENCE.md: exhaustive reference (every command, verb, config key,
  installer, automation, both GUIs, security model, recovery), plus a README
  Documentation section.

README.md

@@ -48,6 +48,13 @@ restore tools, Layer 2 panel integration (module + two GUIs + hooks), install /
 uninstall, and full reference documentation are being built out to track the
 apiscp-kopia feature set. Not yet ready for production use.
 
+## Documentation
+
+- `docs/REFERENCE.md`: exhaustive reference for every command, verb,
+  configuration key, installer, and automation shipped in this repository.
+- `docs/DESIGN.md`: design rationale, and how Borg's native ACL/xattr
+  handling changes the approach taken relative to apiscp-kopia.
+
 ## License
 
 MIT. See `LICENSE`.

bin/borg-apiscp-backup

@@ -32,8 +32,21 @@ done
 
 # --- configuration -------------------------------------------------------------
 CONFIG_FILE="${APISCP_BORG_CONFIG:-/etc/apiscp-borg/config}"
+# Capture run-scoped overrides passed via the environment (e.g. by
+# `borg-apiscp-repo run --site X`, which sets these via systemd-run) BEFORE
+# sourcing the config, then re-apply them AFTER, so a targeted run is not
+# clobbered by the persistent config defaults. `${VAR-x}` (single dash)
+# distinguishes unset (use config) from set-but-empty (an explicit override).
+_env_sites="${BACKUP_SITES-__unset__}"
+_env_system="${BACKUP_SYSTEM-__unset__}"
+_env_paths="${BACKUP_PATHS-__unset__}"
+_env_db="${BACKUP_DATABASES-__unset__}"
 # shellcheck source=/dev/null
 [ -r "$CONFIG_FILE" ] && . "$CONFIG_FILE"
+[ "$_env_sites" != "__unset__" ]  && BACKUP_SITES="$_env_sites"
+[ "$_env_system" != "__unset__" ] && BACKUP_SYSTEM="$_env_system"
+[ "$_env_paths" != "__unset__" ]  && BACKUP_PATHS="$_env_paths"
+[ "$_env_db" != "__unset__" ]     && BACKUP_DATABASES="$_env_db"
 
 : "${APNSCP_ROOT:=/usr/local/apnscp}"
 : "${APNSCP_CMD:=/usr/local/apnscp/bin/cmd}"

docs/REFERENCE.md

@@ -0,0 +1,765 @@
+# apiscp-borg reference
+
+Exhaustive reference for every command, verb, configuration key, installer, and
+automation shipped in this repository. For the design rationale, and for how
+Borg's own metadata handling changes the approach taken by the sibling
+apiscp-kopia project, see `docs/DESIGN.md`.
+
+## 1. Overview
+
+apiscp-borg is a backup engine for ApisCP built on top of
+[BorgBackup](https://www.borgbackup.org/). Unlike kopia, Borg preserves POSIX
+ACLs and extended attributes natively on Linux, so no ACL/xattr sidecar
+mechanism is needed to protect the permission data ApisCP encodes in those
+mechanisms: `borg create` captures them and `borg extract` restores them on its
+own. The plugin's value is instead: running ApisCP's own per-site database
+export before archiving so the logical `.sql` dumps are current, archiving
+each site as its own named set of Borg archives so single-site restore and
+per-site retention are possible, and wiring retention, maintenance, integrity
+checking, scheduling, notifications, one-time key backup, and both a
+CLI and a native ApisCP panel integration around the `borg` binary. It ships
+as two independent layers: a standalone POSIX-sh engine (Layer 1, three
+binaries plus a systemd timer, with zero ApisCP coupling) and a native ApisCP
+integration (Layer 2, a module plus two GUI apps installed into upgrade-safe,
+git-ignored locations). Layer 2 provides two editions of the panel:
+
+- **Appliance-admin edition ("Borg Backups")**: full repository configuration,
+  backup selection, scheduling, retention (prune), maintenance (compact),
+  integrity checking, notifications, one-time key backup, and restore (site,
+  account, system, or database) for any site on the box. `PRIVILEGE_ADMIN`.
+- **Site-owner edition ("My Borg Backups")**: a scoped self-service panel for
+  a single signed-in site owner to restore their own files, databases, or
+  whole account, and to trigger a backup of their own site on demand. It never
+  accepts a site identifier from the caller; the site is always derived from
+  the authenticated session. `PRIVILEGE_SITE`.
+
+Borg's repository model is simpler than kopia's: there are no backend
+"types", no repository server, and no SFTP transport. A repository is just a
+location (`BORG_REPO`, a local path or an `ssh://` URL), a `BORG_PASSPHRASE`,
+a `BORG_ENCRYPTION` mode (`repokey-blake2` by default, so the encryption key
+lives inside the repository and only the passphrase is needed to recover it),
+and an optional `BORG_RSH` for SSH transport options. There are no tagged
+snapshots either: Borg has named archives, and this engine names them with a
+structured prefix (`siteN-shadow`, `siteN-info`, `siteN-db`, `_system`,
+`_custom-<slug>`, each suffixed with Borg's `{now}` timestamp placeholder) so
+per-site operations are glob-addressable. Restore in this release always uses
+the **newest** archive of a given prefix: there is no point-in-time archive
+selection and no browse/subpath picker in the panel (a current limitation,
+noted throughout this document and in the GUI itself).
+
+## 2. Installation
+
+### `install.sh` (Layer 1: the engine)
+
+Installs the standalone engine. Must be run as root. Idempotent.
+
+```sh
+sudo sh install.sh
+```
+
+Unlike apiscp-kopia's installer, this script does **not** install `borg`
+itself: it only checks whether `borg` is on `PATH` and prints a warning if it
+is not. Install `borgbackup` yourself first (`dnf install borgbackup` /
+`apt install borgbackup`).
+
+Flags:
+
+| Flag | Effect |
+|---|---|
+| (none) | Installs the library, binaries, and systemd units; seeds the config. |
+| `-h`, `--help` | Prints the header usage comment and exits 0. |
+
+What it installs, in order:
+
+| Item | Source | Destination | Mode |
+|---|---|---|---|
+| shared library | `lib/apiscp-borg-common.sh` | `/usr/local/lib/apiscp-borg/apiscp-borg-common.sh` | 0644 |
+| engine binary | `bin/borg-apiscp-backup` | `/usr/local/bin/borg-apiscp-backup` | 0755 |
+| repo binary | `bin/borg-apiscp-repo` | `/usr/local/bin/borg-apiscp-repo` | 0755 |
+| restore binary | `bin/borg-apiscp-restore` | `/usr/local/bin/borg-apiscp-restore` | 0755 |
+| systemd service | `systemd/borg-apiscp-backup.service` | `/etc/systemd/system/borg-apiscp-backup.service` | 0644 |
+| systemd timer | `systemd/borg-apiscp-backup.timer` | `/etc/systemd/system/borg-apiscp-backup.timer` | 0644 |
+| config seed | `etc/apiscp-borg.config.example` | `/etc/apiscp-borg/config` | 0600, only if absent (never overwrites an existing config) |
+
+`systemctl daemon-reload` is run after the units are installed. The script
+does **not** enable the timer, install `borg`, or create/connect the Borg
+repository; it prints the manual next steps (install `borgbackup`, set
+`BORG_REPO`/`BORG_PASSPHRASE`, initialise the repository, back up the
+passphrase/key off-machine, one test run, `systemctl enable --now
+borg-apiscp-backup.timer`).
+
+### `install-layer2.sh` (Layer 2: native ApisCP integration)
+
+Installs the panel module, both GUI apps, both menu links, the account
+lifecycle hooks, and a scoped sudoers file. Run as root, after `install.sh`.
+Idempotent. Respects `CP_ROOT` (default `/usr/local/apnscp`) and `PANEL_USER`
+(default `apnscp`).
+
+| Step | Source | Destination | Notes |
+|---|---|---|---|
+| Module | `src/modules/borg.php` | `$CP_ROOT/lib/modules/surrogates/borg.php` | git-ignored by ApisCP (`surrogates/*`), survives `upcp` |
+| Admin GUI app | `src/apps/borg/` (borg.php, application.yml, views/index.blade.php) | `$CP_ROOT/config/custom/apps/borg/` | whole tree ignored |
+| Site-owner GUI app | `src/apps/myborgbackups/` | `$CP_ROOT/config/custom/apps/myborgbackups/` | whole tree ignored |
+| Admin menu link | `src/templates/admin.php` | `$CP_ROOT/config/custom/templates/admin.php` | only written if the file does not already exist; if it exists and already references `/apps/borg`, left untouched; if it exists, does not reference `/apps/borg`, and contains a closing `?>`, a warning is printed and nothing is appended; otherwise the `create_link()` call is appended |
+| Site menu link | `src/templates/site.php` | `$CP_ROOT/config/custom/templates/site.php` | same append-not-clobber logic, keyed on `/apps/myborgbackups` |
+| Account hooks | `src/hooks/{addDomain,suspendDomain,deleteDomain}.sh` | `$CP_ROOT/config/custom/hooks/<event>.sh` | only installed if the destination does not exist, or exists but does not already contain the string `apiscp-borg` (an operator's own hook is left untouched) |
+| Sudoers | generated inline | `/etc/sudoers.d/apiscp-borg` | 0440 root:root, validated with `visudo -cf` before install; installation is skipped with a warning if validation fails |
+
+After installing the sudoers file, the script forces the engine config to be
+root-owned (`chown root:root` + `chmod 0600` on `/etc/apiscp-borg/config` if
+it exists), then restarts the panel (`systemctl restart apnscp`) so the new
+GUI apps and menu entries are picked up; the module itself needs no restart
+and is picked up on the next CLI/API call. Site owners see the "My Borg
+Backups" menu entry after their next login (the menu is session-cached).
+
+**Upgrade-safe locations.** Every Layer 2 destination is one ApisCP
+git-ignores, so a `git pull` / `upcp` upgrade never touches it:
+`lib/modules/surrogates/*`, and the entirety of `config/custom/` (via its own
+`.gitignore` of `*` plus `!.gitignore`).
+
+## 3. Uninstall
+
+`uninstall.sh`, run as root, idempotent.
+
+```sh
+sudo sh uninstall.sh            # default: keep config, staging, key backups
+sudo sh uninstall.sh --purge    # also delete config, staging dir, key backup dir
+```
+
+Always removed (both modes):
+
+- Layer 2: the module surrogate, both GUI app directories (`borg` and
+  `myborgbackups`), the account hooks (only the ones containing the string
+  `apiscp-borg`), the scoped sudoers file, and the `apiscp-borg` lines
+  stripped out of `admin.php`/`site.php` (the template file itself is deleted
+  only if nothing but `<?php` and whitespace remains after stripping). The
+  panel is restarted afterwards.
+- Layer 1: the timer is disabled and stopped, the service is stopped, both
+  systemd unit files and any `borg-apiscp-backup.timer.d` schedule drop-in are
+  removed, `systemctl daemon-reload` runs, the three binaries and the shared
+  library directory are removed, and the Prometheus textfile metric
+  (`apiscp-borg.prom`) is deleted from any of the well-known collector
+  directories.
+
+Kept by default (i.e. **not** removed unless `--purge`):
+
+- `/etc/apiscp-borg/config` (holds the repository passphrase)
+- `/var/lib/apiscp-borg` (the staging area; also where the deleted-accounts
+  audit log lives)
+- the key backup directory (`KEY_BACKUP_DIR`, read out of the config before
+  it would be deleted)
+
+With `--purge`, the config directory and staging directory are deleted; the
+key backup directory is deliberately **not** auto-deleted even under
+`--purge` (its path is only printed, with an explicit `rm -rf` suggestion),
+because it may be the only offline copy of the repository passphrase/key.
+
+**Guarantee, always true, in both modes:** the Borg repository and every
+archive in it are never touched by `uninstall.sh`, and the `borg` binary
+itself is never removed. Uninstalling apiscp-borg does not affect your
+ability to recover backups with plain `borg`, or to reinstall and pick up
+where you left off.
+
+## 4. Configuration reference
+
+Config lives at `/etc/apiscp-borg/config`, POSIX-sh `KEY=value` syntax (no
+spaces around `=`), sourced directly by the three binaries. Every key has a
+built-in default so the engine runs unconfigured on a stock ApisCP box (other
+than `BORG_REPO`, which is required before any real operation).
+`etc/apiscp-borg.config.example` documents the same defaults and is seeded
+(never overwritten) by `install.sh`.
+
+`CONFIG_ALLOWED` in `bin/borg-apiscp-repo` (`config-set`) is the authoritative
+whitelist of keys settable through the CLI/GUI (the module's
+`Borg_Module_Surrogate::ALLOWED_KEYS` mirrors it exactly). A `config-set` /
+`set_option` call for any other key is rejected. Several engine-internals keys
+(`VIRTBASE`, `STAGE`, `SYSTEM_PATHS`, `SYSTEM_GLOBS`, `APNSCP_CMD`,
+`APNSCP_ROOT`, `DB_EXPORT_CMD`, `LOCK_FILE`) are read from the config file if
+present but are **not** in `CONFIG_ALLOWED`; they can only be changed by
+editing the config file directly, not via `config-set`, `set_option`, or
+either GUI.
+
+### Repository
+
+| Key | Meaning | Default | Example |
+|---|---|---|---|
+| `BORG_REPO` | Repository location: a local path or an `ssh://` URL | (none, required) | `ssh://borg@backup.example.net:22/./apiscp-borg` |
+| `BORG_PASSPHRASE` | Repository passphrase, exported into the environment for every `borg` invocation | (none) | generated via `gen-passphrase` |
+| `BORG_ENCRYPTION` | Encryption mode used only when **creating** a new repository | `repokey-blake2` | `keyfile-blake2` |
+| `BORG_RSH` | Extra SSH options passed to `borg` via `BORG_RSH`, for `ssh://` repositories | `ssh -o BatchMode=yes` | `ssh -o BatchMode=yes -i /root/.ssh/borg_key` |
+| `BORG_COMPRESSION` | Compression algorithm passed to `borg create --compression` | `zstd` | `lz4`, `zstd,15`, `none` |
+| `KEY_BACKUP_DIR` | Where `backup-key` writes the one-time passphrase/key export | `/root/apiscp-borg-keys` | (none) |
+
+`repokey-blake2` (the default) stores the encryption key inside the
+repository itself, so only the passphrase is needed to recover it.
+`keyfile-blake2` stores the key under `~/.config/borg` on the machine that
+created the repository and must be backed up separately (`borg key export`,
+handled automatically by `backup-key`/`key-bundle`).
+
+### What to back up
+
+| Key | Meaning | Default | Example |
+|---|---|---|---|
+| `BACKUP_SITES` | `all`, `none`, or a comma-separated list of site ids | `all` | `site1,site3` |
+| `BACKUP_SYSTEM` | Back up the shared system paths as well as per-site data: `1` or `0` | `1` | `0` |
+| `BACKUP_DATABASES` | Dump every site's databases via ApisCP's native per-site export and archive them: `1` or `0` | `1` | `0` |
+| `BACKUP_PATHS` | Extra absolute paths to back up, comma or space separated | (empty) | `/srv/extra,/opt/thing` |
+| `SYSTEM_PATHS` | Shared, non-per-site paths, space separated (not settable via CLI/GUI) | `/etc /opt` | (as default) |
+| `SYSTEM_GLOBS` | Shell globs (space separated) expanded at run time; non-matching globs are skipped, not passed literally to `borg` (not settable via CLI/GUI) | `/var/log/mailer_table* /var/lib/mysql/mysql-grants* /var/lib/pgsql/*/backups /root/apnscp* /root/license*` | (as default) |
+| `CAPTURE_META_SIDECAR` | Whitelisted config key reserved for an optional ACL/xattr text sidecar capture, off by default; Borg already preserves ACLs/xattrs natively in every archive it creates, and no sidecar-capture logic is implemented in the current engine regardless of this setting | `0` | `1` |
+
+### Retention (`borg prune`)
+
+Applied per archive-name prefix (each site's `shadow`, `info`, and `db`
+archives separately, plus `_system-*` and `_custom-*`), both automatically
+after every backup run and on demand via the `prune` verb/button. Any unset
+or blank value means no limit for that bucket. There is no `KEEP_LATEST` and
+no `KEEP_HOURLY` bucket (unlike kopia).
+
+| Key | Meaning | Default | Example |
+|---|---|---|---|
+| `RETENTION_KEEP_WITHIN` | Keep every archive created within this duration (Borg's `--keep-within`, a number plus `H`/`d`/`w`/`m`/`y`) | (none / unlimited) | `7d` |
+| `RETENTION_KEEP_DAILY` | Keep 1 per day for this many days (`--keep-daily`) | (none / unlimited) | `30` |
+| `RETENTION_KEEP_WEEKLY` | Keep 1 per week for this many weeks (`--keep-weekly`) | (none / unlimited) | `52` (1 year) |
+| `RETENTION_KEEP_MONTHLY` | Keep 1 per month for this many months (`--keep-monthly`) | (none / unlimited) | `24` (2 years) |
+| `RETENTION_KEEP_ANNUAL` | Keep 1 per year for this many years (Borg's `--keep-yearly`) | (none / unlimited) | `5` |
+
+### Excludes
+
+| Key | Meaning | Default | Example |
+|---|---|---|---|
+| `BACKUP_EXCLUDES` | `borg create --exclude` glob patterns, comma-separated on disk (one per line in the GUI textarea) | (empty) | `*.log,cache/,node_modules/` |
+
+`*/.borg-restore` and `/.borg-restore` are **always** excluded (hardcoded in
+both `create_archive`/`_create_with_excludes`), so site-owner self-service
+restores staged under that path are never recaptured into a subsequent
+archive.
+
+### Schedule
+
+| Key | Meaning | Default | Example |
+|---|---|---|---|
+| `BACKUP_SCHEDULE` | Time to run backups: `HH:MM` for a daily time, or a full systemd `OnCalendar` expression | `03:30` | `Mon *-*-* 03:30:00` |
+
+### Notifications
+
+| Key | Meaning | Default | Example |
+|---|---|---|---|
+| `NOTIFY_EMAIL` | Recipient address; empty disables email notification entirely | (empty) | `ops@example.com` |
+| `NOTIFY_ON` | `failure` (default, only email on a failed/incomplete run), `always`, or `never` | `failure` | `always` |
+| `NOTIFY_FROM` | Override the `From:` address | `root@$(hostname)` | `backups@example.com` |
+
+### Engine internals
+
+| Key | Meaning | Default | Settable via CLI/GUI? |
+|---|---|---|---|
+| `VIRTBASE` | ApisCP virtual host base; sites are `$VIRTBASE/site<N>` | `/home/virtual` | yes |
+| `STAGE` | Reserved working area for the engine (not exercised by any active code path in this release) | `/var/lib/apiscp-borg` | yes |
+| `METRICS_DIR` | Prometheus node_exporter textfile collector directory; blank autodetects common locations, and metrics are skipped if none is found | (empty / autodetect) | yes |
+| `RUN_DB_EXPORT` | Run ApisCP's per-site database export (`backup_dbs.php`) before archiving: `1` or `0` | `1` | yes |
+| `DB_EXPORT_CMD` | Path to `backup_dbs.php` | `$APNSCP_ROOT/bin/scripts/backup_dbs.php` | no |
+| `APNSCP_CMD` | ApisCP CLI binary, used for native per-site database verbs (`list_databases`/`export`/`import`) | `/usr/local/apnscp/bin/cmd` | no |
+| `APNSCP_ROOT` | Path to the ApisCP installation | `/usr/local/apnscp` | no |
+| `LOCK_FILE` | Single-instance lock file guarding against overlapping backup runs (via `flock`, if available) | `/run/apiscp-borg.lock` | no |
+
+Two further variables are read only via the environment, not the config file
+whitelist: `APISCP_BORG_LIB` (override the shared-library search path) and
+`APISCP_BORG_CONFIG` (override the config file path itself, default
+`/etc/apiscp-borg/config`), plus `APISCP_BORG_LOG_TAG` (the `logger` tag used
+by `akp_log`, default `apiscp-borg`) and `APISCP_BORG_DELETED_LOG` (used only
+by the `deleteDomain` hook, default `/var/lib/apiscp-borg/deleted-accounts.log`).
+`BORG_RELOCATED_REPO_ACCESS_IS_OK` is exported by the engine (default `no`,
+overridable via the real environment) so an unattended run does not block on
+Borg's interactive "repository moved" prompt.
+
+## 5. CLI reference (Layer 1)
+
+All three binaries source `lib/apiscp-borg-common.sh` (search order:
+`$APISCP_BORG_LIB`, `../lib/` relative to the binary,
+`/usr/local/lib/apiscp-borg/`, `/usr/lib/apiscp-borg/`) then
+`$APISCP_BORG_CONFIG` (default `/etc/apiscp-borg/config`). Every subcommand
+below matches the code exactly.
+
+### `borg-apiscp-backup`
+
+Takes no subcommands; running it (with no arguments) performs one full backup
+run and exits 0 if every archive was created, 1 if any failed. Driven by the
+systemd timer/service, or launched detached by `borg-apiscp-repo run`.
+
+Sequence on every run:
+
+1. Acquire `$LOCK_FILE` via `flock` (non-blocking; a second concurrent run
+   aborts immediately) if `flock` is available.
+2. If `RUN_DB_EXPORT=1`, run `$DB_EXPORT_CMD` (ApisCP's `backup_dbs.php`) so
+   per-site `.sql` dumps are current before they are archived.
+3. For every `$VIRTBASE/site<N>` directory (skipping any not selected by
+   `BACKUP_SITES`):
+   - archive `shadow` (if present) as `site<N>-shadow-{now}`;
+   - archive `info` (if present) as `site<N>-info-{now}`;
+   - if `BACKUP_DATABASES=1`: enumerate the site's `mysql` and `pgsql`
+     databases via `$APNSCP_CMD -o json -d <domain> <engine>:list_databases`,
+     export each to a site-relative staging path
+     (`/tmp/.apiscp-borg-db/<engine>/<db>.sql`, resolved inside the site's
+     `fst` namespace by ApisCP), archive that staging tree from its real host
+     location (`<site>/fst/tmp/.apiscp-borg-db`) as `site<N>-db-{now}`, then
+     delete the staging tree from the site;
+   - apply per-archive-prefix retention (`borg prune`) for
+     `site<N>-shadow-*`, `site<N>-info-*`, and `site<N>-db-*` right away, so
+     one site's daily churn is pruned every run rather than waiting for a
+     separate step.
+4. If `BACKUP_SYSTEM=1`: expand `SYSTEM_PATHS` and matching `SYSTEM_GLOBS`
+   entries to those that actually exist, archive them together as
+   `_system-{now}`, then prune `_system-*`.
+5. If `BACKUP_PATHS` is non-empty: for each existing custom path, archive it
+   as `_custom-<slug>-{now}` (slug via `akp_slug`), then prune
+   `_custom-<slug>-*`.
+6. Write Prometheus textfile metrics (always, even on failure or interrupt,
+   via an `EXIT`/`INT`/`TERM` trap) and send an email notification per
+   `NOTIFY_EMAIL`/`NOTIFY_ON`/`NOTIFY_FROM`.
+
+Every archive is created with `borg create --compression
+"${BORG_COMPRESSION:-zstd}"`, plus `--exclude '*/.borg-restore' --exclude
+'/.borg-restore'` and one `--exclude` per `BACKUP_EXCLUDES` entry.
+
+Metrics written to `$METRICS_DIR/apiscp-borg.prom` (or autodetected):
+`apiscp_borg_run_success`, `apiscp_borg_archives_ok`,
+`apiscp_borg_archives_failed`, `apiscp_borg_duration_seconds`,
+`apiscp_borg_db_export_success` (1 ok / 0 failed / -1 skipped), and
+`apiscp_borg_last_run_timestamp_seconds`.
+
+### `borg-apiscp-repo`
+
+Manages the repository and everything that is not a data restore.
+
+| Subcommand | Syntax | Description |
+|---|---|---|
+| `status` | `borg-apiscp-repo status` | Print `repository: $BORG_REPO` followed by `borg info` (first 30 lines, indented). |
+| `init` | `borg-apiscp-repo init` | Create (initialise) the repository at `BORG_REPO` with `BORG_ENCRYPTION`. Requires `BORG_PASSPHRASE` already set. Idempotent: if `borg info` already succeeds against the repository, logs and returns 0 without re-initialising. |
+| `check-access` | `borg-apiscp-repo check-access` | Print `ok` if the repository is reachable and the passphrase is correct, else `error: cannot access ...`. Always exits 0 so a UI can read the message. |
+| `gen-passphrase` | `borg-apiscp-repo gen-passphrase` | Generate a strong random passphrase (`openssl rand -base64 30`, or `/dev/urandom` fallback), store it as `BORG_PASSPHRASE` via `config-set`, and print **only** the passphrase to stdout. |
+| `key-status` | `borg-apiscp-repo key-status` | Print `yes` if the one-time key/passphrase backup sentinel exists at `$KEY_BACKUP_DIR/.apiscp-borg-key-backed-up`, else `no`. |
+| `backup-key` | `borg-apiscp-repo backup-key [DEST] [--force]` | Export `repository.passphrase` (if `BORG_PASSPHRASE` is set), `repository.key` (via `borg key export`, which is a no-op warning for `repokey` repositories since the key already lives in the repository), and a `README.txt` to `DEST` (default `$KEY_BACKUP_DIR`), all 0600. Refuses to redo (logs and returns 0) unless `--force`. |
+| `key-bundle` | `borg-apiscp-repo key-bundle` | Print a self-contained recovery bundle (instructions, the plaintext passphrase, the repository location, and an exported key via `borg key export :: /dev/stdout`) to stdout. For the GUI's one-time post-creation display/download. |
+| `config-get` | `borg-apiscp-repo config-get` | Print every `KEY=VALUE` line from the config file with surrounding quotes stripped (comments and malformed lines skipped). |
+| `config-set` | `borg-apiscp-repo config-set KEY VALUE` | Set one config key, rejecting any key not in `CONFIG_ALLOWED` and any value containing a CR or LF. Value is stored single-quoted (shell-escaped) so the config, which is sourced by the engine, cannot be injected. Writes with mode 0600. |
+| `sites` | `borg-apiscp-repo sites` | List `site<N>\t<domain>` for every ApisCP site under `$VIRTBASE` (domain falls back to the site id if `siteinfo` cannot be read). |
+| `run` | `borg-apiscp-repo run [--site SITE \| --path PATH]` | No args (or `--all`): `systemctl start --no-block borg-apiscp-backup.service` (returns immediately). `--site SITE`: launch a detached one-shot run scoped to just that site (`BACKUP_SITES=SITE BACKUP_SYSTEM=0`); `SITE` must match `site[0-9]*` or be the literal `none`. `--path PATH`: launch a detached one-shot backing up only that one absolute, existing, readable path (`BACKUP_SITES=none BACKUP_SYSTEM=0 BACKUP_PATHS=PATH`). Detached runs use `systemd-run --collect` if available, else a backgrounded `env ... &`. |
+| `running` | `borg-apiscp-repo running` | Print `systemctl is-active borg-apiscp-backup.service` (e.g. `active`, `inactive`). |
+| `prune` | `borg-apiscp-repo prune` | Apply retention (`RETENTION_KEEP_*`, numeric/duration values only) across every archive prefix: each site's `shadow`/`info`/`db` archives separately, plus `_system-*` and `_custom-*`. If no retention key is set, logs and does nothing. |
+| `maintenance` | `borg-apiscp-repo maintenance` | Run `borg compact`, detached (`systemd-run --collect` if available, else backgrounded). |
+| `verify` | `borg-apiscp-repo verify [--data]` | Run `borg check` (repository and archive consistency), detached. With `--data`, adds `--verify-data` to also read file contents back (slow, thorough); without it, only metadata is checked. |
+| `set-schedule` | `borg-apiscp-repo set-schedule [SPEC]` | Set the backup timer's `OnCalendar`. `SPEC` is `HH:MM` (validated 00-23:00-59, becomes `*-*-* HH:MM:00`) or a full `OnCalendar` expression (validated against an allowed character set); falls back to `BACKUP_SCHEDULE` if `SPEC` is omitted. Writes `/etc/systemd/system/borg-apiscp-backup.timer.d/override.conf`, reloads systemd, restarts the timer. Prints the resulting `OnCalendar` value. |
+| `notify-test` | `borg-apiscp-repo notify-test` | Send a test email to `NOTIFY_EMAIL` to confirm mail delivery; fails if `NOTIFY_EMAIL` is unset. |
+| `list-databases` | `borg-apiscp-repo list-databases SITE` | Print `<engine>\t<db>` (engine `mysql` or `pgsql`) for every database ApisCP currently knows for `SITE`, via native per-site enumeration (a host-level dump cannot see per-site databases). `SITE` must match `site[0-9]*`. |
+| (no args / `-h`/`--help`/`help`) | | Print the full header usage comment (via `awk`) and exit 0. |
+| (anything else) | | Print `unknown command: <cmd>` to stderr and exit 2. |
+
+There is no `connect`, `create`, `types`, `repo-info`, `validate-path`,
+`browse`, or `apply-policy` subcommand: those either do not apply to Borg's
+simpler repository model, or (in the case of `repo-info` and `validate-path`)
+are computed directly in the Layer 2 module without a root round trip (see
+section 6).
+
+### `borg-apiscp-restore`
+
+Restores data from a Borg archive. Borg stores archived paths relative
+(leading `/` stripped) and preserves POSIX ACLs and extended attributes
+natively as part of every archive, so restore is a plain `borg extract` with
+`--strip-components` to rebase the archived path under a staging target: no
+separate metadata-replay step is needed (that is the kopia engine's job, not
+Borg's). Archive naming (from the engine): `siteN-shadow-{now}`,
+`siteN-info-{now}`, `siteN-db-{now}`, `_system-{now}`,
+`_custom-<slug>-{now}`. "Newest" means the most recent archive matching a
+given prefix (`borg list --glob-archives PREFIX-* --last 1`).
+
+| Subcommand | Syntax | Description |
+|---|---|---|
+| `list` | `borg-apiscp-restore list [PREFIX]` | List archives, optionally filtered to `PREFIX-*` (e.g. `site1`), else every archive in the repository. |
+| `list-json` | `borg-apiscp-restore list-json PREFIX` | Print raw `borg list --json --glob-archives PREFIX-*`. `PREFIX` is required. Intended for the module/GUI to parse. |
+| `site` | `borg-apiscp-restore site SITE TARGET [--subpath P ...]` | Restore `SITE`'s newest `shadow` and `info` archives into `TARGET/{shadow,info}`. With one or more `--subpath`, restores only those paths (relative to the site root, under `shadow/`) instead of the whole tree; each subpath is rejected if it starts with `/` or contains `..`. Fails if nothing was restored. |
+| `account` | `borg-apiscp-restore account SITE TARGET` | Restore a whole account: files (`shadow`+`info`, fails hard if none exist) plus the newest `SITE-db` archive as a tree into `TARGET/databases` (warns, does not fail, if no database archive exists). |
+| `system` | `borg-apiscp-restore system TARGET` | Restore the newest `_system` archive into `TARGET` via `borg extract --numeric-ids`. |
+| `restore-db` | `borg-apiscp-restore restore-db SITE ENGINE DB TARGET` | Non-destructive: extract `ENGINE/DB.sql` from the newest `SITE-db` archive into `TARGET/DB.sql`, print the restored path. `ENGINE` must be `mysql` or `pgsql`; `DB` must match `[A-Za-z0-9_-]+`. |
+| `import-db` | `borg-apiscp-restore import-db SITE ENGINE DB` | **Destructive.** Restores the newest dump of `SITE`'s `DB` to a temp directory (there is no `DUMPFILE` argument: the newest archived dump is always the one imported), stages it inside the site's own `fst` tree, and imports it into the site's live database via `$APNSCP_CMD -d <domain> <engine>:import`, then removes the staged copy. |
+| `owner-files` | `borg-apiscp-restore owner-files SITE [--subpath P ...]` | Site-owner variant of `site`: restores into the site's own filesystem under `/.borg-restore/files-<epoch>/`, chowned to the site owner, and prints the in-site relative path. |
+| `owner-account` | `borg-apiscp-restore owner-account SITE` | Site-owner whole-account restore into `/.borg-restore/account-<epoch>/` (files, plus databases under `databases/`), chowned to the owner. Prints the in-site relative path. |
+| `owner-restore-db` | `borg-apiscp-restore owner-restore-db SITE ENGINE DB` | Site-owner, non-destructive: restore the newest dump of the site's own database into `/.borg-restore/db-<epoch>/DB.sql`, chowned to the owner. Prints the in-site relative path. |
+| `owner-import-db` | `borg-apiscp-restore owner-import-db SITE ENGINE DB` | Site-owner, **destructive**: equivalent to `import-db` for the site's own database. |
+| (no args / `-h`/`--help`/`help`) | | Print the header usage comment and exit 0. |
+| (anything else) | | Print `unknown command: <cmd>` to stderr and exit 2. |
+
+Every "owner-*" subcommand takes `SITE` as an argument, but in every real
+invocation path that argument is supplied by the trusted Layer 2 module
+(`borg.php`), which derives it from `$this->getAuthContext()->site` (the
+authenticated session), never from a value a site user could control. The
+Layer 1 binaries additionally validate `SITE` matches `site[0-9]*`.
+
+**Current limitation, by design in this release:** restore always uses the
+newest archive of a matching prefix. There is no way, from the CLI or the
+GUI, to restore an older archive by id/time, and `owner-files`/`site` support
+subpath filtering only within the `shadow` tree, not a general browse picker.
+`import-db`/`owner-import-db` take no dump-file argument at all: they always
+restore-and-import the newest `SITE-db` archive's dump for the named
+database.
+
+## 6. Panel/API reference (Layer 2)
+
+`Borg_Module_Surrogate` (`src/modules/borg.php`, installed as
+`lib/modules/surrogates/borg.php`) exposes every verb below as `cpcmd
+borg:<verb>` and to the two GUI apps (`Page_Container::__call` forwards
+`$this->borg_<verb>(...)` calls to the module). Every verb runs one of the
+three Layer 1 binaries as root via a scoped `sudo -n`, with a positional
+printf-style format string so `Util_Process` escapes every argument (see
+section 9).
+
+### Admin verbs (`PRIVILEGE_ADMIN`)
+
+| Verb | Parameters | Description |
+|---|---|---|
+| `get_config` | | Return the current config as an associative array, with any key containing `PASSPHRASE` masked to `********`. |
+| `set_option` | `$key, $value` | Set one whitelisted config key (client-side newline check only; `config-set` is the authoritative validator). |
+| `status` | | Return `borg-apiscp-repo status` output. |
+| `check_access` | | Return `borg-apiscp-repo check-access` output (`ok` or an error message). |
+| `key_backed_up` | | `true` if `key-status` reports `yes`. |
+| `backup_key` | `$dest = null, $force = false` | Run `backup-key`, optionally to a custom destination, optionally forced. |
+| `key_bundle` | | Return the one-time recovery bundle text (secrets included by design). |
+| `run` | `$site = '', $path = ''` | Trigger a backup: `$path` takes precedence over `$site`; with neither, runs everything. `$site` is validated against `^site\d+$`. |
+| `is_running` | | `true` if the backup service is currently active. |
+| `sites` | | Return `siteN => domain` for every ApisCP site, sorted by key. |
+| `site_snapshots` | `$site` | Archives available for a site's files, newest first, as `[{id,time}, ...]`; the archive name is the id. Informational only: restore always uses the newest. |
+| `restore_site` | `$site, $target` | Restore a site's newest `shadow`+`info` archives into a staging directory. There is no point-in-time selection or subpath picker exposed here (unlike the Layer 1 `site` subcommand's `--subpath`); the whole site tree is always restored. |
+| `init_repo` | | Create (initialise) the configured repository. Idempotent; requires a passphrase already set. |
+| `gen_passphrase` | | Generate and store a strong repository passphrase; returns it once. |
+| `set_backup_selection` | `$sites, $system, $paths = []` | Set `BACKUP_SITES` (collapsing to `all` if the selection covers every known site, `none` if empty, otherwise a CSV), `BACKUP_SYSTEM`, and `BACKUP_PATHS` (each path re-validated with `validate_path`). |
+| `set_schedule` | `$when` | Validate `$when` as `HH:MM` or a safe `OnCalendar`-like expression, store it in `BACKUP_SCHEDULE`, then call `set-schedule`. |
+| `maintenance` | | Run `borg compact`, detached. |
+| `verify` | `$data = false` | Run `borg check`, detached; `$data` adds `--verify-data` (reads file contents back). |
+| `notify_test` | | Send a test notification email. |
+| `prune` | | Apply retention (`borg prune`) across every archive prefix right now, per the configured `RETENTION_KEEP_*` settings. Unlike kopia's global policy, there is no separate "apply policy" step: retention lives in config, and `prune` (also run automatically after each backup) is what enforces it. |
+| `repo_info` | | Return `['connected'=>bool, 'location'=>string]`, computed from `get_config()['BORG_REPO']` and a live `check-access` call; there is no repository "type" in the Borg model. |
+| `validate_path` | `$path` | `true` if the path is a plausible absolute path (non-empty, starts with `/`, no CR/LF). Unlike kopia's `validate-path`, there is no corresponding root-level CLI check: this is a best-effort structural check performed entirely in PHP, not authoritative (a path that does not actually exist when the engine runs is simply skipped with a warning in the log). |
+| `sites_with_snapshots` | | Site ids that have at least one `siteN-shadow` archive (for greying out empty sites in the restore picker). |
+| `site_databases` | `$site` | `[{engine, db}, ...]` currently known for a site (live, from ApisCP, not from an archive). |
+| `database_backups` | `$site` | `[{id, time}, ...]` (newest first) of dated `siteN-db` archives for a site. Informational: restore/import always use the newest. |
+| `restore_database` | `$site, $engine, $db, $target` | Non-destructive restore of one database dump to a staging path. |
+| `restore_account` | `$site, $target` | Restore a whole account (files + all databases) to a staging directory. |
+| `import_database` | `$site, $engine, $db` | **Destructive** import of the newest archived dump into the live database. No dump-file parameter: `import-db` always restores and imports the newest `siteN-db` archive itself. |
+| `restore_system` | `$target` | Restore the shared system archive to a staging directory. No subpath parameter is exposed (unlike kopia's `restore_system`). |
+
+### Site-owner verbs (`PRIVILEGE_SITE`)
+
+Every verb below resolves the caller's own site from the ApisCP auth context
+via a private `authSite()` helper (`$this->getAuthContext()->site`, validated
+against `^site\d+$`) and never accepts a site identifier as a parameter. Where
+a database is named, it is additionally checked to belong to that resolved
+site (`ownsDatabase()`) before use.
+
+| Verb | Parameters | Description |
+|---|---|---|
+| `my_site` | | `['site'=>siteN\|'', 'domain'=>string]` for the caller. |
+| `my_snapshots` | | `[{id,time}, ...]` archives available for the caller's own site. Informational only; restore always uses the newest. |
+| `restore_my_files` | | Restore the caller's own files (newest archive) into their own site filesystem (chowned to them), returning the in-site relative path. No point-in-time or subpath selection in this release. |
+| `my_databases` | | `[{engine,db}, ...]` for the caller's own site. |
+| `my_database_backups` | | `[{id,time}, ...]` dated database archives for the caller's own site. |
+| `restore_my_database` | `$engine, $db` | Non-destructive restore of one of the caller's own databases (newest archive) into their own filesystem; rejected if the database is not one the site currently owns. |
+| `import_my_database` | `$engine, $db, $confirm = false` | **Destructive**: import the newest archived dump of one of the caller's own databases back into their live database. Requires `$confirm === true`. |
+| `restore_my_account` | | Restore the caller's whole account (files + all databases, newest archives) into their own filesystem, chowned to them. |
+| `run_my_backup` | | Trigger a detached backup run scoped to only the caller's own site. |
+
+## 7. Automations
+
+### Scheduled backup (systemd timer)
+
+`systemd/borg-apiscp-backup.timer` runs `borg-apiscp-backup.service`
+(`Type=oneshot`, `Nice=10`, `IOSchedulingClass=idle`, `TimeoutStartSec=0` so a
+long backup is never killed as if hung).
+
+| Setting | Value | Purpose |
+|---|---|---|
+| `OnCalendar` | `*-*-* 03:30:00` | Nightly at 03:30, before jitter |
+| `RandomizedDelaySec` | `1800` | Up to 30 minutes of jitter so a fleet of servers does not hit shared storage simultaneously |
+| `Persistent` | `true` | Catch up with a missed run if the machine was off at the scheduled time |
+
+`BACKUP_SCHEDULE` overrides `OnCalendar` without editing the shipped unit:
+`borg-apiscp-repo set-schedule [SPEC]` (or `cpcmd borg:set_schedule`, or the
+GUI time picker) writes
+`/etc/systemd/system/borg-apiscp-backup.timer.d/override.conf` containing an
+empty `OnCalendar=` (clearing the base unit's value) followed by
+`OnCalendar=<computed value>`, then runs `systemctl daemon-reload` and
+`systemctl restart borg-apiscp-backup.timer`. `SPEC` may be `HH:MM` (becomes
+`*-*-* HH:MM:00`) or a full `OnCalendar` expression.
+
+### Retention (`borg prune`)
+
+There is no separate "apply policy" step as with kopia's global policy.
+Retention lives directly in the `RETENTION_KEEP_*` config keys, and `borg
+prune` is what enforces it, in two places:
+
+1. **Automatically, after every backup run**, per archive-name prefix (each
+   site's `shadow`, `info`, and `db` archives pruned separately, plus
+   `_system-*` and `_custom-<slug>-*`), right inside
+   `borg-apiscp-backup`, so retention never falls behind the schedule.
+2. **On demand**, via `borg-apiscp-repo prune` (`cpcmd borg:prune`, or the
+   GUI's "Prune now" button), which sweeps every site's `shadow`/`info`/`db`
+   prefixes plus `_system-*`/`_custom-*` in one pass, using whichever
+   `RETENTION_KEEP_*` values are currently set. If none are set, it logs and
+   does nothing.
+
+### Maintenance and verify (detached)
+
+`borg-apiscp-repo maintenance` runs `borg compact` (reclaims space freed by
+pruned archives); `borg-apiscp-repo verify [--data]` runs `borg check`
+(repository and archive consistency; `--data` also reads back every file's
+content, slower but thorough). Both are launched via `systemd-run --collect`
+when available (falling back to a backgrounded shell job) specifically so a
+web/API caller (the GUI, `cpcmd`) returns immediately rather than blocking on
+a potentially long-running repository operation.
+
+### Email notifications
+
+Configured via `NOTIFY_EMAIL` / `NOTIFY_ON` / `NOTIFY_FROM`. Sent via the
+local `sendmail` (tried at `/usr/sbin/sendmail`, `/usr/lib/sendmail`, or
+`sendmail` on `PATH`); if none is found, mail is skipped as best-effort. After
+every `borg-apiscp-backup` run (success, failure, or an interrupted/crashed
+run, all handled by an `EXIT`/`INT`/`TERM` trap), `notify()` sends a plain
+text summary (host, result, finish time, archives ok/failed, database export
+result, duration, repository location) if `NOTIFY_ON` calls for it: never
+(`never`), always (`always`), or only when the run did not fully succeed
+(`failure`, the default; an interrupted run that never reached its normal end
+also counts as a failure). `borg-apiscp-repo notify-test` (`cpcmd
+borg:notify_test`, GUI "Send test email") sends a one-off confirmation
+message independent of an actual backup run.
+
+### Account-lifecycle hooks
+
+ApisCP runs `config/custom/hooks/<event>.sh` with the site id (`siteN`) as
+`$1` after the named account event. All three hooks below always exit 0
+(hooks cannot interrupt or fail the underlying ApisCP operation) and run
+strictly **after** the event has already completed, not before or during it.
+
+| Hook | Source | Timing | Action |
+|---|---|---|---|
+| `addDomain.sh` | `src/hooks/addDomain.sh` | After a domain/account is created | Kicks off a detached backup scoped to just the new site (`borg-apiscp-repo run --site <site>`), so a brand-new account is protected immediately rather than waiting for the next scheduled run. |
+| `suspendDomain.sh` | `src/hooks/suspendDomain.sh` | After a domain/account is suspended | Takes one final detached backup of the site (same `run --site` call) to capture its state at the moment of suspension, since a suspended account's files still exist on disk. |
+| `deleteDomain.sh` | `src/hooks/deleteDomain.sh` | After a domain/account is deleted | Does **not** purge the site's archives. Only appends a `timestamp\tsite` line to an audit log (`$APISCP_BORG_DELETED_LOG`, default `/var/lib/apiscp-borg/deleted-accounts.log`) and logs to syslog. Existing archives are retained under the repository's normal retention policy; any purge is a deliberate, manual (or retention-window) decision, never automatic. |
+
+All three are installed only if the destination hook file does not already
+exist, or exists but does not already contain the string `apiscp-borg` (an
+operator's own custom hook is never overwritten).
+
+## 8. Both GUIs
+
+### Appliance-admin panel: "Borg Backups" (`src/apps/borg/`)
+
+Registered at `/apps/borg` via `config/custom/templates/admin.php`, under the
+admin menu's "System" category (internal id `services`). Every action maps
+to one `PRIVILEGE_ADMIN` verb from section 6. Sections, top to bottom:
+
+- **Repository**: a compact connected/not-connected status line with a "Show
+  details" toggle (raw `borg info`). When a repository is already connected,
+  the connection fields are prepopulated and disabled behind a "Reconfigure
+  repository" checkbox that must be ticked to unlock them. There is only one
+  repository shape (a path or an `ssh://` URL): fields are the repository
+  location, passphrase, an optional "generate a strong passphrase" checkbox,
+  an encryption-mode selector (`repokey-blake2` / `keyfile-blake2`), and SSH
+  options (`BORG_RSH`); there is no backend "type" selector, no server mode,
+  and no SFTP key upload, unlike the kopia panel. "Save settings" persists;
+  "Check access" confirms the repository is reachable with the current
+  passphrase. Creating a new repository requires ticking an explicit
+  acknowledgement ("I understand that if the repository passphrase/key is
+  lost, the backups are permanently irrecoverable") before "Initialise
+  repository" is enabled; immediately after creation, a one-time
+  red-bordered panel displays the full recovery bundle in a read-only
+  textarea plus a "Download recovery key" button (client-side `Blob`
+  download, no extra request), and is **not** shown again on reload.
+- **What to back up**: a two-box Available/Backing-up multi-select over
+  sites, "System information", and any configured custom paths, with arrow
+  buttons to move entries between boxes and a "Save selection" button, plus
+  a checkbox for whether site databases are included. A separate text field
+  plus "Add custom path" validates and appends an arbitrary absolute path
+  (via `validate_path`) to the backing-up box.
+- **Retention and excludes**: a "Within" duration field (e.g. `7d`, `4w`,
+  `6m`, `1y`) plus four archive-count fields (Daily/Weekly/Monthly/Annual;
+  blank = no limit), and a textarea of exclude globs, one per line (stored
+  comma-separated on disk); a note that `/.borg-restore` is always excluded.
+  "Save retention and excludes" persists the config; "Prune now" runs `borg
+  prune` immediately using whatever is currently saved.
+- **Run**: a status badge (idle / a backup is currently running, which
+  disables the button) and a mode selector (everything / one site / a custom
+  path) with "Run now" (always detached).
+- **Schedule and maintenance**: an HTML time input bound to `BACKUP_SCHEDULE`
+  with "Save schedule" (a systemd `OnCalendar` expression can still be set
+  from the CLI), plus "Run maintenance now" (`borg compact`) and a "verify
+  file contents" checkbox with "Verify now" (`borg check`, both detached).
+- **Notifications**: recipient email, an on-failure/always/never selector,
+  a sender override, "Save", and "Send test email".
+- **Repository key backup**: shows whether the one-time key/passphrase
+  backup sentinel exists, a destination field (defaults to
+  `KEY_BACKUP_DIR`), "Back up key now", and a "force (redo)" checkbox.
+- **Restore**: a single workflow with a "What would you like to restore?"
+  type selector (a site's files / an entire account / a single database /
+  system information) that swaps in the matching sub-panel via JavaScript,
+  rather than four separate pages. A note states plainly that Borg does not
+  yet support point-in-time archive selection or a subpath picker in this
+  release: every restore uses the newest matching archive.
+  - *Site files*: pick a site (sites with no archives are disabled in the
+    dropdown), "Show backups" lists archive dates for information (newest
+    first, up to 6 shown); a target field and "Restore newest archive".
+  - *Entire account*: pick a site (same archive-availability graying), a
+    target directory, "Restore entire account" (files plus all databases
+    under a `databases/` subdirectory).
+  - *System information*: a target directory and "Restore system"
+    (non-destructive).
+  - *Databases*: pick a site, "List databases" shows each database with its
+    engine, the dated backup history (up to 6 most recent shown, for
+    information), and per-row "Restore dump to staging" (safe) versus
+    "Import newest backup into live database" (destructive, requires a
+    ticked confirmation checkbox **and** a JavaScript `confirm()` dialog).
+
+A full-page "running" overlay (spinner + the clicked button's label) appears
+on every form submit and blocks further clicks until the page reloads, since
+Borg operations on a large repository can take a while; the page also
+restores scroll position across the postback so a save action does not jump
+the operator back to the top.
+
+### Site-owner panel: "My Borg Backups" (`src/apps/myborgbackups/`)
+
+Registered at `/apps/myborgbackups` via `config/custom/templates/site.php`,
+under the site panel's "Account" category. Every action maps to one
+`PRIVILEGE_SITE` verb from section 6, and the page never sends a site
+identifier of its own. If the caller's site cannot be resolved from the auth
+context, the whole page shows only a warning to contact the administrator.
+Sections:
+
+- **Restore files**: "Show my backups" lists the caller's own archive dates
+  (informational, up to 6 shown); "Restore newest backup into my files (with
+  permissions)" restores the newest archive into the owner's own site
+  filesystem. A note states there is no point-in-time selection or file
+  picker in this release.
+- **Restore my entire account**: one button, restores all files plus all
+  databases (newest archives) into `/.borg-restore/account-<date>/`
+  (databases under a `databases/` subdirectory).
+- **Databases**: "List my databases" shows engine/name plus dated backup
+  history (informational); per-row "Restore dump to my files" (safe) and
+  "Import newest backup into live database" (destructive, ticked
+  confirmation plus a JS `confirm()`).
+- **Back up now**: triggers an on-demand backup of just the caller's own
+  site, in addition to the scheduled runs.
+
+Every restore result banner states explicitly that the output landed under
+`/.borg-restore/` in the owner's own file space and can be retrieved with the
+File Manager or SFTP. The same "running" overlay and scroll-position
+preservation as the admin app are used here too.
+
+**Common restore landing convention**: every site-owner restore (files,
+database dump, or whole account) is written inside that site's own `fst`
+tree under a timestamped subdirectory of `/.borg-restore/`, and the entire
+`/.borg-restore` directory is then recursively chowned to the site's admin
+user (`akp_owner_chown`, preferring `chown --reference` against the owner's
+home directory, falling back to the numeric uid:gid parsed from the site's
+own `/etc/passwd`) so the owner can browse, download, and delete it through
+their own File Manager or SFTP account.
+
+## 9. Security model
+
+- The Layer 2 module runs inside the ApisCP frontend as the unprivileged
+  `apnscp` user, which cannot read the root-owned Borg repository config or
+  list `/home/virtual`. Every verb therefore shells out to one of the three
+  Layer 1 binaries via `Util_Process::exec('sudo -n <bin> %s %s ...', ...)`
+  (`runTool()`/`runToolRaw()` in `borg.php`), where the binary path is a
+  fixed class constant and every argument is passed through
+  `Util_Process`'s own printf-style escaping, never string-concatenated into
+  a shell command.
+- `install-layer2.sh` installs a scoped `/etc/sudoers.d/apiscp-borg` granting
+  exactly `apnscp ALL=(root) NOPASSWD: <the three binary paths>` (validated
+  with `visudo -cf` before being installed) - nothing else is permitted to
+  run as root. The three binaries themselves therefore are the privilege
+  boundary: they validate every input (site id pattern, database engine
+  must be `mysql`/`pgsql`, database name character set, target must be
+  absolute, subpaths cannot contain `..` or a leading `/`) before acting.
+- `/etc/apiscp-borg/config` (holds the repository passphrase) is kept
+  root-owned (0600), never read or written directly by the panel user; all
+  reads go through `config-get` and all writes through `config-set`, both of
+  which run as root via the same sudoers grant.
+- `config-set` (and the module's `set_option`) enforce a fixed whitelist of
+  settable keys (`CONFIG_ALLOWED` in `bin/borg-apiscp-repo`, mirrored as
+  `Borg_Module_Surrogate::ALLOWED_KEYS`) and reject any value containing a CR
+  or LF, so a caller cannot smuggle extra config lines or inject shell
+  metacharacters into a file that is later `.`-sourced by the engine; the
+  stored value is additionally single-quote-escaped.
+- Site-owner (`PRIVILEGE_SITE`) verbs never accept a site identifier as a
+  parameter. Every one of them calls a private `authSite()` helper that reads
+  `$this->getAuthContext()->site` (set by ApisCP's own auth framework to the
+  authenticated session's site) and validates it against `^site\d+$`; if it
+  cannot be resolved, the verb returns an error or an empty result rather
+  than falling back to any caller-supplied value. Where a database name is
+  also given (e.g. `restore_my_database`, `import_my_database`), it is
+  cross-checked against that resolved site's current database list
+  (`ownsDatabase()`) before use, so a site owner cannot reach another site's
+  data even by guessing a database name. The corresponding Layer 1
+  subcommands (`owner-files`, `owner-restore-db`, `owner-import-db`,
+  `owner-account`) repeat the site-pattern and path-traversal checks
+  independently, so the guarantee does not rest on the module alone.
+- Per-site database isolation: ApisCP's own `<engine>:export`/`<engine>:import`
+  verbs resolve their file argument **inside the target site's own filesystem
+  namespace** (a chroot-like view via `file_make_path`), not the host
+  filesystem, and a host-level dump/restore tool cannot see per-site
+  databases at all. The engine and the CLI tools therefore always go through
+  `$APNSCP_CMD -d <domain> <engine>:{list_databases,export,import}` in the
+  site's own domain context: a dump is written to a site-relative path
+  (e.g. `/tmp/.apiscp-borg-db/mysql/<db>.sql`) which the engine then reads
+  back from its real host-side location under `<site>/fst/...` for
+  archiving, and an import stages the dump under the site's own `fst` before
+  calling the site-context import. This is what makes per-site database
+  backup and restore correct in a multi-tenant ApisCP host where database
+  names can collide or be invisible outside their own site.
+- A Borg repository is locked to one operation at a time (Borg's own
+  repository lock, in addition to the engine's local `$LOCK_FILE`), so
+  concurrent site-owner actions against the same repository queue rather
+  than run in parallel; there is no risk of two writers corrupting the
+  repository.
+
+## 10. Restore and recovery
+
+A Borg repository is encrypted; if the passphrase (and, for keyfile-mode
+repositories, the exported key) are both lost, the data is permanently
+unrecoverable no matter how many copies of the encrypted archives exist.
+`borg-apiscp-repo backup-key` (see section 5) exports everything needed to
+recover onto a fresh machine, and refuses to silently redo that export more
+than once (use `--force` to intentionally redo it after a credential
+rotation).
+
+To recover the repository itself on a brand-new machine, using either the
+persistent key backup directory or the one-time `key-bundle` output:
+
+```sh
+# 1. install borgbackup (your package manager, e.g. dnf/apt install borgbackup)
+
+# 2. obtain the repository passphrase (and, for keyfile-mode repositories,
+#    the exported key), from either:
+#      - the key backup directory (repository.passphrase + repository.key), or
+#      - the one-time key-bundle text saved at repository-creation time
+
+# 3. export the passphrase and the repository location
+export BORG_PASSPHRASE="$(cat repository.passphrase)"
+export BORG_REPO="<the repository location, e.g. /mnt/backups/apiscp-borg or ssh://...>"
+
+# 4. only for keyfile-mode repositories (repokey-blake2 keeps the key IN the
+#    repository, so this step is unnecessary for the default encryption mode):
+borg key import :: repository.key
+
+# 5. confirm the archives are visible
+borg list
+
+# 6. extract whatever is needed, e.g. the newest siteN-shadow archive
+borg extract ::<archive-name>
+```
+
+From there, install apiscp-borg itself (`install.sh`, then point
+`BORG_REPO`/`BORG_PASSPHRASE` at the reconnected repository) to regain the
+per-site restore tooling (`borg-apiscp-restore`), or continue with plain
+`borg list` / `borg extract` if only a bare `borg` binary is available: no
+manual ACL/xattr replay step is needed, because Borg already restored that
+metadata as part of `borg extract`. The repository and its archives are
+never deleted by `uninstall.sh` (with or without `--purge`), so reinstalling
+this project at any point picks up exactly where the repository left off.