apiscp-borg/docs/REFERENCE.md

apiscp-borg reference

Exhaustive reference for every command, verb, configuration key, installer, and automation shipped in this repository. For the design rationale, and for how Borg's own metadata handling changes the approach taken by the sibling apiscp-kopia project, see docs/DESIGN.md.

1. Overview

apiscp-borg is a backup engine for ApisCP built on top of BorgBackup. Unlike kopia, Borg preserves POSIX ACLs and extended attributes natively on Linux, so no ACL/xattr sidecar mechanism is needed to protect the permission data ApisCP encodes in those mechanisms: borg create captures them and borg extract restores them on its own. The plugin's value is instead: running ApisCP's own per-site database export before archiving so the logical .sql dumps are current, archiving each site as its own named set of Borg archives so single-site restore and per-site retention are possible, and wiring retention, maintenance, integrity checking, scheduling, notifications, one-time key backup, and both a CLI and a native ApisCP panel integration around the borg binary. It ships as two independent layers: a standalone POSIX-sh engine (Layer 1, three binaries plus a systemd timer, with zero ApisCP coupling) and a native ApisCP integration (Layer 2, a module plus two GUI apps installed into upgrade-safe, git-ignored locations). Layer 2 provides two editions of the panel:

• Appliance-admin edition ("Borg Backups"): full repository configuration, backup selection, scheduling, retention (prune), maintenance (compact), integrity checking, notifications, one-time key backup, and restore (site, account, system, or database) for any site on the box. PRIVILEGE_ADMIN.
• Site-owner edition ("My Borg Backups"): a scoped self-service panel for a single signed-in site owner to restore their own files, databases, or whole account, and to trigger a backup of their own site on demand. It never accepts a site identifier from the caller; the site is always derived from the authenticated session. PRIVILEGE_SITE.

Borg's repository model is simpler than kopia's: there are no backend "types", no repository server, and no SFTP transport. A repository is just a location (BORG_REPO, a local path or an ssh:// URL), a BORG_PASSPHRASE, a BORG_ENCRYPTION mode (repokey-blake2 by default, so the encryption key lives inside the repository and only the passphrase is needed to recover it), and an optional BORG_RSH for SSH transport options. There are no tagged snapshots either: Borg has named archives, and this engine names them with a structured prefix (siteN-shadow, siteN-info, siteN-db, _system, _custom-<slug>, each suffixed with Borg's {now} timestamp placeholder) so per-site operations are glob-addressable. Restore in this release always uses the newest archive of a given prefix: there is no point-in-time archive selection and no browse/subpath picker in the panel (a current limitation, noted throughout this document and in the GUI itself).

2. Installation

install.sh (Layer 1: the engine)

Installs the standalone engine. Must be run as root. Idempotent.

sudo sh install.sh

Unlike apiscp-kopia's installer, this script does not install borg itself: it only checks whether borg is on PATH and prints a warning if it is not. Install borgbackup yourself first (dnf install borgbackup / apt install borgbackup).

Flags:

Flag	Effect
(none)	Installs the library, binaries, and systemd units; seeds the config.
-h, --help	Prints the header usage comment and exits 0.

What it installs, in order:

Item	Source	Destination	Mode
shared library	lib/apiscp-borg-common.sh	/usr/local/lib/apiscp-borg/apiscp-borg-common.sh	0644
engine binary	bin/borg-apiscp-backup	/usr/local/bin/borg-apiscp-backup	0755
repo binary	bin/borg-apiscp-repo	/usr/local/bin/borg-apiscp-repo	0755
restore binary	bin/borg-apiscp-restore	/usr/local/bin/borg-apiscp-restore	0755
systemd service	systemd/borg-apiscp-backup.service	/etc/systemd/system/borg-apiscp-backup.service	0644
systemd timer	systemd/borg-apiscp-backup.timer	/etc/systemd/system/borg-apiscp-backup.timer	0644
config seed	etc/apiscp-borg.config.example	/etc/apiscp-borg/config	0600, only if absent (never overwrites an existing config)

systemctl daemon-reload is run after the units are installed. The script does not enable the timer, install borg, or create/connect the Borg repository; it prints the manual next steps (install borgbackup, set BORG_REPO/BORG_PASSPHRASE, initialise the repository, back up the passphrase/key off-machine, one test run, systemctl enable --now borg-apiscp-backup.timer).

install-layer2.sh (Layer 2: native ApisCP integration)

Installs the panel module, both GUI apps, both menu links, the account lifecycle hooks, and a scoped sudoers file. Run as root, after install.sh. Idempotent. Respects CP_ROOT (default /usr/local/apnscp) and PANEL_USER (default apnscp).

Step	Source	Destination	Notes
Module	src/modules/borg.php	$CP_ROOT/lib/modules/surrogates/borg.php	git-ignored by ApisCP (surrogates/*), survives upcp
Admin GUI app	src/apps/borg/ (borg.php, application.yml, views/index.blade.php)	$CP_ROOT/config/custom/apps/borg/	whole tree ignored
Site-owner GUI app	src/apps/myborgbackups/	$CP_ROOT/config/custom/apps/myborgbackups/	whole tree ignored
Admin menu link	src/templates/admin.php	$CP_ROOT/config/custom/templates/admin.php	only written if the file does not already exist; if it exists and already references /apps/borg, left untouched; if it exists, does not reference /apps/borg, and contains a closing ?>, a warning is printed and nothing is appended; otherwise the create_link() call is appended
Site menu link	src/templates/site.php	$CP_ROOT/config/custom/templates/site.php	same append-not-clobber logic, keyed on /apps/myborgbackups
Account hooks	src/hooks/{addDomain,suspendDomain,deleteDomain}.sh	$CP_ROOT/config/custom/hooks/<event>.sh	only installed if the destination does not exist, or exists but does not already contain the string apiscp-borg (an operator's own hook is left untouched)
Sudoers	generated inline	/etc/sudoers.d/apiscp-borg	0440 root:root, validated with visudo -cf before install; installation is skipped with a warning if validation fails

After installing the sudoers file, the script forces the engine config to be root-owned (chown root:root + chmod 0600 on /etc/apiscp-borg/config if it exists), then restarts the panel (systemctl restart apnscp) so the new GUI apps and menu entries are picked up; the module itself needs no restart and is picked up on the next CLI/API call. Site owners see the "My Borg Backups" menu entry after their next login (the menu is session-cached).

Upgrade-safe locations. Every Layer 2 destination is one ApisCP git-ignores, so a git pull / upcp upgrade never touches it: lib/modules/surrogates/*, and the entirety of config/custom/ (via its own .gitignore of * plus !.gitignore).

3. Uninstall

uninstall.sh, run as root, idempotent.

sudo sh uninstall.sh            # default: keep config, staging, key backups
sudo sh uninstall.sh --purge    # also delete config, staging dir, key backup dir

Always removed (both modes):

• Layer 2: the module surrogate, both GUI app directories (borg and myborgbackups), the account hooks (only the ones containing the string apiscp-borg), the scoped sudoers file, and the apiscp-borg lines stripped out of admin.php/site.php (the template file itself is deleted only if nothing but <?php and whitespace remains after stripping). The panel is restarted afterwards.
• Layer 1: the timer is disabled and stopped, the service is stopped, both systemd unit files and any borg-apiscp-backup.timer.d schedule drop-in are removed, systemctl daemon-reload runs, the three binaries and the shared library directory are removed, and the Prometheus textfile metric (apiscp-borg.prom) is deleted from any of the well-known collector directories.

Kept by default (i.e. not removed unless --purge):

• /etc/apiscp-borg/config (holds the repository passphrase)
• /var/lib/apiscp-borg (the staging area; also where the deleted-accounts audit log lives)
• the key backup directory (KEY_BACKUP_DIR, read out of the config before it would be deleted)

With --purge, the config directory and staging directory are deleted; the key backup directory is deliberately not auto-deleted even under --purge (its path is only printed, with an explicit rm -rf suggestion), because it may be the only offline copy of the repository passphrase/key.

Guarantee, always true, in both modes: the Borg repository and every archive in it are never touched by uninstall.sh, and the borg binary itself is never removed. Uninstalling apiscp-borg does not affect your ability to recover backups with plain borg, or to reinstall and pick up where you left off.

4. Configuration reference

Config lives at /etc/apiscp-borg/config, POSIX-sh KEY=value syntax (no spaces around =), sourced directly by the three binaries. Every key has a built-in default so the engine runs unconfigured on a stock ApisCP box (other than BORG_REPO, which is required before any real operation). etc/apiscp-borg.config.example documents the same defaults and is seeded (never overwritten) by install.sh.

CONFIG_ALLOWED in bin/borg-apiscp-repo (config-set) is the authoritative whitelist of keys settable through the CLI/GUI (the module's Borg_Module_Surrogate::ALLOWED_KEYS mirrors it exactly). A config-set / set_option call for any other key is rejected. Several engine-internals keys (VIRTBASE, STAGE, SYSTEM_PATHS, SYSTEM_GLOBS, APNSCP_CMD, APNSCP_ROOT, DB_EXPORT_CMD, LOCK_FILE) are read from the config file if present but are not in CONFIG_ALLOWED; they can only be changed by editing the config file directly, not via config-set, set_option, or either GUI.

Repository

Key	Meaning	Default	Example
BORG_REPO	Repository location: a local path or an ssh:// URL	(none, required)	ssh://borg@backup.example.net:22/./apiscp-borg
BORG_PASSPHRASE	Repository passphrase, exported into the environment for every borg invocation	(none)	generated via gen-passphrase
BORG_ENCRYPTION	Encryption mode used only when creating a new repository	repokey-blake2	keyfile-blake2
BORG_RSH	Extra SSH options passed to borg via BORG_RSH, for ssh:// repositories	ssh -o BatchMode=yes	ssh -o BatchMode=yes -i /root/.ssh/borg_key
BORG_COMPRESSION	Compression algorithm passed to borg create --compression	zstd	lz4, zstd,15, none
KEY_BACKUP_DIR	Where backup-key writes the one-time passphrase/key export	/root/apiscp-borg-keys	(none)

repokey-blake2 (the default) stores the encryption key inside the repository itself, so only the passphrase is needed to recover it. keyfile-blake2 stores the key under ~/.config/borg on the machine that created the repository and must be backed up separately (borg key export, handled automatically by backup-key/key-bundle).

What to back up

Key	Meaning	Default	Example
BACKUP_SITES	all, none, or a comma-separated list of site ids	all	site1,site3
BACKUP_SYSTEM	Back up the shared system paths as well as per-site data: 1 or 0	1	0
BACKUP_DATABASES	Dump every site's databases via ApisCP's native per-site export and archive them: 1 or 0	1	0
BACKUP_PATHS	Extra absolute paths to back up, comma or space separated	(empty)	/srv/extra,/opt/thing
SYSTEM_PATHS	Shared, non-per-site paths, space separated (not settable via CLI/GUI)	/etc /opt	(as default)
SYSTEM_GLOBS	Shell globs (space separated) expanded at run time; non-matching globs are skipped, not passed literally to borg (not settable via CLI/GUI)	/var/log/mailer_table* /var/lib/mysql/mysql-grants* /var/lib/pgsql/*/backups /root/apnscp* /root/license*	(as default)
CAPTURE_META_SIDECAR	Whitelisted config key reserved for an optional ACL/xattr text sidecar capture, off by default; Borg already preserves ACLs/xattrs natively in every archive it creates, and no sidecar-capture logic is implemented in the current engine regardless of this setting	0	1

Retention (borg prune)

Applied per archive-name prefix (each site's shadow, info, and db archives separately, plus _system-* and _custom-*), both automatically after every backup run and on demand via the prune verb/button. Any unset or blank value means no limit for that bucket. There is no KEEP_LATEST and no KEEP_HOURLY bucket (unlike kopia).

Key	Meaning	Default	Example
RETENTION_KEEP_WITHIN	Keep every archive created within this duration (Borg's --keep-within, a number plus H/d/w/m/y)	(none / unlimited)	7d
RETENTION_KEEP_DAILY	Keep 1 per day for this many days (--keep-daily)	(none / unlimited)	30
RETENTION_KEEP_WEEKLY	Keep 1 per week for this many weeks (--keep-weekly)	(none / unlimited)	52 (1 year)
RETENTION_KEEP_MONTHLY	Keep 1 per month for this many months (--keep-monthly)	(none / unlimited)	24 (2 years)
RETENTION_KEEP_ANNUAL	Keep 1 per year for this many years (Borg's --keep-yearly)	(none / unlimited)	5

Excludes

Key	Meaning	Default	Example
BACKUP_EXCLUDES	borg create --exclude glob patterns, comma-separated on disk (one per line in the GUI textarea)	(empty)	*.log,cache/,node_modules/

*/.borg-restore and /.borg-restore are always excluded (hardcoded in both create_archive/_create_with_excludes), so site-owner self-service restores staged under that path are never recaptured into a subsequent archive.

Schedule

Key	Meaning	Default	Example
BACKUP_SCHEDULE	Time to run backups: HH:MM for a daily time, or a full systemd OnCalendar expression	03:30	Mon *-*-* 03:30:00

Notifications

Key	Meaning	Default	Example
NOTIFY_EMAIL	Recipient address; empty disables email notification entirely	(empty)	ops@example.com
NOTIFY_ON	failure (default, only email on a failed/incomplete run), always, or never	failure	always
NOTIFY_FROM	Override the From: address	root@$(hostname)	backups@example.com

Engine internals

Key	Meaning	Default	Settable via CLI/GUI?
VIRTBASE	ApisCP virtual host base; sites are $VIRTBASE/site<N>	/home/virtual	yes
STAGE	Reserved working area for the engine (not exercised by any active code path in this release)	/var/lib/apiscp-borg	yes
METRICS_DIR	Prometheus node_exporter textfile collector directory; blank autodetects common locations, and metrics are skipped if none is found	(empty / autodetect)	yes
RUN_DB_EXPORT	Run ApisCP's per-site database export (backup_dbs.php) before archiving: 1 or 0	1	yes
DB_EXPORT_CMD	Path to backup_dbs.php	$APNSCP_ROOT/bin/scripts/backup_dbs.php	no
APNSCP_CMD	ApisCP CLI binary, used for native per-site database verbs (list_databases/export/import)	/usr/local/apnscp/bin/cmd	no
APNSCP_ROOT	Path to the ApisCP installation	/usr/local/apnscp	no
LOCK_FILE	Single-instance lock file guarding against overlapping backup runs (via flock, if available)	/run/apiscp-borg.lock	no

Two further variables are read only via the environment, not the config file whitelist: APISCP_BORG_LIB (override the shared-library search path) and APISCP_BORG_CONFIG (override the config file path itself, default /etc/apiscp-borg/config), plus APISCP_BORG_LOG_TAG (the logger tag used by akp_log, default apiscp-borg) and APISCP_BORG_DELETED_LOG (used only by the deleteDomain hook, default /var/lib/apiscp-borg/deleted-accounts.log). BORG_RELOCATED_REPO_ACCESS_IS_OK is exported by the engine (default no, overridable via the real environment) so an unattended run does not block on Borg's interactive "repository moved" prompt.

5. CLI reference (Layer 1)

All three binaries source lib/apiscp-borg-common.sh (search order: $APISCP_BORG_LIB, ../lib/ relative to the binary, /usr/local/lib/apiscp-borg/, /usr/lib/apiscp-borg/) then $APISCP_BORG_CONFIG (default /etc/apiscp-borg/config). Every subcommand below matches the code exactly.

borg-apiscp-backup

Takes no subcommands; running it (with no arguments) performs one full backup run and exits 0 if every archive was created, 1 if any failed. Driven by the systemd timer/service, or launched detached by borg-apiscp-repo run.

Sequence on every run:

1. Acquire $LOCK_FILE via flock (non-blocking; a second concurrent run aborts immediately) if flock is available.
2. If RUN_DB_EXPORT=1, run $DB_EXPORT_CMD (ApisCP's backup_dbs.php) so per-site .sql dumps are current before they are archived.
3. For every $VIRTBASE/site<N> directory (skipping any not selected by BACKUP_SITES):
   • archive shadow (if present) as site<N>-shadow-{now};
   • archive info (if present) as site<N>-info-{now};
   • if BACKUP_DATABASES=1: enumerate the site's mysql and pgsql databases via $APNSCP_CMD -o json -d <domain> <engine>:list_databases, export each to a site-relative staging path (/tmp/.apiscp-borg-db/<engine>/<db>.sql, resolved inside the site's fst namespace by ApisCP), archive that staging tree from its real host location (<site>/fst/tmp/.apiscp-borg-db) as site<N>-db-{now}, then delete the staging tree from the site;
   • apply per-archive-prefix retention (borg prune) for site<N>-shadow-*, site<N>-info-*, and site<N>-db-* right away, so one site's daily churn is pruned every run rather than waiting for a separate step.
4. If BACKUP_SYSTEM=1: expand SYSTEM_PATHS and matching SYSTEM_GLOBS entries to those that actually exist, archive them together as _system-{now}, then prune _system-*.
5. If BACKUP_PATHS is non-empty: for each existing custom path, archive it as _custom-<slug>-{now} (slug via akp_slug), then prune _custom-<slug>-*.
6. Write Prometheus textfile metrics (always, even on failure or interrupt, via an EXIT/INT/TERM trap) and send an email notification per NOTIFY_EMAIL/NOTIFY_ON/NOTIFY_FROM.

Every archive is created with borg create --compression "${BORG_COMPRESSION:-zstd}", plus --exclude '*/.borg-restore' --exclude '/.borg-restore' and one --exclude per BACKUP_EXCLUDES entry.

Metrics written to $METRICS_DIR/apiscp-borg.prom (or autodetected): apiscp_borg_run_success, apiscp_borg_archives_ok, apiscp_borg_archives_failed, apiscp_borg_duration_seconds, apiscp_borg_db_export_success (1 ok / 0 failed / -1 skipped), and apiscp_borg_last_run_timestamp_seconds.

borg-apiscp-repo

Manages the repository and everything that is not a data restore.

Subcommand	Syntax	Description
status	borg-apiscp-repo status	Print repository: $BORG_REPO followed by borg info (first 30 lines, indented).
init	borg-apiscp-repo init	Create (initialise) the repository at BORG_REPO with BORG_ENCRYPTION. Requires BORG_PASSPHRASE already set. Idempotent: if borg info already succeeds against the repository, logs and returns 0 without re-initialising.
check-access	borg-apiscp-repo check-access	Print ok if the repository is reachable and the passphrase is correct, else error: cannot access .... Always exits 0 so a UI can read the message.
gen-passphrase	borg-apiscp-repo gen-passphrase	Generate a strong random passphrase (openssl rand -base64 30, or /dev/urandom fallback), store it as BORG_PASSPHRASE via config-set, and print only the passphrase to stdout.
key-status	borg-apiscp-repo key-status	Print yes if the one-time key/passphrase backup sentinel exists at $KEY_BACKUP_DIR/.apiscp-borg-key-backed-up, else no.
backup-key	borg-apiscp-repo backup-key [DEST] [--force]	Export repository.passphrase (if BORG_PASSPHRASE is set), repository.key (via borg key export, which is a no-op warning for repokey repositories since the key already lives in the repository), and a README.txt to DEST (default $KEY_BACKUP_DIR), all 0600. Refuses to redo (logs and returns 0) unless --force.
key-bundle	borg-apiscp-repo key-bundle	Print a self-contained recovery bundle (instructions, the plaintext passphrase, the repository location, and an exported key via borg key export :: /dev/stdout) to stdout. For the GUI's one-time post-creation display/download.
config-get	borg-apiscp-repo config-get	Print every KEY=VALUE line from the config file with surrounding quotes stripped (comments and malformed lines skipped).
config-set	borg-apiscp-repo config-set KEY VALUE	Set one config key, rejecting any key not in CONFIG_ALLOWED and any value containing a CR or LF. Value is stored single-quoted (shell-escaped) so the config, which is sourced by the engine, cannot be injected. Writes with mode 0600.
sites	borg-apiscp-repo sites	List site<N>\t<domain> for every ApisCP site under $VIRTBASE (domain falls back to the site id if siteinfo cannot be read).
run	borg-apiscp-repo run [--site SITE | --path PATH]	No args (or --all): systemctl start --no-block borg-apiscp-backup.service (returns immediately). --site SITE: launch a detached one-shot run scoped to just that site (BACKUP_SITES=SITE BACKUP_SYSTEM=0); SITE must match site[0-9]* or be the literal none. --path PATH: launch a detached one-shot backing up only that one absolute, existing, readable path (BACKUP_SITES=none BACKUP_SYSTEM=0 BACKUP_PATHS=PATH). Detached runs use systemd-run --collect if available, else a backgrounded env ... &.
running	borg-apiscp-repo running	Print systemctl is-active borg-apiscp-backup.service (e.g. active, inactive).
prune	borg-apiscp-repo prune	Apply retention (RETENTION_KEEP_*, numeric/duration values only) across every archive prefix: each site's shadow/info/db archives separately, plus _system-* and _custom-*. If no retention key is set, logs and does nothing.
maintenance	borg-apiscp-repo maintenance	Run borg compact, detached (systemd-run --collect if available, else backgrounded).
verify	borg-apiscp-repo verify [--data]	Run borg check (repository and archive consistency), detached. With --data, adds --verify-data to also read file contents back (slow, thorough); without it, only metadata is checked.
set-schedule	borg-apiscp-repo set-schedule [SPEC]	Set the backup timer's OnCalendar. SPEC is HH:MM (validated 00-23:00-59, becomes *-*-* HH:MM:00) or a full OnCalendar expression (validated against an allowed character set); falls back to BACKUP_SCHEDULE if SPEC is omitted. Writes /etc/systemd/system/borg-apiscp-backup.timer.d/override.conf, reloads systemd, restarts the timer. Prints the resulting OnCalendar value.
notify-test	borg-apiscp-repo notify-test	Send a test email to NOTIFY_EMAIL to confirm mail delivery; fails if NOTIFY_EMAIL is unset.
list-databases	borg-apiscp-repo list-databases SITE	Print <engine>\t<db> (engine mysql or pgsql) for every database ApisCP currently knows for SITE, via native per-site enumeration (a host-level dump cannot see per-site databases). SITE must match site[0-9]*.
(no args / -h/--help/help)		Print the full header usage comment (via awk) and exit 0.
(anything else)		Print unknown command: <cmd> to stderr and exit 2.

There is no connect, create, types, repo-info, validate-path, browse, or apply-policy subcommand: those either do not apply to Borg's simpler repository model, or (in the case of repo-info and validate-path) are computed directly in the Layer 2 module without a root round trip (see section 6).

borg-apiscp-restore

Restores data from a Borg archive. Borg stores archived paths relative (leading / stripped) and preserves POSIX ACLs and extended attributes natively as part of every archive, so restore is a plain borg extract with --strip-components to rebase the archived path under a staging target: no separate metadata-replay step is needed (that is the kopia engine's job, not Borg's). Archive naming (from the engine): siteN-shadow-{now}, siteN-info-{now}, siteN-db-{now}, _system-{now}, _custom-<slug>-{now}. "Newest" means the most recent archive matching a given prefix (borg list --glob-archives PREFIX-* --last 1).

Subcommand	Syntax	Description
list	borg-apiscp-restore list [PREFIX]	List archives, optionally filtered to PREFIX-* (e.g. site1), else every archive in the repository.
list-json	borg-apiscp-restore list-json PREFIX	Print raw borg list --json --glob-archives PREFIX-*. PREFIX is required. Intended for the module/GUI to parse.
site	borg-apiscp-restore site SITE TARGET [--subpath P ...]	Restore SITE's newest shadow and info archives into TARGET/{shadow,info}. With one or more --subpath, restores only those paths (relative to the site root, under shadow/) instead of the whole tree; each subpath is rejected if it starts with / or contains ... Fails if nothing was restored.
account	borg-apiscp-restore account SITE TARGET	Restore a whole account: files (shadow+info, fails hard if none exist) plus the newest SITE-db archive as a tree into TARGET/databases (warns, does not fail, if no database archive exists).
system	borg-apiscp-restore system TARGET	Restore the newest _system archive into TARGET via borg extract --numeric-ids.
restore-db	borg-apiscp-restore restore-db SITE ENGINE DB TARGET	Non-destructive: extract ENGINE/DB.sql from the newest SITE-db archive into TARGET/DB.sql, print the restored path. ENGINE must be mysql or pgsql; DB must match [A-Za-z0-9_-]+.
import-db	borg-apiscp-restore import-db SITE ENGINE DB	Destructive. Restores the newest dump of SITE's DB to a temp directory (there is no DUMPFILE argument: the newest archived dump is always the one imported), stages it inside the site's own fst tree, and imports it into the site's live database via $APNSCP_CMD -d <domain> <engine>:import, then removes the staged copy.
owner-files	borg-apiscp-restore owner-files SITE [--subpath P ...]	Site-owner variant of site: restores into the site's own filesystem under /.borg-restore/files-<epoch>/, chowned to the site owner, and prints the in-site relative path.
owner-account	borg-apiscp-restore owner-account SITE	Site-owner whole-account restore into /.borg-restore/account-<epoch>/ (files, plus databases under databases/), chowned to the owner. Prints the in-site relative path.
owner-restore-db	borg-apiscp-restore owner-restore-db SITE ENGINE DB	Site-owner, non-destructive: restore the newest dump of the site's own database into /.borg-restore/db-<epoch>/DB.sql, chowned to the owner. Prints the in-site relative path.
owner-import-db	borg-apiscp-restore owner-import-db SITE ENGINE DB	Site-owner, destructive: equivalent to import-db for the site's own database.
(no args / -h/--help/help)		Print the header usage comment and exit 0.
(anything else)		Print unknown command: <cmd> to stderr and exit 2.

Every "owner-*" subcommand takes SITE as an argument, but in every real invocation path that argument is supplied by the trusted Layer 2 module (borg.php), which derives it from $this->getAuthContext()->site (the authenticated session), never from a value a site user could control. The Layer 1 binaries additionally validate SITE matches site[0-9]*.

Current limitation, by design in this release: restore always uses the newest archive of a matching prefix. There is no way, from the CLI or the GUI, to restore an older archive by id/time, and owner-files/site support subpath filtering only within the shadow tree, not a general browse picker. import-db/owner-import-db take no dump-file argument at all: they always restore-and-import the newest SITE-db archive's dump for the named database.

6. Panel/API reference (Layer 2)

Borg_Module_Surrogate (src/modules/borg.php, installed as lib/modules/surrogates/borg.php) exposes every verb below as cpcmd borg:<verb> and to the two GUI apps (Page_Container::__call forwards $this->borg_<verb>(...) calls to the module). Every verb runs one of the three Layer 1 binaries as root via a scoped sudo -n, with a positional printf-style format string so Util_Process escapes every argument (see section 9).

Admin verbs (PRIVILEGE_ADMIN)

Verb	Parameters	Description
get_config		Return the current config as an associative array, with any key containing PASSPHRASE masked to ********.
set_option	$key, $value	Set one whitelisted config key (client-side newline check only; config-set is the authoritative validator).
status		Return borg-apiscp-repo status output.
check_access		Return borg-apiscp-repo check-access output (ok or an error message).
key_backed_up		true if key-status reports yes.
backup_key	$dest = null, $force = false	Run backup-key, optionally to a custom destination, optionally forced.
key_bundle		Return the one-time recovery bundle text (secrets included by design).
run	$site = '', $path = ''	Trigger a backup: $path takes precedence over $site; with neither, runs everything. $site is validated against ^site\d+$.
is_running		true if the backup service is currently active.
sites		Return siteN => domain for every ApisCP site, sorted by key.
site_snapshots	$site	Archives available for a site's files, newest first, as [{id,time}, ...]; the archive name is the id. Informational only: restore always uses the newest.
restore_site	$site, $target	Restore a site's newest shadow+info archives into a staging directory. There is no point-in-time selection or subpath picker exposed here (unlike the Layer 1 site subcommand's --subpath); the whole site tree is always restored.
init_repo		Create (initialise) the configured repository. Idempotent; requires a passphrase already set.
gen_passphrase		Generate and store a strong repository passphrase; returns it once.
set_backup_selection	$sites, $system, $paths = []	Set BACKUP_SITES (collapsing to all if the selection covers every known site, none if empty, otherwise a CSV), BACKUP_SYSTEM, and BACKUP_PATHS (each path re-validated with validate_path).
set_schedule	$when	Validate $when as HH:MM or a safe OnCalendar-like expression, store it in BACKUP_SCHEDULE, then call set-schedule.
maintenance		Run borg compact, detached.
verify	$data = false	Run borg check, detached; $data adds --verify-data (reads file contents back).
notify_test		Send a test notification email.
prune		Apply retention (borg prune) across every archive prefix right now, per the configured RETENTION_KEEP_* settings. Unlike kopia's global policy, there is no separate "apply policy" step: retention lives in config, and prune (also run automatically after each backup) is what enforces it.
repo_info		Return ['connected'=>bool, 'location'=>string], computed from get_config()['BORG_REPO'] and a live check-access call; there is no repository "type" in the Borg model.
validate_path	$path	true if the path is a plausible absolute path (non-empty, starts with /, no CR/LF). Unlike kopia's validate-path, there is no corresponding root-level CLI check: this is a best-effort structural check performed entirely in PHP, not authoritative (a path that does not actually exist when the engine runs is simply skipped with a warning in the log).
sites_with_snapshots		Site ids that have at least one siteN-shadow archive (for greying out empty sites in the restore picker).
site_databases	$site	[{engine, db}, ...] currently known for a site (live, from ApisCP, not from an archive).
database_backups	$site	[{id, time}, ...] (newest first) of dated siteN-db archives for a site. Informational: restore/import always use the newest.
restore_database	$site, $engine, $db, $target	Non-destructive restore of one database dump to a staging path.
restore_account	$site, $target	Restore a whole account (files + all databases) to a staging directory.
import_database	$site, $engine, $db	Destructive import of the newest archived dump into the live database. No dump-file parameter: import-db always restores and imports the newest siteN-db archive itself.
restore_system	$target	Restore the shared system archive to a staging directory. No subpath parameter is exposed (unlike kopia's restore_system).

Site-owner verbs (PRIVILEGE_SITE)

Every verb below resolves the caller's own site from the ApisCP auth context via a private authSite() helper ($this->getAuthContext()->site, validated against ^site\d+$) and never accepts a site identifier as a parameter. Where a database is named, it is additionally checked to belong to that resolved site (ownsDatabase()) before use.

Verb	Parameters	Description
my_site		['site'=>siteN|'', 'domain'=>string] for the caller.
my_snapshots		[{id,time}, ...] archives available for the caller's own site. Informational only; restore always uses the newest.
restore_my_files		Restore the caller's own files (newest archive) into their own site filesystem (chowned to them), returning the in-site relative path. No point-in-time or subpath selection in this release.
my_databases		[{engine,db}, ...] for the caller's own site.
my_database_backups		[{id,time}, ...] dated database archives for the caller's own site.
restore_my_database	$engine, $db	Non-destructive restore of one of the caller's own databases (newest archive) into their own filesystem; rejected if the database is not one the site currently owns.
import_my_database	$engine, $db, $confirm = false	Destructive: import the newest archived dump of one of the caller's own databases back into their live database. Requires $confirm === true.
restore_my_account		Restore the caller's whole account (files + all databases, newest archives) into their own filesystem, chowned to them.
run_my_backup		Trigger a detached backup run scoped to only the caller's own site.

7. Automations

Scheduled backup (systemd timer)

systemd/borg-apiscp-backup.timer runs borg-apiscp-backup.service (Type=oneshot, Nice=10, IOSchedulingClass=idle, TimeoutStartSec=0 so a long backup is never killed as if hung).

Setting	Value	Purpose
OnCalendar	*-*-* 03:30:00	Nightly at 03:30, before jitter
RandomizedDelaySec	1800	Up to 30 minutes of jitter so a fleet of servers does not hit shared storage simultaneously
Persistent	true	Catch up with a missed run if the machine was off at the scheduled time

BACKUP_SCHEDULE overrides OnCalendar without editing the shipped unit: borg-apiscp-repo set-schedule [SPEC] (or cpcmd borg:set_schedule, or the GUI time picker) writes /etc/systemd/system/borg-apiscp-backup.timer.d/override.conf containing an empty OnCalendar= (clearing the base unit's value) followed by OnCalendar=<computed value>, then runs systemctl daemon-reload and systemctl restart borg-apiscp-backup.timer. SPEC may be HH:MM (becomes *-*-* HH:MM:00) or a full OnCalendar expression.

Retention (borg prune)

There is no separate "apply policy" step as with kopia's global policy. Retention lives directly in the RETENTION_KEEP_* config keys, and borg prune is what enforces it, in two places:

1. Automatically, after every backup run, per archive-name prefix (each site's shadow, info, and db archives pruned separately, plus _system-* and _custom-<slug>-*), right inside borg-apiscp-backup, so retention never falls behind the schedule.
2. On demand, via borg-apiscp-repo prune (cpcmd borg:prune, or the GUI's "Prune now" button), which sweeps every site's shadow/info/db prefixes plus _system-*/_custom-* in one pass, using whichever RETENTION_KEEP_* values are currently set. If none are set, it logs and does nothing.

Maintenance and verify (detached)

borg-apiscp-repo maintenance runs borg compact (reclaims space freed by pruned archives); borg-apiscp-repo verify [--data] runs borg check (repository and archive consistency; --data also reads back every file's content, slower but thorough). Both are launched via systemd-run --collect when available (falling back to a backgrounded shell job) specifically so a web/API caller (the GUI, cpcmd) returns immediately rather than blocking on a potentially long-running repository operation.

Email notifications

Configured via NOTIFY_EMAIL / NOTIFY_ON / NOTIFY_FROM. Sent via the local sendmail (tried at /usr/sbin/sendmail, /usr/lib/sendmail, or sendmail on PATH); if none is found, mail is skipped as best-effort. After every borg-apiscp-backup run (success, failure, or an interrupted/crashed run, all handled by an EXIT/INT/TERM trap), notify() sends a plain text summary (host, result, finish time, archives ok/failed, database export result, duration, repository location) if NOTIFY_ON calls for it: never (never), always (always), or only when the run did not fully succeed (failure, the default; an interrupted run that never reached its normal end also counts as a failure). borg-apiscp-repo notify-test (cpcmd borg:notify_test, GUI "Send test email") sends a one-off confirmation message independent of an actual backup run.

Account-lifecycle hooks

ApisCP runs config/custom/hooks/<event>.sh with the site id (siteN) as $1 after the named account event. All three hooks below always exit 0 (hooks cannot interrupt or fail the underlying ApisCP operation) and run strictly after the event has already completed, not before or during it.

Hook	Source	Timing	Action
addDomain.sh	src/hooks/addDomain.sh	After a domain/account is created	Kicks off a detached backup scoped to just the new site (borg-apiscp-repo run --site <site>), so a brand-new account is protected immediately rather than waiting for the next scheduled run.
suspendDomain.sh	src/hooks/suspendDomain.sh	After a domain/account is suspended	Takes one final detached backup of the site (same run --site call) to capture its state at the moment of suspension, since a suspended account's files still exist on disk.
deleteDomain.sh	src/hooks/deleteDomain.sh	After a domain/account is deleted	Does not purge the site's archives. Only appends a timestamp\tsite line to an audit log ($APISCP_BORG_DELETED_LOG, default /var/lib/apiscp-borg/deleted-accounts.log) and logs to syslog. Existing archives are retained under the repository's normal retention policy; any purge is a deliberate, manual (or retention-window) decision, never automatic.

All three are installed only if the destination hook file does not already exist, or exists but does not already contain the string apiscp-borg (an operator's own custom hook is never overwritten).

8. Both GUIs

Appliance-admin panel: "Borg Backups" (src/apps/borg/)

Registered at /apps/borg via config/custom/templates/admin.php, under the admin menu's "System" category (internal id services). Every action maps to one PRIVILEGE_ADMIN verb from section 6. Sections, top to bottom:

• Repository: a compact connected/not-connected status line with a "Show details" toggle (raw borg info). When a repository is already connected, the connection fields are prepopulated and disabled behind a "Reconfigure repository" checkbox that must be ticked to unlock them. There is only one repository shape (a path or an ssh:// URL): fields are the repository location, passphrase, an optional "generate a strong passphrase" checkbox, an encryption-mode selector (repokey-blake2 / keyfile-blake2), and SSH options (BORG_RSH); there is no backend "type" selector, no server mode, and no SFTP key upload, unlike the kopia panel. "Save settings" persists; "Check access" confirms the repository is reachable with the current passphrase. Creating a new repository requires ticking an explicit acknowledgement ("I understand that if the repository passphrase/key is lost, the backups are permanently irrecoverable") before "Initialise repository" is enabled; immediately after creation, a one-time red-bordered panel displays the full recovery bundle in a read-only textarea plus a "Download recovery key" button (client-side Blob download, no extra request), and is not shown again on reload.
• What to back up: a two-box Available/Backing-up multi-select over sites, "System information", and any configured custom paths, with arrow buttons to move entries between boxes and a "Save selection" button, plus a checkbox for whether site databases are included. A separate text field plus "Add custom path" validates and appends an arbitrary absolute path (via validate_path) to the backing-up box.
• Retention and excludes: a "Within" duration field (e.g. 7d, 4w, 6m, 1y) plus four archive-count fields (Daily/Weekly/Monthly/Annual; blank = no limit), and a textarea of exclude globs, one per line (stored comma-separated on disk); a note that /.borg-restore is always excluded. "Save retention and excludes" persists the config; "Prune now" runs borg prune immediately using whatever is currently saved.
• Run: a status badge (idle / a backup is currently running, which disables the button) and a mode selector (everything / one site / a custom path) with "Run now" (always detached).
• Schedule and maintenance: an HTML time input bound to BACKUP_SCHEDULE with "Save schedule" (a systemd OnCalendar expression can still be set from the CLI), plus "Run maintenance now" (borg compact) and a "verify file contents" checkbox with "Verify now" (borg check, both detached).
• Notifications: recipient email, an on-failure/always/never selector, a sender override, "Save", and "Send test email".
• Repository key backup: shows whether the one-time key/passphrase backup sentinel exists, a destination field (defaults to KEY_BACKUP_DIR), "Back up key now", and a "force (redo)" checkbox.
• Restore: a single workflow with a "What would you like to restore?" type selector (a site's files / an entire account / a single database / system information) that swaps in the matching sub-panel via JavaScript, rather than four separate pages. A note states plainly that Borg does not yet support point-in-time archive selection or a subpath picker in this release: every restore uses the newest matching archive.
  • Site files: pick a site (sites with no archives are disabled in the dropdown), "Show backups" lists archive dates for information (newest first, up to 6 shown); a target field and "Restore newest archive".
  • Entire account: pick a site (same archive-availability graying), a target directory, "Restore entire account" (files plus all databases under a databases/ subdirectory).
  • System information: a target directory and "Restore system" (non-destructive).
  • Databases: pick a site, "List databases" shows each database with its engine, the dated backup history (up to 6 most recent shown, for information), and per-row "Restore dump to staging" (safe) versus "Import newest backup into live database" (destructive, requires a ticked confirmation checkbox and a JavaScript confirm() dialog).

A full-page "running" overlay (spinner + the clicked button's label) appears on every form submit and blocks further clicks until the page reloads, since Borg operations on a large repository can take a while; the page also restores scroll position across the postback so a save action does not jump the operator back to the top.

Site-owner panel: "My Borg Backups" (src/apps/myborgbackups/)

Registered at /apps/myborgbackups via config/custom/templates/site.php, under the site panel's "Account" category. Every action maps to one PRIVILEGE_SITE verb from section 6, and the page never sends a site identifier of its own. If the caller's site cannot be resolved from the auth context, the whole page shows only a warning to contact the administrator. Sections:

• Restore files: "Show my backups" lists the caller's own archive dates (informational, up to 6 shown); "Restore newest backup into my files (with permissions)" restores the newest archive into the owner's own site filesystem. A note states there is no point-in-time selection or file picker in this release.
• Restore my entire account: one button, restores all files plus all databases (newest archives) into /.borg-restore/account-<date>/ (databases under a databases/ subdirectory).
• Databases: "List my databases" shows engine/name plus dated backup history (informational); per-row "Restore dump to my files" (safe) and "Import newest backup into live database" (destructive, ticked confirmation plus a JS confirm()).
• Back up now: triggers an on-demand backup of just the caller's own site, in addition to the scheduled runs.

Every restore result banner states explicitly that the output landed under /.borg-restore/ in the owner's own file space and can be retrieved with the File Manager or SFTP. The same "running" overlay and scroll-position preservation as the admin app are used here too.

Common restore landing convention: every site-owner restore (files, database dump, or whole account) is written inside that site's own fst tree under a timestamped subdirectory of /.borg-restore/, and the entire /.borg-restore directory is then recursively chowned to the site's admin user (akp_owner_chown, preferring chown --reference against the owner's home directory, falling back to the numeric uid:gid parsed from the site's own /etc/passwd) so the owner can browse, download, and delete it through their own File Manager or SFTP account.

9. Security model

• The Layer 2 module runs inside the ApisCP frontend as the unprivileged apnscp user, which cannot read the root-owned Borg repository config or list /home/virtual. Every verb therefore shells out to one of the three Layer 1 binaries via Util_Process::exec('sudo -n <bin> %s %s ...', ...) (runTool()/runToolRaw() in borg.php), where the binary path is a fixed class constant and every argument is passed through Util_Process's own printf-style escaping, never string-concatenated into a shell command.
• install-layer2.sh installs a scoped /etc/sudoers.d/apiscp-borg granting exactly apnscp ALL=(root) NOPASSWD: <the three binary paths> (validated with visudo -cf before being installed) - nothing else is permitted to run as root. The three binaries themselves therefore are the privilege boundary: they validate every input (site id pattern, database engine must be mysql/pgsql, database name character set, target must be absolute, subpaths cannot contain .. or a leading /) before acting.
• /etc/apiscp-borg/config (holds the repository passphrase) is kept root-owned (0600), never read or written directly by the panel user; all reads go through config-get and all writes through config-set, both of which run as root via the same sudoers grant.
• config-set (and the module's set_option) enforce a fixed whitelist of settable keys (CONFIG_ALLOWED in bin/borg-apiscp-repo, mirrored as Borg_Module_Surrogate::ALLOWED_KEYS) and reject any value containing a CR or LF, so a caller cannot smuggle extra config lines or inject shell metacharacters into a file that is later .-sourced by the engine; the stored value is additionally single-quote-escaped.
• Site-owner (PRIVILEGE_SITE) verbs never accept a site identifier as a parameter. Every one of them calls a private authSite() helper that reads $this->getAuthContext()->site (set by ApisCP's own auth framework to the authenticated session's site) and validates it against ^site\d+$; if it cannot be resolved, the verb returns an error or an empty result rather than falling back to any caller-supplied value. Where a database name is also given (e.g. restore_my_database, import_my_database), it is cross-checked against that resolved site's current database list (ownsDatabase()) before use, so a site owner cannot reach another site's data even by guessing a database name. The corresponding Layer 1 subcommands (owner-files, owner-restore-db, owner-import-db, owner-account) repeat the site-pattern and path-traversal checks independently, so the guarantee does not rest on the module alone.
• Per-site database isolation: ApisCP's own <engine>:export/<engine>:import verbs resolve their file argument inside the target site's own filesystem namespace (a chroot-like view via file_make_path), not the host filesystem, and a host-level dump/restore tool cannot see per-site databases at all. The engine and the CLI tools therefore always go through $APNSCP_CMD -d <domain> <engine>:{list_databases,export,import} in the site's own domain context: a dump is written to a site-relative path (e.g. /tmp/.apiscp-borg-db/mysql/<db>.sql) which the engine then reads back from its real host-side location under <site>/fst/... for archiving, and an import stages the dump under the site's own fst before calling the site-context import. This is what makes per-site database backup and restore correct in a multi-tenant ApisCP host where database names can collide or be invisible outside their own site.
• A Borg repository is locked to one operation at a time (Borg's own repository lock, in addition to the engine's local $LOCK_FILE), so concurrent site-owner actions against the same repository queue rather than run in parallel; there is no risk of two writers corrupting the repository.

10. Restore and recovery

A Borg repository is encrypted; if the passphrase (and, for keyfile-mode repositories, the exported key) are both lost, the data is permanently unrecoverable no matter how many copies of the encrypted archives exist. borg-apiscp-repo backup-key (see section 5) exports everything needed to recover onto a fresh machine, and refuses to silently redo that export more than once (use --force to intentionally redo it after a credential rotation).

To recover the repository itself on a brand-new machine, using either the persistent key backup directory or the one-time key-bundle output:

# 1. install borgbackup (your package manager, e.g. dnf/apt install borgbackup)

# 2. obtain the repository passphrase (and, for keyfile-mode repositories,
#    the exported key), from either:
#      - the key backup directory (repository.passphrase + repository.key), or
#      - the one-time key-bundle text saved at repository-creation time

# 3. export the passphrase and the repository location
export BORG_PASSPHRASE="$(cat repository.passphrase)"
export BORG_REPO="<the repository location, e.g. /mnt/backups/apiscp-borg or ssh://...>"

# 4. only for keyfile-mode repositories (repokey-blake2 keeps the key IN the
#    repository, so this step is unnecessary for the default encryption mode):
borg key import :: repository.key

# 5. confirm the archives are visible
borg list

# 6. extract whatever is needed, e.g. the newest siteN-shadow archive
borg extract ::<archive-name>

From there, install apiscp-borg itself (install.sh, then point BORG_REPO/BORG_PASSPHRASE at the reconnected repository) to regain the per-site restore tooling (borg-apiscp-restore), or continue with plain borg list / borg extract if only a bare borg binary is available: no manual ACL/xattr replay step is needed, because Borg already restored that metadata as part of borg extract. The repository and its archives are never deleted by uninstall.sh (with or without --purge), so reinstalling this project at any point picks up exactly where the repository left off.