No backup: agent state exists only on this box #13

New issue

Closed

opened 2026-09-07 23:10:48 +00:00 by agent-pi · 1 comment

agent-pi commented

2026-09-07 23:10:48 +00:00

Collaborator

The ai-coding-box has no backup of any kind. Everything durable lives on a single disk:

/home/agent-pi/.pi — agent memory, skills, session history (~201 MB, irreplaceable)
/home/agent-pi/.ssh — the forge.marvin8.zone signing key (irreplaceable; losing it bricks the agent identity)
/etc/tailscale/authkey — boot-critical enrollment secret
shell/zoxide state, agent config

A disk failure loses the agent identity and all accumulated memory. The NixOS config itself is safe (it lives in this repo), and the wip repos are pushed to the forge, but none of the above is anywhere else.

What is needed: an unattended, encrypted, off-box nightly backup that requires no manual attention, with monitoring so a silent failure is visible.

The ai-coding-box has no backup of any kind. Everything durable lives on a single disk: - `/home/agent-pi/.pi` — agent memory, skills, session history (~201 MB, irreplaceable) - `/home/agent-pi/.ssh` — the forge.marvin8.zone signing key (irreplaceable; losing it bricks the agent identity) - `/etc/tailscale/authkey` — boot-critical enrollment secret - shell/zoxide state, agent config A disk failure loses the agent identity and all accumulated memory. The NixOS config itself is safe (it lives in this repo), and the wip repos are pushed to the forge, but none of the above is anywhere else. What is needed: an unattended, encrypted, off-box nightly backup that requires no manual attention, with monitoring so a silent failure is visible.

agent-pi commented

2026-09-07 23:11:12 +00:00

Author

Collaborator

Findings from exploring the box/repo

restic 0.18.1 is already installed (in environment.systemPackages) but no services.restic.backups.* is configured.
The nixpkgs 26.05 restic module (services.restic.backups.<name>) defaults to running the backup as root (user = "root"), so a cap_dac_read_search security wrapper is unnecessary to read ~/.ssh and /etc/tailscale/authkey. The module also offers createWrapper which installs a restic-<name> CLI with the repository env baked in, for manual snapshots/inspection.
Durable data inventory: ~/.pi 201M, ~/wip 219M (already on the forge — excluded), podman storage 2.8G and uv/nix caches ~0.4G (all re-downloadable — excluded). Real backup target ≈ 202 MB.
uptime.11c8.cloud resolves publicly and is reachable from the box (verified by dig + curl), so an Uptime Kuma push monitor works from here.

Plan of attack

Add services.restic.backups.nightly to hosts/ai-coding-box/default.nix:
- repository: s3:https://s3.ap-southeast-2.wasabisys.com/ai-coding-box-backup-au (new dedicated Wasabi bucket, ap-southeast-2), initialize = true
- root-run (module default), createWrapper = true, runCheck = true
- explicit paths: /etc/tailscale/authkey, /home/agent-pi/.pi, /home/agent-pi/.ssh, /home/agent-pi/.config, /home/agent-pi/.local/share/fish, /home/agent-pi/.local/share/zoxide, /home/agent-pi/devbox.json
- prune: --keep-daily 7 --group-by '' (matches the VPS; longer history is handled by the secondary backup)
- timer: OnCalendar = "*-*-* 02:00:00" (local, box TZ is Australia/Brisbane), Persistent = true, RandomizedDelaySec = "10min"
- no stop/start pre/post units — nothing long-running lives on this box
Monitoring: onSuccess and onFailure units push to Uptime Kuma (status=up / status=down); the push URL is read at runtime from /etc/restic/kuma-push.url (0600) so the token never lands in the repo or the world-readable Nix store.
Secrets (provisioned at console, SETUP.md §3 pattern, 0600 root): /etc/restic/password, /etc/restic/wasabi.env (AWS key pair), /etc/restic/kuma-push.url. SETUP.md gains the provisioning steps + first-run verification.
Key files: hosts/ai-coding-box/default.nix, SETUP.md.
Verification before PR: nix flake check and a local build of the host toplevel. No live-system change until after merge (nixos-rebuild switch, secrets placed, one manual systemctl start restic-backups-nightly, confirm Kuma ping).

**Findings from exploring the box/repo** - `restic` 0.18.1 is already installed (in `environment.systemPackages`) but no `services.restic.backups.*` is configured. - The nixpkgs 26.05 restic module (`services.restic.backups.<name>`) defaults to running the backup as **root** (`user = "root"`), so a `cap_dac_read_search` security wrapper is unnecessary to read `~/.ssh` and `/etc/tailscale/authkey`. The module also offers `createWrapper` which installs a `restic-<name>` CLI with the repository env baked in, for manual snapshots/inspection. - Durable data inventory: `~/.pi` 201M, `~/wip` 219M (already on the forge — excluded), podman storage 2.8G and uv/nix caches ~0.4G (all re-downloadable — excluded). Real backup target ≈ 202 MB. - `uptime.11c8.cloud` resolves publicly and is reachable from the box (verified by `dig` + `curl`), so an Uptime Kuma push monitor works from here. **Plan of attack** - Add `services.restic.backups.nightly` to `hosts/ai-coding-box/default.nix`: - repository: `s3:https://s3.ap-southeast-2.wasabisys.com/ai-coding-box-backup-au` (new dedicated Wasabi bucket, ap-southeast-2), `initialize = true` - root-run (module default), `createWrapper = true`, `runCheck = true` - explicit paths: `/etc/tailscale/authkey`, `/home/agent-pi/.pi`, `/home/agent-pi/.ssh`, `/home/agent-pi/.config`, `/home/agent-pi/.local/share/fish`, `/home/agent-pi/.local/share/zoxide`, `/home/agent-pi/devbox.json` - prune: `--keep-daily 7 --group-by ''` (matches the VPS; longer history is handled by the secondary backup) - timer: `OnCalendar = "*-*-* 02:00:00"` (local, box TZ is Australia/Brisbane), `Persistent = true`, `RandomizedDelaySec = "10min"` - no stop/start pre/post units — nothing long-running lives on this box - Monitoring: `onSuccess` and `onFailure` units push to Uptime Kuma (`status=up` / `status=down`); the push URL is read at runtime from `/etc/restic/kuma-push.url` (0600) so the token never lands in the repo or the world-readable Nix store. - Secrets (provisioned at console, SETUP.md §3 pattern, 0600 root): `/etc/restic/password`, `/etc/restic/wasabi.env` (AWS key pair), `/etc/restic/kuma-push.url`. SETUP.md gains the provisioning steps + first-run verification. - Key files: `hosts/ai-coding-box/default.nix`, `SETUP.md`. - Verification before PR: `nix flake check` and a local build of the host toplevel. No live-system change until after merge (`nixos-rebuild switch`, secrets placed, one manual `systemctl start restic-backups-nightly`, confirm Kuma ping).

agent-pi referenced this issue from a pull request that will close it,

2026-09-07 23:15:33 +00:00

Nightly restic backup to Wasabi #14

agent-pi closed this issue

2026-09-07 23:37:44 +00:00