vpsmgr/docs/architecture.md

Architecture

vpsmgr is a lightweight LXC hosting panel: one Debian 13 container per user account, managed from a web panel and a CLI. Everything ships as a single Go binary (vps = CLI + embedded web panel); Incus, nftables and HAProxy provide the plumbing.

Design goals

vpsmgr is a toy for small machines (≤ 4 GB RAM, small VPS). Storage and memory are treated as scarce, which drives every choice in this document:

• the panel is a single static Go binary — the only new service vpsmgr adds besides Incus and HAProxy;
• the storage pool is sparse (a loop file that only grows as it fills) and the published image is slimmed and the base image deleted, so disk is only consumed by what containers actually use, and clones share the image's blocks;
• container tooling stays minimal (git / python3 are deliberately absent);
• resource stats are sampled once per minute in the background and served from SQLite, so opening a panel does not fan out Incus requests or run commands in containers.

"Lightweight" refers to the panel and this storage/memory discipline, not to a zero-overhead platform: Incus, nftables and HAProxy are the minimal plumbing that makes the panel possible.

Components

install.sh / uninstall.sh / build.sh   # lifecycle + local build
scripts/  00-check 10-incus 20-network 30-haproxy 40-panel 50-image
configs/  reference configs (haproxy / systemd / sudoers)
src/      Go source (single binary: CLI + panel)

• Go binary — CLI commands (vps add/show/del/...) and the HTTPS panel (vps.service). The web templates are embedded (//go:embed).

• Incus (Zabbly LTS 7.0, Debian package) — runs the containers. Storage pool vpsmgr (ZFS by default; btrfs and the throwaway dir backend are alternatives, see “Storage”), bridge incusbr0 (10..0.0/24, octet chosen at install, default 115). The panel talks to the daemon over its Unix-socket REST API (internal/lx, one reusable HTTP connection, no incus process spawn per call). Every operation including exec (provisioning scripts and readiness probes) goes over the API websocket transport — no CLI calls at all. Resource sampling uses the bulk metrics endpoint rather than per-container exec. Fractional CPU quotas (0.1..0.9) are enforced as limits.cpu=1 plus limits.cpu.allowance=<n>ms/100ms — a one-core pin with a time slice.

• nftables — one table inet vpsmgr: DNAT (prerouting+output) for port ranges, MASQUERADE for NAT4. Reload applies the whole ruleset in a single nft -f batch (the generated file starts with a delete-table line so a load error leaves the previous ruleset in place). Restored on boot by vps-nft.service. The reload runs through the sudoers whitelist (the panel daemon is unprivileged).

• UFW — vpsmgr manages its firewall through its own nftables table, so the installer disables UFW when it is active: UFW's default-DROP policy runs before Incus's table inet incus rules and silently kills container IPv4 (no DHCP, no DNS, no forwarded traffic), which makes both the image build and every container's network fail. If you keep UFW, you must add these rules yourself (see 00-check.sh):

  ufw allow in on incusbr0 to any port 67 proto udp   # DHCP
  ufw allow in on incusbr0 to any port 53             # DNS
  ufw route allow in on incusbr0 from 10.<n>.0.0/24    # container forwarding
  

• HAProxy — the domain front, pinned to the 3.4 LTS branch (official performance packages). Port 80 reverse-proxies per Host; 443 routes by SNI with TLS passthrough (the certificate lives in the container, HAProxy never terminates TLS). Runtime configuration is published as immutable generations under /etc/haproxy; see “Domain proxy” below.

• SQLite — users, domains, sessions, bandwidth counters and seven-day minute resource history. Located at /etc/vpsmgr/vpsmgr.db. Migrations run in order on open; the recorded versions must form an unbroken run, and a gap (or a database newer than the binary) refuses to start rather than limping on with a half-migrated schema.

Unprivileged panel

The panel daemon runs as the dedicated unprivileged vps system user (User=vps in vps.service), never as root:

• Incus access comes from group membership: the Incus Unix socket is group incus-admin rw, and vps install adds vps to incus-admin. The full management API is therefore available without any privilege elevation.
• The only root commands the panel may run are pinned in a sudoers whitelist (/etc/sudoers.d/vps, validated with visudo -c before install): nft reloads, systemctl control of haproxy / the panel itself, IPv6 route/neighbor/addr changes, the IPv6-forwarding sysctl and ndppd control. internal/su runs them via sudo -n; anything else fails instead of prompting.
• vps install (the install-time setup) still runs as root because it creates the user, chowns the writable dirs, writes systemd units and loads kernel modules. The long-running vps serve is fully unprivileged.

Users and resources

• User i (the container's slot, idx = v4-last-octet - 1): container IP 10.<n>.0.(i+1) (subnet 10.<n>.0.0/24, second octet chosen at install, default 115), a random SSH port in 30000-31999 (DNAT to container 22), and a whole-hundred block of 100 user ports (DNAT to the container, TCP+UDP). The block a new container takes is drawn from net.user_ports (default 10000-29999 = 200 blocks = 200 containers), a comma-separated set of inclusive port ranges auto-aligned to whole hundreds; narrowing the ranges reduces capacity and the user-port span a host reserves, and affects new containers only — existing ones keep their port blocks even outside the current ranges. The IP slot idx (1..200) and the port block are allocated independently: NextFreeIdx picks the IPv4, allocUserPortBlock picks a free 100-port block inside the configured ranges (start_port UNIQUE is the cross-process backstop).

• IPv4 inbound policy (v4_forward): true enables the SSH/port-block DNAT above. web-only removes that direct DNAT but keeps the public IPv4 visible and allows HAProxy domain forwarding on host ports 80/443 when net.haproxy is enabled. false removes the direct DNAT and stops HAProxy (domains are kept but not served). The NAT4 masquerade stays in every mode so containers still reach IPv4 outbound. Toggle at runtime with vps config set net.v4_forward true|false|web-only (applied immediately).

• Docker compatibility: on hosts running Docker, Docker's default iptables policy (-P FORWARD DROP) would otherwise block forwarded traffic from incusbr0. The installer adds allow rules to the DOCKER-USER chain (both IPv4 and IPv6) and installs a docker.service.d/vpsmgr.conf systemd drop-in so incusbr0 forwarding stays active across Docker restarts without interfering with Docker's container isolation.

• Port 25 is always blocked: the ruleset's forward chain drops port 25 (TCP+UDP, both directions) for all forwarded traffic — permanent anti-spam, only a full uninstall clears it.

• Optional whole /64 per container: on a prefix-mode host whose provider delegates a prefix shorter than /64, net.ipv6_extra_prefix lets a container own a whole /64 out of it alongside its /112 primary — the only block size inside which SLAAC and sub-delegation work. OFF by default and invisible until an operator sets that key; the /64 this host itself uses is never handed out. See ipv6.md.

• Add/Del/Reinstall are serialized by a per-process mutex, and mgr.Add rolls back the container, IPv6 route, nft rules and DB record on any post-launch failure. The DB write is a single transaction (db.CreateUserFull): user row + initial bandwidth row + optional quota commit atomically, so a crash can never leave a half-created user. mgr.Del refuses to drop the DB row when the container cannot actually be removed (it would orphan the container and let NextFreeIdx reuse its IP for a new user); a fresh add also refuses a name/IP already claimed by a live Incus instance, so orphaned containers cannot cause bridge IP conflicts.

• Quotas: CPU (whole cores ≥ 1, or a fraction 0.1..0.9 of one core), memory (MiB), disk (GiB). Disk maps onto the storage driver's quota (ZFS space accounting, btrfs qgroups, or the accepted-but-unenforced dir case) and can only grow, never shrink.

• Quota validity (expiry): each user can carry an optional absolute deadline (users.expires_at, RFC3339 UTC, '' = permanent). When it passes, the 60s loop force-stops the container and disables boot.autostart, and the account goes read-only for both sides. The admin can still extend it (max(now, current) + duration), delete the account, and reset its panel password; the user can still change their own panel password. Extending lifts the lock but leaves autostart off, so the container needs a manual start. The container and its snapshots are never deleted, so a checkpoint shared before expiry stays installable by others.

• Container swap: Incus 7 on cgroup v2 writes memory.swap.max=0 for every container unless limits.memory.swap carries an explicit byte amount (true/false both end up as 0), so without an explicit value containers cannot use the host's swap at all and free inside shows 0. vpsmgr sets limits.memory.swap = limits.memory × incus.swap_ratio (default 0.5) at create/reinstall and on quota changes. The allowance is a per-container cap, not a reservation — the host swap pool is shared, so the sum of all container allowances may over-subscribe a small host swap (e.g. zram); lxcfs clamps the visible SwapTotal to the host's actual swap, so a container never sees more swap than physically exists. Setting incus.swap_ratio re-applies the allowance to every existing container immediately (no restart); vps install does the same for containers created before swap support (upgrade path).

• Bandwidth quota: each user can carry a monthly bandwidth quota in GiB (users.bandwidth_quota_gb, 0 = unlimited; counts upload + download of the current month). The 60s bandwidth sampler enforces it: over-quota containers get their eth0 rate-limited to 1Mbps each direction, back-under (e.g. monthly rollover) is unthrottled. The NIC limits are applied LIVE by Incus via tc (htb qdisc on the host veth) — no container restart — and the manager keeps an in-memory throttle state so Incus is only touched on state changes.

• Domains: the domains table (owner via user_id, PROXY flag, UTC created/updated timestamps) is the single source of truth. The published HAProxy configuration is a derived artifact and is never edited in place:

  /etc/haproxy/
    haproxy.cfg                 # static entry config (frontends + reject backends)
    current -> releases/<gen>/  # the generation in service
    releases/<gen>/
      maps/http.map             # Host -> backend   (port 80)
      maps/sni.map              # SNI  -> backend   (port 443)
      domains.cfg               # one backend per domain
      manifest                  # generation, domain count, timestamp
    generation                  # counter of the last published generation
  

  Every mutation builds a complete new generation from the DB (internal/hpx.PublishAll), validates it with haproxy -c, and only then atomically repoints the current symlink. Because a generation is rendered from scratch, a stale domain can never survive into it, and a rejected candidate leaves the running proxy untouched — the previous generation keeps serving. The three most recent generations are retained for rollback and debugging. Routing lives in the generation's two MAP files (Host -> backend for port 80, SNI -> backend for 443). Maps rather than frontend ACLs because HAProxy refuses duplicate proxy-section names since 3.3 and attaches bare directives to the section opened last — per-domain rules emitted from a second file would either clash or land in the wrong section. A map miss yields an empty backend name, which HAProxy treats as "no rule matched" and falls through to the frontend's reject backend, so an unregistered host or SNI is never forwarded anywhere. PROXY protocol v2 is a TCP-only feature: a flagged domain's 443 backend carries send-proxy-v2 (the container's TLS listener must support it), while port 80 never does — there the client address travels in X-Forwarded-For, which HAProxy adds via option forwardfor. Backend names are sanitizeDomain(domain) (dots → underscores), which is collision-free across the [a-z0-9.-] domain charset. DB and configuration are updated sequentially, not atomically: a domain mutation writes the DB first, then republishes AND reloads (systemctl reload haproxy.service), and rolls the DB back if either fails; a crash or full disk between the steps can leave them out of sync. SyncAllDomains (run on vps install, on vps config set net.v4_forward true, on vps config set net.haproxy true and at every panel start) rebuilds the generation from the DB, repairing any drift. A reload (SIGUSR2 to the master) starts a new worker on the new generation and lets the old one drain its established connections, so publishing a domain never drops a live HTTP or TLS connection; hard-stop-after bounds how long a draining worker may linger. All timestamps are stored as UTC; the admin domain page renders them in the browser's timezone.

• Blocked domains: the admin domain panel keeps a blocked-domains list (a settings key, blocked_domains, one domain per line, managed via a textarea). mgr.AddDomain — the single add path for the user panel — refuses a domain that exactly equals a blocked entry or is a subdomain of one (label boundary: example.co.uk blocks a.example.co.uk but not forexample.co.uk). No wildcards or regex; entries and additions share the same normalization: lowercase, trailing dots stripped, only [a-z0-9.-] allowed (anything else is a hard error — pasting a URL is rejected, not silently rewritten), must end with a letter (which also excludes dotted-quad IPs), must contain at least one dot (no single-word names), every label must start/end with a letter or digit (consecutive hyphens mid-label like xn--p1ai are fine), total ≤ 253 and label ≤ 63. Only the add path is guarded — domains bound before an entry was blocked stay untouched. The save handler validates each line and reports invalid lines by their numbers while saving the rest.

• Audit log: resource-heavy user actions (power start/stop/restart, reinstall, reset root password, domain config changes) are recorded in the audit_log table (actor / action / UTC timestamp). The actor encodes who acted: a plain username for a user's own action, or 000+<username> when an admin acts on that user's resources (usernames can't contain + or start with a digit, so the marker never collides). The admin audit page renders client-side in 500-row chunks fetched from /audit/api with infinite scroll, so the page never renders thousands of rows at once; the latest 5000 rows are kept (pruned on insert, ~1 MB).

• Init script: each user can store a custom shell script (≤ 64 KiB) in their panel. On a successful reinstall it is written to the container over exec stdin (never the host command line — no injection surface) and run detached as root inside the container, logging to /var/log/vpsmgr-init.log there. Shebangs are honored; a hanging script cannot block the reinstall because it is backgrounded. Delivery failure only warns — the reinstall still succeeds.

• Accent color (group-wide): each user can carry an accent color (users.color, a hex string from the admin panel's fixed 10-color palette in admin — black/white/gray are excluded; the palette is the allowlist, so a free-form hex can never be stored). The entry point is deliberately low-key: clicking the username in the admin user table opens a swatch picker with a "clear" action (reset to ''); there is no dedicated row button, so the already-crowded table stays uncluttered. The color feeds two places: the user's 「登录面板」/Login button in the admin table (default is a neutral black-in-light / white-in-dark, themed when a color is set) and the user panel, which injects it as a CSS-variable override — --accent/--head recolor headings, primary buttons and links, and the page background gets a soft color-mix() tint that stays readable in both light and dark mode. The user-panel tint renders only for operator impersonation sessions, so a user logging in with their own password always sees the default theme. Only the admin can set it; users cannot. Setting/clearing is audited as color.update under 000+<user>. The color is shared across a user group (see “User groups” below): writing it updates every member, and a new member inherits the group's existing color.

• Snapshots ("时光机" / time machine): each user can keep up to snapshots.limit disk-only checkpoints (named snap-<UTC>...) of their container and create / delete / restore them from the panel. The panel calls them 检查点/checkpoints and frames the whole feature as a time machine that only goes back: restoring to a checkpoint that is NOT the newest discards the current data and every checkpoint created after it.

  Restore is a single Incus call. On the ZFS pool every container's root storage volume carries zfs.remove_snapshots=true, so Incus performs a ZFS recursive rollback that auto-destroys the newer snapshots in one operation — mirroring the ZFS rule that you cannot roll back to an older snapshot while newer ones exist (cannot be restored due to subsequent snapshot(s)). The setting is applied to the root volume when a container is added or reinstalled, and by an ApplyZFSRemoveSnapshotsToAll pass that vps install runs, so a host upgraded via install.sh --update gets the behaviour immediately without reinstalling containers. It only applies to the ZFS pool; btrfs and dir are skipped (the btrfs driver restores by subvolume copy, which newer snapshots never block, so nothing extra is needed there). As a belt-and-braces fallback, SnapshotRestore checks for the "subsequent snapshot" refusal on a container that somehow lacks the option, deletes the newer snapshots by creation time, and retries once.

  Restoring a checkpoint keeps the current container configuration (quotas, NICs, autostart) — only the disk is rolled back, so the restore never desyncs the Incus configuration from the vpsmgr DB. Reinstalling the container deletes every checkpoint, which the reinstall dialog warns about with the exact count.

• Snapshot sharing (install from someone's checkpoint): a user can publish one checkpoint behind a random v4-UUID code (snapshot_shares, one row per user). Another user enters that code in the reinstall dialog and gets a container cloned copy-on-write from the checkpoint — no image is built, so it costs no real disk until it diverges. The clone is then re-provisioned as the new owner's container: own IP/ports/quota/network, a fresh random hostname, a new root password, a fresh machine-id, and the source's authorized_keys cleared before the new owner's keys are written. The source container and its checkpoint are untouched, so one code can serve many installs, and the init script is deliberately not run (the clone already carries the checkpoint's state).

  Cloning from a checkpoint pins it: ZFS cannot roll the source back past a snapshot a clone depends on. Publishing therefore deletes the older checkpoints, keeping only the published one; the panel warns with the exact count first. Deleting the checkpoint, reinstalling or deleting the container revokes the share with it. A code pointing at the same container being reinstalled is a plain restore (cloning first would destroy the snapshot).

  The feature is off by default and has no panel UI: enable it with vps config set snapshots.share true. While off, the user-side entry points are hidden and new shares and installs are refused; stored codes and every container and checkpoint are left alone, so re-enabling restores the codes as long as their checkpoint exists. A code is dropped only when its checkpoint is deleted or its container goes away, and a code whose checkpoint has vanished fails with an error instead of half-building a container. Installs are audited as reinstall.share.<owner>, shares as snapshot.share.<checkpoint>.

User groups (multi-container users)

One login can manage several containers by grouping panel accounts under a naming convention. There is no group table — the group is derived from the account name at read time, so adding or removing a member needs no migration:

• An account parent and every account named parent-<digits> are one user group. alice is the parent (labelled machine A); alice-1, alice-09 are its children (labelled by their digit suffix, leading zeros preserved).
• The parent part must start and end with a letter. Nesting is rejected: alice-1-2 is not a child, and a -<digits> suffix whose prefix is not a valid parent (alice1-2, a-1) makes the name invalid. Names stay ≤ 31 characters. mgr.ParseUserGroup / UserGroupName are the single definitions, covered by group_test.go.
• Only the admin panel can create a child (ValidateAddName(..., true)). The CLI / vps add path passes AllowChild=false and rejects child names. The admin create form mirrors the rule client-side and previews the group and its size before submitting.

State that is shared across a group:

• Panel password. A brand-new group gets a fresh 20-char password; adding a member to an existing group reuses the group's bcrypt hash, and the one-time credential block prints “same as the existing user-group password” instead of a password. This holds on the CLI too: vps add of a parent name whose group already has members inherits the group hash, and its output says the password was inherited rather than printing one. Changing it (ChangePanelPassword, the path behind both the panel's own change flow and the admin reset) hashes once and rewrites every member in a single transaction, dropping each member's other sessions — the change flow keeps the caller's current token, an admin reset passes an empty token and drops all (db.UpdateUsersPasswordAndDeleteSessions). The container root password is not shared: every container gets its own, and for an inherited group member that value is generated but deliberately not displayed, so the root password must be reset from the panel.
• Accent color. SetGroupColor writes one color to all members (db.UpdateUsersColor, one transaction), and inheritGroupColorLocked gives a newly added member the first color found in its group.
• A container switcher. With more than one member the user panel shows a <select> in the header. Each option is labelled <label>-<cpu>c<mem>g<disk>g — the machine label (A for the base account, else the numeric suffix) plus the three quota values, e.g. 2-4c8g40g; the group-name prefix is dropped because every option shares it. The list is ordered base-first then by numeric suffix (so A, 1, 2, … 10, not lexical). POST /switch-user accepts only a name in the caller's own group, mints a session for the target, drops the old one and audits session.switch.<target>. A warning banner (repeated inside the reinstall and snapshot dialogs) names the current machine label so a destructive action is not aimed at the wrong sibling, and the sticky-notes card title carries the same label (Machine 2) when the group has more than one container — notes themselves stay per-container, so each machine keeps its own encrypted set.

All of it is process-safe: the add path and the password/color changes hold the manager's operation mutex, and the multi-row writes are single transactions. Deleting one member never touches its siblings, and because the group is name- derived it survives the parent's deletion: a leftover child keeps the group name (and its shared credentials) with no A machine, and re-adding the parent name later simply rejoins the same group and inherits its password — the convention working as designed, not a fresh tenant.

Storage

Three backends, all selected by setting VPSMGR_STORAGE when running install.sh (default zfs): zfs, btrfs, dir. zfs and btrfs both provide the full disk model (per-container disk quotas, snapshots, clone-on-create); dir provides none of it. The choice is made explicitly at install and a failed pool is a hard abort — never an automatic fallback. In the Go panel, container management is storage-driver-transparent: there is no driver branch in the panel code, and the single driver-aware helper (lx.EnsureZFSRemoveSnapshots) is a no-op on non-ZFS pools.

• ZFS (default). On first install, 10-incus.sh adopts an existing pool or creates a sparse loop-file pool sized to a share of the free space on /: 80% by default, 90% when ≥ 20 GiB free. New installations never scan, select, format, or modify secondary disks. The loop file only allocates blocks as the pool actually fills. On very small hosts, cap the ZFS ARC (zfs.arc_max) so container memory keeps priority over the pool's cache. New containers get a hard disk limit: the pool sets volume.zfs.use_refquota=true (10-incus.sh plus an EnsurePoolRefQuota pass in vps install), so a volume's usage counts every block it references, including blocks shared through CoW clones. ZFS's default quota charges a clone only for its own delta, which would stop matching what the user sees inside the container. Existing volumes are untouched — pool defaults apply to new ones only.
• btrfs (beta). Selecting VPSMGR_STORAGE=btrfs is flagged as beta and install.sh asks for an explicit confirmation up front (default yes — on a btrfs root there is no ZFS fallback). When / is itself a btrfs filesystem, the pool is created as a native subvolume directly on the host filesystem (incus storage create vpsmgr btrfs source=/var/lib/incus/storage-pools/vpsmgr): reflink clones, no loop-device overhead, no double space accounting, and the whole root's free space is available to the pool. On any other host it becomes a sparse loop-file pool sized exactly like ZFS. Per-container disk limits use btrfs qgroups; snapshots and clone-on-create work natively. 00-check.sh installs btrfs-progs (and skips the ZFS DKMS/kernel-header toolchain entirely). On a btrfs root the swap file is created with btrfs filesystem mkswapfile (a plain fallocate+mkswap swapfile fails there — swap must be NODATACOW and preallocated).
• dir (throwaway test boxes only). Set VPSMGR_STORAGE=dir. dir has no quotas, snapshots or clone-on-create: the admin's disk-limit setting is accepted but not enforced, every container costs a full image copy, and --purge still removes the pool via incus storage delete. In dir mode 00-check.sh skips zfsutils-linux/btrfs-progs and the ZFS module build entirely (no kernel module needed).
• Containers are copy-on-write clones of the image (ZFS clones, btrfs reflinks), so the image's blocks are shared: a well-provisioned image costs one copy no matter how many containers. Because Incus's space accounting counts inherited blocks, image bloat also eats into every container's disk quota — images must stay slim. (Clone-on-create only exists on ZFS/btrfs; dir mode copies the full image per container.)

Image (vpsmgr/debian-sshd)

Built by 50-image.sh from images:debian/13 (fallback images:debian/trixie):

• Installs openssh-server plus universal user tooling: curl, wget, ca-certificates, less, bind9-dnsutils, openssh-client, unzip, nano. (ca-certificates is essential — without it all HTTPS fails.)
• Slims the image before publishing: apt-get clean, removes /var/lib/apt/lists/*, logs and tmp. Without this the image balloons ~50 MiB+ in apt lists alone.
• Deletes the Debian base image afterwards — it is only a build intermediate; the runtime fallback for containers is the remote images:debian/13.
• git / python3 are deliberately not baked in (heavy / opinionated).

Optional RHEL-family image (vpsmgr/alma-sshd)

60-rhel-image.sh (NOT part of install.sh, run by the admin only when wanted) builds an Alma 9 image — rocky builds Rocky 9 instead — with the same hygiene: sshd + universal tooling via dnf, dnf clean all + cache/log removal, base image deleted after publishing.

The reinstall dialog in the user panel enumerates every local vpsmgr/*-sshd image (always offering Debian 13 first) and the user picks one; mgr.Reinstall validates that a picked non-default image still exists. Containers run from either base with the same light provisioning (random hostname, root password, sshd enabled — the service is sshd on RHEL and ssh on Debian).

Optional openSUSE image (vpsmgr/opensuse-sshd)

70-opensuse-image.sh (also NOT part of install.sh) builds an openSUSE Leap 16 image — images:opensuse/16.0, published as vpsmgr/opensuse-sshd — with the same hygiene: sshd + universal tooling via zypper, caches/logs cleaned, base image deleted after publishing. Leap 16 ships systemd-networkd by default, so its IPv6 is configured at runtime by the panel's standard networkd branch (static /128 + fe80::1 gateway), unlike the RHEL-family nmcli path.

Optional Debian dev image (vpsmgr/debian-dev-sshd)

80-debian-dev-image.sh (also NOT part of install.sh) builds a heavy dev toolchain image on top of images:debian/13 — the universal sshd/tooling set from 50-image.sh, plus python3/pip/venv, git, sqlite3, ripgrep, jq, gh, sshpass (all apt-get --no-install-recommends), Go 1.26.7 (arch-aware manual install, SHA-256 verified, /usr/local/go + /etc/profile.d/go.sh) and nvm v0.40.7 + Node.js 24 (baked into /root/.nvm, /etc/profile.d/nodejs.sh). Same hygiene as 50-image.sh: the eth0 IPv6 on-link fix is baked, apt lists/archives, logs, nvm/npm caches and the machine-id are cleaned before publishing, and the base image is deleted afterwards. Expect the published image to be roughly 3-4x the size of vpsmgr/debian-sshd — Go and Node dominate.

Optional Arch Linux image (vpsmgr/arch-sshd)

90-arch-image.sh (also NOT part of install.sh) builds an Arch Linux image from the rolling upstream images:archlinux/current — openssh (both sshd and the ssh client on Arch), ca-certificates, curl/wget, less, bind-tools, unzip, nano via pacman -Syu (so the snapshot is current), with pacman caches/logs and the machine-id cleaned before publishing and the base deleted afterwards. Because Arch is a rolling release the script does NOT skip when the image exists: every run deletes the old vpsmgr/arch-sshd, re-pulls the latest upstream base and rebuilds fresh. The published image's description records the build's date as 8 digits (YYYYMMDD, e.g. 20260824 for a 2026-08-24 build), visible via incus image show vpsmgr/arch-sshd and folded into the Arch intro the user panel shows in the reinstall dialog. Networking follows the openSUSE pattern: Arch ships systemd-networkd, so the panel's networkd branch configures IPv6 at runtime — nothing to bake.

Per-container provisioning

mgr.Provision runs inside every new or reinstalled container and:

1. Sets a random hostname (vps-<8 hex>, crypto/rand) — never the username, so users cannot identify each other from prompts/logs/banners on the internal network. Re-rolled on every reinstall.
2. Disables cloud-init hostname resets (preserve_hostname: true).
3. Sets the root password and ensures sshd is enabled/running.

Container isolation on the bridge

All containers share the single incusbr0 L2 segment 10.<n>.0.0/24 (the second octet is chosen at install; default 10.115.0.0/24). To make sure a scan does not reveal usernames:

• Incus's dnsmasq must NOT serve instance-name DNS: by default it publishes <instance>.lxd records (instance name = username) that turn into username.lxd PTR answers — this is independent of the randomized in-guest hostname, so it would leak usernames anyway. 10-incus.sh therefore sets dns.mode=none on incusbr0 (DHCP and upstream forwarding still work; the search lxd suffix is dropped and reverse lookups fall back to the random guest hostname or the upstream resolver).
• The in-guest hostname is already randomized (see above), so nothing username-derived is ever advertised on the wire.

L2 isolation per container

Every container's eth0 is created (and migrated) with three Incus NIC security options (lx.nicIsolation):

• security.port_isolation=true — the veth is an isolated bridge port (isolated on), so no frames (unicast, multicast, broadcast) flow between containers at L2. This blocks ARP/NDP spoofing, L2 sniffing, and rogue DHCP/DHCPv6/DNS servers.
• security.ipv4_filtering=true + security.ipv6_filtering=true — Incus installs bridge input/forward nftables rules per NIC that only accept ARP/NDP/NA claiming the container's own addresses or MAC, and drop router-advertisements from containers. This protects the host's own ARP/NDP caches from container-side poisoning (a container cannot rewrite the host's neighbor entries).

Observed side effects compared with the flat-bridge baseline:

• Containers cannot reach each other on the private bridge anymore (IPv4 and IPv6 alike) — by design. Any inter-container traffic would have to go through a public address + DNAT; the host does not proxy the private subnet.
• ARP spoofing a neighbor or the gateway, NDP/NA spoofing, and RA router-announcement attacks are all blocked both against other containers and against the host.
• Outbound (IPv4/IPv6), host→container, DNAT port forwards, DHCP lease and IPv6 pass-through (/112 block routed to the container + ndppd on the external interface) are unaffected.

br_netfilter requirement

security.ipv6_filtering only works while the br_netfilter kernel module is loaded, and Incus does not load it itself — a container with the option refuses to boot without it, so after a host reboot every isolated container would fail to start. vps install therefore writes /etc/modules-load.d/br_netfilter.conf and loads the module, so it is present before Incus starts any container. Harmless no-op where the module is built into the kernel.

Security model

• Panel lives behind a random secret url_path; everything off-path returns a bare, headerless 404 (no fingerprint, no auth cost).
• Mutating actions are POST-only; sessions are 3-day HttpOnly+Secure+ SameSite=Lax cookies; a per-IP login rate limiter.
• Sessions are persisted in the DB (hashed tokens, so a copy of the database cannot be replayed): user sessions in sessions, admin sessions in admin_sessions. A panel restart does not log either side out. An admin password change still drops every other admin session.
• The dynamic CPU limit rule is panel-owned: a DB settings row edited on the admin overview, with no config.yaml key. Saving applies it at once, and vps install never resets it. A cap is only kept once its state row is written; if that write fails the pass lifts the cap again, because a cap the panel has no record of could never expire or be switched off.
• The panel daemon is unprivileged (see "Unprivileged panel" above): Incus access via group membership, only whitelisted commands via sudo.
• Containers are Incus-unprivileged with security.nesting=true.

Knowledge base

The operator can author Markdown articles in the admin panel (/knowledge); users open them read-only from a 知识库 / Knowledge base button on the machine card, which opens a large overlay (nearly fullscreen on phones). Content lives in the knowledge table and is rendered server-side by internal/markdown, a dependency-free renderer for a basic subset (headings, fenced code, inline code, bold/italic, links, lists, quotes, rules). It escapes HTML first and allows only http/https/mailto/#/ links, so articles cannot inject markup. A small script adds a copy button to each code block.

Bandwidth accounting

Per-container NIC counters come from Incus. A background goroutine in the panel samples every 60 s through one instances-list request and one bulk metrics request, then accumulates deltas into SQLite (bandwidth table). The panel reads totals from the DB — it never blocks on an exec for bandwidth.

Resource history

The same background sample writes one compact row per managed container and minute to resource_samples. It stores container state, CPU time and derived CPU percentage, memory usage (MemTotal - MemAvailable) and filesystem usage in MiB, process count, network counters, and disk I/O counters. Host resource history is deliberately not stored. Samples older than seven days are deleted automatically; the retention window is currently fixed and not configurable.

The admin overview defines CPU usage as the average of valid samples from the most recent five minutes. Memory and disk usage are the latest successful sample. The persisted history is intentionally ready for a future user-facing resource/time chart and selectable windows without adding live sampling to page requests.

vps install treats a failure to apply guest-side routed IPv6 on one running container as a warning and continues the host upgrade. This is necessary because users can replace the guest OS or network stack; the affected container's host route remains managed, while its guest-side address/default route may need manual repair. The explicit vps ipv6-reapply command still returns an error when any container could not be repaired.

Install / uninstall lifecycle

• uninstall.sh without --purge removes the software but keeps /etc/vpsmgr (config/db/certs) and /etc/haproxy (the static entry config plus every published generation), so a reinstall adopts the previous users/domains/settings. --purge removes those plus containers, the storage pool, the HAProxy package and the Incus package.
• install.sh detects an existing /etc/vpsmgr/config.yaml and adopts it (users/domains survive). 00-ip-ask.sh reuses a previously configured ipv6_subnet / subnet / user_ports instead of re-asking; a legacy slot_range is converted to user_ports once and the new key persisted.
• A pre-HAProxy install is migrated in place by the same install.sh run: 30-haproxy.sh installs and validates HAProxy, stops and removes the Traefik unit/binary/service account (keeping only its config file), republishes the domains from the database and removes the old per-domain YAML directory. net.traefik in an adopted config is read as net.haproxy and rewritten under the new name (see docs/configuration.md).
• install.sh --local-build prints the current git branch and waits 10 s (Ctrl-C to abort) before starting, and always rebuilds rather than reusing an installed stable binary. install.sh --update explicitly re-downloads the latest prebuilt release over an existing binary (conservative: on download/checksum failure the old binary is kept, never replaced by a surprise local build); a plain install.sh re-run skips an existing binary.

See ipv6.md for the optional IPv6 pass-through feature.