- host sysctl (20-network.sh): tcp_rmem/tcp_wmem max, rmem_max and
wmem_max all go from 4194304 to 8388608
- image builds (50/60/70/80/90 + eol-debian11): tcp_rmem/tcp_wmem max
bumped the same way
- autotune defaults (262144) and the min values are untouched
Co-authored-by: CommandCodeBot <noreply@commandcode.ai>
The send half printed a UUID-laden path, a standalone checksum and a paragraph
of hand-off instructions; the receive half printed the URL twice and explained
the outcome in three lines. Same information, fewer lines: one line to say what
is being produced and how big it may get, the version and result on one line,
one receive command with two lines about what it brings, and on the receiving
side one line that confirms the checksum and says which build made the archive.
Co-authored-by: CommandCodeBot <noreply@commandcode.ai>
The sender now offers exactly two, so a transfer cannot end up three-quarters
optimized:
default the storage-driver native stream, written as the pool stores it.
Nothing is decompressed and recompressed at any point.
--portable a tar of the files, zstd-compressed: slower and larger, but any
pool driver can restore it and an older receiving build
understands it.
--compression is gone from the sender: it was the knob that let a native stream
be wrapped in a second compression pass, which cost CPU to save almost nothing
over blocks the pool had already compressed. --optimized stays accepted and is
simply what happens without flags, so commands written against 1.10.x keep
working. The output is one line per fact now instead of three overlapping ones.
The receiving half keeps doing the adapting, because the sender may be any
version:
- it reads the sending pool's driver, the compression and the sending build
from the URL fragment and refuses an optimized stream only when it KNOWS
the driver differs — a stream that arrives without a stated driver (an
older sender, or a hand-written URL) is given the benefit of the doubt and
left to Incus, which reports the mismatch itself;
- it prints what it was handed, and which build produced it, before importing.
Co-authored-by: CommandCodeBot <noreply@commandcode.ai>
A transfer used to describe itself only by its two digests, so the receiving
host found out what it had when Incus either accepted it or did not. The
receiver now knows three more things before it fetches anything, carried in the
URL fragment alongside the checksums (a fragment is never sent to the server,
so the listener still learns nothing about what it serves):
- compression and whether the archive is an --optimized storage-driver stream;
- the sending pool's driver, because an optimized stream is driver-specific
and says nothing about itself: a zfs send stream cannot be unpacked into a
btrfs or dir pool. That check now runs before the metadata fetch, so a
mismatch is refused in a sentence instead of failing an import halfway
through;
- the sending build's version, printed at both ends so an operator can see
which build produced the archive.
parseTransferURL returns one struct rather than ten values, and the
compatibility decision is a pure function with its own tests: a command from an
older build has none of the new fields (zero value = not stated) and one from a
newer build carries extras an older receiver ignores, so plain tar transfers
keep working in both directions across versions. An --optimized archive does
need a receiver that understands it: older builds would try to untar a zfs
stream.
Co-authored-by: CommandCodeBot <noreply@commandcode.ai>
--optimized is Incus's native backup stream (zfs send / btrfs
subvol), not "compressed". It is the fastest path — 190 MiB in
2 s vs 468 MiB in 10 s for the same 175 MiB container — but the
file is pool-driver-specific and unreadable by any other host, so
it must stay opt-in and not be the default.
The default remains a portable zstd-compressed tar, which is small
enough (104 MiB for that same container) and still 2.5x faster than
the old gzip default. gzip is dropped from the CLI: it was the
slowest option, and none of the three paths (tar/gzip, tar/zstd,
optimized) uses it. The error message and usage text are updated.
Co-authored-by: CommandCodeBot <noreply@commandcode.ai>
The quota dialog has grown a second and a third form, and the plain Cancel
button sitting inside the first form's grid was easy to miss — and it is the
only way out. It now lives in the dialog's own footer at the bottom right,
uses the danger styling, and says what it does: 取消且不保存 / Cancel without
saving.
Co-authored-by: CommandCodeBot <noreply@commandcode.ai>
gzip is the slowest link in an export: on a single weak vCPU it streams at
~25-40 MB/s. Same container, same host: gzip 116 MiB in 27 s, zstd 104 MiB in
11 s — smaller and 2.5x faster. --compression none remains for incompressible
data or fast LAN links.
Co-authored-by: CommandCodeBot <noreply@commandcode.ai>
- the transfer URL is built from panel.display_ip (falling back to
public_ip), and display_ip may legitimately be a hostname — a host with one
could not send at all ("cannot build a transfer address from ...").
- the listener only embeds the value in a URL and the peer pins the
certificate by fingerprint, so an IP literal was never a real requirement;
the check now rejects only values that cannot survive a URL, and the error
points at both settings.
Co-authored-by: CommandCodeBot <noreply@commandcode.ai>
- IPv6Block treats a stored index of 0 as "no block": 0 is the block the
bridge gateway lives in, so serving it would put the gateway's own address
on the container. Pool-mode and V4-only accounts store 0 there.
- the v19 migration gets a test that fails its Go step on purpose and asserts
the schema rolls back whole — no column, no index, no version row — and that
the next start re-runs it, plus a second legacy account so the backfill is
proven over more than one row.
- pickIPv6Index is asserted never to hand out the gateway's block, even on an
empty database.
Co-authored-by: CommandCodeBot <noreply@commandcode.ai>
A prefix-mode container's /112 block index was sha256(username)[0:4], so the
suffix of its address was a global constant for a given name: "alice" got the
same block on every host running this panel. An address therefore identified
its owner to anyone who knew the scheme and could enumerate names — the
opposite of what the random container hostname is for.
The index now lives in users.ipv6_index (schema v19) and is random:
- accounts that predate the column are seeded with the value they have been
using, so no address moves and no container is touched;
- vps add picks a free index and retries internally, with the UNIQUE index on
the column as the backstop, so a name can no longer be refused for a reason
as opaque as "your address is taken" (the old checkIPv6BlockCollision). A
block containing the bridge gateway is skipped the same way.
- deleting an account and recreating the same name yields a fresh random index,
and so does a transfer: the value is per-host and stays out of the metadata.
Add launches a container before creating its account row, so within that window
the block can no longer be looked up by name: WireIPv6 and
ConfigureContainerIPv6 now take the block/address the caller already picked.
Blocks are also only meaningful in prefix mode, which the accessors now say.
Verified on a live prefix-mode host: a user created before the change kept her
address (panel and inside the container), a new user came up on a random block
end to end (guest address, ndppd rule, Incus route), deleting and recreating a
name produced a new random block, and a V4-only host still creates users with
no index at all.
Co-authored-by: CommandCodeBot <noreply@commandcode.ai>
A container moving to a host with no IPv6 has to lose the address it carried,
and on RHEL-family guests that means telling NetworkManager, not editing a file.
NetworkManager will not accept the change in the obvious order — a method of
"manual" requires an address, and an address is not allowed under a method of
"disabled" — so the guest still came up with the address from the host it left.
Setting the whole final state in one request is what it accepts.
Verified with an AlmaLinux 9 guest in both directions against a prefix-mode and
a pool-mode host: moved to the prefix host it came back up with one bridged NIC
carrying this host's IPv4 and IPv6 (which required re-enabling the IPv4 that the
pool-mode layout had disabled on that NIC); moved to the pool host it came up
pure IPv4 with ipv6.method disabled and no global address at all.
Co-authored-by: CommandCodeBot <noreply@commandcode.ai>
Unifies what a transfer means on the receiving host, and fixes the gap that
made the rule awkward to live with.
Receiving is now decided entirely by the receiving host's own semantics and
nothing else — the sending host's setup is irrelevant, and the metadata carries
no trace of it. Prefix-mode and IPv4-only hosts give the container the address
they give any other container of theirs. A pool-mode host gives it none: a pool
address is a per-container choice its operator makes, so an imported container
arrives as pure IPv4, on the single bridged NIC that layout uses, consuming
nothing from the pool.
That left a real hole in the panel, which is also fixed here: there was no way
to give a pool address to a container that did not have one, even with the pool
sitting empty, so a container imported into a pool-mode host could never get
IPv6 at all. The quota form now offers the free pool addresses for exactly the
containers that have none. Giving one back is deliberately not offered — an
address handed to a customer stays theirs until the account is deleted — and
the assignment reshapes the container (the IPv4 moves to a bridged eth1, the
address goes on a routed eth0) and wires it on the host, rolling the whole thing
back if any step fails.
Incus decides an update device by device against the set it is replacing, so
that reshape goes in two requests: adding eth1 while it already carries the IPv4
fails with "already defined on another NIC", because the eth0 being changed
still holds it. The NIC is added bare first, and the address arrives as a change
to a device that already exists.
Co-authored-by: CommandCodeBot <noreply@commandcode.ai>
A prefix-mode host and a pool-mode host lay a container out differently —
prefix mode puts the private IPv4 and the routed IPv6 on one bridged eth0, pool
mode puts the public /128 on a routed eth0 and the private IPv4 on a bridged
eth1 — so moving between them needs both halves of the layout to be rebuilt,
not just refilled.
On the host side, the import override now also names every NIC a vpsmgr
container can have and clears every device option this host's own spec does not
decide. It has to: the override can set device keys but never remove a device,
so an eth1 the archive brings and this host has no use for would otherwise be
validated against a network it does not belong to, and an option whose meaning
depends on the NIC's kind ("security.ipv4_filtering" on a routed NIC) fails
outright. Cleared options are the ones this host's spec sets anyway whenever the
mode matches, so the list only bites when the kind of NIC has actually changed.
The spare NIC disappears for good when the configuration is replaced wholesale
a moment later.
Inside the container, one more step was needed in the pool-to-prefix direction:
there the container's eth0 has no DHCP client at all, because on the host it
came from the private address lived on eth1. Without one it never asks for the
address reserved for it and comes up with no IPv4, so every port forward points
at nothing. Its bridged NIC is given back the ordinary setup (the RHEL path
re-enables the NetworkManager profile instead, as that is where its addresses
live).
Verified in both directions between the two test hosts, one installed in prefix
mode and one reinstalled in pool mode: a prefix-mode container moved to the pool
host came up with a routed eth0 carrying an address from that host's pool and a
bridged eth1 carrying its private IPv4, and a pool-mode container moved to the
prefix host came up with a single bridged eth0 carrying both, its old eth1 gone.
In both cases the data was intact, the addresses were the receiving host's own,
and the published SSH port answered through the host's DNAT rules.
Co-authored-by: CommandCodeBot <noreply@commandcode.ai>
The owner should notice nothing but their address, and that is now mostly
true: the quota, the deadline, their SSH keys, their sticky notes and their
init script travel with the archive in a small JSON file served next to it and
checked against its own digest, so nothing has to be retyped on a command line
or remembered from the old machine. The init script is written to the database
and deliberately not run: it belongs to a reinstall, and a migration must not
re-run whatever it did the first time.
The root password is no longer reset either. It lives on the disk, its owner
already knows it, and leaving it alone is the point of a migration that should
feel like nothing happened — verified by logging into the moved container with
the password set on the sending host before it moved.
An account whose deadline has passed is refused outright, with the command to
extend it: starting a container the panel has already locked and stopped is not
something a transfer should do, and an expiry in the past would otherwise be
recreated as a live account. The deadline travels as the absolute date it was,
so moving neither gains nor loses time.
The receiving host checks it has room before fetching anything: the metadata is
small and comes first, and the disk figure in it is the sending host's own, the
same number the export sized its temp file from.
The account's settings are no longer passed as flags. --cpu/--mem/--disk/
--bandwidth/--days on `receive` are gone, along with the code that could have
been used to shrink a disk on the way — quotas may exceed what the target has
room for (the pool is over-committed by design, as it is for `vps add`), but a
migration never reduces one.
Two more things the real daemon taught us, both about the same DHCP handshake:
- A container is handed its static IPv4 through a reservation keyed to its
hardware address, written when the instance is created. Changing that address
afterwards — which is what dropping the source's MAC did — leaves the
reservation naming the old one, and the container then comes up on a dynamic
address while every rule and record points at the static one. The fresh
address is therefore set as part of the import, via X-Incus-config, where
Incus will write the reservation around it. Confirmed on the wire: the
configured, configured-in-the-guest and reserved addresses now agree, and the
lease follows.
- That address arrives over DHCP a moment after the container is up, later than
Readiness means, so it is waited for rather than assumed, with one restart if
the server is still holding the address of the container that was replaced.
Verified on two hosts (Ubuntu 24.04 and 26.04, Incus 7.0.1, zfs): an account
with a key, notes, an init script and a deadline moved with all four intact,
the init script not run, the old root password still working, the transferred
quota in place, its assigned static IPv4 taken and reachable over its published
SSH port; an expired account refused on the sending side with nothing created;
and a failed import rolled back to no account, no container and no files.
Co-authored-by: CommandCodeBot <noreply@commandcode.ai>
Everything below comes from running the feature against two real hosts, where
the assumptions that read fine in the source turned out not to hold.
The receiving side now creates the account instead of replacing one. That is
both simpler and safer: the name must be free, the account is built by the
same code `vps add` uses — same IP, port block, IPv6 assignment, firewall
rules and panel row — and the disk of the container that produced is then
replaced by the transferred one. Nothing that already exists is touched, so
every failure is undone by deleting what was created, and the host ends up
exactly as it was. It also drops the whole staging/rename dance the previous
shape needed.
What the real daemon taught us:
- A direct-stream export (Accept: application/octet-stream) refuses a backup
name: no backup object is created, so the field must not be sent at all.
- Incus validates device configuration while it CREATES the instance, so the
archive's own devices — the source host's IPv6 — are rejected before any
later fix-up can run ("not within network incusbr0 subnet"). This host's
devices therefore go in as part of the import, via X-Incus-devices. Without
them the source's disk quota comes along too: an 8 GiB account arrived on a
host that had allotted 5.
- A static IPv4 is handed out through a DHCP reservation held by the current
lease, and Incus counts a static address as taken by ANY instance that
declares it, stopped ones included. The container being replaced therefore
has to be gone before the replacement starts, or the replacement is given a
dynamic address from the pool — and then nothing can reach it, because the
panel's records and the host's DNAT rules all name the static one.
- The re-key step appends this host's IPv6 address, routed block and gateway
neighbour to the container's networkd file, checking only for its own
values. A container that arrived from another host kept that host's address
as a second global one, so the stanza is now cleared first.
- An archive carries the source's volatile.<nic>.hwaddr, so importing the same
archive twice on one host failed outright with "MAC address ... already
defined on another NIC". Per-instance volatile keys are now dropped, like
the machine-id and SSH host keys before them.
Also from the field: the re-keyed container is checked to actually hold its
assigned IPv4, the quota travels in the printed receive command (--cpu/--mem/
--disk/--bandwidth/--days), and the receiving command's <user> must be free —
receiving onto a taken name is refused before the download rather than after.
Verified on two hosts (Ubuntu 24.04 and 26.04, Incus 7.0.1, zfs): a 316 MiB
container moved and took its data (files, setuid bits, modes, a custom user,
a 200 MiB file), a fresh hostname/machine-id/SSH host key, cleared
authorized_keys, the transferred quota, its assigned static IPv4 and only this
host's IPv6; reachable over its published SSH port. The same archive imported
twice into two accounts; --optimized (zfs-native) likewise; a wrong
certificate pin refused in under a second with nothing created; a mid-import
failure rolled back to no account, no container and no files; idle exit, SIGHUP
and a killed CLI all left no archive behind.
Co-authored-by: CommandCodeBot <noreply@commandcode.ai>
Two vpsmgr hosts that share nothing but a network can now hand a container's
root disk over: `vps transfer send <user>` exports the stopped container to a
single archive and serves it from a temporary listener, and
`vps transfer receive <url> <user>` fetches it, verifies it, and rebuilds the
container from it. It is what an operator needs to move a customer between two
machines in different datacenters, where the existing snapshot sharing cannot
reach: that feature is a copy-on-write clone inside one pool, so no byte ever
leaves the host.
Only the disk travels. Domains, SSH keys, quotas, expiry, bandwidth and
resource history are configuration this host owns and deliberately keeps to
itself -- the two hosts do not share admin keys -- so the receiving side
re-homes the container onto its own address, ports and quota instead of
adopting the sender's.
Decisions worth recording:
- The archive is an instance backup streamed with Accept:
application/octet-stream, so Incus sends the tarball straight down the
socket and nothing is staged in the storage pool first. Publishing an image
would have been the cleaner artefact but it materialises the whole rootfs
inside the pool, which fails on exactly the nearly-full hosts a migration
tends to involve.
- The transfer runs as root from the CLI and has no panel entry. It puts a
user's disk on the network; no panel request should be able to do that.
- The certificate pin and the archive checksum both ride in the URL fragment,
which HTTP clients never transmit, so the pasted command carries the whole
trust decision and the listener never learns either value.
- An import parks the container it replaces under a temporary name and brings
the new one into place only after it starts and provisions. A failure at any
point puts the original back, rather than leaving the user with nothing.
- The listener stops after five idle minutes and the archive is deleted on
every exit path, including SIGINT/SIGHUP during the export itself.
- lx: a deadline-free HTTP client for the transfer (the 60s cap would abort
any real transfer), plus BackupExport/BackupImport/ReplaceConfig/DiskUsage
- lx: ReplaceConfig replaces an imported container's configuration wholesale
while carrying the daemon's volatile.* keys through unchanged
- mgr: the export/import orchestration, including fresh machine-id and SSH
host keys (a copied rootfs otherwise makes two hosts claim one identity)
- mgr: a temporary HTTPS listener with one path, Range support for resuming a
dropped download, and a stalled-connection guard
- cli: `vps transfer send|receive`, live progress on both sides, space
checked against the container's actual size before a long export starts
- tests: pinning, checksum, resume, unknown-token, URL parsing, idle exit
Co-authored-by: CommandCodeBot <noreply@commandcode.ai>
A host whose RTC came up eight hours fast at boot -- systemd-timesyncd
corrected it seconds later -- took its very first resource sample before that
correction, so the row was written with a sample_minute eight hours in the
future. The sampler kept recording normally, but that single row won
MAX(sample_minute) forever, so every reader served the stale boot-time
snapshot: both panels showed all containers Stopped with no CPU or memory until
wall-clock caught up. Restarting does not help, because the row lives in the
database.
Reads cannot tell a mis-dated row from a legitimate one -- a clock jumped
forward and a machine powered off for hours look identical from inside -- so
the guard belongs on the read side, not the write side:
- LatestResourceSamples picks the newest sample at or before now rather than
the newest overall, so one future-dated row can no longer freeze the panels
- ResourceHistory, behind the user panel's 24h chart, skips future-dated
samples for the same reason
- the row is still stored, and the real sample overwrites it in place once
wall-clock reaches that minute, so the series keeps no gap or duplicate
- tested: the future sample is persisted but never served by either reader
Co-authored-by: CommandCodeBot <noreply@commandcode.ai>
Two reports, one of them plainly a bug of mine.
The cursor was never drawn inside vi. The renderer skipped it whenever the
alternate screen was active, on the theory that a full-screen program draws its
own -- but that screen is exactly where a cursor matters most: vi, less and htop
all run there, and an editor without one tells you nothing about where you are
typing. It is drawn whenever the program has not hidden it.
The other report was that Escape left the terminal deaf until the window was
clicked. That does not reproduce here -- a bare Chrome types straight through
after Escape, in vi and at a prompt -- but the symptom is unmistakably focus
leaving the panel's own element, and clicking only helps because the click hands
it back. Since this window is nothing but the terminal, it now takes the
keystroke itself when focus has drifted, and restores focus on the way. Whether
the browser really moves focus on Escape on macOS is still unconfirmed; this
makes the question moot for the user either way.
Typing in vi left the whole session underlined. The stream vi sends contains
xterm's modifyOtherKeys, which carries a private ">" prefix:
CSI > 4 ; m and CSI > 4 ; 2 m
A CSI sequence has four private markers — ? < = > — and the parser only knew
"?". The others were dropped on the floor, so their parameters leaked into the
final byte and ">4;m" was read as SGR 4 (underline on) with nothing left to
turn it off. All four markers are recognised now, and a private sequence is
never treated as SGR. The secondary device-attributes query (CSI > c) also gets
its own reply, so a program asking for it does not sit out its timeout.
Found by capturing the real byte stream from vi over a PTY and replaying it
through the emulator: the flags now come out clean, while a plain CSI 4 m still
turns underline on as it should.
Reported alongside it: after leaving insert mode with Escape the shell seemed
to stop accepting keys until the window was clicked. That does not reproduce
against a bare browser here — the same sequence types ":wq" straight through —
so the likely culprit is an input method leaving its composing flag set, which
would swallow every later key. The guard now trusts our own compositionstart/end
tracking instead of the browser's flag, and a committed composition still
reaches the shell.
The key bar carries only what a browser will not hand over: Ctrl+C, Ctrl+Z,
Ctrl+D, Escape, Tab and the arrows. It is a fallback, not the way to use the
terminal — typing works for everything else, and the bar deliberately does not
take focus so the keyboard keeps it. The arrow buttons ask the emulator for the
sequence, so they follow the mode a full-screen program requested just like the
real keys do.
Paste was broken on Windows and Linux. Ctrl+V fell through to the control-key
mapping and sent 0x16 with preventDefault, so the browser never got to fire a
paste event, and Shift+Insert did the same with the Insert sequence. Both, and
Ctrl+Shift+Insert, are now left to the platform; Cmd+V on macOS already worked.
The control keys were wrong in a way only running it showed: a control byte
cannot be written into an HTML attribute, so data-raw="\u0003" arrived as
six literal characters and the buttons typed them instead of sending them.
They carry a name now and the byte is looked up in JavaScript.
Verified against a real container: the bar's bytes (003, 032, 004, 033, ESC[A-D),
arrows inside less, Tab completion, Ctrl+V and Shift+Insert pasting, Enter
running a two-line paste, and Ctrl+C copying when something is selected.
Two things the key testing turned up.
Alt was wrong on macOS. Option+a arrives as the composed character "å", so
sending e.key put that in the shell instead of Meta-a. Both Alt and Ctrl now
take the letter from e.code (the physical key), falling back to e.key when a
browser does not report it; non-letters keep the plain ASCII mapping, which is
what makes Ctrl+[ an Escape.
The "bye" message was dead: the page never sends it, so the handler arm could
never run.
A browser reaching this panel is often a long way away, so the session is now
built to survive the link rather than be torn down by it.
Liveness first, because the old numbers were wrong: pings every 20s with a 60s
deadline. The previous setting was a single 15-minute timer, and since the
browser answers pings automatically it never fired for an idle user — the one
thing it did do was take fifteen minutes to notice a peer that had vanished.
- the shell outlives the socket. When the client goes away the PTY keeps running
for 120s with its output landing in a 256 KiB ring (with a notice if anything
is dropped). A client that comes back with its token resumes the same shell, so
a tunnel, a cell handover or a sleeping laptop no longer costs a running build.
The token travels in the first frame rather than the URL, is bound to the user,
and lives in sessionStorage — reloading resumes, closing the window lets go.
- the page has its own heartbeat: a browser cannot see protocol-level pings, so
the server also sends a small JSON one and the page reconnects with exponential
backoff and jitter after 45s of silence, up to eight attempts.
- a handshake that neither completes nor fails would otherwise stall the retry
loop for good, so an attempt that has not opened in 15s is abandoned.
- at the per-user cap the client is offered its slots back instead of being
refused, which is the only way to reclaim a window the user can no longer see.
Where there is room, the count is just a notice.
Three bugs the browser testing turned up, all of them real:
- Pty writes had no deadline, so a wedged Incus connection blocked the handler
forever — resume hung after any pause long enough to break that socket.
- a chunk whose write failed was neither delivered nor buffered.
- a session stayed resumable between Close and the end of its drain.
An operator can now turn the whole thing off with `vps config set panel.web_ssh
false`: the button is not rendered and the page, the script and the connection
all refuse. It defaults on, including for a config written before the setting
existed.
The terminal was a modal, which is the wrong container for it: a floating dialog
cannot be dragged to a larger size, moved to another screen or maximised, and
the terminal is exactly the thing that wants more room.
- the button moves up next to Start/Stop/Restart and is renamed Web SSH, which
is what people look for. It is styled apart from every other control on the
page on purpose: black key, green type, monospace, with a ">_" prompt in
front, so it reads as a remote shell at a glance.
- clicking it opens /webssh in its own browser window. That window is resizable
like any other, carries a Fullscreen button, and remembers the size it was
last dragged to so the next open comes back the same way. A blocked popup says
so instead of doing nothing.
- the overview page no longer loads the terminal engine at all; it only exists
in the shell window now.
Verified against a real container in the popup: prompt, typing, and a window
resize reaching the PTY (stty size moved from 26 99 to 16 86 when the terminal
area changed), which is the thing a modal could never do.
Adds a Web SSH-style terminal to the user panel, opened from the overview page.
It is deliberately not SSH. The container runs on the same host, so Incus' own
exec API with a PTY is the transport: no sshd, no keys to distribute, no network
path to arrange, and none of the credential handling a real SSH hop needs.
- lx gains an interactive session beside the fire-and-collect Exec: stdin stays
open, output streams as it is produced, and the window can be resized while the
shell runs. Exec and Attach now share the request-and-dial path.
- the panel serves /terminal over a WebSocket. That handshake is a GET, so the
panel's POST CSRF check never sees it, and SameSite does not cover it the way
it covers a form post — csrf.SameOrigin is what stands in for both. The handler
also takes the container from the session and never from the request, refuses
expired accounts, caps concurrent shells per user, and closes idle ones.
- the browser side is a small VT emulator, hand-written because the panel ships
as a single binary with no build step and a CSP that allows no third-party
scripts. It covers the usual CSI/SGR set, the alternate screen, scrollback,
UTF-8 including wide characters, and an IME via a hidden textarea so composed
text (Chinese, Japanese) reaches the shell. Terminal output is escaped, never
treated as markup.
Two protocol details were worked out against the running daemon rather than
assumed, and both are pinned by tests: an interactive exec returns only fds "0"
and "control" (stdout is folded back into "0"), and the resize message carries
its size as strings inside "args" — top-level width/height are silently ignored.
Checked against a real container: prompt, line wrapping, ANSI colour, CJK
output, less, a full-screen program (top), scrollback and window resize, with
the session ending and reporting its exit status when the shell exits.
Also fixes test isolation, which surfaced while running the suite on this host:
the panel and admin test servers used the default Incus socket, so tests that
drive /power, /reinstall and friends acted on whatever real instance shared a
fixture user's name — on this machine, restarting the real "alice" on every run.
Both now point the client at a socket that cannot exist. The suite also got
faster (panels 12s -> 7s) from no longer waiting on real timeouts.
User-panel knowledge base:
- a blank is now an exam-style underline instead of a box: no background, no
padding, and a height pinned to one line, so it no longer stretches the code
line it sits in. The placeholder is a clear yellow and the value keeps the
code's own colour;
- the width still follows the content, so a long pasted key extends the
underline while the rest of the line stays readable;
- the copy button is always present now, not just once every blank is filled: a
block may legitimately have optional or annotated blanks, and copying then
yields the text with those parts left empty;
- it starts breathing as soon as any blank in that block has content, and the
glow is actually visible (accent background plus a scale pulse) rather than
the button simply appearing;
- "copy all" moved up next to Close on the same row, and says what it does.
The blank styling also had to be scoped to .kb-article: the panel's generic
input[type=text] rule is more specific than a bare class, so it was winning and
painting the blanks as form fields. Caught by looking at the rendered page.
Admin panel: the batch form carried a paragraph of explanation per field.
Dropped the parentheticals and the "either way is fine" notes, leaving one
short hint on the username box.
Creating users one at a time does not scale when they are all carved out of the
same prepared machine, so the admin panel now takes a list:
- one username per line, a shared snapshot UUID, the operator keys to activate
on the new machines, and the resource quotas for the whole batch;
- the whole list is checked up front — syntax, duplicates, names already taken —
and a single problem creates nothing, so a rejected batch never leaves a
half-built account behind;
- the run happens in the background and the page polls for progress. It has to:
each user is a serial clone that takes tens of seconds, while the panel's
WriteTimeout is 60s, and a truncated response would also lose the one-time
passwords, which exist only in memory;
- only one batch runs at a time (creations are serialized by the manager
anyway) and the list is capped, so a pasted wall of text cannot churn the host.
Cloning is a new branch of Manager.Add rather than a parallel code path, so the
identity allocation (IPv4 slot, SSH port, user-port block, IPv6 block), the DB
row, the firewall and the rollback all stay in one tested place. Two clone-only
fixups are mandatory, not optional:
- the clone carries the source's /root/.ssh/authorized_keys, which would let the
owner of the source checkpoint log straight into every new machine, so it is
cleared even when no operator key was ticked;
- it also carries the source's machine-id, and a duplicate DUID makes dnsmasq
drop DHCPv6 leases.
The ticked operator keys reuse the existing admin_keys / admin_key_grants pair:
the batch just writes the grants the user would otherwise tick themselves, and a
failure there is reported next to that user instead of failing the container,
which is already usable at that point.
Checked end to end against a real Incus on the test host: a clone of another
user's snapshot came up with its own machine-id, hostname, IP and SSH port, with
the source's key gone and the operator key written as Vpsmgr-Operator. The host
was left exactly as found (container, nftables files and ruleset all restored).
Co-authored-by: CommandCodeBot <noreply@commandcode.ai>
A clone carries the source checkpoint's whole state, so its init script was
deliberately skipped — re-running it would duplicate whatever the checkpoint's
owner had already done. But someone importing another machine's image often
does want their own bootstrap to run on top of it, so make it a choice:
- the reinstall form gains a "运行初始化脚本" switch that follows the share-code
box: typing a UUID reveals it, clicking back to a plain image clears the code
and hides it again;
- the state is never remembered — it ships unticked, is forced off every time
the modal opens (which also defeats browser form-state restoration), and an
unticked box never reaches the request body;
- rolling back the user's own checkpoint ignores the switch: the state came
back from the checkpoint, not from an image;
- best-effort like a plain reinstall: a delivery failure warns and never fails
the rebuild, since the container is already usable at that point.
ReinstallFromShare takes the flag as a parameter rather than reading it from
anywhere global, so the caller stays the only thing that decodes the form.
Co-authored-by: CommandCodeBot <noreply@commandcode.ai>
An operator can now write a prompt inside a fenced code block, where the
reader fills in the machine-specific part:
REGINNAME=*#*#填写地域信息#*#*
- the hand-written renderer splits the fence body on the directive, escapes
the literal runs as before, and emits an inline text input whose placeholder
and node id both come from the label. The id is a hash of the label (plus an
occurrence counter for repeats), so it is stable across edits and reordering
with no server-side storage, and no value is ever emitted — the answer lives
only in the reader's browser;
- a block containing blanks keeps its copy button hidden until every blank in
that block is filled, then pulses to draw the eye. The copied text is rebuilt
by walking the DOM, because innerText cannot see an <input>'s value;
- a new "copy all" button copies the whole article as Markdown with the same
substitution. It is fence-aware: a directive in prose renders no input, so
substituting it would shift every answer after it;
- the admin editor gains a one-line mention of the syntax, and its preview
styles the blanks so the author sees what the reader will get.
Co-authored-by: CommandCodeBot <noreply@commandcode.ai>
The installed unit started HAProxy with the static entry config alone:
-Ws -f /etc/haproxy/haproxy.cfg
That file declares the frontends, the reject backends and the timeouts; the
per-domain backends and the routing maps live in the generation the panel
publishes, and the maps' paths point at it. So HAProxy came up healthy, bound
80/443 and answered 404 on :80 and closed every :443 connection — no managed
domain was reachable. `haproxy -c` on the entry config alone passes, which is
why the install-time check did not catch it.
- the unit now passes both files, the generation through "current" so a reload
picks up whatever the panel published last;
- 30-haproxy.sh validates exactly that command line (entry + generation) and
requires domains.cfg AND both maps before it considers the bootstrap
generation in place, so the pre-start check can no longer pass a setup that
serves nothing;
- tests: TestUnitLoadsBothConfigFiles parses the unit's own ExecStart and fails
if it does not pass two config files, and TestUnitCommandLineRoutesTraffic
runs that command line against a real HAProxy and asserts a registered host
reaches its backend. Both fail against the old unit.
Co-authored-by: CommandCodeBot <noreply@commandcode.ai>
Two problems with `panel.bandwidth_reset_day`, both around its interaction
with the quota period:
- The user panel hardcoded "至次月 N 号", which is wrong whenever the reset day
is still ahead of today: the period in force started on that day of the
PREVIOUS month, so it ends on that day of THIS one (reset day 22 on the 16th
ends on the 22nd of this month). mgr.BandwidthNextReset derives the real
moment, the page flags whether it lands in this calendar month, and the label
follows. A test also pins it against BandwidthPeriod, so the label and the
rollover can never disagree about the boundary.
- Changing the day DISCARDS every user's accumulated bandwidth totals. That is
not an explicit reset: the counters are keyed by a period derived from the
day, so moving the day moves the key and the next sampler pass sees a new
period. It stays that way (it is the period model), but it is no longer
silent: the key is classified ApplyDestructive, so `vps config set` shows a
confirmation naming the loss, repeats it on the --no-apply path (where the
loss lands at the next panel restart instead), and prints it after applying.
docs/configuration.md explains the mechanism and points at the admin panel's
per-user "重置流量统计" button for a deliberate reset.
TestApplyBandwidthPeriodChangeDiscardsTotals pins the discard so the warning
cannot quietly become wrong, and TestBandwidthResetDayIsDestructive keeps the
classification from regressing to a plain restart.
Co-authored-by: CommandCodeBot <noreply@commandcode.ai>
A host can sit on a pre-1.6 release (distro ndppd, old oneshot vps-ipv6.service)
for a long time before upgrading, so document and lock down how install.sh
converges it:
- the old ndppd/npd6 daemons are disabled and stopped, the unit is replaced by
the in-tree responder, and the legacy per-address kernel proxy_ndp entries
are swept on ipv6-reapply;
- /etc/vpsmgr/ndppd.conf is read AS-IS, so the nested `iface <bridge>` block
and its indentation written by the old panel must keep parsing. Adds
TestLoadRulesAcceptsPreResponderFile to a regression guard, since every
existing container would lose inbound IPv6 if that format ever diverged;
- the old sudoers grants disappear with the rewritten whitelist, and the
distro ndppd package is left installed but inert.
Co-authored-by: CommandCodeBot <noreply@commandcode.ai>
The panel now fronts user domains with HAProxy 3.4 LTS (official performance
packages) instead of Traefik: about 100 MB of RSS and 200 MB of disk saved for
a feature set that never used more than Host routing on :80 and SNI passthrough
on :443.
- hpx: publish the proxy configuration as immutable GENERATIONS. Every domain
mutation renders the whole set from the DB, validates it with the pinned
`haproxy -c`, and only then repoints the `current` symlink, so a rejected
candidate leaves the running proxy untouched and a stale domain can never
survive into a later generation. Routing is carried by two map files
(Host -> backend, SNI -> backend): per-domain frontend rules cannot work from
a second file because HAProxy rejects duplicate section names since 3.3 and
attaches bare directives to the section opened last.
- mgr: republish and reload (SIGUSR2, i.e. `systemctl reload`) after every
domain change, so a change takes effect without dropping established HTTP or
TLS connections. Rollback semantics (DB first, then publish) are unchanged.
- cfg,db: `net.traefik` is renamed to `net.haproxy` with identical values. A
pre-rename config.yaml key and the legacy VPSMGR_TRAEFIK env var are still
read (new name wins), and schema migration v18 moves the settings mirror from
`traefik` to `haproxy`, keeping a newer value when both rows exist.
- 30-haproxy.sh: install the pinned branch from the vendor repository and
migrate a Traefik host in one run — install and validate HAProxy first, then
remove Traefik's binary, unit, service account and dynamic rules (only its
config file is kept, inert). No fallback keeps both proxies alive.
- upgrade-haproxy.sh: follow the pinned branch to its newest patch, with a
configuration check and a graceful reload; `install.sh --update` runs it.
- tests: generation publish/rollback, map rendering, config-key and settings
migrations, plus real-binary tests (skipped unless
VPSMGR_TEST_HAPROXY_BIN is set) that run the generated configuration through
`haproxy -c` and start it in a network namespace.
Co-authored-by: CommandCodeBot <noreply@commandcode.ai>
The release body is the tag's own annotation, but the runner's checkout only
had the commit — no tag object — so `git cat-file -t` failed and every release
silently fell back to "fix some bugs" (v1.8.0 included). The publish job now
fetches tags, and the notes step fetches the tag ref explicitly before reading
it, so the annotation always makes it into the release body.
Co-authored-by: CommandCodeBot <noreply@commandcode.ai>
Addresses two review findings:
- the dynamic CPU limit applied a cap and only then saved the active state. A
failed save left the cap in place with no record, so it could never expire
and switching the rule off would not lift it — the container would stay
pinned to the reduced quota. The caps applied by a pass are now held back
until the state is written; if the write fails they are lifted again (each
back to the user's own quota) and the failure is reported. The quota writer
is behind a one-method interface so the failure path is covered by a test
that fails the state write with a trigger and asserts cap-then-rollback, plus
a test that a failed rollback is reported rather than swallowed
- migrations only compared the highest recorded version, so rows missing from
schema_migrations silently skipped a step and surfaced later as obscure "no
such column" failures. The recorded versions must now form the unbroken run
2..cur (v1 is the baseline, created unconditionally and never recorded) or
the database refuses to open
The third finding (a failed cross-container share install deleting the old
container before the new one is up) is left as is: the user confirms a
reinstall that discards the data, and the retry costs nothing — Delete treats
an already-missing container as success (verified against the live Incus
socket), so the user simply installs again.
Co-authored-by: CommandCodeBot <noreply@commandcode.ai>
- admin CPU card: the "what saving does depends on the checkbox" footnote is
gone; the state line says it instead, and the state/description text is
shorter
- user panel: the reinstall share field is just "或从分享的检查点安装" and its
note "将从该检查点恢复，不安装系统。"; the share warning no longer parenthesises
the contents ("拿到分享码的人可以读取该检查点内的全部数据，包括容器里的凭据。"),
the second confirmation is "确认删除 / 更早的 N 个检查点会被删除，无法恢复。",
the revoke prompt drops the repeated question, and the restore/reset prompts
lose their "…，确认？"/"（重置后无法恢复）" tails
- the share toast only mentions removed checkpoints when there were some
(previously "share code created (0 older checkpoints removed)")
- expiry: "账户已到期，机器已关机并锁定。" and "该账户已到期，仅可延期。"；the
"new deadline = max(now, current) + duration" footnote and "(0 = permanent)"
parentheticals are gone
- knowledge editor: label is a plain "Markdown: headings, lists, …" legend and
the empty preview is a dash instead of an instruction
- docs: the 1.8 sections (expiry, snapshot sharing, refquota, sessions, CPU
rule, knowledge base) are cut roughly in half
Co-authored-by: CommandCodeBot <noreply@commandcode.ai>
- with the rule disabled the parameter inputs are hidden entirely (they were
shown with values that had no effect), and a state line says which of the two
states the rule is in; ticking the box reveals the parameters again
- the parameter fields lost `required`: they are prefilled from the server, the
server validates every range, and a hidden-but-required input makes the
browser refuse to submit with a "not focusable" error
- the save message now matches what happened: "saved and applied now" only when
the rule is enabled, otherwise "saved — it is disabled, so no container is
capped" (nothing is applied, and any capped container was just restored)
- the button is a plain "Save"; the effect depends on the checkbox above it
Co-authored-by: CommandCodeBot <noreply@commandcode.ai>
User panel:
- the reinstall modal drops the system highlight as soon as a share code is
typed, so "nothing selected" is an explicit state that says the container is
rebuilt from a checkpoint rather than from an image; clearing the code
restores the default image, and clicking a tile drops a stale code instead of
letting it silently override the visible selection
- the time-machine actions (share/restore/delete/revoke, plus the sticky-notes
reset) confirm in an in-page dialog styled like the panel instead of
window.confirm(); publishing still asks twice when it would delete the older
checkpoints
Admin panel:
- the knowledge-base preview renders again: it posts a FormData body
(multipart/form-data) and ParseForm alone ignores multipart bodies, so the
server rendered the empty string and the preview box stayed blank
- the dynamic CPU limit rule moves back to the panel: an editable card on the
overview saves into the DB (the single source of truth) and applies on the
spot, restoring capped containers when it is switched off. The cpu_limit.*
config keys and their `vps config set` entries are gone, and `vps install`
no longer touches the rule, so an upgrade cannot reset it
- admin sessions are persisted (new admin_sessions table, hashed tokens) so a
panel restart no longer logs the operator out; a password change from the
panel or the CLI still invalidates every other session
Co-authored-by: CommandCodeBot <noreply@commandcode.ai>
- an expired account may still change its own panel password, and the admin
may still reset it — only the container stays locked. Both routes are
deliberately exempt from the read-only gate and both stay audited
- snapshot sharing is now OFF by default (new installs and upgrades alike):
`snapshots.share` defaults to false and the panel re-derives its DB mirror
from the config file at startup, so even a binary-only upgrade picks the
default up. No admin-panel UI for it; enable with `vps config set
snapshots.share true`
- publishing a checkpoint now asks twice when it would delete older
checkpoints (first the exposure, then the irreversible deletion)
- audit completeness: distinct domain.add/domain.update/domain.delete,
notes.update, and subject suffixes for snapshot.share.<checkpoint>,
reinstall.share.<owner> and quota.expire.<user>, with matching labels in
the audit viewer (knowledge.* labels added too)
Co-authored-by: CommandCodeBot <noreply@commandcode.ai>
- the publish job reads the tag's own message and uses it as the release
body: `git tag -a vX.Y.Z -m "..."` writes the notes shown on the release
page
- a lightweight tag carries no message, so the body falls back to
"fix some bugs"; GitHub still appends its auto-generated compare link
below whatever body is passed
- a signed tag's PGP block is stripped from the notes
- document the annotated-tag flow in docs/development.md
Co-authored-by: CommandCodeBot <noreply@commandcode.ai>
Port of dev-quota-expiry onto 1.8. The branch's other three commits are
already in main, so only this feature is new here; its migration is
renumbered to v16 (1.8 already uses v14/v15).
- users.expires_at (RFC3339 UTC, '' = permanent): the admin sets days at
creation and extends from the users table (72h/30d/90d/180d/custom/forever;
extend = max(now, current) + duration)
- the 60s loop force-stops expired containers (stop force:true) and disables
boot.autostart; the account is locked to read-only — the user panel refuses
every mutation (including the new snapshot-share/unshare routes) and the
admin panel allows only extend + delete
- the user panel shows a live countdown and a locked banner; the admin users
table dims expired rows and tags the name "(-Nd)"
- CLI: vps add --days N, vps quota --days N | --clear-expiry
- audit: quota.expiry (admin change) and quota.expire.<user> (auto stop)
- the container and its checkpoints are never deleted, so a checkpoint shared
before expiry stays installable by others (cloning a stopped container's
snapshot works — verified end to end)
Co-authored-by: CommandCodeBot <noreply@commandcode.ai>
- new `knowledge` table (schema v15) and an admin page (/knowledge) to create,
edit, preview and delete Markdown articles; the Markdown source is capped at
256 KiB and both title and content are required
- `internal/markdown`: a small, dependency-free Markdown renderer for a basic
subset (headings, fenced code, inline code, bold/italic, links, lists,
blockquotes, rules). It escapes HTML first and only allows http/https/mailto/
#/ links, so an article can never inject markup
- the admin editor previews through POST /knowledge-preview, which returns the
rendered HTML as JSON
- the user panel's machine card gains a "知识库 / Knowledge base" button that
opens a large overlay (nearly fullscreen on phones): every article is
embedded rendered, the list opens an article with a back button, and a copy
button is added to each code block
Co-authored-by: CommandCodeBot <noreply@commandcode.ai>
- `snapshots.share` and `cpu_limit.*` are now config.yaml fields managed with
`vps config set` (validated, shown by `vps config list/help`) instead of being
panel/DB-only settings; the admin panel now shows the CPU rule read-only and
the snapshot-sharing card is gone
- both apply immediately: `vps config set` writes the config file, mirrors the
value into the DB and applies it right away — no panel restart — the same
pattern as net.v4_forward. `vps install` refreshes the mirror from the file
- the CPU rule is read from the mirror (falling back to the config file) so the
enforcer and the read-only card pick up a change live; the share toggle too
- drop the now-unused admin routes/handlers, the DB-only rule setting and the
ConfigSet CPU form
Co-authored-by: CommandCodeBot <noreply@commandcode.ai>
- add a snapshot_share_enabled setting with an admin-overview toggle;
absent means ENABLED, so an upgraded host gets the feature on by default
- turning it off hides the user-side share/import entry points and makes
the manager refuse new shares and installs, while existing containers and
checkpoints are left untouched
- the stored codes are deliberately kept in the database (not deleted), so
re-enabling restores them for as long as their checkpoint still exists;
a code whose checkpoint has vanished is reported as invalid instead of
building a half-container, and deleting a checkpoint still removes its code
Co-authored-by: CommandCodeBot <noreply@commandcode.ai>
- a user can publish one checkpoint behind a random v4-UUID code; another
user enters that code in the reinstall dialog and gets a container cloned
copy-on-write from the snapshot (no image is built, so no extra disk until
it diverges), re-provisioned as their own: own IP/ports/disk quota and
network, a fresh hostname/root password/machine-id, the source's
authorized_keys cleared, and no init script run
- publishing deletes the older checkpoints (a clone pins its origin, so they
could never be restored to again); the panel warns with the exact count.
One share per container; deleting the checkpoint/container or reinstalling
revokes it, and a code for the container itself is handled as a restore
- clone via POST /1.0/instances with source.type=copy, then delete the old
container and rename the clone into place; new lx.CloneFromSnapshot /
lx.RenameInstance and a shared instanceSpec() so a clone is configured
exactly like a fresh container
- new snapshot_shares table (schema v14) and /snapshot-share /
/snapshot-unshare panel routes plus the time-machine and reinstall UI
- also fix a refused restore (a checkpoint pinned by a clone) so the
container is always left running, and recognise the "subsequent internal
snapshot(s)" refusal message alongside the existing one
- also give new containers a HARD disk limit: the pool carries
volume.zfs.use_refquota so a volume's usage counts the blocks it
references (including inherited image/snapshot blocks); existing
containers are untouched
Co-authored-by: CommandCodeBot <noreply@commandcode.ai>
