Merge remote-tracking branch 'origin/dev-ndp'

# Conflicts:
#	src/internal/admin/handlers.go
This commit is contained in:
bluebluesoda committed 2026-09-04 08:33:56 +00:00
commit c9b18ffee5
29 files changed
+992 -208

No files matched your search

+1
View File
@@ -7,6 +7,7 @@ Technical documentation for vpsmgr. All docs are in English.
| [architecture.md](architecture.md) | System design: components, storage, network, security, traffic, image, install/uninstall lifecycle |
| [configuration.md](configuration.md) | Full reference for `/etc/vpsmgr/config.yaml` |
| [ipv6.md](ipv6.md) | IPv6 pass-through design: deterministic addresses, prefix support, routing, installer flow |
| [ipv6-ndp-responder.md](ipv6-ndp-responder.md) | Why prefix mode replaced ndppd with an in-tree NDP responder, design hardening, and what to test at scale |
| [development.md](development.md) | Building, testing, releasing, conventions |
See also [`AGENTS.md`](../AGENTS.md) at the repo root — short version aimed at AI coding agents.
+209
View File
@@ -0,0 +1,209 @@
# In-tree NDP responder for prefix mode
This document explains **why** vpsmgr replaced the distro `ndppd` daemon with
its own small NDP responder in **prefix mode**, and what to verify during
wide-scale testing. It is the design rationale for the change on the
`dev-ndp` branch.
## The problem
In prefix mode each container owns a global /112 block described in
[ipv6.md](ipv6.md). For the outside world to reach a container, the provider's
upstream router sends a Neighbor Solicitation (NS) for the container address
on the host's external link; the host must answer with a Neighbor
Advertisement (NA) so the router learns which MAC to forward packets to.
Two mechanisms answered those NSes before:
- The kernel's `proxy_ndp` — but it only answers **single /128 addresses** and
ignores prefix-cover queries, so it cannot serve a whole routed /112.
- **ndppd** — proxied ND for a whole prefix. This was vpsmgr's choice.
### The silent failure
A class of upstream IPv6 providers **drops NAs whose source address is
link-local**. Both `proxy_ndp` and ndppd emit their NA with the host's
link-local source address (`fe80::…`). The router therefore discards the reply,
the container appears unreachable from outside, and — because this happens on
the data plane with no error surfaced anywhere — it fails **silently**:
containers reach *out* fine, but inbound connectivity to the container's
public address returns nothing.
This was confirmed on a real provider with a controlled experiment: binding
the same address directly on `eth0` (so the kernel answers with the *global*
address as the NA source) made the container reachable from outside
immediately.
## The chosen fix
vpsmgr now ships its own minimal NDP responder in `internal/ndp` that answers
Neighbor Solicitations on the external interface with an NA whose **source is
the advertised global address** — exactly what these providers expect. It runs
as a root systemd unit (`vps-ipv6.service`, prefix mode only).
### Wire-format correctness
The responder is small (< 300 lines) but the invariants that matter to a
router are all handled and locked down by unit tests:
- Correct ICMPv6 checksum over the IPv6 pseudo-header.
- Hop limit = 255 (a hard NDP requirement).
- NA flags `0xe0000000` (Router+Solicited+Override) for a normal reply;
`0xc0000000` (Solicited cleared) for a DAD answer.
- Target Link-Layer Address option carrying the host `eth0` MAC.
- DAD solicitations (source `::`) answered as a multicast `ff02::1` NA.
### Why not just patch ndppd?
The ideas of "one project-owned daemon" vs "reuse a mature daemon" were weighed
(the latter is the original report's weaker point). The decision to keep an
in-tree responder was deliberate: ndppd **cannot be configured** to source its
NA from a global address, so no config-level workaround exists; patching it
would mean maintaining a private fork of a C daemon and re-integrating upstream
forever. A ~300-line, unit-tested Go responder that we own and can reason about
was judged the better long-term trade-off. The cost is maintenance ownership.
## Reading the rules without restarting
`ndppd` needed a daemon restart on every config change. The in-tree responder
re-reads `/etc/vpsmgr/ndppd.conf` while it runs:
- At most once per second while idle.
- On-time on the arrival of an NS (by checking the file's mtime), so the very
first solicitation after an `add`/`del` is answered instead of dropped.
As a result `vps add` / `vps del` never restart `vps-ipv6.service`.
## Design hardening (beyond the original report)
Two issues found while reviewing became part of the accepted design.
### A. Confining the root responder's trust surface
`vps-ipv6.service` runs as **root** (it needs a raw `AF_PACKET` socket). Its
input, `/etc/vpsmgr/ndppd.conf`, lives in a directory the unprivileged panel
user writes (the panel updates it on every add/del). Without additional
checks, a compromised panel process could write an arbitrary IPv6 rule and
turn the root responder into an NDP spoofer for any external address.
The responder therefore takes the operator's **routed prefix** as an `allowed`
set and silently drops any rule not contained in it (checked at both reload
points). A compromised rules-file writer can no longer make the host answer for
addresses outside vpsmgr's own prefix.
### B. No hot-looping on a broken interface
`Restart=always` + a responder that exits on error would retry every second
forever. The common trigger is an `ext_if` with no 6-byte Ethernet MAC (a
tunnel, a bare tun, some cloud virtual NICs, a bond without a slave). We added
an `ethernetMAC` pre-check at **install time**: if prefix mode is selected but
the interface cannot answer NDP, the responder is not enabled and `vps install`
warns clearly, instead of letting the service spin in a log loop. The command
itself also exits cleanly (no-op) when IPv6 is disabled or the mode is not prefix.
## What changed on the branch
| File | Change |
|---|---|
| `internal/ndp/proxy.go` | the responder (raw socket, NS→NA, `allowed` confinement) |
| `internal/ndp/proxy_test.go` | wire-format + `filterRules` tests |
| `main.go` | `vps ipv6-proxy` command, prefix `vps-ipv6.service` unit, `ethernetMAC` install check; removed ndppd daemon sudoers grants |
| `internal/mgr/ipv6.go` | `writeNDPPD` rewritten **atomically** (unique temp file + `os.Rename`); all ndppd daemon management (restart/stop/link/pid/alive) removed |
| `internal/mgr/mgr.go` | `Add`/`Reinstall`: configure the guest's IPv6 **before** publishing the NDP rule, so the first SYN never races a half-configured guest |
| `internal/mgr/routed_ipv6.go` | guest script: pin the `fe80::1` gateway neighbor, RA `UseGateway=false`, a **local** /112 route, and a `vpsmgr-ipv6.service` in-container helper that re-applies the local route after a network restart |
| `install.sh` | removed the `ndppd` install/enable block (no longer used in prefix mode) |
| `50-image.sh`, `80-debian-dev-image.sh` | bake `UseGateway=false` into the image's `[IPv6AcceptRA]` |
The real **data-plane** change is confined to prefix mode and to the NDP
handling around it. Login, snapshots, quotas, pool mode, IPv4 and the panel are
untouched.
## Relationship to the existing isolation layer
The responder is not the first NDP-related mechanism in vpsmgr; the container
isolation layer predates it by far (introduced with the Incus port,
`security.port_isolation` + `security.ipv4/ipv6_filtering`, in
`src/internal/lx/lx.go`). These two address **different directions** and do not
conflict:
| Layer | Direction | Defence target | Mechanism |
|---|---|---|---|
| Isolation Layer | **east-west** on the bridge | isolate containers, stop ARP/NDP spoofing, protect the host's own neighbour cache | Incus bridge input rules (`security.ipv4/v6_filtering`, `port_isolation`) |
| This responder | **north-south** on the upstream NIC (`eth0`) | let strict upstreams resolve container /112 addresses | host-side in-tree raw-socket NA replies for inbound NS |
The responder listens on the host's upstream interface for NS aimed at the
routed /112 block; the isolation rules filter *guest*-originated neighbour
packets inside the bridge. Different directions, so they do not overlap in
normal prefix-mode topologies.
### The interaction point: gateway-neighbour pinning
`security.ipv6_filtering` has a deliberate side effect: it drops the **guest's
own NDP lookup** for the bridge gateway `fe80::1` (that reply claims an address
the guest does not own, so the rule discards it). Without a dance-around, a
guest cannot resolve its default route's neighbour — the earlier fix's
filtering breaks the new requirement of egress through the host.
The responder work treats this explicitly, not as a conflict but as a
constraint to plan around: the guest pins `fe80::1` to the bridge MAC (`ip -6
neigh replace \u2026 nud permanent` in the runtime script, and `[Neighbor]` in
networkd), with `UseGateway=false` guarding against a competing RA default
route. So the new code **relies on** the isolation layer's behaviour and
coexists with it — one closes the neighbour-spoofing hole, the other makes
strict upstreams able to reach the block through the existing design.
## Notes on a few deliberate decisions
- **File name kept as `ndppd.conf`.** The format (bare `rule <cidr> {}`) is
ndppd-compatible and unchanged; the name is retained so the panel's Writable
directory layout and the config it renders stay stable. The documentation
makes clear the daemon itself is gone.
- **Atomic write / privilege.** The old buggy author was a *permission* one:
the panel failed to write `ndppd.conf` when an older `vps install` (running
as root) had left it root-owned. The atomic-rename rewrite only depends on the
writable parent directory, never on overwriting a possibly root-owned target.
- **`UseGateway=false`.** Guards against an RA installing a competing dynamic
default route in the guest. It is baked into the images *and* restored
idempotently by the runtime script.
- **Gateway neighbor pinned.** `security.ipv6_filtering` drops the guest's
link-local NDP lookup for the bridge's `fe80::1`; pinning the neighbor to the
bridge MAC restores the guest's ability to send without weakening isolation.
- **Local /112 route.** A normal /112 route only says where packets *go*;
a `local` route makes every address in the block bindable, so a service on
`block::4` is reachable without adding each address separately.
## What to verify in wide-scale testing
The unit tests (build, vet, and `go test ./...`) all pass. Prefix-mode IPv6
data flow still needs real-router validation because a test box without an
upstream IPv6 prefix cannot exercise the NDP path end-to-end. On hosts with a
routed prefix:
1. **Inbound reachability is global-source.** From an external probe (e.g.
Globalping, same as the existing `check-ipv6-support.sh`), verify the
container's public address and at least one non-primary address in its /112
(e.g. `block::4`) reply. This is the exact thing that broke before.
2. **Outbound still works** — `curl -6` from inside the container.
3. **First-connection race.** Immediately after `vps add`, ping the new
container from outside; the first NS must be answered (the on-time mtime
reload), i.e. no "first ping lost".
4. **add/del churn without a service restart** — `journalctl -u vps-ipv6.service`
shows no restarts across several `vps add`/`del`.
5. **Self-healing on boot** — reboot the host; `vps-ipv6.service` starts,
re-runs `ipv6-reapply`, and inbound still works.
6. **Broken `ext_if` fails loudly** — on a host whose ext_if has no MAC,
`vps install` (prefix mode) must *warn and not enable* the responder,
never spin in a start loop.
7. **No competing daemon** — `ndppd.service` is inactive (it is disabled at
install); a competing link-local NA would break the whole fix.
8. **Guest gateway neighbor + local route** — inside a container,
`ip neigh` shows `fe80::1` pinned to the bridge MAC and
`ip -6 route show table local` contains the /112.
## References
- Base behaviour and prefix/pool modes: [ipv6.md](ipv6.md).
- This document's rationale: the original provider bug report plus the
`dev-ndp` branch review (issues A and B).
- The isolation layer this responder coexists with: `src/internal/lx/lx.go`
(`nicIsolation`) and `security.ipv6_filtering`.
-12
View File
@@ -274,18 +274,6 @@ export VPSMGR_IPV4_SUBNET="${VPSMGR_IPV4_SUBNET:-}"
export VPSMGR_IPV6_MODE="${VPSMGR_IPV6_MODE:-}"
export VPSMGR_IPV6_POOL="${VPSMGR_IPV6_POOL:-}"
# NDP proxy for IPv6 pass-through. Prefix mode: each container owns a /112
# block and ndppd relays neighbor discovery on the upstream interface for it.
# Pool mode: kernel proxy_ndp handles each /128 (no ndppd needed). Only
# needed in prefix mode; small, no data-plane involvement.
if [[ "${VPSMGR_IPV6_MODE:-}" == "prefix" && -n "${VPSMGR_IPV6_SUBNET:-}" ]]; then
echo
echo "===== installing ndppd (IPv6 NDP proxy) ====="
apt-get update -qq 2>/dev/null || true
DEBIAN_FRONTEND=noninteractive apt-get install -y -qq ndppd
systemctl enable ndppd.service >/dev/null 2>&1 || true
fi
for step in 00-check 10-incus 20-network 30-traefik 40-panel 50-image; do
echo
echo "===== $step ====="
+1 -1
View File
@@ -75,7 +75,7 @@ systemctl enable ssh
# deterministic block would hand a fresh container that useless address.
if [ -f /etc/systemd/network/eth0.network ]; then
sed -i "s/^DHCP=true\$/DHCP=ipv4/" /etc/systemd/network/eth0.network
printf "\n[IPv6AcceptRA]\nUseOnLinkPrefix=false\nUseRoutePrefix=false\nUseAutonomousPrefix=false\nDHCPv6Client=no\n" >> /etc/systemd/network/eth0.network
printf "\n[IPv6AcceptRA]\nUseGateway=false\nUseOnLinkPrefix=false\nUseRoutePrefix=false\nUseAutonomousPrefix=false\nDHCPv6Client=no\n" >> /etc/systemd/network/eth0.network
fi
# vpsmgr TCP tuning: BBR congestion control, larger autotuned buffers, window
# scaling on, and no slow-start reset after idle (better for long-lived SSH and
+1 -1
View File
@@ -80,7 +80,7 @@ systemctl enable ssh
# host, so the parent prefix must not be treated as on-link
if [ -f /etc/systemd/network/eth0.network ]; then
sed -i "s/^DHCP=true$/DHCP=ipv4/" /etc/systemd/network/eth0.network
printf "\n[IPv6AcceptRA]\nUseOnLinkPrefix=false\nUseRoutePrefix=false\nUseAutonomousPrefix=false\nDHCPv6Client=no\n" >> /etc/systemd/network/eth0.network
printf "\n[IPv6AcceptRA]\nUseGateway=false\nUseOnLinkPrefix=false\nUseRoutePrefix=false\nUseAutonomousPrefix=false\nDHCPv6Client=no\n" >> /etc/systemd/network/eth0.network
fi
# --- Go 1.26.7 (manual install, arch-aware, checksum-verified) ---
GO_VERSION=1.26.7
+2 -2
View File
@@ -927,8 +927,8 @@ func TestAdminOverviewColorUI(t *testing.T) {
`name="color" value="#e11d48"`, `name="color" value="#3b82f6"`,
`name="color" value="#c026d3"`, `name="color" value="#db2777"`,
`style="--ubtn:#16a34a;color:#fff"`,
`button.neutral { --ubtn: #1c1e21;`, // neutral light default
`class="primary neutral" id="cfBtn"`, // create user: no blue
`button.neutral { --ubtn: #1c1e21;`, // neutral light default
`class="primary neutral" id="cfBtn"`, // create user: no blue
`class="primary neutral" id="adminPassBtn"`, // change admin pw: no blue
} {
if !strings.Contains(html, want) {
+55 -43
View File
@@ -75,31 +75,31 @@ type hostView struct {
// userView is one row of the admin user table.
type userView struct {
Name string
Name string
// Color is the operator-assigned accent color ("" = default). It colors
// the bold username and the "login panel" button for quick user spotting.
Color string
State string
Color string
State string
// Status is the persistent lifecycle state (ready/creating/reinstalling/
// failed). Non-ready states are shown to the operator so a crashed
// Add/Reinstall is visible instead of looking like a healthy user.
Status string
Ports string // full user-port block, e.g. 10700-10799 (tooltip)
PortsShort string // compact form, e.g. 107xx
SSHPort string
QuotaCPU string
QuotaMem string
QuotaDisk string
BandwidthGB int // monthly bandwidth quota GiB, 0 = unlimited
CPUUse string
MemUse string
DiskUsed string
UpGB string
DownGB string
BWTotal string // up+down this month, GB — used for table sorting
IPv6 string
Procs int64 // live process count (0 when stopped)
ProcsLimit string // per-container pids.max cap, e.g. "4096"
Status string
Ports string // full user-port block, e.g. 10700-10799 (tooltip)
PortsShort string // compact form, e.g. 107xx
SSHPort string
QuotaCPU string
QuotaMem string
QuotaDisk string
BandwidthGB int // monthly bandwidth quota GiB, 0 = unlimited
CPUUse string
MemUse string
DiskUsed string
UpGB string
DownGB string
BWTotal string // up+down this month, GB — used for table sorting
IPv6 string
Procs int64 // live process count (0 when stopped)
ProcsLimit string // per-container pids.max cap, e.g. "4096"
}
func (s *Server) buildPageData(msg, errMsg string) pageData {
@@ -158,26 +158,26 @@ func (s *Server) loadUsers(d *pageData) {
for _, st := range statuses {
u := st.User
vs = append(vs, userView{
Name: u.Name,
Color: u.Color,
State: st.State,
Status: u.Status,
Ports: mgr.UserPorts(u.StartPort, cfg.PortsPerUser),
PortsShort: mgr.UserPortsShort(u.StartPort),
SSHPort: strconv.Itoa(u.SSHPort),
QuotaCPU: mgr.FormatCPU(u.CPU),
QuotaMem: strconv.Itoa(u.MemMB) + " MiB",
QuotaDisk: strconv.Itoa(u.DiskGB) + " GiB",
BandwidthGB: u.BandwidthQuotaGB,
CPUUse: st.CPUUse,
MemUse: st.MemUse,
DiskUsed: st.DiskUsed,
UpGB: st.UpGB,
DownGB: st.DownGB,
BWTotal: st.BWTotal,
IPv6: st.IPv6,
Procs: st.Procs,
ProcsLimit: lx.DefaultProcessesLimit,
Name: u.Name,
Color: u.Color,
State: st.State,
Status: u.Status,
Ports: mgr.UserPorts(u.StartPort, cfg.PortsPerUser),
PortsShort: mgr.UserPortsShort(u.StartPort),
SSHPort: strconv.Itoa(u.SSHPort),
QuotaCPU: mgr.FormatCPU(u.CPU),
QuotaMem: strconv.Itoa(u.MemMB) + " MiB",
QuotaDisk: strconv.Itoa(u.DiskGB) + " GiB",
BandwidthGB: u.BandwidthQuotaGB,
CPUUse: st.CPUUse,
MemUse: st.MemUse,
DiskUsed: st.DiskUsed,
UpGB: st.UpGB,
DownGB: st.DownGB,
BWTotal: st.BWTotal,
IPv6: st.IPv6,
Procs: st.Procs,
ProcsLimit: lx.DefaultProcessesLimit,
})
}
d.Users = vs
@@ -399,10 +399,12 @@ func (s *Server) handleUserAdd(w http.ResponseWriter, r *http.Request) {
return
}
_ = s.db.AddAuditLog("000+"+name, "user.create")
// The password is always auto-generated and shown once.
// The password is always auto-generated and shown once. The panel address
// is taken from the request's own Host (see panelURL), so it matches
// whatever origin the operator actually used to reach the admin panel.
cred := "user: " + res.User.Name +
"\npassword: " + res.Password +
"\npanel: " + s.cfg.PanelURL("/"+s.cfg.Panel.URLPath)
"\npanel: " + s.panelURL(r, "/"+s.cfg.Panel.URLPath)
// Carry the username as flash data so the modal can offer "log in as".
s.redirectModalData(w, r, s.p(""), cred, res.User.Name)
}
@@ -536,7 +538,7 @@ func (s *Server) handleResetPanelPass(w http.ResponseWriter, r *http.Request) {
s.redirect(w, r, s.p(""), "error: "+err.Error())
return
}
panel := s.cfg.PanelURL("/" + s.cfg.Panel.URLPath)
panel := s.panelURL(r, "/"+s.cfg.Panel.URLPath)
_ = s.db.AddAuditLog("000+"+name, "passwd.reset")
s.redirectModal(w, r, s.p(""), s.t(r, "new_panel_password", name, pass, panel))
}
@@ -624,6 +626,16 @@ func (s *Server) handleLoginAs(w http.ResponseWriter, r *http.Request) {
http.Redirect(w, r, userPrefix+"/", http.StatusFound)
}
// panelURL renders the user-panel address as the browser sees it: https plus
// the request's own Host (not the configured listen port). This keeps the
// address shown in admin modals correct when the panel is reached through a
// reverse proxy on a custom domain or a different port. The scheme is always
// https — the panel only serves TLS and its session cookie is Secure, so it
// cannot function over plain http regardless of the transport in between.
func (s *Server) panelURL(r *http.Request, prefix string) string {
return "https://" + r.Host + prefix
}
// parseMem parses a memory string ("512" or "1G") into MiB, mirroring the CLI.
func parseMem(s string) (int, error) {
s = strings.TrimSpace(s)
+4 -4
View File
@@ -28,9 +28,9 @@ func Ensure(certPath, keyPath, ip string) error {
}
serial, _ := rand.Int(rand.Reader, new(big.Int).Lsh(big.NewInt(1), 128))
cn := ip
if net.ParseIP(ip) == nil {
cn = "localhost"
}
if net.ParseIP(ip) == nil {
cn = "localhost"
}
tmpl := x509.Certificate{
SerialNumber: serial,
Subject: pkix.Name{CommonName: cn},
@@ -41,7 +41,7 @@ func Ensure(certPath, keyPath, ip string) error {
DNSNames: []string{cn},
}
if parsed := net.ParseIP(ip); parsed != nil {
tmpl.IPAddresses = []net.IP{parsed}
tmpl.IPAddresses = []net.IP{parsed}
}
der, err := x509.CreateCertificate(rand.Reader, &tmpl, &tmpl, &key.PublicKey, key)
if err != nil {
+1 -1
View File
@@ -97,4 +97,4 @@ func (d *DB) SetAdminKeyGrants(userID int64, ids []int64) error {
}
}
return tx.Commit()
}
}
+1 -1
View File
@@ -119,4 +119,4 @@ func TestAdminKeyGrants(t *testing.T) {
if len(granted) != 0 {
t.Fatalf("after clear = %d grants, want 0", len(granted))
}
}
}
+1 -1
View File
@@ -83,4 +83,4 @@ func (d *DB) ActiveSSHKeys(userID int64) ([]SSHKey, error) {
out = append(out, k)
}
return out, rows.Err()
}
}
+1 -1
View File
@@ -91,4 +91,4 @@ func TestSSHKeysCascadeOnUserDelete(t *testing.T) {
if len(keys) != 0 {
t.Fatalf("keys survived user delete: %+v", keys)
}
}
}
+1 -1
View File
@@ -26,4 +26,4 @@ func (d *DB) SetStickyNotes(userID int64, data string) error {
_, err := d.sql.Exec(`INSERT INTO sticky_notes(user_id, data) VALUES(?, ?)
ON CONFLICT(user_id) DO UPDATE SET data=excluded.data`, userID, data)
return err
}
}
+1 -1
View File
@@ -71,4 +71,4 @@ func TestStickyNotesRoundTrip(t *testing.T) {
if bobNotes, _ := d.GetStickyNotes(bob.ID); bobNotes != "" {
t.Errorf("bob inherited notes: %q", bobNotes)
}
}
}
-1
View File
@@ -5,7 +5,6 @@ import (
"database/sql"
"fmt"
"math/big"
)
type User struct {
+1 -1
View File
@@ -220,4 +220,4 @@ func TestKeyMarkersSafe(t *testing.T) {
if UserKeyMarker == AdminKeyMarker {
t.Fatal("user and admin markers must be distinct")
}
}
}
+38 -61
View File
@@ -7,6 +7,7 @@ import (
"net"
"os"
"os/exec"
"path/filepath"
"sort"
"strconv"
"strings"
@@ -298,26 +299,13 @@ func (m *Manager) enableProxyNDP() error {
const ndppdConfPath = "/etc/vpsmgr/ndppd.conf"
const ndppdConfLink = "/etc/ndppd.conf"
// linkNDPPDConf points the daemon's default config path at vpsmgr's rendered
// file. Pinned sudo command (no wildcards), idempotent.
func (m *Manager) linkNDPPDConf() error {
_, err := su.Run("/bin/ln", "-sf", ndppdConfPath, ndppdConfLink)
return err
}
// unlinkNDPPDConf removes the root-owned symlink (pinned sudo rm). A missing
// file is not an error.
func (m *Manager) unlinkNDPPDConf() error {
_, err := su.Run("/bin/rm", "-f", ndppdConfLink)
return err
}
// ndppdConf renders /etc/ndppd.conf: one `rule <block>::/112` per container
// under a `proxy <ext_if>` section, so upstream neighbor solicitations for any
// address in a container's block are relayed to the Incus bridge (the container
// answers for the addresses it binds). `add` / `drop` let a single user be
// added or removed without racing the DB transaction in Add/Del. Empty when
// IPv6 is disabled or no container has a block.
// under a `proxy <ext_if>` section, so the in-tree NDP responder knows which
// /112 blocks to answer for on the external link. `add` / `drop` let a single
// user be added or removed without racing the DB transaction in Add/Del.
// Empty when IPv6 is disabled or no container has a block. The format is kept
// ndppd-compatible (each rule line is a bare `rule <cidr> {`), even though the
// daemon is no longer used in prefix mode.
func (m *Manager) ndppdConf(add, drop string) (string, error) {
if !m.cfg.IPv6Enabled() {
return "", nil
@@ -362,62 +350,51 @@ func (m *Manager) ndppdConf(add, drop string) (string, error) {
return b.String(), nil
}
// restartNDPPD (re)starts the ndppd daemon after a config write. ndppd 0.2.x
// has no live reload (SIGHUP terminates it), so a restart is required and its
// liveness is verified afterwards. The distro init script can wedge in a stale
// "active (exited)" state, so a failed start falls back to launching the
// daemon directly. All calls go through the sudoers whitelist.
func (m *Manager) restartNDPPD() error {
if _, err := exec.LookPath("ndppd"); err != nil {
return fmt.Errorf("ndppd is not installed (install.sh installs it when IPv6 is enabled)")
}
_, _ = su.Run("/usr/sbin/service", "ndppd", "restart")
if m.ndppdAlive() {
return nil
}
_, _ = su.Run("/usr/sbin/service", "ndppd", "start")
if m.ndppdAlive() {
return nil
}
// Last resort: start the daemon directly, bypassing the init script.
_, _ = su.Run("/usr/bin/pkill", "-x", "ndppd")
_ = os.Remove("/var/run/ndppd.pid")
if _, err := su.Run("/usr/sbin/ndppd", "-d", "-p", "/var/run/ndppd.pid"); err != nil {
return err
}
return nil
}
// ndppdAlive reports whether an ndppd process is running.
func (m *Manager) ndppdAlive() bool {
out, err := exec.Command("pgrep", "-x", "ndppd").Output()
return err == nil && len(out) > 0
}
// writeNDPPD renders the config for the current container set (plus/minus one
// container) and restarts the daemon. When no container has IPv6 routing the
// daemon is stopped, so a stale config can never misroute.
// container) and atomically rewrites the file. The in-tree NDP responder
// re-reads the file while it runs (vps add/del never restarts it), so the
// rewrite must be atomic: a unique temp file in the (vps-writable) config dir
// is renamed over the target, so the responder never reads a half-written rule
// set and the unprivileged panel never depends on overwriting a possibly
// root-owned old file (rename checks the parent dir, not the target). When no
// container has IPv6 routing the file is removed, so a stale rule can never
// misroute.
func (m *Manager) writeNDPPD(add, drop string) error {
conf, err := m.ndppdConf(add, drop)
if err != nil {
return err
}
if conf == "" {
_, _ = su.Run("/usr/sbin/service", "ndppd", "stop")
_ = os.Remove(ndppdConfPath)
_ = m.unlinkNDPPDConf()
return nil
}
if err := os.WriteFile(ndppdConfPath, []byte(conf), 0o644); err != nil {
tmp, err := os.CreateTemp(filepath.Dir(ndppdConfPath), ".ndppd.conf.*")
if err != nil {
return err
}
// Point the daemon's default config path at the rendered file BEFORE the
// restart, or ndppd exits with "Failed to load configuration file
// '/etc/ndppd.conf'" and every WireIPv6/UnwireIPv6 fails.
if err := m.linkNDPPDConf(); err != nil {
tmpPath := tmp.Name()
removeTemp := true
defer func() {
if removeTemp {
_ = os.Remove(tmpPath)
}
}()
if err := tmp.Chmod(0o644); err != nil {
_ = tmp.Close()
return err
}
return m.restartNDPPD()
if _, err := tmp.WriteString(conf); err != nil {
_ = tmp.Close()
return err
}
if err := tmp.Close(); err != nil {
return err
}
if err := os.Rename(tmpPath, ndppdConfPath); err != nil {
return err
}
removeTemp = false
return nil
}
// WireIPv6 registers a container's /112 with the NDP proxy so its addresses
+11 -11
View File
@@ -166,11 +166,11 @@ func TestIPv6ContainerScript(t *testing.T) {
"DHCP=ipv4", // DHCPv6 off
"s/^DHCP=true$/DHCP=ipv4/", // flips the baked DHCP=true
"n\\[IPv6AcceptRA\\]", // heals mangled old configs (awk regex)
"2602:fada:6*", // stale on-link route flush
"2602:fada:6*", // stale on-link route flush
"ip -6 route flush cache",
"ipv6.method manual", // RHEL: NM owns the IPv6 stack
"ipv6.method manual", // RHEL: NM owns the IPv6 stack
"ipv6.addresses 2602:fada:6::2bd8:6c9:1/128", // deterministic /128
"ipv6.gateway fe80::1", // bridge's fixed link-local gateway
"ipv6.gateway fe80::1", // bridge's fixed link-local gateway
} {
if !strings.Contains(script, want) {
t.Errorf("script missing %q:\n%s", want, script)
@@ -249,15 +249,15 @@ func TestPoolContainerScript(t *testing.T) {
}
// Both stacks: the branch selection and the address/gateway/DHCP markers.
for _, want := range []string{
"nmcli", // RHEL path present
"ipv6.method manual", // RHEL: NM owns the IPv6 stack
"nmcli", // RHEL path present
"ipv6.method manual", // RHEL: NM owns the IPv6 stack
"ipv6.addresses " + addr + "/128", // public /128 bound on eth0
"ipv6.gateway fe80::1", // routed-NIC gateway
"ipv4.method disabled", // v4 lives on eth1 only
"systemd-networkd", // Debian path present
"Address=" + addr + "/128", // Debian: static bind
"Gateway=fe80::1", // Debian: default route
"DHCP=ipv4", // eth1 DHCPv4 on both stacks
"ipv6.gateway fe80::1", // routed-NIC gateway
"ipv4.method disabled", // v4 lives on eth1 only
"systemd-networkd", // Debian path present
"Address=" + addr + "/128", // Debian: static bind
"Gateway=fe80::1", // Debian: default route
"DHCP=ipv4", // eth1 DHCPv4 on both stacks
} {
if !strings.Contains(script, want) {
t.Errorf("script missing %q:\n%s", want, script)
+5 -2
View File
@@ -138,7 +138,10 @@ func (m *Manager) SaveAdminKeys(in []SSHKeyInput) ([]db.AdminKey, error) {
if err != nil {
return nil, err
}
type pending struct{ name, key string; active bool }
type pending struct {
name, key string
active bool
}
seen := map[string]bool{}
kept := map[int64]bool{}
var toAdd []pending
@@ -284,4 +287,4 @@ func applySSHKeysScript(userKeys []string, adminKeys []db.AdminKey) string {
b.WriteString("' >> \"$AUTH\"\n")
}
return b.String()
}
}
+3 -3
View File
@@ -27,8 +27,8 @@ func TestParsePublicKey(t *testing.T) {
{"ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFEZ", "ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFEZ", "", true},
// invalid
{"", "", "", false},
{"ssh-ed25519 AAAA", "", "", false}, // body too short
{"ssh-ed25519 AAAA!!notbase64!!", "", "", false}, // bad body chars
{"ssh-ed25519 AAAA", "", "", false}, // body too short
{"ssh-ed25519 AAAA!!notbase64!!", "", "", false}, // bad body chars
{"ssh-ed25519 AAAA bogus\n; rm -rf /", "", "", false}, // shell chars are not base64 but the regex body is greedy over base64... see below
{"random text", "", "", false},
{"ssh-rsa", "", "", false},
@@ -172,4 +172,4 @@ func TestSaveSSHKeysUpdatesAndDeletes(t *testing.T) {
if len(active) != 0 {
t.Errorf("deactivated key still active: %v", active)
}
}
}
+14 -9
View File
@@ -487,15 +487,18 @@ func (m *Manager) Add(name string, opt AddOptions) (*Result, error) {
}
}
} else {
if err := m.WireIPv6(name); err != nil {
cleanup()
return nil, fmt.Errorf("wire ipv6: %w", err)
}
// Host-routed peer IPv6 (no L2 discovery / MITM between containers).
// Host-routed peer IPv6 (no L2 discovery / MITM between containers):
// bring the guest's /128 + local /112 up FIRST, then publish the NDP
// rule. Publishing last means an outside client's first SYN never
// races the guest still half-configured during `vps add`.
if err := m.ConfigureContainerIPv6(name, ""); err != nil {
cleanup()
return nil, fmt.Errorf("config container ipv6: %w", err)
}
if err := m.WireIPv6(name); err != nil {
cleanup()
return nil, fmt.Errorf("wire ipv6: %w", err)
}
}
u, err := m.db.CreateUserFull(name, hash, ip, idx, sshPort, startPort, opt.CPU, opt.MemMB, opt.DiskGB, opt.BandwidthGB, db.StatusCreating, poolAddr)
if err != nil {
@@ -1315,14 +1318,16 @@ func (m *Manager) Reinstall(name, image string) (string, error) {
}
}
} else {
if err := m.WireIPv6(u.Name); err != nil {
rollback()
return "", fmt.Errorf("wire ipv6: %w", err)
}
// Configure the guest's IPv6 first, then publish the NDP rule, so the
// outside world only learns the /112 once the container is ready.
if err := m.ConfigureContainerIPv6(u.Name, ""); err != nil {
rollback()
return "", fmt.Errorf("config container ipv6: %w", err)
}
if err := m.WireIPv6(u.Name); err != nil {
rollback()
return "", fmt.Errorf("wire ipv6: %w", err)
}
}
// User-defined init script (if any): run it detached inside the container,
// last of all, so it sees the full network. Best-effort — the container is
+1 -1
View File
@@ -15,7 +15,7 @@ func TestValidateName(t *testing.T) {
// Must end with a letter, so digit- and hyphen-terminated names are out.
"", "1alice", "1", "Alice", "ALICE", "alice_", "alice.",
"-alice", "alice-", "a-", "-a", "al-ice-", "al ice", "alice@x",
"al.bob", "alice1", "a1", "vps12345678", "a1b2", strings.Repeat("a", 32), strings.Repeat("a", 31)+"-",
"al.bob", "alice1", "a1", "vps12345678", "a1b2", strings.Repeat("a", 32), strings.Repeat("a", 31) + "-",
}
for _, n := range valid {
if err := ValidateName(n); err != nil {
+109 -7
View File
@@ -2,6 +2,9 @@ package mgr
import (
"fmt"
"net"
"os"
"path/filepath"
"strings"
"vpsmgr/internal/cfg"
@@ -69,9 +72,14 @@ func (m *Manager) ipv6ContainerScriptFor(ipv6, prefix string) (string, error) {
if ipv6 == "" {
return "", nil
}
bridgeMAC := m.bridgeMAC()
if prefix == "" {
return m.poolContainerScript(ipv6)
}
localBlock, err := ipv6LocalBlock(ipv6)
if err != nil {
return "", err
}
flush := fmt.Sprintf(`for r in $(ip -6 route show dev eth0 | awk '{print $1}'); do
case "$r" in
%s*) ip -6 route del "$r" dev eth0 2>/dev/null || true ;;
@@ -79,6 +87,12 @@ func (m *Manager) ipv6ContainerScriptFor(ipv6, prefix string) (string, error) {
done
ip -6 route flush cache 2>/dev/null || true`, prefix)
script := fmt.Sprintf(`set -e
# GATEWAY_MAC is the Incus bridge's MAC: the container's default route is
# fe80::1 (the bridge gateway), but security.ipv6_filtering drops the guest's
# NDP lookup for it, so we pin the neighbor. LOCAL_BLOCK is the container's
# /112; it must be a local route for services to bind any address in it.
GATEWAY_MAC=%q
LOCAL_BLOCK=%q
# Wait for systemd to be ready before touching it: right after boot the
# /run/systemd/private bus socket may not exist yet, and systemctl then fails
# with "Failed to connect to system scope bus". Bounded, cheap.
@@ -99,18 +113,40 @@ if command -v nmcli >/dev/null 2>&1 && ! systemctl is-active systemd-networkd >/
ipv6.addresses %s/128 \
ipv6.gateway fe80::1 2>/dev/null || true
[ -n "$CONN" ] && nmcli con up "$CONN" >/dev/null 2>&1 || true
[ -n "$GATEWAY_MAC" ] && ip -6 neigh replace fe80::1 lladdr "$GATEWAY_MAC" nud permanent dev eth0 2>/dev/null || true
else
CFG=/etc/systemd/network/eth0.network
mkdir -p /etc/systemd/network
[ -f "$CFG" ] || { printf '\n[Match]\nName=eth0\n' > "$CFG"; }
changed=0
# Rebuild the [IPv6AcceptRA] section from scratch — the only way to heal
# both the mangled single-line residue buggy old versions wrote and sections
# missing any option. Canonical contents: the parent prefix is never on-link
# and never a route, no SLAAC address is generated, and the RA's Managed flag
# must not start a DHCPv6 client (a dynamic address would fall outside the
# routed block, which ipv6_filtering drops).
if ! grep -qs '^DHCPv6Client=no$' "$CFG" 2>/dev/null; then
# must not start a DHCPv6 client. UseGateway=false keeps an RA from installing
# a competing dynamic default route.
if ! grep -qs '^UseGateway=false$' "$CFG" 2>/dev/null || \
! grep -qs '^DHCPv6Client=no$' "$CFG" 2>/dev/null; then
awk 'BEGIN{s=0} /^n\[IPv6AcceptRA\]/{next} /^\[IPv6AcceptRA\]$/{s=1;next} s&&/^\[/{s=0} !s' "$CFG" > "$CFG.new" && mv "$CFG.new" "$CFG"
printf '\n[IPv6AcceptRA]\nUseOnLinkPrefix=false\nUseRoutePrefix=false\nUseAutonomousPrefix=false\nDHCPv6Client=no\n' >> "$CFG"
cat >> "$CFG" <<RA
[IPv6AcceptRA]
UseGateway=false
UseOnLinkPrefix=false
UseRoutePrefix=false
UseAutonomousPrefix=false
DHCPv6Client=no
RA
changed=1
fi
# Pin the gateway neighbor in networkd too (security.ipv6_filtering blocks
# the guest's own NDP lookup for the bridge's fe80::1).
if [ -n "$GATEWAY_MAC" ] && ! grep -qs '^LinkLayerAddress='"$GATEWAY_MAC"'$' "$CFG" 2>/dev/null; then
awk 'BEGIN{s=0} /^\[Neighbor\]$/{s=1;next} s&&/^\[/{s=0} !s' "$CFG" > "$CFG.new" && mv "$CFG.new" "$CFG"
cat >> "$CFG" <<NEIGH
[Neighbor]
Address=fe80::1
LinkLayerAddress=$GATEWAY_MAC
NEIGH
changed=1
fi
# Statically bind the /128. A reinstall deletes the container but Incus's
@@ -118,14 +154,32 @@ else
# recreated container a dynamic address instead — binding the /128 directly
# makes IPv6 independent of that.
if ! grep -qs '^Address=%s/128$' "$CFG" 2>/dev/null; then
printf '\n[Address]\nAddress=%s/128\n' %q >> "$CFG"
cat >> "$CFG" <<ADDR
[Address]
Address=%s/128
ADDR
changed=1
fi
# Static default route via the bridge's fixed link-local gateway: with RA
# off-link/route-prefix off there is no RA-provided default, so without this
# the container cannot reach out (only inbound works via the host's proxy).
if ! grep -qs '^Gateway=fe80::1$' "$CFG" 2>/dev/null; then
printf '\n[Route]\nDestination=::/0\nGateway=fe80::1\n' >> "$CFG"
cat >> "$CFG" <<GW
[Route]
Destination=::/0
Gateway=fe80::1
GW
changed=1
fi
# Make the whole routed /112 local to the guest: a normal route only says
# where packets go; a local route makes every address in the block bindable,
# so services on block::4 etc. are reachable without adding each address.
if ! grep -qs '^Destination='"$LOCAL_BLOCK"'$' "$CFG" 2>/dev/null; then
cat >> "$CFG" <<LOCALR
[Route]
Destination=$LOCAL_BLOCK
Type=local
LOCALR
changed=1
fi
# Drop DHCPv6: the dynamic address it would assign is outside the routed
@@ -137,11 +191,59 @@ else
if [ "$changed" = 1 ]; then
systemctl restart systemd-networkd || true
fi
[ -n "$GATEWAY_MAC" ] && ip -6 neigh replace fe80::1 lladdr "$GATEWAY_MAC" nud permanent dev eth0 2>/dev/null || true
fi
%s`, ipv6, ipv6, ipv6, ipv6, flush)
# Make the complete routed /112 local inside the guest for every image stack
# (including NetworkManager), and re-apply it after a network restart via a
# one-shot unit.
ip -6 route replace local "$LOCAL_BLOCK" dev eth0
mkdir -p /etc/systemd/system
cat > /etc/systemd/system/vpsmgr-ipv6.service <<EOF
[Unit]
Description=vpsmgr routed IPv6 local prefix
After=network-online.target
Wants=network-online.target
[Service]
Type=oneshot
ExecStart=/usr/sbin/ip -6 route replace local $LOCAL_BLOCK dev eth0
RemainAfterExit=yes
[Install]
WantedBy=multi-user.target
EOF
systemctl daemon-reload || true
systemctl enable vpsmgr-ipv6.service >/dev/null 2>&1 || true
systemctl restart vpsmgr-ipv6.service >/dev/null 2>&1 || true
%s`, bridgeMAC, localBlock, ipv6, ipv6, ipv6, flush)
return script, nil
}
// ipv6LocalBlock returns the /112 the container's primary address belongs to.
func ipv6LocalBlock(ipv6 string) (string, error) {
ip := net.ParseIP(ipv6)
if ip == nil || ip.To16() == nil || ip.To4() != nil {
return "", fmt.Errorf("invalid IPv6 address %q", ipv6)
}
mask := net.CIDRMask(blockBits, 128)
return (&net.IPNet{IP: ip.To16().Mask(mask), Mask: mask}).String(), nil
}
// bridgeMAC returns the Incus bridge's MAC (used to pin the guest's fe80::1
// gateway neighbor), or "" when the bridge has no Ethernet address.
func (m *Manager) bridgeMAC() string {
path := filepath.Join("/sys/class/net", m.cfg.Incus.Bridge, "address")
b, err := os.ReadFile(path)
if err != nil {
return ""
}
mac, err := net.ParseMAC(strings.TrimSpace(string(b)))
if err != nil || len(mac) != 6 {
return ""
}
return mac.String()
}
// poolContainerScript renders the guest-side network config for a pool
// container, on whichever stack the image ships:
//
+297
View File
@@ -0,0 +1,297 @@
// Package ndp implements the small IPv6 neighbour responder vpsmgr needs for
// routed prefixes. Linux's proxy_ndp and ndppd answer with a link-local
// source address. Some providers reject that response and only accept a
// neighbour advertisement whose source is the advertised global address, so
// vpsmgr emits the advertisement directly on the external Ethernet link.
package ndp
import (
"bufio"
"encoding/binary"
"errors"
"fmt"
"log"
"net"
"os"
"strings"
"time"
"golang.org/x/sys/unix"
)
const (
ethernetHeaderLen = 14
ipv6HeaderLen = 40
icmpv6NS = 135
icmpv6NA = 136
etherTypeIPv6 = 0x86dd
// Router, solicited and override. This matches the advertisement emitted
// by a directly-configured IPv6 address on Linux.
naFlags = uint32(0xe0000000)
)
// Run listens for IPv6 Neighbor Solicitations on external and emits a
// Neighbor Advertisement for targets covered by the CIDR rules in configPath.
// allowed, when non-nil, is the operator's routed prefix: the responder only
// ever answers for a target inside it (and ignores any rule file entry outside
// it). This confines the root raw-socket listener to the operator's own address
// space, so a compromised writer of the rules file cannot turn the host into an
// NDP spoofer for arbitrary external addresses.
//
// The rules file is reread at most once per second while idle, and its mtime is
// checked when an NS arrives. This makes an add/del take effect for that very
// first solicitation without restarting this long-running process.
func Run(configPath, external string, allowed *net.IPNet) error {
iface, err := net.InterfaceByName(external)
if err != nil {
return fmt.Errorf("find external interface %s: %w", external, err)
}
if len(iface.HardwareAddr) != 6 {
return fmt.Errorf("external interface %s has no Ethernet MAC", external)
}
fd, err := unix.Socket(unix.AF_PACKET, unix.SOCK_RAW, int(htons(etherTypeIPv6)))
if err != nil {
return fmt.Errorf("open IPv6 packet socket: %w", err)
}
defer unix.Close(fd)
bind := &unix.SockaddrLinklayer{Ifindex: iface.Index, Protocol: htons(etherTypeIPv6)}
if err := unix.Bind(fd, bind); err != nil {
return fmt.Errorf("bind IPv6 packet socket to %s: %w", external, err)
}
// A timeout lets us reload the rules and observe a clean shutdown signal
// without needing a second control socket.
tv := unix.NsecToTimeval(int64(time.Second))
if err := unix.SetsockoptTimeval(fd, unix.SOL_SOCKET, unix.SO_RCVTIMEO, &tv); err != nil {
return fmt.Errorf("set packet socket timeout: %w", err)
}
mac := [6]byte{}
copy(mac[:], iface.HardwareAddr)
var rules []net.IPNet
var rulesAt time.Time
var rulesModTime time.Time
var lastRuleError time.Time
packet := make([]byte, 4096)
for {
if time.Since(rulesAt) >= time.Second {
next, loadErr := loadRules(configPath)
if loadErr != nil {
// Removing the file is the normal representation of an empty
// container set. Clear the old rules immediately, otherwise a
// deleted container would remain advertised until this process
// restarted. A malformed/read-failed file is different: keep the
// last known-good rules while the writer finishes an update.
if errors.Is(loadErr, errNoRules) || errors.Is(loadErr, os.ErrNotExist) {
rules = nil
rulesAt = time.Now()
rulesModTime = time.Time{}
continue
}
// A config write is atomic from the daemon's point of view in
// normal operation, but tolerate the brief empty/truncated
// window and keep the last known-good rules.
if time.Since(lastRuleError) >= 10*time.Second {
log.Printf("IPv6 NDP rule reload: %v", loadErr)
lastRuleError = time.Now()
}
} else {
rules = filterRules(next, allowed)
rulesAt = time.Now()
if info, statErr := os.Stat(configPath); statErr == nil {
rulesModTime = info.ModTime()
}
}
}
n, _, recvErr := unix.Recvfrom(fd, packet, 0)
if recvErr != nil {
if recvErr == unix.EAGAIN || recvErr == unix.EWOULDBLOCK || recvErr == unix.EINTR {
continue
}
return fmt.Errorf("receive IPv6 packet: %w", recvErr)
}
if n < ethernetHeaderLen+ipv6HeaderLen+24 || len(rules) == 0 {
continue
}
frame := packet[:n]
if binary.BigEndian.Uint16(frame[12:14]) != etherTypeIPv6 || frame[20] != 58 {
continue
}
// ICMPv6 starts immediately after the fixed IPv6 header. Extension
// headers do not occur on Neighbor Solicitations.
icmp := frame[ethernetHeaderLen+ipv6HeaderLen:]
if len(icmp) < 24 || icmp[0] != icmpv6NS || icmp[1] != 0 {
continue
}
// A new rule can be written immediately before the first external NS,
// while the one-second periodic reload is still waiting. Notice an
// atomic rename here so that first NS is answered instead of being
// silently dropped and forcing the client to retry.
if info, statErr := os.Stat(configPath); statErr == nil {
if !info.ModTime().Equal(rulesModTime) {
if next, loadErr := loadRules(configPath); loadErr == nil {
rules = filterRules(next, allowed)
rulesAt = time.Now()
rulesModTime = info.ModTime()
} else if errors.Is(loadErr, errNoRules) || errors.Is(loadErr, os.ErrNotExist) {
rules = nil
rulesAt = time.Now()
rulesModTime = time.Time{}
}
}
} else if errors.Is(statErr, os.ErrNotExist) && !rulesModTime.IsZero() {
rules = nil
rulesAt = time.Now()
rulesModTime = time.Time{}
}
target := net.IP(append([]byte(nil), icmp[8:24]...))
if !matches(target, rules) {
continue
}
if err := sendAdvertisement(fd, iface.Index, mac, frame, target); err != nil {
return fmt.Errorf("send NDP advertisement for %s: %w", target, err)
}
}
}
func loadRules(path string) ([]net.IPNet, error) {
f, err := os.Open(path)
if err != nil {
return nil, err
}
defer f.Close()
var rules []net.IPNet
s := bufio.NewScanner(f)
for s.Scan() {
fields := strings.Fields(s.Text())
if len(fields) < 2 || fields[0] != "rule" {
continue
}
_, network, err := net.ParseCIDR(fields[1])
if err != nil || network.IP.To4() != nil {
continue
}
rules = append(rules, *network)
}
if err := s.Err(); err != nil {
return nil, err
}
if len(rules) == 0 {
return nil, fmt.Errorf("%w: %s", errNoRules, path)
}
return rules, nil
}
var errNoRules = errors.New("no IPv6 rules")
// filterRules drops any rule not contained in allowed. When allowed is nil
// every rule is kept (tests / unconstrained mode).
func filterRules(rules []net.IPNet, allowed *net.IPNet) []net.IPNet {
if allowed == nil {
return rules
}
out := rules[:0]
for _, r := range rules {
if r.IP != nil && allowed.Contains(r.IP) {
out = append(out, r)
}
}
return out
}
func matches(target net.IP, rules []net.IPNet) bool {
for i := range rules {
if rules[i].Contains(target) {
return true
}
}
return false
}
func sendAdvertisement(fd, ifindex int, sourceMAC [6]byte, ns []byte, target net.IP) error {
frame, dstMAC, err := buildAdvertisement(sourceMAC, ns, target)
if err != nil {
return err
}
addr := &unix.SockaddrLinklayer{Ifindex: ifindex, Halen: 6, Protocol: htons(etherTypeIPv6)}
copy(addr.Addr[:], dstMAC[:])
return unix.Sendto(fd, frame, 0, addr)
}
func buildAdvertisement(sourceMAC [6]byte, ns []byte, target net.IP) ([]byte, [6]byte, error) {
var emptyMAC [6]byte
if len(ns) < ethernetHeaderLen+ipv6HeaderLen+24 {
return nil, emptyMAC, fmt.Errorf("short neighbor solicitation")
}
var dstMAC [6]byte
copy(dstMAC[:], ns[6:12])
srcIP := net.IP(ns[22:38])
dstIP := srcIP
if srcIP.Equal(net.IPv6zero) {
// DAD solicitations have no source address and must be multicast.
dstIP = net.ParseIP("ff02::1")
copy(dstMAC[:], []byte{0x33, 0x33, 0x00, 0x00, 0x00, 0x01})
}
target16 := target.To16()
if target16 == nil {
return nil, emptyMAC, fmt.Errorf("invalid IPv6 neighbor target %q", target)
}
icmp := make([]byte, 32)
icmp[0] = icmpv6NA
// icmp[1] is the code, which is zero.
flags := naFlags
if srcIP.Equal(net.IPv6zero) {
// The solicited flag must be clear for a DAD response.
flags = 0xc0000000
}
binary.BigEndian.PutUint32(icmp[4:8], flags)
copy(icmp[8:24], target16)
icmp[24] = 2 // Target Link-Layer Address option.
icmp[25] = 1 // One 8-byte option unit.
copy(icmp[26:32], sourceMAC[:])
ipv6 := make([]byte, ipv6HeaderLen)
ipv6[0] = 0x60
binary.BigEndian.PutUint16(ipv6[4:6], uint16(len(icmp)))
ipv6[6] = 58
ipv6[7] = 255
copy(ipv6[8:24], target16)
copy(ipv6[24:40], dstIP.To16())
pseudo := make([]byte, 40)
copy(pseudo[0:16], target16)
copy(pseudo[16:32], dstIP.To16())
binary.BigEndian.PutUint32(pseudo[32:36], uint32(len(icmp)))
pseudo[39] = 58
binary.BigEndian.PutUint16(icmp[2:4], checksum(append(pseudo, icmp...)))
frame := make([]byte, 0, ethernetHeaderLen+len(ipv6)+len(icmp))
frame = append(frame, dstMAC[:]...)
frame = append(frame, sourceMAC[:]...)
frame = append(frame, 0x86, 0xdd)
frame = append(frame, ipv6...)
frame = append(frame, icmp...)
return frame, dstMAC, nil
}
func checksum(data []byte) uint16 {
var sum uint32
for len(data) >= 2 {
sum += uint32(binary.BigEndian.Uint16(data))
data = data[2:]
}
if len(data) == 1 {
sum += uint32(data[0]) << 8
}
for sum>>16 != 0 {
sum = (sum & 0xffff) + (sum >> 16)
}
return ^uint16(sum)
}
func htons(v uint16) uint16 { return (v<<8)&0xff00 | v>>8 }
+100
View File
@@ -0,0 +1,100 @@
package ndp
import (
"encoding/binary"
"net"
"os"
"path/filepath"
"testing"
)
func TestLoadRulesAndMatches(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "ndppd.conf")
contents := "proxy eth0 {\n rule 2001:db8:1::/112 {\n static\n }\n}\n"
if err := os.WriteFile(path, []byte(contents), 0o600); err != nil {
t.Fatal(err)
}
rules, err := loadRules(path)
if err != nil {
t.Fatal(err)
}
if !matches(net.ParseIP("2001:db8:1::42"), rules) {
t.Fatal("address inside rule did not match")
}
if matches(net.ParseIP("2001:db8:2::42"), rules) {
t.Fatal("address outside rule matched")
}
}
// The A-fix: the responder must only ever advertise into the operator's own
// routed prefix. A rule outside `allowed` is dropped even if the rules file (a
// panel-writable path) names it — so a compromised writer cannot turn the root
// raw-socket responder into an NDP spoofer for arbitrary external addresses.
func TestFilterRules(t *testing.T) {
// A rule inside the allowed prefix is kept.
inside := []net.IPNet{*mustCIDR(t, "2001:db8:aaaa::1/128")}
allowed := mustCIDR(t, "2001:db8:aaaa::/48")
if got := filterRules(inside, allowed); len(got) != 1 {
t.Fatalf("in-prefix rule dropped: %v", got)
}
// A rule outside (or straddling the boundary) is dropped.
outside := []net.IPNet{
*mustCIDR(t, "2001:db8:bbbb::1/128"),
*mustCIDR(t, "2001:db8:aaa1::1/128"),
*mustCIDR(t, "2002:db8:aaaa::1/128"),
}
if got := filterRules(outside, allowed); len(got) != 0 {
t.Fatalf("out-of-prefix rule kept: %v", got)
}
// Mix: only the in-prefix one survives, in order.
mixed := append(append([]net.IPNet{}, inside...), outside[0])
if got := filterRules(mixed, allowed); len(got) != 1 || !got[0].Contains(net.ParseIP("2001:db8:aaaa::1")) {
t.Fatalf("mixed filtering wrong: %v", got)
}
// nil allowed = constrained-less test/unconstrained mode: keep everything.
if got := filterRules(outside, nil); len(got) != len(outside) {
t.Fatalf("nil allowed should keep all rules, got %v", got)
}
}
func mustCIDR(t *testing.T, s string) *net.IPNet {
t.Helper()
_, n, err := net.ParseCIDR(s)
if err != nil {
t.Fatal(err)
}
return n
}
func TestSendAdvertisementUsesTargetAsSource(t *testing.T) {
target := net.ParseIP("2001:db8:1::42")
peer := net.ParseIP("fe80::2")
ns := make([]byte, 14+40+24)
copy(ns[6:12], []byte{0, 1, 2, 3, 4, 5})
copy(ns[22:38], peer.To16())
ns[12], ns[13] = 0x86, 0xdd
ns[20] = 58
ns[14+40] = icmpv6NS
copy(ns[14+40+8:14+40+24], target.To16())
// Build through the same helper used by the raw socket sender; this locks
// down the wire-format invariants that matter to the provider router.
mac := [6]byte{0x10, 0x66, 0x6a, 0xf4, 0x1b, 0x4a}
frame, _, err := buildAdvertisement(mac, ns, target)
if err != nil {
t.Fatal(err)
}
ipv6 := frame[14:54]
icmp := frame[54:]
if got := net.IP(ipv6[8:24]); !got.Equal(target) {
t.Fatalf("NA source = %s, want %s", got, target)
}
if !net.IP(ipv6[24:40]).Equal(peer) {
t.Fatalf("NA destination is not the NS peer")
}
if binary.BigEndian.Uint32(icmp[4:8]) != naFlags {
t.Fatal("NA flags lost router/solicited/override bits")
}
}
+6 -6
View File
@@ -81,11 +81,11 @@ func (s *Server) t(r *http.Request, key string, args ...any) string {
// zh and one en variant toggled by {{if eq .Lang "zh"}}.
func tr(l, key string, args ...any) string {
m := map[string][2]string{ // key -> [zh, en]
"err_bad_login": {"用户名或密码错误", "invalid credentials"},
"err_too_many": {"尝试过于频繁,请 1 分钟后再试", "Too many attempts, please wait 1 minute"},
"err_pass_mismatch": {"两次输入的密码不一致", "The two passwords do not match"},
"new_root_password": {"新的 root 密码:\n%[1]v", "New root password:\n%[1]v"},
"reinstall_done": {"重装完成,新的 root 密码:\n%[1]v", "Reinstall complete. Root password:\n%[1]v"},
"err_bad_login": {"用户名或密码错误", "invalid credentials"},
"err_too_many": {"尝试过于频繁,请 1 分钟后再试", "Too many attempts, please wait 1 minute"},
"err_pass_mismatch": {"两次输入的密码不一致", "The two passwords do not match"},
"new_root_password": {"新的 root 密码:\n%[1]v", "New root password:\n%[1]v"},
"reinstall_done": {"重装完成,新的 root 密码:\n%[1]v", "Reinstall complete. Root password:\n%[1]v"},
"ssh_keys_apply_warn": {"容器当前不可达,密钥已保存,将在下次重装后写入。", "Container unreachable — keys saved and will be applied on the next reinstall."},
}
pair, ok := m[key]
@@ -96,4 +96,4 @@ func tr(l, key string, args ...any) string {
return fmt.Sprintf(pair[0], args...)
}
return fmt.Sprintf(pair[1], args...)
}
}
+3 -3
View File
@@ -369,9 +369,9 @@ func (s *Server) buildData(u *db.User, msg, errMsg string) pageData {
// Snapshots come from Incus (one list call). A failure is non-fatal: the
// page still renders, and the snapshot modal shows the empty state.
if snaps, err := s.mgr.SnapshotList(u.Name); err == nil {
for _, sn := range snaps {
d.Snapshots = append(d.Snapshots, snapshotRow{Name: sn.Name, CreatedAt: sn.CreatedAt, Size: mgr.HumanBytes(sn.Size)})
}
for _, sn := range snaps {
d.Snapshots = append(d.Snapshots, snapshotRow{Name: sn.Name, CreatedAt: sn.CreatedAt, Size: mgr.HumanBytes(sn.Size)})
}
}
d.SnapshotLimit = s.mgr.SnapshotLimit()
keys, _ := s.db.ListSSHKeys(u.ID)
+19 -19
View File
@@ -224,9 +224,9 @@ func TestOverviewShowsMonthlyBandwidth(t *testing.T) {
func TestOverviewDomSaveHiddenByDefault(t *testing.T) {
srv, _ := newTestServer(t)
html := srv.renderToString(t, "overview.html", pageData{
User: &db.User{Name: "alice"},
Prefix: "/" + testSecret,
Domains: []domainRow{{Domain: "example.com", ProxyProtocol: false}},
User: &db.User{Name: "alice"},
Prefix: "/" + testSecret,
Domains: []domainRow{{Domain: "example.com", ProxyProtocol: false}},
V4Forward: true,
})
if !strings.Contains(html, `id="domSave" hidden`) {
@@ -298,10 +298,10 @@ func TestOverviewConnectivityLayout(t *testing.T) {
func TestOverviewShowsBandwidthQuota(t *testing.T) {
srv, _ := newTestServer(t)
html := srv.renderToString(t, "overview.html", pageData{
User: &db.User{Name: "alice"},
UpGB: "3.0",
DownGB: "1.0",
Prefix: "/" + testSecret,
User: &db.User{Name: "alice"},
UpGB: "3.0",
DownGB: "1.0",
Prefix: "/" + testSecret,
BandwidthQuotaGB: 100,
BandwidthUsedGB: "4.0",
BandwidthPct: 4,
@@ -316,14 +316,14 @@ func TestOverviewShowsBandwidthQuota(t *testing.T) {
}
// Over quota: the throttled badge appears.
throttled := srv.renderToString(t, "overview.html", pageData{
User: &db.User{Name: "alice"},
UpGB: "60.0",
DownGB: "41.0",
Prefix: "/" + testSecret,
User: &db.User{Name: "alice"},
UpGB: "60.0",
DownGB: "41.0",
Prefix: "/" + testSecret,
BandwidthQuotaGB: 100,
BandwidthUsedGB: "101.0",
BandwidthPct: 100,
Throttled: true,
Throttled: true,
})
for _, want := range []string{"limited to 1Mbps", "width:100%"} {
if !strings.Contains(throttled, want) {
@@ -1092,11 +1092,11 @@ func TestStatsEndpoint(t *testing.T) {
}
var resp struct {
Points []struct {
M int64 `json:"m"`
C float64 `json:"c"`
Y int64 `json:"y"`
R int64 `json:"r"`
T int64 `json:"t"`
M int64 `json:"m"`
C float64 `json:"c"`
Y int64 `json:"y"`
R int64 `json:"r"`
T int64 `json:"t"`
} `json:"points"`
}
if err := json.Unmarshal(rr.Body.Bytes(), &resp); err != nil {
@@ -1308,8 +1308,8 @@ func TestOverviewAdminKeysDisclosure(t *testing.T) {
Prefix: "/" + testSecret,
Lang: langZh,
AdminSSHKeys: []sshKeyRow{
{ID: 1, Name: "ops", Key: k1, Active: true}, // already granted
{ID: 2, Name: "ci", Key: k2, Active: false}, // not granted
{ID: 1, Name: "ops", Key: k1, Active: true}, // already granted
{ID: 2, Name: "ci", Key: k2, Active: false}, // not granted
},
AdminKeysAnyActive: true, // one key granted -> default expanded
})
+106 -15
View File
@@ -25,6 +25,7 @@ import (
"vpsmgr/internal/fw"
"vpsmgr/internal/inter"
"vpsmgr/internal/mgr"
"vpsmgr/internal/ndp"
"vpsmgr/internal/panel"
"vpsmgr/internal/pw"
"vpsmgr/internal/su"
@@ -91,6 +92,31 @@ RemainAfterExit=yes
WantedBy=multi-user.target
`
// ipv6ProxyUnit is the PREFIX-mode boot unit: it rebuilds the host routes and
// the guest static IPv6 config once (ExecStartPre), then runs the long-lived
// in-process NDP responder whose NA source is the advertised global address
// (ndppd and Linux proxy_ndp use a link-local source that this class of
// provider rejects). Runs as root (raw AF_PACKET socket) and is restarted on
// failure; RestartSec=2 keeps a genuinely broken ext_if from hot-looping. The
// responder re-reads the NDP rule file while it runs, so vps add/del never
// restarts it.
const ipv6ProxyUnit = `# Managed by vpsmgr — installed file, do not edit by hand.
# Changes are overwritten on the next install.
[Unit]
Description=vps IPv6 pass-through routes and NDP responder
After=network-online.target incus.service vps-nft.service
Wants=network-online.target
Before=vps.service
[Service]
Type=simple
ExecStartPre=/usr/local/bin/vps ipv6-reapply
ExecStart=/usr/local/bin/vps ipv6-proxy
Restart=always
RestartSec=2
[Install]
WantedBy=multi-user.target
`
func main() {
log.SetFlags(0)
if len(os.Args) < 2 {
@@ -142,6 +168,11 @@ func main() {
// Re-attach IPv6 routes/proxy_ndp for all existing containers.
// Run by the vps-ipv6.service boot unit and `vps install`.
err = cmdIPv6Reapply()
case "ipv6-proxy":
// Root-only systemd worker for prefix-mode NDP. It must emit the
// advertised global address as the NA source; ndppd and Linux's
// proxy_ndp use a link-local source, which this provider rejects.
err = cmdIPv6Proxy()
case "ip6":
// Root helper behind the sudoers whitelist: validates every argument
// before running the pinned ip -6 operations (see main_ip6.go).
@@ -175,7 +206,7 @@ usage:
vps config list|set|help inspect/change config.yaml (per-field validated edits)
vps version
system:
vps install | serve | ipv6-reapply
vps install | serve | ipv6-reapply | ipv6-proxy
cpu: whole cores >= 1 (e.g. --cpu 2), or a fraction of one core in 0.1..0.9
(e.g. --cpu 0.5 — the container is pinned to one core with a time slice)
bandwidth: monthly quota in GiB (upload + download combined); 0 or empty = unlimited
@@ -361,19 +392,43 @@ func cmdInstall() error {
if err := writeUnit("vps-nft.service", nftUnit); err != nil {
return err
}
// IPv6 pass-through boot unit (re-applies routes/proxy after reboot).
// IPv6 pass-through boot unit. Prefix mode runs the long-lived in-tree NDP
// responder (ipv6ProxyUnit); pool mode re-applies routes/proxy at boot via
// the oneshot ipv6Unit. Stop the old unit before replacing it so an upgrade
// from the previous oneshot/ndppd implementation actually starts the new
// responder, and (for the prefix responder) so a bad ext_if fails the
// install clearly instead of the service hot-looping under Restart=always.
_ = exec.Command("systemctl", "disable", "--now", "vps-ipv6.service").Run()
proxyUsable := true
if c.IPv6Enabled() {
if err := writeUnit("vps-ipv6.service", ipv6Unit); err != nil {
return err
unit := ipv6Unit
if c.IPv6ModeEffective() == cfg.IPv6ModePrefix {
unit = ipv6ProxyUnit
if _, err := ethernetMAC(c.Net.ExtIF); err != nil {
log.Printf("warning: prefix-mode IPv6 needs an Ethernet ext_if (%s): %v — NDP responder not enabled; fix net.ext_if and re-run `vps install`", c.Net.ExtIF, err)
proxyUsable = false
}
}
if proxyUsable {
if err := writeUnit("vps-ipv6.service", unit); err != nil {
return err
}
} else {
_ = os.Remove("/etc/systemd/system/vps-ipv6.service")
}
} else {
_ = os.Remove("/etc/systemd/system/vps-ipv6.service")
}
// Prefix mode uses the vpsmgr raw NDP responder; a distro ndppd listener
// would emit competing (link-local-source) NAs, so disable it explicitly.
_ = exec.Command("systemctl", "disable", "--now", "ndppd.service").Run()
if out, err := exec.Command("systemctl", "daemon-reload").CombinedOutput(); err != nil {
return fmt.Errorf("daemon-reload: %s", strings.TrimSpace(string(out)))
}
if out, err := exec.Command("systemctl", "enable", "--now", "vps-nft.service").CombinedOutput(); err != nil {
return fmt.Errorf("enable vps-nft: %s", strings.TrimSpace(string(out)))
}
if c.IPv6Enabled() {
if c.IPv6Enabled() && proxyUsable {
if out, err := exec.Command("systemctl", "enable", "--now", "vps-ipv6.service").CombinedOutput(); err != nil {
return fmt.Errorf("enable vps-ipv6: %s", strings.TrimSpace(string(out)))
}
@@ -460,6 +515,49 @@ func writeUnit(name, content string) error {
return os.Rename(tmp, p)
}
// ethernetMAC returns the 6-byte hardware address of iface by name, or an
// error when it is missing or has no Ethernet MAC (a tunnel, bare ifb, some
// cloud vtaps, bond without a slave, ...). Prefix-mode NDP needs a real L2 MAC
// to answer neighbor solicitations with, so this is checked both at install
// (so the responder is not enabled on an interface that can never work) and
// in cmdIPv6Proxy (fail clearly rather than looping).
func ethernetMAC(ifname string) (net.HardwareAddr, error) {
iface, err := net.InterfaceByName(ifname)
if err != nil {
return nil, fmt.Errorf("interface %s: %w", ifname, err)
}
if len(iface.HardwareAddr) != 6 {
return nil, fmt.Errorf("interface %s has no Ethernet MAC (%v)", ifname, iface.HardwareAddr)
}
return net.HardwareAddr(append([]byte(nil), iface.HardwareAddr...)), nil
}
// cmdIPv6Proxy runs the prefix-mode NDP responder. Root-only (raw AF_PACKET
// socket); started by vps-ipv6.service's ExecStart. It only ever answers for
// addresses inside the operator's routed prefix (see ndp.Run's `allowed`), so a
// compromised rules-file writer cannot turn the host into an NDP spoofer for
// arbitrary external addresses. Returns nil when IPv6 is not prefix mode, so
// the unit is harmlessly idle outside prefix deployments.
func cmdIPv6Proxy() error {
c, err := cfg.Load()
if err != nil {
return fmt.Errorf("load config: %w", err)
}
if !c.IPv6Enabled() || c.IPv6ModeEffective() != cfg.IPv6ModePrefix {
return nil
}
if _, err := ethernetMAC(c.Net.ExtIF); err != nil {
return err
}
n, err := c.IPv6Network()
if err != nil {
return err
}
// n is the parent routed prefix; hand it to the responder as the only
// address space it may advertise into.
return ndp.Run("/etc/vpsmgr/ndppd.conf", c.Net.ExtIF, n)
}
// ensureVPSUser sets up the unprivileged 'vps' account the panel daemon runs
// as. Idempotent, called by `vps install`:
// - creates the system user if missing
@@ -535,16 +633,9 @@ vps ALL=(root) NOPASSWD: /usr/bin/systemctl is-active --quiet vps.service
# ip — the panel never gets bare ip -6 wildcard rights.
vps ALL=(root) NOPASSWD: /usr/local/bin/vps ip6 *
vps ALL=(root) NOPASSWD: /sbin/sysctl -w net.ipv6.conf.all.forwarding=1
vps ALL=(root) NOPASSWD: /usr/sbin/service ndppd restart
vps ALL=(root) NOPASSWD: /usr/sbin/service ndppd start
vps ALL=(root) NOPASSWD: /usr/sbin/service ndppd stop
vps ALL=(root) NOPASSWD: /usr/bin/pkill -x ndppd
vps ALL=(root) NOPASSWD: /usr/sbin/ndppd -d -p /var/run/ndppd.pid
# ndppd reads /etc/ndppd.conf by default (no -c in the init script), but the
# panel renders the config under /etc/vpsmgr — these pinned commands keep the
# root-owned symlink /etc/ndppd.conf -> /etc/vpsmgr/ndppd.conf in sync.
vps ALL=(root) NOPASSWD: /bin/ln -sf /etc/vpsmgr/ndppd.conf /etc/ndppd.conf
vps ALL=(root) NOPASSWD: /bin/rm -f /etc/ndppd.conf
# Prefix-mode IPv6 NDP is served by the in-tree responder (root systemd unit,
# vps-ipv6.service) reading /etc/vpsmgr/ndppd.conf — the panel no longer
# starts/stops the distro ndppd daemon, so no ndppd/ln-sf grants are needed.
`
if err := os.MkdirAll("/etc/sudoers.d", 0o750); err != nil {
return fmt.Errorf("mkdir /etc/sudoers.d: %w", err)