Piik/docs/operations/self-hosting.md

Self-Hosting Operations

This runbook owns initial single-host service setup. Application releases use deployment; accepted environment values and ports are in configuration reference.

Topology

Browser -- HTTPS/WSS --> nginx :443 --> Screener :8787
Browser -- /rtc/v1 WSS --> nginx :443 --> LiveKit :7880
Browser <---------- DTLS-SRTP P2P ----------> Browser
Browser -------- ordinary STUN/UDP --------> coturn :3478
Browser <--------- DTLS-SRTP/UDP ----------> LiveKit :7882
Screener -------- private RoomService -----> LiveKit :7880

The tracked public baseline uses nginx, a valid Web TLS certificate, time synchronization, coturn 4.17.2 or a newer patched release, and LiveKit Server 1.13.6. LiveKit is required for the accepted SFU fallback and tracked release wrapper. Keep the accepted patch pinned until its replacement passes the same Browser/SFU acceptance boundary. Web and STUN may use different DNS names on the same public IP. Another proxy or P2P-only service shape needs its own matching updater and recovery checks.

Expose only the public listeners in configuration reference. Screener 8787 and LiveKit 7880 stay private.

Tracked Templates

• Screener systemd unit
• LiveKit systemd unit
• LiveKit readiness drop-in
• coturn systemd unit
• STUN-only coturn config
• LiveKit config
• nginx site

Copy templates outside the repository, inject independent secrets through an access-restricted environment file or service manager, and adapt only addresses, domains, certificates, users, and resource limits required by the host.

Initial Setup

1. Configure DNS and TLS for the Web origin and STUN name.
2. Install nginx, coturn, and the pinned LiveKit Server patch. The Screener release is one static binary and needs no language runtime.
3. Create an unprivileged screener service account, /opt/screener/releases, /opt/screener/uploads, and an access-restricted environment file.
4. Provision one verified initial immutable release and atomically point /opt/screener/current to it before enabling the unit. The tracked release wrapper is upgrade-only and deliberately refuses a missing prior release; this repository does not provide a generic first-install transaction.
5. When stable room authority is enabled, use the service-owned /var/lib/screener StateDirectory and include the SQLite file in the host's state backup/recovery policy.
6. Install the tracked service and proxy templates. Bind Screener to loopback on bare metal; containers may bind 0.0.0.0 only when publishing/firewall rules preserve the same boundary.
7. Apply the public-port allowlist and confirm every retired TURN/TCP port is closed.
8. Start coturn and LiveKit before Screener, then nginx. The LiveKit readiness drop-in waits for its private control listener before Screener.

First Go service cutover

The tracked unit starts the Go binary. A Node-to-Go deployment is a one-time infrastructure transaction, not a routine application update: its failure path must restore the old unit and environment before starting the prior release. Prepare and verify that recovery before installing the unit:

sudo install -o root -g root -m 0644 \
  deploy/systemd/screener.service.example \
  /etc/systemd/system/screener.service
sudo systemctl daemon-reload
sudo systemctl show screener.service -p ExecStart -p Environment --no-pager

The output must show /opt/screener/current/screener-server and SCREENER_ENV=production. The staged environment removes NODE_ENV and PEER_ASSISTED_MEDIA, and replaces route NODE_DEBUG with SCREENER_DEBUG=route. Keep credentials, ports, capacity and media configuration unchanged. Do not restart the old release using the Go unit. The routine release wrapper owns only application-symlink recovery; it does not install or restore systemd units.

The first-cutover transaction retains the previous unit, environment and closed SQLite files, applies unit/environment plus release together, and restores those exact values on failure before reloading systemd and restarting Node. After a successful Go cutover, later application releases use the ordinary wrapper.

LiveKit must be dedicated to this Screener application, set room.auto_create: false, and expose its control listener only to the proxy and application host. Screener binds its own listener before touching LiveKit, rejects foreign room names, removes stale managed rooms, confirms the namespace empty, and only then installs signaling. A competing application process must make no LiveKit mutation.

Coturn runs stun-only, no-tcp, and no-tls. It must answer STUN binding and reject TURN allocation. Avoid verbose STUN/ICE logging because infrastructure necessarily observes client IP addresses.

Optional NAT Prediction

Set NAT_PREDICTION_ENABLED=true only while both auxiliary STUN-only listeners are reachable. Add the following to the service-owned coturn configuration, using the local address already bound by the main listener:

aux-server=<PRIVATE_LISTEN_IP>:3479
aux-server=<PRIVATE_LISTEN_IP>:3480

Keep stun-only, no-tcp, and no-tls unchanged. Allow UDP 3479 and 3480 in both the cloud security group and the existing nftables input rule, validate the complete ruleset with nft -c -f /etc/nftables.conf, then restart only coturn.service. Verify ss -lunp shows all three UDP listeners and that an ordinary STUN Binding succeeds on each. Do not add these URLs to production STUN_URLS; enabled Browser connections derive them from the ordinary STUN authority. Keep both listeners and firewall rules for as long as the capability is enabled. Set NAT_PREDICTION_ENABLED=false before removing both aux-server lines and both firewall rules, validate again, and restart coturn.

These same-IP auxiliary ports measure destination-port mapping behavior. They are not a full RFC 5780 alternate-address deployment; that requires a second public IPv4.

Operational Verification

• /healthz returns {"status":"ok"} locally and through the public origin.
• Screener, nginx, coturn, and LiveKit are active with expected restart counts and bounded logs/resources.
• The official Trickle ICE sample obtains an ordinary UDP srflx candidate from the configured STUN service and no relay candidate.
• A direct room presents media on two networks when Peer ICE succeeds.
• A constrained route creates one managed Host publication and only admitted subscriptions over LiveKit UDP; Peer descendants remain ordinary P2P.
• Blocking all UDP reaches a clear bounded failure.
• Restarting Screener in stable mode retains room authority while Browser sessions and media reconnect from fresh process state.

Representative network, mobile, audio, 20-Viewer, and long-running evidence is tracked in verification status, not in this runbook.