651 lines
37 KiB
Markdown
651 lines
37 KiB
Markdown
# Minimal Deployment
|
|
|
|
Last verified against upstream documentation: 2026-08-21.
|
|
|
|
This section documents the repository's UDP-only deployment candidate: one
|
|
Node.js process provides the built Web client, room API, and WebSocket signaling
|
|
behind Caddy or nginx; application ICE advertises only STUN by default; LiveKit
|
|
supplies bounded SFU-root capacity. Normal media remains distributed through
|
|
direct or peer edges whenever those paths work.
|
|
|
|
A deployment may additionally provide one single-node LiveKit process as the
|
|
current controller's automatic final media fallback. This capacity is dormant unless the complete
|
|
`LIVEKIT_URL`/key/secret tuple is configured. It does not replace the P2P path,
|
|
the peer-assisted experiment, or required STUN discovery. LiveKit remains
|
|
ICE/UDP only. Source no longer contains the rejected participant-wide Peer ICE TURN
|
|
candidate; stale `PEER_ICE_TURN_*` keys fail startup even when blank. A
|
|
the selected-edge source tuple is enabled in production only for the controller's
|
|
single exceptional edge. Production ordinary peer ICE remains STUN-only and the
|
|
tracked coturn example remains `stun-only`; the shared host retains its older
|
|
authenticated-relay daemon configuration and firewall range. The application
|
|
does not pre-advertise TURN credentials to ordinary peers.
|
|
|
|
Production currently runs exact `fd76277b05d491af8840b28f3132b7ff445d3cbe`
|
|
on the existing shared public IP, with local/public health 200 and SQLite v3
|
|
integrity. The 915,377-byte artifact has SHA-256
|
|
`8da4b8b2b1ae4b82add615f4367ee447b5ade86c4e58a938e51085940b9867d3`.
|
|
The 2026-08-20T20:53:59Z UTC cutover held the deployment lock for 8,389 ms;
|
|
local health returned 520 ms after service stop (478 ms after the symlink
|
|
switch). Screener, LiveKit, coturn, and nginx are active/running with
|
|
`NRestarts=0`. SQLite v3 contains five rooms including room `1`, with the
|
|
service owner/mode and database hash preserved. `SITE_ACCESS_PASSWORD`,
|
|
`/api/site-access`, and the nginx limiter replaced their retired Host-admission
|
|
names in the prior access cutover; this release preserved the environment hash
|
|
and the old endpoint remains 404. Exact
|
|
`5e4a3679076a9ea2fe7a41fadf4be65a439db450`
|
|
is the immediate rollback target.
|
|
The selected-edge UDP tuple is configured with TTL 120, but no real TURN/SFU
|
|
media canary was triggered; this remains a bounded configuration smoke, not
|
|
broad-rollout media acceptance.
|
|
|
|
## Topology and prerequisites
|
|
|
|
Use two DNS names, for example `share.example.com` and `stun.example.com`. They
|
|
may resolve to the same public IP: the Web ingress owns TCP 443 while the
|
|
self-hosted STUN listener owns UDP 3478.
|
|
|
|
```text
|
|
browser -- HTTPS/WSS --> Caddy or nginx :443 --> Node.js :8787
|
|
browser <------------ DTLS-SRTP P2P ------------> browser
|
|
browser -- STUN binding/UDP --> coturn :3478 (application advertises STUN only)
|
|
browser <---------- DTLS-SRTP/UDP ----------> LiveKit :7882
|
|
```
|
|
|
|
The minimum runtime is Node.js 24 LTS and coturn 4.17.2 or a newer patched
|
|
release. Provision a valid TLS certificate for the Web name. Enable operating system time
|
|
synchronization and keep the Node application port reachable only from its
|
|
reverse proxy. A single Node process is intentional. Active participants and
|
|
signaling remain in memory; an optional protected SQLite file can restore room
|
|
identity after a restart, but it does not restore live WebRTC connections.
|
|
|
|
Build and validate the exact revision before starting it:
|
|
|
|
```sh
|
|
npm ci
|
|
npm run check
|
|
npm start
|
|
```
|
|
|
|
Each immutable release must own an independent dependency tree. Never hard-link
|
|
`node_modules` or another file that deployment may `chown`, `chmod`, replace, or
|
|
remove: metadata changes would also mutate the rollback release. A copy or
|
|
copy-on-write reflink is acceptable only after an inode audit confirms that the
|
|
old and new regular-file sets have zero shared inodes. Do not recursively change
|
|
permissions until that check passes.
|
|
|
|
Transport archives must preserve UTF-8 entry names. Prefer the POSIX release
|
|
archive path and verify the extracted manifest before switching. A Windows ZIP
|
|
used for the 2026-08-20 route-rescue release altered two non-runtime Chinese
|
|
documentation names on Linux. Runtime artifacts were unaffected and the
|
|
immutable upload remains evidence, but do not reuse that packaging method.
|
|
|
|
In a strict-shell deployment, expected service states are data, not command
|
|
failures. Do not call `systemctl is-active` bare under `set -e`/`ERR`: an
|
|
intentionally stopped service returns a nonzero status and can trigger a false rollback. Read
|
|
`ActiveState` with `systemctl show`, compare the returned string explicitly, and
|
|
keep stop, symlink switch, start, health check, and rollback as separate steps.
|
|
Preflight every inspection dependency before taking a backup or changing a
|
|
service. If a host lacks the `sqlite3` CLI, use the pinned Node runtime's
|
|
`node:sqlite` API or stop before the first write. After a start, poll health for
|
|
a short bounded window instead of treating one request during startup as a
|
|
failed release; retain the last failure while still enforcing the deadline.
|
|
|
|
Run `npm start` under a service supervisor that injects the environment, restarts
|
|
on failure, and applies bounded logs. For a simple untracked environment file,
|
|
the equivalent direct launch is
|
|
`node --env-file=.env.production dist/server/server/index.js`. Do not expose port
|
|
8787 to the Internet. The application default is `0.0.0.0` for LAN development
|
|
and container compatibility, while the bare-metal production baseline below
|
|
explicitly sets `LISTEN_HOST=127.0.0.1` because only the same-host reverse proxy
|
|
should connect. A container can instead use `0.0.0.0` and enforce the intended
|
|
boundary with port publishing rules or a host firewall.
|
|
|
|
The optional same-host LiveKit layout needs no additional public hostname:
|
|
`LIVEKIT_URL=wss://share.example.com` and nginx proxies only current `/rtc/v1*`
|
|
requests to LiveKit. Retired `/rtc` and `/rtc/validate` endpoints return 404
|
|
without request-target logging. LiveKit's HTTP/WebSocket listener on TCP 7880 is
|
|
private to nginx, while WebRTC media reaches LiveKit directly on UDP 7882.
|
|
|
|
## Candidate boundary and rollback
|
|
|
|
The accepted target for `share.bonfire.icu` remains distributed and automatic:
|
|
direct/peer UDP first, then an SFU virtual parent feeding normally one or two
|
|
roots, whose peer descendants continue carrying media. Multiple exceptional viewers that cannot attach behind a healthy
|
|
root may consume additional server egress only under a separate explicit cap.
|
|
|
|
The repository requires STUN and ordinary peer connections receive STUN-only
|
|
ICE. The accepted ladder is direct/peer UDP, then the revision-bound SFU/UDP
|
|
virtual parent, then optional authenticated TURN for one controller-selected
|
|
exceptional edge. LiveKit participants receive only revision-bound `sfu-config`
|
|
URL/token messages and negotiate within LiveKit's separate ICE domain. The
|
|
selected-edge TURN config/wire is deployed in the current `c27df22` release with one
|
|
UDP URL and a 120-second credential TTL. It was not exercised by a real media
|
|
session during this cutover.
|
|
|
|
`PEER_ASSISTED_MEDIA=true` is the process-wide topology/SFU switch. Every normal
|
|
room gets its own bounded controller state; ordinary peer connections remain
|
|
STUN-only and only the selected-edge source path may grant TURN to the
|
|
controller's then-current edge. Run candidate releases on an isolated
|
|
instance/hostname and keep
|
|
the old release unchanged for rollback. If the candidate fails, roll back the
|
|
release or instance; do not add a permanent dual-transport branch.
|
|
|
|
A shared-public-IP instance can test candidate behavior while the old TURN
|
|
service stays live, but it cannot prove the clean-port boundary or approve broad
|
|
migration. The room-1 smoke uses this shape: nginx owns 443, LiveKit 1.13.5 owns
|
|
UDP 7882, Peer ICE TURN is disabled, and LiveKit is fail-closed under a 192 MiB
|
|
high/256 MiB hard cgroup limit with restart disabled. Prefer a separate VM/IP
|
|
for the full gate and verify representative external networks and devices.
|
|
|
|
TURN REST authentication proves only HMAC and expiry to coturn. Coturn cannot
|
|
verify a room, route, edge, or connection generation, so the application must
|
|
bind issuance and parent/child rebuild to the current controller state, keep the
|
|
attempt one-use and short-lived, and revalidate every asynchronous boundary.
|
|
Never describe coturn itself as cryptographic selected-edge enforcement.
|
|
HTTPS/WSS always remains TLS/TCP independently.
|
|
|
|
The 2026-08-20 exact-room canary used source `a11a73dfa79d`, inactive release
|
|
`/opt/screener/releases/a11a73dfa79d-r4`, artifact SHA-256
|
|
`C954185869A3A15CCCE642AB72A4CF90770476C117D61182D7728280C30A036F`, and
|
|
backup `/opt/screener/backups/turn-a11-20260820T065933Z`. The candidate
|
|
switch completed at 14:59:33 +08. Local authenticated coturn allocation and
|
|
application/deployment gates passed, but the real direct Host plus Pion Viewer
|
|
did not establish its peer connection. The forced-relay test was therefore not
|
|
run. Issuance was removed first, then coturn and its firewall rules returned to
|
|
the pre-canary authenticated-relay baseline: TCP/UDP 3478 and UDP 49152-49251.
|
|
Exact `7fea60ef6f2ad14a9ac1c23a89a523d91bbb97e4` was restored and advertises no
|
|
TURN credential. The final lock-free check at 15:12:40 +08 found active
|
|
services, zero allocations, and matching health, configuration, firewall, and
|
|
SQLite state.
|
|
No canary test process, browser profile, or transient firewall rule remained.
|
|
The retained release is inactive. This is a canary no-go, not TURN deployment
|
|
or media evidence. Two earlier attempts changed no durable state: one stopped
|
|
before backup/write because `sqlite3` was absent, and one rolled back after a
|
|
single health request landed in the normal startup window. The preflight and
|
|
bounded-poll rules above are the retained fixes.
|
|
|
|
## Production application environment
|
|
|
|
Keep the real values in the process secret store or an untracked, access-restricted
|
|
environment file. Node.js 24 can load such a file with `--env-file`; the tracked
|
|
`.env.example` is only a schema. Generate independent secrets with a password
|
|
manager or a cross-platform Node command such as:
|
|
|
|
```sh
|
|
node -e "console.log(require('node:crypto').randomBytes(32).toString('base64url'))"
|
|
```
|
|
|
|
Use this production baseline:
|
|
|
|
```dotenv
|
|
NODE_ENV=production
|
|
LISTEN_HOST=127.0.0.1
|
|
PORT=8787
|
|
PUBLIC_BASE_URL=https://share.example.com
|
|
ALLOWED_ORIGINS=https://share.example.com
|
|
SITE_ACCESS_PASSWORD=<INDEPENDENT_8_TO_128_BYTE_ACCESS_KEY>
|
|
ROOM_DATABASE_PATH=/var/lib/screener/rooms.sqlite
|
|
ROOM_TTL_SECONDS=14400
|
|
MAX_ROOMS=1000
|
|
MAX_VIEWERS_PER_ROOM=8
|
|
|
|
STUN_URLS=stun:stun.example.com:3478
|
|
```
|
|
|
|
To make automatic SFU fallback capacity available, add the complete tuple:
|
|
|
|
```dotenv
|
|
PEER_ASSISTED_MEDIA=true
|
|
LIVEKIT_URL=wss://share.example.com
|
|
LIVEKIT_API_KEY=<GENERATED_LIVEKIT_API_KEY>
|
|
LIVEKIT_API_SECRET=<INDEPENDENT_SECRET_OF_AT_LEAST_32_BYTES>
|
|
MAX_SFU_ROOTS_PER_ROOM=2
|
|
```
|
|
|
|
The rejected `PEER_ICE_TURN_*` participant-wide tuple is removed; supplying any
|
|
stale key, even blank, fails startup. The source candidate uses the complete
|
|
`SELECTED_EDGE_TURN_URLS`, `SELECTED_EDGE_TURN_SHARED_SECRET`, and
|
|
`SELECTED_EDGE_TURN_CREDENTIAL_TTL_SECONDS` tuple with one explicit TURN/UDP URI,
|
|
an independent secret and bounded TTL. The current production tuple uses the
|
|
coturn UDP endpoint and a 120-second TTL; only the controller-selected edge may
|
|
receive a short-lived grant. Roll back by removing the application tuple before
|
|
changing coturn or firewall state. Credentials never enter URLs, logs, browser
|
|
persistence, room rows, or SQLite.
|
|
|
|
`PEER_ASSISTED_ROOM_IDS` is retired. Supplying it, even blank, fails startup so
|
|
that a stale room-1 deployment cannot silently retain the old scope. With
|
|
`PEER_ASSISTED_MEDIA=true`, every normal room receives peer-assisted routing,
|
|
optional LiveKit fallback, and the same per-room root/fanout/failure guards.
|
|
Every ordinary peer connection remains STUN-only; selected-edge TURN is still
|
|
issued only to a current controller-selected edge. There is no browser control,
|
|
percentage rollout, or second router.
|
|
|
|
`ALLOWED_ORIGINS` must list exact `http` or `https` origins, never `*`.
|
|
`SITE_ACCESS_PASSWORD` is required in production and must contain 8 through
|
|
128 visible ASCII bytes (`0x21` through `0x7e`). It must not be reused for
|
|
LiveKit, TLS, TURN, or another service. It authorizes room creation, Host role,
|
|
and code-only Viewer attempts; it does not replace a private room grant or
|
|
password. Local development and tests may omit it. Supplying a removed access
|
|
key, even blank, fails startup.
|
|
`ROOM_DATABASE_PATH` is optional but requires `SITE_ACCESS_PASSWORD`.
|
|
Omit the database path to keep random temporary rooms; `ROOM_TTL_SECONDS`
|
|
applies only to those rooms.
|
|
`MAX_VIEWERS_PER_ROOM` defaults to 8 and accepts 1 through 16. It is an admission
|
|
limit, not evidence that the publisher can sustain that many streams.
|
|
|
|
The three `LIVEKIT_*` values must either all be absent or all be present, and a
|
|
complete tuple requires `PEER_ASSISTED_MEDIA=true`. An empty tuple keeps the
|
|
optional SDK and server path dormant. `LIVEKIT_URL` must be a plain `ws:` or
|
|
`wss:` origin with no `/rtc` suffix; production requires `wss:`.
|
|
`LIVEKIT_API_SECRET` must contain at least 32 bytes and must not reuse
|
|
`SITE_ACCESS_PASSWORD`. `MAX_SFU_ROOTS_PER_ROOM` defaults to
|
|
2 and accepts only 1 or 2; it is ignored when LiveKit is not configured. These
|
|
credentials authorize short-lived LiveKit room tokens and do not provide E2EE:
|
|
the LiveKit operator can access ordinary SFU media.
|
|
|
|
For the first candidate canary, use an isolated instance and a protected
|
|
persistent room whose ID is stable across restarts. Restart the application and
|
|
verify that both that room and a second normal room contain
|
|
`mediaMode: "peer-assisted"`, with independent route revisions and no shared
|
|
room state. Exercise join, offer and answer, stop, reconnect, and room deletion
|
|
in both rooms. Ordinary Web and
|
|
Native-shaped clients remain STUN-only. With the selected-edge tuple, only
|
|
the current controller-selected edge may receive a one-use grant; all other
|
|
sessions and connections remain STUN-only. Roll back by disabling application
|
|
issuance before restoring the exact recorded pre-canary coturn/firewall baseline, or direct
|
|
traffic to the unchanged old release. Do not treat disabling
|
|
`PEER_ASSISTED_MEDIA` alone as TURN rollback: disable the selected-edge tuple
|
|
independently, and no removed old
|
|
TURN wire is restored. During migration, remove the retired room-ID variable in
|
|
the same coherent change; the process flag explicitly enables the all-room
|
|
controller.
|
|
|
|
Only `POST /api/site-access` accepts the site access secret in an
|
|
`Authorization: Bearer` header from an exact allowed Origin. Success returns a
|
|
12-hour stateless HMAC-SHA256 cookie with `HttpOnly`, `SameSite=Strict`,
|
|
`Path=/`, bounded `Max-Age`, and, under production HTTPS, `Secure` plus an
|
|
`__Host-` name. The cookie contains no account or server-side session ID. There
|
|
is no account database, JWT, session map, or logout endpoint.
|
|
|
|
nginx limits this exact endpoint per source at `5r/m` with `burst=5 nodelay` and
|
|
returns 429 when exhausted. The limiter uses nginx shared memory; do not add the
|
|
secret, Authorization header, request body, or a new source-address field to
|
|
logs. `POST /api/rooms` accepts only the site-access cookie and strict JSON
|
|
with an explicit Viewer policy; a Bearer header is not an alternate creation
|
|
path. `/signal` still admits a cookie-free browser after Origin and capacity
|
|
checks and records the cookie state at upgrade. Host requires it with the room
|
|
Host token. A Viewer may omit it only when presenting a valid, unexpired grant
|
|
for that exact room; otherwise the server rejects before evaluating public-watch
|
|
or a room password.
|
|
|
|
Private rooms are the default. Their invitation is
|
|
`/r/{code}#v={room-scoped-grant}`; the fragment does not enter HTTP or WebSocket
|
|
request targets. The page validates it, writes it only to that room's
|
|
`sessionStorage`, and immediately replaces the visible URL with `/r/{code}`.
|
|
An independent tab without the fragment must first pass site access. A private
|
|
room then requires its room password, while explicit `public-watch` accepts the
|
|
numeric room code. Neither policy
|
|
lets a Viewer create a room or authenticate as Host.
|
|
|
|
Room allocation has two deliberately small policies:
|
|
|
|
- With `ROOM_DATABASE_PATH`, SQLite allocates positive decimal IDs starting at
|
|
`1`. These rooms do not expire, while each private Viewer invitation expires
|
|
after seven days unless rotated earlier. An explicit stop or capture
|
|
track ending stops the current publication and leaves viewers waiting; it does
|
|
not delete the room. A brief signaling disconnect does not stop otherwise
|
|
healthy P2P media.
|
|
- Without `ROOM_DATABASE_PATH`, rooms use random numeric IDs and expire according
|
|
to `ROOM_TTL_SECONDS`; a private grant cannot outlive its room. Stopping sharing
|
|
does not immediately delete it. Public-watch remains available but explicit.
|
|
|
|
The built-in `node:sqlite` schema v3 stores each room ID, a SHA-256 Host token
|
|
digest, one nullable SHA-256 Viewer-grant digest, and optional validated Viewer
|
|
password material in the same `STRICT` row. `NULL` means public-watch or no
|
|
password; checked BLOBs mean private-link/password state. It must not contain
|
|
plaintext tokens or grants, passwords, cookies, names, participants, SDP, ICE
|
|
candidates, IP addresses, TURN credentials, or media.
|
|
Protect and back up the file as service state. The tracked
|
|
systemd unit creates `/var/lib/screener` with `StateDirectory=screener` and mode
|
|
`0700`; the production path above is writable despite `ProtectSystem=strict`.
|
|
The database is designed for one application process, not shared storage across
|
|
multiple instances.
|
|
|
|
For a simple consistent backup, stop `screener.service` before copying the
|
|
SQLite file, then start it again. Restore while the service is stopped, restore
|
|
ownership to the service account and mode `0600`, run SQLite
|
|
`PRAGMA integrity_check`, and only then start the service. An online backup must
|
|
use SQLite's [backup API](https://www.sqlite.org/backup.html) or `VACUUM INTO`;
|
|
do not copy only the live main file while it may have an active journal.
|
|
|
|
The access release changes schema v1 to v2 and has no dual-schema runtime. For
|
|
an existing deployment, stop the service and take a named, immutable v1 copy
|
|
before installing or starting the new binary. Confirm the stopped source reports
|
|
`PRAGMA user_version = 1`, retain the backup outside the release directory, and
|
|
record its checksum. On first v2 startup, one `BEGIN IMMEDIATE` transaction adds
|
|
the checked nullable Viewer digest, writes a different fresh locked-private
|
|
digest to every old room, and advances `user_version` to `2`. Old code-only
|
|
Viewer links intentionally stop working; each existing Host must use rotate once
|
|
to produce a private invitation.
|
|
|
|
After startup, verify `PRAGMA integrity_check`, `user_version = 2`, service
|
|
health, Host reclaim, rotation, and a new Viewer join before removing the
|
|
maintenance boundary. If rollback is required, stop the v2 service, preserve the
|
|
v2 database separately for diagnosis, restore the exact v1 backup with service
|
|
ownership and mode `0600`, verify integrity and `user_version = 1`, and only then
|
|
start the old binary. Never point the old binary at the migrated v2 file.
|
|
|
|
Production completed this migration at 2026-08-20 03:06 +08 on exact
|
|
`bbe4654a7a9b`; its artifact SHA-256 is
|
|
`3C09AF81BCE68B51DD9E36E1C253A880D8D6F24BB498A090CF815AB196C392AA`.
|
|
A stopped, read-only-verified v1 backup retained all four rooms;
|
|
the v2 integrity/schema and locked-private digest checks passed, including room
|
|
`1`. site access and a cookie-free private Viewer denial also passed. An
|
|
earlier artifact attempt was rolled back after hard-linked dependencies let a
|
|
permission change make the rollback release unreadable for 3m11s; the final release has an
|
|
independent dependency tree and zero shared regular-file inodes.
|
|
|
|
The code-only initial-connect recovery deployed at 2026-08-20 09:40 +08 on
|
|
exact `89e6d7649169`; its artifact SHA-256 is
|
|
`39B62039FCFB05281F78FCF85C964C520E2BB2FE5661ADA4758D598A7DF6CEC1`.
|
|
Health returned in 1.09 seconds, the v2 database still contained four rooms and
|
|
room `1`, and environment, nginx, and LiveKit state were unchanged. At 09:37 a
|
|
bare `systemctl is-active` treated the expected stopped state as an error under
|
|
strict shell and caused an immediate healthy rollback; no new application had
|
|
failed. The later explicit `ActiveState` sequence completed successfully.
|
|
|
|
The admission-policy and anonymous-entry cutover deployed between
|
|
2026-08-20 11:13:42 and 11:13:43 +08 on exact
|
|
`05f98d10ecd174427fc969f3ba2d510f12c74eb3`; its artifact SHA-256 is
|
|
`FF938E8D59428F08B3F162DEA6DCF842A4705A94D3153967814CCE9AD6CBD94D`.
|
|
The full gate passed 25 test files/369 tests before the atomic code/environment
|
|
switch. Health and the built asset returned 200; SQLite v2 with four rooms
|
|
including room `1` and zero Screener/LiveKit restarts were preserved. Chrome
|
|
kept all four anonymous entry routes neutral until authorization; the current admission key returned 200,
|
|
while the former and an incorrect key returned 401. Rolling this cutover back
|
|
requires exact `89e6d7649169` together with its environment backup at
|
|
`/etc/screener/backups/05f98d10-precutover-20260820T110654+0800`.
|
|
|
|
One failed validation assertion printed the stateless admission cookie only in
|
|
the operator's private test output. It did not enter the repository, service
|
|
logs, shell history, release artifact, browser profile, or temporary files. No
|
|
value is retained here; the cookie has no server-side state and expires within
|
|
12 hours.
|
|
|
|
Local quality reparenting deployed between 2026-08-20 11:41:22 and 11:41:23
|
|
+08 on exact `d4bc421828c4b74f55195723aace290ffc0e5f9d`; its artifact SHA-256 is
|
|
`3F8CF25F8D989CBDF38DBBCAC4A261C349E44B72D274A944B877540056171A08`.
|
|
The immutable release is `/opt/screener/releases/d4bc421828c4`, its upload is
|
|
`/opt/screener/uploads/screener-d4bc421828c4.tar.gz`, and the pre-switch target
|
|
record is `/opt/screener/backups/current-before-d4bc421828c4.txt`.
|
|
The gate passed 25 test files/378 tests, typecheck and both builds. The 760ms
|
|
code-only switch kept health and `index-YdNLg2E8.js` at 200, SQLite integrity
|
|
and four rooms including room `1`, Screener/LiveKit restarts at 0/0, and
|
|
environment/nginx/LiveKit hashes unchanged. Its immediate rollback is
|
|
`05f98d10ecd1` with the same environment and SQLite v2. A pre-cutover inode gate
|
|
caught two internal esbuild hard links; hash-equal copy replacement broke them
|
|
before the switch, so no running or rollback release was affected.
|
|
At 11:44:15 +08 Screener and LiveKit were still active/running with zero restarts,
|
|
used 39,198,720 and 79,269,888 bytes, and had no error-priority journal entries
|
|
or cgroup high/max/OOM events; host available memory was 373,469,184 bytes.
|
|
No quality/reparent log or real-room trigger appeared, so this verifies the
|
|
deployed code and containment, not reparenting behavior.
|
|
|
|
Viewer presence plus deterministic relay-admission rescue switched from
|
|
`2026-08-20T15:33:21.720345870+08:00` to health-ready at
|
|
`2026-08-20T15:33:22.282829810+08:00`; its gate finished at
|
|
`2026-08-20T15:33:23.051172737+08:00` on exact
|
|
`31bee238bc1e901e823f34e50737e202dc4b04bf`. The 889,710-byte immutable upload
|
|
is `/opt/screener/uploads/screener-31bee238bc1e-20260820T071907Z.zip` with
|
|
SHA-256 `C759A33A50EC3E272F07BCC00F2043CA0042446078472CAFA66CBD406EC925D9`,
|
|
and the active release is `/opt/screener/releases/31bee238bc1e`.
|
|
The gate passed 28 test files/421 tests and both builds. Restart-to-health took
|
|
560.607 ms; 20 ms probing observed 437.137 ms unavailable. External health,
|
|
the asset hash, three neutral entry routes, SQLite integrity/version 2/four
|
|
rooms including room `1`, and the exact revision passed. Screener, LiveKit,
|
|
coturn, and nginx remained active with zero automatic restarts; only Screener
|
|
had the one planned restart. Environment, nginx, LiveKit, coturn, firewall,
|
|
listeners, exact-room/SFU caps, and disabled application TURN tuple were
|
|
unchanged.
|
|
At the `2026-08-20T15:39:07.082219441+08:00` audit, Screener used 38,031,360
|
|
bytes with zero cgroup low/high/max/OOM events. The pre-cutover rollback is exact
|
|
`7fea60ef6f2ad14a9ac1c23a89a523d91bbb97e4` with
|
|
`/opt/screener/backups/31bee238bc1e-precutover-20260820T073321Z`.
|
|
No temporary symlink, deployment process, browser profile, or held deployment
|
|
lock remained; the immutable upload, active release, and backups are retained.
|
|
No real room triggered admission rescue, so this verifies deployment and
|
|
rollback readiness, not the new route transition or media quality.
|
|
|
|
The sole Host display-name cutover attempt used exact source
|
|
`61a87ae47e38abc943cef47b3d7318bb51bf86d6` and artifact
|
|
`screener-61a87ae47e38abc943cef47b3d7318bb51bf86d6-20260820T143538Z.tar.gz`
|
|
(883597 bytes, SHA-256
|
|
`d5b4d6cc567ba96ce37f8eb9934dd5822c64dc084796047d5f6d6217e2e74acb`). It ran at
|
|
`2026-08-20T16:09Z`; the prepare-to-release switch and health gate passed, but a
|
|
post-cutover assertion compared against stale `NRestarts=31` data and triggered
|
|
automatic rollback. The application and database were not the cause. Rollback
|
|
left `fdd14ba0d9b5` active and healthy, with SQLite v3 retaining four rooms
|
|
including room `1`; the current Screener, LiveKit, coturn, and nginx services
|
|
each report `NRestarts=0`. Host display-name remains source-only/not deployed.
|
|
No selected-edge TURN enablement occurred.
|
|
|
|
The all-room routing and selected-edge configuration cutover used exact source
|
|
`cf149df2411798cd632cc92a562b0a35146e0c1b` and immutable artifact
|
|
`screener-cf149df2411798cd632cc92a562b0a35146e0c1b-20260820T173812Z-git.tar.gz`
|
|
(893,953 bytes, SHA-256
|
|
`80918716aefdbb958289b06b6f31551b78dbbaf7fb0f16980eb38d127e158e14`). The
|
|
single lock-held run started at 2026-08-20T17:48:24.554Z and finished at
|
|
2026-08-20T17:48:33.809Z (9,256.904 ms). Screener stopped at
|
|
17:48:29.425Z, the symlink switched at 17:48:29.478Z, and local health was
|
|
ready at 17:48:29.988Z: 562.657 ms from stop, or 509.983 ms from switch.
|
|
The two transient local connection-refused probes were within that normal
|
|
restart window; no browser or media benchmark was run.
|
|
|
|
The final release removed `PEER_ASSISTED_ROOM_IDS`, set
|
|
`PEER_ASSISTED_MEDIA=true` for every room, and configured one selected-edge
|
|
TURN/UDP URL with a 120-second TTL. Ordinary peer ICE remained STUN-only; the
|
|
coturn, nginx, LiveKit, firewall, and listener baselines were preserved, while
|
|
the application environment changed only for the selected tuple and retained
|
|
root ownership and mode 0600. Local/public health, the built asset, three neutral
|
|
routes, and the anonymous site-access boolean passed. SQLite v3 integrity,
|
|
owner/mode, checksum, five-room count, and room `1` were unchanged.
|
|
Screener, LiveKit, coturn, and nginx were active/running with
|
|
`NRestarts=0` at the final audit. The selected-edge tuple was not exercised
|
|
by a real TURN session and no SFU frame or quality result was claimed.
|
|
|
|
This release also carries the Web Host display-name path and the best-effort
|
|
window-scoped display-audio request hint. Native H.264 opt-in source remains
|
|
available in the repository, but no packaged native sender was deployed; VP8
|
|
remains the Web default. The prior stale `NRestarts=31` rollback is historical
|
|
evidence only, and the final wrapper records restart counts without asserting
|
|
an obsolete baseline.
|
|
|
|
The subsequent flagship cutover used exact source
|
|
`c27df2235ecc0be17816f642ca49028e30380a34` and immutable artifact
|
|
`screener-c27df2235ecc0be17816f642ca49028e30380a34-20260820T190108Z-git.tar.gz`
|
|
(912,999 bytes, SHA-256
|
|
`1c7f06e2608414f0a4facf8577c8b49a8fefb831e20826c0e23fa0d1e85240d6`).
|
|
The one lock-held cutover ran from 2026-08-20T19:03:10.569Z to
|
|
19:03:11.232Z: 662 ms total, 552 ms from service stop to local health, and
|
|
543 ms from symlink switch to local health. The environment and SQLite
|
|
checksums were unchanged; SQLite v3 retained five rooms including room `1`
|
|
with owner/mode `screener:screener`/0600. Local/public health, the built asset,
|
|
root, and room route returned 200, all four services remained active/running,
|
|
and exact `cf149df2411798cd632cc92a562b0a35146e0c1b` became the rollback release.
|
|
No browser, TURN, SFU, or media canary was run, so this is deployment/config
|
|
evidence only.
|
|
|
|
Enabling persistence does not migrate rooms that existed only in memory. The
|
|
deployment restart invalidates those temporary links; the first subsequently
|
|
created persistent room receives ID `1`.
|
|
|
|
Production startup requires one to eight syntactically valid `stun:` URLs
|
|
before the server listens. This validates shape only; it does not prove DNS,
|
|
firewall, NAT mappings, or external STUN reachability. The current application
|
|
rejects `TURN_URLS`, `TURN_SHARED_SECRET`, or `TURN_CREDENTIAL_TTL_SECONDS` when
|
|
any key is present, including with an empty value. Remove those obsolete keys
|
|
from the candidate environment; startup fails instead of pretending they enable
|
|
compatibility.
|
|
|
|
The rejected candidate names `PEER_ICE_TURN_URLS`,
|
|
`PEER_ICE_TURN_SHARED_SECRET`, and
|
|
`PEER_ICE_TURN_CREDENTIAL_TTL_SECONDS` must remain absent from production and
|
|
now fail startup even when blank. They are not aliases for the selected-edge
|
|
tuple; ordinary peer edges remain STUN-only while only the current controller
|
|
edge may use the configured selected-edge transport.
|
|
|
|
## HTTPS and WSS ingress
|
|
|
|
Terminate public TLS at Caddy or nginx and proxy every application path to the
|
|
single Node process. `/signal` must preserve the WebSocket upgrade, `Host`, and
|
|
`Origin`, and its idle/read timeout must be longer than a normal sharing session.
|
|
Do not cache `/api/*` responses. Caddy's minimal same-host form is:
|
|
|
|
```caddyfile
|
|
share.example.com {
|
|
reverse_proxy 127.0.0.1:8787
|
|
}
|
|
```
|
|
|
|
An equivalent nginx location is:
|
|
|
|
```nginx
|
|
location / {
|
|
proxy_pass http://127.0.0.1:8787;
|
|
proxy_http_version 1.1;
|
|
proxy_set_header Host $host;
|
|
proxy_set_header Upgrade $http_upgrade;
|
|
proxy_set_header Connection "upgrade";
|
|
proxy_read_timeout 1h;
|
|
}
|
|
```
|
|
|
|
The tracked
|
|
[`share.bonfire.icu.conf.example`](../deploy/nginx/share.bonfire.icu.conf.example)
|
|
follows this simple model and listens directly on public TCP 443. It does not
|
|
need an SNI transport router. Exact `/rtc` and pinned-v0 `/rtc/validate` requests
|
|
return 404 with access and request-line error logging disabled; only `/rtc/v1*`
|
|
is proxied. Current versioned requests carry a short-lived LiveKit JWT in the
|
|
query string, so no `/rtc*` target may fall through to the site's ordinary logs.
|
|
|
|
When Certbot manages the Web certificate, install the tracked
|
|
[`reload-nginx.sh`](../deploy/certbot/reload-nginx.sh) as an executable under
|
|
`/etc/letsencrypt/renewal-hooks/deploy/`. Run `certbot renew --dry-run` after
|
|
the first certificate is installed. This hook reloads only the Web ingress.
|
|
|
|
Keep the proxy's access-log retention bounded and access controlled. Requests to
|
|
`/r/{code}` put the room code in the path, so access logs can contain room codes
|
|
as well as network metadata. They must not be treated as public artifacts. For
|
|
a private room, the grant remains in the fragment and is not part of that
|
|
request target. A public-watch code still requires prior site access.
|
|
|
|
For a process-level liveness probe, send `GET /healthz`. A running process
|
|
returns HTTP 200 with `{"status":"ok"}` and `Cache-Control: no-store`; other
|
|
methods return 405 with `Allow: GET`. The endpoint does not inspect room state,
|
|
STUN/SFU reachability, or any external dependency, so use it only to decide whether
|
|
the Node process can accept HTTP requests. It is not a deployment-readiness or
|
|
end-to-end media check.
|
|
|
|
## Optional single-node LiveKit fallback
|
|
|
|
Use the tracked
|
|
[`livekit.yaml.example`](../deploy/livekit/livekit.yaml.example) and
|
|
[`livekit.service.example`](../deploy/systemd/livekit.service.example) as the
|
|
minimal single-node baseline. Install a reviewed, pinned LiveKit Server release
|
|
from its official release assets, generate an independent key pair with
|
|
`livekit-server generate-keys`, and place only the YAML key mapping in
|
|
`/etc/livekit/keys.yaml`. Keep that file owned by the `livekit` service account
|
|
with mode `0600`; put the same values in Screener's untracked process secrets.
|
|
|
|
The example deliberately omits Redis and every recording, ingress, egress,
|
|
webhook, external-TURN, and embedded-TURN service. It explicitly sets
|
|
`tcp_port: 0` and `allow_tcp_fallback: false`, and points `stun_servers` at the
|
|
deployment's self-hosted STUN listener so pinned LiveKit cannot inherit its
|
|
default public Google STUN servers. Do not add Redis for this one-node workload.
|
|
The service journal is the diagnostic log; keep its retention finite and access
|
|
restricted. Never enable debug/Pion packet logging continuously or persist JWTs,
|
|
SDP, ICE candidates, API secrets, or full `/rtc` and `/rtc/*` request targets.
|
|
|
|
Configure both the host firewall and the provider firewall independently:
|
|
|
|
| Destination | Protocol | Exposure | Purpose |
|
|
| --- | --- | --- | --- |
|
|
| `127.0.0.1:7880` | TCP | Private | nginx to LiveKit signaling/API |
|
|
| public host `:7882` | UDP | Public | WebRTC ICE/UDP mux |
|
|
|
|
LiveKit may listen beyond loopback while it discovers and advertises public
|
|
media addresses, so the public-firewall denial of TCP 7880 is mandatory; nginx
|
|
must be its only ingress. UDP 7882 is a direct media listener and must not be
|
|
placed behind the HTTP reverse proxy. If the host is behind NAT, forward it
|
|
without port translation and verify the advertised candidate from an external
|
|
network. UDP-blocked networks are expected to fail clearly in this candidate;
|
|
do not set `rtc.tcp_port` above zero, enable TCP fallback, or add TURN to make a
|
|
canary pass silently.
|
|
|
|
## Self-hosted STUN
|
|
|
|
This subsection describes the temporary STUN-only deployment template, not the
|
|
enabled selected-edge production environment or the current shared-host coturn
|
|
state. That host retains an older authenticated-relay configuration and TCP/UDP
|
|
3478 plus UDP 49152-49251 firewall range. In this template Screener advertises
|
|
no TURN credential and the canary audit found zero allocations; production
|
|
selected-edge issuance remains limited to the controller's one configured edge
|
|
and still lacks a real forced-relay media acceptance.
|
|
|
|
Copy [`deploy/coturn/turnserver.conf.example`](../deploy/coturn/turnserver.conf.example)
|
|
to an untracked service-owned location and use the tracked
|
|
[`coturn.service.example`](../deploy/systemd/coturn.service.example). The config
|
|
uses coturn's documented `stun-only` mode, ignores every TURN allocation request,
|
|
and disables TCP and TLS client listeners. It has no auth secret, user database,
|
|
relay port range, certificate, quota, or TURN REST credential. Pinned coturn
|
|
4.17.2 deprecates `no-dtls`, so the tracked config does not use that obsolete
|
|
switch or claim a separate DTLS listener gate.
|
|
|
|
On a single-homed public host, automatic listener selection is sufficient. On a
|
|
multi-homed host, set `listening-ip` to the intended public interface. Open only
|
|
UDP 3478 and keep TCP 3478/5349 plus every old relay range closed. The same public
|
|
IP may serve HTTPS/WSS on TCP 443 and LiveKit media on UDP 7882.
|
|
|
|
The example writes informational logs to stdout. Apply finite supervisor
|
|
retention and restrict access because STUN sees client IP addresses. Do not
|
|
enable `verbose` or `log-binding` continuously. `stun-only` is a hard candidate
|
|
boundary, not a hidden unauthenticated TURN service.
|
|
|
|
Open only these public listeners:
|
|
|
|
| Destination | Protocol | Purpose |
|
|
| --- | --- | --- |
|
|
| `share.example.com:443` | TCP | HTTPS and WSS |
|
|
| `stun.example.com:3478` | UDP | STUN binding only |
|
|
| host public address `:7882` | UDP | LiveKit WebRTC ICE/UDP mux |
|
|
|
|
## Verification
|
|
|
|
Run these checks from real external networks before calling the deployment usable:
|
|
|
|
1. Open the official [Trickle ICE sample](https://webrtc.github.io/samples/src/content/peerconnection/trickle-ice/)
|
|
and test the configured STUN URL. It must produce an `srflx` candidate over
|
|
UDP and no `relay` candidate. Confirm TCP 3478/5349 and the old relay range
|
|
are closed externally.
|
|
2. Create a normal Screener room on two different networks. Confirm media flows
|
|
and the selected-pair stats report a non-relay path when direct ICE succeeds.
|
|
3. On a normal room, exhaust a peer route and verify the host plus at
|
|
most two necessary roots select LiveKit UDP 7882. Peer descendants stay on
|
|
ordinary direct UDP and host/relay downstream caps remain two/one.
|
|
4. Block all UDP on one test client. Verify bounded ICE recovery ends in a clear
|
|
connection failure, without an ICE/TCP, TURN/TCP, or long pseudo-connected
|
|
path. No force-relay product branch exists in this candidate.
|
|
5. Exercise root departure, reconnect, SFU unavailable, route prepare rollback,
|
|
stop, and source/profile changes. Unaffected subtrees must not migrate.
|
|
6. Repeat at 1, 3, 5, and 8 viewers across representative consumer networks.
|
|
Record selected protocol, RTT, bitrate, frame rate, packet loss, publisher
|
|
upload/encode load, LiveKit ingress/egress, and final decoded quality. A real
|
|
1:8 session remains unverified until this matrix passes.
|
|
|
|
References: [coturn 4.17.2 release](https://github.com/coturn/coturn/releases/tag/4.17.2),
|
|
[pinned turnserver documentation](https://github.com/coturn/coturn/blob/4.17.2/README.turnserver),
|
|
the [pinned example configuration](https://github.com/coturn/coturn/blob/4.17.2/examples/etc/turnserver.conf),
|
|
[RFC6265bis](https://datatracker.ietf.org/doc/draft-ietf-httpbis-rfc6265bis/),
|
|
[LiveKit firewall guidance](https://docs.livekit.io/deploy/admin/firewall/), and
|
|
the [pinned LiveKit configuration sample](https://github.com/livekit/livekit/blob/v1.13.5/config-sample.yaml).
|