Deploy staging

Terminal and this documentation are static Cloudflare Pages sites. Mainframe staging runs Core, Postgres, SRS, the media indexer, Caddy, world-sim, auto-operator, and monitoring on its own Hetzner server. DVR and review objects use a private Hetzner S3 bucket. It uses real credentials and the normal authentication path.

Surface	Address
Terminal	https://terminal.dev.mainline.inc
Mainframe	https://mainframe.dev.mainline.inc
Documentation	https://docs.terminal.dev.mainline.inc/
Grafana	https://mainframe.dev.mainline.inc/grafana/
Simulator administration	https://mainframe.dev.mainline.inc/sim/admin

These are the configured targets. DNS and TLS must be provisioned before the addresses will work.

1. Provision Pages, the host, and bucket

The separate infra repository owns the Git-integrated Pages projects (terraform/terminal-pages.tf), custom domains, Hetzner server, firewall, and object storage (terraform/terminal-staging.tf). Its Terminal staging resources reuse the account, SSH keys, and providers used by the CC clusters while creating a separate server and private bucket.

Review the Terraform plan before applying it. Use the resulting server IP for PUBLIC_IP. Follow capacity planning when choosing server and disk sizes. The server needs Docker with Compose 2.24.4 or newer and Bun 1.4.2 for the deployment launcher.

Public traffic uses TCP 80 and 443 for HTTP/TLS, UDP and TCP 8000 for WebRTC in staging (MEDIA_RTC_PORT; the default is 8000), and TCP 1935 for authenticated RTMP publishing. SSH access should match the administrator CIDRs configured in Terraform. Postgres, SRS’s API, Prometheus, and exporters are not public services.

Static builds

Both Pages projects use the repository root, Bun 1.4.2, SKIP_DEPENDENCY_INSTALL=1, and the main production branch. Every push to main rebuilds and publishes both sites without touching Docker. Pull requests receive separate preview builds.

Project	Build command	Output
terminal-staging	bun install --frozen-lockfile && bun run --cwd apps/web build	apps/web/dist
terminal-docs-staging	bun install --frozen-lockfile && bun run --cwd docs build	docs/dist

Terminal’s build environment sets VITE_MAINFRAME_URL=https://mainframe.dev.mainline.inc. Mainframe continues to allow the Terminal origin through its configured CORS/auth settings; preview origins are not automatically trusted for staging login. The old Terminal /docs/ URLs redirect to the documentation domain.

The Hetzner console name is mainframe-staging. Its Terraform resource address, /opt/terminal checkout, terminal-staging Compose project, and existing volumes retain their original identities. Renaming the console label does not change the public IP or DNS.

2. Prepare the backend checkout

Place the reviewed Terminal checkout on the server, then run these commands from its root:

bun install --frozen-lockfile
bun run deploy init

This creates deploy/.env.staging with independent random credentials and mode 0600. It refuses to overwrite an existing environment. Set the server IP and S3 credentials, and review the account emails and domains. Keep this file outside version control.

The bucket endpoint and region must match, for example https://hel1.your-objectstorage.com and hel1. The bucket must already exist. Leave BLOB_PRESIGN=0 to serve authenticated playback through Mainframe without configuring public bucket CORS.

3. Validate and launch

bun run deploy config
bun run deploy up
bun run deploy ps

The launcher combines the base, monitoring, and staging Compose files. It validates required fields, enables the selected Core module identities, generates Prometheus scrape targets, builds backend images, and waits for service health checks. Caddy uses the stock image and serves only Mainframe, module routes, and monitoring; it does not build Terminal or docs. A normal rebuild preserves named volumes.

If local map assets are required, build them before deployment using the repository’s map tooling. The stack can otherwise use the frontend’s configured online basemap.

4. Seed the virtual fleet

bun run deploy seed
bun run deploy seed --apply

The first command shows missing records. The second creates them through Mainframe’s authenticated document and fleet APIs. It adds ten SIM-STG-001 through SIM-STG-010 aircraft, one staging flight, four short patrol loops, the border and geofence markings, and non-dispatching watch rules. Re-running preserves existing documents and fleet identities.

Auto-operator runs in auto mode with AUTO_OPERATOR_SERIAL_PREFIX=SIM-STG- and AUTO_OPERATOR_PROVIDER_IDS=world-sim. Mainframe independently checks the same selection. Real aircraft from another provider cannot match it. The aircraft start at different battery levels; four missions leave six aircraft available for charging and rotation.

5. Verify the deployment

Sign in to Terminal with the admin credentials in the environment file, then open Settings → Access → Account and enable two-factor authentication before anyone else signs in; Security → Two-factor requirement set to Admins enforces it for every later administrator. Create the people’s accounts in Settings → Access → Users: each gets a temporary password shown once and must change it at the first sign-in. Confirm the virtual fleet appears, missions become assigned, telemetry advances, and a live feed plays. After at least one DVR segment closes, enter replay and confirm video and positions match the selected time.

In Grafana, inspect service availability, module connections, S3 uploads, simulator publishers, automation freshness, and mission coverage. Check an actual charge → dispatch → return → recharge cycle before calling the continuous run accepted. A long replay history requires a real soak run; an initial healthy boot proves only that the stack started.

Use updates and backups for subsequent releases and real aircraft before enabling the autel or vanagas profiles.

Lithuania basemap assets

The map directory is excluded from Git and must be populated on each new host:

bun scripts/mapctl.ts fetch --area lt --name lt
bun scripts/mapctl.ts assets

Run from /opt/terminal. This installs the full Lithuania extract plus the fonts and sprites in deploy/map, mounted read-only at Caddy’s /srv/map. --area lt matters: the default ao only covers the southeast operating area. Caddy serves /map/* directly with Range and CORS support; native Autel /map/api/v1/* routes continue to go to Autel.

Verify a range request to /map/lt.pmtiles returns 206 with PMTiles magic, and that /map/fonts/Noto%20Sans%20Regular/0-255.pbf and /map/sprites/v4/black.json return 200. A frontend rebuild is unnecessary when installing basemap assets; reload the browser to retry a cached fallback.

Park the simulator for real test operations

On the staging host, from /opt/terminal:

bun deploy/park-world.ts
bun deploy/park-world.ts --apply

The first command previews the affected world-sim entities. Apply saves a Postgres dump, environment copy and identity inventory in deploy/backups/park-world-<timestamp>/, stops world-sim and its simulator-only auto-operator, and removes world and automation from COMPOSE_PROFILES. It retires the provider’s entities through Mainframe’s provider protocol, updating connected clients without restarting Mainframe. No real provider’s entities are changed. The command can be rerun after a partial failure.

Retired aircraft are hidden by default in Assets and Fleet; simulator sites, radars and tracks also leave the live registry. Missions, rules, airframes, recordings and the simulator volume are preserved. Existing history/media retention still applies; parking is not an indefinite archival hold.

To bring the simulator back, add world to the existing COMPOSE_PROFILES line in deploy/.env.staging, preserving every other enabled profile, then run:

bun run deploy up
bun run deploy logs world-sim

World-sim reads its retained volume and announces its assets again. Mainframe reactivates existing identities (or creates them if normal retention has already pruned them). Do not clear volumes or reseed merely to restart. Leave automation disabled for an idle simulator. Add it back only when automatic simulator mission dispatch is wanted, keeping AUTO_OPERATOR_PROVIDER_IDS=world-sim and AUTO_OPERATOR_SERIAL_PREFIX=SIM-STG-. Verify world-sim health, live assets and video after restart; restarting was intentionally not part of parking the simulator.