Skip to content

Seed and clear data

Data operations preview by default. Seeding uses a live, authenticated Mainframe. Clearing runs offline with a database lock. Neither requires enabling development fixtures.

Terminal window
bun run deploy seed staging
bun run deploy seed staging --apply

For a Mainframe reachable directly, run the application tool with admin credentials in the environment:

Terminal window
bun run data seed --profile staging --url https://mainframe.dev.mainline.inc
bun run data seed --profile staging --url https://mainframe.dev.mainline.inc --apply

MAINFRAME_ADMIN_EMAIL and MAINFRAME_ADMIN_PASSWORD authenticate the operation. Use the current password if it has changed since bootstrap. Remote requests require HTTPS.

Seed files live in apps/mainframe/seeds/staging/ and seeds/dev/. The tool validates their document and fleet schemas, adds missing IDs, and waits for world-sim to confirm each new flight or aircraft. It never targets a hardware fleet. Existing revisions and aircraft are preserved. A retired or offline conflicting identity requires investigation instead of an automatic replacement.

Staging deliberately leaves mission launching to the simulator’s operator; Core dispatches only what a person or a module asks for.

Take a backup and stop the deployment before applying a clear:

Terminal window
bun run deploy backup staging
bun run deploy down staging

Bring up only Postgres with the same installation, then preview the clear. The rendered files keep the installation’s project, so the volumes are the same:

Terminal window
bun run deploy compose staging -- up -d postgres
bun run deploy clear staging --scope history

The preview prints the database identity and row counts. To apply, add --apply --confirm followed by that exact database@host:port identity. The tool refuses the operation while a running Mainframe holds its advisory lock.

Scope Cleared
history Samples, snapshots, rollups, events, and command audit rows
operational Everything above, plus missions, markings, alerts, airframe definitions, and their document history

Both scopes preserve users, roles, grants, provider keys, entity identities, stream sequences, media/blob indexes, captures, clips, chat messages, and map catalogs. Recorded video remains in the bucket under media/, capture stills under blobs/captures/, rendered clips with their posters under blobs/clips/ and chat attachments under blobs/chat/; clips are removed by the daily run once older than Mainframe Settings → Configuration → Media → Keep clips, and chat messages (the chat_messages table) with their attachments once older than Mainframe Settings → Configuration → Comms → Keep messages. Use media retention when you intend to delete recordings; do not erase the index while leaving unmanaged objects behind.

Restart with bun run deploy up staging. Seed explicitly if you cleared operational documents. World-sim’s persisted fleet remains separate. Its factory-reset operation removes the simulator overlay; it does not clear Mainframe’s database. Do not reset one side and assume the other was reset too.

clear leaves recordings in the bucket. To start an installation over (database history, recorded video, captures, clips, chat, voice recordings, sensor archives and the recorders’ local volumes) while keeping its users, access, settings, layouts, plans, markings and entity identities, run the wipe on the installation’s host, from /opt/terminal:

Terminal window
bun run deploy wipe staging
bun run deploy wipe staging --apply --confirm terminal-staging/mainline-terminal-dev

The preview prints what will be truncated, how many history documents go and which volumes are emptied, and the exact --confirm value (project/bucket). Apply then:

  1. records the running containers and the backup, TLS and recording units, and stops them (Postgres stays up);
  2. fingerprints every kept table and saves them with pg_dump to deploy/backups/wipes/<project>/configuration.dump;
  3. lists the bucket into objects.sqlite (everything except downloads/, backups/ and keys a kept row references);
  4. clears history in one transaction, with Mainframe’s boot guard set until the wipe has verified the kept tables;
  5. empties the recorder volumes in place, verifies the fingerprints again, lifts the guard and starts the recorded containers and units;
  6. deletes the listed objects in the background, as the systemd unit terminal-wipe-purge (log: purge.log next to the journal).

The installation is back in minutes; the object purge can take hours on Hetzner (one delete per key, retried indefinitely). New recordings never reuse an old key, so the purge runs safely beside them. Every stage is journalled in deploy/backups/wipes/<project>/journal.json: rerun the same command to resume an interrupted wipe, or bun run deploy wipe staging --purge to resume only the purge. When the purge finishes the directory is renamed with the wipe’s id, so the next wipe starts fresh. --no-restart lifts the guard but leaves the stack stopped.

The wipe refuses archives and installations under a preservation hold, a database carrying another maintenance marker, and any table deploy/wipe/plan.ts does not classify. A migration that adds a table must add it there (as kept, cleared or filtered); deploy/wipe/plan.test.ts fails until it does.

At boot Mainframe tombstones stored documents it can no longer read: anything left in the rules collection, captures a watch rule took or annotated, and markings of an unknown kind (such as border). It retires environment entities a weather module left behind; the retention prune removes them later. Settings sections drop fields they no longer have and keep the rest. Saved layouts, local and shared, load without panels this Terminal no longer has.

Seed production documents without a simulator

Section titled “Seed production documents without a simulator”

Use a reviewed directory containing any of layouts.json, markings.json, missions.json, and defs.airframes.json, and for the air picture defs.effectors.json and surveillance.{profiles,config,whitelist,utm,sources}.json (the staging profile carries all of them). Each file is an array of documents using the normal collection schema. Omitted collections are left alone. Mainframe itself seeds any of the four starting layouts it has never stored at boot, in every profile and in archive mode (a deleted one stays deleted), and replaces one stored in the old single-arrangement shape; the bundled profiles carry them too, and seeding preserves existing edits and deleted layouts. The layouts are Feeds (one configuration), Pilot (three), Operations (three), and Analysis (two). New workspaces open Feeds by default; existing workspace selections stay unchanged.

Terminal window
bun run data seed --url https://mainframe.example.com --seed-dir ./production-seed --documents-only
bun run data seed --url https://mainframe.example.com --seed-dir ./production-seed --documents-only --apply

This mode does not require world-sim or create aircraft. Keep admin credentials in the environment or supported secret files. It uses the same authentication, schema validation, audit trail and revision checks as Terminal; it preserves existing IDs and deliberate deletions. Perform offline history maintenance when aircraft are idle: clearing snapshots also removes the last persisted tasking state.