diff --git a/docs/RUNBOOK.md b/docs/RUNBOOK.md new file mode 100644 index 0000000..1d309fc --- /dev/null +++ b/docs/RUNBOOK.md @@ -0,0 +1,93 @@ +# Mark VIII Field Runbook + +Operational procedures for running Mark VIII v1.4.0+ on a WiFi Pineapple +Pager. The reliability subsystem is documented in the README; this file is +the how-to-run-it companion. + +## Pre-engagement checklist (2 minutes) + +From the Dashboard health panel (or `scripts/smoke.sh` over SSH): + +- [ ] Health chip green; guard chip shows `GUARD OK` (not PENDING). +- [ ] Both monitors present (`wlan0mon`, `wlan1mon`). +- [ ] Reliability counters sane: no unexpected boots since last check; + rollbacks/restarts at expected values. +- [ ] Recent events feed shows a clean `boot` entry for this session. +- [ ] RF plan as intended: `PHY0: OPS`, `PHY1:` in the state you want. + +If anything fails: run `sh /tmp/smoke.sh` (re-upload via +`scp scripts/smoke.sh root@:/tmp/` if tmpfs was cleared) and read +which check fails before deploying to a target environment. + +## During an engagement + +- **Prefer phy1 for any uplink** (`POST /api/rfplan/role {"role":"uplink", + "ssid":...,"psk":...}`). phy0 stays fully hoppable for 2.4 GHz work. + Expect 5 GHz recon limited to the uplink's channel while associated. +- **Switching back**: `{"role":"attack"}` tears the STA down with a gated + reload; verify the RF chip flips and wlan1mon hops again. +- **Watch the events feed.** Occasional `guard_fix` entries are the + supervisor healing stock-daemon regressions (STA re-parks, UCI + re-applies) — normal. A stream of them means the stock UI is fighting + you: stop touching the stock PineAP page mid-engagement. +- **Rollback watchdogs** arm around risky operations. If the web UI dies + after an operation, wait ~60 s: the watchdog restores the pre-op snapshot + automatically or promotes the change once healthy. Do NOT power-cycle + before that window passes. +- **Never** hand-edit `/etc/config/pineapd` mid-engagement; use Mark VIII + endpoints. Hand edits race the stock daemon and the reconciler. + +## After an engagement + +1. Stop attacks and switch phy1 to `idle`. +2. Export loot (Handshakes export, HTML/CSV reports). Loot lives under + `/root/loot/**` and survives reboots — pull it off-box anyway. +3. Save a profile (Settings → Reliability), e.g. `post--`. +4. Skim events for `rollback` / `mem_warn` entries worth noting. + +## Recovery procedures + +### Web UI unreachable but device seems alive + +Wait 60 s — a rollback watchdog may be mid-restore. Then SSH: + + curl -fsS http://127.0.0.1:8080/ >/dev/null && echo up + +If still down, check whether a watchdog is running (`ps | grep watchdog`) +and let it finish; if it already rolled back but the UI did not return, +restart the service: `/etc/init.d/pagerwebui start`. + +### Roll everything back to a known-good state + +Settings → Reliability → restore `lastknown-good` (auto-captured whenever +the system has been healthy for 5 minutes), or over SSH: + + python3 /mmc/mk8/releases/current/user/remote_access/pager-webui/server.py \ + --rollback-snapshot lastknown-good + +### After a firmware upgrade or factory reset + +The overlay is wiped; Mark VIII is gone but `/mmc/mk8/` (profiles, +journal, releases) survives. Reinstall: + + ./scripts/deploy.sh --password '' + +The deploy re-installs mk8-guard + service from your release; profiles and +history are still there. Verify with `sh /tmp/smoke.sh`. + +### Device totally unreachable + +Power cycle. On boot, mk8-guard re-applies safe PineAP UCI before pineapd +starts, so the device comes back crash-free even if the stock daemon had +been left in a bad state. If SSH is still refused after full boot, use the +Hak5 first-boot recovery for the firmware image you are on. + +## Known limitations + +- Role drill in smoke needs a real AP: `SMOKE_UPLINK_SSID=... [--write]`. +- While phy1 carries an uplink, 5 GHz recon sees only that channel + (radio physics, not a bug). +- Idle monitors park on one channel between scans; hopping resumes during + recon scans (verified: ch48 -> ch157 -> ch40). +- Journal is capped at 4 x 5 MB on /mmc; counters read the newest ~5000 + entries.