Files
Mark-VIII/docs/RUNBOOK.md

3.9 KiB

Mark VIII Field Runbook

Operational procedures for running Mark VIII v1.4.0+ on a WiFi Pineapple Pager. The reliability subsystem is documented in the README; this file is the how-to-run-it companion.

Pre-engagement checklist (2 minutes)

From the Dashboard health panel (or scripts/smoke.sh over SSH):

  • Health chip green; guard chip shows GUARD OK (not PENDING).
  • Both monitors present (wlan0mon, wlan1mon).
  • Reliability counters sane: no unexpected boots since last check; rollbacks/restarts at expected values.
  • Recent events feed shows a clean boot entry for this session.
  • RF plan as intended: PHY0: OPS, PHY1: in the state you want.

If anything fails: run sh /tmp/smoke.sh (re-upload via scp scripts/smoke.sh root@<ip>:/tmp/ if tmpfs was cleared) and read which check fails before deploying to a target environment.

During an engagement

  • Prefer phy1 for any uplink (POST /api/rfplan/role {"role":"uplink", "ssid":...,"psk":...}). phy0 stays fully hoppable for 2.4 GHz work. Expect 5 GHz recon limited to the uplink's channel while associated.
  • Switching back: {"role":"attack"} tears the STA down with a gated reload; verify the RF chip flips and wlan1mon hops again.
  • Watch the events feed. Occasional guard_fix entries are the supervisor healing stock-daemon regressions (STA re-parks, UCI re-applies) — normal. A stream of them means the stock UI is fighting you: stop touching the stock PineAP page mid-engagement.
  • Rollback watchdogs arm around risky operations. If the web UI dies after an operation, wait ~60 s: the watchdog restores the pre-op snapshot automatically or promotes the change once healthy. Do NOT power-cycle before that window passes.
  • Never hand-edit /etc/config/pineapd mid-engagement; use Mark VIII endpoints. Hand edits race the stock daemon and the reconciler.

After an engagement

  1. Stop attacks and switch phy1 to idle.
  2. Export loot (Handshakes export, HTML/CSV reports). Loot lives under /root/loot/** and survives reboots — pull it off-box anyway.
  3. Save a profile (Settings → Reliability), e.g. post-<site>-<date>.
  4. Skim events for rollback / mem_warn entries worth noting.

Recovery procedures

Web UI unreachable but device seems alive

Wait 60 s — a rollback watchdog may be mid-restore. Then SSH:

curl -fsS http://127.0.0.1:8080/ >/dev/null && echo up

If still down, check whether a watchdog is running (ps | grep watchdog) and let it finish; if it already rolled back but the UI did not return, restart the service: /etc/init.d/pagerwebui start.

Roll everything back to a known-good state

Settings → Reliability → restore lastknown-good (auto-captured whenever the system has been healthy for 5 minutes), or over SSH:

python3 /mmc/mk8/releases/current/user/remote_access/pager-webui/server.py \
    --rollback-snapshot lastknown-good

After a firmware upgrade or factory reset

The overlay is wiped; Mark VIII is gone but /mmc/mk8/ (profiles, journal, releases) survives. Reinstall:

./scripts/deploy.sh --password '<device password>'

The deploy re-installs mk8-guard + service from your release; profiles and history are still there. Verify with sh /tmp/smoke.sh.

Device totally unreachable

Power cycle. On boot, mk8-guard re-applies safe PineAP UCI before pineapd starts, so the device comes back crash-free even if the stock daemon had been left in a bad state. If SSH is still refused after full boot, use the Hak5 first-boot recovery for the firmware image you are on.

Known limitations

  • Role drill in smoke needs a real AP: SMOKE_UPLINK_SSID=... [--write].
  • While phy1 carries an uplink, 5 GHz recon sees only that channel (radio physics, not a bug).
  • Idle monitors park on one channel between scans; hopping resumes during recon scans (verified: ch48 -> ch157 -> ch40).
  • Journal is capped at 4 x 5 MB on /mmc; counters read the newest ~5000 entries.