docs: plan — reliability core implementation
This commit is contained in:
@@ -0,0 +1,983 @@
|
||||
# Mark VIII Reliability Core Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Make the Pager run reliably and consistently as expected via payload-only hardening: boot-time guard, config profiles with rollback watchdogs, RF role manager (uplink on phy1), integrated supervisor with event journal, atomic deploys, and an on-device smoke suite.
|
||||
|
||||
**Architecture:** Three layers inside the existing payload — `mk8-guard` init script (START=49, before the S50 pineapple stack), new `mk8_*.py` stdlib modules imported by `server.py`, and a passive supervisor thread extending the existing health monitor. Persistent state in `/mmc/mk8/` (survives reboots and overlay wipes).
|
||||
|
||||
**Tech Stack:** Python 3 stdlib only (`python3-light` on device: no urllib/http.server/sqlite3 modules), POSIX sh for device scripts, vanilla JS frontend, Bash + sshpass/scp for deploy tooling.
|
||||
|
||||
## Global Constraints
|
||||
|
||||
- Spec: `docs/superpowers/specs/2026-08-22-reliability-core-design.md` (approved).
|
||||
- Device constraints: python3-light stdlib only; BusyBox (no `pkill`; use `killall`/`pidof`); never actively ping pineapd's command socket from loops; never run `wifi reload` outside gated operations.
|
||||
- All persistent state under `/mmc/mk8/`. No writes to stock binaries or `/etc/config` outside reconciler/gated ops.
|
||||
- Tests: stdlib `unittest`, one module per process (tests monkeypatch module state); run pattern:
|
||||
`python3 -m unittest tests.test_<name> -v`
|
||||
- Frontend checks: `node --check <file>` after every JS edit.
|
||||
- Device access is **read-only until Task 10** (deploy + smoke). Password auth via `sshpass -p '<pw>' ssh -o StrictHostKeyChecking=no root@172.16.52.1`.
|
||||
- Branch: `feature/reliability`. Commit after every passing step.
|
||||
- Version: single-source `VERSION` file at repo root; next version `1.4.0`.
|
||||
|
||||
## File Structure
|
||||
|
||||
```
|
||||
payload/user/remote_access/pager-webui/
|
||||
mk8_events.py NEW event journal (JSONL append/rotate/read + counters)
|
||||
mk8_profiles.py NEW UCI snapshot store (/mmc/mk8/profiles)
|
||||
mk8_guard.py NEW known-good invariants + reconcile() + CLI hooks
|
||||
mk8_rfplan.py NEW phy1 RF role manager (uplink/attack/idle)
|
||||
mk8_gate.py NEW risky-op preflight gate + watchdog decision logic
|
||||
mk8-watchdog.sh NEW detached local-liveness rollback watchdog
|
||||
mk8-guard.init NEW START=49 boot guard script (installed to /etc/init.d/mk8-guard)
|
||||
server.py MOD imports, startup hook, h_health extension, /api/reliability/* + /api/rfplan/* routes, gates on risky handlers
|
||||
www/js/views.js MOD health events feed, reliability counters, RF chip, Settings profiles card
|
||||
www/js/app.js MOD nav wiring if needed
|
||||
www/css/app.css MOD styles for new UI elements
|
||||
scripts/deploy.sh MOD VERSION stamping, atomic release swap, post-deploy check, guard install
|
||||
scripts/smoke.sh NEW on-device verification suite
|
||||
VERSION NEW "1.4.0"
|
||||
tests/test_mk8_events.py, test_mk8_profiles.py, test_mk8_guard.py,
|
||||
tests/test_mk8_gate.py, test_mk8_rfplan.py, test_reliability_api.py NEW
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Event journal (`mk8_events.py`)
|
||||
|
||||
**Files:**
|
||||
- Create: `payload/user/remote_access/pager-webui/mk8_events.py`
|
||||
- Test: `tests/test_mk8_events.py`
|
||||
|
||||
**Interfaces:**
|
||||
- Produces: `log_event(kind, sev='info', msg='', meta=None)`; `read_events(limit=100)` → list of dicts newest-first; `counters()` → dict with keys `boots`, `unexpected_boots`, `rollbacks`, `restarts`, `guard_fixes`; `mark_boot()`; constants `MK8_DIR='/mmc/mk8'`, `EVENTS_PATH`, `MAX_BYTES=5*1024*1024`, `KEEP=4`.
|
||||
|
||||
- [ ] **Step 1: Write failing tests**
|
||||
|
||||
```python
|
||||
import json, os, sys, tempfile, unittest
|
||||
sys.path.insert(0, os.path.join(os.path.dirname(__file__), '..', 'payload', 'user', 'remote_access', 'pager-webui'))
|
||||
import mk8_events
|
||||
|
||||
class EventsTest(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.dir = tempfile.mkdtemp()
|
||||
self.old = mk8_events.MK8_DIR
|
||||
mk8_events.MK8_DIR = self.dir
|
||||
mk8_events.EVENTS_PATH = os.path.join(self.dir, 'events.log')
|
||||
|
||||
def tearDown(self):
|
||||
mk8_events.MK8_DIR = self.old
|
||||
|
||||
def test_log_and_read_newest_first(self):
|
||||
mk8_events.log_event('boot', msg='first')
|
||||
mk8_events.log_event('rollback', sev='warn', msg='second', meta={'op': 'wifi'})
|
||||
rows = mk8_events.read_events()
|
||||
self.assertEqual(rows[0]['kind'], 'rollback')
|
||||
self.assertEqual(rows[1]['kind'], 'boot')
|
||||
self.assertEqual(rows[0]['meta'], {'op': 'wifi'})
|
||||
|
||||
def test_counters(self):
|
||||
mk8_events.log_event('boot'); mk8_events.log_event('rollback')
|
||||
mk8_events.log_event('restart'); mk8_events.log_event('guard_fix')
|
||||
c = mk8_events.counters()
|
||||
self.assertEqual(c['boots'], 1)
|
||||
self.assertEqual(c['rollbacks'], 1)
|
||||
self.assertEqual(c['restarts'], 1)
|
||||
self.assertEqual(c['guard_fixes'], 1)
|
||||
|
||||
def test_rotation_keeps_recent(self):
|
||||
mk8_events.MAX_BYTES = 200
|
||||
for i in range(20):
|
||||
mk8_events.log_event('tick', msg='x' * 30)
|
||||
self.assertTrue(len(mk8_events.read_events()) >= 15)
|
||||
self.assertFalse(os.path.exists(mk8_events.EVENTS_PATH + '.4'))
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run to verify failure**
|
||||
|
||||
Run: `python3 -m unittest tests.test_mk8_events -v`
|
||||
Expected: FAIL — `No module named 'mk8_events'`
|
||||
|
||||
- [ ] **Step 3: Implement**
|
||||
|
||||
```python
|
||||
"""Mark VIII reliability event journal. JSONL on /mmc, rotated."""
|
||||
import json, os, threading
|
||||
|
||||
MK8_DIR = '/mmc/mk8'
|
||||
EVENTS_PATH = os.path.join(MK8_DIR, 'events.log')
|
||||
MAX_BYTES = 5 * 1024 * 1024
|
||||
KEEP = 4
|
||||
_LOCK = threading.Lock()
|
||||
|
||||
COUNTER_KINDS = ('boot', 'unexpected_boot', 'rollback', 'restart',
|
||||
'guard_fix')
|
||||
|
||||
|
||||
def _ensure_dir():
|
||||
try:
|
||||
os.makedirs(MK8_DIR, exist_ok=True)
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
|
||||
def log_event(kind, sev='info', msg='', meta=None):
|
||||
entry = {'ts': int(__import__('time').time()), 'kind': str(kind),
|
||||
'sev': sev, 'msg': msg[:500]}
|
||||
if meta:
|
||||
entry['meta'] = meta
|
||||
line = json.dumps(entry) + '\n'
|
||||
with _LOCK:
|
||||
_ensure_dir()
|
||||
try:
|
||||
if os.path.exists(EVENTS_PATH) and \
|
||||
os.path.getsize(EVENTS_PATH) > MAX_BYTES:
|
||||
for i in range(KEEP - 1, 0, -1):
|
||||
src = '%s.%d' % (EVENTS_PATH, i)
|
||||
dst = '%s.%d' % (EVENTS_PATH, i + 1)
|
||||
if os.path.exists(src):
|
||||
os.replace(src, dst)
|
||||
if os.path.exists(EVENTS_PATH):
|
||||
os.replace(EVENTS_PATH, EVENTS_PATH + '.1')
|
||||
with open(EVENTS_PATH, 'a') as f:
|
||||
f.write(line)
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
|
||||
def read_events(limit=100):
|
||||
out = []
|
||||
paths = [EVENTS_PATH + '.%d' % i for i in range(KEEP, 0, -1)]
|
||||
paths.append(EVENTS_PATH)
|
||||
for path in paths:
|
||||
try:
|
||||
with open(path) as f:
|
||||
out.extend(json.loads(l) for l in f if l.strip())
|
||||
except (OSError, ValueError):
|
||||
continue
|
||||
out.sort(key=lambda r: r.get('ts', 0))
|
||||
return out[-limit:][::-1]
|
||||
|
||||
|
||||
def counters():
|
||||
counts = {k: 0 for k in COUNTER_KINDS}
|
||||
for row in read_events(limit=5000):
|
||||
k = row.get('kind')
|
||||
if k in counts:
|
||||
counts[k] += 1
|
||||
return counts
|
||||
|
||||
|
||||
def mark_boot(unexpected=False):
|
||||
log_event('unexpected_boot' if unexpected else 'boot', sev='warn'
|
||||
if unexpected else 'info',
|
||||
msg='service started' + ('' if unexpected else ' cleanly'))
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run tests to pass**
|
||||
|
||||
Run: `python3 -m unittest tests.test_mk8_events -v` → PASS
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add payload/user/remote_access/pager-webui/mk8_events.py tests/test_mk8_events.py
|
||||
git commit -m "feat(reliability): JSONL event journal with rotation and counters"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 2: Profile store (`mk8_profiles.py`)
|
||||
|
||||
**Files:**
|
||||
- Create: `payload/user/remote_access/pager-webui/mk8_profiles.py`
|
||||
- Test: `tests/test_mk8_profiles.py`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: nothing.
|
||||
- Produces: `snapshot(name)` → bool; `list_profiles()` → list of names; `restore(name)` → dict `{ok, restored:[...]}`; `auto_name(op)` → `'pre-<op>-<ts>'`; `promote_lastknown_good()`; `delete(name)`; uses `device_run` injected as module attr `run_cmd(args, timeout=20)` defaulting to `server.device_run` lazily (avoids import cycle: define own `_run` that callers/tests monkeypatch).
|
||||
|
||||
- [ ] **Step 1: Failing tests**
|
||||
|
||||
```python
|
||||
import os, sys, tempfile, unittest
|
||||
sys.path.insert(0, os.path.join(os.path.dirname(__file__), '..', 'payload', 'user', 'remote_access', 'pager-webui'))
|
||||
import mk8_profiles
|
||||
|
||||
CONFIGS = ('pineapd', 'wireless', 'network')
|
||||
|
||||
class ProfilesTest(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.dir = tempfile.mkdtemp()
|
||||
mk8_profiles.PROFILES_DIR = os.path.join(self.dir, 'profiles')
|
||||
self.state = {'pineapd': 'config pineapd\n\toption x y\n',
|
||||
'wireless': 'config wireless\n', 'network': ''}
|
||||
def fake_run(args, timeout=20):
|
||||
a = list(args)
|
||||
if a[:2] == ['uci', 'export']:
|
||||
return (0, self.state.get(a[2], ''), '')
|
||||
if a[:2] == ['uci', 'import'] or a[:2] == ['uci', 'commit']:
|
||||
return (0, '', '')
|
||||
return (0, '', '')
|
||||
self.runs = []
|
||||
mk8_profiles.run_cmd = lambda args, timeout=20: (
|
||||
self.runs.append(list(args)) or fake_run(args, timeout))
|
||||
|
||||
def test_snapshot_and_list(self):
|
||||
self.assertTrue(mk8_profiles.snapshot('testprof'))
|
||||
self.assertIn('testprof', mk8_profiles.list_profiles())
|
||||
|
||||
def test_restore_issues_import_per_config(self):
|
||||
mk8_profiles.snapshot('p1')
|
||||
ok = mk8_profiles.restore('p1')
|
||||
self.assertTrue(ok['ok'])
|
||||
imported = [r for r in self.runs if r[:2] == ['uci', 'import']]
|
||||
self.assertEqual(len(imported), len(CONFIGS))
|
||||
commits = [r for r in self.runs if r[:2] == ['uci', 'commit']]
|
||||
self.assertGreaterEqual(len(commits), 1)
|
||||
|
||||
def test_auto_name_format(self):
|
||||
name = mk8_profiles.auto_name('client_connect')
|
||||
self.assertTrue(name.startswith('pre-client_connect-'))
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Verify failure** — `No module named 'mk8_profiles'`
|
||||
|
||||
- [ ] **Step 3: Implement**
|
||||
|
||||
```python
|
||||
"""UCI profile snapshots under /mmc/mk8/profiles/<name>/{pineapd,wireless,network}"""
|
||||
import os, time
|
||||
|
||||
PROFILES_DIR = '/mmc/mk8/profiles'
|
||||
CONFIGS = ('pineapd', 'wireless', 'network')
|
||||
|
||||
|
||||
def run_cmd(args, timeout=20):
|
||||
"""Lazy import avoids a circular import with server.py; tests monkeypatch."""
|
||||
from server import device_run
|
||||
return device_run(args, timeout=timeout)
|
||||
|
||||
|
||||
def _path(name):
|
||||
return os.path.join(PROFILES_DIR, name)
|
||||
|
||||
|
||||
def snapshot(name):
|
||||
dest = _path(name)
|
||||
try:
|
||||
os.makedirs(dest, exist_ok=True)
|
||||
wrote = False
|
||||
for cfg in CONFIGS:
|
||||
rc, out, err = run_cmd(['uci', 'export', cfg])
|
||||
if rc != 0 or not (out or '').strip():
|
||||
continue
|
||||
with open(os.path.join(dest, cfg + '.uci'), 'w') as f:
|
||||
f.write(out)
|
||||
wrote = True
|
||||
return wrote
|
||||
except OSError:
|
||||
return False
|
||||
|
||||
|
||||
def auto_name(op):
|
||||
return 'pre-%s-%d' % (op, int(time.time()))
|
||||
|
||||
|
||||
def list_profiles():
|
||||
try:
|
||||
return sorted(d for d in os.listdir(PROFILES_DIR)
|
||||
if os.path.isdir(_path(d)))
|
||||
except OSError:
|
||||
return []
|
||||
|
||||
|
||||
def delete(name):
|
||||
import shutil
|
||||
shutil.rmtree(_path(name), ignore_errors=True)
|
||||
|
||||
|
||||
def restore(name):
|
||||
"""Restore configs then commit once per config. Caller runs wifi reload
|
||||
/ service restart as appropriate for the operation."""
|
||||
src = _path(name)
|
||||
restored = []
|
||||
if not os.path.isdir(src):
|
||||
return {'ok': False, 'restored': [], 'error': 'profile not found'}
|
||||
for cfg in CONFIGS:
|
||||
fpath = os.path.join(src, cfg + '.uci')
|
||||
if not os.path.isfile(fpath):
|
||||
continue
|
||||
with open(fpath) as f:
|
||||
text = f.read()
|
||||
rc, _, err = run_cmd(['uci', 'import', cfg], input_data=text)
|
||||
if rc != 0:
|
||||
return {'ok': False, 'restored': restored,
|
||||
'error': 'import failed'}
|
||||
run_cmd(['uci', 'commit', cfg])
|
||||
restored.append(cfg)
|
||||
return {'ok': True, 'restored': restored}
|
||||
|
||||
|
||||
LASTKNOWN_GOOD = 'lastknown-good'
|
||||
|
||||
|
||||
def promote_lastknown_good():
|
||||
"""Replace the lastknown-good profile with the live config."""
|
||||
delete(LASTKNOWN_GOOD)
|
||||
return snapshot(LASTKNOWN_GOOD)
|
||||
```
|
||||
|
||||
Update the Step-1 fake to accept `input_data=None` and record imports:
|
||||
|
||||
```python
|
||||
def fake_run(args, timeout=20, input_data=None):
|
||||
a = list(args)
|
||||
if a[:2] == ['uci', 'import']:
|
||||
self.imports = getattr(self, 'imports', [])
|
||||
self.imports.append((a[2], input_data))
|
||||
return (0, '', '')
|
||||
if a[:2] == ['uci', 'export']:
|
||||
return (0, self.state.get(a[2], ''), '')
|
||||
if a[:2] == ['uci', 'commit']:
|
||||
return (0, '', '')
|
||||
return (0, '', '')
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run to pass.**
|
||||
- [ ] **Step 5: Commit** — `feat(reliability): UCI profile snapshot store`
|
||||
|
||||
---
|
||||
|
||||
### Task 3: Reconciler (`mk8_guard.py`)
|
||||
|
||||
**Files:**
|
||||
- Create: `payload/user/remote_access/pager-webui/mk8_guard.py`
|
||||
- Test: `tests/test_mk8_guard.py`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: `server.PINEAPD_SAFE_UCI` (dict of safe pineapd UCI values), `server._apply_uci_wanted(wanted)`, `server._monitor_down(name)`, `server._raise_monitors()`.
|
||||
- Produces: `WANTED_EXTRA = {'pineapd.@pineapd[0].autossidpool': '0'}`; `POOL_CLEAR_MAX = 20`; `reconcile(clear_pool=True)` → `{'changed': [...], 'pool_cleared': bool}`; `guard_report()` → dict for `/api/health`.
|
||||
|
||||
- [ ] **Step 1: Failing tests**
|
||||
|
||||
Full file — reuse the exact `fake_run` device-mock pattern from `tests/test_health.py`:
|
||||
|
||||
```python
|
||||
import os, sys, unittest
|
||||
sys.path.insert(0, os.path.join(os.path.dirname(__file__), '..', 'payload', 'user', 'remote_access', 'pager-webui'))
|
||||
import server
|
||||
import mk8_guard
|
||||
|
||||
class GuardTest(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.calls = []
|
||||
self.uci = {}
|
||||
self.mon_up = {'wlan0mon': True, 'wlan1mon': True}
|
||||
server._iface_up = lambda name: self.mon_up.get(name, True)
|
||||
|
||||
def fake_run(args, timeout=20, input_data=None):
|
||||
a = list(args)
|
||||
self.calls.append(a)
|
||||
if a[:2] == ['uci', 'get']:
|
||||
key = a[2]
|
||||
if key == 'pineapd.@ssidpool[0].ssid':
|
||||
return (0, ''.join('s%d\n' % i for i in range(self.pool)), '')
|
||||
return (0, self.uci.get(key, '') + '\n', '')
|
||||
if a[:2] == ['uci', 'set']:
|
||||
k, _, v = a[2].partition('=')
|
||||
self.uci[k] = v
|
||||
if a[:2] == ['uci', 'delete']:
|
||||
self.pool = 0
|
||||
return (0, '', '')
|
||||
mk8_guard.device_run = fake_run
|
||||
|
||||
def tearDown(self):
|
||||
server._iface_up = lambda name: True
|
||||
|
||||
def test_applies_all_wanted_when_missing(self):
|
||||
self.pool = 0
|
||||
result = mk8_guard.reconcile(clear_pool=False)
|
||||
sets = [c[2] for c in self.calls if c[:2] == ['uci', 'set']]
|
||||
self.assertEqual(len(sets),
|
||||
len(server.PINEAPD_SAFE_UCI) + len(mk8_guard.WANTED_EXTRA))
|
||||
self.assertTrue(result['changed'])
|
||||
|
||||
def test_clears_large_pool_only(self):
|
||||
self.pool = 25
|
||||
result = mk8_guard.reconcile(clear_pool=True)
|
||||
self.assertTrue(result['pool_cleared'])
|
||||
self.assertIn(['uci', 'delete', 'pineapd.@ssidpool[0].ssid'], self.calls)
|
||||
|
||||
def test_small_pool_untouched(self):
|
||||
self.pool = 5
|
||||
result = mk8_guard.reconcile(clear_pool=True)
|
||||
self.assertFalse(result['pool_cleared'])
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Verify failure** — no module.
|
||||
- [ ] **Step 3: Implement**
|
||||
|
||||
```python
|
||||
"""Boot-time reconciliation of crash-prone PineAP settings."""
|
||||
from server import (_apply_uci_wanted, _monitor_down, _raise_monitors,
|
||||
PINEAPD_SAFE_UCI, device_run)
|
||||
|
||||
WANTED_EXTRA = {'pineapd.@pineapd[0].autossidpool': '0'}
|
||||
POOL_CLEAR_MAX = 20
|
||||
MONITORS = ('wlan0mon', 'wlan1mon')
|
||||
|
||||
|
||||
def _pool_size():
|
||||
rc, out, err = device_run(
|
||||
['uci', 'get', 'pineapd.@ssidpool[0].ssid'])
|
||||
if rc != 0 or not (out or '').strip():
|
||||
return 0
|
||||
return len([s for s in out.strip().split('\\n') if s])
|
||||
|
||||
|
||||
def reconcile(clear_pool=True):
|
||||
changed = _apply_uci_wanted(dict(PINEAPD_SAFE_UCI, **WANTED_EXTRA))
|
||||
pool_cleared = False
|
||||
if clear_pool and _pool_size() > POOL_CLEAR_MAX:
|
||||
device_run(['uci', 'delete', 'pineapd.@ssidpool[0].ssid'])
|
||||
pool_cleared = True
|
||||
if changed or pool_cleared:
|
||||
device_run(['uci', 'commit', 'pineapd'])
|
||||
raised = _raise_monitors() if any(_monitor_down(m) for m in MONITORS) else []
|
||||
return {'changed': changed, 'pool_cleared': pool_cleared,
|
||||
'monitors_raised': raised}
|
||||
|
||||
|
||||
def guard_report():
|
||||
from server import _pending_uci
|
||||
pending = _pending_uci(dict(PINEAPD_SAFE_UCI, **WANTED_EXTRA))
|
||||
return {'in_sync': not pending, 'pending': pending,
|
||||
'pool_size': _pool_size()}
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Pass. Commit:** `feat(reliability): boot-time UCI reconciler`
|
||||
|
||||
---
|
||||
|
||||
### Task 4: Guard init script + install wiring
|
||||
|
||||
**Files:**
|
||||
- Create: `payload/user/remote_access/pager-webui/mk8-guard.init`
|
||||
- Modify: `scripts/deploy.sh` (install block), `payload/user/remote_access/pager-webui/server.py` (CLI flag)
|
||||
|
||||
**Interfaces:** CLI: `python3 server.py --reconcile` runs `mk8_guard.reconcile()` and prints JSON; exit 0 always (boot must not fail).
|
||||
|
||||
- [ ] **Step 1: Write `mk8-guard.init`:**
|
||||
|
||||
```sh
|
||||
#!/bin/sh /etc/rc.common
|
||||
# Mark VIII boot guard: enforce safe PineAP UCI before the S50 stack starts.
|
||||
START=49
|
||||
STOP=90
|
||||
|
||||
GUARD_DIR="/root/payloads/user/remote_access/pager-webui"
|
||||
[ -f "$GUARD_DIR/server.py" ] || GUARD_DIR="/mmc/mk8/releases/current"
|
||||
|
||||
start() {
|
||||
[ -f "$GUARD_DIR/server.py" ] || return 0
|
||||
/usr/bin/python3 "$GUARD_DIR/server.py" --reconcile \
|
||||
>/tmp/mk8-guard.log 2>&1 || true
|
||||
}
|
||||
|
||||
stop() { return 0; }
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Add CLI branch in `server.py` `__main__` (after `--release-pager`):**
|
||||
|
||||
```python
|
||||
if '--reconcile' in sys.argv:
|
||||
try:
|
||||
import mk8_guard
|
||||
print(json.dumps(mk8_guard.reconcile()))
|
||||
except Exception as exc: # boot must never fail here
|
||||
print(json.dumps({'error': str(exc)}))
|
||||
sys.exit(0)
|
||||
```
|
||||
|
||||
- [ ] **Step 3: deploy.sh install block (inside REMOTE_COMMAND before EXTRACT_OK echo):**
|
||||
|
||||
```sh
|
||||
cp -f '$DIR/mk8-guard.init' /etc/init.d/mk8-guard
|
||||
chmod 755 /etc/init.d/mk8-guard
|
||||
/etc/init.d/mk8-guard enable
|
||||
```
|
||||
|
||||
(`$DIR` is the existing remote payload dir var used by the unzip step.)
|
||||
|
||||
- [ ] **Step 4: Local verification:** `python3 -m py_compile payload/user/remote_access/pager-webui/server.py && node --check payload/user/remote_access/pager-webui/www/js/app.js` (JS untouched but cheap sanity). `sh -n scripts/deploy.sh`.
|
||||
- [ ] **Step 5: Commit** — `feat(reliability): START=49 boot guard installed by deploy`
|
||||
|
||||
---
|
||||
|
||||
### Task 5: Supervisor extension of health monitor
|
||||
|
||||
**Files:**
|
||||
- Modify: `payload/user/remote_access/pager-webui/server.py` (`health_check`, `_health_loop`, `h_health`, `serve()` boot sequence)
|
||||
- Test: `tests/test_health.py` (extend), `tests/test_reliability_api.py` (new)
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: `mk8_events`, `mk8_guard.guard_report()`.
|
||||
- Produces in `_health`: `mem_percent`, `events` (last 20), `reliability` counters, `guard` report; boot-marker logic `check_boot_marker()` → bool unexpected; mem sampling `_mem_percent()`.
|
||||
|
||||
- [ ] **Step 1: Failing tests** — add to `tests/test_health.py` (same fake_run pattern already there):
|
||||
|
||||
```python
|
||||
class SupervisorExtrasTest(unittest.TestCase):
|
||||
def runTestWith(self): # helper: reuse existing setUp fake_run
|
||||
pass
|
||||
|
||||
def test_mem_percent_math(self):
|
||||
import tempfile
|
||||
content = 'MemTotal: 250000 kB\nMemAvailable: 100000 kB\n'
|
||||
path = tempfile.mktemp()
|
||||
open(path, 'w').write(content)
|
||||
self.assertEqual(server._mem_percent(path), 60)
|
||||
|
||||
def test_health_reports_events_and_counters(self):
|
||||
import mk8_events
|
||||
mk8_events.log_event('restart', msg='x')
|
||||
status, h = server.h_health(None)
|
||||
self.assertEqual(status, 200)
|
||||
self.assertIn('events', h)
|
||||
self.assertIn('boots', h['reliability'])
|
||||
|
||||
def test_boot_marker_detects_unexpected(self):
|
||||
import mk8_events, tempfile, os
|
||||
marker = tempfile.mktemp()
|
||||
old = server.BOOT_MARKER
|
||||
server.BOOT_MARKER = marker
|
||||
try:
|
||||
open(marker, 'w').write('0')
|
||||
self.assertTrue(server.check_boot_marker())
|
||||
os.unlink(marker)
|
||||
self.assertFalse(server.check_boot_marker())
|
||||
finally:
|
||||
server.BOOT_MARKER = old
|
||||
```
|
||||
|
||||
(`_mem_percent` takes a `path` argument so tests inject a temp file; production call passes no arg.)
|
||||
|
||||
- [ ] **Step 2: Implement** — key code:
|
||||
|
||||
```python
|
||||
def _mem_percent(path='/proc/meminfo'):
|
||||
try:
|
||||
vals = {}
|
||||
with open(path) as f:
|
||||
for line in f:
|
||||
k, v = line.split(':')
|
||||
vals[k] = int(v.strip().split()[0])
|
||||
total = vals.get('MemTotal', 0)
|
||||
avail = vals.get('MemAvailable', vals.get('MemFree', 0))
|
||||
return round(100.0 * (total - avail) / total) if total else 0
|
||||
except (OSError, ValueError):
|
||||
return 0
|
||||
|
||||
MEM_WARN_PERCENT = 85
|
||||
MEM_WARN_STREAK = 5
|
||||
|
||||
# inside health_check(), after monitor repair section:
|
||||
h['mem_percent'] = _mem_percent()
|
||||
if h['mem_percent'] >= MEM_WARN_PERCENT:
|
||||
h['mem_streak'] = h.get('mem_streak', 0) + 1
|
||||
else:
|
||||
h['mem_streak'] = 0
|
||||
if h['mem_streak'] == MEM_WARN_STREAK:
|
||||
mk8_events.log_event('mem_warn', sev='warn',
|
||||
msg='memory above %d%% sustained' % MEM_WARN_PERCENT)
|
||||
|
||||
# restart action gains journaling:
|
||||
h['fixes'] += 1
|
||||
mk8_events.log_event('restart', msg='pineapd restarted by health monitor')
|
||||
```
|
||||
|
||||
Boot marker (called from `startup_env_check` tail):
|
||||
|
||||
```python
|
||||
BOOT_MARKER = '/mmc/mk8/boot.marker'
|
||||
|
||||
def check_boot_marker():
|
||||
import os, mk8_events
|
||||
try:
|
||||
unexpected = os.path.exists(BOOT_MARKER)
|
||||
mk8_events.mark_boot(unexpected=unexpected)
|
||||
with open(BOOT_MARKER, 'w') as f:
|
||||
f.write(str(int(time.time())))
|
||||
return unexpected
|
||||
except OSError:
|
||||
return False
|
||||
```
|
||||
|
||||
`h_health` additions:
|
||||
|
||||
```python
|
||||
import mk8_events, mk8_guard
|
||||
h['reliability'] = mk8_events.counters()
|
||||
h['events'] = mk8_events.read_events(limit=20)
|
||||
h['guard'] = mk8_guard.guard_report()
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Run full test_health + new api test to pass.**
|
||||
- [ ] **Step 4: Commit** — `feat(reliability): supervisor sampling, event feed, boot marker`
|
||||
|
||||
---
|
||||
|
||||
### Task 6: Risky-op gate + rollback watchdog
|
||||
|
||||
**Files:**
|
||||
- Create: `payload/user/remote_access/pager-webui/mk8_gate.py`, `payload/user/remote_access/pager-webui/mk8-watchdog.sh`
|
||||
- Modify: `server.py` (wrap handlers), `deploy.sh` (ship watchdog script)
|
||||
- Test: `tests/test_mk8_gate.py`
|
||||
|
||||
**Interfaces:**
|
||||
- `watchdog_decision(fails, oks, fail_after=6, healthy_after=6)` → `'rollback'|'promote'|None` (pure).
|
||||
- `gated(op, fn)` decorator/context: snapshots `auto_name(op)`, spawns watchdog via `setsid sh mk8-watchdog.sh <profile> ... &`, runs fn, returns `(result, profile)`.
|
||||
- CLI: `server.py --rollback-snapshot <name>` restores profile + wifi reload; `--promote-snapshot <name>` promotes lastknown-good.
|
||||
|
||||
- [ ] **Step 1: Failing decision-table tests**
|
||||
|
||||
```python
|
||||
import os, sys, unittest
|
||||
sys.path.insert(0, os.path.join(os.path.dirname(__file__), '..', 'payload', 'user', 'remote_access', 'pager-webui'))
|
||||
import mk8_gate
|
||||
|
||||
class DecisionTest(unittest.TestCase):
|
||||
def tick(self, state):
|
||||
action, new = mk8_gate.watchdog_decision(state)
|
||||
return action, new
|
||||
|
||||
def test_no_action_below_fail_threshold(self):
|
||||
action, s = self.tick({'fails': 5, 'oks': 0, 'tripped': False})
|
||||
self.assertIsNone(action)
|
||||
self.assertFalse(s['tripped'])
|
||||
|
||||
def test_rollback_at_threshold(self):
|
||||
action, s = self.tick({'fails': 6, 'oks': 0, 'tripped': False})
|
||||
self.assertEqual(action, 'rollback')
|
||||
self.assertTrue(s['tripped'])
|
||||
self.assertEqual(s['oks'], 0)
|
||||
|
||||
def test_promote_after_recovery(self):
|
||||
action, s = self.tick({'fails': 6, 'oks': 6, 'tripped': True})
|
||||
self.assertEqual(action, 'promote')
|
||||
|
||||
def test_no_promote_before_recovery_threshold(self):
|
||||
action, s = self.tick({'fails': 6, 'oks': 5, 'tripped': True})
|
||||
self.assertIsNone(action)
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Implement `mk8_gate.py`**
|
||||
|
||||
```python
|
||||
"""Risky-operation gate: snapshot + detached rollback watchdog."""
|
||||
import subprocess
|
||||
|
||||
WATCHDOG = '/root/payloads/user/remote_access/pager-webui/mk8-watchdog.sh'
|
||||
FAIL_AFTER = 6 # consecutive local-liveness failures -> rollback
|
||||
HEALTHY_AFTER = 6 # consecutive successes after failure -> promote
|
||||
INTERVAL = 5 # seconds between probes
|
||||
|
||||
|
||||
def watchdog_decision(state):
|
||||
"""state: {'fails': int, 'oks': int, 'tripped': bool,
|
||||
'fail_after': 6, 'healthy_after': 6}
|
||||
Returns (action, new_state): action in {'rollback','promote',None}."""
|
||||
s = dict(state)
|
||||
fa = s.get('fail_after', FAIL_AFTER)
|
||||
ha = s.get('healthy_after', HEALTHY_AFTER)
|
||||
if not s['tripped'] and s['fails'] >= fa:
|
||||
return 'rollback', dict(s, tripped=True, oks=0)
|
||||
if s['tripped'] and s['oks'] >= ha:
|
||||
return 'promote', s
|
||||
return None, s
|
||||
|
||||
|
||||
def gated(op, profiles, spawn=None):
|
||||
"""Decorator factory: snapshot config, spawn detached watchdog, run op."""
|
||||
import shlex
|
||||
if spawn is None:
|
||||
def spawn(cmd):
|
||||
subprocess.Popen(cmd, shell=True, start_new_session=True)
|
||||
def deco(fn):
|
||||
def wrapped(*a, **kw):
|
||||
name = profiles.auto_name(op)
|
||||
profiles.snapshot(name)
|
||||
spawn("setsid sh %s %s %d %d %d >/dev/null 2>&1 &" %
|
||||
(shlex.quote(WATCHDOG), shlex.quote(name),
|
||||
INTERVAL, FAIL_AFTER, HEALTHY_AFTER))
|
||||
return fn(*a, **kw)
|
||||
return wrapped
|
||||
return deco
|
||||
```
|
||||
|
||||
(Remove the earlier `NotImplementedError` sketch entirely — this is the final form.)
|
||||
|
||||
`gated` implementation:
|
||||
|
||||
```python
|
||||
def gated(op, profiles, spawn=lambda cmd: subprocess.Popen(
|
||||
cmd, shell=True, start_new_session=True)):
|
||||
"""Decorator factory. profiles = mk8_profiles module."""
|
||||
def deco(fn):
|
||||
def wrapped(*a, **kw):
|
||||
name = profiles.auto_name(op)
|
||||
profiles.snapshot(name)
|
||||
spawn("setsid sh %s %s %d %d %d >/dev/null 2>&1 &"
|
||||
% (WATCHDOG, name, INTERVAL, FAIL_AFTER, HEALTHY_AFTER))
|
||||
return fn(*a, **kw)
|
||||
return wrapped
|
||||
return deco
|
||||
```
|
||||
|
||||
- [ ] **Step 3: `mk8-watchdog.sh`**
|
||||
|
||||
```sh
|
||||
#!/bin/sh
|
||||
# Usage: mk8-watchdog.sh <profile> <interval> <fail_after> <healthy_after>
|
||||
PROFILE="$1"; IV="${2:-5}"; FA="${3:-6}"; HA="${4:-6}"
|
||||
DIR="/root/payloads/user/remote_access/pager-webui"
|
||||
[ -f "$DIR/server.py" ] || DIR="/mmc/mk8/releases/current"
|
||||
fails=0; oks=0; tripped=0
|
||||
probe() {
|
||||
curl -fsS -m 3 http://127.0.0.1:8080/ >/dev/null 2>&1 &&
|
||||
ip link show wlan0mgmt >/dev/null 2>&1 &&
|
||||
! grep -q 'disabled' /sys/class/net/wlan0mgmt/flags 2>/dev/null
|
||||
}
|
||||
while true; do
|
||||
if probe; then
|
||||
fails=0
|
||||
if [ "$tripped" = "1" ]; then
|
||||
oks=$((oks + 1))
|
||||
if [ "$oks" -ge "$HA" ]; then
|
||||
/usr/bin/python3 "$DIR/server.py" --promote-snapshot "$PROFILE" >/dev/null 2>&1
|
||||
exit 0
|
||||
fi
|
||||
fi
|
||||
else
|
||||
fails=$((fails + 1)); oks=0
|
||||
if [ "$tripped" = "0" ] && [ "$fails" -ge "$FA" ]; then
|
||||
tripped=1
|
||||
/usr/bin/python3 "$DIR/server.py" --rollback-snapshot "$PROFILE" >/dev/null 2>&1
|
||||
fi
|
||||
fi
|
||||
sleep "$IV"
|
||||
done
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Wire gates** — decorate `h_pineap_wifi_set_ap`, `_disable_sta_uplink` call sites, enterprise deploy/stop, and any handler issuing `wifi reload`, with `@mk8_gate.gated('<op>', mk8_profiles)`.
|
||||
- [ ] **Step 5: CLI rollback/promote hooks in `server.py` `__main__`:**
|
||||
|
||||
```python
|
||||
if '--rollback-snapshot' in sys.argv:
|
||||
name = sys.argv[sys.argv.index('--rollback-snapshot') + 1]
|
||||
import mk8_profiles
|
||||
result = mk8_profiles.restore(name)
|
||||
device_run(['wifi', 'reload'], timeout=90)
|
||||
import mk8_events
|
||||
mk8_events.log_event('rollback', sev='warn',
|
||||
msg='watchdog restored %s' % name,
|
||||
meta=result)
|
||||
print(json.dumps(result))
|
||||
sys.exit(0)
|
||||
if '--promote-snapshot' in sys.argv:
|
||||
name = sys.argv[sys.argv.index('--promote-snapshot') + 1]
|
||||
import mk8_profiles
|
||||
print(json.dumps({'promoted': mk8_profiles.promote_lastknown_good()}))
|
||||
sys.exit(0)
|
||||
```
|
||||
|
||||
- [ ] **Step 6: Tests pass; `sh -n mk8-watchdog.sh`; commit** — `feat(reliability): risky-op preflight snapshots + detached rollback watchdog`
|
||||
|
||||
---
|
||||
|
||||
### Task 7: RF role manager (`mk8_rfplan.py`)
|
||||
|
||||
**Files:**
|
||||
- Create: `payload/user/remote_access/pager-webui/mk8_rfplan.py`
|
||||
- Modify: `server.py` (routes), UI chip later in Task 8
|
||||
- Test: `tests/test_mk8_rfplan.py`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: `server._pause_hop/_resume_hop/_read_hop`, `server.device_run`, `server._uci_values/_set_uci`, gate from Task 6.
|
||||
- Produces: `current_role()` → `'attack'|'uplink'|'idle'`; `set_role(role, ssid=None, psk=None)` → dict result (gated); ensures exclusivity: attack AP enable paths call `ensure_attack()` which auto-switches uplink→attack first.
|
||||
- Routes: `GET /api/rfplan` , `POST /api/rfplan/role`.
|
||||
|
||||
- [ ] **Step 1: Failing tests** (fake device_run capturing uci/iw calls):
|
||||
|
||||
```python
|
||||
def test_uplink_sets_sta_section_and_pauses_hop(self): ...
|
||||
def test_set_role_uplink_requires_ssid(self): ...
|
||||
def test_exclusivity_switch(self): ...
|
||||
def test_current_role_reads_uci(self): ...
|
||||
```
|
||||
|
||||
Assertions: `uci set wireless.wlan1up.mode=sta`, `.disabled=0`, `.ssid=<ssid>`, hop paused via `_pause_hop` mock, `wifi reload` invoked through gated op only.
|
||||
|
||||
- [ ] **Step 2: Implement core:**
|
||||
|
||||
```python
|
||||
ROLE_KEY = 'mk8.rfplan.role'
|
||||
|
||||
def current_role():
|
||||
from server import _uci_values
|
||||
cfg = _uci_values('wireless.wlan1up') or {}
|
||||
if cfg.get('disabled') != '1' and cfg.get('mode') == 'sta':
|
||||
return 'uplink'
|
||||
return 'idle'
|
||||
|
||||
def set_role(role, ssid=None, psk=None):
|
||||
from server import device_run, _pause_hop, _resume_hop
|
||||
if role == 'uplink':
|
||||
if not ssid:
|
||||
return {'ok': False, 'error': 'ssid required'}
|
||||
cmds = [
|
||||
['uci', 'set', 'wireless.wlan1up=wifi-iface'],
|
||||
['uci', 'set', 'wireless.wlan1up.device=radio1'],
|
||||
['uci', 'set', 'wireless.wlan1up.mode=sta'],
|
||||
['uci', 'set', 'wireless.wlan1up.network=cli'],
|
||||
['uci', 'set', 'wireless.wlan1up.ssid=%s' % ssid],
|
||||
['uci', 'set', 'wireless.wlan1up.encryption=%s'
|
||||
% ('psk2' if psk else 'none')],
|
||||
['uci', 'set', 'wireless.wlan1up.disabled=0'],
|
||||
]
|
||||
if psk:
|
||||
cmds.append(['uci', 'set', 'wireless.wlan1up.key=%s' % psk])
|
||||
for c in cmds:
|
||||
device_run(c)
|
||||
device_run(['uci', 'commit', 'wireless'])
|
||||
_pause_hop()
|
||||
device_run(['wifi', 'reload'], timeout=60)
|
||||
assoc = associated()
|
||||
if not assoc:
|
||||
disable_uplink()
|
||||
_resume_hop()
|
||||
return {'ok': False, 'error': 'association failed; reverted'}
|
||||
return {'ok': True, 'role': 'uplink', 'assoc': assoc}
|
||||
# attack/idle: tear down STA
|
||||
disable_uplink()
|
||||
_resume_hop()
|
||||
return {'ok': True, 'role': role}
|
||||
|
||||
def associated():
|
||||
rc, out, err = device_run(['iw', 'dev', 'wlan1up', 'link'], timeout=10)
|
||||
if rc != 0 or 'Connected' not in (out or ''):
|
||||
return None
|
||||
for line in (out or '').splitlines():
|
||||
line = line.strip()
|
||||
if line.startswith('Connected to '):
|
||||
return line.split()[2]
|
||||
return None
|
||||
|
||||
def disable_uplink():
|
||||
from server import device_run
|
||||
device_run(['uci', 'set', 'wireless.wlan1up.disabled=1'])
|
||||
device_run(['uci', 'commit', 'wireless'])
|
||||
|
||||
def ensure_attack():
|
||||
if current_role() == 'uplink':
|
||||
set_role('attack')
|
||||
```
|
||||
|
||||
Handlers in server.py wrap with gate + journal events. `GET /api/rfplan` returns role + assoc + hop-paused state.
|
||||
|
||||
- [ ] **Step 3: Pass; py_compile; commit** — `feat(reliability): phy1 RF role manager with uplink-on-radio1`
|
||||
|
||||
---
|
||||
|
||||
### Task 8: UI additions
|
||||
|
||||
**Files:**
|
||||
- Modify: `payload/user/remote_access/pager-webui/www/js/views.js`, `www/js/app.js` (nav if needed), `www/css/app.css`
|
||||
|
||||
**Steps (no unit tests; verified by `node --check` + live smoke in Task 10):**
|
||||
|
||||
- [ ] Dashboard health panel: events feed list (ts/kind/sev/msg) + counters row (boots/unexpected/rollbacks/restarts/guard fixes) + guard sync chip; render from `/api/health` new fields; CSS classes `mk8-events-feed`, `mk8-counter-row`.
|
||||
- [ ] Top bar RF chip extension: show `PHY1: UPLINK chNN` when rfplan role is uplink (poll `/api/rfplan` with existing status poll).
|
||||
- [ ] Settings: "Reliability" card — profile save input + Save button (`POST /api/reliability/profile` {name}), profile list with Restore buttons (`POST /api/reliability/restore` {name}), RF role control (role select + SSID/PSK inputs → `POST /api/rfplan/role`).
|
||||
- [ ] Add routes in Task 9's API surface before wiring buttons; keep fetch helpers identical to existing patterns (`apiFetch('/api/...')`).
|
||||
- [ ] `node --check` both JS files; bump cache-bust query `?v=` strings as existing convention does.
|
||||
- [ ] Commit — `feat(ui): reliability panel, profiles card, RF plan controls`
|
||||
|
||||
---
|
||||
|
||||
### Task 9: API routes + atomic deploys + VERSION
|
||||
|
||||
**Files:**
|
||||
- Modify: `server.py` ROUTER block (~line 6804), `scripts/deploy.sh`, create `VERSION`
|
||||
- Test: `tests/test_reliability_api.py`
|
||||
|
||||
- [ ] **Routes:**
|
||||
|
||||
```python
|
||||
ROUTER.add('GET', r'/api/rfplan', h_rfplan_get)
|
||||
ROUTER.add('POST', r'/api/rfplan/role', h_rfplan_post)
|
||||
ROUTER.add('GET', r'/api/reliability/profiles', h_profiles_get)
|
||||
ROUTER.add('POST', r'/api/reliability/profile', h_profile_save)
|
||||
ROUTER.add('POST', r'/api/reliability/restore', h_profile_restore)
|
||||
```
|
||||
|
||||
Handlers thin-wrape `mk8_rfplan` / `mk8_profiles`; restore handler runs inside `mk8_gate.gated('restore_profile', ...)`. All auth-gated automatically by existing middleware.
|
||||
|
||||
- [ ] **VERSION file:** `1.4.0`
|
||||
- [ ] **deploy.sh rework:**
|
||||
1. Read `VERSION` → stamp build copies of `_hak5_manifest.json` (`version`), `payload.sh` header comment, and inject `SERVER_VERSION = 'X'` into staged `server.py` (build dir only, never source tree).
|
||||
2. Remote flow: scp zip to `/tmp/mk8-stage/` → verify sha256 of uploaded zip matches local → stop service → extract to `/mmc/mk8/releases/<ts>/` → repoint `current` symlink atomically (`ln -sfn`) → install/update `/etc/init.d/mk8-guard` + copy `mk8-watchdog.sh` → start → poll `http://127.0.0.1:8080/api/api_ping` (via SSH-local curl) ≤60 s → compare served banner/version → on failure: `ln -sfn` back to previous release + start + exit 1.
|
||||
3. Keep legacy overlay payload dir as symlink target for portal compatibility: `/root/payloads/user/.../pager-webui` → real dir stays, contains pointer script or bind; simplest: leave legacy install untouched and have `current` be canonical (init scripts already fall back to `/mmc/mk8/releases/current`).
|
||||
- [ ] **Tests:** unit-test stamping function `stamp_version(build_dir)` extracted into `scripts/build_common.py` (new) so it is importable: asserts manifest/payload/server contain version; sha256 check tested with tmpfiles.
|
||||
- [ ] **Commit** — `feat(deploy): atomic releases, VERSION single-source, post-deploy verification`
|
||||
|
||||
---
|
||||
|
||||
### Task 10: On-device smoke suite + live verification
|
||||
|
||||
**Files:**
|
||||
- Create: `scripts/smoke.sh`
|
||||
|
||||
- [ ] **smoke.sh checks (each prints PASS/FAIL, non-destructive unless `--write` given):**
|
||||
1. Service up: `curl :8080/api/api_ping`.
|
||||
2. Guard installed: `[ -x /etc/init.d/mk8-guard ]` and enabled symlink exists.
|
||||
3. Invariants: `uci get` each wanted key equals expected; pool size ≤ 20.
|
||||
4. Journal writable + has boot event: tail events.log on device.
|
||||
5. Monitors up: `ip link show wlan0mon/wlan1mon`.
|
||||
6. (`--write`) Bad-value drill: set `pineapd.wlan1mon.bands='2,5'` → run `--reconcile` → expect `'5'`; set pool of 25 SSIDs → reconcile → cleared.
|
||||
7. (`--write`) Role drill: set_role uplink to lab AP → expect assoc; set_role attack → expect monitors hopping again.
|
||||
8. Deploy version match: `/api/health` version == `cat VERSION`.
|
||||
- [ ] **Execution order:** full local unit suite (every `tests/test_*.py` individually) → deploy via `./scripts/deploy.sh --password '<pw>'` → reboot device via SSH → wait for SSH return → rerun smoke.sh → confirm guards survived boot → report.
|
||||
- [ ] **Commit** — `test(smoke): on-device reliability suite` ; final tag `v1.4.0` after user confirmation.
|
||||
|
||||
---
|
||||
|
||||
## Verification matrix (spec → tasks)
|
||||
|
||||
| Spec requirement | Task |
|
||||
|---|---|
|
||||
| Boot guard before S50 | 4 |
|
||||
| Reconciler invariants (5 crash sources) | 3 |
|
||||
| Profiles + lastknown-good | 2 |
|
||||
| Knock-off rollback watchdog (local liveness) | 6 |
|
||||
| RF roles, uplink→phy1, hop pause/resume | 7 |
|
||||
| Supervisor sampling + hysteresis + alerts | 5 |
|
||||
| Event journal on /mmc + boot detection | 1, 5 |
|
||||
| Atomic deploy + version single-source | 9 |
|
||||
| UI health/events/profiles/RF | 8, 9 |
|
||||
| Unit + smoke tests | all, 10 |
|
||||
Reference in New Issue
Block a user