docs: plan recon identity enrichment
This commit is contained in:
@@ -0,0 +1,281 @@
|
||||
# Recon Client Associations and Device Identity Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Add confirmed client-to-SSID/AP associations and best-available manufacturer identity to Recon APIs, tables, focus details, exports, and reports.
|
||||
|
||||
**Architecture:** Keep `recon_scan_data()` as the canonical enrichment boundary. Add cached local OUI resolution and scan-scoped evidence joins in `server.py`, then consume the enriched object shape in existing JSON/CSV/HTML serializers and `views.recon`. Handshake evidence may resolve BSSID; `hostap_client` evidence remains SSID-only because its schema has no BSSID.
|
||||
|
||||
**Tech Stack:** Python 3 standard library, SQLite read-only queries, vanilla JavaScript, existing test suite, no frontend build step.
|
||||
|
||||
## Global Constraints
|
||||
|
||||
- Associations must never be inferred from proximity, channel, timing, or probe requests.
|
||||
- `ssid.type = 5` directed probes are not associations.
|
||||
- `hostap_client` rows create confirmed SSID-only associations and never claim an AP BSSID.
|
||||
- OUI resolution uses Nmap, macchanger, then the built-in map, then `Unknown`.
|
||||
- Missing OUI files or optional association data must not fail Recon.
|
||||
- No external network lookup or wireless/PineAP configuration change is allowed.
|
||||
- Preserve unrelated existing changes in `server.py`, `tests/test_recon.py`, `loot/`, and certificate files.
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Add Cached OUI Identity Resolution
|
||||
|
||||
**Files:**
|
||||
- Modify: `payload/user/remote_access/pager-webui/server.py` near `OUI_VENDORS`, `_oui_prefix`, and `oui_vendor`
|
||||
- Test: `tests/test_recon.py` in `DecodersTest`
|
||||
|
||||
**Interfaces:**
|
||||
- Produces `oui_identity(mac)` returning a JSON-safe dictionary with `manufacturer`, `model`, `oui`, and `source`.
|
||||
- Keeps `oui_vendor(mac)` behavior compatible for existing callers.
|
||||
|
||||
- [ ] **Step 1: Write failing tests**
|
||||
|
||||
Add tests that patch `server.OUI_DATA_PATHS` and `server.open`, reset the identity cache, and assert Nmap wins over macchanger and built-in fallback. Add tests for missing files, unknown global MACs, locally administered MACs, and `model is None`.
|
||||
|
||||
```python
|
||||
def test_oui_identity_prefers_nmap_then_macchanger(self):
|
||||
server._oui_identity_cache = None
|
||||
files = {
|
||||
'/nmap': 'C89E43 Apple Corporation\n',
|
||||
'/mac': 'C89E43 fallback\n',
|
||||
}
|
||||
with mock.patch.object(server, 'OUI_DATA_PATHS', ['/nmap', '/mac']), \
|
||||
mock.patch('builtins.open', side_effect=lambda p, *a, **k:
|
||||
mock.mock_open(read_data=files[p]).return_value):
|
||||
value = server.oui_identity('C89E43648080')
|
||||
self.assertEqual(value['manufacturer'], 'Apple Corporation')
|
||||
self.assertEqual(value['source'], 'nmap')
|
||||
self.assertIsNone(value['model'])
|
||||
|
||||
def test_oui_identity_handles_local_and_unknown(self):
|
||||
server._oui_identity_cache = {}
|
||||
self.assertEqual(server.oui_identity('02:11:22:33:44:55')['manufacturer'],
|
||||
'Local/Randomized')
|
||||
self.assertEqual(server.oui_identity('AA:BB:CC:00:00:01')['manufacturer'],
|
||||
'Unknown')
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run the focused tests and verify failure**
|
||||
|
||||
Run: `python3 -m unittest tests.test_recon.DecodersTest -v`
|
||||
|
||||
Expected: FAIL because `OUI_DATA_PATHS`, `_oui_identity_cache`, and
|
||||
`oui_identity()` do not yet exist.
|
||||
|
||||
- [ ] **Step 3: Implement the smallest resolver**
|
||||
|
||||
Add the two device paths, a process-level cache, parsers for the first six
|
||||
hexadecimal characters in each local database line, source labels `nmap` and
|
||||
`macchanger`, and fallback to `OUI_VENDORS`. Return `model: None` for every
|
||||
current source. Detect the locally administered bit before file lookup.
|
||||
|
||||
- [ ] **Step 4: Run focused and regression tests**
|
||||
|
||||
Run: `python3 -m unittest tests.test_recon.DecodersTest -v`
|
||||
|
||||
Expected: PASS, including the pre-existing `oui_vendor` assertions.
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add payload/user/remote_access/pager-webui/server.py tests/test_recon.py
|
||||
git commit -m "feat: resolve recon device manufacturers locally"
|
||||
```
|
||||
|
||||
### Task 2: Enrich Scan Associations and APs
|
||||
|
||||
**Files:**
|
||||
- Modify: `payload/user/remote_access/pager-webui/server.py` in `recon_scan_data()` and its nearby SQL helpers
|
||||
- Test: `tests/test_recon.py` in `ReconDataTest`
|
||||
|
||||
**Interfaces:**
|
||||
- `recon_scan_data(scan_id, _timeout=20, _limit=None, db=None)` returns existing fields plus AP `device_identity`, `clients`, `client_count`, and client `vendor`, `associations`.
|
||||
|
||||
- [ ] **Step 1: Extend the fixture and write failing tests**
|
||||
|
||||
Add `hostap_client` rows, a second AP/client handshake pair, and a type-5 probe row to `make_db()`. Assert handshake association has SSID/BSSID/source, host-AP association has SSID and no BSSID, type-5 does not associate, duplicate evidence merges sources, AP counts are unique, and client/AP vendors are present.
|
||||
|
||||
```python
|
||||
def test_scan_detail_associations_are_confirmed_only(self):
|
||||
data = server.recon_scan_data(1)
|
||||
client = next(c for c in data['clients'] if c['mac'] == 'AE:77:C0:EB:31:41')
|
||||
self.assertEqual(client['associations'][0]['sources'], ['handshake'])
|
||||
self.assertEqual(client['associations'][0]['ssid'], 'Anderson-5')
|
||||
self.assertEqual(client['associations'][0]['bssid'], 'C8:9E:43:64:80:80')
|
||||
self.assertEqual(data['aps'][0]['client_count'], 1)
|
||||
self.assertNotIn('ProbeOnlySSID', [a['ssid'] for a in client['associations']])
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run the focused test and verify failure**
|
||||
|
||||
Run: `python3 -m unittest tests.test_recon.ReconDataTest -v`
|
||||
|
||||
Expected: FAIL because enriched association fields do not exist.
|
||||
|
||||
- [ ] **Step 3: Add bounded evidence queries**
|
||||
|
||||
Load handshake pairs from the selected scan and resolve AP/client MACs from the
|
||||
already loaded `wifi_device` rows. If `hostap_client` exists, query only rows
|
||||
for the selected scan inside a guarded `try` block; if the table is absent,
|
||||
use an empty list. Do not join type-5 rows into the association map.
|
||||
|
||||
- [ ] **Step 4: Build deterministic deduplicated associations**
|
||||
|
||||
Normalize MACs and use `(client_mac, bssid, ssid)` as the association key,
|
||||
where a missing BSSID is represented separately from any AP row. Merge source
|
||||
names in stable order `handshake`, then `hostap_client`, and retain host-AP
|
||||
timestamps. Attach BSSID-backed associations to matching APs only; attach
|
||||
SSID-only host-AP evidence to clients only.
|
||||
|
||||
- [ ] **Step 5: Add identity and AP/client projection**
|
||||
|
||||
Call `oui_identity()` for every AP BSSID and client MAC. Add `clients` and
|
||||
`client_count` to AP objects, preserving current AP ordering and existing
|
||||
fields. Keep clients with no association and set `associations: []`.
|
||||
|
||||
- [ ] **Step 6: Run the complete Recon tests**
|
||||
|
||||
Run: `python3 -m unittest tests.test_recon -v`
|
||||
|
||||
Expected: PASS for all existing and new tests.
|
||||
|
||||
- [ ] **Step 7: Commit**
|
||||
|
||||
```bash
|
||||
git add payload/user/remote_access/pager-webui/server.py tests/test_recon.py
|
||||
git commit -m "feat: associate recon clients with confirmed networks"
|
||||
```
|
||||
|
||||
### Task 3: Propagate Enriched Data Through Exports
|
||||
|
||||
**Files:**
|
||||
- Modify: `payload/user/remote_access/pager-webui/server.py` in Recon CSV/HTML builders and download handlers
|
||||
- Test: `tests/test_recon.py` in `ReconReportTest`
|
||||
|
||||
**Interfaces:**
|
||||
- Existing JSON downloads preserve the enriched detail object.
|
||||
- Existing CSV and HTML downloads include identity, client counts, and confirmed association details.
|
||||
|
||||
- [ ] **Step 1: Write failing export assertions**
|
||||
|
||||
Assert JSON contains `device_identity` and `associations`, CSV headers/rows
|
||||
contain `Device Identity`, `Client Count`, and semicolon-separated confirmed
|
||||
SSIDs, and HTML contains a `Confirmed Clients` section while excluding the
|
||||
type-5 probe SSID.
|
||||
|
||||
- [ ] **Step 2: Run report tests and verify failure**
|
||||
|
||||
Run: `python3 -m unittest tests.test_recon.ReconReportTest -v`
|
||||
|
||||
Expected: FAIL because serializers currently omit the enrichment.
|
||||
|
||||
- [ ] **Step 3: Implement deterministic flattening and report sections**
|
||||
|
||||
Keep JSON unchanged apart from its enriched source object. Add CSV columns
|
||||
using a stable display identity and `'; '.join()` for multiple association
|
||||
SSIDs. Add AP identity/client count to the existing AP table and a confirmed
|
||||
client table to HTML, escaping all values through the existing HTML helpers.
|
||||
|
||||
- [ ] **Step 4: Run report and full Python tests**
|
||||
|
||||
Run: `python3 -m unittest tests.test_recon tests.test_health tests.test_ws -v`
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add payload/user/remote_access/pager-webui/server.py tests/test_recon.py
|
||||
git commit -m "feat: include recon identity in exports"
|
||||
```
|
||||
|
||||
### Task 4: Update Recon Tables and Focus Details
|
||||
|
||||
**Files:**
|
||||
- Modify: `payload/user/remote_access/pager-webui/www/js/views.js` around `reconDefaultCols`, `colDefs`, table column definitions, and `renderFocus`
|
||||
- Modify: `payload/user/remote_access/pager-webui/www/css/app.css` for compact identity/association detail styling if needed
|
||||
- Test: `tests/test_recon.py` source assertions or existing frontend smoke harness
|
||||
|
||||
**Interfaces:**
|
||||
- Existing `views.recon` consumes enriched AP/client objects without new endpoints.
|
||||
- `pw_recon_cols` migration preserves existing values and adds new defaults.
|
||||
|
||||
- [ ] **Step 1: Add source-level failing assertions**
|
||||
|
||||
Assert the source contains default AP `identity` and `clients` columns, client
|
||||
`vendor` and `associated_ssid` columns, association search values, and a
|
||||
`Confirmed Clients` focus section.
|
||||
|
||||
- [ ] **Step 2: Implement client/AP display helpers**
|
||||
|
||||
Add a helper that formats `device_identity` as manufacturer plus model when
|
||||
model is non-null, otherwise manufacturer plus OUI for unknown values. Add a
|
||||
helper that formats one association SSID or `first SSID +N` and sets the full
|
||||
association summary in the cell `title`.
|
||||
|
||||
- [ ] **Step 3: Update column defaults and settings**
|
||||
|
||||
Replace AP `vendor` with `identity`, add AP `clients`, and add client `vendor`
|
||||
and `associated_ssid` defaults. When loading old settings, merge missing keys
|
||||
from `reconDefaultCols()` rather than discarding the saved preferences.
|
||||
|
||||
- [ ] **Step 4: Update filtering, sorting, and table rendering**
|
||||
|
||||
Include formatted identity and association strings in searchable values,
|
||||
retain numeric sorting for client counts, and render the new columns through
|
||||
the existing table/paginator code.
|
||||
|
||||
- [ ] **Step 5: Add confirmed clients to AP focus**
|
||||
|
||||
Render MAC, vendor, and comma-separated evidence sources from `ap.clients`.
|
||||
Show `No confirmed clients` for an empty list and do not list probe-only
|
||||
records.
|
||||
|
||||
- [ ] **Step 6: Run frontend/source and Python tests**
|
||||
|
||||
Run: `python3 -m unittest discover -s tests -p 'test_*.py' -v`
|
||||
|
||||
Expected: PASS. Then run the project’s existing browser smoke harness if
|
||||
available and verify desktop/mobile Recon rendering without changing device
|
||||
configuration.
|
||||
|
||||
- [ ] **Step 7: Commit**
|
||||
|
||||
```bash
|
||||
git add payload/user/remote_access/pager-webui/www/js/views.js payload/user/remote_access/pager-webui/www/css/app.css tests/test_recon.py
|
||||
git commit -m "feat: show recon client identities and associations"
|
||||
```
|
||||
|
||||
### Task 5: On-Device Read-Only Verification
|
||||
|
||||
**Files:**
|
||||
- No source changes expected.
|
||||
- Evidence: local command output only; do not add credentials or device dumps to git.
|
||||
|
||||
- [ ] **Step 1: Deploy through the project’s normal development/deploy path**
|
||||
|
||||
Use the existing script documented in `README.md`; do not alter wireless or
|
||||
PineAP settings.
|
||||
|
||||
- [ ] **Step 2: Compare API data with read-only SQLite evidence**
|
||||
|
||||
Query `scan`, `wifi_device`, `ssid`, `handshake`, and `hostap_client` using the
|
||||
read-only SQLite URI. Confirm handshake BSSID/client pairs match API
|
||||
associations, host-AP entries are SSID-only, and type-5 rows are absent from
|
||||
associations.
|
||||
|
||||
- [ ] **Step 3: Verify identity and UI behavior**
|
||||
|
||||
Confirm AP identity resolves from an installed local database or fallback,
|
||||
unknown/local MAC labels are honest, Access Points shows client counts and
|
||||
identity, Clients shows vendor/SSID, and the focus sidebar shows confirmed
|
||||
clients at desktop and mobile widths.
|
||||
|
||||
- [ ] **Step 4: Run final verification before claiming completion**
|
||||
|
||||
Run: `git diff --check`, `python3 -m unittest discover -s tests -p 'test_*.py' -v`, and `git status --short`.
|
||||
|
||||
Expected: no whitespace errors, all tests pass, and only intended source
|
||||
changes plus pre-existing worktree changes are present.
|
||||
Reference in New Issue
Block a user