docs: remove superpowers planning documents
This commit is contained in:
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@@ -1,61 +0,0 @@
|
||||
# In-Flight Monitor — Count + Activity Spark
|
||||
|
||||
**Date:** 2026-08-14
|
||||
**Status:** Approved (design)
|
||||
|
||||
## Summary
|
||||
|
||||
Enhance the In-Flight Monitor action so its key shows the **number of
|
||||
in-flight requests** for the configured model plus a **count-trend spark**:
|
||||
a tiny line/area chart of that count over the last ~60s, so activity is
|
||||
visible even during a single long-running request.
|
||||
|
||||
## Layout (72×72 SVG, rendered via `renderInflight`)
|
||||
|
||||
- **Top:** model short name (small, white) — existing.
|
||||
- **Center:** the status. When `ready`:
|
||||
- count > 0 → the **count number**, large/bold (e.g. `3`), on the red
|
||||
`ACTIVE` background, with the existing pulse.
|
||||
- count === 0 → `IDLE` on green — existing.
|
||||
- `loading` → `LOADING`, `stopped` → `OFF`, offline → `OFFLINE`,
|
||||
unconfigured → `NO MODEL`, unknown → `…` — all unchanged.
|
||||
- **Bottom strip (~14px):** the count-trend spark — a small polyline + faint
|
||||
area fill of the model's in-flight count over the last 60 samples, in the
|
||||
state color. Flat at the baseline when idle; spikes when requests come and go.
|
||||
|
||||
## Data Flow
|
||||
|
||||
- The feed side is unchanged — `InflightTracker` already exposes exact
|
||||
per-model counts.
|
||||
- Each key's per-key state (in `inflight-monitor.ts`, keyed by
|
||||
`ev.action.id`) gains:
|
||||
- `history: number[]` — a 60-sample ring buffer of the live count.
|
||||
- `sampler?: ReturnType<typeof setInterval>` — a 1s interval that pushes
|
||||
`runtime.tracker.count(modelId)` into `history` and re-renders.
|
||||
Started in `onWillAppear`, cleared in `onWillDisappear`.
|
||||
- Live feed events (`add`/`remove`/`snapshot`) still re-render the big
|
||||
number immediately via the existing subscription; the spark refreshes at
|
||||
1Hz.
|
||||
- `renderInflight` gains an optional `history?: number[]` field. Spark
|
||||
scale: yMax = `max(max(history), 1)` so a zero line sits at the baseline.
|
||||
|
||||
## Files
|
||||
|
||||
- `src/lib/render.ts` — `renderInflight` accepts `history` and draws the
|
||||
spark; center label shows the count when active.
|
||||
- `src/actions/inflight-monitor.ts` — per-key `history` ring buffer +
|
||||
1s sampler.
|
||||
- `tests/render.test.ts` — new/updated assertions for count rendering and
|
||||
the spark.
|
||||
|
||||
## Error Handling / Edge Cases
|
||||
|
||||
- Spark with <2 samples renders no line (like the GPU chart).
|
||||
- Sampler cleared on `onWillDisappear` (no leaks); multi-key safety
|
||||
preserved (buffer + sampler live in the per-key map entry).
|
||||
- History capped at 60 samples.
|
||||
|
||||
## Out of Scope
|
||||
|
||||
- No change to the GPU Graph action, the feed, or the tracker.
|
||||
- No throughput-bar variant (count trend was chosen).
|
||||
@@ -1,166 +0,0 @@
|
||||
# llama-watch — Stream Deck Plugin Design
|
||||
|
||||
**Date:** 2026-08-14
|
||||
**Status:** Approved (pending spec review)
|
||||
|
||||
## Summary
|
||||
|
||||
A Stream Deck plugin ("llama-watch") that monitors a llama-swap instance
|
||||
(`http://localhost:9292`). Two key-action types:
|
||||
|
||||
1. **In-flight Monitor** — shows whether a specific model currently has a
|
||||
request in flight, with a color-coded state (OFF / LOADING / IDLE / ACTIVE).
|
||||
2. **GPU Graph** — renders a live line chart of a selected GPU metric
|
||||
(utilization %, VRAM %, temperature, power draw, or fan speed) on the key,
|
||||
for a specific GPU or the "All GPUs" aggregate.
|
||||
|
||||
Target device: standard Stream Deck (72x72 px keys), rendered at 144x144 for
|
||||
crispness. macOS only. Local install, with eventual Elgato Marketplace
|
||||
submission as a design goal.
|
||||
|
||||
## Approach
|
||||
|
||||
Official Elgato JS SDK (`streamdeck-jssdk`) + `streamdeck-cli` tooling, a
|
||||
TypeScript Node.js plugin, and `@napi-rs/canvas` for image rendering. The
|
||||
plugin runs locally (launched by the Stream Deck app) and talks directly to
|
||||
the llama-swap instance.
|
||||
|
||||
## Data Sources
|
||||
|
||||
llama-swap exposes everything needed (verified live against
|
||||
`localhost:9292`):
|
||||
|
||||
- **`GET /api/events`** — SSE stream used by the web UI. Emits `inflight`
|
||||
events (operations `snapshot` / `add` / `remove`, per-request detail:
|
||||
model, timestamp, elapsed, bytes) and `modelStatus` events (per-model
|
||||
state: `stopped`, `loading`, `ready`). Also emits `logData`, `uiConfig`,
|
||||
`profileChanged`, `activity` events — ignored.
|
||||
- **`GET /metrics`** — Prometheus text format. Relevant series:
|
||||
- `llamaswap_gpu_util_percent{id,name,uuid}`
|
||||
- `llamaswap_gpu_memory_util_percent{id,name,uuid}`
|
||||
- `llamaswap_gpu_temperature_celsius{id,name,uuid}`
|
||||
- `llamaswap_gpu_power_draw_watts{id,name,uuid}`
|
||||
- `llamaswap_gpu_fan_speed_percent{id,name,uuid}`
|
||||
- **`GET /v1/models`** — model list (with `status.value` loaded/unloaded),
|
||||
used to populate the model dropdown in the property inspector.
|
||||
|
||||
Upstream vLLM metrics (ports 10001/10003) are bound to localhost on the
|
||||
server and are NOT reachable from the client Mac. Not used.
|
||||
|
||||
## Architecture
|
||||
|
||||
The plugin is a single Node process with three connections:
|
||||
|
||||
1. WebSocket to the Stream Deck software (via `streamdeck-jssdk`).
|
||||
2. Persistent SSE connection to `/api/events` (the `EventFeed`).
|
||||
3. 5s-interval polling of `/metrics` (the `MetricsPoller`).
|
||||
|
||||
### Modules
|
||||
|
||||
- **`MetricsPoller`** (pure logic, testable)
|
||||
- Polls `GET /metrics` every **5 seconds** (fixed).
|
||||
- Parses the Prometheus text format into a lookup keyed by metric name ×
|
||||
GPU id.
|
||||
- `getSnapshot(gpuSelector, metric)` resolves any selector × metric
|
||||
combination into a number, including the "All GPUs" aggregate.
|
||||
- Keeps a ring buffer of 60 samples (~5 minutes) per (gpu, metric).
|
||||
- A missed/failed poll skips that sample (no gap in rendering beyond a
|
||||
break in the line).
|
||||
- **`EventFeed`** (pure logic, testable)
|
||||
- Persistent SSE connection to `/api/events`.
|
||||
- Reconnect with exponential backoff (1s → 30s max); the `inflight`
|
||||
`snapshot` operation emitted on connect self-heals the tracker.
|
||||
- Maintains per-model in-flight request counts and per-model state from
|
||||
`modelStatus`.
|
||||
- **Actions**
|
||||
- `InflightMonitor` — one instance per monitored model.
|
||||
- `GpuGraph` — one instance per GPU (or All GPUs) × metric.
|
||||
- **Property inspectors** (`pi/*.html`) — settings UI for both actions.
|
||||
- **Renderer** — draws 144x144 PNGs via `@napi-rs/canvas`.
|
||||
|
||||
### Data Flow
|
||||
|
||||
```
|
||||
llama-swap ──SSE /api/events──▶ EventFeed ──▶ per-model inflight counts + state
|
||||
──HTTP /metrics 5s──▶ MetricsPoller ──▶ ring buffers + aggregates
|
||||
│
|
||||
Stream Deck app ◀──jssdk WS── Actions ────────┘
|
||||
◀──144x144 PNG + title── Renderer
|
||||
```
|
||||
|
||||
### Settings (shared + per action)
|
||||
|
||||
- **Shared:** base URL (default `http://localhost:9292`), optional API
|
||||
key (sent as a header when set; instance currently requires none).
|
||||
- **InflightMonitor:** model (dropdown populated live from `/v1/models`).
|
||||
- **GpuGraph:** GPU selector (dropdown from live GPU list, plus "All GPUs"),
|
||||
metric (dropdown: Utilization %, VRAM %, Temperature, Power draw, Fan
|
||||
speed).
|
||||
|
||||
## Action 1 — In-flight Monitor
|
||||
|
||||
- **States (rendered on key):**
|
||||
- `stopped` / unloaded → grey background, model name, `OFF`
|
||||
- `loading` → amber, `LOADING`
|
||||
- `ready` + 0 requests → green, `IDLE`
|
||||
- `ready` + ≥1 request → red, `ACTIVE` (subtle pulse, re-render ~2 Hz while
|
||||
active)
|
||||
- Model short name displayed on the key; if no model selected, placeholder
|
||||
with an edit hint.
|
||||
- **Press:** opens `http://<base-url>/ui` in the default browser (spawns
|
||||
`open` on macOS).
|
||||
|
||||
## Action 2 — GPU Graph
|
||||
|
||||
- **Rendering (graph dominates the 72x72 key):**
|
||||
- Big line chart of the last 60 samples (5 min @ 5s) filling the key.
|
||||
- Dark background, bright line, subtle filled gradient under the curve.
|
||||
- Small current-value label (e.g. `67%`, `336W`, `55°C`) at top; metric
|
||||
name (e.g. `UTIL`, `PWR`) at bottom.
|
||||
- **Scale:** % metrics fixed 0–100; temperature auto 0–100°C; power
|
||||
auto-scaled to observed max.
|
||||
- **Colors:** severity-based — green → amber → red. Temperature red at
|
||||
≥80°C; util/power/fan scale with level.
|
||||
- **Aggregates ("All GPUs"):** Utilization/VRAM/Fan → average; Temperature →
|
||||
max; Power → sum.
|
||||
- **Render-on-change:** idle/flat history does not re-render every poll.
|
||||
- **Press:** opens the web UI (same as Action 1).
|
||||
|
||||
## Error Handling
|
||||
|
||||
- llama-swap unreachable → dark grey key with `!!` and `OFFLINE`; retry with
|
||||
backoff; auto-recover when server returns.
|
||||
- SSE drop → reconnect with backoff (1s → 30s); self-healing via `snapshot`.
|
||||
- `/metrics` poll timeout → skip sample.
|
||||
- Per-button error states — a bad setting on one key does not affect others.
|
||||
|
||||
## Packaging & Assets
|
||||
|
||||
- Package UUID: `com.bryce.llamawatch` (reverse-DNS, unique).
|
||||
- `streamdeck-cli` scaffolds project, builds TypeScript, packages
|
||||
`.streamDeckPlugin` (zip) → installs via double-click into
|
||||
`~/Library/Application Support/com.elgato.StreamDeck/Plugins/`.
|
||||
- Marketplace-ready from the start:
|
||||
- 256×256 plugin icon + action icons (incl. pressed states) at marketplace
|
||||
spec sizes.
|
||||
- Name "llama-watch", category e.g. "System & Monitoring".
|
||||
- Privacy note: reads GPU metrics + request status from the user's own
|
||||
llama-swap server; only base URL + optional API key stored in Stream
|
||||
Deck's local settings; no data leaves the machine.
|
||||
- No hardcoded secrets/URLs — base URL is user-editable.
|
||||
- Manifest declares macOS only initially.
|
||||
|
||||
## Testing
|
||||
|
||||
- Unit tests with `node:test` for the Prometheus parser, inflight tracker,
|
||||
and aggregate math (avg/max/sum), using **real captured fixtures** from
|
||||
the server instance (metrics body, inflight snapshot, modelStatus payloads).
|
||||
- Manual verification: both actions render correctly on the physical Stream
|
||||
Deck; key press opens the web UI.
|
||||
|
||||
## Out of Scope (YAGNI)
|
||||
|
||||
- Marketplace submission itself (goal for later; assets/metadata prepared now).
|
||||
- Auto-update plumbing.
|
||||
- Windows/Linux support (later add-on; macOS declared in manifest).
|
||||
- Additional metrics or display styles.
|
||||
@@ -1,131 +0,0 @@
|
||||
# Usage Stats + GPU Combinations
|
||||
|
||||
**Date:** 2026-08-14
|
||||
**Status:** Approved (design)
|
||||
|
||||
## Summary
|
||||
|
||||
Two extensions to the existing llama-watch actions:
|
||||
|
||||
1. The In-Flight Monitor action gains a **Usage stats** display mode showing
|
||||
token/request totals and the generation-speed P95 from llama-swap's
|
||||
`/api/metrics/stats` endpoint.
|
||||
2. The GPU Graph action gains **GPU combinations**: a key can aggregate an
|
||||
arbitrary subset of GPUs (e.g. only the two Blackwells).
|
||||
|
||||
## Data Source (verified against llama-swap source)
|
||||
|
||||
- `GET /api/metrics/stats` (no query param) → global aggregates.
|
||||
- `GET /api/metrics/stats?model=<id>` → per-model aggregates.
|
||||
- Response fields used: `total_requests`, `total_input_tokens`,
|
||||
`total_output_tokens`, `gen_histogram.p95`.
|
||||
- `p95` is a **tokens/sec generation-speed percentile**, not latency.
|
||||
- Totals cover llama-swap's in-memory activity retention (default ~1000
|
||||
requests), not lifetime counters.
|
||||
- The `/api/events` SSE `activity` event (payload `{"id": N}`) fires per
|
||||
completed request and is used as a "re-poll stats" trigger. It carries no
|
||||
data itself.
|
||||
- The Prometheus `/metrics` endpoint exposes no token/request/latency data
|
||||
and is not used for this feature.
|
||||
|
||||
## 1. Usage stats data layer
|
||||
|
||||
- **`src/lib/stats.ts`** (new):
|
||||
- `interface UsageStats { totalRequests: number; totalInputTokens: number; totalOutputTokens: number; genP95: number }`.
|
||||
- `parseStats(json: unknown): UsageStats | null` — defensive; returns
|
||||
`null` on malformed input; missing fields default to `0`.
|
||||
- `fetchStats(cfg: LlamaSwapConfig, modelId: string): Promise<UsageStats | null>` —
|
||||
fetches `/api/metrics/stats` (with `?model=` when `modelId` is not
|
||||
`"all"`), returns parsed stats or `null` on HTTP error/parse failure.
|
||||
- **`src/lib/stats-cache.ts`** (new):
|
||||
- `StatsCache` holds `Set<string>` of registered model keys (`"all"` or a
|
||||
model id) and a `Map<key, UsageStats | null>` of last-known values.
|
||||
- One shared 5s interval polls every registered key via `fetchStats`.
|
||||
- `register(key)` / `unregister(key)` start/stop the interval as needed
|
||||
(no ref counting — each key registers/unregisters once).
|
||||
- `get(key): UsageStats | undefined`.
|
||||
- `refresh()` re-polls all registered keys.
|
||||
- Change listener invoked after each poll completes (success or failure).
|
||||
- Fetch failure keeps the last-known value.
|
||||
- **`src/lib/event-feed.ts`**: `decodeEvent` learns the `activity` event —
|
||||
`{ type: "activity", id: number }`. `FeedEvent` (in `inflight-tracker.ts`)
|
||||
gains the `activity` variant.
|
||||
- **`src/lib/runtime.ts`**: runtime owns a `StatsCache` (created alongside
|
||||
feed/poller, reset on config change). Feed `activity` events trigger
|
||||
`statsCache.refresh()` throttled to at most once per 2s. Runtime exposes:
|
||||
- `getStats(modelId: string): UsageStats | undefined`
|
||||
- `watchStats(modelId: string): () => void` (register; returns
|
||||
unsubscribe).
|
||||
- Stats changes flow through the existing `subscribe`/`emit` mechanism so
|
||||
actions re-render on the normal tick.
|
||||
|
||||
## 2. In-Flight action modes
|
||||
|
||||
- New settings (per key, in `inflight-monitor.ts`):
|
||||
- `display: "count" | "usage"` (default `count`).
|
||||
- `primaryStat: "requests" | "input_tokens" | "output_tokens" | "gen_p95"`
|
||||
(default `requests`).
|
||||
- Model setting semantics:
|
||||
- Model select (PI datasource) gains an **"All models"** option, value
|
||||
`all`.
|
||||
- Count mode: `all` → total in-flight across all models; label
|
||||
`ALL MODELS`.
|
||||
- Usage mode: `all` → global stats; otherwise per-model stats.
|
||||
- **Count mode**: unchanged rendering (count number, spark, `IDLE`/
|
||||
`LOADING`/`OFF`/`OFFLINE`). The 1s spark sampler runs only in count mode.
|
||||
- **Usage mode** (`renderUsage` in `render.ts`, dark `#10131a` bg):
|
||||
- top: model short name or `ALL MODELS` (7px).
|
||||
- center (24px bold): the `primaryStat` value, compact-formatted —
|
||||
`< 1000` integer, else `x.xk`, else `x.xM` (`genP95` shows integer t/s).
|
||||
- three small rows (7px): the remaining three stats, labeled `REQ`,
|
||||
`IN`, `OUT`, `P95`.
|
||||
- offline → `OFFLINE` overlay; stats unknown → `--`.
|
||||
- Press opens the UI (unchanged).
|
||||
|
||||
## 3. GPU combinations
|
||||
|
||||
- New setting `gpuCombo?: string` — comma-separated GPU ids, e.g. `"0,2"`.
|
||||
When non-empty it overrides `gpuId`.
|
||||
- Pure helper in `metrics-poller.ts` (exported, tested):
|
||||
`combineSeries(histories: number[][], kind): { value?: number; history: number[] }`
|
||||
— element-wise across the per-GPU rings, aligned by index, applying the
|
||||
existing aggregate rules: `util_percent`/`memory_util_percent`/`fan` →
|
||||
average, `temperature` → max, `power` → sum. A trailing `undefined` value
|
||||
from an empty ring leaves that element undefined.
|
||||
- `gpu-graph.ts` render: for a combo, build per-GPU histories via
|
||||
`poller.getHistory(id, metric)`, combine, use the combined value/history.
|
||||
Unknown ids are filtered out; if none remain → `--`.
|
||||
- Label: `GPU 0+2` (ids joined with `+`).
|
||||
|
||||
## 4. Property inspectors
|
||||
|
||||
- `ui/inflight.html`: add "Display" select (`count`/`usage`) and "Primary
|
||||
stat" select. Both always visible; `primaryStat` has no effect in count
|
||||
mode.
|
||||
- `src/lib/datasources.ts`: the `models` datasource prepends
|
||||
`{ label: "All models", value: "all" }` to its items (so the model select
|
||||
offers both the aggregate and individual models).
|
||||
- `ui/gpu.html`: add "GPU combination (comma-separated ids)" textfield,
|
||||
optional, placeholder `e.g. 0,2`.
|
||||
|
||||
## Error Handling / Edge Cases
|
||||
|
||||
- Stats fetch error → keep last-known values; no `OFFLINE` flip (offline is
|
||||
driven by feed status as today).
|
||||
- Server offline → existing `OFFLINE` overlay in both modes.
|
||||
- Stats not yet loaded → `--` placeholders.
|
||||
- Activity-triggered refresh throttled to 1 per 2s.
|
||||
- Combo with no valid ids → `--`.
|
||||
- No manifest change (both features extend existing actions).
|
||||
|
||||
## Testing
|
||||
|
||||
- `parseStats`: happy path; malformed input; missing fields → `0`.
|
||||
- `StatsCache`: register/unregister lifecycle (interval start/stop),
|
||||
single poll per model, values cached, refresh on demand, failure keeps
|
||||
last value. Uses an injected fetch function.
|
||||
- `decodeEvent`: `activity` event decodes to `{ type: "activity", id }`.
|
||||
- `renderUsage`: big primary stat, small labeled rows, compact formatting
|
||||
(`52.1k`), `--` when no data, `ALL MODELS` label, offline overlay.
|
||||
- `combineSeries`: avg/max/sum per metric; empty/mismatched histories.
|
||||
- In-Flight action: `all` count sums models.
|
||||
Reference in New Issue
Block a user