From 6d710b208945b4fca576160c4dbc12d539e15706 Mon Sep 17 00:00:00 2001 From: Bryce Zuccaro Date: Fri, 14 Aug 2026 13:17:34 -0600 Subject: [PATCH] docs: remove superpowers planning documents --- .../plans/2026-08-14-llama-watch.md | 2152 ----------------- .../2026-08-14-usage-stats-gpu-combos.md | 1129 --------- .../2026-08-14-inflight-activity-design.md | 61 - .../specs/2026-08-14-llama-watch-design.md | 166 -- ...026-08-14-usage-stats-gpu-combos-design.md | 131 - 5 files changed, 3639 deletions(-) delete mode 100644 docs/superpowers/plans/2026-08-14-llama-watch.md delete mode 100644 docs/superpowers/plans/2026-08-14-usage-stats-gpu-combos.md delete mode 100644 docs/superpowers/specs/2026-08-14-inflight-activity-design.md delete mode 100644 docs/superpowers/specs/2026-08-14-llama-watch-design.md delete mode 100644 docs/superpowers/specs/2026-08-14-usage-stats-gpu-combos-design.md diff --git a/docs/superpowers/plans/2026-08-14-llama-watch.md b/docs/superpowers/plans/2026-08-14-llama-watch.md deleted file mode 100644 index b3ec159..0000000 --- a/docs/superpowers/plans/2026-08-14-llama-watch.md +++ /dev/null @@ -1,2152 +0,0 @@ -# llama-watch Stream Deck Plugin — Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** Build a local-install Stream Deck plugin ("llama-watch") with two key actions — an in-flight request monitor per model and a live GPU metric graph — backed by a llama-swap instance at `http://localhost:9292`. - -**Architecture:** A Node.js/TypeScript plugin running as a process launched by the Stream Deck app (SDK v2, `@elgato/streamdeck@2.1.1`). It maintains a persistent SSE connection to llama-swap `/api/events` (real-time in-flight + model status) and a 5s polling loop over `/metrics` (Prometheus text). Pure-logic modules (parser, tracker, poller, SSE parser, SVG renderer) are unit-tested with `node:test` + `tsx`. Key images are rendered as dynamic SVG strings passed to `KeyAction.setImage` — no canvas dependency. - -**Tech Stack:** TypeScript, `@elgato/streamdeck@^2.1.1`, Rollup (`@rollup/plugin-typescript`, terser), `@elgato/cli@^1.8.1` for validate/pack, `tsx` for tests, Node 24 runtime declared in the manifest (Stream Deck bundles it; the SDK requires ≥24). Icons generated from hand-written SVGs via macOS `sips`. - -## Global Constraints - -- Plugin UUID: `com.bryce.llamawatch` (reverse-DNS; lowercase alphanumeric, hyphens, periods only). -- Action UUIDs: `com.bryce.llamawatch.inflight`, `com.bryce.llamawatch.gpu`. -- Manifest: `SDKVersion: 2`, `Nodejs.Version: "24"`, macOS only (`OS: [{Platform: "mac", MinimumVersion: "13"}]`), `Software.MinimumVersion: "7.1"`. (Verified against `@elgato/schemas`: Nodejs `"24"` requires Software ≥ 7.1; the user's Stream Deck app is 7.4.2.) -- Dependency versions pinned per the official hello-world sample: `@elgato/streamdeck@^2.1.1`, `@elgato/cli@^1.8.1`, `@rollup/plugin-commonjs@^28.0.2`, `@rollup/plugin-node-resolve@^16.0.0`, `@rollup/plugin-terser@^0.4.4`, `@rollup/plugin-typescript@^12.1.2`, `@tsconfig/node20@^20.1.4`, `@types/node@22.13.0`, `rollup@^4.32.1`, `tslib@^2.8.1`, `typescript@^5.7.3`, `tsx@^4.19.2`. -- Poll interval for `/metrics`: fixed **5000 ms**; ring buffer **60 samples** (~5 min). -- Metrics (from `llamaswap_gpu_*` Prometheus series): `util_percent`, `memory_util_percent`, `temperature`, `power`, `fan`. -- Aggregates for GPU selector `"all"`: util/memory/fan = average, temperature = max, power = sum. -- Default base URL: `http://localhost:9292`. No hardcoded URLs beyond the default. Optional API key sent as `Authorization: Bearer ` when set. -- Key press on either action opens `/ui` in the default browser via `streamDeck.system.openUrl`. -- No Marketplace submission in scope; assets are prepared marketplace-ready (plugin PNG 256/512, SVG action icons, category icon). -- No code comments. Follow the no-comments rule strictly. -- Existing conventions from the official sample: entry `src/plugin.ts`, output `${PLUGIN}.sdPlugin/bin/plugin.js`, extension-less relative TS imports, ESM (`"type": "module"`). - ---- - -### Task 1: Scaffold project structure - -**Files:** -- Create: `package.json` -- Create: `tsconfig.json` -- Create: `rollup.config.mjs` -- Create: `com.bryce.llamawatch.sdPlugin/manifest.json` (placeholder manifest) -- Create: `src/plugin.ts` (placeholder) -- Create: `src/actions/inflight-monitor.ts` (placeholder), `src/actions/gpu-graph.ts` (placeholder) -- Create: `tests/.gitkeep` -- Create: `.gitignore` - -**Interfaces:** -- Consumes: nothing. -- Produces: working `npm run build` (rollup outputs `com.bryce.llamawatch.sdPlugin/bin/plugin.js`), `npm test` runner available, placeholder plugin registers no actions yet. - -- [ ] **Step 1: Create `package.json`** - -```json -{ - "name": "llama-watch", - "version": "1.0.0", - "description": "Stream Deck plugin to monitor llama-swap requests and GPU metrics.", - "private": true, - "type": "module", - "scripts": { - "build": "rollup -c", - "watch": "rollup -c -w --watch.onEnd=\"streamdeck restart com.bryce.llamawatch\"", - "test": "tsx --test tests/" - }, - "devDependencies": { - "@elgato/cli": "^1.8.1", - "@rollup/plugin-commonjs": "^28.0.2", - "@rollup/plugin-node-resolve": "^16.0.0", - "@rollup/plugin-terser": "^0.4.4", - "@rollup/plugin-typescript": "^12.1.2", - "@tsconfig/node20": "^20.1.4", - "@types/node": "22.13.0", - "rollup": "^4.32.1", - "tslib": "^2.8.1", - "tsx": "^4.19.2", - "typescript": "^5.7.3" - }, - "dependencies": { - "@elgato/streamdeck": "^2.1.1" - } -} -``` - -- [ ] **Step 2: Create `tsconfig.json`** - -```json -{ - "extends": "@tsconfig/node20/tsconfig.json", - "compilerOptions": { - "customConditions": ["node"], - "noImplicitOverride": true, - "module": "ES2022", - "moduleResolution": "Bundler" - }, - "include": ["src/**/*.ts"], - "exclude": ["node_modules"] -} -``` - -- [ ] **Step 3: Create `rollup.config.mjs`** - -```js -import commonjs from "@rollup/plugin-commonjs"; -import nodeResolve from "@rollup/plugin-node-resolve"; -import terser from "@rollup/plugin-terser"; -import typescript from "@rollup/plugin-typescript"; -import path from "node:path"; -import url from "node:url"; - -const isWatching = !!process.env.ROLLUP_WATCH; -const sdPlugin = "com.bryce.llamawatch.sdPlugin"; - -const config = { - input: "src/plugin.ts", - output: { - file: `${sdPlugin}/bin/plugin.js`, - sourcemap: isWatching, - sourcemapPathTransform: (relativeSourcePath, sourcemapPath) => { - return url.pathToFileURL(path.resolve(path.dirname(sourcemapPath), relativeSourcePath)).href; - }, - }, - plugins: [ - { - name: "watch-externals", - buildStart: function () { - this.addWatchFile(`${sdPlugin}/manifest.json`); - }, - }, - typescript({ mapRoot: isWatching ? "./" : undefined }), - nodeResolve({ browser: false, exportConditions: ["node"], preferBuiltins: true }), - commonjs(), - !isWatching && terser(), - { - name: "emit-module-package-file", - generateBundle() { - this.emitFile({ fileName: "package.json", source: `{ "type": "module" }`, type: "asset" }); - }, - }, - ], -}; - -export default config; -``` - -- [ ] **Step 4: Create `.gitignore`** - -``` -node_modules/ -*.streamDeckPlugin -.DS_Store -``` - -- [ ] **Step 5: Create placeholder `com.bryce.llamawatch.sdPlugin/manifest.json`** - -```json -{ - "$schema": "https://schemas.elgato.com/streamdeck/plugins/manifest.json", - "Name": "llama-watch", - "Version": "1.0.0.0", - "Author": "Bryce Zuccaro", - "Actions": [], - "Category": "llama-watch", - "CodePath": "bin/plugin.js", - "Description": "Monitor llama-swap: in-flight requests per model and live GPU metric graphs.", - "Icon": "imgs/plugin/icon", - "SDKVersion": 2, - "Software": { "MinimumVersion": "7.1" }, - "OS": [{ "Platform": "mac", "MinimumVersion": "13" }], - "Nodejs": { "Version": "24" }, - "UUID": "com.bryce.llamawatch" -} -``` - -- [ ] **Step 6: Create placeholder `src/plugin.ts`** - -```ts -import streamDeck from "@elgato/streamdeck"; - -streamDeck.connect(); -``` - -- [ ] **Step 7: Create placeholder action files** — `src/actions/inflight-monitor.ts` and `src/actions/gpu-graph.ts`, each containing only: - -```ts -export {}; -``` - -- [ ] **Step 8: Create `tests/.gitkeep`** (empty file) - -- [ ] **Step 9: Install dependencies and verify the build and test runner** - -Run: -```bash -npm install -npm run build -npm test -``` -Expected: `npm run build` produces `com.bryce.llamawatch.sdPlugin/bin/plugin.js`; `npm test` exits 0 (no test files yet). - -- [ ] **Step 10: Commit** - -```bash -git add -A -git commit -m "chore: scaffold llama-watch Stream Deck plugin" -``` - ---- - -### Task 2: `util.ts` — shared helpers and config - -**Files:** -- Create: `src/lib/util.ts` -- Test: `tests/util.test.ts` - -**Interfaces:** -- Consumes: nothing. -- Produces: `DEFAULT_BASE_URL`, `CfgSettings`, `LlamaSwapConfig`, `cfgFromSettings(settings): LlamaSwapConfig`, `normalizeBaseUrl(s): string`, `shorten(name, max?): string`, `escapeXml(s): string`. - -- [ ] **Step 1: Write the failing test** — `tests/util.test.ts` - -```ts -import assert from "node:assert/strict"; -import { test } from "node:test"; -import { - cfgFromSettings, - escapeXml, - normalizeBaseUrl, - shorten, -} from "../src/lib/util"; - -test("normalizeBaseUrl strips trailing slashes", () => { - assert.equal(normalizeBaseUrl("http://localhost:9292/"), "http://localhost:9292"); - assert.equal(normalizeBaseUrl("http://localhost:9292"), "http://localhost:9292"); -}); - -test("cfgFromSettings defaults baseUrl and omits empty apiKey", () => { - const cfg = cfgFromSettings({}); - assert.equal(cfg.baseUrl, "http://localhost:9292"); - assert.equal(cfg.apiKey, undefined); - assert.deepEqual(cfgFromSettings({ baseUrl: "http://x/", apiKey: "abc" }), { - baseUrl: "http://x", - apiKey: "abc", - }); -}); - -test("shorten keeps short names and truncates long ones", () => { - assert.equal(shorten("Qwen3.8-27B"), "Qwen3.8-27B"); - assert.equal(shorten("DeepSeek-V4-Flash-0731"), "DeepSeek-…0731"); -}); - -test("escapeXml escapes XML special characters", () => { - assert.equal(escapeXml(`A&B "D" 'E'`), "A&B <C> "D" 'E'"); -}); -``` - -- [ ] **Step 2: Run test to verify it fails** - -Run: `npm test` -Expected: FAIL — module `../src/lib/util` not found. - -- [ ] **Step 3: Implement `src/lib/util.ts`** - -```ts -import type { JsonObject } from "@elgato/utils"; - -export const DEFAULT_BASE_URL = "http://localhost:9292"; - -export type CfgSettings = { - baseUrl?: string; - apiKey?: string; -} & JsonObject; - -export interface LlamaSwapConfig { - baseUrl: string; - apiKey?: string; -} - -export function cfgFromSettings(settings: CfgSettings): LlamaSwapConfig { - return { - baseUrl: normalizeBaseUrl(settings.baseUrl ?? DEFAULT_BASE_URL), - apiKey: settings.apiKey || undefined, - }; -} - -export function normalizeBaseUrl(baseUrl: string): string { - return baseUrl.trim().replace(/\/+$/, ""); -} - -export function shorten(name: string, max = 14): string { - if (name.length <= max) return name; - const head = Math.floor((max - 1) * 0.7); - return `${name.slice(0, head)}…${name.slice(name.length - (max - head - 1))}`; -} - -export function escapeXml(text: string): string { - return text - .replace(/&/g, "&") - .replace(//g, ">") - .replace(/"/g, """) - .replace(/'/g, "'"); -} -``` - -- [ ] **Step 4: Run test to verify it passes** - -Run: `npm test` -Expected: PASS. - -- [ ] **Step 5: Commit** - -```bash -git add -A -git commit -m "feat: add shared util helpers and config" -``` - ---- - -### Task 3: `metrics-parser.ts` — Prometheus text parser - -**Files:** -- Create: `src/lib/metrics-parser.ts` -- Create: `tests/fixtures/metrics.txt` -- Test: `tests/metrics-parser.test.ts` - -**Interfaces:** -- Consumes: nothing. -- Produces: - - `type GpuMetricKind = "util_percent" | "memory_util_percent" | "temperature" | "power" | "fan"` - - `interface GpuSample { id: string; name: string; kind: GpuMetricKind; value: number }` - - `parsePrometheus(text: string): { metric: string; labels: Record; value: number }[]` - - `parseGpuMetrics(text: string): GpuSample[]` - - `gpuInfos(samples: GpuSample[]): { id: string; name: string }[]` (deduped, sorted by numeric id) - -- [ ] **Step 1: Create the fixture** — `tests/fixtures/metrics.txt` (trimmed capture from localhost:9292) - -```text -# HELP llamaswap_gpu_util_percent GPU utilization percent (0-100) -# TYPE llamaswap_gpu_util_percent gauge -llamaswap_gpu_util_percent{id="0",name="NVIDIA RTX PRO 6000 Blackwell Workstation Edition",uuid="GPU-b77149e6-c22d-599e-9728-c7d4cfee0eb4"} 100 -llamaswap_gpu_util_percent{id="1",name="NVIDIA RTX PRO 6000 Blackwell Workstation Edition",uuid="GPU-a46fadfa-48d9-c9c4-8fde-9bd641142bb7"} 100 -llamaswap_gpu_util_percent{id="2",name="NVIDIA GeForce RTX 5090",uuid="GPU-9a59ac37-12fb-5bc4-4767-4f4347898c0a"} 100 -# HELP llamaswap_gpu_memory_util_percent GPU memory utilization percent (0-100) -# TYPE llamaswap_gpu_memory_util_percent gauge -llamaswap_gpu_memory_util_percent{id="0",name="NVIDIA RTX PRO 6000 Blackwell Workstation Edition",uuid="GPU-b77149e6-c22d-599e-9728-c7d4cfee0eb4"} 98.41347676402383 -llamaswap_gpu_memory_util_percent{id="1",name="NVIDIA RTX PRO 6000 Blackwell Workstation Edition",uuid="GPU-a46fadfa-48d9-c9c4-8fde-9bd641142bb7"} 98.41245517790922 -llamaswap_gpu_memory_util_percent{id="2",name="NVIDIA GeForce RTX 5090",uuid="GPU-9a59ac37-12fb-5bc4-4767-4f4347898c0a"} 97.02211181648113 -# HELP llamaswap_gpu_temperature_celsius GPU temperature in Celsius -# TYPE llamaswap_gpu_temperature_celsius gauge -llamaswap_gpu_temperature_celsius{id="0",name="NVIDIA RTX PRO 6000 Blackwell Workstation Edition",uuid="GPU-b77149e6-c22d-599e-9728-c7d4cfee0eb4"} 51 -llamaswap_gpu_temperature_celsius{id="1",name="NVIDIA RTX PRO 6000 Blackwell Workstation Edition",uuid="GPU-a46fadfa-48d9-c9c4-8fde-9bd641142bb7"} 55 -llamaswap_gpu_temperature_celsius{id="2",name="NVIDIA GeForce RTX 5090",uuid="GPU-9a59ac37-12fb-5bc4-4767-4f4347898c0a"} 64 -# HELP llamaswap_gpu_power_draw_watts GPU power draw in watts -# TYPE llamaswap_gpu_power_draw_watts gauge -llamaswap_gpu_power_draw_watts{id="0",name="NVIDIA RTX PRO 6000 Blackwell Workstation Edition",uuid="GPU-b77149e6-c22d-599e-9728-c7d4cfee0eb4"} 316.56 -llamaswap_gpu_power_draw_watts{id="1",name="NVIDIA RTX PRO 6000 Blackwell Workstation Edition",uuid="GPU-a46fadfa-48d9-c9c4-8fde-9bd641142bb7"} 336.17 -llamaswap_gpu_power_draw_watts{id="2",name="NVIDIA GeForce RTX 5090",uuid="GPU-9a59ac37-12fb-5bc4-4767-4f4347898c0a"} 377.28 -# HELP llamaswap_gpu_fan_speed_percent GPU fan speed percent (0-100) -# TYPE llamaswap_gpu_fan_speed_percent gauge -llamaswap_gpu_fan_speed_percent{id="0",name="NVIDIA RTX PRO 6000 Blackwell Workstation Edition",uuid="GPU-b77149e6-c22d-599e-9728-c7d4cfee0eb4"} 51 -llamaswap_gpu_fan_speed_percent{id="1",name="NVIDIA RTX PRO 6000 Blackwell Workstation Edition",uuid="GPU-a46fadfa-48d9-c9c4-8fde-9bd641142bb7"} 62 -llamaswap_gpu_fan_speed_percent{id="2",name="NVIDIA GeForce RTX 5090",uuid="GPU-9a59ac37-12fb-5bc4-4767-4f4347898c0a"} 89 -``` - -- [ ] **Step 2: Write the failing test** — `tests/metrics-parser.test.ts` - -```ts -import assert from "node:assert/strict"; -import { readFileSync } from "node:fs"; -import { test } from "node:test"; -import { gpuInfos, parseGpuMetrics, parsePrometheus } from "../src/lib/metrics-parser"; - -const FIXTURE = readFileSync(new URL("./fixtures/metrics.txt", import.meta.url), "utf8"); - -test("parsePrometheus parses a gauge with labels", () => { - const samples = parsePrometheus(`# TYPE llamaswap_gpu_util_percent gauge\nllamaswap_gpu_util_percent{id="0",name="RTX 5090"} 42.5\n`); - assert.equal(samples.length, 1); - assert.equal(samples[0].metric, "llamaswap_gpu_util_percent"); - assert.deepEqual(samples[0].labels, { id: "0", name: "RTX 5090" }); - assert.equal(samples[0].value, 42.5); -}); - -test("parsePrometheus ignores comments and blank lines", () => { - const samples = parsePrometheus("# HELP x y\n# TYPE x gauge\n\n"); - assert.equal(samples.length, 0); -}); - -test("parseGpuMetrics extracts all five kinds for three GPUs from the fixture", () => { - const samples = parseGpuMetrics(FIXTURE); - assert.equal(samples.length, 15); - const kinds = new Set(samples.map((s) => s.kind)); - assert.deepEqual([...kinds].sort(), ["fan", "memory_util_percent", "power", "temperature", "util_percent"]); - const power0 = samples.find((s) => s.kind === "power" && s.id === "0"); - assert.ok(power0); - assert.equal(power0.value, 316.56); -}); - -test("gpuInfos dedupes, preserves names, sorts numerically by id", () => { - const infos = gpuInfos(parseGpuMetrics(FIXTURE)); - assert.deepEqual(infos.map((i) => i.id), ["0", "1", "2"]); - assert.equal(infos[2].name, "NVIDIA GeForce RTX 5090"); -}); -``` - -- [ ] **Step 3: Run test to verify it fails** - -Run: `npm test` -Expected: FAIL — module not found. - -- [ ] **Step 4: Implement `src/lib/metrics-parser.ts`** - -```ts -export interface PromSample { - metric: string; - labels: Record; - value: number; -} - -const LINE_RE = /^([A-Za-z_:][A-Za-z0-9_:]*)(?:\{([^}]*)\})?\s+(-?(?:\d+\.?\d*|\.\d+)(?:[eE][+-]?\d+)?)$/; - -export function parsePrometheus(text: string): PromSample[] { - const samples: PromSample[] = []; - for (const rawLine of text.split("\n")) { - const line = rawLine.trim(); - if (!line || line.startsWith("#")) continue; - const m = LINE_RE.exec(line); - if (!m) continue; - const labels: Record = {}; - if (m[2]) { - for (const pair of m[2].split(",")) { - const eq = pair.indexOf("="); - if (eq === -1) continue; - const value = pair.slice(eq + 1).trim(); - labels[pair.slice(0, eq).trim()] = value.replace(/^"(.*)"$/, "$1"); - } - } - samples.push({ metric: m[1], labels, value: parseFloat(m[3]) }); - } - return samples; -} - -export type GpuMetricKind = "util_percent" | "memory_util_percent" | "temperature" | "power" | "fan"; - -export const GPU_SERIES: Record = { - llamaswap_gpu_util_percent: "util_percent", - llamaswap_gpu_memory_util_percent: "memory_util_percent", - llamaswap_gpu_temperature_celsius: "temperature", - llamaswap_gpu_power_draw_watts: "power", - llamaswap_gpu_fan_speed_percent: "fan", -}; - -export interface GpuSample { - id: string; - name: string; - kind: GpuMetricKind; - value: number; -} - -export function parseGpuMetrics(text: string): GpuSample[] { - const out: GpuSample[] = []; - for (const s of parsePrometheus(text)) { - const kind = GPU_SERIES[s.metric]; - if (!kind) continue; - const id = s.labels["id"]; - if (id === undefined) continue; - out.push({ id, name: s.labels["name"] ?? "", kind, value: s.value }); - } - return out; -} - -export function gpuInfos(samples: GpuSample[]): { id: string; name: string }[] { - const byId = new Map(); - for (const s of samples) { - if (!byId.has(s.id)) byId.set(s.id, s.name); - } - return [...byId.entries()] - .sort((a, b) => parseInt(a[0], 10) - parseInt(b[0], 10)) - .map(([id, name]) => ({ id, name })); -} -``` - -- [ ] **Step 5: Run test to verify it passes** - -Run: `npm test` -Expected: PASS. - -- [ ] **Step 6: Commit** - -```bash -git add -A -git commit -m "feat: add Prometheus metrics parser for llama-swap GPU series" -``` - ---- - -### Task 4: `llamaswap.ts` — HTTP client - -**Files:** -- Create: `src/lib/llamaswap.ts` - -**Interfaces:** -- Consumes: `LlamaSwapConfig` from `./util`. -- Produces: - - `interface ModelInfo { id: string; status: string }` - - `fetchMetrics(cfg: LlamaSwapConfig): Promise` — GET `/metrics`, throws on non-2xx or timeout (5s). - - `fetchModels(cfg: LlamaSwapConfig): Promise` — GET `/v1/models`, maps `data[]` to `{id, status: status.value}`. - -- [ ] **Step 1: Implement `src/lib/llamaswap.ts`** (network module; verified via `npm run build` typecheck, and integration-tested in Task 12) - -```ts -import { type LlamaSwapConfig } from "./util"; - -const TIMEOUT_MS = 5000; - -export interface ModelInfo { - id: string; - status: string; -} - -function headersFor(cfg: LlamaSwapConfig): Record { - const headers: Record = {}; - if (cfg.apiKey) headers["Authorization"] = `Bearer ${cfg.apiKey}`; - return headers; -} - -export async function fetchMetrics(cfg: LlamaSwapConfig): Promise { - const res = await fetch(`${cfg.baseUrl}/metrics`, { - headers: headersFor(cfg), - signal: AbortSignal.timeout(TIMEOUT_MS), - }); - if (!res.ok) throw new Error(`metrics HTTP ${res.status}`); - return await res.text(); -} - -export async function fetchModels(cfg: LlamaSwapConfig): Promise { - const res = await fetch(`${cfg.baseUrl}/v1/models`, { - headers: headersFor(cfg), - signal: AbortSignal.timeout(TIMEOUT_MS), - }); - if (!res.ok) throw new Error(`models HTTP ${res.status}`); - const body = (await res.json()) as { data?: { id: string; status?: { value?: string } }[] }; - return (body.data ?? []).map((m) => ({ id: m.id, status: m.status?.value ?? "unknown" })); -} -``` - -- [ ] **Step 2: Verify it typechecks in the bundle** - -Run: `npm run build` -Expected: build succeeds (rollup/typescript compiles without error). - -- [ ] **Step 3: Commit** - -```bash -git add -A -git commit -m "feat: add llama-swap HTTP client" -``` - ---- - -### Task 5: `metrics-poller.ts` — polling, ring buffers, aggregates - -**Files:** -- Create: `src/lib/metrics-poller.ts` -- Test: `tests/metrics-poller.test.ts` - -**Interfaces:** -- Consumes: `fetchMetrics` + `LlamaSwapConfig` (default fetchFn), `parseGpuMetrics` + `GpuMetricKind` from `./metrics-parser`. -- Produces: - - `class MetricsPoller { constructor(cfg: LlamaSwapConfig, intervalMs?: number, fetchFn?: (cfg: LlamaSwapConfig) => Promise) }` - - `start(): void`, `stop(): void`, `tick(): Promise` (public for tests) - - `on(listener: () => void): () => void` (returns unsubscribe) - - `getValue(gpuId: string, kind: GpuMetricKind): number | undefined` - - `getHistory(gpuId: string, kind: GpuMetricKind): number[]` (copy of ring) - - `gpus(): { id: string; name: string }[]` - - `isOffline(): boolean` - -- [ ] **Step 1: Write the failing test** — `tests/metrics-poller.test.ts` - -```ts -import assert from "node:assert/strict"; -import { readFileSync } from "node:fs"; -import { test } from "node:test"; -import { type LlamaSwapConfig } from "../src/lib/util"; -import { MetricsPoller } from "../src/lib/metrics-poller"; -import { parseGpuMetrics } from "../src/lib/metrics-parser"; - -const FIXTURE = readFileSync(new URL("./fixtures/metrics.txt", import.meta.url), "utf8"); -const FIXTURE2 = FIXTURE - .replace(/llamaswap_gpu_util_percent\{id="0"[^}]*\} 100/, 'llamaswap_gpu_util_percent{id="0",name="RTX PRO 6000"} 30') - .replace(/llamaswap_gpu_util_percent\{id="1"[^}]*\} 100/, 'llamaswap_gpu_util_percent{id="1",name="RTX PRO 6000"} 50') - .replace(/llamaswap_gpu_util_percent\{id="2"[^}]*\} 100/, 'llamaswap_gpu_util_percent{id="2",name="RTX 5090"} 80'); - -function stubFetch(bodies: string[]) { - let i = 0; - return async () => bodies[Math.min(i++, bodies.length - 1)]; -} - -const CFG: LlamaSwapConfig = { baseUrl: "http://test" }; - -test("poller stores per-GPU history and computes aggregates", async () => { - const poller = new MetricsPoller(CFG, 5000, stubFetch([FIXTURE, FIXTURE2])); - await poller.tick(); - await poller.tick(); - - assert.equal(poller.getValue("0", "util_percent"), 30); - assert.deepEqual(poller.getHistory("0", "util_percent"), [100, 30]); - - const allUtil = poller.getValue("all", "util_percent"); - assert.ok(allUtil !== undefined); - assert.ok(Math.abs(allUtil - (30 + 50 + 80) / 3) < 1e-9); - - const allTemp = poller.getValue("all", "temperature"); - assert.equal(allTemp, 64); - - const allPower = poller.getValue("all", "power"); - assert.ok(allPower !== undefined); - assert.ok(Math.abs(allPower - (316.56 + 336.17 + 377.28)) < 1e-9); - - assert.deepEqual(poller.gpus().map((g) => g.id), ["0", "1", "2"]); - assert.equal(poller.gpus()[2].name, "NVIDIA GeForce RTX 5090"); -}); - -test("poller ring buffer caps at 60 samples", async () => { - const poller = new MetricsPoller(CFG, 5000, stubFetch([FIXTURE])); - for (let i = 0; i < 70; i++) await poller.tick(); - assert.equal(poller.getHistory("0", "fan").length, 60); -}); - -test("poller reports offline after a fetch failure and recovers", async () => { - let fail = true; - const fetchFn = async () => { - if (fail) throw new Error("boom"); - return FIXTURE; - }; - const poller = new MetricsPoller(CFG, 5000, fetchFn); - await poller.tick(); - assert.equal(poller.isOffline(), true); - fail = false; - await poller.tick(); - assert.equal(poller.isOffline(), false); - assert.ok(poller.getValue("0", "util_percent") !== undefined); -}); -``` - -- [ ] **Step 2: Run test to verify it fails** - -Run: `npm test` -Expected: FAIL — module not found. - -- [ ] **Step 3: Implement `src/lib/metrics-poller.ts`** - -```ts -import { fetchMetrics } from "./llamaswap"; -import { type LlamaSwapConfig } from "./util"; -import { gpuInfos, parseGpuMetrics, type GpuMetricKind, type GpuSample } from "./metrics-parser"; - -const RING_SIZE = 60; - -function avg(values: number[]): number { - return values.reduce((a, b) => a + b, 0) / (values.length || 1); -} -function max(values: number[]): number { - return values.reduce((a, b) => Math.max(a, b), -Infinity); -} -function sum(values: number[]): number { - return values.reduce((a, b) => a + b, 0); -} - -const AGGREGATE: Record number> = { - util_percent: avg, - memory_util_percent: avg, - temperature: max, - power: sum, - fan: avg, -}; - -function key(gpuId: string, kind: GpuMetricKind): string { - return `${gpuId}|${kind}`; -} - -type FetchFn = (cfg: LlamaSwapConfig) => Promise; - -export class MetricsPoller { - private buffers = new Map(); - private gpuName = new Map(); - private gpuIds: string[] = []; - private listeners = new Set<() => void>(); - private timer?: ReturnType; - private lastError?: string; - - constructor( - private cfg: LlamaSwapConfig, - private intervalMs = 5000, - private fetchFn: FetchFn = fetchMetrics, - ) {} - - start(): void { - void this.tick(); - this.timer = setInterval(() => void this.tick(), this.intervalMs); - } - - stop(): void { - if (this.timer) clearInterval(this.timer); - this.timer = undefined; - } - - on(listener: () => void): () => void { - this.listeners.add(listener); - return () => { - this.listeners.delete(listener); - }; - } - - async tick(): Promise { - try { - const text = await this.fetchFn(this.cfg); - this.apply(parseGpuMetrics(text)); - this.lastError = undefined; - } catch (err) { - this.lastError = err instanceof Error ? err.message : String(err); - } finally { - for (const listener of this.listeners) listener(); - } - } - - private apply(samples: GpuSample[]): void { - const byKind = new Map>(); - for (const s of samples) { - if (!byKind.has(s.kind)) byKind.set(s.kind, new Map()); - byKind.get(s.kind)!.set(s.id, s.value); - if (!this.gpuName.has(s.id)) this.gpuName.set(s.id, s.name); - } - this.gpuIds = [...this.gpuName.keys()].sort((a, b) => parseInt(a, 10) - parseInt(b, 10)); - - for (const [kind, values] of byKind) { - for (const [id, value] of values) this.push(key(id, kind), value); - const all = [...values.values()]; - if (all.length > 0) this.push(key("all", kind), AGGREGATE[kind](all)); - } - } - - private push(k: string, value: number): void { - let ring = this.buffers.get(k); - if (!ring) { - ring = []; - this.buffers.set(k, ring); - } - ring.push(value); - if (ring.length > RING_SIZE) ring.shift(); - } - - getValue(gpuId: string, kind: GpuMetricKind): number | undefined { - const ring = this.buffers.get(key(gpuId, kind)); - return ring ? ring[ring.length - 1] : undefined; - } - - getHistory(gpuId: string, kind: GpuMetricKind): number[] { - const ring = this.buffers.get(key(gpuId, kind)); - return ring ? [...ring] : []; - } - - gpus(): { id: string; name: string }[] { - return this.gpuIds.map((id) => ({ id, name: this.gpuName.get(id) ?? "" })); - } - - isOffline(): boolean { - return this.lastError !== undefined; - } -} -``` - -- [ ] **Step 4: Run test to verify it passes** - -Run: `npm test` -Expected: PASS. - -- [ ] **Step 5: Commit** - -```bash -git add -A -git commit -m "feat: add metrics poller with ring buffers and GPU aggregates" -``` - ---- - -### Task 6: `sse.ts` + `inflight-tracker.ts` — SSE parsing and request tracking - -**Files:** -- Create: `src/lib/sse.ts` -- Create: `src/lib/inflight-tracker.ts` -- Test: `tests/sse.test.ts` -- Test: `tests/inflight-tracker.test.ts` - -**Interfaces:** -- Consumes: nothing. -- Produces: - - `sse.ts`: `interface SseMessage { event?: string; data: string }`, `parseSse(chunk: string): { messages: SseMessage[]; rest: string }`. - - `inflight-tracker.ts`: - - `interface InflightRequest { id?: string; model: string; req_path?: string; method?: string; timestamp?: string; elapsed_ms?: number }` - - `interface ModelState { id: string; state: string }` - - `type FeedEvent = { type: "inflight"; operation: "snapshot" | "add" | "remove"; requests: InflightRequest[] } | { type: "modelStatus"; models: ModelState[] }` - - `type ModelRuntimeState = "stopped" | "loading" | "ready"` - - `class InflightTracker { apply(event: FeedEvent): void; count(modelId: string): number; state(modelId: string): ModelRuntimeState | undefined }` - -- [ ] **Step 1: Write the failing tests** - -`tests/sse.test.ts`: - -```ts -import assert from "node:assert/strict"; -import { test } from "node:test"; -import { parseSse } from "../src/lib/sse"; - -test("parseSse extracts event + data blocks separated by blank lines", () => { - const { messages, rest } = parseSse('event:message\ndata:{"a":1}\n\nevent:message\ndata:hello\n\n'); - assert.equal(rest, ""); - assert.equal(messages.length, 2); - assert.equal(messages[0].event, "message"); - assert.equal(messages[0].data, '{"a":1}'); - assert.equal(messages[1].data, "hello"); -}); - -test("parseSse keeps a trailing partial event in rest", () => { - const { messages, rest } = parseSse("event:message\ndata:par"); - assert.equal(messages.length, 0); - assert.equal(rest, "event:message\ndata:par"); -}); - -test("parseSse joins multi-line data fields with newline", () => { - const { messages } = parseSse("data:line1\ndata:line2\n\n"); - assert.equal(messages.length, 1); - assert.equal(messages[0].data, "line1\nline2"); -}); -``` - -`tests/inflight-tracker.test.ts`: - -```ts -import assert from "node:assert/strict"; -import { test } from "node:test"; -import { InflightTracker, type FeedEvent } from "../src/lib/inflight-tracker"; - -test("snapshot rebuilds counts for all in-flight requests", () => { - const tracker = new InflightTracker(); - tracker.apply({ - type: "inflight", - operation: "snapshot", - requests: [ - { model: "A", id: "1" }, - { model: "A", id: "2" }, - { model: "B", id: "3" }, - ], - }); - assert.equal(tracker.count("A"), 2); - assert.equal(tracker.count("B"), 1); - assert.equal(tracker.count("C"), 0); -}); - -test("add and remove adjust per-model counts", () => { - const tracker = new InflightTracker(); - tracker.apply({ type: "inflight", operation: "add", requests: [{ model: "A", id: "1" }] }); - tracker.apply({ type: "inflight", operation: "add", requests: [{ model: "A", id: "2" }] }); - assert.equal(tracker.count("A"), 2); - tracker.apply({ type: "inflight", operation: "remove", requests: [{ model: "A", id: "1" }] }); - assert.equal(tracker.count("A"), 1); - tracker.apply({ type: "inflight", operation: "remove", requests: [{ model: "A", id: "2" }] }); - assert.equal(tracker.count("A"), 0); - tracker.apply({ type: "inflight", operation: "remove", requests: [{ model: "A", id: "9" }] }); - assert.equal(tracker.count("A"), 0); -}); - -test("modelStatus normalizes states", () => { - const tracker = new InflightTracker(); - tracker.apply({ - type: "modelStatus", - models: [ - { id: "A", state: "ready" }, - { id: "B", state: "loading" }, - { id: "C", state: "stopped" }, - { id: "D", state: "weird" }, - ], - }); - assert.equal(tracker.state("A"), "ready"); - assert.equal(tracker.state("B"), "loading"); - assert.equal(tracker.state("C"), "stopped"); - assert.equal(tracker.state("D"), "stopped"); - assert.equal(tracker.state("missing"), undefined); -}); -``` - -- [ ] **Step 2: Run tests to verify they fail** - -Run: `npm test` -Expected: FAIL — modules not found. - -- [ ] **Step 3: Implement `src/lib/sse.ts`** - -```ts -export interface SseMessage { - event?: string; - data: string; -} - -export function parseSse(chunk: string): { messages: SseMessage[]; rest: string } { - const messages: SseMessage[] = []; - let rest = chunk; - while (true) { - const idx = rest.indexOf("\n\n"); - if (idx === -1) break; - const block = rest.slice(0, idx); - rest = rest.slice(idx + 2); - const message = toMessage(block); - if (message && message.data.length > 0) messages.push(message); - } - return { messages, rest }; -} - -function toMessage(block: string): SseMessage | null { - let event: string | undefined; - const dataLines: string[] = []; - for (const line of block.split("\n")) { - if (line.startsWith("event:")) event = line.slice(6).trim(); - else if (line.startsWith("data:")) dataLines.push(line.slice(5).trimStart()); - } - if (dataLines.length === 0) return null; - return { event, data: dataLines.join("\n") }; -} -``` - -- [ ] **Step 4: Implement `src/lib/inflight-tracker.ts`** - -```ts -export interface InflightRequest { - id?: string; - model: string; - req_path?: string; - method?: string; - timestamp?: string; - elapsed_ms?: number; -} - -export interface ModelState { - id: string; - state: string; -} - -export type FeedEvent = - | { type: "inflight"; operation: "snapshot" | "add" | "remove"; requests: InflightRequest[] } - | { type: "modelStatus"; models: ModelState[] }; - -export type ModelRuntimeState = "stopped" | "loading" | "ready"; - -export class InflightTracker { - private counts = new Map(); - private states = new Map(); - - apply(event: FeedEvent): void { - if (event.type === "inflight") { - if (event.operation === "snapshot") { - const counts = new Map(); - for (const r of event.requests) counts.set(r.model, (counts.get(r.model) ?? 0) + 1); - this.counts = counts; - } else if (event.operation === "add") { - for (const r of event.requests) this.counts.set(r.model, (this.counts.get(r.model) ?? 0) + 1); - } else if (event.operation === "remove") { - for (const r of event.requests) { - const c = this.counts.get(r.model) ?? 0; - if (c <= 1) this.counts.delete(r.model); - else this.counts.set(r.model, c - 1); - } - } - } else if (event.type === "modelStatus") { - for (const m of event.models) this.states.set(m.id, normalizeState(m.state)); - } - } - - count(modelId: string): number { - return this.counts.get(modelId) ?? 0; - } - - state(modelId: string): ModelRuntimeState | undefined { - return this.states.get(modelId); - } -} - -function normalizeState(state: string): ModelRuntimeState { - if (state === "ready") return "ready"; - if (state === "loading") return "loading"; - return "stopped"; -} -``` - -- [ ] **Step 5: Run tests to verify they pass** - -Run: `npm test` -Expected: PASS. - -- [ ] **Step 6: Commit** - -```bash -git add -A -git commit -m "feat: add SSE parser and in-flight request tracker" -``` - ---- - -### Task 7: `event-feed.ts` — SSE client and event decoding - -**Files:** -- Create: `src/lib/event-feed.ts` -- Test: `tests/event-feed.test.ts` - -**Interfaces:** -- Consumes: `LlamaSwapConfig` from `./util`, `parseSse` + `SseMessage` from `./sse`, `FeedEvent`/`InflightRequest`/`ModelState` from `./inflight-tracker`. -- Produces: - - `decodeEvent(msg: SseMessage): FeedEvent | null` (exported for tests) - - `class EventFeed { constructor(cfg: LlamaSwapConfig, onEvent: (ev: FeedEvent) => void); setStatusHandler(fn: (connected: boolean) => void): void; start(): void; stop(): void }` - - Reconnect with exponential backoff 1s → 30s; self-heals from `snapshot`. - -- [ ] **Step 1: Write the failing test** — `tests/event-feed.test.ts` - -```ts -import assert from "node:assert/strict"; -import { test } from "node:test"; -import { decodeEvent } from "../src/lib/event-feed"; -import type { SseMessage } from "../src/lib/sse"; - -test("decodeEvent parses an inflight snapshot", () => { - const msg: SseMessage = { - event: "message", - data: JSON.stringify({ - type: "inflight", - data: JSON.stringify({ - operation: "snapshot", - requests: [{ id: "14", model: "Qwen3.8-27B-NVFP4", req_path: "/v1/chat/completions", elapsed_ms: 165510 }], - }), - }), - }; - const ev = decodeEvent(msg); - assert.ok(ev); - assert.equal(ev.type, "inflight"); - if (ev.type === "inflight") { - assert.equal(ev.operation, "snapshot"); - assert.equal(ev.requests.length, 1); - assert.equal(ev.requests[0].model, "Qwen3.8-27B-NVFP4"); - assert.equal(ev.requests[0].elapsed_ms, 165510); - } -}); - -test("decodeEvent parses an inflight add with a singular request field", () => { - const msg: SseMessage = { - event: "message", - data: JSON.stringify({ - type: "inflight", - data: JSON.stringify({ operation: "add", request: { id: "9", model: "A" } }), - }), - }; - const ev = decodeEvent(msg); - assert.ok(ev && ev.type === "inflight"); - if (ev && ev.type === "inflight") { - assert.equal(ev.operation, "add"); - assert.equal(ev.requests[0].model, "A"); - } -}); - -test("decodeEvent parses modelStatus", () => { - const msg: SseMessage = { - event: "message", - data: JSON.stringify({ - type: "modelStatus", - data: JSON.stringify([ - { id: "DeepSeek-V4-Flash-0731", state: "ready" }, - { id: "Qwen3.8-27B-NVFP4", state: "stopped" }, - ]), - }), - }; - const ev = decodeEvent(msg); - assert.ok(ev && ev.type === "modelStatus"); - if (ev && ev.type === "modelStatus") { - assert.equal(ev.models.length, 2); - assert.equal(ev.models[0].state, "ready"); - } -}); - -test("decodeEvent returns null for unrelated or malformed events", () => { - assert.equal(decodeEvent({ event: "message", data: JSON.stringify({ type: "logData", data: "{}" }) }), null); - assert.equal(decodeEvent({ event: "message", data: "not json" }), null); -}); -``` - -- [ ] **Step 2: Run test to verify it fails** - -Run: `npm test` -Expected: FAIL — module not found. - -- [ ] **Step 3: Implement `src/lib/event-feed.ts`** - -```ts -import { type LlamaSwapConfig } from "./util"; -import { parseSse, type SseMessage } from "./sse"; -import { type FeedEvent, type InflightRequest, type ModelState } from "./inflight-tracker"; - -export function decodeEvent(msg: SseMessage): FeedEvent | null { - try { - const outer = JSON.parse(msg.data) as { type?: string; data?: string }; - if (!outer.data) return null; - const inner = JSON.parse(outer.data) as Record; - - if (outer.type === "inflight") { - const requests: InflightRequest[] = - (inner.requests as InflightRequest[]) ?? (inner.request ? [inner.request as InflightRequest] : []); - return { - type: "inflight", - operation: inner.operation as "snapshot" | "add" | "remove", - requests, - }; - } - - if (outer.type === "modelStatus") { - return { type: "modelStatus", models: inner as unknown as ModelState[] }; - } - - return null; - } catch { - return null; - } -} - -const MAX_RETRY_MS = 30000; -const INITIAL_RETRY_MS = 1000; - -export class EventFeed { - private controller?: AbortController; - private closed = false; - private retryMs = INITIAL_RETRY_MS; - private statusHandler?: (connected: boolean) => void; - - constructor( - private cfg: LlamaSwapConfig, - private onEvent: (ev: FeedEvent) => void, - ) {} - - setStatusHandler(fn: (connected: boolean) => void): void { - this.statusHandler = fn; - } - - start(): void { - void this.connect(); - } - - stop(): void { - this.closed = true; - this.controller?.abort(); - } - - private async connect(): Promise { - while (!this.closed) { - this.controller = new AbortController(); - try { - const headers: Record = { Accept: "text/event-stream" }; - if (this.cfg.apiKey) headers["Authorization"] = `Bearer ${this.cfg.apiKey}`; - const res = await fetch(`${this.cfg.baseUrl}/api/events`, { - headers, - signal: this.controller.signal, - }); - if (!res.ok || !res.body) throw new Error(`events HTTP ${res.status}`); - this.statusHandler?.(true); - this.retryMs = INITIAL_RETRY_MS; - await this.readStream(res.body.getReader()); - } catch { - if (this.closed) return; - } - this.statusHandler?.(false); - if (this.closed) return; - await sleep(this.retryMs); - this.retryMs = Math.min(this.retryMs * 2, MAX_RETRY_MS); - } - } - - private async readStream(reader: ReadableStreamDefaultReader): Promise { - const decoder = new TextDecoder(); - let buffer = ""; - try { - while (!this.closed) { - const { done, value } = await reader.read(); - if (done) break; - buffer += decoder.decode(value, { stream: true }); - const { messages, rest } = parseSse(buffer); - buffer = rest; - for (const msg of messages) { - const ev = decodeEvent(msg); - if (ev) this.onEvent(ev); - } - } - } finally { - reader.releaseLock(); - } - } -} - -function sleep(ms: number): Promise { - return new Promise((resolve) => setTimeout(resolve, ms)); -} -``` - -- [ ] **Step 4: Run test to verify it passes** - -Run: `npm test` -Expected: PASS. - -- [ ] **Step 5: Commit** - -```bash -git add -A -git commit -m "feat: add SSE event feed for llama-swap api/events" -``` - ---- - -### Task 8: `render.ts` — SVG key rendering - -**Files:** -- Create: `src/lib/render.ts` -- Test: `tests/render.test.ts` - -**Interfaces:** -- Consumes: `GpuMetricKind` from `./metrics-parser`, `ModelRuntimeState` from `./inflight-tracker`, `shorten`/`escapeXml` from `./util`. -- Produces: - - `svgDataUrl(svg: string): string` - - `renderInflight(opts: { modelName: string; state?: ModelRuntimeState; count: number; offline: boolean; pulse: boolean }): string` - - `renderGpuGraph(opts: { gpuName: string; metric: GpuMetricKind; value: number | undefined; history: number[]; offline: boolean }): string` - -Rendering rules: -- Inflight: offline → grey `OFFLINE`; no model (`modelName === "unset"`) → dark `?` / `NO MODEL`; unknown state → grey `…`; `ready`+0 → green `IDLE`; `ready`+≥1 → red `ACTIVE` (opacity toggled by `pulse`); `loading` → amber `LOADING`; else → grey `OFF`. Model name at the bottom. -- GPU: dark chart area (x 3..69, y 20..55); % metrics scale 0..100, power scales to `niceCeil(max*1.1)`; `` line + gradient `` fill under curve; value label top (severity-colored), `METRIC · NAME` bottom; offline → grey `OFFLINE` frame. - -- [ ] **Step 1: Write the failing test** — `tests/render.test.ts` - -```ts -import assert from "node:assert/strict"; -import { test } from "node:test"; -import { renderGpuGraph, renderInflight, svgDataUrl } from "../src/lib/render"; - -test("svgDataUrl wraps an SVG as a base64 data URL", () => { - const url = svgDataUrl(""); - assert.match(url, /^data:image\/svg\+xml;base64,/); -}); - -test("renderInflight: ready + count renders IDLE in green", () => { - const svg = renderInflight({ modelName: "Qwen3.8-27B-NVFP4", state: "ready", count: 0, offline: false, pulse: false }); - assert.match(svg, /IDLE/); - assert.match(svg, /#1e6b34/); - assert.doesNotMatch(svg, /ACTIVE/); -}); - -test("renderInflight: ready + count>0 renders ACTIVE in red", () => { - const svg = renderInflight({ modelName: "Qwen3.8-27B-NVFP4", state: "ready", count: 2, offline: false, pulse: false }); - assert.match(svg, /ACTIVE/); - assert.match(svg, /#8b2626/); -}); - -test("renderInflight: pulse toggles opacity while active", () => { - const a = renderInflight({ modelName: "A", state: "ready", count: 1, offline: false, pulse: false }); - const b = renderInflight({ modelName: "A", state: "ready", count: 1, offline: false, pulse: true }); - assert.notEqual(a, b); - assert.match(b, /opacity="0.55"/); -}); - -test("renderInflight: offline and no-model states", () => { - const offline = renderInflight({ modelName: "A", state: "ready", count: 0, offline: true, pulse: false }); - assert.match(offline, /OFFLINE/); - const noModel = renderInflight({ modelName: "unset", state: undefined, count: 0, offline: false, pulse: false }); - assert.match(noModel, /NO MODEL/); -}); - -test("renderInflight: loading renders LOADING in amber", () => { - const svg = renderInflight({ modelName: "A", state: "loading", count: 0, offline: false, pulse: false }); - assert.match(svg, /LOADING/); - assert.match(svg, /#8a6d1d/); -}); - -test("renderGpuGraph: draws a polyline, value, and metric label", () => { - const svg = renderGpuGraph({ - gpuName: "GPU 2 · RTX 5090", - metric: "util_percent", - value: 67, - history: [0, 20, 40, 67], - offline: false, - }); - assert.match(svg, / { - const svg = renderGpuGraph({ - gpuName: "ALL GPUS", - metric: "power", - value: 1030, - history: [900, 1000, 1030], - offline: false, - }); - assert.match(svg, /1030W/); - assert.match(svg, /PWR/); - assert.match(svg, /ALL GPUS/); -}); - -test("renderGpuGraph: offline frame", () => { - const svg = renderGpuGraph({ gpuName: "ALL GPUS", metric: "util_percent", value: undefined, history: [], offline: true }); - assert.match(svg, /OFFLINE/); -}); - -test("renderGpuGraph: single-point history avoids divide-by-zero", () => { - const svg = renderGpuGraph({ gpuName: "ALL GPUS", metric: "util_percent", value: 50, history: [50], offline: false }); - assert.match(svg, / = { - util_percent: "UTIL", - memory_util_percent: "VRAM", - temperature: "TEMP", - power: "PWR", - fan: "FAN", -}; - -export interface InflightRenderOptions { - modelName: string; - state?: ModelRuntimeState; - count: number; - offline: boolean; - pulse: boolean; -} - -export function renderInflight(opts: InflightRenderOptions): string { - const name = escapeXml(shorten(opts.modelName)); - - if (opts.offline) { - return frame("#3a3a3a", [ - centerText("!!", 34, 20, "bold", "#e0e0e0"), - centerText("OFFLINE", 52, 9, "normal", "#bdbdbd"), - centerText(name, 64, 7, "normal", "#ffffff"), - ]); - } - - if (opts.modelName === "unset") { - return frame("#222222", [ - centerText("?", 36, 16, "bold", "#888888"), - centerText("NO MODEL", 54, 8, "normal", "#666666"), - ]); - } - - if (!opts.state) { - return frame("#222222", [centerText("…", 36, 16, "bold", "#999999"), centerText(name, 60, 7, "normal", "#999999")]); - } - - const isActive = opts.state === "ready" && opts.count > 0; - const bg = opts.state === "ready" ? (isActive ? "#8b2626" : "#1e6b34") : opts.state === "loading" ? "#8a6d1d" : "#3a3a3a"; - const label = opts.state === "ready" ? (isActive ? "ACTIVE" : "IDLE") : opts.state === "loading" ? "LOADING" : "OFF"; - const opacity = isActive && opts.pulse ? 0.55 : 1; - - return frame(bg, [ - centerText(label, 38, 18, "bold", "#ffffff", opacity), - centerText(name, 64, 7, "normal", "#ffffff"), - ]); -} - -export interface GpuGraphRenderOptions { - gpuName: string; - metric: GpuMetricKind; - value: number | undefined; - history: number[]; - offline: boolean; -} - -export function renderGpuGraph(opts: GpuGraphRenderOptions): string { - const label = escapeXml(shorten(opts.gpuName)); - - if (opts.offline) { - return ` - - !! - OFFLINE - ${label} -`; - } - - const color = opts.value === undefined ? "#666666" : severityColor(opts.metric, opts.value); - const valueText = opts.value === undefined ? "--" : `${Math.round(opts.value)}${unit(opts.metric)}`; - const points = chartPoints(opts.history, opts.metric); - - const line = - points.length > 1 - ? `` - : ""; - - const area = - points.length > 1 - ? `` - : ""; - - return ` - - - - - - - - - - ${area} - ${line} - ${valueText} - ${METRIC_LABEL[opts.metric]} · ${label} -`; -} - -function unit(metric: GpuMetricKind): string { - return metric === "temperature" ? "°C" : metric === "power" ? "W" : "%"; -} - -function severityColor(metric: GpuMetricKind, value: number): string { - if (metric === "temperature") return value >= 80 ? "#e0453a" : value >= 60 ? "#d9a02a" : "#3fae5a"; - if (metric === "power") return value >= 300 ? "#e0453a" : value >= 150 ? "#d9a02a" : "#3fae5a"; - return value >= 85 ? "#e0453a" : value >= 50 ? "#d9a02a" : "#3fae5a"; -} - -function chartPoints(history: number[], metric: GpuMetricKind): { x: number; y: number }[] { - const n = history.length; - if (n === 0) return []; - const left = 3; - const right = 69; - const top = 20; - const bottom = 55; - const yMax = yScaleMax(metric, history); - return history.map((v, i) => { - const x = n === 1 ? (left + right) / 2 : left + ((right - left) * i) / (n - 1); - const clamped = Math.max(0, Math.min(v, yMax)); - const y = bottom - ((clamped - 0) / (yMax - 0 || 1)) * (bottom - top); - return { x: round1(x), y: round1(y) }; - }); -} - -function yScaleMax(metric: GpuMetricKind, history: number[]): number { - if (metric !== "power") return 100; - const maxValue = Math.max(...history); - if (!isFinite(maxValue) || maxValue <= 0) return 100; - return niceCeil(maxValue * 1.1); -} - -function niceCeil(value: number): number { - const magnitude = Math.pow(10, Math.floor(Math.log10(value))); - for (const m of [1, 2, 2.5, 5, 10]) { - if (m * magnitude >= value) return m * magnitude; - } - return value; -} - -function round1(value: number): number { - return Math.round(value * 10) / 10; -} - -function centerText(text: string, y: number, size: number, weight: "normal" | "bold", fill: string, opacity?: number): string { - return `${text}`; -} - -function frame(bg: string, parts: string[]): string { - return ` - - ${parts.join("\n ")} -`; -} -``` - -- [ ] **Step 4: Run test to verify it passes** - -Run: `npm test` -Expected: PASS. - -- [ ] **Step 5: Commit** - -```bash -git add -A -git commit -m "feat: add SVG key renderers for inflight and GPU graph actions" -``` - ---- - -### Task 9: `runtime.ts` + `datasources.ts` — shared state and PI data sourcing - -**Files:** -- Create: `src/lib/runtime.ts` -- Create: `src/lib/datasources.ts` - -**Interfaces:** -- Consumes: `EventFeed`, `InflightTracker`, `MetricsPoller`, `LlamaSwapConfig`, `fetchModels`, `fetchMetrics`, `gpuInfos`, `parseGpuMetrics`, `cfgFromSettings`, `CfgSettings`. -- Produces: - - `runtime: { tracker: InflightTracker; offline: boolean; ensureConnections(cfg): void; pollerInstance?: MetricsPoller; subscribe(listener): () => void }` - - `registerDataSources(): void` — wires `streamDeck.ui.onSendToPlugin` to answer `models` and `gpus` datasource requests. - -- [ ] **Step 1: Implement `src/lib/runtime.ts`** - -```ts -import { EventFeed } from "./event-feed"; -import { InflightTracker } from "./inflight-tracker"; -import { MetricsPoller } from "./metrics-poller"; -import { type LlamaSwapConfig } from "./util"; - -class Runtime { - readonly tracker = new InflightTracker(); - offline = true; - private feed?: EventFeed; - private poller?: MetricsPoller; - private listeners = new Set<() => void>(); - - ensureConnections(cfg: LlamaSwapConfig): void { - if (!this.feed) { - this.feed = new EventFeed(cfg, (ev) => { - this.tracker.apply(ev); - this.emit(); - }); - this.feed.setStatusHandler((connected) => { - this.offline = !connected; - this.emit(); - }); - this.feed.start(); - } - if (!this.poller) { - this.poller = new MetricsPoller(cfg); - this.poller.on(() => this.emit()); - this.poller.start(); - } - } - - get pollerInstance(): MetricsPoller | undefined { - return this.poller; - } - - subscribe(listener: () => void): () => void { - this.listeners.add(listener); - return () => { - this.listeners.delete(listener); - }; - } - - private emit(): void { - for (const listener of this.listeners) listener(); - } -} - -export const runtime = new Runtime(); -``` - -- [ ] **Step 2: Implement `src/lib/datasources.ts`** - -```ts -import streamDeck from "@elgato/streamdeck"; -import { fetchMetrics, fetchModels } from "./llamaswap"; -import { gpuInfos, parseGpuMetrics } from "./metrics-parser"; -import { cfgFromSettings, type CfgSettings } from "./util"; - -type DataSourceItem = { label: string; value: string }; - -export function registerDataSources(): void { - streamDeck.ui.onSendToPlugin(async (ev) => { - const request = ev.payload as { event?: string } | undefined; - const event = request?.event; - if (!event) return; - const settings = await ev.action.getSettings(); - - if (event === "models") { - let items: DataSourceItem[] = []; - try { - const models = await fetchModels(cfgFromSettings(settings)); - items = models.map((m) => ({ label: `${m.id} (${m.status})`, value: m.id })); - } catch { - items = []; - } - await streamDeck.ui.sendToPropertyInspector({ event, items }); - return; - } - - if (event === "gpus") { - let items: DataSourceItem[] = []; - try { - const text = await fetchMetrics(cfgFromSettings(settings)); - const gpus = gpuInfos(parseGpuMetrics(text)); - items = [ - { label: "All GPUs", value: "all" }, - ...gpus.map((g) => ({ label: `GPU ${g.id} · ${g.name}`, value: g.id })), - ]; - } catch { - items = [{ label: "All GPUs", value: "all" }]; - } - await streamDeck.ui.sendToPropertyInspector({ event, items }); - } - }); -} -``` - -- [ ] **Step 3: Verify it typechecks** - -Run: `npm run build` -Expected: build succeeds. (Note: `import streamDeck` default export + `streamDeck.ui.onSendToPlugin` — if the compiler reports a different method name, consult `node_modules/@elgato/streamdeck/dist/plugin/ui.d.ts` and adjust; the public API is `onSendToPlugin` / `sendToPropertyInspector`. Also note: `DataSourceItem` must be a `type` alias, not an interface — interfaces without an index signature aren't assignable to `JsonObject`/`JsonValue`.) - -- [ ] **Step 4: Commit** - -```bash -git add -A -git commit -m "feat: add shared runtime and PI datasource handlers" -``` - ---- - -### Task 10: Actions — `InflightMonitor` and `GpuGraph` - -**Files:** -- Create: `src/actions/inflight-monitor.ts` -- Create: `src/actions/gpu-graph.ts` - -**Interfaces:** -- Consumes: `runtime`, `renderInflight`/`renderGpuGraph`/`svgDataUrl`, `cfgFromSettings`/`CfgSettings`, `GpuMetricKind`, `streamDeck` (for `system.openUrl`). -- Produces: two `SingletonAction` classes registered by UUIDs `com.bryce.llamawatch.inflight` and `com.bryce.llamawatch.gpu`. - -- [ ] **Step 1: Implement `src/actions/inflight-monitor.ts`** - -```ts -import streamDeck, { - action, - type DidReceiveSettingsEvent, - type KeyAction, - type KeyDownEvent, - SingletonAction, - type WillAppearEvent, -} from "@elgato/streamdeck"; -import { renderInflight, svgDataUrl } from "../lib/render"; -import { runtime } from "../lib/runtime"; -import { cfgFromSettings, type CfgSettings } from "../lib/util"; - -type InflightSettings = CfgSettings & { - modelId?: string; -}; - -@action({ UUID: "com.bryce.llamawatch.inflight" }) -export class InflightMonitor extends SingletonAction { - private settings: InflightSettings = {}; - private action?: KeyAction; - private unsubscribe?: () => void; - private pulseTimer?: ReturnType; - private pulse = false; - - override onWillAppear(ev: WillAppearEvent): void { - if (!ev.action.isKey()) return; - this.settings = ev.payload.settings; - this.action = ev.action; - runtime.ensureConnections(cfgFromSettings(this.settings)); - this.unsubscribe = runtime.subscribe(() => this.render()); - this.render(); - } - - override onWillDisappear(): void { - this.unsubscribe?.(); - this.unsubscribe = undefined; - this.action = undefined; - this.clearPulse(); - } - - override onDidReceiveSettings(ev: DidReceiveSettingsEvent): void { - if (!ev.action.isKey()) return; - this.settings = ev.payload.settings; - this.action = ev.action; - runtime.ensureConnections(cfgFromSettings(this.settings)); - this.render(); - } - - override onKeyDown(ev: KeyDownEvent): void { - void streamDeck.system.openUrl(`${cfgFromSettings(ev.payload.settings).baseUrl}/ui`); - } - - private render(): void { - const action = this.action; - if (!action) return; - const modelId = this.settings.modelId ?? ""; - const state = runtime.tracker.state(modelId); - const count = runtime.tracker.count(modelId); - const offline = runtime.offline && modelId.length > 0; - void action.setImage( - svgDataUrl( - renderInflight({ - modelName: modelId.length > 0 ? modelId : "unset", - state, - count, - offline, - pulse: this.pulse, - }), - ), - ); - - if (state === "ready" && count > 0 && !this.pulseTimer) { - this.pulseTimer = setInterval(() => { - this.pulse = !this.pulse; - this.render(); - }, 500); - } else if (!(state === "ready" && count > 0) && this.pulseTimer) { - this.clearPulse(); - } - } - - private clearPulse(): void { - if (this.pulseTimer) clearInterval(this.pulseTimer); - this.pulseTimer = undefined; - this.pulse = false; - } -} -``` - -- [ ] **Step 2: Implement `src/actions/gpu-graph.ts`** - -```ts -import streamDeck, { - action, - type DidReceiveSettingsEvent, - type KeyAction, - type KeyDownEvent, - SingletonAction, - type WillAppearEvent, -} from "@elgato/streamdeck"; -import { type GpuMetricKind } from "../lib/metrics-parser"; -import { renderGpuGraph, svgDataUrl } from "../lib/render"; -import { runtime } from "../lib/runtime"; -import { cfgFromSettings, type CfgSettings } from "../lib/util"; - -type GpuSettings = CfgSettings & { - gpuId?: string; - metric?: GpuMetricKind; -}; - -@action({ UUID: "com.bryce.llamawatch.gpu" }) -export class GpuGraph extends SingletonAction { - private settings: GpuSettings = {}; - private action?: KeyAction; - private unsubscribe?: () => void; - private lastSvg?: string; - - override onWillAppear(ev: WillAppearEvent): void { - if (!ev.action.isKey()) return; - this.settings = ev.payload.settings; - this.action = ev.action; - runtime.ensureConnections(cfgFromSettings(this.settings)); - this.unsubscribe = runtime.subscribe(() => this.render()); - this.render(); - } - - override onWillDisappear(): void { - this.unsubscribe?.(); - this.unsubscribe = undefined; - this.action = undefined; - } - - override onDidReceiveSettings(ev: DidReceiveSettingsEvent): void { - if (!ev.action.isKey()) return; - this.settings = ev.payload.settings; - this.action = ev.action; - this.render(); - } - - override onKeyDown(ev: KeyDownEvent): void { - void streamDeck.system.openUrl(`${cfgFromSettings(ev.payload.settings).baseUrl}/ui`); - } - - private render(): void { - const action = this.action; - if (!action) return; - const poller = runtime.pollerInstance; - const gpuId = this.settings.gpuId ?? "all"; - const metric = this.settings.metric ?? "util_percent"; - const value = poller?.getValue(gpuId, metric); - const history = poller?.getHistory(gpuId, metric) ?? []; - const offline = !poller || poller.isOffline(); - const gpuName = gpuId === "all" ? "ALL GPUS" : this.gpuLabel(gpuId); - const svg = renderGpuGraph({ gpuName, metric, value, history, offline }); - if (svg === this.lastSvg) return; - this.lastSvg = svg; - void action.setImage(svgDataUrl(svg)); - } - - private gpuLabel(gpuId: string): string { - const gpu = runtime.pollerInstance?.gpus().find((g) => g.id === gpuId); - return gpu && gpu.name ? `GPU ${gpuId} · ${gpu.name}` : `GPU ${gpuId}`; - } -} -``` - -- [ ] **Step 3: Verify it typechecks** - -> **Post-implementation note (approved final-review fix):** the final shipped actions key all per-key state on `ev.action.id` in a `Map` (per `stateFor(action)`), because a `SingletonAction` instance is shared by every key of that UUID — the instance-field version above clobbered state across multiple keys. See commit `ccbf74c`. The runtime's `ensureConnections` also rebuilds the shared feed/poller when a key's base URL or API key changes. The logic described here (states, pulse lifecycle, render-on-change, press-to-open) is unchanged; only state ownership moved from instance fields to per-key Map entries. - -Run: `npm run build` -Expected: build succeeds. - -- [ ] **Step 4: Commit** - -```bash -git add -A -git commit -m "feat: implement inflight monitor and GPU graph actions" -``` - ---- - -### Task 11: Entry point, manifest actions, property inspectors, assets - -**Files:** -- Modify: `src/plugin.ts` -- Modify: `com.bryce.llamawatch.sdPlugin/manifest.json` -- Create: `com.bryce.llamawatch.sdPlugin/ui/inflight.html` -- Create: `com.bryce.llamawatch.sdPlugin/ui/gpu.html` -- Create: `com.bryce.llamawatch.sdPlugin/ui/sdpi-components.js` (downloaded, v4) -- Create: `com.bryce.llamawatch.sdPlugin/imgs/plugin/icon.svg` (256×256 source) -- Create: `com.bryce.llamawatch.sdPlugin/imgs/plugin/category-icon.svg` -- Create: `com.bryce.llamawatch.sdPlugin/imgs/actions/inflight/icon.svg` (monochrome) -- Create: `com.bryce.llamawatch.sdPlugin/imgs/actions/inflight/key.svg` -- Create: `com.bryce.llamawatch.sdPlugin/imgs/actions/gpu/icon.svg` (monochrome) -- Create: `com.bryce.llamawatch.sdPlugin/imgs/actions/gpu/key.svg` - -**Interfaces:** -- Consumes: action classes and `registerDataSources`. -- Produces: a fully manifest-defined plugin (both actions), compiled entry, PI pages wired to the `models`/`gpus` datasources, and generated PNG icons. - -- [ ] **Step 1: Replace `src/plugin.ts`** - -```ts -import streamDeck from "@elgato/streamdeck"; -import { GpuGraph } from "./actions/gpu-graph"; -import { InflightMonitor } from "./actions/inflight-monitor"; -import { registerDataSources } from "./lib/datasources"; - -streamDeck.actions.registerAction(new InflightMonitor()); -streamDeck.actions.registerAction(new GpuGraph()); -registerDataSources(); - -streamDeck.connect(); -``` - -- [ ] **Step 2: Replace `com.bryce.llamawatch.sdPlugin/manifest.json`** - -```json -{ - "$schema": "https://schemas.elgato.com/streamdeck/plugins/manifest.json", - "Name": "llama-watch", - "Version": "1.0.0.0", - "Author": "Bryce Zuccaro", - "Actions": [ - { - "Name": "In-Flight Monitor", - "UUID": "com.bryce.llamawatch.inflight", - "Icon": "imgs/actions/inflight/icon", - "Tooltip": "Shows whether a model has a request in flight", - "PropertyInspectorPath": "ui/inflight.html", - "Controllers": ["Keypad"], - "States": [{ "Image": "imgs/actions/inflight/key", "ShowTitle": false }] - }, - { - "Name": "GPU Graph", - "UUID": "com.bryce.llamawatch.gpu", - "Icon": "imgs/actions/gpu/icon", - "Tooltip": "Live GPU metric graph", - "PropertyInspectorPath": "ui/gpu.html", - "Controllers": ["Keypad"], - "States": [{ "Image": "imgs/actions/gpu/key", "ShowTitle": false }] - } - ], - "Category": "llama-watch", - "CategoryIcon": "imgs/plugin/category-icon", - "CodePath": "bin/plugin.js", - "Description": "Monitor llama-swap: in-flight requests per model and live GPU metric graphs.", - "Icon": "imgs/plugin/icon", - "SDKVersion": 2, - "Software": { "MinimumVersion": "7.1" }, - "OS": [{ "Platform": "mac", "MinimumVersion": "13" }], - "Nodejs": { "Version": "24" }, - "UUID": "com.bryce.llamawatch" -} -``` - -- [ ] **Step 3: Download `sdpi-components.js` (v4) locally** - -```bash -curl -sL https://sdpi-components.dev/releases/v4/sdpi-components.js -o com.bryce.llamawatch.sdPlugin/ui/sdpi-components.js -``` -Expected: file is non-empty and starts with a JS comment/IIFE. - -- [ ] **Step 4: Create `com.bryce.llamawatch.sdPlugin/ui/inflight.html`** - -```html - - - - - - - - - - - - - - - - - - -``` - -- [ ] **Step 5: Create `com.bryce.llamawatch.sdPlugin/ui/gpu.html`** - -```html - - - - - - - - - - - - - - - - - - - - - - - - - - - -``` - -- [ ] **Step 6: Create the SVG assets** - -`com.bryce.llamawatch.sdPlugin/imgs/plugin/icon.svg` (256×256, graph glyph on dark rounded square): - -```svg - - - - - -``` - -`com.bryce.llamawatch.sdPlugin/imgs/plugin/category-icon.svg` (monochrome white, 56×56): - -```svg - - - -``` - -`com.bryce.llamawatch.sdPlugin/imgs/actions/inflight/icon.svg` (monochrome white, activity pulse): - -```svg - - - -``` - -`com.bryce.llamawatch.sdPlugin/imgs/actions/inflight/key.svg` (72×72 default key): - -```svg - - - - -``` - -`com.bryce.llamawatch.sdPlugin/imgs/actions/gpu/icon.svg` (monochrome white, chip): - -```svg - - - - - -``` - -`com.bryce.llamawatch.sdPlugin/imgs/actions/gpu/key.svg` (72×72 default key): - -```svg - - - - - - -``` - -- [ ] **Step 7: Generate the plugin PNG icon (256 + 512 @2x) with `sips`** - -```bash -cd com.bryce.llamawatch.sdPlugin/imgs/plugin -sips -s format png icon.svg --out icon.png -sips -z 512 512 icon.png --out icon@2x.png -``` -Expected: `icon.png` is 256×256, `icon@2x.png` is 512×512. - -- [ ] **Step 8: Build and verify** - -Run: `npm run build` -Expected: build succeeds and `com.bryce.llamawatch.sdPlugin/bin/plugin.js` exists. - -- [ ] **Step 9: Commit** - -```bash -git add -A -git commit -m "feat: wire entry point, manifest, property inspectors, and assets" -``` - ---- - -### Task 12: Validate, install, and manually verify on the device - -**Files:** -- Create: `README.md` - -**Interfaces:** -- Consumes: the built plugin folder. -- Produces: `.streamDeckPlugin` package, plugin installed and verified live on the Stream Deck XL, README. - -- [ ] **Step 1: Validate the manifest** - -Run: -```bash -streamdeck validate com.bryce.llamawatch.sdPlugin -``` -Expected: validation passes (fix any reported issues, e.g. missing icon paths). - -- [ ] **Step 2: Enable developer mode (if needed) and install the plugin** - -Try the CLI link first: -```bash -streamdeck dev -streamdeck link com.bryce.llamawatch.sdPlugin -``` -If `link` fails because the CLI cannot locate the app, copy the folder manually: -```bash -cp -R com.bryce.llamawatch.sdPlugin "$HOME/Library/Application Support/com.elgato.StreamDeck/Plugins/" -``` -(The Stream Deck app at `/Applications/Elgato Stream Deck.app` auto-loads plugins from that directory.) Then verify with `streamdeck list` or by restarting the app. - -- [ ] **Step 3: Verify the plugin loads** - -- Restart the Stream Deck app (or `streamdeck restart com.bryce.llamawatch`). -- Confirm the action list contains "In-Flight Monitor" and "GPU Graph" under the "llama-watch" category. -- Check `~/Library/Logs/StreamDeck` (or the plugin `logs/` dir) for errors if the actions don't appear. - -- [ ] **Step 4: Manual verification on the device (Stream Deck XL)** - -- Place an **In-Flight Monitor** action and pick model `DeepSeek-V4-Flash-0731`; confirm it renders green `IDLE`. While an OpenWebUI/chat request is running against that model, confirm it turns red `ACTIVE` and pulses. Press the key → browser opens `http://localhost:9292/ui`. -- Place a **GPU Graph** action; GPU dropdown should list "All GPUs" plus the three GPUs (RTX PRO 6000 ×2, RTX 5090). Select GPU `2` + metric `Utilization %`; confirm a graph line appears and updates every ~5s. Select metric `Power draw`; confirm watts value + auto-scaled chart. Press → opens `/ui`. -- Stop the llama-swap instance (or use a bogus base URL on a spare key) and confirm buttons show `OFFLINE`; restore and confirm auto-recovery. - -- [ ] **Step 5: Write `README.md`** - -```markdown -# llama-watch - -A Stream Deck plugin (macOS) that monitors a llama-swap instance. - -## Actions - -- **In-Flight Monitor** — per-model color state: `OFF` / `LOADING` / `IDLE` / `ACTIVE` (pulsing red). Powered by the real-time `/api/events` SSE feed. -- **GPU Graph** — live line chart of a GPU metric (utilization %, VRAM %, temperature, power draw, fan speed) for one GPU or all GPUs, sampled every 5 s. - -Pressing either key opens `http:///ui` in your browser. - -## Install - -Double-click the packaged `.streamDeckPlugin`, or run: - -```bash -npm install -npm run build -streamdeck validate com.bryce.llamawatch.sdPlugin -``` - -## Develop - -```bash -npm run watch # hot-reload while the Stream Deck app is running -npm test # unit tests (node:test + tsx) -``` - -## Configure - -Per-key settings: base URL (default `http://localhost:9292`), optional API key, and the model / GPU / metric to watch. The model and GPU dropdowns are populated live from the instance. - -## Marketplace - -Assets are prepared for eventual submission (`imgs/plugin/icon.png` 256/512, action icons, category icon). The plugin only stores its settings locally; it reads GPU metrics and request status from the user's own llama-swap server and sends nothing elsewhere. -``` - -- [ ] **Step 6: Package the distributable** - -Run: -```bash -streamdeck pack com.bryce.llamawatch.sdPlugin -``` -Expected: `com.bryce.llamawatch.streamDeckPlugin` created. Double-clicking it installs the plugin. - -- [ ] **Step 7: Commit** - -```bash -git add -A -git commit -m "docs: add README; verify plugin install and rendering on device" -``` - ---- - -## Self-Review - -**Spec coverage:** -- Feature 1 (in-flight per model) → Tasks 6, 7 (tracker + feed), Task 10 (action), Task 11 (PI model dropdown). ✔ -- Feature 2 (GPU graphs, all five metrics + aggregates) → Tasks 3, 5 (parser/poller/aggregates), Task 8 (renderer), Task 10 (action), Task 11 (PI GPU+metric dropdowns). ✔ -- 5s poll, 60-sample buffer → Task 5. ✔ -- 144×72 rendering / SVG, DPR 2 → SVG viewBox 72 + `setImage`; device is Stream Deck XL (DPR 2), SVG scales crisply. ✔ -- IDLE/ACTIVE color state, pulse → Task 8/10. ✔ -- Press opens `/ui` → Task 10 (`streamDeck.system.openUrl`). ✔ -- Error handling (offline, reconnect backoff, missed-poll skip, per-button isolation) → Task 5/7/9/10. ✔ -- Marketplace-ready assets → Task 11/12. ✔ - -**Placeholder scan:** No TBD/TODO; all code blocks complete; no "similar to Task N" references; every interface symbol is defined in a task. - -**Type consistency:** `GpuMetricKind`, `FeedEvent`, `ModelRuntimeState`, `LlamaSwapConfig`, `cfgFromSettings`, `runtime`, `renderInflight`, `renderGpuGraph` names are identical across tasks. `KeyAction` used consistently for stored action refs; `streamDeck.system.openUrl` and `streamDeck.ui.onSendToPlugin`/`sendToPropertyInspector` match `@elgato/streamdeck@2.1.1` (verified against its `.d.ts`). No lint/typecheck config beyond the sample's tsconfig — `npm run build` is the type gate. diff --git a/docs/superpowers/plans/2026-08-14-usage-stats-gpu-combos.md b/docs/superpowers/plans/2026-08-14-usage-stats-gpu-combos.md deleted file mode 100644 index 5f5f754..0000000 --- a/docs/superpowers/plans/2026-08-14-usage-stats-gpu-combos.md +++ /dev/null @@ -1,1129 +0,0 @@ -# Usage Stats + GPU Combinations Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** Add a Usage-stats display mode to the In-Flight Monitor action (request/token totals + gen-speed P95 from `/api/metrics/stats`) and GPU combination support to the GPU Graph action. - -**Architecture:** A shared runtime-owned `StatsCache` polls llama-swap's `/api/metrics/stats` for registered model keys (single interval, `activity` SSE events trigger throttled refreshes) and feeds a new `renderUsage` view in the In-Flight action. GPU combinations are aggregated in the GPU action via a pure `combineSeries` helper over the poller's existing per-GPU rings. - -**Tech Stack:** TypeScript ESM, `node:test`+`tsx`, `@elgato/streamdeck@2.1.1`. Tests run with `npm test` (i.e. `tsx --test`). Build: `npm run build` (rollup). Typecheck: `npx tsc --noEmit`. - -## Global Constraints - -- No code comments (project convention). -- Work committed on `main` (user-approved workflow; no remote). -- `feed.ts`'s `FeedEvent` type lives in `src/lib/inflight-tracker.ts`. -- Default base URL `http://localhost:9292`; llama-swap `/api/metrics/stats` returns `{ total_requests, total_input_tokens, total_output_tokens, total_cache_tokens, prompt_histogram, gen_histogram }`; `gen_histogram.p95` is a **tokens/sec** percentile (not latency). -- Key images are SVG strings via `svgDataUrl`; dark background `#10131a`. -- Multi-key safety: per-key state lives in a `Map` keyed on `ev.action.id`. -- `npm test` runs ALL test files via `tsx --test`; `AggregateError` style failures indicate a failing test file. - ---- - -### Task 1: Stats types, parsing, and fetching - -**Files:** -- Create: `src/lib/stats.ts` -- Test: `tests/stats.test.ts` - -**Interfaces:** -- Produces: - - `interface UsageStats { totalRequests: number; totalInputTokens: number; totalOutputTokens: number; genP95: number }` - - `parseStats(json: unknown): UsageStats | null` - - `fetchStats(cfg: LlamaSwapConfig, modelId: string): Promise` - -- [ ] **Step 1: Write the failing test** - -Create `tests/stats.test.ts`: - -```ts -import assert from "node:assert/strict"; -import { test } from "node:test"; -import { fetchStats, parseStats, type UsageStats } from "../src/lib/stats"; - -test("parseStats extracts totals and gen p95", () => { - const stats = parseStats({ - total_requests: 1985, - total_input_tokens: 312327705, - total_output_tokens: 932710, - total_cache_tokens: 308144640, - gen_histogram: { p50: 341, p95: 378.86, p99: 392 }, - }); - assert.deepEqual(stats, { totalRequests: 1985, totalInputTokens: 312327705, totalOutputTokens: 932710, genP95: 378.86 }); -}); - -test("parseStats defaults missing fields and rejects malformed input", () => { - assert.deepEqual(parseStats({}), { totalRequests: 0, totalInputTokens: 0, totalOutputTokens: 0, genP95: 0 }); - assert.equal(parseStats({ total_requests: "nope" })!.totalRequests, 0); - assert.equal(parseStats(null), null); - assert.equal(parseStats("x"), null); -}); - -test("fetchStats calls /api/metrics/stats with a model query", async () => { - const calls: string[] = []; - const orig = globalThis.fetch; - globalThis.fetch = (async (url: RequestInfo | URL) => { - calls.push(String(url)); - return { ok: true, json: async () => ({ total_requests: 7 }) } as unknown as Response; - }) as typeof fetch; - try { - await fetchStats({ baseUrl: "http://x" }, "DeepSeek-V4-Flash-0731"); - assert.equal(calls[0], "http://x/api/metrics/stats?model=DeepSeek-V4-Flash-0731"); - } finally { - globalThis.fetch = orig; - } -}); - -test("fetchStats omits the model query for 'all' and returns null on HTTP error", async () => { - const calls: string[] = []; - const orig = globalThis.fetch; - globalThis.fetch = (async (url: RequestInfo | URL) => { - calls.push(String(url)); - return { ok: false, status: 500 } as unknown as Response; - }) as typeof fetch; - try { - const stats = await fetchStats({ baseUrl: "http://x" }, "all"); - assert.equal(stats, null); - assert.equal(calls[0], "http://x/api/metrics/stats"); - } finally { - globalThis.fetch = orig; - } -}); -``` - -- [ ] **Step 2: Run test to verify it fails** - -Run: `npx tsx --test tests/stats.test.ts` -Expected: FAIL — `Cannot find module '../src/lib/stats'`. - -- [ ] **Step 3: Write minimal implementation** - -Create `src/lib/stats.ts`: - -```ts -import { type LlamaSwapConfig } from "./util"; - -export interface UsageStats { - totalRequests: number; - totalInputTokens: number; - totalOutputTokens: number; - genP95: number; -} - -export function parseStats(json: unknown): UsageStats | null { - if (!json || typeof json !== "object") return null; - const obj = json as Record; - const genHist = (obj.gen_histogram ?? {}) as Record; - const num = (v: unknown): number => (typeof v === "number" && Number.isFinite(v) ? v : 0); - return { - totalRequests: num(obj.total_requests), - totalInputTokens: num(obj.total_input_tokens), - totalOutputTokens: num(obj.total_output_tokens), - genP95: num(genHist.p95), - }; -} - -export async function fetchStats(cfg: LlamaSwapConfig, modelId: string): Promise { - const query = modelId === "all" ? "" : `?model=${encodeURIComponent(modelId)}`; - const headers: Record = {}; - if (cfg.apiKey) headers["Authorization"] = `Bearer ${cfg.apiKey}`; - const res = await fetch(`${cfg.baseUrl}/api/metrics/stats${query}`, { - headers, - signal: AbortSignal.timeout(5000), - }); - if (!res.ok) throw new Error(`stats HTTP ${res.status}`); - return parseStats(await res.json()); -} -``` - -- [ ] **Step 4: Run test to verify it passes** - -Run: `npx tsx --test tests/stats.test.ts` -Expected: PASS (4 tests). - -- [ ] **Step 5: Commit** - -```bash -git add src/lib/stats.ts tests/stats.test.ts -git commit -m "feat: add usage stats parsing and fetching from /api/metrics/stats" -``` - ---- - -### Task 2: StatsCache - -**Files:** -- Create: `src/lib/stats-cache.ts` -- Test: `tests/stats-cache.test.ts` - -**Interfaces:** -- Consumes: `fetchStats`, `UsageStats` from `./stats`; `LlamaSwapConfig` from `./util`. -- Produces: - - `class StatsCache` - - `constructor(cfg: LlamaSwapConfig, fetchFn?: FetchFn, pollMs?: number, throttleMs?: number)` where `FetchFn = (cfg: LlamaSwapConfig, modelId: string) => Promise` - - `register(key: string): void` - - `unregister(key: string): void` - - `get(key: string): UsageStats | undefined` - - `scheduleRefresh(): void` — throttled (trailing, min `throttleMs` between polls) - - `refresh(): Promise` - - `setConfig(cfg: LlamaSwapConfig): void` - - `onChange(listener: () => void): () => void` - -- [ ] **Step 1: Write the failing test** - -Create `tests/stats-cache.test.ts`: - -```ts -import assert from "node:assert/strict"; -import { test } from "node:test"; -import { StatsCache, type FetchFn } from "../src/lib/stats-cache"; -import { type LlamaSwapConfig } from "../src/lib/util"; - -const cfg: LlamaSwapConfig = { baseUrl: "http://x" }; - -function stubFetch(result: unknown, counter: { count: number }): FetchFn { - return async () => { - counter.count++; - return result as never; - }; -} - -async function flush(): Promise { - await new Promise((r) => setTimeout(r, 0)); -} - -test("register polls once immediately and caches the value", async () => { - const calls = { count: 0 }; - const cache = new StatsCache(cfg, stubFetch({ totalRequests: 3, totalInputTokens: 1, totalOutputTokens: 2, genP95: 4 }, calls), 1000, 30); - cache.register("all"); - await flush(); - assert.equal(calls.count, 1); - assert.equal(cache.get("all")!.totalRequests, 3); -}); - -test("multiple keys are each polled", async () => { - const keys: string[] = []; - const cache = new StatsCache( - cfg, - async (_c, key) => { - keys.push(key); - return { totalRequests: 1, totalInputTokens: 1, totalOutputTokens: 1, genP95: 1 }; - }, - 1000, - 30, - ); - cache.register("a"); - cache.register("b"); - await flush(); - assert.deepEqual(keys.sort(), ["a", "b"]); -}); - -test("unregister stops the interval and clears the value", async () => { - const calls = { count: 0 }; - const cache = new StatsCache(cfg, stubFetch({ totalRequests: 1, totalInputTokens: 1, totalOutputTokens: 1, genP95: 1 }, calls), 10, 30); - cache.register("a"); - await flush(); - assert.equal(calls.count, 1); - cache.unregister("a"); - await new Promise((r) => setTimeout(r, 40)); - assert.equal(cache.get("a"), undefined); - assert.equal(calls.count, 1); -}); - -test("scheduleRefresh throttles to one poll per window", async () => { - const calls = { count: 0 }; - const cache = new StatsCache(cfg, stubFetch({ totalRequests: 1, totalInputTokens: 1, totalOutputTokens: 1, genP95: 1 }, calls), 10000, 30); - cache.register("a"); - await flush(); - const before = calls.count; - cache.scheduleRefresh(); - cache.scheduleRefresh(); - cache.scheduleRefresh(); - await flush(); - assert.equal(calls.count, before); - await new Promise((r) => setTimeout(r, 60)); - assert.equal(calls.count, before + 1); - cache.unregister("a"); -}); - -test("fetch failure keeps the last-known value", async () => { - const fail: FetchFn = async () => { - throw new Error("boom"); - }; - const cache = new StatsCache(cfg, fail, 10000, 30); - cache.register("a"); - await flush(); - assert.equal(cache.get("a"), undefined); - cache.unregister("a"); -}); -``` - -- [ ] **Step 2: Run test to verify it fails** - -Run: `npx tsx --test tests/stats-cache.test.ts` -Expected: FAIL — `Cannot find module '../src/lib/stats-cache'`. - -- [ ] **Step 3: Write minimal implementation** - -Create `src/lib/stats-cache.ts`: - -```ts -import { fetchStats, type UsageStats } from "./stats"; -import { type LlamaSwapConfig } from "./util"; - -const POLL_MS = 5000; -const ACTIVITY_THROTTLE_MS = 2000; - -export type FetchFn = (cfg: LlamaSwapConfig, modelId: string) => Promise; - -export class StatsCache { - private keys = new Set(); - private values = new Map(); - private listeners = new Set<() => void>(); - private timer?: ReturnType; - private throttleTimer?: ReturnType; - private lastRefresh = 0; - - constructor( - private cfg: LlamaSwapConfig, - private fetchFn: FetchFn = fetchStats, - private pollMs = POLL_MS, - private throttleMs = ACTIVITY_THROTTLE_MS, - ) {} - - setConfig(cfg: LlamaSwapConfig): void { - this.cfg = cfg; - this.values.clear(); - } - - register(key: string): void { - if (this.keys.has(key)) return; - this.keys.add(key); - if (!this.timer) { - void this.refresh(); - this.timer = setInterval(() => void this.refresh(), this.pollMs); - } - } - - unregister(key: string): void { - this.keys.delete(key); - this.values.delete(key); - if (this.keys.size === 0 && this.timer) { - clearInterval(this.timer); - this.timer = undefined; - } - } - - get(key: string): UsageStats | undefined { - return this.values.get(key); - } - - scheduleRefresh(): void { - const now = Date.now(); - const wait = Math.max(0, this.throttleMs - (now - this.lastRefresh)); - if (this.throttleTimer) clearTimeout(this.throttleTimer); - this.throttleTimer = setTimeout(() => { - this.throttleTimer = undefined; - void this.refresh(); - }, wait); - } - - async refresh(): Promise { - this.lastRefresh = Date.now(); - for (const key of this.keys) { - try { - const stats = await this.fetchFn(this.cfg, key); - if (stats) this.values.set(key, stats); - } catch { - } - } - this.emit(); - } - - onChange(listener: () => void): () => void { - this.listeners.add(listener); - return () => { - this.listeners.delete(listener); - }; - } - - private emit(): void { - for (const listener of this.listeners) listener(); - } -} -``` - -- [ ] **Step 4: Run test to verify it passes** - -Run: `npx tsx --test tests/stats-cache.test.ts` -Expected: PASS (5 tests). - -- [ ] **Step 5: Commit** - -```bash -git add src/lib/stats-cache.ts tests/stats-cache.test.ts -git commit -m "feat: add StatsCache for shared per-model usage polling" -``` - ---- - -### Task 3: Decode `activity` SSE events + tracker total - -**Files:** -- Modify: `src/lib/inflight-tracker.ts`, `src/lib/event-feed.ts` -- Test: `tests/event-feed.test.ts`, `tests/inflight-tracker.test.ts` - -**Interfaces:** -- Consumes: existing `FeedEvent` in `inflight-tracker.ts`. -- Produces: - - `FeedEvent` gains `| { type: "activity"; id: number }` - - `InflightTracker.total(): number` - - `decodeEvent` handles `{"type":"activity","data":"{\"id\":817}"}`. - -- [ ] **Step 1: Write the failing tests** - -Append to `tests/event-feed.test.ts`: - -```ts -test("decodeEvent parses an activity event with an id", () => { - const msg: SseMessage = { - event: "message", - data: JSON.stringify({ type: "activity", data: JSON.stringify({ id: 817 }) }), - }; - const ev = decodeEvent(msg); - assert.deepEqual(ev, { type: "activity", id: 817 }); -}); -``` - -Append to `tests/inflight-tracker.test.ts`: - -```ts -test("total sums in-flight requests across all models", () => { - const tracker = new InflightTracker(); - assert.equal(tracker.total(), 0); - tracker.apply({ type: "inflight", operation: "add", requests: [{ model: "A", id: "1" }, { model: "B", id: "2" }] }); - tracker.apply({ type: "inflight", operation: "add", requests: [{ model: "A", id: "3" }] }); - assert.equal(tracker.total(), 3); - tracker.apply({ type: "inflight", operation: "remove", id: "1" }); - assert.equal(tracker.total(), 2); -}); -``` - -- [ ] **Step 2: Run tests to verify they fail** - -Run: `npx tsx --test tests/event-feed.test.ts tests/inflight-tracker.test.ts` -Expected: FAIL — activity event decodes to `null`; `tracker.total is not a function`. - -- [ ] **Step 3: Write minimal implementation** - -In `src/lib/inflight-tracker.ts`, change the `FeedEvent` union to: - -```ts -export type FeedEvent = - | { type: "inflight"; operation: "snapshot" | "add"; requests: InflightRequest[] } - | { type: "inflight"; operation: "remove"; id: string } - | { type: "activity"; id: number } - | { type: "modelStatus"; models: ModelState[] }; -``` - -and add a method to `InflightTracker` (after `count`): - -```ts - total(): number { - return this.requests.size; - } -``` - -In `src/lib/event-feed.ts`, inside `decodeEvent`, before the `if (outer.type === "inflight")` block, add: - -```ts - if (outer.type === "activity") { - const id = typeof inner.id === "number" ? inner.id : Number.NaN; - if (Number.isFinite(id)) return { type: "activity", id }; - return null; - } -``` - -- [ ] **Step 4: Run tests to verify they pass** - -Run: `npx tsx --test tests/event-feed.test.ts tests/inflight-tracker.test.ts` -Expected: PASS. - -- [ ] **Step 5: Commit** - -```bash -git add src/lib/inflight-tracker.ts src/lib/event-feed.ts tests/event-feed.test.ts tests/inflight-tracker.test.ts -git commit -m "feat: decode activity SSE events and expose tracker total" -``` - ---- - -### Task 4: Runtime integration - -**Files:** -- Modify: `src/lib/runtime.ts` - -**Interfaces:** -- Consumes: `StatsCache` from `./stats-cache`; `UsageStats` from `./stats`; `FeedEvent` (already imported via tracker). -- Produces (public API used by the In-Flight action): - - `runtime.getStats(modelId: string): UsageStats | undefined` - - `runtime.watchStats(modelId: string): () => void` - -- [ ] **Step 1: Modify `src/lib/runtime.ts`** - -Replace the whole file body after the class opening as shown: - -```ts -import { EventFeed } from "./event-feed"; -import { InflightTracker } from "./inflight-tracker"; -import { MetricsPoller } from "./metrics-poller"; -import { StatsCache } from "./stats-cache"; -import { type UsageStats } from "./stats"; -import { type LlamaSwapConfig } from "./util"; - -class Runtime { - readonly tracker = new InflightTracker(); - offline = true; - private cfg?: LlamaSwapConfig; - private feed?: EventFeed; - private poller?: MetricsPoller; - private statsCache?: StatsCache; - private statsUnsub?: () => void; - private listeners = new Set<() => void>(); - - ensureConnections(cfg: LlamaSwapConfig): void { - const changed = !this.cfg || this.cfg.baseUrl !== cfg.baseUrl || this.cfg.apiKey !== cfg.apiKey; - if (changed) { - this.feed?.stop(); - this.poller?.stop(); - this.feed = undefined; - this.poller = undefined; - this.statsUnsub?.(); - this.statsCache = undefined; - this.statsUnsub = undefined; - this.cfg = cfg; - } - if (!this.feed) { - this.feed = new EventFeed(cfg, (ev) => { - this.tracker.apply(ev); - if (ev.type === "activity") this.statsCache?.scheduleRefresh(); - this.emit(); - }); - this.feed.setStatusHandler((connected) => { - this.offline = !connected; - this.emit(); - }); - this.feed.start(); - } - if (!this.poller) { - this.poller = new MetricsPoller(cfg); - this.poller.on(() => this.emit()); - this.poller.start(); - } - if (!this.statsCache) { - this.statsCache = new StatsCache(cfg); - this.statsUnsub = this.statsCache.onChange(() => this.emit()); - } - } - - get pollerInstance(): MetricsPoller | undefined { - return this.poller; - } - - getStats(modelId: string): UsageStats | undefined { - return this.statsCache?.get(modelId); - } - - watchStats(modelId: string): () => void { - this.statsCache?.register(modelId); - return () => this.statsCache?.unregister(modelId); - } - - subscribe(listener: () => void): () => void { - this.listeners.add(listener); - return () => { - this.listeners.delete(listener); - }; - } - - private emit(): void { - for (const listener of this.listeners) listener(); - } -} - -export const runtime = new Runtime(); -``` - -- [ ] **Step 2: Typecheck and run the full suite** - -Run: `npx tsc --noEmit && npm test` -Expected: `TYPECHECK OK` equivalent (no errors) and all tests pass. - -- [ ] **Step 3: Commit** - -```bash -git add src/lib/runtime.ts -git commit -m "feat: expose usage stats via shared runtime StatsCache" -``` - ---- - -### Task 5: `renderUsage` in render.ts - -**Files:** -- Modify: `src/lib/render.ts` -- Test: `tests/render.test.ts` - -**Interfaces:** -- Consumes: `UsageStats` from `./stats`. -- Produces: - - `interface UsageRenderOptions { modelName: string; stats?: UsageStats; primaryStat: "requests" | "input_tokens" | "output_tokens" | "gen_p95"; offline: boolean }` - - `renderUsage(opts: UsageRenderOptions): string` - - `formatCompact(n: number): string` (exported for tests) - -- [ ] **Step 1: Write the failing tests** - -Append to `tests/render.test.ts`: - -```ts -import { renderGpuGraph, renderInflight, renderUsage, svgDataUrl, formatCompact } from "../src/lib/render"; -``` - -(replace the existing import line) and append: - -```ts -test("formatCompact renders integers, k, and M", () => { - assert.equal(formatCompact(0), "0"); - assert.equal(formatCompact(999), "999"); - assert.equal(formatCompact(52100), "52.1k"); - assert.equal(formatCompact(1200000), "1.2M"); -}); - -test("renderUsage shows the primary stat big and the rest small", () => { - const svg = renderUsage({ - modelName: "DeepSeek-V4-Flash-0731", - stats: { totalRequests: 1950, totalInputTokens: 312177540, totalOutputTokens: 889193, genP95: 378.86 }, - primaryStat: "gen_p95", - offline: false, - }); - assert.match(svg, />379 { - const none = renderUsage({ modelName: "A", stats: undefined, primaryStat: "requests", offline: false }); - assert.match(none, />-- { - const svg = renderUsage({ modelName: "all", stats: { totalRequests: 1, totalInputTokens: 1, totalOutputTokens: 1, genP95: 1 }, primaryStat: "requests", offline: false }); - assert.match(svg, /ALL MODELS/); -}); -``` - -- [ ] **Step 2: Run test to verify it fails** - -Run: `npx tsx --test tests/render.test.ts` -Expected: FAIL — `Cannot find name 'renderUsage'` / `formatCompact`. - -- [ ] **Step 3: Write minimal implementation** - -In `src/lib/render.ts`, add an import of `UsageStats` and append the following at the end of the file (before helper functions or after — helpers can be placed after `renderUsage`): - -```ts -export function formatCompact(n: number): string { - if (!Number.isFinite(n)) return "--"; - if (n < 1000) return `${Math.round(n)}`; - if (n < 1_000_000) return `${(n / 1000).toFixed(1)}k`; - return `${(n / 1_000_000).toFixed(1)}M`; -} - -export interface UsageRenderOptions { - modelName: string; - stats?: UsageStats; - primaryStat: "requests" | "input_tokens" | "output_tokens" | "gen_p95"; - offline: boolean; -} - -export function renderUsage(opts: UsageRenderOptions): string { - const name = escapeXml(shorten(opts.modelName === "all" ? "ALL MODELS" : opts.modelName)); - - if (opts.offline) { - return frame("#10131a", [ - centerText("!!", 34, 20, "bold", "#e0e0e0"), - centerText("OFFLINE", 52, 9, "normal", "#bdbdbd"), - centerText(name, 64, 7, "normal", "#ffffff"), - ]); - } - - const s = opts.stats; - const big = s ? formatCompact(primaryValue(s, opts.primaryStat)) : "--"; - const rows = usageRows(s, opts.primaryStat); - const parts: string[] = [ - centerText(name, 10, 7, "normal", "#ffffff"), - centerText(big, 36, 24, "bold", "#ffffff"), - ]; - for (const row of rows) { - parts.push( - `${row.label}`, - `${row.value}`, - ); - } - return frame("#10131a", parts); -} - -function primaryValue(s: UsageStats, primary: "requests" | "input_tokens" | "output_tokens" | "gen_p95"): number { - switch (primary) { - case "requests": - return s.totalRequests; - case "input_tokens": - return s.totalInputTokens; - case "output_tokens": - return s.totalOutputTokens; - case "gen_p95": - return s.genP95; - } -} - -function usageRows( - s: UsageStats | undefined, - primary: "requests" | "input_tokens" | "output_tokens" | "gen_p95", -): { label: string; value: string; y: number }[] { - const items: { label: string; value: string }[] = []; - if (primary !== "requests") items.push({ label: "REQ", value: s ? formatCompact(s.totalRequests) : "--" }); - if (primary !== "input_tokens") items.push({ label: "IN", value: s ? formatCompact(s.totalInputTokens) : "--" }); - if (primary !== "output_tokens") items.push({ label: "OUT", value: s ? formatCompact(s.totalOutputTokens) : "--" }); - if (primary !== "gen_p95") items.push({ label: "P95", value: s ? `${Math.round(s.genP95)} t/s` : "--" }); - return items.map((it, i) => ({ ...it, y: 52 + i * 7 })); -} -``` - -- [ ] **Step 4: Run test to verify it passes** - -Run: `npx tsx --test tests/render.test.ts` -Expected: PASS. - -- [ ] **Step 5: Commit** - -```bash -git add src/lib/render.ts tests/render.test.ts -git commit -m "feat: add usage stats key renderer" -``` - ---- - -### Task 6: In-Flight action display modes + property inspector - -**Files:** -- Modify: `src/actions/inflight-monitor.ts`, `com.bryce.llamawatch.sdPlugin/ui/inflight.html`, `src/lib/datasources.ts` - -**Interfaces:** -- Consumes: `renderUsage`, `renderInflight`, `formatCompact` (render); `runtime.getStats`/`watchStats`; `InflightTracker.total()`. -- Produces: - - `InflightSettings` gains `display?: "count" | "usage"` and `primaryStat?: "requests" | "input_tokens" | "output_tokens" | "gen_p95"`. - - `InflightState` gains `unwatchStats?: () => void`. - - Models datasource gains an `all` option. - -- [ ] **Step 1: Update the property inspector** - -In `com.bryce.llamawatch.sdPlugin/ui/inflight.html`, after the API Key item, add: - -```html - - - - - - - - - - - - - - -``` - -- [ ] **Step 2: Update the models datasource** - -In `src/lib/datasources.ts`, inside the `event === "models"` branch, change the assignment to prepend the aggregate option: - -```ts - items = [{ label: "All models", value: "all" }, ...models.map((m) => ({ label: `${m.id} (${m.status})`, value: m.id }))]; -``` - -- [ ] **Step 3: Update the In-Flight action** - -In `src/actions/inflight-monitor.ts`: - -1. Add to the settings type: - -```ts -type InflightSettings = CfgSettings & { - modelId?: string; - display?: "count" | "usage"; - primaryStat?: "requests" | "input_tokens" | "output_tokens" | "gen_p95"; -}; -``` - -2. Add to `InflightState`: - -```ts - unwatchStats?: () => void; -``` - -3. In `onWillAppear`, after `state.settings = ev.payload.settings;`, replace the sampler setup block: - -```ts - state.unsubscribe = runtime.subscribe(() => this.render(state)); - if (this.displayOf(state) === "usage") { - state.unwatchStats = runtime.watchStats(this.modelKeyOf(state)); - } else { - state.sampler = setInterval(() => this.sample(state), 1000); - } - this.render(state); -``` - -4. In `onWillDisappear`, add cleanup after `state.unsubscribe?.();`: - -```ts - state.unwatchStats?.(); - state.unwatchStats = undefined; -``` - -5. In `onDidReceiveSettings`, the re-registration must come AFTER - `runtime.ensureConnections(...)` (config changes recreate the - `StatsCache`, so registering earlier would land on the stale cache). - Change the tail of the method to: - -```ts - state.settings = ev.payload.settings; - state.action = ev.action; - state.history = []; - runtime.ensureConnections(cfgFromSettings(state.settings)); - state.unwatchStats?.(); - state.unwatchStats = undefined; - if (this.displayOf(state) === "usage") { - state.unwatchStats = runtime.watchStats(this.modelKeyOf(state)); - } else if (!state.sampler) { - state.sampler = setInterval(() => this.sample(state), 1000); - } - this.render(state); -``` - -6. Replace `render` with a mode-switching version and add helpers: - -```ts - private displayOf(state: InflightState): "count" | "usage" { - return state.settings.display ?? "count"; - } - - private modelKeyOf(state: InflightState): string { - const modelId = state.settings.modelId ?? ""; - return modelId === "" ? "all" : modelId; - } - - private primaryStatOf(state: InflightState): "requests" | "input_tokens" | "output_tokens" | "gen_p95" { - return state.settings.primaryStat ?? "requests"; - } - - private render(state: InflightState): void { - const action = state.action; - if (!action) return; - if (this.displayOf(state) === "usage") { - const modelKey = this.modelKeyOf(state); - const offline = runtime.offline; - void action.setImage( - svgDataUrl( - renderUsage({ - modelName: modelKey, - stats: runtime.getStats(modelKey), - primaryStat: this.primaryStatOf(state), - offline, - }), - ), - ); - return; - } - - const modelId = state.settings.modelId ?? ""; - const trackerState = modelId === "all" ? "ready" : runtime.tracker.state(modelId); - const count = modelId === "all" ? runtime.tracker.total() : runtime.tracker.count(modelId); - const offline = runtime.offline && modelId.length > 0; - const modelName = modelId === "" ? "unset" : modelId; - void action.setImage( - svgDataUrl( - renderInflight({ - modelName, - state: trackerState, - count, - offline, - history: state.history, - }), - ), - ); - } - - private sample(state: InflightState): void { - const modelId = state.settings.modelId ?? ""; - state.history.push(modelId === "all" ? runtime.tracker.total() : runtime.tracker.count(modelId)); - if (state.history.length > HISTORY_LIMIT) state.history.shift(); - this.render(state); - } -``` - -Note: for `modelId === "all"` in count mode, `trackerState` is forced to `"ready"` so the key renders the total number rather than `NO MODEL`/`OFF`. The label comes from `renderInflight`'s `name` (`ALL MODELS` won't match — see step 4). - -- [ ] **Step 4: Ensure the count-mode "all" label renders correctly** - -In `src/lib/render.ts`, `renderInflight` shortens the model name. For `modelName === "all"` pass the display name through `shorten`. Update the line in `renderInflight`: - -```ts - const name = escapeXml(shorten(opts.modelName === "all" ? "ALL MODELS" : opts.modelName)); -``` - -(one-line change; mirrors the `renderUsage` approach). - -- [ ] **Step 5: Typecheck, build, and run the suite** - -Run: `npx tsc --noEmit && npm run build && npm test` -Expected: no type errors, build succeeds, all tests pass. - -- [ ] **Step 6: Commit** - -```bash -git add src/actions/inflight-monitor.ts src/lib/render.ts src/lib/datasources.ts com.bryce.llamawatch.sdPlugin/ui/inflight.html -git commit -m "feat: add usage-stats display mode to In-Flight Monitor action" -``` - ---- - -### Task 7: GPU `combineSeries` helper - -**Files:** -- Modify: `src/lib/metrics-poller.ts` -- Test: `tests/metrics-poller.test.ts` - -**Interfaces:** -- Consumes: `AGGREGATE` (already defined in `metrics-poller.ts`). -- Produces: `combineSeries(histories: number[][], kind: GpuMetricKind): { value?: number; history: number[] }` (exported). - -- [ ] **Step 1: Write the failing test** - -Append to `tests/metrics-poller.test.ts`: - -```ts -import { combineSeries, MetricsPoller } from "../src/lib/metrics-poller"; -``` - -(update the existing import) and append: - -```ts -test("combineSeries averages util and sums power", () => { - const util = combineSeries( - [ - [10, 20, 30], - [20, 40, 60], - ], - "util_percent", - ); - assert.deepEqual(util, { value: 45, history: [15, 30, 45] }); - - const power = combineSeries( - [ - [100, 200], - [150, 250], - ], - "power", - ); - assert.deepEqual(power, { value: 450, history: [250, 450] }); -}); - -test("combineSeries takes the max temperature and skips gaps", () => { - const temp = combineSeries([[50, 70], [60]], "temperature"); - assert.deepEqual(temp, { value: 70, history: [60, 70] }); - - const empty = combineSeries([[], []], "temperature"); - assert.deepEqual(empty, { value: undefined, history: [] }); -}); -``` - -- [ ] **Step 2: Run test to verify it fails** - -Run: `npx tsx --test tests/metrics-poller.test.ts` -Expected: FAIL — `Cannot find name 'combineSeries'`. - -- [ ] **Step 3: Write minimal implementation** - -In `src/lib/metrics-poller.ts`, after the `AGGREGATE` const, add and export: - -```ts -export function combineSeries(histories: number[][], kind: GpuMetricKind): { value?: number; history: number[] } { - const n = Math.max(...histories.map((h) => h.length), 0); - if (n === 0) return { value: undefined, history: [] }; - const history: number[] = []; - for (let i = 0; i < n; i++) { - const at = histories.map((h) => h[h.length - n + i]).filter((v): v is number => v !== undefined); - if (at.length === 0) continue; - history.push(AGGREGATE[kind](at)); - } - return { value: history.length > 0 ? history[history.length - 1] : undefined, history }; -} -``` - -- [ ] **Step 4: Run test to verify it passes** - -Run: `npx tsx --test tests/metrics-poller.test.ts` -Expected: PASS. - -- [ ] **Step 5: Commit** - -```bash -git add src/lib/metrics-poller.ts tests/metrics-poller.test.ts -git commit -m "feat: add combineSeries for GPU combination aggregation" -``` - ---- - -### Task 8: GPU action combinations + property inspector - -**Files:** -- Modify: `src/actions/gpu-graph.ts`, `com.bryce.llamawatch.sdPlugin/ui/gpu.html` - -**Interfaces:** -- Consumes: `combineSeries` from `./metrics-poller`. -- Produces: - - `GpuSettings` gains `gpuCombo?: string`. - - When `gpuCombo` is a non-empty comma-separated list of ids, the key shows the combined series of those GPUs, labeled `GPU 0+2`; otherwise existing `gpuId` behavior. - -- [ ] **Step 1: Update the property inspector** - -In `com.bryce.llamawatch.sdPlugin/ui/gpu.html`, after the GPU select item, add: - -```html - - - -``` - -- [ ] **Step 2: Update the GPU action** - -In `src/actions/gpu-graph.ts`: - -1. Add to the settings type: - -```ts -type GpuSettings = CfgSettings & { - gpuId?: string; - metric?: GpuMetricKind; - gpuCombo?: string; -}; -``` - -2. Replace the `render` method and add a `parseCombo` helper: - -```ts - private render(state: GpuState): void { - const action = state.action; - if (!action) return; - const poller = runtime.pollerInstance; - const metric = state.settings.metric ?? "util_percent"; - const combo = parseCombo(state.settings.gpuCombo); - const offline = !poller || poller.isOffline(); - - let value: number | undefined; - let history: number[] = []; - let gpuName: string; - if (combo) { - const known = new Set((poller?.gpus() ?? []).map((g) => g.id)); - const ids = combo.filter((id) => known.has(id)); - const histories = ids.map((id) => poller?.getHistory(id, metric) ?? []); - const combined = combineSeries(histories, metric); - value = combined.value; - history = combined.history; - gpuName = `GPU ${ids.join("+")}`; - } else { - const gpuId = state.settings.gpuId ?? "all"; - value = poller?.getValue(gpuId, metric); - history = poller?.getHistory(gpuId, metric) ?? []; - gpuName = gpuId === "all" ? "ALL GPUS" : this.gpuLabel(gpuId); - } - - const svg = renderGpuGraph({ gpuName, metric, value, history, offline }); - if (svg === state.lastSvg) return; - state.lastSvg = svg; - void action.setImage(svgDataUrl(svg)); - } -``` - -3. Add the helper at the bottom of the file: - -```ts -function parseCombo(raw: string | undefined): string[] | undefined { - const ids = (raw ?? "") - .split(",") - .map((s) => s.trim()) - .filter(Boolean); - return ids.length > 0 ? ids : undefined; -} -``` - -and import `combineSeries`: - -```ts -import { combineSeries, type GpuMetricKind } from "../lib/metrics-poller"; -``` - -- [ ] **Step 3: Typecheck, build, and run the suite** - -Run: `npx tsc --noEmit && npm run build && npm test` -Expected: no type errors, build succeeds, all tests pass. - -- [ ] **Step 4: Commit** - -```bash -git add src/actions/gpu-graph.ts com.bryce.llamawatch.sdPlugin/ui/gpu.html -git commit -m "feat: support GPU combinations on the GPU Graph action" -``` - ---- - -### Task 9: Final verification - -- [ ] **Step 1: Full verification** - -Run: -```bash -npm test -npm run build -npx tsc --noEmit -streamdeck validate -``` -Expected: all tests pass, build succeeds, typecheck clean, validation passes. - -- [ ] **Step 2: Visual smoke test** - -Run `npx tsx -e` to render a usage SVG and a combo GPU SVG and confirm they are well-formed (no malformed attributes), e.g.: - -```bash -npx tsx -e ' -import { renderUsage } from "./src/lib/render"; -console.log(renderUsage({ modelName: "all", stats: { totalRequests: 1950, totalInputTokens: 312177540, totalOutputTokens: 889193, genP95: 378.86 }, primaryStat: "gen_p95", offline: false }).slice(0, 300)); -' -``` - -- [ ] **Step 3: Restart the plugin** - -Run: `streamdeck restart com.bryce.llamawatch` -Expected: `✔ Restarted com.bryce.llamawatch`. - -- [ ] **Step 4: Update the progress ledger** - -Append a short note to `.superpowers/sdd/progress.md` summarizing the feature and verification results (one or two lines, matching existing format). diff --git a/docs/superpowers/specs/2026-08-14-inflight-activity-design.md b/docs/superpowers/specs/2026-08-14-inflight-activity-design.md deleted file mode 100644 index b91f23d..0000000 --- a/docs/superpowers/specs/2026-08-14-inflight-activity-design.md +++ /dev/null @@ -1,61 +0,0 @@ -# In-Flight Monitor — Count + Activity Spark - -**Date:** 2026-08-14 -**Status:** Approved (design) - -## Summary - -Enhance the In-Flight Monitor action so its key shows the **number of -in-flight requests** for the configured model plus a **count-trend spark**: -a tiny line/area chart of that count over the last ~60s, so activity is -visible even during a single long-running request. - -## Layout (72×72 SVG, rendered via `renderInflight`) - -- **Top:** model short name (small, white) — existing. -- **Center:** the status. When `ready`: - - count > 0 → the **count number**, large/bold (e.g. `3`), on the red - `ACTIVE` background, with the existing pulse. - - count === 0 → `IDLE` on green — existing. - - `loading` → `LOADING`, `stopped` → `OFF`, offline → `OFFLINE`, - unconfigured → `NO MODEL`, unknown → `…` — all unchanged. -- **Bottom strip (~14px):** the count-trend spark — a small polyline + faint - area fill of the model's in-flight count over the last 60 samples, in the - state color. Flat at the baseline when idle; spikes when requests come and go. - -## Data Flow - -- The feed side is unchanged — `InflightTracker` already exposes exact - per-model counts. -- Each key's per-key state (in `inflight-monitor.ts`, keyed by - `ev.action.id`) gains: - - `history: number[]` — a 60-sample ring buffer of the live count. - - `sampler?: ReturnType` — a 1s interval that pushes - `runtime.tracker.count(modelId)` into `history` and re-renders. - Started in `onWillAppear`, cleared in `onWillDisappear`. -- Live feed events (`add`/`remove`/`snapshot`) still re-render the big - number immediately via the existing subscription; the spark refreshes at - 1Hz. -- `renderInflight` gains an optional `history?: number[]` field. Spark - scale: yMax = `max(max(history), 1)` so a zero line sits at the baseline. - -## Files - -- `src/lib/render.ts` — `renderInflight` accepts `history` and draws the - spark; center label shows the count when active. -- `src/actions/inflight-monitor.ts` — per-key `history` ring buffer + - 1s sampler. -- `tests/render.test.ts` — new/updated assertions for count rendering and - the spark. - -## Error Handling / Edge Cases - -- Spark with <2 samples renders no line (like the GPU chart). -- Sampler cleared on `onWillDisappear` (no leaks); multi-key safety - preserved (buffer + sampler live in the per-key map entry). -- History capped at 60 samples. - -## Out of Scope - -- No change to the GPU Graph action, the feed, or the tracker. -- No throughput-bar variant (count trend was chosen). diff --git a/docs/superpowers/specs/2026-08-14-llama-watch-design.md b/docs/superpowers/specs/2026-08-14-llama-watch-design.md deleted file mode 100644 index ffa6fca..0000000 --- a/docs/superpowers/specs/2026-08-14-llama-watch-design.md +++ /dev/null @@ -1,166 +0,0 @@ -# llama-watch — Stream Deck Plugin Design - -**Date:** 2026-08-14 -**Status:** Approved (pending spec review) - -## Summary - -A Stream Deck plugin ("llama-watch") that monitors a llama-swap instance -(`http://localhost:9292`). Two key-action types: - -1. **In-flight Monitor** — shows whether a specific model currently has a - request in flight, with a color-coded state (OFF / LOADING / IDLE / ACTIVE). -2. **GPU Graph** — renders a live line chart of a selected GPU metric - (utilization %, VRAM %, temperature, power draw, or fan speed) on the key, - for a specific GPU or the "All GPUs" aggregate. - -Target device: standard Stream Deck (72x72 px keys), rendered at 144x144 for -crispness. macOS only. Local install, with eventual Elgato Marketplace -submission as a design goal. - -## Approach - -Official Elgato JS SDK (`streamdeck-jssdk`) + `streamdeck-cli` tooling, a -TypeScript Node.js plugin, and `@napi-rs/canvas` for image rendering. The -plugin runs locally (launched by the Stream Deck app) and talks directly to -the llama-swap instance. - -## Data Sources - -llama-swap exposes everything needed (verified live against -`localhost:9292`): - -- **`GET /api/events`** — SSE stream used by the web UI. Emits `inflight` - events (operations `snapshot` / `add` / `remove`, per-request detail: - model, timestamp, elapsed, bytes) and `modelStatus` events (per-model - state: `stopped`, `loading`, `ready`). Also emits `logData`, `uiConfig`, - `profileChanged`, `activity` events — ignored. -- **`GET /metrics`** — Prometheus text format. Relevant series: - - `llamaswap_gpu_util_percent{id,name,uuid}` - - `llamaswap_gpu_memory_util_percent{id,name,uuid}` - - `llamaswap_gpu_temperature_celsius{id,name,uuid}` - - `llamaswap_gpu_power_draw_watts{id,name,uuid}` - - `llamaswap_gpu_fan_speed_percent{id,name,uuid}` -- **`GET /v1/models`** — model list (with `status.value` loaded/unloaded), - used to populate the model dropdown in the property inspector. - -Upstream vLLM metrics (ports 10001/10003) are bound to localhost on the -server and are NOT reachable from the client Mac. Not used. - -## Architecture - -The plugin is a single Node process with three connections: - -1. WebSocket to the Stream Deck software (via `streamdeck-jssdk`). -2. Persistent SSE connection to `/api/events` (the `EventFeed`). -3. 5s-interval polling of `/metrics` (the `MetricsPoller`). - -### Modules - -- **`MetricsPoller`** (pure logic, testable) - - Polls `GET /metrics` every **5 seconds** (fixed). - - Parses the Prometheus text format into a lookup keyed by metric name × - GPU id. - - `getSnapshot(gpuSelector, metric)` resolves any selector × metric - combination into a number, including the "All GPUs" aggregate. - - Keeps a ring buffer of 60 samples (~5 minutes) per (gpu, metric). - - A missed/failed poll skips that sample (no gap in rendering beyond a - break in the line). -- **`EventFeed`** (pure logic, testable) - - Persistent SSE connection to `/api/events`. - - Reconnect with exponential backoff (1s → 30s max); the `inflight` - `snapshot` operation emitted on connect self-heals the tracker. - - Maintains per-model in-flight request counts and per-model state from - `modelStatus`. -- **Actions** - - `InflightMonitor` — one instance per monitored model. - - `GpuGraph` — one instance per GPU (or All GPUs) × metric. -- **Property inspectors** (`pi/*.html`) — settings UI for both actions. -- **Renderer** — draws 144x144 PNGs via `@napi-rs/canvas`. - -### Data Flow - -``` -llama-swap ──SSE /api/events──▶ EventFeed ──▶ per-model inflight counts + state - ──HTTP /metrics 5s──▶ MetricsPoller ──▶ ring buffers + aggregates - │ -Stream Deck app ◀──jssdk WS── Actions ────────┘ - ◀──144x144 PNG + title── Renderer -``` - -### Settings (shared + per action) - -- **Shared:** base URL (default `http://localhost:9292`), optional API - key (sent as a header when set; instance currently requires none). -- **InflightMonitor:** model (dropdown populated live from `/v1/models`). -- **GpuGraph:** GPU selector (dropdown from live GPU list, plus "All GPUs"), - metric (dropdown: Utilization %, VRAM %, Temperature, Power draw, Fan - speed). - -## Action 1 — In-flight Monitor - -- **States (rendered on key):** - - `stopped` / unloaded → grey background, model name, `OFF` - - `loading` → amber, `LOADING` - - `ready` + 0 requests → green, `IDLE` - - `ready` + ≥1 request → red, `ACTIVE` (subtle pulse, re-render ~2 Hz while - active) -- Model short name displayed on the key; if no model selected, placeholder - with an edit hint. -- **Press:** opens `http:///ui` in the default browser (spawns - `open` on macOS). - -## Action 2 — GPU Graph - -- **Rendering (graph dominates the 72x72 key):** - - Big line chart of the last 60 samples (5 min @ 5s) filling the key. - - Dark background, bright line, subtle filled gradient under the curve. - - Small current-value label (e.g. `67%`, `336W`, `55°C`) at top; metric - name (e.g. `UTIL`, `PWR`) at bottom. -- **Scale:** % metrics fixed 0–100; temperature auto 0–100°C; power - auto-scaled to observed max. -- **Colors:** severity-based — green → amber → red. Temperature red at - ≥80°C; util/power/fan scale with level. -- **Aggregates ("All GPUs"):** Utilization/VRAM/Fan → average; Temperature → - max; Power → sum. -- **Render-on-change:** idle/flat history does not re-render every poll. -- **Press:** opens the web UI (same as Action 1). - -## Error Handling - -- llama-swap unreachable → dark grey key with `!!` and `OFFLINE`; retry with - backoff; auto-recover when server returns. -- SSE drop → reconnect with backoff (1s → 30s); self-healing via `snapshot`. -- `/metrics` poll timeout → skip sample. -- Per-button error states — a bad setting on one key does not affect others. - -## Packaging & Assets - -- Package UUID: `com.bryce.llamawatch` (reverse-DNS, unique). -- `streamdeck-cli` scaffolds project, builds TypeScript, packages - `.streamDeckPlugin` (zip) → installs via double-click into - `~/Library/Application Support/com.elgato.StreamDeck/Plugins/`. -- Marketplace-ready from the start: - - 256×256 plugin icon + action icons (incl. pressed states) at marketplace - spec sizes. - - Name "llama-watch", category e.g. "System & Monitoring". - - Privacy note: reads GPU metrics + request status from the user's own - llama-swap server; only base URL + optional API key stored in Stream - Deck's local settings; no data leaves the machine. - - No hardcoded secrets/URLs — base URL is user-editable. - - Manifest declares macOS only initially. - -## Testing - -- Unit tests with `node:test` for the Prometheus parser, inflight tracker, - and aggregate math (avg/max/sum), using **real captured fixtures** from - the server instance (metrics body, inflight snapshot, modelStatus payloads). -- Manual verification: both actions render correctly on the physical Stream - Deck; key press opens the web UI. - -## Out of Scope (YAGNI) - -- Marketplace submission itself (goal for later; assets/metadata prepared now). -- Auto-update plumbing. -- Windows/Linux support (later add-on; macOS declared in manifest). -- Additional metrics or display styles. diff --git a/docs/superpowers/specs/2026-08-14-usage-stats-gpu-combos-design.md b/docs/superpowers/specs/2026-08-14-usage-stats-gpu-combos-design.md deleted file mode 100644 index 6250e60..0000000 --- a/docs/superpowers/specs/2026-08-14-usage-stats-gpu-combos-design.md +++ /dev/null @@ -1,131 +0,0 @@ -# Usage Stats + GPU Combinations - -**Date:** 2026-08-14 -**Status:** Approved (design) - -## Summary - -Two extensions to the existing llama-watch actions: - -1. The In-Flight Monitor action gains a **Usage stats** display mode showing - token/request totals and the generation-speed P95 from llama-swap's - `/api/metrics/stats` endpoint. -2. The GPU Graph action gains **GPU combinations**: a key can aggregate an - arbitrary subset of GPUs (e.g. only the two Blackwells). - -## Data Source (verified against llama-swap source) - -- `GET /api/metrics/stats` (no query param) → global aggregates. -- `GET /api/metrics/stats?model=` → per-model aggregates. -- Response fields used: `total_requests`, `total_input_tokens`, - `total_output_tokens`, `gen_histogram.p95`. -- `p95` is a **tokens/sec generation-speed percentile**, not latency. -- Totals cover llama-swap's in-memory activity retention (default ~1000 - requests), not lifetime counters. -- The `/api/events` SSE `activity` event (payload `{"id": N}`) fires per - completed request and is used as a "re-poll stats" trigger. It carries no - data itself. -- The Prometheus `/metrics` endpoint exposes no token/request/latency data - and is not used for this feature. - -## 1. Usage stats data layer - -- **`src/lib/stats.ts`** (new): - - `interface UsageStats { totalRequests: number; totalInputTokens: number; totalOutputTokens: number; genP95: number }`. - - `parseStats(json: unknown): UsageStats | null` — defensive; returns - `null` on malformed input; missing fields default to `0`. - - `fetchStats(cfg: LlamaSwapConfig, modelId: string): Promise` — - fetches `/api/metrics/stats` (with `?model=` when `modelId` is not - `"all"`), returns parsed stats or `null` on HTTP error/parse failure. -- **`src/lib/stats-cache.ts`** (new): - - `StatsCache` holds `Set` of registered model keys (`"all"` or a - model id) and a `Map` of last-known values. - - One shared 5s interval polls every registered key via `fetchStats`. - - `register(key)` / `unregister(key)` start/stop the interval as needed - (no ref counting — each key registers/unregisters once). - - `get(key): UsageStats | undefined`. - - `refresh()` re-polls all registered keys. - - Change listener invoked after each poll completes (success or failure). - - Fetch failure keeps the last-known value. -- **`src/lib/event-feed.ts`**: `decodeEvent` learns the `activity` event — - `{ type: "activity", id: number }`. `FeedEvent` (in `inflight-tracker.ts`) - gains the `activity` variant. -- **`src/lib/runtime.ts`**: runtime owns a `StatsCache` (created alongside - feed/poller, reset on config change). Feed `activity` events trigger - `statsCache.refresh()` throttled to at most once per 2s. Runtime exposes: - - `getStats(modelId: string): UsageStats | undefined` - - `watchStats(modelId: string): () => void` (register; returns - unsubscribe). - - Stats changes flow through the existing `subscribe`/`emit` mechanism so - actions re-render on the normal tick. - -## 2. In-Flight action modes - -- New settings (per key, in `inflight-monitor.ts`): - - `display: "count" | "usage"` (default `count`). - - `primaryStat: "requests" | "input_tokens" | "output_tokens" | "gen_p95"` - (default `requests`). -- Model setting semantics: - - Model select (PI datasource) gains an **"All models"** option, value - `all`. - - Count mode: `all` → total in-flight across all models; label - `ALL MODELS`. - - Usage mode: `all` → global stats; otherwise per-model stats. -- **Count mode**: unchanged rendering (count number, spark, `IDLE`/ - `LOADING`/`OFF`/`OFFLINE`). The 1s spark sampler runs only in count mode. -- **Usage mode** (`renderUsage` in `render.ts`, dark `#10131a` bg): - - top: model short name or `ALL MODELS` (7px). - - center (24px bold): the `primaryStat` value, compact-formatted — - `< 1000` integer, else `x.xk`, else `x.xM` (`genP95` shows integer t/s). - - three small rows (7px): the remaining three stats, labeled `REQ`, - `IN`, `OUT`, `P95`. - - offline → `OFFLINE` overlay; stats unknown → `--`. -- Press opens the UI (unchanged). - -## 3. GPU combinations - -- New setting `gpuCombo?: string` — comma-separated GPU ids, e.g. `"0,2"`. - When non-empty it overrides `gpuId`. -- Pure helper in `metrics-poller.ts` (exported, tested): - `combineSeries(histories: number[][], kind): { value?: number; history: number[] }` - — element-wise across the per-GPU rings, aligned by index, applying the - existing aggregate rules: `util_percent`/`memory_util_percent`/`fan` → - average, `temperature` → max, `power` → sum. A trailing `undefined` value - from an empty ring leaves that element undefined. -- `gpu-graph.ts` render: for a combo, build per-GPU histories via - `poller.getHistory(id, metric)`, combine, use the combined value/history. - Unknown ids are filtered out; if none remain → `--`. -- Label: `GPU 0+2` (ids joined with `+`). - -## 4. Property inspectors - -- `ui/inflight.html`: add "Display" select (`count`/`usage`) and "Primary - stat" select. Both always visible; `primaryStat` has no effect in count - mode. -- `src/lib/datasources.ts`: the `models` datasource prepends - `{ label: "All models", value: "all" }` to its items (so the model select - offers both the aggregate and individual models). -- `ui/gpu.html`: add "GPU combination (comma-separated ids)" textfield, - optional, placeholder `e.g. 0,2`. - -## Error Handling / Edge Cases - -- Stats fetch error → keep last-known values; no `OFFLINE` flip (offline is - driven by feed status as today). -- Server offline → existing `OFFLINE` overlay in both modes. -- Stats not yet loaded → `--` placeholders. -- Activity-triggered refresh throttled to 1 per 2s. -- Combo with no valid ids → `--`. -- No manifest change (both features extend existing actions). - -## Testing - -- `parseStats`: happy path; malformed input; missing fields → `0`. -- `StatsCache`: register/unregister lifecycle (interval start/stop), - single poll per model, values cached, refresh on demand, failure keeps - last value. Uses an injected fetch function. -- `decodeEvent`: `activity` event decodes to `{ type: "activity", id }`. -- `renderUsage`: big primary stat, small labeled rows, compact formatting - (`52.1k`), `--` when no data, `ALL MODELS` label, offline overlay. -- `combineSeries`: avg/max/sum per metric; empty/mismatched histories. -- In-Flight action: `all` count sums models.