# llama-watch Stream Deck Plugin — Implementation Plan > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. **Goal:** Build a local-install Stream Deck plugin ("llama-watch") with two key actions — an in-flight request monitor per model and a live GPU metric graph — backed by a llama-swap instance at `http://localhost:9292`. **Architecture:** A Node.js/TypeScript plugin running as a process launched by the Stream Deck app (SDK v2, `@elgato/streamdeck@2.1.1`). It maintains a persistent SSE connection to llama-swap `/api/events` (real-time in-flight + model status) and a 5s polling loop over `/metrics` (Prometheus text). Pure-logic modules (parser, tracker, poller, SSE parser, SVG renderer) are unit-tested with `node:test` + `tsx`. Key images are rendered as dynamic SVG strings passed to `KeyAction.setImage` — no canvas dependency. **Tech Stack:** TypeScript, `@elgato/streamdeck@^2.1.1`, Rollup (`@rollup/plugin-typescript`, terser), `@elgato/cli@^1.8.1` for validate/pack, `tsx` for tests, Node 24 runtime declared in the manifest (Stream Deck bundles it; the SDK requires ≥24). Icons generated from hand-written SVGs via macOS `sips`. ## Global Constraints - Plugin UUID: `com.bryce.llamawatch` (reverse-DNS; lowercase alphanumeric, hyphens, periods only). - Action UUIDs: `com.bryce.llamawatch.inflight`, `com.bryce.llamawatch.gpu`. - Manifest: `SDKVersion: 2`, `Nodejs.Version: "24"`, macOS only (`OS: [{Platform: "mac", MinimumVersion: "13"}]`), `Software.MinimumVersion: "7.1"`. (Verified against `@elgato/schemas`: Nodejs `"24"` requires Software ≥ 7.1; the user's Stream Deck app is 7.4.2.) - Dependency versions pinned per the official hello-world sample: `@elgato/streamdeck@^2.1.1`, `@elgato/cli@^1.8.1`, `@rollup/plugin-commonjs@^28.0.2`, `@rollup/plugin-node-resolve@^16.0.0`, `@rollup/plugin-terser@^0.4.4`, `@rollup/plugin-typescript@^12.1.2`, `@tsconfig/node20@^20.1.4`, `@types/node@22.13.0`, `rollup@^4.32.1`, `tslib@^2.8.1`, `typescript@^5.7.3`, `tsx@^4.19.2`. - Poll interval for `/metrics`: fixed **5000 ms**; ring buffer **60 samples** (~5 min). - Metrics (from `llamaswap_gpu_*` Prometheus series): `util_percent`, `memory_util_percent`, `temperature`, `power`, `fan`. - Aggregates for GPU selector `"all"`: util/memory/fan = average, temperature = max, power = sum. - Default base URL: `http://localhost:9292`. No hardcoded URLs beyond the default. Optional API key sent as `Authorization: Bearer ` when set. - Key press on either action opens `/ui` in the default browser via `streamDeck.system.openUrl`. - No Marketplace submission in scope; assets are prepared marketplace-ready (plugin PNG 256/512, SVG action icons, category icon). - No code comments. Follow the no-comments rule strictly. - Existing conventions from the official sample: entry `src/plugin.ts`, output `${PLUGIN}.sdPlugin/bin/plugin.js`, extension-less relative TS imports, ESM (`"type": "module"`). --- ### Task 1: Scaffold project structure **Files:** - Create: `package.json` - Create: `tsconfig.json` - Create: `rollup.config.mjs` - Create: `com.bryce.llamawatch.sdPlugin/manifest.json` (placeholder manifest) - Create: `src/plugin.ts` (placeholder) - Create: `src/actions/inflight-monitor.ts` (placeholder), `src/actions/gpu-graph.ts` (placeholder) - Create: `tests/.gitkeep` - Create: `.gitignore` **Interfaces:** - Consumes: nothing. - Produces: working `npm run build` (rollup outputs `com.bryce.llamawatch.sdPlugin/bin/plugin.js`), `npm test` runner available, placeholder plugin registers no actions yet. - [ ] **Step 1: Create `package.json`** ```json { "name": "llama-watch", "version": "1.0.0", "description": "Stream Deck plugin to monitor llama-swap requests and GPU metrics.", "private": true, "type": "module", "scripts": { "build": "rollup -c", "watch": "rollup -c -w --watch.onEnd=\"streamdeck restart com.bryce.llamawatch\"", "test": "tsx --test tests/" }, "devDependencies": { "@elgato/cli": "^1.8.1", "@rollup/plugin-commonjs": "^28.0.2", "@rollup/plugin-node-resolve": "^16.0.0", "@rollup/plugin-terser": "^0.4.4", "@rollup/plugin-typescript": "^12.1.2", "@tsconfig/node20": "^20.1.4", "@types/node": "22.13.0", "rollup": "^4.32.1", "tslib": "^2.8.1", "tsx": "^4.19.2", "typescript": "^5.7.3" }, "dependencies": { "@elgato/streamdeck": "^2.1.1" } } ``` - [ ] **Step 2: Create `tsconfig.json`** ```json { "extends": "@tsconfig/node20/tsconfig.json", "compilerOptions": { "customConditions": ["node"], "noImplicitOverride": true, "module": "ES2022", "moduleResolution": "Bundler" }, "include": ["src/**/*.ts"], "exclude": ["node_modules"] } ``` - [ ] **Step 3: Create `rollup.config.mjs`** ```js import commonjs from "@rollup/plugin-commonjs"; import nodeResolve from "@rollup/plugin-node-resolve"; import terser from "@rollup/plugin-terser"; import typescript from "@rollup/plugin-typescript"; import path from "node:path"; import url from "node:url"; const isWatching = !!process.env.ROLLUP_WATCH; const sdPlugin = "com.bryce.llamawatch.sdPlugin"; const config = { input: "src/plugin.ts", output: { file: `${sdPlugin}/bin/plugin.js`, sourcemap: isWatching, sourcemapPathTransform: (relativeSourcePath, sourcemapPath) => { return url.pathToFileURL(path.resolve(path.dirname(sourcemapPath), relativeSourcePath)).href; }, }, plugins: [ { name: "watch-externals", buildStart: function () { this.addWatchFile(`${sdPlugin}/manifest.json`); }, }, typescript({ mapRoot: isWatching ? "./" : undefined }), nodeResolve({ browser: false, exportConditions: ["node"], preferBuiltins: true }), commonjs(), !isWatching && terser(), { name: "emit-module-package-file", generateBundle() { this.emitFile({ fileName: "package.json", source: `{ "type": "module" }`, type: "asset" }); }, }, ], }; export default config; ``` - [ ] **Step 4: Create `.gitignore`** ``` node_modules/ *.streamDeckPlugin .DS_Store ``` - [ ] **Step 5: Create placeholder `com.bryce.llamawatch.sdPlugin/manifest.json`** ```json { "$schema": "https://schemas.elgato.com/streamdeck/plugins/manifest.json", "Name": "llama-watch", "Version": "1.0.0.0", "Author": "Bryce Zuccaro", "Actions": [], "Category": "llama-watch", "CodePath": "bin/plugin.js", "Description": "Monitor llama-swap: in-flight requests per model and live GPU metric graphs.", "Icon": "imgs/plugin/icon", "SDKVersion": 2, "Software": { "MinimumVersion": "7.1" }, "OS": [{ "Platform": "mac", "MinimumVersion": "13" }], "Nodejs": { "Version": "24" }, "UUID": "com.bryce.llamawatch" } ``` - [ ] **Step 6: Create placeholder `src/plugin.ts`** ```ts import streamDeck from "@elgato/streamdeck"; streamDeck.connect(); ``` - [ ] **Step 7: Create placeholder action files** — `src/actions/inflight-monitor.ts` and `src/actions/gpu-graph.ts`, each containing only: ```ts export {}; ``` - [ ] **Step 8: Create `tests/.gitkeep`** (empty file) - [ ] **Step 9: Install dependencies and verify the build and test runner** Run: ```bash npm install npm run build npm test ``` Expected: `npm run build` produces `com.bryce.llamawatch.sdPlugin/bin/plugin.js`; `npm test` exits 0 (no test files yet). - [ ] **Step 10: Commit** ```bash git add -A git commit -m "chore: scaffold llama-watch Stream Deck plugin" ``` --- ### Task 2: `util.ts` — shared helpers and config **Files:** - Create: `src/lib/util.ts` - Test: `tests/util.test.ts` **Interfaces:** - Consumes: nothing. - Produces: `DEFAULT_BASE_URL`, `CfgSettings`, `LlamaSwapConfig`, `cfgFromSettings(settings): LlamaSwapConfig`, `normalizeBaseUrl(s): string`, `shorten(name, max?): string`, `escapeXml(s): string`. - [ ] **Step 1: Write the failing test** — `tests/util.test.ts` ```ts import assert from "node:assert/strict"; import { test } from "node:test"; import { cfgFromSettings, escapeXml, normalizeBaseUrl, shorten, } from "../src/lib/util"; test("normalizeBaseUrl strips trailing slashes", () => { assert.equal(normalizeBaseUrl("http://localhost:9292/"), "http://localhost:9292"); assert.equal(normalizeBaseUrl("http://localhost:9292"), "http://localhost:9292"); }); test("cfgFromSettings defaults baseUrl and omits empty apiKey", () => { const cfg = cfgFromSettings({}); assert.equal(cfg.baseUrl, "http://localhost:9292"); assert.equal(cfg.apiKey, undefined); assert.deepEqual(cfgFromSettings({ baseUrl: "http://x/", apiKey: "abc" }), { baseUrl: "http://x", apiKey: "abc", }); }); test("shorten keeps short names and truncates long ones", () => { assert.equal(shorten("Qwen3.8-27B"), "Qwen3.8-27B"); assert.equal(shorten("DeepSeek-V4-Flash-0731"), "DeepSeek-…0731"); }); test("escapeXml escapes XML special characters", () => { assert.equal(escapeXml(`A&B "D" 'E'`), "A&B <C> "D" 'E'"); }); ``` - [ ] **Step 2: Run test to verify it fails** Run: `npm test` Expected: FAIL — module `../src/lib/util` not found. - [ ] **Step 3: Implement `src/lib/util.ts`** ```ts export const DEFAULT_BASE_URL = "http://localhost:9292"; export interface CfgSettings { baseUrl?: string; apiKey?: string; } export interface LlamaSwapConfig { baseUrl: string; apiKey?: string; } export function cfgFromSettings(settings: CfgSettings): LlamaSwapConfig { return { baseUrl: normalizeBaseUrl(settings.baseUrl ?? DEFAULT_BASE_URL), apiKey: settings.apiKey || undefined, }; } export function normalizeBaseUrl(baseUrl: string): string { return baseUrl.trim().replace(/\/+$/, ""); } export function shorten(name: string, max = 14): string { if (name.length <= max) return name; const head = Math.floor((max - 1) * 0.7); return `${name.slice(0, head)}…${name.slice(name.length - (max - head - 1))}`; } export function escapeXml(text: string): string { return text .replace(/&/g, "&") .replace(//g, ">") .replace(/"/g, """) .replace(/'/g, "'"); } ``` - [ ] **Step 4: Run test to verify it passes** Run: `npm test` Expected: PASS. - [ ] **Step 5: Commit** ```bash git add -A git commit -m "feat: add shared util helpers and config" ``` --- ### Task 3: `metrics-parser.ts` — Prometheus text parser **Files:** - Create: `src/lib/metrics-parser.ts` - Create: `tests/fixtures/metrics.txt` - Test: `tests/metrics-parser.test.ts` **Interfaces:** - Consumes: nothing. - Produces: - `type GpuMetricKind = "util_percent" | "memory_util_percent" | "temperature" | "power" | "fan"` - `interface GpuSample { id: string; name: string; kind: GpuMetricKind; value: number }` - `parsePrometheus(text: string): { metric: string; labels: Record; value: number }[]` - `parseGpuMetrics(text: string): GpuSample[]` - `gpuInfos(samples: GpuSample[]): { id: string; name: string }[]` (deduped, sorted by numeric id) - [ ] **Step 1: Create the fixture** — `tests/fixtures/metrics.txt` (trimmed capture from localhost:9292) ```text # HELP llamaswap_gpu_util_percent GPU utilization percent (0-100) # TYPE llamaswap_gpu_util_percent gauge llamaswap_gpu_util_percent{id="0",name="NVIDIA RTX PRO 6000 Blackwell Workstation Edition",uuid="GPU-b77149e6-c22d-599e-9728-c7d4cfee0eb4"} 100 llamaswap_gpu_util_percent{id="1",name="NVIDIA RTX PRO 6000 Blackwell Workstation Edition",uuid="GPU-a46fadfa-48d9-c9c4-8fde-9bd641142bb7"} 100 llamaswap_gpu_util_percent{id="2",name="NVIDIA GeForce RTX 5090",uuid="GPU-9a59ac37-12fb-5bc4-4767-4f4347898c0a"} 100 # HELP llamaswap_gpu_memory_util_percent GPU memory utilization percent (0-100) # TYPE llamaswap_gpu_memory_util_percent gauge llamaswap_gpu_memory_util_percent{id="0",name="NVIDIA RTX PRO 6000 Blackwell Workstation Edition",uuid="GPU-b77149e6-c22d-599e-9728-c7d4cfee0eb4"} 98.41347676402383 llamaswap_gpu_memory_util_percent{id="1",name="NVIDIA RTX PRO 6000 Blackwell Workstation Edition",uuid="GPU-a46fadfa-48d9-c9c4-8fde-9bd641142bb7"} 98.41245517790922 llamaswap_gpu_memory_util_percent{id="2",name="NVIDIA GeForce RTX 5090",uuid="GPU-9a59ac37-12fb-5bc4-4767-4f4347898c0a"} 97.02211181648113 # HELP llamaswap_gpu_temperature_celsius GPU temperature in Celsius # TYPE llamaswap_gpu_temperature_celsius gauge llamaswap_gpu_temperature_celsius{id="0",name="NVIDIA RTX PRO 6000 Blackwell Workstation Edition",uuid="GPU-b77149e6-c22d-599e-9728-c7d4cfee0eb4"} 51 llamaswap_gpu_temperature_celsius{id="1",name="NVIDIA RTX PRO 6000 Blackwell Workstation Edition",uuid="GPU-a46fadfa-48d9-c9c4-8fde-9bd641142bb7"} 55 llamaswap_gpu_temperature_celsius{id="2",name="NVIDIA GeForce RTX 5090",uuid="GPU-9a59ac37-12fb-5bc4-4767-4f4347898c0a"} 64 # HELP llamaswap_gpu_power_draw_watts GPU power draw in watts # TYPE llamaswap_gpu_power_draw_watts gauge llamaswap_gpu_power_draw_watts{id="0",name="NVIDIA RTX PRO 6000 Blackwell Workstation Edition",uuid="GPU-b77149e6-c22d-599e-9728-c7d4cfee0eb4"} 316.56 llamaswap_gpu_power_draw_watts{id="1",name="NVIDIA RTX PRO 6000 Blackwell Workstation Edition",uuid="GPU-a46fadfa-48d9-c9c4-8fde-9bd641142bb7"} 336.17 llamaswap_gpu_power_draw_watts{id="2",name="NVIDIA GeForce RTX 5090",uuid="GPU-9a59ac37-12fb-5bc4-4767-4f4347898c0a"} 377.28 # HELP llamaswap_gpu_fan_speed_percent GPU fan speed percent (0-100) # TYPE llamaswap_gpu_fan_speed_percent gauge llamaswap_gpu_fan_speed_percent{id="0",name="NVIDIA RTX PRO 6000 Blackwell Workstation Edition",uuid="GPU-b77149e6-c22d-599e-9728-c7d4cfee0eb4"} 51 llamaswap_gpu_fan_speed_percent{id="1",name="NVIDIA RTX PRO 6000 Blackwell Workstation Edition",uuid="GPU-a46fadfa-48d9-c9c4-8fde-9bd641142bb7"} 62 llamaswap_gpu_fan_speed_percent{id="2",name="NVIDIA GeForce RTX 5090",uuid="GPU-9a59ac37-12fb-5bc4-4767-4f4347898c0a"} 89 ``` - [ ] **Step 2: Write the failing test** — `tests/metrics-parser.test.ts` ```ts import assert from "node:assert/strict"; import { readFileSync } from "node:fs"; import { test } from "node:test"; import { gpuInfos, parseGpuMetrics, parsePrometheus } from "../src/lib/metrics-parser"; const FIXTURE = readFileSync(new URL("./fixtures/metrics.txt", import.meta.url), "utf8"); test("parsePrometheus parses a gauge with labels", () => { const samples = parsePrometheus(`# TYPE llamaswap_gpu_util_percent gauge\nllamaswap_gpu_util_percent{id="0",name="RTX 5090"} 42.5\n`); assert.equal(samples.length, 1); assert.equal(samples[0].metric, "llamaswap_gpu_util_percent"); assert.deepEqual(samples[0].labels, { id: "0", name: "RTX 5090" }); assert.equal(samples[0].value, 42.5); }); test("parsePrometheus ignores comments and blank lines", () => { const samples = parsePrometheus("# HELP x y\n# TYPE x gauge\n\n"); assert.equal(samples.length, 0); }); test("parseGpuMetrics extracts all five kinds for three GPUs from the fixture", () => { const samples = parseGpuMetrics(FIXTURE); assert.equal(samples.length, 15); const kinds = new Set(samples.map((s) => s.kind)); assert.deepEqual([...kinds].sort(), ["fan", "memory_util_percent", "power", "temperature", "util_percent"]); const power0 = samples.find((s) => s.kind === "power" && s.id === "0"); assert.ok(power0); assert.equal(power0.value, 316.56); }); test("gpuInfos dedupes, preserves names, sorts numerically by id", () => { const infos = gpuInfos(parseGpuMetrics(FIXTURE)); assert.deepEqual(infos.map((i) => i.id), ["0", "1", "2"]); assert.equal(infos[2].name, "NVIDIA GeForce RTX 5090"); }); ``` - [ ] **Step 3: Run test to verify it fails** Run: `npm test` Expected: FAIL — module not found. - [ ] **Step 4: Implement `src/lib/metrics-parser.ts`** ```ts export interface PromSample { metric: string; labels: Record; value: number; } const LINE_RE = /^([A-Za-z_:][A-Za-z0-9_:]*)(?:\{([^}]*)\})?\s+(-?(?:\d+\.?\d*|\.\d+)(?:[eE][+-]?\d+)?)$/; export function parsePrometheus(text: string): PromSample[] { const samples: PromSample[] = []; for (const rawLine of text.split("\n")) { const line = rawLine.trim(); if (!line || line.startsWith("#")) continue; const m = LINE_RE.exec(line); if (!m) continue; const labels: Record = {}; if (m[2]) { for (const pair of m[2].split(",")) { const eq = pair.indexOf("="); if (eq === -1) continue; const value = pair.slice(eq + 1).trim(); labels[pair.slice(0, eq).trim()] = value.replace(/^"(.*)"$/, "$1"); } } samples.push({ metric: m[1], labels, value: parseFloat(m[3]) }); } return samples; } export type GpuMetricKind = "util_percent" | "memory_util_percent" | "temperature" | "power" | "fan"; export const GPU_SERIES: Record = { llamaswap_gpu_util_percent: "util_percent", llamaswap_gpu_memory_util_percent: "memory_util_percent", llamaswap_gpu_temperature_celsius: "temperature", llamaswap_gpu_power_draw_watts: "power", llamaswap_gpu_fan_speed_percent: "fan", }; export interface GpuSample { id: string; name: string; kind: GpuMetricKind; value: number; } export function parseGpuMetrics(text: string): GpuSample[] { const out: GpuSample[] = []; for (const s of parsePrometheus(text)) { const kind = GPU_SERIES[s.metric]; if (!kind) continue; const id = s.labels["id"]; if (id === undefined) continue; out.push({ id, name: s.labels["name"] ?? "", kind, value: s.value }); } return out; } export function gpuInfos(samples: GpuSample[]): { id: string; name: string }[] { const byId = new Map(); for (const s of samples) { if (!byId.has(s.id)) byId.set(s.id, s.name); } return [...byId.entries()] .sort((a, b) => parseInt(a[0], 10) - parseInt(b[0], 10)) .map(([id, name]) => ({ id, name })); } ``` - [ ] **Step 5: Run test to verify it passes** Run: `npm test` Expected: PASS. - [ ] **Step 6: Commit** ```bash git add -A git commit -m "feat: add Prometheus metrics parser for llama-swap GPU series" ``` --- ### Task 4: `llamaswap.ts` — HTTP client **Files:** - Create: `src/lib/llamaswap.ts` **Interfaces:** - Consumes: `LlamaSwapConfig` from `./util`. - Produces: - `interface ModelInfo { id: string; status: string }` - `fetchMetrics(cfg: LlamaSwapConfig): Promise` — GET `/metrics`, throws on non-2xx or timeout (5s). - `fetchModels(cfg: LlamaSwapConfig): Promise` — GET `/v1/models`, maps `data[]` to `{id, status: status.value}`. - [ ] **Step 1: Implement `src/lib/llamaswap.ts`** (network module; verified via `npm run build` typecheck, and integration-tested in Task 12) ```ts import { type LlamaSwapConfig } from "./util"; const TIMEOUT_MS = 5000; export interface ModelInfo { id: string; status: string; } function headersFor(cfg: LlamaSwapConfig): Record { const headers: Record = {}; if (cfg.apiKey) headers["Authorization"] = `Bearer ${cfg.apiKey}`; return headers; } export async function fetchMetrics(cfg: LlamaSwapConfig): Promise { const res = await fetch(`${cfg.baseUrl}/metrics`, { headers: headersFor(cfg), signal: AbortSignal.timeout(TIMEOUT_MS), }); if (!res.ok) throw new Error(`metrics HTTP ${res.status}`); return await res.text(); } export async function fetchModels(cfg: LlamaSwapConfig): Promise { const res = await fetch(`${cfg.baseUrl}/v1/models`, { headers: headersFor(cfg), signal: AbortSignal.timeout(TIMEOUT_MS), }); if (!res.ok) throw new Error(`models HTTP ${res.status}`); const body = (await res.json()) as { data?: { id: string; status?: { value?: string } }[] }; return (body.data ?? []).map((m) => ({ id: m.id, status: m.status?.value ?? "unknown" })); } ``` - [ ] **Step 2: Verify it typechecks in the bundle** Run: `npm run build` Expected: build succeeds (rollup/typescript compiles without error). - [ ] **Step 3: Commit** ```bash git add -A git commit -m "feat: add llama-swap HTTP client" ``` --- ### Task 5: `metrics-poller.ts` — polling, ring buffers, aggregates **Files:** - Create: `src/lib/metrics-poller.ts` - Test: `tests/metrics-poller.test.ts` **Interfaces:** - Consumes: `fetchMetrics` + `LlamaSwapConfig` (default fetchFn), `parseGpuMetrics` + `GpuMetricKind` from `./metrics-parser`. - Produces: - `class MetricsPoller { constructor(cfg: LlamaSwapConfig, intervalMs?: number, fetchFn?: (cfg: LlamaSwapConfig) => Promise) }` - `start(): void`, `stop(): void`, `tick(): Promise` (public for tests) - `on(listener: () => void): () => void` (returns unsubscribe) - `getValue(gpuId: string, kind: GpuMetricKind): number | undefined` - `getHistory(gpuId: string, kind: GpuMetricKind): number[]` (copy of ring) - `gpus(): { id: string; name: string }[]` - `isOffline(): boolean` - [ ] **Step 1: Write the failing test** — `tests/metrics-poller.test.ts` ```ts import assert from "node:assert/strict"; import { readFileSync } from "node:fs"; import { test } from "node:test"; import { type LlamaSwapConfig } from "../src/lib/util"; import { MetricsPoller } from "../src/lib/metrics-poller"; import { parseGpuMetrics } from "../src/lib/metrics-parser"; const FIXTURE = readFileSync(new URL("./fixtures/metrics.txt", import.meta.url), "utf8"); const FIXTURE2 = FIXTURE .replace(/llamaswap_gpu_util_percent\{id="0"[^}]*\} 100/, 'llamaswap_gpu_util_percent{id="0",name="RTX PRO 6000"} 30') .replace(/llamaswap_gpu_util_percent\{id="1"[^}]*\} 100/, 'llamaswap_gpu_util_percent{id="1",name="RTX PRO 6000"} 50') .replace(/llamaswap_gpu_util_percent\{id="2"[^}]*\} 100/, 'llamaswap_gpu_util_percent{id="2",name="RTX 5090"} 80'); function stubFetch(bodies: string[]) { let i = 0; return async () => bodies[Math.min(i++, bodies.length - 1)]; } const CFG: LlamaSwapConfig = { baseUrl: "http://test" }; test("poller stores per-GPU history and computes aggregates", async () => { const poller = new MetricsPoller(CFG, 5000, stubFetch([FIXTURE, FIXTURE2])); await poller.tick(); await poller.tick(); assert.equal(poller.getValue("0", "util_percent"), 30); assert.deepEqual(poller.getHistory("0", "util_percent"), [100, 30]); const allUtil = poller.getValue("all", "util_percent"); assert.ok(allUtil !== undefined); assert.ok(Math.abs(allUtil - (30 + 50 + 80) / 3) < 1e-9); const allTemp = poller.getValue("all", "temperature"); assert.equal(allTemp, 64); const allPower = poller.getValue("all", "power"); assert.ok(allPower !== undefined); assert.ok(Math.abs(allPower - (316.56 + 336.17 + 377.28)) < 1e-9); assert.deepEqual(poller.gpus().map((g) => g.id), ["0", "1", "2"]); assert.equal(poller.gpus()[2].name, "NVIDIA GeForce RTX 5090"); }); test("poller ring buffer caps at 60 samples", async () => { const poller = new MetricsPoller(CFG, 5000, stubFetch([FIXTURE])); for (let i = 0; i < 70; i++) await poller.tick(); assert.equal(poller.getHistory("0", "fan").length, 60); }); test("poller reports offline after a fetch failure and recovers", async () => { let fail = true; const fetchFn = async () => { if (fail) throw new Error("boom"); return FIXTURE; }; const poller = new MetricsPoller(CFG, 5000, fetchFn); await poller.tick(); assert.equal(poller.isOffline(), true); fail = false; await poller.tick(); assert.equal(poller.isOffline(), false); assert.ok(poller.getValue("0", "util_percent") !== undefined); }); ``` - [ ] **Step 2: Run test to verify it fails** Run: `npm test` Expected: FAIL — module not found. - [ ] **Step 3: Implement `src/lib/metrics-poller.ts`** ```ts import { fetchMetrics } from "./llamaswap"; import { type LlamaSwapConfig } from "./util"; import { gpuInfos, parseGpuMetrics, type GpuMetricKind, type GpuSample } from "./metrics-parser"; const RING_SIZE = 60; function avg(values: number[]): number { return values.reduce((a, b) => a + b, 0) / (values.length || 1); } function max(values: number[]): number { return values.reduce((a, b) => Math.max(a, b), -Infinity); } function sum(values: number[]): number { return values.reduce((a, b) => a + b, 0); } const AGGREGATE: Record number> = { util_percent: avg, memory_util_percent: avg, temperature: max, power: sum, fan: avg, }; function key(gpuId: string, kind: GpuMetricKind): string { return `${gpuId}|${kind}`; } type FetchFn = (cfg: LlamaSwapConfig) => Promise; export class MetricsPoller { private buffers = new Map(); private gpuName = new Map(); private gpuIds: string[] = []; private listeners = new Set<() => void>(); private timer?: ReturnType; private lastError?: string; constructor( private cfg: LlamaSwapConfig, private intervalMs = 5000, private fetchFn: FetchFn = fetchMetrics, ) {} start(): void { void this.tick(); this.timer = setInterval(() => void this.tick(), this.intervalMs); } stop(): void { if (this.timer) clearInterval(this.timer); this.timer = undefined; } on(listener: () => void): () => void { this.listeners.add(listener); return () => { this.listeners.delete(listener); }; } async tick(): Promise { try { const text = await this.fetchFn(this.cfg); this.apply(parseGpuMetrics(text)); this.lastError = undefined; } catch (err) { this.lastError = err instanceof Error ? err.message : String(err); } finally { for (const listener of this.listeners) listener(); } } private apply(samples: GpuSample[]): void { const byKind = new Map>(); for (const s of samples) { if (!byKind.has(s.kind)) byKind.set(s.kind, new Map()); byKind.get(s.kind)!.set(s.id, s.value); if (!this.gpuName.has(s.id)) this.gpuName.set(s.id, s.name); } this.gpuIds = [...this.gpuName.keys()].sort((a, b) => parseInt(a, 10) - parseInt(b, 10)); for (const [kind, values] of byKind) { for (const [id, value] of values) this.push(key(id, kind), value); const all = [...values.values()]; if (all.length > 0) this.push(key("all", kind), AGGREGATE[kind](all)); } } private push(k: string, value: number): void { let ring = this.buffers.get(k); if (!ring) { ring = []; this.buffers.set(k, ring); } ring.push(value); if (ring.length > RING_SIZE) ring.shift(); } getValue(gpuId: string, kind: GpuMetricKind): number | undefined { const ring = this.buffers.get(key(gpuId, kind)); return ring ? ring[ring.length - 1] : undefined; } getHistory(gpuId: string, kind: GpuMetricKind): number[] { const ring = this.buffers.get(key(gpuId, kind)); return ring ? [...ring] : []; } gpus(): { id: string; name: string }[] { return this.gpuIds.map((id) => ({ id, name: this.gpuName.get(id) ?? "" })); } isOffline(): boolean { return this.lastError !== undefined; } } ``` - [ ] **Step 4: Run test to verify it passes** Run: `npm test` Expected: PASS. - [ ] **Step 5: Commit** ```bash git add -A git commit -m "feat: add metrics poller with ring buffers and GPU aggregates" ``` --- ### Task 6: `sse.ts` + `inflight-tracker.ts` — SSE parsing and request tracking **Files:** - Create: `src/lib/sse.ts` - Create: `src/lib/inflight-tracker.ts` - Test: `tests/sse.test.ts` - Test: `tests/inflight-tracker.test.ts` **Interfaces:** - Consumes: nothing. - Produces: - `sse.ts`: `interface SseMessage { event?: string; data: string }`, `parseSse(chunk: string): { messages: SseMessage[]; rest: string }`. - `inflight-tracker.ts`: - `interface InflightRequest { id?: string; model: string; req_path?: string; method?: string; timestamp?: string; elapsed_ms?: number }` - `interface ModelState { id: string; state: string }` - `type FeedEvent = { type: "inflight"; operation: "snapshot" | "add" | "remove"; requests: InflightRequest[] } | { type: "modelStatus"; models: ModelState[] }` - `type ModelRuntimeState = "stopped" | "loading" | "ready"` - `class InflightTracker { apply(event: FeedEvent): void; count(modelId: string): number; state(modelId: string): ModelRuntimeState | undefined }` - [ ] **Step 1: Write the failing tests** `tests/sse.test.ts`: ```ts import assert from "node:assert/strict"; import { test } from "node:test"; import { parseSse } from "../src/lib/sse"; test("parseSse extracts event + data blocks separated by blank lines", () => { const { messages, rest } = parseSse('event:message\ndata:{"a":1}\n\nevent:message\ndata:hello\n\n'); assert.equal(rest, ""); assert.equal(messages.length, 2); assert.equal(messages[0].event, "message"); assert.equal(messages[0].data, '{"a":1}'); assert.equal(messages[1].data, "hello"); }); test("parseSse keeps a trailing partial event in rest", () => { const { messages, rest } = parseSse("event:message\ndata:par"); assert.equal(messages.length, 0); assert.equal(rest, "event:message\ndata:par"); }); test("parseSse joins multi-line data fields with newline", () => { const { messages } = parseSse("data:line1\ndata:line2\n\n"); assert.equal(messages.length, 1); assert.equal(messages[0].data, "line1\nline2"); }); ``` `tests/inflight-tracker.test.ts`: ```ts import assert from "node:assert/strict"; import { test } from "node:test"; import { InflightTracker, type FeedEvent } from "../src/lib/inflight-tracker"; test("snapshot rebuilds counts for all in-flight requests", () => { const tracker = new InflightTracker(); tracker.apply({ type: "inflight", operation: "snapshot", requests: [ { model: "A", id: "1" }, { model: "A", id: "2" }, { model: "B", id: "3" }, ], }); assert.equal(tracker.count("A"), 2); assert.equal(tracker.count("B"), 1); assert.equal(tracker.count("C"), 0); }); test("add and remove adjust per-model counts", () => { const tracker = new InflightTracker(); tracker.apply({ type: "inflight", operation: "add", requests: [{ model: "A", id: "1" }] }); tracker.apply({ type: "inflight", operation: "add", requests: [{ model: "A", id: "2" }] }); assert.equal(tracker.count("A"), 2); tracker.apply({ type: "inflight", operation: "remove", requests: [{ model: "A", id: "1" }] }); assert.equal(tracker.count("A"), 1); tracker.apply({ type: "inflight", operation: "remove", requests: [{ model: "A", id: "2" }] }); assert.equal(tracker.count("A"), 0); tracker.apply({ type: "inflight", operation: "remove", requests: [{ model: "A", id: "9" }] }); assert.equal(tracker.count("A"), 0); }); test("modelStatus normalizes states", () => { const tracker = new InflightTracker(); tracker.apply({ type: "modelStatus", models: [ { id: "A", state: "ready" }, { id: "B", state: "loading" }, { id: "C", state: "stopped" }, { id: "D", state: "weird" }, ], }); assert.equal(tracker.state("A"), "ready"); assert.equal(tracker.state("B"), "loading"); assert.equal(tracker.state("C"), "stopped"); assert.equal(tracker.state("D"), "stopped"); assert.equal(tracker.state("missing"), undefined); }); ``` - [ ] **Step 2: Run tests to verify they fail** Run: `npm test` Expected: FAIL — modules not found. - [ ] **Step 3: Implement `src/lib/sse.ts`** ```ts export interface SseMessage { event?: string; data: string; } export function parseSse(chunk: string): { messages: SseMessage[]; rest: string } { const messages: SseMessage[] = []; let rest = chunk; while (true) { const idx = rest.indexOf("\n\n"); if (idx === -1) break; const block = rest.slice(0, idx); rest = rest.slice(idx + 2); const message = toMessage(block); if (message && message.data.length > 0) messages.push(message); } return { messages, rest }; } function toMessage(block: string): SseMessage | null { let event: string | undefined; const dataLines: string[] = []; for (const line of block.split("\n")) { if (line.startsWith("event:")) event = line.slice(6).trim(); else if (line.startsWith("data:")) dataLines.push(line.slice(5).trimStart()); } if (dataLines.length === 0) return null; return { event, data: dataLines.join("\n") }; } ``` - [ ] **Step 4: Implement `src/lib/inflight-tracker.ts`** ```ts export interface InflightRequest { id?: string; model: string; req_path?: string; method?: string; timestamp?: string; elapsed_ms?: number; } export interface ModelState { id: string; state: string; } export type FeedEvent = | { type: "inflight"; operation: "snapshot" | "add" | "remove"; requests: InflightRequest[] } | { type: "modelStatus"; models: ModelState[] }; export type ModelRuntimeState = "stopped" | "loading" | "ready"; export class InflightTracker { private counts = new Map(); private states = new Map(); apply(event: FeedEvent): void { if (event.type === "inflight") { if (event.operation === "snapshot") { const counts = new Map(); for (const r of event.requests) counts.set(r.model, (counts.get(r.model) ?? 0) + 1); this.counts = counts; } else if (event.operation === "add") { for (const r of event.requests) this.counts.set(r.model, (this.counts.get(r.model) ?? 0) + 1); } else if (event.operation === "remove") { for (const r of event.requests) { const c = this.counts.get(r.model) ?? 0; if (c <= 1) this.counts.delete(r.model); else this.counts.set(r.model, c - 1); } } } else if (event.type === "modelStatus") { for (const m of event.models) this.states.set(m.id, normalizeState(m.state)); } } count(modelId: string): number { return this.counts.get(modelId) ?? 0; } state(modelId: string): ModelRuntimeState | undefined { return this.states.get(modelId); } } function normalizeState(state: string): ModelRuntimeState { if (state === "ready") return "ready"; if (state === "loading") return "loading"; return "stopped"; } ``` - [ ] **Step 5: Run tests to verify they pass** Run: `npm test` Expected: PASS. - [ ] **Step 6: Commit** ```bash git add -A git commit -m "feat: add SSE parser and in-flight request tracker" ``` --- ### Task 7: `event-feed.ts` — SSE client and event decoding **Files:** - Create: `src/lib/event-feed.ts` - Test: `tests/event-feed.test.ts` **Interfaces:** - Consumes: `LlamaSwapConfig` from `./util`, `parseSse` + `SseMessage` from `./sse`, `FeedEvent`/`InflightRequest`/`ModelState` from `./inflight-tracker`. - Produces: - `decodeEvent(msg: SseMessage): FeedEvent | null` (exported for tests) - `class EventFeed { constructor(cfg: LlamaSwapConfig, onEvent: (ev: FeedEvent) => void); setStatusHandler(fn: (connected: boolean) => void): void; start(): void; stop(): void }` - Reconnect with exponential backoff 1s → 30s; self-heals from `snapshot`. - [ ] **Step 1: Write the failing test** — `tests/event-feed.test.ts` ```ts import assert from "node:assert/strict"; import { test } from "node:test"; import { decodeEvent } from "../src/lib/event-feed"; import type { SseMessage } from "../src/lib/sse"; test("decodeEvent parses an inflight snapshot", () => { const msg: SseMessage = { event: "message", data: JSON.stringify({ type: "inflight", data: JSON.stringify({ operation: "snapshot", requests: [{ id: "14", model: "Qwen3.8-27B-NVFP4", req_path: "/v1/chat/completions", elapsed_ms: 165510 }], }), }), }; const ev = decodeEvent(msg); assert.ok(ev); assert.equal(ev.type, "inflight"); if (ev.type === "inflight") { assert.equal(ev.operation, "snapshot"); assert.equal(ev.requests.length, 1); assert.equal(ev.requests[0].model, "Qwen3.8-27B-NVFP4"); assert.equal(ev.requests[0].elapsed_ms, 165510); } }); test("decodeEvent parses an inflight add with a singular request field", () => { const msg: SseMessage = { event: "message", data: JSON.stringify({ type: "inflight", data: JSON.stringify({ operation: "add", request: { id: "9", model: "A" } }), }), }; const ev = decodeEvent(msg); assert.ok(ev && ev.type === "inflight"); if (ev && ev.type === "inflight") { assert.equal(ev.operation, "add"); assert.equal(ev.requests[0].model, "A"); } }); test("decodeEvent parses modelStatus", () => { const msg: SseMessage = { event: "message", data: JSON.stringify({ type: "modelStatus", data: JSON.stringify([ { id: "DeepSeek-V4-Flash-0731", state: "ready" }, { id: "Qwen3.8-27B-NVFP4", state: "stopped" }, ]), }), }; const ev = decodeEvent(msg); assert.ok(ev && ev.type === "modelStatus"); if (ev && ev.type === "modelStatus") { assert.equal(ev.models.length, 2); assert.equal(ev.models[0].state, "ready"); } }); test("decodeEvent returns null for unrelated or malformed events", () => { assert.equal(decodeEvent({ event: "message", data: JSON.stringify({ type: "logData", data: "{}" }) }), null); assert.equal(decodeEvent({ event: "message", data: "not json" }), null); }); ``` - [ ] **Step 2: Run test to verify it fails** Run: `npm test` Expected: FAIL — module not found. - [ ] **Step 3: Implement `src/lib/event-feed.ts`** ```ts import { type LlamaSwapConfig } from "./util"; import { parseSse, type SseMessage } from "./sse"; import { type FeedEvent, type InflightRequest, type ModelState } from "./inflight-tracker"; export function decodeEvent(msg: SseMessage): FeedEvent | null { try { const outer = JSON.parse(msg.data) as { type?: string; data?: string }; if (!outer.data) return null; const inner = JSON.parse(outer.data) as Record; if (outer.type === "inflight") { const requests: InflightRequest[] = (inner.requests as InflightRequest[]) ?? (inner.request ? [inner.request as InflightRequest] : []); return { type: "inflight", operation: inner.operation as "snapshot" | "add" | "remove", requests, }; } if (outer.type === "modelStatus") { return { type: "modelStatus", models: inner as unknown as ModelState[] }; } return null; } catch { return null; } } const MAX_RETRY_MS = 30000; const INITIAL_RETRY_MS = 1000; export class EventFeed { private controller?: AbortController; private closed = false; private retryMs = INITIAL_RETRY_MS; private statusHandler?: (connected: boolean) => void; constructor( private cfg: LlamaSwapConfig, private onEvent: (ev: FeedEvent) => void, ) {} setStatusHandler(fn: (connected: boolean) => void): void { this.statusHandler = fn; } start(): void { void this.connect(); } stop(): void { this.closed = true; this.controller?.abort(); } private async connect(): Promise { while (!this.closed) { this.controller = new AbortController(); try { const headers: Record = { Accept: "text/event-stream" }; if (this.cfg.apiKey) headers["Authorization"] = `Bearer ${this.cfg.apiKey}`; const res = await fetch(`${this.cfg.baseUrl}/api/events`, { headers, signal: this.controller.signal, }); if (!res.ok || !res.body) throw new Error(`events HTTP ${res.status}`); this.statusHandler?.(true); this.retryMs = INITIAL_RETRY_MS; await this.readStream(res.body.getReader()); } catch { if (this.closed) return; } this.statusHandler?.(false); if (this.closed) return; await sleep(this.retryMs); this.retryMs = Math.min(this.retryMs * 2, MAX_RETRY_MS); } } private async readStream(reader: ReadableStreamDefaultReader): Promise { const decoder = new TextDecoder(); let buffer = ""; try { while (!this.closed) { const { done, value } = await reader.read(); if (done) break; buffer += decoder.decode(value, { stream: true }); const { messages, rest } = parseSse(buffer); buffer = rest; for (const msg of messages) { const ev = decodeEvent(msg); if (ev) this.onEvent(ev); } } } finally { reader.releaseLock(); } } } function sleep(ms: number): Promise { return new Promise((resolve) => setTimeout(resolve, ms)); } ``` - [ ] **Step 4: Run test to verify it passes** Run: `npm test` Expected: PASS. - [ ] **Step 5: Commit** ```bash git add -A git commit -m "feat: add SSE event feed for llama-swap api/events" ``` --- ### Task 8: `render.ts` — SVG key rendering **Files:** - Create: `src/lib/render.ts` - Test: `tests/render.test.ts` **Interfaces:** - Consumes: `GpuMetricKind` from `./metrics-parser`, `ModelRuntimeState` from `./inflight-tracker`, `shorten`/`escapeXml` from `./util`. - Produces: - `svgDataUrl(svg: string): string` - `renderInflight(opts: { modelName: string; state?: ModelRuntimeState; count: number; offline: boolean; pulse: boolean }): string` - `renderGpuGraph(opts: { gpuName: string; metric: GpuMetricKind; value: number | undefined; history: number[]; offline: boolean }): string` Rendering rules: - Inflight: offline → grey `OFFLINE`; no model (`modelName === "unset"`) → dark `?` / `NO MODEL`; unknown state → grey `…`; `ready`+0 → green `IDLE`; `ready`+≥1 → red `ACTIVE` (opacity toggled by `pulse`); `loading` → amber `LOADING`; else → grey `OFF`. Model name at the bottom. - GPU: dark chart area (x 3..69, y 20..55); % metrics scale 0..100, power scales to `niceCeil(max*1.1)`; `` line + gradient `` fill under curve; value label top (severity-colored), `METRIC · NAME` bottom; offline → grey `OFFLINE` frame. - [ ] **Step 1: Write the failing test** — `tests/render.test.ts` ```ts import assert from "node:assert/strict"; import { test } from "node:test"; import { renderGpuGraph, renderInflight, svgDataUrl } from "../src/lib/render"; test("svgDataUrl wraps an SVG as a base64 data URL", () => { const url = svgDataUrl(""); assert.match(url, /^data:image\/svg\+xml;base64,/); }); test("renderInflight: ready + count renders IDLE in green", () => { const svg = renderInflight({ modelName: "Qwen3.8-27B-NVFP4", state: "ready", count: 0, offline: false, pulse: false }); assert.match(svg, /IDLE/); assert.match(svg, /#1e6b34/); assert.doesNotMatch(svg, /ACTIVE/); }); test("renderInflight: ready + count>0 renders ACTIVE in red", () => { const svg = renderInflight({ modelName: "Qwen3.8-27B-NVFP4", state: "ready", count: 2, offline: false, pulse: false }); assert.match(svg, /ACTIVE/); assert.match(svg, /#8b2626/); }); test("renderInflight: pulse toggles opacity while active", () => { const a = renderInflight({ modelName: "A", state: "ready", count: 1, offline: false, pulse: false }); const b = renderInflight({ modelName: "A", state: "ready", count: 1, offline: false, pulse: true }); assert.notEqual(a, b); assert.match(b, /opacity="0.55"/); }); test("renderInflight: offline and no-model states", () => { const offline = renderInflight({ modelName: "A", state: "ready", count: 0, offline: true, pulse: false }); assert.match(offline, /OFFLINE/); const noModel = renderInflight({ modelName: "unset", state: undefined, count: 0, offline: false, pulse: false }); assert.match(noModel, /NO MODEL/); }); test("renderInflight: loading renders LOADING in amber", () => { const svg = renderInflight({ modelName: "A", state: "loading", count: 0, offline: false, pulse: false }); assert.match(svg, /LOADING/); assert.match(svg, /#8a6d1d/); }); test("renderGpuGraph: draws a polyline, value, and metric label", () => { const svg = renderGpuGraph({ gpuName: "GPU 2 · RTX 5090", metric: "util_percent", value: 67, history: [0, 20, 40, 67], offline: false, }); assert.match(svg, / { const svg = renderGpuGraph({ gpuName: "ALL GPUS", metric: "power", value: 1030, history: [900, 1000, 1030], offline: false, }); assert.match(svg, /1030W/); assert.match(svg, /PWR/); assert.match(svg, /ALL GPUS/); }); test("renderGpuGraph: offline frame", () => { const svg = renderGpuGraph({ gpuName: "ALL GPUS", metric: "util_percent", value: undefined, history: [], offline: true }); assert.match(svg, /OFFLINE/); }); test("renderGpuGraph: single-point history avoids divide-by-zero", () => { const svg = renderGpuGraph({ gpuName: "ALL GPUS", metric: "util_percent", value: 50, history: [50], offline: false }); assert.match(svg, / = { util_percent: "UTIL", memory_util_percent: "VRAM", temperature: "TEMP", power: "PWR", fan: "FAN", }; export interface InflightRenderOptions { modelName: string; state?: ModelRuntimeState; count: number; offline: boolean; pulse: boolean; } export function renderInflight(opts: InflightRenderOptions): string { const name = escapeXml(shorten(opts.modelName)); if (opts.offline) { return frame("#3a3a3a", [ centerText("!!", 34, 20, "bold", "#e0e0e0"), centerText("OFFLINE", 52, 9, "normal", "#bdbdbd"), centerText(name, 64, 7, "normal", "#ffffff"), ]); } if (opts.modelName === "unset") { return frame("#222222", [ centerText("?", 36, 16, "bold", "#888888"), centerText("NO MODEL", 54, 8, "normal", "#666666"), ]); } if (!opts.state) { return frame("#222222", [centerText("…", 36, 16, "bold", "#999999"), centerText(name, 60, 7, "normal", "#999999")]); } const isActive = opts.state === "ready" && opts.count > 0; const bg = opts.state === "ready" ? (isActive ? "#8b2626" : "#1e6b34") : opts.state === "loading" ? "#8a6d1d" : "#3a3a3a"; const label = opts.state === "ready" ? (isActive ? "ACTIVE" : "IDLE") : opts.state === "loading" ? "LOADING" : "OFF"; const opacity = isActive && opts.pulse ? 0.55 : 1; return frame(bg, [ centerText(label, 38, 18, "bold", "#ffffff", opacity), centerText(name, 64, 7, "normal", "#ffffff"), ]); } export interface GpuGraphRenderOptions { gpuName: string; metric: GpuMetricKind; value: number | undefined; history: number[]; offline: boolean; } export function renderGpuGraph(opts: GpuGraphRenderOptions): string { const label = escapeXml(shorten(opts.gpuName)); if (opts.offline) { return ` !! OFFLINE ${label} `; } const color = opts.value === undefined ? "#666666" : severityColor(opts.metric, opts.value); const valueText = opts.value === undefined ? "--" : `${Math.round(opts.value)}${unit(opts.metric)}`; const points = chartPoints(opts.history, opts.metric); const line = points.length > 1 ? `` : ""; const area = points.length > 1 ? `` : ""; return ` ${area} ${line} ${valueText} ${METRIC_LABEL[opts.metric]} · ${label} `; } function unit(metric: GpuMetricKind): string { return metric === "temperature" ? "°C" : metric === "power" ? "W" : "%"; } function severityColor(metric: GpuMetricKind, value: number): string { if (metric === "temperature") return value >= 80 ? "#e0453a" : value >= 60 ? "#d9a02a" : "#3fae5a"; if (metric === "power") return value >= 300 ? "#e0453a" : value >= 150 ? "#d9a02a" : "#3fae5a"; return value >= 85 ? "#e0453a" : value >= 50 ? "#d9a02a" : "#3fae5a"; } function chartPoints(history: number[], metric: GpuMetricKind): { x: number; y: number }[] { const n = history.length; if (n === 0) return []; const left = 3; const right = 69; const top = 20; const bottom = 55; const yMax = yScaleMax(metric, history); return history.map((v, i) => { const x = n === 1 ? (left + right) / 2 : left + ((right - left) * i) / (n - 1); const clamped = Math.max(0, Math.min(v, yMax)); const y = bottom - ((clamped - 0) / (yMax - 0 || 1)) * (bottom - top); return { x: round1(x), y: round1(y) }; }); } function yScaleMax(metric: GpuMetricKind, history: number[]): number { if (metric !== "power") return 100; const maxValue = Math.max(...history); if (!isFinite(maxValue) || maxValue <= 0) return 100; return niceCeil(maxValue * 1.1); } function niceCeil(value: number): number { const magnitude = Math.pow(10, Math.floor(Math.log10(value))); for (const m of [1, 2, 2.5, 5, 10]) { if (m * magnitude >= value) return m * magnitude; } return value; } function round1(value: number): number { return Math.round(value * 10) / 10; } function centerText(text: string, y: number, size: number, weight: "normal" | "bold", fill: string, opacity?: number): string { return `${text}`; } function frame(bg: string, parts: string[]): string { return ` ${parts.join("\n ")} `; } ``` - [ ] **Step 4: Run test to verify it passes** Run: `npm test` Expected: PASS. - [ ] **Step 5: Commit** ```bash git add -A git commit -m "feat: add SVG key renderers for inflight and GPU graph actions" ``` --- ### Task 9: `runtime.ts` + `datasources.ts` — shared state and PI data sourcing **Files:** - Create: `src/lib/runtime.ts` - Create: `src/lib/datasources.ts` **Interfaces:** - Consumes: `EventFeed`, `InflightTracker`, `MetricsPoller`, `LlamaSwapConfig`, `fetchModels`, `fetchMetrics`, `gpuInfos`, `parseGpuMetrics`, `cfgFromSettings`, `CfgSettings`. - Produces: - `runtime: { tracker: InflightTracker; offline: boolean; ensureConnections(cfg): void; pollerInstance?: MetricsPoller; subscribe(listener): () => void }` - `registerDataSources(): void` — wires `streamDeck.ui.onSendToPlugin` to answer `models` and `gpus` datasource requests. - [ ] **Step 1: Implement `src/lib/runtime.ts`** ```ts import { EventFeed } from "./event-feed"; import { InflightTracker } from "./inflight-tracker"; import { MetricsPoller } from "./metrics-poller"; import { type LlamaSwapConfig } from "./util"; class Runtime { readonly tracker = new InflightTracker(); offline = true; private feed?: EventFeed; private poller?: MetricsPoller; private listeners = new Set<() => void>(); ensureConnections(cfg: LlamaSwapConfig): void { if (!this.feed) { this.feed = new EventFeed(cfg, (ev) => { this.tracker.apply(ev); this.emit(); }); this.feed.setStatusHandler((connected) => { this.offline = !connected; this.emit(); }); this.feed.start(); } if (!this.poller) { this.poller = new MetricsPoller(cfg); this.poller.on(() => this.emit()); this.poller.start(); } } get pollerInstance(): MetricsPoller | undefined { return this.poller; } subscribe(listener: () => void): () => void { this.listeners.add(listener); return () => { this.listeners.delete(listener); }; } private emit(): void { for (const listener of this.listeners) listener(); } } export const runtime = new Runtime(); ``` - [ ] **Step 2: Implement `src/lib/datasources.ts`** ```ts import streamDeck from "@elgato/streamdeck"; import { fetchMetrics, fetchModels } from "./llamaswap"; import { gpuInfos, parseGpuMetrics } from "./metrics-parser"; import { cfgFromSettings, type CfgSettings } from "./util"; interface DataSourceItem { label: string; value: string; } export function registerDataSources(): void { streamDeck.ui.onSendToPlugin(async (ev) => { const request = ev.payload as { event?: string } | undefined; const event = request?.event; if (!event) return; const settings = await ev.action.getSettings(); if (event === "models") { let items: DataSourceItem[] = []; try { const models = await fetchModels(cfgFromSettings(settings)); items = models.map((m) => ({ label: `${m.id} (${m.status})`, value: m.id })); } catch { items = []; } await streamDeck.ui.sendToPropertyInspector({ event, items }); return; } if (event === "gpus") { let items: DataSourceItem[] = []; try { const text = await fetchMetrics(cfgFromSettings(settings)); const gpus = gpuInfos(parseGpuMetrics(text)); items = [ { label: "All GPUs", value: "all" }, ...gpus.map((g) => ({ label: `GPU ${g.id} · ${g.name}`, value: g.id })), ]; } catch { items = [{ label: "All GPUs", value: "all" }]; } await streamDeck.ui.sendToPropertyInspector({ event, items }); } }); } ``` - [ ] **Step 3: Verify it typechecks** Run: `npm run build` Expected: build succeeds. (Note: `import streamDeck` default export + `streamDeck.ui.onSendToPlugin` — if the compiler reports a different method name, consult `node_modules/@elgato/streamdeck/dist/plugin/ui.d.ts` and adjust; the public API is `onSendToPlugin` / `sendToPropertyInspector`.) - [ ] **Step 4: Commit** ```bash git add -A git commit -m "feat: add shared runtime and PI datasource handlers" ``` --- ### Task 10: Actions — `InflightMonitor` and `GpuGraph` **Files:** - Create: `src/actions/inflight-monitor.ts` - Create: `src/actions/gpu-graph.ts` **Interfaces:** - Consumes: `runtime`, `renderInflight`/`renderGpuGraph`/`svgDataUrl`, `cfgFromSettings`/`CfgSettings`, `GpuMetricKind`, `streamDeck` (for `system.openUrl`). - Produces: two `SingletonAction` classes registered by UUIDs `com.bryce.llamawatch.inflight` and `com.bryce.llamawatch.gpu`. - [ ] **Step 1: Implement `src/actions/inflight-monitor.ts`** ```ts import streamDeck, { action, type DidReceiveSettingsEvent, type KeyAction, type KeyDownEvent, SingletonAction, type WillAppearEvent, } from "@elgato/streamdeck"; import { renderInflight, svgDataUrl } from "../lib/render"; import { runtime } from "../lib/runtime"; import { cfgFromSettings, type CfgSettings } from "../lib/util"; type InflightSettings = CfgSettings & { modelId?: string; }; @action({ UUID: "com.bryce.llamawatch.inflight" }) export class InflightMonitor extends SingletonAction { private settings: InflightSettings = {}; private action?: KeyAction; private unsubscribe?: () => void; private pulseTimer?: ReturnType; private pulse = false; override onWillAppear(ev: WillAppearEvent): void { if (!ev.action.isKey()) return; this.settings = ev.payload.settings; this.action = ev.action; runtime.ensureConnections(cfgFromSettings(this.settings)); this.unsubscribe = runtime.subscribe(() => this.render()); this.render(); } override onWillDisappear(): void { this.unsubscribe?.(); this.unsubscribe = undefined; this.action = undefined; this.clearPulse(); } override onDidReceiveSettings(ev: DidReceiveSettingsEvent): void { if (!ev.action.isKey()) return; this.settings = ev.payload.settings; this.action = ev.action; runtime.ensureConnections(cfgFromSettings(this.settings)); this.render(); } override onKeyDown(ev: KeyDownEvent): void { void streamDeck.system.openUrl(`${cfgFromSettings(ev.payload.settings).baseUrl}/ui`); } private render(): void { const action = this.action; if (!action) return; const modelId = this.settings.modelId ?? ""; const state = runtime.tracker.state(modelId); const count = runtime.tracker.count(modelId); const offline = runtime.offline || modelId.length === 0; void action.setImage( svgDataUrl( renderInflight({ modelName: modelId.length > 0 ? modelId : "unset", state, count, offline, pulse: this.pulse, }), ), ); if (state === "ready" && count > 0 && !this.pulseTimer) { this.pulseTimer = setInterval(() => { this.pulse = !this.pulse; this.render(); }, 500); } else if (!(state === "ready" && count > 0) && this.pulseTimer) { this.clearPulse(); } } private clearPulse(): void { if (this.pulseTimer) clearInterval(this.pulseTimer); this.pulseTimer = undefined; this.pulse = false; } } ``` - [ ] **Step 2: Implement `src/actions/gpu-graph.ts`** ```ts import streamDeck, { action, type DidReceiveSettingsEvent, type KeyAction, type KeyDownEvent, SingletonAction, type WillAppearEvent, } from "@elgato/streamdeck"; import { type GpuMetricKind } from "../lib/metrics-parser"; import { renderGpuGraph, svgDataUrl } from "../lib/render"; import { runtime } from "../lib/runtime"; import { cfgFromSettings, type CfgSettings } from "../lib/util"; type GpuSettings = CfgSettings & { gpuId?: string; metric?: GpuMetricKind; }; @action({ UUID: "com.bryce.llamawatch.gpu" }) export class GpuGraph extends SingletonAction { private settings: GpuSettings = {}; private action?: KeyAction; private unsubscribe?: () => void; private lastSvg?: string; override onWillAppear(ev: WillAppearEvent): void { if (!ev.action.isKey()) return; this.settings = ev.payload.settings; this.action = ev.action; runtime.ensureConnections(cfgFromSettings(this.settings)); this.unsubscribe = runtime.subscribe(() => this.render()); this.render(); } override onWillDisappear(): void { this.unsubscribe?.(); this.unsubscribe = undefined; this.action = undefined; } override onDidReceiveSettings(ev: DidReceiveSettingsEvent): void { if (!ev.action.isKey()) return; this.settings = ev.payload.settings; this.action = ev.action; this.render(); } override onKeyDown(ev: KeyDownEvent): void { void streamDeck.system.openUrl(`${cfgFromSettings(ev.payload.settings).baseUrl}/ui`); } private render(): void { const action = this.action; if (!action) return; const poller = runtime.pollerInstance; const gpuId = this.settings.gpuId ?? "all"; const metric = this.settings.metric ?? "util_percent"; const value = poller?.getValue(gpuId, metric); const history = poller?.getHistory(gpuId, metric) ?? []; const offline = !poller || poller.isOffline(); const gpuName = gpuId === "all" ? "ALL GPUS" : this.gpuLabel(gpuId); const svg = renderGpuGraph({ gpuName, metric, value, history, offline }); if (svg === this.lastSvg) return; this.lastSvg = svg; void action.setImage(svgDataUrl(svg)); } private gpuLabel(gpuId: string): string { const gpu = runtime.pollerInstance?.gpus().find((g) => g.id === gpuId); return gpu && gpu.name ? `GPU ${gpuId} · ${gpu.name}` : `GPU ${gpuId}`; } } ``` - [ ] **Step 3: Verify it typechecks** Run: `npm run build` Expected: build succeeds. - [ ] **Step 4: Commit** ```bash git add -A git commit -m "feat: implement inflight monitor and GPU graph actions" ``` --- ### Task 11: Entry point, manifest actions, property inspectors, assets **Files:** - Modify: `src/plugin.ts` - Modify: `com.bryce.llamawatch.sdPlugin/manifest.json` - Create: `com.bryce.llamawatch.sdPlugin/ui/inflight.html` - Create: `com.bryce.llamawatch.sdPlugin/ui/gpu.html` - Create: `com.bryce.llamawatch.sdPlugin/ui/sdpi-components.js` (downloaded, v4) - Create: `com.bryce.llamawatch.sdPlugin/imgs/plugin/icon.svg` (256×256 source) - Create: `com.bryce.llamawatch.sdPlugin/imgs/plugin/category-icon.svg` - Create: `com.bryce.llamawatch.sdPlugin/imgs/actions/inflight/icon.svg` (monochrome) - Create: `com.bryce.llamawatch.sdPlugin/imgs/actions/inflight/key.svg` - Create: `com.bryce.llamawatch.sdPlugin/imgs/actions/gpu/icon.svg` (monochrome) - Create: `com.bryce.llamawatch.sdPlugin/imgs/actions/gpu/key.svg` **Interfaces:** - Consumes: action classes and `registerDataSources`. - Produces: a fully manifest-defined plugin (both actions), compiled entry, PI pages wired to the `models`/`gpus` datasources, and generated PNG icons. - [ ] **Step 1: Replace `src/plugin.ts`** ```ts import streamDeck from "@elgato/streamdeck"; import { GpuGraph } from "./actions/gpu-graph"; import { InflightMonitor } from "./actions/inflight-monitor"; import { registerDataSources } from "./lib/datasources"; streamDeck.actions.registerAction(new InflightMonitor()); streamDeck.actions.registerAction(new GpuGraph()); registerDataSources(); streamDeck.connect(); ``` - [ ] **Step 2: Replace `com.bryce.llamawatch.sdPlugin/manifest.json`** ```json { "$schema": "https://schemas.elgato.com/streamdeck/plugins/manifest.json", "Name": "llama-watch", "Version": "1.0.0.0", "Author": "Bryce Zuccaro", "Actions": [ { "Name": "In-Flight Monitor", "UUID": "com.bryce.llamawatch.inflight", "Icon": "imgs/actions/inflight/icon", "Tooltip": "Shows whether a model has a request in flight", "PropertyInspectorPath": "ui/inflight.html", "Controllers": ["Keypad"], "States": [{ "Image": "imgs/actions/inflight/key", "ShowTitle": false }] }, { "Name": "GPU Graph", "UUID": "com.bryce.llamawatch.gpu", "Icon": "imgs/actions/gpu/icon", "Tooltip": "Live GPU metric graph", "PropertyInspectorPath": "ui/gpu.html", "Controllers": ["Keypad"], "States": [{ "Image": "imgs/actions/gpu/key", "ShowTitle": false }] } ], "Category": "llama-watch", "CategoryIcon": "imgs/plugin/category-icon", "CodePath": "bin/plugin.js", "Description": "Monitor llama-swap: in-flight requests per model and live GPU metric graphs.", "Icon": "imgs/plugin/icon", "SDKVersion": 2, "Software": { "MinimumVersion": "7.1" }, "OS": [{ "Platform": "mac", "MinimumVersion": "13" }], "Nodejs": { "Version": "24" }, "UUID": "com.bryce.llamawatch" } ``` - [ ] **Step 3: Download `sdpi-components.js` (v4) locally** ```bash curl -sL https://sdpi-components.dev/releases/v4/sdpi-components.js -o com.bryce.llamawatch.sdPlugin/ui/sdpi-components.js ``` Expected: file is non-empty and starts with a JS comment/IIFE. - [ ] **Step 4: Create `com.bryce.llamawatch.sdPlugin/ui/inflight.html`** ```html ``` - [ ] **Step 5: Create `com.bryce.llamawatch.sdPlugin/ui/gpu.html`** ```html ``` - [ ] **Step 6: Create the SVG assets** `com.bryce.llamawatch.sdPlugin/imgs/plugin/icon.svg` (256×256, graph glyph on dark rounded square): ```svg ``` `com.bryce.llamawatch.sdPlugin/imgs/plugin/category-icon.svg` (monochrome white, 56×56): ```svg ``` `com.bryce.llamawatch.sdPlugin/imgs/actions/inflight/icon.svg` (monochrome white, activity pulse): ```svg ``` `com.bryce.llamawatch.sdPlugin/imgs/actions/inflight/key.svg` (72×72 default key): ```svg ``` `com.bryce.llamawatch.sdPlugin/imgs/actions/gpu/icon.svg` (monochrome white, chip): ```svg ``` `com.bryce.llamawatch.sdPlugin/imgs/actions/gpu/key.svg` (72×72 default key): ```svg ``` - [ ] **Step 7: Generate the plugin PNG icon (256 + 512 @2x) with `sips`** ```bash cd com.bryce.llamawatch.sdPlugin/imgs/plugin sips -s format png icon.svg --out icon.png sips -z 512 512 icon.png --out icon@2x.png ``` Expected: `icon.png` is 256×256, `icon@2x.png` is 512×512. - [ ] **Step 8: Build and verify** Run: `npm run build` Expected: build succeeds and `com.bryce.llamawatch.sdPlugin/bin/plugin.js` exists. - [ ] **Step 9: Commit** ```bash git add -A git commit -m "feat: wire entry point, manifest, property inspectors, and assets" ``` --- ### Task 12: Validate, install, and manually verify on the device **Files:** - Create: `README.md` **Interfaces:** - Consumes: the built plugin folder. - Produces: `.streamDeckPlugin` package, plugin installed and verified live on the Stream Deck XL, README. - [ ] **Step 1: Validate the manifest** Run: ```bash streamdeck validate com.bryce.llamawatch.sdPlugin ``` Expected: validation passes (fix any reported issues, e.g. missing icon paths). - [ ] **Step 2: Enable developer mode (if needed) and install the plugin** Try the CLI link first: ```bash streamdeck dev streamdeck link com.bryce.llamawatch.sdPlugin ``` If `link` fails because the CLI cannot locate the app, copy the folder manually: ```bash cp -R com.bryce.llamawatch.sdPlugin "$HOME/Library/Application Support/com.elgato.StreamDeck/Plugins/" ``` (The Stream Deck app at `/Applications/Elgato Stream Deck.app` auto-loads plugins from that directory.) Then verify with `streamdeck list` or by restarting the app. - [ ] **Step 3: Verify the plugin loads** - Restart the Stream Deck app (or `streamdeck restart com.bryce.llamawatch`). - Confirm the action list contains "In-Flight Monitor" and "GPU Graph" under the "llama-watch" category. - Check `~/Library/Logs/StreamDeck` (or the plugin `logs/` dir) for errors if the actions don't appear. - [ ] **Step 4: Manual verification on the device (Stream Deck XL)** - Place an **In-Flight Monitor** action and pick model `DeepSeek-V4-Flash-0731`; confirm it renders green `IDLE`. While an OpenWebUI/chat request is running against that model, confirm it turns red `ACTIVE` and pulses. Press the key → browser opens `http://localhost:9292/ui`. - Place a **GPU Graph** action; GPU dropdown should list "All GPUs" plus the three GPUs (RTX PRO 6000 ×2, RTX 5090). Select GPU `2` + metric `Utilization %`; confirm a graph line appears and updates every ~5s. Select metric `Power draw`; confirm watts value + auto-scaled chart. Press → opens `/ui`. - Stop the llama-swap instance (or use a bogus base URL on a spare key) and confirm buttons show `OFFLINE`; restore and confirm auto-recovery. - [ ] **Step 5: Write `README.md`** ```markdown # llama-watch A Stream Deck plugin (macOS) that monitors a llama-swap instance. ## Actions - **In-Flight Monitor** — per-model color state: `OFF` / `LOADING` / `IDLE` / `ACTIVE` (pulsing red). Powered by the real-time `/api/events` SSE feed. - **GPU Graph** — live line chart of a GPU metric (utilization %, VRAM %, temperature, power draw, fan speed) for one GPU or all GPUs, sampled every 5 s. Pressing either key opens `http:///ui` in your browser. ## Install Double-click the packaged `.streamDeckPlugin`, or run: ```bash npm install npm run build streamdeck validate com.bryce.llamawatch.sdPlugin ``` ## Develop ```bash npm run watch # hot-reload while the Stream Deck app is running npm test # unit tests (node:test + tsx) ``` ## Configure Per-key settings: base URL (default `http://localhost:9292`), optional API key, and the model / GPU / metric to watch. The model and GPU dropdowns are populated live from the instance. ## Marketplace Assets are prepared for eventual submission (`imgs/plugin/icon.png` 256/512, action icons, category icon). The plugin only stores its settings locally; it reads GPU metrics and request status from the user's own llama-swap server and sends nothing elsewhere. ``` - [ ] **Step 6: Package the distributable** Run: ```bash streamdeck pack com.bryce.llamawatch.sdPlugin ``` Expected: `com.bryce.llamawatch.streamDeckPlugin` created. Double-clicking it installs the plugin. - [ ] **Step 7: Commit** ```bash git add -A git commit -m "docs: add README; verify plugin install and rendering on device" ``` --- ## Self-Review **Spec coverage:** - Feature 1 (in-flight per model) → Tasks 6, 7 (tracker + feed), Task 10 (action), Task 11 (PI model dropdown). ✔ - Feature 2 (GPU graphs, all five metrics + aggregates) → Tasks 3, 5 (parser/poller/aggregates), Task 8 (renderer), Task 10 (action), Task 11 (PI GPU+metric dropdowns). ✔ - 5s poll, 60-sample buffer → Task 5. ✔ - 144×72 rendering / SVG, DPR 2 → SVG viewBox 72 + `setImage`; device is Stream Deck XL (DPR 2), SVG scales crisply. ✔ - IDLE/ACTIVE color state, pulse → Task 8/10. ✔ - Press opens `/ui` → Task 10 (`streamDeck.system.openUrl`). ✔ - Error handling (offline, reconnect backoff, missed-poll skip, per-button isolation) → Task 5/7/9/10. ✔ - Marketplace-ready assets → Task 11/12. ✔ **Placeholder scan:** No TBD/TODO; all code blocks complete; no "similar to Task N" references; every interface symbol is defined in a task. **Type consistency:** `GpuMetricKind`, `FeedEvent`, `ModelRuntimeState`, `LlamaSwapConfig`, `cfgFromSettings`, `runtime`, `renderInflight`, `renderGpuGraph` names are identical across tasks. `KeyAction` used consistently for stored action refs; `streamDeck.system.openUrl` and `streamDeck.ui.onSendToPlugin`/`sendToPropertyInspector` match `@elgato/streamdeck@2.1.1` (verified against its `.d.ts`). No lint/typecheck config beyond the sample's tsconfig — `npm run build` is the type gate.