Files

T

bzuccaro 2b7764f96d Add Usage Stats + GPU Combinations implementation plan

2026-08-14 12:46:54 -06:00

35 KiB

Raw Blame History

Usage Stats + GPU Combinations Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Add a Usage-stats display mode to the In-Flight Monitor action (request/token totals + gen-speed P95 from /api/metrics/stats) and GPU combination support to the GPU Graph action.

Architecture: A shared runtime-owned StatsCache polls llama-swap's /api/metrics/stats for registered model keys (single interval, activity SSE events trigger throttled refreshes) and feeds a new renderUsage view in the In-Flight action. GPU combinations are aggregated in the GPU action via a pure combineSeries helper over the poller's existing per-GPU rings.

Tech Stack: TypeScript ESM, node:test+tsx, @elgato/streamdeck@2.1.1. Tests run with npm test (i.e. tsx --test). Build: npm run build (rollup). Typecheck: npx tsc --noEmit.

Global Constraints

No code comments (project convention).
Work committed on main (user-approved workflow; no remote).
feed.ts's FeedEvent type lives in src/lib/inflight-tracker.ts.
Default base URL http://localhost:9292; llama-swap /api/metrics/stats returns { total_requests, total_input_tokens, total_output_tokens, total_cache_tokens, prompt_histogram, gen_histogram }; gen_histogram.p95 is a tokens/sec percentile (not latency).
Key images are SVG strings via svgDataUrl; dark background #10131a.
Multi-key safety: per-key state lives in a Map keyed on ev.action.id.
npm test runs ALL test files via tsx --test; AggregateError style failures indicate a failing test file.

Task 1: Stats types, parsing, and fetching

Files:

Create: src/lib/stats.ts
Test: tests/stats.test.ts

Interfaces:

Produces:
- interface UsageStats { totalRequests: number; totalInputTokens: number; totalOutputTokens: number; genP95: number }
- parseStats(json: unknown): UsageStats | null
- fetchStats(cfg: LlamaSwapConfig, modelId: string): Promise<UsageStats | null>
Step 1: Write the failing test

Create tests/stats.test.ts:

import assert from "node:assert/strict";
import { test } from "node:test";
import { fetchStats, parseStats, type UsageStats } from "../src/lib/stats";

test("parseStats extracts totals and gen p95", () => {
  const stats = parseStats({
    total_requests: 1985,
    total_input_tokens: 312327705,
    total_output_tokens: 932710,
    total_cache_tokens: 308144640,
    gen_histogram: { p50: 341, p95: 378.86, p99: 392 },
  });
  assert.deepEqual(stats, { totalRequests: 1985, totalInputTokens: 312327705, totalOutputTokens: 932710, genP95: 378.86 });
});

test("parseStats defaults missing fields and rejects malformed input", () => {
  assert.deepEqual(parseStats({}), { totalRequests: 0, totalInputTokens: 0, totalOutputTokens: 0, genP95: 0 });
  assert.equal(parseStats({ total_requests: "nope" })!.totalRequests, 0);
  assert.equal(parseStats(null), null);
  assert.equal(parseStats("x"), null);
});

test("fetchStats calls /api/metrics/stats with a model query", async () => {
  const calls: string[] = [];
  const orig = globalThis.fetch;
  globalThis.fetch = (async (url: RequestInfo | URL) => {
    calls.push(String(url));
    return { ok: true, json: async () => ({ total_requests: 7 }) } as unknown as Response;
  }) as typeof fetch;
  try {
    await fetchStats({ baseUrl: "http://x" }, "DeepSeek-V4-Flash-0731");
    assert.equal(calls[0], "http://x/api/metrics/stats?model=DeepSeek-V4-Flash-0731");
  } finally {
    globalThis.fetch = orig;
  }
});

test("fetchStats omits the model query for 'all' and returns null on HTTP error", async () => {
  const calls: string[] = [];
  const orig = globalThis.fetch;
  globalThis.fetch = (async (url: RequestInfo | URL) => {
    calls.push(String(url));
    return { ok: false, status: 500 } as unknown as Response;
  }) as typeof fetch;
  try {
    const stats = await fetchStats({ baseUrl: "http://x" }, "all");
    assert.equal(stats, null);
    assert.equal(calls[0], "http://x/api/metrics/stats");
  } finally {
    globalThis.fetch = orig;
  }
});

Step 2: Run test to verify it fails

Run: npx tsx --test tests/stats.test.ts Expected: FAIL — Cannot find module '../src/lib/stats'.

Step 3: Write minimal implementation

Create src/lib/stats.ts:

import { type LlamaSwapConfig } from "./util";

export interface UsageStats {
  totalRequests: number;
  totalInputTokens: number;
  totalOutputTokens: number;
  genP95: number;
}

export function parseStats(json: unknown): UsageStats | null {
  if (!json || typeof json !== "object") return null;
  const obj = json as Record<string, unknown>;
  const genHist = (obj.gen_histogram ?? {}) as Record<string, unknown>;
  const num = (v: unknown): number => (typeof v === "number" && Number.isFinite(v) ? v : 0);
  return {
    totalRequests: num(obj.total_requests),
    totalInputTokens: num(obj.total_input_tokens),
    totalOutputTokens: num(obj.total_output_tokens),
    genP95: num(genHist.p95),
  };
}

export async function fetchStats(cfg: LlamaSwapConfig, modelId: string): Promise<UsageStats | null> {
  const query = modelId === "all" ? "" : `?model=${encodeURIComponent(modelId)}`;
  const headers: Record<string, string> = {};
  if (cfg.apiKey) headers["Authorization"] = `Bearer ${cfg.apiKey}`;
  const res = await fetch(`${cfg.baseUrl}/api/metrics/stats${query}`, {
    headers,
    signal: AbortSignal.timeout(5000),
  });
  if (!res.ok) throw new Error(`stats HTTP ${res.status}`);
  return parseStats(await res.json());
}

Step 4: Run test to verify it passes

Run: npx tsx --test tests/stats.test.ts Expected: PASS (4 tests).

Step 5: Commit

git add src/lib/stats.ts tests/stats.test.ts
git commit -m "feat: add usage stats parsing and fetching from /api/metrics/stats"

Task 2: StatsCache

Files:

Create: src/lib/stats-cache.ts
Test: tests/stats-cache.test.ts

Interfaces:

Consumes: fetchStats, UsageStats from ./stats; LlamaSwapConfig from ./util.
Produces:
- class StatsCache
- constructor(cfg: LlamaSwapConfig, fetchFn?: FetchFn, pollMs?: number, throttleMs?: number) where FetchFn = (cfg: LlamaSwapConfig, modelId: string) => Promise<UsageStats | null>
- register(key: string): void
- unregister(key: string): void
- get(key: string): UsageStats | undefined
- scheduleRefresh(): void — throttled (trailing, min throttleMs between polls)
- refresh(): Promise<void>
- setConfig(cfg: LlamaSwapConfig): void
- onChange(listener: () => void): () => void
Step 1: Write the failing test

Create tests/stats-cache.test.ts:

import assert from "node:assert/strict";
import { test } from "node:test";
import { StatsCache, type FetchFn } from "../src/lib/stats-cache";
import { type LlamaSwapConfig } from "../src/lib/util";

const cfg: LlamaSwapConfig = { baseUrl: "http://x" };

function stubFetch(result: unknown, counter: { count: number }): FetchFn {
  return async () => {
    counter.count++;
    return result as never;
  };
}

async function flush(): Promise<void> {
  await new Promise((r) => setTimeout(r, 0));
}

test("register polls once immediately and caches the value", async () => {
  const calls = { count: 0 };
  const cache = new StatsCache(cfg, stubFetch({ totalRequests: 3, totalInputTokens: 1, totalOutputTokens: 2, genP95: 4 }, calls), 1000, 30);
  cache.register("all");
  await flush();
  assert.equal(calls.count, 1);
  assert.equal(cache.get("all")!.totalRequests, 3);
});

test("multiple keys are each polled", async () => {
  const keys: string[] = [];
  const cache = new StatsCache(
    cfg,
    async (_c, key) => {
      keys.push(key);
      return { totalRequests: 1, totalInputTokens: 1, totalOutputTokens: 1, genP95: 1 };
    },
    1000,
    30,
  );
  cache.register("a");
  cache.register("b");
  await flush();
  assert.deepEqual(keys.sort(), ["a", "b"]);
});

test("unregister stops the interval and clears the value", async () => {
  const calls = { count: 0 };
  const cache = new StatsCache(cfg, stubFetch({ totalRequests: 1, totalInputTokens: 1, totalOutputTokens: 1, genP95: 1 }, calls), 10, 30);
  cache.register("a");
  await flush();
  assert.equal(calls.count, 1);
  cache.unregister("a");
  await new Promise((r) => setTimeout(r, 40));
  assert.equal(cache.get("a"), undefined);
  assert.equal(calls.count, 1);
});

test("scheduleRefresh throttles to one poll per window", async () => {
  const calls = { count: 0 };
  const cache = new StatsCache(cfg, stubFetch({ totalRequests: 1, totalInputTokens: 1, totalOutputTokens: 1, genP95: 1 }, calls), 10000, 30);
  cache.register("a");
  await flush();
  const before = calls.count;
  cache.scheduleRefresh();
  cache.scheduleRefresh();
  cache.scheduleRefresh();
  await flush();
  assert.equal(calls.count, before);
  await new Promise((r) => setTimeout(r, 60));
  assert.equal(calls.count, before + 1);
  cache.unregister("a");
});

test("fetch failure keeps the last-known value", async () => {
  const fail: FetchFn = async () => {
    throw new Error("boom");
  };
  const cache = new StatsCache(cfg, fail, 10000, 30);
  cache.register("a");
  await flush();
  assert.equal(cache.get("a"), undefined);
  cache.unregister("a");
});

Step 2: Run test to verify it fails

Run: npx tsx --test tests/stats-cache.test.ts Expected: FAIL — Cannot find module '../src/lib/stats-cache'.

Step 3: Write minimal implementation

Create src/lib/stats-cache.ts:

import { fetchStats, type UsageStats } from "./stats";
import { type LlamaSwapConfig } from "./util";

const POLL_MS = 5000;
const ACTIVITY_THROTTLE_MS = 2000;

export type FetchFn = (cfg: LlamaSwapConfig, modelId: string) => Promise<UsageStats | null>;

export class StatsCache {
  private keys = new Set<string>();
  private values = new Map<string, UsageStats | undefined>();
  private listeners = new Set<() => void>();
  private timer?: ReturnType<typeof setInterval>;
  private throttleTimer?: ReturnType<typeof setTimeout>;
  private lastRefresh = 0;

  constructor(
    private cfg: LlamaSwapConfig,
    private fetchFn: FetchFn = fetchStats,
    private pollMs = POLL_MS,
    private throttleMs = ACTIVITY_THROTTLE_MS,
  ) {}

  setConfig(cfg: LlamaSwapConfig): void {
    this.cfg = cfg;
    this.values.clear();
  }

  register(key: string): void {
    if (this.keys.has(key)) return;
    this.keys.add(key);
    if (!this.timer) {
      void this.refresh();
      this.timer = setInterval(() => void this.refresh(), this.pollMs);
    }
  }

  unregister(key: string): void {
    this.keys.delete(key);
    this.values.delete(key);
    if (this.keys.size === 0 && this.timer) {
      clearInterval(this.timer);
      this.timer = undefined;
    }
  }

  get(key: string): UsageStats | undefined {
    return this.values.get(key);
  }

  scheduleRefresh(): void {
    const now = Date.now();
    const wait = Math.max(0, this.throttleMs - (now - this.lastRefresh));
    if (this.throttleTimer) clearTimeout(this.throttleTimer);
    this.throttleTimer = setTimeout(() => {
      this.throttleTimer = undefined;
      void this.refresh();
    }, wait);
  }

  async refresh(): Promise<void> {
    this.lastRefresh = Date.now();
    for (const key of this.keys) {
      try {
        const stats = await this.fetchFn(this.cfg, key);
        if (stats) this.values.set(key, stats);
      } catch {
      }
    }
    this.emit();
  }

  onChange(listener: () => void): () => void {
    this.listeners.add(listener);
    return () => {
      this.listeners.delete(listener);
    };
  }

  private emit(): void {
    for (const listener of this.listeners) listener();
  }
}

Step 4: Run test to verify it passes

Run: npx tsx --test tests/stats-cache.test.ts Expected: PASS (5 tests).

Step 5: Commit

git add src/lib/stats-cache.ts tests/stats-cache.test.ts
git commit -m "feat: add StatsCache for shared per-model usage polling"

Task 3: Decode `activity` SSE events + tracker total

Files:

Modify: src/lib/inflight-tracker.ts, src/lib/event-feed.ts
Test: tests/event-feed.test.ts, tests/inflight-tracker.test.ts

Interfaces:

Consumes: existing FeedEvent in inflight-tracker.ts.
Produces:
- FeedEvent gains | { type: "activity"; id: number }
- InflightTracker.total(): number
- decodeEvent handles {"type":"activity","data":"{\"id\":817}"}.
Step 1: Write the failing tests

Append to tests/event-feed.test.ts:

test("decodeEvent parses an activity event with an id", () => {
  const msg: SseMessage = {
    event: "message",
    data: JSON.stringify({ type: "activity", data: JSON.stringify({ id: 817 }) }),
  };
  const ev = decodeEvent(msg);
  assert.deepEqual(ev, { type: "activity", id: 817 });
});

Append to tests/inflight-tracker.test.ts:

test("total sums in-flight requests across all models", () => {
  const tracker = new InflightTracker();
  assert.equal(tracker.total(), 0);
  tracker.apply({ type: "inflight", operation: "add", requests: [{ model: "A", id: "1" }, { model: "B", id: "2" }] });
  tracker.apply({ type: "inflight", operation: "add", requests: [{ model: "A", id: "3" }] });
  assert.equal(tracker.total(), 3);
  tracker.apply({ type: "inflight", operation: "remove", id: "1" });
  assert.equal(tracker.total(), 2);
});

Step 2: Run tests to verify they fail

Run: npx tsx --test tests/event-feed.test.ts tests/inflight-tracker.test.ts Expected: FAIL — activity event decodes to null; tracker.total is not a function.

Step 3: Write minimal implementation

In src/lib/inflight-tracker.ts, change the FeedEvent union to:

export type FeedEvent =
  | { type: "inflight"; operation: "snapshot" | "add"; requests: InflightRequest[] }
  | { type: "inflight"; operation: "remove"; id: string }
  | { type: "activity"; id: number }
  | { type: "modelStatus"; models: ModelState[] };

and add a method to InflightTracker (after count):

  total(): number {
    return this.requests.size;
  }

In src/lib/event-feed.ts, inside decodeEvent, before the if (outer.type === "inflight") block, add:

    if (outer.type === "activity") {
      const id = typeof inner.id === "number" ? inner.id : Number.NaN;
      if (Number.isFinite(id)) return { type: "activity", id };
      return null;
    }

Step 4: Run tests to verify they pass

Run: npx tsx --test tests/event-feed.test.ts tests/inflight-tracker.test.ts Expected: PASS.

Step 5: Commit

git add src/lib/inflight-tracker.ts src/lib/event-feed.ts tests/event-feed.test.ts tests/inflight-tracker.test.ts
git commit -m "feat: decode activity SSE events and expose tracker total"

Task 4: Runtime integration

Files:

Modify: src/lib/runtime.ts

Interfaces:

Consumes: StatsCache from ./stats-cache; UsageStats from ./stats; FeedEvent (already imported via tracker).
Produces (public API used by the In-Flight action):
- runtime.getStats(modelId: string): UsageStats | undefined
- runtime.watchStats(modelId: string): () => void
Step 1: Modify src/lib/runtime.ts

Replace the whole file body after the class opening as shown:

import { EventFeed } from "./event-feed";
import { InflightTracker } from "./inflight-tracker";
import { MetricsPoller } from "./metrics-poller";
import { StatsCache } from "./stats-cache";
import { type UsageStats } from "./stats";
import { type LlamaSwapConfig } from "./util";

class Runtime {
  readonly tracker = new InflightTracker();
  offline = true;
  private cfg?: LlamaSwapConfig;
  private feed?: EventFeed;
  private poller?: MetricsPoller;
  private statsCache?: StatsCache;
  private statsUnsub?: () => void;
  private listeners = new Set<() => void>();

  ensureConnections(cfg: LlamaSwapConfig): void {
    const changed = !this.cfg || this.cfg.baseUrl !== cfg.baseUrl || this.cfg.apiKey !== cfg.apiKey;
    if (changed) {
      this.feed?.stop();
      this.poller?.stop();
      this.feed = undefined;
      this.poller = undefined;
      this.statsUnsub?.();
      this.statsCache = undefined;
      this.statsUnsub = undefined;
      this.cfg = cfg;
    }
    if (!this.feed) {
      this.feed = new EventFeed(cfg, (ev) => {
        this.tracker.apply(ev);
        if (ev.type === "activity") this.statsCache?.scheduleRefresh();
        this.emit();
      });
      this.feed.setStatusHandler((connected) => {
        this.offline = !connected;
        this.emit();
      });
      this.feed.start();
    }
    if (!this.poller) {
      this.poller = new MetricsPoller(cfg);
      this.poller.on(() => this.emit());
      this.poller.start();
    }
    if (!this.statsCache) {
      this.statsCache = new StatsCache(cfg);
      this.statsUnsub = this.statsCache.onChange(() => this.emit());
    }
  }

  get pollerInstance(): MetricsPoller | undefined {
    return this.poller;
  }

  getStats(modelId: string): UsageStats | undefined {
    return this.statsCache?.get(modelId);
  }

  watchStats(modelId: string): () => void {
    this.statsCache?.register(modelId);
    return () => this.statsCache?.unregister(modelId);
  }

  subscribe(listener: () => void): () => void {
    this.listeners.add(listener);
    return () => {
      this.listeners.delete(listener);
    };
  }

  private emit(): void {
    for (const listener of this.listeners) listener();
  }
}

export const runtime = new Runtime();

Step 2: Typecheck and run the full suite

Run: npx tsc --noEmit && npm test Expected: TYPECHECK OK equivalent (no errors) and all tests pass.

Step 3: Commit

git add src/lib/runtime.ts
git commit -m "feat: expose usage stats via shared runtime StatsCache"

Task 5: `renderUsage` in render.ts

Files:

Modify: src/lib/render.ts
Test: tests/render.test.ts

Interfaces:

Consumes: UsageStats from ./stats.
Produces:
- interface UsageRenderOptions { modelName: string; stats?: UsageStats; primaryStat: "requests" | "input_tokens" | "output_tokens" | "gen_p95"; offline: boolean }
- renderUsage(opts: UsageRenderOptions): string
- formatCompact(n: number): string (exported for tests)
Step 1: Write the failing tests

Append to tests/render.test.ts:

import { renderGpuGraph, renderInflight, renderUsage, svgDataUrl, formatCompact } from "../src/lib/render";

(replace the existing import line) and append:

test("formatCompact renders integers, k, and M", () => {
  assert.equal(formatCompact(0), "0");
  assert.equal(formatCompact(999), "999");
  assert.equal(formatCompact(52100), "52.1k");
  assert.equal(formatCompact(1200000), "1.2M");
});

test("renderUsage shows the primary stat big and the rest small", () => {
  const svg = renderUsage({
    modelName: "DeepSeek-V4-Flash-0731",
    stats: { totalRequests: 1950, totalInputTokens: 312177540, totalOutputTokens: 889193, genP95: 378.86 },
    primaryStat: "gen_p95",
    offline: false,
  });
  assert.match(svg, />379</);
  assert.match(svg, /REQ/);
  assert.match(svg, /IN/);
  assert.match(svg, /OUT/);
  assert.match(svg, /312.2M/);
  assert.match(svg, /#10131a/);
});

test("renderUsage renders -- when stats are unknown and OFFLINE when offline", () => {
  const none = renderUsage({ modelName: "A", stats: undefined, primaryStat: "requests", offline: false });
  assert.match(none, />--</);
  const off = renderUsage({ modelName: "A", stats: undefined, primaryStat: "requests", offline: true });
  assert.match(off, /OFFLINE/);
});

test("renderUsage labels the aggregate view ALL MODELS", () => {
  const svg = renderUsage({ modelName: "all", stats: { totalRequests: 1, totalInputTokens: 1, totalOutputTokens: 1, genP95: 1 }, primaryStat: "requests", offline: false });
  assert.match(svg, /ALL MODELS/);
});

Step 2: Run test to verify it fails

Run: npx tsx --test tests/render.test.ts Expected: FAIL — Cannot find name 'renderUsage' / formatCompact.

Step 3: Write minimal implementation

In src/lib/render.ts, add an import of UsageStats and append the following at the end of the file (before helper functions or after — helpers can be placed after renderUsage):

export function formatCompact(n: number): string {
  if (!Number.isFinite(n)) return "--";
  if (n < 1000) return `${Math.round(n)}`;
  if (n < 1_000_000) return `${(n / 1000).toFixed(1)}k`;
  return `${(n / 1_000_000).toFixed(1)}M`;
}

export interface UsageRenderOptions {
  modelName: string;
  stats?: UsageStats;
  primaryStat: "requests" | "input_tokens" | "output_tokens" | "gen_p95";
  offline: boolean;
}

export function renderUsage(opts: UsageRenderOptions): string {
  const name = escapeXml(shorten(opts.modelName === "all" ? "ALL MODELS" : opts.modelName));

  if (opts.offline) {
    return frame("#10131a", [
      centerText("!!", 34, 20, "bold", "#e0e0e0"),
      centerText("OFFLINE", 52, 9, "normal", "#bdbdbd"),
      centerText(name, 64, 7, "normal", "#ffffff"),
    ]);
  }

  const s = opts.stats;
  const big = s ? formatCompact(primaryValue(s, opts.primaryStat)) : "--";
  const rows = usageRows(s, opts.primaryStat);
  const parts: string[] = [
    centerText(name, 10, 7, "normal", "#ffffff"),
    centerText(big, 36, 24, "bold", "#ffffff"),
  ];
  for (const row of rows) {
    parts.push(
      `<text x="4" y="${row.y}" font-family="Arial,sans-serif" font-size="7" fill="#8b93a5">${row.label}</text>`,
      `<text x="68" y="${row.y}" text-anchor="end" font-family="Arial,sans-serif" font-size="7" fill="#ffffff">${row.value}</text>`,
    );
  }
  return frame("#10131a", parts);
}

function primaryValue(s: UsageStats, primary: "requests" | "input_tokens" | "output_tokens" | "gen_p95"): number {
  switch (primary) {
    case "requests":
      return s.totalRequests;
    case "input_tokens":
      return s.totalInputTokens;
    case "output_tokens":
      return s.totalOutputTokens;
    case "gen_p95":
      return s.genP95;
  }
}

function usageRows(
  s: UsageStats | undefined,
  primary: "requests" | "input_tokens" | "output_tokens" | "gen_p95",
): { label: string; value: string; y: number }[] {
  const items: { label: string; value: string }[] = [];
  if (primary !== "requests") items.push({ label: "REQ", value: s ? formatCompact(s.totalRequests) : "--" });
  if (primary !== "input_tokens") items.push({ label: "IN", value: s ? formatCompact(s.totalInputTokens) : "--" });
  if (primary !== "output_tokens") items.push({ label: "OUT", value: s ? formatCompact(s.totalOutputTokens) : "--" });
  if (primary !== "gen_p95") items.push({ label: "P95", value: s ? `${Math.round(s.genP95)} t/s` : "--" });
  return items.map((it, i) => ({ ...it, y: 52 + i * 7 }));
}

Step 4: Run test to verify it passes

Run: npx tsx --test tests/render.test.ts Expected: PASS.

Step 5: Commit

git add src/lib/render.ts tests/render.test.ts
git commit -m "feat: add usage stats key renderer"

Task 6: In-Flight action display modes + property inspector

Files:

Modify: src/actions/inflight-monitor.ts, com.bryce.llamawatch.sdPlugin/ui/inflight.html, src/lib/datasources.ts

Interfaces:

Consumes: renderUsage, renderInflight, formatCompact (render); runtime.getStats/watchStats; InflightTracker.total().
Produces:
- InflightSettings gains display?: "count" | "usage" and primaryStat?: "requests" | "input_tokens" | "output_tokens" | "gen_p95".
- InflightState gains unwatchStats?: () => void.
- Models datasource gains an all option.
Step 1: Update the property inspector

In com.bryce.llamawatch.sdPlugin/ui/inflight.html, after the API Key item, add:

    <sdpi-item label="Display">
      <sdpi-select setting="display" default="count" placeholder="Select a display mode">
        <option value="count">Request count + activity</option>
        <option value="usage">Usage stats</option>
      </sdpi-select>
    </sdpi-item>
    <sdpi-item label="Primary stat (usage view)">
      <sdpi-select setting="primaryStat" default="requests" placeholder="Select a primary stat">
        <option value="requests">Requests</option>
        <option value="input_tokens">Processed tokens</option>
        <option value="output_tokens">Generated tokens</option>
        <option value="gen_p95">Generation speed P95</option>
      </sdpi-select>
    </sdpi-item>

Step 2: Update the models datasource

In src/lib/datasources.ts, inside the event === "models" branch, change the assignment to prepend the aggregate option:

          items = [{ label: "All models", value: "all" }, ...models.map((m) => ({ label: `${m.id} (${m.status})`, value: m.id }))];

Step 3: Update the In-Flight action

In src/actions/inflight-monitor.ts:

Add to the settings type:

type InflightSettings = CfgSettings & {
  modelId?: string;
  display?: "count" | "usage";
  primaryStat?: "requests" | "input_tokens" | "output_tokens" | "gen_p95";
};

Add to InflightState:

  unwatchStats?: () => void;

In onWillAppear, after state.settings = ev.payload.settings;, replace the sampler setup block:

    state.unsubscribe = runtime.subscribe(() => this.render(state));
    if (this.displayOf(state) === "usage") {
      state.unwatchStats = runtime.watchStats(this.modelKeyOf(state));
    } else {
      state.sampler = setInterval(() => this.sample(state), 1000);
    }
    this.render(state);

In onWillDisappear, add cleanup after state.unsubscribe?.();:

    state.unwatchStats?.();
    state.unwatchStats = undefined;

In onDidReceiveSettings, after state.settings = ev.payload.settings;, add re-registration:

    state.unwatchStats?.();
    state.unwatchStats = undefined;
    if (this.displayOf(state) === "usage") {
      state.unwatchStats = runtime.watchStats(this.modelKeyOf(state));
    } else if (!state.sampler) {
      state.sampler = setInterval(() => this.sample(state), 1000);
    }

Replace render with a mode-switching version and add helpers:

  private displayOf(state: InflightState): "count" | "usage" {
    return state.settings.display ?? "count";
  }

  private modelKeyOf(state: InflightState): string {
    const modelId = state.settings.modelId ?? "";
    return modelId === "" ? "all" : modelId;
  }

  private primaryStatOf(state: InflightState): "requests" | "input_tokens" | "output_tokens" | "gen_p95" {
    return state.settings.primaryStat ?? "requests";
  }

  private render(state: InflightState): void {
    const action = state.action;
    if (!action) return;
    if (this.displayOf(state) === "usage") {
      const modelKey = this.modelKeyOf(state);
      const offline = runtime.offline;
      void action.setImage(
        svgDataUrl(
          renderUsage({
            modelName: modelKey,
            stats: runtime.getStats(modelKey),
            primaryStat: this.primaryStatOf(state),
            offline,
          }),
        ),
      );
      return;
    }

    const modelId = state.settings.modelId ?? "";
    const trackerState = modelId === "all" ? "ready" : runtime.tracker.state(modelId);
    const count = modelId === "all" ? runtime.tracker.total() : runtime.tracker.count(modelId);
    const offline = runtime.offline && modelId.length > 0;
    const modelName = modelId === "" ? "unset" : modelId;
    void action.setImage(
      svgDataUrl(
        renderInflight({
          modelName,
          state: trackerState,
          count,
          offline,
          history: state.history,
        }),
      ),
    );
  }

  private sample(state: InflightState): void {
    const modelId = state.settings.modelId ?? "";
    state.history.push(modelId === "all" ? runtime.tracker.total() : runtime.tracker.count(modelId));
    if (state.history.length > HISTORY_LIMIT) state.history.shift();
    this.render(state);
  }

Note: for modelId === "all" in count mode, trackerState is forced to "ready" so the key renders the total number rather than NO MODEL/OFF. The label comes from renderInflight's name (ALL MODELS won't match — see step 4).

Step 4: Ensure the count-mode "all" label renders correctly

In src/lib/render.ts, renderInflight shortens the model name. For modelName === "all" pass the display name through shorten. Update the line in renderInflight:

  const name = escapeXml(shorten(opts.modelName === "all" ? "ALL MODELS" : opts.modelName));

(one-line change; mirrors the renderUsage approach).

Step 5: Typecheck, build, and run the suite

Run: npx tsc --noEmit && npm run build && npm test Expected: no type errors, build succeeds, all tests pass.

Step 6: Commit

git add src/actions/inflight-monitor.ts src/lib/render.ts src/lib/datasources.ts com.bryce.llamawatch.sdPlugin/ui/inflight.html
git commit -m "feat: add usage-stats display mode to In-Flight Monitor action"

Task 7: GPU `combineSeries` helper

Files:

Modify: src/lib/metrics-poller.ts
Test: tests/metrics-poller.test.ts

Interfaces:

Consumes: AGGREGATE (already defined in metrics-poller.ts).
Produces: combineSeries(histories: number[][], kind: GpuMetricKind): { value?: number; history: number[] } (exported).
Step 1: Write the failing test

Append to tests/metrics-poller.test.ts:

import { combineSeries, MetricsPoller } from "../src/lib/metrics-poller";

(update the existing import) and append:

test("combineSeries averages util and sums power", () => {
  const util = combineSeries(
    [
      [10, 20, 30],
      [20, 40, 60],
    ],
    "util_percent",
  );
  assert.deepEqual(util, { value: 45, history: [15, 30, 45] });

  const power = combineSeries(
    [
      [100, 200],
      [150, 250],
    ],
    "power",
  );
  assert.deepEqual(power, { value: 450, history: [250, 450] });
});

test("combineSeries takes the max temperature and skips gaps", () => {
  const temp = combineSeries([[50, 70], [60]], "temperature");
  assert.deepEqual(temp, { value: 70, history: [60, 70] });

  const empty = combineSeries([[], []], "temperature");
  assert.deepEqual(empty, { value: undefined, history: [] });
});

Step 2: Run test to verify it fails

Run: npx tsx --test tests/metrics-poller.test.ts Expected: FAIL — Cannot find name 'combineSeries'.

Step 3: Write minimal implementation

In src/lib/metrics-poller.ts, after the AGGREGATE const, add and export:

export function combineSeries(histories: number[][], kind: GpuMetricKind): { value?: number; history: number[] } {
  const n = Math.max(...histories.map((h) => h.length), 0);
  if (n === 0) return { value: undefined, history: [] };
  const history: number[] = [];
  for (let i = 0; i < n; i++) {
    const at = histories.map((h) => h[h.length - n + i]).filter((v): v is number => v !== undefined);
    if (at.length === 0) continue;
    history.push(AGGREGATE[kind](at));
  }
  return { value: history.length > 0 ? history[history.length - 1] : undefined, history };
}

Step 4: Run test to verify it passes

Run: npx tsx --test tests/metrics-poller.test.ts Expected: PASS.

Step 5: Commit

git add src/lib/metrics-poller.ts tests/metrics-poller.test.ts
git commit -m "feat: add combineSeries for GPU combination aggregation"

Task 8: GPU action combinations + property inspector

Files:

Modify: src/actions/gpu-graph.ts, com.bryce.llamawatch.sdPlugin/ui/gpu.html

Interfaces:

Consumes: combineSeries from ./metrics-poller.
Produces:
- GpuSettings gains gpuCombo?: string.
- When gpuCombo is a non-empty comma-separated list of ids, the key shows the combined series of those GPUs, labeled GPU 0+2; otherwise existing gpuId behavior.
Step 1: Update the property inspector

In com.bryce.llamawatch.sdPlugin/ui/gpu.html, after the GPU select item, add:

    <sdpi-item label="GPU combination">
      <sdpi-textfield setting="gpuCombo" placeholder="e.g. 0,2 (overrides GPU)" />
    </sdpi-item>

Step 2: Update the GPU action

In src/actions/gpu-graph.ts:

Add to the settings type:

type GpuSettings = CfgSettings & {
  gpuId?: string;
  metric?: GpuMetricKind;
  gpuCombo?: string;
};

Replace the render method and add a parseCombo helper:

  private render(state: GpuState): void {
    const action = state.action;
    if (!action) return;
    const poller = runtime.pollerInstance;
    const metric = state.settings.metric ?? "util_percent";
    const combo = parseCombo(state.settings.gpuCombo);
    const offline = !poller || poller.isOffline();

    let value: number | undefined;
    let history: number[] = [];
    let gpuName: string;
    if (combo) {
      const known = new Set((poller?.gpus() ?? []).map((g) => g.id));
      const ids = combo.filter((id) => known.has(id));
      const histories = ids.map((id) => poller?.getHistory(id, metric) ?? []);
      const combined = combineSeries(histories, metric);
      value = combined.value;
      history = combined.history;
      gpuName = `GPU ${ids.join("+")}`;
    } else {
      const gpuId = state.settings.gpuId ?? "all";
      value = poller?.getValue(gpuId, metric);
      history = poller?.getHistory(gpuId, metric) ?? [];
      gpuName = gpuId === "all" ? "ALL GPUS" : this.gpuLabel(gpuId);
    }

    const svg = renderGpuGraph({ gpuName, metric, value, history, offline });
    if (svg === state.lastSvg) return;
    state.lastSvg = svg;
    void action.setImage(svgDataUrl(svg));
  }

Add the helper at the bottom of the file:

function parseCombo(raw: string | undefined): string[] | undefined {
  const ids = (raw ?? "")
    .split(",")
    .map((s) => s.trim())
    .filter(Boolean);
  return ids.length > 0 ? ids : undefined;
}

and import combineSeries:

import { combineSeries, type GpuMetricKind } from "../lib/metrics-poller";

Step 3: Typecheck, build, and run the suite

Run: npx tsc --noEmit && npm run build && npm test Expected: no type errors, build succeeds, all tests pass.

Step 4: Commit

git add src/actions/gpu-graph.ts com.bryce.llamawatch.sdPlugin/ui/gpu.html
git commit -m "feat: support GPU combinations on the GPU Graph action"

Task 9: Final verification

Step 1: Full verification

Run:

npm test
npm run build
npx tsc --noEmit
streamdeck validate

Expected: all tests pass, build succeeds, typecheck clean, validation passes.

Step 2: Visual smoke test

Run npx tsx -e to render a usage SVG and a combo GPU SVG and confirm they are well-formed (no malformed attributes), e.g.:

npx tsx -e '
import { renderUsage } from "./src/lib/render";
console.log(renderUsage({ modelName: "all", stats: { totalRequests: 1950, totalInputTokens: 312177540, totalOutputTokens: 889193, genP95: 378.86 }, primaryStat: "gen_p95", offline: false }).slice(0, 300));
'

Step 3: Restart the plugin

Run: streamdeck restart com.bryce.llamawatch Expected: ✔ Restarted com.bryce.llamawatch.

Step 4: Update the progress ledger

Append a short note to .superpowers/sdd/progress.md summarizing the feature and verification results (one or two lines, matching existing format).

35 KiB Raw Blame History

Usage Stats + GPU Combinations Implementation Plan

Global Constraints

Task 1: Stats types, parsing, and fetching

Task 2: StatsCache

Task 3: Decode activity SSE events + tracker total

Task 4: Runtime integration

Task 5: renderUsage in render.ts

Task 6: In-Flight action display modes + property inspector

Task 7: GPU combineSeries helper

Task 8: GPU action combinations + property inspector

Task 9: Final verification

35 KiB

Raw Blame History

Task 3: Decode `activity` SSE events + tracker total

Task 5: `renderUsage` in render.ts

Task 7: GPU `combineSeries` helper