# llama-watch — Stream Deck Plugin Design

**Date:** 2026-08-14
**Status:** Approved (pending spec review)

## Summary

A Stream Deck plugin ("llama-watch") that monitors a llama-swap instance
(`http://talos.milky.way:9292`). Two key-action types:

1. **In-flight Monitor** — shows whether a specific model currently has a
   request in flight, with a color-coded state (OFF / LOADING / IDLE / ACTIVE).
2. **GPU Graph** — renders a live line chart of a selected GPU metric
   (utilization %, VRAM %, temperature, power draw, or fan speed) on the key,
   for a specific GPU or the "All GPUs" aggregate.

Target device: standard Stream Deck (72x72 px keys), rendered at 144x144 for
crispness. macOS only. Local install, with eventual Elgato Marketplace
submission as a design goal.

## Approach

Official Elgato JS SDK (`streamdeck-jssdk`) + `streamdeck-cli` tooling, a
TypeScript Node.js plugin, and `@napi-rs/canvas` for image rendering. The
plugin runs locally (launched by the Stream Deck app) and talks directly to
the llama-swap instance.

## Data Sources

llama-swap exposes everything needed (verified live against
`talos.milky.way:9292`):

- **`GET /api/events`** — SSE stream used by the web UI. Emits `inflight`
  events (operations `snapshot` / `add` / `remove`, per-request detail:
  model, timestamp, elapsed, bytes) and `modelStatus` events (per-model
  state: `stopped`, `loading`, `ready`). Also emits `logData`, `uiConfig`,
  `profileChanged`, `activity` events — ignored.
- **`GET /metrics`** — Prometheus text format. Relevant series:
  - `llamaswap_gpu_util_percent{id,name,uuid}`
  - `llamaswap_gpu_memory_util_percent{id,name,uuid}`
  - `llamaswap_gpu_temperature_celsius{id,name,uuid}`
  - `llamaswap_gpu_power_draw_watts{id,name,uuid}`
  - `llamaswap_gpu_fan_speed_percent{id,name,uuid}`
- **`GET /v1/models`** — model list (with `status.value` loaded/unloaded),
  used to populate the model dropdown in the property inspector.

Upstream vLLM metrics (ports 10001/10003) are bound to localhost on the
server and are NOT reachable from the client Mac. Not used.

## Architecture

The plugin is a single Node process with three connections:

1. WebSocket to the Stream Deck software (via `streamdeck-jssdk`).
2. Persistent SSE connection to `/api/events` (the `EventFeed`).
3. 5s-interval polling of `/metrics` (the `MetricsPoller`).

### Modules

- **`MetricsPoller`** (pure logic, testable)
  - Polls `GET /metrics` every **5 seconds** (fixed).
  - Parses the Prometheus text format into a lookup keyed by metric name ×
    GPU id.
  - `getSnapshot(gpuSelector, metric)` resolves any selector × metric
    combination into a number, including the "All GPUs" aggregate.
  - Keeps a ring buffer of 60 samples (~5 minutes) per (gpu, metric).
  - A missed/failed poll skips that sample (no gap in rendering beyond a
    break in the line).
- **`EventFeed`** (pure logic, testable)
  - Persistent SSE connection to `/api/events`.
  - Reconnect with exponential backoff (1s → 30s max); the `inflight`
    `snapshot` operation emitted on connect self-heals the tracker.
  - Maintains per-model in-flight request counts and per-model state from
    `modelStatus`.
- **Actions**
  - `InflightMonitor` — one instance per monitored model.
  - `GpuGraph` — one instance per GPU (or All GPUs) × metric.
- **Property inspectors** (`pi/*.html`) — settings UI for both actions.
- **Renderer** — draws 144x144 PNGs via `@napi-rs/canvas`.

### Data Flow

```
llama-swap ──SSE /api/events──▶ EventFeed ──▶ per-model inflight counts + state
           ──HTTP /metrics 5s──▶ MetricsPoller ──▶ ring buffers + aggregates
                                              │
Stream Deck app ◀──jssdk WS── Actions ────────┘
                     ◀──144x144 PNG + title── Renderer
```

### Settings (shared + per action)

- **Shared:** base URL (default `http://talos.milky.way:9292`), optional API
  key (sent as a header when set; instance currently requires none).
- **InflightMonitor:** model (dropdown populated live from `/v1/models`).
- **GpuGraph:** GPU selector (dropdown from live GPU list, plus "All GPUs"),
  metric (dropdown: Utilization %, VRAM %, Temperature, Power draw, Fan
  speed).

## Action 1 — In-flight Monitor

- **States (rendered on key):**
  - `stopped` / unloaded → grey background, model name, `OFF`
  - `loading` → amber, `LOADING`
  - `ready` + 0 requests → green, `IDLE`
  - `ready` + ≥1 request → red, `ACTIVE` (subtle pulse, re-render ~2 Hz while
    active)
- Model short name displayed on the key; if no model selected, placeholder
  with an edit hint.
- **Press:** opens `http://<base-url>/ui` in the default browser (spawns
  `open` on macOS).

## Action 2 — GPU Graph

- **Rendering (graph dominates the 72x72 key):**
  - Big line chart of the last 60 samples (5 min @ 5s) filling the key.
  - Dark background, bright line, subtle filled gradient under the curve.
  - Small current-value label (e.g. `67%`, `336W`, `55°C`) at top; metric
    name (e.g. `UTIL`, `PWR`) at bottom.
- **Scale:** % metrics fixed 0–100; temperature auto 0–100°C; power
  auto-scaled to observed max.
- **Colors:** severity-based — green → amber → red. Temperature red at
  ≥80°C; util/power/fan scale with level.
- **Aggregates ("All GPUs"):** Utilization/VRAM/Fan → average; Temperature →
  max; Power → sum.
- **Render-on-change:** idle/flat history does not re-render every poll.
- **Press:** opens the web UI (same as Action 1).

## Error Handling

- llama-swap unreachable → dark grey key with `!!` and `OFFLINE`; retry with
  backoff; auto-recover when server returns.
- SSE drop → reconnect with backoff (1s → 30s); self-healing via `snapshot`.
- `/metrics` poll timeout → skip sample.
- Per-button error states — a bad setting on one key does not affect others.

## Packaging & Assets

- Package UUID: `com.bryce.llamawatch` (reverse-DNS, unique).
- `streamdeck-cli` scaffolds project, builds TypeScript, packages
  `.streamDeckPlugin` (zip) → installs via double-click into
  `~/Library/Application Support/com.elgato.StreamDeck/Plugins/`.
- Marketplace-ready from the start:
  - 256×256 plugin icon + action icons (incl. pressed states) at marketplace
    spec sizes.
  - Name "llama-watch", category e.g. "System & Monitoring".
  - Privacy note: reads GPU metrics + request status from the user's own
    llama-swap server; only base URL + optional API key stored in Stream
    Deck's local settings; no data leaves the machine.
  - No hardcoded secrets/URLs — base URL is user-editable.
  - Manifest declares macOS only initially.

## Testing

- Unit tests with `node:test` for the Prometheus parser, inflight tracker,
  and aggregate math (avg/max/sum), using **real captured fixtures** from
  the talos instance (metrics body, inflight snapshot, modelStatus payloads).
- Manual verification: both actions render correctly on the physical Stream
  Deck; key press opens the web UI.

## Out of Scope (YAGNI)

- Marketplace submission itself (goal for later; assets/metadata prepared now).
- Auto-update plumbing.
- Windows/Linux support (later add-on; macOS declared in manifest).
- Additional metrics or display styles.