bzuccaro
f98043c21b
On a long-lived SSE connection a missed remove leaves a stale request id in the tracker forever; llama-swap only sends a fresh snapshot on connect, so counts drifted up and stayed stuck. The tracker now timestamps each request id (upserts refresh it) and prunes ids not updated within 120s, on every inflight event and on a 30s runtime interval. Removes also accept numeric ids defensively.
llama-watch
A Stream Deck plugin (macOS) that monitors a llama-swap instance.
Actions
- In-Flight Monitor — per-model request activity. The default view shows
the live in-flight request count with a 60-second activity spark (states:
OFF/LOADING/IDLE/ count). Switch Display to Usage stats to show request/token totals and the generation-speed P95 from/api/metrics/stats, per model or across All models, with a pickable primary stat. Powered by the real-time/api/eventsSSE feed. - GPU Graph — live line chart of a GPU metric (utilization %, VRAM %,
temperature, power draw, fan speed) for one GPU, all GPUs, or an arbitrary
combination (e.g. GPU combination
0,2), sampled every 5 s.
Pressing either key opens http://<base-url>/ui in your browser.
Install
Double-click the packaged .streamDeckPlugin, or run:
npm install
npm run build
streamdeck validate com.bryce.llamawatch.sdPlugin
Develop
npm run watch # hot-reload while the Stream Deck app is running
npm test # unit tests (node:test + tsx)
Configure
Per-key settings: base URL (default http://localhost:9292), optional API key, and the model / GPU / metric to watch. The model and GPU dropdowns are populated live from the instance. Each action instance is independent, so you can place several In-Flight Monitor keys (one per model, or in either display mode) and several GPU Graph keys (one per GPU × metric, or per GPU combination) on the same profile. All keys share a single connection to the configured llama-swap instance, which is re-established automatically if you change the base URL or API key on any key.
Notes
- Usage totals come from llama-swap's
/api/metrics/statsand cover its in-memory activity retention (default ~1000 most recent requests), not lifetime counters. The P95 is a tokens/sec generation-speed percentile. - Keys are SVG-rendered (
render.ts), so no image assets or canvas are needed at runtime.
Marketplace
Assets are prepared for eventual submission (imgs/plugin/icon.png 256/512, action icons, category icon). The plugin only stores its settings locally; it reads GPU metrics and request status from the user's own llama-swap server and sends nothing elsewhere.