51 lines
2.2 KiB
Markdown
51 lines
2.2 KiB
Markdown
# llama-watch
|
||
|
||
A Stream Deck plugin (macOS) that monitors a llama-swap instance.
|
||
|
||
## Actions
|
||
|
||
- **In-Flight Monitor** — per-model request activity. The default view shows
|
||
the live in-flight request count with a 60-second activity spark (states:
|
||
`OFF` / `LOADING` / `IDLE` / count). Switch **Display** to **Usage stats**
|
||
to show request/token totals and the generation-speed P95 from
|
||
`/api/metrics/stats`, per model or across **All models**, with a pickable
|
||
primary stat. Powered by the real-time `/api/events` SSE feed.
|
||
- **GPU Graph** — live line chart of a GPU metric (utilization %, VRAM %,
|
||
temperature, power draw, fan speed) for one GPU, all GPUs, or an arbitrary
|
||
combination (e.g. GPU combination `0,2`), sampled every 5 s.
|
||
|
||
Pressing either key opens `http://<base-url>/ui` in your browser.
|
||
|
||
## Install
|
||
|
||
Double-click the packaged `.streamDeckPlugin`, or run:
|
||
|
||
```bash
|
||
npm install
|
||
npm run build
|
||
streamdeck validate com.bryce.llamawatch.sdPlugin
|
||
```
|
||
|
||
## Develop
|
||
|
||
```bash
|
||
npm run watch # hot-reload while the Stream Deck app is running
|
||
npm test # unit tests (node:test + tsx)
|
||
```
|
||
|
||
## Configure
|
||
|
||
Per-key settings: base URL (default `http://localhost:9292`), optional API key, and the model / GPU / metric to watch. The model and GPU dropdowns are populated live from the instance. Each action instance is independent, so you can place several In-Flight Monitor keys (one per model, or in either display mode) and several GPU Graph keys (one per GPU × metric, or per GPU combination) on the same profile. All keys share a single connection to the configured llama-swap instance, which is re-established automatically if you change the base URL or API key on any key.
|
||
|
||
## Notes
|
||
|
||
- Usage totals come from llama-swap's `/api/metrics/stats` and cover its
|
||
in-memory activity retention (default ~1000 most recent requests), not
|
||
lifetime counters. The P95 is a **tokens/sec** generation-speed percentile.
|
||
- Keys are SVG-rendered (`render.ts`), so no image assets or canvas are
|
||
needed at runtime.
|
||
|
||
## Marketplace
|
||
|
||
Assets are prepared for eventual submission (`imgs/plugin/icon.png` 256/512, action icons, category icon). The plugin only stores its settings locally; it reads GPU metrics and request status from the user's own llama-swap server and sends nothing elsewhere.
|