Using the TUI

Launching

llmux          # no arguments → launches the TUI
llmux tui      # explicit

Running llmux with no subcommand drops you straight into the unified dashboard.

The dashboard

The dashboard is one DataTable showing every profile from both backends. Running containers float to the top so the active workload is always visible without scrolling; within the running / stopped groups, rows stay sorted by (backend, name).

┌ llmux ─ vLLM + llama.cpp ───────────────────────────────────────────────────────────────────┐
│  vLLM 1/2  ·  llama.cpp 0/1  ·  Enter = actions                   │
├─────────────────────────────────────────────────────────────────────┤
│ Status      Backend    Profile      Port   tok/s  Model        Detail  │
│ ● running   vLLM       qwen3-8b     8000   62.4   Qwen3-8B     TP=1    │
│ ○ stopped   llama.cpp  gemma-3-4b   8080   —      gemma-3-4b   —       │
│ ○ stopped   vLLM       qwen3-0-6b   8001   —      Qwen3-0.6B   —       │
└─────────────────────────────────────────────────────────────────────┘
 GPU0 ████████░░░░ 64%  12.1/24GB
 🔍  Estimate HF model memory (press m then type, Enter to run)

A status bar shows running / total counts per backend. Container status preserves healthy, running, starting, unhealthy, paused, restarting, removing, created, exited, dead, unknown, and stopped rather than reducing every non-running state to stopped. The tok/s column shows live generation throughput for each running model, polled from its /metrics endpoint every 3 seconds (see llmux stats). Missing metrics display —, while an HTTP polling failure displays error and raises a notification. A GPU bar below the table auto-refreshes every few seconds, and the table itself re-scans every 5 seconds. The memory-estimator search row is hidden by default to keep the dashboard compact — press m to reveal it (and Esc to hide it again).

Keyboard shortcuts

Move the cursor with ↑ / ↓, then act on the selected row.

Dashboard

The u / p / d / l / e / c / x keys are deliberately hidden from the footer — they do exactly the same thing as picking the matching item from the Enter action menu.

App-level

Log / startup viewers

The action menu

Pressing Enter on a profile opens a context menu. The options depend on whether the container is running:

Prepare (download only) fetches the image and the weights and stops there — see prepare. Benchmark discards one warmup request, runs 3 measured /v1/chat/completions requests, and reports median tokens/second. Render Runtime Env regenerates the selected profile's .runtime file from profiles.yaml. Delete is blocked while a container is running — stop it first. To rename a profile, edit its Name field in the profile form (also only while the container is stopped, since it's named after the profile).

Live monitor

Press v on any row (or pick 📊 Monitor from the action menu) to open a full-screen, btop-style system view. It polls nvidia-smi and each running server's /metrics once a second. Opening it does not require a running container — with nothing up you still get the GPU panel, which is usually the reason to open a monitor at all.

Always shown:

• GPU — every GPU on the box (card name in the panel title): utilization, memory, temperature, power, and PCIe rx/tx as heat bars.

Then one panel per running model — the running list is re-scanned every tick, so a model started elsewhere appears on its own. With exactly one model up you get the full detail view; with several, each gets a compact summary row (tok/s, KV, running/waiting, TTFT, TPOT).

• Throughput — generation and prompt tok/s as a braille graph, with current and peak;
• KV cache — utilization trend as a braille graph, current and peak;
• Requests — running, waiting, completions/s, and preemptions;
• Cache hit — KV, prefix, and external prefix hit rate as heat bars (vLLM; llama.cpp shows KV only);
• Latency — TTFT and E2E percentiles (p50/p95/p99) from the histograms, plus TPOT, queue, and prefill/decode/inference phases (vLLM; llama.cpp shows — where it exposes no histogram).

Both backends work — the differing Prometheus metric names are matched by family, and any metric a backend doesn't expose reads — rather than a made-up value. GPU, backend, and metrics request failures appear as monitor notices; the header lag includes GPU, PCIe, model discovery, and metrics collection for the whole tick. Keys: p pause, r reset peaks, +/- poll interval, l language, Esc/q back.

Terminal monitor (plain)

Press t on any row to suspend the Textual UI and open the same monitor — the same GPU panel, braille graphs, cache-hit, and latency percentiles — as a plain-terminal page in your normal terminal, auto-refreshing every second. No Textual rendering, so it holds up over low-bandwidth SSH or on terminals that render the TUI poorly. Same keys; q (or Ctrl+C) returns.

The same monitor is available without the TUI at all: llmux top [PROFILE]. Omit the profile to show every running model. With a profile, one backend's confirmed match is enough to start the monitor even if the other backend scan fails. If no backend confirms the target and a scan failed, the command exits 2 because the target is unknown; after a complete scan confirms that it is stopped, it exits 1.

Quick Setup

n opens a backend picker, then a Quick Setup form that creates a matching profile + config in one step. For vLLM the form takes a Hugging Face model id, an optional profile name, GPU id, port, GPU-memory-utilization, an Enable-LoRA toggle, an optional config to copy, and optional official-recipe review. llama.cpp provides the equivalent GGUF repository/file and runtime settings. Its Hugging Face file lookup fails explicitly on malformed or partial listings and when the pagination safety cap is reached. Authentication is sent only to the configured Hugging Face origin, never to an off-origin pagination or redirect target.

System screen

Press s to open the selected backend's System screen. Its Docker Images tab can refresh, pull, remove, or build a dev image without starting a container. The Environment tab validates .env.common and renders runtime env files for every profile of that backend; profile action menus can render one selected profile. Token values are never displayed.

Memory estimator

Press m (or click the search box) and type a Hugging Face model id, then Enter. llmux estimates the model's memory footprint and renders it against your detected GPUs:

• A per-GPU bar shows the estimated fit ratio — green / yellow / red, or a red OVER marker when the estimate exceeds available VRAM.
• With multiple GPUs, it also shows the per-GPU figure assuming tensor parallelism (TP=N).

This runs entirely on the dashboard — no profile or container needed.

Container start screen

Picking Start Container opens a full-screen launcher with a Version radio set. The options differ per backend:

Selecting Dev Build reveals Repo URL and Branch inputs, pre-filled from .env.common. Pressing Start streams the startup log live, then switches to following the container's logs. See Dev Builds for the build mechanics.

Profile and config forms

Edit Profile and Edit Config open modal forms:

• The profile form edits the profile name, container name, port, GPU id, the linked config, image_tag, and per-profile environment variables. Existing environment values are shown as <redacted>; leaving a placeholder unchanged preserves its original value. The vLLM form also exposes tensor parallelism and the max_loras, max_lora_rank, and lora_modules fields. Changing the name renames the profile on save rather than creating a second one — the container must be stopped and the new name must be free in both backends. The profile action menu can also clone the profile.
• The config form is a dynamic list of flag rows — add a row, type a flag name (autocompleted from the real flag set of the vllm serve / llama-server image you're running — see Flag autocomplete), and a value. Leaving a boolean flag's value blank stores true. A help line shows a one-line description of the focused flag. Each row also has an on/off switch: flip it off to park a flag without deleting its value (the row dims), flip it back on to restore it. See Toggling parameters on / off. Editing the config's name renames the file and repoints every profile that referenced it.

The profile delete confirmation includes an off-by-default switch to remove the linked config when it is no longer shared. Tracked example configs and configs still referenced by another profile are protected.

Both forms map straight onto profiles.yaml and the config YAML — there is no separate TUI state.

See also

• Profiles / Model Configs — what the forms edit
• Container Lifecycle — what Start / Stop / Logs do
• Dev Builds — the Dev Build start mode
• CLI Reference — the same operations, headless