Container Lifecycle

Startup flow

Starting a profile is more than docker compose up. llmux runs an ordered pipeline and aborts early with a clear message if any step fails.

flowchart TD
    A[Resolve profile from profiles.yaml] --> B[Port conflict check]
    B --> C[Render profile env file]
    C --> D{Backend}
    D -->|vLLM| E[Validate .env.common + linked config]
    D -->|llama.cpp| F[Render override.yaml from config]
    E --> G[GPU conflict warnings]
    F --> G
    G --> H[docker compose up -d]
    H --> I[Post-start health validation]
    I --> J[Container ready]

1. Profile env render

The profile is read from profiles.yaml and rendered to .runtime/<backend>/<name>.env. This file is what docker compose --env-file consumes — it carries CONTAINER_NAME, the port, GPU_ID, CONFIG_NAME, and backend-specific values. It is regenerated on every start, so editing it directly is pointless; edit the profile instead.

2. Config / override render

3. Conflict gates

Before compose up, llmux checks for resource clashes:

• Port conflicts — the target port is probed against running Docker containers and local processes (a raw socket bind). A static overlap between two stopped profiles is ignored.
• GPU conflicts — the profile's gpu_id set is compared against every running container from both backends. Overlap produces a warning.

The TUI offers to stop running llmux profiles sharing the selected GPUs before starting a new model. The CLI provides the same replacement sequence with llmux up PROFILE --replace; --force instead allows both models to run together. Port conflicts outside the replacement targets and failed probes block startup. Replacement never stops unmanaged containers or profiles on unrelated GPUs, and a failed stop prevents the new start.

4. compose up + health validation

llmux runs docker compose ... up -d with the rendered env files. After compose returns, a post-start validation loop polls docker inspect:

• If the container is restarting / exited / dead or becomes unhealthy, startup is reported as failed and the last log lines are printed (this catches e.g. a GPU OOM during engine init).
• Otherwise llmux waits for /v1/models to return a served model id on either backend. A stalled model or an absolute timeout is reported as a startup failure.
• If a cache-size or Docker-log progress probe itself fails, startup stops with that probe error instead of treating it as a quiet loading interval.

The lifecycle commands

Profile names are globally unique, so all commands auto-detect the backend from the name; pass --backend / -b to require one explicitly. These are top-level CLI aliases — llmux up is shorthand for llmux container up.

up — start a profile

llmux up qwen3-0-6b
llmux up qwen3-0-6b --backend vllm

# image overrides
llmux up qwen3-0-6b --tag v0.20.1          # specific vllm/vllm-openai tag
llmux up qwen3-0-6b --tag v0.20.1 --pull   # force --pull always for a non-dev image
llmux up qwen3-0-6b --dev                  # use the locally-built dev image
llmux up gemma-3-4b --tag llamacpp-dev:master

Compose output streams to stdout. The command exits non-zero if startup fails.

prepare — download without starting

prepare runs everything up does except launching the server: it renders the profile env (and llama.cpp's command override), makes sure the image is on disk (pulling it when it isn't), and downloads the weights into your HF cache. Nothing is served and no GPU is touched, so a later up only has to load what is already there.

llmux prepare qwen3-0-6b
llmux prepare gemma-3-4b --backend llamacpp

The download runs inside a throwaway container: vLLM uses the resolved serving image and huggingface_hub.snapshot_download; llama.cpp uses PREPARE_DOWNLOADER_IMAGE because its server image has no Python/Hugging Face client. There is no host-side hf CLI involved. In the TUI the same action is ⬇ Prepare (download only) in a stopped profile's action menu, or the p key on the dashboard.

Before starting and again on cancellation, llmux removes any stale downloader container. Cleanup failure is reported immediately with a non-zero exit instead of leaving a hidden name collision for the next run.

down — stop a profile

llmux down qwen3-0-6b

llmux prefers docker compose down for a clean network teardown. If the runtime env/override cannot be prepared, the container is orphaned, or compose down fails, it still attempts docker stop + docker rm (docker rm -f for llama.cpp). A failed fallback exits non-zero and reports both the original and cleanup causes. Stopping an already-stopped profile is a no-op that exits 0.

logs — stream container logs

llmux logs qwen3-0-6b              # follow (default), last 200 lines first
llmux logs qwen3-0-6b --no-follow  # print recent lines and exit
llmux logs qwen3-0-6b --tail 500   # change the initial backlog

While following, Ctrl+C stops the stream (exit code 130).

ps — list profiles and status

llmux ps                    # all profiles, both backends
llmux ps --backend vllm     # one backend only
llmux ps --running          # only running containers
llmux ps --json             # machine-readable output

Output columns: backend, profile, status, port, gpu, container, model. Status preserves Docker lifecycle and health rather than collapsing them: healthy, unhealthy, starting, running, paused, restarting, removing, created, exited, dead, unknown, or stopped. The model column resolves the vLLM config's model (then profile model_id) or the llama.cpp config's alias (then hf_repo, then model_file).

render-env — regenerate runtime env files

llmux render-env qwen3-0-6b   # one profile
llmux render-env              # re-render every profile
llmux render-env --backend llamacpp

Useful after hand-editing profiles.yaml, or to inspect exactly what compose will see. up does this automatically — you rarely need it directly.

Healthchecks

Both compose definitions declare a Docker healthcheck that curls the in-container /health endpoint every 30s. The start_period is generous to allow for model loading — 600s for vLLM, 300s for llama.cpp — so a slow first load won't be flagged as unhealthy. ps and the TUI dashboard read this health state to show healthy / starting / unhealthy.

See also

• Profiles — the launch recipe up resolves
• Model Configs — what gets rendered into the container at startup
• Using the TUI — the same lifecycle, driven interactively
• Dev Builds — the --dev image path