Overview
—
GPU Utilisation
VRAM
CPU
Memory
Services
Inference Backend
GPU Detail
Storage
Network
Resilience
Last Health Check
Services
vLLM and Ollama are conflicting units — starting one stops the other.
| Service | State | Boot | Port | Answering | Restarts | Control |
|---|
Bulk
Free the GPU stops vLLM and Ollama. Nothing else is touched.
System
the suite's own machinery — timers, boot-time tuning, firewall, driver
| Unit | State | Boot | Restarts | Control |
|---|
Listening sockets
Docker containers
Install history
Configuration backups every changed file, per run
Models
what is on disk, and what is loaded
Add a model from HuggingFace download it, then hand it to vLLM
Currently serving
Pull an Ollama model
HuggingFace cache
Ollama store
Metrics
telemetry recorded every minute — this is how you answer "what was it doing at 3am"
Modules
re-run any part of the installer — every module is idempotent
Installer modules
Output
Configuration
every file that shapes how this machine behaves
Logs
journalctl, live
Health
runs ai-health with repairs disabled — looking never changes anything
Playground
streaming chat against the running backend
Request
Benchmark
concurrent requests — this is what vLLM is for
Configuration
Continuous batching means N requests should finish in barely more
wall-clock time than one. If total time scales linearly with N,
batching is not happening — check ai-ctl logs vllm for
KV-cache preemption.
Above 6, the browser is the bottleneck, not the server.
Chrome opens at most six HTTP/1.1 connections per origin, so a run
of 8 arrives as 6 + 2 and the wall clock roughly doubles. The
aggregate figure stays honest; the per-request times do not. For a
true concurrency sweep use the shell:
ai-bench 32
Result
Endpoints
reach anything on loopback and see what it says