I built Helm’s Host to make running local language models straightforward: choose a model, give it a name, and start a container.

It’s a small Bash-based runner for llama.cpp. It downloads missing GGUF models from Hugging Face, keeps them cached between runs, and exposes an OpenAI-compatible API or a browser interface.

The goals are simple: explicit model selection, minimal configuration, and persistent model storage. The current CUDA build targets my NVIDIA RTX 3090.

Get started

You’ll need a Linux machine with Bash, Docker, and working NVIDIA GPU support through the NVIDIA Container Toolkit.

Clone the project:

git clone https://github.com/derik/helms-host.git
cd helms-host

Create an optional configuration file:

test -e .env || cp .env.example .env

By default, models are stored under /srv/ai/models, and the script checks that /srv/ai is mounted before creating directories.

Edit .env to match your storage. For intentional storage inside the checkout instead of a separate mounted drive, use:

LLAMA_MODELS_DIR=./models
LLAMA_STORAGE_MOUNT=

Run a model

Pass the Hugging Face repository and quantization directly:

./ctl.sh run \
  --model unsloth/Qwen3.8-27B-GGUF:UD-Q4_K_M \
  --alias qwen3.8-27b \
  --mode opencode \
  --ctx-size 32768

The first run builds the Docker image if needed, then downloads and loads the model. Later runs reuse the image and cached weights.

Follow startup progress with:

./ctl.sh logs

Model loading can take a while. Context size and quantization also affect how much GPU memory you need.

Use the API

Once the model is ready, check the available model:

curl http://localhost:8080/v1/models

Send a chat request:

curl http://localhost:8080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "qwen3.8-27b",
    "messages": [
      {"role": "user", "content": "Explain Docker in three sentences."}
    ],
    "max_tokens": 200
  }'

For OpenCode or another compatible client, configure the base URL as http://SERVER:8080/v1 and use your chosen alias as the model ID. The opencode mode starts the API; client configuration is separate.

Open the browser interface

Use webui mode to enable llama.cpp’s native UI:

./ctl.sh run \
  --model unsloth/Qwen3.8-27B-GGUF:UD-Q4_K_M \
  --mode webui \
  --ctx-size 32768

Then visit http://SERVER:8080 on your trusted network.

Adjust settings and manage the container

Named options make individual runs easy to customize. Extra llama.cpp arguments go after --:

./ctl.sh run \
  --model unsloth/Qwen3.8-27B-GGUF:UD-Q4_K_M \
  --port 8081 \
  --ctx-size 16384 \
  -- --temp 0.6 --top-p 0.95

Check or stop the container:

./ctl.sh status
./ctl.sh stop

Stopping preserves downloaded models. A subsequent run replaces Helm’s Host’s previous container. I manually stop other GPU workloads before starting it.

You can find the code and configuration examples on GitHub.