- inference_engine + rocm_container_probe: Digest sha256:4bbbd2e3... (amd64-Manifest sha256:abb28972...), Build b12359, Revision b1b770d90..., OCI-Version v1.0.0-b12359-b1b770d. v1.0.0 (Release 13.11.2026) loest ueber nightly-tag.txt auf b12359 auf; moving tag server-rocm zeigt auf denselben Digest. Semver-Policy upstream ab v1.0.0 — passt zur Digest-Pinning-Konvention. - Kompatibilitaet gegen b12359 llama-server --help geprueft (Image lokal gepullt): alle 26 Exec=-Flags des Quadlets existieren; einzige entfernte Option --defrag-thold (#30565) wird nicht verwendet. ENTRYPOINT /app/llama-server unveraendert. Server-Haertung #32113/#32115/#32098 greift nicht (kein server-seitiges --tools/MCP, keine privaten Media-URLs, OpenWebUI ist Non-Browser-Client). libllama.so.1/36 entfernte Symbole betreffen nur native Bindings — wir sind HTTP-only. - Neue Optionen bewertet (GNAI-010 0.20): FP8-KV f8_e4m3 (auf gfx1030 emuliert, Benchmark vor Adoption, Default q8_0), --moe-cache-cpu-share (nur MoE), --sleep-idle-seconds (kontraproduktiv), --api-key (optional), die Opt-out-Flags --allow-unauthenticated-tools/--media-allow-private-urls bleiben aus. - Doku: GNAI-010 v0.20 + Abschnitt 3.1, GNAI-008 v0.23 + Probe-Abschnitt, Host-README Pin-Block (alter Pin als historisch markiert). - Quadlet Exec bleibt unveraendert. Rollout auf 10.0.0.90 steht noch aus. Generated by Mistral Vibe. Co-Authored-By: Mistral Vibe <vibe@mistral.ai>
LLM serving stack – Ansible control project
Executes the LLM serving stack for exactly one environment on
ghostnet-ki (GNAI-008 section 7.4). Host baseline, Podman runtime,
workload accounts and AMD GPU access are managed by
../../host/ansible; this project never
changes host state.
Layout
inventories/<environment>/– one inventory per KI environment (currentlytesting); deployment uses theansible-deployaccount withbecome: true.playbooks/ai.yml– appliesmodel_storetheninference_engine.roles/model_store/– shared, pinned model weights under/var/lib/ghostnet-ai/shared/models(GNAI-010 sections 3 and 4).roles/inference_engine/– rootless Quadlets, network, image provisioning and the exclusive profile switch for llama.cpp (GNAI-010 sections 5 and 6).
Control node setup
Use the repository-wide control venv built from
../../../shared/ansible/requirements-control.txt
(ansible-core==2.21.4), for example:
python3 -m venv .venv
.venv/bin/pip install -r shared/ansible/requirements-control.txt
cd llm-server/llm/ansible
../../../.venv/bin/ansible-playbook -i inventories/testing playbooks/ai.yml --syntax-check
Usage (Testing)
All commands use the operator deploy key
~/.ssh/id_ed25519_ghostnet_ai_ansible_deploy (see AGENTS.md; no ssh-agent
required):
# Preview only – no changes, no downloads, no service start:
../../../.venv/bin/ansible-playbook -i inventories/testing playbooks/ai.yml \
--check --diff --private-key ~/.ssh/id_ed25519_ghostnet_ai_ansible_deploy
# Provision model weights only (tag models):
../../../.venv/bin/ansible-playbook -i inventories/testing playbooks/ai.yml \
--tags models --private-key ~/.ssh/id_ed25519_ghostnet_ai_ansible_deploy
# Render quadlets and start/verify the selected profile (tag service):
../../../.venv/bin/ansible-playbook -i inventories/testing playbooks/ai.yml \
--tags service --private-key ~/.ssh/id_ed25519_ghostnet_ai_ansible_deploy
inference_engine_active_profile selects which rendered quadlet is started;
all other profiles stay rendered but stopped. Check mode downloads no model
and starts no service. A second identical run must report changed=0
(GNAI-010 section 10.4).
The full technical acceptance (64k/128k start, VRAM distribution, prefill ladder, restart behaviour) is an operator-run protocol documented in GNAI-010; this role automates deployment and basic health only.