Nach Live-Check-Modus gegen ghostnet-ki (ok=23, changed=4, failed=0): - Freiraumprüfung auf existierendes Elternelement (df schlägt im Check-Modus auf dem noch nicht angelegten Store fehl) - Rechte-Durchsetzung nur über vorhandene Artefakte (file-Modul im Check-Modus) - assert ohne loop (2.21 lehnt loop auf assert ab); statische Meldungen, Modell-Anwesenheit im Check-Modus nur informell - find file_type=any für Character-Devices (renderD*) - README: Aufruf mit --private-key nach Workstation-Konvention Preview prognostiziert korrekt: 2 Modellverzeichnisse, 2 SOURCE.yml, Netz-Quadlet, 2 Container-Quadlets; gepinntes ROCm-Image bereits im Store. Generated by Mistral Vibe. Co-Authored-By: Mistral Vibe <vibe@mistral.ai>
62 lines
2.6 KiB
Markdown
62 lines
2.6 KiB
Markdown
# LLM serving stack – Ansible control project
|
||
|
||
Executes the LLM serving stack for exactly one environment on
|
||
`ghostnet-ki` (GNAI-008 section 7.4). Host baseline, Podman runtime,
|
||
workload accounts and AMD GPU access are managed by
|
||
[`../../host/ansible`](../../host/ansible/README.md); this project never
|
||
changes host state.
|
||
|
||
## Layout
|
||
|
||
- `inventories/<environment>/` – one inventory per KI environment
|
||
(currently `testing`); deployment uses the `ansible-deploy` account with
|
||
`become: true`.
|
||
- `playbooks/ai.yml` – applies `model_store` then `inference_engine`.
|
||
- `roles/model_store/` – shared, pinned model weights under
|
||
`/var/lib/ghostnet-ai/shared/models` (GNAI-010 sections 3 and 4).
|
||
- `roles/inference_engine/` – rootless Quadlets, network, image provisioning
|
||
and the exclusive profile switch for llama.cpp (GNAI-010 sections 5 and 6).
|
||
|
||
## Control node setup
|
||
|
||
Use the repository-wide control venv built from
|
||
[`../../../shared/ansible/requirements-control.txt`](../../../shared/ansible/requirements-control.txt)
|
||
(`ansible-core==2.21.4`), for example:
|
||
|
||
```bash
|
||
python3 -m venv .venv
|
||
.venv/bin/pip install -r shared/ansible/requirements-control.txt
|
||
cd llm-server/llm/ansible
|
||
../../../.venv/bin/ansible-playbook -i inventories/testing playbooks/ai.yml --syntax-check
|
||
```
|
||
|
||
## Usage (Testing)
|
||
|
||
All commands use the operator deploy key
|
||
`~/.ssh/id_ed25519_ghostnet_ai_ansible_deploy` (see AGENTS.md; no ssh-agent
|
||
required):
|
||
|
||
```bash
|
||
# Preview only – no changes, no downloads, no service start:
|
||
../../../.venv/bin/ansible-playbook -i inventories/testing playbooks/ai.yml \
|
||
--check --diff --private-key ~/.ssh/id_ed25519_ghostnet_ai_ansible_deploy
|
||
|
||
# Provision model weights only (tag models):
|
||
../../../.venv/bin/ansible-playbook -i inventories/testing playbooks/ai.yml \
|
||
--tags models --private-key ~/.ssh/id_ed25519_ghostnet_ai_ansible_deploy
|
||
|
||
# Render quadlets and start/verify the selected profile (tag service):
|
||
../../../.venv/bin/ansible-playbook -i inventories/testing playbooks/ai.yml \
|
||
--tags service --private-key ~/.ssh/id_ed25519_ghostnet_ai_ansible_deploy
|
||
```
|
||
|
||
`inference_engine_active_profile` selects which rendered quadlet is started;
|
||
all other profiles stay rendered but stopped. Check mode downloads no model
|
||
and starts no service. A second identical run must report `changed=0`
|
||
(GNAI-010 section 10.4).
|
||
|
||
The full technical acceptance (64k/128k start, VRAM distribution, prefill
|
||
ladder, restart behaviour) is an operator-run protocol documented in
|
||
[GNAI-010](../../../docs/platform/llm-server/ai/GNAI-010-TECH_Llama-Serving.md);
|
||
this role automates deployment and basic health only.
|