# LLM serving stack – Ansible control project Executes the LLM serving stack for exactly one environment on `ghostnet-ki` (GNAI-008 section 7.4). Host baseline, Podman runtime, workload accounts and AMD GPU access are managed by [`../../host/ansible`](../../host/ansible/README.md); this project never changes host state. ## Layout - `inventories//` – one inventory per KI environment (currently `testing`); deployment uses the `ansible-deploy` account with `become: true`. - `playbooks/ai.yml` – applies `model_store` then `inference_engine`. - `roles/model_store/` – shared, pinned model weights under `/var/lib/ghostnet-ai/shared/models` (GNAI-010 sections 3 and 4). - `roles/inference_engine/` – rootless Quadlets, network, image provisioning and the exclusive profile switch for llama.cpp (GNAI-010 sections 5 and 6). ## Control node setup Use the repository-wide control venv built from [`../../../shared/ansible/requirements-control.txt`](../../../shared/ansible/requirements-control.txt) (`ansible-core==2.21.4`), for example: ```bash python3 -m venv .venv .venv/bin/pip install -r shared/ansible/requirements-control.txt cd llm-server/llm/ansible ../../../.venv/bin/ansible-playbook -i inventories/testing playbooks/ai.yml --syntax-check ``` ## Usage (Testing) All commands use the operator deploy key `~/.ssh/id_ed25519_ghostnet_ai_ansible_deploy` (see AGENTS.md; no ssh-agent required): ```bash # Preview only – no changes, no downloads, no service start: ../../../.venv/bin/ansible-playbook -i inventories/testing playbooks/ai.yml \ --check --diff --private-key ~/.ssh/id_ed25519_ghostnet_ai_ansible_deploy # Provision model weights only (tag models): ../../../.venv/bin/ansible-playbook -i inventories/testing playbooks/ai.yml \ --tags models --private-key ~/.ssh/id_ed25519_ghostnet_ai_ansible_deploy # Render quadlets and start/verify the selected profile (tag service): ../../../.venv/bin/ansible-playbook -i inventories/testing playbooks/ai.yml \ --tags service --private-key ~/.ssh/id_ed25519_ghostnet_ai_ansible_deploy ``` `inference_engine_active_profile` selects which rendered quadlet is started; all other profiles stay rendered but stopped. Check mode downloads no model and starts no service. A second identical run must report `changed=0` (GNAI-010 section 10.4). The full technical acceptance (64k/128k start, VRAM distribution, prefill ladder, restart behaviour) is an operator-run protocol documented in [GNAI-010](../../../docs/platform/llm-server/ai/GNAI-010-TECH_Llama-Serving.md); this role automates deployment and basic health only.