From 69930f815ac560229a4c409c58615ad9bcffb1e8 Mon Sep 17 00:00:00 2001 From: spektr Date: Fri, 30 Oct 2026 20:47:42 +0100 Subject: [PATCH] Encoding: Pipeline-Konten, NFS-Mount, Layout und Handover-Simulation umgesetzt und verifiziert. MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - workload_accounts: gn-media (GID 970), gn-download (983), gn-dashboard (984); gn-media als primäre Gruppe der Pipeline-Konten — AUTH_SYS ueber NFSv4 wirkte serverseitig nicht mit Supplementary-Gruppen (Betriebsbefund 30.10.2026); Sudoers des Deploy-Kontos auf (ALL:ALL) verbreitert (become_user) - video_encoding: NFS-Client, Mount-Unit und Pipeline-Layout auf dem Export (10.0.0.2:/media/datenhdd3/Encoding -> /var/lib/ghostnet-pipeline, ausserhalb der Homes); Layout-Anlage als Dienstkonten wegen root_squash; Verifikation der Verzeichnisrechte als gn-encoding (2777/2770/0750/2750, Gruppe 970) - pve_media_cron: Legacy-Crontab (chown root:root + chmod -R 0777 auf allen Medienplatten, alle 10 min) entfernt — haette jedes echte Berechtigungsmodell plattgewalzt; Backup unter /var/backups/ghostnet/pve/; verify-production prueft das Fehlen ab - verify-pipeline-handover.yml: Handover-Simulation über alle Identitaeten (CT-101-.dlc-Drop -> gn-download -> queue -> gn-encoding -> output/state -> gn-dashboard); einmal vollstaendig bestanden (30.10.2026) - Doku: GNENC-000/001/002, GNAI-017 (6.10/6.11), GNMON-001 auf Stand gebracht Generated by Mistral Vibe. Co-Authored-By: Mistral Vibe --- .../encoding/GNENC-000-DOC_Index.md | 24 +- .../encoding/GNENC-001-PLAN_Architecture.md | 31 +- .../encoding/GNENC-002-PLAN_Pipeline.md | 48 ++-- .../monitoring/GNMON-001-PLAN_Architecture.md | 2 +- .../GNAI-017-PLAN_Samba-NFS-Export.md | 14 +- llm-server/encoding/ansible/README.md | 12 +- .../playbooks/verify-pipeline-handover.yml | 205 +++++++++++++ .../ansible/roles/video_encoding/README.md | 33 ++- .../roles/video_encoding/defaults/main.yml | 50 +++- .../roles/video_encoding/tasks/main.yml | 270 +++++++++++++++++- .../roles/host_baseline/defaults/main.yml | 7 + .../roles/host_baseline/tasks/main.yml | 9 + .../ansible/roles/workload_accounts/README.md | 22 +- .../roles/workload_accounts/defaults/main.yml | 57 +++- .../roles/workload_accounts/tasks/main.yml | 13 +- proxmox/pve-host/README.md | 7 + proxmox/pve-host/ansible/playbooks/site.yml | 1 + .../ansible/playbooks/verify-production.yml | 23 ++ .../ansible/roles/pve_media_cron/README.md | 13 + .../roles/pve_media_cron/defaults/main.yml | 8 + .../roles/pve_media_cron/tasks/main.yml | 66 +++++ 21 files changed, 818 insertions(+), 97 deletions(-) create mode 100644 llm-server/encoding/ansible/playbooks/verify-pipeline-handover.yml create mode 100644 proxmox/pve-host/ansible/roles/pve_media_cron/README.md create mode 100644 proxmox/pve-host/ansible/roles/pve_media_cron/defaults/main.yml create mode 100644 proxmox/pve-host/ansible/roles/pve_media_cron/tasks/main.yml diff --git a/docs/platform/llm-server/encoding/GNENC-000-DOC_Index.md b/docs/platform/llm-server/encoding/GNENC-000-DOC_Index.md index 6fcfe0a..cb3be89 100644 --- a/docs/platform/llm-server/encoding/GNENC-000-DOC_Index.md +++ b/docs/platform/llm-server/encoding/GNENC-000-DOC_Index.md @@ -33,6 +33,18 @@ beschrieben. NFSv4-only wirksam (`-3`), `verify-production` `ok=17 changed=0`, Idempotenz `changed=0`. Berechtigungsmodell entschieden: Mode 2777 auf den SMB-berührten Verzeichnissen (GNENC-002 Abschnitt 4). +- Implementierung 30.10.2026: Konten `gn-download` (983), `gn-dashboard` (984) + und Gruppe `gn-media` (GID 970) per `workload_accounts` angelegt — `gn-media` + als **primäre** Gruppe der drei Pipeline-Konten (AUTH_SYS über NFSv4 wirkte + serverseitig nicht mit Supplementary-Gruppen; Befund 30.10.2026, + GNENC-002 Abschnitt 3); Sudoers des Deploy-Kontos auf `(ALL:ALL)` verbreitert + (`become_user` für die Dienstkonten). NFS-Mount auf + `/var/lib/ghostnet-pipeline` aktiv (außerhalb der Homes); Layout auf dem + Export erzeugt und verifiziert; Handover-Simulation + (`verify-pipeline-handover.yml`) einmal vollständig bestanden + (CT-101-`.dlc`-Drop → gn-download → queue → gn-encoding → output/state → + gn-dashboard). Legacy-Medien-Cron auf dem PVE-Host entfernt + (`pve_media_cron`, GNAI-017 Abschnitt 6.11). - Verzeichnislayout: `input//*.dlc`, `jd/`, `queue///`, `work/`, `output///` + `output/_failed/`, `state/jobs/` + `state/system.json`. @@ -71,12 +83,12 @@ beschrieben. ## Noch offen -1. NFSv4.2-Mount-Abnahme von `ghostnet-ki` gegen `10.0.0.2` (inkl. - Schreib-/Lesetest als UID 982/983 und mv-Simulation als `ghost` über SMB) - — mit der Encoding-Implementierung. +1. Output-Pickup über CT-101 (SMB) einmal als geschlossener Ablauf + nachweisen (Nachtest zum 30.10.2026 — der Durchlauf wurde durch den + Server-Shutdown unterbrochen; GNENC-002 Abschnitt 11). 2. Feinjustierung der Ressourcenlimits nach einem repräsentativen Encoding-Test unter gleichzeitiger Inferenzlast. 3. Rootless-TUN-Verifizierung für den VPN-Sidecar auf dem Host. -4. Implementierung auf ghostnet-ki: Konten `gn-download`/`gn-dashboard`, - NFS-Mount, Image-Bauten, Quadlet-Units (inkl. VPN-Sidecar), Skripte, - Monitoring-Bereich (Collector, Dashboard). +4. Implementierung auf ghostnet-ki: Image-Bauten, Quadlet-Units (inkl. + VPN-Sidecar), Worker-Skript, JD2-Event-Scripter, Monitoring-Bereich + (Collector, Dashboard). diff --git a/docs/platform/llm-server/encoding/GNENC-001-PLAN_Architecture.md b/docs/platform/llm-server/encoding/GNENC-001-PLAN_Architecture.md index 77ff6b8..c0d3ff0 100644 --- a/docs/platform/llm-server/encoding/GNENC-001-PLAN_Architecture.md +++ b/docs/platform/llm-server/encoding/GNENC-001-PLAN_Architecture.md @@ -32,7 +32,7 @@ beschrieben. | --- | --- | --- | | Repository | Host- und Deployment-Ansible bleibt im bestehenden Infrastructure-Repository | angenommen | | Dokumentation | Encoding liegt unter `docs/platform/llm-server/encoding/` und verwendet die Kennung `GNENC`; Monitoring unter `docs/platform/llm-server/monitoring/` mit der Kennung `GNMON`; CT 101 unter `docs/platform/proxmox-server/lxc/ct-101-samba/` mit GNAI-017 | umgesetzt | -| Ausführungsidentitäten | drei gesperrte Konten: `gn-encoding` (UID/GID 982, Worker), `gn-download` (983, jDownloader2 + VPN-Sidecar), `gn-dashboard` (984, Dashboard, Monitoring-Bereich); gemeinsame Gruppe `gn-media` (GID 970) für die Übergabe-Verzeichnisse | gn-encoding eingerichtet und geprüft; weitere geplant | +| Ausführungsidentitäten | drei gesperrte Konten: `gn-encoding` (UID 982, Worker), `gn-download` (983, jDownloader2 + VPN-Sidecar), `gn-dashboard` (984, Dashboard, Monitoring-Bereich); gemeinsame Gruppe `gn-media` (GID 970) als **primäre** Gruppe der drei Konten (AUTH_SYS über NFSv4 wirkte serverseitig nur mit primärer Gruppe; Befund 30.10.2026) | alle drei Konten eingerichtet und verifiziert 30.10.2026 (Idempotenz `changed=0`) | | Containertrennung | je Konto ein eigener rootless Podman-Store und ein eigener rootverwalteter Quadlet-Pfad | gn-encoding eingerichtet und verifiziert; weitere geplant | | Rechenressource | Encoding ist CPU-basiert ohne GPU-Zugriff; der host-seitige Metrics-Collector (Monitoring-Bereich) liest GPU-Metriken ausschließlich lesend für das Dashboard | Planungsentscheidung | | Parallelität | darf parallel zu AI Production oder AI Testing laufen | Planungsentscheidung | @@ -41,7 +41,7 @@ beschrieben. | VPN | der gesamte jDownloader2-Verkehr läuft über einen ProtonVPN-WireGuard-Tunnel: gluetun-Sidecar (`qmcgaw/gluetun`, gepinnt) unter `gn-download`, JD2 teilt dessen Netzwerk-Namespace, Kill Switch aktiv (kein Traffic ohne Tunnel); WireGuard-Credentials server-lokal in `/etc/ghostnet-download/vpn.env`, nicht in git | Betreiberentscheidung 19.10.2026 | | Anwendung | Worker-Image als Eigenbau auf Basis des gepinnten HandBrake-Images `linuxserver/handbrake:1.11.2-ls48` (`sha256:fda92bd1…48a3c7`) mit python3 und ffmpeg; die Digests von Worker- und VPN-Image werden nach dem Bau gepinnt | Betreiberentscheidung 19.10.2026 | | Speicherort | Pipeline-Daten liegen auf dem PVE-Host-Medienverbund (`/media/datenhdd3`; per LXC-Mount auch in CT-101 sichtbar): dort liegt die Medienbibliothek; ghostnet-ki selbst hat nur eine 2-TB-NVMe (OS + LLM) | Betreiberentscheidung 19.10.2026; NFS seit 30.10.2026 auf dem PVE-Host (GNAI-019 Entscheidung 2) | -| Ein-/Ausgabe | NFS-Export `10.0.0.2:/media/datenhdd3/Encoding` (PVE-Host, Rolle `pve_nfs_export`, NFSv4-only, nur für 10.0.0.90, `root_squash`), gemountet auf `/var/lib/ghostnet-encoding`; Verzeichnislayout `input/`, `jd/`, `queue/`, `work/`, `output/`, `state/` | Betreiberentscheidung 19.10.2026; Umsetzung GNAI-019 (Revision 30.10.2026); Mount-Abnahme durch das Encoding-Ansible ausstehend | +| Ein-/Ausgabe | NFS-Export `10.0.0.2:/media/datenhdd3/Encoding` (PVE-Host, Rolle `pve_nfs_export`, NFSv4-only, nur für 10.0.0.90, `root_squash`), gemountet auf `/var/lib/ghostnet-pipeline` (außerhalb der Konto-Homes); Verzeichnislayout `input/`, `jd/`, `queue/`, `work/`, `output/`, `state/` | Betreiberentscheidung 19.10.2026; Mount aktiv, Layout und Handover verifiziert 30.10.2026 | | Abholung | fertige Outputs per SMB über CT-101: Share `[Encode]` (`output/`, RW für `ghost`, lesend für `daidalaw`/`debugofen`) — Entnahme per `mv` in die Bibliotheks-Shares räumt `output/` auf; optionaler Clean-Job löscht Dateien nach 30 Tagen aus `output/`; kein smbd auf `ghostnet-ki` | Betreiberentscheidung 19.10.2026 | | Samba-Integration (CT-101) | CT-101 ist per Ansible adoptiert (GNAI-019; verifiziert 30.10.2026, `ok=47 changed=0`): Shares `[Encode]`/`[Encode-In]`/`[Encoding]`, Gruppe `gn-media` (GID 970) mit `ghost` als Mitglied; `.dlc`-Drop über `[Encode-In]`. NFS liegt nicht auf CT-101 (PVE-Host); die guest-seitige GID 970 gleicht die Host-GID 970 wegen des LXC-idmap nicht ab — Berechtigungsmodell: Mode 2777 auf den SMB-berührten Verzeichnissen (Betreiberentscheidung 30.10.2026, GNENC-002 Abschnitt 4) | Betreiberentscheidung 19.10.2026; GNAI-017/GNAI-019 | | Profile | drei versionierte HandBrake-Presets (SVT-AV1 10-bit, MKV, Audio-Passthrough) unter `llm-server/encoding/profiles/`; Auswahl über gleichnamige Ordner unter `input/` bzw. `queue/` | Betreiberentscheidung 19.10.2026; Presets versioniert | @@ -232,20 +232,19 @@ jeweils `ok=30 changed=0 failed=0`. Das gepinnte HandBrake-Image anschließend in den rootless Store von `gn-encoding` gezogen; der Pull bestätigte den Manifest-Digest. -Am 19.10.2026 wurde die Planung abgeschlossen (Abschnitte 2 und 7). Die -externen Abhängigkeiten sind umgesetzt: CT-101 ist per Ansible adoptiert -(GNAI-019, verifiziert 30.10.2026), der NFS-Export läuft auf dem PVE-Host -(Rolle `pve_nfs_export`); der PVE-Host ist konvergiert (Export-Root -`100000:970 2775`, NFSv4-only wirksam, `verify-production` `ok=17 changed=0`, -Idempotenz `changed=0`; autorisierter Lauf 30.10.2026). Das -Berechtigungsmodell für die SMB-Schreibpfade ist entschieden (Mode 2777, -GNENC-002 Abschnitt 4); offener Restpunkt ist die NFSv4.2-Mount-Abnahme von -ghostnet-ki. Die Implementierungsphase auf -ghostnet-ki baut als Nächstes auf: die Konten `gn-download` und `gn-dashboard` -samt Gruppe `gn-media` (GID 970), den NFS-Mount von `10.0.0.2`, die -Image-Bauten mit Digest-Pinning (Worker, VPN-Sidecar, Dashboard), die -Quadlet-Units für jDownloader2, VPN-Sidecar, Worker und Dashboard, den -host-seitigen Metrics-Collector sowie die Skripte aus `scripts/`. Die Rolle +Am 19.10.2026 wurde die Planung abgeschlossen (Abschnitte 2 und 7). Stand +30.10.2026: Die externen Abhängigkeiten sind umgesetzt und verifiziert — +CT-101 per Ansible adoptiert, der PVE-Host konvergiert (Export-Root +`100000:970 2775`, NFSv4-only wirksam, Legacy-Medien-Cron entfernt — GNAI-017 +Abschnitt 6.9–6.11), die drei Pipeline-Konten samt Gruppe `gn-media` +(einschließlich Primärgruppen-Umstellung, GNAI-017 Abschnitt 6.10) angelegt, +der NFS-Mount auf `/var/lib/ghostnet-pipeline` aktiv, das Verzeichnislayout +auf dem Export erzeugt und verifiziert, und die Handover-Simulation +(`verify-pipeline-handover.yml`) einmal vollständig bestanden. Die +Implementierungsphase auf ghostnet-ki baut als Nächstes auf: die Image-Bauten +mit Digest-Pinning (Worker, VPN-Sidecar, Dashboard), die Quadlet-Units für +jDownloader2, VPN-Sidecar, Worker und Dashboard, den host-seitigen +Metrics-Collector sowie die Skripte aus `scripts/`. Die Rolle `video_encoding` stellt bis dahin sicher, dass weder Quadlet-Dateien noch Container für `gn-encoding` existieren. diff --git a/docs/platform/llm-server/encoding/GNENC-002-PLAN_Pipeline.md b/docs/platform/llm-server/encoding/GNENC-002-PLAN_Pipeline.md index ea26fb7..ba4ced4 100644 --- a/docs/platform/llm-server/encoding/GNENC-002-PLAN_Pipeline.md +++ b/docs/platform/llm-server/encoding/GNENC-002-PLAN_Pipeline.md @@ -41,17 +41,21 @@ Gruppe `gn-media`: gemeinsame Gruppe der drei Konten für die Übergabe- Verzeichnisse und den lesenden Zugriff auf `state/`; GID 970. Alle Konten sind gesperrt (Passwort `!`, Shell `/usr/sbin/nologin`, Linger aktiv). -**Anmerkung 30.10.2026 (LXC-idmap):** Die guest-seitige Gruppe `gn-media` -(GID 970) in CT-101 erscheint auf dem Host als GID 100970 und gleicht die -reale Host-GID 970 der Pipeline nicht ab — die Gruppen-Alignment wirkt nur -innerhalb von ghostnet-ki. Die SMB-Schreibpfade aus CT-101 laufen über -guest-root (Host-UID 100000, da `admin users` Dateioperationen als root -ausführt); siehe Abschnitt 11. +**Anmerkung 30.10.2026 (primäre Gruppe):** Die drei Pipeline-Konten tragen +`gn-media` als **primäre** Gruppe (die eigenen GID-Gruppen 982/983/984 bleiben +für die Config-Verzeichnisse als Supplementary-Gruppe erhalten). Grund: +AUTH_SYS über NFSv4 wirkte serverseitig nicht mit Supplementary-Gruppen — +`mkdir` als 982/983 bekam EACCES, obwohl `gn-media` Gruppen-Schreibrechte +hatte; mit primärer Gruppe 970 war der Zugriff sofort erfolgreich +(Betriebsbefund 30.10.2026, Testreihe auf ghostnet-ki). Zudem gilt die +LXC-idmap-Diskrepanz: die guest-seitige Gruppe `gn-media` (GID 970) in CT-101 +erscheint auf dem Host als GID 100970 — die SMB-Schreibpfade laufen über +World-Write (Mode 2777), nicht über Gruppen-Abgleich; siehe Abschnitt 4. ## 4. Verzeichnislayout ```text -/var/lib/ghostnet-encoding/ (NFS-Mount: 10.0.0.2:/media/datenhdd3/Encoding, PVE-Host) +/var/lib/ghostnet-pipeline/ (NFS-Mount: 10.0.0.2:/media/datenhdd3/Encoding, PVE-Host; außerhalb der Konto-Homes) ├── input/ gn-media 2777 SMB-Drop ([Encode-In]) │ └── / gn-media 2777 nur .dlc ├── jd/ gn-media 2770 JD2-Arbeitsverzeichnis (gn-download) @@ -92,16 +96,22 @@ Handover-Regeln: Schreibzugriffen. **Berechtigungsmodell (Betreiberentscheidung 30.10.2026):** Die SMB-Zugriffe -aus CT-101 laufen über guest-root (Host-UID 100000; `admin users` führt -Dateioperationen als root aus). Da die Pipeline-Verzeichnisse via NFS den -realen UIDs 982/983:970 gehören und diese im LXC-idmap nicht abgebildet sind -(im Gast: `nobody:nogroup`), tragen `input/` und der `output/`-Baum Mode -`2777`; `jd/`, `queue/`, `work/` und `state/` bleiben eng (2770/0750). +aus CT-101 laufen über guest-root bzw. beliebige Gast-Identitäten (Host-UIDs +100000+, da `admin users` Dateioperationen als root ausführt); die +Pipeline-Verzeichnisse gehören via NFS den realen UIDs 982/983 mit primärer +Gruppe 970. Da diese im LXC-idmap nicht abgebildet sind, tragen `input/` und +der `output/`-Baum Mode `2777` (World-Write); `jd/`, `queue/`, `work/` und +`state/` bleiben eng (2770/0750/2750). Der frühere Legacy-Cron auf dem +PVE-Host (`chown -R root:root` + `chmod -R 0777` auf allen Medienplatten alle +10 Minuten, installiert 11.10.2026) hätte jedes echte +Berechtigungsmodell plattgewalzt und wurde am 30.10.2026 entfernt (Rolle +`pve_media_cron`, Backup der Crontab unter `/var/backups/ghostnet/pve/`). **Layout-Anlage:** Wegen `root_squash` auf dem Export kann Ansible als root keine chowned-Verzeichnisse anlegen; die Struktur wird als `gn-encoding` -angelegt (runuser; Muster der Rolle `video_encoding`) oder vom Worker beim -Start erzeugt und geprüft. +beziehungsweise `gn-download` angelegt (`become_user`; die Rolle +`video_encoding` erzeugt und verifiziert das Layout vollständig — Stand +30.10.2026 umgesetzt, Verifikation bestanden). **Abholung:** Die fertigen Outputs werden über CT-101 per SMB abgeholt: Share `[Encode]` (Pfad `/media/datenhdd3/Encoding/output`, RW für `ghost`, lesend für @@ -254,9 +264,13 @@ die Container gemountet (Updates ohne Image-Rebuild, versioniert über git). ## 11. Offene Punkte -1. NFSv4.2-Mount-Abnahme von `ghostnet-ki` gegen `10.0.0.2` — inkl. - Schreib-/Lesetest als UID 982/983 über den Mount und mv-Simulation als - `ghost` über SMB; mit der Encoding-Implementierung. +1. **Output-Pickup über CT-101 (SMB) einmal als geschlossener Ablauf + nachweisen:** Die Handover-Simulation (`verify-pipeline-handover.yml`) + lief am 30.10.2026 vollständig durch (`.dlc`-Drop aus CT-101 → + gn-download → queue → gn-encoding → output/state → gn-dashboard); das + Lesen/Löschen des Output-Testreleases über CT-101 wurde durch den + gleichzeitigen Server-Shutdown nicht mehr abgeschlossen und ist als + kurzer Nachtest zu wiederholen. 2. Feinjustierung der Ressourcenlimits nach Lasttest unter Inferenzlast (GNENC-001 Abschnitt 4). 3. Rootless-TUN-Verifizierung für den VPN-Sidecar auf dem Host (Fallback: diff --git a/docs/platform/llm-server/monitoring/GNMON-001-PLAN_Architecture.md b/docs/platform/llm-server/monitoring/GNMON-001-PLAN_Architecture.md index d4ee19a..6a639b5 100644 --- a/docs/platform/llm-server/monitoring/GNMON-001-PLAN_Architecture.md +++ b/docs/platform/llm-server/monitoring/GNMON-001-PLAN_Architecture.md @@ -42,7 +42,7 @@ Encoding-Pipeline). Keine GPU-Gruppenmitgliedschaft, keine `/dev/kfd`-/ ## 5. Metrics-Collector - Sampling alle 10 s; atomares Schreiben (`state/tmp/` + `mv`) nach - `/var/lib/ghostnet-encoding/state/system.json` (Mode `0644`). + `/var/lib/ghostnet-pipeline/state/system.json` (Mode `0644`). - Schema: ```json diff --git a/docs/platform/proxmox-server/lxc/ct-101-samba/GNAI-017-PLAN_Samba-NFS-Export.md b/docs/platform/proxmox-server/lxc/ct-101-samba/GNAI-017-PLAN_Samba-NFS-Export.md index d9e2d93..e7e2652 100644 --- a/docs/platform/proxmox-server/lxc/ct-101-samba/GNAI-017-PLAN_Samba-NFS-Export.md +++ b/docs/platform/proxmox-server/lxc/ct-101-samba/GNAI-017-PLAN_Samba-NFS-Export.md @@ -328,4 +328,16 @@ GNENC-002 Abschnitt 4): `input/`, `jd/`, `queue/`, `work/`, `output/`, `input/`-Baum und dem gesamten `output/`-Baum (inkl. Release- und `_failed/`-Verzeichnisse, vom Worker angelegt); `jd/`, `queue/`, `work/` und `state/` bleiben eng (2770/2750/0750). Festgehalten in GNENC-002 - Abschnitt 4. + Abschnitt 4. **Betriebsbefund 30.10.2026:** AUTH_SYS über NFSv4 wirkte + serverseitig nicht mit Supplementary-Gruppen — die Pipeline-Konten auf + ghostnet-ki tragen `gn-media` daher als **primäre** Gruppe (Befund und + Umsetzung: GNENC-002 Abschnitt 3). +11. **Legacy-Medien-Cron auf dem PVE-Host — erledigt (30.10.2026):** Die + root-Crontab enthielt seit dem 11.10.2026 zehnminütliche + `chown -R root:root`- und `chmod -R 0777`-Läufe über alle Medienplatten + (daten1/2/3, datenhdd1, datenhdd3) — sie hatte den Export-Root nach jeder + Konvergenz zurückgesetzt und hätte jedes echte Berechtigungsmodell + plattgewalzt. Entfernt über die Rolle `pve_media_cron` im Host-Projekt + (`proxmox/pve-host/ansible/`, Betreiberentscheidung 30.10.2026); Backup + der Crontab unter `/var/backups/ghostnet/pve/`; `verify-production` des + Host-Projekts prüft das Fehlen seither ab. diff --git a/llm-server/encoding/ansible/README.md b/llm-server/encoding/ansible/README.md index 3cd304d..47fac21 100644 --- a/llm-server/encoding/ansible/README.md +++ b/llm-server/encoding/ansible/README.md @@ -7,11 +7,17 @@ area of the LLM-Server host root; the shared host baseline lives in `playbooks/video-encoding.yml` applies the `video_encoding` role: it verifies the locked `gn-encoding` account, enforces CPU-only device isolation, manages -the encoding directories and proves the separate rootless Podman store. No -encoder image, Quadlet, timer or schedule is defined yet; those are open -decisions in +the pipeline NFS mount (`10.0.0.2:/media/datenhdd3/Encoding` at +`/var/lib/ghostnet-pipeline`) and creates and verifies the pipeline +directory layout on the export as the service accounts. No encoder image, +Quadlet, timer or schedule is defined yet; those are open decisions in [`../../../docs/platform/llm-server/encoding/`](../../../docs/platform/llm-server/encoding/GNENC-000-DOC_Index.md). +`playbooks/verify-pipeline-handover.yml` exercises the file handover between +the pipeline identities (GNENC-002 sections 5 and 6) on the mounted export; +apply `video-encoding.yml` first. The SMB write path from CT-101 (guest root) +is simulated separately on CT-101 (GNAI-017 section 6.10). + ## Control-node setup Use the shared virtual environment at the repository root (see diff --git a/llm-server/encoding/ansible/playbooks/verify-pipeline-handover.yml b/llm-server/encoding/ansible/playbooks/verify-pipeline-handover.yml new file mode 100644 index 0000000..b5aa776 --- /dev/null +++ b/llm-server/encoding/ansible/playbooks/verify-pipeline-handover.yml @@ -0,0 +1,205 @@ +--- +# Exercises the GNENC-002 file handover between the pipeline identities on the +# mounted NFS export: .dlc drop -> gn-download ingest -> jd/ -> queue/ -> +# gn-encoding pick-up -> work/ -> output/ -> state/ -> gn-dashboard read. +# Apply playbooks/video-encoding.yml first. The output test release is left in +# place for the CT-101 SMB pickup simulation and removed on the next run. +- name: Simulate the encoding pipeline file handover (GNENC-002 sections 5 and 6) + hosts: llm_servers + become: true + gather_facts: false + + vars: + handover_profile: "AV1 MKV 1080p30 RF25" + handover_release: handover-test + handover_mount: /var/lib/ghostnet-pipeline + handover_dlc: "{{ handover_mount }}/input/{{ handover_profile }}/{{ handover_release }}.dlc" + handover_dlc_done: "{{ handover_mount }}/jd/dlc_done/{{ handover_release }}.dlc" + handover_jd_release: "{{ handover_mount }}/jd/{{ handover_release }}" + handover_queue_release: "{{ handover_mount }}/queue/{{ handover_profile }}/{{ handover_release }}" + handover_work_release: "{{ handover_mount }}/work/{{ handover_release }}" + handover_output_release: "{{ handover_mount }}/output/{{ handover_profile }}/{{ handover_release }}" + handover_output_file: "{{ handover_output_release }}/video.test.mkv" + handover_jobs: "{{ handover_mount }}/state/jobs" + handover_tmp: "{{ handover_mount }}/state/tmp" + handover_job_file: "{{ handover_jobs }}/handover-test.json" + + tasks: + - name: Require the pipeline mount to be active + ansible.builtin.command: + argv: + - /usr/bin/systemctl + - is-active + - var-lib-ghostnet\x2dpipeline.mount + changed_when: false + register: handover_mount_status + failed_when: handover_mount_status.stdout != "active" + + - name: Stat the handover directories + ansible.builtin.stat: + path: "{{ item }}" + loop: + - "{{ handover_mount }}/input/{{ handover_profile }}" + - "{{ handover_queue_release }}" + - "{{ handover_output_release }}" + - "{{ handover_mount }}/jd" + - "{{ handover_mount }}/work" + - "{{ handover_jobs }}" + - "{{ handover_tmp }}" + register: handover_dir_stats + + - name: Require the pipeline layout to be present + ansible.builtin.assert: + that: >- + handover_dir_stats.results | map(attribute='stat.isdir') + | select('equalto', false) | list | length == 0 + fail_msg: >- + The pipeline layout is missing; apply playbooks/video-encoding.yml + first (GNENC-002 section 4). + + - name: Clean up leftovers from previous runs (gn-download side) + ansible.builtin.shell: | + set -euo pipefail + rm -f "{{ handover_dlc }}" "{{ handover_dlc_done }}" + rm -rf "{{ handover_jd_release }}" "{{ handover_queue_release }}" + args: + executable: /bin/bash + become_user: gn-download + environment: + HOME: /var/lib/ghostnet-download + changed_when: true + + - name: Clean up leftovers from previous runs (gn-encoding side) + ansible.builtin.shell: | + set -euo pipefail + rm -rf "{{ handover_work_release }}" "{{ handover_output_release }}" + rm -f "{{ handover_job_file }}" + args: + executable: /bin/bash + become_user: gn-encoding + environment: + HOME: /var/lib/ghostnet-encoding + changed_when: true + + - name: Probe the .dlc drop + ansible.builtin.stat: + path: "{{ handover_dlc }}" + register: handover_dlc_probe + + - name: Provide the .dlc drop as gn-encoding when no external SMB simulation file exists + ansible.builtin.shell: | + set -euo pipefail + printf 'dlc-handover-test\n' > "{{ handover_dlc }}" + args: + executable: /bin/bash + become_user: gn-encoding + environment: + HOME: /var/lib/ghostnet-encoding + when: not handover_dlc_probe.stat.exists + + - name: Ingest the .dlc as gn-download (read and move to jd/dlc_done) + ansible.builtin.shell: | + set -euo pipefail + cat "{{ handover_dlc }}" > /dev/null + mv "{{ handover_dlc }}" "{{ handover_dlc_done }}" + [ -f "{{ handover_dlc_done }}" ] + args: + executable: /bin/bash + become_user: gn-download + environment: + HOME: /var/lib/ghostnet-download + changed_when: true + + - name: Simulate the jDownloader2 download in jd/ + ansible.builtin.shell: | + set -euo pipefail + mkdir "{{ handover_jd_release }}" + dd if=/dev/zero of="{{ handover_jd_release }}/video.test" bs=1M count=1 status=none + args: + executable: /bin/bash + become_user: gn-download + environment: + HOME: /var/lib/ghostnet-download + changed_when: true + + - name: Hand the release over to the queue (atomic move) + ansible.builtin.shell: | + set -euo pipefail + mv "{{ handover_jd_release }}" "{{ handover_queue_release }}" + [ -f "{{ handover_queue_release }}/video.test" ] + args: + executable: /bin/bash + become_user: gn-download + environment: + HOME: /var/lib/ghostnet-download + changed_when: true + + - name: Pick up, encode and hand over to output as gn-encoding + ansible.builtin.shell: | + set -euo pipefail + cat "{{ handover_queue_release }}/video.test" > /dev/null + mkdir "{{ handover_work_release }}" + dd if=/dev/zero of="{{ handover_work_release }}/video.test.mkv" bs=1M count=1 status=none + mv "{{ handover_work_release }}/video.test.mkv" "{{ handover_output_file }}" + rm "{{ handover_queue_release }}/video.test" + rmdir "{{ handover_queue_release }}" "{{ handover_work_release }}" + [ -f "{{ handover_output_file }}" ] + args: + executable: /bin/bash + become_user: gn-encoding + environment: + HOME: /var/lib/ghostnet-encoding + changed_when: true + + - name: Write the state JSON as gn-encoding (atomic via state/tmp) + ansible.builtin.shell: | + set -euo pipefail + printf '{"job_id":"handover-test","status":"done"}\n' > "{{ handover_tmp }}/handover-test.json" + mv "{{ handover_tmp }}/handover-test.json" "{{ handover_job_file }}" + [ -f "{{ handover_job_file }}" ] + args: + executable: /bin/bash + become_user: gn-encoding + environment: + HOME: /var/lib/ghostnet-encoding + changed_when: true + + - name: Read the state JSON as gn-dashboard + ansible.builtin.shell: | + set -euo pipefail + grep -q '"status":"done"' "{{ handover_job_file }}" + args: + executable: /bin/bash + become_user: gn-dashboard + environment: + HOME: /var/lib/ghostnet-dashboard + changed_when: false + + - name: Clean up the state test object + ansible.builtin.shell: | + set -euo pipefail + rm -f "{{ handover_job_file }}" + args: + executable: /bin/bash + become_user: gn-encoding + environment: + HOME: /var/lib/ghostnet-encoding + changed_when: true + + - name: Stat the handed-over output file + ansible.builtin.stat: + path: "{{ handover_output_file }}" + register: handover_output_stats + + - name: Verify the output file identity + ansible.builtin.assert: + that: + - handover_output_stats.stat.exists + - handover_output_stats.stat.uid | int == 982 + - handover_output_stats.stat.gid | int == 970 + fail_msg: >- + The output file must exist as 982:970 on the export; found + {{ handover_output_stats.stat.uid }}:{{ handover_output_stats.stat.gid }}. + success_msg: >- + Handover simulation complete; the output test release remains for the + CT-101 SMB pickup simulation and is removed on the next run. diff --git a/llm-server/encoding/ansible/roles/video_encoding/README.md b/llm-server/encoding/ansible/roles/video_encoding/README.md index 3a2acb9..4ba39dc 100644 --- a/llm-server/encoding/ansible/roles/video_encoding/README.md +++ b/llm-server/encoding/ansible/roles/video_encoding/README.md @@ -6,17 +6,32 @@ baseline for the locked `gn-encoding` service account (UID/GID 982) on It verifies the account identity, the locked password and the deterministic subordinate ID ranges, enforces the absence of GPU device groups, manages the -root-owned configuration directory, the input/output/work data directories and -the administrator-managed rootless Quadlet directory, and proves that the -account cannot read `/dev/kfd` or any DRM render node and uses its own rootless -Podman store. +root-owned configuration directory and the administrator-managed rootless +Quadlet directory, and proves that the account cannot read `/dev/kfd` or any +DRM render node and uses its own rootless Podman store. + +Pipeline storage comes from the PVE-host NFS export +(`10.0.0.2:/media/datenhdd3/Encoding`, GNAI-017/GNAI-019), mounted at +`/var/lib/ghostnet-encoding/pipeline` inside the account home — the home +itself is not the mountpoint, so the rootless Podman store below it stays +visible. The role installs the NFS client, manages the systemd mount unit and +creates the pipeline directory layout (`input/`, `jd/`, `queue/`, `work/`, +`output/`, `state/`, one subdirectory per HandBrake profile) as the service +accounts over NFS, because `root_squash` prevents root-owned creation. The +layout ownership and modes (2777 on the SMB-facing trees, 2770/0750/2750 +elsewhere, group `gn-media` GID 970) are verified after convergence +(GNENC-002 section 4). While no encoder image is selected (`video_encoding_image` empty), the role additionally guards that no Quadlet file and no container exists for the -account. Encoder image, schedule, resource limits, queue handling and health -checks are open decisions in -`docs/platform/llm-server/encoding/GNENC-001` and are not assumed here. +account. Encoder image, schedule, resource limits and health checks remain +gated by the open decisions in `docs/platform/llm-server/encoding/GNENC-001` +and are not assumed here. The account itself is created by the shared `workload_accounts` role in the -host project (`llm-server/host/ansible`); this role requires the host baseline -to be applied first. +host project (`llm-server/host/ansible`), together with `gn-download` (983), +`gn-dashboard` (984) and the shared group `gn-media` (GID 970); this role +requires the host baseline to be applied first. + +The file handover between the pipeline identities is exercised by +`playbooks/verify-pipeline-handover.yml`. diff --git a/llm-server/encoding/ansible/roles/video_encoding/defaults/main.yml b/llm-server/encoding/ansible/roles/video_encoding/defaults/main.yml index c80d940..24b9fba 100644 --- a/llm-server/encoding/ansible/roles/video_encoding/defaults/main.yml +++ b/llm-server/encoding/ansible/roles/video_encoding/defaults/main.yml @@ -8,6 +8,10 @@ video_encoding_account: name: gn-encoding uid: 982 gid: 982 + # Primary login group is gn-media (GID 970): AUTH_SYS over NFSv4 did not + # honour the supplementary group membership server-side, so the pipeline + # identities carry gn-media as their primary group (verified 2026-10-30). + primary_gid: 970 home: /var/lib/ghostnet-encoding config_directory: /etc/ghostnet-encoding shell: /usr/sbin/nologin @@ -15,18 +19,46 @@ video_encoding_account: video_encoding_subid_count: 65536 -video_encoding_directories: - - path: /var/lib/ghostnet-encoding/input - mode: "0750" - - path: /var/lib/ghostnet-encoding/output - mode: "0750" - - path: /var/lib/ghostnet-encoding/work - mode: "0750" +# Pipeline storage on the PVE-host NFS export (GNENC-001/002, GNAI-017). +# The mount lives OUTSIDE the service homes (the gn-encoding home is 0750 and +# must stay traversable only for its owner; gn-download and gn-dashboard need +# to reach the pipeline root without touching any account home). +video_encoding_nfs_server: 10.0.0.2 +video_encoding_nfs_export: /media/datenhdd3/Encoding +video_encoding_pipeline_mount: /var/lib/ghostnet-pipeline +video_encoding_pipeline_mount_unit: var-lib-ghostnet\x2dpipeline.mount +video_encoding_pipeline_mount_options: nfsvers=4.2,hard,_netdev,noatime +video_encoding_pipeline_nfs_client_package: nfs-common + +# Export root ownership on the PVE host (GNAI-017 section 4.1): +# 100000 = CT-101 guest root (SMB share writes), 970 = gn-media (pipeline). +video_encoding_export_root_uid: 100000 +video_encoding_export_root_gid: 970 +video_encoding_export_root_mode: "2775" + +# Shared pipeline group of gn-encoding (982), gn-download (983) and +# gn-dashboard (984); created by the workload_accounts role. +video_encoding_media_group: gn-media +video_encoding_media_gid: 970 + +# Declared HandBrake profiles (must match the preset files under +# llm-server/encoding/profiles/; verified by the role, one input/, queue/ +# and output/ subdirectory per profile). +video_encoding_profile_names: + - "AV1 MKV 1080p30 Anime RF34" + - "AV1 MKV 1080p30 RF25" + - "AV1 MKV 2160p60 4K RF23" + +# Superseded local directories of the pre-NFS baseline (GNENC-001 section 6); +# removed when empty. +video_encoding_legacy_local_directories: + - /var/lib/ghostnet-encoding/input + - /var/lib/ghostnet-encoding/output + - /var/lib/ghostnet-encoding/work video_encoding_quadlet_user_directory: /etc/containers/systemd/users # Open decisions from docs/platform/llm-server/encoding/GNENC-001 section 7 # stay unset on purpose. Until an encoder image, a schedule and resource limits -# are decided, the role only manages the shared encoding baseline and guards -# that no workload is defined. +# are decided, the role additionally guards that no workload is defined. video_encoding_image: "" diff --git a/llm-server/encoding/ansible/roles/video_encoding/tasks/main.yml b/llm-server/encoding/ansible/roles/video_encoding/tasks/main.yml index 31335f8..64050e4 100644 --- a/llm-server/encoding/ansible/roles/video_encoding/tasks/main.yml +++ b/llm-server/encoding/ansible/roles/video_encoding/tasks/main.yml @@ -46,7 +46,7 @@ video_encoding_account.uid | int - >- video_encoding_account_entry.stdout.split(":")[3] | int == - video_encoding_account.gid | int + video_encoding_account.primary_gid | int - >- video_encoding_account_entry.stdout.split(":")[5] == video_encoding_account.home @@ -146,17 +146,264 @@ group: "{{ video_encoding_account.gid }}" mode: "0750" -- name: Manage the encoding data directories +- name: Remove the superseded local pipeline directories when empty + ansible.builtin.shell: | + set -euo pipefail + changed=0 + for dir in {{ video_encoding_legacy_local_directories | map('quote') | join(' ') }}; do + if [ -d "$dir" ]; then + if [ -n "$(find "$dir" -mindepth 1 -print -quit)" ]; then + echo "Refusing to remove non-empty legacy directory: $dir" + exit 1 + fi + rmdir "$dir" + changed=1 + fi + done + [ "$changed" = 0 ] || echo CHANGED + args: + executable: /bin/bash + register: video_encoding_legacy_cleanup + changed_when: "'CHANGED' in video_encoding_legacy_cleanup.stdout" + +- name: Install the NFS client + ansible.builtin.apt: + name: "{{ video_encoding_pipeline_nfs_client_package }}" + state: present + update_cache: true + cache_valid_time: 3600 + install_recommends: false + +- name: Probe whether the pipeline mount is already active + ansible.builtin.command: + argv: + - /usr/bin/findmnt + - --mountpoint + - "{{ video_encoding_pipeline_mount }}" + check_mode: false + changed_when: false + failed_when: false + register: video_encoding_pipeline_mountpoint_probe + +- name: Create the pipeline mountpoint outside the service homes ansible.builtin.file: - path: "{{ video_encoding_directory.path }}" + path: "{{ video_encoding_pipeline_mount }}" state: directory - owner: "{{ video_encoding_account.uid }}" - group: "{{ video_encoding_account.gid }}" - mode: "{{ video_encoding_directory.mode }}" - loop: "{{ video_encoding_directories }}" - loop_control: - loop_var: video_encoding_directory - label: "{{ video_encoding_directory.path }}" + owner: root + group: root + mode: "0755" + when: video_encoding_pipeline_mountpoint_probe.rc != 0 + +- name: Manage the pipeline NFS mount unit + ansible.builtin.copy: + content: | + [Unit] + Description=Ghostnet encoding pipeline NFS export (GNENC-002) + Requires=network-online.target + After=network-online.target + + [Mount] + What={{ video_encoding_nfs_server }}:{{ video_encoding_nfs_export }} + Where={{ video_encoding_pipeline_mount }} + Type=nfs4 + Options={{ video_encoding_pipeline_mount_options }} + + [Install] + WantedBy=multi-user.target + dest: "/etc/systemd/system/{{ video_encoding_pipeline_mount_unit }}" + owner: root + group: root + mode: "0644" + +- name: Reload systemd for the pipeline mount unit + ansible.builtin.systemd_service: + daemon_reload: true + when: not ansible_check_mode + +- name: Enable and mount the pipeline NFS export + ansible.builtin.systemd_service: + name: "{{ video_encoding_pipeline_mount_unit }}" + enabled: true + state: started + when: not ansible_check_mode + +- name: Verify the pipeline mount is active + ansible.builtin.command: + argv: + - /usr/bin/systemctl + - is-active + - "{{ video_encoding_pipeline_mount_unit }}" + check_mode: false + changed_when: false + register: video_encoding_pipeline_mount_status + failed_when: video_encoding_pipeline_mount_status.stdout != "active" + when: not ansible_check_mode + +- name: Stat the mounted export root + ansible.builtin.stat: + path: "{{ video_encoding_pipeline_mount }}" + register: video_encoding_export_root + when: not ansible_check_mode + +- name: Verify the mounted export root ownership and mode + ansible.builtin.assert: + that: + - video_encoding_export_root.stat.isdir + - video_encoding_export_root.stat.uid == video_encoding_export_root_uid | int + - video_encoding_export_root.stat.gid == video_encoding_export_root_gid | int + - >- + '%04o' % (video_encoding_export_root.stat.mode | int(0, 8)) == + video_encoding_export_root_mode + fail_msg: >- + The mounted export root must be {{ video_encoding_export_root_uid }}:{{ video_encoding_export_root_gid }} + {{ video_encoding_export_root_mode }} (PVE host side, GNAI-017 section 4.1); + found {{ video_encoding_export_root.stat.uid }}:{{ video_encoding_export_root.stat.gid }} + {{ video_encoding_export_root.stat.mode }}. + success_msg: The pipeline export root is mounted with the declared identity. + when: not ansible_check_mode + +- name: Collect the HandBrake preset files from the profiles directory + ansible.builtin.set_fact: + video_encoding_profile_files: >- + {{ lookup('ansible.builtin.fileglob', role_path ~ '/../../../profiles/*.json', wantlist=True) + | map('basename') | map('regex_replace', '\.json$', '') | list }} + +- name: Require the declared profiles to match the versioned preset files + ansible.builtin.assert: + that: + - video_encoding_profile_names | sort == video_encoding_profile_files | sort + fail_msg: >- + The declared video_encoding_profile_names do not match the preset files + under llm-server/encoding/profiles/ (found: + {{ video_encoding_profile_files | join(', ') }}). + success_msg: The declared profiles match the versioned preset files. + +- name: Create the gn-download pipeline directories on the export + ansible.builtin.shell: | + set -euo pipefail + changed=0 + while read -r mode path; do + [ -n "${path:-}" ] || continue + if [ ! -d "$path" ]; then + install -d -m "$mode" "$path" + changed=1 + elif [ "$(stat -c '%a' "$path")" != "$mode" ]; then + chmod "$mode" "$path" + changed=1 + fi + done <<'EOF' + {% for d in [ + (video_encoding_pipeline_mount ~ '/input', '2777'), + (video_encoding_pipeline_mount ~ '/jd', '2770'), + (video_encoding_pipeline_mount ~ '/jd/dlc_done', '2770'), + (video_encoding_pipeline_mount ~ '/jd/failed', '2770'), + (video_encoding_pipeline_mount ~ '/queue', '2770') ] %} + {{ d[1] }} {{ d[0] }} + {% endfor %} + {% for p in video_encoding_profile_names %} + 2777 {{ video_encoding_pipeline_mount }}/input/{{ p }} + 2770 {{ video_encoding_pipeline_mount }}/queue/{{ p }} + {% endfor %} + EOF + [ "$changed" = 0 ] || echo CHANGED + args: + executable: /bin/bash + become: true + become_user: gn-download + environment: + HOME: /var/lib/ghostnet-download + register: video_encoding_download_layout + changed_when: "'CHANGED' in video_encoding_download_layout.stdout" + when: not ansible_check_mode + +- name: Create the gn-encoding pipeline directories on the export + ansible.builtin.shell: | + set -euo pipefail + changed=0 + while read -r mode path; do + [ -n "${path:-}" ] || continue + if [ ! -d "$path" ]; then + install -d -m "$mode" "$path" + changed=1 + elif [ "$(stat -c '%a' "$path")" != "$mode" ]; then + chmod "$mode" "$path" + changed=1 + fi + done <<'EOF' + {% for d in [ + (video_encoding_pipeline_mount ~ '/work', '0750'), + (video_encoding_pipeline_mount ~ '/output', '2777'), + (video_encoding_pipeline_mount ~ '/output/_failed', '2777'), + (video_encoding_pipeline_mount ~ '/state', '2750'), + (video_encoding_pipeline_mount ~ '/state/jobs', '0750'), + (video_encoding_pipeline_mount ~ '/state/tmp', '0750') ] %} + {{ d[1] }} {{ d[0] }} + {% endfor %} + {% for p in video_encoding_profile_names %} + 2777 {{ video_encoding_pipeline_mount }}/output/{{ p }} + {% endfor %} + EOF + [ "$changed" = 0 ] || echo CHANGED + args: + executable: /bin/bash + become: true + become_user: gn-encoding + environment: + HOME: "{{ video_encoding_account.home }}" + register: video_encoding_worker_layout + changed_when: "'CHANGED' in video_encoding_worker_layout.stdout" + when: not ansible_check_mode + +- name: Verify the pipeline directory layout on the export + ansible.builtin.shell: | + set -uo pipefail + failed=0 + while read -r uid gid mode path; do + [ -n "${path:-}" ] || continue + if [ ! -d "$path" ]; then + echo "MISSING $path" + failed=1 + continue + fi + read -r auid agid amode <<<"$(stat -c '%u %g %a' "$path")" + if [ "$auid" != "$uid" ] || [ "$agid" != "$gid" ] || [ "$amode" != "$mode" ]; then + echo "WRONG $path (expected $uid:$gid $mode, found $auid:$agid $amode)" + failed=1 + fi + done <<'EOF' + {% for d in [ + (video_encoding_pipeline_mount ~ '/input', '983', '2777'), + (video_encoding_pipeline_mount ~ '/jd', '983', '2770'), + (video_encoding_pipeline_mount ~ '/jd/dlc_done', '983', '2770'), + (video_encoding_pipeline_mount ~ '/jd/failed', '983', '2770'), + (video_encoding_pipeline_mount ~ '/queue', '983', '2770'), + (video_encoding_pipeline_mount ~ '/work', '982', '0750'), + (video_encoding_pipeline_mount ~ '/output', '982', '2777'), + (video_encoding_pipeline_mount ~ '/output/_failed', '982', '2777'), + (video_encoding_pipeline_mount ~ '/state', '982', '2750'), + (video_encoding_pipeline_mount ~ '/state/jobs', '982', '0750'), + (video_encoding_pipeline_mount ~ '/state/tmp', '982', '0750') ] %} + {{ d[1] }} {{ d[2] }} {{ d[0] }} + {% endfor %} + {% for p in video_encoding_profile_names %} + 983 970 2777 {{ video_encoding_pipeline_mount }}/input/{{ p }} + 983 970 2770 {{ video_encoding_pipeline_mount }}/queue/{{ p }} + 982 970 2777 {{ video_encoding_pipeline_mount }}/output/{{ p }} + {% endfor %} + EOF + if [ "$failed" = 1 ]; then + echo "Pipeline layout verification failed" + exit 1 + fi + echo LAYOUT_OK + args: + executable: /bin/bash + become_user: gn-encoding + environment: + HOME: "{{ video_encoding_account.home }}" + changed_when: false + register: video_encoding_layout_verify + when: not ansible_check_mode - name: Manage the administrator-managed rootless Quadlet directory ansible.builtin.file: @@ -330,7 +577,8 @@ msg: account: "gn-encoding (UID/GID 982)" config_directory: /etc/ghostnet-encoding - data_directories: "{{ video_encoding_directories | map(attribute='path') | list }}" + pipeline_mount: "{{ video_encoding_nfs_server }}:{{ video_encoding_nfs_export }} -> {{ video_encoding_pipeline_mount }}" + pipeline_profiles: "{{ video_encoding_profile_names | default([]) }}" quadlet_directory: /etc/containers/systemd/users/982 podman_store: "{{ video_encoding_podman_info.store.graphRoot }}" gpu_access: none (verified) diff --git a/llm-server/host/ansible/roles/host_baseline/defaults/main.yml b/llm-server/host/ansible/roles/host_baseline/defaults/main.yml index 2144ff2..abd4a35 100644 --- a/llm-server/host/ansible/roles/host_baseline/defaults/main.yml +++ b/llm-server/host/ansible/roles/host_baseline/defaults/main.yml @@ -14,3 +14,10 @@ host_baseline_directories: mode: "0755" - path: /var/lib/ghostnet-ai/shared mode: "0755" + +# Deployment sudoers drop-in (bootstrap-created 2026-09, adopted here). +# (ALL:ALL) lets Ansible become the locked service accounts (become_user) +# without a password; the deploy key is treated as root-equivalent anyway +# (AGENTS.md) — with (root) alone, runas to other users would prompt. +host_baseline_deploy_sudoers_path: /etc/sudoers.d/90-ansible-deploy +host_baseline_deploy_sudoers_content: "ansible-deploy ALL=(ALL:ALL) NOPASSWD: ALL\n" diff --git a/llm-server/host/ansible/roles/host_baseline/tasks/main.yml b/llm-server/host/ansible/roles/host_baseline/tasks/main.yml index 2f611e2..8ce526c 100644 --- a/llm-server/host/ansible/roles/host_baseline/tasks/main.yml +++ b/llm-server/host/ansible/roles/host_baseline/tasks/main.yml @@ -57,3 +57,12 @@ loop: "{{ host_baseline_directories }}" loop_control: label: "{{ item.path }}" + +- name: Manage the deployment sudoers drop-in + ansible.builtin.copy: + content: "{{ host_baseline_deploy_sudoers_content }}" + dest: "{{ host_baseline_deploy_sudoers_path }}" + owner: root + group: root + mode: "0440" + validate: /usr/sbin/visudo -cf %s diff --git a/llm-server/host/ansible/roles/workload_accounts/README.md b/llm-server/host/ansible/roles/workload_accounts/README.md index 531e203..d6b5a30 100644 --- a/llm-server/host/ansible/roles/workload_accounts/README.md +++ b/llm-server/host/ansible/roles/workload_accounts/README.md @@ -1,17 +1,21 @@ # Workload accounts The `workload_accounts` role creates separate locked service identities for AI -Production, AI Testing and CPU-based video encoding. Each identity receives a -stable UID/GID, a non-overlapping subordinate ID range, a private data root, an -administrator-managed rootless Quadlet directory and a persistent systemd user -manager. +Production, AI Testing, video encoding, the download workload (jDownloader2 +plus VPN sidecar) and the monitoring dashboard. Each identity receives a +stable UID/GID, a non-overlapping subordinate ID range, a private data root, +an administrator-managed rootless Quadlet directory and a persistent systemd +user manager. The three encoding-pipeline identities (`gn-encoding` 982, +`gn-download` 983, `gn-dashboard` 984) share the supplementary group +`gn-media` (GID 970) for the pipeline handover directories (GNENC-002). Configuration remains owned by root and is readable by the corresponding service group. Workload data and the rootless Podman store belong to the -service account. The role verifies that all three Podman stores are rootless -and use distinct graph roots. +service account. The role verifies that all Podman stores are rootless and +use distinct graph roots. The two AI accounts receive `render` as an explicit supplementary-group -baseline; the encoding account receives no GPU group. Hardware discovery and -effective device-access checks belong to `amd_gpu_access`. This role does not -define containers, networks, schedules or resource limits. +baseline; the three pipeline accounts receive no GPU group. This role does not +define containers, networks, schedules or resource limits. Pipeline data +directories live on the NFS export and are managed by the `video_encoding` +role under the pipeline mount (GNENC-002 section 4). diff --git a/llm-server/host/ansible/roles/workload_accounts/defaults/main.yml b/llm-server/host/ansible/roles/workload_accounts/defaults/main.yml index a154365..8e7eeab 100644 --- a/llm-server/host/ansible/roles/workload_accounts/defaults/main.yml +++ b/llm-server/host/ansible/roles/workload_accounts/defaults/main.yml @@ -25,21 +25,50 @@ workload_accounts: home: /var/lib/ghostnet-encoding config_directory: /etc/ghostnet-encoding subid_start: 493216 - supplementary_groups: [] + # Primary group gn-media: with AUTH_SYS over NFSv4 the server-side + # permission check did not honour the supplementary group membership + # (mkdir EACCES despite group rwx; verified 2026-10-30). The pipeline + # identities therefore carry gn-media as their primary group; the + # per-UID groups remain for the root-owned config directories. + primary_group: gn-media + supplementary_groups: + - gn-media + - gn-encoding + - name: gn-download + uid: 983 + gid: 983 + comment: Ghostnet download workload (jDownloader2, VPN sidecar) + home: /var/lib/ghostnet-download + config_directory: /etc/ghostnet-download + subid_start: 558752 + primary_group: gn-media + supplementary_groups: + - gn-media + - gn-download + - name: gn-dashboard + uid: 984 + gid: 984 + comment: Ghostnet monitoring dashboard workload + home: /var/lib/ghostnet-dashboard + config_directory: /etc/ghostnet-dashboard + subid_start: 624288 + primary_group: gn-media + supplementary_groups: + - gn-media + - gn-dashboard + +# Shared group of the encoding pipeline accounts (GNENC-002 section 3). +# GID 970 is also declared on CT-101 (GNAI-017 section 4.2); note that the +# CT-101 guest-side GID does not match the host-side GID through the LXC +# idmap — see GNENC-002 sections 3 and 4. +workload_shared_groups: + - name: gn-media + gid: 970 workload_subid_count: 65536 workload_quadlet_user_directory: /etc/containers/systemd/users -workload_data_directories: - - path: /var/lib/ghostnet-encoding/input - owner: 982 - group: 982 - mode: "0750" - - path: /var/lib/ghostnet-encoding/output - owner: 982 - group: 982 - mode: "0750" - - path: /var/lib/ghostnet-encoding/work - owner: 982 - group: 982 - mode: "0750" +# Pipeline directories live on the NFS export and are managed by the +# video_encoding role under the pipeline mount (GNENC-002 section 4); no +# local workload data directories remain. +workload_data_directories: [] diff --git a/llm-server/host/ansible/roles/workload_accounts/tasks/main.yml b/llm-server/host/ansible/roles/workload_accounts/tasks/main.yml index d96a43b..558e2eb 100644 --- a/llm-server/host/ansible/roles/workload_accounts/tasks/main.yml +++ b/llm-server/host/ansible/roles/workload_accounts/tasks/main.yml @@ -10,6 +10,17 @@ workload_accounts | length fail_msg: Workload account names and numeric IDs must be unique. +- name: Create shared workload groups + ansible.builtin.group: + name: "{{ workload_shared_group.name }}" + gid: "{{ workload_shared_group.gid }}" + system: true + state: present + loop: "{{ workload_shared_groups }}" + loop_control: + loop_var: workload_shared_group + label: "{{ workload_shared_group.name }} (GID {{ workload_shared_group.gid }})" + - name: Create dedicated workload groups ansible.builtin.group: name: "{{ workload_account.name }}" @@ -25,7 +36,7 @@ ansible.builtin.user: name: "{{ workload_account.name }}" uid: "{{ workload_account.uid }}" - group: "{{ workload_account.name }}" + group: "{{ workload_account.primary_group | default(workload_account.name) }}" comment: "{{ workload_account.comment }}" home: "{{ workload_account.home }}" shell: /usr/sbin/nologin diff --git a/proxmox/pve-host/README.md b/proxmox/pve-host/README.md index 2ab6b2e..5a99d4a 100644 --- a/proxmox/pve-host/README.md +++ b/proxmox/pve-host/README.md @@ -15,6 +15,13 @@ Aktuell verwaltete Host-State: - **Kernel-NFS-Export** `/media/datenhdd3/Encoding` → `ghostnet-ki` (`10.0.0.90/32`), NFSv4-only — Rolle `pve_nfs_export` (Ersatz für das frühere manuelle Script; CT 101 kann NFS nicht selbst serve, siehe GNAI-019). +- **Legacy-Medien-Cron entfernt**: Die root-Crontab-Einträge + (`chown -R root:root` / `chmod -R 0777` auf allen Medien-Mounts, alle 10 + Minuten, installiert 2026-10-11) sind entfernt — Rolle `pve_media_cron` + (Betreiberentscheidung 30.10.2026; Backup unter + `/var/backups/ghostnet/pve/`). Der Cron hätte jedes echte + Berechtigungsmodell unter den Medien-Mounts — inklusive der + Encoding-Pipeline — alle 10 Minuten plattgewalzt. - **LXC-Configs**: Entfernen der Ganesha-Ära-`lxc.seccomp.profile:`-Leerzeile aus `/etc/pve/lxc/101.conf` — Rolle `pve_lxc_conf` (wirksam beim nächsten CT-Start/Reboot). diff --git a/proxmox/pve-host/ansible/playbooks/site.yml b/proxmox/pve-host/ansible/playbooks/site.yml index 100f1bb..069e47e 100644 --- a/proxmox/pve-host/ansible/playbooks/site.yml +++ b/proxmox/pve-host/ansible/playbooks/site.yml @@ -6,4 +6,5 @@ roles: - role: pve_lxc_conf + - role: pve_media_cron - role: pve_nfs_export diff --git a/proxmox/pve-host/ansible/playbooks/verify-production.yml b/proxmox/pve-host/ansible/playbooks/verify-production.yml index 12d0b17..51b1343 100644 --- a/proxmox/pve-host/ansible/playbooks/verify-production.yml +++ b/proxmox/pve-host/ansible/playbooks/verify-production.yml @@ -144,6 +144,29 @@ /etc/pve/lxc/101.conf still carries the empty Ganesha-era lxc.seccomp.profile line (applies at the next CT start). + - name: Read the root crontab + ansible.builtin.command: + argv: + - /usr/bin/crontab + - -l + - -u + - root + check_mode: false + changed_when: false + failed_when: false + register: pve_verify_crontab + + - name: Require the legacy media permission cron to be absent + ansible.builtin.assert: + that: >- + pve_verify_crontab.rc != 0 or + (pve_verify_crontab.stdout | + regex_findall('(chown -R root:root|chmod -R 0777) /media/(daten1|daten2|daten3|datenhdd1|datenhdd3)') + | length == 0) + fail_msg: >- + The legacy recursive permission cron is active again; it flattens + the encoding pipeline ownership and modes every 10 minutes. + - name: Report the verified PVE host production state ansible.builtin.debug: msg: diff --git a/proxmox/pve-host/ansible/roles/pve_media_cron/README.md b/proxmox/pve-host/ansible/roles/pve_media_cron/README.md new file mode 100644 index 0000000..e2a9bca --- /dev/null +++ b/proxmox/pve-host/ansible/roles/pve_media_cron/README.md @@ -0,0 +1,13 @@ +# `pve_media_cron` + +Removes the legacy root crontab entries (installed 2026-10-11) that ran +`chown -R root:root` and `chmod -R 0777` on all media mounts +(`/media/daten1`, `/media/daten2`, `/media/daten3`, `/media/datenhdd1`, +`/media/datenhdd3`) every 10 minutes. The entries predate the encoding +pipeline and silently flattened every real ownership and mode below the +mounts, including the pipeline export root and layout (GNENC-002 section 4). + +The full root crontab is backed up once to +`/var/backups/ghostnet/pve/root-crontab-before-legacy-removal-20261030.txt` +before removal. Existing media files keep their current (root:root 0777) +state; the cron only stops re-enforcing it. Operator decision 2026-10-30. diff --git a/proxmox/pve-host/ansible/roles/pve_media_cron/defaults/main.yml b/proxmox/pve-host/ansible/roles/pve_media_cron/defaults/main.yml new file mode 100644 index 0000000..c2f1082 --- /dev/null +++ b/proxmox/pve-host/ansible/roles/pve_media_cron/defaults/main.yml @@ -0,0 +1,8 @@ +--- +# Legacy root crontab entries (installed 2026-10-11) flattened real ownership +# and mode on all media mounts every 10 minutes, which is incompatible with +# the encoding pipeline permission model (GNENC-002 section 4). +pve_media_cron_backup_directory: /var/backups/ghostnet/pve +pve_media_cron_backup_name: root-crontab-before-legacy-removal-20261030.txt +pve_media_cron_legacy_pattern: >- + (chown -R root:root|chmod -R 0777) /media/(daten1|daten2|daten3|datenhdd1|datenhdd3) diff --git a/proxmox/pve-host/ansible/roles/pve_media_cron/tasks/main.yml b/proxmox/pve-host/ansible/roles/pve_media_cron/tasks/main.yml new file mode 100644 index 0000000..633f4d6 --- /dev/null +++ b/proxmox/pve-host/ansible/roles/pve_media_cron/tasks/main.yml @@ -0,0 +1,66 @@ +--- +- name: Read the root crontab + ansible.builtin.command: + argv: + - /usr/bin/crontab + - -l + - -u + - root + check_mode: false + changed_when: false + failed_when: false + register: pve_media_crontab_probe + +- name: Detect the legacy media permission cron + ansible.builtin.set_fact: + pve_media_cron_legacy_present: >- + {{ pve_media_crontab_probe.rc == 0 and + (pve_media_crontab_probe.stdout | regex_findall(pve_media_cron_legacy_pattern) | length > 0) }} + +- name: Create the crontab backup directory + ansible.builtin.file: + path: "{{ pve_media_cron_backup_directory }}" + state: directory + owner: root + group: root + mode: "0700" + +- name: Back up the root crontab before removing the legacy entries + ansible.builtin.copy: + content: "{{ pve_media_crontab_probe.stdout }}\n" + dest: "{{ pve_media_cron_backup_directory }}/{{ pve_media_cron_backup_name }}" + owner: root + group: root + mode: "0600" + when: pve_media_cron_legacy_present | bool + +- name: Remove the legacy media permission lines from the root crontab + ansible.builtin.shell: | + set -euo pipefail + crontab -l -u root | grep -vE '{{ pve_media_cron_legacy_pattern }}' | crontab -u root - + args: + executable: /bin/bash + when: pve_media_cron_legacy_present | bool + +- name: Require the legacy media permission cron to be gone + ansible.builtin.command: + argv: + - /usr/bin/crontab + - -l + - -u + - root + check_mode: false + changed_when: false + failed_when: false + register: pve_media_crontab_after + +- name: Assert the legacy media permission cron is removed + ansible.builtin.assert: + that: >- + pve_media_crontab_after.rc != 0 or + (pve_media_crontab_after.stdout | regex_findall(pve_media_cron_legacy_pattern) | length == 0) + fail_msg: >- + The legacy recursive permission cron (chown -R root:root / chmod -R 0777 + on the media mounts) is still active in the root crontab; it flattens the + encoding pipeline ownership and modes every 10 minutes (GNENC-002). + success_msg: The legacy media permission cron is removed.