Encoding: Pipeline-Konten, NFS-Mount, Layout und Handover-Simulation umgesetzt und verifiziert.
- workload_accounts: gn-media (GID 970), gn-download (983), gn-dashboard (984); gn-media als primäre Gruppe der Pipeline-Konten — AUTH_SYS ueber NFSv4 wirkte serverseitig nicht mit Supplementary-Gruppen (Betriebsbefund 30.10.2026); Sudoers des Deploy-Kontos auf (ALL:ALL) verbreitert (become_user) - video_encoding: NFS-Client, Mount-Unit und Pipeline-Layout auf dem Export (10.0.0.2:/media/datenhdd3/Encoding -> /var/lib/ghostnet-pipeline, ausserhalb der Homes); Layout-Anlage als Dienstkonten wegen root_squash; Verifikation der Verzeichnisrechte als gn-encoding (2777/2770/0750/2750, Gruppe 970) - pve_media_cron: Legacy-Crontab (chown root:root + chmod -R 0777 auf allen Medienplatten, alle 10 min) entfernt — haette jedes echte Berechtigungsmodell plattgewalzt; Backup unter /var/backups/ghostnet/pve/; verify-production prueft das Fehlen ab - verify-pipeline-handover.yml: Handover-Simulation über alle Identitaeten (CT-101-.dlc-Drop -> gn-download -> queue -> gn-encoding -> output/state -> gn-dashboard); einmal vollstaendig bestanden (30.10.2026) - Doku: GNENC-000/001/002, GNAI-017 (6.10/6.11), GNMON-001 auf Stand gebracht Generated by Mistral Vibe. Co-Authored-By: Mistral Vibe <vibe@mistral.ai>
This commit is contained in:
@@ -33,6 +33,18 @@ beschrieben.
|
||||
NFSv4-only wirksam (`-3`), `verify-production` `ok=17 changed=0`,
|
||||
Idempotenz `changed=0`. Berechtigungsmodell entschieden: Mode 2777 auf den
|
||||
SMB-berührten Verzeichnissen (GNENC-002 Abschnitt 4).
|
||||
- Implementierung 30.10.2026: Konten `gn-download` (983), `gn-dashboard` (984)
|
||||
und Gruppe `gn-media` (GID 970) per `workload_accounts` angelegt — `gn-media`
|
||||
als **primäre** Gruppe der drei Pipeline-Konten (AUTH_SYS über NFSv4 wirkte
|
||||
serverseitig nicht mit Supplementary-Gruppen; Befund 30.10.2026,
|
||||
GNENC-002 Abschnitt 3); Sudoers des Deploy-Kontos auf `(ALL:ALL)` verbreitert
|
||||
(`become_user` für die Dienstkonten). NFS-Mount auf
|
||||
`/var/lib/ghostnet-pipeline` aktiv (außerhalb der Homes); Layout auf dem
|
||||
Export erzeugt und verifiziert; Handover-Simulation
|
||||
(`verify-pipeline-handover.yml`) einmal vollständig bestanden
|
||||
(CT-101-`.dlc`-Drop → gn-download → queue → gn-encoding → output/state →
|
||||
gn-dashboard). Legacy-Medien-Cron auf dem PVE-Host entfernt
|
||||
(`pve_media_cron`, GNAI-017 Abschnitt 6.11).
|
||||
- Verzeichnislayout: `input/<Profil>/*.dlc`, `jd/`, `queue/<Profil>/<Release>/`,
|
||||
`work/`, `output/<Profil>/<Release>/` + `output/_failed/`, `state/jobs/` +
|
||||
`state/system.json`.
|
||||
@@ -71,12 +83,12 @@ beschrieben.
|
||||
|
||||
## Noch offen
|
||||
|
||||
1. NFSv4.2-Mount-Abnahme von `ghostnet-ki` gegen `10.0.0.2` (inkl.
|
||||
Schreib-/Lesetest als UID 982/983 und mv-Simulation als `ghost` über SMB)
|
||||
— mit der Encoding-Implementierung.
|
||||
1. Output-Pickup über CT-101 (SMB) einmal als geschlossener Ablauf
|
||||
nachweisen (Nachtest zum 30.10.2026 — der Durchlauf wurde durch den
|
||||
Server-Shutdown unterbrochen; GNENC-002 Abschnitt 11).
|
||||
2. Feinjustierung der Ressourcenlimits nach einem repräsentativen Encoding-Test
|
||||
unter gleichzeitiger Inferenzlast.
|
||||
3. Rootless-TUN-Verifizierung für den VPN-Sidecar auf dem Host.
|
||||
4. Implementierung auf ghostnet-ki: Konten `gn-download`/`gn-dashboard`,
|
||||
NFS-Mount, Image-Bauten, Quadlet-Units (inkl. VPN-Sidecar), Skripte,
|
||||
Monitoring-Bereich (Collector, Dashboard).
|
||||
4. Implementierung auf ghostnet-ki: Image-Bauten, Quadlet-Units (inkl.
|
||||
VPN-Sidecar), Worker-Skript, JD2-Event-Scripter, Monitoring-Bereich
|
||||
(Collector, Dashboard).
|
||||
|
||||
@@ -32,7 +32,7 @@ beschrieben.
|
||||
| --- | --- | --- |
|
||||
| Repository | Host- und Deployment-Ansible bleibt im bestehenden Infrastructure-Repository | angenommen |
|
||||
| Dokumentation | Encoding liegt unter `docs/platform/llm-server/encoding/` und verwendet die Kennung `GNENC`; Monitoring unter `docs/platform/llm-server/monitoring/` mit der Kennung `GNMON`; CT 101 unter `docs/platform/proxmox-server/lxc/ct-101-samba/` mit GNAI-017 | umgesetzt |
|
||||
| Ausführungsidentitäten | drei gesperrte Konten: `gn-encoding` (UID/GID 982, Worker), `gn-download` (983, jDownloader2 + VPN-Sidecar), `gn-dashboard` (984, Dashboard, Monitoring-Bereich); gemeinsame Gruppe `gn-media` (GID 970) für die Übergabe-Verzeichnisse | gn-encoding eingerichtet und geprüft; weitere geplant |
|
||||
| Ausführungsidentitäten | drei gesperrte Konten: `gn-encoding` (UID 982, Worker), `gn-download` (983, jDownloader2 + VPN-Sidecar), `gn-dashboard` (984, Dashboard, Monitoring-Bereich); gemeinsame Gruppe `gn-media` (GID 970) als **primäre** Gruppe der drei Konten (AUTH_SYS über NFSv4 wirkte serverseitig nur mit primärer Gruppe; Befund 30.10.2026) | alle drei Konten eingerichtet und verifiziert 30.10.2026 (Idempotenz `changed=0`) |
|
||||
| Containertrennung | je Konto ein eigener rootless Podman-Store und ein eigener rootverwalteter Quadlet-Pfad | gn-encoding eingerichtet und verifiziert; weitere geplant |
|
||||
| Rechenressource | Encoding ist CPU-basiert ohne GPU-Zugriff; der host-seitige Metrics-Collector (Monitoring-Bereich) liest GPU-Metriken ausschließlich lesend für das Dashboard | Planungsentscheidung |
|
||||
| Parallelität | darf parallel zu AI Production oder AI Testing laufen | Planungsentscheidung |
|
||||
@@ -41,7 +41,7 @@ beschrieben.
|
||||
| VPN | der gesamte jDownloader2-Verkehr läuft über einen ProtonVPN-WireGuard-Tunnel: gluetun-Sidecar (`qmcgaw/gluetun`, gepinnt) unter `gn-download`, JD2 teilt dessen Netzwerk-Namespace, Kill Switch aktiv (kein Traffic ohne Tunnel); WireGuard-Credentials server-lokal in `/etc/ghostnet-download/vpn.env`, nicht in git | Betreiberentscheidung 19.10.2026 |
|
||||
| Anwendung | Worker-Image als Eigenbau auf Basis des gepinnten HandBrake-Images `linuxserver/handbrake:1.11.2-ls48` (`sha256:fda92bd1…48a3c7`) mit python3 und ffmpeg; die Digests von Worker- und VPN-Image werden nach dem Bau gepinnt | Betreiberentscheidung 19.10.2026 |
|
||||
| Speicherort | Pipeline-Daten liegen auf dem PVE-Host-Medienverbund (`/media/datenhdd3`; per LXC-Mount auch in CT-101 sichtbar): dort liegt die Medienbibliothek; ghostnet-ki selbst hat nur eine 2-TB-NVMe (OS + LLM) | Betreiberentscheidung 19.10.2026; NFS seit 30.10.2026 auf dem PVE-Host (GNAI-019 Entscheidung 2) |
|
||||
| Ein-/Ausgabe | NFS-Export `10.0.0.2:/media/datenhdd3/Encoding` (PVE-Host, Rolle `pve_nfs_export`, NFSv4-only, nur für 10.0.0.90, `root_squash`), gemountet auf `/var/lib/ghostnet-encoding`; Verzeichnislayout `input/`, `jd/`, `queue/`, `work/`, `output/`, `state/` | Betreiberentscheidung 19.10.2026; Umsetzung GNAI-019 (Revision 30.10.2026); Mount-Abnahme durch das Encoding-Ansible ausstehend |
|
||||
| Ein-/Ausgabe | NFS-Export `10.0.0.2:/media/datenhdd3/Encoding` (PVE-Host, Rolle `pve_nfs_export`, NFSv4-only, nur für 10.0.0.90, `root_squash`), gemountet auf `/var/lib/ghostnet-pipeline` (außerhalb der Konto-Homes); Verzeichnislayout `input/`, `jd/`, `queue/`, `work/`, `output/`, `state/` | Betreiberentscheidung 19.10.2026; Mount aktiv, Layout und Handover verifiziert 30.10.2026 |
|
||||
| Abholung | fertige Outputs per SMB über CT-101: Share `[Encode]` (`output/`, RW für `ghost`, lesend für `daidalaw`/`debugofen`) — Entnahme per `mv` in die Bibliotheks-Shares räumt `output/` auf; optionaler Clean-Job löscht Dateien nach 30 Tagen aus `output/`; kein smbd auf `ghostnet-ki` | Betreiberentscheidung 19.10.2026 |
|
||||
| Samba-Integration (CT-101) | CT-101 ist per Ansible adoptiert (GNAI-019; verifiziert 30.10.2026, `ok=47 changed=0`): Shares `[Encode]`/`[Encode-In]`/`[Encoding]`, Gruppe `gn-media` (GID 970) mit `ghost` als Mitglied; `.dlc`-Drop über `[Encode-In]`. NFS liegt nicht auf CT-101 (PVE-Host); die guest-seitige GID 970 gleicht die Host-GID 970 wegen des LXC-idmap nicht ab — Berechtigungsmodell: Mode 2777 auf den SMB-berührten Verzeichnissen (Betreiberentscheidung 30.10.2026, GNENC-002 Abschnitt 4) | Betreiberentscheidung 19.10.2026; GNAI-017/GNAI-019 |
|
||||
| Profile | drei versionierte HandBrake-Presets (SVT-AV1 10-bit, MKV, Audio-Passthrough) unter `llm-server/encoding/profiles/`; Auswahl über gleichnamige Ordner unter `input/` bzw. `queue/` | Betreiberentscheidung 19.10.2026; Presets versioniert |
|
||||
@@ -232,20 +232,19 @@ jeweils `ok=30 changed=0 failed=0`. Das gepinnte HandBrake-Image
|
||||
anschließend in den rootless Store von `gn-encoding` gezogen; der Pull
|
||||
bestätigte den Manifest-Digest.
|
||||
|
||||
Am 19.10.2026 wurde die Planung abgeschlossen (Abschnitte 2 und 7). Die
|
||||
externen Abhängigkeiten sind umgesetzt: CT-101 ist per Ansible adoptiert
|
||||
(GNAI-019, verifiziert 30.10.2026), der NFS-Export läuft auf dem PVE-Host
|
||||
(Rolle `pve_nfs_export`); der PVE-Host ist konvergiert (Export-Root
|
||||
`100000:970 2775`, NFSv4-only wirksam, `verify-production` `ok=17 changed=0`,
|
||||
Idempotenz `changed=0`; autorisierter Lauf 30.10.2026). Das
|
||||
Berechtigungsmodell für die SMB-Schreibpfade ist entschieden (Mode 2777,
|
||||
GNENC-002 Abschnitt 4); offener Restpunkt ist die NFSv4.2-Mount-Abnahme von
|
||||
ghostnet-ki. Die Implementierungsphase auf
|
||||
ghostnet-ki baut als Nächstes auf: die Konten `gn-download` und `gn-dashboard`
|
||||
samt Gruppe `gn-media` (GID 970), den NFS-Mount von `10.0.0.2`, die
|
||||
Image-Bauten mit Digest-Pinning (Worker, VPN-Sidecar, Dashboard), die
|
||||
Quadlet-Units für jDownloader2, VPN-Sidecar, Worker und Dashboard, den
|
||||
host-seitigen Metrics-Collector sowie die Skripte aus `scripts/`. Die Rolle
|
||||
Am 19.10.2026 wurde die Planung abgeschlossen (Abschnitte 2 und 7). Stand
|
||||
30.10.2026: Die externen Abhängigkeiten sind umgesetzt und verifiziert —
|
||||
CT-101 per Ansible adoptiert, der PVE-Host konvergiert (Export-Root
|
||||
`100000:970 2775`, NFSv4-only wirksam, Legacy-Medien-Cron entfernt — GNAI-017
|
||||
Abschnitt 6.9–6.11), die drei Pipeline-Konten samt Gruppe `gn-media`
|
||||
(einschließlich Primärgruppen-Umstellung, GNAI-017 Abschnitt 6.10) angelegt,
|
||||
der NFS-Mount auf `/var/lib/ghostnet-pipeline` aktiv, das Verzeichnislayout
|
||||
auf dem Export erzeugt und verifiziert, und die Handover-Simulation
|
||||
(`verify-pipeline-handover.yml`) einmal vollständig bestanden. Die
|
||||
Implementierungsphase auf ghostnet-ki baut als Nächstes auf: die Image-Bauten
|
||||
mit Digest-Pinning (Worker, VPN-Sidecar, Dashboard), die Quadlet-Units für
|
||||
jDownloader2, VPN-Sidecar, Worker und Dashboard, den host-seitigen
|
||||
Metrics-Collector sowie die Skripte aus `scripts/`. Die Rolle
|
||||
`video_encoding` stellt bis dahin sicher, dass weder Quadlet-Dateien noch
|
||||
Container für `gn-encoding` existieren.
|
||||
|
||||
|
||||
@@ -41,17 +41,21 @@ Gruppe `gn-media`: gemeinsame Gruppe der drei Konten für die Übergabe-
|
||||
Verzeichnisse und den lesenden Zugriff auf `state/`; GID 970. Alle Konten sind
|
||||
gesperrt (Passwort `!`, Shell `/usr/sbin/nologin`, Linger aktiv).
|
||||
|
||||
**Anmerkung 30.10.2026 (LXC-idmap):** Die guest-seitige Gruppe `gn-media`
|
||||
(GID 970) in CT-101 erscheint auf dem Host als GID 100970 und gleicht die
|
||||
reale Host-GID 970 der Pipeline nicht ab — die Gruppen-Alignment wirkt nur
|
||||
innerhalb von ghostnet-ki. Die SMB-Schreibpfade aus CT-101 laufen über
|
||||
guest-root (Host-UID 100000, da `admin users` Dateioperationen als root
|
||||
ausführt); siehe Abschnitt 11.
|
||||
**Anmerkung 30.10.2026 (primäre Gruppe):** Die drei Pipeline-Konten tragen
|
||||
`gn-media` als **primäre** Gruppe (die eigenen GID-Gruppen 982/983/984 bleiben
|
||||
für die Config-Verzeichnisse als Supplementary-Gruppe erhalten). Grund:
|
||||
AUTH_SYS über NFSv4 wirkte serverseitig nicht mit Supplementary-Gruppen —
|
||||
`mkdir` als 982/983 bekam EACCES, obwohl `gn-media` Gruppen-Schreibrechte
|
||||
hatte; mit primärer Gruppe 970 war der Zugriff sofort erfolgreich
|
||||
(Betriebsbefund 30.10.2026, Testreihe auf ghostnet-ki). Zudem gilt die
|
||||
LXC-idmap-Diskrepanz: die guest-seitige Gruppe `gn-media` (GID 970) in CT-101
|
||||
erscheint auf dem Host als GID 100970 — die SMB-Schreibpfade laufen über
|
||||
World-Write (Mode 2777), nicht über Gruppen-Abgleich; siehe Abschnitt 4.
|
||||
|
||||
## 4. Verzeichnislayout
|
||||
|
||||
```text
|
||||
/var/lib/ghostnet-encoding/ (NFS-Mount: 10.0.0.2:/media/datenhdd3/Encoding, PVE-Host)
|
||||
/var/lib/ghostnet-pipeline/ (NFS-Mount: 10.0.0.2:/media/datenhdd3/Encoding, PVE-Host; außerhalb der Konto-Homes)
|
||||
├── input/ gn-media 2777 SMB-Drop ([Encode-In])
|
||||
│ └── <Profil>/ gn-media 2777 nur .dlc
|
||||
├── jd/ gn-media 2770 JD2-Arbeitsverzeichnis (gn-download)
|
||||
@@ -92,16 +96,22 @@ Handover-Regeln:
|
||||
Schreibzugriffen.
|
||||
|
||||
**Berechtigungsmodell (Betreiberentscheidung 30.10.2026):** Die SMB-Zugriffe
|
||||
aus CT-101 laufen über guest-root (Host-UID 100000; `admin users` führt
|
||||
Dateioperationen als root aus). Da die Pipeline-Verzeichnisse via NFS den
|
||||
realen UIDs 982/983:970 gehören und diese im LXC-idmap nicht abgebildet sind
|
||||
(im Gast: `nobody:nogroup`), tragen `input/` und der `output/`-Baum Mode
|
||||
`2777`; `jd/`, `queue/`, `work/` und `state/` bleiben eng (2770/0750).
|
||||
aus CT-101 laufen über guest-root bzw. beliebige Gast-Identitäten (Host-UIDs
|
||||
100000+, da `admin users` Dateioperationen als root ausführt); die
|
||||
Pipeline-Verzeichnisse gehören via NFS den realen UIDs 982/983 mit primärer
|
||||
Gruppe 970. Da diese im LXC-idmap nicht abgebildet sind, tragen `input/` und
|
||||
der `output/`-Baum Mode `2777` (World-Write); `jd/`, `queue/`, `work/` und
|
||||
`state/` bleiben eng (2770/0750/2750). Der frühere Legacy-Cron auf dem
|
||||
PVE-Host (`chown -R root:root` + `chmod -R 0777` auf allen Medienplatten alle
|
||||
10 Minuten, installiert 11.10.2026) hätte jedes echte
|
||||
Berechtigungsmodell plattgewalzt und wurde am 30.10.2026 entfernt (Rolle
|
||||
`pve_media_cron`, Backup der Crontab unter `/var/backups/ghostnet/pve/`).
|
||||
|
||||
**Layout-Anlage:** Wegen `root_squash` auf dem Export kann Ansible als root
|
||||
keine chowned-Verzeichnisse anlegen; die Struktur wird als `gn-encoding`
|
||||
angelegt (runuser; Muster der Rolle `video_encoding`) oder vom Worker beim
|
||||
Start erzeugt und geprüft.
|
||||
beziehungsweise `gn-download` angelegt (`become_user`; die Rolle
|
||||
`video_encoding` erzeugt und verifiziert das Layout vollständig — Stand
|
||||
30.10.2026 umgesetzt, Verifikation bestanden).
|
||||
|
||||
**Abholung:** Die fertigen Outputs werden über CT-101 per SMB abgeholt: Share
|
||||
`[Encode]` (Pfad `/media/datenhdd3/Encoding/output`, RW für `ghost`, lesend für
|
||||
@@ -254,9 +264,13 @@ die Container gemountet (Updates ohne Image-Rebuild, versioniert über git).
|
||||
|
||||
## 11. Offene Punkte
|
||||
|
||||
1. NFSv4.2-Mount-Abnahme von `ghostnet-ki` gegen `10.0.0.2` — inkl.
|
||||
Schreib-/Lesetest als UID 982/983 über den Mount und mv-Simulation als
|
||||
`ghost` über SMB; mit der Encoding-Implementierung.
|
||||
1. **Output-Pickup über CT-101 (SMB) einmal als geschlossener Ablauf
|
||||
nachweisen:** Die Handover-Simulation (`verify-pipeline-handover.yml`)
|
||||
lief am 30.10.2026 vollständig durch (`.dlc`-Drop aus CT-101 →
|
||||
gn-download → queue → gn-encoding → output/state → gn-dashboard); das
|
||||
Lesen/Löschen des Output-Testreleases über CT-101 wurde durch den
|
||||
gleichzeitigen Server-Shutdown nicht mehr abgeschlossen und ist als
|
||||
kurzer Nachtest zu wiederholen.
|
||||
2. Feinjustierung der Ressourcenlimits nach Lasttest unter Inferenzlast
|
||||
(GNENC-001 Abschnitt 4).
|
||||
3. Rootless-TUN-Verifizierung für den VPN-Sidecar auf dem Host (Fallback:
|
||||
|
||||
@@ -42,7 +42,7 @@ Encoding-Pipeline). Keine GPU-Gruppenmitgliedschaft, keine `/dev/kfd`-/
|
||||
## 5. Metrics-Collector
|
||||
|
||||
- Sampling alle 10 s; atomares Schreiben (`state/tmp/` + `mv`) nach
|
||||
`/var/lib/ghostnet-encoding/state/system.json` (Mode `0644`).
|
||||
`/var/lib/ghostnet-pipeline/state/system.json` (Mode `0644`).
|
||||
- Schema:
|
||||
|
||||
```json
|
||||
|
||||
@@ -328,4 +328,16 @@ GNENC-002 Abschnitt 4): `input/`, `jd/`, `queue/`, `work/`, `output/`,
|
||||
`input/`-Baum und dem gesamten `output/`-Baum (inkl. Release- und
|
||||
`_failed/`-Verzeichnisse, vom Worker angelegt); `jd/`, `queue/`, `work/`
|
||||
und `state/` bleiben eng (2770/2750/0750). Festgehalten in GNENC-002
|
||||
Abschnitt 4.
|
||||
Abschnitt 4. **Betriebsbefund 30.10.2026:** AUTH_SYS über NFSv4 wirkte
|
||||
serverseitig nicht mit Supplementary-Gruppen — die Pipeline-Konten auf
|
||||
ghostnet-ki tragen `gn-media` daher als **primäre** Gruppe (Befund und
|
||||
Umsetzung: GNENC-002 Abschnitt 3).
|
||||
11. **Legacy-Medien-Cron auf dem PVE-Host — erledigt (30.10.2026):** Die
|
||||
root-Crontab enthielt seit dem 11.10.2026 zehnminütliche
|
||||
`chown -R root:root`- und `chmod -R 0777`-Läufe über alle Medienplatten
|
||||
(daten1/2/3, datenhdd1, datenhdd3) — sie hatte den Export-Root nach jeder
|
||||
Konvergenz zurückgesetzt und hätte jedes echte Berechtigungsmodell
|
||||
plattgewalzt. Entfernt über die Rolle `pve_media_cron` im Host-Projekt
|
||||
(`proxmox/pve-host/ansible/`, Betreiberentscheidung 30.10.2026); Backup
|
||||
der Crontab unter `/var/backups/ghostnet/pve/`; `verify-production` des
|
||||
Host-Projekts prüft das Fehlen seither ab.
|
||||
|
||||
@@ -7,11 +7,17 @@ area of the LLM-Server host root; the shared host baseline lives in
|
||||
|
||||
`playbooks/video-encoding.yml` applies the `video_encoding` role: it verifies
|
||||
the locked `gn-encoding` account, enforces CPU-only device isolation, manages
|
||||
the encoding directories and proves the separate rootless Podman store. No
|
||||
encoder image, Quadlet, timer or schedule is defined yet; those are open
|
||||
decisions in
|
||||
the pipeline NFS mount (`10.0.0.2:/media/datenhdd3/Encoding` at
|
||||
`/var/lib/ghostnet-pipeline`) and creates and verifies the pipeline
|
||||
directory layout on the export as the service accounts. No encoder image,
|
||||
Quadlet, timer or schedule is defined yet; those are open decisions in
|
||||
[`../../../docs/platform/llm-server/encoding/`](../../../docs/platform/llm-server/encoding/GNENC-000-DOC_Index.md).
|
||||
|
||||
`playbooks/verify-pipeline-handover.yml` exercises the file handover between
|
||||
the pipeline identities (GNENC-002 sections 5 and 6) on the mounted export;
|
||||
apply `video-encoding.yml` first. The SMB write path from CT-101 (guest root)
|
||||
is simulated separately on CT-101 (GNAI-017 section 6.10).
|
||||
|
||||
## Control-node setup
|
||||
|
||||
Use the shared virtual environment at the repository root (see
|
||||
|
||||
@@ -0,0 +1,205 @@
|
||||
---
|
||||
# Exercises the GNENC-002 file handover between the pipeline identities on the
|
||||
# mounted NFS export: .dlc drop -> gn-download ingest -> jd/ -> queue/ ->
|
||||
# gn-encoding pick-up -> work/ -> output/ -> state/ -> gn-dashboard read.
|
||||
# Apply playbooks/video-encoding.yml first. The output test release is left in
|
||||
# place for the CT-101 SMB pickup simulation and removed on the next run.
|
||||
- name: Simulate the encoding pipeline file handover (GNENC-002 sections 5 and 6)
|
||||
hosts: llm_servers
|
||||
become: true
|
||||
gather_facts: false
|
||||
|
||||
vars:
|
||||
handover_profile: "AV1 MKV 1080p30 RF25"
|
||||
handover_release: handover-test
|
||||
handover_mount: /var/lib/ghostnet-pipeline
|
||||
handover_dlc: "{{ handover_mount }}/input/{{ handover_profile }}/{{ handover_release }}.dlc"
|
||||
handover_dlc_done: "{{ handover_mount }}/jd/dlc_done/{{ handover_release }}.dlc"
|
||||
handover_jd_release: "{{ handover_mount }}/jd/{{ handover_release }}"
|
||||
handover_queue_release: "{{ handover_mount }}/queue/{{ handover_profile }}/{{ handover_release }}"
|
||||
handover_work_release: "{{ handover_mount }}/work/{{ handover_release }}"
|
||||
handover_output_release: "{{ handover_mount }}/output/{{ handover_profile }}/{{ handover_release }}"
|
||||
handover_output_file: "{{ handover_output_release }}/video.test.mkv"
|
||||
handover_jobs: "{{ handover_mount }}/state/jobs"
|
||||
handover_tmp: "{{ handover_mount }}/state/tmp"
|
||||
handover_job_file: "{{ handover_jobs }}/handover-test.json"
|
||||
|
||||
tasks:
|
||||
- name: Require the pipeline mount to be active
|
||||
ansible.builtin.command:
|
||||
argv:
|
||||
- /usr/bin/systemctl
|
||||
- is-active
|
||||
- var-lib-ghostnet\x2dpipeline.mount
|
||||
changed_when: false
|
||||
register: handover_mount_status
|
||||
failed_when: handover_mount_status.stdout != "active"
|
||||
|
||||
- name: Stat the handover directories
|
||||
ansible.builtin.stat:
|
||||
path: "{{ item }}"
|
||||
loop:
|
||||
- "{{ handover_mount }}/input/{{ handover_profile }}"
|
||||
- "{{ handover_queue_release }}"
|
||||
- "{{ handover_output_release }}"
|
||||
- "{{ handover_mount }}/jd"
|
||||
- "{{ handover_mount }}/work"
|
||||
- "{{ handover_jobs }}"
|
||||
- "{{ handover_tmp }}"
|
||||
register: handover_dir_stats
|
||||
|
||||
- name: Require the pipeline layout to be present
|
||||
ansible.builtin.assert:
|
||||
that: >-
|
||||
handover_dir_stats.results | map(attribute='stat.isdir')
|
||||
| select('equalto', false) | list | length == 0
|
||||
fail_msg: >-
|
||||
The pipeline layout is missing; apply playbooks/video-encoding.yml
|
||||
first (GNENC-002 section 4).
|
||||
|
||||
- name: Clean up leftovers from previous runs (gn-download side)
|
||||
ansible.builtin.shell: |
|
||||
set -euo pipefail
|
||||
rm -f "{{ handover_dlc }}" "{{ handover_dlc_done }}"
|
||||
rm -rf "{{ handover_jd_release }}" "{{ handover_queue_release }}"
|
||||
args:
|
||||
executable: /bin/bash
|
||||
become_user: gn-download
|
||||
environment:
|
||||
HOME: /var/lib/ghostnet-download
|
||||
changed_when: true
|
||||
|
||||
- name: Clean up leftovers from previous runs (gn-encoding side)
|
||||
ansible.builtin.shell: |
|
||||
set -euo pipefail
|
||||
rm -rf "{{ handover_work_release }}" "{{ handover_output_release }}"
|
||||
rm -f "{{ handover_job_file }}"
|
||||
args:
|
||||
executable: /bin/bash
|
||||
become_user: gn-encoding
|
||||
environment:
|
||||
HOME: /var/lib/ghostnet-encoding
|
||||
changed_when: true
|
||||
|
||||
- name: Probe the .dlc drop
|
||||
ansible.builtin.stat:
|
||||
path: "{{ handover_dlc }}"
|
||||
register: handover_dlc_probe
|
||||
|
||||
- name: Provide the .dlc drop as gn-encoding when no external SMB simulation file exists
|
||||
ansible.builtin.shell: |
|
||||
set -euo pipefail
|
||||
printf 'dlc-handover-test\n' > "{{ handover_dlc }}"
|
||||
args:
|
||||
executable: /bin/bash
|
||||
become_user: gn-encoding
|
||||
environment:
|
||||
HOME: /var/lib/ghostnet-encoding
|
||||
when: not handover_dlc_probe.stat.exists
|
||||
|
||||
- name: Ingest the .dlc as gn-download (read and move to jd/dlc_done)
|
||||
ansible.builtin.shell: |
|
||||
set -euo pipefail
|
||||
cat "{{ handover_dlc }}" > /dev/null
|
||||
mv "{{ handover_dlc }}" "{{ handover_dlc_done }}"
|
||||
[ -f "{{ handover_dlc_done }}" ]
|
||||
args:
|
||||
executable: /bin/bash
|
||||
become_user: gn-download
|
||||
environment:
|
||||
HOME: /var/lib/ghostnet-download
|
||||
changed_when: true
|
||||
|
||||
- name: Simulate the jDownloader2 download in jd/
|
||||
ansible.builtin.shell: |
|
||||
set -euo pipefail
|
||||
mkdir "{{ handover_jd_release }}"
|
||||
dd if=/dev/zero of="{{ handover_jd_release }}/video.test" bs=1M count=1 status=none
|
||||
args:
|
||||
executable: /bin/bash
|
||||
become_user: gn-download
|
||||
environment:
|
||||
HOME: /var/lib/ghostnet-download
|
||||
changed_when: true
|
||||
|
||||
- name: Hand the release over to the queue (atomic move)
|
||||
ansible.builtin.shell: |
|
||||
set -euo pipefail
|
||||
mv "{{ handover_jd_release }}" "{{ handover_queue_release }}"
|
||||
[ -f "{{ handover_queue_release }}/video.test" ]
|
||||
args:
|
||||
executable: /bin/bash
|
||||
become_user: gn-download
|
||||
environment:
|
||||
HOME: /var/lib/ghostnet-download
|
||||
changed_when: true
|
||||
|
||||
- name: Pick up, encode and hand over to output as gn-encoding
|
||||
ansible.builtin.shell: |
|
||||
set -euo pipefail
|
||||
cat "{{ handover_queue_release }}/video.test" > /dev/null
|
||||
mkdir "{{ handover_work_release }}"
|
||||
dd if=/dev/zero of="{{ handover_work_release }}/video.test.mkv" bs=1M count=1 status=none
|
||||
mv "{{ handover_work_release }}/video.test.mkv" "{{ handover_output_file }}"
|
||||
rm "{{ handover_queue_release }}/video.test"
|
||||
rmdir "{{ handover_queue_release }}" "{{ handover_work_release }}"
|
||||
[ -f "{{ handover_output_file }}" ]
|
||||
args:
|
||||
executable: /bin/bash
|
||||
become_user: gn-encoding
|
||||
environment:
|
||||
HOME: /var/lib/ghostnet-encoding
|
||||
changed_when: true
|
||||
|
||||
- name: Write the state JSON as gn-encoding (atomic via state/tmp)
|
||||
ansible.builtin.shell: |
|
||||
set -euo pipefail
|
||||
printf '{"job_id":"handover-test","status":"done"}\n' > "{{ handover_tmp }}/handover-test.json"
|
||||
mv "{{ handover_tmp }}/handover-test.json" "{{ handover_job_file }}"
|
||||
[ -f "{{ handover_job_file }}" ]
|
||||
args:
|
||||
executable: /bin/bash
|
||||
become_user: gn-encoding
|
||||
environment:
|
||||
HOME: /var/lib/ghostnet-encoding
|
||||
changed_when: true
|
||||
|
||||
- name: Read the state JSON as gn-dashboard
|
||||
ansible.builtin.shell: |
|
||||
set -euo pipefail
|
||||
grep -q '"status":"done"' "{{ handover_job_file }}"
|
||||
args:
|
||||
executable: /bin/bash
|
||||
become_user: gn-dashboard
|
||||
environment:
|
||||
HOME: /var/lib/ghostnet-dashboard
|
||||
changed_when: false
|
||||
|
||||
- name: Clean up the state test object
|
||||
ansible.builtin.shell: |
|
||||
set -euo pipefail
|
||||
rm -f "{{ handover_job_file }}"
|
||||
args:
|
||||
executable: /bin/bash
|
||||
become_user: gn-encoding
|
||||
environment:
|
||||
HOME: /var/lib/ghostnet-encoding
|
||||
changed_when: true
|
||||
|
||||
- name: Stat the handed-over output file
|
||||
ansible.builtin.stat:
|
||||
path: "{{ handover_output_file }}"
|
||||
register: handover_output_stats
|
||||
|
||||
- name: Verify the output file identity
|
||||
ansible.builtin.assert:
|
||||
that:
|
||||
- handover_output_stats.stat.exists
|
||||
- handover_output_stats.stat.uid | int == 982
|
||||
- handover_output_stats.stat.gid | int == 970
|
||||
fail_msg: >-
|
||||
The output file must exist as 982:970 on the export; found
|
||||
{{ handover_output_stats.stat.uid }}:{{ handover_output_stats.stat.gid }}.
|
||||
success_msg: >-
|
||||
Handover simulation complete; the output test release remains for the
|
||||
CT-101 SMB pickup simulation and is removed on the next run.
|
||||
@@ -6,17 +6,32 @@ baseline for the locked `gn-encoding` service account (UID/GID 982) on
|
||||
|
||||
It verifies the account identity, the locked password and the deterministic
|
||||
subordinate ID ranges, enforces the absence of GPU device groups, manages the
|
||||
root-owned configuration directory, the input/output/work data directories and
|
||||
the administrator-managed rootless Quadlet directory, and proves that the
|
||||
account cannot read `/dev/kfd` or any DRM render node and uses its own rootless
|
||||
Podman store.
|
||||
root-owned configuration directory and the administrator-managed rootless
|
||||
Quadlet directory, and proves that the account cannot read `/dev/kfd` or any
|
||||
DRM render node and uses its own rootless Podman store.
|
||||
|
||||
Pipeline storage comes from the PVE-host NFS export
|
||||
(`10.0.0.2:/media/datenhdd3/Encoding`, GNAI-017/GNAI-019), mounted at
|
||||
`/var/lib/ghostnet-encoding/pipeline` inside the account home — the home
|
||||
itself is not the mountpoint, so the rootless Podman store below it stays
|
||||
visible. The role installs the NFS client, manages the systemd mount unit and
|
||||
creates the pipeline directory layout (`input/`, `jd/`, `queue/`, `work/`,
|
||||
`output/`, `state/`, one subdirectory per HandBrake profile) as the service
|
||||
accounts over NFS, because `root_squash` prevents root-owned creation. The
|
||||
layout ownership and modes (2777 on the SMB-facing trees, 2770/0750/2750
|
||||
elsewhere, group `gn-media` GID 970) are verified after convergence
|
||||
(GNENC-002 section 4).
|
||||
|
||||
While no encoder image is selected (`video_encoding_image` empty), the role
|
||||
additionally guards that no Quadlet file and no container exists for the
|
||||
account. Encoder image, schedule, resource limits, queue handling and health
|
||||
checks are open decisions in
|
||||
`docs/platform/llm-server/encoding/GNENC-001` and are not assumed here.
|
||||
account. Encoder image, schedule, resource limits and health checks remain
|
||||
gated by the open decisions in `docs/platform/llm-server/encoding/GNENC-001`
|
||||
and are not assumed here.
|
||||
|
||||
The account itself is created by the shared `workload_accounts` role in the
|
||||
host project (`llm-server/host/ansible`); this role requires the host baseline
|
||||
to be applied first.
|
||||
host project (`llm-server/host/ansible`), together with `gn-download` (983),
|
||||
`gn-dashboard` (984) and the shared group `gn-media` (GID 970); this role
|
||||
requires the host baseline to be applied first.
|
||||
|
||||
The file handover between the pipeline identities is exercised by
|
||||
`playbooks/verify-pipeline-handover.yml`.
|
||||
|
||||
@@ -8,6 +8,10 @@ video_encoding_account:
|
||||
name: gn-encoding
|
||||
uid: 982
|
||||
gid: 982
|
||||
# Primary login group is gn-media (GID 970): AUTH_SYS over NFSv4 did not
|
||||
# honour the supplementary group membership server-side, so the pipeline
|
||||
# identities carry gn-media as their primary group (verified 2026-10-30).
|
||||
primary_gid: 970
|
||||
home: /var/lib/ghostnet-encoding
|
||||
config_directory: /etc/ghostnet-encoding
|
||||
shell: /usr/sbin/nologin
|
||||
@@ -15,18 +19,46 @@ video_encoding_account:
|
||||
|
||||
video_encoding_subid_count: 65536
|
||||
|
||||
video_encoding_directories:
|
||||
- path: /var/lib/ghostnet-encoding/input
|
||||
mode: "0750"
|
||||
- path: /var/lib/ghostnet-encoding/output
|
||||
mode: "0750"
|
||||
- path: /var/lib/ghostnet-encoding/work
|
||||
mode: "0750"
|
||||
# Pipeline storage on the PVE-host NFS export (GNENC-001/002, GNAI-017).
|
||||
# The mount lives OUTSIDE the service homes (the gn-encoding home is 0750 and
|
||||
# must stay traversable only for its owner; gn-download and gn-dashboard need
|
||||
# to reach the pipeline root without touching any account home).
|
||||
video_encoding_nfs_server: 10.0.0.2
|
||||
video_encoding_nfs_export: /media/datenhdd3/Encoding
|
||||
video_encoding_pipeline_mount: /var/lib/ghostnet-pipeline
|
||||
video_encoding_pipeline_mount_unit: var-lib-ghostnet\x2dpipeline.mount
|
||||
video_encoding_pipeline_mount_options: nfsvers=4.2,hard,_netdev,noatime
|
||||
video_encoding_pipeline_nfs_client_package: nfs-common
|
||||
|
||||
# Export root ownership on the PVE host (GNAI-017 section 4.1):
|
||||
# 100000 = CT-101 guest root (SMB share writes), 970 = gn-media (pipeline).
|
||||
video_encoding_export_root_uid: 100000
|
||||
video_encoding_export_root_gid: 970
|
||||
video_encoding_export_root_mode: "2775"
|
||||
|
||||
# Shared pipeline group of gn-encoding (982), gn-download (983) and
|
||||
# gn-dashboard (984); created by the workload_accounts role.
|
||||
video_encoding_media_group: gn-media
|
||||
video_encoding_media_gid: 970
|
||||
|
||||
# Declared HandBrake profiles (must match the preset files under
|
||||
# llm-server/encoding/profiles/; verified by the role, one input/, queue/
|
||||
# and output/ subdirectory per profile).
|
||||
video_encoding_profile_names:
|
||||
- "AV1 MKV 1080p30 Anime RF34"
|
||||
- "AV1 MKV 1080p30 RF25"
|
||||
- "AV1 MKV 2160p60 4K RF23"
|
||||
|
||||
# Superseded local directories of the pre-NFS baseline (GNENC-001 section 6);
|
||||
# removed when empty.
|
||||
video_encoding_legacy_local_directories:
|
||||
- /var/lib/ghostnet-encoding/input
|
||||
- /var/lib/ghostnet-encoding/output
|
||||
- /var/lib/ghostnet-encoding/work
|
||||
|
||||
video_encoding_quadlet_user_directory: /etc/containers/systemd/users
|
||||
|
||||
# Open decisions from docs/platform/llm-server/encoding/GNENC-001 section 7
|
||||
# stay unset on purpose. Until an encoder image, a schedule and resource limits
|
||||
# are decided, the role only manages the shared encoding baseline and guards
|
||||
# that no workload is defined.
|
||||
# are decided, the role additionally guards that no workload is defined.
|
||||
video_encoding_image: ""
|
||||
|
||||
@@ -46,7 +46,7 @@
|
||||
video_encoding_account.uid | int
|
||||
- >-
|
||||
video_encoding_account_entry.stdout.split(":")[3] | int ==
|
||||
video_encoding_account.gid | int
|
||||
video_encoding_account.primary_gid | int
|
||||
- >-
|
||||
video_encoding_account_entry.stdout.split(":")[5] ==
|
||||
video_encoding_account.home
|
||||
@@ -146,17 +146,264 @@
|
||||
group: "{{ video_encoding_account.gid }}"
|
||||
mode: "0750"
|
||||
|
||||
- name: Manage the encoding data directories
|
||||
- name: Remove the superseded local pipeline directories when empty
|
||||
ansible.builtin.shell: |
|
||||
set -euo pipefail
|
||||
changed=0
|
||||
for dir in {{ video_encoding_legacy_local_directories | map('quote') | join(' ') }}; do
|
||||
if [ -d "$dir" ]; then
|
||||
if [ -n "$(find "$dir" -mindepth 1 -print -quit)" ]; then
|
||||
echo "Refusing to remove non-empty legacy directory: $dir"
|
||||
exit 1
|
||||
fi
|
||||
rmdir "$dir"
|
||||
changed=1
|
||||
fi
|
||||
done
|
||||
[ "$changed" = 0 ] || echo CHANGED
|
||||
args:
|
||||
executable: /bin/bash
|
||||
register: video_encoding_legacy_cleanup
|
||||
changed_when: "'CHANGED' in video_encoding_legacy_cleanup.stdout"
|
||||
|
||||
- name: Install the NFS client
|
||||
ansible.builtin.apt:
|
||||
name: "{{ video_encoding_pipeline_nfs_client_package }}"
|
||||
state: present
|
||||
update_cache: true
|
||||
cache_valid_time: 3600
|
||||
install_recommends: false
|
||||
|
||||
- name: Probe whether the pipeline mount is already active
|
||||
ansible.builtin.command:
|
||||
argv:
|
||||
- /usr/bin/findmnt
|
||||
- --mountpoint
|
||||
- "{{ video_encoding_pipeline_mount }}"
|
||||
check_mode: false
|
||||
changed_when: false
|
||||
failed_when: false
|
||||
register: video_encoding_pipeline_mountpoint_probe
|
||||
|
||||
- name: Create the pipeline mountpoint outside the service homes
|
||||
ansible.builtin.file:
|
||||
path: "{{ video_encoding_directory.path }}"
|
||||
path: "{{ video_encoding_pipeline_mount }}"
|
||||
state: directory
|
||||
owner: "{{ video_encoding_account.uid }}"
|
||||
group: "{{ video_encoding_account.gid }}"
|
||||
mode: "{{ video_encoding_directory.mode }}"
|
||||
loop: "{{ video_encoding_directories }}"
|
||||
loop_control:
|
||||
loop_var: video_encoding_directory
|
||||
label: "{{ video_encoding_directory.path }}"
|
||||
owner: root
|
||||
group: root
|
||||
mode: "0755"
|
||||
when: video_encoding_pipeline_mountpoint_probe.rc != 0
|
||||
|
||||
- name: Manage the pipeline NFS mount unit
|
||||
ansible.builtin.copy:
|
||||
content: |
|
||||
[Unit]
|
||||
Description=Ghostnet encoding pipeline NFS export (GNENC-002)
|
||||
Requires=network-online.target
|
||||
After=network-online.target
|
||||
|
||||
[Mount]
|
||||
What={{ video_encoding_nfs_server }}:{{ video_encoding_nfs_export }}
|
||||
Where={{ video_encoding_pipeline_mount }}
|
||||
Type=nfs4
|
||||
Options={{ video_encoding_pipeline_mount_options }}
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
dest: "/etc/systemd/system/{{ video_encoding_pipeline_mount_unit }}"
|
||||
owner: root
|
||||
group: root
|
||||
mode: "0644"
|
||||
|
||||
- name: Reload systemd for the pipeline mount unit
|
||||
ansible.builtin.systemd_service:
|
||||
daemon_reload: true
|
||||
when: not ansible_check_mode
|
||||
|
||||
- name: Enable and mount the pipeline NFS export
|
||||
ansible.builtin.systemd_service:
|
||||
name: "{{ video_encoding_pipeline_mount_unit }}"
|
||||
enabled: true
|
||||
state: started
|
||||
when: not ansible_check_mode
|
||||
|
||||
- name: Verify the pipeline mount is active
|
||||
ansible.builtin.command:
|
||||
argv:
|
||||
- /usr/bin/systemctl
|
||||
- is-active
|
||||
- "{{ video_encoding_pipeline_mount_unit }}"
|
||||
check_mode: false
|
||||
changed_when: false
|
||||
register: video_encoding_pipeline_mount_status
|
||||
failed_when: video_encoding_pipeline_mount_status.stdout != "active"
|
||||
when: not ansible_check_mode
|
||||
|
||||
- name: Stat the mounted export root
|
||||
ansible.builtin.stat:
|
||||
path: "{{ video_encoding_pipeline_mount }}"
|
||||
register: video_encoding_export_root
|
||||
when: not ansible_check_mode
|
||||
|
||||
- name: Verify the mounted export root ownership and mode
|
||||
ansible.builtin.assert:
|
||||
that:
|
||||
- video_encoding_export_root.stat.isdir
|
||||
- video_encoding_export_root.stat.uid == video_encoding_export_root_uid | int
|
||||
- video_encoding_export_root.stat.gid == video_encoding_export_root_gid | int
|
||||
- >-
|
||||
'%04o' % (video_encoding_export_root.stat.mode | int(0, 8)) ==
|
||||
video_encoding_export_root_mode
|
||||
fail_msg: >-
|
||||
The mounted export root must be {{ video_encoding_export_root_uid }}:{{ video_encoding_export_root_gid }}
|
||||
{{ video_encoding_export_root_mode }} (PVE host side, GNAI-017 section 4.1);
|
||||
found {{ video_encoding_export_root.stat.uid }}:{{ video_encoding_export_root.stat.gid }}
|
||||
{{ video_encoding_export_root.stat.mode }}.
|
||||
success_msg: The pipeline export root is mounted with the declared identity.
|
||||
when: not ansible_check_mode
|
||||
|
||||
- name: Collect the HandBrake preset files from the profiles directory
|
||||
ansible.builtin.set_fact:
|
||||
video_encoding_profile_files: >-
|
||||
{{ lookup('ansible.builtin.fileglob', role_path ~ '/../../../profiles/*.json', wantlist=True)
|
||||
| map('basename') | map('regex_replace', '\.json$', '') | list }}
|
||||
|
||||
- name: Require the declared profiles to match the versioned preset files
|
||||
ansible.builtin.assert:
|
||||
that:
|
||||
- video_encoding_profile_names | sort == video_encoding_profile_files | sort
|
||||
fail_msg: >-
|
||||
The declared video_encoding_profile_names do not match the preset files
|
||||
under llm-server/encoding/profiles/ (found:
|
||||
{{ video_encoding_profile_files | join(', ') }}).
|
||||
success_msg: The declared profiles match the versioned preset files.
|
||||
|
||||
- name: Create the gn-download pipeline directories on the export
|
||||
ansible.builtin.shell: |
|
||||
set -euo pipefail
|
||||
changed=0
|
||||
while read -r mode path; do
|
||||
[ -n "${path:-}" ] || continue
|
||||
if [ ! -d "$path" ]; then
|
||||
install -d -m "$mode" "$path"
|
||||
changed=1
|
||||
elif [ "$(stat -c '%a' "$path")" != "$mode" ]; then
|
||||
chmod "$mode" "$path"
|
||||
changed=1
|
||||
fi
|
||||
done <<'EOF'
|
||||
{% for d in [
|
||||
(video_encoding_pipeline_mount ~ '/input', '2777'),
|
||||
(video_encoding_pipeline_mount ~ '/jd', '2770'),
|
||||
(video_encoding_pipeline_mount ~ '/jd/dlc_done', '2770'),
|
||||
(video_encoding_pipeline_mount ~ '/jd/failed', '2770'),
|
||||
(video_encoding_pipeline_mount ~ '/queue', '2770') ] %}
|
||||
{{ d[1] }} {{ d[0] }}
|
||||
{% endfor %}
|
||||
{% for p in video_encoding_profile_names %}
|
||||
2777 {{ video_encoding_pipeline_mount }}/input/{{ p }}
|
||||
2770 {{ video_encoding_pipeline_mount }}/queue/{{ p }}
|
||||
{% endfor %}
|
||||
EOF
|
||||
[ "$changed" = 0 ] || echo CHANGED
|
||||
args:
|
||||
executable: /bin/bash
|
||||
become: true
|
||||
become_user: gn-download
|
||||
environment:
|
||||
HOME: /var/lib/ghostnet-download
|
||||
register: video_encoding_download_layout
|
||||
changed_when: "'CHANGED' in video_encoding_download_layout.stdout"
|
||||
when: not ansible_check_mode
|
||||
|
||||
- name: Create the gn-encoding pipeline directories on the export
|
||||
ansible.builtin.shell: |
|
||||
set -euo pipefail
|
||||
changed=0
|
||||
while read -r mode path; do
|
||||
[ -n "${path:-}" ] || continue
|
||||
if [ ! -d "$path" ]; then
|
||||
install -d -m "$mode" "$path"
|
||||
changed=1
|
||||
elif [ "$(stat -c '%a' "$path")" != "$mode" ]; then
|
||||
chmod "$mode" "$path"
|
||||
changed=1
|
||||
fi
|
||||
done <<'EOF'
|
||||
{% for d in [
|
||||
(video_encoding_pipeline_mount ~ '/work', '0750'),
|
||||
(video_encoding_pipeline_mount ~ '/output', '2777'),
|
||||
(video_encoding_pipeline_mount ~ '/output/_failed', '2777'),
|
||||
(video_encoding_pipeline_mount ~ '/state', '2750'),
|
||||
(video_encoding_pipeline_mount ~ '/state/jobs', '0750'),
|
||||
(video_encoding_pipeline_mount ~ '/state/tmp', '0750') ] %}
|
||||
{{ d[1] }} {{ d[0] }}
|
||||
{% endfor %}
|
||||
{% for p in video_encoding_profile_names %}
|
||||
2777 {{ video_encoding_pipeline_mount }}/output/{{ p }}
|
||||
{% endfor %}
|
||||
EOF
|
||||
[ "$changed" = 0 ] || echo CHANGED
|
||||
args:
|
||||
executable: /bin/bash
|
||||
become: true
|
||||
become_user: gn-encoding
|
||||
environment:
|
||||
HOME: "{{ video_encoding_account.home }}"
|
||||
register: video_encoding_worker_layout
|
||||
changed_when: "'CHANGED' in video_encoding_worker_layout.stdout"
|
||||
when: not ansible_check_mode
|
||||
|
||||
- name: Verify the pipeline directory layout on the export
|
||||
ansible.builtin.shell: |
|
||||
set -uo pipefail
|
||||
failed=0
|
||||
while read -r uid gid mode path; do
|
||||
[ -n "${path:-}" ] || continue
|
||||
if [ ! -d "$path" ]; then
|
||||
echo "MISSING $path"
|
||||
failed=1
|
||||
continue
|
||||
fi
|
||||
read -r auid agid amode <<<"$(stat -c '%u %g %a' "$path")"
|
||||
if [ "$auid" != "$uid" ] || [ "$agid" != "$gid" ] || [ "$amode" != "$mode" ]; then
|
||||
echo "WRONG $path (expected $uid:$gid $mode, found $auid:$agid $amode)"
|
||||
failed=1
|
||||
fi
|
||||
done <<'EOF'
|
||||
{% for d in [
|
||||
(video_encoding_pipeline_mount ~ '/input', '983', '2777'),
|
||||
(video_encoding_pipeline_mount ~ '/jd', '983', '2770'),
|
||||
(video_encoding_pipeline_mount ~ '/jd/dlc_done', '983', '2770'),
|
||||
(video_encoding_pipeline_mount ~ '/jd/failed', '983', '2770'),
|
||||
(video_encoding_pipeline_mount ~ '/queue', '983', '2770'),
|
||||
(video_encoding_pipeline_mount ~ '/work', '982', '0750'),
|
||||
(video_encoding_pipeline_mount ~ '/output', '982', '2777'),
|
||||
(video_encoding_pipeline_mount ~ '/output/_failed', '982', '2777'),
|
||||
(video_encoding_pipeline_mount ~ '/state', '982', '2750'),
|
||||
(video_encoding_pipeline_mount ~ '/state/jobs', '982', '0750'),
|
||||
(video_encoding_pipeline_mount ~ '/state/tmp', '982', '0750') ] %}
|
||||
{{ d[1] }} {{ d[2] }} {{ d[0] }}
|
||||
{% endfor %}
|
||||
{% for p in video_encoding_profile_names %}
|
||||
983 970 2777 {{ video_encoding_pipeline_mount }}/input/{{ p }}
|
||||
983 970 2770 {{ video_encoding_pipeline_mount }}/queue/{{ p }}
|
||||
982 970 2777 {{ video_encoding_pipeline_mount }}/output/{{ p }}
|
||||
{% endfor %}
|
||||
EOF
|
||||
if [ "$failed" = 1 ]; then
|
||||
echo "Pipeline layout verification failed"
|
||||
exit 1
|
||||
fi
|
||||
echo LAYOUT_OK
|
||||
args:
|
||||
executable: /bin/bash
|
||||
become_user: gn-encoding
|
||||
environment:
|
||||
HOME: "{{ video_encoding_account.home }}"
|
||||
changed_when: false
|
||||
register: video_encoding_layout_verify
|
||||
when: not ansible_check_mode
|
||||
|
||||
- name: Manage the administrator-managed rootless Quadlet directory
|
||||
ansible.builtin.file:
|
||||
@@ -330,7 +577,8 @@
|
||||
msg:
|
||||
account: "gn-encoding (UID/GID 982)"
|
||||
config_directory: /etc/ghostnet-encoding
|
||||
data_directories: "{{ video_encoding_directories | map(attribute='path') | list }}"
|
||||
pipeline_mount: "{{ video_encoding_nfs_server }}:{{ video_encoding_nfs_export }} -> {{ video_encoding_pipeline_mount }}"
|
||||
pipeline_profiles: "{{ video_encoding_profile_names | default([]) }}"
|
||||
quadlet_directory: /etc/containers/systemd/users/982
|
||||
podman_store: "{{ video_encoding_podman_info.store.graphRoot }}"
|
||||
gpu_access: none (verified)
|
||||
|
||||
@@ -14,3 +14,10 @@ host_baseline_directories:
|
||||
mode: "0755"
|
||||
- path: /var/lib/ghostnet-ai/shared
|
||||
mode: "0755"
|
||||
|
||||
# Deployment sudoers drop-in (bootstrap-created 2026-09, adopted here).
|
||||
# (ALL:ALL) lets Ansible become the locked service accounts (become_user)
|
||||
# without a password; the deploy key is treated as root-equivalent anyway
|
||||
# (AGENTS.md) — with (root) alone, runas to other users would prompt.
|
||||
host_baseline_deploy_sudoers_path: /etc/sudoers.d/90-ansible-deploy
|
||||
host_baseline_deploy_sudoers_content: "ansible-deploy ALL=(ALL:ALL) NOPASSWD: ALL\n"
|
||||
|
||||
@@ -57,3 +57,12 @@
|
||||
loop: "{{ host_baseline_directories }}"
|
||||
loop_control:
|
||||
label: "{{ item.path }}"
|
||||
|
||||
- name: Manage the deployment sudoers drop-in
|
||||
ansible.builtin.copy:
|
||||
content: "{{ host_baseline_deploy_sudoers_content }}"
|
||||
dest: "{{ host_baseline_deploy_sudoers_path }}"
|
||||
owner: root
|
||||
group: root
|
||||
mode: "0440"
|
||||
validate: /usr/sbin/visudo -cf %s
|
||||
|
||||
@@ -1,17 +1,21 @@
|
||||
# Workload accounts
|
||||
|
||||
The `workload_accounts` role creates separate locked service identities for AI
|
||||
Production, AI Testing and CPU-based video encoding. Each identity receives a
|
||||
stable UID/GID, a non-overlapping subordinate ID range, a private data root, an
|
||||
administrator-managed rootless Quadlet directory and a persistent systemd user
|
||||
manager.
|
||||
Production, AI Testing, video encoding, the download workload (jDownloader2
|
||||
plus VPN sidecar) and the monitoring dashboard. Each identity receives a
|
||||
stable UID/GID, a non-overlapping subordinate ID range, a private data root,
|
||||
an administrator-managed rootless Quadlet directory and a persistent systemd
|
||||
user manager. The three encoding-pipeline identities (`gn-encoding` 982,
|
||||
`gn-download` 983, `gn-dashboard` 984) share the supplementary group
|
||||
`gn-media` (GID 970) for the pipeline handover directories (GNENC-002).
|
||||
|
||||
Configuration remains owned by root and is readable by the corresponding
|
||||
service group. Workload data and the rootless Podman store belong to the
|
||||
service account. The role verifies that all three Podman stores are rootless
|
||||
and use distinct graph roots.
|
||||
service account. The role verifies that all Podman stores are rootless and
|
||||
use distinct graph roots.
|
||||
|
||||
The two AI accounts receive `render` as an explicit supplementary-group
|
||||
baseline; the encoding account receives no GPU group. Hardware discovery and
|
||||
effective device-access checks belong to `amd_gpu_access`. This role does not
|
||||
define containers, networks, schedules or resource limits.
|
||||
baseline; the three pipeline accounts receive no GPU group. This role does not
|
||||
define containers, networks, schedules or resource limits. Pipeline data
|
||||
directories live on the NFS export and are managed by the `video_encoding`
|
||||
role under the pipeline mount (GNENC-002 section 4).
|
||||
|
||||
@@ -25,21 +25,50 @@ workload_accounts:
|
||||
home: /var/lib/ghostnet-encoding
|
||||
config_directory: /etc/ghostnet-encoding
|
||||
subid_start: 493216
|
||||
supplementary_groups: []
|
||||
# Primary group gn-media: with AUTH_SYS over NFSv4 the server-side
|
||||
# permission check did not honour the supplementary group membership
|
||||
# (mkdir EACCES despite group rwx; verified 2026-10-30). The pipeline
|
||||
# identities therefore carry gn-media as their primary group; the
|
||||
# per-UID groups remain for the root-owned config directories.
|
||||
primary_group: gn-media
|
||||
supplementary_groups:
|
||||
- gn-media
|
||||
- gn-encoding
|
||||
- name: gn-download
|
||||
uid: 983
|
||||
gid: 983
|
||||
comment: Ghostnet download workload (jDownloader2, VPN sidecar)
|
||||
home: /var/lib/ghostnet-download
|
||||
config_directory: /etc/ghostnet-download
|
||||
subid_start: 558752
|
||||
primary_group: gn-media
|
||||
supplementary_groups:
|
||||
- gn-media
|
||||
- gn-download
|
||||
- name: gn-dashboard
|
||||
uid: 984
|
||||
gid: 984
|
||||
comment: Ghostnet monitoring dashboard workload
|
||||
home: /var/lib/ghostnet-dashboard
|
||||
config_directory: /etc/ghostnet-dashboard
|
||||
subid_start: 624288
|
||||
primary_group: gn-media
|
||||
supplementary_groups:
|
||||
- gn-media
|
||||
- gn-dashboard
|
||||
|
||||
# Shared group of the encoding pipeline accounts (GNENC-002 section 3).
|
||||
# GID 970 is also declared on CT-101 (GNAI-017 section 4.2); note that the
|
||||
# CT-101 guest-side GID does not match the host-side GID through the LXC
|
||||
# idmap — see GNENC-002 sections 3 and 4.
|
||||
workload_shared_groups:
|
||||
- name: gn-media
|
||||
gid: 970
|
||||
|
||||
workload_subid_count: 65536
|
||||
workload_quadlet_user_directory: /etc/containers/systemd/users
|
||||
|
||||
workload_data_directories:
|
||||
- path: /var/lib/ghostnet-encoding/input
|
||||
owner: 982
|
||||
group: 982
|
||||
mode: "0750"
|
||||
- path: /var/lib/ghostnet-encoding/output
|
||||
owner: 982
|
||||
group: 982
|
||||
mode: "0750"
|
||||
- path: /var/lib/ghostnet-encoding/work
|
||||
owner: 982
|
||||
group: 982
|
||||
mode: "0750"
|
||||
# Pipeline directories live on the NFS export and are managed by the
|
||||
# video_encoding role under the pipeline mount (GNENC-002 section 4); no
|
||||
# local workload data directories remain.
|
||||
workload_data_directories: []
|
||||
|
||||
@@ -10,6 +10,17 @@
|
||||
workload_accounts | length
|
||||
fail_msg: Workload account names and numeric IDs must be unique.
|
||||
|
||||
- name: Create shared workload groups
|
||||
ansible.builtin.group:
|
||||
name: "{{ workload_shared_group.name }}"
|
||||
gid: "{{ workload_shared_group.gid }}"
|
||||
system: true
|
||||
state: present
|
||||
loop: "{{ workload_shared_groups }}"
|
||||
loop_control:
|
||||
loop_var: workload_shared_group
|
||||
label: "{{ workload_shared_group.name }} (GID {{ workload_shared_group.gid }})"
|
||||
|
||||
- name: Create dedicated workload groups
|
||||
ansible.builtin.group:
|
||||
name: "{{ workload_account.name }}"
|
||||
@@ -25,7 +36,7 @@
|
||||
ansible.builtin.user:
|
||||
name: "{{ workload_account.name }}"
|
||||
uid: "{{ workload_account.uid }}"
|
||||
group: "{{ workload_account.name }}"
|
||||
group: "{{ workload_account.primary_group | default(workload_account.name) }}"
|
||||
comment: "{{ workload_account.comment }}"
|
||||
home: "{{ workload_account.home }}"
|
||||
shell: /usr/sbin/nologin
|
||||
|
||||
@@ -15,6 +15,13 @@ Aktuell verwaltete Host-State:
|
||||
- **Kernel-NFS-Export** `/media/datenhdd3/Encoding` → `ghostnet-ki`
|
||||
(`10.0.0.90/32`), NFSv4-only — Rolle `pve_nfs_export` (Ersatz für das
|
||||
frühere manuelle Script; CT 101 kann NFS nicht selbst serve, siehe GNAI-019).
|
||||
- **Legacy-Medien-Cron entfernt**: Die root-Crontab-Einträge
|
||||
(`chown -R root:root` / `chmod -R 0777` auf allen Medien-Mounts, alle 10
|
||||
Minuten, installiert 2026-10-11) sind entfernt — Rolle `pve_media_cron`
|
||||
(Betreiberentscheidung 30.10.2026; Backup unter
|
||||
`/var/backups/ghostnet/pve/`). Der Cron hätte jedes echte
|
||||
Berechtigungsmodell unter den Medien-Mounts — inklusive der
|
||||
Encoding-Pipeline — alle 10 Minuten plattgewalzt.
|
||||
- **LXC-Configs**: Entfernen der Ganesha-Ära-`lxc.seccomp.profile:`-Leerzeile
|
||||
aus `/etc/pve/lxc/101.conf` — Rolle `pve_lxc_conf` (wirksam beim nächsten
|
||||
CT-Start/Reboot).
|
||||
|
||||
@@ -6,4 +6,5 @@
|
||||
|
||||
roles:
|
||||
- role: pve_lxc_conf
|
||||
- role: pve_media_cron
|
||||
- role: pve_nfs_export
|
||||
|
||||
@@ -144,6 +144,29 @@
|
||||
/etc/pve/lxc/101.conf still carries the empty Ganesha-era
|
||||
lxc.seccomp.profile line (applies at the next CT start).
|
||||
|
||||
- name: Read the root crontab
|
||||
ansible.builtin.command:
|
||||
argv:
|
||||
- /usr/bin/crontab
|
||||
- -l
|
||||
- -u
|
||||
- root
|
||||
check_mode: false
|
||||
changed_when: false
|
||||
failed_when: false
|
||||
register: pve_verify_crontab
|
||||
|
||||
- name: Require the legacy media permission cron to be absent
|
||||
ansible.builtin.assert:
|
||||
that: >-
|
||||
pve_verify_crontab.rc != 0 or
|
||||
(pve_verify_crontab.stdout |
|
||||
regex_findall('(chown -R root:root|chmod -R 0777) /media/(daten1|daten2|daten3|datenhdd1|datenhdd3)')
|
||||
| length == 0)
|
||||
fail_msg: >-
|
||||
The legacy recursive permission cron is active again; it flattens
|
||||
the encoding pipeline ownership and modes every 10 minutes.
|
||||
|
||||
- name: Report the verified PVE host production state
|
||||
ansible.builtin.debug:
|
||||
msg:
|
||||
|
||||
@@ -0,0 +1,13 @@
|
||||
# `pve_media_cron`
|
||||
|
||||
Removes the legacy root crontab entries (installed 2026-10-11) that ran
|
||||
`chown -R root:root` and `chmod -R 0777` on all media mounts
|
||||
(`/media/daten1`, `/media/daten2`, `/media/daten3`, `/media/datenhdd1`,
|
||||
`/media/datenhdd3`) every 10 minutes. The entries predate the encoding
|
||||
pipeline and silently flattened every real ownership and mode below the
|
||||
mounts, including the pipeline export root and layout (GNENC-002 section 4).
|
||||
|
||||
The full root crontab is backed up once to
|
||||
`/var/backups/ghostnet/pve/root-crontab-before-legacy-removal-20261030.txt`
|
||||
before removal. Existing media files keep their current (root:root 0777)
|
||||
state; the cron only stops re-enforcing it. Operator decision 2026-10-30.
|
||||
@@ -0,0 +1,8 @@
|
||||
---
|
||||
# Legacy root crontab entries (installed 2026-10-11) flattened real ownership
|
||||
# and mode on all media mounts every 10 minutes, which is incompatible with
|
||||
# the encoding pipeline permission model (GNENC-002 section 4).
|
||||
pve_media_cron_backup_directory: /var/backups/ghostnet/pve
|
||||
pve_media_cron_backup_name: root-crontab-before-legacy-removal-20261030.txt
|
||||
pve_media_cron_legacy_pattern: >-
|
||||
(chown -R root:root|chmod -R 0777) /media/(daten1|daten2|daten3|datenhdd1|datenhdd3)
|
||||
@@ -0,0 +1,66 @@
|
||||
---
|
||||
- name: Read the root crontab
|
||||
ansible.builtin.command:
|
||||
argv:
|
||||
- /usr/bin/crontab
|
||||
- -l
|
||||
- -u
|
||||
- root
|
||||
check_mode: false
|
||||
changed_when: false
|
||||
failed_when: false
|
||||
register: pve_media_crontab_probe
|
||||
|
||||
- name: Detect the legacy media permission cron
|
||||
ansible.builtin.set_fact:
|
||||
pve_media_cron_legacy_present: >-
|
||||
{{ pve_media_crontab_probe.rc == 0 and
|
||||
(pve_media_crontab_probe.stdout | regex_findall(pve_media_cron_legacy_pattern) | length > 0) }}
|
||||
|
||||
- name: Create the crontab backup directory
|
||||
ansible.builtin.file:
|
||||
path: "{{ pve_media_cron_backup_directory }}"
|
||||
state: directory
|
||||
owner: root
|
||||
group: root
|
||||
mode: "0700"
|
||||
|
||||
- name: Back up the root crontab before removing the legacy entries
|
||||
ansible.builtin.copy:
|
||||
content: "{{ pve_media_crontab_probe.stdout }}\n"
|
||||
dest: "{{ pve_media_cron_backup_directory }}/{{ pve_media_cron_backup_name }}"
|
||||
owner: root
|
||||
group: root
|
||||
mode: "0600"
|
||||
when: pve_media_cron_legacy_present | bool
|
||||
|
||||
- name: Remove the legacy media permission lines from the root crontab
|
||||
ansible.builtin.shell: |
|
||||
set -euo pipefail
|
||||
crontab -l -u root | grep -vE '{{ pve_media_cron_legacy_pattern }}' | crontab -u root -
|
||||
args:
|
||||
executable: /bin/bash
|
||||
when: pve_media_cron_legacy_present | bool
|
||||
|
||||
- name: Require the legacy media permission cron to be gone
|
||||
ansible.builtin.command:
|
||||
argv:
|
||||
- /usr/bin/crontab
|
||||
- -l
|
||||
- -u
|
||||
- root
|
||||
check_mode: false
|
||||
changed_when: false
|
||||
failed_when: false
|
||||
register: pve_media_crontab_after
|
||||
|
||||
- name: Assert the legacy media permission cron is removed
|
||||
ansible.builtin.assert:
|
||||
that: >-
|
||||
pve_media_crontab_after.rc != 0 or
|
||||
(pve_media_crontab_after.stdout | regex_findall(pve_media_cron_legacy_pattern) | length == 0)
|
||||
fail_msg: >-
|
||||
The legacy recursive permission cron (chown -R root:root / chmod -R 0777
|
||||
on the media mounts) is still active in the root crontab; it flattens the
|
||||
encoding pipeline ownership and modes every 10 minutes (GNENC-002).
|
||||
success_msg: The legacy media permission cron is removed.
|
||||
Reference in New Issue
Block a user