Encoding: Pipeline-Konten, NFS-Mount, Layout und Handover-Simulation umgesetzt und verifiziert.

- workload_accounts: gn-media (GID 970), gn-download (983), gn-dashboard (984);
  gn-media als primäre Gruppe der Pipeline-Konten — AUTH_SYS ueber NFSv4 wirkte
  serverseitig nicht mit Supplementary-Gruppen (Betriebsbefund 30.10.2026);
  Sudoers des Deploy-Kontos auf (ALL:ALL) verbreitert (become_user)
- video_encoding: NFS-Client, Mount-Unit und Pipeline-Layout auf dem Export
  (10.0.0.2:/media/datenhdd3/Encoding -> /var/lib/ghostnet-pipeline, ausserhalb
  der Homes); Layout-Anlage als Dienstkonten wegen root_squash; Verifikation
  der Verzeichnisrechte als gn-encoding (2777/2770/0750/2750, Gruppe 970)
- pve_media_cron: Legacy-Crontab (chown root:root + chmod -R 0777 auf allen
  Medienplatten, alle 10 min) entfernt — haette jedes echte Berechtigungsmodell
  plattgewalzt; Backup unter /var/backups/ghostnet/pve/; verify-production
  prueft das Fehlen ab
- verify-pipeline-handover.yml: Handover-Simulation über alle Identitaeten
  (CT-101-.dlc-Drop -> gn-download -> queue -> gn-encoding -> output/state ->
  gn-dashboard); einmal vollstaendig bestanden (30.10.2026)
- Doku: GNENC-000/001/002, GNAI-017 (6.10/6.11), GNMON-001 auf Stand gebracht

Generated by Mistral Vibe.
Co-Authored-By: Mistral Vibe <vibe@mistral.ai>
This commit is contained in:
2026-10-30 20:47:42 +01:00
parent 88b063a234
commit 69930f815a
21 changed files with 818 additions and 97 deletions
@@ -33,6 +33,18 @@ beschrieben.
NFSv4-only wirksam (`-3`), `verify-production` `ok=17 changed=0`,
Idempotenz `changed=0`. Berechtigungsmodell entschieden: Mode 2777 auf den
SMB-berührten Verzeichnissen (GNENC-002 Abschnitt 4).
- Implementierung 30.10.2026: Konten `gn-download` (983), `gn-dashboard` (984)
und Gruppe `gn-media` (GID 970) per `workload_accounts` angelegt — `gn-media`
als **primäre** Gruppe der drei Pipeline-Konten (AUTH_SYS über NFSv4 wirkte
serverseitig nicht mit Supplementary-Gruppen; Befund 30.10.2026,
GNENC-002 Abschnitt 3); Sudoers des Deploy-Kontos auf `(ALL:ALL)` verbreitert
(`become_user` für die Dienstkonten). NFS-Mount auf
`/var/lib/ghostnet-pipeline` aktiv (außerhalb der Homes); Layout auf dem
Export erzeugt und verifiziert; Handover-Simulation
(`verify-pipeline-handover.yml`) einmal vollständig bestanden
(CT-101-`.dlc`-Drop → gn-download → queue → gn-encoding → output/state →
gn-dashboard). Legacy-Medien-Cron auf dem PVE-Host entfernt
(`pve_media_cron`, GNAI-017 Abschnitt 6.11).
- Verzeichnislayout: `input/<Profil>/*.dlc`, `jd/`, `queue/<Profil>/<Release>/`,
`work/`, `output/<Profil>/<Release>/` + `output/_failed/`, `state/jobs/` +
`state/system.json`.
@@ -71,12 +83,12 @@ beschrieben.
## Noch offen
1. NFSv4.2-Mount-Abnahme von `ghostnet-ki` gegen `10.0.0.2` (inkl.
Schreib-/Lesetest als UID 982/983 und mv-Simulation als `ghost` über SMB)
— mit der Encoding-Implementierung.
1. Output-Pickup über CT-101 (SMB) einmal als geschlossener Ablauf
nachweisen (Nachtest zum 30.10.2026 — der Durchlauf wurde durch den
Server-Shutdown unterbrochen; GNENC-002 Abschnitt 11).
2. Feinjustierung der Ressourcenlimits nach einem repräsentativen Encoding-Test
unter gleichzeitiger Inferenzlast.
3. Rootless-TUN-Verifizierung für den VPN-Sidecar auf dem Host.
4. Implementierung auf ghostnet-ki: Konten `gn-download`/`gn-dashboard`,
NFS-Mount, Image-Bauten, Quadlet-Units (inkl. VPN-Sidecar), Skripte,
Monitoring-Bereich (Collector, Dashboard).
4. Implementierung auf ghostnet-ki: Image-Bauten, Quadlet-Units (inkl.
VPN-Sidecar), Worker-Skript, JD2-Event-Scripter, Monitoring-Bereich
(Collector, Dashboard).
@@ -32,7 +32,7 @@ beschrieben.
| --- | --- | --- |
| Repository | Host- und Deployment-Ansible bleibt im bestehenden Infrastructure-Repository | angenommen |
| Dokumentation | Encoding liegt unter `docs/platform/llm-server/encoding/` und verwendet die Kennung `GNENC`; Monitoring unter `docs/platform/llm-server/monitoring/` mit der Kennung `GNMON`; CT 101 unter `docs/platform/proxmox-server/lxc/ct-101-samba/` mit GNAI-017 | umgesetzt |
| Ausführungsidentitäten | drei gesperrte Konten: `gn-encoding` (UID/GID 982, Worker), `gn-download` (983, jDownloader2 + VPN-Sidecar), `gn-dashboard` (984, Dashboard, Monitoring-Bereich); gemeinsame Gruppe `gn-media` (GID 970) für die Übergabe-Verzeichnisse | gn-encoding eingerichtet und geprüft; weitere geplant |
| Ausführungsidentitäten | drei gesperrte Konten: `gn-encoding` (UID 982, Worker), `gn-download` (983, jDownloader2 + VPN-Sidecar), `gn-dashboard` (984, Dashboard, Monitoring-Bereich); gemeinsame Gruppe `gn-media` (GID 970) als **primäre** Gruppe der drei Konten (AUTH_SYS über NFSv4 wirkte serverseitig nur mit primärer Gruppe; Befund 30.10.2026) | alle drei Konten eingerichtet und verifiziert 30.10.2026 (Idempotenz `changed=0`) |
| Containertrennung | je Konto ein eigener rootless Podman-Store und ein eigener rootverwalteter Quadlet-Pfad | gn-encoding eingerichtet und verifiziert; weitere geplant |
| Rechenressource | Encoding ist CPU-basiert ohne GPU-Zugriff; der host-seitige Metrics-Collector (Monitoring-Bereich) liest GPU-Metriken ausschließlich lesend für das Dashboard | Planungsentscheidung |
| Parallelität | darf parallel zu AI Production oder AI Testing laufen | Planungsentscheidung |
@@ -41,7 +41,7 @@ beschrieben.
| VPN | der gesamte jDownloader2-Verkehr läuft über einen ProtonVPN-WireGuard-Tunnel: gluetun-Sidecar (`qmcgaw/gluetun`, gepinnt) unter `gn-download`, JD2 teilt dessen Netzwerk-Namespace, Kill Switch aktiv (kein Traffic ohne Tunnel); WireGuard-Credentials server-lokal in `/etc/ghostnet-download/vpn.env`, nicht in git | Betreiberentscheidung 19.10.2026 |
| Anwendung | Worker-Image als Eigenbau auf Basis des gepinnten HandBrake-Images `linuxserver/handbrake:1.11.2-ls48` (`sha256:fda92bd1…48a3c7`) mit python3 und ffmpeg; die Digests von Worker- und VPN-Image werden nach dem Bau gepinnt | Betreiberentscheidung 19.10.2026 |
| Speicherort | Pipeline-Daten liegen auf dem PVE-Host-Medienverbund (`/media/datenhdd3`; per LXC-Mount auch in CT-101 sichtbar): dort liegt die Medienbibliothek; ghostnet-ki selbst hat nur eine 2-TB-NVMe (OS + LLM) | Betreiberentscheidung 19.10.2026; NFS seit 30.10.2026 auf dem PVE-Host (GNAI-019 Entscheidung 2) |
| Ein-/Ausgabe | NFS-Export `10.0.0.2:/media/datenhdd3/Encoding` (PVE-Host, Rolle `pve_nfs_export`, NFSv4-only, nur für 10.0.0.90, `root_squash`), gemountet auf `/var/lib/ghostnet-encoding`; Verzeichnislayout `input/`, `jd/`, `queue/`, `work/`, `output/`, `state/` | Betreiberentscheidung 19.10.2026; Umsetzung GNAI-019 (Revision 30.10.2026); Mount-Abnahme durch das Encoding-Ansible ausstehend |
| Ein-/Ausgabe | NFS-Export `10.0.0.2:/media/datenhdd3/Encoding` (PVE-Host, Rolle `pve_nfs_export`, NFSv4-only, nur für 10.0.0.90, `root_squash`), gemountet auf `/var/lib/ghostnet-pipeline` (außerhalb der Konto-Homes); Verzeichnislayout `input/`, `jd/`, `queue/`, `work/`, `output/`, `state/` | Betreiberentscheidung 19.10.2026; Mount aktiv, Layout und Handover verifiziert 30.10.2026 |
| Abholung | fertige Outputs per SMB über CT-101: Share `[Encode]` (`output/`, RW für `ghost`, lesend für `daidalaw`/`debugofen`) — Entnahme per `mv` in die Bibliotheks-Shares räumt `output/` auf; optionaler Clean-Job löscht Dateien nach 30 Tagen aus `output/`; kein smbd auf `ghostnet-ki` | Betreiberentscheidung 19.10.2026 |
| Samba-Integration (CT-101) | CT-101 ist per Ansible adoptiert (GNAI-019; verifiziert 30.10.2026, `ok=47 changed=0`): Shares `[Encode]`/`[Encode-In]`/`[Encoding]`, Gruppe `gn-media` (GID 970) mit `ghost` als Mitglied; `.dlc`-Drop über `[Encode-In]`. NFS liegt nicht auf CT-101 (PVE-Host); die guest-seitige GID 970 gleicht die Host-GID 970 wegen des LXC-idmap nicht ab — Berechtigungsmodell: Mode 2777 auf den SMB-berührten Verzeichnissen (Betreiberentscheidung 30.10.2026, GNENC-002 Abschnitt 4) | Betreiberentscheidung 19.10.2026; GNAI-017/GNAI-019 |
| Profile | drei versionierte HandBrake-Presets (SVT-AV1 10-bit, MKV, Audio-Passthrough) unter `llm-server/encoding/profiles/`; Auswahl über gleichnamige Ordner unter `input/` bzw. `queue/` | Betreiberentscheidung 19.10.2026; Presets versioniert |
@@ -232,20 +232,19 @@ jeweils `ok=30 changed=0 failed=0`. Das gepinnte HandBrake-Image
anschließend in den rootless Store von `gn-encoding` gezogen; der Pull
bestätigte den Manifest-Digest.
Am 19.10.2026 wurde die Planung abgeschlossen (Abschnitte 2 und 7). Die
externen Abhängigkeiten sind umgesetzt: CT-101 ist per Ansible adoptiert
(GNAI-019, verifiziert 30.10.2026), der NFS-Export läuft auf dem PVE-Host
(Rolle `pve_nfs_export`); der PVE-Host ist konvergiert (Export-Root
`100000:970 2775`, NFSv4-only wirksam, `verify-production` `ok=17 changed=0`,
Idempotenz `changed=0`; autorisierter Lauf 30.10.2026). Das
Berechtigungsmodell für die SMB-Schreibpfade ist entschieden (Mode 2777,
GNENC-002 Abschnitt 4); offener Restpunkt ist die NFSv4.2-Mount-Abnahme von
ghostnet-ki. Die Implementierungsphase auf
ghostnet-ki baut als Nächstes auf: die Konten `gn-download` und `gn-dashboard`
samt Gruppe `gn-media` (GID 970), den NFS-Mount von `10.0.0.2`, die
Image-Bauten mit Digest-Pinning (Worker, VPN-Sidecar, Dashboard), die
Quadlet-Units für jDownloader2, VPN-Sidecar, Worker und Dashboard, den
host-seitigen Metrics-Collector sowie die Skripte aus `scripts/`. Die Rolle
Am 19.10.2026 wurde die Planung abgeschlossen (Abschnitte 2 und 7). Stand
30.10.2026: Die externen Abhängigkeiten sind umgesetzt und verifiziert —
CT-101 per Ansible adoptiert, der PVE-Host konvergiert (Export-Root
`100000:970 2775`, NFSv4-only wirksam, Legacy-Medien-Cron entfernt — GNAI-017
Abschnitt 6.9–6.11), die drei Pipeline-Konten samt Gruppe `gn-media`
(einschließlich Primärgruppen-Umstellung, GNAI-017 Abschnitt 6.10) angelegt,
der NFS-Mount auf `/var/lib/ghostnet-pipeline` aktiv, das Verzeichnislayout
auf dem Export erzeugt und verifiziert, und die Handover-Simulation
(`verify-pipeline-handover.yml`) einmal vollständig bestanden. Die
Implementierungsphase auf ghostnet-ki baut als Nächstes auf: die Image-Bauten
mit Digest-Pinning (Worker, VPN-Sidecar, Dashboard), die Quadlet-Units für
jDownloader2, VPN-Sidecar, Worker und Dashboard, den host-seitigen
Metrics-Collector sowie die Skripte aus `scripts/`. Die Rolle
`video_encoding` stellt bis dahin sicher, dass weder Quadlet-Dateien noch
Container für `gn-encoding` existieren.
@@ -41,17 +41,21 @@ Gruppe `gn-media`: gemeinsame Gruppe der drei Konten für die Übergabe-
Verzeichnisse und den lesenden Zugriff auf `state/`; GID 970. Alle Konten sind
gesperrt (Passwort `!`, Shell `/usr/sbin/nologin`, Linger aktiv).
**Anmerkung 30.10.2026 (LXC-idmap):** Die guest-seitige Gruppe `gn-media`
(GID 970) in CT-101 erscheint auf dem Host als GID 100970 und gleicht die
reale Host-GID 970 der Pipeline nicht ab — die Gruppen-Alignment wirkt nur
innerhalb von ghostnet-ki. Die SMB-Schreibpfade aus CT-101 laufen über
guest-root (Host-UID 100000, da `admin users` Dateioperationen als root
ausführt); siehe Abschnitt 11.
**Anmerkung 30.10.2026 (primäre Gruppe):** Die drei Pipeline-Konten tragen
`gn-media` als **primäre** Gruppe (die eigenen GID-Gruppen 982/983/984 bleiben
für die Config-Verzeichnisse als Supplementary-Gruppe erhalten). Grund:
AUTH_SYS über NFSv4 wirkte serverseitig nicht mit Supplementary-Gruppen —
`mkdir` als 982/983 bekam EACCES, obwohl `gn-media` Gruppen-Schreibrechte
hatte; mit primärer Gruppe 970 war der Zugriff sofort erfolgreich
(Betriebsbefund 30.10.2026, Testreihe auf ghostnet-ki). Zudem gilt die
LXC-idmap-Diskrepanz: die guest-seitige Gruppe `gn-media` (GID 970) in CT-101
erscheint auf dem Host als GID 100970 — die SMB-Schreibpfade laufen über
World-Write (Mode 2777), nicht über Gruppen-Abgleich; siehe Abschnitt 4.
## 4. Verzeichnislayout
```text
/var/lib/ghostnet-encoding/ (NFS-Mount: 10.0.0.2:/media/datenhdd3/Encoding, PVE-Host)
/var/lib/ghostnet-pipeline/ (NFS-Mount: 10.0.0.2:/media/datenhdd3/Encoding, PVE-Host; außerhalb der Konto-Homes)
├── input/ gn-media 2777 SMB-Drop ([Encode-In])
│ └── <Profil>/ gn-media 2777 nur .dlc
├── jd/ gn-media 2770 JD2-Arbeitsverzeichnis (gn-download)
@@ -92,16 +96,22 @@ Handover-Regeln:
Schreibzugriffen.
**Berechtigungsmodell (Betreiberentscheidung 30.10.2026):** Die SMB-Zugriffe
aus CT-101 laufen über guest-root (Host-UID 100000; `admin users` führt
Dateioperationen als root aus). Da die Pipeline-Verzeichnisse via NFS den
realen UIDs 982/983:970 gehören und diese im LXC-idmap nicht abgebildet sind
(im Gast: `nobody:nogroup`), tragen `input/` und der `output/`-Baum Mode
`2777`; `jd/`, `queue/`, `work/` und `state/` bleiben eng (2770/0750).
aus CT-101 laufen über guest-root bzw. beliebige Gast-Identitäten (Host-UIDs
100000+, da `admin users` Dateioperationen als root ausführt); die
Pipeline-Verzeichnisse gehören via NFS den realen UIDs 982/983 mit primärer
Gruppe 970. Da diese im LXC-idmap nicht abgebildet sind, tragen `input/` und
der `output/`-Baum Mode `2777` (World-Write); `jd/`, `queue/`, `work/` und
`state/` bleiben eng (2770/0750/2750). Der frühere Legacy-Cron auf dem
PVE-Host (`chown -R root:root` + `chmod -R 0777` auf allen Medienplatten alle
10 Minuten, installiert 11.10.2026) hätte jedes echte
Berechtigungsmodell plattgewalzt und wurde am 30.10.2026 entfernt (Rolle
`pve_media_cron`, Backup der Crontab unter `/var/backups/ghostnet/pve/`).
**Layout-Anlage:** Wegen `root_squash` auf dem Export kann Ansible als root
keine chowned-Verzeichnisse anlegen; die Struktur wird als `gn-encoding`
angelegt (runuser; Muster der Rolle `video_encoding`) oder vom Worker beim
Start erzeugt und geprüft.
beziehungsweise `gn-download` angelegt (`become_user`; die Rolle
`video_encoding` erzeugt und verifiziert das Layout vollständig — Stand
30.10.2026 umgesetzt, Verifikation bestanden).
**Abholung:** Die fertigen Outputs werden über CT-101 per SMB abgeholt: Share
`[Encode]` (Pfad `/media/datenhdd3/Encoding/output`, RW für `ghost`, lesend für
@@ -254,9 +264,13 @@ die Container gemountet (Updates ohne Image-Rebuild, versioniert über git).
## 11. Offene Punkte
1. NFSv4.2-Mount-Abnahme von `ghostnet-ki` gegen `10.0.0.2` — inkl.
Schreib-/Lesetest als UID 982/983 über den Mount und mv-Simulation als
`ghost` über SMB; mit der Encoding-Implementierung.
1. **Output-Pickup über CT-101 (SMB) einmal als geschlossener Ablauf
nachweisen:** Die Handover-Simulation (`verify-pipeline-handover.yml`)
lief am 30.10.2026 vollständig durch (`.dlc`-Drop aus CT-101 →
gn-download → queue → gn-encoding → output/state → gn-dashboard); das
Lesen/Löschen des Output-Testreleases über CT-101 wurde durch den
gleichzeitigen Server-Shutdown nicht mehr abgeschlossen und ist als
kurzer Nachtest zu wiederholen.
2. Feinjustierung der Ressourcenlimits nach Lasttest unter Inferenzlast
(GNENC-001 Abschnitt 4).
3. Rootless-TUN-Verifizierung für den VPN-Sidecar auf dem Host (Fallback:
@@ -42,7 +42,7 @@ Encoding-Pipeline). Keine GPU-Gruppenmitgliedschaft, keine `/dev/kfd`-/
## 5. Metrics-Collector
- Sampling alle 10 s; atomares Schreiben (`state/tmp/` + `mv`) nach
`/var/lib/ghostnet-encoding/state/system.json` (Mode `0644`).
`/var/lib/ghostnet-pipeline/state/system.json` (Mode `0644`).
- Schema:
```json
@@ -328,4 +328,16 @@ GNENC-002 Abschnitt 4): `input/`, `jd/`, `queue/`, `work/`, `output/`,
`input/`-Baum und dem gesamten `output/`-Baum (inkl. Release- und
`_failed/`-Verzeichnisse, vom Worker angelegt); `jd/`, `queue/`, `work/`
und `state/` bleiben eng (2770/2750/0750). Festgehalten in GNENC-002
Abschnitt 4.
Abschnitt 4. **Betriebsbefund 30.10.2026:** AUTH_SYS über NFSv4 wirkte
serverseitig nicht mit Supplementary-Gruppen — die Pipeline-Konten auf
ghostnet-ki tragen `gn-media` daher als **primäre** Gruppe (Befund und
Umsetzung: GNENC-002 Abschnitt 3).
11. **Legacy-Medien-Cron auf dem PVE-Host — erledigt (30.10.2026):** Die
root-Crontab enthielt seit dem 11.10.2026 zehnminütliche
`chown -R root:root`- und `chmod -R 0777`-Läufe über alle Medienplatten
(daten1/2/3, datenhdd1, datenhdd3) — sie hatte den Export-Root nach jeder
Konvergenz zurückgesetzt und hätte jedes echte Berechtigungsmodell
plattgewalzt. Entfernt über die Rolle `pve_media_cron` im Host-Projekt
(`proxmox/pve-host/ansible/`, Betreiberentscheidung 30.10.2026); Backup
der Crontab unter `/var/backups/ghostnet/pve/`; `verify-production` des
Host-Projekts prüft das Fehlen seither ab.
+9 -3
View File
@@ -7,11 +7,17 @@ area of the LLM-Server host root; the shared host baseline lives in
`playbooks/video-encoding.yml` applies the `video_encoding` role: it verifies
the locked `gn-encoding` account, enforces CPU-only device isolation, manages
the encoding directories and proves the separate rootless Podman store. No
encoder image, Quadlet, timer or schedule is defined yet; those are open
decisions in
the pipeline NFS mount (`10.0.0.2:/media/datenhdd3/Encoding` at
`/var/lib/ghostnet-pipeline`) and creates and verifies the pipeline
directory layout on the export as the service accounts. No encoder image,
Quadlet, timer or schedule is defined yet; those are open decisions in
[`../../../docs/platform/llm-server/encoding/`](../../../docs/platform/llm-server/encoding/GNENC-000-DOC_Index.md).
`playbooks/verify-pipeline-handover.yml` exercises the file handover between
the pipeline identities (GNENC-002 sections 5 and 6) on the mounted export;
apply `video-encoding.yml` first. The SMB write path from CT-101 (guest root)
is simulated separately on CT-101 (GNAI-017 section 6.10).
## Control-node setup
Use the shared virtual environment at the repository root (see
@@ -0,0 +1,205 @@
---
# Exercises the GNENC-002 file handover between the pipeline identities on the
# mounted NFS export: .dlc drop -> gn-download ingest -> jd/ -> queue/ ->
# gn-encoding pick-up -> work/ -> output/ -> state/ -> gn-dashboard read.
# Apply playbooks/video-encoding.yml first. The output test release is left in
# place for the CT-101 SMB pickup simulation and removed on the next run.
- name: Simulate the encoding pipeline file handover (GNENC-002 sections 5 and 6)
hosts: llm_servers
become: true
gather_facts: false
vars:
handover_profile: "AV1 MKV 1080p30 RF25"
handover_release: handover-test
handover_mount: /var/lib/ghostnet-pipeline
handover_dlc: "{{ handover_mount }}/input/{{ handover_profile }}/{{ handover_release }}.dlc"
handover_dlc_done: "{{ handover_mount }}/jd/dlc_done/{{ handover_release }}.dlc"
handover_jd_release: "{{ handover_mount }}/jd/{{ handover_release }}"
handover_queue_release: "{{ handover_mount }}/queue/{{ handover_profile }}/{{ handover_release }}"
handover_work_release: "{{ handover_mount }}/work/{{ handover_release }}"
handover_output_release: "{{ handover_mount }}/output/{{ handover_profile }}/{{ handover_release }}"
handover_output_file: "{{ handover_output_release }}/video.test.mkv"
handover_jobs: "{{ handover_mount }}/state/jobs"
handover_tmp: "{{ handover_mount }}/state/tmp"
handover_job_file: "{{ handover_jobs }}/handover-test.json"
tasks:
- name: Require the pipeline mount to be active
ansible.builtin.command:
argv:
- /usr/bin/systemctl
- is-active
- var-lib-ghostnet\x2dpipeline.mount
changed_when: false
register: handover_mount_status
failed_when: handover_mount_status.stdout != "active"
- name: Stat the handover directories
ansible.builtin.stat:
path: "{{ item }}"
loop:
- "{{ handover_mount }}/input/{{ handover_profile }}"
- "{{ handover_queue_release }}"
- "{{ handover_output_release }}"
- "{{ handover_mount }}/jd"
- "{{ handover_mount }}/work"
- "{{ handover_jobs }}"
- "{{ handover_tmp }}"
register: handover_dir_stats
- name: Require the pipeline layout to be present
ansible.builtin.assert:
that: >-
handover_dir_stats.results | map(attribute='stat.isdir')
| select('equalto', false) | list | length == 0
fail_msg: >-
The pipeline layout is missing; apply playbooks/video-encoding.yml
first (GNENC-002 section 4).
- name: Clean up leftovers from previous runs (gn-download side)
ansible.builtin.shell: |
set -euo pipefail
rm -f "{{ handover_dlc }}" "{{ handover_dlc_done }}"
rm -rf "{{ handover_jd_release }}" "{{ handover_queue_release }}"
args:
executable: /bin/bash
become_user: gn-download
environment:
HOME: /var/lib/ghostnet-download
changed_when: true
- name: Clean up leftovers from previous runs (gn-encoding side)
ansible.builtin.shell: |
set -euo pipefail
rm -rf "{{ handover_work_release }}" "{{ handover_output_release }}"
rm -f "{{ handover_job_file }}"
args:
executable: /bin/bash
become_user: gn-encoding
environment:
HOME: /var/lib/ghostnet-encoding
changed_when: true
- name: Probe the .dlc drop
ansible.builtin.stat:
path: "{{ handover_dlc }}"
register: handover_dlc_probe
- name: Provide the .dlc drop as gn-encoding when no external SMB simulation file exists
ansible.builtin.shell: |
set -euo pipefail
printf 'dlc-handover-test\n' > "{{ handover_dlc }}"
args:
executable: /bin/bash
become_user: gn-encoding
environment:
HOME: /var/lib/ghostnet-encoding
when: not handover_dlc_probe.stat.exists
- name: Ingest the .dlc as gn-download (read and move to jd/dlc_done)
ansible.builtin.shell: |
set -euo pipefail
cat "{{ handover_dlc }}" > /dev/null
mv "{{ handover_dlc }}" "{{ handover_dlc_done }}"
[ -f "{{ handover_dlc_done }}" ]
args:
executable: /bin/bash
become_user: gn-download
environment:
HOME: /var/lib/ghostnet-download
changed_when: true
- name: Simulate the jDownloader2 download in jd/
ansible.builtin.shell: |
set -euo pipefail
mkdir "{{ handover_jd_release }}"
dd if=/dev/zero of="{{ handover_jd_release }}/video.test" bs=1M count=1 status=none
args:
executable: /bin/bash
become_user: gn-download
environment:
HOME: /var/lib/ghostnet-download
changed_when: true
- name: Hand the release over to the queue (atomic move)
ansible.builtin.shell: |
set -euo pipefail
mv "{{ handover_jd_release }}" "{{ handover_queue_release }}"
[ -f "{{ handover_queue_release }}/video.test" ]
args:
executable: /bin/bash
become_user: gn-download
environment:
HOME: /var/lib/ghostnet-download
changed_when: true
- name: Pick up, encode and hand over to output as gn-encoding
ansible.builtin.shell: |
set -euo pipefail
cat "{{ handover_queue_release }}/video.test" > /dev/null
mkdir "{{ handover_work_release }}"
dd if=/dev/zero of="{{ handover_work_release }}/video.test.mkv" bs=1M count=1 status=none
mv "{{ handover_work_release }}/video.test.mkv" "{{ handover_output_file }}"
rm "{{ handover_queue_release }}/video.test"
rmdir "{{ handover_queue_release }}" "{{ handover_work_release }}"
[ -f "{{ handover_output_file }}" ]
args:
executable: /bin/bash
become_user: gn-encoding
environment:
HOME: /var/lib/ghostnet-encoding
changed_when: true
- name: Write the state JSON as gn-encoding (atomic via state/tmp)
ansible.builtin.shell: |
set -euo pipefail
printf '{"job_id":"handover-test","status":"done"}\n' > "{{ handover_tmp }}/handover-test.json"
mv "{{ handover_tmp }}/handover-test.json" "{{ handover_job_file }}"
[ -f "{{ handover_job_file }}" ]
args:
executable: /bin/bash
become_user: gn-encoding
environment:
HOME: /var/lib/ghostnet-encoding
changed_when: true
- name: Read the state JSON as gn-dashboard
ansible.builtin.shell: |
set -euo pipefail
grep -q '"status":"done"' "{{ handover_job_file }}"
args:
executable: /bin/bash
become_user: gn-dashboard
environment:
HOME: /var/lib/ghostnet-dashboard
changed_when: false
- name: Clean up the state test object
ansible.builtin.shell: |
set -euo pipefail
rm -f "{{ handover_job_file }}"
args:
executable: /bin/bash
become_user: gn-encoding
environment:
HOME: /var/lib/ghostnet-encoding
changed_when: true
- name: Stat the handed-over output file
ansible.builtin.stat:
path: "{{ handover_output_file }}"
register: handover_output_stats
- name: Verify the output file identity
ansible.builtin.assert:
that:
- handover_output_stats.stat.exists
- handover_output_stats.stat.uid | int == 982
- handover_output_stats.stat.gid | int == 970
fail_msg: >-
The output file must exist as 982:970 on the export; found
{{ handover_output_stats.stat.uid }}:{{ handover_output_stats.stat.gid }}.
success_msg: >-
Handover simulation complete; the output test release remains for the
CT-101 SMB pickup simulation and is removed on the next run.
@@ -6,17 +6,32 @@ baseline for the locked `gn-encoding` service account (UID/GID 982) on
It verifies the account identity, the locked password and the deterministic
subordinate ID ranges, enforces the absence of GPU device groups, manages the
root-owned configuration directory, the input/output/work data directories and
the administrator-managed rootless Quadlet directory, and proves that the
account cannot read `/dev/kfd` or any DRM render node and uses its own rootless
Podman store.
root-owned configuration directory and the administrator-managed rootless
Quadlet directory, and proves that the account cannot read `/dev/kfd` or any
DRM render node and uses its own rootless Podman store.
Pipeline storage comes from the PVE-host NFS export
(`10.0.0.2:/media/datenhdd3/Encoding`, GNAI-017/GNAI-019), mounted at
`/var/lib/ghostnet-encoding/pipeline` inside the account home — the home
itself is not the mountpoint, so the rootless Podman store below it stays
visible. The role installs the NFS client, manages the systemd mount unit and
creates the pipeline directory layout (`input/`, `jd/`, `queue/`, `work/`,
`output/`, `state/`, one subdirectory per HandBrake profile) as the service
accounts over NFS, because `root_squash` prevents root-owned creation. The
layout ownership and modes (2777 on the SMB-facing trees, 2770/0750/2750
elsewhere, group `gn-media` GID 970) are verified after convergence
(GNENC-002 section 4).
While no encoder image is selected (`video_encoding_image` empty), the role
additionally guards that no Quadlet file and no container exists for the
account. Encoder image, schedule, resource limits, queue handling and health
checks are open decisions in
`docs/platform/llm-server/encoding/GNENC-001` and are not assumed here.
account. Encoder image, schedule, resource limits and health checks remain
gated by the open decisions in `docs/platform/llm-server/encoding/GNENC-001`
and are not assumed here.
The account itself is created by the shared `workload_accounts` role in the
host project (`llm-server/host/ansible`); this role requires the host baseline
to be applied first.
host project (`llm-server/host/ansible`), together with `gn-download` (983),
`gn-dashboard` (984) and the shared group `gn-media` (GID 970); this role
requires the host baseline to be applied first.
The file handover between the pipeline identities is exercised by
`playbooks/verify-pipeline-handover.yml`.
@@ -8,6 +8,10 @@ video_encoding_account:
name: gn-encoding
uid: 982
gid: 982
# Primary login group is gn-media (GID 970): AUTH_SYS over NFSv4 did not
# honour the supplementary group membership server-side, so the pipeline
# identities carry gn-media as their primary group (verified 2026-10-30).
primary_gid: 970
home: /var/lib/ghostnet-encoding
config_directory: /etc/ghostnet-encoding
shell: /usr/sbin/nologin
@@ -15,18 +19,46 @@ video_encoding_account:
video_encoding_subid_count: 65536
video_encoding_directories:
- path: /var/lib/ghostnet-encoding/input
mode: "0750"
- path: /var/lib/ghostnet-encoding/output
mode: "0750"
- path: /var/lib/ghostnet-encoding/work
mode: "0750"
# Pipeline storage on the PVE-host NFS export (GNENC-001/002, GNAI-017).
# The mount lives OUTSIDE the service homes (the gn-encoding home is 0750 and
# must stay traversable only for its owner; gn-download and gn-dashboard need
# to reach the pipeline root without touching any account home).
video_encoding_nfs_server: 10.0.0.2
video_encoding_nfs_export: /media/datenhdd3/Encoding
video_encoding_pipeline_mount: /var/lib/ghostnet-pipeline
video_encoding_pipeline_mount_unit: var-lib-ghostnet\x2dpipeline.mount
video_encoding_pipeline_mount_options: nfsvers=4.2,hard,_netdev,noatime
video_encoding_pipeline_nfs_client_package: nfs-common
# Export root ownership on the PVE host (GNAI-017 section 4.1):
# 100000 = CT-101 guest root (SMB share writes), 970 = gn-media (pipeline).
video_encoding_export_root_uid: 100000
video_encoding_export_root_gid: 970
video_encoding_export_root_mode: "2775"
# Shared pipeline group of gn-encoding (982), gn-download (983) and
# gn-dashboard (984); created by the workload_accounts role.
video_encoding_media_group: gn-media
video_encoding_media_gid: 970
# Declared HandBrake profiles (must match the preset files under
# llm-server/encoding/profiles/; verified by the role, one input/, queue/
# and output/ subdirectory per profile).
video_encoding_profile_names:
- "AV1 MKV 1080p30 Anime RF34"
- "AV1 MKV 1080p30 RF25"
- "AV1 MKV 2160p60 4K RF23"
# Superseded local directories of the pre-NFS baseline (GNENC-001 section 6);
# removed when empty.
video_encoding_legacy_local_directories:
- /var/lib/ghostnet-encoding/input
- /var/lib/ghostnet-encoding/output
- /var/lib/ghostnet-encoding/work
video_encoding_quadlet_user_directory: /etc/containers/systemd/users
# Open decisions from docs/platform/llm-server/encoding/GNENC-001 section 7
# stay unset on purpose. Until an encoder image, a schedule and resource limits
# are decided, the role only manages the shared encoding baseline and guards
# that no workload is defined.
# are decided, the role additionally guards that no workload is defined.
video_encoding_image: ""
@@ -46,7 +46,7 @@
video_encoding_account.uid | int
- >-
video_encoding_account_entry.stdout.split(":")[3] | int ==
video_encoding_account.gid | int
video_encoding_account.primary_gid | int
- >-
video_encoding_account_entry.stdout.split(":")[5] ==
video_encoding_account.home
@@ -146,17 +146,264 @@
group: "{{ video_encoding_account.gid }}"
mode: "0750"
- name: Manage the encoding data directories
- name: Remove the superseded local pipeline directories when empty
ansible.builtin.shell: |
set -euo pipefail
changed=0
for dir in {{ video_encoding_legacy_local_directories | map('quote') | join(' ') }}; do
if [ -d "$dir" ]; then
if [ -n "$(find "$dir" -mindepth 1 -print -quit)" ]; then
echo "Refusing to remove non-empty legacy directory: $dir"
exit 1
fi
rmdir "$dir"
changed=1
fi
done
[ "$changed" = 0 ] || echo CHANGED
args:
executable: /bin/bash
register: video_encoding_legacy_cleanup
changed_when: "'CHANGED' in video_encoding_legacy_cleanup.stdout"
- name: Install the NFS client
ansible.builtin.apt:
name: "{{ video_encoding_pipeline_nfs_client_package }}"
state: present
update_cache: true
cache_valid_time: 3600
install_recommends: false
- name: Probe whether the pipeline mount is already active
ansible.builtin.command:
argv:
- /usr/bin/findmnt
- --mountpoint
- "{{ video_encoding_pipeline_mount }}"
check_mode: false
changed_when: false
failed_when: false
register: video_encoding_pipeline_mountpoint_probe
- name: Create the pipeline mountpoint outside the service homes
ansible.builtin.file:
path: "{{ video_encoding_directory.path }}"
path: "{{ video_encoding_pipeline_mount }}"
state: directory
owner: "{{ video_encoding_account.uid }}"
group: "{{ video_encoding_account.gid }}"
mode: "{{ video_encoding_directory.mode }}"
loop: "{{ video_encoding_directories }}"
loop_control:
loop_var: video_encoding_directory
label: "{{ video_encoding_directory.path }}"
owner: root
group: root
mode: "0755"
when: video_encoding_pipeline_mountpoint_probe.rc != 0
- name: Manage the pipeline NFS mount unit
ansible.builtin.copy:
content: |
[Unit]
Description=Ghostnet encoding pipeline NFS export (GNENC-002)
Requires=network-online.target
After=network-online.target
[Mount]
What={{ video_encoding_nfs_server }}:{{ video_encoding_nfs_export }}
Where={{ video_encoding_pipeline_mount }}
Type=nfs4
Options={{ video_encoding_pipeline_mount_options }}
[Install]
WantedBy=multi-user.target
dest: "/etc/systemd/system/{{ video_encoding_pipeline_mount_unit }}"
owner: root
group: root
mode: "0644"
- name: Reload systemd for the pipeline mount unit
ansible.builtin.systemd_service:
daemon_reload: true
when: not ansible_check_mode
- name: Enable and mount the pipeline NFS export
ansible.builtin.systemd_service:
name: "{{ video_encoding_pipeline_mount_unit }}"
enabled: true
state: started
when: not ansible_check_mode
- name: Verify the pipeline mount is active
ansible.builtin.command:
argv:
- /usr/bin/systemctl
- is-active
- "{{ video_encoding_pipeline_mount_unit }}"
check_mode: false
changed_when: false
register: video_encoding_pipeline_mount_status
failed_when: video_encoding_pipeline_mount_status.stdout != "active"
when: not ansible_check_mode
- name: Stat the mounted export root
ansible.builtin.stat:
path: "{{ video_encoding_pipeline_mount }}"
register: video_encoding_export_root
when: not ansible_check_mode
- name: Verify the mounted export root ownership and mode
ansible.builtin.assert:
that:
- video_encoding_export_root.stat.isdir
- video_encoding_export_root.stat.uid == video_encoding_export_root_uid | int
- video_encoding_export_root.stat.gid == video_encoding_export_root_gid | int
- >-
'%04o' % (video_encoding_export_root.stat.mode | int(0, 8)) ==
video_encoding_export_root_mode
fail_msg: >-
The mounted export root must be {{ video_encoding_export_root_uid }}:{{ video_encoding_export_root_gid }}
{{ video_encoding_export_root_mode }} (PVE host side, GNAI-017 section 4.1);
found {{ video_encoding_export_root.stat.uid }}:{{ video_encoding_export_root.stat.gid }}
{{ video_encoding_export_root.stat.mode }}.
success_msg: The pipeline export root is mounted with the declared identity.
when: not ansible_check_mode
- name: Collect the HandBrake preset files from the profiles directory
ansible.builtin.set_fact:
video_encoding_profile_files: >-
{{ lookup('ansible.builtin.fileglob', role_path ~ '/../../../profiles/*.json', wantlist=True)
| map('basename') | map('regex_replace', '\.json$', '') | list }}
- name: Require the declared profiles to match the versioned preset files
ansible.builtin.assert:
that:
- video_encoding_profile_names | sort == video_encoding_profile_files | sort
fail_msg: >-
The declared video_encoding_profile_names do not match the preset files
under llm-server/encoding/profiles/ (found:
{{ video_encoding_profile_files | join(', ') }}).
success_msg: The declared profiles match the versioned preset files.
- name: Create the gn-download pipeline directories on the export
ansible.builtin.shell: |
set -euo pipefail
changed=0
while read -r mode path; do
[ -n "${path:-}" ] || continue
if [ ! -d "$path" ]; then
install -d -m "$mode" "$path"
changed=1
elif [ "$(stat -c '%a' "$path")" != "$mode" ]; then
chmod "$mode" "$path"
changed=1
fi
done <<'EOF'
{% for d in [
(video_encoding_pipeline_mount ~ '/input', '2777'),
(video_encoding_pipeline_mount ~ '/jd', '2770'),
(video_encoding_pipeline_mount ~ '/jd/dlc_done', '2770'),
(video_encoding_pipeline_mount ~ '/jd/failed', '2770'),
(video_encoding_pipeline_mount ~ '/queue', '2770') ] %}
{{ d[1] }} {{ d[0] }}
{% endfor %}
{% for p in video_encoding_profile_names %}
2777 {{ video_encoding_pipeline_mount }}/input/{{ p }}
2770 {{ video_encoding_pipeline_mount }}/queue/{{ p }}
{% endfor %}
EOF
[ "$changed" = 0 ] || echo CHANGED
args:
executable: /bin/bash
become: true
become_user: gn-download
environment:
HOME: /var/lib/ghostnet-download
register: video_encoding_download_layout
changed_when: "'CHANGED' in video_encoding_download_layout.stdout"
when: not ansible_check_mode
- name: Create the gn-encoding pipeline directories on the export
ansible.builtin.shell: |
set -euo pipefail
changed=0
while read -r mode path; do
[ -n "${path:-}" ] || continue
if [ ! -d "$path" ]; then
install -d -m "$mode" "$path"
changed=1
elif [ "$(stat -c '%a' "$path")" != "$mode" ]; then
chmod "$mode" "$path"
changed=1
fi
done <<'EOF'
{% for d in [
(video_encoding_pipeline_mount ~ '/work', '0750'),
(video_encoding_pipeline_mount ~ '/output', '2777'),
(video_encoding_pipeline_mount ~ '/output/_failed', '2777'),
(video_encoding_pipeline_mount ~ '/state', '2750'),
(video_encoding_pipeline_mount ~ '/state/jobs', '0750'),
(video_encoding_pipeline_mount ~ '/state/tmp', '0750') ] %}
{{ d[1] }} {{ d[0] }}
{% endfor %}
{% for p in video_encoding_profile_names %}
2777 {{ video_encoding_pipeline_mount }}/output/{{ p }}
{% endfor %}
EOF
[ "$changed" = 0 ] || echo CHANGED
args:
executable: /bin/bash
become: true
become_user: gn-encoding
environment:
HOME: "{{ video_encoding_account.home }}"
register: video_encoding_worker_layout
changed_when: "'CHANGED' in video_encoding_worker_layout.stdout"
when: not ansible_check_mode
- name: Verify the pipeline directory layout on the export
ansible.builtin.shell: |
set -uo pipefail
failed=0
while read -r uid gid mode path; do
[ -n "${path:-}" ] || continue
if [ ! -d "$path" ]; then
echo "MISSING $path"
failed=1
continue
fi
read -r auid agid amode <<<"$(stat -c '%u %g %a' "$path")"
if [ "$auid" != "$uid" ] || [ "$agid" != "$gid" ] || [ "$amode" != "$mode" ]; then
echo "WRONG $path (expected $uid:$gid $mode, found $auid:$agid $amode)"
failed=1
fi
done <<'EOF'
{% for d in [
(video_encoding_pipeline_mount ~ '/input', '983', '2777'),
(video_encoding_pipeline_mount ~ '/jd', '983', '2770'),
(video_encoding_pipeline_mount ~ '/jd/dlc_done', '983', '2770'),
(video_encoding_pipeline_mount ~ '/jd/failed', '983', '2770'),
(video_encoding_pipeline_mount ~ '/queue', '983', '2770'),
(video_encoding_pipeline_mount ~ '/work', '982', '0750'),
(video_encoding_pipeline_mount ~ '/output', '982', '2777'),
(video_encoding_pipeline_mount ~ '/output/_failed', '982', '2777'),
(video_encoding_pipeline_mount ~ '/state', '982', '2750'),
(video_encoding_pipeline_mount ~ '/state/jobs', '982', '0750'),
(video_encoding_pipeline_mount ~ '/state/tmp', '982', '0750') ] %}
{{ d[1] }} {{ d[2] }} {{ d[0] }}
{% endfor %}
{% for p in video_encoding_profile_names %}
983 970 2777 {{ video_encoding_pipeline_mount }}/input/{{ p }}
983 970 2770 {{ video_encoding_pipeline_mount }}/queue/{{ p }}
982 970 2777 {{ video_encoding_pipeline_mount }}/output/{{ p }}
{% endfor %}
EOF
if [ "$failed" = 1 ]; then
echo "Pipeline layout verification failed"
exit 1
fi
echo LAYOUT_OK
args:
executable: /bin/bash
become_user: gn-encoding
environment:
HOME: "{{ video_encoding_account.home }}"
changed_when: false
register: video_encoding_layout_verify
when: not ansible_check_mode
- name: Manage the administrator-managed rootless Quadlet directory
ansible.builtin.file:
@@ -330,7 +577,8 @@
msg:
account: "gn-encoding (UID/GID 982)"
config_directory: /etc/ghostnet-encoding
data_directories: "{{ video_encoding_directories | map(attribute='path') | list }}"
pipeline_mount: "{{ video_encoding_nfs_server }}:{{ video_encoding_nfs_export }} -> {{ video_encoding_pipeline_mount }}"
pipeline_profiles: "{{ video_encoding_profile_names | default([]) }}"
quadlet_directory: /etc/containers/systemd/users/982
podman_store: "{{ video_encoding_podman_info.store.graphRoot }}"
gpu_access: none (verified)
@@ -14,3 +14,10 @@ host_baseline_directories:
mode: "0755"
- path: /var/lib/ghostnet-ai/shared
mode: "0755"
# Deployment sudoers drop-in (bootstrap-created 2026-09, adopted here).
# (ALL:ALL) lets Ansible become the locked service accounts (become_user)
# without a password; the deploy key is treated as root-equivalent anyway
# (AGENTS.md) — with (root) alone, runas to other users would prompt.
host_baseline_deploy_sudoers_path: /etc/sudoers.d/90-ansible-deploy
host_baseline_deploy_sudoers_content: "ansible-deploy ALL=(ALL:ALL) NOPASSWD: ALL\n"
@@ -57,3 +57,12 @@
loop: "{{ host_baseline_directories }}"
loop_control:
label: "{{ item.path }}"
- name: Manage the deployment sudoers drop-in
ansible.builtin.copy:
content: "{{ host_baseline_deploy_sudoers_content }}"
dest: "{{ host_baseline_deploy_sudoers_path }}"
owner: root
group: root
mode: "0440"
validate: /usr/sbin/visudo -cf %s
@@ -1,17 +1,21 @@
# Workload accounts
The `workload_accounts` role creates separate locked service identities for AI
Production, AI Testing and CPU-based video encoding. Each identity receives a
stable UID/GID, a non-overlapping subordinate ID range, a private data root, an
administrator-managed rootless Quadlet directory and a persistent systemd user
manager.
Production, AI Testing, video encoding, the download workload (jDownloader2
plus VPN sidecar) and the monitoring dashboard. Each identity receives a
stable UID/GID, a non-overlapping subordinate ID range, a private data root,
an administrator-managed rootless Quadlet directory and a persistent systemd
user manager. The three encoding-pipeline identities (`gn-encoding` 982,
`gn-download` 983, `gn-dashboard` 984) share the supplementary group
`gn-media` (GID 970) for the pipeline handover directories (GNENC-002).
Configuration remains owned by root and is readable by the corresponding
service group. Workload data and the rootless Podman store belong to the
service account. The role verifies that all three Podman stores are rootless
and use distinct graph roots.
service account. The role verifies that all Podman stores are rootless and
use distinct graph roots.
The two AI accounts receive `render` as an explicit supplementary-group
baseline; the encoding account receives no GPU group. Hardware discovery and
effective device-access checks belong to `amd_gpu_access`. This role does not
define containers, networks, schedules or resource limits.
baseline; the three pipeline accounts receive no GPU group. This role does not
define containers, networks, schedules or resource limits. Pipeline data
directories live on the NFS export and are managed by the `video_encoding`
role under the pipeline mount (GNENC-002 section 4).
@@ -25,21 +25,50 @@ workload_accounts:
home: /var/lib/ghostnet-encoding
config_directory: /etc/ghostnet-encoding
subid_start: 493216
supplementary_groups: []
# Primary group gn-media: with AUTH_SYS over NFSv4 the server-side
# permission check did not honour the supplementary group membership
# (mkdir EACCES despite group rwx; verified 2026-10-30). The pipeline
# identities therefore carry gn-media as their primary group; the
# per-UID groups remain for the root-owned config directories.
primary_group: gn-media
supplementary_groups:
- gn-media
- gn-encoding
- name: gn-download
uid: 983
gid: 983
comment: Ghostnet download workload (jDownloader2, VPN sidecar)
home: /var/lib/ghostnet-download
config_directory: /etc/ghostnet-download
subid_start: 558752
primary_group: gn-media
supplementary_groups:
- gn-media
- gn-download
- name: gn-dashboard
uid: 984
gid: 984
comment: Ghostnet monitoring dashboard workload
home: /var/lib/ghostnet-dashboard
config_directory: /etc/ghostnet-dashboard
subid_start: 624288
primary_group: gn-media
supplementary_groups:
- gn-media
- gn-dashboard
# Shared group of the encoding pipeline accounts (GNENC-002 section 3).
# GID 970 is also declared on CT-101 (GNAI-017 section 4.2); note that the
# CT-101 guest-side GID does not match the host-side GID through the LXC
# idmap — see GNENC-002 sections 3 and 4.
workload_shared_groups:
- name: gn-media
gid: 970
workload_subid_count: 65536
workload_quadlet_user_directory: /etc/containers/systemd/users
workload_data_directories:
- path: /var/lib/ghostnet-encoding/input
owner: 982
group: 982
mode: "0750"
- path: /var/lib/ghostnet-encoding/output
owner: 982
group: 982
mode: "0750"
- path: /var/lib/ghostnet-encoding/work
owner: 982
group: 982
mode: "0750"
# Pipeline directories live on the NFS export and are managed by the
# video_encoding role under the pipeline mount (GNENC-002 section 4); no
# local workload data directories remain.
workload_data_directories: []
@@ -10,6 +10,17 @@
workload_accounts | length
fail_msg: Workload account names and numeric IDs must be unique.
- name: Create shared workload groups
ansible.builtin.group:
name: "{{ workload_shared_group.name }}"
gid: "{{ workload_shared_group.gid }}"
system: true
state: present
loop: "{{ workload_shared_groups }}"
loop_control:
loop_var: workload_shared_group
label: "{{ workload_shared_group.name }} (GID {{ workload_shared_group.gid }})"
- name: Create dedicated workload groups
ansible.builtin.group:
name: "{{ workload_account.name }}"
@@ -25,7 +36,7 @@
ansible.builtin.user:
name: "{{ workload_account.name }}"
uid: "{{ workload_account.uid }}"
group: "{{ workload_account.name }}"
group: "{{ workload_account.primary_group | default(workload_account.name) }}"
comment: "{{ workload_account.comment }}"
home: "{{ workload_account.home }}"
shell: /usr/sbin/nologin
+7
View File
@@ -15,6 +15,13 @@ Aktuell verwaltete Host-State:
- **Kernel-NFS-Export** `/media/datenhdd3/Encoding` → `ghostnet-ki`
(`10.0.0.90/32`), NFSv4-only — Rolle `pve_nfs_export` (Ersatz für das
frühere manuelle Script; CT 101 kann NFS nicht selbst serve, siehe GNAI-019).
- **Legacy-Medien-Cron entfernt**: Die root-Crontab-Einträge
(`chown -R root:root` / `chmod -R 0777` auf allen Medien-Mounts, alle 10
Minuten, installiert 2026-10-11) sind entfernt — Rolle `pve_media_cron`
(Betreiberentscheidung 30.10.2026; Backup unter
`/var/backups/ghostnet/pve/`). Der Cron hätte jedes echte
Berechtigungsmodell unter den Medien-Mounts — inklusive der
Encoding-Pipeline — alle 10 Minuten plattgewalzt.
- **LXC-Configs**: Entfernen der Ganesha-Ära-`lxc.seccomp.profile:`-Leerzeile
aus `/etc/pve/lxc/101.conf` — Rolle `pve_lxc_conf` (wirksam beim nächsten
CT-Start/Reboot).
@@ -6,4 +6,5 @@
roles:
- role: pve_lxc_conf
- role: pve_media_cron
- role: pve_nfs_export
@@ -144,6 +144,29 @@
/etc/pve/lxc/101.conf still carries the empty Ganesha-era
lxc.seccomp.profile line (applies at the next CT start).
- name: Read the root crontab
ansible.builtin.command:
argv:
- /usr/bin/crontab
- -l
- -u
- root
check_mode: false
changed_when: false
failed_when: false
register: pve_verify_crontab
- name: Require the legacy media permission cron to be absent
ansible.builtin.assert:
that: >-
pve_verify_crontab.rc != 0 or
(pve_verify_crontab.stdout |
regex_findall('(chown -R root:root|chmod -R 0777) /media/(daten1|daten2|daten3|datenhdd1|datenhdd3)')
| length == 0)
fail_msg: >-
The legacy recursive permission cron is active again; it flattens
the encoding pipeline ownership and modes every 10 minutes.
- name: Report the verified PVE host production state
ansible.builtin.debug:
msg:
@@ -0,0 +1,13 @@
# `pve_media_cron`
Removes the legacy root crontab entries (installed 2026-10-11) that ran
`chown -R root:root` and `chmod -R 0777` on all media mounts
(`/media/daten1`, `/media/daten2`, `/media/daten3`, `/media/datenhdd1`,
`/media/datenhdd3`) every 10 minutes. The entries predate the encoding
pipeline and silently flattened every real ownership and mode below the
mounts, including the pipeline export root and layout (GNENC-002 section 4).
The full root crontab is backed up once to
`/var/backups/ghostnet/pve/root-crontab-before-legacy-removal-20261030.txt`
before removal. Existing media files keep their current (root:root 0777)
state; the cron only stops re-enforcing it. Operator decision 2026-10-30.
@@ -0,0 +1,8 @@
---
# Legacy root crontab entries (installed 2026-10-11) flattened real ownership
# and mode on all media mounts every 10 minutes, which is incompatible with
# the encoding pipeline permission model (GNENC-002 section 4).
pve_media_cron_backup_directory: /var/backups/ghostnet/pve
pve_media_cron_backup_name: root-crontab-before-legacy-removal-20261030.txt
pve_media_cron_legacy_pattern: >-
(chown -R root:root|chmod -R 0777) /media/(daten1|daten2|daten3|datenhdd1|datenhdd3)
@@ -0,0 +1,66 @@
---
- name: Read the root crontab
ansible.builtin.command:
argv:
- /usr/bin/crontab
- -l
- -u
- root
check_mode: false
changed_when: false
failed_when: false
register: pve_media_crontab_probe
- name: Detect the legacy media permission cron
ansible.builtin.set_fact:
pve_media_cron_legacy_present: >-
{{ pve_media_crontab_probe.rc == 0 and
(pve_media_crontab_probe.stdout | regex_findall(pve_media_cron_legacy_pattern) | length > 0) }}
- name: Create the crontab backup directory
ansible.builtin.file:
path: "{{ pve_media_cron_backup_directory }}"
state: directory
owner: root
group: root
mode: "0700"
- name: Back up the root crontab before removing the legacy entries
ansible.builtin.copy:
content: "{{ pve_media_crontab_probe.stdout }}\n"
dest: "{{ pve_media_cron_backup_directory }}/{{ pve_media_cron_backup_name }}"
owner: root
group: root
mode: "0600"
when: pve_media_cron_legacy_present | bool
- name: Remove the legacy media permission lines from the root crontab
ansible.builtin.shell: |
set -euo pipefail
crontab -l -u root | grep -vE '{{ pve_media_cron_legacy_pattern }}' | crontab -u root -
args:
executable: /bin/bash
when: pve_media_cron_legacy_present | bool
- name: Require the legacy media permission cron to be gone
ansible.builtin.command:
argv:
- /usr/bin/crontab
- -l
- -u
- root
check_mode: false
changed_when: false
failed_when: false
register: pve_media_crontab_after
- name: Assert the legacy media permission cron is removed
ansible.builtin.assert:
that: >-
pve_media_crontab_after.rc != 0 or
(pve_media_crontab_after.stdout | regex_findall(pve_media_cron_legacy_pattern) | length == 0)
fail_msg: >-
The legacy recursive permission cron (chown -R root:root / chmod -R 0777
on the media mounts) is still active in the root crontab; it flattens the
encoding pipeline ownership and modes every 10 minutes (GNENC-002).
success_msg: The legacy media permission cron is removed.