97 lines
5.9 KiB
Markdown
97 lines
5.9 KiB
Markdown
## 🤖 GhostNet OpenClaw & Multi-Vendor Ollama Pipeline
|
|
A highly optimized, hardware-accelerated local AI infrastructure leveraging OpenClaw and Ollama running inside containerized isolation. Tailored specifically for Podman (rootless/SELinux) architectures on rolling-release host systems (like CachyOS / Arch Linux).
|
|
------------------------------
|
|
## 🚀 Quick Start
|
|
## 1. Environment Setup
|
|
Clone the repository and copy the environment template to create your local configurations:
|
|
|
|
cp .env.example .env
|
|
|
|
Open .env and configure your system-specific parameters (host paths, custom shared memory allocation, and target model listings).
|
|
## 2. Choose Your Vendor and Boot
|
|
Deploy using podman-compose (native python engine) to ensure hardware mapping instructions are parsed without schema drops:
|
|
|
|
# For AMD Radeon GPUs (Navi 31 / ROCm)
|
|
podman-compose -f compose.amd.yaml up -d --build
|
|
# For NVIDIA GeForce/RTX GPUs (CUDA / CDI)
|
|
podman-compose -f compose.nvidia.yaml up -d --build
|
|
# For Intel Arc / Integrated Xe GPUs (oneAPI / SYCL)
|
|
podman-compose -f compose.intel.yaml up -d --build
|
|
|
|
## 3. Open the Interactive Chat
|
|
Access your localized agent console with a clean, single-line command:
|
|
|
|
podman exec -it openclaw-agent openclaw chat
|
|
|
|
------------------------------
|
|
## 🛠 Technical Architecture & Key Highlights
|
|
This setup relies on unique architectural design patterns engineered to overcome container engines boundaries and system strictness:
|
|
|
|
```
|
|
+-------------------------------------------------------------+
|
|
| Host Hardware (GPU) |
|
|
+-------------------------------------------------------------+
|
|
^
|
|
| Passthrough (CDI / DRI / rwm)
|
|
v
|
|
+-------------------------------------------------------------+
|
|
| ollama (Container) |
|
|
+-------------------------------------------------------------+
|
|
^
|
|
| (Internal Bridge Net: Port 11434)
|
|
v
|
|
+-------------------------------------------------------------+
|
|
| openclaw-ollama-bridge (Sidecar) |
|
|
| - Tunnels 127.0.0.1:11434 directly to ollama:11434 |
|
|
+-------------------------------------------------------------+
|
|
^
|
|
| (Shared Network Namespace)
|
|
v
|
|
+-------------------------------------------------------------+
|
|
| openclaw-agent (Container) |
|
|
| - Runs 'openclaw gateway start' on the main thread |
|
|
| - Injects 'openclaw chat' (TUI) via podman exec |
|
|
+-------------------------------------------------------------+
|
|
|
|
```
|
|
|
|
## 🔒 The Loopback TUI Bypass (The Socat Sidecar)
|
|
|
|
* The Challenge: When invoking the embedded terminal chat user interface (openclaw chat) inside the agent container in local gateway mode, the internal runtime strictly forces inference connections to http://127.0.0.1:11434. It ignores the container's environment OLLAMA_URL variable entirely on this sub-level, causing connection failures to external container hooks.
|
|
* The Solution: An elegant Sidecar design using alpine/socat tied directly to the agent's network stack via network_mode: "service:agent". It spins up an offline-safe TCP tunnel inside the loopback adapter of the agent. When the TUI targets 127.0.0.1, socat seamlessly pipes the payload directly across the secure internal network bridge to the ollama container. Ports no longer need to be exposed to the host machine for production.
|
|
|
|
## 🍱 Monolithic Multi-Vendor Composability
|
|
|
|
* The Challenge: Merging secondary overlays (e.g., -f compose.yaml -f compose.amd.yaml) under podman-compose silently drops nested list arrays like devices: and group_add:. This causes Ollama to drop hardware acceleration without warning, triggering extreme system lockups during 35B model executions due to high CPU thread thrashing.
|
|
* The Solution: Merging everything into highly explicit, standalone files per GPU vendor (compose.amd.yaml, compose.nvidia.yaml, compose.intel.yaml). It decouples vendor configurations entirely and natively injects the target container base image (rocm vs latest) directly through structured Dockerfile ARG bindings.
|
|
|
|
## 🛡 Rootless Storage Mandates (keep-id & ,U)
|
|
|
|
* The Challenge: Running unprivileged Podman engines maps host namespaces heavily. Forcing static user: "${UID}:${GID}" parameters breaks OpenClaw because the container image relies on hardcoded path ownership tied exclusively to UID 1000 (node).
|
|
* The Solution: Leveraging userns_mode: "keep-id" to synchronize permission scopes directly with CachyOS desktop boundaries for configuration syncs. Concurrently, write-heavy workspaces employ the specialized :Z,U Podman storage annotation. This handles real-time user-id chowning in the background automatically, allowing host filesystem edits while maintaining container health.
|
|
|
|
------------------------------
|
|
## 📊 Infrastructure Verification & Operations## Inspect Engine Acceleration
|
|
Verify that Ollama successfully claimed the GPU and offloaded the model weights out of system RAM and completely into the VRAM stack:
|
|
|
|
podman exec -it ollama ollama ps
|
|
|
|
|
|
* Success Output: PROCESSOR column reads 100% GPU.
|
|
* Failure Output: PROCESSOR column reads 100% CPU (indicates mismatched driver mappings or kernel node locks).
|
|
|
|
## Check Active Memory Layer Allocations
|
|
Audit hardware initialization errors directly out of the runner log sequence:
|
|
|
|
podman logs ollama 2>&1 | grep -i -E "amdgpu|rocm|cuda|hip|layers"
|
|
|
|
|
|
* Look out for log signatures stating offloaded X/X layers to GPU to ensure prompt context windows stream back to OpenClaw at maximum token velocity.
|
|
|
|
## Diagnostics & Validation
|
|
|
|
# Shut down the environment cleanly without state deadlocks
|
|
podman-compose -f compose.<vendor>.yaml down
|
|
# Run system integrity and validation checks
|
|
podman exec -it openclaw-agent openclaw doctor
|