Complete rework to a working configuration.
This commit is contained in:
+23
-3
@@ -1,5 +1,25 @@
|
||||
# --- MODEL SELECTION ---
|
||||
MODEL_LIST=ministral-3:3b
|
||||
# ==========================================
|
||||
# OPENCLAW & OLLAMA CONFIGURATION ENVIRONMENT
|
||||
# ==========================================
|
||||
|
||||
# --- NETZWORK ---
|
||||
# --- LLM ENGINE & MODEL SELECTION ---
|
||||
MODEL_LIST=gemma4:e4b,qwen3.6:35b-a3b-q4_K_M
|
||||
OLLAMA_KEEP_ALIVE=-1
|
||||
|
||||
# --- HARDWARE RESOURCES ---
|
||||
# Shared Memory Size (e.g. 16gb for large AMD cards, 8gb for NVIDIA)
|
||||
SHM_SIZE=16gb
|
||||
|
||||
# --- SECURITY & AUTHENTICATION ---
|
||||
OPENCLAW_PASSWORD=mein-sicheres-passwort
|
||||
|
||||
# --- NETWORK ---
|
||||
DNS_SERVER=8.8.8.8
|
||||
|
||||
# --- HOST MOUNT VOLUMES ---
|
||||
# Local Path to Your openclaw.json
|
||||
HOST_OPENCLAW_JSON_PATH=/home/ghost/source/repos/ghostnet-openclaw/config/openclaw.json
|
||||
# Local Path for the AI Workspace (persistent storage for agent's work)
|
||||
HOST_WORKSPACE_PATH=./storage/workspace
|
||||
# Local Path for Additional Repositories or Code Directories for the Agent
|
||||
HOST_REPOS_PATH=/home/ghost/source/repos
|
||||
|
||||
@@ -0,0 +1,96 @@
|
||||
## 🤖 GhostNet OpenClaw & Multi-Vendor Ollama Pipeline
|
||||
A highly optimized, hardware-accelerated local AI infrastructure leveraging OpenClaw and Ollama running inside containerized isolation. Tailored specifically for Podman (rootless/SELinux) architectures on rolling-release host systems (like CachyOS / Arch Linux).
|
||||
------------------------------
|
||||
## 🚀 Quick Start
|
||||
## 1. Environment Setup
|
||||
Clone the repository and copy the environment template to create your local configurations:
|
||||
|
||||
cp .env.example .env
|
||||
|
||||
Open .env and configure your system-specific parameters (host paths, custom shared memory allocation, and target model listings).
|
||||
## 2. Choose Your Vendor and Boot
|
||||
Deploy using podman-compose (native python engine) to ensure hardware mapping instructions are parsed without schema drops:
|
||||
|
||||
# For AMD Radeon GPUs (Navi 31 / ROCm)
|
||||
podman-compose -f compose.amd.yaml up -d --build
|
||||
# For NVIDIA GeForce/RTX GPUs (CUDA / CDI)
|
||||
podman-compose -f compose.nvidia.yaml up -d --build
|
||||
# For Intel Arc / Integrated Xe GPUs (oneAPI / SYCL)
|
||||
podman-compose -f compose.intel.yaml up -d --build
|
||||
|
||||
## 3. Open the Interactive Chat
|
||||
Access your localized agent console with a clean, single-line command:
|
||||
|
||||
podman exec -it openclaw-agent openclaw chat
|
||||
|
||||
------------------------------
|
||||
## 🛠 Technical Architecture & Key Highlights
|
||||
This setup relies on unique architectural design patterns engineered to overcome container engines boundaries and system strictness:
|
||||
|
||||
```
|
||||
+-------------------------------------------------------------+
|
||||
| Host Hardware (GPU) |
|
||||
+-------------------------------------------------------------+
|
||||
^
|
||||
| Passthrough (CDI / DRI / rwm)
|
||||
v
|
||||
+-------------------------------------------------------------+
|
||||
| ollama (Container) |
|
||||
+-------------------------------------------------------------+
|
||||
^
|
||||
| (Internal Bridge Net: Port 11434)
|
||||
v
|
||||
+-------------------------------------------------------------+
|
||||
| openclaw-ollama-bridge (Sidecar) |
|
||||
| - Tunnels 127.0.0.1:11434 directly to ollama:11434 |
|
||||
+-------------------------------------------------------------+
|
||||
^
|
||||
| (Shared Network Namespace)
|
||||
v
|
||||
+-------------------------------------------------------------+
|
||||
| openclaw-agent (Container) |
|
||||
| - Runs 'openclaw gateway start' on the main thread |
|
||||
| - Injects 'openclaw chat' (TUI) via podman exec |
|
||||
+-------------------------------------------------------------+
|
||||
|
||||
```
|
||||
|
||||
## 🔒 The Loopback TUI Bypass (The Socat Sidecar)
|
||||
|
||||
* The Challenge: When invoking the embedded terminal chat user interface (openclaw chat) inside the agent container in local gateway mode, the internal runtime strictly forces inference connections to http://127.0.0.1:11434. It ignores the container's environment OLLAMA_URL variable entirely on this sub-level, causing connection failures to external container hooks.
|
||||
* The Solution: An elegant Sidecar design using alpine/socat tied directly to the agent's network stack via network_mode: "service:agent". It spins up an offline-safe TCP tunnel inside the loopback adapter of the agent. When the TUI targets 127.0.0.1, socat seamlessly pipes the payload directly across the secure internal network bridge to the ollama container. Ports no longer need to be exposed to the host machine for production.
|
||||
|
||||
## 🍱 Monolithic Multi-Vendor Composability
|
||||
|
||||
* The Challenge: Merging secondary overlays (e.g., -f compose.yaml -f compose.amd.yaml) under podman-compose silently drops nested list arrays like devices: and group_add:. This causes Ollama to drop hardware acceleration without warning, triggering extreme system lockups during 35B model executions due to high CPU thread thrashing.
|
||||
* The Solution: Merging everything into highly explicit, standalone files per GPU vendor (compose.amd.yaml, compose.nvidia.yaml, compose.intel.yaml). It decouples vendor configurations entirely and natively injects the target container base image (rocm vs latest) directly through structured Dockerfile ARG bindings.
|
||||
|
||||
## 🛡 Rootless Storage Mandates (keep-id & ,U)
|
||||
|
||||
* The Challenge: Running unprivileged Podman engines maps host namespaces heavily. Forcing static user: "${UID}:${GID}" parameters breaks OpenClaw because the container image relies on hardcoded path ownership tied exclusively to UID 1000 (node).
|
||||
* The Solution: Leveraging userns_mode: "keep-id" to synchronize permission scopes directly with CachyOS desktop boundaries for configuration syncs. Concurrently, write-heavy workspaces employ the specialized :Z,U Podman storage annotation. This handles real-time user-id chowning in the background automatically, allowing host filesystem edits while maintaining container health.
|
||||
|
||||
------------------------------
|
||||
## 📊 Infrastructure Verification & Operations## Inspect Engine Acceleration
|
||||
Verify that Ollama successfully claimed the GPU and offloaded the model weights out of system RAM and completely into the VRAM stack:
|
||||
|
||||
podman exec -it ollama ollama ps
|
||||
|
||||
|
||||
* Success Output: PROCESSOR column reads 100% GPU.
|
||||
* Failure Output: PROCESSOR column reads 100% CPU (indicates mismatched driver mappings or kernel node locks).
|
||||
|
||||
## Check Active Memory Layer Allocations
|
||||
Audit hardware initialization errors directly out of the runner log sequence:
|
||||
|
||||
podman logs ollama 2>&1 | grep -i -E "amdgpu|rocm|cuda|hip|layers"
|
||||
|
||||
|
||||
* Look out for log signatures stating offloaded X/X layers to GPU to ensure prompt context windows stream back to OpenClaw at maximum token velocity.
|
||||
|
||||
## Diagnostics & Validation
|
||||
|
||||
# Shut down the environment cleanly without state deadlocks
|
||||
podman-compose -f compose.<vendor>.yaml down
|
||||
# Run system integrity and validation checks
|
||||
podman exec -it openclaw-agent openclaw doctor
|
||||
+63
-5
@@ -1,11 +1,69 @@
|
||||
services:
|
||||
# --- LLM ENGINE ---
|
||||
ollama:
|
||||
shm_size: '8gb'
|
||||
build:
|
||||
context: ./ollama
|
||||
args:
|
||||
- BASE_IMAGE=ollama/ollama:rocm
|
||||
container_name: ollama
|
||||
restart: always
|
||||
shm_size: ${SHM_SIZE}
|
||||
privileged: true
|
||||
devices:
|
||||
- /dev/dri:/dev/dri
|
||||
- /dev/kfd:/dev/kfd
|
||||
- /dev/dri:/dev/dri:rwm
|
||||
- /dev/kfd:/dev/kfd:rwm
|
||||
group_add:
|
||||
- 44
|
||||
- 109
|
||||
rendersecurity_opt:
|
||||
- label=disable
|
||||
volumes:
|
||||
- ./storage/ollama_models:/root/.ollama
|
||||
environment:
|
||||
# For AMD (Navi 31): 11.0.0 | For APUs: 10.3.0
|
||||
- OLLAMA_HOST=0.0.0.0:11434
|
||||
- MODEL_LIST=${MODEL_LIST}
|
||||
- OLLAMA_KEEP_ALIVE=${OLLAMA_KEEP_ALIVE}
|
||||
- HSA_OVERRIDE_GFX_VERSION=11.0.0
|
||||
#- OLLAMA_VULKAN=1 # only activate on problems with default (rocm)
|
||||
- OLLAMA_FLASH_ATTENTION=1
|
||||
ports:
|
||||
- "11434:11434"
|
||||
dns:
|
||||
- ${DNS_SERVER}
|
||||
networks:
|
||||
- net.ghost.openclaw
|
||||
|
||||
# --- CORE AGENT (GATEWAY) ---
|
||||
agent:
|
||||
image: ghcr.io/openclaw/openclaw:latest
|
||||
container_name: openclaw-agent
|
||||
depends_on:
|
||||
- ollama
|
||||
userns_mode: "keep-id"
|
||||
environment:
|
||||
- OPENCLAW_GATEWAY_MODE=local
|
||||
- OPENCLAW_GATEWAY_PASSWORD=${OPENCLAW_PASSWORD}
|
||||
- OLLAMA_URL=http://127.0.0.1:11434
|
||||
- OLLAMA_API_KEY=ollama-local
|
||||
ports:
|
||||
- "8080:8080"
|
||||
volumes:
|
||||
- ${HOST_OPENCLAW_JSON_PATH}:/home/node/.openclaw/openclaw.json:Z
|
||||
- ${HOST_WORKSPACE_PATH}:/home/node/.openclaw/workspace:Z,U
|
||||
- ${HOST_REPOS_PATH}:/repos:Z,U
|
||||
networks:
|
||||
- net.ghost.openclaw
|
||||
|
||||
# --- NETWORK TUNNEL (SIDECAR) ---
|
||||
ollama-bridge:
|
||||
image: docker.io/alpine/socat:latest
|
||||
container_name: openclaw-ollama-bridge
|
||||
depends_on:
|
||||
- agent
|
||||
network_mode: "service:agent"
|
||||
command: TCP-LISTEN:11434,fork TCP:ollama:11434
|
||||
stop_grace_period: 1s
|
||||
restart: always
|
||||
|
||||
networks:
|
||||
net.ghost.openclaw:
|
||||
driver: bridge
|
||||
|
||||
@@ -0,0 +1,64 @@
|
||||
services:
|
||||
# --- LLM ENGINE ---
|
||||
ollama:
|
||||
build:
|
||||
context: ./ollama
|
||||
args:
|
||||
- BASE_IMAGE=ollama/ollama:rocm
|
||||
container_name: ollama
|
||||
restart: always
|
||||
shm_size: ${SHM_SIZE}
|
||||
privileged: true
|
||||
devices:
|
||||
- /dev/dri:/dev/dri:rwm
|
||||
group_add:
|
||||
- 109
|
||||
volumes:
|
||||
- ./storage/ollama_models:/root/.ollama
|
||||
environment:
|
||||
- OLLAMA_HOST=0.0.0.0:11434
|
||||
- MODEL_LIST=${MODEL_LIST}
|
||||
- OLLAMA_KEEP_ALIVE=${OLLAMA_KEEP_ALIVE}
|
||||
- OLLAMA_FLASH_ATTENTION=1
|
||||
ports:
|
||||
- "11434:11434"
|
||||
dns:
|
||||
- ${DNS_SERVER}
|
||||
networks:
|
||||
- net.ghost.openclaw
|
||||
|
||||
# --- CORE AGENT (GATEWAY) ---
|
||||
agent:
|
||||
image: ghcr.io/openclaw/openclaw:latest
|
||||
container_name: openclaw-agent
|
||||
depends_on:
|
||||
- ollama
|
||||
userns_mode: "keep-id"
|
||||
environment:
|
||||
- OPENCLAW_GATEWAY_MODE=local
|
||||
- OPENCLAW_GATEWAY_PASSWORD=${OPENCLAW_PASSWORD}
|
||||
- OLLAMA_URL=http://127.0.0.1:11434
|
||||
- OLLAMA_API_KEY=ollama-local
|
||||
ports:
|
||||
- "8080:8080"
|
||||
volumes:
|
||||
- ${HOST_OPENCLAW_JSON_PATH}:/home/node/.openclaw/openclaw.json:Z
|
||||
- ${HOST_WORKSPACE_PATH}:/home/node/.openclaw/workspace:Z,U
|
||||
- ${HOST_REPOS_PATH}:/repos:Z,U
|
||||
networks:
|
||||
- net.ghost.openclaw
|
||||
|
||||
# --- NETWORK TUNNEL (SIDECAR) ---
|
||||
ollama-bridge:
|
||||
image: docker.io/alpine/socat:latest
|
||||
container_name: openclaw-ollama-bridge
|
||||
depends_on:
|
||||
- agent
|
||||
network_mode: "service:agent"
|
||||
command: TCP-LISTEN:11434,fork TCP:ollama:11434
|
||||
stop_grace_period: 1s
|
||||
restart: always
|
||||
|
||||
networks:
|
||||
net.ghost.openclaw:
|
||||
driver: bridge
|
||||
+58
-8
@@ -1,12 +1,62 @@
|
||||
services:
|
||||
# --- LLM ENGINE ---
|
||||
ollama:
|
||||
shm_size: '8gb'
|
||||
deploy:
|
||||
resources:
|
||||
reservations:
|
||||
devices:
|
||||
- driver: nvidia
|
||||
count: 1
|
||||
capabilities: [gpu]
|
||||
build:
|
||||
context: ./ollama
|
||||
args:
|
||||
- BASE_IMAGE=ollama/ollama:latest
|
||||
container_name: ollama
|
||||
restart: always
|
||||
shm_size: ${SHM_SIZE}
|
||||
devices:
|
||||
- ://nvidia.com
|
||||
volumes:
|
||||
- ./storage/ollama_models:/root/.ollama
|
||||
environment:
|
||||
- OLLAMA_HOST=0.0.0.0:11434
|
||||
- MODEL_LIST=${MODEL_LIST}
|
||||
- OLLAMA_KEEP_ALIVE=${OLLAMA_KEEP_ALIVE}
|
||||
- OLLAMA_FLASH_ATTENTION=1
|
||||
- NVIDIA_VISIBLE_DEVICES=all
|
||||
ports:
|
||||
- "11434:11434"
|
||||
dns:
|
||||
- ${DNS_SERVER}
|
||||
networks:
|
||||
- net.ghost.openclaw
|
||||
|
||||
# --- CORE AGENT (GATEWAY) ---
|
||||
agent:
|
||||
image: ghcr.io/openclaw/openclaw:latest
|
||||
container_name: openclaw-agent
|
||||
depends_on:
|
||||
- ollama
|
||||
userns_mode: "keep-id"
|
||||
environment:
|
||||
- OPENCLAW_GATEWAY_MODE=local
|
||||
- OPENCLAW_GATEWAY_PASSWORD=${OPENCLAW_PASSWORD}
|
||||
- OLLAMA_URL=http://127.0.0.1:11434
|
||||
- OLLAMA_API_KEY=ollama-local
|
||||
ports:
|
||||
- "8080:8080"
|
||||
volumes:
|
||||
- ${HOST_OPENCLAW_JSON_PATH}:/home/node/.openclaw/openclaw.json:Z
|
||||
- ${HOST_WORKSPACE_PATH}:/home/node/.openclaw/workspace:Z,U
|
||||
- ${HOST_REPOS_PATH}:/repos:Z,U
|
||||
networks:
|
||||
- net.ghost.openclaw
|
||||
|
||||
# --- NETWORK TUNNEL (SIDECAR) ---
|
||||
ollama-bridge:
|
||||
image: docker.io/alpine/socat:latest
|
||||
container_name: openclaw-ollama-bridge
|
||||
depends_on:
|
||||
- agent
|
||||
network_mode: "service:agent"
|
||||
command: TCP-LISTEN:11434,fork TCP:ollama:11434
|
||||
stop_grace_period: 1s
|
||||
restart: always
|
||||
|
||||
networks:
|
||||
net.ghost.openclaw:
|
||||
driver: bridge
|
||||
|
||||
@@ -1,49 +0,0 @@
|
||||
services:
|
||||
# --- LLM ENGINE ---
|
||||
ollama:
|
||||
build: ./providers/llm/ollama
|
||||
container_name: ollama
|
||||
restart: always
|
||||
volumes:
|
||||
- ./storage/ollama_models:/root/.ollama
|
||||
environment:
|
||||
- MODEL_LIST=${MODEL_LIST:-ministral-3:3b}
|
||||
ports:
|
||||
- "11434:11434" # only required for external access
|
||||
dns:
|
||||
- ${DNS_SERVER:-1.1.1.1}
|
||||
networks:
|
||||
- net.ghost.openclaw
|
||||
|
||||
# --- SEARCH ENGINE ---
|
||||
#searxng:
|
||||
# image: docker.io/searxng/searxng:latest
|
||||
# container_name: searxng
|
||||
# restart: always
|
||||
# volumes:
|
||||
# - ./providers/search/engine/settings.yml:/etc/searxng/settings.yml:ro
|
||||
# networks:
|
||||
# - net.ghost.openclaw
|
||||
|
||||
# --- CORE AGENT ---
|
||||
#agent:
|
||||
# build:
|
||||
# context: .
|
||||
# dockerfile: core/agent/docker/Containerfile
|
||||
# container_name: openclaw-agent
|
||||
# depends_on:
|
||||
# - ollama
|
||||
# - searxng
|
||||
# environment:
|
||||
# - OLLAMA_HOST=http://ollama:11434
|
||||
# - SEARXNG_URL=http://searxng:8080
|
||||
# - MCP_SEARXNG_PATH=/app/mcp-bridge/dist/index.js
|
||||
# volumes:
|
||||
# - ./storage/workspace:/app/workspace:rw
|
||||
# - ./storage/knowledge:/app/knowledge:ro
|
||||
# networks:
|
||||
# - net.ghost.openclaw
|
||||
|
||||
networks:
|
||||
net.ghost.openclaw:
|
||||
driver: bridge
|
||||
@@ -0,0 +1,15 @@
|
||||
{
|
||||
"gateway": {
|
||||
"mode": "local",
|
||||
"bind": "lan",
|
||||
"port": 8080,
|
||||
"auth": {
|
||||
"mode": "password"
|
||||
}
|
||||
},
|
||||
"agents": {
|
||||
"defaults": {
|
||||
"model": "ollama/gemma4:e4b"
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,4 +1,5 @@
|
||||
FROM docker.io/ollama/ollama:latest
|
||||
ARG BASE_IMAGE=ollama/ollama:rocm
|
||||
FROM docker.io/${BASE_IMAGE}
|
||||
COPY entrypoint.sh /entrypoint.sh
|
||||
RUN chmod +x /entrypoint.sh
|
||||
ENTRYPOINT ["/entrypoint.sh"]
|
||||
@@ -1,7 +1,8 @@
|
||||
#!/bin/bash
|
||||
|
||||
# --- FUNKTION: Signal-Handling ---
|
||||
# Diese Funktion wird aufgerufen, wenn der Container gestoppt werden soll (SIGTERM/SIGINT)
|
||||
# Diese Funktion wird aufgerufen, wenn der Container gestoppt werden soll
|
||||
# (SIGTERM/SIGINT)
|
||||
cleanup() {
|
||||
echo "--- Abbruchsignal empfangen! Beende Ollama sauber... ---"
|
||||
kill "$OLLAMA_PID" 2>/dev/null
|
||||
@@ -17,43 +18,39 @@ echo "--- Starte Ollama Server im Hintergrund ---"
|
||||
ollama serve &
|
||||
OLLAMA_PID=$!
|
||||
|
||||
# --- SCHRITT 2: Health-Check (Warten bis der Server bereit ist) ---
|
||||
echo "--- Warte auf Ollama Server (TCP Check) ---"
|
||||
# --- SCHRITT 2: Robuster API-Health-Check ---
|
||||
echo "--- Warte auf Ollama Server (API Check) ---"
|
||||
MAX_RETRIES=30
|
||||
COUNT=0
|
||||
# Wir nutzen den Bash-eigenen TCP-Check (benötigt kein curl!)
|
||||
until (exec 3<>/dev/tcp/localhost/11434) &>/dev/null || [ $COUNT -eq $MAX_RETRIES ]; do
|
||||
|
||||
# Wir nutzen ollama list selbst als Healthcheck, das ist 100% verlässlich
|
||||
until ollama list &>/dev/null || [ $COUNT -eq $MAX_RETRIES ]; do
|
||||
sleep 2
|
||||
COUNT=$((COUNT + 1))
|
||||
echo "Warte auf Server... ($COUNT/$MAX_RETRIES)"
|
||||
echo "Warte auf API-Bereitschaft... ($COUNT/$MAX_RETRIES)"
|
||||
done
|
||||
|
||||
# Schließe den Test-Socket sauber
|
||||
exec 3>&- 2>/dev/null || true
|
||||
|
||||
if [ $COUNT -eq $MAX_RETRIES ]; then
|
||||
echo "FEHLER: Ollama Server ist nach 60 Sekunden nicht bereit."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "--- Ollama Server ist online! ---"
|
||||
echo "--- Ollama Server ist online und bereit für Pulls! ---"
|
||||
|
||||
# --- SCHRITT 3: Modelle herunterladen ---
|
||||
# --- SCHRITT 3: Modelle fehlerfrei herunterladen ---
|
||||
# Wir nutzen ein alternatives Parsing, das immun gegen komplexe Strings ist
|
||||
if [ -z "$MODEL_LIST" ]; then
|
||||
echo "WARNUNG: Keine Modelle in MODEL_LIST definiert."
|
||||
else
|
||||
IFS=',' read -ra MODELS <<< "$MODEL_LIST"
|
||||
echo "--- Modell-Check gestartet ---"
|
||||
|
||||
for model in "${MODELS[@]}"; do
|
||||
model=$(echo "$model" | xargs) # Leerzeichen entfernen
|
||||
|
||||
# Ersetzt Kommas durch Leerzeichen und läuft sauber durch jedes Modell
|
||||
for model in $(echo "$MODEL_LIST" | tr ',' ' '); do
|
||||
echo "Prüfe Modell: $model"
|
||||
# Pull ausführen. Wenn es schon da ist, geht das extrem schnell.
|
||||
ollama pull "$model"
|
||||
done
|
||||
echo "--- Alle Modelle sind bereit ---"
|
||||
fi
|
||||
|
||||
# --- SCHRITT 4: Hauptprozess halten ---
|
||||
# Wir warten auf den Ollama-Prozess. Wenn er stirbt, endet auch das Skript.
|
||||
wait $OLLAMA_PID
|
||||
@@ -1,13 +0,0 @@
|
||||
#!/bin/bash
|
||||
# run.sh
|
||||
|
||||
if [ "$1" == "amd" ]; then
|
||||
podman-compose -f compose.yaml -f compose.amd.yaml up -d --build
|
||||
elif [ "$1" == "nvidia" ]; then
|
||||
podman-compose -f compose.yaml -f compose.nvidia.yaml up -d --build
|
||||
elif [ "$1" == "cpu" ]; then
|
||||
podman-compose up -d --build
|
||||
else
|
||||
echo "Please select Hardware: ./run.sh [amd|nvidia|cpu]"
|
||||
podman-compose up -d --build
|
||||
fi
|
||||
Reference in New Issue
Block a user