Add: Working microservice tool infrastructure.

This commit is contained in:
2026-06-03 17:44:51 +02:00
parent be6f193f03
commit d6df397c17
9 changed files with 435 additions and 53 deletions
+2
View File
@@ -7,6 +7,8 @@ MODEL_LIST=gemma4:e4b,qwen3.6:35b-a3b-q4_K_M
OLLAMA_KEEP_ALIVE=-1
# --- HARDWARE RESOURCES ---
# GPU Vendor (e.g. "amd", "nvidia", "intel")
GPU_TYPE=amd
# Shared Memory Size (e.g. 16gb for large AMD cards, 8gb for NVIDIA)
SHM_SIZE=16gb
+51
View File
@@ -0,0 +1,51 @@
# Stand: 01.06.2026
## 📋 Handover & Context Prompt for the Next AI Agent
Context:
We are developing a fully local, containerized AI agent workspace called ghostnet-openclaw on CachyOS (Arch Linux). The stack runs entirely inside unprivileged, rootless Podman containers using a modular, multi-vendor GPU setup managed via podman-compose (the native Python parser).
Current Architecture Stack:
1. LLM Engine (ollama): Running on the official ollama/ollama:rocm base image (built via custom Containerfile and an advanced entrypoint.sh boot-script that auto-polls and pulls models defined in MODEL_LIST). It features successful full AMD ROCm GPU Passthrough (Navi 31) with privileged: true and explicit host mapping for /dev/dri, /dev/kfd, and system render groups (44, 109).
2. Core Agent (openclaw-agent): Running OpenClaw v2026.5.28 in local gateway mode with password authentication. It utilizes userns_mode: "keep-id" to synchronize config file edits with the host alongside :Z,U volume flags to allow rootless Node.js writing to local host workspaces without permission degradation.
3. Network Tunnel Sidecar (openclaw-ollama-bridge): Running an offline-ready alpine/socat:latest image mapping directly into the network namespace of the agent (network_mode: "service:agent"). This tunnels local 127.0.0.1:11434 calls directly to ollama:11434 inside the closed bridge, bypassing a strict OpenClaw TUI bug where the embedded chat utility ignores global container environment variables.
Current Repository State:
* Fully dynamic variables outsourced to a root .env file (supporting SHM_SIZE, MODEL_LIST, OPENCLAW_PASSWORD, DNS servers, and absolute host system paths).
* Split monolithic configurations for various GPU vendors ready to go (compose.amd.yaml, compose.nvidia.yaml, compose.intel.yaml).
* Successfully tested and running gemma4:e4b at 100% GPU (VRAM) streaming tokens fluidly inside the native chat terminal without system freezes or context window pollution.
* Clean operational cycle verified: podman-compose -f compose.amd.yaml up -d --build runs without faults, and podman exec -it openclaw-agent openclaw chat opens the TUI flawlessly.
Next Task / Where to Continue:
The core infrastructure is now solid, secure, and fully GPU-accelerated. The user wants to expand the project from here.
Please analyze this setup and offer actionable next steps, such as:
1. Integrating external search indexers (like a local SearXNG service into the networks block).
2. Setting up explicit messenger interfaces or multi-account policies within the openclaw.json channels structure.
3. Building automated helper utilities (like a start.sh wrapper script to let users dynamically choose their vendor profile upon booting).
# Stand: 03.06.2026
## 📋 Handover & Context Prompt for the Next AI Agent
**Context:**
Wir entwickeln eine lokale, containerisierte AI-Workspace-Infrastruktur namens **ghostnet-openclaw** auf CachyOS. Der Stack nutzt unprivilegierte Podman-Container und eine Multi-Vendor GPU-Strategie.
**Current Architecture Stack:**
1. **LLM Engine (ollama):** Läuft auf `ollama/ollama:rocm`. Voller GPU-Passthrough für AMD Radeon RX 7900 XTX (Navi 31). Ein Entrypoint-Script automatisiert das Pulling der Modelle via `MODEL_LIST`.
2. **Core Agent (openclaw-agent):** OpenClaw v2026.5.28 im Local Gateway Mode. Nutzt `keep-id` und `:Z,U` für persistente Host-Workspaces ohne Permission-Issues.
3. **Socat Bridge:** Tunnelt `127.0.0.1:11434` (Agent) zu `ollama:11434` (Engine), um den TUI-Loopback-Bug zu umgehen.
4. **MCP Infrastructure (The Router):** Ein dedizierter `ghostnet-router` (Python/FastAPI) dient als zentrales Tool-Gateway. Er integriert Skills via Model Context Protocol (MCP).
5. **Search Skill:** Ein `tool-searchfetch` Container (Bridge), der über den Router Suchanfragen an einen lokalen `searxng` Dienst delegiert.
**Current Repository State:**
* **Networking:** Alle Services kommunizieren über ein isoliertes Bridge-Netzwerk. DNS-Auflösung für externe Registeries (Ollama) muss im Podman-Netzwerk stabilisiert werden.
* **Stability:** Die `maintain_connection`-Logik im Router wurde erfolgreich implementiert; der `CancelledError` bei SSE-Handshakes ist behoben.
* **Verification:** End-to-End Test (Client -> Router -> Search-Bridge -> SearXNG) war erfolgreich. Tools wie `searxng_web_search` werden korrekt propagiert.
**Next Tasks / Where to Continue:**
1. **OpenClaw Integration:** Die `openclaw.json` muss so konfiguriert werden, dass der Agent den `ghostnet-router` (SSE-Endpunkt) als primäre Tool-Quelle nutzt.
2. **Modell-Management:** DNS-Problematik in der Ollama-Container-Registry beheben (Registry-Auflösung schlägt aktuell fehl).
3. **Skill Expansion:** Entwicklung weiterer MCP-Bridges (z.B. für lokales Filesystem-Management oder Datenbank-Abfragen), die einfach am Router "angepluggt" werden können.
4. **UX-Automation:** Ein `ghostnet.sh` Wrapper, der die Vendor-Erkennung automatisiert und den Stack konsistent hochfährt.
+54 -52
View File
@@ -1,30 +1,33 @@
## 🤖 GhostNet OpenClaw & Multi-Vendor Ollama Pipeline
A highly optimized, hardware-accelerated local AI infrastructure leveraging OpenClaw and Ollama running inside containerized isolation. Tailored specifically for Podman (rootless/SELinux) architectures on rolling-release host systems (like CachyOS / Arch Linux).
------------------------------
## 🚀 Quick Start
## 1. Environment Setup
Clone the repository and copy the environment template to create your local configurations:
cp .env.example .env
Open .env and configure your system-specific parameters (host paths, custom shared memory allocation, and target model listings).
## 2. Choose Your Vendor and Boot
Deploy using podman-compose (native python engine) to ensure hardware mapping instructions are parsed without schema drops:
# For AMD Radeon GPUs (Navi 31 / ROCm)
podman-compose -f compose.amd.yaml up -d --build
# For NVIDIA GeForce/RTX GPUs (CUDA / CDI)
podman-compose -f compose.nvidia.yaml up -d --build
# For Intel Arc / Integrated Xe GPUs (oneAPI / SYCL)
podman-compose -f compose.intel.yaml up -d --build
## 3. Open the Interactive Chat
Access your localized agent console with a clean, single-line command:
podman exec -it openclaw-agent openclaw chat
A highly optimized, hardware-accelerated local AI infrastructure leveraging OpenClaw, Ollama, and a modular MCP (Model Context Protocol) gateway running inside containerized isolation. Tailored specifically for Podman (rootless/SELinux) architectures on rolling-release host systems (like CachyOS / Arch Linux).
------------------------------
## 🛠 Technical Architecture & Key Highlights
## Quick Start
### 1. Environment Setup
Clone the repository and prepare your environment configuration:
`cp .env.example .env`
Open `.env` and set your `GPU_TYPE` (amd, nvidia, or intel) and your desired models.
### 2. Boot the Stack
Use the integrated management script, which automatically selects the correct vendor-specific compose file:
```bash
# Start the stack in the background
./ghostnet.sh up
```
```bash
# For debugging: Stop everything and start in the foreground (Attached Mode)
./ghostnet.sh attached
```
### 3. Open the Interactive Chat
`podman exec -it openclaw-agent openclaw chat`
## Technical Architecture & Key Highlights
This setup relies on unique architectural design patterns engineered to overcome container engines boundaries and system strictness:
```
@@ -55,42 +58,41 @@ This setup relies on unique architectural design patterns engineered to overcome
```
## 🔒 The Loopback TUI Bypass (The Socat Sidecar)
## MCP Microservice Gateway (The GhostNet Router)
* **The Challenge:** OpenClaw needs to scale its capabilities (Search, Filesystem, DB) without bloating the main agent container or creating dependency hell.
* **The Solution:** A centralized **Python-based MCP Router**. It acts as a single API Gateway that aggregates multiple "Skills" (Bridges).
* **Key Feature:** The Router maintains persistent, asynchronous SSE connections to sub-services (like `searchfetch`) using a robust `maintain_connection` logic, ensuring the agent always has access to live tools without manual re-initialization.
* The Challenge: When invoking the embedded terminal chat user interface (openclaw chat) inside the agent container in local gateway mode, the internal runtime strictly forces inference connections to http://127.0.0.1:11434. It ignores the container's environment OLLAMA_URL variable entirely on this sub-level, causing connection failures to external container hooks.
* The Solution: An elegant Sidecar design using alpine/socat tied directly to the agent's network stack via network_mode: "service:agent". It spins up an offline-safe TCP tunnel inside the loopback adapter of the agent. When the TUI targets 127.0.0.1, socat seamlessly pipes the payload directly across the secure internal network bridge to the ollama container. Ports no longer need to be exposed to the host machine for production.
## The Loopback TUI Bypass (The Socat Sidecar)
* **The Challenge:** The OpenClaw TUI strictly forces connections to `127.0.0.1:11434`, ignoring environment variables.
* **The Solution:** A Socat Sidecar (`network_mode: "service:agent"`) that tunnels local loopback traffic directly to the isolated Ollama container.
## 🍱 Monolithic Multi-Vendor Composability
## Monolithic Multi-Vendor Composability
* Standalone compose files (`compose.amd.yaml` etc.) prevent the "Schema-Drop" bug of `podman-compose`, ensuring that `devices:` and `group_add:` mappings for ROCm/CUDA are never silently discarded.
* The Challenge: Merging secondary overlays (e.g., -f compose.yaml -f compose.amd.yaml) under podman-compose silently drops nested list arrays like devices: and group_add:. This causes Ollama to drop hardware acceleration without warning, triggering extreme system lockups during 35B model executions due to high CPU thread thrashing.
* The Solution: Merging everything into highly explicit, standalone files per GPU vendor (compose.amd.yaml, compose.nvidia.yaml, compose.intel.yaml). It decouples vendor configurations entirely and natively injects the target container base image (rocm vs latest) directly through structured Dockerfile ARG bindings.
## Rootless Storage Mandates (keep-id & ,U)
* Synchronization of host/container UIDs via `userns_mode: "keep-id"` combined with `:Z,U` flags. This allows the Node.js agent to write to host-mounted workspaces while maintaining strict SELinux compliance.
## 🛡 Rootless Storage Mandates (keep-id & ,U)
## Infrastructure Verification
* The Challenge: Running unprivileged Podman engines maps host namespaces heavily. Forcing static user: "${UID}:${GID}" parameters breaks OpenClaw because the container image relies on hardcoded path ownership tied exclusively to UID 1000 (node).
* The Solution: Leveraging userns_mode: "keep-id" to synchronize permission scopes directly with CachyOS desktop boundaries for configuration syncs. Concurrently, write-heavy workspaces employ the specialized :Z,U Podman storage annotation. This handles real-time user-id chowning in the background automatically, allowing host filesystem edits while maintaining container health.
```bash
# Inspect Engine Acceleration
# Success: PROCESSOR reads 100% GPU.
`podman exec -it ollama ollama ps`
```
------------------------------
## 📊 Infrastructure Verification & Operations## Inspect Engine Acceleration
Verify that Ollama successfully claimed the GPU and offloaded the model weights out of system RAM and completely into the VRAM stack:
podman exec -it ollama ollama ps
* Success Output: PROCESSOR column reads 100% GPU.
* Failure Output: PROCESSOR column reads 100% CPU (indicates mismatched driver mappings or kernel node locks).
## Check Active Memory Layer Allocations
Audit hardware initialization errors directly out of the runner log sequence:
podman logs ollama 2>&1 | grep -i -E "amdgpu|rocm|cuda|hip|layers"
* Look out for log signatures stating offloaded X/X layers to GPU to ensure prompt context windows stream back to OpenClaw at maximum token velocity.
## Diagnostics & Validation
```bash
# Check MCP Router Integrity
# Verify that the Gateway is alive and the Search-Bridge is successfully integrated.
curl -N http://tool-router:3000/sse
```
```bash
# Shut down the environment cleanly without state deadlocks
podman-compose -f compose.<vendor>.yaml down
```
```bash
# Run system integrity and validation checks
podman exec -it openclaw-agent openclaw doctor
```
+88 -1
View File
@@ -52,7 +52,7 @@ services:
- ${HOST_REPOS_PATH}:/repos:Z,U
networks:
- net.ghost.openclaw
# --- NETWORK TUNNEL (SIDECAR) ---
ollama-bridge:
image: docker.io/alpine/socat:latest
@@ -64,6 +64,93 @@ services:
stop_grace_period: 1s
restart: always
# --- TOOL ROUTER (GATEWAY) ---
tool-router:
image: docker.io/python:3.12-slim
container_name: tool-router
depends_on:
- tool-searchfetch
init: true
working_dir: /app
environment:
- PORT=3000
command: >
sh -c "pip install --no-cache-dir fastapi uvicorn 'modelcontextprotocol[python-sdk]' &&
uvicorn router:app --host 0.0.0.0 --port 3000"
volumes:
- ./tools/router/src/router.py:/app/router.py:ro,Z
- ./tools/router/config:/app/config:ro,Z
expose:
- "3000"
dns:
- ${DNS_SERVER}
networks:
- net.ghost.openclaw
- net.ghost.tools
# --- SEARCHFETCH TOOL ---
tool-searchfetch:
image: docker.io/nikolaik/python-nodejs:python3.14-nodejs26-slim
container_name: tool-searchfetch
depends_on:
- tool-searchfetch-searxng
networks:
- net.ghost.tools
environment:
- SEARXNG_URL=http://tool-searchfetch-searxng:8080
- MCP_HOST=0.0.0.0
- MCP_PORT=3000
command: >
sh -c "pip install --no-cache-dir mcp-proxy &&
npm install -g mcp-searxng &&
mcp-proxy --host $$MCP_HOST --port $$MCP_PORT --pass-environment -- mcp-searxng"
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:$$MCP_PORT/sse"]
interval: 30s
timeout: 10s
retries: 3
# --- SEARXNG SERVICE FOR SEARCHFETCH TOOL ---
tool-searchfetch-searxng:
image: docker.io/searxng/searxng:latest
container_name: tool-searchfetch-searxng
depends_on:
- tool-searchfetch-valkey
restart: always
volumes:
- ./tools/searchfetch/config/searxng.settings.yml:/etc/searxng/settings.yml:ro,Z
- ./storage/searxng:/tmp:Z
environment:
- SEARXNG_SETTINGS_PATH=/etc/searxng/settings.yml
- SEARXNG_VALKEY_URL=redis://tool-searchfetch-valkey:6379/0
expose:
- "8080"
dns:
- ${DNS_SERVER}
networks:
- net.ghost.tools
tool-searchfetch-valkey:
image: docker.io/valkey/valkey:8-alpine
container_name: tool-searchfetch-valkey
restart: always
volumes:
- ./storage/searxng_valkey:/data:U,Z
networks:
- net.ghost.tools
tool-test:
image: docker.io/nikolaik/python-nodejs:python3.14-nodejs26-slim
container_name: tool-test
command: ["sh", "-c", "while true; do sleep 1000; done"]
dns:
- ${DNS_SERVER}
networks:
- net.ghost.tools
networks:
net.ghost.openclaw:
driver: bridge
internal: true
net.ghost.tools:
driver: bridge
+8
View File
@@ -11,5 +11,13 @@
"defaults": {
"model": "ollama/gemma4:e4b"
}
},
"mcp": {
"servers": {
"router": {
"url": "http://mcp-router:3000/sse",
"transport": "sse"
}
}
}
}
Executable
+67
View File
@@ -0,0 +1,67 @@
#!/bin/bash
# Path to the .env file
ENV_FILE="./.env"
# 1. Determine GPU_TYPE: Shell-Env first, then .env file
if [ -z "$GPU_TYPE" ]; then
if [ -f "$ENV_FILE" ]; then
GPU_TYPE=$(grep '^GPU_TYPE=' "$ENV_FILE" | cut -d '=' -f2)
fi
fi
# 2. Validation: Is the variable now set?
if [ -z "$GPU_TYPE" ]; then
echo "ERROR: 'GPU_TYPE' is not defined!"
echo "💡 Please set it in your shell (export GPU_TYPE=amd) or create an .env file."
exit 1
fi
# 3. Validation: Is the value allowed?
case "$GPU_TYPE" in
amd|intel|nvidia)
COMPOSE_FILE="compose.$GPU_TYPE.yaml"
;;
*)
echo "ERROR: Invalid GPU_TYPE '$GPU_TYPE'!"
echo "Allowed values are: amd, intel, nvidia"
exit 1
esac
# 4. Check if YAML file exists
if [ ! -f "$COMPOSE_FILE" ]; then
echo "ERROR: The file '$COMPOSE_FILE' was not found!"
exit 1
fi
# 5. Execute the command based on user input
COMMAND=$1
case "$COMMAND" in
up)
echo "🚀 Starting Ghostnet ($GPU_TYPE)..."
podman-compose -f "$COMPOSE_FILE" up -d
;;
down)
echo "🛑 Stopping Ghostnet ($GPU_TYPE)..."
podman-compose -f "$COMPOSE_FILE" down
;;
restart)
echo "🔄 Restarting Ghostnet ($GPU_TYPE)..."
podman-compose -f "$COMPOSE_FILE" restart
;;
logs)
podman-compose -f "$COMPOSE_FILE" logs -f
;;
attached)
echo "🔍 Checking for running instances..."
podman-compose -f "$COMPOSE_FILE" down > /dev/null 2>&1
echo "🔍 Starting Ghostnet in foreground ($GPU_TYPE)..."
podman-compose -f "$COMPOSE_FILE" up
;;
*)
echo "Usage: ./ghostnet.sh [up|down|restart|logs|attached]"
echo "Current environment: $GPU_TYPE"
exit 1
;;
esac
+7
View File
@@ -0,0 +1,7 @@
{
"mcpServers": {
"searchfetch": {
"url": "http://tool-searchfetch:3000/sse"
}
}
}
+135
View File
@@ -0,0 +1,135 @@
import asyncio
import json
import logging
import os
import sys
from contextlib import asynccontextmanager
from fastapi import FastAPI, Request
from mcp import ClientSession
from mcp.client.sse import sse_client
from mcp.server import Server
from mcp.server.sse import SseServerTransport
import mcp.types as types
# --- LOGGING ---
log_level_str = os.getenv("LOG_LEVEL", "INFO").upper()
log_level = getattr(logging, log_level_str, logging.INFO)
logging.basicConfig(
level=log_level,
format="%(asctime)s - %(name)s - %(levelname)s - %(message)s",
handlers=[logging.StreamHandler(sys.stdout)]
)
logger = logging.getLogger("ghostnet-router")
# --- GLOBAL STATE ---
# Hier speichern wir die aktiven Sessions zu den Microservices (z.B. searchfetch)
skill_sessions: dict[str, ClientSession] = {}
master_server = Server("GhostNet-Router")
# --- LIFESPAN (Connection Management) ---
@asynccontextmanager
async def lifespan(app: FastAPI):
"""Verwaltet den Lebenszyklus der Verbindungen zu den Microservices."""
config_path = "/app/config/gateway.json"
try:
with open(config_path, "r") as f:
config = json.load(f)
except Exception as e:
logger.error(f"❌ Failed to load config from {config_path}: {e}")
config = {"mcpServers": {}}
# Hintergrund-Tasks für jeden konfigurierten Service starten
tasks = []
for name, info in config.get("mcpServers", {}).items():
url = info.get("url")
if url:
tasks.append(asyncio.create_task(maintain_connection(name, url)))
yield # Der Router ist jetzt bereit und bedient Anfragen
# Cleanup beim Herunterfahren
logger.info("Closing all skill connections...")
for task in tasks:
task.cancel()
logger.info("Router shutdown complete.")
async def maintain_connection(name: str, url: str):
while True:
try:
logger.info(f"🔗 Attempting to connect to Skill '{name}' at {url}")
# Wir nutzen einen ContextManager, der die Verbindung offen hält
async with sse_client(url) as (read_stream, write_stream):
async with ClientSession(read_stream, write_stream) as session:
# 1. Handshake
await session.initialize()
# 2. In den globalen State schreiben
skill_sessions[name] = session
logger.info(f"✅ Skill '{name}' successfully integrated.")
# 3. WICHTIG: Wir müssen hier blockieren, damit wir den
# Context (async with) NICHT verlassen.
# wait_until_disconnected() sollte eigentlich funktionieren,
# aber wir können es mit einem asyncio.Event sicherer machen:
stop_event = asyncio.Event()
# Optional: Ein kleiner Loop, der prüft, ob die Session noch lebt
while not stop_event.is_set():
await asyncio.sleep(1)
# Hier könnte man einen Heartbeat prüfen, falls das SDK das bietet
except Exception as e:
logger.error(f"⚠️ Connection lost to '{name}': {e}. Retrying in 5s...")
finally:
skill_sessions.pop(name, None)
await asyncio.sleep(5)
# --- MCP SERVER LOGIC ---
@master_server.list_tools()
async def handle_list_tools() -> list[types.Tool]:
"""Bündelt die Tool-Listen aller aktuell verbundenen Microservices."""
all_tools = []
for name, session in skill_sessions.items():
try:
result = await session.list_tools()
all_tools.extend(result.tools)
except Exception as e:
logger.error(f"Could not list tools for {name}: {e}")
return all_tools
@master_server.call_tool()
async def handle_call_tool(name: str, arguments: dict | None) -> list[types.TextContent]:
"""Sucht das angeforderte Tool in den verbundenen Services und leitet den Call weiter."""
for skill_name, session in skill_sessions.items():
# Wir prüfen, ob dieser Service das gesuchte Tool anbietet
tools_result = await session.list_tools()
if any(t.name == name for t in tools_result.tools):
logger.info(f"⚡ Relaying tool call '{name}' to Skill '{skill_name}'")
result = await session.call_tool(name, arguments or {})
return result.content
raise ValueError(f"Tool '{name}' not found in any connected GhostNet Skill.")
# --- FASTAPI & SSE TRANSPORT ---
app = FastAPI(title="GhostNet-Router", lifespan=lifespan)
sse = SseServerTransport("/messages")
@app.get("/sse")
async def handle_sse(request: Request):
"""SSE-Endpunkt für den Agenten (OpenClaw)."""
async with sse.connect_sse(request.scope, request.receive, request._send) as (read_stream, write_stream):
await master_server.run(
read_stream,
write_stream,
master_server.create_initialization_options()
)
@app.post("/messages")
async def handle_messages(request: Request):
"""Post-Messages-Endpunkt für den bidirektionalen MCP-Austausch."""
await sse.handle_post_message(request.scope, request.receive, request._send)
@@ -0,0 +1,23 @@
use_default_settings: true
server:
bind_address: 0.0.0.0
port: 8080
secret_key: "48d28819fa08ae64546d354e72dbc6c7a1e8fc5d2ef69be0099dd38d2f22d8d7"
limiter: false
valkey:
url: "redis://tool-searchfetch-valkey:6379/0"
search:
formats:
- html
- json
engines:
- name: radio browser
disabled: true
- name: ahmia
disabled: true
- name: torch
disabled: true