## 🤖 GhostNet OpenClaw & Multi-Vendor Ollama Pipeline A highly optimized, hardware-accelerated local AI infrastructure leveraging OpenClaw, Ollama, and a modular MCP (Model Context Protocol) gateway running inside containerized isolation. Tailored specifically for Podman (rootless/SELinux) architectures on rolling-release host systems (like CachyOS / Arch Linux). ------------------------------ ## Quick Start ### 1. Environment Setup Clone the repository and prepare your environment configuration: `cp .env.example .env` Open `.env` and set your `GPU_TYPE` (amd, nvidia, or intel) and your desired models. Set `OPENCLAW_DEFAULT_MODEL` to the model OpenClaw should use by default. ### 2. Boot the Stack Use the integrated management script, which automatically selects the correct vendor-specific compose file: ```bash # Start the stack in the background ./ghostnet.sh up ``` ```bash # For debugging: Stop everything and start in the foreground (Attached Mode) ./ghostnet.sh attached ``` ### 3. Open the Interactive Chat `podman exec -it openclaw-agent openclaw chat` ## Using the Stack with Cline in VS Code You do not need a second local model runtime for Cline. The existing Ollama container is already published on the host at `http://127.0.0.1:11434`. ### 1. Configure Cline to use Ollama In Cline, select the Ollama provider and use: ```text Base URL: http://127.0.0.1:11434 Model: qwen3-coding ``` Use `ollama list` inside the container or on the host to see the exact model names currently available. Recommended custom profiles for this stack: ```text qwen3-general -> Qwen3 14B, universal profile for OpenClaw chats qwen3-coding -> Qwen3 14B, balanced default for agentic coding qwen25-coding -> Qwen2.5-Coder 14B, code-first edit and repair profile deepseek-coding -> DeepSeek-Coder-V2 16B, long-context repo analysis profile ``` ### 2. Reuse the existing MCP router for tools The GhostNet router exposes an MCP SSE endpoint at `http://127.0.0.1:3000/sse`. Add it to your Cline MCP configuration: ```json { "mcpServers": { "ghostnet-router": { "transport": "sse", "url": "http://127.0.0.1:3000/sse" } } } ``` This lets Cline reuse the same routed tool surface that OpenClaw uses internally. ### 3. What Cline does not need Cline does not need to connect to the OpenClaw gateway on port 8080 for normal local usage. The direct path is: * Cline -> Ollama for model inference * Cline -> GhostNet router for MCP tools If you change the host binding, use `TOOL_ROUTER_BIND` and `TOOL_ROUTER_HOST_PORT` in `.env`. ## Benchmarking Custom Models The repository now ships two complementary benchmark entry points: * `./scripts/benchmark-stack.sh` and `./scripts/benchmark-model-suite.sh` exercise the full OpenClaw agent path. * `./scripts/benchmark-codeneedle-suite.sh` runs the upstream CodeNeedle recall benchmark against the same local Ollama instance. The benchmark flow is split intentionally into runtime artifacts and curated results: * Runtime artifacts go to `.cache/benchmarks/existing-suite/runs/$RUN_ID/`. * Final benchmark outputs go to `docs/benchmarks/existing-suite/$RUN_ID/`. Generated result sets now include more than raw `summary.txt` files: * `report.html` per model for a quick interactive dashboard. * `RESULTS.md` per model for git-friendly lab notes. * `SUITE_COMPARISON.html` for side-by-side multi-model comparison. These reports are generated automatically at the end of the benchmark scripts through `scripts/lib/report-generator.sh`. Current timeout defaults were raised for slower local models and more reliable result capture: * `benchmark-stack.sh`: 600s * `benchmark-model-suite.sh`: 600s * `benchmark-codeneedle-suite.sh`: 600s The current Ollama tuning used for benchmark stability is: * `OLLAMA_KEEP_ALIVE=-1` * `OLLAMA_KV_CACHE_TYPE=q8_0` * `OLLAMA_MAX_LOADED_MODELS=1` * `OLLAMA_NUM_PARALLEL=1` * `SHM_SIZE=16gb` OpenClaw path examples: ```bash ./scripts/benchmark-stack.sh --model ollama/qwen3-coding:latest --lines 140 --timeout-sec 600 --no-start ./scripts/benchmark-model-suite.sh --lines 140 --timeout-sec 600 ``` CodeNeedle path example: ```bash ./scripts/benchmark-codeneedle-suite.sh \ --model qwen3-coding:latest \ --model qwen25-coding:latest \ --model deepseek-coding:latest ``` `benchmark-codeneedle-suite.sh` clones the upstream CodeNeedle repository into `.cache/benchmarks/codeneedle/upstream`, builds its Dockerfile with the configured container engine, and benchmarks the standard `http_server` and `jquery` corpora against `http://127.0.0.1:11434`. Use `--runtime-root`, `--results-root`, or the legacy `--artifact-root` overrides if you want to keep artifacts somewhere else for a run. Typical curated benchmark outputs look like this: ```text docs/benchmarks/existing-suite/model-suite-TIMESTAMP/ ├── qwen3-coding/ │ ├── summary.txt │ ├── assistant-output.txt │ ├── report.html │ └── RESULTS.md ├── qwen25-coding/ │ └── ... └── SUITE_COMPARISON.html ``` ## Technical Architecture & Key Highlights This setup relies on unique architectural design patterns engineered to overcome container engines boundaries and system strictness: ``` +-------------------------------------------------------------+ | Host Hardware (GPU) | +-------------------------------------------------------------+ ^ | Passthrough (CDI / DRI / rwm) v +-------------------------------------------------------------+ | ollama (Container) | +-------------------------------------------------------------+ ^ | (Internal Bridge Net: Port 11434) v +-------------------------------------------------------------+ | openclaw-ollama-bridge (Sidecar) | | - Tunnels 127.0.0.1:11434 directly to ollama:11434 | +-------------------------------------------------------------+ ^ | (Shared Network Namespace) v +-------------------------------------------------------------+ | openclaw-agent (Container) | | - Runs 'openclaw gateway start' on the main thread | | - Injects 'openclaw chat' (TUI) via podman exec | +-------------------------------------------------------------+ ``` ## MCP Microservice Gateway (The GhostNet Router) * **The Challenge:** OpenClaw needs to scale its capabilities (Search, Filesystem, DB) without bloating the main agent container or creating dependency hell. * **The Solution:** A centralized **Python-based MCP Router**. It acts as a single API Gateway that aggregates multiple "Skills" (Bridges). * **Key Feature:** The Router maintains persistent, asynchronous SSE connections to sub-services (like `searchfetch`) using a robust `maintain_connection` logic, ensuring the agent always has access to live tools without manual re-initialization. ## The Loopback TUI Bypass (The Socat Sidecar) * **The Challenge:** The OpenClaw TUI strictly forces connections to `127.0.0.1:11434`, ignoring environment variables. * **The Solution:** A Socat Sidecar (`network_mode: "service:agent"`) that tunnels local loopback traffic directly to the isolated Ollama container. ## Monolithic Multi-Vendor Composability * Standalone compose files (`compose.amd.yaml` etc.) prevent the "Schema-Drop" bug of `podman-compose`, ensuring that `devices:` and `group_add:` mappings for ROCm/CUDA are never silently discarded. ## Rootless Storage Mandates (keep-id & ,U) * Synchronization of host/container UIDs via `userns_mode: "keep-id"` combined with `:Z,U` flags. This allows the Node.js agent to write to host-mounted workspaces while maintaining strict SELinux compliance. ## Infrastructure Verification ```bash # Inspect Engine Acceleration # Success: PROCESSOR reads 100% GPU. `podman exec -it ollama ollama ps` ``` ```bash # Check MCP Router Integrity # Verify that the Gateway is alive and the Search-Bridge is successfully integrated. curl -N http://tool-router:3000/sse ``` ```bash # Shut down the environment cleanly without state deadlocks podman-compose -f compose..yaml down ``` ```bash # Run system integrity and validation checks podman exec -it openclaw-agent openclaw doctor ```