ghostnet-openclaw/README.md

🤖 GhostNet OpenClaw & Multi-Vendor Ollama Pipeline

A highly optimized, hardware-accelerated local AI infrastructure leveraging OpenClaw, Ollama, and a modular MCP (Model Context Protocol) gateway running inside containerized isolation. Tailored specifically for Podman (rootless/SELinux) architectures on rolling-release host systems (like CachyOS / Arch Linux).

---

Quick Start

1. Environment Setup

Clone the repository and prepare your environment configuration: cp .env.example .env

Open .env and set your GPU_TYPE (amd, nvidia, or intel) and your desired models. Set OPENCLAW_DEFAULT_MODEL to the model OpenClaw should use by default.

2. Boot the Stack

Use the integrated management script, which automatically selects the correct vendor-specific compose file:

# Start the stack in the background
./ghostnet.sh up

# For debugging: Stop everything and start in the foreground (Attached Mode)
./ghostnet.sh attached

3. Open the Interactive Chat

podman exec -it openclaw-agent openclaw chat

Using the Stack with Cline in VS Code

You do not need a second local model runtime for Cline. The existing Ollama container is already published on the host at http://127.0.0.1:11434.

1. Configure Cline to use Ollama

In Cline, select the Ollama provider and use:

Base URL: http://127.0.0.1:11434
Model: qwen3-coding

Use ollama list inside the container or on the host to see the exact model names currently available.

Recommended custom profiles for this stack:

qwen3-general      -> Qwen3 14B, universal profile for OpenClaw chats
qwen3-coding       -> Qwen3 14B, balanced default for agentic coding
qwen25-coding      -> Qwen2.5-Coder 14B, code-first edit and repair profile
deepseek-coding    -> DeepSeek-Coder-V2 16B, long-context repo analysis profile

2. Reuse the existing MCP router for tools

The GhostNet router exposes an MCP SSE endpoint at http://127.0.0.1:3000/sse. Add it to your Cline MCP configuration:

{
    "mcpServers": {
        "ghostnet-router": {
            "transport": "sse",
            "url": "http://127.0.0.1:3000/sse"
        }
    }
}

This lets Cline reuse the same routed tool surface that OpenClaw uses internally.

3. What Cline does not need

Cline does not need to connect to the OpenClaw gateway on port 8080 for normal local usage. The direct path is:

• Cline -> Ollama for model inference
• Cline -> GhostNet router for MCP tools

If you change the host binding, use TOOL_ROUTER_BIND and TOOL_ROUTER_HOST_PORT in .env.

Benchmarking Custom Models

The repository now ships two complementary benchmark entry points:

• ./scripts/benchmark-stack.sh and ./scripts/benchmark-model-suite.sh exercise the full OpenClaw agent path.
• ./scripts/benchmark-codeneedle-suite.sh runs the upstream CodeNeedle recall benchmark against the same local Ollama instance.

The benchmark flow is split intentionally into runtime artifacts and curated results:

• Runtime artifacts go to .cache/benchmarks/existing-suite/runs/$RUN_ID/.
• Final benchmark outputs go to docs/benchmarks/existing-suite/$RUN_ID/.

Generated result sets now include more than raw summary.txt files:

• report.html per model for a quick interactive dashboard.
• RESULTS.md per model for git-friendly lab notes.
• SUITE_COMPARISON.html for side-by-side multi-model comparison.

These reports are generated automatically at the end of the benchmark scripts through scripts/lib/report-generator.sh.

Current timeout defaults were raised for slower local models and more reliable result capture:

• benchmark-stack.sh: 600s
• benchmark-model-suite.sh: 600s
• benchmark-codeneedle-suite.sh: 600s

The current Ollama tuning used for benchmark stability is:

• OLLAMA_KEEP_ALIVE=-1
• OLLAMA_KV_CACHE_TYPE=q8_0
• OLLAMA_MAX_LOADED_MODELS=1
• OLLAMA_NUM_PARALLEL=1
• SHM_SIZE=16gb

OpenClaw path examples:

./scripts/benchmark-stack.sh --model ollama/qwen3-coding:latest --lines 140 --timeout-sec 600 --no-start
./scripts/benchmark-model-suite.sh --lines 140 --timeout-sec 600

CodeNeedle path example:

./scripts/benchmark-codeneedle-suite.sh \
    --model qwen3-coding:latest \
    --model qwen25-coding:latest \
    --model deepseek-coding:latest

benchmark-codeneedle-suite.sh clones the upstream CodeNeedle repository into .cache/benchmarks/codeneedle/upstream, builds its Dockerfile with the configured container engine, and benchmarks the standard http_server and jquery corpora against http://127.0.0.1:11434.

Use --runtime-root, --results-root, or the legacy --artifact-root overrides if you want to keep artifacts somewhere else for a run.

Typical curated benchmark outputs look like this:

docs/benchmarks/existing-suite/model-suite-TIMESTAMP/
├── qwen3-coding/
│   ├── summary.txt
│   ├── assistant-output.txt
│   ├── report.html
│   └── RESULTS.md
├── qwen25-coding/
│   └── ...
└── SUITE_COMPARISON.html

Technical Architecture & Key Highlights

This setup relies on unique architectural design patterns engineered to overcome container engines boundaries and system strictness:

+-------------------------------------------------------------+
|                     Host Hardware (GPU)                     |
+-------------------------------------------------------------+
                               ^
                               | Passthrough (CDI / DRI / rwm)
                               v
+-------------------------------------------------------------+
|                     ollama (Container)                      |
+-------------------------------------------------------------+
                               ^
                               | (Internal Bridge Net: Port 11434)
                               v
+-------------------------------------------------------------+
|             openclaw-ollama-bridge (Sidecar)                |
|  - Tunnels 127.0.0.1:11434 directly to ollama:11434         |
+-------------------------------------------------------------+
                               ^
                               | (Shared Network Namespace)
                               v
+-------------------------------------------------------------+
|                openclaw-agent (Container)                   |
|  - Runs 'openclaw gateway start' on the main thread         |
|  - Injects 'openclaw chat' (TUI) via podman exec            |
+-------------------------------------------------------------+

MCP Microservice Gateway (The GhostNet Router)

• The Challenge: OpenClaw needs to scale its capabilities (Search, Filesystem, DB) without bloating the main agent container or creating dependency hell.
• The Solution: A centralized Python-based MCP Router. It acts as a single API Gateway that aggregates multiple "Skills" (Bridges).
• Key Feature: The Router maintains persistent, asynchronous SSE connections to sub-services (like searchfetch) using a robust maintain_connection logic, ensuring the agent always has access to live tools without manual re-initialization.

The Loopback TUI Bypass (The Socat Sidecar)

• The Challenge: The OpenClaw TUI strictly forces connections to 127.0.0.1:11434, ignoring environment variables.
• The Solution: A Socat Sidecar (network_mode: "service:agent") that tunnels local loopback traffic directly to the isolated Ollama container.

Monolithic Multi-Vendor Composability

• Standalone compose files (compose.amd.yaml etc.) prevent the "Schema-Drop" bug of podman-compose, ensuring that devices: and group_add: mappings for ROCm/CUDA are never silently discarded.

Rootless Storage Mandates (keep-id & ,U)

• Synchronization of host/container UIDs via userns_mode: "keep-id" combined with :Z,U flags. This allows the Node.js agent to write to host-mounted workspaces while maintaining strict SELinux compliance.

Infrastructure Verification

# Inspect Engine Acceleration
# Success: PROCESSOR reads 100% GPU.
`podman exec -it ollama ollama ps`

# Check MCP Router Integrity
# Verify that the Gateway is alive and the Search-Bridge is successfully integrated.
curl -N http://tool-router:3000/sse

# Shut down the environment cleanly without state deadlocks
podman-compose -f compose.<vendor>.yaml down

# Run system integrity and validation checks
podman exec -it openclaw-agent openclaw doctor