99 lines
4.4 KiB
Markdown
99 lines
4.4 KiB
Markdown
## 🤖 GhostNet OpenClaw & Multi-Vendor Ollama Pipeline
|
|
A highly optimized, hardware-accelerated local AI infrastructure leveraging OpenClaw, Ollama, and a modular MCP (Model Context Protocol) gateway running inside containerized isolation. Tailored specifically for Podman (rootless/SELinux) architectures on rolling-release host systems (like CachyOS / Arch Linux).
|
|
|
|
------------------------------
|
|
|
|
## Quick Start
|
|
|
|
### 1. Environment Setup
|
|
Clone the repository and prepare your environment configuration:
|
|
`cp .env.example .env`
|
|
|
|
Open `.env` and set your `GPU_TYPE` (amd, nvidia, or intel) and your desired models.
|
|
|
|
### 2. Boot the Stack
|
|
Use the integrated management script, which automatically selects the correct vendor-specific compose file:
|
|
|
|
```bash
|
|
# Start the stack in the background
|
|
./ghostnet.sh up
|
|
```
|
|
|
|
```bash
|
|
# For debugging: Stop everything and start in the foreground (Attached Mode)
|
|
./ghostnet.sh attached
|
|
```
|
|
|
|
### 3. Open the Interactive Chat
|
|
`podman exec -it openclaw-agent openclaw chat`
|
|
|
|
## Technical Architecture & Key Highlights
|
|
This setup relies on unique architectural design patterns engineered to overcome container engines boundaries and system strictness:
|
|
|
|
```
|
|
+-------------------------------------------------------------+
|
|
| Host Hardware (GPU) |
|
|
+-------------------------------------------------------------+
|
|
^
|
|
| Passthrough (CDI / DRI / rwm)
|
|
v
|
|
+-------------------------------------------------------------+
|
|
| ollama (Container) |
|
|
+-------------------------------------------------------------+
|
|
^
|
|
| (Internal Bridge Net: Port 11434)
|
|
v
|
|
+-------------------------------------------------------------+
|
|
| openclaw-ollama-bridge (Sidecar) |
|
|
| - Tunnels 127.0.0.1:11434 directly to ollama:11434 |
|
|
+-------------------------------------------------------------+
|
|
^
|
|
| (Shared Network Namespace)
|
|
v
|
|
+-------------------------------------------------------------+
|
|
| openclaw-agent (Container) |
|
|
| - Runs 'openclaw gateway start' on the main thread |
|
|
| - Injects 'openclaw chat' (TUI) via podman exec |
|
|
+-------------------------------------------------------------+
|
|
|
|
```
|
|
|
|
## MCP Microservice Gateway (The GhostNet Router)
|
|
* **The Challenge:** OpenClaw needs to scale its capabilities (Search, Filesystem, DB) without bloating the main agent container or creating dependency hell.
|
|
* **The Solution:** A centralized **Python-based MCP Router**. It acts as a single API Gateway that aggregates multiple "Skills" (Bridges).
|
|
* **Key Feature:** The Router maintains persistent, asynchronous SSE connections to sub-services (like `searchfetch`) using a robust `maintain_connection` logic, ensuring the agent always has access to live tools without manual re-initialization.
|
|
|
|
## The Loopback TUI Bypass (The Socat Sidecar)
|
|
* **The Challenge:** The OpenClaw TUI strictly forces connections to `127.0.0.1:11434`, ignoring environment variables.
|
|
* **The Solution:** A Socat Sidecar (`network_mode: "service:agent"`) that tunnels local loopback traffic directly to the isolated Ollama container.
|
|
|
|
## Monolithic Multi-Vendor Composability
|
|
* Standalone compose files (`compose.amd.yaml` etc.) prevent the "Schema-Drop" bug of `podman-compose`, ensuring that `devices:` and `group_add:` mappings for ROCm/CUDA are never silently discarded.
|
|
|
|
## Rootless Storage Mandates (keep-id & ,U)
|
|
* Synchronization of host/container UIDs via `userns_mode: "keep-id"` combined with `:Z,U` flags. This allows the Node.js agent to write to host-mounted workspaces while maintaining strict SELinux compliance.
|
|
|
|
## Infrastructure Verification
|
|
|
|
```bash
|
|
# Inspect Engine Acceleration
|
|
# Success: PROCESSOR reads 100% GPU.
|
|
`podman exec -it ollama ollama ps`
|
|
```
|
|
|
|
```bash
|
|
# Check MCP Router Integrity
|
|
# Verify that the Gateway is alive and the Search-Bridge is successfully integrated.
|
|
curl -N http://tool-router:3000/sse
|
|
```
|
|
|
|
```bash
|
|
# Shut down the environment cleanly without state deadlocks
|
|
podman-compose -f compose.<vendor>.yaml down
|
|
```
|
|
|
|
```bash
|
|
# Run system integrity and validation checks
|
|
podman exec -it openclaw-agent openclaw doctor
|
|
```
|