docs: rewrite README with structured overview and quick start guide

Replaces the minimal project description with a comprehensive README including a component overview table, quick start instructions, common Ansible operations, and links to detailed documentation. Aligns with Red Panda Approval™ standards.
2026-03-03 12:49:06 +00:00
parent c7be03a743
commit b4d60f2f38
219 changed files with 34586 additions and 2 deletions
--- a/docs/mcpo.md
+++ b/docs/mcpo.md
@@ -0,0 +1,303 @@
+# MCPO - Model Context Protocol OpenAI-Compatible Proxy
+
+## Overview
+
+MCPO is an OpenAI-compatible proxy that aggregates multiple Model Context Protocol (MCP) servers behind a single HTTP endpoint. It acts as the central MCP gateway for the Agathos sandbox, exposing tools from 13 MCP servers through a unified REST API with interactive Swagger documentation.
+
+**Host:** miranda.incus  
+**Role:** MCP Docker Host  
+**Service Port:** 25530  
+**API Docs:** http://miranda.incus:25530/docs
+
+## Architecture
+
+```
+┌───────────────┐     ┌──────────────────────────────────────────────────────────┐
+│  LLM Client   │     │  Miranda (miranda.incus)                                │
+│  (LobeChat,   │────▶│  ┌────────────────────────────────────────────────────┐  │
+│   Open WebUI, │     │  │  MCPO :25530                                      │  │
+│   VS Code)    │     │  │  OpenAI-compatible proxy                          │  │
+└───────────────┘     │  └─────┬────────────┬────────────┬───────────────────┘  │
+                      │        │            │            │                       │
+                      │  ┌─────▼─────┐ ┌────▼────┐ ┌────▼─────┐                │
+                      │  │  stdio    │ │  Local  │ │  Remote  │                │
+                      │  │  servers  │ │  Docker │ │  servers │                │
+                      │  │           │ │  MCP    │ │          │                │
+                      │  │ • time    │ │         │ │ • athena │                │
+                      │  │ • ctx7    │ │ • neo4j │ │ • github │                │
+                      │  │           │ │ • graf  │ │ • hface  │                │
+                      │  │           │ │ • gitea │ │ • argos  │                │
+                      │  │           │ │         │ │ • rommie │                │
+                      │  │           │ │         │ │ • caliban│                │
+                      │  │           │ │         │ │ • korax  │                │
+                      │  └───────────┘ └─────────┘ └──────────┘                │
+                      └──────────────────────────────────────────────────────────┘
+```
+
+MCPO manages two categories of MCP servers:
+- **stdio servers**: MCPO spawns and manages the process (time, context7)
+- **streamable-http servers**: MCPO proxies to Docker containers on localhost or remote services across the Incus network
+
+## Terraform Resources
+
+### Host Definition
+
+MCPO runs on Miranda, defined in `terraform/containers.tf`:
+
+| Attribute | Value |
+|-----------|-------|
+| Image | noble |
+| Role | mcp_docker_host |
+| Security Nesting | true |
+| AppArmor | unconfined |
+| Proxy: mcp_containers | `0.0.0.0:25530-25539` → `127.0.0.1:25530-25539` |
+| Proxy: mcpo_ports | `0.0.0.0:25560-25569` → `127.0.0.1:25560-25569` |
+
+### Dependencies
+
+| Resource | Relationship |
+|----------|--------------|
+| prospero | Monitoring (Alloy → Loki, Prometheus) |
+| ariel | Neo4j database for neo4j-cypher and neo4j-memory MCP servers |
+| puck | Athena MCP server |
+| caliban | Caliban and Rommie MCP servers |
+
+## Ansible Deployment
+
+### Playbook
+
+```bash
+cd ansible
+ansible-playbook mcpo/deploy.yml
+```
+
+### Files
+
+| File | Purpose |
+|------|---------|
+| `mcpo/deploy.yml` | Main deployment playbook |
+| `mcpo/config.json.j2` | MCP server configuration template |
+| `mcpo/mcpo.service.j2` | Systemd service unit template |
+| `mcpo/restart.yml` | Restart playbook with health check |
+| `mcpo/requirements.txt` | Python package requirements |
+
+### Deployment Steps
+
+1. **Create System User**: `mcpo:mcpo` system account
+2. **Create Directory**: `/srv/mcpo` with restricted permissions
+3. **Backup Config**: Saves existing `config.json` before overwriting
+4. **Template Config**: Renders `config.json.j2` with MCP server definitions
+5. **Install Node.js 22.x**: NodeSource repository for npx-based MCP servers
+6. **Install Python 3.12**: System packages for virtual environment
+7. **Create Virtual Environment**: Python 3.12 venv at `/srv/mcpo/.venv`
+8. **Install pip Packages**: `wheel`, `mcpo`, `mcp-server-time`
+9. **Pre-install Context7**: Downloads `@upstash/context7-mcp` via npx
+10. **Deploy Systemd Service**: Enables and starts `mcpo.service`
+11. **Health Check**: Verifies `http://localhost:25530/docs` returns HTTP 200
+
+## MCP Servers
+
+MCPO aggregates the following MCP servers in `config.json`:
+
+### stdio Servers (managed by MCPO)
+
+| Server | Command | Purpose |
+|--------|---------|---------|
+| `time` | `mcp-server-time` (Python venv) | Current time with timezone support |
+| `upstash-context7` | `npx @upstash/context7-mcp` | Library documentation lookup |
+
+### streamable-http Servers (local Docker containers)
+
+| Server | URL | Purpose |
+|--------|-----|---------|
+| `neo4j-cypher` | `localhost:25531/mcp` | Neo4j Cypher query execution |
+| `neo4j-memory` | `localhost:25532/mcp` | Neo4j knowledge graph memory |
+| `grafana` | `localhost:25533/mcp` | Grafana dashboard and API integration |
+| `gitea` | `localhost:25535/mcp` | Gitea repository management |
+
+### streamable-http Servers (remote services)
+
+| Server | URL | Purpose |
+|--------|-----|---------|
+| `argos-searxng` | `miranda.incus:25534/mcp` | SearXNG search integration |
+| `athena` | `puck.incus:22461/mcp` | Athena knowledge service (auth required) |
+| `github` | `api.githubcopilot.com/mcp/` | GitHub API integration |
+| `rommie` | `caliban.incus:8080/mcp` | Rommie agent interface |
+| `caliban` | `caliban.incus:22021/mcp` | Caliban computer use agent |
+| `korax` | `korax.helu.ca:22021/mcp` | Korax external agent |
+| `huggingface` | `huggingface.co/mcp` | Hugging Face model hub |
+
+## Configuration
+
+### Systemd Service
+
+MCPO runs as a systemd service:
+
+```
+ExecStart=/srv/mcpo/.venv/bin/mcpo --port 25530 --config /srv/mcpo/config.json
+```
+
+- **User:** mcpo
+- **Restart:** always (3s delay)
+- **WorkingDirectory:** /srv/mcpo
+
+### Storage Locations
+
+| Path | Purpose | Owner |
+|------|---------|-------|
+| `/srv/mcpo` | Service directory | mcpo:mcpo |
+| `/srv/mcpo/.venv` | Python virtual environment | mcpo:mcpo |
+| `/srv/mcpo/config.json` | MCP server configuration | mcpo:mcpo |
+| `/srv/mcpo/config.json.bak` | Config backup (pre-deploy) | mcpo:mcpo |
+
+## Required Vault Secrets
+
+Add to `ansible/inventory/group_vars/all/vault.yml`:
+
+| Variable | Purpose |
+|----------|---------|
+| `vault_athena_mcp_auth` | Bearer token for Athena MCP server |
+| `vault_github_personal_access_token` | GitHub personal access token |
+| `vault_huggingface_mcp_token` | Hugging Face API token |
+| `vault_gitea_mcp_access_token` | Gitea personal access token for MCP |
+
+```bash
+ansible-vault edit inventory/group_vars/all/vault.yml
+```
+
+## Host Variables
+
+**File:** `ansible/inventory/host_vars/miranda.incus.yml`
+
+```yaml
+# MCPO Config
+mcpo_user: mcpo
+mcpo_group: mcpo
+mcpo_directory: /srv/mcpo
+mcpo_port: 25530
+argos_mcp_url: http://miranda.incus:25534/mcp
+athena_mcp_auth: "{{ vault_athena_mcp_auth }}"
+athena_mcp_url: http://puck.incus:22461/mcp
+github_personal_access_token: "{{ vault_github_personal_access_token }}"
+neo4j_cypher_mcp_port: 25531
+neo4j_memory_mcp_port: 25532
+caliban_mcp_url: http://caliban.incus:22021/mcp
+korax_mcp_url: http://korax.helu.ca:22021/mcp
+huggingface_mcp_token: "{{ vault_huggingface_mcp_token }}"
+gitea_mcp_port: 25535
+```
+
+## Monitoring
+
+### Loki Logs
+
+MCPO logs are collected via systemd journal by Alloy on Miranda. A relabel rule in Alloy's config tags `mcpo.service` journal entries with `job="mcpo"` so they appear as a dedicated app in Grafana dashboards.
+
+| Log Source | Labels |
+|------------|--------|
+| Systemd journal | `{job="mcpo", hostname="miranda.incus"}` |
+
+The Docker-based MCP servers (neo4j, grafana, gitea) each have dedicated syslog ports forwarded to Loki:
+
+| Server | Syslog Port | Loki Job |
+|--------|-------------|----------|
+| neo4j-cypher | 51431 | `neo4j-cypher` |
+| neo4j-memory | 51432 | `neo4j-memory` |
+| grafana-mcp | 51433 | `grafana_mcp` |
+| argos | 51434 | `argos` |
+| gitea-mcp | 51435 | `gitea-mcp` |
+
+### Grafana
+
+Query MCPO-related logs in Grafana Explore:
+
+```
+{hostname="miranda.incus", job="mcpo"}
+{hostname="miranda.incus", job="gitea-mcp"}
+{hostname="miranda.incus", job="grafana_mcp"}
+```
+
+## Operations
+
+### Start/Stop
+
+```bash
+ssh miranda.incus
+
+# MCPO service
+sudo systemctl start mcpo
+sudo systemctl stop mcpo
+sudo systemctl restart mcpo
+
+# Or use the restart playbook with health check
+cd ansible
+ansible-playbook mcpo/restart.yml
+```
+
+### Health Check
+
+```bash
+# API docs endpoint
+curl http://miranda.incus:25530/docs
+
+# From Miranda itself
+curl http://localhost:25530/docs
+```
+
+### Logs
+
+```bash
+# MCPO systemd journal
+ssh miranda.incus "sudo journalctl -u mcpo -f"
+
+# Docker MCP server logs
+ssh miranda.incus "docker logs -f gitea-mcp"
+ssh miranda.incus "docker logs -f grafana-mcp"
+```
+
+### Adding a New MCP Server
+
+1. Add the server definition to `ansible/mcpo/config.json.j2`
+2. Add any required variables to `ansible/inventory/host_vars/miranda.incus.yml`
+3. Add vault secrets (if needed) to `inventory/group_vars/all/vault.yml`
+4. If Docker-based: create a new `ansible/{service}/deploy.yml` and `docker-compose.yml.j2`
+5. If Docker-based: add a syslog port to Miranda's host vars and Alloy config
+6. Redeploy: `ansible-playbook mcpo/deploy.yml`
+
+## Troubleshooting
+
+### Common Issues
+
+| Symptom | Cause | Resolution |
+|---------|-------|------------|
+| MCPO won't start | Config JSON syntax error | Check `config.json` with `python -m json.tool` |
+| Server shows "unavailable" | Backend MCP server not running | Check Docker containers or remote service status |
+| Context7 timeout on first use | npx downloading package | Wait for download to complete, or re-run pre-install |
+| Health check fails | Port not ready | Increase retry delay, check `journalctl -u mcpo` |
+| stdio server crash loops | Missing runtime dependency | Verify Python venv and Node.js installation |
+
+### Debug Commands
+
+```bash
+# Check MCPO service status
+ssh miranda.incus "sudo systemctl status mcpo"
+
+# Validate config.json syntax
+ssh miranda.incus "python3 -m json.tool /srv/mcpo/config.json"
+
+# List Docker MCP containers
+ssh miranda.incus "docker ps --filter name=mcp"
+
+# Test a specific MCP server endpoint
+ssh miranda.incus "curl -s http://localhost:25531/mcp | head"
+
+# Check MCPO port is listening
+ssh miranda.incus "ss -tlnp | grep 25530"
+```
+
+## References
+
+- **MCPO Repository**: https://github.com/nicobailey/mcpo
+- **MCP Specification**: https://modelcontextprotocol.io/
+- [Ansible Practices](ansible.md)
+- [Agathos Overview](agathos.md)