284 lines
8.0 KiB
Markdown
284 lines
8.0 KiB
Markdown
# Neo4j - Graph Database Platform
|
|
|
|
## Overview
|
|
|
|
Neo4j is a high-performance graph database providing native graph storage and processing. It enables efficient traversal of complex relationships and is used for knowledge graphs, recommendation engines, and connected data analysis. Deployed with the **APOC plugin** enabled for extended stored procedures and functions.
|
|
|
|
**Host:** ariel.incus
|
|
**Role:** graph_database
|
|
**Container Port:** 25554 (HTTP Browser), 7687 (Bolt)
|
|
**External Access:** Direct Bolt connection via `ariel.incus:7687`
|
|
|
|
## Architecture
|
|
|
|
```
|
|
┌──────────────┐ ┌──────────────┐ ┌──────────────┐
|
|
│ Client │─────▶│ Neo4j │◀─────│ Neo4j MCP │
|
|
│ (Browser) │ │ (Ariel) │ │ (Miranda) │
|
|
└──────────────┘ └──────────────┘ └──────────────┘
|
|
│ │
|
|
│ ▼
|
|
│ ┌──────────────┐
|
|
└────────────▶│ Neo4j Browser│
|
|
│ HTTP :25554 │
|
|
└──────────────┘
|
|
```
|
|
|
|
- **Neo4j Browser**: Web-based query interface on port 25554
|
|
- **Bolt Protocol**: Binary protocol on port 7687 for high-performance connections
|
|
- **APOC Plugin**: Extended procedures for import/export, graph algorithms, and utilities
|
|
- **Neo4j MCP Servers**: Connect via Bolt from Miranda for AI agent access
|
|
|
|
## Terraform Resources
|
|
|
|
### Host Definition
|
|
|
|
The service runs on `ariel`, defined in `terraform/containers.tf`:
|
|
|
|
| Attribute | Value |
|
|
|-----------|-------|
|
|
| Image | noble |
|
|
| Role | graph_database |
|
|
| Security Nesting | true |
|
|
| AppArmor | unconfined |
|
|
| Description | Neo4j Host - Ethereal graph connections |
|
|
|
|
### Proxy Devices
|
|
|
|
| Device Name | Listen | Connect |
|
|
|-------------|--------|---------|
|
|
| neo4j_ports | tcp:0.0.0.0:25554 | tcp:127.0.0.1:25554 |
|
|
|
|
### Dependencies
|
|
|
|
| Resource | Relationship |
|
|
|----------|--------------|
|
|
| Prospero | Monitoring stack must exist for Alloy log shipping |
|
|
| Miranda | Neo4j MCP servers connect to Neo4j via Bolt |
|
|
|
|
## Ansible Deployment
|
|
|
|
### Playbook
|
|
|
|
```bash
|
|
cd ansible
|
|
ansible-playbook neo4j/deploy.yml
|
|
```
|
|
|
|
### Files
|
|
|
|
| File | Purpose |
|
|
|------|---------|
|
|
| `neo4j/deploy.yml` | Main deployment playbook |
|
|
| `neo4j/docker-compose.yml.j2` | Docker Compose template |
|
|
| `alloy/ariel/config.alloy.j2` | Alloy log collection config |
|
|
|
|
### Deployment Steps
|
|
|
|
1. **Create System User**: `neo4j:neo4j` system group and user
|
|
2. **Configure ponos Access**: Add ponos user to neo4j group
|
|
3. **Create Directory**: `/srv/neo4j` with proper ownership
|
|
4. **Template Compose File**: Apply `docker-compose.yml.j2`
|
|
5. **Start Service**: Launch via `docker_compose_v2` module
|
|
|
|
## Configuration
|
|
|
|
### Host Variables (`host_vars/ariel.incus.yml`)
|
|
|
|
| Variable | Description | Default |
|
|
|----------|-------------|---------|
|
|
| `neo4j_version` | Neo4j Docker image version | `5.26.0` |
|
|
| `neo4j_user` | System user | `neo4j` |
|
|
| `neo4j_group` | System group | `neo4j` |
|
|
| `neo4j_directory` | Installation directory | `/srv/neo4j` |
|
|
| `neo4j_auth_user` | Database admin username | `neo4j` |
|
|
| `neo4j_auth_password` | Database admin password | `{{ vault_neo4j_auth_password }}` |
|
|
| `neo4j_http_port` | HTTP browser port | `25554` |
|
|
| `neo4j_bolt_port` | Bolt protocol port | `7687` |
|
|
| `neo4j_syslog_port` | Local syslog port for Alloy | `22011` |
|
|
| `neo4j_apoc_unrestricted` | APOC procedures allowed | `apoc.*` |
|
|
|
|
### Vault Variables (`group_vars/all/vault.yml`)
|
|
|
|
| Variable | Description |
|
|
|----------|-------------|
|
|
| `vault_neo4j_auth_password` | Neo4j admin password |
|
|
|
|
### APOC Plugin Configuration
|
|
|
|
The APOC (Awesome Procedures on Cypher) plugin is enabled with the following settings:
|
|
|
|
| Environment Variable | Value | Purpose |
|
|
|---------------------|-------|---------|
|
|
| `NEO4J_PLUGINS` | `["apoc"]` | Install APOC plugin |
|
|
| `NEO4J_apoc_export_file_enabled` | `true` | Allow file exports |
|
|
| `NEO4J_apoc_import_file_enabled` | `true` | Allow file imports |
|
|
| `NEO4J_apoc_import_file_use__neo4j__config` | `true` | Use Neo4j config for imports |
|
|
| `NEO4J_dbms_security_procedures_unrestricted` | `apoc.*` | Allow all APOC procedures |
|
|
|
|
### Docker Volumes
|
|
|
|
| Volume | Mount Point | Purpose |
|
|
|--------|-------------|---------|
|
|
| `neo4j_data` | `/data` | Database files |
|
|
| `neo4j_logs` | `/logs` | Application logs |
|
|
| `neo4j_plugins` | `/plugins` | APOC and other plugins |
|
|
|
|
## Monitoring
|
|
|
|
### Alloy Configuration
|
|
|
|
**File:** `ansible/alloy/ariel/config.alloy.j2`
|
|
|
|
Alloy on Ariel collects:
|
|
- System logs (`/var/log/syslog`, `/var/log/auth.log`)
|
|
- Systemd journal
|
|
- Neo4j Docker container logs via syslog
|
|
|
|
### Loki Logs
|
|
|
|
| Log Source | Labels |
|
|
|------------|--------|
|
|
| Neo4j container | `{job="neo4j", hostname="ariel.incus"}` |
|
|
| System logs | `{job="syslog", hostname="ariel.incus"}` |
|
|
|
|
### Prometheus Metrics
|
|
|
|
Host-level metrics collected via Alloy's Unix exporter:
|
|
|
|
| Metric | Description |
|
|
|--------|-------------|
|
|
| `node_*` | Standard node exporter metrics |
|
|
|
|
### Log Collection Flow
|
|
|
|
```
|
|
Neo4j Container → Syslog (tcp:127.0.0.1:22011) → Alloy → Loki (Prospero)
|
|
```
|
|
|
|
## Operations
|
|
|
|
### Start/Stop
|
|
|
|
```bash
|
|
# Via Docker Compose
|
|
cd /srv/neo4j
|
|
docker compose up -d
|
|
docker compose down
|
|
|
|
# Via Ansible
|
|
ansible-playbook neo4j/deploy.yml
|
|
```
|
|
|
|
### Health Check
|
|
|
|
```bash
|
|
# HTTP Browser
|
|
curl http://ariel.incus:25554
|
|
|
|
# Bolt connection test
|
|
cypher-shell -a bolt://ariel.incus:7687 -u neo4j -p <password> "RETURN 1"
|
|
```
|
|
|
|
### Logs
|
|
|
|
```bash
|
|
# Docker container logs
|
|
docker logs -f neo4j
|
|
|
|
# Via Loki (Grafana Explore)
|
|
{job="neo4j", hostname="ariel.incus"}
|
|
```
|
|
|
|
### Cypher Shell Access
|
|
|
|
```bash
|
|
# SSH to Ariel and exec into container
|
|
ssh ariel.incus
|
|
docker exec -it neo4j cypher-shell -u neo4j -p <password>
|
|
```
|
|
|
|
### Backup
|
|
|
|
Neo4j data persists in Docker volumes. Backup procedures:
|
|
|
|
```bash
|
|
# Stop container for consistent backup
|
|
docker compose -f /srv/neo4j/docker-compose.yml stop
|
|
|
|
# Backup volumes
|
|
docker run --rm -v neo4j_data:/data -v /backup:/backup alpine \
|
|
tar czf /backup/neo4j_data_$(date +%Y%m%d).tar.gz -C /data .
|
|
|
|
# Start container
|
|
docker compose -f /srv/neo4j/docker-compose.yml up -d
|
|
```
|
|
|
|
### Restore
|
|
|
|
```bash
|
|
# Stop container
|
|
docker compose -f /srv/neo4j/docker-compose.yml down
|
|
|
|
# Remove existing volume
|
|
docker volume rm neo4j_data
|
|
|
|
# Create new volume and restore
|
|
docker volume create neo4j_data
|
|
docker run --rm -v neo4j_data:/data -v /backup:/backup alpine \
|
|
tar xzf /backup/neo4j_data_YYYYMMDD.tar.gz -C /data
|
|
|
|
# Start container
|
|
docker compose -f /srv/neo4j/docker-compose.yml up -d
|
|
```
|
|
|
|
## Troubleshooting
|
|
|
|
### Common Issues
|
|
|
|
| Symptom | Cause | Resolution |
|
|
|---------|-------|------------|
|
|
| Container won't start | Auth format issue | Check `NEO4J_AUTH` format is `user/password` |
|
|
| APOC procedures fail | Security restrictions | Verify `neo4j_apoc_unrestricted` includes procedure |
|
|
| Connection refused | Port not exposed | Check Incus proxy device configuration |
|
|
| Bolt connection fails | Wrong port | Use port 7687, not 25554 |
|
|
|
|
### Debug Mode
|
|
|
|
```bash
|
|
# View container startup logs
|
|
docker logs neo4j
|
|
|
|
# Check Neo4j internal logs
|
|
docker exec neo4j cat /logs/debug.log
|
|
```
|
|
|
|
### Verify APOC Installation
|
|
|
|
```cypher
|
|
CALL apoc.help("apoc")
|
|
YIELD name, text
|
|
RETURN name, text LIMIT 10;
|
|
```
|
|
|
|
## Related Services
|
|
|
|
### Neo4j MCP Servers (Miranda)
|
|
|
|
Two MCP servers run on Miranda to provide AI agent access to Neo4j:
|
|
|
|
| Server | Port | Purpose |
|
|
|--------|------|---------|
|
|
| neo4j-cypher | 25531 | Direct Cypher query execution |
|
|
| neo4j-memory | 25532 | Knowledge graph memory operations |
|
|
|
|
See [Neo4j MCP documentation](#neo4j-mcp-servers) for deployment details.
|
|
|
|
## References
|
|
|
|
- [Neo4j Documentation](https://neo4j.com/docs/)
|
|
- [APOC Library Documentation](https://neo4j.com/labs/apoc/)
|
|
- [Terraform Practices](../terraform.md)
|
|
- [Ansible Practices](../ansible.md)
|
|
- [Sandbox Overview](../ouranos.html)
|