r/koios
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Docs: Engineering team

Browse Source
This commit is contained in:
Robert Helewka
2026-05-20 17:34:22 -04:00
parent 248ed0b006
commit c1cc6e26c5
10 changed files with 270 additions and 778 deletions
Download Patch File Download Diff File Expand all files Collapse all files

19
docs/neo4j/Neo4j-breaking-changes.md Normal file
View File

@@ -0,0 +1,19 @@
## Neo4j Version Compatibility Notes
Neo4j had significant breaking changes between version 4.x and 5.x regarding schema introspection:
**Neo4j 5.x+ (current):**
- Use `SHOW INDEXES` instead of `CALL db.indexes()`
- Use `SHOW CONSTRAINTS` instead of `CALL db.constraints()`
- Use `CALL db.schema.visualization()` for full schema (works in both versions)
**Neo4j 4.x and earlier:**
- Use `CALL db.indexes()`
- Use `CALL db.constraints()`
**Safe queries that work across versions:**
- `CALL db.schema.visualization()` - Full schema visualization
- `CALL db.labels()` - Get all node labels
- `CALL db.relationshipTypes()` - Get all relationship types
When querying indexes or constraints, prefer the `SHOW` commands for Neo4j 5+ environments.

75
docs/neo4j/neo4j-engineering.md Normal file
View File

@@ -0,0 +1,75 @@
# Neo4j Knowledge Graph — Engineering Team
You have access to a unified Neo4j knowledge graph shared across fifteen AI assistants (9 personal, 4 work, 2 engineering).
## Principles
1. **Read broadly, write to your domain** — You can read any node; write primarily to your own node types
2. **Always MERGE on `id`** — Check before creating to avoid duplicates
3. **Use consistent IDs** — Format: `{type}_{identifier}_{qualifier}` (e.g., `infra_neo4j_prod`, `proto_mcp_dashboard`)
4. **Always set timestamps** — `created_at` on CREATE, `updated_at` on every SET
5. **Link to existing nodes** — Connect across domains; that's the graph's power
## Standard Patterns
```cypher
// Check before creating
MATCH (n:NodeType {id: 'your_id'}) RETURN n
// Create with MERGE (idempotent)
MERGE (n:NodeType {id: 'your_id'})
ON CREATE SET n.created_at = datetime()
SET n.name = 'Name', n.updated_at = datetime()
// Link to existing nodes
MATCH (a:TypeA {id: 'a_id'}), (b:TypeB {id: 'b_id'})
MERGE (a)-[:RELATIONSHIP]->(b)
```
## Engineering Node Ownership
| Assistant | Domain | Owns |
|-----------|--------|------|
| **Scotty** | Infrastructure & Ops | Infrastructure, Incident |
| **Harper** | Prototyping & Hacking | Prototype, Experiment |
### Scotty's Nodes
| Node | Required | Optional |
|------|----------|----------|
| Infrastructure | id, name, type | status, environment, host, version, notes |
| Incident | id, title, severity | status, date, root_cause, resolution, duration |
### Harper's Nodes
| Node | Required | Optional |
|------|----------|----------|
| Prototype | id, name | status, tech_stack, purpose, outcome, notes |
| Experiment | id, title | hypothesis, result, date, learnings, notes |
## Key Relationships
- Infrastructure -[DEPENDS_ON]-> Infrastructure
- Infrastructure -[HOSTS]-> Project | Prototype
- Incident -[AFFECTED]-> Infrastructure
- Incident -[CAUSED_BY]-> Infrastructure
- Prototype -[DEPLOYED_ON]-> Infrastructure
- Prototype -[SUPPORTS]-> Opportunity
- Prototype -[DEMONSTRATES]-> Technology
- Experiment -[LED_TO]-> Prototype
- Experiment -[VALIDATES]-> MarketTrend
- Prototype -[AUTOMATES]-> Habit | Task
## Cross-Team Reads
- **Work team:** Projects (infrastructure requirements), Opportunities (demo needs), Client SLAs
- **Personal team:** Habits (automation candidates), Goals (tooling support)
- **Universal nodes:** Person, Location, Event, Topic, Goal (shared by all)
## Scotty ↔ Harper Handoff
Harper builds prototypes; Scotty makes them production-grade. Use the messaging system to coordinate handoffs.
## Full Schema Reference
See `docs/neo4j-unified-schema.md` for complete node definitions, all fields, and relationship types.

52
docs/neo4j/neo4j-personal.md Normal file
View File

@@ -0,0 +1,52 @@
# Neo4j Knowledge Graph — Personal Team
You have access to a unified Neo4j knowledge graph shared across fifteen AI assistants (9 personal, 4 work, 2 engineering).
## Principles
1. **Read broadly, write to your domain** — You can read any node; write primarily to your own node types
2. **Always MERGE on `id`** — Check before creating to avoid duplicates
3. **Use consistent IDs** — Format: `{type}_{identifier}_{qualifier}` (e.g., `trip_costarica_2025`, `recipe_carbonara_classic`)
4. **Always set timestamps** — `created_at` on CREATE, `updated_at` on every SET
5. **Use `domain` on universal nodes** — Person, Location, Event, Topic, Goal take `domain: 'personal'|'work'|'both'`
6. **Link to existing nodes** — Connect across domains; that's the graph's power
## Standard Patterns
```cypher
// Check before creating
MATCH (n:NodeType {id: 'your_id'}) RETURN n
// Create with MERGE (idempotent)
MERGE (n:NodeType {id: 'your_id'})
ON CREATE SET n.created_at = datetime()
SET n.name = 'Name', n.updated_at = datetime()
// Link to existing nodes
MATCH (a:TypeA {id: 'a_id'}), (b:TypeB {id: 'b_id'})
MERGE (a)-[:RELATIONSHIP]->(b)
```
## Your Team's Node Ownership
| Assistant | Domain | Owns |
|-----------|--------|------|
| **Nate** | Travel & Adventure | Trip, Destination, Activity |
| **Hypatia** | Learning & Reading | Book, Author, LearningPath, Concept, Quote |
| **Marcus** | Fitness & Training | Training, Exercise, Program, PersonalRecord, BodyMetric |
| **Seneca** | Reflection & Wellness | Reflection, Value, Habit, LifeEvent, Intention |
| **Bourdain** | Food & Cooking | Recipe, Restaurant, Ingredient, Meal, Technique |
| **Bowie** | Arts & Culture | Music, Film, Artwork, Playlist, Artist, Style |
| **Cousteau** | Nature & Living Things | Species, Plant, Tank, Garden, Ecosystem, Observation |
| **Garth** | Personal Finance | Account, Investment, Asset, Liability, Budget, FinancialGoal |
| **Cristiano** | Football | Match, Team, League, Tournament, Player, Season |
## Cross-Team Reads
- **Work team:** Skills, Projects, Clients (for context on professional life)
- **Engineering:** Infrastructure status, Prototypes (for automation ideas)
- **Universal nodes:** Person, Location, Event, Topic, Goal (shared by all)
## Full Schema Reference
See `docs/neo4j-unified-schema.md` for complete node definitions, all fields, and relationship types.

1942
docs/neo4j/neo4j-unified-schema.md Normal file
View File

File diff suppressed because it is too large Load Diff

301
docs/neo4j/neo4j-utils.md Normal file
View File

@@ -0,0 +1,301 @@
# Neo4j Utility Scripts
> Documentation for the database management scripts in `utils/`
---
## Scripts Overview
| Script | Purpose | Destructive? |
|--------|---------|:------------:|
| `neo4j-schema-init.py` | Create constraints, indexes, and sample data | No (idempotent) |
| `neo4j-reset.py` | Wipe all data, constraints, and indexes | **Yes** |
| `neo4j-validate.py` | Comprehensive validation report | No (read-only) |
---
## neo4j-schema-init.py
Creates the foundational schema for the unified knowledge graph: 74 uniqueness constraints, ~94 performance indexes, and 12 sample nodes with 5 cross-domain relationships.
### Usage
```bash
# Interactive — prompts for URI, user, password
python utils/neo4j-schema-init.py
# Specify URI (will prompt for user/password)
python utils/neo4j-schema-init.py --uri bolt://ariel.incus:7687
# Skip sample data creation
python utils/neo4j-schema-init.py --uri bolt://ariel.incus:7687 --skip-samples
# Test-only mode (no schema changes)
python utils/neo4j-schema-init.py --uri bolt://ariel.incus:7687 --test-only
# Quiet mode
python utils/neo4j-schema-init.py --uri bolt://ariel.incus:7687 --quiet
```
### What It Creates
1. **74 uniqueness constraints** — one per node type, on the `id` property
2. **~94 performance indexes** — on name/title, date, type/status/category, and domain fields
3. **12 sample nodes** — spanning all three teams (Personal, Work, Engineering)
4. **5 sample relationships** — demonstrating cross-domain connections
### Idempotent
Safe to run multiple times. Uses `IF NOT EXISTS` for constraints/indexes and `MERGE` for sample data.
---
## neo4j-reset.py
Wipes the database clean. Drops all constraints, indexes, nodes, and relationships.
### Usage
```bash
# Interactive — will prompt for confirmation
python utils/neo4j-reset.py --uri bolt://ariel.incus:7687
# Skip confirmation prompt
python utils/neo4j-reset.py --uri bolt://ariel.incus:7687 --force
```
### What It Does
1. Reports current database contents (node/relationship/constraint/index counts)
2. Drops all constraints
3. Drops all non-lookup indexes
4. Deletes all nodes and relationships (batched for large databases)
5. Verifies the database is clean
### Safety
- Requires typing `yes` to confirm (unless `--force`)
- Shows before/after counts so you know exactly what was removed
---
## neo4j-validate.py
Generates a comprehensive validation report. Share the output to verify the graph is correctly built.
### Usage
```bash
python utils/neo4j-validate.py --uri bolt://ariel.incus:7687
```
### What It Checks
| Section | What's Validated |
|---------|-----------------|
| **Connection** | Database reachable, APOC plugin available |
| **Constraints** | All 74 uniqueness constraints present, no extras |
| **Indexes** | Total count, spot-check of 11 key indexes |
| **Node Labels** | No unexpected labels (detects junk from Memory server, etc.) |
| **Sample Nodes** | All 12 sample nodes exist with correct properties |
| **Sample Relationships** | All 5 cross-domain relationships exist |
| **Relationship Summary** | Total count and breakdown by type |
| **Node Summary** | Total count and breakdown by label |
### Expected Clean Output
```
═════════════════════════════════════════════════════════════════
VALIDATION REPORT — Koios Unified Knowledge Graph
═════════════════════════════════════════════════════════════════
Schema Version: 2.1.0
...
RESULT: ALL 23 CHECKS PASSED ✓
═════════════════════════════════════════════════════════════════
```
---
## Standard Workflow
### Fresh Setup / Clean Slate
```bash
# 1. Wipe everything
python utils/neo4j-reset.py --uri bolt://ariel.incus:7687
# 2. Build schema and sample data
python utils/neo4j-schema-init.py --uri bolt://ariel.incus:7687
# 3. Validate
python utils/neo4j-validate.py --uri bolt://ariel.incus:7687
```
### Routine Validation
```bash
python utils/neo4j-validate.py --uri bolt://ariel.incus:7687
```
### Environment Variables
All three scripts support environment variables to avoid repeated prompts:
```bash
export NEO4J_URI="bolt://ariel.incus:7687"
export NEO4J_USER="neo4j"
export NEO4J_PASSWORD="your-password"
# Then just:
python utils/neo4j-reset.py --force
python utils/neo4j-schema-init.py --skip-docs
python utils/neo4j-validate.py
```
---
## Neo4j Python Driver — Lessons Learned
These patterns were discovered during development and are critical for anyone writing Cypher through the Neo4j Python driver (v5.x / v6.x).
### 1. Use Explicit Transactions for Writes
**Problem:** `session.run()` uses auto-commit transactions that don't reliably commit writes in the Neo4j Python driver 5.x+. Results must be fully consumed or the transaction may not commit.
**Bad — silently fails to persist:**
```python
with driver.session() as session:
session.run("CREATE (n:Person {id: 'test'})")
# Transaction may not commit!
```
**Good — explicit transaction with context manager:**
```python
with driver.session() as session:
with session.begin_transaction() as tx:
tx.run("CREATE (n:Person {id: 'test'})")
# Auto-commits when context exits normally
# Auto-rolls back on exception
```
**Also good — managed write transaction:**
```python
def create_person_tx(tx, name):
result = tx.run("CREATE (a:Person {name: $name}) RETURN a.id AS id", name=name)
record = result.single()
return record["id"]
with driver.session() as session:
node_id = session.execute_write(create_person_tx, "Alice")
```
### 2. Cypher MERGE Clause Ordering
**Problem:** `ON CREATE SET` must come immediately after `MERGE`, before any general `SET` clause. Placing `SET` before `ON CREATE SET` causes a syntax error.
**Bad — syntax error:**
```cypher
MERGE (p:Person {id: 'user_main'})
SET p.name = 'Main User',
p.updated_at = datetime()
ON CREATE SET p.created_at = datetime() -- ERROR: Invalid input 'ON'
```
**Good — correct clause order:**
```cypher
MERGE (p:Person {id: 'user_main'})
ON CREATE SET p.created_at = datetime()
SET p.name = 'Main User',
p.updated_at = datetime()
```
The full MERGE clause order is:
```
MERGE (pattern)
ON CREATE SET ... ← only runs when node is first created
ON MATCH SET ... ← only runs when node already exists (optional)
SET ... ← always runs
```
### 3. Consume Results in Transactions
**Problem:** In managed transactions (`execute_write`), results must be consumed within the transaction function. Unconsumed results can cause issues.
**Good pattern:**
```python
def create_node_tx(tx, node_id):
result = tx.run("MERGE (n:Person {id: $id}) RETURN n.id AS id", id=node_id)
record = result.single() # Consumes the result
return record["id"]
```
### 4. MATCH Returns No Rows ≠ Error
**Problem:** If a `MATCH` clause finds nothing, the query succeeds with zero rows — it does **not** raise an error. This means `MERGE` on a relationship after a failed `MATCH` silently does nothing.
```cypher
-- If person_xyz doesn't exist, this returns 0 rows (no error)
MATCH (p:Person {id: 'person_xyz'})
MATCH (b:Book {id: 'book_abc'})
MERGE (p)-[:COMPLETED]->(b)
-- Zero rows processed, zero relationships created, zero errors
```
**Mitigation:** Always check `result.single()` for `None` to detect this case:
```python
record = result.single()
if record is None:
logger.error("Endpoints not found — no relationship created")
```
### 5. Separate Node and Relationship Transactions
**Problem:** Creating nodes and then matching them for relationships in the same auto-commit transaction can fail because the nodes aren't visible yet within the same transaction scope.
**Good pattern:** Create all nodes in one explicit transaction (commit), then create relationships in a separate explicit transaction:
```python
# Transaction 1: Create nodes
with session.begin_transaction() as tx:
for query in node_queries:
tx.run(query)
# Auto-commits on exit
# Transaction 2: Create relationships (nodes now visible)
with session.begin_transaction() as tx:
for query in relationship_queries:
tx.run(query)
# Auto-commits on exit
```
### 6. MCP Memory Server vs Neo4j Cypher Server
**Problem:** The MCP Memory server (`@modelcontextprotocol/server-memory`) and Neo4j Cypher MCP server can both connect to the same Neo4j instance, but they use completely different data models.
| | Memory Server | Cypher Server |
|---|---|---|
| **Schema** | Fixed: `name`, `type`, `observations` | Your full custom schema |
| **Node labels** | `Memory`, `reference` | Your 74 defined types |
| **Relationships** | Simple string pairs | Rich typed relationships |
| **Query language** | API calls (`search_nodes`) | Full Cypher |
**Resolution:** If you have a custom Neo4j schema, use **only** the Cypher MCP server. Remove the Memory server to prevent it from polluting your graph with its own primitive node types.
---
## Dependencies
```
pip install neo4j
```
All three scripts require the `neo4j` Python package. APOC is optional but recommended (the init script's test suite checks for it).
---
## Version History
| Date | Change |
|------|--------|
| 2025-01-07 | Initial `neo4j-schema-init.py` |
| 2026-02-17 | Added `neo4j-reset.py` and `neo4j-validate.py` |
| 2026-02-17 | Fixed init script: explicit transactions, correct MERGE clause ordering |

57
docs/neo4j/neo4j-work.md Normal file
View File

@@ -0,0 +1,57 @@
# Neo4j Knowledge Graph — Work Team
You have access to a unified Neo4j knowledge graph shared across fifteen AI assistants (9 personal, 4 work, 2 engineering).
## Principles
1. **Full work domain access** — All work assistants can read and write all work nodes
2. **Always MERGE on `id`** — Check before creating to avoid duplicates
3. **Use consistent IDs** — Format: `{type}_{identifier}_{qualifier}` (e.g., `client_acme_corp`, `opp_acme_cx_2025`)
4. **Always set timestamps** — `created_at` on CREATE, `updated_at` on every SET
5. **Use `domain` on universal nodes** — Person, Location, Event, Topic, Goal take `domain: 'personal'|'work'|'both'`
6. **Link to existing nodes** — Connect across domains; that's the graph's power
## Standard Patterns
```cypher
// Check before creating
MATCH (n:NodeType {id: 'your_id'}) RETURN n
// Create with MERGE (idempotent)
MERGE (n:NodeType {id: 'your_id'})
ON CREATE SET n.created_at = datetime()
SET n.name = 'Name', n.updated_at = datetime()
// Link to existing nodes
MATCH (a:TypeA {id: 'a_id'}), (b:TypeB {id: 'b_id'})
MERGE (a)-[:RELATIONSHIP]->(b)
```
## Work Node Types
| Category | Nodes |
|----------|-------|
| **Business** | Client, Contact, Opportunity, Proposal, Project |
| **Market Intelligence** | Vendor, Competitor, MarketTrend, Technology |
| **Content & Visibility** | Content, Publication |
| **Professional Development** | Skill, Certification, Relationship |
| **Daily Operations** | Task, Meeting, Note, Decision |
## Assistant Focus Areas
| Assistant | Primary Focus | Key Nodes |
|-----------|--------------|-----------|
| **Alan** | Strategy & Business Model | Client, Vendor, Competitor, MarketTrend, Decision |
| **Ann** | Marketing & Visibility | Content, Publication, Topic |
| **Jeffrey** | Proposals & Sales | Opportunity, Proposal, Contact |
| **Jarvis** | Daily Execution | Task, Meeting, Note |
## Cross-Team Reads
- **Personal team:** Books (for skill development), Trips (for client travel), Goals (for career alignment)
- **Engineering:** Infrastructure (hosting projects), Prototypes (for client demos)
- **Universal nodes:** Person, Location, Event, Topic, Goal (shared by all)
## Full Schema Reference
See `docs/neo4j-unified-schema.md` for complete node definitions, all fields, and relationship types.

149
docs/neo4j/shared.md Normal file
View File

@@ -0,0 +1,149 @@
# Shared Tools & Infrastructure
## User
You are assisting **Robert Helewka**. Address him as Robert. His node in the Neo4j knowledge graph is `Person {id: "user_main", name: "Robert"}`.
## Your Toolbox (MCP Servers)
MCP tool discovery tells you what each tool does at runtime. This table gives you the operational context that tool descriptions don't:
| Server | Purpose | Location |
|--------|---------|----------|
| **korax** | Shell execution + file operations (Kernos) — primary workbench | korax.helu.ca |
| **neo4j** | Knowledge graph (Cypher queries) | ariel.incus |
| **gitea** | Git repository management | miranda.incus |
| **argos** | Web search + webpage fetching | miranda.incus |
| **rommie** | Computer automation (Agent S, MATE desktop) | caliban.incus |
| **github** | GitHub Copilot MCP | api.githubcopilot.com |
| **context7** | Library/framework documentation lookup | local (npx) |
| **time** | Current time and timezone | local |
**Korax is your workbench.** For shell commands and file operations, use Korax (Kernos MCP). Call `get_shell_config` first to see what commands are whitelisted.
Use the `time` server to check the current date when temporal context matters.
> **Note:** Not every assistant has every server. Your available servers are listed in your FastAgent config.
## Agathos Sandbox
You work within Agathos — a set of Incus containers (LXC) on a 10.10.0.0/24 network, named after moons of Uranus. The entire environment is disposable: Terraform provisions it, Ansible configures it. It can be rebuilt trivially.
Key hosts: ariel (Neo4j), miranda (MCP servers), oberon (Docker/SearXNG), portia (PostgreSQL), prospero (monitoring), puck (apps), sycorax (LLM proxy), caliban (agent automation), titania (HAProxy/SSO).
## Inter-Assistant Graph Messaging
Other assistants may leave you messages as `Note` nodes in the Neo4j knowledge
graph. Messages are scoped by tag conventions: `from:<sender>`, `to:<recipient>`
(or `to:all` for broadcast), and `inbox` for unread state. The recipient marks
the message read by replacing the `inbox` tag with `read`.
This protocol applies to every assistant on every team — Personal (Iolaus),
Work (Mentor), Engineering (Kottos). The shape is identical; only the
`from:`/`to:` tag values change per agent.
### When to read your inbox
Read on demand only. Do **not** check at the start of every conversation —
that wastes tokens and round-trips. Read when:
- The user explicitly asks you to check.
- A scheduler (Daedalus) invokes the inbox-check prompt against you. See
[mentor/docs/inbox_check_prompt.md](../../mentor/docs/inbox_check_prompt.md)
for the canonical scheduler prompt.
- You're picking up cross-domain work and want context from other agents.
### Reading your inbox
Call `read_neo4j_cypher` (substitute your own agent name for `<self>`):
```cypher
MATCH (n:Note)
WHERE n.type = 'assistant_message'
AND ANY(tag IN n.tags WHERE tag IN ['to:<self>', 'to:all'])
AND ANY(tag IN n.tags WHERE tag = 'inbox')
RETURN n.id AS id, n.title AS title, n.content AS content,
n.action_required AS action_required, n.tags AS tags,
n.created_at AS sent_at
ORDER BY n.created_at DESC
```
If messages were returned, mark them all read with a single write
(substituting the actual IDs into `$ids`):
```cypher
MATCH (n:Note)
WHERE n.id IN $ids
SET n.tags = [tag IN n.tags WHERE tag <> 'inbox'] + ['read'],
n.updated_at = datetime()
```
If no messages were returned, skip the write entirely.
Acknowledge messages naturally in conversation. If `action_required: true`,
prioritize addressing the request.
### Sending messages to other assistants
Call `write_neo4j_cypher` with this exact parameterized query (no string
interpolation in the query body — all values come from `params`):
```cypher
MERGE (n:Note {id: $id})
ON CREATE SET n.created_at = datetime()
SET n.title = $title,
n.date = date(),
n.type = 'assistant_message',
n.content = $content,
n.action_required = $action_required,
n.tags = ['from:<self>', $to_tag, 'inbox'],
n.updated_at = datetime()
```
`<self>` is your own agent name (a constant in the query body — `'from:harper'`,
`'from:bourdain'`, etc.). Everything else flows through `params`.
Example `params` (Harper sending Scotty a handoff):
```json
{
"id": "note_2026-05-17_harper_scotty_prod_hardening",
"title": "Prototype ready for production hardening",
"content": "The slack-neo4j bridge is stable. Need your eyes on TLS, systemd, secrets.",
"action_required": true,
"to_tag": "to:scotty"
}
```
Conventions:
- **id** — `note_<YYYY-MM-DD>_<sender>_<recipient>_<short_snake_slug>`. Check
the Time tool for today's date.
- **to_tag** — `to:<recipient>` for a directed message, `to:all` to broadcast.
- **action_required** — `true` when a response is expected, `false` for FYI.
- **Never** use `{placeholder}` syntax in the query body — local models
(Qwen3.5-35B) mishandle it. Pass literal values through `params`.
### Why tag-based `from:` / `to:` (not a `from` property)
The protocol uses tags for both directions (`'from:alan'` AND `'to:jeffrey'`
both live in `n.tags`). This is simpler than splitting into a `from` property
plus a `to:` tag — the local model only has to emit one consistent list,
inbox queries filter on the same array, and there's no second source of truth
to keep in sync.
### Assistant Directory
| Team | Assistants |
|------|-----------|
| **Personal** | shawn, nate, hypatia, marcus, watson, bourdain, david, cousteau, garth, cristiano |
| **Work** | alan, ann, jeffrey, jarvis, aws_sa |
| **Engineering** | scotty, harper |
Watson replaces Seneca (as of 2026-04-28); David replaces Bowie; Shawn is the
personal general assistant (calendar/contacts/email). AWS SA is the work-team
cloud-architecture specialist.
## Graph Error Handling
If a graph query fails, continue the conversation. Mention it briefly and move on. Never expose raw Cypher errors to the user.