Robert Helewka
b4d60f2f38
Replaces the minimal project description with a comprehensive README including a component overview table, quick start instructions, common Ansible operations, and links to detailed documentation. Aligns with Red Panda Approval™ standards.
288 lines
12 KiB
Markdown
288 lines
12 KiB
Markdown
# PostgreSQL - Dual-Deployment Database Layer
|
||
|
||
## Overview
|
||
|
||
PostgreSQL 17 serves as the primary relational database engine for the Agathos sandbox. There are **two separate deployment playbooks**, each targeting a different host with a distinct purpose:
|
||
|
||
| Playbook | Host | Purpose |
|
||
|----------|------|---------|
|
||
| `postgresql/deploy.yml` | **Portia** | Shared multi-tenant database with **pgvector** for AI/vector workloads |
|
||
| `postgresql_ssl/deploy.yml` | **Titania** | Dedicated SSL-enabled database for the **Casdoor** identity provider |
|
||
|
||
**Portia** acts as the central database server for most applications, while **Titania** runs an isolated PostgreSQL instance exclusively for Casdoor, hardened with self-signed SSL certificates for secure external connections.
|
||
|
||
## Architecture
|
||
|
||
```
|
||
┌────────────────────────────────────────────────────┐
|
||
│ Portia (postgresql) │
|
||
┌──────────┐ │ ┌──────────────────────────────────────────────┐ │
|
||
│ Arke │───────────▶│ │ PostgreSQL 17 + pgvector v0.8.0 │ │
|
||
│(Caliban) │ │ │ │ │
|
||
├──────────┤ │ │ Databases: │ │
|
||
│ Gitea │───────────▶│ │ arke ─── openwebui ─── spelunker │ │
|
||
│(Rosalind)│ │ │ gitea ── lobechat ──── nextcloud │ │
|
||
├──────────┤ │ │ anythingllm ────────── hass │ │
|
||
│ Open │───────────▶│ │ │ │
|
||
│ WebUI │ │ │ pgvector enabled in: │ │
|
||
├──────────┤ │ │ arke, lobechat, openwebui, │ │
|
||
│ LobeChat │───────────▶│ │ spelunker, anythingllm │ │
|
||
├──────────┤ │ └──────────────────────────────────────────────┘ │
|
||
│ HASS │───────────▶│ │
|
||
│ + others │ │ PgAdmin available on :25555 │
|
||
└──────────┘ └────────────────────────────────────────────────────┘
|
||
|
||
┌────────────────────────────────────────────────────┐
|
||
│ Titania (postgresql_ssl) │
|
||
┌──────────┐ │ ┌──────────────────────────────────────────────┐ │
|
||
│ Casdoor │──SSL──────▶│ │ PostgreSQL 17 + SSL (self-signed) │ │
|
||
│(Titania) │ (local) │ │ │ │
|
||
└──────────┘ │ │ Database: casdoor (single-purpose) │ │
|
||
│ └──────────────────────────────────────────────┘ │
|
||
└────────────────────────────────────────────────────┘
|
||
```
|
||
|
||
## Terraform Resources
|
||
|
||
### Portia – Shared Database Host
|
||
|
||
Defined in `terraform/containers.tf`:
|
||
|
||
| Attribute | Value |
|
||
|-----------|-------|
|
||
| Image | noble |
|
||
| Role | database |
|
||
| Security Nesting | false |
|
||
| Proxy Devices | `25555 → 80` (PgAdmin web UI) |
|
||
|
||
PostgreSQL port 5432 is **not** exposed externally—applications connect over the private Incus network (`10.10.0.0/16`).
|
||
|
||
### Titania – Proxy & SSO Host
|
||
|
||
| Attribute | Value |
|
||
|-----------|-------|
|
||
| Image | noble |
|
||
| Role | proxy_sso |
|
||
| Security Nesting | true |
|
||
| Proxy Devices | `443 → 8443`, `80 → 8080` (HAProxy) |
|
||
|
||
Titania runs PostgreSQL alongside Casdoor on the same host. Casdoor connects via localhost, so SSL is not required for the local connection despite being available for external clients.
|
||
|
||
## Ansible Deployment
|
||
|
||
### Playbook 1: Shared PostgreSQL with pgvector (Portia)
|
||
|
||
```bash
|
||
cd ansible
|
||
ansible-playbook postgresql/deploy.yml
|
||
```
|
||
|
||
#### Files
|
||
|
||
| File | Purpose |
|
||
|------|---------|
|
||
| `postgresql/deploy.yml` | Multi-tenant PostgreSQL with pgvector |
|
||
|
||
#### Deployment Steps
|
||
|
||
1. **Install build dependencies** — `curl`, `git`, `build-essential`, `vim`, `python3-psycopg2`
|
||
2. **Add PGDG repository** — Official PostgreSQL APT repository
|
||
3. **Install PostgreSQL 17** — Client, server, docs, `libpq-dev`, `server-dev`
|
||
4. **Clone & build pgvector v0.8.0** — Compiled from source against the installed PG version
|
||
5. **Start PostgreSQL** and restart after pgvector installation
|
||
6. **Set data directory permissions** — `700` owned by `postgres:postgres`
|
||
7. **Configure networking** — `listen_addresses = '*'`
|
||
8. **Configure authentication** — `host all all 0.0.0.0/0 md5` in `pg_hba.conf`
|
||
9. **Set admin password** — `postgres` superuser password from vault
|
||
10. **Create application users** — 9 database users (see table below)
|
||
11. **Create application databases** — 9 databases with matching owners
|
||
12. **Enable pgvector** — `CREATE EXTENSION vector` in 5 databases
|
||
|
||
### Playbook 2: SSL-Enabled PostgreSQL (Titania)
|
||
|
||
```bash
|
||
cd ansible
|
||
ansible-playbook postgresql_ssl/deploy.yml
|
||
```
|
||
|
||
#### Files
|
||
|
||
| File | Purpose |
|
||
|------|---------|
|
||
| `postgresql_ssl/deploy.yml` | Single-purpose SSL PostgreSQL for Casdoor |
|
||
|
||
#### Deployment Steps
|
||
|
||
1. **Install dependencies** — `curl`, `python3-psycopg2`, `python3-cryptography`
|
||
2. **Add PGDG repository** — Official PostgreSQL APT repository
|
||
3. **Install PostgreSQL 17** — Client and server only (no dev packages needed)
|
||
4. **Generate SSL certificates** — 4096-bit RSA key, self-signed, 10-year validity
|
||
5. **Configure networking** — `listen_addresses = '*'`
|
||
6. **Enable SSL** — `ssl = on` with cert/key file paths
|
||
7. **Configure tiered authentication** in `pg_hba.conf`:
|
||
- `local` → `peer` (Unix socket, no password)
|
||
- `host 127.0.0.1/32` → `md5` (localhost, no SSL)
|
||
- `host 10.10.0.0/16` → `md5` (Incus network, no SSL)
|
||
- `hostssl 0.0.0.0/0` → `md5` (external, SSL required)
|
||
8. **Set admin password** — `postgres` superuser password from vault
|
||
9. **Create Casdoor user and database** — Single-purpose
|
||
|
||
## User & Database Creation via Host Variables
|
||
|
||
Both playbooks derive all database names, usernames, and passwords from **host variables** defined in the Ansible inventory. No database credentials appear in `group_vars`—everything is scoped to the host that runs PostgreSQL.
|
||
|
||
### Portia Host Variables (`inventory/host_vars/portia.incus.yml`)
|
||
|
||
The `postgresql/deploy.yml` playbook loops over variable pairs to create users and databases. Each application gets three variables defined in Portia's host_vars:
|
||
|
||
| Variable Pattern | Example | Description |
|
||
|-----------------|---------|-------------|
|
||
| `{app}_db_name` | `arke_db_name: arke` | Database name |
|
||
| `{app}_db_user` | `arke_db_user: arke` | Database owner/user |
|
||
| `{app}_db_password` | `arke_db_password: "{{ vault_arke_db_password }}"` | Password (from vault) |
|
||
|
||
#### Application Database Matrix (Portia)
|
||
|
||
| Application | DB Name Variable | DB User Variable | pgvector |
|
||
|-------------|-----------------|-----------------|----------|
|
||
| Arke | `arke_db_name` | `arke_db_user` | ✔ |
|
||
| Open WebUI | `openwebui_db_name` | `openwebui_db_user` | ✔ |
|
||
| Spelunker | `spelunker_db_name` | `spelunker_db_user` | ✔ |
|
||
| Gitea | `gitea_db_name` | `gitea_db_user` | |
|
||
| LobeChat | `lobechat_db_name` | `lobechat_db_user` | ✔ |
|
||
| Nextcloud | `nextcloud_db_name` | `nextcloud_db_user` | |
|
||
| AnythingLLM | `anythingllm_db_name` | `anythingllm_db_user` | ✔ |
|
||
| HASS | `hass_db_name` | `hass_db_user` | |
|
||
| Nike | `nike_db_name` | `nike_db_user` | |
|
||
|
||
#### Additional Portia Variables
|
||
|
||
| Variable | Description |
|
||
|----------|-------------|
|
||
| `postgres_user` | System user (`postgres`) |
|
||
| `postgres_group` | System group (`postgres`) |
|
||
| `postgresql_port` | Port (`5432`) |
|
||
| `postgresql_data_dir` | Data directory (`/var/lib/postgresql`) |
|
||
| `postgres_password` | Admin password (`{{ vault_postgres_password }}`) |
|
||
|
||
### Titania Host Variables (`inventory/host_vars/titania.incus.yml`)
|
||
|
||
The `postgresql_ssl/deploy.yml` playbook creates a single database for Casdoor:
|
||
|
||
| Variable | Value | Description |
|
||
|----------|-------|-------------|
|
||
| `postgresql_ssl_postgres_password` | `{{ vault_postgresql_ssl_postgres_password }}` | Admin password |
|
||
| `postgresql_ssl_port` | `5432` | PostgreSQL port |
|
||
| `postgresql_ssl_cert_path` | `/etc/postgresql/17/main/ssl/server.crt` | SSL certificate |
|
||
| `casdoor_db_name` | `casdoor` | Database name |
|
||
| `casdoor_db_user` | `casdoor` | Database user |
|
||
| `casdoor_db_password` | `{{ vault_casdoor_db_password }}` | Password (from vault) |
|
||
| `casdoor_db_sslmode` | `disable` | Local connection skips SSL |
|
||
|
||
### Adding a New Application Database
|
||
|
||
To add a new application database on Portia:
|
||
|
||
1. **Add variables** to `inventory/host_vars/portia.incus.yml`:
|
||
```yaml
|
||
myapp_db_name: myapp
|
||
myapp_db_user: myapp
|
||
myapp_db_password: "{{ vault_myapp_db_password }}"
|
||
```
|
||
|
||
2. **Add the vault secret** to `inventory/group_vars/all/vault.yml`:
|
||
```yaml
|
||
vault_myapp_db_password: "s3cure-passw0rd"
|
||
```
|
||
|
||
3. **Add the user** to the `Create application database users` loop in `postgresql/deploy.yml`:
|
||
```yaml
|
||
- { user: "{{ myapp_db_user }}", password: "{{ myapp_db_password }}" }
|
||
```
|
||
|
||
4. **Add the database** to the `Create application databases with owners` loop:
|
||
```yaml
|
||
- { name: "{{ myapp_db_name }}", owner: "{{ myapp_db_user }}" }
|
||
```
|
||
|
||
5. **(Optional)** If the application uses vector embeddings, add the database to the `Enable pgvector extension in databases` loop:
|
||
```yaml
|
||
- "{{ myapp_db_name }}"
|
||
```
|
||
|
||
## Operations
|
||
|
||
### Start/Stop
|
||
|
||
```bash
|
||
# On either host
|
||
sudo systemctl start postgresql
|
||
sudo systemctl stop postgresql
|
||
sudo systemctl restart postgresql
|
||
```
|
||
|
||
### Health Check
|
||
|
||
```bash
|
||
# From any Incus host → Portia
|
||
psql -h portia.incus -U postgres -c "SELECT 1;"
|
||
|
||
# From Titania localhost
|
||
sudo -u postgres psql -c "SELECT 1;"
|
||
|
||
# Check pgvector availability
|
||
sudo -u postgres psql -c "SELECT * FROM pg_available_extensions WHERE name = 'vector';"
|
||
```
|
||
|
||
### Logs
|
||
|
||
```bash
|
||
# Systemd journal
|
||
journalctl -u postgresql -f
|
||
|
||
# PostgreSQL log files
|
||
tail -f /var/log/postgresql/postgresql-17-main.log
|
||
|
||
# Loki (via Grafana Explore)
|
||
{job="postgresql"}
|
||
```
|
||
|
||
### Backup
|
||
|
||
```bash
|
||
# Dump a single database
|
||
sudo -u postgres pg_dump myapp > myapp_backup.sql
|
||
|
||
# Dump all databases
|
||
sudo -u postgres pg_dumpall > full_backup.sql
|
||
```
|
||
|
||
### Restore
|
||
|
||
```bash
|
||
# Restore a single database
|
||
sudo -u postgres psql myapp < myapp_backup.sql
|
||
|
||
# Restore all databases
|
||
sudo -u postgres psql < full_backup.sql
|
||
```
|
||
|
||
## Troubleshooting
|
||
|
||
### Common Issues
|
||
|
||
| Symptom | Cause | Resolution |
|
||
|---------|-------|------------|
|
||
| Connection refused from app host | `pg_hba.conf` missing entry | Verify client IP is covered by HBA rules |
|
||
| pgvector extension not found | Built against wrong PG version | Re-run the `Build pgvector with correct pg_config` task |
|
||
| SSL handshake failure (Titania) | Expired or missing certificate | Check `/etc/postgresql/17/main/ssl/server.crt` validity |
|
||
| `FATAL: password authentication failed` | Wrong password in host_vars | Verify vault variable matches and re-run playbook |
|
||
| PgAdmin unreachable on :25555 | Incus proxy device missing | Check `terraform/containers.tf` proxy for Portia |
|
||
|
||
## References
|
||
|
||
- [PostgreSQL 17 Documentation](https://www.postgresql.org/docs/17/)
|
||
- [pgvector GitHub](https://github.com/pgvector/pgvector)
|
||
- [Terraform Practices](terraform.md)
|
||
- [Ansible Practices](ansible.md)
|