mnemosyne/docs/Red Panda Standards_Django_V1-02.md

🐾 Red Panda Approval™

This project follows Red Panda Approval standards — our gold standard for Django application quality. Code must be elegant, reliable, and maintainable to earn the approval of our adorable red panda judges.

The 5 Sacred Django Criteria

1. Fresh Migration Test — Clean migrations from empty database
2. Elegant Simplicity — No unnecessary complexity
3. Observable & Debuggable — Proper logging and error handling
4. Consistent Patterns — Follow Django conventions
5. Actually Works — Passes all checks and serves real user needs

Changelog

• V1-02 — Added Docker Compose environment-scoping standard (per-service environment: blocks), nginx reverse-proxy reference config (Docker DNS resolver, X-Forwarded-Proto preservation, access-log filtering, internal-network allowlists), and Memcached deployment note (bind to 0.0.0.0, not localhost).
• V1-01 — Initial published standards.

Environment Standards

• Virtual environment: ~/env/PROJECT/bin/activate
• Use pyproject.toml for project configuration (no setup.py, no requirements.txt)
• Python version: specified in pyproject.toml
• Dependencies: floor-pinned with ceiling (e.g. Django>=5.2,<6.0)

Dependency Pinning

# Correct — floor pin with ceiling
dependencies = [
    "Django>=5.2,<6.0",
    "djangorestframework>=3.14,<4.0",
    "cryptography>=41.0,<45.0",
]

# Wrong — exact pins in library packages
dependencies = [
    "Django==5.2.7",  # too strict, breaks downstream
]

Exact pins (==) are only appropriate in application-level lock files, not in reusable library packages.

Directory Structure

myproject/ # Git repository root ├── .gitignore ├── README.md ├── pyproject.toml # Project configuration (moved to repo root) ├── docker-compose.yaml # Per-service environment scoping (see below) ├── .env # Docker Compose interpolation source — NOT committed ├── .env.example # Template listing every ${VAR} with which service consumes it │ ├── project/ # Django project root (manage.py lives here) │ ├── manage.py │ ├── Dockerfile │ ├── .env # Local bare-Python dev environment (runserver, celery, etc.) │ │ # Only read by bare-Python runs; NOT by the compose stack │ ├── .env.example │ │ │ ├── config/ # Django configuration module │ │ ├── init.py │ │ ├── settings.py │ │ ├── urls.py │ │ ├── wsgi.py │ │ └── asgi.py │ │ │ ├── accounts/ # Django app │ │ ├── init.py │ │ ├── models.py │ │ ├── views.py │ │ └── urls.py │ │ │ ├── blog/ # Django app │ │ ├── init.py │ │ ├── models.py │ │ ├── views.py │ │ └── urls.py │ │ │ ├── static/ │ │ ├── css/ │ │ └── js/ │ │ │ └── templates/ │ └── base.html │ ├── nginx/ # Nginx configuration (see Nginx Reverse Proxy below) │ └── PROJECT.conf │ ├── db/ # PostgreSQL configuration (if customised) │ └── postgresql.conf │ └── docs/ # Project documentation └── index.md

Settings Structure

• Use a single settings.py file
• Use django-environ or python-dotenv for environment variables
• Never commit .env files to version control
• Provide .env.example with all required variables documented
• Create .gitignore file
• Create a .dockerignore file

Environment Variables

PostgreSQL settings (only if DB_ENGINE=postgresql)

APP_DB_NAME=angelia2
APP_DB_USER=angelia
APP_DB_PASSWORD=changeme
DB_HOST=db
DB_PORT=5432

Docker Compose — Per-Service Environment Scoping

New in V1-02. The monolithic env_file: pattern is deprecated.

Rule: every service declares only the env vars it actually needs

In docker-compose.yaml, each service uses an environment: block listing only the variables that service consumes, with values interpolated from .env (at the repo root) using ${VAR} syntax. Do not use env_file: .env shared across services.

services:
  app:
    image: git.helu.ca/r/myproject:latest
    command: ["web"]
    environment:
      # Django core
      - DJANGO_SETTINGS_MODULE=myproject.settings
      - SECRET_KEY=${SECRET_KEY}
      - DEBUG=${DEBUG}
      - ALLOWED_HOSTS=${ALLOWED_HOSTS}
      - CSRF_TRUSTED_ORIGINS=${CSRF_TRUSTED_ORIGINS}
      # Postgres
      - APP_DB_NAME=${APP_DB_NAME}
      - APP_DB_USER=${APP_DB_USER}
      - APP_DB_PASSWORD=${APP_DB_PASSWORD}
      - DB_HOST=${DB_HOST}
      - DB_PORT=${DB_PORT}
      # ...

  worker:
    image: git.helu.ca/r/myproject:latest
    command: ["worker"]
    environment:
      - DJANGO_SETTINGS_MODULE=myproject.settings
      - SECRET_KEY=${SECRET_KEY}
      - APP_DB_NAME=${APP_DB_NAME}
      # ...
      # NO ALLOWED_HOSTS, CSRF_TRUSTED_ORIGINS, EMAIL_* — worker doesn't serve HTTP

Why this matters

1. Least privilege / blast radius. A compromised MCP container shouldn't see Celery broker credentials or encryption keys. A Celery worker shouldn't see ALLOWED_HOSTS or CSRF config — it doesn't serve HTTP. When every service shares one env file, a misconfigured secret takes down the whole stack instead of just the services that need that secret.

2. Self-documenting surface. Reading docker-compose.yaml immediately tells you what each container depends on. With env_file:, every container has access to every secret and you can't tell from the compose file which service uses which variable.

3. Ansible rendering. The compose file can be converted to a Jinja2 template and rendered per-host by an Ansible role, with secrets pulled from the vault. The ${VAR} pattern is the natural interface between compose and Ansible.

4. Parsing correctness. Docker Compose's env_file: parser does not strip inline # comments, honours CRLF \r as part of values, and handles quoting differently than python-decouple/django-environ. An .env that works with bare-Python manage.py runserver can silently feed a mangled URL (e.g. CELERY_BROKER_URL with a trailing \r or stray comment) to a container. Shell-style ${VAR} interpolation avoids this because the value is unescaped by the shell the same way every time.

.env.example template convention

Annotate each variable with which service(s) consume it:

# --- Django core ------------------------------------------------------------
# Consumed by: app, mcp, worker
SECRET_KEY=change-me-to-a-real-secret-key
DEBUG=False

# --- PostgreSQL ------------------------------------------------------------
# Consumed by: app, mcp, worker
APP_DB_NAME=myproject
APP_DB_USER=myproject
APP_DB_PASSWORD=change-me

# --- Celery / RabbitMQ -----------------------------------------------------
# Consumed by: app (producer), worker (consumer). NOT mcp.
# Percent-encode any password chars with URL meaning: @ : / # % + ? & = and space
CELERY_BROKER_URL=amqp://myproject:change-me@oberon.incus:5672/myproject

Diagnostic: "what did Django actually parse?"

When a service misbehaves on startup (broker 403, DB auth error, unreachable cache), the fastest diagnostic is to print what settings.py actually resolved to — that removes every layer of env-file / interpolation / URL-encoding ambiguity:

docker compose run --rm --no-deps worker \
    python -c "from django.conf import settings; print(repr(settings.CELERY_BROKER_URL))"

docker compose run --rm --no-deps app \
    python -c "from django.conf import settings; print(settings.DATABASES['default'])"

The repr(...) form surfaces CRLF, trailing whitespace, stray quotes, and characters that should have been percent-encoded but weren't.

Broker URL gotcha (documented for every new project)

RabbitMQ connection URLs must percent-encode any password character with URL meaning (@ : / # % + ? & = and space). Kombu's URL parser is strict — an unencoded # in the password is read as the start of a URL fragment, and an unencoded @ shifts the username/host boundary, both causing ACCESS_REFUSED - Login was refused using authentication mechanism PLAIN at worker startup. Bare-Python tests that pass the password as a kwarg rather than a URL won't exhibit this and can mask the bug.

Nginx Reverse Proxy

New in V1-02. Standard reference config for any Red Panda project running behind HAProxy on Titania.

Deployed as a sidecar container in the compose stack, fronting the Django app (gunicorn) and — where applicable — an MCP or streaming service. HAProxy handles TLS termination; nginx is plain HTTP on the internal network.

Required building blocks

1. Docker DNS resolver + variable-based proxy_pass. upstream blocks resolve container hostnames once at startup and cache the IP forever. When docker compose restart app assigns a new IP, nginx returns 502 until fully reloaded. Use:

   resolver 127.0.0.11 valid=10s;
   server {
       set $backend_app http://app:8000;
       location / {
           proxy_pass $backend_app;   # variable → re-resolve each request
       }
   }
   

2. $proxy_x_forwarded_proto map. Inside the compose network $scheme is always http because HAProxy already terminated TLS. Passing $scheme to Django breaks request.is_secure(), secure cookies, and build_absolute_uri(). Preserve the HAProxy header:

   map $http_x_forwarded_proto $proxy_x_forwarded_proto {
       default $http_x_forwarded_proto;
       ""      $scheme;
   }
   # Then in every proxy block:
   proxy_set_header X-Forwarded-Proto $proxy_x_forwarded_proto;
   

3. Access-log suppression for probe paths. HAProxy and Prometheus probe every 15–30 s; logging them floods Loki. The nginx:alpine image has a default http-level access_log, so a server-level access_log is additive, not replacing. You need both:

   map $request_uri $loggable {
       default                     1;
       ~^/live(/|\?|$)             0;
       ~^/ready(/|\?|$)            0;
       ~^/metrics(/|\?|$)          0;
       ~^/healthz(/|\?|$)          0;
       ~^/nginx_status(/|\?|$)     0;
       ~^/mcp/health(/|\?|$)       0;
   }
   
   access_log off;                                   # defeat inherited default
   access_log /dev/stdout combined if=$loggable;     # then install filtered version
   

4. Internal-network allowlist for all probe + metric endpoints. RFC1918 + loopback, applied to /live/, /ready/, /healthz, /metrics, and /nginx_status:

   location = /metrics {
       allow 127.0.0.0/8;     # loopback
       allow 10.0.0.0/8;      # RFC1918 — primary internal (Incus, HAProxy)
       allow 172.16.0.0/12;   # RFC1918 — Docker bridge networks
       allow 192.168.0.0/16;  # RFC1918
       deny  all;
       proxy_pass $backend_app;
       # ...
   }
   

   All four RFC1918 ranges must be present — omitting 172.16.0.0/12 silently breaks scrapes from a Prometheus container on the default Docker bridge.

5. Security headers on the catch-all, marked always so they apply to upstream 4xx/5xx:

   add_header X-Frame-Options        "SAMEORIGIN" always;
   add_header X-Content-Type-Options "nosniff"    always;
   add_header X-XSS-Protection       "1; mode=block" always;
   

   Stronger policies (CSP, Referrer-Policy, HSTS) are set at HAProxy so they're consistent across every backend.

6. Catch-all proxies to Django. nginx should intercept only the paths that need special handling (/static/, /media/, /mcp/, /healthz, /metrics, /nginx_status, the probes). Everything else flows through to Django, which returns its own themed 404 for unrouted paths — not nginx's bare default page.

Minimal reference config

resolver 127.0.0.11 valid=10s;

map $http_x_forwarded_proto $proxy_x_forwarded_proto {
    default $http_x_forwarded_proto;
    ""      $scheme;
}

map $request_uri $loggable {
    default                     1;
    ~^/live(/|\?|$)             0;
    ~^/ready(/|\?|$)            0;
    ~^/metrics(/|\?|$)          0;
    ~^/healthz(/|\?|$)          0;
    ~^/nginx_status(/|\?|$)     0;
}

access_log off;
access_log /dev/stdout combined if=$loggable;

server {
    listen 80 default_server;
    server_name _;

    client_max_body_size 64m;

    set $backend_app http://app:8000;

    add_header X-Frame-Options        "SAMEORIGIN" always;
    add_header X-Content-Type-Options "nosniff"    always;
    add_header X-XSS-Protection       "1; mode=block" always;

    location /static/ {
        alias /var/www/static/;
        access_log off;
        expires 30d;
        add_header Cache-Control "public, immutable";
    }

    location /media/ {
        alias /var/www/media/;
        access_log off;
        expires 7d;
    }

    # Internal-only endpoints — allowlist applied uniformly
    location = /live/      { include /etc/nginx/snippets/internal-only.conf; proxy_pass $backend_app; include /etc/nginx/snippets/proxy-headers.conf; access_log off; }
    location = /ready/     { include /etc/nginx/snippets/internal-only.conf; proxy_pass $backend_app; include /etc/nginx/snippets/proxy-headers.conf; access_log off; }
    location = /metrics    { include /etc/nginx/snippets/internal-only.conf; proxy_pass $backend_app; include /etc/nginx/snippets/proxy-headers.conf; access_log off; }
    location = /nginx_status {
        include /etc/nginx/snippets/internal-only.conf;
        stub_status on;
        access_log off;
    }

    location / {
        proxy_pass $backend_app;
        proxy_http_version 1.1;
        proxy_set_header Host              $host;
        proxy_set_header X-Real-IP         $remote_addr;
        proxy_set_header X-Forwarded-For   $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $proxy_x_forwarded_proto;
        proxy_redirect    off;
        proxy_read_timeout 300s;
    }
}

Projects may inline the proxy_set_header block rather than using snippets; both are acceptable. The important thing is that every proxy_pass has the same four headers (Host, X-Real-IP, X-Forwarded-For, X-Forwarded-Proto using $proxy_x_forwarded_proto).

Memcached

New in V1-02. Memcached is a standard Red Panda dependency. Every Django service uses it for session storage, task-progress tracking, and cheap key-value caching.

• Package: pymemcache (via django.core.cache.backends.memcached.PyMemcacheCache)
• Key prefix: per-app, configured via env var (e.g. KVDB_PREFIX=mnemosyne)
• Cache-key pattern: {app}:{model}:{identifier}:{field}

Deployment convention

Memcached runs as a service on the application host (in Ouranos: a package install per Incus container). Configure it to bind to all interfaces, not just localhost, so:

• Containers on the same host can reach it via the host's LAN name (e.g. puck.incus:11211).
• Other hosts in the lab can reach it for multi-host debugging.

# /etc/memcached.conf on the Docker host
-l 0.0.0.0
-p 11211
-U 0

The bare-Python "localhost:11211 works" default is a dev-only convenience — it breaks as soon as Django moves into a container, because inside the container 127.0.0.1 is the container itself. The production .env must use the LAN-resolvable hostname:

KVDB_LOCATION=puck.incus:11211
KVDB_PREFIX=myproject

Health-check reachability

The Django readiness probe (GET /ready/) must verify Memcached is reachable. If the probe returns 503 and the log shows no cause, hit the endpoint directly to read the JSON body which names the broken dependency:

docker compose exec app curl -sS -o - -w "\nHTTP %{http_code}\n" http://localhost:8000/ready/

Code Organization

• Imports: PEP 8 ordering (stdlib, third-party, local)
• Type hints on function parameters
• CSS: External .css files only (no inline styles, no embedded <style> tags)
• JS: External .js files only (no inline handlers, no embedded <script> blocks)
• Maximum file length: 1000 lines
• If a file exceeds 500 lines, consider splitting by domain concept

Database Conventions

• Migrations run cleanly from empty database
• Never edit deployed migrations
• Use meaningful migration names: --name add_email_to_profile
• One logical change per migration when possible
• Test migrations both forward and backward

Development vs Production

• Development: SQLite
• Production: PostgreSQL

Caching

• Expensive queries are cached
• Cache keys follow naming convention
• TTLs are appropriate (not infinite)
• Invalidation is documented
• Key Naming Pattern: {app}:{model}:{identifier}:{field}
• See the Memcached section above for deployment details

Model Naming

• Model names: singular PascalCase (User, BlogPost, OrderItem)
• Correct English pluralization on related names
• All models have created_at and updated_at
• All models define str and get_absolute_url
• TextChoices used for status fields
• related_name defined on ForeignKey fields
• Related names: plural snake_case with proper English pluralization

Forms

• Use ModelForm with explicit fields list (never all)

Field Naming

• Foreign keys: singular without _id suffix (author, category, parent)
• Boolean fields: use prefixes (is_active, has_permission, can_edit)
• Date fields: use suffixes (created_at, updated_at, published_on)
• Avoid abbreviations (use description, not desc)

Required Model Fields

• All models should include:
  • created_at = models.DateTimeField(auto_now_add=True)
  • updated_at = models.DateTimeField(auto_now=True)
• Consider adding:
  • id = models.UUIDField(primary_key=True) for public-facing models
  • is_active = models.BooleanField(default=True) for soft deletes

Indexing

• Add db_index=True to frequently queried fields
• Use Meta.indexes for composite indexes
• Document why each index exists

Queries

• Use select_related() for foreign keys
• Use prefetch_related() for reverse relations and M2M
• Avoid queries in loops (N+1 problem)
• Use .only() and .defer() for large models
• Add comments explaining complex querysets

Docstrings

• Use Sphinx style docstrings
• Document all public functions, classes, and modules
• Skip docstrings for obvious one-liners and standard Django overrides

Views

• Use Function-Based Views (FBVs) exclusively
• Explicit logic is preferred over implicit inheritance
• Extract shared logic into utility functions

URLs & Identifiers

• Public URLs use short UUIDs (12 characters) via shortuuid
• Never expose sequential IDs in URLs (security/enumeration risk)
• Internal references may use standard UUIDs or PKs

URL Patterns

• Resource-based URLs (RESTful style)
• Namespaced URL names per app
• Trailing slashes (Django default)
• Flat structure preferred over deep nesting

Background Tasks

• All tasks are run synchronously unless the design specifies background tasks are needed for long operations
• Long operations use Celery tasks
• Use Memcached, task progress pattern: {app}:task:{task_id}:progress
• Tasks are idempotent
• Tasks include retry logic
• Tasks live in app/tasks.py
• RabbitMQ is the Message Broker
• Flower Monitoring: Use for debugging failed tasks
• Per-service env scoping: the Celery worker container consumes CELERY_BROKER_URL + LLM_API_SECRETS_ENCRYPTION_KEY + DAEDALUS_S3_* but NOT ALLOWED_HOSTS/CSRF_TRUSTED_ORIGINS/MCP_REQUIRE_AUTH/EMAIL_* (see Docker Compose section)

Testing

• Framework: Django TestCase (not pytest)
• Separate test files per module: test_models.py, test_views.py, test_forms.py

Frontend Standards

New Projects (DaisyUI + Tailwind)

• DaisyUI 4 via CDN for component classes
• Tailwind CSS via CDN for utility classes
• Theme management via Themis (DaisyUI data-theme attribute)
• All apps extend themis/base.html for consistent navigation
• No inline styles or scripts

Existing Projects (Bootstrap 5)

• Bootstrap 5 via CDN
• Bootstrap Icons via CDN
• Bootswatch for theme variants (if applicable)
• django-bootstrap5 and crispy-bootstrap5 for form rendering

Preferred Packages

Core Django

• django>=5.2,<6.0
• django-environ — Environment variables

Authentication & Security

• django-allauth — User management
• django-allauth-2fa — Two-factor authentication

API Development

• djangorestframework>=3.14,<4.0 — REST APIs
• drf-spectacular — OpenAPI/Swagger documentation

Encryption

• cryptography — Fernet encryption for secrets/API keys

Background Tasks

• celery — Async task queue
• django-celery-progress — Progress bars
• flower — Celery monitoring

Caching

• pymemcache — Memcached backend

Database

• psycopg[binary] — PostgreSQL adapter
• shortuuid — Short UUIDs for public URLs

Production

• gunicorn — WSGI server

Shared Apps

• django-heluca-themis — User preferences, themes, key management, navigation

Deprecated / Removed

• pytz — Use stdlib zoneinfo (Python 3.9+, Django 4+)
• Pillow — Only add if your app needs ImageField
• django-heluca-core — Replaced by Themis
• dj-database-url — Use individual Django DB env vars instead

Anti-Patterns to Avoid

Docker Compose

• Don't share a single env_file: across services (see per-service scoping above)
• Don't put secrets in the compose file's environment: block as literals — use ${VAR} interpolation
• Don't commit a populated .env — only .env.example

Nginx

• Don't use upstream blocks for container hostnames without resolver + variable proxy_pass (nginx caches the IP and returns 502 after container restart)
• Don't pass $scheme as X-Forwarded-Proto when behind an external TLS terminator — use $proxy_x_forwarded_proto via the map pattern
• Don't rely on server-level access_log to override the image default — explicitly access_log off; first
• Don't allowlist only 10.0.0.0/8 for /metrics — also include 172.16.0.0/12 for Docker bridge sources

Memcached

• Don't bind to 127.0.0.1 only on a host that runs Docker services — containers can't reach it
• Don't use KVDB_LOCATION=127.0.0.1:11211 in a containerised .env (127.0.0.1 is the container itself)

Models

• Don't use Model.objects.get() without handling DoesNotExist
• Don't use null=True on CharField or TextField (use blank=True, default="")
• Don't use related_name='+' unless you have a specific reason
• Don't override save() for business logic (use signals or service functions)
• Don't use auto_now=True on fields you might need to manually set
• Don't use ForeignKey without specifying on_delete explicitly
• Don't use Meta.ordering on large tables (specify ordering in queries)

Queries

• Don't query inside loops (N+1 problem)
• Don't use .all() when you need a subset
• Don't use raw SQL unless absolutely necessary
• Don't forget select_related() and prefetch_related()

Views

• Don't put business logic in views
• Don't use request.POST.get() without validation (use forms)
• Don't return sensitive data in error messages
• Don't forget login_required decorator on protected views

Forms

• Don't use fields = '__all__' in ModelForm
• Don't trust client-side validation alone
• Don't use exclude in ModelForm (use explicit fields)

Templates

• Don't use {{ variable }} for URLs (use {% url %} tag)
• Don't put logic in templates
• Don't use inline CSS or JavaScript (external files only)
• Don't forget {% csrf_token %} in forms

Security

• Don't store secrets in settings.py (use environment variables)
• Don't commit .env files to version control
• Don't use DEBUG=True in production
• Don't expose sequential IDs in public URLs
• Don't use mark_safe() on user-supplied content
• Don't disable CSRF protection

Imports & Code Style

• Don't use from module import *
• Don't use mutable default arguments
• Don't use bare except: clauses
• Don't ignore linter warnings without documented reason

Migrations

• Don't edit migrations that have been deployed
• Don't use RunPython without a reverse function
• Don't add non-nullable fields without a default value

Celery Tasks

• Don't pass model instances to tasks (pass IDs and re-fetch)
• Don't assume tasks run immediately
• Don't forget retry logic for external service calls
• Don't forget to percent-encode special characters in CELERY_BROKER_URL (@ : / # % + ? & = and space)