ouranos/docs/jupyterlab.md

JupyterLab - Interactive Computing Environment

Overview

JupyterLab is a web-based interactive development environment for notebooks, code, and data. Deployed on Puck as a systemd service running in a Python virtual environment, with OAuth2-Proxy sidecar providing Casdoor SSO authentication.

Host: puck.incus
Role: Application Runtime (Python App Host)
Container Port: 22181 (JupyterLab), 22182 (OAuth2-Proxy)
External Access: https://jupyter.ouranos.helu.ca/ (via HAProxy on Titania)

Architecture

┌──────────┐      ┌────────────┐      ┌─────────────┐      ┌────────────┐
│  Client  │─────▶│  HAProxy   │─────▶│ OAuth2-Proxy│─────▶│ JupyterLab │
│          │      │ (Titania)  │      │   (Puck)    │      │   (Puck)   │
└──────────┘      └────────────┘      └─────────────┘      └────────────┘
                                            │
                                            ▼
                                      ┌───────────┐
                                      │  Casdoor  │
                                      │ (Titania) │
                                      └───────────┘

Authentication Flow

┌──────────┐      ┌────────────┐      ┌─────────────┐      ┌──────────┐
│  Browser │─────▶│  HAProxy   │─────▶│ OAuth2-Proxy│─────▶│ Casdoor  │
│          │      │ (Titania)  │      │   (Puck)    │      │(Titania) │
└──────────┘      └────────────┘      └─────────────┘      └──────────┘
     │                                      │                   │
     │  1. Access jupyter.ouranos.helu.ca   │                   │
     │─────────────────────────────────────▶│                   │
     │  2. No session - redirect to Casdoor │                   │
     │◀─────────────────────────────────────│                   │
     │  3. User authenticates               │                   │
     │─────────────────────────────────────────────────────────▶│
     │  4. Redirect with auth code          │                   │
     │◀─────────────────────────────────────────────────────────│
     │  5. Exchange code, set session cookie│                   │
     │◀─────────────────────────────────────│                   │
     │  6. Proxy to JupyterLab              │                   │
     │◀─────────────────────────────────────│                   │

Deployment

Playbook

cd ansible
ansible-playbook jupyterlab/deploy.yml

Files

File	Purpose
jupyterlab/deploy.yml	Main deployment playbook
jupyterlab/jupyterlab.service.j2	Systemd unit for JupyterLab
jupyterlab/oauth2-proxy-jupyter.service.j2	Systemd unit for OAuth2-Proxy sidecar
jupyterlab/oauth2-proxy-jupyter.cfg.j2	OAuth2-Proxy configuration
jupyterlab/jupyter_lab_config.py.j2	JupyterLab server configuration

Deployment Steps

1. Install Dependencies: python3-venv, nodejs, npm, graphviz
2. Ensure User Exists: robert:robert with home directory
3. Create Directories: Notebooks dir, config dir, log dir
4. Create Virtual Environment: /home/robert/env/jupyter
5. Install Python Packages: jupyterlab, jupyter-ai, langchain-ollama, matplotlib, plotly
6. Install Jupyter Extensions: contrib nbextensions
7. Template Configuration: Apply JupyterLab config
8. Download OAuth2-Proxy: Binary from GitHub releases
9. Template OAuth2-Proxy Config: With Casdoor OIDC settings
10. Start Services: Enable and start both systemd units

Configuration

Key Features

• Jupyter AI: AI assistance via jupyter-ai[all] with LangChain Ollama integration
• Visualization: matplotlib, plotly for data visualization
• Diagrams: Mermaid support via jupyterlab-mermaid
• Extensions: Jupyter contrib nbextensions
• SSO: Casdoor authentication via OAuth2-Proxy sidecar
• WebSocket: Full WebSocket support through reverse proxy

Storage Locations

Path	Purpose	Owner
/home/robert/Notebooks	Notebook files	robert:robert
/home/robert/env/jupyter	Python virtual environment	robert:robert
/etc/jupyterlab	Configuration files	root:robert
/var/log/jupyterlab	Application logs	robert:robert
/etc/oauth2-proxy-jupyter	OAuth2-Proxy config	root:root

Installed Python Packages

Package	Purpose
jupyterlab	Core JupyterLab server
jupyter-ai[all]	AI assistant integration
langchain-ollama	Ollama LLM integration
matplotlib	Data visualization
plotly	Interactive charts
jupyter_contrib_nbextensions	Community extensions
jupyterlab-mermaid	Mermaid diagram support
ipywidgets	Interactive widgets

Logging

• JupyterLab: systemd journal via SyslogIdentifier=jupyterlab
• OAuth2-Proxy: systemd journal via SyslogIdentifier=oauth2-proxy-jupyter
• Alloy Forwarding: Syslog port 51491 → Loki

Access After Deployment

1. Web Interface: https://jupyter.ouranos.helu.ca/
2. Authentication: Redirects to Casdoor SSO login
3. After Login: Full JupyterLab interface with notebook access

Monitoring

Alloy Configuration

File: ansible/alloy/puck/config.alloy.j2

• Log Collection: Syslog port 51491 → Loki
• Job Label: jupyterlab
• System Metrics: Process exporter tracks JupyterLab process

Health Check

• URL: http://puck.incus:22182/ping (OAuth2-Proxy)
• JupyterLab API: http://127.0.0.1:22181/api/status (localhost only)

Required Vault Secrets

Add to ansible/inventory/group_vars/all/vault.yml:

1. OAuth Client ID

vault_jupyter_oauth_client_id: "jupyter-oauth-client"

Requirements:

• Purpose: Client ID for Casdoor OAuth2 application
• Source: Must match clientId in Casdoor application configuration

2. OAuth Client Secret

vault_jupyter_oauth_client_secret: "YourRandomOAuthSecret123!"

Requirements:

• Length: 32+ characters recommended
• Purpose: Client secret for Casdoor OAuth2 authentication
• Generation:

  openssl rand -base64 32
  

3. Cookie Secret

vault_jupyter_oauth2_cookie_secret: "32CharacterRandomStringHere1234"

Requirements:

• Length: Exactly 32 characters (or 16/24 for AES)
• Purpose: Encrypts OAuth2-Proxy session cookies
• Generation:

  openssl rand -base64 32 | head -c 32
  

Host Variables

File: ansible/inventory/host_vars/puck.incus.yml

# JupyterLab Configuration
jupyterlab_user: robert
jupyterlab_group: robert
jupyterlab_notebook_dir: /home/robert/Notebooks
jupyterlab_venv_dir: /home/robert/env/jupyter

# Ports
jupyterlab_port: 22181           # JupyterLab (localhost only)
jupyterlab_proxy_port: 22182     # OAuth2-Proxy (exposed to HAProxy)

# OAuth2-Proxy Configuration
jupyterlab_oauth2_proxy_dir: /etc/oauth2-proxy-jupyter
jupyterlab_oauth2_proxy_version: "7.6.0"
jupyterlab_domain: "ouranos.helu.ca"
jupyterlab_oauth2_oidc_issuer_url: "https://id.ouranos.helu.ca"
jupyterlab_oauth2_redirect_url: "https://jupyter.ouranos.helu.ca/oauth2/callback"

# OAuth2 Credentials (from vault)
jupyterlab_oauth_client_id: "{{ vault_jupyter_oauth_client_id }}"
jupyterlab_oauth_client_secret: "{{ vault_jupyter_oauth_client_secret }}"
jupyterlab_oauth2_cookie_secret: "{{ vault_jupyter_oauth2_cookie_secret }}"

# Alloy Logging
jupyterlab_syslog_port: 51491

OAuth2 / Casdoor SSO

JupyterLab uses OAuth2-Proxy as a sidecar to handle Casdoor authentication. This pattern is simpler than native OAuth for single-user setups.

Why OAuth2-Proxy Sidecar?

Approach	Pros	Cons
OAuth2-Proxy (chosen)	Simple setup, no JupyterLab modification	Extra service to manage
Native JupyterHub OAuth	Integrated solution	More complex, overkill for single user
Token-only auth	Simplest	Less secure, no SSO integration

Casdoor Application Configuration

A JupyterLab application is defined in ansible/casdoor/init_data.json.j2:

Setting	Value
Name	app-jupyter
Client ID	vault_jupyter_oauth_client_id
Redirect URI	https://jupyter.ouranos.helu.ca/oauth2/callback
Grant Types	authorization_code, refresh_token

URL Strategy

URL Type	Address	Used By
OIDC Issuer	https://id.ouranos.helu.ca	OAuth2-Proxy (external)
Redirect URL	https://jupyter.ouranos.helu.ca/oauth2/callback	Browser callback
Upstream	http://127.0.0.1:22181	OAuth2-Proxy → JupyterLab

Deployment Order

1. Deploy Casdoor first (if not already running):

   ansible-playbook casdoor/deploy.yml
   

2. Update HAProxy (add jupyter backend):

   ansible-playbook haproxy/deploy.yml
   

3. Deploy JupyterLab:

   ansible-playbook jupyterlab/deploy.yml
   

4. Update Alloy (for log forwarding):

   ansible-playbook alloy/deploy.yml
   

Integration with Other Services

HAProxy Routing

Backend Configuration (titania.incus.yml):

- subdomain: "jupyter"
  backend_host: "puck.incus"
  backend_port: 22182  # OAuth2-Proxy port
  health_path: "/ping"
  timeout_server: 300s  # WebSocket support

Alloy Log Forwarding

Syslog Configuration (puck/config.alloy.j2):

loki.source.syslog "jupyterlab_logs" {
  listener {
    address  = "127.0.0.1:51491"
    protocol = "tcp"
    labels = {
      job = "jupyterlab",
    }
  }
  forward_to = [loki.write.default.receiver]
}

Troubleshooting

Service Status

ssh puck.incus
sudo systemctl status jupyterlab
sudo systemctl status oauth2-proxy-jupyter

View Logs

# JupyterLab logs
sudo journalctl -u jupyterlab -f

# OAuth2-Proxy logs
sudo journalctl -u oauth2-proxy-jupyter -f

Test JupyterLab Directly (bypass OAuth)

# From puck container
curl http://127.0.0.1:22181/api/status

Test OAuth2-Proxy Health

curl http://puck.incus:22182/ping

Verify Virtual Environment

ssh puck.incus
sudo -u robert /home/robert/env/jupyter/bin/jupyter --version

Common Issues

Issue	Solution
WebSocket disconnects	Verify timeout_server: 300s in HAProxy backend
OAuth redirect loop	Check redirect_url matches Casdoor app config
502 Bad Gateway	Ensure JupyterLab service is running on port 22181
Cookie errors	Verify cookie_secret is exactly 32 characters

Version Information

• Installation Method: Python pip in virtual environment
• JupyterLab Version: Latest stable (pip managed)
• OAuth2-Proxy Version: 7.6.0 (binary from GitHub)
• Update Process: Re-run deployment playbook

References

• JupyterLab Documentation: https://jupyterlab.readthedocs.io/
• OAuth2-Proxy Documentation: https://oauth2-proxy.github.io/oauth2-proxy/
• Jupyter AI: https://jupyter-ai.readthedocs.io/
• Casdoor OIDC: https://casdoor.org/docs/integration/oidc