stellarshenson a296338db1 docs: reorganize README with architecture diagram and improved structure
- Add mermaid diagram illustrating JupyterHub architecture and user flow
- Move screenshots higher in document under dedicated User Interface section
- Visualize relationships between Traefik, Hub, Spawner, and user containers
- Show per-user volumes and shared storage configuration
- Improve document flow: Architecture -> UI -> Features -> Setup
2025-11-04 10:53:12 +01:00
2025-08-08 20:16:08 +02:00
2025-08-08 20:16:08 +02:00
2025-08-08 20:16:08 +02:00
2025-08-08 20:16:08 +02:00
2025-08-08 20:16:08 +02:00
2025-08-10 21:55:59 +02:00
2025-08-07 17:22:30 +02:00
2025-08-07 17:22:30 +02:00
2025-10-26 11:36:19 +01:00

Stellars JupyterHub for Data Science Platform

Docker Pulls Docker Image

Multi-user JupyterHub 4 with Miniforge, Data Science stack, and NativeAuthenticator.

This platform is built to support multiple data scientists on a shared environment with isolated sessions. Powered by JupyterHub, it ensures secure, user-specific access via the NativeAuthenticator plugin. It includes a full data science stack with GPU support (optional), and integrates seamlessly into modern Docker-based workflows.

By default system is capable of automatically detecting NVIDIA CUDA-supported GPU

This deployment provides access to a centralized JupyterHub instance for managing user sessions. Optional integrations such as TensorBoard, MLFlow, or Optuna can be added manually via service extensions.

Architecture

graph TB
    User[User Browser] -->|HTTPS| Traefik[Traefik Proxy<br/>TLS Termination]
    Traefik --> Hub[JupyterHub<br/>Port 8000]

    Hub -->|Authenticates| Auth[NativeAuthenticator<br/>User Management]
    Hub -->|Spawns via| Spawner[DockerSpawner]

    Spawner -->|Creates| Lab1[JupyterLab<br/>User: alice]
    Spawner -->|Creates| Lab2[JupyterLab<br/>User: bob]
    Spawner -->|Creates| Lab3[JupyterLab<br/>User: charlie]

    Lab1 -->|Mounts| Vol1[alice_home<br/>alice_workspace<br/>alice_cache]
    Lab2 -->|Mounts| Vol2[bob_home<br/>bob_workspace<br/>bob_cache]
    Lab3 -->|Mounts| Vol3[charlie_home<br/>charlie_workspace<br/>charlie_cache]

    Lab1 -->|Shared| Shared[jupyterhub_shared<br/>CIFS/NAS Optional]
    Lab2 -->|Shared| Shared
    Lab3 -->|Shared| Shared

    style Hub fill:#fef3c7,stroke:#f59e0b,stroke-width:3px
    style Traefik fill:#e0f2fe,stroke:#0284c7,stroke-width:3px
    style Auth fill:#d1fae5,stroke:#10b981,stroke-width:3px
    style Spawner fill:#e9d5ff,stroke:#a855f7,stroke-width:3px
    style Lab1 fill:#dbeafe,stroke:#3b82f6,stroke-width:2px
    style Lab2 fill:#dbeafe,stroke:#3b82f6,stroke-width:2px
    style Lab3 fill:#dbeafe,stroke:#3b82f6,stroke-width:2px
    style Shared fill:#fee2e2,stroke:#ef4444,stroke-width:2px

Users access JupyterHub through Traefik reverse proxy with TLS termination. After authentication via NativeAuthenticator, JupyterHub spawns isolated JupyterLab containers per user using DockerSpawner. Each user gets dedicated persistent volumes for home directory, workspace files, and cache data, with optional shared storage for collaborative datasets.

User Interface

Restart Server Restart running JupyterLab container directly from the user control panel

Manage Volumes Access volume management when server is stopped

Volume Selection Select individual volumes to reset - home directory, workspace files, or cache data

Features

  • GPU Auto-Detection: Automatic NVIDIA CUDA GPU detection and configuration for spawned user containers
  • User Self-Service: Users can restart their JupyterLab containers and selectively reset persistent volumes (home/workspace/cache) without admin intervention
  • Isolated Environments: Each user gets dedicated JupyterLab container with persistent volumes via DockerSpawner
  • Native Authentication: Built-in user management with NativeAuthenticator supporting self-registration and admin approval
  • Shared Storage: Optional CIFS/NAS mount support for shared datasets across all users
  • Production Ready: Traefik reverse proxy with TLS termination, automatic container updates via Watchtower

References

This project spawns user environments using docker image: stellars/stellars-jupyterlab-ds

Visit the project page for stellars-jupyterlab-ds: https://github.com/stellarshenson/stellars-jupyterlab-ds

Quickstart

Docker Compose

  1. Download compose.yml and config/jupyterhub_config.py config file
  2. Run: docker compose up --no-build
  3. Open https://localhost/jupyterhub in your browser
  4. Add admin user through self-sign-in (user will be authorised automatically)
  5. Log in as admin

Start Scripts

  • start.sh or start.bat standard startup for the environment
  • scripts/build.sh alternatively make build builds required Docker containers

Authentication

This stack uses NativeAuthenticator for user management. Admins can whitelist users or allow self-registration. Passwords are stored securely.

Deployment Notes

  • Ensure config/jupyterhub_config.py is correctly set for your environment (e.g., TLS, admin list).
  • Optional volume mounts and configuration can be modified in jupyterhub_config.py for shared storage.

Customisation

You should customise the deployment by creating a compose_override.yml file.

Custom configuration file

Example below introduces custom config file jupyterhub_config_override.py to use for your deployment:

services:
  jupyterhub:
    volumes:
      - ./config/jupyterhub_config_override.py:/srv/jupyterhub/jupyterhub_config.py:ro # config file (read only)

Enable GPU

No changes required in the configuration if you allow NVidia autodetection to be performed. Otherwise change the ENABLE_GPU_SUPPORT = 1

Changes in your compose_override.yml:

services:
  jupyterhub:
    environment:
      - ENABLE_GPU_SUPPORT=1 # enable NVIDIA GPU, values: 0 - disabled, 1 - enabled, 2 - auto-detect

Enable shared CIFS mount

Changes in your compose_override.yml:

  jupyterhub:
    volumes:
      - ./config/jupyterhub_config_override.py:/srv/jupyterhub/jupyterhub_config.py:ro # config file (read only)
      - jupyterhub_shared_nas:/mnt/shared # cifs share
    
volumes:
  # remote drive for large datasets
  jupyterhub_shared_nas:
    driver: local
    name: jupyterhub_shared_nas
    driver_opts:
      type: cifs
      device: //nas_ip_or_dns_name/data
      o: username=xxxx,password=yyyy,uid=1000,gid=1000

in the config file you will refer to this volume by its name jupyterhub_shared_nas:

# User mounts in the spawned container
c.DockerSpawner.volumes = {
    "jupyterlab-{username}_home": "/home",
    "jupyterlab-{username}_workspace": DOCKER_NOTEBOOK_DIR,
    "jupyterlab-{username}_cache": "/home/lab/.cache",
    "jupyterhub_shared_nas": "/mnt/shared"
}
Languages
Python 38.3%
HTML 34.7%
CSS 10.5%
Shell 8.7%
Batchfile 3.6%
Other 4.2%