Skip to content

Unprivileged on shared HPC

nwp-compose is designed so a user with no elevated privileges on a shared HPC node can do everything on the user path. The only root operation in the whole project is the production hosted-GUI installer (deploy/install.sh), which is an operator tool, not a user tool.

$HOME on a login node is often small or on slow NFS. Every state path honors an override so it can live on scratch:

OverrideRedirectsDefault
NWPC_VENV_DIRthe virtual environment<repo>/.venv
XDG_CACHE_HOMEthe gist/HTTP cache~/.cache/nwp-compose
XDG_DATA_HOMEthe plugin component dir~/.local/share/nwp-compose
CYLC_RUN_DIRCylc run directories~/cylc-run
Terminal window
export NWPC_VENV_DIR=/scratch/$USER/nwpc/venv
export XDG_CACHE_HOME=/scratch/$USER/.cache
export CYLC_RUN_DIR=/scratch/$USER/cylc-run

A read-only or full filesystem degrades gracefully — a cache that cannot be written is a skipped optimization, not a crash.

nwp-compose gui serves a read-only view of your ~/cylc-run/ on 127.0.0.1 only, with five GET-only endpoints and path-traversal protection. On a shared node where the default port is taken, it retries on an ephemeral port and announces the chosen one. Reach it from your laptop with an SSH tunnel — no inbound ports, no service.

The CLI and local GUI need no container at all — that is the simplest path and the one to prefer. Where containers are needed (the CI harness, or a future containerized runtime):

  • Apptainer / Singularity is the NCAR HPC standard and runs rootless by default. It is the documented alternative to the Docker Compose harness — no daemon, no root, runs on Derecho.
  • Rootless Podman is a near-term drop-in where user namespaces are enabled (/proc/sys/user/max_user_namespaces > 0).
  • Cylc’s own scheduler is user-space. Once cylc play is running, the cycling workflow is the recurrence engine — this is the primary answer.
  • To wrap or kick automation, use a user crontab (crontab -e), never a system unit. No systemd --user is assumed (it is unavailable on nodes without XDG_RUNTIME_DIR/lingering).