Skip to content
NSF NCAR Research Applications Laboratory NSF NCAR Research Applications Laboratory nwp-compose

What is nwp-compose?

nwp-compose is a YAML-driven workflow builder for Cylc 8 numerical weather prediction forecasts (MPAS and WRF). You describe a forecast in a single workflow.yaml; the tool validates it, renders a runnable Cylc 8 bundle (flow.cylc + task scripts + namelists + a reproducibility manifest), and hands it to Cylc to install and play.

It exists to consolidate the lab’s scattered, individual bash-and-cron forecast setups into one versioned, validated, reproducible tool — without asking anyone for elevated privileges.

The generate pipeline: workflow.yaml enters the generator, which validates against the frozen schema and component graph then renders the Cylc flow and namelists with sandboxed Jinja2, producing an output bundle of flow.cylc, manifest.json, task scripts and namelists that Cylc installs and plays on the HPC scheduler.
One workflow.yaml in; a validated, runnable Cylc 8 bundle out — with a reproducibility manifest.

The mental model

Section titled “The mental model”

The name echoes docker-compose: you compose a workflow from parts.

  • Modular by component. A forecast is a graph of components — data_ingest, ungrib, init_atmosphere/real, mpas_run/wrf, mpassit/upp, archive_s3. Each declares its inputs, outputs, parameters, platform needs, and the Cylc fragment it owns.
  • Two namelist modes. Templated (you give YAML keys, the tool renders the namelist) or bring-your-own (you give your hand-tuned namelist.input; the tool patches only the cycle-dependent keys and leaves the rest byte-for-byte). See Namelist dual-mode.
  • The schema is the contract. A single frozen v0 component schema is the source of truth for the CLI, the GUI form fields, and the plugin API. The reference flow.cylc is one instance of the schema — not the other way around.

The tool vs. a workflow

Section titled “The tool vs. a workflow”

This repository is the tool. The companion repository cylc-atm-workflow is the reference workflow — “Bahamas 3 km MPAS, built using nwp-compose” — and serves as the golden-path integration test: regenerating its flow.cylc from a YAML must produce a flow that runs to completion against real data. Cylc itself splits the engine (cylc-flow) from individual workflow repos; nwp-compose replicates that split.

What it does not do (initial release)

Section titled “What it does not do (initial release)”

Who it’s for

Section titled “Who it’s for”
AudienceCares aboutStart here
Scientist / modelerParameter values, units, namelists, “why is this value what it is”Tutorial, Authoring
OperatorSLURM directives, cylc trigger, log paths, failoverOperations
Contributor / plugin authorPydantic models, the sandbox contract, the freeze recordComponents, Modernization

Next

Section titled “Next”

Install the tool, then walk your first forecast end to end.