Skip to content

Migrating from bash + cron

If you run forecasts today from a hand-written run_wrf.sh kicked off by a crontab line, this is the page for you. The goal is not to throw away your knowledge — it is to capture it in a validated, reproducible, shareable workflow.yaml so the next person (including future you) inherits a checked artifact instead of a script with a comment that says # don't change this.

Your personal setup already has every concept nwp-compose formalizes — they are just implicit:

In your bash/cron worldIn nwp-compose
0 0,6,12,18 * * * in crontabcycling.period: PT6H + initial/final cycle points
FCST_HOURS=24 in the scriptcycling.forecast_hours_default: 24
ungrib.exe, real.exe, wrf.exe calls in orderthe component graph (ungrib → real → wrf), with order derived and checked
your edited namelist.inputa BYO-namelist component (patched only on cycle keys)
aws s3 cp ... at the endthe archive_s3 component
find . -mtime +7 -delete cleanupthe housekeeping component (a P1D cycle)
“remember to check the boundary interval divides the run length”warning W2, enforced at validate time

The migration is mostly making the implicit explicit — and getting a validator to check the parts you used to hold in your head.

nwp-compose import-cron reads a crontab line and/or a driver script and produces a starter workflow.yaml plus a migration report:

Terminal window
nwp-compose import-cron --cron "0 0,6,12,18 * * *" --script ./run_wrf.sh -o draft.yaml

It will:

  • infer the cycling cadence from the cron hour field (0,6,12,18period: PT6H);
  • grep the script for known executables (ungrib, geogrid, real.exe, wrf.exe, atmosphere_model, mpassit, upp, aws s3 cp) and detect the MPAS-vs-WRF path;
  • emit a valid, runnable starter — the closest shipped template (e.g. conus-wrf-gfs for a WRF script) with your inferred cadence overlaid and a provenance header — so the output actually validates rather than being an incomplete skeleton;
  • write a companion draft.MIGRATION.md report that separates “inferred with confidence” from the TODOs you must resolve (real dates, domain, platform), and reconciles the components it detected against the template’s set.
  1. Import to a draft:

    Terminal window
    nwp-compose import-cron --cron "0 0,6,12,18 * * *" --script ./run_wrf.sh -o draft.yaml
  2. Read draft.MIGRATION.md and resolve every TODO it lists. This is where your domain knowledge goes back in — the importer cannot know your domain bounds, real run window, or platform. Pass --validate-after to have it run validate on the starter immediately.

  3. Keep your namelist. If you trust your hand-tuned namelist.input, switch the model component to BYO mode rather than re-expressing every key. nwp-compose will patch only the cycle-dependent keys and leave the rest byte-for-byte. See Interactive namelist configuration.

  4. Validate — now the parts you used to check by hand are checked for you:

    Terminal window
    nwp-compose validate draft.yaml
  5. Preflight and run on your node:

    Terminal window
    nwp-compose doctor # is this node ready? (prerequisites, paths, quota)
    nwp-compose run draft.yaml --dummy # prove the graph; drop --dummy for the real forecast
  • A validator that blocks CFL-violating time steps, illegal decompositions, and boundary cadences that do not divide the run length — before a six-hour job crashes.
  • A reproducibility manifest (manifest.json) stamped with the tool version, component versions, and a hash of your input — so “this used to work” becomes a diff, not a mystery.
  • A shareable artifact. Hand a colleague your workflow.yaml (or a Gist URL via --from-gist); they get exactly your forecast, validated, without inheriting your $PATH.