Stream: hack-projects

Topic: catalogs-for-MPAS/WRF-models-output

Anderson Banihirwe (Mar 13 2020 at 18:24):

Motivation

• There's a growing list of intake-esm catalogs for a number of collections (CMIP5, CMIP6, CESM-LE). However, there are existing collections/model ouputs that are missing from this list (for e.g. MPAS, WRF), etc...

Proposed Hacking

• Develop utilities for building catalogs for MPAS and WRF model outputs

Desired Collaborators

• Anyone interested in creating (or learning about) intake-esm catalogs in general.

Useful references

• esm-collection-spec: https://github.com/NCAR/esm-collection-spec
• intake-esm: https://intake-esm.readthedocs.io/en/latest/
• intake-esm-datastore: https://github.com/NCAR/intake-esm-datastore
  catalogs-for-MPAS/WRF-models-output

Seth McGinnis (Mar 13 2020 at 18:46):

The NA-CORDEX dataset has WRF data in it, but it's been heavily post-processed. I assume you're targeting unprocessed output files for this hack, so we don't have much overlap. True? In any case, I'd be happy to discuss what we do in the post-processing and why.

Anderson Banihirwe (Mar 13 2020 at 18:51):

Right now, I am open to any direction.. Since I am not a user myself, I could use some input from MPAS and/or WRF users on how MPAS and WRF data are used by most users.

• Are WRF/MPAS model output post-processed into time-series or do most people just use raw model output as is?

Seth McGinnis (Mar 13 2020 at 18:55):

I don't know much about MPAS, but with WRF, the raw outputs are stored in synoptic format (all variables in one file for a short chunk of time). And yes, what most climate users need is timeseries format, where each file has one variable over all time. The raw WRF outputs also have an enormous amount of junk metadata in them that we filter out. And they need some cleanup to be fully CF-compliant.

Anderson Banihirwe (Mar 13 2020 at 19:09):

And yes, what most climate users need is timeseries format, where each file has one variable over all time.

:+1:
I would prefer to stick with time-series as well. From past discussions with @Kevin Paul , @Sheri Mickelson, it appears that
supporting catalogs with synoptic files (at least for CESM) may require a large amount of time/effort due to lack of standards. It sounds like WRF and CESM are in the same boat. Having input on the state of MPAS model may help in deciding how to proceed.

Sheri Mickelson (Mar 26 2020 at 14:09):

@Seth McGinnis I was going to start working on this. Do you have any WRF netCDF files on glade that I can look at in order to see what kind of attributes they attach to the files and what the file structure looks like? I'm hoping to come up with some kind of generic interface that can be used for all NCAR models.

Seth McGinnis (Mar 26 2020 at 17:29):

@Sheri Mickelson We do!
/glade/collections/cdg/work/cordex/raw/wrf-era-25/1979/2D
/glade/collections/cdg/work/cordex/raw/wrf-era-25/1979/3D

I don't know that they're 100% representative of WRF files in general, but that should be at least a decent starting point.

Here's what the corresponding 6-hourly precip data looks like after we've post-processed it:
/glade/collections/cdg/data/cordex/data/raw/NAM-22/6hr/WRF/ERA-Int/eval/pr

Sheri Mickelson (Mar 26 2020 at 17:35):

Thanks @Seth McGinnis

Last updated: May 16 2025 at 17:14 UTC