Stream: hack-projects

Topic: GeoCAT: sigma-coordinate remapping

Matt Long (Mar 15 2020 at 03:06):

Here's a minimum working example of some code that remaps the ocean model from z coordinates to sigma coordinates.  
https://github.com/matt-long/remap_z_to_sigma/tree/master

This is based on code that @Keith Lindsay wrote in F77 and I wrapped with f2py.

I wrote the Python wrapper...as a project to learn Python (this origin is abundantly evident in the code).

While I am using dask.delayedto work on time chunks in parallel, my remapper code is setup to read and write files (i.e. it is doing I/O!).

[You can see my bias here in the command line interface (CLI) I built. I used to do that a lot. As I've starting using Jupyter to interact with my data, I rarely do that anymore.]

The code breaks up input datasets (timeseries) into multiple time-chunk files; it operates on each and concatenate output files with nco.

There are several shortcomings:

• This works, but is not as performant as it could be. This is an embarrassingly parallel problem, each vertical column is independent.
• It's also clunky. Mixing the computation with I/O makes this tool less useful than it could be if it just operated on an xarray.Dataset or DataArrays. This would offer opportunities to explore dask parallelism (and could in theory still support a CLI for files).
• Finally, this code is POP centric, but should be general.

I would like to invite contributions to improving the workflow for this operation. This notebook illustrates the workflow
https://github.com/matt-long/remap_z_to_sigma/blob/master/sigma_remap.ipynb
Depends on data on glade but can work on any POP output (PD variable required).

@Orhan Eroglu would you be willing to lead this effort? @Anderson Banihirwe, we've discussed this before, so would good to have you involved. @Keith Lindsay and @Michael Levy may have important insight and advice.

Some questions:

• Can this functionality be replicated with existing tools in the pangeo stack? (i.e. xhistogram, xgcm)
• If not, where should this functionality be implemented?

cc @geocat, @all

Riley Brady (Mar 15 2020 at 16:00):

This would of course be a very useful tool to have in pop_tools or as a standalone utility. For some inspiration, Spencer Jones wrote https://github.com/cspencerjones/xlayers at the CMIP6 hackathon. I think it wraps some fortran code as well and interfaces with xarray. It's supposedly finished for the most part, but the docs are not great. I haven't tried it yet. But it might serve as some inspiration for how to handle this for POP. Or perhaps it can be leveraged directly. Just some thoughts.

Orhan Eroglu (Mar 23 2020 at 03:57):

Hi Matt,

I noticed with @John Clyne 's reminder that I have missed this message within the last week's email flows. Sorry to you, thanks to John!

My short answer is yes; I would be glad to make this effort. Thanks a lot for the insights you have already given. I will get into the code in the following days and try to come up with an interpretation of mine.

Talk to you soon on this!

Orhan Eroglu (Apr 10 2020 at 20:10):

Hi @Matt Long , @Anderson Banihirwe, @Keith Lindsay, @Michael Levy, and possible others who would be interested and/or have previous effort with this z-to-sigma remapping work.

I will be trying to get familiar with this code as well as potentially usable existing resources such as Pangeo stack from now on. Any recommendations or plans from your expertise on how to kick off and further proceed this effort would be great to hear. I will be reporting my findings as well.

Last updated: May 16 2025 at 17:14 UTC