Tutorial Seminar Series

Because we are unable to host an in-person tutorial for this fall season, the Xdev team has decided to switch to a virtual seminar series. We will be hosting one hour tutorial sessions scheduled for the 2nd and 4th Wednesdays of each month at 1 PM Mountain starting in October. The first tutorial session, covering how to create your first Python script, will be on October 14th. Some tutorials will be 1-off seminars on specific topics, while others will span multiple seminar sessions.

Read more…

Writing multiple netCDF files in parallel with xarray and dask

A typical computation workflow with xarray consists of:

  1. reading one or more netCDF files into an xarray dataset backed by dask using xr.open_mfdataset() or xr.open_dataset(chunks=...),
  2. applying some transformation to the input dataset, and
  3. saving the resulting output to disk in a netCDF file using xr.to_netcdf().

The last step (3) can easily lead to a large netCDF file (>=10GB in size). As a result, this step can take a very long time to complete (since it is run in serial), and sometimes may hang. So, to avoid these issues one can use one of the lesser-used but helpful xarray capabilities: the xr.save_mfdataset() function.

Read more…

Templating isn't just for Web Developers!

Recently, I found myself rambling on in an unprepared attempt to explain templating to a colleague of mine who had no prior experience with frontend design. It didn't take me long to realize that my explanation was convoluted and confusing, not that I stopped talking, mind you. Attribute that to being unprepared, or sleep deprived (i.e., child still not sleeping through the night!), or just a bad teacher.

Read more…

We are Xdev!

More than a year ago, Matthew Long and I started shopping around the idea that a lot of what we do here at NCAR, as well as how we do it, needed a bit of a facelift. Our pitch was simple: As an institution that supports the geoscientific community by providing valuable datasets, supercomputing facilities, and computer models, NCAR needs to exemplify modern best practices in the development and use of these resources. We took inspiration from the remarkably effective Pangeo community, which continues to show enormous success with Open Source development when coupled with modern software best practices, and we started to develop a plan of how to help shift, albeit slowly, both NCAR and---though collaboration with the Pangeo community---the geoscience community toward a better future.

Read more…

Time

The Significance of Time

Time is relative, as my coworker Anderson likes to remind me every time manipulating this coordinate proves to be relatively difficult.

This post is to be the first in a series of my struggles coming from an atmospheric science background and transitioning into software. My hope is that if I detail pain points and headaches I encounter along my journey, you won’t have to.

Read more…