Posts tagged netcdf

Using Kerchunk with CESM Timeseries Data on the Cloud

We benchmark reading a subset of the CESM2-Large Ensemble stored as a collection of netCDF files on the cloud (Amazon / AWS) from Casper. We use a single ensemble member historical experiment with daily data from 1850 to 2009, with a total dataset size of 600+ GB, from 13 netCDF4 files.

We read in two ways:

Read more ...

Writing multiple netCDF files in parallel with xarray and dask

A typical computation workflow with xarray consists of:

reading one or more netCDF files into an xarray dataset backed by dask using xr.open_mfdataset() or xr.open_dataset(chunks=...),


Read more ...