Stream: xarray

Topic: Dask zarr load issues

view this post on Zulip Brian Bonnlander (Jul 16 2021 at 23:06):

Hi,
This might be an intake-esm issue, I'm not sure.

I recently tried writing some Zarr stores that I've written/read back successfully in the past, but now it seems can't read them back in successfully using intake-esm. I had just rebuilt my conda environment, so it's possible I have a newer version of various pangeo-related packages than before.

What is strange is that I can open the Zarr store using xr.open() , but not with this call:

    with dask.config.set(**{'array.slicing.split_large_chunks': False}):
        dsets = col_subset.to_dataset_dict(zarr_kwargs={'consolidated': True}, preprocess=preprocess)

KeyError: '.zmetadata'

My environment includes: 

xarray                    0.18.2             pyhd8ed1ab_0    conda-forge
dask                      2021.7.0           pyhd8ed1ab_0    conda-forge
intake-esm                2021.1.15          pyhd8ed1ab_0    conda-forge
zarr                      2.8.3              pyhd8ed1ab_0    conda-forge

The Zarr write/read steps were working before I rebuilt my conda environment. Unfortunately, I don't know what versions I had before, but something has changed, it seems to me.

view this post on Zulip Brian Bonnlander (Jul 16 2021 at 23:12):

maybe it's the new version of dask, since that has a recent date.

view this post on Zulip Brian Bonnlander (Jul 19 2021 at 17:40):

OK, I think I introduced a bug that was the source of the problem. Sorry for the false alarm.

Last updated: May 16 2025 at 17:14 UTC