Stream: dask

Topic: saving to zarr results in error

view this post on Zulip Mira Berdahl (Jan 26 2022 at 23:52):

Hi,

I'm trying to save a dataArray to a zarr file, and have been able to do this in the past with similar code. However, lately I've run into this issue which I cannot seem to resolve.

Say my DataArray looks like this:

aw_oceanT_global
xarray.DataArraytime: 3600 z_t: 60
Array Chunk
Bytes 1.65 MiB 480 B
Shape (3600, 60) (1, 60)
Count 154875 Tasks 3600 Chunks
Type float64 numpy.ndarray

I do :

aw_oceanT_global.unify_chunks()
aw_oceanT_global.to_dataset(name='TEMP').chunk({'z_t':-1}).to_zarr('/glade/scratch/mberdahl/127kaH11/TEMP/aw_oceanT_global_timeseries_H11.zarr', mode='w')

But receive the following error (just the top bit of it):
distributed.utils - ERROR - 'str' object has no attribute 'text'
Traceback (most recent call last):
File "/glade/work/mberdahl/miniconda/envs/pangeo/lib/python3.9/site-packages/distributed/utils.py", line 681, in log_errors
yield
File "/glade/work/mberdahl/miniconda/envs/pangeo/lib/python3.9/site-packages/distributed/dashboard/components/scheduler.py", line 346, in update
self.root.title.text = title
AttributeError: 'str' object has no attribute 'text'
distributed.utils - ERROR - 'str' object has no attribute 'text'
Traceback (most recent call last):
File "/glade/work/mberdahl/miniconda/envs/pangeo/lib/python3.9/site-packages/distributed/utils.py", line 681, in log_errors
yield
File "/glade/work/mberdahl/miniconda/envs/pangeo/lib/python3.9/site-packages/distributed/dashboard/components/scheduler.py", line 3417, in status_doc
cluster_memory.update()

Looks like I'm running into memory issues maybe?bBut this DataArray isn't any bigger than other arrays I've had success with with the same methods. Any thoughts appreciated!

view this post on Zulip Anderson Banihirwe (Jan 27 2022 at 00:25):

@Mira Berdahl, I don't know why this is failing by looking at the traceback. Can you provide a reproducible test case or a pointer to the notebook you're using?

view this post on Zulip Mira Berdahl (Jan 27 2022 at 02:51):

@Anderson Banihirwe , the path to the notebook is:
/glade/u/home/mberdahl/FirstLook_LIGruns/DASK_scripts/TryAreaWeightingAvg.ipynb
Thanks for taking a look!

Last updated: Jan 30 2022 at 12:01 UTC