Stream: dask

Topic: .zarray ownership

view this post on Zulip Haiying Xu (Apr 30 2021 at 18:02):

I read some netcdf files and convert to zarr files using Dask and Xarray, but the .zarray and .zmetadata ownership is owner read/write only. Is it supposed to be like that? Is it possible for me to change the file permission in the codes?

view this post on Zulip Brian Bonnlander (Apr 30 2021 at 18:11):

The Zarr stores I've created have permissions -rw-r--r-- for those files. That means anyone can read.

view this post on Zulip Anderson Banihirwe (Apr 30 2021 at 18:18):

@Haiying Xu, this was a known issue at some point: https://github.com/zarr-developers/zarr-python/issues/325, and I think it was resolved...

Can you check that the umask for the parent directory you are writing to is the same as your default umask?

view this post on Zulip Haiying Xu (Apr 30 2021 at 18:44):

The directory is 755, my default is 022

view this post on Zulip Brian Bonnlander (Apr 30 2021 at 18:49):

My umask is the same. I don't know if it matters, but in my case I was writing the stores to /glade/scratch, which is visible by anyone logged in. Perhaps if you haven't tried that yet, it might make a difference? If you are doing that already, then I don't know what to suggest...

view this post on Zulip Haiying Xu (Apr 30 2021 at 18:54):

I am writing to scratch. All data and .z* file have 600, directory has 755. Let me try writing to $WORK space to see any change.

view this post on Zulip Brian Bonnlander (Apr 30 2021 at 19:05):

Actually the umask command gives me 0022 as the answer. I am not sure if that extra zero makes a difference. You might check your login scripts to see if there are any umask commands given there. That is all I can think of right now...

view this post on Zulip Brian Bonnlander (Apr 30 2021 at 19:55):

I should also mention I was using xarray 0.17.0:

(pangeo-cordex2) casper-login2:~$. conda list xarray
# packages in environment at /glade/work/bonnland/miniconda3/envs/pangeo-cordex2:
#
# Name                    Version                   Build  Channel
cf_xarray                 0.5.1              pyh44b312d_0    conda-forge
xarray                    0.17.0             pyhd8ed1ab_0    conda-forge

view this post on Zulip Haiying Xu (Apr 30 2021 at 20:06):

I am using the same version of Xarray.

view this post on Zulip Brian Bonnlander (Apr 30 2021 at 20:14):

(pangeo-cordex2) casper-login2:~$ conda list zarr
# packages in environment at /glade/work/bonnland/miniconda3/envs/pangeo-cordex2:
#
# Name                    Version                   Build  Channel
zarr                      2.7.0              pyhd8ed1ab_0    conda-forge

view this post on Zulip Haiying Xu (May 04 2021 at 17:36):

I upgraded zarr to 2.8.1, but still get 600 for .zarray and data.

view this post on Zulip Haiying Xu (May 04 2021 at 17:47):

@Brian Bonnlander Thanks. I used the wrong environment, in fact, after I upgraded zarr, the ownership were changed to right mode.

Last updated: Jan 30 2022 at 12:01 UTC