Stream: python-questions

Topic: intake_esm on cheyenne

view this post on Zulip Precious Mongwe (Mar 18 2020 at 21:01):

I'm having an issue getting CESM-Le data to load using intake_esm on Cheyenne, it works fine on Casper but currently overwhelmed; can't get nodes on casper
Some issue with the json file accessing the data

catalog = intake.open_esm_datastore('/glade/work/mlevy/intake-esm-collection/json/glade-cesm1-le.json')
lens_temp = catalog.search(experiment='20C',component='ocn', variable=['TEMP'], member_id=1)

def preprocess(ds):
keep_vars_temp = ['REGION_MASK', 'z_t', 'z_w','dz', 'TAREA', 'TLONG', 'TLAT', 'time', 'time_bound', 'member_id', 'TEMP', 'O2','SALT']
ds = ds.drop([v for v in ds.variables if v not in keep_vars_temp])
return ds

dq_temp = lens_temp.to_dataset_dict(cdf_kwargs={"chunks": {"time": 12}}, preprocess=preprocess)

Error message I get;
No such file or directory: b'/glade/campaign/cesm/collections/cesmLE/CESM-CAM5-BGC-LE/ocn/proc/tseries/monthly/TEMP/b.e11.B20TRC5CNBDRD.f09_g16.001.pop.h.TEMP.185001-200512.nc'

have anyone dealt with this before or can help?

view this post on Zulip Precious Mongwe (Mar 18 2020 at 21:13):

catalog = intake.open_esm_datastore('/glade/work/mlevy/intake-esm-collection/json/glade-cesm1-le.json')

lens_temp = catalog.search(experiment='20C',component='ocn', variable=['TEMP'], member_id=1)

def preprocess(ds):
keep_vars_temp = ['REGION_MASK', 'z_t', 'z_w','dz', 'TAREA', 'TLONG', 'TLAT', 'time', 'time_bound', 'member_id', 'TEMP', 'O2','SALT']
ds = ds.drop([v for v in ds.variables if v not in keep_vars_temp])
return ds

dq_temp = lens_temp.to_dataset_dict(cdf_kwargs={"chunks": {"time": 12}}, preprocess=preprocess)

view this post on Zulip Keith Lindsay (Mar 18 2020 at 21:22):

@Precious Mongwe, it looks like you are attempting to access a file that is on campaign storage. Campaign storage is only accessible from casper or globus. You cannot access it directly from cheyenne (unless you're using globus).

view this post on Zulip Precious Mongwe (Mar 18 2020 at 21:23):

Noted, Thanks Keith

Last updated: Jan 30 2022 at 12:01 UTC