Precious Mongwe (Mar 18 2020 at 21:01): Precious Mongwe (Mar 18 2020 at 21:01):I'm having an issue getting CESM-Le data to load using intake_esm on Cheyenne, it works fine on Casper but currently overwhelmed; can't get nodes on casper
Some issue with the json file accessing the data
catalog = intake.open_esm_datastore('/glade/work/mlevy/intake-esm-collection/json/glade-cesm1-le.json')
lens_temp = catalog.search(experiment='20C',component='ocn', variable=['TEMP'], member_id=1)
def preprocess(ds):
keep_vars_temp = ['REGION_MASK', 'z_t', 'z_w','dz', 'TAREA', 'TLONG', 'TLAT', 'time', 'time_bound', 'member_id', 'TEMP', 'O2','SALT']
ds = ds.drop([v for v in ds.variables if v not in keep_vars_temp])
return ds
dq_temp = lens_temp.to_dataset_dict(cdf_kwargs={"chunks": {"time": 12}}, preprocess=preprocess)
Error message I get;
No such file or directory: b'/glade/campaign/cesm/collections/cesmLE/CESM-CAM5-BGC-LE/ocn/proc/tseries/monthly/TEMP/b.e11.B20TRC5CNBDRD.f09_g16.001.pop.h.TEMP.185001-200512.nc'
have anyone dealt with this before or can help?
Precious Mongwe (Mar 18 2020 at 21:13):catalog = intake.open_esm_datastore('/glade/work/mlevy/intake-esm-collection/json/glade-cesm1-le.json')
lens_temp = catalog.search(experiment='20C',component='ocn', variable=['TEMP'], member_id=1)
def preprocess(ds):
keep_vars_temp = ['REGION_MASK', 'z_t', 'z_w','dz', 'TAREA', 'TLONG', 'TLAT', 'time', 'time_bound', 'member_id', 'TEMP', 'O2','SALT']
ds = ds.drop([v for v in ds.variables if v not in keep_vars_temp])
return ds
dq_temp = lens_temp.to_dataset_dict(cdf_kwargs={"chunks": {"time": 12}}, preprocess=preprocess)
Keith Lindsay (Mar 18 2020 at 21:22):@Precious Mongwe, it looks like you are attempting to access a file that is on campaign storage. Campaign storage is only accessible from casper or globus. You cannot access it directly from cheyenne (unless you're using globus).
Precious Mongwe (Mar 18 2020 at 21:23):Noted, Thanks Keith
Last updated: May 16 2025 at 17:14 UTC