Stream: python-questions

Topic: Odd Errors Loading NetCDF

view this post on Zulip Brendan Myers (Nov 11 2021 at 21:57):

Hey there everyone!
I am currently trying to open some data from the anemometer on top of the ML. For a very rough outline of what the data looks like, the data from 1996 until 2020 is in .cdf formatting and from 2020 to present is in .nc. In order to open these files I am using xarray.open_mfdataset but am running into an issue. It seems to run fine when I only read in the .nc datafiles but when I try to read in the .cdf files it gives me an error that seems to be saying that the function is not installed. I've included some screenshots in case they help.
Screen-Shot-2021-11-11-at-2.53.41-PM.png
Screen-Shot-2021-11-11-at-2.54.03-PM.png

Seems like a very odd situation to me so I appreciate any thoughts that anyone has (and let me know if you have any questions for me)! Thank you!

view this post on Zulip Max Grover (Nov 11 2021 at 22:35):

You can specify engine='netcdf4' here within the open_mfdataset lines. Also, the times with the cdf files require a bit more pre-processing. You can use a preprocess function such as the following

def fix_times(ds):
    new_time = cftime.num2date(ds.base_time + ds.time_offset, units='seconds since 1970-01-01')
    ds['time'] = new_time
    ds['time'] = ds.indexes['time'].to_datetimeindex()
    return ds.drop(['base_time', 'samp_secs'])

you can pass this into the open_mfdataset line for the .cdf files

ds = xr.open_mfdataset('path/*.cdf', concat_dim='time', preprocess=fix_times)

then, you call plot as shown below

Screen-Shot-2021-11-11-at-3.34.11-PM.png

view this post on Zulip Brendan Myers (Nov 11 2021 at 22:46):

Wow thanks for the quick reply and for all the help! I definitely would not have figured that one out on my own so thank you very much Max!

view this post on Zulip Brendan Myers (Nov 11 2021 at 23:12):

Max, I'm not sure if I implemented something wrong, but I am getting a different error now... Will you look it over really quickly and make sure I didn't mistype something? It looks reasonable to me...
Screen-Shot-2021-11-11-at-4.07.10-PM.png
Screen-Shot-2021-11-11-at-4.08.57-PM.png

view this post on Zulip Max Grover (Nov 12 2021 at 23:03):

Here is a written up example with some interactive plots @Brendan Myers ! Thanks for bringing up this question! https://ncar.github.io/esds/posts/2021/weather-station-data-preprocess/

Last updated: Jan 30 2022 at 12:01 UTC