David Bailey (Jan 26 2024 at 18:03): David Bailey (Jan 26 2024 at 18:03):Hi all,
What is the best way to do a mean on say the last 25 years of data?
field1 = ds1_ann['aice'][::-25,:,:].mean('time')
field2 = ds2_ann['aice'][::-25,:,:].mean('time')
field1 = ds1_ann['aice'].isel(time=[-25]).mean('time')
field2 = ds2_ann['aice'].isel(time=[-25]).mean('time')
Where these are xarray datsets with 38 years of data. I figured out how to do an open_mfdataset on all of the timeseries files. So now I can access the variables in a loop say. In this example, I want to call my plot_diff subroutine with field1 and field2 where these are 2D DataArrays. Or should these be numpy arrays? I then do:
field_diff = field2-field1
in the subroutine and it doesn't like this.
Dave
Michael Levy (Jan 26 2024 at 18:08):@David Bailey what are the dimensions of ds1_ann['aice']? And is this monthly data or annual?
David Bailey (Jan 26 2024 at 18:10):They are 38 x 480 x 540. Then the mean is a dataarray with dimensions 480x540. However, the error is:
NotImplementedError: 'item' is not yet a valid method on dask arrays
Michael Levy (Jan 26 2024 at 18:14):ds1_ann['aice'].isel(time=slice(-25,None)) should shorten ds1_ann['aice'] to the time period you want, so I'd do ds1_ann['aice'].isel(time=slice(-25,None)).mean('time')
Michael Levy (Jan 26 2024 at 18:14):Do you know what line in your notebook is causing the NotImplementedError exception?
David Bailey (Jan 26 2024 at 18:19):Here is the whole error message:
AttributeError Traceback (most recent call last)
File /glade/work/dbailey/conda-envs/cupid-analysis/lib/python3.11/site-packages/xarray/core/ops.py:192, in _call_possibly_missing_method(arg, name, args, kwargs)
191 try:
--> 192 method = getattr(arg, name)
193 except AttributeError:
AttributeError: 'Array' object has no attribute 'item'
During handling of the above exception, another exception occurred:
NotImplementedError Traceback (most recent call last)
Cell In[28], line 6
2 field2 = ds2_ann['aice'].isel(time=[-25]).mean('time')
4 ds1_ann['aice']
----> 6 plot_diff(field1,field2,0.,1.,case1,case2,"N")
Cell In[18], line 67, in plot_diff(field1, field2, field_min, field_max, case1, case2, proj)
64 ax.set_boundary(circle, transform=ax.transAxes)
65 ax.add_feature(cfeature.LAND,zorder=100,edgecolor='k')
---> 67 this=ax.pcolormesh(TLON,
68 TLAT,
69 field_diff,
70 cmap="seismic",vmax=field_std2.0,vmin=-field_std2.0,
71 transform=ccrs.PlateCarree())
72 plt.colorbar(this,orientation='vertical',fraction=0.04,pad=0.01)
73 plt.title(case2+"-"+case1,fontsize=10)
File /glade/work/dbailey/conda-envs/cupid-analysis/lib/python3.11/site-packages/cartopy/mpl/geoaxes.py:315, in _add_transform.<locals>.wrapper(self, *args, **kwargs)
310 raise ValueError(f'Invalid transform: Spherical {func.__name__} '
311 'is not supported - consider using '
312 'PlateCarree/RotatedPole.')
314 kwargs['transform'] = transform
--> 315 return func(self, *args, **kwargs)
File /glade/work/dbailey/conda-envs/cupid-analysis/lib/python3.11/site-packages/cartopy/mpl/geoaxes.py:1781, in GeoAxes.pcolormesh(self, *args, **kwargs)
1778 # Add in an argument checker to handle Matplotlib's potential
1779 # interpolation when coordinate wraps are involved
1780 args, kwargs = self._wrap_args(*args, **kwargs)
-> 1781 result = super().pcolormesh(*args, **kwargs)
1782 # Wrap the quadrilaterals if necessary
1783 result = self._wrap_quadmesh(result, **kwargs)
File /glade/work/dbailey/conda-envs/cupid-analysis/lib/python3.11/site-packages/matplotlib/__init__.py:1465, in _preprocess_data.<locals>.inner(ax, data, *args, **kwargs)
1462 @functools.wraps(func)
1463 def inner(ax, *args, data=None, **kwargs):
1464 if data is None:
-> 1465 return func(ax, *map(sanitize_sequence, args), **kwargs)
1467 bound = new_sig.bind(ax, *args, **kwargs)
1468 auto_label = (bound.arguments.get(label_namer)
1469 or bound.kwargs.get(label_namer))
File /glade/work/dbailey/conda-envs/cupid-analysis/lib/python3.11/site-packages/matplotlib/axes/_axes.py:6298, in Axes.pcolormesh(self, alpha, norm, cmap, vmin, vmax, shading, antialiased, *args, **kwargs)
6293 kwargs.setdefault('snap', mpl.rcParams['pcolormesh.snap'])
6295 collection = mcoll.QuadMesh(
6296 coords, antialiased=antialiased, shading=shading,
6297 array=C, cmap=cmap, norm=norm, alpha=alpha, **kwargs)
-> 6298 collection._scale_norm(norm, vmin, vmax)
6300 coords = coords.reshape(-1, 2) # flatten the grid structure; keep x, y
6302 # Transform from native to data coordinates?
File /glade/work/dbailey/conda-envs/cupid-analysis/lib/python3.11/site-packages/matplotlib/cm.py:432, in ScalarMappable._scale_norm(self, norm, vmin, vmax)
422 """
423 Helper for initial scaling.
424
(...)
429 Note that this method does not set the norm.
430 """
431 if vmin is not None or vmax is not None:
--> 432 self.set_clim(vmin, vmax)
433 if isinstance(norm, colors.Normalize):
434 raise ValueError(
435 "Passing a Normalize instance simultaneously with "
436 "vmin/vmax is not supported. Please pass vmin/vmax "
437 "directly to the norm when creating it.")
File /glade/work/dbailey/conda-envs/cupid-analysis/lib/python3.11/site-packages/matplotlib/cm.py:581, in ScalarMappable.set_clim(self, vmin, vmax)
579 pass
580 if vmin is not None:
--> 581 self.norm.vmin = colors._sanitize_extrema(vmin)
582 if vmax is not None:
583 self.norm.vmax = colors._sanitize_extrema(vmax)
File /glade/work/dbailey/conda-envs/cupid-analysis/lib/python3.11/site-packages/matplotlib/colors.py:208, in _sanitize_extrema(ex)
206 return ex
207 try:
--> 208 ret = ex.item()
209 except AttributeError:
210 ret = float(ex)
File /glade/work/dbailey/conda-envs/cupid-analysis/lib/python3.11/site-packages/xarray/core/ops.py:204, in _values_method_wrapper.<locals>.func(self, *args, **kwargs)
203 def func(self, *args, **kwargs):
--> 204 return _call_possibly_missing_method(self.data, name, args, kwargs)
File /glade/work/dbailey/conda-envs/cupid-analysis/lib/python3.11/site-packages/xarray/core/ops.py:194, in _call_possibly_missing_method(arg, name, args, kwargs)
192 method = getattr(arg, name)
193 except AttributeError:
--> 194 duck_array_ops.fail_on_dask_array_input(arg, func_name=name)
195 if hasattr(arg, "data"):
196 duck_array_ops.fail_on_dask_array_input(arg.data, func_name=name)
File /glade/work/dbailey/conda-envs/cupid-analysis/lib/python3.11/site-packages/xarray/core/duck_array_ops.py:101, in fail_on_dask_array_input(values, msg, func_name)
99 if func_name is None:
100 func_name = inspect.stack()[1][3]
--> 101 raise NotImplementedError(msg % func_name)
NotImplementedError: 'item' is not yet a valid method on dask arrays
Michael Levy (Jan 26 2024 at 18:22):In plot_diff, the arguments vmax=field_std2.0 and vmin=-field_std2.0 to ax.pcolormesh() look awfully suspicious -- field_std2.0 isn't a valid variable name, it's looking for 0 in the field_std2 class or dict. Is field_std2 a list? If so, that should that be field_std2[0]
David Bailey (Jan 26 2024 at 18:24):Sorry, the asterix character disappeared. It is field_std x 2.0. I have a work around. If I do:
field1 = ds1_ann['aice']
Then pass field1[::-25,:,:].mean("time") it works.
Michael Levy (Jan 26 2024 at 18:27):field1[::-25,:,:] is stepping through the whole field1 array with a stride of -25 (starting at the end of the array, I believe); you want field1[-25:,:,:] to get the last 25 years
David Bailey (Jan 26 2024 at 20:03):Sorry, my workaround was not quite right. The problem comes when I try to subtract two DataArrays:
field_diff = field2-field1
However, I just did:
field_diff = field2.values-field1.values
Last updated: May 16 2025 at 17:14 UTC