datetime index · python-questions

Stream: python-questions

Topic: datetime index

Matt Long (Oct 01 2021 at 20:11):

Is anyone aware of an issue arising recently with xarray datetime indexing?

I am having trouble indexing a time coordinate with calendar='noleap'.

I can generate a DataArray with calendar='noleap' as follows:

time = []
for year in range(2005, 2015):
    for month in range(1, 13):
        time.append(
            cftime.datetime(year, month, 15, calendar="noleap")
        )
da = xr.DataArray(np.arange(0, len(time)), dims=("time"),
                  coords={"time": xr.DataArray(time, dims=("time"))})

This indexing operation:

da.sel(time=slice("2005", "2006"))

Used to work! But with a recent xarray update I get the following error:

TypeError: cannot compare cftime.datetime(2010, 1, 15, 0, 0, 0, 0, calendar='noleap', has_year_zero=True) and
cftime.datetime(2005, 1, 1, 0, 0, 0, 0, calendar='gregorian', has_year_zero=False)

But the following does work

time_slice = slice(
    cftime.datetime(2005, 1, 1, calendar="noleap"), cftime.datetime(2014, 12, 31, calendar="noleap")
)
da.sel(time=time_slice)

Anderson Banihirwe (Oct 04 2021 at 14:39):

@Matt Long, something strange is going on during the DataArray instantiation and I haven't figured out what it is... However, the following seems to be working fine:

In [15]: times = xr.cftime_range('2005', '2015', freq='MS', calendar='noleap')

In [16]: da = xr.DataArray(np.arange(0, len(times)), dims=("time"),
    ...:                   coords={"time": xr.DataArray(times, dims=("time"))})

In [17]: da
Out[17]:
<xarray.DataArray (time: 121)>
array([  0,   1,   2,   3,   4,   5,   6,   7,   8,   9,  10,  11,  12,
        13,  14,  15,  16,  17,  18,  19,  20,  21,  22,  23,  24,  25,
        26,  27,  28,  29,  30,  31,  32,  33,  34,  35,  36,  37,  38,
        39,  40,  41,  42,  43,  44,  45,  46,  47,  48,  49,  50,  51,
        52,  53,  54,  55,  56,  57,  58,  59,  60,  61,  62,  63,  64,
        65,  66,  67,  68,  69,  70,  71,  72,  73,  74,  75,  76,  77,
        78,  79,  80,  81,  82,  83,  84,  85,  86,  87,  88,  89,  90,
        91,  92,  93,  94,  95,  96,  97,  98,  99, 100, 101, 102, 103,
       104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116,
       117, 118, 119, 120])
Coordinates:
  * time     (time) object 2005-01-01 00:00:00 ... 2015-01-01 00:00:00

In [18]: da.sel(time=slice("2005", "2006"))
Out[18]:
<xarray.DataArray (time: 24)>
array([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15, 16,
       17, 18, 19, 20, 21, 22, 23])
Coordinates:
  * time     (time) object 2005-01-01 00:00:00 ... 2006-12-01 00:00:00

Matt Long (Oct 04 2021 at 15:18):

Thanks @Anderson Banihirwe, I also noticed that if I used cftime_range I was not able to replicate the problem. This issue is arising during processing of CMIP6 data and the minimal example demonstrates the problem. I am not sure why a cftime_range generated time axis works...but that seems to be a good clue. I poked around the xarray code a bit...but it's not clear to me where the problem is occurring. It's like the code used to effectively create slice.start and slice.stop values consistent with the attributes of the time axis. I have a temporary workaround in my project where I do this manually.

Deepak Cherian (Oct 05 2021 at 08:56):

can you t ry updating cftime. There was some churn recently that broke some tests.

Matt Long (Oct 05 2021 at 12:01):

I updated to cftime=1.5.1 (latest release) and the problem still occurs.

Deepak Cherian (Oct 08 2021 at 04:47):

this is a bug then. Can you file an xarray issue please?

Last updated: Jan 30 2022 at 12:01 UTC