Stream: xarray

Topic: sparse matrix fill_value for PFT regridding

view this post on Zulip Daniel Kennedy (Oct 06 2021 at 18:40):

@Deepak Cherian provided some nice functions to regrid CLM output from [time,PFT] to [time, lat , lon , pft], using the python package sparse.

The solution is working well, except that I can't figure out how to impose a fill_value. I.e. for gridcells that are unpopulated in the sparse matrix, I would like to use fill_value in the dense matrix. I tried passing fill_value to sparse.COO, but that didn't seem to work.

Notebook for testing:
https://github.com/djk2120/ctsm_trendy_2021/blob/main/pft_regridding.ipynb

view this post on Zulip Matt Long (Oct 06 2021 at 18:44):

One approach I have used in regridding application is to remap a field of ones, putting zeroes in where there are missing data, then renormalizing by the sum of the remapped ones:
https://github.com/marbl-ecosys/forcing-Fe-sedflux/blob/96f2a565300ba9d43b373c6ba8537e20d30e4ca1/notebooks/regrid_tools/core.py#L293

Something like this might work for your application.

view this post on Zulip Daniel Kennedy (Oct 06 2021 at 18:46):

That makes sense. Thanks. I think that will work for me.

view this post on Zulip Deepak Cherian (Oct 29 2021 at 16:16):

@Daniel Kennedy I can't see why this doesn't work, you should be able to set np.nan as the fill_value. I think that's what you want for plotting purposes anyway.

import numpy as np
import sparse

sparse.COO(
    coords=[[0, 1, 2], [0, 1, 2]], data=[1.0, 1.0, 1.0], fill_value=np.nan
).todense()

array([[ 1., nan, nan],
       [nan,  1., nan],
       [nan, nan,  1.]])

Note this is not regridding really. It's constructing an nD sparse array from a "compressed" dense 2D array.

Can you try again, and if it doesn't work, can you send me a link to the notebook on cheyenne/casper?

Last updated: May 16 2025 at 17:14 UTC