Creating xarray from multiple numpy arrays - time series

user_10756615 0
Viewed 556

I want to create an Xarray DataArray with the following coordinates using a series of numpy arrays for annual time series data (let's say it's temperature over a uniform 1500X1500 matrix).

('time', 'lon', 'lat') Coordinates:

  • time (time) datetime64[ns] 2000-12-31 2001-12-31 ... 2020-12-31
  • lon (lon) float64 -19.98 -19.93 -19.88 -19.82 ... 54.88 54.93 54.98
  • lat (lat) float64 39.97 39.92 39.87 39.82 ... -34.88 -34.93 -34.98

The original ('raw') time series data I am using is stored as separate files with the filename denoting each year in the time series (i.e, there is no time information provided within the data files themselves, just in the name, temp2000.xxx, temp2001.xxx, etc). I imported each of these individual data files into separate numpy arrays which have a spatial dimension (corresponding to lat/lon above) but no time dimension other than the variable name I assigned.

I am trying to figure out how to combine all these numpy arrays into one multi-dimensional xarray DataArray with the lat/lon of the numpy arrays and the time defined by a time variable (taken from file name).

This is probably straightforward but I can't wrap my head around it.

temp2000 = np.random.rand(1500, 1500)

xll = -20.0
xur = 55.0
yll = -35.0
yur = 40.0
cellsize = 0.1

lon_tup = np.arange(xll, xur, cellsize) + (cellsize / 2)
lat_tup = np.arange(yll, yur, cellsize)
lat_tup = lat_tup[::-1] + (cellsize / 2)
time2 = pd.date_range("2000-01-01", freq="Y", periods=21)

ds = xr.DataArray(
            coords=[time2, lat_tup, lon_tup], dims=["time", "lat", "lon"])

ds["Temperature_2000"] = (["time", "lat", "lon"], temp2000)

The DataArray is created just fine but obviously the numpy array can't be added because it lacks the "time" dimension. Can I force a time dimension via a separate step? Example is just for one time step (2000) with dummy data for illustration purposes.

python-xarray numpy arrays python
2 Answers

You can only initialize a DataArray with dimensions that reflect the actual shape of the data. So, you could either reshape your numpy array to include an extra dimension (e.g. with reshape or np.expand_dims), or create the DataArray as (lat, lon) then add the extra dimension afterwards (e.g. with da.expand_dims), as in this example:

da = xr.DataArray(
    temp2000,
    coords=[lon_tup, lat_tup],
    dims=["lon", "lat"],
)

# expand the array to include a length-1 time dimension
# corresponding to the file's time indicator
da = da.expand_dims(time=pd.Index([2000], name="time"))

Alternatively, you could exclude the time dim until you're ready to concatenate the data:

arrays = []

time = pd.date_range("2000-01-01", freq="Y", periods=21)
years = time.year

for y in years:
    # read in your data as (lat, lon)
    ...

    arrays.append(da)

# concat using a full TimeIndex to give the values of time as well as the name
result = xr.concat(arrays, dim=time)

Note also the difference between xarray Datasets (usually abbreviated ds), which are essentially dictionaries of DataArrays, and DataArrays (usually abbreviated da), which are the fundamental array unit within xarray. Datasets are useful for interacting with storage and organizing your workflows, and can be helpful in applying the same operation across multiple arrays, but in most cases when doing math you want to be working with arrays. See the xarray docs on data structures for more info.

user_38887190

Thanks to Michael Delgado for guidance. Here is my solution:

xll = -20.0
xur = 55.0
yll = -35.0
yur = 40.0
cellsize = 0.1

lon_tup = np.arange(xll, xur, cellsize) + (cellsize / 2)
lat_tup = np.arange(yll, yur, cellsize)
lat_tup = lat_tup[::-1] + (cellsize / 2)

StartYear = 2000
EndYear = 2020
for x in range(StartYear, EndYear):
    # filein would be the data read in from the external file
    filein = np.random.rand(1500, 1500)
    temp = np.resize(filein, (1,1500,1500))
    temp[:, 0, 0] = x
    if x == StartYear:
        array_wbm = temp
    else:
        array_wbm = np.concatenate(([array_wbm, temp]), axis=0)

time = pd.date_range("2000-01-01", freq="Y", periods=21)
years = time.year
da = xr.DataArray(data=array_wbm,
                  coords=[years, lat_tup, lon_tup],
                  dims=["year", "lat", "lon"]
                  )
user_107566150
Related