xarray: best way to "insert" a time slice into a dataset or dataarray

user_5997555 0
Viewed 354

I have a 3-dimensional xarray dataset with the dimensions x, y, and time. Assuming I know that there's a missing observation at timestep n, what would be the best way to insert a timeslice with no-data values?

Here's a working example:

import xarray as xr
import pandas as pd

x = xr.tutorial.load_dataset("air_temperature")

# assuming this is the missing point in time (currently not in the dataset)
missing = "2014-12-31T07:00:00"

# create an "empty" time slice with fillvalues
empty = xr.full_like(x.isel(time=0), -3000)

# fix the time coordinate of the timeslice
empty['time'] = pd.date_range(missing, periods=1)[0]

# before insertion
print(x.time[-5:].values)

# '2014-12-30T18:00:00.000000000' '2014-12-31T00:00:00.000000000'
#  '2014-12-31T06:00:00.000000000' '2014-12-31T12:00:00.000000000'
#  '2014-12-31T18:00:00.000000000']

# concat and sort time
x2 = xr.concat([x, empty], "time").sortby("time")

# after insertion
print(x2.time[-5:].values)

# ['2014-12-31T00:00:00.000000000' '2014-12-31T06:00:00.000000000'
#  '2014-12-31T07:00:00.000000000' '2014-12-31T12:00:00.000000000'
#  '2014-12-31T18:00:00.000000000']

The example works fine, but I'm not sure if that's the best (or even the correct) approach.

My concerns are to use this with bigger datasets, and specifically with dask-array backed datasets.

Is there a better way to fill a missing 2d array? Would it be better to use a dask-backed "fill array" when inserting into a dask-backed dataset?

python-xarray pandas time-series python
1 Answers

You might consider using xarray's reindex method with a constant fill_value for this purpose:

import numpy as np
import xarray as xr

x = xr.tutorial.load_dataset("air_temperature")
missing_time = np.datetime64("2014-12-31T07:00:00")
missing_time_da = xr.DataArray([missing_time], dims=["time"], coords=[[missing_time]])
full_time = xr.concat([x.time, missing_time_da], dim="time")
full = x.reindex(time=full_time, fill_value=-3000.0).sortby("time")

I think both your method and the reindex method will automatically use dask-backed arrays if x is dask-backed.

user_38255330
Related