How to apply gaussian process regression on series problems?

Question

I have been working on a problem as follows, that I wish to perform regression on using Gaussian Process Regressor (GPR):
Input (X): [list1, list2, list3, ....] # All the lists (or arrays) may not be of the same size
Output(y): [value1, value2, value3, ....] # Equal and corresponding to the number of input lists

Code:

X = np.atleast_2d(X)
# X = X.reshape(124, 50, 3)   # 3D representation does not work for GPR
y = np.atleast_2d(y).reshape(-1,1)

# Gaussian process regressor
kernel = RBF(0.1, (10,10))
gp = gpr(kernel=kernel, n_restarts_optimizer=100, alpha = 0.04)
gp.fit(X,y)

But this code only works for one value of one list (within X) at a time, and not for one whole list as an input.

I have tried various representations and have also been able to achieve the representation of multiple features too, but this problem has been bothering me for a couple of weeks. I only need to know how to overcome the couple of hurdles below. I cannot perform the following operations:

Enter 3D shaped data for GPR
(Because of the above) Enter a time series as input and a value as the output

I have been able to implement the same using deep-learning neural networks, but due to the small amount of data that I have, I wanted to check the same for machine learning models. If it is not possible to use GPR, any suggestions on using other ML models like Support and Relevance Vector Machines would also be extremely helpful.

EDIT 1:

As per the response, I would describe my scenario in much more detail. I have 124 materials, and the first 50 trial runs for each material. I, hence, have the following dataframe:

index material_id   trials    strain_ratio
    0       0          2           1.070689
    1       0          3           1.071900
    2       0          4           1.072510
    3       0          5           1.073174
    4       0          6           1.073576
    ...    ...         ...           ...
    6195    123       47           1.071866
    6196    123       48           1.072039
    6197    123       49           1.072053
    6198    123       50           1.072145
    6199    123       51           1.072000

The columns "material id", "trials" and "strain ratio" are the 3 features that I have. So, 124 materials, 50 trials on EACH of the materials and the 3 features make up the desired 3D shape of (124, 50, 3). The output is the breaking point of the material, and I need to predict at which trial the breaking point occurs (could be the 100th trial, or the thousandth) with the available features for the first 50 trials. It is this 3D shape of the input that I wish to pass to the GPR and SVM/RVM, but it is not accepted, as only 2D inputs to such models are allowed. I need to pass in the 50 trials of each material as a list (basically a time series, on the basis of which the breaking point at some point of time in the future is to be predicted).

In the last couple of days, I was able to implement the structure, where any ONE of the features can be passed as an input list to the model in order to predict the breaking point, which is the target of the model. The input list is, hence, of the shape (124, 50) because I can only pass one feature.

Code:

# Inputs to RVM/GPR
X_train = nature_first_50['strain_ratio'].to_numpy()
X_train = np.atleast_2d(X_train)
X_train = X_train.reshape(124, 50)

# Targets
y = breaking_points.copy()
y = np.atleast_2d(y).reshape(-1,1)

Shapes of the input and output:

X_train : (124, 50)
y       : (124, 1)

As is better explained from the shape, 50 previous points in the time series predict one point somewhere in future.

The GPR code is straight forward:

kernel = RBF(0.1, (10,10))
gp = gpr(kernel=kernel, n_restarts_optimizer=100, alpha = 0.04)
gp.fit(X_train, y)

However, I wish to find out a way to feed all 3 features together as the input to predict the breaking point, which is my actual and only query. But the shape

X_train : (124, 50, 3)

will not be accepted by the GPR.

I will also be adding more features in future, and I used 3 features here for the sake of ease of understanding. I have perused the documentation a number of times to get a clue, and other websites and tutorials, but none of those had an answer.

user_14147489 · Answer

It seems like your goal is to do regression on time series. It would help to describe what the shape of your time series data is. To me it would seem you have 50 multivariate (3 dimensional) time series of length 124. As far as I can tell, you are using sklearn.gaussian_process.GaussianProcessRegressor for which the fit function takes an array (time series) of feature vectors (three dimensional) and for each feature vector a target value (y array).

You need to make sure the shape of your data is correct. It seems to me need X.shape == (tsLength, tsDim) and y.shape == (tsLength,) or y.shape == (tsLength,nTargets) in case you want to predict multiple target variables. Note that you can only produce a single model per time series in this way.

It never hurts to give the documentation another look.

All things considered, you should make your question more clear:

Add a working example
Explain wat you are trying to achieve
Explain wat you tried

Additional note: neural networks also fall within the domain of machine learning.

How to apply gaussian process regression on series problems?

1 Answers