I am trying to use GaussianProcess classifier or regressor in an ensemble bagging classifier/regressor. The Gaussian kernel works fine outside the ensemble workflow but as long as it becomes implemented in the ensemble model (bagging here), it generates an error regarding its kernel, declaring that 'CompoundKernel' object has no attribute 'k1'. I regenerated the error using the following simpler code:
import numpy as np
import pandas as pd
from sklearn.gaussian_process import GaussianProcessRegressor
from sklearn.gaussian_process.kernels import CompoundKernel, WhiteKernel, RBF
from sklearn.ensemble import BaggingRegressor
X1 = np.random.exponential(range(0,10), size = 10)
X2 = np.random.poisson(range(0,10), size = 10)
y = np.random.normal(size = 10)
df = pd.DataFrame()
df["X1"] = X1
df["X2"] = X2
df["y"] = y
df1 = df.iloc[:,0:2]
df2 = df.iloc[:,2]
kernel = CompoundKernel([WhiteKernel(noise_level=2), RBF(length_scale=3)])
gaus = GaussianProcessRegressor(kernel = kernel)
bag = BaggingRegressor(n_estimators=10, base_estimator = gaus)
bag.fit(df1, df2) # executing this line generates the error
The following is the error:
AttributeError Traceback (most recent call last)
<ipython-input-32-af2dcdf3f94c> in <module>
2 gaus = GaussianProcessRegressor(kernel = kernel)
3 bag = BaggingRegressor(n_estimators=10, base_estimator = gaus)
----> 4 bag.fit(df1, df2)
......
......
--> 542 k_dims = self.k1.n_dims
543 for i, kernel in enumerate(self.kernels):
544 kernel.theta = theta[i * k_dims:(i + 1) * k_dims]
AttributeError: 'CompoundKernel' object has no attribute 'k1'
Please note that changing the compound kernel to a single kernel (e.g., RBF) solves the issue, but I want to use a hybrid kernel in my model. Do you have any idea on how I can handle this problem?