MLJ: selecting rows and columns for training in evaluate

Viewed 66

I want to implement a kernel ridge regression that also works within MLJ. Moreover, I want to have the option to use either feature vectors or a predefined kernel matrix as in Python sklearn.

When I run this code

const MMI = MLJModelInterface    
MMI.@mlj_model mutable struct KRRModel <: MLJModelInterface.Deterministic
        mu::Float64 = 1::(_ > 0)
        kernel::String = "linear"
end
function MMI.fit(m::KRRModel,verbosity::Int,K,y)
    K = MLJBase.matrix(K)
    fitresult = inv(K+m.mu*I)*y
    cache = nothing
    report = nothing
    return (fitresult,cache,report)
end

   
N = 10
K = randn(N,N)
K = K*K
a = randn(N)
y = K*a + 0.2*randn(N)
m = KRRModel()
kregressor = machine(m,K,y)
cv = CV(; nfolds=6,  shuffle=nothing, rng=nothing)
evaluate!(kregressor, resampling=cv, measure=rms, verbosity=1)

the evaluate! function evaluates the machine on different subsets of rows of K. Due to the Representer Theorem, a kernel ridge regression has a number of nonzero coefficients equal to the number of samples. Hence, a reduced size matrix K[train_rows,train_rows] can be used instead of K[train_rows,:].

To denote I'm using a kernel matrix I'd set m.kernel = "" . How do I make evaluate! select the columns as well as the rows to form a smaller matrix when m.kernel = ""?

This is my first time using MLJ and I'd like to make as few modifications as possible.

1 Answers

Quoting the answer I got on the Julia Discourse from @ablaom

The intended use of evaluate! is to estimate the generalisation error associated with some supervised learning model, by subsampling observations, as in cross-validation, a common use-case. I’m afraid there is no natural way for evaluate! do feature subsampling.

https://alan-turing-institute.github.io/MLJ.jl/dev/evaluating_model_performance/

FYI: There is a version of kernel regression implementing the MLJ model interface, namely kernel partial least squares regression from the package GitHub - lalvim/PartialLeastSquaresRegressor.jl: Implementation of a Partial Least Squares Regressor 2 .

Related