H2O Target Encoder Mojo Model throwing NULL Pointer Exception

Viewed 228

I'm building a target encoder model using the given python example on h2o documentation and trying to predict the target encodings through java using mojo of this model. But the mojo prediction fails on the categories which are present in test data only and not in training data with following error

Exception in thread "main" java.lang.NullPointerException
    at hex.genmodel.algos.targetencoder.TargetEncoderMojoModel.computeEncodings(TargetEncoderMojoModel.java:87)
    at hex.genmodel.algos.targetencoder.TargetEncoderMojoModel.score0(TargetEncoderMojoModel.java:72)
    at hex.genmodel.easy.EasyPredictModelWrapper.predict(EasyPredictModelWrapper.java:889)
    at hex.genmodel.easy.EasyPredictModelWrapper.transformWithTargetEncoding(EasyPredictModelWrapper.java:618)
    at main.main(main.java:26)

After digging into the target encoder mojo, found that categories which are present in test data only, are present in domains.txt, so the target encoder doesn't treat these categories as missing categories. But the target encodings are missing for these categories from encoding_map.ini, so model throws NullPointerException when it tries to access encodings for such categories using encoding_map.ini

Code to train model:

h2o.init()
from h2o.estimators import H2OTargetEncoderEstimator

titanic = h2o.import_file("https://s3.amazonaws.com/h2o-public-test-data/smalldata/gbm_test/titanic.csv")
titanic['survived'] = titanic['survived'].asfactor()
response='survived'

train, test = titanic.split_frame(ratios = [.5], seed = 1234)

encoded_columns = ["home.dest", "cabin", "embarked"]

blended_avg= True
inflection_point = 3
smoothing = 10
noise = 0.15
data_leakage_handling = "k_fold"
fold_column = "kfold_column"
train[fold_column] = train.kfold_column(n_folds=5, seed=3456)

titanic_te = H2OTargetEncoderEstimator(fold_column=fold_column,
                                       data_leakage_handling=data_leakage_handling,
                                       blending=blended_avg,
                                       k=inflection_point,
                                       f=smoothing)

titanic_te.train(x=encoded_columns,
                 y=response,
                 training_frame=train)

titanic_te.download_mojo(get_genmodel_jar=True)

Code to get encodings:

import java.io.*;
import java.util.Arrays;
import hex.genmodel.easy.RowData;
import hex.genmodel.easy.EasyPredictModelWrapper;
import hex.genmodel.easy.prediction.*;
import hex.genmodel.MojoModel;
import hex.genmodel.algos.targetencoder.TargetEncoderMojoModel;

public class main {

    public static void main(String[] args) throws Exception {

        EasyPredictModelWrapper model = new EasyPredictModelWrapper(MojoModel.load("TargetEncoder_model_python_1599838802418_2.zip"));

        String[] temp_home = { "?Havana  Cuba", "Aberdeen / Portland  OR", "Albany  NY", "Altdorf  Switzerland",
                               "Amenia  ND", "Antwerp  Belgium / Stanton  OH", "Asarum  Sweden Brooklyn  NY",
                               "Ascot  Berkshire / Rochester  NY", "Auburn  NY", "Aughnacliff  Co Longford  Ireland New York  NY",
                               "Australia Fingal  ND", "Austria Niagara Falls  NY", "Austria-Hungary",
                               "Austria-Hungary / Germantown  Philadelphia  PA"};

        for(int j=0; j<temp_home.length; j++){
            RowData row = new RowData();
            row.put("cabin", "D43");
            row.put("embarked", "C");
            row.put("home.dest", temp_home[j]);

            TargetEncoderPrediction p = model.transformWithTargetEncoding(row);
            System.out.println(Arrays.toString(p.transformations));
        }
    }
}

Compile Command: javac -cp h2o-genmodel.jar -J-Xmx2g main.java

Run Command: java -cp .:h2o-genmodel.jar main

0 Answers
Related