I'm building a target encoder model using the given python example on h2o documentation and trying to predict the target encodings through java using mojo of this model. But the mojo prediction fails on the categories which are present in test data only and not in training data with following error
Exception in thread "main" java.lang.NullPointerException
at hex.genmodel.algos.targetencoder.TargetEncoderMojoModel.computeEncodings(TargetEncoderMojoModel.java:87)
at hex.genmodel.algos.targetencoder.TargetEncoderMojoModel.score0(TargetEncoderMojoModel.java:72)
at hex.genmodel.easy.EasyPredictModelWrapper.predict(EasyPredictModelWrapper.java:889)
at hex.genmodel.easy.EasyPredictModelWrapper.transformWithTargetEncoding(EasyPredictModelWrapper.java:618)
at main.main(main.java:26)
After digging into the target encoder mojo, found that categories which are present in test data only, are present in domains.txt, so the target encoder doesn't treat these categories as missing categories. But the target encodings are missing for these categories from encoding_map.ini, so model throws NullPointerException when it tries to access encodings for such categories using encoding_map.ini
Code to train model:
h2o.init()
from h2o.estimators import H2OTargetEncoderEstimator
titanic = h2o.import_file("https://s3.amazonaws.com/h2o-public-test-data/smalldata/gbm_test/titanic.csv")
titanic['survived'] = titanic['survived'].asfactor()
response='survived'
train, test = titanic.split_frame(ratios = [.5], seed = 1234)
encoded_columns = ["home.dest", "cabin", "embarked"]
blended_avg= True
inflection_point = 3
smoothing = 10
noise = 0.15
data_leakage_handling = "k_fold"
fold_column = "kfold_column"
train[fold_column] = train.kfold_column(n_folds=5, seed=3456)
titanic_te = H2OTargetEncoderEstimator(fold_column=fold_column,
data_leakage_handling=data_leakage_handling,
blending=blended_avg,
k=inflection_point,
f=smoothing)
titanic_te.train(x=encoded_columns,
y=response,
training_frame=train)
titanic_te.download_mojo(get_genmodel_jar=True)
Code to get encodings:
import java.io.*;
import java.util.Arrays;
import hex.genmodel.easy.RowData;
import hex.genmodel.easy.EasyPredictModelWrapper;
import hex.genmodel.easy.prediction.*;
import hex.genmodel.MojoModel;
import hex.genmodel.algos.targetencoder.TargetEncoderMojoModel;
public class main {
public static void main(String[] args) throws Exception {
EasyPredictModelWrapper model = new EasyPredictModelWrapper(MojoModel.load("TargetEncoder_model_python_1599838802418_2.zip"));
String[] temp_home = { "?Havana Cuba", "Aberdeen / Portland OR", "Albany NY", "Altdorf Switzerland",
"Amenia ND", "Antwerp Belgium / Stanton OH", "Asarum Sweden Brooklyn NY",
"Ascot Berkshire / Rochester NY", "Auburn NY", "Aughnacliff Co Longford Ireland New York NY",
"Australia Fingal ND", "Austria Niagara Falls NY", "Austria-Hungary",
"Austria-Hungary / Germantown Philadelphia PA"};
for(int j=0; j<temp_home.length; j++){
RowData row = new RowData();
row.put("cabin", "D43");
row.put("embarked", "C");
row.put("home.dest", temp_home[j]);
TargetEncoderPrediction p = model.transformWithTargetEncoding(row);
System.out.println(Arrays.toString(p.transformations));
}
}
}
Compile Command: javac -cp h2o-genmodel.jar -J-Xmx2g main.java
Run Command: java -cp .:h2o-genmodel.jar main