unexpected predictions behavior using VW cover

Viewed 85

I am using vowpalwabbit for a contextual bandit problem. I want to use the cover option as explained here.

I am facing 2 issues with this:

  • once the learning phase is over and I use the vw model to make predictions, these predictions (in this case, pmf over the actions) are not stable
  • if I save the model and reload it to memory, the predictions are different

Here is an example (using the python wrapper of VW):

import vowpalwabbit.pyvw as pyvw

data_train = ["1:0:0.5 |features a b", "2:-1:0.5 |features a c", "2:0:0.5 |features b c",
              "1:-2:0.5 |features b d", "2:0:0.5 |features a d", "1:0:0.5 |features a c d",
              "1:-1:0.5 |features a c", "2:-1:0.5 |features a c"]
data_test = ["|features a b", "|features a b"]

model1 = pyvw.vw(cb_explore=2, cover=10)

for data in data_train:
    model1.learn(data)

model1.save("saved_model.model")
model2 = pyvw.vw(cb_explore=2, cover=10, i="saved_model.model")

for data in data_test:
    print(data)
    print(model1.predict(data))
    print(model2.predict(data))

I get the following output:

|features a b
[0.75, 0.25]
[0.5, 0.5]
|features a b
[0.7642977237701416, 0.2357022762298584]
[0.5, 0.5]

As you can see, predictions for model 1 are changing (slightly) while predictions for model 2 (which should be the same as model 1) are different.

If I replace cover with bag, I do not get this problem. What is the explanation for this, and is there a way to fix it in VW?

1 Answers

Thank you for reporting this, this seems like a bug.

I have opened an issue for this here so you can follow the progress.

Related