Numpy array shape mismatch with modAL and skLearn

Viewed 57

I'm stuck on a numpy array shape mismatch error. There are dozens of such questions on StackOverflow but none have helped me solve the problem. I'm struggling with the debugging because the error is raised by library code. Here's the relevant part:

from modAL.models import ActiveLearner

# ... fetch data

print("Beginning debugging logs:")
print(f"classifier: {clf}")
print(f"X_train shape: {X_train.shape}")
print(f"y_train shape: {y_train.shape}")
print(f"X_pool shape: {X_POOL.shape}")

learner = ActiveLearner(
    estimator=clf,
    X_training=X_train,
    y_training=y_train
    )

result = learner.query(X_POOL)

Here is the traceback:

classifier: DecisionTreeClassifier(max_depth=4)
X_train shape: (84, 4926)
y_train shape: (84, 51)
X_pool shape: (997, 4926)


  File "rpc_server.py", line 139, in <module>
    result = learner.query(X_POOL)
  File "/home/ubuntu/venv/lib/python3.8/site-packages/modAL/models/base.py", line 261, in query
    query_result = self.query_strategy(self, X_pool, *query_args, **query_kwargs)
  File "/home/ubuntu/venv/lib/python3.8/site-packages/modAL/uncertainty.py", line 152, in uncertainty_sampling
    uncertainty = classifier_uncertainty(classifier, X, **uncertainty_measure_kwargs)
  File "/home/ubuntu/venv/lib/python3.8/site-packages/modAL/uncertainty.py", line 82, in classifier_uncertainty
    uncertainty = 1 - np.max(classwise_uncertainty, axis=1)
  File "/home/ubuntu/venv/lib/python3.8/site-packages/numpy/core/fromnumeric.py", line 2504, in amax
    return _wrapreduction(a, np.maximum, 'max', axis, None, out, keepdims=keepdims,
  File "/home/ubuntu/venv/lib/python3.8/site-packages/numpy/core/fromnumeric.py", line 86, in _wrapreduction
    return ufunc.reduce(obj, axis, dtype, out, **passkwargs)
ValueError: could not broadcast input array from shape (997,1) into shape (997)

The environment is Python3.8 with:

pandas==1.1.4
numpy==1.16.0
sklearn==0.0
modAL==0.4.0

I believe that (997,1) is just a trivially nested (997) array? It doesn't seem too far off what it needs to be here.

How can I debug this error? Thank you!

1 Answers

The ValueError you are receiving seems to be a problem with the line:

uncertainty = 1 - np.max(classwise_uncertainty, axis=1)

Perhaps you can simply avoid broadcasting altogether at the ufunc.reduce level with:

uncertainty = 1 - (np.max(classwise_uncertainty, axis=1, keepdims=1)).flatten()

I should note however, I was unable to reproduce the Error on my machine. In the worst case scenario, you could simply reformulate that line of code manually (using a loop or two) to reflect the same mathematics it encodes.

Related