I'm trying to learn the basics of Tune. In the following script, I would expect each worker to run for 100 iterations and then end however, the workers end before reaching 100 iterations with state 3 ( TypeError? ). I do not see any error messages so I might be confused as to what is actually supposed to happen. Out of 10 samples, only 2 reach 100 iterations. The rest of the samples are between 5 and 16 iterations.
"""Testing Tune with CartPole."""
import ray
from ray import tune
from ray.tune.schedulers import AsyncHyperBandScheduler
from ray.tune.suggest.bayesopt import BayesOptSearch
if __name__ == "__main__":
tune_metric = "info/learner/default_policy/critic_loss"
space = {"gamma": (0.01, 1)}
algo = BayesOptSearch(
space,
metric=tune_metric,
mode="min",
utility_kwargs={
"kind": "ucb",
"kappa": 2.5,
"xi": 0.0
})
scheduler = AsyncHyperBandScheduler(metric=tune_metric, mode="min")
ray.init()
analysis = tune.run(
"SAC",
stop={"training_iteration": 100},
search_alg=algo,
scheduler=scheduler,
num_samples=10,
config={
"env": "CartPole-v0",
},
)
print("Best config: ", analysis.get_best_config(metric=tune_metric,
mode="min"))
When I attempt to run the following example, the same thing occurs ( mnist pytorch trainable )