I successfully setup a mini Ray cluster (1 head + 1 worker, each with 4 CPU cores) manually. However, I failed to set up it automatically using the Apache Ray autoscaler. The head node starts correctly while the worker node never joins the cluster. Below is my YAML configuration for the autoscaler. Is there anything I did wrong?
cluster_name: my_ray_cluster
min_workers: 8
initial_workers: 8
max_workers: 8
provider:
type: local
head_ip: 10.148.186.178
worker_ips: [10.148.186.18]
auth:
ssh_user: USER_NAME
ssh_private_key: ~/.ssh/id_rsa
# Files or directories to copy to the head and worker nodes.
file_mounts: {
# "/path1/on/remote/machine": "/path1/on/local/machine",
# "/path2/on/remote/machine": "/path2/on/local/machine",
}
head_setup_commands:
- pip3 install ray[debug,dashboard]
setup_commands:
- pip3 install ray[debug,dashboard]
# Command to start ray on the head node. You don't need to change this.
head_start_ray_commands:
- ray stop
- ray start --head --redis-port=6379
worker_start_ray_commands:
- ray stop
- ray start --address=10.148.186.178:6379