How to understand the target of tf.Session?

user_8165066 0
Viewed 1012

The tf.Session has parameter target, it can be '', grpc://ip:port, or tf.train.Server().target, what is the difference between them? And when we create a session, does it mean we create a client, if we have client what is server?

Update

  1. What is the master? Is it chief worker? And what is difference between '' and grpc://localhost:port? I think they are both on the local computer.

Some questions about your local_distributed_benchmark.py

  1. I add print(add_op.device) in create_graph(device1, device2), the output is /job:worker/task:0, why not /job:worker/task:1? And you create tf.Session("grpc://"+host+FLAGS.port1, config=default_config()), if ops is created on other tasks(worker), how does this session run those ops, like init_op = tf.initialize_all_variables() on your program.

  2. If I use sess = tf.Session(config=default_config()) instead of sess = tf.Session("grpc://"+host+FLAGS.port1, config=default_config()), it will raise an error:

Cannot assign a device for operation 'update': Operation was explicitly assigned to /job:worker/task:1 but avail able devices are [ /job:localhost/replica:0/task:0/cpu:0, /job:localhost/replica:0/task:0/gpu:0 ]. What's wrong?

  1. If I delete runcmd("python %s --task=1"%(sys.argv[0])) and change with tf.device(device2): to with tf.device(device1):, the program will output CreateSession still waiting for response from worker: /job:worker/replica:0/task:1. I only use session of target task:0, and don't use any ops on task:1, so why does program wait for task:1 worker to start?
grpc tensorflow python c++
1 Answers

It's the location of session master (the thing that can execute session.run calls).

  • '' is local master (local runtime).

  • tf.train.Server(...).target is local master (distributed runtime). It has the form grpc://localhost:port

  • grpc://ip:port is the master listening on ip

Here's some explanation of terminology (master, worker, client): task assignment in tensorflow distributed process

user_4191160
Related