TensorFlow 2 - does NumPy numerical values and arrays cause new graphs for TF Function?

user_4281353 0
Viewed 197

Does "numerical Python values" stated in the Hands on ML 2 include NumPy int, float, and array? Do we need to explicitly create a TF Tensor or a TF DataSet from a NumPy construct as the argument of a TF Function?

Hands on ML 2 Chapter 12 Auto Graph and Tracing:

By default, a TF Function generates a new graph for every unique set of intput shapes and data types and caches it for subsequent calls. ... However this is only true for tensor arguments: if you pass numerical Python values to a TF Function, a new graph will be generated for every distinct value. ...

if you pass numerical Python values to a TF function, a new graph will be generated for every distinct values. If you call a TF function many times with different numerical Python values, then many graphs will be generated, slowing down your program and using up a lot of RAM (you must delete the TF Function to release it).

Python values should be reserved for arguments that will have few unique values, such as hyper parameters like the number of neurons per layer. This allows TensorFlow to better optimize each variant fo yor model.

The 3rd rule stated in the TensorFlow document Rules of tracing corresponds with Python int, float, boolean, str , etc that will cause a new graph part. But not sure if the 5th rule (all other Python types) applies to NumPy constructs.

A Function determines whether to reuse a traced ConcreteFunction by computing a cache key from an input's args and kwargs. A cache key is a key that identifies a ConcreteFunction based on the input args and kwargs of the Function call, according to the following rules (which may change):

  1. The key generated for a tf.Tensor is its shape and dtype.

  2. The key generated for a tf.Variable is a unique variable id.

  3. The key generated for a Python primitive (like int, float, str) is its value.

  4. The key generated for nested dicts, lists, tuples, namedtuples, and attrs is the flattened tuple of leaf-keys (see nest.flatten). (As a result of this flattening, calling a concrete function with a different nesting structure than the one used during tracing will result in a TypeError).

  5. For all other Python types the key is unique to the object. This way a function or method is traced independently for each instance it is called with.

I suppose the fact tf.numpy_function exists suggests that the TF Function tracing will generate a new graph, but need a definite confirmation.

tensorflow2.0 numpy python
1 Answers

I dont think changing the values inside numpy arrays will generate a new graph. Consider the following minimal code exampls:

@tf.function 
def test(input): 
  print("Tracing with input= ", input)
  tf.print("Executing with input = ", input)

The first print is only executed during tracing, the second for every call. Calling it with a list leads to:

test([1,2])
test([3,4])
>>> Tracing with input =  [1, 2]
>>> Executing with input [1, 2]
>>> Tracing with input =  [3, 4]
>>> Executing with input [3, 4]

whereas calling it with numpy-arrays leads to:

test(np.array([1,2]))
test(np.array([3,4]))
>>> Tracing with input =  Tensor("input:0", shape=(2,), dtype=int32)
>>> Executing with input [1 2]
>>> Executing with input [3, 4]

Here no tracing is done for the second call. This suggests at least that numpy-arrays are handled the same way as tensorflow tensors.

user_164413040
Related