FutureWarning: arrays to stack must be passed as a "sequence" type such as list or tuple. Support for non-sequence iterables is deprecated

user_10908375 0
Viewed 4756

I'm trying to turn a Tensorflow Dataset into a NumPy array and I'm getting a deprecation warning:

FutureWarning: arrays to stack must be passed as a "sequence" type such as list or tuple. Support for non-sequence iterables such as generators is deprecated as of NumPy 1.16 and will raise an error in the future.

What is the "new" way of doing this?

Reproducible example with numpy 1.18.1 and tensorflow 2.1.0:

import tensorflow as tf
import numpy as np

ds = tf.data.Dataset.enumerate(tf.data.Dataset.range(20, 40))

np.vstack(ds)

What I tried: I can only do it with one dimension at a time.

np.fromiter(ds.map(lambda x, y: x), float)
tensorflow numpy python
2 Answers

With just numpy, and using a generator expression:

In [105]: np.stack((np.ones(3) for _ in range(3)))                                      
/usr/local/lib/python3.6/dist-packages/IPython/core/interactiveshell.py:3254: 
FutureWarning: arrays to stack must be passed as a "sequence" type 
such as list or tuple. Support for non-sequence iterables such as generators 
is deprecated as of NumPy 1.16 and will raise an error in the future.:
Out[105]: 
array([[1., 1., 1.],
       [1., 1., 1.],
       [1., 1., 1.]])

Using a list comprehension to create the arrays:

In [106]: np.stack([np.ones(3) for _ in range(3)])                                      
Out[106]: 
array([[1., 1., 1.],
       [1., 1., 1.],
       [1., 1., 1.]])

I don't use tf, so can only guess as to what:

tf.data.Dataset.enumerate(tf.data.Dataset.range(20, 40))

produces. But as I understand it tensorflow has a distinction between tensors that 'generator-like', potential executions (pipeline?), and 'eager' evaluation, in which the tensors, and tensor expressions, are actually evaluated, producing arrays (or similar object). np.stack tries to convert its inputs into arrays.

A code example for Tf.data.Dataset.enumerate:

dataset = tf.data.Dataset.from_tensor_slices([1, 2, 3])
dataset = dataset.enumerate(start=5)
for element in dataset.as_numpy_iterator():
  print(element)

enumerate returns an interator. You still have to iterate it, as in this for loop. OR list(dataset).

https://www.tensorflow.org/api_docs/python/tf/data/Dataset#enumerate

user_9019250

You can use tfds to do so, you can check out the documentation here.

Example how to use:

array = tfds.as_numpy(ds)
user_130280600
Related