Is it possible to get shape of a TFdataset similar to a pandas.shape()?

user_640558 0
Viewed 45

I execute this command I get:

print(df.shape)
(172717, 1521)

but if I convert the dataframe to a tf dataframe:

  df_numpy = df.to_numpy()
  df_tensor = tf.ragged.constant(df_numpy)

I can see rows but not columns:

len(list(df_tensor))
172717

I have a ML model and I want the input layer to be generated dynamically so I need the columns to add them together to create the input shape:

input_shape = (df.shape[1] + df2.shape[1] + df3.shape[1]))

obviously, this works with pandas but it doesn't work with tf datasets because I cannot get the shape.

Is there a way to get around this?

keras tensorflow python
1 Answers

Ragged tensors (tf.RaggedTensor), are tensors with non-uniform shapes, thus using .get_shape(), you only get [num_of_rows, None].

df = pd.DataFrame({'Roll': [1, 2, 3, 4, 5],
                   'Mark': [95, 96, 98, 100, 95],
                   })

tf_df = tf.ragged.constant(df.to_numpy())
tf_df.get_shape()
TensorShape([5, None])

However, here are some of the ways to get full shape info

tf_df.to_sparse().shape
(5, 2)

tf_df.to_tensor().shape
(5, 2)

np.array(tf_df.to_list()).shape
(5, 2)
user_92157800
Related