Dealing with tensor shapes while using Model.fit()

Question

I'm implementing Tacotron2 in Tensorflow for my own purpose and I fail to train it using the Model.fit() method. My tf.data.Dataset generates two tuples (input and output) as follows : (phonemes, mel_spec), (mel_spec, gates) where phonemes are strings of various sizes, mel_spec are spectrogram 2D tensor of various length and fixed channels (80 here), and gates are 1d tensor with same length as mel_spec (representing a stop prediction). Tacotron makes use of teacher forcing, that's why mel_specs is both in input and output tuples.

Since the inputs have different lengths, I use the padded_batch method as following :

dataset = dataset.padded_batch(batch_size, 
        padding_values=((None, None), (None, 1.)) )

With padding value 1 only for the gates. When checking what it returns with print(next(iter(dataset))), everything sounds good : shapes, padding ... Moreover, when feeding a full batch manually like this :

x, y = next(iter(dataset.padded_batch(batch_size, padding_values=((None, None), (None, 1.)) )))
mels, gates = tac(x)

Everything works and it returns tensor with correct shapes.

However it turns out that I can't go through the fit method. When I do this,

dataset = dataset.padded_batch(batch_size, 
        padding_values=((None, None), (None, 1.)) )

"""
train
"""
optimizer = conf["train"]["optimizer"]
epochs = conf["train"]["epochs"]

tac.compile(optimizer=optimizer, loss=tac.criterion)
tac.fit(dataset, epochs=epochs)

I get :

Tacotron2.py:177 call  *
        crop = mels.shape[2] - mels.shape[2]%self.config["n_frames_per_step"]#max_len must be a multiple of n_frames_per_step
TypeError: unsupported operand type(s) for %: 'NoneType' and 'int'

and here is my call function,

def call(self, batch, training=False):

    phon, mels = batch
    x = self.tokenizer(phon)
    x = self.char_embedding(x)
    y = self.encoder(x)
    print(y.shape)


    crop = mels.shape[2] - mels.shape[2]%self.config["n_frames_per_step"]#max_len must be a multiple of n_frames_per_step
    mels, gates = self.decoder(y, mels[:,:,:crop])

    residual = self.decoder.postnet(mels)
    mels_post = mels + residual
return (mels, mels_post), gates

It seems that fit is giving an empty tensor to my model, and since I need to access to the shape of my tensor in order to crop it, it raises an error. I have no idea why Tensorflow is doing this, I've mainly worked so far with Pytorch so I'm still learning the basics of TF. Anybody knows what's wrong with my way of doing it ?

Dealing with tensor shapes while using Model.fit()

0 Answers