Why seed parameter doesn't work with make_tf_dataset function?

Viewed 13

Random seed feature doesn't work with make_tf_dataset function. In the next code:

#Create fake dataset
ratings = spark.createDataFrame([
    {'user_id':0, 'movie_id': 3}, 
    {'user_id': 2, 'movie_id': 5}, 
    {'user_id':4, 'movie_id': 7}, 
    {'user_id':6, 'movie_id': 9}, 
    {'user_id':8, 'movie_id': 11}, 
    {'user_id':10, 'movie_id': 13}, 
    {'user_id':12, 'movie_id': 15}, 
    {'user_id':14, 'movie_id': 17}
])

#Create converter
conv_train = make_spark_converter(ratings)

#Get two datasets(train and train1) from the same converter
with conv_train.make_tf_dataset(batch_size=2, num_epochs=4, seed=1) as train, \
     conv_train.make_tf_dataset(batch_size=2, num_epochs=4, seed=1) as train1:
     
     #Iterate over the datasets and print elements of each batch b and b1 which must be equals
     for i, (b, b1) in enumerate(zip(train, train1)):
         print('batch {0}'.format(i))
         print('m: {0}'.format([i for i in b]))
         print('m1: {0}'.format([i for i in b1]))

The problem is that the batches doesn't have the same elements, even using the same seed.

0 Answers
Related