Random seed feature doesn't work with make_tf_dataset function. In the next code:
#Create fake dataset
ratings = spark.createDataFrame([
{'user_id':0, 'movie_id': 3},
{'user_id': 2, 'movie_id': 5},
{'user_id':4, 'movie_id': 7},
{'user_id':6, 'movie_id': 9},
{'user_id':8, 'movie_id': 11},
{'user_id':10, 'movie_id': 13},
{'user_id':12, 'movie_id': 15},
{'user_id':14, 'movie_id': 17}
])
#Create converter
conv_train = make_spark_converter(ratings)
#Get two datasets(train and train1) from the same converter
with conv_train.make_tf_dataset(batch_size=2, num_epochs=4, seed=1) as train, \
conv_train.make_tf_dataset(batch_size=2, num_epochs=4, seed=1) as train1:
#Iterate over the datasets and print elements of each batch b and b1 which must be equals
for i, (b, b1) in enumerate(zip(train, train1)):
print('batch {0}'.format(i))
print('m: {0}'.format([i for i in b]))
print('m1: {0}'.format([i for i in b1]))
The problem is that the batches doesn't have the same elements, even using the same seed.