What is "seed" for in splitting test-val data in Python and how to come up with a correct number?

user_8073417 0
Viewed 1830

I'm trying to split my image dataset so it can have a training set and validation set. I found this Python's library called split-folders. The syntax is easy to understand

splitfolders.ratio("input_folder", output="output", seed=1337, ratio=(.8, .1, .1), group_prefix=None)

But I don't know about this seed parameter and what it does. The description on the page only says that "a seed makes splits reproducible" and that "it shuffles the items" but it doesn't really explain anything for me. I have googled about it and none of them gave me a clear answer. Anyone can give me a brief explanation?

The default number is 1337, but why? What does it mean to have the seed set to 1337? How did they come up with that number? How do I find the correct seed for my dataset?

random-seed python
1 Answers

When you split your corpus to train, validate, and test set, you randomly assign one data point to one of these three sets. Randomness is traceable using seeds.

Imagine, you have a random generator, a BlackBox, that gives you a series of random numbers; But for each given seed, the sequence it generates will be always identical. For example, for seed=1337, a random generator will always generate a sequence of random numbers like 12,901,110,1,.... on the same computer.

Why we care about tracing the randomness, especially in the case of dividing the corpus for training? Because most of the time, you want to repeat the same experiment, with the same data. So if you do not use the seed value, each time you run the same experiment, you will end up with different settings for training.

The seed value itself is not important, as long as you get it by some value you know it is fixed during your experiments. I personally set it to a prime number.

user_110653470
Related