I'm writing a program (SpringBoot application) and I have to save 15 000 000 records to a table in a DB (I'm using MySQL). My logic states that I have to generate something on back-end (Java) and then to save generated value to the DB, which I did but the problem is that writing those records take so much time. I spent like 7 hours to write ~560 000 records. Now, I'm not sure if that's because my machine is weak but I think this has to do with something else.
I figured that maybe communication with DB is really slow so I tried to use file instead and that looked promising until "OutOfMemoryError occurred while reading file ..." happened.
Generating and printing these values in java console takes around 45 seconds but writing it to DB take forever and it seems file is not an option due to "OutOfMemoryError".
My question is: is there any other way to achieve this goal other than these two approaches? I tried to look for an answer on other places but without success.
EDIT: Way of generating records:
So, I'm using some Generator that creates different combinations and then in forEach I'm saving each record.
public void generateAllCombinations() {
Generator.combination(1, 40)
.simple(7)
.stream()
.forEach(comb -> {
NumberCombination generatedCombination = new NumberCombination(comb.toString(), false, ++NumberCombination.idCounter);
numberOfGeneratedCombinations++;
save(generatedCombination); // Save to the DB
});
}