Dart takes too much RAM while generating random numbers and appending them into a file

Viewed 58

I am doing a project that involves generating 1 billion random 0/1 numbers and storing them in a file. This is my code:

import 'dart:io';
import 'dart:math';

final int COUNT = 1000000000;
final Random randomizer = new Random();
final File file = new File('./dart/dart.txt');

void main() {
  if (file.existsSync()) {
    file.deleteSync();
  }
  file.createSync();
  
  DateTime before = new DateTime.now();
  IOSink sink = file.openWrite(mode: FileMode.append);

  for (int i = 0; i < COUNT; i++) {
    sink.write(nextNumber().toString());
  }
  sink.close();

  DateTime after = new DateTime.now();
  print(after.millisecondsSinceEpoch - before.millisecondsSinceEpoch);
}

int nextNumber() {
  return randomizer.nextInt(2);
}

It works fine with COUNT=10000, taking a dozen milliseconds to generate 10000 random numbers. However, as I raise COUNT to 1 billion, it took about 25GB of my RAM and ran for more than 40 minutes non-stop. I had to kill the process instead of letting it continue running.

How can I fix this?

P.S. I can't change the programming language since using different programming languages to generate such numbers are one aspect of my project. I also use C++, Java, Javascript, PS, Python, etc. to generate such big data, and none of them met this problem. The final file storing the data is less than 1 GB.

0 Answers
Related