I've got too little RAM to finish a calculation because of a large hash. Is there a drop-in Perl module that would let me use the hash without keeping it all in RAM? I expect it to top out around 4GB, and I've got a bit less than 2GB available for the script. I don't think processing time or disk I/O would be an issue.
2 Answers
You can use dbm_open to open a hash connected to a DBM file. These are not particularly sophisticated and can handle shallow hashes of simple keys and values.
For anything more sophisticated, I would recommend using SQLite.
You may try DB_File module (or similar modules).
Memory usage hints: https://www.perlmonks.org/?node_id=146377
Take a look at AnyDBM_File for other similar modules available with rudimentary comparison.
$hash{$key1,$key2} syntax can be used to turn multi level hash into flat (single level) hash.
see $SUBSCRIPT_SEPARATOR in man perlvar for details.
Related