Eat memory using Python

user_764592 0
Viewed 33444

I am trying to create an app that can "Purposely" consume RAM as much as we specify immediately. e.g. I want to consume 512 MB RAM, then the app will consume 512 MB directly.

I have search on the web, most of them are using while loop to fill the ram with variable or data. But I think it is slow way to fill the RAM and might not accurate either.

I am looking for a library in python about memory management. and came across these http://docs.python.org/library/mmap.html. But can't figure out how to use these library to eat the RAM Space in one shot.

I ever saw an mem-eater application, but don't know how they were written...

So, is there any other better suggestion for Fill the RAM with random data immediately? Or Should I just use while loop to fill the data manually but with Multi-Threading to make it faster?

memory memory-management python
6 Answers
x = bytearray(1024*1024*1000)

Eats about 1GB of memory

user_138069340

This function will allocate memory into a list of bytes objects. Each item in the list will effectively be unique and of the same length. The function also logs its allocations. I have tested it up to 3.7 TiB. It uses the humanfriendly package but you can remove its use if you don't want it.

It does use a loop, but at least it lets you optionally customize how much to allocate in each iteration. For example, you can use an 8-fold higher value for multiplier_per_allocation.

import logging
import secrets
from typing import Optional

from humanfriendly import format_size

log = logging.getLogger(__name__)


def fill_memory(*, num_unique_bytes_per_allocation: int = 1024, multiplier_per_allocation: int = 1024 ** 2, max_allocations: Optional[int] = None) -> None:
    """Allocate available memory into a list of effectively unique bytes objects.

    This function is for diagnostic purposes.

    :param num_unique_bytes_per_allocation: Each allocation is created by multiplying a random sequence of bytes of this length.
    :param multiplier_per_allocation: Each allocation is created by multiplying the random sequence of bytes by this number.
    :param max_allocations: Optional number of max allocations.
    """
    # Ref: https://stackoverflow.com/a/66109163/
    num_allocation_bytes = num_unique_bytes_per_allocation * multiplier_per_allocation
    log.info(
        f"Allocating cumulative instances of {num_allocation_bytes:,} bytes ({format_size(num_allocation_bytes)}) each. "
        f"Each allocation uses {num_unique_bytes_per_allocation:,} unique bytes ({format_size(num_unique_bytes_per_allocation)}) "
        f"with a multiplier of {multiplier_per_allocation:,} ({format_size(multiplier_per_allocation)})."
    )

    # Allocate memory
    allocated = []
    num_allocation = 1
    while True:
        unique_bytes_for_allocation = secrets.token_bytes(num_unique_bytes_per_allocation)
        allocated.append(unique_bytes_for_allocation * multiplier_per_allocation)
        num_total_bytes_allocated = num_allocation * num_allocation_bytes
        log.info(f"Used a total of {num_total_bytes_allocated:,} bytes ({format_size(num_total_bytes_allocated)}) via {num_allocation:,} allocations.")
        if max_allocations and (max_allocations == num_allocation):
            break
        num_allocation += 1

Sample output:

>>> import logging
>>> logging.basicConfig(level=logging.INFO)

>>> fill_memory()

INFO:Allocating cumulative instances of 1,073,741,824 bytes (1 GiB) each. Each allocation uses 1,024 unique bytes (1 KiB) with a multiplier of 1,048,576 (1 MiB).
INFO:Used a total of 1,073,741,824 bytes (1 GiB) via 1 allocations.
INFO:Used a total of 2,147,483,648 bytes (2 GiB) via 2 allocations.
INFO:Used a total of 3,221,225,472 bytes (3 GiB) via 3 allocations.
INFO:Used a total of 4,294,967,296 bytes (4 GiB) via 4 allocations.
user_8322300
Related