Space required to keep largest integer array in memory

user_2815227 0
Viewed 248

I am reviewing an algorithm and it keeps an array of integers, the size of input is dynamic. So according to my calculations it can take as much as

  integer MAX_VALUE  * int size  = ?   
      2^31 integers  * 4 bytes   = ?
2147483648 integers  * 4 bytes   = 8 Gigabytes

Is this calculation correct ? would the JVM use this much contiguous space to store the int array or are there other things that one needs to consider ?

jvm memory-management java
2 Answers

The theoretical size of the array would be :

  • numberOfElementsInTheArray * 4 bytes

  • 12 bytes of headers (int[] is an Object). Actually the size of headers depends on the flags you used and on the JVM version you are running this

  • 4 bytes to keep the length of the array

  • padding.

For example: (I am going to use JOL for this):

    int [] x = new int[10];
    for(int i=0;i<10;++i){
        x[i] = 9999;
    }
    System.out.println(GraphLayout.parseInstance((Object)x).toPrintable());

will output:

 [I@7a81197dd object externals:
      ADDRESS       SIZE TYPE PATH                           VALUE
    70fe45268         56 [I                                  [9999, 9999, 9999, 9999, 9999, 9999, 9999, 9999, 9999, 9999]

So it has 56 bytes:

  • 40 for the values themselves (10 ints * 4 bytes)
  • 12 for headers
  • 4 for length
  • 0 for padding

If you change this array to Integer, things change dramatically. Integer is an Object so you will store a reference inside the array (which could be 4 or 8 bytes, depending on UseCompressedOops flag), plus each of the Integer instances will require 2 headers (each Integer is an Object).

    Integer[] y = new Integer[10];
    for(int i=0;i<10;++i){
        y[i] = 9999;
    }

    System.out.println(GraphLayout.parseInstance((Object)y).toFootprint());

which will show:

   [Ljava.lang.Integer;@369f73a2d footprint:
 COUNT       AVG       SUM   DESCRIPTION
     1        56        56   [Ljava.lang.Integer;
    10        16       160   java.lang.Integer
    11                 216   (total)

A total of 216 bytes:

  • 4 bytes for each reference (I have UseCompressedOop turned on), 40 bytes total
  • 12 bytes headers of the array
  • 4 bytes length of the array
  • 0 bytes padding

Each reference from that array points to an Integer, each of those Objects will have 16 bytes:

  • 4 bytes for the inner int they hold
  • 12 bytes headers
  • 0 bytes padding
user_10593720

Array size maximum < Integer.MAX_VALUE

No, your maximum is incorrect.

The limit on number of elements in an array in Java is a little bit less than Integer.MAX_VALUE (2,147,483,647), depending on the version of Java, the host OS, and how Java was compiled. See this Answer by Ivan Mamontov on the Question, Why I can't create an array with large size?.

Yes, biggest array of int ≈ 8 gigs

So the size of a maximum array of int will be roughly ( Integer.MAX_VALUE - 8L ) * 32L bits which is 68,719,476,448 bits which is 8,589,934,556 octets.

So yes, around 8 gigs of memory. And remember: this is contiguous memory for an array. So:

  • There may be significant work on the part of the JVM and host OS to produce such an array depending how fragmented is memory at that moment during runtime.
  • If the host hardware does not have sufficient real memory, you'll be lapsing into virtual memory where the resulting paging may lead to terrible performance.

Always make real-world tests if you really are pushing these limits in your work. And you may want to consider alternative implementations of Java designed for very large memory, such as Zing by Azul Systems.

user_6427060
Related