In Java, what is the best way to determine the size of an object?

Viewed 343325

I have an application that reads a CSV file with piles of data rows. I give the user a summary of the number of rows based on types of data, but I want to make sure that I don't read in too many rows of data and cause OutOfMemoryErrors. Each row translates into an object. Is there an easy way to find out the size of that object programmatically? Is there a reference that defines how large primitive types and object references are for a VM?

Right now, I have code that says read up to 32,000 rows, but I'd also like to have code that says read as many rows as possible until I've used 32MB of memory. Maybe that is a different question, but I'd still like to know.

28 Answers

You can use the java.lang.instrument package.

Compile and put this class in a JAR:

import java.lang.instrument.Instrumentation;

public class ObjectSizeFetcher {
    private static Instrumentation instrumentation;

    public static void premain(String args, Instrumentation inst) {
        instrumentation = inst;
    }

    public static long getObjectSize(Object o) {
        return instrumentation.getObjectSize(o);
    }
}

Add the following to your MANIFEST.MF:

Premain-Class: ObjectSizeFetcher

Use the getObjectSize() method:

public class C {
    private int x;
    private int y;

    public static void main(String [] args) {
        System.out.println(ObjectSizeFetcher.getObjectSize(new C()));
    }
}

Invoke with:

java -javaagent:ObjectSizeFetcherAgent.jar C

Some years back Javaworld had an article on determining the size of composite and potentially nested Java objects, they basically walk through creating a sizeof() implementation in Java. The approach basically builds on other work where people experimentally identified the size of primitives and typical Java objects and then apply that knowledge to a method that recursively walks an object graph to tally the total size.

It is always going to be somewhat less accurate than a native C implementation simply because of the things going on behind the scenes of a class but it should be a good indicator.

Alternatively a SourceForge project appropriately called sizeof that offers a Java5 library with a sizeof() implementation.

P.S. Do not use the serialization approach, there is no correlation between the size of a serialized object and the amount of memory it consumes when live.

Firstly "the size of an object" isn't a well-defined concept in Java. You could mean the object itself, with just its members, the Object and all objects it refers to (the reference graph). You could mean the size in memory or the size on disk. And the JVM is allowed to optimise things like Strings.

So the only correct way is to ask the JVM, with a good profiler (I use YourKit), which probably isn't what you want.

However, from the description above it sounds like each row will be self-contained, and not have a big dependency tree, so the serialization method will probably be a good approximation on most JVMs. The easiest way to do this is as follows:

 Serializable ser;
 ByteArrayOutputStream baos = new ByteArrayOutputStream();
 ObjectOutputStream oos = new ObjectOutputStream(baos);
 oos.writeObject(ser);
 oos.close();
 return baos.size();

Remember that if you have objects with common references this will not give the correct result, and size of serialization will not always match size in memory, but it is a good approximation. The code will be a bit more efficient if you initialise the ByteArrayOutputStream size to a sensible value.

If you would just like to know how much memory is being used in your JVM, and how much is free, you could try something like this:

// Get current size of heap in bytes
long heapSize = Runtime.getRuntime().totalMemory();

// Get maximum size of heap in bytes. The heap cannot grow beyond this size.
// Any attempt will result in an OutOfMemoryException.
long heapMaxSize = Runtime.getRuntime().maxMemory();

// Get amount of free memory within the heap in bytes. This size will increase
// after garbage collection and decrease as new objects are created.
long heapFreeSize = Runtime.getRuntime().freeMemory();

edit: I thought this might be helpful as the question author also stated he would like to have logic that handles "read as many rows as possible until I've used 32MB of memory."

When using JetBrains IntelliJ, first enable "Attach memory agent" in File | Settings | Build, Execution, Deployment | Debugger.

When debugging, right-click a variable of interest and choose "Calculate Retained Size": Calculate Retained Size

You have to walk the objects using reflection. Be careful as you do:

  • Just allocating an object has some overhead in the JVM. The amount varies by JVM so you might make this value a parameter. At least make it a constant (8 bytes?) and apply to anything allocated.
  • Just because byte is theoretically 1 byte doesn't mean it takes just one in memory.
  • There will be loops in object references, so you'll need to keep a HashMap or somesuch using object-equals as the comparator to eliminate infinite loops.

@jodonnell: I like the simplicity of your solution, but many objects aren't Serializable (so this would throw an exception), fields can be transient, and objects can override the standard methods.

You have to measure it with a tool, or estimate it by hand, and it depends on the JVM you are using.

There is some fixed overhead per object. It's JVM-specific, but I usually estimate 40 bytes. Then you have to look at the members of the class. Object references are 4 (8) bytes in a 32-bit (64-bit) JVM. Primitive types are:

  • boolean and byte: 1 byte
  • char and short: 2 bytes
  • int and float: 4 bytes
  • long and double: 8 bytes

Arrays follow the same rules; that is, it's an object reference so that takes 4 (or 8) bytes in your object, and then its length multiplied by the size of its element.

Trying to do it programmatically with calls to Runtime.freeMemory() just doesn't give you much accuracy, because of asynchronous calls to the garbage collector, etc. Profiling the heap with -Xrunhprof or other tools will give you the most accurate results.

I was looking for a runtime calculation of an object size that met the following requirements:

  • Available at runtime with no need to include instrumentation.
  • Works with Java 9+ without access to Unsafe.
  • Is based on the Class only. Not a deep sizeOf that takes into consideration string lengths, array lengths, etc.

The following is based on the core code of the original java specialists article (https://www.javaspecialists.eu/archive/Issue078.html) and a few bits from the Unsafe version in another answer to this question.

I hope someone finds it useful.

public class JavaSize {

    private static final int NR_BITS = Integer.valueOf(System.getProperty("sun.arch.data.model"));
    private static final int BYTE = 8;
    private static final int WORD = NR_BITS / BYTE;
    private static final int HEADER_SIZE = 8;

    public static int sizeOf(Class<?> clazz) {
        int result = 0;

        while (clazz != null) {
            Field[] fields = clazz.getDeclaredFields();
            for (int i = 0; i < fields.length; i++) {
                if (!Modifier.isStatic(fields[i].getModifiers())) {
                    if (fields[i].getType().isPrimitive()) {
                        Class<?> primitiveClass = fields[i].getType();
                        if (primitiveClass == boolean.class || primitiveClass == byte.class) {
                            result += 1;
                        } else if (primitiveClass == short.class) {
                            result += 2;
                        } else if (primitiveClass == int.class || primitiveClass == float.class) {
                            result += 4;
                        } else if (primitiveClass == double.class || primitiveClass == long.class) {
                            result += 8;
                        }

                    } else {
                        // assume compressed references.
                        result += 4;
                    }
                }
            }

            clazz = clazz.getSuperclass();

            // round up to the nearest WORD length.
            if ((result % WORD) != 0) {
                result += WORD - (result % WORD);
            }
        }

        result += HEADER_SIZE;

        return result;
    }
 }

There isn't a method call, if that's what you're asking for. With a little research, I suppose you could write your own. A particular instance has a fixed sized derived from the number of references and primitive values plus instance bookkeeping data. You would simply walk the object graph. The less varied the row types, the easier.

If that's too slow or just more trouble than it's worth, there's always good old-fashioned row counting rule-of-thumbs.

I wrote a quick test once to estimate on the fly:

public class Test1 {

    // non-static nested
    class Nested { }

    // static nested
    static class StaticNested { }

    static long getFreeMemory () {
        // waits for free memory measurement to stabilize
        long init = Runtime.getRuntime().freeMemory(), init2;
        int count = 0;
        do {
            System.out.println("waiting..." + init);
            System.gc();
            try { Thread.sleep(250); } catch (Exception x) { }
            init2 = init;
            init = Runtime.getRuntime().freeMemory();
            if (init == init2) ++ count; else count = 0;
        } while (count < 5);
        System.out.println("ok..." + init);
        return init;
    }

    Test1 () throws InterruptedException {

        Object[] s = new Object[10000];
        Object[] n = new Object[10000];
        Object[] t = new Object[10000];

        long init = getFreeMemory();

        //for (int j = 0; j < 10000; ++ j)
        //    s[j] = new Separate();

        long afters = getFreeMemory();

        for (int j = 0; j < 10000; ++ j)
            n[j] = new Nested();

        long aftersn = getFreeMemory();

        for (int j = 0; j < 10000; ++ j)
            t[j] = new StaticNested();

        long aftersnt = getFreeMemory();

        System.out.println("separate:      " + -(afters - init) + " each=" + -(afters - init) / 10000);
        System.out.println("nested:        " + -(aftersn - afters) + " each=" + -(aftersn - afters) / 10000);
        System.out.println("static nested: " + -(aftersnt - aftersn) + " each=" + -(aftersnt - aftersn) / 10000);

    }

    public static void main (String[] args) throws InterruptedException {
        new Test1();
    }

}

General concept is allocate objects and measure change in free heap space. The key being getFreeMemory(), which requests GC runs and waits for the reported free heap size to stabilize. The output of the above is:

nested:        160000 each=16
static nested: 160000 each=16

Which is what we expect, given alignment behavior and possible heap block header overhead.

The instrumentation method detailed in the accepted answer here the most accurate. The method I described is accurate but only under controlled conditions where no other threads are creating/discarding objects.

A possible year 2022 answer.

https://github.com/ehcache/sizeof

https://mvnrepository.com/artifact/org.ehcache/sizeof

https://mvnrepository.com/artifact/org.ehcache/sizeof/0.4.0

Version 0.4.0 has only a (compile) dependency on

https://mvnrepository.com/artifact/org.slf4j/slf4j-api

which is a good thing.

Sample code:

//import org.ehcache.sizeof.SizeOf;

SizeOf sizeOf = SizeOf.newInstance(); // (1)
long shallowSize = sizeOf.sizeOf(someObject); // (2)
long deepSize = sizeOf.deepSizeOf(someObject); // (3)

Suppose I declare a class named Complex like:

public class Complex {

    private final long real;
    private final long imaginary;

    // omitted
}

In order to see how much memory is allocated to live instances of this class:

$ jmap -histo:live <pid> | grep Complex

 num     #instances         #bytes  class name (module)
-------------------------------------------------------
 327:             1             32  Complex

If your application has the Apache commons lang library as a dependency or is using the Spring framework then you can also use the SerializationUtils class to quickly find out the approximate byte size of any given object.

byte[] data = SerializationUtils.serialize(user);
System.out.println("Approximate object size in bytes " + data.length);

I doubt you want to do it programmatically unless you just want to do it once and store it for future use. It's a costly thing to do. There's no sizeof() operator in Java, and even if there was, it would only count the cost of the references to other objects and the size of the primitives.

One way you could do it is to serialize the thing to a File and look at the size of the file, like this:

Serializable myObject;
ObjectOutputStream oos = new ObjectOutputStream (new FileOutputStream ("obj.ser"));
oos.write (myObject);
oos.close ();

Of course, this assumes that each object is distinct and doesn't contain non-transient references to anything else.

Another strategy would be to take each object and examine its members by reflection and add up the sizes (boolean & byte = 1 byte, short & char = 2 bytes, etc.), working your way down the membership hierarchy. But that's tedious and expensive and ends up doing the same thing the serialization strategy would do.

Related