Backgrounds of heap size measurements

Hello,

after building a JVMTI agent for measuring the size of the java heap (in a first step) I found out about many other ways of measuring the heap. Initially all of them seem to be contradictive to my agent, but step by step I brought some of them in harmony.

My agents is interested in measuring the size of live-objects, thats why it is based on iterateOverReachableObjects().

Currently I have my agents output values and understood the differences which occur on comparison with some of the other techniques.

Those measurements are understood and are no more concadictive:

- JVMTI demo HeapViewer in JDK 1.5

- JVM option -XX:+PrintClassHistogram

- Using -agentlib:hprof=heap=sites to get a heap dump and using jhat to analyse it (which among other things give a total size)

What remains unexplained and contradictive (to my agent and the other techniques) is the output of the following techniques, which ALL give the same values. The values of the following unexplained techniques are always less than the values from the explained techniques above.

- using jstat (formerly jvmstat) and summing up Survivor, Eden and Old Usage

- from inside a java program calling: heapusage = Runtime.getRuntime().totalMemory() - Runtime.getRuntime().freeMemory()

- from inside a java programm too calling: heapusage = ManagementFactory.getMemoryMXBean().getHeapMemoryUsage().getUsed() [which seems equally to the technique above - maybe this bean is based on those Runtime values?]

What I am interested in is why it is that way? What is measured by JVMTI and the other techniques and remains unmeasured by the unexplained techniques? Is it some JVM heap interna, may be something constant pool related or is there another reason for the different values?

Robert

[1840 byte] By [lldaedaluslla] at [2007-10-2 20:30:32]
# 1

I found out about the Bugs 4879849 and 4686462 which correspond to Runtime.getRuntime().totalMemory() and freeMemory(). As I am using JDK/JRE 1.5, are those bugs removed in 1.5?

I got confirmation that there is something wrong with that on running a test programm once on windows xp, once on solaris 9, both with JDK/JRE 1.5.

Rt is what can be gathered with Runtime, Bean are the values gathered by MemoryMXBean.

Results directly after start:

[Windows]:

[Rt]Memory (tot, free, use, max) [Byte]: 2031616, 1857840, 173776, 66650112

[Bean] HeapMemory (init, com, use, max) [Byte]: 0, 2031616, 173776, 66650112

[Bean] NonHeap Memory (init, com, use, max) [Byte]: 8585216, 8880128, 1922768, 100663296

[Solaris]

[Rt]Memory (tot, free, use, max) [Byte]: 385875968, 383862688, 2013280, 957743104

[Bean] HeapMemory (init, com, use, max) [Byte]: 402653184, 385875968, 2013280, 957743104

[Bean] NonHeap Memory (init, com, use, max) [Byte]: 16941056, 17104896, 1877544, 100663296

Here (Solaris) Runtime tells me that once it had around one GB of memory, with a default Xmx set to 64M (or is there a way to configure the default?)! Setting Xmx64M explicitly changes the value of max to around 64MB.

Accepting those errors results in the question, wether jstat (jvmstat) is affected to? The only conncetion to the values of jstat is that totalMemory() - freeMemory() = Surv0Used + Surv1Used + EdenUsed + OldUsed.

lldaedaluslla at 2007-7-13 23:13:36 > top of Java-index,Java HotSpot Virtual Machine,Specifications...
# 2

Hello Robert,

probably you also have to look at the Perm Generation. Some constant string objects may reside in the Perm Generation (those that are created with String.intern()). I would assume that you get those Strings with iterateOverReachableObjects and therefore count them when you use this method. When using jvmstat and tools like this that just look at the heap usage, you would need to add some percentage of the Perm Generation (unfortunately, you don't know how much of your Perm Generation are Strings like that...) to the sizes from young and old heap.

Regards,

Nick.

nicolasmichaela at 2007-7-13 23:13:36 > top of Java-index,Java HotSpot Virtual Machine,Specifications...