long GC pauses; full gc eventually takes 30 seconds+

jre 1.5.0_06,

-Dcom.sun.management.jmxremote -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintTenuringDistribution -XX:NewRatio=3 -XX:MaxTenuringThreshold=3 -XX:NewSize=90M -XX:MaxNewSize=90M -Xmx800M

normally, full gc's occur with < 1 second durations, but eventually, the duration peaks at 30 seconds, then 7 seconds, then back to normal

the heap was at about 290MB (far from full).

setting maxgcpausemillis has no effect (something extraordinary happens?)

there is no obvious pattern in the long pauses (not due to perm resizing or so)

does anyone have experience with sudden long pauses?

we could think of heap being swapped - but we don't know how to monitor this (and nobody was starting memory consuming applications at the time)

tak!

/aksel

[831 byte] By [aksel.schmidta] at [2007-11-26 20:52:02]
# 1
> does anyone have experience with sudden long pauses?Yes, I've seen it, and the reason was what you said. The head was paged out.Try to monitor I/O times on the machine, and see if you have lots of I/O during the long GC.Kaj
kajbja at 2007-7-10 2:17:09 > top of Java-index,Java Essentials,Java Programming...
# 2
A couple of suggestions:1) Try lowering the heap size (-Xmx value) to say, 500M, and see what happens2) Use the concurrent GC ( -XX:+UseConcMarkSweepGC ). Use that with -XX:+CMSClassUnloadingEnabled and -XX:+CMSPermGenSweepingEnabled .
karma-9a at 2007-7-10 2:17:09 > top of Java-index,Java Essentials,Java Programming...
# 3

When you print the GC details what do you see is being reclaimed when you have the long pause? As karma-9 said the concurrent low pause collector will likely help but as this has an associated overhead there might be other possibilities.

EDIT: Also I think NewSize/MaxNewSize should be set to the same value as NewRatio (fourth of the heap)

YoGeea at 2007-7-10 2:17:09 > top of Java-index,Java Essentials,Java Programming...
# 4

> EDIT: Also I think NewSize/MaxNewSize should be set to the same value as NewRatio (fourth of the heap)

yes - there is some confusion there.. the 90MB is used by the vm though

unfortunately we are not able to try out the cms immediately (as we are in production) - and even if this might be the end, we have to test the cms for a long time before entering production. we were hit by crashes caused by cms a year ago when we decided not to use it (http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6386633)

here are some long pauses (days apart - the gc log is embedded in our own loggings)

061221 102753 0750 EVcoreservice.taskoutput.container 75.537: [GC [PSYoungGen: 70133K->19827K(53632K)] 173056K->138149K(172736K), 11.5205585 secs] ##[

070301 122542 0812 EVcoreservice.taskoutput.container 6146.116: [Full GC [PSYoungGen: 768K->0K(90944K)] [PSOldGen: 286976K->213826K(259328K)] 287744K->213826K(350272K) [PSPermGen: 19432K->19432K(23040K)], 30.1171480 secs] ##[

070305 163701 0140 EVcoreservice.taskoutput.container [PSYoungGen: 1256K->0K(90176K)] [PSOldGen: 239309K->197132K(221632K)] 240565K->197132K(311808K) [PSPermGen: 19260K->19039K(23808K)], 20.3872070 secs] ##[

aksel.schmidta at 2007-7-10 2:17:09 > top of Java-index,Java Essentials,Java Programming...
# 5
> 1) Try lowering the heap size (-Xmx value) to say, 500M, and see what happensthanks karma-9as of '1' - can you explain how this should help?/akselMessage was edited by: aksel.schmidt
aksel.schmidta at 2007-7-10 2:17:09 > top of Java-index,Java Essentials,Java Programming...
# 6

Do you ever get an OutOfMemoryError? It looks like you are running out of heap space.

To make these full GC's faster you could try using the parallel compacting collector (-XX:+UseParallelOldGC). If you have multiple CPU's I believe this will help. After that you could try the concurrent low pause collector, as already mentioned.

To be honest you are better off posting this question in the JVM forum (linking to this thread):

http://forum.java.sun.com/forum.jspa?forumID=37

YoGeea at 2007-7-10 2:17:09 > top of Java-index,Java Essentials,Java Programming...
# 7

> Do you ever get an OutOfMemoryError? It looks like you are running out of heap space.

no, we never encountered outofmem

the max heap is set to 800MB (and the machine got 4gb to share with ms sql server)

> To be honest you are better off posting this question in the JVM forum (linking to this thread):

thanks - i guess i will try that

aksel.schmidta at 2007-7-10 2:17:09 > top of Java-index,Java Essentials,Java Programming...
# 8

> > 1) Try lowering the heap size (-Xmx value) to say,

> 500M, and see what happens

> thanks karma-9

>

> as of '1' - can you explain how this should help?

It is possible that the large heap size doesn't leave enough space for allocations needed outside the JVM, keeping in mind that the amount of memory handed by the OS to your app is limited, and in all cases much less than the total amount of memory in your machine. In short, it might be that your heap size is too large for the needs of your application.

Edit:

Actually, I just realized it is explained in great details by the user Caffeine... in the JVM forum thread mentioned .

Message was edited by:

karma-9

karma-9a at 2007-7-10 2:17:09 > top of Java-index,Java Essentials,Java Programming...
# 9
moved to http://forum.java.sun.com/thread.jspa?threadID=5146087
aksel.schmidta at 2007-7-10 2:17:09 > top of Java-index,Java Essentials,Java Programming...