JVM Options

Hello all:

The CPU on the windows machine of this sun java system web server 6.1 sp6 was runing 100%. It has 4 cpus and 4gb memory. The webserver performance came to a halt. It did not accept any new connections and was extremely slow.

The current jvm setting is -Xmx256m. I am following the doc and trying to setup the jvm profiler. I added the jvm option "thread=y, depth=3" in the jvm profiler page. I also added the same option "thread=y, depth=3" in Jvm options page. I then restarted the web server. It created the log.txt file in config directory. I did nothing in server.xml file.

questions are:

1) What is the best way to see where is the bottleneck? Is JVM profiling only way to go? What is the process to identify why the cpu is 100%

2) Where would the jvm profile information store at? After logging in the website I tried to see if log.txt has any information. It has no info.

Thanks in advance for any help!

the current log.txt is..

<code>

JAVA PROFILE 1.0.1, created Wed Mar 21 15:29:20 2007

Header for -agentlib:hprof (or -Xrunhprof) ASCII Output (J2SE 1.5 JVMTI based)

@(#)jvm.hprof.txt1.3 04/02/09

Copyright (c) 2004 Sun Microsystems, Inc. All Rights Reserved.

WARNING! This file format is under development, and is subject to

change without notice.

This file contains the following types of records:

THREAD START

THREAD ENDmark the lifetime of Java threads

TRACErepresents a Java stack trace. Each trace consists

of a series of stack frames. Other records refer to

TRACEs to identify (1) where object allocations have

taken place, (2) the frames in which GC roots were

found, and (3) frequently executed methods.

HEAP DUMPis a complete snapshot of all live objects in the Java

heap. Following distinctions are made:

ROOTroot set as determined by GC

CLSclasses

OBJinstances

ARRarrays

SITESis a sorted list of allocation sites. This identifies

the most heavily allocated object types, and the TRACE

at which those allocations occurred.

CPU SAMPLESis a statistical profile of program execution. The VM

periodically samples all running threads, and assigns

a quantum to active TRACEs in those threads. Entries

in this record are TRACEs ranked by the percentage of

total quanta they consumed; top-ranked TRACEs are

typically hot spots in the program.

CPU TIMEis a profile of program execution obtained by measuring

the time spent in individual methods (excluding the time

spent in callees), as well as by counting the number of

times each method is called. Entries in this record are

TRACEs ranked by the percentage of total CPU time. The

"count" field indicates the number of times each TRACE

is invoked.

MONITOR TIMEis a profile of monitor contention obtained by measuring

the time spent by a thread waiting to enter a monitor.

Entries in this record are TRACEs ranked by the percentage

of total monitor contention time and a brief description

of the monitor. The "count" field indicates the number of

times the monitor was contended at that TRACE.

MONITOR DUMPis a complete snapshot of all the monitors and threads in

the System.

HEAP DUMP, SITES, CPU SAMPLES|TIME and MONITOR DUMP|TIME records are generated

at program exit. They can also be obtained during program execution by typing

Ctrl-\ (on Solaris) or by typing Ctrl-Break (on Win32).

--

</code>

[3638 byte] By [Rvi_20a] at [2007-11-26 22:28:43]
# 1

Hi Rvi_20,

You didn't give any details about neither the application(s) running on the web server nor about the requests that cause the problem. Assuming it is a Java web application, profiling the application using a Java profiler is the best way to identify where the bottleneck is.

When using HPROF, you need to send a SIGQUIT (Ctrl-Break on Windows) to the process so that it dumps the profiling data to disk. Start the server (with the profiling options enabled). Run the test workload which causes the high CPU utilization. Then without stopping the server, send a Ctrl-Break signal to it.

There are also other profilers like OptimizeIt (free trial download) and NetBeans (free from http://www.netbeans.org) that have a nice GUI which makes generating, digesting and navigating the profiled data a lot easier.

The following blog talks about integrating the NetBeans profiler with Web Server 7.0. You could apply the similar steps to Web Server 6.1 too.

http://blogs.sun.com/yamini/entry/integrating_netbeans_profiler_with_web

Arvind_Srinivasana at 2007-7-10 11:32:02 > top of Java-index,Web & Directory Servers,Web Servers...
# 2

Hi!

Thanks for all that information. I will take a look at netbeans and optimize it. This is a Java + servlets + jsp based cots product. It serves dynamic pages. I followed the SJWS 6.1 doc and have done the below steps

1) Added this block in server.xml file.

<!-- hprof options -->

<PROFILER name="hprof" enabled="true">

<JVMOPTIONS>

-Xrunhprof:file=log.txt,options

</JVMOPTIONS>

</PROFILER>

2) Enabled the profiler in profile settings of "JVM Profiler" page and added the JVM option "-Xrunhprof:file=log.txt,thread=y, depth=3"

3) There are currently four jvm options in the jvm options page. they are:

-Djava.security.auth.login.config=d:/Sun/WebServer6.1/https-abc.com/config/logi n.conf

-DJava.util.logging.manager=com.iplanet.ias.server.logging.ServerLogManager

-Xmx256m

-Xrs

After enabling the profiling, I clicked "Apply" and restarted the web server software. It created a file called log.txt.TMP.4744.txt in the config dir.

I am thinking how should I generate the profiler dump information. Should I just open the task manager on the web server machine and select the web server process and press ctrl-break?

i just want to complete the profiler steps in the documentation and see quick info before I explore the other tools.

Thanks again!

Rvi_20a at 2007-7-10 11:32:02 > top of Java-index,Web & Directory Servers,Web Servers...