Sun Application Server hangs after few days

Hi ,

I have a serious problem here using Sun Application Server 8.2 bundled with Creator in Solaris 10.

Here is the scenario

1. i deploy and restart my server on friday

2. it works and runs okay on that day.

3. i come back on monday , i still can go to the login page

but when i click on login it hangs ( doesn't redirect me to the next page )

I still can see some logs are being printed out but somehow it doesn't redirect me to next page .. it just hangs there.

But there is no exception at all.

4. i check the CPU Utilization using prstat and it looks okay and below1%

I don't know what is happening with the App Server after few days.

Below is my domain.xml

<java-config debug-enabled="false" debug-options="-Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=9009" env-classpath-ignored="true" java-home="${com.sun.aas.javaRoot}" javac-options="-g" rmic-options="-iiop -poa -alwaysgenerate -keepgenerated -g" server-classpath="${com.sun.aas.installRoot}/lib/install/applications/jmsra/imqjmsra.jar${path.separator}${com.sun.aas.imqLib}/jaxm-api.jar${path.separator}${com.sun.aas.imqLib}/fscontext.jar${path.separator}${com.sun.aas.antLib}/ant.jar">

<!-- various required jvm-options -->

<jvm-options>-client</jvm-options>

<jvm-options>-Djava.endorsed.dirs=${com.sun.aas.installRoot}/lib/endorsed</jvm-options>

<jvm-options>-Djava.security.policy=${com.sun.aas.instanceRoot}/config/server.policy</jvm-options>

<jvm-options>-Djava.security.auth.login.config=${com.sun.aas.instanceRoot}/config/login.conf</jvm-options>

<jvm-options>-Dsun.rmi.dgc.server.gcInterval=3600000</jvm-options>

<jvm-options>-Dsun.rmi.dgc.client.gcInterval=3600000</jvm-options>

<jvm-options>-Xmx512m</jvm-options>

<jvm-options>-Djavax.net.ssl.keyStore=${com.sun.aas.instanceRoot}/config/keystore.jks</jvm-options>

<jvm-options>-Djavax.net.ssl.trustStore=${com.sun.aas.instanceRoot}/config/cacerts.jks</jvm-options>

<jvm-options>-Djava.ext.dirs=${com.sun.aas.javaRoot}/jre/lib/ext${path.separator}${com.sun.aas.instanceRoot}/lib/ext${path.separator}${com.sun.aas.derbyRoot}/lib</jvm-options>

<jvm-options>-Djdbc.drivers=org.apache.derby.jdbc.ClientDriver</jvm-options>

<jvm-options>-Djavax.management.builder.initial=com.sun.enterprise.admin.server.core.jmx.AppServerMBeanServerBuilder</jvm-options>

<jvm-options>-Dcom.sun.enterprise.config.config_environment_factory_class=com.sun.enterprise.config.serverbeans.AppserverConfigEnvironmentFactory</jvm-options>

<jvm-options>-Dcom.sun.enterprise.taglibs=appserv-jstl.jar,jsf-impl.jar</jvm-options>

<jvm-options>-Dcom.sun.enterprise.taglisteners=jsf-impl.jar</jvm-options>

<jvm-options>-XX:NewRatio=2</jvm-options>

<jvm-options>-XX:PermSize=128m</jvm-options>

<jvm-options>-XX:MaxPermSize=256m</jvm-options>

<jvm-options>-XX:+CMSPermGenSweepingEnabled</jvm-options>

<jvm-options>-XX:+CMSClassUnloadingEnabled</jvm-options>

</java-config>

<thread-pools>

<thread-pool idle-thread-timeout-in-seconds="120" max-thread-pool-size="200" min-thread-pool-size="0" num-work-queues="1" thread-pool-id="thread-pool-1"/>

</thread-pools>

</config>

Anything wrong with it?

Anyone experiencing this before?

Thanks

[4498 byte] By [just_a_kid83] at [2007-11-26 11:35:26]
# 1
HiCould it be because of Hibernate Connection Pooling?My Latest finding is it seems the code is hang when i doHibernateDaoSupport.add(POJO) to the database for audit purposes when user successfully login.
just_a_kid83 at 2007-7-7 3:51:27 > top of Java-index,Development Tools,Java Tools...
# 2
I've used C3PO (wich comes bundled with Hibernate) without problems. On the other hand, why don't you use the application server's built in connection pool? Best regardsAntonio.
antoniovl at 2007-7-7 3:51:27 > top of Java-index,Development Tools,Java Tools...
# 3

Hi Thanks for the reply.

if we use application server built in connection pool i have to define the jdbc resources right?

i am using JSF --> spring --> Hibernate Integration in my web application

my suspect is because after few days of inactivity the old connection is closed due to timeout.

then when i try to login it hangs there ( Because there is an insert statement behind when login is successfull )

if i am using C3P0 will it create a new connection if it detects the old conneciton is died?or validating the connection?

Because it seems when i dont define any connection pool in Hibernate.

it will just return me one connection and use it for the whole session which is not a problem where there is a load of activity.

Maybe that is why when i come back and try to login after few days of inactivity it will hangs at the insert statement ( the old connection which is still open timeout )

<prop key="hibernate.c3p0.minPoolSize">5</prop>

<prop key="hibernate.c3p0.maxPoolSize">25</prop>

<prop key="hibernate.c3p0.timeout">1800</prop>

<prop key="hibernate.c3p0.max_statement">50</prop>

Will the above configuration is sufficient ?

My user is around 15-20 concurrent user.

Thanks

just_a_kid83 at 2007-7-7 3:51:27 > top of Java-index,Development Tools,Java Tools...
# 4

>if we use application server built in connection pool i have to define the jdbc

>resources right?

Yes.

1. In the admin console, go to Resources -> JDBC -> Connection Pool. Here you have to configure the Connection Pool.

2. Choose a name and in the resource type select "ConnectionPoolDataSource". Leave blank the "Database Vendor" field if your database is not listed.

3. Check in your driver's documentation for the Class that implements ConnectionPoolDataSource (eg in PostgreSQL the class name is org.postgresql.ds.PGConnectionPoolDataSource). You'll need the class name in the second screen.

4. In the last screen, pay attention to the "Connection Validation" section. Check the "Required" box, and try with "Validation Method -> table". Create a dummy table in your database with no data, and put the name on the text field.

Then, go to Resources -> JDBC -> JDBC Resources and create the JNDI Resource. Don't forget to prepend "jdbc/" to the selected name (eg. "jdbc/MyDatabase"). Also select the connection pool just created.

Now, in Hibernate you can use something like:

<property name="hibernate.connection.datasource">jdbc/MyDatabase</property>

<property name="dialect">org.hibernate.dialect.PostgreSQLDialect</property>

This should work fine.

Best regards

Antonio.

antoniovl at 2007-7-7 3:51:27 > top of Java-index,Development Tools,Java Tools...
# 5
Hi Thanks What is the difference if i use Server Connection Pooling and Hibernate Connection Pooling? Will it have different effects. Now i already deployed my application with Hibernate connection pooling. Thanks
just_a_kid83 at 2007-7-7 3:51:27 > top of Java-index,Development Tools,Java Tools...
# 6

In most cases is better to use the application server's connection pool (if your app server supports it), unless your app will have to run standalone. In such case then you can use the Connection Pool implementation bundled with Hibernate.

The reason is simple: Connection pools implemented by most application servers are efficient and of high quality.

Best regards

Antonio.

antoniovl at 2007-7-7 3:51:27 > top of Java-index,Development Tools,Java Tools...
# 7
Can you post a jstack output of the server process?Also, which database are you using?
BinodPG at 2007-7-7 3:51:27 > top of Java-index,Development Tools,Java Tools...
# 8
How do i get a jstack of server process?I am using Oracle 9 my web application is using JSF Spring and HibernateThanks
just_a_kid83 at 2007-7-7 3:51:27 > top of Java-index,Development Tools,Java Tools...
# 9
- Use JDK 1.5- JAVA_HOME/bin/jstack <pid> will produce the jstack.
BinodPG at 2007-7-7 3:51:27 > top of Java-index,Development Tools,Java Tools...
# 10

From what i research this few days

it seems the problem is not in the application server

but rather at the connection problem as a lot of people says that there is some problems using DBCP when reconnecting broken connection.

I am using JSF --> Spring --> Hibernate by the way and let Spring manage the configuration for Hibernate and DataSource

Is it because by default the maxWait in DBCP is indefinetly thats why it seems hangs when it tried to perform some database operation.

Now my configuration is as below.

I hope i can see if the problem still persist after this weekend

<bean id="DataSource" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">

<property name="driverClassName" value="oracle.jdbc.OracleDriver"/>

<property name="url" value="jdbc:oracle:thin:@192.168.0.130:1521:DB"/>

<property name="username" value="user"/>

<property name="password" value="password"/>

<property name="testWhileIdle" value="true"/>

<property name="validationQuery" value="select 1 from dual"/>

<property name="maxWait" value="60000"/>

<property name="maxActive" value="50"/>

</bean>

just_a_kid83 at 2007-7-7 3:51:27 > top of Java-index,Development Tools,Java Tools...