# 3
> I don't know why your application doesn't work after
> the server is restarted by the watchdog. Perhaps you
> could describe your application and the observed
> problem in more detail?
>
> The server restart and the associated
> catastrophe-level error message is likely caused by a
> bug in Web Server. Please consider applying the
> latest service pack, 6.0SP11. (In fact, since 6.0 has
> reached its End Of Life period, I think you should
> consider upgrading to a supported version like Sun
> Java System Web Server 7.0.)
Ours is a typical Java/Servlet based application, about 700 users use it. These days we are frequently encountering catastrophe errors on iPlanet logs, after which the site stops functioning correctly. We are considering moving to 6.1SP6 but not sure if that will resolve the issue, as we don't have any repro steps either...
Please find log snippet below...
[03/Jul/2007:04:19:26] catastrophe (11169): UnixSignalHandlerThread: failed to wait on signals. (Interrupted system call)
[03/Jul/2007:04:19:26] failure (11168): Child process admin thread is shutting down
[03/Jul/2007:04:19:30] info ( 8491): Installing a new configuration
[03/Jul/2007:04:19:30] info ( 8491): [LS ls1] http://10.239.234.81, port 80 ready to accept requests
[03/Jul/2007:04:19:30] info ( 8491): A new configuration was successfully installed
[03/Jul/2007:04:19:31] info ( 8491): Using the Java HotSpot(TM) Server VM v1.4.2_10 from Sun Microsystems Inc.
[03/Jul/2007:04:19:31] info ( 8491): Java VM classpath: <snip/>
[03/Jul/2007:04:19:31] info ( 8491): Loading IWSSessionManager by default.
[03/Jul/2007:04:19:31] info ( 8491): IWSSessionManager: Maximum number of sessions is 1000
[03/Jul/2007:04:19:31] info ( 8491): Successfully initialized web application environment (web-apps.xml) for virtual server (https-goldoak1.res.globalone.net)
[03/Jul/2007:04:19:31] info ( 8491): Internal Info: loading servlet /home/gold/jsp/Dispatch.jsp
[03/Jul/2007:04:19:31] info ( 8491): Internal Info: loading servlet /home/gold/jsp/Dispatch.jsp
[03/Jul/2007:04:19:31] info ( 8491): Internal Info: loading servlet /home/gold/jsp/Dispatch.jsp
[03/Jul/2007:04:19:31] info ( 8491): Internal Info: loading servlet /home/gold/jsp/Dispatch.jsp
[03/Jul/2007:04:19:32] info ( 8491): /home/gold/jsp/Dispatch.jsp: init
[03/Jul/2007:04:23:56] config ( 8491): 0.000
[03/Jul/2007:04:23:56] config ( 8491): :
[03/Jul/2007:05:31:26] failure (11168): Watchdog Admin channel has been shutdown
[03/Jul/2007:05:32:40] config ( 8491): Full thread dump Java HotSpot(TM) Server VM (1.4.2_10-b03 mixed mode):
[03/Jul/2007:05:32:40] config ( 8491):
[03/Jul/2007:05:32:40] config ( 8491): "Thread-168"
[03/Jul/2007:05:32:40] config ( 8491): prio=5 tid=0x030a3318
[03/Jul/2007:05:32:40] config ( 8491): nid=0xb4
[03/Jul/2007:05:32:40] config ( 8491): in Object.wait()
[03/Jul/2007:05:32:40] config ( 8491): [1e8de000..1e8df660]
[03/Jul/2007:05:32:40] config ( 8491): at java.lang.Object.wait(Native Method)
[03/Jul/2007:05:32:40] config ( 8491): - waiting on <0x403f81c0> (a com.trilogy.util.Pool)
[03/Jul/2007:05:32:40] config ( 8491): at java.lang.Object.wait(Object.java:429)
[03/Jul/2007:05:32:40] config ( 8491): at com.trilogy.util.Pool.acquireShared(Pool.java:185)
[03/Jul/2007:05:32:40] config ( 8491): - locked <0x403f81c0> (a com.trilogy.util.Pool)
[03/Jul/2007:05:32:40] config ( 8491): at com.trilogy.util.Pool.acquireExclusive(Pool.java:217)
[03/Jul/2007:05:32:40] config ( 8491): at com.trilogy.sc.local.SCLocalSession.beginTransaction(SCLocalSession.java:142)
[03/Jul/2007:05:32:40] config ( 8491): at com.trilogy.mcc.useracl.service.core.UserAclServiceImpl.transact(Unknown Source)
[03/Jul/2007:05:32:40] config ( 8491): at com.trilogy.mcc.useracl.service.core.UserAclServiceImpl.authenticate(Unknown Source)
[03/Jul/2007:05:32:40] config ( 8491): at _jsps._home._gold._jsp._Dispatch_jsp$75.service(_Dispatch_jsp.java:42763)
[03/Jul/2007:05:32:40] config ( 8491): at _jsps._home._gold._jsp._Dispatch_jsp._jspService(_Dispatch_jsp.java:25539)
[03/Jul/2007:05:32:40] config ( 8491): - locked <0x2a9d69b0> (a com.trilogy.twc.server.twc.TWCSession)
[03/Jul/2007:05:32:40] config ( 8491): at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:119)
[03/Jul/2007:05:32:40] config ( 8491): at javax.servlet.http.HttpServlet.service(HttpServlet.java:865)
[03/Jul/2007:05:32:40] config ( 8491): at com.iplanet.server.http.servlet.NSServletRunner.invokeServletService(NSServletR unner.java:919)
[03/Jul/2007:05:32:40] config ( 8491): at com.iplanet.server.http.servlet.NSServletRunner.Service(NSServletRunner.java:48 3)
[03/Jul/2007:05:32:40] config ( 8491):
[03/Jul/2007:05:32:40] config ( 8491): "Thread-167"
[03/Jul/2007:05:32:40] config ( 8491): prio=5 tid=0x030a2680
[03/Jul/2007:05:32:40] config ( 8491): nid=0xb5
[03/Jul/2007:05:32:40] config ( 8491): in Object.wait()
[03/Jul/2007:05:32:40] config ( 8491): [1e8ae000..1e8af660]
[03/Jul/2007:05:32:40] config ( 8491): at java.lang.Object.wait(Native Method)
[03/Jul/2007:05:32:40] config ( 8491): - waiting on <0x403f81c0> (a com.trilogy.util.Pool)
[03/Jul/2007:05:32:40] config ( 8491): at java.lang.Object.wait(Object.java:429)
[03/Jul/2007:05:32:40] config ( 8491): at com.trilogy.util.Pool.acquireShared(Pool.java:185)
[03/Jul/2007:05:32:40] config ( 8491): - locked <0x403f81c0> (a com.trilogy.util.Pool)
[03/Jul/2007:05:32:40] config ( 8491): at com.trilogy.util.Pool.acquireExclusive(Pool.java:217)
[03/Jul/2007:05:32:40] config ( 8491): at com.trilogy.sc.local.SCLocalSession.beginTransaction(SCLocalSession.java:142)
[03/Jul/2007:05:32:40] config ( 8491): at com.trilogy.mcc.useracl.service.core.UserAclServiceImpl.transact(Unknown Source)
[03/Jul/2007:05:32:40] config ( 8491): at com.trilogy.mcc.useracl.service.core.UserAclServiceImpl.authenticate(Unknown Source)
[03/Jul/2007:05:32:40] config ( 8491): at _jsps._home._gold._jsp._Dispatch_jsp$75.service(_Dispatch_jsp.java:42763)
[03/Jul/2007:05:32:40] config ( 8491): at _jsps._home._gold._jsp._Dispatch_jsp._jspService(_Dispatch_jsp.java:25539)
[03/Jul/2007:05:32:40] config ( 8491): - locked <0x2a9b2658> (a com.trilogy.twc.server.twc.TWCSession)
[03/Jul/2007:05:32:40] config ( 8491): at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:119)
[03/Jul/2007:05:32:40] config ( 8491): at javax.servlet.http.HttpServlet.service(HttpServlet.java:865)
[03/Jul/2007:05:32:40] config ( 8491): at com.iplanet.server.http.servlet.NSServletRunner.invokeServletService(NSServletR unner.java:919)
[03/Jul/2007:05:32:40] config ( 8491): at com.iplanet.server.http.servlet.NSServletRunner.Service(NSServletRunner.java:48 3)
[03/Jul/2007:05:32:40] config ( 8491):
<!--comment similar logs for "Thread-XX" -->
[03/Jul/2007:05:32:40] config ( 8491):
[03/Jul/2007:05:32:40] config ( 8491): "Thread-1"
[03/Jul/2007:05:32:40] config ( 8491): prio=5 tid=0x012d64e0
[03/Jul/2007:05:32:40] config ( 8491): nid=0x22
[03/Jul/2007:05:32:40] config ( 8491): waiting on condition
[03/Jul/2007:05:32:40] config ( 8491): [1fd7f000..1fd7fc28]
[03/Jul/2007:05:32:40] config ( 8491): at java.lang.Thread.sleep(Native Method)
[03/Jul/2007:05:32:40] config ( 8491): at com.iplanet.server.http.session.SessionReaper.run(SessionReaper.java:63)
[03/Jul/2007:05:32:40] config ( 8491):
[03/Jul/2007:05:32:40] config ( 8491): "Signal Dispatcher"
[03/Jul/2007:05:32:40] config ( 8491): daemon
[03/Jul/2007:05:32:40] config ( 8491): prio=10 tid=0x00ab7d48
[03/Jul/2007:05:32:40] config ( 8491): nid=0x1d
[03/Jul/2007:05:32:40] config ( 8491): waiting on condition
[03/Jul/2007:05:32:40] config ( 8491): [0..0]
[03/Jul/2007:05:32:40] config ( 8491):
[03/Jul/2007:05:32:40] config ( 8491): "Finalizer"
[03/Jul/2007:05:32:40] config ( 8491): daemon
[03/Jul/2007:05:32:40] config ( 8491): prio=8 tid=0x00ab4a98
[03/Jul/2007:05:32:40] config ( 8491): nid=0x1b
[03/Jul/2007:05:32:40] config ( 8491): in Object.wait()
[03/Jul/2007:05:32:40] config ( 8491): [fb17f000..fb17fc28]
[03/Jul/2007:05:32:40] config ( 8491): at java.lang.Object.wait(Native Method)
[03/Jul/2007:05:32:40] config ( 8491): - waiting on <0x42b58138> (a java.lang.ref.ReferenceQueue$Lock)
[03/Jul/2007:05:32:40] config ( 8491): at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:111)
[03/Jul/2007:05:32:40] config ( 8491): - locked <0x42b58138> (a java.lang.ref.ReferenceQueue$Lock)
[03/Jul/2007:05:32:40] config ( 8491): at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:127)
[03/Jul/2007:05:32:40] config ( 8491): at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)
[03/Jul/2007:05:32:40] config ( 8491):
[03/Jul/2007:05:32:40] config ( 8491): "Reference Handler"
[03/Jul/2007:05:32:40] config ( 8491): daemon
[03/Jul/2007:05:32:40] config ( 8491): prio=10 tid=0x00ab3130
[03/Jul/2007:05:32:40] config ( 8491): nid=0x1a
[03/Jul/2007:05:32:40] config ( 8491): in Object.wait()
[03/Jul/2007:05:32:40] config ( 8491): [fb27f000..fb27fc28]
[03/Jul/2007:05:32:40] config ( 8491): at java.lang.Object.wait(Native Method)
[03/Jul/2007:05:32:40] config ( 8491): - waiting on <0x418da6f0> (a java.lang.ref.Reference$Lock)
[03/Jul/2007:05:32:40] config ( 8491): at java.lang.Object.wait(Object.java:429)
[03/Jul/2007:05:32:40] config ( 8491): at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:115)
[03/Jul/2007:05:32:40] config ( 8491): - locked <0x418da6f0> (a java.lang.ref.Reference$Lock)
[03/Jul/2007:05:32:40] config ( 8491):
[03/Jul/2007:05:32:40] config ( 8491): "main"
[03/Jul/2007:05:32:40] config ( 8491): prio=5 tid=0x003f6880
[03/Jul/2007:05:32:40] config ( 8491): nid=0x14
[03/Jul/2007:05:32:40] config ( 8491): runnable
[03/Jul/2007:05:32:40] config ( 8491): [0..fe23fc8c]
[03/Jul/2007:05:32:40] config ( 8491):
[03/Jul/2007:05:32:40] config ( 8491): "VM Thread"
[03/Jul/2007:05:32:40] config ( 8491): prio=5 tid=0x00ab22e8
[03/Jul/2007:05:32:40] config ( 8491): nid=0x19
[03/Jul/2007:05:32:40] config ( 8491): runnable
[03/Jul/2007:05:32:40] config ( 8491):
[03/Jul/2007:05:32:40] config ( 8491):
[03/Jul/2007:05:32:40] config ( 8491): "VM Periodic Task Thread"
[03/Jul/2007:05:32:40] config ( 8491): prio=10 tid=0x003f46e8
[03/Jul/2007:05:32:40] config ( 8491): nid=0x21
[03/Jul/2007:05:32:40] config ( 8491): waiting on condition
[03/Jul/2007:05:32:40] config ( 8491):
[03/Jul/2007:05:32:40] config ( 8491): "Suspend Checker Thread"
[03/Jul/2007:05:32:40] config ( 8491): prio=10 tid=0x00ab7448
[03/Jul/2007:05:32:40] config ( 8491): nid=0x1c
[03/Jul/2007:05:32:40] config ( 8491): runnable
[03/Jul/2007:05:32:40] config ( 8491):
Asrara at 2007-7-12 23:58:32 >
