Sun Directory Server Crashed

Hi

i have two solaris Box ( solaris 8 and Solaris 10)

and did master-master ldap replication

after few days,

i get this errors log in my Solaris 8 while its working fine in Solaris 10

[19/Dec/2006:17:17:10] config (10607): 0xfe1c0000/var/Sun/mps/lib/libssl3.so

[19/Dec/2006:17:17:10] config (10607):

[19/Dec/2006:17:17:10] config (10607): 0xfe180000/var/Sun/mps/lib/libsmime3.so

[19/Dec/2006:17:17:10] config (10607):

[19/Dec/2006:17:17:10] config (10607): 0xfe080000/var/Sun/mps/lib/libnss3.so

[19/Dec/2006:17:17:10] config (10607):

[19/Dec/2006:17:17:10] config (10607): 0xfe2e0000/var/Sun/mps/bin/https/lib/libnsfc.so

[19/Dec/2006:17:17:10] config (10607):

[19/Dec/2006:17:17:10] config (10607): 0xfe160000/var/Sun/mps/bin/https/lib/libnstp.so

[19/Dec/2006:17:17:10] config (10607):

[19/Dec/2006:17:17:10] config (10607): 0xfe140000/var/Sun/mps/bin/https/lib/libdirmon4.so

[19/Dec/2006:17:17:10] config (10607):

[19/Dec/2006:17:17:10] config (10607): 0xfe120000/var/Sun/mps/bin/https/lib/libnstime.so

[19/Dec/2006:17:17:10] config (10607):

[19/Dec/2006:17:17:10] config (10607): 0xfe060000/var/Sun/mps/bin/https/lib/libsupport.so

[19/Dec/2006:17:17:10] config (10607):

[19/Dec/2006:17:17:10] config (10607): 0xfe040000/var/Sun/mps/bin/https/lib/libares3.so

[19/Dec/2006:17:17:10] config (10607):

[19/Dec/2006:17:17:10] config (10607): 0xfe020000/var/Sun/mps/lib/libssldap50.so

[19/Dec/2006:17:17:10] config (10607):

[19/Dec/2006:17:17:10] config (10607): 0xfe000000/usr/lib/libpthread.so.1

[19/Dec/2006:17:17:10] config (10607):

[19/Dec/2006:17:17:10] config (10607): 0xfdfd0000/usr/lib/libCrun.so.1

[19/Dec/2006:17:17:10] config (10607):

[19/Dec/2006:17:17:10] config (10607): 0xfdfb0000/usr/lib/libmp.so.2

[19/Dec/2006:17:17:10] config (10607):

[19/Dec/2006:17:17:10] config (10607): 0xfdf90000/usr/lib/libaio.so.1

[19/Dec/2006:17:17:10] config (10607):

[19/Dec/2006:17:17:10] config (10607): 0xfdf40000/usr/lib/libresolv.so.2

[19/Dec/2006:17:17:10] config (10607):

[19/Dec/2006:17:17:10] config (10607): 0xfede0000/usr/platform/SUNW,Sun-Fire-480R/lib/libc_psr.so.1

[19/Dec/2006:17:17:10] config (10607):

[19/Dec/2006:17:17:10] config (10607): 0xfded0000/usr/lib/nss_files.so.1

[19/Dec/2006:17:17:10] config (10607):

[19/Dec/2006:17:17:10] config (10607): 0xfdea0000/var/Sun/mps/bin/https/lib/libAdmservPlugin.so

[19/Dec/2006:17:17:10] config (10607):

[19/Dec/2006:17:17:10] config (10607): 0xfde70000/var/Sun/mps/lib/libadmsslutil52.so

[19/Dec/2006:17:17:10] config (10607):

[19/Dec/2006:17:17:10] config (10607): 0xfde40000/v[19/Dec/2006:17:17:10] config (10607): # An error report file has been saved as hs_err_pid10607.log.

[19/Dec/2006:17:17:10] config (10607):

[19/Dec/2006:17:17:10] config (10607): # Please refer to the filefor further information.

[19/Dec/2006:17:17:10] config (10607):

[19/Dec/2006:17:17:10] config (10607): #

[19/Dec/2006:17:17:10] config (10607):

[19/Dec/2006:17:17:14] info (10610): Installing anew configuration

[19/Dec/2006:17:17:14] info (10610): [LS ls1] http://ils1app3.tpcils.com, port 390 ready to accept requests

[19/Dec/2006:17:17:14] info (10610): Anew configuration was successfully installed

[19/Dec/2006:17:17:14] info (10610): Using the Java HotSpot(TM) Server VM v1.4.1_01 from Sun Microsystems Inc.

[19/Dec/2006:17:17:14] info (10610): Java VM classpath: /var/Sun/mps/bin/https/jar/NSServletLayer.jar:/var/Sun/mps/bin/https/jar/NSJavaUtil.jar:/var/Sun/mps/bin/https/jar/NSJavaMiscUtil.jar:/var/Sun/mps/bin/https/jar/servlet.jar:/var/Sun/mps/bin/https/jar/servlet-2.3-filters-api.jar:/var/Sun/mps/bin/https/jar/jspengine.jar:/var/Sun/mps/java/ldapjdk.jar:/var/Sun/mps/java/jss311.jar:

[19/Dec/2006:17:17:14] info (10610): Loading IWSSessionManager by default.

[19/Dec/2006:17:17:14] info (10610): IWSSessionManager: Maximum number of sessions is 1000

[19/Dec/2006:17:17:14] catastrophe (10610): Server crash detected (signal SIGSEGV)

[19/Dec/2006:17:17:14] info (10610): Crash occurred in function PR_Write from module /var/Sun/mps/lib/libnspr4.so

[19/Dec/2006:17:17:14] config (10610):

[19/Dec/2006:17:17:14] config (10610): An unexpected exception has been detected innative code outside the VM.

[19/Dec/2006:17:17:14] config (10610):

[19/Dec/2006:17:17:14] config (10610): Unexpected Signal : 11 occurred at PC=0xFEEBB384

[19/Dec/2006:17:17:14] config (10610):

[19/Dec/2006:17:17:14] config (10610): Function=

[19/Dec/2006:17:17:14] config (10610): PR_Write+0x0

[19/Dec/2006:17:17:14] config (10610):

[19/Dec/2006:17:17:14] config (10610): Library=/var/Sun/mps/lib/libnspr4.so

[19/Dec/2006:17:17:14] config (10610):

[19/Dec/2006:17:17:14] config (10610):

[19/Dec/2006:17:17:14] config (10610): Cannot obtain thread information

[19/Dec/2006:17:17:14] config (10610):

[19/Dec/2006:17:17:14] config (10610):

[19/Dec/2006:17:17:14] config (10610): Dynamic libraries:

[19/Dec/2006:17:17:14] config (10610):

[19/Dec/2006:17:17:14] config (10610): 0x10000 ns-httpd

[19/Dec/2006:17:17:14] config (10610):

[19/Dec/2006:17:17:14] config (10610): 0xff370000/var/Sun/mps/bin/base/jre/lib/sparc/server/libjsig.so

[19/Dec/2006:17:17:14] config (10610):

[19/Dec/2006:17:17:14] config (10610): 0xff350000/var/Sun/mps/bin/https/lib/libsmartheap_smp.so

[19/Dec/2006:17:17:14] config (10610):

[19/Dec/2006:17:17:14] config (10610): 0xff330000/var/Sun/mps/bin/https/lib/liblibdbm.so

[19/Dec/2006:17:17:14] config (10610):

[19/Dec/2006:17:17:14] config (10610): 0xff180000/var/Sun/mps/bin/https/lib/libns-httpd40.so

[19/Dec/2006:17:17:14] config (10610):

[19/Dec/2006:17:17:14] config (10610): 0xff150000/var/Sun/mps/bin/https/lib/liblibsi18n.so

[19/Dec/2006:17:17:14] config (10610):

[19/Dec/2006:17:17:14] config (10610): 0xff130000/var/Sun/mps/bin/https/lib/libgetprop.so

[19/Dec/2006:17:17:14] config (10610):

[19/Dec/2006:17:17:14] config (10610): 0xff110000/var/Sun/mps/bin/https/lib/libnsprwrap.so

[19/Dec/2006:17:17:14] config (10610):

[19/Dec/2006:17:17:14] config (10610): 0xff0c0000/var/Sun/mps/lib/libldap50.so

[19/Dec/2006:17:17:14] config (10610):

[19/Dec/2006:17:17:14] config (10610): 0xff0a0000/var/Sun/mps/lib/libprldap50.so

[19/Dec/2006:17:17:14] config (10610):

[19/Dec/2006:17:17:14] config (10610): 0xff070000/var/Sun/mps/lib/libsasl.so

[19/Dec/2006:17:17:14] config (10610):

[19/Dec/2006:17:17:14] config (10610): 0xfef00000/var/Sun/mps/bin/https/lib/libxerces-c.so

[19/Dec/2006:17:17:14] config (10610):

[19/Dec/2006:17:17:14] config (10610): 0xfee00000/var/Sun/mps/lib/libicuuc.so.2

[19/Dec/2006:17:17:14] config (10610):

[19/Dec/2006:17:17:14] config (10610): 0xfed00000/var/Sun/mps/lib/libicui18n.so.2

[19/Dec/2006:17:17:14] config (10610):

[19/Dec/2006:17:17:14] config (10610): 0xfe400000/var/Sun/mps/lib/libicudata.so.2

[19/Dec/2006:17:17:14] config (10610):

[19/Dec/2006:17:17:14] config (10610): 0xff040000/var/Sun/mps/lib/libplc4.so

[19/Dec/2006:17:17:14] config (10610):

[19/Dec/2006:17:17:14] config (10610): 0xff020000/var/Sun/mps/lib/libplds4.so

[19/Dec/2006:17:17:14] config (10610):

[19/Dec/2006:17:17:14] config (10610): 0xfeeb0000/var/Sun/mps/lib/libnspr4.so

[19/Dec/2006:17:17:14] config (10610):

[19/Dec/2006:17:17:14] config (10610): 0xfee90000/usr/lib/libsocket.so.1

[19/Dec/2006:17:17:14] config (10610):

[19/Dec/2006:17:17:14] config (10610): 0xfe300000/usr/lib/libnsl.so.1

[19/Dec/2006:17:17:14] config (10610):

[19/Dec/2006:17:17:14] config (10610): 0xff010000/usr/lib/libdl.so.1

[19/Dec/2006:17:17:14] config (10610):

[19/Dec/2006:17:17:14] config (10610): 0xfece0000/usr/lib/librt.so.1

[19/Dec/2006:17:17:14] config (10610):

[19/Dec/2006:17:17:14] config (10610): 0xfecc0000/usr/lib/libkstat.so.1

[19/Dec/2006:17:17:14] config (10610):

[19/Dec/2006:17:17:14] config (10610): 0xfec50000/usr/lib/libC.so.5

[19/Dec/2006:17:17:14] config (10610):

[19/Dec/2006:17:17:14] config (10610): 0xfec00000/usr/lib/libm.so.1

[19/Dec/2006:17:17:14] config (10610):

[19/Dec/2006:17:17:14] config (10610): 0xff390000/usr/lib/libw.so.1

[19/Dec/2006:17:17:14] config (10610):

[19/Dec/2006:17:17:14] config (10610): 0xfe3c0000/usr/lib/libthread.so.1

[19/Dec/2006:17:17:14] config (10610):

[19/Dec/2006:17:17:14] config (10610): 0xfe200000/usr/lib/libc.so.1

[19/Dec/2006:17:17:14] config (10610):

[19/Dec/2006:17:17:14] config (10610): 0xfe1c0000/var/Sun/mps/lib/libssl3.so

[19/Dec/2006:17:17:14] config (10610):

[19/Dec/2006:17:17:14] config (10610): 0xfe180000/var/Sun/mps/lib/libsmime3.so

[19/Dec/2006:17:17:14] config (10610):

[19/Dec/2006:17:17:14] config (10610): 0xfe080000/var/Sun/mps/lib/libnss3.so

[19/Dec/2006:17:17:14] config (10610):

[19/Dec/2006:17:17:14] config (10610): 0xfe2e0000/var/Sun/mps/bin/https/lib/libnsfc.so

[19/Dec/2006:17:17:14] config (10610):

[19/Dec/2006:17:17:14] config (10610): 0xfe160000/var/Sun/mps/bin/https/lib/libnstp.so

[19/Dec/2006:17:17:14] config (10610):

[19/Dec/2006:17:17:14] config (10610): 0xfe140000/var/Sun/mps/bin/https/lib/libdirmon4.so

[19/Dec/2006:17:17:14] config (10610):

[19/Dec/2006:17:17:14] config (10610): 0xfe120000/var/Sun/mps/bin/https/lib/libnstime.so

[19/Dec/2006:17:17:14] config (10610):

[19/Dec/2006:17:17:14] config (10610): 0xfe060000/var/Sun/mps/bin/https/lib/libsupport.so

[19/Dec/2006:17:17:14] config (10610):

[19/Dec/2006:17:17:14] config (10610): 0xfe040000/var/Sun/mps/bin/https/lib/libares3.so

[19/Dec/2006:17:17:14] config (10610):

[19/Dec/2006:17:17:14] config (10610): 0xfe020000/var/Sun/mps/lib/libssldap50.so

[19/Dec/2006:17:17:14] config (10610):

[19/Dec/2006:17:17:14] config (10610): 0xfe000000/usr/lib/libpthread.so.1

[19/Dec/2006:17:17:14] config (10610):

[19/Dec/2006:17:17:14] config (10610): 0xfdfd0000/usr/lib/libCrun.so.1

[19/Dec/2006:17:17:14] config (10610):

[19/Dec/2006:17:17:14] config (10610): 0xfdfb0000/usr/lib/libmp.so.2

[19/Dec/2006:17:17:14] config (10610):

[19/Dec/2006:17:17:14] config (10610): 0xfdf90000/usr/lib/libaio.so.1

[19/Dec/2006:17:17:14] config (10610):

[19/Dec/2006:17:17:14] config (10610): 0xfdf40000/usr/lib/libresolv.so.2

[19/Dec/2006:17:17:14] config (10610):

[19/Dec/2006:17:17:14] config (10610): 0xfede0000/usr/platform/SUNW,Sun-Fire-480R/lib/libc_psr.so.1

[19/Dec/2006:17:17:14] config (10610):

[19/Dec/2006:17:17:14] config (10610): 0xfded0000/usr/lib/nss_files.so.1

[19/Dec/2006:17:17:14] config (10610):

[19/Dec/2006:17:17:14] config (10610): 0xfdea0000/var/Sun/mps/bin/https/lib/libAdmservPlugin.so

[19/Dec/2006:17:17:14] config (10610):

[19/Dec/2006:17:17:14] config (10610): 0xfde70000/var/Sun/mps/lib/libadmsslutil52.so

[19/Dec/2006:17:17:14] config (10610):

[11631 byte] By [just_a_kid83a] at [2007-11-26 13:00:24]
# 1

Everytime i restart the ldap server in Solaris 8

the error log will keep grow bigger and reporting the same error

( but weird.. i can still do ldapsearch and so on )

( after sometimes , it will crash and i cant even do ldap search as it will complain operartion error )

[19/Dec/2006:17:59:42] config (16423): 0xfdc40000/var/Sun/mps/bin/https/lib/libConfigFile.so

[19/Dec/2006:17:59:42] config (16423):

[19/Dec/2006:17:59:42] config (16423): 0xfcc00000/var/Sun/mps/bin/base/jre/lib/sparc/server/libjvm.so

[19/Dec/2006:17:59:42] config (16423):

[19/Dec/2006:17:59:42] config (16423): 0xfdac0000/var/Sun/mps/bin/base/jre/lib/sparc/native_threads/libhpi.so

[19/Dec/2006:17:59:42] config (16423):

[19/Dec/2006:17:59:42] config (16423): 0xfda90000/var/Sun/mps/bin/base/jre/lib/sparc/libverify.so

[19/Dec/2006:17:59:42] config (16423):

[19/Dec/2006:17:59:42] config (16423): 0xfda50000/var/Sun/mps/bin/base/jre/lib/sparc/libjava.so

[19/Dec/2006:17:59:42] config (16423):

[19/Dec/2006:17:59:42] config (16423): 0xfda30000/var/Sun/mps/bin/base/jre/lib/sparc/libzip.so

[19/Dec/2006:17:59:42] config (16423):

[19/Dec/2006:17:59:42] config (16423):

[19/Dec/2006:17:59:42] config (16423): Local Time = Tue Dec 19 17:59:42 2006

[19/Dec/2006:17:59:42] config (16423): Elapsed Time = 1

[19/Dec/2006:17:59:42] config (16423):

[19/Dec/2006:17:59:42] config (16423): #

[19/Dec/2006:17:59:42] config (16423):

[19/Dec/2006:17:59:42] config (16423): # The exception above was detected in native code outside the VM

[19/Dec/2006:17:59:42] config (16423):

[19/Dec/2006:17:59:42] config (16423): #

[19/Dec/2006:17:59:42] config (16423):

[19/Dec/2006:17:59:42] config (16423): # Java VM: Java HotSpot(TM) Server VM (1.4.1_01-b01 interpreted mode)

[19/Dec/2006:17:59:42] config (16423):

[19/Dec/2006:17:59:42] config (16423): #

[19/Dec/2006:17:59:42] config (16423):

[19/Dec/2006:17:59:42] config (16423): # An error report file has been saved as hs_err_pid16423.log.

[19/Dec/2006:17:59:42] config (16423):

[19/Dec/2006:17:59:42] config (16423): # Please refer to the file for further information.

[19/Dec/2006:17:59:42] config (16423):

[19/Dec/2006:17:59:42] config (16423): #

[19/Dec/2006:17:59:42] config (16423):

just_a_kid83a at 2007-7-7 17:00:44 > top of Java-index,Web & Directory Servers,Directory Servers...
# 2

i have no idea

whether this is a java version issue

jvm 1.4 in Solaris 8

and 1.5 in solaris 10

when i restart Solaris 8

it will hit this problem

but my another box in Solaris 10 works okay

i guess something wrong when i restarting my Solaris 8 Box since there is master-master replica to Solaris 10?

just_a_kid83a at 2007-7-7 17:00:44 > top of Java-index,Web & Directory Servers,Directory Servers...
# 3

Now Everything Seems okay

i found that the /var directory is running out of space

therefore i deleted some of large unecessary files and restart the ldap

and it works fine until now

why the JVM/LDAP Server crashes when there is no space.. shouldnt be the log saying that there is no enough space or something?

just_a_kid83a at 2007-7-7 17:00:44 > top of Java-index,Web & Directory Servers,Directory Servers...