Apache Policy Agent 2.2 causes segmentation fault when SSL enabled

Policy Agent 2.2

Apache 2.0.59

RedHat Enterprise Linux

I am able to get everything working with a stock compile of the apache source files. If I enable SSL, I am getting a segmentation fault during server startup in the error_log file and it just keeps repeating over and over again.

Log output:

[Tue Oct 17 16:32:07 2006] [info] Init: Initializing OpenSSL library

[Tue Oct 17 16:32:07 2006] [info] Init: Seeding PRNG with 136 bytes of entropy

[Tue Oct 17 16:32:07 2006] [info] Init: Generating temporary RSA private keys (512/1024 bits)

[Tue Oct 17 16:32:08 2006] [info] Init: Generating temporary DH parameters (512/1024 bits)

[Tue Oct 17 16:32:08 2006] [warn] Init: Session Cache is not configured [hint: SSLSessionCache]

[Tue Oct 17 16:32:08 2006] [info] Init: Initializing (virtual) servers for SSL

[Tue Oct 17 16:32:08 2006] [info] Server: Apache/2.0.59, Interface: mod_ssl/2.0.59, Library: OpenSSL/0.9.7a

[Tue Oct 17 16:32:09 2006] [info] Init: Initializing OpenSSL library

[Tue Oct 17 16:32:09 2006] [info] Init: Seeding PRNG with 136 bytes of entropy

[Tue Oct 17 16:32:09 2006] [info] Init: Generating temporary RSA private keys (512/1024 bits)

[Tue Oct 17 16:32:09 2006] [info] Init: Generating temporary DH parameters (512/1024 bits)

[Tue Oct 17 16:32:09 2006] [info] Init: Initializing (virtual) servers for SSL

[Tue Oct 17 16:32:09 2006] [info] Server: Apache/2.0.59, Interface: mod_ssl/2.0.59, Library: OpenSSL/0.9.7a

[Tue Oct 17 16:32:11 2006] [notice] Apache/2.0.59 (Unix) mod_ssl/2.0.59 OpenSSL/0.9.7a configured -- resuming normal operations

[Tue Oct 17 16:32:11 2006] [info] Server built: Oct 17 2006 15:56:10

[Tue Oct 17 16:32:12 2006] [notice] child pid 30711 exit signal Segmentation fault (11)

[Tue Oct 17 16:32:12 2006] [notice] child pid 30712 exit signal Segmentation fault (11)

[Tue Oct 17 16:32:14 2006] [notice] child pid 30767 exit signal Segmentation fault (11)

[Tue Oct 17 16:32:16 2006] [notice] child pid 30795 exit signal Segmentation fault (11)

[Tue Oct 17 16:32:18 2006] [notice] child pid 30824 exit signal Segmentation fault (11)

This also occurs on a 2.0.58 Apache instance as well.

Is there some way I can further debug this to see what is wrong?

[2350 byte] By [billyk] at [2007-11-26 10:52:21]
# 1
The debug log for the agent would be more useful.Set the debug level up to all:5 in the AMAgent.properties file and then look at the agent log.
ChristopherNebergall at 2007-7-7 3:05:12 > top of Java-index,Web & Directory Servers,Directory Servers...
# 2

I am not sure exactly what to look for, but this is what I find:

2006-10-17 17:56:13.166Debug 24845:8271ed8 all: Log::pSetLevelsFromString(): setting log level for module 0 to 5, old level 1.

2006-10-17 17:56:13.166-1 24845:8271ed8 all: =======================================

2006-10-17 17:56:13.166-1 24845:8271ed8 all: Version: Version: 2.2

2006-10-17 17:56:13.166-1 24845:8271ed8 all: Build Date: Tue Oct 4 09:47:30 PDT 2005

2006-10-17 17:56:13.166-1 24845:8271ed8 all: =======================================

.........

2006-10-17 17:56:14.249Debug 24845:8271ed8 PolicyAgent: am_web_init: Using cookie prefix HTTP_.

2006-10-17 17:56:14.249Debug 24845:8271ed8 PolicyAgent: am_web_init: Using cookie max-age 300.

2006-10-17 17:56:14.249Debug 24845:8271ed8 PolicyAgent: Process initialization result:success

2006-10-17 17:56:15.096Debug 24848:82819c8 all: Log::pSetLevelsFromString(): setting log level for module 0 to 5, old level 1.

2006-10-17 17:56:15.097-1 24848:82819c8 all: =======================================

2006-10-17 17:56:15.097-1 24848:82819c8 all: Version: Version: 2.2

2006-10-17 17:56:15.097-1 24848:82819c8 all: Build Date: Tue Oct 4 09:47:30 PDT 2005

2006-10-17 17:56:15.098-1 24848:82819c8 all: =======================================

.........

2006-10-17 17:56:16.225Debug 24848:82819c8 PolicyAgent: am_web_init: Using cookie prefix HTTP_.

2006-10-17 17:56:16.226Debug 24848:82819c8 PolicyAgent: am_web_init: Using cookie max-age 300.

2006-10-17 17:56:16.227Debug 24848:82819c8 PolicyAgent: Process initialization result:success

The only other thing that strikes me as odd is it looks like it initializes itself twice. I see all the messages go through, and then 1 second later the entire initialization process runs through again. Is this normal?

There is a ton of information contained in the log, is there something specific I should be looking for?

I took the properties file just as it is from the initial config_linux output and ran it. Is there something specific that I need to update in here afterwards that would be causing failure?

Also, if I remove the include /..../dsame.conf reference in httpd.conf, the web server starts up just fine.

billyk at 2007-7-7 3:05:12 > top of Java-index,Web & Directory Servers,Directory Servers...
# 3
Whatever was the last thing before it died.Also, if it dropped core you can look at the core with GDB and get a stack trace.
ChristopherNebergall at 2007-7-7 3:05:12 > top of Java-index,Web & Directory Servers,Directory Servers...
# 4

No core file exists.

Running pstack gives the following:

[root@teamsited03 /]# /usr/bin/pstack 15836

Thread 2 (Thread 27175856 (LWP 15839)):

#0 0x006bf7a2 in _dl_sysinfo_int80 () from /lib/ld-linux.so.2

#1 0x00a3dd9c in pthread_cond_timedwait@@GLIBC_2.3.2 ()

#2 0x00ae3df6 in PR_Lock () from /usr/lib/libnspr4.so

#3 0x00ae417f in PR_WaitCondVar () from /usr/lib/libnspr4.so

#4 0x00e18de6 in smi::HTCleaner<smi::SSOTokenEntry>::operator() ()

#5 0x00dc34fa in spin () from /usr/local/agents/apache/lib/libamapc2.so

#6 0x00ae8dc1 in PR_Select () from /usr/lib/libnspr4.so

#7 0x00a3b371 in start_thread () from /lib/tls/libpthread.so.0

#8 0x0079fffe in clone () from /lib/tls/libc.so.6

Thread 1 (Thread -1208142144 (LWP 15836)):

#0 0x006bf7a2 in _dl_sysinfo_int80 () from /lib/ld-linux.so.2

#1 0x00798a41 in newselect_nocancel () from /lib/tls/libc.so.6

#2 0x0068144f in apr_sleep () from /opt/apache/lib/libapr-0.so.0

#3 0x080b66f2 in ap_wait_or_timeout ()

#4 0x080aa731 in ap_mpm_run ()

#5 0x080b079f in main ()

I am not sure if that helps anyone else out there. I will continue checking the apache site for how to get to the bottom of this.

billyk at 2007-7-7 3:05:12 > top of Java-index,Web & Directory Servers,Directory Servers...
# 5
Where you able to intall without SSL.I get Segmentation fault error with the base install
raghu8180 at 2007-7-7 3:05:12 > top of Java-index,Web & Directory Servers,Directory Servers...