What does this mean in my logfiles?

Kent,

This seems to be a common problem. At least, I know I have it.

One of the causes is ipshttpd not handling its sessions properly and

doesn't full close them when they're done. Here's how to check if this is

your problem:

# netstat | grep CLOSE_WAIT | wc -l

If the number returned when you run this on your server is over a thousand

or so when it crashes, then this might be your problem.

My own investigations have led me to believe that the JRE is the culprit.

The good news:

- If you iPlanet support and tell them this, they'll probably send you

HP3, which is sort of a 'beta' SP3.

The bad news:

- I got HP3 and installed it in a test environment but couldn't get my

servers talking once I did.

Good news:

- the real SP3 was supposed to be out last week.

Bad news:

- It's still not out AFAIK.

Good news:

- I have installed a 'cludge' work around until I can solve the real

issue.

Bad news:

- I can give it to you, but then I might have to kill you. (Just

kidding)

Here is what I did:

- On the server as a cron job that runs every 3 minutes, I installed a

perl script that will act as a web browser and try to browse iPS as a normal

user would. If it comes back with the ominous '502 Bad Gateway' error, then

it will restart the ipshttpd service. If it has to restart more than 3 times

in a row (that is, three times without successfully connecting once), it

will create a file in the /opt/SUNWips/public_html/images directory (this

directory doesn't require authentication first before browsing it). Then it

will reboot the whole ipsserver, ipshttpd, and ipsnetletd kit-and-kaboodle..

In case bouncing just ipshttp isn't doing the trick.

Then on the gateway, every 3 minutes it checks for the existence of this

file by trying to browse to it using a similar script. If it does NOT get a

404, it will reboot ipsgateway (the gateway needs to be bounced after the

server gets bounced).

And, of course, any time it does anything it emails me to let me know what

it's doing.

Here's the script:

#!/usr/bin/perl

use LWP::UserAgent;

my $now = localtime;

my $mailprog = '/usr/lib/sendmail -oeq -t';

my $recipient = 'sababineau@tasc.com';

my $sender = 'iplanetchecker@tasc.com';

my $logfile = '/opt/SUNWips/etc/ipshttpd.log';

my $ua = new LWP::UserAgent;

my $req = new HTTP::Request('GET', 'https://iplanet.tasc.com');

my $res = $ua->request($req);

if ($res->is_success) {

$webpage = $res->content;

}

@webpage = split(/\n/,$webpage);

foreach $line (@webpage) {

# In between the /'s, enter in some text that is on the page that comes back

when it's successful, but not on the error pages.

if ($line =~ /Internet Remote Access Portal :: Login/) {

$found = "true";

}

}

if ($found ne "true") {

system('/opt/SUNWips/bin/ipshttpd start >/dev/null');

open (LOG, "<$logfile");

while (<LOG>) { $num = $_; }

$num++;

close(LOG);

open (LOG, ">$logfile");

print LOG $num;

close(LOG);

open (MAIL, "|$mailprog");

print MAIL "To: $recipient\n";

print MAIL "From: $sender\n";

print MAIL "Subject: iPlanet isn't working!\n\n";

print MAIL "iPlanet doesn't seem to be working, so I bounced

</opt/SUNWips/bin/ipshttpd> on $now.\n";

print MAIL "It's been bounced $num time(s) since the last successful

connection.";

close (MAIL);

} else {

open (LOG, ">$logfile");

print LOG "";

close(LOG);

}

exit;

Good luck,

/Steve

sababineau@tasc.com

708764 at 2007-6-29 2:28:58 >