Port 166 Not Responding

Good luck with this one. I opened up a case with Sun that lasted over a month and we never found a solution. One thing you may try is putting the ip in the /var/opt/SUNWsymon/cfg/domain-config.x file in place of the hostname. That is one thing sun suggested. Although it didnt work on all of mine....I think it may have addressed some of the others. It's hard to tell due to the random occurences of this known issue. If you do find a permanent fix please let us know.

Sun also recommended patch 122498...which we then applied. That seemed to temporarily fix the problem, but then it re-appeared. If you have emails setup for ports not responding and responding I get mine 4 min apart. I'm guessing thats because when it polls again 4 min later.

Here's an excerpt of an email i sent to sun during this case. Note the ports for the 15k sc are 166 which is what the case was opened for.

Something else that I just found very interesting that is worth mentioning...

When I originally opened this case it was opened for this issue on the 15k sc's although I had noticed the problem on a couple of our servers loaded with solaris 10. Brief topology...just as an estimate we have probably 140ish running sol 8, maybe 15-20 running 9, and less than 10 servers running sol10 all monitored with sunmc. 2 different installations of SunMC - in two different locations.( our failover location basically).

Anyway I see this problem somewhat consistent on the 15k system controllers and since I had different issues with sol10 and sunmc I mentioned it when I opened the case, but I really wasnt focusing on it at the time..pretty much just wrote it off due to the communication issues with sunmc and the new os(new to our environment at least). I started looking at the solaris 10 box a few minutes ago and discovered this..

8/3/06 12:26PM alarm(PROD Agent on host SUNSERVER01, port 1100 not responding)

8/3/06 12:30PM alarm cleared

8/2/06 4:26AM alarm(PROD Agent on host SUNSERVER01, port 1100 not responding)

8/2/06 4:30AM alarm cleared

Thats on the hour(minute26 instead of minute57) with a 4 min alarm/clear state exactly the same as the 15k sc's do. This solaris10 box is a v440 in a different building reporting to a different sunmc master server. Not even the same sunmc domain.

These are the only servers with this problem. A small number of our solaris 10 boxes and the spare 15k sc's. Our sol10 boxes are 440's and 480's

hurtnn at 2007-7-7 3:10:05 >