smpatch and Sun Update Manager broken
Hello,
I am running Sol 10 11/06 for sparc, fully patched. A few weeks ago, smpatch and Sun Update Manager (SUM) started behaving very badly. smpatch analyze works OK, but smpatch and SUM now hang apparently forever when updates are being installed. smpatch will download selected patches to the spool directory, but will not install them. I can still manually download and install patches with patchadd. I was able to use SUM to remove a patch, but it took an abnormally long time to do so.
Also, when SUM is running, all related processes seem to be duplicated. This is mentioned in:
http://www.sun.com/bigadmin/features/hub_articles/sc_update_rn.jsp
There have been recent patches to the update manager client (e.g. 121118-12). Maybe this or other patches are causing problems? I temporarily removed this patch yesterday, but that didn't seem to help, and I re-installed it.
Does anyone have any ideas as to the source(s) and fixes to these problems?
Thank you.....
# 1
About 85% of the time, I receive no responses to my entries in this forum, and end up replying to myself with fixes and discoveries. My faith in the utility of this forum has drastically declined.
After removing and re-installing all packages and patches related to smpatch and SUM, the above problem still occurs. Most of my hangs occur with smpatch update, not smpatch analyze (both at command line and in SUM).
In the BigAdmin document (link above) about SUM 1.0.9, the "workaround" for the hanging of smpatch is given as "kill the processes and retry." Sun is saying that their software doesn't always work, they don't know why, and the only solution is for users waste their time trying and retrying it in the hopes that it will work. This is total BS.
I have killed and retried many times, all to no avail.
WTF?
# 2
Have you tried running it in debug mode to see what errors might be happening:
smpatch download -C patchpro.log.level=3 -C patchpro.debug=true
I had similar problems and Sun told me to reregister the system (which didn't help), but I will post the directions here for you to try:
Please find below the link containg the instructions for command line
registration:
http://sunsolve.sun.com/search/document.do?assetkey=1-9-82688-1
You will need to create a "registrationprofile.properties" file, containing
the following information:
userName=<your_sun_online_account_name>
password=<your_sun_online_account_password>
hostName=
subscriptionKey=<your contract number>
portalEnabled=false
proxyHostName=
proxyPort=
proxyUserName=
proxyPassword=
Then save the file, e.g.
/tmp/registrationprofile.properties
Then run the following commands as root:
# cacaoadm stop
# cacaoadm status
# /usr/lib/cc-ccr/bin/eraseCCRRepository
# rm /var/scn/persistence/SCN*
# cacaoadm start
# cat /tmp/registrationprofile.properties
# /usr/sbin/sconadm register -a -r /tmp/registrationprofile.properties
# /usr/lib/cc-ccr/bin/ccr -g cns.assetid
# smpatch analyze
# smpatch download
Now on to the part you probably don't want to hear.... I think smpatch stinks. I have had way too many problems with so I dumped it and use 'pca' (patch check advanced) now.You can find it here:
http://www.par.univie.ac.at/solaris/pca/
I like the fact it does not overwrite custom configuration files during patch install, it has a ton of great options, and most of all it "just works".
# 3
Thank you very much for the info! I am also not entirely enamored with smpatch, to say the least.
SUM and smpatch are working a little better since I re-enabled the daily patch analysis and notification icon. There is a bug regarding the icon when smpatch is run from remote systems, so maybe (?) the icon issue has something to do with our problems. Who knows.
I'll look into pca.
Thanks again!
# 4
Yesterday, 4 new patches for Sol 10 were released. I had to spend about 1.5 hours getting SUM and smpatch to work installing the patches. That was quite frustrating and maddening, to say the least, and I had to reboot the system at least once in attempting to get the patches installed. But at least it finally did work, and I did not have to re-register the system. Running smpatch in debug mode did not uncover any obvious errors.
As stated above, re-registering the system is NOT a fix for these problems. There are one or more TERRIBLE BUGS in SUM client 1.0.9/smpatch. The problem may have something to do with the daily update analysis and notification icon, and these features, set via preferences in SUM, appear unstable.
I am very confused about two things:
1) Why did Sun release such woefully bad SUM software, when substantial bugs were known (see release notes)?
2) Why aren't more administrators complaining about these problems? These SUM/smpatch problems have dire consequences for system maintenance, and require IMMEDIATE fixes. When will a new version of the client come out?
# 5
Above I mentioned that smpatch/SUM-related processes are duplicated when smpatch update hangs upon patch installation, and that this problem had been mentioned in the release notes for the 1.0.9 client. I was wrong that the duplicate processes were mentioned in the release notes. They are not, and this process duplication problem could be related to the hanging of smpatch.
Here is the results of "ps -ef | grep patch" when smpatch update hangs upon patch install:
username 136510 18:39:38 ?0:10 /usr/bin/java -jar /usr/lib/patch/swupna.jar
root 2305 22750 18:48:05 ?0:00 /usr/bin/java -Djava.library.path=/usr/lib/cc-ccr/lib com.sun.patchpro.cli.Patc
root 2270 22610 18:45:34 ?0:00 smpatch update -i 120222-18 -i 119115-28 -i 119090-24 -i 120099-08 -C patchpro.
username 1061 10550 18:33:40 ?2:34 /usr/bin/java -client -jar /usr/lib/patch/swupdate.jar
root 1072 10610 18:33:57 ?0:00 sh -c /usr/lib/patch/csm
root 1073 10720 18:34:01 ?0:00 /usr/lib/patch/csm
root 2261 10730 18:45:33 ?0:00 smpatch update -i 120222-18 -i 119115-28 -i 119090-24 -i 120099-08 -C patchpro.
root 2275 22660 18:45:34 ?1:20 /usr/bin/java -Djava.library.path=/usr/lib/cc-ccr/lib com.sun.patchpro.cli.Patc
Note the duplicated smpatch and java processes.
# 6
Ah ha! After I killed the secondary smpatch and java processes (the second of each to start, in this case processes 2270 and 2305), smpatch immediately started to install the patches.
However, one patch Failed:
120099-08
APOC 1.2: Sun Java(tm) Desktop System Configuration Shared Libraries
Failed
Install of update failed. Utility used to install the update could not find the tool to install packages. Utility used to install the update failed with exit code 9.
The problem with installing this patch could be unrelated to the smpatch problem. Hopefully the ability to end the hang of smpatch/SUM by killing the secondary processes will help some users.
# 7
sparcmaster, I'm having the exact same problem. It all started after the latest SUM release. I agree that this release has caused me countless hours of trouble-shooting. Before finding this tread I had tried all the above suggestions to no avail except for killing the secondary smpatch and java processes.
I will try that and post back my results.
# 8
To answer the question, "Why aren't more administrators complaining about these problems?"
I have many times. I have noted it using support cases when I have problems. I have talked with Sun engineers while attending events at Sun offices. I have brought it up at Solaris user group meetings (when Sun employees are present). I have brought it up with my local Sun regional manager.
I can't talk about it any more. I think Sun patching process is horrible. Not only does the software not work well the patching process itself I think is flawed. A great example is when sendmail binaries and configuration files are included in a kernel patch and it overwrites all your custom changes.A kernel patch is a kernel patch, not a sendmail patch. </rant>
Like I said earlier, PCA is the way to go for now.
./pca --install missingrs --safe -a
(Install only missing security/recommended patches and don't overwrite my custom configuration files).
# 9
I had secretly thought that others were having these problems and were telling Sun about it, but I included that question to fire some people up to respond to this thread.
Why in the world would sendmail binaries and config files be included in a kernal patch? That reminds me of a terrible problem I had several years ago at work regarding sendmail and patching. There used to be a security vulnerability bug in sendmail that cause a lot of alarm. Because of the bug, we were not allowed to have the sendmail daemon running, and I had disabled it (Solaris 8). However, after patching my system one day, the sendmail binaries and config files of mine were overwritten, and the process started again after reboot. A remote security scan detected the daemon, and they took away my box. That sucked.
# 10
sendmail patches have been notorious for things like this. I think every sendmail patch has always overwritten my custom configs, enabled sendmail when I have disabled it, etc..
I think SMF will solve the startup problem. They could easily fix the "overwrite config" problem by at least running a post-install script that runs the M4 processor to make the sendmail configs using my custom sendmail.mc file.
Until that happens like I preach.. PCA has the --safe option to stop overwriting configs like this.
# 11
I haven't used PCA, but I will now strongly consider it.
I hope now that Ian Murdock (Debian Founder) has joined Sun as Chief Operating Platforms Officer, he will help to introduce better patch/package management options. Debians APT package manager was in my opinion one of the major foundations to the success of Debian and now of Ubuntu. BSD Ports is also a great one that has allowed several successes.
# 12
I have found that killing the secondary sendmail update and java processes does stop the hang, but always results in the installation failure of one of the patches.
The hangs occur with smpatch update even when there is only one patch to install.
Process duplication appears to be near the heart of this HORRIBLE PROBLEM with Sun Update Manager client 1.0.9.
Please release a new version of the client, SUN!!!*************************
I have not reported all this as a bug to Sun. Does anyone think I should? If so, how? I do not have an active Sun Support contract. Probably others have reported these problems to Sun, correct?
# 13
OK, tried killing the secondary processes and it worked!Thanx sparcmaster.I did get one of the failed patches messages however. I re-ran the update manager and everything worked.
# 14
I have switched forums for this matter to: http://forum.java.sun.com/thread.jspa?threadID=5181777&tstart=0I will not post any further messages to this thread.
