patch proxy not working reliably

We're having problems with the patch proxy.

If I point a relativel unpatched box at it, it offers me 90 patches.

If I try to install them all, it grinds away and installs a few patches successfully.

But most of the patches fail with a message that

Patch was not found on http://proxyserver:3816/solaris.

If I reboot and run it again, it will install a few more patches successfully but fail the same way for the rest.

After about 6 iterations of this I finally got it down to 10 patches to install, but all of those fail.

I then poiint it at the sun server instead of the local proxy, it offers me those 10 + another 3.

And all of those install without problems.

[719 byte] By [robert.cohen] at [2007-11-26 6:03:21]
# 1

We have addressed some of this by using smpatch download first, but even this is frustrating since it keeps failing. The client might get a few patches, then the rest fail and you run it again and you get a few more. If you are lucky and you run it enough times you can actually get all the patches to the client.

It seems like a timout issue on the client while waiting for the proxy to download the patches. I say this because once the patches are cached on the proxy you don't see this with other clients that need those same patches.

Hopefully someone at Sun is listening. Sun, why don't you just let us use wget to populate our proxy's every night until you get your act together on this very poorly written software. We wouldn't be in this fix if you hadn't removed support for it in the first place!

jwbledsoe at 2007-7-6 13:27:53 > top of Java-index,Administration Tools,Sun Update Connection-System...
# 2

I suspect more information on this issue could be gathered from the log files in /var/patchsvr/log/localhost_log.<date>.txt

on the system which is running the Sun Update Connection Proxy.

If you could please post the relevant errors or stack traces you see at the time of failure it would be most helpful in solving this problem.

TIA

-ethan

ejrider at 2007-7-6 13:27:53 > top of Java-index,Administration Tools,Sun Update Connection-System...
# 3

Here's some examples all run the same day, I'll get some logs and put them up too.

# smpatch download

114990-04 has been validated.

The following patches were not downloaded:

109326-17Unexpected error while accessing the requested patch.

116965-13Unexpected error while accessing the requested patch.

116953-03Unexpected error while accessing the requested patch.

109320-15Unexpected error while accessing the requested patch.

112237-12Unexpected error while accessing the requested patch.

108652-92Unexpected error while accessing the requested patch.

# smpatch download

109320-15 has been validated.

112237-12 has been validated.

The following patches were not downloaded:

109326-17Unexpected error while accessing the requested patch.

116965-13Unexpected error while accessing the requested patch.

116953-03Unexpected error while accessing the requested patch.

108652-92Unexpected error while accessing the requested patch.

# smpatch download

The following patches were not downloaded:

109326-17Unexpected error while accessing the requested patch.

116965-13Unexpected error while accessing the requested patch.

116953-03Unexpected error while accessing the requested patch.

108652-92Unexpected error while accessing the requested patch.

# smpatch download

The following patches were not downloaded:

109326-17Unexpected error while accessing the requested patch.

116965-13Unexpected error while accessing the requested patch.

116953-03Unexpected error while accessing the requested patch.

108652-92Unexpected error while accessing the requested patch.

#

jwbledsoe at 2007-7-6 13:27:53 > top of Java-index,Administration Tools,Sun Update Connection-System...
# 4

I ran a new test and put the tarball containing the smpatch download output and all four log files generated for today. Only this attempt is in there.

Sunsolve:/cores/patch_proxy_dl_issue_jwb.tar.gz

Some patches were downloaded and some where not. It appears that the ones that failed were due to resources or authentication, presumably at updateserver. I'm not sure about authentication being an issue since the patch proxy server can download its own patches just fine and I assume the proxysvr uses that server reg as the source.

Here's some snipits from the logs

Cannot establish a network connection with the patch server ("https://getupdates.sun.com/solaris/"). Check your proxy configuration or contact your local system administrator for assistance.

... Trying to use local data to service request.

Cannot establish a network connection with the patch server ("https://getupdates.sun.com/solaris/"). Check your proxy configuration or contact your local system administrator for assistance.

... Trying to use local data to service request.

109320-15 has been validated.

112237-12 has been validated.

Cannot establish a network connection with the patch server ("https://getupdates.sun.com/solaris/"). Check your proxy configuration or contact your local system administrator for assistance.

... Trying to use local data to service request.

Cannot establish a network connection with the patch server ("https://getupdates.sun.com/solaris/"). Check your proxy configuration or contact your local system administrator for assistance.

... Trying to use local data to service request.

////////////

# cat localhost_log.2005-08-03.txt

2005-08-03 11:07:01 Wed Aug 03 11:07:01 PDT 2005(INFO) => com.sun.patchpro.server.PatchProServerServlet@5eb489 <=Protocol version: 2.1

2005-08-03 11:07:01 Wed Aug 03 11:07:01 PDT 2005(DEBUG) => com.sun.patchpro.server.PatchProServerServlet@5eb489 <=Requested Database Name: patchdb

2005-08-03 11:07:01 Wed Aug 03 11:07:01 PDT 2005(DEBUG) => com.sun.patchpro.server.ServerPatchServiceProvider@160877b <=cacheFile is: /jumpstart/patches/patchsvr/Misc/https%3A%2F%2Fgetupdates.sun.com%2Fsolaris%2F% 2Fpatchdb.zip

2005-08-03 11:07:01 Wed Aug 03 11:07:01 PDT 2005(DEBUG) => com.sun.patchpro.server.ServerPatchServiceProvider@160877b <=POST String: action=downloadPatchDB&name=patchdb&version=2.1

2005-08-03 11:07:01 Wed Aug 03 11:07:01 PDT 2005(WARNING) => com.sun.patchpro.util.CachingDownloader@53c3f5 <=Cannot connect to retrieve patchdb, using cached copy: java.security.ProviderException: Could not obtain session

2005-08-03 11:07:01 Wed Aug 03 11:07:01 PDT 2005(DEBUG) => com.sun.patchpro.util.CachingDownloader@53c3f5 <=javax.net.ssl.SSLException: java.security.ProviderException: Could not obtain session

at com.sun.net.ssl.internal.ssl.Alerts.getSSLException(Alerts.java:166)

at com.sun.net.ssl.internal.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1476)

at com.sun.net.ssl.internal.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1443)

at com.sun.net.ssl.internal.ssl.SSLSocketImpl.handleException(SSLSocketImpl.java:1 426)

at com.sun.net.ssl.internal.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:10 45)

at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:405)

at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractD elegateHttpsURLConnection.java:170)

at sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnection.j ava:828)

at com.sun.net.ssl.internal.www.protocol.https.HttpsURLConnectionOldImpl.getOutput Stream(HttpsURLConnectionOldImpl.java:200)

at com.sun.patchpro.util.Downloader.connectToURL(Downloader.java:343)

at com.sun.patchpro.util.CachingDownloader.establishConnection(CachingDownloader.j ava:584)

at com.sun.patchpro.util.CachingDownloader.setSourceURL(CachingDownloader.java:274 )

at com.sun.patchpro.util.CachingDownloader.setupCache(CachingDownloader.java:200)

at com.sun.patchpro.util.CachingDownloader.<init>(CachingDownloader.java:185 )

at com.sun.patchpro.server.ServerPatchServiceProvider.downloadFile(ServerPatchServ iceProvider.java:775)

at com.sun.patchpro.server.ServerPatchServiceProvider.downloadOldPatchDB(ServerPat chServiceProvider.java:443)

at com.sun.patchpro.server.PatchServerProxy.downloadOldPatchDB(PatchServerProxy.ja va:121)

at com.sun.patchpro.server.PatchProServerServlet.doPost(PatchProServerServlet.java :632)

at javax.servlet.http.HttpServlet.service(HttpServlet.java:760)

at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)

at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFil terChain.java:247)

at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain .java:193)

at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java: 243)

at org.apache.catalina.core.StandardPipeline.invokeNext(StandardPipeline.java:566)

at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:472)

at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:943)

at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java: 190)

at org.apache.catalina.core.StandardPipeline.invokeNext(StandardPipeline.java:566)

at org.apache.catalina.valves.CertificatesValve.invoke(CertificatesValve.java:246)

at org.apache.catalina.core.StandardPipeline.invokeNext(StandardPipeline.java:564)

at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:472)

at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:943)

at org.apache.catalina.core.StandardContext.invoke(StandardContext.java:2347)

at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:180)

at org.apache.catalina.core.StandardPipeline.invokeNext(StandardPipeline.java:566)

at org.apache.catalina.valves.ErrorDispatcherValve.invoke(ErrorDispatcherValve.jav a:170)

at org.apache.catalina.core.StandardPipeline.invokeNext(StandardPipeline.java:564)

at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:170)

at org.apache.catalina.core.StandardPipeline.invokeNext(StandardPipeline.java:564)

at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:468)

at org.apache.catalina.core.StandardPipeline.invokeNext(StandardPipeline.java:564)

at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:472)

at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:943)

at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:17 4)

at org.apache.catalina.core.StandardPipeline.invokeNext(StandardPipeline.java:566)

at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:472)

at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:943)

at org.apache.catalina.connector.http.HttpProcessor.process(HttpProcessor.java:102 7)

at org.apache.catalina.connector.http.HttpProcessor.run(HttpProcessor.java:1125)

at java.lang.Thread.run(Thread.java:595)

Caused by: java.security.ProviderException: Could not obtain session

at sun.security.pkcs11.SessionManager.getOpSession(SessionManager.java:122)

at sun.security.pkcs11.Token.getOpSession(Token.java:241)

at sun.security.pkcs11.P11SecureRandom.engineNextBytes(P11SecureRandom.java:92)

at java.security.SecureRandom.nextBytes(SecureRandom.java:413)

at com.sun.net.ssl.internal.ssl.RandomCookie.<init>(RandomCookie.java:34)

at com.sun.net.ssl.internal.ssl.HandshakeMessage$ClientHello.<init>(Handshak eMessage.java:189)

at com.sun.net.ssl.internal.ssl.ClientHandshaker.getKickstartMessage(ClientHandsha ker.java:711)

at com.sun.net.ssl.internal.ssl.Handshaker.kickstart(Handshaker.java:522)

at com.sun.net.ssl.internal.ssl.SSLSocketImpl.kickstartHandshake(SSLSocketImpl.jav a:1101)

at com.sun.net.ssl.internal.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImp l.java:1024)

at com.sun.net.ssl.internal.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:10 38)

... 45 more

jwbledsoe at 2007-7-6 13:27:53 > top of Java-index,Administration Tools,Sun Update Connection-System...
# 5

I do have the same issue on my site.

I have to stop/start patchsvr often and even to bounce the system where runs the proxy to have it work ... sometime ... !!I'm very disapointed by SUN's software quality here !! The fact that now they advertise for that solution which is not working will make then look as "good" as "windows stuff" !!! too bad ...

forum_dev at 2007-7-6 13:27:53 > top of Java-index,Administration Tools,Sun Update Connection-System...
# 6

I am having the exact same issue. It looks like the proxy is getting http code 500 sometimes from getupdates.sun.com, or something like that. These are the two errors I'm seeing:

2005-08-10 09:23:41 Wed Aug 10 09:23:41 CDT 2005(WARNING) => com.sun.patchpro.server.ServerPatchServiceProvider@1815bfb <=com.sun.patchpro.server.DownloadPatchException: Status code 500 returned. An unexpected error occurred inside the server that prevented it from fulfilling the request.

2005-08-10 09:23:41 Wed Aug 10 09:23:41 CDT 2005(CRITICAL) => com.sun.patchpro.server.ServerPatchServiceProvider@1815bfb <=javax.net.ssl.SSLException: java.security.ProviderException: Could not obtain session

the "old" smpatch still works fine. You just have to be careful not to install 119107-03...

_peffley_ at 2007-7-6 13:27:53 > top of Java-index,Administration Tools,Sun Update Connection-System...
# 7

Not directly related to this issue, but I've had problems with smpatch on Solaris 8 and 9 ever since it came out.

Many, many (many many) times I've run smpatch download or smpatch update and watched it sit at "access patches for hostname/ip address" for what seemed like forever. Then, after a couple trips for smokes, I'd ctrl-c and start it over. It would then get maybe 5-10 downloads into the process and hang all over again. So, ctrl-c, and do it again. Eventually it would finish up and I could go home.

That's when I started downloading all the jarballs myself, place them in an nfs export and pointing all the other systems to that location for the "smpatch update" process. Not 100% failsafe, but better than the alternative.

Now, mind you, at the time we had no firewall between these systems and the public Internet (now we do), so I really don't believe any timeouts were on my end. And I know we have big fat pipes to the 'Net from the University backbone. And it happens on all system, regardless if it's an Ultra 5 or a V880.

Sorry, had to get my .02 in, perhaps they've reused the same download code in all the smpatch type of utilities and the same timeouts happen.

In Sun's defense, at least you're _trying_ to give us the tools we need to admin the OS in the "real world".

dyoung2 at 2007-7-6 13:27:53 > top of Java-index,Administration Tools,Sun Update Connection-System...
# 8
I've found that stopping and restarting the patch server seems to help...(on server) patchsvr stop; patchsvr start(on client) smpatch downloadrinse and repeat until they're all in the cache.
_peffley_ at 2007-7-6 13:27:53 > top of Java-index,Administration Tools,Sun Update Connection-System...
# 9

Here's the solution to this problem:

First, stop the proxy:

$ patchsvr stop

See if the crypto services are running:

$ svcs svc:/system/cryptosvc

You'll need to restart them if they are disabled:

$ svcadm enable svc:/system/cryptosvc

Now, you can disable the problematic component:

$ cryptoadm disable metaslot

And restart the proxy:

$ patchsvr start

Solution taken from:

http://forum.sun.com/thread.jspa?threadID=25780&start=10&tstart=0

ForumModerator at 2007-7-6 13:27:53 > top of Java-index,Administration Tools,Sun Update Connection-System...