SocketChannel created in CLOSE_WAIT and never cleaned up - REDUX
Hi,
I am having the same issue as is described in the following bug:
http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6215050
My jvm is :
java version "1.5.0_07"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_07-b03)
Java HotSpot(TM) Client VM (build 1.5.0_07-b03, mixed mode, sharing
running in a win2k3 environment
the bug is listed as closed/fixed, but it seems many people(including myself) are still having the same issue.
in a nutshell, i am accumulating connections on my tomcat servers that stay in a CLOSE_WAIT state...like this:
TCP172.19.1.130:16002172.19.1.130:8080CLOSE_WAIT
TCP172.19.1.130:16005172.19.1.130:8080CLOSE_WAIT
TCP172.19.1.130:16006172.19.1.130:8080CLOSE_WAIT
TCP172.19.1.130:16011172.19.1.130:8080CLOSE_WAIT
TCP172.19.1.130:16012172.19.1.130:8080CLOSE_WAIT
TCP172.19.1.130:16014172.19.1.130:8080CLOSE_WAIT
TCP172.19.1.130:16015172.19.1.130:8080CLOSE_WAIT
TCP172.19.1.130:16017172.19.1.130:8080CLOSE_WAIT
TCP172.19.1.130:16019172.19.1.130:8080CLOSE_WAIT
TCP172.19.1.130:16022172.19.1.130:8080CLOSE_WAIT
TCP172.19.1.130:16023172.19.1.130:8080CLOSE_WAIT
i get so many that network stops functioning and i have to restart tomcat.
Does anyone know if this is being worked on by SUN? Is there another bug report that they are doing their work on? Is there any work-around or fix?
please help, my production systems are not stable!
thanks
Hi,
Thanks for you reply. I thought about what you said and thought it would a good idea to start over......Let me explain my problem.
For a log time now, my tomcat application will lock up. when it is locked I see MANY sockets in a close_wait state(see above). they go away when i restart tomcat. to diagnose the problem, i took the test application from the bug and i changed it to use the commons http client and ran it agains my tomcat servers
the result is that on BOTH the client running the test application and on the tomcat web server, i accumulate close_wait connections.
i guess i am not sure that it is the same problem as the bug, but it does seem to be a jvm issue since on the client, tomcat is not involved. the common layer is java.
here is the test app, please let me know what you think
package com.test;
/*
* $Author: $
* $Revision: $
* $LastChangedDate: $
* $Id: $
*/
import java.io.IOException;
import java.io.InputStream;
import java.net.SocketException;
import org.apache.commons.httpclient.HttpClient;
import org.apache.commons.httpclient.HttpException;
import org.apache.commons.httpclient.HttpMethod;
import org.apache.commons.httpclient.HttpStatus;
import org.apache.commons.httpclient.HttpVersion;
import org.apache.commons.httpclient.methods.GetMethod;
import org.apache.commons.httpclient.params.HttpClientParams;
import org.apache.commons.httpclient.params.HttpConnectionParams;
// Inspired by code from
// http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6215050
public class TestCloseHttp_ForPosting implements Runnable
{
public static final String SMTP_HOSTNAME = "somehost";
public void run()
{
HttpMethod getMeth = null;
String sErrorMsg = null;
String url = "http://" + SMTP_HOSTNAME + "/test/test.html";
try
{
HttpClient client = new HttpClient();
int iHttpTimeOut = 3000;
client.getParams().setParameter("http.protocol.version", HttpVersion.HTTP_1_1);
client.getParams().setParameter("http.socket.timeout", new Integer(iHttpTimeOut));
client.getParams().setParameter(HttpClientParams.ALLOW_CIRCULAR_REDIRECTS, false);
client.getParams().setParameter(HttpConnectionParams.SO_LINGER, 1);
getMeth = new GetMethod(url);
getMeth.setFollowRedirects(false);
int iHttpRetCode = client.executeMethod(getMeth);
if (iHttpRetCode != HttpStatus.SC_OK)
{
sErrorMsg = "client.executeMethod(getMeth)=" + iHttpRetCode;
sErrorMsg += "\n" + url;
System.err.println(sErrorMsg);
}
InputStream in = getMeth.getResponseBodyAsStream();
int iTest = 0;
byte bArr[] = new byte[1024 * 1];
while (iTest > -1)
{
iTest = in.read(bArr);
}
in.close();
long end = System.currentTimeMillis();
}
catch (HttpException e)
{
sErrorMsg = "HttpException: run";
sErrorMsg += "\n" + url;
System.err.println(sErrorMsg);
}
catch (SocketException e)
{
sErrorMsg = "SocketException: run";
sErrorMsg += "\n" + url;
System.err.println(sErrorMsg);
}
catch (IOException e)
{
sErrorMsg = "IOException: run";
sErrorMsg += "\n" + url;
System.err.println(sErrorMsg);
}
catch (Exception e)
{
sErrorMsg = "Exception: run";
sErrorMsg += "\n" + url;
System.err.println(sErrorMsg);
}
finally
{
getMeth.releaseConnection();
}
}
public static void main(String[] args)
{
TestCloseHttp_ForPosting test = new TestCloseHttp_ForPosting();
for (int i = 0; i < 15000; i++)
{
// this bug seems only to appear if different threads are reading the channels
Thread thread = new Thread(test);
thread.start();
try
{
thread.join();
}
catch (InterruptedException e)
{
}
}
System.err.println("Going to sleep.... run netstat -an | grep CLOSE_WAIT ");
while (true)
{
try
{
Thread.sleep(5000);
}
catch (InterruptedException e)
{
}
}
}
}