new to NIO: how to close socketchannels properly?

Hi all,

I've recently started using the NIO packages to handle my networking applications, so i'm very new to how it all works.

I've written a simple server application that streams a constant stream of data to connecting clients, and during stress testing i've noticed a (possible) memory leak in my server.

my stress test involves initially connecting a number (lets say 1000) clients to the server, and at random time intervals, disconnecting a random number of the clients, and connecting that same number of new clients (so there are always 1000 clients connected).

using jconsole to watch the memory usage of my server it constantly increases over time. and letting it run indefinitely results in an out of memory error.

on to my server code:

i initialise the server by:

private AbstractSelector selector;

selector = SelectorProvider.provider().openSelector();

serverChannel = ServerSocketChannel.open();

serverChannel.configureBlocking(false);

InetSocketAddress isa =new InetSocketAddress(getPort());

serverChannel.socket().bind(isa);

serverChannel.register(selector, SelectionKey.OP_ACCEPT);

then start a loop (done in a Tread.run() function) that goes something like this (i've removed try catches and other complexity):

while (notkilled){

selector.select();

Set<SelectionKey> keys = selector.selectedKeys();

Iterator<SelectionKey> iterator = keys.iterator();

while (iterator.hasNext()){

SelectionKey key = iterator.next();

iterator.remove();

if (key.isAcceptable()){

ServerSocketChannel channel = (ServerSocketChannel)key.channel();

SocketChannel client = channel.accept();

client.configureBlocking(false);

SelectionKey readKey = client.register(selector, SelectionKey.OP_WRITE);

readKey.attach(new ClientCallback(client));

}

if (key.isWritable()){

writeToChannel((ClientCallback)key.attachment());

}

}

}

this is the part that i'm worried about. I'm not sure i'm properly detecting disconnects, and i'm not sure I'm getting rid of everything to do with that channel (i.e. the ClientCallback instance attached, the key, and anything else involved).

privatevoid writeToChannel(ClientCallback client){

try{

client.execute();

}catch (IOException e){

try{

System.out.println("closing connection to: " + client.getChannel().socket().getRemoteSocketAddress().toString());

client.getChannel().close();

}catch (IOException e1){

// TODO Auto-generated catch block

e1.printStackTrace();

}

}

}

and here is a very cut back representation of the client callback:

publicclass ClientCallback{

private SocketChannel channel;

public ClientCallback(SocketChannel channel){

this.channel = channel;

}

publicvoid execute()throws IOException{

channel.write(data)// psuedo coded, data exists somewhere

}

public SocketChannel getChannel(){

return channel;

}

}

Can anyone see any obvious memory leak issues in this code? As i said, i'm worried that i'm not closing the channels correctly.

Thanks in advance for any help :)

-Tristan

[4901 byte] By [tkinga] at [2007-11-26 18:58:20]
# 1
If a client disconnects you will get an OP_READ, which you should be registered for, and process. A read on such a channel will return -1 or throw an exception. You should close the channel when either of these occurs.
ejpa at 2007-7-9 20:38:37 > top of Java-index,Core,Core APIs...
# 2

ok, i've added the OP_READ to the registration, and made a simple key.isReadable() section that tries a read. if the read returns -1 i do a channel.close().

memory usage still grows tho.

running a test with 800 clients over 30 mins, the memory usage increases from 1.5mb starting to over 10mb by the end. (not a terribly big increase, but an still an unwanted one)

does a close() cause the garbage collection to clean up all the key's and attachments (i.e. the ClientCallback's) and any other references to the channel? or is there more likely to be a problem somewhere else in my code?

are there any good tools to tell which objects are currently instantiated? jconsole doesn't give much detail.

tkinga at 2007-7-9 20:38:37 > top of Java-index,Core,Core APIs...
# 3

> if the read returns -1 i do a channel.close().

Ditto if the read throws an exception.

> does a close() cause the garbage collection to clean

> up all the key's and attachments (i.e. the

> ClientCallback's) and any other references to the

> channel? or is there more likely to be a problem

> somewhere else in my code?

Close() causes all keys for the channel to be cancelled, which removes them from the selector's keyset on the next iteration of Selector.select(), so if there are no other references to the keys and their attachments they can be garbage-collected. If you have your own references to any of these objects they can't. So you shouldn't.

So I think the memory leak is yours. I have a large-scale NIO server which has been running for months with thousands of clients.

One thing I would do to your code would be to select() for a finite time, not infinity, say 5-10 minutes, and if you get a zero result meaning nothing happened in that interval I would scan for channels which haven't had any events for some other, longer, interval, say an hour or two, and time them out by closing them. The finite select() timeout will also let the key-cancellation action above happen more regularly.

> are there any good tools to tell which objects are

> currently instantiated? jconsole doesn't give much

> detail.

There are several commercial products.

ejpa at 2007-7-9 20:38:37 > top of Java-index,Core,Core APIs...