threaded inner classes & heap memory exhaustion
(_) how can i maximize my threading without running out of
heap memory?
push it to the limit, but throttle back before an
java.lang.OutOfMemoryError.
(_) within 1 threaded classThreadClass, i have two threaded inner classes. for each instance ofThreadClass i only
start one instance of each inner class.
and, i start hundreds ofThreadClass, but not until the previously runningThreadClass object exits, so only one should be running at any given time.
so, what about threaded inner classes?
are they good? bad? cause "OutOfMemoryErrors"?
are those inner threads not dying?
what are common causes of:
java.lang.OutOfMemoryError: java heap space?
my program runs for about 5-minutes, then
bails with the memory error.
how can i drill down and see what
is eating-up all my memory?
thanks.
[983 byte] By [
suppona] at [2007-11-27 5:27:40]
# 1
A Thread class is not the same as a thread of execution. Those inner class based threads of execution are not dying.Why don't you use a thread pool?If not, you need to ensure those inner threads have exited and completed.
# 2
> A Thread class is not the same as a thread of
> execution. Those inner class based threads of
> execution are not dying.
maybe. but this is the way i test a thread's life:
public void run() {
System.out.println("thread start");
....
System.out.println("thread dies and release memory");
}
for each inner thread, and the outer thread, this approach for
testing thread life reveals that they die.
> Why don't you use a thread pool?
ok. i will think about how to do this.
>
> If not, you need to ensure those inner threads have
> exited and completed.
what is a 100% sure check to guarantee a thread exits other than
the one i use above?
note:
the outer thread is running on a remote host, and the inner threads
are running locally. here are the details:
public class BB implements Runnable, FinInterface {
...
public void run() {
// do some work on the remote machine
}
private void startResultsHandler(OisXoos oisX) {
ResultHandler rh = new ResultHandler(oisX);
rh.start();
}
public void startDataProxy(OisXoos oisX, String query) {
DataProxy dp = new DataProxy(oisX, query);
dp.start();
}
public class ResultsHandler extends Thread {
// runs locally; waits for results from servers
..
public void run() {
ObjectInputStream ois = new ObjectInputStream(oisX.input);
Set result = (Set) ois.readObject();
....
}
} // _ class :: _ ResultsHandler _ :: class _
public class DataProxy extends Thread {
// runs locally; performs db queries on behalf of servers
..
public void run() {
ObjectOutputStream oos = new ObjectOutputStream(oisX.output);
while(moreData) {
.... // sql queries
oos.writeObject(data);
}
StartResultsHandler(oisX);
}
} // __ class :: _ DataProxy _ :: class __
}
now, the BB class is not started locally.
the inner threads are started locally to both service data requests
by the BB thread as well as wait for its results.
(_) so, maybe the inner threads cannot exit (but they sure look
like they exit) until their parent BB thread exits.
(_) yet, those inner threads have no knowledge that the BB
thread is running.
externalizing those inner thread classes will put 2-weeks of work
in the dust bin. i want to keep them internal.
thanks.
here this piece of code that controls everything:
while(moreData) {
FinObjects finObj = new BB();
String symb = (String) data_ois.readObject();
OisXoos oisX = RSAdmin.getServer();
oisX.xoos.writeObject(finObj);
finObj.startDataProxy(finObj, oisX, symb);
}
# 3
> > while(moreData) {
>FinObjects finObj = new BB();
> String symb = (String) data_ois.readObject();
>OisXoos oisX = RSAdmin.getServer();
> oisX.xoos.writeObject(finObj);
>finObj.startDataProxy(oisX, symb);
> }
>
i just got really confused...
how can i tell when the finObj memory is released?
its when nothing points to that memory location anymore.
so..
whats going on?
what is referencing finObj ?
finObj.startDataProxy(finObj, oisX, symb);
i eliminated this self-reference to
finObj.startDataProxy(oisX, symb);
and it made no difference.
finally, the error i get is really really, really strange.
there is no stack trace.
here is the complete error report:
Exception in thread "Thread-2034" java.lang.OutOfMemoryError: Java heap space
btw: does this mean i have 2034 threads running?
# 4
(a) Is the ResultHandler ever started? If DataProxy has a lot of data to write to the object output stream it will eventually block, and I think that's what is happening. Start the ResultHandler before DataProxy handles any queries.
(b) Why are DataProxy and ResultHandler separate threads? I would combine them. I don't see what is to be gained by doubling the number of threads in the system
ejpa at 2007-7-12 14:49:26 >
# 5
here is the root cause:
it 100% depends on the amount of data i send from client to server.
sending empty HashSets works with no problem.
sending HashSets with 1700 Doule values bails-out after
about 200-300 sends.
as a test, rather than stuff all the Double values into HashSet,
i tried sending them 1 by 1 (as the code below demonstrates).
and the results are strange.
client-side
int recordCount = 0;
ResultSet rs = statement.executeQuery(query);
while(rs.next()) {
float x = rs.getFloat(1);
objectOutputStream.writeObject(new Float(x));
recordCount++;
}
objectOutputStream.writeObject("end of data");
// "recordCount" always = 1700
server-side
try {
int recordCount = 0;
while(true) {
Float data_from_client = (Float) objectIputStream.readObject();
recordCount++;
}
} catch(ClassCastException e) {}
// only about 200-1500 are sent.
i suspect this is an interesting clue.
and, if i put a Thread.sleep() statement
in the client's write loop, all the data is sent.
sok.setReceiveBufferSize(8056);
?
but made no difference.
yet, i don't find a a good buffer size.
btw
i no longer run ResultsHandler as a Thread.
now its a local method to the DataProxy threaded inner class.
