Diagnosing a "hang" in an Iterator
I have code that takes a list of ID numbers in an ArrayList The size of he ArrayList is 126. IN my code, I create an iterator on the list. For each id, I do a fetch from the database. After I get to 28 ids,the program simply stops. No error, no Exception. It just stops . I wanted to run JConsole but the box is not configured for that (or any X11-type stuff). I want to know what's goig on, like lowmemory. I have guessed it was memory and addd a <maxmemory="512m"/> to the java task of my ant script, but that seemed to amke no difference. How can I diagnose what's wrong? I use iterators all th time and never hit a situation like this. Here's the relevant code snippet:
Iterator idIter = studentIDs.iterator();
while(idIter.hasNext())
{
studentID = (String)idIter.next();
edu.apu.directory.APUPerson student = getStudent(studentID);
writePersonHeader(student);
counter++;
System.out.println("counter:"+counter);
}
studentIDs.size() just before this loop equals 126. When I run this, coutner never gets past 28. NO error. It just stops. Any suggestions? Thanks.
[1140 byte] By [
kdlitwaka] at [2007-10-1 21:26:44]
This code isn't complete. The hang isn't likely to be in the iterator. It's in the getStudent() method, I'll bet. Could an exception be thrown and you haven't really handled it? Look for an empty catch block somewhere that's swallowing an exception without telling you.
I'd move that database code into a separate object and test it out separate from the rest of your code. It's called a DAO object. It ought to have a signature like this:
public interface StudentDAO
{
public Student findById(String studentId);
public List findAll();
public void saveOrUpdate(Student student);
public void delete(Student student);
}
Put your implementation in there and JUnit the snot out of it.
(I don't know what your edu.apu.directory.APUPerson class is. Perhaps that is your "Student".)
If I were running this app, I'd get into a debugger and step through the code until I figured out exactly where the error was occurring. Shouldn't be too hard to do.
%
Here's another performance question: Why do you do a network roundtrip for every student ID when you could do a single query for those student IDs in one network roundtrip, cache them in memory, and then iterate through those?
What you're doing is a classic noob error that kills performance. Minimizing network roundtrips is a good thing to do. It's always slower than an in-memory call. You're trading time for space, and memory is cheap and plentiful.
%
This code isn't complete. The hang isn't likely to be in the iterator. It's in the getStudent() method, I'll bet. Could an exception be thrown and you haven't really handled it? Look for an empty catch block somewhere that's swallowing an exception without telling you.
Thanks but that's not the case. The getStudent method uses a DAO that has been running in production for months and works just fine. There are no empty catch blocks:
private edu.apu.directory.APUPerson getStudent(String studentID)
{
try
{
student = (edu.apu.directory.APUPerson)personDAO.findAPUPerson(studentID);
}
catch (PersonNotFoundException e)
{
System.out.println("error getting student:"+ e.getMessage());
e.printStackTrace(); //To change body of catch statement use File | Settings | File Templates.
}
catch (LookupFailedException e)
{
System.out.println("error getting student:"+ e.getMessage());
e.printStackTrace(); //To change body of catch statement use File | Settings | File Templates.
}
return student;
}
What is of great interest is that if I copy the call to getStudent() out of the Iterator loop and hardcode an id in it, then the number of elements in the ArrayList that the Iterator woudl process went down by one, as though the application simply will not process over 28 calls to the DAO. I don't get any exceptions, and I never code empty catch blocks (bad programming practice, which can cause trouble, as you note).
I'd move that database code into a separate object and test it out separate from the rest of your code. It's called a DAO object. It ought to have a signature like this:
Good idea. Done long ago.
(I don't know what your edu.apu.directory.APUPerson class is. Perhaps that is your "Student".) Yes, it models a student.
If I were running this app, I'd get into a debugger and step through the code until I figured out exactly where the error was occurring. Shouldn't be too hard to do.
I"ll see if I can get that to work, but I'm dubious of it working 28 times and tehen telling me what's wrong the 29th, if it's not a Java error. Thanks.
> Thanks but that's not the case. The getStudent
> method uses a DAO that has been running in production
> for months and works just fine. There are no empty
> catch blocks:
That's better. How are you deploying and running this? You print to System.out. Can you see the console? I'd say it'd be better to write to a log.
> What is of great interest is that if I copy the call
> to getStudent() out of the Iterator loop and hardcode
> an id in it, then the number of elements in the
> ArrayList that the Iterator woudl process went down
> by one, as though the application simply will not
> process over 28 calls to the DAO. I don't get any
> exceptions, and I never code empty catch blocks (bad
> programming practice, which can cause trouble, as you
> note).
You've moved all this into JUnit tests that demonstrate and document the behavior, of course.
> Good idea. Done long ago.
Excellent.
> (I don't know what your edu.apu.directory.APUPerson
> class is. Perhaps that is your "Student".) Yes, it
> models a student.
Very good.
> I"ll see if I can get that to work, but I'm dubious
> of it working 28 times and tehen telling me what's
> wrong the 29th, if it's not a Java error. Thanks.
If by "Java error" you mean a bug in the JVM, I'd put that aside if I were you. I'd bet the farm that the problem is in your code. You'll make more progress if you assume that the problem lies with what's under your control.
Try to set up a JUnit test that demonstrates the behavior. If it's not repeatable, then you've got a real problem.Is the code multi-threaded? Any possibility of deadlocks or race conditions causing this?
%
> I"ll see if I can get that to work, but I'm dubious
> of it working 28 times and tehen telling me what's
> wrong the 29th, if it's not a Java error. Thanks.
If by "Java error" you mean a bug in the JVM, I'd put that aside if I were you. I'd bet the farm that the problem is in your code. You'll make more progress if you assume that the problem lies with what's under your control.
No, I don't mean a JVM error. What I'm saying is this: I build an ArrayList of ids from one database (on Postgres), and then, one at a time, use a DAO that has been working in production, in both tests and when accessed form a sevlet, over and over again, to get a row from an Informix table. My concern with debugging is that, if I'm not getting an exception or the like, I'm not sure what I'll see in a debugger on the 28th iteration of calling the DAO. I don't expect the JVM has a problem. I am running on Linux Red Hat, and running the class from the command line. It's going to be a report on what a web app I've written is doing each day, so the command line is fine for right now. I'm wondering if my JVM is running out of memory. I tied to tell ant to up the maxmemory, but I don't know that this is making any difference in what ant does. Thanks.
Ken
Try to set up a JUnit test that demonstrates the behavior. If it's not repeatable, then you've got a real problem. Is the code multi-threaded? Any possibility of deadlocks or race conditions causing this?
> No, I don't mean a JVM error. What I'm saying is
> this: I build an ArrayList of ids from one database
> (on Postgres), and then, one at a time, use a DAO
> that has been working in production, in both tests
> and when accessed form a sevlet, over and over again,
> to get a row from an Informix table. My concern with
> debugging is that, if I'm not getting an exception or
> the like, I'm not sure what I'll see in a debugger on
> the 28th iteration of calling the DAO.
I guess that's the $1M question, isn't it?
If you have IntelliJ, you can set it up so you can run Tomcat inside it. Maybe you can deploy your servlet in that and use IntelliJ's debugger to see what's going on. A breakpoint at the right spot will help.
> I don't
> expect the JVM has a problem. I am running on Linux
> Red Hat, and running the class from the command line.
> It's going to be a report on what a web app I've
> e written is doing each day, so the command line is
> fine for right now. I'm wondering if my JVM is
> running out of memory.
Why so? If that were the case, wouldn't you see an OutOfMemoryError?
What about the machine where the servlet is deployed? What do the logs for the servlet engine say? Nothing?
What about P6Spy?
http://www.p6spy.com/
That might help you decipher what's going on with the DAO that's accessing Informix.
%
Hi,
Just a quick suggestion, as I've had behaviour like this before. I notice you have a getStudent(studentId) method, which I assumes drag back a student from the database. In this method are you writing your own sql strings and opening db connections and the like? If so, and you're using connection pooling, then it could be you've run out of connections and the program is hanging waiting for one to free up. Make sure all your connections are properly closed.
If not that then I doubt it's a jvm / memory issue - just sysout the hell out of it and look for a block - obviously either in the getStudent or writePersonHeader methods.
wishing you luck....
Ditto: I'd put my bet on the theory that you're failing to close or release somethingin your db access.
Another angle: Is the problem that it dies after 28 accesses, or is there something about 29 that kills it? Try using different student ids or re-arranging the order of the ids and see if it fails on #29 no matter what the value of #29 is, or if it always fails on the same value.
I sincerely doubt that the problem is with the iterator itself. If you're really paranoid about that angle, it's easy enough to prove: Just comment out the getStudent function and replace it with a simple print of the id. If the program still hangs when there is no database access, but all you're doing is looping through whatever the collection is printing values, then okay, maybe I'd start to look at problems with the loop or the collection. But if, as I strongly suspects, it succeeds in printing out more than 29 IDs, then you know that the problem is inside the getStudent function and not in the loop.
