OutOfMemoryError

friends,

I am retrieving records (500,000 and more) from, one or more, databases using two SQL queries. I compare each record and if it I record the comparison result.

Now I am having following problems

1.How do I write to a file so many records and save myself from getting the above error.

2.How do retrieve Rows in batches or remove few rows from result set once there task is finished.

3.How do I catch OutOfMemoryError without terminating or affecting the memory. is there any way to call garbage collection from my program itself. I have tried using more memory for virtual MC but i think its an temporary solution.

[672 byte] By [nirmitdesai07] at [2007-9-30 4:10:42]
# 1

500,000 records or more is kinda big for an in-memory comparison. Database management systems are more suited to handle that kind of operation.

If you are retrieving the records from a single data source, assuming you are using a dbms and not a flat file system, let the dbms do the sorting/selection/filtering.

If you are however retrieving from multiple data sources, and in the absence of a db gateway product, your application should have "some" query optimizing ability for it to know how to tackle the retrieval of the records.

EJPT at 2007-6-29 17:55:07 > top of Java-index,Administration Tools,Sun Connection...