java.lang.outOfMemory
Hello
Our application is trying to read a csv (in bytes) from a report generation software through SDK. When the file size is around 50 Megs..we can read the csv fine. but if the file size is around say 100 Megs...our application throws a java.lang.outOfMemory error while trying to read the data from the report generating software (cognos reportNet). Our heap size is set to 0 (initial) - 3072 (max)
...we tried increasing it to more than 3 gigs..but our server crashes then.
Any suggestions on how we might be able to solve this problem?
Thanks!
# 1
You could stop trying to keep it all in memory.
You could also try to figure out why the in memory data structures increase the size of the data by at least 10.
You could put the data in a database rather than try to do whatever it is that you are doing now and let the database provide all of the functionality that you need.
# 2
Hi,
Since your data is kept in a CSV format indicates to me that you probably like to process the data in rows and in batches and hence its not efficient to read in ALL the data at once.
What i would like to suggest is that you use a BufferedReader and read in the data 1 line at a time or batch-read a couple of lines at a time; i believe you can achieve a far higher efficiency
