PROBLEM : Jakarta POI - writing large Excel files
I have a program that extracts a lot of data from about 1500 txt files. I want to save this data to an excel file.
Everything works fine, but when the amount of data gets too large, I get an java.lang.outOfMemoryError! It works with 200 files.
The problem is, everything is in the memory until all data has been collected, THEN it's written to a file.
Is there a method to partially write a excel file with POI, so I can handle larger amount of data or something like that? starting the program with -xmx or -xms won't help.
Thanks for any help!!
[581 byte] By [
sockena] at [2007-10-2 21:08:13]
Hi, Socken, I am not good at POI.. but First, why do you want to use Excel instead of other DB, such as mysql?Second, can you first save those files into a .csv file, then easily import it into Excel?
Hi! Thanks for your help.
I'm creating a small tool for someone. I have a lot of txt files from which I need to extract certain data. I can't use a DB, has to go into a file. Would be great to save it directly into an Excel file so it doesn't need any further converting.
But if it's not possible, I guess I have to stick the CSV option, that was my first plan anyway!
I know it's a known issue of POI that you can't save large files. Read that on the official site already.
2nd though: Since the speed seems do not matter much for your small utility, can you make the .csv file first, then use POI API to convert it to excel file autmatically? I am not sure if it will solve your problem, but it may work.
I thought of that as well! Might solve my problem yes! But will this work?
I had to read in the CSV file again, and save it to the Excel file with POI. But then again, I had to read everything into memory and save it with POI. So the data size would be exactly the same size, so: Same problem.. Isn't it!?
