hi,
I have to generate millions of records(say of format name|age|ssn|income) and write into a file. Because writing single record everytime will slow down the application, i use a vector. Currently i use some value (1000) , after which i write the vector contents to the file and empty the vector. Is there any way to use the vector capacity to its fullest without facing problems of exception.
What you think is right. Doing insert on database everytime will take time. But how are you moving the content of Vector to database, again within the loop you will have 1000 inserts which is going to take same amount of time.
I suggest two things:
1. Almost all databases support BatchUpdate. Start the batch, insert the records and update the batch
2. If you are taking about having a transaction (start, insert, commit) for each record is taking time, go ahead and start the transaction before you start the work, Commit it after 1000 records and start a new transaction.
In both the cases you don't need to use vector.
Regards,
Venkat
> What you think is right. Doing insert on database
> everytime will take time. But how are you moving the
> content of Vector to database, again within the loop
> you will have 1000 inserts which is going to take
> same amount of time.
>
> I suggest two things:
> 1. Almost all databases support BatchUpdate. Start
> the batch, insert the records and update the batch
> 2. If you are taking about having a transaction
> (start, insert, commit) for each record is taking
> time, go ahead and start the transaction before you
> start the work, Commit it after 1000 records and
> start a new transaction.
>
> In both the cases you don't need to use vector.
>
> Regards,
> Venkat
Is it really efficient to handle a file like a database?
yea, its a file not database. Frm that file as input database will be populated(My job is to generate file). And thanks for suggestion of BufferedWriter. I am using it now.
But still wondering is there anyway to use Vector to its limit, means, wat i did was use a vector and use it as a buffer to write records. I did this
if(v.size()>=1000){
writeToFile(v);
v.clear();
}
I wanted to use(as i was unaware of BufferedWriter) some function instead of '1000' that will let me exploit memory.
Is there any such function?
