Is Vector() limited in size() ?

Hi !

I am actually working on a neuronal system bullshit, and I need to create something like at least 1 million "particles".

So I can generate as many millions particles as I want, but when it comes to putting it in Vector() ... it always stops at something like 104999 particles.

the error is :

"Exception in Thread "main" java.lang.OutofMemoryError

<< no stack trace available >>

BTW : my config(Linux Mandrake 7.2, PIII 1000, 380 RAM) I hope it is sufficient to handle so many particles ... (it is enough to run 3DSMAX on M$ winME)

I first thought that vector were limited in size, because it always stopped at 104999, but afterwards, seeing this out of memory exception, I though I asked too many things for the comp.

What I have tried to do :

1) trimToSize() after all the molecules are created (in series of 10 000 or so)

2) reorganize the vectors so that less need to be added / removed / modified.

3) setting different values for (initial capacity, increment), from (5,1) to (50000,10000) !

What I think could save me :

1) set up a buffer that I could flush after [x] molecules are created (*** preferred ***)

2) store all parameters to a single vector, containing vectors (this would need strong logic to get values from there ...)

If you have any hints, ideas, codes, please post them here !

Regards,

stephmor

PS : yes, I need so many particles ! ;)

[1494 byte] By [stephmor] at [2007-9-26 2:16:08]
# 1

Yes, everything is limited in size because you don't have infinite memory to work in. If you have too much data to fit in memory you will have to store in outside the program (in disk files) and move it in and out as necessary, just as computer programmers have been doing since at least the 1950s.

DrClap at 2007-6-29 9:14:11 > top of Java-index,Archived Forums,Java Programming...
# 2

Hi,

What I wanted to avoid was to write to disk , because the data is gonna change quite often, and randomly.

I don't want a huge pile of dead data, I would prefer "live" data, so the disk solution sounds a bit limited in this way, whereas this data in variables (Vectors) sounds easier to retrieve and analyse, compare, update, and so on and so forth...

Another problem is that the read/write process could increase the processing time . As I will have to handle small particules, but millions of them, if the process increases the whole thing by, say 0.25 seconds, the whole process would take 1000000*0.25 seconds more !

stephmor

PS : barrel organs have been using input files (cards with holes in them) since at least the 1780s.

stephmor at 2007-6-29 9:14:11 > top of Java-index,Archived Forums,Java Programming...
# 3

Although not to familiar with the variuos JVM parameters I do no that you can set some parameters with java command options. You can increase the total amount of memory available to jvm this way.... Theres quite a few options you can use to do this, and well Im not too familiar with any of them!!!

check the tooldocs, info is there about doing this....

In any case though you should try to set up some sort of limits, and perform some sort of disk caching...

J.Prisco at 2007-6-29 9:14:11 > top of Java-index,Archived Forums,Java Programming...