using vectors/array lists
greetings.
i would like to open several text images and look at each entry and place the values into a vector or array list. there are approximately 20 text files, so i created a for loop. i am having two problems.
1. when i initialize the vector/list within the loop everything works relatively well. however, when initialized outside the loop, about 3/4 of the way through the execution, i get a java jeap space OutOfMemoryError. why would this occur only when initialized outside the loop?
2. i would like to add all of the text file values into one LARGE collection. however, as best as i can make out this is not occuring. i have added a print statement to inform me of the size, but the returned value is always the same. since i cannot write a print statement for the collection outside of the loop, i tried initializing an integer prior to the loop and then setting it the size of the collection and performing a similar print statement after the loop is completed. no avail.
any suggestions*?
thank you.
*yes we could devote an entire forum to suggestions you could write, but sticking to helpful suggestions would be greatly appreciated.
[1192 byte] By [
ercricketa] at [2007-10-2 20:21:34]
I don't want to try to guess at what's going on and and what you mean withou seeing code.
When you post code, please use[code] and [/code] tags as described in [url=http://forum.java.sun.com/help.jspa?sec=formatting]Formatting tips[/url] on the message entry page. It makes it much easier to read.
jverda at 2007-7-13 23:04:06 >
Her's a helpful suggestion: Post this message inb a more appropriate forum. It has nothing to do with native methods.
> Her's a helpful suggestion: Post this message inb a> more appropriate forum. It has nothing to do with> native methods.LOLI use category views, and so didn't even notice that.
jverda at 2007-7-13 23:04:06 >
Sure.
import java.io.*;
import java.util.*;
import java.text.*;
import java.lang.*;
import javax.swing.*;//for JOption dialog box
public class BPF_Corpus
{
public static void main (String args[]) throws Exception
{
////////////////////////////////////////////////////////////////////////////////
////////////////////////following code pertains to variables////////////////////
////////////////////////////////////////////////////////////////////////////////
String voxelHeight = JOptionPane.showInputDialog("Enter voxel height");
double numVoxelHeight = Double.parseDouble(voxelHeight);
String voxelDepth = JOptionPane.showInputDialog("Enter voxel depth");
double numVoxelDepth = Double.parseDouble(voxelDepth);
double voxelWidth = numVoxelHeight;
double voxelArea = voxelWidth * numVoxelHeight;
double voxelCube = numVoxelDepth * voxelArea;
double voxel = 0;
double areaIndivi = 0;
double totAreaIndivi = 0;
double volumeIndivi = 0;
double totVolumeIndivi = 0;
double brainAreaIndivi = 0;
double TotalBrainArea;
double brainVolumeIndivi = 0;
double TotalBrainVolume = 0;
double BPF = 0;
double normalizedBPF = 0;
double max = 0;
double min = 200;
double highCutOff;
double lowCutOff;
double BPFnormal = .92;//BPF ratio for normal subject
String input;// image.txt as string
////////////////////////////////////////////////////////////////////////////////
//////////////////////following code pertains to file creation//////////////////
////////////////////////////////////////////////////////////////////////////////
String file = JOptionPane.showInputDialog("Enter execution path");
String filePath = file + "\\";
String files = JOptionPane.showInputDialog("Enter number of files");
int numFiles = Integer.parseInt(files);
String start = JOptionPane.showInputDialog("Enter number of first file");
int numStart = Integer.parseInt(start);
String fileName;
int textSize = 512 * 512;
int count = 0;
int range = 0;
for (int iterator = numStart; iterator < numStart + numFiles; iterator++)
{
Vector corpusVector = new Vector();
List list = new ArrayList ();
FileReader inputImage = new FileReader(filePath + iterator + ".txt");//opens txt file
BufferedReader binputImage = new BufferedReader(inputImage); //reads lines of text
System.out.println(iterator + ".txt" + "\t" + "the count is " + iterator);
while ((input = binputImage.readLine()) != null)
{
StringTokenizer numbers = new StringTokenizer(input);
while (numbers.hasMoreTokens())
{
count++;
String voxelValues = numbers.nextToken(); //reads #s 1by1, line by line
voxel = Double.valueOf(voxelValues).doubleValue();//converts string to numerical (integer) value of pixel
corpusVector.add(voxel);
list.add(voxel);
if (voxel > max)
max = voxel;
if (voxel != -1 && voxel < min)
min = voxel;
}
}
}
}
}
if i place the line Vector corpusVector = new Vector();
before the for loop, i get the error after the 12th iteration.
thanks, let me change topics.
> if i place the line Vector corpusVector = new
> Vector();
> before the for loop, i get the error after the 12th
> iteration.
Oh, sure.
In one case, you're creating one vector, and adding everything to it.
In the other case, you're creating a new vector each iteration and only adding that iteration's stuff to it. As soon as you end an iteration, the vector you created during that iteration is eligible for GC.
If you don't need a single vector that holds all iterations' data and lives beyond the loop, then just put the vector inside the loop.
If you do need that, then you'll need to adjust the memory available to the VM with -Xmx or make smaller objects, or both.
Why are you using both Vector and ArrayList? Unless you have to work with legacy code, ArrayList is preferred.
jverda at 2007-7-13 23:04:06 >
just trying to see what is working and what is not. i comment out one and work with the other. etc...i am going to read your response again and make sure i understand everything.
>
> Why are you using both Vector and ArrayList? Unless
> you have to work with legacy code, ArrayList is
> preferred.
thanks jverd. you were pretty clear. i do need to store everything so it looks as though i need to alter the VM. if you have the time and inclination you can tell me about the VM, but either way, i am going to do some googling and go from there.
also, of the things i looked up on this forum, i am going to follow your advice and stick with the arrayLists.
Thanks loads.
> > if i place the line Vector corpusVector =
> new
> > Vector();
> > before the for loop, i get the error after the
> 12th
> > iteration.
>
> Oh, sure.
>
> In one case, you're creating one vector, and adding
> everything to it.
>
> In the other case, you're creating a new vector each
> iteration and only adding that iteration's stuff to
> it. As soon as you end an iteration, the vector you
> created during that iteration is eligible for GC.
>
> If you don't need a single vector that holds all
> iterations' data and lives beyond the loop, then just
> put the vector inside the loop.
>
> If you do need that, then you'll need to adjust the
> memory available to the VM with -Xmx or make smaller
> objects, or both.
>
> Why are you using both Vector and ArrayList? Unless
> you have to work with legacy code, ArrayList is
> preferred.
i altered the virtual memory from a max of 2678 to 5000, 10000 and then finally to 30,000MB and i am still getting the same error. that seems like a $_*!load of memory. Did i miss something?
I don't know what you're trying to do, and I haven't looked that closely at your code, so I can't offer any specific advice, just a couple of general points.
Think about what you're trying to do, and if you really need all of that data in memory at once to do it. You might consider streaming it in and processing it as it comes by, or breaking it into chunks, and then aggregating the results.
You might want to use a database, or store intermediate results in a file.
You might also look into the Flyweight pattern.
http://www.google.com/search?q=flyweight+pattern
http://www.amazon.com/gp/product/0201633612/102-3021191-3155354?v=glance&n=283155
jverda at 2007-7-13 23:04:06 >
On this line:
voxel = Double.valueOf(voxelValues).doubleValue();
you create a Double object, then get the double primitive out of it.
Immediately following that, you do this:
corpusVector.add(voxel);
list.add(voxel);
Since only objects can be added to Vectors/ArrayLists, the voxel gets autoboxed (twice) back into Double. I don't know if that is affecting your memory use, but it will certainly (and negatively!) impact your speed.
If you want to get the primitive value from the String, you could have done:
voxel = Double.parseDouble(voxelValues);
But, since you want both the Double object (to put in the lists) and the primitive value (for comparing), you might want to do this:
Double voxelObject = Double.valueOf(voxelValues);
voxel = voxelObject.doubleValue();
corpusVector.add(voxelObject);
list.add(voxelObject);
That avoids the performance impact of the autoboxing.
Are your image height/width really doubles? Or, should they be int?
MLRona at 2007-7-13 23:04:06 >
