i am doing some netwrok statistics using java. i have large data set 1.5Millions records from the proxy server.
I wrote some function just put all of the data into an linked list, but i use 300MB ram, can anyone tell me how can i read all of the data but use less ram?
hte proxy log class:
public class proxyLog
{
String machineName;
long timeS;
long timeMS;
String url;
int size;
int latency;
}
THX
> Do you really need to process all the data at a
> time?
>
> There is hardly any situation where you would require
> all the records at a time. Maybe, you should rethink
> your approach to the problem.
Hey, do you know Robin Hood? He is a friend of mine!
> There is hardly any situation where you would require
> all the records at a time. Maybe, you should rethink
I don't think that's true. I'm thinking about wanting to aggregate, find the largest or smallest element, etc.
But in any case, the best answer IMHO is to jam all of your data into a database and handle it through SQL. Databases do a much better job than you could do.
> > There is hardly any situation where you would
> require
> > all the records at a time. Maybe, you should
> rethink
>
> I don't think that's true. I'm thinking about wanting
> to aggregate, find the largest or smallest element,
> etc.
With a linked list of 5 million records?
I never claim that my replies are correct anyway. It's only a nice wild guess.
> But in any case, the best answer IMHO is to jam all
> of your data into a database and handle it through
> SQL. Databases do a much better job than you could do.
Better with a new set of troubles. :-)
