Large Database/Multidimensional Array
I was hoping to get input on how to approach the storage of a large multidimensional array. I am doing independent study and statistical analysis on various topics and think I might need a separate database to hold the data that can be accessed through a JAVA driven GUI.
Although the sizes vary, the structure would be nearly the same. For example, there might be 10 main parent elements. Each of those 10 elements would have 10 of thier own sibling subelements. From there, it would continue through maybe 50 iterations ending with 10^50 leaves alone. In the most extreme cases, there might be a total of 10^54 elements in this tree structure. The values of these elements would be static. What possible solutions are out there for maintaining something of this size?
Using this example, the end goal is to be able to start at one of the first 10 parent elements and traverse down to one of the 10 elements under it until the end is reached. This would be done through a GUI where user input would determine which of the 10 elements to go to, then return what is contained there.
Any help on how, why and where to hold that much data would be greatly appreciated.
[1190 byte] By [
Peelmana] at [2007-11-26 14:52:00]
I think prayer would be a good start. I don't think that 10^54 isa realistic number at all, unless you're the NSA.
I've tried prayer unsuccessfully so far. I'm holding on to hope, though.
How big are each of these elements? How big of a HD can youbuy? You're really talking about quite a lot of data .. more thanthe US Library of Congress... http://www.loc.gov/webcapture/faq.html
Actually 10^54 is roughly 10,000 times larger than the number of atoms that compose the planet Earth. So one wonders where you would get the material to build the RAM chips or hard drives that store this information.
Shouldn't be any problem with the size, you'll just have to wait a bit until atom-based memories are developed. Since the universe is estimated to contain around 6x10^79 atoms, using up around 10^55 (8 bits per byte, remember) atoms still leaves 6x10^24 for the rest of us to use. But the universe will be a bit depopulated. . .
Message was edited by:
ChuckBing
Not sure how much space I could realistically afford. I am determined, but also a college student. The elements could be primitive data types, maybe as small as a char.
Lets continue with my ridiculous error in judgement.So ... to increase scalability, and avoid using costly SCSI drives, what if I used a number of the 150GB WD Raptor drives in a striped array ... how many of those would I need?
You're not paying enough attention. You need more physical materialthan makes up the entire earth to build an adequately large drive.Unless you have some grand cosmic powers of which we areunaware, you are SOL.
> Lets continue with my ridiculous error in judgement.
>
> So ... to increase scalability, and avoid using costly SCSI drives, what
> if I used a number of the 150GB WD Raptor drives in a striped array ...
> how many of those would I need?
Only 10^54/1.5*10^11 == 0.666*10^43 of those drives. If there were ten
billion people on this earth you could give 0.666*10^33 of those disk
drives to each of us, that includes babies, my neighbour who doesn't
even know what a disk drive is and the flower man from whom I buy
flowers every week. You can have their drives.
I think you should get your magnitudes right, i.e. your 'problem' doesn't
make sense.
Can you elaborate a bit more on the actual problem you're facing? Except
for those humonguous numbers which are too large to impress anyone at all.
kind regards,
Jos
> > Lets continue with my ridiculous error in
> judgement.
> >
> > So ... to increase scalability, and avoid using
> costly SCSI drives, what
> > if I used a number of the 150GB WD Raptor drives in
> a striped array ...
> > how many of those would I need?
>
> Only 10^54/1.5*10^11 == 0.666*10^43 of those drives.
> If there were ten
> illion people on this earth you could give
> 0.666*10^33 of those disk
> drives to each of us, that includes babies, my
> neighbour who doesn't
> even know what a disk drive is and the flower man
> from whom I buy
> flowers every week. You can have their drives.
Ok, so you are saying is there is a chance?
(Sarcasm aside, my main problem was knowing if and how this was possible. Now that I know the definite answer to the "if" being no, I will think of something else. I seriously appreciate everyones quick response. Now I will be able to go to bed and catch up on the lack of much needed sleep that likely contributed to my fuzzy math.)
> Ok, so you are saying is there is a chance?Mathematically speaking yes (you're just talking about 10^54 thingies),physically speaking: no, as others have explained before.kind regards,Jos^100^100
What did you have for lunch today?are you serious by asking this question or just joking? :)
