Hardcoding an extremely large ArrayList?
Hi everybody,
I've come up with a weird idea, and I have no clue if it's even remotely possible or rational, but here goes--
I have an extremely large arraylist, which is...let's see...an arraylist of arraylist of arraylist of arraylist. (whew!) Basically, it's 4 levels of an arraylist<String> nested, representing formatted sections of a paragraph, inside paragraphs, inside pages, inside chapters. I'd like to be able to hard-code this somehow into my program, so the user doesn't have to input the raw text file all the time. Basically, I'm looking to create a completely self-contained program, something like an ebook, which will eventually be wrapped in a .exe wrapper.
The only way I can think of to hard-code this big arraylist is to write every single call and insertion into each arraylist, which would take an extremely long time and be a real pain. I imagine that creating the arraylist when the program runs would be easier, like with a BufferedReader, but I don't know if there's a way to include a text file in a jar or class so it's self-contained and doesn't require any user input. Can anyone give me a hand with how, conceptually, I might be able to do this?
Thanks,
Jezzica85
[1251 byte] By [
jezzica85a] at [2007-11-27 8:33:43]
Hardcoding the ArrayLists is not a good idea for the reasons you mentioned, plus the memory that it takes up.You are better off including the text file in the JAR (yes, you can do that) and then reading from it as needed.
RATiXa at 2007-7-12 20:29:49 >
@Op. Yes you can include a textfile in a jar.
kajbja at 2007-7-12 20:29:49 >
Thanks RATIX, and kajbj, good to know...so, how do you include the text file in the JAR? Is there a special way you have to type the filepath then? I have Eclipse 3.2.Thanks,Jezzica85Message was edited by: jezzica85
Really storing an initialization of really big data entities as code is very innefficient. From the sound of things your best bet is to have the data as an XML file and have the program load it from that. Or you might consider a lightweight database (e..g. H2).
If you chose the XML option you can include the XML file in the program's jar. Treat is as a resource, using Class.getResourceAsStream to open it.
... you can also use arrays instead of lists if you don't need to append new data after you have created the structure.
kajbja at 2007-7-12 20:29:49 >
> I have an extremely large arraylist, which is...let's
> see...an arraylist of arraylist of arraylist of
> arraylist. (whew!) Basically, it's 4 levels of an
> arraylist<String> nested, representing formatted
> sections of a paragraph, inside paragraphs, inside
> pages, inside chapters. I'd like to be able to
> hard-code this somehow into my program, so the user
> doesn't have to input the raw text file all the time.
XML maybe the better way to go
there were no replies when I started mine....
Message was edited by:
snic.snac
> Thanks RATIX, and kajbj, good to know...so, how do
> you include the text file in the JAR? Is there a
> special way you have to type the filepath then? I
> have Eclipse 3.2.
You open the file as a resource, so you don't need to use a File object.
Kaj
kajbja at 2007-7-12 20:29:49 >
Thanks for the idea malcolm, can you give me an example of what you mean? I've never heard of that Class method before. Does that mean I can still read it in a FileReader?Thanks,Jezzica85Message was edited by: jezzica85
I can sum up how to solve your problem in three letters.
X M L
If you store your text in xml you can do all manner of interesting formatting things to it and bundle and group sections and paragraphs and what have you in all sorts of interesting ways.
An even better idea would be to teach your application to read and write Open Document Format text which would make your files portable to other applications.
Just my 2 krupplenicks on the subject your milage may of course vary.
PS.
http://java.sun.com/javase/6/docs/api/java/lang/ClassLoader.html#getSystemResourceAsStream(java.lang.String)
kajbja at 2007-7-12 20:29:49 >
I've had the worst time with XML; I can't understand the javadocs. I guess I'm kind of dense in that way but I've tried multiple times and I can't seem to make heads or tails of it. That's why I'm using a text file instead.Jezzica85
Now that's a discreet problem that we can work on. Can you formulate a question about XML that's giving you a particular problem?PS.
Thanks kajbj; can you use relative paths with that method? I imagine that whatever computer the final file for this is on, the path to the text file in the jar would be different.Or am I misunderstanding, and do you not use that method from inside the
Resources are fixed files living alongside the program's .class files. Most IDEs will automatically copy such files into the jar if you put them in the program source directory.
The simplest way is to put the resource right next to the class that reads them, in the same package directory. Then the class can do e.g.
InputStream is = getClass().getResourceAsStream("data.xml");
You should also use this technique to include image files, sound clips etc. It uses the same mechanism to read the resource that the ClassLoader uses to read class files, even if they are in a jar or on line, or both.
If you are going to be searching by chapter, page number, contents etc, then you might want to look at apache lucene. You add documents to a directory and can index keywords to provide ranked searching, wildcard queries, sorting by any field etc. http://lucene.apache.org/java/docs/
YoGeea at 2007-7-21 22:42:35 >
Hi Puck,
Do you really think it would matter if I used XML versus a normal text file? This data structure I've been mentioning is going to be built to matter what format I read in, unless of course somebody gives me an idea for a better one (the structure I have now is the best I could come up with).
Thanks,
Jezzica85
Thanks malcolm, that's really neat. So, I can put the text file in the package? I didn't know you could do that; I thought you could only put class files in a package.Thanks!Jezzica85
XML is just easy to traverse using SAX or DOM, in this case I would probably use DOM.
RATiXa at 2007-7-21 22:42:35 >
late reply:you cant use your original idea because class files have a limit of 64kb, which is probably not enough for your ebook.
Thanks mkoryak, I didn't know that either. So, the resource thing is definitely the way to go. I can still populate the big arraylist without going over the class size limit, right?Thanks again,Jezzica85
right. if you start loading really big stuff into memory you might need to increase java heap size for your program by using this JVM arg:-Xmx256mthis sets your memory to 256 megabytes, which should be enough for anything. i doubt you will have problems either
> Hi Puck,
> Do you really think it would matter if I used XML
> versus a normal text file? This data structure I've
> been mentioning is going to be built to matter what
> format I read in, unless of course somebody gives me
> an idea for a better one (the structure I have now is
> the best I could come up with).
>
> Thanks,
> Jezzica85
It might depend on how much formatting you intend to do and how extensive your documents are. From the sound of it they sound to be fairly substantial and that alone indicates to me that it would be better to use XML. Though if it were me. I would explore the possibilities of using the Open Document format. It's XML based and an emerging standard for document interoperability. Most word processors will read and write it as well. So you could format the document create whatever sections etc. that you needed and then later read them in your favorite word processor.
http://www.google.com/search?hl=en&q=Open+Document
But that's what I would do. If you've got it working with a text file format go with it.
PS.
