I need some way to take an URL specified by the user and to download the source of the page (i.e. what you get when you go View->Source in a browser). The idea is to search through this for specific tags and then pull information from it.
When you download a webpage thats exactly what you are getting -
the source. The web browser is the thing constructing the page.
You can grab the URL bytes and put them in a String.
i just happend to have done this:
u = new URL(requestUrl);
// Read all the text returned by the server
BufferedReader in = new BufferedReader(new InputStreamReader(u.openStream()));
String str = "";
//you add the while loop around this stuff yourself:
str = in.readLine();
in.close();
Heres sample code from another thread
URL url = new URL(...);
URLConnection conn = url.openConnection();
conn.connect();
BufferedInputStream in = new BufferedInputStream(conn.getInputStream());
while ((c = in.read()) != -1) {
// write bytes somewhere
}
EDIT: mkoryak beat me to it
Quick followup question; What can I use to loop to the end of a BufferedReader (I'm used to using Scanner for file input).
depends if you use readLine() do
while(inputLine != null)
but if you use read() do
(while inputLine != -1)
of course you have to add whatever your reading to a string like text+= a.readLine() or a.read() (a being what you declared as your buffered reader)
Message was edited by:
mark07
String line;
while ((line = br.readLine()) != null) {
... process line...
}
Yet another followup question, this one less related.
Is there are way to look for a string that includes the " character in it?
> Is there are way to look for a string that includes
> the " character in it?
I think what you're "really" asking is how to escape a double quote in a string.:
String s = "a \"self-referential\" example";
Thanks for all the help so far! I'm almost done, I just need one more thing...
I want to take the path of an image, and save it to a specified folder. Possible?
> I want to take the path of an image, and save it to a
> specified folder. Possible?
? Could you give an example?
I have an urll " http:// ... .jpg" that I want to save to a folder through java.
> I have an urll "http:// ... .jpg" that I want to save to a folder through java.
It's similar to how you grabbed the contents of the web page URL - but this time you use an input stream, not a reader because you are reading bytes, not characters.
Open a stream using the image URL.
Read the bytes
And write them to a file of your choice
While using this example, what is the "c" variable? Do I make it type byte?
*edit* Apparently I need a boolean. Could somebody show me how to go about writing this?
> Heres sample code from another thread
>
> > URL url = new URL(...);
> URLConnection conn = url.openConnection();
> conn.connect();
> BufferedInputStream in = new
> BufferedInputStream(conn.getInputStream());
> while ((c = in.read()) != -1) {
> // write bytes somewhere
> }
>
>
>
> EDIT: mkoryak beat me to it
Message was edited by:
thefil
My code so far:
String imgName = imagePath.substring(imagePath.lastIndexOf("/") +1);
URL img = new URL(imagePath);
//saving the image
URLConnection conn = img.openConnection();
conn.connect();
BufferedInputStream in = new BufferedInputStream(conn.getInputStream());
//creating output image
FileOutputStream out = null;
PrintStream file = null;
try{
out = new FileOutputStream(imgName);
//file = new PrintStream(out);
boolean c;
while(c=in.read() != -1){
}
}catch(Exception e){
System.out.print("Output failed.");
}
out.close();
in.close();
> boolean c;
>while(c=in.read() != -1) {
>
>}
What happens when you compile this?
in.read() returns an int not a boolean. It is the whole expression "(c = in.read()) != -1" that is boolean.
[Edit] The parentheses are important. You way you have it written c will be a boolean with the value "true" (ie in.read() is not -1) until the end of the file. You should assign an int value to c first so you can later write this value to the file, then compare the value to -1 to obtain a boolean for the while loop.
When I compile it with c as a boolean, it works fine. If I make c an int, I get an error "incompatible types"
*edit* I put brackets around the c=in.read() bit and no more error. Now, excuse my ignorance, but how should I write this?
Message was edited by:
thefil
> If I make c an int, I get an error "incompatible types"
This crossed with my edit. Use the parentheses like the code in TuringPest's reply 3.
I changed the brackets, and it worked. Saved image and everything. But once again I have a new question: How do I write to a subdirectory instead of root? I tried adding "\folder\ "+ imageName with no luck
> I changed the brackets, and it worked. Saved image and everything.
Great.
> How do I write to a subdirectory instead of root?
If you check the API documentation for FileOutputStream you will see its constructor can be a String filename, or a File object. It might be easier to use the File version. You already have the image name (imgName), and you can use this to construct the outputFile. Something like:File outputDir = new File("c:/temp/myImages/");
// ...
String imgName = "foo.jpg"; // or whatever you get with substring()
// A File with the right name, in the output directory
File outputFile = new File(outputDir, imgName);
try {
out = new FileOutputStream(outputFile);
// etc...
}
One last, final, penultimate question that minor and bears no NECCESARY revelance to the project...
In Visual Basic, there is a function called App.Path that returns the directory the application is stored in. This is great for allowing access to subdirectories no matter where the software is stored. I'm about ready to do a final compilation of my project; I just need to know a sort of equivalent in Java. Thanks.
> One last, final, penultimate question
Which? ;)
> In Visual Basic, there is a function called App.Path that returns the directory the
> application is stored in. This is great for allowing access to subdirectories no matter
> where the software is stored.
I don't know Visual Basic, but in Java the nearest we get to an "Application" is a class with some well known entry point (like a main() method). Consequently there needs to be some adjustment in how we consider "where the software is stored".
Most likely the class whose main() method gets invoked will be an entry in a jar file. There are no "subdirectories" as such. Even more radically the application may be a JApplet instance in which case it is entirely up to the server both how and where "the software is stored".
You might want to google - or check the API or your favourite examples site - for the Class method getResource(). (That's a class called Class). Suppose you have an application whose main() method is in a class MyImageGrabber.class and you want to access a text file (outputLocation.txt) that contains the full path of the image output directory. You put the text file "where the software is stored", then say// somewhere in MyImageGrabber.java
URL outputInfoUrl = getClass().getResource("outputLocation.txt");
Now you have a URL you can access (ie read using the techniques of this thread!). Using getResource() ensures you are able to get at this file even if your whole application is bundled up in a jar file or whatever.
Having said that, it is often the case that such files (let's call them data files) should not be placed in the file system relative to the application: often you want the user to have different access rights to data. Consider using an argument to the application to set this:java MyImageGrabber c:\temp\images\Or using "well known" locations like the user's "My Documents" folder for data. Environment variables can also be pressed into service.
[Edit] BTW if, in the example above you addSystem.out.println(outputInfoUrl);
you will be told where the jar file resides: which you were asking about in your other thread.
Unfortunately, that last bit of code just points me to the build folder; where all my classes reside, but no jar. Actually, my only experience with java thus far has been in the classroom, where we only ran files from the editor... if I run my main class file, should it work? Because right now, I'm given an error.
*edit* PS... looks like that wasn't the final question :P Ah, I hate code sometimes
