Pulling URL, Parsing, Displaying
Hello All,
I'm attempting to pull a specified URL (via a specified param in the HTML), parse it, then display the results (preferably by printing out more HTML which the browser can render). However, I have no clue what so ever where I should start. Any direction for any of these steps would be much appriciated!
Cheers-
Austin
[361 byte] By [
NewWave] at [2007-9-26 1:27:03]

Ok as i read your statement i realize this thing that
you want to read all content of a particular URL and want to show them with your chioce.
If i am write then it is very simple
import java.io.*;
import java.net.*;
public class FP
{
public static void main(String args[])
{
String address=args[0];
String fileStr="";
String line;
try {
System.out.println("Fetching contents of " + address);
URL url = new URL(address);
URLConnection con = url.openConnection();
DataInputStream dis = new DataInputStream(con.getInputStream());
line = dis.readLine();
while ( line != null )
{
fileStr=fileStr + line + "\n";
line = dis.readLine();
}
/*******************
you can parse here fileStr
*******************/
System.out.println(fileStr);
}
catch (IOException ioex)
{
System.out.println("Error : " + ioex);
}
}
}
If you want also parsing method tell me i would like to help u ;)
Heya,
Thanks for the quick reply! A few comments though... I need to be able to load this in a webbrowser, and specify the URL of the data as param; see the following example:
<applet code="myapp.class" height="100" width="100">
<param name="url" value="http://my.url.com/">
</applet>
As for the parsing, here is a sample of the data:
Artist_1
Song_1
Source_1
Link_1
Latency_1
.
Artist_2
Song_2
Source_2
Link_2
Latency_2
.
Artist_3
Song_3
Source_3
Link_3
Latency_3
.
Also, is it possible to have the applet output HTML code, which the browser would then render? Thanks again for your help!
Cheers-
Austin
ok Austin
I am also doing same thing in my site but i am not using as applet for the parsing i have created one function which search for particular given variable in the string and can replace with your own text.
I don't know your meaning for parsing is same or different if it is different let me know any way this is the function
public String replaceString(String sStr,String oldStr,String newStr)
{
sStr=(sStr==null?"":sStr);
String strVar=sStr;
String tmpStr="";
String finalStr="";
int stpos=0,endpos=0,strLen=0;
while (true)
{
strLen=strVar.length();
stpos=0;
endpos=strVar.indexOf(oldStr,stpos);
if (endpos==-1)
break;
tmpStr=strVar.substring(stpos,endpos);
tmpStr=tmpStr.concat(newStr);
strVar=strVar.substring(endpos+oldStr.length()>sStr.length()?endpos:endpos+oldStr.length(),strLen);
finalStr=finalStr.concat(tmpStr);
stpos=endpos;
}
finalStr=finalStr.concat(strVar);
return finalStr;
}
In this function u can provide
fileStr=replaceString(fileStr,"searching text","yourtext");
it will search whole text and replace with your text.
If i am wrong mean this is not what u want let me know i will explain u for other problem.
See ya,
Waqar.
Heya,
What I mean by parsing is, once the applet has downloaded the code from the specified URL, I need it to take this data (just a sample):
Artist_1
Song_1
Source_1
Link_1
Latency_1
and change it into:
<TR>
<TD><A HREF="Link_1"><IMG SRC="icon.gif" BORDER="0"></A></TD>
<TD>Artist_1</TD>
<TD>Song_1</TD>
<TD>Source_1</TD>
<TD>Latency</TD>
</TR>
And do that for each block. To see what I mean by a block, look at the example data I posted earlier. Basically, I need it to generate the above HTML code for each set of information, where a set is split up by a '.' (period). Thanks again for your help!
Cheers-
Austin
Are you sure you want to do this from an applet?
1) You can't "pull" an arbitrary URL from an applet without making it signed. (There's a separate forum for signed applets under the security section.) Applets can connect to only the machine where they were loaded.
2) Applets can easily produce HTML code but it's not that simple to show it in a browser. You'd need to a) save the HTML code to a file on the server or b) make a JavaScript function that can open a new browser window and write the HTML to it and open a JavaScript connection through liveconnect.
I think it would be better to do this with server side technologies like JSP, servlets, PHP, CGI, ...
Can u give me code for the page from where u want to get the information or URL of the page so that atleast i can see what is in that page and how i can parse that.ok
This cannot be done on the server side for the pure and simple fact that the client must pull the data from the server, no exceptions. If it is done on server side, the server is left with numerous connections to the host that stores the data, and since this is done nearly 200,000 times a day, I'd like to leave the load on the client instead of the server.
As for getting the applet signed, I'm not forseeing any problems, but was unaware of those security limitations. Thank you for pointing them out!
Cheers-
Austin
Hello,
Thanks again for all your help :) I have posted example data at http://www.new-wave.net/java/example.txt. As for it displaying HTML code, I guess that here is another possability along the lines of what was previously suggested:
Is it possible to get the specified URL, then using the POST method, send the entire contents to my server where a simple C program can handle the html output, and then return it to the browser?
I think that that solution would be much better, any ideas? Thanks!
Cheers-
Austin
> Is it possible to get the specified URL, then using the
> POST method, send the entire contents to my server
> where a simple C program can handle the html output,
> and then return it to the browser?
That's exactly what CGI is, isn't it?
By the way, if the remote data doesn't change that often you can store the html in a file on the server (cacheing, sort of). This can really cut down costs per request.
I can see that doing the processing on the client would be a good idea. If signing the applet isn't a problem I think you can pretty easily write an applet that outputs the HTML to a temporary file and shows it in a browser through the normal methods - no need for javascript.
Heya,
The content will never be the same, and the client must pull the data, so, because I'm much better with server side processing, is it posisble to just write an applet that pulls a specified URL, then sends it to a specified URL via the POST method? That would be easiest because then I could do all the html processing on the server, which is best for the interactivity that is currently built into my infrastructure.
Cheers-
Austin
Well, you are clearly the expert on server side things but why having the applet in between?Yes, you can do a HTTP POST from an applet. Read this page for an explanation: http://java.sun.com/docs/books/tutorial/networking/urls/readingWriting.html
Heya,
Basically what this is doing is searching a peer-to-peer network. When it grabs the data from the specified url, it is grabbing live data. If the server is doing this for every single request, it just ends up returning 0 search results, however, if each client pulls the data, then the desired search results will be returned, but not in a viewable form, which is why it must then be passed to the server for parsing, and then returned. This seems simple, but has prooved to be a bit more challenging. Thanks again.
Cheers-
Austin