Pulling URL, Parsing, Displaying

Hello All,

I'm attempting to pull a specified URL (via a specified param in the HTML), parse it, then display the results (preferably by printing out more HTML which the browser can render). However, I have no clue what so ever where I should start. Any direction for any of these steps would be much appriciated!

Cheers-

Austin

[361 byte] By [NewWave] at [2007-9-26 1:27:03]
# 1

Ok as i read your statement i realize this thing that

you want to read all content of a particular URL and want to show them with your chioce.

If i am write then it is very simple

import java.io.*;

import java.net.*;

public class FP

{

public static void main(String args[])

{

String address=args[0];

String fileStr="";

String line;

try {

System.out.println("Fetching contents of " + address);

URL url = new URL(address);

URLConnection con = url.openConnection();

DataInputStream dis = new DataInputStream(con.getInputStream());

line = dis.readLine();

while ( line != null )

{

fileStr=fileStr + line + "\n";

line = dis.readLine();

}

/*******************

you can parse here fileStr

*******************/

System.out.println(fileStr);

}

catch (IOException ioex)

{

System.out.println("Error : " + ioex);

}

}

}

If you want also parsing method tell me i would like to help u ;)

leewaqar at 2007-6-29 1:10:50 > top of Java-index,Archived Forums,Java Programming...
# 2

Heya,

Thanks for the quick reply! A few comments though... I need to be able to load this in a webbrowser, and specify the URL of the data as param; see the following example:

<applet code="myapp.class" height="100" width="100">

<param name="url" value="http://my.url.com/">

</applet>

As for the parsing, here is a sample of the data:

Artist_1

Song_1

Source_1

Link_1

Latency_1

.

Artist_2

Song_2

Source_2

Link_2

Latency_2

.

Artist_3

Song_3

Source_3

Link_3

Latency_3

.

Also, is it possible to have the applet output HTML code, which the browser would then render? Thanks again for your help!

Cheers-

Austin

NewWave at 2007-6-29 1:10:50 > top of Java-index,Archived Forums,Java Programming...
# 3

ok Austin

I am also doing same thing in my site but i am not using as applet for the parsing i have created one function which search for particular given variable in the string and can replace with your own text.

I don't know your meaning for parsing is same or different if it is different let me know any way this is the function

public String replaceString(String sStr,String oldStr,String newStr)

{

sStr=(sStr==null?"":sStr);

String strVar=sStr;

String tmpStr="";

String finalStr="";

int stpos=0,endpos=0,strLen=0;

while (true)

{

strLen=strVar.length();

stpos=0;

endpos=strVar.indexOf(oldStr,stpos);

if (endpos==-1)

break;

tmpStr=strVar.substring(stpos,endpos);

tmpStr=tmpStr.concat(newStr);

strVar=strVar.substring(endpos+oldStr.length()>sStr.length()?endpos:endpos+oldStr.length(),strLen);

finalStr=finalStr.concat(tmpStr);

stpos=endpos;

}

finalStr=finalStr.concat(strVar);

return finalStr;

}

In this function u can provide

fileStr=replaceString(fileStr,"searching text","yourtext");

it will search whole text and replace with your text.

If i am wrong mean this is not what u want let me know i will explain u for other problem.

See ya,

Waqar.

leewaqar at 2007-6-29 1:10:50 > top of Java-index,Archived Forums,Java Programming...
# 4

Heya,

What I mean by parsing is, once the applet has downloaded the code from the specified URL, I need it to take this data (just a sample):

Artist_1

Song_1

Source_1

Link_1

Latency_1

and change it into:

<TR>

<TD><A HREF="Link_1"><IMG SRC="icon.gif" BORDER="0"></A></TD>

<TD>Artist_1</TD>

<TD>Song_1</TD>

<TD>Source_1</TD>

<TD>Latency</TD>

</TR>

And do that for each block. To see what I mean by a block, look at the example data I posted earlier. Basically, I need it to generate the above HTML code for each set of information, where a set is split up by a '.' (period). Thanks again for your help!

Cheers-

Austin

NewWave at 2007-6-29 1:10:50 > top of Java-index,Archived Forums,Java Programming...
# 5

Are you sure you want to do this from an applet?

1) You can't "pull" an arbitrary URL from an applet without making it signed. (There's a separate forum for signed applets under the security section.) Applets can connect to only the machine where they were loaded.

2) Applets can easily produce HTML code but it's not that simple to show it in a browser. You'd need to a) save the HTML code to a file on the server or b) make a JavaScript function that can open a new browser window and write the HTML to it and open a JavaScript connection through liveconnect.

I think it would be better to do this with server side technologies like JSP, servlets, PHP, CGI, ...

jsalonen at 2007-6-29 1:10:50 > top of Java-index,Archived Forums,Java Programming...
# 6
Can u give me code for the page from where u want to get the information or URL of the page so that atleast i can see what is in that page and how i can parse that.ok
leewaqar at 2007-6-29 1:10:50 > top of Java-index,Archived Forums,Java Programming...
# 7

This cannot be done on the server side for the pure and simple fact that the client must pull the data from the server, no exceptions. If it is done on server side, the server is left with numerous connections to the host that stores the data, and since this is done nearly 200,000 times a day, I'd like to leave the load on the client instead of the server.

As for getting the applet signed, I'm not forseeing any problems, but was unaware of those security limitations. Thank you for pointing them out!

Cheers-

Austin

NewWave at 2007-6-29 1:10:50 > top of Java-index,Archived Forums,Java Programming...
# 8

Hello,

Thanks again for all your help :) I have posted example data at http://www.new-wave.net/java/example.txt. As for it displaying HTML code, I guess that here is another possability along the lines of what was previously suggested:

Is it possible to get the specified URL, then using the POST method, send the entire contents to my server where a simple C program can handle the html output, and then return it to the browser?

I think that that solution would be much better, any ideas? Thanks!

Cheers-

Austin

NewWave at 2007-6-29 1:10:50 > top of Java-index,Archived Forums,Java Programming...
# 9

> Is it possible to get the specified URL, then using the

> POST method, send the entire contents to my server

> where a simple C program can handle the html output,

> and then return it to the browser?

That's exactly what CGI is, isn't it?

By the way, if the remote data doesn't change that often you can store the html in a file on the server (cacheing, sort of). This can really cut down costs per request.

I can see that doing the processing on the client would be a good idea. If signing the applet isn't a problem I think you can pretty easily write an applet that outputs the HTML to a temporary file and shows it in a browser through the normal methods - no need for javascript.

jsalonen at 2007-6-29 1:10:50 > top of Java-index,Archived Forums,Java Programming...
# 10

Heya,

The content will never be the same, and the client must pull the data, so, because I'm much better with server side processing, is it posisble to just write an applet that pulls a specified URL, then sends it to a specified URL via the POST method? That would be easiest because then I could do all the html processing on the server, which is best for the interactivity that is currently built into my infrastructure.

Cheers-

Austin

NewWave at 2007-6-29 1:10:50 > top of Java-index,Archived Forums,Java Programming...
# 11
Well, you are clearly the expert on server side things but why having the applet in between?Yes, you can do a HTTP POST from an applet. Read this page for an explanation: http://java.sun.com/docs/books/tutorial/networking/urls/readingWriting.html
jsalonen at 2007-6-29 1:10:50 > top of Java-index,Archived Forums,Java Programming...
# 12

Heya,

Basically what this is doing is searching a peer-to-peer network. When it grabs the data from the specified url, it is grabbing live data. If the server is doing this for every single request, it just ends up returning 0 search results, however, if each client pulls the data, then the desired search results will be returned, but not in a viewable form, which is why it must then be passed to the server for parsing, and then returned. This seems simple, but has prooved to be a bit more challenging. Thanks again.

Cheers-

Austin

NewWave at 2007-6-29 1:10:50 > top of Java-index,Archived Forums,Java Programming...