Help on Threads in Java
I need some basic help in regards to threading in Java.
I have no history of writing threaded applications in any programming language.
I'm working on a kind of web bot, for a bunch of XML sources. There are 10's of thousands of sources and the timestamp difference between the lastest and last updated item is growing to almost a full day! This is not good enough really.
Here's the basic layout of the main class:
publicclass filer{
privatestaticclass botimplements Runnable{
private mysql db;
privateint botID;
public bot(finalint ID){
botID = ID;
try{
// Some prepared statements here...
}
catch (SQLException ex){System.exit(0);}
catch (NoSuchAlgorithmException ex){System.exit(0);}
}
publicvoid run(){
db =new mysql("localhost","xxx","xxx","xxx");
if (db.connected ==true) spider();
else System.exit(0);
}
finalvoid spider(){
// Spider logic here using botID...
spider();// Recursion...
}
}
publicstaticvoid main(String[] args){
for (int i = 0; i < 51; i++) (new Thread(new bot(i))).start();
}
}
I'm trying to run 50 threads, each doing their task of downloading data, parsing it and inserting where applicable into the MySQL database. 50 should reduce difference in last and latest crawled to 1 hour, which would be great!
Am I missing something? Is there anything special that needs to be taken into consideration when using MySQL?
Any help much appreciated!
