java io program suggestion

Hi all,

I need to write a Java program to insert 1,000,000 records from a text file to the DataBase. Each row in the file represents a record. Each field has it's own fix width. So I guess what I have to do is to define the width for each field, for example 1-4 represents BatchNumber, 5-8 represents CampCode. There are a lot of fields in the file, I guess until 500. Seems like I have to hardcode those field width in the Java program, such as

BufferedReader br =new BufferedReader(new FileReader(filepath));

String s =null;

String[] paramNames;// all the field names in the database

String[] paramVals;// all the field vales read from file

while ((s=br.readLine())!=null)

{

paramVals[0] = s.substring(1, 4)

paramVals[1] = s.substring(5, 10)

.......

}

and then I would have to insert them to the database, such as

StringBuffer sb =new StringBuffer();

sb.append("insert INTO xx_table (");

for (int i = 0; i < paramNames.length(); i++ ){

sb.append(paramVals[i]);

sb.append(",");

}

sb.replace(sb.length()-1, sb.length(),")");

for (int i = 0; i < paramVals.length(); i++ ){

sb.append("'");

sb.append(paramVals[i]);

sb.append("'");

sb.append(",");

}

sb.replace(sb.length()-1, sb.length(),")");

The above codes are tedious, and I think it's a good design. I want to start with a good design so that folks can follow it up later on. Please give me some suggestions and ideal codes.

Thanks in advance

Transistor

[2342 byte] By [popohomaa] at [2007-10-2 11:39:04]
# 1
I don't know if it's a good design or not, what I want to suggest is that before you start doing any design at all, see whether the database you are using has any facilities for bulk importing of data from flat files. If it does, then you should use them if at all possible.
DrClapa at 2007-7-13 5:26:55 > top of Java-index,Other Topics,Patterns & OO Design...
# 2

What RDBMS are you using? Most have native, standalone executables (such as Oracle's imp and exp programs) that can read a flat-file to insert into a database. Some require a little scripting ahead of time (mainly to map the flat file fields to columns in one or more tables), but if you are really inserting over a million records, why do this in Java? Use the tools your RDBMS is already providing, perhaps hooked up to a scheduler if this will be a batch import.

- Saish

Saisha at 2007-7-13 5:26:55 > top of Java-index,Other Topics,Patterns & OO Design...