speed up compile

Hi there,

I'm working with a big project that usually takes 45 min to compile which is a long time. I've noticed that the processor load most of the times is only at 50%. So it should be possible to speed this up, right? Does someone have some hints?

thanks in advance,

Jan

java 1.4.2

win xp

3ghz intel cpu with hyperthreading 1 gb ram

[380 byte] By [jan_techa] at [2007-10-2 1:11:16]
# 1
Turn hyperthreading off
ChuckBinga at 2007-7-15 18:32:09 > top of Java-index,Developer Tools,Java Compiler...
# 2
Thanks, I just tried that. That results in ~ +5% speed however the system feels a lot more sluggish during compile.
jan_techa at 2007-7-15 18:32:09 > top of Java-index,Developer Tools,Java Compiler...
# 3
> I'm working with a big project that usually takes 45> min to compile which is a long time.> win xp> 3ghz intel cpu with hyperthreading 1 gb ramAnd your storage (disk) is local? [As opposed to networked]?
tschodta at 2007-7-15 18:32:09 > top of Java-index,Developer Tools,Java Compiler...
# 4
yes, I compile a local copy of the project
jan_techa at 2007-7-15 18:32:09 > top of Java-index,Developer Tools,Java Compiler...
# 5
You can try passing a larger heap value to the Java vm that is running:javac -J-Xmxnnnm <proggie.java> where nnn is a value greater than 64 (which is probably your default.) This may not help, I've never tried it.
ChuckBinga at 2007-7-15 18:32:09 > top of Java-index,Developer Tools,Java Compiler...
# 6
It is already set to 512 . Nevertheless I'll try to compile with different values and if it has any effect I'll post it here.
jan_techa at 2007-7-15 18:32:09 > top of Java-index,Developer Tools,Java Compiler...
# 7

It's disk access.

Check your import statements. If there are unused ones, remove them. That should reduce disk access (as the compiler checks if the classes are there even if they're not used). Don't know if that will give a substantial performance improvement but it should do something if there's tons of them.

Apart from that, all I can suggest is you try to increase harddisk performance. Try defragmenting, if you're running a slow harddisk and/or controller upgrade those, etc..

There's little else you can do without rearchitecturing to reduce the number of dependencies.

jwentinga at 2007-7-15 18:32:09 > top of Java-index,Developer Tools,Java Compiler...
# 8
To give an indication: our current project is now at over 600 classes and compiles typically in about a minute.That's on a Pentium 3 which isn't even dedicated to the task.
jwentinga at 2007-7-15 18:32:09 > top of Java-index,Developer Tools,Java Compiler...
# 9

Playing with the heap value didn't have an significant effect.

Unfortunately I cannot reengineer the whole project but I'll keep the import statements in mind when changing or creating classes. The number of classes exceeds 10k but still it seems kind of slow. I'll propose to split the project up and create jars for the stable parts.

jan_techa at 2007-7-15 18:32:09 > top of Java-index,Developer Tools,Java Compiler...
# 10
We have 5,578 java files to compile and it only takes about 1 minute to compile. Are you compiling all 10K java files at once with one call to javac or are you individually executing javac for each java file? The latter would be very slow.
Caffeine0001a at 2007-7-15 18:32:09 > top of Java-index,Developer Tools,Java Compiler...
# 11
Woops, I counted my CVS files too. We have 2516 files. First time it took 1 minute 36 seconds. Second time it took 37 seconds.
Caffeine0001a at 2007-7-15 18:32:09 > top of Java-index,Developer Tools,Java Compiler...
# 12
You must be doing something wrong.Try this.Install JDK 1.5.0_05 and unpack the src.zip.On a 1.8 GHz AMD, this takes 5m10s to compile 6552 files, 9823 classes.I did increase the maximum memory size of the compiler to 512 MB.
Peter-Lawreya at 2007-7-15 18:32:09 > top of Java-index,Developer Tools,Java Compiler...
# 13
Ehemm (hesitating ..., but) you are not by any chance using a built system dating back to the good old C/C++ times where each source file is compiled by an individual invocation of the compiler (i.e. javac)?Harald.
pifpafpufa at 2007-7-15 18:32:09 > top of Java-index,Developer Tools,Java Compiler...
# 14

I call an ant script that does all the work. I am a little busy right now but thats what I found out :

compiling the JDK didn't work 100% because of some errors I didn't have time to resolve, however it created some thousand class files pretty fast so I think it should finish in the region of 5 min +-.

Assuming there is nothing wrong with my PC I took a look into the ant file where it seemed that not the actual compiling takes so long but the parsing and compiling of some scripts. At least thats what I think, is there an ant profiling tool ? ;)

I thought there might be some compiler setting that improves the java compile speed, however hearing your compile times it indicates that the problem lies somewhere else.

thanks for your efforts

jan

jan_techa at 2007-7-15 18:32:09 > top of Java-index,Developer Tools,Java Compiler...
# 15

> Ehemm (hesitating ..., but) you are not by any chance

> using a built system dating back to the good old

> C/C++ times where each source file is compiled by an

> individual invocation of the compiler (i.e. javac)?

>

With C/C++ you can very well invoke the compiler on several sources.

posman@proli:/tmp/efe> cat x.c

void foo() { }

posman@proli:/tmp/efe> cat y.c

int main () {

foo();

return 0;

}

posman@proli:/tmp/efe> gcc x.c y.c -o z

posman@proli:/tmp/efe> ./z

posman@proli:/tmp/efe> echo $?

0

BIJ001a at 2007-7-20 14:45:11 > top of Java-index,Developer Tools,Java Compiler...