Performance in Sorting huge data objects
Background
We are extracting data from one source system, manipulating it in java and storing it in another target system.
Problem
We are sorting the java objects and then pushing the data into another system.
Time taken to sort 48000 objects is 24 minutes.
And it keeps on increasing drastically when number of objects increases.
The max size of data that we are looking forward to is around 200,000.
I used the java parameter to SET JAVA_OPTIONS=="-Xmx512m -Xss512k".
Changing it to a lower value increased the time taken for sort.
Would appriciate if someone could give me pointers for performance improvement on this process.
[691 byte] By [
ketansuri] at [2007-9-30 17:20:43]
Normally I doubt it is the sorting that taking up time.java.util.Arrays.sort and java.util.Collections.sort both are pretty efficient, no problem sorting 200,000 items. Like others mentioned, check your performance profile, see what is taking that much time.
yue42 at 2007-7-6 13:43:37 >
> java.util.Arrays.sort and java.util.Collections.sort
> both are pretty efficient, no problem sorting 200,000
> items. Like others mentioned, check your performance
> profile, see what is taking that much time.
In particular, check your comparator object performance.
> Changing it to a lower value increased the time taken
> for sort.
>
> Would appriciate if someone could give me pointers for
> performance improvement on this process.
Are you clocking the sort call only?
What data structure are you sorting?
Are you using your own sort method?
Have you overridden compareTo in the object class?
