Java library for large list sorts in small amount of memory
Hi All
Wondering whether anybody would know of a ready made java library for sorting large lists of objects. We will write ourselves if necessary but I'm hoping this is the sort of problem that has been solved a million times and may hopefully have been wrapped behind some nice interface
What we need to do is something along the line of:
Load a large list with objects (duh!)
If that list fits into an allowed amount of memory great. Sort it and end of story.
If it won't then have some mechanism for spilling over onto disk and perform the best sort algorithm possible given some spill over onto disk.
These sorts need to be used in scenarios such as a random selection of 10% of the records in a 100G unsorted file and delivered sorted, with the jobs doing the sorts being limited to as little as 64MB of memory (these are extremes but ones that happen quite regularly).
Some of you might think "just use a database". Been there. Done that. And for reasons beyond the scope of this post its a no goer.
Thanking you for all your help in advance.
