how to handle garbage collection in a recursive function
I'm writing a web crawler which basically looks like this:
//Entity class representing a page on the web
class Page
{
...
Page[] children;
.....
}
publicvoid crawl(Page parentPage)
{
....
Page[] children = getChildrenUsingSomeFunction(parentPage);
parentPage.setChidren(children);
entitymanager.persist(parentPage);//write parentPage to database
for(Page child : children)
{
crawl(child);
}
.....
}
After writing an object to the database, I want to remove that object from main memory. A page is always referenced by its parent page, except for the root pages, so the garbage collector will never "remove" a page object from memory. But when crawling a million pages, 1 gig of RAM is obviously not sufficient.
How would you solve this problem?

