Caches and SoftReferences
Recently we've been migrating our caches from being based on Apache Commons to ehCache. I'm also putting the finishing touches on a remote caching layer, currently backed by memcached, in order to reduce the database load. Its actually quite impressive how much of a performance gain these two changes make, so I'm looking forward to releasing them into production.
However, we still have memory issues, and while we are moving to 64-bit OSes to increase the JVM's footprint, it doesn't solve the root problem. What we really need to do is convert more caches over and restrict their sizes. Even then, I'm still concerned since a client could hold onto an object for long periods of time and it wouldn't be garbage collected despite the cache discarding the element.
I'm considering creating a client proxy to the real element, so that the cache controls the full lifecycle of the object. The proxy would simply have a soft reference to the element, its cache key, and delegate all calls to the true element. If the element is discarded by the cache (soft reference holds a null), the proxy would request it before making the clients call. On the face of it, this would work pretty well, especially in cases where some developer requests ALL elements of the cache - a set of empty proxies with each key could be returned rather than loading, say, every customer from the database. A very light load would be on the db populating the keys and the element lazily loaded from the remote cache as needed.
What I'm concerned about is serialization and saving of data. I don't think serialization is much of an issue, since if the client really wanted to store the data long-term it would be in an document format. The key would be serializable and thus the element reloaded on usage after deserialization. On saving, what if the client wanted to make changes to the element, such as a customer's address? There's a chance that the element would be discarded between calls and the changes lost. I'm thinking that I'd need to convert to a strong reference on mutator methods until resaved.
Are there any other issues I'm missing? Does anyone have experience with this approach? The main aspect I don't like is that every cache would need to provide a custom proxy implementation. With a large number of caches, developers adding more, and few having the discipline, I'm betting that at best this would be only work on the hottest/largest caches. Is there any good way to generalize this for the average developer so it feels trivially easy to follow this idiom?
Thanks!
