writing log files to CFS
Hi,
On a 3 node cluster connected to se6920 shared storage, we are seeing that writing log files to globally mounted directories is taking much longer than the same action to local UFS. Approx 15 times slower to a global mount - we were writing to the actual mount point.
However, the same application writing the same logs to a sub directory of the mount point has is only 2 times as slow as to the local UFS.
Also, when we made a test of removing 2 out of the 3 nodes from the cluster - the writes to the actual mount point was the same as the writes to the subdirectory in the last test. (presumably less nodes to be checkpointing - but interesting that when all 3 nodes are present logs written to the sub-dir of a global mount point is the same as if only one node present in the cluster)
I know configuration is probably key here, but I was wondering if the problem with slower open/append/close for log file is something inherent to the CFS? (at present we have moved this logging to the local UFS)
Thanks,
Trevor
[1064 byte] By [
trevor_tec] at [2007-11-26 10:47:11]
# 1
Yes, it is inherent in CFS. Consistency needs to be maintained across all nodes and that comes at a cost. Lots of optimisations have been done to this over the years, but it will always be slower than UFS in some cases just because it needs to do more work and has to do it over wires with latency.
All global file systems will suffer some sorts of performance degradation compare with their local counterparts. The degradation of a particular workload will be dependent on the particular design choices made for the global file system.
If you are only writing from one node at any one time make sure you have an HAStoragePlus resource configured to ensure that the primary I/O path is collocated with the application performing the writes.
Applications that don't do allocating writes, or much file creation, e.g. databases, perform much better and are nearer to local FS performance.
Tim
# 2
Thanks Tim.
We have changed our application to write logs locally and copy them to global storage instead.
I'm still not sure though why the open/append works better on the CFS when it is in a sub-directory of the CFS mount point? - this is difficult for me to understand why there is less checking required in sub-directories... should this be true?
Trevor
# 4
Well the architects and engineers for this are at a loss to explain your observations. Some suggestions related to having low memory but not much else. I guess that as long as you have found a suitable workaround then that's all that matters.
Sun Cluster 3.2 should improve the performance of these operations but it will never reach parity with a normally mounted (local) UFS file system.
Tim
