weird T3 iostat numbers
I'm investigating an old T3 has has hideously bad random IO performance. Back in the old days, some dunderhead configured it as a RAID5. But I am seeing some numbers that don't add up. I am doing a dumb old mkfile 2000m, and here are the results compared to a well-striped A5200 and a V400 internal disk.
A440: time mkfile 2000m /junk
real0m33.120s [etc]
iostat -xn 10:
r/sw/skr/skw/s wait actv wsvc_t asvc_t %w %b device
0.0 533.30.0 65990.7 0.0 3.20.06.00 100 c1t0d0
OK, that adds up, 2000 / 33 is about 66 Mbytes/sec
A5200: time mkfile 2000m junk
real0m53.150s
iostat -xn 10:
r/sw/skr/skw/s wait actv wsvc_t asvc_t %w %b device
0.0 199.00.0 10516.2 2.9 13.714.669.1 54 100 c1t2d0
[repeated 8 times - it's 4 stripes of 2 mirrors = about 40 kw/s]
OK, that adds up 2000 / 53 is about 40 MBytes/sec
T3: time mkfile 2000m /junk
real1m5.866s
ostat -xn 10:
r/sw/skr/skw/s wait actv wsvc_t asvc_t %w %b device
0.063.80.0 3458.0 0.0 15.70.0 246.10 100 c5t1d0
BUT 2000 / 65 is about 30 MBytes/sec, but kw/s in the iostat is only 3.5 MBytes/sec.
What am I missing here? Is this a caching sideffect? I have only 256 MB in the T3 cache. The wall cloc time seems reasonable, and I don't seem to be filling it up the cache based on the total wall clock time, but if the cache were working, shouldn't I see fiber channel type speeds in the kw/s stat?
Never seen this inconsistency before....
