Abysmal UFS performance on SB1K...
Hi there,
Not sure if this is the right forum to post, but we'll see...
I am trying to understand why my SB1K gives me abysmal performance on sequential read/write...
My setup is 2x15,000RPM 36GB drives, Solaris 10. I run the Nastran FEA solver, which sometimes creates huge scratch files that are constantly accessed... I direct all my scratch files on my second drive... I am seeing some very long solve time compared to much slower machines. An example would be SB1K 2x900 being 3 times slower than an sgi Octane2 2x400 for solves that use extensive I/O. For solves with little I/O, the SB1K is about twice the speed of the Octane2, which is what I expected. It is not limited to Nastran, other FEA solvers show the same trend...
So, it appears to me that the culprit is the scratch filesystem... I tried to mount the drive with forcedirectio, but it looks like it made things even worse... Does anybody have any suggestion?
[966 byte] By [
unixmuseum] at [2007-11-25 23:05:59]
# 1
B1k doesn't support any 15k disk options, is this standard UFS configuration?
# 2
No, these are drives I bought separately...When you say "doesn't support", you mean "Sun doesn't offer this configuration, therefore wouldn't support it" right?
# 3
Well kind of, the system doesn't have a Sun disk available that has 15k plator speed:
<a href="http://sunsolve.sun.com/handbook_pub/Systems/SunB lade1000/components.html#Disks" target="_blank"> http://sunsolve.sun.com/handbook_pub/Systems/SunBlade1000/co mponents.html#Disks</a>
Just installing a 15k generic disk is not going to yield the performance you might think. There is a way to tune UFS to take advantage of higher speed disks, but I don't think it works properly unless the disk has Sun firmware.
<b>ED</b> Updated spelling mistake.
# 4
<table border="0" align="center" width="90%" cellpadding="3" cellspacing="1"><tr><td class="SmallText"><b>m-lennon wrote</b></td></tr><tr><td class="quote">
Well kind of, the system doesn't have a Sun disk available that has 15k plator speed:
<a href="http://sunsolve.sun.com/handbook_pub/Systems/SunB lade1000/components.html#Disks" target="_blank">http://sunsolve.sun.com/handbook_pub/Systems/SunBlade1000/co mponents.html#Disks</a>
</td></tr></table>
Right, it goes up to 10,000RPM... BTW, $4,100 for 73GB, is that reasonable?
<table border="0" align="center" width="90%" cellpadding="3" cellspacing="1"><tr><td class="SmallText"><b>m-lennon wrote</b></td></tr><tr><td class="quote">
Just installing a 15k generic disk is not going to yield the performance you might think. There is a way to tune UFS to take advantage of higher speed disks, but I don't think it works properly unless the disk has Sun firmware.
</td></tr></table>
YOU'RE KIDDING ME!!!! #&^#$&^#$&#^$#& I had no idea that this kind of crap could even be remotely possible on modern systems... This is no specialty hardware, this is a simple workstation for Pete's sake! Very disturbing news...
I guess I got tricked (or spoiled) by my sgi background... So, you're implying that a simple lack of firmware turns a really nice and fast HD into a clunker... That doesn't sound right, ethically I mean... The drives I put into the SB1K are HP/Compaq drives, brand new, 15,000RPM!!! It's a shame!!!
So now, I got a DS1000 enclosure for the scratch files and have to get hard drives to put in it, am I going to hit the same stuff, i.e. do I have to get Sun hard drives so the performance won't suck?
# 5
I'm posting information here to try and help you, so what's with all the childish crap?
If you don't like the idea of spending $4100 on a disk you can always take the issue up with Sun, I did not suggest that you go buy a Sun disk nor did I try to sell a disk to you, I merely pointed out that there is no 15k disk option supported on a B1k.
I also suggested that the default disk speed revolution for UFS is 3600RPM and unless you tune the file system your disks will rotate at that speed even if the disks are 5400, 7200, 10000 or 15000. I have been experimenting with tuning disk speed on non Sun disks ( EMC Clariion JBOD ) attached to a Blade 1000 and this is why I said that <i>I don't think</i> ( meaning that I am not 100% sure ) it works with non Sun disks. I was willing to share some information with on the subject, but you have just made it apparent that you are an idiot and you don't need any help from me.
idiot... oh sorry I meant to say genius!
# 6
<table border="0" align="center" width="90%" cellpadding="3" cellspacing="1"><tr><td class="SmallText"><b>m-lennon</b></td></tr><tr> ;<td class="quote">
I'm posting information here to try and help you, so what's with all the childish crap?
</td></tr></table>Childish? What is childish about what I said? What's with the attitude? I don't understand why you're reacting like that at all... I do appreciate the information you provided, and reacted to it by asking more question...
<table border="0" align="center" width="90%" cellpadding="3" cellspacing="1"><tr><td class="SmallText"><b>m-lennon</b></td></tr><tr> ;<td class="quote">
If you don't like the idea of spending $4100 on a disk you can always take the issue up with Sun, I did not suggest that you go buy a Sun disk nor did I try to sell a disk to you, I merely pointed out that there is no 15k disk option supported on a B1k.
</td></tr></table>I don't like the idea of spending $4,100 for 73GB drive, you're right... Would you? I don't see why you think I was implying anything personal at all in there... Your reaction puzzles me...
<table border="0" align="center" width="90%" cellpadding="3" cellspacing="1"><tr><td class="SmallText"><b>m-lennon</b></td></tr><tr> ;<td class="quote">
I also suggested that the default disk speed revolution for UFS is 3600RPM and unless you tune the file system your disks will rotate at that speed even if the disks are 5400, 7200, 10000 or 15000. I have been experimenting with tuning disk speed on non Sun disks ( EMC Clariion JBOD ) attached to a Blade 1000 and this is why I said that <i>I don't think</i> ( meaning that I am not 100% sure ) it works with non Sun disks.
</td></tr></table>I read your post again, and there is none of this information (besides "it might not work on non-Sun disks"). The 3,600RPM is interesting information, and I thank you for it...
<table border="0" align="center" width="90%" cellpadding="3" cellspacing="1"><tr><td class="SmallText"><b>m-lennon</b></td></tr><tr> ;<td class="quote">
I was willing to share some information with on the subject, but you have just made it apparent that you are an idiot and you don't need any help from me.
idiot... oh sorry I meant to say genius!
</td></tr></table>
What is this? Who do you just think you are? Is insulting people really necessary? Does that make you feel better? Because it certainly doesn't make you look smarter. Do you really believe that "Senior Member" on a forum is a status that gives you any rights? All you've done there by this ridiculous answer is make this thread, which was a genuine hardware question, look like a Linux kids forum where self-proclaimed experts insult "the noobs"... I really don't understand why you are reacting like that... I think you got it all wrong...
You are way out of line.
# 7
Senior member is a title automatically given to users with a certain number of posts, I didn't choose it, I don't claim to be anything special at all, I don't think that I am smarter than you or anyone else, I couldn't care less! My intensions using this forum is to learn and share knowledge I have with other users. I made my remarks to you because I sensed a tone of sarcasm in your response to my offer to help you. Also the post sounds very much like " why is my Sun system is under performing " but you are using default kernel settings and unsupported components, sounds a bit stupid if you ask me. I'm sorry if my remark insulted you, and I'm sorry if I came across with an attitude, perhaps I misunderstood your first response.
# 8
<table border="0" align="center" width="90%" cellpadding="3" cellspacing="1"><tr><td class="SmallText"><b>m-lennon</b></td></tr><tr> ;<td class="quote">
My intensions using this forum is to learn and share knowledge I have with other users.
</td></tr></table>
Exactly mine too, we are definitelty on the same page! But I will make no judgment calls on what you know or don't know, and definitely not call you names if you don't know something that seems obvious to me.
<table border="0" align="center" width="90%" cellpadding="3" cellspacing="1"><tr><td class="SmallText"><b>m-lennon</b></td></tr><tr> ;<td class="quote">
I made my remarks to you because I sensed a tone of sarcasm in your response to my offer to help you.
</td></tr></table>I can assure you that you sensed wrong. It wasn't sarcasm at all, it was surprise... I obviously welcome (and are thankful for) every single piece of information that could help me understand what the problem is with my system. Besides, calling people names is a little overreacting to sarcasm, don't you think?
<table border="0" align="center" width="90%" cellpadding="3" cellspacing="1"><tr><td class="SmallText"><b>m-lennon</b></td></tr><tr> ;<td class="quote">
Also the post sounds very much like " why is my Sun system is under performing " but you are using default kernel settings and unsupported components, sounds a bit stupid if you ask me.
</td></tr></table>
Yes, I am asking why my Sun is underperforming, and yes, I am using non-Sun hard drives...
You make it sound like a crime... Did you come to this world with some genetic Solaris knowledge or did you learn? Well, picture me as you when you started working on Solaris. One difference I guess is that my job is not to be an expert in Solaris or Sun hardware, I am merely a user who's trying to make Solaris/Sparc work in a world of UNIX-bashing Windows brainwashed.
I have quite a few young engineers working for me, do you think they would want to stay in the company if I was screaming at them and calling them names every time they make a move that looks stupid to me? No, of course not...
And what does "unsupported" mean anyway? In most cases it's mainly a way of shielding support services, which is perfectly understandable (I worked for over a decade for a large ISV, I used the "unsupported" shield many times). But we're talking about a hard drive here, it's not like I'm trying to jamm some EDO RAM into the Sun Blade!
And yes, I am new to Solaris, I don't think that makes me stupid, or what I did stupid. I come from IRIX, which is different in many ways, including the way it deals with storage and bandwith apparently. I'm just learning here!
"Uninformed" would probably be a much more accurate description of my situation wrt Solaris (and definitely more appropriate for a forum).
<table border="0" align="center" width="90%" cellpadding="3" cellspacing="1"><tr><td class="SmallText"><b>m-lennon</b></td></tr><tr> ;<td class="quote">
I'm sorry if my remark insulted you, and I'm sorry if I came across with an attitude, perhaps I misunderstood your first response.
</td></tr></table>Yes, you did misunderstand my response for sure and responded in a very innapropriately harsh way... But hey, that's OK, no hard feelings... Everybody has a bad day every now and then and I've been around long enough to be able to take it.
Like this morning: the SB1K refused to boot. It starts, then a triple-beep and shuts down... A nice feature of the SB1K this power management thing... At least it doesn't boot and blows the circuit up... The bonus is I had an excuse for shutting down two Windows machines to boot the SB1K :-)
Anyway, shall we start from scratch? You made an interesting comment about 3,600RPM default, I'm curious to learn... I'm looking at the doc and don't seem to find a hint...
# 9
What I would be most concerned with using the non Sun disks is the type of internal storage that is used on the Blade 1000. FC-AL: Devices taking part in an arbitrated loop rely on system controller and disk firmware to control Layer FC-1 ( FC encoding and decoding ) and Layer FC-2 ( FC framing and flow control ). Use of non Sun disks could slow down communication between devices on the loop because of AL_PA incompatibilities, which in turn will have a negative impact the application performance. By using an external storage array and an external HBA on the system, FC-1 and FC-2 layer communication is between the external HBA and the array controller, the array controller then communicates with the disks eliminating any possibility of communication related bottlenecks. Disk firmware plays a major role in FC-AL communication, especially with a JBOD scenario and you could consider a Blade 1000 as having a mini JBOD for internal storage. It's unfortunate that your Blade is gone down, I suggest that you attach a serial cable to find out what is causing the problem.
# 10
<table border="0" align="center" width="90%" cellpadding="3" cellspacing="1"><tr><td class="SmallText"><b>m-lennon</b></td></tr><tr> ;<td class="quote">
What I would be most concerned with using the non Sun disks is the type of internal storage that is used on the Blade 1000. FC-AL: Devices taking part in an arbitrated loop rely on system controller and disk firmware to control Layer FC-1 ( FC encoding and decoding ) and Layer FC-2 ( FC framing and flow control ). Use of non Sun disks could slow down communication between devices on the loop because of AL_PA incompatibilities, which in turn will have a negative impact the application performance. By using an external storage array and an external HBA on the system, FC-1 and FC-2 layer communication is between the external HBA and the array controller, the array controller then communicates with the disks eliminating any possibility of communication related bottlenecks. Disk firmware plays a major role in FC-AL communication, especially with a JBOD scenario and you could consider a Blade 1000 as having a mini JBOD for internal storage.
</td></tr></table>Got it! FC is more picky, isn't it? One thing though, Solaris 10 seemed "happy" with the drives, apparently communicating & formatting AOK. But as you explained, there should be an I/O clash somewhere, should I be able to see this anywhere? I don't see anything in messages or syslog...
<table border="0" align="center" width="90%" cellpadding="3" cellspacing="1"><tr><td class="SmallText"><b>m-lennon</b></td></tr><tr> ;<td class="quote">
It's unfortunate that your Blade is gone down, I suggest that you attach a serial cable to find out what is causing the problem.
</td></tr></table>Oh no, no big deal... It was a power-related issue... We've been having all sorts of power issues in the office... This one was fixed very quickly: turning off a couple of Windows machines...
