StorEdge SAN Foundation 4.4.11 - Timeout value for Emulex HBAs
We have recently used the new SAN Foundation 4.4.11 software to configure our storage. MPXIO was enabled and paths were successfully seen to our storage.
Reading the release notes for the SAN Foundation 4.4.11, the following bug was addressed:
6381878 fcp timeout value needs to be tunable
Where is this fcp timeout value located? We have checked both the /kernel/drv/fcp.conf, fp.conf and scsi_vhci.conf and there is no timeout parameter that can be edited.
We are looking for a parameter to edit I/O timeout values for Emulex HBAs, using the Sun SFS software. We understand there are timeout values for QLogic HBAs in the qlc.conf file. Are there any Emulex timeout values located anywhere? We have checked the /kernel/drv/emlxs.conf and no parameter values for timeouts are located here.
Perhaps there is a timeout value that can be edited for the Sun Traffic Manager software. If there is, please point out the parameter that can adjust the I/O timeout.
We have attempted to use the /etc/system parameter of:
set ssd:ssd_io_time=60;
but this appears to have no affect using the Emulex HBAs.
[1151 byte] By [
DotHillSys] at [2007-11-26 11:08:21]
# 1
In working with Sun on the Solaris 10 version of this, I was told to use these lines in /etc/system with the hex values you want to try-
set fcp:fcp_offline_delay=0xa (current default is 20)
set fp:fp_offline_ticker=0xa (current default is 90).
Of course, it hasn't worked on Solaris10 yet 8-(
HTH
Dom
# 2
Thank you for the parameters. This has helped us resolve our problem. The parameters, however, don't take effect from the /etc/system file in Solaris 9. We have had to manually edit the kernel, using adb -kw, to write the values in.
These are the parameters I placed into the /etc/system file, which did not take effect:
set fcp:fcp_offline_delay = 0x3c
set fp:fp_offline_ticker=0x5a
We edited these values using:
adb -kw
fcp_offline_delay/W3c
fcp_offline_delay/X
fcp_offline_delay 3c
# 3
Just curious, had you tried setting the parameters in the fcp.conf and fp.conf files under /kernel/drv?
My understanding is that they should work in either /etc/system or the appropriate .conf file.
Earlier tuning suggestions from Sun always had the values in decimal, but this time around the escalation engineer was stating it needed to be hex. I'm unsure why. And it still doesn't work for me on Solaris 10.
# 4
Just tried placing the parameters in the fcp.conf and fp.conf files under /kernel/drv. They worked. The following parameters took effect:
> /kernel/drv/fcp.conf
fcp_offline_delay=60;
> /kernel/drv/fp.conf
fp_offline_ticker=90;
Have you referred to the Sunsolve article 83639 posted here:
http://sunsolve.sun.com/search/document.do?assetkey=1-9-83639-1
It talks about the name SSFCP_OFFLINE_DELAY.
We now are also in need of these parameter values for Solaris 10. Please update the post if you succeed in finding the correct fcp and fp parameters, or if a patch is required.
# 5
The fixes are planned for Solaris 10 Update 3, scheduled for December. No SFK 4.4.11 for S10.
I have been working with the following T patches- 118855-28 (kernel), 119131-27 (fcp), as well as T118344-14 and T123840-02 (pre req's for the kernel patch).
Thus far trying to set the values in /etc/system has failed though I will try your settings for the .conf files. Will update if it works.
# 6
Yes I was aware of the sunsolve article, my case is based upon it, and the bugs written to address it.
I tried setting fp_offline_ticker=0xa; in the /kernel/drv/fp.conf file, and it worked!
Interestingly, the fcp_offline_delay changed to 0 and I haven't been able to get it to change based on entries in /kernel/drv/fcp.conf (not 0xa or 0xf or no value at all). Will experiment more with that value in /etc/system tomorrow.
# 7
Ok, further testing, it seems that in Solaris 10 x86 fcp_offline_delay must be set only in the /etc/system file, and fp_offline_ticker must only be set in the fp.conf file.
Still need to know (and have asked Sun Support)
1- What is the minimum acceptable value for fcp_offline_delay so we don't violate the fc spec. I can adjust it to 10 in /etc/system.
2- When fcp_offline_delay is *not* set in /etc/system, changing fp_offline_ticker sets it to 0. What is the interrelationship?
# 8
How are you determing the fcp_offline_delay parameter in /etc/system is taking effect in Solaris 10? Are you manually pulling a cable from the switch, and checking the time difference in the time stamps? Or are you verifying the value using either adb or some other Solaris tool?
# 9
Both.
On S10 I use mdb -kw, then fcp_offline_delay /D and fp_offline_ticker /D to verify the kernel settings.
For performance I start a write to my SAN device using dd, in another window I have a tail -f /var/adm/messages going, then I pull the cable.
My issue is that I have multiple connections to multiple switches. When I pull a cable from the switch to the storage, the switch registers the SCN and everything reconfigures very quickly. But if I pull a cable from the hba to the switch, SCN's aren't handled well to that host, so we have to wait for the timeouts to complete before IO retries go off another way. Thus why I need to tune the timeouts to get quicker failover.
# 10
Good information.
I realize these command are likely for Solaris 10 x86. Do you know if these same commands can be used in Solaris 10 SPARC?
The patches you referenced are for Solaris 10 x86. My search on sunsolve in respect to fcp and fp for Solaris 10 only pulls up x86 patches. I've not seen anything for SPARC. Were you provided with patches that can be used in Solaris 10 SPARC?
# 11
No, sorry, x86 is what I'm using for this project. Most times the x86 and Sparc patches are released in tandem so what's in -28 in one is the same in the other.
# 12
FYI, the T patches I was using tend to cause nasty things to happen. I found that you need to be running S10U3, as these patches are looking for libraries that are only included in U3. Specifically S10U3 will roll the Trusted Solaris libraries into the base Solaris code string, thus when you are running those patches and need to do any compiler work they will fail unless you have the TSOL libraries and headers. In fact, you can create a tar archive, but when you attempt to extract the archive it fails. I am highly concerned because these are the same basic OS patches (kernel and fcp driver) and my experience seems to indicate that patching in S10 will be broken if these patches are released as is. Or you will be required to update Solaris to U3 prior to using certain patch levels. Not good.
