Dual HBA port 2 lights flashing

Goal: Trying to get 1 machine - with dual HBA

connected to dual controller

Problem: Unable to get redundant connection to work,

HBA port 2 - all lights flashing.

2 green lights flash at lower controller port 5 also

- looks like 2nd port isn't active ?

Tried: swapped cable, swapped SFP, swapped chan, etc...

note: controller channel report below, no speed chan 5 ?

summary --

HBA: SG-XPCIE2FC-QF4 (Qlogic 4gb dual)

T2000 with Solaris 10 11/06

SE 3510FC w/dual controllers, 12x300gb (no expansion)

port 1 -> chan 0 upper cntrl

port 2 -> chan 5 lower cntrl

- fresh install Solaris 11/06

- install StorEdge 3510 management 2.3

- patch 113723-15 upgrade FW 327R -> 415F

- ssconsole add server

- assign raid controller mgr

- custom config 3 LD's

- HBA port 2 flashing all 3 led's

- stmsboot -e (turn on multipath)

- format shows 3 new LD's (part/labeled them)

- HBA port 2 flashing all 3 led's

- patchadd 119130-32 (qlc patch)

- HBA port 2 flashing all 3 led's

- can NOT get port 2 to stop flashing

- multipathing doesn't show 2nd controller

- device info

mpathadm list initiator-port

Initiator Port: iqn.1986-03.com.sun:01:00144f2b2a7a.459fd2bc,4000002a00ff

Initiator Port: 210100e08ba9bd77

Initiator Port: 210000e08b89bd77

# mpathadm show initiator-port 210100e08ba9bd77

Initiator Port: 210100e08ba9bd77

Transport Type: Fibre Channel

OS Device File: /devices/pci@7c0/pci@0/pci@8/SUNW,qlc@0,1/fp@0,0

# mpathadm show initiator-port 210000e08b89bd77

Initiator Port: 210000e08b89bd77

Transport Type: Fibre Channel

OS Device File: /devices/pci@7c0/pci@0/pci@8/SUNW,qlc@0/fp@0,0

# mpathadm list lu

/dev/rdsk/c4t600C0FF0000000000A7EF83C6918B300d0s2

Total Path Count: 1

Operational Path Count: 1

/dev/rdsk/c4t600C0FF0000000000A7EF83293102500d0s2

Total Path Count: 1

Operational Path Count: 1

/dev/rdsk/c4t600C0FF0000000000A7EF82174E56600d0s2

Total Path Count: 1

Operational Path Count: 1

mpathadm show lu /dev/rdsk/c4t600C0FF0000000000A7EF83C6918B300d0s2

Logical Unit: /dev/rdsk/c4t600C0FF0000000000A7EF83C6918B300d0s2

mpath-support: libmpscsi_vhci.so

Vendor: SUN

Product: StorEdge 3510

Revision: 415F

Name Type: unknown type

Name: 600c0ff0000000000a7ef83c6918b300

Asymmetric: no

Current Load Balance: round-robin

Logical Unit Group ID: NA

Auto Failback: on

Auto Probing: NA

Paths:

Initiator Port Name: 210000e08b89bd77

Target Port Name: 216000c0ff8a7ef8

Override Path: NA

Path State: OK

Disabled: no

Target Ports:

Name: 216000c0ff8a7ef8

Relative ID: 0

# cfgadm -al

Ap_IdTypeReceptacle OccupantCondition

c0scsi-busconnectedconfiguredunknown

c0::dsk/c0t0d0diskconnectedconfiguredunknown

c0::dsk/c0t1d0diskconnectedconfiguredunknown

c0::dsk/c0t2d0diskconnectedconfiguredunknown

c0::dsk/c0t3d0diskconnectedconfiguredunknown

c1scsi-busconnectedconfiguredunknown

c1::dsk/c1t0d0CD-ROMconnectedconfiguredunknown

c2fc-private connectedconfiguredunknown

c2::216000c0ff8a7ef8 diskconnectedconfiguredunknown

c3fc connectedunconfigured unknown

# luxadm probe

No Network Array enclosures found in /dev/es

Found Fibre Channel device(s):

Node WWN:206000c0ff0a7ef8 Device Type:Disk device

Logical Path:/dev/rdsk/c4t600C0FF0000000000A7EF83C6918B300d0s2

Node WWN:206000c0ff0a7ef8 Device Type:Disk device

Logical Path:/dev/rdsk/c4t600C0FF0000000000A7EF82174E56600d0s2

Node WWN:206000c0ff0a7ef8 Device Type:Disk device

Logical Path:/dev/rdsk/c4t600C0FF0000000000A7EF83293102500d0s2

CONTROLLER INFO

sccli> show configuration

...abreviated...

SCCli version 2.3.0

built 2006.03.15.09.49

build 12 for solaris-sparc

* inquiry-data

Vendor: SUN

Product: StorEdge 3510

Revision: 415F

Peripheral Device Type: 0x0

NVRAM Defaults: 415F 3510 S470F

Bootrecord version: 1.31H

Serial Number: 0A7EF8

Page 80 Serial Number: 0A7EF82174E56600

Page 83 Logical Unit Device ID: 600C0FF0000000000A7EF82174E56600

Page 83 Target Device ID: 206000C0FF0A7EF8

Page D0 Fibre Channel Address: A7 (id 40)

Page D0 Node Name: 206000C0FF0A7EF8

Page D0 Port Name: 216000C0FF8A7EF8

Device Type: Primary

unique-identifier: A7EF8

* redundancy-mode

Primary controller serial number: 8105312

Primary controller location: Upper

Redundancy mode: Active-Active

Redundancy status: Enabled

Secondary controller serial number: 8104517

* channels

Ch TypeMediaSpeedWidth PID / SID

--

0 HostFC(L)2GSerial 40 / N/A

1 HostFC(L)N/ASerial N/A / 42

2 DRV+RCC FC(L)2GSerial 14 / 15

3 DRV+RCC FC(L)2GSerial 14 / 15

4 HostFC(L)2GSerial 44 / N/A

5 HostFC(L)N/ASerial N/A / 46 <<-- odd ?

6 HostLANN/ASerial N/A / N/A

* lun-maps

Ch Tgt LUNld/lvID-Partition Assigned Filter Map

-

0 400ld02174E566-00Primary

0 401ld132931025-00Primary

0 402ld23C6918B3-00Primary

* port-WWNs

Ch IdWWPN

-

0 40216000C0FF8A7EF8

1 42226000C0FFAA7EF8

4 44256000C0FFCA7EF8

5 46266000C0FFEA7EF8

* host-wwn-names

sccli: No host WWN names were found.

[5681 byte] By [htsgurua] at [2007-11-26 17:40:30]
# 1

Connect to channel 4 on the 3510, per your notes below Channels 0 and 4 are assigned to the primary controller (where your luns are) and channels 1 and 5 are assigned to the secondary controller, which has nothing associated with it.

On the 3510s, the IO ports are located on the controllers, but are not tied directly to the controllers. Ports 0 on both the top and bottom controller are electrically connetced, same for the other ports as well.

domburnsa at 2007-7-9 0:08:40 > top of Java-index,Storage Forums,Storage General Discussion...
# 2

Appreciate response...

Okay. I see what you mean about controller 'owned' channels now.

I moved fiber to chan 4 on bottom controller.

Touched /reconfigure - rebooted - reset controller.

No joy. Same blinking lights (except from chan 4 now.)

2 green lights below SPF both blink together.

Did...

# mpathadm list initiator-port

Initiator Port: iqn.1986-03.com.sun:01:00144f2b2a7a.459fd2bc,4000002a00ff

Initiator Port: 210100e08ba9bd77

Initiator Port: 210000e08b89bd77

# mpathadm list lu

/dev/rdsk/c4t600C0FF0000000000A7EF83C6918B300d0s2

Total Path Count: 1

Operational Path Count: 1

/dev/rdsk/c4t600C0FF0000000000A7EF83293102500d0s2

Total Path Count: 1

Operational Path Count: 1

/dev/rdsk/c4t600C0FF0000000000A7EF82174E56600d0s2

Total Path Count: 1

Operational Path Count: 1

Path count is still 1.

Any ideas? I'm stumped.

htsgurua at 2007-7-9 0:08:40 > top of Java-index,Storage Forums,Storage General Discussion...
# 3

Hope you don't mind me chipping in...

Did you map the luns to the channel? ie port 4.

I guess you did this for channel 0 , hence you can see the disks from one fbre only. I can only think thats all thats wrong.

Just moving the fibre cable to port 4 is not good enough on its own.

Hope this helps :)

Cheers

Garyg

northgate-isa at 2007-7-9 0:08:40 > top of Java-index,Storage Forums,Storage General Discussion...
# 4

Thank you for advice!

You were correct, I did need to assign LUN's to port 4. Once I did

the controller port 4 green lights stayed green steady.

However, I 'still' have flashing lights on port 2 of HBA...

I turned off multipathing and did reconfigure reboot - still

no joy.

I'm guessing - somehow, port 2 on HBA isn't active.

To test port 4 of the controller...

- took brand new SG-XPCI1FC-QF4 HBA and

plugged it into a spare V210.

- plugged fiber from new card into chan 4 lower controller

- rebooted (reconfigure)

- got green lights immediately, both HBA and port 4

- format shows 3 drives just fine.

I believe this narrows the problem to port 2 of the dual HBA but,

not sure why. I can't run qlctest, because it's part of SUNWvts.

Found on Qlogic site, drivers for Sun Solaris 10. Not sure what to do?

I can't tell if this is a config problem -or- a driver problem.

Prefer to use Sun drivers, whenever possible, so I'm hesitant to put

Qlogic drivers on my brand new T2000.

VERY cruious if anybody has successfully used the 4gb dual HBA on

the T2000.

Truely appreciate any ideas anyone has!! Thanks!

htsgurua at 2007-7-9 0:08:40 > top of Java-index,Storage Forums,Storage General Discussion...
# 5

I am just curious, with both fibres in the Dual HBA ie port 0 and 4.

What does the OBP see?

I would guess you must see all luns on each HBA interface.

If you do, then it must be a driver/software issue, am guessing here.

However if you don't see the luns on the 2nd port at OBP then it must be a hardware error, I would then in this case swap the HBA.

I have only used single 4GB cards so far, with Solaris 10 and Solaris 9 9/05 update 9 with SFS 4.4.11

Solaris 9 needed SFS 4.4.11 for the 4GB cards and 9 update 9 for v245 hardware.

Cheers

Garyg

northgate-isa at 2007-7-9 0:08:40 > top of Java-index,Storage Forums,Storage General Discussion...
# 6
Just need to add that with Solaris 9 I had to use a PCIX card not PCIE as this does not work on 9.Sorry.Garyg
northgate-isa at 2007-7-9 0:08:40 > top of Java-index,Storage Forums,Storage General Discussion...
# 7

problem starting to look like harware or firmware...

installed patch 124750-02 T2000 firmware update

installed patch 123305-02 Qlogic firmware update

after updates: port 1 works fine, port 2 still flashing lights

as long as fiber port lights flash - there is no communication occuring

so, there are no LD's, LUN's to report even at OBP.

fiber cable is proven good, controller port is proven good...

guessing it's a bad HBA GBIC port, can't explain it any other way.

turned on extended-logging=1; in qlc.conf

/var/adm/messages reports a problem on port 2 during boot...

qlc(0): ql_check_isp_firmware: Load RISC code

qlc(0): ql_fw_ready: mailbox_reg[1] = 4h

ql_async_event, 8013h LIP received

etc...

NOTICE: Qlogic qlc(0): Loop ONLINE

NOTICE: qlc(0): Firmware version 4.0.23

qlc(1): ql_check_isp_firmware: Load RISC code

qlc(1): ql_fw_ready: mailbox_reg[1] = 4h

NOTICE: Qlogic qlc(1): Loop OFFLINE

NOTICE: qlc(1): Firmware version 4.0.23

qlc(1): ql_fw_ready: failed = 100h

htsgurua at 2007-7-9 0:08:40 > top of Java-index,Storage Forums,Storage General Discussion...
# 8
Final check would be to swap cables between the two hba ports, but I'd look to swap out the hba.... good luck!
domburnsa at 2007-7-9 0:08:40 > top of Java-index,Storage Forums,Storage General Discussion...
# 9

When fiber cable is swapped between ports

- hba port 1 still works (sees LD's fine)

- hba port 2 doesn't (still flashing).

Opened Sun ticket, they agree, bad card. Very rare, just lucky I guess :-)

But, wont' know for sure until swapped out...

Appreciate everyones help.

htsgurua at 2007-7-9 0:08:40 > top of Java-index,Storage Forums,Storage General Discussion...