How to install StorEdge 3510FC arrary?
Hi experts,
My commpany purchased a StorEdge 3510FC and one expansion unit, equiped with a SunFire V490 server with Solaris 10 3/05. I just make DAS connections between server and 3510FC.
When it powered on, I checked the result and I found there is nothing else by using cfgadm.
root@walden # cfgadm -al
Ap_Id Type ReceptacleOccupantCondition
c0scsi-busconnectedconfiguredunknown
c0::dsk/c0t0d0CD-ROMconnectedconfiguredunknown
c1fc-privateconnectedconfiguredunknown
c1::21000014c3713f26disk connectedconfiguredunknown
c1::21000014c37156a1disk connectedconfiguredunknown
c2fc-privateconnectedconfiguredunknown
c2::226000c0ffa9cadadisk connectedconfiguredunknown
c3fcconnectedunconfigured unknown
c4fc-privateconnectedconfiguredunknown
c4::216000c0ff89cadadisk connectedconfiguredunknown
c5fcconnectedunconfigured unknown
usb0/1 unknownemptyunconfigured ok
usb0/2 unknownemptyunconfigured ok
usb0/3 unknownemptyunconfigured ok
usb0/4 unknownemptyunconfigured ok
I am just wondering each 3510FC box with 12 disks. However, I just see two disks on the above list. Is there anything wrong on my 3510FC?
Thanks
Jeff.
[1253 byte] By [
Jeff_C] at [2007-11-26 9:36:54]
# 1
The important part is:
c2 fc-private connected configured unknown
c2::226000c0ffa9cada disk connected configured unknown
c4 fc-private connected configured unknown
c4::216000c0ff89cada disk connected configured unknown
Basically, there are two ports on the 3510 that have fibre connections to your 490. On the 3510 is one logical drive (LD) and this has been exported as two luns to your host (one down each channel or port). A LD is a raid group, most probably RAID 5.
You mentioned an expansion unit. I guess this is a JBOD attached to your 3510. Whether or not the JBOD disks are part of the LD needs to be determined.
Without going into details, you can see your disks but they belong to LD's. It is up to you to configure what you require on the 3510.
If you have 22 available (2 global spares not used) 300 GB drives, doing a format on those two luns will probably produce 3.2 TB each.
Enjoy!!
Stephen
# 2
It is interesting. From format command, I just found a 173G disk. Where 's the other ones? I have 24 173Gb disks, haven't I?Any idea?
# 3
I reconfigured our 3510FC & expansion unit. Physcial connections are listed as below.
FC0, FC5 to V490.
FC2,FC3 to expansion unit.
Meanwhile, I have reconfigured the all disks as two LDs and partitioned them as four slices. However, after rebooted the machine, I cannot find the disks from format command. I suppose it is special step that I have missed in Solaris 10.
# 4
When you say partitioned them, exactly what are you doing?Have you partitioned the two LD's by slicing them up as LUNs and making them available to the hosts?CheersStephen
# 5
Stephen,
Exactly like what you said. Partitoned LD since each LUN only support 2TB disk spaces.
Since I configured two machines attached with 3510FC. Personal, I configured two LUNs on the primary and secondary controller respectively. One V440 server consider as a primary server which can see two disks, another V440 server is working as a standy server. In case one of controller is broken, you suppose I can use sccli command to unmap the LUNs and move connections to the standy server which use alternative paths to primary & secondary controller. This is my thought. However, I still found there is a problem when I unmaped the LUN device and it was still there by using format command.
In general, what do you do in the active/standy environment?
Cheers
Jeff
# 6
Just to confirm this..
You have a 3510 with two controllers and a JBOD attached. You want to share the two hosts on the one 3510. I suppose you have two fibre connections to each of the hosts. If that is right, then you don't have to really worry about a controller failing as you would have redundant paths to each host.
When you unmap a lun you probably need to do a cfgadm -c unconfigure <the lun> to let Solaris know it is no longer there. Then just for fun do a devfsadm -v -C to clean up the device path.
One of these days, I will figure out what the actual disk size is allowed on Solaris 10. I know that it has been over 1 TB for sometime now but only with UFS (and probably ZFS). Veratis needs 4.1 and Solaris 10 to go above 1 TB.
# 7
Stephen,
One more question.
When I run the command " cfgadm -c unconfigure", I always get the error message.
Please refer to the following output
root@panda # cfgadm -al
Ap_Id Type ReceptacleOccupantCondition
c0scsi-busconnectedconfiguredunknown
c0::dsk/c0t0d0CD-ROMconnectedconfiguredunknown
c1scsi-busconnectedconfiguredunknown
c1::dsk/c1t0d0disk connectedconfiguredunknown
c1::dsk/c1t1d0disk connectedconfiguredunknown
c1::dsk/c1t2d0disk connectedconfiguredunknown
c1::dsk/c1t3d0disk connectedconfiguredunknown
c2scsi-busconnectedunconfigured unknown
c3fc-privateconnectedconfiguredunknown
c3::266000c0ffe99e73ESI connectedconfiguredunknown
c4fc-privateconnectedconfiguredunknown
c4::256000c0ffc99e73ESI connectedconfiguredunknown
usb0/1 unknownemptyunconfigured ok
usb0/2 unknownemptyunconfigured ok
usb1/1 unknownemptyunconfigured ok
usb1/2 unknownemptyunconfigured ok
root@panda # cfgadm -c unconfigure c3::266000c0ffe99e73
cfgadm: Configuration operation not supported
root@panda # cfgadm -c unconfigure c3
cfgadm: Configuration operation not supported
As I know, both active LUNs are only available on another server. However, on panda, there is still a old LUN device exist. It cannot be deleted.
Any idea? Stephen.
Cheers
Jeff
# 8
I recall seeing that before and I think it was because the system thought the luns were still being used by the system or it was just unsure of what was happening. I saw it with Veritas on a really complex system which was similiar to what you are doing in that a 3510 was sharing with two hosts.
There is a force option in cfgadm.The error you are seeing is just a sort of failsafe option to make you sure you know what you are doing.
If you are absolutely sure the luns are not being used, do a:
cfgadm -f -c unconfigure ....
then do a:
devfsadm -vC
and see what spits out.
The host that gave me the problem was ok once I cleaned up the device trees and fixed Veritas' knowledge of the long gone LUN's. I think I might have had to reboot the system. It was about a year ago now. Once I started using switches between my storage and systems, life got a lot easier.
Same thing might happen if using any sort of volume manager.
Stephen
# 9
Stephen,
"cfgadm -f" still does not work. I rebooted the machine, the invalid LUN disapeared. I suppose this is because I use CLI command on the another server. It just look likes out-of-band management and the device was removed abruptly. In my opinion, Solaris cannot identify this problem and leave this invalid LUN. I suppose I can use cfgadm to unconfigure LUNs on this machine then execute CLI command to get rid of two LUNs. Then, on another server, execute CLI command to reestablish two LUNs. It should be working fine.
Problem is that cfgadm command does not seems to support the fibre channels. I are looking into it.
Anyway, I am learning a lot, before that, I knew nothing about Sun DAS.
Jeff
# 10
Hi Jeff,
cfgadm is definitely your friend for all sorts of dynamic reconfiguration (DR) issues. Perhaps there is a set procedures to do what you have done. I have moved lots of storage over the last couple of years and never had any problems other than the one I mentioned earlier which was to do with sharing a 3510. I have lots of confidence in cfgadm as I was Sun's DR champion in my region on this.
I would look at using cfgadm to remove the luns, then unmap the luns to remove the storage before reassigning new luns to new hosts.
Good luck with your future endeavours and I hope you have learnt something from all this.
Regards
Stephen
