# [OmniOS-discuss] multipathing against compellent

Majeed mabuqu at ilstu.edu
Thu Oct 18 16:45:15 EDT 2012

- Previous message: [OmniOS-discuss] multipathing against compellent
- Next message: [OmniOS-discuss] hostid missing or corrupt on /etc/hostid
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

    Sorry for the spam everyone, wrong list :(

    On 10/18/2012 03:35 PM, Majeed wrote:
    > Hi All, As noted below, we have attempted to test the illumos based
    > omniOS in our datacenter on a dell R610 connected through dual cisco
    > fiber channel fabrics to a compellent SAN with dual controllers. We
    > got pretty far with an out of the box setup until the last
    > multipathing bit. I was hoping there might be some additional eyes on
    > this list that may be able to provide some insight.
    >
    > Thanks in advance for taking the time to read through my following
    > notes on this topic,
    > Majeed Qulbain
    >
    >
    >
    >
    > On 10/18/2012 03:06 PM, Majeed wrote:
    >> Thanks Theo, that makes sense. I will definitely do that.
    >>
    >> Majeed
    >>
    >> On 10/18/2012 02:58 PM, Theo Schlossnagle wrote:
    >>> This is a more general Illumos problem, you will open your
    >>> troubleshooting to a larger audience by interfacing with the
    >>> illumos-discuss list.  I encourage you to post there! I'm subscribed
    >>> there as well, but I haven't done multipathing against compellent and
    >>> it appears no one else on the omnios-discuss list has either.
    >>> Anything learned in the illumos-discuss group can be applied directly
    >>> to OmniOS.
    >>>
    >>> On Thu, Oct 18, 2012 at 3:51 PM, Majeed<mabuqu at ilstu.edu>  wrote:
    >>>> Hi Again, just wanted to update the list with my latest progress and
    >>>> thoughts on this topic. I think the issue lies in
    >>>> libmpscsi_vhci.so's lack
    >>>> of support for multipathing with compellent.
    >>>>
    >>>> When I do "mpathadm show mpath-support libmpscsi_vhci.so" on omniOS
    >>>> I see
    >>>> many venders and models listed but not compellent. We are going to
    >>>> spin up a
    >>>> solaris 11 machine to see what it looks like. I wonder if this was
    >>>> forked
    >>>> just before SUN/oracle added support for compellent (i could be way
    >>>> off base
    >>>> here).
    >>>>
    >>>> When I do the same command on solaris 10 u10, at the very bottom of
    >>>> the list
    >>>> i see.
    >>>>                  Vendor:  COMPELNT
    >>>>                  Product:  Compellent Vol
    >>>>                  Revision:
    >>>>                  Supported Load Balance Types:
    >>>>                          round-robin
    >>>>
    >>>> I have verified that multipathing is working properly on other
    >>>> arrays. I
    >>>> zoned up omniOS with all controllers on a 6140 (CSM200) that sit in
    >>>> multiple
    >>>> fabrics, mapped a lun, and it looks wonderful. mpatham reports as
    >>>> expected,
    >>>> and all paths are condensed to a single block device in format's
    >>>> output.
    >>>>
    >>>> % mpathadm list LU
    >>>>      /dev/rdsk/c0t600A0B800033816200001391507FF856d0s2
    >>>>          Total Path Count: 4
    >>>>          Operational Path Count: 4
    >>>>
    >>>>
    >>>>
    >>>> It seems there are some options in /kernel/drv/scsi_vhci.conf to force
    >>>> scsi_vhci to accept 3rd party devices (hopefully compellent) for
    >>>> failover,
    >>>> but this is kinda getting over my head. It seems like if I had a
    >>>> proper
    >>>> ddi-forceload ''misc/scsi_vhci/scsi_vhci_f_<x>" file for compellent
    >>>> and I
    >>>> knew the proper VenderID and ProductID for the compellent devices,
    >>>> i might
    >>>> be able to force it. Im not sure if can use the information from
    >>>> the format
    >>>> inquiry to populate these values.
    >>>>
    >>>> format>  inquiry
    >>>> Vendor:   COMPELNT
    >>>> Product:  Compellent Vol
    >>>> Revision: 0505
    >>>>
    >>>>
    >>>> Any thoughts on this would be excellent.
    >>>>
    >>>> Thanks,
    >>>> Majeed Qulbain
    >>>>
    >>>>
    >>>>
    >>>> On 10/18/2012 09:30 AM, Majeed wrote:
    >>>>> I just noticed that we do have access to the mpathadm command by
    >>>>> installing the "pkg:/storage/mpathadm at 0.5.11-0.151002" package. After
    >>>>> installing I don't see any paths with "mpathadm list LU" which is
    >>>>> what im
    >>>>> used to on solaris 10 (i know I should be comparing to solaris 11,
    >>>>> just
    >>>>> havn't played with it much). I wonder if there are any other bits
    >>>>> that I
    >>>>> need to install or other configuration tweaks to get this going.
    >>>>>
    >>>>> Thanks in advance for any pointers,
    >>>>> Majeed
    >>>>>
    >>>>> Some other related commands that might assist troubleshooting:
    >>>>>
    >>>>> % uname -a
    >>>>> SunOS jhx011 5.11 omnios-eae537b i86pc i386 i86pc
    >>>>>
    >>>>> % mpathadm list initiator-port
    >>>>> Initiator Port:  w5d4ae520937bbb00
    >>>>> Initiator Port:
    >>>>> iqn.1986-03.com.sun:01:b06a64f606ff.507f35b3,4000002a00ff
    >>>>> Initiator Port:  w5d4ae520937bbb00
    >>>>> Initiator Port:  2101001b32b40767
    >>>>> Initiator Port:  2100001b32940767
    >>>>>
    >>>>> % luxadm -e port
    >>>>> /devices/pci at 0,0/pci8086,3410 at 9/pci1077,143 at 0/fp at 0,0:devctl
    >>>>> CONNECTED
    >>>>> /devices/pci at 0,0/pci8086,3410 at 9/pci1077,143 at 0,1/fp at 0,0:devctl
    >>>>> CONNECTED
    >>>>>
    >>>>> % luxadm -v probe
    >>>>>
    >>>>> Found Fibre Channel device(s):
    >>>>>    Node WWN:5000d310004a0602  Device Type:Disk device
    >>>>>      Logical Path:/dev/rdsk/c16t5000D310004A064Bd1s2
    >>>>>    Node WWN:5000d310004a0602  Device Type:Disk device
    >>>>>      Logical Path:/dev/rdsk/c17t5000D310004A064Cd1s2
    >>>>>
    >>>>> % luxadm disp /dev/rdsk/c16t5000D310004A064Bd1s2
    >>>>> DEVICE PROPERTIES for disk: /dev/rdsk/c16t5000D310004A064Bd1s2
    >>>>>    Vendor:        COMPELNT
    >>>>>    Product ID:        Compellent Vol
    >>>>>    Revision:        0505
    >>>>>    Serial Num:        00004a06-00000014
    >>>>>    Unformatted capacity:    512000.000 MBytes
    >>>>>    Read Cache:        Enabled
    >>>>>      Minimum prefetch:    0x100
    >>>>>      Maximum prefetch:    0xf0
    >>>>>    Device Type:        Disk device
    >>>>>    Path(s):
    >>>>>
    >>>>>    /dev/rdsk/c16t5000D310004A064Bd1s2
    >>>>>
    >>>>> /devices/pci at 0,0/pci8086,3410 at 9/pci1077,143 at 0/fp at 0,0/disk at w5000d310004a064b,1:c,raw
    >>>>>
    >>>>>     Controller          /dev/cfg/c16
    >>>>>      Device Address        5000d310004a064b,1
    >>>>>      Host controller port WWN    2100001b32940767
    >>>>>     Controller          /dev/cfg/c17
    >>>>>      Device Address        5000d310004a064c,1
    >>>>>      Host controller port WWN    2101001b32b40767
    >>>>>
    >>>>>
    >>>>> On 10/17/2012 06:25 PM, Majeed wrote:
    >>>>>> Hi All, We are doing some testing with omniOS in our datacenter
    >>>>>> and would
    >>>>>> love connect it up to our compellent san via fc connectivity. I
    >>>>>> slapped
    >>>>>> omniOS onto a dell r610 that has a qlogic dual port hba (each
    >>>>>> port is pathed
    >>>>>> to separate fabrics), performed zoning in each fabric, and
    >>>>>> assigned some
    >>>>>> luns. Now im not sure if compellent supports Solaris 11 (i can
    >>>>>> contact them
    >>>>>> and see about that), but it definately support solaris 10 which
    >>>>>> is what we
    >>>>>> use on other production systems.
    >>>>>>
    >>>>>> I'v actually made it pretty far through the process which is really
    >>>>>> exciting, except for the last multipathing bit. After mapping a
    >>>>>> volume to
    >>>>>> the host I can't seem to condense all the devices down to a single
    >>>>>> mulipathed device. It seems multipathing is enabled for the fp
    >>>>>> driver. I
    >>>>>> noticed there is not a mpathadm command.
    >>>>>>
    >>>>>> Anyone have any ideas? I'll post some output from the commands i
    >>>>>> think of
    >>>>>> below. Let me know if you would like to see any other information.
    >>>>>>
    >>>>>> Thanks much,
    >>>>>> Majeed
    >>>>>>
    >>>>>>
    >>>>>> On solaris 10 the main thing we have to do is add the following to
    >>>>>> /kernel/drv/scsi_vhci.conf (reconfigure reboot)
    >>>>>> device-type-scsi-options-list =
    >>>>>> "COMPELNTCompellent Vol", "symmetric-option";
    >>>>>> symmetric-option = 0x1000000;
    >>>>>>
    >>>>>>
    >>>>>> Here is the format output:
    >>>>>> AVAILABLE DISK SELECTIONS:
    >>>>>>         0. c13t5000CCA0213ED1E1d0<HITACHI-HUC106030CSS600-A360
    >>>>>> cyl 36470
    >>>>>> alt 2 hd 255 sec 63>
    >>>>>>
    >>>>>> /pci at 0,0/pci8086,2940 at 1c/pci1028,1f1e at 0/iport at 40/disk at w5000cca0213ed1e1,0
    >>>>>>
    >>>>>>         1. c14t5000CCA0213EFC7Dd0<HITACHI-HUC106030CSS600-A360
    >>>>>> cyl 36469
    >>>>>> alt 2 hd 255 sec 63>
    >>>>>>
    >>>>>> /pci at 0,0/pci8086,2940 at 1c/pci1028,1f1e at 0/iport at 80/disk at w5000cca0213efc7d,0
    >>>>>>
    >>>>>>         2. c16t5000D310004A064Bd1<COMPELNT-CompellentVol-0505 cyl
    >>>>>> 65268
    >>>>>> alt 2 hd 255 sec 63>
    >>>>>>
    >>>>>> /pci at 0,0/pci8086,3410 at 9/pci1077,143 at 0/fp at 0,0/disk at w5000d310004a064b,1
    >>>>>>
    >>>>>>         3. c17t5000D310004A064Cd1<COMPELNT-CompellentVol-0505 cyl
    >>>>>> 65268
    >>>>>> alt 2 hd 255 sec 63>
    >>>>>>
    >>>>>> /pci at 0,0/pci8086,3410 at 9/pci1077,143 at 0,1/fp at 0,0/disk at w5000d310004a064c,1
    >>>>>>
    >>>>>> Specify disk (enter its number): Specify disk (enter its number):
    >>>>>>
    >>>>>>
    >>>>>>
    >>>>>> % fcinfo hba-port
    >>>>>> HBA Port WWN: 2100001b32940767
    >>>>>>      Port Mode: Initiator
    >>>>>>      Port ID: 91c00
    >>>>>>      OS Device Name: /dev/cfg/c16
    >>>>>>      Manufacturer: QLogic Corp.
    >>>>>>      Model: 375-3356-02
    >>>>>>      Firmware Version: 05.03.01
    >>>>>>      FCode/BIOS Version:  BIOS: 2.02; fcode: 2.01; EFI: 2.00;
    >>>>>>      Serial Number: 0402H00-0941774287
    >>>>>>      Driver Name: qlc
    >>>>>>      Driver Version: 20100408-3.01
    >>>>>>      Type: N-port
    >>>>>>      State: online
    >>>>>>      Supported Speeds: 1Gb 2Gb 4Gb
    >>>>>>      Current Speed: 4Gb
    >>>>>>      Node WWN: 2000001b32940767
    >>>>>>      Max NPIV Ports: 127
    >>>>>>      NPIV port list:
    >>>>>> HBA Port WWN: 2101001b32b40767
    >>>>>>      Port Mode: Initiator
    >>>>>>      Port ID: a1900
    >>>>>>      OS Device Name: /dev/cfg/c17
    >>>>>>      Manufacturer: QLogic Corp.
    >>>>>>      Model: 375-3356-02
    >>>>>>      Firmware Version: 05.03.01
    >>>>>>      FCode/BIOS Version:  BIOS: 2.02; fcode: 2.01; EFI: 2.00;
    >>>>>>      Serial Number: 0402H00-0941774287
    >>>>>>      Driver Name: qlc
    >>>>>>      Driver Version: 20100408-3.01
    >>>>>>      Type: N-port
    >>>>>>      State: online
    >>>>>>      Supported Speeds: 1Gb 2Gb 4Gb
    >>>>>>      Current Speed: 4Gb
    >>>>>>      Node WWN: 2001001b32b40767
    >>>>>>      Max NPIV Ports: 127
    >>>>>>      NPIV port list:
    >>>>>>
    >>>>>>
    >>>>>> % cfgadm -al -o show_SCSI_LUN
    >>>>>> Ap_Id                          Type         Receptacle   Occupant
    >>>>>> Condition
    >>>>>> c16                            fc-fabric    connected    configured
    >>>>>> unknown
    >>>>>> c16::5000d310004a0649          unknown      connected
    >>>>>> unconfigured
    >>>>>> unknown
    >>>>>> c16::5000d310004a064b,1        disk         connected    configured
    >>>>>> unknown
    >>>>>> c17                            fc-fabric    connected    configured
    >>>>>> unknown
    >>>>>> c17::5000d310004a064a          unknown      connected
    >>>>>> unconfigured
    >>>>>> unknown
    >>>>>> c17::5000d310004a064c,1        disk         connected    configured
    >>>>>> unknown
    >>>>>>
    >>>>>>
    >>>>>> _______________________________________________
    >
    > _______________________________________________
    > OmniOS-discuss mailing list
    > OmniOS-discuss at lists.omniti.com
    > http://lists.omniti.com/mailman/listinfo/omnios-discuss

    --
    Majeed Abu-Qulbain
    Lead Unix Admin - CISS
    309-438-3455
    Illinois State University

- Previous message: [OmniOS-discuss] multipathing against compellent
- Next message: [OmniOS-discuss] hostid missing or corrupt on /etc/hostid
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

More information about the OmniOS-discuss mailing list