# [OmniOS-discuss] Trouble getting Supermicro board to see all its PCI-e slots

Sašo Kiselkov skiselkov.ml at gmail.com
Tue Apr 30 06:16:57 EDT 2013

- Previous message: [OmniOS-discuss] Problem with passed-through HBA
- Next message: [OmniOS-discuss] flash to IT
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

    I've got a bunch of new Supermicro X9DRW-7TPF+ boards, with on-board
    10-gigy sweetness by Intel. The motherboard is loaded with two Intel
    E5-2630 CPUs with 128GB of RAM. The boards come in the proprietary WIO
    form factor from Supermicro, so to get to the PCI-e slots, I'm using the
    appropriate riser cards, to get to them. The risers are:

    (looking from the front of the server)
    Left-hand side:	 RSC-R2UW-4E8
                     four x8 PCI-e slots (two to CPU1, two to CPU2)
    Right-hand side: RSC-R2UW-2E8R
                     two x8 PCI-e slots (two to CPU2)

    However, and here's the kicker, I can't get OmniOS bloody to see
    anything plugged into any slot other than those that lead to CPU1 (I'm
    trying a pair of LSI SAS HBAs). And yet the BIOS sees everything just
    fine and I can access the cards, the boot ROMs get installed properly
    and I can see all of the attached disks. Allocation of device data above
    4G is disabled in the BIOS. The "lspci" utility from
    pkg:/system/pciutils doesn't see them either.

    Strangely, however, prtdiag sees the slots as populated:

    # prtdiag -v
    ...
    ==== Upgradeable Slots ====================================

    ID  Status    Type             Description
    --- --------- ---------------- ----------------------------
    1   available Unknown          CPU1 Slot1
    2   available Unknown          CPU1 Slot2
    3   in use    Unknown          CPU2 Slot3
    4   in use    Unknown          CPU2 Slot4
    5   available Unknown          CPU2 Slot5
    6   available Unknown          CPU2 Slot6

    (the cards work in slots 1 & 2, but nowhere else)

    Motherboard manual:
    http://www.supermicro.com/manuals/motherboard/C606_602/MNL-1420.pdf

    Can anybody help please? I'm seriously out of ideas here...

    Here's the dmesg on anything PCI related:

    # dmesg  | grep -i pci
    Apr 30 05:12:32 head1 npe: [ID 236367 kern.info] PCI Express-device:
    pci8086,3c04 at 2, pcieb1
    Apr 30 05:12:32 head1 genunix: [ID 936769 kern.info] pcieb1 is
    /pci at 0,0/pci8086,3c04 at 2
    Apr 30 05:12:32 head1 npe: [ID 236367 kern.info] PCI Express-device:
    pci8086,3c06 at 2,2, pcieb2
    Apr 30 05:12:32 head1 genunix: [ID 936769 kern.info] pcieb2 is
    /pci at 0,0/pci8086,3c06 at 2,2
    Apr 30 05:12:32 head1 npe: [ID 236367 kern.info] PCI Express-device:
    pci8086,3c07 at 2,3, pcieb3
    Apr 30 05:12:32 head1 genunix: [ID 936769 kern.info] pcieb3 is
    /pci at 0,0/pci8086,3c07 at 2,3
    Apr 30 05:12:32 head1 npe: [ID 236367 kern.info] PCI Express-device:
    pci8086,1d3e at 11, pcieb6
    Apr 30 05:12:32 head1 genunix: [ID 936769 kern.info] pcieb6 is
    /pci at 0,0/pci8086,1d3e at 11
    Apr 30 05:12:32 head1 npe: [ID 236367 kern.info] PCI Express-device:
    pci8086,244e at 1e, pci_pci0
    Apr 30 05:12:32 head1 genunix: [ID 936769 kern.info] pci_pci0 is
    /pci at 0,0/pci8086,244e at 1e
    Apr 30 05:12:32 head1 pcieb: [ID 586369 kern.info] PCIE-device:
    pci10e3,8113 at 0, pcieb7
    Apr 30 05:12:32 head1 npe: [ID 236367 kern.info] PCI Express-device:
    pci10e3,8113 at 0, pcieb7
    Apr 30 05:12:32 head1 genunix: [ID 936769 kern.info] pcieb7 is
    /pci at 0,0/pci8086,3c06 at 2,2/pci10e3,8113 at 0
    Apr 30 05:12:33 head1 npe: [ID 236367 kern.info] PCI Express-device:
    pci15d9,720 at 1a, ehci0
    Apr 30 05:12:33 head1 genunix: [ID 936769 kern.info] ehci0 is
    /pci at 0,0/pci15d9,720 at 1a
    Apr 30 05:12:34 head1 npe: [ID 236367 kern.info] PCI Express-device:
    pci15d9,720 at 1d, ehci1
    Apr 30 05:12:34 head1 genunix: [ID 936769 kern.info] ehci1 is
    /pci at 0,0/pci15d9,720 at 1d
    Apr 30 05:12:34 head1 genunix: [ID 936769 kern.info] mouse80420 is
    /pci at 0,0/isa at 1f/i8042 at 1,60/mouse at 1
    Apr 30 05:12:37 head1 genunix: [ID 936769 kern.info] hubd0 is
    /pci at 0,0/pci15d9,720 at 1a/hub at 1
    Apr 30 05:12:37 head1 genunix: [ID 408114 kern.info]
    /pci at 0,0/pci15d9,720 at 1a/hub at 1 (hubd0) online
    Apr 30 05:12:37 head1 genunix: [ID 936769 kern.info] hubd1 is
    /pci at 0,0/pci15d9,720 at 1d/hub at 1
    Apr 30 05:12:37 head1 genunix: [ID 408114 kern.info]
    /pci at 0,0/pci15d9,720 at 1d/hub at 1 (hubd1) online
    Apr 30 05:12:38 head1 genunix: [ID 936769 kern.info] usb_mid0 is
    /pci at 0,0/pci15d9,720 at 1a/hub at 1/device at 3
    Apr 30 05:12:38 head1 genunix: [ID 408114 kern.info]
    /pci at 0,0/pci15d9,720 at 1a/hub at 1/device at 3 (usb_mid0) online
    Apr 30 05:12:38 head1 genunix: [ID 936769 kern.info] hid0 is
    /pci at 0,0/pci15d9,720 at 1a/hub at 1/device at 3/mouse at 0
    Apr 30 05:12:38 head1 genunix: [ID 408114 kern.info]
    /pci at 0,0/pci15d9,720 at 1a/hub at 1/device at 3/mouse at 0 (hid0) online
    Apr 30 05:12:38 head1 genunix: [ID 936769 kern.info] hid1 is
    /pci at 0,0/pci15d9,720 at 1a/hub at 1/device at 3/keyboard at 1
    Apr 30 05:12:38 head1 genunix: [ID 408114 kern.info]
    /pci at 0,0/pci15d9,720 at 1a/hub at 1/device at 3/keyboard at 1 (hid1) online
    Apr 30 05:12:41 head1 genunix: [ID 936769 kern.info] asy0 is
    /pci at 0,0/isa at 1f/asy at 1,3f8
    Apr 30 05:12:41 head1 genunix: [ID 936769 kern.info] asy1 is
    /pci at 0,0/isa at 1f/asy at 1,2f8

    --
    Saso

- Previous message: [OmniOS-discuss] Problem with passed-through HBA
- Next message: [OmniOS-discuss] flash to IT
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

More information about the OmniOS-discuss mailing list