# [OmniOS-discuss] Ang: Re: Ang: Re: infiniband

Greg Mason gmason at msu.edu
Fri Oct 24 18:46:47 UTC 2014

- Previous message: [OmniOS-discuss] Ang: Re: Ang: Re: infiniband
- Next message: [OmniOS-discuss] Ang: Re: Ang: Re: infiniband
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

    Enabling IB support for our ZFS-based filers is important enough to us that we have made the decision to migrate away from OmniOS/Illumos over to FreeBSD. FreeBSD 9 has a partial OFED stack, FreeBSD 10 is further along, and development continues for IB support in FreeBSD. I’ve successfully tested a Mellanox ConnectX-3 card on our FDR fabric on a FreeBSD 10.0-RELEASE system. This was with the subnet manager running on a Mellanox switch (which we regularly upgrade).

    If you’d like more details, please contact me off-list, I’m happy to share :)

    -Greg

    Greg Mason
    Michigan State University
    Institute for Cyber-Enabled Research
    High Performance Computing Center

    web: www.hpcc.msu.edu
    email: gmason at msu.edu

    > On Oct 24, 2014, at 11:35 AM, Ian Kaufman <ikaufman at eng.ucsd.edu> wrote:
    >
    > I wonder if Nexenta has added more/better support for IB? I believe
    > that Illumian 1.0 is finally out (after having run the Alpha for
    > years), and I know that they did have some interest in supporting IB
    > originally. If they have, then I would hope some of that support would
    > make it back upstream.
    >
    > Mayhaps I have an email to send out ...
    >
    > Ian
    >
    > On Fri, Oct 24, 2014 at 7:58 AM, Ian Kaufman <ikaufman at eng.ucsd.edu> wrote:
    >> I use a QLogic 12300 with SM built in. I haven't had any issues,
    >> saving my cluster frontend's cycles for other things.
    >>
    >> Ian
    >>
    >> On Fri, Oct 24, 2014 at 7:55 AM, Michael Rasmussen <mir at miras.org> wrote:
    >>> On Fri, 24 Oct 2014 13:17:39 +0200
    >>> Johan Kragsterman <johan.kragsterman at capvert.se> wrote:
    >>>
    >>>>
    >>>> Well, I can only find one person in that thread that BELIEVES that running SM on a switch is preferable to run OpenSM software. He hardly comes with any documentation of this, and I can't see any other that agrees...
    >>>>
    >>>> But of coarse, if you provide too little resources for OpenSM it will cause problems, or if you have any problems with the S/W or the configuration...
    >>>>
    >>> I think the issue here is that OpenSM is a single threaded application
    >>> so when the CPU which is assigned to it maxes out performance will
    >>> drop. So I guess it comes down to a trade-off between the
    >>> performance of the host CPU and the performance of the switch hardware.
    >>>
    >>> --
    >>> Hilsen/Regards
    >>> Michael Rasmussen
    >>>
    >>> Get my public GnuPG keys:
    >>> michael <at> rasmussen <dot> cc
    >>> http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E
    >>> mir <at> datanom <dot> net
    >>> http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C
    >>> mir <at> miras <dot> org
    >>> http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917
    >>> --------------------------------------------------------------
    >>> /usr/games/fortune -es says:
    >>> The default Magic Word, "Abracadabra", actually is a corruption of the
    >>> Hebrew phrase "ha-Bracha dab'ra" which means "pronounce the blessing".
    >>>
    >>> _______________________________________________
    >>> OmniOS-discuss mailing list
    >>> OmniOS-discuss at lists.omniti.com
    >>> http://lists.omniti.com/mailman/listinfo/omnios-discuss
    >>>
    >>
    >>
    >>
    >> --
    >> Ian Kaufman
    >> Research Systems Administrator
    >> UC San Diego, Jacobs School of Engineering ikaufman AT ucsd DOT edu
    >
    >
    >
    > --
    > Ian Kaufman
    > Research Systems Administrator
    > UC San Diego, Jacobs School of Engineering ikaufman AT ucsd DOT edu
    > _______________________________________________
    > OmniOS-discuss mailing list
    > OmniOS-discuss at lists.omniti.com
    > http://lists.omniti.com/mailman/listinfo/omnios-discuss

    -------------- next part --------------
    An HTML attachment was scrubbed...
    URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141024/c3986dbb/attachment.html>

- Previous message: [OmniOS-discuss] Ang: Re: Ang: Re: infiniband
- Next message: [OmniOS-discuss] Ang: Re: Ang: Re: infiniband
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

More information about the OmniOS-discuss mailing list