# [OmniOS-discuss] Low latency networking

Richard Elling richard.elling at richardelling.com
Sat Aug 4 23:06:18 EDT 2012

- Previous message: [OmniOS-discuss] Low latency networking
- Next message: [OmniOS-discuss] Low latency networking
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

    Hi Chris,
    Check all cable ends for errors. In an extreme case, we sent an Engineer to Germany
    to solve a "performance problem" that turned out to be a bad SFP+ transceiver. The bill
    was, shall we say, higher than the average service call :-)
     -- richard

    On Aug 4, 2012, at 4:41 PM, Chris Nagele wrote:

    > I've been testing an environment before we send it over to the data
    > center. The setup is an OmniOS install (Supermicro X9, 24 drives,
    > DDRdrive for ZIL) used as an NFS file server and multiple Debian
    > machines connected via directly attached 10GB SFP+ twin-axial cables.
    > We went with the 10Gb in order to get lower latency to compensate for
    > the high TCP overhead of NFS.
    >
    > For some reason, I can't get ping times below 0.2ms when using OmniOS.
    > If I install Debian on the same machines I get consistent 0.08ms
    > between the machines. In addition, iperf shows a maximum of 6.6Gb/s on
    > OmniOS, and about 9.89Gb/s on Debian.
    >
    > We are using the x520-DA2, which is well supported in Illumos. I also
    > tried on Nexenta and OpenIndiana with the same result. I was thinking
    > of trying different cards (Myricom maybe), different cabling (CX4?),
    > or even Infiniband since it is known for low latency.
    >
    > Any recommendations? I am wondering if I just need to adjust some
    > tunables or maybe update a driver.
    >
    > Thanks,
    > Chris
    > _______________________________________________
    > OmniOS-discuss mailing list
    > OmniOS-discuss at lists.omniti.com
    > http://lists.omniti.com/mailman/listinfo/omnios-discuss

    --
    ZFS Performance and Training
    Richard.Elling at RichardElling.com
    +1-760-896-4422

    -------------- next part --------------
    An HTML attachment was scrubbed...
    URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20120804/23423944/attachment.html>

- Previous message: [OmniOS-discuss] Low latency networking
- Next message: [OmniOS-discuss] Low latency networking
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

More information about the OmniOS-discuss mailing list