# [OmniOS-discuss] The ixgbe driver, Lindsay Lohan, and the Greek economy

Chris Siebenmann cks at cs.toronto.edu
Mon Feb 23 16:29:48 UTC 2015

- Previous message: [OmniOS-discuss] The ixgbe driver, Lindsay Lohan, and the Greek economy
- Next message: [OmniOS-discuss] recovering a faulted disk
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

    > Chris, thanks for your specific details. I'd appreciate it if you
    > could tell me which copper NIC you tried, as well as to pass on the
    > iSCSI tuning parameters.

     Our copper NIC experience is with onboard X540-AT2 ports on SuperMicro
    hardware (which have the guaranteed 10-20 msec lock hold) and dual-port
    82599EB TN cards (which have some sort of driver/hardware failure under
    load that eventually leads to 2-second lock holds). I can't recommend
    either with the current driver; we had to revert to 1G networking in
    order to get stable servers.

     The iSCSI parameter modifications we do, across both initiators and
    targets, are:

    	initialr2t		no
    	firstburstlength	128k
    	maxrecvdataseglen	128k		[only on Linux backends]
    	maxxmitdataseglen	128k		[only on Linux backends]

    The OmniOS initiator doesn't need tuning for more than the first two
    parameters; on the Linux backends we tune up all four. My extended
    thoughts on these tuning parameters and why we touch them can be found
    here:

       http://utcc.utoronto.ca/~cks/space/blog/tech/UnderstandingiSCSIProtocol
       http://utcc.utoronto.ca/~cks/space/blog/tech/LikelyISCSITuning

    The short version is that these parameters probably only make a small
    difference but their overall goal is to do 128KB ZFS reads and writes
    in single iSCSI operations (although they will be fragmented at the TCP
    layer) and to do iSCSI writes without a back-and-forth delay between
    initiator and target (that's 'initialr2t no').

     I think basically everyone should use InitialR2T set to no and in
    fact that it should be the software default. These days only unusually
    limited iSCSI targets should need it to be otherwise and they can change
    their setting for it (initiator and target must both agree to it being
    'yes', so either can veto it).

    	- cks

- Previous message: [OmniOS-discuss] The ixgbe driver, Lindsay Lohan, and the Greek economy
- Next message: [OmniOS-discuss] recovering a faulted disk
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

More information about the OmniOS-discuss mailing list