`<html>
<head>
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">Hi Dale,<br>
<br>
Am 17.05.16 um 20:55 schrieb Dale Ghent:<br>
</div>
<blockquote
cite="mid:DDDC4D69-75C1-4B13-9DE8-EF20D8074632@omniti.com"
type="cite">
<pre wrap="">On May 17, 2016, at 8:30 AM, Stephan Budach <a class="moz-txt-link-rfc2396E" href="mailto:stephan.budach@JVM.DE"><stephan.budach@JVM.DE></a> wrote:
</pre>
<blockquote type="cite">
<pre wrap="">I have checked all of my ixgbe interfaces and they all report that now flow controll is in place, as you can see:
root@zfsha01colt:/root# dladm show-linkprop -p flowctrl ixgbe0
LINK PROPERTY PERM VALUE DEFAULT POSSIBLE
ixgbe0 flowctrl rw no no no,tx,rx,bi
root@zfsha01colt:/root# dladm show-linkprop -p flowctrl ixgbe1
LINK PROPERTY PERM VALUE DEFAULT POSSIBLE
ixgbe1 flowctrl rw no no no,tx,rx,bi
root@zfsha01colt:/root# dladm show-linkprop -p flowctrl ixgbe2
LINK PROPERTY PERM VALUE DEFAULT POSSIBLE
ixgbe2 flowctrl rw no no no,tx,rx,bi
root@zfsha01colt:/root# dladm show-linkprop -p flowctrl ixgbe3
LINK PROPERTY PERM VALUE DEFAULT POSSIBLE
ixgbe3 flowctrl rw no no no,tx,rx,bi
I then checked the ports on the Nexus switches and found out, that they do have outbound-flowcontrol enabled, but that is the case on any of those Nexus ports, including those, where this issue doesn't exist.
</pre>
</blockquote>
<pre wrap="">
Optimally you would have flow control turned off on both sides, as the switch still expects the ixgbe NIC to respond appropriately. To be honest, the only time to use ethernet flow control is if you are operating the interfaces for higher-level protocols which do not provide any sort of direct flow control themselves, such as FCoE. If the vast majority of traffic is TCP, leave it to the TCP stack to manage any local congestion on the link.
/dale
</pre>
</blockquote>
I just wanted to wrap this up… I recently swapped that old Sun
server with a new Supermicro X10-type, which has 4 10 GbE NICs on
board, installed OmniOS r018 and my RSF-1 cluster software on it.
Configured my two LACP aggregations and there hasn't been any issue
since.<br>
So, either it's something on the old server - it's a Sun Fire
X4170M2 - or something on the Intel cards.<br>
<br>
Cheers,<br>
Stephan<br>
<pre class="moz-signature" cols="72">
</pre>
</body>
</html>
`