`<div dir="ltr">We're running a mixed system with connectX 1 and 2, and IB only cards work great with the hermon driver. We haven't had any luck with CX3, but hear that it is possible. Performance has been quite good, and we haven't needed to worry about it so far (knock on wood)<div>
<br></div><div>I did a bunch of digging to figure out the OFED front. Basically:</div><div> - Sun did the port themselves</div><div> - they wrote native drivers that provide a different API than the linux drivers</div><div>
- they took their ball and went home</div><div><br></div><div>Mellanox has only had indirect involvement in this, and afaict, there has never been any open activity on this. </div><div><br></div><div>Syoyo Fujita did a forward port of the last public release of the userland source bits (from <a href="http://sunfreeware.com">sunfreeware.com</a>, iirc), and there is a binary package of the userland bits that works fine for omnios (at least for us). I haven't managed to get a full build working myself. This bits are here:</div>
<div> - <a href="http://github.com/syoyo/solaris-infiniband-tools">github.com/syoyo/solaris-infiniband-tools</a></div><div><br></div><div>Unfortunately, this doesn't do you much good on the CX3 front; the drivers need to be done/reworked for illumos directly. </div>
<div><br></div><div>We're interested in this in order to build high bandwidth servers; we're going to need 60-80 gbit in the next year or so, which means RDMA is the only option. </div><div> -nld</div></div><div class="gmail_extra">
<br><br><div class="gmail_quote">On Wed, Jan 29, 2014 at 12:38 PM, Chris Zembower <span dir="ltr"><<a href="mailto:zembower@criterion.com" target="_blank">zembower@criterion.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr"><br><div>Hello,</div><div><br></div><div>I'm trying to dig up some information on Illumos (or Solaris) compatibility with the Mellanox ConnectX-3 VPI HCA. From what I can gather, the Hermon driver (for 1st-gen ConnectX) may work with this hardware with tweaking, but I can't seem to confirm that.</div>
<div><br></div><div>OmniOS is my server OS of choice (I have several systems running), but the necessity of a working IB fabric here has led me to implement a ZFS on Linux box for this particular purpose. I'm currently using LIO/targetcli to export block SRP targets, and I'm not happy with it at all. Really wishing I had Comstar for this.</div>
<div><br></div><div>Anyone here using IB with OmniOS? Anyone aware of a roadmap or active development of new IB drivers? Why isn't there an Illumos OFED port happening? No interest? It's thriving on the Linux side of the world...</div>
<div><br></div><div>Please don't lecture me on the merits of 10GbE/FC over Infiniband, I've heard it all. :-)</div><div><br></div><div>Any insights or suggestions are very much appreciated!</div><div><br></div><div>
Best,</div><div>Chris</div></div>
<br>_______________________________________________<br>
OmniOS-discuss mailing list<br>
<a href="mailto:OmniOS-discuss@lists.omniti.com">OmniOS-discuss@lists.omniti.com</a><br>
<a href="http://lists.omniti.com/mailman/listinfo/omnios-discuss" target="_blank">http://lists.omniti.com/mailman/listinfo/omnios-discuss</a><br>
<br></blockquote></div><br></div>
`