From andykinney at advantagecom.net  Sat Oct  1 00:51:19 2016
From: andykinney at advantagecom.net (Andrew Kinney)
Date: Fri, 30 Sep 2016 17:51:19 -0700
Subject: [OmniOS-discuss] smartctl and SCT ERC on omnios
In-Reply-To: <57EEECB2.5030701@advantagecom.net>
References: <57EEECB2.5030701@advantagecom.net>
Message-ID: <57EF0887.6080501@advantagecom.net>

To add to this, it *appears* I can *set* scterc on omnios, but not 
*read* scterc.

# smartctl -l scterc,70,70 -d sat,12 /dev/rdsk/c4t0d0s0
smartctl 6.5 2016-05-07 r4318 [i386-pc-solaris2.11] (local build)
Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org

SCT Error Recovery Control set to:
            Read:     70 (7.0 seconds)
           Write:     70 (7.0 seconds)

# smartctl -l scterc -d sat,12 /dev/rdsk/c4t0d0s0
smartctl 6.5 2016-05-07 r4318 [i386-pc-solaris2.11] (local build)
Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org

SMART WRITE LOG returns COUNT and LBA_LOW register unchanged
SCT (Get) Error Recovery Control command failed


Without being able to read the values, I'm taking it on faith that it is 
actually set since the set command didn't error out.

Regarding the error message:
"The meat of this patch extends the WRITE_LOG command to return the ATA 
registers back to the higher level code. As it was, smartmontools had 
some provision for returning 512 byte sectors as results of ioctl()s, 
but not the contents of the ATA registers. Before ERC appeared, no other 
ATA commands return a result in this way. When a read or write ERC time 
is requested, the current ERC value are returned in SectorCount and 
LBA_Low, as a low and high byte. The number is in deciseconds."

This was from http://abatis.org.uk/projects/erc/ where he discusses how 
smartmontools was patched back in 2009 to get smartctl to read these 
values properly on Linux. It would seem smartctl is getting the values 
back, but on Solaris it doesn't know how to interpret and display the data.


I hooked up one of these SSDs to a Windows machine and was able to read 
and set scterc. From the factory, it comes as "disabled", which explains 
the trouble I had with the long error recovery (2 minutes).

--
Andrew


Andrew Kinney wrote:
> Hello,
>
> I have a Samsung 845DC Pro SSD that I need to respond more timely during
> problems with flash cells. We had an incident where it took 2 minutes to
> fail a cell erase and bring in a spare cell, which is too long for a
> redundant ZFS log device on a pool with a dataset with property
> sync=always serving iSCSI volumes.
>
> smartctl shows it supports SCT ERC:
>
> # smartctl -x -d sat,12 /dev/rdsk/c4t0d0s0
>
> smartctl 6.5 2016-05-07 r4318 [i386-pc-solaris2.11] (local build)
> Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org
>
> === START OF INFORMATION SECTION ===
> Device Model: Samsung SSD 845DC PRO 400GB
> <snip>
> ATA Version is: ACS-2, ATA8-ACS T13/1699-D revision 4c
> SATA Version is: SATA 3.1, 6.0 Gb/s (current: 6.0 Gb/s)
> <snip>
> SCT capabilities: (0x003d) SCT Status supported.
> SCT Error Recovery Control supported.
> SCT Feature Control supported.
> SCT Data Table supported.
> <snip>
>
>
> I should be able to read its current SCT ERC settings, but get an error:
>
> # smartctl -l scterc -d sat,12 /dev/rdsk/c4t0d0s0
> smartctl 6.5 2016-05-07 r4318 [i386-pc-solaris2.11] (local build)
> Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org
>
> SMART WRITE LOG returns COUNT and LBA_LOW register unchanged
> SCT (Get) Error Recovery Control command failed
>
>
> I would go read/write the scterc values on a different OS as a
> workaround, but I've read that Samsung devices do not persist CCTL
> values across reboots. At present, I don't know what it's set to or if
> it's being honored.
>
> It appears that this is a limitation of smartmontools on Solaris and its
> derivatives.
>
> Any ideas how I can read and set the scterc values on Omnios?
>
> --
> Andrew
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>

From danmcd at omniti.com  Sat Oct  1 01:56:32 2016
From: danmcd at omniti.com (Dan McDonald)
Date: Fri, 30 Sep 2016 21:56:32 -0400
Subject: [OmniOS-discuss] extended attributes in LX zone
In-Reply-To: <7DCFD721-221D-42C7-A497-A4A7E323B3FB@lji.org>
References: <4F6BE78F-D4F9-4C7A-B024-F925023B86CF@lji.org>
	<C4180B3F-2964-438C-9828-12EBA0C1AB5E@omniti.com>
	<7DCFD721-221D-42C7-A497-A4A7E323B3FB@lji.org>
Message-ID: <1B0C1A14-8979-4DBD-ACD0-B52D18E23643@omniti.com>


> On Sep 30, 2016, at 6:57 PM, Michael Talbott <mtalbott at lji.org> wrote:
> 
> Bummer. Oh well. How about presenting a zvol as a raw device like shown here: http://docs.oracle.com/cd/E19253-01/819-5461/gbebi/index.html
> 
> I did this, shutdown and rebooted the zone.. and the path the the volume appears under /dev/zvol/dsk/ssdpool/ but that directory is empty when it should contain the name of my zvol :(
> 
> I figure if I can get that working, then I could format that vol in ext4 so it'd have a way of properly handling xattrs (or so I hope).. Otherwise.. I guess I could present a zvol as an iscsi volume to the linux host/zone/entity and then format that ext4 over iscsi, but I'd rather not add the networking stack in there if I don't have to.
> 
> Any ideas on getting zvols as raw devices in an lx zone?

You MIGHT be able to use zonecfg to bring one in.  E.g.

add device
set match=/dev/zvol/dsk/data/oi-data-disk
end

I haven't tested this yet.  That example three extra lines for zonecfg comes from a standard lipkg zone.

ALSO, it may be the case that for an LX zone, such a zvol device entry would appear in /native.

Dan


From daleg at omniti.com  Sat Oct  1 03:52:52 2016
From: daleg at omniti.com (Dale Ghent)
Date: Fri, 30 Sep 2016 23:52:52 -0400
Subject: [OmniOS-discuss] extended attributes in LX zone
In-Reply-To: <7DCFD721-221D-42C7-A497-A4A7E323B3FB@lji.org>
References: <4F6BE78F-D4F9-4C7A-B024-F925023B86CF@lji.org>
	<C4180B3F-2964-438C-9828-12EBA0C1AB5E@omniti.com>
	<7DCFD721-221D-42C7-A497-A4A7E323B3FB@lji.org>
Message-ID: <ECE32D93-C067-4898-893C-E2ECB5496B9C@omniti.com>


> On Sep 30, 2016, at 6:57 PM, Michael Talbott <mtalbott at lji.org> wrote:
> 
> Bummer. Oh well. How about presenting a zvol as a raw device like shown here: http://docs.oracle.com/cd/E19253-01/819-5461/gbebi/index.html
> 
> I did this, shutdown and rebooted the zone.. and the path the the volume appears under /dev/zvol/dsk/ssdpool/ but that directory is empty when it should contain the name of my zvol :(
> 
> I figure if I can get that working, then I could format that vol in ext4 so it'd have a way of properly handling xattrs (or so I hope).. Otherwise.. I guess I could present a zvol as an iscsi volume to the linux host/zone/entity and then format that ext4 over iscsi, but I'd rather not add the networking stack in there if I don't have to.

Negative, won't work because the illumos kernel doesn't grok ext4.

In Linux-land, the get/setxattr calls are made from an app, and those calls are handed in glibc, which turn around and execute the linux kernel syscall for getting and setting xattrs.

This syscalls then run the vfs_get/setxattr VFS layer op that each filesystem in Linux implements if it supports xattrs, which then in turn do the FS-specfic actions in the relevant FS driver to get and set xattrs on the file's on-disk structure.

For xattrs to work with LX brands, the get/setxattr functions in lx_xattr.c would need to be more complete and provide a direct translation layer to ZFS-style xattrs.

/dale

From skiselkov.ml at gmail.com  Sat Oct  1 19:36:23 2016
From: skiselkov.ml at gmail.com (Saso Kiselkov)
Date: Sat, 1 Oct 2016 21:36:23 +0200
Subject: [OmniOS-discuss] rge_intr troubles
Message-ID: <8eb7ec55-f090-6bcd-7e5c-9d7179b079a8@gmail.com>

So I'm playing around with a box that has an on-board Realtek NIC and
periodically, about once every 2-5 minutes, the network just goes out to
lunch and stops responding to ping or attempts to send anything from
the box. I noticed that while doing so, the box is getting floored by
interrupts from the NIC, so I see tons of rge_intr activity and one CPU
core receiving about 160000 interrupts per second (other cores are idle).

I've tried a number of things I've found on the internet, but to no
avail. I tried setting msi_enable=0 and msix_enable=0 in
/kernel/drv/rge.conf, but that didn't help. Disabling flow control via

# ndd -set /dev/rge0 adv_pause_cap 0
# ndd -set /dev/rge0 adv_asym_pause_cap 0

didn't help either. I tried OmniOS r151018, OpenIndiana hipster
(2016.04) and even the old OpenIndiana 151a8 (all 64-bit).

Not being a driver person myself, I have no idea where to look next.
I've attached prtconf -v and lspci -vvvxx output for PCI IDs and config
info. Also attached are screenshots of mpstat and dtrace profile at
997hz running on the CPU that's bogged down with interrupts.

-- 
Saso
-------------- next part --------------
A non-text attachment was scrubbed...
Name: lspci.gz
Type: application/x-gzip
Size: 5507 bytes
Desc: not available
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20161001/cf5cb8a2/attachment-0002.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: prtconf.gz
Type: application/x-gzip
Size: 14324 bytes
Desc: not available
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20161001/cf5cb8a2/attachment-0003.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: dtrace.jpg
Type: image/jpeg
Size: 55022 bytes
Desc: not available
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20161001/cf5cb8a2/attachment-0002.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: mpstat.jpg
Type: image/jpeg
Size: 74331 bytes
Desc: not available
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20161001/cf5cb8a2/attachment-0003.jpg>

From mir at miras.org  Sat Oct  1 21:13:35 2016
From: mir at miras.org (Michael Rasmussen)
Date: Sat, 1 Oct 2016 23:13:35 +0200
Subject: [OmniOS-discuss] rge_intr troubles
In-Reply-To: <8eb7ec55-f090-6bcd-7e5c-9d7179b079a8@gmail.com>
References: <8eb7ec55-f090-6bcd-7e5c-9d7179b079a8@gmail.com>
Message-ID: <20161001231335.03eca0ee@sleipner.datanom.net>

On Sat, 1 Oct 2016 21:36:23 +0200
Saso Kiselkov <skiselkov.ml at gmail.com> wrote:

> 
> Not being a driver person myself, I have no idea where to look next.
> I've attached prtconf -v and lspci -vvvxx output for PCI IDs and config
> info. Also attached are screenshots of mpstat and dtrace profile at
> 997hz running on the CPU that's bogged down with interrupts.
> 
Have you tried to disable csum and hardware offloading?

-- 
Hilsen/Regards
Michael Rasmussen

Get my public GnuPG keys:
michael <at> rasmussen <dot> cc
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E
mir <at> datanom <dot> net
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C
mir <at> miras <dot> org
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917
--------------------------------------------------------------
/usr/games/fortune -es says:
rugged, adj.:
	Too heavy to lift.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 163 bytes
Desc: OpenPGP digital signature
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20161001/4fce3123/attachment.bin>

From skiselkov.ml at gmail.com  Sat Oct  1 21:21:40 2016
From: skiselkov.ml at gmail.com (Saso Kiselkov)
Date: Sat, 1 Oct 2016 23:21:40 +0200
Subject: [OmniOS-discuss] rge_intr troubles
In-Reply-To: <20161001231335.03eca0ee@sleipner.datanom.net>
References: <8eb7ec55-f090-6bcd-7e5c-9d7179b079a8@gmail.com>
	<20161001231335.03eca0ee@sleipner.datanom.net>
Message-ID: <13c89c38-3679-e1fa-49b5-5cb87c8e6016@gmail.com>

On 10/1/16 11:13 PM, Michael Rasmussen wrote:
> hardware offloading

Just found the article that mentions dohwcksum. Set it to 0 and testing
now. Any other tunables I should be aware of?

Cheers,
-- 
Saso

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20161001/3a4ee16a/attachment.bin>

From mir at miras.org  Sat Oct  1 21:32:06 2016
From: mir at miras.org (Michael Rasmussen)
Date: Sat, 1 Oct 2016 23:32:06 +0200
Subject: [OmniOS-discuss] rge_intr troubles
In-Reply-To: <13c89c38-3679-e1fa-49b5-5cb87c8e6016@gmail.com>
References: <8eb7ec55-f090-6bcd-7e5c-9d7179b079a8@gmail.com>
	<20161001231335.03eca0ee@sleipner.datanom.net>
	<13c89c38-3679-e1fa-49b5-5cb87c8e6016@gmail.com>
Message-ID: <20161001233206.154cb73a@sleipner.datanom.net>

On Sat, 1 Oct 2016 23:21:40 +0200
Saso Kiselkov <skiselkov.ml at gmail.com> wrote:

> On 10/1/16 11:13 PM, Michael Rasmussen wrote:
> > hardware offloading  
> 
> Just found the article that mentions dohwcksum. Set it to 0 and testing
> now. Any other tunables I should be aware of?
> 
LRO: https://docs.oracle.com/cd/E53394_01/html/E54790/goryb.html


-- 
Hilsen/Regards
Michael Rasmussen

Get my public GnuPG keys:
michael <at> rasmussen <dot> cc
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E
mir <at> datanom <dot> net
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C
mir <at> miras <dot> org
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917
--------------------------------------------------------------
/usr/games/fortune -es says:
If it doesn't smell yet, it's pretty fresh.
		-- Dave Johnson, on dead seagulls
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 163 bytes
Desc: OpenPGP digital signature
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20161001/d1d5e9af/attachment.bin>

From skiselkov.ml at gmail.com  Sat Oct  1 21:32:32 2016
From: skiselkov.ml at gmail.com (Saso Kiselkov)
Date: Sat, 1 Oct 2016 23:32:32 +0200
Subject: [OmniOS-discuss] rge_intr troubles
In-Reply-To: <20161001231335.03eca0ee@sleipner.datanom.net>
References: <8eb7ec55-f090-6bcd-7e5c-9d7179b079a8@gmail.com>
	<20161001231335.03eca0ee@sleipner.datanom.net>
Message-ID: <3f440494-b82f-e10b-d582-ff4f670e596d@gmail.com>

On 10/1/16 11:13 PM, Michael Rasmussen wrote:
> On Sat, 1 Oct 2016 21:36:23 +0200
> Saso Kiselkov <skiselkov.ml at gmail.com> wrote:
> 
>>
>> Not being a driver person myself, I have no idea where to look next.
>> I've attached prtconf -v and lspci -vvvxx output for PCI IDs and config
>> info. Also attached are screenshots of mpstat and dtrace profile at
>> 997hz running on the CPU that's bogged down with interrupts.
>>
> Have you tried to disable csum and hardware offloading?

Unfortunately, setting ip:dohwcksum=0 in /etc/system didn't help.

-- 
Saso

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20161001/6a040a8c/attachment.bin>

From skiselkov.ml at gmail.com  Sat Oct  1 21:42:34 2016
From: skiselkov.ml at gmail.com (Saso Kiselkov)
Date: Sat, 1 Oct 2016 23:42:34 +0200
Subject: [OmniOS-discuss] rge_intr troubles
In-Reply-To: <20161001233206.154cb73a@sleipner.datanom.net>
References: <8eb7ec55-f090-6bcd-7e5c-9d7179b079a8@gmail.com>
	<20161001231335.03eca0ee@sleipner.datanom.net>
	<13c89c38-3679-e1fa-49b5-5cb87c8e6016@gmail.com>
	<20161001233206.154cb73a@sleipner.datanom.net>
Message-ID: <db00c4a6-40c7-f946-0d9f-8ccd31f16705@gmail.com>

On 10/1/16 11:32 PM, Michael Rasmussen wrote:
> On Sat, 1 Oct 2016 23:21:40 +0200
> Saso Kiselkov <skiselkov.ml at gmail.com> wrote:
> 
>> On 10/1/16 11:13 PM, Michael Rasmussen wrote:
>>> hardware offloading  
>>
>> Just found the article that mentions dohwcksum. Set it to 0 and testing
>> now. Any other tunables I should be aware of?
>>
> LRO: https://docs.oracle.com/cd/E53394_01/html/E54790/goryb.html
>

LRO isn't in Illumos' rge driver, at least the option doesn't appear to
be in rge's dladm show-linkprop.

-- 
Saso

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20161001/ae39ce31/attachment.bin>

From mir at miras.org  Sat Oct  1 21:45:56 2016
From: mir at miras.org (Michael Rasmussen)
Date: Sat, 1 Oct 2016 23:45:56 +0200
Subject: [OmniOS-discuss] rge_intr troubles
In-Reply-To: <3f440494-b82f-e10b-d582-ff4f670e596d@gmail.com>
References: <8eb7ec55-f090-6bcd-7e5c-9d7179b079a8@gmail.com>
	<20161001231335.03eca0ee@sleipner.datanom.net>
	<3f440494-b82f-e10b-d582-ff4f670e596d@gmail.com>
Message-ID: <20161001234556.6f701e1f@sleipner.datanom.net>

On Sat, 1 Oct 2016 23:32:32 +0200
Saso Kiselkov <skiselkov.ml at gmail.com> wrote:

> 
> Unfortunately, setting ip:dohwcksum=0 in /etc/system didn't help.
> 
You don't have a left-over intel nic to plug in?

-- 
Hilsen/Regards
Michael Rasmussen

Get my public GnuPG keys:
michael <at> rasmussen <dot> cc
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E
mir <at> datanom <dot> net
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C
mir <at> miras <dot> org
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917
--------------------------------------------------------------
/usr/games/fortune -es says:
Man 1:	Ask me the what the most important thing about telling a
good joke is.

Man 2:	OK, what is the most impo --

Man 1:	______TIMING!
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 163 bytes
Desc: OpenPGP digital signature
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20161001/f9bd57d8/attachment-0001.bin>

From skiselkov.ml at gmail.com  Sat Oct  1 22:00:43 2016
From: skiselkov.ml at gmail.com (Saso Kiselkov)
Date: Sun, 2 Oct 2016 00:00:43 +0200
Subject: [OmniOS-discuss] rge_intr troubles
In-Reply-To: <20161001234556.6f701e1f@sleipner.datanom.net>
References: <8eb7ec55-f090-6bcd-7e5c-9d7179b079a8@gmail.com>
	<20161001231335.03eca0ee@sleipner.datanom.net>
	<3f440494-b82f-e10b-d582-ff4f670e596d@gmail.com>
	<20161001234556.6f701e1f@sleipner.datanom.net>
Message-ID: <8f370a86-71f7-bb3b-21da-047a51b889ba@gmail.com>

On 10/1/16 11:45 PM, Michael Rasmussen wrote:
> On Sat, 1 Oct 2016 23:32:32 +0200
> Saso Kiselkov <skiselkov.ml at gmail.com> wrote:
> 
>>
>> Unfortunately, setting ip:dohwcksum=0 in /etc/system didn't help.
>>
> You don't have a left-over intel nic to plug in?

Sadly no. I might end up buying one, but first I'd like to try getting
this to work.

-- 
Saso


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20161002/3d7994e7/attachment.bin>

From skiselkov.ml at gmail.com  Sat Oct  1 22:01:26 2016
From: skiselkov.ml at gmail.com (Saso Kiselkov)
Date: Sun, 2 Oct 2016 00:01:26 +0200
Subject: [OmniOS-discuss] [discuss] Re: [networking] rge_intr troubles
In-Reply-To: <E6D41BFB-1EEB-436D-BA3E-337FFE5B8EEE@elemental.org>
References: <8eb7ec55-f090-6bcd-7e5c-9d7179b079a8@gmail.com>
	<E6D41BFB-1EEB-436D-BA3E-337FFE5B8EEE@elemental.org>
Message-ID: <feac5ff6-d0bb-fb1d-10b8-ce70740db215@gmail.com>

On 10/1/16 11:45 PM, Dale Ghent wrote:
> 
>> On Oct 1, 2016, at 3:36 PM, Saso Kiselkov <skiselkov.ml at gmail.com> wrote:
>>
>> So I'm playing around with a box that has an on-board Realtek NIC and
>> periodically, about once every 2-5 minutes, the network just goes out to
>> lunch and stops responding to ping or attempts to send anything from
>> the box. I noticed that while doing so, the box is getting floored by
>> interrupts from the NIC, so I see tons of rge_intr activity and one CPU
>> core receiving about 160000 interrupts per second (other cores are idle).
> 
> One core getting all the interrupts is expected, as both these chips and the driver do not support RSS.
> 
> The key thing here is to see what rge_intr() is actually doing. It has 2 outcomes: It identifies the interrupt type, processes it, then returns to the DDI that it was claimed. IF it doesn't identify the interrupt, rge_intr() returns and reports unclaimed to the DDI.
> 
> Knowing this info would be a good first step in figuring out what's going on.

Every time, we're returning through the bottom of the function,
returning DDI_INTR_CLAIMED. So the interrupt is meant for us.

-- 
Saso

From skiselkov.ml at gmail.com  Sat Oct  1 22:05:49 2016
From: skiselkov.ml at gmail.com (Saso Kiselkov)
Date: Sun, 2 Oct 2016 00:05:49 +0200
Subject: [OmniOS-discuss] [discuss] Re: [networking] rge_intr troubles
In-Reply-To: <E6D41BFB-1EEB-436D-BA3E-337FFE5B8EEE@elemental.org>
References: <8eb7ec55-f090-6bcd-7e5c-9d7179b079a8@gmail.com>
	<E6D41BFB-1EEB-436D-BA3E-337FFE5B8EEE@elemental.org>
Message-ID: <c39b194c-828f-d613-02fb-50247a79abe2@gmail.com>

On 10/1/16 11:45 PM, Dale Ghent wrote:
> 
>> On Oct 1, 2016, at 3:36 PM, Saso Kiselkov <skiselkov.ml at gmail.com> wrote:
>>
>> So I'm playing around with a box that has an on-board Realtek NIC and
>> periodically, about once every 2-5 minutes, the network just goes out to
>> lunch and stops responding to ping or attempts to send anything from
>> the box. I noticed that while doing so, the box is getting floored by
>> interrupts from the NIC, so I see tons of rge_intr activity and one CPU
>> core receiving about 160000 interrupts per second (other cores are idle).
> 
> One core getting all the interrupts is expected, as both these chips and the driver do not support RSS.
> 
> The key thing here is to see what rge_intr() is actually doing. It has 2 outcomes: It identifies the interrupt type, processes it, then returns to the DDI that it was claimed. IF it doesn't identify the interrupt, rge_intr() returns and reports unclaimed to the DDI.
> 
> Knowing this info would be a good first step in figuring out what's going on.

Btw: current working on modifying the driver to give us int_status
through dtrace, so I know what kind of interrupt we're dealing with.

-- 
Saso


From skiselkov.ml at gmail.com  Sat Oct  1 22:13:06 2016
From: skiselkov.ml at gmail.com (Saso Kiselkov)
Date: Sun, 2 Oct 2016 00:13:06 +0200
Subject: [OmniOS-discuss] [discuss] Re: [networking] rge_intr troubles
In-Reply-To: <E6D41BFB-1EEB-436D-BA3E-337FFE5B8EEE@elemental.org>
References: <8eb7ec55-f090-6bcd-7e5c-9d7179b079a8@gmail.com>
	<E6D41BFB-1EEB-436D-BA3E-337FFE5B8EEE@elemental.org>
Message-ID: <d80990e0-6dc5-dd98-e5f2-d86980d1ff1e@gmail.com>

On 10/1/16 11:45 PM, Dale Ghent wrote:
> 
>> On Oct 1, 2016, at 3:36 PM, Saso Kiselkov <skiselkov.ml at gmail.com> wrote:
>>
>> So I'm playing around with a box that has an on-board Realtek NIC and
>> periodically, about once every 2-5 minutes, the network just goes out to
>> lunch and stops responding to ping or attempts to send anything from
>> the box. I noticed that while doing so, the box is getting floored by
>> interrupts from the NIC, so I see tons of rge_intr activity and one CPU
>> core receiving about 160000 interrupts per second (other cores are idle).
> 
> One core getting all the interrupts is expected, as both these chips and the driver do not support RSS.
> 
> The key thing here is to see what rge_intr() is actually doing. It has 2 outcomes: It identifies the interrupt type, processes it, then returns to the DDI that it was claimed. IF it doesn't identify the interrupt, rge_intr() returns and reports unclaimed to the DDI.
> 
> Knowing this info would be a good first step in figuring out what's going on.

It's a whole buttload of RT_93c46_COMMOND_REG (0x50) interrupts.

-- 
Saso

From skiselkov.ml at gmail.com  Sat Oct  1 22:15:53 2016
From: skiselkov.ml at gmail.com (Saso Kiselkov)
Date: Sun, 2 Oct 2016 00:15:53 +0200
Subject: [OmniOS-discuss] [discuss] Re: [networking] rge_intr troubles
In-Reply-To: <E6D41BFB-1EEB-436D-BA3E-337FFE5B8EEE@elemental.org>
References: <8eb7ec55-f090-6bcd-7e5c-9d7179b079a8@gmail.com>
	<E6D41BFB-1EEB-436D-BA3E-337FFE5B8EEE@elemental.org>
Message-ID: <f00e2726-12b0-680c-d8f9-9a43269d4c58@gmail.com>

On 10/1/16 11:45 PM, Dale Ghent wrote:
> 
>> On Oct 1, 2016, at 3:36 PM, Saso Kiselkov <skiselkov.ml at gmail.com> wrote:
>>
>> So I'm playing around with a box that has an on-board Realtek NIC and
>> periodically, about once every 2-5 minutes, the network just goes out to
>> lunch and stops responding to ping or attempts to send anything from
>> the box. I noticed that while doing so, the box is getting floored by
>> interrupts from the NIC, so I see tons of rge_intr activity and one CPU
>> core receiving about 160000 interrupts per second (other cores are idle).
> 
> One core getting all the interrupts is expected, as both these chips and the driver do not support RSS.
> 
> The key thing here is to see what rge_intr() is actually doing. It has 2 outcomes: It identifies the interrupt type, processes it, then returns to the DDI that it was claimed. IF it doesn't identify the interrupt, rge_intr() returns and reports unclaimed to the DDI.
> 
> Knowing this info would be a good first step in figuring out what's going on.

Gah, I'm an idiot, it's actually a bitmask of two things:

RX_FIFO_OVERFLOW_INT | NO_RXDESC_INT

Apparently, we don't give it enough rx descriptors. Trying to now figure
out where to change that...

-- 
Saso

From skiselkov.ml at gmail.com  Sun Oct  2 12:23:05 2016
From: skiselkov.ml at gmail.com (Saso Kiselkov)
Date: Sun, 2 Oct 2016 14:23:05 +0200
Subject: [OmniOS-discuss] [discuss] Re: [networking] rge_intr troubles
In-Reply-To: <E7CEA449-1AFC-4FD7-9293-13EFDBCC818A@damore.org>
References: <8eb7ec55-f090-6bcd-7e5c-9d7179b079a8@gmail.com>
	<E6D41BFB-1EEB-436D-BA3E-337FFE5B8EEE@elemental.org>
	<f00e2726-12b0-680c-d8f9-9a43269d4c58@gmail.com>
	<18830ded-6d62-4528-a587-66a4ea0bc1db@joyent.com>
	<8a2fd932-3a0f-a160-f6d1-b42974544af9@gmail.com>
	<E7CEA449-1AFC-4FD7-9293-13EFDBCC818A@damore.org>
Message-ID: <6a7f9e0a-dfaa-56b7-186b-afc4018c4b0b@gmail.com>

Well what I know so far is that after a fixed number of packets
(probably after filling up the RX ring), we get a storm of
RX_FIFO_OVERFLOW_INT | NO_RXDESC_INT interrupts, probably because the
adapter is telling us that it filled up the RX ring and we didn't let it
know that we dequeued the old packets. Unfortunately, for the life of
me, I can't figure out how we're supposed to let it know that. I've been
staring at the drivers (both our and FreeBSD's) for hours and to me it's
all just a jumble of "DMA sync this, write reg that".

-- 
Saso

On 10/2/16 3:40 AM, Garrett D'Amore wrote:
> probably we should do something.  like reap the descriptors. i am afk but the usual strategy is to treat these kinds of interrupts just like normal rx. after that you should ack the interrupt of course. 
> 
> Sent from my iPhone
> 
>> On Oct 1, 2016, at 6:31 PM, Saso Kiselkov <skiselkov.ml at gmail.com> wrote:
>>
>>> On 10/2/16 12:23 AM, Robert Mustacchi wrote:
>>>> On 10/1/16 15:15 , Saso Kiselkov wrote:
>>>>> On 10/1/16 11:45 PM, Dale Ghent wrote:
>>>>>
>>>>>> On Oct 1, 2016, at 3:36 PM, Saso Kiselkov <skiselkov.ml at gmail.com> wrote:
>>>>>>
>>>>>> So I'm playing around with a box that has an on-board Realtek NIC and
>>>>>> periodically, about once every 2-5 minutes, the network just goes out to
>>>>>> lunch and stops responding to ping or attempts to send anything from
>>>>>> the box. I noticed that while doing so, the box is getting floored by
>>>>>> interrupts from the NIC, so I see tons of rge_intr activity and one CPU
>>>>>> core receiving about 160000 interrupts per second (other cores are idle).
>>>>>
>>>>> One core getting all the interrupts is expected, as both these chips and the driver do not support RSS.
>>>>>
>>>>> The key thing here is to see what rge_intr() is actually doing. It has 2 outcomes: It identifies the interrupt type, processes it, then returns to the DDI that it was claimed. IF it doesn't identify the interrupt, rge_intr() returns and reports unclaimed to the DDI.
>>>>>
>>>>> Knowing this info would be a good first step in figuring out what's going on.
>>>>
>>>> Gah, I'm an idiot, it's actually a bitmask of two things:
>>>>
>>>> RX_FIFO_OVERFLOW_INT | NO_RXDESC_INT
>>>>
>>>> Apparently, we don't give it enough rx descriptors. Trying to now figure
>>>> out where to change that...
>>>
>>> There'll always be cases where we don't have enough rx descriptors for
>>> devices. Presumably we shouldn't actually care about receiving that
>>> interrupt. Do you happen to have a specification for the device handy?
>>>
>>> Given that we're not doing anything with the NO_RXDESC_INT, we probably
>>> should just mask it on the device if possible.
>>
>> Just as a general FYI, I'm dealing with 8168G version of the MAC.
>> FreeBSD does have a driver that supports it, but since the driver there
>> appears home-grown (similar to ours), trying to transplant it would be a
>> major undertaking. I'll try to identify the major differences between
>> the versions we support and the 8168G, but of course, this being
>> hardware, they are many and few of them make any logical sense.
>>
>> --
>> Saso
>>
> 
> 
> -------------------------------------------
> illumos-networking
> Archives: https://www.listbox.com/member/archive/182193/=now
> RSS Feed: https://www.listbox.com/member/archive/rss/182193/22721964-fe287663
> Modify Your Subscription: https://www.listbox.com/member/?member_id=22721964&id_secret=22721964-d1c6dd60
> Powered by Listbox: http://www.listbox.com
> 


From garrett at damore.org  Sun Oct  2 16:59:06 2016
From: garrett at damore.org (Garrett D'Amore)
Date: Sun, 2 Oct 2016 09:59:06 -0700
Subject: [OmniOS-discuss] [discuss] Re: [networking] rge_intr troubles
In-Reply-To: <6a7f9e0a-dfaa-56b7-186b-afc4018c4b0b@gmail.com>
References: <8eb7ec55-f090-6bcd-7e5c-9d7179b079a8@gmail.com>
	<E6D41BFB-1EEB-436D-BA3E-337FFE5B8EEE@elemental.org>
	<f00e2726-12b0-680c-d8f9-9a43269d4c58@gmail.com>
	<18830ded-6d62-4528-a587-66a4ea0bc1db@joyent.com>
	<8a2fd932-3a0f-a160-f6d1-b42974544af9@gmail.com>
	<E7CEA449-1AFC-4FD7-9293-13EFDBCC818A@damore.org>
	<6a7f9e0a-dfaa-56b7-186b-afc4018c4b0b@gmail.com>
Message-ID: <CAAYoGd-B5+90D7uj=G2QnWQ7eW12uXQnWp6H98ETA9XU3dwpGg@mail.gmail.com>

Let me take a look.

On Sun, Oct 2, 2016 at 5:23 AM, Saso Kiselkov <skiselkov.ml at gmail.com>
wrote:

> Well what I know so far is that after a fixed number of packets
> (probably after filling up the RX ring), we get a storm of
> RX_FIFO_OVERFLOW_INT | NO_RXDESC_INT interrupts, probably because the
> adapter is telling us that it filled up the RX ring and we didn't let it
> know that we dequeued the old packets. Unfortunately, for the life of
> me, I can't figure out how we're supposed to let it know that. I've been
> staring at the drivers (both our and FreeBSD's) for hours and to me it's
> all just a jumble of "DMA sync this, write reg that".
>
> --
> Saso
>
> On 10/2/16 3:40 AM, Garrett D'Amore wrote:
> > probably we should do something.  like reap the descriptors. i am afk
> but the usual strategy is to treat these kinds of interrupts just like
> normal rx. after that you should ack the interrupt of course.
> >
> > Sent from my iPhone
> >
> >> On Oct 1, 2016, at 6:31 PM, Saso Kiselkov <skiselkov.ml at gmail.com>
> wrote:
> >>
> >>> On 10/2/16 12:23 AM, Robert Mustacchi wrote:
> >>>> On 10/1/16 15:15 , Saso Kiselkov wrote:
> >>>>> On 10/1/16 11:45 PM, Dale Ghent wrote:
> >>>>>
> >>>>>> On Oct 1, 2016, at 3:36 PM, Saso Kiselkov <skiselkov.ml at gmail.com>
> wrote:
> >>>>>>
> >>>>>> So I'm playing around with a box that has an on-board Realtek NIC
> and
> >>>>>> periodically, about once every 2-5 minutes, the network just goes
> out to
> >>>>>> lunch and stops responding to ping or attempts to send anything from
> >>>>>> the box. I noticed that while doing so, the box is getting floored
> by
> >>>>>> interrupts from the NIC, so I see tons of rge_intr activity and one
> CPU
> >>>>>> core receiving about 160000 interrupts per second (other cores are
> idle).
> >>>>>
> >>>>> One core getting all the interrupts is expected, as both these chips
> and the driver do not support RSS.
> >>>>>
> >>>>> The key thing here is to see what rge_intr() is actually doing. It
> has 2 outcomes: It identifies the interrupt type, processes it, then
> returns to the DDI that it was claimed. IF it doesn't identify the
> interrupt, rge_intr() returns and reports unclaimed to the DDI.
> >>>>>
> >>>>> Knowing this info would be a good first step in figuring out what's
> going on.
> >>>>
> >>>> Gah, I'm an idiot, it's actually a bitmask of two things:
> >>>>
> >>>> RX_FIFO_OVERFLOW_INT | NO_RXDESC_INT
> >>>>
> >>>> Apparently, we don't give it enough rx descriptors. Trying to now
> figure
> >>>> out where to change that...
> >>>
> >>> There'll always be cases where we don't have enough rx descriptors for
> >>> devices. Presumably we shouldn't actually care about receiving that
> >>> interrupt. Do you happen to have a specification for the device handy?
> >>>
> >>> Given that we're not doing anything with the NO_RXDESC_INT, we probably
> >>> should just mask it on the device if possible.
> >>
> >> Just as a general FYI, I'm dealing with 8168G version of the MAC.
> >> FreeBSD does have a driver that supports it, but since the driver there
> >> appears home-grown (similar to ours), trying to transplant it would be a
> >> major undertaking. I'll try to identify the major differences between
> >> the versions we support and the 8168G, but of course, this being
> >> hardware, they are many and few of them make any logical sense.
> >>
> >> --
> >> Saso
> >>
> >
> >
>
>
> -------------------------------------------
> illumos-networking
> Archives: https://www.listbox.com/member/archive/182193/=now
> RSS Feed: https://www.listbox.com/member/archive/rss/182193/
> 22035935-aad458cf
> Modify Your Subscription: https://www.listbox.com/
> member/?member_id=22035935&id_secret=22035935-4809d4da
> Powered by Listbox: http://www.listbox.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20161002/10387627/attachment.html>

From garrett at damore.org  Sun Oct  2 17:20:11 2016
From: garrett at damore.org (Garrett D'Amore)
Date: Sun, 2 Oct 2016 10:20:11 -0700
Subject: [OmniOS-discuss] [discuss] Re: [networking] rge_intr troubles
In-Reply-To: <CAAYoGd-B5+90D7uj=G2QnWQ7eW12uXQnWp6H98ETA9XU3dwpGg@mail.gmail.com>
References: <8eb7ec55-f090-6bcd-7e5c-9d7179b079a8@gmail.com>
	<E6D41BFB-1EEB-436D-BA3E-337FFE5B8EEE@elemental.org>
	<f00e2726-12b0-680c-d8f9-9a43269d4c58@gmail.com>
	<18830ded-6d62-4528-a587-66a4ea0bc1db@joyent.com>
	<8a2fd932-3a0f-a160-f6d1-b42974544af9@gmail.com>
	<E7CEA449-1AFC-4FD7-9293-13EFDBCC818A@damore.org>
	<6a7f9e0a-dfaa-56b7-186b-afc4018c4b0b@gmail.com>
	<CAAYoGd-B5+90D7uj=G2QnWQ7eW12uXQnWp6H98ETA9XU3dwpGg@mail.gmail.com>
Message-ID: <CAAYoGd_LXKrbUiJY76WCCuCWe-cfWvFmya3-zZSPB_FD0DN-Hw@mail.gmail.com>

Yuck.  This driver does the loan up thing ? which is ugly and pointless on
?modern? gigE hardware, bcopy is faster.  Especially as this driver is
probably only ever used with 1500 byte frames (it cannot do conventional
jumbo frames, but can do frames up to 7k? which is just bizarre!)  It looks
like this was copy-paste developed from nge, which explains some of the
sadness here.

But as a first test, you can try calling rge_receive().  The simplest way I
can see to do that is to OR in the value of RGE_NO_RXDESC_INT in the check
at 1495. (Btw the ordering of the checks at 1488 and 1495 are suboptimal,
as the rx interrupt should be the *hot* code path.

One thing you might also try doing is changing the value in rge.h for
RGE_RECV_COPY_SIZE from 256 to something much larger ? 8000 will be larger
than the largest possible rge frame.  I have a theory that part of the
problem you are encountering may be due to being out of buffers to loan up,
and the screwy handling for that case.

If I were doing maintenance on this driver, I would gut all the rx loan up
code, and just bcopy.  On CPUs produced in the past decade or so, its
cheaper to bcopy than to do the loan up.  Especially if you rip out all the
checks you have to do to make the decision about which to do.  (And the
driver becomes much leaner and simpler.)


On Sun, Oct 2, 2016 at 9:59 AM, Garrett D'Amore <garrett at damore.org> wrote:

> Let me take a look.
>
> On Sun, Oct 2, 2016 at 5:23 AM, Saso Kiselkov <skiselkov.ml at gmail.com>
> wrote:
>
>> Well what I know so far is that after a fixed number of packets
>> (probably after filling up the RX ring), we get a storm of
>> RX_FIFO_OVERFLOW_INT | NO_RXDESC_INT interrupts, probably because the
>> adapter is telling us that it filled up the RX ring and we didn't let it
>> know that we dequeued the old packets. Unfortunately, for the life of
>> me, I can't figure out how we're supposed to let it know that. I've been
>> staring at the drivers (both our and FreeBSD's) for hours and to me it's
>> all just a jumble of "DMA sync this, write reg that".
>>
>> --
>> Saso
>>
>> On 10/2/16 3:40 AM, Garrett D'Amore wrote:
>> > probably we should do something.  like reap the descriptors. i am afk
>> but the usual strategy is to treat these kinds of interrupts just like
>> normal rx. after that you should ack the interrupt of course.
>> >
>> > Sent from my iPhone
>> >
>> >> On Oct 1, 2016, at 6:31 PM, Saso Kiselkov <skiselkov.ml at gmail.com>
>> wrote:
>> >>
>> >>> On 10/2/16 12:23 AM, Robert Mustacchi wrote:
>> >>>> On 10/1/16 15:15 , Saso Kiselkov wrote:
>> >>>>> On 10/1/16 11:45 PM, Dale Ghent wrote:
>> >>>>>
>> >>>>>> On Oct 1, 2016, at 3:36 PM, Saso Kiselkov <skiselkov.ml at gmail.com>
>> wrote:
>> >>>>>>
>> >>>>>> So I'm playing around with a box that has an on-board Realtek NIC
>> and
>> >>>>>> periodically, about once every 2-5 minutes, the network just goes
>> out to
>> >>>>>> lunch and stops responding to ping or attempts to send anything
>> from
>> >>>>>> the box. I noticed that while doing so, the box is getting floored
>> by
>> >>>>>> interrupts from the NIC, so I see tons of rge_intr activity and
>> one CPU
>> >>>>>> core receiving about 160000 interrupts per second (other cores are
>> idle).
>> >>>>>
>> >>>>> One core getting all the interrupts is expected, as both these
>> chips and the driver do not support RSS.
>> >>>>>
>> >>>>> The key thing here is to see what rge_intr() is actually doing. It
>> has 2 outcomes: It identifies the interrupt type, processes it, then
>> returns to the DDI that it was claimed. IF it doesn't identify the
>> interrupt, rge_intr() returns and reports unclaimed to the DDI.
>> >>>>>
>> >>>>> Knowing this info would be a good first step in figuring out what's
>> going on.
>> >>>>
>> >>>> Gah, I'm an idiot, it's actually a bitmask of two things:
>> >>>>
>> >>>> RX_FIFO_OVERFLOW_INT | NO_RXDESC_INT
>> >>>>
>> >>>> Apparently, we don't give it enough rx descriptors. Trying to now
>> figure
>> >>>> out where to change that...
>> >>>
>> >>> There'll always be cases where we don't have enough rx descriptors for
>> >>> devices. Presumably we shouldn't actually care about receiving that
>> >>> interrupt. Do you happen to have a specification for the device handy?
>> >>>
>> >>> Given that we're not doing anything with the NO_RXDESC_INT, we
>> probably
>> >>> should just mask it on the device if possible.
>> >>
>> >> Just as a general FYI, I'm dealing with 8168G version of the MAC.
>> >> FreeBSD does have a driver that supports it, but since the driver there
>> >> appears home-grown (similar to ours), trying to transplant it would be
>> a
>> >> major undertaking. I'll try to identify the major differences between
>> >> the versions we support and the 8168G, but of course, this being
>> >> hardware, they are many and few of them make any logical sense.
>> >>
>> >> --
>> >> Saso
>> >>
>> >
>> >
>>
>>
>> -------------------------------------------
>> illumos-networking
>> Archives: https://www.listbox.com/member/archive/182193/=now
>> RSS Feed: https://www.listbox.com/member/archive/rss/182193/22035935-
>> aad458cf
>> Modify Your Subscription: https://www.listbox.com/member
>> /?member_id=22035935&id_secret=22035935-4809d4da
>> Powered by Listbox: http://www.listbox.com
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20161002/19440062/attachment.html>

From matej at zunaj.si  Sun Oct  2 17:33:55 2016
From: matej at zunaj.si (=?utf-8?Q?Matej_=C5=BDerovnik?=)
Date: Sun, 2 Oct 2016 19:33:55 +0200
Subject: [OmniOS-discuss] zpool Write Bottlenecks
In-Reply-To: <CE812361-A027-4C79-99C5-75B1C4C860BD@lji.org>
References: <34569661-54B5-4C13-9662-D3C088EC7BFB@lji.org>
	<FA018B31-B458-431D-8F0E-2334DF933D2C@omniti.com>
	<27B73CAE-7A40-42F6-AD37-1556015D6D4E@lji.org>
	<alpine.GSO.2.20.1609300817350.13801@freddy.simplesystems.org>
	<CE812361-A027-4C79-99C5-75B1C4C860BD@lji.org>
Message-ID: <34C206B4-A5FC-4132-8B4D-EFF7A541C2AC@zunaj.si>

You could also try benchmarking with 'fio' tool, where you have some more control over the benchmark. 

I?m also having some troubles with iSCSI performance, where I can get up to 3GB/s with local tests, but can?t go over 500MB/s via network (10gbit, latency is minimal, since both servers are on the same switch). 

Matej

> On 30 Sep 2016, at 18:46, Michael Talbott <mtalbott at lji.org> wrote:
> 
> Ah. That explains it. At first I figured the single stream couldn't be the case since a dd from /dev/zero to /dev/null was in excess of 24GB/s. But what you said  about TX groups explains it all. I ran several dd's in parallel and now see all of the disks getting fully saturated.
> 
> Thanks
> 
> 
>> On Sep 30, 2016, at 6:36 AM, Bob Friesenhahn <bfriesen at simple.dallas.tx.us> wrote:
>> 
>> On Thu, 29 Sep 2016, Michael Talbott wrote:
>> 
>>> I'm very aware that dd is not the best tool for measuring disk performance in most cases. And I know the read throughput number is way off b/c of zfs caching mechanisms. However it does work in this case to illustrate the point in my case of a write throttle somewhere in the system. If anyone needs me to test with some other
>> 
>> I think that Dale is correct that your benchmark may be only benchmarking 'dd'.  The performance of single-threaded 'dd' will be entirely driven by read performance of /dev/zero followed by write latency per operation on writes.  In your case, the issue is almost certainly that the write latency is a fixed value (regardless of the number of disks) and so there is a fixed maximum write rate regardless of the performance of the underlying store.
>> 
>> If you were to run two 'dd's in parallel, you may see an increase in the write rate.  This is the first thing to try.
>> 
>> With zfs, a "write" goes into a buffer associated with a transaction group (TXG) which is written to disk based on a maximum elapsed time, or the maximum size of the TXG being hit.  The maximum size of the TXG is estimated based on the available total RAM and zfs's guess as to the time to write the TXG (based on throughput), as well as tunables.
>> 
>> For a brief time, the writer proces will not be able to write any data at all as the current TXG is finished (resulting in writes) and a new one is started.
>> 
>> At the lowest level, the TXG can not be completed until all involved disks have written their data and completed a cache sync operation. The latency of the cache sync is driven by the latency of the slowest disk drive (+HBA) involved.  Getting rid of any disks which exhibit abnormally high latency would help with the transaction times.
>> 
>> Bob
>> -- 
>> Bob Friesenhahn
>> bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
>> GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/
> 
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss


From skiselkov.ml at gmail.com  Sun Oct  2 19:45:15 2016
From: skiselkov.ml at gmail.com (Saso Kiselkov)
Date: Sun, 2 Oct 2016 21:45:15 +0200
Subject: [OmniOS-discuss] [discuss] Re: [networking] rge_intr troubles
In-Reply-To: <CAAYoGd_LXKrbUiJY76WCCuCWe-cfWvFmya3-zZSPB_FD0DN-Hw@mail.gmail.com>
References: <8eb7ec55-f090-6bcd-7e5c-9d7179b079a8@gmail.com>
	<E6D41BFB-1EEB-436D-BA3E-337FFE5B8EEE@elemental.org>
	<f00e2726-12b0-680c-d8f9-9a43269d4c58@gmail.com>
	<18830ded-6d62-4528-a587-66a4ea0bc1db@joyent.com>
	<8a2fd932-3a0f-a160-f6d1-b42974544af9@gmail.com>
	<E7CEA449-1AFC-4FD7-9293-13EFDBCC818A@damore.org>
	<6a7f9e0a-dfaa-56b7-186b-afc4018c4b0b@gmail.com>
	<CAAYoGd-B5+90D7uj=G2QnWQ7eW12uXQnWp6H98ETA9XU3dwpGg@mail.gmail.com>
	<CAAYoGd_LXKrbUiJY76WCCuCWe-cfWvFmya3-zZSPB_FD0DN-Hw@mail.gmail.com>
Message-ID: <e415c27d-f03c-8d22-df6a-17336396ecc6@gmail.com>

Thanks for the suggestions! Results below:

On 10/2/16 7:20 PM, Garrett D'Amore wrote:
> But as a first test, you can try calling rge_receive().  The simplest
> way I can see to do that is to OR in the value of RGE_NO_RXDESC_INT in
> the check at 1495. (Btw the ordering of the checks at 1488 and 1495 are
> suboptimal, as the rx interrupt should be the *hot* code path.

Sadly, this won't help. RGE_RX_INT is already a composite of
RX_OK_INT | RX_ERR_INT | NO_RXDESC_INT

> One thing you might also try doing is changing the value in rge.h for
> RGE_RECV_COPY_SIZE from 256 to something much larger ? 8000 will be
> larger than the largest possible rge frame.  I have a theory that part
> of the problem you are encountering may be due to being out of buffers
> to loan up, and the screwy handling for that case.

Tried this, but to no avail. It still goes into the sad place. Sadly,
this stupid box doesn't even have a serial port with which I could
provide you access. And I'm out of PCI-e slots to shove in an Intel NIC,
so the on-board Realtek POS is my last option. Well that, or using Linux
with Illumos as a KVM on top of it, but even saying that out loud leaves
a bad taste in my mouth...

Cheers,
-- 
Saso

From garrett at damore.org  Sun Oct  2 20:09:49 2016
From: garrett at damore.org (Garrett D'Amore)
Date: Sun, 2 Oct 2016 13:09:49 -0700
Subject: [OmniOS-discuss] [discuss] Re: [networking] rge_intr troubles
In-Reply-To: <e415c27d-f03c-8d22-df6a-17336396ecc6@gmail.com>
References: <8eb7ec55-f090-6bcd-7e5c-9d7179b079a8@gmail.com>
	<E6D41BFB-1EEB-436D-BA3E-337FFE5B8EEE@elemental.org>
	<f00e2726-12b0-680c-d8f9-9a43269d4c58@gmail.com>
	<18830ded-6d62-4528-a587-66a4ea0bc1db@joyent.com>
	<8a2fd932-3a0f-a160-f6d1-b42974544af9@gmail.com>
	<E7CEA449-1AFC-4FD7-9293-13EFDBCC818A@damore.org>
	<6a7f9e0a-dfaa-56b7-186b-afc4018c4b0b@gmail.com>
	<CAAYoGd-B5+90D7uj=G2QnWQ7eW12uXQnWp6H98ETA9XU3dwpGg@mail.gmail.com>
	<CAAYoGd_LXKrbUiJY76WCCuCWe-cfWvFmya3-zZSPB_FD0DN-Hw@mail.gmail.com>
	<e415c27d-f03c-8d22-df6a-17336396ecc6@gmail.com>
Message-ID: <DAD79D3B-4543-42C8-AD2C-82D96BA0EB8C@damore.org>

ah so maybe we need to change the logic so that the exhaustion of descriptors takes precedence over the rx fifo overrun.  

Sent from my iPhone

> On Oct 2, 2016, at 12:45 PM, Saso Kiselkov <skiselkov.ml at gmail.com> wrote:
> 
> Thanks for the suggestions! Results below:
> 
>> On 10/2/16 7:20 PM, Garrett D'Amore wrote:
>> But as a first test, you can try calling rge_receive().  The simplest
>> way I can see to do that is to OR in the value of RGE_NO_RXDESC_INT in
>> the check at 1495. (Btw the ordering of the checks at 1488 and 1495 are
>> suboptimal, as the rx interrupt should be the *hot* code path.
> 
> Sadly, this won't help. RGE_RX_INT is already a composite of
> RX_OK_INT | RX_ERR_INT | NO_RXDESC_INT
> 
>> One thing you might also try doing is changing the value in rge.h for
>> RGE_RECV_COPY_SIZE from 256 to something much larger ? 8000 will be
>> larger than the largest possible rge frame.  I have a theory that part
>> of the problem you are encountering may be due to being out of buffers
>> to loan up, and the screwy handling for that case.
> 
> Tried this, but to no avail. It still goes into the sad place. Sadly,
> this stupid box doesn't even have a serial port with which I could
> provide you access. And I'm out of PCI-e slots to shove in an Intel NIC,
> so the on-board Realtek POS is my last option. Well that, or using Linux
> with Illumos as a KVM on top of it, but even saying that out loud leaves
> a bad taste in my mouth...
> 
> Cheers,
> -- 
> Saso
> 
> 
> -------------------------------------------
> illumos-networking
> Archives: https://www.listbox.com/member/archive/182193/=now
> RSS Feed: https://www.listbox.com/member/archive/rss/182193/22035935-aad458cf
> Modify Your Subscription: https://www.listbox.com/member/?member_id=22035935&id_secret=22035935-4809d4da
> Powered by Listbox: http://www.listbox.com

From skiselkov.ml at gmail.com  Sun Oct  2 20:32:53 2016
From: skiselkov.ml at gmail.com (Saso Kiselkov)
Date: Sun, 2 Oct 2016 22:32:53 +0200
Subject: [OmniOS-discuss] [discuss] Re: [networking] rge_intr troubles
In-Reply-To: <DAD79D3B-4543-42C8-AD2C-82D96BA0EB8C@damore.org>
References: <8eb7ec55-f090-6bcd-7e5c-9d7179b079a8@gmail.com>
	<E6D41BFB-1EEB-436D-BA3E-337FFE5B8EEE@elemental.org>
	<f00e2726-12b0-680c-d8f9-9a43269d4c58@gmail.com>
	<18830ded-6d62-4528-a587-66a4ea0bc1db@joyent.com>
	<8a2fd932-3a0f-a160-f6d1-b42974544af9@gmail.com>
	<E7CEA449-1AFC-4FD7-9293-13EFDBCC818A@damore.org>
	<6a7f9e0a-dfaa-56b7-186b-afc4018c4b0b@gmail.com>
	<CAAYoGd-B5+90D7uj=G2QnWQ7eW12uXQnWp6H98ETA9XU3dwpGg@mail.gmail.com>
	<CAAYoGd_LXKrbUiJY76WCCuCWe-cfWvFmya3-zZSPB_FD0DN-Hw@mail.gmail.com>
	<e415c27d-f03c-8d22-df6a-17336396ecc6@gmail.com>
	<DAD79D3B-4543-42C8-AD2C-82D96BA0EB8C@damore.org>
Message-ID: <03ab6cbc-facf-622b-4c56-689a93ccd6ee@gmail.com>

I'm willing to test anything you suggest.

-- 
Saso

On 10/2/16 10:09 PM, Garrett D'Amore wrote:
> ah so maybe we need to change the logic so that the exhaustion of descriptors takes precedence over the rx fifo overrun.  
> 
> Sent from my iPhone
> 
>> On Oct 2, 2016, at 12:45 PM, Saso Kiselkov <skiselkov.ml at gmail.com> wrote:
>>
>> Thanks for the suggestions! Results below:
>>
>>> On 10/2/16 7:20 PM, Garrett D'Amore wrote:
>>> But as a first test, you can try calling rge_receive().  The simplest
>>> way I can see to do that is to OR in the value of RGE_NO_RXDESC_INT in
>>> the check at 1495. (Btw the ordering of the checks at 1488 and 1495 are
>>> suboptimal, as the rx interrupt should be the *hot* code path.
>>
>> Sadly, this won't help. RGE_RX_INT is already a composite of
>> RX_OK_INT | RX_ERR_INT | NO_RXDESC_INT
>>
>>> One thing you might also try doing is changing the value in rge.h for
>>> RGE_RECV_COPY_SIZE from 256 to something much larger ? 8000 will be
>>> larger than the largest possible rge frame.  I have a theory that part
>>> of the problem you are encountering may be due to being out of buffers
>>> to loan up, and the screwy handling for that case.
>>
>> Tried this, but to no avail. It still goes into the sad place. Sadly,
>> this stupid box doesn't even have a serial port with which I could
>> provide you access. And I'm out of PCI-e slots to shove in an Intel NIC,
>> so the on-board Realtek POS is my last option. Well that, or using Linux
>> with Illumos as a KVM on top of it, but even saying that out loud leaves
>> a bad taste in my mouth...
>>
>> Cheers,
>> --
>> Saso
>>
> 
> 
> -------------------------------------------
> illumos-discuss
> Archives: https://www.listbox.com/member/archive/182180/=now
> RSS Feed: https://www.listbox.com/member/archive/rss/182180/22722377-e9306e56
> Modify Your Subscription: https://www.listbox.com/member/?member_id=22722377&id_secret=22722377-08ac87bf
> Powered by Listbox: http://www.listbox.com
> 


From garrett at damore.org  Sun Oct  2 20:37:39 2016
From: garrett at damore.org (Garrett D'Amore)
Date: Sun, 2 Oct 2016 13:37:39 -0700
Subject: [OmniOS-discuss] [discuss] Re: [networking] rge_intr troubles
In-Reply-To: <03ab6cbc-facf-622b-4c56-689a93ccd6ee@gmail.com>
References: <8eb7ec55-f090-6bcd-7e5c-9d7179b079a8@gmail.com>
	<E6D41BFB-1EEB-436D-BA3E-337FFE5B8EEE@elemental.org>
	<f00e2726-12b0-680c-d8f9-9a43269d4c58@gmail.com>
	<18830ded-6d62-4528-a587-66a4ea0bc1db@joyent.com>
	<8a2fd932-3a0f-a160-f6d1-b42974544af9@gmail.com>
	<E7CEA449-1AFC-4FD7-9293-13EFDBCC818A@damore.org>
	<6a7f9e0a-dfaa-56b7-186b-afc4018c4b0b@gmail.com>
	<CAAYoGd-B5+90D7uj=G2QnWQ7eW12uXQnWp6H98ETA9XU3dwpGg@mail.gmail.com>
	<CAAYoGd_LXKrbUiJY76WCCuCWe-cfWvFmya3-zZSPB_FD0DN-Hw@mail.gmail.com>
	<e415c27d-f03c-8d22-df6a-17336396ecc6@gmail.com>
	<DAD79D3B-4543-42C8-AD2C-82D96BA0EB8C@damore.org>
	<03ab6cbc-facf-622b-4c56-689a93ccd6ee@gmail.com>
Message-ID: <D3452844-42DB-4C91-A37C-D23AD7865855@damore.org>

so do the rx logic first even if the fifo overflow is set.  im not in front of the code but refer to the if-else block I referenced earlier. 

Sent from my iPhone

> On Oct 2, 2016, at 1:32 PM, Saso Kiselkov <skiselkov.ml at gmail.com> wrote:
> 
> I'm willing to test anything you suggest.
> 
> -- 
> Saso
> 
>> On 10/2/16 10:09 PM, Garrett D'Amore wrote:
>> ah so maybe we need to change the logic so that the exhaustion of descriptors takes precedence over the rx fifo overrun.  
>> 
>> Sent from my iPhone
>> 
>>> On Oct 2, 2016, at 12:45 PM, Saso Kiselkov <skiselkov.ml at gmail.com> wrote:
>>> 
>>> Thanks for the suggestions! Results below:
>>> 
>>>> On 10/2/16 7:20 PM, Garrett D'Amore wrote:
>>>> But as a first test, you can try calling rge_receive().  The simplest
>>>> way I can see to do that is to OR in the value of RGE_NO_RXDESC_INT in
>>>> the check at 1495. (Btw the ordering of the checks at 1488 and 1495 are
>>>> suboptimal, as the rx interrupt should be the *hot* code path.
>>> 
>>> Sadly, this won't help. RGE_RX_INT is already a composite of
>>> RX_OK_INT | RX_ERR_INT | NO_RXDESC_INT
>>> 
>>>> One thing you might also try doing is changing the value in rge.h for
>>>> RGE_RECV_COPY_SIZE from 256 to something much larger ? 8000 will be
>>>> larger than the largest possible rge frame.  I have a theory that part
>>>> of the problem you are encountering may be due to being out of buffers
>>>> to loan up, and the screwy handling for that case.
>>> 
>>> Tried this, but to no avail. It still goes into the sad place. Sadly,
>>> this stupid box doesn't even have a serial port with which I could
>>> provide you access. And I'm out of PCI-e slots to shove in an Intel NIC,
>>> so the on-board Realtek POS is my last option. Well that, or using Linux
>>> with Illumos as a KVM on top of it, but even saying that out loud leaves
>>> a bad taste in my mouth...
>>> 
>>> Cheers,
>>> --
>>> Saso
> 
> 
> -------------------------------------------
> illumos-networking
> Archives: https://www.listbox.com/member/archive/182193/=now
> RSS Feed: https://www.listbox.com/member/archive/rss/182193/22035935-aad458cf
> Modify Your Subscription: https://www.listbox.com/member/?member_id=22035935&id_secret=22035935-4809d4da
> Powered by Listbox: http://www.listbox.com

From skiselkov.ml at gmail.com  Sun Oct  2 20:51:09 2016
From: skiselkov.ml at gmail.com (Saso Kiselkov)
Date: Sun, 2 Oct 2016 22:51:09 +0200
Subject: [OmniOS-discuss] [discuss] Re: [networking] rge_intr troubles
In-Reply-To: <D3452844-42DB-4C91-A37C-D23AD7865855@damore.org>
References: <8eb7ec55-f090-6bcd-7e5c-9d7179b079a8@gmail.com>
	<E6D41BFB-1EEB-436D-BA3E-337FFE5B8EEE@elemental.org>
	<f00e2726-12b0-680c-d8f9-9a43269d4c58@gmail.com>
	<18830ded-6d62-4528-a587-66a4ea0bc1db@joyent.com>
	<8a2fd932-3a0f-a160-f6d1-b42974544af9@gmail.com>
	<E7CEA449-1AFC-4FD7-9293-13EFDBCC818A@damore.org>
	<6a7f9e0a-dfaa-56b7-186b-afc4018c4b0b@gmail.com>
	<CAAYoGd-B5+90D7uj=G2QnWQ7eW12uXQnWp6H98ETA9XU3dwpGg@mail.gmail.com>
	<CAAYoGd_LXKrbUiJY76WCCuCWe-cfWvFmya3-zZSPB_FD0DN-Hw@mail.gmail.com>
	<e415c27d-f03c-8d22-df6a-17336396ecc6@gmail.com>
	<DAD79D3B-4543-42C8-AD2C-82D96BA0EB8C@damore.org>
	<03ab6cbc-facf-622b-4c56-689a93ccd6ee@gmail.com>
	<D3452844-42DB-4C91-A37C-D23AD7865855@damore.org>
Message-ID: <e81171b0-4c8c-ceff-1535-667527c09656@gmail.com>

But we already do an rge_receive on receiving NO_RXDESC_INT. The code
sequence is:

 1) get interrupt status word (line 1479)
 2) if interrupt is RX_FIFO_OVERFLOW_INT, modify int_mask variable to
    exclude further RX_FIFO_OVERFLOW_INT (line 1488)
 3) otherwise if interrupt contains NO_RXDESC_INT (among other things),
    modify int_mask to include RX_FIFO_OVERFLOW_INT (line 1495)
 4) if interrupt contains NO_RXDESC_INT (among other things), do
    rge_receive (line 1510)
 ...
 5) update interrupt mask according to steps 2 & 3.

So after receiving the interrupt, the first thing we pretty much do is
rge_receive anyway. I'm sorry I can't be of more help here, but this is
all completely foreign to me. I suspect the problem has more to do with
the fact that we never really properly dequeue packets from the RX ring,
so once the NIC runs out of descriptors, it just keeps on notifying us
that it needs more RX descriptors. The reason I think is that this
problem appears to be triggered strictly by the number of packets received.

-- 
Saso

On 10/2/16 10:37 PM, Garrett D'Amore wrote:
> so do the rx logic first even if the fifo overflow is set.  im not in front of the code but refer to the if-else block I referenced earlier. 
> 
> Sent from my iPhone
> 
>> On Oct 2, 2016, at 1:32 PM, Saso Kiselkov <skiselkov.ml at gmail.com> wrote:
>>
>> I'm willing to test anything you suggest.
>>
>> -- 
>> Saso
>>
>>> On 10/2/16 10:09 PM, Garrett D'Amore wrote:
>>> ah so maybe we need to change the logic so that the exhaustion of descriptors takes precedence over the rx fifo overrun.  
>>>
>>> Sent from my iPhone
>>>
>>>> On Oct 2, 2016, at 12:45 PM, Saso Kiselkov <skiselkov.ml at gmail.com> wrote:
>>>>
>>>> Thanks for the suggestions! Results below:
>>>>
>>>>> On 10/2/16 7:20 PM, Garrett D'Amore wrote:
>>>>> But as a first test, you can try calling rge_receive().  The simplest
>>>>> way I can see to do that is to OR in the value of RGE_NO_RXDESC_INT in
>>>>> the check at 1495. (Btw the ordering of the checks at 1488 and 1495 are
>>>>> suboptimal, as the rx interrupt should be the *hot* code path.
>>>>
>>>> Sadly, this won't help. RGE_RX_INT is already a composite of
>>>> RX_OK_INT | RX_ERR_INT | NO_RXDESC_INT
>>>>
>>>>> One thing you might also try doing is changing the value in rge.h for
>>>>> RGE_RECV_COPY_SIZE from 256 to something much larger ? 8000 will be
>>>>> larger than the largest possible rge frame.  I have a theory that part
>>>>> of the problem you are encountering may be due to being out of buffers
>>>>> to loan up, and the screwy handling for that case.
>>>>
>>>> Tried this, but to no avail. It still goes into the sad place. Sadly,
>>>> this stupid box doesn't even have a serial port with which I could
>>>> provide you access. And I'm out of PCI-e slots to shove in an Intel NIC,
>>>> so the on-board Realtek POS is my last option. Well that, or using Linux
>>>> with Illumos as a KVM on top of it, but even saying that out loud leaves
>>>> a bad taste in my mouth...
>>>>
>>>> Cheers,
>>>> --
>>>> Saso
>>
>>
> 
> 
> -------------------------------------------
> illumos-discuss
> Archives: https://www.listbox.com/member/archive/182180/=now
> RSS Feed: https://www.listbox.com/member/archive/rss/182180/22722377-e9306e56
> Modify Your Subscription: https://www.listbox.com/member/?member_id=22722377&id_secret=22722377-08ac87bf
> Powered by Listbox: http://www.listbox.com
> 


From garrett at damore.org  Mon Oct  3 01:55:37 2016
From: garrett at damore.org (Garrett D'Amore)
Date: Sun, 2 Oct 2016 18:55:37 -0700
Subject: [OmniOS-discuss] [discuss] Re: [networking] rge_intr troubles
In-Reply-To: <e415c27d-f03c-8d22-df6a-17336396ecc6@gmail.com>
References: <8eb7ec55-f090-6bcd-7e5c-9d7179b079a8@gmail.com>
	<E6D41BFB-1EEB-436D-BA3E-337FFE5B8EEE@elemental.org>
	<f00e2726-12b0-680c-d8f9-9a43269d4c58@gmail.com>
	<18830ded-6d62-4528-a587-66a4ea0bc1db@joyent.com>
	<8a2fd932-3a0f-a160-f6d1-b42974544af9@gmail.com>
	<E7CEA449-1AFC-4FD7-9293-13EFDBCC818A@damore.org>
	<6a7f9e0a-dfaa-56b7-186b-afc4018c4b0b@gmail.com>
	<CAAYoGd-B5+90D7uj=G2QnWQ7eW12uXQnWp6H98ETA9XU3dwpGg@mail.gmail.com>
	<CAAYoGd_LXKrbUiJY76WCCuCWe-cfWvFmya3-zZSPB_FD0DN-Hw@mail.gmail.com>
	<e415c27d-f03c-8d22-df6a-17336396ecc6@gmail.com>
Message-ID: <4555194F-E01D-4ACC-B586-F77403AE416D@damore.org>

i will try to have another look at this tomorrow.  i am too tired tonight. :)

Sent from my iPhone

> On Oct 2, 2016, at 12:45 PM, Saso Kiselkov <skiselkov.ml at gmail.com> wrote:
> 
> Thanks for the suggestions! Results below:
> 
>> On 10/2/16 7:20 PM, Garrett D'Amore wrote:
>> But as a first test, you can try calling rge_receive().  The simplest
>> way I can see to do that is to OR in the value of RGE_NO_RXDESC_INT in
>> the check at 1495. (Btw the ordering of the checks at 1488 and 1495 are
>> suboptimal, as the rx interrupt should be the *hot* code path.
> 
> Sadly, this won't help. RGE_RX_INT is already a composite of
> RX_OK_INT | RX_ERR_INT | NO_RXDESC_INT
> 
>> One thing you might also try doing is changing the value in rge.h for
>> RGE_RECV_COPY_SIZE from 256 to something much larger ? 8000 will be
>> larger than the largest possible rge frame.  I have a theory that part
>> of the problem you are encountering may be due to being out of buffers
>> to loan up, and the screwy handling for that case.
> 
> Tried this, but to no avail. It still goes into the sad place. Sadly,
> this stupid box doesn't even have a serial port with which I could
> provide you access. And I'm out of PCI-e slots to shove in an Intel NIC,
> so the on-board Realtek POS is my last option. Well that, or using Linux
> with Illumos as a KVM on top of it, but even saying that out loud leaves
> a bad taste in my mouth...
> 
> Cheers,
> -- 
> Saso
> 
> 
> -------------------------------------------
> illumos-networking
> Archives: https://www.listbox.com/member/archive/182193/=now
> RSS Feed: https://www.listbox.com/member/archive/rss/182193/22035935-aad458cf
> Modify Your Subscription: https://www.listbox.com/member/?member_id=22035935&id_secret=22035935-4809d4da
> Powered by Listbox: http://www.listbox.com

From mir at miras.org  Mon Oct  3 20:39:40 2016
From: mir at miras.org (Michael Rasmussen)
Date: Mon, 3 Oct 2016 22:39:40 +0200
Subject: [OmniOS-discuss] Supermicro X10SLL-F
Message-ID: <20161003223940.7e954c22@sleipner.datanom.net>

Hi all,

I recently swapped motherboard in an Omnios server to Supermicro
X10SLL-F and system came up again without any problems so this is all fine
and dandy. But one thing that literally took hours to solve was getting
the board to even post. So here is the story and solution to anybody
hitting the same problem as I.

Symptom: Board gets power since power led lights and IPMI heartbeat led
blinks, but no post what so ever. Access to IPMI through web browser
works but system is reported to be off and cannot be started. Post
snooping shows f0.
Solution: 
1) Power off the system and pop the CMOS battery. Wait a couple of
minuts before going to step 2.
2) Power on system without CMOS battery (Single user mode preferable)
3) When system is powered up fully consult IPMI to ensure everything is
OK and the power system off again
4) Reinsert CMOS battery and power system on again.

Every thing should work again.

Outstanding unresolved issues:
Case fans are Arctic F8 PWM
(http://www.newegg.com/Product/Product.aspx?Item=N82E16835186072)
and Arctic F12 PWM
(http://www.newegg.com/Product/Product.aspx?Item=N82E16835186033)
Problem: Even though fan speed is controlled by Supermicro fan mode no
speed readings are available (display N/A). Maybe this is because the
Arctic fans are all revision 1 but to sure to have working speed
readings I have ordered new fans from Noctua.
A couple of Noctua NF-A8 PWM
(http://www.newegg.com/Product/Product.aspx?Item=9SIA9PV3Y64158)
A couple of Noctua NF-S12A
(http://www.newegg.com/Product/Product.aspx?Item=N82E16835608038)

Anybody having experience with these fans on supermicro boards?

-- 
Hilsen/Regards
Michael Rasmussen

Get my public GnuPG keys:
michael <at> rasmussen <dot> cc
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E
mir <at> datanom <dot> net
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C
mir <at> miras <dot> org
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917
--------------------------------------------------------------
/usr/games/fortune -es says:
Life is a yo-yo, and mankind ties knots in the string.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 163 bytes
Desc: OpenPGP digital signature
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20161003/f1078e3b/attachment.bin>

From groups at tierarzt-mueller.de  Tue Oct  4 09:31:05 2016
From: groups at tierarzt-mueller.de (Alexander Lesle)
Date: Tue, 4 Oct 2016 11:31:05 +0200
Subject: [OmniOS-discuss] Supermicro X10SLL-F
In-Reply-To: <20161003223940.7e954c22@sleipner.datanom.net>
References: <20161003223940.7e954c22@sleipner.datanom.net>
Message-ID: <294740579.20161004113105@tierarzt-mueller.de>

Hello Michael Rasmussen and List,

maybe the fans from supermicro are the better way.
https://www.supermicro.com/support/resources/Thermal/#FAN

On Oktober, 03 2016, 22:39 <Michael Rasmussen> wrote in [1]:

> Outstanding unresolved issues:
> Case fans are Arctic F8 PWM
> (http://www.newegg.com/Product/Product.aspx?Item=N82E16835186072)
> and Arctic F12 PWM
> (http://www.newegg.com/Product/Product.aspx?Item=N82E16835186033)
> Problem: Even though fan speed is controlled by Supermicro fan mode no
> speed readings are available (display N/A). Maybe this is because the
> Arctic fans are all revision 1 but to sure to have working speed
> readings I have ordered new fans from Noctua.
> A couple of Noctua NF-A8 PWM
> (http://www.newegg.com/Product/Product.aspx?Item=9SIA9PV3Y64158)
> A couple of Noctua NF-S12A
> (http://www.newegg.com/Product/Product.aspx?Item=N82E16835608038)

> Anybody having experience with these fans on supermicro boards?

-- 
Best Regards
Alexander
Oktober, 04 2016
........
[1] mid:20161003223940.7e954c22 at sleipner.datanom.net
........


From mir at miras.org  Tue Oct  4 09:59:12 2016
From: mir at miras.org (mir at miras.org)
Date: Tue, 04 Oct 2016 11:59:12 +0200
Subject: [OmniOS-discuss] Supermicro X10SLL-F
In-Reply-To: <294740579.20161004113105@tierarzt-mueller.de>
References: <20161003223940.7e954c22@sleipner.datanom.net>
	<294740579.20161004113105@tierarzt-mueller.de>
Message-ID: <e4a4a1478dc0dc7502309f6291d05499@miras.org>

On 2016-10-04 11:31, Alexander Lesle wrote:
> Hello Michael Rasmussen and List,
> 
> maybe the fans from supermicro are the better way.
> https://www.supermicro.com/support/resources/Thermal/#FAN
> 
I have solved the issue with Arctic fans. It was as mentioned the yellow 
wire which is separated on arctic fans which I had forgotten to attached 
the extender connector.

Everything works now except for the 12 cm fans which seems to spin at a 
lower RPM than Supermicro accepts for FANA header (means alarm condition 
activated and all fans running at full speed) so cross connecting with a 
8 cm fan have solved the issue.

I will however install my new Noctua fans when I get them since I have 
already made the purchase and because the current fans is near end of 
life anyway.

----

This mail was virus scanned and spam checked before delivery.
This mail is also DKIM signed. See header dkim-signature.


From timo.raatikainen at sourapple.fi  Tue Oct  4 12:20:36 2016
From: timo.raatikainen at sourapple.fi (Timo Raatikainen)
Date: Tue, 4 Oct 2016 15:20:36 +0300
Subject: [OmniOS-discuss] Supermicro X10SLL-F
In-Reply-To: <e4a4a1478dc0dc7502309f6291d05499@miras.org>
References: <20161003223940.7e954c22@sleipner.datanom.net>
	<294740579.20161004113105@tierarzt-mueller.de>
	<e4a4a1478dc0dc7502309f6291d05499@miras.org>
Message-ID: <B7D2B0B6-1D21-4CCC-AC56-D9B0DDB80636@sourapple.fi>

Hi,

You can change the fan thresholds on a Supermicro board using ipmitool. I also use Noctua fans and have successfully lowered the thresholds.

See https://calvin.me/quick-how-to-decrease-ipmi-fan-threshold/ for more specific instructions. Ipmitool should be available in OmniOS repo, but I havent checked lately. You can also use ipmitool from another computer via network.

--
Timo Raatikainen



> On 4 Oct 2016, at 12.59, mir at miras.org wrote:
> 
> On 2016-10-04 11:31, Alexander Lesle wrote:
>> Hello Michael Rasmussen and List,
>> maybe the fans from supermicro are the better way.
>> https://www.supermicro.com/support/resources/Thermal/#FAN
> I have solved the issue with Arctic fans. It was as mentioned the yellow wire which is separated on arctic fans which I had forgotten to attached the extender connector.
> 
> Everything works now except for the 12 cm fans which seems to spin at a lower RPM than Supermicro accepts for FANA header (means alarm condition activated and all fans running at full speed) so cross connecting with a 8 cm fan have solved the issue.
> 
> I will however install my new Noctua fans when I get them since I have already made the purchase and because the current fans is near end of life anyway.
> 
> ----
> 
> This mail was virus scanned and spam checked before delivery.
> This mail is also DKIM signed. See header dkim-signature.
> 
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss


From mir at miras.org  Tue Oct  4 15:53:41 2016
From: mir at miras.org (Michael Rasmussen)
Date: Tue, 4 Oct 2016 17:53:41 +0200
Subject: [OmniOS-discuss] Supermicro X10SLL-F
In-Reply-To: <B7D2B0B6-1D21-4CCC-AC56-D9B0DDB80636@sourapple.fi>
References: <20161003223940.7e954c22@sleipner.datanom.net>
	<294740579.20161004113105@tierarzt-mueller.de>
	<e4a4a1478dc0dc7502309f6291d05499@miras.org>
	<B7D2B0B6-1D21-4CCC-AC56-D9B0DDB80636@sourapple.fi>
Message-ID: <20161004175341.06132902@sleipner.datanom.net>

On Tue, 4 Oct 2016 15:20:36 +0300
Timo Raatikainen <timo.raatikainen at sourapple.fi> wrote:

> Hi,
> 
> You can change the fan thresholds on a Supermicro board using ipmitool. I also use Noctua fans and have successfully lowered the thresholds.
> 
> See https://calvin.me/quick-how-to-decrease-ipmi-fan-threshold/ for more specific instructions. Ipmitool should be available in OmniOS repo, but I havent checked lately. You can also use ipmitool from another computer via network.
> 
I have tried the solution from the article even did sensor thresh FANA
lower 100 100 100 without success. The problem is that the arctic f12 fans RPM is (for 12V) between 450 (0% PWM) and 1350 (100% PWM) which apparently confuses Supermicro IPMI since it interprets this to a RPM span -? to ? resulting in fan read error

And ipmitool is available in Omnios -> system/management/ipmitool


-- 
Hilsen/Regards
Michael Rasmussen

Get my public GnuPG keys:
michael <at> rasmussen <dot> cc
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E
mir <at> datanom <dot> net
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C
mir <at> miras <dot> org
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917
--------------------------------------------------------------
/usr/games/fortune -es says:
Parenthesise to avoid ambiguity.
            - The Elements of Programming Style (Kernighan & Plaugher)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 163 bytes
Desc: OpenPGP digital signature
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20161004/6095fb93/attachment.bin>

From skeltonr at btconnect.com  Tue Oct  4 18:33:26 2016
From: skeltonr at btconnect.com (Richard Skelton)
Date: Tue, 04 Oct 2016 19:33:26 +0100
Subject: [OmniOS-discuss] omniti/database/mysql-55 keeps on stopping
Message-ID: <57F3F5F6.3080703@btconnect.com>

Hi,
I have installed the
pkg://ms.omniti.com/omniti/database/mysql-55 at 5.5.50-0.151014:20160722T162252Z
in a zone on my OmniOS v11 r151018 system
Which has stopped and I don't see any core file.
Looking in the .err log file I can see my restart but no message why it
crashed.

160913 22:19:58  InnoDB: Waiting for the background threads to start
160913 22:19:59 InnoDB: 5.5.50 started; log sequence number 0
160913 22:19:59 [Note] Server hostname (bind-address): '0.0.0.0'; port: 3306
160913 22:19:59 [Note]   - '0.0.0.0' resolves to '0.0.0.0';
160913 22:19:59 [Note] Server socket created on IP: '0.0.0.0'.
160913 22:19:59 [Note] Event Scheduler: Loaded 0 events
160913 22:19:59 [Note] /opt/mysql55/bin/mysqld: ready for connections.
Version: '5.5.50'  socket: '/tmp/mysql.sock'  port: 3306  MySQL
Community Server (GPL)
161003 20:48:44 mysqld_safe Starting mysqld daemon with databases from
/opt/mysql55/data
161003 20:48:44 [Note] /opt/mysql55/bin/mysqld (mysqld 5.5.50) starting
as process 4098 ...
161003 20:48:44 [Note] Plugin 'FEDERATED' is disabled.
161003 20:48:44 InnoDB: The InnoDB memory heap is disabled
161003 20:48:44 InnoDB: Mutexes and rw_locks use GCC atomic builtins
161003 20:48:44 InnoDB: Compressed tables use zlib 1.2.3
161003 20:48:44 InnoDB: Initializing buffer pool, size = 128.0M
161003 20:48:44 InnoDB: Completed initialization of buffer pool
161003 20:48:44 InnoDB: highest supported file format is Barracuda.
InnoDB: The log sequence number in ibdata files does not match
InnoDB: the log sequence number in the ib_logfiles!
161003 20:48:44  InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files...
InnoDB: Restoring possible half-written data pages from the doublewrite
InnoDB: buffer...
161003 20:48:44  InnoDB: Waiting for the background threads to start
161003 20:48:45 InnoDB: 5.5.50 started; log sequence number 1595675
161003 20:48:45 [Note] Server hostname (bind-address): '0.0.0.0'; port: 3306
161003 20:48:45 [Note]   - '0.0.0.0' resolves to '0.0.0.0';
161003 20:48:45 [Note] Server socket created on IP: '0.0.0.0'.
161003 20:48:45 [Note] Event Scheduler: Loaded 0 events
161003 20:48:45 [Note] /opt/mysql55/bin/mysqld: ready for connections.


Can I set more detailed logging ?

Cheers
Richard.

From ghestin at hotmail.com  Wed Oct  5 13:50:59 2016
From: ghestin at hotmail.com (ludo zor)
Date: Wed, 5 Oct 2016 13:50:59 +0000
Subject: [OmniOS-discuss] omnios LTS hpaculi / hpssascli p410i support
Message-ID: <DB3PR05MB087316A494F4FAF57021DB06A5C40@DB3PR05MB0873.eurprd05.prod.outlook.com>

Hello
I migrating a dl370g6 solaris 10 , hpacucli 9.0-24.0 ( for monitoring raid ) to the last omnios lts .
When a I try all version of hpacucli or hpssascli on it , i always have the same message , "controller not found" .
Can you help me ?

Thanks

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20161005/06e5b79e/attachment.html>

From danmcd at omniti.com  Wed Oct  5 14:35:33 2016
From: danmcd at omniti.com (Dan McDonald)
Date: Wed, 5 Oct 2016 10:35:33 -0400
Subject: [OmniOS-discuss] omnios LTS hpaculi / hpssascli p410i support
In-Reply-To: <DB3PR05MB087316A494F4FAF57021DB06A5C40@DB3PR05MB0873.eurprd05.prod.outlook.com>
References: <DB3PR05MB087316A494F4FAF57021DB06A5C40@DB3PR05MB0873.eurprd05.prod.outlook.com>
Message-ID: <AF7DD884-0C0F-48C3-83DB-A9D3A3DEE729@omniti.com>


> On Oct 5, 2016, at 9:50 AM, ludo zor <ghestin at hotmail.com> wrote:
> 
> Hello
> I migrating a dl370g6 solaris 10 , hpacucli 9.0-24.0 ( for monitoring raid ) to the last omnios lts .
> When a I try all version of hpacucli or hpssascli on it , i always have the same message , "controller not found" .
> Can you help me ?
> 

PLEASE do not use Oracle Solaris drivers on OmniOS.  There are likely to be subtle incompatibilities.

If you cannot see your controller under the OmniOS LTS media, please see if the more recent OmniOS Stable or OmniOS Bloody release see it.  If they do, a backport of the driver may be possible (esp. to a paying customer).

Sorry,
Dan


From eric.sproul at circonus.com  Wed Oct  5 14:46:23 2016
From: eric.sproul at circonus.com (Eric Sproul)
Date: Wed, 5 Oct 2016 10:46:23 -0400
Subject: [OmniOS-discuss] omnios LTS hpaculi / hpssascli p410i support
In-Reply-To: <AF7DD884-0C0F-48C3-83DB-A9D3A3DEE729@omniti.com>
References: <DB3PR05MB087316A494F4FAF57021DB06A5C40@DB3PR05MB0873.eurprd05.prod.outlook.com>
	<AF7DD884-0C0F-48C3-83DB-A9D3A3DEE729@omniti.com>
Message-ID: <CAO8hXRADbmchJqbcAfaeFm9RQ9j86V0uyxqnOmjT-QZFtY41Bg@mail.gmail.com>

On Wed, Oct 5, 2016 at 10:35 AM, Dan McDonald <danmcd at omniti.com> wrote:
>
>> On Oct 5, 2016, at 9:50 AM, ludo zor <ghestin at hotmail.com> wrote:
>>
>> Hello
>> I migrating a dl370g6 solaris 10 , hpacucli 9.0-24.0 ( for monitoring raid ) to the last omnios lts .
>> When a I try all version of hpacucli or hpssascli on it , i always have the same message , "controller not found" .
>> Can you help me ?
>>
>
> PLEASE do not use Oracle Solaris drivers on OmniOS.  There are likely to be subtle incompatibilities.

I don't think these are drivers-- they sound like admin utilities,
though I'm not too familiar with HP hardware.  I would first check
that the illumos cpqary3 driver is attaching to the hardware:

# grep cpqary3 /etc/path_to_inst

If this shows nothing, then for some reason the driver did not attach.
The kernel log might have some messages related to that.

Can you install pkg:/system/pciutils and post the output of
`/usr/sbin/lspci -nn` for this machine?  The PCI ID for the HBA would
need to be one of the IDs that cpqary3 knows about, which are listed
in /etc/driver_aliases.

Eric

From daleg at omniti.com  Wed Oct  5 15:10:54 2016
From: daleg at omniti.com (Dale Ghent)
Date: Wed, 5 Oct 2016 11:10:54 -0400
Subject: [OmniOS-discuss] omnios LTS hpaculi / hpssascli p410i support
In-Reply-To: <DB3PR05MB087316A494F4FAF57021DB06A5C40@DB3PR05MB0873.eurprd05.prod.outlook.com>
References: <DB3PR05MB087316A494F4FAF57021DB06A5C40@DB3PR05MB0873.eurprd05.prod.outlook.com>
Message-ID: <D94FE1C1-066B-4B5D-A8D0-75FB8FA8438A@omniti.com>


> On Oct 5, 2016, at 9:50 AM, ludo zor <ghestin at hotmail.com> wrote:
> 
> Hello
> I migrating a dl370g6 solaris 10 , hpacucli 9.0-24.0 ( for monitoring raid ) to the last omnios lts .
> When a I try all version of hpacucli or hpssascli on it , i always have the same message , "controller not found" .
> Can you help me ?

You are using binaries and utilities made for Solaris 10 on OmniOS. There is no guarantee of compatibility, as these are two very different OSes that have grown apart over the years. It is possible that, while your binaries do execute on OmniOS, your hpacucli is looking for and trying to use a facility that has since changed or disappeared.

/dale

From geoffn at gnaa.net  Wed Oct  5 19:57:22 2016
From: geoffn at gnaa.net (Geoff Nordli)
Date: Wed, 5 Oct 2016 12:57:22 -0700
Subject: [OmniOS-discuss] upgrade from r151008
Message-ID: <e3b8f6cf-92ff-4333-a7c8-26607cd9d733@gnaa.net>

Hi.

Do you think I will be able to upgrade from r151008 to r151014 LTS?

What about going to current stable of r151018?

thanks,
Geoff /
/

/
/

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20161005/c44b976c/attachment.html>

From danmcd at omniti.com  Wed Oct  5 20:11:48 2016
From: danmcd at omniti.com (Dan McDonald)
Date: Wed, 5 Oct 2016 16:11:48 -0400
Subject: [OmniOS-discuss] upgrade from r151008
In-Reply-To: <e3b8f6cf-92ff-4333-a7c8-26607cd9d733@gnaa.net>
References: <e3b8f6cf-92ff-4333-a7c8-26607cd9d733@gnaa.net>
Message-ID: <B9B07CB6-D30D-4B9E-A636-2EC2A439991B@omniti.com>

Wow!  We stopped supporting 008 a long time ago.

Follow the upgrade directions on the wiki for 014 (involves a publisher change AND a package policy change) and you should be in good shape.

Should be little difference between 018 and 014 with respect to a jump from 018.  For both, make sure you've changed the OmniOS publisher policy to require signatures.  It should all be on the upgrade pages.

Dan 

Sent from my iPhone (typos, autocorrect, and all)

> On Oct 5, 2016, at 3:57 PM, Geoff Nordli <geoffn at gnaa.net> wrote:
> 
> Hi.
> 
> Do you think I will be able to upgrade from r151008 to r151014 LTS?
> 
> What about going to current stable of r151018? 
> 
> thanks,
> Geoff 
> 
> 
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20161005/a3b7a2ff/attachment-0001.html>

From geoffn at gnaa.net  Wed Oct  5 23:03:49 2016
From: geoffn at gnaa.net (Geoff Nordli)
Date: Wed, 5 Oct 2016 16:03:49 -0700
Subject: [OmniOS-discuss] upgrade from r151008
In-Reply-To: <B9B07CB6-D30D-4B9E-A636-2EC2A439991B@omniti.com>
References: <e3b8f6cf-92ff-4333-a7c8-26607cd9d733@gnaa.net>
	<B9B07CB6-D30D-4B9E-A636-2EC2A439991B@omniti.com>
Message-ID: <c10dcf54-6bb7-cb7e-fd08-ab774bdb95d7@gnaa.net>

It has been a good box.

But I should have upgraded it sooner than I did.

Always a risk when you wait too long between versions.

thanks!!

Geoff


On 2016-10-05 01:11 PM, Dan McDonald wrote:
> Wow!  We stopped supporting 008 a long time ago.
>
> Follow the upgrade directions on the wiki for 014 (involves a 
> publisher change AND a package policy change) and you should be in 
> good shape.
>
> Should be little difference between 018 and 014 with respect to a jump 
> from 018.  For both, make sure you've changed the OmniOS publisher 
> policy to require signatures.  It should all be on the upgrade pages.
>
> Dan
>
> Sent from my iPhone (typos, autocorrect, and all)
>
> On Oct 5, 2016, at 3:57 PM, Geoff Nordli <geoffn at gnaa.net 
> <mailto:geoffn at gnaa.net>> wrote:
>
>> Hi.
>>
>> Do you think I will be able to upgrade from r151008 to r151014 LTS?
>>
>> What about going to current stable of r151018?
>>
>> thanks,
>> Geoff /
>> /
>>
>> /
>> /
>>
>> _______________________________________________
>> OmniOS-discuss mailing list
>> OmniOS-discuss at lists.omniti.com <mailto:OmniOS-discuss at lists.omniti.com>
>> http://lists.omniti.com/mailman/listinfo/omnios-discuss

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20161005/76668f47/attachment.html>

From danmcd at omniti.com  Wed Oct  5 23:32:07 2016
From: danmcd at omniti.com (Dan McDonald)
Date: Wed, 5 Oct 2016 19:32:07 -0400
Subject: [OmniOS-discuss] upgrade from r151008
In-Reply-To: <c10dcf54-6bb7-cb7e-fd08-ab774bdb95d7@gnaa.net>
References: <e3b8f6cf-92ff-4333-a7c8-26607cd9d733@gnaa.net>
	<B9B07CB6-D30D-4B9E-A636-2EC2A439991B@omniti.com>
	<c10dcf54-6bb7-cb7e-fd08-ab774bdb95d7@gnaa.net>
Message-ID: <F383D5AF-B644-4A36-9351-42B285C79350@omniti.com>


> On Oct 5, 2016, at 7:03 PM, Geoff Nordli <geoffn at gnaa.net> wrote:
> 
> Always a risk when you wait too long between versions. 
> 
> 

We tested 006 --> 014 pre-release of 014.  008 --> 014 should be equally workable.

Dan


From johan.kragsterman at capvert.se  Thu Oct  6 09:00:58 2016
From: johan.kragsterman at capvert.se (Johan Kragsterman)
Date: Thu, 6 Oct 2016 11:00:58 +0200
Subject: [OmniOS-discuss] stmf trouble, crash and dump
Message-ID: <OF96A5BCAC.424ED968-ONC1258041.0030483C-C1258044.00318733@inse.com>

Hi!


Got a problem here a couple of days ago when I ran a snapshot stream over fibre channel on my home/business/devel server to the clone backup server.

Systems: OmniOS 5.11 ? ? omnios-r151018-95eaa7e on both systems, initiator on one, and and target on the other. Also same hardware: Dell precision workstation with dual xeon 6-cores and 96 GB registred ram, intel quad port gb nic, and qlogic QLE2462 HBA's.

Configured with one lun provisioned from the target/backup system to the initiator system as a backup lun, and that backup lun configured as a zpool in the initiator system. I should also say, that I run this Fc connection point-to-point, no switch is involved, and it's a single fibre pair, 10 m.

I did a zfs send/recv of a snapshot, and I thought it took a long time. It was around 67 GB. Then the initiator system crashed and dumped. It rebooted, and I got into it again without any trouble.
What I immediately saw was that the zpool "backpool" that was backed by the Fc lun was not present any longer. I made a zpool import, and it was back there again. I did another test, sent a much smaller snap, this was around 450 MB, and that worked fine, although I thought it took a lot of time.

I once again tried with the bigger snap, and same thing happened, system crashed and dumped. I got those two dump files, but I don't know wether this might be a problem on the target system or the initiator side.

I can provide access to dump files.

Here is some information from the two systems that I find interesting:

The initiator system, omni:

omni:

root at omni:/var/log# dmesg | grep qlc
Oct  2 18:33:08 omni qlc: [ID 439991 kern.info] NOTICE: Qlogic qlc(0,0): Loop OFFLINE


root at omni:/# dmesg | grep scsi
Oct  2 18:34:58 omni scsi: [ID 243001 kern.info] /pci at 19,0/pci8086,3410 at 9/pci1077,138 at 0/fp at 0,0 (fcp0):
Oct  2 18:34:58 omni genunix: [ID 408114 kern.info] /scsi_vhci/disk at g600144f0c648ae73000057ef6d370001 (sd5) offline
Oct  2 18:34:58 omni genunix: [ID 483743 kern.info] /scsi_vhci/disk at g600144f0c648ae73000057ef6d370001 (sd5) multipath status: failed: path 4 fp0/disk at w2101001b32a19a92,0 is offline
root at omni:/# dmesg | grep multipath
Oct  2 18:34:58 omni genunix: [ID 483743 kern.info] /scsi_vhci/disk at g600144f0c648ae73000057ef6d370001 (sd5) multipath status: failed: path 4 fp0/disk at w2101001b32a19a92,0 is offline

As you can see, the loop is marked offline at the occasion for the crash. But notably strange, there is also an info of a failed multipath...? Why this? There is no multipath here...

The target system, omni2:

root at omni2:/root# grep stmf /var/adm/messages
Oct  2 09:56:37 omni2 pseudo: [ID 129642 kern.info] pseudo-device: stmf_sbd0
Oct  2 09:56:37 omni2 genunix: [ID 936769 kern.info] stmf_sbd0 is /pseudo/stmf_sbd at 0
Oct  2 09:56:46 omni2 pseudo: [ID 129642 kern.info] pseudo-device: stmf0
Oct  2 09:56:46 omni2 genunix: [ID 936769 kern.info] stmf0 is /pseudo/stmf at 0
Oct  2 09:57:31 omni2 pseudo: [ID 129642 kern.info] pseudo-device: stmf0
Oct  2 09:57:31 omni2 genunix: [ID 936769 kern.info] stmf0 is /pseudo/stmf at 0
Oct  2 09:57:31 omni2 stmf_sbd: [ID 690249 kern.warning] WARNING: ioctl(DKIOCINFO) failed 25


There is this warning, ioctl(DKCINFO) failed 25, that I tried to find out what it is about, but not succeeded.


Perhaps it is just so simple that the Fc connection isn't good enough. The cable shouldn't be a problem, since it is brand new, but it could of coarse be something with the HBA's. I could get another cable for doing multipath, and see how that would work, but let's start with this first.

Best regards from/Med v?nliga h?lsningar fr?n

Johan Kragsterman

Capvert



From danmcd at omniti.com  Thu Oct  6 14:35:45 2016
From: danmcd at omniti.com (Dan McDonald)
Date: Thu, 6 Oct 2016 10:35:45 -0400
Subject: [OmniOS-discuss] stmf trouble, crash and dump
In-Reply-To: <OF96A5BCAC.424ED968-ONC1258041.0030483C-C1258044.00318733@inse.com>
References: <OF96A5BCAC.424ED968-ONC1258041.0030483C-C1258044.00318733@inse.com>
Message-ID: <531FE7BF-1D78-4AB2-BA7B-9C214A21E5F3@omniti.com>


> On Oct 6, 2016, at 5:00 AM, Johan Kragsterman <johan.kragsterman at capvert.se> wrote:
> 
> I can provide access to dump files.

Please share those, or at least show the "$c" output from mdb when you post a "kernel panic" email.

Thank you,
Dan


From johan.kragsterman at capvert.se  Thu Oct  6 16:00:39 2016
From: johan.kragsterman at capvert.se (Johan Kragsterman)
Date: Thu, 6 Oct 2016 18:00:39 +0200
Subject: [OmniOS-discuss] Ang:  stmf trouble, crash and dump
In-Reply-To: <OF96A5BCAC.424ED968-ONC1258041.0030483C-C1258044.00318733@inse.com>
References: <OF96A5BCAC.424ED968-ONC1258041.0030483C-C1258044.00318733@inse.com>
Message-ID: <OF5C14E196.4CF82F02-ONC1258044.0057F38B-C1258044.0057F38E@inse.com>


Hi!

Reponding my own mail instead of Dan's, because he didn't have history included. See further down the mdb output. It is obvious that the pool hang, but not why...


-----"OmniOS-discuss" <omnios-discuss-bounces at lists.omniti.com> skrev: -----
Till: omnios-discuss at lists.omniti.com
Fr?n: Johan Kragsterman 
S?nt av: "OmniOS-discuss" 
Datum: 2016-10-06 11:03
?rende: [OmniOS-discuss] stmf trouble, crash and dump

Hi!


Got a problem here a couple of days ago when I ran a snapshot stream over fibre channel on my home/business/devel server to the clone backup server.

Systems: OmniOS 5.11 ? ? omnios-r151018-95eaa7e on both systems, initiator on one, and and target on the other. Also same hardware: Dell precision workstation with dual xeon 6-cores and 96 GB registred ram, intel quad port gb nic, and qlogic QLE2462 HBA's.

Configured with one lun provisioned from the target/backup system to the initiator system as a backup lun, and that backup lun configured as a zpool in the initiator system. I should also say, that I run this Fc connection point-to-point, no switch is involved, and it's a single fibre pair, 10 m.

I did a zfs send/recv of a snapshot, and I thought it took a long time. It was around 67 GB. Then the initiator system crashed and dumped. It rebooted, and I got into it again without any trouble.
What I immediately saw was that the zpool "backpool" that was backed by the Fc lun was not present any longer. I made a zpool import, and it was back there again. I did another test, sent a much smaller snap, this was around 450 MB, and that worked fine, although I thought it took a lot of time.

I once again tried with the bigger snap, and same thing happened, system crashed and dumped. I got those two dump files, but I don't know wether this might be a problem on the target system or the initiator side.

I can provide access to dump files.

Here is some information from the two systems that I find interesting:

The initiator system, omni:

omni:

root at omni:/var/log# dmesg | grep qlc
Oct ?2 18:33:08 omni qlc: [ID 439991 kern.info] NOTICE: Qlogic qlc(0,0): Loop OFFLINE


root at omni:/# dmesg | grep scsi
Oct ?2 18:34:58 omni scsi: [ID 243001 kern.info] /pci at 19,0/pci8086,3410 at 9/pci1077,138 at 0/fp at 0,0 (fcp0):
Oct ?2 18:34:58 omni genunix: [ID 408114 kern.info] /scsi_vhci/disk at g600144f0c648ae73000057ef6d370001 (sd5) offline
Oct ?2 18:34:58 omni genunix: [ID 483743 kern.info] /scsi_vhci/disk at g600144f0c648ae73000057ef6d370001 (sd5) multipath status: failed: path 4 fp0/disk at w2101001b32a19a92,0 is offline
root at omni:/# dmesg | grep multipath
Oct ?2 18:34:58 omni genunix: [ID 483743 kern.info] /scsi_vhci/disk at g600144f0c648ae73000057ef6d370001 (sd5) multipath status: failed: path 4 fp0/disk at w2101001b32a19a92,0 is offline

As you can see, the loop is marked offline at the occasion for the crash. But notably strange, there is also an info of a failed multipath...? Why this? There is no multipath here...

The target system, omni2:

root at omni2:/root# grep stmf /var/adm/messages
Oct ?2 09:56:37 omni2 pseudo: [ID 129642 kern.info] pseudo-device: stmf_sbd0
Oct ?2 09:56:37 omni2 genunix: [ID 936769 kern.info] stmf_sbd0 is /pseudo/stmf_sbd at 0
Oct ?2 09:56:46 omni2 pseudo: [ID 129642 kern.info] pseudo-device: stmf0
Oct ?2 09:56:46 omni2 genunix: [ID 936769 kern.info] stmf0 is /pseudo/stmf at 0
Oct ?2 09:57:31 omni2 pseudo: [ID 129642 kern.info] pseudo-device: stmf0
Oct ?2 09:57:31 omni2 genunix: [ID 936769 kern.info] stmf0 is /pseudo/stmf at 0
Oct ?2 09:57:31 omni2 stmf_sbd: [ID 690249 kern.warning] WARNING: ioctl(DKIOCINFO) failed 25


There is this warning, ioctl(DKCINFO) failed 25, that I tried to find out what it is about, but not succeeded.


Perhaps it is just so simple that the Fc connection isn't good enough. The cable shouldn't be a problem, since it is brand new, but it could of coarse be something with the HBA's. I could get another cable for doing multipath, and see how that would work, but let's start with this first.




mdb output from the initiator system here. Exactly the same from both crash dumps. So no questions there, the question is why...? In the target system I only found that warning "stmf_sbd: [ID 690249 kern.warning] WARNING: ioctl(DKIOCINFO) failed 25"




root at omni:/var/crash/unknown# mdb unix.0 vmcore.0
Loading modules: [ unix genunix specfs dtrace mac cpu.generic uppc pcplusmp scsi_vhci zfs mpt sd ip hook neti sockfs arp usba uhci mm stmf stmf_sbd md lofs mpt_sas sata random idm cpc crypto kvm ufs logindmux nsmb ptm smbsrv nfs ]
> ::status
debugging crash dump vmcore.0 (64-bit) from omni
operating system: 5.11 omnios-r151018-95eaa7e (i86pc)
image uuid: 0fdcc7e7-aaf9-4d9d-cc75-ac766a3c3b5a
panic message: I/O to pool 'backpool' appears to be hung.
dump content: kernel pages only
> C
debugging crash dump vmcore.0 (64-bit) from omni
operating system: 5.11 omnios-r151018-95eaa7e (i86pc)
image uuid: 0fdcc7e7-aaf9-4d9d-cc75-ac766a3c3b5a
panic message: I/O to pool 'backpool' appears to be hung.
dump content: kernel pages only
>  ::panicinfo
             cpu               20
          thread ffffff007ba86c40
         message I/O to pool 'backpool' appears to be hung.
             rdi fffffffff7a72290
             rsi ffffff007ba869c0
             rdx ffffff007ba86c40
             rcx ffffff196891f274
              r8               20
              r9                a
             rax ffffff007ba869e0
             rbx ffffff25200971c8
             rbp ffffff007ba86a20
             r10                0
             r11 ffffff007ba868c0
             r12 ffffff19ae36b000
             r13       e984f1e438
             r14 ffffff2520096cc0
             r15 ffffff753a6df538
          fsbase fffffd7fff061a40
          gsbase ffffff195ca63a80
              ds               38
              es               38
              fs                0
              gs                0
       trapno                0     
             err                0
             rip fffffffffb860190
              cs               30
          rflags              246
             rsp ffffff007ba869a8
              ss               38
          gdt_hi                0
          gdt_lo         5000ffff
          idt_hi                0
          idt_lo         4000ffff
             ldt                0
            task               70
             cr0         8005003b
             cr2     7ffcb34d6618
             cr3          c000000
             cr4             26f8
> 



root at omni:/var/crash/unknown# mdb unix.1 vmcore.1
Loading modules: [ unix genunix specfs dtrace mac cpu.generic uppc pcplusmp scsi_vhci zfs mpt sd ip hook neti sockfs arp usba uhci mm stmf stmf_sbd idm mpt_sas sata cpc crypto md kvm random lofs ufs logindmux nsmb ptm smbsrv nfs ]
> ::status
debugging crash dump vmcore.1 (64-bit) from omni
operating system: 5.11 omnios-r151018-95eaa7e (i86pc)
image uuid: 748b6c72-5dec-c92c-a155-f1788f51b3fd
panic message: I/O to pool 'backpool' appears to be hung.
dump content: kernel pages only
> C
debugging crash dump vmcore.1 (64-bit) from omni
operating system: 5.11 omnios-r151018-95eaa7e (i86pc)
image uuid: 748b6c72-5dec-c92c-a155-f1788f51b3fd
panic message: I/O to pool 'backpool' appears to be hung.
dump content: kernel pages only
> ::panicinfo
             cpu                2
          thread ffffff007a4afc40
         message I/O to pool 'backpool' appears to be hung.
             rdi fffffffff7a72290
             rsi ffffff007a4af9c0
             rdx ffffff007a4afc40
             rcx ffffff1964e842ee
              r8               20
              r9                a
             rax ffffff007a4af9e0
             rbx ffffff1966da1188
             rbp ffffff007a4afa20
             r10                0
             r11 ffffff007a4af8c0
             r12 ffffff1997848000
             r13       e9840bec87
             r14 ffffff1966da0c80
             r15 ffffff19e954c718
          fsbase fffffd7fff072a40
          gsbase ffffff195c0e6040
              ds               4b
              es               4b
              fs                0
              gs                0
          trapno                0     
             err                0
             rip fffffffffb860190
              cs               30
          rflags              246
             rsp ffffff007a4af9a8
              ss               38
          gdt_hi                0
          gdt_lo         d000ffff
          idt_hi                0
          idt_lo         c000ffff
             ldt                0
            task               70
             cr0         8005003b
             cr2     7ffd18fc0568
             cr3          c400000
             cr4             26f8
> 






























Best regards from/Med v?nliga h?lsningar fr?n

Johan Kragsterman

Capvert


_______________________________________________
OmniOS-discuss mailing list
OmniOS-discuss at lists.omniti.com
http://lists.omniti.com/mailman/listinfo/omnios-discuss




From eric.sproul at circonus.com  Thu Oct  6 16:07:46 2016
From: eric.sproul at circonus.com (Eric Sproul)
Date: Thu, 6 Oct 2016 12:07:46 -0400
Subject: [OmniOS-discuss] Ang: stmf trouble, crash and dump
In-Reply-To: <OF5C14E196.4CF82F02-ONC1258044.0057F38B-C1258044.0057F38E@inse.com>
References: <OF96A5BCAC.424ED968-ONC1258041.0030483C-C1258044.00318733@inse.com>
	<OF5C14E196.4CF82F02-ONC1258044.0057F38B-C1258044.0057F38E@inse.com>
Message-ID: <CAO8hXRAQqsuwHVoZ6ucFfCK4jAb=BNSVDxC6tyTFNFv0fyNa9A@mail.gmail.com>

On Thu, Oct 6, 2016 at 12:00 PM, Johan Kragsterman
<johan.kragsterman at capvert.se> wrote:
>
> Hi!
>
> Reponding my own mail instead of Dan's, because he didn't have history included. See further down the mdb output. It is obvious that the pool hang, but not why...

As it happens, I just wrote a blog post about how to drill down on
this type of crash and determine if it's a single disk or multiple
that had I/O hangs: http://www.circonus.com/crash-dump-necromancy/

I'm happy to answer questions about that method, if needed.
Eric

From johan.kragsterman at capvert.se  Thu Oct  6 16:14:13 2016
From: johan.kragsterman at capvert.se (Johan Kragsterman)
Date: Thu, 6 Oct 2016 18:14:13 +0200
Subject: [OmniOS-discuss] Ang: Re:  Ang: stmf trouble, crash and dump
In-Reply-To: <CAO8hXRAQqsuwHVoZ6ucFfCK4jAb=BNSVDxC6tyTFNFv0fyNa9A@mail.gmail.com>
References: <CAO8hXRAQqsuwHVoZ6ucFfCK4jAb=BNSVDxC6tyTFNFv0fyNa9A@mail.gmail.com>,
	<OF96A5BCAC.424ED968-ONC1258041.0030483C-C1258044.00318733@inse.com>
	<OF5C14E196.4CF82F02-ONC1258044.0057F38B-C1258044.0057F38E@inse.com>
Message-ID: <OF35D11F7A.33360790-ONC1258044.0059318F-C1258044.00593190@inse.com>


Hi!


Thanks, Eric, I check it out...


-----Eric Sproul <eric.sproul at circonus.com> skrev: -----
Till: Johan Kragsterman <johan.kragsterman at capvert.se>
Fr?n: Eric Sproul <eric.sproul at circonus.com>
Datum: 2016-10-06 18:07
Kopia: omnios-discuss <omnios-discuss at lists.omniti.com>
?rende: Re: [OmniOS-discuss] Ang: stmf trouble, crash and dump

On Thu, Oct 6, 2016 at 12:00 PM, Johan Kragsterman
<johan.kragsterman at capvert.se> wrote:
>
> Hi!
>
> Reponding my own mail instead of Dan's, because he didn't have history included. See further down the mdb output. It is obvious that the pool hang, but not why...

As it happens, I just wrote a blog post about how to drill down on
this type of crash and determine if it's a single disk or multiple
that had I/O hangs: http://www.circonus.com/crash-dump-necromancy/

I'm happy to answer questions about that method, if needed.
Eric






From johan.kragsterman at capvert.se  Thu Oct  6 16:36:24 2016
From: johan.kragsterman at capvert.se (Johan Kragsterman)
Date: Thu, 6 Oct 2016 18:36:24 +0200
Subject: [OmniOS-discuss] Ang:  Ang: Re:  Ang: stmf trouble, crash and dump
In-Reply-To: <OF35D11F7A.33360790-ONC1258044.0059318F-C1258044.00593190@inse.com>
References: <OF35D11F7A.33360790-ONC1258044.0059318F-C1258044.00593190@inse.com>,
	<CAO8hXRAQqsuwHVoZ6ucFfCK4jAb=BNSVDxC6tyTFNFv0fyNa9A@mail.gmail.com>,
	<OF96A5BCAC.424ED968-ONC1258041.0030483C-C1258044.00318733@inse.com>
	<OF5C14E196.4CF82F02-ONC1258044.0057F38B-C1258044.0057F38E@inse.com>
Message-ID: <OF536D4D4E.89C6EEDD-ONC1258044.005B3948-C1258044.005B3949@inse.com>


Hi!


While I'm doing this, is it someone that knows something about that error msg from the target system I mentioned earlier??

"stmf_sbd: [ID 690249 kern.warning] WARNING: ioctl(DKIOCINFO) failed 25"


Best regards from/Med v?nliga h?lsningar fr?n

Johan Kragsterman

Capvert


-----"OmniOS-discuss" <omnios-discuss-bounces at lists.omniti.com> skrev: -----
Till: Eric Sproul <eric.sproul at circonus.com>
Fr?n: Johan Kragsterman 
S?nt av: "OmniOS-discuss" 
Datum: 2016-10-06 18:15
Kopia: omnios-discuss <omnios-discuss at lists.omniti.com>
?rende: [OmniOS-discuss] Ang: Re: Ang: stmf trouble, crash and dump

Hi!


Thanks, Eric, I check it out...


-----Eric Sproul <eric.sproul at circonus.com> skrev: -----
Till: Johan Kragsterman <johan.kragsterman at capvert.se>
Fr?n: Eric Sproul <eric.sproul at circonus.com>
Datum: 2016-10-06 18:07
Kopia: omnios-discuss <omnios-discuss at lists.omniti.com>
?rende: Re: [OmniOS-discuss] Ang: stmf trouble, crash and dump

On Thu, Oct 6, 2016 at 12:00 PM, Johan Kragsterman
<johan.kragsterman at capvert.se> wrote:
>
> Hi!
>
> Reponding my own mail instead of Dan's, because he didn't have history included. See further down the mdb output. It is obvious that the pool hang, but not why...

As it happens, I just wrote a blog post about how to drill down on
this type of crash and determine if it's a single disk or multiple
that had I/O hangs: http://www.circonus.com/crash-dump-necromancy/

I'm happy to answer questions about that method, if needed.
Eric





_______________________________________________
OmniOS-discuss mailing list
OmniOS-discuss at lists.omniti.com
http://lists.omniti.com/mailman/listinfo/omnios-discuss




From eric.sproul at circonus.com  Thu Oct  6 16:46:38 2016
From: eric.sproul at circonus.com (Eric Sproul)
Date: Thu, 6 Oct 2016 12:46:38 -0400
Subject: [OmniOS-discuss] Ang: Re: Ang: stmf trouble, crash and dump
In-Reply-To: <OF536D4D4E.89C6EEDD-ONC1258044.005B3948-C1258044.005B3949@inse.com>
References: <CAO8hXRAQqsuwHVoZ6ucFfCK4jAb=BNSVDxC6tyTFNFv0fyNa9A@mail.gmail.com>
	<OF96A5BCAC.424ED968-ONC1258041.0030483C-C1258044.00318733@inse.com>
	<OF5C14E196.4CF82F02-ONC1258044.0057F38B-C1258044.0057F38E@inse.com>
	<OF35D11F7A.33360790-ONC1258044.0059318F-C1258044.00593190@inse.com>
	<OF536D4D4E.89C6EEDD-ONC1258044.005B3948-C1258044.005B3949@inse.com>
Message-ID: <CAO8hXRBdN+8SvMqCtGMunNeRW2huLPorw7Tr+5-Sn25p-S8XMw@mail.gmail.com>

On Thu, Oct 6, 2016 at 12:36 PM, Johan Kragsterman
<johan.kragsterman at capvert.se> wrote:
>
> Hi!
>
>
> While I'm doing this, is it someone that knows something about that error msg from the target system I mentioned earlier??
>
> "stmf_sbd: [ID 690249 kern.warning] WARNING: ioctl(DKIOCINFO) failed 25"

DKIOCINFO is the macro for the ioctl that gets info from a disk:
http://src.illumos.org/source/xref/illumos-gate/usr/src/uts/common/sys/dkio.h#166

I am not a COMSTAR expert but I believe your warning is coming from here:
http://src.illumos.org/source/xref/illumos-gate/usr/src/uts/common/io/comstar/lu/stmf_sbd/sbd.c#1581

It looks like COMSTAR was trying to query one of the zvols, and
failed.  This is almost assuredly a consequence of the hung pool.

Eric

From danmcd at omniti.com  Thu Oct  6 16:50:20 2016
From: danmcd at omniti.com (Dan McDonald)
Date: Thu, 6 Oct 2016 12:50:20 -0400
Subject: [OmniOS-discuss] Ang:  Ang: Re:  Ang: stmf trouble,
 crash and dump
In-Reply-To: <OF536D4D4E.89C6EEDD-ONC1258044.005B3948-C1258044.005B3949@inse.com>
References: <OF35D11F7A.33360790-ONC1258044.0059318F-C1258044.00593190@inse.com>
	<CAO8hXRAQqsuwHVoZ6ucFfCK4jAb=BNSVDxC6tyTFNFv0fyNa9A@mail.gmail.com>
	<OF96A5BCAC.424ED968-ONC1258041.0030483C-C1258044.00318733@inse.com>
	<OF5C14E196.4CF82F02-ONC1258044.0057F38B-C1258044.0057F38E@inse.com>
	<OF536D4D4E.89C6EEDD-ONC1258044.005B3948-C1258044.005B3949@inse.com>
Message-ID: <12AF765A-81CC-44C1-AE0A-795B10F37972@omniti.com>


> On Oct 6, 2016, at 12:36 PM, Johan Kragsterman <johan.kragsterman at capvert.se> wrote:
> 
> 
> While I'm doing this, is it someone that knows something about that error msg from the target system I mentioned earlier??
> 
> "stmf_sbd: [ID 690249 kern.warning] WARNING: ioctl(DKIOCINFO) failed 25"
> 

Error 25 is "ENOTTY", aka. "Inappropriate ioctl for device":

http://src.illumos.org/source/xref/illumos-gate/usr/src/uts/common/sys/errno.h#77

The error itself comes because while STMF thinks the underlying storage is a zvol, apparently the DKIOCINFO ioctl failed with the above erro:

http://src.illumos.org/source/xref/illumos-gate/usr/src/uts/common/io/comstar/lu/stmf_sbd/sbd.c#1575

As Eric (who beat me to the punch here) said, it was likely caused by a hung pool.

Dan


From johan.kragsterman at capvert.se  Thu Oct  6 17:45:51 2016
From: johan.kragsterman at capvert.se (Johan Kragsterman)
Date: Thu, 6 Oct 2016 19:45:51 +0200
Subject: [OmniOS-discuss] Ang: Re:  Ang:  Ang: Re:  Ang: stmf trouble,
	crash and dump
In-Reply-To: <12AF765A-81CC-44C1-AE0A-795B10F37972@omniti.com>
References: <12AF765A-81CC-44C1-AE0A-795B10F37972@omniti.com>,
	<OF35D11F7A.33360790-ONC1258044.0059318F-C1258044.00593190@inse.com>
	<CAO8hXRAQqsuwHVoZ6ucFfCK4jAb=BNSVDxC6tyTFNFv0fyNa9A@mail.gmail.com>
	<OF96A5BCAC.424ED968-ONC1258041.0030483C-C1258044.00318733@inse.com>
	<OF5C14E196.4CF82F02-ONC1258044.0057F38B-C1258044.0057F38E@inse.com>
	<OF536D4D4E.89C6EEDD-ONC1258044.005B3948-C1258044.005B3949@inse.com>
Message-ID: <OF8340BCB7.2D054AAD-ONC1258044.0060A896-C1258044.006194E3@inse.com>


Hi!




-----Dan McDonald <danmcd at omniti.com> skrev: -----
Till: Johan Kragsterman <johan.kragsterman at capvert.se>, omnios-discuss <omnios-discuss at lists.omniti.com>
Fr?n: Dan McDonald <danmcd at omniti.com>
Datum: 2016-10-06 18:50
Kopia: Dan McDonald <danmcd at omniti.com>
?rende: Re: [OmniOS-discuss] Ang: Ang: Re: Ang: stmf trouble, crash and dump

> On Oct 6, 2016, at 12:36 PM, Johan Kragsterman <johan.kragsterman at capvert.se> wrote:
> 
> 
> While I'm doing this, is it someone that knows something about that error msg from the target system I mentioned earlier??
> 
> "stmf_sbd: [ID 690249 kern.warning] WARNING: ioctl(DKIOCINFO) failed 25"
> 

Error 25 is "ENOTTY", aka. "Inappropriate ioctl for device":

http://src.illumos.org/source/xref/illumos-gate/usr/src/uts/common/sys/errno.h#77

The error itself comes because while STMF thinks the underlying storage is a zvol, apparently the DKIOCINFO ioctl failed with the above erro:

http://src.illumos.org/source/xref/illumos-gate/usr/src/uts/common/io/comstar/lu/stmf_sbd/sbd.c#1575

As Eric (who beat me to the punch here) said, it was likely caused by a hung pool.

Dan






No, it is the other way around, actually! This error comes from the target system, not the system that crashed, that was the intitiator system. The underlying device IS a zvol on the target system!
So the question I ask myself, since I saw this error, is if this could be a/the problem that caused the lu/pool on the initiator system to go off line? Though there were also the loop offline  msg on the initiator:

root at omni:/var/log# dmesg | grep qlc
Oct  2 18:33:08 omni qlc: [ID 439991 kern.info] NOTICE: Qlogic qlc(0,0): Loop OFFLINE

And what puzzles me is the multipath messages, because there is no multipath devices on the system:

root at omni:/# dmesg | grep scsi
Oct  2 18:34:58 omni scsi: [ID 243001 kern.info] /pci at 19,0/pci8086,3410 at 9/pci1077,138 at 0/fp at 0,0 (fcp0):
Oct  2 18:34:58 omni genunix: [ID 408114 kern.info] /scsi_vhci/disk at g600144f0c648ae73000057ef6d370001 (sd5) offline
Oct  2 18:34:58 omni genunix: [ID 483743 kern.info] /scsi_vhci/disk at g600144f0c648ae73000057ef6d370001 (sd5) multipath status: failed: path 4 fp0/disk at w2101001b32a19a92,0 is offline
root at omni:/# dmesg | grep multipath
Oct  2 18:34:58 omni genunix: [ID 483743 kern.info] /scsi_vhci/disk at g600144f0c648ae73000057ef6d370001 (sd5) multipath status: failed: path 4 fp0/disk at w2101001b32a19a92,0 is offline

Johan




From danmcd at omniti.com  Fri Oct  7 01:02:47 2016
From: danmcd at omniti.com (Dan McDonald)
Date: Thu, 6 Oct 2016 21:02:47 -0400
Subject: [OmniOS-discuss] Bloody update -- NOW INCLUDES LX Beta SUPPORT
Message-ID: <0B830A7A-6FA6-46FA-AC6C-C138C13212E5@omniti.com>

It's here!

LX Support is now folded into illumos-omnios:master, and therefore is now ready for OmniOS bloody updates... like this one.

uname -v says omnios-master-3fd4b56, and that is the commit of illumos-omnios:master which generated this update.  omnios-build is at 5f132cf.

New with this update:

* LX Beta SUPPORT -- If you've been following the LX threads here, you know what's been happening.  A new README.OmniOS on the top-level of illumos-omnios covers what was once in the "io-lx" gate.  Going forward, this support is considered to be Beta.  This will include when r151020 ships.  We need to sort out various administrative issues, as well as keep up with Joyent's progress.  Feedback will be appreciated.

* NSS is now 3.26

* libdbus-glib is now 0.108

* dbus is 1.11.4

* diffutils is now 3.5

* expat is now 2.2.0

* gawk is now 4.1.4

* gettext is now 0.19.8.1

* Git is now 2.10.0

* gnu-make is now 4.2.1

* gnump/gmp is now 6.1.1


The http://pkg.omniti.com/omnios/LX/ repo is now obsolete.  For this update, it matches bloody, but going forward it will not receive updates until the next bloody cycle, IF AT ALL.

I've also updated release media, as seen here:  https://omnios.omniti.com/wiki.php/Installation

Happy updating, and happy LX-ing!
Dan


From bfriesen at simple.dallas.tx.us  Fri Oct  7 13:43:46 2016
From: bfriesen at simple.dallas.tx.us (Bob Friesenhahn)
Date: Fri, 7 Oct 2016 08:43:46 -0500 (CDT)
Subject: [OmniOS-discuss] Bloody update -- NOW INCLUDES LX Beta SUPPORT
In-Reply-To: <0B830A7A-6FA6-46FA-AC6C-C138C13212E5@omniti.com>
References: <0B830A7A-6FA6-46FA-AC6C-C138C13212E5@omniti.com>
Message-ID: <alpine.GSO.2.20.1610070836420.3266@freddy.simplesystems.org>

On Thu, 6 Oct 2016, Dan McDonald wrote:
>
> LX Support is now folded into illumos-omnios:master, and therefore is now ready for OmniOS bloody updates... like this one.

This is very exciting.

Is anyone able to confirm if programs compiled GCC or Clang with 
Address Sanitizer (ASAN, https://code.google.com/p/address-sanitizer/) 
enabled are able to function properly in a LX zone?  Programs compiled 
with ASAN use 20GB of virtual memory.  I am not sure if any special OS 
run-time support is required.

I ask this because my main reason to use a Linux OS (vs some other OS 
like OmniOS) is to check my software using ASAN and valgrind for 
errors.

Bob
-- 
Bob Friesenhahn
bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/

From garland at bustle.com  Fri Oct  7 15:44:38 2016
From: garland at bustle.com (Garland McAlexander)
Date: Fri, 7 Oct 2016 11:44:38 -0400
Subject: [OmniOS-discuss] Issues with 10GbE - X540
Message-ID: <CAMUDemsjCcQxje1Mj2gqVuK3sB8zOmQN_4nYeg0RrWgt2YGgDw@mail.gmail.com>

Hi All,

I'm unable to link at 10GbE with my onboard X540 (X10DRW-iT). It works
perfectly fine at 10GbE if using Windows or CentOS on the same machine.
Just to be safe, I've tried multiple switches and Cat6a cabling.

Any tips on how to get this going?

-- 

Garland McAlexander

Technology Manager

E: garland at bustle.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20161007/e5324c5d/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 1758 bytes
Desc: not available
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20161007/e5324c5d/attachment.png>

From daleg at omniti.com  Fri Oct  7 17:07:38 2016
From: daleg at omniti.com (Dale Ghent)
Date: Fri, 7 Oct 2016 13:07:38 -0400
Subject: [OmniOS-discuss] Issues with 10GbE - X540
In-Reply-To: <CAMUDemsjCcQxje1Mj2gqVuK3sB8zOmQN_4nYeg0RrWgt2YGgDw@mail.gmail.com>
References: <CAMUDemsjCcQxje1Mj2gqVuK3sB8zOmQN_4nYeg0RrWgt2YGgDw@mail.gmail.com>
Message-ID: <E4249D67-A9B8-4BFF-90FD-62A8245D4CC6@omniti.com>

On Oct 7, 2016, at 11:44 AM, Garland McAlexander <garland at bustle.com> wrote:
> 
> Hi All,
> 
> I'm unable to link at 10GbE with my onboard X540 (X10DRW-iT). It works perfectly fine at 10GbE if using Windows or CentOS on the same machine. Just to be safe, I've tried multiple switches and Cat6a cabling.
> 
> Any tips on how to get this going?

Are you getting a 1Gb link instead of a 10Gb link?

Can you run this:

dladm set-linkprop en_10gfdx_cap=1 ixgbeN

And check the link speed afterwards (dladm show-phys)

/dale


From mtalbott at lji.org  Fri Oct  7 19:02:29 2016
From: mtalbott at lji.org (Michael Talbott)
Date: Fri, 7 Oct 2016 12:02:29 -0700
Subject: [OmniOS-discuss] extended attributes in LX zone
In-Reply-To: <ECE32D93-C067-4898-893C-E2ECB5496B9C@omniti.com>
References: <4F6BE78F-D4F9-4C7A-B024-F925023B86CF@lji.org>
	<C4180B3F-2964-438C-9828-12EBA0C1AB5E@omniti.com>
	<7DCFD721-221D-42C7-A497-A4A7E323B3FB@lji.org>
	<ECE32D93-C067-4898-893C-E2ECB5496B9C@omniti.com>
Message-ID: <414911B0-9721-4B8B-9E7A-E669427669CB@lji.org>

Bummer. Oh well. Just fyi, I have BeeGFS running successfully on OmniOS in LX zones and is blazing fast. All of the services have been running in  LX zones just fine so far except for the metadata server due to EA needs. So I just put the metadata server in a KVM instance until hopefully some day down the road xattrs are made available to LX zones ;)

THANK YOU LX ZONES!!!


> On Sep 30, 2016, at 8:52 PM, Dale Ghent <daleg at omniti.com> wrote:
> 
> 
>> On Sep 30, 2016, at 6:57 PM, Michael Talbott <mtalbott at lji.org> wrote:
>> 
>> Bummer. Oh well. How about presenting a zvol as a raw device like shown here: http://docs.oracle.com/cd/E19253-01/819-5461/gbebi/index.html
>> 
>> I did this, shutdown and rebooted the zone.. and the path the the volume appears under /dev/zvol/dsk/ssdpool/ but that directory is empty when it should contain the name of my zvol :(
>> 
>> I figure if I can get that working, then I could format that vol in ext4 so it'd have a way of properly handling xattrs (or so I hope).. Otherwise.. I guess I could present a zvol as an iscsi volume to the linux host/zone/entity and then format that ext4 over iscsi, but I'd rather not add the networking stack in there if I don't have to.
> 
> Negative, won't work because the illumos kernel doesn't grok ext4.
> 
> In Linux-land, the get/setxattr calls are made from an app, and those calls are handed in glibc, which turn around and execute the linux kernel syscall for getting and setting xattrs.
> 
> This syscalls then run the vfs_get/setxattr VFS layer op that each filesystem in Linux implements if it supports xattrs, which then in turn do the FS-specfic actions in the relevant FS driver to get and set xattrs on the file's on-disk structure.
> 
> For xattrs to work with LX brands, the get/setxattr functions in lx_xattr.c would need to be more complete and provide a direct translation layer to ZFS-style xattrs.
> 
> /dale


From daleg at omniti.com  Fri Oct  7 19:46:00 2016
From: daleg at omniti.com (Dale Ghent)
Date: Fri, 7 Oct 2016 15:46:00 -0400
Subject: [OmniOS-discuss] extended attributes in LX zone
In-Reply-To: <414911B0-9721-4B8B-9E7A-E669427669CB@lji.org>
References: <4F6BE78F-D4F9-4C7A-B024-F925023B86CF@lji.org>
	<C4180B3F-2964-438C-9828-12EBA0C1AB5E@omniti.com>
	<7DCFD721-221D-42C7-A497-A4A7E323B3FB@lji.org>
	<ECE32D93-C067-4898-893C-E2ECB5496B9C@omniti.com>
	<414911B0-9721-4B8B-9E7A-E669427669CB@lji.org>
Message-ID: <09FBAFD3-EA71-4A8E-ABF9-0DBBD43501B2@omniti.com>


Good to hear. With ZFS gaining traction in the Linux world, you might want to talk to the BeeGFS owners/maintainers about making their metadata server more filesystem-agnostic.

> On Oct 7, 2016, at 3:02 PM, Michael Talbott <mtalbott at lji.org> wrote:
> 
> Bummer. Oh well. Just fyi, I have BeeGFS running successfully on OmniOS in LX zones and is blazing fast. All of the services have been running in  LX zones just fine so far except for the metadata server due to EA needs. So I just put the metadata server in a KVM instance until hopefully some day down the road xattrs are made available to LX zones ;)
> 
> THANK YOU LX ZONES!!!
> 
> 
>> On Sep 30, 2016, at 8:52 PM, Dale Ghent <daleg at omniti.com> wrote:
>> 
>> 
>>> On Sep 30, 2016, at 6:57 PM, Michael Talbott <mtalbott at lji.org> wrote:
>>> 
>>> Bummer. Oh well. How about presenting a zvol as a raw device like shown here: http://docs.oracle.com/cd/E19253-01/819-5461/gbebi/index.html
>>> 
>>> I did this, shutdown and rebooted the zone.. and the path the the volume appears under /dev/zvol/dsk/ssdpool/ but that directory is empty when it should contain the name of my zvol :(
>>> 
>>> I figure if I can get that working, then I could format that vol in ext4 so it'd have a way of properly handling xattrs (or so I hope).. Otherwise.. I guess I could present a zvol as an iscsi volume to the linux host/zone/entity and then format that ext4 over iscsi, but I'd rather not add the networking stack in there if I don't have to.
>> 
>> Negative, won't work because the illumos kernel doesn't grok ext4.
>> 
>> In Linux-land, the get/setxattr calls are made from an app, and those calls are handed in glibc, which turn around and execute the linux kernel syscall for getting and setting xattrs.
>> 
>> This syscalls then run the vfs_get/setxattr VFS layer op that each filesystem in Linux implements if it supports xattrs, which then in turn do the FS-specfic actions in the relevant FS driver to get and set xattrs on the file's on-disk structure.
>> 
>> For xattrs to work with LX brands, the get/setxattr functions in lx_xattr.c would need to be more complete and provide a direct translation layer to ZFS-style xattrs.
>> 
>> /dale
> 


From robertfantini at gmail.com  Sat Oct  8 01:27:05 2016
From: robertfantini at gmail.com (Robert Fantini)
Date: Fri, 7 Oct 2016 21:27:05 -0400
Subject: [OmniOS-discuss] using proxmox in a kvm running on napp-it/omnios
Message-ID: <CALnM7znr3Je2yA3j7JejVWGX=btsQ-zrL+dTJ8ZbuhkUOJewuw@mail.gmail.com>

 We use proxmox and  napp-it/omnios zfs iscsi storage  for our 30 KVM' s
and 10 LXC's .

I want to cut down the amount of hardware we use. currently that is 2
storage and 3 pve systems.

 I think it should be possible  to run proxmox with in a kvm running on
omnios . then using local storage shared with iscsi to proxmox.

 I've a couple of quesitons:

 Is there anything wrong with going down that path?

 Is there documentation for setting up a kvm and using an iso file to
install operating system using omnios ?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20161007/39741a83/attachment.html>

From alka at hfg-gmuend.de  Sat Oct  8 08:35:47 2016
From: alka at hfg-gmuend.de (=?UTF-8?Q?G=c3=bcnther_Alka?=)
Date: Sat, 8 Oct 2016 10:35:47 +0200
Subject: [OmniOS-discuss] Plex mediaserver in an LX zone
In-Reply-To: <09FBAFD3-EA71-4A8E-ABF9-0DBBD43501B2@omniti.com>
References: <4F6BE78F-D4F9-4C7A-B024-F925023B86CF@lji.org>
	<C4180B3F-2964-438C-9828-12EBA0C1AB5E@omniti.com>
	<7DCFD721-221D-42C7-A497-A4A7E323B3FB@lji.org>
	<ECE32D93-C067-4898-893C-E2ECB5496B9C@omniti.com>
	<414911B0-9721-4B8B-9E7A-E669427669CB@lji.org>
	<09FBAFD3-EA71-4A8E-ABF9-0DBBD43501B2@omniti.com>
Message-ID: <98277e8c-f0dd-77b7-ee27-309d2ae7139f@hfg-gmuend.de>

about LX
I have not yet worked with this concept.
Main problem seems the lack of documentation, blogs and howtos or a 
simple ready to use example for OmniOS.

A good start would be a documented example like a Plex LX mediaserver.
Has anyone used this in SmartOS and can give some informations about 
doing the same with OmniOS? I will add a support menu for handling and 
management of LX containers in napp-it then as I suppose that many non 
storage related features and add-ons are LX containers in future.

see http://lightsandshapes.com/plex-on-smartos.html

Gea/ napp-it.org



From mir at miras.org  Sat Oct  8 10:40:31 2016
From: mir at miras.org (Michael Rasmussen)
Date: Sat, 8 Oct 2016 12:40:31 +0200
Subject: [OmniOS-discuss] using proxmox in a kvm running on
 napp-it/omnios
In-Reply-To: <CALnM7znr3Je2yA3j7JejVWGX=btsQ-zrL+dTJ8ZbuhkUOJewuw@mail.gmail.com>
References: <CALnM7znr3Je2yA3j7JejVWGX=btsQ-zrL+dTJ8ZbuhkUOJewuw@mail.gmail.com>
Message-ID: <20161008124031.55d3aa9f@sleipner.datanom.net>

On Fri, 7 Oct 2016 21:27:05 -0400
Robert Fantini <robertfantini at gmail.com> wrote:

> 
>  I think it should be possible  to run proxmox with in a kvm running on
> omnios . then using local storage shared with iscsi to proxmox.
> 
Since Omnios does not support nested KVM you will only be able to run
containers.

-- 
Hilsen/Regards
Michael Rasmussen

Get my public GnuPG keys:
michael <at> rasmussen <dot> cc
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E
mir <at> datanom <dot> net
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C
mir <at> miras <dot> org
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917
--------------------------------------------------------------
/usr/games/fortune -es says:
The future is a race between education and catastrophe.
		-- H. G. Wells
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 163 bytes
Desc: OpenPGP digital signature
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20161008/9e4c4d52/attachment-0001.bin>

From denis at denisandyuki.net  Sat Oct  8 12:28:19 2016
From: denis at denisandyuki.net (Denis Cheong)
Date: Sat, 8 Oct 2016 23:28:19 +1100
Subject: [OmniOS-discuss] Plex mediaserver in an LX zone
In-Reply-To: <98277e8c-f0dd-77b7-ee27-309d2ae7139f@hfg-gmuend.de>
References: <4F6BE78F-D4F9-4C7A-B024-F925023B86CF@lji.org>
	<C4180B3F-2964-438C-9828-12EBA0C1AB5E@omniti.com>
	<7DCFD721-221D-42C7-A497-A4A7E323B3FB@lji.org>
	<ECE32D93-C067-4898-893C-E2ECB5496B9C@omniti.com>
	<414911B0-9721-4B8B-9E7A-E669427669CB@lji.org>
	<09FBAFD3-EA71-4A8E-ABF9-0DBBD43501B2@omniti.com>
	<98277e8c-f0dd-77b7-ee27-309d2ae7139f@hfg-gmuend.de>
Message-ID: <CABOCf+iaGGXaHBdGrt3U6Leh8i3qGJUfKPOvun441zJ6bhK0VQ@mail.gmail.com>

I am running Plex within an LX zone, there is very little that is
particularly special about it.  The article you referred to is a good start.

Publish any required file systems over lofs, configure your file ownership
and permissions correctly, and just run it as if it's Ubuntu.




On 8 October 2016 at 19:35, G?nther Alka <alka at hfg-gmuend.de> wrote:

> about LX
> I have not yet worked with this concept.
> Main problem seems the lack of documentation, blogs and howtos or a simple
> ready to use example for OmniOS.
>
> A good start would be a documented example like a Plex LX mediaserver.
> Has anyone used this in SmartOS and can give some informations about doing
> the same with OmniOS? I will add a support menu for handling and management
> of LX containers in napp-it then as I suppose that many non storage related
> features and add-ons are LX containers in future.
>
> see http://lightsandshapes.com/plex-on-smartos.html
>
> Gea/ napp-it.org
>
>
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20161008/53c5caac/attachment.html>

From skeltonr at btconnect.com  Tue Oct 11 07:47:31 2016
From: skeltonr at btconnect.com (Richard Skelton)
Date: Tue, 11 Oct 2016 08:47:31 +0100
Subject: [OmniOS-discuss] LX zone console broken
Message-ID: <57FC9913.3080609@btconnect.com>

Hi,
I have a working LX zone which I am using to run Review Board:
https://www.reviewboard.org/news <https://www.reviewboard.org/news/>
I am using the bloody ISO image I downloaded at the weekend.
The only problem I have is the console login prompt fails.
If i do :-
# zlogin -C lx0
I see a no prompt.
I can access the zone with :-
# zlogin lx0
Looking in /var/log/messages I can see :-
Oct 10 20:35:13 lx0 /sbin/mingetty[1371]: console: vhangup() failed
Oct 10 20:35:18 lx0 init: tty (console) main process (1371) terminated
with status 1
Oct 10 20:35:18 lx0 init: tty (console) main process ended, respawning
Oct 10 20:35:18 lx0 /sbin/mingetty[1374]: console: vhangup() failed

What am I missing?

Cheers

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20161011/164987c8/attachment.html>

From danmcd at omniti.com  Tue Oct 11 14:13:03 2016
From: danmcd at omniti.com (Dan McDonald)
Date: Tue, 11 Oct 2016 10:13:03 -0400
Subject: [OmniOS-discuss] LX zone console broken
In-Reply-To: <57FC9913.3080609@btconnect.com>
References: <57FC9913.3080609@btconnect.com>
Message-ID: <1B02D76B-1341-4559-B166-DAC6AEF8ED26@omniti.com>


> On Oct 11, 2016, at 3:47 AM, Richard Skelton <skeltonr at btconnect.com> wrote:
> 
> Hi,
> I have a working LX zone which I am using to run Review Board: https://www.reviewboard.org/news
> I am using the bloody ISO image I downloaded at the weekend.
> The only problem I have is the console login prompt fails.
> If i do :-
> # zlogin -C lx0
> I see a no prompt.
> I can access the zone with :-
> # zlogin lx0
> Looking in /var/log/messages I can see :-
> Oct 10 20:35:13 lx0 /sbin/mingetty[1371]: console: vhangup() failed
> Oct 10 20:35:18 lx0 init: tty (console) main process (1371) terminated with status 1
> Oct 10 20:35:18 lx0 init: tty (console) main process ended, respawning
> Oct 10 20:35:18 lx0 /sbin/mingetty[1374]: console: vhangup() failed

Which image did you install?

I've seen lots of console problems, even on SmartOS.

Dan


From skeltonr at btconnect.com  Tue Oct 11 16:03:15 2016
From: skeltonr at btconnect.com (Richard Skelton)
Date: Tue, 11 Oct 2016 17:03:15 +0100
Subject: [OmniOS-discuss] LX zone console broken
In-Reply-To: <1B02D76B-1341-4559-B166-DAC6AEF8ED26@omniti.com>
References: <57FC9913.3080609@btconnect.com>
	<1B02D76B-1341-4559-B166-DAC6AEF8ED26@omniti.com>
Message-ID: <57FD0D43.6020705@btconnect.com>

Hi Dan,
I used :-
curl -o centos6.zss.gz
https://images.joyent.com/images/5b7e86e4-2797-11e6-b7d7-cbe6a1a48791/file
zoneadm -z lx0 install -s /root/centos6.zss.gz
zoneadm -z lx0 boot;zlogin -C lx0

Dan McDonald wrote:
>> On Oct 11, 2016, at 3:47 AM, Richard Skelton <skeltonr at btconnect.com> wrote:
>>
>> Hi,
>> I have a working LX zone which I am using to run Review Board: https://www.reviewboard.org/news
>> I am using the bloody ISO image I downloaded at the weekend.
>> The only problem I have is the console login prompt fails.
>> If i do :-
>> # zlogin -C lx0
>> I see a no prompt.
>> I can access the zone with :-
>> # zlogin lx0
>> Looking in /var/log/messages I can see :-
>> Oct 10 20:35:13 lx0 /sbin/mingetty[1371]: console: vhangup() failed
>> Oct 10 20:35:18 lx0 init: tty (console) main process (1371) terminated with status 1
>> Oct 10 20:35:18 lx0 init: tty (console) main process ended, respawning
>> Oct 10 20:35:18 lx0 /sbin/mingetty[1374]: console: vhangup() failed
>>     
>
> Which image did you install?
>
> I've seen lots of console problems, even on SmartOS.
>
> Dan
>
>
>   
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20161011/8455b8ed/attachment.html>

From skeltonr at btconnect.com  Tue Oct 11 16:27:35 2016
From: skeltonr at btconnect.com (Richard Skelton)
Date: Tue, 11 Oct 2016 17:27:35 +0100
Subject: [OmniOS-discuss] LX zone console broken
In-Reply-To: <1B02D76B-1341-4559-B166-DAC6AEF8ED26@omniti.com>
References: <57FC9913.3080609@btconnect.com>
	<1B02D76B-1341-4559-B166-DAC6AEF8ED26@omniti.com>
Message-ID: <57FD12F7.1010704@btconnect.com>

Hi Dan,
I used :-
curl -o centos6.zss.gz
https://images.joyent.com/images/5b7e86e4-2797-11e6-b7d7-cbe6a1a48791/file
zoneadm -z lx0 install -s /root/centos6.zss.gz
zoneadm -z lx0 boot;zlogin -C lx0

Dan McDonald wrote:
>> On Oct 11, 2016, at 3:47 AM, Richard Skelton <skeltonr at btconnect.com> wrote:
>>
>> Hi,
>> I have a working LX zone which I am using to run Review Board: https://www.reviewboard.org/news
>> I am using the bloody ISO image I downloaded at the weekend.
>> The only problem I have is the console login prompt fails.
>> If i do :-
>> # zlogin -C lx0
>> I see a no prompt.
>> I can access the zone with :-
>> # zlogin lx0
>> Looking in /var/log/messages I can see :-
>> Oct 10 20:35:13 lx0 /sbin/mingetty[1371]: console: vhangup() failed
>> Oct 10 20:35:18 lx0 init: tty (console) main process (1371) terminated with status 1
>> Oct 10 20:35:18 lx0 init: tty (console) main process ended, respawning
>> Oct 10 20:35:18 lx0 /sbin/mingetty[1374]: console: vhangup() failed
>>     
>
> Which image did you install?
>
> I've seen lots of console problems, even on SmartOS.
>
> Dan
>
>
>   
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20161011/68d8c7db/attachment.html>

From danmcd at omniti.com  Tue Oct 11 16:52:37 2016
From: danmcd at omniti.com (Dan McDonald)
Date: Tue, 11 Oct 2016 12:52:37 -0400
Subject: [OmniOS-discuss] LX zone console broken
In-Reply-To: <57FD12F7.1010704@btconnect.com>
References: <57FC9913.3080609@btconnect.com>
	<1B02D76B-1341-4559-B166-DAC6AEF8ED26@omniti.com>
	<57FD12F7.1010704@btconnect.com>
Message-ID: <32CA0CA0-4CE1-40D8-BFE7-A6FE9CA7F659@omniti.com>


> On Oct 11, 2016, at 12:27 PM, Richard Skelton <skeltonr at btconnect.com> wrote:
> 
> Hi Dan,
> I used :-
> curl -o centos6.zss.gz https://images.joyent.com/images/5b7e86e4-2797-11e6-b7d7-cbe6a1a48791/file
> zoneadm -z lx0 install -s /root/centos6.zss.gz
> zoneadm -z lx0 boot;zlogin -C lx0

Yeah, I'm seeing this too on my CentOS 6 zone as well.

I'm going to check with SmartOS, and see if this is something I fubared, or if it's an upstream (i.e. their) problem too.  I suspect I mismerged something.  It's fairly new, because I swear I used to see CentOS 6 consoles during the io-lx days.

Dan


From danmcd at omniti.com  Tue Oct 11 17:01:19 2016
From: danmcd at omniti.com (Dan McDonald)
Date: Tue, 11 Oct 2016 13:01:19 -0400
Subject: [OmniOS-discuss] LX zone console broken
In-Reply-To: <32CA0CA0-4CE1-40D8-BFE7-A6FE9CA7F659@omniti.com>
References: <57FC9913.3080609@btconnect.com>
	<1B02D76B-1341-4559-B166-DAC6AEF8ED26@omniti.com>
	<57FD12F7.1010704@btconnect.com>
	<32CA0CA0-4CE1-40D8-BFE7-A6FE9CA7F659@omniti.com>
Message-ID: <77EFCE94-0C91-408E-9F79-E679DEAA8CB2@omniti.com>


> On Oct 11, 2016, at 12:52 PM, Dan McDonald <danmcd at omniti.com> wrote:
> 
> I'm going to check with SmartOS, and see if this is something I fubared, or if it's an upstream (i.e. their) problem too.  I suspect I mismerged something.  It's fairly new, because I swear I used to see CentOS 6 consoles during the io-lx days.

Not mismerge, but missing bugfix:

    OS-5685 LTP vhangup02 fails and passes

Could you do me a favor and test a replacement lx_brand binary?  Please respond unicast, and we'll report back to the list once your post-fix findings match mine.  :)

Thanks,
Dan


From waldenvik at gmx.com  Tue Oct 11 17:53:23 2016
From: waldenvik at gmx.com (Martin Waldenvik)
Date: Tue, 11 Oct 2016 19:53:23 +0200
Subject: [OmniOS-discuss] Slow ssh transfers
Message-ID: <157b4e0a238.2790.a146d0101469d673ab957f4bbe410253@gmx.com>

Hi

I'm wondering if anyone can shed some light on a problem I have. I'm using 
lftp sftp a lot and it is very slow.  I can work around that sometimes by 
using another server. But I am also using zrep (using ssh) to replicate my 
pool to a backup server. This is also slow. Transfer is about 40 megabyte 
(omnios  to omnios). When doing this on freebsd it is about 100 megabyte 
and it saturates my gigabit link (freebsd to freebsd). Same hardware.

I'm using the latest Omios LTS release and openssh. Is there any way to 
speed this up on Omnios?

Regards
Martin



From gearboxes at outlook.com  Tue Oct 11 17:53:35 2016
From: gearboxes at outlook.com (Machine Man)
Date: Tue, 11 Oct 2016 17:53:35 +0000
Subject: [OmniOS-discuss] NVMe/kvm
Message-ID: <BN3PR01MB12403CA7BF39D791E3DC2BFBE7DA0@BN3PR01MB1240.prod.exchangelabs.com>

I'm looking to deploy an all NVMe system, 2x 400GB for rpool and 4x 800GB for the data pool.
I have had poor performance for virtual machines using either 10k or 15k drives and also adding mirrored ZIL and L2 cache SSD. All SAS of course. This got expensive fast and only striped mirror vdevs got to acceptable performance and ended up with far more TB than needed by tossing spindles at it. It also did not matter how I played around with blocksizes for the VM zvols. The IO requirements for some of the machines is too high.

I'm looking to build a system that will be running 7 virtual machines.
4x Windows standard 2012r2 1x domain controller, 2x IIS systems (low client access) and 1x running an FTP commercial product.
1x SQL 2014 (only 300GB max size)
2x Windows 10 pro workstations

The chassis will be a Supermicro SYS-1028U-TN10RT+
192GB memory

My thought is to use stripped mirrors on the data pool and enable dedup on all the vm disks except the SQL server.

The reason for not using a commercial product like VMware is snapshots and offsite replication.
We don?t have an environment with more than 5 VMs and no Microsoft SQL at the moment. We would prefer to spend on hardware rather than licensing and maintenance cost with added admin of constant software updates for hypervisor backup software and replication pieces etc.

The 2 questions I have are:

1)      Are all issues with NVMe resolved?

2)      Is this beyond of what OmniOS was designed for and should I rather look at a commercial products for this type of deployment?

Thanks,


Sent from my Windows 10 phone

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20161011/193a89d2/attachment.html>

From skeltonr at btconnect.com  Tue Oct 11 18:20:42 2016
From: skeltonr at btconnect.com (Richard Skelton)
Date: Tue, 11 Oct 2016 19:20:42 +0100
Subject: [OmniOS-discuss] LX zone console broken
In-Reply-To: <77EFCE94-0C91-408E-9F79-E679DEAA8CB2@omniti.com>
References: <57FC9913.3080609@btconnect.com>
	<1B02D76B-1341-4559-B166-DAC6AEF8ED26@omniti.com>
	<57FD12F7.1010704@btconnect.com>
	<32CA0CA0-4CE1-40D8-BFE7-A6FE9CA7F659@omniti.com>
	<77EFCE94-0C91-408E-9F79-E679DEAA8CB2@omniti.com>
Message-ID: <57FD2D7A.1020308@btconnect.com>

Hi All,
Dan provided an updated lx_brand file which I installed in
/usr/kernel/brand/amd64/lx_brand
After a reboot of the global zone all is now working :-)

CentOS release 6.8 (Final)
Kernel 2.6.32 on an x86_64

lx0 login:



Dan McDonald wrote:
>> On Oct 11, 2016, at 12:52 PM, Dan McDonald <danmcd at omniti.com> wrote:
>>
>> I'm going to check with SmartOS, and see if this is something I fubared, or if it's an upstream (i.e. their) problem too.  I suspect I mismerged something.  It's fairly new, because I swear I used to see CentOS 6 consoles during the io-lx days.
>>     
>
> Not mismerge, but missing bugfix:
>
>     OS-5685 LTP vhangup02 fails and passes
>
> Could you do me a favor and test a replacement lx_brand binary?  Please respond unicast, and we'll report back to the list once your post-fix findings match mine.  :)
>
> Thanks,
> Dan
>
>
>   
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20161011/81560d1a/attachment.html>

From danmcd at omniti.com  Tue Oct 11 19:14:46 2016
From: danmcd at omniti.com (Dan McDonald)
Date: Tue, 11 Oct 2016 15:14:46 -0400
Subject: [OmniOS-discuss] Slow ssh transfers
In-Reply-To: <157b4e0a238.2790.a146d0101469d673ab957f4bbe410253@gmx.com>
References: <157b4e0a238.2790.a146d0101469d673ab957f4bbe410253@gmx.com>
Message-ID: <398961A7-5F81-4731-87B9-B3A829946245@omniti.com>

Dumb question - is your FreeBSD frame size jumbo? 

Also, try this on OmniOS:

ipadm set-prop -p send_buf=1048576 tcp
ipadm set-prop -p recv_buf=1048576 tcp

And see if that helps if the frame size isn't smaller on OmniOS.

Dan

Sent from my iPhone (typos, autocorrect, and all)

> On Oct 11, 2016, at 1:53 PM, Martin Waldenvik <waldenvik at gmx.com> wrote:
> 
> Hi
> 
> I'm wondering if anyone can shed some light on a problem I have. I'm using lftp sftp a lot and it is very slow.  I can work around that sometimes by using another server. But I am also using zrep (using ssh) to replicate my pool to a backup server. This is also slow. Transfer is about 40 megabyte (omnios  to omnios). When doing this on freebsd it is about 100 megabyte and it saturates my gigabit link (freebsd to freebsd). Same hardware.
> 
> I'm using the latest Omios LTS release and openssh. Is there any way to speed this up on Omnios?
> 
> Regards
> Martin
> 
> 
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss

From waldenvik at gmx.com  Tue Oct 11 19:58:48 2016
From: waldenvik at gmx.com (Martin Waldenvik)
Date: Tue, 11 Oct 2016 21:58:48 +0200
Subject: [OmniOS-discuss] Slow ssh transfers
In-Reply-To: <398961A7-5F81-4731-87B9-B3A829946245@omniti.com>
References: <157b4e0a238.2790.a146d0101469d673ab957f4bbe410253@gmx.com>
	<398961A7-5F81-4731-87B9-B3A829946245@omniti.com>
Message-ID: <d18a0fd0-c5da-3ccb-225e-16666042405c@gmx.com>

On 10/11/2016 09:14 PM, Dan McDonald wrote:
> Dumb question - is your FreeBSD frame size jumbo?
>
> Also, try this on OmniOS:
>
> ipadm set-prop -p send_buf=1048576 tcp
> ipadm set-prop -p recv_buf=1048576 tcp
>
> And see if that helps if the frame size isn't smaller on OmniOS.
>
> Dan
>
> Sent from my iPhone (typos, autocorrect, and all)
>
>> On Oct 11, 2016, at 1:53 PM, Martin Waldenvik <waldenvik at gmx.com> wrote:
>>
>> Hi
>>
>> I'm wondering if anyone can shed some light on a problem I have. I'm using lftp sftp a lot and it is very slow.  I can work around that sometimes by using another server. But I am also using zrep (using ssh) to replicate my pool to a backup server. This is also slow. Transfer is about 40 megabyte (omnios  to omnios). When doing this on freebsd it is about 100 megabyte and it saturates my gigabit link (freebsd to freebsd). Same hardware.
>>
>> I'm using the latest Omios LTS release and openssh. Is there any way to speed this up on Omnios?
>>
>> Regards
>> Martin
>>
>>
>> _______________________________________________
>> OmniOS-discuss mailing list
>> OmniOS-discuss at lists.omniti.com
>> http://lists.omniti.com/mailman/listinfo/omnios-discuss

Hi Dan

Thanks for your input. I have had this problem for a long time, then i 
thought is was the old SunSSH but the problem seems to persist with OpenSSH.

I have not touched the frame size, the only thing i have is LACP with 
L3,L4. Intel NIC on a supermicro board.

Tried to set send and recv buffer but there was no difference. It's 
about the same speed with both sending and recieving ~40-45M/s.

I have found that zrep is simple enough for me that manually replicates 
my system on an irregular basis. Maybe there is something that simple 
around that works with netcat.

Regards
Martin

From jimklimov at cos.ru  Tue Oct 11 20:50:34 2016
From: jimklimov at cos.ru (Jim Klimov)
Date: Tue, 11 Oct 2016 22:50:34 +0200
Subject: [OmniOS-discuss] Slow ssh transfers
In-Reply-To: <d18a0fd0-c5da-3ccb-225e-16666042405c@gmx.com>
References: <157b4e0a238.2790.a146d0101469d673ab957f4bbe410253@gmx.com>
	<398961A7-5F81-4731-87B9-B3A829946245@omniti.com>
	<d18a0fd0-c5da-3ccb-225e-16666042405c@gmx.com>
Message-ID: <B36EBFF7-9B4F-4FB9-B350-42489C41681B@cos.ru>

11 ??????? 2016??. 21:58:48 CEST, Martin Waldenvik <waldenvik at gmx.com> ?????:
>On 10/11/2016 09:14 PM, Dan McDonald wrote:
>> Dumb question - is your FreeBSD frame size jumbo?
>>
>> Also, try this on OmniOS:
>>
>> ipadm set-prop -p send_buf=1048576 tcp
>> ipadm set-prop -p recv_buf=1048576 tcp
>>
>> And see if that helps if the frame size isn't smaller on OmniOS.
>>
>> Dan
>>
>> Sent from my iPhone (typos, autocorrect, and all)
>>
>>> On Oct 11, 2016, at 1:53 PM, Martin Waldenvik <waldenvik at gmx.com>
>wrote:
>>>
>>> Hi
>>>
>>> I'm wondering if anyone can shed some light on a problem I have. I'm
>using lftp sftp a lot and it is very slow.  I can work around that
>sometimes by using another server. But I am also using zrep (using ssh)
>to replicate my pool to a backup server. This is also slow. Transfer is
>about 40 megabyte (omnios  to omnios). When doing this on freebsd it is
>about 100 megabyte and it saturates my gigabit link (freebsd to
>freebsd). Same hardware.
>>>
>>> I'm using the latest Omios LTS release and openssh. Is there any way
>to speed this up on Omnios?
>>>
>>> Regards
>>> Martin
>>>
>>>
>>> _______________________________________________
>>> OmniOS-discuss mailing list
>>> OmniOS-discuss at lists.omniti.com
>>> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>
>Hi Dan
>
>Thanks for your input. I have had this problem for a long time, then i 
>thought is was the old SunSSH but the problem seems to persist with
>OpenSSH.
>
>I have not touched the frame size, the only thing i have is LACP with 
>L3,L4. Intel NIC on a supermicro board.
>
>Tried to set send and recv buffer but there was no difference. It's 
>about the same speed with both sending and recieving ~40-45M/s.
>
>I have found that zrep is simple enough for me that manually replicates
>
>my system on an irregular basis. Maybe there is something that simple 
>around that works with netcat.
>
>Regards
>Martin
>_______________________________________________
>OmniOS-discuss mailing list
>OmniOS-discuss at lists.omniti.com
>http://lists.omniti.com/mailman/listinfo/omnios-discuss

Does this problem hit any ssh transfers, or in particular zfs send/recv? The latter is known to be bursty and benefits a lot from caching by mbuffer on at least one side of the link.

Also for curiosity, did you try the hpc ssh patches (build your own), and/or reduced-security protocols (arcfour), and/or inline compression (ssh -z or explicit gzip/gzcat in the pipe)?

Jim
--
Typos courtesy of K-9 Mail on my Samsung Android

From jstockett at molalla.com  Tue Oct 11 22:03:06 2016
From: jstockett at molalla.com (Jeff Stockett)
Date: Tue, 11 Oct 2016 22:03:06 +0000
Subject: [OmniOS-discuss] problem with boot-archive after upgrade
Message-ID: <136C13E89D22BB468B2A7025993639736E12895F@EXMCCMB.molalla.com>

I upgraded a system from 012 to 018 just now and everything went totally smoothly - system rebooted fine.  However, the boot environment names weren't what I wanted and since beadm can't rename the active one, I create a new one, activated it, and rebooted again.  On reboot this second (and subsequent times), I'm getting this in /var/src/log/system-boot-archive:default.log:

[ Mar 20 14:25:01 Executing start method ("/lib/svc/method/boot-archive"). ]
cannot find: /etc/cluster/nodeid: No such file or directory
cannot find: /etc/devices/mdi_ib_cache: No such file or directory
cannot find: /etc/devices/retire_store: No such file or directory
[ Mar 20 14:25:50 Method "start" exited with status 0. ]
[ May 18 05:20:09 Enabled. ]
[ May 18 05:20:14 Executing start method ("/lib/svc/method/boot-archive"). ]
[ May 18 05:21:15 Method or service exit timed out.  Killing contract 16. ]
[ May 18 05:21:15 Method "start" failed due to signal KILL. ]

The system eventually comes up to a single user mode login, and if I login as root, and enter "svcadm clear system/boot-archive" and/or maybe just wait a while longer, then logout the system eventually comes up and works fine in multi-user mode, but I was wondering if there is an easy way to fix whatever I've done to shoot myself in the foot here?  I tried "bootadm update-archive" but that doesn't appear to fix it.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20161011/55e87d1a/attachment.html>

From guorong.koh at gmail.com  Wed Oct 12 00:02:25 2016
From: guorong.koh at gmail.com (Guo-Rong Koh)
Date: Wed, 12 Oct 2016 10:32:25 +1030
Subject: [OmniOS-discuss] Slow ssh transfers
In-Reply-To: <d18a0fd0-c5da-3ccb-225e-16666042405c@gmx.com>
References: <157b4e0a238.2790.a146d0101469d673ab957f4bbe410253@gmx.com>
	<398961A7-5F81-4731-87B9-B3A829946245@omniti.com>
	<d18a0fd0-c5da-3ccb-225e-16666042405c@gmx.com>
Message-ID: <1476230545.603.5.camel@gmail.com>

On Tue, 2016-10-11 at 21:58 +0200, Martin Waldenvik wrote:
> On 10/11/2016 09:14 PM, Dan McDonald wrote:
> > Dumb question - is your FreeBSD frame size jumbo?
> > 
> > Also, try this on OmniOS:
> > 
> > ipadm set-prop -p send_buf=1048576 tcp
> > ipadm set-prop -p recv_buf=1048576 tcp
> > 
> > And see if that helps if the frame size isn't smaller on OmniOS.
> > 
> > Dan
> > 
> > Sent from my iPhone (typos, autocorrect, and all)
> > 
> > > > > > > > > On Oct 11, 2016, at 1:53 PM, Martin Waldenvik <waldenvik at gmx.com>
wrote:
> > > 
> > > Hi
> > > 
> > > > > > > > > > > > > > > > > > > > > I'm wondering if anyone can shed some light on a problem I have.
I'm using lftp sftp a lot and it is very slow.??I can work around
that sometimes by using another server. But I am also using zrep
(using ssh) to replicate my pool to a backup server. This is also
slow. Transfer is about 40 megabyte (omnios??to omnios). When
doing this on freebsd it is about 100 megabyte and it saturates
my gigabit link (freebsd to freebsd). Same hardware.
> > > 
> > > > > > I'm using the latest Omios LTS release and openssh. Is there any
way to speed this up on Omnios?
> > > 
> > > Regards
> > > Martin
> > > 
> > > 
> > > _______________________________________________
> > > OmniOS-discuss mailing list
> > > OmniOS-discuss at lists.omniti.com
> > > http://lists.omniti.com/mailman/listinfo/omnios-discuss
> 
> Hi Dan
> 
> > Thanks for your input. I have had this problem for a long time, then
i?
> > thought is was the old SunSSH but the problem seems to persist with
OpenSSH.
> 
> > I have not touched the frame size, the only thing i have is LACP
with?
> L3,L4. Intel NIC on a supermicro board.
> 
> Tried to set send and recv buffer but there was no difference. It's?
> about the same speed with both sending and recieving ~40-45M/s.
> 
> > I have found that zrep is simple enough for me that manually
replicates?
> > my system on an irregular basis. Maybe there is something that
simple?
> around that works with netcat.
Hi Martin,

Have you considered changing/choosing the cipher used in the sftp
transfer?
I found something like this:
https://turecki.net/content/getting-most-out-ssh-hardware-acceleration-
tuning-aes-ni
enabled scp to saturate my gigabit link whilst saving CPU.

Prior to that, the default negotiated cipher seemed to peak at ~50MB/s.

regards,
Guo-Rong
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20161012/d403a444/attachment.html>

From danmcd at omniti.com  Wed Oct 12 02:39:41 2016
From: danmcd at omniti.com (Dan McDonald)
Date: Tue, 11 Oct 2016 22:39:41 -0400
Subject: [OmniOS-discuss] LX zone console broken
In-Reply-To: <57FD2D7A.1020308@btconnect.com>
References: <57FC9913.3080609@btconnect.com>
	<1B02D76B-1341-4559-B166-DAC6AEF8ED26@omniti.com>
	<57FD12F7.1010704@btconnect.com>
	<32CA0CA0-4CE1-40D8-BFE7-A6FE9CA7F659@omniti.com>
	<77EFCE94-0C91-408E-9F79-E679DEAA8CB2@omniti.com>
	<57FD2D7A.1020308@btconnect.com>
Message-ID: <806FD6D2-3C9D-4634-B580-249C8DE2424E@omniti.com>

The bloody repo server has an updated LX brand package with this fix.

Dan

Sent from my iPhone (typos, autocorrect, and all)

> On Oct 11, 2016, at 2:20 PM, Richard Skelton <skeltonr at btconnect.com> wrote:
> 
> Hi All,
> Dan provided an updated lx_brand file which I installed in /usr/kernel/brand/amd64/lx_brand
> After a reboot of the global zone all is now working :-)
> 
> CentOS release 6.8 (Final)
> Kernel 2.6.32 on an x86_64
> 
> lx0 login: 
> 
> 
> 
> Dan McDonald wrote:
>> 
>>> On Oct 11, 2016, at 12:52 PM, Dan McDonald <danmcd at omniti.com> wrote:
>>> 
>>> I'm going to check with SmartOS, and see if this is something I fubared, or if it's an upstream (i.e. their) problem too.  I suspect I mismerged something.  It's fairly new, because I swear I used to see CentOS 6 consoles during the io-lx days.
>>>     
>> 
>> Not mismerge, but missing bugfix:
>> 
>>     OS-5685 LTP vhangup02 fails and passes
>> 
>> Could you do me a favor and test a replacement lx_brand binary?  Please respond unicast, and we'll report back to the list once your post-fix findings match mine.  :)
>> 
>> Thanks,
>> Dan
>> 
>> 
>>   
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20161011/451b7ff7/attachment.html>

From natxo.asenjo at gmail.com  Wed Oct 12 12:54:09 2016
From: natxo.asenjo at gmail.com (Natxo Asenjo)
Date: Wed, 12 Oct 2016 14:54:09 +0200
Subject: [OmniOS-discuss] cifs smb 2.1 errors when using windows 10 backup
	tool
Message-ID: <CAHBEJzViu_wumS_21QE0BAjAvSv7sO5KMxJA_eKL5s+RRY8QDg@mail.gmail.com>

hi,

There is  a w10 system at the home and it's a bit critical for the wife
;-), so I want to have this up and running asap if some kind of trouble
happens with it.

Anyway, this windows 10 OS has a backup tool that allows you to dump an
image to a network share using cifs, obviously.

So I have a zfs file system, shared it for cifs, set the permissions for
the share. Start the backup, and it will fail consistently when it has
reached 57%.

After much searching, I found this:
https://social.technet.microsoft.com/Forums/windows/en-US/c2032e37-ebe9-4221-9d37-5525fcdaae19/windows-8-backup-win7-file-recovery-system-image-creation-to-nas-device-fails-with-error?forum=w8itprogeneral

And yes, after:

# sharectl set -p max_protocol=1 smb

and restarting the cifs service, now the backup is running correctly (97%
and counting).

Apparently it has to do with 'resilency support', whatever that may be:

https://support.microsoft.com/en-us/kb/2920193

Which is a shame, because now the cifs host is limited to smb 1.5 instead
of 2.1 (but at least it works).

Hopefully I spare somebody some unnecessary pain with this info. And it
would be awesome if this 'resiliciency support' made it to the cifs server
in a next version ;-)

Thanks!

--
Groeten,
natxo
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20161012/9a18cc46/attachment-0001.html>

From waldenvik at gmx.com  Wed Oct 12 14:21:43 2016
From: waldenvik at gmx.com (Martin Waldenvik)
Date: Wed, 12 Oct 2016 16:21:43 +0200
Subject: [OmniOS-discuss] Slow ssh transfers
In-Reply-To: <1476230545.603.5.camel@gmail.com>
References: <157b4e0a238.2790.a146d0101469d673ab957f4bbe410253@gmx.com>
	<398961A7-5F81-4731-87B9-B3A829946245@omniti.com>
	<d18a0fd0-c5da-3ccb-225e-16666042405c@gmx.com>
	<1476230545.603.5.camel@gmail.com>
Message-ID: <2848247d-c853-48f3-fa29-6fc8e6ade124@gmx.com>

On 10/12/2016 02:02 AM, Guo-Rong Koh wrote:
> On Tue, 2016-10-11 at 21:58 +0200, Martin Waldenvik wrote:
>> On 10/11/2016 09:14 PM, Dan McDonald wrote:
>>> Dumb question - is your FreeBSD frame size jumbo? Also, try this on
>>> OmniOS: ipadm set-prop -p send_buf=1048576 tcp ipadm set-prop -p
>>> recv_buf=1048576 tcp And see if that helps if the frame size isn't
>>> smaller on OmniOS. Dan Sent from my iPhone (typos, autocorrect, and all)
>>>> On Oct 11, 2016, at 1:53 PM, Martin Waldenvik <waldenvik at gmx.com
>>>> <mailto:waldenvik at gmx.com>> wrote: Hi I'm wondering if anyone can
>>>> shed some light on a problem I have. I'm using lftp sftp a lot and
>>>> it is very slow. I can work around that sometimes by using another
>>>> server. But I am also using zrep (using ssh) to replicate my pool to
>>>> a backup server. This is also slow. Transfer is about 40 megabyte
>>>> (omnios to omnios). When doing this on freebsd it is about 100
>>>> megabyte and it saturates my gigabit link (freebsd to freebsd). Same
>>>> hardware. I'm using the latest Omios LTS release and openssh. Is
>>>> there any way to speed this up on Omnios? Regards Martin
>>>> _______________________________________________ OmniOS-discuss
>>>> mailing list OmniOS-discuss at lists.omniti.com
>>>> <mailto:OmniOS-discuss at lists.omniti.com>
>>>> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>>
>>
>> Hi Dan
>>
>> Thanks for your input. I have had this problem for a long time, then i
>> thought is was the old SunSSH but the problem seems to persist with OpenSSH.
>>
>> I have not touched the frame size, the only thing i have is LACP with
>> L3,L4. Intel NIC on a supermicro board.
>>
>> Tried to set send and recv buffer but there was no difference. It's
>> about the same speed with both sending and recieving ~40-45M/s.
>>
>> I have found that zrep is simple enough for me that manually replicates
>> my system on an irregular basis. Maybe there is something that simple
>> around that works with netcat.
>
> Hi Martin,
>
> Have you considered changing/choosing the cipher used in the sftp transfer?
> I found something like this:
> https://turecki.net/content/getting-most-out-ssh-hardware-acceleration-tuning-aes-ni
> enabled scp to saturate my gigabit link whilst saving CPU.
>
> Prior to that, the default negotiated cipher seemed to peak at ~50MB/s.
>
> regards,
> Guo-Rong
>
>
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>
Thank you so much Guo-Rong for your help. When i changed from:
Ciphers 
chacha20-poly1305 at openssh.com,aes256-gcm at openssh.com,aes128-gcm at openssh.com,aes256-ctr,aes192-ctr,aes128-ctr

to Ciphers 
aes256-gcm at openssh.com,aes128-gcm at openssh.com,aes256-ctr,aes192-ctr,aes128-ctr

I get full speed. I am so happy for this.

Warm regards
Martin


From danmcd at omniti.com  Wed Oct 12 14:37:11 2016
From: danmcd at omniti.com (Dan McDonald)
Date: Wed, 12 Oct 2016 10:37:11 -0400
Subject: [OmniOS-discuss] Slow ssh transfers
In-Reply-To: <2848247d-c853-48f3-fa29-6fc8e6ade124@gmx.com>
References: <157b4e0a238.2790.a146d0101469d673ab957f4bbe410253@gmx.com>
	<398961A7-5F81-4731-87B9-B3A829946245@omniti.com>
	<d18a0fd0-c5da-3ccb-225e-16666042405c@gmx.com>
	<1476230545.603.5.camel@gmail.com>
	<2848247d-c853-48f3-fa29-6fc8e6ade124@gmx.com>
Message-ID: <4A5B352A-6251-42F3-83D5-FBEC30E0ABD6@omniti.com>


> On Oct 12, 2016, at 10:21 AM, Martin Waldenvik <waldenvik at gmx.com> wrote:
>> 
> Thank you so much Guo-Rong for your help. When i changed from:
> Ciphers chacha20-poly1305 at openssh.com,aes256-gcm at openssh.com,aes128-gcm at openssh.com,aes256-ctr,aes192-ctr,aes128-ctr
> 
> to Ciphers aes256-gcm at openssh.com,aes128-gcm at openssh.com,aes256-ctr,aes192-ctr,aes128-ctr
> 
> I get full speed. I am so happy for this.

I'm curious about one thing.  Are your ciphers on FreeBSD pre-set to the reduced list (i.e. no chacha) by default?  Or does chacha20 run faster on FreeBSD?

Thanks,
Dan


From daleg at omniti.com  Wed Oct 12 14:41:07 2016
From: daleg at omniti.com (Dale Ghent)
Date: Wed, 12 Oct 2016 10:41:07 -0400
Subject: [OmniOS-discuss] problem with boot-archive after upgrade
In-Reply-To: <136C13E89D22BB468B2A7025993639736E12895F@EXMCCMB.molalla.com>
References: <136C13E89D22BB468B2A7025993639736E12895F@EXMCCMB.molalla.com>
Message-ID: <333BA485-2796-4744-A22F-F30355EE4B36@omniti.com>


> On Oct 11, 2016, at 6:03 PM, Jeff Stockett <jstockett at molalla.com> wrote:
> 
> I upgraded a system from 012 to 018 just now and everything went totally smoothly ? system rebooted fine.  However, the boot environment names weren?t what I wanted and since beadm can?t rename the active one, I create a new one, activated it, and rebooted again.  On reboot this second (and subsequent times), I?m getting this in /var/src/log/system-boot-archive:default.log:
>  
> [ Mar 20 14:25:01 Executing start method ("/lib/svc/method/boot-archive"). ]
> cannot find: /etc/cluster/nodeid: No such file or directory
> cannot find: /etc/devices/mdi_ib_cache: No such file or directory
> cannot find: /etc/devices/retire_store: No such file or directory
> [ Mar 20 14:25:50 Method "start" exited with status 0. ]
> [ May 18 05:20:09 Enabled. ]
> [ May 18 05:20:14 Executing start method ("/lib/svc/method/boot-archive"). ]
> [ May 18 05:21:15 Method or service exit timed out.  Killing contract 16. ]
> [ May 18 05:21:15 Method "start" failed due to signal KILL. ]
>  
> The system eventually comes up to a single user mode login, and if I login as root, and enter ?svcadm clear system/boot-archive? and/or maybe just wait a while longer, then logout the system eventually comes up and works fine in multi-user mode, but I was wondering if there is an easy way to fix whatever I?ve done to shoot myself in the foot here?  I tried ?bootadm update-archive? but that doesn?t appear to fix it.

When you manually ran 'bootadm update-archive' did it give you any output such as "updating..." , or did it silently exit?

If the latter, do something like 'touch /etc/system' first, then run it, as bootadm will regenerate the boot archive only if it detects any changes in the files its concerned with. And yes, I've seen this same problem you speak of on rare occasion, with a rebuild of the boot archive being the cure.

/dale


From waldenvik at gmx.com  Wed Oct 12 14:45:20 2016
From: waldenvik at gmx.com (Martin Waldenvik)
Date: Wed, 12 Oct 2016 16:45:20 +0200
Subject: [OmniOS-discuss] Slow ssh transfers
In-Reply-To: <4A5B352A-6251-42F3-83D5-FBEC30E0ABD6@omniti.com>
References: <157b4e0a238.2790.a146d0101469d673ab957f4bbe410253@gmx.com>
	<398961A7-5F81-4731-87B9-B3A829946245@omniti.com>
	<d18a0fd0-c5da-3ccb-225e-16666042405c@gmx.com>
	<1476230545.603.5.camel@gmail.com>
	<2848247d-c853-48f3-fa29-6fc8e6ade124@gmx.com>
	<4A5B352A-6251-42F3-83D5-FBEC30E0ABD6@omniti.com>
Message-ID: <f792f4ed-3d55-c2e3-3341-c9ab70e04516@gmx.com>

On 10/12/2016 04:37 PM, Dan McDonald wrote:
>
>> On Oct 12, 2016, at 10:21 AM, Martin Waldenvik <waldenvik at gmx.com> wrote:
>>>
>> Thank you so much Guo-Rong for your help. When i changed from:
>> Ciphers chacha20-poly1305 at openssh.com,aes256-gcm at openssh.com,aes128-gcm at openssh.com,aes256-ctr,aes192-ctr,aes128-ctr
>>
>> to Ciphers aes256-gcm at openssh.com,aes128-gcm at openssh.com,aes256-ctr,aes192-ctr,aes128-ctr
>>
>> I get full speed. I am so happy for this.
>
> I'm curious about one thing.  Are your ciphers on FreeBSD pre-set to the reduced list (i.e. no chacha) by default?  Or does chacha20 run faster on FreeBSD?
>
> Thanks,
> Dan
>

Hi Dan

I have used those ciphers in that order on all my servers and it works 
in full speed i both freebsd and linux but not in Omnios.

I have configured my ssh-servers as recommended here:

https://stribika.github.io/2015/01/04/secure-secure-shell.html

Regards
Martin

From daleg at omniti.com  Wed Oct 12 14:53:01 2016
From: daleg at omniti.com (Dale Ghent)
Date: Wed, 12 Oct 2016 10:53:01 -0400
Subject: [OmniOS-discuss] Slow ssh transfers
In-Reply-To: <f792f4ed-3d55-c2e3-3341-c9ab70e04516@gmx.com>
References: <157b4e0a238.2790.a146d0101469d673ab957f4bbe410253@gmx.com>
	<398961A7-5F81-4731-87B9-B3A829946245@omniti.com>
	<d18a0fd0-c5da-3ccb-225e-16666042405c@gmx.com>
	<1476230545.603.5.camel@gmail.com>
	<2848247d-c853-48f3-fa29-6fc8e6ade124@gmx.com>
	<4A5B352A-6251-42F3-83D5-FBEC30E0ABD6@omniti.com>
	<f792f4ed-3d55-c2e3-3341-c9ab70e04516@gmx.com>
Message-ID: <74E530D3-33DA-42AD-8A18-54D9676BC41F@omniti.com>


> On Oct 12, 2016, at 10:45 AM, Martin Waldenvik <waldenvik at gmx.com> wrote:
> 
> On 10/12/2016 04:37 PM, Dan McDonald wrote:
>> 
>>> On Oct 12, 2016, at 10:21 AM, Martin Waldenvik <waldenvik at gmx.com> wrote:
>>>> 
>>> Thank you so much Guo-Rong for your help. When i changed from:
>>> Ciphers chacha20-poly1305 at openssh.com,aes256-gcm at openssh.com,aes128-gcm at openssh.com,aes256-ctr,aes192-ctr,aes128-ctr
>>> 
>>> to Ciphers aes256-gcm at openssh.com,aes128-gcm at openssh.com,aes256-ctr,aes192-ctr,aes128-ctr
>>> 
>>> I get full speed. I am so happy for this.
>> 
>> I'm curious about one thing.  Are your ciphers on FreeBSD pre-set to the reduced list (i.e. no chacha) by default?  Or does chacha20 run faster on FreeBSD?
>> 
>> Thanks,
>> Dan
>> 
> 
> Hi Dan
> 
> I have used those ciphers in that order on all my servers and it works in full speed i both freebsd and linux but not in Omnios.
> 
> I have configured my ssh-servers as recommended here:
> 
> https://stribika.github.io/2015/01/04/secure-secure-shell.html

Can you provide the output of 'ssh -Q cipher' from both your FreeBSD and OmniOS boxes?

/dale

From jdg117 at elvis.arl.psu.edu  Wed Oct 12 15:00:58 2016
From: jdg117 at elvis.arl.psu.edu (John D Groenveld)
Date: Wed, 12 Oct 2016 11:00:58 -0400
Subject: [OmniOS-discuss] Slow ssh transfers
In-Reply-To: Your message of "Wed, 12 Oct 2016 16:45:20 +0200."
	<f792f4ed-3d55-c2e3-3341-c9ab70e04516@gmx.com> 
References: <157b4e0a238.2790.a146d0101469d673ab957f4bbe410253@gmx.com>
	<398961A7-5F81-4731-87B9-B3A829946245@omniti.com>
	<d18a0fd0-c5da-3ccb-225e-16666042405c@gmx.com>
	<1476230545.603.5.camel@gmail.com>
	<2848247d-c853-48f3-fa29-6fc8e6ade124@gmx.com>
	<4A5B352A-6251-42F3-83D5-FBEC30E0ABD6@omniti.com>
	<f792f4ed-3d55-c2e3-3341-c9ab70e04516@gmx.com> 
Message-ID: <201610121500.u9CF0wQF018308@elvis.arl.psu.edu>

In message <f792f4ed-3d55-c2e3-3341-c9ab70e04516 at gmx.com>, Martin Waldenvik wri
tes:
>I have used those ciphers in that order on all my servers and it works 
>in full speed i both freebsd and linux but not in Omnios.

If your FreeBSD and OmniOS installs are on similar hardware,
openssl(1) speed benchmark might be interesting.

John
groenveld at acm.org


From waldenvik at gmx.com  Wed Oct 12 15:02:27 2016
From: waldenvik at gmx.com (Martin Waldenvik)
Date: Wed, 12 Oct 2016 17:02:27 +0200
Subject: [OmniOS-discuss] Slow ssh transfers
In-Reply-To: <74E530D3-33DA-42AD-8A18-54D9676BC41F@omniti.com>
References: <157b4e0a238.2790.a146d0101469d673ab957f4bbe410253@gmx.com>
	<398961A7-5F81-4731-87B9-B3A829946245@omniti.com>
	<d18a0fd0-c5da-3ccb-225e-16666042405c@gmx.com>
	<1476230545.603.5.camel@gmail.com>
	<2848247d-c853-48f3-fa29-6fc8e6ade124@gmx.com>
	<4A5B352A-6251-42F3-83D5-FBEC30E0ABD6@omniti.com>
	<f792f4ed-3d55-c2e3-3341-c9ab70e04516@gmx.com>
	<74E530D3-33DA-42AD-8A18-54D9676BC41F@omniti.com>
Message-ID: <409c3004-6182-5285-a02c-de532002fca9@gmx.com>

On 10/12/2016 04:53 PM, Dale Ghent wrote:
>
>> On Oct 12, 2016, at 10:45 AM, Martin Waldenvik <waldenvik at gmx.com> wrote:
>>
>> On 10/12/2016 04:37 PM, Dan McDonald wrote:
>>>
>>>> On Oct 12, 2016, at 10:21 AM, Martin Waldenvik <waldenvik at gmx.com> wrote:
>>>>>
>>>> Thank you so much Guo-Rong for your help. When i changed from:
>>>> Ciphers chacha20-poly1305 at openssh.com,aes256-gcm at openssh.com,aes128-gcm at openssh.com,aes256-ctr,aes192-ctr,aes128-ctr
>>>>
>>>> to Ciphers aes256-gcm at openssh.com,aes128-gcm at openssh.com,aes256-ctr,aes192-ctr,aes128-ctr
>>>>
>>>> I get full speed. I am so happy for this.
>>>
>>> I'm curious about one thing.  Are your ciphers on FreeBSD pre-set to the reduced list (i.e. no chacha) by default?  Or does chacha20 run faster on FreeBSD?
>>>
>>> Thanks,
>>> Dan
>>>
>>
>> Hi Dan
>>
>> I have used those ciphers in that order on all my servers and it works in full speed i both freebsd and linux but not in Omnios.
>>
>> I have configured my ssh-servers as recommended here:
>>
>> https://stribika.github.io/2015/01/04/secure-secure-shell.html
>
> Can you provide the output of 'ssh -Q cipher' from both your FreeBSD and OmniOS boxes?
>
> /dale
>

Hi

Seems to be the same

Omnios box - ssh -Q cipher
3des-cbc
blowfish-cbc
cast128-cbc
arcfour
arcfour128
arcfour256
aes128-cbc
aes192-cbc
aes256-cbc
rijndael-cbc at lysator.liu.se
aes128-ctr
aes192-ctr
aes256-ctr
aes128-gcm at openssh.com
aes256-gcm at openssh.com
chacha20-poly1305 at openssh.com


FreeBSD box - ssh -Q cipher:
3des-cbc
blowfish-cbc
cast128-cbc
arcfour
arcfour128
arcfour256
aes128-cbc
aes192-cbc
aes256-cbc
rijndael-cbc at lysator.liu.se
aes128-ctr
aes192-ctr
aes256-ctr
aes128-gcm at openssh.com
aes256-gcm at openssh.com
chacha20-poly1305 at openssh.com


From garland at bustle.com  Wed Oct 12 15:59:21 2016
From: garland at bustle.com (Garland McAlexander)
Date: Wed, 12 Oct 2016 11:59:21 -0400
Subject: [OmniOS-discuss] Issues with 10GbE - X540
In-Reply-To: <E4249D67-A9B8-4BFF-90FD-62A8245D4CC6@omniti.com>
References: <CAMUDemsjCcQxje1Mj2gqVuK3sB8zOmQN_4nYeg0RrWgt2YGgDw@mail.gmail.com>
	<E4249D67-A9B8-4BFF-90FD-62A8245D4CC6@omniti.com>
Message-ID: <CAMUDemv7xVGVhi+Gaesc8Ecf4n5s6-_fdfdbOaAqUck5BzNW0Q@mail.gmail.com>

Yes, I'm getting a 1GbE link.

Randomly it'll work as 10GbE after a reboot, but another reboot will cause
it to go back to 1GbE.

Running 'dladm set-linkprop en_10gfdx_cap=1 ixgbeN' has no change.

On Fri, Oct 7, 2016 at 1:07 PM, Dale Ghent <daleg at omniti.com> wrote:

> On Oct 7, 2016, at 11:44 AM, Garland McAlexander <garland at bustle.com>
> wrote:
> >
> > Hi All,
> >
> > I'm unable to link at 10GbE with my onboard X540 (X10DRW-iT). It works
> perfectly fine at 10GbE if using Windows or CentOS on the same machine.
> Just to be safe, I've tried multiple switches and Cat6a cabling.
> >
> > Any tips on how to get this going?
>
> Are you getting a 1Gb link instead of a 10Gb link?
>
> Can you run this:
>
> dladm set-linkprop en_10gfdx_cap=1 ixgbeN
>
> And check the link speed afterwards (dladm show-phys)
>
> /dale
>
>


-- 

Garland McAlexander

Technology Manager

E: garland at bustle.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20161012/e8936e41/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 1758 bytes
Desc: not available
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20161012/e8936e41/attachment.png>

From gearboxes at outlook.com  Wed Oct 12 16:00:23 2016
From: gearboxes at outlook.com (Machine Man)
Date: Wed, 12 Oct 2016 16:00:23 +0000
Subject: [OmniOS-discuss] KVM +NVMe + dedup
Message-ID: <BN3PR01MB1240C4A0B110C33B701882FEE7DD0@BN3PR01MB1240.prod.exchangelabs.com>

I am looking to build an all NVMe system using:

SYS-1028U-TN10RT+


OS will be on a mirror pair of 400GB and there will be a data pool of 4x 800GB in striped mirror pair. (there will be one 2x small VMs on rpool)

The system will run 7 virtual machines (most will be Windows 2012R2 and one a 300GB SQL 2014 server) configured in KVM on OmniOS.

The main reason for not using 10K or 15K drives is performance and dedup. I would like to get down to the a 1U chassis for this build.

I have not been that successful with SAS 10/15K  configuration and using KVM. I have tried different block sizes and some tuning and it could just be the load of these systems.

We have a number of system running for years now, but never considered dedup largely due to fear of all the negative post, but most could also be due incorrect hardware configurations.


My main reason for posting is to perhaps get feedback if this route can be considered or to stick to 2U 24x drive chassis and use more spindles without dedup. The system will have 192GB of memory and the VMs will not even consume close to 100GB.

CPUs will be 2x 2.6Ghz 6core E5-2600 v3.

The systems are fairly critical and the reason for not using VMware or commercial products is for snapshots and to cut down on how many products we need to update (hypervisor, backup software, replication components).


Also, when using dedup, is there a higher disk of data corruption in the event of power outage?


Thanks,
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20161012/78298c2c/attachment.html>

From waldenvik at gmx.com  Wed Oct 12 16:05:14 2016
From: waldenvik at gmx.com (Martin Waldenvik)
Date: Wed, 12 Oct 2016 18:05:14 +0200
Subject: [OmniOS-discuss] Slow ssh transfers
In-Reply-To: <201610121500.u9CF0wQF018308@elvis.arl.psu.edu>
References: <157b4e0a238.2790.a146d0101469d673ab957f4bbe410253@gmx.com>
	<398961A7-5F81-4731-87B9-B3A829946245@omniti.com>
	<d18a0fd0-c5da-3ccb-225e-16666042405c@gmx.com>
	<1476230545.603.5.camel@gmail.com>
	<2848247d-c853-48f3-fa29-6fc8e6ade124@gmx.com>
	<4A5B352A-6251-42F3-83D5-FBEC30E0ABD6@omniti.com>
	<f792f4ed-3d55-c2e3-3341-c9ab70e04516@gmx.com>
	<201610121500.u9CF0wQF018308@elvis.arl.psu.edu>
Message-ID: <0d3ebb99-c2ec-b23c-d559-374f7a2a7be4@gmx.com>

On 10/12/2016 05:00 PM, John D Groenveld wrote:
> In message <f792f4ed-3d55-c2e3-3341-c9ab70e04516 at gmx.com>, Martin Waldenvik wri
> tes:
>> I have used those ciphers in that order on all my servers and it works
>> in full speed i both freebsd and linux but not in Omnios.
>
> If your FreeBSD and OmniOS installs are on similar hardware,
> openssl(1) speed benchmark might be interesting.
>
> John
> groenveld at acm.org
>
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>
Hi

Not really similar. Freebsd is a xeon 1260L (sandy bridge, 2.4GHz) and 
Omnios a xeon 1230L (haswell, 1.8GHz).

Ran some tests. https://calomel.org/aesni_ssl_performance.html

omnios - ./openssl speed -elapsed -evp chacha

The 'numbers' are in 1000s of bytes per second processed.
type      16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes
chacha    111089.26k   148893.74k  155299.16k  159068.16k   159257.94k

freebsd - ./openssl speed -elapsed -evp chacha

The 'numbers' are in 1000s of bytes per second processed.
type     16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes
chacha   160908.79k   258242.61k  274012.32k  277625.70k   278313.86k

omnios - ./openssl speed -elapsed -evp aes-256-gcm

The 'numbers' are in 1000s of bytes per second processed.
type        16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes
aes-256-gcm 36546.42k    39239.49k    39289.51k    39519.91k  39613.78k

freebsd - ./openssl speed -elapsed -evp aes-256-gcm

The 'numbers' are in 1000s of bytes per second processed.
type        16 bytes    64 bytes    256 bytes   1024 bytes   8192 bytes
aes-256-gcm 222688.71k  638351.04k  889610.92k  993149.57k   1017445.72k


Martin

From gordon.w.ross at gmail.com  Wed Oct 12 16:07:49 2016
From: gordon.w.ross at gmail.com (Gordon Ross)
Date: Wed, 12 Oct 2016 12:07:49 -0400
Subject: [OmniOS-discuss] cifs smb 2.1 errors when using windows 10
	backup tool
In-Reply-To: <CAHBEJzViu_wumS_21QE0BAjAvSv7sO5KMxJA_eKL5s+RRY8QDg@mail.gmail.com>
References: <CAHBEJzViu_wumS_21QE0BAjAvSv7sO5KMxJA_eKL5s+RRY8QDg@mail.gmail.com>
Message-ID: <CAD0Ztp1XsD-47mYKEN4RDqYE_aWG-G=ehVYHiQ9kAbf1PxZVnQ@mail.gmail.com>

Interesting finding re. VHD support needing resiliency support in SMB 2.1.
Thanks for the KB article: https://support.microsoft.com/en-us/kb/2920193

We have resiliency support in NexentaStor 5.0 (shameless plug:)
https://nexenta.com/products/nexentastor
We should work on upstreaming that code soon.


On Wed, Oct 12, 2016 at 8:54 AM, Natxo Asenjo <natxo.asenjo at gmail.com> wrote:
> hi,
>
> There is  a w10 system at the home and it's a bit critical for the wife ;-),
> so I want to have this up and running asap if some kind of trouble happens
> with it.
>
> Anyway, this windows 10 OS has a backup tool that allows you to dump an
> image to a network share using cifs, obviously.
>
> So I have a zfs file system, shared it for cifs, set the permissions for the
> share. Start the backup, and it will fail consistently when it has reached
> 57%.
>
> After much searching, I found this:
> https://social.technet.microsoft.com/Forums/windows/en-US/c2032e37-ebe9-4221-9d37-5525fcdaae19/windows-8-backup-win7-file-recovery-system-image-creation-to-nas-device-fails-with-error?forum=w8itprogeneral
>
> And yes, after:
>
> # sharectl set -p max_protocol=1 smb
>
> and restarting the cifs service, now the backup is running correctly (97%
> and counting).
>
> Apparently it has to do with 'resilency support', whatever that may be:
>
> https://support.microsoft.com/en-us/kb/2920193
>
> Which is a shame, because now the cifs host is limited to smb 1.5 instead of
> 2.1 (but at least it works).
>
> Hopefully I spare somebody some unnecessary pain with this info. And it
> would be awesome if this 'resiliciency support' made it to the cifs server
> in a next version ;-)
>
> Thanks!
>
> --
> Groeten,
> natxo
>
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>

From daleg at omniti.com  Wed Oct 12 16:19:14 2016
From: daleg at omniti.com (Dale Ghent)
Date: Wed, 12 Oct 2016 12:19:14 -0400
Subject: [OmniOS-discuss] Issues with 10GbE - X540
In-Reply-To: <CAMUDemv7xVGVhi+Gaesc8Ecf4n5s6-_fdfdbOaAqUck5BzNW0Q@mail.gmail.com>
References: <CAMUDemsjCcQxje1Mj2gqVuK3sB8zOmQN_4nYeg0RrWgt2YGgDw@mail.gmail.com>
	<E4249D67-A9B8-4BFF-90FD-62A8245D4CC6@omniti.com>
	<CAMUDemv7xVGVhi+Gaesc8Ecf4n5s6-_fdfdbOaAqUck5BzNW0Q@mail.gmail.com>
Message-ID: <67CB4EEE-82F5-4B46-94E8-1A6696B99774@omniti.com>


I trust you set N to the interface instance of ixgbe :) (eg; ixgbe0)

how about setting that to 0, then back to 1?

/dale

> On Oct 12, 2016, at 11:59 AM, Garland McAlexander <garland at bustle.com> wrote:
> 
> Yes, I'm getting a 1GbE link. 
> 
> Randomly it'll work as 10GbE after a reboot, but another reboot will cause it to go back to 1GbE.
> 
> Running 'dladm set-linkprop en_10gfdx_cap=1 ixgbeN' has no change.
> 
> On Fri, Oct 7, 2016 at 1:07 PM, Dale Ghent <daleg at omniti.com> wrote:
> On Oct 7, 2016, at 11:44 AM, Garland McAlexander <garland at bustle.com> wrote:
> >
> > Hi All,
> >
> > I'm unable to link at 10GbE with my onboard X540 (X10DRW-iT). It works perfectly fine at 10GbE if using Windows or CentOS on the same machine. Just to be safe, I've tried multiple switches and Cat6a cabling.
> >
> > Any tips on how to get this going?
> 
> Are you getting a 1Gb link instead of a 10Gb link?
> 
> Can you run this:
> 
> dladm set-linkprop en_10gfdx_cap=1 ixgbeN
> 
> And check the link speed afterwards (dladm show-phys)
> 
> /dale
> 
> 
> 
> 
> -- 
> Garland McAlexander
> 
> Technology Manager
> 
> E: garland at bustle.com
> 
> <image001.png>
> 


From garland at bustle.com  Wed Oct 12 16:23:50 2016
From: garland at bustle.com (Garland McAlexander)
Date: Wed, 12 Oct 2016 12:23:50 -0400
Subject: [OmniOS-discuss] Issues with 10GbE - X540
In-Reply-To: <67CB4EEE-82F5-4B46-94E8-1A6696B99774@omniti.com>
References: <CAMUDemsjCcQxje1Mj2gqVuK3sB8zOmQN_4nYeg0RrWgt2YGgDw@mail.gmail.com>
	<E4249D67-A9B8-4BFF-90FD-62A8245D4CC6@omniti.com>
	<CAMUDemv7xVGVhi+Gaesc8Ecf4n5s6-_fdfdbOaAqUck5BzNW0Q@mail.gmail.com>
	<67CB4EEE-82F5-4B46-94E8-1A6696B99774@omniti.com>
Message-ID: <CAMUDemsJkEN6twgt6Oo0puY+xf35LKDvQb_4fPTcLsgNJf-ofA@mail.gmail.com>

Hah, yes, I tried with both interface 0 and 1.

Changing that back and forth has no change.

Both ixgbe0 and ixgbe1 will show as either 1000 when my switch ports are
set to auto, and will show as STATE - Down and SPEED - 0 when I set my
switch ports to 10GbE.

On Wed, Oct 12, 2016 at 12:19 PM, Dale Ghent <daleg at omniti.com> wrote:

>
> I trust you set N to the interface instance of ixgbe :) (eg; ixgbe0)
>
> how about setting that to 0, then back to 1?
>
> /dale
>
> > On Oct 12, 2016, at 11:59 AM, Garland McAlexander <garland at bustle.com>
> wrote:
> >
> > Yes, I'm getting a 1GbE link.
> >
> > Randomly it'll work as 10GbE after a reboot, but another reboot will
> cause it to go back to 1GbE.
> >
> > Running 'dladm set-linkprop en_10gfdx_cap=1 ixgbeN' has no change.
> >
> > On Fri, Oct 7, 2016 at 1:07 PM, Dale Ghent <daleg at omniti.com> wrote:
> > On Oct 7, 2016, at 11:44 AM, Garland McAlexander <garland at bustle.com>
> wrote:
> > >
> > > Hi All,
> > >
> > > I'm unable to link at 10GbE with my onboard X540 (X10DRW-iT). It works
> perfectly fine at 10GbE if using Windows or CentOS on the same machine.
> Just to be safe, I've tried multiple switches and Cat6a cabling.
> > >
> > > Any tips on how to get this going?
> >
> > Are you getting a 1Gb link instead of a 10Gb link?
> >
> > Can you run this:
> >
> > dladm set-linkprop en_10gfdx_cap=1 ixgbeN
> >
> > And check the link speed afterwards (dladm show-phys)
> >
> > /dale
> >
> >
> >
> >
> > --
> > Garland McAlexander
> >
> > Technology Manager
> >
> > E: garland at bustle.com
> >
> > <image001.png>
> >
>
>


-- 

Garland McAlexander

Technology Manager

E: garland at bustle.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20161012/51a60ed3/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 1758 bytes
Desc: not available
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20161012/51a60ed3/attachment.png>

From daleg at omniti.com  Wed Oct 12 16:25:53 2016
From: daleg at omniti.com (Dale Ghent)
Date: Wed, 12 Oct 2016 12:25:53 -0400
Subject: [OmniOS-discuss] Issues with 10GbE - X540
In-Reply-To: <CAMUDemsJkEN6twgt6Oo0puY+xf35LKDvQb_4fPTcLsgNJf-ofA@mail.gmail.com>
References: <CAMUDemsjCcQxje1Mj2gqVuK3sB8zOmQN_4nYeg0RrWgt2YGgDw@mail.gmail.com>
	<E4249D67-A9B8-4BFF-90FD-62A8245D4CC6@omniti.com>
	<CAMUDemv7xVGVhi+Gaesc8Ecf4n5s6-_fdfdbOaAqUck5BzNW0Q@mail.gmail.com>
	<67CB4EEE-82F5-4B46-94E8-1A6696B99774@omniti.com>
	<CAMUDemsJkEN6twgt6Oo0puY+xf35LKDvQb_4fPTcLsgNJf-ofA@mail.gmail.com>
Message-ID: <288A481D-7277-46DC-9415-B5EE77D210F1@omniti.com>


Oof, I realized I may not have been clear in how I worded that. What I meant to ask is:

What happens when you set en_10gfdx_cap to 0, then back to 1 ?

> On Oct 12, 2016, at 12:23 PM, Garland McAlexander <garland at bustle.com> wrote:
> 
> Hah, yes, I tried with both interface 0 and 1.
> 
> Changing that back and forth has no change.
> 
> Both ixgbe0 and ixgbe1 will show as either 1000 when my switch ports are set to auto, and will show as STATE - Down and SPEED - 0 when I set my switch ports to 10GbE.
> 
> On Wed, Oct 12, 2016 at 12:19 PM, Dale Ghent <daleg at omniti.com> wrote:
> 
> I trust you set N to the interface instance of ixgbe :) (eg; ixgbe0)
> 
> how about setting that to 0, then back to 1?
> 
> /dale
> 
> > On Oct 12, 2016, at 11:59 AM, Garland McAlexander <garland at bustle.com> wrote:
> >
> > Yes, I'm getting a 1GbE link.
> >
> > Randomly it'll work as 10GbE after a reboot, but another reboot will cause it to go back to 1GbE.
> >
> > Running 'dladm set-linkprop en_10gfdx_cap=1 ixgbeN' has no change.
> >
> > On Fri, Oct 7, 2016 at 1:07 PM, Dale Ghent <daleg at omniti.com> wrote:
> > On Oct 7, 2016, at 11:44 AM, Garland McAlexander <garland at bustle.com> wrote:
> > >
> > > Hi All,
> > >
> > > I'm unable to link at 10GbE with my onboard X540 (X10DRW-iT). It works perfectly fine at 10GbE if using Windows or CentOS on the same machine. Just to be safe, I've tried multiple switches and Cat6a cabling.
> > >
> > > Any tips on how to get this going?
> >
> > Are you getting a 1Gb link instead of a 10Gb link?
> >
> > Can you run this:
> >
> > dladm set-linkprop en_10gfdx_cap=1 ixgbeN
> >
> > And check the link speed afterwards (dladm show-phys)
> >
> > /dale
> >
> >
> >
> >
> > --
> > Garland McAlexander
> >
> > Technology Manager
> >
> > E: garland at bustle.com
> >
> > <image001.png>
> >
> 
> 
> 
> 
> -- 
> Garland McAlexander
> 
> Technology Manager
> 
> E: garland at bustle.com
> 
> <image001.png>
> 


From garland at bustle.com  Wed Oct 12 16:29:14 2016
From: garland at bustle.com (Garland McAlexander)
Date: Wed, 12 Oct 2016 12:29:14 -0400
Subject: [OmniOS-discuss] Issues with 10GbE - X540
In-Reply-To: <288A481D-7277-46DC-9415-B5EE77D210F1@omniti.com>
References: <CAMUDemsjCcQxje1Mj2gqVuK3sB8zOmQN_4nYeg0RrWgt2YGgDw@mail.gmail.com>
	<E4249D67-A9B8-4BFF-90FD-62A8245D4CC6@omniti.com>
	<CAMUDemv7xVGVhi+Gaesc8Ecf4n5s6-_fdfdbOaAqUck5BzNW0Q@mail.gmail.com>
	<67CB4EEE-82F5-4B46-94E8-1A6696B99774@omniti.com>
	<CAMUDemsJkEN6twgt6Oo0puY+xf35LKDvQb_4fPTcLsgNJf-ofA@mail.gmail.com>
	<288A481D-7277-46DC-9415-B5EE77D210F1@omniti.com>
Message-ID: <CAMUDemv3+ntp1u_iB+LMMKQUv8PwJJGyOaSob1TmwEmiW9xXNA@mail.gmail.com>

No no, I understood.

When I set it to 0 and back to 1, the link will lose connection.

On Wed, Oct 12, 2016 at 12:25 PM, Dale Ghent <daleg at omniti.com> wrote:

>
> Oof, I realized I may not have been clear in how I worded that. What I
> meant to ask is:
>
> What happens when you set en_10gfdx_cap to 0, then back to 1 ?
>
> > On Oct 12, 2016, at 12:23 PM, Garland McAlexander <garland at bustle.com>
> wrote:
> >
> > Hah, yes, I tried with both interface 0 and 1.
> >
> > Changing that back and forth has no change.
> >
> > Both ixgbe0 and ixgbe1 will show as either 1000 when my switch ports are
> set to auto, and will show as STATE - Down and SPEED - 0 when I set my
> switch ports to 10GbE.
> >
> > On Wed, Oct 12, 2016 at 12:19 PM, Dale Ghent <daleg at omniti.com> wrote:
> >
> > I trust you set N to the interface instance of ixgbe :) (eg; ixgbe0)
> >
> > how about setting that to 0, then back to 1?
> >
> > /dale
> >
> > > On Oct 12, 2016, at 11:59 AM, Garland McAlexander <garland at bustle.com>
> wrote:
> > >
> > > Yes, I'm getting a 1GbE link.
> > >
> > > Randomly it'll work as 10GbE after a reboot, but another reboot will
> cause it to go back to 1GbE.
> > >
> > > Running 'dladm set-linkprop en_10gfdx_cap=1 ixgbeN' has no change.
> > >
> > > On Fri, Oct 7, 2016 at 1:07 PM, Dale Ghent <daleg at omniti.com> wrote:
> > > On Oct 7, 2016, at 11:44 AM, Garland McAlexander <garland at bustle.com>
> wrote:
> > > >
> > > > Hi All,
> > > >
> > > > I'm unable to link at 10GbE with my onboard X540 (X10DRW-iT). It
> works perfectly fine at 10GbE if using Windows or CentOS on the same
> machine. Just to be safe, I've tried multiple switches and Cat6a cabling.
> > > >
> > > > Any tips on how to get this going?
> > >
> > > Are you getting a 1Gb link instead of a 10Gb link?
> > >
> > > Can you run this:
> > >
> > > dladm set-linkprop en_10gfdx_cap=1 ixgbeN
> > >
> > > And check the link speed afterwards (dladm show-phys)
> > >
> > > /dale
> > >
> > >
> > >
> > >
> > > --
> > > Garland McAlexander
> > >
> > > Technology Manager
> > >
> > > E: garland at bustle.com
> > >
> > > <image001.png>
> > >
> >
> >
> >
> >
> > --
> > Garland McAlexander
> >
> > Technology Manager
> >
> > E: garland at bustle.com
> >
> > <image001.png>
> >
>
>


-- 

Garland McAlexander

Technology Manager

E: garland at bustle.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20161012/a790d3ca/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 1758 bytes
Desc: not available
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20161012/a790d3ca/attachment.png>

From daleg at omniti.com  Wed Oct 12 16:34:07 2016
From: daleg at omniti.com (Dale Ghent)
Date: Wed, 12 Oct 2016 12:34:07 -0400
Subject: [OmniOS-discuss] Slow ssh transfers
In-Reply-To: <0d3ebb99-c2ec-b23c-d559-374f7a2a7be4@gmx.com>
References: <157b4e0a238.2790.a146d0101469d673ab957f4bbe410253@gmx.com>
	<398961A7-5F81-4731-87B9-B3A829946245@omniti.com>
	<d18a0fd0-c5da-3ccb-225e-16666042405c@gmx.com>
	<1476230545.603.5.camel@gmail.com>
	<2848247d-c853-48f3-fa29-6fc8e6ade124@gmx.com>
	<4A5B352A-6251-42F3-83D5-FBEC30E0ABD6@omniti.com>
	<f792f4ed-3d55-c2e3-3341-c9ab70e04516@gmx.com>
	<201610121500.u9CF0wQF018308@elvis.arl.psu.edu>
	<0d3ebb99-c2ec-b23c-d559-374f7a2a7be4@gmx.com>
Message-ID: <270983B5-48B3-4CA3-AB7F-503B4C075A1F@omniti.com>


> On Oct 12, 2016, at 12:05 PM, Martin Waldenvik <waldenvik at gmx.com> wrote:
> 
> omnios - ./openssl speed -elapsed -evp aes-256-gcm
> 
> The 'numbers' are in 1000s of bytes per second processed.
> type        16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes
> aes-256-gcm 36546.42k    39239.49k    39289.51k    39519.91k  39613.78k
> 
> freebsd - ./openssl speed -elapsed -evp aes-256-gcm
> 
> The 'numbers' are in 1000s of bytes per second processed.
> type        16 bytes    64 bytes    256 bytes   1024 bytes   8192 bytes
> aes-256-gcm 222688.71k  638351.04k  889610.92k  993149.57k   1017445.72k
> 

That's still quite slow on the OmniOS box. On a 3.5GHz Haswell-E (E3-1241 v3) I get:

type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes
aes-256-gcm     394656.68k  1001238.31k  2094818.99k  2407235.93k  2689916.93k

Perhaps you may want to check the BIOS settings of your OmniOS box and ensure that "AES-NI Instructions" are turned on. Your hardware vendor might have that termed differently - AES Acceleration and such.

/dale

From daleg at omniti.com  Wed Oct 12 16:34:58 2016
From: daleg at omniti.com (Dale Ghent)
Date: Wed, 12 Oct 2016 12:34:58 -0400
Subject: [OmniOS-discuss] Issues with 10GbE - X540
In-Reply-To: <CAMUDemv3+ntp1u_iB+LMMKQUv8PwJJGyOaSob1TmwEmiW9xXNA@mail.gmail.com>
References: <CAMUDemsjCcQxje1Mj2gqVuK3sB8zOmQN_4nYeg0RrWgt2YGgDw@mail.gmail.com>
	<E4249D67-A9B8-4BFF-90FD-62A8245D4CC6@omniti.com>
	<CAMUDemv7xVGVhi+Gaesc8Ecf4n5s6-_fdfdbOaAqUck5BzNW0Q@mail.gmail.com>
	<67CB4EEE-82F5-4B46-94E8-1A6696B99774@omniti.com>
	<CAMUDemsJkEN6twgt6Oo0puY+xf35LKDvQb_4fPTcLsgNJf-ofA@mail.gmail.com>
	<288A481D-7277-46DC-9415-B5EE77D210F1@omniti.com>
	<CAMUDemv3+ntp1u_iB+LMMKQUv8PwJJGyOaSob1TmwEmiW9xXNA@mail.gmail.com>
Message-ID: <70524BEE-C6C7-4DD3-82C2-CF0977B12325@omniti.com>


Interesting. So your switch configuration is set to nail the ports at 10Gbe rather than allow autoneg?

/dale

> On Oct 12, 2016, at 12:29 PM, Garland McAlexander <garland at bustle.com> wrote:
> 
> No no, I understood.
> 
> When I set it to 0 and back to 1, the link will lose connection.
> 
> On Wed, Oct 12, 2016 at 12:25 PM, Dale Ghent <daleg at omniti.com> wrote:
> 
> Oof, I realized I may not have been clear in how I worded that. What I meant to ask is:
> 
> What happens when you set en_10gfdx_cap to 0, then back to 1 ?
> 
> > On Oct 12, 2016, at 12:23 PM, Garland McAlexander <garland at bustle.com> wrote:
> >
> > Hah, yes, I tried with both interface 0 and 1.
> >
> > Changing that back and forth has no change.
> >
> > Both ixgbe0 and ixgbe1 will show as either 1000 when my switch ports are set to auto, and will show as STATE - Down and SPEED - 0 when I set my switch ports to 10GbE.
> >
> > On Wed, Oct 12, 2016 at 12:19 PM, Dale Ghent <daleg at omniti.com> wrote:
> >
> > I trust you set N to the interface instance of ixgbe :) (eg; ixgbe0)
> >
> > how about setting that to 0, then back to 1?
> >
> > /dale
> >
> > > On Oct 12, 2016, at 11:59 AM, Garland McAlexander <garland at bustle.com> wrote:
> > >
> > > Yes, I'm getting a 1GbE link.
> > >
> > > Randomly it'll work as 10GbE after a reboot, but another reboot will cause it to go back to 1GbE.
> > >
> > > Running 'dladm set-linkprop en_10gfdx_cap=1 ixgbeN' has no change.
> > >
> > > On Fri, Oct 7, 2016 at 1:07 PM, Dale Ghent <daleg at omniti.com> wrote:
> > > On Oct 7, 2016, at 11:44 AM, Garland McAlexander <garland at bustle.com> wrote:
> > > >
> > > > Hi All,
> > > >
> > > > I'm unable to link at 10GbE with my onboard X540 (X10DRW-iT). It works perfectly fine at 10GbE if using Windows or CentOS on the same machine. Just to be safe, I've tried multiple switches and Cat6a cabling.
> > > >
> > > > Any tips on how to get this going?
> > >
> > > Are you getting a 1Gb link instead of a 10Gb link?
> > >
> > > Can you run this:
> > >
> > > dladm set-linkprop en_10gfdx_cap=1 ixgbeN
> > >
> > > And check the link speed afterwards (dladm show-phys)
> > >
> > > /dale
> > >
> > >
> > >
> > >
> > > --
> > > Garland McAlexander
> > >
> > > Technology Manager
> > >
> > > E: garland at bustle.com
> > >
> > > <image001.png>
> > >
> >
> >
> >
> >
> > --
> > Garland McAlexander
> >
> > Technology Manager
> >
> > E: garland at bustle.com
> >
> > <image001.png>
> >
> 
> 
> 
> 
> -- 
> Garland McAlexander
> 
> Technology Manager
> 
> E: garland at bustle.com
> 
> <image001.png>
> 


From garland at bustle.com  Wed Oct 12 16:35:54 2016
From: garland at bustle.com (Garland McAlexander)
Date: Wed, 12 Oct 2016 12:35:54 -0400
Subject: [OmniOS-discuss] Issues with 10GbE - X540
In-Reply-To: <70524BEE-C6C7-4DD3-82C2-CF0977B12325@omniti.com>
References: <CAMUDemsjCcQxje1Mj2gqVuK3sB8zOmQN_4nYeg0RrWgt2YGgDw@mail.gmail.com>
	<E4249D67-A9B8-4BFF-90FD-62A8245D4CC6@omniti.com>
	<CAMUDemv7xVGVhi+Gaesc8Ecf4n5s6-_fdfdbOaAqUck5BzNW0Q@mail.gmail.com>
	<67CB4EEE-82F5-4B46-94E8-1A6696B99774@omniti.com>
	<CAMUDemsJkEN6twgt6Oo0puY+xf35LKDvQb_4fPTcLsgNJf-ofA@mail.gmail.com>
	<288A481D-7277-46DC-9415-B5EE77D210F1@omniti.com>
	<CAMUDemv3+ntp1u_iB+LMMKQUv8PwJJGyOaSob1TmwEmiW9xXNA@mail.gmail.com>
	<70524BEE-C6C7-4DD3-82C2-CF0977B12325@omniti.com>
Message-ID: <CAMUDemvgqkQTwT8VDkmZ-wjCfpaNEkwrR-T1O4Ms-m0zf+14vQ@mail.gmail.com>

They're autoneg, but I've been trying everything (including forcing to
10GbE).

For testing, I currently have one port set to autoneg, and one port to
10GbE to try and find a solution.

On Wed, Oct 12, 2016 at 12:34 PM, Dale Ghent <daleg at omniti.com> wrote:

>
> Interesting. So your switch configuration is set to nail the ports at
> 10Gbe rather than allow autoneg?
>
> /dale
>
> > On Oct 12, 2016, at 12:29 PM, Garland McAlexander <garland at bustle.com>
> wrote:
> >
> > No no, I understood.
> >
> > When I set it to 0 and back to 1, the link will lose connection.
> >
> > On Wed, Oct 12, 2016 at 12:25 PM, Dale Ghent <daleg at omniti.com> wrote:
> >
> > Oof, I realized I may not have been clear in how I worded that. What I
> meant to ask is:
> >
> > What happens when you set en_10gfdx_cap to 0, then back to 1 ?
> >
> > > On Oct 12, 2016, at 12:23 PM, Garland McAlexander <garland at bustle.com>
> wrote:
> > >
> > > Hah, yes, I tried with both interface 0 and 1.
> > >
> > > Changing that back and forth has no change.
> > >
> > > Both ixgbe0 and ixgbe1 will show as either 1000 when my switch ports
> are set to auto, and will show as STATE - Down and SPEED - 0 when I set my
> switch ports to 10GbE.
> > >
> > > On Wed, Oct 12, 2016 at 12:19 PM, Dale Ghent <daleg at omniti.com> wrote:
> > >
> > > I trust you set N to the interface instance of ixgbe :) (eg; ixgbe0)
> > >
> > > how about setting that to 0, then back to 1?
> > >
> > > /dale
> > >
> > > > On Oct 12, 2016, at 11:59 AM, Garland McAlexander <
> garland at bustle.com> wrote:
> > > >
> > > > Yes, I'm getting a 1GbE link.
> > > >
> > > > Randomly it'll work as 10GbE after a reboot, but another reboot will
> cause it to go back to 1GbE.
> > > >
> > > > Running 'dladm set-linkprop en_10gfdx_cap=1 ixgbeN' has no change.
> > > >
> > > > On Fri, Oct 7, 2016 at 1:07 PM, Dale Ghent <daleg at omniti.com> wrote:
> > > > On Oct 7, 2016, at 11:44 AM, Garland McAlexander <garland at bustle.com>
> wrote:
> > > > >
> > > > > Hi All,
> > > > >
> > > > > I'm unable to link at 10GbE with my onboard X540 (X10DRW-iT). It
> works perfectly fine at 10GbE if using Windows or CentOS on the same
> machine. Just to be safe, I've tried multiple switches and Cat6a cabling.
> > > > >
> > > > > Any tips on how to get this going?
> > > >
> > > > Are you getting a 1Gb link instead of a 10Gb link?
> > > >
> > > > Can you run this:
> > > >
> > > > dladm set-linkprop en_10gfdx_cap=1 ixgbeN
> > > >
> > > > And check the link speed afterwards (dladm show-phys)
> > > >
> > > > /dale
> > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Garland McAlexander
> > > >
> > > > Technology Manager
> > > >
> > > > E: garland at bustle.com
> > > >
> > > > <image001.png>
> > > >
> > >
> > >
> > >
> > >
> > > --
> > > Garland McAlexander
> > >
> > > Technology Manager
> > >
> > > E: garland at bustle.com
> > >
> > > <image001.png>
> > >
> >
> >
> >
> >
> > --
> > Garland McAlexander
> >
> > Technology Manager
> >
> > E: garland at bustle.com
> >
> > <image001.png>
> >
>
>


-- 

Garland McAlexander

Technology Manager

E: garland at bustle.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20161012/3166bc1c/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 1758 bytes
Desc: not available
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20161012/3166bc1c/attachment.png>

From waldenvik at gmx.com  Wed Oct 12 16:39:07 2016
From: waldenvik at gmx.com (Martin Waldenvik)
Date: Wed, 12 Oct 2016 18:39:07 +0200
Subject: [OmniOS-discuss] Slow ssh transfers
In-Reply-To: <270983B5-48B3-4CA3-AB7F-503B4C075A1F@omniti.com>
References: <157b4e0a238.2790.a146d0101469d673ab957f4bbe410253@gmx.com>
	<398961A7-5F81-4731-87B9-B3A829946245@omniti.com>
	<d18a0fd0-c5da-3ccb-225e-16666042405c@gmx.com>
	<1476230545.603.5.camel@gmail.com>
	<2848247d-c853-48f3-fa29-6fc8e6ade124@gmx.com>
	<4A5B352A-6251-42F3-83D5-FBEC30E0ABD6@omniti.com>
	<f792f4ed-3d55-c2e3-3341-c9ab70e04516@gmx.com>
	<201610121500.u9CF0wQF018308@elvis.arl.psu.edu>
	<0d3ebb99-c2ec-b23c-d559-374f7a2a7be4@gmx.com>
	<270983B5-48B3-4CA3-AB7F-503B4C075A1F@omniti.com>
Message-ID: <1afc4341-e212-ec85-c467-543ac201a34a@gmx.com>

On 10/12/2016 06:34 PM, Dale Ghent wrote:
>
>> On Oct 12, 2016, at 12:05 PM, Martin Waldenvik <waldenvik at gmx.com> wrote:
>>
>> omnios - ./openssl speed -elapsed -evp aes-256-gcm
>>
>> The 'numbers' are in 1000s of bytes per second processed.
>> type        16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes
>> aes-256-gcm 36546.42k    39239.49k    39289.51k    39519.91k  39613.78k
>>
>> freebsd - ./openssl speed -elapsed -evp aes-256-gcm
>>
>> The 'numbers' are in 1000s of bytes per second processed.
>> type        16 bytes    64 bytes    256 bytes   1024 bytes   8192 bytes
>> aes-256-gcm 222688.71k  638351.04k  889610.92k  993149.57k   1017445.72k
>>
>
> That's still quite slow on the OmniOS box. On a 3.5GHz Haswell-E (E3-1241 v3) I get:
>
> type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes
> aes-256-gcm     394656.68k  1001238.31k  2094818.99k  2407235.93k  2689916.93k
>
> Perhaps you may want to check the BIOS settings of your OmniOS box and ensure that "AES-NI Instructions" are turned on. Your hardware vendor might have that termed differently - AES Acceleration and such.
>
> /dale
>
Must be someting wrong with what i did. Used libressl to test chacha. 
This is with built in openssl:

type        16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes
aes-256-gcm 309921.38k   784682.82k  1614856.62k 1867160.92k  2117028.52k

Looks a lot better.

Martin

From natxo.asenjo at gmail.com  Wed Oct 12 17:11:52 2016
From: natxo.asenjo at gmail.com (Natxo Asenjo)
Date: Wed, 12 Oct 2016 19:11:52 +0200
Subject: [OmniOS-discuss] cifs smb 2.1 errors when using windows 10
	backup tool
In-Reply-To: <136C13E89D22BB468B2A7025993639736E1293D1@EXMCCMB.molalla.com>
References: <CAHBEJzViu_wumS_21QE0BAjAvSv7sO5KMxJA_eKL5s+RRY8QDg@mail.gmail.com>
	<136C13E89D22BB468B2A7025993639736E1293D1@EXMCCMB.molalla.com>
Message-ID: <CAHBEJzWtSfD3afDXbcX-9fGfqCJZAnugmik=Rz_443JLkj4TDQ@mail.gmail.com>

On Wed, Oct 12, 2016 at 5:00 PM, Jeff Stockett <jstockett at molalla.com>
wrote:

> FWIW, you could create a backup volume and share it via iscsi to your
> wife?s PC ? quite a few commands to accomplish on the omnios side but easy
> if you have nappit already.  Once you attach it, run windows backup, and it
> will immediately see it as an available backup location and initialize it
> for just that purpose if you let it.  Then you could leave your CIFS shares
> 2.1.
>

sure, you can offer an iscsi disk but recovering bare metal from iscsi
targets is not as easy as from a share ;-) and well, as I read in the
thread, Nexenta already has this and will upstream their work (thanks!
awesome work), so for the time being I guess I can either turn smb 1 on/off
every now and then for a backup, or just live with a slower cifs protocol.

--
Groeten,
natxo
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20161012/4f383727/attachment-0001.html>

From natxo.asenjo at gmail.com  Wed Oct 12 17:12:05 2016
From: natxo.asenjo at gmail.com (Natxo Asenjo)
Date: Wed, 12 Oct 2016 19:12:05 +0200
Subject: [OmniOS-discuss] cifs smb 2.1 errors when using windows 10
	backup tool
In-Reply-To: <CAD0Ztp1XsD-47mYKEN4RDqYE_aWG-G=ehVYHiQ9kAbf1PxZVnQ@mail.gmail.com>
References: <CAHBEJzViu_wumS_21QE0BAjAvSv7sO5KMxJA_eKL5s+RRY8QDg@mail.gmail.com>
	<CAD0Ztp1XsD-47mYKEN4RDqYE_aWG-G=ehVYHiQ9kAbf1PxZVnQ@mail.gmail.com>
Message-ID: <CAHBEJzVcFiTwyJjj_XLnRcxVwUTdHnMSy4ZHCk0Gz8njM0pZiA@mail.gmail.com>

On Wed, Oct 12, 2016 at 6:07 PM, Gordon Ross <gordon.w.ross at gmail.com>
wrote:

> Interesting finding re. VHD support needing resiliency support in SMB 2.1.
> Thanks for the KB article: https://support.microsoft.com/en-us/kb/2920193
>
> We have resiliency support in NexentaStor 5.0 (shameless plug:)
> https://nexenta.com/products/nexentastor
> We should work on upstreaming that code soon.
>
>
> On Wed, Oct 12, 2016 at 8:54 AM, Natxo Asenjo <natxo.asenjo at gmail.com>
> wrote:
> > hi,
> >
> > There is  a w10 system at the home and it's a bit critical for the wife
> ;-),
> > so I want to have this up and running asap if some kind of trouble
> happens
> > with it.
> >
> > Anyway, this windows 10 OS has a backup tool that allows you to dump an
> > image to a network share using cifs, obviously.
> >
> > So I have a zfs file system, shared it for cifs, set the permissions for
> the
> > share. Start the backup, and it will fail consistently when it has
> reached
> > 57%.
> >
> > After much searching, I found this:
> > https://social.technet.microsoft.com/Forums/windows/
> en-US/c2032e37-ebe9-4221-9d37-5525fcdaae19/windows-8-backup-
> win7-file-recovery-system-image-creation-to-nas-device-
> fails-with-error?forum=w8itprogeneral
> >
> > And yes, after:
> >
> > # sharectl set -p max_protocol=1 smb
> >
> > and restarting the cifs service, now the backup is running correctly (97%
> > and counting).
> >
> > Apparently it has to do with 'resilency support', whatever that may be:
> >
> > https://support.microsoft.com/en-us/kb/2920193
> >
> > Which is a shame, because now the cifs host is limited to smb 1.5
> instead of
> > 2.1 (but at least it works).
> >
> > Hopefully I spare somebody some unnecessary pain with this info. And it
> > would be awesome if this 'resiliciency support' made it to the cifs
> server
> > in a next version ;-)
> >
> > Thanks!
> >
> > --
> > Groeten,
> > natxo
> >
> > _______________________________________________
> > OmniOS-discuss mailing list
> > OmniOS-discuss at lists.omniti.com
> > http://lists.omniti.com/mailman/listinfo/omnios-discuss
> >
>



-- 
--
Groeten,
natxo
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20161012/6498c219/attachment.html>

From natxo.asenjo at gmail.com  Wed Oct 12 17:14:22 2016
From: natxo.asenjo at gmail.com (Natxo Asenjo)
Date: Wed, 12 Oct 2016 19:14:22 +0200
Subject: [OmniOS-discuss] cifs smb 2.1 errors when using windows 10
	backup tool
In-Reply-To: <CAHBEJzVcFiTwyJjj_XLnRcxVwUTdHnMSy4ZHCk0Gz8njM0pZiA@mail.gmail.com>
References: <CAHBEJzViu_wumS_21QE0BAjAvSv7sO5KMxJA_eKL5s+RRY8QDg@mail.gmail.com>
	<CAD0Ztp1XsD-47mYKEN4RDqYE_aWG-G=ehVYHiQ9kAbf1PxZVnQ@mail.gmail.com>
	<CAHBEJzVcFiTwyJjj_XLnRcxVwUTdHnMSy4ZHCk0Gz8njM0pZiA@mail.gmail.com>
Message-ID: <CAHBEJzUocCpTUPqiimcFY1MwwSaVf12s-Qf4pKLbFaFuaNHQqg@mail.gmail.com>

hi,

>
> On Wed, Oct 12, 2016 at 6:07 PM, Gordon Ross <gordon.w.ross at gmail.com>
> wrote:
>
>> Interesting finding re. VHD support needing resiliency support in SMB 2.1.
>> Thanks for the KB article: https://support.microsoft.com/en-us/kb/2920193
>>
>> We have resiliency support in NexentaStor 5.0 (shameless plug:)
>> https://nexenta.com/products/nexentastor
>> We should work on upstreaming that code soon.
>>
>
That would be terrific! Thanks for your efforts for making an awesome cifs
service.

--
Groeten,
natxo
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20161012/b7d54f69/attachment.html>

From tom-omnios-discuss at tom.bn-ulm.de  Wed Oct 12 17:49:16 2016
From: tom-omnios-discuss at tom.bn-ulm.de (Thomas Wagner)
Date: Wed, 12 Oct 2016 19:49:16 +0200
Subject: [OmniOS-discuss] Slow ssh transfers
In-Reply-To: <409c3004-6182-5285-a02c-de532002fca9@gmx.com>
References: <157b4e0a238.2790.a146d0101469d673ab957f4bbe410253@gmx.com>
	<398961A7-5F81-4731-87B9-B3A829946245@omniti.com>
	<d18a0fd0-c5da-3ccb-225e-16666042405c@gmx.com>
	<1476230545.603.5.camel@gmail.com>
	<2848247d-c853-48f3-fa29-6fc8e6ade124@gmx.com>
	<4A5B352A-6251-42F3-83D5-FBEC30E0ABD6@omniti.com>
	<f792f4ed-3d55-c2e3-3341-c9ab70e04516@gmx.com>
	<74E530D3-33DA-42AD-8A18-54D9676BC41F@omniti.com>
	<409c3004-6182-5285-a02c-de532002fca9@gmx.com>
Message-ID: <20161012174916.GI25497@wagner-net.com>


The SSH Daemon is in most cases the limiting party. 
The ssh client is allowed to support much more older 
and weaker algorithms and protocols.
Therefore I would compare the Ciphers and macs the
Server supports.

If you run the ssh client with "ssh -vvv" then you see
what the SSH server offers and what the client can do.

Preferences can be tweaked from the server side if the
order inside the Chipers / KexAlgorithms / macs
are changed. The same is the case on the client side
if e.g. the allowed algorithms are locked down to a secure
and fast enough Cipher / mac.


I've seen on older CPUs that they limit transfer speed
early if newer algorithms are used. The hardware
accelleration does not help in all cases, as they can't
support all algorithms with accelerated subtasks.
Are junks very small? Then HW-accelleration is usually
killed by the overhead.

So I would experiment with the algorithms / macs and try 
finding one which is accellerated on both sides.

Thomas


> >
> > Can you provide the output of 'ssh -Q cipher' from both your FreeBSD and 
> > OmniOS boxes?
> >
> > /dale
> >
> 
>  Hi
> 
>  Seems to be the same
> 
>  Omnios box - ssh -Q cipher
>  3des-cbc
>  blowfish-cbc
>  cast128-cbc
>  arcfour
>  arcfour128
>  arcfour256
>  aes128-cbc
>  aes192-cbc
>  aes256-cbc
>  rijndael-cbc at lysator.liu.se
>  aes128-ctr
>  aes192-ctr
>  aes256-ctr
>  aes128-gcm at openssh.com
>  aes256-gcm at openssh.com
>  chacha20-poly1305 at openssh.com
> 
> 
>  FreeBSD box - ssh -Q cipher:
>  3des-cbc
>  blowfish-cbc
>  cast128-cbc
>  arcfour
>  arcfour128
>  arcfour256
>  aes128-cbc
>  aes192-cbc
>  aes256-cbc
>  rijndael-cbc at lysator.liu.se
>  aes128-ctr
>  aes192-ctr
>  aes256-ctr
>  aes128-gcm at openssh.com
>  aes256-gcm at openssh.com
>  chacha20-poly1305 at openssh.com
> 
>  _______________________________________________
>  OmniOS-discuss mailing list
>  OmniOS-discuss at lists.omniti.com
>  http://lists.omniti.com/mailman/listinfo/omnios-discuss
> 

-- 
-- 
Thomas Wagner

------------------------------------------------------------------------
Service rund um UNIX(TM),     Wagner Network Services, Thomas Wagner
Solaris(TM), Linux(TM)        Eschenweg 21, 89174 Altheim, Germany
Windows(TM)                   TEL: +49-731-9807799, FAX: +49-731-9807711
Telekommunikation, LAN,       MOBILE/CELL: +49-171-6135989
Internet-Service, Elektronik  EMAIL: wagner at wagner-net.com

From alka at hfg-gmuend.de  Wed Oct 12 18:40:58 2016
From: alka at hfg-gmuend.de (=?UTF-8?Q?G=c3=bcnther_Alka?=)
Date: Wed, 12 Oct 2016 20:40:58 +0200
Subject: [OmniOS-discuss] LX zones and a small Howto
In-Reply-To: <806FD6D2-3C9D-4634-B580-249C8DE2424E@omniti.com>
References: <57FC9913.3080609@btconnect.com>
	<1B02D76B-1341-4559-B166-DAC6AEF8ED26@omniti.com>
	<57FD12F7.1010704@btconnect.com>
	<32CA0CA0-4CE1-40D8-BFE7-A6FE9CA7F659@omniti.com>
	<77EFCE94-0C91-408E-9F79-E679DEAA8CB2@omniti.com>
	<57FD2D7A.1020308@btconnect.com>
	<806FD6D2-3C9D-4634-B580-249C8DE2424E@omniti.com>
Message-ID: <8d20c700-06e8-26bd-e4c9-bdf172328af6@hfg-gmuend.de>

Gratulation!

I have also tried the CentOS example and it works for me.
I have created a small howto setup LX and the VM, see 
http://www.napp-it.org/doc/downloads/zones.pdf

I have not used SmartOS in the past so the concept was at first not 
clear to me.
What I must say is, that this seems to be a real killer innovation for 
OmniOS as you can run OmniOS as a very solid ZFS filer with additional 
Linux services running directly especially with preconfigured services 
as a container where you only need a copy and import/run after some 
small editings of a config file like shares or nics.

What I will add to the current concept in napp-it is deploying 
containers together with the config file (zone.cfg) like it is done on 
ESXi where you can simply provide a folder with the VM that you can 
simply import and run. An option is that one deploy LX images as a 
folder imagefile with the zone config file within and the container with 
the OS as a subfolder.

Gea

Am 12.10.2016 um 04:39 schrieb Dan McDonald:
> The bloody repo server has an updated LX brand package with this fix.
>
> Dan
>
> Sent from my iPhone (typos, autocorrect, and all)
>
> On Oct 11, 2016, at 2:20 PM, Richard Skelton <skeltonr at btconnect.com 
> <mailto:skeltonr at btconnect.com>> wrote:
>
>> Hi All,
>> Dan provided an updated lx_brand file which I installed in 
>> /usr/kernel/brand/amd64/lx_brand
>> After a reboot of the global zone all is now working :-)
>>
>> CentOS release 6.8 (Final)
>> Kernel 2.6.32 on an x86_64
>>
>> lx0 login:
>>
>>
>>
>> Dan McDonald wrote:
>>>> On Oct 11, 2016, at 12:52 PM, Dan McDonald<danmcd at omniti.com>  wrote:
>>>>
>>>> I'm going to check with SmartOS, and see if this is something I fubared, or if it's an upstream (i.e. their) problem too.  I suspect I mismerged something.  It's fairly new, because I swear I used to see CentOS 6 consoles during the io-lx days.
>>>>      
>>>
>>> Not mismerge, but missing bugfix:
>>>
>>>      OS-5685 LTP vhangup02 fails and passes
>>>
>>> Could you do me a favor and test a replacement lx_brand binary?  Please respond unicast, and we'll report back to the list once your post-fix findings match mine.  :)
>>>
>>> Thanks,
>>> Dan
>>>
>>>
>>>    
>
>
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss

-- 

Guenther Alka, Dipl.-Ing. (FH)
Leiter des Rechenzentrums
head of computer center

Tel 07171 602 627
Fax 07171 69259
guenther.alka at hfg-gmuend.de
http://rz.hfg-gmuend.de

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20161012/4b582a4f/attachment.html>

From danmcd at omniti.com  Wed Oct 12 18:52:30 2016
From: danmcd at omniti.com (Dan McDonald)
Date: Wed, 12 Oct 2016 14:52:30 -0400
Subject: [OmniOS-discuss] LX zones and a small Howto
In-Reply-To: <8d20c700-06e8-26bd-e4c9-bdf172328af6@hfg-gmuend.de>
References: <57FC9913.3080609@btconnect.com>
	<1B02D76B-1341-4559-B166-DAC6AEF8ED26@omniti.com>
	<57FD12F7.1010704@btconnect.com>
	<32CA0CA0-4CE1-40D8-BFE7-A6FE9CA7F659@omniti.com>
	<77EFCE94-0C91-408E-9F79-E679DEAA8CB2@omniti.com>
	<57FD2D7A.1020308@btconnect.com>
	<806FD6D2-3C9D-4634-B580-249C8DE2424E@omniti.com>
	<8d20c700-06e8-26bd-e4c9-bdf172328af6@hfg-gmuend.de>
Message-ID: <BA43476B-337E-457F-9649-1BEA2D2FD504@omniti.com>


> On Oct 12, 2016, at 2:40 PM, G?nther Alka <alka at hfg-gmuend.de> wrote:
> 
> What I must say is, that this seems to be a real killer innovation for OmniOS as you can run OmniOS as a very solid ZFS filer with additional Linux services running directly especially with preconfigured services as a container where you only need a copy and import/run after some small editings of a config file like shares or nics.

Joyent did most of the hard work here with LX -- credit where due, please. :)

By integrating it into OmniOS, though, you get the improvment of a fully-working global zone (where many sharing services can only function... e.g. no NFS in an NGZ currently) with the capability for closer-to-native Linux services (not as much running KVM in a zone now, for example).

Thanks for the napp-it update!
Dan


From alka at hfg-gmuend.de  Wed Oct 12 20:10:55 2016
From: alka at hfg-gmuend.de (=?UTF-8?Q?G=c3=bcnther_Alka?=)
Date: Wed, 12 Oct 2016 22:10:55 +0200
Subject: [OmniOS-discuss] LX zones and a small Howto
In-Reply-To: <BA43476B-337E-457F-9649-1BEA2D2FD504@omniti.com>
References: <57FC9913.3080609@btconnect.com>
	<1B02D76B-1341-4559-B166-DAC6AEF8ED26@omniti.com>
	<57FD12F7.1010704@btconnect.com>
	<32CA0CA0-4CE1-40D8-BFE7-A6FE9CA7F659@omniti.com>
	<77EFCE94-0C91-408E-9F79-E679DEAA8CB2@omniti.com>
	<57FD2D7A.1020308@btconnect.com>
	<806FD6D2-3C9D-4634-B580-249C8DE2424E@omniti.com>
	<8d20c700-06e8-26bd-e4c9-bdf172328af6@hfg-gmuend.de>
	<BA43476B-337E-457F-9649-1BEA2D2FD504@omniti.com>
Message-ID: <d0924a68-3eba-5465-3a5b-0d7f5219bb17@hfg-gmuend.de>

Most important is the success of the Illumos or Open-ZFS platform.
LX containers are from Joyent (btw have you read this article 
https://www.infoq.com/news/2016/03/containers-summit-nyc ), SMB2+ and 
sequential resilvering are or may come from Nexenta and native 
encryption from ZoL with many features or additions that strengthen the 
platform from others versus Oracle or Linux.

ESXi, SmartOS or Proxmox are very good as a cloud or virtualisation 
platform where you can run or use ZFS  but OmniOS is a perfect NAS/SAN 
filer or server OS with the option to run a Linux container. This is 
indeed new.

Thanks to all

Gea

Am 12.10.2016 um 20:52 schrieb Dan McDonald:
>> On Oct 12, 2016, at 2:40 PM, G?nther Alka <alka at hfg-gmuend.de> wrote:
>>
>> What I must say is, that this seems to be a real killer innovation for OmniOS as you can run OmniOS as a very solid ZFS filer with additional Linux services running directly especially with preconfigured services as a container where you only need a copy and import/run after some small editings of a config file like shares or nics.
> Joyent did most of the hard work here with LX -- credit where due, please. :)
>
> By integrating it into OmniOS, though, you get the improvment of a fully-working global zone (where many sharing services can only function... e.g. no NFS in an NGZ currently) with the capability for closer-to-native Linux services (not as much running KVM in a zone now, for example).
>
> Thanks for the napp-it update!
> Dan
>

-- 

Guenther Alka, Dipl.-Ing. (FH)
Leiter des Rechenzentrums
head of computer center

Tel 07171 602 627
Fax 07171 69259
guenther.alka at hfg-gmuend.de
http://rz.hfg-gmuend.de


From jstockett at molalla.com  Wed Oct 12 21:52:07 2016
From: jstockett at molalla.com (Jeff Stockett)
Date: Wed, 12 Oct 2016 21:52:07 +0000
Subject: [OmniOS-discuss] problem with boot-archive after upgrade
In-Reply-To: <333BA485-2796-4744-A22F-F30355EE4B36@omniti.com>
References: <136C13E89D22BB468B2A7025993639736E12895F@EXMCCMB.molalla.com>
	<333BA485-2796-4744-A22F-F30355EE4B36@omniti.com>
Message-ID: <136C13E89D22BB468B2A7025993639736E12B5A2@EXMCCMB.molalla.com>

> > I upgraded a system from 012 to 018 just now and everything went totally
> smoothly ? system rebooted fine.  However, the boot environment names
> weren?t what I wanted and since beadm can?t rename the active one, I create
> a new one, activated it, and rebooted again.  On reboot this second (and
> subsequent times), I?m getting this in /var/src/log/system-boot-
> archive:default.log:
> >
> > [ Mar 20 14:25:01 Executing start method
> > ("/lib/svc/method/boot-archive"). ] cannot find: /etc/cluster/nodeid:
> > No such file or directory cannot find: /etc/devices/mdi_ib_cache: No
> > such file or directory cannot find: /etc/devices/retire_store: No such
> > file or directory [ Mar 20 14:25:50 Method "start" exited with status
> > 0. ] [ May 18 05:20:09 Enabled. ] [ May 18 05:20:14 Executing start
> > method ("/lib/svc/method/boot-archive"). ] [ May 18 05:21:15 Method or
> > service exit timed out.  Killing contract 16. ] [ May 18 05:21:15
> > Method "start" failed due to signal KILL. ]
> >
> > The system eventually comes up to a single user mode login, and if I login
> as root, and enter ?svcadm clear system/boot-archive? and/or maybe just
> wait a while longer, then logout the system eventually comes up and works
> fine in multi-user mode, but I was wondering if there is an easy way to fix
> whatever I?ve done to shoot myself in the foot here?  I tried ?bootadm
> update-archive? but that doesn?t appear to fix it.
> 
> When you manually ran 'bootadm update-archive' did it give you any output
> such as "updating..." , or did it silently exit?
> 
> If the latter, do something like 'touch /etc/system' first, then run it, as
> bootadm will regenerate the boot archive only if it detects any changes in the
> files its concerned with. And yes, I've seen this same problem you speak of
> on rare occasion, with a rebuild of the boot archive being the cure.

Thanks for your reply - here is what I get when I try your suggestion which looks promising - but when I reboot, still same problem.

root at zfs01:/root# touch /etc/system; bootadm update-archive -v
cannot find: /etc/cluster/nodeid: No such file or directory
cannot find: /etc/devices/mdi_ib_cache: No such file or directory
    changed /etc/system
mkisofs not found, creating UFS archive
updating /platform/i86pc/amd64/boot_archive
updating /platform/i86pc/boot_archive
root at zfs01:/root#

If you or anyone else has any other ideas, I'm all ears, otherwise I'll probably just live with it since it doesn't seem to cause problems other than slowing the initial boot.

From ergi.thanasko at avsquad.com  Thu Oct 13 23:13:59 2016
From: ergi.thanasko at avsquad.com (Ergi Thanasko)
Date: Thu, 13 Oct 2016 23:13:59 +0000
Subject: [OmniOS-discuss] Network throughout 1GB/sec
In-Reply-To: <3419113d-b3ba-7ac7-30cb-945f6d2192ff@hfg-gmuend.de>
References: <56B30910-D0CB-4C7A-9848-313358007C6E@avsquad.com>
	<3419113d-b3ba-7ac7-30cb-945f6d2192ff@hfg-gmuend.de>
Message-ID: <AB670D9B-2944-46F1-9117-1157E0902AE1@avsquad.com>

Hi Gea,
Your property values are a bit off, here is what  i got from EMC
https://community.emc.com/docs/DOC-39156

  *   ipadm set-prop ?p send_buf =2097152 tcp
  *   ipadm set-prop ?p recv_buf=2097152 tcp
  *   ipadm set-prop ?p max_buf=16777216 tcp
  *   ipadm set-prop ?p _cwnd_max=8388608 tcp

also the power of 2 in twenty is 1048576.

Can you confirm that the set-prop -p values should the 2 in the power of 20 and 21st
[cid:449A210A-451D-40E1-97EA-98BA5E7FA558 at fios-router.home]

On Sep 16, 2016, at 12:57 PM, Guenther Alka <alka at hfg-gmuend.de<mailto:alka at hfg-gmuend.de>> wrote:

I have made some investigations into 10G and found that 300-400MB/s is expected with default settings. Improvements are possible up to 1000MB/s via mtu 9000 and if you increase ip buffers ex
max_buf=4097152 tcp
send_buf=2048576 tcp
recv_buf=2048576 tcp,

NFS lockd servers (ex 1024), NFS number of threads (ex 64) and NFS transfer size (ex 1048576)

http://napp-it.org/doc/downloads/performance_smb2.pdf


Gea

Am 16.09.2016 um 19:43 schrieb Ergi Thanasko:
Hi all,
We have a a few servers  conected via 10g nic  LACP, some of them have  4nic and some have 6nic in a link aggregation mode. We been moving a lot of data around and we are trying to get the maximum performance. I have seen zpool can deliver  2-3GB accumulated  throughput. Iperf does about 600-800MB/sec between those two servers.
Given  the hardware that we have and the zpool performance,   we expected   to see some serious data transfer rates  however we only see around 200-300MB/sec average  using rsync or copy paste over NFS.  Standard MTU 1500 and nfs block size.  I want to ask the community what to do get some higher throughout and the application level. I hear ZFS send/receive   or ZFS shadow does work faster but it does snapshots. Out data (Terabytes) is constantly evolving   and we prefer something in the nature of rsync  but to utilize the network hardware.

If Anyone has a hardware setup that can see 1GB/sec  throughput  and does not mind sharing?
Any software  that  use multithreads  sessions to move data around  zfs friendly? We do not mind getting going with a commercial solution like camvault or veeam if they work.

Thank you for your time



_______________________________________________
OmniOS-discuss mailing list
OmniOS-discuss at lists.omniti.com
http://lists.omniti.com/mailman/listinfo/omnios-discuss

_______________________________________________
OmniOS-discuss mailing list
OmniOS-discuss at lists.omniti.com
http://lists.omniti.com/mailman/listinfo/omnios-discuss

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20161013/d9805b10/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ErgiThanasko.png
Type: image/png
Size: 7910 bytes
Desc: ErgiThanasko.png
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20161013/d9805b10/attachment-0001.png>

From terminator2202 at web.de  Fri Oct 14 10:37:49 2016
From: terminator2202 at web.de (terminator2202 at web.de)
Date: Fri, 14 Oct 2016 12:37:49 +0200
Subject: [OmniOS-discuss] vmware and omnios problem with passthru
Message-ID: <trinity-a65f24b3-aa85-4070-a5e6-132f98fc325c-1476441469266@3capp-webde-bap38>

An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20161014/832a31da/attachment.html>

From daleg at omniti.com  Fri Oct 14 14:30:05 2016
From: daleg at omniti.com (Dale Ghent)
Date: Fri, 14 Oct 2016 10:30:05 -0400
Subject: [OmniOS-discuss] vmware and omnios problem with passthru
In-Reply-To: <trinity-a65f24b3-aa85-4070-a5e6-132f98fc325c-1476441469266@3capp-webde-bap38>
References: <trinity-a65f24b3-aa85-4070-a5e6-132f98fc325c-1476441469266@3capp-webde-bap38>
Message-ID: <FDEACE29-D446-45E8-BE98-82194644B8E4@omniti.com>


Interesting that it seems to happen upon making the ixgbe driver do work. However, vmware logs don't do us much good here as they don't actually give us any valuable information. Would would need to see the crash info from OmniOS and a the crash dump.

/dale

> On Oct 14, 2016, at 6:37 AM, terminator2202 at web.de wrote:
> 
> Hello,
> i have some problems with omnios and VMware 6U2. I am not sure if it is VMware related or omnios related.
>  
> First, my  Hardware:
> X10SRL-F with XEON CPU
> 64GB ECC RAM
> 2x M1015 SAS Controller with IT Mode (Firmware P19)
> 1x Intel 540x-T1
> and one or two other PCIe Devices
>  
> I have some issues with Passthru some PCIe devices.
> i create a VM with the newest omnios_bloody (bloody_20161006.iso).
> After that i passthru the 2x M1015 and import the exisiting pool -> no problem -> run's stable.
> (see log WITHOUT problems/crash: http://pastebin.com/Y0G5YHEg )
>  
> After that i tried to passthru my intel x540-T1 NIC (and later a i210-NIC onboard-Port). About 10 seconds after omibos boot the vm just crashes.
> Sometimes it is stable for a bit longer and i can login and execute dladm show-phys or show-link but after that i crashes instantly.
> (see log WITH problems/crash: http://pastebin.com/gPNLrdg7 )
>  
> I also tried to install omnios without vmware (native) and i have no problems at all. Omnios regocnize all discs on M1015 and x540 NIC and i210 NIC.
> I guess the issues are msi-x/interups related, but maybe you can help with that. 
>  
> Thanks, manuel
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss


From geoffn at gnaa.net  Fri Oct 14 19:52:00 2016
From: geoffn at gnaa.net (Geoff Nordli)
Date: Fri, 14 Oct 2016 12:52:00 -0700
Subject: [OmniOS-discuss] upgrade from r151008
In-Reply-To: <F383D5AF-B644-4A36-9351-42B285C79350@omniti.com>
References: <e3b8f6cf-92ff-4333-a7c8-26607cd9d733@gnaa.net>
	<B9B07CB6-D30D-4B9E-A636-2EC2A439991B@omniti.com>
	<c10dcf54-6bb7-cb7e-fd08-ab774bdb95d7@gnaa.net>
	<F383D5AF-B644-4A36-9351-42B285C79350@omniti.com>
Message-ID: <0585465a-1242-a904-6123-a1edacdf6633@gnaa.net>


On 2016-10-05 04:32 PM, Dan McDonald wrote:
>> On Oct 5, 2016, at 7:03 PM, Geoff Nordli <geoffn at gnaa.net> wrote:
>>
>> Always a risk when you wait too long between versions.
>>
>>
> We tested 006 --> 014 pre-release of 014.  008 --> 014 should be equally workable.
>
> Dan
>


Upgrade worked perfectly.  006 --> 018.

Nothing fancy on it though, just a basic install as a storage server.

Happy Friday!!

Geoff

From daleg at omniti.com  Fri Oct 14 19:56:01 2016
From: daleg at omniti.com (Dale Ghent)
Date: Fri, 14 Oct 2016 15:56:01 -0400
Subject: [OmniOS-discuss] upgrade from r151008
In-Reply-To: <0585465a-1242-a904-6123-a1edacdf6633@gnaa.net>
References: <e3b8f6cf-92ff-4333-a7c8-26607cd9d733@gnaa.net>
	<B9B07CB6-D30D-4B9E-A636-2EC2A439991B@omniti.com>
	<c10dcf54-6bb7-cb7e-fd08-ab774bdb95d7@gnaa.net>
	<F383D5AF-B644-4A36-9351-42B285C79350@omniti.com>
	<0585465a-1242-a904-6123-a1edacdf6633@gnaa.net>
Message-ID: <57213295-D6C4-46F1-AB0C-8C2F090FE1C4@omniti.com>


> On Oct 14, 2016, at 3:52 PM, Geoff Nordli <geoffn at gnaa.net> wrote:
> 
> 
> On 2016-10-05 04:32 PM, Dan McDonald wrote:
>>> On Oct 5, 2016, at 7:03 PM, Geoff Nordli <geoffn at gnaa.net> wrote:
>>> 
>>> Always a risk when you wait too long between versions.
>>> 
>>> 
>> We tested 006 --> 014 pre-release of 014.  008 --> 014 should be equally workable.
>> 
>> Dan
>> 
> 
> 
> Upgrade worked perfectly.  006 --> 018.
> 
> Nothing fancy on it though, just a basic install as a storage server.
> 

Remember to upgrade your zpools. Also, 020 is coming soon enough :)

/dale


From terminator2202 at web.de  Sat Oct 15 14:31:23 2016
From: terminator2202 at web.de (terminator2202 at web.de)
Date: Sat, 15 Oct 2016 16:31:23 +0200
Subject: [OmniOS-discuss] vmware and omnios problem with passthru
In-Reply-To: <FDEACE29-D446-45E8-BE98-82194644B8E4@omniti.com>
References: <trinity-a65f24b3-aa85-4070-a5e6-132f98fc325c-1476441469266@3capp-webde-bap38>,
	<FDEACE29-D446-45E8-BE98-82194644B8E4@omniti.com>
Message-ID: <trinity-b1076b5f-a0b3-4942-85e8-980bc51a7cbf-1476541883120@3capp-webde-bs44>

An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20161015/c1717bc2/attachment.html>

From danmcd at omniti.com  Tue Oct 18 17:40:54 2016
From: danmcd at omniti.com (Dan McDonald)
Date: Tue, 18 Oct 2016 13:40:54 -0400
Subject: [OmniOS-discuss] NEED FEEDBACK:  VIM8 and "vi" behavior
Message-ID: <17FB4959-79E3-49E9-BA59-397468C936D6@omniti.com>

VIM8 is in the omnios-build master branch for next-bloody and r151020.

VIM8's defaults are more VIM-like and less classic-VI like.

Would it kill people if I made "vi" act like "vim -C"?  Or would that be MORE surprising?

Curious,
Dan


From gate03 at landcroft.co.uk  Tue Oct 18 21:27:22 2016
From: gate03 at landcroft.co.uk (Michael Mounteney)
Date: Wed, 19 Oct 2016 07:27:22 +1000
Subject: [OmniOS-discuss] NEED FEEDBACK:  VIM8 and "vi" behavior
In-Reply-To: <17FB4959-79E3-49E9-BA59-397468C936D6@omniti.com>
References: <17FB4959-79E3-49E9-BA59-397468C936D6@omniti.com>
Message-ID: <20161019072722.1b979323@punda-mlia>

On Tue, 18 Oct 2016 13:40:54 -0400
Dan McDonald <danmcd at omniti.com> wrote:

> VIM8 is in the omnios-build master branch for next-bloody and r151020.
> 
> VIM8's defaults are more VIM-like and less classic-VI like.
> 
> Would it kill people if I made "vi" act like "vim -C"?  Or would that
> be MORE surprising?

No strong opinion here.  I use vim on Linux so am used to it, and am
slightly caught off guard by the differences of vi on OmniOS.

______________
Michael Mounteney

From bfriesen at simple.dallas.tx.us  Wed Oct 19 02:45:11 2016
From: bfriesen at simple.dallas.tx.us (Bob Friesenhahn)
Date: Tue, 18 Oct 2016 21:45:11 -0500 (CDT)
Subject: [OmniOS-discuss] Firewall in OmniOS?
Message-ID: <alpine.GSO.2.20.1610182135560.17782@freddy.simplesystems.org>

While I do have a dedicated low-power box I can use, it came to mind 
that I could use OmniOS on a more powerful Internet-facing server and 
use it as the firewall for my network (screening and session-oriented 
NAT).  Assuming that the Internet-facing interface is sufficiently 
battened down, this should provide optimum performance for other 
software running on the same system while providing good security.

I know that ipfilter is available in OmniOS but have heard that it is 
a bit dated when it comes to NAT.

Has anyone tried (or is it feasible) to install OpenBSD or FreeBSD 
(perhaps pfSense or OPNsense) in a KVM instance (running in a zone) to 
provide an effective firewall for LANs?

Bob
-- 
Bob Friesenhahn
bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/

From johan.kragsterman at capvert.se  Wed Oct 19 07:33:31 2016
From: johan.kragsterman at capvert.se (Johan Kragsterman)
Date: Wed, 19 Oct 2016 09:33:31 +0200
Subject: [OmniOS-discuss] Ang:  Firewall in OmniOS?
In-Reply-To: <alpine.GSO.2.20.1610182135560.17782@freddy.simplesystems.org>
References: <alpine.GSO.2.20.1610182135560.17782@freddy.simplesystems.org>
Message-ID: <OFB3256B86.AFA81B08-ONC1258051.002936F7-C1258051.0029857F@inse.com>


Hi, Bob!




-----"OmniOS-discuss" <omnios-discuss-bounces at lists.omniti.com> skrev: -----
Till: omnios-discuss at lists.omniti.com
Fr?n: Bob Friesenhahn 
S?nt av: "OmniOS-discuss" 
Datum: 2016-10-19 02:47
?rende: [OmniOS-discuss] Firewall in OmniOS?

While I do have a dedicated low-power box I can use, it came to mind 
that I could use OmniOS on a more powerful Internet-facing server and 
use it as the firewall for my network (screening and session-oriented 
NAT). ?Assuming that the Internet-facing interface is sufficiently 
battened down, this should provide optimum performance for other 
software running on the same system while providing good security.

I know that ipfilter is available in OmniOS but have heard that it is 
a bit dated when it comes to NAT.

Has anyone tried (or is it feasible) to install OpenBSD or FreeBSD 
(perhaps pfSense or OPNsense) in a KVM instance (running in a zone) to 
provide an effective firewall for LANs?



Yeah, I do that since several yrs with PfSense. I use it a lot, it is of great help, since it is so easy to handle and has a lot of great features.

I think I have 5 virtual interfaces on it, both strictly internal and the ones going out of the box.

You can say that whatever I handle in my infrastructure, PfSense is involved. I like fixed dhcp addressing, so most interfaces provides that.

Rgrds Johan





Bob
-- 
Bob Friesenhahn
bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer, ? ?http://www.GraphicsMagick.org/
_______________________________________________

OmniOS-discuss mailing list
OmniOS-discuss at lists.omniti.com
http://lists.omniti.com/mailman/listinfo/omnios-discuss













From omnios at citrus-it.net  Wed Oct 19 12:10:41 2016
From: omnios at citrus-it.net (Andy Fiddaman)
Date: Wed, 19 Oct 2016 12:10:41 +0000 (UTC)
Subject: [OmniOS-discuss] NEED FEEDBACK:  VIM8 and "vi" behavior
In-Reply-To: <17FB4959-79E3-49E9-BA59-397468C936D6@omniti.com>
References: <17FB4959-79E3-49E9-BA59-397468C936D6@omniti.com>
Message-ID: <alpine.GSO.2.00.1610191209140.7726@erncre.pvgehf-vg.arg>


On Tue, 18 Oct 2016, Dan McDonald wrote:

; VIM8 is in the omnios-build master branch for next-bloody and r151020.
;
; VIM8's defaults are more VIM-like and less classic-VI like.
;
; Would it kill people if I made "vi" act like "vim -C"?  Or would that be MORE surprising?

No preference here. We replace /usr/share/vim/vimrc with our own version
anyway.

Andy
-- 
Citrus IT Limited | +44 (0)870 199 8000 | enquiries at citrus-it.co.uk
Rock House Farm | Green Moor | Wortley | Sheffield | S35 7DQ
Registered in England and Wales | Company number 4899123


From danmcd at omniti.com  Wed Oct 19 15:58:25 2016
From: danmcd at omniti.com (Dan McDonald)
Date: Wed, 19 Oct 2016 11:58:25 -0400
Subject: [OmniOS-discuss] Firewall in OmniOS?
In-Reply-To: <alpine.GSO.2.20.1610182135560.17782@freddy.simplesystems.org>
References: <alpine.GSO.2.20.1610182135560.17782@freddy.simplesystems.org>
Message-ID: <A32FD975-06DA-43EC-A394-994934690107@omniti.com>

I still use ipfilter, but only as a NAT.

Porting one of the others to OmniOS/illumos would require a lot of work.  (Hell, part of me thinks writing a NAT-only hooks module would be easier.)

My $.02,
Dan


From gary at genashor.com  Wed Oct 19 16:17:44 2016
From: gary at genashor.com (Gary Gendel)
Date: Wed, 19 Oct 2016 12:17:44 -0400
Subject: [OmniOS-discuss] Firewall in OmniOS?
In-Reply-To: <A32FD975-06DA-43EC-A394-994934690107@omniti.com>
References: <alpine.GSO.2.20.1610182135560.17782@freddy.simplesystems.org>
	<A32FD975-06DA-43EC-A394-994934690107@omniti.com>
Message-ID: <a17b2795-e672-d233-404c-b0c9cf81a357@genashor.com>

I used OpenIndiana and OmniOS with ipfilter in a router configuration 
for years without any serious issues.  I was also providing DNS, DHCP 
and IPv6 over v4. However, one of the drawbacks of this is that, if you 
use this as your WAN/LAN router, you bring your whole network down with 
the occasional reboot.

I ended up picking up a MicroTik RB750 for about $50.  This is no longer 
made but the "hEX" replacement looks sweet.  These have Cisco-level 
functionality, a tiny footprint, and low power consumption.  I've never 
looked back.  I'm running a split-horizon DNS with hairpin routing rules 
so the internal network machines get the internal ip addresses for the 
services.  It reboots in a few seconds so there is minimal downtime when 
I upgrade the software.

Adding my $.02 too.

Gary

On 10/19/2016 11:58 AM, Dan McDonald wrote:
> I still use ipfilter, but only as a NAT.
>
> Porting one of the others to OmniOS/illumos would require a lot of work.  (Hell, part of me thinks writing a NAT-only hooks module would be easier.)
>
> My $.02,
> Dan
>
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss



-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3703 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20161019/aede22f6/attachment.bin>

From danmcd at omniti.com  Wed Oct 19 18:11:15 2016
From: danmcd at omniti.com (Dan McDonald)
Date: Wed, 19 Oct 2016 14:11:15 -0400
Subject: [OmniOS-discuss] OpenSSH Update
Message-ID: <2E60B827-AF11-4DD1-958E-4C710E26BD5C@omniti.com>

There's a small memory leak in OpenSSH that, if properly attacked, can cause OpenSSH to eat a bunch of memory.

If you're using OpenSSH on supported releases (014, 016, 018), please utter "pkg update" now.  Your OpenSSH service will restart automatically with the fixed sshd after updating is completed.

Happy patching!
Dan


From skiselkov.ml at gmail.com  Wed Oct 19 20:55:07 2016
From: skiselkov.ml at gmail.com (Saso Kiselkov)
Date: Wed, 19 Oct 2016 22:55:07 +0200
Subject: [OmniOS-discuss] r151014 on top of Linux KVM spontaneously reboots
Message-ID: <3e7253c1-ed6d-39fb-20b6-fe7e4582835c@gmail.com>

So I've been trying to get illumos-gate built on a fresh r151014 install
in a Linux KVM host (Ubuntu 16.04 64-bit). It's running on a standard
qcow2 16GB DATA disk, 2GB of vRAM, 4 cores on the Core i5 CPU and
console redirected to serial. Other than that, no tuning. The problem
is, when I start the gate build process, randomly after about 15-30
minutes of building, the machine just spontaneously reboots. No panic,
no log message in libvirt, nothing (even when kmdb was loaded). Out of
nothing, the VM just hits a reset and reappears in the bootloader.

Has anybody else seen this?

Cheers,
-- 
Saso

From danmcd at omniti.com  Wed Oct 19 21:02:21 2016
From: danmcd at omniti.com (Dan McDonald)
Date: Wed, 19 Oct 2016 17:02:21 -0400
Subject: [OmniOS-discuss] r151014 on top of Linux KVM spontaneously
	reboots
In-Reply-To: <3e7253c1-ed6d-39fb-20b6-fe7e4582835c@gmail.com>
References: <3e7253c1-ed6d-39fb-20b6-fe7e4582835c@gmail.com>
Message-ID: <BADDC572-E755-429D-A26F-76A939270F1D@omniti.com>

Not sure how hard it'd be, but can you see if 018 has similar problems?

Dan

Sent from my iPhone (typos, autocorrect, and all)

> On Oct 19, 2016, at 4:55 PM, Saso Kiselkov <skiselkov.ml at gmail.com> wrote:
> 
> So I've been trying to get illumos-gate built on a fresh r151014 install
> in a Linux KVM host (Ubuntu 16.04 64-bit). It's running on a standard
> qcow2 16GB DATA disk, 2GB of vRAM, 4 cores on the Core i5 CPU and
> console redirected to serial. Other than that, no tuning. The problem
> is, when I start the gate build process, randomly after about 15-30
> minutes of building, the machine just spontaneously reboots. No panic,
> no log message in libvirt, nothing (even when kmdb was loaded). Out of
> nothing, the VM just hits a reset and reappears in the bootloader.
> 
> Has anybody else seen this?
> 
> Cheers,
> -- 
> Saso
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss

From skiselkov.ml at gmail.com  Wed Oct 19 21:07:36 2016
From: skiselkov.ml at gmail.com (Saso Kiselkov)
Date: Wed, 19 Oct 2016 23:07:36 +0200
Subject: [OmniOS-discuss] r151014 on top of Linux KVM spontaneously
 reboots
In-Reply-To: <BADDC572-E755-429D-A26F-76A939270F1D@omniti.com>
References: <3e7253c1-ed6d-39fb-20b6-fe7e4582835c@gmail.com>
	<BADDC572-E755-429D-A26F-76A939270F1D@omniti.com>
Message-ID: <a9b877a7-324a-5bd7-14c8-cdca0c88f864@gmail.com>

On 10/19/16 11:02 PM, Dan McDonald wrote:
> Not sure how hard it'd be, but can you see if 018 has similar problems?

I'm gonna try to give it a go.

-- 
Saso

>> On Oct 19, 2016, at 4:55 PM, Saso Kiselkov <skiselkov.ml at gmail.com> wrote:
>>
>> So I've been trying to get illumos-gate built on a fresh r151014 install
>> in a Linux KVM host (Ubuntu 16.04 64-bit). It's running on a standard
>> qcow2 16GB DATA disk, 2GB of vRAM, 4 cores on the Core i5 CPU and
>> console redirected to serial. Other than that, no tuning. The problem
>> is, when I start the gate build process, randomly after about 15-30
>> minutes of building, the machine just spontaneously reboots. No panic,
>> no log message in libvirt, nothing (even when kmdb was loaded). Out of
>> nothing, the VM just hits a reset and reappears in the bootloader.
>>
>> Has anybody else seen this?
>>
>> Cheers,
>> -- 
>> Saso
>> _______________________________________________
>> OmniOS-discuss mailing list
>> OmniOS-discuss at lists.omniti.com
>> http://lists.omniti.com/mailman/listinfo/omnios-discuss


From david.ledger at ivdcs.co.uk  Wed Oct 19 21:14:11 2016
From: david.ledger at ivdcs.co.uk (David Ledger)
Date: Wed, 19 Oct 2016 22:14:11 +0100
Subject: [OmniOS-discuss] NEED FEEDBACK:  VIM8 and "vi" behavior
In-Reply-To: <17FB4959-79E3-49E9-BA59-397468C936D6@omniti.com>
References: <17FB4959-79E3-49E9-BA59-397468C936D6@omniti.com>
Message-ID: <9B403F44-2C9C-413A-AF10-B9B2EF071231@ivdcs.co.uk>


> On 18 Oct 2016, at 18:40, Dan McDonald <danmcd at omniti.com> wrote:
> 
> VIM8 is in the omnios-build master branch for next-bloody and r151020.
> 
> VIM8's defaults are more VIM-like and less classic-VI like.
> 
> Would it kill people if I made "vi" act like "vim -C"?  Or would that be MORE surprising?
> 
> Curious,
> Dan

As long as I can configure things so that I have a vi that works like that in Bsd 4.3 it?s ok. The thing I fall over most often is the simple ^U^U to scroll back to the last change. It?s muscle memory and loses me two changes on some versions.

David - Unix since ?83



From skiselkov.ml at gmail.com  Wed Oct 19 22:00:11 2016
From: skiselkov.ml at gmail.com (Saso Kiselkov)
Date: Thu, 20 Oct 2016 00:00:11 +0200
Subject: [OmniOS-discuss] r151014 on top of Linux KVM spontaneously
 reboots
In-Reply-To: <BADDC572-E755-429D-A26F-76A939270F1D@omniti.com>
References: <3e7253c1-ed6d-39fb-20b6-fe7e4582835c@gmail.com>
	<BADDC572-E755-429D-A26F-76A939270F1D@omniti.com>
Message-ID: <065cd587-8ec6-f4ea-b282-6d277fbfa94c@gmail.com>

On 10/19/16 11:02 PM, Dan McDonald wrote:
> Not sure how hard it'd be, but can you see if 018 has similar problems?

Tried it, still happens. After about 10 minutes of building, VM
spontaneously reboots.

-- 
Saso

>> On Oct 19, 2016, at 4:55 PM, Saso Kiselkov <skiselkov.ml at gmail.com> wrote:
>>
>> So I've been trying to get illumos-gate built on a fresh r151014 install
>> in a Linux KVM host (Ubuntu 16.04 64-bit). It's running on a standard
>> qcow2 16GB DATA disk, 2GB of vRAM, 4 cores on the Core i5 CPU and
>> console redirected to serial. Other than that, no tuning. The problem
>> is, when I start the gate build process, randomly after about 15-30
>> minutes of building, the machine just spontaneously reboots. No panic,
>> no log message in libvirt, nothing (even when kmdb was loaded). Out of
>> nothing, the VM just hits a reset and reappears in the bootloader.
>>
>> Has anybody else seen this?
>>
>> Cheers,
>> -- 
>> Saso
>> _______________________________________________
>> OmniOS-discuss mailing list
>> OmniOS-discuss at lists.omniti.com
>> http://lists.omniti.com/mailman/listinfo/omnios-discuss


From danmcd at omniti.com  Thu Oct 20 04:15:51 2016
From: danmcd at omniti.com (Dan McDonald)
Date: Thu, 20 Oct 2016 00:15:51 -0400
Subject: [OmniOS-discuss] Likely last bloody update before r151020
Message-ID: <BF6EDC2D-A986-44A1-9A16-39EC98AE2AAA@omniti.com>

I hope this is the bloody that will form the r151020 branch.  Please note that because of the sheer volume of upstream illumos-gate activity, we froze pulls from -gate early this time, early enough to not include BSD Loader and Python 2.7 support.  A few cherrypicks may happen pre-020, and a couple have already happened:

- illumos 4498, update ACPI to version 6.x
- A series of nvme and blkdev fixes culminating in illumos 7382, basic NVMe 1.1 support.

uname -v says omnios-master-2f9273c, corresponding to the illumos-omnios:master commit, and omnios-build is at commit 8d79e11.  This is likely to be where we branch r151020.

Staying with Python2.6 means that if you wish to compile illumos-gate on ANY OmniOS, you need to add these two lines to your .env file:

+# Starting with illumos bug 5969, upstream defaults to Python2.7.  We still
+# use Python2.6 as of r151020
+export PYTHON_VERSION="2.6"
+export PYTHON_PKGVERS="-26"
+

The shipping "/opt/onbld/env/omnios-illumos-gate" file has these lines in it now.

The rest of the software in omnios-build has been updated as well.  Including:

* GNU grep is now 2.26
* GNU tar is now 1.29
* gzip is now 1.8
* iperf is now 3.1.3
* isc-dhcp is now 4.3.5
* iso-codes are now 3.70
* libidn is now 1.33
* libxslt is now 1.1.29
* mercurial is now 3.9.2
* mozilla-nss-nspr is up to 3.25 and 4.12, respectively
* nghttp2 is new, at version 1.15.0
* p7zip is now at 16.02.
* pciutils are now at 3.5.2
* pcre is now at 8.39
* python26-coverage is now at  4.2
* python26-lxml is now at 3.6.4
* python26-numpy is now at 1.11.2
* python26-ply is now at 3.9
* python26-pylint is now at 1.6.4
* readline is now at 7.0, but we will ship the 6.3 libraries from r151018 to maintain linked-application compatibility
* screen is now at 4.4.0
* sigcpp is now at 2.10.0
* sqlite3 is now at 3.15.0
* tmux is now at 2.3
* vim is now at 8.0 (and "vi" is, for now, going to behave as if you invoked vim, because of how 8.0 works - see the other thread on omnios-discuss about this)


This is an repo-only update.

Happy updating!
Dan


From Josh.Barton at usurf.usu.edu  Thu Oct 20 13:30:23 2016
From: Josh.Barton at usurf.usu.edu (Josh Barton)
Date: Thu, 20 Oct 2016 13:30:23 +0000
Subject: [OmniOS-discuss] IPS and Updating
Message-ID: <E82AA97B-77D0-4C3E-BBD8-B156429F55F8@usurf.usu.edu>

I have set up an IPS repository for all the software we have compiled and added it as a publisher on one of or boxes. As it turns out, running "pkg update" fails because OmniOs tried to download he updates from my personal repository rather than directly from
Omniti. Is there a work around for this? Is it possible that I added my custom publisher incorrectly?

Josh Barton


USU Research Foundation
1695 Research Park Way
North Logan UT, 84321
Office: (435) 713-3089  Cell: (435) 770-4179

From danmcd at omniti.com  Thu Oct 20 13:36:36 2016
From: danmcd at omniti.com (Dan McDonald)
Date: Thu, 20 Oct 2016 09:36:36 -0400
Subject: [OmniOS-discuss] IPS and Updating
In-Reply-To: <E82AA97B-77D0-4C3E-BBD8-B156429F55F8@usurf.usu.edu>
References: <E82AA97B-77D0-4C3E-BBD8-B156429F55F8@usurf.usu.edu>
Message-ID: <CE31EABD-6D6E-4954-907D-E5CC603790A6@omniti.com>


> On Oct 20, 2016, at 9:30 AM, Josh Barton <Josh.Barton at usurf.usu.edu> wrote:
> 
> I have set up an IPS repository for all the software we have compiled and added it as a publisher on one of or boxes. As it turns out, running "pkg update" fails because OmniOs tried to download he updates from my personal repository rather than directly from
> Omniti. Is there a work around for this? Is it possible that I added my custom publisher incorrectly?

You need to share more specific information for folks to help.

Please share the output of:

	pkg publisher

and

	pkg update -v

So people can see what package update triggered the failure.

There are a lot of ways things can go wrong, so we need to narrow things down.

Dan


From skiselkov.ml at gmail.com  Thu Oct 20 14:47:12 2016
From: skiselkov.ml at gmail.com (Saso Kiselkov)
Date: Thu, 20 Oct 2016 16:47:12 +0200
Subject: [OmniOS-discuss] r151014 on top of Linux KVM spontaneously
 reboots
In-Reply-To: <BADDC572-E755-429D-A26F-76A939270F1D@omniti.com>
References: <3e7253c1-ed6d-39fb-20b6-fe7e4582835c@gmail.com>
	<BADDC572-E755-429D-A26F-76A939270F1D@omniti.com>
Message-ID: <8dca4287-62b7-f09c-466a-0b440e623a3b@gmail.com>

On 10/19/16 11:02 PM, Dan McDonald wrote:
> Not sure how hard it'd be, but can you see if 018 has similar problems?

Managed to get this resolved by upping amount of vram. My suspicion was
roused because I saw a shell command return with "fork: not enough
space". After I went from 2G to 4G of vram and swap from 1g to 2g, full
build /w lint went through without a hitch, twice. Maybe just increasing
swap would have been enough, at the expense of the speed of the build.

Cheers,
-- 
Saso

From tim at multitalents.net  Fri Oct 21 03:19:24 2016
From: tim at multitalents.net (Tim Rice)
Date: Thu, 20 Oct 2016 20:19:24 -0700 (PDT)
Subject: [OmniOS-discuss] NEED FEEDBACK: VIM8 and "vi" behavior
In-Reply-To: <17FB4959-79E3-49E9-BA59-397468C936D6@omniti.com>
References: <17FB4959-79E3-49E9-BA59-397468C936D6@omniti.com>
Message-ID: <alpine.UW2.2.11.1610202013560.28443@server01.int.multitalents.net>


Hi Dan,

On Tue, 18 Oct 2016, Dan McDonald wrote:

> VIM8 is in the omnios-build master branch for next-bloody and r151020.
> 
> VIM8's defaults are more VIM-like and less classic-VI like.
> 
> Would it kill people if I made "vi" act like "vim -C"?  Or would that be MORE surprising?

On the contrary, having "vi" act like vi would be great. On Linux distros
I always have to patch the system vimrc so I don't get annoyed all the 
time. If someone wants vim "features" they can invoke vim.

> Curious,
> Dan
> 

-- 
Tim Rice				Multitalents
tim at multitalents.net



From alka at hfg-gmuend.de  Fri Oct 21 15:04:35 2016
From: alka at hfg-gmuend.de (Guenther Alka)
Date: Fri, 21 Oct 2016 17:04:35 +0200
Subject: [OmniOS-discuss] LX zones and a small Howto
In-Reply-To: <BA43476B-337E-457F-9649-1BEA2D2FD504@omniti.com>
References: <57FC9913.3080609@btconnect.com>
	<1B02D76B-1341-4559-B166-DAC6AEF8ED26@omniti.com>
	<57FD12F7.1010704@btconnect.com>
	<32CA0CA0-4CE1-40D8-BFE7-A6FE9CA7F659@omniti.com>
	<77EFCE94-0C91-408E-9F79-E679DEAA8CB2@omniti.com>
	<57FD2D7A.1020308@btconnect.com>
	<806FD6D2-3C9D-4634-B580-249C8DE2424E@omniti.com>
	<8d20c700-06e8-26bd-e4c9-bdf172328af6@hfg-gmuend.de>
	<BA43476B-337E-457F-9649-1BEA2D2FD504@omniti.com>
Message-ID: <d8f9902b-38e7-d989-2d09-dd9a5702d15a@hfg-gmuend.de>

I have added a menu in napp-it to select, download and preconfigure LX images from Joyent
beside the menu to configure/ attach/ start/ stop/ detach/ delete

http://www.napp-it.org/doc/downloads/zones.pdf

Whats not yet clear to me is the way to configure lx zones with 
zoneadm/zonecfg according to the settings in the Joyent XML file like 
network ip/dhcp, router, netmask, gateway, maxram, ssh, mount zfs 
filesystems to name a few.

btw
what about kvm branded zones from Joyent?
the concept is very new to me....

you may discuss at
https://forums.servethehome.com/index.php?threads/omnios-now-includes-lx-support-from-joyent-smartos.11577/



>> On Oct 12, 2016, at 2:40 PM, G?nther Alka <alka at hfg-gmuend.de> wrote:
>>
>> What I must say is, that this seems to be a real killer innovation for OmniOS as you can run OmniOS as a very solid ZFS filer with additional Linux services running directly especially with preconfigured services as a container where you only need a copy and import/run after some small editings of a config file like shares or nics.
> Joyent did most of the hard work here with LX -- credit where due, please. :)
>
> By integrating it into OmniOS, though, you get the improvment of a fully-working global zone (where many sharing services can only function... e.g. no NFS in an NGZ currently) with the capability for closer-to-native Linux services (not as much running KVM in a zone now, for example).
>
> Thanks for the napp-it update!
> Dan
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20161021/211e1ab8/attachment.html>

From danmcd at omniti.com  Fri Oct 21 17:20:52 2016
From: danmcd at omniti.com (Dan McDonald)
Date: Fri, 21 Oct 2016 13:20:52 -0400
Subject: [OmniOS-discuss] LX zones and a small Howto
In-Reply-To: <d8f9902b-38e7-d989-2d09-dd9a5702d15a@hfg-gmuend.de>
References: <57FC9913.3080609@btconnect.com>
	<1B02D76B-1341-4559-B166-DAC6AEF8ED26@omniti.com>
	<57FD12F7.1010704@btconnect.com>
	<32CA0CA0-4CE1-40D8-BFE7-A6FE9CA7F659@omniti.com>
	<77EFCE94-0C91-408E-9F79-E679DEAA8CB2@omniti.com>
	<57FD2D7A.1020308@btconnect.com>
	<806FD6D2-3C9D-4634-B580-249C8DE2424E@omniti.com>
	<8d20c700-06e8-26bd-e4c9-bdf172328af6@hfg-gmuend.de>
	<BA43476B-337E-457F-9649-1BEA2D2FD504@omniti.com>
	<d8f9902b-38e7-d989-2d09-dd9a5702d15a@hfg-gmuend.de>
Message-ID: <12F5F8D5-9676-4B63-B9A8-2B896EFEEE1F@omniti.com>


> On Oct 21, 2016, at 11:04 AM, Guenther Alka <alka at hfg-gmuend.de> wrote:
> 
> I have added a menu in napp-it to select, download and preconfigure LX images from Joyent 
> beside the menu to configure/ attach/ start/ stop/ detach/ delete
> 
> http://www.napp-it.org/doc/downloads/zones.pdf
> 
> Whats not yet clear to me is the way to configure lx zones with zoneadm/zonecfg according to the settings in the Joyent XML file like network ip/dhcp, router, netmask, gateway, maxram, ssh, mount zfs filesystems to name a few.

We didn't port all of SmartOS, we ported the LX zones.

For now, there are a few specific zonecfg(1M) parameters for kernel-version and network setting.  We hope to have /native/sbin/ipadm persistence before r151020 ships.  Otherwise, you treat an LX zone like you would any other, using zonecfg(1M) where appropriate.

> btw
> what about kvm branded zones from Joyent?
> the concept is very new to me....

A KVM zone instantiates a zone, but instead of running a child init(1M) process, it runs KVM directly.  It wraps a KVM process in it's own zone, whose sole purpose is to service KVM.  We don't have these in OmniOS.  Like I said, we weren't porting all of SmartOS.

> you may discuss at
> https://forums.servethehome.com/index.php?threads/omnios-now-includes-lx-support-from-joyent-smartos.11577/

I'm trying to get r151020 out the door, so I can't for the time being.

Dan


From johan.kragsterman at capvert.se  Sat Oct 22 11:51:47 2016
From: johan.kragsterman at capvert.se (Johan Kragsterman)
Date: Sat, 22 Oct 2016 13:51:47 +0200
Subject: [OmniOS-discuss] pass H/W device to LX zone
Message-ID: <OF672D7EA9.0BBCEC6A-ONC1258054.004116F7-C1258054.00412AA6@inse.com>


Hi!


I wonder if it is, or will be, possible to pass a hardware device(pci, usb, whatever) that has no omnios driver support, to an LX zone, where it has driver support?


Best regards from/Med v?nliga h?lsningar fr?n

Johan Kragsterman

Capvert



From jimklimov at cos.ru  Sat Oct 22 14:31:21 2016
From: jimklimov at cos.ru (Jim Klimov)
Date: Sat, 22 Oct 2016 16:31:21 +0200
Subject: [OmniOS-discuss] pass H/W device to LX zone
In-Reply-To: <OF672D7EA9.0BBCEC6A-ONC1258054.004116F7-C1258054.00412AA6@inse.com>
References: <OF672D7EA9.0BBCEC6A-ONC1258054.004116F7-C1258054.00412AA6@inse.com>
Message-ID: <8F13D020-7DEA-40E4-9BB3-CDA5A4DF5E3B@cos.ru>

22 ??????? 2016??. 13:51:47 CEST, Johan Kragsterman <johan.kragsterman at capvert.se> ?????:
>
>Hi!
>
>
>I wonder if it is, or will be, possible to pass a hardware device(pci,
>usb, whatever) that has no omnios driver support, to an LX zone, where
>it has driver support?
>
>
>Best regards from/Med v?nliga h?lsningar fr?n
>
>Johan Kragsterman
>
>Capvert
>
>
>_______________________________________________
>OmniOS-discuss mailing list
>OmniOS-discuss at lists.omniti.com
>http://lists.omniti.com/mailman/listinfo/omnios-discuss

Your question actually provokes a larger one: will it ever be possible to run a linux HW driver in an LX zone?

@Johan: Initially I wanted to bluntly say "of course not" because an LX zone has the illumos kernel and a layer to emulate some syscalls and translate them from linux to solaris - which is ok for some userland software but likely not for linux kernel modules (which is what drivers are, mostly). But then again, what if some drivers use this set of syscalls (or some others not difficult to provide mappings for) and might just run?

Jim
--
Typos courtesy of K-9 Mail on my Samsung Android

From danmcd at omniti.com  Sat Oct 22 18:07:06 2016
From: danmcd at omniti.com (Dan McDonald)
Date: Sat, 22 Oct 2016 14:07:06 -0400
Subject: [OmniOS-discuss] pass H/W device to LX zone
In-Reply-To: <8F13D020-7DEA-40E4-9BB3-CDA5A4DF5E3B@cos.ru>
References: <OF672D7EA9.0BBCEC6A-ONC1258054.004116F7-C1258054.00412AA6@inse.com>
	<8F13D020-7DEA-40E4-9BB3-CDA5A4DF5E3B@cos.ru>
Message-ID: <A55736DD-2E80-433F-9177-2BE8CCC2771B@omniti.com>


> On Oct 22, 2016, at 10:31 AM, Jim Klimov <jimklimov at cos.ru> wrote:
> 
> 
> @Johan: Initially I wanted to bluntly say "of course not" because an LX zone has the illumos kernel and a layer to emulate some syscalls and translate them from linux to solaris - which is ok for some userland software but likely not for linux kernel modules (which is what drivers are, mostly). But then again, what if some drivers use this set of syscalls (or some others not difficult to provide mappings for) and might just run?

Your blunt answer is generally right.  LX zones have the illumos kernel underneath.  Unless there's some amazing work going on in the in-kernel-emulation I'm not aware of, you won't be able to run Linux device drivers ever in an LX Zone.

There's another part to this tale as well, but I will not commit it to email, especially an archived list.  Let's just say it involves GPL and leave it at that.  :-P

Dan


From bfriesen at simple.dallas.tx.us  Sat Oct 22 19:33:28 2016
From: bfriesen at simple.dallas.tx.us (Bob Friesenhahn)
Date: Sat, 22 Oct 2016 14:33:28 -0500 (CDT)
Subject: [OmniOS-discuss] pass H/W device to LX zone
In-Reply-To: <8F13D020-7DEA-40E4-9BB3-CDA5A4DF5E3B@cos.ru>
References: <OF672D7EA9.0BBCEC6A-ONC1258054.004116F7-C1258054.00412AA6@inse.com>
	<8F13D020-7DEA-40E4-9BB3-CDA5A4DF5E3B@cos.ru>
Message-ID: <alpine.GSO.2.20.1610221426500.17782@freddy.simplesystems.org>

On Sat, 22 Oct 2016, Jim Klimov wrote:

> 22 ??????? 2016??. 13:51:47 CEST, Johan Kragsterman <johan.kragsterman at capvert.se> ?????:
>>
>> I wonder if it is, or will be, possible to pass a hardware device(pci,
>> usb, whatever) that has no omnios driver support, to an LX zone, where
>> it has driver support?

In my experience the Linux kernel interface to its drivers is quite 
unstable and driver code requires tweaks to work with the updated 
kernel.  There is a tight coupling between drivers and the kernel so 
that they have to be compiled with the kernel, or re-compiled based on 
specific kernel headers.  The driver interfaces are not mostly stable 
like they are with Solaris/Illumos.

The reason why Linux works with so many devices is due to the many 
interested parties who work to keep it working.

If someone was to compile the driver from source code they could use 
it with the Illumos kernel on their own system without violating GPL. 
Any small error might result in the OS needing to be re-installed 
and/or total loss of data.

Bob
-- 
Bob Friesenhahn
bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/

From info.fmo at gmx.de  Sun Oct 23 01:06:28 2016
From: info.fmo at gmx.de (Frank M.)
Date: Sun, 23 Oct 2016 03:06:28 +0200
Subject: [OmniOS-discuss] Fwd: NVMe in ESXi-Passthrough crashes
In-Reply-To: <1344112614.20161023025416@gmx.de>
References: <1344112614.20161023025416@gmx.de>
Message-ID: <1878224113.20161023030628@gmx.de>

Perhaps my post is here better, because I use OmniOS of course...
...and the next stable OmniOS is in sight...

This is a forwarded message
From: Frank M. <info.fmo at gmx.de>
To: Illumos Discussion <discuss at lists.illumos.org>
Date: Sunday, October 23, 2016, 2:54:16 AM
Subject: NVMe in ESXi-Passthrough crashes

===8<==============Original message text===============
Hi,

I?ve installed the new bloody as a new test virtual machine. The vm
crashes whenever there is a access to the nvme device.
Tests with a "special nvme driver" on the last stable had no problems.
I tested a Samsung SM951 on ESXi 6.0 U2 on a Supermicro X10DRi-T4+.
Tomorrow I will test shortly a Samsung SM961 on another Server (HP).
I hope, you will find a solution until the next stable release...
  
greets from Germany
Frank

===8<===========End of original message text===========



-- 
Best regards,
 Frank                            mailto:info.fmo at gmx.de


From danmcd at omniti.com  Sun Oct 23 01:11:09 2016
From: danmcd at omniti.com (Dan McDonald)
Date: Sat, 22 Oct 2016 21:11:09 -0400
Subject: [OmniOS-discuss] Fwd: NVMe in ESXi-Passthrough crashes
In-Reply-To: <1878224113.20161023030628@gmx.de>
References: <1344112614.20161023025416@gmx.de>
	<1878224113.20161023030628@gmx.de>
Message-ID: <B4BED540-E685-47B7-859F-D974A93AB215@omniti.com>


> On Oct 22, 2016, at 9:06 PM, Frank M. <info.fmo at gmx.de> wrote:
> 
> 
> I?ve installed the new bloody as a new test virtual machine. The vm
> crashes whenever there is a access to the nvme device.
> Tests with a "special nvme driver" on the last stable had no problems.
> I tested a Samsung SM951 on ESXi 6.0 U2 on a Supermicro X10DRi-T4+.
> Tomorrow I will test shortly a Samsung SM961 on another Server (HP).
> I hope, you will find a solution until the next stable release...

Oddly enough:

1.) r151020 (next stable) is coming soon (November).

2.) Due to some interesting circumstances, the NVMe fixes in bloody & 020 may be backported to LTS (014) and last-stable (018).

Dan


From info.fmo at gmx.de  Sun Oct 23 14:09:05 2016
From: info.fmo at gmx.de (Frank M.)
Date: Sun, 23 Oct 2016 16:09:05 +0200
Subject: [OmniOS-discuss] Fwd: NVMe in ESXi-Passthrough crashes
In-Reply-To: <B4BED540-E685-47B7-859F-D974A93AB215@omniti.com>
References: <1344112614.20161023025416@gmx.de>
	<1878224113.20161023030628@gmx.de>
	<B4BED540-E685-47B7-859F-D974A93AB215@omniti.com>
Message-ID: <1987622123.20161023160905@gmx.de>

Hi,

I?ve tested now on another machine (HP DL380 G6).
The new bloody OmniOS crashes here too...

Frank

Sunday, October 23, 2016, 3:11:09 AM, you wrote:


>> On Oct 22, 2016, at 9:06 PM, Frank M. <info.fmo at gmx.de> wrote:
>> 
>> 
>> I?ve installed the new bloody as a new test virtual machine. The vm
>> crashes whenever there is a access to the nvme device.
>> Tests with a "special nvme driver" on the last stable had no problems.
>> I tested a Samsung SM951 on ESXi 6.0 U2 on a Supermicro X10DRi-T4+.
>> Tomorrow I will test shortly a Samsung SM961 on another Server (HP).
>> I hope, you will find a solution until the next stable release...

DM> Oddly enough:

DM> 1.) r151020 (next stable) is coming soon (November).

DM> 2.) Due to some interesting circumstances, the NVMe fixes in
DM> bloody & 020 may be backported to LTS (014) and last-stable (018).

DM> Dan




From an3s.annema at gmail.com  Mon Oct 24 19:04:07 2016
From: an3s.annema at gmail.com (Andries Annema)
Date: Mon, 24 Oct 2016 21:04:07 +0200
Subject: [OmniOS-discuss] Tun/Tap device installed but not showing up on
	OmniOS
Message-ID: <bb5ed86b-b803-9340-b68b-85995edbfb48@gmail.com>

Hello there, guys,

THE PROBLEM IN SHORT or TL;DR:
I have installed the tun/tap device within the global zone of a r151014 
machine, but the /dev/tun and /dev/tap devices don't show up. What am I 
missing here?


THE PROBLEM IN MORE DETAIL:
I'm trying to get OpenVPN up and running as a server from within a 
non-global zone. For this the TUN/TAP device driver is needed within the 
global zone and passed on to the non-global zone. Seems straight 
forward, and I actually did succeed on one system, but my main 
production server is playing tricks on me.

As I'm not running just one OmniOS server anymore, it seemed obvious 
that I needed my own package repository and not build software from 
source on each production server.
So, this is what I did - at first, all of it on a virtual OmniOS r151014 
machine:
1. Set up a local IPS package repository within another NGZ,
2. set up a build environment within even another NGZ,
3. within the build environment, I downloaded the TUN/TAP source code 
from https://github.com/kaizawa/tuntap
4. configure --prefix=/proto/install/path/, make, make install
5. Created a manifest, specify package contents, etc etc, "pkglint" to 
verify before publishing,
6. and published the package to my local repo.
7. Then from the global zone on this VM, with "pkg set-publisher" I 
added this repo,
8. and installed my brand new tuntap-package with "pkg install 
network/tuntap".
9. to verify "ls -l /dev/tun /dev/tap" and indeed the devices showed up. 
And not just these links, but also the actual devices listed under 
"/devices/pseudo/clone at 0:[tun|tap]". Awesome! My very first built and 
published package!

Now, to get more serious, I repeated all this on my actual production 
server. But there it goes funny at step 9: the devices don't show up. 
The files that are included within the IPS package however, are indeed 
installed. No error/warning during install whatsoever. Reinstalled the 
tuntap a few times. No effect. Rebooted the system and reinstalled. 
Still no effect.
The system hadn't been updated in over a year (431 days uptime...), so I 
figured maybe I'm missing a package or something. I uninstalled the 
tuntap-device, updated the system, tried again: still no luck.
Fiddling around with "modinfo" and "modload" didn't help either: 
"modinfo" doesn't show up a tun or tap device and "modload" is unable to 
load it.

Then I tried step 7-9 on a second and quite similarly build OmniOS 
r151014 system, a system that is much younger and had all its packages 
up to date to begin with (not that I'm sure that the cause of the 
problem lies in there, but it usually is a good/better starting point) 
and guess what? Two beautiful new devices are listed at /dev/tun and 
/dev/tap.
Here, also "modinfo" shows nicely a tun and a tap device.


So my questions are:
A. Where do I have to look for feedback/logs on the installation 
procedure to get more insight, if any, on where it failed? (The whole 
logging on Unix/Linux and alikes is one of many subjects that I have to 
master sometime).
B. How come it does install nicely on one system, but does not on 
another (seemingly identical or at least very similar) system?
C. The most important question: how can I get these devices actually 
showing up?

Any help would be greatly appreciated! Thanks.
We only learn by making mistakes and screwing things up, right! ;)

Regards,
Andries


From mailinglists at qutic.com  Tue Oct 25 10:59:50 2016
From: mailinglists at qutic.com (qutic development)
Date: Tue, 25 Oct 2016 12:59:50 +0200
Subject: [OmniOS-discuss] Tun/Tap device installed but not showing up on
	OmniOS
In-Reply-To: <bb5ed86b-b803-9340-b68b-85995edbfb48@gmail.com>
References: <bb5ed86b-b803-9340-b68b-85995edbfb48@gmail.com>
Message-ID: <86177927-04CE-43E4-89ED-A25458641F65@qutic.com>

Hi Andries,

> Am 24.10.2016 um 21:04 schrieb Andries Annema <an3s.annema at gmail.com>:
> 
> I have installed the tun/tap device within the global zone of a r151014 machine, but the /dev/tun and /dev/tap devices don't show up. What am I missing here?

I use the following build script to get a working module for r151014:

https://github.com/jfqd/omnios-build/tree/151006/build/tuntap

- Stefan

From tom-omnios-discuss at tom.bn-ulm.de  Tue Oct 25 12:41:25 2016
From: tom-omnios-discuss at tom.bn-ulm.de (Thomas Wagner)
Date: Tue, 25 Oct 2016 14:41:25 +0200
Subject: [OmniOS-discuss] Tun/Tap device installed but not showing up	on
	OmniOS
In-Reply-To: <bb5ed86b-b803-9340-b68b-85995edbfb48@gmail.com>
References: <bb5ed86b-b803-9340-b68b-85995edbfb48@gmail.com>
Message-ID: <20161025124125.GC15500@wagner-net.com>

Hi Andries,

does your IPS manifest have an "action" to add the driver entries "tun" and "tap"?

If you scroll down here to the %action section: https://sourceforge.net/p/pkgbuild/code/HEAD/tree/spec-files-extra/trunk/SFEtun.spec
This results if built with "pkgtool"/"pkbbuild" to an entry in the manifest file like this:
driver name=tap
driver name=tun

A ready-made package is available here: http://sfe.opencsw.org/quickrepolinks
(fully independent of OpenCSW project, but kindly hosted by them)

Btw, it would be cool if you could share your build recipes and / or 
participate in one of the public repositories for build recipes.

Regards,
Thomas


From dasaint.blair at gmail.com  Tue Oct 25 19:11:13 2016
From: dasaint.blair at gmail.com (Matt Mabis)
Date: Tue, 25 Oct 2016 13:11:13 -0600
Subject: [OmniOS-discuss] OmniOS (Joined Active Directory & KRB5.Conf + NFS
 = NFS Disconnects in ESXI on DC Reboot)
Message-ID: <CAJpROcQEh82AfToJjEKtCMCKVg_qb25NeQ3nkZVE+jKVStfMKA@mail.gmail.com>

Hey All,

Was wondering if you could help me out, using b13298F but i have seen this
in other versions too where i joined the Omni to the Domain so i could use
ACLs and UserID Mappings for SMB.  however when i reboot one of the 2 DCs i
have it looks like it loses all access to the NFS Shares till that DC Comes
back online.

>From my troubleshooting i have figured out it has something to do with
Kerberos and NFS together if the System cannot talk to Kerberos NFS Drops
the connections (Even though i am doing Root @ IP Range) based ESXi
Mappings for NFS v3  I believe Kerberos and NFS are intertwined in some
sense cause even though i can ping the IP its like i just get inherently
denied access when that DC Is down...

Trying to make sure if i have to take these DC's down for
updates/patches/maintenance they arent going to take down my entire NFS
structure...

Any Help would be appreciated.


KRB5.CONF File
See Data *Changed domain name to DOMAIN.LOCAL* but all other information is
correct 192.168.30.14 is DC1 and 192.168.30.15 is DC2.. (When i power down
DC1 is when i start loosing connection)


#
# CDDL HEADER START
#
# The contents of this file are subject to the terms of the
# Common Development and Distribution License (the "License").
# You may not use this file except in compliance with the License.
#
# You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE
# or http://www.opensolaris.org/os/licensing.
# See the License for the specific language governing permissions
# and limitations under the License.
#
# When distributing Covered Code, include this CDDL HEADER in each
# file and include the License file at usr/src/OPENSOLARIS.LICENSE.
# If applicable, add the following below this CDDL HEADER, with the
# fields enclosed by brackets "[]" replaced with your own identifying
# information: Portions Copyright [yyyy] [name of copyright owner]
#
# CDDL HEADER END
#
#
# Copyright 2007 Sun Microsystems, Inc.  All rights reserved.
# Use is subject to license terms.
#
# ident  ?  "%Z%%M%  ?  %I%  ?  %E% SMI"
#

# krb5.conf template
# In order to complete this configuration file
# you will need to replace the __<name>__ placeholders
# with appropriate values for your network and uncomment the
# appropriate entries.
#
[libdefaults]
        default_realm = DOMAIN.LOCAL


[realms]
        DOMAIN.LOCAL = {
                kdc = 192.168.30.14
                kdc = 192.168.30.15
                admin_server = 192.168.30.15
                kpasswd_server = 192.168.30.15
                kpasswd_protocol = SET_CHANGE
        }

[domain_realm]
        .domain.local = DOMAIN.LOCAL

[logging]
        default = FILE:/var/krb5/kdc.log
        kdc = FILE:/var/krb5/kdc.log
  ?  kdc_rotate = {

# How often to rotate kdc.log. Logs will get rotated no more
# often than the period, and less often if the KDC is not used
# frequently.

  ?    ?  period = 1d

# how many versions of kdc.log to keep around (kdc.log.0, kdc.log.1, ...)

  ?    ?  versions = 10
  ?  }

[appdefaults]
  ?  kinit = {
  ?    ?  renewable = true
  ?    ?  forwardable= true
  ?  }
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20161025/d4aacec8/attachment.html>

From asc1111 at gmail.com  Wed Oct 26 22:08:24 2016
From: asc1111 at gmail.com (Aaron Curry)
Date: Wed, 26 Oct 2016 16:08:24 -0600
Subject: [OmniOS-discuss] OmniOS (Joined Active Directory & KRB5.Conf +
 NFS = NFS Disconnects in ESXI on DC Reboot)
In-Reply-To: <CAJpROcQEh82AfToJjEKtCMCKVg_qb25NeQ3nkZVE+jKVStfMKA@mail.gmail.com>
References: <CAJpROcQEh82AfToJjEKtCMCKVg_qb25NeQ3nkZVE+jKVStfMKA@mail.gmail.com>
Message-ID: <CAOqBcP9H=Kn0JWa7BA1Y7xhqTA_xiL8jnGT1JfkkHipA19FBMQ@mail.gmail.com>

I had a similar problem. Turned out to be a DNS issue. Our DNS was running
on the DC. When the DC was down, NFS wouldn't work. All I had to do was add
our router as a nameserver in resolv.conf and it started working again. It
seemed to be that as long as OmniOS has someone responding to DNS requests
NFS will work, even if the DNS has no local info.

Hope that helps.

Aaron
On Oct 25, 2016 1:13 PM, "Matt Mabis" <dasaint.blair at gmail.com> wrote:

> Hey All,
>
> Was wondering if you could help me out, using b13298F but i have seen this
> in other versions too where i joined the Omni to the Domain so i could use
> ACLs and UserID Mappings for SMB.  however when i reboot one of the 2 DCs i
> have it looks like it loses all access to the NFS Shares till that DC Comes
> back online.
>
> From my troubleshooting i have figured out it has something to do with
> Kerberos and NFS together if the System cannot talk to Kerberos NFS Drops
> the connections (Even though i am doing Root @ IP Range) based ESXi
> Mappings for NFS v3  I believe Kerberos and NFS are intertwined in some
> sense cause even though i can ping the IP its like i just get inherently
> denied access when that DC Is down...
>
> Trying to make sure if i have to take these DC's down for
> updates/patches/maintenance they arent going to take down my entire NFS
> structure...
>
> Any Help would be appreciated.
>
>
> KRB5.CONF File
> See Data *Changed domain name to DOMAIN.LOCAL* but all other information
> is correct 192.168.30.14 is DC1 and 192.168.30.15 is DC2.. (When i power
> down DC1 is when i start loosing connection)
>
>
> #
> # CDDL HEADER START
> #
> # The contents of this file are subject to the terms of the
> # Common Development and Distribution License (the "License").
> # You may not use this file except in compliance with the License.
> #
> # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE
> # or http://www.opensolaris.org/os/licensing.
> # See the License for the specific language governing permissions
> # and limitations under the License.
> #
> # When distributing Covered Code, include this CDDL HEADER in each
> # file and include the License file at usr/src/OPENSOLARIS.LICENSE.
> # If applicable, add the following below this CDDL HEADER, with the
> # fields enclosed by brackets "[]" replaced with your own identifying
> # information: Portions Copyright [yyyy] [name of copyright owner]
> #
> # CDDL HEADER END
> #
> #
> # Copyright 2007 Sun Microsystems, Inc.  All rights reserved.
> # Use is subject to license terms.
> #
> # ident  ?  "%Z%%M%  ?  %I%  ?  %E% SMI"
> #
>
> # krb5.conf template
> # In order to complete this configuration file
> # you will need to replace the __<name>__ placeholders
> # with appropriate values for your network and uncomment the
> # appropriate entries.
> #
> [libdefaults]
>         default_realm = DOMAIN.LOCAL
>
>
> [realms]
>         DOMAIN.LOCAL = {
>                 kdc = 192.168.30.14
>                 kdc = 192.168.30.15
>                 admin_server = 192.168.30.15
>                 kpasswd_server = 192.168.30.15
>                 kpasswd_protocol = SET_CHANGE
>         }
>
> [domain_realm]
>         .domain.local = DOMAIN.LOCAL
>
> [logging]
>         default = FILE:/var/krb5/kdc.log
>         kdc = FILE:/var/krb5/kdc.log
>   ?  kdc_rotate = {
>
> # How often to rotate kdc.log. Logs will get rotated no more
> # often than the period, and less often if the KDC is not used
> # frequently.
>
>   ?    ?  period = 1d
>
> # how many versions of kdc.log to keep around (kdc.log.0, kdc.log.1, ...)
>
>   ?    ?  versions = 10
>   ?  }
>
> [appdefaults]
>   ?  kinit = {
>   ?    ?  renewable = true
>   ?    ?  forwardable= true
>   ?  }
>
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20161026/40cdf41b/attachment-0001.html>

From dasaint.blair at gmail.com  Wed Oct 26 22:13:36 2016
From: dasaint.blair at gmail.com (Matt Mabis)
Date: Wed, 26 Oct 2016 16:13:36 -0600
Subject: [OmniOS-discuss] OmniOS (Joined Active Directory & KRB5.Conf +
 NFS = NFS Disconnects in ESXI on DC Reboot)
In-Reply-To: <CAOqBcP9H=Kn0JWa7BA1Y7xhqTA_xiL8jnGT1JfkkHipA19FBMQ@mail.gmail.com>
References: <CAJpROcQEh82AfToJjEKtCMCKVg_qb25NeQ3nkZVE+jKVStfMKA@mail.gmail.com>
	<CAOqBcP9H=Kn0JWa7BA1Y7xhqTA_xiL8jnGT1JfkkHipA19FBMQ@mail.gmail.com>
Message-ID: <CAJpROcTfAZ_hbDrxEapXUuugnhf5uZW20mfYYhP54GxpjnGuAQ@mail.gmail.com>

It does, i got another email from another member directly that explained
the issue as well, i however did have my router in there and another active
DNS controller and yet it still happened...  The other guy recommended to
get around this eaiser to put the esxi host names in the /etc/hosts file to
ensure that dns was ok....

Funny part is i didnt think it would be DNS as my hosts were using IPs to
connect not DNS names..   My Resolv.conf had the following info and even
tho .15 and .1 were both online as soon as i lost .14 it lost
connectivity... so i think theres still something wonky with the NFS
protocol as it shouldnt have lost connection consistently... it would go up
and down like it was always trying to hit 14 first and lost connection then
found 15 then reconnected... then rechecked at 14 then lost it....

192.168.30.14 - DC1
192.168.30.15 - DC2
192.168.30.1 - Router

On Wed, Oct 26, 2016 at 4:08 PM, Aaron Curry <asc1111 at gmail.com> wrote:

> I had a similar problem. Turned out to be a DNS issue. Our DNS was running
> on the DC. When the DC was down, NFS wouldn't work. All I had to do was add
> our router as a nameserver in resolv.conf and it started working again. It
> seemed to be that as long as OmniOS has someone responding to DNS requests
> NFS will work, even if the DNS has no local info.
>
> Hope that helps.
>
> Aaron
> On Oct 25, 2016 1:13 PM, "Matt Mabis" <dasaint.blair at gmail.com> wrote:
>
>> Hey All,
>>
>> Was wondering if you could help me out, using b13298F but i have seen
>> this in other versions too where i joined the Omni to the Domain so i could
>> use ACLs and UserID Mappings for SMB.  however when i reboot one of the 2
>> DCs i have it looks like it loses all access to the NFS Shares till that DC
>> Comes back online.
>>
>> From my troubleshooting i have figured out it has something to do with
>> Kerberos and NFS together if the System cannot talk to Kerberos NFS Drops
>> the connections (Even though i am doing Root @ IP Range) based ESXi
>> Mappings for NFS v3  I believe Kerberos and NFS are intertwined in some
>> sense cause even though i can ping the IP its like i just get inherently
>> denied access when that DC Is down...
>>
>> Trying to make sure if i have to take these DC's down for
>> updates/patches/maintenance they arent going to take down my entire NFS
>> structure...
>>
>> Any Help would be appreciated.
>>
>>
>> KRB5.CONF File
>> See Data *Changed domain name to DOMAIN.LOCAL* but all other information
>> is correct 192.168.30.14 is DC1 and 192.168.30.15 is DC2.. (When i power
>> down DC1 is when i start loosing connection)
>>
>>
>> #
>> # CDDL HEADER START
>> #
>> # The contents of this file are subject to the terms of the
>> # Common Development and Distribution License (the "License").
>> # You may not use this file except in compliance with the License.
>> #
>> # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE
>> # or http://www.opensolaris.org/os/licensing.
>> # See the License for the specific language governing permissions
>> # and limitations under the License.
>> #
>> # When distributing Covered Code, include this CDDL HEADER in each
>> # file and include the License file at usr/src/OPENSOLARIS.LICENSE.
>> # If applicable, add the following below this CDDL HEADER, with the
>> # fields enclosed by brackets "[]" replaced with your own identifying
>> # information: Portions Copyright [yyyy] [name of copyright owner]
>> #
>> # CDDL HEADER END
>> #
>> #
>> # Copyright 2007 Sun Microsystems, Inc.  All rights reserved.
>> # Use is subject to license terms.
>> #
>> # ident  ?  "%Z%%M%  ?  %I%  ?  %E% SMI"
>> #
>>
>> # krb5.conf template
>> # In order to complete this configuration file
>> # you will need to replace the __<name>__ placeholders
>> # with appropriate values for your network and uncomment the
>> # appropriate entries.
>> #
>> [libdefaults]
>>         default_realm = DOMAIN.LOCAL
>>
>>
>> [realms]
>>         DOMAIN.LOCAL = {
>>                 kdc = 192.168.30.14
>>                 kdc = 192.168.30.15
>>                 admin_server = 192.168.30.15
>>                 kpasswd_server = 192.168.30.15
>>                 kpasswd_protocol = SET_CHANGE
>>         }
>>
>> [domain_realm]
>>         .domain.local = DOMAIN.LOCAL
>>
>> [logging]
>>         default = FILE:/var/krb5/kdc.log
>>         kdc = FILE:/var/krb5/kdc.log
>>   ?  kdc_rotate = {
>>
>> # How often to rotate kdc.log. Logs will get rotated no more
>> # often than the period, and less often if the KDC is not used
>> # frequently.
>>
>>   ?    ?  period = 1d
>>
>> # how many versions of kdc.log to keep around (kdc.log.0, kdc.log.1, ...)
>>
>>   ?    ?  versions = 10
>>   ?  }
>>
>> [appdefaults]
>>   ?  kinit = {
>>   ?    ?  renewable = true
>>   ?    ?  forwardable= true
>>   ?  }
>>
>> _______________________________________________
>> OmniOS-discuss mailing list
>> OmniOS-discuss at lists.omniti.com
>> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20161026/5e44be1d/attachment.html>

From apenner.it at gmail.com  Thu Oct 27 21:38:53 2016
From: apenner.it at gmail.com (Artem Penner)
Date: Thu, 27 Oct 2016 21:38:53 +0000
Subject: [OmniOS-discuss]  Only grub after boot
Message-ID: <CAE9csxO_TijF-bMWRPze9=3On=dpzF+sarAqy=mXtR_fYjZ5bQ@mail.gmail.com>

Hi, i have the same problem.
When I try to install OmniOS on HGST HUC101818CS4204
<https://www.hgst.com/ru/products/hard-drives/ultrastar-c10k1800> which
connected through LSI SAS 9300-16e HBA, boot hangs on "GRUB _" screen.
In same JBOD I successfully install OS on HUSMH8020BSS204 SSD.

I also test Solaris 11.3, but same problem appeared.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20161027/1f14ca96/attachment.html>

From danmcd at omniti.com  Thu Oct 27 22:09:57 2016
From: danmcd at omniti.com (Dan McDonald)
Date: Thu, 27 Oct 2016 18:09:57 -0400
Subject: [OmniOS-discuss] Full-kernel updates for r151014 and r151018
Message-ID: <5C8B2AB9-D041-4266-898C-D6A24A9C97B3@omniti.com>

LTS and Stable users, please run "pkg update" for a full illumos update.  It will create a new BE, and require a reboot.

Both are receiving backports, and some critical kernel bugfixes.  Backports include support for NVME1.1 and the Kaby Lake I219 Gigabit ethernet driver.

The new uname -v for r151014 is omnios-cac2b76, and for r151018 is omnios-r151018-258cc99.

Thanks,
Dan


From henson at acm.org  Thu Oct 27 22:23:13 2016
From: henson at acm.org (Paul B. Henson)
Date: Thu, 27 Oct 2016 15:23:13 -0700
Subject: [OmniOS-discuss] Full-kernel updates for r151014 and r151018
In-Reply-To: <5C8B2AB9-D041-4266-898C-D6A24A9C97B3@omniti.com>
References: <5C8B2AB9-D041-4266-898C-D6A24A9C97B3@omniti.com>
Message-ID: <1fb801d230a0$b5732ad0$20598070$@acm.org>

> From: Dan McDonald
> Sent: Thursday, October 27, 2016 3:10 PM
> 
> Both are receiving backports, and some critical kernel bugfixes.
Backports
> include support for NVME1.1 and the Kaby Lake I219 Gigabit ethernet
driver.

Could I trouble you to elaborate on "critical kernel bug fixes"? Are we
talking "drop everything else and get these done tonight" fixes, or
"schedule for the next standard maintenance window" fixes?



From danmcd at omniti.com  Thu Oct 27 23:05:21 2016
From: danmcd at omniti.com (Dan McDonald)
Date: Thu, 27 Oct 2016 19:05:21 -0400
Subject: [OmniOS-discuss] Full-kernel updates for r151014 and r151018
In-Reply-To: <1fb801d230a0$b5732ad0$20598070$@acm.org>
References: <5C8B2AB9-D041-4266-898C-D6A24A9C97B3@omniti.com>
	<1fb801d230a0$b5732ad0$20598070$@acm.org>
Message-ID: <57D3304A-105A-4976-91AE-E2B9BFF3075D@omniti.com>

The exploits are nontrivial, but these do have CVE numbers assigned to them (to be announced).

Dan

Sent from my iPhone (typos, autocorrect, and all)

On Oct 27, 2016, at 6:23 PM, Paul B. Henson <henson at acm.org> wrote:

>> From: Dan McDonald
>> Sent: Thursday, October 27, 2016 3:10 PM
>> 
>> Both are receiving backports, and some critical kernel bugfixes.
> Backports
>> include support for NVME1.1 and the Kaby Lake I219 Gigabit ethernet
> driver.
> 
> Could I trouble you to elaborate on "critical kernel bug fixes"? Are we
> talking "drop everything else and get these done tonight" fixes, or
> "schedule for the next standard maintenance window" fixes?
> 
> 

From apenner.it at gmail.com  Fri Oct 28 07:40:40 2016
From: apenner.it at gmail.com (Artem Penner)
Date: Fri, 28 Oct 2016 07:40:40 +0000
Subject: [OmniOS-discuss] Only grub after boot
In-Reply-To: <CAE9csxO_TijF-bMWRPze9=3On=dpzF+sarAqy=mXtR_fYjZ5bQ@mail.gmail.com>
References: <CAE9csxO_TijF-bMWRPze9=3On=dpzF+sarAqy=mXtR_fYjZ5bQ@mail.gmail.com>
Message-ID: <CAE9csxM3C0pG_+eXYEqbpaijVqg5UOJacwFDgaDkXXaLibthnA@mail.gmail.com>

I try to install grub manually on disk, but have the same result.
After that i try to install grub on usb flash, now stage2 is loading, but
can't find rpool. When i try to list disks in grub shell, grub also hangs.

my hardware:
Supermicro 2U CSE-825TQ-R740LPB
motherboard: SuperMicro 2xCPU X10DRI-LN4+ E5-2600v3/v4 UpTo2x12DIMM/
10xSATA3/ C612 RAID 0/1/5/10/ 4xGE/ 2xPCIx16, 3xPCIx8, 1xPCIx4 (in
x8)(13.68" x 13")
SAS controller: LSI SAS9300-16E SGL SAS 12G
disks: Hitachi HUSMH8020BSS204 - this is ok, bootable, no problem
          Hitachi SAS 3.0 1800Gb HUC101818CS4204 Ultrastar C10K1800  - this
is disk, from which i can't boot.

??, 28 ???. 2016 ?. ? 0:38, Artem Penner <apenner.it at gmail.com>:

> Hi, i have the same problem.
> When I try to install OmniOS on HGST HUC101818CS4204
> <https://www.hgst.com/ru/products/hard-drives/ultrastar-c10k1800> which
> connected through LSI SAS 9300-16e HBA, boot hangs on "GRUB _" screen.
> In same JBOD I successfully install OS on HUSMH8020BSS204 SSD.
>
> I also test Solaris 11.3, but same problem appeared.
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20161028/0e2425df/attachment-0001.html>

From an3s.annema at gmail.com  Mon Oct 31 19:32:12 2016
From: an3s.annema at gmail.com (Andries Annema)
Date: Mon, 31 Oct 2016 20:32:12 +0100
Subject: [OmniOS-discuss] FW: Tun/Tap device installed but not showing up on
	OmniOS
In-Reply-To: <003601d233ab$fed56db0$fc804910$@gmail.com>
References: <bb5ed86b-b803-9340-b68b-85995edbfb48@gmail.com>
	<86177927-04CE-43E4-89ED-A25458641F65@qutic.com>
	<003601d233ab$fed56db0$fc804910$@gmail.com>
Message-ID: <003801d233ad$7b091fc0$711b5f40$@gmail.com>

(Forgot the include the discuss list. Sorry)

-----Original Message-----
From: Andries Annema [mailto:an3s.annema at gmail.com] 
Sent: maandag 31 oktober 2016 20:22
To: 'qutic development'
Subject: RE: [OmniOS-discuss] Tun/Tap device installed but not showing up on
OmniOS

Thanks Stefan, for your suggestion.
However, when I tried to build stuff using the OmniOS build scripts that are
available on github some weeks ago, I was unable to master the process; it
kept throwing errors at me and I was unable to find some decent,
to-the-point documentation to get on top of it.
That may be due to beginners error, but for me the learning curve was too
steep at the moment and I figured I had to go back to the "classic" Solaris
build procedure as described here, for example:
http://www.oracle.com/technetwork/articles/servers-storage-admin/o11-097-cre
ate-pkg-ips-524496.html
With that, I managed to build and publish a number of packages, so for the
time being I think I'll stick with that - unless someone has a _really_ good
starting guide for the github-way.

In the meantime, the suggestions as given by Thomas got me to solve my
tun/tap-issue - I'll reply to that separately.

Regards,
Andries


-----Original Message-----
From: OmniOS-discuss [mailto:omnios-discuss-bounces at lists.omniti.com] On
Behalf Of qutic development
Sent: dinsdag 25 oktober 2016 13:00
To: omnios-discuss
Subject: Re: [OmniOS-discuss] Tun/Tap device installed but not showing up on
OmniOS

Hi Andries,

> Am 24.10.2016 um 21:04 schrieb Andries Annema <an3s.annema at gmail.com>:
> 
> I have installed the tun/tap device within the global zone of a r151014
machine, but the /dev/tun and /dev/tap devices don't show up. What am I
missing here?

I use the following build script to get a working module for r151014:

https://github.com/jfqd/omnios-build/tree/151006/build/tuntap

- Stefan
_______________________________________________
OmniOS-discuss mailing list
OmniOS-discuss at lists.omniti.com
http://lists.omniti.com/mailman/listinfo/omnios-discuss



From an3s.annema at gmail.com  Mon Oct 31 19:32:23 2016
From: an3s.annema at gmail.com (Andries Annema)
Date: Mon, 31 Oct 2016 20:32:23 +0100
Subject: [OmniOS-discuss] FW: Tun/Tap device installed but not showing up on
	OmniOS
In-Reply-To: <003701d233ad$3c310510$b4930f30$@gmail.com>
References: <bb5ed86b-b803-9340-b68b-85995edbfb48@gmail.com>
	<20161025124125.GC15500@wagner-net.com>
	<003701d233ad$3c310510$b4930f30$@gmail.com>
Message-ID: <003901d233ad$813b9490$83b2bdb0$@gmail.com>

(Forgot the include the discuss list. Sorry)

-----Original Message-----
From: Andries Annema [mailto:an3s.annema at gmail.com] 
Sent: maandag 31 oktober 2016 20:30
To: 'Thomas Wagner'
Subject: RE: [OmniOS-discuss] Tun/Tap device installed but not showing up on
OmniOS

Thomas, thanks!
Your suggestion to include the lines:
driver name=tap
driver name=tun
in the manifest solved it for me! Wow.

Although the build process with "pkgtool"/"pkgbuild" and the script as shown
by your first link seem old, and even though they probably might still work,
I used the build process as documented by Solaris here:
http://www.oracle.com/technetwork/articles/servers-storage-admin/o11-097-cre
ate-pkg-ips-524496.html
Combined this with the two "driver name"-lines and bingo!

Your second link - of the OpenCSW project - got me a nice manifest to
compare to and with it polish some other settings. Nice. 

I am not sure however on how to participate in a public repository or share
my recipes any other way; I'm still learning and am not sure, security wise,
on how to allow access to my personal repository and maintain it on a public
level. Suggestions are always welcome though.

Thanks again!
Greetings,
Andries


-----Original Message-----
From: Thomas Wagner [mailto:tom-omnios-discuss at tom.bn-ulm.de] 
Sent: dinsdag 25 oktober 2016 14:41
To: Andries Annema
Cc: omnios-discuss
Subject: Re: [OmniOS-discuss] Tun/Tap device installed but not showing up on
OmniOS

Hi Andries,

does your IPS manifest have an "action" to add the driver entries "tun" and
"tap"?

If you scroll down here to the %action section:
https://sourceforge.net/p/pkgbuild/code/HEAD/tree/spec-files-extra/trunk/SFE
tun.spec
This results if built with "pkgtool"/"pkbbuild" to an entry in the manifest
file like this:
driver name=tap
driver name=tun

A ready-made package is available here:
http://sfe.opencsw.org/quickrepolinks
(fully independent of OpenCSW project, but kindly hosted by them)

Btw, it would be cool if you could share your build recipes and / or 
participate in one of the public repositories for build recipes.

Regards,
Thomas




