From danmcd at omniti.com  Thu Sep  1 22:18:57 2016
From: danmcd at omniti.com (Dan McDonald)
Date: Thu, 1 Sep 2016 18:18:57 -0400
Subject: [OmniOS-discuss] New updates for LX Zones for OmniOS - zonecfg(1M)
	networking
Message-ID: <855DE2C3-A631-45CE-BEBD-D674A77576C4@omniti.com>

Aka. "A tale of two LX zonecfgs".

I've updated the io-lx-public repo

	https://github.com/danmcd/io-lx-public/

AND the IPS server for bloody+LX:

	http://pkg.omniti.com/omnios/LX/

to include the Joyent-style zonecfg(1M) method of LX zone network configuration.  For example, a worked LX zone before would have:

	# zonecfg -z lx0 export
	create -b
	set zonepath=/zones/lx0
	set brand=lx
	set autoboot=false
	set ip-type=exclusive
	add net
	set physical=lx0
	end
	add attr
	set name=kernel-version
	set type=string
	set value=2.6.32
	end
	#

But now, you can add attributes to pre-configure the LX zone's basic networking (changes prefixed with '+'):

 	# zonecfg -z lx0 export
	create -b
	set zonepath=/zones/lx0
	set brand=lx
	set autoboot=false
	set ip-type=exclusive
	add net
	set physical=lx0
+	add property (name=gateway,value="192.168.0.1")
+	add property (name=ips,value="192.168.0.69/24")
+	add property (name=primary,value="true")
	end
+	add attr
+	set name=dns-domain
+	set type=string
+	set value=example.com
+	end
+	add attr
+	set name=resolvers
+	set type=string
+	set value=192.168.0.1
+	end
	add attr
	set name=kernel-version
	set type=string
	set value=2.6.32
	end
	#

And your LX zone, thanks to the lxinit(1M) binary will do the right thing.

Update your LX test boxes to this, and see how it works.

Thanks,
Dan


From danmcd at omniti.com  Thu Sep  1 22:25:24 2016
From: danmcd at omniti.com (Dan McDonald)
Date: Thu, 1 Sep 2016 18:25:24 -0400
Subject: [OmniOS-discuss] New updates for LX Zones for OmniOS -
	zonecfg(1M) networking
In-Reply-To: <855DE2C3-A631-45CE-BEBD-D674A77576C4@omniti.com>
References: <855DE2C3-A631-45CE-BEBD-D674A77576C4@omniti.com>
Message-ID: <66468812-08E9-40CE-8E98-1853723B5690@omniti.com>


> On Sep 1, 2016, at 6:18 PM, Dan McDonald <danmcd at omniti.com> wrote:
> 
> Aka. "A tale of two LX zonecfgs".
> 
> I've updated the io-lx-public repo
> 
> 	https://github.com/danmcd/io-lx-public/
> 
> AND the IPS server for bloody+LX:
> 
> 	http://pkg.omniti.com/omnios/LX/

I almost forgot:  The webrevs are updated too:

	http://kebe.com/~danmcd/webrevs/io-lx-public/

	http://kebe.com/~danmcd/webrevs/io-lx-public-vs-joyent/

Dan


From mir at miras.org  Thu Sep  1 23:19:59 2016
From: mir at miras.org (Michael Rasmussen)
Date: Fri, 2 Sep 2016 01:19:59 +0200
Subject: [OmniOS-discuss] New updates for LX Zones for OmniOS -
 zonecfg(1M) networking
In-Reply-To: <66468812-08E9-40CE-8E98-1853723B5690@omniti.com>
References: <855DE2C3-A631-45CE-BEBD-D674A77576C4@omniti.com>
	<66468812-08E9-40CE-8E98-1853723B5690@omniti.com>
Message-ID: <20160902011959.4455e3de@sleipner.datanom.net>

On Thu, 1 Sep 2016 18:25:24 -0400
Dan McDonald <danmcd at omniti.com> wrote:

> 
> I almost forgot:  The webrevs are updated too:
> 
> 	http://kebe.com/~danmcd/webrevs/io-lx-public/
> 
> 	http://kebe.com/~danmcd/webrevs/io-lx-public-vs-joyent/
> 
For both links:
Forbidden

You don't have permission to access /~danmcd/webrevs/io-lx-public/ on this server.

???

-- 
Hilsen/Regards
Michael Rasmussen

Get my public GnuPG keys:
michael <at> rasmussen <dot> cc
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E
mir <at> datanom <dot> net
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C
mir <at> miras <dot> org
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917
--------------------------------------------------------------
/usr/games/fortune -es says:
I can resist anything but temptation.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 163 bytes
Desc: OpenPGP digital signature
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20160902/476e7d45/attachment.bin>

From danmcd at omniti.com  Fri Sep  2 00:53:16 2016
From: danmcd at omniti.com (Dan McDonald)
Date: Thu, 1 Sep 2016 20:53:16 -0400
Subject: [OmniOS-discuss] New updates for LX Zones for OmniOS -
	zonecfg(1M) networking
In-Reply-To: <20160902011959.4455e3de@sleipner.datanom.net>
References: <855DE2C3-A631-45CE-BEBD-D674A77576C4@omniti.com>
	<66468812-08E9-40CE-8E98-1853723B5690@omniti.com>
	<20160902011959.4455e3de@sleipner.datanom.net>
Message-ID: <46548612-3343-47EE-9FDA-0798FFB53DD9@omniti.com>


> On Sep 1, 2016, at 7:19 PM, Michael Rasmussen <mir at miras.org> wrote:
> 
> On Thu, 1 Sep 2016 18:25:24 -0400
> Dan McDonald <danmcd at omniti.com> wrote:
> 
>> 
>> I almost forgot:  The webrevs are updated too:
>> 
>> 	http://kebe.com/~danmcd/webrevs/io-lx-public/
>> 
>> 	http://kebe.com/~danmcd/webrevs/io-lx-public-vs-joyent/
>> 
> For both links:
> Forbidden
> 
> You don't have permission to access /~danmcd/webrevs/io-lx-public/ on this server.

Please try now.

Sorry about that,
Dan


From hannohirschberger at googlemail.com  Fri Sep  2 08:25:45 2016
From: hannohirschberger at googlemail.com (Hanno Hirschberger)
Date: Fri, 2 Sep 2016 10:25:45 +0200
Subject: [OmniOS-discuss] Missing /devices/pseudo/kvm* with Intel Xeon
	E3-1240 v3
Message-ID: <e26603ac-7324-dcc4-0c1d-d567a94581f4@googlemail.com>

Hey guys!

I run OmniOS r151014 on a SuperMicro server with an Intel Xeon E3-1240 
v3 CPU. We need kvm for a qemu-kvm machine but the kvm device is missing.

The symlink /dev/kvm is there but there's no /devices/pseudo/kvm at 0:kvm 
where it's pointing. I enabled Intel VT-x on the mainboard. The kernel 
module files are present but OmniOS is unable to load them ("file not 
found" - obviously). So I'm stuck now and don't know how to analyze this 
further. Please help me out!

Do you have any hints or encountered a similar problem? Is the processor 
unsupported by any chance?

The mainboard is a Supermicro Server Board X10SLM-F. Let me know if you 
need anything.

Best regards,

Hanno

From danmcd at omniti.com  Fri Sep  2 14:06:02 2016
From: danmcd at omniti.com (Dan McDonald)
Date: Fri, 2 Sep 2016 10:06:02 -0400
Subject: [OmniOS-discuss] Missing /devices/pseudo/kvm* with Intel Xeon
	E3-1240 v3
In-Reply-To: <e26603ac-7324-dcc4-0c1d-d567a94581f4@googlemail.com>
References: <e26603ac-7324-dcc4-0c1d-d567a94581f4@googlemail.com>
Message-ID: <2D8A88BA-5AF7-4B00-836A-0F38F5374A2D@omniti.com>


> On Sep 2, 2016, at 4:25 AM, Hanno Hirschberger <hannohirschberger at googlemail.com> wrote:
> 
> Hey guys!
> 
> I run OmniOS r151014 on a SuperMicro server with an Intel Xeon E3-1240 v3 CPU. We need kvm for a qemu-kvm machine but the kvm device is missing.

This has nothing to do with the processor.

> The symlink /dev/kvm is there but there's no /devices/pseudo/kvm at 0:kvm where it's pointing. I enabled Intel VT-x on the mainboard. The kernel module files are present but OmniOS is unable to load them ("file not found" - obviously). So I'm stuck now and don't know how to analyze this further. Please help me out!

Please do this:

	pkg list kvm

You should see two lines of output:

	r151014(~)[0]% pkg list kvm
	NAME (PUBLISHER)                                  VERSION                    IFO
	driver/virtualization/kvm                         1.0.5.11-0.151014          i--
	system/kvm                                        1.0.5.11-0.151014          i--
	r151014(~)[0]% 

Are those packages missing?  If so, "pkg install" them both.

It's strange that they would be missing, however, given they're part of the "entire" consolidation.  If they are there in "pkg list", you may wish to "pkg verify driver/virtualization/kvm system/kvm" to make sure.

Dan



From danmcd at omniti.com  Sat Sep  3 05:48:18 2016
From: danmcd at omniti.com (Dan McDonald)
Date: Sat, 3 Sep 2016 01:48:18 -0400
Subject: [OmniOS-discuss] I broke Centos/Redhat LX zones,
	plus -s now takes ZFS filesystem/snapshot
Message-ID: <527A65A1-E20F-415F-AF3D-514F22F5582B@omniti.com>

I've just updated the LX brand package on the http://pkg.omniti.com/omnios/LX repo.

Besides fixing Centos/Redhat *and* busybox LX zones, I've included a new method for -s on install, where it takes a filesystem argument, in lieu of a pathname to a ZFS send stream.  If you have a "images" dataset with filesystems in it, you can just do:

	zoneadm -z lx0 install -s rpool/images/centos6

for example.

Sorry about the bug, and enjoy the image-clone-install,
Dan


From johan.kragsterman at capvert.se  Sat Sep  3 11:05:02 2016
From: johan.kragsterman at capvert.se (Johan Kragsterman)
Date: Sat, 3 Sep 2016 13:05:02 +0200
Subject: [OmniOS-discuss] stmfadm add-hg-member -F -g cluster1
	wwn.2100001B321EDFB9
Message-ID: <OF71E5A70E.624EFDD0-ONC1258023.003C553F-C1258023.003CE2C7@inse.com>


Hi!


I'd like to hear from people using omnios as a scsi target appliance, how/if they use host group members in multiple groups, or just in one?

The use case for putting a member in multiple(or perhaps just two) host groups, would be a cluster where you can have a host group for the entire cluster, with all members included there, and then individual host groups for each host for os/boot or other private lu's.

The other way to do this is of coarse to provide multiple views for the lu's that should be shared across the cluster. IMHO, though, a single cluster host group would be more simple to handle.

So, how do YOU do it? And why...?

Opinions pls...?


Best regards from/Med v?nliga h?lsningar fr?n

Johan Kragsterman

Capvert


From alka at hfg-gmuend.de  Sat Sep  3 11:57:30 2016
From: alka at hfg-gmuend.de (Guenther Alka)
Date: Sat, 3 Sep 2016 13:57:30 +0200
Subject: [OmniOS-discuss] stmfadm add-hg-member -F -g cluster1
 wwn.2100001B321EDFB9
In-Reply-To: <OF71E5A70E.624EFDD0-ONC1258023.003C553F-C1258023.003CE2C7@inse.com>
References: <OF71E5A70E.624EFDD0-ONC1258023.003C553F-C1258023.003CE2C7@inse.com>
Message-ID: <2dcd8728-0c68-23c6-296e-ba2bdc8eef77@hfg-gmuend.de>

You can control access and visibility of LUNs either based on Initiators 
(host groups), targets (target groups), network links (target portal 
groups) or views that make a logical unit visible in a target. Every 
solution works with a different use case idea.

In napp-it I am currently working on full appliance redundancy with 
Z-Raid pool mirroring over iSCSI for a storage and service failover 
cluster.  I have played with the solutions above but in the end I 
decided to use none of them but use manual target discovery with 
switchable IP aliases for iSCSI targets to restrict and switchover LUNs 
and for NFS/SMB failover between the active master storage appliance and 
the slave. The reason was simplicity and that this works on a lower 
level that does not rely to Comstar timings and delays.

Gea

Am 03.09.2016 um 13:05 schrieb Johan Kragsterman:
> Hi!
>
>
> I'd like to hear from people using omnios as a scsi target appliance, how/if they use host group members in multiple groups, or just in one?
>
> The use case for putting a member in multiple(or perhaps just two) host groups, would be a cluster where you can have a host group for the entire cluster, with all members included there, and then individual host groups for each host for os/boot or other private lu's.
>
> The other way to do this is of coarse to provide multiple views for the lu's that should be shared across the cluster. IMHO, though, a single cluster host group would be more simple to handle.
>
> So, how do YOU do it? And why...?
>
> Opinions pls...?
>
>
> Best regards from/Med v?nliga h?lsningar fr?n
>
> Johan Kragsterman
>
> Capvert
>
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss


From johan.kragsterman at capvert.se  Sat Sep  3 12:15:56 2016
From: johan.kragsterman at capvert.se (Johan Kragsterman)
Date: Sat, 3 Sep 2016 14:15:56 +0200
Subject: [OmniOS-discuss] Ang: Re: stmfadm add-hg-member -F -g cluster1
	wwn.2100001B321EDFB9
In-Reply-To: <2dcd8728-0c68-23c6-296e-ba2bdc8eef77@hfg-gmuend.de>
References: <2dcd8728-0c68-23c6-296e-ba2bdc8eef77@hfg-gmuend.de>,
	<OF71E5A70E.624EFDD0-ONC1258023.003C553F-C1258023.003CE2C7@inse.com>
Message-ID: <OFBA6013C4.630F55A1-ONC1258023.00434D4C-C1258023.0043608B@inse.com>


Hi!

Thanks for the input, Gea!


-----"OmniOS-discuss" <omnios-discuss-bounces at lists.omniti.com> skrev: -----
Till: omnios-discuss at lists.omniti.com
Fr?n: Guenther Alka 
S?nt av: "OmniOS-discuss" 
Datum: 2016-09-03 12:01
?rende: Re: [OmniOS-discuss] stmfadm add-hg-member -F -g cluster1 wwn.2100001B321EDFB9

You can control access and visibility of LUNs either based on Initiators 
(host groups), targets (target groups), network links (target portal 
groups) or views that make a logical unit visible in a target. Every 
solution works with a different use case idea.

In napp-it I am currently working on full appliance redundancy with 
Z-Raid pool mirroring over iSCSI for a storage and service failover 
cluster. ?I have played with the solutions above but in the end I 
decided to use none of them but use manual target discovery with 
switchable IP aliases for iSCSI targets to restrict and switchover LUNs 
and for NFS/SMB failover between the active master storage appliance and 
the slave. The reason was simplicity and that this works on a lower 
level that does not rely to Comstar timings and delays.

Gea



I mainly use FC, and never iScsi, so my control mechanisms are host and target groups, and views. My setups are often that I run clusters with multiple host machines, which runs multiple virtual machines, and if I should map every lu with a new view it would be a hell of a job. It is much easier to just put the wwn in the cluster host group, than to maintain all these views.


Rgrds Johan



Am 03.09.2016 um 13:05 schrieb Johan Kragsterman:
> Hi!
>
>
> I'd like to hear from people using omnios as a scsi target appliance, how/if they use host group members in multiple groups, or just in one?
>
> The use case for putting a member in multiple(or perhaps just two) host groups, would be a cluster where you can have a host group for the entire cluster, with all members included there, and then individual host groups for each host for os/boot or other private lu's.
>
> The other way to do this is of coarse to provide multiple views for the lu's that should be shared across the cluster. IMHO, though, a single cluster host group would be more simple to handle.
>
> So, how do YOU do it? And why...?
>
> Opinions pls...?
>
>
> Best regards from/Med v?nliga h?lsningar fr?n
>
> Johan Kragsterman
>
> Capvert
>
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss

_______________________________________________
OmniOS-discuss mailing list
OmniOS-discuss at lists.omniti.com
http://lists.omniti.com/mailman/listinfo/omnios-discuss




From bfriesen at simple.dallas.tx.us  Sat Sep  3 21:42:53 2016
From: bfriesen at simple.dallas.tx.us (Bob Friesenhahn)
Date: Sat, 3 Sep 2016 16:42:53 -0500 (CDT)
Subject: [OmniOS-discuss] LSI 9300-16i SAS and OmniOS?
Message-ID: <alpine.GSO.2.20.1609031639550.19594@freddy.simplesystems.org>

Is the LSI 9300-16i SAS support in OmniOS known to be robust?

I am planning to use it with the SuperMicro SC836BA-R920B 3U case and 
Supermicro X10DRi-T4+ motherboard to build a storage server.

Is there anything about this HBA and motherboard that might cause an 
issue?

Thanks,

Bob
-- 
Bob Friesenhahn
bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/

From peter.tribble at gmail.com  Sun Sep  4 09:00:36 2016
From: peter.tribble at gmail.com (Peter Tribble)
Date: Sun, 4 Sep 2016 10:00:36 +0100
Subject: [OmniOS-discuss] LSI 9300-16i SAS and OmniOS?
In-Reply-To: <alpine.GSO.2.20.1609031639550.19594@freddy.simplesystems.org>
References: <alpine.GSO.2.20.1609031639550.19594@freddy.simplesystems.org>
Message-ID: <CAEgYsbFdQiOj+sPkcuo+Y5UACRvtyPq-c8NJ3dYv=Khr1C9J=w@mail.gmail.com>

On Sat, Sep 3, 2016 at 10:42 PM, Bob Friesenhahn <
bfriesen at simple.dallas.tx.us> wrote:

> Is the LSI 9300-16i SAS support in OmniOS known to be robust?
>
> I am planning to use it with the SuperMicro SC836BA-R920B 3U case and
> Supermicro X10DRi-T4+ motherboard to build a storage server.
>
> Is there anything about this HBA and motherboard that might cause an issue?
>

Should be good. We're using that X10DRi-T4+ motherboard with the LSI
9300-8i HBA (in the 2U chassis,
where the 16i wouldn't fit), and everything has worked perfectly so far.

-- 
-Peter Tribble
http://www.petertribble.co.uk/ - http://ptribble.blogspot.com/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20160904/0ce8c96e/attachment.html>

From bfriesen at simple.dallas.tx.us  Sun Sep  4 14:46:45 2016
From: bfriesen at simple.dallas.tx.us (Bob Friesenhahn)
Date: Sun, 4 Sep 2016 09:46:45 -0500 (CDT)
Subject: [OmniOS-discuss] LSI 9300-16i SAS and OmniOS?
In-Reply-To: <CAEgYsbFdQiOj+sPkcuo+Y5UACRvtyPq-c8NJ3dYv=Khr1C9J=w@mail.gmail.com>
References: <alpine.GSO.2.20.1609031639550.19594@freddy.simplesystems.org>
	<CAEgYsbFdQiOj+sPkcuo+Y5UACRvtyPq-c8NJ3dYv=Khr1C9J=w@mail.gmail.com>
Message-ID: <alpine.GSO.2.20.1609040938250.19594@freddy.simplesystems.org>

On Sun, 4 Sep 2016, Peter Tribble wrote:

> On Sat, Sep 3, 2016 at 10:42 PM, Bob Friesenhahn <
> bfriesen at simple.dallas.tx.us> wrote:
>
>> Is the LSI 9300-16i SAS support in OmniOS known to be robust?
>>
>> I am planning to use it with the SuperMicro SC836BA-R920B 3U case and
>> Supermicro X10DRi-T4+ motherboard to build a storage server.
>>
>> Is there anything about this HBA and motherboard that might cause an issue?
>>
>
> Should be good. We're using that X10DRi-T4+ motherboard with the LSI
> 9300-8i HBA (in the 2U chassis,
> where the 16i wouldn't fit), and everything has worked perfectly so far.

That is good to hear.  I did not see any hardware on the motherboard 
which is not supported by OmniOS.

Something which is not clear to me (or to my computer builder) is if 
using a 12gb SAS HBA and 12gb SAS disks with a passive backplane 
described by SuperMicro as being 3g/6g might cause issues.  It is not 
clear if the end devices somehow detect that the chassis is certified 
for 6g or if they will use 12g and possibly work just fine, or 
encounter issues.  The signalling and cables for 6g and 12g are the 
same, but with 12g using a higher clocking rate.

Bob
-- 
Bob Friesenhahn
bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/

From matej at zunaj.si  Mon Sep  5 06:58:57 2016
From: matej at zunaj.si (=?utf-8?Q?Matej_=C5=BDerovnik?=)
Date: Mon, 5 Sep 2016 08:58:57 +0200
Subject: [OmniOS-discuss] LSI 9300-16i SAS and OmniOS?
In-Reply-To: <alpine.GSO.2.20.1609040938250.19594@freddy.simplesystems.org>
References: <alpine.GSO.2.20.1609031639550.19594@freddy.simplesystems.org>
	<CAEgYsbFdQiOj+sPkcuo+Y5UACRvtyPq-c8NJ3dYv=Khr1C9J=w@mail.gmail.com>
	<alpine.GSO.2.20.1609040938250.19594@freddy.simplesystems.org>
Message-ID: <B3D0A761-9BC3-45DB-A89A-691DC8410F7F@zunaj.si>

We are using SAS3 drives with LSI 9300-8e and SC837E26-RJBOD1 and it?s working great for the last 4 months.

I?m not sure if it?s running SAS2 or SAS3, since I dont see those information nowhere (I checked messages and sas3ircu). It only says SAS

Matej

> On 04 Sep 2016, at 16:46, Bob Friesenhahn <bfriesen at simple.dallas.tx.us> wrote:
> 
> On Sun, 4 Sep 2016, Peter Tribble wrote:
> 
>> On Sat, Sep 3, 2016 at 10:42 PM, Bob Friesenhahn <
>> bfriesen at simple.dallas.tx.us> wrote:
>> 
>>> Is the LSI 9300-16i SAS support in OmniOS known to be robust?
>>> 
>>> I am planning to use it with the SuperMicro SC836BA-R920B 3U case and
>>> Supermicro X10DRi-T4+ motherboard to build a storage server.
>>> 
>>> Is there anything about this HBA and motherboard that might cause an issue?
>>> 
>> 
>> Should be good. We're using that X10DRi-T4+ motherboard with the LSI
>> 9300-8i HBA (in the 2U chassis,
>> where the 16i wouldn't fit), and everything has worked perfectly so far.
> 
> That is good to hear.  I did not see any hardware on the motherboard which is not supported by OmniOS.
> 
> Something which is not clear to me (or to my computer builder) is if using a 12gb SAS HBA and 12gb SAS disks with a passive backplane described by SuperMicro as being 3g/6g might cause issues.  It is not clear if the end devices somehow detect that the chassis is certified for 6g or if they will use 12g and possibly work just fine, or encounter issues.  The signalling and cables for 6g and 12g are the same, but with 12g using a higher clocking rate.
> 
> Bob
> -- 
> Bob Friesenhahn
> bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
> GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss


From hannohirschberger at googlemail.com  Mon Sep  5 08:13:23 2016
From: hannohirschberger at googlemail.com (Hanno Hirschberger)
Date: Mon, 5 Sep 2016 10:13:23 +0200
Subject: [OmniOS-discuss] Missing /devices/pseudo/kvm* with Intel Xeon
 E3-1240 v3
In-Reply-To: <2D8A88BA-5AF7-4B00-836A-0F38F5374A2D@omniti.com>
References: <e26603ac-7324-dcc4-0c1d-d567a94581f4@googlemail.com>
	<2D8A88BA-5AF7-4B00-836A-0F38F5374A2D@omniti.com>
Message-ID: <564c13ea-848a-7acc-d60d-c05b24207b1b@googlemail.com>

Sorry Dan, forgot to reply to the mailing list.

On 02.09.2016 16:06, Dan McDonald wrote:
> Please do this:
> 	pkg list kvm
> [...]
>
> Are those packages missing?  If so, "pkg install" them both.

Both are installed. Here's the output:

NAME (PUBLISHER)           VERSION             IFO
driver/virtualization/kvm  1.0.5.11-0.151014   i--
system/kvm                 1.0.5.11-0.151014   i--

> [...]  If they are there in "pkg list", you may wish to "pkg verify driver/virtualization/kvm system/kvm" to make sure.

Now it's getting interesting. Here's the error the verification is 
telling me:

PACKAGE                                 STATUS
pkg://omnios/drive/virtualization/kvm   ERROR

    driver: kvm
       etc/name_to_major: 'kvm' entry not present

I checked the file and indeed there's no such entry. I mounted a few 
older boot environments. There's no 'kvm' entry in any of the 
/etc/name_to_major files. Do you have any idea how to tackle this?

Hanno

From an3s.annema at gmail.com  Mon Sep  5 09:51:06 2016
From: an3s.annema at gmail.com (Andries Annema)
Date: Mon, 5 Sep 2016 11:51:06 +0200
Subject: [OmniOS-discuss] r151014 repo offline?
In-Reply-To: <F387C521-2FF0-439D-A9CD-43FECADBEF86@omniti.com>
References: <8ecefd54-9758-98c8-d83e-0935d0aec4fd@gmail.com>
	<52CBF697-0C20-4665-A7FF-119B3381F0DB@omniti.com>
	<op.yle9cbzojdf4tt@ws01-win7>
	<a4e0f888-6eb4-be71-74bd-b30bb08fa601@gmail.com>
	<F387C521-2FF0-439D-A9CD-43FECADBEF86@omniti.com>
Message-ID: <000301d2075b$0b0f5be0$212e13a0$@gmail.com>

Hi Dan,

I hate to bring this matter up again, but it seems there's still something
loose here.

This page on itself works flawlessly:
http://pkg.omniti.com/omnios/r151014/en/index.shtml
But upon hitting the "browse" button next to the tiny little "Release and
Branch" pull-down menu, set to the default value of "11-0.151014",  the
catalog.shtml-page that should come up fails with a read timeout. The
address bar then shows this URL:
http://pkg.omniti.com/omnios/r151014/en/catalog.shtml?version=entire%4011%2C
5.11-0.151014&action=Browse 

However - and this surprised me and it might give you a hint on where the
loose screw is located - when selecting "All" in that pull-down menu, after
about 30 seconds (maybe a bit slow?), a catalog.shtml-page with a package
list is indeed displayed. The address bar then shows this URL:
http://pkg.omniti.com/omnios/r151014/en/catalog.shtml?version=&action=Browse


Furthermore, when accessing from OmniOS by running:
$ pkgrepo info -s http://pkg.omniti.com/omnios/r151014
it reports:
  PUBLISHER PACKAGES STATUS           UPDATED
  omnios    1022     processing       2016-08-10T18:35:55.166990Z
Not sure if the "processing" status here is of any concern.

While investigating this and writing this report, I also noticed that for
some time the whole thing seemed to collapse: suddenly the "pkgrepo info"
command reported "500 Internal Server Error" and later "Framework error:
code: 28 reason: Operation too slow.", and all the webpages mentioned above
timed out.
I waited half an hour or so and then it seemed to have gotten on its feet
again, as all responded as reported above. 

This repo seems a bit shaky. Hope this report is of any help in getting it
blazing again.

Regards,
Andries


-----Original Message-----
From: Dan McDonald [mailto:danmcd at omniti.com] 
Sent: zaterdag 30 juli 2016 22:01
To: omnios-discuss
Cc: Andreas Luka; Andries Annema; Dan McDonald
Subject: Re: [OmniOS-discuss] r151014 repo offline?

I checked the internal-side of things, and also had problems with
catalog.shtml.

I've since done "pkgrepo rebuild" and "pkgrepo refresh", and I got a
catalog.shtml list, but it took a long time.

I will investigate further on Monday.

Thanks for the report,
Dan



From davide.poletto at gmail.com  Mon Sep  5 12:03:18 2016
From: davide.poletto at gmail.com (Davide Poletto)
Date: Mon, 5 Sep 2016 14:03:18 +0200
Subject: [OmniOS-discuss] r151014 repo offline?
In-Reply-To: <000301d2075b$0b0f5be0$212e13a0$@gmail.com>
References: <8ecefd54-9758-98c8-d83e-0935d0aec4fd@gmail.com>
	<52CBF697-0C20-4665-A7FF-119B3381F0DB@omniti.com>
	<op.yle9cbzojdf4tt@ws01-win7>
	<a4e0f888-6eb4-be71-74bd-b30bb08fa601@gmail.com>
	<F387C521-2FF0-439D-A9CD-43FECADBEF86@omniti.com>
	<000301d2075b$0b0f5be0$212e13a0$@gmail.com>
Message-ID: <CANKMAMbtdP=qTv6=h0WXNRP6YsDw2mKDWtuXOpDSaH5K4Hv5uQ@mail.gmail.com>

Me too, here it's the same:

pkgrepo info -s http://pkg.omniti.com/omnios/r151014
PUBLISHER PACKAGES STATUS           UPDATED
omnios    1022     processing       2016-08-10T18:35:55.166990Z

And, as Andries pointed out yet, just browsing the 11-0.151014 Release
returns a solid "Inactivity Timeout" (an HTTP/1.1 504 Connection Timed Out
181660 ms for the GET) showing a web page with this URL:

http://pkg.omniti.com/omnios/r151014/en/catalog.shtml?version=entire%4011%2C5.11-0.151014&action=Browse

AFAIK browsing the http://pkg.omniti.com/omnios/r151014/en/index.shtml
doesn't look stable at all (few minutes ago I was able to reach the page
very slowly, now it ends with a timeout)

Tested using two different connections through two different ISPs (Italy).

On Mon, Sep 5, 2016 at 11:51 AM, Andries Annema <an3s.annema at gmail.com>
wrote:

> Hi Dan,
>
> I hate to bring this matter up again, but it seems there's still something
> loose here.
>
> This page on itself works flawlessly:
> http://pkg.omniti.com/omnios/r151014/en/index.shtml
> But upon hitting the "browse" button next to the tiny little "Release and
> Branch" pull-down menu, set to the default value of "11-0.151014",  the
> catalog.shtml-page that should come up fails with a read timeout. The
> address bar then shows this URL:
> http://pkg.omniti.com/omnios/r151014/en/catalog.shtml?versio
> n=entire%4011%2C
> 5.11-0.151014&action=Browse
> <http://pkg.omniti.com/omnios/r151014/en/catalog.shtml?version=entire%4011%2C5.11-0.151014&action=Browse>
>
> However - and this surprised me and it might give you a hint on where the
> loose screw is located - when selecting "All" in that pull-down menu, after
> about 30 seconds (maybe a bit slow?), a catalog.shtml-page with a package
> list is indeed displayed. The address bar then shows this URL:
> http://pkg.omniti.com/omnios/r151014/en/catalog.shtml?versio
> n=&action=Browse
>
>
> Furthermore, when accessing from OmniOS by running:
> $ pkgrepo info -s http://pkg.omniti.com/omnios/r151014
> it reports:
>   PUBLISHER PACKAGES STATUS           UPDATED
>   omnios    1022     processing       2016-08-10T18:35:55.166990Z
> Not sure if the "processing" status here is of any concern.
>
> While investigating this and writing this report, I also noticed that for
> some time the whole thing seemed to collapse: suddenly the "pkgrepo info"
> command reported "500 Internal Server Error" and later "Framework error:
> code: 28 reason: Operation too slow.", and all the webpages mentioned above
> timed out.
> I waited half an hour or so and then it seemed to have gotten on its feet
> again, as all responded as reported above.
>
> This repo seems a bit shaky. Hope this report is of any help in getting it
> blazing again.
>
> Regards,
> Andries
>
>
> -----Original Message-----
> From: Dan McDonald [mailto:danmcd at omniti.com]
> Sent: zaterdag 30 juli 2016 22:01
> To: omnios-discuss
> Cc: Andreas Luka; Andries Annema; Dan McDonald
> Subject: Re: [OmniOS-discuss] r151014 repo offline?
>
> I checked the internal-side of things, and also had problems with
> catalog.shtml.
>
> I've since done "pkgrepo rebuild" and "pkgrepo refresh", and I got a
> catalog.shtml list, but it took a long time.
>
> I will investigate further on Monday.
>
> Thanks for the report,
> Dan
>
>
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20160905/ac9bba63/attachment.html>

From info.fmo at gmx.de  Tue Sep  6 13:18:06 2016
From: info.fmo at gmx.de (Frank M.)
Date: Tue, 6 Sep 2016 15:18:06 +0200
Subject: [OmniOS-discuss] LSI 9300-16i SAS and OmniOS?
In-Reply-To: <alpine.GSO.2.20.1609040938250.19594@freddy.simplesystems.org>
References: <alpine.GSO.2.20.1609031639550.19594@freddy.simplesystems.org>
	<CAEgYsbFdQiOj+sPkcuo+Y5UACRvtyPq-c8NJ3dYv=Khr1C9J=w@mail.gmail.com>
	<alpine.GSO.2.20.1609040938250.19594@freddy.simplesystems.org>
Message-ID: <976878138.20160906151806@gmx.de>

Hi,

I have an new system with a LSI 9300-16i in slot3 of a X10DRi-T4+
motherboard on a 6G-Backplane.
In my short tests with barmetal OmniOS there were no problems. But I
would use VMware-virtualized OmniOS with passthrough LSI-HBA. And here
I have massive problems... The disks/pools crash after a short time
with low load.
I testet the HBA in another PCIe-slot - I havn?t seen any problems, but
I havn?t looked into the logfile.

Supermicro recommended to set jumper JI2C1 and JI2C2 to
2-3 - but they were set to this already. So I set the jumper to 1-2.
Now disks/pools crash still earlier.

The HBA gets very hot, but I testet it also with "special" airflow -
the same....

I attached the logfile...

What does it mean "FastPath":
----
Sep  6 15:07:05 test-san1 scsi: [ID 243001 kern.info]   w5000cca24200d4fd FastPath Capable and Enabled
Sep  6 15:07:05 test-san1 scsi: [ID 243001 kern.info]   w4433221106000000 FastPath Capable but Disabled
Sep  6 15:07:05 test-san1 scsi: [ID 243001 kern.info]   w5000cca24200d4c9 FastPath Capable and Enabled
Sep  6 15:07:05 test-san1 scsi: [ID 243001 kern.info]   w5000cca2426fff85 FastPath Capable and Enabled
Sep  6 15:07:05 test-san1 scsi: [ID 243001 kern.info]   w5000cca24200d33d FastPath Capable and Enabled
Sep  6 15:07:05 test-san1 scsi: [ID 243001 kern.info]   w5000cca24200d4c9 FastPath Capable and Enabled
----
and later:
----
Sep  6 15:10:19 test-san1 scsi: [ID 107833 kern.warning] WARNING: /pci at 0,0/pci15ad,7a0 at 18/pci1000,3130 at 0 (mpt_sas19):
Sep  6 15:10:19 test-san1       MPT Firmware Fault, code: 5854
Sep  6 15:10:21 test-san1 scsi: [ID 365881 kern.info] /pci at 0,0/pci15ad,7a0 at 18/pci1000,3130 at 0 (mpt_sas19):
Sep  6 15:10:21 test-san1       MPT Firmware version v12.0.0.0 (SAS3008)
Sep  6 15:10:21 test-san1 scsi: [ID 365881 kern.info] /pci at 0,0/pci15ad,7a0 at 18/pci1000,3130 at 0 (mpt_sas19):
Sep  6 15:10:21 test-san1       mpt_sas19 SAS 3 Supported
Sep  6 15:10:21 test-san1 scsi: [ID 365881 kern.info] /pci at 0,0/pci15ad,7a0 at 18/pci1000,3130 at 0 (mpt_sas19):
Sep  6 15:10:21 test-san1       mpt19: IOC Operational.
Sep  6 15:10:28 test-san1 scsi: [ID 243001 kern.info]   w4433221100000000 FastPath Capable but Disabled
Sep  6 15:10:28 test-san1 last message repeated 1 time
Sep  6 15:10:28 test-san1 scsi: [ID 243001 kern.info]   w4433221101000000 FastPath Capable but Disabled
Sep  6 15:10:28 test-san1 last message repeated 1 time
Sep  6 15:10:29 test-san1 scsi: [ID 243001 kern.info]   w4433221102000000 FastPath Capable but Disabled
Sep  6 15:10:29 test-san1 last message repeated 1 time
Sep  6 15:10:29 test-san1 scsi: [ID 243001 kern.info]   w4433221103000000 FastPath Capable but Disabled
Sep  6 15:10:29 test-san1 last message repeated 1 time
Sep  6 15:10:29 test-san1 scsi: [ID 243001 kern.info]   w4433221106000000 FastPath Capable but Disabled
Sep  6 15:10:29 test-san1 last message repeated 1 time
Sep  6 15:10:29 test-san1 scsi: [ID 243001 kern.info]   w4433221107000000 FastPath Capable but Disabled
Sep  6 15:10:29 test-san1 last message repeated 1 time
Sep  6 15:10:31 test-san1 scsi: [ID 107833 kern.warning] WARNING: /pci at 0,0/pci15ad,7a0 at 18/pci1000,3130 at 0 (mpt_sas19):
Sep  6 15:10:31 test-san1       MPT Firmware Fault, code: 5854
Sep  6 15:10:32 test-san1 scsi: [ID 365881 kern.info] /pci at 0,0/pci15ad,7a0 at 18/pci1000,3130 at 0 (mpt_sas19):
Sep  6 15:10:32 test-san1       MPT Firmware version v12.0.0.0 (SAS3008)
Sep  6 15:10:32 test-san1 scsi: [ID 365881 kern.info] /pci at 0,0/pci15ad,7a0 at 18/pci1000,3130 at 0 (mpt_sas19):
Sep  6 15:10:32 test-san1       mpt_sas19 SAS 3 Supported
Sep  6 15:10:32 test-san1 scsi: [ID 365881 kern.info] /pci at 0,0/pci15ad,7a0 at 18/pci1000,3130 at 0 (mpt_sas19):
Sep  6 15:10:32 test-san1       mpt19: IOC Operational.
Sep  6 15:10:40 test-san1 scsi: [ID 243001 kern.info]   w4433221100000000 FastPath Capable but Disabled
Sep  6 15:10:40 test-san1 scsi: [ID 243001 kern.info]   w4433221101000000 FastPath Capable but Disabled
Sep  6 15:10:40 test-san1 scsi: [ID 243001 kern.info]   w4433221102000000 FastPath Capable but Disabled
Sep  6 15:10:40 test-san1 scsi: [ID 243001 kern.info]   w4433221103000000 FastPath Capable but Disabled
Sep  6 15:10:41 test-san1 scsi: [ID 243001 kern.info]   w4433221106000000 FastPath Capable but Disabled
Sep  6 15:10:41 test-san1 scsi: [ID 243001 kern.info]   w4433221107000000 FastPath Capable but Disabled
----

For information:
pool1 6x SATA-SSD Raidz2
pool2 4x SAS-HDD 2 mirrored vdev with SATA-SSD l2arc

I hope anybody has any useful hints...

PS:
Tonight I want to test with only one HDD...

Frank


Sunday, September 4, 2016, 4:46:45 PM, you wrote:

BF> On Sun, 4 Sep 2016, Peter Tribble wrote:

>> On Sat, Sep 3, 2016 at 10:42 PM, Bob Friesenhahn <
>> bfriesen at simple.dallas.tx.us> wrote:
>>
>>> Is the LSI 9300-16i SAS support in OmniOS known to be robust?
>>>
>>> I am planning to use it with the SuperMicro SC836BA-R920B 3U case and
>>> Supermicro X10DRi-T4+ motherboard to build a storage server.
>>>
>>> Is there anything about this HBA and motherboard that might cause an issue?
>>>
>>
>> Should be good. We're using that X10DRi-T4+ motherboard with the LSI
>> 9300-8i HBA (in the 2U chassis,
>> where the 16i wouldn't fit), and everything has worked perfectly so far.

BF> That is good to hear.  I did not see any hardware on the motherboard 
BF> which is not supported by OmniOS.

BF> Something which is not clear to me (or to my computer builder) is if 
BF> using a 12gb SAS HBA and 12gb SAS disks with a passive backplane 
BF> described by SuperMicro as being 3g/6g might cause issues.  It is not 
BF> clear if the end devices somehow detect that the chassis is certified 
BF> for 6g or if they will use 12g and possibly work just fine, or 
BF> encounter issues.  The signalling and cables for 6g and 12g are the 
BF> same, but with 12g using a higher clocking rate.

BF> Bob



-- 
Best regards,
 Frank                            mailto:info.fmo at gmx.de

---
Diese E-Mail wurde von Avast Antivirus-Software auf Viren gepr?ft.
https://www.avast.com/antivirus
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: messages-test-san.txt
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20160906/473c3aa0/attachment-0001.txt>

From danmcd at omniti.com  Tue Sep  6 14:53:27 2016
From: danmcd at omniti.com (Dan McDonald)
Date: Tue, 6 Sep 2016 10:53:27 -0400
Subject: [OmniOS-discuss] Missing /devices/pseudo/kvm* with Intel Xeon
	E3-1240 v3
In-Reply-To: <564c13ea-848a-7acc-d60d-c05b24207b1b@googlemail.com>
References: <e26603ac-7324-dcc4-0c1d-d567a94581f4@googlemail.com>
	<2D8A88BA-5AF7-4B00-836A-0F38F5374A2D@omniti.com>
	<564c13ea-848a-7acc-d60d-c05b24207b1b@googlemail.com>
Message-ID: <4DD6D14B-495C-401A-9525-8AF41230DF98@omniti.com>


> On Sep 5, 2016, at 4:13 AM, Hanno Hirschberger <hannohirschberger at googlemail.com> wrote:
> 
> 
> I checked the file and indeed there's no such entry. I mounted a few older boot environments. There's no 'kvm' entry in any of the /etc/name_to_major files. Do you have any idea how to tackle this?

Try re-running devfsadm:

	# devfsadm
	# grep kvm /etc/name_to_major

Also, perhaps "pkg verify kvm" may be useful.

Dan


From vizoster at gmail.com  Tue Sep  6 16:56:37 2016
From: vizoster at gmail.com (Vizo Allman)
Date: Tue, 6 Sep 2016 12:56:37 -0400
Subject: [OmniOS-discuss] new to omnios and zfs...
Message-ID: <CADdbb3w+40cSgBB9h5WXXd_7oGpBh9eURiT-TymeQVZGNok4pA@mail.gmail.com>

question: very new to zfs but looking at it to hold my VMs for kvm server
over NFS
I have some samsung 1tb pro ssd drives
 can I use those ? I am having a little trouble parsing the info I am
seeing online.

For the server I was planning on using a Supermicro with a 12 core cpu and
32 gigs of RAM it has a LSI-9211 in IT mode. I eventually want to use this
setup type in prodution for some small networks.

Any advise is appreciated.


-- 
"Nor aught availed him now
To have build in Heaven high towers    Nor did he scape
By all his engines
But was headlong sent with his industrious crew
To build in Hell
"Milton, "Paradise Lost"
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20160906/8785f04e/attachment.html>

From bfriesen at simple.dallas.tx.us  Tue Sep  6 17:31:45 2016
From: bfriesen at simple.dallas.tx.us (Bob Friesenhahn)
Date: Tue, 6 Sep 2016 12:31:45 -0500 (CDT)
Subject: [OmniOS-discuss] new to omnios and zfs...
In-Reply-To: <CADdbb3w+40cSgBB9h5WXXd_7oGpBh9eURiT-TymeQVZGNok4pA@mail.gmail.com>
References: <CADdbb3w+40cSgBB9h5WXXd_7oGpBh9eURiT-TymeQVZGNok4pA@mail.gmail.com>
Message-ID: <alpine.GSO.2.20.1609061227330.8984@freddy.simplesystems.org>

On Tue, 6 Sep 2016, Vizo Allman wrote:

> question: very new to zfs but looking at it to hold my VMs for kvm server
> over NFS
> I have some samsung 1tb pro ssd drives
> can I use those ? I am having a little trouble parsing the info I am
> seeing online.

What would be the purpose of these Samsung drives (e.g. synchronous 
write intent log, L2 ARC, main store) in your server?

For use as a synchronous intent log you should look for SSDs which 
support a super-cap to assure that all confirmed data is commited to 
underlying FLASH if the power fails.

Bob
-- 
Bob Friesenhahn
bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/

From vizoster at gmail.com  Tue Sep  6 17:44:47 2016
From: vizoster at gmail.com (Vizo Allman)
Date: Tue, 6 Sep 2016 13:44:47 -0400
Subject: [OmniOS-discuss] new to omnios and zfs...
In-Reply-To: <alpine.GSO.2.20.1609061227330.8984@freddy.simplesystems.org>
References: <CADdbb3w+40cSgBB9h5WXXd_7oGpBh9eURiT-TymeQVZGNok4pA@mail.gmail.com>
	<alpine.GSO.2.20.1609061227330.8984@freddy.simplesystems.org>
Message-ID: <CADdbb3xn509gpYgAt+WkeVKYsE8o4KJNun0xQgbeb23gpVR+_A@mail.gmail.com>

It would be the main store. I have 5 of these drives I will use them to
store my VM using linux kvm over NFS share to hold my mysql and ms sql VM
with DB. Small for now but I would like a similar setup for a small
production network that has the same services.

Hardware: supermicro mbd-h8sgl-f
with a 12 core AMD opteron
32 Gigs ECC memory (can add more)
LSI-9211 in IT mode.
 Two Intel? 82574L controllers Dual-Port Gigabit Ethernet


On Tue, Sep 6, 2016 at 1:31 PM, Bob Friesenhahn <
bfriesen at simple.dallas.tx.us> wrote:

> On Tue, 6 Sep 2016, Vizo Allman wrote:
>
> question: very new to zfs but looking at it to hold my VMs for kvm server
>> over NFS
>> I have some samsung 1tb pro ssd drives
>> can I use those ? I am having a little trouble parsing the info I am
>> seeing online.
>>
>
> What would be the purpose of these Samsung drives (e.g. synchronous write
> intent log, L2 ARC, main store) in your server?
>
> For use as a synchronous intent log you should look for SSDs which support
> a super-cap to assure that all confirmed data is commited to underlying
> FLASH if the power fails.
>
> Bob
> --
> Bob Friesenhahn
> bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
> GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/
>



-- 
"Nor aught availed him now
To have build in Heaven high towers    Nor did he scape
By all his engines
But was headlong sent with his industrious crew
To build in Hell
"Milton, "Paradise Lost"
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20160906/f624ec9b/attachment.html>

From nagele at wildbit.com  Tue Sep  6 17:50:14 2016
From: nagele at wildbit.com (Chris Nagele)
Date: Tue, 6 Sep 2016 13:50:14 -0400
Subject: [OmniOS-discuss] new to omnios and zfs...
In-Reply-To: <CADdbb3w+40cSgBB9h5WXXd_7oGpBh9eURiT-TymeQVZGNok4pA@mail.gmail.com>
References: <CADdbb3w+40cSgBB9h5WXXd_7oGpBh9eURiT-TymeQVZGNok4pA@mail.gmail.com>
Message-ID: <CAHfYOdXGUkzzihACpmBNJm-4Nz52oumqGQn8NcMEXhm4JZ49Rw@mail.gmail.com>

What kind of chassis are you looking at? I've tried the all SSD route
a few times, but you really want to use SAS drives and the cost is not
worth it. We ended up going with HGST (0B31069) for the log and cache,
and then larger 4TB or 6TB drives using 3 way mirrors for data.

Chris

From vizoster at gmail.com  Tue Sep  6 17:56:08 2016
From: vizoster at gmail.com (Vizo Allman)
Date: Tue, 6 Sep 2016 13:56:08 -0400
Subject: [OmniOS-discuss] new to omnios and zfs...
In-Reply-To: <CAHfYOdXGUkzzihACpmBNJm-4Nz52oumqGQn8NcMEXhm4JZ49Rw@mail.gmail.com>
References: <CADdbb3w+40cSgBB9h5WXXd_7oGpBh9eURiT-TymeQVZGNok4pA@mail.gmail.com>
	<CAHfYOdXGUkzzihACpmBNJm-4Nz52oumqGQn8NcMEXhm4JZ49Rw@mail.gmail.com>
Message-ID: <CADdbb3wygeky2KchLR23dtEPU1G84uVRwDoSZfkpZqJYeLtvkQ@mail.gmail.com>

https://www.supermicro.com/Aplus/system/1U/1012/AS-1012G-MTF.cfm

I added the lsi-9211. I actually have a few like this on hand

On Tue, Sep 6, 2016 at 1:50 PM, Chris Nagele <nagele at wildbit.com> wrote:

> What kind of chassis are you looking at? I've tried the all SSD route
> a few times, but you really want to use SAS drives and the cost is not
> worth it. We ended up going with HGST (0B31069) for the log and cache,
> and then larger 4TB or 6TB drives using 3 way mirrors for data.
>
> Chris
>



-- 
"Nor aught availed him now
To have build in Heaven high towers    Nor did he scape
By all his engines
But was headlong sent with his industrious crew
To build in Hell
"Milton, "Paradise Lost"
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20160906/9277a762/attachment.html>

From robertfantini at gmail.com  Tue Sep  6 21:08:56 2016
From: robertfantini at gmail.com (Robert Fantini)
Date: Tue, 6 Sep 2016 17:08:56 -0400
Subject: [OmniOS-discuss] how to find which disk is ' target 1 lun0'
Message-ID: <CALnM7znHQfNsmLDxWVMbV=mBRHOwzP8tZdpL7y5NY1QWQw95qQ@mail.gmail.com>

Hello

Our system froze , console had  time out errors on  target 1  lun 0 .

I've check around on the system and on napp-it menus but could not find for
certain which disk is target 1 lun 0 .

Can someone tell me a way to do so?


best regards,

Robert Fantini
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20160906/927decc2/attachment-0001.html>

From gate03 at landcroft.co.uk  Wed Sep  7 11:15:28 2016
From: gate03 at landcroft.co.uk (Michael Mounteney)
Date: Wed, 7 Sep 2016 21:15:28 +1000
Subject: [OmniOS-discuss] PHP on LTS
Message-ID: <20160907211528.35bd852c@kendall>

OK, my secret shame:  I want to install PHP on r151014.  How ?  Latest
version available.

______________
Michael Mounteney

From jimklimov at cos.ru  Wed Sep  7 12:15:02 2016
From: jimklimov at cos.ru (Jim Klimov)
Date: Wed, 07 Sep 2016 14:15:02 +0200
Subject: [OmniOS-discuss] how to find which disk is ' target 1 lun0'
In-Reply-To: <CALnM7znHQfNsmLDxWVMbV=mBRHOwzP8tZdpL7y5NY1QWQw95qQ@mail.gmail.com>
References: <CALnM7znHQfNsmLDxWVMbV=mBRHOwzP8tZdpL7y5NY1QWQw95qQ@mail.gmail.com>
Message-ID: <03BEDB1C-A762-496C-90D8-72E04A6C54D6@cos.ru>

6 ???????? 2016??. 23:08:56 CEST, Robert Fantini <robertfantini at gmail.com> ?????:
>Hello
>
>Our system froze , console had  time out errors on  target 1  lun 0 .
>
>I've check around on the system and on napp-it menus but could not find
>for
>certain which disk is target 1 lun 0 .
>
>Can someone tell me a way to do so?
>
>
>best regards,
>
>Robert Fantini
>
>
>------------------------------------------------------------------------
>
>_______________________________________________
>OmniOS-discuss mailing list
>OmniOS-discuss at lists.omniti.com
>http://lists.omniti.com/mailman/listinfo/omnios-discuss

Possibly dmesg van glean some details (it sort of lists various na es the devices go by as it enumerates and attaches storage device nodes). Or 'zpool status' can say what's faulty.

Good luck,
Jim 
--
Typos courtesy of K-9 Mail on my Samsung Android

From jdg117 at elvis.arl.psu.edu  Thu Sep  8 15:17:13 2016
From: jdg117 at elvis.arl.psu.edu (John D Groenveld)
Date: Thu, 08 Sep 2016 11:17:13 -0400
Subject: [OmniOS-discuss] PHP on LTS
In-Reply-To: Your message of "Wed, 07 Sep 2016 21:15:28 +1000."
	<20160907211528.35bd852c@kendall> 
References: <20160907211528.35bd852c@kendall> 
Message-ID: <201609081517.u88FHDxt013817@elvis.arl.psu.edu>

In message <20160907211528.35bd852c at kendall>, Michael Mounteney writes:
>OK, my secret shame:  I want to install PHP on r151014.  How ?  Latest
>version available.

On r151018, PHP-7 builds from source:
# pkg install developer/gcc51 system/header gnu-make
$ env PATH=/usr/bin:/usr/sbin:/usr/ccs/bin:/usr/gnu/bin:/opt/gcc-5.1.0/bin:/opt/apache2/bin:/opt/postgres/bin \
CC=gcc CFLAGS="-m64 -std=gnu99" LDFLAGS=-m64 \
./configure --prefix=/opt/apache2/php-7 \
--with-apxs2=/opt/apache2/bin/apxs \
--with-pgsql=/opt/postgres

John
groenveld at acm.org

From pkam at bloom.pl  Mon Sep 12 09:41:46 2016
From: pkam at bloom.pl ([BLOOM] Piotr Kaminski)
Date: Mon, 12 Sep 2016 11:41:46 +0200
Subject: [OmniOS-discuss] otherwise stable system sudden crashing
In-Reply-To: <04B0A5C1206D824F8D310C5B3DB28CC9597B93@vEX01.mindstorm-internet.local>
References: <04B0A5C1206D824F8D310C5B3DB28CC9597B93@vEX01.mindstorm-internet.local>
Message-ID: <5debae5d-53de-a928-650e-c8e49544d3e4@bloom.pl>

Hi everybody

I've just subscribed to the list, although I have been using OmniOS for
a a few years.

I would like go back to the issue raised by Floris van Essen.
Some time ago I had the same issue. I upgraded Omnios from 151014 to
151016 and had the same issue - the host rebooted when accessed by Linux
client.
I reverted to 151014 because that is production file server. Rock solid
and stable since then.

Now, on completely different system - it is new install of OmniOS 151018
VM under ESXi 6, with passthrough LSI card, the system was solid while
serving NFS shares. And then I turned on some CIFS shares, one of them
is mounted on Ubuntu system. OmniOS has crashed twice in 2 hours!

I am desperate now. It is a new important server and it cannot crash.
In the whole internet I have found only one post from Floris with
similar looking issue.
Anyone able to help?
With regards

-- 
Piotr Kami?ski
PGP KeyID 0x804df875a899f9ac



From danmcd at omniti.com  Mon Sep 12 14:42:32 2016
From: danmcd at omniti.com (Dan McDonald)
Date: Mon, 12 Sep 2016 10:42:32 -0400
Subject: [OmniOS-discuss] otherwise stable system sudden crashing
In-Reply-To: <5debae5d-53de-a928-650e-c8e49544d3e4@bloom.pl>
References: <04B0A5C1206D824F8D310C5B3DB28CC9597B93@vEX01.mindstorm-internet.local>
	<5debae5d-53de-a928-650e-c8e49544d3e4@bloom.pl>
Message-ID: <325F9F2D-CE1A-4A4F-9B04-32846B05988B@omniti.com>


> On Sep 12, 2016, at 5:41 AM, [BLOOM] Piotr Kaminski <pkam at bloom.pl> wrote:
> 
> Now, on completely different system - it is new install of OmniOS 151018
> VM under ESXi 6, with passthrough LSI card, the system was solid while
> serving NFS shares. And then I turned on some CIFS shares, one of them
> is mounted on Ubuntu system. OmniOS has crashed twice in 2 hours!

You need to share core files.  "OmniOS crashes" is bad news, but without the corefiles, it's hard to know what is happening.

Dan


From skeltonr at btconnect.com  Mon Sep 12 17:23:07 2016
From: skeltonr at btconnect.com (Richard Skelton)
Date: Mon, 12 Sep 2016 18:23:07 +0100
Subject: [OmniOS-discuss] Zone network fails to work.
Message-ID: <57D6E47B.4020504@btconnect.com>

Hi,
I have a Sun X4170 with igb network.
I am using omnios-r151018-ae3141d
Here are my steps to create a zone called fred :-
dladm show-phys
LINK         MEDIA                STATE      SPEED  DUPLEX    DEVICE
usbecm0      Ethernet             down       10     full      usbecm0
igb2         Ethernet             unknown    0      half      igb2
igb0         Ethernet             up         1000   full      igb0
igb3         Ethernet             unknown    0      half      igb3
igb1         Ethernet             unknown    0      half      igb1


dladm create-vnic -l igb0 fred
zfs create -o mountpoint=/zones rpool/zones

/usr/sbin/zonecfg -z fred

create
set zonepath=/zones/fred
set autoboot=true
set limitpriv=default,dtrace_proc,dtrace_user
set ip-type=exclusive
add net
set physical=fred
end
verify
commit
exitz
oneadm -z fred install
zoneadm -z fred boot
zlogin -e+ -C fred

dladm show-vnic
LINK         OVER         SPEED  MACADDRESS        MACADDRTYPE         VID
fred      ?            1000   2:8:20:f4:52:49   random              0

ipadm create-if fred

ipadm create-addr -T static -a 192.168.240.214/24 fred/v4static
route -p add default 192.168.240.254

ping 192.168.240.214
192.168.240.214 is alive
ping 192.168.240.254
no answer from 192.168.240.254


ifconfig  
lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu
8232 index 1
    inet 127.0.0.1 netmask ff000000
fred: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2
    inet 192.168.240.214 netmask ffffff00 broadcast 192.168.240.255
    ether 2:8:20:f4:52:49
lo0: flags=2002000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv6,VIRTUAL> mtu
8252 index 1
    inet6 ::1/128
fred: flags=20002000840<RUNNING,MULTICAST,IPv6> mtu 1500 index 2
    inet6 ::/0
    ether 2:8:20:f4:52:49


What am I missing :-(

Cheers
Richard

From danmcd at omniti.com  Mon Sep 12 17:36:30 2016
From: danmcd at omniti.com (Dan McDonald)
Date: Mon, 12 Sep 2016 13:36:30 -0400
Subject: [OmniOS-discuss] Zone network fails to work.
In-Reply-To: <57D6E47B.4020504@btconnect.com>
References: <57D6E47B.4020504@btconnect.com>
Message-ID: <0AFBE5AF-7824-45CF-B78F-442E6E21C57B@omniti.com>

vNICS MUST end in a number.  That's a requirement.  "fred0" or "fred23" is fine, but "fred" breaks things.

Dan


From danmcd at omniti.com  Mon Sep 12 19:05:14 2016
From: danmcd at omniti.com (Dan McDonald)
Date: Mon, 12 Sep 2016 15:05:14 -0400
Subject: [OmniOS-discuss] otherwise stable system sudden crashing
In-Reply-To: <db3713d5-e68d-266b-0e86-bfd7f858941e@bloom.pl>
References: <04B0A5C1206D824F8D310C5B3DB28CC9597B93@vEX01.mindstorm-internet.local>
	<5debae5d-53de-a928-650e-c8e49544d3e4@bloom.pl>
	<325F9F2D-CE1A-4A4F-9B04-32846B05988B@omniti.com>
	<CAK9SPSg7S=jyYb1TfTMMr74-goNSs8g7U74m2Bpb=uRxvUzgow@mail.gmail.com>
	<CBC44860-EDCF-4AAE-8A41-2E8FED2F0291@omniti.com>
	<db3713d5-e68d-266b-0e86-bfd7f858941e@bloom.pl>
Message-ID: <63B18CF9-C802-4A30-8B26-B1DE7A155B31@omniti.com>

I took a look at your dumps.

Both vmdump.0 and vmdump1 have the same stack:

vpanic()
0xfffffffffba90de8()
zfs_acl_ids_create+0x4d2(ffffff0799931ab0, 2, ffffff00194bdcb0, ffffff055d97ddb0
, 0, ffffff00194bdbf0)
zfs_make_xattrdir+0x96(ffffff0799931ab0, ffffff00194bdcb0, ffffff00194be018, 
ffffff055d97ddb0)
zfs_get_xattrdir+0x103(ffffff0799931ab0, ffffff00194be018, ffffff055d97ddb0, e)
zfs_lookup+0x1b6(ffffff056969d740, fffffffffbb78cd0, ffffff00194be018, 
ffffff00194bdf30, e, ffffff055db38740)
fop_lookup+0xa2(ffffff056969d740, fffffffffbb78cd0, ffffff00194be018, 
ffffff00194bdf30, e, ffffff055db38740)
xattr_dir_realdir+0xb3(ffffff079edb1880, ffffff00194be018, 6, ffffff055d97ddb0, 
0)
xattr_lookup_cb+0x65(ffffff079edb1880, ffffff05da046a00, ffffff00194be1b0, 
ffffff00194be0b8, ffffff055d97ddb0, 80000)
gfs_dir_lookup_dynamic+0x7c(fffffffffbb5d3d0, ffffff0563cea1f8, ffffff05da046a00
, ffffff079edb1880, ffffff00194be1b0, ffffff055d97ddb0)
gfs_dir_lookup+0x18c(ffffff079edb1880, ffffff05da046a00, ffffff00194be608, 
ffffff055d97ddb0, 80000, ffffff00194be55c)
gfs_vop_lookup+0x35(ffffff079edb1880, ffffff05da046a00, ffffff00194be608, 0, 
80000, 0)
fop_lookup+0xa2(ffffff079edb1880, ffffff05da046a00, ffffff00194be608, 0, 80000, 
0)

Do you know exactly what you did to reproduce this panic?




After r151018 was released, there were at least FOUR changes near the code that's panicking on you:

6762 POSIX write should imply DELETE_CHILD on directories - and some additional considerations
6763 aclinherit=restricted masks inherited permissions by group perms (groupmask)
6764 zfs issues with inheritance flags during chmod(2) with aclmode=passthrough
6765 zfs_zaccess_delete() comments do not accurately reflect delete permissions for ACLs

Unfortunately, none of those bugs mention a kernel panic.


Are you willing to try OmniOS bloody (with DEBUG enabled) on a be on your system?  Or at least are you willing to run a DEBUG kernel on your r151018 environment?  Either one of these would be most helpful.

Thanks,
Dan


From skeltonr at btconnect.com  Mon Sep 12 20:35:33 2016
From: skeltonr at btconnect.com (Richard Skelton)
Date: Mon, 12 Sep 2016 21:35:33 +0100
Subject: [OmniOS-discuss] Zone network fails to work.
In-Reply-To: <57D6E47B.4020504@btconnect.com>
References: <57D6E47B.4020504@btconnect.com>
Message-ID: <57D71195.7000601@btconnect.com>

Sorry in my haste I posted fred not fred5
See corrections below :-)

Richard Skelton wrote:
> Hi,
> I have a Sun X4170 with igb network.
> I am using omnios-r151018-ae3141d
> Here are my steps to create a zone called fred5 :-
> dladm show-phys
> LINK         MEDIA                STATE      SPEED  DUPLEX    DEVICE
> usbecm0      Ethernet             down       10     full      usbecm0
> igb2         Ethernet             unknown    0      half      igb2
> igb0         Ethernet             up         1000   full      igb0
> igb3         Ethernet             unknown    0      half      igb3
> igb1         Ethernet             unknown    0      half      igb1
>
>
> dladm create-vnic -l igb0 fred5
> zfs create -o mountpoint=/zones rpool/zones
>
> /usr/sbin/zonecfg -z fred5
>
> create
> set zonepath=/zones/fred5
> set autoboot=true
> set limitpriv=default,dtrace_proc,dtrace_user
> set ip-type=exclusive
> add net
> set physical=fred5
> end
> verify
> commit
> exitz
> oneadm -z fred5 install
> zoneadm -z fred5 boot
> zlogin -e+ -C fred5
>
> dladm show-vnic
> LINK         OVER         SPEED  MACADDRESS        MACADDRTYPE         VID
> fred5      ?            1000   2:8:20:f4:52:49   random              0
>
> ipadm create-if fred5
>
> ipadm create-addr -T static -a 192.168.240.214/24 fred5/v4static
> route -p add default 192.168.240.254
>
> ping 192.168.240.214
> 192.168.240.214 is alive
> ping 192.168.240.254
> no answer from 192.168.240.254
>
>
> ifconfig  
> lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu
> 8232 index 1
>     inet 127.0.0.1 netmask ff000000
> fred5: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2
>     inet 192.168.240.214 netmask ffffff00 broadcast 192.168.240.255
>     ether 2:8:20:f4:52:49
> lo0: flags=2002000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv6,VIRTUAL> mtu
> 8252 index 1
>     inet6 ::1/128
> fred5: flags=20002000840<RUNNING,MULTICAST,IPv6> mtu 1500 index 2
>     inet6 ::/0
>     ether 2:8:20:f4:52:49
>
>
> What am I missing :-(
>
> Cheers
> Richard
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>   

From danmcd at omniti.com  Wed Sep 14 15:52:50 2016
From: danmcd at omniti.com (Dan McDonald)
Date: Wed, 14 Sep 2016 11:52:50 -0400
Subject: [OmniOS-discuss] CVE-2016-7167 in curl now fixed
Message-ID: <881257C1-9D69-42BD-99EC-4D65DDBEBB5F@omniti.com>

All supported repos (r15101[468], bloody) now have updated cURL packages with a patch for the aforementioned CVE.

Please update at your convenience.  A reboot is not required, but some services may need to be restarted that don't have proper SMF restart dependencies in them.

Dan


From an3s.annema at gmail.com  Thu Sep 15 18:15:05 2016
From: an3s.annema at gmail.com (Andries Annema)
Date: Thu, 15 Sep 2016 20:15:05 +0200
Subject: [OmniOS-discuss] Zone network fails to work.
In-Reply-To: <57D71195.7000601@btconnect.com>
References: <57D6E47B.4020504@btconnect.com> <57D71195.7000601@btconnect.com>
Message-ID: <74892e16-b91d-83f3-a000-cf422ab815b7@gmail.com>

Hi Richard,

About two years ago I bumped into the same, or at least a similar, 
problem on my brand new r151014 rig: the system was able to ping hosts 
on the same network segment, but a ping to my router/gateway resulted in 
"no answer from <router> / Unknown host".

In my notes from back then I find that I ended up in verifying the arp 
cache by entering:

# arp -a

The resulting list did not show the router, so I added it manually:

# arp -s <router IP> <router MAC>

A page that might shed some more light on the matter: 
https://www.petri.com/csc_arp_cache

I have no clue as to why a single host on the local network would not 
show up in that arp table while others would, but back then the above 
solved my problem. Haven't had the issue since then, so it might be 
worth giving it a try.
If it doesn't help you, then I hope someone who really knows the ins and 
outs of networking is listening in. :-)

Regards,
Andries


On 2016-09-12 22:35, Richard Skelton wrote:
> Sorry in my haste I posted fred not fred5
> See corrections below :-)
>
> Richard Skelton wrote:
>> Hi,
>> I have a Sun X4170 with igb network.
>> I am using omnios-r151018-ae3141d
>> Here are my steps to create a zone called fred5 :-
>> dladm show-phys
>> LINK         MEDIA                STATE      SPEED  DUPLEX    DEVICE
>> usbecm0      Ethernet             down       10     full      usbecm0
>> igb2         Ethernet             unknown    0      half      igb2
>> igb0         Ethernet             up         1000   full      igb0
>> igb3         Ethernet             unknown    0      half      igb3
>> igb1         Ethernet             unknown    0      half      igb1
>>
>>
>> dladm create-vnic -l igb0 fred5
>> zfs create -o mountpoint=/zones rpool/zones
>>
>> /usr/sbin/zonecfg -z fred5
>>
>> create
>> set zonepath=/zones/fred5
>> set autoboot=true
>> set limitpriv=default,dtrace_proc,dtrace_user
>> set ip-type=exclusive
>> add net
>> set physical=fred5
>> end
>> verify
>> commit
>> exitz
>> oneadm -z fred5 install
>> zoneadm -z fred5 boot
>> zlogin -e+ -C fred5
>>
>> dladm show-vnic
>> LINK         OVER         SPEED  MACADDRESS        MACADDRTYPE         VID
>> fred5      ?            1000   2:8:20:f4:52:49   random              0
>>
>> ipadm create-if fred5
>>
>> ipadm create-addr -T static -a 192.168.240.214/24 fred5/v4static
>> route -p add default 192.168.240.254
>>
>> ping 192.168.240.214
>> 192.168.240.214 is alive
>> ping 192.168.240.254
>> no answer from 192.168.240.254
>>
>>
>> ifconfig
>> lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu
>> 8232 index 1
>>      inet 127.0.0.1 netmask ff000000
>> fred5: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2
>>      inet 192.168.240.214 netmask ffffff00 broadcast 192.168.240.255
>>      ether 2:8:20:f4:52:49
>> lo0: flags=2002000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv6,VIRTUAL> mtu
>> 8252 index 1
>>      inet6 ::1/128
>> fred5: flags=20002000840<RUNNING,MULTICAST,IPv6> mtu 1500 index 2
>>      inet6 ::/0
>>      ether 2:8:20:f4:52:49
>>
>>
>> What am I missing :-(
>>
>> Cheers
>> Richard
>> _______________________________________________
>> OmniOS-discuss mailing list
>> OmniOS-discuss at lists.omniti.com
>> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>>    
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss


From ergi.thanasko at avsquad.com  Fri Sep 16 17:43:30 2016
From: ergi.thanasko at avsquad.com (Ergi Thanasko)
Date: Fri, 16 Sep 2016 17:43:30 +0000
Subject: [OmniOS-discuss] Network throughout 1GB/sec
Message-ID: <56B30910-D0CB-4C7A-9848-313358007C6E@avsquad.com>

Hi all,
We have a a few servers  conected via 10g nic  LACP, some of them have  4nic and some have 6nic in a link aggregation mode. We been moving a lot of data around and we are trying to get the maximum performance. I have seen zpool can deliver  2-3GB accumulated  throughput. Iperf does about 600-800MB/sec between those two servers. 
Given  the hardware that we have and the zpool performance,   we expected   to see some serious data transfer rates  however we only see around 200-300MB/sec average  using rsync or copy paste over NFS.  Standard MTU 1500 and nfs block size.  I want to ask the community what to do get some higher throughout and the application level. I hear ZFS send/receive   or ZFS shadow does work faster but it does snapshots. Out data (Terabytes) is constantly evolving   and we prefer something in the nature of rsync  but to utilize the network hardware. 

If Anyone has a hardware setup that can see 1GB/sec  throughput  and does not mind sharing? 
Any software  that  use multithreads  sessions to move data around  zfs friendly? We do not mind getting going with a commercial solution like camvault or veeam if they work.

Thank you for your time 

 



From bfriesen at simple.dallas.tx.us  Fri Sep 16 19:33:58 2016
From: bfriesen at simple.dallas.tx.us (Bob Friesenhahn)
Date: Fri, 16 Sep 2016 14:33:58 -0500 (CDT)
Subject: [OmniOS-discuss] Network throughout 1GB/sec
In-Reply-To: <56B30910-D0CB-4C7A-9848-313358007C6E@avsquad.com>
References: <56B30910-D0CB-4C7A-9848-313358007C6E@avsquad.com>
Message-ID: <alpine.GSO.2.20.1609161430080.28439@freddy.simplesystems.org>

On Fri, 16 Sep 2016, Ergi Thanasko wrote:

> Given the hardware that we have and the zpool performance, we 
> expected to see some serious data transfer rates however we only see 
> around 200-300MB/sec average using rsync or copy paste over NFS. 
> Standard MTU 1500 and nfs block size.  I want to ask the community

Are these read rates or write rates?  Read rates should be able to 
come close to pool or wire limits.  Write rates over NFS are primarily 
dominated by latency on a per-writer basis.

Increasing the MTU to 9k has been shown to improve throughput quite a 
lot for large transfers.

Bob
-- 
Bob Friesenhahn
bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/

From ergi.thanasko at avsquad.com  Fri Sep 16 19:38:32 2016
From: ergi.thanasko at avsquad.com (Ergi Thanasko)
Date: Fri, 16 Sep 2016 19:38:32 +0000
Subject: [OmniOS-discuss] Network throughout 1GB/sec
In-Reply-To: <alpine.GSO.2.20.1609161430080.28439@freddy.simplesystems.org>
References: <56B30910-D0CB-4C7A-9848-313358007C6E@avsquad.com>
	<alpine.GSO.2.20.1609161430080.28439@freddy.simplesystems.org>
Message-ID: <6A6FC908-63A2-476D-AA93-0EFC6281D8C6@avsquad.com>

Hi Bob,
We were testing it between two similar servers   rsynic and copy paste  both ways ( read/write) was the same around 300MB/sec average. Of course the speed test on the pools provide  higher throughput  around 600MB/sec
[cid:449A210A-451D-40E1-97EA-98BA5E7FA558 at fios-router.home]

On Sep 16, 2016, at 12:33 PM, Bob Friesenhahn <bfriesen at simple.dallas.tx.us<mailto:bfriesen at simple.dallas.tx.us>> wrote:

On Fri, 16 Sep 2016, Ergi Thanasko wrote:

Given the hardware that we have and the zpool performance, we expected to see some serious data transfer rates however we only see around 200-300MB/sec average using rsync or copy paste over NFS. Standard MTU 1500 and nfs block size.  I want to ask the community

Are these read rates or write rates?  Read rates should be able to come close to pool or wire limits.  Write rates over NFS are primarily dominated by latency on a per-writer basis.

Increasing the MTU to 9k has been shown to improve throughput quite a lot for large transfers.

Bob
--
Bob Friesenhahn
bfriesen at simple.dallas.tx.us<mailto:bfriesen at simple.dallas.tx.us>, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20160916/9e3a611e/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ErgiThanasko.png
Type: image/png
Size: 7910 bytes
Desc: ErgiThanasko.png
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20160916/9e3a611e/attachment-0001.png>

From mtalbott at lji.org  Fri Sep 16 19:53:42 2016
From: mtalbott at lji.org (Michael Talbott)
Date: Fri, 16 Sep 2016 12:53:42 -0700
Subject: [OmniOS-discuss] Network throughout 1GB/sec
In-Reply-To: <6A6FC908-63A2-476D-AA93-0EFC6281D8C6@avsquad.com>
References: <56B30910-D0CB-4C7A-9848-313358007C6E@avsquad.com>
	<alpine.GSO.2.20.1609161430080.28439@freddy.simplesystems.org>
	<6A6FC908-63A2-476D-AA93-0EFC6281D8C6@avsquad.com>
Message-ID: <BFF3FA33-43D9-4E64-A796-C7B151B2E706@lji.org>

Jumbo frames are a major help. Also, try using multiple streams (break single rsync job into multiple jobs). Also be sure to use rsync native protocol and don't tunnel it over ssh. Then there's bbcp which can push a single copy operation into multiple streams to fully saturate your disks/network ;) https://www.slac.stanford.edu/~abh/bbcp/


Michael


> On Sep 16, 2016, at 12:38 PM, Ergi Thanasko <ergi.thanasko at avsquad.com> wrote:
> 
> Hi Bob,
> We were testing it between two similar servers   rsynic and copy paste  both ways ( read/write) was the same around 300MB/sec average. Of course the speed test on the pools provide  higher throughput  around 600MB/sec  
> <ErgiThanasko.png>
> 
>> On Sep 16, 2016, at 12:33 PM, Bob Friesenhahn <bfriesen at simple.dallas.tx.us <mailto:bfriesen at simple.dallas.tx.us>> wrote:
>> 
>> On Fri, 16 Sep 2016, Ergi Thanasko wrote:
>> 
>>> Given the hardware that we have and the zpool performance, we expected to see some serious data transfer rates however we only see around 200-300MB/sec average using rsync or copy paste over NFS. Standard MTU 1500 and nfs block size.  I want to ask the community
>> 
>> Are these read rates or write rates?  Read rates should be able to come close to pool or wire limits.  Write rates over NFS are primarily dominated by latency on a per-writer basis.
>> 
>> Increasing the MTU to 9k has been shown to improve throughput quite a lot for large transfers.
>> 
>> Bob
>> -- 
>> Bob Friesenhahn
>> bfriesen at simple.dallas.tx.us <mailto:bfriesen at simple.dallas.tx.us>, http://www.simplesystems.org/users/bfriesen/ <http://www.simplesystems.org/users/bfriesen/>
>> GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/ <http://www.graphicsmagick.org/>
> 
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20160916/750c3c26/attachment.html>

From alka at hfg-gmuend.de  Fri Sep 16 19:57:53 2016
From: alka at hfg-gmuend.de (Guenther Alka)
Date: Fri, 16 Sep 2016 21:57:53 +0200
Subject: [OmniOS-discuss] Network throughout 1GB/sec
In-Reply-To: <56B30910-D0CB-4C7A-9848-313358007C6E@avsquad.com>
References: <56B30910-D0CB-4C7A-9848-313358007C6E@avsquad.com>
Message-ID: <3419113d-b3ba-7ac7-30cb-945f6d2192ff@hfg-gmuend.de>

I have made some investigations into 10G and found that 300-400MB/s is 
expected with default settings. Improvements are possible up to 1000MB/s 
via mtu 9000 and if you increase ip buffers ex
max_buf=4097152 tcp
send_buf=2048576 tcp
recv_buf=2048576 tcp,

NFS lockd servers (ex 1024), NFS number of threads (ex 64) and NFS 
transfer size (ex 1048576)

http://napp-it.org/doc/downloads/performance_smb2.pdf


Gea

Am 16.09.2016 um 19:43 schrieb Ergi Thanasko:
> Hi all,
> We have a a few servers  conected via 10g nic  LACP, some of them have  4nic and some have 6nic in a link aggregation mode. We been moving a lot of data around and we are trying to get the maximum performance. I have seen zpool can deliver  2-3GB accumulated  throughput. Iperf does about 600-800MB/sec between those two servers.
> Given  the hardware that we have and the zpool performance,   we expected   to see some serious data transfer rates  however we only see around 200-300MB/sec average  using rsync or copy paste over NFS.  Standard MTU 1500 and nfs block size.  I want to ask the community what to do get some higher throughout and the application level. I hear ZFS send/receive   or ZFS shadow does work faster but it does snapshots. Out data (Terabytes) is constantly evolving   and we prefer something in the nature of rsync  but to utilize the network hardware.
>
> If Anyone has a hardware setup that can see 1GB/sec  throughput  and does not mind sharing?
> Any software  that  use multithreads  sessions to move data around  zfs friendly? We do not mind getting going with a commercial solution like camvault or veeam if they work.
>
> Thank you for your time
>
>   
>
>
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss


From bfriesen at simple.dallas.tx.us  Fri Sep 16 20:23:46 2016
From: bfriesen at simple.dallas.tx.us (Bob Friesenhahn)
Date: Fri, 16 Sep 2016 15:23:46 -0500 (CDT)
Subject: [OmniOS-discuss] Network throughout 1GB/sec
In-Reply-To: <6A6FC908-63A2-476D-AA93-0EFC6281D8C6@avsquad.com>
References: <56B30910-D0CB-4C7A-9848-313358007C6E@avsquad.com>
	<alpine.GSO.2.20.1609161430080.28439@freddy.simplesystems.org>
	<6A6FC908-63A2-476D-AA93-0EFC6281D8C6@avsquad.com>
Message-ID: <alpine.GSO.2.20.1609161518430.28439@freddy.simplesystems.org>

On Fri, 16 Sep 2016, Ergi Thanasko wrote:

> We were testing it between two similar servers rsynic and copy paste 
> both ways ( read/write) was the same around 300MB/sec average. Of 
> course the speed test on the pools provide higher throughput around 
> 600MB/sec

I am not sure what 'copy paste' means, but rsync can be used in a 
couple of different ways.  One way is that rsync is writing over NFS 
to the server.  Another way is that rsync communicates via some means 
(ssh, netcat) to another rsync running on the server.  Rsync must 
always query/check to see if/what data is existing while it does the 
copy.

It is useful to see that if you do two copies at the same time if the 
combined rate is much more than 300MB/sec average.

Bob
-- 
Bob Friesenhahn
bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/

From davide.poletto at gmail.com  Fri Sep 16 22:40:04 2016
From: davide.poletto at gmail.com (Davide Poletto)
Date: Sat, 17 Sep 2016 00:40:04 +0200
Subject: [OmniOS-discuss] Network throughout 1GB/sec
In-Reply-To: <56B30910-D0CB-4C7A-9848-313358007C6E@avsquad.com>
References: <56B30910-D0CB-4C7A-9848-313358007C6E@avsquad.com>
Message-ID: <CANKMAMZ7ffQ+F1ZwoQMwTFVHdEJ8GRR=G0kt178iEvNBPiaBNA@mail.gmail.com>

Hope not be wrong here but Port Trunking usage deserves its part in the
whole picture too: be aware that using Port Trunking (with LACP as per IEEE
802.3ad) between your Servers' NIC and your 10Gb Switching infrastructure -
and this happens by aggregating "n" identical ports together on both link's
ends, as you wrote - doesn't consequently mean that your
"one-Host-to-one-Host" generated traffic will be able to use and will be
able to saturate all those "n" 10Gb based links concurrently...

On Sep 16, 2016 7:45 PM, "Ergi Thanasko" <ergi.thanasko at avsquad.com> wrote:

> Hi all,
> We have a a few servers  conected via 10g nic  LACP, some of them have
> 4nic and some have 6nic in a link aggregation mode. We been moving a lot of
> data around and we are trying to get the maximum performance. I have seen
> zpool can deliver  2-3GB accumulated  throughput. Iperf does about
> 600-800MB/sec between those two servers.
> Given  the hardware that we have and the zpool performance,   we expected
>  to see some serious data transfer rates  however we only see around
> 200-300MB/sec average  using rsync or copy paste over NFS.  Standard MTU
> 1500 and nfs block size.  I want to ask the community what to do get some
> higher throughout and the application level. I hear ZFS send/receive   or
> ZFS shadow does work faster but it does snapshots. Out data (Terabytes) is
> constantly evolving   and we prefer something in the nature of rsync  but
> to utilize the network hardware.
>
> If Anyone has a hardware setup that can see 1GB/sec  throughput  and does
> not mind sharing?
> Any software  that  use multithreads  sessions to move data around  zfs
> friendly? We do not mind getting going with a commercial solution like
> camvault or veeam if they work.
>
> Thank you for your time
>
>
>
>
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20160917/07ad7d9c/attachment.html>

From daleg at omniti.com  Sat Sep 17 00:24:39 2016
From: daleg at omniti.com (Dale Ghent)
Date: Fri, 16 Sep 2016 20:24:39 -0400
Subject: [OmniOS-discuss] Network throughout 1GB/sec
In-Reply-To: <56B30910-D0CB-4C7A-9848-313358007C6E@avsquad.com>
References: <56B30910-D0CB-4C7A-9848-313358007C6E@avsquad.com>
Message-ID: <86D8AFD6-91E5-4AA4-A6CF-6285665BC257@omniti.com>

Are you doing the rsync over ssh? You might want to look into using HPN-SSH:

https://www.psc.edu/index.php/hpn-ssh

/dale

> On Sep 16, 2016, at 1:43 PM, Ergi Thanasko <ergi.thanasko at avsquad.com> wrote:
> 
> Hi all,
> We have a a few servers  conected via 10g nic  LACP, some of them have  4nic and some have 6nic in a link aggregation mode. We been moving a lot of data around and we are trying to get the maximum performance. I have seen zpool can deliver  2-3GB accumulated  throughput. Iperf does about 600-800MB/sec between those two servers. 
> Given  the hardware that we have and the zpool performance,   we expected   to see some serious data transfer rates  however we only see around 200-300MB/sec average  using rsync or copy paste over NFS.  Standard MTU 1500 and nfs block size.  I want to ask the community what to do get some higher throughout and the application level. I hear ZFS send/receive   or ZFS shadow does work faster but it does snapshots. Out data (Terabytes) is constantly evolving   and we prefer something in the nature of rsync  but to utilize the network hardware. 
> 
> If Anyone has a hardware setup that can see 1GB/sec  throughput  and does not mind sharing? 
> Any software  that  use multithreads  sessions to move data around  zfs friendly? We do not mind getting going with a commercial solution like camvault or veeam if they work.
> 
> Thank you for your time 
> 
> 
> 
> 
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss


From ergi.thanasko at avsquad.com  Sat Sep 17 00:37:12 2016
From: ergi.thanasko at avsquad.com (Ergi Thanasko)
Date: Sat, 17 Sep 2016 00:37:12 +0000
Subject: [OmniOS-discuss] Network throughout 1GB/sec
In-Reply-To: <86D8AFD6-91E5-4AA4-A6CF-6285665BC257@omniti.com>
References: <56B30910-D0CB-4C7A-9848-313358007C6E@avsquad.com>
	<86D8AFD6-91E5-4AA4-A6CF-6285665BC257@omniti.com>
Message-ID: <580E52D0-B98D-4836-A5B4-9AF7ED8FDD47@avsquad.com>

no that is even slower, just rsync over mounted  nfs, a multithreaded rsync does work have better performance.

[cid:449A210A-451D-40E1-97EA-98BA5E7FA558 at fios-router.home]

On Sep 16, 2016, at 5:24 PM, Dale Ghent <daleg at omniti.com<mailto:daleg at omniti.com>> wrote:

Are you doing the rsync over ssh? You might want to look into using HPN-SSH:

https://www.psc.edu/index.php/hpn-ssh

/dale

On Sep 16, 2016, at 1:43 PM, Ergi Thanasko <ergi.thanasko at avsquad.com> wrote:

Hi all,
We have a a few servers  conected via 10g nic  LACP, some of them have  4nic and some have 6nic in a link aggregation mode. We been moving a lot of data around and we are trying to get the maximum performance. I have seen zpool can deliver  2-3GB accumulated  throughput. Iperf does about 600-800MB/sec between those two servers.
Given  the hardware that we have and the zpool performance,   we expected   to see some serious data transfer rates  however we only see around 200-300MB/sec average  using rsync or copy paste over NFS.  Standard MTU 1500 and nfs block size.  I want to ask the community what to do get some higher throughout and the application level. I hear ZFS send/receive   or ZFS shadow does work faster but it does snapshots. Out data (Terabytes) is constantly evolving   and we prefer something in the nature of rsync  but to utilize the network hardware.

If Anyone has a hardware setup that can see 1GB/sec  throughput  and does not mind sharing?
Any software  that  use multithreads  sessions to move data around  zfs friendly? We do not mind getting going with a commercial solution like camvault or veeam if they work.

Thank you for your time




_______________________________________________
OmniOS-discuss mailing list
OmniOS-discuss at lists.omniti.com
http://lists.omniti.com/mailman/listinfo/omnios-discuss


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20160917/bdb9faa4/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ErgiThanasko.png
Type: image/png
Size: 7910 bytes
Desc: ErgiThanasko.png
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20160917/bdb9faa4/attachment-0001.png>

From ergi.thanasko at avsquad.com  Sat Sep 17 00:57:44 2016
From: ergi.thanasko at avsquad.com (Ergi Thanasko)
Date: Sat, 17 Sep 2016 00:57:44 +0000
Subject: [OmniOS-discuss] Network throughout 1GB/sec
In-Reply-To: <3419113d-b3ba-7ac7-30cb-945f6d2192ff@hfg-gmuend.de>
References: <56B30910-D0CB-4C7A-9848-313358007C6E@avsquad.com>
	<3419113d-b3ba-7ac7-30cb-945f6d2192ff@hfg-gmuend.de>
Message-ID: <2B866604-28CC-4629-A5D8-BE5DF5EFD061@avsquad.com>

HI Gea,
Great info, are you seeing 1000MB/s doing iperf or actually transfer rates  rsync, cp bbcp?.


> On Sep 16, 2016, at 12:57 PM, Guenther Alka <alka at hfg-gmuend.de> wrote:
> 
> I have made some investigations into 10G and found that 300-400MB/s is expected with default settings. Improvements are possible up to 1000MB/s via mtu 9000 and if you increase ip buffers ex
> max_buf=4097152 tcp
> send_buf=2048576 tcp
> recv_buf=2048576 tcp,
> 
> NFS lockd servers (ex 1024), NFS number of threads (ex 64) and NFS transfer size (ex 1048576)
> 
> http://napp-it.org/doc/downloads/performance_smb2.pdf
> 
> 
> Gea
> 
> Am 16.09.2016 um 19:43 schrieb Ergi Thanasko:
>> Hi all,
>> We have a a few servers  conected via 10g nic  LACP, some of them have  4nic and some have 6nic in a link aggregation mode. We been moving a lot of data around and we are trying to get the maximum performance. I have seen zpool can deliver  2-3GB accumulated  throughput. Iperf does about 600-800MB/sec between those two servers.
>> Given  the hardware that we have and the zpool performance,   we expected   to see some serious data transfer rates  however we only see around 200-300MB/sec average  using rsync or copy paste over NFS.  Standard MTU 1500 and nfs block size.  I want to ask the community what to do get some higher throughout and the application level. I hear ZFS send/receive   or ZFS shadow does work faster but it does snapshots. Out data (Terabytes) is constantly evolving   and we prefer something in the nature of rsync  but to utilize the network hardware.
>> 
>> If Anyone has a hardware setup that can see 1GB/sec  throughput  and does not mind sharing?
>> Any software  that  use multithreads  sessions to move data around  zfs friendly? We do not mind getting going with a commercial solution like camvault or veeam if they work.
>> 
>> Thank you for your time
>> 
>>  
>> 
>> _______________________________________________
>> OmniOS-discuss mailing list
>> OmniOS-discuss at lists.omniti.com
>> http://lists.omniti.com/mailman/listinfo/omnios-discuss
> 
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss


From stephan.budach at jvm.de  Sat Sep 17 05:47:18 2016
From: stephan.budach at jvm.de (Stephan Budach)
Date: Sat, 17 Sep 2016 07:47:18 +0200
Subject: [OmniOS-discuss] Network throughout 1GB/sec
In-Reply-To: <CANKMAMZ7ffQ+F1ZwoQMwTFVHdEJ8GRR=G0kt178iEvNBPiaBNA@mail.gmail.com>
References: <56B30910-D0CB-4C7A-9848-313358007C6E@avsquad.com>
	<CANKMAMZ7ffQ+F1ZwoQMwTFVHdEJ8GRR=G0kt178iEvNBPiaBNA@mail.gmail.com>
Message-ID: <b8075f5d-137c-42e4-4577-b3721b863467@jvm.de>

Am 17.09.16 um 00:40 schrieb Davide Poletto:
>
> Hope not be wrong here but Port Trunking usage deserves its part in 
> the whole picture too: be aware that using Port Trunking (with LACP as 
> per IEEE 802.3ad) between your Servers' NIC and your 10Gb Switching 
> infrastructure - and this happens by aggregating "n" identical ports 
> together on both link's ends, as you wrote - doesn't consequently mean 
> that your "one-Host-to-one-Host" generated traffic will be able to use 
> and will be able to saturate all those "n" 10Gb based links 
> concurrently...
>
>

+1

It's a common misunderstanding that bundling a couple of ports together 
with LACP results in bigger throughput, as one connection will always 
only deliver it's maximum physical bandwidth. There are bonding modes, 
which support that, but you'd have to be able to configure this end to 
end, with some round-robin settings, as of on wich NIC the next packet 
will be stuffed into the "pipe". Regular 802.3ad will not give you that, 
as it has been designed for redundancy, not performance.

So, LACP gives you mutiple read streams up to the number of NICs 
involved in the bundle, but writes from the host will always be on one 
NIC for any given connection. The classic algorithm is based on the 
source and target MAC addresses. So you will not be able to get 2 to 3 
GB/s over a 10GbE LACP bundle.

Cheers,
Stephan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20160917/909f03bd/attachment.html>

From alka at hfg-gmuend.de  Sat Sep 17 08:50:54 2016
From: alka at hfg-gmuend.de (Guenther Alka)
Date: Sat, 17 Sep 2016 10:50:54 +0200
Subject: [OmniOS-discuss] Network throughout 1GB/sec
In-Reply-To: <2B866604-28CC-4629-A5D8-BE5DF5EFD061@avsquad.com>
References: <56B30910-D0CB-4C7A-9848-313358007C6E@avsquad.com>
	<3419113d-b3ba-7ac7-30cb-945f6d2192ff@hfg-gmuend.de>
	<2B866604-28CC-4629-A5D8-BE5DF5EFD061@avsquad.com>
Message-ID: <7e30147c-be84-5b6c-272e-56431b892f48@hfg-gmuend.de>

hi
Intention of this test and tuning cycle was to check 4k video editing 
capability
from OSX and Windows to a Solaris or OmniOS storage over 10G/40G to SSD 
or NVMe storage.

With tunings and large 4k videofiles I was able to get about 900MB/s on 
write with peaks up to 1000 MB/s over SMB 2.1 tested with the video 
editing tool AJA on Windows and speed test on OSX. These are more or 
less sequential tests with a lot of large files. Values on Solaris were 
slightly better than on OmniOS so I asume OS or driver defaults are more 
optimized there at least regarding 10G/40G. Reads were always slower and 
more critical to settings and cablings. NFS values on OSX were not 
nearly as good and quite disappointing (at least on OSX).

With smaller NTSC/PAL video settings (test uses many small files then) 
performance went down to 500-600 MB/s with writes and a little lower on 
reads.

I currently do some tests with i40e on Intel XL710 where the difference 
is heavy with 2200MB write on Solaris and 1500 MB/s on OmniOS with same 
settings while reads are currently a disaster at least on Windows with 
up to 300 MB/s on Solaris and 150MB/s on OmniOS.

In best cases these values were nearly as good as iperf values so near 
wire speed.

I would not asume that you can reach high performance values with rsync. 
Zfs send should be faster as it creates a filestream. I woukd not expect 
that they can come close to the above even with zfs send over mbuffer or 
netcat. A pure copy should be as fast from/to OSX orWindows or with cp 
over netcat.

Gea


Am 17.09.2016 um 02:57 schrieb Ergi Thanasko:
> HI Gea,
> Great info, are you seeing 1000MB/s doing iperf or actually transfer rates  rsync, cp bbcp?.
>
>
>> On Sep 16, 2016, at 12:57 PM, Guenther Alka <alka at hfg-gmuend.de> wrote:
>>
>> I have made some investigations into 10G and found that 300-400MB/s is expected with default settings. Improvements are possible up to 1000MB/s via mtu 9000 and if you increase ip buffers ex
>> max_buf=4097152 tcp
>> send_buf=2048576 tcp
>> recv_buf=2048576 tcp,
>>
>> NFS lockd servers (ex 1024), NFS number of threads (ex 64) and NFS transfer size (ex 1048576)
>>
>> http://napp-it.org/doc/downloads/performance_smb2.pdf
>>
>>
>> Gea
>>
>> Am 16.09.2016 um 19:43 schrieb Ergi Thanasko:
>>> Hi all,
>>> We have a a few servers  conected via 10g nic  LACP, some of them have  4nic and some have 6nic in a link aggregation mode. We been moving a lot of data around and we are trying to get the maximum performance. I have seen zpool can deliver  2-3GB accumulated  throughput. Iperf does about 600-800MB/sec between those two servers.
>>> Given  the hardware that we have and the zpool performance,   we expected   to see some serious data transfer rates  however we only see around 200-300MB/sec average  using rsync or copy paste over NFS.  Standard MTU 1500 and nfs block size.  I want to ask the community what to do get some higher throughout and the application level. I hear ZFS send/receive   or ZFS shadow does work faster but it does snapshots. Out data (Terabytes) is constantly evolving   and we prefer something in the nature of rsync  but to utilize the network hardware.
>>>
>>> If Anyone has a hardware setup that can see 1GB/sec  throughput  and does not mind sharing?
>>> Any software  that  use multithreads  sessions to move data around  zfs friendly? We do not mind getting going with a commercial solution like camvault or veeam if they work.
>>>
>>> Thank you for your time
>>>
>>>   
>>>
>>> _______________________________________________
>>> OmniOS-discuss mailing list
>>> OmniOS-discuss at lists.omniti.com
>>> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>> _______________________________________________
>> OmniOS-discuss mailing list
>> OmniOS-discuss at lists.omniti.com
>> http://lists.omniti.com/mailman/listinfo/omnios-discuss


From mir at miras.org  Sat Sep 17 21:03:44 2016
From: mir at miras.org (Michael Rasmussen)
Date: Sat, 17 Sep 2016 23:03:44 +0200
Subject: [OmniOS-discuss] kernel panic
Message-ID: <20160917230344.493adaee@sleipner.datanom.net>

Hi all,

On my Omnios r151014 (omnios-b13298f) I had a sudden kernel panic this
evening with the following display in /var/adm/messages:

Sep 17 21:35:29 nas unix: [ID 836849 kern.notice] 
Sep 17 21:35:29 nas ^Mpanic[cpu2]/thread=ffffff071b9a8400: 
Sep 17 21:35:29 nas genunix: [ID 683410 kern.notice] BAD TRAP: type=d (#gp General protection) rp=ffffff00206eb700 addr=ffffff06e6fd8d00
Sep 17 21:35:29 nas unix: [ID 100000 kern.notice] 
Sep 17 21:35:29 nas unix: [ID 839527 kern.notice] perl: 
Sep 17 21:35:29 nas unix: [ID 753105 kern.notice] #gp General protection
Sep 17 21:35:29 nas unix: [ID 358286 kern.notice] addr=0xffffff06e6fd8d00
Sep 17 21:35:29 nas unix: [ID 243837 kern.notice] pid=18522, pc=0xfffffffffbb4e61f, sp=0xffffff00206eb7f0, eflags=0x10282
Sep 17 21:35:29 nas unix: [ID 211416 kern.notice] cr0: 80050033<pg,wp,ne,et,mp,pe> cr4: 6f8<xmme,fxsr,pge,mce,pae,pse,de>
Sep 17 21:35:29 nas unix: [ID 624947 kern.notice] cr2: 817824c
Sep 17 21:35:29 nas unix: [ID 625075 kern.notice] cr3: 3d63d5000
Sep 17 21:35:29 nas unix: [ID 625715 kern.notice] cr8: c
Sep 17 21:35:29 nas unix: [ID 100000 kern.notice] 
Sep 17 21:35:29 nas unix: [ID 592667 kern.notice]       rdi: ffffff06e6fd8d00 rsi: 1fffffe0ed7a1000 rdx:             1000
Sep 17 21:35:29 nas unix: [ID 592667 kern.notice]       rcx: ffffff00206eba90  r8: ffffff00206eb9f0  r9:             1000
Sep 17 21:35:29 nas unix: [ID 592667 kern.notice]       rax: ffffff071db23ad8 rbx: ffffff06e6fd8d00 rbp: ffffff00206eb8a0
Sep 17 21:35:29 nas unix: [ID 592667 kern.notice]       r10: baddcafebaddcafe r11:                0 r12: ffffff00206eb9f0
Sep 17 21:35:29 nas unix: [ID 592667 kern.notice]       r13: ffffff00206eba90 r14: 1fffffe0ed7a1000 r15: ffffff06e6fd8d00
Sep 17 21:35:29 nas unix: [ID 592667 kern.notice]       fsb:                0 gsb: ffffff06e6106500  ds:               4b
Sep 17 21:35:29 nas unix: [ID 592667 kern.notice]        es:               4b  fs:                0  gs:              1c3
Sep 17 21:35:29 nas unix: [ID 592667 kern.notice]       trp:                d err:                0 rip: fffffffffbb4e61f
Sep 17 21:35:29 nas unix: [ID 592667 kern.notice]        cs:               30 rfl:            10282 rsp: ffffff00206eb7f0
Sep 17 21:35:29 nas unix: [ID 266532 kern.notice]        ss:                0
Sep 17 21:35:29 nas unix: [ID 100000 kern.notice] 
Sep 17 21:35:29 nas genunix: [ID 655072 kern.notice] ffffff00206eb5e0 unix:real_mode_stop_cpu_stage2_end+9e13 ()
Sep 17 21:35:29 nas genunix: [ID 655072 kern.notice] ffffff00206eb6f0 unix:trap+a30 ()
Sep 17 21:35:29 nas genunix: [ID 655072 kern.notice] ffffff00206eb700 unix:cmntrap+e6 ()
Sep 17 21:35:29 nas genunix: [ID 655072 kern.notice] ffffff00206eb8a0 genunix:fop_getpage+3f ()
Sep 17 21:35:29 nas genunix: [ID 655072 kern.notice] ffffff00206eb960 genunix:anon_getpage+18d ()
Sep 17 21:35:29 nas genunix: [ID 655072 kern.notice] ffffff00206eba80 genunix:segvn_faultpage+168 ()
Sep 17 21:35:29 nas genunix: [ID 655072 kern.notice] ffffff00206ebc50 genunix:segvn_fault+8e6 ()
Sep 17 21:35:29 nas genunix: [ID 655072 kern.notice] ffffff00206ebd60 genunix:as_fault+31a ()
Sep 17 21:35:29 nas genunix: [ID 655072 kern.notice] ffffff00206ebdf0 unix:pagefault+96 ()
Sep 17 21:35:29 nas genunix: [ID 655072 kern.notice] ffffff00206ebf00 unix:trap+2c7 ()
Sep 17 21:35:29 nas genunix: [ID 655072 kern.notice] ffffff00206ebf10 unix:cmntrap+e6 ()
Sep 17 21:35:29 nas unix: [ID 100000 kern.notice] 
Sep 17 21:35:29 nas genunix: [ID 672855 kern.notice] syncing file systems...
Sep 17 21:35:30 nas genunix: [ID 904073 kern.notice]  done
Sep 17 21:35:31 nas genunix: [ID 111219 kern.notice] dumping to /dev/zvol/dsk/rpool/dump, offset 65536, content: kernel
Sep 17 21:35:31 nas genunix: [ID 405573 kern.info] NOTICE: ahci1: ahci_tran_reset_dport port 0 reset port
Sep 17 21:35:31 nas genunix: [ID 405573 kern.info] NOTICE: ahci1: ahci_tran_reset_dport port 1 reset port

Anybody knows what is going on here?

NB. I have the kernel dump file in case anybody should be interested.

-- 
Hilsen/Regards
Michael Rasmussen

Get my public GnuPG keys:
michael <at> rasmussen <dot> cc
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E
mir <at> datanom <dot> net
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C
mir <at> miras <dot> org
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917
--------------------------------------------------------------
/usr/games/fortune -es says:
	"Life and death are seldom logical."
	"But attaining a desired goal always is."
		-- McCoy and Spock, "The Galileo Seven", stardate 2821.7
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 163 bytes
Desc: OpenPGP digital signature
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20160917/c6f8cad4/attachment.bin>

From mir at miras.org  Sat Sep 17 22:08:10 2016
From: mir at miras.org (Michael Rasmussen)
Date: Sun, 18 Sep 2016 00:08:10 +0200
Subject: [OmniOS-discuss] kernel panic
In-Reply-To: <20160917230344.493adaee@sleipner.datanom.net>
References: <20160917230344.493adaee@sleipner.datanom.net>
Message-ID: <20160918000810.49ca7650@sleipner.datanom.net>

On Sat, 17 Sep 2016 23:03:44 +0200
Michael Rasmussen <mir at miras.org> wrote:

> Hi all,
> 
> On my Omnios r151014 (omnios-b13298f) I had a sudden kernel panic this
> evening with the following display in /var/adm/messages:
> 
And after reboot:
Sep 17 21:51:07 nas savecore: [ID 833026 auth.error] Decompress the crash dump w
ith
Sep 17 21:51:07 nas 'savecore -vf /var/crash/unknown/vmdump.2'
Sep 17 21:51:08 nas fmd: [ID 377184 daemon.error] SUNW-MSG-ID: SUNOS-8000-KL, TY
PE: Defect, VER: 1, SEVERITY: Major
Sep 17 21:51:08 nas EVENT-TIME: Sat Sep 17 21:51:07 CEST 2016
Sep 17 21:51:08 nas PLATFORM: To-Be-Filled-By-O.E.M., CSN: -, HOSTNAME: nas
Sep 17 21:51:08 nas SOURCE: software-diagnosis, REV: 0.1
Sep 17 21:51:08 nas EVENT-ID: 914a5680-21c2-6d8b-e476-e7be57cf40e9
Sep 17 21:51:08 nas DESC: The system has rebooted after a kernel panic.  Refer t
o http://illumos.org/msg/SUNOS-8000-KL for more information.
Sep 17 21:51:08 nas AUTO-RESPONSE: The failed system image was dumped to the dum
p device.  If savecore is enabled (see dumpadm(1M)) a copy of the dump will be w
ritten to the savecore directory /var/crash/unknown.
Sep 17 21:51:08 nas IMPACT: There may be some performance impact while the panic
 is copied to the savecore directory.  Disk space usage by panics can be substan
tial.
Sep 17 21:51:08 nas REC-ACTION: If savecore is not enabled then please take step
s to preserve the crash image.
Sep 17 21:51:08 nas Use 'fmdump -Vp -u 914a5680-21c2-6d8b-e476-e7be57cf40e9' to 
view more panic detail.  Please refer to the knowledge article for additional in
formation.



-- 
Hilsen/Regards
Michael Rasmussen

Get my public GnuPG keys:
michael <at> rasmussen <dot> cc
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E
mir <at> datanom <dot> net
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C
mir <at> miras <dot> org
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917
--------------------------------------------------------------
/usr/games/fortune -es says:
She was good at playing abstract confusion in the same way a midget is
good at being short.
		-- Clive James, on Marilyn Monroe
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 163 bytes
Desc: OpenPGP digital signature
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20160918/035c52a8/attachment-0001.bin>

From jimklimov at cos.ru  Sun Sep 18 06:38:59 2016
From: jimklimov at cos.ru (Jim Klimov)
Date: Sun, 18 Sep 2016 08:38:59 +0200
Subject: [OmniOS-discuss] Network throughout 1GB/sec
In-Reply-To: <580E52D0-B98D-4836-A5B4-9AF7ED8FDD47@avsquad.com>
References: <56B30910-D0CB-4C7A-9848-313358007C6E@avsquad.com>
	<86D8AFD6-91E5-4AA4-A6CF-6285665BC257@omniti.com>
	<580E52D0-B98D-4836-A5B4-9AF7ED8FDD47@avsquad.com>
Message-ID: <E13BFB14-BB3C-48A9-9F7C-D5E1D9EDD381@cos.ru>

17 ???????? 2016??. 2:37:12 CEST, Ergi Thanasko <ergi.thanasko at avsquad.com> ?????:
>no that is even slower, just rsync over mounted  nfs, a multithreaded
>rsync does work have better performance.
>
>[cid:449A210A-451D-40E1-97EA-98BA5E7FA558 at fios-router.home]
>
>On Sep 16, 2016, at 5:24 PM, Dale Ghent
><daleg at omniti.com<mailto:daleg at omniti.com>> wrote:
>
>Are you doing the rsync over ssh? You might want to look into using
>HPN-SSH:
>
>https://www.psc.edu/index.php/hpn-ssh
>
>/dale
>
>On Sep 16, 2016, at 1:43 PM, Ergi Thanasko <ergi.thanasko at avsquad.com>
>wrote:
>
>Hi all,
>We have a a few servers  conected via 10g nic  LACP, some of them have 
>4nic and some have 6nic in a link aggregation mode. We been moving a
>lot of data around and we are trying to get the maximum performance. I
>have seen zpool can deliver  2-3GB accumulated  throughput. Iperf does
>about 600-800MB/sec between those two servers.
>Given  the hardware that we have and the zpool performance,   we
>expected   to see some serious data transfer rates  however we only see
>around 200-300MB/sec average  using rsync or copy paste over NFS. 
>Standard MTU 1500 and nfs block size.  I want to ask the community what
>to do get some higher throughout and the application level. I hear ZFS
>send/receive   or ZFS shadow does work faster but it does snapshots.
>Out data (Terabytes) is constantly evolving   and we prefer something
>in the nature of rsync  but to utilize the network hardware.
>
>If Anyone has a hardware setup that can see 1GB/sec  throughput  and
>does not mind sharing?
>Any software  that  use multithreads  sessions to move data around  zfs
>friendly? We do not mind getting going with a commercial solution like
>camvault or veeam if they work.
>
>Thank you for your time
>
>
>
>
>_______________________________________________
>OmniOS-discuss mailing list
>OmniOS-discuss at lists.omniti.com
>http://lists.omniti.com/mailman/listinfo/omnios-discuss
>
>
>
>
>------------------------------------------------------------------------
>
>_______________________________________________
>OmniOS-discuss mailing list
>OmniOS-discuss at lists.omniti.com
>http://lists.omniti.com/mailman/listinfo/omnios-discuss

Note that with rsync over nfs (or any seemingly local FS) your sender spawns a recipient process which must also read the existing data to see what differences rsync should transfer. So over your links you read the entirety of data from remote host, and send something back. Unless your other side has a pathologically weak cpu (think hobby ARM board), you gotta be faster off with real networked rsync (native or rsh/ssh etc.), where sender and recipient run on respective hosts and process the bulk of data locally.

Also NFS I/O by design defaults to synchronous writes, so forfeits write-caching (with ZFS you can have ZIL on SSD to mitigate that performance impact and keep data security guarantees); some other OSes pretend they are fast by ignoring the sync requirement, and then you have fun times after a crash, failover etc. of the NFS server side.

Jim
--
Typos courtesy of K-9 Mail on my Samsung Android

From skeltonr at btconnect.com  Sun Sep 18 08:55:24 2016
From: skeltonr at btconnect.com (Richard Skelton)
Date: Sun, 18 Sep 2016 09:55:24 +0100
Subject: [OmniOS-discuss] I would like to create a zfs file system in my zone
Message-ID: <57DE567C.6050100@btconnect.com>

Hi,
I would like to create a zfs file system in my zone and create snapshots
like I would in a global zone.
In the zone I see:-
root at fred5:/root# zfs list
NAME                           USED  AVAIL  REFER  MOUNTPOINT
rpool                         38.6G  93.2G  23.5K  /rpool
rpool/zones                   2.94G  93.2G    19K  /zones
rpool/zones/fred5           1.85G  93.2G    19K  /zones/fred5
rpool/zones/fred5/ROOT      1.85G  93.2G    19K  legacy
rpool/zones/fred5/ROOT/zbe  1.85G  93.2G  1.85G  legacy
root at fred5:/root# ls /zones/fred5
ls: cannot access '/zones/fred5': No such file or directory
root at fred5:/root# zpool list
NAME    SIZE  ALLOC   FREE  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
rpool   136G  30.3G   106G         -    19%    22%  1.00x  ONLINE  -
root at fred5:/root#

Before I install any optional software I would like to create a zfs file
system and mount it on /opt in the zone.
How can I do this?

Cheers
Richard

From peter.tribble at gmail.com  Sun Sep 18 09:32:24 2016
From: peter.tribble at gmail.com (Peter Tribble)
Date: Sun, 18 Sep 2016 10:32:24 +0100
Subject: [OmniOS-discuss] I would like to create a zfs file system in my
	zone
In-Reply-To: <57DE567C.6050100@btconnect.com>
References: <57DE567C.6050100@btconnect.com>
Message-ID: <CAEgYsbEZs7OUURiLYxV-qOCaof7+h78Bzk6ZUtJpo7o08EW=oQ@mail.gmail.com>

On Sun, Sep 18, 2016 at 9:55 AM, Richard Skelton <skeltonr at btconnect.com>
wrote:

> Hi,
> I would like to create a zfs file system in my zone and create snapshots
> like I would in a global zone.
> In the zone I see:-
> root at fred5:/root# zfs list
> NAME                           USED  AVAIL  REFER  MOUNTPOINT
> rpool                         38.6G  93.2G  23.5K  /rpool
> rpool/zones                   2.94G  93.2G    19K  /zones
> rpool/zones/fred5           1.85G  93.2G    19K  /zones/fred5
> rpool/zones/fred5/ROOT      1.85G  93.2G    19K  legacy
> rpool/zones/fred5/ROOT/zbe  1.85G  93.2G  1.85G  legacy
> root at fred5:/root# ls /zones/fred5
> ls: cannot access '/zones/fred5': No such file or directory
>

So the way it works is this: the dataset

rpool/zones/fred5/ROOT

is delegated to the zone. Anything above that is associated with the global
zone;
that dataset or below is associated with the local zone. You get to see all
the parents
in the hierarchy, with their mountpoints in their own context, which is a
bit confusing.


> root at fred5:/root# zpool list
> NAME    SIZE  ALLOC   FREE  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
> rpool   136G  30.3G   106G         -    19%    22%  1.00x  ONLINE  -
> root at fred5:/root#
>
> Before I install any optional software I would like to create a zfs file
> system and mount it on /opt in the zone.
> How can I do this?
>

In this case, it could be:

zfs create -o mountpoint=/opt rpool/zones/fred5/ROOT/opt

However, I wouldn't normally do this. If I was creating datasets for use by
a
zone, I would create a separate dataset such as rpool/fred5 in the global
zone
(with mountpoint set to none), use 'add dataset' in the zone configuration
to
delegate that dataset to the zone, and use that to create my own filesystems
in the zone. The aim being to separate my own data from that managed by the
operating system.

(Although the same issue of conflicts arises with the use of /opt, which the
OS might think it has control of. Creating /opt/local or /opt/company-name
one level down might be a better choice.)

-- 
-Peter Tribble
http://www.petertribble.co.uk/ - http://ptribble.blogspot.com/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20160918/3a577f25/attachment.html>

From bfriesen at simple.dallas.tx.us  Sun Sep 18 14:44:22 2016
From: bfriesen at simple.dallas.tx.us (Bob Friesenhahn)
Date: Sun, 18 Sep 2016 09:44:22 -0500 (CDT)
Subject: [OmniOS-discuss] I would like to create a zfs file system in my
 zone
In-Reply-To: <57DE567C.6050100@btconnect.com>
References: <57DE567C.6050100@btconnect.com>
Message-ID: <alpine.GSO.2.20.1609180934530.5577@freddy.simplesystems.org>

On Sun, 18 Sep 2016, Richard Skelton wrote:
>
> Before I install any optional software I would like to create a zfs file
> system and mount it on /opt in the zone.
> How can I do this?

Don't attempt to do this because OmniOS delivers some essential 
package content to /opt (e.g. gcc run-time libraries) and you will 
encounter booting problems after an upgrade (I was slightly singed by 
that).  Default OmniOS has /opt in the boot environment so if you move 
it out of the boot environment then the installation will be 
inconsistent.

You should be free to mount on mountpoints within /opt.

Bob
-- 
Bob Friesenhahn
bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/

From vab at bb-c.de  Sun Sep 18 14:47:24 2016
From: vab at bb-c.de (Volker A. Brandt)
Date: Sun, 18 Sep 2016 16:47:24 +0200
Subject: [OmniOS-discuss] I would like to create a zfs file system in my
	zone
In-Reply-To: <CAEgYsbEZs7OUURiLYxV-qOCaof7+h78Bzk6ZUtJpo7o08EW=oQ@mail.gmail.com>
References: <57DE567C.6050100@btconnect.com>
	<CAEgYsbEZs7OUURiLYxV-qOCaof7+h78Bzk6ZUtJpo7o08EW=oQ@mail.gmail.com>
Message-ID: <22494.43260.50648.709397@shelob.bb-c.de>

Peter Tribble writes:
> On Sun, Sep 18, 2016 at 9:55 AM, Richard Skelton <skeltonr at btconnect.com>
> wrote:
>     Before I install any optional software I would like to create a zfs file
>     system and mount it on /opt in the zone.
>     How can I do this?
> 
> 
> In this case, it could be:
> 
> zfs create -o mountpoint=/opt rpool/zones/fred5/ROOT/opt

On Solaris 11, a "zfs create -o mountpoint=/opt rpool/opt"  will also
work.  (Can't test on OmniOS ATM as my OmniOS system is down.)

This has the advantage that the ZFS dataset will appear as "rpool/opt"
in the zone, and as "<zone root>/<zone name>/opt" in the global zone.

> However, I wouldn't normally do this. If I was creating datasets for use by a
> zone, I would create a separate dataset such as rpool/fred5 in the global zone
> (with mountpoint set to none), use 'add dataset' in the zone configuration to
> delegate that dataset to the zone, and use that to create my own filesystems
> in the zone. The aim being to separate my own data from that managed by the
> operating system.
> 
> (Although the same issue of conflicts arises with the use of /opt, which the
> OS might think it has control of. Creating /opt/local or /opt/company-name
> one level down might be a better choice.)

One thing to remember is that the directory /opt normally is part of the
zone root, and thus part of the managed boot environments (BEs).  If you
create a dataset and mount it under /opt (or /opt/local), it will not
be part of any BE.  If you then create IPS packages that install things
in /opt, you will run into inconsistencies in the package database when
you switch back and forth between BEs.


Regards -- Volker
-- 
------------------------------------------------------------------------
Volker A. Brandt               Consulting and Support for Oracle Solaris
Brandt & Brandt Computer GmbH                   WWW: http://www.bb-c.de/
Am Wiesenpfad 6, 53340 Meckenheim, GERMANY            Email: vab at bb-c.de
Handelsregister: Amtsgericht Bonn, HRB 10513              Schuhgr??e: 46
Gesch?ftsf?hrer: Rainer J.H. Brandt und Volker A. Brandt

"When logic and proportion have fallen sloppy dead"

From jdg117 at elvis.arl.psu.edu  Sun Sep 18 22:40:53 2016
From: jdg117 at elvis.arl.psu.edu (John D Groenveld)
Date: Sun, 18 Sep 2016 18:40:53 -0400
Subject: [OmniOS-discuss] CMake fail due to ncurses vs curses
Message-ID: <201609182240.u8IMerGf015467@elvis.arl.psu.edu>

Has anyone got past this conflict?

$ env PATH=/usr/bin:/usr/sbin:/usr/ccs/bin:/usr/gnu/bin:/usr/gnu/i386-pc-solaris2.11/bin CC=gcc CXX=g++ CFLAGS=-m64 CXXFLAGS=-m64 LDFLAGS=-m64 ./bootstrap --system-libs --prefix=/opt/cmake --no-system-libarchive --no-system-jsoncpp --parallel=8

[ 87%] Linking CXX executable ../bin/ccmake
Undefined                       first referenced
 symbol                             in file
initscr32                           CMakeFiles/ccmake.dir/CursesDialog/ccmake.cxx.o  (symbol belongs to implicit dependency /usr/lib/64/libcurses.so.1)
w32attron                           CMakeFiles/ccmake.dir/CursesDialog/cmCursesLongMessageForm.cxx.o  (symbol belongs to implicit dependency /usr/lib/64/libcurses.so.1)
w32attroff                          CMakeFiles/ccmake.dir/CursesDialog/cmCursesLongMessageForm.cxx.o  (symbol belongs to implicit dependency /usr/lib/64/libcurses.so.1)
ld: fatal: symbol referencing errors. No output written to ../bin/ccmake
collect2: error: ld returned 1 exit status
Source/CMakeFiles/ccmake.dir/build.make:419: recipe for target 'bin/ccmake' failed
gmake[2]: *** [bin/ccmake] Error 1
CMakeFiles/Makefile2:1696: recipe for target 'Source/CMakeFiles/ccmake.dir/all' failed
gmake[1]: *** [Source/CMakeFiles/ccmake.dir/all] Error 2
Makefile:160: recipe for target 'all' failed
gmake: *** [all] Error 2

I can add -lcurses to Source/CMakeFiles/ccmake.dir/link.txt, below,
but that seems like a Bad Idea[tm].
nm(1) suggests that initscr32 and friends are defined in libncurses.so
How do I chase down the reference to libcurses functions?

/usr/bin/g++   -m64    -m64 CMakeFiles/ccmake.dir/CursesDialog/cmCursesOptionsWidget.cxx.o CMakeFiles/ccmake.dir/CursesDialog/cmCursesBoolWidget.cxx.o CMakeFiles/ccmake.dir/CursesDialog/cmCursesCacheEntryComposite.cxx.o CMakeFiles/ccmake.dir/CursesDialog/cmCursesDummyWidget.cxx.o CMakeFiles/ccmake.dir/CursesDialog/cmCursesFilePathWidget.cxx.o CMakeFiles/ccmake.dir/CursesDialog/cmCursesForm.cxx.o CMakeFiles/ccmake.dir/CursesDialog/cmCursesLabelWidget.cxx.o CMakeFiles/ccmake.dir/CursesDialog/cmCursesLongMessageForm.cxx.o CMakeFiles/ccmake.dir/CursesDialog/cmCursesMainForm.cxx.o CMakeFiles/ccmake.dir/CursesDialog/cmCursesPathWidget.cxx.o CMakeFiles/ccmake.dir/CursesDialog/cmCursesStringWidget.cxx.o CMakeFiles/ccmake.dir/CursesDialog/cmCursesWidget.cxx.o CMakeFiles/ccmake.dir/CursesDialog/ccmake.cxx.o  -o ../bin/ccmake  -L/usr/lib/64  libCMakeLib.a /usr/lib/64/libform.so /usr/lib/64/libncurses.so kwsys/libcmsys.a -ldl /usr/lib/64/libexpat.so ../Utilities/cmlibarchive/libarchive/!
 libcmlibarchive.a -lz /usr/lib/64/liblzma.so -lmd -lbz2 ../Utilities/cmcompress/libcmcompress.a /usr/lib/64/libcurl.so ../Utilities/cmjsoncpp/libcmjsoncpp.a -Wl,-R/usr/lib/64

TIA, hope to post a build recipe for GitLab CE this week.
John
groenveld at acm.org

From lotheac at iki.fi  Mon Sep 19 07:52:33 2016
From: lotheac at iki.fi (Lauri Tirkkonen)
Date: Mon, 19 Sep 2016 10:52:33 +0300
Subject: [OmniOS-discuss] CMake fail due to ncurses vs curses
In-Reply-To: <201609182240.u8IMerGf015467@elvis.arl.psu.edu>
References: <201609182240.u8IMerGf015467@elvis.arl.psu.edu>
Message-ID: <20160919075232.GA11664@kekkonen.niksula.hut.fi>

On Sun, Sep 18 2016 18:40:53 -0400, John D Groenveld wrote:
> Has anyone got past this conflict?
> 
> $ env PATH=/usr/bin:/usr/sbin:/usr/ccs/bin:/usr/gnu/bin:/usr/gnu/i386-pc-solaris2.11/bin CC=gcc CXX=g++ CFLAGS=-m64 CXXFLAGS=-m64 LDFLAGS=-m64 ./bootstrap --system-libs --prefix=/opt/cmake --no-system-libarchive --no-system-jsoncpp --parallel=8
> 
> [ 87%] Linking CXX executable ../bin/ccmake

I just built it without the curses interface. See
https://github.com/niksula/omnios-build-scripts/blob/master/cmake/build.sh#L43

-- 
Lauri Tirkkonen | lotheac @ IRCnet

From danmcd at omniti.com  Mon Sep 19 15:35:33 2016
From: danmcd at omniti.com (Dan McDonald)
Date: Mon, 19 Sep 2016 11:35:33 -0400
Subject: [OmniOS-discuss] kernel panic
In-Reply-To: <20160918000810.49ca7650@sleipner.datanom.net>
References: <20160917230344.493adaee@sleipner.datanom.net>
	<20160918000810.49ca7650@sleipner.datanom.net>
Message-ID: <A1323C87-F043-447C-B973-0F090CDEAC03@omniti.com>


> On Sep 17, 2016, at 6:08 PM, Michael Rasmussen <mir at miras.org> wrote:
> 
> And after reboot:
> Sep 17 21:51:07 nas savecore: [ID 833026 auth.error] Decompress the crash dump w
> ith
> Sep 17 21:51:07 nas 'savecore -vf /var/crash/unknown/vmdump.2'
> Sep 17 21:51:08 nas fmd: [ID 377184 daemon.error] SUNW-MSG-ID: SUNOS-8000-KL, TY
> PE: Defect, VER: 1, SEVERITY: Major

FMA always gets a fault report after a kernel panic.

I see you have a vmdump.2 present.  Can you make that downloadable somewhere? I'd like to take a closer look.

The stack trace looks... almost random.  It could be a double-fault (though I thought those were explicitly called out), but it looks like it panicked while handling a userland fault.  This is going to sound like I'm pointing fingers elsewhere, but how old is your HW?

Dan


From mir at miras.org  Mon Sep 19 15:51:28 2016
From: mir at miras.org (Michael Rasmussen)
Date: Mon, 19 Sep 2016 17:51:28 +0200
Subject: [OmniOS-discuss] kernel panic
In-Reply-To: <A1323C87-F043-447C-B973-0F090CDEAC03@omniti.com>
References: <20160917230344.493adaee@sleipner.datanom.net>
	<20160918000810.49ca7650@sleipner.datanom.net>
	<A1323C87-F043-447C-B973-0F090CDEAC03@omniti.com>
Message-ID: <20160919175128.11d667e3@sleipner.datanom.net>

On Mon, 19 Sep 2016 11:35:33 -0400
Dan McDonald <danmcd at omniti.com> wrote:

> 
> I see you have a vmdump.2 present.  Can you make that downloadable somewhere? I'd like to take a closer look.
> 
ftp://ftp.datanom.net/pub/vmdump.2

> The stack trace looks... almost random.  It could be a double-fault (though I thought those were explicitly called out), but it looks like it panicked while handling a userland fault.  This is going to sound like I'm pointing fingers elsewhere, but how old is your HW?
> 
From smbios:
Motherboard
  Manufacturer: ASRock
  Product: A75 Extreme6

CPU
  Manufacturer: AMD
  Version: AMD A8-3800 APU with Radeon(tm) HD Graphics
  Location Tag: CPUSocket

  Family: 72 (AMD A-series)
  Family Ext: 72 (AMD A-series)
  CPUID: 0x178bfbff00300f10
  Type: 3 (central processor)
  Socket Upgrade: 41 (socket FM1)
  Socket Status: Populated
  Processor Status: 1 (enabled)
  Supported Voltages: 1.2V
  Core Count: 4
  Cores Enabled: 4
  Thread Count: 4
  Processor Characteristics: 0x4
        SMB_PRC_64BIT (64-bit capable)
  External Clock Speed: Unknown
  Maximum Speed: 2400MHz
  Current Speed: 2400MHz
         L1 Cache: 23
         L2 Cache: 24
         L3 Cache: None


-- 
Hilsen/Regards
Michael Rasmussen

Get my public GnuPG keys:
michael <at> rasmussen <dot> cc
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E
mir <at> datanom <dot> net
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C
mir <at> miras <dot> org
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917
--------------------------------------------------------------
/usr/games/fortune -es says:
"There was a boy called Eustace Clarence Scrubb, and he almost deserved
it." -- C. S. Lewis, "The Chronicles of Narnia"
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 163 bytes
Desc: OpenPGP digital signature
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20160919/e3760b87/attachment.bin>

From mir at miras.org  Mon Sep 19 15:55:54 2016
From: mir at miras.org (Michael Rasmussen)
Date: Mon, 19 Sep 2016 17:55:54 +0200
Subject: [OmniOS-discuss] kernel panic
In-Reply-To: <20160919175128.11d667e3@sleipner.datanom.net>
References: <20160917230344.493adaee@sleipner.datanom.net>
	<20160918000810.49ca7650@sleipner.datanom.net>
	<A1323C87-F043-447C-B973-0F090CDEAC03@omniti.com>
	<20160919175128.11d667e3@sleipner.datanom.net>
Message-ID: <20160919175554.43bfc2dc@sleipner.datanom.net>

On Mon, 19 Sep 2016 17:51:28 +0200
Michael Rasmussen <mir at miras.org> wrote:

> ftp://ftp.datanom.net/pub/vmdump.2
> 
Also via http:
http://ftp.datanom.net/vmdump.2

-- 
Hilsen/Regards
Michael Rasmussen

Get my public GnuPG keys:
michael <at> rasmussen <dot> cc
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E
mir <at> datanom <dot> net
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C
mir <at> miras <dot> org
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917
--------------------------------------------------------------
/usr/games/fortune -es says:
Neil Armstrong tripped.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 163 bytes
Desc: OpenPGP digital signature
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20160919/fe6a169a/attachment.bin>

From danmcd at omniti.com  Mon Sep 19 17:33:02 2016
From: danmcd at omniti.com (Dan McDonald)
Date: Mon, 19 Sep 2016 13:33:02 -0400
Subject: [OmniOS-discuss] kernel panic
In-Reply-To: <20160919175128.11d667e3@sleipner.datanom.net>
References: <20160917230344.493adaee@sleipner.datanom.net>
	<20160918000810.49ca7650@sleipner.datanom.net>
	<A1323C87-F043-447C-B973-0F090CDEAC03@omniti.com>
	<20160919175128.11d667e3@sleipner.datanom.net>
Message-ID: <0C836AC0-20D3-4358-BB4D-F1FC63189431@omniti.com>


> On Sep 19, 2016, at 11:51 AM, Michael Rasmussen <mir at miras.org> wrote:
> 
> On Mon, 19 Sep 2016 11:35:33 -0400
> Dan McDonald <danmcd at omniti.com> wrote:
> 
>> 
>> I see you have a vmdump.2 present.  Can you make that downloadable somewhere? I'd like to take a closer look.
>> 
> ftp://ftp.datanom.net/pub/vmdump.2

This process:

R  15776   2671     10     10      0 0x4a004000 ffffff075dadd0a0 perl
        T  0xffffff072006ec60 <TS_SLEEP>

Hit a page fault:

> 0xffffff071b9a8400::findstack -v
stack pointer for thread ffffff071b9a8400: ffffff00206eb1f0
  ffffff00206eb290 page_ctr_sub_internal+0x65(0, 3, ffffff0012581270, 1)
  ffffff00206eb2f0 page_ctr_sub+0x7e(0, 3, ffffff0012581270, 1)
  ffffff00206eb410 page_get_mnode_freelist+0x3ff(0, 4, 3, 0, 202000b)
  ffffff00206eb4e0 page_get_freelist+0x16d(286, 0, baddcafebaddcafe, 
  fffffffffb934d9f, ffffff00206eb510, ffffff06e6fd8d00, ffffffff00000008)
  ffffff00206eb550 param_preset()
  ffffff00206eb5e0 0xfffffffffb8480fc()
  ffffff00206eb6f0 trap+0xa30(ffffff00206eb700, ffffff06e6fd8d00, 2)
  ffffff00206eb700 0xfffffffffb8001d6()
  ffffff00206eb8a0 fop_getpage+0x3f(ffffff06e6fd8d00, 1fffffe0ed7a1000, 1000, 
  ffffff00206eba90, ffffff00206eb9f0, 1000, ffffff0760c25938, 8178000, 
  ffffff0000000002, ffffff071db23ad8, 0)
  ffffff00206eb960 anon_getpage+0x18d(ffffff00206eba08, ffffff00206eba90, 
  ffffff00206eb9f0, 1000, ffffff0760c25938, 8178000, 2, ffffff071db23ad8)
  ffffff00206eba80 segvn_faultpage+0x168(ffffff07213e2020, ffffff0760c25938, 
  8178000, 116000, 0, ffffff00206ebb50, 0, 1, 2, 1)
  ffffff00206ebc50 segvn_fault+0x8e6(ffffff07213e2020, ffffff0760c25938, 8178000
  , 1000, 1, 2)
  ffffff00206ebd60 as_fault+0x31a(ffffff07213e2020, ffffff072245adc8, 817824c, 1
  , 1, 2)
  ffffff00206ebdf0 pagefault+0x96(817824c, 1, 2, 0)
  ffffff00206ebf00 trap+0x2c7(ffffff00206ebf10, 817824c, 2)
  ffffff00206ebf10 0xfffffffffb8001d6()
> 

And then trapped further while doing fop_getpage().  What's your swap device?

I also couldn't help but notice this:

> ffffff06e6fd8d00::print vnode_t v_vfsp
v_vfsp = 0xbaddcafebaddcafe
> 


WHOA!

> ffffff06e6fd8d00::whatis
ffffff06e6fd8d00 is ffffff06e6fd85c0+740, allocated from rootnex_dmahdl
> 


So fop_getpage() was sent a vnode_t that at least had one field uninitialized (0xbaddcafe).  Turns out the whole wad is:

> ffffff06e6fd8d00/10P
0xffffff06e6fd8d00:             0xbaddcafebaddcafe0xbaddcafebaddcafe
                0xbaddcafebaddcafe0xbaddcafebaddcafe0xbaddcafebaddcafe
                0xbaddcafebaddcafe0xbaddcafebaddcafe0xbaddcafebaddcafe
                0xbaddcafebaddcafe0xbaddcafebaddcafe
> 

The anon cache entry fed into anon_getpage seems legit, but corrupt somehow:

> ffffff00206eba08/P
0xffffff00206eba08:             0xffffff076bd0a5c0
> 0xffffff076bd0a5c0::print "struct anon"
{
    an_vp = 0xffffff06e6fd8d00
    an_pvp = 0
    an_off = 0x1fffffe0ed7a1000
    an_poff = 0
    an_hash = 0
    an_refcnt = 0x1
}
> 0xffffff06e6fd8d00::whatis
ffffff06e6fd8d00 is ffffff06e6fd85c0+740, allocated from rootnex_dmahdl
> 0xffffff06e6fd8d00/20P    
0xffffff06e6fd8d00:             0xbaddcafebaddcafe0xbaddcafebaddcafe
                0xbaddcafebaddcafe0xbaddcafebaddcafe0xbaddcafebaddcafe
                0xbaddcafebaddcafe0xbaddcafebaddcafe0xbaddcafebaddcafe
                0xbaddcafebaddcafe0xbaddcafebaddcafe0xbaddcafebaddcafe
                0xbaddcafebaddcafe0xbaddcafebaddcafe0xbaddcafebaddcafe
                0xbaddcafebaddcafe0xbaddcafebaddcafe0xbaddcafebaddcafe
                0xbaddcafebaddcafe0xbaddcafebaddcafe0xbaddcafebaddcafe
> 0xffffff076bd0a5c0::whatis
ffffff076bd0a5c0 is allocated from anon_cache
> 

Generally speaking, this is memory corruption.  What caused it I'm not sure, however.

Dan



From mir at miras.org  Mon Sep 19 17:43:13 2016
From: mir at miras.org (Michael Rasmussen)
Date: Mon, 19 Sep 2016 19:43:13 +0200
Subject: [OmniOS-discuss] kernel panic
In-Reply-To: <0C836AC0-20D3-4358-BB4D-F1FC63189431@omniti.com>
References: <20160917230344.493adaee@sleipner.datanom.net>
	<20160918000810.49ca7650@sleipner.datanom.net>
	<A1323C87-F043-447C-B973-0F090CDEAC03@omniti.com>
	<20160919175128.11d667e3@sleipner.datanom.net>
	<0C836AC0-20D3-4358-BB4D-F1FC63189431@omniti.com>
Message-ID: <20160919194313.2f302ae8@sleipner.datanom.net>

On Mon, 19 Sep 2016 13:33:02 -0400
Dan McDonald <danmcd at omniti.com> wrote:

> This process:
> 
> R  15776   2671     10     10      0 0x4a004000 ffffff075dadd0a0 perl
>         T  0xffffff072006ec60 <TS_SLEEP>
> 
> 
Only perl running here:
# ps -ef |grep perl
    root  2684  2674   3   Sep 17 ?         267:07 /usr/bin/perl /var/web-gui/data/napp-it/zfsos/_lib/scripts/socketserver.pl -l h
    root  2656     1   0   Sep 17 ?           0:31 /usr/perl5/bin/perl /usr/lib/intrd
    root  2695  2684   0   Sep 17 ?          53:48 /usr/bin/perl /var/web-gui/data/napp-it/zfsos/_lib/scripts/socketserver.pl -l h

> And then trapped further while doing fop_getpage().  What's your swap device?
> 
# swap -l
swapfile             dev    swaplo   blocks     free
/dev/zvol/dsk/rpool/swap 91,2         8  4194296  4194296


-- 
Hilsen/Regards
Michael Rasmussen

Get my public GnuPG keys:
michael <at> rasmussen <dot> cc
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E
mir <at> datanom <dot> net
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C
mir <at> miras <dot> org
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917
--------------------------------------------------------------
/usr/games/fortune -es says:
Beggar to well-dressed businessman:
	"Could you spare $20.95 for a fifth of Chivas?"
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 163 bytes
Desc: OpenPGP digital signature
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20160919/a9bf81eb/attachment.bin>

From danmcd at omniti.com  Mon Sep 19 17:49:16 2016
From: danmcd at omniti.com (Dan McDonald)
Date: Mon, 19 Sep 2016 13:49:16 -0400
Subject: [OmniOS-discuss] kernel panic
In-Reply-To: <20160919194313.2f302ae8@sleipner.datanom.net>
References: <20160917230344.493adaee@sleipner.datanom.net>
	<20160918000810.49ca7650@sleipner.datanom.net>
	<A1323C87-F043-447C-B973-0F090CDEAC03@omniti.com>
	<20160919175128.11d667e3@sleipner.datanom.net>
	<0C836AC0-20D3-4358-BB4D-F1FC63189431@omniti.com>
	<20160919194313.2f302ae8@sleipner.datanom.net>
Message-ID: <1FEC6CA6-491A-4621-8596-9C880069C464@omniti.com>


> On Sep 19, 2016, at 1:43 PM, Michael Rasmussen <mir at miras.org> wrote:
> 
> On Mon, 19 Sep 2016 13:33:02 -0400
> Dan McDonald <danmcd at omniti.com> wrote:
> 
>> This process:
>> 
>> R  15776   2671     10     10      0 0x4a004000 ffffff075dadd0a0 perl
>>        T  0xffffff072006ec60 <TS_SLEEP>
>> 
>> 
> Only perl running here:
> # ps -ef |grep perl
>    root  2684  2674   3   Sep 17 ?         267:07 /usr/bin/perl /var/web-gui/data/napp-it/zfsos/_lib/scripts/socketserver.pl -l h
>    root  2656     1   0   Sep 17 ?           0:31 /usr/perl5/bin/perl /usr/lib/intrd
>    root  2695  2684   0   Sep 17 ?          53:48 /usr/bin/perl /var/web-gui/data/napp-it/zfsos/_lib/scripts/socketserver.pl -l h

Remember, a page fault isn't a bad thing.  It just means the process has a paged swapped out:

> ::ps -f !grep 15776
R  15776   2671     10     10      0 0x4a004000 ffffff075dadd0a0 perl /var/web-gui/data/scripts/agent-nappit.pl '1474140713_hmp' ''
> 

And THAT was the particular process running.

>> And then trapped further while doing fop_getpage().  What's your swap device?
>> 
> # swap -l
> swapfile             dev    swaplo   blocks     free
> /dev/zvol/dsk/rpool/swap 91,2         8  4194296  4194296

SO you've 4G of swap on rpool.  Typical out of the box deployment.

Something corrupted the anon cache entry, making it point to something in unused/unallocated/un-something hyperspace.  The big question is what.  This is why I asked about your hardware.  It wasn't obvious, but since you're running NAPP-IT, I assume you're using ECC memory?

Dan


From mir at miras.org  Mon Sep 19 17:52:54 2016
From: mir at miras.org (Michael Rasmussen)
Date: Mon, 19 Sep 2016 19:52:54 +0200
Subject: [OmniOS-discuss] kernel panic
In-Reply-To: <0C836AC0-20D3-4358-BB4D-F1FC63189431@omniti.com>
References: <20160917230344.493adaee@sleipner.datanom.net>
	<20160918000810.49ca7650@sleipner.datanom.net>
	<A1323C87-F043-447C-B973-0F090CDEAC03@omniti.com>
	<20160919175128.11d667e3@sleipner.datanom.net>
	<0C836AC0-20D3-4358-BB4D-F1FC63189431@omniti.com>
Message-ID: <20160919195254.55709606@sleipner.datanom.net>

On Mon, 19 Sep 2016 13:33:02 -0400
Dan McDonald <danmcd at omniti.com> wrote:

> 
> Generally speaking, this is memory corruption.  What caused it I'm not sure, however.
> 
A bit flip caused by radiation from Cosmos? ;-)

-- 
Hilsen/Regards
Michael Rasmussen

Get my public GnuPG keys:
michael <at> rasmussen <dot> cc
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E
mir <at> datanom <dot> net
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C
mir <at> miras <dot> org
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917
--------------------------------------------------------------
/usr/games/fortune -es says:
My doctorate's in Literature, but it seems like a pretty good pulse to
me.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 163 bytes
Desc: OpenPGP digital signature
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20160919/f001ff79/attachment.bin>

From mir at miras.org  Mon Sep 19 17:56:31 2016
From: mir at miras.org (Michael Rasmussen)
Date: Mon, 19 Sep 2016 19:56:31 +0200
Subject: [OmniOS-discuss] kernel panic
In-Reply-To: <1FEC6CA6-491A-4621-8596-9C880069C464@omniti.com>
References: <20160917230344.493adaee@sleipner.datanom.net>
	<20160918000810.49ca7650@sleipner.datanom.net>
	<A1323C87-F043-447C-B973-0F090CDEAC03@omniti.com>
	<20160919175128.11d667e3@sleipner.datanom.net>
	<0C836AC0-20D3-4358-BB4D-F1FC63189431@omniti.com>
	<20160919194313.2f302ae8@sleipner.datanom.net>
	<1FEC6CA6-491A-4621-8596-9C880069C464@omniti.com>
Message-ID: <20160919195631.76ecba37@sleipner.datanom.net>

On Mon, 19 Sep 2016 13:49:16 -0400
Dan McDonald <danmcd at omniti.com> wrote:

> 
> Something corrupted the anon cache entry, making it point to something in unused/unallocated/un-something hyperspace.  The big question is what.  This is why I asked about your hardware.  It wasn't obvious, but since you're running NAPP-IT, I assume you're using ECC memory?
> 
I am afraid not. No support on the MB. Box is to be Upgraded sometime
in the near future but serves a purpose for the moment.

-- 
Hilsen/Regards
Michael Rasmussen

Get my public GnuPG keys:
michael <at> rasmussen <dot> cc
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E
mir <at> datanom <dot> net
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C
mir <at> miras <dot> org
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917
--------------------------------------------------------------
/usr/games/fortune -es says:
My doctorate's in Literature, but it seems like a pretty good pulse to
me.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 163 bytes
Desc: OpenPGP digital signature
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20160919/8b4b295e/attachment.bin>

From robertfantini at gmail.com  Wed Sep 21 14:33:35 2016
From: robertfantini at gmail.com (Robert Fantini)
Date: Wed, 21 Sep 2016 10:33:35 -0400
Subject: [OmniOS-discuss] fan control
In-Reply-To: <CALnM7zkmWwLOHuVjG2nMgb_8i83ObTnPu8PKdEtbpm3XMd23nQ@mail.gmail.com>
References: <CALnM7zkmWwLOHuVjG2nMgb_8i83ObTnPu8PKdEtbpm3XMd23nQ@mail.gmail.com>
Message-ID: <CALnM7z=iRt+gyoOOCQS4pV5MBGnVLckR0WZvSTv_eF6N7hU6ng@mail.gmail.com>

Just to close this off,

 Our issue was hardware related.

Motherboard / chassis / power supply mismatch on a testing system.

New system fans work as expected.

On Tue, May 24, 2016 at 7:28 PM, Robert Fantini <robertfantini at gmail.com>
wrote:

> we've installed napp-it to a couple of systems .   they run a lot louder -
> fans do not seem to be controlled from software.
>
> is there a software package that that can be installed to handle sensors
> and fans?
>
> Or is this something to deal with in Supermicro bios?
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20160921/46387395/attachment.html>

From geoffn at gnaa.net  Wed Sep 21 18:40:51 2016
From: geoffn at gnaa.net (Geoff Nordli)
Date: Wed, 21 Sep 2016 11:40:51 -0700
Subject: [OmniOS-discuss] upgrading to r151018 -- conflicting actions
Message-ID: <914b01c5-2351-22a5-b6ea-86b4579849ac@gnaa.net>

Hi.

When I try to upgrade to r151018 from r151016 it can't complete.

#pkg update
Creating Plan (Checking for conflicting actions): /
pkg update: The following packages deliver conflicting action types to 
usr/share/man/man4/ssh_config.4:

   link:
pkg://omnios/service/network/ssh-common at 0.5.11,5.11-0.151018:20160603T032215Z
   file:
pkg://omnios/network/openssh at 7.2.2,5.11-0.151018:20160412T153954Z

These packages may not be installed together. Any non-conflicting set may
be, or the packages must be corrected before they can be installed.

The following packages all deliver file actions to etc/ssh/ssh_config:

pkg://omnios/network/openssh at 7.2.2,5.11-0.151018:20160412T153954Z
pkg://omnios/service/network/ssh-common at 0.5.11,5.11-0.151018:20160603T032215Z

These packages may not be installed together. Any non-conflicting set may
be, or the packages must be corrected before they can be installed.


these are the ssh packages currently installed:

#pkg list | grep ssh
network/openssh 7.2.2-0.151016             i--
service/network/ssh 0.5.11-0.151016            i--

Any suggestions on how to get it upgraded?

thanks,

Geoff



From danmcd at omniti.com  Wed Sep 21 18:50:11 2016
From: danmcd at omniti.com (Dan McDonald)
Date: Wed, 21 Sep 2016 14:50:11 -0400
Subject: [OmniOS-discuss] upgrading to r151018 -- conflicting actions
In-Reply-To: <914b01c5-2351-22a5-b6ea-86b4579849ac@gnaa.net>
References: <914b01c5-2351-22a5-b6ea-86b4579849ac@gnaa.net>
Message-ID: <D86CADEB-B97F-4D84-8B1D-3AA2DE6AE0F8@omniti.com>


> On Sep 21, 2016, at 2:40 PM, Geoff Nordli <geoffn at gnaa.net> wrote:
> 
> Hi.
> 
> When I try to upgrade to r151018 from r151016 it can't complete.
> 
> #pkg update
> Creating Plan (Checking for conflicting actions): /
> pkg update: The following packages deliver conflicting action types to usr/share/man/man4/ssh_config.4:
> 
>  link:
> pkg://omnios/service/network/ssh-common at 0.5.11,5.11-0.151018:20160603T032215Z
>  file:
> pkg://omnios/network/openssh at 7.2.2,5.11-0.151018:20160412T153954Z
> 
> These packages may not be installed together. Any non-conflicting set may
> be, or the packages must be corrected before they can be installed.
> 
> The following packages all deliver file actions to etc/ssh/ssh_config:
> 
> pkg://omnios/network/openssh at 7.2.2,5.11-0.151018:20160412T153954Z
> pkg://omnios/service/network/ssh-common at 0.5.11,5.11-0.151018:20160603T032215Z
> 
> These packages may not be installed together. Any non-conflicting set may
> be, or the packages must be corrected before they can be installed.

Add these flags to the 'pkg upgrade' command:

	--reject network/ssh-common --reject service/network/ssh --reject omnios/network/ssh --reject ssh/ssh-key

which will convert you from SunSSH to OpenSSH (which is good, because SunSSH goes away after r151018).

Dan


From geoffn at gnaa.net  Wed Sep 21 20:36:37 2016
From: geoffn at gnaa.net (Geoff Nordli)
Date: Wed, 21 Sep 2016 13:36:37 -0700
Subject: [OmniOS-discuss] upgrading to r151018 -- conflicting actions
In-Reply-To: <D86CADEB-B97F-4D84-8B1D-3AA2DE6AE0F8@omniti.com>
References: <914b01c5-2351-22a5-b6ea-86b4579849ac@gnaa.net>
	<D86CADEB-B97F-4D84-8B1D-3AA2DE6AE0F8@omniti.com>
Message-ID: <d0d646f0-df27-a023-ec45-6bdd4abd5a9e@gnaa.net>



On 2016-09-21 11:50 AM, Dan McDonald wrote:
>> On Sep 21, 2016, at 2:40 PM, Geoff Nordli<geoffn at gnaa.net>  wrote:
>>
>> Hi.
>>
>> When I try to upgrade to r151018 from r151016 it can't complete.
>>
>> #pkg update
>> Creating Plan (Checking for conflicting actions): /
>> pkg update: The following packages deliver conflicting action types to usr/share/man/man4/ssh_config.4:
>>
>>   link:
>> pkg://omnios/service/network/ssh-common at 0.5.11,5.11-0.151018:20160603T032215Z
>>   file:
>> pkg://omnios/network/openssh at 7.2.2,5.11-0.151018:20160412T153954Z
>>
>> These packages may not be installed together. Any non-conflicting set may
>> be, or the packages must be corrected before they can be installed.
>>
>> The following packages all deliver file actions to etc/ssh/ssh_config:
>>
>> pkg://omnios/network/openssh at 7.2.2,5.11-0.151018:20160412T153954Z
>> pkg://omnios/service/network/ssh-common at 0.5.11,5.11-0.151018:20160603T032215Z
>>
>> These packages may not be installed together. Any non-conflicting set may
>> be, or the packages must be corrected before they can be installed.
> Add these flags to the 'pkg upgrade' command:
>
> 	--reject network/ssh-common --reject service/network/ssh --reject omnios/network/ssh --reject ssh/ssh-key
>
> which will convert you from SunSSH to OpenSSH (which is good, because SunSSH goes away after r151018).
>
> Dan
>

thanks Dan.

This command did the trick:

  pkg update --reject network/ssh-common --reject service/network/ssh 
--reject ssh/ssh-key


I couldn't do the

--reject network/ssh-common --reject service/network/ssh --reject omnios/network/ssh --reject ssh/ssh-key


pkg update: The following pattern(s) did not match any allowable 
packages.  Try
using a different matching pattern, or refreshing publisher information:

         omnios/network/ssh


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20160921/719e5a6f/attachment-0001.html>

From danmcd at omniti.com  Wed Sep 21 20:50:24 2016
From: danmcd at omniti.com (Dan McDonald)
Date: Wed, 21 Sep 2016 16:50:24 -0400
Subject: [OmniOS-discuss] Bloody Update for September 21st & LX
	update/question
References: <7C1C6107-5405-4F40-84DD-7BF0076FD918@kebe.com>
Message-ID: <48EF546C-ED7A-4A39-8310-68D360DEB359@omniti.com>

Hello!

I'm starting to wind down LX for now, and wind up the r151020 release.  I'm leaning toward folding in LX support (see below) into stock OmniOS for r151020 as "LX Beta", with fixes and supportability showing up in r151022 (next LTS).

uname -a says omnios-master-a10017b, which means illumos-omnios was at commit a10017b, and omnios-build was at commit a0dbf1f.

New with this bloody update:

- Bash is now 4.4

- OpenSSH support/SunSSH EOL is now easier to upgrade to.  Instead of --reject lines, if you have SunSSH installed, you'll still *SEE* those packages, but they are now empty and "renamed" to the OpenSSH packages.  Post-upgrade "pkg list | grep ssh" can show you what I mean if you're still on SunSSH.

- Curl is now 7.50.3 (and was updated earlier as part of the security fix)

- SEVERAL NVMe fixes.

- nlockmgr now more reliable at boot time (7231 nlockmgr failing to start up during bootup)

- Various ZFS bugfixes



ALSO UPDATED is the bloody+LX repo:  http://pkg.omniti.com/omnios/LX/.  It is updated to illumos-joyent changes since their commit 6a41e72, and now quiesced.  One bug was also fixed... SMAP support on Broadwell and newer Intel processors broke LX zones on OmniOS, because of a mismerge.

I'M VERY CLOSE to folding in LX support into illumos-omnios's master branch, prior to construction of the r151020 branch.  If people have opinions on this topic, I'd very much like to hear it.  Also, is anyone besides OmniTI developers using LX for OmniOS?

Thanks,
Dan

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20160921/2a996b02/attachment.html>

From dain.bentley at gmail.com  Wed Sep 21 21:08:00 2016
From: dain.bentley at gmail.com (Dain Bentley)
Date: Wed, 21 Sep 2016 21:08:00 +0000 (UTC)
Subject: [OmniOS-discuss] Bloody Update for September 21st & LX
 update/question
In-Reply-To: <48EF546C-ED7A-4A39-8310-68D360DEB359@omniti.com>
References: <7C1C6107-5405-4F40-84DD-7BF0076FD918@kebe.com>
	<48EF546C-ED7A-4A39-8310-68D360DEB359@omniti.com>
Message-ID: <C459D3C3A9FD081A.4344D62A-44BE-4893-B872-5BE8EE41A992@mail.outlook.com>

I use it

Get Outlook for iOS




On Wed, Sep 21, 2016 at 4:51 PM -0400, "Dan McDonald" <danmcd at omniti.com> wrote:










Hello!

I'm starting to wind down LX for now, and wind up the r151020 release. ?I'm leaning toward folding in LX support (see below) into stock OmniOS for r151020 as "LX Beta", with fixes and supportability showing up in r151022 (next LTS).

uname -a says omnios-master-a10017b, which means illumos-omnios was at commit a10017b, and omnios-build was at commit a0dbf1f.

New with this bloody update:

- Bash is now 4.4

- OpenSSH support/SunSSH EOL is now easier to upgrade to. ?Instead of --reject lines, if you have SunSSH installed, you'll still *SEE* those packages, but they are now empty and "renamed" to the OpenSSH packages. ?Post-upgrade "pkg list | grep ssh" can show you what I mean if you're still on SunSSH.

- Curl is now 7.50.3 (and was updated earlier as part of the security fix)

- SEVERAL NVMe fixes.

- nlockmgr now more reliable at boot time (7231 nlockmgr failing to start up during bootup)

- Various ZFS bugfixes



ALSO UPDATED is the bloody+LX repo: ?http://pkg.omniti.com/omnios/LX/. ?It is updated to illumos-joyent changes since their commit 6a41e72, and now quiesced. ?One bug was also fixed... SMAP support on Broadwell and newer Intel processors broke LX zones on OmniOS, because of a mismerge.

I'M VERY CLOSE to folding in LX support into illumos-omnios's master branch, prior to construction of the r151020 branch. ?If people have opinions on this topic, I'd very much like to hear it. ?Also, is anyone besides OmniTI developers using LX for OmniOS?

Thanks,
Dan






-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20160921/b4c48ff6/attachment.html>

From danmcd at omniti.com  Thu Sep 22 18:22:39 2016
From: danmcd at omniti.com (Dan McDonald)
Date: Thu, 22 Sep 2016 14:22:39 -0400
Subject: [OmniOS-discuss] OpenSSL 1.0.2i is out
Message-ID: <FB1C583C-F4BA-47D3-9E3A-843E89577BE7@omniti.com>

And available for "pkg update" on all supported OmniOS releases, plus bloody.

Dan


From skeltonr at btconnect.com  Thu Sep 22 19:42:53 2016
From: skeltonr at btconnect.com (Richard Skelton)
Date: Thu, 22 Sep 2016 20:42:53 +0100
Subject: [OmniOS-discuss] How can I increase the number of file descriptors
 for the apache service
Message-ID: <57E4343D.1080809@btconnect.com>

Hi,
I have installed the apache22 package from ms.omniti.com and enabled the
service included.
I now fined the httpd has only 256 file descriptors.
plimit 1188
1188:   /opt/apache22/bin/amd64/httpd -f /opt/apache22/conf/httpd.conf
-k star
   resource              current         maximum
  time(seconds)         unlimited       unlimited
  file(blocks)          unlimited       unlimited
  data(kbytes)          unlimited       unlimited
  stack(kbytes)         10240           unlimited
  coredump(blocks)      unlimited       unlimited
  nofiles(descriptors)  256             65536
  vmemory(kbytes)       unlimited       unlimited

The service starts httpd :-
/opt/apache22/bin/httpd -f /opt/apache22/conf/httpd.conf -k start
Found that by running :-
svccfg -s svc:/network/http:apache22 listprop start/exec
start/exec  astring  "/opt/apache22/bin/httpd -f
/opt/apache22/conf/httpd.conf -k start"

How do I set the file descriptors to 1024?

Cheers
Richard


From mtalbott at lji.org  Fri Sep 23 01:07:56 2016
From: mtalbott at lji.org (Michael Talbott)
Date: Thu, 22 Sep 2016 18:07:56 -0700
Subject: [OmniOS-discuss] LX zone empty error messages
Message-ID: <8EB27AEE-6D90-4C18-8CE4-5FE1F9D5CCA8@lji.org>

So I'm finally getting around to trying out LX zones and can't seem to get the zone installed for some reason. Here's what I've done and where I'm stuck:

# Fresh install of latest bloody
# Setup basic networking
# pkg update -v
# reboot
# pkg set-publisher -G http://pkg.omniti.com/omnios/bloody/ -g http://pkg.omniti.com/omnios/LX/ omnios
# pkg install brand/lx
# curl -o /root/centos6.zss.gz https://images.joyent.com/images/5b7e86e4-2797-11e6-b7d7-cbe6a1a48791/file <https://images.joyent.com/images/5b7e86e4-2797-11e6-b7d7-cbe6a1a48791/file>

# zonecfg -z lx0
# create -b
# set zonepath=/zones/lx0
# set brand=lx
# set autoboot=false
# set ip-type=exclusive
# add net
# set physical=lx0
# add property (name=gateway,value="192.168.0.1")
# add property (name=ips,value="192.168.0.69/24")
# add property (name=primary,value="true")
# end
# add attr
# set name=dns-domain
# set type=string
# set value=example.com
# end
# add attr
# set name=resolvers
# set type=string
# set value=192.168.0.1
# end
# add attr
# set name=kernel-version
# set type=string
# set value=2.6.32
# end
# commit

-- This is where it breaks
# zoneadm -z lx0 install -s /root/centos6.zss.gz

I get this.. 
root at bloodybox:/root# zoneadm -z lx0 install -s /root/centos6.zss.gz
WARNING: skipping network interface 'lx0': object not found
ERROR: 

Ok, I think I can fix the iface issue like so:
# dladm create-vnic lx0 -l e1000g0

But, the empty error message persists and the zone never gets installed :(

root at bloodybox:/root# zoneadm -z lx0 uninstall 
root at bloodybox:/root# zoneadm -z lx0 install -s /root/centos6.zss.gz
ERROR: 

And for giggles, I gunzipped the file and tried it with the unzipped version. Same deal. Any ideas?.. I'm running this as a VM for testing before I put it on bare metal. Could that be the issue or did I miss something critical above? I wish the error message said *something* ;) Any other way of getting some extra verbose output to see where it's dying?

Thanks

Michael
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20160922/663e90a0/attachment-0001.html>

From danmcd at omniti.com  Fri Sep 23 01:42:18 2016
From: danmcd at omniti.com (Dan McDonald)
Date: Thu, 22 Sep 2016 21:42:18 -0400
Subject: [OmniOS-discuss] LX zone empty error messages
In-Reply-To: <8EB27AEE-6D90-4C18-8CE4-5FE1F9D5CCA8@lji.org>
References: <8EB27AEE-6D90-4C18-8CE4-5FE1F9D5CCA8@lji.org>
Message-ID: <F3A1B957-5525-4120-9FC5-6FF7E6A47E8D@omniti.com>


> On Sep 22, 2016, at 9:07 PM, Michael Talbott <mtalbott at lji.org> wrote:

<SNIP!>

> And for giggles, I gunzipped the file and tried it with the unzipped version. Same deal. Any ideas?.. I'm running this as a VM for testing before I put it on bare metal. Could that be the issue or did I miss something critical above? I wish the error message said *something* ;) Any other way of getting some extra verbose output to see where it's dying?

Weird.

One thing you can try is not only to uninstall the zone, but also delete its configuration (after exporting it) and then re-create it.

I'm trying to reproduce your bug right now.  I got this:

zonecfg:lx2> export
create -b
set zonepath=/zones/lx2
set brand=lx
set autoboot=false
set ip-type=exclusive
add net
set physical=lx2
add property (name=gateway,value="x.y.z.1")
add property (name=primary,value="true")
add property (name=ips,value="x.y.z.246/24")
end
add attr
set name=dns-domain
set type=string
set value=x.y.z.net
end
add attr
set name=resolvers
set type=string
set value=x.y.z.1
end
add attr
set name=kernel-version
set type=string
set value=2.6.32
end
zonecfg:lx2> exit
# zoneadm list -cv
  ID NAME             STATUS     PATH                           BRAND    IP    
   0 global           running    /                              ipkg     shared
   4 lx1              running    /zones/lx1                     lx       excl  
   - tz2              installed  /zones/tz2                     lipkg    excl  
   - lx0              installed  /zones/lx0                     lx       excl  
   - lx2              configured /zones/lx2                     lx       excl  
# zoneadm -z lx2 install -s /rpool/lx-stuff/joyent-images/centos6.gz 
WARNING: skipping network interface 'lx2': object not found
A ZFS file system has been created for this zone.
# zoneadm list -cv
  ID NAME             STATUS     PATH                           BRAND    IP    
   0 global           running    /                              ipkg     shared
   4 lx1              running    /zones/lx1                     lx       excl  
   - tz2              installed  /zones/tz2                     lipkg    excl  
   - lx0              installed  /zones/lx0                     lx       excl  
   - lx2              installed  /zones/lx2                     lx       excl  
# 


Okay... dumb question:  is "/zones" on your box its own ZFS dataset?

# zfs list | grep /zones
rpool/zones                        8.57G  34.6G    20K  /zones
rpool/zones/lx0                    1.33G  34.6G  1.87G  /zones/lx0
rpool/zones/lx1                    1.86G  34.6G  1.81G  /zones/lx1
rpool/zones/lx2                     787M  34.6G   787M  /zones/lx2
rpool/zones/tz2                    4.62G  34.6G  20.5K  /zones/tz2
rpool/zones/tz2/ROOT               4.62G  34.6G    19K  legacy
rpool/zones/tz2/ROOT/zbe-14        2.32M  34.6G  1.31G  legacy
rpool/zones/tz2/ROOT/zbe-15        2.50M  34.6G  1.24G  legacy
rpool/zones/tz2/ROOT/zbe-16         100K  34.6G  1.11G  legacy
rpool/zones/tz2/ROOT/zbe-17        1.89M  34.6G  1.25G  legacy
rpool/zones/tz2/ROOT/zbe-21        1.89M  34.6G  1.26G  legacy
rpool/zones/tz2/ROOT/zbe-22        2.55M  34.6G  1.26G  legacy
rpool/zones/tz2/ROOT/zbe-25        10.4M  34.6G  1.12G  legacy
rpool/zones/tz2/ROOT/zbe-29          73K  34.6G  1.12G  legacy
rpool/zones/tz2/ROOT/zbe-30        28.8M  34.6G  1.12G  legacy
rpool/zones/tz2/ROOT/zbe-31        18.5M  34.6G  1.11G  legacy
rpool/zones/tz2/ROOT/zbe-32        4.55G  34.6G  1.13G  legacy
rpool/zones/tz2/ROOT/zbe-33        68.5K  34.6G  1.13G  legacy
# 


It should be, and MAYBE that's causing your problem?

Dan


From mtalbott at lji.org  Fri Sep 23 02:08:45 2016
From: mtalbott at lji.org (Michael Talbott)
Date: Thu, 22 Sep 2016 19:08:45 -0700
Subject: [OmniOS-discuss] LX zone empty error messages
In-Reply-To: <F3A1B957-5525-4120-9FC5-6FF7E6A47E8D@omniti.com>
References: <8EB27AEE-6D90-4C18-8CE4-5FE1F9D5CCA8@lji.org>
	<F3A1B957-5525-4120-9FC5-6FF7E6A47E8D@omniti.com>
Message-ID: <CAA8A986-2E2F-4CED-ACCC-6FBCED89D5B0@lji.org>

Ah HA! So, I did not have a zfs dataset at /zones. I created one, retried the process and the same ominous message popped up :( Then I rebooted and tried again, and success! Hurray! Now I can carry on with my testing :)

Thanks!


> On Sep 22, 2016, at 6:42 PM, Dan McDonald <danmcd at omniti.com> wrote:
> 
> 
>> On Sep 22, 2016, at 9:07 PM, Michael Talbott <mtalbott at lji.org> wrote:
> 
> <SNIP!>
> 
>> And for giggles, I gunzipped the file and tried it with the unzipped version. Same deal. Any ideas?.. I'm running this as a VM for testing before I put it on bare metal. Could that be the issue or did I miss something critical above? I wish the error message said *something* ;) Any other way of getting some extra verbose output to see where it's dying?
> 
> Weird.
> 
> One thing you can try is not only to uninstall the zone, but also delete its configuration (after exporting it) and then re-create it.
> 
> I'm trying to reproduce your bug right now.  I got this:
> 
> zonecfg:lx2> export
> create -b
> set zonepath=/zones/lx2
> set brand=lx
> set autoboot=false
> set ip-type=exclusive
> add net
> set physical=lx2
> add property (name=gateway,value="x.y.z.1")
> add property (name=primary,value="true")
> add property (name=ips,value="x.y.z.246/24")
> end
> add attr
> set name=dns-domain
> set type=string
> set value=x.y.z.net
> end
> add attr
> set name=resolvers
> set type=string
> set value=x.y.z.1
> end
> add attr
> set name=kernel-version
> set type=string
> set value=2.6.32
> end
> zonecfg:lx2> exit
> # zoneadm list -cv
>  ID NAME             STATUS     PATH                           BRAND    IP    
>   0 global           running    /                              ipkg     shared
>   4 lx1              running    /zones/lx1                     lx       excl  
>   - tz2              installed  /zones/tz2                     lipkg    excl  
>   - lx0              installed  /zones/lx0                     lx       excl  
>   - lx2              configured /zones/lx2                     lx       excl  
> # zoneadm -z lx2 install -s /rpool/lx-stuff/joyent-images/centos6.gz 
> WARNING: skipping network interface 'lx2': object not found
> A ZFS file system has been created for this zone.
> # zoneadm list -cv
>  ID NAME             STATUS     PATH                           BRAND    IP    
>   0 global           running    /                              ipkg     shared
>   4 lx1              running    /zones/lx1                     lx       excl  
>   - tz2              installed  /zones/tz2                     lipkg    excl  
>   - lx0              installed  /zones/lx0                     lx       excl  
>   - lx2              installed  /zones/lx2                     lx       excl  
> # 
> 
> 
> Okay... dumb question:  is "/zones" on your box its own ZFS dataset?
> 
> # zfs list | grep /zones
> rpool/zones                        8.57G  34.6G    20K  /zones
> rpool/zones/lx0                    1.33G  34.6G  1.87G  /zones/lx0
> rpool/zones/lx1                    1.86G  34.6G  1.81G  /zones/lx1
> rpool/zones/lx2                     787M  34.6G   787M  /zones/lx2
> rpool/zones/tz2                    4.62G  34.6G  20.5K  /zones/tz2
> rpool/zones/tz2/ROOT               4.62G  34.6G    19K  legacy
> rpool/zones/tz2/ROOT/zbe-14        2.32M  34.6G  1.31G  legacy
> rpool/zones/tz2/ROOT/zbe-15        2.50M  34.6G  1.24G  legacy
> rpool/zones/tz2/ROOT/zbe-16         100K  34.6G  1.11G  legacy
> rpool/zones/tz2/ROOT/zbe-17        1.89M  34.6G  1.25G  legacy
> rpool/zones/tz2/ROOT/zbe-21        1.89M  34.6G  1.26G  legacy
> rpool/zones/tz2/ROOT/zbe-22        2.55M  34.6G  1.26G  legacy
> rpool/zones/tz2/ROOT/zbe-25        10.4M  34.6G  1.12G  legacy
> rpool/zones/tz2/ROOT/zbe-29          73K  34.6G  1.12G  legacy
> rpool/zones/tz2/ROOT/zbe-30        28.8M  34.6G  1.12G  legacy
> rpool/zones/tz2/ROOT/zbe-31        18.5M  34.6G  1.11G  legacy
> rpool/zones/tz2/ROOT/zbe-32        4.55G  34.6G  1.13G  legacy
> rpool/zones/tz2/ROOT/zbe-33        68.5K  34.6G  1.13G  legacy
> # 
> 
> 
> It should be, and MAYBE that's causing your problem?
> 
> Dan
> 


From danmcd at omniti.com  Fri Sep 23 02:31:43 2016
From: danmcd at omniti.com (Dan McDonald)
Date: Thu, 22 Sep 2016 22:31:43 -0400
Subject: [OmniOS-discuss] LX zone empty error messages
In-Reply-To: <CAA8A986-2E2F-4CED-ACCC-6FBCED89D5B0@lji.org>
References: <8EB27AEE-6D90-4C18-8CE4-5FE1F9D5CCA8@lji.org>
	<F3A1B957-5525-4120-9FC5-6FF7E6A47E8D@omniti.com>
	<CAA8A986-2E2F-4CED-ACCC-6FBCED89D5B0@lji.org>
Message-ID: <2F65EF55-2B24-4D80-AF26-062BB729593F@omniti.com>


> On Sep 22, 2016, at 10:08 PM, Michael Talbott <mtalbott at lji.org> wrote:
> 
> Ah HA! So, I did not have a zfs dataset at /zones. I created one, retried the process and the same ominous message popped up :( Then I rebooted and tried again, and success! Hurray! Now I can carry on with my testing :)

Sounds like, however, there needs to be better diagnostics somewhere.  Thanks for adding that to the tests.  (I suspect the Joyent-originated brand code assumes /zones always exists as a ZFS dataset, as it does in all SmartOS deployments.)

Thanks!
Dan


From danmcd at omniti.com  Fri Sep 23 21:29:31 2016
From: danmcd at omniti.com (Dan McDonald)
Date: Fri, 23 Sep 2016 17:29:31 -0400
Subject: [OmniOS-discuss] LX zone empty error messages
In-Reply-To: <CAA8A986-2E2F-4CED-ACCC-6FBCED89D5B0@lji.org>
References: <8EB27AEE-6D90-4C18-8CE4-5FE1F9D5CCA8@lji.org>
	<F3A1B957-5525-4120-9FC5-6FF7E6A47E8D@omniti.com>
	<CAA8A986-2E2F-4CED-ACCC-6FBCED89D5B0@lji.org>
Message-ID: <FCC63FB6-3C03-4669-8B53-B3007275C3FE@omniti.com>


> On Sep 22, 2016, at 10:08 PM, Michael Talbott <mtalbott at lji.org> wrote:
> 
> Ah HA! So, I did not have a zfs dataset at /zones. I created one, retried the process and the same ominous message popped up :( Then I rebooted and tried again, and success! Hurray! Now I can carry on with my testing :)

Followup, as much for me as for you,

The failure isn't more verbose because the zones-common function get_zonepath_ds() requires a variable be set by the brand script, which our LX zone code does not, currently.  We need to fix that, so instead of:

ERROR:

we can instead print something like what the ipkg/lipkg brand does:

ERROR:  the zonepath must be a ZFS dataset.
The parent directory of the zonepath must be a ZFS dataset so that the
zonepath ZFS dataset can be created properly.

Now I'm beginning to understand why the brand scripts of ipkg were brought into Oracle's version of illumos-gate for Oracle Solaris.  Some of the ipkg/lipkg items can/should be more common across other brands as well.

Until that day, however, I will make sure the lx_install script has a proper $f_no_ds variable set.

Thanks again, Michael!
Dan


From garland at bustle.com  Fri Sep 23 23:25:03 2016
From: garland at bustle.com (Garland McAlexander)
Date: Fri, 23 Sep 2016 19:25:03 -0400
Subject: [OmniOS-discuss] Installation Failing (Reboots)
Message-ID: <CAMUDemvCkBEDLp_2ePQB+H=B=UnEOPf-EeqVRFPqmYGT___rFg@mail.gmail.com>

Hi All,

Trying to install the latest stable Omnios via USB.

After booting from USB, I am greeted by the Omnios logo. One that proceeds
into another screen, it loads a few things, gets to "syncing system files"
and immediately after that something flashes (unable to see the text, even
when spamming pause/break) and my server reboots.

I have tried to recreate the USB multiple times, and I've done MD5 checking
on my download.

Any assistance would be absolutely fantastic.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20160923/35a221d5/attachment.html>

From bfriesen at simple.dallas.tx.us  Fri Sep 23 23:53:32 2016
From: bfriesen at simple.dallas.tx.us (Bob Friesenhahn)
Date: Fri, 23 Sep 2016 18:53:32 -0500 (CDT)
Subject: [OmniOS-discuss] Installation Failing (Reboots)
In-Reply-To: <CAMUDemvCkBEDLp_2ePQB+H=B=UnEOPf-EeqVRFPqmYGT___rFg@mail.gmail.com>
References: <CAMUDemvCkBEDLp_2ePQB+H=B=UnEOPf-EeqVRFPqmYGT___rFg@mail.gmail.com>
Message-ID: <alpine.GSO.2.20.1609231852460.5577@freddy.simplesystems.org>

On Fri, 23 Sep 2016, Garland McAlexander wrote:

> Hi All,
>
> Trying to install the latest stable Omnios via USB.
>
> After booting from USB, I am greeted by the Omnios logo. One that proceeds
> into another screen, it loads a few things, gets to "syncing system files"
> and immediately after that something flashes (unable to see the text, even
> when spamming pause/break) and my server reboots.
>
> I have tried to recreate the USB multiple times, and I've done MD5 checking
> on my download.
>
> Any assistance would be absolutely fantastic.

The most likely cause of the problem would be due to using USB3 
hardware with an OS which only supports USB1 and USB2.

Bob
-- 
Bob Friesenhahn
bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/

From mtalbott at lji.org  Sat Sep 24 19:00:04 2016
From: mtalbott at lji.org (Michael Talbott)
Date: Sat, 24 Sep 2016 12:00:04 -0700
Subject: [OmniOS-discuss] Reversed All-In-One
Message-ID: <D4F51F44-B4A6-49B3-8180-88A0ECD44EE9@lji.org>

So I just had this crazy idea while thinking about lx branded zones.. I'm loving lx zones. What a great piece of computing voodoo. I'd love to say there's no more need for things like ESX in an all in one setup, but, there's still windows and some random others that has their niche needs on a box.

Anyway, I know there's plenty of folks out there that use ESXi as the root system then virtualize OmniOS and pass through storage controller cards to it to become a storage VM for their other VMs. I started thinking "what-if" we reversed that process to get the best of ALL worlds. That is, run OmniOS as root system and control storage natively and run all linux needs in zones. Then if, and a big if, we could run ESXi in an lx zone.. And IF that is even possible, another big if.. IF we can present a zone with a ZFS volume as a raw block device, ESXi could then use it's native VMFS format on said block device. If that were so, that would eliminate all traditional latencies and bottlenecks caused by the network stack be it iSCSI or NFS and provide all the benefits of ZFS such as checksums and compression inline to ESX.. Obviously it couldn't vmotion in a cluster since the storage would be local to that box, but, the thought is currently for an all in one setup and not a cluster ;)

So I guess the first question is, does anyone think that ESXi's kernel is anywhere close enough to a linux kernel for this to even be feasible to run in an LX zone or am I totally just daydreaming here? I have my suspicions, but, that's all they are at this point.

Second, and regardless of the above, does anyone have any good info on how to create a custom lx image? Surely it's not as simple as installing it on some box and then use tar and/or dd?


Michael

From paul.jochum at nokia.com  Sun Sep 25 14:36:26 2016
From: paul.jochum at nokia.com (Paul Jochum)
Date: Sun, 25 Sep 2016 09:36:26 -0500
Subject: [OmniOS-discuss] Reversed All-In-One
In-Reply-To: <D4F51F44-B4A6-49B3-8180-88A0ECD44EE9@lji.org>
References: <D4F51F44-B4A6-49B3-8180-88A0ECD44EE9@lji.org>
Message-ID: <d7269495-87eb-d2a3-335b-6ba6ccf367e9@nokia.com>

Hi Michael:

     As to your 2nd qustion, about having good info on how to create a 
custom lx image, I have used the following for smartOS: 
https://github.com/joyent/centos-lx-brand-image-builder
     (I am not sure how closely OmniOS tracks the smartOS lx branded 
zones, but wanted to pass it along in case it helps).  There are more 
links (in case you don't want to use centos as the base at:
https://wiki.smartos.org/display/DOC/LX+Branded+Zones#LXBrandedZones-Creatingyourownzonedataset

Paul


On 09/24/2016 02:00 PM, Michael Talbott wrote:
> So I just had this crazy idea while thinking about lx branded zones.. I'm loving lx zones. What a great piece of computing voodoo. I'd love to say there's no more need for things like ESX in an all in one setup, but, there's still windows and some random others that has their niche needs on a box.
>
> Anyway, I know there's plenty of folks out there that use ESXi as the root system then virtualize OmniOS and pass through storage controller cards to it to become a storage VM for their other VMs. I started thinking "what-if" we reversed that process to get the best of ALL worlds. That is, run OmniOS as root system and control storage natively and run all linux needs in zones. Then if, and a big if, we could run ESXi in an lx zone.. And IF that is even possible, another big if.. IF we can present a zone with a ZFS volume as a raw block device, ESXi could then use it's native VMFS format on said block device. If that were so, that would eliminate all traditional latencies and bottlenecks caused by the network stack be it iSCSI or NFS and provide all the benefits of ZFS such as checksums and compression inline to ESX.. Obviously it couldn't vmotion in a cluster since the storage would be local to that box, but, the thought is currently for an all in one setup and not a cluster!
>    ;)
>
> So I guess the first question is, does anyone think that ESXi's kernel is anywhere close enough to a linux kernel for this to even be feasible to run in an LX zone or am I totally just daydreaming here? I have my suspicions, but, that's all they are at this point.
>
> Second, and regardless of the above, does anyone have any good info on how to create a custom lx image? Surely it's not as simple as installing it on some box and then use tar and/or dd?
>
>
> Michael
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss



From jimklimov at cos.ru  Sun Sep 25 16:48:13 2016
From: jimklimov at cos.ru (Jim Klimov)
Date: Sun, 25 Sep 2016 18:48:13 +0200
Subject: [OmniOS-discuss] Reversed All-In-One
In-Reply-To: <d7269495-87eb-d2a3-335b-6ba6ccf367e9@nokia.com>
References: <D4F51F44-B4A6-49B3-8180-88A0ECD44EE9@lji.org>
	<d7269495-87eb-d2a3-335b-6ba6ccf367e9@nokia.com>
Message-ID: <7179CAB7-F16D-4014-8284-DDE77703BDE0@cos.ru>

25 ???????? 2016??. 16:36:26 CEST, Paul Jochum <paul.jochum at nokia.com> ?????:
>Hi Michael:
>
>     As to your 2nd qustion, about having good info on how to create a 
>custom lx image, I have used the following for smartOS: 
>https://github.com/joyent/centos-lx-brand-image-builder
>     (I am not sure how closely OmniOS tracks the smartOS lx branded 
>zones, but wanted to pass it along in case it helps).  There are more 
>links (in case you don't want to use centos as the base at:
>https://wiki.smartos.org/display/DOC/LX+Branded+Zones#LXBrandedZones-Creatingyourownzonedataset
>
>Paul
>
>
>On 09/24/2016 02:00 PM, Michael Talbott wrote:
>> So I just had this crazy idea while thinking about lx branded zones..
>I'm loving lx zones. What a great piece of computing voodoo. I'd love
>to say there's no more need for things like ESX in an all in one setup,
>but, there's still windows and some random others that has their niche
>needs on a box.
>>
>> Anyway, I know there's plenty of folks out there that use ESXi as the
>root system then virtualize OmniOS and pass through storage controller
>cards to it to become a storage VM for their other VMs. I started
>thinking "what-if" we reversed that process to get the best of ALL
>worlds. That is, run OmniOS as root system and control storage natively
>and run all linux needs in zones. Then if, and a big if, we could run
>ESXi in an lx zone.. And IF that is even possible, another big if.. IF
>we can present a zone with a ZFS volume as a raw block device, ESXi
>could then use it's native VMFS format on said block device. If that
>were so, that would eliminate all traditional latencies and bottlenecks
>caused by the network stack be it iSCSI or NFS and provide all the
>benefits of ZFS such as checksums and compression inline to ESX..
>Obviously it couldn't vmotion in a cluster since the storage would be
>local to that box, but, the thought is currently for an all in one
>setup and not a clust!
> er!
>>    ;)
>>
>> So I guess the first question is, does anyone think that ESXi's
>kernel is anywhere close enough to a linux kernel for this to even be
>feasible to run in an LX zone or am I totally just daydreaming here? I
>have my suspicions, but, that's all they are at this point.
>>
>> Second, and regardless of the above, does anyone have any good info
>on how to create a custom lx image? Surely it's not as simple as
>installing it on some box and then use tar and/or dd?
>>
>>
>> Michael
>> _______________________________________________
>> OmniOS-discuss mailing list
>> OmniOS-discuss at lists.omniti.com
>> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>
>
>_______________________________________________
>OmniOS-discuss mailing list
>OmniOS-discuss at lists.omniti.com
>http://lists.omniti.com/mailman/listinfo/omnios-discuss

As far as I know, ESX used a custom kernel as hypervosor (closer to BSD than Linux, perhaps because the latter would have to get opensourced). The linuxish environment seen in older (3.x) versions and hidden but uncoverable in newer ones is a controlling VM (IIRC, may be wrong here) allowed to grab the console and control the HW resources to present them back to hypervisor, and indeed based on Linux (quite compatible to redhat back then). To the point where I could compile kernel modules like md-raid and drivers to use setups other than certified officially.

I guess the entanglement there is too intimate to port elsewhere, and why would you - given that you can run KVMs or VirtualBoxes natively in a global or local zone? ;)

Jim
--
Typos courtesy of K-9 Mail on my Samsung Android

From jimklimov at cos.ru  Sun Sep 25 16:55:51 2016
From: jimklimov at cos.ru (Jim Klimov)
Date: Sun, 25 Sep 2016 18:55:51 +0200
Subject: [OmniOS-discuss] Bloody Update for September 21st & LX
 update/question
In-Reply-To: <C459D3C3A9FD081A.4344D62A-44BE-4893-B872-5BE8EE41A992@mail.outlook.com>
References: <7C1C6107-5405-4F40-84DD-7BF0076FD918@kebe.com>
	<48EF546C-ED7A-4A39-8310-68D360DEB359@omniti.com>
	<C459D3C3A9FD081A.4344D62A-44BE-4893-B872-5BE8EE41A992@mail.outlook.com>
Message-ID: <CA0682FA-DB74-4905-BCFB-2EEA20A5F3FD@cos.ru>

21 ???????? 2016??. 23:08:00 CEST, Dain Bentley <dain.bentley at gmail.com> ?????:
>I use it
>
>Get Outlook for iOS
>
>
>
>
>On Wed, Sep 21, 2016 at 4:51 PM -0400, "Dan McDonald"
><danmcd at omniti.com> wrote:
>
>
>
>
>
>
>
>
>
>
>Hello!
>
>I'm starting to wind down LX for now, and wind up the r151020 release.
>?I'm leaning toward folding in LX support (see below) into stock OmniOS
>for r151020 as "LX Beta", with fixes and supportability showing up in
>r151022 (next LTS).
>
>uname -a says omnios-master-a10017b, which means illumos-omnios was at
>commit a10017b, and omnios-build was at commit a0dbf1f.
>
>New with this bloody update:
>
>- Bash is now 4.4
>
>- OpenSSH support/SunSSH EOL is now easier to upgrade to. ?Instead of
>--reject lines, if you have SunSSH installed, you'll still *SEE* those
>packages, but they are now empty and "renamed" to the OpenSSH packages.
>?Post-upgrade "pkg list | grep ssh" can show you what I mean if you're
>still on SunSSH.
>
>- Curl is now 7.50.3 (and was updated earlier as part of the security
>fix)
>
>- SEVERAL NVMe fixes.
>
>- nlockmgr now more reliable at boot time (7231 nlockmgr failing to
>start up during bootup)
>
>- Various ZFS bugfixes
>
>
>
>ALSO UPDATED is the bloody+LX repo: ?http://pkg.omniti.com/omnios/LX/.
>?It is updated to illumos-joyent changes since their commit 6a41e72,
>and now quiesced. ?One bug was also fixed... SMAP support on Broadwell
>and newer Intel processors broke LX zones on OmniOS, because of a
>mismerge.
>
>I'M VERY CLOSE to folding in LX support into illumos-omnios's master
>branch, prior to construction of the r151020 branch. ?If people have
>opinions on this topic, I'd very much like to hear it. ?Also, is anyone
>besides OmniTI developers using LX for OmniOS?
>
>Thanks,
>Dan
>
>
>
>
>
>
>
>
>------------------------------------------------------------------------
>
>_______________________________________________
>OmniOS-discuss mailing list
>OmniOS-discuss at lists.omniti.com
>http://lists.omniti.com/mailman/listinfo/omnios-discuss

I play with it. :)

Given desktop (laptop) usage of OI Hipster, I also honestly look forward to when LX stabilizes in OmniOS so I can snatch it and use also while commuting, experimenting or otherwise not near an OmniOS server ;)

Jim
--
Typos courtesy of K-9 Mail on my Samsung Android

From garland at bustle.com  Sun Sep 25 17:02:49 2016
From: garland at bustle.com (Garland McAlexander)
Date: Sun, 25 Sep 2016 13:02:49 -0400
Subject: [OmniOS-discuss] Installation Failing (Reboots)
In-Reply-To: <alpine.GSO.2.20.1609231852460.5577@freddy.simplesystems.org>
References: <CAMUDemvCkBEDLp_2ePQB+H=B=UnEOPf-EeqVRFPqmYGT___rFg@mail.gmail.com>
	<alpine.GSO.2.20.1609231852460.5577@freddy.simplesystems.org>
Message-ID: <CAMUDemvDxRT_xapdaqzfJgcAH6mTUehf2D2SVtT0g5X0sAe3TA@mail.gmail.com>

So would a workaround be installing it via a USB2.0 drive, or will USB not
work at all due to the USB3.0 slot?

It seems odd that it would get to the point of loading, only to fail if it
was USB related.

On Fri, Sep 23, 2016 at 7:53 PM, Bob Friesenhahn <
bfriesen at simple.dallas.tx.us> wrote:

> On Fri, 23 Sep 2016, Garland McAlexander wrote:
>
> Hi All,
>>
>> Trying to install the latest stable Omnios via USB.
>>
>> After booting from USB, I am greeted by the Omnios logo. One that proceeds
>> into another screen, it loads a few things, gets to "syncing system files"
>> and immediately after that something flashes (unable to see the text, even
>> when spamming pause/break) and my server reboots.
>>
>> I have tried to recreate the USB multiple times, and I've done MD5
>> checking
>> on my download.
>>
>> Any assistance would be absolutely fantastic.
>>
>
> The most likely cause of the problem would be due to using USB3 hardware
> with an OS which only supports USB1 and USB2.
>
> Bob
> --
> Bob Friesenhahn
> bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
> GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20160925/a0a7c456/attachment.html>

From danmcd at omniti.com  Sun Sep 25 19:34:51 2016
From: danmcd at omniti.com (Dan McDonald)
Date: Sun, 25 Sep 2016 15:34:51 -0400
Subject: [OmniOS-discuss] Reversed All-In-One
In-Reply-To: <d7269495-87eb-d2a3-335b-6ba6ccf367e9@nokia.com>
References: <D4F51F44-B4A6-49B3-8180-88A0ECD44EE9@lji.org>
	<d7269495-87eb-d2a3-335b-6ba6ccf367e9@nokia.com>
Message-ID: <EA4DBB9B-9FF9-404A-9381-F6DBA3C72874@omniti.com>


> On Sep 25, 2016, at 10:36 AM, Paul Jochum <paul.jochum at nokia.com> wrote:
> 
> 
>    As to your 2nd qustion, about having good info on how to create a custom lx image, I have used the following for smartOS: https://github.com/joyent/centos-lx-brand-image-builder
>    (I am not sure how closely OmniOS tracks the smartOS lx branded zones, but wanted to pass it along in case it helps).  There are more links (in case you don't want to use centos as the base at:
> https://wiki.smartos.org/display/DOC/LX+Branded+Zones#LXBrandedZones-Creatingyourownzonedataset

The LX Beta for OmniOS (r151020 will have LX Beta in it) has its install script capable of ZFS send streams (-s with an absolute pathname) or ZFS snapshots-or-datasets (-s using a zfs dataset name), or tarballs (-t).  The -s data was intended to be used with either SmartOS compressed ZFS send streams or datasets managed by SmartOS's imgadm(1M) command.

Dan


From danmcd at omniti.com  Sun Sep 25 19:36:01 2016
From: danmcd at omniti.com (Dan McDonald)
Date: Sun, 25 Sep 2016 15:36:01 -0400
Subject: [OmniOS-discuss] Installation Failing (Reboots)
In-Reply-To: <CAMUDemvDxRT_xapdaqzfJgcAH6mTUehf2D2SVtT0g5X0sAe3TA@mail.gmail.com>
References: <CAMUDemvCkBEDLp_2ePQB+H=B=UnEOPf-EeqVRFPqmYGT___rFg@mail.gmail.com>
	<alpine.GSO.2.20.1609231852460.5577@freddy.simplesystems.org>
	<CAMUDemvDxRT_xapdaqzfJgcAH6mTUehf2D2SVtT0g5X0sAe3TA@mail.gmail.com>
Message-ID: <942E11EF-14FC-479C-92D6-8C19ABBAD789@omniti.com>


> On Sep 25, 2016, at 1:02 PM, Garland McAlexander <garland at bustle.com> wrote:
> 
> So would a workaround be installing it via a USB2.0 drive, or will USB not work at all due to the USB3.0 slot?

The USB3.0 slot presents as currently-unsupported XHCI.  You either need to convinced your bios to do EHCI (USB2) or wait for XHCI support to land in illumos & OmniOS.

Dan


From matej at zunaj.si  Mon Sep 26 07:04:25 2016
From: matej at zunaj.si (=?utf-8?Q?Matej_=C5=BDerovnik?=)
Date: Mon, 26 Sep 2016 09:04:25 +0200
Subject: [OmniOS-discuss] Installation Failing (Reboots)
In-Reply-To: <942E11EF-14FC-479C-92D6-8C19ABBAD789@omniti.com>
References: <CAMUDemvCkBEDLp_2ePQB+H=B=UnEOPf-EeqVRFPqmYGT___rFg@mail.gmail.com>
	<alpine.GSO.2.20.1609231852460.5577@freddy.simplesystems.org>
	<CAMUDemvDxRT_xapdaqzfJgcAH6mTUehf2D2SVtT0g5X0sAe3TA@mail.gmail.com>
	<942E11EF-14FC-479C-92D6-8C19ABBAD789@omniti.com>
Message-ID: <EB2990FA-5E81-4DE0-8961-ACB7D14070F1@zunaj.si>

Usually one can disable USB3 support in BIOS and then it should work.

Matej

> On 25 Sep 2016, at 21:36, Dan McDonald <danmcd at omniti.com> wrote:
> 
> 
>> On Sep 25, 2016, at 1:02 PM, Garland McAlexander <garland at bustle.com> wrote:
>> 
>> So would a workaround be installing it via a USB2.0 drive, or will USB not work at all due to the USB3.0 slot?
> 
> The USB3.0 slot presents as currently-unsupported XHCI.  You either need to convinced your bios to do EHCI (USB2) or wait for XHCI support to land in illumos & OmniOS.
> 
> Dan
> 
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss


From danmcd at omniti.com  Mon Sep 26 14:45:49 2016
From: danmcd at omniti.com (Dan McDonald)
Date: Mon, 26 Sep 2016 10:45:49 -0400
Subject: [OmniOS-discuss] HEADS UP --> ANOTHER OpenSSL update
References: <8CF9A138-CFCC-47B9-82AE-03E7217D2FE2@omniti.com>
Message-ID: <157A1C64-B0CD-43B5-815B-AD5B9985EBEE@omniti.com>

Say hello to 1.0.2j.  Thank goodness I follow security types on twitter or I'd have missed this pre-OpenZFS.  (Anyone on this list who's AT OpenZFS today, say hi!  I'm wearing a "There's no place like usr/src/uts" shirt.)

All of the repos have updated (and signed for 01[468]) OpenSSL 1.0.2j packages now.

Happy re-updating!
Dan


From garland at bustle.com  Mon Sep 26 15:04:23 2016
From: garland at bustle.com (Garland McAlexander)
Date: Mon, 26 Sep 2016 11:04:23 -0400
Subject: [OmniOS-discuss] Installation Failing (Reboots)
In-Reply-To: <EB2990FA-5E81-4DE0-8961-ACB7D14070F1@zunaj.si>
References: <CAMUDemvCkBEDLp_2ePQB+H=B=UnEOPf-EeqVRFPqmYGT___rFg@mail.gmail.com>
	<alpine.GSO.2.20.1609231852460.5577@freddy.simplesystems.org>
	<CAMUDemvDxRT_xapdaqzfJgcAH6mTUehf2D2SVtT0g5X0sAe3TA@mail.gmail.com>
	<942E11EF-14FC-479C-92D6-8C19ABBAD789@omniti.com>
	<EB2990FA-5E81-4DE0-8961-ACB7D14070F1@zunaj.si>
Message-ID: <CAMUDemufri7o2_gH=fVeDe=g0BNjuJ+a01YKbuS+FKBOc2QY8A@mail.gmail.com>

I've disabled all USB 3.0 support in BIOS, but it's still failing to boot
from USB. I still get the splash screen, and then a few lines of text until
it reboots itself.

On Mon, Sep 26, 2016 at 3:04 AM, Matej ?erovnik <matej at zunaj.si> wrote:

> Usually one can disable USB3 support in BIOS and then it should work.
>
> Matej
>
> > On 25 Sep 2016, at 21:36, Dan McDonald <danmcd at omniti.com> wrote:
> >
> >
> >> On Sep 25, 2016, at 1:02 PM, Garland McAlexander <garland at bustle.com>
> wrote:
> >>
> >> So would a workaround be installing it via a USB2.0 drive, or will USB
> not work at all due to the USB3.0 slot?
> >
> > The USB3.0 slot presents as currently-unsupported XHCI.  You either need
> to convinced your bios to do EHCI (USB2) or wait for XHCI support to land
> in illumos & OmniOS.
> >
> > Dan
> >
> > _______________________________________________
> > OmniOS-discuss mailing list
> > OmniOS-discuss at lists.omniti.com
> > http://lists.omniti.com/mailman/listinfo/omnios-discuss
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20160926/1207d002/attachment.html>

From daleg at omniti.com  Mon Sep 26 15:32:26 2016
From: daleg at omniti.com (Dale Ghent)
Date: Mon, 26 Sep 2016 11:32:26 -0400
Subject: [OmniOS-discuss] Installation Failing (Reboots)
In-Reply-To: <CAMUDemufri7o2_gH=fVeDe=g0BNjuJ+a01YKbuS+FKBOc2QY8A@mail.gmail.com>
References: <CAMUDemvCkBEDLp_2ePQB+H=B=UnEOPf-EeqVRFPqmYGT___rFg@mail.gmail.com>
	<alpine.GSO.2.20.1609231852460.5577@freddy.simplesystems.org>
	<CAMUDemvDxRT_xapdaqzfJgcAH6mTUehf2D2SVtT0g5X0sAe3TA@mail.gmail.com>
	<942E11EF-14FC-479C-92D6-8C19ABBAD789@omniti.com>
	<EB2990FA-5E81-4DE0-8961-ACB7D14070F1@zunaj.si>
	<CAMUDemufri7o2_gH=fVeDe=g0BNjuJ+a01YKbuS+FKBOc2QY8A@mail.gmail.com>
Message-ID: <21FA36AF-0B6F-4FC3-A48E-0EA0A024A835@omniti.com>


2 options until the USB3 XHCI driver is ready (which it is close to being so)

1) Put in a USB 2 PCIe add-on card at least temporarily.
2) If your box has IPMI and virtual CDROM/remote disk KVM services, use those instead.

> On Sep 26, 2016, at 11:04 AM, Garland McAlexander <garland at bustle.com> wrote:
> 
> I've disabled all USB 3.0 support in BIOS, but it's still failing to boot from USB. I still get the splash screen, and then a few lines of text until it reboots itself. 
> 
> On Mon, Sep 26, 2016 at 3:04 AM, Matej ?erovnik <matej at zunaj.si> wrote:
> Usually one can disable USB3 support in BIOS and then it should work.
> 
> Matej
> 
> > On 25 Sep 2016, at 21:36, Dan McDonald <danmcd at omniti.com> wrote:
> >
> >
> >> On Sep 25, 2016, at 1:02 PM, Garland McAlexander <garland at bustle.com> wrote:
> >>
> >> So would a workaround be installing it via a USB2.0 drive, or will USB not work at all due to the USB3.0 slot?
> >
> > The USB3.0 slot presents as currently-unsupported XHCI.  You either need to convinced your bios to do EHCI (USB2) or wait for XHCI support to land in illumos & OmniOS.
> >
> > Dan
> >
> > _______________________________________________
> > OmniOS-discuss mailing list
> > OmniOS-discuss at lists.omniti.com
> > http://lists.omniti.com/mailman/listinfo/omnios-discuss
> 
> 
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss


From lkateley at kateley.com  Mon Sep 26 15:54:07 2016
From: lkateley at kateley.com (Linda Kateley)
Date: Mon, 26 Sep 2016 10:54:07 -0500
Subject: [OmniOS-discuss] Reversed All-In-One
In-Reply-To: <7179CAB7-F16D-4014-8284-DDE77703BDE0@cos.ru>
References: <D4F51F44-B4A6-49B3-8180-88A0ECD44EE9@lji.org>
	<d7269495-87eb-d2a3-335b-6ba6ccf367e9@nokia.com>
	<7179CAB7-F16D-4014-8284-DDE77703BDE0@cos.ru>
Message-ID: <aca79c1d-e1ca-8f06-4a60-d58a3e45d2ae@kateley.com>

+ I run with virtual box on top of omni. Also really digging Proxmox 
which has native zfs.

Linda


On 9/25/16 11:48 AM, Jim Klimov wrote:
> 25 ???????? 2016 ?. 16:36:26 CEST, Paul Jochum <paul.jochum at nokia.com> ?????:
>> Hi Michael:
>>
>>      As to your 2nd qustion, about having good info on how to create a
>> custom lx image, I have used the following for smartOS:
>> https://github.com/joyent/centos-lx-brand-image-builder
>>      (I am not sure how closely OmniOS tracks the smartOS lx branded
>> zones, but wanted to pass it along in case it helps).  There are more
>> links (in case you don't want to use centos as the base at:
>> https://wiki.smartos.org/display/DOC/LX+Branded+Zones#LXBrandedZones-Creatingyourownzonedataset
>>
>> Paul
>>
>>
>> On 09/24/2016 02:00 PM, Michael Talbott wrote:
>>> So I just had this crazy idea while thinking about lx branded zones..
>> I'm loving lx zones. What a great piece of computing voodoo. I'd love
>> to say there's no more need for things like ESX in an all in one setup,
>> but, there's still windows and some random others that has their niche
>> needs on a box.
>>> Anyway, I know there's plenty of folks out there that use ESXi as the
>> root system then virtualize OmniOS and pass through storage controller
>> cards to it to become a storage VM for their other VMs. I started
>> thinking "what-if" we reversed that process to get the best of ALL
>> worlds. That is, run OmniOS as root system and control storage natively
>> and run all linux needs in zones. Then if, and a big if, we could run
>> ESXi in an lx zone.. And IF that is even possible, another big if.. IF
>> we can present a zone with a ZFS volume as a raw block device, ESXi
>> could then use it's native VMFS format on said block device. If that
>> were so, that would eliminate all traditional latencies and bottlenecks
>> caused by the network stack be it iSCSI or NFS and provide all the
>> benefits of ZFS such as checksums and compression inline to ESX..
>> Obviously it couldn't vmotion in a cluster since the storage would be
>> local to that box, but, the thought is currently for an all in one
>> setup and not a clust!
>> er!
>>>     ;)
>>>
>>> So I guess the first question is, does anyone think that ESXi's
>> kernel is anywhere close enough to a linux kernel for this to even be
>> feasible to run in an LX zone or am I totally just daydreaming here? I
>> have my suspicions, but, that's all they are at this point.
>>> Second, and regardless of the above, does anyone have any good info
>> on how to create a custom lx image? Surely it's not as simple as
>> installing it on some box and then use tar and/or dd?
>>>
>>> Michael
>>> _______________________________________________
>>> OmniOS-discuss mailing list
>>> OmniOS-discuss at lists.omniti.com
>>> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>>
>> _______________________________________________
>> OmniOS-discuss mailing list
>> OmniOS-discuss at lists.omniti.com
>> http://lists.omniti.com/mailman/listinfo/omnios-discuss
> As far as I know, ESX used a custom kernel as hypervosor (closer to BSD than Linux, perhaps because the latter would have to get opensourced). The linuxish environment seen in older (3.x) versions and hidden but uncoverable in newer ones is a controlling VM (IIRC, may be wrong here) allowed to grab the console and control the HW resources to present them back to hypervisor, and indeed based on Linux (quite compatible to redhat back then). To the point where I could compile kernel modules like md-raid and drivers to use setups other than certified officially.
>
> I guess the entanglement there is too intimate to port elsewhere, and why would you - given that you can run KVMs or VirtualBoxes natively in a global or local zone? ;)
>
> Jim
> --
> Typos courtesy of K-9 Mail on my Samsung Android
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss


From garland at bustle.com  Mon Sep 26 15:56:03 2016
From: garland at bustle.com (Garland McAlexander)
Date: Mon, 26 Sep 2016 11:56:03 -0400
Subject: [OmniOS-discuss] Installation Failing (Reboots)
In-Reply-To: <21FA36AF-0B6F-4FC3-A48E-0EA0A024A835@omniti.com>
References: <CAMUDemvCkBEDLp_2ePQB+H=B=UnEOPf-EeqVRFPqmYGT___rFg@mail.gmail.com>
	<alpine.GSO.2.20.1609231852460.5577@freddy.simplesystems.org>
	<CAMUDemvDxRT_xapdaqzfJgcAH6mTUehf2D2SVtT0g5X0sAe3TA@mail.gmail.com>
	<942E11EF-14FC-479C-92D6-8C19ABBAD789@omniti.com>
	<EB2990FA-5E81-4DE0-8961-ACB7D14070F1@zunaj.si>
	<CAMUDemufri7o2_gH=fVeDe=g0BNjuJ+a01YKbuS+FKBOc2QY8A@mail.gmail.com>
	<21FA36AF-0B6F-4FC3-A48E-0EA0A024A835@omniti.com>
Message-ID: <CAMUDemudWTH7SLFm8qAU+1aKde0xWoBvi4PP8K4GEz_cdOtmLw@mail.gmail.com>

Just attempted via IPMI CDROM and the same issue persists, gets to splash
screen, loads a few lines of text and forces a reboot.

On Mon, Sep 26, 2016 at 11:32 AM, Dale Ghent <daleg at omniti.com> wrote:

>
> 2 options until the USB3 XHCI driver is ready (which it is close to being
> so)
>
> 1) Put in a USB 2 PCIe add-on card at least temporarily.
> 2) If your box has IPMI and virtual CDROM/remote disk KVM services, use
> those instead.
>
> > On Sep 26, 2016, at 11:04 AM, Garland McAlexander <garland at bustle.com>
> wrote:
> >
> > I've disabled all USB 3.0 support in BIOS, but it's still failing to
> boot from USB. I still get the splash screen, and then a few lines of text
> until it reboots itself.
> >
> > On Mon, Sep 26, 2016 at 3:04 AM, Matej ?erovnik <matej at zunaj.si> wrote:
> > Usually one can disable USB3 support in BIOS and then it should work.
> >
> > Matej
> >
> > > On 25 Sep 2016, at 21:36, Dan McDonald <danmcd at omniti.com> wrote:
> > >
> > >
> > >> On Sep 25, 2016, at 1:02 PM, Garland McAlexander <garland at bustle.com>
> wrote:
> > >>
> > >> So would a workaround be installing it via a USB2.0 drive, or will
> USB not work at all due to the USB3.0 slot?
> > >
> > > The USB3.0 slot presents as currently-unsupported XHCI.  You either
> need to convinced your bios to do EHCI (USB2) or wait for XHCI support to
> land in illumos & OmniOS.
> > >
> > > Dan
> > >
> > > _______________________________________________
> > > OmniOS-discuss mailing list
> > > OmniOS-discuss at lists.omniti.com
> > > http://lists.omniti.com/mailman/listinfo/omnios-discuss
> >
> >
> > _______________________________________________
> > OmniOS-discuss mailing list
> > OmniOS-discuss at lists.omniti.com
> > http://lists.omniti.com/mailman/listinfo/omnios-discuss
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20160926/e577a407/attachment-0001.html>

From mir at miras.org  Mon Sep 26 16:01:47 2016
From: mir at miras.org (Michael Rasmussen)
Date: Mon, 26 Sep 2016 18:01:47 +0200
Subject: [OmniOS-discuss] Installation Failing (Reboots)
In-Reply-To: <CAMUDemudWTH7SLFm8qAU+1aKde0xWoBvi4PP8K4GEz_cdOtmLw@mail.gmail.com>
References: <CAMUDemvCkBEDLp_2ePQB+H=B=UnEOPf-EeqVRFPqmYGT___rFg@mail.gmail.com>
	<alpine.GSO.2.20.1609231852460.5577@freddy.simplesystems.org>
	<CAMUDemvDxRT_xapdaqzfJgcAH6mTUehf2D2SVtT0g5X0sAe3TA@mail.gmail.com>
	<942E11EF-14FC-479C-92D6-8C19ABBAD789@omniti.com>
	<EB2990FA-5E81-4DE0-8961-ACB7D14070F1@zunaj.si>
	<CAMUDemufri7o2_gH=fVeDe=g0BNjuJ+a01YKbuS+FKBOc2QY8A@mail.gmail.com>
	<21FA36AF-0B6F-4FC3-A48E-0EA0A024A835@omniti.com>
	<CAMUDemudWTH7SLFm8qAU+1aKde0xWoBvi4PP8K4GEz_cdOtmLw@mail.gmail.com>
Message-ID: <20160926180147.58c7e5d3@sleipner.datanom.net>

On Mon, 26 Sep 2016 11:56:03 -0400
Garland McAlexander <garland at bustle.com> wrote:

> Just attempted via IPMI CDROM and the same issue persists, gets to splash
> screen, loads a few lines of text and forces a reboot.
> 
If that's the case then it can hardly be a USB-only issue. Maybe the
chipset on the motherboard or the CPU is to new?

What chipset and CPU is on the board?

-- 
Hilsen/Regards
Michael Rasmussen

Get my public GnuPG keys:
michael <at> rasmussen <dot> cc
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E
mir <at> datanom <dot> net
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C
mir <at> miras <dot> org
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917
--------------------------------------------------------------
/usr/games/fortune -es says:
Make sure your code "does nothing" gracefully.
            - The Elements of Programming Style (Kernighan & Plaugher)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 163 bytes
Desc: OpenPGP digital signature
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20160926/69a10411/attachment.bin>

From garland at bustle.com  Mon Sep 26 16:03:26 2016
From: garland at bustle.com (Garland McAlexander)
Date: Mon, 26 Sep 2016 12:03:26 -0400
Subject: [OmniOS-discuss] Installation Failing (Reboots)
In-Reply-To: <20160926180147.58c7e5d3@sleipner.datanom.net>
References: <CAMUDemvCkBEDLp_2ePQB+H=B=UnEOPf-EeqVRFPqmYGT___rFg@mail.gmail.com>
	<alpine.GSO.2.20.1609231852460.5577@freddy.simplesystems.org>
	<CAMUDemvDxRT_xapdaqzfJgcAH6mTUehf2D2SVtT0g5X0sAe3TA@mail.gmail.com>
	<942E11EF-14FC-479C-92D6-8C19ABBAD789@omniti.com>
	<EB2990FA-5E81-4DE0-8961-ACB7D14070F1@zunaj.si>
	<CAMUDemufri7o2_gH=fVeDe=g0BNjuJ+a01YKbuS+FKBOc2QY8A@mail.gmail.com>
	<21FA36AF-0B6F-4FC3-A48E-0EA0A024A835@omniti.com>
	<CAMUDemudWTH7SLFm8qAU+1aKde0xWoBvi4PP8K4GEz_cdOtmLw@mail.gmail.com>
	<20160926180147.58c7e5d3@sleipner.datanom.net>
Message-ID: <CAMUDemueiho3AVwHPS8Xtra=K924QtSRKwkE2XYVx1t1QXaFWw@mail.gmail.com>

The board is a Supermicro X10DRW-iT which is a C612 chipset.

On Mon, Sep 26, 2016 at 12:01 PM, Michael Rasmussen <mir at miras.org> wrote:

> On Mon, 26 Sep 2016 11:56:03 -0400
> Garland McAlexander <garland at bustle.com> wrote:
>
> > Just attempted via IPMI CDROM and the same issue persists, gets to splash
> > screen, loads a few lines of text and forces a reboot.
> >
> If that's the case then it can hardly be a USB-only issue. Maybe the
> chipset on the motherboard or the CPU is to new?
>
> What chipset and CPU is on the board?
>
> --
> Hilsen/Regards
> Michael Rasmussen
>
> Get my public GnuPG keys:
> michael <at> rasmussen <dot> cc
> http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E
> mir <at> datanom <dot> net
> http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C
> mir <at> miras <dot> org
> http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917
> --------------------------------------------------------------
> /usr/games/fortune -es says:
> Make sure your code "does nothing" gracefully.
>             - The Elements of Programming Style (Kernighan & Plaugher)
>
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20160926/e8b304bb/attachment.html>

From mir at miras.org  Mon Sep 26 16:07:33 2016
From: mir at miras.org (Michael Rasmussen)
Date: Mon, 26 Sep 2016 18:07:33 +0200
Subject: [OmniOS-discuss] Installation Failing (Reboots)
In-Reply-To: <CAMUDemueiho3AVwHPS8Xtra=K924QtSRKwkE2XYVx1t1QXaFWw@mail.gmail.com>
References: <CAMUDemvCkBEDLp_2ePQB+H=B=UnEOPf-EeqVRFPqmYGT___rFg@mail.gmail.com>
	<alpine.GSO.2.20.1609231852460.5577@freddy.simplesystems.org>
	<CAMUDemvDxRT_xapdaqzfJgcAH6mTUehf2D2SVtT0g5X0sAe3TA@mail.gmail.com>
	<942E11EF-14FC-479C-92D6-8C19ABBAD789@omniti.com>
	<EB2990FA-5E81-4DE0-8961-ACB7D14070F1@zunaj.si>
	<CAMUDemufri7o2_gH=fVeDe=g0BNjuJ+a01YKbuS+FKBOc2QY8A@mail.gmail.com>
	<21FA36AF-0B6F-4FC3-A48E-0EA0A024A835@omniti.com>
	<CAMUDemudWTH7SLFm8qAU+1aKde0xWoBvi4PP8K4GEz_cdOtmLw@mail.gmail.com>
	<20160926180147.58c7e5d3@sleipner.datanom.net>
	<CAMUDemueiho3AVwHPS8Xtra=K924QtSRKwkE2XYVx1t1QXaFWw@mail.gmail.com>
Message-ID: <20160926180733.5cee0873@sleipner.datanom.net>

On Mon, 26 Sep 2016 12:03:26 -0400
Garland McAlexander <garland at bustle.com> wrote:

> The board is a Supermicro X10DRW-iT which is a C612 chipset.
> 
Are you on latest BIOS firmware?

-- 
Hilsen/Regards
Michael Rasmussen

Get my public GnuPG keys:
michael <at> rasmussen <dot> cc
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E
mir <at> datanom <dot> net
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C
mir <at> miras <dot> org
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917
--------------------------------------------------------------
/usr/games/fortune -es says:
Absinthe makes the tart grow fonder.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 163 bytes
Desc: OpenPGP digital signature
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20160926/95f20f64/attachment.bin>

From garland at bustle.com  Mon Sep 26 16:10:41 2016
From: garland at bustle.com (Garland McAlexander)
Date: Mon, 26 Sep 2016 12:10:41 -0400
Subject: [OmniOS-discuss] Installation Failing (Reboots)
In-Reply-To: <20160926180733.5cee0873@sleipner.datanom.net>
References: <CAMUDemvCkBEDLp_2ePQB+H=B=UnEOPf-EeqVRFPqmYGT___rFg@mail.gmail.com>
	<alpine.GSO.2.20.1609231852460.5577@freddy.simplesystems.org>
	<CAMUDemvDxRT_xapdaqzfJgcAH6mTUehf2D2SVtT0g5X0sAe3TA@mail.gmail.com>
	<942E11EF-14FC-479C-92D6-8C19ABBAD789@omniti.com>
	<EB2990FA-5E81-4DE0-8961-ACB7D14070F1@zunaj.si>
	<CAMUDemufri7o2_gH=fVeDe=g0BNjuJ+a01YKbuS+FKBOc2QY8A@mail.gmail.com>
	<21FA36AF-0B6F-4FC3-A48E-0EA0A024A835@omniti.com>
	<CAMUDemudWTH7SLFm8qAU+1aKde0xWoBvi4PP8K4GEz_cdOtmLw@mail.gmail.com>
	<20160926180147.58c7e5d3@sleipner.datanom.net>
	<CAMUDemueiho3AVwHPS8Xtra=K924QtSRKwkE2XYVx1t1QXaFWw@mail.gmail.com>
	<20160926180733.5cee0873@sleipner.datanom.net>
Message-ID: <CAMUDemtaumY2KJmLxoPt1Z2++Y26DshWYnxCr0kMCraGCeWOpQ@mail.gmail.com>

I'm on 2.0a, which looks to be the latest version.

On Mon, Sep 26, 2016 at 12:07 PM, Michael Rasmussen <mir at miras.org> wrote:

> On Mon, 26 Sep 2016 12:03:26 -0400
> Garland McAlexander <garland at bustle.com> wrote:
>
> > The board is a Supermicro X10DRW-iT which is a C612 chipset.
> >
> Are you on latest BIOS firmware?
>
> --
> Hilsen/Regards
> Michael Rasmussen
>
> Get my public GnuPG keys:
> michael <at> rasmussen <dot> cc
> http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E
> mir <at> datanom <dot> net
> http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C
> mir <at> miras <dot> org
> http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917
> --------------------------------------------------------------
> /usr/games/fortune -es says:
> Absinthe makes the tart grow fonder.
>
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20160926/2a5d8d82/attachment.html>

From johan.kragsterman at capvert.se  Mon Sep 26 16:29:47 2016
From: johan.kragsterman at capvert.se (Johan Kragsterman)
Date: Mon, 26 Sep 2016 18:29:47 +0200
Subject: [OmniOS-discuss] Ang: Re:  Installation Failing (Reboots)
In-Reply-To: <CAMUDemtaumY2KJmLxoPt1Z2++Y26DshWYnxCr0kMCraGCeWOpQ@mail.gmail.com>
References: <CAMUDemtaumY2KJmLxoPt1Z2++Y26DshWYnxCr0kMCraGCeWOpQ@mail.gmail.com>,
	<CAMUDemvCkBEDLp_2ePQB+H=B=UnEOPf-EeqVRFPqmYGT___rFg@mail.gmail.com>	<alpine.GSO.2.20.1609231852460.5577@freddy.simplesystems.org>
	<CAMUDemvDxRT_xapdaqzfJgcAH6mTUehf2D2SVtT0g5X0sAe3TA@mail.gmail.com>	<942E11EF-14FC-479C-92D6-8C19ABBAD789@omniti.com>
	<EB2990FA-5E81-4DE0-8961-ACB7D14070F1@zunaj.si>	<CAMUDemufri7o2_gH=fVeDe=g0BNjuJ+a01YKbuS+FKBOc2QY8A@mail.gmail.com>
	<21FA36AF-0B6F-4FC3-A48E-0EA0A024A835@omniti.com>	<CAMUDemudWTH7SLFm8qAU+1aKde0xWoBvi4PP8K4GEz_cdOtmLw@mail.gmail.com>
	<20160926180147.58c7e5d3@sleipner.datanom.net>	<CAMUDemueiho3AVwHPS8Xtra=K924QtSRKwkE2XYVx1t1QXaFWw@mail.gmail.com>
	<20160926180733.5cee0873@sleipner.datanom.net>
Message-ID: <OFCAB0B4E0.364AE4AF-ONC125803A.005A9E26-C125803A.005A9E28@inse.com>


Hi!

I've seen this sometime, if I remember right, when I forgot to disable C-states.

So, have you checked that C-states is disabled?

Rgrds /J


-----"OmniOS-discuss" <omnios-discuss-bounces at lists.omniti.com> skrev: -----
Till: Michael Rasmussen <mir at miras.org>
Fr?n: Garland McAlexander 
S?nt av: "OmniOS-discuss" 
Datum: 2016-09-26 16:11
Kopia: omnios-discuss <omnios-discuss at lists.omniti.com>
?rende: Re: [OmniOS-discuss] Installation Failing (Reboots)

I'm on 2.0a, which looks to be the latest version.?

On Mon, Sep 26, 2016 at 12:07 PM, Michael Rasmussen <mir at miras.org> wrote:
On Mon, 26 Sep 2016 12:03:26 -0400
Garland McAlexander <garland at bustle.com> wrote:

> The board is a Supermicro X10DRW-iT which is a C612 chipset.
>
Are you on latest BIOS firmware?

--
Hilsen/Regards
Michael Rasmussen

Get my public GnuPG keys:
michael <at> rasmussen <dot> cc
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E
mir <at> datanom <dot> net
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C
mir <at> miras <dot> org
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917
--------------------------------------------------------------
/usr/games/fortune -es says:
Absinthe makes the tart grow fonder.

_______________________________________________
OmniOS-discuss mailing list
OmniOS-discuss at lists.omniti.com
http://lists.omniti.com/mailman/listinfo/omnios-discuss


_______________________________________________
OmniOS-discuss mailing list
OmniOS-discuss at lists.omniti.com
http://lists.omniti.com/mailman/listinfo/omnios-discuss




From garland at bustle.com  Mon Sep 26 16:34:31 2016
From: garland at bustle.com (Garland McAlexander)
Date: Mon, 26 Sep 2016 12:34:31 -0400
Subject: [OmniOS-discuss] Installation Failing (Reboots)
In-Reply-To: <OFCAB0B4E0.364AE4AF-ONC125803A.005A9E26-C125803A.005A9E28@inse.com>
References: <CAMUDemvCkBEDLp_2ePQB+H=B=UnEOPf-EeqVRFPqmYGT___rFg@mail.gmail.com>
	<alpine.GSO.2.20.1609231852460.5577@freddy.simplesystems.org>
	<CAMUDemvDxRT_xapdaqzfJgcAH6mTUehf2D2SVtT0g5X0sAe3TA@mail.gmail.com>
	<942E11EF-14FC-479C-92D6-8C19ABBAD789@omniti.com>
	<EB2990FA-5E81-4DE0-8961-ACB7D14070F1@zunaj.si>
	<CAMUDemufri7o2_gH=fVeDe=g0BNjuJ+a01YKbuS+FKBOc2QY8A@mail.gmail.com>
	<21FA36AF-0B6F-4FC3-A48E-0EA0A024A835@omniti.com>
	<CAMUDemudWTH7SLFm8qAU+1aKde0xWoBvi4PP8K4GEz_cdOtmLw@mail.gmail.com>
	<20160926180147.58c7e5d3@sleipner.datanom.net>
	<CAMUDemueiho3AVwHPS8Xtra=K924QtSRKwkE2XYVx1t1QXaFWw@mail.gmail.com>
	<20160926180733.5cee0873@sleipner.datanom.net>
	<CAMUDemtaumY2KJmLxoPt1Z2++Y26DshWYnxCr0kMCraGCeWOpQ@mail.gmail.com>
	<OFCAB0B4E0.364AE4AF-ONC125803A.005A9E26-C125803A.005A9E28@inse.com>
Message-ID: <CAMUDemvpmuK4dB29jpNr3EA70vHzAXjfBV+kfxOgdbEJNgdA_A@mail.gmail.com>

Hi,

I currently have all power technology (P, C, T) states disabled.

On Mon, Sep 26, 2016 at 12:29 PM, Johan Kragsterman <
johan.kragsterman at capvert.se> wrote:

>
> Hi!
>
> I've seen this sometime, if I remember right, when I forgot to disable
> C-states.
>
> So, have you checked that C-states is disabled?
>
> Rgrds /J
>
>
> -----"OmniOS-discuss" <omnios-discuss-bounces at lists.omniti.com> skrev:
> -----
> Till: Michael Rasmussen <mir at miras.org>
> Fr?n: Garland McAlexander
> S?nt av: "OmniOS-discuss"
> Datum: 2016-09-26 16:11
> Kopia: omnios-discuss <omnios-discuss at lists.omniti.com>
> ?rende: Re: [OmniOS-discuss] Installation Failing (Reboots)
>
> I'm on 2.0a, which looks to be the latest version.
>
> On Mon, Sep 26, 2016 at 12:07 PM, Michael Rasmussen <mir at miras.org> wrote:
> On Mon, 26 Sep 2016 12:03:26 -0400
> Garland McAlexander <garland at bustle.com> wrote:
>
> > The board is a Supermicro X10DRW-iT which is a C612 chipset.
> >
> Are you on latest BIOS firmware?
>
> --
> Hilsen/Regards
> Michael Rasmussen
>
> Get my public GnuPG keys:
> michael <at> rasmussen <dot> cc
> http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E
> mir <at> datanom <dot> net
> http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C
> mir <at> miras <dot> org
> http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917
> --------------------------------------------------------------
> /usr/games/fortune -es says:
> Absinthe makes the tart grow fonder.
>
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>
>
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20160926/1abd9c20/attachment-0001.html>

From mir at miras.org  Mon Sep 26 16:41:45 2016
From: mir at miras.org (Michael Rasmussen)
Date: Mon, 26 Sep 2016 18:41:45 +0200
Subject: [OmniOS-discuss] Ang: Re:  Installation Failing (Reboots)
In-Reply-To: <OFCAB0B4E0.364AE4AF-ONC125803A.005A9E26-C125803A.005A9E28@inse.com>
References: <CAMUDemtaumY2KJmLxoPt1Z2++Y26DshWYnxCr0kMCraGCeWOpQ@mail.gmail.com>
	<CAMUDemvCkBEDLp_2ePQB+H=B=UnEOPf-EeqVRFPqmYGT___rFg@mail.gmail.com>
	<alpine.GSO.2.20.1609231852460.5577@freddy.simplesystems.org>
	<CAMUDemvDxRT_xapdaqzfJgcAH6mTUehf2D2SVtT0g5X0sAe3TA@mail.gmail.com>
	<942E11EF-14FC-479C-92D6-8C19ABBAD789@omniti.com>
	<EB2990FA-5E81-4DE0-8961-ACB7D14070F1@zunaj.si>
	<CAMUDemufri7o2_gH=fVeDe=g0BNjuJ+a01YKbuS+FKBOc2QY8A@mail.gmail.com>
	<21FA36AF-0B6F-4FC3-A48E-0EA0A024A835@omniti.com>
	<CAMUDemudWTH7SLFm8qAU+1aKde0xWoBvi4PP8K4GEz_cdOtmLw@mail.gmail.com>
	<20160926180147.58c7e5d3@sleipner.datanom.net>
	<CAMUDemueiho3AVwHPS8Xtra=K924QtSRKwkE2XYVx1t1QXaFWw@mail.gmail.com>
	<20160926180733.5cee0873@sleipner.datanom.net>
	<OFCAB0B4E0.364AE4AF-ONC125803A.005A9E26-C125803A.005A9E28@inse.com>
Message-ID: <20160926184145.012ef299@sleipner.datanom.net>

On Mon, 26 Sep 2016 18:29:47 +0200
Johan Kragsterman <johan.kragsterman at capvert.se> wrote:

> Hi!
> 
> I've seen this sometime, if I remember right, when I forgot to disable C-states.
> 
> So, have you checked that C-states is disabled?
> 
I have never heard of enabled C-states should prevent booting? I have
heard of C-states influence stability in the long run.

-- 
Hilsen/Regards
Michael Rasmussen

Get my public GnuPG keys:
michael <at> rasmussen <dot> cc
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E
mir <at> datanom <dot> net
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C
mir <at> miras <dot> org
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917
--------------------------------------------------------------
/usr/games/fortune -es says:
God is dead and I don't feel all too well either....
		-- Ralph Moonen
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 163 bytes
Desc: OpenPGP digital signature
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20160926/71fbcaff/attachment.bin>

From gate03 at landcroft.co.uk  Mon Sep 26 19:58:23 2016
From: gate03 at landcroft.co.uk (Michael Mounteney)
Date: Tue, 27 Sep 2016 05:58:23 +1000
Subject: [OmniOS-discuss] Installation Failing (Reboots)
In-Reply-To: <CAMUDemudWTH7SLFm8qAU+1aKde0xWoBvi4PP8K4GEz_cdOtmLw@mail.gmail.com>
References: <CAMUDemvCkBEDLp_2ePQB+H=B=UnEOPf-EeqVRFPqmYGT___rFg@mail.gmail.com>
	<alpine.GSO.2.20.1609231852460.5577@freddy.simplesystems.org>
	<CAMUDemvDxRT_xapdaqzfJgcAH6mTUehf2D2SVtT0g5X0sAe3TA@mail.gmail.com>
	<942E11EF-14FC-479C-92D6-8C19ABBAD789@omniti.com>
	<EB2990FA-5E81-4DE0-8961-ACB7D14070F1@zunaj.si>
	<CAMUDemufri7o2_gH=fVeDe=g0BNjuJ+a01YKbuS+FKBOc2QY8A@mail.gmail.com>
	<21FA36AF-0B6F-4FC3-A48E-0EA0A024A835@omniti.com>
	<CAMUDemudWTH7SLFm8qAU+1aKde0xWoBvi4PP8K4GEz_cdOtmLw@mail.gmail.com>
Message-ID: <20160927055823.778d3b9e@kendall>

On Mon, 26 Sep 2016 11:56:03 -0400
Garland McAlexander <garland at bustle.com> wrote:

> Just attempted via IPMI CDROM and the same issue persists, gets to
> splash screen, loads a few lines of text and forces a reboot.

I had this once and it turned out to be defective media.  Try another
stick?

______________
Michael Mounteney

From garland at bustle.com  Mon Sep 26 19:59:06 2016
From: garland at bustle.com (Garland McAlexander)
Date: Mon, 26 Sep 2016 15:59:06 -0400
Subject: [OmniOS-discuss] Installation Failing (Reboots)
In-Reply-To: <20160927055823.778d3b9e@kendall>
References: <CAMUDemvCkBEDLp_2ePQB+H=B=UnEOPf-EeqVRFPqmYGT___rFg@mail.gmail.com>
	<alpine.GSO.2.20.1609231852460.5577@freddy.simplesystems.org>
	<CAMUDemvDxRT_xapdaqzfJgcAH6mTUehf2D2SVtT0g5X0sAe3TA@mail.gmail.com>
	<942E11EF-14FC-479C-92D6-8C19ABBAD789@omniti.com>
	<EB2990FA-5E81-4DE0-8961-ACB7D14070F1@zunaj.si>
	<CAMUDemufri7o2_gH=fVeDe=g0BNjuJ+a01YKbuS+FKBOc2QY8A@mail.gmail.com>
	<21FA36AF-0B6F-4FC3-A48E-0EA0A024A835@omniti.com>
	<CAMUDemudWTH7SLFm8qAU+1aKde0xWoBvi4PP8K4GEz_cdOtmLw@mail.gmail.com>
	<20160927055823.778d3b9e@kendall>
Message-ID: <CAMUDemtTp0CpMuuA_s8EFECopDRoJGy1Np=xpPY4LnEMLNjRig@mail.gmail.com>

I've tried multiple images via IPMI now, and I've md5 verified them all
before attempting to install.

On Mon, Sep 26, 2016 at 3:58 PM, Michael Mounteney <gate03 at landcroft.co.uk>
wrote:

> On Mon, 26 Sep 2016 11:56:03 -0400
> Garland McAlexander <garland at bustle.com> wrote:
>
> > Just attempted via IPMI CDROM and the same issue persists, gets to
> > splash screen, loads a few lines of text and forces a reboot.
>
> I had this once and it turned out to be defective media.  Try another
> stick?
>
> ______________
> Michael Mounteney
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20160926/a31b5b63/attachment.html>

From jeff at 1beyond.com  Mon Sep 26 20:19:42 2016
From: jeff at 1beyond.com (Jeff Berkembrock)
Date: Mon, 26 Sep 2016 16:19:42 -0400
Subject: [OmniOS-discuss] Installation Failing (Reboots)
In-Reply-To: <CAMUDemtTp0CpMuuA_s8EFECopDRoJGy1Np=xpPY4LnEMLNjRig@mail.gmail.com>
References: <CAMUDemvCkBEDLp_2ePQB+H=B=UnEOPf-EeqVRFPqmYGT___rFg@mail.gmail.com>
	<alpine.GSO.2.20.1609231852460.5577@freddy.simplesystems.org>
	<CAMUDemvDxRT_xapdaqzfJgcAH6mTUehf2D2SVtT0g5X0sAe3TA@mail.gmail.com>
	<942E11EF-14FC-479C-92D6-8C19ABBAD789@omniti.com>
	<EB2990FA-5E81-4DE0-8961-ACB7D14070F1@zunaj.si>
	<CAMUDemufri7o2_gH=fVeDe=g0BNjuJ+a01YKbuS+FKBOc2QY8A@mail.gmail.com>
	<21FA36AF-0B6F-4FC3-A48E-0EA0A024A835@omniti.com>
	<CAMUDemudWTH7SLFm8qAU+1aKde0xWoBvi4PP8K4GEz_cdOtmLw@mail.gmail.com>
	<20160927055823.778d3b9e@kendall>
	<CAMUDemtTp0CpMuuA_s8EFECopDRoJGy1Np=xpPY4LnEMLNjRig@mail.gmail.com>
Message-ID: <CAEGbm64iz4F06HThW5vMChJuJJsrxepeSjLdBZrhGGNZHmiKuw@mail.gmail.com>

Now that you're using IPMI, try to press Print Screen right as the text
flashes on the screen. You might be able to snap a screenshot of it and
read what it says.

Best Regards,

Jeff Berkembrock
www.1beyond.com
617-591-2200 x2212

On Mon, Sep 26, 2016 at 3:59 PM, Garland McAlexander <garland at bustle.com>
wrote:

> I've tried multiple images via IPMI now, and I've md5 verified them all
> before attempting to install.
>
> On Mon, Sep 26, 2016 at 3:58 PM, Michael Mounteney <gate03 at landcroft.co.uk
> > wrote:
>
>> On Mon, 26 Sep 2016 11:56:03 -0400
>> Garland McAlexander <garland at bustle.com> wrote:
>>
>> > Just attempted via IPMI CDROM and the same issue persists, gets to
>> > splash screen, loads a few lines of text and forces a reboot.
>>
>> I had this once and it turned out to be defective media.  Try another
>> stick?
>>
>> ______________
>> Michael Mounteney
>>
>
>
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20160926/f02c73cc/attachment.html>

From garland at bustle.com  Mon Sep 26 20:31:24 2016
From: garland at bustle.com (Garland McAlexander)
Date: Mon, 26 Sep 2016 16:31:24 -0400
Subject: [OmniOS-discuss] Installation Failing (Reboots)
In-Reply-To: <CAEGbm64iz4F06HThW5vMChJuJJsrxepeSjLdBZrhGGNZHmiKuw@mail.gmail.com>
References: <CAMUDemvCkBEDLp_2ePQB+H=B=UnEOPf-EeqVRFPqmYGT___rFg@mail.gmail.com>
	<alpine.GSO.2.20.1609231852460.5577@freddy.simplesystems.org>
	<CAMUDemvDxRT_xapdaqzfJgcAH6mTUehf2D2SVtT0g5X0sAe3TA@mail.gmail.com>
	<942E11EF-14FC-479C-92D6-8C19ABBAD789@omniti.com>
	<EB2990FA-5E81-4DE0-8961-ACB7D14070F1@zunaj.si>
	<CAMUDemufri7o2_gH=fVeDe=g0BNjuJ+a01YKbuS+FKBOc2QY8A@mail.gmail.com>
	<21FA36AF-0B6F-4FC3-A48E-0EA0A024A835@omniti.com>
	<CAMUDemudWTH7SLFm8qAU+1aKde0xWoBvi4PP8K4GEz_cdOtmLw@mail.gmail.com>
	<20160927055823.778d3b9e@kendall>
	<CAMUDemtTp0CpMuuA_s8EFECopDRoJGy1Np=xpPY4LnEMLNjRig@mail.gmail.com>
	<CAEGbm64iz4F06HThW5vMChJuJJsrxepeSjLdBZrhGGNZHmiKuw@mail.gmail.com>
Message-ID: <CAMUDemv2JiXa=YPzKhnBRNAs_u2ODUefHsAi1tJnEOzO9_LC6Q@mail.gmail.com>

That lead me to recording the IPMI session and I now get this:



?

On Mon, Sep 26, 2016 at 4:19 PM, Jeff Berkembrock <jeff at 1beyond.com> wrote:

> Now that you're using IPMI, try to press Print Screen right as the text
> flashes on the screen. You might be able to snap a screenshot of it and
> read what it says.
>
> Best Regards,
>
> Jeff Berkembrock
> www.1beyond.com
> 617-591-2200 x2212
>
> On Mon, Sep 26, 2016 at 3:59 PM, Garland McAlexander <garland at bustle.com>
> wrote:
>
>> I've tried multiple images via IPMI now, and I've md5 verified them all
>> before attempting to install.
>>
>> On Mon, Sep 26, 2016 at 3:58 PM, Michael Mounteney <
>> gate03 at landcroft.co.uk> wrote:
>>
>>> On Mon, 26 Sep 2016 11:56:03 -0400
>>> Garland McAlexander <garland at bustle.com> wrote:
>>>
>>> > Just attempted via IPMI CDROM and the same issue persists, gets to
>>> > splash screen, loads a few lines of text and forces a reboot.
>>>
>>> I had this once and it turned out to be defective media.  Try another
>>> stick?
>>>
>>> ______________
>>> Michael Mounteney
>>>
>>
>>
>> _______________________________________________
>> OmniOS-discuss mailing list
>> OmniOS-discuss at lists.omniti.com
>> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20160926/f27dda95/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Screen Shot 2016-09-26 at 4.30.42 PM.png
Type: image/png
Size: 122333 bytes
Desc: not available
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20160926/f27dda95/attachment-0001.png>

From josh at sysmgr.org  Mon Sep 26 20:33:48 2016
From: josh at sysmgr.org (Joshua M. Clulow)
Date: Mon, 26 Sep 2016 13:33:48 -0700
Subject: [OmniOS-discuss] Installation Failing (Reboots)
In-Reply-To: <CAMUDemtTp0CpMuuA_s8EFECopDRoJGy1Np=xpPY4LnEMLNjRig@mail.gmail.com>
References: <CAMUDemvCkBEDLp_2ePQB+H=B=UnEOPf-EeqVRFPqmYGT___rFg@mail.gmail.com>
	<alpine.GSO.2.20.1609231852460.5577@freddy.simplesystems.org>
	<CAMUDemvDxRT_xapdaqzfJgcAH6mTUehf2D2SVtT0g5X0sAe3TA@mail.gmail.com>
	<942E11EF-14FC-479C-92D6-8C19ABBAD789@omniti.com>
	<EB2990FA-5E81-4DE0-8961-ACB7D14070F1@zunaj.si>
	<CAMUDemufri7o2_gH=fVeDe=g0BNjuJ+a01YKbuS+FKBOc2QY8A@mail.gmail.com>
	<21FA36AF-0B6F-4FC3-A48E-0EA0A024A835@omniti.com>
	<CAMUDemudWTH7SLFm8qAU+1aKde0xWoBvi4PP8K4GEz_cdOtmLw@mail.gmail.com>
	<20160927055823.778d3b9e@kendall>
	<CAMUDemtTp0CpMuuA_s8EFECopDRoJGy1Np=xpPY4LnEMLNjRig@mail.gmail.com>
Message-ID: <CAEwA5n+fgcuS9jWBMqdyJ7sSwRyOnuH6-+gRrXsUi-+SeuezzA@mail.gmail.com>

On 26 September 2016 at 12:59, Garland McAlexander <garland at bustle.com> wrote:
> I've tried multiple images via IPMI now, and I've md5 verified them all
> before attempting to install.

Does your system have serial-over-LAN functionality available through
IPMI?  I would recommend trying to get the system to boot using the
serial port as the console -- that way, when the text flashes up
you'll be able to capture it before the system reboots.

I've used the OmniOS installer via the serial port (in a virtual
machine) before.  I think that all I had to do was edit the grub menu
entry and add the appropriate console settings, e.g.

    -B console=ttyb,ttyb-mode="115200,8,n,1,-"

(NB: that's for "ttyb", aka COM2.  If your IPMI serial-over-LAN port
is COM1 you'll want ttya instead.)

-- 
Joshua M. Clulow
UNIX Admin/Developer
http://blog.sysmgr.org

From garland at bustle.com  Mon Sep 26 20:50:05 2016
From: garland at bustle.com (Garland McAlexander)
Date: Mon, 26 Sep 2016 16:50:05 -0400
Subject: [OmniOS-discuss] Installation Failing (Reboots)
In-Reply-To: <CAEwA5n+fgcuS9jWBMqdyJ7sSwRyOnuH6-+gRrXsUi-+SeuezzA@mail.gmail.com>
References: <CAMUDemvCkBEDLp_2ePQB+H=B=UnEOPf-EeqVRFPqmYGT___rFg@mail.gmail.com>
	<alpine.GSO.2.20.1609231852460.5577@freddy.simplesystems.org>
	<CAMUDemvDxRT_xapdaqzfJgcAH6mTUehf2D2SVtT0g5X0sAe3TA@mail.gmail.com>
	<942E11EF-14FC-479C-92D6-8C19ABBAD789@omniti.com>
	<EB2990FA-5E81-4DE0-8961-ACB7D14070F1@zunaj.si>
	<CAMUDemufri7o2_gH=fVeDe=g0BNjuJ+a01YKbuS+FKBOc2QY8A@mail.gmail.com>
	<21FA36AF-0B6F-4FC3-A48E-0EA0A024A835@omniti.com>
	<CAMUDemudWTH7SLFm8qAU+1aKde0xWoBvi4PP8K4GEz_cdOtmLw@mail.gmail.com>
	<20160927055823.778d3b9e@kendall>
	<CAMUDemtTp0CpMuuA_s8EFECopDRoJGy1Np=xpPY4LnEMLNjRig@mail.gmail.com>
	<CAEwA5n+fgcuS9jWBMqdyJ7sSwRyOnuH6-+gRrXsUi-+SeuezzA@mail.gmail.com>
Message-ID: <CAMUDemsj+00ZZNDELmARNb_L2Xzx+6S7Ambihs58UBT-bNsCVg@mail.gmail.com>

https://youtu.be/JlbxQg01AdA Here it is in action.

On Mon, Sep 26, 2016 at 4:33 PM, Joshua M. Clulow <josh at sysmgr.org> wrote:

> On 26 September 2016 at 12:59, Garland McAlexander <garland at bustle.com>
> wrote:
> > I've tried multiple images via IPMI now, and I've md5 verified them all
> > before attempting to install.
>
> Does your system have serial-over-LAN functionality available through
> IPMI?  I would recommend trying to get the system to boot using the
> serial port as the console -- that way, when the text flashes up
> you'll be able to capture it before the system reboots.
>
> I've used the OmniOS installer via the serial port (in a virtual
> machine) before.  I think that all I had to do was edit the grub menu
> entry and add the appropriate console settings, e.g.
>
>     -B console=ttyb,ttyb-mode="115200,8,n,1,-"
>
> (NB: that's for "ttyb", aka COM2.  If your IPMI serial-over-LAN port
> is COM1 you'll want ttya instead.)
>
> --
> Joshua M. Clulow
> UNIX Admin/Developer
> http://blog.sysmgr.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20160926/032f1990/attachment.html>

From josh at sysmgr.org  Mon Sep 26 20:52:49 2016
From: josh at sysmgr.org (Joshua M. Clulow)
Date: Mon, 26 Sep 2016 13:52:49 -0700
Subject: [OmniOS-discuss] Installation Failing (Reboots)
In-Reply-To: <CAMUDemv2JiXa=YPzKhnBRNAs_u2ODUefHsAi1tJnEOzO9_LC6Q@mail.gmail.com>
References: <CAMUDemvCkBEDLp_2ePQB+H=B=UnEOPf-EeqVRFPqmYGT___rFg@mail.gmail.com>
	<alpine.GSO.2.20.1609231852460.5577@freddy.simplesystems.org>
	<CAMUDemvDxRT_xapdaqzfJgcAH6mTUehf2D2SVtT0g5X0sAe3TA@mail.gmail.com>
	<942E11EF-14FC-479C-92D6-8C19ABBAD789@omniti.com>
	<EB2990FA-5E81-4DE0-8961-ACB7D14070F1@zunaj.si>
	<CAMUDemufri7o2_gH=fVeDe=g0BNjuJ+a01YKbuS+FKBOc2QY8A@mail.gmail.com>
	<21FA36AF-0B6F-4FC3-A48E-0EA0A024A835@omniti.com>
	<CAMUDemudWTH7SLFm8qAU+1aKde0xWoBvi4PP8K4GEz_cdOtmLw@mail.gmail.com>
	<20160927055823.778d3b9e@kendall>
	<CAMUDemtTp0CpMuuA_s8EFECopDRoJGy1Np=xpPY4LnEMLNjRig@mail.gmail.com>
	<CAEGbm64iz4F06HThW5vMChJuJJsrxepeSjLdBZrhGGNZHmiKuw@mail.gmail.com>
	<CAMUDemv2JiXa=YPzKhnBRNAs_u2ODUefHsAi1tJnEOzO9_LC6Q@mail.gmail.com>
Message-ID: <CAEwA5nLunaPuc+d3WUZGcg1BLkqjY=mUcM=7dKQL-ZJRh_VQ3A@mail.gmail.com>

On 26 September 2016 at 13:31, Garland McAlexander <garland at bustle.com> wrote:
> That lead me to recording the IPMI session and I now get this:

I've seen that message ("unsupported mixed cpu monitor/mwait support
detected") in another thread recently.  In that case, I believe the
solution was to reset the BIOS to an "optimised performance" factory
defaults set.

Can you try various default BIOS settings that to see if that allows
the system to boot?


Cheers.

-- 
Joshua M. Clulow
UNIX Admin/Developer
http://blog.sysmgr.org

From garland at bustle.com  Mon Sep 26 21:07:04 2016
From: garland at bustle.com (Garland McAlexander)
Date: Mon, 26 Sep 2016 17:07:04 -0400
Subject: [OmniOS-discuss] Installation Failing (Reboots)
In-Reply-To: <CAEwA5nLunaPuc+d3WUZGcg1BLkqjY=mUcM=7dKQL-ZJRh_VQ3A@mail.gmail.com>
References: <CAMUDemvCkBEDLp_2ePQB+H=B=UnEOPf-EeqVRFPqmYGT___rFg@mail.gmail.com>
	<alpine.GSO.2.20.1609231852460.5577@freddy.simplesystems.org>
	<CAMUDemvDxRT_xapdaqzfJgcAH6mTUehf2D2SVtT0g5X0sAe3TA@mail.gmail.com>
	<942E11EF-14FC-479C-92D6-8C19ABBAD789@omniti.com>
	<EB2990FA-5E81-4DE0-8961-ACB7D14070F1@zunaj.si>
	<CAMUDemufri7o2_gH=fVeDe=g0BNjuJ+a01YKbuS+FKBOc2QY8A@mail.gmail.com>
	<21FA36AF-0B6F-4FC3-A48E-0EA0A024A835@omniti.com>
	<CAMUDemudWTH7SLFm8qAU+1aKde0xWoBvi4PP8K4GEz_cdOtmLw@mail.gmail.com>
	<20160927055823.778d3b9e@kendall>
	<CAMUDemtTp0CpMuuA_s8EFECopDRoJGy1Np=xpPY4LnEMLNjRig@mail.gmail.com>
	<CAEGbm64iz4F06HThW5vMChJuJJsrxepeSjLdBZrhGGNZHmiKuw@mail.gmail.com>
	<CAMUDemv2JiXa=YPzKhnBRNAs_u2ODUefHsAi1tJnEOzO9_LC6Q@mail.gmail.com>
	<CAEwA5nLunaPuc+d3WUZGcg1BLkqjY=mUcM=7dKQL-ZJRh_VQ3A@mail.gmail.com>
Message-ID: <CAMUDemsr1P-ZhjjskECBtr_fcwfgYdEFCWb81VZ=x7xyAxwwjQ@mail.gmail.com>

Spot on!

Resetting to optimized defaults did the trick. Such strange behaviour.

Thanks a TON for the advice!

On Mon, Sep 26, 2016 at 4:52 PM, Joshua M. Clulow <josh at sysmgr.org> wrote:

> On 26 September 2016 at 13:31, Garland McAlexander <garland at bustle.com>
> wrote:
> > That lead me to recording the IPMI session and I now get this:
>
> I've seen that message ("unsupported mixed cpu monitor/mwait support
> detected") in another thread recently.  In that case, I believe the
> solution was to reset the BIOS to an "optimised performance" factory
> defaults set.
>
> Can you try various default BIOS settings that to see if that allows
> the system to boot?
>
>
> Cheers.
>
> --
> Joshua M. Clulow
> UNIX Admin/Developer
> http://blog.sysmgr.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20160926/0c38d729/attachment.html>

From info.fmo at gmx.de  Mon Sep 26 22:14:28 2016
From: info.fmo at gmx.de (Frank M.)
Date: Tue, 27 Sep 2016 00:14:28 +0200
Subject: [OmniOS-discuss] bloody update error pkg
Message-ID: <53794389.20160927001428@gmx.de>

Hi,

I wanted to update my 151019-test-installation:

root at test-san1:/root# pkg update
Checking that pkg(5) is up to date -

Errors were encountered while attempting to retrieve package or file data for
the requested operation.
Details follow:

http protocol error: code: 404 reason: Not Found
URL: 'http://pkg.omniti.com/omnios/bloody/omnios/manifest/0/package%2Fpkg at 0.5.11%2C5.11-0.151019%3A20160824T050133Z' (happened 4 times)


Frank


---
Diese E-Mail wurde von Avast Antivirus-Software auf Viren gepr?ft.
https://www.avast.com/antivirus


From danmcd at omniti.com  Mon Sep 26 22:28:43 2016
From: danmcd at omniti.com (Dan McDonald)
Date: Mon, 26 Sep 2016 18:28:43 -0400
Subject: [OmniOS-discuss] bloody update error pkg
In-Reply-To: <53794389.20160927001428@gmx.de>
References: <53794389.20160927001428@gmx.de>
Message-ID: <C7BFDD47-ED05-41C4-9478-399D43FB25AA@omniti.com>

Pardon the top-post.

I'm rebuilding the package index, which should fix this.  I believe it's papering over a possible problem in our front-end.

Try in another 20 minutes or so.

Dan


> On Sep 26, 2016, at 6:14 PM, Frank M. <info.fmo at gmx.de> wrote:
> 
> Hi,
> 
> I wanted to update my 151019-test-installation:
> 
> root at test-san1:/root# pkg update
> Checking that pkg(5) is up to date -
> 
> Errors were encountered while attempting to retrieve package or file data for
> the requested operation.
> Details follow:
> 
> http protocol error: code: 404 reason: Not Found
> URL: 'http://pkg.omniti.com/omnios/bloody/omnios/manifest/0/package%2Fpkg at 0.5.11%2C5.11-0.151019%3A20160824T050133Z' (happened 4 times)
> 
> 
> Frank
> 
> 
> ---
> Diese E-Mail wurde von Avast Antivirus-Software auf Viren gepr?ft.
> https://www.avast.com/antivirus
> 
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss


From bouktaib.mohssin at gmail.com  Tue Sep 27 11:23:42 2016
From: bouktaib.mohssin at gmail.com (Bouktaib Mohssin)
Date: Tue, 27 Sep 2016 12:23:42 +0100
Subject: [OmniOS-discuss] Booting omnios from grub2
Message-ID: <CAELhy=PBBxi3MGwHD-r68XzsMwhNsf3MP=pRrRvhMBoApXk0Hg@mail.gmail.com>

?Hello Community

I'm trying to boot omnios from grub2 using my ubuntu server ,i've triedit
with smartos & it work (
https://wiki.smartos.org/display/DOC/Booting+SmartOS+from+GRUB2)
my question is how to boot omnios from grub2

thanks :)?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20160927/9efa6a06/attachment-0001.html>

From nshalman at omniti.com  Tue Sep 27 13:21:02 2016
From: nshalman at omniti.com (Nahum Shalman)
Date: Tue, 27 Sep 2016 09:21:02 -0400
Subject: [OmniOS-discuss] Booting omnios from grub2
In-Reply-To: <CAELhy=PBBxi3MGwHD-r68XzsMwhNsf3MP=pRrRvhMBoApXk0Hg@mail.gmail.com>
References: <CAELhy=PBBxi3MGwHD-r68XzsMwhNsf3MP=pRrRvhMBoApXk0Hg@mail.gmail.com>
Message-ID: <CAOc1FHhYTb9S4Jn5==Rpv2-Pzebkzp-9BLDMoa=zYav2tQmhZQ@mail.gmail.com>

Hi,

I see I will now have to pay for my sins for having written that wiki
page...
The short answer is that SmartOS is far more amenable to booting from
arbitrary boot loaders than OmniOS.

With SmartOS all you care about is booting the correct kernel and boot
archive.

OmniOS has boot environments (BEs). Booting into the correct BE requires
more information on the kernel command line than SmartOS does.
The `beadm` tool on OmniOS integrates with GRUB but has no integration for
GRUB2.

In the future, upstream illumos has recently landed a port of the FreeBSD
loader which has many of the same desirable features as GRUB2, but also
includes BE/beadm integration.

 Also, you'd have to be certain that the version of GRUB2 in question is
compatible with the pool features of the pool that OmniOS is on.

There are possibly some very clunky ways to get GRUB2 to look at the GRUB
configuration from OmniOS, but without explanation of *why* you want to do
this and how you expect it to behave, I'm not prepared to opine about
precisely how difficult it would be.

Could you clarify what you're trying to accomplish?

-Nahum

On Tue, Sep 27, 2016 at 7:23 AM, Bouktaib Mohssin <
bouktaib.mohssin at gmail.com> wrote:

> ?Hello Community
>
> I'm trying to boot omnios from grub2 using my ubuntu server ,i've triedit
> with smartos & it work (https://wiki.smartos.org/
> display/DOC/Booting+SmartOS+from+GRUB2)
> my question is how to boot omnios from grub2
>
> thanks :)?
>
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20160927/cd12393e/attachment.html>

From mir at miras.org  Wed Sep 28 07:42:07 2016
From: mir at miras.org (Michael Rasmussen)
Date: Wed, 28 Sep 2016 09:42:07 +0200
Subject: [OmniOS-discuss] Changing motherboard
Message-ID: <20160928094207.2d8a65b2@sleipner.datanom.net>

Hi all,

In the coming days I am going to change motherboard in an Omnios
server. then change also involves change CPU from AMD to Intel. The
system has two pools - rpool and a storage pool. The server has a bond
on two add-in 82574L PCIe which will be replaced by two on-board
one i217LM and one i210AT. And add-in card hosts a dual Infiniband
which will be transferred. The storage pool is hosted on a SAS HBA
which likewise will be transferred. the rpool is connected through
on-board SATA which is also the way it will be connected on the new
board.

My hope is that the server just comes up on the new board but are there
anything special that should be done before shutting down the old
system? Maybe export the storage pool?

-- 
Hilsen/Regards
Michael Rasmussen

Get my public GnuPG keys:
michael <at> rasmussen <dot> cc
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E
mir <at> datanom <dot> net
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C
mir <at> miras <dot> org
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917
--------------------------------------------------------------
/usr/games/fortune -es says:
Somewhere, just out of sight, the unicorns are gathering.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 163 bytes
Desc: OpenPGP digital signature
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20160928/9be1b190/attachment.bin>

From alka at hfg-gmuend.de  Wed Sep 28 14:29:12 2016
From: alka at hfg-gmuend.de (Guenther Alka)
Date: Wed, 28 Sep 2016 16:29:12 +0200
Subject: [OmniOS-discuss] Changing motherboard
In-Reply-To: <20160928094207.2d8a65b2@sleipner.datanom.net>
References: <20160928094207.2d8a65b2@sleipner.datanom.net>
Message-ID: <3fafe557-86dc-248c-941b-cba7298d73bb@hfg-gmuend.de>

I would probably

- export the datapool
- delete all nic interfaces

then try to reboot from the disk on the new system

- create new nic interfaces and ip adresses
- import the datapool

Gea

Am 28.09.2016 um 09:42 schrieb Michael Rasmussen:
> Hi all,
>
> In the coming days I am going to change motherboard in an Omnios
> server. then change also involves change CPU from AMD to Intel. The
> system has two pools - rpool and a storage pool. The server has a bond
> on two add-in 82574L PCIe which will be replaced by two on-board
> one i217LM and one i210AT. And add-in card hosts a dual Infiniband
> which will be transferred. The storage pool is hosted on a SAS HBA
> which likewise will be transferred. the rpool is connected through
> on-board SATA which is also the way it will be connected on the new
> board.
>
> My hope is that the server just comes up on the new board but are there
> anything special that should be done before shutting down the old
> system? Maybe export the storage pool?
>
>
>
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20160928/d8edeb8f/attachment.html>

From johan.kragsterman at capvert.se  Wed Sep 28 14:46:39 2016
From: johan.kragsterman at capvert.se (Johan Kragsterman)
Date: Wed, 28 Sep 2016 16:46:39 +0200
Subject: [OmniOS-discuss] Ang: Re:  Changing motherboard
In-Reply-To: <3fafe557-86dc-248c-941b-cba7298d73bb@hfg-gmuend.de>
References: <3fafe557-86dc-248c-941b-cba7298d73bb@hfg-gmuend.de>,
	<20160928094207.2d8a65b2@sleipner.datanom.net>
Message-ID: <OFAA147028.7E460EF3-ONC125803C.00512CF0-C125803C.00512CF1@inse.com>


Hi!


-----"OmniOS-discuss" <omnios-discuss-bounces at lists.omniti.com> skrev: -----
Till: omnios-discuss at lists.omniti.com
Fr?n: Guenther Alka 
S?nt av: "OmniOS-discuss" 
Datum: 2016-09-28 16:34
?rende: Re: [OmniOS-discuss] Changing motherboard

I would probably

- export the datapool
- delete all nic interfaces

then try to reboot from the disk on the new system

- create new nic interfaces and ip adresses
- import the datapool

Gea





Yeah, it can be a mess with old nic interfaces. I've also been thinking about how to do that. You should delete the eth if's, but perhaps you should also delete the phy's? dladm delete-phys?
I remember when I swapped mobo some years ago, and used the old system disk, I got this problem with the old phy's. I deleted them in the new system, but ran into some problems due to that, don't remember now what it was.

/J






Am 28.09.2016 um 09:42 schrieb Michael Rasmussen:
Hi all,

In the coming days I am going to change motherboard in an Omnios
server. then change also involves change CPU from AMD to Intel. The
system has two pools - rpool and a storage pool. The server has a bond
on two add-in 82574L PCIe which will be replaced by two on-board
one i217LM and one i210AT. And add-in card hosts a dual Infiniband
which will be transferred. The storage pool is hosted on a SAS HBA
which likewise will be transferred. the rpool is connected through
on-board SATA which is also the way it will be connected on the new
board.

My hope is that the server just comes up on the new board but are there
anything special that should be done before shutting down the old
system? Maybe export the storage pool?



_______________________________________________
OmniOS-discuss mailing list
OmniOS-discuss at lists.omniti.com
http://lists.omniti.com/mailman/listinfo/omnios-discuss

_______________________________________________
OmniOS-discuss mailing list
OmniOS-discuss at lists.omniti.com
http://lists.omniti.com/mailman/listinfo/omnios-discuss




From mir at miras.org  Wed Sep 28 17:15:28 2016
From: mir at miras.org (Michael Rasmussen)
Date: Wed, 28 Sep 2016 19:15:28 +0200
Subject: [OmniOS-discuss] Changing motherboard
In-Reply-To: <3fafe557-86dc-248c-941b-cba7298d73bb@hfg-gmuend.de>
References: <20160928094207.2d8a65b2@sleipner.datanom.net>
	<3fafe557-86dc-248c-941b-cba7298d73bb@hfg-gmuend.de>
Message-ID: <20160928191528.5b89a93b@sleipner.datanom.net>

On Wed, 28 Sep 2016 16:29:12 +0200
Guenther Alka <alka at hfg-gmuend.de> wrote:

> 
> - export the datapool
> - delete all nic interfaces
> 
Do you think deleting nics also includes the Infiniband
ports/partitions given they reside on a dedicated HCA?

-- 
Hilsen/Regards
Michael Rasmussen

Get my public GnuPG keys:
michael <at> rasmussen <dot> cc
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E
mir <at> datanom <dot> net
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C
mir <at> miras <dot> org
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917
--------------------------------------------------------------
/usr/games/fortune -es says:
He hated being thought of as one of those people that wore stupid
ornamental armour. It was gilt by association.
		-- Terry Pratchett, "Night Watch"
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 163 bytes
Desc: OpenPGP digital signature
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20160928/63c7310e/attachment.bin>

From mir at miras.org  Thu Sep 29 00:12:27 2016
From: mir at miras.org (Michael Rasmussen)
Date: Thu, 29 Sep 2016 02:12:27 +0200
Subject: [OmniOS-discuss] vlan and vnic
Message-ID: <20160929021227.2fe5f04f@sleipner.datanom.net>

Hi all,

I have been wondering what is the best way to create vlans in omnios.
use dladm create-vlan and assign IP to the vlan
or
use dladm create-vnic with a vlan tag and assign IP to the vnic

What are the pros and cons of each?

-- 
Hilsen/Regards
Michael Rasmussen

Get my public GnuPG keys:
michael <at> rasmussen <dot> cc
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E
mir <at> datanom <dot> net
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C
mir <at> miras <dot> org
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917
--------------------------------------------------------------
/usr/games/fortune -es says:
Kirk to Enterprise -- beam down yeoman Rand and a six-pack.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 163 bytes
Desc: OpenPGP digital signature
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20160929/425350f6/attachment-0001.bin>

From daleg at omniti.com  Thu Sep 29 06:43:54 2016
From: daleg at omniti.com (Dale Ghent)
Date: Thu, 29 Sep 2016 02:43:54 -0400
Subject: [OmniOS-discuss] vlan and vnic
In-Reply-To: <20160929021227.2fe5f04f@sleipner.datanom.net>
References: <20160929021227.2fe5f04f@sleipner.datanom.net>
Message-ID: <B9757DE7-5F7E-4EDA-9CCF-16CC4215213B@omniti.com>


> On Sep 28, 2016, at 8:12 PM, Michael Rasmussen <mir at miras.org> wrote:
> 
> Hi all,
> 
> I have been wondering what is the best way to create vlans in omnios.
> use dladm create-vlan and assign IP to the vlan
> or
> use dladm create-vnic with a vlan tag and assign IP to the vnic
> 
> What are the pros and cons of each?

Functionally as far as VLANs go, they are equivalent. However, VNICs make things more flexible.

VNICs can consume a aggregated set of physical interfaces, or a single physical interface. VNICs in turn can be assigned to Zones.

Here's an operational example that we use:

1) 2 physical interfaces: igb0 and igb1

2) igb0 and igb1 are made into an aggregated NIC we will name "aggr0" (dladm create-aggr ... aggr0) 

Now we have a redundant Layer 2. On the switch, we assign the ports these are connected to all the tagged VLANs that should participate.

3) For the global zone, we make a new VNIC from that aggr. Let's say the global zone's interface is on vlan 5. Thus, we do:
 
gz# dladm create-vnic -d aggr0 -v 5 global0

Okay, good. Now the global zone has its interface (named "global0"), now you just assign it a IP and default route and all that using the standard commands (ipadm, route -p add ...) If you don't have more than 1 physical interface, then you can ignore those first 3 steps.

4) Now we have some zones we want to add, but they need to talk to other VLANs, because VLAN 5 that the global zone is on is a private network that they should not see. Well, we create a VNIC for each zone and assign it to the zone with zonecfg:

 gz# dladm create-vnic -d aggr0 -v 20 foozone0

 gz# zonecfg -z foozone
 zonecfg:foozone> add net
 zonecfg:foozone> set physical=foozone0
...

Now the zone "foozone" will have its own "NIC" that can participate only on VLAN 20.

/dale

From alka at hfg-gmuend.de  Thu Sep 29 09:58:42 2016
From: alka at hfg-gmuend.de (Guenther Alka)
Date: Thu, 29 Sep 2016 11:58:42 +0200
Subject: [OmniOS-discuss] Changing motherboard
In-Reply-To: <20160928191528.5b89a93b@sleipner.datanom.net>
References: <20160928094207.2d8a65b2@sleipner.datanom.net>
	<3fafe557-86dc-248c-941b-cba7298d73bb@hfg-gmuend.de>
	<20160928191528.5b89a93b@sleipner.datanom.net>
Message-ID: <bc9b8a03-248d-3b52-2132-303bd1d8d6ed@hfg-gmuend.de>

I have not tried IB -
If they change link name/numbering then yes


Am 28.09.2016 um 19:15 schrieb Michael Rasmussen:
> On Wed, 28 Sep 2016 16:29:12 +0200
> Guenther Alka <alka at hfg-gmuend.de> wrote:
>
>> - export the datapool
>> - delete all nic interfaces
>>
> Do you think deleting nics also includes the Infiniband
> ports/partitions given they reside on a dedicated HCA?
>
>
>
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20160929/560e8af6/attachment.html>

From paul.jochum at nokia.com  Thu Sep 29 14:54:39 2016
From: paul.jochum at nokia.com (Paul Jochum)
Date: Thu, 29 Sep 2016 09:54:39 -0500
Subject: [OmniOS-discuss] Low powered and cheap OmniOS server?
Message-ID: <03507cc7-d011-f58a-3bd3-1424935e9a5c@nokia.com>

Hi All:

     Looking for the latest recommendations on a low powered and cheap 
OmniOS server.  This would be for home use, looking to use it as way of 
backing up Linux and Windows boxes.  Anyone have a recommendation?

thanks in advance,
Paul

From davide.poletto at gmail.com  Thu Sep 29 15:08:59 2016
From: davide.poletto at gmail.com (Davide Poletto)
Date: Thu, 29 Sep 2016 17:08:59 +0200
Subject: [OmniOS-discuss] Low powered and cheap OmniOS server?
In-Reply-To: <03507cc7-d011-f58a-3bd3-1424935e9a5c@nokia.com>
References: <03507cc7-d011-f58a-3bd3-1424935e9a5c@nokia.com>
Message-ID: <CANKMAMaqVo5SXJgqi1R7B3GCQ4-Hk46QSb+kdX5_i0tLT_Mr6A@mail.gmail.com>

My take: HPE ProLiant MicroServer Gen8
<http://www8.hp.com/us/en/products/proliant-servers/product-detail.html?oid=5379860>
(the one G1610T CPU based) or a Gen7 too: sufficiently silent (not
completely silent since has 2 fans), it doesn't consume so much power (with
Toshiba DT01ACAxxx drives is really silent - no vibrations), keep in mind
that it needs expensive "transposers" if you're planning to use SSD 2,5"
disks on its 3,5" bays (it has 4)...overall IMHO, it's compact enough and
it works flawlessly with OmniOS (tested on N36L, N40L and N54L which was
Gen 7). I have a few at home mostly equipped with 8GB or 16GB ECC Ram (a
must with ZFS, clearly) and Toshiba 1TB/2TB HDDs. OS is installed on a
dedicated 40GB Intel SSD 320 series connected into the optical drive bay
through its 5th SATA cable (internal). If you're brave enough you can test
OmniOS on a USB Flash...but I hard endurance troubles (so I switched to SSD
for the peace of my mind)...NAS4Free (FreeBSD based) runs flawlessly on USB
Stick on a pair of MicroServer I deployed as long term ZFS backup boxes
(really rock solid...on a Gen 7...about 37k working hours without a single
glitch). Hope it helps.

On Thu, Sep 29, 2016 at 4:54 PM, Paul Jochum <paul.jochum at nokia.com> wrote:

> Hi All:
>
>     Looking for the latest recommendations on a low powered and cheap
> OmniOS server.  This would be for home use, looking to use it as way of
> backing up Linux and Windows boxes.  Anyone have a recommendation?
>
> thanks in advance,
> Paul
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20160929/fb9c3094/attachment.html>

From mir at miras.org  Thu Sep 29 15:40:20 2016
From: mir at miras.org (Michael Rasmussen)
Date: Thu, 29 Sep 2016 17:40:20 +0200
Subject: [OmniOS-discuss] Low powered and cheap OmniOS server?
In-Reply-To: <03507cc7-d011-f58a-3bd3-1424935e9a5c@nokia.com>
References: <03507cc7-d011-f58a-3bd3-1424935e9a5c@nokia.com>
Message-ID: <20160929174020.63d2944c@sleipner.datanom.net>

On Thu, 29 Sep 2016 09:54:39 -0500
Paul Jochum <paul.jochum at nokia.com> wrote:

> Hi All:
> 
>      Looking for the latest recommendations on a low powered and cheap OmniOS server.  This would be for home use, looking to use it as way of backing up Linux and Windows boxes.  Anyone have a recommendation?
> 
You could build one yourself based on Intel I3-4(1|3)xxT (35W) using a
supermicro x10sll-f and you will have a solid build for $300
(MB+CPU+RAM).

-- 
Hilsen/Regards
Michael Rasmussen

Get my public GnuPG keys:
michael <at> rasmussen <dot> cc
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E
mir <at> datanom <dot> net
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C
mir <at> miras <dot> org
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917
--------------------------------------------------------------
/usr/games/fortune -es says:
A mathematician is a device for turning coffee into theorems.
		-- P. Erdos
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 163 bytes
Desc: OpenPGP digital signature
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20160929/a318bec7/attachment.bin>

From davide.poletto at gmail.com  Thu Sep 29 15:47:49 2016
From: davide.poletto at gmail.com (Davide Poletto)
Date: Thu, 29 Sep 2016 17:47:49 +0200
Subject: [OmniOS-discuss] Low powered and cheap OmniOS server?
In-Reply-To: <20160929174020.63d2944c@sleipner.datanom.net>
References: <03507cc7-d011-f58a-3bd3-1424935e9a5c@nokia.com>
	<20160929174020.63d2944c@sleipner.datanom.net>
Message-ID: <CANKMAMZ7CTz2-i9Tf4bYrcQ0bxyaevPR8XR4XaqieEO1W=ErNw@mail.gmail.com>

Uh...forgot...it's also cheap: the G1610T equipped with 4GB ECC - diskless
- should cost 170 Euro excluding VAT...and it has HPE iLO4 for management
on shared LAN (2x1Gbps). My 2 cents.

On Thu, Sep 29, 2016 at 5:40 PM, Michael Rasmussen <mir at miras.org> wrote:

> On Thu, 29 Sep 2016 09:54:39 -0500
> Paul Jochum <paul.jochum at nokia.com> wrote:
>
> > Hi All:
> >
> >      Looking for the latest recommendations on a low powered and cheap
> OmniOS server.  This would be for home use, looking to use it as way of
> backing up Linux and Windows boxes.  Anyone have a recommendation?
> >
> You could build one yourself based on Intel I3-4(1|3)xxT (35W) using a
> supermicro x10sll-f and you will have a solid build for $300
> (MB+CPU+RAM).
>
> --
> Hilsen/Regards
> Michael Rasmussen
>
> Get my public GnuPG keys:
> michael <at> rasmussen <dot> cc
> http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E
> mir <at> datanom <dot> net
> http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C
> mir <at> miras <dot> org
> http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917
> --------------------------------------------------------------
> /usr/games/fortune -es says:
> A mathematician is a device for turning coffee into theorems.
>                 -- P. Erdos
>
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20160929/1eb1c640/attachment-0001.html>

From mir at miras.org  Thu Sep 29 15:52:17 2016
From: mir at miras.org (Michael Rasmussen)
Date: Thu, 29 Sep 2016 17:52:17 +0200
Subject: [OmniOS-discuss] vlan and vnic
In-Reply-To: <B9757DE7-5F7E-4EDA-9CCF-16CC4215213B@omniti.com>
References: <20160929021227.2fe5f04f@sleipner.datanom.net>
	<B9757DE7-5F7E-4EDA-9CCF-16CC4215213B@omniti.com>
Message-ID: <20160929175217.0e55356b@sleipner.datanom.net>

On Thu, 29 Sep 2016 02:43:54 -0400
Dale Ghent <daleg at omniti.com> wrote:

> 
> Functionally as far as VLANs go, they are equivalent. However, VNICs make things more flexible.
> 
> VNICs can consume a aggregated set of physical interfaces, or a single physical interface. VNICs in turn can be assigned to Zones.
Another feature in vnic not available in vlan is that a vnic can be
assigned a mac which can come in handy. A vlan cannot be assigned a mac
so I assume a vlan inherits the mac from the link.

Given all this I have a hard time to find any pros for vlans compared
to vnics.

-- 
Hilsen/Regards
Michael Rasmussen

Get my public GnuPG keys:
michael <at> rasmussen <dot> cc
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E
mir <at> datanom <dot> net
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C
mir <at> miras <dot> org
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917
--------------------------------------------------------------
/usr/games/fortune -es says:
On the Internet, no one knows you're using Windows NT
		-- Submitted by Ramiro Estrugo, restrugo at fateware.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 163 bytes
Desc: OpenPGP digital signature
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20160929/ca07b02a/attachment.bin>

From bfriesen at simple.dallas.tx.us  Thu Sep 29 15:52:34 2016
From: bfriesen at simple.dallas.tx.us (Bob Friesenhahn)
Date: Thu, 29 Sep 2016 10:52:34 -0500 (CDT)
Subject: [OmniOS-discuss] Low powered and cheap OmniOS server?
In-Reply-To: <03507cc7-d011-f58a-3bd3-1424935e9a5c@nokia.com>
References: <03507cc7-d011-f58a-3bd3-1424935e9a5c@nokia.com>
Message-ID: <alpine.GSO.2.20.1609291045310.13864@freddy.simplesystems.org>

On Thu, 29 Sep 2016, Paul Jochum wrote:

> Hi All:
>
>    Looking for the latest recommendations on a low powered and cheap OmniOS 
> server.  This would be for home use, looking to use it as way of backing up 
> Linux and Windows boxes.  Anyone have a recommendation?

I am very happy with my 1U SuperMicro server based on the Xeon-D SOC. 
It takes hardly any power at all, makes very little noise, and 
performance is pretty good.  The system boots from two Samsung AHCI 
M.2 drives (one M.2 slot is via an add-in card).

The main drawback is that even though the chassis provides 8 hot-swap 
SFF drive bays, the SOC only supports 6 SATA interfaces so only 6 of 
those bays can be used without consuming the single card slot for a 
HBA.  If one adds an HBA then 8 hot-swap SFF slots are available, but 
only one M.2 slot is then available for use as a boot device.

I am not sure how much total storage can be attained with 6 SFF drives 
for backup server purposes.

Bob
-- 
Bob Friesenhahn
bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/

From natxo.asenjo at gmail.com  Thu Sep 29 18:33:32 2016
From: natxo.asenjo at gmail.com (Natxo Asenjo)
Date: Thu, 29 Sep 2016 20:33:32 +0200
Subject: [OmniOS-discuss] Low powered and cheap OmniOS server?
In-Reply-To: <03507cc7-d011-f58a-3bd3-1424935e9a5c@nokia.com>
References: <03507cc7-d011-f58a-3bd3-1424935e9a5c@nokia.com>
Message-ID: <CAHBEJzUagByaqVgmX4YEDPPqOP=ux_eucA9Wr-xubN0qiHJ_WA@mail.gmail.com>

hi,

On Thu, Sep 29, 2016 at 4:54 PM, Paul Jochum <paul.jochum at nokia.com> wrote:

> Hi All:
>
>     Looking for the latest recommendations on a low powered and cheap
> OmniOS server.  This would be for home use, looking to use it as way of
> backing up Linux and Windows boxes.  Anyone have a recommendation?
>

HP microserver 40NL, with a  IBM serveraid m1015 sas/sata raid controller
in jbod mode.

I use the internal 4 sata 2 connectors with 4x3Tib hard drives, and I
bought one of these (not available any more, unfortunately):
https://azerty.nl/0-1651-403300/sharkoon-sata-quickport-intern-6-bay-.html

I replaced the cd/dvd drive with that bay expansion, I have 2 laptop drives
as rpool and one ssd as cache. If I run out of space I can add 3 2,5'' hard
drives to the pool, or create a new pool. It even has a e-sata connection,
so I could add an external enclosure as well.

I got the kvm module as well, so it's quite a convenient litlle thing.

Not the fastest cpu, but hey, for that price and low consumption .. ;-)
--
Groeten,
natxo
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20160929/2a6b8e6b/attachment.html>

From bfriesen at simple.dallas.tx.us  Fri Sep 30 02:14:39 2016
From: bfriesen at simple.dallas.tx.us (Bob Friesenhahn)
Date: Thu, 29 Sep 2016 21:14:39 -0500 (CDT)
Subject: [OmniOS-discuss] vlan and vnic
In-Reply-To: <20160929175217.0e55356b@sleipner.datanom.net>
References: <20160929021227.2fe5f04f@sleipner.datanom.net>
	<B9757DE7-5F7E-4EDA-9CCF-16CC4215213B@omniti.com>
	<20160929175217.0e55356b@sleipner.datanom.net>
Message-ID: <alpine.GSO.2.20.1609292112530.13801@freddy.simplesystems.org>

On Thu, 29 Sep 2016, Michael Rasmussen wrote:
>
> Given all this I have a hard time to find any pros for vlans compared
> to vnics.

Obviously this is an Apples vs Orangutans comparison.  A vnic has a 
MAC which uniquely identifies an interface and a vlan is used to 
separate traffic (like a virtual switch) on the Ethernet.

Bob
-- 
Bob Friesenhahn
bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/

From mtalbott at lji.org  Fri Sep 30 02:21:40 2016
From: mtalbott at lji.org (Michael Talbott)
Date: Thu, 29 Sep 2016 19:21:40 -0700
Subject: [OmniOS-discuss] zpool Write Bottlenecks
Message-ID: <34569661-54B5-4C13-9662-D3C088EC7BFB@lji.org>

Hi, I'm trying to find a way to achieve massive write speeds with some decent hardware which will be used for some parallel computing needs (bioinformatics). Eventually if all goes well and my testing succeeds.. I'll be duplicating this setup and run BeeGFS in a few LX zones (THANK YOU LX ZONES!!!) to run some truly massive parallel computing storage happiness, but, I'd like to tune this box as much as possible first.

For this setup I'm using an Intel S2600GZ board and 2 x E5-2640 (six cores ea) @ 2.5GHz and there's 144GB ECC ram installed. I have 3 x SAS2008 based LSI cards in that box. 1 of those is connected to 8 internal SSDs and the other 2 cards (4 ports) are connected to a 45 bay drive enclosure. And then there's 2 intel x 2 port 10ge cards for connectivity.

I've created so many different zpools in different configurations between straight up striped with no redundancy, radiz2, raidz3.. used multipath, non-multipath'd with 8x phy links instead of 4x multipath links, etc, etc in order to find the magic combination for maximum performance, but there's something somewhere capping raw throughput and I can't seem to find it.

Now the crazy part is I can for instance, create zpoolA with ~20 drives (via cardA and attached only to backplaneA), create zpoolB with another ~20 drives (via cardB and attached only to backplaneB), and each of them gets the same performance individually (~1GB/s write and 2.5GB/s read). So, my thought is if I destroy zpoolB and attach all those drives to zpoolA via additional vdevs.. It should double the performance or make some sort of significant improvement, but, nope, roughly the same speed.. Then I thought, ok, well maybe it's a slowest vdev sort of thing.. So then I created vdevs such each vdev used half it's drives from backplaneA and the other half from backplaneB.. That would force data distribution between the cards for each vdev and double the speed and get me to 2GB/s write.. but, nope. same deal. 1G/s write and 2.5G read :(

When I create the pool from scratch, for each vdev I add I see a linear increase in performance until I hit about 4-5 vdevs.. That's where the performance flatlines and no matter what I do beyond that it just wont go any faster :(

Also, if I create a pure SSD pool with cardC, the linear read/write performance of those are hitting the exact same numbers as the others :( Bottom line, no matter what pool configuration I use, no matter what recordsize is set in zfs, I'm always getting capped with roughly 1GB/s write and 2.5GB/s read.

I thought maybe there wasn't enough PCIe lanes to run all of those cards at 8x, but, that's not the case, this board can run 6 x 8 lane PCIe 3.0 cards at full speed.. I booted it up in linux to use lspci -vv to make sure of it (since I'm not sure how to view pcie speeds in OmniOS), and sure enough, everything is running with 8x width, so that's not it.

Oh, and just fyi, this is my super simple throughput testing script that I run with compression disabled on the tested pool.

START=$(date +%s)
/usr/gnu/bin/dd if=/dev/zero of=/datastore/testdd bs=1M count=10k
sync
echo $(($(date +%s)-START))

My goal is to find a way to achieve at least 2GB/s write and 4GB/s read which I think is theoretically possbile with this hardware..

Anyone have any ideas of what could be limiting this or how to remedy it? Could it be the mpt_sas driver itself somehow throttling access to all these devices? Or maybe do I need to do some sort of irq-cpu pinning type of magic?


Thanks,

Michael


From daleg at omniti.com  Fri Sep 30 02:46:22 2016
From: daleg at omniti.com (Dale Ghent)
Date: Thu, 29 Sep 2016 22:46:22 -0400
Subject: [OmniOS-discuss] zpool Write Bottlenecks
In-Reply-To: <34569661-54B5-4C13-9662-D3C088EC7BFB@lji.org>
References: <34569661-54B5-4C13-9662-D3C088EC7BFB@lji.org>
Message-ID: <FA018B31-B458-431D-8F0E-2334DF933D2C@omniti.com>


Awesome that you're using LX Zones in a way with BeeGFS.

A note on your testing methodology, however:
http://lethargy.org/~jesus/writes/disk-benchmarking-with-dd-dont/#.V-3RUqOZPOY

> On Sep 29, 2016, at 10:21 PM, Michael Talbott <mtalbott at lji.org> wrote:
> 
> Hi, I'm trying to find a way to achieve massive write speeds with some decent hardware which will be used for some parallel computing needs (bioinformatics). Eventually if all goes well and my testing succeeds.. I'll be duplicating this setup and run BeeGFS in a few LX zones (THANK YOU LX ZONES!!!) to run some truly massive parallel computing storage happiness, but, I'd like to tune this box as much as possible first.
> 
> For this setup I'm using an Intel S2600GZ board and 2 x E5-2640 (six cores ea) @ 2.5GHz and there's 144GB ECC ram installed. I have 3 x SAS2008 based LSI cards in that box. 1 of those is connected to 8 internal SSDs and the other 2 cards (4 ports) are connected to a 45 bay drive enclosure. And then there's 2 intel x 2 port 10ge cards for connectivity.
> 
> I've created so many different zpools in different configurations between straight up striped with no redundancy, radiz2, raidz3.. used multipath, non-multipath'd with 8x phy links instead of 4x multipath links, etc, etc in order to find the magic combination for maximum performance, but there's something somewhere capping raw throughput and I can't seem to find it.
> 
> Now the crazy part is I can for instance, create zpoolA with ~20 drives (via cardA and attached only to backplaneA), create zpoolB with another ~20 drives (via cardB and attached only to backplaneB), and each of them gets the same performance individually (~1GB/s write and 2.5GB/s read). So, my thought is if I destroy zpoolB and attach all those drives to zpoolA via additional vdevs.. It should double the performance or make some sort of significant improvement, but, nope, roughly the same speed.. Then I thought, ok, well maybe it's a slowest vdev sort of thing.. So then I created vdevs such each vdev used half it's drives from backplaneA and the other half from backplaneB.. That would force data distribution between the cards for each vdev and double the speed and get me to 2GB/s write.. but, nope. same deal. 1G/s write and 2.5G read :(
> 
> When I create the pool from scratch, for each vdev I add I see a linear increase in performance until I hit about 4-5 vdevs.. That's where the performance flatlines and no matter what I do beyond that it just wont go any faster :(
> 
> Also, if I create a pure SSD pool with cardC, the linear read/write performance of those are hitting the exact same numbers as the others :( Bottom line, no matter what pool configuration I use, no matter what recordsize is set in zfs, I'm always getting capped with roughly 1GB/s write and 2.5GB/s read.
> 
> I thought maybe there wasn't enough PCIe lanes to run all of those cards at 8x, but, that's not the case, this board can run 6 x 8 lane PCIe 3.0 cards at full speed.. I booted it up in linux to use lspci -vv to make sure of it (since I'm not sure how to view pcie speeds in OmniOS), and sure enough, everything is running with 8x width, so that's not it.
> 
> Oh, and just fyi, this is my super simple throughput testing script that I run with compression disabled on the tested pool.
> 
> START=$(date +%s)
> /usr/gnu/bin/dd if=/dev/zero of=/datastore/testdd bs=1M count=10k
> sync
> echo $(($(date +%s)-START))
> 
> My goal is to find a way to achieve at least 2GB/s write and 4GB/s read which I think is theoretically possbile with this hardware..
> 
> Anyone have any ideas of what could be limiting this or how to remedy it? Could it be the mpt_sas driver itself somehow throttling access to all these devices? Or maybe do I need to do some sort of irq-cpu pinning type of magic?
> 
> 
> Thanks,
> 
> Michael
> 
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss


From mtalbott at lji.org  Fri Sep 30 05:19:38 2016
From: mtalbott at lji.org (Michael Talbott)
Date: Thu, 29 Sep 2016 22:19:38 -0700
Subject: [OmniOS-discuss] zpool Write Bottlenecks
In-Reply-To: <FA018B31-B458-431D-8F0E-2334DF933D2C@omniti.com>
References: <34569661-54B5-4C13-9662-D3C088EC7BFB@lji.org>
	<FA018B31-B458-431D-8F0E-2334DF933D2C@omniti.com>
Message-ID: <27B73CAE-7A40-42F6-AD37-1556015D6D4E@lji.org>

I'm very aware that dd is not the best tool for measuring disk performance in most cases. And I know the read throughput number is way off b/c of zfs caching mechanisms. However it does work in this case to illustrate the point in my case of a write throttle somewhere in the system. If anyone needs me to test with some other tool for illustrative purposes, I can do that too. It's just so odd that 1 card with a given set of disks attached in a pool provide roughly the same net throughput as 2 cards with 2 sets of disks in said pool.

But, for the nay-sayers of dd testing, I'll provide you with this.. Here's an example of using 2 x 11 disk raidz3 vdevs where all 22 drives live on one backplane attached to 1 card with an 8x phy sas connection. And then adding 2 more 11 disk raidz3 vdevs that are connected to that system on a separate card (also using an 8x phy sas link). No compression. Running bonnie++ to saturate the disks while I pull the numbers from iotop.

The following is an output of iotop (which uses dtrace for measuring disk io)
http://www.brendangregg.com/DTrace/iotop <http://www.brendangregg.com/DTrace/iotop>

Here's the 22 drive pool (all attached to 1 card):
------------------------------------------------------------------------------------------------------
2016 Sep 30 00:02:42,  load: 3.17,  disk_r:      0 KB,  disk_w: 3394084 KB

  UID    PID   PPID CMD              DEVICE  MAJ MIN D            BYTES
    0   7630      0 zpool-datastore  sd124   194 7936 W        161812480
    0   7630      0 zpool-datastore  sd118   194 7552 W        161820672
    0   7630      0 zpool-datastore  sd127   194 8128 W        161845248
    0   7630      0 zpool-datastore  sd128   194 8192 W        161845248
    0   7630      0 zpool-datastore  sd122   194 7808 W        161849344
    0   7630      0 zpool-datastore  sd119   194 7616 W        161857536
    0   7630      0 zpool-datastore  sd121   194 7744 W        161857536
    0   7630      0 zpool-datastore  sd125   194 8000 W        161865728
    0   7630      0 zpool-datastore  sd123   194 7872 W        161869824
    0   7630      0 zpool-datastore  sd126   194 8064 W        161873920
    0   7630      0 zpool-datastore  sd120   194 7680 W        161906688
    0   7630      0 zpool-datastore  sd136   194 8704 W        165916672
    0   7630      0 zpool-datastore  sd137   194 8768 W        165916672
    0   7630      0 zpool-datastore  sd138   194 8832 W        165933056
    0   7630      0 zpool-datastore  sd135   194 8640 W        165937152
    0   7630      0 zpool-datastore  sd139   194 8896 W        165941248
    0   7630      0 zpool-datastore  sd134   194 8576 W        165945344
    0   7630      0 zpool-datastore  sd130   194 8320 W        165974016
    0   7630      0 zpool-datastore  sd129   194 8256 W        165978112
    0   7630      0 zpool-datastore  sd132   194 8448 W        165994496
    0   7630      0 zpool-datastore  sd133   194 8512 W        165994496
    0   7630      0 zpool-datastore  sd131   194 8384 W        166006784

------------------------------------------------------------------------------------------------------


And here's the pool extended with 2 more raidz3s with 2 cards
notice it's almost LITERALLY cut in HALF per drive!


------------------------------------------------------------------------------------------------------


2016 Sep 30 00:01:07,  load: 4.59,  disk_r:      8 KB,  disk_w: 3609852 KB

  UID    PID   PPID CMD              DEVICE  MAJ MIN D            BYTES
    0   4550      0 zpool-datastore  sd133   194 8512 W         76558336
    0   4550      0 zpool-datastore  sd132   194 8448 W         76566528
    0   4550      0 zpool-datastore  sd135   194 8640 W         76570624
    0   4550      0 zpool-datastore  sd134   194 8576 W         76574720
    0   4550      0 zpool-datastore  sd136   194 8704 W         76582912
    0   4550      0 zpool-datastore  sd131   194 8384 W         76611584
    0   4550      0 zpool-datastore  sd130   194 8320 W         76644352
    0   4550      0 zpool-datastore  sd137   194 8768 W         76648448
    0   4550      0 zpool-datastore  sd129   194 8256 W         76660736
    0   4550      0 zpool-datastore  sd138   194 8832 W         76713984
    0   4550      0 zpool-datastore  sd139   194 8896 W         77369344
    0   4550      0 zpool-datastore  sd113   194 7232 W         77770752
    0   4550      0 zpool-datastore  sd115   194 7360 W         77832192
    0   4550      0 zpool-datastore  sd114   194 7296 W         77836288
    0   4550      0 zpool-datastore  sd111   194 7104 W         77840384
    0   4550      0 zpool-datastore  sd112   194 7168 W         77840384
    0   4550      0 zpool-datastore  sd108   194 6912 W         77844480
    0   4550      0 zpool-datastore  sd110   194 7040 W         77864960
    0   4550      0 zpool-datastore  sd116   194 7424 W         77873152
    0   4550      0 zpool-datastore  sd107   194 6848 W         77914112
    0   4550      0 zpool-datastore  sd106   194 6784 W         77918208
    0   4550      0 zpool-datastore  sd109   194 6976 W         77926400
    0   4550      0 zpool-datastore  sd128   194 8192 W         78938112
    0   4550      0 zpool-datastore  sd118   194 7552 W         78979072
    0   4550      0 zpool-datastore  sd125   194 8000 W         78991360
    0   4550      0 zpool-datastore  sd120   194 7680 W         78999552
    0   4550      0 zpool-datastore  sd127   194 8128 W         79007744
    0   4550      0 zpool-datastore  sd126   194 8064 W         79011840
    0   4550      0 zpool-datastore  sd119   194 7616 W         79020032
    0   4550      0 zpool-datastore  sd123   194 7872 W         79020032
    0   4550      0 zpool-datastore  sd122   194 7808 W         79048704
    0   4550      0 zpool-datastore  sd124   194 7936 W         79056896
    0   4550      0 zpool-datastore  sd121   194 7744 W         79065088
    0   4550      0 zpool-datastore  sd105   194 6720 W         82460672
    0   4550      0 zpool-datastore  sd102   194 6528 W         82468864
    0   4550      0 zpool-datastore  sd101   194 6464 W         82477056
    0   4550      0 zpool-datastore  sd104   194 6656 W         82477056
    0   4550      0 zpool-datastore  sd103   194 6592 W         82481152
    0   4550      0 zpool-datastore  sd141   194 9024 W         82489344
    0   4550      0 zpool-datastore  sd99    194 6336 W         82493440
    0   4550      0 zpool-datastore  sd98    194 6272 W         82501632
    0   4550      0 zpool-datastore  sd140   194 8960 W         82513920
    0   4550      0 zpool-datastore  sd97    194 6208 W         82538496
    0   4550      0 zpool-datastore  sd100   194 6400 W         82542592

------------------------------------------------------------------------------------------------------




Any thoughts of how to discover and/or overcome the true bottleneck is much appreciated.

Thanks

Michael



> On Sep 29, 2016, at 7:46 PM, Dale Ghent <daleg at omniti.com> wrote:
> 
> 
> Awesome that you're using LX Zones in a way with BeeGFS.
> 
> A note on your testing methodology, however:
> http://lethargy.org/~jesus/writes/disk-benchmarking-with-dd-dont/#.V-3RUqOZPOY
> 
>> On Sep 29, 2016, at 10:21 PM, Michael Talbott <mtalbott at lji.org> wrote:
>> 
>> Hi, I'm trying to find a way to achieve massive write speeds with some decent hardware which will be used for some parallel computing needs (bioinformatics). Eventually if all goes well and my testing succeeds.. I'll be duplicating this setup and run BeeGFS in a few LX zones (THANK YOU LX ZONES!!!) to run some truly massive parallel computing storage happiness, but, I'd like to tune this box as much as possible first.
>> 
>> For this setup I'm using an Intel S2600GZ board and 2 x E5-2640 (six cores ea) @ 2.5GHz and there's 144GB ECC ram installed. I have 3 x SAS2008 based LSI cards in that box. 1 of those is connected to 8 internal SSDs and the other 2 cards (4 ports) are connected to a 45 bay drive enclosure. And then there's 2 intel x 2 port 10ge cards for connectivity.
>> 
>> I've created so many different zpools in different configurations between straight up striped with no redundancy, radiz2, raidz3.. used multipath, non-multipath'd with 8x phy links instead of 4x multipath links, etc, etc in order to find the magic combination for maximum performance, but there's something somewhere capping raw throughput and I can't seem to find it.
>> 
>> Now the crazy part is I can for instance, create zpoolA with ~20 drives (via cardA and attached only to backplaneA), create zpoolB with another ~20 drives (via cardB and attached only to backplaneB), and each of them gets the same performance individually (~1GB/s write and 2.5GB/s read). So, my thought is if I destroy zpoolB and attach all those drives to zpoolA via additional vdevs.. It should double the performance or make some sort of significant improvement, but, nope, roughly the same speed.. Then I thought, ok, well maybe it's a slowest vdev sort of thing.. So then I created vdevs such each vdev used half it's drives from backplaneA and the other half from backplaneB.. That would force data distribution between the cards for each vdev and double the speed and get me to 2GB/s write.. but, nope. same deal. 1G/s write and 2.5G read :(
>> 
>> When I create the pool from scratch, for each vdev I add I see a linear increase in performance until I hit about 4-5 vdevs.. That's where the performance flatlines and no matter what I do beyond that it just wont go any faster :(
>> 
>> Also, if I create a pure SSD pool with cardC, the linear read/write performance of those are hitting the exact same numbers as the others :( Bottom line, no matter what pool configuration I use, no matter what recordsize is set in zfs, I'm always getting capped with roughly 1GB/s write and 2.5GB/s read.
>> 
>> I thought maybe there wasn't enough PCIe lanes to run all of those cards at 8x, but, that's not the case, this board can run 6 x 8 lane PCIe 3.0 cards at full speed.. I booted it up in linux to use lspci -vv to make sure of it (since I'm not sure how to view pcie speeds in OmniOS), and sure enough, everything is running with 8x width, so that's not it.
>> 
>> Oh, and just fyi, this is my super simple throughput testing script that I run with compression disabled on the tested pool.
>> 
>> START=$(date +%s)
>> /usr/gnu/bin/dd if=/dev/zero of=/datastore/testdd bs=1M count=10k
>> sync
>> echo $(($(date +%s)-START))
>> 
>> My goal is to find a way to achieve at least 2GB/s write and 4GB/s read which I think is theoretically possbile with this hardware..
>> 
>> Anyone have any ideas of what could be limiting this or how to remedy it? Could it be the mpt_sas driver itself somehow throttling access to all these devices? Or maybe do I need to do some sort of irq-cpu pinning type of magic?
>> 
>> 
>> Thanks,
>> 
>> Michael
>> 
>> _______________________________________________
>> OmniOS-discuss mailing list
>> OmniOS-discuss at lists.omniti.com
>> http://lists.omniti.com/mailman/listinfo/omnios-discuss
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20160929/8b1a114a/attachment-0001.html>

From lkateley at kateley.com  Fri Sep 30 05:25:41 2016
From: lkateley at kateley.com (Linda Kateley)
Date: Fri, 30 Sep 2016 00:25:41 -0500
Subject: [OmniOS-discuss] zpool Write Bottlenecks
In-Reply-To: <27B73CAE-7A40-42F6-AD37-1556015D6D4E@lji.org>
References: <34569661-54B5-4C13-9662-D3C088EC7BFB@lji.org>
	<FA018B31-B458-431D-8F0E-2334DF933D2C@omniti.com>
	<27B73CAE-7A40-42F6-AD37-1556015D6D4E@lji.org>
Message-ID: <0239a6ea-52fb-5e56-1fcf-cddd7c6dcea1@kateley.com>

One of the things I do is turn caching off. on a dataset #zfs get 
primarycache=none dataset. Give me better disk performance

lk


On 9/30/16 12:19 AM, Michael Talbott wrote:
> I'm very aware that dd is not the best tool for measuring disk 
> performance in most cases. And I know the read throughput number is 
> way off b/c of zfs caching mechanisms. However it does work in this 
> case to illustrate the point in my case of a write throttle somewhere 
> in the system. If anyone needs me to test with some other tool for 
> illustrative purposes, I can do that too. It's just so odd that 1 card 
> with a given set of disks attached in a pool provide roughly the same 
> net throughput as 2 cards with 2 sets of disks in said pool.
>
> But, for the nay-sayers of dd testing, I'll provide you with this.. 
> Here's an example of using 2 x 11 disk raidz3 vdevs where all 22 
> drives live on one backplane attached to 1 card with an 8x phy sas 
> connection. And then adding 2 more 11 disk raidz3 vdevs that are 
> connected to that system on a separate card (also using an 8x phy sas 
> link). No compression. Running bonnie++ to saturate the disks while I 
> pull the numbers from iotop.
>
> The following is an output of iotop (which uses dtrace for measuring 
> disk io)
> http://www.brendangregg.com/DTrace/iotop
>
> Here's the 22 drive pool (all attached to 1 card):
> ------------------------------------------------------------------------------------------------------
> 2016 Sep 30 00:02:42,  load: 3.17,  disk_r:      0 KB,  disk_w: 3394084 KB
>
>   UID    PID   PPID CMD              DEVICE  MAJ MIN D    BYTES
>     0   7630      0 zpool-datastore  sd124   194 7936 W  161812480
>     0   7630      0 zpool-datastore  sd118   194 7552 W  161820672
>     0   7630      0 zpool-datastore  sd127   194 8128 W  161845248
>     0   7630      0 zpool-datastore  sd128   194 8192 W  161845248
>     0   7630      0 zpool-datastore  sd122   194 7808 W  161849344
>     0   7630      0 zpool-datastore  sd119   194 7616 W  161857536
>     0   7630      0 zpool-datastore  sd121   194 7744 W  161857536
>     0   7630      0 zpool-datastore  sd125   194 8000 W  161865728
>     0   7630      0 zpool-datastore  sd123   194 7872 W  161869824
>     0   7630      0 zpool-datastore  sd126   194 8064 W  161873920
>     0   7630      0 zpool-datastore  sd120   194 7680 W  161906688
>     0   7630      0 zpool-datastore  sd136   194 8704 W  165916672
>     0   7630      0 zpool-datastore  sd137   194 8768 W  165916672
>     0   7630      0 zpool-datastore  sd138   194 8832 W  165933056
>     0   7630      0 zpool-datastore  sd135   194 8640 W  165937152
>     0   7630      0 zpool-datastore  sd139   194 8896 W  165941248
>     0   7630      0 zpool-datastore  sd134   194 8576 W  165945344
>     0   7630      0 zpool-datastore  sd130   194 8320 W  165974016
>     0   7630      0 zpool-datastore  sd129   194 8256 W  165978112
>     0   7630      0 zpool-datastore  sd132   194 8448 W  165994496
>     0   7630      0 zpool-datastore  sd133   194 8512 W  165994496
>     0   7630      0 zpool-datastore  sd131   194 8384 W  166006784
>
> ------------------------------------------------------------------------------------------------------
>
>
> And here's the pool extended with 2 more raidz3s with 2 cards
> notice it's almost LITERALLY cut in HALF per drive!
>
>
> ------------------------------------------------------------------------------------------------------
>
>
> 2016 Sep 30 00:01:07,  load: 4.59,  disk_r:      8 KB,  disk_w: 3609852 KB
>
>   UID    PID   PPID CMD              DEVICE  MAJ MIN D    BYTES
>     0   4550      0 zpool-datastore  sd133   194 8512 W   76558336
>     0   4550      0 zpool-datastore  sd132   194 8448 W   76566528
>     0   4550      0 zpool-datastore  sd135   194 8640 W   76570624
>     0   4550      0 zpool-datastore  sd134   194 8576 W   76574720
>     0   4550      0 zpool-datastore  sd136   194 8704 W   76582912
>     0   4550      0 zpool-datastore  sd131   194 8384 W   76611584
>     0   4550      0 zpool-datastore  sd130   194 8320 W   76644352
>     0   4550      0 zpool-datastore  sd137   194 8768 W   76648448
>     0   4550      0 zpool-datastore  sd129   194 8256 W   76660736
>     0   4550      0 zpool-datastore  sd138   194 8832 W   76713984
>     0   4550      0 zpool-datastore  sd139   194 8896 W   77369344
>     0   4550      0 zpool-datastore  sd113   194 7232 W   77770752
>     0   4550      0 zpool-datastore  sd115   194 7360 W   77832192
>     0   4550      0 zpool-datastore  sd114   194 7296 W   77836288
>     0   4550      0 zpool-datastore  sd111   194 7104 W   77840384
>     0   4550      0 zpool-datastore  sd112   194 7168 W   77840384
>     0   4550      0 zpool-datastore  sd108   194 6912 W   77844480
>     0   4550      0 zpool-datastore  sd110   194 7040 W   77864960
>     0   4550      0 zpool-datastore  sd116   194 7424 W   77873152
>     0   4550      0 zpool-datastore  sd107   194 6848 W   77914112
>     0   4550      0 zpool-datastore  sd106   194 6784 W   77918208
>     0   4550      0 zpool-datastore  sd109   194 6976 W   77926400
>     0   4550      0 zpool-datastore  sd128   194 8192 W   78938112
>     0   4550      0 zpool-datastore  sd118   194 7552 W   78979072
>     0   4550      0 zpool-datastore  sd125   194 8000 W   78991360
>     0   4550      0 zpool-datastore  sd120   194 7680 W   78999552
>     0   4550      0 zpool-datastore  sd127   194 8128 W   79007744
>     0   4550      0 zpool-datastore  sd126   194 8064 W   79011840
>     0   4550      0 zpool-datastore  sd119   194 7616 W   79020032
>     0   4550      0 zpool-datastore  sd123   194 7872 W   79020032
>     0   4550      0 zpool-datastore  sd122   194 7808 W   79048704
>     0   4550      0 zpool-datastore  sd124   194 7936 W   79056896
>     0   4550      0 zpool-datastore  sd121   194 7744 W   79065088
>     0   4550      0 zpool-datastore  sd105   194 6720 W   82460672
>     0   4550      0 zpool-datastore  sd102   194 6528 W   82468864
>     0   4550      0 zpool-datastore  sd101   194 6464 W   82477056
>     0   4550      0 zpool-datastore  sd104   194 6656 W   82477056
>     0   4550      0 zpool-datastore  sd103   194 6592 W   82481152
>     0   4550      0 zpool-datastore  sd141   194 9024 W   82489344
>     0   4550      0 zpool-datastore  sd99    194 6336 W   82493440
>     0   4550      0 zpool-datastore  sd98    194 6272 W   82501632
>     0   4550      0 zpool-datastore  sd140   194 8960 W   82513920
>     0   4550      0 zpool-datastore  sd97    194 6208 W   82538496
>     0   4550      0 zpool-datastore  sd100   194 6400 W   82542592
>
> ------------------------------------------------------------------------------------------------------
>
>
>
>
> Any thoughts of how to discover and/or overcome the true bottleneck is 
> much appreciated.
>
> Thanks
>
> Michael
>
>
>
>> On Sep 29, 2016, at 7:46 PM, Dale Ghent <daleg at omniti.com 
>> <mailto:daleg at omniti.com>> wrote:
>>
>>
>> Awesome that you're using LX Zones in a way with BeeGFS.
>>
>> A note on your testing methodology, however:
>> http://lethargy.org/~jesus/writes/disk-benchmarking-with-dd-dont/#.V-3RUqOZPOY 
>> <http://lethargy.org/%7Ejesus/writes/disk-benchmarking-with-dd-dont/#.V-3RUqOZPOY>
>>
>>> On Sep 29, 2016, at 10:21 PM, Michael Talbott <mtalbott at lji.org> wrote:
>>>
>>> Hi, I'm trying to find a way to achieve massive write speeds with 
>>> some decent hardware which will be used for some parallel computing 
>>> needs (bioinformatics). Eventually if all goes well and my testing 
>>> succeeds.. I'll be duplicating this setup and run BeeGFS in a few LX 
>>> zones (THANK YOU LX ZONES!!!) to run some truly massive parallel 
>>> computing storage happiness, but, I'd like to tune this box as much 
>>> as possible first.
>>>
>>> For this setup I'm using an Intel S2600GZ board and 2 x E5-2640 (six 
>>> cores ea) @ 2.5GHz and there's 144GB ECC ram installed. I have 3 x 
>>> SAS2008 based LSI cards in that box. 1 of those is connected to 8 
>>> internal SSDs and the other 2 cards (4 ports) are connected to a 45 
>>> bay drive enclosure. And then there's 2 intel x 2 port 10ge cards 
>>> for connectivity.
>>>
>>> I've created so many different zpools in different configurations 
>>> between straight up striped with no redundancy, radiz2, raidz3.. 
>>> used multipath, non-multipath'd with 8x phy links instead of 4x 
>>> multipath links, etc, etc in order to find the magic combination for 
>>> maximum performance, but there's something somewhere capping raw 
>>> throughput and I can't seem to find it.
>>>
>>> Now the crazy part is I can for instance, create zpoolA with ~20 
>>> drives (via cardA and attached only to backplaneA), create zpoolB 
>>> with another ~20 drives (via cardB and attached only to backplaneB), 
>>> and each of them gets the same performance individually (~1GB/s 
>>> write and 2.5GB/s read). So, my thought is if I destroy zpoolB and 
>>> attach all those drives to zpoolA via additional vdevs.. It should 
>>> double the performance or make some sort of significant improvement, 
>>> but, nope, roughly the same speed.. Then I thought, ok, well maybe 
>>> it's a slowest vdev sort of thing.. So then I created vdevs such 
>>> each vdev used half it's drives from backplaneA and the other half 
>>> from backplaneB.. That would force data distribution between the 
>>> cards for each vdev and double the speed and get me to 2GB/s write.. 
>>> but, nope. same deal. 1G/s write and 2.5G read :(
>>>
>>> When I create the pool from scratch, for each vdev I add I see a 
>>> linear increase in performance until I hit about 4-5 vdevs.. That's 
>>> where the performance flatlines and no matter what I do beyond that 
>>> it just wont go any faster :(
>>>
>>> Also, if I create a pure SSD pool with cardC, the linear read/write 
>>> performance of those are hitting the exact same numbers as the 
>>> others :( Bottom line, no matter what pool configuration I use, no 
>>> matter what recordsize is set in zfs, I'm always getting capped with 
>>> roughly 1GB/s write and 2.5GB/s read.
>>>
>>> I thought maybe there wasn't enough PCIe lanes to run all of those 
>>> cards at 8x, but, that's not the case, this board can run 6 x 8 lane 
>>> PCIe 3.0 cards at full speed.. I booted it up in linux to use lspci 
>>> -vv to make sure of it (since I'm not sure how to view pcie speeds 
>>> in OmniOS), and sure enough, everything is running with 8x width, so 
>>> that's not it.
>>>
>>> Oh, and just fyi, this is my super simple throughput testing script 
>>> that I run with compression disabled on the tested pool.
>>>
>>> START=$(date +%s)
>>> /usr/gnu/bin/dd if=/dev/zero of=/datastore/testdd bs=1M count=10k
>>> sync
>>> echo $(($(date +%s)-START))
>>>
>>> My goal is to find a way to achieve at least 2GB/s write and 4GB/s 
>>> read which I think is theoretically possbile with this hardware..
>>>
>>> Anyone have any ideas of what could be limiting this or how to 
>>> remedy it? Could it be the mpt_sas driver itself somehow throttling 
>>> access to all these devices? Or maybe do I need to do some sort of 
>>> irq-cpu pinning type of magic?
>>>
>>>
>>> Thanks,
>>>
>>> Michael
>>>
>>> _______________________________________________
>>> OmniOS-discuss mailing list
>>> OmniOS-discuss at lists.omniti.com
>>> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>>
>
>
>
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20160930/de47fe2d/attachment-0001.html>

From bfriesen at simple.dallas.tx.us  Fri Sep 30 13:36:02 2016
From: bfriesen at simple.dallas.tx.us (Bob Friesenhahn)
Date: Fri, 30 Sep 2016 08:36:02 -0500 (CDT)
Subject: [OmniOS-discuss] zpool Write Bottlenecks
In-Reply-To: <27B73CAE-7A40-42F6-AD37-1556015D6D4E@lji.org>
References: <34569661-54B5-4C13-9662-D3C088EC7BFB@lji.org>
	<FA018B31-B458-431D-8F0E-2334DF933D2C@omniti.com>
	<27B73CAE-7A40-42F6-AD37-1556015D6D4E@lji.org>
Message-ID: <alpine.GSO.2.20.1609300817350.13801@freddy.simplesystems.org>

On Thu, 29 Sep 2016, Michael Talbott wrote:

> I'm very aware that dd is not the best tool for measuring disk 
> performance in most cases. And I know the read throughput number is 
> way off b/c of zfs caching mechanisms. However it does work in this 
> case to illustrate the point in my case of a write throttle 
> somewhere in the system. If anyone needs me to test with some other

I think that Dale is correct that your benchmark may be only 
benchmarking 'dd'.  The performance of single-threaded 'dd' will be 
entirely driven by read performance of /dev/zero followed by write 
latency per operation on writes.  In your case, the issue is almost 
certainly that the write latency is a fixed value (regardless of the 
number of disks) and so there is a fixed maximum write rate regardless 
of the performance of the underlying store.

If you were to run two 'dd's in parallel, you may see an increase in 
the write rate.  This is the first thing to try.

With zfs, a "write" goes into a buffer associated with a transaction 
group (TXG) which is written to disk based on a maximum elapsed time, 
or the maximum size of the TXG being hit.  The maximum size of the TXG 
is estimated based on the available total RAM and zfs's guess as to 
the time to write the TXG (based on throughput), as well as tunables.

For a brief time, the writer proces will not be able to write any data 
at all as the current TXG is finished (resulting in writes) and a new 
one is started.

At the lowest level, the TXG can not be completed until all involved 
disks have written their data and completed a cache sync operation. 
The latency of the cache sync is driven by the latency of the slowest 
disk drive (+HBA) involved.  Getting rid of any disks which exhibit 
abnormally high latency would help with the transaction times.

Bob
-- 
Bob Friesenhahn
bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/

From mtalbott at lji.org  Fri Sep 30 16:46:23 2016
From: mtalbott at lji.org (Michael Talbott)
Date: Fri, 30 Sep 2016 09:46:23 -0700
Subject: [OmniOS-discuss] zpool Write Bottlenecks
In-Reply-To: <alpine.GSO.2.20.1609300817350.13801@freddy.simplesystems.org>
References: <34569661-54B5-4C13-9662-D3C088EC7BFB@lji.org>
	<FA018B31-B458-431D-8F0E-2334DF933D2C@omniti.com>
	<27B73CAE-7A40-42F6-AD37-1556015D6D4E@lji.org>
	<alpine.GSO.2.20.1609300817350.13801@freddy.simplesystems.org>
Message-ID: <CE812361-A027-4C79-99C5-75B1C4C860BD@lji.org>

Ah. That explains it. At first I figured the single stream couldn't be the case since a dd from /dev/zero to /dev/null was in excess of 24GB/s. But what you said  about TX groups explains it all. I ran several dd's in parallel and now see all of the disks getting fully saturated.

Thanks


> On Sep 30, 2016, at 6:36 AM, Bob Friesenhahn <bfriesen at simple.dallas.tx.us> wrote:
> 
> On Thu, 29 Sep 2016, Michael Talbott wrote:
> 
>> I'm very aware that dd is not the best tool for measuring disk performance in most cases. And I know the read throughput number is way off b/c of zfs caching mechanisms. However it does work in this case to illustrate the point in my case of a write throttle somewhere in the system. If anyone needs me to test with some other
> 
> I think that Dale is correct that your benchmark may be only benchmarking 'dd'.  The performance of single-threaded 'dd' will be entirely driven by read performance of /dev/zero followed by write latency per operation on writes.  In your case, the issue is almost certainly that the write latency is a fixed value (regardless of the number of disks) and so there is a fixed maximum write rate regardless of the performance of the underlying store.
> 
> If you were to run two 'dd's in parallel, you may see an increase in the write rate.  This is the first thing to try.
> 
> With zfs, a "write" goes into a buffer associated with a transaction group (TXG) which is written to disk based on a maximum elapsed time, or the maximum size of the TXG being hit.  The maximum size of the TXG is estimated based on the available total RAM and zfs's guess as to the time to write the TXG (based on throughput), as well as tunables.
> 
> For a brief time, the writer proces will not be able to write any data at all as the current TXG is finished (resulting in writes) and a new one is started.
> 
> At the lowest level, the TXG can not be completed until all involved disks have written their data and completed a cache sync operation. The latency of the cache sync is driven by the latency of the slowest disk drive (+HBA) involved.  Getting rid of any disks which exhibit abnormally high latency would help with the transaction times.
> 
> Bob
> -- 
> Bob Friesenhahn
> bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
> GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/


From mtalbott at lji.org  Fri Sep 30 21:30:21 2016
From: mtalbott at lji.org (Michael Talbott)
Date: Fri, 30 Sep 2016 14:30:21 -0700
Subject: [OmniOS-discuss] extended attributes in LX zone
Message-ID: <4F6BE78F-D4F9-4C7A-B024-F925023B86CF@lji.org>

So I've hit a bit of a brick wall in my testing to get BeeGFS working in an LX zone :( The metadata server needs extended attributes. I know zfs has a different implementation of EAs, but they are there.. So.. Is it possible for an LX zone to somehow use them with native linux tools like getfattr, setfattr, etc? I tried adding xattr and user_xattr to the zone's fstab file hoping just maybe it was that simple, but, no dice. Any ideas?

Thanks

Michael

From daleg at omniti.com  Fri Sep 30 22:40:03 2016
From: daleg at omniti.com (Dale Ghent)
Date: Fri, 30 Sep 2016 18:40:03 -0400
Subject: [OmniOS-discuss] extended attributes in LX zone
In-Reply-To: <4F6BE78F-D4F9-4C7A-B024-F925023B86CF@lji.org>
References: <4F6BE78F-D4F9-4C7A-B024-F925023B86CF@lji.org>
Message-ID: <C4180B3F-2964-438C-9828-12EBA0C1AB5E@omniti.com>


> On Sep 30, 2016, at 5:30 PM, Michael Talbott <mtalbott at lji.org> wrote:
> 
> So I've hit a bit of a brick wall in my testing to get BeeGFS working in an LX zone :( The metadata server needs extended attributes. I know zfs has a different implementation of EAs, but they are there.. So.. Is it possible for an LX zone to somehow use them with native linux tools like getfattr, setfattr, etc? I tried adding xattr and user_xattr to the zone's fstab file hoping just maybe it was that simple, but, no dice. Any ideas?

No support for linux get/setxattr syscalls right now in LX brants other than superficial faking to please certain circumstances:

https://grok.elemental.org/source/xref/io-lx-public/usr/src/uts/common/brand/lx/syscall/lx_xattr.c#33

/dale

From andykinney at advantagecom.net  Fri Sep 30 22:52:34 2016
From: andykinney at advantagecom.net (Andrew Kinney)
Date: Fri, 30 Sep 2016 15:52:34 -0700
Subject: [OmniOS-discuss] smartctl and SCT ERC on omnios
Message-ID: <57EEECB2.5030701@advantagecom.net>

Hello,

I have a Samsung 845DC Pro SSD that I need to respond more timely during 
problems with flash cells. We had an incident where it took 2 minutes to 
fail a cell erase and bring in a spare cell, which is too long for a 
redundant ZFS log device on a pool with a dataset with property 
sync=always serving iSCSI volumes.

smartctl shows it supports SCT ERC:

# smartctl -x -d sat,12 /dev/rdsk/c4t0d0s0

smartctl 6.5 2016-05-07 r4318 [i386-pc-solaris2.11] (local build)
Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Device Model:     Samsung SSD 845DC PRO 400GB
<snip>
ATA Version is:   ACS-2, ATA8-ACS T13/1699-D revision 4c
SATA Version is:  SATA 3.1, 6.0 Gb/s (current: 6.0 Gb/s)
<snip>
SCT capabilities:              (0x003d) SCT Status supported.
                                         SCT Error Recovery Control 
supported.
                                         SCT Feature Control supported.
                                         SCT Data Table supported.
<snip>


I should be able to read its current SCT ERC settings, but get an error:

# smartctl -l scterc -d sat,12 /dev/rdsk/c4t0d0s0
smartctl 6.5 2016-05-07 r4318 [i386-pc-solaris2.11] (local build)
Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org

SMART WRITE LOG returns COUNT and LBA_LOW register unchanged
SCT (Get) Error Recovery Control command failed


I would go read/write the scterc values on a different OS as a 
workaround, but I've read that Samsung devices do not persist CCTL 
values across reboots. At present, I don't know what it's set to or if 
it's being honored.

It appears that this is a limitation of smartmontools on Solaris and its 
derivatives.

Any ideas how I can read and set the scterc values on Omnios?

--
Andrew

From mtalbott at lji.org  Fri Sep 30 22:57:26 2016
From: mtalbott at lji.org (Michael Talbott)
Date: Fri, 30 Sep 2016 15:57:26 -0700
Subject: [OmniOS-discuss] extended attributes in LX zone
In-Reply-To: <C4180B3F-2964-438C-9828-12EBA0C1AB5E@omniti.com>
References: <4F6BE78F-D4F9-4C7A-B024-F925023B86CF@lji.org>
	<C4180B3F-2964-438C-9828-12EBA0C1AB5E@omniti.com>
Message-ID: <7DCFD721-221D-42C7-A497-A4A7E323B3FB@lji.org>

Bummer. Oh well. How about presenting a zvol as a raw device like shown here: http://docs.oracle.com/cd/E19253-01/819-5461/gbebi/index.html <http://docs.oracle.com/cd/E19253-01/819-5461/gbebi/index.html>

I did this, shutdown and rebooted the zone.. and the path the the volume appears under /dev/zvol/dsk/ssdpool/ but that directory is empty when it should contain the name of my zvol :(

I figure if I can get that working, then I could format that vol in ext4 so it'd have a way of properly handling xattrs (or so I hope).. Otherwise.. I guess I could present a zvol as an iscsi volume to the linux host/zone/entity and then format that ext4 over iscsi, but I'd rather not add the networking stack in there if I don't have to.

Any ideas on getting zvols as raw devices in an lx zone?


________________________
Michael Talbott
Systems Administrator
La Jolla Institute

> On Sep 30, 2016, at 3:40 PM, Dale Ghent <daleg at omniti.com> wrote:
> 
> 
>> On Sep 30, 2016, at 5:30 PM, Michael Talbott <mtalbott at lji.org> wrote:
>> 
>> So I've hit a bit of a brick wall in my testing to get BeeGFS working in an LX zone :( The metadata server needs extended attributes. I know zfs has a different implementation of EAs, but they are there.. So.. Is it possible for an LX zone to somehow use them with native linux tools like getfattr, setfattr, etc? I tried adding xattr and user_xattr to the zone's fstab file hoping just maybe it was that simple, but, no dice. Any ideas?
> 
> No support for linux get/setxattr syscalls right now in LX brants other than superficial faking to please certain circumstances:
> 
> https://grok.elemental.org/source/xref/io-lx-public/usr/src/uts/common/brand/lx/syscall/lx_xattr.c#33
> 
> /dale

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20160930/6c9b6d1c/attachment.html>

