From lists at mcintyreweb.com  Mon Dec  1 01:33:00 2014
From: lists at mcintyreweb.com (Hugh McIntyre)
Date: Sun, 30 Nov 2014 17:33:00 -0800
Subject: [OmniOS-discuss] r151006 -> r151012 upgrade problems after
	bash/openssl patches ...
Message-ID: <547BC54C.6060805@mcintyreweb.com>


I have a backup system which I'm trying to upgrade from 151006 to 151012 
-- I stayed on 151006 for a long time since this is just a backup 
system, but now want to upgrade because of some stuck zpool IO issues.

I'm starting by trying to upgrade from 151006->151008 (based on the 
release notes because of package signatures) and following the 
instructions under 
http://omnios.omniti.com/wiki.php/Upgrade_r151006_r151008.

The eventual problem is that when I run "pkg update 
--be-name=omnios-r151008 entire at 11,5.11-0.151008" it fails because of 
conflicting package versions.  Although I had previously frozen at 
r151006, the conflicting packages seem to be omnios/shell/bash, 
omnios/system/library, and omnios/library/security/openssl.

Apparently the r151008 version of OmniOS does not work with the 
bash/heartbleed patches, which I cannot really complain about since 
r151008 is so old.  But is there still a way to perform the upgrade, or 
am I stuck with needing to do a clean install?

Hugh.


PS: failing output:

# pkg update --be-name=omnios-r151008 entire at 11,5.11-0.151008
Creating Plan /
pkg update: No matching version of entire can be installed:
   Reject:  pkg://omnios/entire at 11,5.11-0.151008:20131204T230829Z
            pkg://omnios/entire at 11,5.11-0.151008:20131205T191306Z
   Reason:  All versions matching 'require' dependency 
pkg:/compress/p7zip at 9.20.1,5.11-0.151008 are rejected
     Reject: 
pkg://omnios/compress/p7zip at 9.20.1,5.11-0.151008:20131204T022414Z
     Reason:  All versions matching 'require' dependency pkg:/shell/bash 
are rejected
       Reject:  pkg://omnios/shell/bash at 4.2,5.11-0.151002:20120401T175628Z
                pkg://omnios/shell/bash at 4.2,5.11-0.151004:20121014T211709Z
 
pkg://omnios/shell/bash at 4.2.45,5.11-0.151006:20130506T192652Z
 
pkg://omnios/shell/bash at 4.2.45,5.11-0.151006:20140924T200933Z
       Reason:  Excluded by proposed incorporation 
'incorporation/jeos/omnios-userland'
                Newer version 
pkg://omnios/shell/bash at 4.2.53,5.11-0.151006:20141006T151920Z is already 
installed
       Reject: 
pkg://omnios/shell/bash at 4.2.45,5.11-0.151008:20131204T024624Z
       Reason:  Newer version 
pkg://omnios/shell/bash at 4.2.53,5.11-0.151006:20141006T151920Z is already 
installed
       Reject: 
pkg://omnios/shell/bash at 4.2.48,5.11-0.151006:20140924T201441Z
 
pkg://omnios/shell/bash at 4.2.49,5.11-0.151006:20140926T220552Z
 
pkg://omnios/shell/bash at 4.2.49,5.11-0.151006:20140929T154717Z
 
pkg://omnios/shell/bash at 4.2.50,5.11-0.151006:20140929T161532Z
 
pkg://omnios/shell/bash at 4.2.51,5.11-0.151006:20141001T182748Z
 
pkg://omnios/shell/bash at 4.2.52,5.11-0.151006:20141003T191548Z
       Reason:  Excluded by proposed incorporation 
'incorporation/jeos/omnios-userland'
                Newer version 
pkg://omnios/shell/bash at 4.2.53,5.11-0.151006:20141006T151920Z is already 
installed
       Reject: 
pkg://omnios/shell/bash at 4.2.53,5.11-0.151006:20141006T151920Z
       Reason:  All versions matching 'require' dependency 
pkg:/system/library are rejected
         Reject: 
pkg://omnios/system/library at 0.5.11,5.11-0.151002:20120401T180219Z
 
pkg://omnios/system/library at 0.5.11,5.11-0.151002:20120418T225425Z
 
pkg://omnios/system/library at 0.5.11,5.11-0.151004:20121011T224353Z
 
pkg://omnios/system/library at 0.5.11,5.11-0.151006:20130506T161124Z
         Reason:  Excluded by proposed incorporation 
'consolidation/osnet/osnet-incorporation'
                  Excluded by proposed incorporation 
'incorporation/jeos/illumos-gate'
                  Newer version 
pkg://omnios/system/library at 0.5.11,5.11-0.151006:20130516T142123Z is 
already installed
         Reject: 
pkg://omnios/system/library at 0.5.11,5.11-0.151006:20130516T142123Z
         Reason:  All versions matching 'require' dependency 
pkg:/SUNWcs at 0.5.11,5.11-0.151006 are rejected
           Reject: 
pkg://omnios/SUNWcs at 0.5.11,5.11-0.151006:20130506T161045Z
           Reason:  Excluded by proposed incorporation 
'consolidation/osnet/osnet-incorporation'
                    Excluded by proposed incorporation 
'incorporation/jeos/illumos-gate'
                    Newer version 
pkg://omnios/SUNWcs at 0.5.11,5.11-0.151006:20140622T160108Z is already 
installed
           Reject: 
pkg://omnios/SUNWcs at 0.5.11,5.11-0.151006:20140622T160108Z
           Reason:  Excluded by proposed incorporation 
'consolidation/osnet/osnet-incorporation'
                    Excluded by proposed incorporation 
'incorporation/jeos/illumos-gate'
           Reject: 
pkg://omnios/SUNWcs at 0.5.11,5.11-0.151008:20131204T022241Z
           Reason:  All versions matching 'require' dependency 
pkg:/install/beadm at 0.5.11,5.11-0.151008 are rejected
             Reject: 
pkg://omnios/install/beadm at 0.5.11,5.11-0.151008:20131204T024150Z
             Reason:  All versions matching 'require' dependency 
pkg:/runtime/python-26 at 2.6.8,5.11-0.151008 are rejected
               Reject: 
pkg://omnios/runtime/python-26 at 2.6.8,5.11-0.151008:20131204T024531Z
               Reason:  All versions matching 'require' dependency 
pkg:/library/security/openssl are rejected
                 Reject: 
pkg://omnios/library/security/openssl at 1.0.1,5.11-0.151002:20120401T175247Z
 
pkg://omnios/library/security/openssl at 1.0.1.1,5.11-0.151002:20120419T140207Z
 
pkg://omnios/library/security/openssl at 1.0.1.3,5.11-0.151002:20120510T203327Z
 
pkg://omnios/library/security/openssl at 1.0.1.3,5.11-0.151004:20121014T200457Z
 
pkg://omnios/library/security/openssl at 1.0.1.4,5.11-0.151004:20130208T215759Z
 
pkg://omnios/library/security/openssl at 1.0.1.4,5.11-0.151004:20130208T230413Z
 
pkg://omnios/library/security/openssl at 1.0.1.5,5.11-0.151006:20130506T185419Z
                 Reason:  Excluded by proposed incorporation 
'incorporation/jeos/omnios-userland'
                          Newer version 
pkg://omnios/library/security/openssl at 1.0.1.10,5.11-0.151006:20141015T154022Z 
is already installed
                 Reject: 
pkg://omnios/library/security/openssl at 1.0.1.5,5.11-0.151008:20131204T024253Z
                 Reason:  Newer version 
pkg://omnios/library/security/openssl at 1.0.1.10,5.11-0.151006:20141015T154022Z 
is already installed
                 Reject: 
pkg://omnios/library/security/openssl at 1.0.1.6,5.11-0.151006:20140110T154549Z
                 Reason:  Excluded by proposed incorporation 
'incorporation/jeos/omnios-userland'
                          Newer version 
pkg://omnios/library/security/openssl at 1.0.1.10,5.11-0.151006:20141015T154022Z 
is already installed
                 Reject: 
pkg://omnios/library/security/openssl at 1.0.1.6,5.11-0.151008:20140110T173804Z
                 Reason:  Newer version 
pkg://omnios/library/security/openssl at 1.0.1.10,5.11-0.151006:20141015T154022Z 
is already installed
                 Reject: 
pkg://omnios/library/security/openssl at 1.0.1.7,5.11-0.151006:20140407T211430Z
                 Reason:  Excluded by proposed incorporation 
'incorporation/jeos/omnios-userland'
                          Newer version 
pkg://omnios/library/security/openssl at 1.0.1.10,5.11-0.151006:20141015T154022Z 
is already installed
                 Reject: 
pkg://omnios/library/security/openssl at 1.0.1.7,5.11-0.151008:20140407T220403Z
 
pkg://omnios/library/security/openssl at 1.0.1.7,5.11-0.151008:20140408T142844Z
                 Reason:  Newer version 
pkg://omnios/library/security/openssl at 1.0.1.10,5.11-0.151006:20141015T154022Z 
is already installed
                 Reject: 
pkg://omnios/library/security/openssl at 1.0.1.8,5.11-0.151006:20140605T131009Z
                 Reason:  Excluded by proposed incorporation 
'incorporation/jeos/omnios-userland'
                          Newer version 
pkg://omnios/library/security/openssl at 1.0.1.10,5.11-0.151006:20141015T154022Z 
is already installed
                 Reject: 
pkg://omnios/library/security/openssl at 1.0.1.8,5.11-0.151008:20140605T140630Z
                 Reason:  Newer version 
pkg://omnios/library/security/openssl at 1.0.1.10,5.11-0.151006:20141015T154022Z 
is already installed
                 Reject: 
pkg://omnios/library/security/openssl at 1.0.1.9,5.11-0.151006:20140807T034412Z
                 Reason:  Excluded by proposed incorporation 
'incorporation/jeos/omnios-userland'
                          Newer version 
pkg://omnios/library/security/openssl at 1.0.1.10,5.11-0.151006:20141015T154022Z 
is already installed
                 Reject: 
pkg://omnios/library/security/openssl at 1.0.1.9,5.11-0.151008:20140807T035111Z
                 Reason:  Newer version 
pkg://omnios/library/security/openssl at 1.0.1.10,5.11-0.151006:20141015T154022Z 
is already installed
                 Reject: 
pkg://omnios/library/security/openssl at 1.0.1.10,5.11-0.151006:20141015T154022Z
                 Reason:  Excluded by proposed incorporation 
'incorporation/jeos/omnios-userland'
                  Excluded by proposed incorporation 
'consolidation/osnet/osnet-incorporation'
                  Excluded by proposed incorporation 
'incorporation/jeos/illumos-gate'
         Reject: 
pkg://omnios/system/library at 0.5.11,5.11-0.151008:20131204T024825Z
         Reason:  All versions matching 'require' dependency 
pkg:/SUNWcs at 0.5.11,5.11-0.151008 are rejected
                Excluded by proposed incorporation 
'incorporation/jeos/omnios-userland'
   Reject:  pkg://omnios/entire at 11,5.11-0.151008:20131205T195242Z
   Reason:  All versions matching 'require' dependency 
pkg:/SUNWcs at 0.5.11,5.11-0.151008 are rejected
     Reject:  pkg://omnios/SUNWcs at 0.5.11,5.11-0.151008:20131204T022241Z
     Reason:  All versions matching 'require' dependency 
pkg:/install/beadm at 0.5.11,5.11-0.151008 are rejected
       Reject: 
pkg://omnios/install/beadm at 0.5.11,5.11-0.151008:20131204T024150Z
       Reason:  All versions matching 'require' dependency 
pkg:/runtime/python-26 at 2.6.8,5.11-0.151008 are rejected
         Reject: 
pkg://omnios/runtime/python-26 at 2.6.8,5.11-0.151008:20131204T024531Z
         Reason:  All versions matching 'require' dependency 
pkg:/library/security/openssl are rejected
           Reject: 
pkg://omnios/library/security/openssl at 1.0.1,5.11-0.151002:20120401T175247Z
 
pkg://omnios/library/security/openssl at 1.0.1.1,5.11-0.151002:20120419T140207Z
 
pkg://omnios/library/security/openssl at 1.0.1.3,5.11-0.151002:20120510T203327Z
 
pkg://omnios/library/security/openssl at 1.0.1.3,5.11-0.151004:20121014T200457Z
 
pkg://omnios/library/security/openssl at 1.0.1.4,5.11-0.151004:20130208T215759Z
 
pkg://omnios/library/security/openssl at 1.0.1.4,5.11-0.151004:20130208T230413Z
 
pkg://omnios/library/security/openssl at 1.0.1.5,5.11-0.151006:20130506T185419Z
           Reason:  Excluded by proposed incorporation 
'incorporation/jeos/omnios-userland'
                    Newer version 
pkg://omnios/library/security/openssl at 1.0.1.10,5.11-0.151006:20141015T154022Z 
is already installed
           Reject: 
pkg://omnios/library/security/openssl at 1.0.1.5,5.11-0.151008:20131204T024253Z
           Reason:  Newer version 
pkg://omnios/library/security/openssl at 1.0.1.10,5.11-0.151006:20141015T154022Z 
is already installed
           Reject: 
pkg://omnios/library/security/openssl at 1.0.1.6,5.11-0.151006:20140110T154549Z
           Reason:  Excluded by proposed incorporation 
'incorporation/jeos/omnios-userland'
                    Newer version 
pkg://omnios/library/security/openssl at 1.0.1.10,5.11-0.151006:20141015T154022Z 
is already installed
           Reject: 
pkg://omnios/library/security/openssl at 1.0.1.6,5.11-0.151008:20140110T173804Z
           Reason:  Newer version 
pkg://omnios/library/security/openssl at 1.0.1.10,5.11-0.151006:20141015T154022Z 
is already installed
           Reject: 
pkg://omnios/library/security/openssl at 1.0.1.7,5.11-0.151006:20140407T211430Z
           Reason:  Excluded by proposed incorporation 
'incorporation/jeos/omnios-userland'
                    Newer version 
pkg://omnios/library/security/openssl at 1.0.1.10,5.11-0.151006:20141015T154022Z 
is already installed
           Reject: 
pkg://omnios/library/security/openssl at 1.0.1.7,5.11-0.151008:20140407T220403Z
 
pkg://omnios/library/security/openssl at 1.0.1.7,5.11-0.151008:20140408T142844Z
           Reason:  Newer version 
pkg://omnios/library/security/openssl at 1.0.1.10,5.11-0.151006:20141015T154022Z 
is already installed
           Reject: 
pkg://omnios/library/security/openssl at 1.0.1.8,5.11-0.151006:20140605T131009Z
           Reason:  Excluded by proposed incorporation 
'incorporation/jeos/omnios-userland'
                    Newer version 
pkg://omnios/library/security/openssl at 1.0.1.10,5.11-0.151006:20141015T154022Z 
is already installed
           Reject: 
pkg://omnios/library/security/openssl at 1.0.1.8,5.11-0.151008:20140605T140630Z
           Reason:  Newer version 
pkg://omnios/library/security/openssl at 1.0.1.10,5.11-0.151006:20141015T154022Z 
is already installed
           Reject: 
pkg://omnios/library/security/openssl at 1.0.1.9,5.11-0.151006:20140807T034412Z
           Reason:  Excluded by proposed incorporation 
'incorporation/jeos/omnios-userland'
                    Newer version 
pkg://omnios/library/security/openssl at 1.0.1.10,5.11-0.151006:20141015T154022Z 
is already installed
           Reject: 
pkg://omnios/library/security/openssl at 1.0.1.9,5.11-0.151008:20140807T035111Z
           Reason:  Newer version 
pkg://omnios/library/security/openssl at 1.0.1.10,5.11-0.151006:20141015T154022Z 
is already installed
           Reject: 
pkg://omnios/library/security/openssl at 1.0.1.10,5.11-0.151006:20141015T154022Z
           Reason:  Excluded by proposed incorporation 
'incorporation/jeos/omnios-userland'


From alexmcwhirter at mojovapes.net  Mon Dec  1 15:28:40 2014
From: alexmcwhirter at mojovapes.net (Alex McWhirter)
Date: Mon, 1 Dec 2014 10:28:40 -0500
Subject: [OmniOS-discuss] [PATCH] isc-dhcp to work with VNIC's via DLPI
Message-ID: <14F85ED3-C0B7-42DE-BAD3-8C0931E8D429@mojovapes.net>

As of now, the network/service/isc-dhcp package uses UDP sockets. The build.sh file is using the options ??enable-use-sockets? and ??enable-ipv4-pktinfo?. More details on this workaround can be found here. https://kb.isc.org/article/AA-01040/0/Building-ISC-DHCP-on-Solaris-11.html <https://kb.isc.org/article/AA-01040/0/Building-ISC-DHCP-on-Solaris-11.html>


The problem with using UDP sockets is that unicast clients cannot correctly obtain an IP without a relay, which really isn?t ideal. I?ve sent this patch to the isc-dhcp team, but i figured it might be found more quickly useful here.

I?ve created two patches for isc-dhcp that allow dhcpd to use DLPI with VNIC?s and other vanity named interfaces. the modified files are ?./configure.ac? and ?./common/dlpi.c? Below is the diff info.





--- /Users/alexmcwhirter/Downloads/dhcp-4.3.1 2/common/dlpi.c	2014-07-29 18:41:04.000000000 -0400
+++ /Users/alexmcwhirter/Desktop/dhcp-4.3.1/common/dlpi.c	2014-11-30 20:48:39.000000000 -0500
@@ -130,7 +130,14 @@
 
 #define DLPI_MAXDLBUF		8192	/* Buffer size */
 #define DLPI_MAXDLADDR		1024	/* Max address size */
-#define DLPI_DEVDIR		"/dev/"	/* Device directory */
+
+/* Solaris 11 / Illumos 11 uses vanity names for ethernet devices
+ * and places them in /dev/net */
+#ifdef __SOLARIS_2_11
+# define DLPI_DEVDIR		"/dev/net/"	/* Device directory */
+#else
+# define DLPI_DEVDIR		"/dev/"	/* Device directory */
+#endif
 
 static int dlpiopen(const char *ifname);
 static int dlpiunit (char *ifname);
@@ -794,9 +801,13 @@
 	ep = cp = ifname;
 	while (*ep)
 		ep++;
-	/* And back up to the first digit (unit number) */
-	while ((*(ep - 1) >= '0' && *(ep - 1) <= '9') || *(ep - 1) == ':')
-		ep--;
+    /* Solaris 11 / Illumos 11 should not have the number removed from the 
+     * interface name */
+    if (__SOLARIS_2_11 != 1) {
+        /* And back up to the first digit (unit number) */
+        while ((*(ep - 1) >= '0' && *(ep - 1) <= '9') || *(ep - 1) == ':')
+            ep--;
+    }
 	
 	/* Copy everything up to the unit number */
 	while (cp < ep) {




--- /Users/alexmcwhirter/Downloads/dhcp-4.3.1 2/configure.ac	2014-08-06 18:35:33.000000000 -0400
+++ /Users/alexmcwhirter/Desktop/dhcp-4.3.1/configure.ac	2014-11-30 20:50:11.000000000 -0500
@@ -13,6 +13,13 @@
 
 AC_CANONICAL_HOST
 
+# Solaris 11 / Illumos 11 is identified as solaris2.11
+AM_CONDITIONAL([SOLARIS_2_11], [test x$host_os = xsolaris2.11])
+
+# If host_os is solaris2.11 then define __SOLARIS_2_11
+AM_COND_IF([SOLARIS_2_11],[AC_DEFINE([__SOLARIS_2_11], [1],
+    [Define if host_os is solaris2.11])]) 
+
 # We want to turn on warnings if we are using gcc and the user did 
 # not specify CFLAGS. The autoconf check for the C compiler sets the
 # CFLAGS if gcc is used, so we will save it before we run that check.



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141201/3b9a4153/attachment.html>

From danmcd at omniti.com  Mon Dec  1 15:57:31 2014
From: danmcd at omniti.com (Dan McDonald)
Date: Mon, 1 Dec 2014 10:57:31 -0500
Subject: [OmniOS-discuss] Release 12 - KVM poor network performance
In-Reply-To: <BAY167-W576E805BCA6E92E076512EAE7F0@phx.gbl>
References: <BAY167-W576E805BCA6E92E076512EAE7F0@phx.gbl>
Message-ID: <5B32DB03-16B1-4DB0-B0A6-81221203AEDF@omniti.com>

One other question, which I should've asked earlier:  You're using OmniOS as the KVM host, and your performance drops are on Windows guests, right?

Thanks,
Dan


From danmcd at omniti.com  Mon Dec  1 16:12:44 2014
From: danmcd at omniti.com (Dan McDonald)
Date: Mon, 1 Dec 2014 11:12:44 -0500
Subject: [OmniOS-discuss] r151006 -> r151012 upgrade problems after
	bash/openssl patches ...
In-Reply-To: <547BC54C.6060805@mcintyreweb.com>
References: <547BC54C.6060805@mcintyreweb.com>
Message-ID: <7AA0387B-1F68-4464-A21C-7E860FA2FB5C@omniti.com>

You will likely have better luck directly upgrading to 012.

You can try using a spare boot environment (BE) to make it less damaging going forward.

	beadm create 012-be -e 006-be

	beadm mount 012-be /mnt

	pkg -R /mnt set-publisher -G http://pkg.omniti.com/omnios/stable -g http://pkg.omniti.com/omnios/r151012 omnios

	pkg -R /mnt update

	beadm umount /mnt

	beadm activate 012-be

We didn't update 008 with the heartbleed fixes because 008 was discontinued.

Hope this helps,
Dan


From danmcd at omniti.com  Mon Dec  1 16:19:50 2014
From: danmcd at omniti.com (Dan McDonald)
Date: Mon, 1 Dec 2014 11:19:50 -0500
Subject: [OmniOS-discuss] [PATCH] isc-dhcp to work with VNIC's via DLPI
In-Reply-To: <14F85ED3-C0B7-42DE-BAD3-8C0931E8D429@mojovapes.net>
References: <14F85ED3-C0B7-42DE-BAD3-8C0931E8D429@mojovapes.net>
Message-ID: <BD1E8409-5A12-41AC-97E8-78C240F7BF11@omniti.com>


> On Dec 1, 2014, at 10:28 AM, Alex McWhirter <alexmcwhirter at mojovapes.net> wrote:
> 
> As of now, the network/service/isc-dhcp package uses UDP sockets. The build.sh file is using the options ??enable-use-sockets? and ??enable-ipv4-pktinfo?. More details on this workaround can be found here. https://kb.isc.org/article/AA-01040/0/Building-ISC-DHCP-on-Solaris-11.html

Got it.

> The problem with using UDP sockets is that unicast clients cannot correctly obtain an IP without a relay, which really isn?t ideal. I?ve sent this patch to the isc-dhcp team, but i figured it might be found more quickly useful here.
> 
> I?ve created two patches for isc-dhcp that allow dhcpd to use DLPI with VNIC?s and other vanity named interfaces. the modified files are ?./configure.ac? and ?./common/dlpi.c? Below is the diff info.

Thank you!

BTW, it's just "illumos" not "illumos 11", but that's a tiny nit.

My question for you is this:  Are you suggesting OmniOS build ISC DHCP using the DLPI method/configure-flags instead, provided your fixes are in place?  If so, I have a followup question:  What testing have you performed?  I'm not saying no, but I just want to know what all you've tested alongside these fixes.

Thanks, I really appreciate the contribution!
Dan


From henson at acm.org  Mon Dec  1 22:36:03 2014
From: henson at acm.org (Paul B. Henson)
Date: Mon, 1 Dec 2014 14:36:03 -0800
Subject: [OmniOS-discuss] common-factor key exchange
In-Reply-To: <20141201055654.2a3e23f8@punda-mlia>
References: <20141128191703.00a9df99@punda-mlia>	<20141130025557.GI29549@bender.unx.csupomona.edu>
	<20141201055654.2a3e23f8@punda-mlia>
Message-ID: <234301d00db7$32bdcc20$98396460$@acm.org>

> From: Michael Mounteney
> Sent: Sunday, November 30, 2014 11:57 AM
>
> Port {something other than 22}
> PasswordAuthentication no
> PermitEmptyPasswords no
> UsePAM no
> PrintMotd no
> PrintLastLog no
> UsePrivilegeSeparation sandbox          # Default for new installations.
> ClientAliveInterval 61
> Subsystem       sftp    /usr/lib/misc/sftp-server
> AcceptEnv LANG LC_*

Hmm, I restarted my Gentoo sshd with that exact config (other than port=22),
and had no problems connecting to it from omnios, using
diffie-hellman-group-exchange-sha1 for the key exchange. I'm running
net-misc/openssh-6.6_p1-r1...

I've got no idea why yours is behaving differently. What use flags do you
have set? Mine are:

[ebuild   R    ] net-misc/openssh-6.6_p1-r1  USE="X hpn kerberos pam -X509
-bindist -ldap -ldns -libedit (-selinux) -skey -static -tcpd" 0 kB




From gate03 at landcroft.co.uk  Mon Dec  1 23:41:49 2014
From: gate03 at landcroft.co.uk (Michael Mounteney)
Date: Tue, 2 Dec 2014 09:41:49 +1000
Subject: [OmniOS-discuss] common-factor key exchange
In-Reply-To: <234301d00db7$32bdcc20$98396460$@acm.org>
References: <20141128191703.00a9df99@punda-mlia>
	<20141130025557.GI29549@bender.unx.csupomona.edu>
	<20141201055654.2a3e23f8@punda-mlia>
	<234301d00db7$32bdcc20$98396460$@acm.org>
Message-ID: <20141202094149.43d9f5d9@punda-mlia>

On Mon, 1 Dec 2014 14:36:03 -0800
"Paul B. Henson" <henson at acm.org> wrote:

> I've got no idea why yours is behaving differently. What use flags do
> you have set? Mine are:
> 
> [ebuild   R    ] net-misc/openssh-6.6_p1-r1  USE="X hpn kerberos pam
> -X509 -bindist -ldap -ldns -libedit (-selinux) -skey -static -tcpd" 0
> kB

My use flags are

equery u openssh
[ Legend : U - final flag setting for installation]
[        : I - package is installed with flag     ]
[ Colors : set, unset                             ]
 * Found these USE flags for net-misc/openssh-6.7_p1-r3:
 U I
 + + X        : Add support for X11
 - - X509     : Adds support for X.509 certificate authentication
 - - bindist  : Disable EC/RC5 algorithms in OpenSSL for patent reasons.
 + + hpn      : Enable high performance ssh
 - - kerberos : Add kerberos support
 + + ldap     : Add support for storing SSH public keys in LDAP
 - - ldns     : Use LDNS for DNSSEC/SSHFP validation.
 - - libedit  : Use the libedit library (replacement for readline)
 + + pam      : Add support for PAM (Pluggable Authentication Modules) - DANGEROUS to arbitrarily flip
 + + pie      : Build programs as Position Independent Executables (a security hardening technique)
 - - sctp     : Support for Stream Control Transmission Protocol
 - - skey     : Enable S/Key (Single use password) authentication support
 - - static   : !!do not set this during bootstrap!! Causes binaries to be statically linked instead of dynamically

but the reason for the problem is that the older algorithms have been removed from openssh-6.7.  I just downgraded to 6.6 on one machine and once again was able to ssh in from OmniOS.  Upgrade to 6.7 again and the common kex problem re-arose.

Michael.

From gearboxes at outlook.com  Tue Dec  2 02:54:28 2014
From: gearboxes at outlook.com (Machine Man)
Date: Mon, 1 Dec 2014 21:54:28 -0500
Subject: [OmniOS-discuss] Release 12 - KVM poor network performance
In-Reply-To: <5B32DB03-16B1-4DB0-B0A6-81221203AEDF@omniti.com>
References: <BAY167-W576E805BCA6E92E076512EAE7F0@phx.gbl>,
	<5B32DB03-16B1-4DB0-B0A6-81221203AEDF@omniti.com>
Message-ID: <BAY167-W38DE39333963E0C7CA5AAAE7A0@phx.gbl>

Correct. The copy would also stall quickly and pool utilization would be at 10%.
 
I have not tested NFS for example, but I am sure that would have been discussed if that was happening. I did have to do zfs send/recv between two nodes on the same subnet and did not notice any performance issue with one node was on r12. 
Both currently are back to running r10. 
I will build a test system with a fresh install of the latest version and see when time permits.
 
Thanks,
 
> Subject: Re: [OmniOS-discuss] Release 12 - KVM poor network performance
> From: danmcd at omniti.com
> Date: Mon, 1 Dec 2014 10:57:31 -0500
> CC: omnios-discuss at lists.omniti.com
> To: gearboxes at outlook.com
> 
> One other question, which I should've asked earlier:  You're using OmniOS as the KVM host, and your performance drops are on Windows guests, right?
> 
> Thanks,
> Dan
> 
 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141201/6d50978f/attachment.html>

From lists at mcintyreweb.com  Tue Dec  2 07:38:52 2014
From: lists at mcintyreweb.com (Hugh McIntyre)
Date: Mon, 01 Dec 2014 23:38:52 -0800
Subject: [OmniOS-discuss] r151006 -> r151012 upgrade problems after
 bash/openssl patches ...
In-Reply-To: <7AA0387B-1F68-4464-A21C-7E860FA2FB5C@omniti.com>
References: <547BC54C.6060805@mcintyreweb.com>
	<7AA0387B-1F68-4464-A21C-7E860FA2FB5C@omniti.com>
Message-ID: <547D6C8C.4020307@mcintyreweb.com>

Thanks Dan!

This seems to have worked very painlessly, modulo needing to use "-G 
http://pkg.omniti.com/omnios/release" instead of "-G 
http://pkg.omniti.com/omnios/stable".

Hugh.


On 12/1/14 8:12 AM, Dan McDonald wrote:
> You will likely have better luck directly upgrading to 012.
>
> You can try using a spare boot environment (BE) to make it less damaging going forward.
>
> 	beadm create 012-be -e 006-be
>
> 	beadm mount 012-be /mnt
>
> 	pkg -R /mnt set-publisher -G http://pkg.omniti.com/omnios/stable -g http://pkg.omniti.com/omnios/r151012 omnios
>
> 	pkg -R /mnt update
>
> 	beadm umount /mnt
>
> 	beadm activate 012-be
>
> We didn't update 008 with the heartbleed fixes because 008 was discontinued.
>
> Hope this helps,
> Dan
>

From piiv at zhaw.ch  Tue Dec  2 11:05:10 2014
From: piiv at zhaw.ch (Vincenzo Pii)
Date: Tue, 2 Dec 2014 12:05:10 +0100
Subject: [OmniOS-discuss]  OmniOS Packaging: override installation PREFIX
Message-ID: <CALOiZS06ENCCEa9vvAZBxz7O4ShjLB0TFPCynygi5yQ8mw8rzQ@mail.gmail.com>

I am trying to package something on OmniOS following the instructions here:

http://omnios.omniti.com/wiki.php/PackagingForOmniOS.

The PREFIX in the config.sh file is set to /usr/local:

    # Default prefix for packages (may be overridden)
    PREFIX=/usr/local

In my build.sh I want to set a different prefix (for this specific package
only), so I am doing something like

    export PREFIX=/opt
    export CONFIGURE_OPTS="--prefix=$PREFIX"

However, this doesn't seem to work, as files are still installed under
/usr/local.

What is the correct way to override the global PREFIX?

Thanks,
Vincenzo.

-- 
Vincenzo Pii
Researcher, InIT Cloud Computing Lab
Zurich University of Applied Sciences (ZHAW)
blog.zhaw.ch/icclab
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141202/a35bc57b/attachment.html>

From lotheac at iki.fi  Tue Dec  2 11:30:25 2014
From: lotheac at iki.fi (Lauri Tirkkonen)
Date: Tue, 2 Dec 2014 13:30:25 +0200
Subject: [OmniOS-discuss] OmniOS Packaging: override installation PREFIX
In-Reply-To: <CALOiZS06ENCCEa9vvAZBxz7O4ShjLB0TFPCynygi5yQ8mw8rzQ@mail.gmail.com>
References: <CALOiZS06ENCCEa9vvAZBxz7O4ShjLB0TFPCynygi5yQ8mw8rzQ@mail.gmail.com>
Message-ID: <20141202113025.GB11049@gutsman.lotheac.fi>

On Tue, Dec 02 2014 12:05:10 +0100, Vincenzo Pii wrote:
> In my build.sh I want to set a different prefix (for this specific package
> only), so I am doing something like
> 
>     export PREFIX=/opt
>     export CONFIGURE_OPTS="--prefix=$PREFIX"
> 
> However, this doesn't seem to work, as files are still installed under
> /usr/local.
> 
> What is the correct way to override the global PREFIX?

Try this:

    PREFIX=/opt
    reset_configure_opts

-- 
Lauri Tirkkonen | +358 50 5341376 | lotheac @ IRCnet

From piiv at zhaw.ch  Tue Dec  2 12:39:38 2014
From: piiv at zhaw.ch (Vincenzo Pii)
Date: Tue, 2 Dec 2014 13:39:38 +0100
Subject: [OmniOS-discuss] OmniOS Packaging: override installation PREFIX
In-Reply-To: <03c26df9fdba46c18c505889f8ca095c@SRV-MAIL-001.zhaw.ch>
References: <CALOiZS06ENCCEa9vvAZBxz7O4ShjLB0TFPCynygi5yQ8mw8rzQ@mail.gmail.com>
	<03c26df9fdba46c18c505889f8ca095c@SRV-MAIL-001.zhaw.ch>
Message-ID: <CALOiZS1iJP=Kyz=HHOSfs3m5NnUUOSytPOK5g6fRj0-h3KB4Ew@mail.gmail.com>

Many thanks Lauri, that worked (no need for the CONFIGURE_OPTS)!

2014-12-02 12:30 GMT+01:00 Lauri Tirkkonen <lotheac at iki.fi>:

> On Tue, Dec 02 2014 12:05:10 +0100, Vincenzo Pii wrote:
> > In my build.sh I want to set a different prefix (for this specific
> package
> > only), so I am doing something like
> >
> >     export PREFIX=/opt
> >     export CONFIGURE_OPTS="--prefix=$PREFIX"
> >
> > However, this doesn't seem to work, as files are still installed under
> > /usr/local.
> >
> > What is the correct way to override the global PREFIX?
>
> Try this:
>
>     PREFIX=/opt
>     reset_configure_opts
>
> --
> Lauri Tirkkonen | +358 50 5341376 | lotheac @ IRCnet
>



-- 
Vincenzo Pii
Researcher, InIT Cloud Computing Lab
Zurich University of Applied Sciences (ZHAW)
blog.zhaw.ch/icclab
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141202/c44dd346/attachment.html>

From filip.marvan at aira.cz  Tue Dec  2 14:31:55 2014
From: filip.marvan at aira.cz (Filip Marvan)
Date: Tue, 2 Dec 2014 15:31:55 +0100
Subject: [OmniOS-discuss] zpool import strange informations
Message-ID: <3BE0DEED8863E5429BAE4CAEDF624565039C1B330761@AIRA-SRV.aira.local>

Hi,

 

I have one storage server on OmniOS 12 with one data pool:

 

  pool: raid_10

state: ONLINE

  scan: none requested

config:

 

        NAME        STATE     READ WRITE CKSUM

        raid_10     ONLINE       0     0     0

          mirror-0  ONLINE       0     0     0

            c3t2d0  ONLINE       0     0     0

            c3t3d0  ONLINE       0     0     0

          mirror-1  ONLINE       0     0     0

            c3t4d0  ONLINE       0     0     0

            c3t5d0  ONLINE       0     0     0

        logs

          c3t0d0    ONLINE       0     0     0

 

errors: No known data errors

 

  pool: rpool

state: ONLINE

  scan: none requested

config:

 

        NAME        STATE     READ WRITE CKSUM

        rpool       ONLINE       0     0     0

          c3t1d0s0  ONLINE       0     0     0

 

errors: No known data errors

 

There are no other disks in this storage server. Bud if I write command
zpool import, I can see:

  pool: tank

     id: 4916680739993977334

  state: UNAVAIL

status: The pool was last accessed by another system.

action: The pool cannot be imported due to damaged devices or data.

   see: http://illumos.org/msg/ZFS-8000-EY

config:

 

        tank                     UNAVAIL  insufficient replicas

          raidz2-1               UNAVAIL  insufficient replicas

            /dev/label/mfront8   UNAVAIL  cannot open

            /dev/label/mfront9   UNAVAIL  cannot open

            /dev/label/mfront10  UNAVAIL  cannot open

            c3t2d0p0             ONLINE

            /dev/label/mfront12  UNAVAIL  cannot open

            /dev/label/mfront13  UNAVAIL  cannot open

            /dev/label/mfront14  UNAVAIL  cannot open

            c3t5d0               ONLINE

 

I never created such "tank" pool. Where are stored informations about this
(un)importable pool, how can I delete them? Why c3t5d0 seems to be as part
of two different pools at the same time?

 

Thank you for any help.

Filip Marvan

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141202/47846175/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 6247 bytes
Desc: not available
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141202/47846175/attachment-0001.bin>

From cal-s at blue-bolt.com  Tue Dec  2 14:47:50 2014
From: cal-s at blue-bolt.com (Cal Sawyer)
Date: Tue, 02 Dec 2014 14:47:50 +0000
Subject: [OmniOS-discuss] Upgrading from OI to OmniOS
Message-ID: <547DD116.7040401@blue-bolt.com>

Hi

I have a server hosting 65TB of RAIDZ2 storage that's currently running 
OI (oi_151a8), which i really want to get off of given that OI has 
bitten the dust support-wise.  I found one brief discussion on the topic 
but it wasn't terribly detailed and was missing critical steps like 
configuration protection/retention. I'd like to preserve network and esp 
LDAP (client) settings as they were kind of a pain to establish in the 
first instance for quick reapplication. If anyone has or is interested 
in cobbling together a checklist of steps to upgrade OI to OmniOS that 
takes configuration into account, i'd very much appreciate seeing it.

thank you

- cal sawyer

From zmalone at omniti.com  Tue Dec  2 14:57:59 2014
From: zmalone at omniti.com (Zach Malone)
Date: Tue, 2 Dec 2014 09:57:59 -0500
Subject: [OmniOS-discuss] Postgres on Stable
In-Reply-To: <CALLZ=vV6u2H1WC6D+a_a9qiUSpOs0jBxwq=YjvRVnCDVowNLdA@mail.gmail.com>
References: <20141128191821.43382469@punda-mlia>
	<CACLsApuL0vfHhn7FAQw8OU+uPd=j=qEGRUVWAtCtA1VXoo+TVA@mail.gmail.com>
	<CA+FJ0zsFukbMSdnTRxqdO1fU_zR69_=bH4UDJkh3mdB+pmtABQ@mail.gmail.com>
	<CALLZ=vXs6XDheAGuE2aDJigNaHti4bVe7npAc2qiiqK45zh0sQ@mail.gmail.com>
	<CALLZ=vXQ=4qn=4SWAtAnBksf=dSVXkzFRrfWffe3HyoLB9b+1g@mail.gmail.com>
	<20141129071835.1daa0799@punda-mlia>
	<CALLZ=vU4MNQ2cZQgTHguzq4Ok-UywT7HTr5c6Pm05g4QyR8=7Q@mail.gmail.com>
	<CALLZ=vV6u2H1WC6D+a_a9qiUSpOs0jBxwq=YjvRVnCDVowNLdA@mail.gmail.com>
Message-ID: <CALLZ=vXMbwm=jO6EhHCkMZGTbk55f_YVm9-jc+Gjce4ppoA8ag@mail.gmail.com>

A vagrant box for r151012 is now listed on the download page.
--Zach

On Fri, Nov 28, 2014 at 6:37 PM, Zach Malone <zmalone at omniti.com> wrote:
>>>   No suitable version of installed package pkg://omnios/entire at 11,5.11-0.151012:20141027T191658Z found
>>>     Reject:  pkg://omnios/entire at 11,5.11-0.151012:20141027T191658Z
>>>     Reason:  Excluded by proposed incorporation 'omniti/database/postgresql-934'
>>>              This version is excluded by installed incorporation pkg://omnios/entire at 11,5.11-0.151012:20141027T191658Z
>>
>> I hadn't realized we incorporated against the OS for postgres, I'll
>> spin a new postgresql-934 or 935 on a 012 machine when I get a chance.
>> It looks like 012 doesn't have a Vagrant box published yet either,
>> which will make this take a little longer.
>
> I built postgresql-935 for you on a r151012 system, and published it
> to the omniti-ms repo.  Want to give it a shot?  Past installing it, I
> have not tried it on a production system (I'm planning on moving some
> systems to r151012 this month), and I have not rolled any of the other
> postgres libraries or modules that we publish for 9.2.
>
> I also noticed that we're missing a Vagrant box for r151012, I'll see
> if I can get that up next week.
> --Zach

From a.angelov at gmail.com  Tue Dec  2 14:57:44 2014
From: a.angelov at gmail.com (Angel Angelov)
Date: Tue, 2 Dec 2014 16:57:44 +0200
Subject: [OmniOS-discuss] PCIe dedicated device for ZIL
Message-ID: <CAO-7L5Fzico5MsnYET9ODndQMerJJNfq=jmfzWTrxorrZ_DzEA@mail.gmail.com>

Hi guys,

does anyone played with some of these PCIe devices on the market as a
cheaper ZIL dedicated device instead of this one
<http://www.hgst.com/solid-state-storage/enterprise-ssd/sas-ssd/zeusram-sas-ssd>
for example that would work with OmniOS out of the box?

Here's one of the products I have found in the Internet googling around but
it's way too big for the purpose:
http://www.kingspec.com/en/product_xx187.html

I am trying to find a good price/performance balance when full SSD storage
based pool is in use.

Thanks in advance.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141202/31a774ec/attachment.html>

From zmalone at omniti.com  Tue Dec  2 15:12:16 2014
From: zmalone at omniti.com (Zach Malone)
Date: Tue, 2 Dec 2014 10:12:16 -0500
Subject: [OmniOS-discuss] zpool import strange informations
In-Reply-To: <3BE0DEED8863E5429BAE4CAEDF624565039C1B330761@AIRA-SRV.aira.local>
References: <3BE0DEED8863E5429BAE4CAEDF624565039C1B330761@AIRA-SRV.aira.local>
Message-ID: <CALLZ=vV4D6yDai9mdnq7Z6odinxFARSSCTtz_nHyEyD4Jfz4=A@mail.gmail.com>

If I was to guess,  c3t2d0p0 was once a part of another pool called
"tank", and when you do an import, it's attempting to reassemble the
rest of tank, and sees that a drive named c3t5d0 used to be in the
pool it knew as tank.

Tank is a pretty standard pool name, so I'm guessing the drive used to
be in another system, or the system itself used to be running a
different OS.
--Zach

On Tue, Dec 2, 2014 at 9:31 AM, Filip Marvan <filip.marvan at aira.cz> wrote:
> Hi,
>
>
>
> I have one storage server on OmniOS 12 with one data pool:
>
>
>
>   pool: raid_10
>
> state: ONLINE
>
>   scan: none requested
>
> config:
>
>
>
>         NAME        STATE     READ WRITE CKSUM
>
>         raid_10     ONLINE       0     0     0
>
>           mirror-0  ONLINE       0     0     0
>
>             c3t2d0  ONLINE       0     0     0
>
>             c3t3d0  ONLINE       0     0     0
>
>           mirror-1  ONLINE       0     0     0
>
>             c3t4d0  ONLINE       0     0     0
>
>             c3t5d0  ONLINE       0     0     0
>
>         logs
>
>           c3t0d0    ONLINE       0     0     0
>
>
>
> errors: No known data errors
>
>
>
>   pool: rpool
>
> state: ONLINE
>
>   scan: none requested
>
> config:
>
>
>
>         NAME        STATE     READ WRITE CKSUM
>
>         rpool       ONLINE       0     0     0
>
>           c3t1d0s0  ONLINE       0     0     0
>
>
>
> errors: No known data errors
>
>
>
> There are no other disks in this storage server. Bud if I write command
> zpool import, I can see:
>
>   pool: tank
>
>      id: 4916680739993977334
>
>   state: UNAVAIL
>
> status: The pool was last accessed by another system.
>
> action: The pool cannot be imported due to damaged devices or data.
>
>    see: http://illumos.org/msg/ZFS-8000-EY
>
> config:
>
>
>
>         tank                     UNAVAIL  insufficient replicas
>
>           raidz2-1               UNAVAIL  insufficient replicas
>
>             /dev/label/mfront8   UNAVAIL  cannot open
>
>             /dev/label/mfront9   UNAVAIL  cannot open
>
>             /dev/label/mfront10  UNAVAIL  cannot open
>
>             c3t2d0p0             ONLINE
>
>             /dev/label/mfront12  UNAVAIL  cannot open
>
>             /dev/label/mfront13  UNAVAIL  cannot open
>
>             /dev/label/mfront14  UNAVAIL  cannot open
>
>             c3t5d0               ONLINE
>
>
>
> I never created such "tank" pool. Where are stored informations about this
> (un)importable pool, how can I delete them? Why c3t5d0 seems to be as part
> of two different pools at the same time?
>
>
>
> Thank you for any help.
>
> Filip Marvan
>
>
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>

From danmcd at omniti.com  Tue Dec  2 15:19:08 2014
From: danmcd at omniti.com (Dan McDonald)
Date: Tue, 2 Dec 2014 10:19:08 -0500
Subject: [OmniOS-discuss] r151006 -> r151012 upgrade problems after
	bash/openssl patches ...
In-Reply-To: <547D6C8C.4020307@mcintyreweb.com>
References: <547BC54C.6060805@mcintyreweb.com>
	<7AA0387B-1F68-4464-A21C-7E860FA2FB5C@omniti.com>
	<547D6C8C.4020307@mcintyreweb.com>
Message-ID: <73038663-F1B9-4033-B800-D2643E8BB1B9@omniti.com>


> On Dec 2, 2014, at 2:38 AM, Hugh McIntyre <lists at mcintyreweb.com> wrote:
> 
> Thanks Dan!
> 
> This seems to have worked very painlessly, modulo needing to use "-G http://pkg.omniti.com/omnios/release" instead of "-G http://pkg.omniti.com/omnios/stable".

The other solution would've been for me to update 008's appropriate packages to stop pkg(5) from seizing up.  I'm glad the 006->012 path works, though, as the next release is also the next LTS, so we MUST be able to support that.

Thanks,
Dan


From filip.marvan at aira.cz  Tue Dec  2 15:20:25 2014
From: filip.marvan at aira.cz (Filip Marvan)
Date: Tue, 2 Dec 2014 16:20:25 +0100
Subject: [OmniOS-discuss] zpool import strange informations
In-Reply-To: <CALLZ=vV4D6yDai9mdnq7Z6odinxFARSSCTtz_nHyEyD4Jfz4=A@mail.gmail.com>
References: <3BE0DEED8863E5429BAE4CAEDF624565039C1B330761@AIRA-SRV.aira.local>
	<CALLZ=vV4D6yDai9mdnq7Z6odinxFARSSCTtz_nHyEyD4Jfz4=A@mail.gmail.com>
Message-ID: <3BE0DEED8863E5429BAE4CAEDF624565039C1B33076D@AIRA-SRV.aira.local>

Yes, you are probably right. But I'm thinking, why that information wasn't 
overwriten when I add whole c3t2d0 to a new mirror? Is it possible to use 
c3t2d0 in one pool, and c3t2d0p0 in another?

Thank you,
Filip

-----Original Message-----
From: Zach Malone [mailto:zmalone at omniti.com]
Sent: Tuesday, December 02, 2014 4:12 PM
To: Filip Marvan
Cc: omnios-discuss at lists.omniti.com
Subject: Re: [OmniOS-discuss] zpool import strange informations

If I was to guess,  c3t2d0p0 was once a part of another pool called "tank", 
and when you do an import, it's attempting to reassemble the rest of tank, and 
sees that a drive named c3t5d0 used to be in the pool it knew as tank.

Tank is a pretty standard pool name, so I'm guessing the drive used to be in 
another system, or the system itself used to be running a different OS.
--Zach

On Tue, Dec 2, 2014 at 9:31 AM, Filip Marvan <filip.marvan at aira.cz> wrote:
> Hi,
>
>
>
> I have one storage server on OmniOS 12 with one data pool:
>
>
>
>   pool: raid_10
>
> state: ONLINE
>
>   scan: none requested
>
> config:
>
>
>
>         NAME        STATE     READ WRITE CKSUM
>
>         raid_10     ONLINE       0     0     0
>
>           mirror-0  ONLINE       0     0     0
>
>             c3t2d0  ONLINE       0     0     0
>
>             c3t3d0  ONLINE       0     0     0
>
>           mirror-1  ONLINE       0     0     0
>
>             c3t4d0  ONLINE       0     0     0
>
>             c3t5d0  ONLINE       0     0     0
>
>         logs
>
>           c3t0d0    ONLINE       0     0     0
>
>
>
> errors: No known data errors
>
>
>
>   pool: rpool
>
> state: ONLINE
>
>   scan: none requested
>
> config:
>
>
>
>         NAME        STATE     READ WRITE CKSUM
>
>         rpool       ONLINE       0     0     0
>
>           c3t1d0s0  ONLINE       0     0     0
>
>
>
> errors: No known data errors
>
>
>
> There are no other disks in this storage server. Bud if I write
> command zpool import, I can see:
>
>   pool: tank
>
>      id: 4916680739993977334
>
>   state: UNAVAIL
>
> status: The pool was last accessed by another system.
>
> action: The pool cannot be imported due to damaged devices or data.
>
>    see: http://illumos.org/msg/ZFS-8000-EY
>
> config:
>
>
>
>         tank                     UNAVAIL  insufficient replicas
>
>           raidz2-1               UNAVAIL  insufficient replicas
>
>             /dev/label/mfront8   UNAVAIL  cannot open
>
>             /dev/label/mfront9   UNAVAIL  cannot open
>
>             /dev/label/mfront10  UNAVAIL  cannot open
>
>             c3t2d0p0             ONLINE
>
>             /dev/label/mfront12  UNAVAIL  cannot open
>
>             /dev/label/mfront13  UNAVAIL  cannot open
>
>             /dev/label/mfront14  UNAVAIL  cannot open
>
>             c3t5d0               ONLINE
>
>
>
> I never created such "tank" pool. Where are stored informations about
> this (un)importable pool, how can I delete them? Why c3t5d0 seems to
> be as part of two different pools at the same time?
>
>
>
> Thank you for any help.
>
> Filip Marvan
>
>
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 6247 bytes
Desc: not available
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141202/2035f27c/attachment-0001.bin>

From danmcd at omniti.com  Tue Dec  2 15:35:36 2014
From: danmcd at omniti.com (Dan McDonald)
Date: Tue, 2 Dec 2014 10:35:36 -0500
Subject: [OmniOS-discuss] Release 12 KVM poor network performance
In-Reply-To: <BAY167-W945359F73CCDEE97B19CBE837F0@phx.gbl>
References: <BAY167-W945359F73CCDEE97B19CBE837F0@phx.gbl>
Message-ID: <34BE292D-F302-4E1C-BCAA-6FD226AABA70@omniti.com>


> On Nov 29, 2014, at 3:42 PM, Machine Man <heinrich.vanriel at outlook.com> wrote:
> 
> Experience slow network performance after upgrading.
> Copying from another windows system (physical machine) it drops form the 85MB/s on R10 to between 9 and 14MB/s. Two systems experience the exact same symptom. Both make use of igb driver. 
> Copy tests are done between windows 8.1 and Windows 2012R2 VM.
> Changing to virtio for nic has no impact other than causing kvm to crash after copying ~400MB. 
>  
> Any ideas? 

We are behind the illumos-kvm-cmd distribution because it requires changes in illumos that Joyent hasn't pushed upstream just yet (the "vnd" project).  I'm hoping to address this for '014, but I have other 014-related things to do first.

Sorry,
Dan



From richard.elling at richardelling.com  Tue Dec  2 17:38:23 2014
From: richard.elling at richardelling.com (Richard Elling)
Date: Tue, 2 Dec 2014 09:38:23 -0800
Subject: [OmniOS-discuss] zpool import strange informations
In-Reply-To: <3BE0DEED8863E5429BAE4CAEDF624565039C1B33076D@AIRA-SRV.aira.local>
References: <3BE0DEED8863E5429BAE4CAEDF624565039C1B330761@AIRA-SRV.aira.local>
	<CALLZ=vV4D6yDai9mdnq7Z6odinxFARSSCTtz_nHyEyD4Jfz4=A@mail.gmail.com>
	<3BE0DEED8863E5429BAE4CAEDF624565039C1B33076D@AIRA-SRV.aira.local>
Message-ID: <21BA9319-381E-4303-8B49-93B7693FFCE7@richardelling.com>


On Dec 2, 2014, at 7:20 AM, Filip Marvan <filip.marvan at aira.cz> wrote:

> Yes, you are probably right. But I'm thinking, why that information wasn't 
> overwriten when I add whole c3t2d0 to a new mirror?

p0 is an fdisk partition for the whole disk.
c3t2d0 is really a shorthand notation for c3t2d0s0.
c3t2d0 is also in a fdisk partition of type "Solaris2"

So you'll need to check your partitioning and see where exactly the blocks are
allocated to solve the mystery.

> Is it possible to use 
> c3t2d0 in one pool, and c3t2d0p0 in another?

In general, no. The p0 fdisk partition overlaps whatever fdisk partition is being used
for c3t2d0 -- an invitation to data corruption.

In general, it is considered bad practice to use p0 (or any p*) for ZFS pool creation.
There is a darn good reason the ZFS docs refer to disks as c3t2d0 -- if you do this
consistently, then you'll never have partition nightmares.

HTH
 -- richard

> 
> Thank you,
> Filip
> 
> -----Original Message-----
> From: Zach Malone [mailto:zmalone at omniti.com]
> Sent: Tuesday, December 02, 2014 4:12 PM
> To: Filip Marvan
> Cc: omnios-discuss at lists.omniti.com
> Subject: Re: [OmniOS-discuss] zpool import strange informations
> 
> If I was to guess,  c3t2d0p0 was once a part of another pool called "tank", 
> and when you do an import, it's attempting to reassemble the rest of tank, and 
> sees that a drive named c3t5d0 used to be in the pool it knew as tank.
> 
> Tank is a pretty standard pool name, so I'm guessing the drive used to be in 
> another system, or the system itself used to be running a different OS.
> --Zach
> 
> On Tue, Dec 2, 2014 at 9:31 AM, Filip Marvan <filip.marvan at aira.cz> wrote:
>> Hi,
>> 
>> 
>> 
>> I have one storage server on OmniOS 12 with one data pool:
>> 
>> 
>> 
>>  pool: raid_10
>> 
>> state: ONLINE
>> 
>>  scan: none requested
>> 
>> config:
>> 
>> 
>> 
>>        NAME        STATE     READ WRITE CKSUM
>> 
>>        raid_10     ONLINE       0     0     0
>> 
>>          mirror-0  ONLINE       0     0     0
>> 
>>            c3t2d0  ONLINE       0     0     0
>> 
>>            c3t3d0  ONLINE       0     0     0
>> 
>>          mirror-1  ONLINE       0     0     0
>> 
>>            c3t4d0  ONLINE       0     0     0
>> 
>>            c3t5d0  ONLINE       0     0     0
>> 
>>        logs
>> 
>>          c3t0d0    ONLINE       0     0     0
>> 
>> 
>> 
>> errors: No known data errors
>> 
>> 
>> 
>>  pool: rpool
>> 
>> state: ONLINE
>> 
>>  scan: none requested
>> 
>> config:
>> 
>> 
>> 
>>        NAME        STATE     READ WRITE CKSUM
>> 
>>        rpool       ONLINE       0     0     0
>> 
>>          c3t1d0s0  ONLINE       0     0     0
>> 
>> 
>> 
>> errors: No known data errors
>> 
>> 
>> 
>> There are no other disks in this storage server. Bud if I write
>> command zpool import, I can see:
>> 
>>  pool: tank
>> 
>>     id: 4916680739993977334
>> 
>>  state: UNAVAIL
>> 
>> status: The pool was last accessed by another system.
>> 
>> action: The pool cannot be imported due to damaged devices or data.
>> 
>>   see: http://illumos.org/msg/ZFS-8000-EY
>> 
>> config:
>> 
>> 
>> 
>>        tank                     UNAVAIL  insufficient replicas
>> 
>>          raidz2-1               UNAVAIL  insufficient replicas
>> 
>>            /dev/label/mfront8   UNAVAIL  cannot open
>> 
>>            /dev/label/mfront9   UNAVAIL  cannot open
>> 
>>            /dev/label/mfront10  UNAVAIL  cannot open
>> 
>>            c3t2d0p0             ONLINE
>> 
>>            /dev/label/mfront12  UNAVAIL  cannot open
>> 
>>            /dev/label/mfront13  UNAVAIL  cannot open
>> 
>>            /dev/label/mfront14  UNAVAIL  cannot open
>> 
>>            c3t5d0               ONLINE
>> 
>> 
>> 
>> I never created such "tank" pool. Where are stored informations about
>> this (un)importable pool, how can I delete them? Why c3t5d0 seems to
>> be as part of two different pools at the same time?
>> 
>> 
>> 
>> Thank you for any help.
>> 
>> Filip Marvan
>> 
>> 
>> _______________________________________________
>> OmniOS-discuss mailing list
>> OmniOS-discuss at lists.omniti.com
>> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>> 
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss


From sim.ple at live.nl  Tue Dec  2 23:17:24 2014
From: sim.ple at live.nl (Randy S)
Date: Wed, 3 Dec 2014 00:17:24 +0100
Subject: [OmniOS-discuss] test omnios on OI machine
Message-ID: <DUB124-W4FAE9E8C04253565F247FE07A0@phx.gbl>

Hi,

I have an OI 7 machine running for quite a while now and installed Omnios r10 into a boot env to see how it behaves on the same hardware. 

The machine has a mirrored rpool created on two sas disks which are connected to two seperate LSI SAS9207-4i4e  HBA's .  Each sas disk is connected to its own LSI HBA.

When I boot the machine into OI, the rpool mirror is reported as expected. (Running like this for over a year)

When I boot it into Omnios the rpool mirror is degraded because Omnios cannot see one of the HBA's.
Lsiutil reports only one hba ,
sasinfo hba -v , same thing
echo "::mptsas -t" | mdb -k = ditto
So as far as I can see omnios could not detect one of the hba's

I have cleaned out path_to_inst, done a reconfiguration reboot and devfsadm -Cv but no succes.

I have also installed omnios on OI systems with only one LSI SAS9207-4i4e with no problem. Can someone tell me if it is known if omnios has problems with a dual hba system or how to force omnios to detect the other HBA ? We have also tested with multipath disabled and enabled.

Rgds,

R
 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141203/8180d2bd/attachment.html>

From danmcd at omniti.com  Tue Dec  2 23:35:42 2014
From: danmcd at omniti.com (Dan McDonald)
Date: Tue, 2 Dec 2014 18:35:42 -0500
Subject: [OmniOS-discuss] test omnios on OI machine
In-Reply-To: <DUB124-W4FAE9E8C04253565F247FE07A0@phx.gbl>
References: <DUB124-W4FAE9E8C04253565F247FE07A0@phx.gbl>
Message-ID: <13009DC9-7FA5-4938-8042-F2AF74DDC695@omniti.com>


> On Dec 2, 2014, at 6:17 PM, Randy S <sim.ple at live.nl> wrote:
> 
> Hi,
> 
> I have an OI 7 machine running for quite a while now and installed Omnios r10 into a boot env to see how it behaves on the same hardware. 

Try upgrading r10 to r12 and see if the same thing happens please.

Next, utter "dmesg | grep mpt_sas" to see if there are any complaints from the kernel.

Dan


From sim.ple at live.nl  Wed Dec  3 00:01:33 2014
From: sim.ple at live.nl (Randy S)
Date: Wed, 3 Dec 2014 01:01:33 +0100
Subject: [OmniOS-discuss] test omnios on OI machine
In-Reply-To: <DUB124-W31A653A65DB219489C3FBAE07A0@phx.gbl>
References: <DUB124-W4FAE9E8C04253565F247FE07A0@phx.gbl>,
	<13009DC9-7FA5-4938-8042-F2AF74DDC695@omniti.com>,
	<DUB124-W31A653A65DB219489C3FBAE07A0@phx.gbl>
Message-ID: <DUB124-W4D2197DE98771D3138C6BE07B0@phx.gbl>


Sorry should have send this to the list.


Subject: RE: [OmniOS-discuss] test omnios on OI machine
Date: Wed, 3 Dec 2014 00:59:33 +0100




Hi Dan,

It's been a while...

I wanted to stay on r10 since that has been in use by quite a community for quite a while now. R12 is still new so usually unforseen hicups can be expected. (We are not frontrunners regarding systems intended for serious use  ;-) ). Have been testing omnios for quite a while on systems with one hba now. All was satisfactory. 

forgot to mention, I did do an fmdump -ev and saw 
Dec 02 23:32:37.1675 ereport.fs.zfs.vdev.open_failed       0x0077a03af6b00001
Dec 02 23:32:37.1675 ereport.fs.zfs.vdev.open_failed       0x0077a03af6b00001
Dec 02 23:35:17.6783 ereport.sensor.failure                0x02cd95a4cde05801
Dec 02 23:35:17.6868 ereport.sensor.failure                0x02cd9dbb53605801

I guess this is because the one controller cannot be found.


dmesg | grep mpt_sa (done just now (not sleepy yet)) gives :

Dec  3 00:42:44 omni scsi: [ID 583861 kern.info] sd68 at mpt_sas3: unit-address w5000c50057ad9161,0: w5000c50057ad9161,0
Dec  3 00:42:44 omni scsi: [ID 583861 kern.info] sd69 at mpt_sas3: unit-address w5000c50057adb269,0: w5000c50057adb269,0
Dec  3 00:42:45 omni scsi: [ID 583861 kern.info] sd70 at mpt_sas3: unit-address w5000c50057adea91,0: w5000c50057adea91,0
Dec  3 00:42:45 omni scsi: [ID 583861 kern.info] ses2 at mpt_sas3: unit-address w5003048000b18d3d,0: w5003048000b18d3d,0
Dec  3 00:42:45 omni scsi: [ID 583861 kern.info] mpt_sas4 at mpt_sas0: scsi-iport 20
Dec  3 00:42:45 omni genunix: [ID 936769 kern.info] mpt_sas4 is /pci at 0,0/pci8086,3c04 at 2/pci1000,3030 at 0/iport at 20
Dec  3 00:42:45 omni genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,3c04 at 2/pci1000,3030 at 0/iport at 20 (mpt_sas4) online
Dec  3 00:42:45 omni scsi: [ID 583861 kern.info] sd71 at mpt_sas4: unit-address w5001517bb27f79f7,0: w5001517bb27f79f7,0
Dec  3 00:42:45 omni scsi: [ID 583861 kern.info] mpt_sas5 at mpt_sas0: scsi-iport v0
Dec  3 00:42:45 omni genunix: [ID 936769 kern.info] mpt_sas5 is /pci at 0,0/pci8086,3c04 at 2/pci1000,3030 at 0/iport at v0
Dec  3 00:42:45 omni genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,3c04 at 2/pci1000,3030 at 0/iport at v0 (mpt_sas5) online

Thanks,

R


> Subject: Re: [OmniOS-discuss] test omnios on OI machine
> From: danmcd at omniti.com
> Date: Tue, 2 Dec 2014 18:35:42 -0500
> CC: omnios-discuss at lists.omniti.com
> To: sim.ple at live.nl
> 
> 
> > On Dec 2, 2014, at 6:17 PM, Randy S <sim.ple at live.nl> wrote:
> > 
> > Hi,
> > 
> > I have an OI 7 machine running for quite a while now and installed Omnios r10 into a boot env to see how it behaves on the same hardware. 
> 
> Try upgrading r10 to r12 and see if the same thing happens please.
> 
> Next, utter "dmesg | grep mpt_sas" to see if there are any complaints from the kernel.
> 
> Dan
> 
 		 	   		   		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141203/ab30f7cc/attachment.html>

From sim.ple at live.nl  Wed Dec  3 00:45:12 2014
From: sim.ple at live.nl (Randy S)
Date: Wed, 3 Dec 2014 01:45:12 +0100
Subject: [OmniOS-discuss] test omnios on OI machine
In-Reply-To: <63802B04-B4E9-4FC6-8059-EDAD86FBADBE@omniti.com>
References: <DUB124-W4FAE9E8C04253565F247FE07A0@phx.gbl>
	<,<13009DC9-7FA5-4938-8042-F2AF74DDC695@omniti.com> <>>
	<DUB124-W31A653A65DB219489C3FBAE07A0@phx.gbl>,
	<63802B04-B4E9-4FC6-8059-EDAD86FBADBE@omniti.com>
Message-ID: <DUB124-W28749FC3157E8298D365CAE07B0@phx.gbl>

Hi, was indeed intended for the list.

dmesg | grep mptsas

gives nothing (no output). 

Hope you have an idea of what to do next. 

Going to test a current omnios with one hba by adding another to it to see what happens tomorrow.  

Will be testing with new Dell hardware and solarflare cards probably in the comming weeks also. If you are interrested, will keep you posted. 



> Subject: Re: [OmniOS-discuss] test omnios on OI machine
> From: danmcd at omniti.com
> Date: Tue, 2 Dec 2014 19:22:15 -0500
> CC: danmcd at omniti.com
> To: sim.ple at live.nl
> 
> Please share these with the community in the future, unless you're sharing confidential data of some sort.
> 
> 
> > On Dec 2, 2014, at 6:59 PM, Randy S <sim.ple at live.nl> wrote:
> > 
> > Hi Dan,
> > 
> > It's been a while...
> > 
> > I wanted to stay on r10 since that has been in use by quite a community for quite a while now. R12 is still new so usually unforseen hicups can be expected. (We are not frontrunners regarding systems intended for serious use  ;-) ). Have been testing omnios for quite a while on systems with one hba now. All was satisfactory. 
> > 
> > forgot to mention, I did do an fmdump -ev and saw 
> > Dec 02 23:32:37.1675 ereport.fs.zfs.vdev.open_failed       0x0077a03af6b00001
> > Dec 02 23:32:37.1675 ereport.fs.zfs.vdev.open_failed       0x0077a03af6b00001
> > Dec 02 23:35:17.6783 ereport.sensor.failure                0x02cd95a4cde05801
> > Dec 02 23:35:17.6868 ereport.sensor.failure                0x02cd9dbb53605801
> > 
> > I guess this is because the one controller cannot be found.
> 
> That's a fair assessment.
> 
> > dmesg | grep mpt_sa (done just now (not sleepy yet)) gives :
> > 
> > Dec  3 00:42:44 omni scsi: [ID 583861 kern.info] sd68 at mpt_sas3: unit-address w5000c50057ad9161,0: w5000c50057ad9161,0
> > Dec  3 00:42:44 omni scsi: [ID 583861 kern.info] sd69 at mpt_sas3: unit-address w5000c50057adb269,0: w5000c50057adb269,0
> > Dec  3 00:42:45 omni scsi: [ID 583861 kern.info] sd70 at mpt_sas3: unit-address w5000c50057adea91,0: w5000c50057adea91,0
> > Dec  3 00:42:45 omni scsi: [ID 583861 kern.info] ses2 at mpt_sas3: unit-address w5003048000b18d3d,0: w5003048000b18d3d,0
> > Dec  3 00:42:45 omni scsi: [ID 583861 kern.info] mpt_sas4 at mpt_sas0: scsi-iport 20
> > Dec  3 00:42:45 omni genunix: [ID 936769 kern.info] mpt_sas4 is /pci at 0,0/pci8086,3c04 at 2/pci1000,3030 at 0/iport at 20
> > Dec  3 00:42:45 omni genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,3c04 at 2/pci1000,3030 at 0/iport at 20 (mpt_sas4) online
> > Dec  3 00:42:45 omni scsi: [ID 583861 kern.info] sd71 at mpt_sas4: unit-address w5001517bb27f79f7,0: w5001517bb27f79f7,0
> > Dec  3 00:42:45 omni scsi: [ID 583861 kern.info] mpt_sas5 at mpt_sas0: scsi-iport v0
> > Dec  3 00:42:45 omni genunix: [ID 936769 kern.info] mpt_sas5 is /pci at 0,0/pci8086,3c04 at 2/pci1000,3030 at 0/iport at v0
> > Dec  3 00:42:45 omni genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,3c04 at 2/pci1000,3030 at 0/iport at v0 (mpt_sas5) online
> 
> Interesting... note that two mpt_sas controllers (4 and 5) came online.  I wonder why one isn't showing up?  Grep for "mptsas" as well, just in case?
> 
> Dan
> 
 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141203/b24ff1d2/attachment-0001.html>

From henson at acm.org  Wed Dec  3 02:03:12 2014
From: henson at acm.org (Paul B. Henson)
Date: Tue, 2 Dec 2014 18:03:12 -0800
Subject: [OmniOS-discuss] common-factor key exchange
In-Reply-To: <4fcd8a9de3d5ab64e565365c78cb3105@kivalo.at>
References: <20141128191703.00a9df99@punda-mlia>
	<4fcd8a9de3d5ab64e565365c78cb3105@kivalo.at>
Message-ID: <306f01d00e9d$4d16ab50$e74401f0$@acm.org>

> From: Christian Kivalo
> Sent: Friday, November 28, 2014 1:55 AM
>
> I would be interestent in replacing the default ssh package with OpenSSH
on
> the OmniOS installation, but the ssh package seems to be bound to @entire
so
> this looks to me as if this is not as easy as uninstall/install to replace
the ssh
> daemon/client...

I'm definitely interested in that as well, although I haven't actually had
the time to try and run into the issue you found :).

Any official omniti perspective on making openssh the default and legacy sun
ssh the alternative option?

Thanks.


From henson at acm.org  Wed Dec  3 02:05:54 2014
From: henson at acm.org (Paul B. Henson)
Date: Tue, 2 Dec 2014 18:05:54 -0800
Subject: [OmniOS-discuss] common-factor key exchange
In-Reply-To: <20141202094149.43d9f5d9@punda-mlia>
References: <20141128191703.00a9df99@punda-mlia>	<20141130025557.GI29549@bender.unx.csupomona.edu>	<20141201055654.2a3e23f8@punda-mlia>	<234301d00db7$32bdcc20$98396460$@acm.org>
	<20141202094149.43d9f5d9@punda-mlia>
Message-ID: <314901d00e9d$ade49f00$09addd00$@acm.org>

> From: Michael Mounteney
> Sent: Monday, December 01, 2014 3:42 PM
>
> but the reason for the problem is that the older algorithms have been
> removed from openssh-6.7.  I just downgraded to 6.6 on one machine and
> once again was able to ssh in from OmniOS.  Upgrade to 6.7 again and the
> common kex problem re-arose.

Ah, that must have gone stable right after I did my last update. Guess I
will have this issue to look forward to for our next update cycle 8-/...


From gate03 at landcroft.co.uk  Wed Dec  3 03:16:57 2014
From: gate03 at landcroft.co.uk (Michael Mounteney)
Date: Wed, 3 Dec 2014 13:16:57 +1000
Subject: [OmniOS-discuss] common-factor key exchange
In-Reply-To: <314901d00e9d$ade49f00$09addd00$@acm.org>
References: <20141128191703.00a9df99@punda-mlia>
	<20141130025557.GI29549@bender.unx.csupomona.edu>
	<20141201055654.2a3e23f8@punda-mlia>
	<234301d00db7$32bdcc20$98396460$@acm.org>
	<20141202094149.43d9f5d9@punda-mlia>
	<314901d00e9d$ade49f00$09addd00$@acm.org>
Message-ID: <20141203131657.11875145@punda-mlia>

On Tue, 2 Dec 2014 18:05:54 -0800
"Paul B. Henson" <henson at acm.org> wrote:

> Ah, that must have gone stable right after I did my last update.
> Guess I will have this issue to look forward to for our next update
> cycle 8-/...

I did consider holding on to 6.6 but there must be a good reason for the
issuance of 6.7.  As I said originally, I'd rather go up with OmniOS
than down with Gentoo.

Michael.

From henson at acm.org  Wed Dec  3 03:51:14 2014
From: henson at acm.org (Paul B. Henson)
Date: Tue, 2 Dec 2014 19:51:14 -0800
Subject: [OmniOS-discuss] common-factor key exchange
In-Reply-To: <20141203131657.11875145@punda-mlia>
References: <20141128191703.00a9df99@punda-mlia>
	<20141130025557.GI29549@bender.unx.csupomona.edu>
	<20141201055654.2a3e23f8@punda-mlia>
	<234301d00db7$32bdcc20$98396460$@acm.org>
	<20141202094149.43d9f5d9@punda-mlia>
	<314901d00e9d$ade49f00$09addd00$@acm.org>
	<20141203131657.11875145@punda-mlia>
Message-ID: <20141203035114.GL29549@bender.unx.csupomona.edu>

On Wed, Dec 03, 2014 at 01:16:57PM +1000, Michael Mounteney wrote:

> I did consider holding on to 6.6 but there must be a good reason for the
> issuance of 6.7.  As I said originally, I'd rather go up with OmniOS
> than down with Gentoo.

Looks like it fixes this security bug:

https://bugs.gentoo.org/show_bug.cgi?id=505942

Don't think many people are using SSHFP DNS records, so probably not a
big deal at this point.

I think the odds of illumos ssh getting updated are pretty low, so we'll
have to wait and see what the omnios powers that be think of switching
the distribution to openssh...


From moo at wuffers.net  Wed Dec  3 06:02:40 2014
From: moo at wuffers.net (wuffers)
Date: Wed, 3 Dec 2014 01:02:40 -0500
Subject: [OmniOS-discuss] Bad ZeusRAM? How to tell if another component is
	causing issues?
Message-ID: <CA+tR_Kyxzf6XEczbgaxCsvKdtJ74C-+_0SkMjs=rmDufiCAJsQ@mail.gmail.com>

I'm at home just looking into the health of our SAN and came across a bunch
of errors on the Stec ZeusRAM (in a mirrored log configuration):

# iostat -En
c12t5000A72B300780FFd0 Soft Errors: 0 Hard Errors: 1 Transport Errors: 5224
Vendor: STEC     Product: ZeusRAM          Revision: C018 Serial No:
STM000170C98
Size: 8.00GB <8000000000 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 391 Predictive Failure Analysis: 0

#fmdump -eV
Dec 03 2014 00:26:22.592888816 ereport.io.scsi.cmd.disk.recovered
nvlist version: 0
        class = ereport.io.scsi.cmd.disk.recovered
        ena = 0xd38b237e7ed02001
        detector = (embedded nvlist)
        nvlist version: 0
                version = 0x0
                scheme = dev
                device-path = /pci at 0,0/pci8086,3c08 at 3/pci1000,3030 at 0/iport at f
/disk at w5000a72b300780ff,0
                devid = id1,sd at n5000a720300780ff
        (end detector)

        devid = id1,sd at n5000a720300780ff
        driver-assessment = recovered
        op-code = 0x2a
        cdb = 0x2a 0x0 0x0 0x2d 0xda 0x0 0x0 0x0 0xf8 0x0
        pkt-reason = 0x0
        pkt-state = 0x1f
        pkt-stats = 0x0
        __ttl = 0x1
        __tod = 0x547e9efe 0x2356c3f0

# dmesg
Dec  3 00:28:24 san1 scsi: [ID 365881 kern.info] /pci at 0,0/pci8086,3c08 at 3
/pci1000,3030 at 0 (mpt_sas1):
Dec  3 00:28:24 san1    Log info 0x31120303 received for target 10
w5000a72b300780ff.
Dec  3 00:28:24 san1    scsi_status=0x0, ioc_status=0x804b, scsi_state=0xc

from format:

57. c12t5000A72B300780FFd0 <STEC-ZeusRAM-C018-7.45GB>
          /pci at 0,0/pci8086,3c08 at 3/pci1000,3030 at 0/iport at f
/disk at w5000a72b300780ff,0

Both fmdump and dmesg has these errors repeating over and over. Everything
seems to point to the drive. I suppose I would have to physically move the
drive to eliminate cable, backplane or controller issues. Is there another
way to tell just by these error logs or is the physical test the way to go?

Are logs enough to justify an RMA?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141203/eb6f9b85/attachment.html>

From moo at wuffers.net  Wed Dec  3 06:11:59 2014
From: moo at wuffers.net (wuffers)
Date: Wed, 3 Dec 2014 01:11:59 -0500
Subject: [OmniOS-discuss] Slow scrub performance
In-Reply-To: <CA+tR_KyPZs38vOoSZjN5ob7GS2j=V-ZvCuY_abgXpqS62UjWPA@mail.gmail.com>
References: <CA+tR_KxTmft+hK9U8HLgqmPOdZf0+9q67+Kg9p1JwX6sg5drAA@mail.gmail.com>
	<C7904250-3A94-4D4B-8D62-D49FDE808008@richardelling.com>
	<CA+tR_KwX_1HN4tVa+-ZOFJk2mN7RE-nFh31sMcTNo7TJJjfyLg@mail.gmail.com>
	<CA+tR_KxSt3VVeUVZAOL4FxWsRL3M-PwpBF6JbHVXv5Xx66Jozg@mail.gmail.com>
	<BE32B046-05EC-43D2-AC46-7649FAC34C2A@richardelling.com>
	<CA+tR_KyPZs38vOoSZjN5ob7GS2j=V-ZvCuY_abgXpqS62UjWPA@mail.gmail.com>
Message-ID: <CA+tR_KxhWH95EoAgOtkMp6qkTMXMsusgkP87VhO4g6rhqJSx+A@mail.gmail.com>

Just wanted to update this thread with my results. I had the scrub started
since late July, and checked on it at the end of Oct:

  pool: tank
 state: ONLINE
  scan: scrub in progress since Tue Jul 29 15:41:27 2014
    7.65T scanned out of 24.5T at 1024K/s, (scan is slow, no estimated time)
    0 repaired, 31.16% done

So it was scrubbing away for ~90 days and got a bit over 31% with
vdev_scrub_max set to 5.

In anticipation of having to power off the head nodes for the building's
power maintenance and not wanting to waste the scrub so far, I pushed the
vdev_scrub_max to 10. The scrub then proceeded to accelerate to about 8%
per day, and ended:

  pool: tank
 state: ONLINE
  scan: scrub repaired 0 in 2372h54m with 0 errors on Wed Nov  5 11:36:00
2014

My pool wasn't busy so I was okay with it having so much time slice, but
this obviously isn't the best thing to do. Is this really normal, or is
there something wrong with my config?

On Thu, Jul 31, 2014 at 3:44 PM, wuffers <moo at wuffers.net> wrote:

> This is going to be long winded as well (apologies!).. lots of pasted data.
>
> On Thu, Jul 31, 2014 at 1:37 AM, Richard Elling <
> richard.elling at richardelling.com> wrote:
>
>>
>> The %busy for controllers is a sum of the %busy for all disks on the
>> controller, so
>> is can be large, but overall isn't interesting. With HDDs, there is no
>> way you can
>> saturate the controller, so we don't really care what the %busy shows.
>>
>> The more important item is that the number of read ops is fairly low for
>> all but 4 disks.
>> Since you didn't post the pool configuration, we can only guess that they
>> might be a
>> souce of the bottleneck.
>>
>> You're seeing a lot of reads from the cache devices. How much RAM does
>> this system
>> have?
>>
>>
> I realized that the busy % was a sum after looking through some of that
> data, but good to know that it isn't very relevant.
>
> The pool configuration was in the original post, but here it is again
> (after re-attaching the mirror log device). Just saw your edit, but this
> has been updated from the original post anyways.
>
>   pool: tank
>  state: ONLINE
>   scan: scrub in progress since Tue Jul 29 15:41:27 2014
>     82.5G scanned out of 24.5T at 555K/s, (scan is slow, no estimated time)
>     0 repaired, 0.33% done
> config:
>
>         NAME                        STATE     READ WRITE CKSUM
>         tank                        ONLINE       0     0     0
>           mirror-0                  ONLINE       0     0     0
>             c1t5000C50055F9F637d0   ONLINE       0     0     0
>             c1t5000C50055F9EF2Fd0   ONLINE       0     0     0
>           mirror-1                  ONLINE       0     0     0
>             c1t5000C50055F87D97d0   ONLINE       0     0     0
>             c1t5000C50055F9D3B3d0   ONLINE       0     0     0
>           mirror-2                  ONLINE       0     0     0
>             c1t5000C50055E6606Fd0   ONLINE       0     0     0
>             c1t5000C50055F9F92Bd0   ONLINE       0     0     0
>           mirror-3                  ONLINE       0     0     0
>             c1t5000C50055F856CFd0   ONLINE       0     0     0
>             c1t5000C50055F9FE87d0   ONLINE       0     0     0
>           mirror-4                  ONLINE       0     0     0
>             c1t5000C50055F84A97d0   ONLINE       0     0     0
>             c1t5000C50055FA0AF7d0   ONLINE       0     0     0
>           mirror-5                  ONLINE       0     0     0
>             c1t5000C50055F9D3E3d0   ONLINE       0     0     0
>             c1t5000C50055F9F0B3d0   ONLINE       0     0     0
>           mirror-6                  ONLINE       0     0     0
>             c1t5000C50055F8A46Fd0   ONLINE       0     0     0
>             c1t5000C50055F9FB8Bd0   ONLINE       0     0     0
>           mirror-7                  ONLINE       0     0     0
>             c1t5000C50055F8B21Fd0   ONLINE       0     0     0
>             c1t5000C50055F9F89Fd0   ONLINE       0     0     0
>           mirror-8                  ONLINE       0     0     0
>             c1t5000C50055F8BE3Fd0   ONLINE       0     0     0
>             c1t5000C50055F9E123d0   ONLINE       0     0     0
>           mirror-9                  ONLINE       0     0     0
>             c1t5000C50055F9379Bd0   ONLINE       0     0     0
>             c1t5000C50055F9E7D7d0   ONLINE       0     0     0
>           mirror-10                 ONLINE       0     0     0
>             c1t5000C50055E65F0Fd0   ONLINE       0     0     0
>             c1t5000C50055F9F80Bd0   ONLINE       0     0     0
>           mirror-11                 ONLINE       0     0     0
>             c1t5000C50055F8A22Bd0   ONLINE       0     0     0
>             c1t5000C50055F8D48Fd0   ONLINE       0     0     0
>           mirror-12                 ONLINE       0     0     0
>             c1t5000C50055E65807d0   ONLINE       0     0     0
>             c1t5000C50055F8BFA3d0   ONLINE       0     0     0
>           mirror-13                 ONLINE       0     0     0
>             c1t5000C50055E579F7d0   ONLINE       0     0     0
>             c1t5000C50055E65877d0   ONLINE       0     0     0
>           mirror-14                 ONLINE       0     0     0
>             c1t5000C50055F9FA1Fd0   ONLINE       0     0     0
>             c1t5000C50055F8CDA7d0   ONLINE       0     0     0
>           mirror-15                 ONLINE       0     0     0
>             c1t5000C50055F8BF9Bd0   ONLINE       0     0     0
>             c1t5000C50055F9A607d0   ONLINE       0     0     0
>           mirror-16                 ONLINE       0     0     0
>             c1t5000C50055E66503d0   ONLINE       0     0     0
>             c1t5000C50055E4FDE7d0   ONLINE       0     0     0
>           mirror-17                 ONLINE       0     0     0
>             c1t5000C50055F8E017d0   ONLINE       0     0     0
>             c1t5000C50055F9F3EBd0   ONLINE       0     0     0
>           mirror-18                 ONLINE       0     0     0
>             c1t5000C50055F8B80Fd0   ONLINE       0     0     0
>             c1t5000C50055F9F63Bd0   ONLINE       0     0     0
>           mirror-19                 ONLINE       0     0     0
>             c1t5000C50055F84FB7d0   ONLINE       0     0     0
>             c1t5000C50055F9FEABd0   ONLINE       0     0     0
>           mirror-20                 ONLINE       0     0     0
>             c1t5000C50055F8CCAFd0   ONLINE       0     0     0
>             c1t5000C50055F9F91Bd0   ONLINE       0     0     0
>           mirror-21                 ONLINE       0     0     0
>             c1t5000C50055E65ABBd0   ONLINE       0     0     0
>             c1t5000C50055F8905Fd0   ONLINE       0     0     0
>           mirror-22                 ONLINE       0     0     0
>             c1t5000C50055E57A5Fd0   ONLINE       0     0     0
>             c1t5000C50055F87E73d0   ONLINE       0     0     0
>           mirror-23                 ONLINE       0     0     0
>             c1t5000C50055E66053d0   ONLINE       0     0     0
>             c1t5000C50055E66B63d0   ONLINE       0     0     0
>           mirror-24                 ONLINE       0     0     0
>             c1t5000C50055F8723Bd0   ONLINE       0     0     0
>             c1t5000C50055F8C3ABd0   ONLINE       0     0     0
>         logs
>           mirror-25                 ONLINE       0     0     0
>             c2t5000A72A3007811Dd0   ONLINE       0     0     0
>             c12t5000A72B300780FFd0  ONLINE       0     0     0
>         cache
>           c2t500117310015D579d0     ONLINE       0     0     0
>           c2t50011731001631FDd0     ONLINE       0     0     0
>           c12t500117310015D59Ed0    ONLINE       0     0     0
>           c12t500117310015D54Ed0    ONLINE       0     0     0
>         spares
>           c1t5000C50055FA2AEFd0     AVAIL
>           c1t5000C50055E595B7d0     AVAIL
>
> Basically, this is 2 head nodes (Supermicro 826BE26) connected to a
> Supermicro 847E26 JBOD, using LSI 9207s. There are 52 Seagate ST4000NM0023s
> (4TB SAS drives) in 25 mirror pairs plus 2 which are spares. There are 4
> Smart Optimus 400GB SSDs as cache drives, and 2 Stec ZeusRAMs for slogs.
> They're wired in such a way that both nodes can see all the drives (data,
> cache and log), and the data drives are on separate controllers than the
> cache/slog devices. RSF-1 was also specced in here but not in use at the
> moment. All the SAN traffic is through InfiniBand (SRP). Each head unit has
> 256GB of RAM. Dedupe is not in use and all the latest feature flags are
> enabled.
>
> An arc_summary output:
>
> System Memory:
>          Physical RAM:  262103 MB
>          Free Memory :  10273 MB
>          LotsFree:      4095 MB
>
> ZFS Tunables (/etc/system):
>          set zfs:zfs_arc_shrink_shift = 10
>
> ARC Size:
>          Current Size:             225626 MB (arcsize)
>          Target Size (Adaptive):   225626 MB (c)
>          Min Size (Hard Limit):    8190 MB (zfs_arc_min)
>          Max Size (Hard Limit):    261079 MB (zfs_arc_max)
>
> ARC Size Breakdown:
>          Most Recently Used Cache Size:          10%    23290 MB (p)
>          Most Frequently Used Cache Size:        89%    202335 MB (c-p)
>
> ARC Efficency:
>          Cache Access Total:             27377320465
>          Cache Hit Ratio:      93%       25532510784    [Defined State for
> buffer]
>          Cache Miss Ratio:      6%       1844809681     [Undefined State
> for Buffer]
>          REAL Hit Ratio:       92%       25243933796    [MRU/MFU Hits Only]
>
>          Data Demand   Efficiency:    95%
>          Data Prefetch Efficiency:    40%
>
>         CACHE HITS BY CACHE LIST:
>           Anon:                       --%        Counter Rolled.
>           Most Recently Used:         18%        4663226393 (mru)       [
> Return Customer ]
>           Most Frequently Used:       80%        20580707403 (mfu)
>      [ Frequent Customer ]
>           Most Recently Used Ghost:    0%        176686906 (mru_ghost)  [
> Return Customer Evicted, Now Back ]
>           Most Frequently Used Ghost:  0%        126286869 (mfu_ghost)  [
> Frequent Customer Evicted, Now Back ]
>         CACHE HITS BY DATA TYPE:
>           Demand Data:                95%        24413671342
>           Prefetch Data:               1%        358419705
>           Demand Metadata:             2%        698314899
>           Prefetch Metadata:           0%        62104838
>         CACHE MISSES BY DATA TYPE:
>           Demand Data:                69%        1277347273
>           Prefetch Data:              28%        519579788
>           Demand Metadata:             2%        39512363
>           Prefetch Metadata:           0%        8370257
>
> And a sample of arcstat (deleted first line of output):
>
> # arcstat -f
> read,hits,miss,hit%,l2read,l2hits,l2miss,l2hit%,arcsz,l2size,l2asize 1
>
> read  hits  miss  hit%  l2read  l2hits  l2miss  l2hit%  arcsz  l2size
>  l2asize
> 5.9K  4.6K  1.3K    78    1.3K    1.2K      80      93   220G    1.6T
> 901G
> 6.7K  5.2K  1.5K    76    1.5K    1.3K     250      83   220G    1.6T
> 901G
> 7.0K  5.3K  1.7K    76    1.7K    1.4K     316      81   220G    1.6T
> 901G
> 6.5K  5.3K  1.2K    80    1.2K    1.1K     111      91   220G    1.6T
> 901G
> 6.4K  5.2K  1.2K    81    1.2K    1.1K     100      91   220G    1.6T
> 901G
> 7.2K  5.6K  1.6K    78    1.6K    1.3K     289      81   220G    1.6T
> 901G
> 8.5K  6.8K  1.7K    80    1.7K    1.3K     351      79   220G    1.6T
> 901G
> 7.5K  5.9K  1.6K    78    1.6K    1.3K     282      82   220G    1.6T
> 901G
> 6.7K  5.6K  1.1K    83    1.1K     991     123      88   220G    1.6T
> 901G
> 6.8K  5.5K  1.3K    80    1.3K    1.1K     234      82   220G    1.6T
> 901G
>
> Interesting to see only an l2asize of 901G even though I should have
> more.. 373G x 4 is just under 1.5TB of raw storage. The compressed l2arc
> size is 1.6TB, while actual used space is 901G. I expect more to be used.
> Perhaps Saso can comment on this portion, if he's following this thread
> (snipped from "zpool iostat -v"):
>
> cache                           -      -      -      -      -      -
>   c2t500117310015D579d0      373G     8M    193     16  2.81M   394K
>   c2t50011731001631FDd0      373G  5.29M    194     15  2.85M   360K
>   c12t500117310015D59Ed0     373G  5.50M    191     17  2.74M   368K
>   c12t500117310015D54Ed0     373G  5.57M    200     14  2.89M   300K
>
> (from this discussion here:
> http://lists.omniti.com/pipermail/omnios-discuss/2014-February/002287.html),
> and the uptime on this is currently around ~58 days, so it should have had
> enough time to rotate through the l2arc "rotor".
>
>
>> methinks the scrub I/Os are getting starved and since they are low
>> priority, they
>> could get very starved. In general, I wouldn't worry about it, but I
>> understand
>> why you might be nervous. Keep in mind that in ZFS scrubs are intended to
>> find
>> errors on idle data, not frequently accessed data.
>>
>> more far below...
>>
>>
> I'm worried because there's no way the scrub will ever complete before the
> next reboot. Regular scrubs are important, right?
>
>
>> ok, so the pool is issuing 720 read iops, including resilver workload, vs
>> 1298 write iops.
>> There is plenty of I/O capacity left on the table here, as you can see by
>> the %busy being
>> so low.
>>
>> So I think the pool is not scheduling scrub I/Os very well. You can
>> increase the number of
>> scrub I/Os in the scheduler by adjusting the zfs_vdev_scrub_max_active
>> tunable. The
>> default is 2, but you'll have to consider that a share (in the stock
>> market sense) where
>> the active sync reads and writes are getting 10 each. You can try bumping
>> up the value
>> and see what happens over some time, perhaps 10 minutes or so -- too
>> short of a time
>> and you won't get a good feeling for the impact (try this in off-peak
>> time).
>> echo zfs_vdev_scrub_max_active/W0t5 | mdb -kw
>> will change the value from 2 to 5, increasing its share of the total I/O
>> workload.
>>
>> You can see the progress of scan (scrubs do scan) workload by looking at
>> the ZFS
>> debug messages.
>> echo ::zfs_dbgmsg | mdb -k
>> These will look mysterious... they are. But the interesting bits are
>> about how many blocks
>> are visited in some amount of time (txg sync interval). Ideally, this
>> will change as you
>> adjust zfs_vdev_scrub_max_active.
>>  -- richard
>>
>>
> Actually, you used the data from before the resilver. During resilver this
> was the activity on the pool:
>
> 3883.5 1357.7 40141.6 60739.5 22.8 38.6    4.4    7.4  54 100 tank
>
> Are you looking at an individual drive's busy % or the pool's busy % to
> determine whether it's "busy"? During the resilver this was the activity on
> the drives (a sample - between 38-45%, whereas during the scrub the
> individual drives were 2-5% busy):
>
>    59.5   30.0  553.2  741.8  0.0  0.9    0.0    9.8   0  45
> c1t5000C50055F8A46Fd0
>    57.4   22.5  504.0  724.5  0.0  0.8    0.0    9.6   0  41
> c1t5000C50055F856CFd0
>    58.4   24.6  531.4  786.9  0.0  0.7    0.0    8.4   0  38
> c1t5000C50055E6606Fd0
>
> But yes, without the resilver the busy % was much less (during the scrub
> each individual drive was 2-4% busy). I've pasted the current iostat output
> further below.
>
> With the zfs_vdev_scrub_max_active at the default of 2, it was doing an
> average of 162 blocks:
>
> doing scan sync txg 26678243; bm=897/1/0/15785978
> scanned dataset 897 (tank/vmware-64k-5tb-1) with min=1 max=26652167;
> pausing=1
> visited 162 blocks in 6090ms
> doing scan sync txg 26678244; bm=897/1/0/15786126
> scanned dataset 897 (tank/vmware-64k-5tb-1) with min=1 max=26652167;
> pausing=1
> visited 162 blocks in 6094ms
>
> After changing it to 5, and waiting about 20 mins, I'm not seeing anything
> significantly different:
>
> doing scan sync txg 26678816; bm=897/1/0/37082043
> scanned dataset 897 (tank/vmware-64k-5tb-1) with min=1 max=26652167;
> pausing=1
> visited 163 blocks in 6154ms
> doing scan sync txg 26678817; bm=897/1/0/37082193
> scanned dataset 897 (tank/vmware-64k-5tb-1) with min=1 max=26652167;
> pausing=1
> visited 162 blocks in 6128ms
>
>   pool: tank
>  state: ONLINE
>   scan: scrub in progress since Tue Jul 29 15:41:27 2014
>     97.0G scanned out of 24.5T at 599K/s, (scan is slow, no estimated time)
>     0 repaired, 0.39% done
>
> I'll keep the zfs_vdev_scrub_max_active tunable to 5, as it doesn't
> appear to be impacting too much, and monitor for changes. What's strange to
> me is that it was "humming" along at 5.5MB/s at the 2 week mark but is now
> 10x slower (compared to before reattaching the mirror log device). It
> *seems* marginally faster, from 541K/s to almost 600K/s..
>
> This is the current activity from "iostat -xnCz 60 2":
>
>                     extended device statistics
>     r/s    w/s   kr/s   kw/s wait actv wsvc_t asvc_t  %w  %b device
>   158.8 1219.2 3717.8 39969.8  0.0  1.6    0.0    1.1   0 139 c1
>     3.6   35.1   86.2  730.9  0.0  0.0    0.0    0.9   0   3
> c1t5000C50055F8723Bd0
>     3.7   19.9   83.7  789.9  0.0  0.0    0.0    1.4   0   3
> c1t5000C50055E66B63d0
>     2.7   22.5   60.8  870.9  0.0  0.0    0.0    1.1   0   2
> c1t5000C50055F87E73d0
>     2.4   27.9   66.0  765.8  0.0  0.0    0.0    0.8   0   2
> c1t5000C50055F8BFA3d0
>     2.8   17.9   64.9  767.0  0.0  0.0    0.0    0.8   0   1
> c1t5000C50055F9E123d0
>     3.1   26.1   73.8  813.3  0.0  0.0    0.0    0.9   0   2
> c1t5000C50055F9F0B3d0
>     3.1   15.5   79.4  783.4  0.0  0.0    0.0    1.3   0   2
> c1t5000C50055F9D3B3d0
>     3.8   38.6   86.2  826.8  0.0  0.1    0.0    1.2   0   4
> c1t5000C50055E4FDE7d0
>     3.8   15.4   93.0  822.3  0.0  0.0    0.0    1.5   0   3
> c1t5000C50055F9A607d0
>     3.0   25.7   79.4  719.7  0.0  0.0    0.0    0.9   0   2
> c1t5000C50055F8CDA7d0
>     3.2   26.5   69.0  824.3  0.0  0.0    0.0    1.1   0   3
> c1t5000C50055E65877d0
>     3.7   42.6   79.2  834.1  0.0  0.1    0.0    1.3   0   5
> c1t5000C50055F9E7D7d0
>     3.3   23.2   79.5  778.0  0.0  0.0    0.0    1.2   0   3
> c1t5000C50055FA0AF7d0
>     3.4   30.2   77.0  805.9  0.0  0.0    0.0    0.9   0   3
> c1t5000C50055F9FE87d0
>     3.0   15.4   72.6  795.0  0.0  0.0    0.0    1.6   0   3
> c1t5000C50055F9F91Bd0
>     2.5   38.1   61.1  859.4  0.0  0.1    0.0    1.6   0   5
> c1t5000C50055F9FEABd0
>     2.1   13.2   42.7  801.6  0.0  0.0    0.0    1.6   0   2
> c1t5000C50055F9F63Bd0
>     3.0   20.0   62.6  766.6  0.0  0.0    0.0    1.1   0   2
> c1t5000C50055F9F3EBd0
>     3.7   24.3   80.2  807.9  0.0  0.0    0.0    1.0   0   2
> c1t5000C50055F9F80Bd0
>     3.2   35.2   66.1  852.4  0.0  0.0    0.0    1.2   0   4
> c1t5000C50055F9FB8Bd0
>     3.9   30.6   84.7  845.7  0.0  0.0    0.0    0.8   0   3
> c1t5000C50055F9F92Bd0
>     2.7   18.1   68.8  831.4  0.0  0.0    0.0    1.4   0   2
> c1t5000C50055F8905Fd0
>     2.7   17.7   61.4  762.1  0.0  0.0    0.0    1.0   0   2
> c1t5000C50055F8D48Fd0
>     3.5   17.5   87.8  749.7  0.0  0.0    0.0    1.7   0   3
> c1t5000C50055F9F89Fd0
>     2.6   13.7   58.6  780.9  0.0  0.0    0.0    1.7   0   3
> c1t5000C50055F9EF2Fd0
>     3.3   34.9   74.5  730.9  0.0  0.0    0.0    0.8   0   3
> c1t5000C50055F8C3ABd0
>     3.1   19.3   64.7  789.9  0.0  0.0    0.0    1.0   0   2
> c1t5000C50055E66053d0
>     3.8   38.5   82.9  826.8  0.0  0.1    0.0    1.3   0   4
> c1t5000C50055E66503d0
>     3.7   25.8   91.4  813.3  0.0  0.0    0.0    0.8   0   2
> c1t5000C50055F9D3E3d0
>     2.2   37.9   52.5  859.4  0.0  0.0    0.0    1.1   0   4
> c1t5000C50055F84FB7d0
>     2.8   20.0   62.8  766.6  0.0  0.0    0.0    1.0   0   2
> c1t5000C50055F8E017d0
>     3.9   26.1   86.5  824.3  0.0  0.0    0.0    1.1   0   3
> c1t5000C50055E579F7d0
>     3.1   27.7   79.9  765.8  0.0  0.0    0.0    1.2   0   3
> c1t5000C50055E65807d0
>     2.9   22.8   76.3  778.0  0.0  0.0    0.0    1.1   0   3
> c1t5000C50055F84A97d0
>     3.6   15.3   89.0  783.4  0.0  0.0    0.0    1.7   0   3
> c1t5000C50055F87D97d0
>     2.8   13.8   77.9  780.9  0.0  0.0    0.0    1.5   0   2
> c1t5000C50055F9F637d0
>     2.1   18.3   51.4  831.4  0.0  0.0    0.0    1.1   0   2
> c1t5000C50055E65ABBd0
>     3.1   15.4   70.9  822.3  0.0  0.0    0.0    1.2   0   2
> c1t5000C50055F8BF9Bd0
>     3.2   17.9   75.5  762.1  0.0  0.0    0.0    1.2   0   2
> c1t5000C50055F8A22Bd0
>     3.7   42.4   83.3  834.1  0.0  0.1    0.0    1.1   0   5
> c1t5000C50055F9379Bd0
>     4.0   22.7   86.8  870.9  0.0  0.0    0.0    1.0   0   2
> c1t5000C50055E57A5Fd0
>     2.6   15.5   67.5  795.0  0.0  0.0    0.0    1.4   0   2
> c1t5000C50055F8CCAFd0
>     2.9   13.2   65.4  801.6  0.0  0.0    0.0    1.9   0   3
> c1t5000C50055F8B80Fd0
>     3.3   25.7   82.7  719.7  0.0  0.0    0.0    1.1   0   3
> c1t5000C50055F9FA1Fd0
>     4.0   24.0   84.9  807.9  0.0  0.0    0.0    1.1   0   3
> c1t5000C50055E65F0Fd0
>     2.8   18.4   69.5  767.0  0.0  0.0    0.0    1.0   0   2
> c1t5000C50055F8BE3Fd0
>     3.3   17.6   81.6  749.7  0.0  0.0    0.0    1.4   0   3
> c1t5000C50055F8B21Fd0
>     3.3   35.1   64.2  852.4  0.0  0.0    0.0    1.1   0   4
> c1t5000C50055F8A46Fd0
>     3.5   30.0   82.1  805.9  0.0  0.0    0.0    0.9   0   3
> c1t5000C50055F856CFd0
>     3.9   30.4   89.5  845.7  0.0  0.0    0.0    0.9   0   3
> c1t5000C50055E6606Fd0
>   429.4  133.6 5933.0 8163.0  0.0  0.2    0.0    0.3   0  12 c2
>   215.8   28.4 2960.4  677.7  0.0  0.1    0.0    0.2   0   5
> c2t500117310015D579d0
>   213.7   27.4 2972.6  654.1  0.0  0.1    0.0    0.2   0   5
> c2t50011731001631FDd0
>     0.0   77.8    0.0 6831.2  0.0  0.1    0.0    0.8   0   2
> c2t5000A72A3007811Dd0
>     0.0   12.3    0.0   46.8  0.0  0.0    0.0    0.0   0   0 c4
>     0.0    6.2    0.0   23.4  0.0  0.0    0.0    0.0   0   0 c4t0d0
>     0.0    6.1    0.0   23.4  0.0  0.0    0.0    0.0   0   0 c4t1d0
>   418.4  134.8 5663.1 8197.8  0.0  0.2    0.0    0.3   0  11 c12
>     0.0   77.8    0.0 6831.2  0.0  0.1    0.0    0.8   0   2
> c12t5000A72B300780FFd0
>   203.5   29.7 2738.0  715.8  0.0  0.1    0.0    0.2   0   5
> c12t500117310015D59Ed0
>   214.9   27.2 2925.2  650.8  0.0  0.1    0.0    0.2   0   5
> c12t500117310015D54Ed0
>     0.0   11.3    0.0   46.8  0.0  0.0    0.7    0.1   0   0 rpool
>  1006.7 1478.2 15313.9 56330.7 30.4  2.0   12.2    0.8   6  64 tank
>
> Seems the pool is busy at 64% but the individual drives are not taxed at
> all (this load is virtually identical to when the scrub was running before
> the resilver was triggered). Still not too sure how to interpret this data.
> Is the system over-stressed? Is there really a bottleneck somewhere, or
> just need to fine tune some settings?
>
> Going to try some iometer results via the VM I/O Analyzer.
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141203/e4be543e/attachment-0001.html>

From richard.elling at richardelling.com  Wed Dec  3 06:27:59 2014
From: richard.elling at richardelling.com (Richard Elling)
Date: Tue, 2 Dec 2014 22:27:59 -0800
Subject: [OmniOS-discuss] Slow scrub performance
In-Reply-To: <CA+tR_KxhWH95EoAgOtkMp6qkTMXMsusgkP87VhO4g6rhqJSx+A@mail.gmail.com>
References: <CA+tR_KxTmft+hK9U8HLgqmPOdZf0+9q67+Kg9p1JwX6sg5drAA@mail.gmail.com>
	<C7904250-3A94-4D4B-8D62-D49FDE808008@richardelling.com>
	<CA+tR_KwX_1HN4tVa+-ZOFJk2mN7RE-nFh31sMcTNo7TJJjfyLg@mail.gmail.com>
	<CA+tR_KxSt3VVeUVZAOL4FxWsRL3M-PwpBF6JbHVXv5Xx66Jozg@mail.gmail.com>
	<BE32B046-05EC-43D2-AC46-7649FAC34C2A@richardelling.com>
	<CA+tR_KyPZs38vOoSZjN5ob7GS2j=V-ZvCuY_abgXpqS62UjWPA@mail.gmail.com>
	<CA+tR_KxhWH95EoAgOtkMp6qkTMXMsusgkP87VhO4g6rhqJSx+A@mail.gmail.com>
Message-ID: <A270CBE0-2078-4A7B-AD77-4B6D9C397DB7@richardelling.com>


On Dec 2, 2014, at 10:11 PM, wuffers <moo at wuffers.net> wrote:

> Just wanted to update this thread with my results. I had the scrub started since late July, and checked on it at the end of Oct:
> 
>   pool: tank
>  state: ONLINE
>   scan: scrub in progress since Tue Jul 29 15:41:27 2014
>     7.65T scanned out of 24.5T at 1024K/s, (scan is slow, no estimated time)
>     0 repaired, 31.16% done
> 
> So it was scrubbing away for ~90 days and got a bit over 31% with vdev_scrub_max set to 5.
> 
> In anticipation of having to power off the head nodes for the building's power maintenance and not wanting to waste the scrub so far, I pushed the vdev_scrub_max to 10. The scrub then proceeded to accelerate to about 8% per day, and ended:
> 
>   pool: tank
>  state: ONLINE
>   scan: scrub repaired 0 in 2372h54m with 0 errors on Wed Nov  5 11:36:00 2014
> 
> My pool wasn't busy so I was okay with it having so much time slice, but this obviously isn't the best thing to do. Is this really normal, or is there something wrong with my config?

The default settings for zfs_vdev_scrub_max_active are (mostly) a guess. It is expected
that you can tune. It would not be surprising for the default to change as we get more,
varied field experience. Thanks for sharing your results!
 -- richard

> 
> On Thu, Jul 31, 2014 at 3:44 PM, wuffers <moo at wuffers.net> wrote:
> This is going to be long winded as well (apologies!).. lots of pasted data.
> 
> On Thu, Jul 31, 2014 at 1:37 AM, Richard Elling <richard.elling at richardelling.com> wrote:
> 
> The %busy for controllers is a sum of the %busy for all disks on the controller, so
> is can be large, but overall isn't interesting. With HDDs, there is no way you can 
> saturate the controller, so we don't really care what the %busy shows.
> 
> The more important item is that the number of read ops is fairly low for all but 4 disks.
> Since you didn't post the pool configuration, we can only guess that they might be a
> souce of the bottleneck. 
> 
> You're seeing a lot of reads from the cache devices. How much RAM does this system
> have?
> 
> 
> I realized that the busy % was a sum after looking through some of that data, but good to know that it isn't very relevant. 
> 
> The pool configuration was in the original post, but here it is again (after re-attaching the mirror log device). Just saw your edit, but this has been updated from the original post anyways.
> 
>   pool: tank
>  state: ONLINE
>   scan: scrub in progress since Tue Jul 29 15:41:27 2014
>     82.5G scanned out of 24.5T at 555K/s, (scan is slow, no estimated time)
>     0 repaired, 0.33% done
> config:
> 
>         NAME                        STATE     READ WRITE CKSUM
>         tank                        ONLINE       0     0     0
>           mirror-0                  ONLINE       0     0     0
>             c1t5000C50055F9F637d0   ONLINE       0     0     0
>             c1t5000C50055F9EF2Fd0   ONLINE       0     0     0
>           mirror-1                  ONLINE       0     0     0
>             c1t5000C50055F87D97d0   ONLINE       0     0     0
>             c1t5000C50055F9D3B3d0   ONLINE       0     0     0
>           mirror-2                  ONLINE       0     0     0
>             c1t5000C50055E6606Fd0   ONLINE       0     0     0
>             c1t5000C50055F9F92Bd0   ONLINE       0     0     0
>           mirror-3                  ONLINE       0     0     0
>             c1t5000C50055F856CFd0   ONLINE       0     0     0
>             c1t5000C50055F9FE87d0   ONLINE       0     0     0
>           mirror-4                  ONLINE       0     0     0
>             c1t5000C50055F84A97d0   ONLINE       0     0     0
>             c1t5000C50055FA0AF7d0   ONLINE       0     0     0
>           mirror-5                  ONLINE       0     0     0
>             c1t5000C50055F9D3E3d0   ONLINE       0     0     0
>             c1t5000C50055F9F0B3d0   ONLINE       0     0     0
>           mirror-6                  ONLINE       0     0     0
>             c1t5000C50055F8A46Fd0   ONLINE       0     0     0
>             c1t5000C50055F9FB8Bd0   ONLINE       0     0     0
>           mirror-7                  ONLINE       0     0     0
>             c1t5000C50055F8B21Fd0   ONLINE       0     0     0
>             c1t5000C50055F9F89Fd0   ONLINE       0     0     0
>           mirror-8                  ONLINE       0     0     0
>             c1t5000C50055F8BE3Fd0   ONLINE       0     0     0
>             c1t5000C50055F9E123d0   ONLINE       0     0     0
>           mirror-9                  ONLINE       0     0     0
>             c1t5000C50055F9379Bd0   ONLINE       0     0     0
>             c1t5000C50055F9E7D7d0   ONLINE       0     0     0
>           mirror-10                 ONLINE       0     0     0
>             c1t5000C50055E65F0Fd0   ONLINE       0     0     0
>             c1t5000C50055F9F80Bd0   ONLINE       0     0     0
>           mirror-11                 ONLINE       0     0     0
>             c1t5000C50055F8A22Bd0   ONLINE       0     0     0
>             c1t5000C50055F8D48Fd0   ONLINE       0     0     0
>           mirror-12                 ONLINE       0     0     0
>             c1t5000C50055E65807d0   ONLINE       0     0     0
>             c1t5000C50055F8BFA3d0   ONLINE       0     0     0
>           mirror-13                 ONLINE       0     0     0
>             c1t5000C50055E579F7d0   ONLINE       0     0     0
>             c1t5000C50055E65877d0   ONLINE       0     0     0
>           mirror-14                 ONLINE       0     0     0
>             c1t5000C50055F9FA1Fd0   ONLINE       0     0     0
>             c1t5000C50055F8CDA7d0   ONLINE       0     0     0
>           mirror-15                 ONLINE       0     0     0
>             c1t5000C50055F8BF9Bd0   ONLINE       0     0     0
>             c1t5000C50055F9A607d0   ONLINE       0     0     0
>           mirror-16                 ONLINE       0     0     0
>             c1t5000C50055E66503d0   ONLINE       0     0     0
>             c1t5000C50055E4FDE7d0   ONLINE       0     0     0
>           mirror-17                 ONLINE       0     0     0
>             c1t5000C50055F8E017d0   ONLINE       0     0     0
>             c1t5000C50055F9F3EBd0   ONLINE       0     0     0
>           mirror-18                 ONLINE       0     0     0
>             c1t5000C50055F8B80Fd0   ONLINE       0     0     0
>             c1t5000C50055F9F63Bd0   ONLINE       0     0     0
>           mirror-19                 ONLINE       0     0     0
>             c1t5000C50055F84FB7d0   ONLINE       0     0     0
>             c1t5000C50055F9FEABd0   ONLINE       0     0     0
>           mirror-20                 ONLINE       0     0     0
>             c1t5000C50055F8CCAFd0   ONLINE       0     0     0
>             c1t5000C50055F9F91Bd0   ONLINE       0     0     0
>           mirror-21                 ONLINE       0     0     0
>             c1t5000C50055E65ABBd0   ONLINE       0     0     0
>             c1t5000C50055F8905Fd0   ONLINE       0     0     0
>           mirror-22                 ONLINE       0     0     0
>             c1t5000C50055E57A5Fd0   ONLINE       0     0     0
>             c1t5000C50055F87E73d0   ONLINE       0     0     0
>           mirror-23                 ONLINE       0     0     0
>             c1t5000C50055E66053d0   ONLINE       0     0     0
>             c1t5000C50055E66B63d0   ONLINE       0     0     0
>           mirror-24                 ONLINE       0     0     0
>             c1t5000C50055F8723Bd0   ONLINE       0     0     0
>             c1t5000C50055F8C3ABd0   ONLINE       0     0     0
>         logs
>           mirror-25                 ONLINE       0     0     0
>             c2t5000A72A3007811Dd0   ONLINE       0     0     0
>             c12t5000A72B300780FFd0  ONLINE       0     0     0
>         cache
>           c2t500117310015D579d0     ONLINE       0     0     0
>           c2t50011731001631FDd0     ONLINE       0     0     0
>           c12t500117310015D59Ed0    ONLINE       0     0     0
>           c12t500117310015D54Ed0    ONLINE       0     0     0
>         spares
>           c1t5000C50055FA2AEFd0     AVAIL
>           c1t5000C50055E595B7d0     AVAIL
> 
> Basically, this is 2 head nodes (Supermicro 826BE26) connected to a Supermicro 847E26 JBOD, using LSI 9207s. There are 52 Seagate ST4000NM0023s (4TB SAS drives) in 25 mirror pairs plus 2 which are spares. There are 4 Smart Optimus 400GB SSDs as cache drives, and 2 Stec ZeusRAMs for slogs. They're wired in such a way that both nodes can see all the drives (data, cache and log), and the data drives are on separate controllers than the cache/slog devices. RSF-1 was also specced in here but not in use at the moment. All the SAN traffic is through InfiniBand (SRP). Each head unit has 256GB of RAM. Dedupe is not in use and all the latest feature flags are enabled.
>  
> An arc_summary output:
> 
> System Memory:
>          Physical RAM:  262103 MB
>          Free Memory :  10273 MB
>          LotsFree:      4095 MB
> 
> ZFS Tunables (/etc/system):
>          set zfs:zfs_arc_shrink_shift = 10
> 
> ARC Size:
>          Current Size:             225626 MB (arcsize)
>          Target Size (Adaptive):   225626 MB (c)
>          Min Size (Hard Limit):    8190 MB (zfs_arc_min)
>          Max Size (Hard Limit):    261079 MB (zfs_arc_max)
> 
> ARC Size Breakdown:
>          Most Recently Used Cache Size:          10%    23290 MB (p)
>          Most Frequently Used Cache Size:        89%    202335 MB (c-p)
> 
> ARC Efficency:
>          Cache Access Total:             27377320465
>          Cache Hit Ratio:      93%       25532510784    [Defined State for buffer]
>          Cache Miss Ratio:      6%       1844809681     [Undefined State for Buffer]
>          REAL Hit Ratio:       92%       25243933796    [MRU/MFU Hits Only]
> 
>          Data Demand   Efficiency:    95%
>          Data Prefetch Efficiency:    40%
> 
>         CACHE HITS BY CACHE LIST:
>           Anon:                       --%        Counter Rolled.
>           Most Recently Used:         18%        4663226393 (mru)       [ Return Customer ]
>           Most Frequently Used:       80%        20580707403 (mfu)              [ Frequent Customer ]
>           Most Recently Used Ghost:    0%        176686906 (mru_ghost)  [ Return Customer Evicted, Now Back ]
>           Most Frequently Used Ghost:  0%        126286869 (mfu_ghost)  [ Frequent Customer Evicted, Now Back ]
>         CACHE HITS BY DATA TYPE:
>           Demand Data:                95%        24413671342
>           Prefetch Data:               1%        358419705
>           Demand Metadata:             2%        698314899
>           Prefetch Metadata:           0%        62104838
>         CACHE MISSES BY DATA TYPE:
>           Demand Data:                69%        1277347273
>           Prefetch Data:              28%        519579788
>           Demand Metadata:             2%        39512363
>           Prefetch Metadata:           0%        8370257
> 
> And a sample of arcstat (deleted first line of output):
> 
> # arcstat -f read,hits,miss,hit%,l2read,l2hits,l2miss,l2hit%,arcsz,l2size,l2asize 1
> 
> read  hits  miss  hit%  l2read  l2hits  l2miss  l2hit%  arcsz  l2size  l2asize
> 5.9K  4.6K  1.3K    78    1.3K    1.2K      80      93   220G    1.6T     901G
> 6.7K  5.2K  1.5K    76    1.5K    1.3K     250      83   220G    1.6T     901G
> 7.0K  5.3K  1.7K    76    1.7K    1.4K     316      81   220G    1.6T     901G
> 6.5K  5.3K  1.2K    80    1.2K    1.1K     111      91   220G    1.6T     901G
> 6.4K  5.2K  1.2K    81    1.2K    1.1K     100      91   220G    1.6T     901G
> 7.2K  5.6K  1.6K    78    1.6K    1.3K     289      81   220G    1.6T     901G
> 8.5K  6.8K  1.7K    80    1.7K    1.3K     351      79   220G    1.6T     901G
> 7.5K  5.9K  1.6K    78    1.6K    1.3K     282      82   220G    1.6T     901G
> 6.7K  5.6K  1.1K    83    1.1K     991     123      88   220G    1.6T     901G
> 6.8K  5.5K  1.3K    80    1.3K    1.1K     234      82   220G    1.6T     901G
> 
> Interesting to see only an l2asize of 901G even though I should have more.. 373G x 4 is just under 1.5TB of raw storage. The compressed l2arc size is 1.6TB, while actual used space is 901G. I expect more to be used. Perhaps Saso can comment on this portion, if he's following this thread (snipped from "zpool iostat -v"):
> 
> cache                           -      -      -      -      -      -
>   c2t500117310015D579d0      373G     8M    193     16  2.81M   394K
>   c2t50011731001631FDd0      373G  5.29M    194     15  2.85M   360K
>   c12t500117310015D59Ed0     373G  5.50M    191     17  2.74M   368K
>   c12t500117310015D54Ed0     373G  5.57M    200     14  2.89M   300K
> 
> (from this discussion here: http://lists.omniti.com/pipermail/omnios-discuss/2014-February/002287.html), and the uptime on this is currently around ~58 days, so it should have had enough time to rotate through the l2arc "rotor".
> 
> 
> methinks the scrub I/Os are getting starved and since they are low priority, they 
> could get very starved. In general, I wouldn't worry about it, but I understand 
> why you might be nervous. Keep in mind that in ZFS scrubs are intended to find 
> errors on idle data, not frequently accessed data.
> 
> more far below...
> 
> 
> I'm worried because there's no way the scrub will ever complete before the next reboot. Regular scrubs are important, right?
> 
> 
> ok, so the pool is issuing 720 read iops, including resilver workload, vs 1298 write iops.
> There is plenty of I/O capacity left on the table here, as you can see by the %busy being
> so low.
> 
> So I think the pool is not scheduling scrub I/Os very well. You can increase the number of
> scrub I/Os in the scheduler by adjusting the zfs_vdev_scrub_max_active tunable. The
> default is 2, but you'll have to consider that a share (in the stock market sense) where
> the active sync reads and writes are getting 10 each. You can try bumping up the value
> and see what happens over some time, perhaps 10 minutes or so -- too short of a time
> and you won't get a good feeling for the impact (try this in off-peak time).
> 	echo zfs_vdev_scrub_max_active/W0t5 | mdb -kw
> will change the value from 2 to 5, increasing its share of the total I/O workload.
> 
> You can see the progress of scan (scrubs do scan) workload by looking at the ZFS
> debug messages.
> 	echo ::zfs_dbgmsg | mdb -k
> These will look mysterious... they are. But the interesting bits are about how many blocks
> are visited in some amount of time (txg sync interval). Ideally, this will change as you 
> adjust zfs_vdev_scrub_max_active.
>  -- richard
> 
> 
> Actually, you used the data from before the resilver. During resilver this was the activity on the pool:
> 
> 3883.5 1357.7 40141.6 60739.5 22.8 38.6    4.4    7.4  54 100 tank
> 
> Are you looking at an individual drive's busy % or the pool's busy % to determine whether it's "busy"? During the resilver this was the activity on the drives (a sample - between 38-45%, whereas during the scrub the individual drives were 2-5% busy):
> 
>    59.5   30.0  553.2  741.8  0.0  0.9    0.0    9.8   0  45 c1t5000C50055F8A46Fd0
>    57.4   22.5  504.0  724.5  0.0  0.8    0.0    9.6   0  41 c1t5000C50055F856CFd0
>    58.4   24.6  531.4  786.9  0.0  0.7    0.0    8.4   0  38 c1t5000C50055E6606Fd0
> 
> But yes, without the resilver the busy % was much less (during the scrub each individual drive was 2-4% busy). I've pasted the current iostat output further below.
> 
> With the zfs_vdev_scrub_max_active at the default of 2, it was doing an average of 162 blocks:
> 
> doing scan sync txg 26678243; bm=897/1/0/15785978
> scanned dataset 897 (tank/vmware-64k-5tb-1) with min=1 max=26652167; pausing=1
> visited 162 blocks in 6090ms
> doing scan sync txg 26678244; bm=897/1/0/15786126
> scanned dataset 897 (tank/vmware-64k-5tb-1) with min=1 max=26652167; pausing=1
> visited 162 blocks in 6094ms
> 
> After changing it to 5, and waiting about 20 mins, I'm not seeing anything significantly different:
> 
> doing scan sync txg 26678816; bm=897/1/0/37082043
> scanned dataset 897 (tank/vmware-64k-5tb-1) with min=1 max=26652167; pausing=1
> visited 163 blocks in 6154ms
> doing scan sync txg 26678817; bm=897/1/0/37082193
> scanned dataset 897 (tank/vmware-64k-5tb-1) with min=1 max=26652167; pausing=1
> visited 162 blocks in 6128ms
> 
>   pool: tank
>  state: ONLINE
>   scan: scrub in progress since Tue Jul 29 15:41:27 2014
>     97.0G scanned out of 24.5T at 599K/s, (scan is slow, no estimated time)
>     0 repaired, 0.39% done
> 
> I'll keep the zfs_vdev_scrub_max_active tunable to 5, as it doesn't appear to be impacting too much, and monitor for changes. What's strange to me is that it was "humming" along at 5.5MB/s at the 2 week mark but is now 10x slower (compared to before reattaching the mirror log device). It *seems* marginally faster, from 541K/s to almost 600K/s..
> 
> This is the current activity from "iostat -xnCz 60 2":
> 
>                     extended device statistics
>     r/s    w/s   kr/s   kw/s wait actv wsvc_t asvc_t  %w  %b device
>   158.8 1219.2 3717.8 39969.8  0.0  1.6    0.0    1.1   0 139 c1
>     3.6   35.1   86.2  730.9  0.0  0.0    0.0    0.9   0   3 c1t5000C50055F8723Bd0
>     3.7   19.9   83.7  789.9  0.0  0.0    0.0    1.4   0   3 c1t5000C50055E66B63d0
>     2.7   22.5   60.8  870.9  0.0  0.0    0.0    1.1   0   2 c1t5000C50055F87E73d0
>     2.4   27.9   66.0  765.8  0.0  0.0    0.0    0.8   0   2 c1t5000C50055F8BFA3d0
>     2.8   17.9   64.9  767.0  0.0  0.0    0.0    0.8   0   1 c1t5000C50055F9E123d0
>     3.1   26.1   73.8  813.3  0.0  0.0    0.0    0.9   0   2 c1t5000C50055F9F0B3d0
>     3.1   15.5   79.4  783.4  0.0  0.0    0.0    1.3   0   2 c1t5000C50055F9D3B3d0
>     3.8   38.6   86.2  826.8  0.0  0.1    0.0    1.2   0   4 c1t5000C50055E4FDE7d0
>     3.8   15.4   93.0  822.3  0.0  0.0    0.0    1.5   0   3 c1t5000C50055F9A607d0
>     3.0   25.7   79.4  719.7  0.0  0.0    0.0    0.9   0   2 c1t5000C50055F8CDA7d0
>     3.2   26.5   69.0  824.3  0.0  0.0    0.0    1.1   0   3 c1t5000C50055E65877d0
>     3.7   42.6   79.2  834.1  0.0  0.1    0.0    1.3   0   5 c1t5000C50055F9E7D7d0
>     3.3   23.2   79.5  778.0  0.0  0.0    0.0    1.2   0   3 c1t5000C50055FA0AF7d0
>     3.4   30.2   77.0  805.9  0.0  0.0    0.0    0.9   0   3 c1t5000C50055F9FE87d0
>     3.0   15.4   72.6  795.0  0.0  0.0    0.0    1.6   0   3 c1t5000C50055F9F91Bd0
>     2.5   38.1   61.1  859.4  0.0  0.1    0.0    1.6   0   5 c1t5000C50055F9FEABd0
>     2.1   13.2   42.7  801.6  0.0  0.0    0.0    1.6   0   2 c1t5000C50055F9F63Bd0
>     3.0   20.0   62.6  766.6  0.0  0.0    0.0    1.1   0   2 c1t5000C50055F9F3EBd0
>     3.7   24.3   80.2  807.9  0.0  0.0    0.0    1.0   0   2 c1t5000C50055F9F80Bd0
>     3.2   35.2   66.1  852.4  0.0  0.0    0.0    1.2   0   4 c1t5000C50055F9FB8Bd0
>     3.9   30.6   84.7  845.7  0.0  0.0    0.0    0.8   0   3 c1t5000C50055F9F92Bd0
>     2.7   18.1   68.8  831.4  0.0  0.0    0.0    1.4   0   2 c1t5000C50055F8905Fd0
>     2.7   17.7   61.4  762.1  0.0  0.0    0.0    1.0   0   2 c1t5000C50055F8D48Fd0
>     3.5   17.5   87.8  749.7  0.0  0.0    0.0    1.7   0   3 c1t5000C50055F9F89Fd0
>     2.6   13.7   58.6  780.9  0.0  0.0    0.0    1.7   0   3 c1t5000C50055F9EF2Fd0
>     3.3   34.9   74.5  730.9  0.0  0.0    0.0    0.8   0   3 c1t5000C50055F8C3ABd0
>     3.1   19.3   64.7  789.9  0.0  0.0    0.0    1.0   0   2 c1t5000C50055E66053d0
>     3.8   38.5   82.9  826.8  0.0  0.1    0.0    1.3   0   4 c1t5000C50055E66503d0
>     3.7   25.8   91.4  813.3  0.0  0.0    0.0    0.8   0   2 c1t5000C50055F9D3E3d0
>     2.2   37.9   52.5  859.4  0.0  0.0    0.0    1.1   0   4 c1t5000C50055F84FB7d0
>     2.8   20.0   62.8  766.6  0.0  0.0    0.0    1.0   0   2 c1t5000C50055F8E017d0
>     3.9   26.1   86.5  824.3  0.0  0.0    0.0    1.1   0   3 c1t5000C50055E579F7d0
>     3.1   27.7   79.9  765.8  0.0  0.0    0.0    1.2   0   3 c1t5000C50055E65807d0
>     2.9   22.8   76.3  778.0  0.0  0.0    0.0    1.1   0   3 c1t5000C50055F84A97d0
>     3.6   15.3   89.0  783.4  0.0  0.0    0.0    1.7   0   3 c1t5000C50055F87D97d0
>     2.8   13.8   77.9  780.9  0.0  0.0    0.0    1.5   0   2 c1t5000C50055F9F637d0
>     2.1   18.3   51.4  831.4  0.0  0.0    0.0    1.1   0   2 c1t5000C50055E65ABBd0
>     3.1   15.4   70.9  822.3  0.0  0.0    0.0    1.2   0   2 c1t5000C50055F8BF9Bd0
>     3.2   17.9   75.5  762.1  0.0  0.0    0.0    1.2   0   2 c1t5000C50055F8A22Bd0
>     3.7   42.4   83.3  834.1  0.0  0.1    0.0    1.1   0   5 c1t5000C50055F9379Bd0
>     4.0   22.7   86.8  870.9  0.0  0.0    0.0    1.0   0   2 c1t5000C50055E57A5Fd0
>     2.6   15.5   67.5  795.0  0.0  0.0    0.0    1.4   0   2 c1t5000C50055F8CCAFd0
>     2.9   13.2   65.4  801.6  0.0  0.0    0.0    1.9   0   3 c1t5000C50055F8B80Fd0
>     3.3   25.7   82.7  719.7  0.0  0.0    0.0    1.1   0   3 c1t5000C50055F9FA1Fd0
>     4.0   24.0   84.9  807.9  0.0  0.0    0.0    1.1   0   3 c1t5000C50055E65F0Fd0
>     2.8   18.4   69.5  767.0  0.0  0.0    0.0    1.0   0   2 c1t5000C50055F8BE3Fd0
>     3.3   17.6   81.6  749.7  0.0  0.0    0.0    1.4   0   3 c1t5000C50055F8B21Fd0
>     3.3   35.1   64.2  852.4  0.0  0.0    0.0    1.1   0   4 c1t5000C50055F8A46Fd0
>     3.5   30.0   82.1  805.9  0.0  0.0    0.0    0.9   0   3 c1t5000C50055F856CFd0
>     3.9   30.4   89.5  845.7  0.0  0.0    0.0    0.9   0   3 c1t5000C50055E6606Fd0
>   429.4  133.6 5933.0 8163.0  0.0  0.2    0.0    0.3   0  12 c2
>   215.8   28.4 2960.4  677.7  0.0  0.1    0.0    0.2   0   5 c2t500117310015D579d0
>   213.7   27.4 2972.6  654.1  0.0  0.1    0.0    0.2   0   5 c2t50011731001631FDd0
>     0.0   77.8    0.0 6831.2  0.0  0.1    0.0    0.8   0   2 c2t5000A72A3007811Dd0
>     0.0   12.3    0.0   46.8  0.0  0.0    0.0    0.0   0   0 c4
>     0.0    6.2    0.0   23.4  0.0  0.0    0.0    0.0   0   0 c4t0d0
>     0.0    6.1    0.0   23.4  0.0  0.0    0.0    0.0   0   0 c4t1d0
>   418.4  134.8 5663.1 8197.8  0.0  0.2    0.0    0.3   0  11 c12
>     0.0   77.8    0.0 6831.2  0.0  0.1    0.0    0.8   0   2 c12t5000A72B300780FFd0
>   203.5   29.7 2738.0  715.8  0.0  0.1    0.0    0.2   0   5 c12t500117310015D59Ed0
>   214.9   27.2 2925.2  650.8  0.0  0.1    0.0    0.2   0   5 c12t500117310015D54Ed0
>     0.0   11.3    0.0   46.8  0.0  0.0    0.7    0.1   0   0 rpool
>  1006.7 1478.2 15313.9 56330.7 30.4  2.0   12.2    0.8   6  64 tank
> 
> Seems the pool is busy at 64% but the individual drives are not taxed at all (this load is virtually identical to when the scrub was running before the resilver was triggered). Still not too sure how to interpret this data. Is the system over-stressed? Is there really a bottleneck somewhere, or just need to fine tune some settings?
> 
> Going to try some iometer results via the VM I/O Analyzer.
> 
> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141202/e622f0fc/attachment-0001.html>

From richard.elling at richardelling.com  Wed Dec  3 06:31:23 2014
From: richard.elling at richardelling.com (Richard Elling)
Date: Tue, 2 Dec 2014 22:31:23 -0800
Subject: [OmniOS-discuss] Bad ZeusRAM? How to tell if another component
	is causing issues?
In-Reply-To: <CA+tR_Kyxzf6XEczbgaxCsvKdtJ74C-+_0SkMjs=rmDufiCAJsQ@mail.gmail.com>
References: <CA+tR_Kyxzf6XEczbgaxCsvKdtJ74C-+_0SkMjs=rmDufiCAJsQ@mail.gmail.com>
Message-ID: <5601F3FA-4BD6-4E2B-B365-8640605BAECC@richardelling.com>


On Dec 2, 2014, at 10:02 PM, wuffers <moo at wuffers.net> wrote:

> I'm at home just looking into the health of our SAN and came across a bunch of errors on the Stec ZeusRAM (in a mirrored log configuration):
> 
> # iostat -En
> c12t5000A72B300780FFd0 Soft Errors: 0 Hard Errors: 1 Transport Errors: 5224

ZeusRAMs are more sensitive to noisy fabric or cables than some other drives.
At the OS level, one symptom is transport errors, but that is only one view of the
fabric and you'll need more than one view to reach root cause.

Check the drive's health using something like sg3_utils:
	sg_logs -a /dev/rdsk/c#t#d#
and look for link stats, especially loss of DWORD sync and running disparity errors.
 -- richard

> Vendor: STEC     Product: ZeusRAM          Revision: C018 Serial No: STM000170C98
> Size: 8.00GB <8000000000 bytes>
> Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
> Illegal Request: 391 Predictive Failure Analysis: 0
> 
> #fmdump -eV
> Dec 03 2014 00:26:22.592888816 ereport.io.scsi.cmd.disk.recovered
> nvlist version: 0
>         class = ereport.io.scsi.cmd.disk.recovered
>         ena = 0xd38b237e7ed02001
>         detector = (embedded nvlist)
>         nvlist version: 0
>                 version = 0x0
>                 scheme = dev
>                 device-path = /pci at 0,0/pci8086,3c08 at 3/pci1000,3030 at 0/iport at f/disk at w5000a72b300780ff,0
>                 devid = id1,sd at n5000a720300780ff
>         (end detector)
> 
>         devid = id1,sd at n5000a720300780ff
>         driver-assessment = recovered
>         op-code = 0x2a
>         cdb = 0x2a 0x0 0x0 0x2d 0xda 0x0 0x0 0x0 0xf8 0x0
>         pkt-reason = 0x0
>         pkt-state = 0x1f
>         pkt-stats = 0x0
>         __ttl = 0x1
>         __tod = 0x547e9efe 0x2356c3f0
> 
> # dmesg
> Dec  3 00:28:24 san1 scsi: [ID 365881 kern.info] /pci at 0,0/pci8086,3c08 at 3/pci1000,3030 at 0 (mpt_sas1):
> Dec  3 00:28:24 san1    Log info 0x31120303 received for target 10 w5000a72b300780ff.
> Dec  3 00:28:24 san1    scsi_status=0x0, ioc_status=0x804b, scsi_state=0xc
> 
> from format:
> 
> 57. c12t5000A72B300780FFd0 <STEC-ZeusRAM-C018-7.45GB>
>           /pci at 0,0/pci8086,3c08 at 3/pci1000,3030 at 0/iport at f/disk at w5000a72b300780ff,0
> 
> Both fmdump and dmesg has these errors repeating over and over. Everything seems to point to the drive. I suppose I would have to physically move the drive to eliminate cable, backplane or controller issues. Is there another way to tell just by these error logs or is the physical test the way to go? 
> 
> Are logs enough to justify an RMA? 
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141202/52c8b070/attachment.html>

From piiv at zhaw.ch  Wed Dec  3 09:33:30 2014
From: piiv at zhaw.ch (Vincenzo Pii)
Date: Wed, 3 Dec 2014 10:33:30 +0100
Subject: [OmniOS-discuss]  packaging - unwanted directory group owner
Message-ID: <CALOiZS2bv2SZ4Ew6dhjyqHOG2LxbWy52NvspsWhAXUM=ygyBnw@mail.gmail.com>

I am on my way to package some source code for OmniOS.

For one package I have this problem, as seen from the res file:

    dir  path=etc/init.d owner=root group=bin mode=0755

This is changing the owner of the /etc/init.d dir to bin (should be sys)
and so conflicts with other packages.

If I try to install this, I get

    pkg install: The requested change to the system attempts to install
multiple actions
    for dir 'etc/init.d' with conflicting attributes:

    1 package delivers 'dir group=bin mode=0755 owner=root path=etc/init.d':
    [...]
    5 packages deliver 'dir group=sys mode=0755 owner=root path=etc/init.d':
    [...]

Now, I can't find where in the source code this change is happening (e.g.,
chgrp'ing or chown'ing the folder), but I don't even know if this "bin"
group comes from some default setting of the packaging template (so not
directly from one of the install scripts in the code).

My questions are:

1) Is there a way to manually edit the res file (<pkg_name>.p5m.int.3.res)
and just rebuild the package? The edit would consist in changing
"group=bin" to "group=sys" (I am 100% sure this won't be a problem for this
package).

2) If "group=bin" comes from some default, can I override this setting?

Many thanks!

-- 
Vincenzo Pii
Researcher, InIT Cloud Computing Lab
Zurich University of Applied Sciences (ZHAW)
blog.zhaw.ch/icclab
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141203/d68ffae8/attachment.html>

From piiv at zhaw.ch  Wed Dec  3 09:48:05 2014
From: piiv at zhaw.ch (Vincenzo Pii)
Date: Wed, 3 Dec 2014 10:48:05 +0100
Subject: [OmniOS-discuss] packaging - unwanted directory group owner
In-Reply-To: <03c0db4c2df34d0fb257362cba456703@SRV-MAIL-001.zhaw.ch>
References: <03c0db4c2df34d0fb257362cba456703@SRV-MAIL-001.zhaw.ch>
Message-ID: <CALOiZS2KUDXNsKgVLbUaH1GKWMG5M7KXpjGx4nzR-KKH4g9nBQ@mail.gmail.com>

Ok, problem solved :D!

I added a "local.mog" file to my build directory (besides build.sh), with
the following content:

    <transform dir  path=etc/init.d -> set group sys>

I realized I could do that by browsing the omnios-build repo, for the bash
package:

https://github.com/omniti-labs/omnios-build/blob/abbfd1697849f6f8a84be3f94317b317798236a1/build/bash/local.mog

2014-12-03 10:33 GMT+01:00 Pii Vincenzo Maria (piiv) <piiv at zhaw.ch>:

>  I am on my way to package some source code for OmniOS.
>
>  For one package I have this problem, as seen from the res file:
>
>      dir  path=etc/init.d owner=root group=bin mode=0755
>
>  This is changing the owner of the /etc/init.d dir to bin (should be sys)
> and so conflicts with other packages.
>
>  If I try to install this, I get
>
>      pkg install: The requested change to the system attempts to install
> multiple actions
>     for dir 'etc/init.d' with conflicting attributes:
>
>      1 package delivers 'dir group=bin mode=0755 owner=root
> path=etc/init.d':
>     [...]
>     5 packages deliver 'dir group=sys mode=0755 owner=root
> path=etc/init.d':
>     [...]
>
>  Now, I can't find where in the source code this change is happening
> (e.g., chgrp'ing or chown'ing the folder), but I don't even know if this
> "bin" group comes from some default setting of the packaging template (so
> not directly from one of the install scripts in the code).
>
>  My questions are:
>
>  1) Is there a way to manually edit the res file
> (<pkg_name>.p5m.int.3.res) and just rebuild the package? The edit would
> consist in changing "group=bin" to "group=sys" (I am 100% sure this won't
> be a problem for this package).
>
>  2) If "group=bin" comes from some default, can I override this setting?
>
>  Many thanks!
>
>  --
>  Vincenzo Pii
>  Researcher, InIT Cloud Computing Lab
> Zurich University of Applied Sciences (ZHAW)
> blog.zhaw.ch/icclab
>



-- 
Vincenzo Pii
Researcher, InIT Cloud Computing Lab
Zurich University of Applied Sciences (ZHAW)
blog.zhaw.ch/icclab
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141203/40592fc0/attachment.html>

From lotheac at iki.fi  Wed Dec  3 10:08:11 2014
From: lotheac at iki.fi (Lauri Tirkkonen)
Date: Wed, 3 Dec 2014 12:08:11 +0200
Subject: [OmniOS-discuss] packaging - unwanted directory group owner
In-Reply-To: <CALOiZS2bv2SZ4Ew6dhjyqHOG2LxbWy52NvspsWhAXUM=ygyBnw@mail.gmail.com>
References: <CALOiZS2bv2SZ4Ew6dhjyqHOG2LxbWy52NvspsWhAXUM=ygyBnw@mail.gmail.com>
Message-ID: <20141203100811.GA17394@gutsman.lotheac.fi>

On Wed, Dec 03 2014 10:33:30 +0100, Vincenzo Pii wrote:
> If I try to install this, I get
> 
>     pkg install: The requested change to the system attempts to install
> multiple actions
>     for dir 'etc/init.d' with conflicting attributes:
> 
>     1 package delivers 'dir group=bin mode=0755 owner=root path=etc/init.d':
>     [...]
>     5 packages deliver 'dir group=sys mode=0755 owner=root path=etc/init.d':
>     [...]

pkglint(1) will also warn you about this. I'm under the impression
pkglint was integrated into omnios-build:template, so you could also
catch this before publishing the package.

-- 
Lauri Tirkkonen | +358 50 5341376 | lotheac @ IRCnet

From sim.ple at live.nl  Wed Dec  3 10:37:31 2014
From: sim.ple at live.nl (Randy S)
Date: Wed, 3 Dec 2014 11:37:31 +0100
Subject: [OmniOS-discuss] test omnios on OI machine
In-Reply-To: <DUB124-W28749FC3157E8298D365CAE07B0@phx.gbl>
References: <DUB124-W4FAE9E8C04253565F247FE07A0@phx.gbl>, <,
	<13009DC9-7FA5-4938-8042-F2AF74DDC695@omniti.com>
	<>>, <DUB124-W31A653A65DB219489C3FBAE07A0@phx.gbl>, ,
	<63802B04-B4E9-4FC6-8059-EDAD86FBADBE@omniti.com>,
	<DUB124-W28749FC3157E8298D365CAE07B0@phx.gbl>
Message-ID: <DUB124-W15E9AE203C73CD4F0BBB3FE07B0@phx.gbl>


Anybody else had problems with two or more sas hba's in an omnios system ?
How was this solved?

From: sim.ple at live.nl
To: omnios-discuss at lists.omniti.com
Date: Wed, 3 Dec 2014 01:45:12 +0100
Subject: Re: [OmniOS-discuss] test omnios on OI machine




Hi, was indeed intended for the list.

dmesg | grep mptsas

gives nothing (no output). 

Hope you have an idea of what to do next. 

Going to test a current omnios with one hba by adding another to it to see what happens tomorrow.  

Will be testing with new Dell hardware and solarflare cards probably in the comming weeks also. If you are interrested, will keep you posted. 



> Subject: Re: [OmniOS-discuss] test omnios on OI machine
> From: danmcd at omniti.com
> Date: Tue, 2 Dec 2014 19:22:15 -0500
> CC: danmcd at omniti.com
> To: sim.ple at live.nl
> 
> Please share these with the community in the future, unless you're sharing confidential data of some sort.
> 
> 
> > On Dec 2, 2014, at 6:59 PM, Randy S <sim.ple at live.nl> wrote:
> > 
> > Hi Dan,
> > 
> > It's been a while...
> > 
> > I wanted to stay on r10 since that has been in use by quite a community for quite a while now. R12 is still new so usually unforseen hicups can be expected. (We are not frontrunners regarding systems intended for serious use  ;-) ). Have been testing omnios for quite a while on systems with one hba now. All was satisfactory. 
> > 
> > forgot to mention, I did do an fmdump -ev and saw 
> > Dec 02 23:32:37.1675 ereport.fs.zfs.vdev.open_failed       0x0077a03af6b00001
> > Dec 02 23:32:37.1675 ereport.fs.zfs.vdev.open_failed       0x0077a03af6b00001
> > Dec 02 23:35:17.6783 ereport.sensor.failure                0x02cd95a4cde05801
> > Dec 02 23:35:17.6868 ereport.sensor.failure                0x02cd9dbb53605801
> > 
> > I guess this is because the one controller cannot be found.
> 
> That's a fair assessment.
> 
> > dmesg | grep mpt_sa (done just now (not sleepy yet)) gives :
> > 
> > Dec  3 00:42:44 omni scsi: [ID 583861 kern.info] sd68 at mpt_sas3: unit-address w5000c50057ad9161,0: w5000c50057ad9161,0
> > Dec  3 00:42:44 omni scsi: [ID 583861 kern.info] sd69 at mpt_sas3: unit-address w5000c50057adb269,0: w5000c50057adb269,0
> > Dec  3 00:42:45 omni scsi: [ID 583861 kern.info] sd70 at mpt_sas3: unit-address w5000c50057adea91,0: w5000c50057adea91,0
> > Dec  3 00:42:45 omni scsi: [ID 583861 kern.info] ses2 at mpt_sas3: unit-address w5003048000b18d3d,0: w5003048000b18d3d,0
> > Dec  3 00:42:45 omni scsi: [ID 583861 kern.info] mpt_sas4 at mpt_sas0: scsi-iport 20
> > Dec  3 00:42:45 omni genunix: [ID 936769 kern.info] mpt_sas4 is /pci at 0,0/pci8086,3c04 at 2/pci1000,3030 at 0/iport at 20
> > Dec  3 00:42:45 omni genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,3c04 at 2/pci1000,3030 at 0/iport at 20 (mpt_sas4) online
> > Dec  3 00:42:45 omni scsi: [ID 583861 kern.info] sd71 at mpt_sas4: unit-address w5001517bb27f79f7,0: w5001517bb27f79f7,0
> > Dec  3 00:42:45 omni scsi: [ID 583861 kern.info] mpt_sas5 at mpt_sas0: scsi-iport v0
> > Dec  3 00:42:45 omni genunix: [ID 936769 kern.info] mpt_sas5 is /pci at 0,0/pci8086,3c04 at 2/pci1000,3030 at 0/iport at v0
> > Dec  3 00:42:45 omni genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,3c04 at 2/pci1000,3030 at 0/iport at v0 (mpt_sas5) online
> 
> Interesting... note that two mpt_sas controllers (4 and 5) came online.  I wonder why one isn't showing up?  Grep for "mptsas" as well, just in case?
> 
> Dan
> 
 		 	   		  

_______________________________________________
OmniOS-discuss mailing list
OmniOS-discuss at lists.omniti.com
http://lists.omniti.com/mailman/listinfo/omnios-discuss 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141203/8af2a351/attachment-0001.html>

From gate03 at landcroft.co.uk  Wed Dec  3 20:28:47 2014
From: gate03 at landcroft.co.uk (Michael Mounteney)
Date: Thu, 4 Dec 2014 06:28:47 +1000
Subject: [OmniOS-discuss] help with ipfilter rule for DHCP client
Message-ID: <20141204062847.6d1b24d9@punda-mlia>

Hello, I tightened up my ipfilter rules and somehow cut out DHCP.
Could someone who knows DHCP better than me please have a look and
tell me what's missing.  I pulled ipf.conf from an Oracle page so it
must be slightly different in some respect.

What I have now is

pass in on e1000g1 all
pass in on e1000g0 all
pass out on e1000g1 all
pass out on e1000g0 all

where:

e1000g0 is connected to my ISP's cable modem and picks up its address
via DHCP in the range 192.168.0.2 to 192.168.0.254
e1000g1 is the local network interface, statically set to 192.168.1.1

It was working after I changed the rules to those below, even after
restarting service ipfilter, but I suppose it didn't have to renew the
lease at that time.

Dec  3 16:47:34 world /sbin/dhcpagent[106]: [ID 490758 daemon.error] send_pkt_internal: cannot send REQUEST packet to server (will retry in 63 seconds): Network is unreachable
Dec  3 16:48:34 world ipf: [ID 774698 kern.info] IP Filter: v4.1.9, running.
Dec  3 16:48:39 world /sbin/dhcpagent[106]: [ID 778557 daemon.warning] configure_v4_lease: no IP broadcast specified for e1000g0, making best guess

The rules were as at http://pastebin.com/4aYyZhJ8 -- posted there to avoid a long message.

Can I also suggest that for the sake of anyone searching for this later, that you copy and repost any relevant lines from the rule set.

Thanks in anticipation, Michael.

From sim.ple at live.nl  Wed Dec  3 10:59:11 2014
From: sim.ple at live.nl (Randy S)
Date: Wed, 3 Dec 2014 11:59:11 +0100
Subject: [OmniOS-discuss] test omnios on OI machine
In-Reply-To: <DUB124-W15E9AE203C73CD4F0BBB3FE07B0@phx.gbl>
References: <DUB124-W4FAE9E8C04253565F247FE07A0@phx.gbl>,
	<,,<13009DC9-7FA5-4938-8042-F2AF74DDC695@omniti.com>,<>>,
	<DUB124-W31A653A65DB219489C3FBAE07A0@phx.gbl>,
	, , <63802B04-B4E9-4FC6-8059-EDAD86FBADBE@omniti.com>, ,
	<DUB124-W28749FC3157E8298D365CAE07B0@phx.gbl>,
	<DUB124-W15E9AE203C73CD4F0BBB3FE07B0@phx.gbl>
Message-ID: <DUB124-W12DFDF024A9F155D414B06E07B0@phx.gbl>

when I do 

cfgadm -al
Ap_Id                          Type         Receptacle   Occupant     Condition
c7                             scsi-sas     connected    configured   unknown
c7::dsk/c7t500000E118E31592d0  disk         connected    configured   unknown
c9                             scsi-sas     connected    configured   unknown
c9::dsk/c9t5000C50055B2A5B1d0  disk         connected    configured   unknown
c9::dsk/c9t5000C50055B2A9A9d0  disk         connected    configured   unknown
c9::dsk/c9t5000C50055B2A139d0  disk         connected    configured   unknown
c9::dsk/c9t5000C50055B2A245d0  disk         connected    configured   unknown
c9::dsk/c9t5000C50055B2B081d0  disk         connected    configured   unknown
c9::dsk/c9t5000C50055B2B249d0  disk         connected    configured   unknown
c9::dsk/c9t5000C50055B2B255d0  disk         connected    configured   unknown
c9::dsk/c9t5000C50055B2BD6Dd0  disk         connected    configured   unknown
c9::es/ses0                    ESI          connected    configured   unknown
c9::es/ses1                    ESI          connected    configured   unknown
c9::es/ses2                    ESI          connected    configured   unknown
c9::smp/expd0                  smp          connected    configured   unknown
c9::smp/expd1                  smp          connected    configured   unknown
c9::smp/expd2                  smp          connected    configured   unknown
c10                            scsi-sas     connected    configured   unknown
c10::dsk/c10t5001517BB27F79F7  disk         connected    configured   unknown
c11                            scsi-sas     connected    unconfigured unknown
usb2/1                         usb-hub      connected    configured   ok
usb2/1.1                       unknown      empty        unconfigured ok
usb2/1.2                       unknown      empty        unconfigured ok
usb2/1.3                       unknown      empty        unconfigured ok
usb2/1.4                       unknown      empty        unconfigured ok
usb2/1.5                       unknown      empty        unconfigured ok
usb2/1.6                       unknown      empty        unconfigured ok
usb2/2                         unknown      empty        unconfigured ok
usb3/1                         usb-hub      connected    configured   ok
usb3/1.1                       unknown      empty        unconfigured ok
usb3/1.2                       unknown      empty        unconfigured ok
usb3/1.3                       unknown      empty        unconfigured ok
usb3/1.4                       usb-device   connected    configured   ok
usb3/1.5                       unknown      empty        unconfigured ok
usb3/1.6                       unknown      empty        unconfigured ok
usb3/1.7                       unknown      empty        unconfigured ok
usb3/1.8                       unknown      empty        unconfigured ok
usb3/2                         unknown      empty        unconfigured ok


i see c11 as unconfigured. Could this be the cause ?

From: sim.ple at live.nl
To: omnios-discuss at lists.omniti.com
Date: Wed, 3 Dec 2014 11:37:31 +0100
Subject: Re: [OmniOS-discuss] test omnios on OI machine





Anybody else had problems with two or more sas hba's in an omnios system ?
How was this solved?

From: sim.ple at live.nl
To: omnios-discuss at lists.omniti.com
Date: Wed, 3 Dec 2014 01:45:12 +0100
Subject: Re: [OmniOS-discuss] test omnios on OI machine




Hi, was indeed intended for the list.

dmesg | grep mptsas

gives nothing (no output). 

Hope you have an idea of what to do next. 

Going to test a current omnios with one hba by adding another to it to see what happens tomorrow.  

Will be testing with new Dell hardware and solarflare cards probably in the comming weeks also. If you are interrested, will keep you posted. 



> Subject: Re: [OmniOS-discuss] test omnios on OI machine
> From: danmcd at omniti.com
> Date: Tue, 2 Dec 2014 19:22:15 -0500
> CC: danmcd at omniti.com
> To: sim.ple at live.nl
> 
> Please share these with the community in the future, unless you're sharing confidential data of some sort.
> 
> 
> > On Dec 2, 2014, at 6:59 PM, Randy S <sim.ple at live.nl> wrote:
> > 
> > Hi Dan,
> > 
> > It's been a while...
> > 
> > I wanted to stay on r10 since that has been in use by quite a community for quite a while now. R12 is still new so usually unforseen hicups can be expected. (We are not frontrunners regarding systems intended for serious use  ;-) ). Have been testing omnios for quite a while on systems with one hba now. All was satisfactory. 
> > 
> > forgot to mention, I did do an fmdump -ev and saw 
> > Dec 02 23:32:37.1675 ereport.fs.zfs.vdev.open_failed       0x0077a03af6b00001
> > Dec 02 23:32:37.1675 ereport.fs.zfs.vdev.open_failed       0x0077a03af6b00001
> > Dec 02 23:35:17.6783 ereport.sensor.failure                0x02cd95a4cde05801
> > Dec 02 23:35:17.6868 ereport.sensor.failure                0x02cd9dbb53605801
> > 
> > I guess this is because the one controller cannot be found.
> 
> That's a fair assessment.
> 
> > dmesg | grep mpt_sa (done just now (not sleepy yet)) gives :
> > 
> > Dec  3 00:42:44 omni scsi: [ID 583861 kern.info] sd68 at mpt_sas3: unit-address w5000c50057ad9161,0: w5000c50057ad9161,0
> > Dec  3 00:42:44 omni scsi: [ID 583861 kern.info] sd69 at mpt_sas3: unit-address w5000c50057adb269,0: w5000c50057adb269,0
> > Dec  3 00:42:45 omni scsi: [ID 583861 kern.info] sd70 at mpt_sas3: unit-address w5000c50057adea91,0: w5000c50057adea91,0
> > Dec  3 00:42:45 omni scsi: [ID 583861 kern.info] ses2 at mpt_sas3: unit-address w5003048000b18d3d,0: w5003048000b18d3d,0
> > Dec  3 00:42:45 omni scsi: [ID 583861 kern.info] mpt_sas4 at mpt_sas0: scsi-iport 20
> > Dec  3 00:42:45 omni genunix: [ID 936769 kern.info] mpt_sas4 is /pci at 0,0/pci8086,3c04 at 2/pci1000,3030 at 0/iport at 20
> > Dec  3 00:42:45 omni genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,3c04 at 2/pci1000,3030 at 0/iport at 20 (mpt_sas4) online
> > Dec  3 00:42:45 omni scsi: [ID 583861 kern.info] sd71 at mpt_sas4: unit-address w5001517bb27f79f7,0: w5001517bb27f79f7,0
> > Dec  3 00:42:45 omni scsi: [ID 583861 kern.info] mpt_sas5 at mpt_sas0: scsi-iport v0
> > Dec  3 00:42:45 omni genunix: [ID 936769 kern.info] mpt_sas5 is /pci at 0,0/pci8086,3c04 at 2/pci1000,3030 at 0/iport at v0
> > Dec  3 00:42:45 omni genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,3c04 at 2/pci1000,3030 at 0/iport at v0 (mpt_sas5) online
> 
> Interesting... note that two mpt_sas controllers (4 and 5) came online.  I wonder why one isn't showing up?  Grep for "mptsas" as well, just in case?
> 
> Dan
> 
 		 	   		  

_______________________________________________
OmniOS-discuss mailing list
OmniOS-discuss at lists.omniti.com
http://lists.omniti.com/mailman/listinfo/omnios-discuss 		 	   		  

_______________________________________________
OmniOS-discuss mailing list
OmniOS-discuss at lists.omniti.com
http://lists.omniti.com/mailman/listinfo/omnios-discuss 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141203/e98004f9/attachment.html>

From gate03 at landcroft.co.uk  Wed Dec  3 12:05:46 2014
From: gate03 at landcroft.co.uk (Michael Mounteney)
Date: Wed, 3 Dec 2014 22:05:46 +1000
Subject: [OmniOS-discuss] help with ipfilter rule for DHCP client
In-Reply-To: <20141204062847.6d1b24d9@punda-mlia>
References: <20141204062847.6d1b24d9@punda-mlia>
Message-ID: <20141203220546.1eed4dcb@punda-mlia>

Answering my own question:

# Allow access to ISP's specified DHCP server for cable or DSL networks.
# The first rule obtains the IP of the server.
# The second obtains an address from it.
pass out log quick on e1000g0 proto udp from any to any port = 67 keep state
pass out quick on e1000g0 proto udp from any to 192.168.0.1 port = 67 keep state

How about that?  Anything better?

From jimklimov at cos.ru  Wed Dec  3 12:02:26 2014
From: jimklimov at cos.ru (Jim Klimov)
Date: Wed, 03 Dec 2014 13:02:26 +0100
Subject: [OmniOS-discuss] KVM compatibility with Linux
Message-ID: <9BA9C418-73C1-4AB1-82D0-AA1A99F48594@cos.ru>

A team I know uses a Linux server hosting some KVM machines for continuous integration and testing of their work. Now there is discussion about backing it up and/or replicating, and I proposed to set up an omnios box for dependable storage and VM capabilities (at least vbox, or maybe kvm).

A question popped up - how comparable (features, hw support) and compatible (copy-paste the disk image and config) are the kvm implementations in linux and illumos/omnios today?

And what is the state of libvirt in illumos/omnios regarding management of VMs and similar environments (zones, vboxes, maybe BEs, etc.)?

Thanks in advance,
Jim

--
Typos courtesy of K-9 Mail on my Samsung Android

From sim.ple at live.nl  Wed Dec  3 12:44:35 2014
From: sim.ple at live.nl (Randy S)
Date: Wed, 3 Dec 2014 13:44:35 +0100
Subject: [OmniOS-discuss] Upgrading from OI to OmniOS
In-Reply-To: <DUB124-W9CEC7807C3A19C2612FEDE07B0@phx.gbl>
References: <547DD116.7040401@blue-bolt.com>,
	<DUB124-W9CEC7807C3A19C2612FEDE07B0@phx.gbl>
Message-ID: <DUB124-W52130E84F44FC86051EE9FE07B0@phx.gbl>

should have send this to the list.... again.... ;-)



Hi,

The safest way I could think of to do this is as follows:
create a bootenv in OI en install Omnios fresh into this
Start omnios and if you need any settings you can mount the be of OI to e.g. /mnt and copy the config files from there to the Omnios install.

I had to keep track of config files in /etc and /kernel/drv 
I have noticed that both oses are quite compatible

Above procedure can first be tested with vm's 

Do not upgrade any pools untill you are satisfied with the new situation since there might be a diff in zfs version. If you do you won't be able to go back to your OI install.

Please first test with vm's!

Rgds

R



> Date: Tue, 2 Dec 2014 14:47:50 +0000
> From: cal-s at blue-bolt.com
> To: omnios-discuss at lists.omniti.com
> Subject: [OmniOS-discuss] Upgrading from OI to OmniOS
> 
> Hi
> 
> I have a server hosting 65TB of RAIDZ2 storage that's currently running 
> OI (oi_151a8), which i really want to get off of given that OI has 
> bitten the dust support-wise.  I found one brief discussion on the topic 
> but it wasn't terribly detailed and was missing critical steps like 
> configuration protection/retention. I'd like to preserve network and esp 
> LDAP (client) settings as they were kind of a pain to establish in the 
> first instance for quick reapplication. If anyone has or is interested 
> in cobbling together a checklist of steps to upgrade OI to OmniOS that 
> takes configuration into account, i'd very much appreciate seeing it.
> 
> thank you
> 
> - cal sawyer
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
 		 	   		   		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141203/dfc5cd9f/attachment.html>

From eric.sproul at circonus.com  Wed Dec  3 14:49:04 2014
From: eric.sproul at circonus.com (Eric Sproul)
Date: Wed, 3 Dec 2014 09:49:04 -0500
Subject: [OmniOS-discuss] packaging - unwanted directory group owner
In-Reply-To: <CALOiZS2KUDXNsKgVLbUaH1GKWMG5M7KXpjGx4nzR-KKH4g9nBQ@mail.gmail.com>
References: <03c0db4c2df34d0fb257362cba456703@SRV-MAIL-001.zhaw.ch>
	<CALOiZS2KUDXNsKgVLbUaH1GKWMG5M7KXpjGx4nzR-KKH4g9nBQ@mail.gmail.com>
Message-ID: <CAO8hXRAEFMdUmTaQwsfq9-qMnTWDKBBdQF8cy-Z7TjkJJDGHKQ@mail.gmail.com>

On Wed, Dec 3, 2014 at 4:48 AM, Vincenzo Pii <piiv at zhaw.ch> wrote:
> Ok, problem solved :D!
>
> I added a "local.mog" file to my build directory (besides build.sh), with
> the following content:
>
>     <transform dir  path=etc/init.d -> set group sys>
>
> I realized I could do that by browsing the omnios-build repo, for the bash
> package:
>
> https://github.com/omniti-labs/omnios-build/blob/abbfd1697849f6f8a84be3f94317b317798236a1/build/bash/local.mog

Vincenzo,
If you're interested, the IPS Developer's Guide, linked from
http://omnios.omniti.com/wiki.php/MoreInfo was very helpful to me when
I was first learning how to package for IPS.  Among many other things,
it describes the various ways you can programmatically transform pkg
manifests to fine-tune your packages.

Eric

From filip.marvan at aira.cz  Wed Dec  3 14:53:36 2014
From: filip.marvan at aira.cz (Filip Marvan)
Date: Wed, 3 Dec 2014 15:53:36 +0100
Subject: [OmniOS-discuss] zpool import strange informations
In-Reply-To: <21BA9319-381E-4303-8B49-93B7693FFCE7@richardelling.com>
References: <3BE0DEED8863E5429BAE4CAEDF624565039C1B330761@AIRA-SRV.aira.local>
	<CALLZ=vV4D6yDai9mdnq7Z6odinxFARSSCTtz_nHyEyD4Jfz4=A@mail.gmail.com>
	<3BE0DEED8863E5429BAE4CAEDF624565039C1B33076D@AIRA-SRV.aira.local>
	<21BA9319-381E-4303-8B49-93B7693FFCE7@richardelling.com>
Message-ID: <3BE0DEED8863E5429BAE4CAEDF624565039C1B3307EF@AIRA-SRV.aira.local>

Hi,

thank you for clarification. Yes, it is possible, that these disk was used
in some ZFS pool before, they aren't new and I don't know their history.

So I checked both, c3t2d0 and c3t5d0 disks (disks, that are "online" in
importable "tank" pool) and I found nothing strange.

      Partition   Status    Type          Start   End   Length    %
      =========   ======    ============  =====   ===   ======   ===
          1                 EFI               0  60800    60801    100

There is only one partition on both disk, which should be online and used by
my "raid_10" pool (as I posted before). So do you have any idea, where on
disks are stored information, that they are part of some another pool and
how can get rid of it?

Thank you very much,
Filip


-----Original Message-----
From: Richard Elling [mailto:richard.elling at richardelling.com] 
Sent: Tuesday, December 02, 2014 6:38 PM
To: Filip Marvan
Cc: Zach Malone; omnios-discuss at lists.omniti.com
Subject: Re: [OmniOS-discuss] zpool import strange informations


On Dec 2, 2014, at 7:20 AM, Filip Marvan <filip.marvan at aira.cz> wrote:

> Yes, you are probably right. But I'm thinking, why that information 
> wasn't overwriten when I add whole c3t2d0 to a new mirror?

p0 is an fdisk partition for the whole disk.
c3t2d0 is really a shorthand notation for c3t2d0s0.
c3t2d0 is also in a fdisk partition of type "Solaris2"

So you'll need to check your partitioning and see where exactly the blocks
are allocated to solve the mystery.

> Is it possible to use
> c3t2d0 in one pool, and c3t2d0p0 in another?

In general, no. The p0 fdisk partition overlaps whatever fdisk partition is
being used for c3t2d0 -- an invitation to data corruption.

In general, it is considered bad practice to use p0 (or any p*) for ZFS pool
creation.
There is a darn good reason the ZFS docs refer to disks as c3t2d0 -- if you
do this consistently, then you'll never have partition nightmares.

HTH
 -- richard

> 
> Thank you,
> Filip
> 
> -----Original Message-----
> From: Zach Malone [mailto:zmalone at omniti.com]
> Sent: Tuesday, December 02, 2014 4:12 PM
> To: Filip Marvan
> Cc: omnios-discuss at lists.omniti.com
> Subject: Re: [OmniOS-discuss] zpool import strange informations
> 
> If I was to guess,  c3t2d0p0 was once a part of another pool called 
> "tank", and when you do an import, it's attempting to reassemble the 
> rest of tank, and sees that a drive named c3t5d0 used to be in the pool it
knew as tank.
> 
> Tank is a pretty standard pool name, so I'm guessing the drive used to 
> be in another system, or the system itself used to be running a different
OS.
> --Zach
> 
> On Tue, Dec 2, 2014 at 9:31 AM, Filip Marvan <filip.marvan at aira.cz> wrote:
>> Hi,
>> 
>> 
>> 
>> I have one storage server on OmniOS 12 with one data pool:
>> 
>> 
>> 
>>  pool: raid_10
>> 
>> state: ONLINE
>> 
>>  scan: none requested
>> 
>> config:
>> 
>> 
>> 
>>        NAME        STATE     READ WRITE CKSUM
>> 
>>        raid_10     ONLINE       0     0     0
>> 
>>          mirror-0  ONLINE       0     0     0
>> 
>>            c3t2d0  ONLINE       0     0     0
>> 
>>            c3t3d0  ONLINE       0     0     0
>> 
>>          mirror-1  ONLINE       0     0     0
>> 
>>            c3t4d0  ONLINE       0     0     0
>> 
>>            c3t5d0  ONLINE       0     0     0
>> 
>>        logs
>> 
>>          c3t0d0    ONLINE       0     0     0
>> 
>> 
>> 
>> errors: No known data errors
>> 
>> 
>> 
>>  pool: rpool
>> 
>> state: ONLINE
>> 
>>  scan: none requested
>> 
>> config:
>> 
>> 
>> 
>>        NAME        STATE     READ WRITE CKSUM
>> 
>>        rpool       ONLINE       0     0     0
>> 
>>          c3t1d0s0  ONLINE       0     0     0
>> 
>> 
>> 
>> errors: No known data errors
>> 
>> 
>> 
>> There are no other disks in this storage server. Bud if I write 
>> command zpool import, I can see:
>> 
>>  pool: tank
>> 
>>     id: 4916680739993977334
>> 
>>  state: UNAVAIL
>> 
>> status: The pool was last accessed by another system.
>> 
>> action: The pool cannot be imported due to damaged devices or data.
>> 
>>   see: http://illumos.org/msg/ZFS-8000-EY
>> 
>> config:
>> 
>> 
>> 
>>        tank                     UNAVAIL  insufficient replicas
>> 
>>          raidz2-1               UNAVAIL  insufficient replicas
>> 
>>            /dev/label/mfront8   UNAVAIL  cannot open
>> 
>>            /dev/label/mfront9   UNAVAIL  cannot open
>> 
>>            /dev/label/mfront10  UNAVAIL  cannot open
>> 
>>            c3t2d0p0             ONLINE
>> 
>>            /dev/label/mfront12  UNAVAIL  cannot open
>> 
>>            /dev/label/mfront13  UNAVAIL  cannot open
>> 
>>            /dev/label/mfront14  UNAVAIL  cannot open
>> 
>>            c3t5d0               ONLINE
>> 
>> 
>> 
>> I never created such "tank" pool. Where are stored informations about 
>> this (un)importable pool, how can I delete them? Why c3t5d0 seems to 
>> be as part of two different pools at the same time?
>> 
>> 
>> 
>> Thank you for any help.
>> 
>> Filip Marvan
>> 
>> 
>> _______________________________________________
>> OmniOS-discuss mailing list
>> OmniOS-discuss at lists.omniti.com
>> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>> 
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 6247 bytes
Desc: not available
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141203/e5ce0299/attachment.bin>

From piiv at zhaw.ch  Wed Dec  3 14:54:09 2014
From: piiv at zhaw.ch (Vincenzo Pii)
Date: Wed, 3 Dec 2014 15:54:09 +0100
Subject: [OmniOS-discuss] packaging - unwanted directory group owner
In-Reply-To: <75636d9956eb410dbadc8850c9dd420c@SRV-MAIL-001.zhaw.ch>
References: <03c0db4c2df34d0fb257362cba456703@SRV-MAIL-001.zhaw.ch>
	<CALOiZS2KUDXNsKgVLbUaH1GKWMG5M7KXpjGx4nzR-KKH4g9nBQ@mail.gmail.com>
	<75636d9956eb410dbadc8850c9dd420c@SRV-MAIL-001.zhaw.ch>
Message-ID: <CALOiZS2GLPaLMxuakNNcM=cMrWKMBywYBnC5K1Kf2RToff=_yQ@mail.gmail.com>

Wow, many thanks Eric.

Could someone with super powers add the link in [1] at the end of [2]? I
wish that link was there since 2 days ago :D!
(I cannot edit the wiki.)

[1]  http://omnios.omniti.com/wiki.php/MoreInfo#IPSpkg5
[2]  http://omnios.omniti.com/wiki.php/PackagingForOmniOS

2014-12-03 15:49 GMT+01:00 Eric Sproul <eric.sproul at circonus.com>:

> On Wed, Dec 3, 2014 at 4:48 AM, Vincenzo Pii <piiv at zhaw.ch> wrote:
> > Ok, problem solved :D!
> >
> > I added a "local.mog" file to my build directory (besides build.sh), with
> > the following content:
> >
> >     <transform dir  path=etc/init.d -> set group sys>
> >
> > I realized I could do that by browsing the omnios-build repo, for the
> bash
> > package:
> >
> >
> https://github.com/omniti-labs/omnios-build/blob/abbfd1697849f6f8a84be3f94317b317798236a1/build/bash/local.mog
>
> Vincenzo,
> If you're interested, the IPS Developer's Guide, linked from
> http://omnios.omniti.com/wiki.php/MoreInfo was very helpful to me when
> I was first learning how to package for IPS.  Among many other things,
> it describes the various ways you can programmatically transform pkg
> manifests to fine-tune your packages.
>
> Eric
>



-- 
Vincenzo Pii
Researcher, InIT Cloud Computing Lab
Zurich University of Applied Sciences (ZHAW)
blog.zhaw.ch/icclab
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141203/ebca0073/attachment-0001.html>

From eric.sproul at circonus.com  Wed Dec  3 15:02:59 2014
From: eric.sproul at circonus.com (Eric Sproul)
Date: Wed, 3 Dec 2014 10:02:59 -0500
Subject: [OmniOS-discuss] packaging - unwanted directory group owner
In-Reply-To: <CALOiZS2GLPaLMxuakNNcM=cMrWKMBywYBnC5K1Kf2RToff=_yQ@mail.gmail.com>
References: <03c0db4c2df34d0fb257362cba456703@SRV-MAIL-001.zhaw.ch>
	<CALOiZS2KUDXNsKgVLbUaH1GKWMG5M7KXpjGx4nzR-KKH4g9nBQ@mail.gmail.com>
	<75636d9956eb410dbadc8850c9dd420c@SRV-MAIL-001.zhaw.ch>
	<CALOiZS2GLPaLMxuakNNcM=cMrWKMBywYBnC5K1Kf2RToff=_yQ@mail.gmail.com>
Message-ID: <CAO8hXRB=FMxGJ=mO56-rV54_XSEQxLYNYxBMY_uGJV5opKzTaQ@mail.gmail.com>

On Wed, Dec 3, 2014 at 9:54 AM, Vincenzo Pii <piiv at zhaw.ch> wrote:
> Wow, many thanks Eric.
>
> Could someone with super powers add the link in [1] at the end of [2]? I
> wish that link was there since 2 days ago :D!
> (I cannot edit the wiki.)
>
> [1]  http://omnios.omniti.com/wiki.php/MoreInfo#IPSpkg5
> [2]  http://omnios.omniti.com/wiki.php/PackagingForOmniOS

Done.  I put it at the top.  :)

From sim.ple at live.nl  Wed Dec  3 16:11:32 2014
From: sim.ple at live.nl (Randy S)
Date: Wed, 3 Dec 2014 17:11:32 +0100
Subject: [OmniOS-discuss] test omnios on OI machine / solved
In-Reply-To: <DUB124-W12DFDF024A9F155D414B06E07B0@phx.gbl>
References: <DUB124-W4FAE9E8C04253565F247FE07A0@phx.gbl>, , <, ,
	<13009DC9-7FA5-4938-8042-F2AF74DDC695@omniti.com>, <>>, ,
	<DUB124-W31A653A65DB219489C3FBAE07A0@phx.gbl>, , , 
	, <63802B04-B4E9-4FC6-8059-EDAD86FBADBE@omniti.com>,
	, , <DUB124-W28749FC3157E8298D365CAE07B0@phx.gbl>, ,
	<DUB124-W15E9AE203C73CD4F0BBB3FE07B0@phx.gbl>,
	<DUB124-W12DFDF024A9F155D414B06E07B0@phx.gbl>
Message-ID: <DUB124-W11981608C18CCD499912BAE07B0@phx.gbl>

As a last resort I just turned off the machine and kept it off for a few minutes. Turned it back on and both controllers are now properly detected.

R

From: sim.ple at live.nl
To: omnios-discuss at lists.omniti.com
Date: Wed, 3 Dec 2014 11:59:11 +0100
Subject: Re: [OmniOS-discuss] test omnios on OI machine




when I do 

cfgadm -al
Ap_Id                          Type         Receptacle   Occupant     Condition
c7                             scsi-sas     connected    configured   unknown
c7::dsk/c7t500000E118E31592d0  disk         connected    configured   unknown
c9                             scsi-sas     connected    configured   unknown
c9::dsk/c9t5000C50055B2A5B1d0  disk         connected    configured   unknown
c9::dsk/c9t5000C50055B2A9A9d0  disk         connected    configured   unknown
c9::dsk/c9t5000C50055B2A139d0  disk         connected    configured   unknown
c9::dsk/c9t5000C50055B2A245d0  disk         connected    configured   unknown
c9::dsk/c9t5000C50055B2B081d0  disk         connected    configured   unknown
c9::dsk/c9t5000C50055B2B249d0  disk         connected    configured   unknown
c9::dsk/c9t5000C50055B2B255d0  disk         connected    configured   unknown
c9::dsk/c9t5000C50055B2BD6Dd0  disk         connected    configured   unknown
c9::es/ses0                    ESI          connected    configured   unknown
c9::es/ses1                    ESI          connected    configured   unknown
c9::es/ses2                    ESI          connected    configured   unknown
c9::smp/expd0                  smp          connected    configured   unknown
c9::smp/expd1                  smp          connected    configured   unknown
c9::smp/expd2                  smp          connected    configured   unknown
c10                            scsi-sas     connected    configured   unknown
c10::dsk/c10t5001517BB27F79F7  disk         connected    configured   unknown
c11                            scsi-sas     connected    unconfigured unknown
usb2/1                         usb-hub      connected    configured   ok
usb2/1.1                       unknown      empty        unconfigured ok
usb2/1.2                       unknown      empty        unconfigured ok
usb2/1.3                       unknown      empty        unconfigured ok
usb2/1.4                       unknown      empty        unconfigured ok
usb2/1.5                       unknown      empty        unconfigured ok
usb2/1.6                       unknown      empty        unconfigured ok
usb2/2                         unknown      empty        unconfigured ok
usb3/1                         usb-hub      connected    configured   ok
usb3/1.1                       unknown      empty        unconfigured ok
usb3/1.2                       unknown      empty        unconfigured ok
usb3/1.3                       unknown      empty        unconfigured ok
usb3/1.4                       usb-device   connected    configured   ok
usb3/1.5                       unknown      empty        unconfigured ok
usb3/1.6                       unknown      empty        unconfigured ok
usb3/1.7                       unknown      empty        unconfigured ok
usb3/1.8                       unknown      empty        unconfigured ok
usb3/2                         unknown      empty        unconfigured ok


i see c11 as unconfigured. Could this be the cause ?

From: sim.ple at live.nl
To: omnios-discuss at lists.omniti.com
Date: Wed, 3 Dec 2014 11:37:31 +0100
Subject: Re: [OmniOS-discuss] test omnios on OI machine





Anybody else had problems with two or more sas hba's in an omnios system ?
How was this solved?

From: sim.ple at live.nl
To: omnios-discuss at lists.omniti.com
Date: Wed, 3 Dec 2014 01:45:12 +0100
Subject: Re: [OmniOS-discuss] test omnios on OI machine




Hi, was indeed intended for the list.

dmesg | grep mptsas

gives nothing (no output). 

Hope you have an idea of what to do next. 

Going to test a current omnios with one hba by adding another to it to see what happens tomorrow.  

Will be testing with new Dell hardware and solarflare cards probably in the comming weeks also. If you are interrested, will keep you posted. 



> Subject: Re: [OmniOS-discuss] test omnios on OI machine
> From: danmcd at omniti.com
> Date: Tue, 2 Dec 2014 19:22:15 -0500
> CC: danmcd at omniti.com
> To: sim.ple at live.nl
> 
> Please share these with the community in the future, unless you're sharing confidential data of some sort.
> 
> 
> > On Dec 2, 2014, at 6:59 PM, Randy S <sim.ple at live.nl> wrote:
> > 
> > Hi Dan,
> > 
> > It's been a while...
> > 
> > I wanted to stay on r10 since that has been in use by quite a community for quite a while now. R12 is still new so usually unforseen hicups can be expected. (We are not frontrunners regarding systems intended for serious use  ;-) ). Have been testing omnios for quite a while on systems with one hba now. All was satisfactory. 
> > 
> > forgot to mention, I did do an fmdump -ev and saw 
> > Dec 02 23:32:37.1675 ereport.fs.zfs.vdev.open_failed       0x0077a03af6b00001
> > Dec 02 23:32:37.1675 ereport.fs.zfs.vdev.open_failed       0x0077a03af6b00001
> > Dec 02 23:35:17.6783 ereport.sensor.failure                0x02cd95a4cde05801
> > Dec 02 23:35:17.6868 ereport.sensor.failure                0x02cd9dbb53605801
> > 
> > I guess this is because the one controller cannot be found.
> 
> That's a fair assessment.
> 
> > dmesg | grep mpt_sa (done just now (not sleepy yet)) gives :
> > 
> > Dec  3 00:42:44 omni scsi: [ID 583861 kern.info] sd68 at mpt_sas3: unit-address w5000c50057ad9161,0: w5000c50057ad9161,0
> > Dec  3 00:42:44 omni scsi: [ID 583861 kern.info] sd69 at mpt_sas3: unit-address w5000c50057adb269,0: w5000c50057adb269,0
> > Dec  3 00:42:45 omni scsi: [ID 583861 kern.info] sd70 at mpt_sas3: unit-address w5000c50057adea91,0: w5000c50057adea91,0
> > Dec  3 00:42:45 omni scsi: [ID 583861 kern.info] ses2 at mpt_sas3: unit-address w5003048000b18d3d,0: w5003048000b18d3d,0
> > Dec  3 00:42:45 omni scsi: [ID 583861 kern.info] mpt_sas4 at mpt_sas0: scsi-iport 20
> > Dec  3 00:42:45 omni genunix: [ID 936769 kern.info] mpt_sas4 is /pci at 0,0/pci8086,3c04 at 2/pci1000,3030 at 0/iport at 20
> > Dec  3 00:42:45 omni genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,3c04 at 2/pci1000,3030 at 0/iport at 20 (mpt_sas4) online
> > Dec  3 00:42:45 omni scsi: [ID 583861 kern.info] sd71 at mpt_sas4: unit-address w5001517bb27f79f7,0: w5001517bb27f79f7,0
> > Dec  3 00:42:45 omni scsi: [ID 583861 kern.info] mpt_sas5 at mpt_sas0: scsi-iport v0
> > Dec  3 00:42:45 omni genunix: [ID 936769 kern.info] mpt_sas5 is /pci at 0,0/pci8086,3c04 at 2/pci1000,3030 at 0/iport at v0
> > Dec  3 00:42:45 omni genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,3c04 at 2/pci1000,3030 at 0/iport at v0 (mpt_sas5) online
> 
> Interesting... note that two mpt_sas controllers (4 and 5) came online.  I wonder why one isn't showing up?  Grep for "mptsas" as well, just in case?
> 
> Dan
> 
 		 	   		  

_______________________________________________
OmniOS-discuss mailing list
OmniOS-discuss at lists.omniti.com
http://lists.omniti.com/mailman/listinfo/omnios-discuss 		 	   		  

_______________________________________________
OmniOS-discuss mailing list
OmniOS-discuss at lists.omniti.com
http://lists.omniti.com/mailman/listinfo/omnios-discuss 		 	   		  

_______________________________________________
OmniOS-discuss mailing list
OmniOS-discuss at lists.omniti.com
http://lists.omniti.com/mailman/listinfo/omnios-discuss 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141203/9af25fea/attachment-0001.html>

From richard.elling at richardelling.com  Wed Dec  3 16:58:43 2014
From: richard.elling at richardelling.com (Richard Elling)
Date: Wed, 3 Dec 2014 08:58:43 -0800
Subject: [OmniOS-discuss] test omnios on OI machine
In-Reply-To: <DUB124-W15E9AE203C73CD4F0BBB3FE07B0@phx.gbl>
References: <DUB124-W4FAE9E8C04253565F247FE07A0@phx.gbl>, <,
	<13009DC9-7FA5-4938-8042-F2AF74DDC695@omniti.com> <>>,
	<DUB124-W31A653A65DB219489C3FBAE07A0@phx.gbl>, ,
	<63802B04-B4E9-4FC6-8059-EDAD86FBADBE@omniti.com>,
	<DUB124-W28749FC3157E8298D365CAE07B0@phx.gbl>
	<DUB124-W15E9AE203C73CD4F0BBB3FE07B0@phx.gbl>
Message-ID: <EDE6C088-3821-41B1-A947-97DB7D46AE05@richardelling.com>

I'm glad you got it running ok. Helpful hint below...

On Dec 3, 2014, at 2:37 AM, Randy S <sim.ple at live.nl> wrote:

> 
> Anybody else had problems with two or more sas hba's in an omnios system ?
> How was this solved?
> 
> From: sim.ple at live.nl
> To: omnios-discuss at lists.omniti.com
> Date: Wed, 3 Dec 2014 01:45:12 +0100
> Subject: Re: [OmniOS-discuss] test omnios on OI machine
> 
> Hi, was indeed intended for the list.
> 
> dmesg | grep mptsas

dmesg is brain dead. All it does is (from the script itself, /usr/bin/dmesg):
	/usr/bin/cat -sv `/usr/bin/ls -tr1 /var/adm/messages.? 2>/dev/null` \
		/var/adm/messages | /usr/bin/tail -200

to show you the last 200 lines in the messages file. For a large system, there can 
easily be 1000+ lines of boot-time messages. You're better off looking at the messages
file directly:
	grep mptsas /var/adm/messages

For a better method of identifying loaded drivers, use prtconf:
	prtconf -D
shows the device tree and the attached drivers.

HTH
 -- richard

> 
> gives nothing (no output). 
> 
> Hope you have an idea of what to do next. 
> 
> Going to test a current omnios with one hba by adding another to it to see what happens tomorrow.  
> 
> Will be testing with new Dell hardware and solarflare cards probably in the comming weeks also. If you are interrested, will keep you posted. 
> 
> 
> 
> > Subject: Re: [OmniOS-discuss] test omnios on OI machine
> > From: danmcd at omniti.com
> > Date: Tue, 2 Dec 2014 19:22:15 -0500
> > CC: danmcd at omniti.com
> > To: sim.ple at live.nl
> > 
> > Please share these with the community in the future, unless you're sharing confidential data of some sort.
> > 
> > 
> > > On Dec 2, 2014, at 6:59 PM, Randy S <sim.ple at live.nl> wrote:
> > > 
> > > Hi Dan,
> > > 
> > > It's been a while...
> > > 
> > > I wanted to stay on r10 since that has been in use by quite a community for quite a while now. R12 is still new so usually unforseen hicups can be expected. (We are not frontrunners regarding systems intended for serious use ;-) ). Have been testing omnios for quite a while on systems with one hba now. All was satisfactory. 
> > > 
> > > forgot to mention, I did do an fmdump -ev and saw 
> > > Dec 02 23:32:37.1675 ereport.fs.zfs.vdev.open_failed 0x0077a03af6b00001
> > > Dec 02 23:32:37.1675 ereport.fs.zfs.vdev.open_failed 0x0077a03af6b00001
> > > Dec 02 23:35:17.6783 ereport.sensor.failure 0x02cd95a4cde05801
> > > Dec 02 23:35:17.6868 ereport.sensor.failure 0x02cd9dbb53605801
> > > 
> > > I guess this is because the one controller cannot be found.
> > 
> > That's a fair assessment.
> > 
> > > dmesg | grep mpt_sa (done just now (not sleepy yet)) gives :
> > > 
> > > Dec 3 00:42:44 omni scsi: [ID 583861 kern.info] sd68 at mpt_sas3: unit-address w5000c50057ad9161,0: w5000c50057ad9161,0
> > > Dec 3 00:42:44 omni scsi: [ID 583861 kern.info] sd69 at mpt_sas3: unit-address w5000c50057adb269,0: w5000c50057adb269,0
> > > Dec 3 00:42:45 omni scsi: [ID 583861 kern.info] sd70 at mpt_sas3: unit-address w5000c50057adea91,0: w5000c50057adea91,0
> > > Dec 3 00:42:45 omni scsi: [ID 583861 kern.info] ses2 at mpt_sas3: unit-address w5003048000b18d3d,0: w5003048000b18d3d,0
> > > Dec 3 00:42:45 omni scsi: [ID 583861 kern.info] mpt_sas4 at mpt_sas0: scsi-iport 20
> > > Dec 3 00:42:45 omni genunix: [ID 936769 kern.info] mpt_sas4 is /pci at 0,0/pci8086,3c04 at 2/pci1000,3030 at 0/iport at 20
> > > Dec 3 00:42:45 omni genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,3c04 at 2/pci1000,3030 at 0/iport at 20 (mpt_sas4) online
> > > Dec 3 00:42:45 omni scsi: [ID 583861 kern.info] sd71 at mpt_sas4: unit-address w5001517bb27f79f7,0: w5001517bb27f79f7,0
> > > Dec 3 00:42:45 omni scsi: [ID 583861 kern.info] mpt_sas5 at mpt_sas0: scsi-iport v0
> > > Dec 3 00:42:45 omni genunix: [ID 936769 kern.info] mpt_sas5 is /pci at 0,0/pci8086,3c04 at 2/pci1000,3030 at 0/iport at v0
> > > Dec 3 00:42:45 omni genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,3c04 at 2/pci1000,3030 at 0/iport at v0 (mpt_sas5) online
> > 
> > Interesting... note that two mpt_sas controllers (4 and 5) came online. I wonder why one isn't showing up? Grep for "mptsas" as well, just in case?
> > 
> > Dan
> > 
> 
> _______________________________________________ OmniOS-discuss mailing list OmniOS-discuss at lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141203/16fe3c55/attachment.html>

From richard.elling at richardelling.com  Wed Dec  3 17:02:08 2014
From: richard.elling at richardelling.com (Richard Elling)
Date: Wed, 3 Dec 2014 09:02:08 -0800
Subject: [OmniOS-discuss] zpool import strange informations
In-Reply-To: <3BE0DEED8863E5429BAE4CAEDF624565039C1B3307EF@AIRA-SRV.aira.local>
References: <3BE0DEED8863E5429BAE4CAEDF624565039C1B330761@AIRA-SRV.aira.local>
	<CALLZ=vV4D6yDai9mdnq7Z6odinxFARSSCTtz_nHyEyD4Jfz4=A@mail.gmail.com>
	<3BE0DEED8863E5429BAE4CAEDF624565039C1B33076D@AIRA-SRV.aira.local>
	<21BA9319-381E-4303-8B49-93B7693FFCE7@richardelling.com>
	<3BE0DEED8863E5429BAE4CAEDF624565039C1B3307EF@AIRA-SRV.aira.local>
Message-ID: <6E5A567E-6AC9-475F-8FBD-3133C72E58D1@richardelling.com>


On Dec 3, 2014, at 6:53 AM, Filip Marvan <filip.marvan at aira.cz> wrote:

> Hi,
> 
> thank you for clarification. Yes, it is possible, that these disk was used
> in some ZFS pool before, they aren't new and I don't know their history.
> 
> So I checked both, c3t2d0 and c3t5d0 disks (disks, that are "online" in
> importable "tank" pool) and I found nothing strange.
> 
>      Partition   Status    Type          Start   End   Length    %
>      =========   ======    ============  =====   ===   ======   ===
>          1                 EFI               0  60800    60801    100
> 
> There is only one partition on both disk, which should be online and used by
> my "raid_10" pool (as I posted before). So do you have any idea, where on
> disks are stored information, that they are part of some another pool and
> how can get rid of it?

Try this instead:
	for i in /dev/dsk/c3t2d0*
	do
		echo $i
		zdb -l $i
	done

This will show you the ZFS labels on each and every slice/partition for the disk.
Ideally, you'll only see one set of ZFS labels for each disk.
 -- richard

> 
> Thank you very much,
> Filip
> 
> 
> -----Original Message-----
> From: Richard Elling [mailto:richard.elling at richardelling.com] 
> Sent: Tuesday, December 02, 2014 6:38 PM
> To: Filip Marvan
> Cc: Zach Malone; omnios-discuss at lists.omniti.com
> Subject: Re: [OmniOS-discuss] zpool import strange informations
> 
> 
> On Dec 2, 2014, at 7:20 AM, Filip Marvan <filip.marvan at aira.cz> wrote:
> 
>> Yes, you are probably right. But I'm thinking, why that information 
>> wasn't overwriten when I add whole c3t2d0 to a new mirror?
> 
> p0 is an fdisk partition for the whole disk.
> c3t2d0 is really a shorthand notation for c3t2d0s0.
> c3t2d0 is also in a fdisk partition of type "Solaris2"
> 
> So you'll need to check your partitioning and see where exactly the blocks
> are allocated to solve the mystery.
> 
>> Is it possible to use
>> c3t2d0 in one pool, and c3t2d0p0 in another?
> 
> In general, no. The p0 fdisk partition overlaps whatever fdisk partition is
> being used for c3t2d0 -- an invitation to data corruption.
> 
> In general, it is considered bad practice to use p0 (or any p*) for ZFS pool
> creation.
> There is a darn good reason the ZFS docs refer to disks as c3t2d0 -- if you
> do this consistently, then you'll never have partition nightmares.
> 
> HTH
> -- richard
> 
>> 
>> Thank you,
>> Filip
>> 
>> -----Original Message-----
>> From: Zach Malone [mailto:zmalone at omniti.com]
>> Sent: Tuesday, December 02, 2014 4:12 PM
>> To: Filip Marvan
>> Cc: omnios-discuss at lists.omniti.com
>> Subject: Re: [OmniOS-discuss] zpool import strange informations
>> 
>> If I was to guess,  c3t2d0p0 was once a part of another pool called 
>> "tank", and when you do an import, it's attempting to reassemble the 
>> rest of tank, and sees that a drive named c3t5d0 used to be in the pool it
> knew as tank.
>> 
>> Tank is a pretty standard pool name, so I'm guessing the drive used to 
>> be in another system, or the system itself used to be running a different
> OS.
>> --Zach
>> 
>> On Tue, Dec 2, 2014 at 9:31 AM, Filip Marvan <filip.marvan at aira.cz> wrote:
>>> Hi,
>>> 
>>> 
>>> 
>>> I have one storage server on OmniOS 12 with one data pool:
>>> 
>>> 
>>> 
>>> pool: raid_10
>>> 
>>> state: ONLINE
>>> 
>>> scan: none requested
>>> 
>>> config:
>>> 
>>> 
>>> 
>>>       NAME        STATE     READ WRITE CKSUM
>>> 
>>>       raid_10     ONLINE       0     0     0
>>> 
>>>         mirror-0  ONLINE       0     0     0
>>> 
>>>           c3t2d0  ONLINE       0     0     0
>>> 
>>>           c3t3d0  ONLINE       0     0     0
>>> 
>>>         mirror-1  ONLINE       0     0     0
>>> 
>>>           c3t4d0  ONLINE       0     0     0
>>> 
>>>           c3t5d0  ONLINE       0     0     0
>>> 
>>>       logs
>>> 
>>>         c3t0d0    ONLINE       0     0     0
>>> 
>>> 
>>> 
>>> errors: No known data errors
>>> 
>>> 
>>> 
>>> pool: rpool
>>> 
>>> state: ONLINE
>>> 
>>> scan: none requested
>>> 
>>> config:
>>> 
>>> 
>>> 
>>>       NAME        STATE     READ WRITE CKSUM
>>> 
>>>       rpool       ONLINE       0     0     0
>>> 
>>>         c3t1d0s0  ONLINE       0     0     0
>>> 
>>> 
>>> 
>>> errors: No known data errors
>>> 
>>> 
>>> 
>>> There are no other disks in this storage server. Bud if I write 
>>> command zpool import, I can see:
>>> 
>>> pool: tank
>>> 
>>>    id: 4916680739993977334
>>> 
>>> state: UNAVAIL
>>> 
>>> status: The pool was last accessed by another system.
>>> 
>>> action: The pool cannot be imported due to damaged devices or data.
>>> 
>>>  see: http://illumos.org/msg/ZFS-8000-EY
>>> 
>>> config:
>>> 
>>> 
>>> 
>>>       tank                     UNAVAIL  insufficient replicas
>>> 
>>>         raidz2-1               UNAVAIL  insufficient replicas
>>> 
>>>           /dev/label/mfront8   UNAVAIL  cannot open
>>> 
>>>           /dev/label/mfront9   UNAVAIL  cannot open
>>> 
>>>           /dev/label/mfront10  UNAVAIL  cannot open
>>> 
>>>           c3t2d0p0             ONLINE
>>> 
>>>           /dev/label/mfront12  UNAVAIL  cannot open
>>> 
>>>           /dev/label/mfront13  UNAVAIL  cannot open
>>> 
>>>           /dev/label/mfront14  UNAVAIL  cannot open
>>> 
>>>           c3t5d0               ONLINE
>>> 
>>> 
>>> 
>>> I never created such "tank" pool. Where are stored informations about 
>>> this (un)importable pool, how can I delete them? Why c3t5d0 seems to 
>>> be as part of two different pools at the same time?
>>> 
>>> 
>>> 
>>> Thank you for any help.
>>> 
>>> Filip Marvan
>>> 
>>> 
>>> _______________________________________________
>>> OmniOS-discuss mailing list
>>> OmniOS-discuss at lists.omniti.com
>>> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>>> 
>> _______________________________________________
>> OmniOS-discuss mailing list
>> OmniOS-discuss at lists.omniti.com
>> http://lists.omniti.com/mailman/listinfo/omnios-discuss
> 


From sim.ple at live.nl  Wed Dec  3 17:04:43 2014
From: sim.ple at live.nl (Randy S)
Date: Wed, 3 Dec 2014 18:04:43 +0100
Subject: [OmniOS-discuss] test omnios on OI machine
In-Reply-To: <EDE6C088-3821-41B1-A947-97DB7D46AE05@richardelling.com>
References: <DUB124-W4FAE9E8C04253565F247FE07A0@phx.gbl>, <,
	<13009DC9-7FA5-4938-8042-F2AF74DDC695@omniti.com> <>>,
	<DUB124-W31A653A65DB219489C3FBAE07A0@phx.gbl>, ,
	<63802B04-B4E9-4FC6-8059-EDAD86FBADBE@omniti.com>,
	<DUB124-W28749FC3157E8298D365CAE07B0@phx.gbl>
	<DUB124-W15E9AE203C73CD4F0BBB3FE07B0@phx.gbl>,
	<EDE6C088-3821-41B1-A947-97DB7D46AE05@richardelling.com>
Message-ID: <DUB124-W114E0BF69CB86241E14344E07B0@phx.gbl>

Thank you very much for the hint, indeed helpful.

Subject: Re: [OmniOS-discuss] test omnios on OI machine
From: richard.elling at richardelling.com
Date: Wed, 3 Dec 2014 08:58:43 -0800
CC: omnios-discuss at lists.omniti.com
To: sim.ple at live.nl

I'm glad you got it running ok. Helpful hint below...
On Dec 3, 2014, at 2:37 AM, Randy S <sim.ple at live.nl> wrote:



Anybody else had problems with two or more sas hba's in an omnios system ?
How was this solved?

From: sim.ple at live.nl
To: omnios-discuss at lists.omniti.com
Date: Wed, 3 Dec 2014 01:45:12 +0100
Subject: Re: [OmniOS-discuss] test omnios on OI machine




Hi, was indeed intended for the list.

dmesg | grep mptsas

dmesg is brain dead. All it does is (from the script itself, /usr/bin/dmesg):	/usr/bin/cat -sv `/usr/bin/ls -tr1 /var/adm/messages.? 2>/dev/null` \
		/var/adm/messages | /usr/bin/tail -200
to show you the last 200 lines in the messages file. For a large system, there can easily be 1000+ lines of boot-time messages. You're better off looking at the messagesfile directly:	grep mptsas /var/adm/messages
For a better method of identifying loaded drivers, use prtconf:	prtconf -Dshows the device tree and the attached drivers.
HTH -- richard

gives nothing (no output). 

Hope you have an idea of what to do next. 

Going to test a current omnios with one hba by adding another to it to see what happens tomorrow.  

Will be testing with new Dell hardware and solarflare cards probably in the comming weeks also. If you are interrested, will keep you posted. 



> Subject: Re: [OmniOS-discuss] test omnios on OI machine
> From: danmcd at omniti.com
> Date: Tue, 2 Dec 2014 19:22:15 -0500
> CC: danmcd at omniti.com
> To: sim.ple at live.nl
> 
> Please share these with the community in the future, unless you're sharing confidential data of some sort.
> 
> 
> > On Dec 2, 2014, at 6:59 PM, Randy S <sim.ple at live.nl> wrote:
> > 
> > Hi Dan,
> > 
> > It's been a while...
> > 
> > I wanted to stay on r10 since that has been in use by quite a community for quite a while now. R12 is still new so usually unforseen hicups can be expected. (We are not frontrunners regarding systems intended for serious use  ;-) ). Have been testing omnios for quite a while on systems with one hba now. All was satisfactory. 
> > 
> > forgot to mention, I did do an fmdump -ev and saw 
> > Dec 02 23:32:37.1675 ereport.fs.zfs.vdev.open_failed       0x0077a03af6b00001
> > Dec 02 23:32:37.1675 ereport.fs.zfs.vdev.open_failed       0x0077a03af6b00001
> > Dec 02 23:35:17.6783 ereport.sensor.failure                0x02cd95a4cde05801
> > Dec 02 23:35:17.6868 ereport.sensor.failure                0x02cd9dbb53605801
> > 
> > I guess this is because the one controller cannot be found.
> 
> That's a fair assessment.
> 
> > dmesg | grep mpt_sa (done just now (not sleepy yet)) gives :
> > 
> > Dec  3 00:42:44 omni scsi: [ID 583861 kern.info] sd68 at mpt_sas3: unit-address w5000c50057ad9161,0: w5000c50057ad9161,0
> > Dec  3 00:42:44 omni scsi: [ID 583861 kern.info] sd69 at mpt_sas3: unit-address w5000c50057adb269,0: w5000c50057adb269,0
> > Dec  3 00:42:45 omni scsi: [ID 583861 kern.info] sd70 at mpt_sas3: unit-address w5000c50057adea91,0: w5000c50057adea91,0
> > Dec  3 00:42:45 omni scsi: [ID 583861 kern.info] ses2 at mpt_sas3: unit-address w5003048000b18d3d,0: w5003048000b18d3d,0
> > Dec  3 00:42:45 omni scsi: [ID 583861 kern.info] mpt_sas4 at mpt_sas0: scsi-iport 20
> > Dec  3 00:42:45 omni genunix: [ID 936769 kern.info] mpt_sas4 is /pci at 0,0/pci8086,3c04 at 2/pci1000,3030 at 0/iport at 20
> > Dec  3 00:42:45 omni genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,3c04 at 2/pci1000,3030 at 0/iport at 20 (mpt_sas4) online
> > Dec  3 00:42:45 omni scsi: [ID 583861 kern.info] sd71 at mpt_sas4: unit-address w5001517bb27f79f7,0: w5001517bb27f79f7,0
> > Dec  3 00:42:45 omni scsi: [ID 583861 kern.info] mpt_sas5 at mpt_sas0: scsi-iport v0
> > Dec  3 00:42:45 omni genunix: [ID 936769 kern.info] mpt_sas5 is /pci at 0,0/pci8086,3c04 at 2/pci1000,3030 at 0/iport at v0
> > Dec  3 00:42:45 omni genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,3c04 at 2/pci1000,3030 at 0/iport at v0 (mpt_sas5) online
> 
> Interesting... note that two mpt_sas controllers (4 and 5) came online.  I wonder why one isn't showing up?  Grep for "mptsas" as well, just in case?
> 
> Dan
> 
 		 	   		  

_______________________________________________
OmniOS-discuss mailing list
OmniOS-discuss at lists.omniti.com
http://lists.omniti.com/mailman/listinfo/omnios-discuss 		 	   		  
_______________________________________________
OmniOS-discuss mailing list
OmniOS-discuss at lists.omniti.com
http://lists.omniti.com/mailman/listinfo/omnios-discuss

 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141203/0dceac33/attachment-0001.html>

From steve at linuxsuite.org  Wed Dec  3 17:06:40 2014
From: steve at linuxsuite.org (steve at linuxsuite.org)
Date: Wed, 3 Dec 2014 12:06:40 -0500
Subject: [OmniOS-discuss] Missing device but format shows it exists?
Message-ID: <5ec7cb4824606ee7cd7c719281608b1c.squirrel@emailmg.netfirms.com>


 Howdy!

          This seems wierd....  I am adding cache and log mirrors and
everything
works fine but then their is a missing device, but format shows the partition
is there. devfsadmn does not add the device file.   d0s0 - d0s6 and d0s8-d015
are there but d0s7  is missing?

 thoughts?

   -steve



# zpool add dfs2_data8 cache c1t5d0s7
cannot open 'c1t5d0s7': no such device in /dev/dsk
must be a full path or shorthand device name

root at live-dfs-2:~# ls /dev/dsk/c1t5d0s7
ls: cannot access /dev/dsk/c1t5d0s7: No such file or directory

root at live-dfs-2:~# ls /dev/dsk/c1t5d0s*
/dev/dsk/c1t5d0s0
/dev/dsk/c1t5d0s1
 /dev/dsk/c1t5d0s2
 /dev/dsk/c1t5d0s3
/dev/dsk/c1t5d0s4
/dev/dsk/c1t5d0s5
 /dev/dsk/c1t5d0s6
 /dev/dsk/c1t5d0s8
/dev/dsk/c1t5d0s9
/dev/dsk/c1t5d0s10
 /dev/dsk/c1t5d0s11
 /dev/dsk/c1t5d0s12
 /dev/dsk/c1t5d0s13
 /dev/dsk/c1t5d0s14
 /dev/dsk/c1t5d0s15


 # format -e
Specify disk (enter its number): 4
selecting c1t5d0
[disk formatted]
/dev/dsk/c1t5d0s0 is in use as a cache device for ZFS pool dfs2_data1. 
Please see zpool(1M).
/dev/dsk/c1t5d0s1 is in use as a cache device for ZFS pool dfs2_data2. 
Please see zpool(1M).
/dev/dsk/c1t5d0s2 is in use as a cache device for ZFS pool dfs2_data3. 
Please see zpool(1M).
/dev/dsk/c1t5d0s3 is in use as a cache device for ZFS pool dfs2_data4. 
Please see zpool(1M).
/dev/dsk/c1t5d0s4 is in use as a cache device for ZFS pool dfs2_data5. 
Please see zpool(1M).
/dev/dsk/c1t5d0s5 is in use as a cache device for ZFS pool dfs2_data6. 
Please see zpool(1M).
/dev/dsk/c1t5d0s6 is in use as a cache device for ZFS pool dfs2_data7. 
Please see zpool(1M).


format> ver

Volume name = <        >
ascii name  = <DELL-PERC 6/i-1.22-111.25GB>
bytes/sector    =  512
sectors = 233308159
accessible sectors = 233308126
Part      Tag    Flag     First Sector         Size         Last Sector
  0        usr    wm               256       12.00GB          25166079
  1        usr    wm          25166080       12.00GB          50331903
  2        usr    wm          50331904       12.00GB          75497727
  3        usr    wm          75497728       12.00GB          100663551
  4        usr    wm         100663552       12.00GB          125829375
  5        usr    wm         125829376       12.00GB          150995199
  6        usr    wm         150995200       12.00GB          176161023
  7        usr    wm         176161024       12.00GB          201326847
  8   reserved    wm         233291743        8.00MB          233308126

format>



From jboren at drakecooper.com  Thu Dec  4 00:10:29 2014
From: jboren at drakecooper.com (Joseph Boren)
Date: Wed, 3 Dec 2014 17:10:29 -0700
Subject: [OmniOS-discuss] anyone doing DirectPath I/O?
Message-ID: <CAGoaUhO9kRCopXH5+wojk4U-ict8=JpbwS9U9RtBxcO86F6_zQ@mail.gmail.com>

Greetings,

I was wondering if anyone was using DirectPath, specifically for exclusive
use of a drive controller and attached drives to a specific VM.    I have a
use case that would seem to be a good fit for this, so I played around with
a couple of RAID controllers I had, and was able to get one (3ware 9650se)
configured for directpath, but none of the drives attached would show to
OmniOS, regardless of how I configured them in the raid bois (JBOD,
individual disks, etc).  I know that controller is poorly supported and I
was curious if anyone was using DirectPath  this way in production and what
kind of drive controller/HBA/whatever was working.  Also any "for the love
of god don't do it this way" scenarios?  I seem to be really adept at
finding and trying those out first....

Thanks a ton and best regards,

-jb-
*Joseph Boren*

IT Specialist
*DRAKE COOPER*
+ c: (208) 891-2128 + o: (208) 342-0925
+ 416 S. 8th St., Boise, ID 83702
+ w: drakecooper.com + f: /drakecooper <http://facebook.com/drakecooper> +
t: @drakecooper <http://twitter.com/drakecooper>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141203/48b18a97/attachment.html>

From chip at innovates.com  Thu Dec  4 03:01:51 2014
From: chip at innovates.com (Schweiss, Chip)
Date: Wed, 3 Dec 2014 21:01:51 -0600
Subject: [OmniOS-discuss] anyone doing DirectPath I/O?
In-Reply-To: <CAGoaUhO9kRCopXH5+wojk4U-ict8=JpbwS9U9RtBxcO86F6_zQ@mail.gmail.com>
References: <CAGoaUhO9kRCopXH5+wojk4U-ict8=JpbwS9U9RtBxcO86F6_zQ@mail.gmail.com>
Message-ID: <CALeZrrTCE=N5arT7_LFCLex5NJXyxMugNwWrgO_mbmOaHXeEcQ@mail.gmail.com>

I've been using DirectPath with ESXi 5.0 with an LSI HBA for almost 2 years
to an OpenIndiana server.  It's been very stable:

root at zfs01:~# uptime
 21:01pm  up 649 days  6:58,  2 users,  load average: 0.46, 0.30, 0.25


On Wed, Dec 3, 2014 at 6:10 PM, Joseph Boren <jboren at drakecooper.com> wrote:

> Greetings,
>
> I was wondering if anyone was using DirectPath, specifically for exclusive
> use of a drive controller and attached drives to a specific VM.    I have a
> use case that would seem to be a good fit for this, so I played around with
> a couple of RAID controllers I had, and was able to get one (3ware 9650se)
> configured for directpath, but none of the drives attached would show to
> OmniOS, regardless of how I configured them in the raid bois (JBOD,
> individual disks, etc).  I know that controller is poorly supported and I
> was curious if anyone was using DirectPath  this way in production and what
> kind of drive controller/HBA/whatever was working.  Also any "for the love
> of god don't do it this way" scenarios?  I seem to be really adept at
> finding and trying those out first....
>
> Thanks a ton and best regards,
>
> -jb-
> *Joseph Boren*
>
> IT Specialist
> *DRAKE COOPER*
> + c: (208) 891-2128 + o: (208) 342-0925
> + 416 S. 8th St., Boise, ID 83702
> + w: drakecooper.com + f: /drakecooper <http://facebook.com/drakecooper> +
>  t: @drakecooper <http://twitter.com/drakecooper>
>
>
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141203/b8b6fba8/attachment.html>

From gate03 at landcroft.co.uk  Thu Dec  4 05:30:51 2014
From: gate03 at landcroft.co.uk (Michael Mounteney)
Date: Thu, 4 Dec 2014 15:30:51 +1000
Subject: [OmniOS-discuss] adding cua/a as a second login
Message-ID: <20141204153051.3e17ac8f@punda-mlia>

There's plenty of stuff about configuring the serial port as the
console on login, but I would like to ADD the serial port as a second
login;  that is, keep the normal console login but have cua/a as a
second login access point.  Is this possible ?  Something to do with
SAF on Solaris, IIRC.

Michael.

From tim at multitalents.net  Thu Dec  4 07:21:54 2014
From: tim at multitalents.net (Tim Rice)
Date: Wed, 3 Dec 2014 23:21:54 -0800 (PST)
Subject: [OmniOS-discuss] adding cua/a as a second login
In-Reply-To: <20141204153051.3e17ac8f@punda-mlia>
References: <20141204153051.3e17ac8f@punda-mlia>
Message-ID: <alpine.UW2.2.11.1412032317360.27489@server01.int.multitalents.net>

On Thu, 4 Dec 2014, Michael Mounteney wrote:

> There's plenty of stuff about configuring the serial port as the
> console on login, but I would like to ADD the serial port as a second
> login;  that is, keep the normal console login but have cua/a as a
> second login access point.  Is this possible ?  Something to do with
> SAF on Solaris, IIRC.
> 
> Michael.

Yes, read the saf, sacadm, and pmadm man pages.
The default install will probably already have a ttymon running.
......
# sacadm -l
PMTAG          PMTYPE         FLGS RCNT STATUS     COMMAND
zsmon          ttymon         -    0    ENABLED    /usr/lib/saf/ttymon #
......

And here are the ports in zsmon
.......
# pmadm -l
PMTAG          PMTYPE         SVCTAG         FLGS ID       <PMSPECIFIC>
zsmon          ttymon         ttya           u    root     /dev/term/a I - /usr/bin/login - 9600 ldterm,ttcompat ttya login:  - tvi925 y  #
zsmon          ttymon         ttyb           u    root     /dev/term/b I - /usr/bin/login - 9600 ldterm,ttcompat ttyb login:  - tvi925 y  #
.......

-- 
Tim Rice				Multitalents	(707) 456-1146
tim at multitalents.net



From tim at multitalents.net  Thu Dec  4 07:11:04 2014
From: tim at multitalents.net (Tim Rice)
Date: Wed, 3 Dec 2014 23:11:04 -0800 (PST)
Subject: [OmniOS-discuss] anyone doing DirectPath I/O?
In-Reply-To: <CALeZrrTCE=N5arT7_LFCLex5NJXyxMugNwWrgO_mbmOaHXeEcQ@mail.gmail.com>
References: <CAGoaUhO9kRCopXH5+wojk4U-ict8=JpbwS9U9RtBxcO86F6_zQ@mail.gmail.com>
	<CALeZrrTCE=N5arT7_LFCLex5NJXyxMugNwWrgO_mbmOaHXeEcQ@mail.gmail.com>
Message-ID: <alpine.UW2.2.11.1412032251490.27489@server01.int.multitalents.net>

On Wed, 3 Dec 2014, Schweiss, Chip wrote:

| I've been using DirectPath with ESXi 5.0 with an LSI HBA for almost 2 years
| to an OpenIndiana server.  It's been very stable:

My all in one box (Supermicro MBD-X9SCM-F-0) was on ESXi 5.1 for
about a year with a LSI SAS3801E and a LSI SAS9211-4i. I've since
updated to ESXi 5.5 update2. The OmniOS r151006 VM provides NFS
storage to the ESXi host for (currently) 15 running VMS as well
as well as NFS home directories for various other computers around
the office. Other than drives failing, it's been working reasonably
well for what it is.

| 
| root at zfs01:~# uptime
|  21:01pm  up 649 days  6:58,  2 users,  load average: 0.46, 0.30, 0.25
| 
| 
| On Wed, Dec 3, 2014 at 6:10 PM, Joseph Boren <jboren at drakecooper.com> wrote:
| 
| > Greetings,
| >
| > I was wondering if anyone was using DirectPath, specifically for exclusive
| > use of a drive controller and attached drives to a specific VM.    I have a
| > use case that would seem to be a good fit for this, so I played around with
| > a couple of RAID controllers I had, and was able to get one (3ware 9650se)
| > configured for directpath, but none of the drives attached would show to
| > OmniOS, regardless of how I configured them in the raid bois (JBOD,
| > individual disks, etc).  I know that controller is poorly supported and I
| > was curious if anyone was using DirectPath  this way in production and what
| > kind of drive controller/HBA/whatever was working.  Also any "for the love
| > of god don't do it this way" scenarios?  I seem to be really adept at
| > finding and trying those out first....
| >
| > Thanks a ton and best regards,
| >
| > -jb-
| > *Joseph Boren*
| >
| > IT Specialist
| > *DRAKE COOPER*
| > + c: (208) 891-2128 + o: (208) 342-0925
| > + 416 S. 8th St., Boise, ID 83702
| > + w: drakecooper.com + f: /drakecooper <http://facebook.com/drakecooper> +
| >  t: @drakecooper <http://twitter.com/drakecooper>
| >
| >
| > _______________________________________________
| > OmniOS-discuss mailing list
| > OmniOS-discuss at lists.omniti.com
| > http://lists.omniti.com/mailman/listinfo/omnios-discuss
| >
| >
| 

-- 
Tim Rice				Multitalents	(707) 456-1146
tim at multitalents.net



From sjorge+ml at blackdot.be  Thu Dec  4 08:35:04 2014
From: sjorge+ml at blackdot.be (Jorge Schrauwen)
Date: Thu, 04 Dec 2014 09:35:04 +0100
Subject: [OmniOS-discuss] adding cua/a as a second login
In-Reply-To: <20141204153051.3e17ac8f@punda-mlia>
References: <20141204153051.3e17ac8f@punda-mlia>
Message-ID: <dde4299711c6bd8e1f3a24a3fd0b91ce@blackdot.be>

Hey,

I have this setup on my SuperMicro server to have a 2nd console on the 
SoL.
I documented it here: 
https://docu.blackdot.be/snipets/solaris/misc-serial-console

Hopefully this is helpful.

Regards

Jorge



On 2014-12-04 06:30, Michael Mounteney wrote:
> There's plenty of stuff about configuring the serial port as the
> console on login, but I would like to ADD the serial port as a second
> login;  that is, keep the normal console login but have cua/a as a
> second login access point.  Is this possible ?  Something to do with
> SAF on Solaris, IIRC.
> 
> Michael.
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss

From sim.ple at live.nl  Thu Dec  4 10:51:53 2014
From: sim.ple at live.nl (Randy S)
Date: Thu, 4 Dec 2014 11:51:53 +0100
Subject: [OmniOS-discuss] test omnios on OI machine / ehh
In-Reply-To: <DUB124-W114E0BF69CB86241E14344E07B0@phx.gbl>
References: <DUB124-W4FAE9E8C04253565F247FE07A0@phx.gbl>,
	<,,<13009DC9-7FA5-4938-8042-F2AF74DDC695@omniti.com>
	<>>,,<DUB124-W31A653A65DB219489C3FBAE07A0@phx.gbl>,
	, , <63802B04-B4E9-4FC6-8059-EDAD86FBADBE@omniti.com>, ,
	<DUB124-W28749FC3157E8298D365CAE07B0@phx.gbl>,
	<DUB124-W15E9AE203C73CD4F0BBB3FE07B0@phx.gbl>, ,
	<EDE6C088-3821-41B1-A947-97DB7D46AE05@richardelling.com>,
	<DUB124-W114E0BF69CB86241E14344E07B0@phx.gbl>
Message-ID: <DUB124-W4285BB313B6DD742A8F0AE0780@phx.gbl>

Did a cold reboot and controller = seen
I did my stuff. After a while I did a devfsadm -Cv , removed path_to_inst and a reboot -- -r and controller = gone

It's indeed the c11 (see previous message), prtconf -c configure c11 , does nothing (causes no log messages either).
Is there a way to configure this manually without having to reboot?

The strange thing is that the OI has no problem with this at all. 

R

From: sim.ple at live.nl
To: omnios-discuss at lists.omniti.com
Date: Wed, 3 Dec 2014 18:04:43 +0100
Subject: Re: [OmniOS-discuss] test omnios on OI machine




Thank you very much for the hint, indeed helpful.

Subject: Re: [OmniOS-discuss] test omnios on OI machine
From: richard.elling at richardelling.com
Date: Wed, 3 Dec 2014 08:58:43 -0800
CC: omnios-discuss at lists.omniti.com
To: sim.ple at live.nl

I'm glad you got it running ok. Helpful hint below...
On Dec 3, 2014, at 2:37 AM, Randy S <sim.ple at live.nl> wrote:



Anybody else had problems with two or more sas hba's in an omnios system ?
How was this solved?

From: sim.ple at live.nl
To: omnios-discuss at lists.omniti.com
Date: Wed, 3 Dec 2014 01:45:12 +0100
Subject: Re: [OmniOS-discuss] test omnios on OI machine




Hi, was indeed intended for the list.

dmesg | grep mptsas

dmesg is brain dead. All it does is (from the script itself, /usr/bin/dmesg):	/usr/bin/cat -sv `/usr/bin/ls -tr1 /var/adm/messages.? 2>/dev/null` \
		/var/adm/messages | /usr/bin/tail -200
to show you the last 200 lines in the messages file. For a large system, there can easily be 1000+ lines of boot-time messages. You're better off looking at the messagesfile directly:	grep mptsas /var/adm/messages
For a better method of identifying loaded drivers, use prtconf:	prtconf -Dshows the device tree and the attached drivers.
HTH -- richard

gives nothing (no output). 

Hope you have an idea of what to do next. 

Going to test a current omnios with one hba by adding another to it to see what happens tomorrow.  

Will be testing with new Dell hardware and solarflare cards probably in the comming weeks also. If you are interrested, will keep you posted. 



> Subject: Re: [OmniOS-discuss] test omnios on OI machine
> From: danmcd at omniti.com
> Date: Tue, 2 Dec 2014 19:22:15 -0500
> CC: danmcd at omniti.com
> To: sim.ple at live.nl
> 
> Please share these with the community in the future, unless you're sharing confidential data of some sort.
> 
> 
> > On Dec 2, 2014, at 6:59 PM, Randy S <sim.ple at live.nl> wrote:
> > 
> > Hi Dan,
> > 
> > It's been a while...
> > 
> > I wanted to stay on r10 since that has been in use by quite a community for quite a while now. R12 is still new so usually unforseen hicups can be expected. (We are not frontrunners regarding systems intended for serious use  ;-) ). Have been testing omnios for quite a while on systems with one hba now. All was satisfactory. 
> > 
> > forgot to mention, I did do an fmdump -ev and saw 
> > Dec 02 23:32:37.1675 ereport.fs.zfs.vdev.open_failed       0x0077a03af6b00001
> > Dec 02 23:32:37.1675 ereport.fs.zfs.vdev.open_failed       0x0077a03af6b00001
> > Dec 02 23:35:17.6783 ereport.sensor.failure                0x02cd95a4cde05801
> > Dec 02 23:35:17.6868 ereport.sensor.failure                0x02cd9dbb53605801
> > 
> > I guess this is because the one controller cannot be found.
> 
> That's a fair assessment.
> 
> > dmesg | grep mpt_sa (done just now (not sleepy yet)) gives :
> > 
> > Dec  3 00:42:44 omni scsi: [ID 583861 kern.info] sd68 at mpt_sas3: unit-address w5000c50057ad9161,0: w5000c50057ad9161,0
> > Dec  3 00:42:44 omni scsi: [ID 583861 kern.info] sd69 at mpt_sas3: unit-address w5000c50057adb269,0: w5000c50057adb269,0
> > Dec  3 00:42:45 omni scsi: [ID 583861 kern.info] sd70 at mpt_sas3: unit-address w5000c50057adea91,0: w5000c50057adea91,0
> > Dec  3 00:42:45 omni scsi: [ID 583861 kern.info] ses2 at mpt_sas3: unit-address w5003048000b18d3d,0: w5003048000b18d3d,0
> > Dec  3 00:42:45 omni scsi: [ID 583861 kern.info] mpt_sas4 at mpt_sas0: scsi-iport 20
> > Dec  3 00:42:45 omni genunix: [ID 936769 kern.info] mpt_sas4 is /pci at 0,0/pci8086,3c04 at 2/pci1000,3030 at 0/iport at 20
> > Dec  3 00:42:45 omni genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,3c04 at 2/pci1000,3030 at 0/iport at 20 (mpt_sas4) online
> > Dec  3 00:42:45 omni scsi: [ID 583861 kern.info] sd71 at mpt_sas4: unit-address w5001517bb27f79f7,0: w5001517bb27f79f7,0
> > Dec  3 00:42:45 omni scsi: [ID 583861 kern.info] mpt_sas5 at mpt_sas0: scsi-iport v0
> > Dec  3 00:42:45 omni genunix: [ID 936769 kern.info] mpt_sas5 is /pci at 0,0/pci8086,3c04 at 2/pci1000,3030 at 0/iport at v0
> > Dec  3 00:42:45 omni genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,3c04 at 2/pci1000,3030 at 0/iport at v0 (mpt_sas5) online
> 
> Interesting... note that two mpt_sas controllers (4 and 5) came online.  I wonder why one isn't showing up?  Grep for "mptsas" as well, just in case?
> 
> Dan
> 
 		 	   		  

_______________________________________________
OmniOS-discuss mailing list
OmniOS-discuss at lists.omniti.com
http://lists.omniti.com/mailman/listinfo/omnios-discuss 		 	   		  
_______________________________________________
OmniOS-discuss mailing list
OmniOS-discuss at lists.omniti.com
http://lists.omniti.com/mailman/listinfo/omnios-discuss

 		 	   		  

_______________________________________________
OmniOS-discuss mailing list
OmniOS-discuss at lists.omniti.com
http://lists.omniti.com/mailman/listinfo/omnios-discuss 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141204/a7541130/attachment.html>

From sim.ple at live.nl  Thu Dec  4 12:16:47 2014
From: sim.ple at live.nl (Randy S)
Date: Thu, 4 Dec 2014 13:16:47 +0100
Subject: [OmniOS-discuss] Upgrading from OI to OmniOS
In-Reply-To: <DUB124-W40839C8BAE4785656A8636E0780@phx.gbl>
References: <547DD116.7040401@blue-bolt.com>, ,
	<DUB124-W9CEC7807C3A19C2612FEDE07B0@phx.gbl>,
	<DUB124-W52130E84F44FC86051EE9FE07B0@phx.gbl>,
	<547F4668.9040309@osn.de>,
	<DUB124-W40839C8BAE4785656A8636E0780@phx.gbl>
Message-ID: <DUB124-W29010B848697C5D5B7B0F1E0780@phx.gbl>



From: sim.ple at live.nl
To: jg at osn.de
Subject: RE: [OmniOS-discuss] Upgrading from OI to OmniOS
Date: Thu, 4 Dec 2014 12:45:10 +0100




Hi,
 Not a stupid question at all.
I fought with this for quite a while. I did it in quite an elaborate way and I am certain this can be done in fewer steps so I hope that one of the guru's in here can step in to correct my way of doing this.
Perhaps with a pkg install entire sort of a command ?

But anyway the following works for me :

On a vm I install the Omnios vm  and configure it
In this vm I do a beadm create BEname.
Do:
beadm mount BEname /mnt

create a compressed file of this BE

tar --create --bzip2 --absolute-names 
--preserve-permissions --preserve-order --verbose 
--file=/tmp/omni.tar.bz2 
/mnt

Copy this to the machine where you want to install to e.g. /tmp

On the destination (vm)machine create a BE ( beadm create whatever and also mount it on e.g /mnt)
cd to /mnt on the destination and rm -rf *

untar the bz2 file (cd /tmp) :

tar --extract --bzip2 --absolute-names --preserve-permissions --same-owner --verbose  --file=omni.tar.bz2

This will populate the /mnt with the omnios be
On this machine (destination) I also do a  

:;zpool status rpool 

to identify the disks used by rpool and then do e.g. 

ls -l /dev/dsk/c4t50014EE002EC010Bd0s0

and register the physical device in the bootenv.rc file of the omnios

:;vi /mnt/boot/solaris/bootenv.rc

and add e.g. lines like:

setprop bootpath /scsi_vhci/disk at g50014ee002ec010b:asetprop altbootpath /scsi_vhci/disk at g50014ee0ad96d82a:a <== in case of mirrorsetprop bootfile kernel/amd64/unix
After this do:
:;/usr/sbin/devfsadm -r /mnt

Now you can reboot into your new Omni bootenv and clean stuff up. Again do not upgrade zfs version until satiesfied.

I am certain this can be a lot better but this works 4 me (being an autodidact)

HTH
And as others usually end their helpful hints,... typo's courtesy of ehh my keyboard...

R



> Date: Wed, 3 Dec 2014 18:20:40 +0100
> From: jg at osn.de
> To: sim.ple at live.nl
> Subject: Re: [OmniOS-discuss] Upgrading from OI to OmniOS
> 
> Hi Randy,
> 
> sorry this may be a stupid question, but how can I install
> a fresh Omnios in an existing bootenv? Or how do you create
> the bootenv? bootadm create ?
> 
> Kind regards,
> 
>   - Joerg
> 
> On 03.12.2014 13:44, Randy S wrote:
> > should have send this to the list.... again.... ;-)
> >
> > ------------------------------------------------------------------------
> >
> >
> > Hi,
> >
> > The safest way I could think of to do this is as follows:
> > create a bootenv in OI en install Omnios fresh into this
> > Start omnios and if you need any settings you can mount the be of OI to
> > e.g. /mnt and copy the config files from there to the Omnios install.
> >
> > I had to keep track of config files in /etc and /kernel/drv
> > I have noticed that both oses are quite compatible
> >
> > Above procedure can first be tested with vm's
> >
> > Do not upgrade any pools untill you are satisfied with the new situation
> > since there might be a diff in zfs version. If you do you won't be able
> > to go back to your OI install.
> >
> > Please first test with vm's!
> >
> > Rgds
> >
> > R
> >
> >
> >
> >> Date: Tue, 2 Dec 2014 14:47:50 +0000
> >> From: cal-s at blue-bolt.com
> >> To: omnios-discuss at lists.omniti.com
> >> Subject: [OmniOS-discuss] Upgrading from OI to OmniOS
> >>
> >> Hi
> >>
> >> I have a server hosting 65TB of RAIDZ2 storage that's currently running
> >> OI (oi_151a8), which i really want to get off of given that OI has
> >> bitten the dust support-wise.  I found one brief discussion on the topic
> >> but it wasn't terribly detailed and was missing critical steps like
> >> configuration protection/retention. I'd like to preserve network and esp
> >> LDAP (client) settings as they were kind of a pain to establish in the
> >> first instance for quick reapplication. If anyone has or is interested
> >> in cobbling together a checklist of steps to upgrade OI to OmniOS that
> >> takes configuration into account, i'd very much appreciate seeing it.
> >>
> >> thank you
> >>
> >> - cal sawyer
> >> _______________________________________________
> >> OmniOS-discuss mailing list
> >> OmniOS-discuss at lists.omniti.com
> >> http://lists.omniti.com/mailman/listinfo/omnios-discuss
> >
> >
> > _______________________________________________
> > OmniOS-discuss mailing list
> > OmniOS-discuss at lists.omniti.com
> > http://lists.omniti.com/mailman/listinfo/omnios-discuss
> >
> 
> -- 
> OSN Online Service Nuernberg GmbH, Bucher Str. 78, 90408 Nuernberg
> Tel: +49 911 39905-0 - Fax: +49 911 39905-55 - http://www.osn.de
> HRB 15022 Nuernberg, USt-Id: DE189301263, GF: Joerg Goltermann
 		 	   		   		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141204/99f5e518/attachment-0001.html>

From jklimek at gmail.com  Thu Dec  4 18:20:04 2014
From: jklimek at gmail.com (John Klimek)
Date: Thu, 4 Dec 2014 13:20:04 -0500
Subject: [OmniOS-discuss] NFSv4 id mapping only working on client but not
	server?
Message-ID: <CAOEv1kYdT=Cj-K7KarcEFizXKe8g8aehKm5uaHORqfqzAxUFoA@mail.gmail.com>

I'm running OmniOS (v5.11, omnios-10b9c79, September 2014) as my server and
Ubuntu 14.04 as my client.  I'm only using sys (?) authentication and not
Kerberos.

I've correctly set my NFS domain name but it's only partially working.

Here's an example:

Server:  user 'media', uid 1000.
Client:  user 'media', uid 2000.

If I create a file as 'media' on the server, the client correctly maps the
id to the local 'media' user.  However, if I create a file on the client,
the server will show a uid and doesn't correctly map to the local 'media'
uid.

What can I have setup wrong?

I can't find any debugging or logging options for nfsmapid...

Also, does the uidmap and gidmap share.nfs options only work for NFSv3?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141204/98099886/attachment.html>

From danmcd at omniti.com  Thu Dec  4 18:29:50 2014
From: danmcd at omniti.com (Dan McDonald)
Date: Thu, 4 Dec 2014 13:29:50 -0500
Subject: [OmniOS-discuss] NFSv4 id mapping only working on client but
	not server?
In-Reply-To: <CAOEv1kYdT=Cj-K7KarcEFizXKe8g8aehKm5uaHORqfqzAxUFoA@mail.gmail.com>
References: <CAOEv1kYdT=Cj-K7KarcEFizXKe8g8aehKm5uaHORqfqzAxUFoA@mail.gmail.com>
Message-ID: <CD8F3202-61E1-44EE-87E5-6C93A450FDBB@omniti.com>


> On Dec 4, 2014, at 1:20 PM, John Klimek <jklimek at gmail.com> wrote:
> 
> I'm running OmniOS (v5.11, omnios-10b9c79, September 2014) as my server and Ubuntu 14.04 as my client.  I'm only using sys (?) authentication and not Kerberos.
>  
> I've correctly set my NFS domain name but it's only partially working.
>  
> Here's an example:
>  
> Server:  user 'media', uid 1000.
> Client:  user 'media', uid 2000.
>  
> If I create a file as 'media' on the server, the client correctly maps the id to the local 'media' user.  However, if I create a file on the client, the server will show a uid and doesn't correctly map to the local 'media' uid.
>  
> What can I have setup wrong?
>  

Are you sharing using zfs(1M) sharing properties?  (zfs set sharenfs=on <filesystem>)

You need to make sure you set the uidmap property when you share your filesystem.  I believe, you want this:

	zfs set sharenfs='uidmap=2000:media' <filesystem>

Try that out for size.

Dan


From danmcd at omniti.com  Thu Dec  4 18:32:28 2014
From: danmcd at omniti.com (Dan McDonald)
Date: Thu, 4 Dec 2014 13:32:28 -0500
Subject: [OmniOS-discuss] NFSv4 id mapping only working on client but
	not server?
In-Reply-To: <CD8F3202-61E1-44EE-87E5-6C93A450FDBB@omniti.com>
References: <CAOEv1kYdT=Cj-K7KarcEFizXKe8g8aehKm5uaHORqfqzAxUFoA@mail.gmail.com>
	<CD8F3202-61E1-44EE-87E5-6C93A450FDBB@omniti.com>
Message-ID: <CDF9ECCE-BF4D-46FB-9B06-9B7D98EA8348@omniti.com>


> On Dec 4, 2014, at 1:29 PM, Dan McDonald <danmcd at omniti.com> wrote:
> 
> 
>> On Dec 4, 2014, at 1:20 PM, John Klimek <jklimek at gmail.com> wrote:
>> 
>> I'm running OmniOS (v5.11, omnios-10b9c79, September 2014) as my server and Ubuntu 14.04 as my client.  I'm only using sys (?) authentication and not Kerberos.
>> 
>> I've correctly set my NFS domain name but it's only partially working.
>> 
>> Here's an example:
>> 
>> Server:  user 'media', uid 1000.
>> Client:  user 'media', uid 2000.
>> 
>> If I create a file as 'media' on the server, the client correctly maps the id to the local 'media' user.  However, if I create a file on the client, the server will show a uid and doesn't correctly map to the local 'media' uid.
>> 
>> What can I have setup wrong?
>> 
> 
> Are you sharing using zfs(1M) sharing properties?  (zfs set sharenfs=on <filesystem>)
> 
> You need to make sure you set the uidmap property when you share your filesystem.  I believe, you want this:
> 
> 	zfs set sharenfs='uidmap=2000:media' <filesystem>

You may need to also specify the IP address of the particular client:

	zfs set sharenfs='uidmap=2000:media:<hostname>' <filesystem>

Dan


From jboren at drakecooper.com  Thu Dec  4 18:32:35 2014
From: jboren at drakecooper.com (Joseph Boren)
Date: Thu, 4 Dec 2014 11:32:35 -0700
Subject: [OmniOS-discuss] anyone doing DirectPath I/O?
In-Reply-To: <alpine.UW2.2.11.1412032251490.27489@server01.int.multitalents.net>
References: <CAGoaUhO9kRCopXH5+wojk4U-ict8=JpbwS9U9RtBxcO86F6_zQ@mail.gmail.com>
	<CALeZrrTCE=N5arT7_LFCLex5NJXyxMugNwWrgO_mbmOaHXeEcQ@mail.gmail.com>
	<alpine.UW2.2.11.1412032251490.27489@server01.int.multitalents.net>
Message-ID: <CAGoaUhMgR-babko710PfW2kyM0i_89LXD-Rvm7CRcgEQRFhZ9Q@mail.gmail.com>

Thanks guys, that's very helpful.

Tim that sounds very close to the use-case i had in my head.  I've run into
a couple of situations where a VM sitting on ESXi on particular hardware
was much more performant than the exact same system, bare-metal on the
exact same hardware.  Once I actually P2V'd a server to a temp location,
loaded ESXi on the physical server, migrated the P2V'd guest back and it
was crazy how much faster and stabler it was.  So now days I tend to want
to run ESXi on the bare-metal, but I didn't have any experience with
DirectPath, and it looked perfect for this situation.

Thanks again gentlemen, much appreciated.

Best regards,

-jb-
*Joseph Boren*

IT Specialist
*DRAKE COOPER*
+ c: (208) 891-2128 + o: (208) 342-0925
+ 416 S. 8th St., Boise, ID 83702
+ w: drakecooper.com + f: /drakecooper <http://facebook.com/drakecooper> +
t: @drakecooper <http://twitter.com/drakecooper>


On Thu, Dec 4, 2014 at 12:11 AM, Tim Rice <tim at multitalents.net> wrote:

> On Wed, 3 Dec 2014, Schweiss, Chip wrote:
>
> | I've been using DirectPath with ESXi 5.0 with an LSI HBA for almost 2
> years
> | to an OpenIndiana server.  It's been very stable:
>
> My all in one box (Supermicro MBD-X9SCM-F-0) was on ESXi 5.1 for
> about a year with a LSI SAS3801E and a LSI SAS9211-4i. I've since
> updated to ESXi 5.5 update2. The OmniOS r151006 VM provides NFS
> storage to the ESXi host for (currently) 15 running VMS as well
> as well as NFS home directories for various other computers around
> the office. Other than drives failing, it's been working reasonably
> well for what it is.
>
> |
> | root at zfs01:~# uptime
> |  21:01pm  up 649 days  6:58,  2 users,  load average: 0.46, 0.30, 0.25
> |
> |
> | On Wed, Dec 3, 2014 at 6:10 PM, Joseph Boren <jboren at drakecooper.com>
> wrote:
> |
> | > Greetings,
> | >
> | > I was wondering if anyone was using DirectPath, specifically for
> exclusive
> | > use of a drive controller and attached drives to a specific VM.    I
> have a
> | > use case that would seem to be a good fit for this, so I played around
> with
> | > a couple of RAID controllers I had, and was able to get one (3ware
> 9650se)
> | > configured for directpath, but none of the drives attached would show
> to
> | > OmniOS, regardless of how I configured them in the raid bois (JBOD,
> | > individual disks, etc).  I know that controller is poorly supported
> and I
> | > was curious if anyone was using DirectPath  this way in production and
> what
> | > kind of drive controller/HBA/whatever was working.  Also any "for the
> love
> | > of god don't do it this way" scenarios?  I seem to be really adept at
> | > finding and trying those out first....
> | >
> | > Thanks a ton and best regards,
> | >
> | > -jb-
> | > *Joseph Boren*
> | >
> | > IT Specialist
> | > *DRAKE COOPER*
> | > + c: (208) 891-2128 + o: (208) 342-0925
> | > + 416 S. 8th St., Boise, ID 83702
> | > + w: drakecooper.com + f: /drakecooper <
> http://facebook.com/drakecooper> +
> | >  t: @drakecooper <http://twitter.com/drakecooper>
> | >
> | >
> | > _______________________________________________
> | > OmniOS-discuss mailing list
> | > OmniOS-discuss at lists.omniti.com
> | > http://lists.omniti.com/mailman/listinfo/omnios-discuss
> | >
> | >
> |
>
> --
> Tim Rice                                Multitalents    (707) 456-1146
> tim at multitalents.net
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141204/3b51a0fc/attachment.html>

From mir at miras.org  Thu Dec  4 18:37:11 2014
From: mir at miras.org (Michael Rasmussen)
Date: Thu, 4 Dec 2014 19:37:11 +0100
Subject: [OmniOS-discuss] NFSv4 id mapping only working on client but
 not server?
In-Reply-To: <CAOEv1kYdT=Cj-K7KarcEFizXKe8g8aehKm5uaHORqfqzAxUFoA@mail.gmail.com>
References: <CAOEv1kYdT=Cj-K7KarcEFizXKe8g8aehKm5uaHORqfqzAxUFoA@mail.gmail.com>
Message-ID: <20141204193711.03524183@sleipner.datanom.net>

On Thu, 4 Dec 2014 13:20:04 -0500
John Klimek <jklimek at gmail.com> wrote:

> 
> What can I have setup wrong?
> 
> I can't find any debugging or logging options for nfsmapid...
> 
> Also, does the uidmap and gidmap share.nfs options only work for NFSv3?
Solaris and derivatives implementation of NFS ACL is not compliant to the Linux NFS ACL.
More directly it is the ACL in Linux which is not POSIX conformant so to avoid problems you
should add the mount option noacl in your Linux fstab file. Noacl will instruct Omnios NFS
to revert to plain old uid/gid.

-- 
Hilsen/Regards
Michael Rasmussen

Get my public GnuPG keys:
michael <at> rasmussen <dot> cc
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E
mir <at> datanom <dot> net
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C
mir <at> miras <dot> org
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917
--------------------------------------------------------------
/usr/games/fortune -es says:
Don't just echo the code with comments - make every comment count.
            - The Elements of Programming Style (Kernighan & Plaugher)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 181 bytes
Desc: OpenPGP digital signature
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141204/2ac0a1ac/attachment-0001.bin>

From jklimek at gmail.com  Thu Dec  4 18:55:11 2014
From: jklimek at gmail.com (John Klimek)
Date: Thu, 4 Dec 2014 13:55:11 -0500
Subject: [OmniOS-discuss] NFSv4 id mapping only working on client but
	not server?
In-Reply-To: <20141204193711.03524183@sleipner.datanom.net>
References: <CAOEv1kYdT=Cj-K7KarcEFizXKe8g8aehKm5uaHORqfqzAxUFoA@mail.gmail.com>
	<20141204193711.03524183@sleipner.datanom.net>
Message-ID: <CAOEv1kaoxj=AREpfuRo8saHFhW18i0CmgSd9dfwLZEtCUv8pmw@mail.gmail.com>

Thanks, I will give this a try.

However, when I create a file if I check the acl on Omni, it does have
extended permissions and looks correct.  It's only the uid and gid that I'm
trying to get mapped correctly.

Do you still think that noacl will solve that problem?

(I'm migrating some virtual machines so I can't try out the setting you
mentioned until a few more minutes...)

On Thu, Dec 4, 2014 at 1:37 PM, Michael Rasmussen <mir at miras.org> wrote:

> On Thu, 4 Dec 2014 13:20:04 -0500
> John Klimek <jklimek at gmail.com> wrote:
>
> >
> > What can I have setup wrong?
> >
> > I can't find any debugging or logging options for nfsmapid...
> >
> > Also, does the uidmap and gidmap share.nfs options only work for NFSv3?
> Solaris and derivatives implementation of NFS ACL is not compliant to the
> Linux NFS ACL.
> More directly it is the ACL in Linux which is not POSIX conformant so to
> avoid problems you
> should add the mount option noacl in your Linux fstab file. Noacl will
> instruct Omnios NFS
> to revert to plain old uid/gid.
>
> --
> Hilsen/Regards
> Michael Rasmussen
>
> Get my public GnuPG keys:
> michael <at> rasmussen <dot> cc
> http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E
> mir <at> datanom <dot> net
> http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C
> mir <at> miras <dot> org
> http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917
> --------------------------------------------------------------
> /usr/games/fortune -es says:
> Don't just echo the code with comments - make every comment count.
>             - The Elements of Programming Style (Kernighan & Plaugher)
>
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141204/cc61d5fd/attachment.html>

From mir at miras.org  Thu Dec  4 19:11:07 2014
From: mir at miras.org (Michael Rasmussen)
Date: Thu, 4 Dec 2014 20:11:07 +0100
Subject: [OmniOS-discuss] NFSv4 id mapping only working on client but
 not server?
In-Reply-To: <CAOEv1kaoxj=AREpfuRo8saHFhW18i0CmgSd9dfwLZEtCUv8pmw@mail.gmail.com>
References: <CAOEv1kYdT=Cj-K7KarcEFizXKe8g8aehKm5uaHORqfqzAxUFoA@mail.gmail.com>
	<20141204193711.03524183@sleipner.datanom.net>
	<CAOEv1kaoxj=AREpfuRo8saHFhW18i0CmgSd9dfwLZEtCUv8pmw@mail.gmail.com>
Message-ID: <20141204201107.43ad5928@sleipner.datanom.net>

On Thu, 4 Dec 2014 13:55:11 -0500
John Klimek <jklimek at gmail.com> wrote:

> Thanks, I will give this a try.
> 
> However, when I create a file if I check the acl on Omni, it does have
> extended permissions and looks correct.  It's only the uid and gid that I'm
> trying to get mapped correctly.
> 
> Do you still think that noacl will solve that problem?
> 
Yes, because you want to avoid Omnios presents ACL which is incompatible with Linux ACL.
Read my explanation of fix here:
http://forum.proxmox.com/threads/15793-CT-creation-on-NFS-Share?p=81530#post81530

-- 
Hilsen/Regards
Michael Rasmussen

Get my public GnuPG keys:
michael <at> rasmussen <dot> cc
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E
mir <at> datanom <dot> net
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C
mir <at> miras <dot> org
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917
--------------------------------------------------------------
/usr/games/fortune -es says:
<lilo> Fairlight: udp is the light margarine of tcp/ip transport protocols :)
		-- Seen on #Linux
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 181 bytes
Desc: OpenPGP digital signature
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141204/b7571aa4/attachment.bin>

From henson at acm.org  Thu Dec  4 20:38:19 2014
From: henson at acm.org (Paul B. Henson)
Date: Thu, 4 Dec 2014 12:38:19 -0800
Subject: [OmniOS-discuss] NFSv4 id mapping only working on client but
	not server?
In-Reply-To: <20141204193711.03524183@sleipner.datanom.net>
References: <CAOEv1kYdT=Cj-K7KarcEFizXKe8g8aehKm5uaHORqfqzAxUFoA@mail.gmail.com>
	<20141204193711.03524183@sleipner.datanom.net>
Message-ID: <4ce401d01002$3efbe920$bcf3bb60$@acm.org>

> From: Michael Rasmussen
> Sent: Thursday, December 04, 2014 10:37 AM
>
> > Also, does the uidmap and gidmap share.nfs options only work for NFSv3?
> Solaris and derivatives implementation of NFS ACL is not compliant to the
> Linux NFS ACL.

Linux supports NFSv4 ACLs perfectly fine, unless you have some requirement
to use an older version of the protocol, just mount with v4. You can use the
nfs4_editfacl, nfs4_getfacl, and nfs4_setfacl commands under linux to
manipulate ZFS ACLs over NFS.

 


From danmcd at omniti.com  Thu Dec  4 20:50:29 2014
From: danmcd at omniti.com (Dan McDonald)
Date: Thu, 4 Dec 2014 15:50:29 -0500
Subject: [OmniOS-discuss] OmniOS Bloody December 4th update
Message-ID: <879935CC-27DE-4F12-BB23-50DE4AC815B2@omniti.com>

New with this release (r151013-20141204):

- No change on the omnios-build master branch, still revision 31f07b8

- illumos-omnios master branch, revision aeccc33

- Many man page updates.

- TCP change that will positively affect off-link-peer performance (see illumos 5295)

- ZFS improvements and bugfixes, including better metadata caching in the ARC.

- mpt_sas(7d) now uses 64-bit DMA.

- SMB/CIFS bugfixes

- Built in Sun SSH supports EOW extension.

Install media and Kayak root & miniroot are also available.

Dan


From henson at acm.org  Thu Dec  4 20:59:56 2014
From: henson at acm.org (Paul B. Henson)
Date: Thu, 4 Dec 2014 12:59:56 -0800
Subject: [OmniOS-discuss] NFSv4 id mapping only working on client but
	not server?
In-Reply-To: <20141204201107.43ad5928@sleipner.datanom.net>
References: <CAOEv1kYdT=Cj-K7KarcEFizXKe8g8aehKm5uaHORqfqzAxUFoA@mail.gmail.com>	<20141204193711.03524183@sleipner.datanom.net>	<CAOEv1kaoxj=AREpfuRo8saHFhW18i0CmgSd9dfwLZEtCUv8pmw@mail.gmail.com>
	<20141204201107.43ad5928@sleipner.datanom.net>
Message-ID: <4ced01d01005$44363af0$cca2b0d0$@acm.org>

> From: Michael Rasmussen
> Sent: Thursday, December 04, 2014 11:11 AM
>
> Yes, because you want to avoid Omnios presents ACL which is incompatible
> with Linux ACL.

I don't believe the ACL has anything to do with NFSv4 id mapping? And a ZFS
ACL presented over NFSv4 is perfectly compatible with Linux. It's not a
Linux POSIX ACL, and cannot be manipulated with getfacl/setfacl, you need to
use the nfs4 ACL tools, but it works fine.

> http://forum.proxmox.com/threads/15793-CT-creation-on-NFS-
> Share?p=81530#post81530

In that thread, the user fails to chmod via NFS:

chmod: changing permissions of `/mnt/pve/proxCT/private/108.tmp': Operation
not permitted

The root cause of which was a setting of restricted for aclmode:

vdev1/proxCT aclmode             restricted
local

Per the man page "An aclmode property of restricted will cause the chmod(2)
operation to return an error when used on any file or directory which has a
non-trivial ACL whose entries can not be represented by a mode."

The user could have set the inherited ACL on the initial filesystem to a
trivial ACL, in which case chmod would've worked fine over NFS.

In any case, I don't see anything in that thread that seems relevant to
NFSv4 id mapping, which unless I misunderstand is the problem the OP is
trying to resolve.

On that subject, NFSv4 id mapping seems to be working fine for me between an
omnios client and server. On the server, the file system is mounted as:

/export/user/henson on export/user/henson
read/write/nosetuid/nodevices/nonbmand/exec/xattr/atime/dev=2c5025c

And exported as:

/export/user/henson     -       nfs     nosuid,sec=krb5i,sec=krb5p

with the domain set:

$ sharectl get -p nfsmapid_domain nfs 
nfsmapid_domain=csupomona.edu

if I create a file on the server, it has the correct ownership:

$ touch test_server
$ ls -l test_server 
-rw-r--r--+ 1 henson csupomona 0 Dec  4 12:50 test_server

on the client, the NFS export is mounted as:

/mnt on files-www.csupomona.edu:/export/user/henson
remote/read/write/setuid/devices/sec=krb5p/xattr/dev=85c0008 on Thu Dec  4
12:50:01 2014

the client has the same domain:

$ sharectl get -p nfsmapid_domain nfs 
nfsmapid_domain=csupomona.edu

The file created on the server shows up with the correct ownership:

$ ls -l test_server 
-rw-r--r--+ 1 henson csupomona 0 Dec  4 12:50 test_server

A file created on the client has the correct ownership:

$ touch test_client
$ ls -l test_client 
-rw-r--r--+ 1 henson csupomona 0 Dec  4 12:52 test_client

And viewed back on the server, still correct:

$ ls -l test_client 
-rw-r--r--+ 1 henson csupomona 0 Dec  4 12:52 test_client



From gate03 at landcroft.co.uk  Thu Dec  4 21:31:11 2014
From: gate03 at landcroft.co.uk (Michael Mounteney)
Date: Fri, 5 Dec 2014 07:31:11 +1000
Subject: [OmniOS-discuss] adding cua/a as a second login
In-Reply-To: <dde4299711c6bd8e1f3a24a3fd0b91ce@blackdot.be>
References: <20141204153051.3e17ac8f@punda-mlia>
	<dde4299711c6bd8e1f3a24a3fd0b91ce@blackdot.be>
Message-ID: <20141205073111.0762e2b1@punda-mlia>

On Thu, 04 Dec 2014 09:35:04 +0100
Jorge Schrauwen <sjorge+ml at blackdot.be> wrote:

> Hey,
> 
> I have this setup on my SuperMicro server to have a 2nd console on
> the SoL.
> I documented it here: 
> https://docu.blackdot.be/snipets/solaris/misc-serial-console
> 
> Hopefully this is helpful.  

It certainly is, Jorge;  thanks very much.  Working very well.  The
only thing I'd like to change is to limit root to logging in on the two
known ports /dev/console and /dev/ttya, but that's a small point.
Thank you for a very clear document.

Michael.

From henson at acm.org  Thu Dec  4 22:22:55 2014
From: henson at acm.org (Paul B. Henson)
Date: Thu, 4 Dec 2014 14:22:55 -0800
Subject: [OmniOS-discuss] adding cua/a as a second login
In-Reply-To: <20141205073111.0762e2b1@punda-mlia>
References: <20141204153051.3e17ac8f@punda-mlia>	<dde4299711c6bd8e1f3a24a3fd0b91ce@blackdot.be>
	<20141205073111.0762e2b1@punda-mlia>
Message-ID: <4cff01d01010$dc508510$94f18f30$@acm.org>

> From: Michael Mounteney
> Sent: Thursday, December 04, 2014 1:31 PM
>
> It certainly is, Jorge;  thanks very much.  Working very well.  The
> only thing I'd like to change is to limit root to logging in on the two
> known ports /dev/console and /dev/ttya, but that's a small point.

Hmm, reviewing the source to login, if CONSOLE is set to the default
/dev/console, root login is allowed on either /dev/console or /dev/vt/*. If
it is set to anything else, root login is allowed only from the device it is
set to. If not set, root login is allowed on any device.

It would be pretty trivial to extend the current code:

} else {
	if (strcmp(ttyn, Console) == 0)
		return;
}

To allow CONSOLE to be a list of devices rather than a single device:

char *state;
char *test_console;
for (test_console = strtok_r(Console, ",", &state); test_console != NULL,
test_console = strtok_r(NULL, ",", &state)) {
	if (strcmp(ttyn, test_console) == 0)
		return;
}

I'm not sure if anything else pays attention to the CONSOLE definition in
/etc/default/login that might get confused though.

If you open a ticket in the illumos issue tracker requesting this feature, I
might take a shot at implementing it :). I'm hoping to get an illumos-gate
development environment going under omnios stable over the Christmas break
(the OI vm I was using got trashed quite a while ago, and I just can't bring
myself to install another OI box <sigh>), and this would be a simple test of
it.



From gate03 at landcroft.co.uk  Thu Dec  4 22:50:57 2014
From: gate03 at landcroft.co.uk (Michael Mounteney)
Date: Fri, 5 Dec 2014 08:50:57 +1000
Subject: [OmniOS-discuss] adding cua/a as a second login
In-Reply-To: <4cff01d01010$dc508510$94f18f30$@acm.org>
References: <20141204153051.3e17ac8f@punda-mlia>
	<dde4299711c6bd8e1f3a24a3fd0b91ce@blackdot.be>
	<20141205073111.0762e2b1@punda-mlia>
	<4cff01d01010$dc508510$94f18f30$@acm.org>
Message-ID: <20141205085057.5e000d9e@punda-mlia>

On Thu, 4 Dec 2014 14:22:55 -0800
"Paul B. Henson" <henson at acm.org> wrote:

> If you open a ticket in the illumos issue tracker requesting this
> feature, I might take a shot at implementing it :). I'm hoping to get
> an illumos-gate development environment going under omnios stable
> over the Christmas break (the OI vm I was using got trashed quite a
> while ago, and I just can't bring myself to install another OI box
> <sigh>), and this would be a simple test of it.

Paul:  https://www.illumos.org/issues/5391

Thanks, Michael.


From gate03 at landcroft.co.uk  Fri Dec  5 01:08:42 2014
From: gate03 at landcroft.co.uk (Michael Mounteney)
Date: Fri, 5 Dec 2014 11:08:42 +1000
Subject: [OmniOS-discuss] UPS serial non-standard, WHY?
Message-ID: <20141205110842.2da41bea@punda-mlia>

OK, not an OmniOS matter but as a lot of people here will be using
UPSs, I hope it's alright.

I made what is apparently the common but incorrect assumption that the
serial monitoring port on the back of my APC 700 UPS had a standard
serial pinout but for some reason the TxD and RxD are swapped.  Is this
just some random brain-&*^# on the part of an android within APC, or is
there some actual reason for it?

Don't plug a standard serial cable into the UPS:  it instantly powers
off.  No fooling.

This is the first step in getting UPS software monitoring in place.  I
presume that the software of choice is NUT?  I can't find it in a
repo so I suppose it's a build-it-yourself job.

Michael.

From protonwrangler at gmail.com  Fri Dec  5 03:07:00 2014
From: protonwrangler at gmail.com (Warren Marts)
Date: Thu, 4 Dec 2014 20:07:00 -0700
Subject: [OmniOS-discuss] UPS serial non-standard, WHY?
In-Reply-To: <20141205110842.2da41bea@punda-mlia>
References: <20141205110842.2da41bea@punda-mlia>
Message-ID: <CAN8fiVu9132=bb2UQsXJTnNtvnEzGcQJN2tchxFGHNq=hkoNEg@mail.gmail.com>

Serial ports have two wiring standards DTE (data Terminal equipment)
and DCE (data Communication equipment), with the data pins and transmit
enables swapped.

Connecting DTE to DCE you use a straight-through cable; here both are DTE
so you need a crossover or "null modem" cable.

On Thu, Dec 4, 2014 at 6:08 PM, Michael Mounteney <gate03 at landcroft.co.uk>
wrote:

> OK, not an OmniOS matter but as a lot of people here will be using
> UPSs, I hope it's alright.
>
> I made what is apparently the common but incorrect assumption that the
> serial monitoring port on the back of my APC 700 UPS had a standard
> serial pinout but for some reason the TxD and RxD are swapped.  Is this
> just some random brain-&*^# on the part of an android within APC, or is
> there some actual reason for it?
>
> Don't plug a standard serial cable into the UPS:  it instantly powers
> off.  No fooling.
>
> This is the first step in getting UPS software monitoring in place.  I
> presume that the software of choice is NUT?  I can't find it in a
> repo so I suppose it's a build-it-yourself job.
>
> Michael.
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141204/8e4ebfdd/attachment.html>

From orilali at evogene.com  Fri Dec  5 06:56:34 2014
From: orilali at evogene.com (Kobi Schwartz)
Date: Fri, 05 Dec 2014 08:56:34 +0200
Subject: [OmniOS-discuss] Kernel stability.
Message-ID: <54815722.3020909@evogene.com>

Hi,

My name is Kobi and I work with a company called Evogene.

We have a storage server based on a Supermicro X9DRD-7LN4F (Intel(R) 
Xeon(R) CPU E5-2609, 64GB RAM) equipped with Intel X520-DA2 10Gb 
Ethernet card.

Two weeks ago we encountered a problem which regards Omnios (r15012) and 
OpenIndiana (151_a8) kernel stability.

The system is a clean install of r151012, only configuration is 
networking and zpool (raidz3 1 vdev of 8 disks of data and 3 parity).

The storage has a single NFS share mounted (nfsvers=3) on an 8 grid 
nodes (running CentOS6.6) with the following mount options :

async,tcp,vers=3,noacl,timeo=20,rw,bg,hard,intr,rsize=65536,wsize=65536 
0 0.

Under normal conditions (random reads and writes) we get great 
performance and stability but if we increase nfs usage the Kernel memory 
usage increases until the machine eventually halts.

To test it we used on each node iozone ( Version $Revision: 3.394 ) 
which simultaneously write and reads 12 files with a size of 500MB.
(iozone -r 64k -i 0 -i 1 -s 500m -t 12 -F 
/evolon2/iozone/`hostname`.{1..12}).

While the iozone's are running the memory of Kernel (as seen by echo 
'::memstat'| mdb -k) is steadily increasing until the machine is dead.

For clarification, when the machine is dead/halts:
There is no crash files.
There is no new messages in /var/adm/messages.
It doesn't respond to pings (and obviously other network services).
Waiting doesn't help.
It can only be brought to life by power cycling it.

I attached 3 files: memstat, top and arc-kstat which I hope will assist 
to understand why the storage died/halted

Thank you in advance,

Kobi.


memstat

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141205/761856c7/attachment-0001.html>
-------------- next part --------------
Thu Dec  4 16:14:39 IST 2014

l2_size                   = 0                   
other_size                = 16487240            
c_max                     = 67601473536         
hash_elements             = 36191               
duplicate_reads           = 0                   
prefetch_data_hits        = 537                 
l2_read_bytes             = 0                   
demand_metadata_misses    = 12438               
prefetch_data_misses      = 1540                
hash_chain_max            = 2                   
l2_hits                   = 0                   
l2_compress_failures      = 0                   
mfu_ghost_hits            = 0                   
l2_rw_clash               = 0                   
memory_throttle_count     = 0                   
crtime                    = 4.537754192         
arc_meta_used             = 97172664            
hash_elements_max         = 41359               
size                      = 5312841912          
hits                      = 750510              
mfu_hits                  = 507924              
prefetch_metadata_hits    = 10297               
hash_chains               = 54                  
evict_l2_cached           = 0                   
l2_writes_sent            = 0                   
l2_writes_error           = 0                   
deleted                   = 15                  
hdr_size                  = 12559216            
l2_compress_zeros         = 0                   
l2_abort_lowmem           = 0                   
l2_io_error               = 0                   
l2_hdr_size               = 0                   
l2_cksum_bad              = 0                   
l2_evict_reading          = 0                   
arc_meta_limit            = 16900368384         
prefetch_metadata_misses  = 3381                
l2_evict_lock_retry       = 0                   
hash_collisions           = 519                 
l2_writes_hdr_miss        = 0                   
evict_l2_ineligible       = 4096                
l2_write_bytes            = 0                   
data_size                 = 5283795456          
c                         = 67601473536         
l2_asize                  = 0                   
class                     = misc                
mutex_miss                = 0                   
snaptime                  = 1052.750004426      
duplicate_buffers_size    = 0                   
l2_misses                 = 0                   
mru_ghost_hits            = 0                   
demand_metadata_hits      = 28989               
arc_meta_max              = 97686696            
c_min                     = 2146100480          
mru_hits                  = 231752              
demand_data_misses        = 2615                
duplicate_buffers         = 0                   
demand_data_hits          = 710687              
l2_free_on_write          = 0                   
l2_feeds                  = 0                   
misses                    = 19974               
evict_skip                = 1214                
evict_l2_eligible         = 71168               
p                         = 33800736768         
l2_compress_successes     = 0                   
recycle_miss              = 0                   
l2_writes_done            = 0                   
----------

Thu Dec  4 16:14:42 IST 2014

l2_size                   = 0                   
other_size                = 16557200            
c_max                     = 67601473536         
hash_elements             = 35929               
duplicate_reads           = 0                   
prefetch_data_hits        = 537                 
l2_read_bytes             = 0                   
demand_metadata_misses    = 12446               
prefetch_data_misses      = 1540                
hash_chain_max            = 2                   
l2_hits                   = 0                   
l2_compress_failures      = 0                   
mfu_ghost_hits            = 0                   
l2_rw_clash               = 0                   
memory_throttle_count     = 0                   
crtime                    = 4.537754192         
arc_meta_used             = 97848832            
hash_elements_max         = 41359               
size                      = 5242345984          
hits                      = 765628              
mfu_hits                  = 519125              
prefetch_metadata_hits    = 10297               
hash_chains               = 53                  
evict_l2_cached           = 0                   
l2_writes_sent            = 0                   
l2_writes_error           = 0                   
deleted                   = 15                  
hdr_size                  = 12559216            
l2_compress_zeros         = 0                   
l2_abort_lowmem           = 0                   
l2_io_error               = 0                   
l2_hdr_size               = 0                   
l2_cksum_bad              = 0                   
l2_evict_reading          = 0                   
arc_meta_limit            = 16900368384         
prefetch_metadata_misses  = 3381                
l2_evict_lock_retry       = 0                   
hash_collisions           = 544                 
l2_writes_hdr_miss        = 0                   
evict_l2_ineligible       = 4096                
l2_write_bytes            = 0                   
data_size                 = 5213229568          
c                         = 67601473536         
l2_asize                  = 0                   
class                     = misc                
mutex_miss                = 0                   
snaptime                  = 1055.756679096      
duplicate_buffers_size    = 0                   
l2_misses                 = 0                   
mru_ghost_hits            = 0                   
demand_metadata_hits      = 28999               
arc_meta_max              = 97815864            
c_min                     = 2146100480          
mru_hits                  = 235669              
demand_data_misses        = 2615                
duplicate_buffers         = 0                   
demand_data_hits          = 725795              
l2_free_on_write          = 0                   
l2_feeds                  = 0                   
misses                    = 19982               
evict_skip                = 1214                
evict_l2_eligible         = 71168               
p                         = 33800736768         
l2_compress_successes     = 0                   
recycle_miss              = 0                   
l2_writes_done            = 0                   
----------

Thu Dec  4 16:14:45 IST 2014

l2_size                   = 0                   
other_size                = 16815440            
c_max                     = 67601473536         
hash_elements             = 34070               
duplicate_reads           = 0                   
prefetch_data_hits        = 537                 
l2_read_bytes             = 0                   
demand_metadata_misses    = 12446               
prefetch_data_misses      = 1540                
hash_chain_max            = 2                   
l2_hits                   = 0                   
l2_compress_failures      = 0                   
mfu_ghost_hits            = 0                   
l2_rw_clash               = 0                   
memory_throttle_count     = 0                   
crtime                    = 4.537754192         
arc_meta_used             = 97619648            
hash_elements_max         = 41359               
size                      = 5487483584          
hits                      = 791669              
mfu_hits                  = 537122              
prefetch_metadata_hits    = 10297               
hash_chains               = 50                  
evict_l2_cached           = 0                   
l2_writes_sent            = 0                   
l2_writes_error           = 0                   
deleted                   = 15                  
hdr_size                  = 12559216            
l2_compress_zeros         = 0                   
l2_abort_lowmem           = 0                   
l2_io_error               = 0                   
l2_hdr_size               = 0                   
l2_cksum_bad              = 0                   
l2_evict_reading          = 0                   
arc_meta_limit            = 16900368384         
prefetch_metadata_misses  = 3381                
l2_evict_lock_retry       = 0                   
hash_collisions           = 581                 
l2_writes_hdr_miss        = 0                   
evict_l2_ineligible       = 4096                
l2_write_bytes            = 0                   
data_size                 = 5458108928          
c                         = 67601473536         
l2_asize                  = 0                   
class                     = misc                
mutex_miss                = 0                   
snaptime                  = 1058.767796546      
duplicate_buffers_size    = 0                   
l2_misses                 = 0                   
mru_ghost_hits            = 0                   
demand_metadata_hits      = 29003               
arc_meta_max              = 97968040            
c_min                     = 2146100480          
mru_hits                  = 243713              
demand_data_misses        = 2615                
duplicate_buffers         = 0                   
demand_data_hits          = 751832              
l2_free_on_write          = 0                   
l2_feeds                  = 0                   
misses                    = 19982               
evict_skip                = 1214                
evict_l2_eligible         = 71168               
p                         = 33800736768         
l2_compress_successes     = 0                   
recycle_miss              = 0                   
l2_writes_done            = 0                   
----------

Thu Dec  4 16:14:48 IST 2014

l2_size                   = 0                   
other_size                = 16858360            
c_max                     = 67601473536         
hash_elements             = 36794               
duplicate_reads           = 0                   
prefetch_data_hits        = 537                 
l2_read_bytes             = 0                   
demand_metadata_misses    = 12454               
prefetch_data_misses      = 1540                
hash_chain_max            = 2                   
l2_hits                   = 0                   
l2_compress_failures      = 0                   
mfu_ghost_hits            = 0                   
l2_rw_clash               = 0                   
memory_throttle_count     = 0                   
crtime                    = 4.537754192         
arc_meta_used             = 98313832            
hash_elements_max         = 41359               
size                      = 5462880872          
hits                      = 802717              
mfu_hits                  = 544723              
prefetch_metadata_hits    = 10297               
hash_chains               = 75                  
evict_l2_cached           = 0                   
l2_writes_sent            = 0                   
l2_writes_error           = 0                   
deleted                   = 15                  
hdr_size                  = 12559216            
l2_compress_zeros         = 0                   
l2_abort_lowmem           = 0                   
l2_io_error               = 0                   
l2_hdr_size               = 0                   
l2_cksum_bad              = 0                   
l2_evict_reading          = 0                   
arc_meta_limit            = 16900368384         
prefetch_metadata_misses  = 3381                
l2_evict_lock_retry       = 0                   
hash_collisions           = 626                 
l2_writes_hdr_miss        = 0                   
evict_l2_ineligible       = 4096                
l2_write_bytes            = 0                   
data_size                 = 5433463296          
c                         = 67601473536         
l2_asize                  = 0                   
class                     = misc                
mutex_miss                = 0                   
snaptime                  = 1061.787519516      
duplicate_buffers_size    = 0                   
l2_misses                 = 0                   
mru_ghost_hits            = 0                   
demand_metadata_hits      = 29022               
arc_meta_max              = 98312872            
c_min                     = 2146100480          
mru_hits                  = 247160              
demand_data_misses        = 2615                
duplicate_buffers         = 0                   
demand_data_hits          = 762861              
l2_free_on_write          = 0                   
l2_feeds                  = 0                   
misses                    = 19990               
evict_skip                = 1214                
evict_l2_eligible         = 71168               
p                         = 33800736768         
l2_compress_successes     = 0                   
recycle_miss              = 0                   
l2_writes_done            = 0                   
----------

Thu Dec  4 16:14:51 IST 2014

l2_size                   = 0                   
other_size                = 16888240            
c_max                     = 67601473536         
hash_elements             = 40590               
duplicate_reads           = 0                   
prefetch_data_hits        = 537                 
l2_read_bytes             = 0                   
demand_metadata_misses    = 12458               
prefetch_data_misses      = 1540                
hash_chain_max            = 2                   
l2_hits                   = 0                   
l2_compress_failures      = 0                   
mfu_ghost_hits            = 0                   
l2_rw_clash               = 0                   
memory_throttle_count     = 0                   
crtime                    = 4.537754192         
arc_meta_used             = 98425632            
hash_elements_max         = 42641               
size                      = 5669168928          
hits                      = 815255              
mfu_hits                  = 552397              
prefetch_metadata_hits    = 10297               
hash_chains               = 86                  
evict_l2_cached           = 0                   
l2_writes_sent            = 0                   
l2_writes_error           = 0                   
deleted                   = 15                  
hdr_size                  = 12559216            
l2_compress_zeros         = 0                   
l2_abort_lowmem           = 0                   
l2_io_error               = 0                   
l2_hdr_size               = 0                   
l2_cksum_bad              = 0                   
l2_evict_reading          = 0                   
arc_meta_limit            = 16900368384         
prefetch_metadata_misses  = 3381                
l2_evict_lock_retry       = 0                   
hash_collisions           = 664                 
l2_writes_hdr_miss        = 0                   
evict_l2_ineligible       = 4096                
l2_write_bytes            = 0                   
data_size                 = 5639721472          
c                         = 67601473536         
l2_asize                  = 0                   
class                     = misc                
mutex_miss                = 0                   
snaptime                  = 1064.791199302      
duplicate_buffers_size    = 0                   
l2_misses                 = 0                   
mru_ghost_hits            = 0                   
demand_metadata_hits      = 29029               
arc_meta_max              = 98388464            
c_min                     = 2146100480          
mru_hits                  = 252024              
demand_data_misses        = 2615                
duplicate_buffers         = 0                   
demand_data_hits          = 775392              
l2_free_on_write          = 0                   
l2_feeds                  = 0                   
misses                    = 19994               
evict_skip                = 1214                
evict_l2_eligible         = 71168               
p                         = 33800736768         
l2_compress_successes     = 0                   
recycle_miss              = 0                   
l2_writes_done            = 0                   
----------

Thu Dec  4 16:14:54 IST 2014

l2_size                   = 0                   
other_size                = 17262640            
c_max                     = 67601473536         
hash_elements             = 36737               
duplicate_reads           = 0                   
prefetch_data_hits        = 537                 
l2_read_bytes             = 0                   
demand_metadata_misses    = 12462               
prefetch_data_misses      = 1540                
hash_chain_max            = 2                   
l2_hits                   = 0                   
l2_compress_failures      = 0                   
mfu_ghost_hits            = 0                   
l2_rw_clash               = 0                   
memory_throttle_count     = 0                   
crtime                    = 4.537754192         
arc_meta_used             = 98304416            
hash_elements_max         = 42641               
size                      = 6396890528          
hits                      = 840144              
mfu_hits                  = 570407              
prefetch_metadata_hits    = 10297               
hash_chains               = 80                  
evict_l2_cached           = 0                   
l2_writes_sent            = 0                   
l2_writes_error           = 0                   
deleted                   = 15                  
hdr_size                  = 12559216            
l2_compress_zeros         = 0                   
l2_abort_lowmem           = 0                   
l2_io_error               = 0                   
l2_hdr_size               = 0                   
l2_cksum_bad              = 0                   
l2_evict_reading          = 0                   
arc_meta_limit            = 16900368384         
prefetch_metadata_misses  = 3381                
l2_evict_lock_retry       = 0                   
hash_collisions           = 690                 
l2_writes_hdr_miss        = 0                   
evict_l2_ineligible       = 4096                
l2_write_bytes            = 0                   
data_size                 = 6367068672          
c                         = 67601473536         
l2_asize                  = 0                   
class                     = misc                
mutex_miss                = 0                   
snaptime                  = 1067.797220708      
duplicate_buffers_size    = 0                   
l2_misses                 = 0                   
mru_ghost_hits            = 0                   
demand_metadata_hits      = 29038               
arc_meta_max              = 98730168            
c_min                     = 2146100480          
mru_hits                  = 258903              
demand_data_misses        = 2615                
duplicate_buffers         = 0                   
demand_data_hits          = 800272              
l2_free_on_write          = 0                   
l2_feeds                  = 0                   
misses                    = 19998               
evict_skip                = 1214                
evict_l2_eligible         = 71168               
p                         = 33800736768         
l2_compress_successes     = 0                   
recycle_miss              = 0                   
l2_writes_done            = 0                   
----------

Thu Dec  4 16:14:57 IST 2014

l2_size                   = 0                   
other_size                = 17330840            
c_max                     = 67601473536         
hash_elements             = 39702               
duplicate_reads           = 0                   
prefetch_data_hits        = 537                 
l2_read_bytes             = 0                   
demand_metadata_misses    = 12466               
prefetch_data_misses      = 1540                
hash_chain_max            = 2                   
l2_hits                   = 0                   
l2_compress_failures      = 0                   
mfu_ghost_hits            = 0                   
l2_rw_clash               = 0                   
memory_throttle_count     = 0                   
crtime                    = 4.537754192         
arc_meta_used             = 99036168            
hash_elements_max         = 42641               
size                      = 6258292744          
hits                      = 851791              
mfu_hits                  = 578190              
prefetch_metadata_hits    = 10297               
hash_chains               = 63                  
evict_l2_cached           = 0                   
l2_writes_sent            = 0                   
l2_writes_error           = 0                   
deleted                   = 15                  
hdr_size                  = 12559216            
l2_compress_zeros         = 0                   
l2_abort_lowmem           = 0                   
l2_io_error               = 0                   
l2_hdr_size               = 0                   
l2_cksum_bad              = 0                   
l2_evict_reading          = 0                   
arc_meta_limit            = 16900368384         
prefetch_metadata_misses  = 3381                
l2_evict_lock_retry       = 0                   
hash_collisions           = 710                 
l2_writes_hdr_miss        = 0                   
evict_l2_ineligible       = 4096                
l2_write_bytes            = 0                   
data_size                 = 6228402688          
c                         = 67601473536         
l2_asize                  = 0                   
class                     = misc                
mutex_miss                = 0                   
snaptime                  = 1070.806236777      
duplicate_buffers_size    = 0                   
l2_misses                 = 0                   
mru_ghost_hits            = 0                   
demand_metadata_hits      = 29044               
arc_meta_max              = 99036168            
c_min                     = 2146100480          
mru_hits                  = 262767              
demand_data_misses        = 2615                
duplicate_buffers         = 0                   
demand_data_hits          = 811913              
l2_free_on_write          = 0                   
l2_feeds                  = 0                   
misses                    = 20002               
evict_skip                = 1214                
evict_l2_eligible         = 71168               
p                         = 33800736768         
l2_compress_successes     = 0                   
recycle_miss              = 0                   
l2_writes_done            = 0                   
----------

Thu Dec  4 16:15:00 IST 2014

l2_size                   = 0                   
other_size                = 17276800            
c_max                     = 67601473536         
hash_elements             = 39938               
duplicate_reads           = 0                   
prefetch_data_hits        = 537                 
l2_read_bytes             = 0                   
demand_metadata_misses    = 12470               
prefetch_data_misses      = 1540                
hash_chain_max            = 2                   
l2_hits                   = 0                   
l2_compress_failures      = 0                   
mfu_ghost_hits            = 0                   
l2_rw_clash               = 0                   
memory_throttle_count     = 0                   
crtime                    = 4.537754192         
arc_meta_used             = 98994416            
hash_elements_max         = 42641               
size                      = 5947348208          
hits                      = 854591              
mfu_hits                  = 580209              
prefetch_metadata_hits    = 10297               
hash_chains               = 64                  
evict_l2_cached           = 0                   
l2_writes_sent            = 0                   
l2_writes_error           = 0                   
deleted                   = 15                  
hdr_size                  = 12559216            
l2_compress_zeros         = 0                   
l2_abort_lowmem           = 0                   
l2_io_error               = 0                   
l2_hdr_size               = 0                   
l2_cksum_bad              = 0                   
l2_evict_reading          = 0                   
arc_meta_limit            = 16900368384         
prefetch_metadata_misses  = 3381                
l2_evict_lock_retry       = 0                   
hash_collisions           = 713                 
l2_writes_hdr_miss        = 0                   
evict_l2_ineligible       = 4096                
l2_write_bytes            = 0                   
data_size                 = 5917512192          
c                         = 67601473536         
l2_asize                  = 0                   
class                     = misc                
mutex_miss                = 0                   
snaptime                  = 1073.811610852      
duplicate_buffers_size    = 0                   
l2_misses                 = 0                   
mru_ghost_hits            = 0                   
demand_metadata_hits      = 29051               
arc_meta_max              = 99051168            
c_min                     = 2146100480          
mru_hits                  = 263548              
demand_data_misses        = 2615                
duplicate_buffers         = 0                   
demand_data_hits          = 814706              
l2_free_on_write          = 0                   
l2_feeds                  = 0                   
misses                    = 20006               
evict_skip                = 1214                
evict_l2_eligible         = 71168               
p                         = 33800736768         
l2_compress_successes     = 0                   
recycle_miss              = 0                   
l2_writes_done            = 0                   
----------

Thu Dec  4 16:15:03 IST 2014

l2_size                   = 0                   
other_size                = 17471000            
c_max                     = 67601473536         
hash_elements             = 38061               
duplicate_reads           = 0                   
prefetch_data_hits        = 537                 
l2_read_bytes             = 0                   
demand_metadata_misses    = 12470               
prefetch_data_misses      = 1540                
hash_chain_max            = 2                   
l2_hits                   = 0                   
l2_compress_failures      = 0                   
mfu_ghost_hits            = 0                   
l2_rw_clash               = 0                   
memory_throttle_count     = 0                   
crtime                    = 4.537754192         
arc_meta_used             = 98656136            
hash_elements_max         = 42641               
size                      = 6162492296          
hits                      = 881460              
mfu_hits                  = 598502              
prefetch_metadata_hits    = 10297               
hash_chains               = 77                  
evict_l2_cached           = 0                   
l2_writes_sent            = 0                   
l2_writes_error           = 0                   
deleted                   = 15                  
hdr_size                  = 12559216            
l2_compress_zeros         = 0                   
l2_abort_lowmem           = 0                   
l2_io_error               = 0                   
l2_hdr_size               = 0                   
l2_cksum_bad              = 0                   
l2_evict_reading          = 0                   
arc_meta_limit            = 16900368384         
prefetch_metadata_misses  = 3381                
l2_evict_lock_retry       = 0                   
hash_collisions           = 756                 
l2_writes_hdr_miss        = 0                   
evict_l2_ineligible       = 4096                
l2_write_bytes            = 0                   
data_size                 = 6132462080          
c                         = 67601473536         
l2_asize                  = 0                   
class                     = misc                
mutex_miss                = 0                   
snaptime                  = 1076.82642884       
duplicate_buffers_size    = 0                   
l2_misses                 = 0                   
mru_ghost_hits            = 0                   
demand_metadata_hits      = 29052               
arc_meta_max              = 99051168            
c_min                     = 2146100480          
mru_hits                  = 272124              
demand_data_misses        = 2615                
duplicate_buffers         = 0                   
demand_data_hits          = 841574              
l2_free_on_write          = 0                   
l2_feeds                  = 0                   
misses                    = 20006               
evict_skip                = 1214                
evict_l2_eligible         = 71168               
p                         = 33800736768         
l2_compress_successes     = 0                   
recycle_miss              = 0                   
l2_writes_done            = 0                   
----------

Thu Dec  4 16:15:06 IST 2014

l2_size                   = 0                   
other_size                = 17531840            
c_max                     = 67601473536         
hash_elements             = 38754               
duplicate_reads           = 0                   
prefetch_data_hits        = 537                 
l2_read_bytes             = 0                   
demand_metadata_misses    = 12478               
prefetch_data_misses      = 1540                
hash_chain_max            = 2                   
l2_hits                   = 0                   
l2_compress_failures      = 0                   
mfu_ghost_hits            = 0                   
l2_rw_clash               = 0                   
memory_throttle_count     = 0                   
crtime                    = 4.537754192         
arc_meta_used             = 98766128            
hash_elements_max         = 42641               
size                      = 6131800368          
hits                      = 891251              
mfu_hits                  = 605449              
prefetch_metadata_hits    = 10297               
hash_chains               = 83                  
evict_l2_cached           = 0                   
l2_writes_sent            = 0                   
l2_writes_error           = 0                   
deleted                   = 15                  
hdr_size                  = 12559216            
l2_compress_zeros         = 0                   
l2_abort_lowmem           = 0                   
l2_io_error               = 0                   
l2_hdr_size               = 0                   
l2_cksum_bad              = 0                   
l2_evict_reading          = 0                   
arc_meta_limit            = 16900368384         
prefetch_metadata_misses  = 3381                
l2_evict_lock_retry       = 0                   
hash_collisions           = 774                 
l2_writes_hdr_miss        = 0                   
evict_l2_ineligible       = 4096                
l2_write_bytes            = 0                   
data_size                 = 6101709312          
c                         = 67601473536         
l2_asize                  = 0                   
class                     = misc                
mutex_miss                = 0                   
snaptime                  = 1079.87068534       
duplicate_buffers_size    = 0                   
l2_misses                 = 0                   
mru_ghost_hits            = 0                   
demand_metadata_hits      = 29066               
arc_meta_max              = 99360176            
c_min                     = 2146100480          
mru_hits                  = 274968              
demand_data_misses        = 2615                
duplicate_buffers         = 0                   
demand_data_hits          = 851351              
l2_free_on_write          = 0                   
l2_feeds                  = 0                   
misses                    = 20014               
evict_skip                = 1214                
evict_l2_eligible         = 71168               
p                         = 33800736768         
l2_compress_successes     = 0                   
recycle_miss              = 0                   
l2_writes_done            = 0                   
----------

Thu Dec  4 16:15:09 IST 2014

l2_size                   = 0                   
other_size                = 17593040            
c_max                     = 67601473536         
hash_elements             = 40255               
duplicate_reads           = 0                   
prefetch_data_hits        = 537                 
l2_read_bytes             = 0                   
demand_metadata_misses    = 12478               
prefetch_data_misses      = 1540                
hash_chain_max            = 2                   
l2_hits                   = 0                   
l2_compress_failures      = 0                   
mfu_ghost_hits            = 0                   
l2_rw_clash               = 0                   
memory_throttle_count     = 0                   
crtime                    = 4.537754192         
arc_meta_used             = 98860096            
hash_elements_max         = 42641               
size                      = 6205425728          
hits                      = 902579              
mfu_hits                  = 612627              
prefetch_metadata_hits    = 10297               
hash_chains               = 81                  
evict_l2_cached           = 0                   
l2_writes_sent            = 0                   
l2_writes_error           = 0                   
deleted                   = 15                  
hdr_size                  = 12559216            
l2_compress_zeros         = 0                   
l2_abort_lowmem           = 0                   
l2_io_error               = 0                   
l2_hdr_size               = 0                   
l2_cksum_bad              = 0                   
l2_evict_reading          = 0                   
arc_meta_limit            = 16900368384         
prefetch_metadata_misses  = 3381                
l2_evict_lock_retry       = 0                   
hash_collisions           = 794                 
l2_writes_hdr_miss        = 0                   
evict_l2_ineligible       = 4096                
l2_write_bytes            = 0                   
data_size                 = 6175273472          
c                         = 67601473536         
l2_asize                  = 0                   
class                     = misc                
mutex_miss                = 0                   
snaptime                  = 1082.875826687      
duplicate_buffers_size    = 0                   
l2_misses                 = 0                   
mru_ghost_hits            = 0                   
demand_metadata_hits      = 29066               
arc_meta_max              = 99360176            
c_min                     = 2146100480          
mru_hits                  = 279118              
demand_data_misses        = 2615                
duplicate_buffers         = 0                   
demand_data_hits          = 862679              
l2_free_on_write          = 0                   
l2_feeds                  = 0                   
misses                    = 20014               
evict_skip                = 1214                
evict_l2_eligible         = 71168               
p                         = 33800736768         
l2_compress_successes     = 0                   
recycle_miss              = 0                   
l2_writes_done            = 0                   
----------

Thu Dec  4 16:15:12 IST 2014

l2_size                   = 0                   
other_size                = 17598240            
c_max                     = 67601473536         
hash_elements             = 43117               
duplicate_reads           = 0                   
prefetch_data_hits        = 537                 
l2_read_bytes             = 0                   
demand_metadata_misses    = 12478               
prefetch_data_misses      = 1540                
hash_chain_max            = 2                   
l2_hits                   = 0                   
l2_compress_failures      = 0                   
mfu_ghost_hits            = 0                   
l2_rw_clash               = 0                   
memory_throttle_count     = 0                   
crtime                    = 4.537754192         
arc_meta_used             = 98865296            
hash_elements_max         = 43118               
size                      = 6013541520          
hits                      = 905362              
mfu_hits                  = 613511              
prefetch_metadata_hits    = 10297               
hash_chains               = 111                 
evict_l2_cached           = 0                   
l2_writes_sent            = 0                   
l2_writes_error           = 0                   
deleted                   = 15                  
hdr_size                  = 12559216            
l2_compress_zeros         = 0                   
l2_abort_lowmem           = 0                   
l2_io_error               = 0                   
l2_hdr_size               = 0                   
l2_cksum_bad              = 0                   
l2_evict_reading          = 0                   
arc_meta_limit            = 16900368384         
prefetch_metadata_misses  = 3381                
l2_evict_lock_retry       = 0                   
hash_collisions           = 831                 
l2_writes_hdr_miss        = 0                   
evict_l2_ineligible       = 4096                
l2_write_bytes            = 0                   
data_size                 = 5983384064          
c                         = 67601473536         
l2_asize                  = 0                   
class                     = misc                
mutex_miss                = 0                   
snaptime                  = 1085.879514131      
duplicate_buffers_size    = 0                   
l2_misses                 = 0                   
mru_ghost_hits            = 0                   
demand_metadata_hits      = 29066               
arc_meta_max              = 99360176            
c_min                     = 2146100480          
mru_hits                  = 281017              
demand_data_misses        = 2615                
duplicate_buffers         = 0                   
demand_data_hits          = 865462              
l2_free_on_write          = 0                   
l2_feeds                  = 0                   
misses                    = 20014               
evict_skip                = 1214                
evict_l2_eligible         = 71168               
p                         = 33800736768         
l2_compress_successes     = 0                   
recycle_miss              = 0                   
l2_writes_done            = 0                   
----------

Thu Dec  4 16:15:15 IST 2014

l2_size                   = 0                   
other_size                = 17603440            
c_max                     = 67601473536         
hash_elements             = 43174               
duplicate_reads           = 0                   
prefetch_data_hits        = 537                 
l2_read_bytes             = 0                   
demand_metadata_misses    = 12482               
prefetch_data_misses      = 1540                
hash_chain_max            = 2                   
l2_hits                   = 0                   
l2_compress_failures      = 0                   
mfu_ghost_hits            = 0                   
l2_rw_clash               = 0                   
memory_throttle_count     = 0                   
crtime                    = 4.537754192         
arc_meta_used             = 99480800            
hash_elements_max         = 43929               
size                      = 5807325408          
hits                      = 908319              
mfu_hits                  = 615797              
prefetch_metadata_hits    = 10297               
hash_chains               = 102                 
evict_l2_cached           = 0                   
l2_writes_sent            = 0                   
l2_writes_error           = 0                   
deleted                   = 15                  
hdr_size                  = 12559216            
l2_compress_zeros         = 0                   
l2_abort_lowmem           = 0                   
l2_io_error               = 0                   
l2_hdr_size               = 0                   
l2_cksum_bad              = 0                   
l2_evict_reading          = 0                   
arc_meta_limit            = 16900368384         
prefetch_metadata_misses  = 3381                
l2_evict_lock_retry       = 0                   
hash_collisions           = 832                 
l2_writes_hdr_miss        = 0                   
evict_l2_ineligible       = 4096                
l2_write_bytes            = 0                   
data_size                 = 5777162752          
c                         = 67601473536         
l2_asize                  = 0                   
class                     = misc                
mutex_miss                = 0                   
snaptime                  = 1088.884358936      
duplicate_buffers_size    = 0                   
l2_misses                 = 0                   
mru_ghost_hits            = 0                   
demand_metadata_hits      = 29075               
arc_meta_max              = 99480800            
c_min                     = 2146100480          
mru_hits                  = 281688              
demand_data_misses        = 2615                
duplicate_buffers         = 0                   
demand_data_hits          = 868410              
l2_free_on_write          = 0                   
l2_feeds                  = 0                   
misses                    = 20018               
evict_skip                = 1214                
evict_l2_eligible         = 71168               
p                         = 33800736768         
l2_compress_successes     = 0                   
recycle_miss              = 0                   
l2_writes_done            = 0                   
----------

Thu Dec  4 16:15:18 IST 2014

l2_size                   = 0                   
other_size                = 17668200            
c_max                     = 67601473536         
hash_elements             = 41509               
duplicate_reads           = 0                   
prefetch_data_hits        = 537                 
l2_read_bytes             = 0                   
demand_metadata_misses    = 12488               
prefetch_data_misses      = 1540                
hash_chain_max            = 2                   
l2_hits                   = 0                   
l2_compress_failures      = 0                   
mfu_ghost_hits            = 0                   
l2_rw_clash               = 0                   
memory_throttle_count     = 0                   
crtime                    = 4.537754192         
arc_meta_used             = 98984408            
hash_elements_max         = 43929               
size                      = 5884816856          
hits                      = 927065              
mfu_hits                  = 630052              
prefetch_metadata_hits    = 10297               
hash_chains               = 93                  
evict_l2_cached           = 0                   
l2_writes_sent            = 0                   
l2_writes_error           = 0                   
deleted                   = 15                  
hdr_size                  = 12559216            
l2_compress_zeros         = 0                   
l2_abort_lowmem           = 0                   
l2_io_error               = 0                   
l2_hdr_size               = 0                   
l2_cksum_bad              = 0                   
l2_evict_reading          = 0                   
arc_meta_limit            = 16900368384         
prefetch_metadata_misses  = 3381                
l2_evict_lock_retry       = 0                   
hash_collisions           = 843                 
l2_writes_hdr_miss        = 0                   
evict_l2_ineligible       = 4096                
l2_write_bytes            = 0                   
data_size                 = 5854589440          
c                         = 67601473536         
l2_asize                  = 0                   
class                     = misc                
mutex_miss                = 0                   
snaptime                  = 1091.888514904      
duplicate_buffers_size    = 0                   
l2_misses                 = 0                   
mru_ghost_hits            = 0                   
demand_metadata_hits      = 29087               
arc_meta_max              = 99568328            
c_min                     = 2146100480          
mru_hits                  = 286179              
demand_data_misses        = 2615                
duplicate_buffers         = 0                   
demand_data_hits          = 887144              
l2_free_on_write          = 0                   
l2_feeds                  = 0                   
misses                    = 20024               
evict_skip                = 1214                
evict_l2_eligible         = 71168               
p                         = 33800736768         
l2_compress_successes     = 0                   
recycle_miss              = 0                   
l2_writes_done            = 0                   
----------

Thu Dec  4 16:15:21 IST 2014

l2_size                   = 0                   
other_size                = 17646400            
c_max                     = 67601473536         
hash_elements             = 39031               
duplicate_reads           = 0                   
prefetch_data_hits        = 537                 
l2_read_bytes             = 0                   
demand_metadata_misses    = 12488               
prefetch_data_misses      = 1540                
hash_chain_max            = 2                   
l2_hits                   = 0                   
l2_compress_failures      = 0                   
mfu_ghost_hits            = 0                   
l2_rw_clash               = 0                   
memory_throttle_count     = 0                   
crtime                    = 4.537754192         
arc_meta_used             = 98978992            
hash_elements_max         = 43929               
size                      = 5865412784          
hits                      = 955131              
mfu_hits                  = 649821              
prefetch_metadata_hits    = 10297               
hash_chains               = 76                  
evict_l2_cached           = 0                   
l2_writes_sent            = 0                   
l2_writes_error           = 0                   
deleted                   = 15                  
hdr_size                  = 12559216            
l2_compress_zeros         = 0                   
l2_abort_lowmem           = 0                   
l2_io_error               = 0                   
l2_hdr_size               = 0                   
l2_cksum_bad              = 0                   
l2_evict_reading          = 0                   
arc_meta_limit            = 16900368384         
prefetch_metadata_misses  = 3381                
l2_evict_lock_retry       = 0                   
hash_collisions           = 857                 
l2_writes_hdr_miss        = 0                   
evict_l2_ineligible       = 4096                
l2_write_bytes            = 0                   
data_size                 = 5835207168          
c                         = 67601473536         
l2_asize                  = 0                   
class                     = misc                
mutex_miss                = 0                   
snaptime                  = 1094.945818674      
duplicate_buffers_size    = 0                   
l2_misses                 = 0                   
mru_ghost_hits            = 0                   
demand_metadata_hits      = 29087               
arc_meta_max              = 99568328            
c_min                     = 2146100480          
mru_hits                  = 294476              
demand_data_misses        = 2615                
duplicate_buffers         = 0                   
demand_data_hits          = 915210              
l2_free_on_write          = 0                   
l2_feeds                  = 0                   
misses                    = 20024               
evict_skip                = 1214                
evict_l2_eligible         = 71168               
p                         = 33800736768         
l2_compress_successes     = 0                   
recycle_miss              = 0                   
l2_writes_done            = 0                   
----------

Thu Dec  4 16:15:24 IST 2014

l2_size                   = 0                   
other_size                = 17752040            
c_max                     = 67601473536         
hash_elements             = 41863               
duplicate_reads           = 0                   
prefetch_data_hits        = 537                 
l2_read_bytes             = 0                   
demand_metadata_misses    = 12492               
prefetch_data_misses      = 1540                
hash_chain_max            = 2                   
l2_hits                   = 0                   
l2_compress_failures      = 0                   
mfu_ghost_hits            = 0                   
l2_rw_clash               = 0                   
memory_throttle_count     = 0                   
crtime                    = 4.537754192         
arc_meta_used             = 99137880            
hash_elements_max         = 43929               
size                      = 6192858456          
hits                      = 968809              
mfu_hits                  = 659725              
prefetch_metadata_hits    = 10297               
hash_chains               = 87                  
evict_l2_cached           = 0                   
l2_writes_sent            = 0                   
l2_writes_error           = 0                   
deleted                   = 15                  
hdr_size                  = 12559216            
l2_compress_zeros         = 0                   
l2_abort_lowmem           = 0                   
l2_io_error               = 0                   
l2_hdr_size               = 0                   
l2_cksum_bad              = 0                   
l2_evict_reading          = 0                   
arc_meta_limit            = 16900368384         
prefetch_metadata_misses  = 3381                
l2_evict_lock_retry       = 0                   
hash_collisions           = 880                 
l2_writes_hdr_miss        = 0                   
evict_l2_ineligible       = 4096                
l2_write_bytes            = 0                   
data_size                 = 6162547200          
c                         = 67601473536         
l2_asize                  = 0                   
class                     = misc                
mutex_miss                = 0                   
snaptime                  = 1097.957144074      
duplicate_buffers_size    = 0                   
l2_misses                 = 0                   
mru_ghost_hits            = 0                   
demand_metadata_hits      = 29094               
arc_meta_max              = 99660136            
c_min                     = 2146100480          
mru_hits                  = 298250              
demand_data_misses        = 2615                
duplicate_buffers         = 0                   
demand_data_hits          = 928881              
l2_free_on_write          = 0                   
l2_feeds                  = 0                   
misses                    = 20028               
evict_skip                = 1214                
evict_l2_eligible         = 71168               
p                         = 33800736768         
l2_compress_successes     = 0                   
recycle_miss              = 0                   
l2_writes_done            = 0                   
----------

Thu Dec  4 16:15:27 IST 2014

l2_size                   = 0                   
other_size                = 17651840            
c_max                     = 67601473536         
hash_elements             = 44052               
duplicate_reads           = 0                   
prefetch_data_hits        = 537                 
l2_read_bytes             = 0                   
demand_metadata_misses    = 12496               
prefetch_data_misses      = 1540                
hash_chain_max            = 2                   
l2_hits                   = 0                   
l2_compress_failures      = 0                   
mfu_ghost_hits            = 0                   
l2_rw_clash               = 0                   
memory_throttle_count     = 0                   
crtime                    = 4.537754192         
arc_meta_used             = 99623408            
hash_elements_max         = 44527               
size                      = 5948239344          
hits                      = 972940              
mfu_hits                  = 661821              
prefetch_metadata_hits    = 10297               
hash_chains               = 92                  
evict_l2_cached           = 0                   
l2_writes_sent            = 0                   
l2_writes_error           = 0                   
deleted                   = 15                  
hdr_size                  = 12559216            
l2_compress_zeros         = 0                   
l2_abort_lowmem           = 0                   
l2_io_error               = 0                   
l2_hdr_size               = 0                   
l2_cksum_bad              = 0                   
l2_evict_reading          = 0                   
arc_meta_limit            = 16900368384         
prefetch_metadata_misses  = 3381                
l2_evict_lock_retry       = 0                   
hash_collisions           = 891                 
l2_writes_hdr_miss        = 0                   
evict_l2_ineligible       = 4096                
l2_write_bytes            = 0                   
data_size                 = 5918028288          
c                         = 67601473536         
l2_asize                  = 0                   
class                     = misc                
mutex_miss                = 0                   
snaptime                  = 1100.964341965      
duplicate_buffers_size    = 0                   
l2_misses                 = 0                   
mru_ghost_hits            = 0                   
demand_metadata_hits      = 29100               
arc_meta_max              = 99660136            
c_min                     = 2146100480          
mru_hits                  = 300285              
demand_data_misses        = 2615                
duplicate_buffers         = 0                   
demand_data_hits          = 933006              
l2_free_on_write          = 0                   
l2_feeds                  = 0                   
misses                    = 20032               
evict_skip                = 1214                
evict_l2_eligible         = 71168               
p                         = 33800736768         
l2_compress_successes     = 0                   
recycle_miss              = 0                   
l2_writes_done            = 0                   
----------

Thu Dec  4 16:15:30 IST 2014

l2_size                   = 0                   
other_size                = 17736840            
c_max                     = 67601473536         
hash_elements             = 39286               
duplicate_reads           = 0                   
prefetch_data_hits        = 537                 
l2_read_bytes             = 0                   
demand_metadata_misses    = 12500               
prefetch_data_misses      = 1540                
hash_chain_max            = 2                   
l2_hits                   = 0                   
l2_compress_failures      = 0                   
mfu_ghost_hits            = 0                   
l2_rw_clash               = 0                   
memory_throttle_count     = 0                   
crtime                    = 4.537754192         
arc_meta_used             = 99134968            
hash_elements_max         = 44527               
size                      = 5866748408          
hits                      = 999296              
mfu_hits                  = 681359              
prefetch_metadata_hits    = 10297               
hash_chains               = 89                  
evict_l2_cached           = 0                   
l2_writes_sent            = 0                   
l2_writes_error           = 0                   
deleted                   = 15                  
hdr_size                  = 12559216            
l2_compress_zeros         = 0                   
l2_abort_lowmem           = 0                   
l2_io_error               = 0                   
l2_hdr_size               = 0                   
l2_cksum_bad              = 0                   
l2_evict_reading          = 0                   
arc_meta_limit            = 16900368384         
prefetch_metadata_misses  = 3381                
l2_evict_lock_retry       = 0                   
hash_collisions           = 922                 
l2_writes_hdr_miss        = 0                   
evict_l2_ineligible       = 4096                
l2_write_bytes            = 0                   
data_size                 = 5836452352          
c                         = 67601473536         
l2_asize                  = 0                   
class                     = misc                
mutex_miss                = 0                   
snaptime                  = 1103.989100171      
duplicate_buffers_size    = 0                   
l2_misses                 = 0                   
mru_ghost_hits            = 0                   
demand_metadata_hits      = 29108               
arc_meta_max              = 99718496            
c_min                     = 2146100480          
mru_hits                  = 307103              
demand_data_misses        = 2615                
duplicate_buffers         = 0                   
demand_data_hits          = 959354              
l2_free_on_write          = 0                   
l2_feeds                  = 0                   
misses                    = 20036               
evict_skip                = 1214                
evict_l2_eligible         = 71168               
p                         = 33800736768         
l2_compress_successes     = 0                   
recycle_miss              = 0                   
l2_writes_done            = 0                   
----------

Thu Dec  4 16:15:33 IST 2014

l2_size                   = 0                   
other_size                = 17708240            
c_max                     = 67601473536         
hash_elements             = 43427               
duplicate_reads           = 0                   
prefetch_data_hits        = 537                 
l2_read_bytes             = 0                   
demand_metadata_misses    = 12500               
prefetch_data_misses      = 1540                
hash_chain_max            = 2                   
l2_hits                   = 0                   
l2_compress_failures      = 0                   
mfu_ghost_hits            = 0                   
l2_rw_clash               = 0                   
memory_throttle_count     = 0                   
crtime                    = 4.537754192         
arc_meta_used             = 99106368            
hash_elements_max         = 44527               
size                      = 5953882688          
hits                      = 1009876             
mfu_hits                  = 687985              
prefetch_metadata_hits    = 10297               
hash_chains               = 107                 
evict_l2_cached           = 0                   
l2_writes_sent            = 0                   
l2_writes_error           = 0                   
deleted                   = 15                  
hdr_size                  = 12559216            
l2_compress_zeros         = 0                   
l2_abort_lowmem           = 0                   
l2_io_error               = 0                   
l2_hdr_size               = 0                   
l2_cksum_bad              = 0                   
l2_evict_reading          = 0                   
arc_meta_limit            = 16900368384         
prefetch_metadata_misses  = 3381                
l2_evict_lock_retry       = 0                   
hash_collisions           = 959                 
l2_writes_hdr_miss        = 0                   
evict_l2_ineligible       = 4096                
l2_write_bytes            = 0                   
data_size                 = 5923615232          
c                         = 67601473536         
l2_asize                  = 0                   
class                     = misc                
mutex_miss                = 0                   
snaptime                  = 1106.99462468       
duplicate_buffers_size    = 0                   
l2_misses                 = 0                   
mru_ghost_hits            = 0                   
demand_metadata_hits      = 29109               
arc_meta_max              = 99718496            
c_min                     = 2146100480          
mru_hits                  = 311057              
demand_data_misses        = 2615                
duplicate_buffers         = 0                   
demand_data_hits          = 969933              
l2_free_on_write          = 0                   
l2_feeds                  = 0                   
misses                    = 20036               
evict_skip                = 1214                
evict_l2_eligible         = 71168               
p                         = 33800736768         
l2_compress_successes     = 0                   
recycle_miss              = 0                   
l2_writes_done            = 0                   
----------

Thu Dec  4 16:15:36 IST 2014

l2_size                   = 0                   
other_size                = 17820440            
c_max                     = 67601473536         
hash_elements             = 38756               
duplicate_reads           = 0                   
prefetch_data_hits        = 537                 
l2_read_bytes             = 0                   
demand_metadata_misses    = 12508               
prefetch_data_misses      = 1540                
hash_chain_max            = 2                   
l2_hits                   = 0                   
l2_compress_failures      = 0                   
mfu_ghost_hits            = 0                   
l2_rw_clash               = 0                   
memory_throttle_count     = 0                   
crtime                    = 4.537754192         
arc_meta_used             = 99222664            
hash_elements_max         = 44527               
size                      = 5931061384          
hits                      = 1034158             
mfu_hits                  = 705727              
prefetch_metadata_hits    = 10297               
hash_chains               = 79                  
evict_l2_cached           = 0                   
l2_writes_sent            = 0                   
l2_writes_error           = 0                   
deleted                   = 15                  
hdr_size                  = 12559216            
l2_compress_zeros         = 0                   
l2_abort_lowmem           = 0                   
l2_io_error               = 0                   
l2_hdr_size               = 0                   
l2_cksum_bad              = 0                   
l2_evict_reading          = 0                   
arc_meta_limit            = 16900368384         
prefetch_metadata_misses  = 3381                
l2_evict_lock_retry       = 0                   
hash_collisions           = 970                 
l2_writes_hdr_miss        = 0                   
evict_l2_ineligible       = 4096                
l2_write_bytes            = 0                   
data_size                 = 5900681728          
c                         = 67601473536         
l2_asize                  = 0                   
class                     = misc                
mutex_miss                = 0                   
snaptime                  = 1109.998397046      
duplicate_buffers_size    = 0                   
l2_misses                 = 0                   
mru_ghost_hits            = 0                   
demand_metadata_hits      = 29126               
arc_meta_max              = 99816784            
c_min                     = 2146100480          
mru_hits                  = 317597              
demand_data_misses        = 2615                
duplicate_buffers         = 0                   
demand_data_hits          = 994198              
l2_free_on_write          = 0                   
l2_feeds                  = 0                   
misses                    = 20044               
evict_skip                = 1214                
evict_l2_eligible         = 71168               
p                         = 33800736768         
l2_compress_successes     = 0                   
recycle_miss              = 0                   
l2_writes_done            = 0                   
----------

Thu Dec  4 16:15:39 IST 2014

l2_size                   = 0                   
other_size                = 17808640            
c_max                     = 67601473536         
hash_elements             = 44706               
duplicate_reads           = 0                   
prefetch_data_hits        = 537                 
l2_read_bytes             = 0                   
demand_metadata_misses    = 12508               
prefetch_data_misses      = 1540                
hash_chain_max            = 2                   
l2_hits                   = 0                   
l2_compress_failures      = 0                   
mfu_ghost_hits            = 0                   
l2_rw_clash               = 0                   
memory_throttle_count     = 0                   
crtime                    = 4.537754192         
arc_meta_used             = 99212592            
hash_elements_max         = 44706               
size                      = 6196209968          
hits                      = 1039940             
mfu_hits                  = 709638              
prefetch_metadata_hits    = 10297               
hash_chains               = 103                 
evict_l2_cached           = 0                   
l2_writes_sent            = 0                   
l2_writes_error           = 0                   
deleted                   = 15                  
hdr_size                  = 12560944            
l2_compress_zeros         = 0                   
l2_abort_lowmem           = 0                   
l2_io_error               = 0                   
l2_hdr_size               = 0                   
l2_cksum_bad              = 0                   
l2_evict_reading          = 0                   
arc_meta_limit            = 16900368384         
prefetch_metadata_misses  = 3381                
l2_evict_lock_retry       = 0                   
hash_collisions           = 1014                
l2_writes_hdr_miss        = 0                   
evict_l2_ineligible       = 4096                
l2_write_bytes            = 0                   
data_size                 = 6165840384          
c                         = 67601473536         
l2_asize                  = 0                   
class                     = misc                
mutex_miss                = 0                   
snaptime                  = 1113.002157194      
duplicate_buffers_size    = 0                   
l2_misses                 = 0                   
mru_ghost_hits            = 0                   
demand_metadata_hits      = 29126               
arc_meta_max              = 99816784            
c_min                     = 2146100480          
mru_hits                  = 319468              
demand_data_misses        = 2615                
duplicate_buffers         = 0                   
demand_data_hits          = 999980              
l2_free_on_write          = 0                   
l2_feeds                  = 0                   
misses                    = 20044               
evict_skip                = 1214                
evict_l2_eligible         = 71168               
p                         = 33800736768         
l2_compress_successes     = 0                   
recycle_miss              = 0                   
l2_writes_done            = 0                   
----------

Thu Dec  4 16:15:42 IST 2014

l2_size                   = 0                   
other_size                = 17820240            
c_max                     = 67601473536         
hash_elements             = 42808               
duplicate_reads           = 0                   
prefetch_data_hits        = 537                 
l2_read_bytes             = 0                   
demand_metadata_misses    = 12516               
prefetch_data_misses      = 1540                
hash_chain_max            = 2                   
l2_hits                   = 0                   
l2_compress_failures      = 0                   
mfu_ghost_hits            = 0                   
l2_rw_clash               = 0                   
memory_throttle_count     = 0                   
crtime                    = 4.537754192         
arc_meta_used             = 99240576            
hash_elements_max         = 46675               
size                      = 6415390336          
hits                      = 1059255             
mfu_hits                  = 723283              
prefetch_metadata_hits    = 10297               
hash_chains               = 108                 
evict_l2_cached           = 0                   
l2_writes_sent            = 0                   
l2_writes_error           = 0                   
deleted                   = 15                  
hdr_size                  = 12560944            
l2_compress_zeros         = 0                   
l2_abort_lowmem           = 0                   
l2_io_error               = 0                   
l2_hdr_size               = 0                   
l2_cksum_bad              = 0                   
l2_evict_reading          = 0                   
arc_meta_limit            = 16900368384         
prefetch_metadata_misses  = 3381                
l2_evict_lock_retry       = 0                   
hash_collisions           = 1042                
l2_writes_hdr_miss        = 0                   
evict_l2_ineligible       = 4096                
l2_write_bytes            = 0                   
data_size                 = 6385009152          
c                         = 67601473536         
l2_asize                  = 0                   
class                     = misc                
mutex_miss                = 0                   
snaptime                  = 1116.00772062       
duplicate_buffers_size    = 0                   
l2_misses                 = 0                   
mru_ghost_hits            = 0                   
demand_metadata_hits      = 29140               
arc_meta_max              = 99824704            
c_min                     = 2146100480          
mru_hits                  = 325138              
demand_data_misses        = 2615                
duplicate_buffers         = 0                   
demand_data_hits          = 1019281             
l2_free_on_write          = 0                   
l2_feeds                  = 0                   
misses                    = 20052               
evict_skip                = 1214                
evict_l2_eligible         = 71168               
p                         = 33800736768         
l2_compress_successes     = 0                   
recycle_miss              = 0                   
l2_writes_done            = 0                   
----------

Thu Dec  4 16:15:45 IST 2014

l2_size                   = 0                   
other_size                = 17674040            
c_max                     = 67601473536         
hash_elements             = 44891               
duplicate_reads           = 0                   
prefetch_data_hits        = 537                 
l2_read_bytes             = 0                   
demand_metadata_misses    = 12516               
prefetch_data_misses      = 1540                
hash_chain_max            = 2                   
l2_hits                   = 0                   
l2_compress_failures      = 0                   
mfu_ghost_hits            = 0                   
l2_rw_clash               = 0                   
memory_throttle_count     = 0                   
crtime                    = 4.537754192         
arc_meta_used             = 99094376            
hash_elements_max         = 46675               
size                      = 5900262248          
hits                      = 1068199             
mfu_hits                  = 728474              
prefetch_metadata_hits    = 10297               
hash_chains               = 130                 
evict_l2_cached           = 0                   
l2_writes_sent            = 0                   
l2_writes_error           = 0                   
deleted                   = 15                  
hdr_size                  = 12560944            
l2_compress_zeros         = 0                   
l2_abort_lowmem           = 0                   
l2_io_error               = 0                   
l2_hdr_size               = 0                   
l2_cksum_bad              = 0                   
l2_evict_reading          = 0                   
arc_meta_limit            = 16900368384         
prefetch_metadata_misses  = 3381                
l2_evict_lock_retry       = 0                   
hash_collisions           = 1081                
l2_writes_hdr_miss        = 0                   
evict_l2_ineligible       = 4096                
l2_write_bytes            = 0                   
data_size                 = 5870027264          
c                         = 67601473536         
l2_asize                  = 0                   
class                     = misc                
mutex_miss                = 0                   
snaptime                  = 1119.021235208      
duplicate_buffers_size    = 0                   
l2_misses                 = 0                   
mru_ghost_hits            = 0                   
demand_metadata_hits      = 29140               
arc_meta_max              = 99824704            
c_min                     = 2146100480          
mru_hits                  = 328891              
demand_data_misses        = 2615                
duplicate_buffers         = 0                   
demand_data_hits          = 1028225             
l2_free_on_write          = 0                   
l2_feeds                  = 0                   
misses                    = 20052               
evict_skip                = 1214                
evict_l2_eligible         = 71168               
p                         = 33800736768         
l2_compress_successes     = 0                   
recycle_miss              = 0                   
l2_writes_done            = 0                   
----------

Thu Dec  4 16:15:48 IST 2014

l2_size                   = 0                   
other_size                = 17766200            
c_max                     = 67601473536         
hash_elements             = 39994               
duplicate_reads           = 0                   
prefetch_data_hits        = 537                 
l2_read_bytes             = 0                   
demand_metadata_misses    = 12524               
prefetch_data_misses      = 1540                
hash_chain_max            = 2                   
l2_hits                   = 0                   
l2_compress_failures      = 0                   
mfu_ghost_hits            = 0                   
l2_rw_clash               = 0                   
memory_throttle_count     = 0                   
crtime                    = 4.537754192         
arc_meta_used             = 99751784            
hash_elements_max         = 46675               
size                      = 5771420520          
hits                      = 1088758             
mfu_hits                  = 743653              
prefetch_metadata_hits    = 10297               
hash_chains               = 94                  
evict_l2_cached           = 0                   
l2_writes_sent            = 0                   
l2_writes_error           = 0                   
deleted                   = 15                  
hdr_size                  = 12560944            
l2_compress_zeros         = 0                   
l2_abort_lowmem           = 0                   
l2_io_error               = 0                   
l2_hdr_size               = 0                   
l2_cksum_bad              = 0                   
l2_evict_reading          = 0                   
arc_meta_limit            = 16900368384         
prefetch_metadata_misses  = 3381                
l2_evict_lock_retry       = 0                   
hash_collisions           = 1092                
l2_writes_hdr_miss        = 0                   
evict_l2_ineligible       = 4096                
l2_write_bytes            = 0                   
data_size                 = 5741093376          
c                         = 67601473536         
l2_asize                  = 0                   
class                     = misc                
mutex_miss                = 0                   
snaptime                  = 1122.026835395      
duplicate_buffers_size    = 0                   
l2_misses                 = 0                   
mru_ghost_hits            = 0                   
demand_metadata_hits      = 29147               
arc_meta_max              = 99824704            
c_min                     = 2146100480          
mru_hits                  = 334271              
demand_data_misses        = 2615                
duplicate_buffers         = 0                   
demand_data_hits          = 1048777             
l2_free_on_write          = 0                   
l2_feeds                  = 0                   
misses                    = 20060               
evict_skip                = 1214                
evict_l2_eligible         = 71168               
p                         = 33800736768         
l2_compress_successes     = 0                   
recycle_miss              = 0                   
l2_writes_done            = 0                   
----------

Thu Dec  4 16:15:51 IST 2014

l2_size                   = 0                   
other_size                = 17829240            
c_max                     = 67601473536         
hash_elements             = 43380               
duplicate_reads           = 0                   
prefetch_data_hits        = 537                 
l2_read_bytes             = 0                   
demand_metadata_misses    = 12524               
prefetch_data_misses      = 1540                
hash_chain_max            = 2                   
l2_hits                   = 0                   
l2_compress_failures      = 0                   
mfu_ghost_hits            = 0                   
l2_rw_clash               = 0                   
memory_throttle_count     = 0                   
crtime                    = 4.537754192         
arc_meta_used             = 99249576            
hash_elements_max         = 46675               
size                      = 6035421608          
hits                      = 1096297             
mfu_hits                  = 749091              
prefetch_metadata_hits    = 10297               
hash_chains               = 87                  
evict_l2_cached           = 0                   
l2_writes_sent            = 0                   
l2_writes_error           = 0                   
deleted                   = 15                  
hdr_size                  = 12560944            
l2_compress_zeros         = 0                   
l2_abort_lowmem           = 0                   
l2_io_error               = 0                   
l2_hdr_size               = 0                   
l2_cksum_bad              = 0                   
l2_evict_reading          = 0                   
arc_meta_limit            = 16900368384         
prefetch_metadata_misses  = 3381                
l2_evict_lock_retry       = 0                   
hash_collisions           = 1102                
l2_writes_hdr_miss        = 0                   
evict_l2_ineligible       = 4096                
l2_write_bytes            = 0                   
data_size                 = 6005031424          
c                         = 67601473536         
l2_asize                  = 0                   
class                     = misc                
mutex_miss                = 0                   
snaptime                  = 1125.031042354      
duplicate_buffers_size    = 0                   
l2_misses                 = 0                   
mru_ghost_hits            = 0                   
demand_metadata_hits      = 29154               
arc_meta_max              = 99829992            
c_min                     = 2146100480          
mru_hits                  = 336372              
demand_data_misses        = 2615                
duplicate_buffers         = 0                   
demand_data_hits          = 1056309             
l2_free_on_write          = 0                   
l2_feeds                  = 0                   
misses                    = 20060               
evict_skip                = 1214                
evict_l2_eligible         = 71168               
p                         = 33800736768         
l2_compress_successes     = 0                   
recycle_miss              = 0                   
l2_writes_done            = 0                   
----------

Thu Dec  4 16:15:54 IST 2014

l2_size                   = 0                   
other_size                = 17900600            
c_max                     = 67601473536         
hash_elements             = 39997               
duplicate_reads           = 0                   
prefetch_data_hits        = 537                 
l2_read_bytes             = 0                   
demand_metadata_misses    = 12528               
prefetch_data_misses      = 1540                
hash_chain_max            = 2                   
l2_hits                   = 0                   
l2_compress_failures      = 0                   
mfu_ghost_hits            = 0                   
l2_rw_clash               = 0                   
memory_throttle_count     = 0                   
crtime                    = 4.537754192         
arc_meta_used             = 99390568            
hash_elements_max         = 46675               
size                      = 5948399720          
hits                      = 1120591             
mfu_hits                  = 765902              
prefetch_metadata_hits    = 10297               
hash_chains               = 83                  
evict_l2_cached           = 0                   
l2_writes_sent            = 0                   
l2_writes_error           = 0                   
deleted                   = 15                  
hdr_size                  = 12560944            
l2_compress_zeros         = 0                   
l2_abort_lowmem           = 0                   
l2_io_error               = 0                   
l2_hdr_size               = 0                   
l2_cksum_bad              = 0                   
l2_evict_reading          = 0                   
arc_meta_limit            = 16900368384         
prefetch_metadata_misses  = 3381                
l2_evict_lock_retry       = 0                   
hash_collisions           = 1118                
l2_writes_hdr_miss        = 0                   
evict_l2_ineligible       = 4096                
l2_write_bytes            = 0                   
data_size                 = 5917938176          
c                         = 67601473536         
l2_asize                  = 0                   
class                     = misc                
mutex_miss                = 0                   
snaptime                  = 1128.034852134      
duplicate_buffers_size    = 0                   
l2_misses                 = 0                   
mru_ghost_hits            = 0                   
demand_metadata_hits      = 29163               
arc_meta_max              = 99971728            
c_min                     = 2146100480          
mru_hits                  = 343855              
demand_data_misses        = 2615                
duplicate_buffers         = 0                   
demand_data_hits          = 1080594             
l2_free_on_write          = 0                   
l2_feeds                  = 0                   
misses                    = 20064               
evict_skip                = 1214                
evict_l2_eligible         = 71168               
p                         = 33800736768         
l2_compress_successes     = 0                   
recycle_miss              = 0                   
l2_writes_done            = 0                   
----------

Thu Dec  4 16:15:57 IST 2014

l2_size                   = 0                   
other_size                = 17931600            
c_max                     = 67601473536         
hash_elements             = 44425               
duplicate_reads           = 0                   
prefetch_data_hits        = 537                 
l2_read_bytes             = 0                   
demand_metadata_misses    = 12532               
prefetch_data_misses      = 1540                
hash_chain_max            = 2                   
l2_hits                   = 0                   
l2_compress_failures      = 0                   
mfu_ghost_hits            = 0                   
l2_rw_clash               = 0                   
memory_throttle_count     = 0                   
crtime                    = 4.537754192         
arc_meta_used             = 100027776           
hash_elements_max         = 46675               
size                      = 5889137024          
hits                      = 1123794             
mfu_hits                  = 768068              
prefetch_metadata_hits    = 10297               
hash_chains               = 102                 
evict_l2_cached           = 0                   
l2_writes_sent            = 0                   
l2_writes_error           = 0                   
deleted                   = 15                  
hdr_size                  = 12560944            
l2_compress_zeros         = 0                   
l2_abort_lowmem           = 0                   
l2_io_error               = 0                   
l2_hdr_size               = 0                   
l2_cksum_bad              = 0                   
l2_evict_reading          = 0                   
arc_meta_limit            = 16900368384         
prefetch_metadata_misses  = 3381                
l2_evict_lock_retry       = 0                   
hash_collisions           = 1137                
l2_writes_hdr_miss        = 0                   
evict_l2_ineligible       = 4096                
l2_write_bytes            = 0                   
data_size                 = 5858644480          
c                         = 67601473536         
l2_asize                  = 0                   
class                     = misc                
mutex_miss                = 0                   
snaptime                  = 1131.040711699      
duplicate_buffers_size    = 0                   
l2_misses                 = 0                   
mru_ghost_hits            = 0                   
demand_metadata_hits      = 29170               
arc_meta_max              = 99994208            
c_min                     = 2146100480          
mru_hits                  = 344892              
demand_data_misses        = 2615                
duplicate_buffers         = 0                   
demand_data_hits          = 1083790             
l2_free_on_write          = 0                   
l2_feeds                  = 0                   
misses                    = 20068               
evict_skip                = 1214                
evict_l2_eligible         = 71168               
p                         = 33800736768         
l2_compress_successes     = 0                   
recycle_miss              = 0                   
l2_writes_done            = 0                   
----------

Thu Dec  4 16:16:00 IST 2014

l2_size                   = 0                   
other_size                = 17803560            
c_max                     = 67601473536         
hash_elements             = 42523               
duplicate_reads           = 0                   
prefetch_data_hits        = 537                 
l2_read_bytes             = 0                   
demand_metadata_misses    = 12532               
prefetch_data_misses      = 1540                
hash_chain_max            = 2                   
l2_hits                   = 0                   
l2_compress_failures      = 0                   
mfu_ghost_hits            = 0                   
l2_rw_clash               = 0                   
memory_throttle_count     = 0                   
crtime                    = 4.537754192         
arc_meta_used             = 99334488            
hash_elements_max         = 46675               
size                      = 5995791704          
hits                      = 1137123             
mfu_hits                  = 776941              
prefetch_metadata_hits    = 10297               
hash_chains               = 84                  
evict_l2_cached           = 0                   
l2_writes_sent            = 0                   
l2_writes_error           = 0                   
deleted                   = 15                  
hdr_size                  = 12560944            
l2_compress_zeros         = 0                   
l2_abort_lowmem           = 0                   
l2_io_error               = 0                   
l2_hdr_size               = 0                   
l2_cksum_bad              = 0                   
l2_evict_reading          = 0                   
arc_meta_limit            = 16900368384         
prefetch_metadata_misses  = 3381                
l2_evict_lock_retry       = 0                   
hash_collisions           = 1145                
l2_writes_hdr_miss        = 0                   
evict_l2_ineligible       = 4096                
l2_write_bytes            = 0                   
data_size                 = 5965427200          
c                         = 67601473536         
l2_asize                  = 0                   
class                     = misc                
mutex_miss                = 0                   
snaptime                  = 1134.044734783      
duplicate_buffers_size    = 0                   
l2_misses                 = 0                   
mru_ghost_hits            = 0                   
demand_metadata_hits      = 29171               
arc_meta_max              = 100042776           
c_min                     = 2146100480          
mru_hits                  = 349348              
demand_data_misses        = 2615                
duplicate_buffers         = 0                   
demand_data_hits          = 1097118             
l2_free_on_write          = 0                   
l2_feeds                  = 0                   
misses                    = 20068               
evict_skip                = 1214                
evict_l2_eligible         = 71168               
p                         = 33800736768         
l2_compress_successes     = 0                   
recycle_miss              = 0                   
l2_writes_done            = 0                   
----------

Thu Dec  4 16:16:03 IST 2014

l2_size                   = 0                   
other_size                = 18005000            
c_max                     = 67601473536         
hash_elements             = 40668               
duplicate_reads           = 0                   
prefetch_data_hits        = 537                 
l2_read_bytes             = 0                   
demand_metadata_misses    = 12540               
prefetch_data_misses      = 1540                
hash_chain_max            = 2                   
l2_hits                   = 0                   
l2_compress_failures      = 0                   
mfu_ghost_hits            = 0                   
l2_rw_clash               = 0                   
memory_throttle_count     = 0                   
crtime                    = 4.537754192         
arc_meta_used             = 100146232           
hash_elements_max         = 46675               
size                      = 5979957304          
hits                      = 1157458             
mfu_hits                  = 791193              
prefetch_metadata_hits    = 10297               
hash_chains               = 85                  
evict_l2_cached           = 0                   
l2_writes_sent            = 0                   
l2_writes_error           = 0                   
deleted                   = 15                  
hdr_size                  = 12560944            
l2_compress_zeros         = 0                   
l2_abort_lowmem           = 0                   
l2_io_error               = 0                   
l2_hdr_size               = 0                   
l2_cksum_bad              = 0                   
l2_evict_reading          = 0                   
arc_meta_limit            = 16900368384         
prefetch_metadata_misses  = 3381                
l2_evict_lock_retry       = 0                   
hash_collisions           = 1162                
l2_writes_hdr_miss        = 0                   
evict_l2_ineligible       = 4096                
l2_write_bytes            = 0                   
data_size                 = 5949391360          
c                         = 67601473536         
l2_asize                  = 0                   
class                     = misc                
mutex_miss                = 0                   
snaptime                  = 1137.04862393       
duplicate_buffers_size    = 0                   
l2_misses                 = 0                   
mru_ghost_hits            = 0                   
demand_metadata_hits      = 29187               
arc_meta_max              = 100145632           
c_min                     = 2146100480          
mru_hits                  = 355431              
demand_data_misses        = 2615                
duplicate_buffers         = 0                   
demand_data_hits          = 1117437             
l2_free_on_write          = 0                   
l2_feeds                  = 0                   
misses                    = 20076               
evict_skip                = 1214                
evict_l2_eligible         = 71168               
p                         = 33800736768         
l2_compress_successes     = 0                   
recycle_miss              = 0                   
l2_writes_done            = 0                   
----------

Thu Dec  4 16:16:06 IST 2014

l2_size                   = 0                   
other_size                = 18052040            
c_max                     = 67601473536         
hash_elements             = 45931               
duplicate_reads           = 0                   
prefetch_data_hits        = 537                 
l2_read_bytes             = 0                   
demand_metadata_misses    = 12540               
prefetch_data_misses      = 1540                
hash_chain_max            = 2                   
l2_hits                   = 0                   
l2_compress_failures      = 0                   
mfu_ghost_hits            = 0                   
l2_rw_clash               = 0                   
memory_throttle_count     = 0                   
crtime                    = 4.537754192         
arc_meta_used             = 99615736            
hash_elements_max         = 46725               
size                      = 6264246264          
hits                      = 1168001             
mfu_hits                  = 798095              
prefetch_metadata_hits    = 10297               
hash_chains               = 98                  
evict_l2_cached           = 0                   
l2_writes_sent            = 0                   
l2_writes_error           = 0                   
deleted                   = 15                  
hdr_size                  = 12560944            
l2_compress_zeros         = 0                   
l2_abort_lowmem           = 0                   
l2_io_error               = 0                   
l2_hdr_size               = 0                   
l2_cksum_bad              = 0                   
l2_evict_reading          = 0                   
arc_meta_limit            = 16900368384         
prefetch_metadata_misses  = 3381                
l2_evict_lock_retry       = 0                   
hash_collisions           = 1192                
l2_writes_hdr_miss        = 0                   
evict_l2_ineligible       = 4096                
l2_write_bytes            = 0                   
data_size                 = 6233633280          
c                         = 67601473536         
l2_asize                  = 0                   
class                     = misc                
mutex_miss                = 0                   
snaptime                  = 1140.052771613      
duplicate_buffers_size    = 0                   
l2_misses                 = 0                   
mru_ghost_hits            = 0                   
demand_metadata_hits      = 29188               
arc_meta_max              = 100153872           
c_min                     = 2146100480          
mru_hits                  = 359072              
demand_data_misses        = 2615                
duplicate_buffers         = 0                   
demand_data_hits          = 1127979             
l2_free_on_write          = 0                   
l2_feeds                  = 0                   
misses                    = 20076               
evict_skip                = 1214                
evict_l2_eligible         = 71168               
p                         = 33800736768         
l2_compress_successes     = 0                   
recycle_miss              = 0                   
l2_writes_done            = 0                   
----------

Thu Dec  4 16:16:09 IST 2014

l2_size                   = 0                   
other_size                = 18144240            
c_max                     = 67601473536         
hash_elements             = 44399               
duplicate_reads           = 0                   
prefetch_data_hits        = 537                 
l2_read_bytes             = 0                   
demand_metadata_misses    = 12548               
prefetch_data_misses      = 1540                
hash_chain_max            = 2                   
l2_hits                   = 0                   
l2_compress_failures      = 0                   
mfu_ghost_hits            = 0                   
l2_rw_clash               = 0                   
memory_throttle_count     = 0                   
crtime                    = 4.537754192         
arc_meta_used             = 99752992            
hash_elements_max         = 47118               
size                      = 6250227744          
hits                      = 1186886             
mfu_hits                  = 811377              
prefetch_metadata_hits    = 10297               
hash_chains               = 155                 
evict_l2_cached           = 0                   
l2_writes_sent            = 0                   
l2_writes_error           = 0                   
deleted                   = 15                  
hdr_size                  = 12560944            
l2_compress_zeros         = 0                   
l2_abort_lowmem           = 0                   
l2_io_error               = 0                   
l2_hdr_size               = 0                   
l2_cksum_bad              = 0                   
l2_evict_reading          = 0                   
arc_meta_limit            = 16900368384         
prefetch_metadata_misses  = 3381                
l2_evict_lock_retry       = 0                   
hash_collisions           = 1302                
l2_writes_hdr_miss        = 0                   
evict_l2_ineligible       = 4096                
l2_write_bytes            = 0                   
data_size                 = 6219522560          
c                         = 67601473536         
l2_asize                  = 0                   
class                     = misc                
mutex_miss                = 0                   
snaptime                  = 1143.058190189      
duplicate_buffers_size    = 0                   
l2_misses                 = 0                   
mru_ghost_hits            = 0                   
demand_metadata_hits      = 29202               
arc_meta_max              = 100285688           
c_min                     = 2146100480          
mru_hits                  = 364675              
demand_data_misses        = 2615                
duplicate_buffers         = 0                   
demand_data_hits          = 1146850             
l2_free_on_write          = 0                   
l2_feeds                  = 0                   
misses                    = 20084               
evict_skip                = 1214                
evict_l2_eligible         = 71168               
p                         = 33800736768         
l2_compress_successes     = 0                   
recycle_miss              = 0                   
l2_writes_done            = 0                   
----------

Thu Dec  4 16:16:12 IST 2014

l2_size                   = 0                   
other_size                = 18053440            
c_max                     = 67601473536         
hash_elements             = 46005               
duplicate_reads           = 0                   
prefetch_data_hits        = 537                 
l2_read_bytes             = 0                   
demand_metadata_misses    = 12552               
prefetch_data_misses      = 1540                
hash_chain_max            = 2                   
l2_hits                   = 0                   
l2_compress_failures      = 0                   
mfu_ghost_hits            = 0                   
l2_rw_clash               = 0                   
memory_throttle_count     = 0                   
crtime                    = 4.537754192         
arc_meta_used             = 100254576           
hash_elements_max         = 47118               
size                      = 6189538160          
hits                      = 1194015             
mfu_hits                  = 815523              
prefetch_metadata_hits    = 10297               
hash_chains               = 129                 
evict_l2_cached           = 0                   
l2_writes_sent            = 0                   
l2_writes_error           = 0                   
deleted                   = 15                  
hdr_size                  = 12560944            
l2_compress_zeros         = 0                   
l2_abort_lowmem           = 0                   
l2_io_error               = 0                   
l2_hdr_size               = 0                   
l2_cksum_bad              = 0                   
l2_evict_reading          = 0                   
arc_meta_limit            = 16900368384         
prefetch_metadata_misses  = 3381                
l2_evict_lock_retry       = 0                   
hash_collisions           = 1311                
l2_writes_hdr_miss        = 0                   
evict_l2_ineligible       = 397312              
l2_write_bytes            = 0                   
data_size                 = 6158923776          
c                         = 6190066904          
l2_asize                  = 0                   
class                     = misc                
mutex_miss                = 0                   
snaptime                  = 1146.06243032       
duplicate_buffers_size    = 0                   
l2_misses                 = 0                   
mru_ghost_hits            = 0                   
demand_metadata_hits      = 29214               
arc_meta_max              = 100285688           
c_min                     = 2146100480          
mru_hits                  = 367658              
demand_data_misses        = 2615                
duplicate_buffers         = 0                   
demand_data_hits          = 1153967             
l2_free_on_write          = 0                   
l2_feeds                  = 0                   
misses                    = 20088               
evict_skip                = 1214                
evict_l2_eligible         = 1011712             
p                         = 3096114796          
l2_compress_successes     = 0                   
recycle_miss              = 1                   
l2_writes_done            = 0                   
----------

Thu Dec  4 16:16:15 IST 2014

l2_size                   = 0                   
other_size                = 17695320            
c_max                     = 67601473536         
hash_elements             = 44156               
duplicate_reads           = 0                   
prefetch_data_hits        = 537                 
l2_read_bytes             = 0                   
demand_metadata_misses    = 12556               
prefetch_data_misses      = 1540                
hash_chain_max            = 2                   
l2_hits                   = 0                   
l2_compress_failures      = 0                   
mfu_ghost_hits            = 0                   
l2_rw_clash               = 0                   
memory_throttle_count     = 0                   
crtime                    = 4.537754192         
arc_meta_used             = 99370392            
hash_elements_max         = 47845               
size                      = 6062509464          
hits                      = 1214370             
mfu_hits                  = 831061              
prefetch_metadata_hits    = 10297               
hash_chains               = 121                 
evict_l2_cached           = 0                   
l2_writes_sent            = 0                   
l2_writes_error           = 0                   
deleted                   = 15                  
hdr_size                  = 12583744            
l2_compress_zeros         = 0                   
l2_abort_lowmem           = 0                   
l2_io_error               = 0                   
l2_hdr_size               = 0                   
l2_cksum_bad              = 0                   
l2_evict_reading          = 0                   
arc_meta_limit            = 16900368384         
prefetch_metadata_misses  = 3381                
l2_evict_lock_retry       = 0                   
hash_collisions           = 1369                
l2_writes_hdr_miss        = 0                   
evict_l2_ineligible       = 132255744           
l2_write_bytes            = 0                   
data_size                 = 6032230400          
c                         = 6190066904          
l2_asize                  = 0                   
class                     = misc                
mutex_miss                = 1                   
snaptime                  = 1149.080481297      
duplicate_buffers_size    = 0                   
l2_misses                 = 0                   
mru_ghost_hits            = 0                   
demand_metadata_hits      = 29222               
arc_meta_max              = 100285688           
c_min                     = 2146100480          
mru_hits                  = 372475              
demand_data_misses        = 2615                
duplicate_buffers         = 0                   
demand_data_hits          = 1174314             
l2_free_on_write          = 0                   
l2_feeds                  = 0                   
misses                    = 20092               
evict_skip                = 1214                
evict_l2_eligible         = 285229056           
p                         = 3401023084          
l2_compress_successes     = 0                   
recycle_miss              = 219                 
l2_writes_done            = 0                   
----------

Thu Dec  4 16:16:18 IST 2014

l2_size                   = 0                   
other_size                = 17003120            
c_max                     = 67601473536         
hash_elements             = 49196               
duplicate_reads           = 0                   
prefetch_data_hits        = 537                 
l2_read_bytes             = 0                   
demand_metadata_misses    = 12556               
prefetch_data_misses      = 1540                
hash_chain_max            = 2                   
l2_hits                   = 0                   
l2_compress_failures      = 0                   
mfu_ghost_hits            = 0                   
l2_rw_clash               = 0                   
memory_throttle_count     = 0                   
crtime                    = 4.537754192         
arc_meta_used             = 97914576            
hash_elements_max         = 49196               
size                      = 5597583056          
hits                      = 1214720             
mfu_hits                  = 831312              
prefetch_metadata_hits    = 10297               
hash_chains               = 158                 
evict_l2_cached           = 0                   
l2_writes_sent            = 0                   
l2_writes_error           = 0                   
deleted                   = 15                  
hdr_size                  = 11820128            
l2_compress_zeros         = 0                   
l2_abort_lowmem           = 0                   
l2_io_error               = 0                   
l2_hdr_size               = 0                   
l2_cksum_bad              = 0                   
l2_evict_reading          = 0                   
arc_meta_limit            = 16900368384         
prefetch_metadata_misses  = 3381                
l2_evict_lock_retry       = 0                   
hash_collisions           = 1414                
l2_writes_hdr_miss        = 0                   
evict_l2_ineligible       = 132255744           
l2_write_bytes            = 0                   
data_size                 = 5568759808          
c                         = 5859249032          
l2_asize                  = 0                   
class                     = misc                
mutex_miss                = 1                   
snaptime                  = 1152.093743051      
duplicate_buffers_size    = 0                   
l2_misses                 = 0                   
mru_ghost_hits            = 0                   
demand_metadata_hits      = 29222               
arc_meta_max              = 100285688           
c_min                     = 2146100480          
mru_hits                  = 372574              
demand_data_misses        = 2615                
duplicate_buffers         = 0                   
demand_data_hits          = 1174664             
l2_free_on_write          = 0                   
l2_feeds                  = 0                   
misses                    = 20092               
evict_skip                = 1214                
evict_l2_eligible         = 285491200           
p                         = 3295003257          
l2_compress_successes     = 0                   
recycle_miss              = 219                 
l2_writes_done            = 0                   
----------

Thu Dec  4 16:16:21 IST 2014

l2_size                   = 0                   
other_size                = 15860920            
c_max                     = 67601473536         
hash_elements             = 44130               
duplicate_reads           = 0                   
prefetch_data_hits        = 537                 
l2_read_bytes             = 0                   
demand_metadata_misses    = 12564               
prefetch_data_misses      = 1540                
hash_chain_max            = 2                   
l2_hits                   = 0                   
l2_compress_failures      = 0                   
mfu_ghost_hits            = 0                   
l2_rw_clash               = 0                   
memory_throttle_count     = 0                   
crtime                    = 4.537754192         
arc_meta_used             = 91144424            
hash_elements_max         = 49670               
size                      = 4804212968          
hits                      = 1244167             
mfu_hits                  = 851345              
prefetch_metadata_hits    = 10297               
hash_chains               = 102                 
evict_l2_cached           = 0                   
l2_writes_sent            = 0                   
l2_writes_error           = 0                   
deleted                   = 1063                
hdr_size                  = 11332144            
l2_compress_zeros         = 0                   
l2_abort_lowmem           = 0                   
l2_io_error               = 0                   
l2_hdr_size               = 0                   
l2_cksum_bad              = 0                   
l2_evict_reading          = 0                   
arc_meta_limit            = 16900368384         
prefetch_metadata_misses  = 3381                
l2_evict_lock_retry       = 0                   
hash_collisions           = 1433                
l2_writes_hdr_miss        = 0                   
evict_l2_ineligible       = 137154560           
l2_write_bytes            = 0                   
data_size                 = 4777019904          
c                         = 4879960441          
l2_asize                  = 0                   
class                     = misc                
mutex_miss                = 9                   
snaptime                  = 1155.099409293      
duplicate_buffers_size    = 0                   
l2_misses                 = 0                   
mru_ghost_hits            = 24                  
demand_metadata_hits      = 29238               
arc_meta_max              = 100285688           
c_min                     = 2146100480          
mru_hits                  = 381988              
demand_data_misses        = 2639                
duplicate_buffers         = 0                   
demand_data_hits          = 1204095             
l2_free_on_write          = 0                   
l2_feeds                  = 0                   
misses                    = 20124               
evict_skip                = 1214                
evict_l2_eligible         = 1330413056          
p                         = 3274886866          
l2_compress_successes     = 0                   
recycle_miss              = 519                 
l2_writes_done            = 0                   
----------

Thu Dec  4 16:16:24 IST 2014

l2_size                   = 0                   
other_size                = 15703800            
c_max                     = 67601473536         
hash_elements             = 41125               
duplicate_reads           = 0                   
prefetch_data_hits        = 537                 
l2_read_bytes             = 0                   
demand_metadata_misses    = 12572               
prefetch_data_misses      = 1540                
hash_chain_max            = 2                   
l2_hits                   = 0                   
l2_compress_failures      = 0                   
mfu_ghost_hits            = 0                   
l2_rw_clash               = 0                   
memory_throttle_count     = 0                   
crtime                    = 4.537754192         
arc_meta_used             = 85702920            
hash_elements_max         = 49670               
size                      = 4396642568          
hits                      = 1257727             
mfu_hits                  = 860930              
prefetch_metadata_hits    = 10297               
hash_chains               = 92                  
evict_l2_cached           = 0                   
l2_writes_sent            = 0                   
l2_writes_error           = 0                   
deleted                   = 3379                
hdr_size                  = 10493456            
l2_compress_zeros         = 0                   
l2_abort_lowmem           = 0                   
l2_io_error               = 0                   
l2_hdr_size               = 0                   
l2_cksum_bad              = 0                   
l2_evict_reading          = 0                   
arc_meta_limit            = 16900368384         
prefetch_metadata_misses  = 3383                
l2_evict_lock_retry       = 0                   
hash_collisions           = 1442                
l2_writes_hdr_miss        = 0                   
evict_l2_ineligible       = 142102528           
l2_write_bytes            = 0                   
data_size                 = 4370445312          
c                         = 4396690358          
l2_asize                  = 0                   
class                     = misc                
mutex_miss                = 9                   
snaptime                  = 1158.10348092       
duplicate_buffers_size    = 0                   
l2_misses                 = 0                   
mru_ghost_hits            = 56                  
demand_metadata_hits      = 29252               
arc_meta_max              = 100285688           
c_min                     = 2146100480          
mru_hits                  = 385963              
demand_data_misses        = 2669                
duplicate_buffers         = 0                   
demand_data_hits          = 1217641             
l2_free_on_write          = 0                   
l2_feeds                  = 0                   
misses                    = 20164               
evict_skip                = 1214                
evict_l2_eligible         = 1491629056          
p                         = 2997814168          
l2_compress_successes     = 0                   
recycle_miss              = 821                 
l2_writes_done            = 0                   
----------

Thu Dec  4 16:16:27 IST 2014

l2_size                   = 0                   
other_size                = 14938800            
c_max                     = 67601473536         
hash_elements             = 39052               
duplicate_reads           = 0                   
prefetch_data_hits        = 537                 
l2_read_bytes             = 0                   
demand_metadata_misses    = 12576               
prefetch_data_misses      = 1540                
hash_chain_max            = 2                   
l2_hits                   = 0                   
l2_compress_failures      = 0                   
mfu_ghost_hits            = 0                   
l2_rw_clash               = 0                   
memory_throttle_count     = 0                   
crtime                    = 4.537754192         
arc_meta_used             = 80821952            
hash_elements_max         = 49670               
size                      = 3828283072          
hits                      = 1264949             
mfu_hits                  = 864148              
prefetch_metadata_hits    = 10297               
hash_chains               = 75                  
evict_l2_cached           = 0                   
l2_writes_sent            = 0                   
l2_writes_error           = 0                   
deleted                   = 10138               
hdr_size                  = 8953872             
l2_compress_zeros         = 0                   
l2_abort_lowmem           = 0                   
l2_io_error               = 0                   
l2_hdr_size               = 0                   
l2_cksum_bad              = 0                   
l2_evict_reading          = 0                   
arc_meta_limit            = 16900368384         
prefetch_metadata_misses  = 3383                
l2_evict_lock_retry       = 0                   
hash_collisions           = 1454                
l2_writes_hdr_miss        = 0                   
evict_l2_ineligible       = 144691200           
l2_write_bytes            = 0                   
data_size                 = 3804390400          
c                         = 3830309700          
l2_asize                  = 0                   
class                     = misc                
mutex_miss                = 12                  
snaptime                  = 1161.109480472      
duplicate_buffers_size    = 0                   
l2_misses                 = 0                   
mru_ghost_hits            = 77                  
demand_metadata_hits      = 29256               
arc_meta_max              = 100285688           
c_min                     = 2146100480          
mru_hits                  = 389967              
demand_data_misses        = 2692                
duplicate_buffers         = 0                   
demand_data_hits          = 1224859             
l2_free_on_write          = 0                   
l2_feeds                  = 0                   
misses                    = 20191               
evict_skip                = 1214                
evict_l2_eligible         = 1951429632          
p                         = 2764099257          
l2_compress_successes     = 0                   
recycle_miss              = 972                 
l2_writes_done            = 0                   
----------

-------------- next part --------------
Thu Dec  4 16:14:02 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                    5667640             22139   34%
ZFS File Data             1171645              4576    7%
Anon                        26071               101    0%
Exec and libs                1181                 4    0%
Page cache                   4609                18    0%
Free (cachelist)             7723                30    0%
Free (freelist)           9887541             38623   59%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:14:06 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                    6380424             24923   38%
ZFS File Data             1171645              4576    7%
Anon                        26096               101    0%
Exec and libs                1175                 4    0%
Page cache                   4610                18    0%
Free (cachelist)             7729                30    0%
Free (freelist)           9174731             35838   55%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:14:09 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                    6524733             25487   39%
ZFS File Data             1184093              4625    7%
Anon                        26096               101    0%
Exec and libs                1181                 4    0%
Page cache                   4612                18    0%
Free (cachelist)             7722                30    0%
Free (freelist)           9017973             35226   54%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:14:12 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                    7290171             28477   43%
ZFS File Data             1184093              4625    7%
Anon                        26122               102    0%
Exec and libs                1181                 4    0%
Page cache                   4612                18    0%
Free (cachelist)             7723                30    0%
Free (freelist)           8252508             32236   49%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:14:16 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                    7456599             29127   44%
ZFS File Data             1190237              4649    7%
Anon                        26122               102    0%
Exec and libs                1181                 4    0%
Page cache                   4613                18    0%
Free (cachelist)             7723                30    0%
Free (freelist)           8079935             31562   48%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:14:19 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                    7480950             29222   45%
ZFS File Data             1513949              5913    9%
Anon                        26178               102    0%
Exec and libs                1180                 4    0%
Page cache                   4615                18    0%
Free (cachelist)             7724                30    0%
Free (freelist)           7731814             30202   46%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:14:25 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                    7481258             29223   45%
ZFS File Data             1513949              5913    9%
Anon                        26174               102    0%
Exec and libs                1181                 4    0%
Page cache                   4616                18    0%
Free (cachelist)             7723                30    0%
Free (freelist)           7731509             30201   46%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:14:28 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                    7482405             29228   45%
ZFS File Data             1577181              6160    9%
Anon                        26200               102    0%
Exec and libs                1181                 4    0%
Page cache                   4617                18    0%
Free (cachelist)             7723                30    0%
Free (freelist)           7667103             29949   46%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:14:31 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                    7482519             29228   45%
ZFS File Data             1577181              6160    9%
Anon                        26226               102    0%
Exec and libs                1181                 4    0%
Page cache                   4618                18    0%
Free (cachelist)             7723                30    0%
Free (freelist)           7666962             29949   46%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:14:35 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                    7482590             29228   45%
ZFS File Data             1577181              6160    9%
Anon                        26226               102    0%
Exec and libs                1180                 4    0%
Page cache                   4619                18    0%
Free (cachelist)             7723                30    0%
Free (freelist)           7666891             29948   46%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:14:38 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                    7482751             29229   45%
ZFS File Data             1577181              6160    9%
Anon                        26252               102    0%
Exec and libs                1181                 4    0%
Page cache                   4619                18    0%
Free (cachelist)             7723                30    0%
Free (freelist)           7666703             29948   46%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:14:41 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                    7482755             29229   45%
ZFS File Data             1577181              6160    9%
Anon                        26252               102    0%
Exec and libs                1181                 4    0%
Page cache                   4622                18    0%
Free (cachelist)             7723                30    0%
Free (freelist)           7666696             29948   46%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:14:44 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                    7483021             29230   45%
ZFS File Data             1577181              6160    9%
Anon                        26278               102    0%
Exec and libs                1181                 4    0%
Page cache                   4623                18    0%
Free (cachelist)             7723                30    0%
Free (freelist)           7666403             29946   46%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:14:48 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                    7483581             29232   45%
ZFS File Data             1577181              6160    9%
Anon                        26303               102    0%
Exec and libs                1181                 4    0%
Page cache                   4622                18    0%
Free (cachelist)             7724                30    0%
Free (freelist)           7665818             29944   46%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:14:51 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                    7645816             29866   46%
ZFS File Data             1577181              6160    9%
Anon                        26328               102    0%
Exec and libs                1181                 4    0%
Page cache                   4624                18    0%
Free (cachelist)             7723                30    0%
Free (freelist)           7503557             29310   45%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:14:55 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                    7645904             29866   46%
ZFS File Data             1577181              6160    9%
Anon                        26328               102    0%
Exec and libs                1181                 4    0%
Page cache                   4625                18    0%
Free (cachelist)             7723                30    0%
Free (freelist)           7503468             29310   45%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:14:58 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                    7753278             30286   46%
ZFS File Data             1577181              6160    9%
Anon                        26355               102    0%
Exec and libs                1181                 4    0%
Page cache                   4626                18    0%
Free (cachelist)             7723                30    0%
Free (freelist)           7396066             28890   44%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:15:01 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                    8085909             31585   48%
ZFS File Data             1577181              6160    9%
Anon                        26380               103    0%
Exec and libs                1179                 4    0%
Page cache                   4627                18    0%
Free (cachelist)             7725                30    0%
Free (freelist)           7063409             27591   42%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:15:04 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                    8085951             31585   48%
ZFS File Data             1577181              6160    9%
Anon                        26380               103    0%
Exec and libs                1181                 4    0%
Page cache                   4628                18    0%
Free (cachelist)             7723                30    0%
Free (freelist)           7063366             27591   42%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:15:08 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                    8209025             32066   49%
ZFS File Data             1577181              6160    9%
Anon                        26407               103    0%
Exec and libs                1181                 4    0%
Page cache                   4630                18    0%
Free (cachelist)             7723                30    0%
Free (freelist)           6940263             27110   41%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:15:11 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                    8763032             34230   52%
ZFS File Data             1577181              6160    9%
Anon                        26432               103    0%
Exec and libs                1181                 4    0%
Page cache                   4630                18    0%
Free (cachelist)             7723                30    0%
Free (freelist)           6386231             24946   38%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:15:14 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                    9388716             36674   56%
ZFS File Data             1577181              6160    9%
Anon                        26432               103    0%
Exec and libs                1181                 4    0%
Page cache                   4631                18    0%
Free (cachelist)             7724                30    0%
Free (freelist)           5760545             22502   34%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:15:17 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                    9586189             37446   57%
ZFS File Data             1577181              6160    9%
Anon                        26459               103    0%
Exec and libs                1181                 4    0%
Page cache                   4632                18    0%
Free (cachelist)             7723                30    0%
Free (freelist)           5563045             21730   33%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:15:20 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                    9967338             38934   59%
ZFS File Data             1577181              6160    9%
Anon                        26459               103    0%
Exec and libs                1181                 4    0%
Page cache                   4633                18    0%
Free (cachelist)             7723                30    0%
Free (freelist)           5181895             20241   31%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:15:24 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                   10087818             39405   60%
ZFS File Data             1577181              6160    9%
Anon                        26484               103    0%
Exec and libs                1181                 4    0%
Page cache                   4635                18    0%
Free (cachelist)             7723                30    0%
Free (freelist)           5061388             19771   30%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:15:27 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                   10620046             41484   63%
ZFS File Data             1577181              6160    9%
Anon                        26510               103    0%
Exec and libs                1157                 4    0%
Page cache                   4633                18    0%
Free (cachelist)             7750                30    0%
Free (freelist)           4529133             17691   27%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:15:30 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                   10728161             41906   64%
ZFS File Data             1577181              6160    9%
Anon                        26510               103    0%
Exec and libs                1181                 4    0%
Page cache                   4637                18    0%
Free (cachelist)             7723                30    0%
Free (freelist)           4421017             17269   26%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:15:34 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                   11230924             43870   67%
ZFS File Data             1577181              6160    9%
Anon                        26536               103    0%
Exec and libs                1181                 4    0%
Page cache                   4637                18    0%
Free (cachelist)             7723                30    0%
Free (freelist)           3918228             15305   23%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:15:38 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                   11277074             44051   67%
ZFS File Data             1577981              6163    9%
Anon                        26562               103    0%
Exec and libs                1181                 4    0%
Page cache                   4638                18    0%
Free (cachelist)             7723                30    0%
Free (freelist)           3871251             15122   23%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:15:41 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                   11779320             46012   70%
ZFS File Data             1577981              6163    9%
Anon                        26562               103    0%
Exec and libs                1180                 4    0%
Page cache                   4639                18    0%
Free (cachelist)             7724                30    0%
Free (freelist)           3369004             13160   20%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:15:44 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                   11968305             46751   71%
ZFS File Data             1577981              6163    9%
Anon                        26588               103    0%
Exec and libs                1181                 4    0%
Page cache                   4641                18    0%
Free (cachelist)             7723                30    0%
Free (freelist)           3179991             12421   19%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:15:47 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                   12424008             48531   74%
ZFS File Data             1577981              6163    9%
Anon                        26614               103    0%
Exec and libs                1181                 4    0%
Page cache                   4642                18    0%
Free (cachelist)             7723                30    0%
Free (freelist)           2724261             10641   16%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:15:51 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                   12708257             49641   76%
ZFS File Data             1577981              6163    9%
Anon                        26614               103    0%
Exec and libs                1181                 4    0%
Page cache                   4643                18    0%
Free (cachelist)             7723                30    0%
Free (freelist)           2440011              9531   15%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:15:54 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                   12960558             50627   77%
ZFS File Data             1577981              6163    9%
Anon                        26640               104    0%
Exec and libs                1181                 4    0%
Page cache                   4644                18    0%
Free (cachelist)             7723                30    0%
Free (freelist)           2187683              8545   13%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:15:58 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                   13441434             52505   80%
ZFS File Data             1577981              6163    9%
Anon                        26664               104    0%
Exec and libs                1181                 4    0%
Page cache                   4645                18    0%
Free (cachelist)             7723                30    0%
Free (freelist)           1706782              6667   10%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:16:01 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                   13724075             53609   82%
ZFS File Data             1577981              6163    9%
Anon                        26691               104    0%
Exec and libs                1181                 4    0%
Page cache                   4646                18    0%
Free (cachelist)             7723                30    0%
Free (freelist)           1424113              5562    8%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:16:05 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                   14075430             54982   84%
ZFS File Data             1577981              6163    9%
Anon                        26691               104    0%
Exec and libs                1181                 4    0%
Page cache                   4647                18    0%
Free (cachelist)             7723                30    0%
Free (freelist)           1072757              4190    6%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:16:08 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                   14302316             55868   85%
ZFS File Data             1577981              6163    9%
Anon                        26723               104    0%
Exec and libs                1181                 4    0%
Page cache                   4649                18    0%
Free (cachelist)             7723                30    0%
Free (freelist)            845837              3304    5%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:16:12 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                   14585839             56975   87%
ZFS File Data             1541308              6020    9%
Anon                        26743               104    0%
Exec and libs                1181                 4    0%
Page cache                   4649                18    0%
Free (cachelist)             7724                30    0%
Free (freelist)            598966              2339    4%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:16:16 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                   14682403             57353   88%
ZFS File Data             1408152              5500    8%
Anon                        26768               104    0%
Exec and libs                1181                 4    0%
Page cache                   4651                18    0%
Free (cachelist)             7723                30    0%
Free (freelist)            635532              2482    4%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:16:19 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                   15056031             58812   90%
ZFS File Data             1163626              4545    7%
Anon                        26778               104    0%
Exec and libs                1181                 4    0%
Page cache                   4652                18    0%
Free (cachelist)             7723                30    0%
Free (freelist)            506419              1978    3%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:16:23 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                   15285492             59708   91%
ZFS File Data             1008029              3937    6%
Anon                        26795               104    0%
Exec and libs                1181                 4    0%
Page cache                   4653                18    0%
Free (cachelist)             7429                29    0%
Free (freelist)            432831              1690    3%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:16:27 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                   15705417             61349   94%
ZFS File Data              857418              3349    5%
Anon                        21468                83    0%
Exec and libs                  73                 0    0%
Page cache                    721                 2    0%
Free (cachelist)            14622                57    0%
Free (freelist)            166691               651    1%

Total                    16766410             65493
Physical                 16766409             65493


-------------- next part --------------
last pid:   996;  load avg:  5.95,  3.04,  1.31;  up 0+00:19:43                                                                                                                 16:16:27
59 processes: 58 sleeping, 1 on cpu
CPU states: 30.6% idle,  1.6% user, 67.8% kernel,  0.0% iowait,  0.0% swap
Kernel: 97556 ctxsw, 2440 trap, 448800 intr, 27430 syscall, 1 fork, 1407 flt
Memory: 64G phys mem, 933M free mem, 4096M total swap, 4096M free swap

   PID USERNAME NLWP PRI NICE  SIZE   RES STATE    TIME    CPU COMMAND
   675 daemon     10  60  -20 2904K 1968K sleep    2:28  4.35% nfsd
   607 root        1  59    0 8308K 6668K sleep    0:00  0.01% perl
   631 root        1  59    0   26M   25M cpu/5    0:00  0.01% top
   711 root        1  59    0 5888K 4180K sleep    0:00  0.01% perl
   705 root        1  59    0 4712K 2628K sleep    0:00  0.00% perl
   633 root        1  59    0 7576K 4828K sleep    0:00  0.00% sshd
   712 root        1  59    0 1644K 1040K sleep    0:00  0.00% tee
   622 root        1  59    0 7576K 4828K sleep    0:00  0.00% sshd
   645 root        1  59    0 7576K 4832K sleep    0:00  0.00% sshd
    10 root       14  59    0 7684K 5664K sleep    0:01  0.00% svc.startd
   580 root       27  59    0   23M   16M sleep    0:01  0.00% fmd
   706 root        1  59    0 1644K 1040K sleep    0:00  0.00% tee
    12 root       19  59    0 9704K 8372K sleep    0:04  0.00% svc.configd
   241 root        7  59    0 4456K 3220K sleep    0:00  0.00% devfsadm
   294 root       34  59    0 7480K 4936K sleep    0:00  0.00% nscd
   367 root        1  59    0 1656K  940K sleep    0:00  0.00% utmpd
   658 daemon      2  60  -20 2888K 1912K sleep    0:00  0.00% lockd
   232 root        5  60  -20 2712K 1600K sleep    0:00  0.00% zonestatd
   322 root        4  59    0   11M 8224K sleep    0:00  0.00% hald
   301 root        5  59    0 5232K 4324K sleep    0:00  0.00% picld
   487 root        3  59    0 5272K 3688K sleep    0:00  0.00% inetd
   331 root        2  59    0 6660K 3368K sleep    0:00  0.00% hald-runner
   225 root       17  59    0 6164K 3312K sleep    0:00  0.00% syseventd
   632 root        1  59    0 5804K 3192K sleep    0:00  0.00% sshd
   644 root        1  59    0 5804K 3192K sleep    0:00  0.00% sshd
   612 root        1  59    0 6508K 3180K sleep    0:00  0.00% hald-addon-netw
   614 root        1  59    0 6508K 3180K sleep    0:00  0.00% hald-addon-cpuf
   621 root        1  59    0 5804K 3096K sleep    0:00  0.00% sshd
    49 netadm      3  59    0 4084K 2836K sleep    0:00  0.00% ipmgmtd
   673 root       13  59    0 4588K 2744K sleep    0:00  0.00% mountd


From henson at acm.org  Fri Dec  5 08:09:17 2014
From: henson at acm.org (Paul B. Henson)
Date: Fri, 5 Dec 2014 00:09:17 -0800
Subject: [OmniOS-discuss] Kernel stability.
In-Reply-To: <54815722.3020909@evogene.com>
References: <54815722.3020909@evogene.com>
Message-ID: <20141205080917.GM29549@bender.unx.csupomona.edu>

On Fri, Dec 05, 2014 at 08:56:34AM +0200, Kobi Schwartz wrote:
>
> It can only be brought to life by power cycling it.

I'm in no way qualified to diagnose this :), but perhaps you could set
your box up to panic on NMI:

http://www.cuddletech.com/blog/pivot/entry.php?id=1044

And next time this happens, instead of just resetting, force a panic and
get a kernel dump? That would be helpful to someone who actually would
be qualified to look at the issue...

BTW, on newer boxes you need:

set apix:apic_panic_on_nmi = 1 

instead of:

set pcplusmp:apic_kmdb_on_nmi=1

I usually just set both to cover the bases...

From filip.marvan at aira.cz  Fri Dec  5 09:10:58 2014
From: filip.marvan at aira.cz (Filip Marvan)
Date: Fri, 5 Dec 2014 10:10:58 +0100
Subject: [OmniOS-discuss] zpool import strange informations
In-Reply-To: <6E5A567E-6AC9-475F-8FBD-3133C72E58D1@richardelling.com>
References: <3BE0DEED8863E5429BAE4CAEDF624565039C1B330761@AIRA-SRV.aira.local>
	<CALLZ=vV4D6yDai9mdnq7Z6odinxFARSSCTtz_nHyEyD4Jfz4=A@mail.gmail.com>
	<3BE0DEED8863E5429BAE4CAEDF624565039C1B33076D@AIRA-SRV.aira.local>
	<21BA9319-381E-4303-8B49-93B7693FFCE7@richardelling.com>
	<3BE0DEED8863E5429BAE4CAEDF624565039C1B3307EF@AIRA-SRV.aira.local>
	<6E5A567E-6AC9-475F-8FBD-3133C72E58D1@richardelling.com>
Message-ID: <3BE0DEED8863E5429BAE4CAEDF624565039C1B3308DC@AIRA-SRV.aira.local>

Hi, 

great! Thank you.

With this command, I can see something like that:

/dev/dsk/c3t2d0
LABEL 2
and
LABEL 3 
--------------------------------------------
    version: 15
    name: 'tank'
    state: 0
    txg: 121888
    pool_guid: 4916680739993977334
    hostid: 3848478874
    hostname: 'xxx'
    top_guid: 11305277212649201582
    guid: 11245012245677467134
...

And for
/dev/dsk/c3t2d0s0

LABEL 0-3
--------------------------------------------
    version: 5000
    name: 'raid_10'
    state: 0
    txg: 7494567
    pool_guid: 1071744107269411238
    hostid: 134742016
    hostname: 'dnas'
    top_guid: 565884742309860805
    guid: 13206765472826992314
    vdev_children: 3

So it seems, that /dev/dsk/c3t2d0 have some bad (old) labels, and
/dev/dsk/c3t2d0s0 have correct one. How can I remove the bad unused labels
and how can I avoid this in future when I create new pool with formerly used
disks?

Thank you very much,
Filip


-----Original Message-----
From: Richard Elling [mailto:richard.elling at richardelling.com] 
Sent: Wednesday, December 03, 2014 6:02 PM
To: Filip Marvan
Cc: Zach Malone; omnios-discuss at lists.omniti.com
Subject: Re: [OmniOS-discuss] zpool import strange informations

Try this instead:
	for i in /dev/dsk/c3t2d0*
	do
		echo $i
		zdb -l $i
	done

This will show you the ZFS labels on each and every slice/partition for the
disk.
Ideally, you'll only see one set of ZFS labels for each disk.
 -- richard

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 6247 bytes
Desc: not available
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141205/0a62d92f/attachment.bin>

From orilali at evogene.com  Fri Dec  5 10:19:59 2014
From: orilali at evogene.com (Kobi Schwartz)
Date: Fri, 05 Dec 2014 12:19:59 +0200
Subject: [OmniOS-discuss] Kernel stability.
In-Reply-To: <20141205080917.GM29549@bender.unx.csupomona.edu>
References: <54815722.3020909@evogene.com>
	<20141205080917.GM29549@bender.unx.csupomona.edu>
Message-ID: <548186CF.9090207@evogene.com>

Hi Paul,

Thanks for the tip. I already changed /etc/system.

Kobi.

On 12/05/2014 10:09 AM, Paul B. Henson wrote:
> On Fri, Dec 05, 2014 at 08:56:34AM +0200, Kobi Schwartz wrote:
>> It can only be brought to life by power cycling it.
> I'm in no way qualified to diagnose this :), but perhaps you could set
> your box up to panic on NMI:
>
> http://www.cuddletech.com/blog/pivot/entry.php?id=1044
>
> And next time this happens, instead of just resetting, force a panic and
> get a kernel dump? That would be helpful to someone who actually would
> be qualified to look at the issue...
>
> BTW, on newer boxes you need:
>
> set apix:apic_panic_on_nmi = 1
>
> instead of:
>
> set pcplusmp:apic_kmdb_on_nmi=1
>
> I usually just set both to cover the bases...
>


From gate03 at landcroft.co.uk  Fri Dec  5 21:00:52 2014
From: gate03 at landcroft.co.uk (Michael Mounteney)
Date: Sat, 6 Dec 2014 07:00:52 +1000
Subject: [OmniOS-discuss] UPS serial non-standard, WHY?
In-Reply-To: <20141205080411.GI19035@wagner-net.com>
References: <20141205110842.2da41bea@punda-mlia>
	<20141205080411.GI19035@wagner-net.com>
Message-ID: <20141206070052.6367a1a8@punda-mlia>

On Fri, 5 Dec 2014 09:04:11 +0100
Thomas Wagner <wagner at wagner-net.com> wrote:

> Michael,
> I can't say something on the serial connection.
> But I believe that some of the PINs like "carrier detect"
> are used to signal power conditions and in the other direction
> tell the UPS to power on or off.
> Maybe the "null-modem" case where the special cable from APC
> swaps everything.

It's not a proper 'null modem' (DTE) layout;  just pins 2 and 3 are
swapped.

http://www.networkupstools.org/ups-protocols/apcsmart.html

> About NUT, I compiled this for the SFE repository (RPM style
> build recipes).

Thanks but I'm going to bite the bullet and work out how to build the
packages I want in my own repository.  So far:  OpenLDAP and NUT, and
I'm sure there'll be more.  I'm going to have to do it at some stage.

Michael.

From jklimek at gmail.com  Fri Dec  5 13:32:01 2014
From: jklimek at gmail.com (John Klimek)
Date: Fri, 5 Dec 2014 08:32:01 -0500
Subject: [OmniOS-discuss] NFSv4 id mapping only working on client but
	not server?
In-Reply-To: <4ced01d01005$44363af0$cca2b0d0$@acm.org>
References: <CAOEv1kYdT=Cj-K7KarcEFizXKe8g8aehKm5uaHORqfqzAxUFoA@mail.gmail.com>
	<20141204193711.03524183@sleipner.datanom.net>
	<CAOEv1kaoxj=AREpfuRo8saHFhW18i0CmgSd9dfwLZEtCUv8pmw@mail.gmail.com>
	<20141204201107.43ad5928@sleipner.datanom.net>
	<4ced01d01005$44363af0$cca2b0d0$@acm.org>
Message-ID: <CAOEv1kZo394CCe7tTRaTNd_tPo7evc2j+72FVGw-QrRS4o7LQg@mail.gmail.com>

Paul, thanks!  You understand exactly what I'm asking and what my problem
is.

Something seems to be going wrong (or incorrectly configured) with the NFS
id mapping services on my Linux client or OmniOS server.

I've setup the nfsmapid_domain and I believe it's working because before
configuring it, everything was showing as nobody/nobody, but afterwards it
shows the correct names.  (on the client)  I can also see in the client log
and it's correctly mapping user at domain.com in the Linux rpc.idmapd domain.

The problem is that when I create a file on the client, the server (OmniOS)
is showing the UID of the user that created the file instead of mapping it
using the name to the local UID.

It looks like I can avoid the problem if I synchronize UIDs and GIDs across
all servers (it's a small home network so it's not a big problem and I'm
not using LDAP or NIS), but NFSv4 was supposed to avoid this problem by
using name mapping so you didn't have to synchronize UIDs/GIDs.



On Thu, Dec 4, 2014 at 3:59 PM, Paul B. Henson <henson at acm.org> wrote:

> > From: Michael Rasmussen
> > Sent: Thursday, December 04, 2014 11:11 AM
> >
> > Yes, because you want to avoid Omnios presents ACL which is incompatible
> > with Linux ACL.
>
> I don't believe the ACL has anything to do with NFSv4 id mapping? And a ZFS
> ACL presented over NFSv4 is perfectly compatible with Linux. It's not a
> Linux POSIX ACL, and cannot be manipulated with getfacl/setfacl, you need
> to
> use the nfs4 ACL tools, but it works fine.
>
> > http://forum.proxmox.com/threads/15793-CT-creation-on-NFS-
> > Share?p=81530#post81530
>
> In that thread, the user fails to chmod via NFS:
>
> chmod: changing permissions of `/mnt/pve/proxCT/private/108.tmp': Operation
> not permitted
>
> The root cause of which was a setting of restricted for aclmode:
>
> vdev1/proxCT aclmode             restricted
> local
>
> Per the man page "An aclmode property of restricted will cause the chmod(2)
> operation to return an error when used on any file or directory which has a
> non-trivial ACL whose entries can not be represented by a mode."
>
> The user could have set the inherited ACL on the initial filesystem to a
> trivial ACL, in which case chmod would've worked fine over NFS.
>
> In any case, I don't see anything in that thread that seems relevant to
> NFSv4 id mapping, which unless I misunderstand is the problem the OP is
> trying to resolve.
>
> On that subject, NFSv4 id mapping seems to be working fine for me between
> an
> omnios client and server. On the server, the file system is mounted as:
>
> /export/user/henson on export/user/henson
> read/write/nosetuid/nodevices/nonbmand/exec/xattr/atime/dev=2c5025c
>
> And exported as:
>
> /export/user/henson     -       nfs     nosuid,sec=krb5i,sec=krb5p
>
> with the domain set:
>
> $ sharectl get -p nfsmapid_domain nfs
> nfsmapid_domain=csupomona.edu
>
> if I create a file on the server, it has the correct ownership:
>
> $ touch test_server
> $ ls -l test_server
> -rw-r--r--+ 1 henson csupomona 0 Dec  4 12:50 test_server
>
> on the client, the NFS export is mounted as:
>
> /mnt on files-www.csupomona.edu:/export/user/henson
> remote/read/write/setuid/devices/sec=krb5p/xattr/dev=85c0008 on Thu Dec  4
> 12:50:01 2014
>
> the client has the same domain:
>
> $ sharectl get -p nfsmapid_domain nfs
> nfsmapid_domain=csupomona.edu
>
> The file created on the server shows up with the correct ownership:
>
> $ ls -l test_server
> -rw-r--r--+ 1 henson csupomona 0 Dec  4 12:50 test_server
>
> A file created on the client has the correct ownership:
>
> $ touch test_client
> $ ls -l test_client
> -rw-r--r--+ 1 henson csupomona 0 Dec  4 12:52 test_client
>
> And viewed back on the server, still correct:
>
> $ ls -l test_client
> -rw-r--r--+ 1 henson csupomona 0 Dec  4 12:52 test_client
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141205/d2a79d69/attachment-0001.html>

From ikaufman at eng.ucsd.edu  Fri Dec  5 16:53:13 2014
From: ikaufman at eng.ucsd.edu (Ian Kaufman)
Date: Fri, 5 Dec 2014 08:53:13 -0800
Subject: [OmniOS-discuss] NFSv4 id mapping only working on client but
	not server?
In-Reply-To: <CAOEv1kZo394CCe7tTRaTNd_tPo7evc2j+72FVGw-QrRS4o7LQg@mail.gmail.com>
References: <CAOEv1kYdT=Cj-K7KarcEFizXKe8g8aehKm5uaHORqfqzAxUFoA@mail.gmail.com>
	<20141204193711.03524183@sleipner.datanom.net>
	<CAOEv1kaoxj=AREpfuRo8saHFhW18i0CmgSd9dfwLZEtCUv8pmw@mail.gmail.com>
	<20141204201107.43ad5928@sleipner.datanom.net>
	<4ced01d01005$44363af0$cca2b0d0$@acm.org>
	<CAOEv1kZo394CCe7tTRaTNd_tPo7evc2j+72FVGw-QrRS4o7LQg@mail.gmail.com>
Message-ID: <CAPJtH1gJEqs3oRGEqa5t4PD1vrBKkqGtCv68LE2UZTbnnFsSog@mail.gmail.com>

As far as I recall, AUTH_UNIX (aka AUTH_SYS) uses RPC, and RPC has not
been augmented to support NFSv4 yet.

>From an old thread in the OpenSolaris boards:
-------------------------------------
Note also, that although NFSv4 uses strings for uid/gid/domain, the
underlying RPC layer uses the same authentication credentials as in
previous NFS versions and other RPC programs.

Since it's the RPC authentication that is used to control access, etc,
don't expect too much from the NFSv4 name/id mapping. It's useful for ls
-l listings, etc, but not for authentication.
-------------------------------------


On Fri, Dec 5, 2014 at 5:32 AM, John Klimek <jklimek at gmail.com> wrote:
> Paul, thanks!  You understand exactly what I'm asking and what my problem
> is.
>
> Something seems to be going wrong (or incorrectly configured) with the NFS
> id mapping services on my Linux client or OmniOS server.
>
> I've setup the nfsmapid_domain and I believe it's working because before
> configuring it, everything was showing as nobody/nobody, but afterwards it
> shows the correct names.  (on the client)  I can also see in the client log
> and it's correctly mapping user at domain.com in the Linux rpc.idmapd domain.
>
> The problem is that when I create a file on the client, the server (OmniOS)
> is showing the UID of the user that created the file instead of mapping it
> using the name to the local UID.
>
> It looks like I can avoid the problem if I synchronize UIDs and GIDs across
> all servers (it's a small home network so it's not a big problem and I'm not
> using LDAP or NIS), but NFSv4 was supposed to avoid this problem by using
> name mapping so you didn't have to synchronize UIDs/GIDs.
>
>
>
> On Thu, Dec 4, 2014 at 3:59 PM, Paul B. Henson <henson at acm.org> wrote:
>>
>> > From: Michael Rasmussen
>> > Sent: Thursday, December 04, 2014 11:11 AM
>> >
>> > Yes, because you want to avoid Omnios presents ACL which is incompatible
>> > with Linux ACL.
>>
>> I don't believe the ACL has anything to do with NFSv4 id mapping? And a
>> ZFS
>> ACL presented over NFSv4 is perfectly compatible with Linux. It's not a
>> Linux POSIX ACL, and cannot be manipulated with getfacl/setfacl, you need
>> to
>> use the nfs4 ACL tools, but it works fine.
>>
>> > http://forum.proxmox.com/threads/15793-CT-creation-on-NFS-
>> > Share?p=81530#post81530
>>
>> In that thread, the user fails to chmod via NFS:
>>
>> chmod: changing permissions of `/mnt/pve/proxCT/private/108.tmp':
>> Operation
>> not permitted
>>
>> The root cause of which was a setting of restricted for aclmode:
>>
>> vdev1/proxCT aclmode             restricted
>> local
>>
>> Per the man page "An aclmode property of restricted will cause the
>> chmod(2)
>> operation to return an error when used on any file or directory which has
>> a
>> non-trivial ACL whose entries can not be represented by a mode."
>>
>> The user could have set the inherited ACL on the initial filesystem to a
>> trivial ACL, in which case chmod would've worked fine over NFS.
>>
>> In any case, I don't see anything in that thread that seems relevant to
>> NFSv4 id mapping, which unless I misunderstand is the problem the OP is
>> trying to resolve.
>>
>> On that subject, NFSv4 id mapping seems to be working fine for me between
>> an
>> omnios client and server. On the server, the file system is mounted as:
>>
>> /export/user/henson on export/user/henson
>> read/write/nosetuid/nodevices/nonbmand/exec/xattr/atime/dev=2c5025c
>>
>> And exported as:
>>
>> /export/user/henson     -       nfs     nosuid,sec=krb5i,sec=krb5p
>>
>> with the domain set:
>>
>> $ sharectl get -p nfsmapid_domain nfs
>> nfsmapid_domain=csupomona.edu
>>
>> if I create a file on the server, it has the correct ownership:
>>
>> $ touch test_server
>> $ ls -l test_server
>> -rw-r--r--+ 1 henson csupomona 0 Dec  4 12:50 test_server
>>
>> on the client, the NFS export is mounted as:
>>
>> /mnt on files-www.csupomona.edu:/export/user/henson
>> remote/read/write/setuid/devices/sec=krb5p/xattr/dev=85c0008 on Thu Dec  4
>> 12:50:01 2014
>>
>> the client has the same domain:
>>
>> $ sharectl get -p nfsmapid_domain nfs
>> nfsmapid_domain=csupomona.edu
>>
>> The file created on the server shows up with the correct ownership:
>>
>> $ ls -l test_server
>> -rw-r--r--+ 1 henson csupomona 0 Dec  4 12:50 test_server
>>
>> A file created on the client has the correct ownership:
>>
>> $ touch test_client
>> $ ls -l test_client
>> -rw-r--r--+ 1 henson csupomona 0 Dec  4 12:52 test_client
>>
>> And viewed back on the server, still correct:
>>
>> $ ls -l test_client
>> -rw-r--r--+ 1 henson csupomona 0 Dec  4 12:52 test_client
>>
>>
>
>
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>



-- 
Ian Kaufman
Research Systems Administrator
UC San Diego, Jacobs School of Engineering ikaufman AT ucsd DOT edu

From henson at acm.org  Fri Dec  5 19:31:26 2014
From: henson at acm.org (Paul B. Henson)
Date: Fri, 5 Dec 2014 11:31:26 -0800
Subject: [OmniOS-discuss] NFSv4 id mapping only working on client but
	not server?
In-Reply-To: <CAPJtH1gJEqs3oRGEqa5t4PD1vrBKkqGtCv68LE2UZTbnnFsSog@mail.gmail.com>
References: <CAOEv1kYdT=Cj-K7KarcEFizXKe8g8aehKm5uaHORqfqzAxUFoA@mail.gmail.com>
	<20141204193711.03524183@sleipner.datanom.net>
	<CAOEv1kaoxj=AREpfuRo8saHFhW18i0CmgSd9dfwLZEtCUv8pmw@mail.gmail.com>
	<20141204201107.43ad5928@sleipner.datanom.net>
	<4ced01d01005$44363af0$cca2b0d0$@acm.org>
	<CAOEv1kZo394CCe7tTRaTNd_tPo7evc2j+72FVGw-QrRS4o7LQg@mail.gmail.com>
	<CAPJtH1gJEqs3oRGEqa5t4PD1vrBKkqGtCv68LE2UZTbnnFsSog@mail.gmail.com>
Message-ID: <57b501d010c2$11d3d6e0$357b84a0$@acm.org>

> From: Ian Kaufman
> Sent: Friday, December 05, 2014 8:53 AM
>
> As far as I recall, AUTH_UNIX (aka AUTH_SYS) uses RPC, and RPC has not
> been augmented to support NFSv4 yet.

Ah, right; I forgot about that annoyance :(. My systems are using kerberized NFS, which doesn't rely on uid/gid being passed over the wire. I remember now when we needed to integrate a system that wouldn't do kerberos, and we had the exact same uid mismatch issues. I don't know why they didn't introduce a new mechanism such as AUTH_SYSNAME with NFSv4 that would be identical to AUTH_SYS but use string identifiers rather than hardcoded id's over the wire, that's quite a deficiency.

For a small scale, it's a bit of a pain/overkill to have to set up a kerberos infrastructure. I've never tried secure RPC using Diffie Hellman, that might work if you're purely Solaris/illumos, but I don't believe linux supports that. And I'm not even sure whether or not that uses string identifiers or numeric IDs on the wire?


From rt at steait.net  Fri Dec  5 19:56:11 2014
From: rt at steait.net (Rune Tipsmark)
Date: Fri, 5 Dec 2014 19:56:11 +0000
Subject: [OmniOS-discuss] PCIe dedicated device for ZIL
In-Reply-To: <CAO-7L5Fzico5MsnYET9ODndQMerJJNfq=jmfzWTrxorrZ_DzEA@mail.gmail.com>
References: <CAO-7L5Fzico5MsnYET9ODndQMerJJNfq=jmfzWTrxorrZ_DzEA@mail.gmail.com>
Message-ID: <1417809372716.19446@steait.net>

its a good idea, I use Fusion IO SLC drives but I still see a limit of about 750 MB/sec which is too little, also I need to run multiple streams to achieve this, if I only use a single data stream I only get around 350 MB/sec. I makes no difference if I use 1 or 4 SLC drives, speed remains pretty much the same +/-  5%.


After testing some mSata drives in one of Intels NUCs it became interesting to investigate if it would be possible to get a motherboard with say 10 or 20 mSata slots, I even asked SuperMicros engineers and they said no coz its not an enterprise product.


However there are some PCIe cards available where you can attach mSata drives and even raid them and while I havent tried this yet, I think it could deliver some nice cheap SLOG perforformance


check http://www.addonics.com/products/ad4mspx2.php


Maybe there are better products out there, this was just what I came across initially.


Maybe even completely skip a dedicated log device and set the pool to sync=always and base it on for example 8 of the above PCIe cards each with 4 1TB drives and you could have 32TB raw SSD ultrafast storage in a 3unit and with compression and possibly dedup you could easily store a lot of data on even when mirroring.


This is probably my next project anyway so interested to see what you find out in terms of upping the speed and lowering the latency.


br,

Rune



________________________________
From: OmniOS-discuss <omnios-discuss-bounces at lists.omniti.com> on behalf of Angel Angelov <a.angelov at gmail.com>
Sent: Tuesday, December 2, 2014 3:57 PM
To: omnios-discuss at lists.omniti.com
Subject: [OmniOS-discuss] PCIe dedicated device for ZIL

Hi guys,

does anyone played with some of these PCIe devices on the market as a cheaper ZIL dedicated device instead of this one<http://www.hgst.com/solid-state-storage/enterprise-ssd/sas-ssd/zeusram-sas-ssd> for example that would work with OmniOS out of the box?

Here's one of the products I have found in the Internet googling around but it's way too big for the purpose:
http://www.kingspec.com/en/product_xx187.html

I am trying to find a good price/performance balance when full SSD storage based pool is in use.

Thanks in advance.



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141205/a4158906/attachment.html>

From fwp at deepthought.com  Sun Dec  7 22:43:53 2014
From: fwp at deepthought.com (Frank Pittel)
Date: Sun, 7 Dec 2014 16:43:53 -0600
Subject: [OmniOS-discuss] NFSv4 id mapping only working on client but
 not server?
In-Reply-To: <CAOEv1kaoxj=AREpfuRo8saHFhW18i0CmgSd9dfwLZEtCUv8pmw@mail.gmail.com>
References: <CAOEv1kYdT=Cj-K7KarcEFizXKe8g8aehKm5uaHORqfqzAxUFoA@mail.gmail.com>
	<20141204193711.03524183@sleipner.datanom.net>
	<CAOEv1kaoxj=AREpfuRo8saHFhW18i0CmgSd9dfwLZEtCUv8pmw@mail.gmail.com>
Message-ID: <20141207224353.GA13916@warlock.deepthought.com>

I've tried doing this with NFS4 a couple of years ago and what I found was that
while the UID mapping worked for things like "ls" anytime you tried to actually
do anything with the file RPC calls were made and since that's outside of NFS they
failed.

It was a couple of years ago now since I went through it but the upshot is that
at least then uid mapping didn't work in any meaningful way!

The Other Frank



On Thu, Dec 04, 2014 at 01:55:11PM -0500, John Klimek wrote:
> Thanks, I will give this a try.
> 
> However, when I create a file if I check the acl on Omni, it does have
> extended permissions and looks correct.  It's only the uid and gid that I'm
> trying to get mapped correctly.
> 
> Do you still think that noacl will solve that problem?
> 
> (I'm migrating some virtual machines so I can't try out the setting you
> mentioned until a few more minutes...)
> 
> On Thu, Dec 4, 2014 at 1:37 PM, Michael Rasmussen <mir at miras.org> wrote:
> 
> > On Thu, 4 Dec 2014 13:20:04 -0500
> > John Klimek <jklimek at gmail.com> wrote:
> >
> > >
> > > What can I have setup wrong?
> > >
> > > I can't find any debugging or logging options for nfsmapid...
> > >
> > > Also, does the uidmap and gidmap share.nfs options only work for NFSv3?
> > Solaris and derivatives implementation of NFS ACL is not compliant to the
> > Linux NFS ACL.
> > More directly it is the ACL in Linux which is not POSIX conformant so to
> > avoid problems you
> > should add the mount option noacl in your Linux fstab file. Noacl will
> > instruct Omnios NFS
> > to revert to plain old uid/gid.
> >
> > --
> > Hilsen/Regards
> > Michael Rasmussen
> >
> > Get my public GnuPG keys:
> > michael <at> rasmussen <dot> cc
> > http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E
> > mir <at> datanom <dot> net
> > http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C
> > mir <at> miras <dot> org
> > http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917
> > --------------------------------------------------------------
> > /usr/games/fortune -es says:
> > Don't just echo the code with comments - make every comment count.
> >             - The Elements of Programming Style (Kernighan & Plaugher)
> >
> > _______________________________________________
> > OmniOS-discuss mailing list
> > OmniOS-discuss at lists.omniti.com
> > http://lists.omniti.com/mailman/listinfo/omnios-discuss
> >
> >

> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss


From jimklimov at cos.ru  Mon Dec  8 08:50:33 2014
From: jimklimov at cos.ru (Jim Klimov)
Date: Mon, 08 Dec 2014 09:50:33 +0100
Subject: [OmniOS-discuss] Timezones
Message-ID: <6E7D35D0-98DC-4B63-8683-A861AC1A6D05@cos.ru>

Hello all,

The /usr/share/lib/zoneinfo/src/README mentions that the Solaris zoneinfo is based on Olson tzdata 2010k.

Is that true or was the data since then updated and just not the readme?

Is it safe to remove this directory and replace by either vanilla rebuild of a recent tzdata (i.e. 2014j) or some transformation thereof to make it more solaris-compatible? 

Or should it be overlaid by whatever the new version provides?

The set of filenames is noticeably different between the two variants (installed in OS and manually compiled with zic)...

Also, do the distributions like OmniOS or OI, or the illumos-gate itself, track the IANA (exOlson) tzdata to provide some recent version in the os/net builds?

Thanks,
Jim Klimov
--
Typos courtesy of K-9 Mail on my Samsung Android

From jklimek at gmail.com  Mon Dec  8 13:25:54 2014
From: jklimek at gmail.com (John Klimek)
Date: Mon, 8 Dec 2014 08:25:54 -0500
Subject: [OmniOS-discuss] NFSv4 id mapping only working on client but
	not server?
In-Reply-To: <20141207224353.GA13916@warlock.deepthought.com>
References: <CAOEv1kYdT=Cj-K7KarcEFizXKe8g8aehKm5uaHORqfqzAxUFoA@mail.gmail.com>
	<20141204193711.03524183@sleipner.datanom.net>
	<CAOEv1kaoxj=AREpfuRo8saHFhW18i0CmgSd9dfwLZEtCUv8pmw@mail.gmail.com>
	<20141207224353.GA13916@warlock.deepthought.com>
Message-ID: <CAOEv1kYze9=As8jtTYAG88O+0hn0ECvLGrY9_HiLZdGu1-23uA@mail.gmail.com>

Thanks everybody.

Are you guys saying that Solaris and Linux are incompatible for sharing
NFSv4 ACLs (because of RPC differences, etc) ?

On Sun, Dec 7, 2014 at 5:43 PM, Frank Pittel <fwp at deepthought.com> wrote:

> I've tried doing this with NFS4 a couple of years ago and what I found was
> that
> while the UID mapping worked for things like "ls" anytime you tried to
> actually
> do anything with the file RPC calls were made and since that's outside of
> NFS they
> failed.
>
> It was a couple of years ago now since I went through it but the upshot is
> that
> at least then uid mapping didn't work in any meaningful way!
>
> The Other Frank
>
>
>
> On Thu, Dec 04, 2014 at 01:55:11PM -0500, John Klimek wrote:
> > Thanks, I will give this a try.
> >
> > However, when I create a file if I check the acl on Omni, it does have
> > extended permissions and looks correct.  It's only the uid and gid that
> I'm
> > trying to get mapped correctly.
> >
> > Do you still think that noacl will solve that problem?
> >
> > (I'm migrating some virtual machines so I can't try out the setting you
> > mentioned until a few more minutes...)
> >
> > On Thu, Dec 4, 2014 at 1:37 PM, Michael Rasmussen <mir at miras.org> wrote:
> >
> > > On Thu, 4 Dec 2014 13:20:04 -0500
> > > John Klimek <jklimek at gmail.com> wrote:
> > >
> > > >
> > > > What can I have setup wrong?
> > > >
> > > > I can't find any debugging or logging options for nfsmapid...
> > > >
> > > > Also, does the uidmap and gidmap share.nfs options only work for
> NFSv3?
> > > Solaris and derivatives implementation of NFS ACL is not compliant to
> the
> > > Linux NFS ACL.
> > > More directly it is the ACL in Linux which is not POSIX conformant so
> to
> > > avoid problems you
> > > should add the mount option noacl in your Linux fstab file. Noacl will
> > > instruct Omnios NFS
> > > to revert to plain old uid/gid.
> > >
> > > --
> > > Hilsen/Regards
> > > Michael Rasmussen
> > >
> > > Get my public GnuPG keys:
> > > michael <at> rasmussen <dot> cc
> > > http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E
> > > mir <at> datanom <dot> net
> > > http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C
> > > mir <at> miras <dot> org
> > > http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917
> > > --------------------------------------------------------------
> > > /usr/games/fortune -es says:
> > > Don't just echo the code with comments - make every comment count.
> > >             - The Elements of Programming Style (Kernighan & Plaugher)
> > >
> > > _______________________________________________
> > > OmniOS-discuss mailing list
> > > OmniOS-discuss at lists.omniti.com
> > > http://lists.omniti.com/mailman/listinfo/omnios-discuss
> > >
> > >
>
> > _______________________________________________
> > OmniOS-discuss mailing list
> > OmniOS-discuss at lists.omniti.com
> > http://lists.omniti.com/mailman/listinfo/omnios-discuss
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141208/b4c3635e/attachment-0001.html>

From jim at cos.ru  Mon Dec  8 12:57:34 2014
From: jim at cos.ru (Jim Klimov)
Date: Mon, 08 Dec 2014 13:57:34 +0100
Subject: [OmniOS-discuss] FYI: Split-root installs are also applicable to
	OmniOS
Message-ID: <fc77547e7ff1.5485ae4e@cos.ru>


 Hello all,

 Now that I'm playing with OmniOS as well, I am verifying my old Solaris/OpenSolaris/OI recipes with it. In particular, the "split-root installation" procedure worked flawlessly via copy-paste from my old article (now updated to smooth some non-critical rough edges):

http://wiki.openindiana.org/oi/Advanced+-+Split-root+installation#Advanced-Split-rootinstallation-Upgrades

 Same caveats as before are applicable, i.e. that "pkg upgrade" which invokes "beadm clone" un-does much of the effort regarding dataset fine-tuning (compression/quota/... on components of the rootfs), so the script provided with the article is recommended as the way to clone the global zone BE.




 My other procedures (ZFS pools as SMF services, and Zones as SMF services) installed without apparent errors, but currently this box is a single-disk rpool-only machine without zones, so there was no practical testing of that part yet.




 On a side note, adding pkgsrc was a breeze, except that it does refer to some libraries like "/lib/64/libz.so.1" or "/lib/64/libxml2.so.2" - but creating corresponding symlinks to objects in /usr/lib/{amd64,} helped the installation. According to http://www.kdump.cn/forums/viewtopic.php?id=1325 just touching these FS objects would've helped ("These are pkgsrc deps, not linker deps") and anyhow ldd'ing the binaries refers to /usr/lib/... paths.




 HTH,
 Jim Klimov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141208/309998c3/attachment.html>

From danmcd at omniti.com  Mon Dec  8 14:22:55 2014
From: danmcd at omniti.com (Dan McDonald)
Date: Mon, 8 Dec 2014 09:22:55 -0500
Subject: [OmniOS-discuss] Timezones
In-Reply-To: <6E7D35D0-98DC-4B63-8683-A861AC1A6D05@cos.ru>
References: <6E7D35D0-98DC-4B63-8683-A861AC1A6D05@cos.ru>
Message-ID: <94A534D5-D8BD-4426-B0AF-90E6D48F6802@omniti.com>

Updating the zoneinfo is highly annoying, and the README is wrong - if you look in the Illumos source (I upstreamed this too), it's up to one of the 2014 updates.

Thanks for the README catch.  When 014 gets closer to release, I'll be updating the zoneinfo again.

Dan

Sent from my iPhone (typos, autocorrect, and all)

> On Dec 8, 2014, at 3:50 AM, Jim Klimov <jimklimov at cos.ru> wrote:
> 
> Hello all,
> 
> The /usr/share/lib/zoneinfo/src/README mentions that the Solaris zoneinfo is based on Olson tzdata 2010k.
> 
> Is that true or was the data since then updated and just not the readme?
> 
> Is it safe to remove this directory and replace by either vanilla rebuild of a recent tzdata (i.e. 2014j) or some transformation thereof to make it more solaris-compatible? 
> 
> Or should it be overlaid by whatever the new version provides?
> 
> The set of filenames is noticeably different between the two variants (installed in OS and manually compiled with zic)...
> 
> Also, do the distributions like OmniOS or OI, or the illumos-gate itself, track the IANA (exOlson) tzdata to provide some recent version in the os/net builds?
> 
> Thanks,
> Jim Klimov
> --
> Typos courtesy of K-9 Mail on my Samsung Android
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss

From heinrich.vanriel at outlook.com  Tue Dec  2 18:35:06 2014
From: heinrich.vanriel at outlook.com (Machine Man)
Date: Tue, 2 Dec 2014 13:35:06 -0500
Subject: [OmniOS-discuss] Release 12 KVM poor network performance
Message-ID: <BAY405-EAS14228BC389BDD6B60D7519837A0@phx.gbl>

I sent this as one of my aliases and it was not allowed to be delivered since I am not register with it. I sent this message days ago and then resent it with the alias that is registered.
Apologies.
________________________________
From: Dan McDonald<mailto:danmcd at omniti.com>
Sent: ?12/?2/?2014 10:45
To: Machine Man<mailto:heinrich.vanriel at outlook.com>
Cc: omnios-discuss at lists.omniti.com<mailto:omnios-discuss at lists.omniti.com>
Subject: Re: [OmniOS-discuss] Release 12 KVM poor network performance


> On Nov 29, 2014, at 3:42 PM, Machine Man <heinrich.vanriel at outlook.com> wrote:
>
> Experience slow network performance after upgrading.
> Copying from another windows system (physical machine) it drops form the 85MB/s on R10 to between 9 and 14MB/s. Two systems experience the exact same symptom. Both make use of igb driver.
> Copy tests are done between windows 8.1 and Windows 2012R2 VM.
> Changing to virtio for nic has no impact other than causing kvm to crash after copying ~400MB.
>
> Any ideas?

We are behind the illumos-kvm-cmd distribution because it requires changes in illumos that Joyent hasn't pushed upstream just yet (the "vnd" project).  I'm hoping to address this for '014, but I have other 014-related things to do first.

Sorry,
Dan


_______________________________________________
OmniOS-discuss mailing list
OmniOS-discuss at lists.omniti.com
http://lists.omniti.com/mailman/listinfo/omnios-discuss
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141202/64709ab6/attachment-0001.html>

From lkateley at kateley.com  Thu Dec  4 15:33:24 2014
From: lkateley at kateley.com (Linda Kateley)
Date: Thu, 04 Dec 2014 09:33:24 -0600
Subject: [OmniOS-discuss] anyone doing DirectPath I/O?
In-Reply-To: <alpine.UW2.2.11.1412032251490.27489@server01.int.multitalents.net>
References: <CAGoaUhO9kRCopXH5+wojk4U-ict8=JpbwS9U9RtBxcO86F6_zQ@mail.gmail.com>	<CALeZrrTCE=N5arT7_LFCLex5NJXyxMugNwWrgO_mbmOaHXeEcQ@mail.gmail.com>
	<alpine.UW2.2.11.1412032251490.27489@server01.int.multitalents.net>
Message-ID: <54807EC4.6020201@kateley.com>

Just curious, how fast is the nfs in this config?

linda

On 12/4/14, 1:11 AM, Tim Rice wrote:
> On Wed, 3 Dec 2014, Schweiss, Chip wrote:
>
> | I've been using DirectPath with ESXi 5.0 with an LSI HBA for almost 2 years
> | to an OpenIndiana server.  It's been very stable:
>
> My all in one box (Supermicro MBD-X9SCM-F-0) was on ESXi 5.1 for
> about a year with a LSI SAS3801E and a LSI SAS9211-4i. I've since
> updated to ESXi 5.5 update2. The OmniOS r151006 VM provides NFS
> storage to the ESXi host for (currently) 15 running VMS as well
> as well as NFS home directories for various other computers around
> the office. Other than drives failing, it's been working reasonably
> well for what it is.
>
> |
> | root at zfs01:~# uptime
> |  21:01pm  up 649 days  6:58,  2 users,  load average: 0.46, 0.30, 0.25
> |
> |
> | On Wed, Dec 3, 2014 at 6:10 PM, Joseph Boren <jboren at drakecooper.com> wrote:
> |
> | > Greetings,
> | >
> | > I was wondering if anyone was using DirectPath, specifically for exclusive
> | > use of a drive controller and attached drives to a specific VM.    I have a
> | > use case that would seem to be a good fit for this, so I played around with
> | > a couple of RAID controllers I had, and was able to get one (3ware 9650se)
> | > configured for directpath, but none of the drives attached would show to
> | > OmniOS, regardless of how I configured them in the raid bois (JBOD,
> | > individual disks, etc).  I know that controller is poorly supported and I
> | > was curious if anyone was using DirectPath  this way in production and what
> | > kind of drive controller/HBA/whatever was working.  Also any "for the love
> | > of god don't do it this way" scenarios?  I seem to be really adept at
> | > finding and trying those out first....
> | >
> | > Thanks a ton and best regards,
> | >
> | > -jb-
> | > *Joseph Boren*
> | >
> | > IT Specialist
> | > *DRAKE COOPER*
> | > + c: (208) 891-2128 + o: (208) 342-0925
> | > + 416 S. 8th St., Boise, ID 83702
> | > + w: drakecooper.com + f: /drakecooper <http://facebook.com/drakecooper> +
> | >  t: @drakecooper <http://twitter.com/drakecooper>
> | >
> | >
> | > _______________________________________________
> | > OmniOS-discuss mailing list
> | > OmniOS-discuss at lists.omniti.com
> | > http://lists.omniti.com/mailman/listinfo/omnios-discuss
> | >
> | >
> |
>


From orilali at evogene.com  Thu Dec  4 17:51:46 2014
From: orilali at evogene.com (Koby Schwartz)
Date: Thu, 04 Dec 2014 19:51:46 +0200
Subject: [OmniOS-discuss] Kernel stability.
Message-ID: <54809F32.2080703@evogene.com>

Hi,

My name is Kobi and I work with a company called Evogene.

We have a storage server based on a Supermicro X9DRD-7LN4F (Intel(R) 
Xeon(R) CPU E5-2609, 64GB RAM) equipped with Intel X520-DA2 10Gb 
Ethernet card.

Two weeks ago we encountered a problem which regards Omnios (r15012) and 
OpenIndiana (151_a8) kernel stability.

The system is a clean install of r151012, only configuration is 
networking and zpool (raidz3 1 vdev of 8 disks of data and 3 parity).

The storage has a single NFS share mounted (nfsvers=3) on an 8 grid 
nodes (running CentOS6.6) with the following mount options :

async,tcp,vers=3,noacl,timeo=20,rw,bg,hard,intr,rsize=65536,wsize=65536 0 0.

Under normal conditions (random reads and writes) we get great 
performance and stability but if we increase nfs usage the Kernel memory 
usage increases until the machine eventually halts.

To test it we used on each node iozone ( Version $Revision: 3.394 ) 
which simultaneously write and reads 12 files with a size of 500MB.
(iozone -r 64k -i 0 -i 1 -s 500m -t 12 -F 
/evolon2/iozone/`hostname`.{1..12}).

While the iozone's are running the memory of Kernel (as seen by echo 
'::memstat'| mdb -k) is steadily increasing until the machine is dead.

For clarification, when the machine is dead/halts:
There is no crash files.
There is no new messages in /var/adm/messages.
It doesn't respond to pings (and obviously other network services).
Waiting doesn't help.
It can only be brought to life by power cycling it.

I attached 3 files: memstat, top and arc-kstat which I hope will assist 
to understand why the storage died/halted

  Thank you in advance,

Kobi.

-------------- next part --------------
Thu Dec  4 16:14:02 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                    5667640             22139   34%
ZFS File Data             1171645              4576    7%
Anon                        26071               101    0%
Exec and libs                1181                 4    0%
Page cache                   4609                18    0%
Free (cachelist)             7723                30    0%
Free (freelist)           9887541             38623   59%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:14:06 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                    6380424             24923   38%
ZFS File Data             1171645              4576    7%
Anon                        26096               101    0%
Exec and libs                1175                 4    0%
Page cache                   4610                18    0%
Free (cachelist)             7729                30    0%
Free (freelist)           9174731             35838   55%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:14:09 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                    6524733             25487   39%
ZFS File Data             1184093              4625    7%
Anon                        26096               101    0%
Exec and libs                1181                 4    0%
Page cache                   4612                18    0%
Free (cachelist)             7722                30    0%
Free (freelist)           9017973             35226   54%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:14:12 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                    7290171             28477   43%
ZFS File Data             1184093              4625    7%
Anon                        26122               102    0%
Exec and libs                1181                 4    0%
Page cache                   4612                18    0%
Free (cachelist)             7723                30    0%
Free (freelist)           8252508             32236   49%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:14:16 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                    7456599             29127   44%
ZFS File Data             1190237              4649    7%
Anon                        26122               102    0%
Exec and libs                1181                 4    0%
Page cache                   4613                18    0%
Free (cachelist)             7723                30    0%
Free (freelist)           8079935             31562   48%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:14:19 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                    7480950             29222   45%
ZFS File Data             1513949              5913    9%
Anon                        26178               102    0%
Exec and libs                1180                 4    0%
Page cache                   4615                18    0%
Free (cachelist)             7724                30    0%
Free (freelist)           7731814             30202   46%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:14:25 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                    7481258             29223   45%
ZFS File Data             1513949              5913    9%
Anon                        26174               102    0%
Exec and libs                1181                 4    0%
Page cache                   4616                18    0%
Free (cachelist)             7723                30    0%
Free (freelist)           7731509             30201   46%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:14:28 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                    7482405             29228   45%
ZFS File Data             1577181              6160    9%
Anon                        26200               102    0%
Exec and libs                1181                 4    0%
Page cache                   4617                18    0%
Free (cachelist)             7723                30    0%
Free (freelist)           7667103             29949   46%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:14:31 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                    7482519             29228   45%
ZFS File Data             1577181              6160    9%
Anon                        26226               102    0%
Exec and libs                1181                 4    0%
Page cache                   4618                18    0%
Free (cachelist)             7723                30    0%
Free (freelist)           7666962             29949   46%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:14:35 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                    7482590             29228   45%
ZFS File Data             1577181              6160    9%
Anon                        26226               102    0%
Exec and libs                1180                 4    0%
Page cache                   4619                18    0%
Free (cachelist)             7723                30    0%
Free (freelist)           7666891             29948   46%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:14:38 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                    7482751             29229   45%
ZFS File Data             1577181              6160    9%
Anon                        26252               102    0%
Exec and libs                1181                 4    0%
Page cache                   4619                18    0%
Free (cachelist)             7723                30    0%
Free (freelist)           7666703             29948   46%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:14:41 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                    7482755             29229   45%
ZFS File Data             1577181              6160    9%
Anon                        26252               102    0%
Exec and libs                1181                 4    0%
Page cache                   4622                18    0%
Free (cachelist)             7723                30    0%
Free (freelist)           7666696             29948   46%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:14:44 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                    7483021             29230   45%
ZFS File Data             1577181              6160    9%
Anon                        26278               102    0%
Exec and libs                1181                 4    0%
Page cache                   4623                18    0%
Free (cachelist)             7723                30    0%
Free (freelist)           7666403             29946   46%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:14:48 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                    7483581             29232   45%
ZFS File Data             1577181              6160    9%
Anon                        26303               102    0%
Exec and libs                1181                 4    0%
Page cache                   4622                18    0%
Free (cachelist)             7724                30    0%
Free (freelist)           7665818             29944   46%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:14:51 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                    7645816             29866   46%
ZFS File Data             1577181              6160    9%
Anon                        26328               102    0%
Exec and libs                1181                 4    0%
Page cache                   4624                18    0%
Free (cachelist)             7723                30    0%
Free (freelist)           7503557             29310   45%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:14:55 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                    7645904             29866   46%
ZFS File Data             1577181              6160    9%
Anon                        26328               102    0%
Exec and libs                1181                 4    0%
Page cache                   4625                18    0%
Free (cachelist)             7723                30    0%
Free (freelist)           7503468             29310   45%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:14:58 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                    7753278             30286   46%
ZFS File Data             1577181              6160    9%
Anon                        26355               102    0%
Exec and libs                1181                 4    0%
Page cache                   4626                18    0%
Free (cachelist)             7723                30    0%
Free (freelist)           7396066             28890   44%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:15:01 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                    8085909             31585   48%
ZFS File Data             1577181              6160    9%
Anon                        26380               103    0%
Exec and libs                1179                 4    0%
Page cache                   4627                18    0%
Free (cachelist)             7725                30    0%
Free (freelist)           7063409             27591   42%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:15:04 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                    8085951             31585   48%
ZFS File Data             1577181              6160    9%
Anon                        26380               103    0%
Exec and libs                1181                 4    0%
Page cache                   4628                18    0%
Free (cachelist)             7723                30    0%
Free (freelist)           7063366             27591   42%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:15:08 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                    8209025             32066   49%
ZFS File Data             1577181              6160    9%
Anon                        26407               103    0%
Exec and libs                1181                 4    0%
Page cache                   4630                18    0%
Free (cachelist)             7723                30    0%
Free (freelist)           6940263             27110   41%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:15:11 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                    8763032             34230   52%
ZFS File Data             1577181              6160    9%
Anon                        26432               103    0%
Exec and libs                1181                 4    0%
Page cache                   4630                18    0%
Free (cachelist)             7723                30    0%
Free (freelist)           6386231             24946   38%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:15:14 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                    9388716             36674   56%
ZFS File Data             1577181              6160    9%
Anon                        26432               103    0%
Exec and libs                1181                 4    0%
Page cache                   4631                18    0%
Free (cachelist)             7724                30    0%
Free (freelist)           5760545             22502   34%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:15:17 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                    9586189             37446   57%
ZFS File Data             1577181              6160    9%
Anon                        26459               103    0%
Exec and libs                1181                 4    0%
Page cache                   4632                18    0%
Free (cachelist)             7723                30    0%
Free (freelist)           5563045             21730   33%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:15:20 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                    9967338             38934   59%
ZFS File Data             1577181              6160    9%
Anon                        26459               103    0%
Exec and libs                1181                 4    0%
Page cache                   4633                18    0%
Free (cachelist)             7723                30    0%
Free (freelist)           5181895             20241   31%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:15:24 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                   10087818             39405   60%
ZFS File Data             1577181              6160    9%
Anon                        26484               103    0%
Exec and libs                1181                 4    0%
Page cache                   4635                18    0%
Free (cachelist)             7723                30    0%
Free (freelist)           5061388             19771   30%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:15:27 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                   10620046             41484   63%
ZFS File Data             1577181              6160    9%
Anon                        26510               103    0%
Exec and libs                1157                 4    0%
Page cache                   4633                18    0%
Free (cachelist)             7750                30    0%
Free (freelist)           4529133             17691   27%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:15:30 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                   10728161             41906   64%
ZFS File Data             1577181              6160    9%
Anon                        26510               103    0%
Exec and libs                1181                 4    0%
Page cache                   4637                18    0%
Free (cachelist)             7723                30    0%
Free (freelist)           4421017             17269   26%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:15:34 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                   11230924             43870   67%
ZFS File Data             1577181              6160    9%
Anon                        26536               103    0%
Exec and libs                1181                 4    0%
Page cache                   4637                18    0%
Free (cachelist)             7723                30    0%
Free (freelist)           3918228             15305   23%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:15:38 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                   11277074             44051   67%
ZFS File Data             1577981              6163    9%
Anon                        26562               103    0%
Exec and libs                1181                 4    0%
Page cache                   4638                18    0%
Free (cachelist)             7723                30    0%
Free (freelist)           3871251             15122   23%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:15:41 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                   11779320             46012   70%
ZFS File Data             1577981              6163    9%
Anon                        26562               103    0%
Exec and libs                1180                 4    0%
Page cache                   4639                18    0%
Free (cachelist)             7724                30    0%
Free (freelist)           3369004             13160   20%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:15:44 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                   11968305             46751   71%
ZFS File Data             1577981              6163    9%
Anon                        26588               103    0%
Exec and libs                1181                 4    0%
Page cache                   4641                18    0%
Free (cachelist)             7723                30    0%
Free (freelist)           3179991             12421   19%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:15:47 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                   12424008             48531   74%
ZFS File Data             1577981              6163    9%
Anon                        26614               103    0%
Exec and libs                1181                 4    0%
Page cache                   4642                18    0%
Free (cachelist)             7723                30    0%
Free (freelist)           2724261             10641   16%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:15:51 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                   12708257             49641   76%
ZFS File Data             1577981              6163    9%
Anon                        26614               103    0%
Exec and libs                1181                 4    0%
Page cache                   4643                18    0%
Free (cachelist)             7723                30    0%
Free (freelist)           2440011              9531   15%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:15:54 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                   12960558             50627   77%
ZFS File Data             1577981              6163    9%
Anon                        26640               104    0%
Exec and libs                1181                 4    0%
Page cache                   4644                18    0%
Free (cachelist)             7723                30    0%
Free (freelist)           2187683              8545   13%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:15:58 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                   13441434             52505   80%
ZFS File Data             1577981              6163    9%
Anon                        26664               104    0%
Exec and libs                1181                 4    0%
Page cache                   4645                18    0%
Free (cachelist)             7723                30    0%
Free (freelist)           1706782              6667   10%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:16:01 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                   13724075             53609   82%
ZFS File Data             1577981              6163    9%
Anon                        26691               104    0%
Exec and libs                1181                 4    0%
Page cache                   4646                18    0%
Free (cachelist)             7723                30    0%
Free (freelist)           1424113              5562    8%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:16:05 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                   14075430             54982   84%
ZFS File Data             1577981              6163    9%
Anon                        26691               104    0%
Exec and libs                1181                 4    0%
Page cache                   4647                18    0%
Free (cachelist)             7723                30    0%
Free (freelist)           1072757              4190    6%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:16:08 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                   14302316             55868   85%
ZFS File Data             1577981              6163    9%
Anon                        26723               104    0%
Exec and libs                1181                 4    0%
Page cache                   4649                18    0%
Free (cachelist)             7723                30    0%
Free (freelist)            845837              3304    5%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:16:12 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                   14585839             56975   87%
ZFS File Data             1541308              6020    9%
Anon                        26743               104    0%
Exec and libs                1181                 4    0%
Page cache                   4649                18    0%
Free (cachelist)             7724                30    0%
Free (freelist)            598966              2339    4%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:16:16 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                   14682403             57353   88%
ZFS File Data             1408152              5500    8%
Anon                        26768               104    0%
Exec and libs                1181                 4    0%
Page cache                   4651                18    0%
Free (cachelist)             7723                30    0%
Free (freelist)            635532              2482    4%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:16:19 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                   15056031             58812   90%
ZFS File Data             1163626              4545    7%
Anon                        26778               104    0%
Exec and libs                1181                 4    0%
Page cache                   4652                18    0%
Free (cachelist)             7723                30    0%
Free (freelist)            506419              1978    3%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:16:23 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                   15285492             59708   91%
ZFS File Data             1008029              3937    6%
Anon                        26795               104    0%
Exec and libs                1181                 4    0%
Page cache                   4653                18    0%
Free (cachelist)             7429                29    0%
Free (freelist)            432831              1690    3%

Total                    16766410             65493
Physical                 16766409             65493

Thu Dec  4 16:16:27 IST 2014
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                   15705417             61349   94%
ZFS File Data              857418              3349    5%
Anon                        21468                83    0%
Exec and libs                  73                 0    0%
Page cache                    721                 2    0%
Free (cachelist)            14622                57    0%
Free (freelist)            166691               651    1%

Total                    16766410             65493
Physical                 16766409             65493

-------------- next part --------------
last pid:   996;  load avg:  5.95,  3.04,  1.31;  up 0+00:19:43                                                                                                                 16:16:27
59 processes: 58 sleeping, 1 on cpu
CPU states: 30.6% idle,  1.6% user, 67.8% kernel,  0.0% iowait,  0.0% swap
Kernel: 97556 ctxsw, 2440 trap, 448800 intr, 27430 syscall, 1 fork, 1407 flt
Memory: 64G phys mem, 933M free mem, 4096M total swap, 4096M free swap

   PID USERNAME NLWP PRI NICE  SIZE   RES STATE    TIME    CPU COMMAND
   675 daemon     10  60  -20 2904K 1968K sleep    2:28  4.35% nfsd
   607 root        1  59    0 8308K 6668K sleep    0:00  0.01% perl
   631 root        1  59    0   26M   25M cpu/5    0:00  0.01% top
   711 root        1  59    0 5888K 4180K sleep    0:00  0.01% perl
   705 root        1  59    0 4712K 2628K sleep    0:00  0.00% perl
   633 root        1  59    0 7576K 4828K sleep    0:00  0.00% sshd
   712 root        1  59    0 1644K 1040K sleep    0:00  0.00% tee
   622 root        1  59    0 7576K 4828K sleep    0:00  0.00% sshd
   645 root        1  59    0 7576K 4832K sleep    0:00  0.00% sshd
    10 root       14  59    0 7684K 5664K sleep    0:01  0.00% svc.startd
   580 root       27  59    0   23M   16M sleep    0:01  0.00% fmd
   706 root        1  59    0 1644K 1040K sleep    0:00  0.00% tee
    12 root       19  59    0 9704K 8372K sleep    0:04  0.00% svc.configd
   241 root        7  59    0 4456K 3220K sleep    0:00  0.00% devfsadm
   294 root       34  59    0 7480K 4936K sleep    0:00  0.00% nscd
   367 root        1  59    0 1656K  940K sleep    0:00  0.00% utmpd
   658 daemon      2  60  -20 2888K 1912K sleep    0:00  0.00% lockd
   232 root        5  60  -20 2712K 1600K sleep    0:00  0.00% zonestatd
   322 root        4  59    0   11M 8224K sleep    0:00  0.00% hald
   301 root        5  59    0 5232K 4324K sleep    0:00  0.00% picld
   487 root        3  59    0 5272K 3688K sleep    0:00  0.00% inetd
   331 root        2  59    0 6660K 3368K sleep    0:00  0.00% hald-runner
   225 root       17  59    0 6164K 3312K sleep    0:00  0.00% syseventd
   632 root        1  59    0 5804K 3192K sleep    0:00  0.00% sshd
   644 root        1  59    0 5804K 3192K sleep    0:00  0.00% sshd
   612 root        1  59    0 6508K 3180K sleep    0:00  0.00% hald-addon-netw
   614 root        1  59    0 6508K 3180K sleep    0:00  0.00% hald-addon-cpuf
   621 root        1  59    0 5804K 3096K sleep    0:00  0.00% sshd
    49 netadm      3  59    0 4084K 2836K sleep    0:00  0.00% ipmgmtd
   673 root       13  59    0 4588K 2744K sleep    0:00  0.00% mountd
-------------- next part --------------
Thu Dec  4 16:14:39 IST 2014

l2_size                   = 0                   
other_size                = 16487240            
c_max                     = 67601473536         
hash_elements             = 36191               
duplicate_reads           = 0                   
prefetch_data_hits        = 537                 
l2_read_bytes             = 0                   
demand_metadata_misses    = 12438               
prefetch_data_misses      = 1540                
hash_chain_max            = 2                   
l2_hits                   = 0                   
l2_compress_failures      = 0                   
mfu_ghost_hits            = 0                   
l2_rw_clash               = 0                   
memory_throttle_count     = 0                   
crtime                    = 4.537754192         
arc_meta_used             = 97172664            
hash_elements_max         = 41359               
size                      = 5312841912          
hits                      = 750510              
mfu_hits                  = 507924              
prefetch_metadata_hits    = 10297               
hash_chains               = 54                  
evict_l2_cached           = 0                   
l2_writes_sent            = 0                   
l2_writes_error           = 0                   
deleted                   = 15                  
hdr_size                  = 12559216            
l2_compress_zeros         = 0                   
l2_abort_lowmem           = 0                   
l2_io_error               = 0                   
l2_hdr_size               = 0                   
l2_cksum_bad              = 0                   
l2_evict_reading          = 0                   
arc_meta_limit            = 16900368384         
prefetch_metadata_misses  = 3381                
l2_evict_lock_retry       = 0                   
hash_collisions           = 519                 
l2_writes_hdr_miss        = 0                   
evict_l2_ineligible       = 4096                
l2_write_bytes            = 0                   
data_size                 = 5283795456          
c                         = 67601473536         
l2_asize                  = 0                   
class                     = misc                
mutex_miss                = 0                   
snaptime                  = 1052.750004426      
duplicate_buffers_size    = 0                   
l2_misses                 = 0                   
mru_ghost_hits            = 0                   
demand_metadata_hits      = 28989               
arc_meta_max              = 97686696            
c_min                     = 2146100480          
mru_hits                  = 231752              
demand_data_misses        = 2615                
duplicate_buffers         = 0                   
demand_data_hits          = 710687              
l2_free_on_write          = 0                   
l2_feeds                  = 0                   
misses                    = 19974               
evict_skip                = 1214                
evict_l2_eligible         = 71168               
p                         = 33800736768         
l2_compress_successes     = 0                   
recycle_miss              = 0                   
l2_writes_done            = 0                   
----------

Thu Dec  4 16:14:42 IST 2014

l2_size                   = 0                   
other_size                = 16557200            
c_max                     = 67601473536         
hash_elements             = 35929               
duplicate_reads           = 0                   
prefetch_data_hits        = 537                 
l2_read_bytes             = 0                   
demand_metadata_misses    = 12446               
prefetch_data_misses      = 1540                
hash_chain_max            = 2                   
l2_hits                   = 0                   
l2_compress_failures      = 0                   
mfu_ghost_hits            = 0                   
l2_rw_clash               = 0                   
memory_throttle_count     = 0                   
crtime                    = 4.537754192         
arc_meta_used             = 97848832            
hash_elements_max         = 41359               
size                      = 5242345984          
hits                      = 765628              
mfu_hits                  = 519125              
prefetch_metadata_hits    = 10297               
hash_chains               = 53                  
evict_l2_cached           = 0                   
l2_writes_sent            = 0                   
l2_writes_error           = 0                   
deleted                   = 15                  
hdr_size                  = 12559216            
l2_compress_zeros         = 0                   
l2_abort_lowmem           = 0                   
l2_io_error               = 0                   
l2_hdr_size               = 0                   
l2_cksum_bad              = 0                   
l2_evict_reading          = 0                   
arc_meta_limit            = 16900368384         
prefetch_metadata_misses  = 3381                
l2_evict_lock_retry       = 0                   
hash_collisions           = 544                 
l2_writes_hdr_miss        = 0                   
evict_l2_ineligible       = 4096                
l2_write_bytes            = 0                   
data_size                 = 5213229568          
c                         = 67601473536         
l2_asize                  = 0                   
class                     = misc                
mutex_miss                = 0                   
snaptime                  = 1055.756679096      
duplicate_buffers_size    = 0                   
l2_misses                 = 0                   
mru_ghost_hits            = 0                   
demand_metadata_hits      = 28999               
arc_meta_max              = 97815864            
c_min                     = 2146100480          
mru_hits                  = 235669              
demand_data_misses        = 2615                
duplicate_buffers         = 0                   
demand_data_hits          = 725795              
l2_free_on_write          = 0                   
l2_feeds                  = 0                   
misses                    = 19982               
evict_skip                = 1214                
evict_l2_eligible         = 71168               
p                         = 33800736768         
l2_compress_successes     = 0                   
recycle_miss              = 0                   
l2_writes_done            = 0                   
----------

Thu Dec  4 16:14:45 IST 2014

l2_size                   = 0                   
other_size                = 16815440            
c_max                     = 67601473536         
hash_elements             = 34070               
duplicate_reads           = 0                   
prefetch_data_hits        = 537                 
l2_read_bytes             = 0                   
demand_metadata_misses    = 12446               
prefetch_data_misses      = 1540                
hash_chain_max            = 2                   
l2_hits                   = 0                   
l2_compress_failures      = 0                   
mfu_ghost_hits            = 0                   
l2_rw_clash               = 0                   
memory_throttle_count     = 0                   
crtime                    = 4.537754192         
arc_meta_used             = 97619648            
hash_elements_max         = 41359               
size                      = 5487483584          
hits                      = 791669              
mfu_hits                  = 537122              
prefetch_metadata_hits    = 10297               
hash_chains               = 50                  
evict_l2_cached           = 0                   
l2_writes_sent            = 0                   
l2_writes_error           = 0                   
deleted                   = 15                  
hdr_size                  = 12559216            
l2_compress_zeros         = 0                   
l2_abort_lowmem           = 0                   
l2_io_error               = 0                   
l2_hdr_size               = 0                   
l2_cksum_bad              = 0                   
l2_evict_reading          = 0                   
arc_meta_limit            = 16900368384         
prefetch_metadata_misses  = 3381                
l2_evict_lock_retry       = 0                   
hash_collisions           = 581                 
l2_writes_hdr_miss        = 0                   
evict_l2_ineligible       = 4096                
l2_write_bytes            = 0                   
data_size                 = 5458108928          
c                         = 67601473536         
l2_asize                  = 0                   
class                     = misc                
mutex_miss                = 0                   
snaptime                  = 1058.767796546      
duplicate_buffers_size    = 0                   
l2_misses                 = 0                   
mru_ghost_hits            = 0                   
demand_metadata_hits      = 29003               
arc_meta_max              = 97968040            
c_min                     = 2146100480          
mru_hits                  = 243713              
demand_data_misses        = 2615                
duplicate_buffers         = 0                   
demand_data_hits          = 751832              
l2_free_on_write          = 0                   
l2_feeds                  = 0                   
misses                    = 19982               
evict_skip                = 1214                
evict_l2_eligible         = 71168               
p                         = 33800736768         
l2_compress_successes     = 0                   
recycle_miss              = 0                   
l2_writes_done            = 0                   
----------

Thu Dec  4 16:14:48 IST 2014

l2_size                   = 0                   
other_size                = 16858360            
c_max                     = 67601473536         
hash_elements             = 36794               
duplicate_reads           = 0                   
prefetch_data_hits        = 537                 
l2_read_bytes             = 0                   
demand_metadata_misses    = 12454               
prefetch_data_misses      = 1540                
hash_chain_max            = 2                   
l2_hits                   = 0                   
l2_compress_failures      = 0                   
mfu_ghost_hits            = 0                   
l2_rw_clash               = 0                   
memory_throttle_count     = 0                   
crtime                    = 4.537754192         
arc_meta_used             = 98313832            
hash_elements_max         = 41359               
size                      = 5462880872          
hits                      = 802717              
mfu_hits                  = 544723              
prefetch_metadata_hits    = 10297               
hash_chains               = 75                  
evict_l2_cached           = 0                   
l2_writes_sent            = 0                   
l2_writes_error           = 0                   
deleted                   = 15                  
hdr_size                  = 12559216            
l2_compress_zeros         = 0                   
l2_abort_lowmem           = 0                   
l2_io_error               = 0                   
l2_hdr_size               = 0                   
l2_cksum_bad              = 0                   
l2_evict_reading          = 0                   
arc_meta_limit            = 16900368384         
prefetch_metadata_misses  = 3381                
l2_evict_lock_retry       = 0                   
hash_collisions           = 626                 
l2_writes_hdr_miss        = 0                   
evict_l2_ineligible       = 4096                
l2_write_bytes            = 0                   
data_size                 = 5433463296          
c                         = 67601473536         
l2_asize                  = 0                   
class                     = misc                
mutex_miss                = 0                   
snaptime                  = 1061.787519516      
duplicate_buffers_size    = 0                   
l2_misses                 = 0                   
mru_ghost_hits            = 0                   
demand_metadata_hits      = 29022               
arc_meta_max              = 98312872            
c_min                     = 2146100480          
mru_hits                  = 247160              
demand_data_misses        = 2615                
duplicate_buffers         = 0                   
demand_data_hits          = 762861              
l2_free_on_write          = 0                   
l2_feeds                  = 0                   
misses                    = 19990               
evict_skip                = 1214                
evict_l2_eligible         = 71168               
p                         = 33800736768         
l2_compress_successes     = 0                   
recycle_miss              = 0                   
l2_writes_done            = 0                   
----------

Thu Dec  4 16:14:51 IST 2014

l2_size                   = 0                   
other_size                = 16888240            
c_max                     = 67601473536         
hash_elements             = 40590               
duplicate_reads           = 0                   
prefetch_data_hits        = 537                 
l2_read_bytes             = 0                   
demand_metadata_misses    = 12458               
prefetch_data_misses      = 1540                
hash_chain_max            = 2                   
l2_hits                   = 0                   
l2_compress_failures      = 0                   
mfu_ghost_hits            = 0                   
l2_rw_clash               = 0                   
memory_throttle_count     = 0                   
crtime                    = 4.537754192         
arc_meta_used             = 98425632            
hash_elements_max         = 42641               
size                      = 5669168928          
hits                      = 815255              
mfu_hits                  = 552397              
prefetch_metadata_hits    = 10297               
hash_chains               = 86                  
evict_l2_cached           = 0                   
l2_writes_sent            = 0                   
l2_writes_error           = 0                   
deleted                   = 15                  
hdr_size                  = 12559216            
l2_compress_zeros         = 0                   
l2_abort_lowmem           = 0                   
l2_io_error               = 0                   
l2_hdr_size               = 0                   
l2_cksum_bad              = 0                   
l2_evict_reading          = 0                   
arc_meta_limit            = 16900368384         
prefetch_metadata_misses  = 3381                
l2_evict_lock_retry       = 0                   
hash_collisions           = 664                 
l2_writes_hdr_miss        = 0                   
evict_l2_ineligible       = 4096                
l2_write_bytes            = 0                   
data_size                 = 5639721472          
c                         = 67601473536         
l2_asize                  = 0                   
class                     = misc                
mutex_miss                = 0                   
snaptime                  = 1064.791199302      
duplicate_buffers_size    = 0                   
l2_misses                 = 0                   
mru_ghost_hits            = 0                   
demand_metadata_hits      = 29029               
arc_meta_max              = 98388464            
c_min                     = 2146100480          
mru_hits                  = 252024              
demand_data_misses        = 2615                
duplicate_buffers         = 0                   
demand_data_hits          = 775392              
l2_free_on_write          = 0                   
l2_feeds                  = 0                   
misses                    = 19994               
evict_skip                = 1214                
evict_l2_eligible         = 71168               
p                         = 33800736768         
l2_compress_successes     = 0                   
recycle_miss              = 0                   
l2_writes_done            = 0                   
----------

Thu Dec  4 16:14:54 IST 2014

l2_size                   = 0                   
other_size                = 17262640            
c_max                     = 67601473536         
hash_elements             = 36737               
duplicate_reads           = 0                   
prefetch_data_hits        = 537                 
l2_read_bytes             = 0                   
demand_metadata_misses    = 12462               
prefetch_data_misses      = 1540                
hash_chain_max            = 2                   
l2_hits                   = 0                   
l2_compress_failures      = 0                   
mfu_ghost_hits            = 0                   
l2_rw_clash               = 0                   
memory_throttle_count     = 0                   
crtime                    = 4.537754192         
arc_meta_used             = 98304416            
hash_elements_max         = 42641               
size                      = 6396890528          
hits                      = 840144              
mfu_hits                  = 570407              
prefetch_metadata_hits    = 10297               
hash_chains               = 80                  
evict_l2_cached           = 0                   
l2_writes_sent            = 0                   
l2_writes_error           = 0                   
deleted                   = 15                  
hdr_size                  = 12559216            
l2_compress_zeros         = 0                   
l2_abort_lowmem           = 0                   
l2_io_error               = 0                   
l2_hdr_size               = 0                   
l2_cksum_bad              = 0                   
l2_evict_reading          = 0                   
arc_meta_limit            = 16900368384         
prefetch_metadata_misses  = 3381                
l2_evict_lock_retry       = 0                   
hash_collisions           = 690                 
l2_writes_hdr_miss        = 0                   
evict_l2_ineligible       = 4096                
l2_write_bytes            = 0                   
data_size                 = 6367068672          
c                         = 67601473536         
l2_asize                  = 0                   
class                     = misc                
mutex_miss                = 0                   
snaptime                  = 1067.797220708      
duplicate_buffers_size    = 0                   
l2_misses                 = 0                   
mru_ghost_hits            = 0                   
demand_metadata_hits      = 29038               
arc_meta_max              = 98730168            
c_min                     = 2146100480          
mru_hits                  = 258903              
demand_data_misses        = 2615                
duplicate_buffers         = 0                   
demand_data_hits          = 800272              
l2_free_on_write          = 0                   
l2_feeds                  = 0                   
misses                    = 19998               
evict_skip                = 1214                
evict_l2_eligible         = 71168               
p                         = 33800736768         
l2_compress_successes     = 0                   
recycle_miss              = 0                   
l2_writes_done            = 0                   
----------

Thu Dec  4 16:14:57 IST 2014

l2_size                   = 0                   
other_size                = 17330840            
c_max                     = 67601473536         
hash_elements             = 39702               
duplicate_reads           = 0                   
prefetch_data_hits        = 537                 
l2_read_bytes             = 0                   
demand_metadata_misses    = 12466               
prefetch_data_misses      = 1540                
hash_chain_max            = 2                   
l2_hits                   = 0                   
l2_compress_failures      = 0                   
mfu_ghost_hits            = 0                   
l2_rw_clash               = 0                   
memory_throttle_count     = 0                   
crtime                    = 4.537754192         
arc_meta_used             = 99036168            
hash_elements_max         = 42641               
size                      = 6258292744          
hits                      = 851791              
mfu_hits                  = 578190              
prefetch_metadata_hits    = 10297               
hash_chains               = 63                  
evict_l2_cached           = 0                   
l2_writes_sent            = 0                   
l2_writes_error           = 0                   
deleted                   = 15                  
hdr_size                  = 12559216            
l2_compress_zeros         = 0                   
l2_abort_lowmem           = 0                   
l2_io_error               = 0                   
l2_hdr_size               = 0                   
l2_cksum_bad              = 0                   
l2_evict_reading          = 0                   
arc_meta_limit            = 16900368384         
prefetch_metadata_misses  = 3381                
l2_evict_lock_retry       = 0                   
hash_collisions           = 710                 
l2_writes_hdr_miss        = 0                   
evict_l2_ineligible       = 4096                
l2_write_bytes            = 0                   
data_size                 = 6228402688          
c                         = 67601473536         
l2_asize                  = 0                   
class                     = misc                
mutex_miss                = 0                   
snaptime                  = 1070.806236777      
duplicate_buffers_size    = 0                   
l2_misses                 = 0                   
mru_ghost_hits            = 0                   
demand_metadata_hits      = 29044               
arc_meta_max              = 99036168            
c_min                     = 2146100480          
mru_hits                  = 262767              
demand_data_misses        = 2615                
duplicate_buffers         = 0                   
demand_data_hits          = 811913              
l2_free_on_write          = 0                   
l2_feeds                  = 0                   
misses                    = 20002               
evict_skip                = 1214                
evict_l2_eligible         = 71168               
p                         = 33800736768         
l2_compress_successes     = 0                   
recycle_miss              = 0                   
l2_writes_done            = 0                   
----------

Thu Dec  4 16:15:00 IST 2014

l2_size                   = 0                   
other_size                = 17276800            
c_max                     = 67601473536         
hash_elements             = 39938               
duplicate_reads           = 0                   
prefetch_data_hits        = 537                 
l2_read_bytes             = 0                   
demand_metadata_misses    = 12470               
prefetch_data_misses      = 1540                
hash_chain_max            = 2                   
l2_hits                   = 0                   
l2_compress_failures      = 0                   
mfu_ghost_hits            = 0                   
l2_rw_clash               = 0                   
memory_throttle_count     = 0                   
crtime                    = 4.537754192         
arc_meta_used             = 98994416            
hash_elements_max         = 42641               
size                      = 5947348208          
hits                      = 854591              
mfu_hits                  = 580209              
prefetch_metadata_hits    = 10297               
hash_chains               = 64                  
evict_l2_cached           = 0                   
l2_writes_sent            = 0                   
l2_writes_error           = 0                   
deleted                   = 15                  
hdr_size                  = 12559216            
l2_compress_zeros         = 0                   
l2_abort_lowmem           = 0                   
l2_io_error               = 0                   
l2_hdr_size               = 0                   
l2_cksum_bad              = 0                   
l2_evict_reading          = 0                   
arc_meta_limit            = 16900368384         
prefetch_metadata_misses  = 3381                
l2_evict_lock_retry       = 0                   
hash_collisions           = 713                 
l2_writes_hdr_miss        = 0                   
evict_l2_ineligible       = 4096                
l2_write_bytes            = 0                   
data_size                 = 5917512192          
c                         = 67601473536         
l2_asize                  = 0                   
class                     = misc                
mutex_miss                = 0                   
snaptime                  = 1073.811610852      
duplicate_buffers_size    = 0                   
l2_misses                 = 0                   
mru_ghost_hits            = 0                   
demand_metadata_hits      = 29051               
arc_meta_max              = 99051168            
c_min                     = 2146100480          
mru_hits                  = 263548              
demand_data_misses        = 2615                
duplicate_buffers         = 0                   
demand_data_hits          = 814706              
l2_free_on_write          = 0                   
l2_feeds                  = 0                   
misses                    = 20006               
evict_skip                = 1214                
evict_l2_eligible         = 71168               
p                         = 33800736768         
l2_compress_successes     = 0                   
recycle_miss              = 0                   
l2_writes_done            = 0                   
----------

Thu Dec  4 16:15:03 IST 2014

l2_size                   = 0                   
other_size                = 17471000            
c_max                     = 67601473536         
hash_elements             = 38061               
duplicate_reads           = 0                   
prefetch_data_hits        = 537                 
l2_read_bytes             = 0                   
demand_metadata_misses    = 12470               
prefetch_data_misses      = 1540                
hash_chain_max            = 2                   
l2_hits                   = 0                   
l2_compress_failures      = 0                   
mfu_ghost_hits            = 0                   
l2_rw_clash               = 0                   
memory_throttle_count     = 0                   
crtime                    = 4.537754192         
arc_meta_used             = 98656136            
hash_elements_max         = 42641               
size                      = 6162492296          
hits                      = 881460              
mfu_hits                  = 598502              
prefetch_metadata_hits    = 10297               
hash_chains               = 77                  
evict_l2_cached           = 0                   
l2_writes_sent            = 0                   
l2_writes_error           = 0                   
deleted                   = 15                  
hdr_size                  = 12559216            
l2_compress_zeros         = 0                   
l2_abort_lowmem           = 0                   
l2_io_error               = 0                   
l2_hdr_size               = 0                   
l2_cksum_bad              = 0                   
l2_evict_reading          = 0                   
arc_meta_limit            = 16900368384         
prefetch_metadata_misses  = 3381                
l2_evict_lock_retry       = 0                   
hash_collisions           = 756                 
l2_writes_hdr_miss        = 0                   
evict_l2_ineligible       = 4096                
l2_write_bytes            = 0                   
data_size                 = 6132462080          
c                         = 67601473536         
l2_asize                  = 0                   
class                     = misc                
mutex_miss                = 0                   
snaptime                  = 1076.82642884       
duplicate_buffers_size    = 0                   
l2_misses                 = 0                   
mru_ghost_hits            = 0                   
demand_metadata_hits      = 29052               
arc_meta_max              = 99051168            
c_min                     = 2146100480          
mru_hits                  = 272124              
demand_data_misses        = 2615                
duplicate_buffers         = 0                   
demand_data_hits          = 841574              
l2_free_on_write          = 0                   
l2_feeds                  = 0                   
misses                    = 20006               
evict_skip                = 1214                
evict_l2_eligible         = 71168               
p                         = 33800736768         
l2_compress_successes     = 0                   
recycle_miss              = 0                   
l2_writes_done            = 0                   
----------

Thu Dec  4 16:15:06 IST 2014

l2_size                   = 0                   
other_size                = 17531840            
c_max                     = 67601473536         
hash_elements             = 38754               
duplicate_reads           = 0                   
prefetch_data_hits        = 537                 
l2_read_bytes             = 0                   
demand_metadata_misses    = 12478               
prefetch_data_misses      = 1540                
hash_chain_max            = 2                   
l2_hits                   = 0                   
l2_compress_failures      = 0                   
mfu_ghost_hits            = 0                   
l2_rw_clash               = 0                   
memory_throttle_count     = 0                   
crtime                    = 4.537754192         
arc_meta_used             = 98766128            
hash_elements_max         = 42641               
size                      = 6131800368          
hits                      = 891251              
mfu_hits                  = 605449              
prefetch_metadata_hits    = 10297               
hash_chains               = 83                  
evict_l2_cached           = 0                   
l2_writes_sent            = 0                   
l2_writes_error           = 0                   
deleted                   = 15                  
hdr_size                  = 12559216            
l2_compress_zeros         = 0                   
l2_abort_lowmem           = 0                   
l2_io_error               = 0                   
l2_hdr_size               = 0                   
l2_cksum_bad              = 0                   
l2_evict_reading          = 0                   
arc_meta_limit            = 16900368384         
prefetch_metadata_misses  = 3381                
l2_evict_lock_retry       = 0                   
hash_collisions           = 774                 
l2_writes_hdr_miss        = 0                   
evict_l2_ineligible       = 4096                
l2_write_bytes            = 0                   
data_size                 = 6101709312          
c                         = 67601473536         
l2_asize                  = 0                   
class                     = misc                
mutex_miss                = 0                   
snaptime                  = 1079.87068534       
duplicate_buffers_size    = 0                   
l2_misses                 = 0                   
mru_ghost_hits            = 0                   
demand_metadata_hits      = 29066               
arc_meta_max              = 99360176            
c_min                     = 2146100480          
mru_hits                  = 274968              
demand_data_misses        = 2615                
duplicate_buffers         = 0                   
demand_data_hits          = 851351              
l2_free_on_write          = 0                   
l2_feeds                  = 0                   
misses                    = 20014               
evict_skip                = 1214                
evict_l2_eligible         = 71168               
p                         = 33800736768         
l2_compress_successes     = 0                   
recycle_miss              = 0                   
l2_writes_done            = 0                   
----------

Thu Dec  4 16:15:09 IST 2014

l2_size                   = 0                   
other_size                = 17593040            
c_max                     = 67601473536         
hash_elements             = 40255               
duplicate_reads           = 0                   
prefetch_data_hits        = 537                 
l2_read_bytes             = 0                   
demand_metadata_misses    = 12478               
prefetch_data_misses      = 1540                
hash_chain_max            = 2                   
l2_hits                   = 0                   
l2_compress_failures      = 0                   
mfu_ghost_hits            = 0                   
l2_rw_clash               = 0                   
memory_throttle_count     = 0                   
crtime                    = 4.537754192         
arc_meta_used             = 98860096            
hash_elements_max         = 42641               
size                      = 6205425728          
hits                      = 902579              
mfu_hits                  = 612627              
prefetch_metadata_hits    = 10297               
hash_chains               = 81                  
evict_l2_cached           = 0                   
l2_writes_sent            = 0                   
l2_writes_error           = 0                   
deleted                   = 15                  
hdr_size                  = 12559216            
l2_compress_zeros         = 0                   
l2_abort_lowmem           = 0                   
l2_io_error               = 0                   
l2_hdr_size               = 0                   
l2_cksum_bad              = 0                   
l2_evict_reading          = 0                   
arc_meta_limit            = 16900368384         
prefetch_metadata_misses  = 3381                
l2_evict_lock_retry       = 0                   
hash_collisions           = 794                 
l2_writes_hdr_miss        = 0                   
evict_l2_ineligible       = 4096                
l2_write_bytes            = 0                   
data_size                 = 6175273472          
c                         = 67601473536         
l2_asize                  = 0                   
class                     = misc                
mutex_miss                = 0                   
snaptime                  = 1082.875826687      
duplicate_buffers_size    = 0                   
l2_misses                 = 0                   
mru_ghost_hits            = 0                   
demand_metadata_hits      = 29066               
arc_meta_max              = 99360176            
c_min                     = 2146100480          
mru_hits                  = 279118              
demand_data_misses        = 2615                
duplicate_buffers         = 0                   
demand_data_hits          = 862679              
l2_free_on_write          = 0                   
l2_feeds                  = 0                   
misses                    = 20014               
evict_skip                = 1214                
evict_l2_eligible         = 71168               
p                         = 33800736768         
l2_compress_successes     = 0                   
recycle_miss              = 0                   
l2_writes_done            = 0                   
----------

Thu Dec  4 16:15:12 IST 2014

l2_size                   = 0                   
other_size                = 17598240            
c_max                     = 67601473536         
hash_elements             = 43117               
duplicate_reads           = 0                   
prefetch_data_hits        = 537                 
l2_read_bytes             = 0                   
demand_metadata_misses    = 12478               
prefetch_data_misses      = 1540                
hash_chain_max            = 2                   
l2_hits                   = 0                   
l2_compress_failures      = 0                   
mfu_ghost_hits            = 0                   
l2_rw_clash               = 0                   
memory_throttle_count     = 0                   
crtime                    = 4.537754192         
arc_meta_used             = 98865296            
hash_elements_max         = 43118               
size                      = 6013541520          
hits                      = 905362              
mfu_hits                  = 613511              
prefetch_metadata_hits    = 10297               
hash_chains               = 111                 
evict_l2_cached           = 0                   
l2_writes_sent            = 0                   
l2_writes_error           = 0                   
deleted                   = 15                  
hdr_size                  = 12559216            
l2_compress_zeros         = 0                   
l2_abort_lowmem           = 0                   
l2_io_error               = 0                   
l2_hdr_size               = 0                   
l2_cksum_bad              = 0                   
l2_evict_reading          = 0                   
arc_meta_limit            = 16900368384         
prefetch_metadata_misses  = 3381                
l2_evict_lock_retry       = 0                   
hash_collisions           = 831                 
l2_writes_hdr_miss        = 0                   
evict_l2_ineligible       = 4096                
l2_write_bytes            = 0                   
data_size                 = 5983384064          
c                         = 67601473536         
l2_asize                  = 0                   
class                     = misc                
mutex_miss                = 0                   
snaptime                  = 1085.879514131      
duplicate_buffers_size    = 0                   
l2_misses                 = 0                   
mru_ghost_hits            = 0                   
demand_metadata_hits      = 29066               
arc_meta_max              = 99360176            
c_min                     = 2146100480          
mru_hits                  = 281017              
demand_data_misses        = 2615                
duplicate_buffers         = 0                   
demand_data_hits          = 865462              
l2_free_on_write          = 0                   
l2_feeds                  = 0                   
misses                    = 20014               
evict_skip                = 1214                
evict_l2_eligible         = 71168               
p                         = 33800736768         
l2_compress_successes     = 0                   
recycle_miss              = 0                   
l2_writes_done            = 0                   
----------

Thu Dec  4 16:15:15 IST 2014

l2_size                   = 0                   
other_size                = 17603440            
c_max                     = 67601473536         
hash_elements             = 43174               
duplicate_reads           = 0                   
prefetch_data_hits        = 537                 
l2_read_bytes             = 0                   
demand_metadata_misses    = 12482               
prefetch_data_misses      = 1540                
hash_chain_max            = 2                   
l2_hits                   = 0                   
l2_compress_failures      = 0                   
mfu_ghost_hits            = 0                   
l2_rw_clash               = 0                   
memory_throttle_count     = 0                   
crtime                    = 4.537754192         
arc_meta_used             = 99480800            
hash_elements_max         = 43929               
size                      = 5807325408          
hits                      = 908319              
mfu_hits                  = 615797              
prefetch_metadata_hits    = 10297               
hash_chains               = 102                 
evict_l2_cached           = 0                   
l2_writes_sent            = 0                   
l2_writes_error           = 0                   
deleted                   = 15                  
hdr_size                  = 12559216            
l2_compress_zeros         = 0                   
l2_abort_lowmem           = 0                   
l2_io_error               = 0                   
l2_hdr_size               = 0                   
l2_cksum_bad              = 0                   
l2_evict_reading          = 0                   
arc_meta_limit            = 16900368384         
prefetch_metadata_misses  = 3381                
l2_evict_lock_retry       = 0                   
hash_collisions           = 832                 
l2_writes_hdr_miss        = 0                   
evict_l2_ineligible       = 4096                
l2_write_bytes            = 0                   
data_size                 = 5777162752          
c                         = 67601473536         
l2_asize                  = 0                   
class                     = misc                
mutex_miss                = 0                   
snaptime                  = 1088.884358936      
duplicate_buffers_size    = 0                   
l2_misses                 = 0                   
mru_ghost_hits            = 0                   
demand_metadata_hits      = 29075               
arc_meta_max              = 99480800            
c_min                     = 2146100480          
mru_hits                  = 281688              
demand_data_misses        = 2615                
duplicate_buffers         = 0                   
demand_data_hits          = 868410              
l2_free_on_write          = 0                   
l2_feeds                  = 0                   
misses                    = 20018               
evict_skip                = 1214                
evict_l2_eligible         = 71168               
p                         = 33800736768         
l2_compress_successes     = 0                   
recycle_miss              = 0                   
l2_writes_done            = 0                   
----------

Thu Dec  4 16:15:18 IST 2014

l2_size                   = 0                   
other_size                = 17668200            
c_max                     = 67601473536         
hash_elements             = 41509               
duplicate_reads           = 0                   
prefetch_data_hits        = 537                 
l2_read_bytes             = 0                   
demand_metadata_misses    = 12488               
prefetch_data_misses      = 1540                
hash_chain_max            = 2                   
l2_hits                   = 0                   
l2_compress_failures      = 0                   
mfu_ghost_hits            = 0                   
l2_rw_clash               = 0                   
memory_throttle_count     = 0                   
crtime                    = 4.537754192         
arc_meta_used             = 98984408            
hash_elements_max         = 43929               
size                      = 5884816856          
hits                      = 927065              
mfu_hits                  = 630052              
prefetch_metadata_hits    = 10297               
hash_chains               = 93                  
evict_l2_cached           = 0                   
l2_writes_sent            = 0                   
l2_writes_error           = 0                   
deleted                   = 15                  
hdr_size                  = 12559216            
l2_compress_zeros         = 0                   
l2_abort_lowmem           = 0                   
l2_io_error               = 0                   
l2_hdr_size               = 0                   
l2_cksum_bad              = 0                   
l2_evict_reading          = 0                   
arc_meta_limit            = 16900368384         
prefetch_metadata_misses  = 3381                
l2_evict_lock_retry       = 0                   
hash_collisions           = 843                 
l2_writes_hdr_miss        = 0                   
evict_l2_ineligible       = 4096                
l2_write_bytes            = 0                   
data_size                 = 5854589440          
c                         = 67601473536         
l2_asize                  = 0                   
class                     = misc                
mutex_miss                = 0                   
snaptime                  = 1091.888514904      
duplicate_buffers_size    = 0                   
l2_misses                 = 0                   
mru_ghost_hits            = 0                   
demand_metadata_hits      = 29087               
arc_meta_max              = 99568328            
c_min                     = 2146100480          
mru_hits                  = 286179              
demand_data_misses        = 2615                
duplicate_buffers         = 0                   
demand_data_hits          = 887144              
l2_free_on_write          = 0                   
l2_feeds                  = 0                   
misses                    = 20024               
evict_skip                = 1214                
evict_l2_eligible         = 71168               
p                         = 33800736768         
l2_compress_successes     = 0                   
recycle_miss              = 0                   
l2_writes_done            = 0                   
----------

Thu Dec  4 16:15:21 IST 2014

l2_size                   = 0                   
other_size                = 17646400            
c_max                     = 67601473536         
hash_elements             = 39031               
duplicate_reads           = 0                   
prefetch_data_hits        = 537                 
l2_read_bytes             = 0                   
demand_metadata_misses    = 12488               
prefetch_data_misses      = 1540                
hash_chain_max            = 2                   
l2_hits                   = 0                   
l2_compress_failures      = 0                   
mfu_ghost_hits            = 0                   
l2_rw_clash               = 0                   
memory_throttle_count     = 0                   
crtime                    = 4.537754192         
arc_meta_used             = 98978992            
hash_elements_max         = 43929               
size                      = 5865412784          
hits                      = 955131              
mfu_hits                  = 649821              
prefetch_metadata_hits    = 10297               
hash_chains               = 76                  
evict_l2_cached           = 0                   
l2_writes_sent            = 0                   
l2_writes_error           = 0                   
deleted                   = 15                  
hdr_size                  = 12559216            
l2_compress_zeros         = 0                   
l2_abort_lowmem           = 0                   
l2_io_error               = 0                   
l2_hdr_size               = 0                   
l2_cksum_bad              = 0                   
l2_evict_reading          = 0                   
arc_meta_limit            = 16900368384         
prefetch_metadata_misses  = 3381                
l2_evict_lock_retry       = 0                   
hash_collisions           = 857                 
l2_writes_hdr_miss        = 0                   
evict_l2_ineligible       = 4096                
l2_write_bytes            = 0                   
data_size                 = 5835207168          
c                         = 67601473536         
l2_asize                  = 0                   
class                     = misc                
mutex_miss                = 0                   
snaptime                  = 1094.945818674      
duplicate_buffers_size    = 0                   
l2_misses                 = 0                   
mru_ghost_hits            = 0                   
demand_metadata_hits      = 29087               
arc_meta_max              = 99568328            
c_min                     = 2146100480          
mru_hits                  = 294476              
demand_data_misses        = 2615                
duplicate_buffers         = 0                   
demand_data_hits          = 915210              
l2_free_on_write          = 0                   
l2_feeds                  = 0                   
misses                    = 20024               
evict_skip                = 1214                
evict_l2_eligible         = 71168               
p                         = 33800736768         
l2_compress_successes     = 0                   
recycle_miss              = 0                   
l2_writes_done            = 0                   
----------

Thu Dec  4 16:15:24 IST 2014

l2_size                   = 0                   
other_size                = 17752040            
c_max                     = 67601473536         
hash_elements             = 41863               
duplicate_reads           = 0                   
prefetch_data_hits        = 537                 
l2_read_bytes             = 0                   
demand_metadata_misses    = 12492               
prefetch_data_misses      = 1540                
hash_chain_max            = 2                   
l2_hits                   = 0                   
l2_compress_failures      = 0                   
mfu_ghost_hits            = 0                   
l2_rw_clash               = 0                   
memory_throttle_count     = 0                   
crtime                    = 4.537754192         
arc_meta_used             = 99137880            
hash_elements_max         = 43929               
size                      = 6192858456          
hits                      = 968809              
mfu_hits                  = 659725              
prefetch_metadata_hits    = 10297               
hash_chains               = 87                  
evict_l2_cached           = 0                   
l2_writes_sent            = 0                   
l2_writes_error           = 0                   
deleted                   = 15                  
hdr_size                  = 12559216            
l2_compress_zeros         = 0                   
l2_abort_lowmem           = 0                   
l2_io_error               = 0                   
l2_hdr_size               = 0                   
l2_cksum_bad              = 0                   
l2_evict_reading          = 0                   
arc_meta_limit            = 16900368384         
prefetch_metadata_misses  = 3381                
l2_evict_lock_retry       = 0                   
hash_collisions           = 880                 
l2_writes_hdr_miss        = 0                   
evict_l2_ineligible       = 4096                
l2_write_bytes            = 0                   
data_size                 = 6162547200          
c                         = 67601473536         
l2_asize                  = 0                   
class                     = misc                
mutex_miss                = 0                   
snaptime                  = 1097.957144074      
duplicate_buffers_size    = 0                   
l2_misses                 = 0                   
mru_ghost_hits            = 0                   
demand_metadata_hits      = 29094               
arc_meta_max              = 99660136            
c_min                     = 2146100480          
mru_hits                  = 298250              
demand_data_misses        = 2615                
duplicate_buffers         = 0                   
demand_data_hits          = 928881              
l2_free_on_write          = 0                   
l2_feeds                  = 0                   
misses                    = 20028               
evict_skip                = 1214                
evict_l2_eligible         = 71168               
p                         = 33800736768         
l2_compress_successes     = 0                   
recycle_miss              = 0                   
l2_writes_done            = 0                   
----------

Thu Dec  4 16:15:27 IST 2014

l2_size                   = 0                   
other_size                = 17651840            
c_max                     = 67601473536         
hash_elements             = 44052               
duplicate_reads           = 0                   
prefetch_data_hits        = 537                 
l2_read_bytes             = 0                   
demand_metadata_misses    = 12496               
prefetch_data_misses      = 1540                
hash_chain_max            = 2                   
l2_hits                   = 0                   
l2_compress_failures      = 0                   
mfu_ghost_hits            = 0                   
l2_rw_clash               = 0                   
memory_throttle_count     = 0                   
crtime                    = 4.537754192         
arc_meta_used             = 99623408            
hash_elements_max         = 44527               
size                      = 5948239344          
hits                      = 972940              
mfu_hits                  = 661821              
prefetch_metadata_hits    = 10297               
hash_chains               = 92                  
evict_l2_cached           = 0                   
l2_writes_sent            = 0                   
l2_writes_error           = 0                   
deleted                   = 15                  
hdr_size                  = 12559216            
l2_compress_zeros         = 0                   
l2_abort_lowmem           = 0                   
l2_io_error               = 0                   
l2_hdr_size               = 0                   
l2_cksum_bad              = 0                   
l2_evict_reading          = 0                   
arc_meta_limit            = 16900368384         
prefetch_metadata_misses  = 3381                
l2_evict_lock_retry       = 0                   
hash_collisions           = 891                 
l2_writes_hdr_miss        = 0                   
evict_l2_ineligible       = 4096                
l2_write_bytes            = 0                   
data_size                 = 5918028288          
c                         = 67601473536         
l2_asize                  = 0                   
class                     = misc                
mutex_miss                = 0                   
snaptime                  = 1100.964341965      
duplicate_buffers_size    = 0                   
l2_misses                 = 0                   
mru_ghost_hits            = 0                   
demand_metadata_hits      = 29100               
arc_meta_max              = 99660136            
c_min                     = 2146100480          
mru_hits                  = 300285              
demand_data_misses        = 2615                
duplicate_buffers         = 0                   
demand_data_hits          = 933006              
l2_free_on_write          = 0                   
l2_feeds                  = 0                   
misses                    = 20032               
evict_skip                = 1214                
evict_l2_eligible         = 71168               
p                         = 33800736768         
l2_compress_successes     = 0                   
recycle_miss              = 0                   
l2_writes_done            = 0                   
----------

Thu Dec  4 16:15:30 IST 2014

l2_size                   = 0                   
other_size                = 17736840            
c_max                     = 67601473536         
hash_elements             = 39286               
duplicate_reads           = 0                   
prefetch_data_hits        = 537                 
l2_read_bytes             = 0                   
demand_metadata_misses    = 12500               
prefetch_data_misses      = 1540                
hash_chain_max            = 2                   
l2_hits                   = 0                   
l2_compress_failures      = 0                   
mfu_ghost_hits            = 0                   
l2_rw_clash               = 0                   
memory_throttle_count     = 0                   
crtime                    = 4.537754192         
arc_meta_used             = 99134968            
hash_elements_max         = 44527               
size                      = 5866748408          
hits                      = 999296              
mfu_hits                  = 681359              
prefetch_metadata_hits    = 10297               
hash_chains               = 89                  
evict_l2_cached           = 0                   
l2_writes_sent            = 0                   
l2_writes_error           = 0                   
deleted                   = 15                  
hdr_size                  = 12559216            
l2_compress_zeros         = 0                   
l2_abort_lowmem           = 0                   
l2_io_error               = 0                   
l2_hdr_size               = 0                   
l2_cksum_bad              = 0                   
l2_evict_reading          = 0                   
arc_meta_limit            = 16900368384         
prefetch_metadata_misses  = 3381                
l2_evict_lock_retry       = 0                   
hash_collisions           = 922                 
l2_writes_hdr_miss        = 0                   
evict_l2_ineligible       = 4096                
l2_write_bytes            = 0                   
data_size                 = 5836452352          
c                         = 67601473536         
l2_asize                  = 0                   
class                     = misc                
mutex_miss                = 0                   
snaptime                  = 1103.989100171      
duplicate_buffers_size    = 0                   
l2_misses                 = 0                   
mru_ghost_hits            = 0                   
demand_metadata_hits      = 29108               
arc_meta_max              = 99718496            
c_min                     = 2146100480          
mru_hits                  = 307103              
demand_data_misses        = 2615                
duplicate_buffers         = 0                   
demand_data_hits          = 959354              
l2_free_on_write          = 0                   
l2_feeds                  = 0                   
misses                    = 20036               
evict_skip                = 1214                
evict_l2_eligible         = 71168               
p                         = 33800736768         
l2_compress_successes     = 0                   
recycle_miss              = 0                   
l2_writes_done            = 0                   
----------

Thu Dec  4 16:15:33 IST 2014

l2_size                   = 0                   
other_size                = 17708240            
c_max                     = 67601473536         
hash_elements             = 43427               
duplicate_reads           = 0                   
prefetch_data_hits        = 537                 
l2_read_bytes             = 0                   
demand_metadata_misses    = 12500               
prefetch_data_misses      = 1540                
hash_chain_max            = 2                   
l2_hits                   = 0                   
l2_compress_failures      = 0                   
mfu_ghost_hits            = 0                   
l2_rw_clash               = 0                   
memory_throttle_count     = 0                   
crtime                    = 4.537754192         
arc_meta_used             = 99106368            
hash_elements_max         = 44527               
size                      = 5953882688          
hits                      = 1009876             
mfu_hits                  = 687985              
prefetch_metadata_hits    = 10297               
hash_chains               = 107                 
evict_l2_cached           = 0                   
l2_writes_sent            = 0                   
l2_writes_error           = 0                   
deleted                   = 15                  
hdr_size                  = 12559216            
l2_compress_zeros         = 0                   
l2_abort_lowmem           = 0                   
l2_io_error               = 0                   
l2_hdr_size               = 0                   
l2_cksum_bad              = 0                   
l2_evict_reading          = 0                   
arc_meta_limit            = 16900368384         
prefetch_metadata_misses  = 3381                
l2_evict_lock_retry       = 0                   
hash_collisions           = 959                 
l2_writes_hdr_miss        = 0                   
evict_l2_ineligible       = 4096                
l2_write_bytes            = 0                   
data_size                 = 5923615232          
c                         = 67601473536         
l2_asize                  = 0                   
class                     = misc                
mutex_miss                = 0                   
snaptime                  = 1106.99462468       
duplicate_buffers_size    = 0                   
l2_misses                 = 0                   
mru_ghost_hits            = 0                   
demand_metadata_hits      = 29109               
arc_meta_max              = 99718496            
c_min                     = 2146100480          
mru_hits                  = 311057              
demand_data_misses        = 2615                
duplicate_buffers         = 0                   
demand_data_hits          = 969933              
l2_free_on_write          = 0                   
l2_feeds                  = 0                   
misses                    = 20036               
evict_skip                = 1214                
evict_l2_eligible         = 71168               
p                         = 33800736768         
l2_compress_successes     = 0                   
recycle_miss              = 0                   
l2_writes_done            = 0                   
----------

Thu Dec  4 16:15:36 IST 2014

l2_size                   = 0                   
other_size                = 17820440            
c_max                     = 67601473536         
hash_elements             = 38756               
duplicate_reads           = 0                   
prefetch_data_hits        = 537                 
l2_read_bytes             = 0                   
demand_metadata_misses    = 12508               
prefetch_data_misses      = 1540                
hash_chain_max            = 2                   
l2_hits                   = 0                   
l2_compress_failures      = 0                   
mfu_ghost_hits            = 0                   
l2_rw_clash               = 0                   
memory_throttle_count     = 0                   
crtime                    = 4.537754192         
arc_meta_used             = 99222664            
hash_elements_max         = 44527               
size                      = 5931061384          
hits                      = 1034158             
mfu_hits                  = 705727              
prefetch_metadata_hits    = 10297               
hash_chains               = 79                  
evict_l2_cached           = 0                   
l2_writes_sent            = 0                   
l2_writes_error           = 0                   
deleted                   = 15                  
hdr_size                  = 12559216            
l2_compress_zeros         = 0                   
l2_abort_lowmem           = 0                   
l2_io_error               = 0                   
l2_hdr_size               = 0                   
l2_cksum_bad              = 0                   
l2_evict_reading          = 0                   
arc_meta_limit            = 16900368384         
prefetch_metadata_misses  = 3381                
l2_evict_lock_retry       = 0                   
hash_collisions           = 970                 
l2_writes_hdr_miss        = 0                   
evict_l2_ineligible       = 4096                
l2_write_bytes            = 0                   
data_size                 = 5900681728          
c                         = 67601473536         
l2_asize                  = 0                   
class                     = misc                
mutex_miss                = 0                   
snaptime                  = 1109.998397046      
duplicate_buffers_size    = 0                   
l2_misses                 = 0                   
mru_ghost_hits            = 0                   
demand_metadata_hits      = 29126               
arc_meta_max              = 99816784            
c_min                     = 2146100480          
mru_hits                  = 317597              
demand_data_misses        = 2615                
duplicate_buffers         = 0                   
demand_data_hits          = 994198              
l2_free_on_write          = 0                   
l2_feeds                  = 0                   
misses                    = 20044               
evict_skip                = 1214                
evict_l2_eligible         = 71168               
p                         = 33800736768         
l2_compress_successes     = 0                   
recycle_miss              = 0                   
l2_writes_done            = 0                   
----------

Thu Dec  4 16:15:39 IST 2014

l2_size                   = 0                   
other_size                = 17808640            
c_max                     = 67601473536         
hash_elements             = 44706               
duplicate_reads           = 0                   
prefetch_data_hits        = 537                 
l2_read_bytes             = 0                   
demand_metadata_misses    = 12508               
prefetch_data_misses      = 1540                
hash_chain_max            = 2                   
l2_hits                   = 0                   
l2_compress_failures      = 0                   
mfu_ghost_hits            = 0                   
l2_rw_clash               = 0                   
memory_throttle_count     = 0                   
crtime                    = 4.537754192         
arc_meta_used             = 99212592            
hash_elements_max         = 44706               
size                      = 6196209968          
hits                      = 1039940             
mfu_hits                  = 709638              
prefetch_metadata_hits    = 10297               
hash_chains               = 103                 
evict_l2_cached           = 0                   
l2_writes_sent            = 0                   
l2_writes_error           = 0                   
deleted                   = 15                  
hdr_size                  = 12560944            
l2_compress_zeros         = 0                   
l2_abort_lowmem           = 0                   
l2_io_error               = 0                   
l2_hdr_size               = 0                   
l2_cksum_bad              = 0                   
l2_evict_reading          = 0                   
arc_meta_limit            = 16900368384         
prefetch_metadata_misses  = 3381                
l2_evict_lock_retry       = 0                   
hash_collisions           = 1014                
l2_writes_hdr_miss        = 0                   
evict_l2_ineligible       = 4096                
l2_write_bytes            = 0                   
data_size                 = 6165840384          
c                         = 67601473536         
l2_asize                  = 0                   
class                     = misc                
mutex_miss                = 0                   
snaptime                  = 1113.002157194      
duplicate_buffers_size    = 0                   
l2_misses                 = 0                   
mru_ghost_hits            = 0                   
demand_metadata_hits      = 29126               
arc_meta_max              = 99816784            
c_min                     = 2146100480          
mru_hits                  = 319468              
demand_data_misses        = 2615                
duplicate_buffers         = 0                   
demand_data_hits          = 999980              
l2_free_on_write          = 0                   
l2_feeds                  = 0                   
misses                    = 20044               
evict_skip                = 1214                
evict_l2_eligible         = 71168               
p                         = 33800736768         
l2_compress_successes     = 0                   
recycle_miss              = 0                   
l2_writes_done            = 0                   
----------

Thu Dec  4 16:15:42 IST 2014

l2_size                   = 0                   
other_size                = 17820240            
c_max                     = 67601473536         
hash_elements             = 42808               
duplicate_reads           = 0                   
prefetch_data_hits        = 537                 
l2_read_bytes             = 0                   
demand_metadata_misses    = 12516               
prefetch_data_misses      = 1540                
hash_chain_max            = 2                   
l2_hits                   = 0                   
l2_compress_failures      = 0                   
mfu_ghost_hits            = 0                   
l2_rw_clash               = 0                   
memory_throttle_count     = 0                   
crtime                    = 4.537754192         
arc_meta_used             = 99240576            
hash_elements_max         = 46675               
size                      = 6415390336          
hits                      = 1059255             
mfu_hits                  = 723283              
prefetch_metadata_hits    = 10297               
hash_chains               = 108                 
evict_l2_cached           = 0                   
l2_writes_sent            = 0                   
l2_writes_error           = 0                   
deleted                   = 15                  
hdr_size                  = 12560944            
l2_compress_zeros         = 0                   
l2_abort_lowmem           = 0                   
l2_io_error               = 0                   
l2_hdr_size               = 0                   
l2_cksum_bad              = 0                   
l2_evict_reading          = 0                   
arc_meta_limit            = 16900368384         
prefetch_metadata_misses  = 3381                
l2_evict_lock_retry       = 0                   
hash_collisions           = 1042                
l2_writes_hdr_miss        = 0                   
evict_l2_ineligible       = 4096                
l2_write_bytes            = 0                   
data_size                 = 6385009152          
c                         = 67601473536         
l2_asize                  = 0                   
class                     = misc                
mutex_miss                = 0                   
snaptime                  = 1116.00772062       
duplicate_buffers_size    = 0                   
l2_misses                 = 0                   
mru_ghost_hits            = 0                   
demand_metadata_hits      = 29140               
arc_meta_max              = 99824704            
c_min                     = 2146100480          
mru_hits                  = 325138              
demand_data_misses        = 2615                
duplicate_buffers         = 0                   
demand_data_hits          = 1019281             
l2_free_on_write          = 0                   
l2_feeds                  = 0                   
misses                    = 20052               
evict_skip                = 1214                
evict_l2_eligible         = 71168               
p                         = 33800736768         
l2_compress_successes     = 0                   
recycle_miss              = 0                   
l2_writes_done            = 0                   
----------

Thu Dec  4 16:15:45 IST 2014

l2_size                   = 0                   
other_size                = 17674040            
c_max                     = 67601473536         
hash_elements             = 44891               
duplicate_reads           = 0                   
prefetch_data_hits        = 537                 
l2_read_bytes             = 0                   
demand_metadata_misses    = 12516               
prefetch_data_misses      = 1540                
hash_chain_max            = 2                   
l2_hits                   = 0                   
l2_compress_failures      = 0                   
mfu_ghost_hits            = 0                   
l2_rw_clash               = 0                   
memory_throttle_count     = 0                   
crtime                    = 4.537754192         
arc_meta_used             = 99094376            
hash_elements_max         = 46675               
size                      = 5900262248          
hits                      = 1068199             
mfu_hits                  = 728474              
prefetch_metadata_hits    = 10297               
hash_chains               = 130                 
evict_l2_cached           = 0                   
l2_writes_sent            = 0                   
l2_writes_error           = 0                   
deleted                   = 15                  
hdr_size                  = 12560944            
l2_compress_zeros         = 0                   
l2_abort_lowmem           = 0                   
l2_io_error               = 0                   
l2_hdr_size               = 0                   
l2_cksum_bad              = 0                   
l2_evict_reading          = 0                   
arc_meta_limit            = 16900368384         
prefetch_metadata_misses  = 3381                
l2_evict_lock_retry       = 0                   
hash_collisions           = 1081                
l2_writes_hdr_miss        = 0                   
evict_l2_ineligible       = 4096                
l2_write_bytes            = 0                   
data_size                 = 5870027264          
c                         = 67601473536         
l2_asize                  = 0                   
class                     = misc                
mutex_miss                = 0                   
snaptime                  = 1119.021235208      
duplicate_buffers_size    = 0                   
l2_misses                 = 0                   
mru_ghost_hits            = 0                   
demand_metadata_hits      = 29140               
arc_meta_max              = 99824704            
c_min                     = 2146100480          
mru_hits                  = 328891              
demand_data_misses        = 2615                
duplicate_buffers         = 0                   
demand_data_hits          = 1028225             
l2_free_on_write          = 0                   
l2_feeds                  = 0                   
misses                    = 20052               
evict_skip                = 1214                
evict_l2_eligible         = 71168               
p                         = 33800736768         
l2_compress_successes     = 0                   
recycle_miss              = 0                   
l2_writes_done            = 0                   
----------

Thu Dec  4 16:15:48 IST 2014

l2_size                   = 0                   
other_size                = 17766200            
c_max                     = 67601473536         
hash_elements             = 39994               
duplicate_reads           = 0                   
prefetch_data_hits        = 537                 
l2_read_bytes             = 0                   
demand_metadata_misses    = 12524               
prefetch_data_misses      = 1540                
hash_chain_max            = 2                   
l2_hits                   = 0                   
l2_compress_failures      = 0                   
mfu_ghost_hits            = 0                   
l2_rw_clash               = 0                   
memory_throttle_count     = 0                   
crtime                    = 4.537754192         
arc_meta_used             = 99751784            
hash_elements_max         = 46675               
size                      = 5771420520          
hits                      = 1088758             
mfu_hits                  = 743653              
prefetch_metadata_hits    = 10297               
hash_chains               = 94                  
evict_l2_cached           = 0                   
l2_writes_sent            = 0                   
l2_writes_error           = 0                   
deleted                   = 15                  
hdr_size                  = 12560944            
l2_compress_zeros         = 0                   
l2_abort_lowmem           = 0                   
l2_io_error               = 0                   
l2_hdr_size               = 0                   
l2_cksum_bad              = 0                   
l2_evict_reading          = 0                   
arc_meta_limit            = 16900368384         
prefetch_metadata_misses  = 3381                
l2_evict_lock_retry       = 0                   
hash_collisions           = 1092                
l2_writes_hdr_miss        = 0                   
evict_l2_ineligible       = 4096                
l2_write_bytes            = 0                   
data_size                 = 5741093376          
c                         = 67601473536         
l2_asize                  = 0                   
class                     = misc                
mutex_miss                = 0                   
snaptime                  = 1122.026835395      
duplicate_buffers_size    = 0                   
l2_misses                 = 0                   
mru_ghost_hits            = 0                   
demand_metadata_hits      = 29147               
arc_meta_max              = 99824704            
c_min                     = 2146100480          
mru_hits                  = 334271              
demand_data_misses        = 2615                
duplicate_buffers         = 0                   
demand_data_hits          = 1048777             
l2_free_on_write          = 0                   
l2_feeds                  = 0                   
misses                    = 20060               
evict_skip                = 1214                
evict_l2_eligible         = 71168               
p                         = 33800736768         
l2_compress_successes     = 0                   
recycle_miss              = 0                   
l2_writes_done            = 0                   
----------

Thu Dec  4 16:15:51 IST 2014

l2_size                   = 0                   
other_size                = 17829240            
c_max                     = 67601473536         
hash_elements             = 43380               
duplicate_reads           = 0                   
prefetch_data_hits        = 537                 
l2_read_bytes             = 0                   
demand_metadata_misses    = 12524               
prefetch_data_misses      = 1540                
hash_chain_max            = 2                   
l2_hits                   = 0                   
l2_compress_failures      = 0                   
mfu_ghost_hits            = 0                   
l2_rw_clash               = 0                   
memory_throttle_count     = 0                   
crtime                    = 4.537754192         
arc_meta_used             = 99249576            
hash_elements_max         = 46675               
size                      = 6035421608          
hits                      = 1096297             
mfu_hits                  = 749091              
prefetch_metadata_hits    = 10297               
hash_chains               = 87                  
evict_l2_cached           = 0                   
l2_writes_sent            = 0                   
l2_writes_error           = 0                   
deleted                   = 15                  
hdr_size                  = 12560944            
l2_compress_zeros         = 0                   
l2_abort_lowmem           = 0                   
l2_io_error               = 0                   
l2_hdr_size               = 0                   
l2_cksum_bad              = 0                   
l2_evict_reading          = 0                   
arc_meta_limit            = 16900368384         
prefetch_metadata_misses  = 3381                
l2_evict_lock_retry       = 0                   
hash_collisions           = 1102                
l2_writes_hdr_miss        = 0                   
evict_l2_ineligible       = 4096                
l2_write_bytes            = 0                   
data_size                 = 6005031424          
c                         = 67601473536         
l2_asize                  = 0                   
class                     = misc                
mutex_miss                = 0                   
snaptime                  = 1125.031042354      
duplicate_buffers_size    = 0                   
l2_misses                 = 0                   
mru_ghost_hits            = 0                   
demand_metadata_hits      = 29154               
arc_meta_max              = 99829992            
c_min                     = 2146100480          
mru_hits                  = 336372              
demand_data_misses        = 2615                
duplicate_buffers         = 0                   
demand_data_hits          = 1056309             
l2_free_on_write          = 0                   
l2_feeds                  = 0                   
misses                    = 20060               
evict_skip                = 1214                
evict_l2_eligible         = 71168               
p                         = 33800736768         
l2_compress_successes     = 0                   
recycle_miss              = 0                   
l2_writes_done            = 0                   
----------

Thu Dec  4 16:15:54 IST 2014

l2_size                   = 0                   
other_size                = 17900600            
c_max                     = 67601473536         
hash_elements             = 39997               
duplicate_reads           = 0                   
prefetch_data_hits        = 537                 
l2_read_bytes             = 0                   
demand_metadata_misses    = 12528               
prefetch_data_misses      = 1540                
hash_chain_max            = 2                   
l2_hits                   = 0                   
l2_compress_failures      = 0                   
mfu_ghost_hits            = 0                   
l2_rw_clash               = 0                   
memory_throttle_count     = 0                   
crtime                    = 4.537754192         
arc_meta_used             = 99390568            
hash_elements_max         = 46675               
size                      = 5948399720          
hits                      = 1120591             
mfu_hits                  = 765902              
prefetch_metadata_hits    = 10297               
hash_chains               = 83                  
evict_l2_cached           = 0                   
l2_writes_sent            = 0                   
l2_writes_error           = 0                   
deleted                   = 15                  
hdr_size                  = 12560944            
l2_compress_zeros         = 0                   
l2_abort_lowmem           = 0                   
l2_io_error               = 0                   
l2_hdr_size               = 0                   
l2_cksum_bad              = 0                   
l2_evict_reading          = 0                   
arc_meta_limit            = 16900368384         
prefetch_metadata_misses  = 3381                
l2_evict_lock_retry       = 0                   
hash_collisions           = 1118                
l2_writes_hdr_miss        = 0                   
evict_l2_ineligible       = 4096                
l2_write_bytes            = 0                   
data_size                 = 5917938176          
c                         = 67601473536         
l2_asize                  = 0                   
class                     = misc                
mutex_miss                = 0                   
snaptime                  = 1128.034852134      
duplicate_buffers_size    = 0                   
l2_misses                 = 0                   
mru_ghost_hits            = 0                   
demand_metadata_hits      = 29163               
arc_meta_max              = 99971728            
c_min                     = 2146100480          
mru_hits                  = 343855              
demand_data_misses        = 2615                
duplicate_buffers         = 0                   
demand_data_hits          = 1080594             
l2_free_on_write          = 0                   
l2_feeds                  = 0                   
misses                    = 20064               
evict_skip                = 1214                
evict_l2_eligible         = 71168               
p                         = 33800736768         
l2_compress_successes     = 0                   
recycle_miss              = 0                   
l2_writes_done            = 0                   
----------

Thu Dec  4 16:15:57 IST 2014

l2_size                   = 0                   
other_size                = 17931600            
c_max                     = 67601473536         
hash_elements             = 44425               
duplicate_reads           = 0                   
prefetch_data_hits        = 537                 
l2_read_bytes             = 0                   
demand_metadata_misses    = 12532               
prefetch_data_misses      = 1540                
hash_chain_max            = 2                   
l2_hits                   = 0                   
l2_compress_failures      = 0                   
mfu_ghost_hits            = 0                   
l2_rw_clash               = 0                   
memory_throttle_count     = 0                   
crtime                    = 4.537754192         
arc_meta_used             = 100027776           
hash_elements_max         = 46675               
size                      = 5889137024          
hits                      = 1123794             
mfu_hits                  = 768068              
prefetch_metadata_hits    = 10297               
hash_chains               = 102                 
evict_l2_cached           = 0                   
l2_writes_sent            = 0                   
l2_writes_error           = 0                   
deleted                   = 15                  
hdr_size                  = 12560944            
l2_compress_zeros         = 0                   
l2_abort_lowmem           = 0                   
l2_io_error               = 0                   
l2_hdr_size               = 0                   
l2_cksum_bad              = 0                   
l2_evict_reading          = 0                   
arc_meta_limit            = 16900368384         
prefetch_metadata_misses  = 3381                
l2_evict_lock_retry       = 0                   
hash_collisions           = 1137                
l2_writes_hdr_miss        = 0                   
evict_l2_ineligible       = 4096                
l2_write_bytes            = 0                   
data_size                 = 5858644480          
c                         = 67601473536         
l2_asize                  = 0                   
class                     = misc                
mutex_miss                = 0                   
snaptime                  = 1131.040711699      
duplicate_buffers_size    = 0                   
l2_misses                 = 0                   
mru_ghost_hits            = 0                   
demand_metadata_hits      = 29170               
arc_meta_max              = 99994208            
c_min                     = 2146100480          
mru_hits                  = 344892              
demand_data_misses        = 2615                
duplicate_buffers         = 0                   
demand_data_hits          = 1083790             
l2_free_on_write          = 0                   
l2_feeds                  = 0                   
misses                    = 20068               
evict_skip                = 1214                
evict_l2_eligible         = 71168               
p                         = 33800736768         
l2_compress_successes     = 0                   
recycle_miss              = 0                   
l2_writes_done            = 0                   
----------

Thu Dec  4 16:16:00 IST 2014

l2_size                   = 0                   
other_size                = 17803560            
c_max                     = 67601473536         
hash_elements             = 42523               
duplicate_reads           = 0                   
prefetch_data_hits        = 537                 
l2_read_bytes             = 0                   
demand_metadata_misses    = 12532               
prefetch_data_misses      = 1540                
hash_chain_max            = 2                   
l2_hits                   = 0                   
l2_compress_failures      = 0                   
mfu_ghost_hits            = 0                   
l2_rw_clash               = 0                   
memory_throttle_count     = 0                   
crtime                    = 4.537754192         
arc_meta_used             = 99334488            
hash_elements_max         = 46675               
size                      = 5995791704          
hits                      = 1137123             
mfu_hits                  = 776941              
prefetch_metadata_hits    = 10297               
hash_chains               = 84                  
evict_l2_cached           = 0                   
l2_writes_sent            = 0                   
l2_writes_error           = 0                   
deleted                   = 15                  
hdr_size                  = 12560944            
l2_compress_zeros         = 0                   
l2_abort_lowmem           = 0                   
l2_io_error               = 0                   
l2_hdr_size               = 0                   
l2_cksum_bad              = 0                   
l2_evict_reading          = 0                   
arc_meta_limit            = 16900368384         
prefetch_metadata_misses  = 3381                
l2_evict_lock_retry       = 0                   
hash_collisions           = 1145                
l2_writes_hdr_miss        = 0                   
evict_l2_ineligible       = 4096                
l2_write_bytes            = 0                   
data_size                 = 5965427200          
c                         = 67601473536         
l2_asize                  = 0                   
class                     = misc                
mutex_miss                = 0                   
snaptime                  = 1134.044734783      
duplicate_buffers_size    = 0                   
l2_misses                 = 0                   
mru_ghost_hits            = 0                   
demand_metadata_hits      = 29171               
arc_meta_max              = 100042776           
c_min                     = 2146100480          
mru_hits                  = 349348              
demand_data_misses        = 2615                
duplicate_buffers         = 0                   
demand_data_hits          = 1097118             
l2_free_on_write          = 0                   
l2_feeds                  = 0                   
misses                    = 20068               
evict_skip                = 1214                
evict_l2_eligible         = 71168               
p                         = 33800736768         
l2_compress_successes     = 0                   
recycle_miss              = 0                   
l2_writes_done            = 0                   
----------

Thu Dec  4 16:16:03 IST 2014

l2_size                   = 0                   
other_size                = 18005000            
c_max                     = 67601473536         
hash_elements             = 40668               
duplicate_reads           = 0                   
prefetch_data_hits        = 537                 
l2_read_bytes             = 0                   
demand_metadata_misses    = 12540               
prefetch_data_misses      = 1540                
hash_chain_max            = 2                   
l2_hits                   = 0                   
l2_compress_failures      = 0                   
mfu_ghost_hits            = 0                   
l2_rw_clash               = 0                   
memory_throttle_count     = 0                   
crtime                    = 4.537754192         
arc_meta_used             = 100146232           
hash_elements_max         = 46675               
size                      = 5979957304          
hits                      = 1157458             
mfu_hits                  = 791193              
prefetch_metadata_hits    = 10297               
hash_chains               = 85                  
evict_l2_cached           = 0                   
l2_writes_sent            = 0                   
l2_writes_error           = 0                   
deleted                   = 15                  
hdr_size                  = 12560944            
l2_compress_zeros         = 0                   
l2_abort_lowmem           = 0                   
l2_io_error               = 0                   
l2_hdr_size               = 0                   
l2_cksum_bad              = 0                   
l2_evict_reading          = 0                   
arc_meta_limit            = 16900368384         
prefetch_metadata_misses  = 3381                
l2_evict_lock_retry       = 0                   
hash_collisions           = 1162                
l2_writes_hdr_miss        = 0                   
evict_l2_ineligible       = 4096                
l2_write_bytes            = 0                   
data_size                 = 5949391360          
c                         = 67601473536         
l2_asize                  = 0                   
class                     = misc                
mutex_miss                = 0                   
snaptime                  = 1137.04862393       
duplicate_buffers_size    = 0                   
l2_misses                 = 0                   
mru_ghost_hits            = 0                   
demand_metadata_hits      = 29187               
arc_meta_max              = 100145632           
c_min                     = 2146100480          
mru_hits                  = 355431              
demand_data_misses        = 2615                
duplicate_buffers         = 0                   
demand_data_hits          = 1117437             
l2_free_on_write          = 0                   
l2_feeds                  = 0                   
misses                    = 20076               
evict_skip                = 1214                
evict_l2_eligible         = 71168               
p                         = 33800736768         
l2_compress_successes     = 0                   
recycle_miss              = 0                   
l2_writes_done            = 0                   
----------

Thu Dec  4 16:16:06 IST 2014

l2_size                   = 0                   
other_size                = 18052040            
c_max                     = 67601473536         
hash_elements             = 45931               
duplicate_reads           = 0                   
prefetch_data_hits        = 537                 
l2_read_bytes             = 0                   
demand_metadata_misses    = 12540               
prefetch_data_misses      = 1540                
hash_chain_max            = 2                   
l2_hits                   = 0                   
l2_compress_failures      = 0                   
mfu_ghost_hits            = 0                   
l2_rw_clash               = 0                   
memory_throttle_count     = 0                   
crtime                    = 4.537754192         
arc_meta_used             = 99615736            
hash_elements_max         = 46725               
size                      = 6264246264          
hits                      = 1168001             
mfu_hits                  = 798095              
prefetch_metadata_hits    = 10297               
hash_chains               = 98                  
evict_l2_cached           = 0                   
l2_writes_sent            = 0                   
l2_writes_error           = 0                   
deleted                   = 15                  
hdr_size                  = 12560944            
l2_compress_zeros         = 0                   
l2_abort_lowmem           = 0                   
l2_io_error               = 0                   
l2_hdr_size               = 0                   
l2_cksum_bad              = 0                   
l2_evict_reading          = 0                   
arc_meta_limit            = 16900368384         
prefetch_metadata_misses  = 3381                
l2_evict_lock_retry       = 0                   
hash_collisions           = 1192                
l2_writes_hdr_miss        = 0                   
evict_l2_ineligible       = 4096                
l2_write_bytes            = 0                   
data_size                 = 6233633280          
c                         = 67601473536         
l2_asize                  = 0                   
class                     = misc                
mutex_miss                = 0                   
snaptime                  = 1140.052771613      
duplicate_buffers_size    = 0                   
l2_misses                 = 0                   
mru_ghost_hits            = 0                   
demand_metadata_hits      = 29188               
arc_meta_max              = 100153872           
c_min                     = 2146100480          
mru_hits                  = 359072              
demand_data_misses        = 2615                
duplicate_buffers         = 0                   
demand_data_hits          = 1127979             
l2_free_on_write          = 0                   
l2_feeds                  = 0                   
misses                    = 20076               
evict_skip                = 1214                
evict_l2_eligible         = 71168               
p                         = 33800736768         
l2_compress_successes     = 0                   
recycle_miss              = 0                   
l2_writes_done            = 0                   
----------

Thu Dec  4 16:16:09 IST 2014

l2_size                   = 0                   
other_size                = 18144240            
c_max                     = 67601473536         
hash_elements             = 44399               
duplicate_reads           = 0                   
prefetch_data_hits        = 537                 
l2_read_bytes             = 0                   
demand_metadata_misses    = 12548               
prefetch_data_misses      = 1540                
hash_chain_max            = 2                   
l2_hits                   = 0                   
l2_compress_failures      = 0                   
mfu_ghost_hits            = 0                   
l2_rw_clash               = 0                   
memory_throttle_count     = 0                   
crtime                    = 4.537754192         
arc_meta_used             = 99752992            
hash_elements_max         = 47118               
size                      = 6250227744          
hits                      = 1186886             
mfu_hits                  = 811377              
prefetch_metadata_hits    = 10297               
hash_chains               = 155                 
evict_l2_cached           = 0                   
l2_writes_sent            = 0                   
l2_writes_error           = 0                   
deleted                   = 15                  
hdr_size                  = 12560944            
l2_compress_zeros         = 0                   
l2_abort_lowmem           = 0                   
l2_io_error               = 0                   
l2_hdr_size               = 0                   
l2_cksum_bad              = 0                   
l2_evict_reading          = 0                   
arc_meta_limit            = 16900368384         
prefetch_metadata_misses  = 3381                
l2_evict_lock_retry       = 0                   
hash_collisions           = 1302                
l2_writes_hdr_miss        = 0                   
evict_l2_ineligible       = 4096                
l2_write_bytes            = 0                   
data_size                 = 6219522560          
c                         = 67601473536         
l2_asize                  = 0                   
class                     = misc                
mutex_miss                = 0                   
snaptime                  = 1143.058190189      
duplicate_buffers_size    = 0                   
l2_misses                 = 0                   
mru_ghost_hits            = 0                   
demand_metadata_hits      = 29202               
arc_meta_max              = 100285688           
c_min                     = 2146100480          
mru_hits                  = 364675              
demand_data_misses        = 2615                
duplicate_buffers         = 0                   
demand_data_hits          = 1146850             
l2_free_on_write          = 0                   
l2_feeds                  = 0                   
misses                    = 20084               
evict_skip                = 1214                
evict_l2_eligible         = 71168               
p                         = 33800736768         
l2_compress_successes     = 0                   
recycle_miss              = 0                   
l2_writes_done            = 0                   
----------

Thu Dec  4 16:16:12 IST 2014

l2_size                   = 0                   
other_size                = 18053440            
c_max                     = 67601473536         
hash_elements             = 46005               
duplicate_reads           = 0                   
prefetch_data_hits        = 537                 
l2_read_bytes             = 0                   
demand_metadata_misses    = 12552               
prefetch_data_misses      = 1540                
hash_chain_max            = 2                   
l2_hits                   = 0                   
l2_compress_failures      = 0                   
mfu_ghost_hits            = 0                   
l2_rw_clash               = 0                   
memory_throttle_count     = 0                   
crtime                    = 4.537754192         
arc_meta_used             = 100254576           
hash_elements_max         = 47118               
size                      = 6189538160          
hits                      = 1194015             
mfu_hits                  = 815523              
prefetch_metadata_hits    = 10297               
hash_chains               = 129                 
evict_l2_cached           = 0                   
l2_writes_sent            = 0                   
l2_writes_error           = 0                   
deleted                   = 15                  
hdr_size                  = 12560944            
l2_compress_zeros         = 0                   
l2_abort_lowmem           = 0                   
l2_io_error               = 0                   
l2_hdr_size               = 0                   
l2_cksum_bad              = 0                   
l2_evict_reading          = 0                   
arc_meta_limit            = 16900368384         
prefetch_metadata_misses  = 3381                
l2_evict_lock_retry       = 0                   
hash_collisions           = 1311                
l2_writes_hdr_miss        = 0                   
evict_l2_ineligible       = 397312              
l2_write_bytes            = 0                   
data_size                 = 6158923776          
c                         = 6190066904          
l2_asize                  = 0                   
class                     = misc                
mutex_miss                = 0                   
snaptime                  = 1146.06243032       
duplicate_buffers_size    = 0                   
l2_misses                 = 0                   
mru_ghost_hits            = 0                   
demand_metadata_hits      = 29214               
arc_meta_max              = 100285688           
c_min                     = 2146100480          
mru_hits                  = 367658              
demand_data_misses        = 2615                
duplicate_buffers         = 0                   
demand_data_hits          = 1153967             
l2_free_on_write          = 0                   
l2_feeds                  = 0                   
misses                    = 20088               
evict_skip                = 1214                
evict_l2_eligible         = 1011712             
p                         = 3096114796          
l2_compress_successes     = 0                   
recycle_miss              = 1                   
l2_writes_done            = 0                   
----------

Thu Dec  4 16:16:15 IST 2014

l2_size                   = 0                   
other_size                = 17695320            
c_max                     = 67601473536         
hash_elements             = 44156               
duplicate_reads           = 0                   
prefetch_data_hits        = 537                 
l2_read_bytes             = 0                   
demand_metadata_misses    = 12556               
prefetch_data_misses      = 1540                
hash_chain_max            = 2                   
l2_hits                   = 0                   
l2_compress_failures      = 0                   
mfu_ghost_hits            = 0                   
l2_rw_clash               = 0                   
memory_throttle_count     = 0                   
crtime                    = 4.537754192         
arc_meta_used             = 99370392            
hash_elements_max         = 47845               
size                      = 6062509464          
hits                      = 1214370             
mfu_hits                  = 831061              
prefetch_metadata_hits    = 10297               
hash_chains               = 121                 
evict_l2_cached           = 0                   
l2_writes_sent            = 0                   
l2_writes_error           = 0                   
deleted                   = 15                  
hdr_size                  = 12583744            
l2_compress_zeros         = 0                   
l2_abort_lowmem           = 0                   
l2_io_error               = 0                   
l2_hdr_size               = 0                   
l2_cksum_bad              = 0                   
l2_evict_reading          = 0                   
arc_meta_limit            = 16900368384         
prefetch_metadata_misses  = 3381                
l2_evict_lock_retry       = 0                   
hash_collisions           = 1369                
l2_writes_hdr_miss        = 0                   
evict_l2_ineligible       = 132255744           
l2_write_bytes            = 0                   
data_size                 = 6032230400          
c                         = 6190066904          
l2_asize                  = 0                   
class                     = misc                
mutex_miss                = 1                   
snaptime                  = 1149.080481297      
duplicate_buffers_size    = 0                   
l2_misses                 = 0                   
mru_ghost_hits            = 0                   
demand_metadata_hits      = 29222               
arc_meta_max              = 100285688           
c_min                     = 2146100480          
mru_hits                  = 372475              
demand_data_misses        = 2615                
duplicate_buffers         = 0                   
demand_data_hits          = 1174314             
l2_free_on_write          = 0                   
l2_feeds                  = 0                   
misses                    = 20092               
evict_skip                = 1214                
evict_l2_eligible         = 285229056           
p                         = 3401023084          
l2_compress_successes     = 0                   
recycle_miss              = 219                 
l2_writes_done            = 0                   
----------

Thu Dec  4 16:16:18 IST 2014

l2_size                   = 0                   
other_size                = 17003120            
c_max                     = 67601473536         
hash_elements             = 49196               
duplicate_reads           = 0                   
prefetch_data_hits        = 537                 
l2_read_bytes             = 0                   
demand_metadata_misses    = 12556               
prefetch_data_misses      = 1540                
hash_chain_max            = 2                   
l2_hits                   = 0                   
l2_compress_failures      = 0                   
mfu_ghost_hits            = 0                   
l2_rw_clash               = 0                   
memory_throttle_count     = 0                   
crtime                    = 4.537754192         
arc_meta_used             = 97914576            
hash_elements_max         = 49196               
size                      = 5597583056          
hits                      = 1214720             
mfu_hits                  = 831312              
prefetch_metadata_hits    = 10297               
hash_chains               = 158                 
evict_l2_cached           = 0                   
l2_writes_sent            = 0                   
l2_writes_error           = 0                   
deleted                   = 15                  
hdr_size                  = 11820128            
l2_compress_zeros         = 0                   
l2_abort_lowmem           = 0                   
l2_io_error               = 0                   
l2_hdr_size               = 0                   
l2_cksum_bad              = 0                   
l2_evict_reading          = 0                   
arc_meta_limit            = 16900368384         
prefetch_metadata_misses  = 3381                
l2_evict_lock_retry       = 0                   
hash_collisions           = 1414                
l2_writes_hdr_miss        = 0                   
evict_l2_ineligible       = 132255744           
l2_write_bytes            = 0                   
data_size                 = 5568759808          
c                         = 5859249032          
l2_asize                  = 0                   
class                     = misc                
mutex_miss                = 1                   
snaptime                  = 1152.093743051      
duplicate_buffers_size    = 0                   
l2_misses                 = 0                   
mru_ghost_hits            = 0                   
demand_metadata_hits      = 29222               
arc_meta_max              = 100285688           
c_min                     = 2146100480          
mru_hits                  = 372574              
demand_data_misses        = 2615                
duplicate_buffers         = 0                   
demand_data_hits          = 1174664             
l2_free_on_write          = 0                   
l2_feeds                  = 0                   
misses                    = 20092               
evict_skip                = 1214                
evict_l2_eligible         = 285491200           
p                         = 3295003257          
l2_compress_successes     = 0                   
recycle_miss              = 219                 
l2_writes_done            = 0                   
----------

Thu Dec  4 16:16:21 IST 2014

l2_size                   = 0                   
other_size                = 15860920            
c_max                     = 67601473536         
hash_elements             = 44130               
duplicate_reads           = 0                   
prefetch_data_hits        = 537                 
l2_read_bytes             = 0                   
demand_metadata_misses    = 12564               
prefetch_data_misses      = 1540                
hash_chain_max            = 2                   
l2_hits                   = 0                   
l2_compress_failures      = 0                   
mfu_ghost_hits            = 0                   
l2_rw_clash               = 0                   
memory_throttle_count     = 0                   
crtime                    = 4.537754192         
arc_meta_used             = 91144424            
hash_elements_max         = 49670               
size                      = 4804212968          
hits                      = 1244167             
mfu_hits                  = 851345              
prefetch_metadata_hits    = 10297               
hash_chains               = 102                 
evict_l2_cached           = 0                   
l2_writes_sent            = 0                   
l2_writes_error           = 0                   
deleted                   = 1063                
hdr_size                  = 11332144            
l2_compress_zeros         = 0                   
l2_abort_lowmem           = 0                   
l2_io_error               = 0                   
l2_hdr_size               = 0                   
l2_cksum_bad              = 0                   
l2_evict_reading          = 0                   
arc_meta_limit            = 16900368384         
prefetch_metadata_misses  = 3381                
l2_evict_lock_retry       = 0                   
hash_collisions           = 1433                
l2_writes_hdr_miss        = 0                   
evict_l2_ineligible       = 137154560           
l2_write_bytes            = 0                   
data_size                 = 4777019904          
c                         = 4879960441          
l2_asize                  = 0                   
class                     = misc                
mutex_miss                = 9                   
snaptime                  = 1155.099409293      
duplicate_buffers_size    = 0                   
l2_misses                 = 0                   
mru_ghost_hits            = 24                  
demand_metadata_hits      = 29238               
arc_meta_max              = 100285688           
c_min                     = 2146100480          
mru_hits                  = 381988              
demand_data_misses        = 2639                
duplicate_buffers         = 0                   
demand_data_hits          = 1204095             
l2_free_on_write          = 0                   
l2_feeds                  = 0                   
misses                    = 20124               
evict_skip                = 1214                
evict_l2_eligible         = 1330413056          
p                         = 3274886866          
l2_compress_successes     = 0                   
recycle_miss              = 519                 
l2_writes_done            = 0                   
----------

Thu Dec  4 16:16:24 IST 2014

l2_size                   = 0                   
other_size                = 15703800            
c_max                     = 67601473536         
hash_elements             = 41125               
duplicate_reads           = 0                   
prefetch_data_hits        = 537                 
l2_read_bytes             = 0                   
demand_metadata_misses    = 12572               
prefetch_data_misses      = 1540                
hash_chain_max            = 2                   
l2_hits                   = 0                   
l2_compress_failures      = 0                   
mfu_ghost_hits            = 0                   
l2_rw_clash               = 0                   
memory_throttle_count     = 0                   
crtime                    = 4.537754192         
arc_meta_used             = 85702920            
hash_elements_max         = 49670               
size                      = 4396642568          
hits                      = 1257727             
mfu_hits                  = 860930              
prefetch_metadata_hits    = 10297               
hash_chains               = 92                  
evict_l2_cached           = 0                   
l2_writes_sent            = 0                   
l2_writes_error           = 0                   
deleted                   = 3379                
hdr_size                  = 10493456            
l2_compress_zeros         = 0                   
l2_abort_lowmem           = 0                   
l2_io_error               = 0                   
l2_hdr_size               = 0                   
l2_cksum_bad              = 0                   
l2_evict_reading          = 0                   
arc_meta_limit            = 16900368384         
prefetch_metadata_misses  = 3383                
l2_evict_lock_retry       = 0                   
hash_collisions           = 1442                
l2_writes_hdr_miss        = 0                   
evict_l2_ineligible       = 142102528           
l2_write_bytes            = 0                   
data_size                 = 4370445312          
c                         = 4396690358          
l2_asize                  = 0                   
class                     = misc                
mutex_miss                = 9                   
snaptime                  = 1158.10348092       
duplicate_buffers_size    = 0                   
l2_misses                 = 0                   
mru_ghost_hits            = 56                  
demand_metadata_hits      = 29252               
arc_meta_max              = 100285688           
c_min                     = 2146100480          
mru_hits                  = 385963              
demand_data_misses        = 2669                
duplicate_buffers         = 0                   
demand_data_hits          = 1217641             
l2_free_on_write          = 0                   
l2_feeds                  = 0                   
misses                    = 20164               
evict_skip                = 1214                
evict_l2_eligible         = 1491629056          
p                         = 2997814168          
l2_compress_successes     = 0                   
recycle_miss              = 821                 
l2_writes_done            = 0                   
----------

Thu Dec  4 16:16:27 IST 2014

l2_size                   = 0                   
other_size                = 14938800            
c_max                     = 67601473536         
hash_elements             = 39052               
duplicate_reads           = 0                   
prefetch_data_hits        = 537                 
l2_read_bytes             = 0                   
demand_metadata_misses    = 12576               
prefetch_data_misses      = 1540                
hash_chain_max            = 2                   
l2_hits                   = 0                   
l2_compress_failures      = 0                   
mfu_ghost_hits            = 0                   
l2_rw_clash               = 0                   
memory_throttle_count     = 0                   
crtime                    = 4.537754192         
arc_meta_used             = 80821952            
hash_elements_max         = 49670               
size                      = 3828283072          
hits                      = 1264949             
mfu_hits                  = 864148              
prefetch_metadata_hits    = 10297               
hash_chains               = 75                  
evict_l2_cached           = 0                   
l2_writes_sent            = 0                   
l2_writes_error           = 0                   
deleted                   = 10138               
hdr_size                  = 8953872             
l2_compress_zeros         = 0                   
l2_abort_lowmem           = 0                   
l2_io_error               = 0                   
l2_hdr_size               = 0                   
l2_cksum_bad              = 0                   
l2_evict_reading          = 0                   
arc_meta_limit            = 16900368384         
prefetch_metadata_misses  = 3383                
l2_evict_lock_retry       = 0                   
hash_collisions           = 1454                
l2_writes_hdr_miss        = 0                   
evict_l2_ineligible       = 144691200           
l2_write_bytes            = 0                   
data_size                 = 3804390400          
c                         = 3830309700          
l2_asize                  = 0                   
class                     = misc                
mutex_miss                = 12                  
snaptime                  = 1161.109480472      
duplicate_buffers_size    = 0                   
l2_misses                 = 0                   
mru_ghost_hits            = 77                  
demand_metadata_hits      = 29256               
arc_meta_max              = 100285688           
c_min                     = 2146100480          
mru_hits                  = 389967              
demand_data_misses        = 2692                
duplicate_buffers         = 0                   
demand_data_hits          = 1224859             
l2_free_on_write          = 0                   
l2_feeds                  = 0                   
misses                    = 20191               
evict_skip                = 1214                
evict_l2_eligible         = 1951429632          
p                         = 2764099257          
l2_compress_successes     = 0                   
recycle_miss              = 972                 
l2_writes_done            = 0                   
----------

From jklimek at gmail.com  Thu Dec  4 18:12:54 2014
From: jklimek at gmail.com (John Klimek)
Date: Thu, 4 Dec 2014 13:12:54 -0500
Subject: [OmniOS-discuss] NFSv4 id mapping only working on client but not
	server?
Message-ID: <CAOEv1kbHMCaDvotg84i7M_pOzVJuOCuquVtFDpO84GJTUpTYkQ@mail.gmail.com>

I'm running OmniOS (v5.11, omnios-10b9c79, September 2014) as my server and
Ubuntu 14.04 as my client.  I'm only using sys (?) authentication and not
Kerberos.

I've correctly set my NFS domain name but it's only partially working.

Here's an example:

Server:  user 'media', uid 1000.
Client:  user 'media', uid 2000.

If I create a file as 'media' on the server, the client correctly maps the
id to the local 'media' user.  However, if I create a file on the client,
the server will show a uid and doesn't correctly map to the local 'media'
uid.

What can I have setup wrong?

I can't find any debugging or logging options for nfsmapid...

Also, does the uidmap and gidmap share.nfs options only work for NFSv3?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141204/33e2b80f/attachment.html>

From jim at cos.ru  Mon Dec  8 13:29:18 2014
From: jim at cos.ru (Jim Klimov)
Date: Mon, 08 Dec 2014 14:29:18 +0100
Subject: [OmniOS-discuss] Bug in omnios packaging
Message-ID: <fc775c566949.5485b5be@cos.ru>


 Hi all,




 Found a small issue in some of the omnios (ms.omniti.com) packages: they include symlinks that point to the build-host's absolute paths, i.e.:




 # find /opt/omni -ls | grep tmp
  12295    0 lrwxrwxrwx   1 root     root           66 Dec  8 13:39 /opt/omni/bin/pbzcat -> /tmp/build_esproul/omniti_compress_pbzip2_pkg//opt/omni/bin/pbzip2
  12296    0 lrwxrwxrwx   1 root     root           66 Dec  8 13:39 /opt/omni/bin/pbunzip2 -> /tmp/build_esproul/omniti_compress_pbzip2_pkg//opt/omni/bin/pbzip2
  12302    0 lrwxrwxrwx   1 root     root           68 Dec  8 13:39 /opt/omni/bin/amd64/unpigz -> /tmp/build_esproul/omniti_compress_pigz_pkg//opt/omni/bin/amd64/pigz




 Cheers,

 //Jim
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141208/1a96405d/attachment.html>

From eric.sproul at circonus.com  Mon Dec  8 15:24:00 2014
From: eric.sproul at circonus.com (Eric Sproul)
Date: Mon, 8 Dec 2014 10:24:00 -0500
Subject: [OmniOS-discuss] Bug in omnios packaging
In-Reply-To: <fc775c566949.5485b5be@cos.ru>
References: <fc775c566949.5485b5be@cos.ru>
Message-ID: <CAO8hXRDDpjN_vrK0_25w-H7gn3o+SEzprGVgAkzFqyVgD33pxA@mail.gmail.com>

On Mon, Dec 8, 2014 at 8:29 AM, Jim Klimov <jim at cos.ru> wrote:
> Hi all,
>
>
> Found a small issue in some of the omnios (ms.omniti.com) packages: they
> include symlinks that point to the build-host's absolute paths, i.e.:
>
>
> # find /opt/omni -ls | grep tmp
>  12295    0 lrwxrwxrwx   1 root     root           66 Dec  8 13:39
> /opt/omni/bin/pbzcat ->
> /tmp/build_esproul/omniti_compress_pbzip2_pkg//opt/omni/bin/pbzip2
>  12296    0 lrwxrwxrwx   1 root     root           66 Dec  8 13:39
> /opt/omni/bin/pbunzip2 ->
> /tmp/build_esproul/omniti_compress_pbzip2_pkg//opt/omni/bin/pbzip2
>  12302    0 lrwxrwxrwx   1 root     root           68 Dec  8 13:39
> /opt/omni/bin/amd64/unpigz ->
> /tmp/build_esproul/omniti_compress_pigz_pkg//opt/omni/bin/amd64/pigz

Haha... busted!  Thanks for the bug report.  For whatever reason,
these compression tools all have very non-standard build procedures,
and make all sorts of silly assumptions.  We'll get it fixed.

Eric

From danmcd at omniti.com  Mon Dec  8 15:52:48 2014
From: danmcd at omniti.com (Dan McDonald)
Date: Mon, 8 Dec 2014 10:52:48 -0500
Subject: [OmniOS-discuss] Timezones
In-Reply-To: <94A534D5-D8BD-4426-B0AF-90E6D48F6802@omniti.com>
References: <6E7D35D0-98DC-4B63-8683-A861AC1A6D05@cos.ru>
	<94A534D5-D8BD-4426-B0AF-90E6D48F6802@omniti.com>
Message-ID: <88FF83DA-E619-490C-94E2-C37173BEAC8B@omniti.com>


> On Dec 8, 2014, at 9:22 AM, Dan McDonald <danmcd at omniti.com> wrote:
> 
> Updating the zoneinfo is highly annoying, and the README is wrong - if you look in the Illumos source (I upstreamed this too), it's up to one of the 2014 updates.
> 
> Thanks for the README catch.  When 014 gets closer to release, I'll be updating the zoneinfo again.

The annoying parts of all of this are documented in $ILLUMOS_GATE/usr/src/cmd/zic/README.illumos

Couldn't remember that until I had my workstation in front of me.

Dan


From chip at innovates.com  Mon Dec  8 16:09:02 2014
From: chip at innovates.com (Schweiss, Chip)
Date: Mon, 8 Dec 2014 10:09:02 -0600
Subject: [OmniOS-discuss] Parity error on path mpt_sas2
In-Reply-To: <3BE0DEED8863E5429BAE4CAEDF624565039C1B0DACD1@AIRA-SRV.aira.local>
References: <3BE0DEED8863E5429BAE4CAEDF624565039C1B0DAAD0@AIRA-SRV.aira.local>
	<1439760B-8D8B-40F9-8A62-4CFE4A9483C1@omniti.com>
	<3BE0DEED8863E5429BAE4CAEDF624565039C1B0DAADA@AIRA-SRV.aira.local>
	<CAOqBcP_12FaH77o+3nrGVm7KU3x1XCk3B-9uenKRRL63t5wdyg@mail.gmail.com>
	<3BE0DEED8863E5429BAE4CAEDF624565039C1B0DACD1@AIRA-SRV.aira.local>
Message-ID: <CALeZrrRhsV5i2AJF2aRrLxQvR9LSckiEax59gX5XXAYAkpCS+Q@mail.gmail.com>

I've got some new LSI HBAs I'm trying to downgrade from firmware version 20
to 18.

I'm getting errors when trying to downgrade:

Attempting to flash firmware to LSI SAS SAS2308_2(D1) :

        Executing Operation: Flash Firmware Image

                Firmware Image has a Valid Checksum.
                Firmware Version 18.00.00.00
                Firmware Image compatible with Controller.

                Valid NVDATA Image found.
                NVDATA Version 11.00.00.00
                Checking for a compatible NVData image...

                NVDATA Device ID and Chip Revision match verified.
                ERROR: Cannot downgrade NVDATA version 14.00.00.06
                       to 11.00.110000.00.

                ERROR: Failed to get valid NVDATA image from File!

                Firmware Image Validation Failed!

Tried downgrading bios:

Attempting to flash Boot Service to LSI SAS SAS2308_2(D1) :

                Validating BIOS Image...

                BIOS Header Signature is Valid

                BIOS Image has a Valid Checksum.

                BIOS PCI Structure Signature Valid.

                BIOS Image Compatible with the SAS Controller.

                Attempting to Flash BIOS Image...

                BIOS Version in flash    : 07.39.00.00
                BIOS Version from File  : 07.35.00.00
                Skipping flash since file version is not greater than
existing.

                Flash BIOS Image Failed!


I've tried sas2flash from P20 and P18.

Can someone fill me in on the trick to downgrade?

-Chip


On Thu, Nov 27, 2014 at 9:03 AM, Filip Marvan <filip.marvan at aira.cz> wrote:

> Thank you for your help Aaron! I downgraded firmware to P18 and there are
> no more errors today.
>
> It seems, that there is something bad with P20 firmware!
>
>
>
> You saved me a lot of time J
>
>
>
> Filip
>
>
>
>
>
> *From:* Aaron Curry [mailto:asc1111 at gmail.com]
> *Sent:* Tuesday, November 25, 2014 5:46 PM
> *To:* Filip Marvan
> *Cc:* Dan McDonald; omnios-discuss at lists.omniti.com
> *Subject:* Re: [OmniOS-discuss] Parity error on path mpt_sas2
>
>
>
> I had this exact same problem recently when setting up a new home
> server... same controller, same firmware (P20). The errors were on all
> disks attached to the controller, but only on high read activity. Writes
> did not generate the errors. I downgraded the firmware to P18 (which is
> what we are using at work) and the errors went away. Has anyone had success
> with this controller and the P20 firmware?
>
>
>
> Aaron
>
>
>
> On Tue, Nov 25, 2014 at 8:30 AM, Filip Marvan <filip.marvan at aira.cz>
> wrote:
>
> Hi Dan,
>
> thanks for reply.
> Yes, errors are on all 5 disks in the same RAIDZ pool. No problem on other
> disks in different pool but on the same SAS cable.
> So maybe I only had bad luck with that disks.
>
> Filip
>
>
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>
>
>
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141208/e69d42b5/attachment-0001.html>

From matthew.lagoe at subrigo.net  Mon Dec  8 16:15:08 2014
From: matthew.lagoe at subrigo.net (Matthew Lagoe)
Date: Mon, 8 Dec 2014 08:15:08 -0800
Subject: [OmniOS-discuss] Parity error on path mpt_sas2
In-Reply-To: <CALeZrrRhsV5i2AJF2aRrLxQvR9LSckiEax59gX5XXAYAkpCS+Q@mail.gmail.com>
References: <3BE0DEED8863E5429BAE4CAEDF624565039C1B0DAAD0@AIRA-SRV.aira.local>	<1439760B-8D8B-40F9-8A62-4CFE4A9483C1@omniti.com>	<3BE0DEED8863E5429BAE4CAEDF624565039C1B0DAADA@AIRA-SRV.aira.local>	<CAOqBcP_12FaH77o+3nrGVm7KU3x1XCk3B-9uenKRRL63t5wdyg@mail.gmail.com>	<3BE0DEED8863E5429BAE4CAEDF624565039C1B0DACD1@AIRA-SRV.aira.local>
	<CALeZrrRhsV5i2AJF2aRrLxQvR9LSckiEax59gX5XXAYAkpCS+Q@mail.gmail.com>
Message-ID: <004501d01302$257c9b90$7075d2b0$@subrigo.net>

I think you have to use a special flag to force it

 

From: OmniOS-discuss [mailto:omnios-discuss-bounces at lists.omniti.com] On Behalf Of Schweiss, Chip
Sent: Monday, December 08, 2014 08:09 AM
To: Filip Marvan
Cc: omnios-discuss at lists.omniti.com
Subject: Re: [OmniOS-discuss] Parity error on path mpt_sas2

 

I've got some new LSI HBAs I'm trying to downgrade from firmware version 20 to 18.

I'm getting errors when trying to downgrade:

Attempting to flash firmware to LSI SAS SAS2308_2(D1) :

        Executing Operation: Flash Firmware Image

                Firmware Image has a Valid Checksum.
                Firmware Version 18.00.00.00
                Firmware Image compatible with Controller.

                Valid NVDATA Image found.
                NVDATA Version 11.00.00.00
                Checking for a compatible NVData image...

                NVDATA Device ID and Chip Revision match verified.
                ERROR: Cannot downgrade NVDATA version 14.00.00.06
                       to 11.00.110000.00.

                ERROR: Failed to get valid NVDATA image from File!

                Firmware Image Validation Failed!

Tried downgrading bios:

Attempting to flash Boot Service to LSI SAS SAS2308_2(D1) :

                Validating BIOS Image...

                BIOS Header Signature is Valid

                BIOS Image has a Valid Checksum.

                BIOS PCI Structure Signature Valid.

                BIOS Image Compatible with the SAS Controller.

                Attempting to Flash BIOS Image...

                BIOS Version in flash    : 07.39.00.00
                BIOS Version from File  : 07.35.00.00
                Skipping flash since file version is not greater than existing.

                Flash BIOS Image Failed!



I've tried sas2flash from P20 and P18.   

Can someone fill me in on the trick to downgrade?  

-Chip

 

 

On Thu, Nov 27, 2014 at 9:03 AM, Filip Marvan <filip.marvan at aira.cz> wrote:

Thank you for your help Aaron! I downgraded firmware to P18 and there are no more errors today.

It seems, that there is something bad with P20 firmware!

 

You saved me a lot of time J

 

Filip

 

 

From: Aaron Curry [mailto:asc1111 at gmail.com] 
Sent: Tuesday, November 25, 2014 5:46 PM
To: Filip Marvan
Cc: Dan McDonald; omnios-discuss at lists.omniti.com
Subject: Re: [OmniOS-discuss] Parity error on path mpt_sas2

 

I had this exact same problem recently when setting up a new home server... same controller, same firmware (P20). The errors were on all disks attached to the controller, but only on high read activity. Writes did not generate the errors. I downgraded the firmware to P18 (which is what we are using at work) and the errors went away. Has anyone had success with this controller and the P20 firmware?

 

Aaron

 

On Tue, Nov 25, 2014 at 8:30 AM, Filip Marvan <filip.marvan at aira.cz> wrote:

Hi Dan,

thanks for reply.
Yes, errors are on all 5 disks in the same RAIDZ pool. No problem on other
disks in different pool but on the same SAS cable.
So maybe I only had bad luck with that disks.

Filip


_______________________________________________
OmniOS-discuss mailing list
OmniOS-discuss at lists.omniti.com
http://lists.omniti.com/mailman/listinfo/omnios-discuss

 


_______________________________________________
OmniOS-discuss mailing list
OmniOS-discuss at lists.omniti.com
http://lists.omniti.com/mailman/listinfo/omnios-discuss

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141208/56289266/attachment.html>

From ikaufman at eng.ucsd.edu  Mon Dec  8 16:54:10 2014
From: ikaufman at eng.ucsd.edu (Ian Kaufman)
Date: Mon, 8 Dec 2014 08:54:10 -0800
Subject: [OmniOS-discuss] NFSv4 id mapping only working on client but
	not server?
In-Reply-To: <CAOEv1kYze9=As8jtTYAG88O+0hn0ECvLGrY9_HiLZdGu1-23uA@mail.gmail.com>
References: <CAOEv1kYdT=Cj-K7KarcEFizXKe8g8aehKm5uaHORqfqzAxUFoA@mail.gmail.com>
	<20141204193711.03524183@sleipner.datanom.net>
	<CAOEv1kaoxj=AREpfuRo8saHFhW18i0CmgSd9dfwLZEtCUv8pmw@mail.gmail.com>
	<20141207224353.GA13916@warlock.deepthought.com>
	<CAOEv1kYze9=As8jtTYAG88O+0hn0ECvLGrY9_HiLZdGu1-23uA@mail.gmail.com>
Message-ID: <CAPJtH1gTA9z4got9Ou=Zc9BxjmULGq3-NhF=50YgMT6Vqx3n8w@mail.gmail.com>

No, what we are saying is NFSv4 and RPC are not compatible right now,
and thus AUTH_SYS/AUTH_UNIX will not map UIDs by name, but by number
over RPC. If you are not going to use Kerberos or LDAP, then you need
to sync UIDs/UIDNumber in /etc/passwd. It's not that Solaris and Linux
are incompatible, it's that RPC does not support NFSv4.

Ian

On Mon, Dec 8, 2014 at 5:25 AM, John Klimek <jklimek at gmail.com> wrote:
> Thanks everybody.
>
> Are you guys saying that Solaris and Linux are incompatible for sharing
> NFSv4 ACLs (because of RPC differences, etc) ?
>
> On Sun, Dec 7, 2014 at 5:43 PM, Frank Pittel <fwp at deepthought.com> wrote:
>>
>> I've tried doing this with NFS4 a couple of years ago and what I found was
>> that
>> while the UID mapping worked for things like "ls" anytime you tried to
>> actually
>> do anything with the file RPC calls were made and since that's outside of
>> NFS they
>> failed.
>>
>> It was a couple of years ago now since I went through it but the upshot is
>> that
>> at least then uid mapping didn't work in any meaningful way!
>>
>> The Other Frank
>>
>>
>>
>> On Thu, Dec 04, 2014 at 01:55:11PM -0500, John Klimek wrote:
>> > Thanks, I will give this a try.
>> >
>> > However, when I create a file if I check the acl on Omni, it does have
>> > extended permissions and looks correct.  It's only the uid and gid that
>> > I'm
>> > trying to get mapped correctly.
>> >
>> > Do you still think that noacl will solve that problem?
>> >
>> > (I'm migrating some virtual machines so I can't try out the setting you
>> > mentioned until a few more minutes...)
>> >
>> > On Thu, Dec 4, 2014 at 1:37 PM, Michael Rasmussen <mir at miras.org> wrote:
>> >
>> > > On Thu, 4 Dec 2014 13:20:04 -0500
>> > > John Klimek <jklimek at gmail.com> wrote:
>> > >
>> > > >
>> > > > What can I have setup wrong?
>> > > >
>> > > > I can't find any debugging or logging options for nfsmapid...
>> > > >
>> > > > Also, does the uidmap and gidmap share.nfs options only work for
>> > > > NFSv3?
>> > > Solaris and derivatives implementation of NFS ACL is not compliant to
>> > > the
>> > > Linux NFS ACL.
>> > > More directly it is the ACL in Linux which is not POSIX conformant so
>> > > to
>> > > avoid problems you
>> > > should add the mount option noacl in your Linux fstab file. Noacl will
>> > > instruct Omnios NFS
>> > > to revert to plain old uid/gid.
>> > >
>> > > --
>> > > Hilsen/Regards
>> > > Michael Rasmussen
>> > >
>> > > Get my public GnuPG keys:
>> > > michael <at> rasmussen <dot> cc
>> > > http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E
>> > > mir <at> datanom <dot> net
>> > > http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C
>> > > mir <at> miras <dot> org
>> > > http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917
>> > > --------------------------------------------------------------
>> > > /usr/games/fortune -es says:
>> > > Don't just echo the code with comments - make every comment count.
>> > >             - The Elements of Programming Style (Kernighan & Plaugher)
>> > >
>> > > _______________________________________________
>> > > OmniOS-discuss mailing list
>> > > OmniOS-discuss at lists.omniti.com
>> > > http://lists.omniti.com/mailman/listinfo/omnios-discuss
>> > >
>> > >
>>
>> > _______________________________________________
>> > OmniOS-discuss mailing list
>> > OmniOS-discuss at lists.omniti.com
>> > http://lists.omniti.com/mailman/listinfo/omnios-discuss
>>
>
>
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>



-- 
Ian Kaufman
Research Systems Administrator
UC San Diego, Jacobs School of Engineering ikaufman AT ucsd DOT edu

From tim at multitalents.net  Mon Dec  8 18:05:34 2014
From: tim at multitalents.net (Tim Rice)
Date: Mon, 8 Dec 2014 10:05:34 -0800 (PST)
Subject: [OmniOS-discuss] Parity error on path mpt_sas2
In-Reply-To: <CALeZrrRhsV5i2AJF2aRrLxQvR9LSckiEax59gX5XXAYAkpCS+Q@mail.gmail.com>
References: <3BE0DEED8863E5429BAE4CAEDF624565039C1B0DAAD0@AIRA-SRV.aira.local>
	<1439760B-8D8B-40F9-8A62-4CFE4A9483C1@omniti.com>
	<3BE0DEED8863E5429BAE4CAEDF624565039C1B0DAADA@AIRA-SRV.aira.local>
	<CAOqBcP_12FaH77o+3nrGVm7KU3x1XCk3B-9uenKRRL63t5wdyg@mail.gmail.com>
	<3BE0DEED8863E5429BAE4CAEDF624565039C1B0DACD1@AIRA-SRV.aira.local>
	<CALeZrrRhsV5i2AJF2aRrLxQvR9LSckiEax59gX5XXAYAkpCS+Q@mail.gmail.com>
Message-ID: <alpine.UW2.2.11.1412080955080.21692@server01.int.multitalents.net>

On Mon, 8 Dec 2014, Schweiss, Chip wrote:

| I've got some new LSI HBAs I'm trying to downgrade from firmware version 20
| to 18.
| 
| I'm getting errors when trying to downgrade:

Try erasing first. On the 6G cards it is "sas2flsh -o -e 6".
Once erased, be sure to flash before rebooting or the card will
need to be sent back.

| Attempting to flash firmware to LSI SAS SAS2308_2(D1) :
| 
|         Executing Operation: Flash Firmware Image
| 
|                 Firmware Image has a Valid Checksum.
|                 Firmware Version 18.00.00.00
|                 Firmware Image compatible with Controller.
| 
|                 Valid NVDATA Image found.
|                 NVDATA Version 11.00.00.00
|                 Checking for a compatible NVData image...
| 
|                 NVDATA Device ID and Chip Revision match verified.
|                 ERROR: Cannot downgrade NVDATA version 14.00.00.06
|                        to 11.00.110000.00.
| 
|                 ERROR: Failed to get valid NVDATA image from File!
| 
|                 Firmware Image Validation Failed!
| 
| Tried downgrading bios:
| 
| Attempting to flash Boot Service to LSI SAS SAS2308_2(D1) :
| 
|                 Validating BIOS Image...
| 
|                 BIOS Header Signature is Valid
| 
|                 BIOS Image has a Valid Checksum.
| 
|                 BIOS PCI Structure Signature Valid.
| 
|                 BIOS Image Compatible with the SAS Controller.
| 
|                 Attempting to Flash BIOS Image...
| 
|                 BIOS Version in flash    : 07.39.00.00
|                 BIOS Version from File  : 07.35.00.00
|                 Skipping flash since file version is not greater than
| existing.
| 
|                 Flash BIOS Image Failed!
| 
| 
| I've tried sas2flash from P20 and P18.
| 
| Can someone fill me in on the trick to downgrade?
| 
| -Chip
| 
| 
| On Thu, Nov 27, 2014 at 9:03 AM, Filip Marvan <filip.marvan at aira.cz> wrote:
| 
| > Thank you for your help Aaron! I downgraded firmware to P18 and there are
| > no more errors today.
| >
| > It seems, that there is something bad with P20 firmware!
| >
| >
| >
| > You saved me a lot of time J
| >
| >
| >
| > Filip
| >
| >
| >
| >
| >
| > *From:* Aaron Curry [mailto:asc1111 at gmail.com]
| > *Sent:* Tuesday, November 25, 2014 5:46 PM
| > *To:* Filip Marvan
| > *Cc:* Dan McDonald; omnios-discuss at lists.omniti.com
| > *Subject:* Re: [OmniOS-discuss] Parity error on path mpt_sas2
| >
| >
| >
| > I had this exact same problem recently when setting up a new home
| > server... same controller, same firmware (P20). The errors were on all
| > disks attached to the controller, but only on high read activity. Writes
| > did not generate the errors. I downgraded the firmware to P18 (which is
| > what we are using at work) and the errors went away. Has anyone had success
| > with this controller and the P20 firmware?
| >
| >
| >
| > Aaron
| >
| >
| >
| > On Tue, Nov 25, 2014 at 8:30 AM, Filip Marvan <filip.marvan at aira.cz>
| > wrote:
| >
| > Hi Dan,
| >
| > thanks for reply.
| > Yes, errors are on all 5 disks in the same RAIDZ pool. No problem on other
| > disks in different pool but on the same SAS cable.
| > So maybe I only had bad luck with that disks.
| >
| > Filip
| >

-- 
Tim Rice				Multitalents
tim at multitalents.net



From eric.sproul at circonus.com  Mon Dec  8 18:17:13 2014
From: eric.sproul at circonus.com (Eric Sproul)
Date: Mon, 8 Dec 2014 13:17:13 -0500
Subject: [OmniOS-discuss] Bug in omnios packaging
In-Reply-To: <CAO8hXRDDpjN_vrK0_25w-H7gn3o+SEzprGVgAkzFqyVgD33pxA@mail.gmail.com>
References: <fc775c566949.5485b5be@cos.ru>
	<CAO8hXRDDpjN_vrK0_25w-H7gn3o+SEzprGVgAkzFqyVgD33pxA@mail.gmail.com>
Message-ID: <CAO8hXRDEBiEmOpGFF0=ac4Vd3d-7FCxfPtbDbKTKSPkEVj74tg@mail.gmail.com>

On Mon, Dec 8, 2014 at 10:24 AM, Eric Sproul <eric.sproul at circonus.com> wrote:
> Haha... busted!  Thanks for the bug report.  For whatever reason,
> these compression tools all have very non-standard build procedures,
> and make all sorts of silly assumptions.  We'll get it fixed.

The fix was to make the symlink target relative instead of absolute.

pkg://ms.omniti.com/omniti/compress/pbzip2 at 1.1.8,5.11-0.151006:20141208T180943Z
is the updated package.

Thanks again for pointing that out!
Eric

From henson at acm.org  Mon Dec  8 20:23:20 2014
From: henson at acm.org (Paul B. Henson)
Date: Mon, 8 Dec 2014 12:23:20 -0800
Subject: [OmniOS-discuss] NFSv4 id mapping only working on client
	but	not server?
In-Reply-To: <CAPJtH1gTA9z4got9Ou=Zc9BxjmULGq3-NhF=50YgMT6Vqx3n8w@mail.gmail.com>
References: <CAOEv1kYdT=Cj-K7KarcEFizXKe8g8aehKm5uaHORqfqzAxUFoA@mail.gmail.com>	<20141204193711.03524183@sleipner.datanom.net>	<CAOEv1kaoxj=AREpfuRo8saHFhW18i0CmgSd9dfwLZEtCUv8pmw@mail.gmail.com>	<20141207224353.GA13916@warlock.deepthought.com>	<CAOEv1kYze9=As8jtTYAG88O+0hn0ECvLGrY9_HiLZdGu1-23uA@mail.gmail.com>
	<CAPJtH1gTA9z4got9Ou=Zc9BxjmULGq3-NhF=50YgMT6Vqx3n8w@mail.gmail.com>
Message-ID: <674b01d01324$d0e28180$72a78480$@acm.org>

> From: Ian Kaufman
> Sent: Monday, December 08, 2014 8:54 AM
> 
> No, what we are saying is NFSv4 and RPC are not compatible right now,
> and thus AUTH_SYS/AUTH_UNIX will not map UIDs by name, but by number
> over RPC. If you are not going to use Kerberos or LDAP, then you need
> to sync UIDs/UIDNumber in /etc/passwd. It's not that Solaris and Linux
> are incompatible, it's that RPC does not support NFSv4.

Well, I don't think it's technically accurate to say NFSv4 and RPC are not
compatible, nor that RPC does not support NFSv4.

It's really more of an issue that the NFSv4 protocol failed to address ID
mapping completely for nonsecure RPC methods. For secure RPC (such as
Kerberos), identifiers are passed over the wire symbolically as strings, and
the ID mapper on each side translates them to numeric identifiers. However,
for insecure RPC using AUTH_SYS, NFSv4 continues to pass raw uid/gid
identifiers over the wire.

This limitation of the protocol renders ID mapping somewhat useless for
NFSv4 over insecure RPC :(. Ideally they could address it in v4.1 or v4.2 by
creating a new insecure AUTH_SYS_NAMES protocol, which would simply use
symbolic string identifiers over the wire like secure NFS does. I guess
there is not much interest or concern about this in the communities that
deal with NFS protocol specifications 8-/. Perhaps the majority of
deployments for people in decision-making capacity use secure NFS and thus
are not impacted by this deficiency...



From ikaufman at eng.ucsd.edu  Mon Dec  8 20:37:25 2014
From: ikaufman at eng.ucsd.edu (Ian Kaufman)
Date: Mon, 8 Dec 2014 12:37:25 -0800
Subject: [OmniOS-discuss] NFSv4 id mapping only working on client but
	not server?
In-Reply-To: <674b01d01324$d0e28180$72a78480$@acm.org>
References: <CAOEv1kYdT=Cj-K7KarcEFizXKe8g8aehKm5uaHORqfqzAxUFoA@mail.gmail.com>
	<20141204193711.03524183@sleipner.datanom.net>
	<CAOEv1kaoxj=AREpfuRo8saHFhW18i0CmgSd9dfwLZEtCUv8pmw@mail.gmail.com>
	<20141207224353.GA13916@warlock.deepthought.com>
	<CAOEv1kYze9=As8jtTYAG88O+0hn0ECvLGrY9_HiLZdGu1-23uA@mail.gmail.com>
	<CAPJtH1gTA9z4got9Ou=Zc9BxjmULGq3-NhF=50YgMT6Vqx3n8w@mail.gmail.com>
	<674b01d01324$d0e28180$72a78480$@acm.org>
Message-ID: <CAPJtH1h_Y_3rQm5-SNaUAPcnQxvJ4CKEUdni4kzmXQhuVhH8ZA@mail.gmail.com>

Yes, I oversimplified things. The issue is that AUTH_SYS/AUTH_UNIX
does not support NFSv4. AUTH_SYS uses the UID, not the name, so the
mapping fails. So, when RPC uses AUTH_SYS, NFSv4 is SOL.

http://thread.gmane.org/gmane.linux.nfsv4/7103/focus=7105

Supposedly. at least on the client side, this has been fixed somewhere
upstream. However, the server side is not.

https://bugzilla.linux-nfs.org/show_bug.cgi?id=226

Regardless, my point is that this is not a Solaris/Linux issue, as a
Linux server and Linux clients would be in the same boat.

On Mon, Dec 8, 2014 at 12:23 PM, Paul B. Henson <henson at acm.org> wrote:
>> From: Ian Kaufman
>> Sent: Monday, December 08, 2014 8:54 AM
>>
>> No, what we are saying is NFSv4 and RPC are not compatible right now,
>> and thus AUTH_SYS/AUTH_UNIX will not map UIDs by name, but by number
>> over RPC. If you are not going to use Kerberos or LDAP, then you need
>> to sync UIDs/UIDNumber in /etc/passwd. It's not that Solaris and Linux
>> are incompatible, it's that RPC does not support NFSv4.
>
> Well, I don't think it's technically accurate to say NFSv4 and RPC are not
> compatible, nor that RPC does not support NFSv4.
>
> It's really more of an issue that the NFSv4 protocol failed to address ID
> mapping completely for nonsecure RPC methods. For secure RPC (such as
> Kerberos), identifiers are passed over the wire symbolically as strings, and
> the ID mapper on each side translates them to numeric identifiers. However,
> for insecure RPC using AUTH_SYS, NFSv4 continues to pass raw uid/gid
> identifiers over the wire.
>
> This limitation of the protocol renders ID mapping somewhat useless for
> NFSv4 over insecure RPC :(. Ideally they could address it in v4.1 or v4.2 by
> creating a new insecure AUTH_SYS_NAMES protocol, which would simply use
> symbolic string identifiers over the wire like secure NFS does. I guess
> there is not much interest or concern about this in the communities that
> deal with NFS protocol specifications 8-/. Perhaps the majority of
> deployments for people in decision-making capacity use secure NFS and thus
> are not impacted by this deficiency...
>
>



-- 
Ian Kaufman
Research Systems Administrator
UC San Diego, Jacobs School of Engineering ikaufman AT ucsd DOT edu

From henson at acm.org  Mon Dec  8 21:42:32 2014
From: henson at acm.org (Paul B. Henson)
Date: Mon, 8 Dec 2014 13:42:32 -0800
Subject: [OmniOS-discuss] NFSv4 id mapping only working on client but
	not server?
In-Reply-To: <CAPJtH1h_Y_3rQm5-SNaUAPcnQxvJ4CKEUdni4kzmXQhuVhH8ZA@mail.gmail.com>
References: <CAOEv1kYdT=Cj-K7KarcEFizXKe8g8aehKm5uaHORqfqzAxUFoA@mail.gmail.com>
	<20141204193711.03524183@sleipner.datanom.net>
	<CAOEv1kaoxj=AREpfuRo8saHFhW18i0CmgSd9dfwLZEtCUv8pmw@mail.gmail.com>
	<20141207224353.GA13916@warlock.deepthought.com>
	<CAOEv1kYze9=As8jtTYAG88O+0hn0ECvLGrY9_HiLZdGu1-23uA@mail.gmail.com>
	<CAPJtH1gTA9z4got9Ou=Zc9BxjmULGq3-NhF=50YgMT6Vqx3n8w@mail.gmail.com>
	<674b01d01324$d0e28180$72a78480$@acm.org>
	<CAPJtH1h_Y_3rQm5-SNaUAPcnQxvJ4CKEUdni4kzmXQhuVhH8ZA@mail.gmail.com>
Message-ID: <6e8301d0132f$e1702c40$a45084c0$@acm.org>

> From: Ian Kaufman
> Sent: Monday, December 08, 2014 12:37 PM
> 
> Yes, I oversimplified things. The issue is that AUTH_SYS/AUTH_UNIX
> does not support NFSv4. AUTH_SYS uses the UID, not the name, so the
> mapping fails. So, when RPC uses AUTH_SYS, NFSv4 is SOL.

I apologize for being a pedant; it's a character flaw :). Your wording implies that you cannot use AUTH_SYS with NFSv4, which is not true. NFSv4 works perfectly fine with AUTH_SYS as long as you maintain synchronization of uid/gid's on both sides. I would pick NFSv4 with AUTH_SYS over NFSv3 with AUTH_SYS. Both require that you maintain uid synchronization, so it's not like you're gaining something by falling back, and why miss out on the other features of NFSv4?

> Supposedly. at least on the client side, this has been fixed somewhere
> upstream. However, the server side is not.
> 
> https://bugzilla.linux-nfs.org/show_bug.cgi?id=226

I don't know if I would call this "fixed" 8-/. They are basically just disabling the idmapper and passing raw uid/gid info at the NFS level to match the raw info at the RPC level. I guess it makes it less confusing in such a scenario, because you're always broken if they don't match on each side rather than only broken in some cases. An actual fix would be introducing an RPC mechanism using names such as AUTH_SYS_NAME, one of these days maybe I'll find the time to go hassle some NFS developers and see why they don't just do that and make everything simpler.

> Regardless, my point is that this is not a Solaris/Linux issue, as a
> Linux server and Linux clients would be in the same boat.

Agreed. It is a deficiency in the NFSv4 protocol :(...



From info at houseofancients.nl  Mon Dec  8 23:14:58 2014
From: info at houseofancients.nl (Floris van Essen ..:: House of Ancients Amstafs ::..)
Date: Mon, 8 Dec 2014 23:14:58 +0000
Subject: [OmniOS-discuss] LACP Omnios , igb and C3750
Message-ID: <356582D1FC91784992ABB4265A16ED487EBAE58F@vEX01.mindstorm-internet.local>

Guys,
been having a odd issue with my Omnios box and c3750
the omnios box has 6 interfaces ( 2 e1000g and 4 igb)
somehow the LACP bonding between the igb's and my cisco's will not stay in sync, while the same lacp config runs fine over the e1000's :
channel 5 = e1000g's channel 6 and 7 are the igb's :
Channel group 5 neighbors
Partner's information:
LACP port Admin Oper Port Port
Port Flags Priority Dev ID Age key Key Number State
Gi1/0/14 FA 4096 0030.48d5.ec94 23s 0x0 0x3E8 0x2 0x3F
Gi2/0/14 FA 4096 0030.48d5.ec94 23s 0x0 0x3E8 0x1 0x3F
Channel group 6 neighbors
Partner's information:
LACP port Admin Oper Port Port
Port Flags Priority Dev ID Age key Key Number State
Gi1/0/10 FA 4096 a036.9f02.c13a 0s 0x0 0x3EA 0x6 0x47
Gi1/0/12 FA 4096 a036.9f02.c13a 0s 0x0 0x3EA 0x5 0x47
Channel group 7 neighbors
Partner's information:
LACP port Admin Oper Port Port
Port Flags Priority Dev ID Age key Key Number State
Gi2/0/10 FA 4096 a036.9f02.c138 0s 0x0 0x3E9 0x4 0x47
Gi2/0/11 FA 4096 a036.9f02.c138 0s 0x0 0x3E9 0x3 0x47
LINK POLICY ADDRPOLICY LACPACTIVITY LACPTIMER FLAGS
aggr0 L4 auto active short -----
aggr1 L4 auto active short -----
aggr2 L4 auto active short -----
aggr0 -- up 1000Mb full static x.x.x.x 255.255.255.0 x:x:x:x:x:x 1500
e1000g0 attached up 1000Mb full - - - x:x:x:x:x:x 1500
e1000g1 attached up 1000Mb full - - - x:x:x:x:x:x 1500
aggr1 -- up 1000Mb full static x.x.x.x 255.255.255.0 x:x:x:x:x:x 1500
igb0 attached up 1000Mb full - - - x:x:x:x:x:x 1500
igb1 attached up 1000Mb full - - - x:x:x:x:x:x 1500
aggr2 -- up 1000Mb full static x.x.x.x 255.255.255.0 x:x:x:x:x:x 1500
igb2 attached up 1000Mb full - - - x:x:x:x:x:x 1500
igb3 attached up 1000Mb full - - - x:x:x:x:x:x 1500
058673: Dec 8 22:15:51.593: lacp_mux Gi2/0/10 - mux: during state WAITING, got event 1(selected)
058674: Dec 8 22:15:51.593: @@@ lacp_mux Gi2/0/10 - mux: WAITING -> WAITING
058675: Dec 8 22:15:51.593: LACP: Gi2/0/10 lacp_action_mx_waiting entered
058676: Dec 8 22:15:51.593: LACP: timer lacp_w(Gi2/0/10) started with interval 2000.
058677: Dec 8 22:15:51.895: LACP :lacp_bugpak: Receive LACP-PDU packet via Gi2/0/10
058678: Dec 8 22:15:51.895: LACP : packet size: 124
058679: Dec 8 22:15:51.895: LACP: pdu: subtype: 1, version: 1
058680: Dec 8 22:15:51.895: LACP: Act: tlv:1, tlv-len:20, key:0x3E9, p-pri:0x1000, p:0x4, p-state:0x47,
s-pri:0x1000, s-mac:a036.9f02.c138
058681: Dec 8 22:15:51.895: LACP: Part: tlv:2, tlv-len:20, key:0x0, p-pri:0x0, p:0x0, p-state:0xA,
s-pri:0x0, s-mac:0000.0000.0000
058682: Dec 8 22:15:51.895: LACP: col-tlv:3, col-tlv-len:16, col-max-d:0xA
058683: Dec 8 22:15:51.895: LACP: term-tlv:0 termr-tlv-len:0
058684: Dec 8 22:15:51.895: LACP: Gi2/0/10 LACP packet received, processing
058685: Dec 8 22:15:51.895: lacp_rx Gi2/0/10 - rx: during state CURRENT, got event 5(recv_lacpdu)
058686: Dec 8 22:15:51.895: @@@ lacp_rx Gi2/0/10 - rx: CURRENT -> CURRENT
058687: Dec 8 22:15:51.895: LACP: Gi2/0/10 lacp_action_rx_current entered
058688: Dec 8 22:15:51.895: LACP: Gi2/0/10 set to UNSELECTED
058689: Dec 8 22:15:51.895: lacp_mux Gi2/0/10 - mux: during state WAITING, got event 3(unselected)
058690: Dec 8 22:15:51.895: @@@ lacp_mux Gi2/0/10 - mux: WAITING -> DETACHED
058691: Dec 8 22:15:51.895: LACP: Gi2/0/10 lacp_action_mx_detached entered
058692: Dec 8 22:15:51.895: LACP: Gi2/0/10 Resetting hw_sw constraints
058693: Dec 8 22:15:51.895: LACP: lacp_insert_partner_cd_inhibitor: didn't change sync flag.
058694: Dec 8 22:15:51.895: LACP: lacp_send_lacpdu: (Gi2/0/10) About to send the 110 LACPDU
058695: Dec 8 22:15:51.895: LACP :lacp_bugpak: Send LACP-PDU packet via Gi2/0/10
058696: Dec 8 22:15:51.895: LACP : packet size: 124
058697: Dec 8 22:15:51.895: LACP: pdu: subtype: 1, version: 1
058698: Dec 8 22:15:51.895: LACP: Act: tlv:1, tlv-len:20, key:0x7, p-pri:0x8000, p:0x20B, p-state:0x5,
s-pri:0x8000, s-mac:0019.aa89.b580
058699: Dec 8 22:15:51.895: LACP: Part: tlv:2, tlv-len:20, key:0x3E9, p-pri:0x1000, p:0x4, p-state:0x47,
s-pri:0x1000, s-mac:a036.9f02.c138
058700: Dec 8 22:15:51.895: LACP: col-tlv:3, col-tlv-len:16, col-max-d:0x8000
058701: Dec 8 22:15:51.895: LACP: term-tlv:0 termr-tlv-len:0
058702: Dec 8 22:15:51.895: LACP: lacp_write: LACP 124 bytes out Gi2/0/10
058703: Dec 8 22:15:51.895: lacp_handle_standby_port_internal called, depth = 1
058704: Dec 8 22:15:51.895: LACP: recordPDU Gi2/0/10 LACP PDU Rcvd. Partners oper state is hex 47
058705: Dec 8 22:15:51.895: LACP: recordPDU Gi2/0/10 Partner out of sync
058706: Dec 8 22:15:51.895: LACP: Gi2/0/10 Partners oper state is hex 47
058707: Dec 8 22:15:51.895: LACP: timer lacp_c_l(Gi2/0/10) started with interval 90000.
058708: Dec 8 22:15:51.895: LACP: Gi2/0/10 LAG_PARTNER_UP.
058709: Dec 8 22:15:51.895: LACP: Gi2/0/10 LAG unchanged
058710: Dec 8 22:15:51.903: LACP: Gi2/0/10 STANDBY aggregator hex address is 5FB71A4
058711: Dec 8 22:15:51.903: LACP: Gi2/0/10 set to STANDBY
058712: Dec 8 22:15:51.903: lacp_mux Gi2/0/10 - mux: during state DETACHED, got event 2(standby)
058713: Dec 8 22:15:51.903: @@@ lacp_mux Gi2/0/10 - mux: DETACHED -> WAITING
058714: Dec 8 22:15:51.903: LACP: Gi2/0/10 lacp_action_mx_waiting entered
058715: Dec 8 22:15:51.903: @@@ lacp_mux Gi2/0/10 - mux: WAITING -> WAITING
058716: Dec 8 22:15:51.903: lacp_mux Gi2/0/10 - mux: during state WAITING, got event 6(outof_sync) (ignored)
058717: Dec 8 22:15:51.903: LACP :lacp_bugpak: Receive LACP-PDU packet via Gi2/0/11
058718: Dec 8 22:15:51.903: LACP : packet size: 124
058719: Dec 8 22:15:51.903: LACP: pdu: subtype: 1, version: 1
058720: Dec 8 22:15:51.903: LACP: Act: tlv:1, tlv-len:20, key:0x3E9, p-pri:0x1000, p:0x3, p-state:0x47,
s-pri:0x1000, s-mac:a036.9f02.c138
058721: Dec 8 22:15:51.903: LACP: Part: tlv:2, tlv-len:20, key:0x0, p-pri:0x0, p:0x0, p-state:0xA,
s-pri:0x0, s-mac:0000.0000.0000
058722: Dec 8 22:15:51.903: LACP: col-tlv:3, col-tlv-len:16, col-max-d:0xA
058723: Dec 8 22:15:51.903: LACP: term-tlv:0 termr-tlv-len:0
058724: Dec 8 22:15:51.903: LACP: Gi2/0/11 LACP packet received, processing
058725: Dec 8 22:15:51.903: lacp_rx Gi2/0/11 - rx: during state CURRENT, got event 5(recv_lacpdu)
058726: Dec 8 22:15:51.903: @@@ lacp_rx Gi2/0/11 - rx: CURRENT -> CURRENT
058727: Dec 8 22:15:51.903: LACP: Gi2/0/11 lacp_action_rx_current entered
058728: Dec 8 22:15:51.903: LACP: Gi2/0/11 set to UNSELECTED
058729: Dec 8 22:15:51.903: lacp_mux Gi2/0/11 - mux: during state WAITING, got event 3(unselected)
058730: Dec 8 22:15:51.903: @@@ lacp_mux Gi2/0/11 - mux: WAITING -> DETACHED
058731: Dec 8 22:15:51.903: LACP: Gi2/0/11 lacp_action_mx_detached entered
058732: Dec 8 22:15:51.903: LACP: Gi2/0/11 Resetting hw_sw constraints
058733: Dec 8 22:15:51.903: LACP: lacp_insert_partner_cd_inhibitor: didn't change sync flag.
058734: Dec 8 22:15:51.903: LACP: lacp_send_lacpdu: (Gi2/0/11) About to send the 110 LACPDU
058735: Dec 8 22:15:51.903: LACP :lacp_bugpak: Send LACP-PDU packet via Gi2/0/11
058736: Dec 8 22:15:51.903: LACP : packet size: 124
058737: Dec 8 22:15:51.903: LACP: pdu: subtype: 1, version: 1
058738: Dec 8 22:15:51.903: LACP: Act: tlv:1, tlv-len:20, key:0x7, p-pri:0x8000, p:0x20C, p-state:0x5,
s-pri:0x8000, s-mac:0019.aa89.b580
058739: Dec 8 22:15:51.903: LACP: Part: tlv:2, tlv-len:20, key:0x3E9, p-pri:0x1000, p:0x3, p-state:0x47,
s-pri:0x1000, s-mac:a036.9f02.c138


so in short, i see it coming up, shows out of sync, and down it goes...
have tried short times, long times active , passive... nothing will work


Met vriendelijke groet / With kind regards,



               Floris van Essen / Claudia Dufornee
                House of Ancients Amstaffs

               info at houseofancients.nl<mailto:info at houseofancients.nl>
               www.houseofancients.nl


...:: House of Ancients ::...
American Staffordshire Terriers

+31-628-161-350
+31-614-198-389
Het Perk 48
4903 RB
Oosterhout
Netherlands
www.houseofancients.nl
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141208/7723a4bf/attachment-0001.html>

From filip.marvan at aira.cz  Tue Dec  9 09:06:01 2014
From: filip.marvan at aira.cz (Filip Marvan)
Date: Tue, 9 Dec 2014 10:06:01 +0100
Subject: [OmniOS-discuss] Parity error on path mpt_sas2
In-Reply-To: <004501d01302$257c9b90$7075d2b0$@subrigo.net>
References: <3BE0DEED8863E5429BAE4CAEDF624565039C1B0DAAD0@AIRA-SRV.aira.local>
	<1439760B-8D8B-40F9-8A62-4CFE4A9483C1@omniti.com>
	<3BE0DEED8863E5429BAE4CAEDF624565039C1B0DAADA@AIRA-SRV.aira.local>
	<CAOqBcP_12FaH77o+3nrGVm7KU3x1XCk3B-9uenKRRL63t5wdyg@mail.gmail.com>
	<3BE0DEED8863E5429BAE4CAEDF624565039C1B0DACD1@AIRA-SRV.aira.local>
	<CALeZrrRhsV5i2AJF2aRrLxQvR9LSckiEax59gX5XXAYAkpCS+Q@mail.gmail.com>
	<004501d01302$257c9b90$7075d2b0$@subrigo.net>
Message-ID: <3BE0DEED8863E5429BAE4CAEDF624565039C1B0D8E3F@AIRA-SRV.aira.local>

Hi,

first you have to erase P20 firmware with:

sas2flsh -o -e 6

now do not reboot(!) and flash P18 with

sas2flsh -f XXXXXX.bin -b mptsas2.rom

Filip


From: OmniOS-discuss [mailto:omnios-discuss-bounces at lists.omniti.com] On Behalf Of Schweiss, Chip
Sent: Monday, December 08, 2014 08:09 AM
To: Filip Marvan
Cc: omnios-discuss at lists.omniti.com
Subject: Re: [OmniOS-discuss] Parity error on path mpt_sas2

I've got some new LSI HBAs I'm trying to downgrade from firmware version 20 to 18.
I'm getting errors when trying to downgrade:

Attempting to flash firmware to LSI SAS SAS2308_2(D1) :

        Executing Operation: Flash Firmware Image

                Firmware Image has a Valid Checksum.
                Firmware Version 18.00.00.00
                Firmware Image compatible with Controller.

                Valid NVDATA Image found.
                NVDATA Version 11.00.00.00
                Checking for a compatible NVData image...

                NVDATA Device ID and Chip Revision match verified.
                ERROR: Cannot downgrade NVDATA version 14.00.00.06
                       to 11.00.110000.00.

                ERROR: Failed to get valid NVDATA image from File!

                Firmware Image Validation Failed!
Tried downgrading bios:

Attempting to flash Boot Service to LSI SAS SAS2308_2(D1) :

                Validating BIOS Image...

                BIOS Header Signature is Valid

                BIOS Image has a Valid Checksum.

                BIOS PCI Structure Signature Valid.

                BIOS Image Compatible with the SAS Controller.

                Attempting to Flash BIOS Image...

                BIOS Version in flash    : 07.39.00.00
                BIOS Version from File  : 07.35.00.00
                Skipping flash since file version is not greater than existing.

                Flash BIOS Image Failed!
I've tried sas2flash from P20 and P18.

Can someone fill me in on the trick to downgrade?
-Chip


On Thu, Nov 27, 2014 at 9:03 AM, Filip Marvan <filip.marvan at aira.cz<mailto:filip.marvan at aira.cz>> wrote:
Thank you for your help Aaron! I downgraded firmware to P18 and there are no more errors today.
It seems, that there is something bad with P20 firmware!

You saved me a lot of time :)

Filip


From: Aaron Curry [mailto:asc1111 at gmail.com<mailto:asc1111 at gmail.com>]
Sent: Tuesday, November 25, 2014 5:46 PM
To: Filip Marvan
Cc: Dan McDonald; omnios-discuss at lists.omniti.com<mailto:omnios-discuss at lists.omniti.com>
Subject: Re: [OmniOS-discuss] Parity error on path mpt_sas2

I had this exact same problem recently when setting up a new home server... same controller, same firmware (P20). The errors were on all disks attached to the controller, but only on high read activity. Writes did not generate the errors. I downgraded the firmware to P18 (which is what we are using at work) and the errors went away. Has anyone had success with this controller and the P20 firmware?

Aaron

On Tue, Nov 25, 2014 at 8:30 AM, Filip Marvan <filip.marvan at aira.cz<mailto:filip.marvan at aira.cz>> wrote:
Hi Dan,

thanks for reply.
Yes, errors are on all 5 disks in the same RAIDZ pool. No problem on other
disks in different pool but on the same SAS cable.
So maybe I only had bad luck with that disks.

Filip


_______________________________________________
OmniOS-discuss mailing list
OmniOS-discuss at lists.omniti.com<mailto:OmniOS-discuss at lists.omniti.com>
http://lists.omniti.com/mailman/listinfo/omnios-discuss


_______________________________________________
OmniOS-discuss mailing list
OmniOS-discuss at lists.omniti.com<mailto:OmniOS-discuss at lists.omniti.com>
http://lists.omniti.com/mailman/listinfo/omnios-discuss

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141209/6f8c5e63/attachment.html>

From chip at innovates.com  Tue Dec  9 14:49:30 2014
From: chip at innovates.com (Schweiss, Chip)
Date: Tue, 9 Dec 2014 08:49:30 -0600
Subject: [OmniOS-discuss] Parity error on path mpt_sas2
In-Reply-To: <3BE0DEED8863E5429BAE4CAEDF624565039C1B0D8E3F@AIRA-SRV.aira.local>
References: <3BE0DEED8863E5429BAE4CAEDF624565039C1B0DAAD0@AIRA-SRV.aira.local>
	<1439760B-8D8B-40F9-8A62-4CFE4A9483C1@omniti.com>
	<3BE0DEED8863E5429BAE4CAEDF624565039C1B0DAADA@AIRA-SRV.aira.local>
	<CAOqBcP_12FaH77o+3nrGVm7KU3x1XCk3B-9uenKRRL63t5wdyg@mail.gmail.com>
	<3BE0DEED8863E5429BAE4CAEDF624565039C1B0DACD1@AIRA-SRV.aira.local>
	<CALeZrrRhsV5i2AJF2aRrLxQvR9LSckiEax59gX5XXAYAkpCS+Q@mail.gmail.com>
	<004501d01302$257c9b90$7075d2b0$@subrigo.net>
	<3BE0DEED8863E5429BAE4CAEDF624565039C1B0D8E3F@AIRA-SRV.aira.local>
Message-ID: <CALeZrrTqymmNX3E=NpZr2kJokFHK6xB850NLjDh12Wo9o4Mcng@mail.gmail.com>

I'm still fighting this.

Under OmniOS the erase function is disabled in sas2flash for solaris.

I couldn't seem to find an iso bootable DOS that I could stage files in.
FreeDOS1.0 won't mount the CD, 1.1 got rid of the live mode.    Tried a
Linux rescue CD and I get " ERROR: Erase Flash Operation Failed!"   Tried
both P18 sas2flash and P20 sas2flash.

What OS environment are you running this in?



On Tue, Dec 9, 2014 at 3:06 AM, Filip Marvan <filip.marvan at aira.cz> wrote:

>  Hi,
>
>
>
> first you have to erase P20 firmware with:
>
>
>
> sas2flsh -o -e 6
>
>
>
> now do not reboot(!) and flash P18 with
>
>
>
> sas2flsh -f XXXXXX.bin -b mptsas2.rom
>
>
>
> Filip
>
>
>
>
>
> *From:* OmniOS-discuss [mailto:omnios-discuss-bounces at lists.omniti.com] *On
> Behalf Of *Schweiss, Chip
> *Sent:* Monday, December 08, 2014 08:09 AM
> *To:* Filip Marvan
> *Cc:* omnios-discuss at lists.omniti.com
> *Subject:* Re: [OmniOS-discuss] Parity error on path mpt_sas2
>
>
>
> I've got some new LSI HBAs I'm trying to downgrade from firmware version
> 20 to 18.
>
> I'm getting errors when trying to downgrade:
>
> Attempting to flash firmware to LSI SAS SAS2308_2(D1) :
>
>         Executing Operation: Flash Firmware Image
>
>                 Firmware Image has a Valid Checksum.
>                 Firmware Version 18.00.00.00
>                 Firmware Image compatible with Controller.
>
>                 Valid NVDATA Image found.
>                 NVDATA Version 11.00.00.00
>                 Checking for a compatible NVData image...
>
>                 NVDATA Device ID and Chip Revision match verified.
>                 ERROR: Cannot downgrade NVDATA version 14.00.00.06
>                        to 11.00.110000.00.
>
>                 ERROR: Failed to get valid NVDATA image from File!
>
>                 Firmware Image Validation Failed!
>
> Tried downgrading bios:
>
> Attempting to flash Boot Service to LSI SAS SAS2308_2(D1) :
>
>                 Validating BIOS Image...
>
>                 BIOS Header Signature is Valid
>
>                 BIOS Image has a Valid Checksum.
>
>                 BIOS PCI Structure Signature Valid.
>
>                 BIOS Image Compatible with the SAS Controller.
>
>                 Attempting to Flash BIOS Image...
>
>                 BIOS Version in flash    : 07.39.00.00
>                 BIOS Version from File  : 07.35.00.00
>                 Skipping flash since file version is not greater than
> existing.
>
>                 Flash BIOS Image Failed!
>
> I've tried sas2flash from P20 and P18.
>
> Can someone fill me in on the trick to downgrade?
>
> -Chip
>
>
>
>
>
> On Thu, Nov 27, 2014 at 9:03 AM, Filip Marvan <filip.marvan at aira.cz>
> wrote:
>
> Thank you for your help Aaron! I downgraded firmware to P18 and there are
> no more errors today.
>
> It seems, that there is something bad with P20 firmware!
>
>
>
> You saved me a lot of time J
>
>
>
> Filip
>
>
>
>
>
> *From:* Aaron Curry [mailto:asc1111 at gmail.com]
> *Sent:* Tuesday, November 25, 2014 5:46 PM
> *To:* Filip Marvan
> *Cc:* Dan McDonald; omnios-discuss at lists.omniti.com
> *Subject:* Re: [OmniOS-discuss] Parity error on path mpt_sas2
>
>
>
> I had this exact same problem recently when setting up a new home
> server... same controller, same firmware (P20). The errors were on all
> disks attached to the controller, but only on high read activity. Writes
> did not generate the errors. I downgraded the firmware to P18 (which is
> what we are using at work) and the errors went away. Has anyone had success
> with this controller and the P20 firmware?
>
>
>
> Aaron
>
>
>
> On Tue, Nov 25, 2014 at 8:30 AM, Filip Marvan <filip.marvan at aira.cz>
> wrote:
>
> Hi Dan,
>
> thanks for reply.
> Yes, errors are on all 5 disks in the same RAIDZ pool. No problem on other
> disks in different pool but on the same SAS cable.
> So maybe I only had bad luck with that disks.
>
> Filip
>
>
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>
>
>
>
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141209/a2a52702/attachment-0001.html>

From filip.marvan at aira.cz  Tue Dec  9 14:58:05 2014
From: filip.marvan at aira.cz (Filip Marvan)
Date: Tue, 9 Dec 2014 15:58:05 +0100
Subject: [OmniOS-discuss] Parity error on path mpt_sas2
In-Reply-To: <CALeZrrTqymmNX3E=NpZr2kJokFHK6xB850NLjDh12Wo9o4Mcng@mail.gmail.com>
References: <3BE0DEED8863E5429BAE4CAEDF624565039C1B0DAAD0@AIRA-SRV.aira.local>
	<1439760B-8D8B-40F9-8A62-4CFE4A9483C1@omniti.com>
	<3BE0DEED8863E5429BAE4CAEDF624565039C1B0DAADA@AIRA-SRV.aira.local>
	<CAOqBcP_12FaH77o+3nrGVm7KU3x1XCk3B-9uenKRRL63t5wdyg@mail.gmail.com>
	<3BE0DEED8863E5429BAE4CAEDF624565039C1B0DACD1@AIRA-SRV.aira.local>
	<CALeZrrRhsV5i2AJF2aRrLxQvR9LSckiEax59gX5XXAYAkpCS+Q@mail.gmail.com>
	<004501d01302$257c9b90$7075d2b0$@subrigo.net>
	<3BE0DEED8863E5429BAE4CAEDF624565039C1B0D8E3F@AIRA-SRV.aira.local>
	<CALeZrrTqymmNX3E=NpZr2kJokFHK6xB850NLjDh12Wo9o4Mcng@mail.gmail.com>
Message-ID: <3BE0DEED8863E5429BAE4CAEDF624565039C1B330ACE@AIRA-SRV.aira.local>

Yes, you have to use bootable DOS and DOS version of sas2flash.

There is an awesome tool for creating bootable USB stick with DOS called Rufus



http://rufus.akeo.ie/



Filip





From: Schweiss, Chip [mailto:chip at innovates.com]
Sent: Tuesday, December 09, 2014 3:50 PM
To: Filip Marvan
Cc: Matthew Lagoe; omnios-discuss at lists.omniti.com
Subject: Re: [OmniOS-discuss] Parity error on path mpt_sas2



I'm still fighting this.

Under OmniOS the erase function is disabled in sas2flash for solaris.

I couldn't seem to find an iso bootable DOS that I could stage files in. 
FreeDOS1.0 won't mount the CD, 1.1 got rid of the live mode.    Tried a Linux 
rescue CD and I get " ERROR: Erase Flash Operation Failed!"   Tried both P18 
sas2flash and P20 sas2flash.

What OS environment are you running this in?





On Tue, Dec 9, 2014 at 3:06 AM, Filip Marvan <filip.marvan at aira.cz> wrote:

Hi,



first you have to erase P20 firmware with:



sas2flsh -o -e 6



now do not reboot(!) and flash P18 with



sas2flsh -f XXXXXX.bin -b mptsas2.rom



Filip





From: OmniOS-discuss [mailto:omnios-discuss-bounces at lists.omniti.com] On 
Behalf Of Schweiss, Chip
Sent: Monday, December 08, 2014 08:09 AM
To: Filip Marvan
Cc: omnios-discuss at lists.omniti.com
Subject: Re: [OmniOS-discuss] Parity error on path mpt_sas2



I've got some new LSI HBAs I'm trying to downgrade from firmware version 20 to 
18.

I'm getting errors when trying to downgrade:

Attempting to flash firmware to LSI SAS SAS2308_2(D1) :

        Executing Operation: Flash Firmware Image

                Firmware Image has a Valid Checksum.
                Firmware Version 18.00.00.00
                Firmware Image compatible with Controller.

                Valid NVDATA Image found.
                NVDATA Version 11.00.00.00
                Checking for a compatible NVData image...

                NVDATA Device ID and Chip Revision match verified.
                ERROR: Cannot downgrade NVDATA version 14.00.00.06
                       to 11.00.110000.00.

                ERROR: Failed to get valid NVDATA image from File!

                Firmware Image Validation Failed!

Tried downgrading bios:

Attempting to flash Boot Service to LSI SAS SAS2308_2(D1) :

                Validating BIOS Image...

                BIOS Header Signature is Valid

                BIOS Image has a Valid Checksum.

                BIOS PCI Structure Signature Valid.

                BIOS Image Compatible with the SAS Controller.

                Attempting to Flash BIOS Image...

                BIOS Version in flash    : 07.39.00.00
                BIOS Version from File  : 07.35.00.00
                Skipping flash since file version is not greater than 
existing.

                Flash BIOS Image Failed!

I've tried sas2flash from P20 and P18.

Can someone fill me in on the trick to downgrade?

-Chip





On Thu, Nov 27, 2014 at 9:03 AM, Filip Marvan <filip.marvan at aira.cz> wrote:

Thank you for your help Aaron! I downgraded firmware to P18 and there are no 
more errors today.

It seems, that there is something bad with P20 firmware!



You saved me a lot of time J



Filip





From: Aaron Curry [mailto:asc1111 at gmail.com]
Sent: Tuesday, November 25, 2014 5:46 PM
To: Filip Marvan
Cc: Dan McDonald; omnios-discuss at lists.omniti.com
Subject: Re: [OmniOS-discuss] Parity error on path mpt_sas2



I had this exact same problem recently when setting up a new home server... 
same controller, same firmware (P20). The errors were on all disks attached to 
the controller, but only on high read activity. Writes did not generate the 
errors. I downgraded the firmware to P18 (which is what we are using at work) 
and the errors went away. Has anyone had success with this controller and the 
P20 firmware?



Aaron



On Tue, Nov 25, 2014 at 8:30 AM, Filip Marvan <filip.marvan at aira.cz> wrote:

Hi Dan,

thanks for reply.
Yes, errors are on all 5 disks in the same RAIDZ pool. No problem on other
disks in different pool but on the same SAS cable.
So maybe I only had bad luck with that disks.

Filip


_______________________________________________
OmniOS-discuss mailing list
OmniOS-discuss at lists.omniti.com
http://lists.omniti.com/mailman/listinfo/omnios-discuss




_______________________________________________
OmniOS-discuss mailing list
OmniOS-discuss at lists.omniti.com
http://lists.omniti.com/mailman/listinfo/omnios-discuss





-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141209/b78e598c/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 6247 bytes
Desc: not available
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141209/b78e598c/attachment-0001.bin>

From hasslerd at gmx.li  Tue Dec  9 15:51:03 2014
From: hasslerd at gmx.li (Dominik Hassler)
Date: Tue, 09 Dec 2014 16:51:03 +0100
Subject: [OmniOS-discuss] Parity error on path mpt_sas2
In-Reply-To: <CALeZrrTqymmNX3E=NpZr2kJokFHK6xB850NLjDh12Wo9o4Mcng@mail.gmail.com>
References: <3BE0DEED8863E5429BAE4CAEDF624565039C1B0DAAD0@AIRA-SRV.aira.local>	<1439760B-8D8B-40F9-8A62-4CFE4A9483C1@omniti.com>	<3BE0DEED8863E5429BAE4CAEDF624565039C1B0DAADA@AIRA-SRV.aira.local>	<CAOqBcP_12FaH77o+3nrGVm7KU3x1XCk3B-9uenKRRL63t5wdyg@mail.gmail.com>	<3BE0DEED8863E5429BAE4CAEDF624565039C1B0DACD1@AIRA-SRV.aira.local>	<CALeZrrRhsV5i2AJF2aRrLxQvR9LSckiEax59gX5XXAYAkpCS+Q@mail.gmail.com>	<004501d01302$257c9b90$7075d2b0$@subrigo.net>	<3BE0DEED8863E5429BAE4CAEDF624565039C1B0D8E3F@AIRA-SRV.aira.local>
	<CALeZrrTqymmNX3E=NpZr2kJokFHK6xB850NLjDh12Wo9o4Mcng@mail.gmail.com>
Message-ID: <54871A67.3040805@gmx.li>

Chip,

when I had to reflash an onboard LSI2308 I used the UEFI shell. No
fiddling around with boot disks etc.

Not sure, if it is applicable to your problem.

On 12/09/2014 03:49 PM, Schweiss, Chip wrote:
> I'm still fighting this.
> 
> Under OmniOS the erase function is disabled in sas2flash for solaris.
> 
> I couldn't seem to find an iso bootable DOS that I could stage files
> in.  FreeDOS1.0 won't mount the CD, 1.1 got rid of the live mode.   
> Tried a Linux rescue CD and I get " ERROR: Erase Flash Operation
> Failed!"   Tried both P18 sas2flash and P20 sas2flash.
> 
> What OS environment are you running this in?  
> 
> 
> 
> On Tue, Dec 9, 2014 at 3:06 AM, Filip Marvan <filip.marvan at aira.cz
> <mailto:filip.marvan at aira.cz>> wrote:
> 
>     Hi,____
> 
>     __ __
> 
>     first you have to erase P20 firmware with:____
> 
>     __ __
> 
>     sas2flsh -o -e 6____
> 
>     __ __
> 
>     now do not reboot(!) and flash P18 with____
> 
>     __ __
> 
>     sas2flsh -f XXXXXX.bin -b mptsas2.rom____
> 
>     __ __
> 
>     Filip____
> 
>     __ __
> 
>     __ __
> 
>     *From:*OmniOS-discuss
>     [mailto:omnios-discuss-bounces at lists.omniti.com
>     <mailto:omnios-discuss-bounces at lists.omniti.com>] *On Behalf Of
>     *Schweiss, Chip
>     *Sent:* Monday, December 08, 2014 08:09 AM
>     *To:* Filip Marvan
>     *Cc:* omnios-discuss at lists.omniti.com
>     <mailto:omnios-discuss at lists.omniti.com>
>     *Subject:* Re: [OmniOS-discuss] Parity error on path mpt_sas2____
> 
>     __ __
> 
>     I've got some new LSI HBAs I'm trying to downgrade from firmware
>     version 20 to 18.____
> 
>     I'm getting errors when trying to downgrade:
> 
>     Attempting to flash firmware to LSI SAS SAS2308_2(D1) :
> 
>             Executing Operation: Flash Firmware Image
> 
>                     Firmware Image has a Valid Checksum.
>                     Firmware Version 18.00.00.00
>                     Firmware Image compatible with Controller.
> 
>                     Valid NVDATA Image found.
>                     NVDATA Version 11.00.00.00
>                     Checking for a compatible NVData image...
> 
>                     NVDATA Device ID and Chip Revision match verified.
>                     ERROR: Cannot downgrade NVDATA version 14.00.00.06
>                            to 11.00.110000.00.
> 
>                     ERROR: Failed to get valid NVDATA image from File!
> 
>                     Firmware Image Validation Failed!____
> 
>     Tried downgrading bios:
> 
>     Attempting to flash Boot Service to LSI SAS SAS2308_2(D1) :
> 
>                     Validating BIOS Image...
> 
>                     BIOS Header Signature is Valid
> 
>                     BIOS Image has a Valid Checksum.
> 
>                     BIOS PCI Structure Signature Valid.
> 
>                     BIOS Image Compatible with the SAS Controller.
> 
>                     Attempting to Flash BIOS Image...
> 
>                     BIOS Version in flash    : 07.39.00.00
>                     BIOS Version from File  : 07.35.00.00
>                     Skipping flash since file version is not greater
>     than existing.
> 
>                     Flash BIOS Image Failed!____
> 
>     I've tried sas2flash from P20 and P18.  
> 
>     Can someone fill me in on the trick to downgrade?  ____
> 
>     -Chip____
> 
>     __ __
> 
>     __ __
> 
>     On Thu, Nov 27, 2014 at 9:03 AM, Filip Marvan <filip.marvan at aira.cz
>     <mailto:filip.marvan at aira.cz>> wrote:____
> 
>     Thank you for your help Aaron! I downgraded firmware to P18 and
>     there are no more errors today.____
> 
>     It seems, that there is something bad with P20 firmware!____
> 
>      ____
> 
>     You saved me a lot of time J____
> 
>      ____
> 
>     Filip____
> 
>      ____
> 
>      ____
> 
>     *From:*Aaron Curry [mailto:asc1111 at gmail.com
>     <mailto:asc1111 at gmail.com>]
>     *Sent:* Tuesday, November 25, 2014 5:46 PM
>     *To:* Filip Marvan
>     *Cc:* Dan McDonald; omnios-discuss at lists.omniti.com
>     <mailto:omnios-discuss at lists.omniti.com>
>     *Subject:* Re: [OmniOS-discuss] Parity error on path mpt_sas2____
> 
>      ____
> 
>     I had this exact same problem recently when setting up a new home
>     server... same controller, same firmware (P20). The errors were on
>     all disks attached to the controller, but only on high read
>     activity. Writes did not generate the errors. I downgraded the
>     firmware to P18 (which is what we are using at work) and the errors
>     went away. Has anyone had success with this controller and the P20
>     firmware?____
> 
>      ____
> 
>     Aaron____
> 
>      ____
> 
>     On Tue, Nov 25, 2014 at 8:30 AM, Filip Marvan <filip.marvan at aira.cz
>     <mailto:filip.marvan at aira.cz>> wrote:____
> 
>         Hi Dan,
> 
>         thanks for reply.
>         Yes, errors are on all 5 disks in the same RAIDZ pool. No
>         problem on other
>         disks in different pool but on the same SAS cable.
>         So maybe I only had bad luck with that disks.
> 
>         Filip
> 
> 
>         _______________________________________________
>         OmniOS-discuss mailing list
>         OmniOS-discuss at lists.omniti.com
>         <mailto:OmniOS-discuss at lists.omniti.com>
>         http://lists.omniti.com/mailman/listinfo/omnios-discuss____
> 
>      ____
> 
> 
>     _______________________________________________
>     OmniOS-discuss mailing list
>     OmniOS-discuss at lists.omniti.com <mailto:OmniOS-discuss at lists.omniti.com>
>     http://lists.omniti.com/mailman/listinfo/omnios-discuss____
> 
>     __ __
> 
> 
> 
> 
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
> 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0xF9ECC6A5.asc
Type: application/pgp-keys
Size: 2686 bytes
Desc: not available
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141209/783d64df/attachment.bin>

From danmcd at omniti.com  Tue Dec  9 17:10:50 2014
From: danmcd at omniti.com (Dan McDonald)
Date: Tue, 9 Dec 2014 12:10:50 -0500
Subject: [OmniOS-discuss] Please UPDATE now
Message-ID: <6D8B3BE9-18D5-44A1-A921-37D0ED337DA6@omniti.com>

Hello OmniOS users!

Illumos bug 5421 was fixed in all OmniOS repos, and the r151012/Stable install media has been updated as well.  This bug had allowed an ordinary user in the global zone to kernel-panic the machine.  That bug is now fixed in illumos-gate, and all SUPPORTED OmniOS revisions:

	- bloody
	- r151012 (aka. Stable)
	- r151010 (aka. previous Stable)
	- r151006 (aka. Long-Term Support)

If you are on one of these supported OmniOS revisions, run "pkg update" now and reboot.  I requested a CVE number for all illumos distros, but the CVE folks haven't gotten back to me yet.

Thank you!
Dan McDonald -- OmniOS Engineering


From henson at acm.org  Tue Dec  9 17:47:24 2014
From: henson at acm.org (Paul B. Henson)
Date: Tue, 9 Dec 2014 09:47:24 -0800
Subject: [OmniOS-discuss] Please UPDATE now
In-Reply-To: <6D8B3BE9-18D5-44A1-A921-37D0ED337DA6@omniti.com>
References: <6D8B3BE9-18D5-44A1-A921-37D0ED337DA6@omniti.com>
Message-ID: <CDE7637A-D68E-4580-840C-113EF8BCB03B@acm.org>

Is this only an issue if a malicious user intentionally crashes the system, or could it also potentially occur under regular use? IE, if you have a system with no local users only providing network services, would this still be a critical patch or could it wait for a more convenient installation schedule? The bug report isn't particularly detailed, it's not clear when/why devzvol_readdir() would call strchr or what would cause that call to return NULL. 

Thanks...

> On Dec 9, 2014, at 9:10 AM, Dan McDonald <danmcd at omniti.com> wrote:
> 
> Hello OmniOS users!
> 
> Illumos bug 5421 was fixed in all OmniOS repos, and the r151012/Stable install media has been updated as well.  This bug had allowed an ordinary user in the global zone to kernel-panic the machine.  That bug is now fixed in illumos-gate, and all SUPPORTED OmniOS revisions:
> 
>    - bloody
>    - r151012 (aka. Stable)
>    - r151010 (aka. previous Stable)
>    - r151006 (aka. Long-Term Support)
> 
> If you are on one of these supported OmniOS revisions, run "pkg update" now and reboot.  I requested a CVE number for all illumos distros, but the CVE folks haven't gotten back to me yet.
> 
> Thank you!
> Dan McDonald -- OmniOS Engineering
> 
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141209/e855eb75/attachment.html>

From danmcd at omniti.com  Tue Dec  9 18:37:21 2014
From: danmcd at omniti.com (Dan McDonald)
Date: Tue, 9 Dec 2014 13:37:21 -0500
Subject: [OmniOS-discuss] Please UPDATE now
In-Reply-To: <CDE7637A-D68E-4580-840C-113EF8BCB03B@acm.org>
References: <6D8B3BE9-18D5-44A1-A921-37D0ED337DA6@omniti.com>
	<CDE7637A-D68E-4580-840C-113EF8BCB03B@acm.org>
Message-ID: <F8FD43A1-21B4-4141-9F6F-396773BDC1E3@omniti.com>

Only specifically buggy code or malicious use will cause the panic.  The deployment as you describe (esp. No local users) has reduced the risk enough where you can likely wait for your window.

Dan

Sent from my iPhone (typos, autocorrect, and all)

> On Dec 9, 2014, at 12:47 PM, Paul B. Henson <henson at acm.org> wrote:
> 
> Is this only an issue if a malicious user intentionally crashes the system, or could it also potentially occur under regular use? IE, if you have a system with no local users only providing network services, would this still be a critical patch or could it wait for a more convenient installation schedule? The bug report isn't particularly detailed, it's not clear when/why devzvol_readdir() would call strchr or what would cause that call to return NULL. 
> 
> Thanks...
> 
>> On Dec 9, 2014, at 9:10 AM, Dan McDonald <danmcd at omniti.com> wrote:
>> 
>> Hello OmniOS users!
>> 
>> Illumos bug 5421 was fixed in all OmniOS repos, and the r151012/Stable install media has been updated as well.  This bug had allowed an ordinary user in the global zone to kernel-panic the machine.  That bug is now fixed in illumos-gate, and all SUPPORTED OmniOS revisions:
>> 
>>    - bloody
>>    - r151012 (aka. Stable)
>>    - r151010 (aka. previous Stable)
>>    - r151006 (aka. Long-Term Support)
>> 
>> If you are on one of these supported OmniOS revisions, run "pkg update" now and reboot.  I requested a CVE number for all illumos distros, but the CVE folks haven't gotten back to me yet.
>> 
>> Thank you!
>> Dan McDonald -- OmniOS Engineering
>> 
>> _______________________________________________
>> OmniOS-discuss mailing list
>> OmniOS-discuss at lists.omniti.com
>> http://lists.omniti.com/mailman/listinfo/omnios-discuss
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141209/7fd3c229/attachment.html>

From brogyi at gmail.com  Tue Dec  9 23:03:44 2014
From: brogyi at gmail.com (=?windows-1252?Q?Brogy=E1nyi_J=F3zsef?=)
Date: Wed, 10 Dec 2014 00:03:44 +0100
Subject: [OmniOS-discuss] Parity error on path mpt_sas2
In-Reply-To: <54871A67.3040805@gmx.li>
References: <3BE0DEED8863E5429BAE4CAEDF624565039C1B0DAAD0@AIRA-SRV.aira.local>	<1439760B-8D8B-40F9-8A62-4CFE4A9483C1@omniti.com>	<3BE0DEED8863E5429BAE4CAEDF624565039C1B0DAADA@AIRA-SRV.aira.local>	<CAOqBcP_12FaH77o+3nrGVm7KU3x1XCk3B-9uenKRRL63t5wdyg@mail.gmail.com>	<3BE0DEED8863E5429BAE4CAEDF624565039C1B0DACD1@AIRA-SRV.aira.local>	<CALeZrrRhsV5i2AJF2aRrLxQvR9LSckiEax59gX5XXAYAkpCS+Q@mail.gmail.com>	<004501d01302$257c9b90$7075d2b0$@subrigo.net>	<3BE0DEED8863E5429BAE4CAEDF624565039C1B0D8E3F@AIRA-SRV.aira.local>	<CALeZrrTqymmNX3E=NpZr2kJokFHK6xB850NLjDh12Wo9o4Mcng@mail.gmail.com>
	<54871A67.3040805@gmx.li>
Message-ID: <54877FD0.7060605@gmail.com>

Here is my short description about UEFI flashing. Please modify your 
version.

download 9211_8i_Package_P18_IR_IT_Firmware_BIOS_for_MSDOS_Windows
9211_8i_Package_P18_IR_IT_Firmware_BIOS_for_MSDOS_Windows\Firmware\HBA_9211_8i_IT\2118it.bin
9211_8i_Package_P20_IR_IT_Firmware_BIOS_for_MSDOS_Windows\sasbios_rel\mptsas2.rom
download Installer_P18_for_UEFI
Installer_P18_for_UEFI\sas2flash_efi_ebc_rel\sas2flash.efi
Shell.efi/shellx64.efi ### Both name are working. 
https://svn.code.sf.net/p/edk2/code/trunk/edk2/EdkShellBinPkg/FullShell/X64/Shell_Full.efi
### 
https://svn.code.sf.net/p/edk2/code/trunk/edk2/ShellBinPkg/UefiShell/X64/Shell.efi 
### The first link what I worked for me. The second is working with some 
Asus mb.
Format stick MBR and FAT, and there?s no need to make a bootable disk 
since we?ll be using EFI.
map ### Not need if you know it.
mount fs2 ### My stick name.
fs2:
ls
cd 9211 ### It was my directory name.
sa2flash.efi -o -e 6 ### Do not restart or switch it off after this step!!!
sa2flash.efi -o -f 2118it.bin -b mptsas2.rom
sa2flash.efi -listall ### For check.
sa2flash.efi -list ### For check.

From tobi at oetiker.ch  Wed Dec 10 00:41:54 2014
From: tobi at oetiker.ch (Tobias Oetiker)
Date: Wed, 10 Dec 2014 01:41:54 +0100 (CET)
Subject: [OmniOS-discuss] kvm io 10 times slower after r151010 -> r151012
	upgrade
Message-ID: <alpine.DEB.2.10.1412100127050.1990@engelberg.oetiker.ch>

So tonight, we finally took the plunge and upgraded our zfs/kvm
server to r151012 ... the results were terrible. The kvm booted
very slowly and all networking felt really slow ... so I did a
little test:

  ubutu-14.04-guest$ dd if=/dev/zero bs=1M count=20 | ssh omnios-r151012-host dd of=/dev/null
  20971520 bytes (21 MB) copied, 6.27333 s, 3.3 MB/s

  ubutu-14.04-guest$ ssh omnios-r151012-host dd if=/dev/zero bs=1M count=20 | dd of=/dev/null
  20971520 bytes transferred in 8.010208 secs (2618099 bytes/sec)

These numbers were obtained using virtio net drivers but switching
to e1000 did not significantly change things.

So we booted back into r151010 again ... the difference was
immediately apparent ... but there are also number to back this up.

  ubutu-14.04-guest$ dd if=/dev/zero bs=1M count=20 | ssh omnios-r151010-host dd of=/dev/null
  20971520 bytes (21 MB) copied, 0.812479 s, 25.8 MB/s

  ubutu-14.04-guest$ ssh omnios-r151010-host dd if=/dev/zero bs=1M count=20 | dd of=/dev/null
  20971520 bytes (21 MB) copied, 0.545423 s, 38.5 MB/s

as you can see the difference in guest network performance is
roughly one magnitude ...

I have not tested disk performance explicitly, but even booting a
windows host took ages ... so I suspect whatever is causing this
influences all kvm guest IO.

cheers
tobi

-- 
Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten, Switzerland
www.oetiker.ch tobi at oetiker.ch +41 62 775 9902


From danmcd at omniti.com  Wed Dec 10 00:59:47 2014
From: danmcd at omniti.com (Dan McDonald)
Date: Tue, 9 Dec 2014 19:59:47 -0500
Subject: [OmniOS-discuss] kvm io 10 times slower after r151010 ->
	r151012 upgrade
In-Reply-To: <alpine.DEB.2.10.1412100127050.1990@engelberg.oetiker.ch>
References: <alpine.DEB.2.10.1412100127050.1990@engelberg.oetiker.ch>
Message-ID: <E7BB97A8-7F0E-4DA3-B1F5-E407975A54F4@omniti.com>

> 
> I have not tested disk performance explicitly, but even booting a
> windows host took ages ... so I suspect whatever is causing this
> influences all kvm guest IO.

What's really REALLY weird about this is that we did not alter anything about how we built KVM between these releases.

Tell me, can you run "lockstat sleep <time-of-test>" in the global zone while you run your KVM tests?  They will produce a lot of output, but they may be very informative about what's going on.

Also, I'd be curious if you might (BEs and rpool space being available) upgrade a BE to bloody and repeat your tests?

We don't have the facilities to stress out VMs like this, which is why we didn't notice this before 012 went out the door.  Clearly something's messing up KVM performance (you're not the first to report this, but you seem to have a decent environment for comparisons).  Before the next stable (and incidentally long-term-support as well) release, I hope to have these problems cleared up.  One thing that should happen soon is that Joyent is upstreaming the VND changes into illumos-gate, which will allow us to be fully caught up to their illumos-kvm-cmd source, which we've frozen at revision 1c6181be55d1cadc4426069960688307a6083131 since r151010.

Thanks, and I wish I could be of more immediate assistance!

Dan


From tobi at oetiker.ch  Wed Dec 10 05:14:56 2014
From: tobi at oetiker.ch (Tobias Oetiker)
Date: Wed, 10 Dec 2014 06:14:56 +0100 (CET)
Subject: [OmniOS-discuss] kvm io 10 times slower after r151010 ->
 r151012 upgrade
In-Reply-To: <E7BB97A8-7F0E-4DA3-B1F5-E407975A54F4@omniti.com>
References: <alpine.DEB.2.10.1412100127050.1990@engelberg.oetiker.ch>
	<E7BB97A8-7F0E-4DA3-B1F5-E407975A54F4@omniti.com>
Message-ID: <alpine.DEB.2.10.1412100604030.4496@engelberg.oetiker.ch>

Yesterday Dan McDonald wrote:

> >
> > I have not tested disk performance explicitly, but even booting a
> > windows host took ages ... so I suspect whatever is causing this
> > influences all kvm guest IO.
>
> What's really REALLY weird about this is that we did not alter
> anything about how we built KVM between these releases.
>
> Tell me, can you run "lockstat sleep <time-of-test>" in the
> global zone while you run your KVM tests?  They will produce a
> lot of output, but they may be very informative about what's
> going on.
>

Unfortunately I don't have a spare 'big' iron box to play with, but
we should be able to do some downtime tonight to run that lockstat
sleep experiment and also to do some simple disk io test (with dd).

> Also, I'd be curious if you might (BEs and rpool space being
> available) upgrade a BE to bloody and repeat your tests?

> We don't have the facilities to stress out VMs like this, which
> is why we didn't notice this before 012 went out the door.
> Clearly something's messing up KVM performance (you're not the
> first to report this, but you seem to have a decent environment
> for comparisons).  Before the next stable (and incidentally
> long-term-support as well) release, I hope to have these problems
> cleared up.  One thing that should happen soon is that Joyent is
> upstreaming the VND changes into illumos-gate, which will allow
> us to be fully caught up to their illumos-kvm-cmd source, which
> we've frozen at revision 1c6181be55d1cadc4426069960688307a6083131
> since r151010.

I know that there was no kvm change ... so this must be some side
effect of another modificiation ...

What seems odd, is that only 2 (or 3) few people reported this
problem on the list.  After all, it's not something that was
difficult to notice.  After the upgrade the kvm guests really are
almost un-usable for interactive work involing network or disk IO,
especially when compared to before.

This leads me to suspect, that either only very few people are
using omnios as a kvm server OR it is also a hardware dependent
problem.

I was also wondering if we should try to boot the current smaros on
the box just to see what it does to kvm perf. But as I said, it is
a production machine, so it is all a bit tricky.

cheers
tobi

-- 
Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten, Switzerland
www.oetiker.ch tobi at oetiker.ch +41 62 775 9902


From stephan.budach at JVM.DE  Wed Dec 10 09:16:31 2014
From: stephan.budach at JVM.DE (Stephan Budach)
Date: Wed, 10 Dec 2014 10:16:31 +0100
Subject: [OmniOS-discuss] ipadm hangs on interface creation
Message-ID: <54880F6F.2010904@jvm.de>

Hi all,

I have run into a situation, where ipadm hangs upon trying to create a 
new interface on my ixgbe adaptors. ipadm create-if ixgbe[0|1|2] didn't 
return and either got killed eventually by the system or by kill -9, 
which I issued against that process. I just tried to create another if 
on the on-board NIC igb1, which still works without issues.

What could cause ipadm to hang this way and is there a way to get ipadm 
working again on the ixgbe adaptors, preferreably without having to boot 
the OmniOS server, since it is serving a NFS export from the remaining 
functioning ixgbe3 port to my Oracle VM servers.

Thanks,
budy


From gate03 at landcroft.co.uk  Wed Dec 10 10:15:40 2014
From: gate03 at landcroft.co.uk (Michael Mounteney)
Date: Wed, 10 Dec 2014 20:15:40 +1000
Subject: [OmniOS-discuss] kvm io 10 times slower after r151010 ->
 r151012 upgrade
In-Reply-To: <alpine.DEB.2.10.1412100604030.4496@engelberg.oetiker.ch>
References: <alpine.DEB.2.10.1412100127050.1990@engelberg.oetiker.ch>
	<E7BB97A8-7F0E-4DA3-B1F5-E407975A54F4@omniti.com>
	<alpine.DEB.2.10.1412100604030.4496@engelberg.oetiker.ch>
Message-ID: <20141210201540.6fe5cfee@emeritus>

On Wed, 10 Dec 2014 06:14:56 +0100 (CET)
Tobias Oetiker <tobi at oetiker.ch> wrote:

> This leads me to suspect, that either only very few people are
> using omnios as a kvm server OR it is also a hardware dependent
> problem.

I think it must be.  I'm running KVM (Gentoo Linux guests) and have
just gone from 151010 to 151012.  I haven't carried-out any
quantitative assessment, but didn't notice any slowdown.  For the
record, my KVM invocation is:

/usr/bin/qemu-system-x86_64 \
        -name "Gentoo "$WHAT \
        -cpu host \
        -boot order=d \
        -enable-kvm \
        -vnc cortex:$VNC \
        -smp 1,maxcpus=1,cores=2 \
        -m 1024 \
        -no-hpet \
        -localtime \
        -kernel /gentoo/kernel-source/linux-3.17.4-gentoo-vbox/arch/x86/boot/bzImage \
        -append "root=/dev/vda1 init=/usr/lib/systemd/systemd quiet" \
        -drive file=/dev/zvol/dsk/rpool/vol/Gentoo-KVM-${WHAT},cache=none,if=virtio,index=0 \
        -drive file=/dev/zvol/dsk/rpool/vol/Linuswap-${WHAT},cache=none,if=virtio,index=1 \
        -net nic,vlan=0,macaddr=$mac,model=virtio,name=ncard1 \
        -net vnic,vlan=0,name=net0,ifname=$VNIC,macaddr=$mac \
        -monitor telnet:127.0.0.1:${monitor},server,nowait \
        -vga std \
        -daemonize

where ${WHAT} is either KDE or XFCE.  Machine is a Supermicro 5017C-LF with 1 x Intel Xeon E3-1240V2 3.40 GHz.4 Cores , 4 Threads,8Mb cache and 8 GiB RAM.

If you are interested in any performance figures, let me know any tar or dd etc. commands you'd like me to run.

Michael.

From tobi at oetiker.ch  Wed Dec 10 10:58:27 2014
From: tobi at oetiker.ch (Tobias Oetiker)
Date: Wed, 10 Dec 2014 11:58:27 +0100 (CET)
Subject: [OmniOS-discuss] kvm io 10 times slower after r151010 ->
 r151012 upgrade
In-Reply-To: <20141210201540.6fe5cfee@emeritus>
References: <alpine.DEB.2.10.1412100127050.1990@engelberg.oetiker.ch>
	<E7BB97A8-7F0E-4DA3-B1F5-E407975A54F4@omniti.com>
	<alpine.DEB.2.10.1412100604030.4496@engelberg.oetiker.ch>
	<20141210201540.6fe5cfee@emeritus>
Message-ID: <alpine.DEB.2.10.1412101129310.4496@engelberg.oetiker.ch>

Hi Michael,

Today Michael Mounteney wrote:

> On Wed, 10 Dec 2014 06:14:56 +0100 (CET)
> Tobias Oetiker <tobi at oetiker.ch> wrote:
>
> > This leads me to suspect, that either only very few people are
> > using omnios as a kvm server OR it is also a hardware dependent
> > problem.
>
> I think it must be.  I'm running KVM (Gentoo Linux guests) and have
> just gone from 151010 to 151012.  I haven't carried-out any
> quantitative assessment, but didn't notice any slowdown.  For the
> record, my KVM invocation is:
>
> /usr/bin/qemu-system-x86_64 \
>         -name "Gentoo "$WHAT \
>         -cpu host \
>         -boot order=d \
>         -enable-kvm \
>         -vnc cortex:$VNC \
>         -smp 1,maxcpus=1,cores=2 \
>         -m 1024 \
>         -no-hpet \
>         -localtime \
>         -kernel /gentoo/kernel-source/linux-3.17.4-gentoo-vbox/arch/x86/boot/bzImage \
>         -append "root=/dev/vda1 init=/usr/lib/systemd/systemd quiet" \
>         -drive file=/dev/zvol/dsk/rpool/vol/Gentoo-KVM-${WHAT},cache=none,if=virtio,index=0 \
>         -drive file=/dev/zvol/dsk/rpool/vol/Linuswap-${WHAT},cache=none,if=virtio,index=1 \
>         -net nic,vlan=0,macaddr=$mac,model=virtio,name=ncard1 \
>         -net vnic,vlan=0,name=net0,ifname=$VNIC,macaddr=$mac \
>         -monitor telnet:127.0.0.1:${monitor},server,nowait \
>         -vga std \
>         -daemonize
>
> where ${WHAT} is either KDE or XFCE.  Machine is a Supermicro 5017C-LF with 1 x Intel Xeon E3-1240V2 3.40 GHz.4 Cores , 4 Threads,8Mb cache and 8 GiB RAM.
>
> If you are interested in any performance figures, let me know any tar or dd etc. commands you'd like me to run.

yes, a very simple test:

guest$ dd if=/dev/zero bs=1M count=20 | ssh host dd of=/dev/null
guest$ ssh host dd if=/dev/zero bs=1M count=20 | dd of=/dev/null

and since I suspect that the disk io suffers too but due to
caching, maybe reading just a bit off the disk device might be an
interesting test:

  dd if=/dev/vda of=/dev/zero bs=1M count=1000

cheers
tobi


> Michael.
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>
>

-- 
Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten, Switzerland
www.oetiker.ch tobi at oetiker.ch +41 62 775 9902


From chip at innovates.com  Wed Dec 10 15:37:28 2014
From: chip at innovates.com (Schweiss, Chip)
Date: Wed, 10 Dec 2014 09:37:28 -0600
Subject: [OmniOS-discuss] Parity error on path mpt_sas2
In-Reply-To: <54871A67.3040805@gmx.li>
References: <3BE0DEED8863E5429BAE4CAEDF624565039C1B0DAAD0@AIRA-SRV.aira.local>
	<1439760B-8D8B-40F9-8A62-4CFE4A9483C1@omniti.com>
	<3BE0DEED8863E5429BAE4CAEDF624565039C1B0DAADA@AIRA-SRV.aira.local>
	<CAOqBcP_12FaH77o+3nrGVm7KU3x1XCk3B-9uenKRRL63t5wdyg@mail.gmail.com>
	<3BE0DEED8863E5429BAE4CAEDF624565039C1B0DACD1@AIRA-SRV.aira.local>
	<CALeZrrRhsV5i2AJF2aRrLxQvR9LSckiEax59gX5XXAYAkpCS+Q@mail.gmail.com>
	<004501d01302$257c9b90$7075d2b0$@subrigo.net>
	<3BE0DEED8863E5429BAE4CAEDF624565039C1B0D8E3F@AIRA-SRV.aira.local>
	<CALeZrrTqymmNX3E=NpZr2kJokFHK6xB850NLjDh12Wo9o4Mcng@mail.gmail.com>
	<54871A67.3040805@gmx.li>
Message-ID: <CALeZrrSNXXV_aXyo2bP==7OQ2_h8uC1UZEK=82gA6bK8Go889Q@mail.gmail.com>

On Tue, Dec 9, 2014 at 9:51 AM, Dominik Hassler <hasslerd at gmx.li> wrote:

> Chip,
>
> when I had to reflash an onboard LSI2308 I used the UEFI shell. No
> fiddling around with boot disks etc.
>
> Not sure, if it is applicable to your problem.
>

That was the trick.  Under DOS I kept getting "ERROR: Failed to initialize
PAL."  Turns out this is a problem on X9 and newer Supermicro boards.

With the efi sas2flash version on a USB stick it was quite easy.  The
system BIOS has a boot override to drop into the UEFI shell from there I
could see the USB stick and run the erase and flash.

Thanks for everyone's input on this.  Hopefully I will only be upgrading in
the future and can stick to the Solaris version of sas2flash.

-Chip


> On 12/09/2014 03:49 PM, Schweiss, Chip wrote:
> > I'm still fighting this.
> >
> > Under OmniOS the erase function is disabled in sas2flash for solaris.
> >
> > I couldn't seem to find an iso bootable DOS that I could stage files
> > in.  FreeDOS1.0 won't mount the CD, 1.1 got rid of the live mode.
> > Tried a Linux rescue CD and I get " ERROR: Erase Flash Operation
> > Failed!"   Tried both P18 sas2flash and P20 sas2flash.
> >
> > What OS environment are you running this in?
> >
> >
> >
> > On Tue, Dec 9, 2014 at 3:06 AM, Filip Marvan <filip.marvan at aira.cz
> > <mailto:filip.marvan at aira.cz>> wrote:
> >
> >     Hi,____
> >
> >     __ __
> >
> >     first you have to erase P20 firmware with:____
> >
> >     __ __
> >
> >     sas2flsh -o -e 6____
> >
> >     __ __
> >
> >     now do not reboot(!) and flash P18 with____
> >
> >     __ __
> >
> >     sas2flsh -f XXXXXX.bin -b mptsas2.rom____
> >
> >     __ __
> >
> >     Filip____
> >
> >     __ __
> >
> >     __ __
> >
> >     *From:*OmniOS-discuss
> >     [mailto:omnios-discuss-bounces at lists.omniti.com
> >     <mailto:omnios-discuss-bounces at lists.omniti.com>] *On Behalf Of
> >     *Schweiss, Chip
> >     *Sent:* Monday, December 08, 2014 08:09 AM
> >     *To:* Filip Marvan
> >     *Cc:* omnios-discuss at lists.omniti.com
> >     <mailto:omnios-discuss at lists.omniti.com>
> >     *Subject:* Re: [OmniOS-discuss] Parity error on path mpt_sas2____
> >
> >     __ __
> >
> >     I've got some new LSI HBAs I'm trying to downgrade from firmware
> >     version 20 to 18.____
> >
> >     I'm getting errors when trying to downgrade:
> >
> >     Attempting to flash firmware to LSI SAS SAS2308_2(D1) :
> >
> >             Executing Operation: Flash Firmware Image
> >
> >                     Firmware Image has a Valid Checksum.
> >                     Firmware Version 18.00.00.00
> >                     Firmware Image compatible with Controller.
> >
> >                     Valid NVDATA Image found.
> >                     NVDATA Version 11.00.00.00
> >                     Checking for a compatible NVData image...
> >
> >                     NVDATA Device ID and Chip Revision match verified.
> >                     ERROR: Cannot downgrade NVDATA version 14.00.00.06
> >                            to 11.00.110000.00.
> >
> >                     ERROR: Failed to get valid NVDATA image from File!
> >
> >                     Firmware Image Validation Failed!____
> >
> >     Tried downgrading bios:
> >
> >     Attempting to flash Boot Service to LSI SAS SAS2308_2(D1) :
> >
> >                     Validating BIOS Image...
> >
> >                     BIOS Header Signature is Valid
> >
> >                     BIOS Image has a Valid Checksum.
> >
> >                     BIOS PCI Structure Signature Valid.
> >
> >                     BIOS Image Compatible with the SAS Controller.
> >
> >                     Attempting to Flash BIOS Image...
> >
> >                     BIOS Version in flash    : 07.39.00.00
> >                     BIOS Version from File  : 07.35.00.00
> >                     Skipping flash since file version is not greater
> >     than existing.
> >
> >                     Flash BIOS Image Failed!____
> >
> >     I've tried sas2flash from P20 and P18.
> >
> >     Can someone fill me in on the trick to downgrade?  ____
> >
> >     -Chip____
> >
> >     __ __
> >
> >     __ __
> >
> >     On Thu, Nov 27, 2014 at 9:03 AM, Filip Marvan <filip.marvan at aira.cz
> >     <mailto:filip.marvan at aira.cz>> wrote:____
> >
> >     Thank you for your help Aaron! I downgraded firmware to P18 and
> >     there are no more errors today.____
> >
> >     It seems, that there is something bad with P20 firmware!____
> >
> >      ____
> >
> >     You saved me a lot of time J____
> >
> >      ____
> >
> >     Filip____
> >
> >      ____
> >
> >      ____
> >
> >     *From:*Aaron Curry [mailto:asc1111 at gmail.com
> >     <mailto:asc1111 at gmail.com>]
> >     *Sent:* Tuesday, November 25, 2014 5:46 PM
> >     *To:* Filip Marvan
> >     *Cc:* Dan McDonald; omnios-discuss at lists.omniti.com
> >     <mailto:omnios-discuss at lists.omniti.com>
> >     *Subject:* Re: [OmniOS-discuss] Parity error on path mpt_sas2____
> >
> >      ____
> >
> >     I had this exact same problem recently when setting up a new home
> >     server... same controller, same firmware (P20). The errors were on
> >     all disks attached to the controller, but only on high read
> >     activity. Writes did not generate the errors. I downgraded the
> >     firmware to P18 (which is what we are using at work) and the errors
> >     went away. Has anyone had success with this controller and the P20
> >     firmware?____
> >
> >      ____
> >
> >     Aaron____
> >
> >      ____
> >
> >     On Tue, Nov 25, 2014 at 8:30 AM, Filip Marvan <filip.marvan at aira.cz
> >     <mailto:filip.marvan at aira.cz>> wrote:____
> >
> >         Hi Dan,
> >
> >         thanks for reply.
> >         Yes, errors are on all 5 disks in the same RAIDZ pool. No
> >         problem on other
> >         disks in different pool but on the same SAS cable.
> >         So maybe I only had bad luck with that disks.
> >
> >         Filip
> >
> >
> >         _______________________________________________
> >         OmniOS-discuss mailing list
> >         OmniOS-discuss at lists.omniti.com
> >         <mailto:OmniOS-discuss at lists.omniti.com>
> >         http://lists.omniti.com/mailman/listinfo/omnios-discuss____
> >
> >      ____
> >
> >
> >     _______________________________________________
> >     OmniOS-discuss mailing list
> >     OmniOS-discuss at lists.omniti.com <mailto:
> OmniOS-discuss at lists.omniti.com>
> >     http://lists.omniti.com/mailman/listinfo/omnios-discuss____
> >
> >     __ __
> >
> >
> >
> >
> > _______________________________________________
> > OmniOS-discuss mailing list
> > OmniOS-discuss at lists.omniti.com
> > http://lists.omniti.com/mailman/listinfo/omnios-discuss
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141210/05e19e40/attachment-0001.html>

From danmcd at omniti.com  Wed Dec 10 15:53:21 2014
From: danmcd at omniti.com (Dan McDonald)
Date: Wed, 10 Dec 2014 10:53:21 -0500
Subject: [OmniOS-discuss] ipadm hangs on interface creation
In-Reply-To: <54880F6F.2010904@jvm.de>
References: <54880F6F.2010904@jvm.de>
Message-ID: <2636F307-5701-42BD-8765-E4AAD2ED61EE@omniti.com>


> On Dec 10, 2014, at 4:16 AM, Stephan Budach <stephan.budach at JVM.DE> wrote:
> 
> Hi all,
> 
> I have run into a situation, where ipadm hangs upon trying to create a new interface on my ixgbe adaptors. ipadm create-if ixgbe[0|1|2] didn't return and either got killed eventually by the system or by kill -9, which I issued against that process. I just tried to create another if on the on-board NIC igb1, which still works without issues.

What does "pstack `pgrep ipadm`" say when they hang?  And what's your exact incantation?  And "killed by the system", what exactly do you mean?  Did ipadm(1M) time out?

> What could cause ipadm to hang this way and is there a way to get ipadm working again on the ixgbe adaptors, preferreably without having to boot the OmniOS server, since it is serving a NFS export from the remaining functioning ixgbe3 port to my Oracle VM servers.

I'll need more information before I can help you.

Dan


From stephan.budach at JVM.DE  Wed Dec 10 16:12:10 2014
From: stephan.budach at JVM.DE (Stephan Budach)
Date: Wed, 10 Dec 2014 17:12:10 +0100
Subject: [OmniOS-discuss] ipadm hangs on interface creation
In-Reply-To: <2636F307-5701-42BD-8765-E4AAD2ED61EE@omniti.com>
References: <54880F6F.2010904@jvm.de>
	<2636F307-5701-42BD-8765-E4AAD2ED61EE@omniti.com>
Message-ID: <548870DA.6080308@jvm.de>

Hi Dan,

I actually don't know the term "incantation" yet, but I assume, that you 
wanted to know the release of OmniOS I am running? If not, please 
correct me on that.
What is really funny, is that running pstack against the ipadm process, 
somehow brought things back in line:

root at nfsvmpool01:~# pkg info entire
           Name: entire
        Summary: Incorporation to constrain core system packages to same 
build
    Description: This package constrains core system package versions to 
the same
                 build and provides a minimal set of additional packages.
          State: Installed
      Publisher: omnios
        Version: 11
  Build Release: 5.11
         Branch: 0.151006
Packaging Date: Mon Oct 27 19:15:00 2014
           Size: 0.00 B
           FMRI: pkg://omnios/entire at 11,5.11-0.151006:20141027T191500Z

I was running ipadm create-if ixgbe3 in another session when I ran 
pstack as suggested:

root at nfsvmpool01:~# pstack `pgrep ipadm`
12397:    ipadm create-if ixgbe3
  fef07723 open     (8047190, 2, fec9404c)
  feef24f8 open     (8047190, 2, fec9404c, 8068db8, 11, feffb0a4) + 105
  fec935f8 i_dlpi_open (8068db8, 80475dc, 10, 1, 8047f00, 5) + ed
  fec93768 i_dlpi_style1_open (8068db0, 804761c, 20, fec9386d, 3, 3) + 2a
  fec939a8 dlpi_open (8047f00, 8047cd4, 10, fedf1541) + 149
  fedf17fb i_ipadm_plumb_if (8068968, 8047f00, 2, 3) + 2cb
  fedf0ed4 i_ipadm_create_if (8068968, 8047f00, 2, 3, 0, 3) + 9a
  fedf0fc2 ipadm_create_if (8068968, 8047f00, 0, 3) + e3
  0805553e do_create_if (2) + 8f
  08052e72 main     (80555ca, feffb0a4, 8047e30, 80525c7, 3, 8047e3c) + df
  080525c7 _start   (3, 8047ef0, 8047ef6, 8047f00, 0, 8047f07) + 83

ipadm returned and created the interface, just as if nothing ever 
happened. Afterwards I was able to create and delete interfaces without 
any issue on the ixgbe adaptors.

Cheers,
budy


Am 10.12.14 um 16:53 schrieb Dan McDonald:
>> On Dec 10, 2014, at 4:16 AM, Stephan Budach <stephan.budach at JVM.DE> wrote:
>>
>> Hi all,
>>
>> I have run into a situation, where ipadm hangs upon trying to create a new interface on my ixgbe adaptors. ipadm create-if ixgbe[0|1|2] didn't return and either got killed eventually by the system or by kill -9, which I issued against that process. I just tried to create another if on the on-board NIC igb1, which still works without issues.
> What does "pstack `pgrep ipadm`" say when they hang?  And what's your exact incantation?  And "killed by the system", what exactly do you mean?  Did ipadm(1M) time out?
>
>> What could cause ipadm to hang this way and is there a way to get ipadm working again on the ixgbe adaptors, preferreably without having to boot the OmniOS server, since it is serving a NFS export from the remaining functioning ixgbe3 port to my Oracle VM servers.
> I'll need more information before I can help you.
>
> Dan
>


-- 
Stephan Budach
Deputy Managing Director
Jung von Matt/it-services GmbH
Glash?ttenstra?e 79
20357 Hamburg


Tel: +49 40-4321-1353
Fax: +49 40-4321-1114
E-Mail: stephan.budach at jvm.de
Internet: http://www.jvm.com

Gesch?ftsf?hrer: Frank Wilhelm, Stephan Budach (stellv.)
AG HH HRB 98380


From danmcd at omniti.com  Wed Dec 10 16:20:46 2014
From: danmcd at omniti.com (Dan McDonald)
Date: Wed, 10 Dec 2014 11:20:46 -0500
Subject: [OmniOS-discuss] ipadm hangs on interface creation
In-Reply-To: <548870DA.6080308@jvm.de>
References: <54880F6F.2010904@jvm.de>
	<2636F307-5701-42BD-8765-E4AAD2ED61EE@omniti.com>
	<548870DA.6080308@jvm.de>
Message-ID: <E948E1BE-A338-46EC-BB7F-4C179DA006C1@omniti.com>


> On Dec 10, 2014, at 11:12 AM, Stephan Budach <stephan.budach at jvm.de> wrote:
> 
> Hi Dan,
> 
> I actually don't know the term "incantation" yet, but I assume, that you wanted to know the release of OmniOS I am running?

I meant what exact command-line arguments you typed for ipadm(1M).  But that's okay, because it's good to know the other things you're about to show me.

> What is really funny, is that running pstack against the ipadm process, somehow brought things back in line:
> 
> root at nfsvmpool01:~# pkg info entire
>          Name: entire
>       Summary: Incorporation to constrain core system packages to same build
>   Description: This package constrains core system package versions to the same
>                build and provides a minimal set of additional packages.
>         State: Installed
>     Publisher: omnios
>       Version: 11
> Build Release: 5.11
>        Branch: 0.151006
> Packaging Date: Mon Oct 27 19:15:00 2014
>          Size: 0.00 B
>          FMRI: pkg://omnios/entire at 11,5.11-0.151006:20141027T191500Z
> 
> I was running ipadm create-if ixgbe3 in another session when I ran pstack as suggested:
> 
> root at nfsvmpool01:~# pstack `pgrep ipadm`
> 12397:    ipadm create-if ixgbe3
> fef07723 open     (8047190, 2, fec9404c)
> feef24f8 open     (8047190, 2, fec9404c, 8068db8, 11, feffb0a4) + 105
> fec935f8 i_dlpi_open (8068db8, 80475dc, 10, 1, 8047f00, 5) + ed
> fec93768 i_dlpi_style1_open (8068db0, 804761c, 20, fec9386d, 3, 3) + 2a
> fec939a8 dlpi_open (8047f00, 8047cd4, 10, fedf1541) + 149
> fedf17fb i_ipadm_plumb_if (8068968, 8047f00, 2, 3) + 2cb
> fedf0ed4 i_ipadm_create_if (8068968, 8047f00, 2, 3, 0, 3) + 9a
> fedf0fc2 ipadm_create_if (8068968, 8047f00, 0, 3) + e3
> 0805553e do_create_if (2) + 8f
> 08052e72 main     (80555ca, feffb0a4, 8047e30, 80525c7, 3, 8047e3c) + df
> 080525c7 _start   (3, 8047ef0, 8047ef6, 8047f00, 0, 8047f07) + 83
> 
> ipadm returned and created the interface, just as if nothing ever happened. Afterwards I was able to create and delete interfaces without any issue on the ixgbe adaptors.

That's VERY odd, albeit with pleasant results.  I wouldn't think pstack would unblock something like what you showed me.  You're running 006 (long-term support), but I'm not seeing any major changes in the plumbing of interfaces between then and now.

You mentioned a desire to not reboot your system.  Did you plug in these ixgbes while booted (i.e. hotplugging)?  Or were they always there, and you just hadn't configured them until now?

It seems like you tickled a race condition or some other odd combination of circumstances that inspecting the ipadm(1M) process jostled free.  I don't have any ixgbe handy at the moment, but had I, I'd like to know if this is a reproducible problem.  The only theory I can think of is that something weird happened while ipadm was communicating with ipmgmtd, and inspecting the process with pstack allowed a context switch to occur.

Thank you, and while it's nice to see things working, I'm sorry I don't have a better explanation about what happened, or the ability to immediately reproduce this bug.

Dan


From stephan.budach at JVM.DE  Wed Dec 10 16:38:20 2014
From: stephan.budach at JVM.DE (Stephan Budach)
Date: Wed, 10 Dec 2014 17:38:20 +0100
Subject: [OmniOS-discuss] ipadm hangs on interface creation
In-Reply-To: <E948E1BE-A338-46EC-BB7F-4C179DA006C1@omniti.com>
References: <54880F6F.2010904@jvm.de>
	<2636F307-5701-42BD-8765-E4AAD2ED61EE@omniti.com>
	<548870DA.6080308@jvm.de>
	<E948E1BE-A338-46EC-BB7F-4C179DA006C1@omniti.com>
Message-ID: <548876FC.9040108@jvm.de>

Am 10.12.14 um 17:20 schrieb Dan McDonald:
>> On Dec 10, 2014, at 11:12 AM, Stephan Budach <stephan.budach at jvm.de> wrote:
>>
>> Hi Dan,
>>
>> I actually don't know the term "incantation" yet, but I assume, that you wanted to know the release of OmniOS I am running?
> I meant what exact command-line arguments you typed for ipadm(1M).  But that's okay, because it's good to know the other things you're about to show me.
>
>> What is really funny, is that running pstack against the ipadm process, somehow brought things back in line:
>>
>> root at nfsvmpool01:~# pkg info entire
>>           Name: entire
>>        Summary: Incorporation to constrain core system packages to same build
>>    Description: This package constrains core system package versions to the same
>>                 build and provides a minimal set of additional packages.
>>          State: Installed
>>      Publisher: omnios
>>        Version: 11
>> Build Release: 5.11
>>         Branch: 0.151006
>> Packaging Date: Mon Oct 27 19:15:00 2014
>>           Size: 0.00 B
>>           FMRI: pkg://omnios/entire at 11,5.11-0.151006:20141027T191500Z
>>
>> I was running ipadm create-if ixgbe3 in another session when I ran pstack as suggested:
>>
>> root at nfsvmpool01:~# pstack `pgrep ipadm`
>> 12397:    ipadm create-if ixgbe3
>> fef07723 open     (8047190, 2, fec9404c)
>> feef24f8 open     (8047190, 2, fec9404c, 8068db8, 11, feffb0a4) + 105
>> fec935f8 i_dlpi_open (8068db8, 80475dc, 10, 1, 8047f00, 5) + ed
>> fec93768 i_dlpi_style1_open (8068db0, 804761c, 20, fec9386d, 3, 3) + 2a
>> fec939a8 dlpi_open (8047f00, 8047cd4, 10, fedf1541) + 149
>> fedf17fb i_ipadm_plumb_if (8068968, 8047f00, 2, 3) + 2cb
>> fedf0ed4 i_ipadm_create_if (8068968, 8047f00, 2, 3, 0, 3) + 9a
>> fedf0fc2 ipadm_create_if (8068968, 8047f00, 0, 3) + e3
>> 0805553e do_create_if (2) + 8f
>> 08052e72 main     (80555ca, feffb0a4, 8047e30, 80525c7, 3, 8047e3c) + df
>> 080525c7 _start   (3, 8047ef0, 8047ef6, 8047f00, 0, 8047f07) + 83
>>
>> ipadm returned and created the interface, just as if nothing ever happened. Afterwards I was able to create and delete interfaces without any issue on the ixgbe adaptors.
> That's VERY odd, albeit with pleasant results.  I wouldn't think pstack would unblock something like what you showed me.  You're running 006 (long-term support), but I'm not seeing any major changes in the plumbing of interfaces between then and now.
Yes, it is? ;)
>
> You mentioned a desire to not reboot your system.  Did you plug in these ixgbes while booted (i.e. hotplugging)?  Or were they always there, and you just hadn't configured them until now?
The two cards had been in the system right from the start and I already 
configured one port (ixgbe0) to connect to a 10GbE port on our 6509, 
which went without issues. I wanted to resolve this issue without 
rebooting, since rebooting would have killed 53 VMs that are residing on 
my NFS export that is running over ixgbe0.
> It seems like you tickled a race condition or some other odd combination of circumstances that inspecting the ipadm(1M) process jostled free.  I don't have any ixgbe handy at the moment, but had I, I'd like to know if this is a reproducible problem.  The only theory I can think of is that something weird happened while ipadm was communicating with ipmgmtd, and inspecting the process with pstack allowed a context switch to occur.
I can maybe help with that as well? I tried to set the MTU to the 
maximum, that the driver would accept and I ended up with a mtu of 15500 
(what a odd number anyway) which I set using dladm on ixgbe3. When i 
tried to create the interface using ipadm, that was the moment, when 
things went quirky. Although I trimmed the mtu down to a reasonable 9216 
ipadm wouldn't create any longer and would hang at creating interfaces 
on any of the remaining 3 ports of my T540-X2's.
>
> Thank you, and while it's nice to see things working, I'm sorry I don't have a better explanation about what happened, or the ability to immediately reproduce this bug.
>
> Dan
>
Ha ha, no prob - you already provided a great deal of help!

Thanks a lot,
budy

From ikaufman at eng.ucsd.edu  Wed Dec 10 17:01:43 2014
From: ikaufman at eng.ucsd.edu (Ian Kaufman)
Date: Wed, 10 Dec 2014 09:01:43 -0800
Subject: [OmniOS-discuss] weird library issue
Message-ID: <CAPJtH1ghB83XiJonib0qxTu3oZ=28XcKFOj7FHRm9hu-AB+Kuw@mail.gmail.com>

Hi all,

I have a situation where a bunch of libs are complaining about missing
the following:

libc.so.1 (ILLUMOS_0.7)

or

libc.so.1 (ILLUMOS_0.6)

This is happening on one system, but I have others that are fine. A
standard libc.so.1 exists, but various libs are looking for the tagged
one.

Any ideas?

Thanks,

Ian

-- 
Ian Kaufman
Research Systems Administrator
UC San Diego, Jacobs School of Engineering ikaufman AT ucsd DOT edu

From danmcd at omniti.com  Wed Dec 10 18:15:54 2014
From: danmcd at omniti.com (Dan McDonald)
Date: Wed, 10 Dec 2014 13:15:54 -0500
Subject: [OmniOS-discuss] weird library issue
In-Reply-To: <CAPJtH1ghB83XiJonib0qxTu3oZ=28XcKFOj7FHRm9hu-AB+Kuw@mail.gmail.com>
References: <CAPJtH1ghB83XiJonib0qxTu3oZ=28XcKFOj7FHRm9hu-AB+Kuw@mail.gmail.com>
Message-ID: <B3CAABA1-3A42-4CA4-9F20-D825E03BA499@omniti.com>


> On Dec 10, 2014, at 12:01 PM, Ian Kaufman <ikaufman at eng.ucsd.edu> wrote:
> 
> Hi all,
> 
> I have a situation where a bunch of libs are complaining about missing
> the following:
> 
> libc.so.1 (ILLUMOS_0.7)
> 
> or
> 
> libc.so.1 (ILLUMOS_0.6)
> 
> This is happening on one system, but I have others that are fine. A
> standard libc.so.1 exists, but various libs are looking for the tagged
> one.
> 
> Any ideas?

You'll need to provide more data.  It's as if something you compiled was compiled against a later version of libc, and then moved to an earlier version of libc.

Dan


From ikaufman at eng.ucsd.edu  Wed Dec 10 18:42:35 2014
From: ikaufman at eng.ucsd.edu (Ian Kaufman)
Date: Wed, 10 Dec 2014 10:42:35 -0800
Subject: [OmniOS-discuss] weird library issue
In-Reply-To: <B3CAABA1-3A42-4CA4-9F20-D825E03BA499@omniti.com>
References: <CAPJtH1ghB83XiJonib0qxTu3oZ=28XcKFOj7FHRm9hu-AB+Kuw@mail.gmail.com>
	<B3CAABA1-3A42-4CA4-9F20-D825E03BA499@omniti.com>
Message-ID: <CAPJtH1hkxOAeB0Uf_Dbww6xiWeav83BOOhoRun_8JNTfAFE+sA@mail.gmail.com>

It looks like this system is in some weird state where there are
packages from 151006 and 151008.

For example, /etc/release shows:

root at massive02-b:~# cat /etc/release
  OmniOS v11 r151008
  Copyright 2013 OmniTI Computer Consulting, Inc. All rights reserved.
  Use is subject to license terms.

but, when I try to do a pkg update, I see this (multiple repeating lines):

  No suitable version of required package
pkg://omnios/incorporation/jeos/omnios-userland at 11,5.11-0.151008:20131204T231613Z
found:
    Reject:  pkg://omnios/incorporation/jeos/omnios-userland at 11,5.11-0.151008:20131204T231613Z
    Reason:  A version for 'incorporate' dependency on
pkg:/library/security/openssl at 1.0.1,5.11-0.151008 cannot be found
  No suitable version of required package
pkg://omnios/incorporation/jeos/omnios-userland at 11,5.11-0.151004:20121024T180535Z
found:
    Reject:  pkg://omnios/incorporation/jeos/omnios-userland at 11,5.11-0.151004:20121024T180535Z
    Reason:  A version for 'incorporate' dependency on
pkg:/archiver/gnu-tar at 1.26,5.11-0.151004 cannot be found
  No suitable version of required package
pkg://omnios/incorporation/jeos/omnios-userland at 11,5.11-0.151006:20130506T214442Z
found:
    Reject:  pkg://omnios/incorporation/jeos/omnios-userland at 11,5.11-0.151006:20130506T214442Z
    Reason:  A version for 'incorporate' dependency on
pkg:/archiver/gnu-tar at 1.26,5.11-0.151006 cannot be found

and for openssl, I see this:

root at massive02-b:~# pkg info library/security/openssl
          Name: library/security/openssl
       Summary: openssl - A toolkit for Secure Sockets Layer (SSL
v2/v3) and Transport Layer (TLS v1) protocols and general purpose
cryptographic library
         State: Installed
     Publisher: omnios
       Version: 1.0.1.10 (1.0.1j)
 Build Release: 5.11
        Branch: 0.151006
Packaging Date: October 15, 2014 03:40:22 PM
          Size: 18.08 MB
          FMRI:
pkg://omnios/library/security/openssl at 1.0.1.10,5.11-0.151006:20141015T154022Z
root at massive02-b:~# pkg install
pkg:/library/security/openssl at 1.0.1,5.11-0.151008
Creating Plan /
pkg install: No matching version of library/security/openssl can be installed:
  Reject:  pkg://omnios/library/security/openssl at 1.0.1.5,5.11-0.151008:20131204T024253Z
           pkg://omnios/library/security/openssl at 1.0.1.6,5.11-0.151008:20140110T173804Z
           pkg://omnios/library/security/openssl at 1.0.1.7,5.11-0.151008:20140407T220403Z
           pkg://omnios/library/security/openssl at 1.0.1.7,5.11-0.151008:20140408T142844Z
           pkg://omnios/library/security/openssl at 1.0.1.8,5.11-0.151008:20140605T140630Z
           pkg://omnios/library/security/openssl at 1.0.1.9,5.11-0.151008:20140807T035111Z
  Reason:  Newer version
pkg://omnios/library/security/openssl at 1.0.1.10,5.11-0.151006:20141015T154022Z
is already installed

Ian

On Wed, Dec 10, 2014 at 10:15 AM, Dan McDonald <danmcd at omniti.com> wrote:
>
>> On Dec 10, 2014, at 12:01 PM, Ian Kaufman <ikaufman at eng.ucsd.edu> wrote:
>>
>> Hi all,
>>
>> I have a situation where a bunch of libs are complaining about missing
>> the following:
>>
>> libc.so.1 (ILLUMOS_0.7)
>>
>> or
>>
>> libc.so.1 (ILLUMOS_0.6)
>>
>> This is happening on one system, but I have others that are fine. A
>> standard libc.so.1 exists, but various libs are looking for the tagged
>> one.
>>
>> Any ideas?
>
> You'll need to provide more data.  It's as if something you compiled was compiled against a later version of libc, and then moved to an earlier version of libc.
>
> Dan
>



-- 
Ian Kaufman
Research Systems Administrator
UC San Diego, Jacobs School of Engineering ikaufman AT ucsd DOT edu

From danmcd at omniti.com  Wed Dec 10 18:55:00 2014
From: danmcd at omniti.com (Dan McDonald)
Date: Wed, 10 Dec 2014 13:55:00 -0500
Subject: [OmniOS-discuss] weird library issue
In-Reply-To: <CAPJtH1hkxOAeB0Uf_Dbww6xiWeav83BOOhoRun_8JNTfAFE+sA@mail.gmail.com>
References: <CAPJtH1ghB83XiJonib0qxTu3oZ=28XcKFOj7FHRm9hu-AB+Kuw@mail.gmail.com>
	<B3CAABA1-3A42-4CA4-9F20-D825E03BA499@omniti.com>
	<CAPJtH1hkxOAeB0Uf_Dbww6xiWeav83BOOhoRun_8JNTfAFE+sA@mail.gmail.com>
Message-ID: <A4828F6E-1C68-4107-9C5D-00363774C73F@omniti.com>


> On Dec 10, 2014, at 1:42 PM, Ian Kaufman <ikaufman at eng.ucsd.edu> wrote:
> 
> It looks like this system is in some weird state where there are
> packages from 151006 and 151008.

Oh my.

Yes, you've been bitten by the pkg upgrade problems between 006 and 008.  This was before my time here, so I'm not sure how best to extricate yourself from things.

If you want, you can jump forward to the current stable release (r151012).  If you want to try that, and you don't have zones, do this:


1.) Create a new BE, call it 012 for this example:  beadm create 012

2.) Mount the BE on /mnt:	beadm mount 012 /mnt

3.) Change the publisher on the new BE to point to r151012's repo:
	pkg -R /mnt set-publisher -G http://pkg.omniti.com/omnios/release/ -g http://pkg.omniti.com/omnios/r151012/ omnios

4.) Apply the update to the new BE:   pkg -R /mnt update

5.) Unmount the BE:   beadm umount 012

6.) Either reboot and use the GRUB menu to try out the new 012 BE, or "beadm activate 012" to make it go there on the next reboot.


Dan


From stephan.budach at JVM.DE  Wed Dec 10 19:07:47 2014
From: stephan.budach at JVM.DE (Stephan Budach)
Date: Wed, 10 Dec 2014 20:07:47 +0100
Subject: [OmniOS-discuss] ipadm hangs on interface creation
In-Reply-To: <548876FC.9040108@jvm.de>
References: <54880F6F.2010904@jvm.de>
	<2636F307-5701-42BD-8765-E4AAD2ED61EE@omniti.com>
	<548870DA.6080308@jvm.de>
	<E948E1BE-A338-46EC-BB7F-4C179DA006C1@omniti.com>
	<548876FC.9040108@jvm.de>
Message-ID: <54889A03.7080307@jvm.de>

Am 10.12.14 um 17:38 schrieb Stephan Budach:
> Am 10.12.14 um 17:20 schrieb Dan McDonald:
>>> On Dec 10, 2014, at 11:12 AM, Stephan Budach <stephan.budach at jvm.de> 
>>> wrote:
>>>
>>> Hi Dan,
>>>
>>> I actually don't know the term "incantation" yet, but I assume, that 
>>> you wanted to know the release of OmniOS I am running?
>> I meant what exact command-line arguments you typed for ipadm(1M).  
>> But that's okay, because it's good to know the other things you're 
>> about to show me.
>>
>>> What is really funny, is that running pstack against the ipadm 
>>> process, somehow brought things back in line:
>>>
>>> root at nfsvmpool01:~# pkg info entire
>>>           Name: entire
>>>        Summary: Incorporation to constrain core system packages to 
>>> same build
>>>    Description: This package constrains core system package versions 
>>> to the same
>>>                 build and provides a minimal set of additional 
>>> packages.
>>>          State: Installed
>>>      Publisher: omnios
>>>        Version: 11
>>> Build Release: 5.11
>>>         Branch: 0.151006
>>> Packaging Date: Mon Oct 27 19:15:00 2014
>>>           Size: 0.00 B
>>>           FMRI: pkg://omnios/entire at 11,5.11-0.151006:20141027T191500Z
>>>
>>> I was running ipadm create-if ixgbe3 in another session when I ran 
>>> pstack as suggested:
>>>
>>> root at nfsvmpool01:~# pstack `pgrep ipadm`
>>> 12397:    ipadm create-if ixgbe3
>>> fef07723 open     (8047190, 2, fec9404c)
>>> feef24f8 open     (8047190, 2, fec9404c, 8068db8, 11, feffb0a4) + 105
>>> fec935f8 i_dlpi_open (8068db8, 80475dc, 10, 1, 8047f00, 5) + ed
>>> fec93768 i_dlpi_style1_open (8068db0, 804761c, 20, fec9386d, 3, 3) + 2a
>>> fec939a8 dlpi_open (8047f00, 8047cd4, 10, fedf1541) + 149
>>> fedf17fb i_ipadm_plumb_if (8068968, 8047f00, 2, 3) + 2cb
>>> fedf0ed4 i_ipadm_create_if (8068968, 8047f00, 2, 3, 0, 3) + 9a
>>> fedf0fc2 ipadm_create_if (8068968, 8047f00, 0, 3) + e3
>>> 0805553e do_create_if (2) + 8f
>>> 08052e72 main     (80555ca, feffb0a4, 8047e30, 80525c7, 3, 8047e3c) 
>>> + df
>>> 080525c7 _start   (3, 8047ef0, 8047ef6, 8047f00, 0, 8047f07) + 83
>>>
>>> ipadm returned and created the interface, just as if nothing ever 
>>> happened. Afterwards I was able to create and delete interfaces 
>>> without any issue on the ixgbe adaptors.
>> That's VERY odd, albeit with pleasant results.  I wouldn't think 
>> pstack would unblock something like what you showed me.  You're 
>> running 006 (long-term support), but I'm not seeing any major changes 
>> in the plumbing of interfaces between then and now.
> Yes, it is? ;)
>>
>> You mentioned a desire to not reboot your system.  Did you plug in 
>> these ixgbes while booted (i.e. hotplugging)?  Or were they always 
>> there, and you just hadn't configured them until now?
> The two cards had been in the system right from the start and I 
> already configured one port (ixgbe0) to connect to a 10GbE port on our 
> 6509, which went without issues. I wanted to resolve this issue 
> without rebooting, since rebooting would have killed 53 VMs that are 
> residing on my NFS export that is running over ixgbe0.
>> It seems like you tickled a race condition or some other odd 
>> combination of circumstances that inspecting the ipadm(1M) process 
>> jostled free.  I don't have any ixgbe handy at the moment, but had I, 
>> I'd like to know if this is a reproducible problem.  The only theory 
>> I can think of is that something weird happened while ipadm was 
>> communicating with ipmgmtd, and inspecting the process with pstack 
>> allowed a context switch to occur.
> I can maybe help with that as well? I tried to set the MTU to the 
> maximum, that the driver would accept and I ended up with a mtu of 
> 15500 (what a odd number anyway) which I set using dladm on ixgbe3. 
> When i tried to create the interface using ipadm, that was the moment, 
> when things went quirky. Although I trimmed the mtu down to a 
> reasonable 9216 ipadm wouldn't create any longer and would hang at 
> creating interfaces on any of the remaining 3 ports of my T540-X2's.
>>
>> Thank you, and while it's nice to see things working, I'm sorry I 
>> don't have a better explanation about what happened, or the ability 
>> to immediately reproduce this bug.
>>
>> Dan
>>
> Ha ha, no prob - you already provided a great deal of help!
>
> Thanks a lot,
> budy
Ahh? here we go again?

root at nfsvmpool01:~#  dladm show-linkprop -p mtu
LINK         PROPERTY        PERM VALUE          DEFAULT POSSIBLE
igb0         mtu             rw   1500           1500 60-9000
igb1         mtu             rw   1500           1500 60-9000
igb2         mtu             rw   1500           1500 60-9000
igb3         mtu             rw   1500           1500 60-9000
ixgbe2       mtu             rw   1500           1500 1500-15500
ixgbe0       mtu             rw   1500           1500 1500-15500


root at nfsvmpool01:~#  pstack `pgrep dladm`
23948:  dladm show-linkprop -p mtu

What I did was to remove the ixgbe3 interface and try to set the mtu to 
9216 like this:

dladm set-linkprop -p mtu=9216 ixgbe3

I seem only to be able to kill dladm by killing it using kill -9 <PID>

Cheers,
budy

From danmcd at omniti.com  Wed Dec 10 19:12:15 2014
From: danmcd at omniti.com (Dan McDonald)
Date: Wed, 10 Dec 2014 14:12:15 -0500
Subject: [OmniOS-discuss] ipadm hangs on interface creation
In-Reply-To: <54889A03.7080307@jvm.de>
References: <54880F6F.2010904@jvm.de>
	<2636F307-5701-42BD-8765-E4AAD2ED61EE@omniti.com>
	<548870DA.6080308@jvm.de>
	<E948E1BE-A338-46EC-BB7F-4C179DA006C1@omniti.com>
	<548876FC.9040108@jvm.de> <54889A03.7080307@jvm.de>
Message-ID: <2504F084-440B-4E08-9F0F-68CA2038C449@omniti.com>

You said it was ipadm earlier, not dladm.

> On Dec 10, 2014, at 2:07 PM, Stephan Budach <stephan.budach at jvm.de> wrote:
> Ahh? here we go again?
> 
> root at nfsvmpool01:~#  dladm show-linkprop -p mtu
> LINK         PROPERTY        PERM VALUE          DEFAULT POSSIBLE
> igb0         mtu             rw   1500           1500 60-9000
> igb1         mtu             rw   1500           1500 60-9000
> igb2         mtu             rw   1500           1500 60-9000
> igb3         mtu             rw   1500           1500 60-9000
> ixgbe2       mtu             rw   1500           1500 1500-15500
> ixgbe0       mtu             rw   1500           1500 1500-15500
> 
> 
> root at nfsvmpool01:~#  pstack `pgrep dladm`
> 23948:  dladm show-linkprop -p mtu
> 
> What I did was to remove the ixgbe3 interface and try to set the mtu to 9216 like this:

How did you remove the ixgbe3 interface?  With delete-phys?

> dladm set-linkprop -p mtu=9216 ixgbe3
> 
> I seem only to be able to kill dladm by killing it using kill -9 <PID>

It may be hanging because you removed it.

Generally speaking, you shouldn't remove hardware entities (vs. created entities like vnics, aggrs, etc.) with dladm.  They are harmless lying in the background, if you don't have them configured up with ifconfig or ipadm, you won't be using them.

Dan


From stephan.budach at JVM.DE  Wed Dec 10 19:23:32 2014
From: stephan.budach at JVM.DE (Stephan Budach)
Date: Wed, 10 Dec 2014 20:23:32 +0100
Subject: [OmniOS-discuss] ipadm hangs on interface creation
In-Reply-To: <2504F084-440B-4E08-9F0F-68CA2038C449@omniti.com>
References: <54880F6F.2010904@jvm.de>
	<2636F307-5701-42BD-8765-E4AAD2ED61EE@omniti.com>
	<548870DA.6080308@jvm.de>
	<E948E1BE-A338-46EC-BB7F-4C179DA006C1@omniti.com>
	<548876FC.9040108@jvm.de> <54889A03.7080307@jvm.de>
	<2504F084-440B-4E08-9F0F-68CA2038C449@omniti.com>
Message-ID: <54889DB4.40803@jvm.de>

Am 10.12.14 um 20:12 schrieb Dan McDonald:
> You said it was ipadm earlier, not dladm.
Well, yes it was. I am sure that ipadm will now hang as well.
>> On Dec 10, 2014, at 2:07 PM, Stephan Budach <stephan.budach at jvm.de> wrote:
>> Ahh? here we go again?
>>
>> root at nfsvmpool01:~#  dladm show-linkprop -p mtu
>> LINK         PROPERTY        PERM VALUE          DEFAULT POSSIBLE
>> igb0         mtu             rw   1500           1500 60-9000
>> igb1         mtu             rw   1500           1500 60-9000
>> igb2         mtu             rw   1500           1500 60-9000
>> igb3         mtu             rw   1500           1500 60-9000
>> ixgbe2       mtu             rw   1500           1500 1500-15500
>> ixgbe0       mtu             rw   1500           1500 1500-15500
>>
>>
>> root at nfsvmpool01:~#  pstack `pgrep dladm`
>> 23948:  dladm show-linkprop -p mtu
>>
>> What I did was to remove the ixgbe3 interface and try to set the mtu to 9216 like this:
> How did you remove the ixgbe3 interface?  With delete-phys?
>
>> dladm set-linkprop -p mtu=9216 ixgbe3
>>
>> I seem only to be able to kill dladm by killing it using kill -9 <PID>
> It may be hanging because you removed it.
>
> Generally speaking, you shouldn't remove hardware entities (vs. created entities like vnics, aggrs, etc.) with dladm.  They are harmless lying in the background, if you don't have them configured up with ifconfig or ipadm, you won't be using them.
>
> Dan
>
Maybe I have told things not clearly enough. I have created the 
interface like this:

ipadm create-if ixgbe3
ipadm create-addr -T static -a 192.168.1.1/24 ixgbe3/v4static

This worked, but I wanted to change the MTU, so I deciced to remove this 
config again:

ipadm delete-addr ixgb3/v4static
ipadm delete-if ixgbe3

Then I tried to set the MTU of ixgbe3 like this:
dladm set-linkprop -p mtu=9216 ixgbe3

I can't remember, if that already stalled, however, now dladm 
show-linkprop hangs and it does need to get killed -9.

Cheers,
budy


From danmcd at omniti.com  Wed Dec 10 19:28:37 2014
From: danmcd at omniti.com (Dan McDonald)
Date: Wed, 10 Dec 2014 14:28:37 -0500
Subject: [OmniOS-discuss] ipadm hangs on interface creation
In-Reply-To: <54889DB4.40803@jvm.de>
References: <54880F6F.2010904@jvm.de>
	<2636F307-5701-42BD-8765-E4AAD2ED61EE@omniti.com>
	<548870DA.6080308@jvm.de>
	<E948E1BE-A338-46EC-BB7F-4C179DA006C1@omniti.com>
	<548876FC.9040108@jvm.de> <54889A03.7080307@jvm.de>
	<2504F084-440B-4E08-9F0F-68CA2038C449@omniti.com>
	<54889DB4.40803@jvm.de>
Message-ID: <1BEB9CF7-779C-4003-BCDB-4038136846AA@omniti.com>


> On Dec 10, 2014, at 2:23 PM, Stephan Budach <stephan.budach at jvm.de> wrote:
> 
> This worked, but I wanted to change the MTU, so I deciced to remove this config again:
> 
> ipadm delete-addr ixgb3/v4static
> ipadm delete-if ixgbe3

Interesting.  Your "dladm show-linkprop" output didn't have anything for ixgbe3.  I wonder if that happened because of the delete-if for ipadm?

> Then I tried to set the MTU of ixgbe3 like this:
> dladm set-linkprop -p mtu=9216 ixgbe3
> 
> I can't remember, if that already stalled, however, now dladm show-linkprop hangs and it does need to get killed -9.

It's probably not able to find ixgbe3, and hanging.  The question is how did ixgbe3 disappear from the list of links (vs. ip addrs)?

Dan


From stephan.budach at JVM.DE  Wed Dec 10 19:56:08 2014
From: stephan.budach at JVM.DE (Stephan Budach)
Date: Wed, 10 Dec 2014 20:56:08 +0100
Subject: [OmniOS-discuss] ipadm hangs on interface creation
In-Reply-To: <1BEB9CF7-779C-4003-BCDB-4038136846AA@omniti.com>
References: <54880F6F.2010904@jvm.de>
	<2636F307-5701-42BD-8765-E4AAD2ED61EE@omniti.com>
	<548870DA.6080308@jvm.de>
	<E948E1BE-A338-46EC-BB7F-4C179DA006C1@omniti.com>
	<548876FC.9040108@jvm.de> <54889A03.7080307@jvm.de>
	<2504F084-440B-4E08-9F0F-68CA2038C449@omniti.com>
	<54889DB4.40803@jvm.de>
	<1BEB9CF7-779C-4003-BCDB-4038136846AA@omniti.com>
Message-ID: <5488A558.5010401@jvm.de>

Am 10.12.14 um 20:28 schrieb Dan McDonald:
>> On Dec 10, 2014, at 2:23 PM, Stephan Budach <stephan.budach at jvm.de> wrote:
>>
>> This worked, but I wanted to change the MTU, so I deciced to remove this config again:
>>
>> ipadm delete-addr ixgb3/v4static
>> ipadm delete-if ixgbe3
> Interesting.  Your "dladm show-linkprop" output didn't have anything for ixgbe3.  I wonder if that happened because of the delete-if for ipadm?
>
>> Then I tried to set the MTU of ixgbe3 like this:
>> dladm set-linkprop -p mtu=9216 ixgbe3
>>
>> I can't remember, if that already stalled, however, now dladm show-linkprop hangs and it does need to get killed -9.
> It's probably not able to find ixgbe3, and hanging.  The question is how did ixgbe3 disappear from the list of links (vs. ip addrs)?
>
I don't know? it seems to have serious issues with the MTU specifically?

root at nfsvmpool01:~# dladm show-phys
LINK         MEDIA                STATE      SPEED DUPLEX    DEVICE
igb0         Ethernet             up         1000 full      igb0
igb1         Ethernet             unknown    1000 full      igb1
igb2         Ethernet             unknown    0 half      igb2
igb3         Ethernet             unknown    0 half      igb3
ixgbe2       Ethernet             down       0 unknown   ixgbe2
ixgbe0       Ethernet             up         10000 full      ixgbe0
ixgbe3       Ethernet             up         10000 full      ixgbe3
ixgbe1       Ethernet             down       0 unknown   ixgbe1

root at nfsvmpool01:~# dladm show-linkprop -p mtu
LINK         PROPERTY        PERM VALUE DEFAULT        POSSIBLE
igb0         mtu             rw   1500 1500           60-9000
igb1         mtu             rw   1500 1500           60-9000
igb2         mtu             rw   1500 1500           60-9000
igb3         mtu             rw   1500 1500           60-9000
ixgbe2       mtu             rw   1500 1500           1500-15500
ixgbe0       mtu             rw   1500 1500           1500-15500
^C

So, dladm shows the phys status, but it isn't able to tell me about the 
MTU of ixgbe3??

Needless to mention that ipadm also doesn't work. I will try to reset 
the MTU back to 1500 again and seem if I can get the interface back.

Cheers,
budy


From stephan.budach at JVM.DE  Wed Dec 10 20:14:59 2014
From: stephan.budach at JVM.DE (Stephan Budach)
Date: Wed, 10 Dec 2014 21:14:59 +0100
Subject: [OmniOS-discuss] ipadm hangs on interface creation
In-Reply-To: <5488A558.5010401@jvm.de>
References: <54880F6F.2010904@jvm.de>
	<2636F307-5701-42BD-8765-E4AAD2ED61EE@omniti.com>
	<548870DA.6080308@jvm.de>
	<E948E1BE-A338-46EC-BB7F-4C179DA006C1@omniti.com>
	<548876FC.9040108@jvm.de> <54889A03.7080307@jvm.de>
	<2504F084-440B-4E08-9F0F-68CA2038C449@omniti.com>
	<54889DB4.40803@jvm.de>
	<1BEB9CF7-779C-4003-BCDB-4038136846AA@omniti.com>
	<5488A558.5010401@jvm.de>
Message-ID: <5488A9C3.3080903@jvm.de>

Am 10.12.14 um 20:56 schrieb Stephan Budach:
> I don't know? it seems to have serious issues with the MTU specifically?
>
> root at nfsvmpool01:~# dladm show-phys
> LINK         MEDIA                STATE      SPEED DUPLEX DEVICE
> igb0         Ethernet             up         1000 full      igb0
> igb1         Ethernet             unknown    1000 full      igb1
> igb2         Ethernet             unknown    0 half      igb2
> igb3         Ethernet             unknown    0 half      igb3
> ixgbe2       Ethernet             down       0 unknown   ixgbe2
> ixgbe0       Ethernet             up         10000 full ixgbe0
> ixgbe3       Ethernet             up         10000 full ixgbe3
> ixgbe1       Ethernet             down       0 unknown   ixgbe1
>
> root at nfsvmpool01:~# dladm show-linkprop -p mtu
> LINK         PROPERTY        PERM VALUE DEFAULT        POSSIBLE
> igb0         mtu             rw   1500 1500           60-9000
> igb1         mtu             rw   1500 1500           60-9000
> igb2         mtu             rw   1500 1500           60-9000
> igb3         mtu             rw   1500 1500           60-9000
> ixgbe2       mtu             rw   1500 1500           1500-15500
> ixgbe0       mtu             rw   1500 1500           1500-15500
> ^C
>
> So, dladm shows the phys status, but it isn't able to tell me about 
> the MTU of ixgbe3??
>
> Needless to mention that ipadm also doesn't work. I will try to reset 
> the MTU back to 1500 again and seem if I can get the interface back. 
Now, this one? I don't get it. ;)

root at nfsvmpool01:~# ipadm create-if ixgbe3
root at nfsvmpool01:~# ipadm show-if
IFNAME     STATE    CURRENT      PERSISTENT
lo0        ok       -m-v------46 ---
igb0       ok       bm--------46 -46
ixgbe0     ok       bm--------46 -46
ixgbe3     down     bm--------46 -46
root at nfsvmpool01:~# ipadm create-addr -T static -a 192.168.1.1/24 
ixgbe3/v4static
root at nfsvmpool01:~# ping 192.168.1.2
192.168.1.2 is alive
root at nfsvmpool01:~# dladm show-linkprop -p mtu
LINK         PROPERTY        PERM VALUE DEFAULT        POSSIBLE
igb0         mtu             rw   1500 1500           60-9000
igb1         mtu             rw   1500 1500           60-9000
igb2         mtu             rw   1500 1500           60-9000
igb3         mtu             rw   1500 1500           60-9000
ixgbe2       mtu             rw   1500 1500           1500-15500
ixgbe0       mtu             rw   1500 1500           1500-15500
ixgbe3       mtu             rw   9216 1500           1500-15500
ixgbe1       mtu             rw   1500 1500           1500-15500

I do have a OmniOS 012 system, where all this works without these issues? ;)

Now, I won#t change the mtu that soon again, I guess?

Cheers,
budy

From danmcd at omniti.com  Wed Dec 10 20:17:27 2014
From: danmcd at omniti.com (Dan McDonald)
Date: Wed, 10 Dec 2014 15:17:27 -0500
Subject: [OmniOS-discuss] ipadm hangs on interface creation
In-Reply-To: <5488A9C3.3080903@jvm.de>
References: <54880F6F.2010904@jvm.de>
	<2636F307-5701-42BD-8765-E4AAD2ED61EE@omniti.com>
	<548870DA.6080308@jvm.de>
	<E948E1BE-A338-46EC-BB7F-4C179DA006C1@omniti.com>
	<548876FC.9040108@jvm.de> <54889A03.7080307@jvm.de>
	<2504F084-440B-4E08-9F0F-68CA2038C449@omniti.com>
	<54889DB4.40803@jvm.de>
	<1BEB9CF7-779C-4003-BCDB-4038136846AA@omniti.com>
	<5488A558.5010401@jvm.de> <5488A9C3.3080903@jvm.de>
Message-ID: <360BDE82-002E-40B9-A746-C4328FE8776D@omniti.com>


> On Dec 10, 2014, at 3:14 PM, Stephan Budach <stephan.budach at jvm.de> wrote:
> 
>> 
> Now, this one? I don't get it. ;)
> 
> root at nfsvmpool01:~# ipadm create-if ixgbe3
> root at nfsvmpool01:~# ipadm show-if
> IFNAME     STATE    CURRENT      PERSISTENT
> lo0        ok       -m-v------46 ---
> igb0       ok       bm--------46 -46
> ixgbe0     ok       bm--------46 -46
> ixgbe3     down     bm--------46 -46
> root at nfsvmpool01:~# ipadm create-addr -T static -a 192.168.1.1/24 ixgbe3/v4static
> root at nfsvmpool01:~# ping 192.168.1.2
> 192.168.1.2 is alive
> root at nfsvmpool01:~# dladm show-linkprop -p mtu
> LINK         PROPERTY        PERM VALUE DEFAULT        POSSIBLE
> igb0         mtu             rw   1500 1500           60-9000
> igb1         mtu             rw   1500 1500           60-9000
> igb2         mtu             rw   1500 1500           60-9000
> igb3         mtu             rw   1500 1500           60-9000
> ixgbe2       mtu             rw   1500 1500           1500-15500
> ixgbe0       mtu             rw   1500 1500           1500-15500
> ixgbe3       mtu             rw   9216 1500           1500-15500
> ixgbe1       mtu             rw   1500 1500           1500-15500

Call me old-fashioned, but I'm used to doing this all with ifconfig(1M) and editing /etc/hostname.X files.  I know I should learn this new stuff, but I never have had the wherewithal until now.

> I do have a OmniOS 012 system, where all this works without these issues? ;)

REALLY?  So this fails on 006, but works on 012?!  I quickly looked at illumos-gate to see what all got fixed if anything, and I didn't see anything that would've necessarily improved things.

> Now, I won#t change the mtu that soon again, I guess?

I guess.

Sorry I couldn't provide an authoritative answer,
Dan


From stephan.budach at JVM.DE  Wed Dec 10 20:27:11 2014
From: stephan.budach at JVM.DE (Stephan Budach)
Date: Wed, 10 Dec 2014 21:27:11 +0100
Subject: [OmniOS-discuss] ipadm hangs on interface creation
In-Reply-To: <360BDE82-002E-40B9-A746-C4328FE8776D@omniti.com>
References: <54880F6F.2010904@jvm.de>
	<2636F307-5701-42BD-8765-E4AAD2ED61EE@omniti.com>
	<548870DA.6080308@jvm.de>
	<E948E1BE-A338-46EC-BB7F-4C179DA006C1@omniti.com>
	<548876FC.9040108@jvm.de> <54889A03.7080307@jvm.de>
	<2504F084-440B-4E08-9F0F-68CA2038C449@omniti.com>
	<54889DB4.40803@jvm.de>
	<1BEB9CF7-779C-4003-BCDB-4038136846AA@omniti.com>
	<5488A558.5010401@jvm.de> <5488A9C3.3080903@jvm.de>
	<360BDE82-002E-40B9-A746-C4328FE8776D@omniti.com>
Message-ID: <5488AC9F.9000400@jvm.de>

Hi Dan,

Am 10.12.14 um 21:17 schrieb Dan McDonald:
>> On Dec 10, 2014, at 3:14 PM, Stephan Budach <stephan.budach at jvm.de> wrote:
>>
>> Now, this one? I don't get it. ;)
>>
>> root at nfsvmpool01:~# ipadm create-if ixgbe3
>> root at nfsvmpool01:~# ipadm show-if
>> IFNAME     STATE    CURRENT      PERSISTENT
>> lo0        ok       -m-v------46 ---
>> igb0       ok       bm--------46 -46
>> ixgbe0     ok       bm--------46 -46
>> ixgbe3     down     bm--------46 -46
>> root at nfsvmpool01:~# ipadm create-addr -T static -a 192.168.1.1/24 ixgbe3/v4static
>> root at nfsvmpool01:~# ping 192.168.1.2
>> 192.168.1.2 is alive
>> root at nfsvmpool01:~# dladm show-linkprop -p mtu
>> LINK         PROPERTY        PERM VALUE DEFAULT        POSSIBLE
>> igb0         mtu             rw   1500 1500           60-9000
>> igb1         mtu             rw   1500 1500           60-9000
>> igb2         mtu             rw   1500 1500           60-9000
>> igb3         mtu             rw   1500 1500           60-9000
>> ixgbe2       mtu             rw   1500 1500           1500-15500
>> ixgbe0       mtu             rw   1500 1500           1500-15500
>> ixgbe3       mtu             rw   9216 1500           1500-15500
>> ixgbe1       mtu             rw   1500 1500           1500-15500
> Call me old-fashioned, but I'm used to doing this all with ifconfig(1M) and editing /etc/hostname.X files.  I know I should learn this new stuff, but I never have had the wherewithal until now.
>
>> I do have a OmniOS 012 system, where all this works without these issues? ;)
> REALLY?  So this fails on 006, but works on 012?!  I quickly looked at illumos-gate to see what all got fixed if anything, and I didn't see anything that would've necessarily improved things.
>
>> Now, I won#t change the mtu that soon again, I guess?
> I guess.
>
> Sorry I couldn't provide an authoritative answer,
> Dan

Ha ha - never mind! Sometimes, I find it enough to chat about such 
issues with someone.
I do have to admit, that on the 012 system, none of the ixgbe ports were 
already in use. You provided mental care and that alone helped. ;)
Besides, you' re able to make something of the illumos sources - one 
thing that really escapes me?

Cheers,
budy

From danmcd at omniti.com  Wed Dec 10 22:10:43 2014
From: danmcd at omniti.com (Dan McDonald)
Date: Wed, 10 Dec 2014 17:10:43 -0500
Subject: [OmniOS-discuss] NEWS: r151008 packages retroactively separated
	into own repo
References: <CAO8hXRBrNVAZrKJW=MvPBSDv99dpoA5ZYMdiJuSRWUaT6H8tJw@mail.gmail.com>
Message-ID: <63929042-B3DF-4A99-ABBF-8EF9DC11DCEE@omniti.com>

Thank you to Eric Sproul (now of Circonus)!  Here's his good news:

.  .  .

Good news everyone!

As many of you are aware, the current practice for core OmniOS
packaging is to separate packages for a given release into their own
repository.  This was prompted by all the headaches caused when
r151008 packages were added to the release repo where r151006 (and
earlier) lived.  As of about an hour ago, I have split out the r151008
packages from http://pkg.omniti.com/omnios/release/ into
http://pkg.omniti.com/omnios/r151008/ .

The good news is that LTS users now no longer need to take
extraordinary measures to stay on r151006.  We'll be updating the
documentation shortly.  If you've already done the pkg freeze stuff,
you can either leave it in place and plan to remove it when you're
ready to upgrade to the next LTS, or go ahead and remove it now.
Either way, you will never again see post-r151006 packages in the
/omnios/release/ location.

Since r151008 is EOL this is essentially just an archival operation--
no new packages will ever be published for this release.  Those still
running r151008 (pro tip: you should upgrade!) will no longer be able
to install any existing packages unless you switch publisher URLs:
pkg set-publisher -G <existing-release-url> -g <new-008-url-above> omnios

.  .  .

I will reiterate the protip mentioned earlier. You should not be running 008 anywhere anymore.

Thanks again to Eric!

Dan


From ikaufman at eng.ucsd.edu  Thu Dec 11 00:17:49 2014
From: ikaufman at eng.ucsd.edu (Ian Kaufman)
Date: Wed, 10 Dec 2014 16:17:49 -0800
Subject: [OmniOS-discuss] weird library issue
In-Reply-To: <A4828F6E-1C68-4107-9C5D-00363774C73F@omniti.com>
References: <CAPJtH1ghB83XiJonib0qxTu3oZ=28XcKFOj7FHRm9hu-AB+Kuw@mail.gmail.com>
	<B3CAABA1-3A42-4CA4-9F20-D825E03BA499@omniti.com>
	<CAPJtH1hkxOAeB0Uf_Dbww6xiWeav83BOOhoRun_8JNTfAFE+sA@mail.gmail.com>
	<A4828F6E-1C68-4107-9C5D-00363774C73F@omniti.com>
Message-ID: <CAPJtH1h4NCKN6AUC3fByRmnZmTvTArCUxKP-OuNMN6iDKsoC4w@mail.gmail.com>

Thanks Dan.

Alas, I get stuck here:

root at massive02-b:~# pkg -R /mnt update
Creating Plan -
pkg update: The certificate which issued this
certificate:/C=US/ST=Maryland/O=OmniTI/OU=OmniOS/CN=OmniOS r151012
Release Signing Certificate/emailAddress=omnios-support at omniti.com
could not be found. The issuer
is:/C=US/ST=Maryland/L=Fulton/O=OmniTI/CN=OmniTI Certificate Authority
The package involved
is:pkg://omnios/locale/gu at 0.5.11,5.11-0.151012:20140913T033547Z

Ian

On Wed, Dec 10, 2014 at 10:55 AM, Dan McDonald <danmcd at omniti.com> wrote:
>
>> On Dec 10, 2014, at 1:42 PM, Ian Kaufman <ikaufman at eng.ucsd.edu> wrote:
>>
>> It looks like this system is in some weird state where there are
>> packages from 151006 and 151008.
>
> Oh my.
>
> Yes, you've been bitten by the pkg upgrade problems between 006 and 008.  This was before my time here, so I'm not sure how best to extricate yourself from things.
>
> If you want, you can jump forward to the current stable release (r151012).  If you want to try that, and you don't have zones, do this:
>
>
> 1.) Create a new BE, call it 012 for this example:  beadm create 012
>
> 2.) Mount the BE on /mnt:       beadm mount 012 /mnt
>
> 3.) Change the publisher on the new BE to point to r151012's repo:
>         pkg -R /mnt set-publisher -G http://pkg.omniti.com/omnios/release/ -g http://pkg.omniti.com/omnios/r151012/ omnios
>
> 4.) Apply the update to the new BE:   pkg -R /mnt update
>
> 5.) Unmount the BE:   beadm umount 012
>
> 6.) Either reboot and use the GRUB menu to try out the new 012 BE, or "beadm activate 012" to make it go there on the next reboot.
>
>
> Dan
>



-- 
Ian Kaufman
Research Systems Administrator
UC San Diego, Jacobs School of Engineering ikaufman AT ucsd DOT edu

From alexmcwhirter at mojovapes.net  Thu Dec 11 00:47:28 2014
From: alexmcwhirter at mojovapes.net (Alex McWhirter)
Date: Wed, 10 Dec 2014 19:47:28 -0500
Subject: [OmniOS-discuss] Change Zone Hostname
Message-ID: <7F6BAE74-996F-459E-B035-6422E41904B8@mojovapes.net>

I have a small cluster of servers hosting zones over multiple VLAN?s. I would like to name the zones so that they don?t run into naming collisions. For example, instead of ?web0? i would like to name them ?2049_web0? and ?2050_web0?. The problem is that this set?s the hostname to ?XXXX_web0?. Is there a way to change the hostname to be something different than the zone name?

I had a look at /etc/nodname and it is an empty file when used in a zone.

From tim at multitalents.net  Thu Dec 11 06:30:01 2014
From: tim at multitalents.net (Tim Rice)
Date: Wed, 10 Dec 2014 22:30:01 -0800 (PST)
Subject: [OmniOS-discuss] Change Zone Hostname
In-Reply-To: <7F6BAE74-996F-459E-B035-6422E41904B8@mojovapes.net>
References: <7F6BAE74-996F-459E-B035-6422E41904B8@mojovapes.net>
Message-ID: <alpine.UW2.2.11.1412102228350.4175@server01.int.multitalents.net>

On Wed, 10 Dec 2014, Alex McWhirter wrote:

> I have a small cluster of servers hosting zones over multiple VLAN?s. I would like to name the zones so that they don?t run into naming collisions. For example, instead of ?web0? i would like to name them ?2049_web0? and ?2050_web0?. The problem is that this set?s the hostname to ?XXXX_web0?. Is there a way to change the hostname to be something different than the zone name?
> 
> I had a look at /etc/nodname and it is an empty file when used in a zone.

What happens if you put the hostname you want in /etc/nodname ?


-- 
Tim Rice				Multitalents
tim at multitalents.net

From alexmcwhirter at mojovapes.net  Thu Dec 11 06:48:34 2014
From: alexmcwhirter at mojovapes.net (Alex McWhirter)
Date: Thu, 11 Dec 2014 01:48:34 -0500
Subject: [OmniOS-discuss] Change Zone Hostname
In-Reply-To: <alpine.UW2.2.11.1412102228350.4175@server01.int.multitalents.net>
References: <7F6BAE74-996F-459E-B035-6422E41904B8@mojovapes.net>
	<alpine.UW2.2.11.1412102228350.4175@server01.int.multitalents.net>
Message-ID: <D992C95D-FF1E-4BFA-A7AB-91789067E952@mojovapes.net>

Yea, some services picked it up but others did not. It seems the identity/node service does not have the config variable set for nodename. I attempted to set it but received and error saying that the variable didn't exist.

I don't think this was ever an intended function for a zone.

Sent from my iPhone

> On Dec 11, 2014, at 1:30 AM, Tim Rice <tim at multitalents.net> wrote:
> 
>> On Wed, 10 Dec 2014, Alex McWhirter wrote:
>> 
>> I have a small cluster of servers hosting zones over multiple VLAN?s. I would like to name the zones so that they don?t run into naming collisions. For example, instead of ?web0? i would like to name them ?2049_web0? and ?2050_web0?. The problem is that this set?s the hostname to ?XXXX_web0?. Is there a way to change the hostname to be something different than the zone name?
>> 
>> I had a look at /etc/nodname and it is an empty file when used in a zone.
> 
> What happens if you put the hostname you want in /etc/nodname ?
> 
> 
> -- 
> Tim Rice                Multitalents
> tim at multitalents.net

From danmcd at omniti.com  Thu Dec 11 07:16:05 2014
From: danmcd at omniti.com (Dan McDonald)
Date: Thu, 11 Dec 2014 02:16:05 -0500
Subject: [OmniOS-discuss] Change Zone Hostname
In-Reply-To: <alpine.UW2.2.11.1412102228350.4175@server01.int.multitalents.net>
References: <7F6BAE74-996F-459E-B035-6422E41904B8@mojovapes.net>
	<alpine.UW2.2.11.1412102228350.4175@server01.int.multitalents.net>
Message-ID: <4C2DF45B-2E0B-4624-BB35-DA3EC09CD51C@omniti.com>

Editing nodename and rebooting the zone will give it a different name.  I do this on my own HDC.

Dan

Sent from my iPhone (typos, autocorrect, and all)

> On Dec 11, 2014, at 1:30 AM, Tim Rice <tim at multitalents.net> wrote:
> 
>> On Wed, 10 Dec 2014, Alex McWhirter wrote:
>> 
>> I have a small cluster of servers hosting zones over multiple VLAN?s. I would like to name the zones so that they don?t run into naming collisions. For example, instead of ?web0? i would like to name them ?2049_web0? and ?2050_web0?. The problem is that this set?s the hostname to ?XXXX_web0?. Is there a way to change the hostname to be something different than the zone name?
>> 
>> I had a look at /etc/nodname and it is an empty file when used in a zone.
> 
> What happens if you put the hostname you want in /etc/nodname ?
> 
> 
> -- 
> Tim Rice                Multitalents
> tim at multitalents.net
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss

From tim at multitalents.net  Thu Dec 11 07:18:45 2014
From: tim at multitalents.net (Tim Rice)
Date: Wed, 10 Dec 2014 23:18:45 -0800 (PST)
Subject: [OmniOS-discuss] Change Zone Hostname
In-Reply-To: <D992C95D-FF1E-4BFA-A7AB-91789067E952@mojovapes.net>
References: <7F6BAE74-996F-459E-B035-6422E41904B8@mojovapes.net>
	<alpine.UW2.2.11.1412102228350.4175@server01.int.multitalents.net>
	<D992C95D-FF1E-4BFA-A7AB-91789067E952@mojovapes.net>
Message-ID: <alpine.UW2.2.11.1412102313460.4175@server01.int.multitalents.net>

On Thu, 11 Dec 2014, Alex McWhirter wrote:

| Yea, some services picked it up but others did not. It seems the identity/node service does not have the config variable set for nodename. I attempted to set it but received and error saying that the variable didn't exist.
| 
| I don't think this was ever an intended function for a zone.

Well it does on solaris 10 so I thought it might be worth a try.
.....
# uname -n
ftp22
# zonename
ftp
# cat /etc/nodename
ftp22
#
.....

| 
| Sent from my iPhone
| 
| > On Dec 11, 2014, at 1:30 AM, Tim Rice <tim at multitalents.net> wrote:
| > 
| >> On Wed, 10 Dec 2014, Alex McWhirter wrote:
| >> 
| >> I have a small cluster of servers hosting zones over multiple VLAN?s. I would like to name the zones so that they don?t run into naming collisions. For example, instead of ?web0? i would like to name them ?2049_web0? and ?2050_web0?. The problem is that this set?s the hostname to ?XXXX_web0?. Is there a way to change the hostname to be something different than the zone name?
| >> 
| >> I had a look at /etc/nodname and it is an empty file when used in a zone.
| > 
| > What happens if you put the hostname you want in /etc/nodname ?
| > 
| > 
| > -- 
| > Tim Rice                Multitalents
| > tim at multitalents.net
| _______________________________________________
| OmniOS-discuss mailing list
| OmniOS-discuss at lists.omniti.com
| http://lists.omniti.com/mailman/listinfo/omnios-discuss
| 
| 
| 

-- 
Tim Rice				Multitalents	(707) 456-1146
tim at multitalents.net

From alexmcwhirter at mojovapes.net  Thu Dec 11 07:26:40 2014
From: alexmcwhirter at mojovapes.net (Alex McWhirter)
Date: Thu, 11 Dec 2014 02:26:40 -0500
Subject: [OmniOS-discuss] Change Zone Hostname
In-Reply-To: <alpine.UW2.2.11.1412102313460.4175@server01.int.multitalents.net>
References: <7F6BAE74-996F-459E-B035-6422E41904B8@mojovapes.net>
	<alpine.UW2.2.11.1412102228350.4175@server01.int.multitalents.net>
	<D992C95D-FF1E-4BFA-A7AB-91789067E952@mojovapes.net>
	<alpine.UW2.2.11.1412102313460.4175@server01.int.multitalents.net>
Message-ID: <40F72E42-C88F-4909-8CD7-EAD05AABF072@mojovapes.net>

I had been toying around with a lot of different settings, it's possible I just messed something up in the process. Let me try again with a fresh zone.

Sent from my iPhone

> On Dec 11, 2014, at 2:18 AM, Tim Rice <tim at multitalents.net> wrote:
> 
> On Thu, 11 Dec 2014, Alex McWhirter wrote:
> 
> | Yea, some services picked it up but others did not. It seems the identity/node service does not have the config variable set for nodename. I attempted to set it but received and error saying that the variable didn't exist.
> | 
> | I don't think this was ever an intended function for a zone.
> 
> Well it does on solaris 10 so I thought it might be worth a try.
> .....
> # uname -n
> ftp22
> # zonename
> ftp
> # cat /etc/nodename
> ftp22
> #
> .....
> 
> | 
> | Sent from my iPhone
> | 
> | > On Dec 11, 2014, at 1:30 AM, Tim Rice <tim at multitalents.net> wrote:
> | > 
> | >> On Wed, 10 Dec 2014, Alex McWhirter wrote:
> | >> 
> | >> I have a small cluster of servers hosting zones over multiple VLAN?s. I would like to name the zones so that they don?t run into naming collisions. For example, instead of ?web0? i would like to name them ?2049_web0? and ?2050_web0?. The problem is that this set?s the hostname to ?XXXX_web0?. Is there a way to change the hostname to be something different than the zone name?
> | >> 
> | >> I had a look at /etc/nodname and it is an empty file when used in a zone.
> | > 
> | > What happens if you put the hostname you want in /etc/nodname ?
> | > 
> | > 
> | > -- 
> | > Tim Rice                Multitalents
> | > tim at multitalents.net
> | _______________________________________________
> | OmniOS-discuss mailing list
> | OmniOS-discuss at lists.omniti.com
> | http://lists.omniti.com/mailman/listinfo/omnios-discuss
> | 
> | 
> | 
> 
> -- 
> Tim Rice                Multitalents    (707) 456-1146
> tim at multitalents.net

From jimklimov at cos.ru  Thu Dec 11 10:38:05 2014
From: jimklimov at cos.ru (Jim Klimov)
Date: Thu, 11 Dec 2014 11:38:05 +0100
Subject: [OmniOS-discuss] Change Zone Hostname
In-Reply-To: <40F72E42-C88F-4909-8CD7-EAD05AABF072@mojovapes.net>
References: <7F6BAE74-996F-459E-B035-6422E41904B8@mojovapes.net>
	<alpine.UW2.2.11.1412102228350.4175@server01.int.multitalents.net>
	<D992C95D-FF1E-4BFA-A7AB-91789067E952@mojovapes.net>
	<alpine.UW2.2.11.1412102313460.4175@server01.int.multitalents.net>
	<40F72E42-C88F-4909-8CD7-EAD05AABF072@mojovapes.net>
Message-ID: <472E6EE3-F06B-48B5-97AC-E6EEC513A7A1@cos.ru>

11 ??????? 2014??. 8:26:40 CET, Alex McWhirter <alexmcwhirter at mojovapes.net> ?????:
>I had been toying around with a lot of different settings, it's
>possible I just messed something up in the process. Let me try again
>with a fresh zone.
>
>Sent from my iPhone
>
>> On Dec 11, 2014, at 2:18 AM, Tim Rice <tim at multitalents.net> wrote:
>> 
>> On Thu, 11 Dec 2014, Alex McWhirter wrote:
>> 
>> | Yea, some services picked it up but others did not. It seems the
>identity/node service does not have the config variable set for
>nodename. I attempted to set it but received and error saying that the
>variable didn't exist.
>> | 
>> | I don't think this was ever an intended function for a zone.
>> 
>> Well it does on solaris 10 so I thought it might be worth a try.
>> .....
>> # uname -n
>> ftp22
>> # zonename
>> ftp
>> # cat /etc/nodename
>> ftp22
>> #
>> .....
>> 
>> | 
>> | Sent from my iPhone
>> | 
>> | > On Dec 11, 2014, at 1:30 AM, Tim Rice <tim at multitalents.net>
>wrote:
>> | > 
>> | >> On Wed, 10 Dec 2014, Alex McWhirter wrote:
>> | >> 
>> | >> I have a small cluster of servers hosting zones over multiple
>VLAN?s. I would like to name the zones so that they don?t run into
>naming collisions. For example, instead of ?web0? i would like to name
>them ?2049_web0? and ?2050_web0?. The problem is that this set?s the
>hostname to ?XXXX_web0?. Is there a way to change the hostname to be
>something different than the zone name?
>> | >> 
>> | >> I had a look at /etc/nodname and it is an empty file when used
>in a zone.
>> | > 
>> | > What happens if you put the hostname you want in /etc/nodname ?
>> | > 
>> | > 
>> | > -- 
>> | > Tim Rice                Multitalents
>> | > tim at multitalents.net
>> | _______________________________________________
>> | OmniOS-discuss mailing list
>> | OmniOS-discuss at lists.omniti.com
>> | http://lists.omniti.com/mailman/listinfo/omnios-discuss
>> | 
>> | 
>> | 
>> 
>> -- 
>> Tim Rice                Multitalents    (707) 456-1146
>> tim at multitalents.net
>_______________________________________________
>OmniOS-discuss mailing list
>OmniOS-discuss at lists.omniti.com
>http://lists.omniti.com/mailman/listinfo/omnios-discuss

Did you reboot it after changing the file?

Generally running "hostname `cat /etc/nodename" should've done the job too. 
--
Typos courtesy of K-9 Mail on my Samsung Android

From tobi at oetiker.ch  Thu Dec 11 15:44:37 2014
From: tobi at oetiker.ch (Tobias Oetiker)
Date: Thu, 11 Dec 2014 16:44:37 +0100 (CET)
Subject: [OmniOS-discuss] live lsi firmware upgrade P17->P19
Message-ID: <alpine.DEB.2.10.1412111638310.4496@engelberg.oetiker.ch>

I am looking at upgrading the firmware of our LSI HBAs to P19 since
we suspect that our use of P17 is the cause for disk timeouts we
are seeing every few weeks.

I am wondering, is it save to flash the HBAs from a running omnios system,
and if so, has anyone written some notes down on the process and
tools to use?

cheers
tobi


-- 
Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten, Switzerland
www.oetiker.ch tobi at oetiker.ch +41 62 775 9902


From mmabis at vmware.com  Thu Dec 11 16:36:13 2014
From: mmabis at vmware.com (Matthew Mabis)
Date: Thu, 11 Dec 2014 16:36:13 +0000
Subject: [OmniOS-discuss] live lsi firmware upgrade P17->P19
In-Reply-To: <alpine.DEB.2.10.1412111638310.4496@engelberg.oetiker.ch>
References: <alpine.DEB.2.10.1412111638310.4496@engelberg.oetiker.ch>
Message-ID: <50c221a6047b444682d4f0654a3b3163@EX13-MBX-001.vmware.com>

Just an FYI

I wanted to pass some information along that I became aware of earlier today. There is a fairly serious firmware issue with the entire LSI 92XX HBA product line. The currently shipping firmware version for the 92XX HBA line is P20 which was released on September 22. The P20 firmware contains a bug that will show up as IO timeouts, disk errors and corruption on nearly every storage drive on LSI?s compatibility list. I became aware of the issue after observing a catastrophic ZFS storage failure and tracking it back to the HBAs. After isolating the issue I contacted LSI support who informed me of the bug and the fact that they have not publicly documented the issue. LSI support also informed me that the only fix for this issue is to return any effected HBA to the P19 firmware.


Matt Mabis


________________________________________
From: OmniOS-discuss <omnios-discuss-bounces at lists.omniti.com> on behalf of Tobias Oetiker <tobi at oetiker.ch>
Sent: Thursday, December 11, 2014 7:44 AM
To: omnios-discuss at lists.omniti.com
Subject: [OmniOS-discuss] live lsi firmware upgrade P17->P19

I am looking at upgrading the firmware of our LSI HBAs to P19 since
we suspect that our use of P17 is the cause for disk timeouts we
are seeing every few weeks.

I am wondering, is it save to flash the HBAs from a running omnios system,
and if so, has anyone written some notes down on the process and
tools to use?

cheers
tobi


--
Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten, Switzerland
www.oetiker.ch tobi at oetiker.ch +41 62 775 9902

_______________________________________________
OmniOS-discuss mailing list
OmniOS-discuss at lists.omniti.com
https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.omniti.com_mailman_listinfo_omnios-2Ddiscuss&d=AAICAg&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtXt-uEs&r=6lrHKBYs7FGYRAd987joFNP44jNhX5Ha5IypFnBFBJE&m=ffEtrdn5eO8TKjy27ecAvbkgdKvSgf_ODZ0kSPuqz50&s=HyYgGdm8SWTXlGepFA7xj-5PYCtJBlSmai-tLvol3aQ&e=

From mir at miras.org  Thu Dec 11 17:16:22 2014
From: mir at miras.org (Michael Rasmussen)
Date: Thu, 11 Dec 2014 18:16:22 +0100
Subject: [OmniOS-discuss] live lsi firmware upgrade P17->P19
In-Reply-To: <50c221a6047b444682d4f0654a3b3163@EX13-MBX-001.vmware.com>
References: <alpine.DEB.2.10.1412111638310.4496@engelberg.oetiker.ch>
	<50c221a6047b444682d4f0654a3b3163@EX13-MBX-001.vmware.com>
Message-ID: <20141211181622.4bf08199@sleipner.datanom.net>

On Thu, 11 Dec 2014 16:36:13 +0000
Matthew Mabis <mmabis at vmware.com> wrote:

have not publicly documented the issue. LSI support also informed me that the only fix for this issue is to return any effected HBA to the P19 firmware.
> 
Someone on this list has also claimed that P18 should be a safe harbor.

-- 
Hilsen/Regards
Michael Rasmussen

Get my public GnuPG keys:
michael <at> rasmussen <dot> cc
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E
mir <at> datanom <dot> net
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C
mir <at> miras <dot> org
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917
--------------------------------------------------------------
/usr/games/fortune -es says:
"On the Internet, no one knows you're using Windows NT"
(Submitted by Ramiro Estrugo, restrugo at fateware.com)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 181 bytes
Desc: OpenPGP digital signature
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141211/6a0b7002/attachment-0001.bin>

From gearboxes at outlook.com  Thu Dec 11 17:47:49 2014
From: gearboxes at outlook.com (Machine Man)
Date: Thu, 11 Dec 2014 12:47:49 -0500
Subject: [OmniOS-discuss] KVM - Windows guest disks
Message-ID: <BAY167-W997A361E0D4A40CF22B8AEAE630@phx.gbl>



Hi,


Are there any pointers for
create the underlying ZFS volume for the windows disk.


Currently I am using block
volume created at 4k. 


The servers have LSI 2308
controllers in IT mode and I am seeing high disk queue lengths with the system
not doing really anything. The system was just installed. It also boots slow
and the pool is maxed out. 


Pool consists of 6x 7200K NL-SAS
2 drives raidz2.



 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141211/1ab0de96/attachment.html>

From sjorge+ml at blackdot.be  Thu Dec 11 17:55:57 2014
From: sjorge+ml at blackdot.be (Jorge Schrauwen)
Date: Thu, 11 Dec 2014 18:55:57 +0100
Subject: [OmniOS-discuss] KVM - Windows guest disks
In-Reply-To: <BAY167-W997A361E0D4A40CF22B8AEAE630@phx.gbl>
References: <BAY167-W997A361E0D4A40CF22B8AEAE630@phx.gbl>
Message-ID: <593b44443c150ad3a8a1d7af9d6e2285@blackdot.be>

Are you running r012? Some other people reported poor IO with KVM 
earlier this week.

On 2014-12-11 18:47, Machine Man wrote:

> Hi,
> Are there any pointers for create the underlying ZFS volume for the 
> windows disk.
> Currently I am using block volume created at 4k.
> The servers have LSI 2308 controllers in IT mode and I am seeing high 
> disk queue lengths with the system not doing really anything. The 
> system was just installed. It also boots slow and the pool is maxed 
> out.
> Pool consists of 6x 7200K NL-SAS 2 drives raidz2.
> 
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss [1]


Links:
------
[1] http://lists.omniti.com/mailman/listinfo/omnios-discuss

From chip at innovates.com  Thu Dec 11 18:30:45 2014
From: chip at innovates.com (Schweiss, Chip)
Date: Thu, 11 Dec 2014 12:30:45 -0600
Subject: [OmniOS-discuss] live lsi firmware upgrade P17->P19
In-Reply-To: <alpine.DEB.2.10.1412111638310.4496@engelberg.oetiker.ch>
References: <alpine.DEB.2.10.1412111638310.4496@engelberg.oetiker.ch>
Message-ID: <CALeZrrREh+RHcdPpUCw0MYAmEkV0qkD3Xs1u4N0gdGhZ3sBSVA@mail.gmail.com>

On Thu, Dec 11, 2014 at 9:44 AM, Tobias Oetiker <tobi at oetiker.ch> wrote:

> I am looking at upgrading the firmware of our LSI HBAs to P19 since
> we suspect that our use of P17 is the cause for disk timeouts we
> are seeing every few weeks.
>
> I am wondering, is it save to flash the HBAs from a running omnios system,
> and if so, has anyone written some notes down on the process and
> tools to use?
>
>
I don't believe that is possible on a live system.  I have upgraded using
the Solaris sas2flash utility on a system with the pool exported.   The
last step of the flash function is to reset the adapter which fails
requiring the system to be rebooted.

What I haven't tried is disconnecting the SAS cables then applying the
update.   I suspect this may work, but is only a useful method if you have
redundant SAS paths and can do one HBA at a time.

These issues are why I run HA systems and can migrate my pools from server
to server and perform maintenance at a leisurely pace and confirm
everything is the way I want it before putting the server back in service.

-Chip




> cheers
> tobi
>
>
> --
> Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten, Switzerland
> www.oetiker.ch tobi at oetiker.ch +41 62 775 9902
>
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141211/d09ba6cd/attachment.html>

From joeveliscos at gmail.com  Thu Dec 11 21:00:53 2014
From: joeveliscos at gmail.com (Joe Veliscos)
Date: Thu, 11 Dec 2014 22:00:53 +0100
Subject: [OmniOS-discuss] Python difference
Message-ID: <CAHNoB+1iJ=TNq9Z0sZ-WO-BMtx--tXcrXkVyWvcu3UfS-W+CCg@mail.gmail.com>

Hi,

I have an application working on Omnios r10 which depends on certain python
libraries. There are certain environment variables in place which point to
the location of those libraries.

I have updated the r10 machine to r12. The application now cannot be
started with errors stating that the needed libraries cannot be found in
the given paths.

Maybe somebody can tell me what the difference is in python version between
the two omnios releases. Python -V gives me 2.6.8. on both versions.

I am looking for pointers I can use to make this application work.

Rgds

Joe
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141211/718750eb/attachment.html>

From danmcd at omniti.com  Thu Dec 11 21:26:23 2014
From: danmcd at omniti.com (Dan McDonald)
Date: Thu, 11 Dec 2014 16:26:23 -0500
Subject: [OmniOS-discuss] Python difference
In-Reply-To: <CAHNoB+1iJ=TNq9Z0sZ-WO-BMtx--tXcrXkVyWvcu3UfS-W+CCg@mail.gmail.com>
References: <CAHNoB+1iJ=TNq9Z0sZ-WO-BMtx--tXcrXkVyWvcu3UfS-W+CCg@mail.gmail.com>
Message-ID: <56C3D19D-B648-4F02-A7CA-BC24A31C385E@omniti.com>


> On Dec 11, 2014, at 4:00 PM, Joe Veliscos <joeveliscos at gmail.com> wrote:
> 
> Hi,
> 
> I have an application working on Omnios r10 which depends on certain python libraries. There are certain environment variables in place which point to the location of those libraries.
> 
> I have updated the r10 machine to r12. The application now cannot be started with errors stating that the needed libraries cannot be found in the given paths. 

Share the errors please?  I'll need more details.

> Maybe somebody can tell me what the difference is in python version between the two omnios releases. Python -V gives me 2.6.8. on both versions. 

We updated supplemental python libraries, which may contribute to what you're seeing.  Also, we had not updated the "entire" metapackage on 010 to show we were actually running 2.6.8.

Dan


From joeveliscos at gmail.com  Thu Dec 11 22:23:49 2014
From: joeveliscos at gmail.com (Joe Veliscos)
Date: Thu, 11 Dec 2014 23:23:49 +0100
Subject: [OmniOS-discuss] Python difference
In-Reply-To: <56C3D19D-B648-4F02-A7CA-BC24A31C385E@omniti.com>
References: <CAHNoB+1iJ=TNq9Z0sZ-WO-BMtx--tXcrXkVyWvcu3UfS-W+CCg@mail.gmail.com>
	<56C3D19D-B648-4F02-A7CA-BC24A31C385E@omniti.com>
Message-ID: <CAHNoB+1pePnQuUNHSBKLN14975bqHPX1C6F1E7nciZjkE2qQDw@mail.gmail.com>

Hi

The command that I execute :
root#crm
abort: couldn't find crm libraries in [/opt/ha/sbin
/usr/lib/python2.6/vendor-packages/setuptools-0.6c11-py2.6.egg
/opt/ha/lib/python2.6/site-packages /usr/lib/python26.zip
/usr/lib/python2.6 /usr/lib/python2.6/plat-sunos5 /usr/lib/python2.6/lib-tk
/usr/lib/python2.6/lib-old /usr/lib/python2.6/lib-dynload
/usr/lib/python2.6/site-packages /usr/lib/python2.6/vendor-packages]
(check your install and PYTHONPATH)

I have the following environment variables set:
export PYTHONPATH=/opt/ha/lib/python2.6/site-packages
export PATH=/opt/ha/bin:/opt/ha/sbin:$PATH
export OCF_ROOT=/opt/ha/lib/ocf
export OCF_AGENTS=/opt/ha/lib/ocf/resource.d/heartbeat

I have exactly the same in an r10 release (pre upgrade to rr12) where there
is no problem

I did a truss -d crm and it seems that many files it searches for are not
found.  Snippets of the output (very long file) hope this helps:

Below (as I understand it ) some searches which it can resolve:

0.0098    resolvepath("/usr/lib/amd64/ld.so.1", "/lib/amd64/ld.so.1", 1023)
= 18
 0.0100    resolvepath("/usr/bin/amd64/python2.6",
"/usr/bin/amd64/python2.6", 1023) = 24
 0.0101    stat("/usr/bin/amd64/python2.6", 0xFFFFFD7FFFDFF910) = 0
 0.0103    open("/var/ld/64/ld.config", O_RDONLY)        Err#2 ENOENT
 0.0105    stat("/usr/gnu/lib/amd64/libpython2.6.so.1.0",
0xFFFFFD7FFFDFF000) Err#2 ENOENT
 0.0106    stat("/usr/lib/amd64/libpython2.6.so.1.0", 0xFFFFFD7FFFDFF000) =
0
 0.0108    resolvepath("/usr/lib/amd64/libpython2.6.so.1.0",
"/usr/lib/amd64/libpython2.6.so.1.0", 1023) = 34
 0.0110    open("/usr/lib/amd64/libpython2.6.so.1.0", O_RDONLY) = 3
 0.0112    mmapobj(3, MMOBJ_INTERPRET, 0xFFFFFD7FFF350AB8,
0xFFFFFD7FFFDFEB5C, 0x00000000) = 0
 0.0113    close(3)                    = 0
 0.0115    mmap(0x00000000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC,
MAP_PRIVATE|MAP_ANON, 4294967295, 0) = 0xFFFFFD7FFF340000
 0.0117    memcntl(0xFFFFFD7FFEAB0000, 457808, MC_ADVISE, MADV_WILLNEED, 0,
0) = 0
 0.0118    stat("/usr/gnu/lib/amd64/libsocket.so.1", 0xFFFFFD7FFFDFF000)
Err#2 ENOENT
 0.0120    stat("/usr/lib/amd64/libsocket.so.1", 0xFFFFFD7FFFDFF000) = 0
 0.0121    resolvepath("/usr/lib/amd64/libsocket.so.1",
"/lib/amd64/libsocket.so.1", 1023) = 25
 0.0123    open("/usr/lib/amd64/libsocket.so.1", O_RDONLY)    = 3
 0.0125    mmapobj(3, MMOBJ_INTERPRET, 0xFFFFFD7FFF340A18,
0xFFFFFD7FFFDFEB5C, 0x00000000) = 0
 0.0127    close(3)                    = 0
 0.0128    mmap(0x00000000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC,
MAP_PRIVATE|MAP_ANON, 4294967295, 0) = 0xFFFFFD7FFF330000
 0.0129    memcntl(0xFFFFFD7FFEA80000, 32240, MC_ADVISE, MADV_WILLNEED, 0,
0) = 0
 0.0130    stat("/usr/gnu/lib/amd64/libnsl.so.1", 0xFFFFFD7FFFDFF000) Err#2
ENOENT
 0.0132    stat("/usr/lib/amd64/libnsl.so.1", 0xFFFFFD7FFFDFF000) = 0
 0.0134    resolvepath("/usr/lib/amd64/libnsl.so.1",
"/lib/amd64/libnsl.so.1", 1023) = 22
 0.0135    open("/usr/lib/amd64/libnsl.so.1", O_RDONLY)    = 3
 0.0137    mmapobj(3, MMOBJ_INTERPRET, 0xFFFFFD7FFF3309C8,
0xFFFFFD7FFFDFEB5C, 0x00000000) = 0
 0.0139    close(3)                    = 0
 0.0140    mmap(0x00000000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC,
MAP_PRIVATE|MAP_ANON, 4294967295, 0) = 0xFFFFFD7FFF320000
 0.0141    memcntl(0xFFFFFD7FFEDD0000, 180072, MC_ADVISE, MADV_WILLNEED, 0,
0) = 0
 0.0142    stat("/usr/gnu/lib/amd64/libm.so.2", 0xFFFFFD7FFFDFF000) Err#2
ENOENT
 0.0144    stat("/usr/lib/amd64/libm.so.2", 0xFFFFFD7FFFDFF000) = 0
 0.0145    resolvepath("/usr/lib/amd64/libm.so.2", "/lib/amd64/libm.so.2",
1023) = 20
 0.0147    open("/usr/lib/amd64/libm.so.2", O_RDONLY)    = 3
 0.0149    mmapobj(3, MMOBJ_INTERPRET, 0xFFFFFD7FFF3209F8,
0xFFFFFD7FFFDFEB5C, 0x00000000) = 0
 0.0150    close(3)                    = 0
 0.0151    mmap(0x00000000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC,
MAP_PRIVATE|MAP_ANON, 4294967295, 0) = 0xFFFFFD7FFF310000
 0.0153    memcntl(0xFFFFFD7FFEEE0000, 58680, MC_ADVISE, MADV_WILLNEED, 0,
0) = 0
 0.0154    stat("/usr/gnu/lib/amd64/libc.so.1", 0xFFFFFD7FFFDFF000) Err#2
ENOENT
 0.0156    stat("/usr/lib/amd64/libc.so.1", 0xFFFFFD7FFFDFF000) = 0
 0.0157    resolvepath("/usr/lib/amd64/libc.so.1", "/lib/amd64/libc.so.1",
1023) = 20
 0.0159    open("/usr/lib/amd64/libc.so.1", O_RDONLY)    = 3
 0.0161    mmapobj(3, MMOBJ_INTERPRET, 0xFFFFFD7FFF310920,
0xFFFFFD7FFFDFEB5C, 0x00000000) = 0
 0.0162    close(3)                    = 0
 0.0163    mmap(0x00000000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC,
MAP_PRIVATE|MAP_ANON, 4294967295, 0) = 0xFFFFFD7FFF160000
 0.0165    memcntl(0xFFFFFD7FFF170000, 477048, MC_ADVISE, MADV_WILLNEED, 0,
0) = 0
 0.0166    stat("/lib/64/libsocket.so.1", 0xFFFFFD7FFFDFF000) = 0
 0.0168    resolvepath("/lib/64/libsocket.so.1",
"/lib/amd64/libsocket.so.1", 1023) = 25
 0.0170    stat("/lib/64/libnsl.so.1", 0xFFFFFD7FFFDFF000)    = 0
 0.0171    resolvepath("/lib/64/libnsl.so.1", "/lib/amd64/libnsl.so.1",
1023) = 22
 0.0173    stat("/lib/64/libm.so.2", 0xFFFFFD7FFFDFF000)    = 0
 0.0174    resolvepath("/lib/64/libm.so.2", "/lib/amd64/libm.so.2", 1023) =
20
 0.0177    stat("/usr/gnu/lib/amd64/libgcc_s.so.1", 0xFFFFFD7FFFDFF000)
Err#2 ENOENT
 0.0179    stat("/lib/64/libgcc_s.so.1", 0xFFFFFD7FFFDFF000) Err#2 ENOENT
 0.0180    stat("/usr/lib/64/libgcc_s.so.1", 0xFFFFFD7FFFDFF000) = 0
 0.0185    resolvepath("/usr/lib/64/libgcc_s.so.1",
"/usr/lib/amd64/libgcc_s.so.1", 1023) = 28
 0.0187    open("/usr/lib/64/libgcc_s.so.1", O_RDONLY)    = 3

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Below searches which it cannot resolve:

0.0329    fstat(2, 0xFFFFFD7FFFDFF920)            = 0
 0.0331    readlink("/usr/bin/python", "python2.6", 1024)    = 9
 0.0333    readlink("/usr/bin/python2.6", 0xFFFFFD7FFFDFF5B0, 1024) Err#22
EINVAL
 0.0335    stat("/usr/bin/Modules/Setup", 0xFFFFFD7FFFDFF5B0) Err#2 ENOENT
 0.0336    stat("/usr/bin/lib/python2.6/os.py", 0xFFFFFD7FFFDFF5B0) Err#2
ENOENT
 0.0338    stat("/usr/bin/lib/python2.6/os.pyc", 0xFFFFFD7FFFDFF5B0) Err#2
ENOENT
 0.0342    stat("/usr/lib/python2.6/os.py", 0xFFFFFD7FFFDFF5B0) = 0
 0.0344    stat("/usr/bin/Modules/Setup", 0xFFFFFD7FFFDFF120) Err#2 ENOENT
 0.0345    stat("/usr/bin/lib/python2.6/lib-dynload", 0xFFFFFD7FFFDFF120)
Err#2 ENOENT
 0.0347    stat("/usr/lib/python2.6/lib-dynload", 0xFFFFFD7FFFDFF120) = 0
 0.0351    brk(0x00485250)                    = 0

 0.0456    sysconfig(_CONFIG_SIGRT_MAX)            = 73
 0.0459    stat("/opt/ha/lib/python2.6/site-packages", 0xFFFFFD7FFFDFDF90)
= 0
 0.0461    stat("/opt/ha/lib/python2.6/site-packages", 0xFFFFFD7FFFDFE340)
= 0
 0.0462    stat("/opt/ha/lib/python2.6/site-packages/site",
0xFFFFFD7FFFDFE640) Err#2 ENOENT
 0.0464    stat("/opt/ha/lib/python2.6/site-packages/site",
0xFFFFFD7FFFDFEAD0) Err#2 ENOENT
 0.0465    open("/opt/ha/lib/python2.6/site-packages/64/site.so", O_RDONLY)
Err#2 ENOENT
 0.0467    stat("/opt/ha/lib/python2.6/site-packages/site",
0xFFFFFD7FFFDFEAD0) Err#2 ENOENT
 0.0469    open("/opt/ha/lib/python2.6/site-packages/64/sitemodule.so",
O_RDONLY) Err#2 ENOENT
 0.0470    stat("/opt/ha/lib/python2.6/site-packages/site",
0xFFFFFD7FFFDFEAD0) Err#2 ENOENT
 0.0471    open("/opt/ha/lib/python2.6/site-packages/site.py", O_RDONLY)
Err#2 ENOENT
 0.0473    stat("/opt/ha/lib/python2.6/site-packages/site",
0xFFFFFD7FFFDFEAD0) Err#2 ENOENT
 0.0474    open("/opt/ha/lib/python2.6/site-packages/site.pyc", O_RDONLY)
Err#2 ENOENT
 0.0476    stat("/usr/lib/python26.zip", 0xFFFFFD7FFFDFDF90) Err#2 ENOENT
 0.0477    stat("/usr/lib", 0xFFFFFD7FFFDFDF90)        = 0
 0.0478    stat("/usr/lib/python26.zip", 0xFFFFFD7FFFDFE340) Err#2 ENOENT
 0.0480    stat("/usr/lib/python2.6/", 0xFFFFFD7FFFDFDF90)    = 0
 0.0481    stat("/usr/lib/python2.6/", 0xFFFFFD7FFFDFE340)    = 0
 0.0482    stat("/usr/lib/python2.6/site", 0xFFFFFD7FFFDFE640) Err#2 ENOENT
 0.0483    stat("/usr/lib/python2.6/site", 0xFFFFFD7FFFDFEAD0) Err#2 ENOENT
 0.0485    open("/usr/lib/python2.6/64/site.so", O_RDONLY)    Err#2 ENOENT
 0.0486    stat("/usr/lib/python2.6/site", 0xFFFFFD7FFFDFEAD0) Err#2 ENOENT
 0.0487    open("/usr/lib/python2.6/64/sitemodule.so", O_RDONLY) Err#2
ENOENT
 0.0488    stat("/usr/lib/python2.6/site", 0xFFFFFD7FFFDFEAD0) Err#2 ENOENT
 0.0490    open("/usr/lib/python2.6/site.py", O_RDONLY)    = 3
 0.0491    fstat(3, 0xFFFFFD7FFFDFEA90)            = 0
 0.0492    open("/usr/lib/python2.6/site.pyc", O_RDONLY)    = 4
 0.0493    fstat(4, 0xFFFFFD7FFFDFE8D0)            = 0
 0.0494    brk(0x004D5250)                    = 0
 0.0496    brk(0x004D9250)                    = 0
 0.0498    fstat(4, 0xFFFFFD7FFFDFE800)            = 0
 0.0498    ioctl(4, TCGETA, 0xFFFFFD7FFFDFE880)        Err#25 ENOTTY
 0.0500    read(4, "D1F2\r\nA09390 S c\0\0\0".., 18944)    = 18651

0.0514    stat("/opt/ha/lib/python2.6/site-packages/os",
0xFFFFFD7FFFDFD320) Err#2 ENOENT
 0.0516    stat("/opt/ha/lib/python2.6/site-packages/os",
0xFFFFFD7FFFDFD7B0) Err#2 ENOENT
 0.0517    open("/opt/ha/lib/python2.6/site-packages/64/os.so", O_RDONLY)
Err#2 ENOENT
 0.0519    stat("/opt/ha/lib/python2.6/site-packages/os",
0xFFFFFD7FFFDFD7B0) Err#2 ENOENT
 0.0520    open("/opt/ha/lib/python2.6/site-packages/64/osmodule.so",
O_RDONLY) Err#2 ENOENT
 0.0521    stat("/opt/ha/lib/python2.6/site-packages/os",
0xFFFFFD7FFFDFD7B0) Err#2 ENOENT
 0.0522    open("/opt/ha/lib/python2.6/site-packages/os.py", O_RDONLY)
Err#2 ENOENT
 0.0524    stat("/opt/ha/lib/python2.6/site-packages/os",
0xFFFFFD7FFFDFD7B0) Err#2 ENOENT
 0.0525    open("/opt/ha/lib/python2.6/site-packages/os.pyc", O_RDONLY)
Err#2 ENOENT
 0.0527    stat("/usr/lib/python2.6/os", 0xFFFFFD7FFFDFD320) Err#2 ENOENT
 0.0528    stat("/usr/lib/python2.6/os", 0xFFFFFD7FFFDFD7B0) Err#2 ENOENT
 0.0529    open("/usr/lib/python2.6/64/os.so", O_RDONLY)    Err#2 ENOENT
 0.0530    stat("/usr/lib/python2.6/os", 0xFFFFFD7FFFDFD7B0) Err#2 ENOENT
 0.0532    open("/usr/lib/python2.6/64/osmodule.so", O_RDONLY) Err#2 ENOENT
 0.0533    stat("/usr/lib/python2.6/os", 0xFFFFFD7FFFDFD7B0) Err#2 ENOENT
 0.0534    open("/usr/lib/python2.6/os.py", O_RDONLY)    = 4
 0.0536    fstat(4, 0xFFFFFD7FFFDFD770)            = 0
 0.0537    open("/usr/lib/python2.6/os.pyc", O_RDONLY)    = 5
 0.0538    fstat(5, 0xFFFFFD7FFFDFD5B0)            = 0
 0.0539    fstat(5, 0xFFFFFD7FFFDFD4E0)            = 0
 0.0540    ioctl(5, TCGETA, 0xFFFFFD7FFFDFD560)        Err#25 ENOTTY
 0.0541    read(5, "D1F2\r\n9F9390 S c\0\0\0".., 26112)    = 25702

 0.0568    stat("/opt/ha/lib/python2.6/site-packages/posixpath",
0xFFFFFD7FFFDFC000) Err#2 ENOENT
 0.0570    stat("/opt/ha/lib/python2.6/site-packages/posixpath",
0xFFFFFD7FFFDFC490) Err#2 ENOENT
 0.0572    open("/opt/ha/lib/python2.6/site-packages/64/posixpath.so",
O_RDONLY) Err#2 ENOENT
 0.0573    stat("/opt/ha/lib/python2.6/site-packages/posixpath",
0xFFFFFD7FFFDFC490) Err#2 ENOENT
 0.0575
open("/opt/ha/lib/python2.6/site-packages/64/posixpathmodule.so", O_RDONLY)
Err#2 ENOENT
 0.0576    stat("/opt/ha/lib/python2.6/site-packages/posixpath",
0xFFFFFD7FFFDFC490) Err#2 ENOENT
 0.0577    open("/opt/ha/lib/python2.6/site-packages/posixpath.py",
O_RDONLY) Err#2 ENOENT
 0.0579    stat("/opt/ha/lib/python2.6/site-packages/posixpath",
0xFFFFFD7FFFDFC490) Err#2 ENOENT
 0.0580    open("/opt/ha/lib/python2.6/site-packages/posixpath.pyc",
O_RDONLY) Err#2 ENOENT
 0.0582    stat("/usr/lib/python2.6/posixpath", 0xFFFFFD7FFFDFC000) Err#2
ENOENT
 0.0583    stat("/usr/lib/python2.6/posixpath", 0xFFFFFD7FFFDFC490) Err#2
ENOENT
 0.0584    open("/usr/lib/python2.6/64/posixpath.so", O_RDONLY) Err#2 ENOENT
 0.0585    stat("/usr/lib/python2.6/posixpath", 0xFFFFFD7FFFDFC490) Err#2
ENOENT
 0.0587    open("/usr/lib/python2.6/64/posixpathmodule.so", O_RDONLY) Err#2
ENOENT
 0.0588    stat("/usr/lib/python2.6/posixpath", 0xFFFFFD7FFFDFC490) Err#2
ENOENT
 0.0589    open("/usr/lib/python2.6/posixpath.py", O_RDONLY) = 5
 0.0591    fstat(5, 0xFFFFFD7FFFDFC450)            = 0
 0.0592    open("/usr/lib/python2.6/posixpath.pyc", O_RDONLY) = 6
 0.0593    fstat(6, 0xFFFFFD7FFFDFC290)            = 0


And the list goes on.

Hope there's a solution for this.


Joe


On Thu, Dec 11, 2014 at 10:26 PM, Dan McDonald <danmcd at omniti.com> wrote:
>
>
> > On Dec 11, 2014, at 4:00 PM, Joe Veliscos <joeveliscos at gmail.com> wrote:
> >
> > Hi,
> >
> > I have an application working on Omnios r10 which depends on certain
> python libraries. There are certain environment variables in place which
> point to the location of those libraries.
> >
> > I have updated the r10 machine to r12. The application now cannot be
> started with errors stating that the needed libraries cannot be found in
> the given paths.
>
> Share the errors please?  I'll need more details.
>
> > Maybe somebody can tell me what the difference is in python version
> between the two omnios releases. Python -V gives me 2.6.8. on both versions.
>
> We updated supplemental python libraries, which may contribute to what
> you're seeing.  Also, we had not updated the "entire" metapackage on 010 to
> show we were actually running 2.6.8.
>
> Dan
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141211/a2335797/attachment-0001.html>

From rt at steait.net  Thu Dec 11 22:25:48 2014
From: rt at steait.net (Rune Tipsmark)
Date: Thu, 11 Dec 2014 22:25:48 +0000
Subject: [OmniOS-discuss] hangs on reboot
Message-ID: <1418336750819.83182@steait.net>

hi all,



I got a bunch (3) installations of omnios on SuperMicro hardware and all 3 have issues rebooting. They simply hang and never ever reboot.



The install is latest version and I only added the storage-server package and installed napp-it and changed the fibre channel setting in /kernel/drv/emlxs.conf target-mode=1



two nics igb0 and igb1 configured as aggregation (aggr0)



besides this its 100% default install, not 10 minutes old...



Any ideas?

br,

Rune
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141211/95886ed3/attachment.html>

From henson at acm.org  Thu Dec 11 22:32:26 2014
From: henson at acm.org (Paul B. Henson)
Date: Thu, 11 Dec 2014 14:32:26 -0800
Subject: [OmniOS-discuss] hangs on reboot
In-Reply-To: <1418336750819.83182@steait.net>
References: <1418336750819.83182@steait.net>
Message-ID: <893101d01592$59168140$0b4383c0$@acm.org>

> Rune Tipsmark
> Sent: Thursday, December 11, 2014 2:26 PM
> 
> I got a bunch (3) installations of omnios on SuperMicro hardware and all 3
> have issues rebooting. They simply hang and never ever reboot.

Disable fast reboot:

svccfg -s "system/boot-config:default" setprop
config/fastreboot_default=false
svccfg -s "system/boot-config:default" setprop
config/fastreboot_onpanic=false
svcadm refresh svc:/system/boot-config:default

Some systems (it seems particularly common on supermicro hardware) tend to
wedge up with fast reboot enabled. If you want to verify this is the issue
before changing the configuration, try 'reboot -p' which should do a one
time regular reboot without changing the default.

Dan, there was some talk of making fast reboot disabled the default, and
having people who wanted it need to enable it, rather than the other way
around? Any thoughts?



From danmcd at omniti.com  Thu Dec 11 22:37:18 2014
From: danmcd at omniti.com (Dan McDonald)
Date: Thu, 11 Dec 2014 17:37:18 -0500
Subject: [OmniOS-discuss] Python difference
In-Reply-To: <CAHNoB+1pePnQuUNHSBKLN14975bqHPX1C6F1E7nciZjkE2qQDw@mail.gmail.com>
References: <CAHNoB+1iJ=TNq9Z0sZ-WO-BMtx--tXcrXkVyWvcu3UfS-W+CCg@mail.gmail.com>
	<56C3D19D-B648-4F02-A7CA-BC24A31C385E@omniti.com>
	<CAHNoB+1pePnQuUNHSBKLN14975bqHPX1C6F1E7nciZjkE2qQDw@mail.gmail.com>
Message-ID: <02CECDF1-3154-40CD-B120-38C082C0D1AF@omniti.com>

Did you look to see if there's anything in /opt/ha/lib/python2.6/site-packages ?

The massive ENOENTs you're showing me are just library searching.  See how several path and filename combos are tried?

HA should be more clear about how it failed.  Also, look toward the end of the truss, and just look for open calls.  Once you see a string of failures that doesn't end with a success, you'll know what's missing.

Dan

Sent from my iPhone (typos, autocorrect, and all)

> On Dec 11, 2014, at 5:23 PM, Joe Veliscos <joeveliscos at gmail.com> wrote:
> 
> Hi 
> 
> The command that I execute :
> root#crm
> abort: couldn't find crm libraries in [/opt/ha/sbin /usr/lib/python2.6/vendor-packages/setuptools-0.6c11-py2.6.egg /opt/ha/lib/python2.6/site-packages /usr/lib/python26.zip /usr/lib/python2.6 /usr/lib/python2.6/plat-sunos5 /usr/lib/python2.6/lib-tk /usr/lib/python2.6/lib-old /usr/lib/python2.6/lib-dynload /usr/lib/python2.6/site-packages /usr/lib/python2.6/vendor-packages]
> (check your install and PYTHONPATH)
> 
> I have the following environment variables set:
> export PYTHONPATH=/opt/ha/lib/python2.6/site-packages
> export PATH=/opt/ha/bin:/opt/ha/sbin:$PATH
> export OCF_ROOT=/opt/ha/lib/ocf
> export OCF_AGENTS=/opt/ha/lib/ocf/resource.d/heartbeat
> 
> I have exactly the same in an r10 release (pre upgrade to rr12) where there is no problem
> 
> I did a truss -d crm and it seems that many files it searches for are not found.  Snippets of the output (very long file) hope this helps:
> 
> Below (as I understand it ) some searches which it can resolve:
> 
> 0.0098    resolvepath("/usr/lib/amd64/ld.so.1", "/lib/amd64/ld.so.1", 1023) = 18
>  0.0100    resolvepath("/usr/bin/amd64/python2.6", "/usr/bin/amd64/python2.6", 1023) = 24
>  0.0101    stat("/usr/bin/amd64/python2.6", 0xFFFFFD7FFFDFF910) = 0
>  0.0103    open("/var/ld/64/ld.config", O_RDONLY)        Err#2 ENOENT
>  0.0105    stat("/usr/gnu/lib/amd64/libpython2.6.so.1.0", 0xFFFFFD7FFFDFF000) Err#2 ENOENT
>  0.0106    stat("/usr/lib/amd64/libpython2.6.so.1.0", 0xFFFFFD7FFFDFF000) = 0
>  0.0108    resolvepath("/usr/lib/amd64/libpython2.6.so.1.0", "/usr/lib/amd64/libpython2.6.so.1.0", 1023) = 34
>  0.0110    open("/usr/lib/amd64/libpython2.6.so.1.0", O_RDONLY) = 3
>  0.0112    mmapobj(3, MMOBJ_INTERPRET, 0xFFFFFD7FFF350AB8, 0xFFFFFD7FFFDFEB5C, 0x00000000) = 0
>  0.0113    close(3)                    = 0
>  0.0115    mmap(0x00000000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANON, 4294967295, 0) = 0xFFFFFD7FFF340000
>  0.0117    memcntl(0xFFFFFD7FFEAB0000, 457808, MC_ADVISE, MADV_WILLNEED, 0, 0) = 0
>  0.0118    stat("/usr/gnu/lib/amd64/libsocket.so.1", 0xFFFFFD7FFFDFF000) Err#2 ENOENT
>  0.0120    stat("/usr/lib/amd64/libsocket.so.1", 0xFFFFFD7FFFDFF000) = 0
>  0.0121    resolvepath("/usr/lib/amd64/libsocket.so.1", "/lib/amd64/libsocket.so.1", 1023) = 25
>  0.0123    open("/usr/lib/amd64/libsocket.so.1", O_RDONLY)    = 3
>  0.0125    mmapobj(3, MMOBJ_INTERPRET, 0xFFFFFD7FFF340A18, 0xFFFFFD7FFFDFEB5C, 0x00000000) = 0
>  0.0127    close(3)                    = 0
>  0.0128    mmap(0x00000000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANON, 4294967295, 0) = 0xFFFFFD7FFF330000
>  0.0129    memcntl(0xFFFFFD7FFEA80000, 32240, MC_ADVISE, MADV_WILLNEED, 0, 0) = 0
>  0.0130    stat("/usr/gnu/lib/amd64/libnsl.so.1", 0xFFFFFD7FFFDFF000) Err#2 ENOENT
>  0.0132    stat("/usr/lib/amd64/libnsl.so.1", 0xFFFFFD7FFFDFF000) = 0
>  0.0134    resolvepath("/usr/lib/amd64/libnsl.so.1", "/lib/amd64/libnsl.so.1", 1023) = 22
>  0.0135    open("/usr/lib/amd64/libnsl.so.1", O_RDONLY)    = 3
>  0.0137    mmapobj(3, MMOBJ_INTERPRET, 0xFFFFFD7FFF3309C8, 0xFFFFFD7FFFDFEB5C, 0x00000000) = 0
>  0.0139    close(3)                    = 0
>  0.0140    mmap(0x00000000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANON, 4294967295, 0) = 0xFFFFFD7FFF320000
>  0.0141    memcntl(0xFFFFFD7FFEDD0000, 180072, MC_ADVISE, MADV_WILLNEED, 0, 0) = 0
>  0.0142    stat("/usr/gnu/lib/amd64/libm.so.2", 0xFFFFFD7FFFDFF000) Err#2 ENOENT
>  0.0144    stat("/usr/lib/amd64/libm.so.2", 0xFFFFFD7FFFDFF000) = 0
>  0.0145    resolvepath("/usr/lib/amd64/libm.so.2", "/lib/amd64/libm.so.2", 1023) = 20
>  0.0147    open("/usr/lib/amd64/libm.so.2", O_RDONLY)    = 3
>  0.0149    mmapobj(3, MMOBJ_INTERPRET, 0xFFFFFD7FFF3209F8, 0xFFFFFD7FFFDFEB5C, 0x00000000) = 0
>  0.0150    close(3)                    = 0
>  0.0151    mmap(0x00000000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANON, 4294967295, 0) = 0xFFFFFD7FFF310000
>  0.0153    memcntl(0xFFFFFD7FFEEE0000, 58680, MC_ADVISE, MADV_WILLNEED, 0, 0) = 0
>  0.0154    stat("/usr/gnu/lib/amd64/libc.so.1", 0xFFFFFD7FFFDFF000) Err#2 ENOENT
>  0.0156    stat("/usr/lib/amd64/libc.so.1", 0xFFFFFD7FFFDFF000) = 0
>  0.0157    resolvepath("/usr/lib/amd64/libc.so.1", "/lib/amd64/libc.so.1", 1023) = 20
>  0.0159    open("/usr/lib/amd64/libc.so.1", O_RDONLY)    = 3
>  0.0161    mmapobj(3, MMOBJ_INTERPRET, 0xFFFFFD7FFF310920, 0xFFFFFD7FFFDFEB5C, 0x00000000) = 0
>  0.0162    close(3)                    = 0
>  0.0163    mmap(0x00000000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANON, 4294967295, 0) = 0xFFFFFD7FFF160000
>  0.0165    memcntl(0xFFFFFD7FFF170000, 477048, MC_ADVISE, MADV_WILLNEED, 0, 0) = 0
>  0.0166    stat("/lib/64/libsocket.so.1", 0xFFFFFD7FFFDFF000) = 0
>  0.0168    resolvepath("/lib/64/libsocket.so.1", "/lib/amd64/libsocket.so.1", 1023) = 25
>  0.0170    stat("/lib/64/libnsl.so.1", 0xFFFFFD7FFFDFF000)    = 0
>  0.0171    resolvepath("/lib/64/libnsl.so.1", "/lib/amd64/libnsl.so.1", 1023) = 22
>  0.0173    stat("/lib/64/libm.so.2", 0xFFFFFD7FFFDFF000)    = 0
>  0.0174    resolvepath("/lib/64/libm.so.2", "/lib/amd64/libm.so.2", 1023) = 20
>  0.0177    stat("/usr/gnu/lib/amd64/libgcc_s.so.1", 0xFFFFFD7FFFDFF000) Err#2 ENOENT
>  0.0179    stat("/lib/64/libgcc_s.so.1", 0xFFFFFD7FFFDFF000) Err#2 ENOENT
>  0.0180    stat("/usr/lib/64/libgcc_s.so.1", 0xFFFFFD7FFFDFF000) = 0
>  0.0185    resolvepath("/usr/lib/64/libgcc_s.so.1", "/usr/lib/amd64/libgcc_s.so.1", 1023) = 28
>  0.0187    open("/usr/lib/64/libgcc_s.so.1", O_RDONLY)    = 3
> 
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> Below searches which it cannot resolve:
> 
> 0.0329    fstat(2, 0xFFFFFD7FFFDFF920)            = 0
>  0.0331    readlink("/usr/bin/python", "python2.6", 1024)    = 9
>  0.0333    readlink("/usr/bin/python2.6", 0xFFFFFD7FFFDFF5B0, 1024) Err#22 EINVAL
>  0.0335    stat("/usr/bin/Modules/Setup", 0xFFFFFD7FFFDFF5B0) Err#2 ENOENT
>  0.0336    stat("/usr/bin/lib/python2.6/os.py", 0xFFFFFD7FFFDFF5B0) Err#2 ENOENT
>  0.0338    stat("/usr/bin/lib/python2.6/os.pyc", 0xFFFFFD7FFFDFF5B0) Err#2 ENOENT
>  0.0342    stat("/usr/lib/python2.6/os.py", 0xFFFFFD7FFFDFF5B0) = 0
>  0.0344    stat("/usr/bin/Modules/Setup", 0xFFFFFD7FFFDFF120) Err#2 ENOENT
>  0.0345    stat("/usr/bin/lib/python2.6/lib-dynload", 0xFFFFFD7FFFDFF120) Err#2 ENOENT
>  0.0347    stat("/usr/lib/python2.6/lib-dynload", 0xFFFFFD7FFFDFF120) = 0
>  0.0351    brk(0x00485250)                    = 0
> 
>  0.0456    sysconfig(_CONFIG_SIGRT_MAX)            = 73
>  0.0459    stat("/opt/ha/lib/python2.6/site-packages", 0xFFFFFD7FFFDFDF90) = 0
>  0.0461    stat("/opt/ha/lib/python2.6/site-packages", 0xFFFFFD7FFFDFE340) = 0
>  0.0462    stat("/opt/ha/lib/python2.6/site-packages/site", 0xFFFFFD7FFFDFE640) Err#2 ENOENT
>  0.0464    stat("/opt/ha/lib/python2.6/site-packages/site", 0xFFFFFD7FFFDFEAD0) Err#2 ENOENT
>  0.0465    open("/opt/ha/lib/python2.6/site-packages/64/site.so", O_RDONLY) Err#2 ENOENT
>  0.0467    stat("/opt/ha/lib/python2.6/site-packages/site", 0xFFFFFD7FFFDFEAD0) Err#2 ENOENT
>  0.0469    open("/opt/ha/lib/python2.6/site-packages/64/sitemodule.so", O_RDONLY) Err#2 ENOENT
>  0.0470    stat("/opt/ha/lib/python2.6/site-packages/site", 0xFFFFFD7FFFDFEAD0) Err#2 ENOENT
>  0.0471    open("/opt/ha/lib/python2.6/site-packages/site.py", O_RDONLY) Err#2 ENOENT
>  0.0473    stat("/opt/ha/lib/python2.6/site-packages/site", 0xFFFFFD7FFFDFEAD0) Err#2 ENOENT
>  0.0474    open("/opt/ha/lib/python2.6/site-packages/site.pyc", O_RDONLY) Err#2 ENOENT
>  0.0476    stat("/usr/lib/python26.zip", 0xFFFFFD7FFFDFDF90) Err#2 ENOENT
>  0.0477    stat("/usr/lib", 0xFFFFFD7FFFDFDF90)        = 0
>  0.0478    stat("/usr/lib/python26.zip", 0xFFFFFD7FFFDFE340) Err#2 ENOENT
>  0.0480    stat("/usr/lib/python2.6/", 0xFFFFFD7FFFDFDF90)    = 0
>  0.0481    stat("/usr/lib/python2.6/", 0xFFFFFD7FFFDFE340)    = 0
>  0.0482    stat("/usr/lib/python2.6/site", 0xFFFFFD7FFFDFE640) Err#2 ENOENT
>  0.0483    stat("/usr/lib/python2.6/site", 0xFFFFFD7FFFDFEAD0) Err#2 ENOENT
>  0.0485    open("/usr/lib/python2.6/64/site.so", O_RDONLY)    Err#2 ENOENT
>  0.0486    stat("/usr/lib/python2.6/site", 0xFFFFFD7FFFDFEAD0) Err#2 ENOENT
>  0.0487    open("/usr/lib/python2.6/64/sitemodule.so", O_RDONLY) Err#2 ENOENT
>  0.0488    stat("/usr/lib/python2.6/site", 0xFFFFFD7FFFDFEAD0) Err#2 ENOENT
>  0.0490    open("/usr/lib/python2.6/site.py", O_RDONLY)    = 3
>  0.0491    fstat(3, 0xFFFFFD7FFFDFEA90)            = 0
>  0.0492    open("/usr/lib/python2.6/site.pyc", O_RDONLY)    = 4
>  0.0493    fstat(4, 0xFFFFFD7FFFDFE8D0)            = 0
>  0.0494    brk(0x004D5250)                    = 0
>  0.0496    brk(0x004D9250)                    = 0
>  0.0498    fstat(4, 0xFFFFFD7FFFDFE800)            = 0
>  0.0498    ioctl(4, TCGETA, 0xFFFFFD7FFFDFE880)        Err#25 ENOTTY
>  0.0500    read(4, "D1F2\r\nA09390 S c\0\0\0".., 18944)    = 18651
> 
> 0.0514    stat("/opt/ha/lib/python2.6/site-packages/os", 0xFFFFFD7FFFDFD320) Err#2 ENOENT
>  0.0516    stat("/opt/ha/lib/python2.6/site-packages/os", 0xFFFFFD7FFFDFD7B0) Err#2 ENOENT
>  0.0517    open("/opt/ha/lib/python2.6/site-packages/64/os.so", O_RDONLY) Err#2 ENOENT
>  0.0519    stat("/opt/ha/lib/python2.6/site-packages/os", 0xFFFFFD7FFFDFD7B0) Err#2 ENOENT
>  0.0520    open("/opt/ha/lib/python2.6/site-packages/64/osmodule.so", O_RDONLY) Err#2 ENOENT
>  0.0521    stat("/opt/ha/lib/python2.6/site-packages/os", 0xFFFFFD7FFFDFD7B0) Err#2 ENOENT
>  0.0522    open("/opt/ha/lib/python2.6/site-packages/os.py", O_RDONLY) Err#2 ENOENT
>  0.0524    stat("/opt/ha/lib/python2.6/site-packages/os", 0xFFFFFD7FFFDFD7B0) Err#2 ENOENT
>  0.0525    open("/opt/ha/lib/python2.6/site-packages/os.pyc", O_RDONLY) Err#2 ENOENT
>  0.0527    stat("/usr/lib/python2.6/os", 0xFFFFFD7FFFDFD320) Err#2 ENOENT
>  0.0528    stat("/usr/lib/python2.6/os", 0xFFFFFD7FFFDFD7B0) Err#2 ENOENT
>  0.0529    open("/usr/lib/python2.6/64/os.so", O_RDONLY)    Err#2 ENOENT
>  0.0530    stat("/usr/lib/python2.6/os", 0xFFFFFD7FFFDFD7B0) Err#2 ENOENT
>  0.0532    open("/usr/lib/python2.6/64/osmodule.so", O_RDONLY) Err#2 ENOENT
>  0.0533    stat("/usr/lib/python2.6/os", 0xFFFFFD7FFFDFD7B0) Err#2 ENOENT
>  0.0534    open("/usr/lib/python2.6/os.py", O_RDONLY)    = 4
>  0.0536    fstat(4, 0xFFFFFD7FFFDFD770)            = 0
>  0.0537    open("/usr/lib/python2.6/os.pyc", O_RDONLY)    = 5
>  0.0538    fstat(5, 0xFFFFFD7FFFDFD5B0)            = 0
>  0.0539    fstat(5, 0xFFFFFD7FFFDFD4E0)            = 0
>  0.0540    ioctl(5, TCGETA, 0xFFFFFD7FFFDFD560)        Err#25 ENOTTY
>  0.0541    read(5, "D1F2\r\n9F9390 S c\0\0\0".., 26112)    = 25702
> 
>  0.0568    stat("/opt/ha/lib/python2.6/site-packages/posixpath", 0xFFFFFD7FFFDFC000) Err#2 ENOENT
>  0.0570    stat("/opt/ha/lib/python2.6/site-packages/posixpath", 0xFFFFFD7FFFDFC490) Err#2 ENOENT
>  0.0572    open("/opt/ha/lib/python2.6/site-packages/64/posixpath.so", O_RDONLY) Err#2 ENOENT
>  0.0573    stat("/opt/ha/lib/python2.6/site-packages/posixpath", 0xFFFFFD7FFFDFC490) Err#2 ENOENT
>  0.0575    open("/opt/ha/lib/python2.6/site-packages/64/posixpathmodule.so", O_RDONLY) Err#2 ENOENT
>  0.0576    stat("/opt/ha/lib/python2.6/site-packages/posixpath", 0xFFFFFD7FFFDFC490) Err#2 ENOENT
>  0.0577    open("/opt/ha/lib/python2.6/site-packages/posixpath.py", O_RDONLY) Err#2 ENOENT
>  0.0579    stat("/opt/ha/lib/python2.6/site-packages/posixpath", 0xFFFFFD7FFFDFC490) Err#2 ENOENT
>  0.0580    open("/opt/ha/lib/python2.6/site-packages/posixpath.pyc", O_RDONLY) Err#2 ENOENT
>  0.0582    stat("/usr/lib/python2.6/posixpath", 0xFFFFFD7FFFDFC000) Err#2 ENOENT
>  0.0583    stat("/usr/lib/python2.6/posixpath", 0xFFFFFD7FFFDFC490) Err#2 ENOENT
>  0.0584    open("/usr/lib/python2.6/64/posixpath.so", O_RDONLY) Err#2 ENOENT
>  0.0585    stat("/usr/lib/python2.6/posixpath", 0xFFFFFD7FFFDFC490) Err#2 ENOENT
>  0.0587    open("/usr/lib/python2.6/64/posixpathmodule.so", O_RDONLY) Err#2 ENOENT
>  0.0588    stat("/usr/lib/python2.6/posixpath", 0xFFFFFD7FFFDFC490) Err#2 ENOENT
>  0.0589    open("/usr/lib/python2.6/posixpath.py", O_RDONLY) = 5
>  0.0591    fstat(5, 0xFFFFFD7FFFDFC450)            = 0
>  0.0592    open("/usr/lib/python2.6/posixpath.pyc", O_RDONLY) = 6
>  0.0593    fstat(6, 0xFFFFFD7FFFDFC290)            = 0
> 
> 
> And the list goes on.
> 
> Hope there's a solution for this.
> 
> 
> Joe
> 
> 
>> On Thu, Dec 11, 2014 at 10:26 PM, Dan McDonald <danmcd at omniti.com> wrote:
>> 
>> > On Dec 11, 2014, at 4:00 PM, Joe Veliscos <joeveliscos at gmail.com> wrote:
>> >
>> > Hi,
>> >
>> > I have an application working on Omnios r10 which depends on certain python libraries. There are certain environment variables in place which point to the location of those libraries.
>> >
>> > I have updated the r10 machine to r12. The application now cannot be started with errors stating that the needed libraries cannot be found in the given paths.
>> 
>> Share the errors please?  I'll need more details.
>> 
>> > Maybe somebody can tell me what the difference is in python version between the two omnios releases. Python -V gives me 2.6.8. on both versions.
>> 
>> We updated supplemental python libraries, which may contribute to what you're seeing.  Also, we had not updated the "entire" metapackage on 010 to show we were actually running 2.6.8.
>> 
>> Dan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141211/8503b2c1/attachment-0001.html>

From danmcd at omniti.com  Thu Dec 11 22:39:37 2014
From: danmcd at omniti.com (Dan McDonald)
Date: Thu, 11 Dec 2014 17:39:37 -0500
Subject: [OmniOS-discuss] hangs on reboot
In-Reply-To: <1418336750819.83182@steait.net>
References: <1418336750819.83182@steait.net>
Message-ID: <EFBA16E3-3165-4164-B700-53A80F6D48CE@omniti.com>

Nothing printed out on the console?

And try eliminating the target mode first - just in case.

Also, is one of your NICs a dual IPMI/host NIC?  If so, disable the IPMI portion, we don't cope with shared NIC ports like that.

Dan

Sent from my iPhone (typos, autocorrect, and all)

> On Dec 11, 2014, at 5:25 PM, Rune Tipsmark <rt at steait.net> wrote:
> 
> hi all,
> 
>  
> 
> I got a bunch (3) installations of omnios on SuperMicro hardware and all 3 have issues rebooting. They simply hang and never ever reboot.
> 
>  
> 
> The install is latest version and I only added the storage-server package and installed napp-it and changed the fibre channel setting in /kernel/drv/emlxs.conf target-mode=1
> 
>  
> 
> two nics igb0 and igb1 configured as aggregation (aggr0)
> 
>  
> 
> besides this its 100% default install, not 10 minutes old...
> 
>  
> 
> Any ideas?
> 
> br,
> 
> Rune
> 
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141211/d5fa6bc6/attachment.html>

From danmcd at omniti.com  Thu Dec 11 22:40:49 2014
From: danmcd at omniti.com (Dan McDonald)
Date: Thu, 11 Dec 2014 17:40:49 -0500
Subject: [OmniOS-discuss] hangs on reboot
In-Reply-To: <893101d01592$59168140$0b4383c0$@acm.org>
References: <1418336750819.83182@steait.net>
	<893101d01592$59168140$0b4383c0$@acm.org>
Message-ID: <28B0569C-BB7A-47B2-9F73-90EB2C1D56F8@omniti.com>

Thanks for the fast reboot catch!

Yes, were contemplating it for '014 AND for upstreaming.  Not sure yet if it'll happen, but this thread makes a compelling case.

Dan

Sent from my iPhone (typos, autocorrect, and all)

On Dec 11, 2014, at 5:32 PM, Paul B. Henson <henson at acm.org> wrote:

>> Rune Tipsmark
>> Sent: Thursday, December 11, 2014 2:26 PM
>> 
>> I got a bunch (3) installations of omnios on SuperMicro hardware and all 3
>> have issues rebooting. They simply hang and never ever reboot.
> 
> Disable fast reboot:
> 
> svccfg -s "system/boot-config:default" setprop
> config/fastreboot_default=false
> svccfg -s "system/boot-config:default" setprop
> config/fastreboot_onpanic=false
> svcadm refresh svc:/system/boot-config:default
> 
> Some systems (it seems particularly common on supermicro hardware) tend to
> wedge up with fast reboot enabled. If you want to verify this is the issue
> before changing the configuration, try 'reboot -p' which should do a one
> time regular reboot without changing the default.
> 
> Dan, there was some talk of making fast reboot disabled the default, and
> having people who wanted it need to enable it, rather than the other way
> around? Any thoughts?
> 
> 
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss

From rt at steait.net  Thu Dec 11 22:41:35 2014
From: rt at steait.net (Rune Tipsmark)
Date: Thu, 11 Dec 2014 22:41:35 +0000
Subject: [OmniOS-discuss] hangs on reboot
In-Reply-To: <EFBA16E3-3165-4164-B700-53A80F6D48CE@omniti.com>
References: <1418336750819.83182@steait.net>,
	<EFBA16E3-3165-4164-B700-53A80F6D48CE@omniti.com>
Message-ID: <1418337697672.85350@steait.net>

NIC's are not shared with IPMI.



I will try the target-mode first, however on the 3rd box I actually have Infiniband and it also caused the same issue without touching the target-mode for FC.



I have a 4th system on an old HP server and it reboots perfect every time, no target-mode and has Infiniband as well... I am leaning towards something with the SuperMicro hardware but can't really pinpoint it.

br,

Rune

________________________________
From: Dan McDonald <danmcd at omniti.com>
Sent: Thursday, December 11, 2014 11:39 PM
To: Rune Tipsmark
Cc: omnios-discuss at lists.omniti.com
Subject: Re: [OmniOS-discuss] hangs on reboot

Nothing printed out on the console?

And try eliminating the target mode first - just in case.

Also, is one of your NICs a dual IPMI/host NIC?  If so, disable the IPMI portion, we don't cope with shared NIC ports like that.

Dan

Sent from my iPhone (typos, autocorrect, and all)

On Dec 11, 2014, at 5:25 PM, Rune Tipsmark <rt at steait.net<mailto:rt at steait.net>> wrote:


hi all,



I got a bunch (3) installations of omnios on SuperMicro hardware and all 3 have issues rebooting. They simply hang and never ever reboot.



The install is latest version and I only added the storage-server package and installed napp-it and changed the fibre channel setting in /kernel/drv/emlxs.conf target-mode=1



two nics igb0 and igb1 configured as aggregation (aggr0)



besides this its 100% default install, not 10 minutes old...



Any ideas?

br,

Rune

_______________________________________________
OmniOS-discuss mailing list
OmniOS-discuss at lists.omniti.com<mailto:OmniOS-discuss at lists.omniti.com>
http://lists.omniti.com/mailman/listinfo/omnios-discuss
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141211/8734f8a9/attachment.html>

From rt at steait.net  Thu Dec 11 22:48:39 2014
From: rt at steait.net (Rune Tipsmark)
Date: Thu, 11 Dec 2014 22:48:39 +0000
Subject: [OmniOS-discuss] hangs on reboot
In-Reply-To: <EFBA16E3-3165-4164-B700-53A80F6D48CE@omniti.com>
References: <1418336750819.83182@steait.net>,
	<EFBA16E3-3165-4164-B700-53A80F6D48CE@omniti.com>
Message-ID: <1418338121673.68415@steait.net>

still same... output can be seen here:

http://i.imgur.com/BuwaGGn.png



________________________________
From: Dan McDonald <danmcd at omniti.com>
Sent: Thursday, December 11, 2014 11:39 PM
To: Rune Tipsmark
Cc: omnios-discuss at lists.omniti.com
Subject: Re: [OmniOS-discuss] hangs on reboot

Nothing printed out on the console?

And try eliminating the target mode first - just in case.

Also, is one of your NICs a dual IPMI/host NIC?  If so, disable the IPMI portion, we don't cope with shared NIC ports like that.

Dan

Sent from my iPhone (typos, autocorrect, and all)

On Dec 11, 2014, at 5:25 PM, Rune Tipsmark <rt at steait.net<mailto:rt at steait.net>> wrote:


hi all,



I got a bunch (3) installations of omnios on SuperMicro hardware and all 3 have issues rebooting. They simply hang and never ever reboot.



The install is latest version and I only added the storage-server package and installed napp-it and changed the fibre channel setting in /kernel/drv/emlxs.conf target-mode=1



two nics igb0 and igb1 configured as aggregation (aggr0)



besides this its 100% default install, not 10 minutes old...



Any ideas?

br,

Rune

_______________________________________________
OmniOS-discuss mailing list
OmniOS-discuss at lists.omniti.com<mailto:OmniOS-discuss at lists.omniti.com>
http://lists.omniti.com/mailman/listinfo/omnios-discuss
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141211/a98de1d9/attachment.html>

From rt at steait.net  Thu Dec 11 23:04:53 2014
From: rt at steait.net (Rune Tipsmark)
Date: Thu, 11 Dec 2014 23:04:53 +0000
Subject: [OmniOS-discuss] hangs on reboot
In-Reply-To: <893101d01592$59168140$0b4383c0$@acm.org>
References: <1418336750819.83182@steait.net>,
	<893101d01592$59168140$0b4383c0$@acm.org>
Message-ID: <1418339095532.45737@steait.net>

didn't even see this one in the thread, reboot -p did it for me.

I have a system where I had done the below but it still hangs, will try -p on it during the weekend.

thx,
br
Rune
________________________________________
From: Paul Henson <paul.b.henson at gmail.com> on behalf of Paul B. Henson <henson at acm.org>
Sent: Thursday, December 11, 2014 11:32 PM
To: Rune Tipsmark; omnios-discuss at lists.omniti.com
Subject: RE: [OmniOS-discuss] hangs on reboot

> Rune Tipsmark
> Sent: Thursday, December 11, 2014 2:26 PM
>
> I got a bunch (3) installations of omnios on SuperMicro hardware and all 3
> have issues rebooting. They simply hang and never ever reboot.

Disable fast reboot:

svccfg -s "system/boot-config:default" setprop
config/fastreboot_default=false
svccfg -s "system/boot-config:default" setprop
config/fastreboot_onpanic=false
svcadm refresh svc:/system/boot-config:default

Some systems (it seems particularly common on supermicro hardware) tend to
wedge up with fast reboot enabled. If you want to verify this is the issue
before changing the configuration, try 'reboot -p' which should do a one
time regular reboot without changing the default.

Dan, there was some talk of making fast reboot disabled the default, and
having people who wanted it need to enable it, rather than the other way
around? Any thoughts?



From joeveliscos at gmail.com  Thu Dec 11 23:23:25 2014
From: joeveliscos at gmail.com (Joe Veliscos)
Date: Fri, 12 Dec 2014 00:23:25 +0100
Subject: [OmniOS-discuss] Python difference
In-Reply-To: <02CECDF1-3154-40CD-B120-38C082C0D1AF@omniti.com>
References: <CAHNoB+1iJ=TNq9Z0sZ-WO-BMtx--tXcrXkVyWvcu3UfS-W+CCg@mail.gmail.com>
	<56C3D19D-B648-4F02-A7CA-BC24A31C385E@omniti.com>
	<CAHNoB+1pePnQuUNHSBKLN14975bqHPX1C6F1E7nciZjkE2qQDw@mail.gmail.com>
	<02CECDF1-3154-40CD-B120-38C082C0D1AF@omniti.com>
Message-ID: <CAHNoB+382iSS3faB8vOLOGz6cvGWzW_6s4oysCZnUbTyhgeUxw@mail.gmail.com>

Yes I did look in the /opt/ha/lib/python2.6/site-packages .It contains maps
with crm (HA) specific python modules .

At the end of the truss there's a long list of unsuccesfull open calls for
modules many of which are to a readline module e.g. (short list)

/usr/lib/python2.6/lib-dynload/cStringIO
/usr/lib/python2.6/lib-tk/cStringIO.pyc
/usr/lib/python2.6/plat-sunos5/64/cStringIO.so

/opt/ha/lib/python2.6/site-packages/readline
/opt/ha/lib/python2.6/site-packages/64/readline.so
/opt/ha/lib/python2.6/site-packages/readline.py

These are nowhere to be found on the system!

How would I go about getting such into the system? Compile python from
source or did the update go wrong? Just trying to understand the situation.

Thanks for your help.

Joe



On Thu, Dec 11, 2014 at 11:37 PM, Dan McDonald <danmcd at omniti.com> wrote:
>
> Did you look to see if there's anything in
> /opt/ha/lib/python2.6/site-packages ?
>
> The massive ENOENTs you're showing me are just library searching.  See how
> several path and filename combos are tried?
>
> HA should be more clear about how it failed.  Also, look toward the end of
> the truss, and just look for open calls.  Once you see a string of failures
> that doesn't end with a success, you'll know what's missing.
>
> Dan
>
> Sent from my iPhone (typos, autocorrect, and all)
>
> On Dec 11, 2014, at 5:23 PM, Joe Veliscos <joeveliscos at gmail.com> wrote:
>
> Hi
>
> The command that I execute :
> root#crm
> abort: couldn't find crm libraries in [/opt/ha/sbin
> /usr/lib/python2.6/vendor-packages/setuptools-0.6c11-py2.6.egg
> /opt/ha/lib/python2.6/site-packages /usr/lib/python26.zip
> /usr/lib/python2.6 /usr/lib/python2.6/plat-sunos5 /usr/lib/python2.6/lib-tk
> /usr/lib/python2.6/lib-old /usr/lib/python2.6/lib-dynload
> /usr/lib/python2.6/site-packages /usr/lib/python2.6/vendor-packages]
> (check your install and PYTHONPATH)
>
> I have the following environment variables set:
> export PYTHONPATH=/opt/ha/lib/python2.6/site-packages
> export PATH=/opt/ha/bin:/opt/ha/sbin:$PATH
> export OCF_ROOT=/opt/ha/lib/ocf
> export OCF_AGENTS=/opt/ha/lib/ocf/resource.d/heartbeat
>
> I have exactly the same in an r10 release (pre upgrade to rr12) where
> there is no problem
>
> I did a truss -d crm and it seems that many files it searches for are not
> found.  Snippets of the output (very long file) hope this helps:
>
> Below (as I understand it ) some searches which it can resolve:
>
> 0.0098    resolvepath("/usr/lib/amd64/ld.so.1", "/lib/amd64/ld.so.1",
> 1023) = 18
>  0.0100    resolvepath("/usr/bin/amd64/python2.6",
> "/usr/bin/amd64/python2.6", 1023) = 24
>  0.0101    stat("/usr/bin/amd64/python2.6", 0xFFFFFD7FFFDFF910) = 0
>  0.0103    open("/var/ld/64/ld.config", O_RDONLY)        Err#2 ENOENT
>  0.0105    stat("/usr/gnu/lib/amd64/libpython2.6.so.1.0",
> 0xFFFFFD7FFFDFF000) Err#2 ENOENT
>  0.0106    stat("/usr/lib/amd64/libpython2.6.so.1.0", 0xFFFFFD7FFFDFF000)
> = 0
>  0.0108    resolvepath("/usr/lib/amd64/libpython2.6.so.1.0",
> "/usr/lib/amd64/libpython2.6.so.1.0", 1023) = 34
>  0.0110    open("/usr/lib/amd64/libpython2.6.so.1.0", O_RDONLY) = 3
>  0.0112    mmapobj(3, MMOBJ_INTERPRET, 0xFFFFFD7FFF350AB8,
> 0xFFFFFD7FFFDFEB5C, 0x00000000) = 0
>  0.0113    close(3)                    = 0
>  0.0115    mmap(0x00000000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC,
> MAP_PRIVATE|MAP_ANON, 4294967295, 0) = 0xFFFFFD7FFF340000
>  0.0117    memcntl(0xFFFFFD7FFEAB0000, 457808, MC_ADVISE, MADV_WILLNEED,
> 0, 0) = 0
>  0.0118    stat("/usr/gnu/lib/amd64/libsocket.so.1", 0xFFFFFD7FFFDFF000)
> Err#2 ENOENT
>  0.0120    stat("/usr/lib/amd64/libsocket.so.1", 0xFFFFFD7FFFDFF000) = 0
>  0.0121    resolvepath("/usr/lib/amd64/libsocket.so.1",
> "/lib/amd64/libsocket.so.1", 1023) = 25
>  0.0123    open("/usr/lib/amd64/libsocket.so.1", O_RDONLY)    = 3
>  0.0125    mmapobj(3, MMOBJ_INTERPRET, 0xFFFFFD7FFF340A18,
> 0xFFFFFD7FFFDFEB5C, 0x00000000) = 0
>  0.0127    close(3)                    = 0
>  0.0128    mmap(0x00000000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC,
> MAP_PRIVATE|MAP_ANON, 4294967295, 0) = 0xFFFFFD7FFF330000
>  0.0129    memcntl(0xFFFFFD7FFEA80000, 32240, MC_ADVISE, MADV_WILLNEED, 0,
> 0) = 0
>  0.0130    stat("/usr/gnu/lib/amd64/libnsl.so.1", 0xFFFFFD7FFFDFF000)
> Err#2 ENOENT
>  0.0132    stat("/usr/lib/amd64/libnsl.so.1", 0xFFFFFD7FFFDFF000) = 0
>  0.0134    resolvepath("/usr/lib/amd64/libnsl.so.1",
> "/lib/amd64/libnsl.so.1", 1023) = 22
>  0.0135    open("/usr/lib/amd64/libnsl.so.1", O_RDONLY)    = 3
>  0.0137    mmapobj(3, MMOBJ_INTERPRET, 0xFFFFFD7FFF3309C8,
> 0xFFFFFD7FFFDFEB5C, 0x00000000) = 0
>  0.0139    close(3)                    = 0
>  0.0140    mmap(0x00000000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC,
> MAP_PRIVATE|MAP_ANON, 4294967295, 0) = 0xFFFFFD7FFF320000
>  0.0141    memcntl(0xFFFFFD7FFEDD0000, 180072, MC_ADVISE, MADV_WILLNEED,
> 0, 0) = 0
>  0.0142    stat("/usr/gnu/lib/amd64/libm.so.2", 0xFFFFFD7FFFDFF000) Err#2
> ENOENT
>  0.0144    stat("/usr/lib/amd64/libm.so.2", 0xFFFFFD7FFFDFF000) = 0
>  0.0145    resolvepath("/usr/lib/amd64/libm.so.2", "/lib/amd64/libm.so.2",
> 1023) = 20
>  0.0147    open("/usr/lib/amd64/libm.so.2", O_RDONLY)    = 3
>  0.0149    mmapobj(3, MMOBJ_INTERPRET, 0xFFFFFD7FFF3209F8,
> 0xFFFFFD7FFFDFEB5C, 0x00000000) = 0
>  0.0150    close(3)                    = 0
>  0.0151    mmap(0x00000000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC,
> MAP_PRIVATE|MAP_ANON, 4294967295, 0) = 0xFFFFFD7FFF310000
>  0.0153    memcntl(0xFFFFFD7FFEEE0000, 58680, MC_ADVISE, MADV_WILLNEED, 0,
> 0) = 0
>  0.0154    stat("/usr/gnu/lib/amd64/libc.so.1", 0xFFFFFD7FFFDFF000) Err#2
> ENOENT
>  0.0156    stat("/usr/lib/amd64/libc.so.1", 0xFFFFFD7FFFDFF000) = 0
>  0.0157    resolvepath("/usr/lib/amd64/libc.so.1", "/lib/amd64/libc.so.1",
> 1023) = 20
>  0.0159    open("/usr/lib/amd64/libc.so.1", O_RDONLY)    = 3
>  0.0161    mmapobj(3, MMOBJ_INTERPRET, 0xFFFFFD7FFF310920,
> 0xFFFFFD7FFFDFEB5C, 0x00000000) = 0
>  0.0162    close(3)                    = 0
>  0.0163    mmap(0x00000000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC,
> MAP_PRIVATE|MAP_ANON, 4294967295, 0) = 0xFFFFFD7FFF160000
>  0.0165    memcntl(0xFFFFFD7FFF170000, 477048, MC_ADVISE, MADV_WILLNEED,
> 0, 0) = 0
>  0.0166    stat("/lib/64/libsocket.so.1", 0xFFFFFD7FFFDFF000) = 0
>  0.0168    resolvepath("/lib/64/libsocket.so.1",
> "/lib/amd64/libsocket.so.1", 1023) = 25
>  0.0170    stat("/lib/64/libnsl.so.1", 0xFFFFFD7FFFDFF000)    = 0
>  0.0171    resolvepath("/lib/64/libnsl.so.1", "/lib/amd64/libnsl.so.1",
> 1023) = 22
>  0.0173    stat("/lib/64/libm.so.2", 0xFFFFFD7FFFDFF000)    = 0
>  0.0174    resolvepath("/lib/64/libm.so.2", "/lib/amd64/libm.so.2", 1023)
> = 20
>  0.0177    stat("/usr/gnu/lib/amd64/libgcc_s.so.1", 0xFFFFFD7FFFDFF000)
> Err#2 ENOENT
>  0.0179    stat("/lib/64/libgcc_s.so.1", 0xFFFFFD7FFFDFF000) Err#2 ENOENT
>  0.0180    stat("/usr/lib/64/libgcc_s.so.1", 0xFFFFFD7FFFDFF000) = 0
>  0.0185    resolvepath("/usr/lib/64/libgcc_s.so.1",
> "/usr/lib/amd64/libgcc_s.so.1", 1023) = 28
>  0.0187    open("/usr/lib/64/libgcc_s.so.1", O_RDONLY)    = 3
>
>
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> Below searches which it cannot resolve:
>
> 0.0329    fstat(2, 0xFFFFFD7FFFDFF920)            = 0
>  0.0331    readlink("/usr/bin/python", "python2.6", 1024)    = 9
>  0.0333    readlink("/usr/bin/python2.6", 0xFFFFFD7FFFDFF5B0, 1024) Err#22
> EINVAL
>  0.0335    stat("/usr/bin/Modules/Setup", 0xFFFFFD7FFFDFF5B0) Err#2 ENOENT
>  0.0336    stat("/usr/bin/lib/python2.6/os.py", 0xFFFFFD7FFFDFF5B0) Err#2
> ENOENT
>  0.0338    stat("/usr/bin/lib/python2.6/os.pyc", 0xFFFFFD7FFFDFF5B0) Err#2
> ENOENT
>  0.0342    stat("/usr/lib/python2.6/os.py", 0xFFFFFD7FFFDFF5B0) = 0
>  0.0344    stat("/usr/bin/Modules/Setup", 0xFFFFFD7FFFDFF120) Err#2 ENOENT
>  0.0345    stat("/usr/bin/lib/python2.6/lib-dynload", 0xFFFFFD7FFFDFF120)
> Err#2 ENOENT
>  0.0347    stat("/usr/lib/python2.6/lib-dynload", 0xFFFFFD7FFFDFF120) = 0
>  0.0351    brk(0x00485250)                    = 0
>
>  0.0456    sysconfig(_CONFIG_SIGRT_MAX)            = 73
>  0.0459    stat("/opt/ha/lib/python2.6/site-packages", 0xFFFFFD7FFFDFDF90)
> = 0
>  0.0461    stat("/opt/ha/lib/python2.6/site-packages", 0xFFFFFD7FFFDFE340)
> = 0
>  0.0462    stat("/opt/ha/lib/python2.6/site-packages/site",
> 0xFFFFFD7FFFDFE640) Err#2 ENOENT
>  0.0464    stat("/opt/ha/lib/python2.6/site-packages/site",
> 0xFFFFFD7FFFDFEAD0) Err#2 ENOENT
>  0.0465    open("/opt/ha/lib/python2.6/site-packages/64/site.so",
> O_RDONLY) Err#2 ENOENT
>  0.0467    stat("/opt/ha/lib/python2.6/site-packages/site",
> 0xFFFFFD7FFFDFEAD0) Err#2 ENOENT
>  0.0469    open("/opt/ha/lib/python2.6/site-packages/64/sitemodule.so",
> O_RDONLY) Err#2 ENOENT
>  0.0470    stat("/opt/ha/lib/python2.6/site-packages/site",
> 0xFFFFFD7FFFDFEAD0) Err#2 ENOENT
>  0.0471    open("/opt/ha/lib/python2.6/site-packages/site.py", O_RDONLY)
> Err#2 ENOENT
>  0.0473    stat("/opt/ha/lib/python2.6/site-packages/site",
> 0xFFFFFD7FFFDFEAD0) Err#2 ENOENT
>  0.0474    open("/opt/ha/lib/python2.6/site-packages/site.pyc", O_RDONLY)
> Err#2 ENOENT
>  0.0476    stat("/usr/lib/python26.zip", 0xFFFFFD7FFFDFDF90) Err#2 ENOENT
>  0.0477    stat("/usr/lib", 0xFFFFFD7FFFDFDF90)        = 0
>  0.0478    stat("/usr/lib/python26.zip", 0xFFFFFD7FFFDFE340) Err#2 ENOENT
>  0.0480    stat("/usr/lib/python2.6/", 0xFFFFFD7FFFDFDF90)    = 0
>  0.0481    stat("/usr/lib/python2.6/", 0xFFFFFD7FFFDFE340)    = 0
>  0.0482    stat("/usr/lib/python2.6/site", 0xFFFFFD7FFFDFE640) Err#2 ENOENT
>  0.0483    stat("/usr/lib/python2.6/site", 0xFFFFFD7FFFDFEAD0) Err#2 ENOENT
>  0.0485    open("/usr/lib/python2.6/64/site.so", O_RDONLY)    Err#2 ENOENT
>  0.0486    stat("/usr/lib/python2.6/site", 0xFFFFFD7FFFDFEAD0) Err#2 ENOENT
>  0.0487    open("/usr/lib/python2.6/64/sitemodule.so", O_RDONLY) Err#2
> ENOENT
>  0.0488    stat("/usr/lib/python2.6/site", 0xFFFFFD7FFFDFEAD0) Err#2 ENOENT
>  0.0490    open("/usr/lib/python2.6/site.py", O_RDONLY)    = 3
>  0.0491    fstat(3, 0xFFFFFD7FFFDFEA90)            = 0
>  0.0492    open("/usr/lib/python2.6/site.pyc", O_RDONLY)    = 4
>  0.0493    fstat(4, 0xFFFFFD7FFFDFE8D0)            = 0
>  0.0494    brk(0x004D5250)                    = 0
>  0.0496    brk(0x004D9250)                    = 0
>  0.0498    fstat(4, 0xFFFFFD7FFFDFE800)            = 0
>  0.0498    ioctl(4, TCGETA, 0xFFFFFD7FFFDFE880)        Err#25 ENOTTY
>  0.0500    read(4, "D1F2\r\nA09390 S c\0\0\0".., 18944)    = 18651
>
> 0.0514    stat("/opt/ha/lib/python2.6/site-packages/os",
> 0xFFFFFD7FFFDFD320) Err#2 ENOENT
>  0.0516    stat("/opt/ha/lib/python2.6/site-packages/os",
> 0xFFFFFD7FFFDFD7B0) Err#2 ENOENT
>  0.0517    open("/opt/ha/lib/python2.6/site-packages/64/os.so", O_RDONLY)
> Err#2 ENOENT
>  0.0519    stat("/opt/ha/lib/python2.6/site-packages/os",
> 0xFFFFFD7FFFDFD7B0) Err#2 ENOENT
>  0.0520    open("/opt/ha/lib/python2.6/site-packages/64/osmodule.so",
> O_RDONLY) Err#2 ENOENT
>  0.0521    stat("/opt/ha/lib/python2.6/site-packages/os",
> 0xFFFFFD7FFFDFD7B0) Err#2 ENOENT
>  0.0522    open("/opt/ha/lib/python2.6/site-packages/os.py", O_RDONLY)
> Err#2 ENOENT
>  0.0524    stat("/opt/ha/lib/python2.6/site-packages/os",
> 0xFFFFFD7FFFDFD7B0) Err#2 ENOENT
>  0.0525    open("/opt/ha/lib/python2.6/site-packages/os.pyc", O_RDONLY)
> Err#2 ENOENT
>  0.0527    stat("/usr/lib/python2.6/os", 0xFFFFFD7FFFDFD320) Err#2 ENOENT
>  0.0528    stat("/usr/lib/python2.6/os", 0xFFFFFD7FFFDFD7B0) Err#2 ENOENT
>  0.0529    open("/usr/lib/python2.6/64/os.so", O_RDONLY)    Err#2 ENOENT
>  0.0530    stat("/usr/lib/python2.6/os", 0xFFFFFD7FFFDFD7B0) Err#2 ENOENT
>  0.0532    open("/usr/lib/python2.6/64/osmodule.so", O_RDONLY) Err#2 ENOENT
>  0.0533    stat("/usr/lib/python2.6/os", 0xFFFFFD7FFFDFD7B0) Err#2 ENOENT
>  0.0534    open("/usr/lib/python2.6/os.py", O_RDONLY)    = 4
>  0.0536    fstat(4, 0xFFFFFD7FFFDFD770)            = 0
>  0.0537    open("/usr/lib/python2.6/os.pyc", O_RDONLY)    = 5
>  0.0538    fstat(5, 0xFFFFFD7FFFDFD5B0)            = 0
>  0.0539    fstat(5, 0xFFFFFD7FFFDFD4E0)            = 0
>  0.0540    ioctl(5, TCGETA, 0xFFFFFD7FFFDFD560)        Err#25 ENOTTY
>  0.0541    read(5, "D1F2\r\n9F9390 S c\0\0\0".., 26112)    = 25702
>
>  0.0568    stat("/opt/ha/lib/python2.6/site-packages/posixpath",
> 0xFFFFFD7FFFDFC000) Err#2 ENOENT
>  0.0570    stat("/opt/ha/lib/python2.6/site-packages/posixpath",
> 0xFFFFFD7FFFDFC490) Err#2 ENOENT
>  0.0572    open("/opt/ha/lib/python2.6/site-packages/64/posixpath.so",
> O_RDONLY) Err#2 ENOENT
>  0.0573    stat("/opt/ha/lib/python2.6/site-packages/posixpath",
> 0xFFFFFD7FFFDFC490) Err#2 ENOENT
>  0.0575
> open("/opt/ha/lib/python2.6/site-packages/64/posixpathmodule.so", O_RDONLY)
> Err#2 ENOENT
>  0.0576    stat("/opt/ha/lib/python2.6/site-packages/posixpath",
> 0xFFFFFD7FFFDFC490) Err#2 ENOENT
>  0.0577    open("/opt/ha/lib/python2.6/site-packages/posixpath.py",
> O_RDONLY) Err#2 ENOENT
>  0.0579    stat("/opt/ha/lib/python2.6/site-packages/posixpath",
> 0xFFFFFD7FFFDFC490) Err#2 ENOENT
>  0.0580    open("/opt/ha/lib/python2.6/site-packages/posixpath.pyc",
> O_RDONLY) Err#2 ENOENT
>  0.0582    stat("/usr/lib/python2.6/posixpath", 0xFFFFFD7FFFDFC000) Err#2
> ENOENT
>  0.0583    stat("/usr/lib/python2.6/posixpath", 0xFFFFFD7FFFDFC490) Err#2
> ENOENT
>  0.0584    open("/usr/lib/python2.6/64/posixpath.so", O_RDONLY) Err#2
> ENOENT
>  0.0585    stat("/usr/lib/python2.6/posixpath", 0xFFFFFD7FFFDFC490) Err#2
> ENOENT
>  0.0587    open("/usr/lib/python2.6/64/posixpathmodule.so", O_RDONLY)
> Err#2 ENOENT
>  0.0588    stat("/usr/lib/python2.6/posixpath", 0xFFFFFD7FFFDFC490) Err#2
> ENOENT
>  0.0589    open("/usr/lib/python2.6/posixpath.py", O_RDONLY) = 5
>  0.0591    fstat(5, 0xFFFFFD7FFFDFC450)            = 0
>  0.0592    open("/usr/lib/python2.6/posixpath.pyc", O_RDONLY) = 6
>  0.0593    fstat(6, 0xFFFFFD7FFFDFC290)            = 0
>
>
> And the list goes on.
>
> Hope there's a solution for this.
>
>
> Joe
>
>
> On Thu, Dec 11, 2014 at 10:26 PM, Dan McDonald <danmcd at omniti.com> wrote:
>>
>>
>> > On Dec 11, 2014, at 4:00 PM, Joe Veliscos <joeveliscos at gmail.com>
>> wrote:
>> >
>> > Hi,
>> >
>> > I have an application working on Omnios r10 which depends on certain
>> python libraries. There are certain environment variables in place which
>> point to the location of those libraries.
>> >
>> > I have updated the r10 machine to r12. The application now cannot be
>> started with errors stating that the needed libraries cannot be found in
>> the given paths.
>>
>> Share the errors please?  I'll need more details.
>>
>> > Maybe somebody can tell me what the difference is in python version
>> between the two omnios releases. Python -V gives me 2.6.8. on both versions.
>>
>> We updated supplemental python libraries, which may contribute to what
>> you're seeing.  Also, we had not updated the "entire" metapackage on 010 to
>> show we were actually running 2.6.8.
>>
>> Dan
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141212/fa5b1710/attachment-0001.html>

From alexmcwhirter at mojovapes.net  Thu Dec 11 23:35:07 2014
From: alexmcwhirter at mojovapes.net (Alex McWhirter)
Date: Thu, 11 Dec 2014 18:35:07 -0500
Subject: [OmniOS-discuss] Network Killed before iSCSI is unmounted,
	hangs on shutdown.
Message-ID: <9F6A55C8-45D9-4DEF-9480-4D2914081FB9@mojovapes.net>

I have a client that is using iSCSI mounts as zone storage. Upon system shutdown the host kills the network before unmounting the iSCSI zpools. This causes the system to hang as it attempts to unmount the zpool after the disks have gone offline. Whats the best way to prevent something like this?

From danmcd at omniti.com  Thu Dec 11 23:42:52 2014
From: danmcd at omniti.com (Dan McDonald)
Date: Thu, 11 Dec 2014 18:42:52 -0500
Subject: [OmniOS-discuss] Python difference
In-Reply-To: <CAHNoB+382iSS3faB8vOLOGz6cvGWzW_6s4oysCZnUbTyhgeUxw@mail.gmail.com>
References: <CAHNoB+1iJ=TNq9Z0sZ-WO-BMtx--tXcrXkVyWvcu3UfS-W+CCg@mail.gmail.com>
	<56C3D19D-B648-4F02-A7CA-BC24A31C385E@omniti.com>
	<CAHNoB+1pePnQuUNHSBKLN14975bqHPX1C6F1E7nciZjkE2qQDw@mail.gmail.com>
	<02CECDF1-3154-40CD-B120-38C082C0D1AF@omniti.com>
	<CAHNoB+382iSS3faB8vOLOGz6cvGWzW_6s4oysCZnUbTyhgeUxw@mail.gmail.com>
Message-ID: <B57B98E5-3EFE-4062-B92C-88295A4116C3@omniti.com>


> On Dec 11, 2014, at 6:23 PM, Joe Veliscos <joeveliscos at gmail.com> wrote:
> 
> Yes I did look in the /opt/ha/lib/python2.6/site-packages .It contains maps with crm (HA) specific python modules .
> 
> At the end of the truss there's a long list of unsuccesfull open calls for modules many of which are to a readline module e.g. (short list)
> 
> /usr/lib/python2.6/lib-dynload/cStringIO
> /usr/lib/python2.6/lib-tk/cStringIO.pyc
> /usr/lib/python2.6/plat-sunos5/64/cStringIO.so
> 
> /opt/ha/lib/python2.6/site-packages/readline
> /opt/ha/lib/python2.6/site-packages/64/readline.so
> /opt/ha/lib/python2.6/site-packages/readline.py
> 
> These are nowhere to be found on the system!
> 
> How would I go about getting such into the system? Compile python from source or did the update go wrong? Just trying to understand the situation.

But they are on your r151010 BE?  If you have the old BE, use:

	beadm mount old-be /mnt

and go poking around in there to see where they are.  Also, how do you install the HA packages?  Perhaps you need to reinstall them?

BTW, I don't know much about HA, but maybe someone else on this list does, especially with how it interacts with r151012.

Dan



From joeveliscos at gmail.com  Fri Dec 12 00:31:41 2014
From: joeveliscos at gmail.com (Joe Veliscos)
Date: Fri, 12 Dec 2014 01:31:41 +0100
Subject: [OmniOS-discuss] Python difference
In-Reply-To: <B57B98E5-3EFE-4062-B92C-88295A4116C3@omniti.com>
References: <CAHNoB+1iJ=TNq9Z0sZ-WO-BMtx--tXcrXkVyWvcu3UfS-W+CCg@mail.gmail.com>
	<56C3D19D-B648-4F02-A7CA-BC24A31C385E@omniti.com>
	<CAHNoB+1pePnQuUNHSBKLN14975bqHPX1C6F1E7nciZjkE2qQDw@mail.gmail.com>
	<02CECDF1-3154-40CD-B120-38C082C0D1AF@omniti.com>
	<CAHNoB+382iSS3faB8vOLOGz6cvGWzW_6s4oysCZnUbTyhgeUxw@mail.gmail.com>
	<B57B98E5-3EFE-4062-B92C-88295A4116C3@omniti.com>
Message-ID: <CAHNoB+3P0jyNsN5AWD3Vaij=npLGEo2WnpkB+EDBK4ko-d7obQ@mail.gmail.com>

Just checked,
Most of them are in the /usr/lib/python2.6/ and sub directories.
The HA package was created by sazo, usually a frequent member of this list.

I seems there's a diff in the installed python packages between releases so
I'll have to play around to get this working I think.

Thanks for all your help and pointers again,

High regards,

Joe

On Fri, Dec 12, 2014 at 12:42 AM, Dan McDonald <danmcd at omniti.com> wrote:
>
>
> > On Dec 11, 2014, at 6:23 PM, Joe Veliscos <joeveliscos at gmail.com> wrote:
> >
> > Yes I did look in the /opt/ha/lib/python2.6/site-packages .It contains
> maps with crm (HA) specific python modules .
> >
> > At the end of the truss there's a long list of unsuccesfull open calls
> for modules many of which are to a readline module e.g. (short list)
> >
> > /usr/lib/python2.6/lib-dynload/cStringIO
> > /usr/lib/python2.6/lib-tk/cStringIO.pyc
> > /usr/lib/python2.6/plat-sunos5/64/cStringIO.so
> >
> > /opt/ha/lib/python2.6/site-packages/readline
> > /opt/ha/lib/python2.6/site-packages/64/readline.so
> > /opt/ha/lib/python2.6/site-packages/readline.py
> >
> > These are nowhere to be found on the system!
> >
> > How would I go about getting such into the system? Compile python from
> source or did the update go wrong? Just trying to understand the situation.
>
> But they are on your r151010 BE?  If you have the old BE, use:
>
>         beadm mount old-be /mnt
>
> and go poking around in there to see where they are.  Also, how do you
> install the HA packages?  Perhaps you need to reinstall them?
>
> BTW, I don't know much about HA, but maybe someone else on this list does,
> especially with how it interacts with r151012.
>
> Dan
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141212/d10905d1/attachment.html>

From alexmcwhirter at mojovapes.net  Fri Dec 12 06:12:48 2014
From: alexmcwhirter at mojovapes.net (Alex McWhirter)
Date: Fri, 12 Dec 2014 01:12:48 -0500
Subject: [OmniOS-discuss] Network Killed before iSCSI is unmounted,
	hangs on shutdown.
In-Reply-To: <9F6A55C8-45D9-4DEF-9480-4D2914081FB9@mojovapes.net>
References: <9F6A55C8-45D9-4DEF-9480-4D2914081FB9@mojovapes.net>
Message-ID: <A481FB7D-9DA6-42C9-827E-629575D8EE09@mojovapes.net>

This is an exact description of the error i?m having. Changing the timeout did not solve this problem. Nor did adding /network/iscsi/initiator as a dependecy of /system/filesystem/local

http://everycity.co.uk/alasdair/2010/03/solaris-iscsi-initiator-reboots/ <http://everycity.co.uk/alasdair/2010/03/solaris-iscsi-initiator-reboots/>

Hopefully someone can chime in on this, i really need zones to work over shared storage.



> On Dec 11, 2014, at 6:35 PM, Alex McWhirter <alexmcwhirter at mojovapes.net> wrote:
> 
> I have a client that is using iSCSI mounts as zone storage. Upon system shutdown the host kills the network before unmounting the iSCSI zpools. This causes the system to hang as it attempts to unmount the zpool after the disks have gone offline. Whats the best way to prevent something like this?
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141212/384bda97/attachment.html>

From jimklimov at cos.ru  Fri Dec 12 06:36:56 2014
From: jimklimov at cos.ru (Jim Klimov)
Date: Fri, 12 Dec 2014 07:36:56 +0100
Subject: [OmniOS-discuss] Network Killed before iSCSI is unmounted,
	hangs on shutdown.
In-Reply-To: <9F6A55C8-45D9-4DEF-9480-4D2914081FB9@mojovapes.net>
References: <9F6A55C8-45D9-4DEF-9480-4D2914081FB9@mojovapes.net>
Message-ID: <E2F2BA30-54AA-46DE-BF0A-07EE092A0E14@cos.ru>

12 ??????? 2014??. 0:35:07 CET, Alex McWhirter <alexmcwhirter at mojovapes.net> ?????:
>I have a client that is using iSCSI mounts as zone storage. Upon system
>shutdown the host kills the network before unmounting the iSCSI zpools.
>This causes the system to hang as it attempts to unmount the zpool
>after the disks have gone offline. Whats the best way to prevent
>something like this?
>_______________________________________________
>OmniOS-discuss mailing list
>OmniOS-discuss at lists.omniti.com
>http://lists.omniti.com/mailman/listinfo/omnios-discuss

Hi, this seems exactly like a job for my half-baked procedures (manual work needed during setup) to wrap each pool, zone and iscsi service into SMF instances so you can set up dependencies for ordered startup-shutdown.
The first two I use on a number of machines (Solaris 10, OpenSolaris Nevada, OpenIndiana and now starting with OmniOS). Don't have much iscsi beside initial experiments though...

See here:

http://wiki.openindiana.org/display/oi/Zones+as+SMF+services

http://wiki.openindiana.org/oi/Advanced+-+ZFS+Pools+as+SMF+services+and+iSCSI+loopback+mounts

Hope this helps,
Jim Klimov

--
Typos courtesy of K-9 Mail on my Samsung Android

From jimklimov at cos.ru  Fri Dec 12 06:42:38 2014
From: jimklimov at cos.ru (Jim Klimov)
Date: Fri, 12 Dec 2014 07:42:38 +0100
Subject: [OmniOS-discuss] Network Killed before iSCSI is unmounted,
	hangs on shutdown.
In-Reply-To: <A481FB7D-9DA6-42C9-827E-629575D8EE09@mojovapes.net>
References: <9F6A55C8-45D9-4DEF-9480-4D2914081FB9@mojovapes.net>
	<A481FB7D-9DA6-42C9-827E-629575D8EE09@mojovapes.net>
Message-ID: <15349DD3-62FD-4FE7-AABE-3E76AE636DE7@cos.ru>

12 ??????? 2014??. 7:12:48 CET, Alex McWhirter <alexmcwhirter at mojovapes.net> ?????:
>This is an exact description of the error i?m having. Changing the
>timeout did not solve this problem. Nor did adding
>/network/iscsi/initiator as a dependecy of /system/filesystem/local
>
>http://everycity.co.uk/alasdair/2010/03/solaris-iscsi-initiator-reboots/
><http://everycity.co.uk/alasdair/2010/03/solaris-iscsi-initiator-reboots/>
>
>Hopefully someone can chime in on this, i really need zones to work
>over shared storage.
>
>
>
>> On Dec 11, 2014, at 6:35 PM, Alex McWhirter
><alexmcwhirter at mojovapes.net> wrote:
>> 
>> I have a client that is using iSCSI mounts as zone storage. Upon
>system shutdown the host kills the network before unmounting the iSCSI
>zpools. This causes the system to hang as it attempts to unmount the
>zpool after the disks have gone offline. Whats the best way to prevent
>something like this?
>> _______________________________________________
>> OmniOS-discuss mailing list
>> OmniOS-discuss at lists.omniti.com
>> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>
>
>
>------------------------------------------------------------------------
>
>_______________________________________________
>OmniOS-discuss mailing list
>OmniOS-discuss at lists.omniti.com
>http://lists.omniti.com/mailman/listinfo/omnios-discuss

Just in case: you did do "svcadm refresh servicename" after svccfg to actually apply the new attributes, right?
--
Typos courtesy of K-9 Mail on my Samsung Android

From jimklimov at cos.ru  Fri Dec 12 07:52:23 2014
From: jimklimov at cos.ru (Jim Klimov)
Date: Fri, 12 Dec 2014 08:52:23 +0100
Subject: [OmniOS-discuss] Java paths in OmniOS
Message-ID: <AFA0DC98-08AB-43A5-9019-023EB9140883@cos.ru>

Hi all,

While trying to install Java in OmniOS, I found it delivers one. However, it has 'simplified' the path structure by removing essentially the tree for different jvm versions under /usr/jdk, instead making it equivalent to /usr/java...

In Sun packages, there were /usr/jdk/instances/$manyjavaversions/, a /usr/jdk/latest symlink pointing to ./instances/$somejavaversion and a /usr/java symlink to ./jdk/latest (typically JAVA_HOME=/usr/java) and some 20 binaries like /usr/bin/javac being symlinks to ../java/bin/javac. 

Symlinks like the 'latest' could be easily changed by an admin to choose a particular JVM as default. While the $manyjavaversions could be managed by packaging, this also allowed to pluck in zip-installs and/or customized jvm's (custom timezone updates, ca's, jce settings etc.) to jse certain jvm versions for some obscure software which did not like updates for example.

The new structure all but forbids such customization; was there and particular benefit from changing to it?

Does anything in OmniOS or omniti-ms packages depend on the new paths (can I locally change back to old structure without something strangely misbehaving)?

Thanks,
Jim
--
Typos courtesy of K-9 Mail on my Samsung Android

From jimklimov at cos.ru  Fri Dec 12 07:58:33 2014
From: jimklimov at cos.ru (Jim Klimov)
Date: Fri, 12 Dec 2014 08:58:33 +0100
Subject: [OmniOS-discuss] hangs on reboot
In-Reply-To: <B5E5DBE6-67C1-4E63-BDA9-B52C40E4B545@cos.ru>
References: <1418336750819.83182@steait.net>
	<EFBA16E3-3165-4164-B700-53A80F6D48CE@omniti.com>
	<1418337697672.85350@steait.net>
	<B5E5DBE6-67C1-4E63-BDA9-B52C40E4B545@cos.ru>
Message-ID: <B515960D-28BB-462A-AADF-464D10112679@cos.ru>

12 ??????? 2014??. 8:35:18 CET, Jim Klimov <jimklimov at cos.ru> ?????:
>11 ??????? 2014??. 23:41:35 CET, Rune Tipsmark <rt at steait.net> ?????:
>>NIC's are not shared with IPMI.
>>
>>
>>
>>I will try the target-mode first, however on the 3rd box I actually
>>have Infiniband and it also caused the same issue without touching the
>>target-mode for FC.
>>
>>
>>
>>I have a 4th system on an old HP server and it reboots perfect every
>>time, no target-mode and has Infiniband as well... I am leaning
>towards
>>something with the SuperMicro hardware but can't really pinpoint it.
>>
>>br,
>>
>>Rune
>>
>>________________________________
>>From: Dan McDonald <danmcd at omniti.com>
>>Sent: Thursday, December 11, 2014 11:39 PM
>>To: Rune Tipsmark
>>Cc: omnios-discuss at lists.omniti.com
>>Subject: Re: [OmniOS-discuss] hangs on reboot
>>
>>Nothing printed out on the console?
>>
>>And try eliminating the target mode first - just in case.
>>
>>Also, is one of your NICs a dual IPMI/host NIC?  If so, disable the
>>IPMI portion, we don't cope with shared NIC ports like that.
>>
>>Dan
>>
>>Sent from my iPhone (typos, autocorrect, and all)
>>
>>On Dec 11, 2014, at 5:25 PM, Rune Tipsmark
>><rt at steait.net<mailto:rt at steait.net>> wrote:
>>
>>
>>hi all,
>>
>>
>>
>>I got a bunch (3) installations of omnios on SuperMicro hardware and
>>all 3 have issues rebooting. They simply hang and never ever reboot.
>>
>>
>>
>>The install is latest version and I only added the storage-server
>>package and installed napp-it and changed the fibre channel setting in
>>/kernel/drv/emlxs.conf target-mode=1
>>
>>
>>
>>two nics igb0 and igb1 configured as aggregation (aggr0)
>>
>>
>>
>>besides this its 100% default install, not 10 minutes old...
>>
>>
>>
>>Any ideas?
>>
>>br,
>>
>>Rune
>>
>>_______________________________________________
>>OmniOS-discuss mailing list
>>OmniOS-discuss at lists.omniti.com<mailto:OmniOS-discuss at lists.omniti.com>
>>http://lists.omniti.com/mailman/listinfo/omnios-discuss
>>
>>
>>------------------------------------------------------------------------
>>
>>_______________________________________________
>>OmniOS-discuss mailing list
>>OmniOS-discuss at lists.omniti.com
>>http://lists.omniti.com/mailman/listinfo/omnios-discuss
>
>I've installed latest bloody on an HP Z400 workstation last week with
>an lsi 9212 addon hba, and it too needed to disable fastboot.
>
>IIRC this is somehow tied to ability of all drivers to quiesce like for
>standby - if some one refuses to, suspend or fastboot fail.
>IMHO the fastboot handler should have a reasonable/configurable timeout
>after 'stand by for reboot' to do slowboot if that expires.
>
>Alternately on servers you can set up bmc-watchdog and have the mobo
>reboot itself. But then use long timeouts (10-15 min) and/or take care
>that the watchdog driver is among the first pieces of software that
>runs in your bootup or especially 'live' repair-boots.
>
>HTH,
>Jim 
>--
>Typos courtesy of K-9 Mail on my Samsung Android

Forgot to add my 2c: if the driver quiesce is at fault, maybe it is also a signal for devs to improve at least nominal hw support for new usb3, video chips, maybe some sata HBAs? 
Even if they don't perform optimally (i.e. let usb3 be as slow as usb2 or usb1, or let radeon be a vesa-video, but let it work without errors) -- they should not impede system work either.

Jim
--
Typos courtesy of K-9 Mail on my Samsung Android

From jimklimov at cos.ru  Fri Dec 12 07:35:18 2014
From: jimklimov at cos.ru (Jim Klimov)
Date: Fri, 12 Dec 2014 08:35:18 +0100
Subject: [OmniOS-discuss] hangs on reboot
In-Reply-To: <1418337697672.85350@steait.net>
References: <1418336750819.83182@steait.net>
	<EFBA16E3-3165-4164-B700-53A80F6D48CE@omniti.com>
	<1418337697672.85350@steait.net>
Message-ID: <B5E5DBE6-67C1-4E63-BDA9-B52C40E4B545@cos.ru>

11 ??????? 2014??. 23:41:35 CET, Rune Tipsmark <rt at steait.net> ?????:
>NIC's are not shared with IPMI.
>
>
>
>I will try the target-mode first, however on the 3rd box I actually
>have Infiniband and it also caused the same issue without touching the
>target-mode for FC.
>
>
>
>I have a 4th system on an old HP server and it reboots perfect every
>time, no target-mode and has Infiniband as well... I am leaning towards
>something with the SuperMicro hardware but can't really pinpoint it.
>
>br,
>
>Rune
>
>________________________________
>From: Dan McDonald <danmcd at omniti.com>
>Sent: Thursday, December 11, 2014 11:39 PM
>To: Rune Tipsmark
>Cc: omnios-discuss at lists.omniti.com
>Subject: Re: [OmniOS-discuss] hangs on reboot
>
>Nothing printed out on the console?
>
>And try eliminating the target mode first - just in case.
>
>Also, is one of your NICs a dual IPMI/host NIC?  If so, disable the
>IPMI portion, we don't cope with shared NIC ports like that.
>
>Dan
>
>Sent from my iPhone (typos, autocorrect, and all)
>
>On Dec 11, 2014, at 5:25 PM, Rune Tipsmark
><rt at steait.net<mailto:rt at steait.net>> wrote:
>
>
>hi all,
>
>
>
>I got a bunch (3) installations of omnios on SuperMicro hardware and
>all 3 have issues rebooting. They simply hang and never ever reboot.
>
>
>
>The install is latest version and I only added the storage-server
>package and installed napp-it and changed the fibre channel setting in
>/kernel/drv/emlxs.conf target-mode=1
>
>
>
>two nics igb0 and igb1 configured as aggregation (aggr0)
>
>
>
>besides this its 100% default install, not 10 minutes old...
>
>
>
>Any ideas?
>
>br,
>
>Rune
>
>_______________________________________________
>OmniOS-discuss mailing list
>OmniOS-discuss at lists.omniti.com<mailto:OmniOS-discuss at lists.omniti.com>
>http://lists.omniti.com/mailman/listinfo/omnios-discuss
>
>
>------------------------------------------------------------------------
>
>_______________________________________________
>OmniOS-discuss mailing list
>OmniOS-discuss at lists.omniti.com
>http://lists.omniti.com/mailman/listinfo/omnios-discuss

I've installed latest bloody on an HP Z400 workstation last week with an lsi 9212 addon hba, and it too needed to disable fastboot.

IIRC this is somehow tied to ability of all drivers to quiesce like for standby - if some one refuses to, suspend or fastboot fail.
IMHO the fastboot handler should have a reasonable/configurable timeout after 'stand by for reboot' to do slowboot if that expires.

Alternately on servers you can set up bmc-watchdog and have the mobo reboot itself. But then use long timeouts (10-15 min) and/or take care that the watchdog driver is among the first pieces of software that runs in your bootup or especially 'live' repair-boots.

HTH,
Jim 
--
Typos courtesy of K-9 Mail on my Samsung Android

From stephan.budach at JVM.DE  Fri Dec 12 11:24:33 2014
From: stephan.budach at JVM.DE (Stephan Budach)
Date: Fri, 12 Dec 2014 12:24:33 +0100
Subject: [OmniOS-discuss] hangs on reboot
In-Reply-To: <B515960D-28BB-462A-AADF-464D10112679@cos.ru>
References: <1418336750819.83182@steait.net>	<EFBA16E3-3165-4164-B700-53A80F6D48CE@omniti.com>	<1418337697672.85350@steait.net>	<B5E5DBE6-67C1-4E63-BDA9-B52C40E4B545@cos.ru>
	<B515960D-28BB-462A-AADF-464D10112679@cos.ru>
Message-ID: <548AD071.5070306@jvm.de>

Am 12.12.14 um 08:58 schrieb Jim Klimov:
> 12 ??????? 2014 ?. 8:35:18 CET, Jim Klimov <jimklimov at cos.ru> ?????:
>> 11 ??????? 2014 ?. 23:41:35 CET, Rune Tipsmark <rt at steait.net> ?????:
>>> NIC's are not shared with IPMI.
>>>
>>>
>>>
>>> I will try the target-mode first, however on the 3rd box I actually
>>> have Infiniband and it also caused the same issue without touching the
>>> target-mode for FC.
>>>
>>>
>>>
>>> I have a 4th system on an old HP server and it reboots perfect every
>>> time, no target-mode and has Infiniband as well... I am leaning
>> towards
>>> something with the SuperMicro hardware but can't really pinpoint it.
>>>
>>> br,
>>>
>>> Rune
>>>
>>> ________________________________
>>> From: Dan McDonald <danmcd at omniti.com>
>>> Sent: Thursday, December 11, 2014 11:39 PM
>>> To: Rune Tipsmark
>>> Cc: omnios-discuss at lists.omniti.com
>>> Subject: Re: [OmniOS-discuss] hangs on reboot
>>>
>>> Nothing printed out on the console?
>>>
>>> And try eliminating the target mode first - just in case.
>>>
>>> Also, is one of your NICs a dual IPMI/host NIC?  If so, disable the
>>> IPMI portion, we don't cope with shared NIC ports like that.
>>>
>>> Dan
>>>
>>> Sent from my iPhone (typos, autocorrect, and all)
>>>
>>> On Dec 11, 2014, at 5:25 PM, Rune Tipsmark
>>> <rt at steait.net<mailto:rt at steait.net>> wrote:
>>>
>>>
>>> hi all,
>>>
>>>
>>>
>>> I got a bunch (3) installations of omnios on SuperMicro hardware and
>>> all 3 have issues rebooting. They simply hang and never ever reboot.
>>>
>>>
>>>
>>> The install is latest version and I only added the storage-server
>>> package and installed napp-it and changed the fibre channel setting in
>>> /kernel/drv/emlxs.conf target-mode=1
>>>
>>>
>>>
>>> two nics igb0 and igb1 configured as aggregation (aggr0)
>>>
>>>
>>>
>>> besides this its 100% default install, not 10 minutes old...
>>>
>>>
>>>
>>> Any ideas?
>>>
>>> br,
>>>
>>> Rune
>>>
>>> _______________________________________________
>>> OmniOS-discuss mailing list
>>> OmniOS-discuss at lists.omniti.com<mailto:OmniOS-discuss at lists.omniti.com>
>>> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>>>
>>>
>>> ------------------------------------------------------------------------
>>>
>>> _______________________________________________
>>> OmniOS-discuss mailing list
>>> OmniOS-discuss at lists.omniti.com
>>> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>> I've installed latest bloody on an HP Z400 workstation last week with
>> an lsi 9212 addon hba, and it too needed to disable fastboot.
>>
>> IIRC this is somehow tied to ability of all drivers to quiesce like for
>> standby - if some one refuses to, suspend or fastboot fail.
>> IMHO the fastboot handler should have a reasonable/configurable timeout
>> after 'stand by for reboot' to do slowboot if that expires.
>>
>> Alternately on servers you can set up bmc-watchdog and have the mobo
>> reboot itself. But then use long timeouts (10-15 min) and/or take care
>> that the watchdog driver is among the first pieces of software that
>> runs in your bootup or especially 'live' repair-boots.
>>
>> HTH,
>> Jim
>> --
>> Typos courtesy of K-9 Mail on my Samsung Android
> Forgot to add my 2c: if the driver quiesce is at fault, maybe it is also a signal for devs to improve at least nominal hw support for new usb3, video chips, maybe some sata HBAs?
> Even if they don't perform optimally (i.e. let usb3 be as slow as usb2 or usb1, or let radeon be a vesa-video, but let it work without errors) -- they should not impede system work either.
>
> Jim
> --
I have recently bought a new SuperMicro which showed exactly the same 
behaviour regarding fast reboots. Rebooting through the prom however, 
worked each time I can remember trying it. That system had been 
originally installed using the release 006 of OmniOS. After upgrading to 
012 stable, these issues seem to have gone. The only thing that is 
different on the new host is, that it's equipped with a SSD DOM instead 
of a "traditional" HD,. This DOM is stuffed in SATA0 and somewhat cabled 
to the MB, so it's not just a dumb one. This cable seems to be a bit 
picky about its connection of the plug that goes into the DOM.

As for 012? I just performed 4 fast reboots in a row without issues on 
my new SuperMicro box?

Cheers,
budy

From jimklimov at cos.ru  Fri Dec 12 12:57:37 2014
From: jimklimov at cos.ru (Jim Klimov)
Date: Fri, 12 Dec 2014 13:57:37 +0100
Subject: [OmniOS-discuss] Parity error on path mpt_sas2
In-Reply-To: <3BE0DEED8863E5429BAE4CAEDF624565039C1B330ACE@AIRA-SRV.aira.local>
References: <3BE0DEED8863E5429BAE4CAEDF624565039C1B0DAAD0@AIRA-SRV.aira.local>
	<1439760B-8D8B-40F9-8A62-4CFE4A9483C1@omniti.com>
	<3BE0DEED8863E5429BAE4CAEDF624565039C1B0DAADA@AIRA-SRV.aira.local>
	<CAOqBcP_12FaH77o+3nrGVm7KU3x1XCk3B-9uenKRRL63t5wdyg@mail.gmail.com>
	<3BE0DEED8863E5429BAE4CAEDF624565039C1B0DACD1@AIRA-SRV.aira.local>
	<CALeZrrRhsV5i2AJF2aRrLxQvR9LSckiEax59gX5XXAYAkpCS+Q@mail.gmail.com>
	<004501d01302$257c9b90$7075d2b0$@subrigo.net>
	<3BE0DEED8863E5429BAE4CAEDF624565039C1B0D8E3F@AIRA-SRV.aira.local>
	<CALeZrrTqymmNX3E=NpZr2kJokFHK6xB850NLjDh12Wo9o4Mcng@mail.gmail.com>
	<3BE0DEED8863E5429BAE4CAEDF624565039C1B330ACE@AIRA-SRV.aira.local>
Message-ID: <B59886DC-DD4E-4A64-B081-10240CDD25D0@cos.ru>

9 ??????? 2014??. 15:58:05 CET, Filip Marvan <filip.marvan at aira.cz> ?????:
>Yes, you have to use bootable DOS and DOS version of sas2flash.
>
>There is an awesome tool for creating bootable USB stick with DOS
>called Rufus
>
>
>
>http://rufus.akeo.ie/
>
>
>
>Filip
>
>
>
>
>
>From: Schweiss, Chip [mailto:chip at innovates.com]
>Sent: Tuesday, December 09, 2014 3:50 PM
>To: Filip Marvan
>Cc: Matthew Lagoe; omnios-discuss at lists.omniti.com
>Subject: Re: [OmniOS-discuss] Parity error on path mpt_sas2
>
>
>
>I'm still fighting this.
>
>Under OmniOS the erase function is disabled in sas2flash for solaris.
>
>I couldn't seem to find an iso bootable DOS that I could stage files
>in. 
>FreeDOS1.0 won't mount the CD, 1.1 got rid of the live mode.    Tried a
>Linux 
>rescue CD and I get " ERROR: Erase Flash Operation Failed!"   Tried
>both P18 
>sas2flash and P20 sas2flash.
>
>What OS environment are you running this in?
>
>
>
>
>
>On Tue, Dec 9, 2014 at 3:06 AM, Filip Marvan <filip.marvan at aira.cz>
>wrote:
>
>Hi,
>
>
>
>first you have to erase P20 firmware with:
>
>
>
>sas2flsh -o -e 6
>
>
>
>now do not reboot(!) and flash P18 with
>
>
>
>sas2flsh -f XXXXXX.bin -b mptsas2.rom
>
>
>
>Filip
>
>
>
>
>
>From: OmniOS-discuss [mailto:omnios-discuss-bounces at lists.omniti.com]
>On 
>Behalf Of Schweiss, Chip
>Sent: Monday, December 08, 2014 08:09 AM
>To: Filip Marvan
>Cc: omnios-discuss at lists.omniti.com
>Subject: Re: [OmniOS-discuss] Parity error on path mpt_sas2
>
>
>
>I've got some new LSI HBAs I'm trying to downgrade from firmware
>version 20 to 
>18.
>
>I'm getting errors when trying to downgrade:
>
>Attempting to flash firmware to LSI SAS SAS2308_2(D1) :
>
>        Executing Operation: Flash Firmware Image
>
>                Firmware Image has a Valid Checksum.
>                Firmware Version 18.00.00.00
>                Firmware Image compatible with Controller.
>
>                Valid NVDATA Image found.
>                NVDATA Version 11.00.00.00
>                Checking for a compatible NVData image...
>
>                NVDATA Device ID and Chip Revision match verified.
>                ERROR: Cannot downgrade NVDATA version 14.00.00.06
>                       to 11.00.110000.00.
>
>                ERROR: Failed to get valid NVDATA image from File!
>
>                Firmware Image Validation Failed!
>
>Tried downgrading bios:
>
>Attempting to flash Boot Service to LSI SAS SAS2308_2(D1) :
>
>                Validating BIOS Image...
>
>                BIOS Header Signature is Valid
>
>                BIOS Image has a Valid Checksum.
>
>                BIOS PCI Structure Signature Valid.
>
>                BIOS Image Compatible with the SAS Controller.
>
>                Attempting to Flash BIOS Image...
>
>                BIOS Version in flash    : 07.39.00.00
>                BIOS Version from File  : 07.35.00.00
>                Skipping flash since file version is not greater than 
>existing.
>
>                Flash BIOS Image Failed!
>
>I've tried sas2flash from P20 and P18.
>
>Can someone fill me in on the trick to downgrade?
>
>-Chip
>
>
>
>
>
>On Thu, Nov 27, 2014 at 9:03 AM, Filip Marvan <filip.marvan at aira.cz>
>wrote:
>
>Thank you for your help Aaron! I downgraded firmware to P18 and there
>are no 
>more errors today.
>
>It seems, that there is something bad with P20 firmware!
>
>
>
>You saved me a lot of time J
>
>
>
>Filip
>
>
>
>
>
>From: Aaron Curry [mailto:asc1111 at gmail.com]
>Sent: Tuesday, November 25, 2014 5:46 PM
>To: Filip Marvan
>Cc: Dan McDonald; omnios-discuss at lists.omniti.com
>Subject: Re: [OmniOS-discuss] Parity error on path mpt_sas2
>
>
>
>I had this exact same problem recently when setting up a new home
>server... 
>same controller, same firmware (P20). The errors were on all disks
>attached to 
>the controller, but only on high read activity. Writes did not generate
>the 
>errors. I downgraded the firmware to P18 (which is what we are using at
>work) 
>and the errors went away. Has anyone had success with this controller
>and the 
>P20 firmware?
>
>
>
>Aaron
>
>
>
>On Tue, Nov 25, 2014 at 8:30 AM, Filip Marvan <filip.marvan at aira.cz>
>wrote:
>
>Hi Dan,
>
>thanks for reply.
>Yes, errors are on all 5 disks in the same RAIDZ pool. No problem on
>other
>disks in different pool but on the same SAS cable.
>So maybe I only had bad luck with that disks.
>
>Filip
>
>
>_______________________________________________
>OmniOS-discuss mailing list
>OmniOS-discuss at lists.omniti.com
>http://lists.omniti.com/mailman/listinfo/omnios-discuss
>
>
>
>
>_______________________________________________
>OmniOS-discuss mailing list
>OmniOS-discuss at lists.omniti.com
>http://lists.omniti.com/mailman/listinfo/omnios-discuss
>
>
>
>
>
>
>
>------------------------------------------------------------------------
>
>_______________________________________________
>OmniOS-discuss mailing list
>OmniOS-discuss at lists.omniti.com
>http://lists.omniti.com/mailman/listinfo/omnios-discuss

Thanks all for the insightful thread. I found a usb image instead, with freedos and batch files prepared to reflash lsi - see http://joelinoff.com/blog/?p=588

HTH, Jim
--
Typos courtesy of K-9 Mail on my Samsung Android

From chip at innovates.com  Fri Dec 12 14:33:55 2014
From: chip at innovates.com (Schweiss, Chip)
Date: Fri, 12 Dec 2014 08:33:55 -0600
Subject: [OmniOS-discuss] hangs on reboot
In-Reply-To: <548AD071.5070306@jvm.de>
References: <1418336750819.83182@steait.net>
	<EFBA16E3-3165-4164-B700-53A80F6D48CE@omniti.com>
	<1418337697672.85350@steait.net>
	<B5E5DBE6-67C1-4E63-BDA9-B52C40E4B545@cos.ru>
	<B515960D-28BB-462A-AADF-464D10112679@cos.ru> <548AD071.5070306@jvm.de>
Message-ID: <CALeZrrSqdjk5fhDa0AGLJ+H1xxeeZTPQDMqY-nLTWFzpdnsf3g@mail.gmail.com>

There is one situation that svccfg changes are not turning off fast
reboot.   If the system has a panic the next boot will have an automated
reboot that gets hung.

I'm assuming this is the crash dump being collected then a reboot
initiated.

Anyone know how to fix this one or how to collect better information on
where the fast reboot is coming from?

-Chip

On Fri, Dec 12, 2014 at 5:24 AM, Stephan Budach <stephan.budach at jvm.de>
wrote:
>
> Am 12.12.14 um 08:58 schrieb Jim Klimov:
>
>  12 ??????? 2014 ?. 8:35:18 CET, Jim Klimov <jimklimov at cos.ru> ?????:
>>
>>> 11 ??????? 2014 ?. 23:41:35 CET, Rune Tipsmark <rt at steait.net> ?????:
>>>
>>>> NIC's are not shared with IPMI.
>>>>
>>>>
>>>>
>>>> I will try the target-mode first, however on the 3rd box I actually
>>>> have Infiniband and it also caused the same issue without touching the
>>>> target-mode for FC.
>>>>
>>>>
>>>>
>>>> I have a 4th system on an old HP server and it reboots perfect every
>>>> time, no target-mode and has Infiniband as well... I am leaning
>>>>
>>> towards
>>>
>>>> something with the SuperMicro hardware but can't really pinpoint it.
>>>>
>>>> br,
>>>>
>>>> Rune
>>>>
>>>> ________________________________
>>>> From: Dan McDonald <danmcd at omniti.com>
>>>> Sent: Thursday, December 11, 2014 11:39 PM
>>>> To: Rune Tipsmark
>>>> Cc: omnios-discuss at lists.omniti.com
>>>> Subject: Re: [OmniOS-discuss] hangs on reboot
>>>>
>>>> Nothing printed out on the console?
>>>>
>>>> And try eliminating the target mode first - just in case.
>>>>
>>>> Also, is one of your NICs a dual IPMI/host NIC?  If so, disable the
>>>> IPMI portion, we don't cope with shared NIC ports like that.
>>>>
>>>> Dan
>>>>
>>>> Sent from my iPhone (typos, autocorrect, and all)
>>>>
>>>> On Dec 11, 2014, at 5:25 PM, Rune Tipsmark
>>>> <rt at steait.net<mailto:rt at steait.net>> wrote:
>>>>
>>>>
>>>> hi all,
>>>>
>>>>
>>>>
>>>> I got a bunch (3) installations of omnios on SuperMicro hardware and
>>>> all 3 have issues rebooting. They simply hang and never ever reboot.
>>>>
>>>>
>>>>
>>>> The install is latest version and I only added the storage-server
>>>> package and installed napp-it and changed the fibre channel setting in
>>>> /kernel/drv/emlxs.conf target-mode=1
>>>>
>>>>
>>>>
>>>> two nics igb0 and igb1 configured as aggregation (aggr0)
>>>>
>>>>
>>>>
>>>> besides this its 100% default install, not 10 minutes old...
>>>>
>>>>
>>>>
>>>> Any ideas?
>>>>
>>>> br,
>>>>
>>>> Rune
>>>>
>>>> _______________________________________________
>>>> OmniOS-discuss mailing list
>>>> OmniOS-discuss at lists.omniti.com<mailto:OmniOS-discuss at lists.omniti.com>
>>>> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>>>>
>>>>
>>>> ------------------------------------------------------------
>>>> ------------
>>>>
>>>> _______________________________________________
>>>> OmniOS-discuss mailing list
>>>> OmniOS-discuss at lists.omniti.com
>>>> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>>>>
>>> I've installed latest bloody on an HP Z400 workstation last week with
>>> an lsi 9212 addon hba, and it too needed to disable fastboot.
>>>
>>> IIRC this is somehow tied to ability of all drivers to quiesce like for
>>> standby - if some one refuses to, suspend or fastboot fail.
>>> IMHO the fastboot handler should have a reasonable/configurable timeout
>>> after 'stand by for reboot' to do slowboot if that expires.
>>>
>>> Alternately on servers you can set up bmc-watchdog and have the mobo
>>> reboot itself. But then use long timeouts (10-15 min) and/or take care
>>> that the watchdog driver is among the first pieces of software that
>>> runs in your bootup or especially 'live' repair-boots.
>>>
>>> HTH,
>>> Jim
>>> --
>>> Typos courtesy of K-9 Mail on my Samsung Android
>>>
>> Forgot to add my 2c: if the driver quiesce is at fault, maybe it is also
>> a signal for devs to improve at least nominal hw support for new usb3,
>> video chips, maybe some sata HBAs?
>> Even if they don't perform optimally (i.e. let usb3 be as slow as usb2 or
>> usb1, or let radeon be a vesa-video, but let it work without errors) --
>> they should not impede system work either.
>>
>> Jim
>> --
>>
> I have recently bought a new SuperMicro which showed exactly the same
> behaviour regarding fast reboots. Rebooting through the prom however,
> worked each time I can remember trying it. That system had been originally
> installed using the release 006 of OmniOS. After upgrading to 012 stable,
> these issues seem to have gone. The only thing that is different on the new
> host is, that it's equipped with a SSD DOM instead of a "traditional" HD,.
> This DOM is stuffed in SATA0 and somewhat cabled to the MB, so it's not
> just a dumb one. This cable seems to be a bit picky about its connection of
> the plug that goes into the DOM.
>
> As for 012? I just performed 4 fast reboots in a row without issues on my
> new SuperMicro box?
>
> Cheers,
> budy
>
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141212/cdb6824a/attachment-0001.html>

From eric.sproul at circonus.com  Fri Dec 12 15:08:14 2014
From: eric.sproul at circonus.com (Eric Sproul)
Date: Fri, 12 Dec 2014 10:08:14 -0500
Subject: [OmniOS-discuss] Java paths in OmniOS
In-Reply-To: <AFA0DC98-08AB-43A5-9019-023EB9140883@cos.ru>
References: <AFA0DC98-08AB-43A5-9019-023EB9140883@cos.ru>
Message-ID: <CAO8hXRC=YYpFQqmBpHfAUEwpZTt19t_rKz+zK_xL2=1jK30ZdQ@mail.gmail.com>

On Fri, Dec 12, 2014 at 2:52 AM, Jim Klimov <jimklimov at cos.ru> wrote:
> The new structure all but forbids such customization; was there and particular benefit from changing to it?

The Java that ships with OmniOS is only there to satisfy runtime
dependencies for certain things that need it.  You are almost always
better off getting your own JDK/JRE and installing it however you
wish.  The Oracle Java downloads work just fine, and so does OpenJDK,
though that's trickier to build yourself (but you can look at the
omnios-build script for hints).

Eric

From zmalone at omniti.com  Fri Dec 12 15:25:32 2014
From: zmalone at omniti.com (Zach Malone)
Date: Fri, 12 Dec 2014 10:25:32 -0500
Subject: [OmniOS-discuss] Python difference
In-Reply-To: <CAHNoB+1pePnQuUNHSBKLN14975bqHPX1C6F1E7nciZjkE2qQDw@mail.gmail.com>
References: <CAHNoB+1iJ=TNq9Z0sZ-WO-BMtx--tXcrXkVyWvcu3UfS-W+CCg@mail.gmail.com>
	<56C3D19D-B648-4F02-A7CA-BC24A31C385E@omniti.com>
	<CAHNoB+1pePnQuUNHSBKLN14975bqHPX1C6F1E7nciZjkE2qQDw@mail.gmail.com>
Message-ID: <CALLZ=vV1FoK_XdTmqXRSetrnZ70bE_WdvS0uQa_SSNipxokFvg@mail.gmail.com>

/opt/ha is locally built, it isn't delivered by OmniOS.  I'm not sure
why it would go missing on upgrade, how did you initially create it?
--Zach

On Thu, Dec 11, 2014 at 5:23 PM, Joe Veliscos <joeveliscos at gmail.com> wrote:
> Hi
>
> The command that I execute :
> root#crm
> abort: couldn't find crm libraries in [/opt/ha/sbin
> /usr/lib/python2.6/vendor-packages/setuptools-0.6c11-py2.6.egg
> /opt/ha/lib/python2.6/site-packages /usr/lib/python26.zip /usr/lib/python2.6
> /usr/lib/python2.6/plat-sunos5 /usr/lib/python2.6/lib-tk
> /usr/lib/python2.6/lib-old /usr/lib/python2.6/lib-dynload
> /usr/lib/python2.6/site-packages /usr/lib/python2.6/vendor-packages]
> (check your install and PYTHONPATH)
>
> I have the following environment variables set:
> export PYTHONPATH=/opt/ha/lib/python2.6/site-packages
> export PATH=/opt/ha/bin:/opt/ha/sbin:$PATH
> export OCF_ROOT=/opt/ha/lib/ocf
> export OCF_AGENTS=/opt/ha/lib/ocf/resource.d/heartbeat
>
> I have exactly the same in an r10 release (pre upgrade to rr12) where there
> is no problem
>
> I did a truss -d crm and it seems that many files it searches for are not
> found.  Snippets of the output (very long file) hope this helps:
>
> Below (as I understand it ) some searches which it can resolve:
>
> 0.0098    resolvepath("/usr/lib/amd64/ld.so.1", "/lib/amd64/ld.so.1", 1023)
> = 18
>  0.0100    resolvepath("/usr/bin/amd64/python2.6",
> "/usr/bin/amd64/python2.6", 1023) = 24
>  0.0101    stat("/usr/bin/amd64/python2.6", 0xFFFFFD7FFFDFF910) = 0
>  0.0103    open("/var/ld/64/ld.config", O_RDONLY)        Err#2 ENOENT
>  0.0105    stat("/usr/gnu/lib/amd64/libpython2.6.so.1.0",
> 0xFFFFFD7FFFDFF000) Err#2 ENOENT
>  0.0106    stat("/usr/lib/amd64/libpython2.6.so.1.0", 0xFFFFFD7FFFDFF000) =
> 0
>  0.0108    resolvepath("/usr/lib/amd64/libpython2.6.so.1.0",
> "/usr/lib/amd64/libpython2.6.so.1.0", 1023) = 34
>  0.0110    open("/usr/lib/amd64/libpython2.6.so.1.0", O_RDONLY) = 3
>  0.0112    mmapobj(3, MMOBJ_INTERPRET, 0xFFFFFD7FFF350AB8,
> 0xFFFFFD7FFFDFEB5C, 0x00000000) = 0
>  0.0113    close(3)                    = 0
>  0.0115    mmap(0x00000000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC,
> MAP_PRIVATE|MAP_ANON, 4294967295, 0) = 0xFFFFFD7FFF340000
>  0.0117    memcntl(0xFFFFFD7FFEAB0000, 457808, MC_ADVISE, MADV_WILLNEED, 0,
> 0) = 0
>  0.0118    stat("/usr/gnu/lib/amd64/libsocket.so.1", 0xFFFFFD7FFFDFF000)
> Err#2 ENOENT
>  0.0120    stat("/usr/lib/amd64/libsocket.so.1", 0xFFFFFD7FFFDFF000) = 0
>  0.0121    resolvepath("/usr/lib/amd64/libsocket.so.1",
> "/lib/amd64/libsocket.so.1", 1023) = 25
>  0.0123    open("/usr/lib/amd64/libsocket.so.1", O_RDONLY)    = 3
>  0.0125    mmapobj(3, MMOBJ_INTERPRET, 0xFFFFFD7FFF340A18,
> 0xFFFFFD7FFFDFEB5C, 0x00000000) = 0
>  0.0127    close(3)                    = 0
>  0.0128    mmap(0x00000000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC,
> MAP_PRIVATE|MAP_ANON, 4294967295, 0) = 0xFFFFFD7FFF330000
>  0.0129    memcntl(0xFFFFFD7FFEA80000, 32240, MC_ADVISE, MADV_WILLNEED, 0,
> 0) = 0
>  0.0130    stat("/usr/gnu/lib/amd64/libnsl.so.1", 0xFFFFFD7FFFDFF000) Err#2
> ENOENT
>  0.0132    stat("/usr/lib/amd64/libnsl.so.1", 0xFFFFFD7FFFDFF000) = 0
>  0.0134    resolvepath("/usr/lib/amd64/libnsl.so.1",
> "/lib/amd64/libnsl.so.1", 1023) = 22
>  0.0135    open("/usr/lib/amd64/libnsl.so.1", O_RDONLY)    = 3
>  0.0137    mmapobj(3, MMOBJ_INTERPRET, 0xFFFFFD7FFF3309C8,
> 0xFFFFFD7FFFDFEB5C, 0x00000000) = 0
>  0.0139    close(3)                    = 0
>  0.0140    mmap(0x00000000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC,
> MAP_PRIVATE|MAP_ANON, 4294967295, 0) = 0xFFFFFD7FFF320000
>  0.0141    memcntl(0xFFFFFD7FFEDD0000, 180072, MC_ADVISE, MADV_WILLNEED, 0,
> 0) = 0
>  0.0142    stat("/usr/gnu/lib/amd64/libm.so.2", 0xFFFFFD7FFFDFF000) Err#2
> ENOENT
>  0.0144    stat("/usr/lib/amd64/libm.so.2", 0xFFFFFD7FFFDFF000) = 0
>  0.0145    resolvepath("/usr/lib/amd64/libm.so.2", "/lib/amd64/libm.so.2",
> 1023) = 20
>  0.0147    open("/usr/lib/amd64/libm.so.2", O_RDONLY)    = 3
>  0.0149    mmapobj(3, MMOBJ_INTERPRET, 0xFFFFFD7FFF3209F8,
> 0xFFFFFD7FFFDFEB5C, 0x00000000) = 0
>  0.0150    close(3)                    = 0
>  0.0151    mmap(0x00000000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC,
> MAP_PRIVATE|MAP_ANON, 4294967295, 0) = 0xFFFFFD7FFF310000
>  0.0153    memcntl(0xFFFFFD7FFEEE0000, 58680, MC_ADVISE, MADV_WILLNEED, 0,
> 0) = 0
>  0.0154    stat("/usr/gnu/lib/amd64/libc.so.1", 0xFFFFFD7FFFDFF000) Err#2
> ENOENT
>  0.0156    stat("/usr/lib/amd64/libc.so.1", 0xFFFFFD7FFFDFF000) = 0
>  0.0157    resolvepath("/usr/lib/amd64/libc.so.1", "/lib/amd64/libc.so.1",
> 1023) = 20
>  0.0159    open("/usr/lib/amd64/libc.so.1", O_RDONLY)    = 3
>  0.0161    mmapobj(3, MMOBJ_INTERPRET, 0xFFFFFD7FFF310920,
> 0xFFFFFD7FFFDFEB5C, 0x00000000) = 0
>  0.0162    close(3)                    = 0
>  0.0163    mmap(0x00000000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC,
> MAP_PRIVATE|MAP_ANON, 4294967295, 0) = 0xFFFFFD7FFF160000
>  0.0165    memcntl(0xFFFFFD7FFF170000, 477048, MC_ADVISE, MADV_WILLNEED, 0,
> 0) = 0
>  0.0166    stat("/lib/64/libsocket.so.1", 0xFFFFFD7FFFDFF000) = 0
>  0.0168    resolvepath("/lib/64/libsocket.so.1",
> "/lib/amd64/libsocket.so.1", 1023) = 25
>  0.0170    stat("/lib/64/libnsl.so.1", 0xFFFFFD7FFFDFF000)    = 0
>  0.0171    resolvepath("/lib/64/libnsl.so.1", "/lib/amd64/libnsl.so.1",
> 1023) = 22
>  0.0173    stat("/lib/64/libm.so.2", 0xFFFFFD7FFFDFF000)    = 0
>  0.0174    resolvepath("/lib/64/libm.so.2", "/lib/amd64/libm.so.2", 1023) =
> 20
>  0.0177    stat("/usr/gnu/lib/amd64/libgcc_s.so.1", 0xFFFFFD7FFFDFF000)
> Err#2 ENOENT
>  0.0179    stat("/lib/64/libgcc_s.so.1", 0xFFFFFD7FFFDFF000) Err#2 ENOENT
>  0.0180    stat("/usr/lib/64/libgcc_s.so.1", 0xFFFFFD7FFFDFF000) = 0
>  0.0185    resolvepath("/usr/lib/64/libgcc_s.so.1",
> "/usr/lib/amd64/libgcc_s.so.1", 1023) = 28
>  0.0187    open("/usr/lib/64/libgcc_s.so.1", O_RDONLY)    = 3
>
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> Below searches which it cannot resolve:
>
> 0.0329    fstat(2, 0xFFFFFD7FFFDFF920)            = 0
>  0.0331    readlink("/usr/bin/python", "python2.6", 1024)    = 9
>  0.0333    readlink("/usr/bin/python2.6", 0xFFFFFD7FFFDFF5B0, 1024) Err#22
> EINVAL
>  0.0335    stat("/usr/bin/Modules/Setup", 0xFFFFFD7FFFDFF5B0) Err#2 ENOENT
>  0.0336    stat("/usr/bin/lib/python2.6/os.py", 0xFFFFFD7FFFDFF5B0) Err#2
> ENOENT
>  0.0338    stat("/usr/bin/lib/python2.6/os.pyc", 0xFFFFFD7FFFDFF5B0) Err#2
> ENOENT
>  0.0342    stat("/usr/lib/python2.6/os.py", 0xFFFFFD7FFFDFF5B0) = 0
>  0.0344    stat("/usr/bin/Modules/Setup", 0xFFFFFD7FFFDFF120) Err#2 ENOENT
>  0.0345    stat("/usr/bin/lib/python2.6/lib-dynload", 0xFFFFFD7FFFDFF120)
> Err#2 ENOENT
>  0.0347    stat("/usr/lib/python2.6/lib-dynload", 0xFFFFFD7FFFDFF120) = 0
>  0.0351    brk(0x00485250)                    = 0
>
>  0.0456    sysconfig(_CONFIG_SIGRT_MAX)            = 73
>  0.0459    stat("/opt/ha/lib/python2.6/site-packages", 0xFFFFFD7FFFDFDF90) =
> 0
>  0.0461    stat("/opt/ha/lib/python2.6/site-packages", 0xFFFFFD7FFFDFE340) =
> 0
>  0.0462    stat("/opt/ha/lib/python2.6/site-packages/site",
> 0xFFFFFD7FFFDFE640) Err#2 ENOENT
>  0.0464    stat("/opt/ha/lib/python2.6/site-packages/site",
> 0xFFFFFD7FFFDFEAD0) Err#2 ENOENT
>  0.0465    open("/opt/ha/lib/python2.6/site-packages/64/site.so", O_RDONLY)
> Err#2 ENOENT
>  0.0467    stat("/opt/ha/lib/python2.6/site-packages/site",
> 0xFFFFFD7FFFDFEAD0) Err#2 ENOENT
>  0.0469    open("/opt/ha/lib/python2.6/site-packages/64/sitemodule.so",
> O_RDONLY) Err#2 ENOENT
>  0.0470    stat("/opt/ha/lib/python2.6/site-packages/site",
> 0xFFFFFD7FFFDFEAD0) Err#2 ENOENT
>  0.0471    open("/opt/ha/lib/python2.6/site-packages/site.py", O_RDONLY)
> Err#2 ENOENT
>  0.0473    stat("/opt/ha/lib/python2.6/site-packages/site",
> 0xFFFFFD7FFFDFEAD0) Err#2 ENOENT
>  0.0474    open("/opt/ha/lib/python2.6/site-packages/site.pyc", O_RDONLY)
> Err#2 ENOENT
>  0.0476    stat("/usr/lib/python26.zip", 0xFFFFFD7FFFDFDF90) Err#2 ENOENT
>  0.0477    stat("/usr/lib", 0xFFFFFD7FFFDFDF90)        = 0
>  0.0478    stat("/usr/lib/python26.zip", 0xFFFFFD7FFFDFE340) Err#2 ENOENT
>  0.0480    stat("/usr/lib/python2.6/", 0xFFFFFD7FFFDFDF90)    = 0
>  0.0481    stat("/usr/lib/python2.6/", 0xFFFFFD7FFFDFE340)    = 0
>  0.0482    stat("/usr/lib/python2.6/site", 0xFFFFFD7FFFDFE640) Err#2 ENOENT
>  0.0483    stat("/usr/lib/python2.6/site", 0xFFFFFD7FFFDFEAD0) Err#2 ENOENT
>  0.0485    open("/usr/lib/python2.6/64/site.so", O_RDONLY)    Err#2 ENOENT
>  0.0486    stat("/usr/lib/python2.6/site", 0xFFFFFD7FFFDFEAD0) Err#2 ENOENT
>  0.0487    open("/usr/lib/python2.6/64/sitemodule.so", O_RDONLY) Err#2
> ENOENT
>  0.0488    stat("/usr/lib/python2.6/site", 0xFFFFFD7FFFDFEAD0) Err#2 ENOENT
>  0.0490    open("/usr/lib/python2.6/site.py", O_RDONLY)    = 3
>  0.0491    fstat(3, 0xFFFFFD7FFFDFEA90)            = 0
>  0.0492    open("/usr/lib/python2.6/site.pyc", O_RDONLY)    = 4
>  0.0493    fstat(4, 0xFFFFFD7FFFDFE8D0)            = 0
>  0.0494    brk(0x004D5250)                    = 0
>  0.0496    brk(0x004D9250)                    = 0
>  0.0498    fstat(4, 0xFFFFFD7FFFDFE800)            = 0
>  0.0498    ioctl(4, TCGETA, 0xFFFFFD7FFFDFE880)        Err#25 ENOTTY
>  0.0500    read(4, "D1F2\r\nA09390 S c\0\0\0".., 18944)    = 18651
>
> 0.0514    stat("/opt/ha/lib/python2.6/site-packages/os", 0xFFFFFD7FFFDFD320)
> Err#2 ENOENT
>  0.0516    stat("/opt/ha/lib/python2.6/site-packages/os",
> 0xFFFFFD7FFFDFD7B0) Err#2 ENOENT
>  0.0517    open("/opt/ha/lib/python2.6/site-packages/64/os.so", O_RDONLY)
> Err#2 ENOENT
>  0.0519    stat("/opt/ha/lib/python2.6/site-packages/os",
> 0xFFFFFD7FFFDFD7B0) Err#2 ENOENT
>  0.0520    open("/opt/ha/lib/python2.6/site-packages/64/osmodule.so",
> O_RDONLY) Err#2 ENOENT
>  0.0521    stat("/opt/ha/lib/python2.6/site-packages/os",
> 0xFFFFFD7FFFDFD7B0) Err#2 ENOENT
>  0.0522    open("/opt/ha/lib/python2.6/site-packages/os.py", O_RDONLY) Err#2
> ENOENT
>  0.0524    stat("/opt/ha/lib/python2.6/site-packages/os",
> 0xFFFFFD7FFFDFD7B0) Err#2 ENOENT
>  0.0525    open("/opt/ha/lib/python2.6/site-packages/os.pyc", O_RDONLY)
> Err#2 ENOENT
>  0.0527    stat("/usr/lib/python2.6/os", 0xFFFFFD7FFFDFD320) Err#2 ENOENT
>  0.0528    stat("/usr/lib/python2.6/os", 0xFFFFFD7FFFDFD7B0) Err#2 ENOENT
>  0.0529    open("/usr/lib/python2.6/64/os.so", O_RDONLY)    Err#2 ENOENT
>  0.0530    stat("/usr/lib/python2.6/os", 0xFFFFFD7FFFDFD7B0) Err#2 ENOENT
>  0.0532    open("/usr/lib/python2.6/64/osmodule.so", O_RDONLY) Err#2 ENOENT
>  0.0533    stat("/usr/lib/python2.6/os", 0xFFFFFD7FFFDFD7B0) Err#2 ENOENT
>  0.0534    open("/usr/lib/python2.6/os.py", O_RDONLY)    = 4
>  0.0536    fstat(4, 0xFFFFFD7FFFDFD770)            = 0
>  0.0537    open("/usr/lib/python2.6/os.pyc", O_RDONLY)    = 5
>  0.0538    fstat(5, 0xFFFFFD7FFFDFD5B0)            = 0
>  0.0539    fstat(5, 0xFFFFFD7FFFDFD4E0)            = 0
>  0.0540    ioctl(5, TCGETA, 0xFFFFFD7FFFDFD560)        Err#25 ENOTTY
>  0.0541    read(5, "D1F2\r\n9F9390 S c\0\0\0".., 26112)    = 25702
>
>  0.0568    stat("/opt/ha/lib/python2.6/site-packages/posixpath",
> 0xFFFFFD7FFFDFC000) Err#2 ENOENT
>  0.0570    stat("/opt/ha/lib/python2.6/site-packages/posixpath",
> 0xFFFFFD7FFFDFC490) Err#2 ENOENT
>  0.0572    open("/opt/ha/lib/python2.6/site-packages/64/posixpath.so",
> O_RDONLY) Err#2 ENOENT
>  0.0573    stat("/opt/ha/lib/python2.6/site-packages/posixpath",
> 0xFFFFFD7FFFDFC490) Err#2 ENOENT
>  0.0575    open("/opt/ha/lib/python2.6/site-packages/64/posixpathmodule.so",
> O_RDONLY) Err#2 ENOENT
>  0.0576    stat("/opt/ha/lib/python2.6/site-packages/posixpath",
> 0xFFFFFD7FFFDFC490) Err#2 ENOENT
>  0.0577    open("/opt/ha/lib/python2.6/site-packages/posixpath.py",
> O_RDONLY) Err#2 ENOENT
>  0.0579    stat("/opt/ha/lib/python2.6/site-packages/posixpath",
> 0xFFFFFD7FFFDFC490) Err#2 ENOENT
>  0.0580    open("/opt/ha/lib/python2.6/site-packages/posixpath.pyc",
> O_RDONLY) Err#2 ENOENT
>  0.0582    stat("/usr/lib/python2.6/posixpath", 0xFFFFFD7FFFDFC000) Err#2
> ENOENT
>  0.0583    stat("/usr/lib/python2.6/posixpath", 0xFFFFFD7FFFDFC490) Err#2
> ENOENT
>  0.0584    open("/usr/lib/python2.6/64/posixpath.so", O_RDONLY) Err#2 ENOENT
>  0.0585    stat("/usr/lib/python2.6/posixpath", 0xFFFFFD7FFFDFC490) Err#2
> ENOENT
>  0.0587    open("/usr/lib/python2.6/64/posixpathmodule.so", O_RDONLY) Err#2
> ENOENT
>  0.0588    stat("/usr/lib/python2.6/posixpath", 0xFFFFFD7FFFDFC490) Err#2
> ENOENT
>  0.0589    open("/usr/lib/python2.6/posixpath.py", O_RDONLY) = 5
>  0.0591    fstat(5, 0xFFFFFD7FFFDFC450)            = 0
>  0.0592    open("/usr/lib/python2.6/posixpath.pyc", O_RDONLY) = 6
>  0.0593    fstat(6, 0xFFFFFD7FFFDFC290)            = 0
>
>
> And the list goes on.
>
> Hope there's a solution for this.
>
>
> Joe
>
>
> On Thu, Dec 11, 2014 at 10:26 PM, Dan McDonald <danmcd at omniti.com> wrote:
>>
>>
>> > On Dec 11, 2014, at 4:00 PM, Joe Veliscos <joeveliscos at gmail.com> wrote:
>> >
>> > Hi,
>> >
>> > I have an application working on Omnios r10 which depends on certain
>> > python libraries. There are certain environment variables in place which
>> > point to the location of those libraries.
>> >
>> > I have updated the r10 machine to r12. The application now cannot be
>> > started with errors stating that the needed libraries cannot be found in the
>> > given paths.
>>
>> Share the errors please?  I'll need more details.
>>
>> > Maybe somebody can tell me what the difference is in python version
>> > between the two omnios releases. Python -V gives me 2.6.8. on both versions.
>>
>> We updated supplemental python libraries, which may contribute to what
>> you're seeing.  Also, we had not updated the "entire" metapackage on 010 to
>> show we were actually running 2.6.8.
>>
>> Dan
>>
>
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>

From info at houseofancients.nl  Fri Dec 12 15:24:34 2014
From: info at houseofancients.nl (Floris van Essen ..:: House of Ancients Amstafs ::..)
Date: Fri, 12 Dec 2014 15:24:34 +0000
Subject: [OmniOS-discuss] LACP Omnios , igb and C3750
In-Reply-To: <~B548633c30000.5486a0500000.0002.mml.3857832108@vEX01.mindstorm-internet.local>
References: <~B548633c30000.5486a0500000.0002.mml.3857832108@vEX01.mindstorm-internet.local>
Message-ID: <356582D1FC91784992ABB4265A16ED487EF447DC@vEX01.mindstorm-internet.local>

Hi All,

this just keeps getting stranger by the minute.
after removing the aggr's and setting a static ip on each interface,
i can ping from the host itself, but not through a wire, direct or through a switch.
plugged in a new intel i350, same deal...
when looking at the driver, i see that the is a update date of november 6....

does anybody have an idea what could be going on ?

Met vriendelijke groet / With kind regards,



               Floris van Essen

Van: OmniOS-discuss [mailto:omnios-discuss-bounces at lists.omniti.com] Namens Floris van Essen ..:: House of Ancients Amstafs ::..
Verzonden: dinsdag 9 december 2014 0:15
Aan: omnios-discuss at lists.omniti.com
Onderwerp: [OmniOS-discuss] LACP Omnios , igb and C3750

Guys,
been having a odd issue with my Omnios box and c3750
the omnios box has 6 interfaces ( 2 e1000g and 4 igb)
somehow the LACP bonding between the igb's and my cisco's will not stay in sync, while the same lacp config runs fine over the e1000's :
channel 5 = e1000g's channel 6 and 7 are the igb's :
Channel group 5 neighbors
Partner's information:
LACP port Admin Oper Port Port
Port Flags Priority Dev ID Age key Key Number State
Gi1/0/14 FA 4096 0030.48d5.ec94 23s 0x0 0x3E8 0x2 0x3F
Gi2/0/14 FA 4096 0030.48d5.ec94 23s 0x0 0x3E8 0x1 0x3F
Channel group 6 neighbors
Partner's information:
LACP port Admin Oper Port Port
Port Flags Priority Dev ID Age key Key Number State
Gi1/0/10 FA 4096 a036.9f02.c13a 0s 0x0 0x3EA 0x6 0x47
Gi1/0/12 FA 4096 a036.9f02.c13a 0s 0x0 0x3EA 0x5 0x47
Channel group 7 neighbors
Partner's information:
LACP port Admin Oper Port Port
Port Flags Priority Dev ID Age key Key Number State
Gi2/0/10 FA 4096 a036.9f02.c138 0s 0x0 0x3E9 0x4 0x47
Gi2/0/11 FA 4096 a036.9f02.c138 0s 0x0 0x3E9 0x3 0x47
LINK POLICY ADDRPOLICY LACPACTIVITY LACPTIMER FLAGS
aggr0 L4 auto active short -----
aggr1 L4 auto active short -----
aggr2 L4 auto active short -----
aggr0 -- up 1000Mb full static x.x.x.x 255.255.255.0 x:x:x:x:x:x 1500
e1000g0 attached up 1000Mb full - - - x:x:x:x:x:x 1500
e1000g1 attached up 1000Mb full - - - x:x:x:x:x:x 1500
aggr1 -- up 1000Mb full static x.x.x.x 255.255.255.0 x:x:x:x:x:x 1500
igb0 attached up 1000Mb full - - - x:x:x:x:x:x 1500
igb1 attached up 1000Mb full - - - x:x:x:x:x:x 1500
aggr2 -- up 1000Mb full static x.x.x.x 255.255.255.0 x:x:x:x:x:x 1500
igb2 attached up 1000Mb full - - - x:x:x:x:x:x 1500
igb3 attached up 1000Mb full - - - x:x:x:x:x:x 1500
058673: Dec 8 22:15:51.593: lacp_mux Gi2/0/10 - mux: during state WAITING, got event 1(selected)
058674: Dec 8 22:15:51.593: @@@ lacp_mux Gi2/0/10 - mux: WAITING -> WAITING
058675: Dec 8 22:15:51.593: LACP: Gi2/0/10 lacp_action_mx_waiting entered
058676: Dec 8 22:15:51.593: LACP: timer lacp_w(Gi2/0/10) started with interval 2000.
058677: Dec 8 22:15:51.895: LACP :lacp_bugpak: Receive LACP-PDU packet via Gi2/0/10
058678: Dec 8 22:15:51.895: LACP : packet size: 124
058679: Dec 8 22:15:51.895: LACP: pdu: subtype: 1, version: 1
058680: Dec 8 22:15:51.895: LACP: Act: tlv:1, tlv-len:20, key:0x3E9, p-pri:0x1000, p:0x4, p-state:0x47,
s-pri:0x1000, s-mac:a036.9f02.c138
058681: Dec 8 22:15:51.895: LACP: Part: tlv:2, tlv-len:20, key:0x0, p-pri:0x0, p:0x0, p-state:0xA,
s-pri:0x0, s-mac:0000.0000.0000
058682: Dec 8 22:15:51.895: LACP: col-tlv:3, col-tlv-len:16, col-max-d:0xA
058683: Dec 8 22:15:51.895: LACP: term-tlv:0 termr-tlv-len:0
058684: Dec 8 22:15:51.895: LACP: Gi2/0/10 LACP packet received, processing
058685: Dec 8 22:15:51.895: lacp_rx Gi2/0/10 - rx: during state CURRENT, got event 5(recv_lacpdu)
058686: Dec 8 22:15:51.895: @@@ lacp_rx Gi2/0/10 - rx: CURRENT -> CURRENT
058687: Dec 8 22:15:51.895: LACP: Gi2/0/10 lacp_action_rx_current entered
058688: Dec 8 22:15:51.895: LACP: Gi2/0/10 set to UNSELECTED
058689: Dec 8 22:15:51.895: lacp_mux Gi2/0/10 - mux: during state WAITING, got event 3(unselected)
058690: Dec 8 22:15:51.895: @@@ lacp_mux Gi2/0/10 - mux: WAITING -> DETACHED
058691: Dec 8 22:15:51.895: LACP: Gi2/0/10 lacp_action_mx_detached entered
058692: Dec 8 22:15:51.895: LACP: Gi2/0/10 Resetting hw_sw constraints
058693: Dec 8 22:15:51.895: LACP: lacp_insert_partner_cd_inhibitor: didn't change sync flag.
058694: Dec 8 22:15:51.895: LACP: lacp_send_lacpdu: (Gi2/0/10) About to send the 110 LACPDU
058695: Dec 8 22:15:51.895: LACP :lacp_bugpak: Send LACP-PDU packet via Gi2/0/10
058696: Dec 8 22:15:51.895: LACP : packet size: 124
058697: Dec 8 22:15:51.895: LACP: pdu: subtype: 1, version: 1
058698: Dec 8 22:15:51.895: LACP: Act: tlv:1, tlv-len:20, key:0x7, p-pri:0x8000, p:0x20B, p-state:0x5,
s-pri:0x8000, s-mac:0019.aa89.b580
058699: Dec 8 22:15:51.895: LACP: Part: tlv:2, tlv-len:20, key:0x3E9, p-pri:0x1000, p:0x4, p-state:0x47,
s-pri:0x1000, s-mac:a036.9f02.c138
058700: Dec 8 22:15:51.895: LACP: col-tlv:3, col-tlv-len:16, col-max-d:0x8000
058701: Dec 8 22:15:51.895: LACP: term-tlv:0 termr-tlv-len:0
058702: Dec 8 22:15:51.895: LACP: lacp_write: LACP 124 bytes out Gi2/0/10
058703: Dec 8 22:15:51.895: lacp_handle_standby_port_internal called, depth = 1
058704: Dec 8 22:15:51.895: LACP: recordPDU Gi2/0/10 LACP PDU Rcvd. Partners oper state is hex 47
058705: Dec 8 22:15:51.895: LACP: recordPDU Gi2/0/10 Partner out of sync
058706: Dec 8 22:15:51.895: LACP: Gi2/0/10 Partners oper state is hex 47
058707: Dec 8 22:15:51.895: LACP: timer lacp_c_l(Gi2/0/10) started with interval 90000.
058708: Dec 8 22:15:51.895: LACP: Gi2/0/10 LAG_PARTNER_UP.
058709: Dec 8 22:15:51.895: LACP: Gi2/0/10 LAG unchanged
058710: Dec 8 22:15:51.903: LACP: Gi2/0/10 STANDBY aggregator hex address is 5FB71A4
058711: Dec 8 22:15:51.903: LACP: Gi2/0/10 set to STANDBY
058712: Dec 8 22:15:51.903: lacp_mux Gi2/0/10 - mux: during state DETACHED, got event 2(standby)
058713: Dec 8 22:15:51.903: @@@ lacp_mux Gi2/0/10 - mux: DETACHED -> WAITING
058714: Dec 8 22:15:51.903: LACP: Gi2/0/10 lacp_action_mx_waiting entered
058715: Dec 8 22:15:51.903: @@@ lacp_mux Gi2/0/10 - mux: WAITING -> WAITING
058716: Dec 8 22:15:51.903: lacp_mux Gi2/0/10 - mux: during state WAITING, got event 6(outof_sync) (ignored)
058717: Dec 8 22:15:51.903: LACP :lacp_bugpak: Receive LACP-PDU packet via Gi2/0/11
058718: Dec 8 22:15:51.903: LACP : packet size: 124
058719: Dec 8 22:15:51.903: LACP: pdu: subtype: 1, version: 1
058720: Dec 8 22:15:51.903: LACP: Act: tlv:1, tlv-len:20, key:0x3E9, p-pri:0x1000, p:0x3, p-state:0x47,
s-pri:0x1000, s-mac:a036.9f02.c138
058721: Dec 8 22:15:51.903: LACP: Part: tlv:2, tlv-len:20, key:0x0, p-pri:0x0, p:0x0, p-state:0xA,
s-pri:0x0, s-mac:0000.0000.0000
058722: Dec 8 22:15:51.903: LACP: col-tlv:3, col-tlv-len:16, col-max-d:0xA
058723: Dec 8 22:15:51.903: LACP: term-tlv:0 termr-tlv-len:0
058724: Dec 8 22:15:51.903: LACP: Gi2/0/11 LACP packet received, processing
058725: Dec 8 22:15:51.903: lacp_rx Gi2/0/11 - rx: during state CURRENT, got event 5(recv_lacpdu)
058726: Dec 8 22:15:51.903: @@@ lacp_rx Gi2/0/11 - rx: CURRENT -> CURRENT
058727: Dec 8 22:15:51.903: LACP: Gi2/0/11 lacp_action_rx_current entered
058728: Dec 8 22:15:51.903: LACP: Gi2/0/11 set to UNSELECTED
058729: Dec 8 22:15:51.903: lacp_mux Gi2/0/11 - mux: during state WAITING, got event 3(unselected)
058730: Dec 8 22:15:51.903: @@@ lacp_mux Gi2/0/11 - mux: WAITING -> DETACHED
058731: Dec 8 22:15:51.903: LACP: Gi2/0/11 lacp_action_mx_detached entered
058732: Dec 8 22:15:51.903: LACP: Gi2/0/11 Resetting hw_sw constraints
058733: Dec 8 22:15:51.903: LACP: lacp_insert_partner_cd_inhibitor: didn't change sync flag.
058734: Dec 8 22:15:51.903: LACP: lacp_send_lacpdu: (Gi2/0/11) About to send the 110 LACPDU
058735: Dec 8 22:15:51.903: LACP :lacp_bugpak: Send LACP-PDU packet via Gi2/0/11
058736: Dec 8 22:15:51.903: LACP : packet size: 124
058737: Dec 8 22:15:51.903: LACP: pdu: subtype: 1, version: 1
058738: Dec 8 22:15:51.903: LACP: Act: tlv:1, tlv-len:20, key:0x7, p-pri:0x8000, p:0x20C, p-state:0x5,
s-pri:0x8000, s-mac:0019.aa89.b580
058739: Dec 8 22:15:51.903: LACP: Part: tlv:2, tlv-len:20, key:0x3E9, p-pri:0x1000, p:0x3, p-state:0x47,
s-pri:0x1000, s-mac:a036.9f02.c138


so in short, i see it coming up, shows out of sync, and down it goes...
have tried short times, long times active , passive... nothing will work


Met vriendelijke groet / With kind regards,



               Floris van Essen / Claudia Dufornee
                House of Ancients Amstaffs

               info at houseofancients.nl<mailto:info at houseofancients.nl>
               www.houseofancients.nl


...:: House of Ancients ::...
American Staffordshire Terriers

+31-628-161-350
+31-614-198-389
Het Perk 48
4903 RB
Oosterhout
Netherlands
www.houseofancients.nl<http://www.houseofancients.nl>

...:: House of Ancients ::...
American Staffordshire Terriers

+31-628-161-350
+31-614-198-389
Het Perk 48
4903 RB
Oosterhout
Netherlands
www.houseofancients.nl
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141212/75fcb99b/attachment-0001.html>

From danmcd at omniti.com  Fri Dec 12 16:35:25 2014
From: danmcd at omniti.com (Dan McDonald)
Date: Fri, 12 Dec 2014 11:35:25 -0500
Subject: [OmniOS-discuss] LACP Omnios , igb and C3750
In-Reply-To: <356582D1FC91784992ABB4265A16ED487EF447DC@vEX01.mindstorm-internet.local>
References: <~B548633c30000.5486a0500000.0002.mml.3857832108@vEX01.mindstorm-internet.local>
	<356582D1FC91784992ABB4265A16ED487EF447DC@vEX01.mindstorm-internet.local>
Message-ID: <DD4737A6-D508-4759-8F6D-3869F259A3D2@omniti.com>

Sorry for not replying earlier.

> On Dec 12, 2014, at 10:24 AM, Floris van Essen ..:: House of Ancients Amstafs ::.. <info at houseofancients.nl> wrote:
> 
> Hi All,
>  
> this just keeps getting stranger by the minute.
> after removing the aggr's and setting a static ip on each interface,
> i can ping from the host itself, but not through a wire, direct or through a switch.
> plugged in a new intel i350, same deal... 
> when looking at the driver, i see that the is a update date of november 6....
>  
> does anybody have an idea what could be going on ?

You were smart to reduce it to individual igb links first.

Please remind me what revision of OmniOS you have?

Also, please share the output of:

- dladm show-ether
- dladm show-link
- ifconfig -a
- netstat -rnv

Thanks,
Dan



From info at houseofancients.nl  Fri Dec 12 16:49:26 2014
From: info at houseofancients.nl (Floris van Essen ..:: House of Ancients Amstafs ::..)
Date: Fri, 12 Dec 2014 16:49:26 +0000
Subject: [OmniOS-discuss] LACP Omnios , igb and C3750
In-Reply-To: <DD4737A6-D508-4759-8F6D-3869F259A3D2@omniti.com>
References: <~B548633c30000.5486a0500000.0002.mml.3857832108@vEX01.mindstorm-internet.local>
	<356582D1FC91784992ABB4265A16ED487EF447DC@vEX01.mindstorm-internet.local>
	<DD4737A6-D508-4759-8F6D-3869F259A3D2@omniti.com>
Message-ID: <356582D1FC91784992ABB4265A16ED487EF46909@vEX01.mindstorm-internet.local>

Hi Dann,

No problem, just happy you did :-)

Right, so as might remember I had to reinstall because I was running bloody R11 , but there was a problem upgrading , so just did a fresh install
Had to skip r12 because I simply couldn't install it, as there was a installer issue with r12...

So here we are again, running bloody r13 , fully updated :-)

# dladm show-ether
LINK            PTYPE    STATE    AUTO  SPEED-DUPLEX                    PAUSE
e1000g1         current  up       yes   1G-f                            bi
e1000g0         current  up       yes   1G-f                            bi
igb0            current  up       yes   1G-f                            bi
igb1            current  up       yes   1G-f                            bi
igb2            current  up       yes   1G-f                            bi
igb3            current  up       yes   1G-f                            bi
# dladm show-link
LINK        CLASS     MTU    STATE    BRIDGE     OVER
e1000g1     phys      1500   up       --         --
e1000g0     phys      1500   up       --         --
aggr0       aggr      1500   up       --         e1000g0 e1000g1
igb0        phys      1500   up       --         --
igb1        phys      1500   up       --         --
igb2        phys      1500   up       --         --
igb3        phys      1500   up       --         --
# ifconfig -a
lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1
        inet 127.0.0.1 netmask ff000000
aggr0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2
        inet x.x.x.11 netmask ffffff00 broadcast x.x.x.255
        ether 0:30:48:d5:ec:94
igb0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 3
        inet 192.168.0.1 netmask ffffff00 broadcast 192.168.0.255
        ether a0:36:9f:2:c2:6c
igb1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 4
        inet 192.168.0.2 netmask ffffff00 broadcast 192.168.0.255
        ether a0:36:9f:2:c2:6d
igb2: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 5
        inet 192.168.0.3 netmask ffffff00 broadcast 192.168.0.255
        ether a0:36:9f:2:c2:6e
igb3: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 6
        inet 192.168.0.4 netmask ffffff00 broadcast 192.168.0.255
        ether a0:36:9f:2:c2:6f
lo0: flags=2002000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv6,VIRTUAL> mtu 8252 index 1
        inet6 ::1/128
aggr0: flags=20002000840<RUNNING,MULTICAST,IPv6> mtu 1500 index 2
        inet6 ::/0
        ether 0:30:48:d5:ec:94
igb0: flags=20002000840<RUNNING,MULTICAST,IPv6> mtu 1500 index 3
        inet6 ::/0
        ether a0:36:9f:2:c2:6c
igb1: flags=20002000840<RUNNING,MULTICAST,IPv6> mtu 1500 index 4
        inet6 ::/0
        ether a0:36:9f:2:c2:6d
igb2: flags=20002000840<RUNNING,MULTICAST,IPv6> mtu 1500 index 5
        inet6 ::/0
        ether a0:36:9f:2:c2:6e
igb3: flags=20002000840<RUNNING,MULTICAST,IPv6> mtu 1500 index 6
        inet6 ::/0
        ether a0:36:9f:2:c2:6f
# netstat -rnv

IRE Table: IPv4
  Destination             Mask           Gateway          Device  MTU  Ref Flg  Out  In/Fwd
-------------------- --------------- -------------------- ------ ----- --- --- ----- ------
default              0.0.0.0         x.x.x.254                 0   3 UG   68263      0
5.135.127.96         255.255.255.255 x.x.x.1                   0   2 UHD      2      0
37.187.62.87         255.255.255.255 x.x.x.1                   0   2 UHD     79      0
46.19.141.134        255.255.255.255 x.x.x.1                   0   2 UHD    210      0
46.105.201.50        255.255.255.255 x.x.x.1                   0   2 UHD     35      0
77.250.233.134       255.255.255.255 x.x.x.1                   0   2 UHD     53      0
127.0.0.1            255.255.255.255 127.0.0.1            lo0     8232   2 UH    9921   9921
130.233.43.4         255.255.255.255 x.x.x.1                   0   2 UHD      2      0
192.168.0.0          255.255.255.0   192.168.0.4          igb3    1500   3 U       18      0
192.168.0.0          255.255.255.0   192.168.0.3          igb2    1500   2 U        0      0
192.168.0.0          255.255.255.0   192.168.0.2          igb1    1500   2 U        0      0
192.168.0.0          255.255.255.0   192.168.0.1          igb0    1500   2 U        0      0
x.x.x.0        255.255.255.0   x.x.x.11       aggr0   1500   6 U      680      0
199.15.226.221       255.255.255.255 x.x.x.1                   0   2 UHD      8      0

IRE Table: IPv6
  Destination/Mask            Gateway                    If    MTU  Ref Flags  Out   In/Fwd
--------------------------- --------------------------- ----- ----- --- ----- ------ ------
::1                         ::1                         lo0    8252   2 UH      1118   1118
#


Met vriendelijke groet / With kind regards,



???????????????Floris van Essen 

-----Oorspronkelijk bericht-----
Van: Dan McDonald [mailto:danmcd at omniti.com] 
Verzonden: vrijdag 12 december 2014 17:35
Aan: Floris van Essen ..:: House of Ancients Amstafs ::..
CC: omnios-discuss at lists.omniti.com
Onderwerp: Re: [OmniOS-discuss] LACP Omnios , igb and C3750

Sorry for not replying earlier.

> On Dec 12, 2014, at 10:24 AM, Floris van Essen ..:: House of Ancients Amstafs ::.. <info at houseofancients.nl> wrote:
> 
> Hi All,
>  
> this just keeps getting stranger by the minute.
> after removing the aggr's and setting a static ip on each interface, i 
> can ping from the host itself, but not through a wire, direct or through a switch.
> plugged in a new intel i350, same deal... 
> when looking at the driver, i see that the is a update date of november 6....
>  
> does anybody have an idea what could be going on ?

You were smart to reduce it to individual igb links first.

Please remind me what revision of OmniOS you have?

Also, please share the output of:

- dladm show-ether
- dladm show-link
- ifconfig -a
- netstat -rnv

Thanks,
Dan


...:: House of Ancients ::...
American Staffordshire Terriers

+31-628-161-350
+31-614-198-389
Het Perk 48
4903 RB
Oosterhout
Netherlands
www.houseofancients.nl

From danmcd at omniti.com  Fri Dec 12 16:54:27 2014
From: danmcd at omniti.com (Dan McDonald)
Date: Fri, 12 Dec 2014 11:54:27 -0500
Subject: [OmniOS-discuss] LACP Omnios , igb and C3750
In-Reply-To: <356582D1FC91784992ABB4265A16ED487EF46909@vEX01.mindstorm-internet.local>
References: <~B548633c30000.5486a0500000.0002.mml.3857832108@vEX01.mindstorm-internet.local>
	<356582D1FC91784992ABB4265A16ED487EF447DC@vEX01.mindstorm-internet.local>
	<DD4737A6-D508-4759-8F6D-3869F259A3D2@omniti.com>
	<356582D1FC91784992ABB4265A16ED487EF46909@vEX01.mindstorm-internet.local>
Message-ID: <C77D681D-D52C-4278-9E45-ACDBAE6ED281@omniti.com>


> On Dec 12, 2014, at 11:49 AM, Floris van Essen ..:: House of Ancients Amstafs ::.. <info at houseofancients.nl> wrote:
> 
> Hi Dann,
> 
> No problem, just happy you did :-)
> 
> Right, so as might remember I had to reinstall because I was running bloody R11 , but there was a problem upgrading , so just did a fresh install
> Had to skip r12 because I simply couldn't install it, as there was a installer issue with r12...

Weird.  I JUST updated the 012 install media thanks to illumos #5421, so you may want to try that again.

> So here we are again, running bloody r13 , fully updated :-)
> 
> # dladm show-ether
> LINK            PTYPE    STATE    AUTO  SPEED-DUPLEX                    PAUSE
> e1000g1         current  up       yes   1G-f                            bi
> e1000g0         current  up       yes   1G-f                            bi
> igb0            current  up       yes   1G-f                            bi
> igb1            current  up       yes   1G-f                            bi
> igb2            current  up       yes   1G-f                            bi
> igb3            current  up       yes   1G-f                            bi
> # dladm show-link
> LINK        CLASS     MTU    STATE    BRIDGE     OVER
> e1000g1     phys      1500   up       --         --
> e1000g0     phys      1500   up       --         --
> aggr0       aggr      1500   up       --         e1000g0 e1000g1
> igb0        phys      1500   up       --         --
> igb1        phys      1500   up       --         --
> igb2        phys      1500   up       --         --
> igb3        phys      1500   up       --         --

Tis all looks sane.  You're showing 1Gig, full dupliex, and up.  This seems sane.

> # ifconfig -a
> lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1
>        inet 127.0.0.1 netmask ff000000
> aggr0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2
>        inet x.x.x.11 netmask ffffff00 broadcast x.x.x.255
>        ether 0:30:48:d5:ec:94

And you have no problems pinging x.x.x.0/24 addresses?  Or do you?

> igb0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 3
>        inet 192.168.0.1 netmask ffffff00 broadcast 192.168.0.255
>        ether a0:36:9f:2:c2:6c
> igb1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 4
>        inet 192.168.0.2 netmask ffffff00 broadcast 192.168.0.255
>        ether a0:36:9f:2:c2:6d
> igb2: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 5
>        inet 192.168.0.3 netmask ffffff00 broadcast 192.168.0.255
>        ether a0:36:9f:2:c2:6e
> igb3: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 6
>        inet 192.168.0.4 netmask ffffff00 broadcast 192.168.0.255
>        ether a0:36:9f:2:c2:6f

Now I can totally see some potential confusion here.  You've four addrs all in the same netstack and all with the same prefix.  Now granted, they're different MAC addresses, but packets coming in on one interface may have their return traffic going out another.

If these are the problem, try using just one (igb0) for starters and "ifconfig igbX down" for 1, 2, and 3.

Also, I need to ask --> are either of your e1000g0's shared IPMI/host links?  If so, disable the sharing feature in the BIOS.  We don't cope with shared-with-IPMI NICs.

Dan


From info at houseofancients.nl  Fri Dec 12 17:10:36 2014
From: info at houseofancients.nl (Floris van Essen ..:: House of Ancients Amstafs ::..)
Date: Fri, 12 Dec 2014 17:10:36 +0000
Subject: [OmniOS-discuss] LACP Omnios , igb and C3750
In-Reply-To: <C77D681D-D52C-4278-9E45-ACDBAE6ED281@omniti.com>
References: <~B548633c30000.5486a0500000.0002.mml.3857832108@vEX01.mindstorm-internet.local>
	<356582D1FC91784992ABB4265A16ED487EF447DC@vEX01.mindstorm-internet.local>
	<DD4737A6-D508-4759-8F6D-3869F259A3D2@omniti.com>
	<356582D1FC91784992ABB4265A16ED487EF46909@vEX01.mindstorm-internet.local>
	<C77D681D-D52C-4278-9E45-ACDBAE6ED281@omniti.com>
Message-ID: <356582D1FC91784992ABB4265A16ED487EF46934@vEX01.mindstorm-internet.local>






> On Dec 12, 2014, at 11:49 AM, Floris van Essen ..:: House of Ancients Amstafs ::.. <info at houseofancients.nl> wrote:
> 
> Hi Dann,
> 
> No problem, just happy you did :-)
> 
> Right, so as might remember I had to reinstall because I was running 
> bloody R11 , but there was a problem upgrading , so just did a fresh install Had to skip r12 because I simply couldn't install it, as there was a installer issue with r12...

>> Weird.  I JUST updated the 012 install media thanks to illumos #5421, so you may want to try that again.

Did the installation 2 weeks ago, so guessing that that was before you did the changes.
No problem, I'll update to stable next release :-)

> So here we are again, running bloody r13 , fully updated :-)
> 
> # dladm show-ether
> LINK            PTYPE    STATE    AUTO  SPEED-DUPLEX                    PAUSE
> e1000g1         current  up       yes   1G-f                            bi
> e1000g0         current  up       yes   1G-f                            bi
> igb0            current  up       yes   1G-f                            bi
> igb1            current  up       yes   1G-f                            bi
> igb2            current  up       yes   1G-f                            bi
> igb3            current  up       yes   1G-f                            bi
> # dladm show-link
> LINK        CLASS     MTU    STATE    BRIDGE     OVER
> e1000g1     phys      1500   up       --         --
> e1000g0     phys      1500   up       --         --
> aggr0       aggr      1500   up       --         e1000g0 e1000g1
> igb0        phys      1500   up       --         --
> igb1        phys      1500   up       --         --
> igb2        phys      1500   up       --         --
> igb3        phys      1500   up       --         --

>>Tis all looks sane.  You're showing 1Gig, full dupliex, and up.  This seems sane.

> # ifconfig -a
> lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1
>        inet 127.0.0.1 netmask ff000000
> aggr0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2
>        inet x.x.x.11 netmask ffffff00 broadcast x.x.x.255
>        ether 0:30:48:d5:ec:94

And you have no problems pinging x.x.x.0/24 addresses?  Or do you?

Non, actually connecting on that interface with ssh

> igb0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 3
>        inet 192.168.0.1 netmask ffffff00 broadcast 192.168.0.255
>        ether a0:36:9f:2:c2:6c
> igb1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 4
>        inet 192.168.0.2 netmask ffffff00 broadcast 192.168.0.255
>        ether a0:36:9f:2:c2:6d
> igb2: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 5
>        inet 192.168.0.3 netmask ffffff00 broadcast 192.168.0.255
>        ether a0:36:9f:2:c2:6e
> igb3: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 6
>        inet 192.168.0.4 netmask ffffff00 broadcast 192.168.0.255
>        ether a0:36:9f:2:c2:6f

>>Now I can totally see some potential confusion here.  You've four addrs all in the same netstack and all with the same prefix.  Now granted, they're different MAC addresses, but packets coming in on one interface may have their return traffic going out >>another.

>>If these are the problem, try using just one (igb0) for starters and "ifconfig igbX down" for 1, 2, and 3.

That is exactly what i did.. for each test brought the others down
Just to add, when doing a  SH ARP on my cisco's, don't see the macs popping up, which is even more worring , no arps on a switch usually is very bad news

>>Also, I need to ask --> are either of your e1000g0's shared IPMI/host links?  If so, disable the sharing feature in the BIOS.  We don't cope with shared-with-IPMI NICs.

They are not... besides.. the e1000G's ( and their aggr) are working fully as expected.
To make it more interesting.. this was the same setup as I had on r11 , and then it did seem to work 100%

Dan

...:: House of Ancients ::...
American Staffordshire Terriers

+31-628-161-350
+31-614-198-389
Het Perk 48
4903 RB
Oosterhout
Netherlands
www.houseofancients.nl

From danmcd at omniti.com  Fri Dec 12 17:21:24 2014
From: danmcd at omniti.com (Dan McDonald)
Date: Fri, 12 Dec 2014 12:21:24 -0500
Subject: [OmniOS-discuss] LACP Omnios , igb and C3750
In-Reply-To: <356582D1FC91784992ABB4265A16ED487EF46934@vEX01.mindstorm-internet.local>
References: <~B548633c30000.5486a0500000.0002.mml.3857832108@vEX01.mindstorm-internet.local>
	<356582D1FC91784992ABB4265A16ED487EF447DC@vEX01.mindstorm-internet.local>
	<DD4737A6-D508-4759-8F6D-3869F259A3D2@omniti.com>
	<356582D1FC91784992ABB4265A16ED487EF46909@vEX01.mindstorm-internet.local>
	<C77D681D-D52C-4278-9E45-ACDBAE6ED281@omniti.com>
	<356582D1FC91784992ABB4265A16ED487EF46934@vEX01.mindstorm-internet.local>
Message-ID: <F928B2C6-2760-4FCB-B368-81F840F61A03@omniti.com>

> 
>>> If these are the problem, try using just one (igb0) for starters and "ifconfig igbX down" for 1, 2, and 3.
> 
> That is exactly what i did.. for each test brought the others down
> Just to add, when doing a  SH ARP on my cisco's, don't see the macs popping up, which is even more worring , no arps on a switch usually is very bad news

And you've eliminated the switch as a problem?  I ask because of this recent push (don't let the commit date fool you, this went in in November):

commit b607c8a3bdb27d4fde6e3fc4bb6617a1d91bdca7
Author: Keith M Wesolowski <wesolows at foobazco.org>
Date:   Tue Aug 5 22:08:11 2014 +0000

    5337 igb/ixgbe mishandle raw packets if cable problem
    Reviewed by: Robert Mustacchi <rm at joyent.com>
    Reviewed by: Dan McDonald <danmcd at nexenta.com>
    Reviewed by: Jason King <jason.brian.king at gmail.com>
    Reviewed by: Garrett D'Amore <garrett at damore.org>
    Approved by: Richard Lowe <richlowe at richlowe.net>

 usr/src/uts/common/io/igb/igb_tx.c     | 10 +++++++---
 usr/src/uts/common/io/ixgbe/ixgbe_tx.c |  6 ++++--
 2 files changed, 11 insertions(+), 5 deletions(-)

This is the ONLY change since the February of 2014 support for I354 that's gone into the Intel GigE NIC code.

>>> Also, I need to ask --> are either of your e1000g0's shared IPMI/host links?  If so, disable the sharing feature in the BIOS.  We don't cope with shared-with-IPMI NICs.
> 
> They are not... besides.. the e1000G's ( and their aggr) are working fully as expected.
> To make it more interesting.. this was the same setup as I had on r11 , and then it did seem to work 100%

The only difference between 011 & 012 and 013 is the above commit.  You *could*, if you're feeling adventurous, swap out the "igb" driver in 013 with one from 012 or 011.  And if you unplumb all of the igbs (both in IPv4 and IPv6), you MAY be able to do it without rebooting.

Do you know which PCI ID your igb (it's an I350, right?) board is?  "prtconf -d | grep -i network" should be a quick way to show me that.

Dan


From info at houseofancients.nl  Fri Dec 12 17:27:42 2014
From: info at houseofancients.nl (Floris van Essen ..:: House of Ancients Amstafs ::..)
Date: Fri, 12 Dec 2014 17:27:42 +0000
Subject: [OmniOS-discuss] LACP Omnios , igb and C3750
In-Reply-To: <F928B2C6-2760-4FCB-B368-81F840F61A03@omniti.com>
References: <~B548633c30000.5486a0500000.0002.mml.3857832108@vEX01.mindstorm-internet.local>
	<356582D1FC91784992ABB4265A16ED487EF447DC@vEX01.mindstorm-internet.local>
	<DD4737A6-D508-4759-8F6D-3869F259A3D2@omniti.com>
	<356582D1FC91784992ABB4265A16ED487EF46909@vEX01.mindstorm-internet.local>
	<C77D681D-D52C-4278-9E45-ACDBAE6ED281@omniti.com>
	<356582D1FC91784992ABB4265A16ED487EF46934@vEX01.mindstorm-internet.local>
	<F928B2C6-2760-4FCB-B368-81F840F61A03@omniti.com>
Message-ID: <356582D1FC91784992ABB4265A16ED487EF46973@vEX01.mindstorm-internet.local>


> 
>>> If these are the problem, try using just one (igb0) for starters and "ifconfig igbX down" for 1, 2, and 3.
> 
> That is exactly what i did.. for each test brought the others down 
> Just to add, when doing a  SH ARP on my cisco's, don't see the macs 
> popping up, which is even more worring , no arps on a switch usually 
> is very bad news

And you've eliminated the switch as a problem?  I ask because of this recent push (don't let the commit date fool you, this went in in November):

Yes... using the e1000g's and the same switch/etherchannel works flawless, also, using one of my vmware machines, and testing the same etherchannel works perfect.
Beside this is a cisco c3750 e, running 12.2 firmware, which is about as proven as the way to rome :-)

commit b607c8a3bdb27d4fde6e3fc4bb6617a1d91bdca7
Author: Keith M Wesolowski <wesolows at foobazco.org>
Date:   Tue Aug 5 22:08:11 2014 +0000

    5337 igb/ixgbe mishandle raw packets if cable problem
    Reviewed by: Robert Mustacchi <rm at joyent.com>
    Reviewed by: Dan McDonald <danmcd at nexenta.com>
    Reviewed by: Jason King <jason.brian.king at gmail.com>
    Reviewed by: Garrett D'Amore <garrett at damore.org>
    Approved by: Richard Lowe <richlowe at richlowe.net>

 usr/src/uts/common/io/igb/igb_tx.c     | 10 +++++++---
 usr/src/uts/common/io/ixgbe/ixgbe_tx.c |  6 ++++--
 2 files changed, 11 insertions(+), 5 deletions(-)

This is the ONLY change since the February of 2014 support for I354 that's gone into the Intel GigE NIC code.

>>> Also, I need to ask --> are either of your e1000g0's shared IPMI/host links?  If so, disable the sharing feature in the BIOS.  We don't cope with shared-with-IPMI NICs.
> 
> They are not... besides.. the e1000G's ( and their aggr) are working fully as expected.
> To make it more interesting.. this was the same setup as I had on r11 
> , and then it did seem to work 100%

The only difference between 011 & 012 and 013 is the above commit.  You *could*, if you're feeling adventurous, swap out the "igb" driver in 013 with one from 012 or 011.  And if you unplumb all of the igbs (both in IPv4 and IPv6), you MAY be able to do it without rebooting.

As i can't use the machine for it's primairy function ( ISCSI target utilizing those i350's), rebooting it's any issue.
Just not sure as HOW to change that driver, so would be great if you'd have some pointers

Do you know which PCI ID your igb (it's an I350, right?) board is?  "prtconf -d | grep -i network" should be a quick way to show me that.

pci8086,1 (pciex8086,1521) [Intel Corporation I350 Gigabit Network Connection], instance #0
            pci8086,1 (pciex8086,1521) [Intel Corporation I350 Gigabit Network Connection], instance #1
            pci8086,1 (pciex8086,1521) [Intel Corporation I350 Gigabit Network Connection], instance #2
            pci8086,1 (pciex8086,1521) [Intel Corporation I350 Gigabit Network Connection], instance #3


floris
...:: House of Ancients ::...
American Staffordshire Terriers

+31-628-161-350
+31-614-198-389
Het Perk 48
4903 RB
Oosterhout
Netherlands
www.houseofancients.nl

From danmcd at omniti.com  Fri Dec 12 17:32:04 2014
From: danmcd at omniti.com (Dan McDonald)
Date: Fri, 12 Dec 2014 12:32:04 -0500
Subject: [OmniOS-discuss] LACP Omnios , igb and C3750
In-Reply-To: <356582D1FC91784992ABB4265A16ED487EF46973@vEX01.mindstorm-internet.local>
References: <~B548633c30000.5486a0500000.0002.mml.3857832108@vEX01.mindstorm-internet.local>
	<356582D1FC91784992ABB4265A16ED487EF447DC@vEX01.mindstorm-internet.local>
	<DD4737A6-D508-4759-8F6D-3869F259A3D2@omniti.com>
	<356582D1FC91784992ABB4265A16ED487EF46909@vEX01.mindstorm-internet.local>
	<C77D681D-D52C-4278-9E45-ACDBAE6ED281@omniti.com>
	<356582D1FC91784992ABB4265A16ED487EF46934@vEX01.mindstorm-internet.local>
	<F928B2C6-2760-4FCB-B368-81F840F61A03@omniti.com>
	<356582D1FC91784992ABB4265A16ED487EF46973@vEX01.mindstorm-internet.local>
Message-ID: <037DE647-E2D1-4186-B2F2-39B0715BF509@omniti.com>

> 
> And you've eliminated the switch as a problem?  I ask because of this recent push (don't let the commit date fool you, this went in in November):
> 
> Yes... using the e1000g's and the same switch/etherchannel works flawless, also, using one of my vmware machines, and testing the same etherchannel works perfect.
> Beside this is a cisco c3750 e, running 12.2 firmware, which is about as proven as the way to rome :-)

Okay, if the e1000gs work, then fine.

I'm assuming you've checked out the cables as well...

> 
> As i can't use the machine for it's primairy function ( ISCSI target utilizing those i350's), rebooting it's any issue.
> Just not sure as HOW to change that driver, so would be great if you'd have some pointers

You're running bloody, so you can do these normally not-recommended-for-users steps:

1.) Backup the original igb:   cp /kernel/drv/amd64/igb <somewhere>

2.) Find an igb from 011 or 012.  Exercise left to the reader.  If you have an old boot-environment, "beadm mount" it and grab it from there.

3.) Replace /kernel/drv/amd64/igb with the older version.

4.) Utter "bootadm update-archive" to make sure the replacement igb is there.

5.) Reboot.


Dan


From info at houseofancients.nl  Fri Dec 12 17:36:02 2014
From: info at houseofancients.nl (Floris van Essen ..:: House of Ancients Amstafs ::..)
Date: Fri, 12 Dec 2014 17:36:02 +0000
Subject: [OmniOS-discuss] LACP Omnios , igb and C3750
In-Reply-To: <037DE647-E2D1-4186-B2F2-39B0715BF509@omniti.com>
References: <~B548633c30000.5486a0500000.0002.mml.3857832108@vEX01.mindstorm-internet.local>
	<356582D1FC91784992ABB4265A16ED487EF447DC@vEX01.mindstorm-internet.local>
	<DD4737A6-D508-4759-8F6D-3869F259A3D2@omniti.com>
	<356582D1FC91784992ABB4265A16ED487EF46909@vEX01.mindstorm-internet.local>
	<C77D681D-D52C-4278-9E45-ACDBAE6ED281@omniti.com>
	<356582D1FC91784992ABB4265A16ED487EF46934@vEX01.mindstorm-internet.local>
	<F928B2C6-2760-4FCB-B368-81F840F61A03@omniti.com>
	<356582D1FC91784992ABB4265A16ED487EF46973@vEX01.mindstorm-internet.local>
	<037DE647-E2D1-4186-B2F2-39B0715BF509@omniti.com>
Message-ID: <356582D1FC91784992ABB4265A16ED487EF46A07@vEX01.mindstorm-internet.local>


> 
> And you've eliminated the switch as a problem?  I ask because of this recent push (don't let the commit date fool you, this went in in November):
> 
> Yes... using the e1000g's and the same switch/etherchannel works flawless, also, using one of my vmware machines, and testing the same etherchannel works perfect.
> Beside this is a cisco c3750 e, running 12.2 firmware, which is about as proven as the way to rome :-)

Okay, if the e1000gs work, then fine.

I'm assuming you've checked out the cables as well...

New cables, cables tested with e1000 also perfect

> 
> As i can't use the machine for it's primairy function ( ISCSI target utilizing those i350's), rebooting it's any issue.
> Just not sure as HOW to change that driver, so would be great if you'd have some pointers

You're running bloody, so you can do these normally not-recommended-for-users steps:

1.) Backup the original igb:   cp /kernel/drv/amd64/igb <somewhere>

2.) Find an igb from 011 or 012.  Exercise left to the reader.  If you have an old boot-environment, "beadm mount" it and grab it from there.

As i had to do a reinstall , there's no r11 stuff left.
R12 was never installed... 

Is there a way to download this somewere from github or maybe some source ?

3.) Replace /kernel/drv/amd64/igb with the older version.

4.) Utter "bootadm update-archive" to make sure the replacement igb is there.

5.) Reboot.


Dan

...:: House of Ancients ::...
American Staffordshire Terriers

+31-628-161-350
+31-614-198-389
Het Perk 48
4903 RB
Oosterhout
Netherlands
www.houseofancients.nl

From info at houseofancients.nl  Fri Dec 12 19:05:23 2014
From: info at houseofancients.nl (Floris van Essen ..:: House of Ancients Amstafs ::..)
Date: Fri, 12 Dec 2014 19:05:23 +0000
Subject: [OmniOS-discuss] LACP Omnios , igb and C3750
In-Reply-To: <037DE647-E2D1-4186-B2F2-39B0715BF509@omniti.com>
References: <~B548633c30000.5486a0500000.0002.mml.3857832108@vEX01.mindstorm-internet.local>
	<356582D1FC91784992ABB4265A16ED487EF447DC@vEX01.mindstorm-internet.local>
	<DD4737A6-D508-4759-8F6D-3869F259A3D2@omniti.com>
	<356582D1FC91784992ABB4265A16ED487EF46909@vEX01.mindstorm-internet.local>
	<C77D681D-D52C-4278-9E45-ACDBAE6ED281@omniti.com>
	<356582D1FC91784992ABB4265A16ED487EF46934@vEX01.mindstorm-internet.local>
	<F928B2C6-2760-4FCB-B368-81F840F61A03@omniti.com>
	<356582D1FC91784992ABB4265A16ED487EF46973@vEX01.mindstorm-internet.local>
	<037DE647-E2D1-4186-B2F2-39B0715BF509@omniti.com>
Message-ID: <356582D1FC91784992ABB4265A16ED487EF47A6B@vEX01.mindstorm-internet.local>

Is there anyone running i350's and r10,r11 or r12 that would be able to help out here ?
Would like the igb file, for testing


Met vriendelijke groet / With kind regards,



???????????????Floris van Essen 

-----Oorspronkelijk bericht-----
Van: Dan McDonald [mailto:danmcd at omniti.com] 
Verzonden: vrijdag 12 december 2014 18:32
Aan: Floris van Essen ..:: House of Ancients Amstafs ::..
CC: omnios-discuss at lists.omniti.com
Onderwerp: Re: [OmniOS-discuss] LACP Omnios , igb and C3750

> 
> And you've eliminated the switch as a problem?  I ask because of this recent push (don't let the commit date fool you, this went in in November):
> 
> Yes... using the e1000g's and the same switch/etherchannel works flawless, also, using one of my vmware machines, and testing the same etherchannel works perfect.
> Beside this is a cisco c3750 e, running 12.2 firmware, which is about as proven as the way to rome :-)

Okay, if the e1000gs work, then fine.

I'm assuming you've checked out the cables as well...

> 
> As i can't use the machine for it's primairy function ( ISCSI target utilizing those i350's), rebooting it's any issue.
> Just not sure as HOW to change that driver, so would be great if you'd have some pointers

You're running bloody, so you can do these normally not-recommended-for-users steps:

1.) Backup the original igb:   cp /kernel/drv/amd64/igb <somewhere>

2.) Find an igb from 011 or 012.  Exercise left to the reader.  If you have an old boot-environment, "beadm mount" it and grab it from there.

3.) Replace /kernel/drv/amd64/igb with the older version.

4.) Utter "bootadm update-archive" to make sure the replacement igb is there.

5.) Reboot.


Dan

...:: House of Ancients ::...
American Staffordshire Terriers

+31-628-161-350
+31-614-198-389
Het Perk 48
4903 RB
Oosterhout
Netherlands
www.houseofancients.nl

From eric.sproul at circonus.com  Fri Dec 12 19:15:18 2014
From: eric.sproul at circonus.com (Eric Sproul)
Date: Fri, 12 Dec 2014 14:15:18 -0500
Subject: [OmniOS-discuss] LACP Omnios , igb and C3750
In-Reply-To: <356582D1FC91784992ABB4265A16ED487EF47A6B@vEX01.mindstorm-internet.local>
References: <~B548633c30000.5486a0500000.0002.mml.3857832108@vEX01.mindstorm-internet.local>
	<356582D1FC91784992ABB4265A16ED487EF447DC@vEX01.mindstorm-internet.local>
	<DD4737A6-D508-4759-8F6D-3869F259A3D2@omniti.com>
	<356582D1FC91784992ABB4265A16ED487EF46909@vEX01.mindstorm-internet.local>
	<C77D681D-D52C-4278-9E45-ACDBAE6ED281@omniti.com>
	<356582D1FC91784992ABB4265A16ED487EF46934@vEX01.mindstorm-internet.local>
	<F928B2C6-2760-4FCB-B368-81F840F61A03@omniti.com>
	<356582D1FC91784992ABB4265A16ED487EF46973@vEX01.mindstorm-internet.local>
	<037DE647-E2D1-4186-B2F2-39B0715BF509@omniti.com>
	<356582D1FC91784992ABB4265A16ED487EF47A6B@vEX01.mindstorm-internet.local>
Message-ID: <CAO8hXRB141AoSttXbYF_igw-fbNvg9Kq1L51EKWt5P1rRQnKCg@mail.gmail.com>

On Fri, Dec 12, 2014 at 2:05 PM, Floris van Essen ..:: House of
Ancients Amstafs ::.. <info at houseofancients.nl> wrote:
> Is there anyone running i350's and r10,r11 or r12 that would be able to help out here ?
> Would like the igb file, for testing

Hi Floris,
You can fetch the file directly from the pkg server, without having
any additional publisher URLs configured:

$ pkg contents -mr -g http://pkg.omniti.com/omnios/r151012/
driver/network/igb | grep kernel/drv/amd64/igb

file a4f79660b1593166be621166e8e1afc9b45af993
chash=c95cbda5496b6643ae331e0a1630eb74b55200c4 elfarch=i386 elfbits=64
elfhash=c10bc6c62e118456093689921d7794ae60684ae1 group=sys mode=0755
owner=root path=kernel/drv/amd64/igb pkg.csize=117774 pkg.size=384592
reboot-needed=true variant.opensolaris.zone=global

The second field of the result is a SHA1 hash of the file which is its
index in the repo, so you can:

curl http://pkg.omniti.com/omnios/r151012/file/1/a4f79660b1593166be621166e8e1afc9b45af993
| gzip -dc > igb.r151012.amd64

and verify:

$ sha1sum igb.r151012.amd64

a4f79660b1593166be621166e8e1afc9b45af993  igb.r151012.amd64

Then copy that file into place as Dan described.

See http://omnios.omniti.com/wiki.php/FetchIPSFilesWithoutPkg for details.

Eric

From danmcd at omniti.com  Fri Dec 12 19:23:00 2014
From: danmcd at omniti.com (Dan McDonald)
Date: Fri, 12 Dec 2014 14:23:00 -0500
Subject: [OmniOS-discuss] LACP Omnios , igb and C3750
In-Reply-To: <CAO8hXRB141AoSttXbYF_igw-fbNvg9Kq1L51EKWt5P1rRQnKCg@mail.gmail.com>
References: <~B548633c30000.5486a0500000.0002.mml.3857832108@vEX01.mindstorm-internet.local>
	<356582D1FC91784992ABB4265A16ED487EF447DC@vEX01.mindstorm-internet.local>
	<DD4737A6-D508-4759-8F6D-3869F259A3D2@omniti.com>
	<356582D1FC91784992ABB4265A16ED487EF46909@vEX01.mindstorm-internet.local>
	<C77D681D-D52C-4278-9E45-ACDBAE6ED281@omniti.com>
	<356582D1FC91784992ABB4265A16ED487EF46934@vEX01.mindstorm-internet.local>
	<F928B2C6-2760-4FCB-B368-81F840F61A03@omniti.com>
	<356582D1FC91784992ABB4265A16ED487EF46973@vEX01.mindstorm-internet.local>
	<037DE647-E2D1-4186-B2F2-39B0715BF509@omniti.com>
	<356582D1FC91784992ABB4265A16ED487EF47A6B@vEX01.mindstorm-internet.local>
	<CAO8hXRB141AoSttXbYF_igw-fbNvg9Kq1L51EKWt5P1rRQnKCg@mail.gmail.com>
Message-ID: <827529EA-5494-45FD-9181-B54B3553C6B4@omniti.com>

I'd sent him an igb binary from 012 offline.

Floris --> if you didn't get it, maybe your mail server filtered me out?!

Dan


From info at houseofancients.nl  Fri Dec 12 20:06:40 2014
From: info at houseofancients.nl (Floris van Essen ..:: House of Ancients Amstafs ::..)
Date: Fri, 12 Dec 2014 20:06:40 +0000
Subject: [OmniOS-discuss] LACP Omnios , igb and C3750
In-Reply-To: <CAO8hXRB141AoSttXbYF_igw-fbNvg9Kq1L51EKWt5P1rRQnKCg@mail.gmail.com>
References: <~B548633c30000.5486a0500000.0002.mml.3857832108@vEX01.mindstorm-internet.local>
	<356582D1FC91784992ABB4265A16ED487EF447DC@vEX01.mindstorm-internet.local>
	<DD4737A6-D508-4759-8F6D-3869F259A3D2@omniti.com>
	<356582D1FC91784992ABB4265A16ED487EF46909@vEX01.mindstorm-internet.local>
	<C77D681D-D52C-4278-9E45-ACDBAE6ED281@omniti.com>
	<356582D1FC91784992ABB4265A16ED487EF46934@vEX01.mindstorm-internet.local>
	<F928B2C6-2760-4FCB-B368-81F840F61A03@omniti.com>
	<356582D1FC91784992ABB4265A16ED487EF46973@vEX01.mindstorm-internet.local>
	<037DE647-E2D1-4186-B2F2-39B0715BF509@omniti.com>
	<356582D1FC91784992ABB4265A16ED487EF47A6B@vEX01.mindstorm-internet.local>
	<CAO8hXRB141AoSttXbYF_igw-fbNvg9Kq1L51EKWt5P1rRQnKCg@mail.gmail.com>
Message-ID: <356582D1FC91784992ABB4265A16ED487EF47B49@vEX01.mindstorm-internet.local>

Right,

Wish i had better news, copied the file i got from Dan, create boot archive and rebooted
Disable on the switch the ports not needed for first test

Checked the boot log to see if it didn?t show funny stuff, of course I saw the e1000g's and their aggr passing correctly :
Dec 12 20:40:43 PSD01 genunix: [ID 805372 kern.info] pcplusmp: pciex8086,1096 (e1000g) instance 0 irq 0x33 vector 0x60 ioapic 0xff intin 0xff is bound to cpu 3
Dec 12 20:40:43 PSD01 genunix: [ID 469746 kern.info] NOTICE: e1000g0 registered
Dec 12 20:40:43 PSD01 genunix: [ID 805372 kern.info] pcplusmp: pciex8086,1096 (e1000g) instance 1 irq 0x34 vector 0x61 ioapic 0xff intin 0xff is bound to cpu 0
Dec 12 20:40:43 PSD01 genunix: [ID 469746 kern.info] NOTICE: e1000g1 registered
Dec 12 20:40:43 PSD01 genunix: [ID 469746 kern.info] NOTICE: aggr1000 registered

Then the IGB's :

Dec 12 20:40:50 PSD01 genunix: [ID 805372 kern.info] pcplusmp: pciex8086,1521 (igb) instance 0 irq 0x37 vector 0x62 ioapic 0xff intin 0xff is bound to cpu 3
Dec 12 20:40:50 PSD01 genunix: [ID 805372 kern.info] pcplusmp: pciex8086,1521 (igb) instance 0 irq 0x38 vector 0x63 ioapic 0xff intin 0xff is bound to cpu 0
Dec 12 20:40:50 PSD01 genunix: [ID 469746 kern.info] NOTICE: igb0 registered
Dec 12 20:40:50 PSD01 genunix: [ID 611667 kern.info] NOTICE: igb0: igb 2.3.8-ish
Dec 12 20:40:50 PSD01 genunix: [ID 805372 kern.info] pcplusmp: pciex8086,1521 (igb) instance 1 irq 0x39 vector 0x64 ioapic 0xff intin 0xff is bound to cpu 1
Dec 12 20:40:50 PSD01 genunix: [ID 805372 kern.info] pcplusmp: pciex8086,1521 (igb) instance 1 irq 0x3a vector 0x65 ioapic 0xff intin 0xff is bound to cpu 2
Dec 12 20:40:50 PSD01 genunix: [ID 469746 kern.info] NOTICE: igb1 registered
Dec 12 20:40:50 PSD01 genunix: [ID 611667 kern.info] NOTICE: igb1: igb 2.3.8-ish
Dec 12 20:40:50 PSD01 genunix: [ID 805372 kern.info] pcplusmp: pciex8086,1521 (igb) instance 2 irq 0x3b vector 0x66 ioapic 0xff intin 0xff is bound to cpu 3
Dec 12 20:40:50 PSD01 genunix: [ID 805372 kern.info] pcplusmp: pciex8086,1521 (igb) instance 2 irq 0x3c vector 0x67 ioapic 0xff intin 0xff is bound to cpu 0
Dec 12 20:40:50 PSD01 genunix: [ID 469746 kern.info] NOTICE: igb2 registered
Dec 12 20:40:50 PSD01 genunix: [ID 611667 kern.info] NOTICE: igb2: igb 2.3.8-ish
Dec 12 20:40:50 PSD01 genunix: [ID 805372 kern.info] pcplusmp: pciex8086,1521 (igb) instance 3 irq 0x3d vector 0x68 ioapic 0xff intin 0xff is bound to cpu 1
Dec 12 20:40:50 PSD01 genunix: [ID 805372 kern.info] pcplusmp: pciex8086,1521 (igb) instance 3 irq 0x3e vector 0x69 ioapic 0xff intin 0xff is bound to cpu 2
Dec 12 20:40:50 PSD01 genunix: [ID 469746 kern.info] NOTICE: igb3 registered
Dec 12 20:40:50 PSD01 genunix: [ID 611667 kern.info] NOTICE: igb3: igb 2.3.8-ish

Seems 100% ok so After that removed if's and recreated just one of them (IGB3) :

ipadm create-addr -T static -a 192.168.0.1/24 igb3/v4

let's check :
# dladm show-ether
LINK            PTYPE    STATE    AUTO  SPEED-DUPLEX                    PAUSE
e1000g1         current  up       yes   1G-f                            bi
e1000g0         current  up       yes   1G-f                            bi
igb0            current  down     yes   0M-h                            bi
igb1            current  unknown  yes   0M-h                            bi
igb2            current  unknown  yes   0M-h                            bi
igb3            current  up       yes   1G-f                            bi

awesome

root at PSD01:/root# dladm show-link
LINK        CLASS     MTU    STATE    BRIDGE     OVER
e1000g1     phys      1500   up       --         --
e1000g0     phys      1500   up       --         --
aggr0       aggr      1500   up       --         e1000g0 e1000g1
igb0        phys      1500   down     --         --
igb1        phys      1500   unknown  --         --
igb2        phys      1500   unknown  --         --
igb3        phys      1500   up       --         --

awesome

root at PSD01:/root# ifconfig -a
lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1
        inet 127.0.0.1 netmask ff000000
aggr0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2
        inet x.x.x.11 netmask ffffff00 broadcast 192.168.254.255
        ether 0:30:48:d5:ec:94
igb3: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 10
        inet 192.168.0.1 netmask ffffff00 broadcast 192.168.0.255
        ether a0:36:9f:2:c2:6f
lo0: flags=2002000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv6,VIRTUAL> mtu 8252 index 1
        inet6 ::1/128
aggr0: flags=20002000840<RUNNING,MULTICAST,IPv6> mtu 1500 index 2
        inet6 ::/0
        ether 0:30:48:d5:ec:94
igb3: flags=20002000840<RUNNING,MULTICAST,IPv6> mtu 1500 index 10
        inet6 ::/0
        ether a0:36:9f:2:c2:6f

awesome 

root at PSD01:/root# ping 192.168.0.1
192.168.0.1 is alive

core-stack#sh arp | inc a036.9f:2.c26f
core-stack#
oh oh 

let's see if we can ping another host in the same subnet :

root at PSD01:/root# ping 192.168.0.5
no answer from 192.168.0.5
root at PSD01:/root#

let check the arp again :

core-stack#sh arp | inc a036.9f:2.c26f
core-stack#

so I'm (again ) baffled 

Met vriendelijke groet 

-----Oorspronkelijk bericht-----
Van: Eric Sproul [mailto:eric.sproul at circonus.com] 
Verzonden: vrijdag 12 december 2014 20:15
Aan: Floris van Essen ..:: House of Ancients Amstafs ::..
CC: Dan McDonald; omnios-discuss at lists.omniti.com
Onderwerp: Re: [OmniOS-discuss] LACP Omnios , igb and C3750

On Fri, Dec 12, 2014 at 2:05 PM, Floris van Essen ..:: House of Ancients Amstafs ::.. <info at houseofancients.nl> wrote:
> Is there anyone running i350's and r10,r11 or r12 that would be able to help out here ?
> Would like the igb file, for testing

Hi Floris,
You can fetch the file directly from the pkg server, without having any additional publisher URLs configured:

$ pkg contents -mr -g http://pkg.omniti.com/omnios/r151012/
driver/network/igb | grep kernel/drv/amd64/igb

file a4f79660b1593166be621166e8e1afc9b45af993
chash=c95cbda5496b6643ae331e0a1630eb74b55200c4 elfarch=i386 elfbits=64
elfhash=c10bc6c62e118456093689921d7794ae60684ae1 group=sys mode=0755 owner=root path=kernel/drv/amd64/igb pkg.csize=117774 pkg.size=384592 reboot-needed=true variant.opensolaris.zone=global

The second field of the result is a SHA1 hash of the file which is its index in the repo, so you can:

curl http://pkg.omniti.com/omnios/r151012/file/1/a4f79660b1593166be621166e8e1afc9b45af993
| gzip -dc > igb.r151012.amd64

and verify:

$ sha1sum igb.r151012.amd64

a4f79660b1593166be621166e8e1afc9b45af993  igb.r151012.amd64

Then copy that file into place as Dan described.

See http://omnios.omniti.com/wiki.php/FetchIPSFilesWithoutPkg for details.

Eric

...:: House of Ancients ::...
American Staffordshire Terriers

+31-628-161-350
+31-614-198-389
Het Perk 48
4903 RB
Oosterhout
Netherlands
www.houseofancients.nl

From danmcd at omniti.com  Fri Dec 12 20:10:21 2014
From: danmcd at omniti.com (Dan McDonald)
Date: Fri, 12 Dec 2014 15:10:21 -0500
Subject: [OmniOS-discuss] LACP Omnios , igb and C3750
In-Reply-To: <356582D1FC91784992ABB4265A16ED487EF47B49@vEX01.mindstorm-internet.local>
References: <~B548633c30000.5486a0500000.0002.mml.3857832108@vEX01.mindstorm-internet.local>
	<356582D1FC91784992ABB4265A16ED487EF447DC@vEX01.mindstorm-internet.local>
	<DD4737A6-D508-4759-8F6D-3869F259A3D2@omniti.com>
	<356582D1FC91784992ABB4265A16ED487EF46909@vEX01.mindstorm-internet.local>
	<C77D681D-D52C-4278-9E45-ACDBAE6ED281@omniti.com>
	<356582D1FC91784992ABB4265A16ED487EF46934@vEX01.mindstorm-internet.local>
	<F928B2C6-2760-4FCB-B368-81F840F61A03@omniti.com>
	<356582D1FC91784992ABB4265A16ED487EF46973@vEX01.mindstorm-internet.local>
	<037DE647-E2D1-4186-B2F2-39B0715BF509@omniti.com>
	<356582D1FC91784992ABB4265A16ED487EF47A6B@vEX01.mindstorm-internet.local>
	<CAO8hXRB141AoSttXbYF_igw-fbNvg9Kq1L51EKWt5P1rRQnKCg@mail.gmail.com>
	<356582D1FC91784992ABB4265A16ED487EF47B49@vEX01.mindstorm-internet.local>
Message-ID: <6A38C745-16C1-4C64-BE3B-0EB6D2EAFD77@omniti.com>


> On Dec 12, 2014, at 3:06 PM, Floris van Essen ..:: House of Ancients Amstafs ::.. <info at houseofancients.nl> wrote:
> 
> ipadm create-addr -T static -a 192.168.0.1/24 igb3/v4
> 
> let's check :
> # dladm show-ether
> LINK            PTYPE    STATE    AUTO  SPEED-DUPLEX                    PAUSE
> e1000g1         current  up       yes   1G-f                            bi
> e1000g0         current  up       yes   1G-f                            bi
> igb0            current  down     yes   0M-h                            bi
> igb1            current  unknown  yes   0M-h                            bi
> igb2            current  unknown  yes   0M-h                            bi
> igb3            current  up       yes   1G-f                            bi
> 
> awesome
> 
> root at PSD01:/root# dladm show-link
> LINK        CLASS     MTU    STATE    BRIDGE     OVER
> e1000g1     phys      1500   up       --         --
> e1000g0     phys      1500   up       --         --
> aggr0       aggr      1500   up       --         e1000g0 e1000g1
> igb0        phys      1500   down     --         --
> igb1        phys      1500   unknown  --         --
> igb2        phys      1500   unknown  --         --
> igb3        phys      1500   up       --         --
> 
> awesome
> 
> root at PSD01:/root# ifconfig -a
> lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1
>        inet 127.0.0.1 netmask ff000000
> aggr0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2
>        inet x.x.x.11 netmask ffffff00 broadcast 192.168.254.255
>        ether 0:30:48:d5:ec:94
> igb3: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 10
>        inet 192.168.0.1 netmask ffffff00 broadcast 192.168.0.255
>        ether a0:36:9f:2:c2:6f
> lo0: flags=2002000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv6,VIRTUAL> mtu 8252 index 1
>        inet6 ::1/128
> aggr0: flags=20002000840<RUNNING,MULTICAST,IPv6> mtu 1500 index 2
>        inet6 ::/0
>        ether 0:30:48:d5:ec:94
> igb3: flags=20002000840<RUNNING,MULTICAST,IPv6> mtu 1500 index 10
>        inet6 ::/0
>        ether a0:36:9f:2:c2:6f
> 
> awesome 
> 
> root at PSD01:/root# ping 192.168.0.1
> 192.168.0.1 is alive

NOTE:  Pinging the local address doesn't prove anything about a NIC.

> core-stack#sh arp | inc a036.9f:2.c26f
> core-stack#
> oh oh 
> 
> let's see if we can ping another host in the same subnet :
> 
> root at PSD01:/root# ping 192.168.0.5
> no answer from 192.168.0.5
> root at PSD01:/root#
> 
> let check the arp again :
> 
> core-stack#sh arp | inc a036.9f:2.c26f
> core-stack#
> 
> so I'm (again ) baffled 

I'm actually relieved.  I think your card is broken.  The old driver doesn't improve your situation from what was previously happening.

See if putting that card in another machine helps.  Or perhaps just re-seating it.

Dan


From info at houseofancients.nl  Fri Dec 12 20:36:16 2014
From: info at houseofancients.nl (Floris van Essen ..:: House of Ancients Amstafs ::..)
Date: Fri, 12 Dec 2014 20:36:16 +0000
Subject: [OmniOS-discuss] LACP Omnios , igb and C3750
In-Reply-To: <6A38C745-16C1-4C64-BE3B-0EB6D2EAFD77@omniti.com>
References: <~B548633c30000.5486a0500000.0002.mml.3857832108@vEX01.mindstorm-internet.local>
	<356582D1FC91784992ABB4265A16ED487EF447DC@vEX01.mindstorm-internet.local>
	<DD4737A6-D508-4759-8F6D-3869F259A3D2@omniti.com>
	<356582D1FC91784992ABB4265A16ED487EF46909@vEX01.mindstorm-internet.local>
	<C77D681D-D52C-4278-9E45-ACDBAE6ED281@omniti.com>
	<356582D1FC91784992ABB4265A16ED487EF46934@vEX01.mindstorm-internet.local>
	<F928B2C6-2760-4FCB-B368-81F840F61A03@omniti.com>
	<356582D1FC91784992ABB4265A16ED487EF46973@vEX01.mindstorm-internet.local>
	<037DE647-E2D1-4186-B2F2-39B0715BF509@omniti.com>
	<356582D1FC91784992ABB4265A16ED487EF47A6B@vEX01.mindstorm-internet.local>
	<CAO8hXRB141AoSttXbYF_igw-fbNvg9Kq1L51EKWt5P1rRQnKCg@mail.gmail.com>
	<356582D1FC91784992ABB4265A16ED487EF47B49@vEX01.mindstorm-internet.local>
	<6A38C745-16C1-4C64-BE3B-0EB6D2EAFD77@omniti.com>
Message-ID: <356582D1FC91784992ABB4265A16ED487EF47BBB@vEX01.mindstorm-internet.local>

Hi Dan,

So.. 
Pulled a working i350 ( connected to the same switch) from one of my vmware hosts.
This would be the 3rd broke i350 

Plugged it in, went into BIOS, reset config data

Still no joy no ping, not arp

My last option would be an HP NC quad nic, but I don't think that is supported by omnios



Met vriendelijke groet / With kind regards,



???????????????Floris van Essen

-----Oorspronkelijk bericht-----
Van: Dan McDonald [mailto:danmcd at omniti.com] 
Verzonden: vrijdag 12 december 2014 21:10
Aan: Floris van Essen ..:: House of Ancients Amstafs ::..
CC: Eric Sproul; omnios-discuss at lists.omniti.com
Onderwerp: Re: [OmniOS-discuss] LACP Omnios , igb and C3750


> On Dec 12, 2014, at 3:06 PM, Floris van Essen ..:: House of Ancients Amstafs ::.. <info at houseofancients.nl> wrote:
> 
> ipadm create-addr -T static -a 192.168.0.1/24 igb3/v4
> 
> let's check :
> # dladm show-ether
> LINK            PTYPE    STATE    AUTO  SPEED-DUPLEX                    PAUSE
> e1000g1         current  up       yes   1G-f                            bi
> e1000g0         current  up       yes   1G-f                            bi
> igb0            current  down     yes   0M-h                            bi
> igb1            current  unknown  yes   0M-h                            bi
> igb2            current  unknown  yes   0M-h                            bi
> igb3            current  up       yes   1G-f                            bi
> 
> awesome
> 
> root at PSD01:/root# dladm show-link
> LINK        CLASS     MTU    STATE    BRIDGE     OVER
> e1000g1     phys      1500   up       --         --
> e1000g0     phys      1500   up       --         --
> aggr0       aggr      1500   up       --         e1000g0 e1000g1
> igb0        phys      1500   down     --         --
> igb1        phys      1500   unknown  --         --
> igb2        phys      1500   unknown  --         --
> igb3        phys      1500   up       --         --
> 
> awesome
> 
> root at PSD01:/root# ifconfig -a
> lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1
>        inet 127.0.0.1 netmask ff000000
> aggr0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2
>        inet x.x.x.11 netmask ffffff00 broadcast 192.168.254.255
>        ether 0:30:48:d5:ec:94
> igb3: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 10
>        inet 192.168.0.1 netmask ffffff00 broadcast 192.168.0.255
>        ether a0:36:9f:2:c2:6f
> lo0: flags=2002000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv6,VIRTUAL> mtu 8252 index 1
>        inet6 ::1/128
> aggr0: flags=20002000840<RUNNING,MULTICAST,IPv6> mtu 1500 index 2
>        inet6 ::/0
>        ether 0:30:48:d5:ec:94
> igb3: flags=20002000840<RUNNING,MULTICAST,IPv6> mtu 1500 index 10
>        inet6 ::/0
>        ether a0:36:9f:2:c2:6f
> 
> awesome
> 
> root at PSD01:/root# ping 192.168.0.1
> 192.168.0.1 is alive

NOTE:  Pinging the local address doesn't prove anything about a NIC.

> core-stack#sh arp | inc a036.9f:2.c26f core-stack# oh oh
> 
> let's see if we can ping another host in the same subnet :
> 
> root at PSD01:/root# ping 192.168.0.5
> no answer from 192.168.0.5
> root at PSD01:/root#
> 
> let check the arp again :
> 
> core-stack#sh arp | inc a036.9f:2.c26f core-stack#
> 
> so I'm (again ) baffled

I'm actually relieved.  I think your card is broken.  The old driver doesn't improve your situation from what was previously happening.

See if putting that card in another machine helps.  Or perhaps just re-seating it.

Dan

...:: House of Ancients ::...
American Staffordshire Terriers

+31-628-161-350
+31-614-198-389
Het Perk 48
4903 RB
Oosterhout
Netherlands
www.houseofancients.nl

From danmcd at omniti.com  Fri Dec 12 20:50:15 2014
From: danmcd at omniti.com (Dan McDonald)
Date: Fri, 12 Dec 2014 15:50:15 -0500
Subject: [OmniOS-discuss] LACP Omnios , igb and C3750
In-Reply-To: <356582D1FC91784992ABB4265A16ED487EF47BBB@vEX01.mindstorm-internet.local>
References: <~B548633c30000.5486a0500000.0002.mml.3857832108@vEX01.mindstorm-internet.local>
	<356582D1FC91784992ABB4265A16ED487EF447DC@vEX01.mindstorm-internet.local>
	<DD4737A6-D508-4759-8F6D-3869F259A3D2@omniti.com>
	<356582D1FC91784992ABB4265A16ED487EF46909@vEX01.mindstorm-internet.local>
	<C77D681D-D52C-4278-9E45-ACDBAE6ED281@omniti.com>
	<356582D1FC91784992ABB4265A16ED487EF46934@vEX01.mindstorm-internet.local>
	<F928B2C6-2760-4FCB-B368-81F840F61A03@omniti.com>
	<356582D1FC91784992ABB4265A16ED487EF46973@vEX01.mindstorm-internet.local>
	<037DE647-E2D1-4186-B2F2-39B0715BF509@omniti.com>
	<356582D1FC91784992ABB4265A16ED487EF47A6B@vEX01.mindstorm-internet.local>
	<CAO8hXRB141AoSttXbYF_igw-fbNvg9Kq1L51EKWt5P1rRQnKCg@mail.gmail.com>
	<356582D1FC91784992ABB4265A16ED487EF47B49@vEX01.mindstorm-internet.local>
	<6A38C745-16C1-4C64-BE3B-0EB6D2EAFD77@omniti.com>
	<356582D1FC91784992ABB4265A16ED487EF47BBB@vEX01.mindstorm-internet.local>
Message-ID: <25D05256-180A-4C56-BADE-A4E216AACCF4@omniti.com>


> On Dec 12, 2014, at 3:36 PM, Floris van Essen ..:: House of Ancients Amstafs ::.. <info at houseofancients.nl> wrote:
> 
> Hi Dan,
> 
> So.. 
> Pulled a working i350 ( connected to the same switch) from one of my vmware hosts.
> This would be the 3rd broke i350 
> 
> Plugged it in, went into BIOS, reset config data
> 
> Still no joy no ping, not arp

grep igb /var/adm/messages --> any complaints from the kernel?

Dan


From info at houseofancients.nl  Fri Dec 12 21:03:36 2014
From: info at houseofancients.nl (Floris van Essen ..:: House of Ancients Amstafs ::..)
Date: Fri, 12 Dec 2014 21:03:36 +0000
Subject: [OmniOS-discuss] LACP Omnios , igb and C3750
In-Reply-To: <25D05256-180A-4C56-BADE-A4E216AACCF4@omniti.com>
References: <~B548633c30000.5486a0500000.0002.mml.3857832108@vEX01.mindstorm-internet.local>
	<356582D1FC91784992ABB4265A16ED487EF447DC@vEX01.mindstorm-internet.local>
	<DD4737A6-D508-4759-8F6D-3869F259A3D2@omniti.com>
	<356582D1FC91784992ABB4265A16ED487EF46909@vEX01.mindstorm-internet.local>
	<C77D681D-D52C-4278-9E45-ACDBAE6ED281@omniti.com>
	<356582D1FC91784992ABB4265A16ED487EF46934@vEX01.mindstorm-internet.local>
	<F928B2C6-2760-4FCB-B368-81F840F61A03@omniti.com>
	<356582D1FC91784992ABB4265A16ED487EF46973@vEX01.mindstorm-internet.local>
	<037DE647-E2D1-4186-B2F2-39B0715BF509@omniti.com>
	<356582D1FC91784992ABB4265A16ED487EF47A6B@vEX01.mindstorm-internet.local>
	<CAO8hXRB141AoSttXbYF_igw-fbNvg9Kq1L51EKWt5P1rRQnKCg@mail.gmail.com>
	<356582D1FC91784992ABB4265A16ED487EF47B49@vEX01.mindstorm-internet.local>
	<6A38C745-16C1-4C64-BE3B-0EB6D2EAFD77@omniti.com>
	<356582D1FC91784992ABB4265A16ED487EF47BBB@vEX01.mindstorm-internet.local>
	<25D05256-180A-4C56-BADE-A4E216AACCF4@omniti.com>
Message-ID: <356582D1FC91784992ABB4265A16ED487EF47C4C@vEX01.mindstorm-internet.local>

Looks good me :

Dec 12 22:00:17 PSD01 genunix: [ID 805372 kern.info] pcplusmp: pciex8086,1521 (igb) instance 3 irq 0x35 vector 0x62 ioapic 0xff intin 0xff is bound to cpu 1
Dec 12 22:00:17 PSD01 genunix: [ID 805372 kern.info] pcplusmp: pciex8086,1521 (igb) instance 3 irq 0x36 vector 0x63 ioapic 0xff intin 0xff is bound to cpu 2
Dec 12 22:00:17 PSD01 genunix: [ID 469746 kern.info] NOTICE: igb3 registered
Dec 12 22:00:17 PSD01 genunix: [ID 611667 kern.info] NOTICE: igb3: igb 2.3.8-ish
Dec 12 22:00:21 PSD01 genunix: [ID 435574 kern.info] NOTICE: igb3 link up, 1000 Mbps, full duplex
Dec 12 22:00:23 PSD01 genunix: [ID 805372 kern.info] pcplusmp: pciex8086,1521 (igb) instance 0 irq 0x39 vector 0x64 ioapic 0xff intin 0xff is bound to cpu 1
Dec 12 22:00:23 PSD01 genunix: [ID 805372 kern.info] pcplusmp: pciex8086,1521 (igb) instance 0 irq 0x3a vector 0x65 ioapic 0xff intin 0xff is bound to cpu 2
Dec 12 22:00:23 PSD01 genunix: [ID 469746 kern.info] NOTICE: igb0 registered
Dec 12 22:00:23 PSD01 genunix: [ID 611667 kern.info] NOTICE: igb0: igb 2.3.8-ish
Dec 12 22:00:23 PSD01 genunix: [ID 805372 kern.info] pcplusmp: pciex8086,1521 (igb) instance 1 irq 0x3b vector 0x66 ioapic 0xff intin 0xff is bound to cpu 3
Dec 12 22:00:23 PSD01 genunix: [ID 805372 kern.info] pcplusmp: pciex8086,1521 (igb) instance 1 irq 0x3c vector 0x67 ioapic 0xff intin 0xff is bound to cpu 0
Dec 12 22:00:23 PSD01 genunix: [ID 469746 kern.info] NOTICE: igb1 registered
Dec 12 22:00:23 PSD01 genunix: [ID 611667 kern.info] NOTICE: igb1: igb 2.3.8-ish
Dec 12 22:00:23 PSD01 genunix: [ID 805372 kern.info] pcplusmp: pciex8086,1521 (igb) instance 2 irq 0x3d vector 0x68 ioapic 0xff intin 0xff is bound to cpu 1
Dec 12 22:00:23 PSD01 genunix: [ID 805372 kern.info] pcplusmp: pciex8086,1521 (igb) instance 2 irq 0x3e vector 0x69 ioapic 0xff intin 0xff is bound to cpu 2
Dec 12 22:00:23 PSD01 genunix: [ID 469746 kern.info] NOTICE: igb2 registered
Dec 12 22:00:23 PSD01 genunix: [ID 611667 kern.info] NOTICE: igb2: igb 2.3.8-ish
Dec 12 22:00:39 PSD01 genunix: [ID 736570 kern.info] NOTICE: igb0 unregistered
Dec 12 22:00:39 PSD01 genunix: [ID 736570 kern.info] NOTICE: igb1 unregistered
Dec 12 22:00:39 PSD01 genunix: [ID 736570 kern.info] NOTICE: igb2 unregistered
Dec 12 22:00:40 PSD01 genunix: [ID 805372 kern.info] pcplusmp: pciex8086,1521 (igb) instance 0 irq 0x39 vector 0x64 ioapic 0xff intin 0xff is bound to cpu 3
Dec 12 22:00:40 PSD01 genunix: [ID 805372 kern.info] pcplusmp: pciex8086,1521 (igb) instance 0 irq 0x3a vector 0x65 ioapic 0xff intin 0xff is bound to cpu 0
Dec 12 22:00:40 PSD01 genunix: [ID 469746 kern.info] NOTICE: igb0 registered
Dec 12 22:00:40 PSD01 genunix: [ID 611667 kern.info] NOTICE: igb0: igb 2.3.8-ish
Dec 12 22:00:40 PSD01 genunix: [ID 805372 kern.info] pcplusmp: pciex8086,1521 (igb) instance 1 irq 0x3b vector 0x66 ioapic 0xff intin 0xff is bound to cpu 1
Dec 12 22:00:40 PSD01 genunix: [ID 805372 kern.info] pcplusmp: pciex8086,1521 (igb) instance 1 irq 0x3c vector 0x67 ioapic 0xff intin 0xff is bound to cpu 2
Dec 12 22:00:40 PSD01 genunix: [ID 469746 kern.info] NOTICE: igb1 registered
Dec 12 22:00:40 PSD01 genunix: [ID 611667 kern.info] NOTICE: igb1: igb 2.3.8-ish
Dec 12 22:00:40 PSD01 genunix: [ID 805372 kern.info] pcplusmp: pciex8086,1521 (igb) instance 2 irq 0x3d vector 0x68 ioapic 0xff intin 0xff is bound to cpu 3
Dec 12 22:00:40 PSD01 genunix: [ID 805372 kern.info] pcplusmp: pciex8086,1521 (igb) instance 2 irq 0x3e vector 0x69 ioapic 0xff intin 0xff is bound to cpu 0
Dec 12 22:00:40 PSD01 genunix: [ID 469746 kern.info] NOTICE: igb2 registered
Dec 12 22:00:40 PSD01 genunix: [ID 611667 kern.info] NOTICE: igb2: igb 2.3.8-ish

Met vriendelijke groet / With kind regards,



???????????????Floris van Essen 

-----Oorspronkelijk bericht-----
Van: Dan McDonald [mailto:danmcd at omniti.com] 
Verzonden: vrijdag 12 december 2014 21:50
Aan: Floris van Essen ..:: House of Ancients Amstafs ::..
CC: Eric Sproul; omnios-discuss at lists.omniti.com
Onderwerp: Re: [OmniOS-discuss] LACP Omnios , igb and C3750


> On Dec 12, 2014, at 3:36 PM, Floris van Essen ..:: House of Ancients Amstafs ::.. <info at houseofancients.nl> wrote:
> 
> Hi Dan,
> 
> So.. 
> Pulled a working i350 ( connected to the same switch) from one of my vmware hosts.
> This would be the 3rd broke i350 
> 
> Plugged it in, went into BIOS, reset config data
> 
> Still no joy no ping, not arp

grep igb /var/adm/messages --> any complaints from the kernel?

Dan

...:: House of Ancients ::...
American Staffordshire Terriers

+31-628-161-350
+31-614-198-389
Het Perk 48
4903 RB
Oosterhout
Netherlands
www.houseofancients.nl

From info at houseofancients.nl  Sat Dec 13 10:37:34 2014
From: info at houseofancients.nl (Floris van Essen ..:: House of Ancients Amstafs ::..)
Date: Sat, 13 Dec 2014 10:37:34 +0000
Subject: [OmniOS-discuss] LACP Omnios , igb and C3750
In-Reply-To: <~B548b22460000.548b52030004.0002.mml.372676210@vEX01.mindstorm-internet.local>
References: <~B548633c30000.5486a0500000.0002.mml.3857832108@vEX01.mindstorm-internet.local>
	<356582D1FC91784992ABB4265A16ED487EF447DC@vEX01.mindstorm-internet.local>
	<DD4737A6-D508-4759-8F6D-3869F259A3D2@omniti.com>
	<356582D1FC91784992ABB4265A16ED487EF46909@vEX01.mindstorm-internet.local>
	<C77D681D-D52C-4278-9E45-ACDBAE6ED281@omniti.com>
	<~B548b22460000.548b52030004.0002.mml.372676210@vEX01.mindstorm-internet.local>
Message-ID: <356582D1FC91784992ABB4265A16ED487F2CAE34@vEX01.mindstorm-internet.local>

So,

To add to the weirdness.

Got ready to reinstall the whole machine, to set the interface back to DHCP, set it back to VLAN I could use for reinstalling, was about to clear the arp table, and then I noticed something very strange :
	
	Sh arp
	Internet  x.x.x.104        52   a036.9f17.abc0  ARPA   Vlanx

Wait, that is the mac of my IGB interface...  did I miss something ?
	
	ifconfig
	igb0: flags=1004843<UP,BROADCAST,RUNNING,MULTICAST,DHCP,IPv4> mtu 1500 index 13
       	 inet 0.0.0.0 netmask ff000000
       	 ether a0:36:9f:17:ab:c0

so it did do an ARP, did get an IP from DHCP server, but it's simply not registering on the Omnios box...
This means hardware, and drivers are ok!

then I booted the machine using a KNOPPIX os, just to confirm my suspicions about the hardware and got ip x.x.x.104 on IGB0

I think this is something within Omnios , however, would like some pointers where to look

Met vriendelijke groet / With kind regards,



???????????????Floris van Essen

-----Oorspronkelijk bericht-----
Van: OmniOS-discuss [mailto:omnios-discuss-bounces at lists.omniti.com] Namens Floris van Essen ..:: House of Ancients Amstafs ::..
Verzonden: vrijdag 12 december 2014 18:11
Aan: Dan McDonald
CC: omnios-discuss at lists.omniti.com
Onderwerp: Re: [OmniOS-discuss] LACP Omnios , igb and C3750






> On Dec 12, 2014, at 11:49 AM, Floris van Essen ..:: House of Ancients Amstafs ::.. <info at houseofancients.nl> wrote:
> 
> Hi Dann,
> 
> No problem, just happy you did :-)
> 
> Right, so as might remember I had to reinstall because I was running 
> bloody R11 , but there was a problem upgrading , so just did a fresh install Had to skip r12 because I simply couldn't install it, as there was a installer issue with r12...

>> Weird.  I JUST updated the 012 install media thanks to illumos #5421, so you may want to try that again.

Did the installation 2 weeks ago, so guessing that that was before you did the changes.
No problem, I'll update to stable next release :-)

> So here we are again, running bloody r13 , fully updated :-)
> 
> # dladm show-ether
> LINK            PTYPE    STATE    AUTO  SPEED-DUPLEX                    PAUSE
> e1000g1         current  up       yes   1G-f                            bi
> e1000g0         current  up       yes   1G-f                            bi
> igb0            current  up       yes   1G-f                            bi
> igb1            current  up       yes   1G-f                            bi
> igb2            current  up       yes   1G-f                            bi
> igb3            current  up       yes   1G-f                            bi
> # dladm show-link
> LINK        CLASS     MTU    STATE    BRIDGE     OVER
> e1000g1     phys      1500   up       --         --
> e1000g0     phys      1500   up       --         --
> aggr0       aggr      1500   up       --         e1000g0 e1000g1
> igb0        phys      1500   up       --         --
> igb1        phys      1500   up       --         --
> igb2        phys      1500   up       --         --
> igb3        phys      1500   up       --         --

>>Tis all looks sane.  You're showing 1Gig, full dupliex, and up.  This seems sane.

> # ifconfig -a
> lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1
>        inet 127.0.0.1 netmask ff000000
> aggr0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2
>        inet x.x.x.11 netmask ffffff00 broadcast x.x.x.255
>        ether 0:30:48:d5:ec:94

And you have no problems pinging x.x.x.0/24 addresses?  Or do you?

Non, actually connecting on that interface with ssh

> igb0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 3
>        inet 192.168.0.1 netmask ffffff00 broadcast 192.168.0.255
>        ether a0:36:9f:2:c2:6c
> igb1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 4
>        inet 192.168.0.2 netmask ffffff00 broadcast 192.168.0.255
>        ether a0:36:9f:2:c2:6d
> igb2: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 5
>        inet 192.168.0.3 netmask ffffff00 broadcast 192.168.0.255
>        ether a0:36:9f:2:c2:6e
> igb3: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 6
>        inet 192.168.0.4 netmask ffffff00 broadcast 192.168.0.255
>        ether a0:36:9f:2:c2:6f

>>Now I can totally see some potential confusion here.  You've four addrs all in the same netstack and all with the same prefix.  Now granted, they're different MAC addresses, but packets coming in on one interface may have their return traffic going out >>another.

>>If these are the problem, try using just one (igb0) for starters and "ifconfig igbX down" for 1, 2, and 3.

That is exactly what i did.. for each test brought the others down Just to add, when doing a  SH ARP on my cisco's, don't see the macs popping up, which is even more worring , no arps on a switch usually is very bad news

>>Also, I need to ask --> are either of your e1000g0's shared IPMI/host links?  If so, disable the sharing feature in the BIOS.  We don't cope with shared-with-IPMI NICs.

They are not... besides.. the e1000G's ( and their aggr) are working fully as expected.
To make it more interesting.. this was the same setup as I had on r11 , and then it did seem to work 100%

Dan

...:: House of Ancients ::...
American Staffordshire Terriers

+31-628-161-350
+31-614-198-389
Het Perk 48
4903 RB
Oosterhout
Netherlands
www.houseofancients.nl
_______________________________________________
OmniOS-discuss mailing list
OmniOS-discuss at lists.omniti.com
http://lists.omniti.com/mailman/listinfo/omnios-discuss
...:: House of Ancients ::...
American Staffordshire Terriers

+31-628-161-350
+31-614-198-389
Het Perk 48
4903 RB
Oosterhout
Netherlands
www.houseofancients.nl

From joeveliscos at gmail.com  Sat Dec 13 13:50:48 2014
From: joeveliscos at gmail.com (Joe Veliscos)
Date: Sat, 13 Dec 2014 14:50:48 +0100
Subject: [OmniOS-discuss] Python difference
In-Reply-To: <CALLZ=vV1FoK_XdTmqXRSetrnZ70bE_WdvS0uQa_SSNipxokFvg@mail.gmail.com>
References: <CAHNoB+1iJ=TNq9Z0sZ-WO-BMtx--tXcrXkVyWvcu3UfS-W+CCg@mail.gmail.com>
	<56C3D19D-B648-4F02-A7CA-BC24A31C385E@omniti.com>
	<CAHNoB+1pePnQuUNHSBKLN14975bqHPX1C6F1E7nciZjkE2qQDw@mail.gmail.com>
	<CALLZ=vV1FoK_XdTmqXRSetrnZ70bE_WdvS0uQa_SSNipxokFvg@mail.gmail.com>
Message-ID: <CAHNoB+2YVfFvwT189oPzfJQX+VHX_RJR_4P8jHND6hSeW9QfXg@mail.gmail.com>

/opt/ha  was created by the HA install package.
It is not missing after upgrade but many python files on the other hand,
are removed from the system by the upgrade from r10 to r12.




On Fri, Dec 12, 2014 at 4:25 PM, Zach Malone <zmalone at omniti.com> wrote:
>
> /opt/ha is locally built, it isn't delivered by OmniOS.  I'm not sure
> why it would go missing on upgrade, how did you initially create it?
> --Zach
>
> On Thu, Dec 11, 2014 at 5:23 PM, Joe Veliscos <joeveliscos at gmail.com>
> wrote:
> > Hi
> >
> > The command that I execute :
> > root#crm
> > abort: couldn't find crm libraries in [/opt/ha/sbin
> > /usr/lib/python2.6/vendor-packages/setuptools-0.6c11-py2.6.egg
> > /opt/ha/lib/python2.6/site-packages /usr/lib/python26.zip
> /usr/lib/python2.6
> > /usr/lib/python2.6/plat-sunos5 /usr/lib/python2.6/lib-tk
> > /usr/lib/python2.6/lib-old /usr/lib/python2.6/lib-dynload
> > /usr/lib/python2.6/site-packages /usr/lib/python2.6/vendor-packages]
> > (check your install and PYTHONPATH)
> >
> > I have the following environment variables set:
> > export PYTHONPATH=/opt/ha/lib/python2.6/site-packages
> > export PATH=/opt/ha/bin:/opt/ha/sbin:$PATH
> > export OCF_ROOT=/opt/ha/lib/ocf
> > export OCF_AGENTS=/opt/ha/lib/ocf/resource.d/heartbeat
> >
> > I have exactly the same in an r10 release (pre upgrade to rr12) where
> there
> > is no problem
> >
> > I did a truss -d crm and it seems that many files it searches for are not
> > found.  Snippets of the output (very long file) hope this helps:
> >
> > Below (as I understand it ) some searches which it can resolve:
> >
> > 0.0098    resolvepath("/usr/lib/amd64/ld.so.1", "/lib/amd64/ld.so.1",
> 1023)
> > = 18
> >  0.0100    resolvepath("/usr/bin/amd64/python2.6",
> > "/usr/bin/amd64/python2.6", 1023) = 24
> >  0.0101    stat("/usr/bin/amd64/python2.6", 0xFFFFFD7FFFDFF910) = 0
> >  0.0103    open("/var/ld/64/ld.config", O_RDONLY)        Err#2 ENOENT
> >  0.0105    stat("/usr/gnu/lib/amd64/libpython2.6.so.1.0",
> > 0xFFFFFD7FFFDFF000) Err#2 ENOENT
> >  0.0106    stat("/usr/lib/amd64/libpython2.6.so.1.0",
> 0xFFFFFD7FFFDFF000) =
> > 0
> >  0.0108    resolvepath("/usr/lib/amd64/libpython2.6.so.1.0",
> > "/usr/lib/amd64/libpython2.6.so.1.0", 1023) = 34
> >  0.0110    open("/usr/lib/amd64/libpython2.6.so.1.0", O_RDONLY) = 3
> >  0.0112    mmapobj(3, MMOBJ_INTERPRET, 0xFFFFFD7FFF350AB8,
> > 0xFFFFFD7FFFDFEB5C, 0x00000000) = 0
> >  0.0113    close(3)                    = 0
> >  0.0115    mmap(0x00000000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC,
> > MAP_PRIVATE|MAP_ANON, 4294967295, 0) = 0xFFFFFD7FFF340000
> >  0.0117    memcntl(0xFFFFFD7FFEAB0000, 457808, MC_ADVISE, MADV_WILLNEED,
> 0,
> > 0) = 0
> >  0.0118    stat("/usr/gnu/lib/amd64/libsocket.so.1", 0xFFFFFD7FFFDFF000)
> > Err#2 ENOENT
> >  0.0120    stat("/usr/lib/amd64/libsocket.so.1", 0xFFFFFD7FFFDFF000) = 0
> >  0.0121    resolvepath("/usr/lib/amd64/libsocket.so.1",
> > "/lib/amd64/libsocket.so.1", 1023) = 25
> >  0.0123    open("/usr/lib/amd64/libsocket.so.1", O_RDONLY)    = 3
> >  0.0125    mmapobj(3, MMOBJ_INTERPRET, 0xFFFFFD7FFF340A18,
> > 0xFFFFFD7FFFDFEB5C, 0x00000000) = 0
> >  0.0127    close(3)                    = 0
> >  0.0128    mmap(0x00000000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC,
> > MAP_PRIVATE|MAP_ANON, 4294967295, 0) = 0xFFFFFD7FFF330000
> >  0.0129    memcntl(0xFFFFFD7FFEA80000, 32240, MC_ADVISE, MADV_WILLNEED,
> 0,
> > 0) = 0
> >  0.0130    stat("/usr/gnu/lib/amd64/libnsl.so.1", 0xFFFFFD7FFFDFF000)
> Err#2
> > ENOENT
> >  0.0132    stat("/usr/lib/amd64/libnsl.so.1", 0xFFFFFD7FFFDFF000) = 0
> >  0.0134    resolvepath("/usr/lib/amd64/libnsl.so.1",
> > "/lib/amd64/libnsl.so.1", 1023) = 22
> >  0.0135    open("/usr/lib/amd64/libnsl.so.1", O_RDONLY)    = 3
> >  0.0137    mmapobj(3, MMOBJ_INTERPRET, 0xFFFFFD7FFF3309C8,
> > 0xFFFFFD7FFFDFEB5C, 0x00000000) = 0
> >  0.0139    close(3)                    = 0
> >  0.0140    mmap(0x00000000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC,
> > MAP_PRIVATE|MAP_ANON, 4294967295, 0) = 0xFFFFFD7FFF320000
> >  0.0141    memcntl(0xFFFFFD7FFEDD0000, 180072, MC_ADVISE, MADV_WILLNEED,
> 0,
> > 0) = 0
> >  0.0142    stat("/usr/gnu/lib/amd64/libm.so.2", 0xFFFFFD7FFFDFF000) Err#2
> > ENOENT
> >  0.0144    stat("/usr/lib/amd64/libm.so.2", 0xFFFFFD7FFFDFF000) = 0
> >  0.0145    resolvepath("/usr/lib/amd64/libm.so.2",
> "/lib/amd64/libm.so.2",
> > 1023) = 20
> >  0.0147    open("/usr/lib/amd64/libm.so.2", O_RDONLY)    = 3
> >  0.0149    mmapobj(3, MMOBJ_INTERPRET, 0xFFFFFD7FFF3209F8,
> > 0xFFFFFD7FFFDFEB5C, 0x00000000) = 0
> >  0.0150    close(3)                    = 0
> >  0.0151    mmap(0x00000000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC,
> > MAP_PRIVATE|MAP_ANON, 4294967295, 0) = 0xFFFFFD7FFF310000
> >  0.0153    memcntl(0xFFFFFD7FFEEE0000, 58680, MC_ADVISE, MADV_WILLNEED,
> 0,
> > 0) = 0
> >  0.0154    stat("/usr/gnu/lib/amd64/libc.so.1", 0xFFFFFD7FFFDFF000) Err#2
> > ENOENT
> >  0.0156    stat("/usr/lib/amd64/libc.so.1", 0xFFFFFD7FFFDFF000) = 0
> >  0.0157    resolvepath("/usr/lib/amd64/libc.so.1",
> "/lib/amd64/libc.so.1",
> > 1023) = 20
> >  0.0159    open("/usr/lib/amd64/libc.so.1", O_RDONLY)    = 3
> >  0.0161    mmapobj(3, MMOBJ_INTERPRET, 0xFFFFFD7FFF310920,
> > 0xFFFFFD7FFFDFEB5C, 0x00000000) = 0
> >  0.0162    close(3)                    = 0
> >  0.0163    mmap(0x00000000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC,
> > MAP_PRIVATE|MAP_ANON, 4294967295, 0) = 0xFFFFFD7FFF160000
> >  0.0165    memcntl(0xFFFFFD7FFF170000, 477048, MC_ADVISE, MADV_WILLNEED,
> 0,
> > 0) = 0
> >  0.0166    stat("/lib/64/libsocket.so.1", 0xFFFFFD7FFFDFF000) = 0
> >  0.0168    resolvepath("/lib/64/libsocket.so.1",
> > "/lib/amd64/libsocket.so.1", 1023) = 25
> >  0.0170    stat("/lib/64/libnsl.so.1", 0xFFFFFD7FFFDFF000)    = 0
> >  0.0171    resolvepath("/lib/64/libnsl.so.1", "/lib/amd64/libnsl.so.1",
> > 1023) = 22
> >  0.0173    stat("/lib/64/libm.so.2", 0xFFFFFD7FFFDFF000)    = 0
> >  0.0174    resolvepath("/lib/64/libm.so.2", "/lib/amd64/libm.so.2",
> 1023) =
> > 20
> >  0.0177    stat("/usr/gnu/lib/amd64/libgcc_s.so.1", 0xFFFFFD7FFFDFF000)
> > Err#2 ENOENT
> >  0.0179    stat("/lib/64/libgcc_s.so.1", 0xFFFFFD7FFFDFF000) Err#2 ENOENT
> >  0.0180    stat("/usr/lib/64/libgcc_s.so.1", 0xFFFFFD7FFFDFF000) = 0
> >  0.0185    resolvepath("/usr/lib/64/libgcc_s.so.1",
> > "/usr/lib/amd64/libgcc_s.so.1", 1023) = 28
> >  0.0187    open("/usr/lib/64/libgcc_s.so.1", O_RDONLY)    = 3
> >
> >
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > Below searches which it cannot resolve:
> >
> > 0.0329    fstat(2, 0xFFFFFD7FFFDFF920)            = 0
> >  0.0331    readlink("/usr/bin/python", "python2.6", 1024)    = 9
> >  0.0333    readlink("/usr/bin/python2.6", 0xFFFFFD7FFFDFF5B0, 1024)
> Err#22
> > EINVAL
> >  0.0335    stat("/usr/bin/Modules/Setup", 0xFFFFFD7FFFDFF5B0) Err#2
> ENOENT
> >  0.0336    stat("/usr/bin/lib/python2.6/os.py", 0xFFFFFD7FFFDFF5B0) Err#2
> > ENOENT
> >  0.0338    stat("/usr/bin/lib/python2.6/os.pyc", 0xFFFFFD7FFFDFF5B0)
> Err#2
> > ENOENT
> >  0.0342    stat("/usr/lib/python2.6/os.py", 0xFFFFFD7FFFDFF5B0) = 0
> >  0.0344    stat("/usr/bin/Modules/Setup", 0xFFFFFD7FFFDFF120) Err#2
> ENOENT
> >  0.0345    stat("/usr/bin/lib/python2.6/lib-dynload", 0xFFFFFD7FFFDFF120)
> > Err#2 ENOENT
> >  0.0347    stat("/usr/lib/python2.6/lib-dynload", 0xFFFFFD7FFFDFF120) = 0
> >  0.0351    brk(0x00485250)                    = 0
> >
> >  0.0456    sysconfig(_CONFIG_SIGRT_MAX)            = 73
> >  0.0459    stat("/opt/ha/lib/python2.6/site-packages",
> 0xFFFFFD7FFFDFDF90) =
> > 0
> >  0.0461    stat("/opt/ha/lib/python2.6/site-packages",
> 0xFFFFFD7FFFDFE340) =
> > 0
> >  0.0462    stat("/opt/ha/lib/python2.6/site-packages/site",
> > 0xFFFFFD7FFFDFE640) Err#2 ENOENT
> >  0.0464    stat("/opt/ha/lib/python2.6/site-packages/site",
> > 0xFFFFFD7FFFDFEAD0) Err#2 ENOENT
> >  0.0465    open("/opt/ha/lib/python2.6/site-packages/64/site.so",
> O_RDONLY)
> > Err#2 ENOENT
> >  0.0467    stat("/opt/ha/lib/python2.6/site-packages/site",
> > 0xFFFFFD7FFFDFEAD0) Err#2 ENOENT
> >  0.0469    open("/opt/ha/lib/python2.6/site-packages/64/sitemodule.so",
> > O_RDONLY) Err#2 ENOENT
> >  0.0470    stat("/opt/ha/lib/python2.6/site-packages/site",
> > 0xFFFFFD7FFFDFEAD0) Err#2 ENOENT
> >  0.0471    open("/opt/ha/lib/python2.6/site-packages/site.py", O_RDONLY)
> > Err#2 ENOENT
> >  0.0473    stat("/opt/ha/lib/python2.6/site-packages/site",
> > 0xFFFFFD7FFFDFEAD0) Err#2 ENOENT
> >  0.0474    open("/opt/ha/lib/python2.6/site-packages/site.pyc", O_RDONLY)
> > Err#2 ENOENT
> >  0.0476    stat("/usr/lib/python26.zip", 0xFFFFFD7FFFDFDF90) Err#2 ENOENT
> >  0.0477    stat("/usr/lib", 0xFFFFFD7FFFDFDF90)        = 0
> >  0.0478    stat("/usr/lib/python26.zip", 0xFFFFFD7FFFDFE340) Err#2 ENOENT
> >  0.0480    stat("/usr/lib/python2.6/", 0xFFFFFD7FFFDFDF90)    = 0
> >  0.0481    stat("/usr/lib/python2.6/", 0xFFFFFD7FFFDFE340)    = 0
> >  0.0482    stat("/usr/lib/python2.6/site", 0xFFFFFD7FFFDFE640) Err#2
> ENOENT
> >  0.0483    stat("/usr/lib/python2.6/site", 0xFFFFFD7FFFDFEAD0) Err#2
> ENOENT
> >  0.0485    open("/usr/lib/python2.6/64/site.so", O_RDONLY)    Err#2
> ENOENT
> >  0.0486    stat("/usr/lib/python2.6/site", 0xFFFFFD7FFFDFEAD0) Err#2
> ENOENT
> >  0.0487    open("/usr/lib/python2.6/64/sitemodule.so", O_RDONLY) Err#2
> > ENOENT
> >  0.0488    stat("/usr/lib/python2.6/site", 0xFFFFFD7FFFDFEAD0) Err#2
> ENOENT
> >  0.0490    open("/usr/lib/python2.6/site.py", O_RDONLY)    = 3
> >  0.0491    fstat(3, 0xFFFFFD7FFFDFEA90)            = 0
> >  0.0492    open("/usr/lib/python2.6/site.pyc", O_RDONLY)    = 4
> >  0.0493    fstat(4, 0xFFFFFD7FFFDFE8D0)            = 0
> >  0.0494    brk(0x004D5250)                    = 0
> >  0.0496    brk(0x004D9250)                    = 0
> >  0.0498    fstat(4, 0xFFFFFD7FFFDFE800)            = 0
> >  0.0498    ioctl(4, TCGETA, 0xFFFFFD7FFFDFE880)        Err#25 ENOTTY
> >  0.0500    read(4, "D1F2\r\nA09390 S c\0\0\0".., 18944)    = 18651
> >
> > 0.0514    stat("/opt/ha/lib/python2.6/site-packages/os",
> 0xFFFFFD7FFFDFD320)
> > Err#2 ENOENT
> >  0.0516    stat("/opt/ha/lib/python2.6/site-packages/os",
> > 0xFFFFFD7FFFDFD7B0) Err#2 ENOENT
> >  0.0517    open("/opt/ha/lib/python2.6/site-packages/64/os.so", O_RDONLY)
> > Err#2 ENOENT
> >  0.0519    stat("/opt/ha/lib/python2.6/site-packages/os",
> > 0xFFFFFD7FFFDFD7B0) Err#2 ENOENT
> >  0.0520    open("/opt/ha/lib/python2.6/site-packages/64/osmodule.so",
> > O_RDONLY) Err#2 ENOENT
> >  0.0521    stat("/opt/ha/lib/python2.6/site-packages/os",
> > 0xFFFFFD7FFFDFD7B0) Err#2 ENOENT
> >  0.0522    open("/opt/ha/lib/python2.6/site-packages/os.py", O_RDONLY)
> Err#2
> > ENOENT
> >  0.0524    stat("/opt/ha/lib/python2.6/site-packages/os",
> > 0xFFFFFD7FFFDFD7B0) Err#2 ENOENT
> >  0.0525    open("/opt/ha/lib/python2.6/site-packages/os.pyc", O_RDONLY)
> > Err#2 ENOENT
> >  0.0527    stat("/usr/lib/python2.6/os", 0xFFFFFD7FFFDFD320) Err#2 ENOENT
> >  0.0528    stat("/usr/lib/python2.6/os", 0xFFFFFD7FFFDFD7B0) Err#2 ENOENT
> >  0.0529    open("/usr/lib/python2.6/64/os.so", O_RDONLY)    Err#2 ENOENT
> >  0.0530    stat("/usr/lib/python2.6/os", 0xFFFFFD7FFFDFD7B0) Err#2 ENOENT
> >  0.0532    open("/usr/lib/python2.6/64/osmodule.so", O_RDONLY) Err#2
> ENOENT
> >  0.0533    stat("/usr/lib/python2.6/os", 0xFFFFFD7FFFDFD7B0) Err#2 ENOENT
> >  0.0534    open("/usr/lib/python2.6/os.py", O_RDONLY)    = 4
> >  0.0536    fstat(4, 0xFFFFFD7FFFDFD770)            = 0
> >  0.0537    open("/usr/lib/python2.6/os.pyc", O_RDONLY)    = 5
> >  0.0538    fstat(5, 0xFFFFFD7FFFDFD5B0)            = 0
> >  0.0539    fstat(5, 0xFFFFFD7FFFDFD4E0)            = 0
> >  0.0540    ioctl(5, TCGETA, 0xFFFFFD7FFFDFD560)        Err#25 ENOTTY
> >  0.0541    read(5, "D1F2\r\n9F9390 S c\0\0\0".., 26112)    = 25702
> >
> >  0.0568    stat("/opt/ha/lib/python2.6/site-packages/posixpath",
> > 0xFFFFFD7FFFDFC000) Err#2 ENOENT
> >  0.0570    stat("/opt/ha/lib/python2.6/site-packages/posixpath",
> > 0xFFFFFD7FFFDFC490) Err#2 ENOENT
> >  0.0572    open("/opt/ha/lib/python2.6/site-packages/64/posixpath.so",
> > O_RDONLY) Err#2 ENOENT
> >  0.0573    stat("/opt/ha/lib/python2.6/site-packages/posixpath",
> > 0xFFFFFD7FFFDFC490) Err#2 ENOENT
> >  0.0575
> open("/opt/ha/lib/python2.6/site-packages/64/posixpathmodule.so",
> > O_RDONLY) Err#2 ENOENT
> >  0.0576    stat("/opt/ha/lib/python2.6/site-packages/posixpath",
> > 0xFFFFFD7FFFDFC490) Err#2 ENOENT
> >  0.0577    open("/opt/ha/lib/python2.6/site-packages/posixpath.py",
> > O_RDONLY) Err#2 ENOENT
> >  0.0579    stat("/opt/ha/lib/python2.6/site-packages/posixpath",
> > 0xFFFFFD7FFFDFC490) Err#2 ENOENT
> >  0.0580    open("/opt/ha/lib/python2.6/site-packages/posixpath.pyc",
> > O_RDONLY) Err#2 ENOENT
> >  0.0582    stat("/usr/lib/python2.6/posixpath", 0xFFFFFD7FFFDFC000) Err#2
> > ENOENT
> >  0.0583    stat("/usr/lib/python2.6/posixpath", 0xFFFFFD7FFFDFC490) Err#2
> > ENOENT
> >  0.0584    open("/usr/lib/python2.6/64/posixpath.so", O_RDONLY) Err#2
> ENOENT
> >  0.0585    stat("/usr/lib/python2.6/posixpath", 0xFFFFFD7FFFDFC490) Err#2
> > ENOENT
> >  0.0587    open("/usr/lib/python2.6/64/posixpathmodule.so", O_RDONLY)
> Err#2
> > ENOENT
> >  0.0588    stat("/usr/lib/python2.6/posixpath", 0xFFFFFD7FFFDFC490) Err#2
> > ENOENT
> >  0.0589    open("/usr/lib/python2.6/posixpath.py", O_RDONLY) = 5
> >  0.0591    fstat(5, 0xFFFFFD7FFFDFC450)            = 0
> >  0.0592    open("/usr/lib/python2.6/posixpath.pyc", O_RDONLY) = 6
> >  0.0593    fstat(6, 0xFFFFFD7FFFDFC290)            = 0
> >
> >
> > And the list goes on.
> >
> > Hope there's a solution for this.
> >
> >
> > Joe
> >
> >
> > On Thu, Dec 11, 2014 at 10:26 PM, Dan McDonald <danmcd at omniti.com>
> wrote:
> >>
> >>
> >> > On Dec 11, 2014, at 4:00 PM, Joe Veliscos <joeveliscos at gmail.com>
> wrote:
> >> >
> >> > Hi,
> >> >
> >> > I have an application working on Omnios r10 which depends on certain
> >> > python libraries. There are certain environment variables in place
> which
> >> > point to the location of those libraries.
> >> >
> >> > I have updated the r10 machine to r12. The application now cannot be
> >> > started with errors stating that the needed libraries cannot be found
> in the
> >> > given paths.
> >>
> >> Share the errors please?  I'll need more details.
> >>
> >> > Maybe somebody can tell me what the difference is in python version
> >> > between the two omnios releases. Python -V gives me 2.6.8. on both
> versions.
> >>
> >> We updated supplemental python libraries, which may contribute to what
> >> you're seeing.  Also, we had not updated the "entire" metapackage on
> 010 to
> >> show we were actually running 2.6.8.
> >>
> >> Dan
> >>
> >
> > _______________________________________________
> > OmniOS-discuss mailing list
> > OmniOS-discuss at lists.omniti.com
> > http://lists.omniti.com/mailman/listinfo/omnios-discuss
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141213/f46f160c/attachment-0001.html>

From info at houseofancients.nl  Sat Dec 13 14:01:04 2014
From: info at houseofancients.nl (Floris van Essen ..:: House of Ancients Amstafs ::..)
Date: Sat, 13 Dec 2014 14:01:04 +0000
Subject: [OmniOS-discuss] LACP Omnios , igb and C3750
In-Reply-To: <356582D1FC91784992ABB4265A16ED487F2CAE34@vEX01.mindstorm-internet.local>
References: <~B548633c30000.5486a0500000.0002.mml.3857832108@vEX01.mindstorm-internet.local>
	<356582D1FC91784992ABB4265A16ED487EF447DC@vEX01.mindstorm-internet.local>
	<DD4737A6-D508-4759-8F6D-3869F259A3D2@omniti.com>
	<356582D1FC91784992ABB4265A16ED487EF46909@vEX01.mindstorm-internet.local>
	<C77D681D-D52C-4278-9E45-ACDBAE6ED281@omniti.com>
	<~B548b22460000.548b52030004.0002.mml.372676210@vEX01.mindstorm-internet.local>
	<356582D1FC91784992ABB4265A16ED487F2CAE34@vEX01.mindstorm-internet.local>
Message-ID: <356582D1FC91784992ABB4265A16ED487F2CCEBF@vEX01.mindstorm-internet.local>

So i replaced the Igb adapter with a HP NC375t, loaded the drivers from HP,left cables and switch alone :

ntxn0  	 ntxn0  	 Ethernet  	 up  	 1000  	 full  	 BOUND  	 192.168.249.104  	 255.255.255.0  	 78:ac:c0:11:d4:88  	 1500  	 ok  	    	   
 ntxn1  	 ntxn1  	 Ethernet  	 unknown  	 1000  	 full  	 static  	 unset  	 -  	 -  	 1500  	   	    	   
 ntxn2  	 ntxn2  	 Ethernet  	 unknown  	 1000  	 full  	 static  	 unset  	 -  	 -  	 1500  	   	    	   
 ntxn3  	 ntxn3  	 Ethernet  	 unknown  	 1000  	 full  	 static  	 unset  	 -  	 -  	 1500  	   	

this all lead me to conclude that there is something wrong with the IGB implementation as it sits
Weather it be drivers, or something more to the background..don't know, but would love to find out

Met vriendelijke groet / With kind regards,



???????????????Floris van Essen

-----Oorspronkelijk bericht-----
Van: Floris van Essen ..:: House of Ancients Amstafs ::.. 
Verzonden: zaterdag 13 december 2014 11:38
Aan: Dan McDonald
CC: omnios-discuss at lists.omniti.com
Onderwerp: Re: [OmniOS-discuss] LACP Omnios , igb and C3750

So,

To add to the weirdness.

Got ready to reinstall the whole machine, to set the interface back to DHCP, set it back to VLAN I could use for reinstalling, was about to clear the arp table, and then I noticed something very strange :
	
	Sh arp
	Internet  x.x.x.104        52   a036.9f17.abc0  ARPA   Vlanx

Wait, that is the mac of my IGB interface...  did I miss something ?
	
	ifconfig
	igb0: flags=1004843<UP,BROADCAST,RUNNING,MULTICAST,DHCP,IPv4> mtu 1500 index 13
       	 inet 0.0.0.0 netmask ff000000
       	 ether a0:36:9f:17:ab:c0

so it did do an ARP, did get an IP from DHCP server, but it's simply not registering on the Omnios box...
This means hardware, and drivers are ok!

then I booted the machine using a KNOPPIX os, just to confirm my suspicions about the hardware and got ip x.x.x.104 on IGB0

I think this is something within Omnios , however, would like some pointers where to look

Met vriendelijke groet / With kind regards,



???????????????Floris van Essen

-----Oorspronkelijk bericht-----
Van: OmniOS-discuss [mailto:omnios-discuss-bounces at lists.omniti.com] Namens Floris van Essen ..:: House of Ancients Amstafs ::..
Verzonden: vrijdag 12 december 2014 18:11
Aan: Dan McDonald
CC: omnios-discuss at lists.omniti.com
Onderwerp: Re: [OmniOS-discuss] LACP Omnios , igb and C3750






> On Dec 12, 2014, at 11:49 AM, Floris van Essen ..:: House of Ancients Amstafs ::.. <info at houseofancients.nl> wrote:
> 
> Hi Dann,
> 
> No problem, just happy you did :-)
> 
> Right, so as might remember I had to reinstall because I was running 
> bloody R11 , but there was a problem upgrading , so just did a fresh install Had to skip r12 because I simply couldn't install it, as there was a installer issue with r12...

>> Weird.  I JUST updated the 012 install media thanks to illumos #5421, so you may want to try that again.

Did the installation 2 weeks ago, so guessing that that was before you did the changes.
No problem, I'll update to stable next release :-)

> So here we are again, running bloody r13 , fully updated :-)
> 
> # dladm show-ether
> LINK            PTYPE    STATE    AUTO  SPEED-DUPLEX                    PAUSE
> e1000g1         current  up       yes   1G-f                            bi
> e1000g0         current  up       yes   1G-f                            bi
> igb0            current  up       yes   1G-f                            bi
> igb1            current  up       yes   1G-f                            bi
> igb2            current  up       yes   1G-f                            bi
> igb3            current  up       yes   1G-f                            bi
> # dladm show-link
> LINK        CLASS     MTU    STATE    BRIDGE     OVER
> e1000g1     phys      1500   up       --         --
> e1000g0     phys      1500   up       --         --
> aggr0       aggr      1500   up       --         e1000g0 e1000g1
> igb0        phys      1500   up       --         --
> igb1        phys      1500   up       --         --
> igb2        phys      1500   up       --         --
> igb3        phys      1500   up       --         --

>>Tis all looks sane.  You're showing 1Gig, full dupliex, and up.  This seems sane.

> # ifconfig -a
> lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1
>        inet 127.0.0.1 netmask ff000000
> aggr0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2
>        inet x.x.x.11 netmask ffffff00 broadcast x.x.x.255
>        ether 0:30:48:d5:ec:94

And you have no problems pinging x.x.x.0/24 addresses?  Or do you?

Non, actually connecting on that interface with ssh

> igb0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 3
>        inet 192.168.0.1 netmask ffffff00 broadcast 192.168.0.255
>        ether a0:36:9f:2:c2:6c
> igb1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 4
>        inet 192.168.0.2 netmask ffffff00 broadcast 192.168.0.255
>        ether a0:36:9f:2:c2:6d
> igb2: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 5
>        inet 192.168.0.3 netmask ffffff00 broadcast 192.168.0.255
>        ether a0:36:9f:2:c2:6e
> igb3: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 6
>        inet 192.168.0.4 netmask ffffff00 broadcast 192.168.0.255
>        ether a0:36:9f:2:c2:6f

>>Now I can totally see some potential confusion here.  You've four addrs all in the same netstack and all with the same prefix.  Now granted, they're different MAC addresses, but packets coming in on one interface may have their return traffic going out >>another.

>>If these are the problem, try using just one (igb0) for starters and "ifconfig igbX down" for 1, 2, and 3.

That is exactly what i did.. for each test brought the others down Just to add, when doing a  SH ARP on my cisco's, don't see the macs popping up, which is even more worring , no arps on a switch usually is very bad news

>>Also, I need to ask --> are either of your e1000g0's shared IPMI/host links?  If so, disable the sharing feature in the BIOS.  We don't cope with shared-with-IPMI NICs.

They are not... besides.. the e1000G's ( and their aggr) are working fully as expected.
To make it more interesting.. this was the same setup as I had on r11 , and then it did seem to work 100%

Dan

...:: House of Ancients ::...
American Staffordshire Terriers

+31-628-161-350
+31-614-198-389
Het Perk 48
4903 RB
Oosterhout
Netherlands
www.houseofancients.nl
_______________________________________________
OmniOS-discuss mailing list
OmniOS-discuss at lists.omniti.com
http://lists.omniti.com/mailman/listinfo/omnios-discuss
...:: House of Ancients ::...
American Staffordshire Terriers

+31-628-161-350
+31-614-198-389
Het Perk 48
4903 RB
Oosterhout
Netherlands
www.houseofancients.nl

...:: House of Ancients ::...
American Staffordshire Terriers

+31-628-161-350
+31-614-198-389
Het Perk 48
4903 RB
Oosterhout
Netherlands
www.houseofancients.nl

From danmcd at omniti.com  Sat Dec 13 16:12:35 2014
From: danmcd at omniti.com (Dan McDonald)
Date: Sat, 13 Dec 2014 11:12:35 -0500
Subject: [OmniOS-discuss] LACP Omnios , igb and C3750
In-Reply-To: <356582D1FC91784992ABB4265A16ED487F2CCEBF@vEX01.mindstorm-internet.local>
References: <~B548633c30000.5486a0500000.0002.mml.3857832108@vEX01.mindstorm-internet.local>
	<356582D1FC91784992ABB4265A16ED487EF447DC@vEX01.mindstorm-internet.local>
	<DD4737A6-D508-4759-8F6D-3869F259A3D2@omniti.com>
	<356582D1FC91784992ABB4265A16ED487EF46909@vEX01.mindstorm-internet.local>
	<C77D681D-D52C-4278-9E45-ACDBAE6ED281@omniti.com>
	<~B548b22460000.548b52030004.0002.mml.372676210@vEX01.mindstorm-internet.local>
	<356582D1FC91784992ABB4265A16ED487F2CAE34@vEX01.mindstorm-internet.local>
	<356582D1FC91784992ABB4265A16ED487F2CCEBF@vEX01.mindstorm-internet.local>
Message-ID: <1D812FAE-40BA-4464-8896-208E2FCF01DB@omniti.com>

I thought I'd mentioned this on the thread, but does the machine attempt to use one of the IGB interfaces for IPMI as well?  We do not support dual-use NICs like that.  (Check your BIOS.)

Dan



Sent from my iPhone (typos, autocorrect, and all)

> On Dec 13, 2014, at 9:01 AM, Floris van Essen ..:: House of Ancients Amstafs ::.. <info at houseofancients.nl> wrote:
> 
> So i replaced the Igb adapter with a HP NC375t, loaded the drivers from HP,left cables and switch alone :
> 
> ntxn0       ntxn0       Ethernet       up       1000       full       BOUND       192.168.249.104       255.255.255.0       78:ac:c0:11:d4:88       1500       ok                 
> ntxn1       ntxn1       Ethernet       unknown       1000       full       static       unset       -       -       1500                        
> ntxn2       ntxn2       Ethernet       unknown       1000       full       static       unset       -       -       1500                        
> ntxn3       ntxn3       Ethernet       unknown       1000       full       static       unset       -       -       1500             
> 
> this all lead me to conclude that there is something wrong with the IGB implementation as it sits
> Weather it be drivers, or something more to the background..don't know, but would love to find out
> 
> Met vriendelijke groet / With kind regards,
> 
> 
> 
>                Floris van Essen
> 
> -----Oorspronkelijk bericht-----
> Van: Floris van Essen ..:: House of Ancients Amstafs ::.. 
> Verzonden: zaterdag 13 december 2014 11:38
> Aan: Dan McDonald
> CC: omnios-discuss at lists.omniti.com
> Onderwerp: Re: [OmniOS-discuss] LACP Omnios , igb and C3750
> 
> So,
> 
> To add to the weirdness.
> 
> Got ready to reinstall the whole machine, to set the interface back to DHCP, set it back to VLAN I could use for reinstalling, was about to clear the arp table, and then I noticed something very strange :
>    
>    Sh arp
>    Internet  x.x.x.104        52   a036.9f17.abc0  ARPA   Vlanx
> 
> Wait, that is the mac of my IGB interface...  did I miss something ?
>    
>    ifconfig
>    igb0: flags=1004843<UP,BROADCAST,RUNNING,MULTICAST,DHCP,IPv4> mtu 1500 index 13
>            inet 0.0.0.0 netmask ff000000
>            ether a0:36:9f:17:ab:c0
> 
> so it did do an ARP, did get an IP from DHCP server, but it's simply not registering on the Omnios box...
> This means hardware, and drivers are ok!
> 
> then I booted the machine using a KNOPPIX os, just to confirm my suspicions about the hardware and got ip x.x.x.104 on IGB0
> 
> I think this is something within Omnios , however, would like some pointers where to look
> 
> Met vriendelijke groet / With kind regards,
> 
> 
> 
>                Floris van Essen
> 
> -----Oorspronkelijk bericht-----
> Van: OmniOS-discuss [mailto:omnios-discuss-bounces at lists.omniti.com] Namens Floris van Essen ..:: House of Ancients Amstafs ::..
> Verzonden: vrijdag 12 december 2014 18:11
> Aan: Dan McDonald
> CC: omnios-discuss at lists.omniti.com
> Onderwerp: Re: [OmniOS-discuss] LACP Omnios , igb and C3750
> 
> 
> 
> 
> 
> 
>> On Dec 12, 2014, at 11:49 AM, Floris van Essen ..:: House of Ancients Amstafs ::.. <info at houseofancients.nl> wrote:
>> 
>> Hi Dann,
>> 
>> No problem, just happy you did :-)
>> 
>> Right, so as might remember I had to reinstall because I was running 
>> bloody R11 , but there was a problem upgrading , so just did a fresh install Had to skip r12 because I simply couldn't install it, as there was a installer issue with r12...
> 
>>> Weird.  I JUST updated the 012 install media thanks to illumos #5421, so you may want to try that again.
> 
> Did the installation 2 weeks ago, so guessing that that was before you did the changes.
> No problem, I'll update to stable next release :-)
> 
>> So here we are again, running bloody r13 , fully updated :-)
>> 
>> # dladm show-ether
>> LINK            PTYPE    STATE    AUTO  SPEED-DUPLEX                    PAUSE
>> e1000g1         current  up       yes   1G-f                            bi
>> e1000g0         current  up       yes   1G-f                            bi
>> igb0            current  up       yes   1G-f                            bi
>> igb1            current  up       yes   1G-f                            bi
>> igb2            current  up       yes   1G-f                            bi
>> igb3            current  up       yes   1G-f                            bi
>> # dladm show-link
>> LINK        CLASS     MTU    STATE    BRIDGE     OVER
>> e1000g1     phys      1500   up       --         --
>> e1000g0     phys      1500   up       --         --
>> aggr0       aggr      1500   up       --         e1000g0 e1000g1
>> igb0        phys      1500   up       --         --
>> igb1        phys      1500   up       --         --
>> igb2        phys      1500   up       --         --
>> igb3        phys      1500   up       --         --
> 
>>> Tis all looks sane.  You're showing 1Gig, full dupliex, and up.  This seems sane.
> 
>> # ifconfig -a
>> lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1
>>       inet 127.0.0.1 netmask ff000000
>> aggr0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2
>>       inet x.x.x.11 netmask ffffff00 broadcast x.x.x.255
>>       ether 0:30:48:d5:ec:94
> 
> And you have no problems pinging x.x.x.0/24 addresses?  Or do you?
> 
> Non, actually connecting on that interface with ssh
> 
>> igb0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 3
>>       inet 192.168.0.1 netmask ffffff00 broadcast 192.168.0.255
>>       ether a0:36:9f:2:c2:6c
>> igb1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 4
>>       inet 192.168.0.2 netmask ffffff00 broadcast 192.168.0.255
>>       ether a0:36:9f:2:c2:6d
>> igb2: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 5
>>       inet 192.168.0.3 netmask ffffff00 broadcast 192.168.0.255
>>       ether a0:36:9f:2:c2:6e
>> igb3: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 6
>>       inet 192.168.0.4 netmask ffffff00 broadcast 192.168.0.255
>>       ether a0:36:9f:2:c2:6f
> 
>>> Now I can totally see some potential confusion here.  You've four addrs all in the same netstack and all with the same prefix.  Now granted, they're different MAC addresses, but packets coming in on one interface may have their return traffic going out >>another.
> 
>>> If these are the problem, try using just one (igb0) for starters and "ifconfig igbX down" for 1, 2, and 3.
> 
> That is exactly what i did.. for each test brought the others down Just to add, when doing a  SH ARP on my cisco's, don't see the macs popping up, which is even more worring , no arps on a switch usually is very bad news
> 
>>> Also, I need to ask --> are either of your e1000g0's shared IPMI/host links?  If so, disable the sharing feature in the BIOS.  We don't cope with shared-with-IPMI NICs.
> 
> They are not... besides.. the e1000G's ( and their aggr) are working fully as expected.
> To make it more interesting.. this was the same setup as I had on r11 , and then it did seem to work 100%
> 
> Dan
> 
> ...:: House of Ancients ::...
> American Staffordshire Terriers
> 
> +31-628-161-350
> +31-614-198-389
> Het Perk 48
> 4903 RB
> Oosterhout
> Netherlands
> www.houseofancients.nl
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
> ...:: House of Ancients ::...
> American Staffordshire Terriers
> 
> +31-628-161-350
> +31-614-198-389
> Het Perk 48
> 4903 RB
> Oosterhout
> Netherlands
> www.houseofancients.nl
> 
> ...:: House of Ancients ::...
> American Staffordshire Terriers
> 
> +31-628-161-350
> +31-614-198-389
> Het Perk 48
> 4903 RB
> Oosterhout
> Netherlands
> www.houseofancients.nl

From jesus at omniti.com  Sat Dec 13 17:14:27 2014
From: jesus at omniti.com (Theo Schlossnagle)
Date: Sat, 13 Dec 2014 09:14:27 -0800
Subject: [OmniOS-discuss] Python difference
In-Reply-To: <CAHNoB+2YVfFvwT189oPzfJQX+VHX_RJR_4P8jHND6hSeW9QfXg@mail.gmail.com>
References: <CAHNoB+1iJ=TNq9Z0sZ-WO-BMtx--tXcrXkVyWvcu3UfS-W+CCg@mail.gmail.com>
	<56C3D19D-B648-4F02-A7CA-BC24A31C385E@omniti.com>
	<CAHNoB+1pePnQuUNHSBKLN14975bqHPX1C6F1E7nciZjkE2qQDw@mail.gmail.com>
	<CALLZ=vV1FoK_XdTmqXRSetrnZ70bE_WdvS0uQa_SSNipxokFvg@mail.gmail.com>
	<CAHNoB+2YVfFvwT189oPzfJQX+VHX_RJR_4P8jHND6hSeW9QfXg@mail.gmail.com>
Message-ID: <CACLsAptmZ9QFHeXXkZW3wigPnNywq=5SHx_+sg77TzWuxvJOGQ@mail.gmail.com>

A package (like ha) should either rely on the system python packages /
versions it needs to work (thus preventing your upgrade to 012) or more
reasonably rely on its own known-good python.  I don't believe OmniOS makes
any release-to-release compatibility guarantees for python (or perl or java
for that matter). The only reason they even exist in the core is because we
have other core requirements that depend upon them.

On Sat, Dec 13, 2014 at 5:50 AM, Joe Veliscos <joeveliscos at gmail.com> wrote:
>
> /opt/ha  was created by the HA install package.
> It is not missing after upgrade but many python files on the other hand,
> are removed from the system by the upgrade from r10 to r12.
>
>
>
>
> On Fri, Dec 12, 2014 at 4:25 PM, Zach Malone <zmalone at omniti.com> wrote:
>>
>> /opt/ha is locally built, it isn't delivered by OmniOS.  I'm not sure
>> why it would go missing on upgrade, how did you initially create it?
>> --Zach
>>
>> On Thu, Dec 11, 2014 at 5:23 PM, Joe Veliscos <joeveliscos at gmail.com>
>> wrote:
>> > Hi
>> >
>> > The command that I execute :
>> > root#crm
>> > abort: couldn't find crm libraries in [/opt/ha/sbin
>> > /usr/lib/python2.6/vendor-packages/setuptools-0.6c11-py2.6.egg
>> > /opt/ha/lib/python2.6/site-packages /usr/lib/python26.zip
>> /usr/lib/python2.6
>> > /usr/lib/python2.6/plat-sunos5 /usr/lib/python2.6/lib-tk
>> > /usr/lib/python2.6/lib-old /usr/lib/python2.6/lib-dynload
>> > /usr/lib/python2.6/site-packages /usr/lib/python2.6/vendor-packages]
>> > (check your install and PYTHONPATH)
>> >
>> > I have the following environment variables set:
>> > export PYTHONPATH=/opt/ha/lib/python2.6/site-packages
>> > export PATH=/opt/ha/bin:/opt/ha/sbin:$PATH
>> > export OCF_ROOT=/opt/ha/lib/ocf
>> > export OCF_AGENTS=/opt/ha/lib/ocf/resource.d/heartbeat
>> >
>> > I have exactly the same in an r10 release (pre upgrade to rr12) where
>> there
>> > is no problem
>> >
>> > I did a truss -d crm and it seems that many files it searches for are
>> not
>> > found.  Snippets of the output (very long file) hope this helps:
>> >
>> > Below (as I understand it ) some searches which it can resolve:
>> >
>> > 0.0098    resolvepath("/usr/lib/amd64/ld.so.1", "/lib/amd64/ld.so.1",
>> 1023)
>> > = 18
>> >  0.0100    resolvepath("/usr/bin/amd64/python2.6",
>> > "/usr/bin/amd64/python2.6", 1023) = 24
>> >  0.0101    stat("/usr/bin/amd64/python2.6", 0xFFFFFD7FFFDFF910) = 0
>> >  0.0103    open("/var/ld/64/ld.config", O_RDONLY)        Err#2 ENOENT
>> >  0.0105    stat("/usr/gnu/lib/amd64/libpython2.6.so.1.0",
>> > 0xFFFFFD7FFFDFF000) Err#2 ENOENT
>> >  0.0106    stat("/usr/lib/amd64/libpython2.6.so.1.0",
>> 0xFFFFFD7FFFDFF000) =
>> > 0
>> >  0.0108    resolvepath("/usr/lib/amd64/libpython2.6.so.1.0",
>> > "/usr/lib/amd64/libpython2.6.so.1.0", 1023) = 34
>> >  0.0110    open("/usr/lib/amd64/libpython2.6.so.1.0", O_RDONLY) = 3
>> >  0.0112    mmapobj(3, MMOBJ_INTERPRET, 0xFFFFFD7FFF350AB8,
>> > 0xFFFFFD7FFFDFEB5C, 0x00000000) = 0
>> >  0.0113    close(3)                    = 0
>> >  0.0115    mmap(0x00000000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC,
>> > MAP_PRIVATE|MAP_ANON, 4294967295, 0) = 0xFFFFFD7FFF340000
>> >  0.0117    memcntl(0xFFFFFD7FFEAB0000, 457808, MC_ADVISE,
>> MADV_WILLNEED, 0,
>> > 0) = 0
>> >  0.0118    stat("/usr/gnu/lib/amd64/libsocket.so.1", 0xFFFFFD7FFFDFF000)
>> > Err#2 ENOENT
>> >  0.0120    stat("/usr/lib/amd64/libsocket.so.1", 0xFFFFFD7FFFDFF000) = 0
>> >  0.0121    resolvepath("/usr/lib/amd64/libsocket.so.1",
>> > "/lib/amd64/libsocket.so.1", 1023) = 25
>> >  0.0123    open("/usr/lib/amd64/libsocket.so.1", O_RDONLY)    = 3
>> >  0.0125    mmapobj(3, MMOBJ_INTERPRET, 0xFFFFFD7FFF340A18,
>> > 0xFFFFFD7FFFDFEB5C, 0x00000000) = 0
>> >  0.0127    close(3)                    = 0
>> >  0.0128    mmap(0x00000000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC,
>> > MAP_PRIVATE|MAP_ANON, 4294967295, 0) = 0xFFFFFD7FFF330000
>> >  0.0129    memcntl(0xFFFFFD7FFEA80000, 32240, MC_ADVISE, MADV_WILLNEED,
>> 0,
>> > 0) = 0
>> >  0.0130    stat("/usr/gnu/lib/amd64/libnsl.so.1", 0xFFFFFD7FFFDFF000)
>> Err#2
>> > ENOENT
>> >  0.0132    stat("/usr/lib/amd64/libnsl.so.1", 0xFFFFFD7FFFDFF000) = 0
>> >  0.0134    resolvepath("/usr/lib/amd64/libnsl.so.1",
>> > "/lib/amd64/libnsl.so.1", 1023) = 22
>> >  0.0135    open("/usr/lib/amd64/libnsl.so.1", O_RDONLY)    = 3
>> >  0.0137    mmapobj(3, MMOBJ_INTERPRET, 0xFFFFFD7FFF3309C8,
>> > 0xFFFFFD7FFFDFEB5C, 0x00000000) = 0
>> >  0.0139    close(3)                    = 0
>> >  0.0140    mmap(0x00000000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC,
>> > MAP_PRIVATE|MAP_ANON, 4294967295, 0) = 0xFFFFFD7FFF320000
>> >  0.0141    memcntl(0xFFFFFD7FFEDD0000, 180072, MC_ADVISE,
>> MADV_WILLNEED, 0,
>> > 0) = 0
>> >  0.0142    stat("/usr/gnu/lib/amd64/libm.so.2", 0xFFFFFD7FFFDFF000)
>> Err#2
>> > ENOENT
>> >  0.0144    stat("/usr/lib/amd64/libm.so.2", 0xFFFFFD7FFFDFF000) = 0
>> >  0.0145    resolvepath("/usr/lib/amd64/libm.so.2",
>> "/lib/amd64/libm.so.2",
>> > 1023) = 20
>> >  0.0147    open("/usr/lib/amd64/libm.so.2", O_RDONLY)    = 3
>> >  0.0149    mmapobj(3, MMOBJ_INTERPRET, 0xFFFFFD7FFF3209F8,
>> > 0xFFFFFD7FFFDFEB5C, 0x00000000) = 0
>> >  0.0150    close(3)                    = 0
>> >  0.0151    mmap(0x00000000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC,
>> > MAP_PRIVATE|MAP_ANON, 4294967295, 0) = 0xFFFFFD7FFF310000
>> >  0.0153    memcntl(0xFFFFFD7FFEEE0000, 58680, MC_ADVISE, MADV_WILLNEED,
>> 0,
>> > 0) = 0
>> >  0.0154    stat("/usr/gnu/lib/amd64/libc.so.1", 0xFFFFFD7FFFDFF000)
>> Err#2
>> > ENOENT
>> >  0.0156    stat("/usr/lib/amd64/libc.so.1", 0xFFFFFD7FFFDFF000) = 0
>> >  0.0157    resolvepath("/usr/lib/amd64/libc.so.1",
>> "/lib/amd64/libc.so.1",
>> > 1023) = 20
>> >  0.0159    open("/usr/lib/amd64/libc.so.1", O_RDONLY)    = 3
>> >  0.0161    mmapobj(3, MMOBJ_INTERPRET, 0xFFFFFD7FFF310920,
>> > 0xFFFFFD7FFFDFEB5C, 0x00000000) = 0
>> >  0.0162    close(3)                    = 0
>> >  0.0163    mmap(0x00000000, 4096, PROT_READ|PROT_WRITE|PROT_EXEC,
>> > MAP_PRIVATE|MAP_ANON, 4294967295, 0) = 0xFFFFFD7FFF160000
>> >  0.0165    memcntl(0xFFFFFD7FFF170000, 477048, MC_ADVISE,
>> MADV_WILLNEED, 0,
>> > 0) = 0
>> >  0.0166    stat("/lib/64/libsocket.so.1", 0xFFFFFD7FFFDFF000) = 0
>> >  0.0168    resolvepath("/lib/64/libsocket.so.1",
>> > "/lib/amd64/libsocket.so.1", 1023) = 25
>> >  0.0170    stat("/lib/64/libnsl.so.1", 0xFFFFFD7FFFDFF000)    = 0
>> >  0.0171    resolvepath("/lib/64/libnsl.so.1", "/lib/amd64/libnsl.so.1",
>> > 1023) = 22
>> >  0.0173    stat("/lib/64/libm.so.2", 0xFFFFFD7FFFDFF000)    = 0
>> >  0.0174    resolvepath("/lib/64/libm.so.2", "/lib/amd64/libm.so.2",
>> 1023) =
>> > 20
>> >  0.0177    stat("/usr/gnu/lib/amd64/libgcc_s.so.1", 0xFFFFFD7FFFDFF000)
>> > Err#2 ENOENT
>> >  0.0179    stat("/lib/64/libgcc_s.so.1", 0xFFFFFD7FFFDFF000) Err#2
>> ENOENT
>> >  0.0180    stat("/usr/lib/64/libgcc_s.so.1", 0xFFFFFD7FFFDFF000) = 0
>> >  0.0185    resolvepath("/usr/lib/64/libgcc_s.so.1",
>> > "/usr/lib/amd64/libgcc_s.so.1", 1023) = 28
>> >  0.0187    open("/usr/lib/64/libgcc_s.so.1", O_RDONLY)    = 3
>> >
>> >
>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>> > Below searches which it cannot resolve:
>> >
>> > 0.0329    fstat(2, 0xFFFFFD7FFFDFF920)            = 0
>> >  0.0331    readlink("/usr/bin/python", "python2.6", 1024)    = 9
>> >  0.0333    readlink("/usr/bin/python2.6", 0xFFFFFD7FFFDFF5B0, 1024)
>> Err#22
>> > EINVAL
>> >  0.0335    stat("/usr/bin/Modules/Setup", 0xFFFFFD7FFFDFF5B0) Err#2
>> ENOENT
>> >  0.0336    stat("/usr/bin/lib/python2.6/os.py", 0xFFFFFD7FFFDFF5B0)
>> Err#2
>> > ENOENT
>> >  0.0338    stat("/usr/bin/lib/python2.6/os.pyc", 0xFFFFFD7FFFDFF5B0)
>> Err#2
>> > ENOENT
>> >  0.0342    stat("/usr/lib/python2.6/os.py", 0xFFFFFD7FFFDFF5B0) = 0
>> >  0.0344    stat("/usr/bin/Modules/Setup", 0xFFFFFD7FFFDFF120) Err#2
>> ENOENT
>> >  0.0345    stat("/usr/bin/lib/python2.6/lib-dynload",
>> 0xFFFFFD7FFFDFF120)
>> > Err#2 ENOENT
>> >  0.0347    stat("/usr/lib/python2.6/lib-dynload", 0xFFFFFD7FFFDFF120) =
>> 0
>> >  0.0351    brk(0x00485250)                    = 0
>> >
>> >  0.0456    sysconfig(_CONFIG_SIGRT_MAX)            = 73
>> >  0.0459    stat("/opt/ha/lib/python2.6/site-packages",
>> 0xFFFFFD7FFFDFDF90) =
>> > 0
>> >  0.0461    stat("/opt/ha/lib/python2.6/site-packages",
>> 0xFFFFFD7FFFDFE340) =
>> > 0
>> >  0.0462    stat("/opt/ha/lib/python2.6/site-packages/site",
>> > 0xFFFFFD7FFFDFE640) Err#2 ENOENT
>> >  0.0464    stat("/opt/ha/lib/python2.6/site-packages/site",
>> > 0xFFFFFD7FFFDFEAD0) Err#2 ENOENT
>> >  0.0465    open("/opt/ha/lib/python2.6/site-packages/64/site.so",
>> O_RDONLY)
>> > Err#2 ENOENT
>> >  0.0467    stat("/opt/ha/lib/python2.6/site-packages/site",
>> > 0xFFFFFD7FFFDFEAD0) Err#2 ENOENT
>> >  0.0469    open("/opt/ha/lib/python2.6/site-packages/64/sitemodule.so",
>> > O_RDONLY) Err#2 ENOENT
>> >  0.0470    stat("/opt/ha/lib/python2.6/site-packages/site",
>> > 0xFFFFFD7FFFDFEAD0) Err#2 ENOENT
>> >  0.0471    open("/opt/ha/lib/python2.6/site-packages/site.py", O_RDONLY)
>> > Err#2 ENOENT
>> >  0.0473    stat("/opt/ha/lib/python2.6/site-packages/site",
>> > 0xFFFFFD7FFFDFEAD0) Err#2 ENOENT
>> >  0.0474    open("/opt/ha/lib/python2.6/site-packages/site.pyc",
>> O_RDONLY)
>> > Err#2 ENOENT
>> >  0.0476    stat("/usr/lib/python26.zip", 0xFFFFFD7FFFDFDF90) Err#2
>> ENOENT
>> >  0.0477    stat("/usr/lib", 0xFFFFFD7FFFDFDF90)        = 0
>> >  0.0478    stat("/usr/lib/python26.zip", 0xFFFFFD7FFFDFE340) Err#2
>> ENOENT
>> >  0.0480    stat("/usr/lib/python2.6/", 0xFFFFFD7FFFDFDF90)    = 0
>> >  0.0481    stat("/usr/lib/python2.6/", 0xFFFFFD7FFFDFE340)    = 0
>> >  0.0482    stat("/usr/lib/python2.6/site", 0xFFFFFD7FFFDFE640) Err#2
>> ENOENT
>> >  0.0483    stat("/usr/lib/python2.6/site", 0xFFFFFD7FFFDFEAD0) Err#2
>> ENOENT
>> >  0.0485    open("/usr/lib/python2.6/64/site.so", O_RDONLY)    Err#2
>> ENOENT
>> >  0.0486    stat("/usr/lib/python2.6/site", 0xFFFFFD7FFFDFEAD0) Err#2
>> ENOENT
>> >  0.0487    open("/usr/lib/python2.6/64/sitemodule.so", O_RDONLY) Err#2
>> > ENOENT
>> >  0.0488    stat("/usr/lib/python2.6/site", 0xFFFFFD7FFFDFEAD0) Err#2
>> ENOENT
>> >  0.0490    open("/usr/lib/python2.6/site.py", O_RDONLY)    = 3
>> >  0.0491    fstat(3, 0xFFFFFD7FFFDFEA90)            = 0
>> >  0.0492    open("/usr/lib/python2.6/site.pyc", O_RDONLY)    = 4
>> >  0.0493    fstat(4, 0xFFFFFD7FFFDFE8D0)            = 0
>> >  0.0494    brk(0x004D5250)                    = 0
>> >  0.0496    brk(0x004D9250)                    = 0
>> >  0.0498    fstat(4, 0xFFFFFD7FFFDFE800)            = 0
>> >  0.0498    ioctl(4, TCGETA, 0xFFFFFD7FFFDFE880)        Err#25 ENOTTY
>> >  0.0500    read(4, "D1F2\r\nA09390 S c\0\0\0".., 18944)    = 18651
>> >
>> > 0.0514    stat("/opt/ha/lib/python2.6/site-packages/os",
>> 0xFFFFFD7FFFDFD320)
>> > Err#2 ENOENT
>> >  0.0516    stat("/opt/ha/lib/python2.6/site-packages/os",
>> > 0xFFFFFD7FFFDFD7B0) Err#2 ENOENT
>> >  0.0517    open("/opt/ha/lib/python2.6/site-packages/64/os.so",
>> O_RDONLY)
>> > Err#2 ENOENT
>> >  0.0519    stat("/opt/ha/lib/python2.6/site-packages/os",
>> > 0xFFFFFD7FFFDFD7B0) Err#2 ENOENT
>> >  0.0520    open("/opt/ha/lib/python2.6/site-packages/64/osmodule.so",
>> > O_RDONLY) Err#2 ENOENT
>> >  0.0521    stat("/opt/ha/lib/python2.6/site-packages/os",
>> > 0xFFFFFD7FFFDFD7B0) Err#2 ENOENT
>> >  0.0522    open("/opt/ha/lib/python2.6/site-packages/os.py", O_RDONLY)
>> Err#2
>> > ENOENT
>> >  0.0524    stat("/opt/ha/lib/python2.6/site-packages/os",
>> > 0xFFFFFD7FFFDFD7B0) Err#2 ENOENT
>> >  0.0525    open("/opt/ha/lib/python2.6/site-packages/os.pyc", O_RDONLY)
>> > Err#2 ENOENT
>> >  0.0527    stat("/usr/lib/python2.6/os", 0xFFFFFD7FFFDFD320) Err#2
>> ENOENT
>> >  0.0528    stat("/usr/lib/python2.6/os", 0xFFFFFD7FFFDFD7B0) Err#2
>> ENOENT
>> >  0.0529    open("/usr/lib/python2.6/64/os.so", O_RDONLY)    Err#2 ENOENT
>> >  0.0530    stat("/usr/lib/python2.6/os", 0xFFFFFD7FFFDFD7B0) Err#2
>> ENOENT
>> >  0.0532    open("/usr/lib/python2.6/64/osmodule.so", O_RDONLY) Err#2
>> ENOENT
>> >  0.0533    stat("/usr/lib/python2.6/os", 0xFFFFFD7FFFDFD7B0) Err#2
>> ENOENT
>> >  0.0534    open("/usr/lib/python2.6/os.py", O_RDONLY)    = 4
>> >  0.0536    fstat(4, 0xFFFFFD7FFFDFD770)            = 0
>> >  0.0537    open("/usr/lib/python2.6/os.pyc", O_RDONLY)    = 5
>> >  0.0538    fstat(5, 0xFFFFFD7FFFDFD5B0)            = 0
>> >  0.0539    fstat(5, 0xFFFFFD7FFFDFD4E0)            = 0
>> >  0.0540    ioctl(5, TCGETA, 0xFFFFFD7FFFDFD560)        Err#25 ENOTTY
>> >  0.0541    read(5, "D1F2\r\n9F9390 S c\0\0\0".., 26112)    = 25702
>> >
>> >  0.0568    stat("/opt/ha/lib/python2.6/site-packages/posixpath",
>> > 0xFFFFFD7FFFDFC000) Err#2 ENOENT
>> >  0.0570    stat("/opt/ha/lib/python2.6/site-packages/posixpath",
>> > 0xFFFFFD7FFFDFC490) Err#2 ENOENT
>> >  0.0572    open("/opt/ha/lib/python2.6/site-packages/64/posixpath.so",
>> > O_RDONLY) Err#2 ENOENT
>> >  0.0573    stat("/opt/ha/lib/python2.6/site-packages/posixpath",
>> > 0xFFFFFD7FFFDFC490) Err#2 ENOENT
>> >  0.0575
>> open("/opt/ha/lib/python2.6/site-packages/64/posixpathmodule.so",
>> > O_RDONLY) Err#2 ENOENT
>> >  0.0576    stat("/opt/ha/lib/python2.6/site-packages/posixpath",
>> > 0xFFFFFD7FFFDFC490) Err#2 ENOENT
>> >  0.0577    open("/opt/ha/lib/python2.6/site-packages/posixpath.py",
>> > O_RDONLY) Err#2 ENOENT
>> >  0.0579    stat("/opt/ha/lib/python2.6/site-packages/posixpath",
>> > 0xFFFFFD7FFFDFC490) Err#2 ENOENT
>> >  0.0580    open("/opt/ha/lib/python2.6/site-packages/posixpath.pyc",
>> > O_RDONLY) Err#2 ENOENT
>> >  0.0582    stat("/usr/lib/python2.6/posixpath", 0xFFFFFD7FFFDFC000)
>> Err#2
>> > ENOENT
>> >  0.0583    stat("/usr/lib/python2.6/posixpath", 0xFFFFFD7FFFDFC490)
>> Err#2
>> > ENOENT
>> >  0.0584    open("/usr/lib/python2.6/64/posixpath.so", O_RDONLY) Err#2
>> ENOENT
>> >  0.0585    stat("/usr/lib/python2.6/posixpath", 0xFFFFFD7FFFDFC490)
>> Err#2
>> > ENOENT
>> >  0.0587    open("/usr/lib/python2.6/64/posixpathmodule.so", O_RDONLY)
>> Err#2
>> > ENOENT
>> >  0.0588    stat("/usr/lib/python2.6/posixpath", 0xFFFFFD7FFFDFC490)
>> Err#2
>> > ENOENT
>> >  0.0589    open("/usr/lib/python2.6/posixpath.py", O_RDONLY) = 5
>> >  0.0591    fstat(5, 0xFFFFFD7FFFDFC450)            = 0
>> >  0.0592    open("/usr/lib/python2.6/posixpath.pyc", O_RDONLY) = 6
>> >  0.0593    fstat(6, 0xFFFFFD7FFFDFC290)            = 0
>> >
>> >
>> > And the list goes on.
>> >
>> > Hope there's a solution for this.
>> >
>> >
>> > Joe
>> >
>> >
>> > On Thu, Dec 11, 2014 at 10:26 PM, Dan McDonald <danmcd at omniti.com>
>> wrote:
>> >>
>> >>
>> >> > On Dec 11, 2014, at 4:00 PM, Joe Veliscos <joeveliscos at gmail.com>
>> wrote:
>> >> >
>> >> > Hi,
>> >> >
>> >> > I have an application working on Omnios r10 which depends on certain
>> >> > python libraries. There are certain environment variables in place
>> which
>> >> > point to the location of those libraries.
>> >> >
>> >> > I have updated the r10 machine to r12. The application now cannot be
>> >> > started with errors stating that the needed libraries cannot be
>> found in the
>> >> > given paths.
>> >>
>> >> Share the errors please?  I'll need more details.
>> >>
>> >> > Maybe somebody can tell me what the difference is in python version
>> >> > between the two omnios releases. Python -V gives me 2.6.8. on both
>> versions.
>> >>
>> >> We updated supplemental python libraries, which may contribute to what
>> >> you're seeing.  Also, we had not updated the "entire" metapackage on
>> 010 to
>> >> show we were actually running 2.6.8.
>> >>
>> >> Dan
>> >>
>> >
>> > _______________________________________________
>> > OmniOS-discuss mailing list
>> > OmniOS-discuss at lists.omniti.com
>> > http://lists.omniti.com/mailman/listinfo/omnios-discuss
>> >
>>
>
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>
>

-- 

Theo Schlossnagle

http://omniti.com/is/theo-schlossnagle
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141213/35fde131/attachment-0001.html>

From info at houseofancients.nl  Sat Dec 13 17:34:47 2014
From: info at houseofancients.nl (Floris van Essen ..:: House of Ancients Amstafs ::..)
Date: Sat, 13 Dec 2014 17:34:47 +0000
Subject: [OmniOS-discuss] LACP Omnios , igb and C3750
In-Reply-To: <1D812FAE-40BA-4464-8896-208E2FCF01DB@omniti.com>
References: <~B548633c30000.5486a0500000.0002.mml.3857832108@vEX01.mindstorm-internet.local>
	<356582D1FC91784992ABB4265A16ED487EF447DC@vEX01.mindstorm-internet.local>
	<DD4737A6-D508-4759-8F6D-3869F259A3D2@omniti.com>
	<356582D1FC91784992ABB4265A16ED487EF46909@vEX01.mindstorm-internet.local>
	<C77D681D-D52C-4278-9E45-ACDBAE6ED281@omniti.com>
	<~B548b22460000.548b52030004.0002.mml.372676210@vEX01.mindstorm-internet.local>
	<356582D1FC91784992ABB4265A16ED487F2CAE34@vEX01.mindstorm-internet.local>
	<356582D1FC91784992ABB4265A16ED487F2CCEBF@vEX01.mindstorm-internet.local>
	<1D812FAE-40BA-4464-8896-208E2FCF01DB@omniti.com>
Message-ID: <356582D1FC91784992ABB4265A16ED487F2CCF63@vEX01.mindstorm-internet.local>

Hi Dan,

Yes you did mention it,and I confirmed that is not the case.
System Configuration: Supermicro X7DB8
BIOS Configuration: Phoenix Technologies LTD 2.1c 07/04/2011
BMC Configuration: IPMI 1.0 (unknown)


Met vriendelijke groet / With kind regards,



???????????????Floris van Essen 

-----Oorspronkelijk bericht-----
Van: Dan McDonald [mailto:danmcd at omniti.com] 
Verzonden: zaterdag 13 december 2014 17:13
Aan: Floris van Essen ..:: House of Ancients Amstafs ::..
CC: omnios-discuss at lists.omniti.com
Onderwerp: Re: [OmniOS-discuss] LACP Omnios , igb and C3750

I thought I'd mentioned this on the thread, but does the machine attempt to use one of the IGB interfaces for IPMI as well?  We do not support dual-use NICs like that.  (Check your BIOS.)

Dan



Sent from my iPhone (typos, autocorrect, and all)

> On Dec 13, 2014, at 9:01 AM, Floris van Essen ..:: House of Ancients Amstafs ::.. <info at houseofancients.nl> wrote:
> 
> So i replaced the Igb adapter with a HP NC375t, loaded the drivers from HP,left cables and switch alone :
> 
> ntxn0       ntxn0       Ethernet       up       1000       full       BOUND       192.168.249.104       255.255.255.0       78:ac:c0:11:d4:88       1500       ok                 
> ntxn1       ntxn1       Ethernet       unknown       1000       full       static       unset       -       -       1500                        
> ntxn2       ntxn2       Ethernet       unknown       1000       full       static       unset       -       -       1500                        
> ntxn3       ntxn3       Ethernet       unknown       1000       full       static       unset       -       -       1500             
> 
> this all lead me to conclude that there is something wrong with the 
> IGB implementation as it sits Weather it be drivers, or something more 
> to the background..don't know, but would love to find out
> 
> Met vriendelijke groet / With kind regards,
> 
> 
> 
>                Floris van Essen
> 
> -----Oorspronkelijk bericht-----
> Van: Floris van Essen ..:: House of Ancients Amstafs ::.. 
> Verzonden: zaterdag 13 december 2014 11:38
> Aan: Dan McDonald
> CC: omnios-discuss at lists.omniti.com
> Onderwerp: Re: [OmniOS-discuss] LACP Omnios , igb and C3750
> 
> So,
> 
> To add to the weirdness.
> 
> Got ready to reinstall the whole machine, to set the interface back to DHCP, set it back to VLAN I could use for reinstalling, was about to clear the arp table, and then I noticed something very strange :
>    
>    Sh arp
>    Internet  x.x.x.104        52   a036.9f17.abc0  ARPA   Vlanx
> 
> Wait, that is the mac of my IGB interface...  did I miss something ?
>    
>    ifconfig
>    igb0: flags=1004843<UP,BROADCAST,RUNNING,MULTICAST,DHCP,IPv4> mtu 1500 index 13
>            inet 0.0.0.0 netmask ff000000
>            ether a0:36:9f:17:ab:c0
> 
> so it did do an ARP, did get an IP from DHCP server, but it's simply not registering on the Omnios box...
> This means hardware, and drivers are ok!
> 
> then I booted the machine using a KNOPPIX os, just to confirm my 
> suspicions about the hardware and got ip x.x.x.104 on IGB0
> 
> I think this is something within Omnios , however, would like some 
> pointers where to look
> 
> Met vriendelijke groet / With kind regards,
> 
> 
> 
>                Floris van Essen
> 
> -----Oorspronkelijk bericht-----
> Van: OmniOS-discuss [mailto:omnios-discuss-bounces at lists.omniti.com] Namens Floris van Essen ..:: House of Ancients Amstafs ::..
> Verzonden: vrijdag 12 december 2014 18:11
> Aan: Dan McDonald
> CC: omnios-discuss at lists.omniti.com
> Onderwerp: Re: [OmniOS-discuss] LACP Omnios , igb and C3750
> 
> 
> 
> 
> 
> 
>> On Dec 12, 2014, at 11:49 AM, Floris van Essen ..:: House of Ancients Amstafs ::.. <info at houseofancients.nl> wrote:
>> 
>> Hi Dann,
>> 
>> No problem, just happy you did :-)
>> 
>> Right, so as might remember I had to reinstall because I was running 
>> bloody R11 , but there was a problem upgrading , so just did a fresh install Had to skip r12 because I simply couldn't install it, as there was a installer issue with r12...
> 
>>> Weird.  I JUST updated the 012 install media thanks to illumos #5421, so you may want to try that again.
> 
> Did the installation 2 weeks ago, so guessing that that was before you did the changes.
> No problem, I'll update to stable next release :-)
> 
>> So here we are again, running bloody r13 , fully updated :-)
>> 
>> # dladm show-ether
>> LINK            PTYPE    STATE    AUTO  SPEED-DUPLEX                    PAUSE
>> e1000g1         current  up       yes   1G-f                            bi
>> e1000g0         current  up       yes   1G-f                            bi
>> igb0            current  up       yes   1G-f                            bi
>> igb1            current  up       yes   1G-f                            bi
>> igb2            current  up       yes   1G-f                            bi
>> igb3            current  up       yes   1G-f                            bi
>> # dladm show-link
>> LINK        CLASS     MTU    STATE    BRIDGE     OVER
>> e1000g1     phys      1500   up       --         --
>> e1000g0     phys      1500   up       --         --
>> aggr0       aggr      1500   up       --         e1000g0 e1000g1
>> igb0        phys      1500   up       --         --
>> igb1        phys      1500   up       --         --
>> igb2        phys      1500   up       --         --
>> igb3        phys      1500   up       --         --
> 
>>> Tis all looks sane.  You're showing 1Gig, full dupliex, and up.  This seems sane.
> 
>> # ifconfig -a
>> lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1
>>       inet 127.0.0.1 netmask ff000000
>> aggr0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2
>>       inet x.x.x.11 netmask ffffff00 broadcast x.x.x.255
>>       ether 0:30:48:d5:ec:94
> 
> And you have no problems pinging x.x.x.0/24 addresses?  Or do you?
> 
> Non, actually connecting on that interface with ssh
> 
>> igb0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 3
>>       inet 192.168.0.1 netmask ffffff00 broadcast 192.168.0.255
>>       ether a0:36:9f:2:c2:6c
>> igb1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 4
>>       inet 192.168.0.2 netmask ffffff00 broadcast 192.168.0.255
>>       ether a0:36:9f:2:c2:6d
>> igb2: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 5
>>       inet 192.168.0.3 netmask ffffff00 broadcast 192.168.0.255
>>       ether a0:36:9f:2:c2:6e
>> igb3: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 6
>>       inet 192.168.0.4 netmask ffffff00 broadcast 192.168.0.255
>>       ether a0:36:9f:2:c2:6f
> 
>>> Now I can totally see some potential confusion here.  You've four addrs all in the same netstack and all with the same prefix.  Now granted, they're different MAC addresses, but packets coming in on one interface may have their return traffic going out >>another.
> 
>>> If these are the problem, try using just one (igb0) for starters and "ifconfig igbX down" for 1, 2, and 3.
> 
> That is exactly what i did.. for each test brought the others down 
> Just to add, when doing a  SH ARP on my cisco's, don't see the macs 
> popping up, which is even more worring , no arps on a switch usually 
> is very bad news
> 
>>> Also, I need to ask --> are either of your e1000g0's shared IPMI/host links?  If so, disable the sharing feature in the BIOS.  We don't cope with shared-with-IPMI NICs.
> 
> They are not... besides.. the e1000G's ( and their aggr) are working fully as expected.
> To make it more interesting.. this was the same setup as I had on r11 
> , and then it did seem to work 100%
> 
> Dan
> 
> ...:: House of Ancients ::...
> American Staffordshire Terriers
> 
> +31-628-161-350
> +31-614-198-389
> Het Perk 48
> 4903 RB
> Oosterhout
> Netherlands
> www.houseofancients.nl
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
> ...:: House of Ancients ::...
> American Staffordshire Terriers
> 
> +31-628-161-350
> +31-614-198-389
> Het Perk 48
> 4903 RB
> Oosterhout
> Netherlands
> www.houseofancients.nl
> 
> ...:: House of Ancients ::...
> American Staffordshire Terriers
> 
> +31-628-161-350
> +31-614-198-389
> Het Perk 48
> 4903 RB
> Oosterhout
> Netherlands
> www.houseofancients.nl
...:: House of Ancients ::...
American Staffordshire Terriers

+31-628-161-350
+31-614-198-389
Het Perk 48
4903 RB
Oosterhout
Netherlands
www.houseofancients.nl

From tim at multitalents.net  Sat Dec 13 19:52:28 2014
From: tim at multitalents.net (Tim Rice)
Date: Sat, 13 Dec 2014 11:52:28 -0800 (PST)
Subject: [OmniOS-discuss] Python difference
In-Reply-To: <CACLsAptmZ9QFHeXXkZW3wigPnNywq=5SHx_+sg77TzWuxvJOGQ@mail.gmail.com>
References: <CAHNoB+1iJ=TNq9Z0sZ-WO-BMtx--tXcrXkVyWvcu3UfS-W+CCg@mail.gmail.com>
	<56C3D19D-B648-4F02-A7CA-BC24A31C385E@omniti.com>
	<CAHNoB+1pePnQuUNHSBKLN14975bqHPX1C6F1E7nciZjkE2qQDw@mail.gmail.com>
	<CALLZ=vV1FoK_XdTmqXRSetrnZ70bE_WdvS0uQa_SSNipxokFvg@mail.gmail.com>
	<CAHNoB+2YVfFvwT189oPzfJQX+VHX_RJR_4P8jHND6hSeW9QfXg@mail.gmail.com>
	<CACLsAptmZ9QFHeXXkZW3wigPnNywq=5SHx_+sg77TzWuxvJOGQ@mail.gmail.com>
Message-ID: <alpine.UW2.2.11.1412131114420.24872@server01.int.multitalents.net>

On Sat, 13 Dec 2014, Theo Schlossnagle wrote:

| A package (like ha) should either rely on the system python packages /
| versions it needs to work (thus preventing your upgrade to 012) or more
| reasonably rely on its own known-good python.  I don't believe OmniOS makes
| any release-to-release compatibility guarantees for python (or perl or java
| for that matter). The only reason they even exist in the core is because we
| have other core requirements that depend upon them.

Anyone coming from a UNIX background knowing that OmniOS has
it's roots in OpenSolaris would reasonably expect that something
depending on programs/libs found in /usr/bin,/lib,/usr/lib would
continue to work on the next release. If omniti chooses not to
have any compatibility guarantees for python (or perl or java),
they should be moved to a separate tree (perhaps /usr/omnios or
even /usr/omnios-private). Kind of like how Solaris put the unstable
bits in /usr/sfw.


-- 
Tim Rice				Multitalents
tim at multitalents.net



From jesus at omniti.com  Sat Dec 13 20:06:24 2014
From: jesus at omniti.com (Theo Schlossnagle)
Date: Sat, 13 Dec 2014 12:06:24 -0800
Subject: [OmniOS-discuss] Python difference
In-Reply-To: <alpine.UW2.2.11.1412131114420.24872@server01.int.multitalents.net>
References: <CAHNoB+1iJ=TNq9Z0sZ-WO-BMtx--tXcrXkVyWvcu3UfS-W+CCg@mail.gmail.com>
	<56C3D19D-B648-4F02-A7CA-BC24A31C385E@omniti.com>
	<CAHNoB+1pePnQuUNHSBKLN14975bqHPX1C6F1E7nciZjkE2qQDw@mail.gmail.com>
	<CALLZ=vV1FoK_XdTmqXRSetrnZ70bE_WdvS0uQa_SSNipxokFvg@mail.gmail.com>
	<CAHNoB+2YVfFvwT189oPzfJQX+VHX_RJR_4P8jHND6hSeW9QfXg@mail.gmail.com>
	<CACLsAptmZ9QFHeXXkZW3wigPnNywq=5SHx_+sg77TzWuxvJOGQ@mail.gmail.com>
	<alpine.UW2.2.11.1412131114420.24872@server01.int.multitalents.net>
Message-ID: <CACLsAps6uMsbc1ZqiLSqLEj3d9gO7vpQm57_fpsc1HizBOydoQ@mail.gmail.com>

Agreed.  I think we've had a considerable amount of regret that python was
not put into to a "system internals" path.

And I know I regret having Java and Perl in there at all.  It's been stated
a few times that as soon as the two bits that depend on Java get removed or
replaced that Java will go out the window.

Also, having used distros from every modern and many not so modern unicies,
I would say that any that provided perl or python in their distro did so
with horrible woe (at least in hindsight).

On Sat, Dec 13, 2014 at 11:52 AM, Tim Rice <tim at multitalents.net> wrote:
>
> On Sat, 13 Dec 2014, Theo Schlossnagle wrote:
>
> | A package (like ha) should either rely on the system python packages /
> | versions it needs to work (thus preventing your upgrade to 012) or more
> | reasonably rely on its own known-good python.  I don't believe OmniOS
> makes
> | any release-to-release compatibility guarantees for python (or perl or
> java
> | for that matter). The only reason they even exist in the core is because
> we
> | have other core requirements that depend upon them.
>
> Anyone coming from a UNIX background knowing that OmniOS has
> it's roots in OpenSolaris would reasonably expect that something
> depending on programs/libs found in /usr/bin,/lib,/usr/lib would
> continue to work on the next release. If omniti chooses not to
> have any compatibility guarantees for python (or perl or java),
> they should be moved to a separate tree (perhaps /usr/omnios or
> even /usr/omnios-private). Kind of like how Solaris put the unstable
> bits in /usr/sfw.
>
>
> --
> Tim Rice                                Multitalents
> tim at multitalents.net
>
>
>

-- 

Theo Schlossnagle

http://omniti.com/is/theo-schlossnagle
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141213/fee2338c/attachment.html>

From henson at acm.org  Sat Dec 13 21:53:58 2014
From: henson at acm.org (Paul B. Henson)
Date: Sat, 13 Dec 2014 13:53:58 -0800
Subject: [OmniOS-discuss] LACP Omnios , igb and C3750
In-Reply-To: <1D812FAE-40BA-4464-8896-208E2FCF01DB@omniti.com>
References: <~B548633c30000.5486a0500000.0002.mml.3857832108@vEX01.mindstorm-internet.local>
	<356582D1FC91784992ABB4265A16ED487EF447DC@vEX01.mindstorm-internet.local>
	<DD4737A6-D508-4759-8F6D-3869F259A3D2@omniti.com>
	<356582D1FC91784992ABB4265A16ED487EF46909@vEX01.mindstorm-internet.local>
	<C77D681D-D52C-4278-9E45-ACDBAE6ED281@omniti.com>
	<~B548b22460000.548b52030004.0002.mml.372676210@vEX01.mindstorm-internet.local>
	<356582D1FC91784992ABB4265A16ED487F2CAE34@vEX01.mindstorm-internet.local>
	<356582D1FC91784992ABB4265A16ED487F2CCEBF@vEX01.mindstorm-internet.local>
	<1D812FAE-40BA-4464-8896-208E2FCF01DB@omniti.com>
Message-ID: <20141213215358.GQ29549@bender.unx.csupomona.edu>

On Sat, Dec 13, 2014 at 11:12:35AM -0500, Dan McDonald wrote:
> I thought I'd mentioned this on the thread, but does the machine
> attempt to use one of the IGB interfaces for IPMI as well?  We do not
> support dual-use NICs like that.  (Check your BIOS.)

Hmm, I had thought that the "shared" IPMI port was invisible to the OS?
That there was an internal three-port switch, with one port connected to
the physical jack on the back of the server, and then the other two
wired internally to the NIC on the motherboard and the NIC on the BMC?

Do these boxes do something else that actually tweaks with the hardware
on the motherboard NIC when the BIOS is set to shared?


From doug at will.to  Sat Dec 13 22:07:40 2014
From: doug at will.to (Doug Hughes)
Date: Sat, 13 Dec 2014 17:07:40 -0500
Subject: [OmniOS-discuss] LACP Omnios , igb and C3750
In-Reply-To: <20141213215358.GQ29549@bender.unx.csupomona.edu>
References: <~B548633c30000.5486a0500000.0002.mml.3857832108@vEX01.mindstorm-internet.local>	<356582D1FC91784992ABB4265A16ED487EF447DC@vEX01.mindstorm-internet.local>	<DD4737A6-D508-4759-8F6D-3869F259A3D2@omniti.com>	<356582D1FC91784992ABB4265A16ED487EF46909@vEX01.mindstorm-internet.local>	<C77D681D-D52C-4278-9E45-ACDBAE6ED281@omniti.com>	<~B548b22460000.548b52030004.0002.mml.372676210@vEX01.mindstorm-internet.local>	<356582D1FC91784992ABB4265A16ED487F2CAE34@vEX01.mindstorm-internet.local>	<356582D1FC91784992ABB4265A16ED487F2CCEBF@vEX01.mindstorm-internet.local>	<1D812FAE-40BA-4464-8896-208E2FCF01DB@omniti.com>
	<20141213215358.GQ29549@bender.unx.csupomona.edu>
Message-ID: <548CB8AC.30205@will.to>


On 12/13/2014 4:53 PM, Paul B. Henson wrote:
> On Sat, Dec 13, 2014 at 11:12:35AM -0500, Dan McDonald wrote:
>> I thought I'd mentioned this on the thread, but does the machine
>> attempt to use one of the IGB interfaces for IPMI as well?  We do not
>> support dual-use NICs like that.  (Check your BIOS.)
> Hmm, I had thought that the "shared" IPMI port was invisible to the OS?
> That there was an internal three-port switch, with one port connected to
> the physical jack on the back of the server, and then the other two
> wired internally to the NIC on the motherboard and the NIC on the BMC?
>
> Do these boxes do something else that actually tweaks with the hardware
> on the motherboard NIC when the BIOS is set to shared?
>
That's the typical setup with IPMI NIC = shared. The OS is totally 
unaware that there's another device using the same hardware. The amazing 
thing is that while the OS is using the physical port at 1gbit the BMC 
is using it at 100mbits/sec on the same physical port, all without 
either knowing about the other. IMHO, the best way to do this is to set 
the IPMI to use a different VLAN than the host, but that's dependent on 
the features of your BMC and network hardware. (tagged to BMC, untagged 
to host).

The other options are 'dedicated' where there's a dedicated IPMI/BMC 
port, and 'failover' which has a dedicated port, but can failover to the 
motherboard/host port. (crazy: some of them even support bonding across 
the ports for the BMC; who thought this was a good idea?!)



From rt at steait.net  Sun Dec 14 13:27:39 2014
From: rt at steait.net (Rune Tipsmark)
Date: Sun, 14 Dec 2014 13:27:39 +0000
Subject: [OmniOS-discuss] latency spikes ~every hour
Message-ID: <1418563662763.71128@steait.net>

hi all,



All my vSphere (ESXi5.1) hosts experience a big spike in latency every hour or so.

I tested on Infiniband iSER and SRP and also 4Gbit FC and 8GBit FC. All exhibit the same behavior so I don't think its the connection that is causing this.

When I modify the arc_shrink_shift 10 (192GB Ram in the System) it helps a bit, the spikes are still with the same regularity but latency peaks at about ~5000ms for the most part. if I leave arc_shrink_shift at the default value they can be higher - up to 15000ms.

Looking at the latency as seen from the vSphere hosts the average is usually below 1ms for most datastores.



Any ideas what can cause this or what can be done to fix?

br,

Rune
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141214/64db7337/attachment.html>

From jimklimov at cos.ru  Sun Dec 14 15:28:48 2014
From: jimklimov at cos.ru (Jim Klimov)
Date: Sun, 14 Dec 2014 16:28:48 +0100
Subject: [OmniOS-discuss] LACP Omnios , igb and C3750
In-Reply-To: <548CB8AC.30205@will.to>
References: <~B548633c30000.5486a0500000.0002.mml.3857832108@vEX01.mindstorm-internet.local>
	<356582D1FC91784992ABB4265A16ED487EF447DC@vEX01.mindstorm-internet.local>
	<DD4737A6-D508-4759-8F6D-3869F259A3D2@omniti.com>
	<356582D1FC91784992ABB4265A16ED487EF46909@vEX01.mindstorm-internet.local>
	<C77D681D-D52C-4278-9E45-ACDBAE6ED281@omniti.com>
	<~B548b22460000.548b52030004.0002.mml.372676210@vEX01.mindstorm-internet.local>
	<356582D1FC91784992ABB4265A16ED487F2CAE34@vEX01.mindstorm-internet.local>
	<356582D1FC91784992ABB4265A16ED487F2CCEBF@vEX01.mindstorm-internet.local>
	<1D812FAE-40BA-4464-8896-208E2FCF01DB@omniti.com>
	<20141213215358.GQ29549@bender.unx.csupomona.edu>
	<548CB8AC.30205@will.to>
Message-ID: <62E80049-87C8-4B24-8C42-7B8AF13F2589@cos.ru>

13 ??????? 2014??. 23:07:40 CET, Doug Hughes <doug at will.to> ?????:
>
>On 12/13/2014 4:53 PM, Paul B. Henson wrote:
>> On Sat, Dec 13, 2014 at 11:12:35AM -0500, Dan McDonald wrote:
>>> I thought I'd mentioned this on the thread, but does the machine
>>> attempt to use one of the IGB interfaces for IPMI as well?  We do
>not
>>> support dual-use NICs like that.  (Check your BIOS.)
>> Hmm, I had thought that the "shared" IPMI port was invisible to the
>OS?
>> That there was an internal three-port switch, with one port connected
>to
>> the physical jack on the back of the server, and then the other two
>> wired internally to the NIC on the motherboard and the NIC on the
>BMC?
>>
>> Do these boxes do something else that actually tweaks with the
>hardware
>> on the motherboard NIC when the BIOS is set to shared?
>>
>That's the typical setup with IPMI NIC = shared. The OS is totally 
>unaware that there's another device using the same hardware. The
>amazing 
>thing is that while the OS is using the physical port at 1gbit the BMC 
>is using it at 100mbits/sec on the same physical port, all without 
>either knowing about the other. IMHO, the best way to do this is to set
>
>the IPMI to use a different VLAN than the host, but that's dependent on
>
>the features of your BMC and network hardware. (tagged to BMC, untagged
>
>to host).
>
>The other options are 'dedicated' where there's a dedicated IPMI/BMC 
>port, and 'failover' which has a dedicated port, but can failover to
>the 
>motherboard/host port. (crazy: some of them even support bonding across
>
>the ports for the BMC; who thought this was a good idea?!)
>
>
>_______________________________________________
>OmniOS-discuss mailing list
>OmniOS-discuss at lists.omniti.com
>http://lists.omniti.com/mailman/listinfo/omnios-discuss

Imho it varies with hardware and firmware. Recent supermicros apparently offer, among other configurable options, failover between a dedicated ipmi port (deemed more reliable) and the connection shared over a port of the server (saves on cabling) if both ports have a cable. According to forum critique, this implementation is currently prone to failure if the datacenter takes time to power on and for the switches (STP) to converge and begin passing frames. The ipmi module sticks to one choice it made during bootup and no longer fails over if that link goes down. Something of the sort, they say.

What i did have experience with was with sun x2100, where they had 2*bge + 2*nge (less reliable in practice under load) interfaces, and one of the better ones was shared with ipmi, and a reboot of the host using the interface caused the ipmi module to lose connection. So essentially that port was to be left unconfigured in the OS, for remote management to remain usable when you need it.

Jim
--
Typos courtesy of K-9 Mail on my Samsung Android

From rt at steait.net  Sun Dec 14 15:44:57 2014
From: rt at steait.net (Rune Tipsmark)
Date: Sun, 14 Dec 2014 15:44:57 +0000
Subject: [OmniOS-discuss] Fibre Target problems
In-Reply-To: <5415604A.7030408@gmail.com>
References: <541155D5.8080406@gmail.com>
	<562C2ABFE2448D4E95E8910A85488708260B0262@HEIMAT.cm3c.local>,
	<5415604A.7030408@gmail.com>
Message-ID: <1418571900149.76065@steait.net>

did you ever find a solution?
I have the same problem on a SuperMicro based system... FC drops and it causes Windows to loose connection and copying files fails... 
br,
Rune
________________________________________
From: OmniOS-discuss <omnios-discuss-bounces at lists.omniti.com> on behalf of Mark <mark0x01 at gmail.com>
Sent: Sunday, September 14, 2014 11:30 AM
To: omnios-discuss at lists.omniti.com
Subject: Re: [OmniOS-discuss] Fibre Target problems

On 11/09/2014 9:25 p.m., OSN | Marian Fischer wrote:
> Hi,
>
> do you have Sync=disabled in ZFS / ZPOOL settings?
> If not, this can cause the slow speed ...

No, but I didn't with OI either, and it has no issues achieving 400+
Mbytes/sec and doesn't have link loss issues.

>
> Mit besten Gruessen
>
> Marian Fischer
> --
> OSN Online Service Nuernberg GmbH             http://www.osn.de
> Bucher Str. 78                                        Tel: 0911/39905-0
> 90408 Nuernberg                               Fax: 0911/39905-55
> HRB 15022 Nuernberg, Ust-Id: DE189301263      GF: Joerg Goltermann
>
> -----Urspr?ngliche Nachricht-----
> Von: OmniOS-discuss [mailto:omnios-discuss-bounces at lists.omniti.com] Im
> Auftrag von Mark
> Gesendet: Donnerstag, 11. September 2014 09:57
> An: omnios-discuss at lists.omniti.com
> Betreff: [OmniOS-discuss] Fibre Target problems
>
> I have a Supermicro X7DWN+ based server with 24 x 4Tb SAS disks on a LSI
> 6G IT mode hba.
>
> After configuring a bunch of 10Tb Luns and presenting to a Win2012
> server, write is very slow < 6 Mb/sec, and writing causes the Fc link to
> drop repeatedly.
> Reads reach about 400Mb/sec.
>
> OmniOS version is r151006_057.
>
> I've tried two different 4G and an 8G QLogic adapters, via switch or
> direct, but there is no change in behaviour. Only 1 HBA and path.
>
> The odd thing is the exact same hardware and os setup (fct, zpool etc.)
> works well with OI or Solaris 11/11, with writes getting around 4-500
> Mb/sec.
> I had to start with OI147 and work up, as the later text installer is
> very buggy. Upgrading OI with qlt installed fails.
>
> I'm at a bit of a loss as to a likely cause.
>
> Anyone have any pointers ?
>
>
> Some details and log:
>
> HBA Port WWN: 2100001b320aba27
>           Port Mode: Target
>           Port ID: 10100
>           OS Device Name: Not Applicable
>           Manufacturer: QLogic Corp.
>           Model: QLE2460
>           Firmware Version: 5.2.1
>           FCode/BIOS Version: N/A
>           Serial Number: not available
>           Driver Name: COMSTAR QLT
>           Driver Version: 20100505-1.05
>           Type: F-port
>           State: online
>           Supported Speeds: 1Gb 2Gb 4Gb
>           Current Speed: 4Gb
>           Node WWN: 2000001b320aba27
>           Link Error Statistics:
>                   Link Failure Count: 0
>                   Loss of Sync Count: 0
>                   Loss of Signal Count: 0
>                   Primitive Seq Protocol Error Count: 0
>                   Invalid Tx Word Count: 4
>                   Invalid CRC Count: 0
>
>
>
> prtconf
>
> value='ISP2432-based 4Gb Fibre Channel to PCI Express HBA'
> name='subsystem-name' type=string items=1
> value='unknown subsystem'
> Device Minor Nodes:dev=(3,1)
> dev_path=/pci at 0,0/pci8086,4027 at 7/pci8086,3500 at 0/pci8086,3510 at 0/pci1077,137 at 0
> :qlt1
>
> echo "*stmf_trace_buf/s" | mdb -k
>
> 0xffffff090f7c0000:             qlt1,0:0001318: iport is ffffff090f1921b8
>
> :0003718: Imported the LU 600144f09cdd9224000053f56a5c0005
> :0003719: Imported the LU 600144f09cdd9224000053f56a5d0006
> :0003721: Imported the LU 600144f09cdd9224000053f56a5d0007
> :0003722: Imported the LU 600144f09cdd9224000053f56a5e0008
> :0003725: Imported the LU 600144f09cdd9224000053f56a5e0009
> :0003726: Imported the LU 600144f09cdd9224000053f56a5f000a
> :0003728: Imported the LU 600144f09cdd9224000053f56a5f000b
> qlt1,0:0003815: Async event 8010 mb1=f8e8 mb2=c108, mb3=0, mb5=3362, mb6=0
> qlt1,0:0003815: Async event 8011 mb1=3 mb2=c108, mb3=0, mb5=3362, mb6=0
> qlt1,0:0003815: port state change from 0 to e
> qlt1,0:0003815: Async event 8014 mb1=ffff mb2=6, mb3=0, mb5=3362, mb6=0
> qlt1,0:0003815: Posting unsol ELS 3 (PLOGI) rp_id=e8 lp_id=ef
> qlt1,0:0003815: Rcvd PLOGI with wrong lportid ef, expecting 0. Killing ELS.
> qlt1,0:0003815: port state change from e to 4
> qlt1,0:0004216: Posting unsol ELS 3 (PLOGI) rp_id=e8 lp_id=ef
> qlt1,0:0004216: Processing unsol ELS 3 (PLOGI) rp_id=e8
> qlt1,0:0004216: Posting unsol ELS 20 (PRLI) rp_id=e8 lp_id=ef
> qlt1,0:0004216: Processing unsol ELS 20 (PRLI) rp_id=e8
> qlt1,0:7247736: Posting unsol ELS 5 (LOGO) rp_id=e8 lp_id=ef
> qlt1,0:7247736: Processing unsol ELS 5 (LOGO) rp_id=e8
> qlt1,0:7247736: handling LOGO rp_id e8. Triggering cleanup
> :7248836: fct_port_shutdown: port-ffffff090f1920b8, fct_process_logo:
> unable to clean up I/O. iport-ffffff090f1921b8, icmd-ffffff0931089a00
> qlt1,0:7248836: port state change from 4 to 11
> :7248836: fct_port_shutdown: port-ffffff090f1920b8, fct_process_logo:
> unable to clean up I/O. iport-ffffff090f1921b8, icmd-ffffff0931089a00
> qlt1,0:7248836: port state change from 11 to 11
> :7248836: fct_port_shutdown: port-ffffff090f1920b8, fct_process_logo:
> unable to clean up I/O. iport-ffffff090f1921b8, icmd-ffffff0931089a00
> qlt1,0:7248936: port state change from 11 to 11
> :7248936: fct_port_shutdown: port-ffffff090f1920b8, fct_process_logo:
> unable to clean up I/O. iport-ffffff090f1921b8, icmd-ffffff0931089a00
> qlt1,0:7249036: port state change from 11 to 11
> :7249036: fct_port_shutdown: port-ffffff090f1920b8, fct_process_logo:
> unable to clean up I/O. iport-ffffff090f1921b8, icmd-ffffff0931089a00
> qlt1,0:7249136: port state change from 11 to 11
> qlt1,0:7249136: Killing ELS 5 cond 1
> qlt1,0:7249136: Processing sol ELS 5 (LOGO) rp_id=e8
> qlt1,0:7249136: handling LOGO rp_id e8. Triggering cleanup
> qlt1,0:7249236: port state change from 11 to 11
> qlt1,0:7249336: port state change from 11 to 11
> qlt1,0:7249336: port state change from 11 to 11
> qlt1,0:7249436: port state change from 11 to 11
> qlt1,0:7249536: port state change from 11 to 11
> qlt1,0:7249536: handling LOGO rp_id e8, waiting for cmds to drain
> qlt1,0:7249636: port state change from 11 to 11
> qlt1,0:7249736: port state change from 11 to 11
> qlt1,0:7249836: port state change from 11 to 11
> qlt1,0:7249936: port state change from 11 to 11
> qlt1,0:7250036: port state change from 11 to 11
> qlt1,0:7250136: port state change from 11 to 11
> qlt1,0:7250236: port state change from 11 to 11
> :7250236: fct_port_shutdown: port-ffffff090f1920b8, fct_process_logo:
> unable to clean up I/O. iport-ffffff090f1921b8, icmd-ffffff091c5549d0
> qlt1,0:7250336: port state change from 11 to 11
> :7250336: fct_port_shutdown: port-ffffff090f1920b8, fct_process_logo:
> unable to clean up I/O. iport-ffffff090f1921b8, icmd-ffffff091c5549d0
> qlt1,0:7250336: port state change from 11 to 11
> :7250336: fct_port_shutdown: port-ffffff090f1920b8, fct_process_logo:
> unable to clean up I/O. iport-ffffff090f1921b8, icmd-ffffff091c5549d0
> qlt1,0:7250436: port state change from 11 to 11
> :7250436: fct_port_shutdown: port-ffffff090f1920b8, fct_process_logo:
> unable to clean up I/O. iport-ffffff090f1921b8, icmd-ffffff091c5549d0
> qlt1,0:7250536: port state change from 11 to 11
> :7250536: fct_port_shutdown: port-ffffff090f1920b8, fct_process_logo:
> unable to clean up I/O. iport-ffffff090f1921b8, icmd-ffffff091c5549d0
> qlt1,0:7250636: port state change from 11 to 11
> :7250636: fct_port_shutdown: port-ffffff090f1920b8, fct_process_logo:
> unable to clean up I/O. iport-ffffff090f1921b8, icmd-ffffff091c5549d0
> qlt1,0:7250736: port state change from 11 to 11
> :7250736: fct_port_shutdown: port-ffffff090f1920b8, fct_process_logo:
> unable to clean up I/O. iport-ffffff090f1921b8, icmd-ffffff091c5549d0
> qlt1,0:7250836: port state change from 11 to 11
> :7250836: fct_port_shutdown: port-ffffff090f1920b8, fct_process_logo:
> unable to clean up I/O. iport-ffffff090f1921b8, icmd-ffffff091c5549d0
> qlt1,0:7250936: port state change from 11 to 11
> :7250936: fct_port_shutdown: port-ffffff090f1920b8, fct_process_logo:
> unable to clean up I/O. iport-ffffff090f1921b8, icmd-ffffff091c5549d0
> qlt1,0:7251136: port state change from 11 to 11
> :7251136: fct_port_shutdown: port-ffffff090f1920b8, fct_process_logo:
> unable to clean up I/O. iport-ffffff090f1921b8, icmd-ffffff091c5549d0
> qlt1,0:7251236: port state change from 11 to 11
>
>
>
>
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>
>
>
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>

_______________________________________________
OmniOS-discuss mailing list
OmniOS-discuss at lists.omniti.com
http://lists.omniti.com/mailman/listinfo/omnios-discuss

From richard.elling at richardelling.com  Sun Dec 14 15:48:10 2014
From: richard.elling at richardelling.com (Richard Elling)
Date: Sun, 14 Dec 2014 07:48:10 -0800
Subject: [OmniOS-discuss] hangs on reboot
In-Reply-To: <28B0569C-BB7A-47B2-9F73-90EB2C1D56F8@omniti.com>
References: <1418336750819.83182@steait.net>
	<893101d01592$59168140$0b4383c0$@acm.org>
	<28B0569C-BB7A-47B2-9F73-90EB2C1D56F8@omniti.com>
Message-ID: <1372E447-00C5-4BA6-953F-154178050022@RichardElling.com>


> On Dec 11, 2014, at 2:40 PM, Dan McDonald <danmcd at omniti.com> wrote:
> 
> Thanks for the fast reboot catch!
> 
> Yes, were contemplating it for '014 AND for upstreaming.  Not sure yet if it'll happen, but this thread makes a compelling case.

FWIW, even Oracle has not been successful in getting fastboot to work
consistently in x86. We always disable it, as do some other illumos distros.
Make your life easier, disable by default.


 -- richard

> 
> Dan
> 
> Sent from my iPhone (typos, autocorrect, and all)
> 
> On Dec 11, 2014, at 5:32 PM, Paul B. Henson <henson at acm.org> wrote:
> 
>>> Rune Tipsmark
>>> Sent: Thursday, December 11, 2014 2:26 PM
>>> 
>>> I got a bunch (3) installations of omnios on SuperMicro hardware and all 3
>>> have issues rebooting. They simply hang and never ever reboot.
>> 
>> Disable fast reboot:
>> 
>> svccfg -s "system/boot-config:default" setprop
>> config/fastreboot_default=false
>> svccfg -s "system/boot-config:default" setprop
>> config/fastreboot_onpanic=false
>> svcadm refresh svc:/system/boot-config:default
>> 
>> Some systems (it seems particularly common on supermicro hardware) tend to
>> wedge up with fast reboot enabled. If you want to verify this is the issue
>> before changing the configuration, try 'reboot -p' which should do a one
>> time regular reboot without changing the default.
>> 
>> Dan, there was some talk of making fast reboot disabled the default, and
>> having people who wanted it need to enable it, rather than the other way
>> around? Any thoughts?
>> 
>> 
>> _______________________________________________
>> OmniOS-discuss mailing list
>> OmniOS-discuss at lists.omniti.com
>> http://lists.omniti.com/mailman/listinfo/omnios-discuss
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss

From henson at acm.org  Mon Dec 15 01:36:59 2014
From: henson at acm.org (Paul B. Henson)
Date: Sun, 14 Dec 2014 17:36:59 -0800
Subject: [OmniOS-discuss] LACP Omnios , igb and C3750
In-Reply-To: <548CB8AC.30205@will.to>
References: <356582D1FC91784992ABB4265A16ED487EF447DC@vEX01.mindstorm-internet.local>
	<DD4737A6-D508-4759-8F6D-3869F259A3D2@omniti.com>
	<356582D1FC91784992ABB4265A16ED487EF46909@vEX01.mindstorm-internet.local>
	<C77D681D-D52C-4278-9E45-ACDBAE6ED281@omniti.com>
	<~B548b22460000.548b52030004.0002.mml.372676210@vEX01.mindstorm-internet.local>
	<356582D1FC91784992ABB4265A16ED487F2CAE34@vEX01.mindstorm-internet.local>
	<356582D1FC91784992ABB4265A16ED487F2CCEBF@vEX01.mindstorm-internet.local>
	<1D812FAE-40BA-4464-8896-208E2FCF01DB@omniti.com>
	<20141213215358.GQ29549@bender.unx.csupomona.edu>
	<548CB8AC.30205@will.to>
Message-ID: <20141215013659.GZ29549@bender.unx.csupomona.edu>

On Sat, Dec 13, 2014 at 05:07:40PM -0500, Doug Hughes wrote:

> That's the typical setup with IPMI NIC = shared. The OS is totally 
> unaware that there's another device using the same hardware.

Yeah, that's why I'm confused about Dan saying omnios doesn't support
it, I'm not sure how it would know it was happening. Unless newer Intel
chipsets do something weird to make it happen rather than having a
separate mini-switch.

> The amazing 
> thing is that while the OS is using the physical port at 1gbit the BMC 
> is using it at 100mbits/sec on the same physical port, all without 
> either knowing about the other.

If you think of it as an embedded 3 port switch it's not that amazing :).

From henson at acm.org  Mon Dec 15 04:30:37 2014
From: henson at acm.org (Paul B. Henson)
Date: Sun, 14 Dec 2014 20:30:37 -0800
Subject: [OmniOS-discuss] state of building illumos-gate on omnios-stable
Message-ID: <20141215043037.GA29549@bender.unx.csupomona.edu>

One of my winter break plans is to get an illumos-gate dev environment
going again :). A while back there was some talk of possibly putting up
some wiki pages on configuring omnios-stable to build upstream
illumos-gate, but I still see only the "Setting Up A Basic Dev
Environment" at the moment.

I found https://docu.blackdot.be/snipets/solaris/build-zone, which talks
about how to get an OmniOS r151012 zone configured for illumos
development, and seems like a good starting point. Just wondering if
anybody had any further advice or possible hangups to keep an eye out
for, or if maybe there was some official omnios wiki page about to be
pushed out ;).

Thanks much...


From danmcd at omniti.com  Mon Dec 15 04:38:45 2014
From: danmcd at omniti.com (Dan McDonald)
Date: Sun, 14 Dec 2014 23:38:45 -0500
Subject: [OmniOS-discuss] state of building illumos-gate on omnios-stable
In-Reply-To: <20141215043037.GA29549@bender.unx.csupomona.edu>
References: <20141215043037.GA29549@bender.unx.csupomona.edu>
Message-ID: <B4CE795E-1991-4685-94CB-EFF84A4C1279@omniti.com>

Upstream needs several changes.  See http://kebe.com/~danmcd/webrevs/for-omnios

Dan

Sent from my iPhone (typos, autocorrect, and all)

> On Dec 14, 2014, at 11:30 PM, Paul B. Henson <henson at acm.org> wrote:
> 
> One of my winter break plans is to get an illumos-gate dev environment
> going again :). A while back there was some talk of possibly putting up
> some wiki pages on configuring omnios-stable to build upstream
> illumos-gate, but I still see only the "Setting Up A Basic Dev
> Environment" at the moment.
> 
> I found https://docu.blackdot.be/snipets/solaris/build-zone, which talks
> about how to get an OmniOS r151012 zone configured for illumos
> development, and seems like a good starting point. Just wondering if
> anybody had any further advice or possible hangups to keep an eye out
> for, or if maybe there was some official omnios wiki page about to be
> pushed out ;).
> 
> Thanks much...
> 
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss

From mark0x01 at gmail.com  Mon Dec 15 06:19:18 2014
From: mark0x01 at gmail.com (Mark)
Date: Mon, 15 Dec 2014 19:19:18 +1300
Subject: [OmniOS-discuss] Fibre Target problems
In-Reply-To: <1418571900149.76065@steait.net>
References: <541155D5.8080406@gmail.com>	<562C2ABFE2448D4E95E8910A85488708260B0262@HEIMAT.cm3c.local>,
	<5415604A.7030408@gmail.com> <1418571900149.76065@steait.net>
Message-ID: <548E7D66.6080208@gmail.com>

On 15/12/2014 4:44 a.m., Rune Tipsmark wrote:
> did you ever find a solution?
> I have the same problem on a SuperMicro based system... FC drops and it causes Windows to loose connection and copying files fails...
> br,
> Rune

I installed an older OpenIndiana version to work around the issue.

One thing I didn't check was if the fc target cache was disabled (may be 
the current default), so may be worth checking that.

Mark.


From henson at acm.org  Mon Dec 15 08:33:40 2014
From: henson at acm.org (Paul B. Henson)
Date: Mon, 15 Dec 2014 00:33:40 -0800
Subject: [OmniOS-discuss] state of building illumos-gate on omnios-stable
In-Reply-To: <B4CE795E-1991-4685-94CB-EFF84A4C1279@omniti.com>
References: <20141215043037.GA29549@bender.unx.csupomona.edu>
	<B4CE795E-1991-4685-94CB-EFF84A4C1279@omniti.com>
Message-ID: <20141215083340.GB29549@bender.unx.csupomona.edu>

On Sun, Dec 14, 2014 at 11:38:45PM -0500, Dan McDonald wrote:
> Upstream needs several changes.  See http://kebe.com/~danmcd/webrevs/for-omnios

Thanks for the pointer. As I recall the last thread on this subject, I
thought it was possible to build upstream as-is with the caveat you
needed to disable building one or two components and that lint was
noisier than one would like?

Hmm, or maybe I'm misremembering, and that was the state if these not as
yet accepted upstream patches were actually committed.

How's progress on pushing that upstream :)? There's a handful of things
I'd really like to do but I just can't bring myself to spend the
time to spin up an OI box <sigh>. Is my impression that stable OI is
dead and hipster is a never-ending alpha mistaken? If not, I'm surprised
there isn't more interest in getting the gate to build on a stable and
well maintained illumos distribution 8-/.

Thanks...


From richard at netbsd.org  Mon Dec 15 10:50:08 2014
From: richard at netbsd.org (Richard PALO)
Date: Mon, 15 Dec 2014 11:50:08 +0100
Subject: [OmniOS-discuss] state of building illumos-gate on omnios-stable
In-Reply-To: <20141215043037.GA29549@bender.unx.csupomona.edu>
References: <20141215043037.GA29549@bender.unx.csupomona.edu>
Message-ID: <m6mecf$met$1@ger.gmane.org>

Le 15/12/14 05:30, Paul B. Henson a ?crit :
> One of my winter break plans is to get an illumos-gate dev environment
> going again :). A while back there was some talk of possibly putting up
> some wiki pages on configuring omnios-stable to build upstream
> illumos-gate, but I still see only the "Setting Up A Basic Dev
> Environment" at the moment.
>
> I found https://docu.blackdot.be/snipets/solaris/build-zone, which talks
> about how to get an OmniOS r151012 zone configured for illumos
> development, and seems like a good starting point. Just wondering if
> anybody had any further advice or possible hangups to keep an eye out
> for, or if maybe there was some official omnios wiki page about to be
> pushed out ;).
>
> Thanks much...
>

One possibility, that is if you use what you build, is to start from 
omnios bloody.

For example,  git remote -v (doctored up for <your github id>)
> illumos-joyent  git://github.com/joyent/illumos-joyent (fetch)
> illumos-joyent  git://github.com/joyent/illumos-joyent (push)
> illumos-omnios  git://github.com/omniti-labs/illumos-omnios.git (push)
> illumos-omnios  git://github.com/omniti-labs/illumos-omnios.git (fetch)
> origin  ssh://git at github.com/<your github id>/illumos-gate (fetch)
> origin  ssh://git at github.com/<your github id>/illumos-gate (push)
> upstream        git://github.com/illumos/illumos-gate (fetch)
> upstream        git://github.com/illumos/illumos-gate (push)

by having your development branches rebased or merged from
illumos-omnios, you can now test with
'pkg update -v --be-name=<test BE name>'

For upstream, you can then create a branch using upstream and then
cherry picking your patchset.

I've been doing this since summer after dropping hipster.

For myself, my diffs for illumos.sh and nightly.sh:

> richard at omnis:/home/richard/src/illumos-gate$ diff illumos.sh usr/src/tools/env/illumos.sh
> 47c47
> < export NIGHTLY_OPTIONS='-FnCDAmprt'
> ---
> > export NIGHTLY_OPTIONS='-FnCDAlmprt'
> 60c60
> < export GATE='illumos-gate'
> ---
> > export GATE='testws'
> 162,164d161
> < if [[ -d "$CODEMGR_WS/.git" ]]; then
> < export VERSION="$GATE-$(git log -1 --format=%h)"
> < else
> 166d162
> < fi
> 235d230
> < # export ENABLE_SMB_PRINTING='#'
> 237,244d231
> <
> < export GCC_ROOT=/opt/gcc-4.4.4
> < export CW_NO_SHADOW=1
> < export ONNV_BUILDNUM=151013
> < export RELEASE_DATE=`git log --pretty=format:%cd --date=short -n 1 | tr - .`
> < export i386_LINT=/opt/SUNWspro/bin/lint
> < export amd64_LINT=/opt/SUNWspro/bin/lint
> < export MULTI_PROTO=yes

and
> richard at omnis:/home/richard/src/illumos-gate$ diff nightly.sh usr/src/tools/scripts/nightly.sh
> 2159,2161c2159,2161
> <       checkpaths $arg $checkroot > $SRC/check-paths.out 2>&1
> <       if [[ -s $SRC/check-paths.out ]]; then
> <               tee -a $LOGFILE < $SRC/check-paths.out >> $mail_msg_file
> ---
> >       checkpaths $arg $checkroot > $SRC/checkpaths.out 2>&1
> >       if [[ -s $SRC/checkpaths.out ]]; then
> >               tee -a $LOGFILE < $SRC/checkpaths.out >> $mail_msg_file

(I also reverted .gitignore from illumos-omnios to use upstream...)

If you have any particular questions about my config, offline is perhaps 
best.

It will be great when illumos-omnios and upstream converge 'au maximum'.


From sjorge+ml at blackdot.be  Mon Dec 15 12:26:02 2014
From: sjorge+ml at blackdot.be (Jorge Schrauwen)
Date: Mon, 15 Dec 2014 13:26:02 +0100
Subject: [OmniOS-discuss] state of building illumos-gate on omnios-stable
In-Reply-To: <20141215043037.GA29549@bender.unx.csupomona.edu>
References: <20141215043037.GA29549@bender.unx.csupomona.edu>
Message-ID: <44531b95aa7d9a09499e71edd48d2bd9@blackdot.be>

Hi Paul,

If while setting up your env you found obvious things lacking on 
https://docu.blackdot.be/snipets/solaris/build-zone, feel free to drop 
me a line on #omnios (sjorge) or here on the mailing list and I will 
update the page.

Other than that, good luck!

Regards

Jorge


On 2014-12-15 05:30, Paul B. Henson wrote:
> One of my winter break plans is to get an illumos-gate dev environment
> going again :). A while back there was some talk of possibly putting up
> some wiki pages on configuring omnios-stable to build upstream
> illumos-gate, but I still see only the "Setting Up A Basic Dev
> Environment" at the moment.
> 
> I found https://docu.blackdot.be/snipets/solaris/build-zone, which 
> talks
> about how to get an OmniOS r151012 zone configured for illumos
> development, and seems like a good starting point. Just wondering if
> anybody had any further advice or possible hangups to keep an eye out
> for, or if maybe there was some official omnios wiki page about to be
> pushed out ;).
> 
> Thanks much...
> 
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss

From rt at steait.net  Mon Dec 15 12:27:39 2014
From: rt at steait.net (Rune Tipsmark)
Date: Mon, 15 Dec 2014 12:27:39 +0000
Subject: [OmniOS-discuss] latency spikes ~every hour
In-Reply-To: <1418563662763.71128@steait.net>
References: <1418563662763.71128@steait.net>
Message-ID: <1418646462467.87062@steait.net>

ok I removed some of my SLOG devices and currently I am only using a single SLOG (no mirror or anything) and no spikes seen since.

I wonder why multiple SLOG devices would cause this.

br.

Rune

________________________________
From: OmniOS-discuss <omnios-discuss-bounces at lists.omniti.com> on behalf of Rune Tipsmark <rt at steait.net>
Sent: Sunday, December 14, 2014 2:27 PM
To: omnios-discuss at lists.omniti.com
Subject: [OmniOS-discuss] latency spikes ~every hour


hi all,



All my vSphere (ESXi5.1) hosts experience a big spike in latency every hour or so.

I tested on Infiniband iSER and SRP and also 4Gbit FC and 8GBit FC. All exhibit the same behavior so I don't think its the connection that is causing this.

When I modify the arc_shrink_shift 10 (192GB Ram in the System) it helps a bit, the spikes are still with the same regularity but latency peaks at about ~5000ms for the most part. if I leave arc_shrink_shift at the default value they can be higher - up to 15000ms.

Looking at the latency as seen from the vSphere hosts the average is usually below 1ms for most datastores.



Any ideas what can cause this or what can be done to fix?

br,

Rune
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141215/364a8335/attachment-0001.html>

From rt at steait.net  Mon Dec 15 12:45:15 2014
From: rt at steait.net (Rune Tipsmark)
Date: Mon, 15 Dec 2014 12:45:15 +0000
Subject: [OmniOS-discuss] Fibre Target problems
In-Reply-To: <548E7D66.6080208@gmail.com>
References: <541155D5.8080406@gmail.com>
	<562C2ABFE2448D4E95E8910A85488708260B0262@HEIMAT.cm3c.local>,
	<5415604A.7030408@gmail.com>
	<1418571900149.76065@steait.net>,<548E7D66.6080208@gmail.com>
Message-ID: <1418647518088.75378@steait.net>

where do you check that?
br,
Rune
________________________________________
From: Mark <mark0x01 at gmail.com>
Sent: Monday, December 15, 2014 7:19 AM
To: Rune Tipsmark; omnios-discuss at lists.omniti.com
Subject: Re: [OmniOS-discuss] Fibre Target problems

On 15/12/2014 4:44 a.m., Rune Tipsmark wrote:
> did you ever find a solution?
> I have the same problem on a SuperMicro based system... FC drops and it causes Windows to loose connection and copying files fails...
> br,
> Rune

I installed an older OpenIndiana version to work around the issue.

One thing I didn't check was if the fc target cache was disabled (may be
the current default), so may be worth checking that.

Mark.


From alexmcwhirter at vantagetitle.com  Thu Dec 11 00:45:48 2014
From: alexmcwhirter at vantagetitle.com (Alex McWhirter)
Date: Wed, 10 Dec 2014 19:45:48 -0500
Subject: [OmniOS-discuss] Change Zone Hostname
Message-ID: <1C434EF5-8635-4DBA-8317-45DEA492A920@vantagetitle.com>

I have a small cluster of servers hosting zones over multiple VLAN?s. I would like to name the zones so that they don?t run into naming collisions. For example, instead of ?web0? i would like to name them ?2049_web0? and ?2050_web0?. The problem is that this set?s the hostname to ?XXXX_web0?. Is there a way to change the hostname to be something different than the zone name?

I had a look at /etc/nodname and it is an empty file when used in a zone.

From jorge at blackdot.be  Fri Dec 12 08:10:53 2014
From: jorge at blackdot.be (Jorge Schrauwen)
Date: Fri, 12 Dec 2014 09:10:53 +0100
Subject: [OmniOS-discuss] hangs on reboot
In-Reply-To: <28B0569C-BB7A-47B2-9F73-90EB2C1D56F8@omniti.com>
References: <1418336750819.83182@steait.net>
	<893101d01592$59168140$0b4383c0$@acm.org>
	<28B0569C-BB7A-47B2-9F73-90EB2C1D56F8@omniti.com>
Message-ID: <ad985b1381f8ddcb8d6ead7fc548c40b@blackdot.be>

SuperMicro boards seem quite popular too... anybody ever figure out why 
they are hanging on fast reboot? I have the same issue but it only hangs 
about 1/3 of the time. (Fastreboot is now off though)

---
~ sjorge

On 2014-12-11 23:40, Dan McDonald wrote:
> Thanks for the fast reboot catch!
> 
> Yes, were contemplating it for '014 AND for upstreaming.  Not sure yet
> if it'll happen, but this thread makes a compelling case.
> 
> Dan
> 
> Sent from my iPhone (typos, autocorrect, and all)
> 
> On Dec 11, 2014, at 5:32 PM, Paul B. Henson <henson at acm.org> wrote:
> 
>>> Rune Tipsmark
>>> Sent: Thursday, December 11, 2014 2:26 PM
>>> 
>>> I got a bunch (3) installations of omnios on SuperMicro hardware and 
>>> all 3
>>> have issues rebooting. They simply hang and never ever reboot.
>> 
>> Disable fast reboot:
>> 
>> svccfg -s "system/boot-config:default" setprop
>> config/fastreboot_default=false
>> svccfg -s "system/boot-config:default" setprop
>> config/fastreboot_onpanic=false
>> svcadm refresh svc:/system/boot-config:default
>> 
>> Some systems (it seems particularly common on supermicro hardware) 
>> tend to
>> wedge up with fast reboot enabled. If you want to verify this is the 
>> issue
>> before changing the configuration, try 'reboot -p' which should do a 
>> one
>> time regular reboot without changing the default.
>> 
>> Dan, there was some talk of making fast reboot disabled the default, 
>> and
>> having people who wanted it need to enable it, rather than the other 
>> way
>> around? Any thoughts?
>> 
>> 
>> _______________________________________________
>> OmniOS-discuss mailing list
>> OmniOS-discuss at lists.omniti.com
>> http://lists.omniti.com/mailman/listinfo/omnios-discuss
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss

From matthew.lagoe at subrigo.net  Mon Dec 15 19:16:14 2014
From: matthew.lagoe at subrigo.net (Matthew Lagoe)
Date: Mon, 15 Dec 2014 11:16:14 -0800
Subject: [OmniOS-discuss] hangs on reboot
In-Reply-To: <ad985b1381f8ddcb8d6ead7fc548c40b@blackdot.be>
References: <1418336750819.83182@steait.net>	<893101d01592$59168140$0b4383c0$@acm.org>	<28B0569C-BB7A-47B2-9F73-90EB2C1D56F8@omniti.com>
	<ad985b1381f8ddcb8d6ead7fc548c40b@blackdot.be>
Message-ID: <005e01d0189b$99e2f5d0$cda8e170$@subrigo.net>

Ive had the same issue with dell servers as well as supermicro

Seems fast reboot just has issues, I always disable it as I have yet to see
it work

-----Original Message-----
From: OmniOS-discuss [mailto:omnios-discuss-bounces at lists.omniti.com] On
Behalf Of Jorge Schrauwen
Sent: Friday, December 12, 2014 12:11 AM
To: Dan McDonald
Cc: omnios-discuss at lists.omniti.com
Subject: Re: [OmniOS-discuss] hangs on reboot

SuperMicro boards seem quite popular too... anybody ever figure out why they
are hanging on fast reboot? I have the same issue but it only hangs about
1/3 of the time. (Fastreboot is now off though)

---
~ sjorge

On 2014-12-11 23:40, Dan McDonald wrote:
> Thanks for the fast reboot catch!
> 
> Yes, were contemplating it for '014 AND for upstreaming.  Not sure yet 
> if it'll happen, but this thread makes a compelling case.
> 
> Dan
> 
> Sent from my iPhone (typos, autocorrect, and all)
> 
> On Dec 11, 2014, at 5:32 PM, Paul B. Henson <henson at acm.org> wrote:
> 
>>> Rune Tipsmark
>>> Sent: Thursday, December 11, 2014 2:26 PM
>>> 
>>> I got a bunch (3) installations of omnios on SuperMicro hardware and 
>>> all 3 have issues rebooting. They simply hang and never ever reboot.
>> 
>> Disable fast reboot:
>> 
>> svccfg -s "system/boot-config:default" setprop 
>> config/fastreboot_default=false svccfg -s 
>> "system/boot-config:default" setprop config/fastreboot_onpanic=false 
>> svcadm refresh svc:/system/boot-config:default
>> 
>> Some systems (it seems particularly common on supermicro hardware) 
>> tend to wedge up with fast reboot enabled. If you want to verify this 
>> is the issue before changing the configuration, try 'reboot -p' which 
>> should do a one time regular reboot without changing the default.
>> 
>> Dan, there was some talk of making fast reboot disabled the default, 
>> and having people who wanted it need to enable it, rather than the 
>> other way around? Any thoughts?
>> 
>> 
>> _______________________________________________
>> OmniOS-discuss mailing list
>> OmniOS-discuss at lists.omniti.com
>> http://lists.omniti.com/mailman/listinfo/omnios-discuss
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
_______________________________________________
OmniOS-discuss mailing list
OmniOS-discuss at lists.omniti.com
http://lists.omniti.com/mailman/listinfo/omnios-discuss



From danmcd at omniti.com  Mon Dec 15 20:34:15 2014
From: danmcd at omniti.com (Dan McDonald)
Date: Mon, 15 Dec 2014 15:34:15 -0500
Subject: [OmniOS-discuss] [PATCH] isc-dhcp to work with VNIC's via DLPI
In-Reply-To: <BE930D3E-A6A5-4109-85BD-1D5C3D1A848E@mojovapes.net>
References: <14F85ED3-C0B7-42DE-BAD3-8C0931E8D429@mojovapes.net>
	<BD1E8409-5A12-41AC-97E8-78C240F7BF11@omniti.com>
	<BE930D3E-A6A5-4109-85BD-1D5C3D1A848E@mojovapes.net>
Message-ID: <5B181158-1EA2-441D-8FD9-9C8ADB8F85F4@omniti.com>

Sorry for not getting back to you on this sooner.

Have you tried putting your patches into the omnios-build tree?  What you'd have to do is:

	1.) mkdir $OMNIOS_BUILD/build/isc-dhcp/patches/

	2.) Put your patch in a file in the aforementioned directory:  say "use-dlpi.patch".

	3.) Edit .../patches/series so it has one line:  "use-dlpi.patch".

	4.) Run ./build.sh in "isc-dhcp".  It should generate the right binaries.

I might be able to do this later in the week, but since you have experience with it already, I thought maybe you could try it.

Thanks,
Dan


From danmcd at omniti.com  Mon Dec 15 20:35:15 2014
From: danmcd at omniti.com (Dan McDonald)
Date: Mon, 15 Dec 2014 15:35:15 -0500
Subject: [OmniOS-discuss] Change Zone Hostname
In-Reply-To: <1C434EF5-8635-4DBA-8317-45DEA492A920@vantagetitle.com>
References: <1C434EF5-8635-4DBA-8317-45DEA492A920@vantagetitle.com>
Message-ID: <99C856C5-2927-45DC-A5E4-C398A4A74BE0@omniti.com>


> On Dec 10, 2014, at 7:45 PM, Alex McWhirter <alexmcwhirter at vantagetitle.com> wrote:
> 
> I have a small cluster of servers hosting zones over multiple VLAN?s. I would like to name the zones so that they don?t run into naming collisions. For example, instead of ?web0? i would like to name them ?2049_web0? and ?2050_web0?. The problem is that this set?s the hostname to ?XXXX_web0?. Is there a way to change the hostname to be something different than the zone name?
> 
> I had a look at /etc/nodname and it is an empty file when used in a zone.

Did you ask this earlier and I'm seeing a duplicate?  Or did someone else ask this?

You can have /etc/nodename override the zone's name for a given zone.  E.g. my "webserver" zone isn't hostnamed "webserver", because I have a different name in /etc/nodename for that zone.  You will have to reboot the zone, however, for the change to stick.

Dan



From rt at steait.net  Mon Dec 15 20:43:58 2014
From: rt at steait.net (Rune Tipsmark)
Date: Mon, 15 Dec 2014 20:43:58 +0000
Subject: [OmniOS-discuss] dedup causes zfs/omnios to drop connections.
Message-ID: <1418676240949.15076@steait.net>

hi all,



got a new system I was intending on using as backup repository. Whenever dedup is enabled it dies after anywhere between 5 and 30 minutes. I need to reboot OmniOS to get it back online.

the files being copied onto the zfs vols are rather large, about ~2TB each... if I copy smaller files, say 400GB or so, it takes longer for it to crash.



what can be done to fix this? after Windows (initiator) looses the connection (both Fibre Channel and iSCSI) I still see a lot of disk activity using iostat - disks remain active for minutes after the copying has died... its like ZFS cannot handle dedup of large files..

br,

Rune
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141215/41c60603/attachment.html>

From danmcd at omniti.com  Mon Dec 15 20:53:47 2014
From: danmcd at omniti.com (Dan McDonald)
Date: Mon, 15 Dec 2014 15:53:47 -0500
Subject: [OmniOS-discuss] dedup causes zfs/omnios to drop connections.
In-Reply-To: <1418676240949.15076@steait.net>
References: <1418676240949.15076@steait.net>
Message-ID: <B41C6DFF-8557-4440-B9E1-E3130FDB0F1B@omniti.com>


> On Dec 15, 2014, at 3:43 PM, Rune Tipsmark <rt at steait.net> wrote:
> 
> hi all,
>  
> got a new system I was intending on using as backup repository. Whenever dedup is enabled it dies after anywhere between 5 and 30 minutes. I need to reboot OmniOS to get it back online.
> the files being copied onto the zfs vols are rather large, about ~2TB each... if I copy smaller files, say 400GB or so, it takes longer for it to crash.
>  
> what can be done to fix this? after Windows (initiator) looses the connection (both Fibre Channel and iSCSI) I still see a lot of disk activity using iostat - disks remain active for minutes after the copying has died... its like ZFS cannot handle dedup of large files.. 

Dedup is a memory pig and not very well implemented in ZFS.  I'd highly recommend against it in production.  Either that, or really increase your memory for your system in question.  There was some work going on at Nexenta to perhaps put the dedup tables (DDTs) onto a dedicated slog-like device, but I believe that work stalled.

Sorry,
Dan



From henson at acm.org  Mon Dec 15 20:55:13 2014
From: henson at acm.org (Paul B. Henson)
Date: Mon, 15 Dec 2014 12:55:13 -0800
Subject: [OmniOS-discuss] Change Zone Hostname
In-Reply-To: <99C856C5-2927-45DC-A5E4-C398A4A74BE0@omniti.com>
References: <1C434EF5-8635-4DBA-8317-45DEA492A920@vantagetitle.com>
	<99C856C5-2927-45DC-A5E4-C398A4A74BE0@omniti.com>
Message-ID: <a23f01d018a9$6e4bdeb0$4ae39c10$@acm.org>

> From: Dan McDonald
> Sent: Monday, December 15, 2014 12:35 PM
>
> Did you ask this earlier and I'm seeing a duplicate?  Or did someone else ask
> this?

It does seem familiar, plus it has an old date? I believe earlier this month I saw what appeared to be some list reposts too.

> You can have /etc/nodename override the zone's name for a given zone.
> E.g. my "webserver" zone isn't hostnamed "webserver", because I have a
> different name in /etc/nodename for that zone.  You will have to reboot the
> zone, however, for the change to stick.

Just as a data point, I set up a couple of zones over the weekend, and after updating nodename and rebooting the zone, the replacement hostname worked fine.


From jtyocum at uw.edu  Mon Dec 15 20:51:36 2014
From: jtyocum at uw.edu (John Yocum)
Date: Mon, 15 Dec 2014 12:51:36 -0800
Subject: [OmniOS-discuss] dedup causes zfs/omnios to drop connections.
In-Reply-To: <1418676240949.15076@steait.net>
References: <1418676240949.15076@steait.net>
Message-ID: <548F49D8.9060601@uw.edu>

On 12/15/2014 12:43 PM, Rune Tipsmark wrote:
> hi all,
> 
>  
> 
> got a new system I was intending on using as backup repository. Whenever
> dedup is enabled it dies after anywhere between 5 and 30 minutes. I need
> to reboot OmniOS to get it back online.
> 
> the files being copied onto the zfs vols are rather large, about ~2TB
> each... if I copy smaller files, say 400GB or so, it takes longer for it
> to crash.
> 
>  
> 
> what can be done to fix this? after Windows (initiator) looses the
> connection (both Fibre Channel and iSCSI) I still see a lot of disk
> activity using iostat - disks remain active for minutes after the
> copying has died... its like ZFS cannot handle dedup of large files..
> 
> br,
> 
> Rune
> 

How much RAM do you have? And, how big is your ZFS pool? I ask, as dedup
requires a ton of RAM. Helpful reference on the issue
http://utcc.utoronto.ca/~cks/space/blog/solaris/ZFSDedupMemoryProblem

-- 
John Yocum, Systems Administrator, DEOHS

From henson at acm.org  Mon Dec 15 21:00:08 2014
From: henson at acm.org (Paul B. Henson)
Date: Mon, 15 Dec 2014 13:00:08 -0800
Subject: [OmniOS-discuss] state of building illumos-gate on omnios-stable
In-Reply-To: <44531b95aa7d9a09499e71edd48d2bd9@blackdot.be>
References: <20141215043037.GA29549@bender.unx.csupomona.edu>
	<44531b95aa7d9a09499e71edd48d2bd9@blackdot.be>
Message-ID: <a39d01d018aa$1e5f3f90$5b1dbeb0$@acm.org>

> From: Jorge Schrauwen
> Sent: Monday, December 15, 2014 4:26 AM
> 
> If while setting up your env you found obvious things lacking on
> https://docu.blackdot.be/snipets/solaris/build-zone, feel free to drop
> me a line on #omnios (sjorge) or here on the mailing list and I will
> update the page.

Cool, thanks. Just to clarify, did you apply the pending illumos-gate
patches Dan pointed out to get things to compile? As it sounded like without
them it wouldn't compile.



From richard at netbsd.org  Mon Dec 15 21:00:48 2014
From: richard at netbsd.org (Richard PALO)
Date: Mon, 15 Dec 2014 22:00:48 +0100
Subject: [OmniOS-discuss] pull request: revert  a6f10d64
Message-ID: <m6ni61$inq$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Dan suggested I post this here to get a go-ahead.
This gist effectively reverts the following which seems unnecessary, at least nowadays:
> commit a6f10d643d66d62130a867a42efa06ce8cb93f70
> Author: Theo Schlossnagle <jesus at omniti.com>
> Date:   Fri Mar 30 03:42:02 2012 +0000
> 
>     two .gitignore files that cover a completed build
> 
> diff --git a/.gitignore b/.gitignore

https://gist.github.com/risto3/adf3f9068eff759f06a7

cheers
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEcBAEBAgAGBQJUj0wAAAoJECAB22fHtp27ZtAH+wZk2tjcGXnlzKD4DNf5r9s/
4fBJ++ojxTPbO3jQuE5e0XpMAbADPQLgR+A6vT07AYoruRIx81uUHg3nihJNlrXx
Um8DRTKViQcyUHWsiRXYUI8NgF4oeGryKohyfphZGbZYLPlHqmjxB4S3WpFKNkPx
TgKOpHzInc9uMqQbNTQNM/SA9h8ydo3pWhl1Y/SYQl2TyxvYmHCOpcImsqaLmiUT
wF/OhoQxVYHy1LHZwDOSR4rHAvqYhaHwhWCtnVqVZbkbGpjhHBHBSLN8G+/UDKd3
GM9PxZGJyB1axsP2I5YwuQFFQXDnjP5veZONaMsc9y5YbLAseOU45uXI9QbCtGM=
=6G0k
-----END PGP SIGNATURE-----


From hasslerd at gmx.li  Mon Dec 15 21:11:55 2014
From: hasslerd at gmx.li (Dominik Hassler)
Date: Mon, 15 Dec 2014 22:11:55 +0100
Subject: [OmniOS-discuss] dedup causes zfs/omnios to drop connections.
In-Reply-To: <B41C6DFF-8557-4440-B9E1-E3130FDB0F1B@omniti.com>
References: <1418676240949.15076@steait.net>
	<B41C6DFF-8557-4440-B9E1-E3130FDB0F1B@omniti.com>
Message-ID: <548F4E9B.6060904@gmx.li>

Hi,

we used dedup on a production machine w/ 256 GB RAM, but disabled it
after a couple of days due to huge performance impact.

I would not recommend to use dedup even when having "enough" RAM.

On 12/15/2014 09:53 PM, Dan McDonald wrote:
> 
>> On Dec 15, 2014, at 3:43 PM, Rune Tipsmark <rt at steait.net> wrote:
>>
>> hi all,
>>  
>> got a new system I was intending on using as backup repository. Whenever dedup is enabled it dies after anywhere between 5 and 30 minutes. I need to reboot OmniOS to get it back online.
>> the files being copied onto the zfs vols are rather large, about ~2TB each... if I copy smaller files, say 400GB or so, it takes longer for it to crash.
>>  
>> what can be done to fix this? after Windows (initiator) looses the connection (both Fibre Channel and iSCSI) I still see a lot of disk activity using iostat - disks remain active for minutes after the copying has died... its like ZFS cannot handle dedup of large files.. 
> 
> Dedup is a memory pig and not very well implemented in ZFS.  I'd highly recommend against it in production.  Either that, or really increase your memory for your system in question.  There was some work going on at Nexenta to perhaps put the dedup tables (DDTs) onto a dedicated slog-like device, but I believe that work stalled.
> 
> Sorry,
> Dan
> 
> 
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
> 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0xF9ECC6A5.asc
Type: application/pgp-keys
Size: 2686 bytes
Desc: not available
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141215/9a2b3d99/attachment.bin>

From eric.sproul at circonus.com  Mon Dec 15 21:15:51 2014
From: eric.sproul at circonus.com (Eric Sproul)
Date: Mon, 15 Dec 2014 16:15:51 -0500
Subject: [OmniOS-discuss] Change Zone Hostname
In-Reply-To: <a23f01d018a9$6e4bdeb0$4ae39c10$@acm.org>
References: <1C434EF5-8635-4DBA-8317-45DEA492A920@vantagetitle.com>
	<99C856C5-2927-45DC-A5E4-C398A4A74BE0@omniti.com>
	<a23f01d018a9$6e4bdeb0$4ae39c10$@acm.org>
Message-ID: <CAO8hXRD9HQmkmRXeByMC9jGomQQiCBwJ-C42T6+Xn5S09d8d3w@mail.gmail.com>

FYI, the mail that appeared today was held in the moderator queue,
having been sent from a non-subscribed address.  I just went through
the queue earlier today.

From danmcd at omniti.com  Mon Dec 15 21:22:14 2014
From: danmcd at omniti.com (Dan McDonald)
Date: Mon, 15 Dec 2014 16:22:14 -0500
Subject: [OmniOS-discuss] pull request: revert  a6f10d64
In-Reply-To: <m6ni61$inq$1@ger.gmane.org>
References: <m6ni61$inq$1@ger.gmane.org>
Message-ID: <04937852-0D6E-4375-B0AC-064C489089FB@omniti.com>


> On Dec 15, 2014, at 4:00 PM, Richard PALO <richard at netbsd.org> wrote:
> 
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Dan suggested I post this here to get a go-ahead.
> This gist effectively reverts the following which seems unnecessary, at least nowadays:
>> commit a6f10d643d66d62130a867a42efa06ce8cb93f70
>> Author: Theo Schlossnagle <jesus at omniti.com>
>> Date:   Fri Mar 30 03:42:02 2012 +0000
>> 
>>    two .gitignore files that cover a completed build
>> 
>> diff --git a/.gitignore b/.gitignore
> 
> https://gist.github.com/risto3/adf3f9068eff759f06a7

I'm building nightly on the bloody build box, with the master branch of illumos-omnios with this fix.  If it's silent, then I'm okay taking this in.

If there's historical reasons this happened originally, I'd love to hear them on this thread.

Dan


From rt at steait.net  Mon Dec 15 22:17:09 2014
From: rt at steait.net (Rune Tipsmark)
Date: Mon, 15 Dec 2014 22:17:09 +0000
Subject: [OmniOS-discuss] dedup causes zfs/omnios to drop connections.
In-Reply-To: <548F4E9B.6060904@gmx.li>
References: <1418676240949.15076@steait.net>
	<B41C6DFF-8557-4440-B9E1-E3130FDB0F1B@omniti.com>,
	<548F4E9B.6060904@gmx.li>
Message-ID: <1418681831295.66175@steait.net>

we have only 24GB of ram on this system...
I was under the impression it would not require much when the block size was larger, we are on 64kb, so would expect around 2GB per 1TB.
yet we cannot even get more than a few TB on the system before it dies. 

the main purpose with this system was dedup, no problem with slow speed... its for backup only, so could not care less if its rather slow - but not responding is not acceptable.
br,
Rune
________________________________________
From: OmniOS-discuss <omnios-discuss-bounces at lists.omniti.com> on behalf of Dominik Hassler <hasslerd at gmx.li>
Sent: Monday, December 15, 2014 10:11 PM
To: omnios-discuss at lists.omniti.com
Subject: Re: [OmniOS-discuss] dedup causes zfs/omnios to drop connections.

Hi,

we used dedup on a production machine w/ 256 GB RAM, but disabled it
after a couple of days due to huge performance impact.

I would not recommend to use dedup even when having "enough" RAM.

On 12/15/2014 09:53 PM, Dan McDonald wrote:
>
>> On Dec 15, 2014, at 3:43 PM, Rune Tipsmark <rt at steait.net> wrote:
>>
>> hi all,
>>
>> got a new system I was intending on using as backup repository. Whenever dedup is enabled it dies after anywhere between 5 and 30 minutes. I need to reboot OmniOS to get it back online.
>> the files being copied onto the zfs vols are rather large, about ~2TB each... if I copy smaller files, say 400GB or so, it takes longer for it to crash.
>>
>> what can be done to fix this? after Windows (initiator) looses the connection (both Fibre Channel and iSCSI) I still see a lot of disk activity using iostat - disks remain active for minutes after the copying has died... its like ZFS cannot handle dedup of large files..
>
> Dedup is a memory pig and not very well implemented in ZFS.  I'd highly recommend against it in production.  Either that, or really increase your memory for your system in question.  There was some work going on at Nexenta to perhaps put the dedup tables (DDTs) onto a dedicated slog-like device, but I believe that work stalled.
>
> Sorry,
> Dan
>
>
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>

From sjorge+ml at blackdot.be  Tue Dec 16 00:15:58 2014
From: sjorge+ml at blackdot.be (Jorge Schrauwen)
Date: Tue, 16 Dec 2014 01:15:58 +0100
Subject: [OmniOS-discuss] state of building illumos-gate on omnios-stable
In-Reply-To: <a39d01d018aa$1e5f3f90$5b1dbeb0$@acm.org>
References: <20141215043037.GA29549@bender.unx.csupomona.edu>
	<44531b95aa7d9a09499e71edd48d2bd9@blackdot.be>
	<a39d01d018aa$1e5f3f90$5b1dbeb0$@acm.org>
Message-ID: <b3191e564438bd020eca36cb9cecae04@blackdot.be>

illumos-omnios works fine without.
illumos-gate needs dan's patches to compile (but it still gives lint 
message IIRC)

On 2014-12-15 22:00, Paul B. Henson wrote:
>> From: Jorge Schrauwen
>> Sent: Monday, December 15, 2014 4:26 AM
>> 
>> If while setting up your env you found obvious things lacking on
>> https://docu.blackdot.be/snipets/solaris/build-zone, feel free to drop
>> me a line on #omnios (sjorge) or here on the mailing list and I will
>> update the page.
> 
> Cool, thanks. Just to clarify, did you apply the pending illumos-gate
> patches Dan pointed out to get things to compile? As it sounded like 
> without
> them it wouldn't compile.

From danmcd at omniti.com  Tue Dec 16 02:10:32 2014
From: danmcd at omniti.com (Dan McDonald)
Date: Mon, 15 Dec 2014 21:10:32 -0500
Subject: [OmniOS-discuss] pull request: revert  a6f10d64
In-Reply-To: <04937852-0D6E-4375-B0AC-064C489089FB@omniti.com>
References: <m6ni61$inq$1@ger.gmane.org>
	<04937852-0D6E-4375-B0AC-064C489089FB@omniti.com>
Message-ID: <5F94DC7D-8858-4876-AAAE-90808CE965CF@omniti.com>


> On Dec 15, 2014, at 4:22 PM, Dan McDonald <danmcd at omniti.com> wrote:
>> 
>> https://gist.github.com/risto3/adf3f9068eff759f06a7
> 
> I'm building nightly on the bloody build box, with the master branch of illumos-omnios with this fix.  If it's silent, then I'm okay taking this in.
> 
> If there's historical reasons this happened originally, I'd love to hear them on this thread.

illumos-omnios build went through w/o a hitch.

Unless I hear strong objections, I'm taking this one in tomorrow (Tuesday).

Dan


From henson at acm.org  Tue Dec 16 02:20:10 2014
From: henson at acm.org (Paul B. Henson)
Date: Mon, 15 Dec 2014 18:20:10 -0800
Subject: [OmniOS-discuss] state of building illumos-gate on omnios-stable
In-Reply-To: <m6mecf$met$1@ger.gmane.org>
References: <20141215043037.GA29549@bender.unx.csupomona.edu>
	<m6mecf$met$1@ger.gmane.org>
Message-ID: <ac1301d018d6$d2e75bb0$78b61310$@acm.org>

> From: Richard PALO
> Sent: Monday, December 15, 2014 2:50 AM
> 
> One possibility, that is if you use what you build, is to start from
> omnios bloody.

Hmm, do you mean actually develop against and test with illumos-omnios
bloody, and then apply the resultant patch to illumos-gate, and RTI using
the build log from omnios bloody? 

> For myself, my diffs for illumos.sh and nightly.sh:

Thanks, hopefully I can take this and some of the other information I've
gotten and get something going.



From danmcd at omniti.com  Tue Dec 16 02:24:07 2014
From: danmcd at omniti.com (Dan McDonald)
Date: Mon, 15 Dec 2014 21:24:07 -0500
Subject: [OmniOS-discuss] state of building illumos-gate on omnios-stable
In-Reply-To: <ac1301d018d6$d2e75bb0$78b61310$@acm.org>
References: <20141215043037.GA29549@bender.unx.csupomona.edu>
	<m6mecf$met$1@ger.gmane.org>
	<ac1301d018d6$d2e75bb0$78b61310$@acm.org>
Message-ID: <80F9B9EA-5308-4CAA-90B8-1206B03033B9@omniti.com>


> On Dec 15, 2014, at 9:20 PM, Paul B. Henson <henson at acm.org> wrote:
> 
>> From: Richard PALO
>> Sent: Monday, December 15, 2014 2:50 AM
>> 
>> One possibility, that is if you use what you build, is to start from
>> omnios bloody.
> 
> Hmm, do you mean actually develop against and test with illumos-omnios
> bloody, and then apply the resultant patch to illumos-gate, and RTI using
> the build log from omnios bloody?

Depending on what you do, that MAY be sufficient.  I might do the dead-chicken dance and build your submitted diff against stock illumos-gate to be sure, but that's my own RTI advocate opinion.

>> For myself, my diffs for illumos.sh and nightly.sh:
> 
> Thanks, hopefully I can take this and some of the other information I've
> gotten and get something going.

Go for it!

Dan


From danmcd at omniti.com  Tue Dec 16 02:25:43 2014
From: danmcd at omniti.com (Dan McDonald)
Date: Mon, 15 Dec 2014 21:25:43 -0500
Subject: [OmniOS-discuss] state of building illumos-gate on omnios-stable
In-Reply-To: <80F9B9EA-5308-4CAA-90B8-1206B03033B9@omniti.com>
References: <20141215043037.GA29549@bender.unx.csupomona.edu>
	<m6mecf$met$1@ger.gmane.org>
	<ac1301d018d6$d2e75bb0$78b61310$@acm.org>
	<80F9B9EA-5308-4CAA-90B8-1206B03033B9@omniti.com>
Message-ID: <B2D8B406-D3F1-427E-889C-15B99EE2812F@omniti.com>


> On Dec 15, 2014, at 9:24 PM, Dan McDonald <danmcd at omniti.com> wrote:
>> 
>> Thanks, hopefully I can take this and some of the other information I've
>> gotten and get something going.
> 
> Go for it!

One important safety tip:

** If you're doing an illumos-omnios nightly you want to actually ONU over a given build of OmniOS, make sure ONNV_BUILDNUM is set to the release number of your target OmniOS.  E.g. If I have a bloody machine I wish to ONU with new bits, I better make sure ONNV_BUILDNUM=151013, or if I have a current-stable machine I wish to ONU with new bits, I better make sure ONNV_BUILDNUM=151012.

FYI,
Dan


From chip at innovates.com  Tue Dec 16 14:02:12 2014
From: chip at innovates.com (Schweiss, Chip)
Date: Tue, 16 Dec 2014 08:02:12 -0600
Subject: [OmniOS-discuss] NFSv4 id mapping only working on client but
	not server?
In-Reply-To: <6e8301d0132f$e1702c40$a45084c0$@acm.org>
References: <CAOEv1kYdT=Cj-K7KarcEFizXKe8g8aehKm5uaHORqfqzAxUFoA@mail.gmail.com>
	<20141204193711.03524183@sleipner.datanom.net>
	<CAOEv1kaoxj=AREpfuRo8saHFhW18i0CmgSd9dfwLZEtCUv8pmw@mail.gmail.com>
	<20141207224353.GA13916@warlock.deepthought.com>
	<CAOEv1kYze9=As8jtTYAG88O+0hn0ECvLGrY9_HiLZdGu1-23uA@mail.gmail.com>
	<CAPJtH1gTA9z4got9Ou=Zc9BxjmULGq3-NhF=50YgMT6Vqx3n8w@mail.gmail.com>
	<674b01d01324$d0e28180$72a78480$@acm.org>
	<CAPJtH1h_Y_3rQm5-SNaUAPcnQxvJ4CKEUdni4kzmXQhuVhH8ZA@mail.gmail.com>
	<6e8301d0132f$e1702c40$a45084c0$@acm.org>
Message-ID: <CALeZrrTufELRVgC0=PEG=X-7x4i5EAOMiGgRmsSiyZ3RjJWQBg@mail.gmail.com>

It seems there a many ways to map ID in NFSv4, is there a way to not map
them at all?

I'm working to configure an OmniOS NFS server that will serve several
domains simultaneously.  Each which has their own ID map with many
conflicting uid/gid across the domains.

All the current file systems being migrated are NFSv3 with AUTH_SYS.   I'd
consider moving them all to kerberos authentication, but something tells me
that may be impossible with the multiple domains.

-Chip



On Mon, Dec 8, 2014 at 3:42 PM, Paul B. Henson <henson at acm.org> wrote:
>
> > From: Ian Kaufman
> > Sent: Monday, December 08, 2014 12:37 PM
> >
> > Yes, I oversimplified things. The issue is that AUTH_SYS/AUTH_UNIX
> > does not support NFSv4. AUTH_SYS uses the UID, not the name, so the
> > mapping fails. So, when RPC uses AUTH_SYS, NFSv4 is SOL.
>
> I apologize for being a pedant; it's a character flaw :). Your wording
> implies that you cannot use AUTH_SYS with NFSv4, which is not true. NFSv4
> works perfectly fine with AUTH_SYS as long as you maintain synchronization
> of uid/gid's on both sides. I would pick NFSv4 with AUTH_SYS over NFSv3
> with AUTH_SYS. Both require that you maintain uid synchronization, so it's
> not like you're gaining something by falling back, and why miss out on the
> other features of NFSv4?
>
> > Supposedly. at least on the client side, this has been fixed somewhere
> > upstream. However, the server side is not.
> >
> > https://bugzilla.linux-nfs.org/show_bug.cgi?id=226
>
> I don't know if I would call this "fixed" 8-/. They are basically just
> disabling the idmapper and passing raw uid/gid info at the NFS level to
> match the raw info at the RPC level. I guess it makes it less confusing in
> such a scenario, because you're always broken if they don't match on each
> side rather than only broken in some cases. An actual fix would be
> introducing an RPC mechanism using names such as AUTH_SYS_NAME, one of
> these days maybe I'll find the time to go hassle some NFS developers and
> see why they don't just do that and make everything simpler.
>
> > Regardless, my point is that this is not a Solaris/Linux issue, as a
> > Linux server and Linux clients would be in the same boat.
>
> Agreed. It is a deficiency in the NFSv4 protocol :(...
>
>
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141216/721ed0ee/attachment.html>

From cscoman at gmail.com  Tue Dec 16 15:12:43 2014
From: cscoman at gmail.com (Jason Cox)
Date: Tue, 16 Dec 2014 07:12:43 -0800
Subject: [OmniOS-discuss] dedup causes zfs/omnios to drop connections.
In-Reply-To: <1418681831295.66175@steait.net>
References: <1418676240949.15076@steait.net>
	<B41C6DFF-8557-4440-B9E1-E3130FDB0F1B@omniti.com>
	<548F4E9B.6060904@gmx.li> <1418681831295.66175@steait.net>
Message-ID: <CAC4WUHp+tdBcpPbs+GGh60e8OjFwphEwbip-1bQooX4Uk02qTA@mail.gmail.com>

I have found that turning on compression gets more savings when storing my
backups then I got with dedup on. I run gzip-9 for my backup dataset. I
think dedup is good if you are storing a lot of files that really are the
same. Your mileage may vary compared to mine. I do a full backup once a
week and incrementals daily.

On Mon, Dec 15, 2014 at 2:17 PM, Rune Tipsmark <rt at steait.net> wrote:
>
> we have only 24GB of ram on this system...
> I was under the impression it would not require much when the block size
> was larger, we are on 64kb, so would expect around 2GB per 1TB.
> yet we cannot even get more than a few TB on the system before it dies.
>
> the main purpose with this system was dedup, no problem with slow speed...
> its for backup only, so could not care less if its rather slow - but not
> responding is not acceptable.
> br,
> Rune
> ________________________________________
> From: OmniOS-discuss <omnios-discuss-bounces at lists.omniti.com> on behalf
> of Dominik Hassler <hasslerd at gmx.li>
> Sent: Monday, December 15, 2014 10:11 PM
> To: omnios-discuss at lists.omniti.com
> Subject: Re: [OmniOS-discuss] dedup causes zfs/omnios to drop connections.
>
> Hi,
>
> we used dedup on a production machine w/ 256 GB RAM, but disabled it
> after a couple of days due to huge performance impact.
>
> I would not recommend to use dedup even when having "enough" RAM.
>
> On 12/15/2014 09:53 PM, Dan McDonald wrote:
> >
> >> On Dec 15, 2014, at 3:43 PM, Rune Tipsmark <rt at steait.net> wrote:
> >>
> >> hi all,
> >>
> >> got a new system I was intending on using as backup repository.
> Whenever dedup is enabled it dies after anywhere between 5 and 30 minutes.
> I need to reboot OmniOS to get it back online.
> >> the files being copied onto the zfs vols are rather large, about ~2TB
> each... if I copy smaller files, say 400GB or so, it takes longer for it to
> crash.
> >>
> >> what can be done to fix this? after Windows (initiator) looses the
> connection (both Fibre Channel and iSCSI) I still see a lot of disk
> activity using iostat - disks remain active for minutes after the copying
> has died... its like ZFS cannot handle dedup of large files..
> >
> > Dedup is a memory pig and not very well implemented in ZFS.  I'd highly
> recommend against it in production.  Either that, or really increase your
> memory for your system in question.  There was some work going on at
> Nexenta to perhaps put the dedup tables (DDTs) onto a dedicated slog-like
> device, but I believe that work stalled.
> >
> > Sorry,
> > Dan
> >
> >
> > _______________________________________________
> > OmniOS-discuss mailing list
> > OmniOS-discuss at lists.omniti.com
> > http://lists.omniti.com/mailman/listinfo/omnios-discuss
> >
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>


-- 
Jason Cox
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141216/7beae212/attachment-0001.html>

From danmcd at omniti.com  Tue Dec 16 19:33:23 2014
From: danmcd at omniti.com (Dan McDonald)
Date: Tue, 16 Dec 2014 14:33:23 -0500
Subject: [OmniOS-discuss] pull request: revert  a6f10d64
In-Reply-To: <5F94DC7D-8858-4876-AAAE-90808CE965CF@omniti.com>
References: <m6ni61$inq$1@ger.gmane.org>
	<04937852-0D6E-4375-B0AC-064C489089FB@omniti.com>
	<5F94DC7D-8858-4876-AAAE-90808CE965CF@omniti.com>
Message-ID: <ADEED67C-6A29-44A5-9F34-DF51F5F8A5A9@omniti.com>


> On Dec 15, 2014, at 9:10 PM, Dan McDonald <danmcd at omniti.com> wrote:
> 
> 
>> On Dec 15, 2014, at 4:22 PM, Dan McDonald <danmcd at omniti.com> wrote:
>>> 
>>> https://gist.github.com/risto3/adf3f9068eff759f06a7
>> 
>> I'm building nightly on the bloody build box, with the master branch of illumos-omnios with this fix.  If it's silent, then I'm okay taking this in.
>> 
>> If there's historical reasons this happened originally, I'd love to hear them on this thread.
> 
> illumos-omnios build went through w/o a hitch.
> 
> Unless I hear strong objections, I'm taking this one in tomorrow (Tuesday).

It's in now:

	https://github.com/omniti-labs/illumos-omnios/commit/0c8258bb76d866ec46d30d1b3429fbd766237ef6

Dan


From henson at acm.org  Wed Dec 17 00:27:35 2014
From: henson at acm.org (Paul B. Henson)
Date: Tue, 16 Dec 2014 16:27:35 -0800
Subject: [OmniOS-discuss] NFSv4 id mapping only working on client but
	not server?
In-Reply-To: <CALeZrrTufELRVgC0=PEG=X-7x4i5EAOMiGgRmsSiyZ3RjJWQBg@mail.gmail.com>
References: <CAOEv1kYdT=Cj-K7KarcEFizXKe8g8aehKm5uaHORqfqzAxUFoA@mail.gmail.com>
	<20141204193711.03524183@sleipner.datanom.net>
	<CAOEv1kaoxj=AREpfuRo8saHFhW18i0CmgSd9dfwLZEtCUv8pmw@mail.gmail.com>
	<20141207224353.GA13916@warlock.deepthought.com>
	<CAOEv1kYze9=As8jtTYAG88O+0hn0ECvLGrY9_HiLZdGu1-23uA@mail.gmail.com>
	<CAPJtH1gTA9z4got9Ou=Zc9BxjmULGq3-NhF=50YgMT6Vqx3n8w@mail.gmail.com>
	<674b01d01324$d0e28180$72a78480$@acm.org>
	<CAPJtH1h_Y_3rQm5-SNaUAPcnQxvJ4CKEUdni4kzmXQhuVhH8ZA@mail.gmail.com>
	<6e8301d0132f$e1702c40$a45084c0$@acm.org>
	<CALeZrrTufELRVgC0=PEG=X-7x4i5EAOMiGgRmsSiyZ3RjJWQBg@mail.gmail.com>
Message-ID: <b8b501d01990$43e301c0$cba90540$@acm.org>

> From: Schweiss, Chip
> Sent: Tuesday, December 16, 2014 6:02 AM
> 
> It seems there a many ways to map ID in NFSv4, is there a way to not map
> them at all?

I believe linux supports disabling ID mapping and using raw uid/gids on the wire instead of strings, but I don't think illumos does?

> All the current file systems being migrated are NFSv3 with AUTH_SYS.   I'd
> consider moving them all to kerberos authentication, but something tells me
> that may be impossible with the multiple domains.

Multiple Kerberos realms too? I don't think illumos can have more than one kerberos realm defined for NFS...



From ikaufman at eng.ucsd.edu  Wed Dec 17 00:29:39 2014
From: ikaufman at eng.ucsd.edu (Ian Kaufman)
Date: Tue, 16 Dec 2014 16:29:39 -0800
Subject: [OmniOS-discuss] NFSv4 id mapping only working on client but
	not server?
In-Reply-To: <b8b501d01990$43e301c0$cba90540$@acm.org>
References: <CAOEv1kYdT=Cj-K7KarcEFizXKe8g8aehKm5uaHORqfqzAxUFoA@mail.gmail.com>
	<20141204193711.03524183@sleipner.datanom.net>
	<CAOEv1kaoxj=AREpfuRo8saHFhW18i0CmgSd9dfwLZEtCUv8pmw@mail.gmail.com>
	<20141207224353.GA13916@warlock.deepthought.com>
	<CAOEv1kYze9=As8jtTYAG88O+0hn0ECvLGrY9_HiLZdGu1-23uA@mail.gmail.com>
	<CAPJtH1gTA9z4got9Ou=Zc9BxjmULGq3-NhF=50YgMT6Vqx3n8w@mail.gmail.com>
	<674b01d01324$d0e28180$72a78480$@acm.org>
	<CAPJtH1h_Y_3rQm5-SNaUAPcnQxvJ4CKEUdni4kzmXQhuVhH8ZA@mail.gmail.com>
	<6e8301d0132f$e1702c40$a45084c0$@acm.org>
	<CALeZrrTufELRVgC0=PEG=X-7x4i5EAOMiGgRmsSiyZ3RjJWQBg@mail.gmail.com>
	<b8b501d01990$43e301c0$cba90540$@acm.org>
Message-ID: <CAPJtH1i1QhsSaLymo=yB75TvPVp0amqNfwxe=yZ_5zuVv+K3Tw@mail.gmail.com>

There has been some work on multi-domain Kerberos in NFSv4, going back
to 2010. Not sure where things stand though.

https://www.ietf.org/proceedings/10mar/slides/nfsv4-5.pdf

Ian

On Tue, Dec 16, 2014 at 4:27 PM, Paul B. Henson <henson at acm.org> wrote:
>> From: Schweiss, Chip
>> Sent: Tuesday, December 16, 2014 6:02 AM
>>
>> It seems there a many ways to map ID in NFSv4, is there a way to not map
>> them at all?
>
> I believe linux supports disabling ID mapping and using raw uid/gids on the wire instead of strings, but I don't think illumos does?
>
>> All the current file systems being migrated are NFSv3 with AUTH_SYS.   I'd
>> consider moving them all to kerberos authentication, but something tells me
>> that may be impossible with the multiple domains.
>
> Multiple Kerberos realms too? I don't think illumos can have more than one kerberos realm defined for NFS...
>
>



-- 
Ian Kaufman
Research Systems Administrator
UC San Diego, Jacobs School of Engineering ikaufman AT ucsd DOT edu

From tim at multitalents.net  Wed Dec 17 06:31:10 2014
From: tim at multitalents.net (Tim Rice)
Date: Tue, 16 Dec 2014 22:31:10 -0800 (PST)
Subject: [OmniOS-discuss] anyone doing DirectPath I/O?
In-Reply-To: <54807EC4.6020201@kateley.com>
References: <CAGoaUhO9kRCopXH5+wojk4U-ict8=JpbwS9U9RtBxcO86F6_zQ@mail.gmail.com>
	<CALeZrrTCE=N5arT7_LFCLex5NJXyxMugNwWrgO_mbmOaHXeEcQ@mail.gmail.com>
	<alpine.UW2.2.11.1412032251490.27489@server01.int.multitalents.net>
	<54807EC4.6020201@kateley.com>
Message-ID: <alpine.UW2.2.11.1412162134480.27899@server01.int.multitalents.net>

On Thu, 4 Dec 2014, Linda Kateley wrote:

| Just curious, how fast is the nfs in this config?

Is fast enough a good enough answer? :-)

The only actual hard data I have (over any length of time) is
what my backup program reports. So picture 3 evolving configurations.
First the system common to all 3, a Solaris Express Community Edition
box with 4 SATA drives in a raidz configuration. That's where
the backups get written via ftp. (Yes I know you asked about NFS,
we'll get to that)

Now to establish a baseline, when the machine being backed up,
a UnixWare 7.1.4 box, was running native backups would average
29.85G/hr and verify would average 49.10G/hr.

Now we introduce the all in one box mentioned below. The OmniOS
r151006 VM is providing NFS storage to it's ESXi host for the
UnixWare 7.1.4 VM. Both VMs using E1000 nics. Now backups
are averaging 31.19G/hr and verify 50.44G/hr. This is with no
compression (on the UnixWare backup program) so the increase in
CPU power would not be helping there. So obviously NFS is not
getting in the way here. Faster then the native 6 drive raid
in the HP DL385.

Now let's say your ESXi was a little behind in it's patches and
you load all the current patches. Oops. Your entire system
falls on it's face. Network throughput is at least an order of
magnitude slower.

More info here

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2078636

http://kb.vmware.com/selfservice/microsites/search.do?cmd=displayKC&docType=kc&externalId=2092809

So after switching to vmxnet3 nics on the OmniOS VM and reconfiguring,
things were back to normal. Actually it seemed better than before.
The current numbers seem to confirm this. Backups are averaging
33.32G/hr and verify 56.42G/hr.

Now what I have noticed over the life of this system (installed July 2013)
although not recently, is what another poster described where from
time to time everything seems to lock up for about 5 to 15 seconds.

I hope this helps answer your question.

| 
| linda
| 
| On 12/4/14, 1:11 AM, Tim Rice wrote:
| > On Wed, 3 Dec 2014, Schweiss, Chip wrote:
| > 
| > | I've been using DirectPath with ESXi 5.0 with an LSI HBA for almost 2
| > years
| > | to an OpenIndiana server.  It's been very stable:
| > 
| > My all in one box (Supermicro MBD-X9SCM-F-0) was on ESXi 5.1 for
| > about a year with a LSI SAS3801E and a LSI SAS9211-4i. I've since
| > updated to ESXi 5.5 update2. The OmniOS r151006 VM provides NFS
| > storage to the ESXi host for (currently) 15 running VMS as well
| > as well as NFS home directories for various other computers around
| > the office. Other than drives failing, it's been working reasonably
| > well for what it is.
| > 
| > |
| > | root at zfs01:~# uptime
| > |  21:01pm  up 649 days  6:58,  2 users,  load average: 0.46, 0.30, 0.25
| > |
| > |
| > | On Wed, Dec 3, 2014 at 6:10 PM, Joseph Boren <jboren at drakecooper.com>
| > wrote:
| > |
| > | > Greetings,
| > | >
| > | > I was wondering if anyone was using DirectPath, specifically for
| > exclusive
| > | > use of a drive controller and attached drives to a specific VM.    I
| > have a
| > | > use case that would seem to be a good fit for this, so I played around
| > with
| > | > a couple of RAID controllers I had, and was able to get one (3ware
| > 9650se)
| > | > configured for directpath, but none of the drives attached would show to
| > | > OmniOS, regardless of how I configured them in the raid bois (JBOD,
| > | > individual disks, etc).  I know that controller is poorly supported and
| > I
| > | > was curious if anyone was using DirectPath  this way in production and
| > what
| > | > kind of drive controller/HBA/whatever was working.  Also any "for the
| > love
| > | > of god don't do it this way" scenarios?  I seem to be really adept at
| > | > finding and trying those out first....
| > | >
| > | > Thanks a ton and best regards,
| > | >
| > | > -jb-
| > | > *Joseph Boren*
| > | >
| > | > IT Specialist
| > | > *DRAKE COOPER*
| > | > + c: (208) 891-2128 + o: (208) 342-0925
| > | > + 416 S. 8th St., Boise, ID 83702
| > | > + w: drakecooper.com + f: /drakecooper <http://facebook.com/drakecooper>
| > +
| > | >  t: @drakecooper <http://twitter.com/drakecooper>
| > | >
| > | >
| > | > _______________________________________________
| > | > OmniOS-discuss mailing list
| > | > OmniOS-discuss at lists.omniti.com
| > | > http://lists.omniti.com/mailman/listinfo/omnios-discuss
| > | >
| > | >
| > |
| > 
| 
| _______________________________________________
| OmniOS-discuss mailing list
| OmniOS-discuss at lists.omniti.com
| http://lists.omniti.com/mailman/listinfo/omnios-discuss
| 
| 
| 

-- 
Tim Rice				Multitalents	(707) 456-1146
tim at multitalents.net



From mark0x01 at gmail.com  Wed Dec 17 06:44:17 2014
From: mark0x01 at gmail.com (Mark)
Date: Wed, 17 Dec 2014 19:44:17 +1300
Subject: [OmniOS-discuss] Fibre Target problems
In-Reply-To: <1418647518088.75378@steait.net>
References: <541155D5.8080406@gmail.com>	<562C2ABFE2448D4E95E8910A85488708260B0262@HEIMAT.cm3c.local>,
	<5415604A.7030408@gmail.com> <1418571900149.76065@steait.net>,
	<548E7D66.6080208@gmail.com> <1418647518088.75378@steait.net>
Message-ID: <54912641.9020402@gmail.com>

On 16/12/2014 1:45 a.m., Rune Tipsmark wrote:
> where do you check that?
> br,
> Rune
> ________________________________________
> From: Mark <mark0x01 at gmail.com>
> Sent: Monday, December 15, 2014 7:19 AM
> To: Rune Tipsmark; omnios-discuss at lists.omniti.com
> Subject: Re: [OmniOS-discuss] Fibre Target problems
>
> On 15/12/2014 4:44 a.m., Rune Tipsmark wrote:
>> did you ever find a solution?
>> I have the same problem on a SuperMicro based system... FC drops and it causes Windows to loose connection and copying files fails...
>> br,
>> Rune
>
> I installed an older OpenIndiana version to work around the issue.
>
> One thing I didn't check was if the fc target cache was disabled (may be
> the current default), so may be worth checking that.
>
> Mark.
>

To check

stmfadm list-lu -v
LU Name: 600144F0EBE24300000051118E770001
     Operational Status: Online
     Provider Name     : sbd
     Alias             : /dev/zvol/rdsk/dpool/vol1
     View Entry Count  : 1
     Data File         : /dev/zvol/rdsk/dpool/vol1
     Meta File         : not set
     Size              : 2147483648
     Block Size        : 512
     Management URL    : not set
     Vendor ID         : OI
     Product ID        : COMSTAR
     Serial Num        : not set
     Write Protect     : Disabled
     Writeback Cache   : Enabled              <<<<<<<<<<<<<<<
     Access State      : Active


you can change properties with stmfadm modify-lu

I'm not sure of the exact syntax, so check the man page.




From tobi at oetiker.ch  Wed Dec 17 07:54:39 2014
From: tobi at oetiker.ch (Tobias Oetiker)
Date: Wed, 17 Dec 2014 08:54:39 +0100 (CET)
Subject: [OmniOS-discuss] ipadm issues
Message-ID: <alpine.DEB.2.10.1412170838580.25138@froburg.oetiker.ch>

I seem to run into trouble with ipadm ...

here is the latest

 root at simplon:~# ipadm  show-addr
 ADDROBJ           TYPE     STATE        ADDR
 lo0/v4            static   ok           127.0.0.1/8
 mplon0/v4static static   ok           118.124.138.203/28
 mplon0/v4intern static   ok           192.168.1.4/24
 lo0/v6            static   ok           ::1/128
 mplon0/v6       addrconf disabled     ::
 mplon0/v6a      static   disabled     2001:2620:102d::cb/64
 root at mplon:~# ipadm  enable-addr -t mplon0/v6
 ipadm: could not enable address: Object not found
 root at mplon:~# ipadm  enable-addr -t mplon0/v6a
 ipadm: could not enable address: Object not found

I was able to solve this to an extent by removing the ip6 stuff
from /etc/ipadm.conf, and then running ipdam again but it would not
let me add permanent config:

 root at mplon:~# ipadm create-addr -T static -a 2001:2620:102d::cb/64 mplon0/v6a
 ipadm: Could not create address: Persistent operation on temporary object

At least adding a temporary address worked:

 ipadm create-addr -t -T static -a 2001:2620:102d::cb/64 mplon0/v6a

how to make ipadm see the light ?

cheers
tobi

-- 
Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten, Switzerland
www.oetiker.ch tobi at oetiker.ch +41 62 775 9902


From johan.kragsterman at capvert.se  Wed Dec 17 10:02:29 2014
From: johan.kragsterman at capvert.se (Johan Kragsterman)
Date: Wed, 17 Dec 2014 11:02:29 +0100
Subject: [OmniOS-discuss] Ang: Re:  Fibre Target problems
In-Reply-To: <54912641.9020402@gmail.com>
References: <54912641.9020402@gmail.com>,
	<541155D5.8080406@gmail.com>	<562C2ABFE2448D4E95E8910A85488708260B0262@HEIMAT.cm3c.local>,
	<5415604A.7030408@gmail.com> <1418571900149.76065@steait.net>,
	<548E7D66.6080208@gmail.com> <1418647518088.75378@steait.net>
Message-ID: <OF38E261E9.36EEC230-ONC1257DB1.003728EF-C1257DB1.003728F2@inse.com>

Hi!

-----"OmniOS-discuss" <omnios-discuss-bounces at lists.omniti.com> skrev: -----
Till: Rune Tipsmark <rt at steait.net>, "omnios-discuss at lists.omniti.com" <omnios-discuss at lists.omniti.com>
Fr?n: Mark 
S?nt av: "OmniOS-discuss" 
Datum: 2014-12-17 07:45
?rende: Re: [OmniOS-discuss] Fibre Target problems

On 16/12/2014 1:45 a.m., Rune Tipsmark wrote:
> where do you check that?
> br,
> Rune
> ________________________________________
> From: Mark <mark0x01 at gmail.com>
> Sent: Monday, December 15, 2014 7:19 AM
> To: Rune Tipsmark; omnios-discuss at lists.omniti.com
> Subject: Re: [OmniOS-discuss] Fibre Target problems
>
> On 15/12/2014 4:44 a.m., Rune Tipsmark wrote:
>> did you ever find a solution?
>> I have the same problem on a SuperMicro based system... FC drops and it causes Windows to loose connection and copying files fails...
>> br,
>> Rune
>
> I installed an older OpenIndiana version to work around the issue.
>
> One thing I didn't check was if the fc target cache was disabled (may be
> the current default), so may be worth checking that.
>
> Mark.
>

To check

stmfadm list-lu -v
LU Name: 600144F0EBE24300000051118E770001
?? ? Operational Status: Online
?? ? Provider Name ? ? : sbd
?? ? Alias ? ? ? ? ? ? : /dev/zvol/rdsk/dpool/vol1
?? ? View Entry Count ?: 1
?? ? Data File ? ? ? ? : /dev/zvol/rdsk/dpool/vol1
?? ? Meta File ? ? ? ? : not set
?? ? Size ? ? ? ? ? ? ?: 2147483648
?? ? Block Size ? ? ? ?: 512
?? ? Management URL ? ?: not set
?? ? Vendor ID ? ? ? ? : OI
?? ? Product ID ? ? ? ?: COMSTAR
?? ? Serial Num ? ? ? ?: not set
?? ? Write Protect ? ? : Disabled
?? ? Writeback Cache ? : Enabled ? ? ? ? ? ? ?<<<<<<<<<<<<<<<
?? ? Access State ? ? ?: Active


you can change properties with stmfadm modify-lu

I'm not sure of the exact syntax, so check the man page.





I've followed this thread, since it is in my line of work.

I'd be interested to know the progress of this. Are you guys still having this problem?

And about the setup:

The hosts seem to be windows, right? Have you seen this problem with other host OS's? Are they windows on hardware, or windows VM's? If VM's, which hypervisor and are the LU's presented directly to the VM's or through the hypervisor? Do you use NPIV? Other interesting information?

Regards Johan







_______________________________________________
OmniOS-discuss mailing list
OmniOS-discuss at lists.omniti.com
http://lists.omniti.com/mailman/listinfo/omnios-discuss




From jesus at omniti.com  Wed Dec 17 13:48:57 2014
From: jesus at omniti.com (Theo Schlossnagle)
Date: Wed, 17 Dec 2014 08:48:57 -0500
Subject: [OmniOS-discuss] ipadm issues
In-Reply-To: <alpine.DEB.2.10.1412170838580.25138@froburg.oetiker.ch>
References: <alpine.DEB.2.10.1412170838580.25138@froburg.oetiker.ch>
Message-ID: <CACLsApvJ40y91-dbjAbwf6atexrhEknA7R9pZs-jvteVdoT4OA@mail.gmail.com>

I think you have to have an addrconf ipv6 first, then you can add the
static one.

ipadm create-addr -T addrconf mplon0/v6a
ipadm create-addr -T static -a 2001:2620:102d::cb/64 mplon0/v6static

On Wed, Dec 17, 2014 at 2:54 AM, Tobias Oetiker <tobi at oetiker.ch> wrote:
>
> I seem to run into trouble with ipadm ...
>
> here is the latest
>
>  root at simplon:~# ipadm  show-addr
>  ADDROBJ           TYPE     STATE        ADDR
>  lo0/v4            static   ok           127.0.0.1/8
>  mplon0/v4static static   ok           118.124.138.203/28
>  mplon0/v4intern static   ok           192.168.1.4/24
>  lo0/v6            static   ok           ::1/128
>  mplon0/v6       addrconf disabled     ::
>  mplon0/v6a      static   disabled     2001:2620:102d::cb/64
>  root at mplon:~# ipadm  enable-addr -t mplon0/v6
>  ipadm: could not enable address: Object not found
>  root at mplon:~# ipadm  enable-addr -t mplon0/v6a
>  ipadm: could not enable address: Object not found
>
> I was able to solve this to an extent by removing the ip6 stuff
> from /etc/ipadm.conf, and then running ipdam again but it would not
> let me add permanent config:
>
>  root at mplon:~# ipadm create-addr -T static -a 2001:2620:102d::cb/64
> mplon0/v6a
>  ipadm: Could not create address: Persistent operation on temporary object
>
> At least adding a temporary address worked:
>
>  ipadm create-addr -t -T static -a 2001:2620:102d::cb/64 mplon0/v6a
>
> how to make ipadm see the light ?
>
> cheers
> tobi
>
> --
> Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten, Switzerland
> www.oetiker.ch tobi at oetiker.ch +41 62 775 9902
>
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>


-- 

Theo Schlossnagle

http://omniti.com/is/theo-schlossnagle
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141217/1f864e59/attachment.html>

From tobi at oetiker.ch  Wed Dec 17 13:52:50 2014
From: tobi at oetiker.ch (Tobias Oetiker)
Date: Wed, 17 Dec 2014 14:52:50 +0100 (CET)
Subject: [OmniOS-discuss] ipadm issues
In-Reply-To: <CACLsApvJ40y91-dbjAbwf6atexrhEknA7R9pZs-jvteVdoT4OA@mail.gmail.com>
References: <alpine.DEB.2.10.1412170838580.25138@froburg.oetiker.ch>
	<CACLsApvJ40y91-dbjAbwf6atexrhEknA7R9pZs-jvteVdoT4OA@mail.gmail.com>
Message-ID: <alpine.DEB.2.10.1412171452000.1861@froburg.oetiker.ch>

Today Theo Schlossnagle wrote:

> I think you have to have an addrconf ipv6 first, then you can add the
> static one.
>
> ipadm create-addr -T addrconf mplon0/v6a
> ipadm create-addr -T static -a 2001:2620:102d::cb/64 mplon0/v6static

oh ... I did that ... sorry ... not mentioned ... same problem ...
ipadm only accepts the setting with the -t switch

cheers
tobi

>
> On Wed, Dec 17, 2014 at 2:54 AM, Tobias Oetiker <tobi at oetiker.ch> wrote:
> >
> > I seem to run into trouble with ipadm ...
> >
> > here is the latest
> >
> >  root at simplon:~# ipadm  show-addr
> >  ADDROBJ           TYPE     STATE        ADDR
> >  lo0/v4            static   ok           127.0.0.1/8
> >  mplon0/v4static static   ok           118.124.138.203/28
> >  mplon0/v4intern static   ok           192.168.1.4/24
> >  lo0/v6            static   ok           ::1/128
> >  mplon0/v6       addrconf disabled     ::
> >  mplon0/v6a      static   disabled     2001:2620:102d::cb/64
> >  root at mplon:~# ipadm  enable-addr -t mplon0/v6
> >  ipadm: could not enable address: Object not found
> >  root at mplon:~# ipadm  enable-addr -t mplon0/v6a
> >  ipadm: could not enable address: Object not found
> >
> > I was able to solve this to an extent by removing the ip6 stuff
> > from /etc/ipadm.conf, and then running ipdam again but it would not
> > let me add permanent config:
> >
> >  root at mplon:~# ipadm create-addr -T static -a 2001:2620:102d::cb/64
> > mplon0/v6a
> >  ipadm: Could not create address: Persistent operation on temporary object
> >
> > At least adding a temporary address worked:
> >
> >  ipadm create-addr -t -T static -a 2001:2620:102d::cb/64 mplon0/v6a
> >
> > how to make ipadm see the light ?
> >
> > cheers
> > tobi
> >
> > --
> > Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten, Switzerland
> > www.oetiker.ch tobi at oetiker.ch +41 62 775 9902
> >
> > _______________________________________________
> > OmniOS-discuss mailing list
> > OmniOS-discuss at lists.omniti.com
> > http://lists.omniti.com/mailman/listinfo/omnios-discuss
> >
>
>
>

-- 
Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten, Switzerland
www.oetiker.ch tobi at oetiker.ch +41 62 775 9902


From danmcd at omniti.com  Wed Dec 17 15:47:19 2014
From: danmcd at omniti.com (Dan McDonald)
Date: Wed, 17 Dec 2014 10:47:19 -0500
Subject: [OmniOS-discuss] ipadm issues
In-Reply-To: <alpine.DEB.2.10.1412170838580.25138@froburg.oetiker.ch>
References: <alpine.DEB.2.10.1412170838580.25138@froburg.oetiker.ch>
Message-ID: <7690B42A-370A-489F-BE2C-B24DFA726B43@omniti.com>


> On Dec 17, 2014, at 2:54 AM, Tobias Oetiker <tobi at oetiker.ch> wrote:
> 
> I seem to run into trouble with ipadm ...
> 
> here is the latest

Silly question --> what does "ipadm show-if" say?

Dan


From danmcd at omniti.com  Wed Dec 17 16:00:18 2014
From: danmcd at omniti.com (Dan McDonald)
Date: Wed, 17 Dec 2014 11:00:18 -0500
Subject: [OmniOS-discuss] ipadm issues
In-Reply-To: <7690B42A-370A-489F-BE2C-B24DFA726B43@omniti.com>
References: <alpine.DEB.2.10.1412170838580.25138@froburg.oetiker.ch>
	<7690B42A-370A-489F-BE2C-B24DFA726B43@omniti.com>
Message-ID: <2C02DC1A-7DDD-4B01-B11E-26A75D4ECB0E@omniti.com>


> On Dec 17, 2014, at 10:47 AM, Dan McDonald <danmcd at omniti.com> wrote:
> 
> 
>> On Dec 17, 2014, at 2:54 AM, Tobias Oetiker <tobi at oetiker.ch> wrote:
>> 
>> I seem to run into trouble with ipadm ...
>> 
>> here is the latest
> 
> Silly question --> what does "ipadm show-if" say?

Also, I noticed this:

mplon0/v6       addrconf disabled     ::

Try this:

	ipadm enable-if mplon0/v6

Dan


From danmcd at omniti.com  Wed Dec 17 16:15:12 2014
From: danmcd at omniti.com (Dan McDonald)
Date: Wed, 17 Dec 2014 11:15:12 -0500
Subject: [OmniOS-discuss] ipadm issues
In-Reply-To: <2C02DC1A-7DDD-4B01-B11E-26A75D4ECB0E@omniti.com>
References: <alpine.DEB.2.10.1412170838580.25138@froburg.oetiker.ch>
	<7690B42A-370A-489F-BE2C-B24DFA726B43@omniti.com>
	<2C02DC1A-7DDD-4B01-B11E-26A75D4ECB0E@omniti.com>
Message-ID: <4B6A601F-8DE0-47AB-84AD-0C3ABC47C559@omniti.com>


> On Dec 17, 2014, at 11:00 AM, Dan McDonald <danmcd at omniti.com> wrote:
> 
> 
>> On Dec 17, 2014, at 10:47 AM, Dan McDonald <danmcd at omniti.com> wrote:
>> 
>> 
>>> On Dec 17, 2014, at 2:54 AM, Tobias Oetiker <tobi at oetiker.ch> wrote:
>>> 
>>> I seem to run into trouble with ipadm ...
>>> 
>>> here is the latest
>> 
>> Silly question --> what does "ipadm show-if" say?
> 
> Also, I noticed this:
> 
> mplon0/v6       addrconf disabled     ::

Yeah... it's the disabled part that's likely getting you.  I tried some experiments on one of my VMs, and once things were "enabled" life got a lot better.

Dan


From jimklimov at cos.ru  Wed Dec 17 21:16:37 2014
From: jimklimov at cos.ru (Jim Klimov)
Date: Wed, 17 Dec 2014 22:16:37 +0100
Subject: [OmniOS-discuss] latency spikes ~every hour
In-Reply-To: <1418646462467.87062@steait.net>
References: <1418563662763.71128@steait.net> <1418646462467.87062@steait.net>
Message-ID: <7429C727-AFAF-4C6E-9EA6-C27B4F291CA2@cos.ru>

15 ??????? 2014??. 13:27:39 CET, Rune Tipsmark <rt at steait.net> ?????:
>ok I removed some of my SLOG devices and currently I am only using a
>single SLOG (no mirror or anything) and no spikes seen since.
>
>I wonder why multiple SLOG devices would cause this.
>
>br.
>
>Rune
>
>________________________________
>From: OmniOS-discuss <omnios-discuss-bounces at lists.omniti.com> on
>behalf of Rune Tipsmark <rt at steait.net>
>Sent: Sunday, December 14, 2014 2:27 PM
>To: omnios-discuss at lists.omniti.com
>Subject: [OmniOS-discuss] latency spikes ~every hour
>
>
>hi all,
>
>
>
>All my vSphere (ESXi5.1) hosts experience a big spike in latency every
>hour or so.
>
>I tested on Infiniband iSER and SRP and also 4Gbit FC and 8GBit FC. All
>exhibit the same behavior so I don't think its the connection that is
>causing this.
>
>When I modify the arc_shrink_shift 10 (192GB Ram in the System) it
>helps a bit, the spikes are still with the same regularity but latency
>peaks at about ~5000ms for the most part. if I leave arc_shrink_shift
>at the default value they can be higher - up to 15000ms.
>
>Looking at the latency as seen from the vSphere hosts the average is
>usually below 1ms for most datastores.
>
>
>
>Any ideas what can cause this or what can be done to fix?
>
>br,
>
>Rune
>
>
>------------------------------------------------------------------------
>
>_______________________________________________
>OmniOS-discuss mailing list
>OmniOS-discuss at lists.omniti.com
>http://lists.omniti.com/mailman/listinfo/omnios-discuss

One idea stems from old experience: there used to be two triggers to start a TXG sync, where cached (async) writes do actually get to disk: a timeout and a filled-up buffer. While the sync is underway, a system indeed feels locked, new i/o's are delayed, etc. In such cases the tuning was to decrease time or rather size limits, to make the txg syncs shorter and faster, though at risk of more fragmentation of on-disk data. 

This is easily seen in iostat or zpool iostat as a considerable spike in write traffic (with your ram size, the metadata is probably cached and there are not many reads needed to complete the write), occurring regularly (i.e. 5secs).

Your hourly hiccups = backups starting up, maybe?

It may be that the mechanism or the tuning handles were changed in the past year or two in illumos-zfs, but it may be that your (unnamed?) distrp still has that behavior.

--
Typos courtesy of K-9 Mail on my Samsung Android

From tobi at oetiker.ch  Wed Dec 17 23:30:58 2014
From: tobi at oetiker.ch (Tobias Oetiker)
Date: Thu, 18 Dec 2014 00:30:58 +0100 (CET)
Subject: [OmniOS-discuss] ipadm issues
In-Reply-To: <2C02DC1A-7DDD-4B01-B11E-26A75D4ECB0E@omniti.com>
References: <alpine.DEB.2.10.1412170838580.25138@froburg.oetiker.ch>
	<7690B42A-370A-489F-BE2C-B24DFA726B43@omniti.com>
	<2C02DC1A-7DDD-4B01-B11E-26A75D4ECB0E@omniti.com>
Message-ID: <alpine.DEB.2.10.1412180025460.13169@froburg.oetiker.ch>

Hi Dan,

Yesterday Dan McDonald wrote:

>
> > On Dec 17, 2014, at 10:47 AM, Dan McDonald <danmcd at omniti.com> wrote:
> >
> >
> >> On Dec 17, 2014, at 2:54 AM, Tobias Oetiker <tobi at oetiker.ch> wrote:
> >>
> >> I seem to run into trouble with ipadm ...
> >>
> >> here is the latest
> >
> > Silly question --> what does "ipadm show-if" say?
>
> Also, I noticed this:
>
> mplon0/v6       addrconf disabled     ::
>
> Try this:
>
> 	ipadm enable-if mplon0/v6

I tried enable-addr ... maybe that was the problem ...
since I have now setup the temporary interface ... I can't test
this ...

on another note though ...

as I setup addr conf, I got both the linklocal address as well as a
selfassigned global address ... how can I tell ipadm NOT todo that
and to be happy with the linklocal address, since I am assigning a
fixed address by hand

cheers
tobi


> Dan
>
>

-- 
Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten, Switzerland
www.oetiker.ch tobi at oetiker.ch +41 62 775 9902


From danmcd at omniti.com  Wed Dec 17 23:55:27 2014
From: danmcd at omniti.com (Dan McDonald)
Date: Wed, 17 Dec 2014 18:55:27 -0500
Subject: [OmniOS-discuss] ipadm issues
In-Reply-To: <alpine.DEB.2.10.1412180025460.13169@froburg.oetiker.ch>
References: <alpine.DEB.2.10.1412170838580.25138@froburg.oetiker.ch>
	<7690B42A-370A-489F-BE2C-B24DFA726B43@omniti.com>
	<2C02DC1A-7DDD-4B01-B11E-26A75D4ECB0E@omniti.com>
	<alpine.DEB.2.10.1412180025460.13169@froburg.oetiker.ch>
Message-ID: <7D2BC64A-F3BC-4F86-9A97-B119A0646927@omniti.com>


> On Dec 17, 2014, at 6:30 PM, Tobias Oetiker <tobi at oetiker.ch> wrote:
> 
> Hi Dan,
> 
> Yesterday Dan McDonald wrote:
> 
>> 
>>> On Dec 17, 2014, at 10:47 AM, Dan McDonald <danmcd at omniti.com> wrote:
>>> 
>>> 
>>>> On Dec 17, 2014, at 2:54 AM, Tobias Oetiker <tobi at oetiker.ch> wrote:
>>>> 
>>>> I seem to run into trouble with ipadm ...
>>>> 
>>>> here is the latest
>>> 
>>> Silly question --> what does "ipadm show-if" say?
>> 
>> Also, I noticed this:
>> 
>> mplon0/v6       addrconf disabled     ::
>> 
>> Try this:
>> 
>> 	ipadm enable-if mplon0/v6
> 
> I tried enable-addr ... maybe that was the problem ...
> since I have now setup the temporary interface ... I can't test
> this ...
> 
> on another note though ...
> 
> as I setup addr conf, I got both the linklocal address as well as a
> selfassigned global address ... how can I tell ipadm NOT todo that
> and to be happy with the linklocal address, since I am assigning a
> fixed address by hand

The ipadm(1M) page says:

     ipadm create-addr [-t] -T addrconf [-i interface-id]
        [-p {stateful|stateless}={yes|no},..] addrobj

<SNIP!>

             The -p option (also --prop) indicates  which  method
             of auto-configuration should be used.

I believe you'll want to disable both stateful and stateless, or at the very least, stateful (which grabs prefixes via in.ndpd).

Dan


From danmcd at omniti.com  Thu Dec 18 02:44:34 2014
From: danmcd at omniti.com (Dan McDonald)
Date: Wed, 17 Dec 2014 21:44:34 -0500
Subject: [OmniOS-discuss] Followon review --> Changes that allow
	illumos-gate to build on OmniOS
Message-ID: <72C4BB3D-7C98-406B-9254-8CE1D5AEE956@omniti.com>

Now that the java fixes are their own thing, let's revisit the remaining changes in:

	http://kebe.com/~danmcd/webrevs/for-omnios/

All of these together, combined with 4719, will allow illumos-gate to build on OmniOS, AND allow an OmniOS box to ONU to the results, provided the .env file is properly populated.

The aforementioned directory contains:

check-rtime/

* Some check_rtime modifications for the version of things in OmniOS

ig-on-omnios.env

* A sample .env file used for building illumos-gate on OmniOS.  Look for the extra export variables you'll need.

java-updates

* Now points to illumos 4719, see previous mail.

lose-ipp-listener

* Lose the IPP listener if so specified in your .env file, which depends on bits not in OmniOS.
* See the sample .env file above for how it works.

lose-shell-styleguide

* Lose the dependency on docbooks, only existing in the Shell Style Guide.
* Apparently not only is this out in OmniOS already, but in SmartOS as well.

lose-sysidtool-dep

* Lose the SMF dependency on sysidtool.
* I've been told this only ever affected people trying to move from old OpenSolaris to OI.

modern-netsmnp

* Allow linkage with more recent versions of net-snmp.

more-lintfixes

* Lint fixes that the OmniOS sunstudio12.1 lint are picky about.
* Work done by Rich Lowe.  Thank you, Rich!

omnios-mail-msg.txt

* See the small number of remaining lint errors, ALL having to do with openssl versions.

perl-fixes

* More perl fixes to work with OmniOS's alternative approach to perl support.
* This has also been tested, I believe, by the OpenIndiana Hipster people.

After 4719 goes back, I'd REALLY like to get these 7 other changesets back as well.  They'll be NOPs as far as OI is concerned, but they'll help out OmniOS immensely, and possibly other distros as well.

Thanks,
Dan McDonald

From danmcd at omniti.com  Thu Dec 18 02:45:52 2014
From: danmcd at omniti.com (Dan McDonald)
Date: Wed, 17 Dec 2014 21:45:52 -0500
Subject: [OmniOS-discuss] Followon review --> Changes that allow
	illumos-gate to build on OmniOS
In-Reply-To: <72C4BB3D-7C98-406B-9254-8CE1D5AEE956@omniti.com>
References: <72C4BB3D-7C98-406B-9254-8CE1D5AEE956@omniti.com>
Message-ID: <0054A484-17B3-4B6E-821F-6056388E10AD@omniti.com>

Two other things:

1.) I believe these changes could go back independently of 4719, but without the benefit of helping OmniOS directly.

2.) Also, I understand I'll need to file bugs for these diffs.

Thanks,
Dan


From stephan.budach at JVM.DE  Thu Dec 18 07:54:05 2014
From: stephan.budach at JVM.DE (Stephan Budach)
Date: Thu, 18 Dec 2014 08:54:05 +0100
Subject: [OmniOS-discuss] No gcc after pkg install developer/gcc48 on r012
Message-ID: <5492881D.5060504@jvm.de>

Good morning,

I wanted to install gcc to compile znapzend on my r012 box, so I issued:

root at nfsvmpool02:/root# pkg install developer/gcc48
            Packages to install:  6
        Create boot environment: No
Create backup boot environment: No

DOWNLOAD                                  PKGS FILES    XFER (MB)
Completed                                  6/6   1537/1537 165.1/165.1

PHASE                                        ACTIONS
Install Phase                              2104/2104

PHASE                                          ITEMS
Package State Update Phase                       6/6
Image State Update Phase                         2/2
root at nfsvmpool02:/root# pkg install gnu-make
            Packages to install:  1
        Create boot environment: No
Create backup boot environment: No

DOWNLOAD                                  PKGS FILES    XFER (MB)
Completed                                  1/1 31/31      0.5/0.5

PHASE                                        ACTIONS
Install Phase                                  98/98

PHASE                                          ITEMS
Package State Update Phase                       1/1
Image State Update Phase                         2/2


However, gcc is not available afterwards:

root at nfsvmpool02:/root# gcc
-bash: gcc: command not found

I did the same on my r006 box using pkg install developer/gcc47, which 
installed gcc as expected.
What did I do wrong? ;)

Thanks,
budy

From henson at acm.org  Thu Dec 18 08:15:39 2014
From: henson at acm.org (Paul B. Henson)
Date: Thu, 18 Dec 2014 00:15:39 -0800
Subject: [OmniOS-discuss] No gcc after pkg install developer/gcc48 on
	r012
In-Reply-To: <5492881D.5060504@jvm.de>
References: <5492881D.5060504@jvm.de>
Message-ID: <3268807D-987E-459B-AB4D-D651121F8801@acm.org>

gcc on omnios isn't installed in the default path, you need to add /opt/gccXX/bin to your path. I'm pretty sure it's always been like that, so if you installed and used gcc on an earlier version, it's likely you added it to your path to do so. 



> On Dec 17, 2014, at 11:54 PM, Stephan Budach <stephan.budach at JVM.DE> wrote:
> 
> Good morning,
> 
> I wanted to install gcc to compile znapzend on my r012 box, so I issued:
> 
> root at nfsvmpool02:/root# pkg install developer/gcc48
>           Packages to install:  6
>       Create boot environment: No
> Create backup boot environment: No
> 
> DOWNLOAD                                  PKGS FILES    XFER (MB)
> Completed                                  6/6   1537/1537 165.1/165.1
> 
> PHASE                                        ACTIONS
> Install Phase                              2104/2104
> 
> PHASE                                          ITEMS
> Package State Update Phase                       6/6
> Image State Update Phase                         2/2
> root at nfsvmpool02:/root# pkg install gnu-make
>           Packages to install:  1
>       Create boot environment: No
> Create backup boot environment: No
> 
> DOWNLOAD                                  PKGS FILES    XFER (MB)
> Completed                                  1/1 31/31      0.5/0.5
> 
> PHASE                                        ACTIONS
> Install Phase                                  98/98
> 
> PHASE                                          ITEMS
> Package State Update Phase                       1/1
> Image State Update Phase                         2/2
> 
> 
> However, gcc is not available afterwards:
> 
> root at nfsvmpool02:/root# gcc
> -bash: gcc: command not found
> 
> I did the same on my r006 box using pkg install developer/gcc47, which installed gcc as expected.
> What did I do wrong? ;)
> 
> Thanks,
> budy
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss

From ml.omnios-discuss at valo.at  Thu Dec 18 08:53:52 2014
From: ml.omnios-discuss at valo.at (Christian Kivalo)
Date: Thu, 18 Dec 2014 09:53:52 +0100
Subject: [OmniOS-discuss] No gcc after pkg install developer/gcc48 on
	r012
In-Reply-To: <5492881D.5060504@jvm.de>
References: <5492881D.5060504@jvm.de>
Message-ID: <c18130faf575c050192c5cd4a684269e@kivalo.at>

Good Morning,

I ran into the exact same problem yesterday and found this very informative
page about setting up a basic dev environment on OmniOS:
Link http://omnios.omniti.com/wiki.php/DevEnv

I added the gcc bin dir to my $PATH and installed all the packages mentioned on
that site and znapzend compiled.

-christian

On Thu, 18 Dec 2014 08:54:05 +0100 Stephan Budach <stephan.budach at jvm.de> wrote

> Good morning,
> 
> I wanted to install gcc to compile znapzend on my r012 box, so I issued:
> 
> root at nfsvmpool02:/root# pkg install developer/gcc48
>             Packages to install:  6
>         Create boot environment: No
> Create backup boot environment: No
> 
> DOWNLOAD                                  PKGS FILES    XFER (MB)
> Completed                                  6/6   1537/1537 165.1/165.1
> 
> PHASE                                        ACTIONS
> Install Phase                              2104/2104
> 
> PHASE                                          ITEMS
> Package State Update Phase                       6/6
> Image State Update Phase                         2/2
> root at nfsvmpool02:/root# pkg install gnu-make
>             Packages to install:  1
>         Create boot environment: No
> Create backup boot environment: No
> 
> DOWNLOAD                                  PKGS FILES    XFER (MB)
> Completed                                  1/1 31/31      0.5/0.5
> 
> PHASE                                        ACTIONS
> Install Phase                                  98/98
> 
> PHASE                                          ITEMS
> Package State Update Phase                       1/1
> Image State Update Phase                         2/2
> 
> 
> However, gcc is not available afterwards:
> 
> root at nfsvmpool02:/root# gcc
> -bash: gcc: command not found
> 
> I did the same on my r006 box using pkg install developer/gcc47, which 
> installed gcc as expected.
> What did I do wrong? ;)
> 
> Thanks,
> budy
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss



From stephan.budach at JVM.DE  Thu Dec 18 10:42:35 2014
From: stephan.budach at JVM.DE (Stephan Budach)
Date: Thu, 18 Dec 2014 11:42:35 +0100
Subject: [OmniOS-discuss] No gcc after pkg install developer/gcc48 on
 r012
In-Reply-To: <c18130faf575c050192c5cd4a684269e@kivalo.at>
References: <5492881D.5060504@jvm.de>
	<c18130faf575c050192c5cd4a684269e@kivalo.at>
Message-ID: <5492AF9B.8020907@jvm.de>

Am 18.12.14 um 09:53 schrieb Christian Kivalo:
> Good Morning,
>
> I ran into the exact same problem yesterday and found this very informative
> page about setting up a basic dev environment on OmniOS:
> Link http://omnios.omniti.com/wiki.php/DevEnv
>
> I added the gcc bin dir to my $PATH and installed all the packages mentioned on
> that site and znapzend compiled.
>
> -christian

Thanks - it seems, that I rushed over those extra packages. Thanks for 
the heads-up!
After actually completing the installation of the dev tools, I am also 
able to compile znapzend, which is a very neat tool, btw.!

Thanks,
budy
>
> On Thu, 18 Dec 2014 08:54:05 +0100 Stephan Budach <stephan.budach at jvm.de> wrote
>
>> Good morning,
>>
>> I wanted to install gcc to compile znapzend on my r012 box, so I issued:
>>
>> root at nfsvmpool02:/root# pkg install developer/gcc48
>>              Packages to install:  6
>>          Create boot environment: No
>> Create backup boot environment: No
>>
>> DOWNLOAD                                  PKGS FILES    XFER (MB)
>> Completed                                  6/6   1537/1537 165.1/165.1
>>
>> PHASE                                        ACTIONS
>> Install Phase                              2104/2104
>>
>> PHASE                                          ITEMS
>> Package State Update Phase                       6/6
>> Image State Update Phase                         2/2
>> root at nfsvmpool02:/root# pkg install gnu-make
>>              Packages to install:  1
>>          Create boot environment: No
>> Create backup boot environment: No
>>
>> DOWNLOAD                                  PKGS FILES    XFER (MB)
>> Completed                                  1/1 31/31      0.5/0.5
>>
>> PHASE                                        ACTIONS
>> Install Phase                                  98/98
>>
>> PHASE                                          ITEMS
>> Package State Update Phase                       1/1
>> Image State Update Phase                         2/2
>>
>>
>> However, gcc is not available afterwards:
>>
>> root at nfsvmpool02:/root# gcc
>> -bash: gcc: command not found
>>
>> I did the same on my r006 box using pkg install developer/gcc47, which
>> installed gcc as expected.
>> What did I do wrong? ;)
>>
>> Thanks,
>> budy
>> _______________________________________________
>> OmniOS-discuss mailing list
>> OmniOS-discuss at lists.omniti.com
>> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss


-- 
Stephan Budach
Deputy Managing Director
Jung von Matt/it-services GmbH
Glash?ttenstra?e 79
20357 Hamburg


Tel: +49 40-4321-1353
Fax: +49 40-4321-1114
E-Mail: stephan.budach at jvm.de
Internet: http://www.jvm.com

Gesch?ftsf?hrer: Frank Wilhelm, Stephan Budach (stellv.)
AG HH HRB 98380


From johan.kragsterman at capvert.se  Thu Dec 18 15:57:59 2014
From: johan.kragsterman at capvert.se (Johan Kragsterman)
Date: Thu, 18 Dec 2014 16:57:59 +0100
Subject: [OmniOS-discuss] CoreOS
Message-ID: <OF9FC50875.7A47B1B0-ONC1257DB2.00579EAF-C1257DB2.0057B4FE@inse.com>


Hi!


 I've been looking at CoreOS and finds it interesting! Since I'd like to have OmniOS as the platform, I need to run CoreOS as a KVM guest. Haven't tested yet, but I downloaded the startscript for qemu, and it looks a little bit "too much" for Illumos KVM...

 It would be nice to get some views on people that have been considering this as well, perhaps some already tested or already running...?

 I've seen that Frederic Alix on this list been blogging about it, but haven't seen if he managed to run it as a KVM guest on OmniOS.

 For me it seems to be some complications at first startup, mainly. It doesn't seem to be reachable by VNC...

 Hope to get some input from you guys...


Best regards from/Med v?nliga h?lsningar fr?n

Johan Kragsterman

Capvert


From doug at will.to  Thu Dec 18 16:26:40 2014
From: doug at will.to (Doug Hughes)
Date: Thu, 18 Dec 2014 11:26:40 -0500
Subject: [OmniOS-discuss] latent problem with networking
Message-ID: <CAOpmc6xj7EyKWryW6JUof39n9E_=SvcA9iwcpDKzoad5ERazTA@mail.gmail.com>

the machine has been up for a while. It seems like something funky is going
on in the network stack. it pings ok.. ssh inbound new sessions no longer
work. The master ssh process isn't bound to port 22 any more (pfiles).
Active ssh session to this machine continues to work, however.

Also, pkg install gets stuck, looks like on DNS.

But the DNS server is responsive from other hosts, and I can see all
request/response from the client on the DNS server: the client omniOS
machine cannot see the responses from the server, though. Ping between
omnios machine and DNS server is fully functional (initiated from either
side). There is no firewall between them.

I can see existing ssh packets (pre-existing this strange new condition)
going back and forth quite well (also to DNS server) - demonstrating that
there is no intervening network problem.

I can see tcp/udp packets reaching the server from the omniOS machine and
responses going back (but not showing to snoop on omniOS)

Here's the simplest test... I start up ttcp -r on the server, it binds to
port 5001, listening. I run snoop.. Then I try to connect to 5001 from
another machine. I see the packets in snoop, but the accept call on the
omniOS machine never returns. Something seems wonky in network land. Has
anybody seen this? THe machine has been up for weeks without any problems.

  OmniOS v11 r151012
  Copyright 2014 OmniTI Computer Consulting, Inc. All rights reserved.
  Use is subject to license terms.

Regular/plain Intel chipset:
e1000g0:
root at xyr-r:/root# dladm show-link e1000g0
LINK        CLASS     MTU    STATE    BRIDGE     OVER
e1000g0     phys      1500   up       --         --
root at xyr-r:/root# dladm show-phys e1000g0
LINK         MEDIA                STATE      SPEED  DUPLEX    DEVICE
e1000g0      Ethernet             up         1000   full      e1000g0

e1000 prtdiag excerpt:
                    name='device-name' type=string items=1
                        value='82574L Gigabit Network Connection'
                    name='subsystem-name' type=string items=1
                        value='unknown subsystem'
                Device Minor Nodes:
                    dev=(112,1)
                        dev_path=/pci at 0,0/pci8086,1d14 at 1c,2/pci122e,10d3 at 0
:e1000g0


Ideas?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141218/2ebe397f/attachment.html>

From sjorge+ml at blackdot.be  Thu Dec 18 16:37:17 2014
From: sjorge+ml at blackdot.be (Jorge Schrauwen)
Date: Thu, 18 Dec 2014 17:37:17 +0100
Subject: [OmniOS-discuss] CoreOS
In-Reply-To: <OF9FC50875.7A47B1B0-ONC1257DB2.00579EAF-C1257DB2.0057B4FE@inse.com>
References: <OF9FC50875.7A47B1B0-ONC1257DB2.00579EAF-C1257DB2.0057B4FE@inse.com>
Message-ID: <e69c6d0959ec12af17df866b720f5e14@blackdot.be>

Something like this will probably work:


  /usr/bin/qemu-system-x86_64
   -name coreos \
   -enable-kvm \
   -no-hpet \
   -m 4096
   -cpu Nehalem \
   -smp sockets=1,cores=4,threads=2 \
   -rtc base=utc,driftfix=slew \
   -pidfile /tank/coreo/coreos.pid  \
   -monitor unix:/tank/coreo/coreos.monitor,server,nowait,nodelay  \
   -vga std  \
   -vnc :1  \
   -nographic \
   -drive 
file=/tank/coreos/coreos.iso,if=ide,media=cdrom,index=0,cache=none \
   -drive 
file=/dev/zvol/rdsk/tank/coreos/disk0,if=virtio,media=disk,index=0,cache=none,boot=on 
\
   -boot order=cd,once=d \
   -device 
virtio-net-pci,mac=02:08:20:0c:04:d2,tx=timer,x-txtimer=200000,x-txburst=128,vlan=0 
\
   -net vnic,vlan=0,name=net1,ifname=vcoreos0 \
   -chardev 
socket,id=serial0,path=/tank/coreos/coreos.console,server,nowait \
   -serial chardev:serial0 \
   -usb \
   -usbdevice tablet \
   -daemonize

You should get vnc at port 5901, seemed to boot for me but I did not 
complete the install.

Regards

Jorge

On 2014-12-18 16:57, Johan Kragsterman wrote:
> Hi!
> 
> 
>  I've been looking at CoreOS and finds it interesting! Since I'd like
> to have OmniOS as the platform, I need to run CoreOS as a KVM guest.
> Haven't tested yet, but I downloaded the startscript for qemu, and it
> looks a little bit "too much" for Illumos KVM...
> 
>  It would be nice to get some views on people that have been
> considering this as well, perhaps some already tested or already
> running...?
> 
>  I've seen that Frederic Alix on this list been blogging about it, but
> haven't seen if he managed to run it as a KVM guest on OmniOS.
> 
>  For me it seems to be some complications at first startup, mainly. It
> doesn't seem to be reachable by VNC...
> 
>  Hope to get some input from you guys...
> 
> 
> Best regards from/Med v?nliga h?lsningar fr?n
> 
> Johan Kragsterman
> 
> Capvert
> 
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss

From johan.kragsterman at capvert.se  Thu Dec 18 16:57:11 2014
From: johan.kragsterman at capvert.se (Johan Kragsterman)
Date: Thu, 18 Dec 2014 17:57:11 +0100
Subject: [OmniOS-discuss] Ang: Re:  CoreOS
In-Reply-To: <e69c6d0959ec12af17df866b720f5e14@blackdot.be>
References: <e69c6d0959ec12af17df866b720f5e14@blackdot.be>,
	<OF9FC50875.7A47B1B0-ONC1257DB2.00579EAF-C1257DB2.0057B4FE@inse.com>
Message-ID: <OF79C7D6B0.90D89DC4-ONC1257DB2.005C37AA-C1257DB2.005D205C@inse.com>

Jorge, I was thinking about you when I posted this! I thought you would be a possible contributor to this thread...  More furhter down...


-----Jorge Schrauwen <sjorge+ml at blackdot.be> skrev: -----
Till: Johan Kragsterman <johan.kragsterman at capvert.se>
Fr?n: Jorge Schrauwen <sjorge+ml at blackdot.be>
Datum: 2014-12-18 17:38
Kopia: omnios-discuss at lists.omniti.com
?rende: Re: [OmniOS-discuss] CoreOS

Something like this will probably work:


??/usr/bin/qemu-system-x86_64
?? -name coreos \
?? -enable-kvm \
?? -no-hpet \
?? -m 4096
?? -cpu Nehalem \
?? -smp sockets=1,cores=4,threads=2 \
?? -rtc base=utc,driftfix=slew \
?? -pidfile /tank/coreo/coreos.pid ?\
?? -monitor unix:/tank/coreo/coreos.monitor,server,nowait,nodelay ?\
?? -vga std ?\
?? -vnc :1 ?\
?? -nographic \
?? -drive 
file=/tank/coreos/coreos.iso,if=ide,media=cdrom,index=0,cache=none \
?? -drive 
file=/dev/zvol/rdsk/tank/coreos/disk0,if=virtio,media=disk,index=0,cache=none,boot=on 
\
?? -boot order=cd,once=d \
?? -device 
virtio-net-pci,mac=02:08:20:0c:04:d2,tx=timer,x-txtimer=200000,x-txburst=128,vlan=0 
\
?? -net vnic,vlan=0,name=net1,ifname=vcoreos0 \
?? -chardev 
socket,id=serial0,path=/tank/coreos/coreos.console,server,nowait \
?? -serial chardev:serial0 \
?? -usb \
?? -usbdevice tablet \
?? -daemonize

You should get vnc at port 5901, seemed to boot for me but I did not 
complete the install.



At the CoreOS site they say: Start like this: 

./coreos_production_qemu.sh -nographic

and they pass on that string -nographic  ...?

It makes me wonder, because they tell you to connect with the instans only over ssh with: ssh -l core -p 2222 localhost   ...

So I'm not sure if it is possible to connect via VNC...did you actually check VNC, to confirm you had a VNC connection?

It should boot and run from the image r/o, so perhaps you just need one "disk"? I can see you got two configured, or at least the iso file, and then a disk. Don't you think it would be enough with just the image file?

Perhaps I just try...



Regards

Jorge












On 2014-12-18 16:57, Johan Kragsterman wrote:
> Hi!
> 
> 
> ?I've been looking at CoreOS and finds it interesting! Since I'd like
> to have OmniOS as the platform, I need to run CoreOS as a KVM guest.
> Haven't tested yet, but I downloaded the startscript for qemu, and it
> looks a little bit "too much" for Illumos KVM...
> 
> ?It would be nice to get some views on people that have been
> considering this as well, perhaps some already tested or already
> running...?
> 
> ?I've seen that Frederic Alix on this list been blogging about it, but
> haven't seen if he managed to run it as a KVM guest on OmniOS.
> 
> ?For me it seems to be some complications at first startup, mainly. It
> doesn't seem to be reachable by VNC...
> 
> ?Hope to get some input from you guys...
> 
> 
> Best regards from/Med v?nliga h?lsningar fr?n
> 
> Johan Kragsterman
> 
> Capvert
> 
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss




From sjorge+ml at blackdot.be  Thu Dec 18 17:06:17 2014
From: sjorge+ml at blackdot.be (Jorge Schrauwen)
Date: Thu, 18 Dec 2014 18:06:17 +0100
Subject: [OmniOS-discuss] Ang: Re:  CoreOS
In-Reply-To: <OF79C7D6B0.90D89DC4-ONC1257DB2.005C37AA-C1257DB2.005D205C@inse.com>
References: <e69c6d0959ec12af17df866b720f5e14@blackdot.be>,
	<OF9FC50875.7A47B1B0-ONC1257DB2.00579EAF-C1257DB2.0057B4FE@inse.com>
	<OF79C7D6B0.90D89DC4-ONC1257DB2.005C37AA-C1257DB2.005D205C@inse.com>
Message-ID: <509071d00c17bdf9229b0be3eb8bfe8b@blackdot.be>



On 2014-12-18 17:57, Johan Kragsterman wrote:
> Jorge, I was thinking about you when I posted this! I thought you
> would be a possible contributor to this thread...  More furhter
> down...
> 
> 
> -----Jorge Schrauwen <sjorge+ml at blackdot.be> skrev: -----
> Till: Johan Kragsterman <johan.kragsterman at capvert.se>
> Fr?n: Jorge Schrauwen <sjorge+ml at blackdot.be>
> Datum: 2014-12-18 17:38
> Kopia: omnios-discuss at lists.omniti.com
> ?rende: Re: [OmniOS-discuss] CoreOS
> 
> Something like this will probably work:
> 
> 
> ??/usr/bin/qemu-system-x86_64
> ?? -name coreos \
> ?? -enable-kvm \
> ?? -no-hpet \
> ?? -m 4096
> ?? -cpu Nehalem \
> ?? -smp sockets=1,cores=4,threads=2 \
> ?? -rtc base=utc,driftfix=slew \
> ?? -pidfile /tank/coreo/coreos.pid ?\
> ?? -monitor unix:/tank/coreo/coreos.monitor,server,nowait,nodelay ?\
> ?? -vga std ?\
> ?? -vnc :1 ?\
> ?? -nographic \
> ?? -drive
> file=/tank/coreos/coreos.iso,if=ide,media=cdrom,index=0,cache=none \
> ?? -drive
> file=/dev/zvol/rdsk/tank/coreos/disk0,if=virtio,media=disk,index=0,cache=none,boot=on
> \
> ?? -boot order=cd,once=d \
> ?? -device
> virtio-net-pci,mac=02:08:20:0c:04:d2,tx=timer,x-txtimer=200000,x-txburst=128,vlan=0
> \
> ?? -net vnic,vlan=0,name=net1,ifname=vcoreos0 \
> ?? -chardev
> socket,id=serial0,path=/tank/coreos/coreos.console,server,nowait \
> ?? -serial chardev:serial0 \
> ?? -usb \
> ?? -usbdevice tablet \
> ?? -daemonize
> 
> You should get vnc at port 5901, seemed to boot for me but I did not
> complete the install.
> 
> 
> 
> At the CoreOS site they say: Start like this:
> 
> ./coreos_production_qemu.sh -nographic
> 
> and they pass on that string -nographic  ...?
> 
> It makes me wonder, because they tell you to connect with the instans
> only over ssh with: ssh -l core -p 2222 localhost   ...
> 
> So I'm not sure if it is possible to connect via VNC...did you
> actually check VNC, to confirm you had a VNC connection?
> 
> It should boot and run from the image r/o, so perhaps you just need
> one "disk"? I can see you got two configured, or at least the iso
> file, and then a disk. Don't you think it would be enough with just
> the image file?
> 
> Perhaps I just try...
I used the install iso to see if it booted.
-nographic just mean don't spawn a graphical console AKA SDL or simular 
window. It does not prevent '-vnc :1' from working.

> 
> 
> 
> Regards
> 
> Jorge
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> On 2014-12-18 16:57, Johan Kragsterman wrote:
>> Hi!
>> 
>> 
>> ?I've been looking at CoreOS and finds it interesting! Since I'd like
>> to have OmniOS as the platform, I need to run CoreOS as a KVM guest.
>> Haven't tested yet, but I downloaded the startscript for qemu, and it
>> looks a little bit "too much" for Illumos KVM...
>> 
>> ?It would be nice to get some views on people that have been
>> considering this as well, perhaps some already tested or already
>> running...?
>> 
>> ?I've seen that Frederic Alix on this list been blogging about it, but
>> haven't seen if he managed to run it as a KVM guest on OmniOS.
>> 
>> ?For me it seems to be some complications at first startup, mainly. It
>> doesn't seem to be reachable by VNC...
>> 
>> ?Hope to get some input from you guys...
>> 
>> 
>> Best regards from/Med v?nliga h?lsningar fr?n
>> 
>> Johan Kragsterman
>> 
>> Capvert
>> 
>> _______________________________________________
>> OmniOS-discuss mailing list
>> OmniOS-discuss at lists.omniti.com
>> http://lists.omniti.com/mailman/listinfo/omnios-discuss

From danmcd at omniti.com  Thu Dec 18 18:21:09 2014
From: danmcd at omniti.com (Dan McDonald)
Date: Thu, 18 Dec 2014 13:21:09 -0500
Subject: [OmniOS-discuss] latent problem with networking
In-Reply-To: <CAOpmc6xj7EyKWryW6JUof39n9E_=SvcA9iwcpDKzoad5ERazTA@mail.gmail.com>
References: <CAOpmc6xj7EyKWryW6JUof39n9E_=SvcA9iwcpDKzoad5ERazTA@mail.gmail.com>
Message-ID: <C3AF1D99-E674-466E-9F22-E57B33707F29@omniti.com>


> On Dec 18, 2014, at 11:26 AM, Doug Hughes <doug at will.to> wrote:
> 
> 
> Here's the simplest test... I start up ttcp -r on the server, it binds to port 5001, listening. I run snoop.. Then I try to connect to 5001 from another machine. I see the packets in snoop, but the accept call on the omniOS machine never returns. Something seems wonky in network land. Has anybody seen this? THe machine has been up for weeks without any problems.
> 
>   OmniOS v11 r151012
>   Copyright 2014 OmniTI Computer Consulting, Inc. All rights reserved.
>   Use is subject to license terms.
> 
> Regular/plain Intel chipset:
> e1000g0: 
> root at xyr-r:/root# dladm show-link e1000g0
> LINK        CLASS     MTU    STATE    BRIDGE     OVER
> e1000g0     phys      1500   up       --         --
> root at xyr-r:/root# dladm show-phys e1000g0
> LINK         MEDIA                STATE      SPEED  DUPLEX    DEVICE
> e1000g0      Ethernet             up         1000   full      e1000g0
> 
> e1000 prtdiag excerpt:
>                     name='device-name' type=string items=1
>                         value='82574L Gigabit Network Connection'

I can't recall if this chipset has problems or not.  I want to say it might, BUT I'm not sure, so I won't point fingers.

>                     name='subsystem-name' type=string items=1
>                         value='unknown subsystem'
>                 Device Minor Nodes:
>                     dev=(112,1)
>                         dev_path=/pci at 0,0/pci8086,1d14 at 1c,2/pci122e,10d3 at 0:e1000g0
> Ideas?

If you've the disk space, please utter "savecore -L" while your machine is in this state.  It might be nice to have the system state while things are failing.

Do you see any complaints from e1000g in /var/adm/messages?

It's like the NIC or the driver stopped receiving packets.

One thing you could do is unplumb and replumb the interface.  That may make the kernel reset the driver.

	ifconfig e1000g0 unplumb
	ifconfig e1000g0 plumb <addr/prefix> up

If that doesn't work, you may also need to modunload the driver before replumbing.

	ifconfig e1000g0 unplumb
	modinfo | grep e1000g
	modunload -i <number from modinfo line>
	ifconfig e1000g0 plumb ....

If modunload complains, you will need to unplumb the v6 interface ("ifconfig e1000g0 inet6 unplumb") or maybe disable some other services temporarily.

Dan


From doug at will.to  Thu Dec 18 18:30:39 2014
From: doug at will.to (Doug Hughes)
Date: Thu, 18 Dec 2014 13:30:39 -0500
Subject: [OmniOS-discuss] latent problem with networking
In-Reply-To: <C3AF1D99-E674-466E-9F22-E57B33707F29@omniti.com>
References: <CAOpmc6xj7EyKWryW6JUof39n9E_=SvcA9iwcpDKzoad5ERazTA@mail.gmail.com>
	<C3AF1D99-E674-466E-9F22-E57B33707F29@omniti.com>
Message-ID: <CAOpmc6wAWBrqDXXMn3doCey88jzX94Q-wZwVYxuH8+_rB7L_cw@mail.gmail.com>

well, we figured it out..

it was pretty silly actually.. It looks like for this machine, at this
location, and without the network/routing/route disabled, it was picking up
a *second* default route.. so some of the packets (seemingly acks and other
TCP activity -- somewhat important!) were ending up at this other router
which belongs to a peer organization and we're not making it all the way to
the remote side under certain circumstances. Once that second default route
was removed, everything was fixed. It never affected ping, and my existing
ssh was working fine. I have no idea why this suddenly started causing a
problem!

I'm glad it turned out to be something simple.


On Thu, Dec 18, 2014 at 1:21 PM, Dan McDonald <danmcd at omniti.com> wrote:
>
>
> > On Dec 18, 2014, at 11:26 AM, Doug Hughes <doug at will.to> wrote:
> >
> >
> > Here's the simplest test... I start up ttcp -r on the server, it binds
> to port 5001, listening. I run snoop.. Then I try to connect to 5001 from
> another machine. I see the packets in snoop, but the accept call on the
> omniOS machine never returns. Something seems wonky in network land. Has
> anybody seen this? THe machine has been up for weeks without any problems.
> >
> >   OmniOS v11 r151012
> >   Copyright 2014 OmniTI Computer Consulting, Inc. All rights reserved.
> >   Use is subject to license terms.
> >
> > Regular/plain Intel chipset:
> > e1000g0:
> > root at xyr-r:/root# dladm show-link e1000g0
> > LINK        CLASS     MTU    STATE    BRIDGE     OVER
> > e1000g0     phys      1500   up       --         --
> > root at xyr-r:/root# dladm show-phys e1000g0
> > LINK         MEDIA                STATE      SPEED  DUPLEX    DEVICE
> > e1000g0      Ethernet             up         1000   full      e1000g0
> >
> > e1000 prtdiag excerpt:
> >                     name='device-name' type=string items=1
> >                         value='82574L Gigabit Network Connection'
>
> I can't recall if this chipset has problems or not.  I want to say it
> might, BUT I'm not sure, so I won't point fingers.
>
> >                     name='subsystem-name' type=string items=1
> >                         value='unknown subsystem'
> >                 Device Minor Nodes:
> >                     dev=(112,1)
> >                         dev_path=/pci at 0,0/pci8086,1d14 at 1c
> ,2/pci122e,10d3 at 0:e1000g0
> > Ideas?
>
> If you've the disk space, please utter "savecore -L" while your machine is
> in this state.  It might be nice to have the system state while things are
> failing.
>
> Do you see any complaints from e1000g in /var/adm/messages?
>
> It's like the NIC or the driver stopped receiving packets.
>
> One thing you could do is unplumb and replumb the interface.  That may
> make the kernel reset the driver.
>
>         ifconfig e1000g0 unplumb
>         ifconfig e1000g0 plumb <addr/prefix> up
>
> If that doesn't work, you may also need to modunload the driver before
> replumbing.
>
>         ifconfig e1000g0 unplumb
>         modinfo | grep e1000g
>         modunload -i <number from modinfo line>
>         ifconfig e1000g0 plumb ....
>
> If modunload complains, you will need to unplumb the v6 interface
> ("ifconfig e1000g0 inet6 unplumb") or maybe disable some other services
> temporarily.
>
> Dan
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141218/87da12ce/attachment.html>

From danmcd at omniti.com  Thu Dec 18 18:34:18 2014
From: danmcd at omniti.com (Dan McDonald)
Date: Thu, 18 Dec 2014 13:34:18 -0500
Subject: [OmniOS-discuss] latent problem with networking
In-Reply-To: <CAOpmc6wAWBrqDXXMn3doCey88jzX94Q-wZwVYxuH8+_rB7L_cw@mail.gmail.com>
References: <CAOpmc6xj7EyKWryW6JUof39n9E_=SvcA9iwcpDKzoad5ERazTA@mail.gmail.com>
	<C3AF1D99-E674-466E-9F22-E57B33707F29@omniti.com>
	<CAOpmc6wAWBrqDXXMn3doCey88jzX94Q-wZwVYxuH8+_rB7L_cw@mail.gmail.com>
Message-ID: <59A588F7-8C4D-4198-94A1-3CCF9BE91E22@omniti.com>


> On Dec 18, 2014, at 1:30 PM, Doug Hughes <doug at will.to> wrote:
> 
> well, we figured it out..
> 
> it was pretty silly actually.. It looks like for this machine, at this location, and without the network/routing/route disabled, it was picking up a *second* default route.. so some of the packets (seemingly acks and other TCP activity -- somewhat important!) were ending up at this other router which belongs to a peer organization and we're not making it all the way to the remote side under certain circumstances. Once that second default route was removed, everything was fixed. It never affected ping, and my existing ssh was working fine. I have no idea why this suddenly started causing a problem!
> 
> I'm glad it turned out to be something simple.

Me too.  When I see questions about drivers, I assume the routing situation had already been inspected.  Glad you figured that out on your own, and thank you for the update!

Dan


From mir at miras.org  Thu Dec 18 18:39:17 2014
From: mir at miras.org (Michael Rasmussen)
Date: Thu, 18 Dec 2014 19:39:17 +0100
Subject: [OmniOS-discuss] latent problem with networking
In-Reply-To: <C3AF1D99-E674-466E-9F22-E57B33707F29@omniti.com>
References: <CAOpmc6xj7EyKWryW6JUof39n9E_=SvcA9iwcpDKzoad5ERazTA@mail.gmail.com>
	<C3AF1D99-E674-466E-9F22-E57B33707F29@omniti.com>
Message-ID: <20141218193917.7f2e2138@sleipner.datanom.net>

On Thu, 18 Dec 2014 13:21:09 -0500
Dan McDonald <danmcd at omniti.com> wrote:

> 
> I can't recall if this chipset has problems or not.  I want to say it might, BUT I'm not sure, so I won't point fingers.
> 
I have run with these chipsets on a Hudson based AMD board since 151008
was bloody (Sep/Oct 2013). It has been rock solid since then and still
is. 

-- 
Hilsen/Regards
Michael Rasmussen

Get my public GnuPG keys:
michael <at> rasmussen <dot> cc
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E
mir <at> datanom <dot> net
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C
mir <at> miras <dot> org
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917
--------------------------------------------------------------
/usr/games/fortune -es says:
You can't have everything.  Where would you put it?
		-- Steven Wright
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 181 bytes
Desc: OpenPGP digital signature
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141218/a1410e7f/attachment.bin>

From danmcd at omniti.com  Thu Dec 18 19:12:20 2014
From: danmcd at omniti.com (Dan McDonald)
Date: Thu, 18 Dec 2014 14:12:20 -0500
Subject: [OmniOS-discuss] Last OmniOS bloody update for 2014 (long,
	please read)
Message-ID: <B2AB195E-D342-4061-8179-E43D2DBF3D36@omniti.com>

This will be the last OmniOS bloody update for the calendar year, AND you likely won't see the next one until the end of January.  Please read on for what's new, and why the big upcoming gap.

Only out to the pkg servers this time, this OmniOS bloody update (r151013-20141218):

- omnios-build master branch, revision 9d22dd3

- Fix for os.mknod() in 64-bit runtimes.  If any python wizards in the audience can help me upstream this, I'd appreciate it.

- New M2Crypto features needed for CRL handling, part of the pkg(5) modernization effort.

- illumos-omnios master branch, revision eb4c8f3 (last illumos-gate merge 9c30721)

- Fast reboot is now DISABLED by default.

- Non-interactive mode for ypinit(1M)

- NFS bugfixes (illumos 5436 & 5440)

- Several man page updates

- Hung mount in one zone no longer affects boot/halt of another (illumos 5419)

- Underlying infrastructure for global-zone rulesets for zones using ipfilter.
 (If this interests you, please let me know so an RFE can be tracked in the OmniOS ipkg brand)

- One mpt_sas hang-on-reset bug fixed (illumos 5297)

- Two ZFS bugfixes (illumos 5377 & 5369-5370).

.  .  .

We are suspending bloody updates during the month of January to run tests on a modernization of pkg(5).  I may be altering the bloody pkg repo during this time, but I would HIGHLY recommend that users who update to this version save a backup BE afterwards (using beadm(1m)) in case they accidentially take unannounced experimental updates during the month of January.

This modernization of pkg(5) has been bootstrapped by becoming a downstream child of the OpenIndiana Hipster work.  I am in contact with OI pkg(5) people, and of course everything we do will be open-sourced (testing it all is the hard part).  I've found bugs in Python thanks to this work, so I know it's being helpful.  We have the pkg(5) test suite passing 100% in API tests, and ~90% in CLI tests, where almost all of those failures are either to either assumptions about where apache22 lives, or by having the existence of python2.7 in /usr/lib/python2.7... which runs a bit counter to OmniOS's "Keep your sh*t to yourself" philosophy.

This pkg(5) work is difficult.  Once it's in an OmniOS bloody update, I hope to aid illumos community members in upstreaming pieces from other distros, as well as our own.  During January, I hope to define what will be in r151014, which is not only our next Stable release, but also our next Long-Term Support release.  This bloody period is critical to r151014's success, so if you have spare machines to run this bloody on, PLEASE do so, as you'll be helping the OmniOS community.

Thanks,
Dan McDonald -- OmniOS Engineering

From danmcd at omniti.com  Thu Dec 18 22:28:43 2014
From: danmcd at omniti.com (Dan McDonald)
Date: Thu, 18 Dec 2014 17:28:43 -0500
Subject: [OmniOS-discuss] UPDATE NOW:  git vulnerability
Message-ID: <8F42DE29-F036-4444-AF66-68855B63EE9D@omniti.com>

See here:

	https://github.com/blog/1938-vulnerability-announced-update-your-git-clients

All supported repos (006/LTS, 010/previous-stable, 012/current-stable, 013/bloody) are now updated with an appropriately patched version of git.

Dan


From kangcool2002 at yahoo.co.uk  Thu Dec 18 22:48:05 2014
From: kangcool2002 at yahoo.co.uk (a a)
Date: Thu, 18 Dec 2014 22:48:05 +0000 (UTC)
Subject: [OmniOS-discuss] Who is good at recovering zfs pools/files?
Message-ID: <922776323.500535.1418942885007.JavaMail.yahoo@jws11122.mail.ir2.yahoo.com>

Hi.
I am currently working for a London based charity who had bought an NAS appliance which is based on OmniOS.
Unfortunately its rolled over taking lots of video files before it could be backed up.
I am not sure if they have a support contract with Omni, and I have told them to talk to Omni, but they really need some of these video files back.
So who is good at this as its been a while since I last looked or had an issue with zfs?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141218/039f036d/attachment-0001.html>

From henson at acm.org  Fri Dec 19 00:53:28 2014
From: henson at acm.org (Paul B. Henson)
Date: Thu, 18 Dec 2014 16:53:28 -0800
Subject: [OmniOS-discuss] UPDATE NOW:  git vulnerability
In-Reply-To: <8F42DE29-F036-4444-AF66-68855B63EE9D@omniti.com>
References: <8F42DE29-F036-4444-AF66-68855B63EE9D@omniti.com>
Message-ID: <19e401d01b26$36899d80$a39cd880$@acm.org>

> From: Dan McDonald
> Sent: Thursday, December 18, 2014 2:29 PM
> 
> All supported repos (006/LTS, 010/previous-stable, 012/current-stable,
> 013/bloody) are now updated with an appropriately patched version of git.

This bug only impacts users on case insensitive filesystems, so unless
you're running it on a zfs filesystem with casesensitivity= insensitive,
you're not going to be vulnerable to this issue on an illumos system...

Not that you shouldn't update anyway as a general precaution (and a big
thanks to Dan for making it available so quickly), but if for some reason
updating would be problematic it's probably not a crisis for you.



From henson at acm.org  Fri Dec 19 03:05:24 2014
From: henson at acm.org (Paul B. Henson)
Date: Thu, 18 Dec 2014 19:05:24 -0800
Subject: [OmniOS-discuss] Last OmniOS bloody update for 2014 (long,
	please read)
In-Reply-To: <B2AB195E-D342-4061-8179-E43D2DBF3D36@omniti.com>
References: <B2AB195E-D342-4061-8179-E43D2DBF3D36@omniti.com>
Message-ID: <1a2d01d01b38$a3e114a0$eba33de0$@acm.org>

> From: Dan McDonald
> Sent: Thursday, December 18, 2014 11:12 AM
> 
> - Underlying infrastructure for global-zone rulesets for zones using
ipfilter.

Hmm, what exactly is this? Right now my zones are using exclusive stacks and
just running ipfilter in the zone...



From danmcd at omniti.com  Fri Dec 19 03:20:31 2014
From: danmcd at omniti.com (Dan McDonald)
Date: Thu, 18 Dec 2014 22:20:31 -0500
Subject: [OmniOS-discuss] Last OmniOS bloody update for 2014 (long,
	please read)
In-Reply-To: <1a2d01d01b38$a3e114a0$eba33de0$@acm.org>
References: <B2AB195E-D342-4061-8179-E43D2DBF3D36@omniti.com>
	<1a2d01d01b38$a3e114a0$eba33de0$@acm.org>
Message-ID: <EBD6F5A3-D0E6-427B-B104-CE4DF2A7FD2A@omniti.com>


> On Dec 18, 2014, at 10:05 PM, Paul B. Henson <henson at acm.org> wrote:
> 
>> From: Dan McDonald
>> Sent: Thursday, December 18, 2014 11:12 AM
>> 
>> - Underlying infrastructure for global-zone rulesets for zones using
> ipfilter.
> 
> Hmm, what exactly is this? Right now my zones are using exclusive stacks and
> just running ipfilter in the zone...

This came from Joyent.  Imagine if you, the GZ admin, want to further clamp down a zone you're renting or just transferring root for.  That's what this feature does.

It doesn't have set-at-boot time properties, because that needs to be added to your zone's brand code.  Brand code is still outside illumos-gate for illumos distros, including OmniOS.

Dan


From garrett at damore.org  Fri Dec 19 17:37:58 2014
From: garrett at damore.org (Garrett D'Amore)
Date: Fri, 19 Dec 2014 09:37:58 -0800
Subject: [OmniOS-discuss] [developer] Followon review --> Changes that
	allow illumos-gate to build on OmniOS
In-Reply-To: <72C4BB3D-7C98-406B-9254-8CE1D5AEE956@omniti.com>
References: <72C4BB3D-7C98-406B-9254-8CE1D5AEE956@omniti.com>
Message-ID: <6200A265-2B7C-4DF6-A64F-449A21DEE8F5@damore.org>

I?d like to see the shell style guide go entirely not just the html.

The docbook is hard to use, but more importantly the guide itself got little review, and I think it may refer to ksh93isms that I think we may wish to avoid in the future.  Shell style should only make use of POSIX shell compatible capabilities.

Its my hope someday to extract ksh93 from the gate in favor of a lighter weight alternative such as dash.  I have already done some work on this front, though its not ready for wider consumption yet.

	- Garrett

> On Dec 17, 2014, at 6:44 PM, Dan McDonald via illumos-developer <developer at lists.illumos.org> wrote:
> 
> Now that the java fixes are their own thing, let's revisit the remaining changes in:
> 
> 	http://kebe.com/~danmcd/webrevs/for-omnios/
> 
> All of these together, combined with 4719, will allow illumos-gate to build on OmniOS, AND allow an OmniOS box to ONU to the results, provided the .env file is properly populated.
> 
> The aforementioned directory contains:
> 
> check-rtime/
> 
> * Some check_rtime modifications for the version of things in OmniOS
> 
> ig-on-omnios.env
> 
> * A sample .env file used for building illumos-gate on OmniOS.  Look for the extra export variables you'll need.
> 
> java-updates
> 
> * Now points to illumos 4719, see previous mail.
> 
> lose-ipp-listener
> 
> * Lose the IPP listener if so specified in your .env file, which depends on bits not in OmniOS.
> * See the sample .env file above for how it works.
> 
> lose-shell-styleguide
> 
> * Lose the dependency on docbooks, only existing in the Shell Style Guide.
> * Apparently not only is this out in OmniOS already, but in SmartOS as well.
> 
> lose-sysidtool-dep
> 
> * Lose the SMF dependency on sysidtool.
> * I've been told this only ever affected people trying to move from old OpenSolaris to OI.
> 
> modern-netsmnp
> 
> * Allow linkage with more recent versions of net-snmp.
> 
> more-lintfixes
> 
> * Lint fixes that the OmniOS sunstudio12.1 lint are picky about.
> * Work done by Rich Lowe.  Thank you, Rich!
> 
> omnios-mail-msg.txt
> 
> * See the small number of remaining lint errors, ALL having to do with openssl versions.
> 
> perl-fixes
> 
> * More perl fixes to work with OmniOS's alternative approach to perl support.
> * This has also been tested, I believe, by the OpenIndiana Hipster people.
> 
> After 4719 goes back, I'd REALLY like to get these 7 other changesets back as well.  They'll be NOPs as far as OI is concerned, but they'll help out OmniOS immensely, and possibly other distros as well.
> 
> Thanks,
> Dan McDonald
> 
> -------------------------------------------
> illumos-developer
> Archives: https://www.listbox.com/member/archive/182179/=now
> RSS Feed: https://www.listbox.com/member/archive/rss/182179/21239177-3604570e
> Modify Your Subscription: https://www.listbox.com/member/?member_id=21239177&id_secret=21239177-2d0c9337
> Powered by Listbox: http://www.listbox.com


From garrett at damore.org  Fri Dec 19 17:34:07 2014
From: garrett at damore.org (Garrett D'Amore)
Date: Fri, 19 Dec 2014 09:34:07 -0800
Subject: [OmniOS-discuss] [developer] Followon review --> Changes that
	allow illumos-gate to build on OmniOS
In-Reply-To: <72C4BB3D-7C98-406B-9254-8CE1D5AEE956@omniti.com>
References: <72C4BB3D-7C98-406B-9254-8CE1D5AEE956@omniti.com>
Message-ID: <906F536C-0EAE-4D13-B315-1576888CF093@damore.org>

on the ipp listener cleanup, I think you can probably avoid a two step calculation of the package list in pkg/Makefile by declaring a variable automatically that is the inverse of ENABLE_IPP_PRINTING.

Somehting like this:

$(ENABLE_IPP_PIRNTING)NO_ENABLE_IPP_PRINTING=$(POUND_SIGN)

But really as I think about this, making more packages optional seems like a good thing.

So probably we should instead create a list of packages to suppress, append to it, and filter that way.  (Think scaling this to more than just the IPP printing problem.)

	- Garrett

> On Dec 17, 2014, at 6:44 PM, Dan McDonald via illumos-developer <developer at lists.illumos.org> wrote:
> 
> Now that the java fixes are their own thing, let's revisit the remaining changes in:
> 
> 	http://kebe.com/~danmcd/webrevs/for-omnios/
> 
> All of these together, combined with 4719, will allow illumos-gate to build on OmniOS, AND allow an OmniOS box to ONU to the results, provided the .env file is properly populated.
> 
> The aforementioned directory contains:
> 
> check-rtime/
> 
> * Some check_rtime modifications for the version of things in OmniOS
> 
> ig-on-omnios.env
> 
> * A sample .env file used for building illumos-gate on OmniOS.  Look for the extra export variables you'll need.
> 
> java-updates
> 
> * Now points to illumos 4719, see previous mail.
> 
> lose-ipp-listener
> 
> * Lose the IPP listener if so specified in your .env file, which depends on bits not in OmniOS.
> * See the sample .env file above for how it works.
> 
> lose-shell-styleguide
> 
> * Lose the dependency on docbooks, only existing in the Shell Style Guide.
> * Apparently not only is this out in OmniOS already, but in SmartOS as well.
> 
> lose-sysidtool-dep
> 
> * Lose the SMF dependency on sysidtool.
> * I've been told this only ever affected people trying to move from old OpenSolaris to OI.
> 
> modern-netsmnp
> 
> * Allow linkage with more recent versions of net-snmp.
> 
> more-lintfixes
> 
> * Lint fixes that the OmniOS sunstudio12.1 lint are picky about.
> * Work done by Rich Lowe.  Thank you, Rich!
> 
> omnios-mail-msg.txt
> 
> * See the small number of remaining lint errors, ALL having to do with openssl versions.
> 
> perl-fixes
> 
> * More perl fixes to work with OmniOS's alternative approach to perl support.
> * This has also been tested, I believe, by the OpenIndiana Hipster people.
> 
> After 4719 goes back, I'd REALLY like to get these 7 other changesets back as well.  They'll be NOPs as far as OI is concerned, but they'll help out OmniOS immensely, and possibly other distros as well.
> 
> Thanks,
> Dan McDonald
> 
> -------------------------------------------
> illumos-developer
> Archives: https://www.listbox.com/member/archive/182179/=now
> RSS Feed: https://www.listbox.com/member/archive/rss/182179/21239177-3604570e
> Modify Your Subscription: https://www.listbox.com/member/?member_id=21239177&id_secret=21239177-2d0c9337
> Powered by Listbox: http://www.listbox.com


From danmcd at omniti.com  Fri Dec 19 20:39:43 2014
From: danmcd at omniti.com (Dan McDonald)
Date: Fri, 19 Dec 2014 15:39:43 -0500
Subject: [OmniOS-discuss] UPDATE TIME (again)!
Message-ID: <225409DE-A25E-4901-A0C5-B6C9FF2D03F5@omniti.com>

The NTP server version is now 4.2.8, which I've gone ahead and placed on ALL OmniOS supported releases (006/LTS, 010/last-Stable, 012/Stable, 013/bloody).  Please utter "pkg update" at your earliest convenience.  This is not a requires-reboot update.

Relevant CVEs are: CVE-2014-9296, 9295, 9294, 9293

Thanks,
Dan


From danmcd at omniti.com  Fri Dec 19 20:52:35 2014
From: danmcd at omniti.com (Dan McDonald)
Date: Fri, 19 Dec 2014 15:52:35 -0500
Subject: [OmniOS-discuss] UPDATE TIME (again)!
In-Reply-To: <225409DE-A25E-4901-A0C5-B6C9FF2D03F5@omniti.com>
References: <225409DE-A25E-4901-A0C5-B6C9FF2D03F5@omniti.com>
Message-ID: <6E6F9316-ABE7-43C0-862E-F18C1C7C6D2E@omniti.com>


> On Dec 19, 2014, at 3:39 PM, Dan McDonald <danmcd at omniti.com> wrote:
> 
> The NTP server version is now 4.2.8, which I've gone ahead and placed on ALL OmniOS supported releases (006/LTS, 010/last-Stable, 012/Stable, 013/bloody).  Please utter "pkg update" at your earliest convenience.  This is not a requires-reboot update.
> 
> Relevant CVEs are: CVE-2014-9296, 9295, 9294, 9293

NOTE:  You will also need to "svcadm restart ntp" after this update.

Dan


From henson at acm.org  Sat Dec 20 02:46:31 2014
From: henson at acm.org (Paul B. Henson)
Date: Fri, 19 Dec 2014 18:46:31 -0800
Subject: [OmniOS-discuss] UPDATE TIME (again)!
In-Reply-To: <225409DE-A25E-4901-A0C5-B6C9FF2D03F5@omniti.com>
References: <225409DE-A25E-4901-A0C5-B6C9FF2D03F5@omniti.com>
Message-ID: <1e3201d01bff$2b8521a0$828f64e0$@acm.org>

> From: Dan McDonald
> Sent: Friday, December 19, 2014 12:40 PM
> 
> The NTP server version is now 4.2.8, which I've gone ahead and placed on
> ALL OmniOS supported releases (006/LTS, 010/last-Stable, 012/Stable,
> 013/bloody).  Please utter "pkg update" at your earliest convenience.
This is
> not a requires-reboot update.

The only thing I've used classic ntpd on for ages are boxes with actual
hardware time devices on them. It's got a history of being bloated with the
occasional security issue like this 8-/. I've been using openntpd for pretty
much everything else, but the portable version is unfortunately getting
pretty dated :(. I've been thinking of looking into Chrony
(http://chrony.tuxfamily.org/ ) as a possible simple/lightweight replacement
for openntpd. Has anybody tried that under omnios?



From danmcd at omniti.com  Sat Dec 20 02:51:06 2014
From: danmcd at omniti.com (Dan McDonald)
Date: Fri, 19 Dec 2014 21:51:06 -0500
Subject: [OmniOS-discuss] UPDATE TIME (again)!
In-Reply-To: <1e3201d01bff$2b8521a0$828f64e0$@acm.org>
References: <225409DE-A25E-4901-A0C5-B6C9FF2D03F5@omniti.com>
	<1e3201d01bff$2b8521a0$828f64e0$@acm.org>
Message-ID: <B841404D-8654-4484-80C9-61FE7FFA1595@omniti.com>


> On Dec 19, 2014, at 9:46 PM, Paul B. Henson <henson at acm.org> wrote:
> I've been thinking of looking into Chrony
> (http://chrony.tuxfamily.org/ ) as a possible simple/lightweight replacement
> for openntpd. Has anybody tried that under omnios?

I don't think so, but if you can get it to work, you should try and upstream your diffs.  I've done this for other open-source projects (ptp2d, e.g.), and it's been rather rewarding.  If you have problems getting it working, let people know here, or on the illumos developer's list.

Dan


From henson at acm.org  Sat Dec 20 02:55:02 2014
From: henson at acm.org (Paul B. Henson)
Date: Fri, 19 Dec 2014 18:55:02 -0800
Subject: [OmniOS-discuss] Last OmniOS bloody update for 2014 (long,
	please read)
In-Reply-To: <EBD6F5A3-D0E6-427B-B104-CE4DF2A7FD2A@omniti.com>
References: <B2AB195E-D342-4061-8179-E43D2DBF3D36@omniti.com>
	<1a2d01d01b38$a3e114a0$eba33de0$@acm.org>
	<EBD6F5A3-D0E6-427B-B104-CE4DF2A7FD2A@omniti.com>
Message-ID: <211001d01c00$5c74f190$155ed4b0$@acm.org>

> From: Dan McDonald
> Sent: Thursday, December 18, 2014 7:21 PM
> 
> This came from Joyent.  Imagine if you, the GZ admin, want to further
clamp
> down a zone you're renting or just transferring root for.  That's what
this
> feature does.

Ooh, cool. I can see how that would be helpful if you are deploying a zone
that will have an untrustworthy entity with privileges and you want to keep
them from abusing the network.


From danmcd at omniti.com  Sat Dec 20 03:01:10 2014
From: danmcd at omniti.com (Dan McDonald)
Date: Fri, 19 Dec 2014 22:01:10 -0500
Subject: [OmniOS-discuss] Last OmniOS bloody update for 2014 (long,
	please read)
In-Reply-To: <211001d01c00$5c74f190$155ed4b0$@acm.org>
References: <B2AB195E-D342-4061-8179-E43D2DBF3D36@omniti.com>
	<1a2d01d01b38$a3e114a0$eba33de0$@acm.org>
	<EBD6F5A3-D0E6-427B-B104-CE4DF2A7FD2A@omniti.com>
	<211001d01c00$5c74f190$155ed4b0$@acm.org>
Message-ID: <54E99D52-1987-42AA-BAB3-F476C4192A97@omniti.com>


> On Dec 19, 2014, at 9:55 PM, Paul B. Henson <henson at acm.org> wrote:
> , cool. I can see how that would be helpful if you are deploying a zone
> that will have an untrustworthy entity with privileges and you want to keep
> them from abusing the network.
> 

For it to be REALLY useful, some brand code will have to happen.  You need something to read /etc/ipf/zones/<blah> from the global zone, and only the brand code that handles "boot" methods can do it right.  I'm not doing that right now.

Dan


From johan.kragsterman at capvert.se  Sat Dec 20 11:47:47 2014
From: johan.kragsterman at capvert.se (Johan Kragsterman)
Date: Sat, 20 Dec 2014 12:47:47 +0100
Subject: [OmniOS-discuss] Ang: Re: Ang: Re:  CoreOS
In-Reply-To: <509071d00c17bdf9229b0be3eb8bfe8b@blackdot.be>
References: <509071d00c17bdf9229b0be3eb8bfe8b@blackdot.be>,
	<e69c6d0959ec12af17df866b720f5e14@blackdot.be>,
	<OF9FC50875.7A47B1B0-ONC1257DB2.00579EAF-C1257DB2.0057B4FE@inse.com>
	<OF79C7D6B0.90D89DC4-ONC1257DB2.005C37AA-C1257DB2.005D205C@inse.com>
Message-ID: <OF2DB776DD.80304056-ONC1257DB4.003F1ED2-C1257DB4.0040CCBD@inse.com>

Hi, Jorge and all!


I would be interested in discussing this further, but perhaps omnios-discuss isn't the right place? Since I don't know if this is omnios/illumos/coreos specific...

I did some experimenting:

I only used CoreOS stable in my tests.

I tried the iso, but the iso isn't full featured, and doesn't run docker out of the box. And the docker implementation is of coarse what everybody is interested in. I got it to boot without problems, but I had big problems with VNC keymapping due to my Swedish keyboard and perhaps my Swedish client computer. So I could actually never do something with it, and since it is not full featured, it is not what I want to use.

So instead, I downloaded the img file for qemu, created a volume, and dd'ed the image to the volume, and then set this volume as boot. That went fine, to get it to boot. But then, with the default boot option in grub, it panicked, and restarted every 60 seconds.

I stopped the grub booting, and chosed the B option. That didn't panic, but it didn't work either, it was too much that didn't work. But option A went fine, no panic, and everything seem to work more or less without problems. The only problem here seem to be that I can't log in, due to the "first log in"-principles they seem to have: It is only possible to log in via ssh, which means the network have to be up, and I couldn't get the network to come up....so there I am right now...

Regards Johan


-----Jorge Schrauwen <sjorge+ml at blackdot.be> skrev: -----
Till: Johan Kragsterman <johan.kragsterman at capvert.se>
Fr?n: Jorge Schrauwen <sjorge+ml at blackdot.be>
Datum: 2014-12-18 18:07
Kopia: omnios-discuss at lists.omniti.com
?rende: Re: Ang: Re: [OmniOS-discuss] CoreOS


On 2014-12-18 17:57, Johan Kragsterman wrote:
> Jorge, I was thinking about you when I posted this! I thought you
> would be a possible contributor to this thread... ?More furhter
> down...
> 
> 
> -----Jorge Schrauwen <sjorge+ml at blackdot.be> skrev: -----
> Till: Johan Kragsterman <johan.kragsterman at capvert.se>
> Fr?n: Jorge Schrauwen <sjorge+ml at blackdot.be>
> Datum: 2014-12-18 17:38
> Kopia: omnios-discuss at lists.omniti.com
> ?rende: Re: [OmniOS-discuss] CoreOS
> 
> Something like this will probably work:
> 
> 
> ??/usr/bin/qemu-system-x86_64
> ?? -name coreos \
> ?? -enable-kvm \
> ?? -no-hpet \
> ?? -m 4096
> ?? -cpu Nehalem \
> ?? -smp sockets=1,cores=4,threads=2 \
> ?? -rtc base=utc,driftfix=slew \
> ?? -pidfile /tank/coreo/coreos.pid ?\
> ?? -monitor unix:/tank/coreo/coreos.monitor,server,nowait,nodelay ?\
> ?? -vga std ?\
> ?? -vnc :1 ?\
> ?? -nographic \
> ?? -drive
> file=/tank/coreos/coreos.iso,if=ide,media=cdrom,index=0,cache=none \
> ?? -drive
> file=/dev/zvol/rdsk/tank/coreos/disk0,if=virtio,media=disk,index=0,cache=none,boot=on
> \
> ?? -boot order=cd,once=d \
> ?? -device
> virtio-net-pci,mac=02:08:20:0c:04:d2,tx=timer,x-txtimer=200000,x-txburst=128,vlan=0
> \
> ?? -net vnic,vlan=0,name=net1,ifname=vcoreos0 \
> ?? -chardev
> socket,id=serial0,path=/tank/coreos/coreos.console,server,nowait \
> ?? -serial chardev:serial0 \
> ?? -usb \
> ?? -usbdevice tablet \
> ?? -daemonize
> 
> You should get vnc at port 5901, seemed to boot for me but I did not
> complete the install.
> 
> 
> 
> At the CoreOS site they say: Start like this:
> 
> ./coreos_production_qemu.sh -nographic
> 
> and they pass on that string -nographic ?...?
> 
> It makes me wonder, because they tell you to connect with the instans
> only over ssh with: ssh -l core -p 2222 localhost ? ...
> 
> So I'm not sure if it is possible to connect via VNC...did you
> actually check VNC, to confirm you had a VNC connection?
> 
> It should boot and run from the image r/o, so perhaps you just need
> one "disk"? I can see you got two configured, or at least the iso
> file, and then a disk. Don't you think it would be enough with just
> the image file?
> 
> Perhaps I just try...
I used the install iso to see if it booted.
-nographic just mean don't spawn a graphical console AKA SDL or simular 
window. It does not prevent '-vnc :1' from working.

> 
> 
> 
> Regards
> 
> Jorge
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> On 2014-12-18 16:57, Johan Kragsterman wrote:
>> Hi!
>> 
>> 
>> ?I've been looking at CoreOS and finds it interesting! Since I'd like
>> to have OmniOS as the platform, I need to run CoreOS as a KVM guest.
>> Haven't tested yet, but I downloaded the startscript for qemu, and it
>> looks a little bit "too much" for Illumos KVM...
>> 
>> ?It would be nice to get some views on people that have been
>> considering this as well, perhaps some already tested or already
>> running...?
>> 
>> ?I've seen that Frederic Alix on this list been blogging about it, but
>> haven't seen if he managed to run it as a KVM guest on OmniOS.
>> 
>> ?For me it seems to be some complications at first startup, mainly. It
>> doesn't seem to be reachable by VNC...
>> 
>> ?Hope to get some input from you guys...
>> 
>> 
>> Best regards from/Med v?nliga h?lsningar fr?n
>> 
>> Johan Kragsterman
>> 
>> Capvert
>> 
>> _______________________________________________
>> OmniOS-discuss mailing list
>> OmniOS-discuss at lists.omniti.com
>> http://lists.omniti.com/mailman/listinfo/omnios-discuss




From sjorge+ml at blackdot.be  Sat Dec 20 13:36:50 2014
From: sjorge+ml at blackdot.be (Jorge Schrauwen)
Date: Sat, 20 Dec 2014 14:36:50 +0100
Subject: [OmniOS-discuss] Ang: Re: Ang: Re:  CoreOS
In-Reply-To: <OF2DB776DD.80304056-ONC1257DB4.003F1ED2-C1257DB4.0040CCBD@inse.com>
References: <509071d00c17bdf9229b0be3eb8bfe8b@blackdot.be>, 
	<e69c6d0959ec12af17df866b720f5e14@blackdot.be>,
	<OF9FC50875.7A47B1B0-ONC1257DB2.00579EAF-C1257DB2.0057B4FE@inse.com>
	<OF79C7D6B0.90D89DC4-ONC1257DB2.005C37AA-C1257DB2.005D205C@inse.com>
	<OF2DB776DD.80304056-ONC1257DB4.003F1ED2-C1257DB4.0040CCBD@inse.com>
Message-ID: <bbfb9966b67a5349dc56a8522bd0202c@blackdot.be>

Hey Johan,

I just poked at the qemu image... it seems it wants some stuff not in 
our old qemu-kvm fork. e.g. fsdev (mouting a filesystem from host to 
guest).

But let's try anyway!

# convert qcow2 to raw
qemu-img convert coreos_production_qemu_image.img 
coreos_production_qemu_image.dd
# dump this on our zvol
dd if=coreos_production_qemu_image.dd 
of=/dev/zvol/rdsk/core/vms/hosts/coreos/disk0

We now have the correctly formatted data on our zvol...

On the plus side it does output nicely to ttya if added to a vm :)

So... here is where the kernel dies: (oh it does some kexec bits which 
are a PITA)
---
[    0.001000] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 3.17.2 #2
[    0.001000] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2007
[    0.001000]  0000000000000008 ffff88007a3e7db8 ffffffff814e8915 
000000000000e
[    0.001000]  ffffffff81798190 ffff88007a3e7e38 ffffffff814e4c97 
0000000000006
[    0.001000]  0000000000000008 ffff88007a3e7e48 ffff88007a3e7de8 
00000000fffb0
[    0.001000] Call Trace:
[    0.001000]  [<ffffffff814e8915>] dump_stack+0x46/0x58
[    0.001000]  [<ffffffff814e4c97>] panic+0xc1/0x1f5
[    0.001000]  [<ffffffff81b10205>] setup_IO_APIC+0x7d6/0x83d
[    0.001000]  [<ffffffff81b0d0cb>] native_smp_prepare_cpus+0x2bc/0x337
[    0.001000]  [<ffffffff81afc1a1>] kernel_init_freeable+0xcd/0x212
[    0.001000]  [<ffffffff814daf30>] ? rest_init+0x80/0x80
[    0.001000]  [<ffffffff814daf3e>] kernel_init+0xe/0xf0
[    0.001000]  [<ffffffff814ef8fc>] ret_from_fork+0x7c/0xb0
[    0.001000]  [<ffffffff814daf30>] ? rest_init+0x80/0x80
[    0.001000] Rebooting in 60 seconds..
---

I actually also have this on a ubuntu vm I am using, it needs noapic 
kernel option... on the grub prompt (really nice is coreos seems to have 
grub + console on both tty0 (vga) and ttyS0 (serial ttya).

Woo hoo we got past that bit where it fails on IO-APIC, now we just hang 
on smpboot :(

---
[    0.001000] CPU: Physical Processor ID: 0
[    0.001000] CPU: Processor Core ID: 0
[    0.001000] mce: CPU supports 10 MCE banks
[    0.001000] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0
[    0.001000] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0
[    0.001000] Freeing SMP alternatives memory: 20K (ffffffff82fa1000 - 
fffffff)
[    0.001000] ftrace: allocating 19518 entries in 77 pages
[    0.001000] smpboot: CPU0: Intel QEMU Virtual CPU version 0.14.1 
(fam: 06, m
---

Pretty much stuck here... I tried some variations of cpu type (qemu64, 
Nehalem and host) I also tried using one vcpu but still stuck.

Let's just cripple the entire thing and plow are way through: adding 
'nosmp noapic noacpi'

So yeah at this point coreos is pretty useless... but we fly past 
smpboot!
And... land here:

---
[    0.239823] scsi host0: ata_piix
[    0.239823] scsi host1: ata_piix
[    0.239823] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc000 
irq 14
[    0.239823] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc008 
irq 15
---

If it is docker you want, you may as well look at SmartOS's LX Brand 
stuff, they are racing towards workable docker zones.

But since I came this far, lets see if I can make it to the finish...
I am using virtio... lets try scsi... nothing... ide... nothing...

So this is were it ends.

Our qemu-kvm fork is probably just too old.

Regards

Jorge



On 2014-12-20 12:47, Johan Kragsterman wrote:
> Hi, Jorge and all!
> 
> 
> I would be interested in discussing this further, but perhaps
> omnios-discuss isn't the right place? Since I don't know if this is
> omnios/illumos/coreos specific...
> 
> I did some experimenting:
> 
> I only used CoreOS stable in my tests.
> 
> I tried the iso, but the iso isn't full featured, and doesn't run
> docker out of the box. And the docker implementation is of coarse what
> everybody is interested in. I got it to boot without problems, but I
> had big problems with VNC keymapping due to my Swedish keyboard and
> perhaps my Swedish client computer. So I could actually never do
> something with it, and since it is not full featured, it is not what I
> want to use.
> 
> So instead, I downloaded the img file for qemu, created a volume, and
> dd'ed the image to the volume, and then set this volume as boot. That
> went fine, to get it to boot. But then, with the default boot option
> in grub, it panicked, and restarted every 60 seconds.
> 
> I stopped the grub booting, and chosed the B option. That didn't
> panic, but it didn't work either, it was too much that didn't work.
> But option A went fine, no panic, and everything seem to work more or
> less without problems. The only problem here seem to be that I can't
> log in, due to the "first log in"-principles they seem to have: It is
> only possible to log in via ssh, which means the network have to be
> up, and I couldn't get the network to come up....so there I am right
> now...
> 
> Regards Johan
> 
> 
> -----Jorge Schrauwen <sjorge+ml at blackdot.be> skrev: -----
> Till: Johan Kragsterman <johan.kragsterman at capvert.se>
> Fr?n: Jorge Schrauwen <sjorge+ml at blackdot.be>
> Datum: 2014-12-18 18:07
> Kopia: omnios-discuss at lists.omniti.com
> ?rende: Re: Ang: Re: [OmniOS-discuss] CoreOS
> 
> 
> On 2014-12-18 17:57, Johan Kragsterman wrote:
>> Jorge, I was thinking about you when I posted this! I thought you
>> would be a possible contributor to this thread... ?More furhter
>> down...
>> 
>> 
>> -----Jorge Schrauwen <sjorge+ml at blackdot.be> skrev: -----
>> Till: Johan Kragsterman <johan.kragsterman at capvert.se>
>> Fr?n: Jorge Schrauwen <sjorge+ml at blackdot.be>
>> Datum: 2014-12-18 17:38
>> Kopia: omnios-discuss at lists.omniti.com
>> ?rende: Re: [OmniOS-discuss] CoreOS
>> 
>> Something like this will probably work:
>> 
>> 
>> ??/usr/bin/qemu-system-x86_64
>> ?? -name coreos \
>> ?? -enable-kvm \
>> ?? -no-hpet \
>> ?? -m 4096
>> ?? -cpu Nehalem \
>> ?? -smp sockets=1,cores=4,threads=2 \
>> ?? -rtc base=utc,driftfix=slew \
>> ?? -pidfile /tank/coreo/coreos.pid ?\
>> ?? -monitor unix:/tank/coreo/coreos.monitor,server,nowait,nodelay ?\
>> ?? -vga std ?\
>> ?? -vnc :1 ?\
>> ?? -nographic \
>> ?? -drive
>> file=/tank/coreos/coreos.iso,if=ide,media=cdrom,index=0,cache=none \
>> ?? -drive
>> file=/dev/zvol/rdsk/tank/coreos/disk0,if=virtio,media=disk,index=0,cache=none,boot=on
>> \
>> ?? -boot order=cd,once=d \
>> ?? -device
>> virtio-net-pci,mac=02:08:20:0c:04:d2,tx=timer,x-txtimer=200000,x-txburst=128,vlan=0
>> \
>> ?? -net vnic,vlan=0,name=net1,ifname=vcoreos0 \
>> ?? -chardev
>> socket,id=serial0,path=/tank/coreos/coreos.console,server,nowait \
>> ?? -serial chardev:serial0 \
>> ?? -usb \
>> ?? -usbdevice tablet \
>> ?? -daemonize
>> 
>> You should get vnc at port 5901, seemed to boot for me but I did not
>> complete the install.
>> 
>> 
>> 
>> At the CoreOS site they say: Start like this:
>> 
>> ./coreos_production_qemu.sh -nographic
>> 
>> and they pass on that string -nographic ?...?
>> 
>> It makes me wonder, because they tell you to connect with the instans
>> only over ssh with: ssh -l core -p 2222 localhost ? ...
>> 
>> So I'm not sure if it is possible to connect via VNC...did you
>> actually check VNC, to confirm you had a VNC connection?
>> 
>> It should boot and run from the image r/o, so perhaps you just need
>> one "disk"? I can see you got two configured, or at least the iso
>> file, and then a disk. Don't you think it would be enough with just
>> the image file?
>> 
>> Perhaps I just try...
> I used the install iso to see if it booted.
> -nographic just mean don't spawn a graphical console AKA SDL or simular
> window. It does not prevent '-vnc :1' from working.
> 
>> 
>> 
>> 
>> Regards
>> 
>> Jorge
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> On 2014-12-18 16:57, Johan Kragsterman wrote:
>>> Hi!
>>> 
>>> 
>>> ?I've been looking at CoreOS and finds it interesting! Since I'd like
>>> to have OmniOS as the platform, I need to run CoreOS as a KVM guest.
>>> Haven't tested yet, but I downloaded the startscript for qemu, and it
>>> looks a little bit "too much" for Illumos KVM...
>>> 
>>> ?It would be nice to get some views on people that have been
>>> considering this as well, perhaps some already tested or already
>>> running...?
>>> 
>>> ?I've seen that Frederic Alix on this list been blogging about it, 
>>> but
>>> haven't seen if he managed to run it as a KVM guest on OmniOS.
>>> 
>>> ?For me it seems to be some complications at first startup, mainly. 
>>> It
>>> doesn't seem to be reachable by VNC...
>>> 
>>> ?Hope to get some input from you guys...
>>> 
>>> 
>>> Best regards from/Med v?nliga h?lsningar fr?n
>>> 
>>> Johan Kragsterman
>>> 
>>> Capvert
>>> 
>>> _______________________________________________
>>> OmniOS-discuss mailing list
>>> OmniOS-discuss at lists.omniti.com
>>> http://lists.omniti.com/mailman/listinfo/omnios-discuss

From rt at steait.net  Mon Dec 22 18:59:19 2014
From: rt at steait.net (Rune Tipsmark)
Date: Mon, 22 Dec 2014 18:59:19 +0000
Subject: [OmniOS-discuss] mount/create volume lu from snapshot
Message-ID: <1419274759953.32966@steait.net>

hi all,



I have two omnios boxes and zfs replication going between the two every 30 min.



I am replicating a volume lu pool01/vol01 from hostA to hostB



how can I mount this or create a volume lu out of it on my destination box?



br,

Rune
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141222/255706a2/attachment.html>

From danmcd at omniti.com  Mon Dec 22 19:07:06 2014
From: danmcd at omniti.com (Dan McDonald)
Date: Mon, 22 Dec 2014 14:07:06 -0500
Subject: [OmniOS-discuss] mount/create volume lu from snapshot
In-Reply-To: <1419274759953.32966@steait.net>
References: <1419274759953.32966@steait.net>
Message-ID: <4522F01B-DADE-423E-8526-0A28E809D9A3@omniti.com>


> On Dec 22, 2014, at 1:59 PM, Rune Tipsmark <rt at steait.net> wrote:
> 
> hi all,
>  
> I have two omnios boxes and zfs replication going between the two every 30 min.
>  
> I am replicating a volume lu pool01/vol01 from hostA to hostB
>  
> how can I mount this or create a volume lu out of it on my destination box?
>  

"stmfadm create-lu" does this for a given zvol.

Read the stmfadm(1M) man page for more details.

Dan


From rt at steait.net  Mon Dec 22 19:24:46 2014
From: rt at steait.net (Rune Tipsmark)
Date: Mon, 22 Dec 2014 19:24:46 +0000
Subject: [OmniOS-discuss] mount/create volume lu from snapshot
In-Reply-To: <4522F01B-DADE-423E-8526-0A28E809D9A3@omniti.com>
References: <1419274759953.32966@steait.net>,
	<4522F01B-DADE-423E-8526-0A28E809D9A3@omniti.com>
Message-ID: <1419276286622.95214@steait.net>

hi Dan,

tried that, get data file error.

I tried zfs list and found my snapshot

pool01/PHTVOL01 at 1419269859_repli_zfs_zfs20_nr_4

then stmfadm create-lu pool01/PHTVOL01 at 1419269859_repli_zfs_zfs20_nr_4

doesn't work...  also tried pool01/PHTVOL01, no luck either.

which file do I need to use?

br,
Rune

________________________________________
From: Dan McDonald <danmcd at omniti.com>
Sent: Monday, December 22, 2014 8:07 PM
To: Rune Tipsmark
Cc: omnios-discuss at lists.omniti.com
Subject: Re: [OmniOS-discuss] mount/create volume lu from snapshot

> On Dec 22, 2014, at 1:59 PM, Rune Tipsmark <rt at steait.net> wrote:
>
> hi all,
>
> I have two omnios boxes and zfs replication going between the two every 30 min.
>
> I am replicating a volume lu pool01/vol01 from hostA to hostB
>
> how can I mount this or create a volume lu out of it on my destination box?
>

"stmfadm create-lu" does this for a given zvol.

Read the stmfadm(1M) man page for more details.

Dan


From danmcd at omniti.com  Mon Dec 22 19:26:52 2014
From: danmcd at omniti.com (Dan McDonald)
Date: Mon, 22 Dec 2014 14:26:52 -0500
Subject: [OmniOS-discuss] mount/create volume lu from snapshot
In-Reply-To: <1419276286622.95214@steait.net>
References: <1419274759953.32966@steait.net> <,
	<4522F01B-DADE-423E-8526-0A28E809D9A3@omniti.com> <>>
	<1419276286622.95214@steait.net>
Message-ID: <6E1EAF64-BD33-48E3-A32E-37AE6DC1C57E@omniti.com>


> On Dec 22, 2014, at 2:24 PM, Rune Tipsmark <rt at steait.net> wrote:
> 
> hi Dan,
> 
> tried that, get data file error.
> 
> I tried zfs list and found my snapshot
> 
> pool01/PHTVOL01 at 1419269859_repli_zfs_zfs20_nr_4
> 
> then stmfadm create-lu pool01/PHTVOL01 at 1419269859_repli_zfs_zfs20_nr_4
> 
> doesn't work...  also tried pool01/PHTVOL01, no luck either.
> 
> which file do I need to use?

/dev/zvol/dsk/pool01/PHTVOL01

If you utter "stmf list-lu" on your original machine, you'll see pathnames.

Dan


From geoffn at gnaa.net  Mon Dec 22 20:41:00 2014
From: geoffn at gnaa.net (Geoff Nordli)
Date: Mon, 22 Dec 2014 12:41:00 -0800
Subject: [OmniOS-discuss] mount/create volume lu from snapshot
In-Reply-To: <1419276286622.95214@steait.net>
References: <1419274759953.32966@steait.net>,
	<4522F01B-DADE-423E-8526-0A28E809D9A3@omniti.com>
	<1419276286622.95214@steait.net>
Message-ID: <549881DC.9050905@gnaa.net>

On 14-12-22 11:24 AM, Rune Tipsmark wrote:
> hi Dan,
>
> tried that, get data file error.
>
> I tried zfs list and found my snapshot
>
> pool01/PHTVOL01 at 1419269859_repli_zfs_zfs20_nr_4
>
> then stmfadm create-lu pool01/PHTVOL01 at 1419269859_repli_zfs_zfs20_nr_4
>
> doesn't work...  also tried pool01/PHTVOL01, no luck either.
>
> which file do I need to use?
>
> br,
> Rune
>
> ________________________________________
> From: Dan McDonald <danmcd at omniti.com>
> Sent: Monday, December 22, 2014 8:07 PM
> To: Rune Tipsmark
> Cc: omnios-discuss at lists.omniti.com
> Subject: Re: [OmniOS-discuss] mount/create volume lu from snapshot
>
>> On Dec 22, 2014, at 1:59 PM, Rune Tipsmark <rt at steait.net> wrote:
>>
>> hi all,
>>
>> I have two omnios boxes and zfs replication going between the two every 30 min.
>>
>> I am replicating a volume lu pool01/vol01 from hostA to hostB
>>
>> how can I mount this or create a volume lu out of it on my destination box?
>>
> "stmfadm create-lu" does this for a given zvol.
>
> Read the stmfadm(1M) man page for more details.
>
> Dan
>
> ____________

Rune, you are going to need to create a zfs clone of the snapshot before 
you create the lun.

Geoff




From gate03 at landcroft.co.uk  Tue Dec 23 00:51:53 2014
From: gate03 at landcroft.co.uk (Michael Mounteney)
Date: Tue, 23 Dec 2014 10:51:53 +1000
Subject: [OmniOS-discuss] KVM within a child zone
Message-ID: <20141223105153.5d50749a@emeritus>

Has anyone done this?  The main obstacle in my own investigation is
that the zonecfg command "add device / set match=/dev/net/vnick0 ; end"
is ignored so I can't network the VM.

Michael.

From gate03 at landcroft.co.uk  Tue Dec 23 02:04:02 2014
From: gate03 at landcroft.co.uk (Michael Mounteney)
Date: Tue, 23 Dec 2014 12:04:02 +1000
Subject: [OmniOS-discuss] sudden loss of networking
Message-ID: <20141223120402.6714c763@emeritus>

Just now, r151012 lost networking.  I was able to get in via a serial
console (very relieved I took the trouble to set that up) but otherwise
the machine wasn't even pingable.  Thanks to the wonders of NFS, a
reboot preserved everything --- I wasn't running any VMs at the time.

svcs -a showed that network/physical:default was online.  I didn't
really know what else to do.  Any suggestions for next time ?  assuming
I can get in on the serial console.

Michael.

From danmcd at omniti.com  Tue Dec 23 02:27:00 2014
From: danmcd at omniti.com (Dan McDonald)
Date: Mon, 22 Dec 2014 21:27:00 -0500
Subject: [OmniOS-discuss] sudden loss of networking
In-Reply-To: <20141223120402.6714c763@emeritus>
References: <20141223120402.6714c763@emeritus>
Message-ID: <A6B226FF-FAAF-4F70-BADF-F0AD7DD66376@omniti.com>


> On Dec 22, 2014, at 9:04 PM, Michael Mounteney <gate03 at landcroft.co.uk> wrote:
> 
> Just now, r151012 lost networking.  I was able to get in via a serial
> console (very relieved I took the trouble to set that up) but otherwise
> the machine wasn't even pingable.  Thanks to the wonders of NFS, a
> reboot preserved everything --- I wasn't running any VMs at the time.
> 
> svcs -a showed that network/physical:default was online.  I didn't
> really know what else to do.  Any suggestions for next time ?  assuming
> I can get in on the serial console.

Lots of possibilities when you "lose networking".  Lots of information helps tell what happened.

Output of these commands:
	- ifconfig -a
	- dladm show-link
	- dladm show-phys
	- dladm show-ether
	- netstat -rnv

Also, make sure you didn't lose routing, as opposed to losing the link.  See if a node on the same subnet can ping it (you said it wasn't pingable, but where did you ping from?).

Someone recently here mentioned having two default routes installed because in.routed got fed bad information, for example.

Dan


From gate03 at landcroft.co.uk  Tue Dec 23 02:29:10 2014
From: gate03 at landcroft.co.uk (Michael Mounteney)
Date: Tue, 23 Dec 2014 12:29:10 +1000
Subject: [OmniOS-discuss] NTP server needs restarting to be usable
Message-ID: <20141223122910.7457532c@emeritus>

After the reboot referred-to in my earlier message, the NTP server
wasn't really working properly.  At first ntpclient on a Linux box was
reporting:

41994 07614.540  rejected packet: LI==3
41994 07629.556  rejected packet: LI==3
41994 07644.571  rejected packet: LI==3
41994 07659.586  rejected packet: LI==3

then after "svcadm disable ntp ; svcadm enable ntp" on the server:

41994 07860.947  rejected packet: abs(DISP)>65536
41994 07875.963  rejected packet: abs(DISP)>65536
41994 07890.978  rejected packet: abs(DISP)>65536
41994 07905.993  rejected packet: abs(DISP)>65536

then after "svcadm disable ntp ; sleep 30 ; svcadm enable ntp":

41994 07996.069  rejected packet: abs(DISP)>65536
41994 08011.084     397.0    122.7   -5266.1 215988.2  -3528976
41994 08026.099     334.0    113.8   -5872.0 216217.0  -3528976
41994 08041.115     345.0    122.2   -6198.3 216445.9  -3528976

so it took two restarts to get it going properly.  Any ideas
why ?  /etc/inet/ntp.conf is:

driftfile /var/ntp/ntp.drift
statsdir /var/ntp/ntpstats/
filegen peerstats file peerstats type day enable
filegen loopstats file loopstats type day enable
restrict default ignore # Deny everyone by default
restrict 127.0.0.1      # Allow localhost full access
restrict 192.168.1.0 mask 255.255.255.0 kod nomodify notrap nopeer
pool 0.pool.ntp.org iburst
pool 1.pool.ntp.org iburst
pool 2.pool.ntp.org iburst
pool 3.pool.ntp.org iburst
restrict 0.pool.ntp.org mask 255.255.255.255 nomodify notrap noquery
restrict 1.pool.ntp.org mask 255.255.255.255 nomodify notrap noquery
restrict 2.pool.ntp.org mask 255.255.255.255 nomodify notrap noquery
restrict 3.pool.ntp.org mask 255.255.255.255 nomodify notrap noquery

Michael.

From protonwrangler at gmail.com  Tue Dec 23 02:34:52 2014
From: protonwrangler at gmail.com (Warren Marts)
Date: Mon, 22 Dec 2014 19:34:52 -0700
Subject: [OmniOS-discuss] KVM within a child zone
In-Reply-To: <20141223105153.5d50749a@emeritus>
References: <20141223105153.5d50749a@emeritus>
Message-ID: <CAN8fiVvGCwA9eW9AT4YZ81xfoqwRedLyME+peTdxbZT_i+jorQ@mail.gmail.com>

KVM virtual machines on illumos each already run in a zone - with a
specific minimal brand, and qemu-kvm as the zone's primary process. So you
can apply global zone resources limits to arbitrate each vm.

On Mon, Dec 22, 2014 at 5:51 PM, Michael Mounteney <gate03 at landcroft.co.uk>
wrote:
>
> Has anyone done this?  The main obstacle in my own investigation is
> that the zonecfg command "add device / set match=/dev/net/vnick0 ; end"
> is ignored so I can't network the VM.
>
> Michael.
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141222/97b2f2ef/attachment.html>

From danmcd at omniti.com  Tue Dec 23 02:40:30 2014
From: danmcd at omniti.com (Dan McDonald)
Date: Mon, 22 Dec 2014 21:40:30 -0500
Subject: [OmniOS-discuss] NTP server needs restarting to be usable
In-Reply-To: <20141223122910.7457532c@emeritus>
References: <20141223122910.7457532c@emeritus>
Message-ID: <B9AF327D-6992-44E3-9201-3D4F0BA98724@omniti.com>


> On Dec 22, 2014, at 9:29 PM, Michael Mounteney <gate03 at landcroft.co.uk> wrote:
> 
> After the reboot referred-to in my earlier message, the NTP server
> wasn't really working properly.  At first ntpclient on a Linux box was
> reporting:

Did you "pkg update" to the very latest NTP server?  There's a security vulnerability attached to it, you know.

And you must "svcadm restart ntp" after the update.

Dan


From gate03 at landcroft.co.uk  Tue Dec 23 02:42:15 2014
From: gate03 at landcroft.co.uk (Michael Mounteney)
Date: Tue, 23 Dec 2014 12:42:15 +1000
Subject: [OmniOS-discuss] sudden loss of networking
In-Reply-To: <A6B226FF-FAAF-4F70-BADF-F0AD7DD66376@omniti.com>
References: <20141223120402.6714c763@emeritus>
	<A6B226FF-FAAF-4F70-BADF-F0AD7DD66376@omniti.com>
Message-ID: <20141223124215.534116b3@emeritus>

On Mon, 22 Dec 2014 21:27:00 -0500
Dan McDonald <danmcd at omniti.com> wrote:

> Lots of possibilities when you "lose networking".  Lots of
> information helps tell what happened.
> 
> Output of these commands:
> 	- ifconfig -a
> 	- dladm show-link
> 	- dladm show-phys
> 	- dladm show-ether
> 	- netstat -rnv

Thanks Dan;  I've saved those commands in a script which hopefully I'll
be able to run if it happens again, via the serial port.

> Also, make sure you didn't lose routing, as opposed to losing the
> link.  See if a node on the same subnet can ping it (you said it
> wasn't pingable, but where did you ping from?).

I was pinging from a machine on the same subnet;  it's just a simple
home setup here with two interfaces:  one connected to the ISP's cable
modem and the other to a hub/router and all the other machines, on the
192.168.1.0/24 subnet.

> Someone recently here mentioned having two default routes installed
> because in.routed got fed bad information, for example.

Maybe the diagnostics will reveal this to be the cause.

Michael.

From gate03 at landcroft.co.uk  Tue Dec 23 02:46:16 2014
From: gate03 at landcroft.co.uk (Michael Mounteney)
Date: Tue, 23 Dec 2014 12:46:16 +1000
Subject: [OmniOS-discuss] NTP server needs restarting to be usable
In-Reply-To: <B9AF327D-6992-44E3-9201-3D4F0BA98724@omniti.com>
References: <20141223122910.7457532c@emeritus>
	<B9AF327D-6992-44E3-9201-3D4F0BA98724@omniti.com>
Message-ID: <20141223124616.6298d7b9@emeritus>

On Mon, 22 Dec 2014 21:40:30 -0500
Dan McDonald <danmcd at omniti.com> wrote:

> Did you "pkg update" to the very latest NTP server?  There's a
> security vulnerability attached to it, you know.

Oh yes.

root at world:/root# pkg list ntp
NAME (PUBLISHER)                                  VERSION                    IFO
service/network/ntp                               4.2.8-0.151012             i--

> And you must "svcadm restart ntp" after the update.

This only happened after the machine rebooted, as referred-to in my
earlier message.

Maybe something to do with the server coming up before networking is
ready?  Just a thought.

Michael.

From bdha at mirrorshades.net  Tue Dec 23 02:50:58 2014
From: bdha at mirrorshades.net (Bryan Horstmann-Allen)
Date: Mon, 22 Dec 2014 21:50:58 -0500
Subject: [OmniOS-discuss] KVM within a child zone
In-Reply-To: <CAN8fiVvGCwA9eW9AT4YZ81xfoqwRedLyME+peTdxbZT_i+jorQ@mail.gmail.com>
References: <20141223105153.5d50749a@emeritus>
	<CAN8fiVvGCwA9eW9AT4YZ81xfoqwRedLyME+peTdxbZT_i+jorQ@mail.gmail.com>
Message-ID: <20141223025058.GB98666@lab.int.icgroup.com>

+------------------------------------------------------------------------------
| On 2014-12-22 19:34:52, Warren Marts wrote:
| 
| KVM virtual machines on illumos each already run in a zone - with a
| specific minimal brand, and qemu-kvm as the zone's primary process. So you
| can apply global zone resources limits to arbitrate each vm.

This is only true on SmartOS, not OmniOS.
-- 
bdha

From jesus at omniti.com  Tue Dec 23 03:07:57 2014
From: jesus at omniti.com (Theo Schlossnagle)
Date: Mon, 22 Dec 2014 22:07:57 -0500
Subject: [OmniOS-discuss] KVM within a child zone
In-Reply-To: <CAN8fiVvGCwA9eW9AT4YZ81xfoqwRedLyME+peTdxbZT_i+jorQ@mail.gmail.com>
References: <20141223105153.5d50749a@emeritus>
	<CAN8fiVvGCwA9eW9AT4YZ81xfoqwRedLyME+peTdxbZT_i+jorQ@mail.gmail.com>
Message-ID: <CACLsApsVx0S1qkb+YCf4WD5YxMMnHty0DHLWEXuoMntdAWrt+Q@mail.gmail.com>

This doesn't happen unless you explicit do that on "illumos."  You are
describing the default behavior on SmartOS, qemu-kvm is just a process on
illumos, you'd have to setup the minimal zone, etc.

On Mon, Dec 22, 2014 at 9:34 PM, Warren Marts <protonwrangler at gmail.com>
wrote:
>
> KVM virtual machines on illumos each already run in a zone - with a
> specific minimal brand, and qemu-kvm as the zone's primary process. So you
> can apply global zone resources limits to arbitrate each vm.
>
> On Mon, Dec 22, 2014 at 5:51 PM, Michael Mounteney <gate03 at landcroft.co.uk
> > wrote:
>>
>> Has anyone done this?  The main obstacle in my own investigation is
>> that the zonecfg command "add device / set match=/dev/net/vnick0 ; end"
>> is ignored so I can't network the VM.
>>
>> Michael.
>> _______________________________________________
>> OmniOS-discuss mailing list
>> OmniOS-discuss at lists.omniti.com
>> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>>
>
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>
>

-- 

Theo Schlossnagle

http://omniti.com/is/theo-schlossnagle
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141222/b2598eff/attachment.html>

From takashiary at gmail.com  Tue Dec 23 03:50:24 2014
From: takashiary at gmail.com (takashi ary)
Date: Tue, 23 Dec 2014 12:50:24 +0900
Subject: [OmniOS-discuss] NTP server needs restarting to be usable
In-Reply-To: <20141223124616.6298d7b9@emeritus>
References: <20141223122910.7457532c@emeritus>
	<B9AF327D-6992-44E3-9201-3D4F0BA98724@omniti.com>
	<20141223124616.6298d7b9@emeritus>
Message-ID: <CANXaD7i=JJTvqNBHnbmOTE6cUK2PNadARj_vJUygN=3euB=Hgw@mail.gmail.com>

Hello,

please try using the following setup
------------------------------------------------------------
svccfg -s svc:/network/ntp addpg name-services dependency
svccfg -s svc:/network/ntp setprop name-services/entities = fmri:
svc:/milestone/name-services:default
svccfg -s svc:/network/ntp setprop name-services/grouping = astring:
optional_all
svccfg -s svc:/network/ntp setprop name-services/restart_on = astring: none
svccfg -s svc:/network/ntp setprop name-services/type= astring: service

svccfg -s svc:/network/ntp addpg routing-setup dependency
svccfg -s svc:/network/ntp setprop routing-setup/entities = fmri:
svc:/network/routing-setup:default
svccfg -s svc:/network/ntp setprop routing-setup/grouping = astring:
optional_all
svccfg -s svc:/network/ntp setprop routing-setup/restart_on = astring: none
svccfg -s svc:/network/ntp setprop routing-setup/type= astring: service
------------------------------------------------------------

Thanks

From gate03 at landcroft.co.uk  Tue Dec 23 05:16:28 2014
From: gate03 at landcroft.co.uk (Michael Mounteney)
Date: Tue, 23 Dec 2014 15:16:28 +1000
Subject: [OmniOS-discuss] NTP server needs restarting to be usable
In-Reply-To: <CANXaD7i=JJTvqNBHnbmOTE6cUK2PNadARj_vJUygN=3euB=Hgw@mail.gmail.com>
References: <20141223122910.7457532c@emeritus>
	<B9AF327D-6992-44E3-9201-3D4F0BA98724@omniti.com>
	<20141223124616.6298d7b9@emeritus>
	<CANXaD7i=JJTvqNBHnbmOTE6cUK2PNadARj_vJUygN=3euB=Hgw@mail.gmail.com>
Message-ID: <20141223151628.597a12b0@emeritus>

On Tue, 23 Dec 2014 12:50:24 +0900
takashi ary <takashiary at gmail.com> wrote:

> Hello,
> 
> please try using the following setup
> [...]

Thank you Takashi:  that does appear to fix the problem, although the
skew hovers around 5 ms.

Michael.

From sjorge+ml at blackdot.be  Tue Dec 23 08:01:16 2014
From: sjorge+ml at blackdot.be (Jorge Schrauwen)
Date: Tue, 23 Dec 2014 09:01:16 +0100
Subject: [OmniOS-discuss] KVM within a child zone
In-Reply-To: <20141223105153.5d50749a@emeritus>
References: <20141223105153.5d50749a@emeritus>
Message-ID: <93bb2197e9cbb0d20a5968402954dd05@blackdot.be>

IIRC setting the zone to exclusive netstack works to include the 
/dev/net/vnicX device:
---
set ip-type=exclusive
add net
set physical=zleonov0
end
---
-(~)-[.]-{ ls -l /dev/net                                      
}-(root at leonov)-
total 0
crw-rw-rw-   1 root     sys      263, 1010 Dec 20 18:59 zleonov0
---

Regards

Jorge


On 2014-12-23 01:51, Michael Mounteney wrote:
> Has anyone done this?  The main obstacle in my own investigation is
> that the zonecfg command "add device / set match=/dev/net/vnick0 ; end"
> is ignored so I can't network the VM.
> 
> Michael.
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss

From gate03 at landcroft.co.uk  Tue Dec 23 09:28:47 2014
From: gate03 at landcroft.co.uk (Michael Mounteney)
Date: Tue, 23 Dec 2014 19:28:47 +1000
Subject: [OmniOS-discuss] KVM within a child zone
In-Reply-To: <93bb2197e9cbb0d20a5968402954dd05@blackdot.be>
References: <20141223105153.5d50749a@emeritus>
	<93bb2197e9cbb0d20a5968402954dd05@blackdot.be>
Message-ID: <20141223192847.74648386@emeritus>

On Tue, 23 Dec 2014 09:01:16 +0100
Jorge Schrauwen <sjorge+ml at blackdot.be> wrote:

> IIRC setting the zone to exclusive netstack works to include the 
> /dev/net/vnicX device:

That gets it a bit closer but I still need the real interface (e1000g1)
for vnc and sshing in etc. and I can't give that to the zone
exclusively.

Michael.

From sjorge+ml at blackdot.be  Tue Dec 23 09:42:19 2014
From: sjorge+ml at blackdot.be (Jorge Schrauwen)
Date: Tue, 23 Dec 2014 10:42:19 +0100
Subject: [OmniOS-discuss] KVM within a child zone
In-Reply-To: <20141223192847.74648386@emeritus>
References: <20141223105153.5d50749a@emeritus>
	<93bb2197e9cbb0d20a5968402954dd05@blackdot.be>
	<20141223192847.74648386@emeritus>
Message-ID: <a516f45c7f5aa191cb5440d942604d26@blackdot.be>

Well just use a socket for vnc, then you can spawn socat in the gz to 
explose the socket over tcp :)

This deals with a serial port as a socket...
https://blackdot.be/2013/07/qemu-kvm-monitor-and-serial-console-over-sockets-with-minicom/
But:

socat unix-listen:/vms/leonov/run/vnc.sock tcp4:0.0.0.0:5901 &

works just as well :)

For qemu the option is -vnc unix:/run/vnc.sock (assuming /vms/leonov is 
your zone root)
I tend do drop vnc, serial0 and monitoring interface as unix sockets for 
all my qemu processes. Then I use a small wrapper I wrote (kvmcon - 
https://github.com/sjorge/omnios-build-blackdot/blob/master/build/system-kvmcon/files/kvmcon) 
to connect to them.

I use this in combination with kvmadm, works reasonably well. This 
should not be to hard to adapt to a qemu process running in a zone.

Regards

Jorge


On 2014-12-23 10:28, Michael Mounteney wrote:
> On Tue, 23 Dec 2014 09:01:16 +0100
> Jorge Schrauwen <sjorge+ml at blackdot.be> wrote:
> 
>> IIRC setting the zone to exclusive netstack works to include the
>> /dev/net/vnicX device:
> 
> That gets it a bit closer but I still need the real interface (e1000g1)
> for vnc and sshing in etc. and I can't give that to the zone
> exclusively.
> 
> Michael.
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss

From lotheac at iki.fi  Tue Dec 23 09:44:01 2014
From: lotheac at iki.fi (Lauri Tirkkonen)
Date: Tue, 23 Dec 2014 11:44:01 +0200
Subject: [OmniOS-discuss] KVM within a child zone
In-Reply-To: <20141223192847.74648386@emeritus>
References: <20141223105153.5d50749a@emeritus>
	<93bb2197e9cbb0d20a5968402954dd05@blackdot.be>
	<20141223192847.74648386@emeritus>
Message-ID: <20141223094401.GA18650@gutsman.lotheac.fi>

On Tue, Dec 23 2014 19:28:47 +1000, Michael Mounteney wrote:
> That gets it a bit closer but I still need the real interface (e1000g1)
> for vnc and sshing in etc. and I can't give that to the zone
> exclusively.

You can create a vnic over your physical interface (with dladm(1M)) and
then give that to the zone.

-- 
Lauri Tirkkonen | +358 50 5341376 | lotheac @ IRCnet

From natxo.asenjo at gmail.com  Tue Dec 23 09:56:30 2014
From: natxo.asenjo at gmail.com (Natxo Asenjo)
Date: Tue, 23 Dec 2014 10:56:30 +0100
Subject: [OmniOS-discuss] KVM within a child zone
In-Reply-To: <20141223094401.GA18650@gutsman.lotheac.fi>
References: <20141223105153.5d50749a@emeritus>
	<93bb2197e9cbb0d20a5968402954dd05@blackdot.be>
	<20141223192847.74648386@emeritus>
	<20141223094401.GA18650@gutsman.lotheac.fi>
Message-ID: <CAHBEJzU40nPE3G5Fzuhw0f52D8BUhe2unEDRpYajzEFnmsHxug@mail.gmail.com>

On Tue, Dec 23, 2014 at 10:44 AM, Lauri Tirkkonen <lotheac at iki.fi> wrote:

> On Tue, Dec 23 2014 19:28:47 +1000, Michael Mounteney wrote:
> > That gets it a bit closer but I still need the real interface (e1000g1)
> > for vnc and sshing in etc. and I can't give that to the zone
> > exclusively.
>
> You can create a vnic over your physical interface (with dladm(1M)) and
> then give that to the zone.
>

my thoghts exactly. This is what I do in such cases:

First let's see the real network hardware:

# dladm show-phys
LINK         MEDIA                STATE      SPEED  DUPLEX    DEVICE
bge0         Ethernet             up         1000   full      bge0

then create a vnic on top of that

# dladm create-vnic -l bge0 vnic1

check it's there:

# dladm show-vnic
LINK         OVER         SPEED  MACADDRESS        MACADDRTYPE         VID
vnic1        bge0         1000   2:8:20:ae:f6:c0   random              0

create a zone and assign it the new vnic:

global # zonecfg -z sapacc
sapacc: No such zone configured
Use 'create' to begin configuring a new zone.
zonecfg:sapacc> create
zonecfg:sapacc> set zonepath=/export/zones/sapacc
zonecfg:sapacc> set autoboot=false
zonecfg:sapacc> set ip-type=exclusive
zonecfg:sapacc> add net
zonecfg:sapacc:net> set physical=vnic1
zonecfg:sapacc:net> end
zonecfg:sapacc> verify
zonecfg:sapacc> commit
zonecfg:sapacc> exit


And then configure the vnic inside the zone like you would normally
--
Groeten,
natxo
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141223/355ebfd1/attachment.html>

From gate03 at landcroft.co.uk  Tue Dec 23 10:48:15 2014
From: gate03 at landcroft.co.uk (Michael Mounteney)
Date: Tue, 23 Dec 2014 20:48:15 +1000
Subject: [OmniOS-discuss] KVM within a child zone
In-Reply-To: <CAHBEJzU40nPE3G5Fzuhw0f52D8BUhe2unEDRpYajzEFnmsHxug@mail.gmail.com>
References: <20141223105153.5d50749a@emeritus>
	<93bb2197e9cbb0d20a5968402954dd05@blackdot.be>
	<20141223192847.74648386@emeritus>
	<20141223094401.GA18650@gutsman.lotheac.fi>
	<CAHBEJzU40nPE3G5Fzuhw0f52D8BUhe2unEDRpYajzEFnmsHxug@mail.gmail.com>
Message-ID: <20141223204815.63e06058@emeritus>

On Tue, 23 Dec 2014 10:56:30 +0100
Natxo Asenjo <natxo.asenjo at gmail.com> wrote:

Thanks to you and everyone else for their help.  I hope I'm not
polluting the list with newbie noise but hopefully this will stay on
record and help out someone else later.  My knowledge of vnics, sockets
etc. is thin, to say the least.

It is now working in a fashion.  Here's the KVM zone:

root at world:/root# zonecfg -z KVM export
create -b
set zonepath=/zone/KVM
set brand=ipkg
set autoboot=true
set ip-type=exclusive
add net
set physical=vnick0
end
add device
set match=/dev/kvm
end
add device
set match=/dev/dld
end
add device
set match=/dev/zvol/dsk/rpool/vol/*
end
add dataset
set name=rpool/vol
end

I don't need that dataset but anyway.

Within the zone, the kvm launch command is:

/usr/bin/qemu-system-x86_64 \
        -name "Gentoo XFCE" \
        -cpu host \
        -boot order=d \
        -enable-kvm \
        -chardev socket,id=monitor,path=/var/run/monitor.sock,server,nowait \
	-monitor chardev:monitor \
	-chardev socket,id=serial0,path=/var/run/console.sock,server,nowait \
	-serial chardev:serial0 \
	-vnc unix:/var/run/XFCE.sock \
        -smp 1,maxcpus=1,cores=2 \
        -m 1024 \
        -no-hpet \
        -localtime \
        -kernel ./bzImage \
        -append "root=/dev/vda1 init=/usr/lib/systemd/systemd quiet" \
        -drive file=/dev/zvol/dsk/rpool/vol/Gentoo-KVM-XFCE,cache=none,if=virtio,index=0 \
	-drive file=/dev/zvol/dsk/rpool/vol/Linuswap-XFCE,cache=none,if=virtio,index=1 \
	-net nic,vlan=0,macaddr=$mac,model=virtio,name=ncard1 \
	-net vnic,vlan=0,name=net0,ifname=5905,macaddr=$mac \
	-vga std \
	-daemonize

and that works inasmuch as the guest assigns an IP of 192.168.1.47 to
its interface, and I can ssh and xdmcp into it from elsewhere.

Now the three sockets are visible in the global zone:

root at world:/root# ls -ltr /zone/KVM/root/var/run/*.sock 
srwxr-xr-x 1 root root 0 Dec 23 20:04 /zone/KVM/root/var/run/monitor.sock
srwxr-xr-x 1 root root 0 Dec 23 20:04 /zone/KVM/root/var/run/console.sock
srwxr-x--- 1 root root 0 Dec 23 20:04 /zone/KVM/root/var/run/XFCE.sock

so how would I connect a VNC viewer on another machine on the same
subnet to the VM ?  I need this for when the guest fails to start up for whatever reason.

Michael.

From vab at bb-c.de  Tue Dec 23 15:20:36 2014
From: vab at bb-c.de (Volker A. Brandt)
Date: Tue, 23 Dec 2014 16:20:36 +0100
Subject: [OmniOS-discuss] NTP server needs restarting to be usable
In-Reply-To: <B9AF327D-6992-44E3-9201-3D4F0BA98724@omniti.com>
References: <20141223122910.7457532c@emeritus>
	<B9AF327D-6992-44E3-9201-3D4F0BA98724@omniti.com>
Message-ID: <21657.34884.272541.710083@glaurung.bb-c.de>

Hi Dan!


Thanks for all of your efforts.  OmniOS is a solid part of our
IT infrastucture here.

You wrote (to Michael Mounteney):
> Did you "pkg update" to the very latest NTP server?  There's a
> security vulnerability attached to it, you know.

As an aside, I just updated from 10 to 12 and am now running:

# cat /etc/release
  OmniOS v11 r151012
  Copyright 2014 OmniTI Computer Consulting, Inc. All rights reserved.
  Use is subject to license terms.
# uname -a
SunOS nfs2 5.11 omnios-10b9c79 i86pc i386 i86pc

Update was straightforward and worked just fine.  But I noticed a strange
path in the /usr/share/doc/ntp4 subdirectory:

  # pkg list -Hv ntp at latest
  pkg://omnios/service/network/ntp at 4.2.8,5.11-0.151012:20141219T194032Z        i--

  # pkg contents -H pkg://omnios/service/network/ntp at 4.2.8,5.11-0.151012:20141219T194032Z
  etc/inet
  etc/inet/ntp.client
  [...]
  usr/share/doc/ntp4/ntpsweep.html
  usr/share/doc/ntp4/ntptrace.html
  usr/share/doc/ntp4/{}
  usr/share/doc/ntp4/{}/9400n.jpg
  usr/share/doc/ntp4/{}/a-ux
  usr/share/doc/ntp4/{}/access.html
  usr/share/doc/ntp4/{}/accopt.html
  usr/share/doc/ntp4/{}/accopt.txt
  usr/share/doc/ntp4/{}/aix
  usr/share/doc/ntp4/{}/alice11.gif
  usr/share/doc/ntp4/{}/alice13.gif
  usr/share/doc/ntp4/{}/alice15.gif
  usr/share/doc/ntp4/{}/alice23.gif
  usr/share/doc/ntp4/{}/alice31.gif
  usr/share/doc/ntp4/{}/alice32.gif
  usr/share/doc/ntp4/{}/alice35.gif
  usr/share/doc/ntp4/{}/alice38.gif
  usr/share/doc/ntp4/{}/alice44.gif
  usr/share/doc/ntp4/{}/alice47.gif
  usr/share/doc/ntp4/{}/alice51.gif
  usr/share/doc/ntp4/{}/alice61.gif
  usr/share/doc/ntp4/{}/assoc.html
  usr/share/doc/ntp4/{}/audio.html
  usr/share/doc/ntp4/{}/audio.txt
  [...]

I guess these braces come from a failed variable expansion... :-)

> And you must "svcadm restart ntp" after the update.

Couldn't this be automated by tacking a restart_fmri attribute on to
some file that is guaranteed to be different, such as the ntpd binary?
(It needs to be different from the existing binary for the 
 restart_fmri to fire.)

E.g.

  file 8b55f1839681e3b537177922f31821a19d5f5ba4 path=usr/lib/inet/ntpd \
    owner=root group=bin mode=0555 \
    chash=4da9e4ef17575df37441057da515a15e57ab4938 elfarch=i386 elfbits=32 \
    elfhash=991e353f618ea696ebe95403e70519d71cb3d0b2 pkg.csize=403734 \
    pkg.size=1013996 restart_fmri=svc:/network/ntp:default


Regards -- Volker
-- 
------------------------------------------------------------------------
Volker A. Brandt               Consulting and Support for Oracle Solaris
Brandt & Brandt Computer GmbH                   WWW: http://www.bb-c.de/
Am Wiesenpfad 6, 53340 Meckenheim, GERMANY            Email: vab at bb-c.de
Handelsregister: Amtsgericht Bonn, HRB 10513              Schuhgr??e: 46
Gesch?ftsf?hrer: Rainer J.H. Brandt und Volker A. Brandt

"When logic and proportion have fallen sloppy dead"

From danmcd at omniti.com  Tue Dec 23 15:40:47 2014
From: danmcd at omniti.com (Dan McDonald)
Date: Tue, 23 Dec 2014 10:40:47 -0500
Subject: [OmniOS-discuss] NTP server needs restarting to be usable
In-Reply-To: <21657.34884.272541.710083@glaurung.bb-c.de>
References: <20141223122910.7457532c@emeritus>
	<B9AF327D-6992-44E3-9201-3D4F0BA98724@omniti.com>
	<21657.34884.272541.710083@glaurung.bb-c.de>
Message-ID: <ADE707D5-59C1-4457-9896-7824D5DCA9D0@omniti.com>


> On Dec 23, 2014, at 10:20 AM, Volker A. Brandt <vab at bb-c.de> wrote:
> Update was straightforward and worked just fine.  But I noticed a strange
> path in the /usr/share/doc/ntp4 subdirectory:
> 
>  # pkg list -Hv ntp at latest
>  pkg://omnios/service/network/ntp at 4.2.8,5.11-0.151012:20141219T194032Z        i--
> 
>  # pkg contents -H pkg://omnios/service/network/ntp at 4.2.8,5.11-0.151012:20141219T194032Z
>  etc/inet
>  etc/inet/ntp.client
>  [...]
>  usr/share/doc/ntp4/ntpsweep.html
>  usr/share/doc/ntp4/ntptrace.html
>  usr/share/doc/ntp4/{}
>  usr/share/doc/ntp4/{}/9400n.jpg
>  usr/share/doc/ntp4/{}/a-ux
>  usr/share/doc/ntp4/{}/access.html
>  usr/share/doc/ntp4/{}/accopt.html
>  usr/share/doc/ntp4/{}/accopt.txt
>  usr/share/doc/ntp4/{}/aix
>  usr/share/doc/ntp4/{}/alice11.gif
>  usr/share/doc/ntp4/{}/alice13.gif
>  usr/share/doc/ntp4/{}/alice15.gif
>  usr/share/doc/ntp4/{}/alice23.gif
>  usr/share/doc/ntp4/{}/alice31.gif
>  usr/share/doc/ntp4/{}/alice32.gif
>  usr/share/doc/ntp4/{}/alice35.gif
>  usr/share/doc/ntp4/{}/alice38.gif
>  usr/share/doc/ntp4/{}/alice44.gif
>  usr/share/doc/ntp4/{}/alice47.gif
>  usr/share/doc/ntp4/{}/alice51.gif
>  usr/share/doc/ntp4/{}/alice61.gif
>  usr/share/doc/ntp4/{}/assoc.html
>  usr/share/doc/ntp4/{}/audio.html
>  usr/share/doc/ntp4/{}/audio.txt
>  [...]
> 
> I guess these braces come from a failed variable expansion... :-)

It's possible they came in with the 4.2.7 update (which was introduced in 012).

>> And you must "svcadm restart ntp" after the update.
> 
> Couldn't this be automated by tacking a restart_fmri attribute on to
> some file that is guaranteed to be different, such as the ntpd binary?
> (It needs to be different from the existing binary for the 
> restart_fmri to fire.)
> 
> E.g.
> 
>  file 8b55f1839681e3b537177922f31821a19d5f5ba4 path=usr/lib/inet/ntpd \
>    owner=root group=bin mode=0555 \
>    chash=4da9e4ef17575df37441057da515a15e57ab4938 elfarch=i386 elfbits=32 \
>    elfhash=991e353f618ea696ebe95403e70519d71cb3d0b2 pkg.csize=403734 \
>    pkg.size=1013996 restart_fmri=svc:/network/ntp:default

If that change happens, it should probably be placed somewhere in a .mog file in the ntp build directory under omnios-build.

Dan


From gate03 at landcroft.co.uk  Tue Dec 23 20:14:46 2014
From: gate03 at landcroft.co.uk (Michael Mounteney)
Date: Wed, 24 Dec 2014 06:14:46 +1000
Subject: [OmniOS-discuss] KVM within a child zone
In-Reply-To: <20141223204815.63e06058@emeritus>
References: <20141223105153.5d50749a@emeritus>
	<93bb2197e9cbb0d20a5968402954dd05@blackdot.be>
	<20141223192847.74648386@emeritus>
	<20141223094401.GA18650@gutsman.lotheac.fi>
	<CAHBEJzU40nPE3G5Fzuhw0f52D8BUhe2unEDRpYajzEFnmsHxug@mail.gmail.com>
	<20141223204815.63e06058@emeritus>
Message-ID: <20141224061446.078bf13d@emeritus>

Brilliant, with a bit more fiddling I've got it so that the KVM
instances can be started within the child zone, AND shut down cleanly
via the command "echo system_powerdown | nc -U /var/run/KDE.monsock" so
the next step is to put them into a service so that they can be brought
up and down within SMF.

Thanks again to all for their help.

Michael.

From hasslerd at gmx.li  Tue Dec 23 23:00:03 2014
From: hasslerd at gmx.li (Dominik Hassler)
Date: Wed, 24 Dec 2014 00:00:03 +0100
Subject: [OmniOS-discuss] KVM within a child zone
In-Reply-To: <20141224061446.078bf13d@emeritus>
References: <20141223105153.5d50749a@emeritus>	<93bb2197e9cbb0d20a5968402954dd05@blackdot.be>	<20141223192847.74648386@emeritus>	<20141223094401.GA18650@gutsman.lotheac.fi>	<CAHBEJzU40nPE3G5Fzuhw0f52D8BUhe2unEDRpYajzEFnmsHxug@mail.gmail.com>	<20141223204815.63e06058@emeritus>
	<20141224061446.078bf13d@emeritus>
Message-ID: <5499F3F3.7010408@gmx.li>

Michael,

you might wanna give kvmadm a try ( https://github.com/hadfl/kvmadm ).

that'll help you putting your kvms under smf control.

at least you should be able to extract all the infos from there...

On 12/23/2014 09:14 PM, Michael Mounteney wrote:
> Brilliant, with a bit more fiddling I've got it so that the KVM
> instances can be started within the child zone, AND shut down cleanly
> via the command "echo system_powerdown | nc -U /var/run/KDE.monsock" so
> the next step is to put them into a service so that they can be brought
> up and down within SMF.
> 
> Thanks again to all for their help.
> 
> Michael.
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
> 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0xF9ECC6A5.asc
Type: application/pgp-keys
Size: 2686 bytes
Desc: not available
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141224/0f092e1e/attachment.bin>

From gate03 at landcroft.co.uk  Tue Dec 23 23:56:22 2014
From: gate03 at landcroft.co.uk (Michael Mounteney)
Date: Wed, 24 Dec 2014 09:56:22 +1000
Subject: [OmniOS-discuss] KVM within a child zone
In-Reply-To: <5499F3F3.7010408@gmx.li>
References: <20141223105153.5d50749a@emeritus>
	<93bb2197e9cbb0d20a5968402954dd05@blackdot.be>
	<20141223192847.74648386@emeritus>
	<20141223094401.GA18650@gutsman.lotheac.fi>
	<CAHBEJzU40nPE3G5Fzuhw0f52D8BUhe2unEDRpYajzEFnmsHxug@mail.gmail.com>
	<20141223204815.63e06058@emeritus>
	<20141224061446.078bf13d@emeritus> <5499F3F3.7010408@gmx.li>
Message-ID: <20141224095622.3867fe34@emeritus>

On Wed, 24 Dec 2014 00:00:03 +0100
Dominik Hassler <hasslerd at gmx.li> wrote:

> Michael,
> 
> you might wanna give kvmadm a try ( https://github.com/hadfl/kvmadm ).
> 
> that'll help you putting your kvms under smf control.
> 
> at least you should be able to extract all the infos from there...

Thanks Dominic, yes, very similar to Jorges suggestion.  I have to work
out how to do instances with SMF, because I have two VMs, which differ
in (1) into which image they boot and (2) the port number of the VNC
connection.  It'll be very nice to have everything under proper control.

Michael.

From sjorge+ml at blackdot.be  Wed Dec 24 08:42:39 2014
From: sjorge+ml at blackdot.be (Jorge Schrauwen)
Date: Wed, 24 Dec 2014 09:42:39 +0100
Subject: [OmniOS-discuss] KVM within a child zone
In-Reply-To: <20141224095622.3867fe34@emeritus>
References: <20141223105153.5d50749a@emeritus>
	<93bb2197e9cbb0d20a5968402954dd05@blackdot.be>
	<20141223192847.74648386@emeritus>
	<20141223094401.GA18650@gutsman.lotheac.fi>
	<CAHBEJzU40nPE3G5Fzuhw0f52D8BUhe2unEDRpYajzEFnmsHxug@mail.gmail.com>
	<20141223204815.63e06058@emeritus> <20141224061446.078bf13d@emeritus>
	<5499F3F3.7010408@gmx.li> <20141224095622.3867fe34@emeritus>
Message-ID: <032d07135554c37dbcebe78b144e9da1@blackdot.be>

Well I actually suggested Dominik's kvmadm so they are identical not 
similar :D

kvmadm can certainly do that, lots of features have been added over the 
last few months and It's pretty solid now. Dominik also has an amazing 
response time if you open a feature request.

Regards

Jorge


On 2014-12-24 00:56, Michael Mounteney wrote:
> On Wed, 24 Dec 2014 00:00:03 +0100
> Dominik Hassler <hasslerd at gmx.li> wrote:
> 
>> Michael,
>> 
>> you might wanna give kvmadm a try ( https://github.com/hadfl/kvmadm ).
>> 
>> that'll help you putting your kvms under smf control.
>> 
>> at least you should be able to extract all the infos from there...
> 
> Thanks Dominic, yes, very similar to Jorges suggestion.  I have to work
> out how to do instances with SMF, because I have two VMs, which differ
> in (1) into which image they boot and (2) the port number of the VNC
> connection.  It'll be very nice to have everything under proper 
> control.
> 
> Michael.
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss

From jimklimov at cos.ru  Thu Dec 25 10:18:01 2014
From: jimklimov at cos.ru (Jim Klimov)
Date: Thu, 25 Dec 2014 11:18:01 +0100
Subject: [OmniOS-discuss] KVM within a child zone
In-Reply-To: <20141223105153.5d50749a@emeritus>
References: <20141223105153.5d50749a@emeritus>
Message-ID: <ED8DBC1F-B424-4ED5-BC3C-0F3569837755@cos.ru>

23 ??????? 2014??. 1:51:53 CET, Michael Mounteney <gate03 at landcroft.co.uk> ?????:
>Has anyone done this?  The main obstacle in my own investigation is
>that the zonecfg command "add device / set match=/dev/net/vnick0 ; end"
>is ignored so I can't network the VM.
>
>Michael.
>_______________________________________________
>OmniOS-discuss mailing list
>OmniOS-discuss at lists.omniti.com
>http://lists.omniti.com/mailman/listinfo/omnios-discuss

Are ypu sure tuere is no typo?
i.e. vnick0 vs. vnic0 ?
--
Typos courtesy of K-9 Mail on my Samsung Android

From gate03 at landcroft.co.uk  Thu Dec 25 10:26:39 2014
From: gate03 at landcroft.co.uk (Michael Mounteney)
Date: Thu, 25 Dec 2014 20:26:39 +1000
Subject: [OmniOS-discuss] KVM within a child zone
In-Reply-To: <ED8DBC1F-B424-4ED5-BC3C-0F3569837755@cos.ru>
References: <20141223105153.5d50749a@emeritus>
	<ED8DBC1F-B424-4ED5-BC3C-0F3569837755@cos.ru>
Message-ID: <20141225202639.48716756@emeritus>

On Thu, 25 Dec 2014 11:18:01 +0100
Jim Klimov <jimklimov at cos.ru> wrote:

> Are ypu sure tuere is no typo?
> i.e. vnick0 vs. vnic0 ?

Quite sure, Jim.  My choice of name, whimsical as it might be.

Michael.

From sergei25 at gmail.com  Fri Dec 26 22:36:25 2014
From: sergei25 at gmail.com (sergei)
Date: Fri, 26 Dec 2014 14:36:25 -0800
Subject: [OmniOS-discuss] Sneak modified scsi_vhci.conf in under installer ?
Message-ID: <CABZ-ChSBSf4gkbzUW7wXumSq87g4ERycz0wu9QQnnx2vqwtpVQ@mail.gmail.com>

Hi

The disks I want to install OmniOS to are TOSHIBA AL13SEB300 model which
scsi_vhci won't take over without proper conf file listing this model under
"scsi-vhci-failover-override" line. Right now those disk device path starts
with /pci instead of /scsi_vhci. Yet they are showing up in format output
ok. What is the trick to fix this without rebuilding install ISO image ?

I could install into one of (larger) Seagates and then mirror/remove mirror
to move boot OS to the proper disks. Is there any easier way ?

I think it would benefit Omni if you could keep scsi_vhci with at least
some updates. I see Nexenta does include bunch of models into it's default
scsi_vhci.conf.

S
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141226/4d61589c/attachment.html>

From richard.elling at richardelling.com  Sat Dec 27 18:54:01 2014
From: richard.elling at richardelling.com (Richard Elling)
Date: Sat, 27 Dec 2014 10:54:01 -0800
Subject: [OmniOS-discuss] Sneak modified scsi_vhci.conf in under
	installer ?
In-Reply-To: <CABZ-ChSBSf4gkbzUW7wXumSq87g4ERycz0wu9QQnnx2vqwtpVQ@mail.gmail.com>
References: <CABZ-ChSBSf4gkbzUW7wXumSq87g4ERycz0wu9QQnnx2vqwtpVQ@mail.gmail.com>
Message-ID: <2C40A4F8-BCBB-4B4C-BAC6-390595CA89BD@richardelling.com>


> On Dec 26, 2014, at 2:36 PM, sergei <sergei25 at gmail.com> wrote:
> 
> Hi
> 
> The disks I want to install OmniOS to are TOSHIBA AL13SEB300 model which scsi_vhci won't take over without proper conf file listing this model under "scsi-vhci-failover-override" line. Right now those disk device path starts with /pci instead of /scsi_vhci. Yet they are showing up in format output ok. What is the trick to fix this without rebuilding install ISO image ?

easy -- don't rebuild the install ISO image :-)

> 
> I could install into one of (larger) Seagates and then mirror/remove mirror to move boot OS to the proper disks. Is there any easier way ?

ugh, too much work.
Try this (I'm sure I blogged this a few times, or maybe in the Nexenta knowledge base?)
I'm sure the procedure is in the email archives...

1. go ahead and do the installation.
2. boot into newly installed OS
3. edit scsi_vhci.conf
4. shutdown
5. boot from install media, go to shell
6. import rpool
7. export rpool
8. reboot into newly installed OS

ZFS is tolerant of path changes, but you have to trick the boot process.

> 
> I think it would benefit Omni if you could keep scsi_vhci with at least some updates. I see Nexenta does include bunch of models into it's default scsi_vhci.conf.

The root cause is a deficiency in detecting multiple ports. The workaround is to override
in scsi_vhci.conf. The fix is known, just need to find the time...
 -- richard


From sergei25 at gmail.com  Sat Dec 27 19:26:48 2014
From: sergei25 at gmail.com (sergei)
Date: Sat, 27 Dec 2014 11:26:48 -0800
Subject: [OmniOS-discuss] Sneak modified scsi_vhci.conf in under
	installer ?
In-Reply-To: <2C40A4F8-BCBB-4B4C-BAC6-390595CA89BD@richardelling.com>
References: <CABZ-ChSBSf4gkbzUW7wXumSq87g4ERycz0wu9QQnnx2vqwtpVQ@mail.gmail.com>
	<2C40A4F8-BCBB-4B4C-BAC6-390595CA89BD@richardelling.com>
Message-ID: <CABZ-ChSFgBhUWR5PbWpz0+Gim=HiQYJjXd0OAwH4FQUjZ3j3Rg@mail.gmail.com>

Richard

The thing is - installer does not show those two disks in the list. So I
can't install into the disks that are supposed to be boot disks. There only
larger Seagate drives offered as install target. My guess was that
installed filters disks by their device path, not letting /pci ones through
? That is why I wanted to have proper scsi_vhci before the installer. Was
hoping this can be done with mdb.

Re: scsi_vhci.conf - systems with 3+ years uptime now are very picky about
disk replacements (model/vendor). Almost have to feed them Seagate
exclusively. If I told anyone that this server won't take TOSHIBA SAS drive
but will be more happy with Seagate - I doubt many people would take it
seriously. And these days I see lots of TOSHIBA disks that come as
replacements. Almost makes one wish for a *some* tool to add entries to
vhci and make them active at runtime.


On Sat, Dec 27, 2014 at 10:54 AM, Richard Elling <
richard.elling at richardelling.com> wrote:

>
> > On Dec 26, 2014, at 2:36 PM, sergei <sergei25 at gmail.com> wrote:
> >
> > Hi
> >
> > The disks I want to install OmniOS to are TOSHIBA AL13SEB300 model which
> scsi_vhci won't take over without proper conf file listing this model under
> "scsi-vhci-failover-override" line. Right now those disk device path starts
> with /pci instead of /scsi_vhci. Yet they are showing up in format output
> ok. What is the trick to fix this without rebuilding install ISO image ?
>
> easy -- don't rebuild the install ISO image :-)
>
> >
> > I could install into one of (larger) Seagates and then mirror/remove
> mirror to move boot OS to the proper disks. Is there any easier way ?
>
> ugh, too much work.
> Try this (I'm sure I blogged this a few times, or maybe in the Nexenta
> knowledge base?)
> I'm sure the procedure is in the email archives...
>
> 1. go ahead and do the installation.
> 2. boot into newly installed OS
> 3. edit scsi_vhci.conf
> 4. shutdown
> 5. boot from install media, go to shell
> 6. import rpool
> 7. export rpool
> 8. reboot into newly installed OS
>
> ZFS is tolerant of path changes, but you have to trick the boot process.
>
> >
> > I think it would benefit Omni if you could keep scsi_vhci with at least
> some updates. I see Nexenta does include bunch of models into it's default
> scsi_vhci.conf.
>
> The root cause is a deficiency in detecting multiple ports. The workaround
> is to override
> in scsi_vhci.conf. The fix is known, just need to find the time...
>  -- richard
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141227/25e317a3/attachment.html>

From gate03 at landcroft.co.uk  Mon Dec 29 06:40:29 2014
From: gate03 at landcroft.co.uk (Michael Mounteney)
Date: Mon, 29 Dec 2014 16:40:29 +1000
Subject: [OmniOS-discuss] Postgres on Stable
In-Reply-To: <CALLZ=vV6u2H1WC6D+a_a9qiUSpOs0jBxwq=YjvRVnCDVowNLdA@mail.gmail.com>
References: <20141128191821.43382469@punda-mlia>
	<CACLsApuL0vfHhn7FAQw8OU+uPd=j=qEGRUVWAtCtA1VXoo+TVA@mail.gmail.com>
	<CA+FJ0zsFukbMSdnTRxqdO1fU_zR69_=bH4UDJkh3mdB+pmtABQ@mail.gmail.com>
	<CALLZ=vXs6XDheAGuE2aDJigNaHti4bVe7npAc2qiiqK45zh0sQ@mail.gmail.com>
	<CALLZ=vXQ=4qn=4SWAtAnBksf=dSVXkzFRrfWffe3HyoLB9b+1g@mail.gmail.com>
	<20141129071835.1daa0799@punda-mlia>
	<CALLZ=vU4MNQ2cZQgTHguzq4Ok-UywT7HTr5c6Pm05g4QyR8=7Q@mail.gmail.com>
	<CALLZ=vV6u2H1WC6D+a_a9qiUSpOs0jBxwq=YjvRVnCDVowNLdA@mail.gmail.com>
Message-ID: <20141229164029.0b4c386c@emeritus>

On Fri, 28 Nov 2014 18:37:09 -0500
Zach Malone <zmalone at omniti.com> wrote:

> I built postgresql-935 for you on a r151012 system, and published it
> to the omniti-ms repo.  Want to give it a shot?  Past installing it, I
> have not tried it on a production system (I'm planning on moving some
> systems to r151012 this month), and I have not rolled any of the other
> postgres libraries or modules that we publish for 9.2.

Zac, can I be really cheeky and ask for this for bloody, r151013 ?
I've just switched over to that distro, and there are no versions of
Postgres available from the standard repositories.

Much thanks in anticipation.

Michael.

From chip at innovates.com  Mon Dec 29 12:52:04 2014
From: chip at innovates.com (Schweiss, Chip)
Date: Mon, 29 Dec 2014 06:52:04 -0600
Subject: [OmniOS-discuss] Postgres on Stable
In-Reply-To: <20141229164029.0b4c386c@emeritus>
References: <20141128191821.43382469@punda-mlia>
	<CACLsApuL0vfHhn7FAQw8OU+uPd=j=qEGRUVWAtCtA1VXoo+TVA@mail.gmail.com>
	<CA+FJ0zsFukbMSdnTRxqdO1fU_zR69_=bH4UDJkh3mdB+pmtABQ@mail.gmail.com>
	<CALLZ=vXs6XDheAGuE2aDJigNaHti4bVe7npAc2qiiqK45zh0sQ@mail.gmail.com>
	<CALLZ=vXQ=4qn=4SWAtAnBksf=dSVXkzFRrfWffe3HyoLB9b+1g@mail.gmail.com>
	<20141129071835.1daa0799@punda-mlia>
	<CALLZ=vU4MNQ2cZQgTHguzq4Ok-UywT7HTr5c6Pm05g4QyR8=7Q@mail.gmail.com>
	<CALLZ=vV6u2H1WC6D+a_a9qiUSpOs0jBxwq=YjvRVnCDVowNLdA@mail.gmail.com>
	<20141229164029.0b4c386c@emeritus>
Message-ID: <CALeZrrTLa2QFFZqJMs15koYjTh1AB4_QT=Qq_NJAmE85+Oa6iQ@mail.gmail.com>

I use OpenCSW for Postgres and many other packages on OmniOS.

While targeted at Solaris, OpenCSW packages have worked flawlessly for me
on OmniOS.

http://www.opencsw.org/

-Chip

On Mon, Dec 29, 2014 at 12:40 AM, Michael Mounteney <gate03 at landcroft.co.uk>
wrote:

> On Fri, 28 Nov 2014 18:37:09 -0500
> Zach Malone <zmalone at omniti.com> wrote:
>
> > I built postgresql-935 for you on a r151012 system, and published it
> > to the omniti-ms repo.  Want to give it a shot?  Past installing it, I
> > have not tried it on a production system (I'm planning on moving some
> > systems to r151012 this month), and I have not rolled any of the other
> > postgres libraries or modules that we publish for 9.2.
>
> Zac, can I be really cheeky and ask for this for bloody, r151013 ?
> I've just switched over to that distro, and there are no versions of
> Postgres available from the standard repositories.
>
> Much thanks in anticipation.
>
> Michael.
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141229/8e3fdcfb/attachment.html>

From rt at steait.net  Mon Dec 29 21:59:22 2014
From: rt at steait.net (Rune Tipsmark)
Date: Mon, 29 Dec 2014 21:59:22 +0000
Subject: [OmniOS-discuss] zfs destroy takes forever,
	how do I set async_destroy?
Message-ID: <1419890360886.59904@steait.net>

hi all,

as stated above, I have a server where I am syncing some 70 or so TB from and it has some very large snapshots and a destroy takes forever... has run for hours and hours now, 100% disk busy...I read there was a feature flag async_destroy but I don't seem to be able to find it.



Any ideas? the box is more or less useless and I have more rather large volumes I need to sync to a remote device.



br,

Rune
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141229/fe0b0f59/attachment.html>

From rt at steait.net  Wed Dec 31 03:21:44 2014
From: rt at steait.net (Rune Tipsmark)
Date: Wed, 31 Dec 2014 03:21:44 +0000
Subject: [OmniOS-discuss] zfs destroy takes forever,
 how do I set async_destroy?
In-Reply-To: <1419890360886.59904@steait.net>
References: <1419890360886.59904@steait.net>
Message-ID: <1419996103098.13209@steait.net>

I found out the feature is already enabled, I guess destroying very large snapshots just takes a very long time regardless...

br,

Rune

________________________________
From: OmniOS-discuss <omnios-discuss-bounces at lists.omniti.com> on behalf of Rune Tipsmark <rt at steait.net>
Sent: Monday, December 29, 2014 10:59 PM
To: omnios-discuss
Subject: [OmniOS-discuss] zfs destroy takes forever, how do I set async_destroy?


hi all,

as stated above, I have a server where I am syncing some 70 or so TB from and it has some very large snapshots and a destroy takes forever... has run for hours and hours now, 100% disk busy...I read there was a feature flag async_destroy but I don't seem to be able to find it.



Any ideas? the box is more or less useless and I have more rather large volumes I need to sync to a remote device.



br,

Rune
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141231/d9767166/attachment.html>

From Kevin.Swab at ColoState.EDU  Wed Dec 31 19:25:00 2014
From: Kevin.Swab at ColoState.EDU (Kevin Swab)
Date: Wed, 31 Dec 2014 12:25:00 -0700
Subject: [OmniOS-discuss] slow drive response times
Message-ID: <54A44D8C.5090302@ColoState.EDU>

Hello Everyone,

	We've been running OmniOS on a number of SuperMicro 36bay chassis, with
Supermicro motherboards, LSI SAS controllers (9211-8i & 9207-8i) and
various SAS HDD's.  These systems are serving block storage via Comstar
and Qlogic FC HBA's, and have been running well for several years.

	The problem we've got is that as the drives age, some of them start to
perform slowly (intermittently) without failing - no zpool or iostat
errors, and nothing logged in /var/adm/messages.  The slow performance
can be seen as high average service times in iostat or sar.

	When these service times get above 500ms, they start to cause IO
timeouts on the downstream storage consumers, which is bad...

	I'm wondering - is there a way to tune OmniOS' behavior so that it
doesn't try so hard to complete IOs to these slow disks, and instead
just gives up and fails them?

	I found an old post from 2011 which states that some tunables exist,
but are ignored by the mpt_sas driver:

http://everycity.co.uk/alasdair/2011/05/adjusting-drive-timeouts-with-mdb-on-solaris-or-openindiana/

Does anyone know the current status of these tunables, or have any other
suggestions that might help?

Thanks,
Kevin


-- 
-------------------------------------------------------------------
Kevin Swab                          UNIX Systems Administrator
ACNS                                Colorado State University
Phone: (970)491-6572                Email: Kevin.Swab at ColoState.EDU
GPG Fingerprint: 7026 3F66 A970 67BD 6F17  8EB8 8A7D 142F 2392 791C

From groups at tierarzt-mueller.de  Wed Dec 31 20:40:48 2014
From: groups at tierarzt-mueller.de (Alexander Lesle)
Date: Wed, 31 Dec 2014 21:40:48 +0100
Subject: [OmniOS-discuss] Kernel panic - I cant find the problem
Message-ID: <1531165597.20141231214048@tierarzt-mueller.de>

Hello All,

today it was Update-time for my servers. After typing reboot the
system hang on reboot. Ok, looking by iKVM what happend and there I
see on the screen that the server was restarted after a kernel panic.

I cant solve the problem because I dont understand where the problem
is.
Please help.

,-----[  ]-----
|
| root at aio:/root# fmdump -Vp -u da148174-fd91-4572-b639-b4fef8253f70
| TIME                           UUID                                
| SUNW-MSG-ID
| 
| Dec 16 2014 15:30:39.077759000 da148174-fd91-4572-b639-b4fef8253f70
| SUNOS-8000-KL
| 
| 
| nvlist version: 0
|         version = 0x0
|         class = list.suspect
|         uuid = da148174-fd91-4572-b639-b4fef8253f70
|         code = SUNOS-8000-KL
|         diag-time = 1418740239 75809
|         de = fmd:///module/software-diagnosis
|         fault-list-sz = 0x1
|         fault-list = (array of embedded nvlists)
|         (start fault-list[0])
|         nvlist version: 0
|                 version = 0x0
|                 class = defect.sunos.kernel.panic
|                 certainty = 0x64
|                 asru =
| sw:///:path=/var/crash/unknown/.da148174-fd91-4572-b639-b4fef8253f70
| 
|                 resource =
| sw:///:path=/var/crash/unknown/.da148174-fd91-4572-b639-b4fef8253f70
| 
|                 savecore-succcess = 1
|                 dump-dir = /var/crash/unknown
|                 dump-files = vmdump.2
|                 os-instance-uuid =
| da148174-fd91-4572-b639-b4fef8253f70
| 
|                 panicstr = BAD TRAP: type=e (#pf Page fault)
| rp=ffffff00103bd790 addr=fffffffff7afe4f0
| 
|                 panicstack = unix:real_mode_stop_cpu_stage2_end+9e23
| () | unix:trap+db3 () | unix:cmntrap+e6 () | cmlb:cmlb_validate+0 () |
| sd:sdopen+35f () | genunix:dev_open+30 () | specfs:spec_open+206 () |
| genunix:fop_open+89 () | genunix:vn_openat+234 () | genunix:copen+20c
| () | genunix:openat32+27 () | genunix:open32+25 () |
| unix:brand_sys_sysenter+1c9 () |
| 
|                 crashtime = 1418740182
|                 panic-time = Tue Dec 16 15:29:42 2014 CET
|         (end fault-list[0])
| 
|         fault-status = 0x1
|         severity = Major
|         __ttl = 0x1
|         __tod = 0x5490420f 0x4a28218
| 
| root at aio:/root#
|
`-------------------

Attachments / Dateianlagen:
----------------------------
[1] messages.1


Happy new year to you all.
-- 
Best Regards
Alexander
Dezember, 31 2014
-------------- next part --------------
A non-text attachment was scrubbed...
Name: messages.1
Type: application/octet-stream
Size: 40334 bytes
Desc: not available
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20141231/b0da8d98/attachment-0001.obj>

From richard.elling at richardelling.com  Wed Dec 31 22:22:13 2014
From: richard.elling at richardelling.com (Richard Elling)
Date: Wed, 31 Dec 2014 14:22:13 -0800
Subject: [OmniOS-discuss] slow drive response times
In-Reply-To: <54A44D8C.5090302@ColoState.EDU>
References: <54A44D8C.5090302@ColoState.EDU>
Message-ID: <DEAD2247-D2F4-4D3F-8C56-3B821DE958F5@richardelling.com>


> On Dec 31, 2014, at 11:25 AM, Kevin Swab <Kevin.Swab at colostate.edu> wrote:
> 
> Hello Everyone,
> 
> 	We've been running OmniOS on a number of SuperMicro 36bay chassis, with
> Supermicro motherboards, LSI SAS controllers (9211-8i & 9207-8i) and
> various SAS HDD's.  These systems are serving block storage via Comstar
> and Qlogic FC HBA's, and have been running well for several years.
> 
> 	The problem we've got is that as the drives age, some of them start to
> perform slowly (intermittently) without failing - no zpool or iostat
> errors, and nothing logged in /var/adm/messages.  The slow performance
> can be seen as high average service times in iostat or sar.

Look at the drive's error logs using sg_logs (-a for all)

> 
> 	When these service times get above 500ms, they start to cause IO
> timeouts on the downstream storage consumers, which is bad...

500 milliseconds is not unusual for a busy HDD with SCSI TCQ or SATA NCQ

> 
> 	I'm wondering - is there a way to tune OmniOS' behavior so that it
> doesn't try so hard to complete IOs to these slow disks, and instead
> just gives up and fails them?

Yes, the tuning in Alasdair's blog should work as he describes. More below...

> 
> 	I found an old post from 2011 which states that some tunables exist,
> but are ignored by the mpt_sas driver:
> 
> http://everycity.co.uk/alasdair/2011/05/adjusting-drive-timeouts-with-mdb-on-solaris-or-openindiana/
> 
> Does anyone know the current status of these tunables, or have any other
> suggestions that might help?

These tunables are on the order of seconds. The default, 60, is obviously too big
unless you have old, slow, SCSI CD-ROMs. But setting it below the manufacturer's
internal limit (default or tuned) can lead to an unstable system. Some vendors are
better than others at documenting these, but in any case you'll need to see their spec.
Expect values on the order of 6 to 15 seconds for modern HDDs and SSDs.

There are a lot of tunables in this area at all levels of the architecture. OOB, the OmniOS
settings ensure stable behaviour. Tuning any layer without understanding the others can
lead to unstable systems, as demonstrated by your current downstream consumers.
 -- richard


> 
> Thanks,
> Kevin
> 
> 
> -- 
> -------------------------------------------------------------------
> Kevin Swab                          UNIX Systems Administrator
> ACNS                                Colorado State University
> Phone: (970)491-6572                Email: Kevin.Swab at ColoState.EDU
> GPG Fingerprint: 7026 3F66 A970 67BD 6F17  8EB8 8A7D 142F 2392 791C
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss


From henson at acm.org  Wed Dec 31 22:50:28 2014
From: henson at acm.org (Paul B. Henson)
Date: Wed, 31 Dec 2014 14:50:28 -0800
Subject: [OmniOS-discuss] adding cua/a as a second login
In-Reply-To: <20141205085057.5e000d9e@punda-mlia>
References: <20141204153051.3e17ac8f@punda-mlia>
	<dde4299711c6bd8e1f3a24a3fd0b91ce@blackdot.be>
	<20141205073111.0762e2b1@punda-mlia>
	<4cff01d01010$dc508510$94f18f30$@acm.org>
	<20141205085057.5e000d9e@punda-mlia>
Message-ID: <20141231225028.GH29549@bender.unx.csupomona.edu>

On Fri, Dec 05, 2014 at 08:50:57AM +1000, Michael Mounteney wrote:

> Paul:  https://www.illumos.org/issues/5391

I finished the code change to make this work. If you'd like to test it
out, you can find an omnios-stable compatible binary at:

	https://www.cpp.edu/~henson/tmp/login

You can either boot into a new BE to test it, or just make a backup copy
of /usr/bin/login and copy in the test one.  I tested it myself and
believe it functions correctly, but be sure to have an active root login
up in another window to either boot back into the original BE or copy
the original login binary back into place if you have any problems.

I left the console on the frame buffer and set up a secondary serial
login as described at

	https://docu.blackdot.be/snipets/solaris/misc-serial-console

Which is what I believe you did; with a setting of:

CONSOLE=/dev/console,/dev/term/a

I can log in as root on both the framebuffer console and the serial
login. Note that setting CONSOLE to just /dev/console also implicitly
allows the virtual terminals access as root too, adding the second
device removes that implicit access so they would need to be enumerated
individually to maintain it. They're not even enabled by default so
that's probably not an issue for you.

Now I just need to go update the documentation, put up a webrev, and see
what the dev community thinks about it ;).

Let me know if you have any questions or problems.

