From jim at cos.ru  Wed Jul  1 13:20:56 2015
From: jim at cos.ru (Jim Klimov)
Date: Wed, 01 Jul 2015 15:20:56 +0200
Subject: [OmniOS-discuss] pkg(5) in alternate roots
Message-ID: <fc776a65623.55940558@cos.ru>

Hello all,


Due to my split-root configurations, I often do OI/OmniOS upgrades in a pre-created cloned BE (automated by https://github.com/jimklimov/illumos-splitroot-scripts since the generic beadm/zfs clone do a poor job with customized zfs attributes on child datasets). As part of an upgrade, I do a chroot into the mounted new BE to use the pkg(5) "in place" (and with altroot of "/") and if this fails - I fall back to alternate root (-R) with the unchanged pkg(5) version installed in the original BE. In either case, I pass the "--deny-new-be" parameter to pkg(5). (relevant code snippet is presented below in this message)


However there are some packages whose upgrades seem to require a "non-Live BE" installation. And even though I install to an alternate BE, they fail. Is there some way around this, beside fixing pkg(5) itself?


For example, recently I got the state below: the pkg program got updated in the new BE, but it could not be used via chroot due to "live image" concerns, and the old pkg(5) in original BE did not suffice for the upgrades. A reboot into the new BE and another upgrade request fixes this, but that's kind of unwieldy. And installing new pkg into old BE goes somewhat against the purpose of BEs (unchanged old state I can roll back into - thus including escape from broken pkg(5) as any other bit of system software).


Screenshot from the script:



===== Updating PKG software itself
Packages to update: 1


DOWNLOAD ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?PKGS ? ? ? ? FILES ? ?XFER (MB) ? SPEED
Completed ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?1/1 ? ? ? ? 16/16 ? ? ?0.3/0.3 ?303k/s


PHASE ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?ITEMS
Removing old actions ? ? ? ? ? ? ? ? ? ? ? ? ? ? 1/1
Updating modified actions ? ? ? ? ? ? ? ? ? ? ?18/18
Updating package state database ? ? ? ? ? ? ? ? Done
Updating package cache ? ? ? ? ? ? ? ? ? ? ? ? ? 1/1
Updating image state ? ? ? ? ? ? ? ? ? ? ? ? ? ?Done
Creating fast lookup database ? ? ? ? ? ? ? ? ? Done
Reading search index ? ? ? ? ? ? ? ? ? ? ? ? ? ?Done
Updating search index ? ? ? ? ? ? ? ? ? ? ? ? ? ?1/1



===== Updating the image with new PKG software via chroot
pkg update: The proposed operation cannot be performed on a live image.
===== Updating the image with old PKG software via altroot
WARNING: pkg(5) appears to be out of date, and should be updated before
running update. ?Please update pkg(5) by executing 'pkg install
pkg:/package/pkg' as a privileged user and then retry the update.
===== Updating the image with old PKG software via altroot and allowed refresh
WARNING: pkg(5) appears to be out of date, and should be updated before
running update. ?Please update pkg(5) by executing 'pkg install
pkg:/package/pkg' as a privileged user and then retry the update.




The corresponding part of the script (and exact command-lines involved) are here:


...

? ? ? ? { echo "===== Updating PKG software itself"
? ? ? ? ? ### This clause should fail if no 'pkg' updates were available, or if a
? ? ? ? ? ### chrooted upgrade attempt with the new 'pkg' failed - both ways fall
? ? ? ? ? ### through to altroot upgrade attempt
? ? ? ? ? if /usr/bin/pkg -R "$BENEW_MNT" update --no-refresh --accept --deny-new-be --no-backup-be pkg; then \
? ? ? ? ? ? echo "===== Updating the image with new PKG software via chroot"
? ? ? ? ? ? chroot "$BENEW_MNT" /usr/bin/pkg -R / image-update --no-refresh --accept --deny-new-be --no-backup-be
? ? ? ? ? else false; fi; } || \
? ? ? ? { echo "===== Updating the image with old PKG software via altroot"
? ? ? ? ? /usr/bin/pkg -R "$BENEW_MNT" image-update --no-refresh --accept --deny-new-be --no-backup-be; } || \
? ? ? ? { echo "===== Updating the image with old PKG software via altroot and allowed refresh"
? ? ? ? ? /usr/bin/pkg -R "$BENEW_MNT" image-update --accept --deny-new-be --no-backup-be; }
...




Essentially, what I want is for pkg(5) to correctly guess, or be told by the caller, that it in fact does not modify a "live" image and so can do anything dangerous it would please to do.


Thanks in advance,
Jim Klimov




-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150701/f08316f8/attachment-0001.html>

From danmcd at omniti.com  Wed Jul  1 17:40:27 2015
From: danmcd at omniti.com (Dan McDonald)
Date: Wed, 1 Jul 2015 13:40:27 -0400
Subject: [OmniOS-discuss] pkg(5) in alternate roots
In-Reply-To: <fc776a65623.55940558@cos.ru>
References: <fc776a65623.55940558@cos.ru>
Message-ID: <A941A78E-B723-4386-B215-799DAC44F8E1@omniti.com>


> On Jul 1, 2015, at 9:20 AM, Jim Klimov <jim at cos.ru> wrote:
> 
> The corresponding part of the script (and exact command-lines involved) are here:
> 
> 
> ...
> 
>         { echo "===== Updating PKG software itself"
>           ### This clause should fail if no 'pkg' updates were available, or if a
>           ### chrooted upgrade attempt with the new 'pkg' failed - both ways fall
>           ### through to altroot upgrade attempt
>           if /usr/bin/pkg -R "$BENEW_MNT" update --no-refresh --accept --deny-new-be --no-backup-be pkg; then \
>             echo "===== Updating the image with new PKG software via chroot"
>             chroot "$BENEW_MNT" /usr/bin/pkg -R / image-update --no-refresh --accept --deny-new-be --no-backup-be
>           else false; fi; } || \
>         { echo "===== Updating the image with old PKG software via altroot"
>           /usr/bin/pkg -R "$BENEW_MNT" image-update --no-refresh --accept --deny-new-be --no-backup-be; } || \
>         { echo "===== Updating the image with old PKG software via altroot and allowed refresh"
>           /usr/bin/pkg -R "$BENEW_MNT" image-update --accept --deny-new-be --no-backup-be; }

Can you run these, one at a time, by hand, so we all can see the exact output per command that's run?  It'll help at least me get a better bead on what's wrong.

Thanks,
Dan



From jimklimov at cos.ru  Wed Jul  1 18:21:29 2015
From: jimklimov at cos.ru (Jim Klimov)
Date: Wed, 01 Jul 2015 20:21:29 +0200
Subject: [OmniOS-discuss] pkg(5) in alternate roots
In-Reply-To: <fc776a65623.55940558@cos.ru>
References: <fc776a65623.55940558@cos.ru>
Message-ID: <1005F94C-AA59-4E20-95F2-375DB05094B4@cos.ru>

1 ???? 2015??. 15:20:56 CEST, Jim Klimov <jim at cos.ru> ?????:
>Hello all,
>
>
>Due to my split-root configurations, I often do OI/OmniOS upgrades in a
>pre-created cloned BE (automated by
>https://github.com/jimklimov/illumos-splitroot-scripts since the
>generic beadm/zfs clone do a poor job with customized zfs attributes on
>child datasets). As part of an upgrade, I do a chroot into the mounted
>new BE to use the pkg(5) "in place" (and with altroot of "/") and if
>this fails - I fall back to alternate root (-R) with the unchanged
>pkg(5) version installed in the original BE. In either case, I pass the
>"--deny-new-be" parameter to pkg(5). (relevant code snippet is
>presented below in this message)
>
>
>However there are some packages whose upgrades seem to require a
>"non-Live BE" installation. And even though I install to an alternate
>BE, they fail. Is there some way around this, beside fixing pkg(5)
>itself?
>
>
>For example, recently I got the state below: the pkg program got
>updated in the new BE, but it could not be used via chroot due to "live
>image" concerns, and the old pkg(5) in original BE did not suffice for
>the upgrades. A reboot into the new BE and another upgrade request
>fixes this, but that's kind of unwieldy. And installing new pkg into
>old BE goes somewhat against the purpose of BEs (unchanged old state I
>can roll back into - thus including escape from broken pkg(5) as any
>other bit of system software).
>
>
>Screenshot from the script:
>
>
>
>===== Updating PKG software itself
>Packages to update: 1
>
>
>DOWNLOAD ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?PKGS ? ? ? ? FILES ? ?XFER (MB)
>? SPEED
>Completed ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?1/1 ? ? ? ? 16/16 ? ? ?0.3/0.3
>?303k/s
>
>
>PHASE ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?ITEMS
>Removing old actions ? ? ? ? ? ? ? ? ? ? ? ? ? ? 1/1
>Updating modified actions ? ? ? ? ? ? ? ? ? ? ?18/18
>Updating package state database ? ? ? ? ? ? ? ? Done
>Updating package cache ? ? ? ? ? ? ? ? ? ? ? ? ? 1/1
>Updating image state ? ? ? ? ? ? ? ? ? ? ? ? ? ?Done
>Creating fast lookup database ? ? ? ? ? ? ? ? ? Done
>Reading search index ? ? ? ? ? ? ? ? ? ? ? ? ? ?Done
>Updating search index ? ? ? ? ? ? ? ? ? ? ? ? ? ?1/1
>
>
>
>===== Updating the image with new PKG software via chroot
>pkg update: The proposed operation cannot be performed on a live image.
>===== Updating the image with old PKG software via altroot
>WARNING: pkg(5) appears to be out of date, and should be updated before
>running update. ?Please update pkg(5) by executing 'pkg install
>pkg:/package/pkg' as a privileged user and then retry the update.
>===== Updating the image with old PKG software via altroot and allowed
>refresh
>WARNING: pkg(5) appears to be out of date, and should be updated before
>running update. ?Please update pkg(5) by executing 'pkg install
>pkg:/package/pkg' as a privileged user and then retry the update.
>
>
>
>
>The corresponding part of the script (and exact command-lines involved)
>are here:
>
>
>...
>
>? ? ? ? { echo "===== Updating PKG software itself"
>? ? ? ? ? ### This clause should fail if no 'pkg' updates were
>available, or if a
>? ? ? ? ? ### chrooted upgrade attempt with the new 'pkg' failed - both
>ways fall
>? ? ? ? ? ### through to altroot upgrade attempt
>? ? ? ? ? if /usr/bin/pkg -R "$BENEW_MNT" update --no-refresh --accept
>--deny-new-be --no-backup-be pkg; then \
>? ? ? ? ? ? echo "===== Updating the image with new PKG software via
>chroot"
>? ? ? ? ? ? chroot "$BENEW_MNT" /usr/bin/pkg -R / image-update
>--no-refresh --accept --deny-new-be --no-backup-be
>? ? ? ? ? else false; fi; } || \
>? ? ? ? { echo "===== Updating the image with old PKG software via
>altroot"
>? ? ? ? ? /usr/bin/pkg -R "$BENEW_MNT" image-update --no-refresh
>--accept --deny-new-be --no-backup-be; } || \
>? ? ? ? { echo "===== Updating the image with old PKG software via
>altroot and allowed refresh"
>? ? ? ? ? /usr/bin/pkg -R "$BENEW_MNT" image-update --accept
>--deny-new-be --no-backup-be; }
>...
>
>
>
>
>Essentially, what I want is for pkg(5) to correctly guess, or be told
>by the caller, that it in fact does not modify a "live" image and so
>can do anything dangerous it would please to do.
>
>
>Thanks in advance,
>Jim Klimov
>
>
>
>
>
>
>------------------------------------------------------------------------
>
>_______________________________________________
>OmniOS-discuss mailing list
>OmniOS-discuss at lists.omniti.com
>http://lists.omniti.com/mailman/listinfo/omnios-discuss

I guess i found the workaround - the pkg(5) script honors some PKG_LIVE_ROOT variable, for unittesting from what i gather, that along with -R allows to not create BEs for packages that otherwise need them.

I guess this is not a committed api, but will have to do ;)

Jim
--
Typos courtesy of K-9 Mail on my Samsung Android

From danmcd at omniti.com  Wed Jul  1 20:39:38 2015
From: danmcd at omniti.com (Dan McDonald)
Date: Wed, 1 Jul 2015 16:39:38 -0400
Subject: [OmniOS-discuss] REVIEW - splitting out as and sccs from
	soon-to-be-in-ON make
Message-ID: <0781FF0E-FEEB-476A-8EBE-5ACE20AE2172@omniti.com>

Rich Lowe (Cc:ed here) is going to be putting back dmake(1) into ON/illumos VERY soon.

To that end, I've restructured omnios-build to assist the inevitable merge.  Here's a webrev:

	http://kebe.com/~danmcd/webrevs/oob-split-as/

that should cover most everything.  I'm including some useful command-line output to show the results:

bloody(oob-split-as/build)[0]% pkg contents -r -g /tmp/kebe.repo as > /tmp/as   bloody(oob-split-as/build)[0]% pkg contents -r -g /tmp/kebe.repo sccs > /tmp/sccs
bloody(oob-split-as/build)[0]% pkg contents make | sort -u > /tmp/before
bloody(oob-split-as/build)[0]% cat /tmp/as /tmp/sccs | sort -u > /tmp/after
bloody(oob-split-as/build)[0]% wc -l /tmp/{before,after}
      87 /tmp/before
      76 /tmp/after
     163 total
bloody(oob-split-as/build)[0]% diff /tmp/{before,after}
8d7
< usr/bin/make
28d26
< usr/ccs/bin/make
42d39
< usr/ccs/lib/svr4.make
44d40
< usr/lib/amd64/libmakestate.so.1
71d66
< usr/lib/libmakestate.so.1
82,87d76
< usr/lib/svr4.make
< usr/share/lib
< usr/share/lib/make
< usr/share/lib/make/make.rules
< usr/share/lib/make/svr4.make.rules
< usr/xpg4/bin/make
bloody(oob-split-as/build)[1]% pkgrepo list -s /tmp/kebe.repo/
PUBLISHER NAME                                          O VERSION
omnios    developer/as                                    0.5.11-0.151015:20150701T200724Z
omnios    developer/versioning/sccs                       0.5.11-0.151015:20150701T200711Z
bloody(oob-split-as/build)[0]% 


Thanks!
Dan


From jim at cos.ru  Thu Jul  2 09:33:31 2015
From: jim at cos.ru (Jim Klimov)
Date: Thu, 02 Jul 2015 11:33:31 +0200
Subject: [OmniOS-discuss] pkg(5) in alternate roots
In-Reply-To: <A941A78E-B723-4386-B215-799DAC44F8E1@omniti.com>
References: <fc776a65623.55940558@cos.ru>
	<A941A78E-B723-4386-B215-799DAC44F8E1@omniti.com>
Message-ID: <fc788a8932ba.5595218b@cos.ru>

??: Dan McDonald <danmcd at omniti.com>
????: Wednesday, July 1, 2015 19:43
> 
> > On Jul 1, 2015, at 9:20 AM, Jim Klimov <jim at cos.ru> wrote:
> > 
> > The corresponding part of the script (and exact command-lines 
> involved) are here:
> > 
> > 
> > ...
> > 
> >???????? { echo "===== 
> Updating PKG software itself"
> >?????????? 
> ### This clause should fail if no 'pkg' updates were available, 
> or if a
> >?????????? 
> ### chrooted upgrade attempt with the new 'pkg' failed - both 
> ways fall
> >?????????? 
> ### through to altroot upgrade attempt
> >?????????? if 
> /usr/bin/pkg -R "$BENEW_MNT" update --no-refresh --accept --deny-
> new-be --no-backup-be pkg; then \
> >???????????? echo "===== Updating the image with new PKG software via chroot"
> >???????????? chroot "$BENEW_MNT" /usr/bin/pkg -R / image-update --no-refresh --accept --deny-new-be --no-backup-be
> >?????????? 
> else false; fi; } || \
> >???????? { echo "===== 
> Updating the image with old PKG software via altroot"
> >?????????? 
> /usr/bin/pkg -R "$BENEW_MNT" image-update --no-refresh --accept -
> -deny-new-be --no-backup-be; } || \
> >???????? { echo "===== 
> Updating the image with old PKG software via altroot and allowed 
> refresh">?????????? /usr/bin/pkg -R "$BENEW_MNT" image-update --accept --deny-new-be --no-backup-be; }
> 
> Can you run these, one at a time, by hand, so we all can see the 
> exact output per command that's run?? It'll help at least 
> me get a better bead on what's wrong.
> 
> Thanks,
> Dan


Most of the output was in the screenshot just above that, and I guess I found a workaround?
that should suffice (will test more today), but for the sake of completeness, here goes:


The old BE was OmniOS bloody updated a month ago (20150602Z070407), new one with an updated pkg program was made yesterday, starting as the old one's clone (export BENEW=omnios-20150701Z065339-test) and the relevant tree was mounted into "/a" with my splitroot scripts from github:



###############################


:; ./beadm-mount.sh
omnios-20150701Z065339-test does not exist or appear to be a valid BE.
Please check that the name of the BE provided is correct.
=== Will clone omnios-20150602Z070407 into omnios-20150701Z065339-test, ok? Using these settings:
BENEW=omnios-20150701Z065339-test
BENEW_DS=rpool/ROOT/omnios-20150701Z065339-test
BENEW_MNT=/a
BENEW_MPT=/a
BEOLD=omnios-20150602Z070407
BEOLD_DS=rpool/ROOT/omnios-20150602Z070407
BEOLD_MNT=/
BEOLD_MPT=/
EXCLUDE_ATTRS='org.opensolaris.libbe:uuid|canmount|mountpoint'
RPOOL=rpool
RPOOLALT=
RPOOL_ROOT=rpool/ROOT
RPOOL_SHARED=rpool/SHARED
? ? (Press ENTER or CTRL+C)


Created successfully
=== Created omnios-20150701Z065339-test based on omnios-20150602Z070407
=== Trying to replicate ZFS attributes from original to new BE...
===== ''
org.opensolaris.libbe:policy=static
org.openindiana.caiman:install=ready
===== '/opt'
compression=gzip-9
===== '/opt/local'
compression=gzip-9
===== '/usr'
compression=gzip-9
===== '/usr/local'
compression=gzip-9
===== '/var'
compression=gzip-9
===== '/var/db'
===== '/var/db/pkgin'
===== '/var/pkg'
compression=gzip-9
===== '/var/pkg/publisher'
=== Replicated custom ZFS attributes
Mounted successfully on: '/a'
=== Mounted omnios-20150701Z065339-test at /a
rpool/ROOT/omnios-20150701Z065339-test ? 153387008 ? ? ?721348 ? 124154515 ? ? 1% ? ?/a
rpool/ROOT/omnios-20150701Z065339-test/usr ? 153387008 ? ? ?758981 ? 124154515 ? ? 1% ? ?/a/usr
rpool/ROOT/omnios-20150701Z065339-test/usr/local ? 153387008 ? ? ? ? ?28 ? 124154515 ? ? 1% ? ?/a/usr/local
rpool/ROOT/omnios-20150701Z065339-test/opt ? 153387008 ? ? 1207927 ? 124154515 ? ? 1% ? ?/a/opt
rpool/ROOT/omnios-20150701Z065339-test/opt/local ? 153387008 ? ? ?811058 ? 124154515 ? ? 1% ? ?/a/opt/local
rpool/ROOT/omnios-20150701Z065339-test/var ? 153387008 ? ? ? 18600 ? 124154515 ? ? 1% ? ?/a/var
rpool/ROOT/omnios-20150701Z065339-test/var/db ? 153387008 ? ? ? ? ?33 ? 124154515 ? ? 1% ? ?/a/var/db
rpool/ROOT/omnios-20150701Z065339-test/var/db/pkgin ? 153387008 ? ? ?665190 ? 124154515 ? ? 1% ? ?/a/var/db/pkgin
rpool/ROOT/omnios-20150701Z065339-test/var/pkg ? 153387008 ? ? ? 38370 ? 124154515 ? ? 1% ? ?/a/var/pkg
rpool/ROOT/omnios-20150701Z065339-test/var/pkg/publisher ? 153387008 ? ? ? ?9364 ? 124154515 ? ? 1% ? ?/a/var/pkg/publisher
=== BE clone completed!


=== To upgrade from upstream do:
pkg -R "/a" image-update --deny-new-be --no-backup-be && touch "/a/reconfigure" && bootadm update-archive -R "/a" && beadm umount "omnios-20150701Z065339-test"


TS="`date -u "+%Y%m%dZ%H%M%S"`"
zfs snapshot -r "rpool/SHARED at postupgrade-$TS"
zfs snapshot -r "rpool/ROOT/omnios-20150701Z065339-test at postupgrade-$TS"


beadm activate "omnios-20150701Z065339-test"
===


INFO: Unmounting any previous traces (if any - may fail), just in case
=== Unmounting BE omnios-20150701Z065339-test under '/a'...
===== beadm-unmounting omnios-20150701Z065339-test (/a)...
Unmounted successfully
Mounted successfully on: '/a'
BE ? ? ? ? ? ? ? ? ? ? ? ? ?Active Mountpoint Space Policy Created
omnios-20150701Z065339-test - ? ? ?/a ? ? ? ? 15.0K static 2015-07-01 17:09
===== lofs-mount '/tmp' at '/a/tmp'
===== lofs-mount '/proc' at '/a/proc'
===== lofs-mount '/dev' at '/a/dev'
===== lofs-mount '/devices' at '/a/devices'
===== lofs-mount '/var/adm' at '/a/var/adm'
===== lofs-mount '/var/cores' at '/a/var/cores'
===== lofs-mount '/var/crash' at '/a/var/crash'
===== lofs-mount '/var/log' at '/a/var/log'
===== lofs-mount '/var/mail' at '/a/var/mail'
===== lofs-mount '/var/spool/clientmqueue' at '/a/var/spool/clientmqueue'
===== lofs-mount '/var/spool/mqueue' at '/a/var/spool/mqueue'
===== lofs-mount '/var/tmp' at '/a/var/tmp'


BE ? ? ? ? ? ? ? ? ? ? ? ? ?Active Mountpoint Space Policy Created
omnios-20150701Z065339-test - ? ? ?/a ? ? ? ? 15.0K static 2015-07-01 17:09
rpool/ROOT/omnios-20150701Z065339-test ? 153387008 ? ? ?721348 ? 124154491 ? ? 1% ? ?/a
rpool/ROOT/omnios-20150701Z065339-test/usr ? 153387008 ? ? ?758981 ? 124154491 ? ? 1% ? ?/a/usr
rpool/ROOT/omnios-20150701Z065339-test/usr/local ? 153387008 ? ? ? ? ?28 ? 124154491 ? ? 1% ? ?/a/usr/local
rpool/ROOT/omnios-20150701Z065339-test/opt ? 153387008 ? ? 1207927 ? 124154491 ? ? 1% ? ?/a/opt
rpool/ROOT/omnios-20150701Z065339-test/opt/local ? 153387008 ? ? ?811058 ? 124154491 ? ? 1% ? ?/a/opt/local
rpool/ROOT/omnios-20150701Z065339-test/var ? 153387008 ? ? ? 18600 ? 124154491 ? ? 1% ? ?/a/var
rpool/ROOT/omnios-20150701Z065339-test/var/db ? 153387008 ? ? ? ? ?33 ? 124154491 ? ? 1% ? ?/a/var/db
rpool/ROOT/omnios-20150701Z065339-test/var/db/pkgin ? 153387008 ? ? ?665190 ? 124154491 ? ? 1% ? ?/a/var/db/pkgin
rpool/ROOT/omnios-20150701Z065339-test/var/pkg ? 153387008 ? ? ? 38370 ? 124154491 ? ? 1% ? ?/a/var/pkg
rpool/ROOT/omnios-20150701Z065339-test/var/pkg/publisher ? 153387008 ? ? ? ?9364 ? 124154491 ? ? 1% ? ?/a/var/pkg/publisher
/tmp ? ? ? ? ? ? ? ? ? ?14744832 ? ? ? ? ?16 ? ?14744816 ? ? 1% ? ?/a/tmp
/proc ? ? ? ? ? ? ? ? ? ? ? ? ?0 ? ? ? ? ? 0 ? ? ? ? ? 0 ? ? 0% ? ?/a/proc
/dev ? ? ? ? ? ? ? ? ? ? ? ? ? 0 ? ? ? ? ? 0 ? ? ? ? ? 0 ? ? 0% ? ?/a/dev
/devices ? ? ? ? ? ? ? ? ? ? ? 0 ? ? ? ? ? 0 ? ? ? ? ? 0 ? ? 0% ? ?/a/devices
/var/adm ? ? ? ? ? ? ? 124172025 ? ? ? ?3071 ? 124168954 ? ? 1% ? ?/a/var/adm
/var/cores ? ? ? ? ? ? ? 5241366 ? ? ? ? ?19 ? ? 5241347 ? ? 1% ? ?/a/var/cores
/var/crash ? ? ? ? ? ? ? 5242880 ? ? ? ? ?19 ? ? 5242861 ? ? 1% ? ?/a/var/crash
/var/log ? ? ? ? ? ? ? 124173841 ? ? ? ? 337 ? 124173504 ? ? 1% ? ?/a/var/log
/var/mail ? ? ? ? ? ? ?124173921 ? ? ? ? 114 ? 124173807 ? ? 1% ? ?/a/var/mail
/var/spool/clientmqueue ? ? 2096461 ? ? ? ? ?26 ? ? 2096435 ? ? 1% ? ?/a/var/spool/clientmqueue
/var/spool/mqueue ? ? ? ?2096392 ? ? ? ? ?30 ? ? 2096362 ? ? 1% ? ?/a/var/spool/mqueue
/var/tmp ? ? ? ? ? ? ? 124515436 ? ? ?360944 ? 124154491 ? ? 1% ? ?/a/var/tmp


=== SUCCESS, (un)mounting operations completed for BE 'omnios-20150701Z065339-test' at '/a'


###############################




So, the tree above is mounted. Updating pkg under /a altroot itself (note the script does this via "pkg -R /a update pkg", but in the example below I do it via chroot to verify it generally works for this):



###############################

:; ?chroot /a /usr/bin/pkg -R / ?update pkg
WARNING: The boot environment being modified is not the active one. ?Changes
made will not be reflected on the next boot.


? ? ? ? ? ? Packages to update: ? 1
? ? ? ?Create boot environment: ?No
Create backup boot environment: Yes


DOWNLOAD ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?PKGS ? ? ? ? FILES ? ?XFER (MB) ? SPEED
Completed ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?1/1 ? ? ? ? 16/16 ? ? ?0.3/0.3 ?183k/s


PHASE ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?ITEMS
Removing old actions ? ? ? ? ? ? ? ? ? ? ? ? ? ? 1/1
Updating modified actions ? ? ? ? ? ? ? ? ? ? ?18/18
Updating package state database ? ? ? ? ? ? ? ? Done
Updating package cache ? ? ? ? ? ? ? ? ? ? ? ? ? 1/1
Updating image state ? ? ? ? ? ? ? ? ? ? ? ? ? ?Done
Creating fast lookup database ? ? ? ? ? ? ? ? ? Done
Reading search index ? ? ? ? ? ? ? ? ? ? ? ? ? ?Done
Updating search index ? ? ? ? ? ? ? ? ? ? ? ? ? ?1/1
###############################


Now, trying to update packages inside that altroot in different ways...


What the script tries?in the chroot:

###############################

:; chroot /a /usr/bin/pkg -R / image-update --no-refresh --accept --deny-new-be --no-backup-be
Creating Plan (Package planning: ?425/427): /
pkg update: The proposed operation cannot be performed on a live image.
###############################


Trying to cheat with a non-verbatim "/" altroot string in the chroot:

###############################

:; chroot /a /usr/bin/pkg -R /// image-update --no-refresh --accept --deny-new-be --no-backup-be
pkg update: The proposed operation cannot be performed on a live image.
###############################


Trying the two other ways the script attempts (original BE's old pkg over a new BE's packages)

###############################

:;?/usr/bin/pkg -R /a image-update --no-refresh --accept --deny-new-be --no-backup-be

Checking that pkg(5) is up to date -



WARNING: pkg(5) appears to be out of date, and should be updated before
running update. ?Please update pkg(5) by executing 'pkg install
pkg:/package/pkg' as a privileged user and then retry the update.
###############################


###############################

:; /usr/bin/pkg -R /a image-update --accept --deny-new-be --no-backup-be
WARNING: pkg(5) appears to be out of date, and should be updated before
running update. ?Please update pkg(5) by executing 'pkg install
pkg:/package/pkg' as a privileged user and then retry the update.
###############################




Finally, reading through the sources at /usr/bin/pkg and?/usr/lib/python2.*/vendor-packages/pkg/ I tracked the error message to decision points to is_liveroot() to obscure envvars and/or configuration settings that cause some boolean regarding required new BE's to be true or false regardless of package-reboot settings.


Here is a piece which made me think it is intended for debug and so might not be a piece of committed API (would love to be wrong on this point):?https://java.net/projects/ips/sources/pkg-gate/content/src/tests/api/t_bootenv.py#62


So, using PKG_LIVE_ROOT envvar, but no -R for altroot - no banana:

###############################

:; PKG_LIVE_ROOT=/ chroot /a /usr/bin/pkg image-update --no-refresh --accept --deny-new-be --no-backup-be
pkg update: The proposed operation cannot be performed on a live image.
###############################



Using both PKG_LIVE_ROOT envvar and -R for altroot - this is what ultimately succeeded, though only with a non-verbatim-"/" root directory string in this envvar:

###############################
:; PKG_LIVE_ROOT=/ chroot /a /usr/bin/pkg -R / image-update --no-refresh --accept --deny-new-be --no-backup-be
pkg update: The proposed operation cannot be performed on a live image.


:; PKG_LIVE_ROOT=/// chroot /a /usr/bin/pkg -R /// ?image-update --no-refresh --accept --deny-new-be --no-backup-be

?Packages to remove: ? 1
Packages to install: ? 2
?Packages to update: 424
?Services to change: ? 1


DOWNLOAD ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?PKGS ? ? ? ? FILES ? ?XFER (MB) ? SPEED
SUNWcs ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? 1/427 ? ? ?476/7699 ? ?6.4/418.8 ?1.1M/s^C





:; PKG_LIVE_ROOT=/// chroot /a /usr/bin/pkg -R / image-update --no-refresh --accept --deny-new-be --no-backup-be
?Packages to remove: ? 1
Packages to install: ? 2
?Packages to update: 424
?Services to change: ? 1


DOWNLOAD ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?PKGS ? ? ? ? FILES ? ?XFER (MB) ? SPEED
SUNWcs ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? 1/427 ? ? ? 96/7699 ? ?1.2/418.8 ?516k/s^C
###############################


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150702/1dabe9c7/attachment-0001.html>

From graham at thestephensdomain.com  Thu Jul  2 12:42:19 2015
From: graham at thestephensdomain.com (Graham Stephens)
Date: Thu, 02 Jul 2015 13:42:19 +0100
Subject: [OmniOS-discuss] KVM vnics not starting
In-Reply-To: <558E843E.4070108@thestephensdomain.com>
References: <558D18FD.20103@thestephensdomain.com>	<201506261842.t5QIgpQC006523@elvis.arl.psu.edu>
	<558E843E.4070108@thestephensdomain.com>
Message-ID: <559531AB.4090704@thestephensdomain.com>



On 27/06/2015 12:08, Graham Stephens wrote:
>
>
> On 26/06/2015 19:42, John D Groenveld wrote:
>> In message <558D18FD.20103 at thestephensdomain.com>, Graham Stephens
>> writes:
>>> Does anyone else here have problems trying to start a kvm due to vnics
>>> not initializing properly? It seems to me that the more vnics you add,
>>> the more likely the problem is to occur.
>>
>> No.
>>
>> What's your reproducable configuration?
>> What version OmniOS?
>> How many VMs and VNICs?
>> What's your KVM invocations?
>>
>> John
>> groenveld at acm.org
> -------------------
>
> I also get these messages in the global log, but only AFTER the kvm has
> managed to start:
>
> Jun 27 09:13:19 eppigw01 genunix: [ID 420667 kern.info] kvm_lapic_reset:
> vcpu=ffffff04ffa6d000, id=0, base_msr= fee00100 PRIx64
> base_address=fee00000
> Jun 27 09:13:19 eppigw01 genunix: [ID 710719 kern.info] vmcs revision_id
> = f
> Jun 27 09:13:19 eppigw01 genunix: [ID 420667 kern.info] kvm_lapic_reset:
> vcpu=ffffff04ffa65000, id=1, base_msr= fee00000 PRIx64
> base_address=fee00000
> Jun 27 09:13:19 eppigw01 genunix: [ID 710719 kern.info] vmcs revision_id
> = f
> Jun 27 09:13:21 eppigw01 genunix: [ID 391722 kern.info] unhandled wrmsr:
> 0xfed8c25b data 53
> Jun 27 09:13:21 eppigw01 last message repeated 1 time
> Jun 27 09:13:21 eppigw01 genunix: [ID 391722 kern.info] unhandled wrmsr:
> 0xff3fd4a0 data fffffd7fffdfea70
> Jun 27 09:13:21 eppigw01 last message repeated 1 time
> Jun 27 09:13:21 eppigw01 genunix: [ID 391722 kern.info] unhandled wrmsr:
> 0xffdff078 data fffffd7fffdfe900
> Jun 27 09:13:21 eppigw01 last message repeated 1 time
> Jun 27 09:13:21 eppigw01 genunix: [ID 291337 kern.info] vcpu 1 received
> sipi with vector # 10
> Jun 27 09:13:21 eppigw01 genunix: [ID 420667 kern.info] kvm_lapic_reset:
> vcpu=ffffff04ffa65000, id=1, base_msr= fee00800 PRIx64
> base_address=fee00000
> Jun 27 09:13:37 eppigw01 genunix: [ID 713435 kern.info] unhandled rdmsr:
> 0x756e6547
> Jun 27 09:13:37 eppigw01 genunix: [ID 391722 kern.info] unhandled wrmsr:
> 0x0 data 0
> Jun 27 09:13:44 eppigw01 genunix: [ID 291337 kern.info] vcpu 1 received
> sipi with vector # 10
> Jun 27 09:13:44 eppigw01 genunix: [ID 420667 kern.info] kvm_lapic_reset:
> vcpu=ffffff04ffa65000, id=1, base_msr= fee00800 PRIx64
> base_address=fee00000
> Jun 27 09:13:44 eppigw01 genunix: [ID 713435 kern.info] unhandled rdmsr:
> 0x756e6547
> Jun 27 09:13:44 eppigw01 genunix: [ID 391722 kern.info] unhandled wrmsr:
> 0x0 data 0
> Jun 27 09:13:44 eppigw01 genunix: [ID 713435 kern.info] unhandled rdmsr:
> 0x756e6547
> Jun 27 09:13:44 eppigw01 genunix: [ID 391722 kern.info] unhandled wrmsr:
> 0x0 data 0
>
> (sorry for the email client formatting)
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
---------------------------------------------------------

An update to tell tales of weirdness...

One of two things did "something":

1) pkg update in the non-global zone for Openssl & ntp

2) I attempted to turn off the new CPUID feature in the bios (new from 
2008!) as I didn't know what it was for, with a resultant OmniOS crash 
at startup. So I changed it back. ;)

One of these two things appears to have helped with kvm startup. Early 
days, sure, but I have restarted three or four times without issue. I'll 
let you know if it decides not to play nice again.

From graham at thestephensdomain.com  Thu Jul  2 13:39:08 2015
From: graham at thestephensdomain.com (Graham Stephens)
Date: Thu, 02 Jul 2015 14:39:08 +0100
Subject: [OmniOS-discuss] KVM vnics not starting
In-Reply-To: <559531AB.4090704@thestephensdomain.com>
References: <559531AB.4090704@thestephensdomain.com>
Message-ID: <55953EFC.4030705@thestephensdomain.com>


On 27/06/2015 12:08, Graham Stephens wrote:
>
>
> On 26/06/2015 19:42, John D Groenveld wrote:
>> In message <558D18FD.20103 at thestephensdomain.com>, Graham Stephens
>> writes:
>>> Does anyone else here have problems trying to start a kvm due to vnics
>>> not initializing properly? It seems to me that the more vnics you add,
>>> the more likely the problem is to occur.
>>
>> No.
>>
>> What's your reproducable configuration?
>> What version OmniOS?
>> How many VMs and VNICs?
>> What's your KVM invocations?
>>
>> John
>> groenveld at acm.org
> -------------------
>
> I also get these messages in the global log, but only AFTER the kvm has
> managed to start:
>
> Jun 27 09:13:19 eppigw01 genunix: [ID 420667 kern.info] kvm_lapic_reset:
> vcpu=ffffff04ffa6d000, id=0, base_msr= fee00100 PRIx64
> base_address=fee00000
> Jun 27 09:13:19 eppigw01 genunix: [ID 710719 kern.info] vmcs revision_id
> = f
> Jun 27 09:13:19 eppigw01 genunix: [ID 420667 kern.info] kvm_lapic_reset:
> vcpu=ffffff04ffa65000, id=1, base_msr= fee00000 PRIx64
> base_address=fee00000
> Jun 27 09:13:19 eppigw01 genunix: [ID 710719 kern.info] vmcs revision_id
> = f
> Jun 27 09:13:21 eppigw01 genunix: [ID 391722 kern.info] unhandled wrmsr:
> 0xfed8c25b data 53
> Jun 27 09:13:21 eppigw01 last message repeated 1 time
> Jun 27 09:13:21 eppigw01 genunix: [ID 391722 kern.info] unhandled wrmsr:
> 0xff3fd4a0 data fffffd7fffdfea70
> Jun 27 09:13:21 eppigw01 last message repeated 1 time
> Jun 27 09:13:21 eppigw01 genunix: [ID 391722 kern.info] unhandled wrmsr:
> 0xffdff078 data fffffd7fffdfe900
> Jun 27 09:13:21 eppigw01 last message repeated 1 time
> Jun 27 09:13:21 eppigw01 genunix: [ID 291337 kern.info] vcpu 1 received
> sipi with vector # 10
> Jun 27 09:13:21 eppigw01 genunix: [ID 420667 kern.info] kvm_lapic_reset:
> vcpu=ffffff04ffa65000, id=1, base_msr= fee00800 PRIx64
> base_address=fee00000
> Jun 27 09:13:37 eppigw01 genunix: [ID 713435 kern.info] unhandled rdmsr:
> 0x756e6547
> Jun 27 09:13:37 eppigw01 genunix: [ID 391722 kern.info] unhandled wrmsr:
> 0x0 data 0
> Jun 27 09:13:44 eppigw01 genunix: [ID 291337 kern.info] vcpu 1 received
> sipi with vector # 10
> Jun 27 09:13:44 eppigw01 genunix: [ID 420667 kern.info] kvm_lapic_reset:
> vcpu=ffffff04ffa65000, id=1, base_msr= fee00800 PRIx64
> base_address=fee00000
> Jun 27 09:13:44 eppigw01 genunix: [ID 713435 kern.info] unhandled rdmsr:
> 0x756e6547
> Jun 27 09:13:44 eppigw01 genunix: [ID 391722 kern.info] unhandled wrmsr:
> 0x0 data 0
> Jun 27 09:13:44 eppigw01 genunix: [ID 713435 kern.info] unhandled rdmsr:
> 0x756e6547
> Jun 27 09:13:44 eppigw01 genunix: [ID 391722 kern.info] unhandled wrmsr:
> 0x0 data 0
>
> (sorry for the email client formatting)
---------------------------------------------------------

An update to tell tales of weirdness...

One of two things did "something":

1) pkg update in the non-global zone for Openssl & ntp

2) I attempted to turn off the new CPUID feature in the bios (new from
2008!) as I didn't know what it was for, with a resultant OmniOS crash
at startup. So I changed it back. ;)

One of these two things appears to have helped with kvm startup. Early
days, sure, but I have restarted three or four times without issue. I'll
let you know if it decides not to play nice again.

----------------------------------------------------------

I spoke too soon - I added another vnic and it took half a dozen 
attempts to start. :(

From johan.kragsterman at capvert.se  Thu Jul  2 13:55:07 2015
From: johan.kragsterman at capvert.se (Johan Kragsterman)
Date: Thu, 2 Jul 2015 15:55:07 +0200
Subject: [OmniOS-discuss] SSD rpool degraded
Message-ID: <OFABACE78C.9AA09E81-ONC1257E76.004B4A1F-C1257E76.004C7537@inse.com>


Hi!

I got a degraded rpool, consisting of a mirror of two SSD's. I feel unsure about if it really is the SSD that have failed, since it is enterprise grade and haven't been running that long

I would like to know if there is a way to figure out wether it is the SATA port or the SSD that have failed.

The zpool status looks like this:

NAME          STATE     READ WRITE CKSUM
        rpool         DEGRADED     0     0     0
          mirror-0    DEGRADED     0     0     0
            c2t0d0s0  ONLINE       0     0     0
            c2t1d0s0  FAULTED      1   191     0  too many errors



dmesg containes this:

Jun 29 00:39:47 omni2 genunix: [ID 517647 kern.warning] WARNING: ahci0: watchdog port 1 satapkt 0xffffff065eb76860 timed out
Jun 29 00:39:58 omni2 genunix: [ID 860969 kern.warning] WARNING: ahci0: ahci_port_reset port 1 the device hardware has been initialized and the power-up diagnostics failed
Jun 29 00:39:59 omni2 genunix: [ID 801845 kern.info] /pci at 0,0/pci1028,26e at 1f,2:
Jun 29 00:39:59 omni2  SATA port 1 error
Jun 29 00:39:59 omni2 genunix: [ID 801845 kern.info] /pci at 0,0/pci1028,26e at 1f,2:
Jun 29 00:39:59 omni2  SATA port 1 error
Jun 29 00:39:59 omni2 genunix: [ID 801845 kern.info] /pci at 0,0/pci1028,26e at 1f,2:
Jun 29 00:39:59 omni2  SATA port 1 error
Jun 29 00:39:59 omni2 genunix: [ID 801845 kern.info] /pci at 0,0/pci1028,26e at 1f,2:
Jun 29 00:39:59 omni2  SATA port 1 error
Jun 29 00:39:59 omni2 genunix: [ID 801845 kern.info] /pci at 0,0/pci1028,26e at 1f,2:
Jun 29 00:39:59 omni2  SATA port 1 error
Jun 29 00:39:59 omni2 genunix: [ID 801845 kern.info] /pci at 0,0/pci1028,26e at 1f,2:
Jun 29 00:39:59 omni2  SATA port 1 error
Jun 29 00:39:59 omni2 genunix: [ID 801845 kern.info] /pci at 0,0/pci1028,26e at 1f,2:
Jun 29 00:39:59 omni2  SATA port 1 error
Jun 29 00:39:59 omni2 genunix: [ID 801845 kern.info] /pci at 0,0/pci1028,26e at 1f,2:
Jun 29 00:39:59 omni2  SATA port 1 error
Jun 29 00:39:59 omni2 genunix: [ID 801845 kern.info] /pci at 0,0/pci1028,26e at 1f,2:
Jun 29 00:39:59 omni2  SATA port 1 error
Jun 29 00:39:59 omni2 genunix: [ID 801845 kern.info] /pci at 0,0/pci1028,26e at 1f,2:
Jun 29 00:39:59 omni2  SATA port 1 error
Jun 29 00:39:59 omni2 genunix: [ID 801845 kern.info] /pci at 0,0/pci1028,26e at 1f,2:
Jun 29 00:39:59 omni2  SATA port 1 error
Jun 29 00:40:14 omni2 fmd: [ID 377184 daemon.error] SUNW-MSG-ID: ZFS-8000-FD, TYPE: Fault, VER: 1, SEVERITY: Major
Jun 29 00:40:14 omni2 EVENT-TIME: Mon Jun 29 00:40:14 CEST 2015
Jun 29 00:40:14 omni2 PLATFORM: Precision-WorkStation-T5500, CSN: BCLJ55J, HOSTNAME: omni2
Jun 29 00:40:14 omni2 SOURCE: zfs-diagnosis, REV: 1.0
Jun 29 00:40:14 omni2 EVENT-ID: e44ba921-004f-61f8-cbdf-8f1ebf0d57c0
Jun 29 00:40:14 omni2 DESC: The number of I/O errors associated with a ZFS device exceeded
Jun 29 00:40:14 omni2        acceptable levels.  Refer to http://illumos.org/msg/ZFS-8000-FD for more information.
Jun 29 00:40:14 omni2 AUTO-RESPONSE: The device has been offlined and marked as faulted.  An attempt
Jun 29 00:40:14 omni2        will be made to activate a hot spare if available. 
Jun 29 00:40:14 omni2 IMPACT: Fault tolerance of the pool may be compromised.
Jun 29 00:40:14 omni2 REC-ACTION: Run 'zpool status -x' and replace the bad device.



>From that it looks like zfs hinting that it is the device, not the port...




cfgadm -al:

root at omni2:/root# cfgadm -al
Ap_Id                          Type         Receptacle   Occupant     Condition
c3                             scsi-sas     connected    unconfigured unknown
c5                             scsi-sas     connected    configured   unknown
c5::w5000c50078e5135e,0        disk-path    connected    configured   unknown
c8                             scsi-sas     connected    configured   unknown
c8::w5000c5007ffee30b,0        disk-path    connected    configured   unknown
c9                             scsi-sas     connected    configured   unknown
c9::w500a0751034af6dc,0        disk-path    connected    configured   unknown
sata1/0::dsk/c2t0d0            disk         connected    configured   ok
sata1/1                        sata-port    disconnected unconfigured failed
sata1/2                        sata-port    empty        unconfigured ok
sata1/3                        sata-port    empty        unconfigured ok
sata1/4                        sata-port    empty        unconfigured ok
sata1/5                        sata-port    empty        unconfigured ok


But with cfgadm I get unsure again...

Someone know...?


Best regards from/Med v?nliga h?lsningar fr?n

Johan Kragsterman

Capvert


From yavoritomov at gmail.com  Thu Jul  2 14:41:04 2015
From: yavoritomov at gmail.com (Yavor Tomov)
Date: Thu, 2 Jul 2015 09:41:04 -0500
Subject: [OmniOS-discuss] SSD rpool degraded
In-Reply-To: <OFABACE78C.9AA09E81-ONC1257E76.004B4A1F-C1257E76.004C7537@inse.com>
References: <OFABACE78C.9AA09E81-ONC1257E76.004B4A1F-C1257E76.004C7537@inse.com>
Message-ID: <CAJ5wh+7BAE-zKDHZxzViLLxod2bFd+iUw64-oTQSR=Gd=ni7qw@mail.gmail.com>

First make sure you have a backup of your data. SSD are famous of failing
at the same time if they were installed at the same time. Also if you are
using the pool by write intensive application SSDs can die quick. Depending
on the drive you should be able to look at the Erase Count attribute from
the SMART information. The value should be above a 100, if under some data
loss is possible, if "0" the ssd will stop working, all this dependence on
the manufacture. The easiest thing will be to replace the drive and see if
the errors come back.

Good Luck
Yavor Tomov

On Thu, Jul 2, 2015 at 8:55 AM, Johan Kragsterman <
johan.kragsterman at capvert.se> wrote:

>
> Hi!
>
> I got a degraded rpool, consisting of a mirror of two SSD's. I feel unsure
> about if it really is the SSD that have failed, since it is enterprise
> grade and haven't been running that long
>
> I would like to know if there is a way to figure out wether it is the SATA
> port or the SSD that have failed.
>
> The zpool status looks like this:
>
> NAME          STATE     READ WRITE CKSUM
>         rpool         DEGRADED     0     0     0
>           mirror-0    DEGRADED     0     0     0
>             c2t0d0s0  ONLINE       0     0     0
>             c2t1d0s0  FAULTED      1   191     0  too many errors
>
>
>
> dmesg containes this:
>
> Jun 29 00:39:47 omni2 genunix: [ID 517647 kern.warning] WARNING: ahci0:
> watchdog port 1 satapkt 0xffffff065eb76860 timed out
> Jun 29 00:39:58 omni2 genunix: [ID 860969 kern.warning] WARNING: ahci0:
> ahci_port_reset port 1 the device hardware has been initialized and the
> power-up diagnostics failed
> Jun 29 00:39:59 omni2 genunix: [ID 801845 kern.info] /pci at 0
> ,0/pci1028,26e at 1f,2:
> Jun 29 00:39:59 omni2  SATA port 1 error
> Jun 29 00:39:59 omni2 genunix: [ID 801845 kern.info] /pci at 0
> ,0/pci1028,26e at 1f,2:
> Jun 29 00:39:59 omni2  SATA port 1 error
> Jun 29 00:39:59 omni2 genunix: [ID 801845 kern.info] /pci at 0
> ,0/pci1028,26e at 1f,2:
> Jun 29 00:39:59 omni2  SATA port 1 error
> Jun 29 00:39:59 omni2 genunix: [ID 801845 kern.info] /pci at 0
> ,0/pci1028,26e at 1f,2:
> Jun 29 00:39:59 omni2  SATA port 1 error
> Jun 29 00:39:59 omni2 genunix: [ID 801845 kern.info] /pci at 0
> ,0/pci1028,26e at 1f,2:
> Jun 29 00:39:59 omni2  SATA port 1 error
> Jun 29 00:39:59 omni2 genunix: [ID 801845 kern.info] /pci at 0
> ,0/pci1028,26e at 1f,2:
> Jun 29 00:39:59 omni2  SATA port 1 error
> Jun 29 00:39:59 omni2 genunix: [ID 801845 kern.info] /pci at 0
> ,0/pci1028,26e at 1f,2:
> Jun 29 00:39:59 omni2  SATA port 1 error
> Jun 29 00:39:59 omni2 genunix: [ID 801845 kern.info] /pci at 0
> ,0/pci1028,26e at 1f,2:
> Jun 29 00:39:59 omni2  SATA port 1 error
> Jun 29 00:39:59 omni2 genunix: [ID 801845 kern.info] /pci at 0
> ,0/pci1028,26e at 1f,2:
> Jun 29 00:39:59 omni2  SATA port 1 error
> Jun 29 00:39:59 omni2 genunix: [ID 801845 kern.info] /pci at 0
> ,0/pci1028,26e at 1f,2:
> Jun 29 00:39:59 omni2  SATA port 1 error
> Jun 29 00:39:59 omni2 genunix: [ID 801845 kern.info] /pci at 0
> ,0/pci1028,26e at 1f,2:
> Jun 29 00:39:59 omni2  SATA port 1 error
> Jun 29 00:40:14 omni2 fmd: [ID 377184 daemon.error] SUNW-MSG-ID:
> ZFS-8000-FD, TYPE: Fault, VER: 1, SEVERITY: Major
> Jun 29 00:40:14 omni2 EVENT-TIME: Mon Jun 29 00:40:14 CEST 2015
> Jun 29 00:40:14 omni2 PLATFORM: Precision-WorkStation-T5500, CSN: BCLJ55J,
> HOSTNAME: omni2
> Jun 29 00:40:14 omni2 SOURCE: zfs-diagnosis, REV: 1.0
> Jun 29 00:40:14 omni2 EVENT-ID: e44ba921-004f-61f8-cbdf-8f1ebf0d57c0
> Jun 29 00:40:14 omni2 DESC: The number of I/O errors associated with a ZFS
> device exceeded
> Jun 29 00:40:14 omni2        acceptable levels.  Refer to
> http://illumos.org/msg/ZFS-8000-FD for more information.
> Jun 29 00:40:14 omni2 AUTO-RESPONSE: The device has been offlined and
> marked as faulted.  An attempt
> Jun 29 00:40:14 omni2        will be made to activate a hot spare if
> available.
> Jun 29 00:40:14 omni2 IMPACT: Fault tolerance of the pool may be
> compromised.
> Jun 29 00:40:14 omni2 REC-ACTION: Run 'zpool status -x' and replace the
> bad device.
>
>
>
> From that it looks like zfs hinting that it is the device, not the port...
>
>
>
>
> cfgadm -al:
>
> root at omni2:/root# cfgadm -al
> Ap_Id                          Type         Receptacle   Occupant
>  Condition
> c3                             scsi-sas     connected    unconfigured
> unknown
> c5                             scsi-sas     connected    configured
>  unknown
> c5::w5000c50078e5135e,0        disk-path    connected    configured
>  unknown
> c8                             scsi-sas     connected    configured
>  unknown
> c8::w5000c5007ffee30b,0        disk-path    connected    configured
>  unknown
> c9                             scsi-sas     connected    configured
>  unknown
> c9::w500a0751034af6dc,0        disk-path    connected    configured
>  unknown
> sata1/0::dsk/c2t0d0            disk         connected    configured   ok
> sata1/1                        sata-port    disconnected unconfigured
> failed
> sata1/2                        sata-port    empty        unconfigured ok
> sata1/3                        sata-port    empty        unconfigured ok
> sata1/4                        sata-port    empty        unconfigured ok
> sata1/5                        sata-port    empty        unconfigured ok
>
>
> But with cfgadm I get unsure again...
>
> Someone know...?
>
>
> Best regards from/Med v?nliga h?lsningar fr?n
>
> Johan Kragsterman
>
> Capvert
>
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150702/d6645ace/attachment-0001.html>

From johan.kragsterman at capvert.se  Thu Jul  2 15:03:38 2015
From: johan.kragsterman at capvert.se (Johan Kragsterman)
Date: Thu, 2 Jul 2015 17:03:38 +0200
Subject: [OmniOS-discuss] Ang: Re:  SSD rpool degraded
In-Reply-To: <CAJ5wh+7BAE-zKDHZxzViLLxod2bFd+iUw64-oTQSR=Gd=ni7qw@mail.gmail.com>
References: <CAJ5wh+7BAE-zKDHZxzViLLxod2bFd+iUw64-oTQSR=Gd=ni7qw@mail.gmail.com>,
	<OFABACE78C.9AA09E81-ONC1257E76.004B4A1F-C1257E76.004C7537@inse.com>
Message-ID: <OF0E1A47A6.A2CC56F2-ONC1257E76.00522E1F-C1257E76.0052BB22@inse.com>


Hi!


It is only rpool, no data, but of coarse configuration data. So there are no write-intensive applications writing to the drive, only log. Not even swap or dump, they are on different devices.

It would be nice to know if there was a way to tell wether it is the drive or the port, though...


Best regards from/Med v?nliga h?lsningar fr?n

Johan Kragsterman

Capvert


-----Yavor Tomov <yavoritomov at gmail.com> skrev: -----
Till: Johan Kragsterman <johan.kragsterman at capvert.se>
Fr?n: Yavor Tomov <yavoritomov at gmail.com>
Datum: 2015-07-02 16:41
Kopia: omnios-discuss at lists.omniti.com
?rende: Re: [OmniOS-discuss] SSD rpool degraded

First make sure you have a backup of your data. SSD are famous of failing at the same time if they were installed at the same time. Also if you are using the pool by write intensive application SSDs can die quick. Depending on the drive you should be able to look at the Erase Count attribute from the SMART information. The value should be above a 100, if under some data loss is possible, if "0" the ssd will stop working, all this dependence on the manufacture. The easiest thing will be to replace the drive and see if the errors come back.?

Good Luck
Yavor Tomov

On Thu, Jul 2, 2015 at 8:55 AM, Johan Kragsterman <johan.kragsterman at capvert.se> wrote:

Hi!

I got a degraded rpool, consisting of a mirror of two SSD's. I feel unsure about if it really is the SSD that have failed, since it is enterprise grade and haven't been running that long

I would like to know if there is a way to figure out wether it is the SATA port or the SSD that have failed.

The zpool status looks like this:

NAME? ? ? ? ? STATE? ? ?READ WRITE CKSUM
? ? ? ? rpool? ? ? ? ?DEGRADED? ? ?0? ? ?0? ? ?0
? ? ? ? ? mirror-0? ? DEGRADED? ? ?0? ? ?0? ? ?0
? ? ? ? ? ? c2t0d0s0? ONLINE? ? ? ?0? ? ?0? ? ?0
? ? ? ? ? ? c2t1d0s0? FAULTED? ? ? 1? ?191? ? ?0? too many errors



dmesg containes this:

Jun 29 00:39:47 omni2 genunix: [ID 517647 kern.warning] WARNING: ahci0: watchdog port 1 satapkt 0xffffff065eb76860 timed out
Jun 29 00:39:58 omni2 genunix: [ID 860969 kern.warning] WARNING: ahci0: ahci_port_reset port 1 the device hardware has been initialized and the power-up diagnostics failed
Jun 29 00:39:59 omni2 genunix: [ID 801845 kern.info] /pci at 0,0/pci1028,26e at 1f,2:
Jun 29 00:39:59 omni2? SATA port 1 error
Jun 29 00:39:59 omni2 genunix: [ID 801845 kern.info] /pci at 0,0/pci1028,26e at 1f,2:
Jun 29 00:39:59 omni2? SATA port 1 error
Jun 29 00:39:59 omni2 genunix: [ID 801845 kern.info] /pci at 0,0/pci1028,26e at 1f,2:
Jun 29 00:39:59 omni2? SATA port 1 error
Jun 29 00:39:59 omni2 genunix: [ID 801845 kern.info] /pci at 0,0/pci1028,26e at 1f,2:
Jun 29 00:39:59 omni2? SATA port 1 error
Jun 29 00:39:59 omni2 genunix: [ID 801845 kern.info] /pci at 0,0/pci1028,26e at 1f,2:
Jun 29 00:39:59 omni2? SATA port 1 error
Jun 29 00:39:59 omni2 genunix: [ID 801845 kern.info] /pci at 0,0/pci1028,26e at 1f,2:
Jun 29 00:39:59 omni2? SATA port 1 error
Jun 29 00:39:59 omni2 genunix: [ID 801845 kern.info] /pci at 0,0/pci1028,26e at 1f,2:
Jun 29 00:39:59 omni2? SATA port 1 error
Jun 29 00:39:59 omni2 genunix: [ID 801845 kern.info] /pci at 0,0/pci1028,26e at 1f,2:
Jun 29 00:39:59 omni2? SATA port 1 error
Jun 29 00:39:59 omni2 genunix: [ID 801845 kern.info] /pci at 0,0/pci1028,26e at 1f,2:
Jun 29 00:39:59 omni2? SATA port 1 error
Jun 29 00:39:59 omni2 genunix: [ID 801845 kern.info] /pci at 0,0/pci1028,26e at 1f,2:
Jun 29 00:39:59 omni2? SATA port 1 error
Jun 29 00:39:59 omni2 genunix: [ID 801845 kern.info] /pci at 0,0/pci1028,26e at 1f,2:
Jun 29 00:39:59 omni2? SATA port 1 error
Jun 29 00:40:14 omni2 fmd: [ID 377184 daemon.error] SUNW-MSG-ID: ZFS-8000-FD, TYPE: Fault, VER: 1, SEVERITY: Major
Jun 29 00:40:14 omni2 EVENT-TIME: Mon Jun 29 00:40:14 CEST 2015
Jun 29 00:40:14 omni2 PLATFORM: Precision-WorkStation-T5500, CSN: BCLJ55J, HOSTNAME: omni2
Jun 29 00:40:14 omni2 SOURCE: zfs-diagnosis, REV: 1.0
Jun 29 00:40:14 omni2 EVENT-ID: e44ba921-004f-61f8-cbdf-8f1ebf0d57c0
Jun 29 00:40:14 omni2 DESC: The number of I/O errors associated with a ZFS device exceeded
Jun 29 00:40:14 omni2? ? ? ? acceptable levels.? Refer to http://illumos.org/msg/ZFS-8000-FD for more information.
Jun 29 00:40:14 omni2 AUTO-RESPONSE: The device has been offlined and marked as faulted.? An attempt
Jun 29 00:40:14 omni2? ? ? ? will be made to activate a hot spare if available.
Jun 29 00:40:14 omni2 IMPACT: Fault tolerance of the pool may be compromised.
Jun 29 00:40:14 omni2 REC-ACTION: Run 'zpool status -x' and replace the bad device.



>From that it looks like zfs hinting that it is the device, not the port...




cfgadm -al:

root at omni2:/root# cfgadm -al
Ap_Id? ? ? ? ? ? ? ? ? ? ? ? ? Type? ? ? ? ?Receptacle? ?Occupant? ? ?Condition
c3? ? ? ? ? ? ? ? ? ? ? ? ? ? ?scsi-sas? ? ?connected? ? unconfigured unknown
c5? ? ? ? ? ? ? ? ? ? ? ? ? ? ?scsi-sas? ? ?connected? ? configured? ?unknown
c5::w5000c50078e5135e,0? ? ? ? disk-path? ? connected? ? configured? ?unknown
c8? ? ? ? ? ? ? ? ? ? ? ? ? ? ?scsi-sas? ? ?connected? ? configured? ?unknown
c8::w5000c5007ffee30b,0? ? ? ? disk-path? ? connected? ? configured? ?unknown
c9? ? ? ? ? ? ? ? ? ? ? ? ? ? ?scsi-sas? ? ?connected? ? configured? ?unknown
c9::w500a0751034af6dc,0? ? ? ? disk-path? ? connected? ? configured? ?unknown
sata1/0::dsk/c2t0d0? ? ? ? ? ? disk? ? ? ? ?connected? ? configured? ?ok
sata1/1? ? ? ? ? ? ? ? ? ? ? ? sata-port? ? disconnected unconfigured failed
sata1/2? ? ? ? ? ? ? ? ? ? ? ? sata-port? ? empty? ? ? ? unconfigured ok
sata1/3? ? ? ? ? ? ? ? ? ? ? ? sata-port? ? empty? ? ? ? unconfigured ok
sata1/4? ? ? ? ? ? ? ? ? ? ? ? sata-port? ? empty? ? ? ? unconfigured ok
sata1/5? ? ? ? ? ? ? ? ? ? ? ? sata-port? ? empty? ? ? ? unconfigured ok


But with cfgadm I get unsure again...

Someone know...?


Best regards from/Med v?nliga h?lsningar fr?n

Johan Kragsterman

Capvert

_______________________________________________
OmniOS-discuss mailing list
OmniOS-discuss at lists.omniti.com
http://lists.omniti.com/mailman/listinfo/omnios-discuss




From daleg at omniti.com  Thu Jul  2 15:37:20 2015
From: daleg at omniti.com (Dale Ghent)
Date: Thu, 2 Jul 2015 11:37:20 -0400
Subject: [OmniOS-discuss] Ang: Re:  SSD rpool degraded
In-Reply-To: <OF0E1A47A6.A2CC56F2-ONC1257E76.00522E1F-C1257E76.0052BB22@inse.com>
References: <CAJ5wh+7BAE-zKDHZxzViLLxod2bFd+iUw64-oTQSR=Gd=ni7qw@mail.gmail.com>
	<OFABACE78C.9AA09E81-ONC1257E76.004B4A1F-C1257E76.004C7537@inse.com>
	<OF0E1A47A6.A2CC56F2-ONC1257E76.00522E1F-C1257E76.0052BB22@inse.com>
Message-ID: <224BA774-B09F-4EB9-AA3D-7B6431EBA216@omniti.com>


> On Jul 2, 2015, at 11:03 AM, Johan Kragsterman <johan.kragsterman at capvert.se> wrote:
> 
> It is only rpool, no data, but of coarse configuration data. So there are no write-intensive applications writing to the drive, only log. Not even swap or dump, they are on different devices.
> 
> It would be nice to know if there was a way to tell wether it is the drive or the port, though...

Considering the nature of the error message, you will have to start with the drive. The drive, at least, can send telemetry. A physical port cannot. Make sure your cables are seated correctly and so on. The AHCI driver sent a command to power on the drive and initialize it, and it did not receive the expected response.

Drives (enterprise-grade or not) will fail very young (due to manufacturing defect) and, if they survive their first months in operation, they can go for years before failing. You might have a drive that fell out of the crib, or just a bad cable/connector. Sometimes these things take some hands-on detective work.

/dale
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 494 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150702/7b8cd7b9/attachment.bin>

From danmcd at omniti.com  Thu Jul  2 15:59:42 2015
From: danmcd at omniti.com (Dan McDonald)
Date: Thu, 2 Jul 2015 11:59:42 -0400
Subject: [OmniOS-discuss] pkg(5) in alternate roots
In-Reply-To: <fc788a8932ba.5595218b@cos.ru>
References: <fc776a65623.55940558@cos.ru>
	<A941A78E-B723-4386-B215-799DAC44F8E1@omniti.com>
	<fc788a8932ba.5595218b@cos.ru>
Message-ID: <A9A9066A-B8DB-4E6A-861C-BFAD79156A2F@omniti.com>


> On Jul 2, 2015, at 5:33 AM, Jim Klimov <jim at cos.ru> wrote:
> 
> Most of the output was in the screenshot just above that, and I guess I found a workaround 
> that should suffice (will test more today), but for the sake of completeness, here goes:

Thank you.

Dan


From ryan at zinascii.com  Fri Jul  3 15:29:48 2015
From: ryan at zinascii.com (Ryan Zezeski)
Date: Fri, 03 Jul 2015 11:29:48 -0400
Subject: [OmniOS-discuss] zpool list -p
In-Reply-To: <22AE6795-CD87-4FAA-B9BB-3C173012C7B7@licenser.net>
References: <1E73E845-FD72-496D-9E7C-650580B7E305@licenser.net>
	<CAED-Y_DQx0QE0VRqjOe6Cr4Xr2AffXPu63iCHv2_kDVif-mpYA@mail.gmail.com>
	<22AE6795-CD87-4FAA-B9BB-3C173012C7B7@licenser.net>
Message-ID: <m2pp4945ub.fsf@zinascii.com>


Heinz Nikolaus Gies writes:

> zpool upgrade -v shows the same version on both systems.
>
> I would suspect that Joyent has modified the zpool utility, but it seems like a sensible change.
>

This is indeed a local modification in SmartOS's libzfs.

/// START DIFF ///
diff -ur omniti-labs/illumos-omnios/usr/src/lib/libzfs/common/libzfs_pool.c joyent/illumos-joyent/usr/src/lib/libzfs/common/libzfs_pool.c
--- omniti-labs/illumos-omnios/usr/src/lib/libzfs/common/libzfs_pool.c  2015-07-03 10:33:48.000000000 -0400
+++ joyent/illumos-joyent/usr/src/lib/libzfs/common/libzfs_pool.c       2015-03-03 12:27:24.000000000 -0500
@@ -270,6 +270,12 @@
        case PROP_TYPE_NUMBER:
                intval = zpool_get_prop_int(zhp, prop, &src);

+               if (literal && prop != ZPOOL_PROP_HEALTH) {
+                       (void) snprintf(buf, len, "%llu",
+                           (u_longlong_t)intval);
+                       break;
+               }
+
                switch (prop) {
                case ZPOOL_PROP_SIZE:
                case ZPOOL_PROP_ALLOCATED:
/// END DIFF ///

This short-circuits all properties except for ZPOOL_PROP_HEALTH when the
literal flag is true (e.g. when passing -p to zpool).

-Z

>
>
>> On May 30, 2015, at 19:20, Krzysztof Grzempa <grzempek at gmail.com> wrote:
>> 
>> Did you compare ZFS versions on both OS'es ? This might changed over some new version..
>> 
>> 2015-05-30 17:58 GMT+02:00 Heinz Nikolaus Gies <heinz at licenser.net <mailto:heinz at licenser.net>>:
>> I was looking at the output of zpool list today, comparing it with what I?d get on SmartOS and noticed that when using the -p flag for parable output the deduplication factor is still presented as a string (or floatish type) instead of a integer vlaue. It seems to me a bit wrong for parable output.
>> 
>> If there is a reason behind that decision it?s fine and I?ll gladly work around it, but it feels like an oversight.
>> 
>> 
>> Cheers,
>> Heinz
>> 
>> Here a quick glance:
>> 
>> OmniOS:
>> /usr/sbin/zpool list -pH -oname,size,alloc,free,dedup,health
>> data    7971459301376   6405101887488   1566357413888   1.00x   ONLINE
>> rpool   249108103168    121560741376    127547361792    1.00x   ONLINE
>> 
>> SmartOS:
>> list -pH -oname,size,alloc,free,dedup,health
>> zones   319975063552    51935040512     268040023040    100     ONLINE
>> ---
>> Cheers,
>> Heinz Nikolaus Gies
>> heinz at licenser.net <mailto:heinz at licenser.net>
>> 
>> 
>> 
>> 
>> _______________________________________________
>> OmniOS-discuss mailing list
>> OmniOS-discuss at lists.omniti.com <mailto:OmniOS-discuss at lists.omniti.com>
>> http://lists.omniti.com/mailman/listinfo/omnios-discuss <http://lists.omniti.com/mailman/listinfo/omnios-discuss>
>> 
>> 
>
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss


From omnios at citrus-it.net  Thu Jul  9 11:23:55 2015
From: omnios at citrus-it.net (Andy Fiddaman)
Date: Thu, 9 Jul 2015 11:23:55 +0000 (UTC)
Subject: [OmniOS-discuss] [developer] HEADS UP -- illumos-gate can now
 be built on OmniOS r151014 or later
In-Reply-To: <D8F62323-340B-4334-8BBC-9DE7198ADAC9@omniti.com>
References: <D8F62323-340B-4334-8BBC-9DE7198ADAC9@omniti.com>
Message-ID: <alpine.GSO.2.00.1507091040020.2334@erncre.pvgehf-vg.arg>


On Tue, 5 May 2015, Dan McDonald wrote:

; With the pushes of 4719, 5878, and 5879, one may now build stock illumos-gate
; on OmniOS, revisions r151014 or later.

Dan,

I'm finally getting around to trying this on one my r151014 zones - good
timing since my OI KVM instance isn't booting any more for some reason and
I have a few sets of mr_sas patches I want to push.

It's mostly looking good but I'm getting errors in mail_msg for openssl
components. Do you know if this s something to do with my environment
or a problem with the gate at the moment?

Thanks,

Andy


==== Nightly distributed build started:   Wed Jul  8 21:44:56 UTC 2015 ====
==== Nightly distributed build completed: Wed Jul  8 22:42:56 UTC 2015 ====

==== Total build time ====

real    0:57:59

==== Build environment ====

/usr/bin/uname
SunOS build 5.11 omnios-170cea2 i86pc i386 i86pc

/opt/SUNWspro/bin/dmake
dmake: Sun Distributed Make 7.9 SunOS_i386 2009/06/03
number of concurrent jobs = 4

32-bit compiler
/opt/onbld/bin/i386/cw -_gcc
cw version 1.30 (SHADOW MODE DISABLED)
primary: /opt/gcc-4.4.4//bin/gcc
gcc (GCC) 4.4.4

64-bit compiler
/opt/onbld/bin/i386/cw -_gcc
cw version 1.30 (SHADOW MODE DISABLED)
primary: /opt/gcc-4.4.4//bin/gcc
gcc (GCC) 4.4.4

/usr/java/bin/javac
openjdk full version "1.7.0_76-b31"

/usr/bin/as
as: Sun Compiler Common 12 SunOS_i386 snv_121 08/03/2009

/usr/ccs/bin/ld
ld: Software Generation Utilities - Solaris Link Editors: 5.11-1.1750 (illumos)

Build project:  user.root
Build taskid:   450834

==== Nightly argument issues ====


==== Build version ====

illumos-gate

==== Make clobber ERRORS ====


==== Make tools clobber ERRORS ====


==== Tools build errors ====


==== Build errors (non-DEBUG) ====

The following command caused the error:
dmake: Warning: Don't know how to make target `/root/illumos/illumos-gate/proto/root_i386/licenses/usr/src/common/openssl/LICENSE'
dmake: Warning: Don't know how to make target `/root/illumos/illumos-gate/proto/root_i386/licenses/usr/src/common/openssl/LICENSE.descrip'
dmake: Warning: Target `/root/illumos/illumos-gate/proto/root_i386/licenses/usr/src/cmd/acpihpd/THIRDPARTYLICENSE' not remade because of errors
The following command caused the error:
dmake: Warning: Target `stage-licenses' not remade because of errors
The following command caused the error:
dmake: Warning: Target `install' not remade because of errors

==== Build warnings (non-DEBUG) ====

dmake: Warning: Don't know how to make target `/root/illumos/illumos-gate/proto/root_i386/licenses/usr/src/common/openssl/LICENSE'
dmake: Warning: Don't know how to make target `/root/illumos/illumos-gate/proto/root_i386/licenses/usr/src/common/openssl/LICENSE.descrip'
dmake: Warning: Target `/root/illumos/illumos-gate/proto/root_i386/licenses/usr/src/cmd/acpihpd/THIRDPARTYLICENSE' not remade because of errors
dmake: Warning: Target `stage-licenses' not remade because of errors
dmake: Warning: Target `install' not remade because of errors

==== Elapsed build time (non-DEBUG) ====

real    18:27.1
user  1:29:38.5
sys     11:02.9

==== Build noise differences (non-DEBUG) ====


==== Build errors (DEBUG) ====


==== Build warnings (DEBUG) ====


==== Elapsed build time (DEBUG) ====

real    15:45.3
user  1:21:10.0
sys      8:52.1

==== Build noise differences (DEBUG) ====


==== package build errors (DEBUG) ====


==== 'dmake lint' of src ERRORS ====


==== Elapsed time of 'dmake lint' of src ====

real    15:32.1
user    44:44.1
sys      6:02.6

==== lint warnings src ====


==== lint noise differences src ====


==== cstyle/hdrchk errors ====


==== Find core files ====


==== Check lists of files ====


-- 
Citrus IT Limited | +44 (0)870 199 8000 | enquiries at citrus-it.co.uk
Rock House Farm | Green Moor | Wortley | Sheffield | S35 7DQ
Registered in England and Wales | Company number 4899123


From danmcd at omniti.com  Thu Jul  9 14:10:54 2015
From: danmcd at omniti.com (Dan McDonald)
Date: Thu, 9 Jul 2015 10:10:54 -0400
Subject: [OmniOS-discuss] [developer] HEADS UP -- illumos-gate can now
	be built on OmniOS r151014 or later
In-Reply-To: <alpine.GSO.2.00.1507091040020.2334@erncre.pvgehf-vg.arg>
References: <D8F62323-340B-4334-8BBC-9DE7198ADAC9@omniti.com>
	<alpine.GSO.2.00.1507091040020.2334@erncre.pvgehf-vg.arg>
Message-ID: <EB7EAE73-93BD-446B-9B9F-3A0DE2C26992@omniti.com>


> On Jul 9, 2015, at 7:23 AM, Andy Fiddaman <omnios at citrus-it.net> wrote:


> ==== Build errors (non-DEBUG) ====
> 
> The following command caused the error:
> dmake: Warning: Don't know how to make target `/root/illumos/illumos-gate/proto/root_i386/licenses/usr/src/common/openssl/LICENSE'
> dmake: Warning: Don't know how to make target `/root/illumos/illumos-gate/proto/root_i386/licenses/usr/src/common/openssl/LICENSE.descrip'
> dmake: Warning: Target `/root/illumos/illumos-gate/proto/root_i386/licenses/usr/src/cmd/acpihpd/THIRDPARTYLICENSE' not remade because of errors
> The following command caused the error:
> dmake: Warning: Target `stage-licenses' not remade because of errors
> The following command caused the error:
> dmake: Warning: Target `install' not remade because of errors
> 
> ==== Build warnings (non-DEBUG) ====
> 
> dmake: Warning: Don't know how to make target `/root/illumos/illumos-gate/proto/root_i386/licenses/usr/src/common/openssl/LICENSE'
> dmake: Warning: Don't know how to make target `/root/illumos/illumos-gate/proto/root_i386/licenses/usr/src/common/openssl/LICENSE.descrip'
> dmake: Warning: Target `/root/illumos/illumos-gate/proto/root_i386/licenses/usr/src/cmd/acpihpd/THIRDPARTYLICENSE' not remade because of errors
> dmake: Warning: Target `stage-licenses' not remade because of errors
> dmake: Warning: Target `install' not remade because of errors
> 
> ==== Elapsed build time (non-DEBUG) ====

Are you missing these files?  That's the explanation off the top of my head.

Otherwise I'd have to look at the whole nightly.log, and I won't be able to do that at all today, possibly this week.

Dan


From danmcd at omniti.com  Thu Jul  9 14:39:56 2015
From: danmcd at omniti.com (Dan McDonald)
Date: Thu, 9 Jul 2015 10:39:56 -0400
Subject: [OmniOS-discuss] OpenSSL updates
Message-ID: <362A29DC-5589-425A-A878-0F127DA6E0E4@omniti.com>

Bloody and 014 are now up to OpenSSL 1.0.2d.

012 and 006 are now up to OpenSSL 1.0.1p.

Please utter "pkg update" ASAP, and as an added measure, restart any openssl-using services.

Thanks,
Dan


From nsmith at careyweb.com  Thu Jul  9 15:23:13 2015
From: nsmith at careyweb.com (Nate Smith)
Date: Thu, 9 Jul 2015 11:23:13 -0400
Subject: [OmniOS-discuss] pkg update failed due to full root system (which
	isn't full)
Message-ID: <656d2d4c-6a86-47e8-9ff4-89d6f6568c06@careyweb.com>

So I went to run a yum update..

Last login: Tue Jun 30 14:01:50 2015 from 10.1.0.10
OmniOS 5.11     omnios-10b9c79  September 2014
root at host:/root# pkg update
            Packages to update: 307
       Create boot environment: Yes
Create backup boot environment:  No

pkg: Insufficent disk space available (426.36 MB)for estimated need (476.64 MB) for Root filesystem

So I did a df and this is my first line

root at storm3:/etc# df -k
Filesystem                                                    1K-blocks       Used   Available Use% Mounted on
/usr/lib/libc/libc_hwcap1.so.1                                  4525952    4089451      436501  91% /lib/libc.so.1
swap                                                            6103004        312     6102692   1% /etc/svc/volatile
swap                                                            6106932       4240     6102692   1% /tmp
swap                                                            6102744         52     6102692   1% /var/run
rpool/export                                                     436520         19      436501   1% /export
rpool/export/home                                                436520         19      436501   1% /export/home
rpool                                                            436527         26      436501   1% /rpool

What is with that first line? Anyone ever seen this before?

Here's my vfstab

root at host:/etc# cat vfstab 
#device         device          mount           FS      fsck    mount   mount
#to mount       to fsck         point           type    pass    at boot options
#
/devices        -               /devices        devfs   -       no      -
/proc           -               /proc           proc    -       no      -
ctfs            -               /system/contract ctfs   -       no      -
objfs           -               /system/object  objfs   -       no      -
sharefs         -               /etc/dfs/sharetab       sharefs -       no      -
fd              -               /dev/fd         fd      -       no      -
swap            -               /tmp            tmpfs   -       yes     -

Baffled here. Any help appreciated.

-Nate



From jimklimov at cos.ru  Thu Jul  9 16:38:39 2015
From: jimklimov at cos.ru (Jim Klimov)
Date: Thu, 09 Jul 2015 18:38:39 +0200
Subject: [OmniOS-discuss] pkg update failed due to full root system
	(which	isn't full)
In-Reply-To: <656d2d4c-6a86-47e8-9ff4-89d6f6568c06@careyweb.com>
References: <656d2d4c-6a86-47e8-9ff4-89d6f6568c06@careyweb.com>
Message-ID: <82715C36-3249-4A8C-9C26-44AFE29D6163@cos.ru>

9 ???? 2015??. 17:23:13 CEST, Nate Smith <nsmith at careyweb.com> ?????:
>So I went to run a yum update..
>
>Last login: Tue Jun 30 14:01:50 2015 from 10.1.0.10
>OmniOS 5.11     omnios-10b9c79  September 2014
>root at host:/root# pkg update
>            Packages to update: 307
>       Create boot environment: Yes
>Create backup boot environment:  No
>
>pkg: Insufficent disk space available (426.36 MB)for estimated need
>(476.64 MB) for Root filesystem
>
>So I did a df and this is my first line
>
>root at storm3:/etc# df -k
>Filesystem                                                    1K-blocks
>      Used   Available Use% Mounted on
>/usr/lib/libc/libc_hwcap1.so.1                                  4525952
>   4089451      436501  91% /lib/libc.so.1
>swap                                                            6103004
>       312     6102692   1% /etc/svc/volatile
>swap                                                            6106932
>      4240     6102692   1% /tmp
>swap                                                            6102744
>        52     6102692   1% /var/run
>rpool/export                                                     436520
>        19      436501   1% /export
>rpool/export/home                                                436520
>        19      436501   1% /export/home
>rpool                                                            436527
>        26      436501   1% /rpool
>
>What is with that first line? Anyone ever seen this before?
>
>Here's my vfstab
>
>root at host:/etc# cat vfstab 
>#device         device          mount           FS      fsck    mount  
>mount
>#to mount       to fsck         point           type    pass    at boot
>options
>#
>/devices        -               /devices        devfs   -       no     
>-
>/proc           -               /proc           proc    -       no     
>-
>ctfs            -               /system/contract ctfs   -       no     
>-
>objfs           -               /system/object  objfs   -       no     
>-
>sharefs         -               /etc/dfs/sharetab       sharefs -      
>no      -
>fd              -               /dev/fd         fd      -       no     
>-
>swap            -               /tmp            tmpfs   -       yes    
>-
>
>Baffled here. Any help appreciated.
>
>-Nate
>
>
>_______________________________________________
>OmniOS-discuss mailing list
>OmniOS-discuss at lists.omniti.com
>http://lists.omniti.com/mailman/listinfo/omnios-discuss

Your rpool available space (end of listing) is indeed small.
The hwcap line is an overlaid library variant best optimized for the hardware you have.

Jim
--
Typos courtesy of K-9 Mail on my Samsung Android

From venture37 at gmail.com  Thu Jul  9 20:53:42 2015
From: venture37 at gmail.com (Sevan / Venture37)
Date: Thu, 9 Jul 2015 21:53:42 +0100
Subject: [OmniOS-discuss] gettext 0.18.3.1 on OmniOS: libgomp.so.1 not
	found
In-Reply-To: <20130924.231808.1311489290245807538.ryo_on@yk.rim.or.jp>
References: <20130922.072119.1528732886952694589.ryo_on@yk.rim.or.jp>
	<l1m2e9$nq4$1@ger.gmane.org>
	<20130924.231808.1311489290245807538.ryo_on@yk.rim.or.jp>
Message-ID: <CA+U3Mf5FTx6zKVYrk_px+eZ=TWh-N_LLYnHLsRw=nkLr_GCe5A@mail.gmail.com>

On 24 September 2013 at 15:18, Ryo ONODERA <ryo_on at yk.rim.or.jp> wrote:
> As you expected, my real problem is seen in pkgsrc build.
> I do not wat to use PREFER_PKGSRC=yes, so I will use crle command.

As a temporary workaround I went down the crle route
crle -c /var/ld/ld.config -l /lib:/usr/lib:/opt/gcc-4.8.1/lib

With that, I was able to build databases/shared-mime-info which
previously failed at the configured stage due to
checking for gmsgfmt... /usr/bin/gmsgfmt
ld.so.1: xgettext: fatal: libgomp.so.1: open failed: No such file or directory
ld.so.1: msgmerge: fatal: libgomp.so.1: open failed: No such file or directory
ld.so.1: msgfmt: fatal: libgomp.so.1: open failed: No such file or directory
configure: error: GNU gettext tools not found; required for intltool

Causing 2306 to break.

Ideally the gettext package provided in omnios should have the correct
rpath passed to it during build so it's not an issue but before we get
to that, here's a yak called omnios-build which needs shaving and
documenting correctly because out of the box on a r151014 zone it's
incapable of creating the repo needed or talking to a pre-prepared
repo.


Sevan / Venture37

From lkateley at kateley.com  Thu Jul  9 21:21:03 2015
From: lkateley at kateley.com (Linda Kateley)
Date: Thu, 09 Jul 2015 16:21:03 -0500
Subject: [OmniOS-discuss] big zfs storage?
Message-ID: <559EE5BF.4040900@kateley.com>

Hey is there anyone out there running big zfs on omni?

I have been doing mostly zol and freebsd for the last year but have to 
build a 300+TB box and i want to come back home to roots(solaris). 
Feeling kind of hesitant :) Also, if you had to do over, is there 
anything you would do different.

Also, what is the go to HBA these days? Seems like i saw stable code for 
lsi 3008?

TIA

linda



From ikaufman at eng.ucsd.edu  Thu Jul  9 21:37:17 2015
From: ikaufman at eng.ucsd.edu (Ian Kaufman)
Date: Thu, 9 Jul 2015 14:37:17 -0700
Subject: [OmniOS-discuss] big zfs storage?
In-Reply-To: <559EE5BF.4040900@kateley.com>
References: <559EE5BF.4040900@kateley.com>
Message-ID: <CAPJtH1iVm+coAWBeDSfPicS4xRiXDUUu1F8Z_Zj9o1ts4jAs7g@mail.gmail.com>

Hi Linda,

I have two dual-headed OmniOS system attached to two 60 bay JBODs w/
3TB drives. Using raidz2 and hot spares, each JBOD has about 106TB
usable, and i use a single filesystem per JBOD. We may or may not add
another JBOD, but we will be expanding into the PB realm eventually. I
use napp-it to make things easier to manage, and I will start testing
some HA/Failover.

I also have two ~70TB usable systems.

Ian

On Thu, Jul 9, 2015 at 2:21 PM, Linda Kateley <lkateley at kateley.com> wrote:
> Hey is there anyone out there running big zfs on omni?
>
> I have been doing mostly zol and freebsd for the last year but have to build
> a 300+TB box and i want to come back home to roots(solaris). Feeling kind of
> hesitant :) Also, if you had to do over, is there anything you would do
> different.
>
> Also, what is the go to HBA these days? Seems like i saw stable code for lsi
> 3008?
>
> TIA
>
> linda
>
>
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss



-- 
Ian Kaufman
Research Systems Administrator
UC San Diego, Jacobs School of Engineering ikaufman AT ucsd DOT edu

From omnios at citrus-it.net  Thu Jul  9 22:31:41 2015
From: omnios at citrus-it.net (Andy Fiddaman)
Date: Thu, 9 Jul 2015 22:31:41 +0000 (UTC)
Subject: [OmniOS-discuss] [developer] HEADS UP -- illumos-gate can now
 be built on OmniOS r151014 or later
In-Reply-To: <EB7EAE73-93BD-446B-9B9F-3A0DE2C26992@omniti.com>
References: <D8F62323-340B-4334-8BBC-9DE7198ADAC9@omniti.com>
	<alpine.GSO.2.00.1507091040020.2334@erncre.pvgehf-vg.arg>
	<EB7EAE73-93BD-446B-9B9F-3A0DE2C26992@omniti.com>
Message-ID: <alpine.GSO.2.00.1507092230470.10796@erncre.pvgehf-vg.arg>


On Thu, 9 Jul 2015, Dan McDonald wrote:
; Are you missing these files?  That's the explanation off the top of my head.
;

No, they seem to be built and there too.

build# (7) ls -l
/root/illumos/illumos-gate/proto/root_i386/licenses/usr/src/common/openssl/LICENSE
-rw-r--r--   1 root     8190        6.1K Aug 18  2010
/root/illumos/illumos-gate/proto/root_i386/licenses/usr/src/common/openssl/LICENSE

I'll track it down - just hoped it would be something you'd seen before.

Thanks for the reply,

Andy
-- 
Citrus IT Limited | +44 (0)870 199 8000 | enquiries at citrus-it.co.uk
Rock House Farm | Green Moor | Wortley | Sheffield | S35 7DQ
Registered in England and Wales | Company number 4899123


From chip at innovates.com  Fri Jul 10 02:30:15 2015
From: chip at innovates.com (Schweiss, Chip)
Date: Thu, 9 Jul 2015 21:30:15 -0500
Subject: [OmniOS-discuss] big zfs storage?
In-Reply-To: <559EE5BF.4040900@kateley.com>
References: <559EE5BF.4040900@kateley.com>
Message-ID: <CALeZrrQ1nVjCFiTPEnDdpvCyHHDw7pm6z3tKH=678gHtR3c5tw@mail.gmail.com>

Linda,

I have 3.5 PB running under OmniOS.  All my systems have LSI 2108 HBAs
which is considered the best choice for HBAs.

Illumos leaves a bit to be desired with handling faults from disks or SAS
problems, but things under OmniOS have been improving, much thanks to Dan
McDonald and OmniTI.   We have a paid support on all of our production
systems with OmniTI.  Their response and dedication has been very good.
Other than the occasional panic and restart from a disk failure, OmniOS has
been solid.   ZFS of course never has lost a single bit of information.

I'd be curious why you're looking to move, have there been specific
problems under BSD or ZoL?  I've been slowly evaluating FreeBSD ZFS, but of
course the skeletons in the closet never seem to come out until you do
something big.

-Chip

On Thu, Jul 9, 2015 at 4:21 PM, Linda Kateley <lkateley at kateley.com> wrote:

> Hey is there anyone out there running big zfs on omni?
>
> I have been doing mostly zol and freebsd for the last year but have to
> build a 300+TB box and i want to come back home to roots(solaris). Feeling
> kind of hesitant :) Also, if you had to do over, is there anything you
> would do different.
>
> Also, what is the go to HBA these days? Seems like i saw stable code for
> lsi 3008?
>
> TIA
>
> linda
>
>
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150709/c3cb5160/attachment-0001.html>

From omnios at citrus-it.net  Fri Jul 10 13:39:54 2015
From: omnios at citrus-it.net (Andy Fiddaman)
Date: Fri, 10 Jul 2015 13:39:54 +0000 (UTC)
Subject: [OmniOS-discuss] [developer] HEADS UP -- illumos-gate can now
 be built on OmniOS r151014 or later
In-Reply-To: <EB7EAE73-93BD-446B-9B9F-3A0DE2C26992@omniti.com>
References: <D8F62323-340B-4334-8BBC-9DE7198ADAC9@omniti.com>
	<alpine.GSO.2.00.1507091040020.2334@erncre.pvgehf-vg.arg>
	<EB7EAE73-93BD-446B-9B9F-3A0DE2C26992@omniti.com>
Message-ID: <alpine.GSO.2.00.1507101329510.19413@erncre.pvgehf-vg.arg>


On Thu, 9 Jul 2015, Dan McDonald wrote:

; Are you missing these files?  That's the explanation off the top of my head.
;
; Otherwise I'd have to look at the whole nightly.log, and I won't be able to do that at all today, possibly this week.

I am not getting much further with finding the root cause of this. It does
seem to be a fault in the build process but in that case I don't understand
why I don't see it when building on OI. I might have to try and recover my
OI machine to compare.

Here's what I've found so far:

Once nightly is finished, the OpenSSL licence file is present in
proto/root_i386/licenses/usr/src/common/openssl/LICENSE

and the nightly log says that it failed to build it once then didn't need
to after that:

build-env# (154) grep openssl/LICEN ~/log/log.2015-07-10.12:47/nightly.log
dmake: Warning: Don't know how to make target `/root/illumos/illumos-gate/proto/root_i386/licenses/usr/src/common/openssl/LICENSE'
make: Warning: Don't know how to make target `/root/illumos/illumos-gate/proto/root_i386/licenses/usr/src/common/openssl/LICENSE.descrip'
`/root/illumos/illumos-gate/proto/root_i386/licenses/usr/src/common/openssl/LICENSE' is up to date.
`/root/illumos/illumos-gate/proto/root_i386/licenses/usr/src/common/openssl/LICENSE.descrip' is up to date.
`/root/illumos/illumos-gate/proto/root_i386/licenses/usr/src/common/openssl/LICENSE' is up to date.
`/root/illumos/illumos-gate/proto/root_i386/licenses/usr/src/common/openssl/LICENSE.descrip' is up to date.

If I run the make command that builds the file by hand, it says it is
up-to-date:

build-env# (146) pwd
/root/illumos/illumos-gate/usr/src/pkg
build-env# (147) make -e -f Makefile.lic \
    PKGDEBUG= LICROOT=$PARENT_WS/proto/root_i386/licenses \
    $PARENT_WS/proto/root_i386/licenses/usr/src/common/openssl/LICENSE
`/root/illumos/illumos-gate/proto/root_i386/licenses/usr/src/common/openssl/LICENSE' is up to date.

but if I remove it and attempt to rebuild it:

build-env# (148) rm /root/illumos/illumos-gate/proto/root_i386/licenses/usr/src/common/openssl/LICENSE
build-env# (149) make -e -f Makefile.lic \
    PKGDEBUG= LICROOT=$PARENT_WS/proto/root_i386/licenses \
    $PARENT_WS/proto/root_i386/licenses/usr/src/common/openssl/LICENSE
make: Fatal error: Don't know how to make target `/root/illumos/illumos-gate/proto/root_i386/licenses/usr/src/common/openssl/LICENSE'

This is because the stage-licenses target just tries to copy the licence
file from the usr/src tree and there is no usr/src/common/openssl in the
gate.

So I understand why it is failing but not yet what process ends up creating
proto/root_i386/licenses/usr/src/common/openssl/LICENSE nor why I am not
getting a clean build here. It looks like it might be the order in which
things are built - it would be interesting to see the output of
grep openssl/LIC from someone else's nightly.log to compare.

I'll keep digging but any help appreciated,

Thanks,

Andy

-- 
Citrus IT Limited | +44 (0)870 199 8000 | enquiries at citrus-it.co.uk
Rock House Farm | Green Moor | Wortley | Sheffield | S35 7DQ
Registered in England and Wales | Company number 4899123


From chip at innovates.com  Fri Jul 10 17:31:13 2015
From: chip at innovates.com (Schweiss, Chip)
Date: Fri, 10 Jul 2015 12:31:13 -0500
Subject: [OmniOS-discuss] big zfs storage?
In-Reply-To: <559FF4DE.4040202@kateley.com>
References: <559EE5BF.4040900@kateley.com>
	<CALeZrrQ1nVjCFiTPEnDdpvCyHHDw7pm6z3tKH=678gHtR3c5tw@mail.gmail.com>
	<559FF4DE.4040202@kateley.com>
Message-ID: <CALeZrrTSMhsD0K4Vw1khAVE=XDtThJ21kyGk-5DHvy5A_oBuug@mail.gmail.com>

Unfortunately for the past couple years panics on disk failure has been the
norm.   All my production systems are HA with RSF-1, so at least things
come back online relatively quick.  There are quite a few open tickets in
the Illumos bug tracker related to mpt_sas related panics.

Most of the work to fix these problems has been committed in the past year,
though problems still exist.  For example, my systems are dual path SAS,
however, mpt_sas will panic if you pull a cable instead of dropping a path
to the disks.  Dan McDonald is actively working to resolve this.   He is
also pushing a bug fix in genunix from Nexenta that appears to fix a lot of
the panic problems.   I'll know for sure in a few months after I see a disk
or two drop if it truly fixes things.  Hans Rosenfeld at Nexenta is
responsible for most of the updates to mpt_sas including support for 3008
(12G SAS).

I haven't run any 12G SAS yet, but plan to on my next build in a couple
months.   This will be about 300TB using an 84 disk JBOD.  All the code
from Nexenta to support the 3008 appears to be in Illumos now, and they
fully support it so I suspect it's pretty stable now.  From what I
understand there may be some 12G performance fixes coming sometime.

The fault manager is nice when the system doesn't panic.  When it panics,
the fault manger never gets a chance to take action.  It is still the
consensus that is is better to run pools without hot spares because there
are situations the fault manager will do bad things.   I witnessed this
myself when building a system and the fault manger replaced 5 disks in a
raidz2 vdev inside 1 minute, trashing the pool.   I haven't completely
yield to the "best practice".  I now run one hot spare per pool.  I figure
with raidz2, the odds of the fault manager causing something catastrophic
is much less possible.

-Chip



On Fri, Jul 10, 2015 at 11:37 AM, Linda Kateley <lkateley at kateley.com>
wrote:

>  I have to build and maintain my own system. I usually help others build(i
> teach zfs and freenas classes/consulting). I really love fault management
> in solaris and miss it. Just thought since it's my system and I get to
> choose I would use omni. I have 20+ years using solaris and only 2 on
> freebsd.
>
> I like freebsd for how well tuned for zfs oob. I miss the network, v12n
> and resource controls in solaris.
>
> Concerned about panics on disk failure. Is that common?
>
>
linda
>
>
> On 7/9/15 9:30 PM, Schweiss, Chip wrote:
>
>   Linda,
>
>  I have 3.5 PB running under OmniOS.  All my systems have LSI 2108 HBAs
> which is considered the best choice for HBAs.
>
> Illumos leaves a bit to be desired with handling faults from disks or SAS
> problems, but things under OmniOS have been improving, much thanks to Dan
> McDonald and OmniTI.   We have a paid support on all of our production
> systems with OmniTI.  Their response and dedication has been very good.
> Other than the occasional panic and restart from a disk failure, OmniOS has
> been solid.   ZFS of course never has lost a single bit of information.
>
>  I'd be curious why you're looking to move, have there been specific
> problems under BSD or ZoL?  I've been slowly evaluating FreeBSD ZFS, but of
> course the skeletons in the closet never seem to come out until you do
> something big.
>
>  -Chip
>
> On Thu, Jul 9, 2015 at 4:21 PM, Linda Kateley <lkateley at kateley.com>
> wrote:
>
>> Hey is there anyone out there running big zfs on omni?
>>
>> I have been doing mostly zol and freebsd for the last year but have to
>> build a 300+TB box and i want to come back home to roots(solaris). Feeling
>> kind of hesitant :) Also, if you had to do over, is there anything you
>> would do different.
>>
>> Also, what is the go to HBA these days? Seems like i saw stable code for
>> lsi 3008?
>>
>> TIA
>>
>> linda
>>
>>
>> _______________________________________________
>> OmniOS-discuss mailing list
>> OmniOS-discuss at lists.omniti.com
>> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>>
>
>
> --
> Linda Kateley
> Kateley Company
> Skype ID-kateleycohttp://kateleyco.com
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150710/66b82b30/attachment.html>

From lslusser at gmail.com  Fri Jul 10 18:04:24 2015
From: lslusser at gmail.com (Liam Slusser)
Date: Fri, 10 Jul 2015 11:04:24 -0700
Subject: [OmniOS-discuss] big zfs storage?
In-Reply-To: <CALeZrrTSMhsD0K4Vw1khAVE=XDtThJ21kyGk-5DHvy5A_oBuug@mail.gmail.com>
References: <559EE5BF.4040900@kateley.com>
	<CALeZrrQ1nVjCFiTPEnDdpvCyHHDw7pm6z3tKH=678gHtR3c5tw@mail.gmail.com>
	<559FF4DE.4040202@kateley.com>
	<CALeZrrTSMhsD0K4Vw1khAVE=XDtThJ21kyGk-5DHvy5A_oBuug@mail.gmail.com>
Message-ID: <CAESZ+_-zWA31os-tw+uvW5+FRCXwM6dt_VxgoNOAtq6fA==d_Q@mail.gmail.com>

I have two 800T ZFS systems on OmniOS and a bunch of smaller <50T systems.
Things generally work very well.  We loose a disk here and there but its
never resulted in downtime.  They're all on Dell hardware with LSI or Dell
PERC controllers.

Putting in smaller disk failure parameters, so disks fail quicker, was a
big help when something does go wrong with a disk.

thanks,
liam


On Fri, Jul 10, 2015 at 10:31 AM, Schweiss, Chip <chip at innovates.com> wrote:

> Unfortunately for the past couple years panics on disk failure has been
> the norm.   All my production systems are HA with RSF-1, so at least things
> come back online relatively quick.  There are quite a few open tickets in
> the Illumos bug tracker related to mpt_sas related panics.
>
> Most of the work to fix these problems has been committed in the past
> year, though problems still exist.  For example, my systems are dual path
> SAS, however, mpt_sas will panic if you pull a cable instead of dropping a
> path to the disks.  Dan McDonald is actively working to resolve this.   He
> is also pushing a bug fix in genunix from Nexenta that appears to fix a lot
> of the panic problems.   I'll know for sure in a few months after I see a
> disk or two drop if it truly fixes things.  Hans Rosenfeld at Nexenta is
> responsible for most of the updates to mpt_sas including support for 3008
> (12G SAS).
>
> I haven't run any 12G SAS yet, but plan to on my next build in a couple
> months.   This will be about 300TB using an 84 disk JBOD.  All the code
> from Nexenta to support the 3008 appears to be in Illumos now, and they
> fully support it so I suspect it's pretty stable now.  From what I
> understand there may be some 12G performance fixes coming sometime.
>
> The fault manager is nice when the system doesn't panic.  When it panics,
> the fault manger never gets a chance to take action.  It is still the
> consensus that is is better to run pools without hot spares because there
> are situations the fault manager will do bad things.   I witnessed this
> myself when building a system and the fault manger replaced 5 disks in a
> raidz2 vdev inside 1 minute, trashing the pool.   I haven't completely
> yield to the "best practice".  I now run one hot spare per pool.  I figure
> with raidz2, the odds of the fault manager causing something catastrophic
> is much less possible.
>
> -Chip
>
>
>
> On Fri, Jul 10, 2015 at 11:37 AM, Linda Kateley <lkateley at kateley.com>
> wrote:
>
>>  I have to build and maintain my own system. I usually help others
>> build(i teach zfs and freenas classes/consulting). I really love fault
>> management in solaris and miss it. Just thought since it's my system and I
>> get to choose I would use omni. I have 20+ years using solaris and only 2
>> on freebsd.
>>
>> I like freebsd for how well tuned for zfs oob. I miss the network, v12n
>> and resource controls in solaris.
>>
>> Concerned about panics on disk failure. Is that common?
>>
>>
> linda
>>
>>
>> On 7/9/15 9:30 PM, Schweiss, Chip wrote:
>>
>>   Linda,
>>
>>  I have 3.5 PB running under OmniOS.  All my systems have LSI 2108 HBAs
>> which is considered the best choice for HBAs.
>>
>> Illumos leaves a bit to be desired with handling faults from disks or SAS
>> problems, but things under OmniOS have been improving, much thanks to Dan
>> McDonald and OmniTI.   We have a paid support on all of our production
>> systems with OmniTI.  Their response and dedication has been very good.
>> Other than the occasional panic and restart from a disk failure, OmniOS has
>> been solid.   ZFS of course never has lost a single bit of information.
>>
>>  I'd be curious why you're looking to move, have there been specific
>> problems under BSD or ZoL?  I've been slowly evaluating FreeBSD ZFS, but of
>> course the skeletons in the closet never seem to come out until you do
>> something big.
>>
>>  -Chip
>>
>> On Thu, Jul 9, 2015 at 4:21 PM, Linda Kateley <lkateley at kateley.com>
>> wrote:
>>
>>> Hey is there anyone out there running big zfs on omni?
>>>
>>> I have been doing mostly zol and freebsd for the last year but have to
>>> build a 300+TB box and i want to come back home to roots(solaris). Feeling
>>> kind of hesitant :) Also, if you had to do over, is there anything you
>>> would do different.
>>>
>>> Also, what is the go to HBA these days? Seems like i saw stable code for
>>> lsi 3008?
>>>
>>> TIA
>>>
>>> linda
>>>
>>>
>>> _______________________________________________
>>> OmniOS-discuss mailing list
>>> OmniOS-discuss at lists.omniti.com
>>> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>>>
>>
>>
>> --
>> Linda Kateley
>> Kateley Company
>> Skype ID-kateleycohttp://kateleyco.com
>>
>>
>
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150710/94a731fa/attachment-0001.html>

From lkateley at kateley.com  Sat Jul 11 20:11:42 2015
From: lkateley at kateley.com (Linda Kateley)
Date: Sat, 11 Jul 2015 15:11:42 -0500
Subject: [OmniOS-discuss] big zfs storage?
In-Reply-To: <CALeZrrTSMhsD0K4Vw1khAVE=XDtThJ21kyGk-5DHvy5A_oBuug@mail.gmail.com>
References: <559EE5BF.4040900@kateley.com>
	<CALeZrrQ1nVjCFiTPEnDdpvCyHHDw7pm6z3tKH=678gHtR3c5tw@mail.gmail.com>
	<559FF4DE.4040202@kateley.com>
	<CALeZrrTSMhsD0K4Vw1khAVE=XDtThJ21kyGk-5DHvy5A_oBuug@mail.gmail.com>
Message-ID: <55A1787E.1050600@kateley.com>

This really saddens me. To me my favorite part of solaris was FMA. But 
happy about the 3008.

thanks

linda

On 7/10/15 12:31 PM, Schweiss, Chip wrote:
> Unfortunately for the past couple years panics on disk failure has 
> been the norm.   All my production systems are HA with RSF-1, so at 
> least things come back online relatively quick.  There are quite a few 
> open tickets in the Illumos bug tracker related to mpt_sas related 
> panics.
>
> Most of the work to fix these problems has been committed in the past 
> year, though problems still exist.  For example, my systems are dual 
> path SAS, however, mpt_sas will panic if you pull a cable instead of 
> dropping a path to the disks.  Dan McDonald is actively working to 
> resolve this.   He is also pushing a bug fix in genunix from Nexenta 
> that appears to fix a lot of the panic problems. I'll know for sure in 
> a few months after I see a disk or two drop if it truly fixes things.  
> Hans Rosenfeld at Nexenta is responsible for most of the updates to 
> mpt_sas including support for 3008 (12G SAS).
>
> I haven't run any 12G SAS yet, but plan to on my next build in a 
> couple months.   This will be about 300TB using an 84 disk JBOD.  All 
> the code from Nexenta to support the 3008 appears to be in Illumos 
> now, and they fully support it so I suspect it's pretty stable now.  
> From what I understand there may be some 12G performance fixes coming 
> sometime.
>
> The fault manager is nice when the system doesn't panic. When it 
> panics, the fault manger never gets a chance to take action.  It is 
> still the consensus that is is better to run pools without hot spares 
> because there are situations the fault manager will do bad things.   I 
> witnessed this myself when building a system and the fault manger 
> replaced 5 disks in a raidz2 vdev inside 1 minute, trashing the pool. 
>   I haven't completely yield to the "best practice".  I now run one 
> hot spare per pool.  I figure with raidz2, the odds of the fault 
> manager causing something catastrophic is much less possible.
>
> -Chip
>
>
>
> On Fri, Jul 10, 2015 at 11:37 AM, Linda Kateley <lkateley at kateley.com 
> <mailto:lkateley at kateley.com>> wrote:
>
>     I have to build and maintain my own system. I usually help others
>     build(i teach zfs and freenas classes/consulting). I really love
>     fault management in solaris and miss it. Just thought since it's
>     my system and I get to choose I would use omni. I have 20+ years
>     using solaris and only 2 on freebsd.
>
>     I like freebsd for how well tuned for zfs oob. I miss the network,
>     v12n and resource controls in solaris.
>
>     Concerned about panics on disk failure. Is that common?
>
>     linda
>
>
>     On 7/9/15 9:30 PM, Schweiss, Chip wrote:
>>     Linda,
>>
>>     I have 3.5 PB running under OmniOS.  All my systems have LSI 2108
>>     HBAs which is considered the best choice for HBAs.
>>
>>     Illumos leaves a bit to be desired with handling faults from
>>     disks or SAS problems, but things under OmniOS have been
>>     improving, much thanks to Dan McDonald and OmniTI.   We have a
>>     paid support on all of our production systems with OmniTI.  Their
>>     response and dedication has been very good.  Other than the
>>     occasional panic and restart from a disk failure, OmniOS has been
>>     solid.   ZFS of course never has lost a single bit of information.
>>
>>     I'd be curious why you're looking to move, have there been
>>     specific problems under BSD or ZoL?  I've been slowly evaluating
>>     FreeBSD ZFS, but of course the skeletons in the closet never seem
>>     to come out until you do something big.
>>
>>     -Chip
>>
>>     On Thu, Jul 9, 2015 at 4:21 PM, Linda Kateley
>>     <lkateley at kateley.com <mailto:lkateley at kateley.com>> wrote:
>>
>>         Hey is there anyone out there running big zfs on omni?
>>
>>         I have been doing mostly zol and freebsd for the last year
>>         but have to build a 300+TB box and i want to come back home
>>         to roots(solaris). Feeling kind of hesitant :) Also, if you
>>         had to do over, is there anything you would do different.
>>
>>         Also, what is the go to HBA these days? Seems like i saw
>>         stable code for lsi 3008?
>>
>>         TIA
>>
>>         linda
>>
>>
>>         _______________________________________________
>>         OmniOS-discuss mailing list
>>         OmniOS-discuss at lists.omniti.com
>>         <mailto:OmniOS-discuss at lists.omniti.com>
>>         http://lists.omniti.com/mailman/listinfo/omnios-discuss
>>
>>
>
>     -- 
>     Linda Kateley
>     Kateley Company
>     Skype ID-kateleyco
>     http://kateleyco.com
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150711/aaf282e2/attachment.html>

From bdha at mirrorshades.net  Sat Jul 11 20:45:04 2015
From: bdha at mirrorshades.net (Bryan Horstmann-Allen)
Date: Sat, 11 Jul 2015 16:45:04 -0400
Subject: [OmniOS-discuss] big zfs storage?
In-Reply-To: <55A1787E.1050600@kateley.com>
References: <559EE5BF.4040900@kateley.com>
	<CALeZrrQ1nVjCFiTPEnDdpvCyHHDw7pm6z3tKH=678gHtR3c5tw@mail.gmail.com>
	<559FF4DE.4040202@kateley.com>
	<CALeZrrTSMhsD0K4Vw1khAVE=XDtThJ21kyGk-5DHvy5A_oBuug@mail.gmail.com>
	<55A1787E.1050600@kateley.com>
Message-ID: <E5767AF6-D274-423D-83F8-04DBC070A250@mirrorshades.net>

FWIW I have not had panics on disk failure. The pool will hang, and often not recover until a reboot -- depending on the disk failure mode. 

I blame sd for this rather than ZFS or FMA, but at the end of the day it is still suboptimal behavior.
-- 
bdha

> On Jul 11, 2015, at 16:11, Linda Kateley <lkateley at kateley.com> wrote:
> 
> This really saddens me. To me my favorite part of solaris was FMA. But happy about the 3008.
> 
> thanks 
> 
> linda
> 
>> On 7/10/15 12:31 PM, Schweiss, Chip wrote:
>> Unfortunately for the past couple years panics on disk failure has been the norm.   All my production systems are HA with RSF-1, so at least things come back online relatively quick.  There are quite a few open tickets in the Illumos bug tracker related to mpt_sas related panics.   
>> 
>> Most of the work to fix these problems has been committed in the past year, though problems still exist.  For example, my systems are dual path SAS, however, mpt_sas will panic if you pull a cable instead of dropping a path to the disks.  Dan McDonald is actively working to resolve this.   He is also pushing a bug fix in genunix from Nexenta that appears to fix a lot of the panic problems.   I'll know for sure in a few months after I see a disk or two drop if it truly fixes things.  Hans Rosenfeld at Nexenta is responsible for most of the updates to mpt_sas including support for 3008 (12G SAS).
>> 
>> I haven't run any 12G SAS yet, but plan to on my next build in a couple months.   This will be about 300TB using an 84 disk JBOD.  All the code from Nexenta to support the 3008 appears to be in Illumos now, and they fully support it so I suspect it's pretty stable now.  From what I understand there may be some 12G performance fixes coming sometime.   
>> 
>> The fault manager is nice when the system doesn't panic.  When it panics, the fault manger never gets a chance to take action.  It is still the consensus that is is better to run pools without hot spares because there are situations the fault manager will do bad things.   I witnessed this myself when building a system and the fault manger replaced 5 disks in a raidz2 vdev inside 1 minute, trashing the pool.   I haven't completely yield to the "best practice".  I now run one hot spare per pool.  I figure with raidz2, the odds of the fault manager causing               something catastrophic is much less possible. 
>> 
>> -Chip
>> 
>> 
>> 
>>> On Fri, Jul 10, 2015 at 11:37 AM, Linda Kateley <lkateley at kateley.com> wrote:
>>> I have to build and maintain my own system. I usually help others build(i teach zfs and freenas classes/consulting). I really love fault management in solaris and miss it. Just thought since it's my system and I get to choose I would use omni. I have 20+ years using solaris and only 2 on freebsd. 
>>> 
>>> I like freebsd for how well tuned for zfs oob. I miss the network, v12n and resource controls in solaris. 
>>> 
>>> Concerned about panics on disk failure. Is that common?
>>> linda
>>> 
>>> 
>>>> On 7/9/15 9:30 PM, Schweiss, Chip wrote:
>>>> Linda,
>>>> 
>>>> I have 3.5 PB running under OmniOS.  All my systems have LSI 2108 HBAs which is considered the best choice for HBAs.   
>>>> 
>>>> Illumos leaves a bit to be desired with handling faults from disks or SAS problems, but things under OmniOS have been improving, much thanks to Dan McDonald and OmniTI.   We have a paid support on all of our production systems with OmniTI.  Their response and dedication has been very good.  Other than the occasional panic and restart from a disk failure, OmniOS has been solid.   ZFS of course never has lost a single bit of information. 
>>>> 
>>>> I'd be curious why you're looking to move, have there been specific problems under BSD or ZoL?  I've been slowly evaluating FreeBSD ZFS, but of course the skeletons in the closet never seem to come out until you do something big.
>>>> 
>>>> -Chip
>>>> 
>>>>> On Thu, Jul 9, 2015 at 4:21 PM, Linda Kateley <lkateley at kateley.com> wrote:
>>>>> Hey is there anyone out there running big zfs on omni?
>>>>> 
>>>>> I have been doing mostly zol and freebsd for the last year but have to build a 300+TB box and i want to come back home to roots(solaris). Feeling kind of hesitant :) Also, if you had to do over, is there anything you would do different.
>>>>> 
>>>>> Also, what is the go to HBA these days? Seems like i saw stable code for lsi 3008?
>>>>> 
>>>>> TIA
>>>>> 
>>>>> linda
>>>>> 
>>>>> 
>>>>> _______________________________________________
>>>>> OmniOS-discuss mailing list
>>>>> OmniOS-discuss at lists.omniti.com
>>>>> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>>> 
>>>  -- 
>>> Linda Kateley
>>> Kateley Company
>>> Skype ID-kateleyco
>>> http://kateleyco.com
> 
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150711/8a6e4391/attachment-0001.html>

From derek at umiacs.umd.edu  Sun Jul 12 02:43:35 2015
From: derek at umiacs.umd.edu (Derek Yarnell)
Date: Sat, 11 Jul 2015 22:43:35 -0400
Subject: [OmniOS-discuss] ZFS crash/reboot loop
Message-ID: <55A1D457.7060207@umiacs.umd.edu>

Hi,

We just have had a catastrophic event on one of our OmniOS r14 file
servers.  In what seems to have been triggered by the weekly scrub of
its one large zfs pool (~100T) it panics.  This made it basically reboot
continually and we have installed a second copy of OmniOS r14 in the
mean time.  We are able to mount the pool readonly and are currently
securing the data as soon as possible.

The underlying zpool (zvol00) was a Nexenta that was imported on June
18th and had gotten through more than one weekly scrub since we had
upgraded.  We had not upgraded the pool from 28 yet and we are not using
dedup.  I have the the vmdump[0] file available if that is useful from
when we tried running `zpool import zvol00`.

The hardware should be fully compatible Supermicro X8DTH-i/6/iF/6F,
Hitachi/HGST SAS drives, STEC ZeusRAM slogs, STEC l2arc drives with LSI
2008 controllers.

[0] - https://obj.umiacs.umd.edu/derek_support/vmdump.0


TIME                           UUID
SUNW-MSG-ID
Jul 11 2015 21:23:40.015738000 f120cfb4-7e83-c576-ca89-e522daf796e7
SUNOS-8000-KL

  TIME                 CLASS                                 ENA
  Jul 11 21:23:39.9681 ireport.os.sunos.panic.dump_available
0x0000000000000000
  Jul 11 21:23:24.1537 ireport.os.sunos.panic.dump_pending_on_device
0x0000000000000000

nvlist version: 0
        version = 0x0
        class = list.suspect
        uuid = f120cfb4-7e83-c576-ca89-e522daf796e7
        code = SUNOS-8000-KL
        diag-time = 1436664219 981033
        de = fmd:///module/software-diagnosis
        fault-list-sz = 0x1
        fault-list = (array of embedded nvlists)
        (start fault-list[0])
        nvlist version: 0
                version = 0x0
                class = defect.sunos.kernel.panic
                certainty = 0x64
                asru =
sw:///:path=/var/crash/unknown/.f120cfb4-7e83-c576-ca89-e522daf796e7
                resource =
sw:///:path=/var/crash/unknown/.f120cfb4-7e83-c576-ca89-e522daf796e7
                savecore-succcess = 1
                dump-dir = /var/crash/unknown
                dump-files = vmdump.0
                os-instance-uuid = f120cfb4-7e83-c576-ca89-e522daf796e7
                panicstr = assertion failed: c < (1ULL << 24) >> 9
(0x7fffffffffffff < 0x8000), file: ../../common/fs/zfs/zio.c, line: 221
                panicstack = fffffffffba8b13d () | zfs:zio_buf_alloc+49
() | zfs:arc_get_data_buf+12b () | zfs:arc_buf_alloc+d2 () |
zfs:arc_read+15b () | zfs:dsl_scan_prefetch+da () |
zfs:dsl_scan_recurse+13c () | zfs:dsl_scan_visitbp+fd () |
zfs:dsl_scan_visitdnode+b4 () | zfs:dsl_scan_recurse+435 () |
zfs:dsl_scan_visitbp+fd () | zfs:dsl_scan_recurse+1b9 () |
zfs:dsl_scan_visitbp+fd () | zfs:dsl_scan_recurse+1b9 () |
zfs:dsl_scan_visitbp+fd () | zfs:dsl_scan_visitdnode+b4 () |
zfs:dsl_scan_recurse+496 () | zfs:dsl_scan_visitbp+fd () |
zfs:dsl_scan_visit_rootbp+5d () | zfs:dsl_scan_visit+273 () |
zfs:dsl_scan_sync+247 () | zfs:spa_sync+2b3 () | zfs:txg_sync_thread+227
() | unix:thread_start+8 () |
                crashtime = 1436664019
                panic-time = Sat Jul 11 21:20:19 2015 EDT
        (end fault-list[0])

        fault-status = 0x1
        severity = Major
        __ttl = 0x1
        __tod = 0x55a1c19c 0xf02490

### In readonly mode
root at cbcbomni02:/root# zdb -e -bcsvL zvol00
assertion failed for thread 0xfffffd7fff162a40, thread-id 1: !claimed ||
!(zh->zh_flags & ZIL_CLAIM_LR_SEQ_VALID) || (max_blk_seq ==
claim_blk_seq && max_lr_seq == claim_lr_seq), file
../../../uts/common/fs/zfs/zil.c, line 367
Abort (core dumped)

### After mounting in readonly mode
  pool: zvol00
 state: ONLINE
status: The pool is formatted using a legacy on-disk format.  The pool can
        still be used, but some features are unavailable.
action: Upgrade the pool using 'zpool upgrade'.  Once this is done, the
        pool will no longer be accessible on software that does not
support feature
        flags.
  scan: scrub in progress since Sat Jul 11 11:00:02 2015
    2.24G scanned out of 69.5T at 1/s, (scan is slow, no estimated time)
    0 repaired, 0.00% done
config:

        NAME                       STATE     READ WRITE CKSUM
        zvol00                     ONLINE       0     0     0
          raidz2-0                 ONLINE       0     0     0
            c4t5000CCA01A91D6F9d0  ONLINE       0     0     0
            c4t5000CCA01A9D4B6Dd0  ONLINE       0     0     0
            c4t5000CCA01A9ED1A9d0  ONLINE       0     0     0
            c4t5000CCA01A9ED311d0  ONLINE       0     0     0
            c4t5000CCA01A9ED345d0  ONLINE       0     0     0
            c4t5000CCA01A9EDD21d0  ONLINE       0     0     0
          raidz2-1                 ONLINE       0     0     0
            c4t5000CCA01A9EE1D1d0  ONLINE       0     0     0
            c4t5000CCA01A9EE3D9d0  ONLINE       0     0     0
            c4t5000CCA01A9F6659d0  ONLINE       0     0     0
            c4t5000CCA01A9F69C5d0  ONLINE       0     0     0
            c4t5000CCA01A9F81A9d0  ONLINE       0     0     0
            c4t5000CCA01AA139CDd0  ONLINE       0     0     0
          raidz2-2                 ONLINE       0     0     0
            c4t5000CCA01AA139D9d0  ONLINE       0     0     0
            c4t5000CCA01AA1CEE9d0  ONLINE       0     0     0
            c4t5000CCA01AA387C9d0  ONLINE       0     0     0
            c4t5000CCA01AA38A89d0  ONLINE       0     0     0
            c4t5000CCA01AA38A91d0  ONLINE       0     0     0
            c4t5000CCA01AA38A95d0  ONLINE       0     0     0
          raidz2-3                 ONLINE       0     0     0
            c5t5000CCA01A9F69F5d0  ONLINE       0     0     0
            c5t5000CCA01A9F816Dd0  ONLINE       0     0     0
            c5t5000CCA01AA389F1d0  ONLINE       0     0     0
            c5t5000CCA01AA38A4Dd0  ONLINE       0     0     0
            c5t5000CCA01AA38A61d0  ONLINE       0     0     0
            c5t5000CCA01AA390F1d0  ONLINE       0     0     0
          raidz2-5                 ONLINE       0     0     0
            c2t5000CCA02490EC4Dd0  ONLINE       0     0     0
            c2t5000CCA02490ED7Dd0  ONLINE       0     0     0
            c2t5000CCA02490EE79d0  ONLINE       0     0     0
            c2t5000CCA02490F0B1d0  ONLINE       0     0     0
            c2t5000CCA02490F271d0  ONLINE       0     0     0
            c2t5000CCA02490F865d0  ONLINE       0     0     0
          raidz2-6                 ONLINE       0     0     0
            c2t5000CCA02490EDB5d0  ONLINE       0     0     0
            c2t5000CCA02490F0D1d0  ONLINE       0     0     0
            c2t5000CCA02490F2F9d0  ONLINE       0     0     0
            c2t5000CCA02490F66Dd0  ONLINE       0     0     0
            c2t5000CCA0249CB6A9d0  ONLINE       0     0     0
            c2t5000CCA0249CB6D9d0  ONLINE       0     0     0
        logs
          mirror-4                 ONLINE       0     0     0
            c4t5000A72A3003CBAAd0  ONLINE       0     0     0
            c4t5000A72A300737B5d0  ONLINE       0     0     0
        cache
          c4t5000A72A3006BD31d0    ONLINE       0     0     0
          c4t5000A72A3006BD37d0    ONLINE       0     0     0


zvol00  size                           108T                           -
zvol00  capacity                       64%                            -
zvol00  altroot                        -
default
zvol00  health                         ONLINE                         -
zvol00  guid                           15199497356660648465           local
zvol00  version                        28                             local
zvol00  bootfs                         -
default
zvol00  delegation                     on
default
zvol00  autoreplace                    on                             local
zvol00  cachefile                      -
default
zvol00  failmode                       continue                       local
zvol00  listsnapshots                  off
default
zvol00  autoexpand                     off
default
zvol00  dedupditto                     0
default
zvol00  dedupratio                     1.00x                          -
zvol00  free                           39.0T                          -
zvol00  allocated                      69.5T                          -
zvol00  readonly                       on                             -
zvol00  comment                        -
default
zvol00  expandsize                     -                              -
zvol00  freeing                        0                              local
zvol00  fragmentation                  0%                             -
zvol00  leaked                         0                              local
zvol00  feature at async_destroy          disabled                       local
zvol00  feature at empty_bpobj            disabled                       local
zvol00  feature at lz4_compress           disabled                       local
zvol00  feature at multi_vdev_crash_dump  disabled                       local
zvol00  feature at spacemap_histogram     disabled                       local
zvol00  feature at enabled_txg            disabled                       local
zvol00  feature at hole_birth             disabled                       local
zvol00  feature at extensible_dataset     disabled                       local
zvol00  feature at embedded_data          disabled                       local
zvol00  feature at bookmarks              disabled                       local
zvol00  feature at filesystem_limits      disabled                       local
zvol00  feature at large_blocks           disabled                       local

2015-06-18.22:10:56 zpool import -f zvol00
2015-06-18.22:11:07 zpool status
2015-06-18.22:45:08 [txg:12396996] open pool version 28; software
version 5000/5; uts cbcbomni02 5.11 omnios-170cea2 i86pc
2015-06-20.11:00:03 [txg:12427001] scan setup func=1 mintxg=0
maxtxg=12427001
2015-06-20.11:00:11 zpool scrub rpool zvol00
2015-06-24.20:39:57 [txg:12493454] scan done complete=1
2015-06-27.11:00:03 [txg:12544493] scan setup func=1 mintxg=0
maxtxg=12544493
2015-06-27.11:00:11 zpool scrub rpool zvol00
2015-07-02.14:13:43 [txg:12620332] scan done complete=1
2015-07-04.11:00:03 [txg:12652793] scan setup func=1 mintxg=0
maxtxg=12652793
2015-07-04.11:00:11 zpool scrub rpool zvol00
2015-07-06.03:58:52 [txg:12678221] scan done complete=1
2015-07-11.11:00:02 [txg:12797698] scan setup func=1 mintxg=0
maxtxg=12797698
2015-07-11.11:00:12 zpool scrub rpool zvol00
2015-07-11.13:17:32 [txg:12797754] open pool version 28; software
version 5000/5; uts cbcbomni02 5.11 omnios-170cea2 i86pc


root at cbcbomni02:/var/crash/unknown# iostat -Ezn
c1t5000C5004CD3C173d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: SEAGATE  Product: ST9146852SS      Revision: 0005 Serial No:
6TB3DYXJ0000B24
Size: 146.82GB <146815737856 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
c4t5000A72A3003CBAAd0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: STEC     Product: ZeusRAM          Revision: C018 Serial No:
STM000131E94
Size: 8.00GB <8000000000 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 9 Predictive Failure Analysis: 0
c4t5000A72A300737B5d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: STEC     Product: ZeusRAM          Revision: C018 Serial No:
STM00016C0F4
Size: 8.00GB <8000000000 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 9 Predictive Failure Analysis: 0
c4t5000A72A3006BD31d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: STEC     Product: Z16IZF2E-400UCV  Revision: E493 Serial No:
STM000164420
Size: 400.09GB <400088457216 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
c4t5000A72A3006BD37d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: STEC     Product: Z16IZF2E-400UCV  Revision: E493 Serial No:
STM000164425
Size: 400.09GB <400088457216 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
c4t5000CCA01A9F69C5d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: HITACHI  Product: HUS723030ALS640  Revision: A222 Serial No:
YHJUNUEG
Size: 3000.59GB <3000592982016 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
c4t5000CCA01A9EE1D1d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: HITACHI  Product: HUS723030ALS640  Revision: A222 Serial No:
YHJUBSSG
Size: 3000.59GB <3000592982016 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
c4t5000CCA01A9EE3D9d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: HITACHI  Product: HUS723030ALS640  Revision: A222 Serial No:
YHJUBWYG
Size: 3000.59GB <3000592982016 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
c4t5000CCA01A9ED311d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: HITACHI  Product: HUS723030ALS640  Revision: A222 Serial No:
YHJUAT9G
Size: 3000.59GB <3000592982016 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
c4t5000CCA01AA387C9d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: HITACHI  Product: HUS723030ALS640  Revision: A222 Serial No:
YHJWY0GG
Size: 3000.59GB <3000592982016 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
c4t5000CCA01AA38A89d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: HITACHI  Product: HUS723030ALS640  Revision: A222 Serial No:
YHJWY64G
Size: 3000.59GB <3000592982016 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
c4t5000CCA01AA1CEE9d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: HITACHI  Product: HUS723030ALS640  Revision: A222 Serial No:
YHJVZN8G
Size: 3000.59GB <3000592982016 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
c4t5000CCA01AA139CDd0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: HITACHI  Product: HUS723030ALS640  Revision: A222 Serial No:
YHJVNREG
Size: 3000.59GB <3000592982016 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
c4t5000CCA01AA139D9d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: HITACHI  Product: HUS723030ALS640  Revision: A222 Serial No:
YHJVNRJG
Size: 3000.59GB <3000592982016 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
c4t5000CCA01A91D6F9d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: HITACHI  Product: HUS723030ALS640  Revision: A222 Serial No:
YHJL6DND
Size: 3000.59GB <3000592982016 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
c4t5000CCA01A9ED1A9d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: HITACHI  Product: HUS723030ALS640  Revision: A222 Serial No:
YHJUAPDG
Size: 3000.59GB <3000592982016 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
c4t5000CCA01A9EDD21d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: HITACHI  Product: HUS723030ALS640  Revision: A222 Serial No:
YHJUBG2G
Size: 3000.59GB <3000592982016 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
c4t5000CCA01A9ED345d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: HITACHI  Product: HUS723030ALS640  Revision: A222 Serial No:
YHJUATRG
Size: 3000.59GB <3000592982016 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
c4t5000CCA01A9F81A9d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: HITACHI  Product: HUS723030ALS640  Revision: A222 Serial No:
YHJURDSG
Size: 3000.59GB <3000592982016 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
c4t5000CCA01A9F6659d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: HITACHI  Product: HUS723030ALS640  Revision: A222 Serial No:
YHJUNLBG
Size: 3000.59GB <3000592982016 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
c4t5000CCA01A9D4B6Dd0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: HITACHI  Product: HUS723030ALS640  Revision: A222 Serial No:
YHJTHPSG
Size: 3000.59GB <3000592982016 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
c4t5000CCA01AA38A95d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: HITACHI  Product: HUS723030ALS640  Revision: A222 Serial No:
YHJWY67G
Size: 3000.59GB <3000592982016 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
c4t5000CCA01AA38A91d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: HITACHI  Product: HUS723030ALS640  Revision: A222 Serial No:
YHJWY66G
Size: 3000.59GB <3000592982016 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
c3t0d0           Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: Generic  Product: Flash Disk       Revision: 8.07 Serial No:
Size: 8.05GB <8054112256 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 83 Predictive Failure Analysis: 0
c5t5000CCA01AA38A61d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: HITACHI  Product: HUS723030ALS640  Revision: A222 Serial No:
YHJWY5UG
Size: 3000.59GB <3000592982016 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
c5t5000CCA01AA389F1d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: HITACHI  Product: HUS723030ALS640  Revision: A222 Serial No:
YHJWY4XG
Size: 3000.59GB <3000592982016 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
c5t5000CCA01AA390F1d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: HITACHI  Product: HUS723030ALS640  Revision: A222 Serial No:
YHJWYMBG
Size: 3000.59GB <3000592982016 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
c5t5000CCA01AA38A4Dd0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: HITACHI  Product: HUS723030ALS640  Revision: A222 Serial No:
YHJWY5NG
Size: 3000.59GB <3000592982016 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
c5t5000CCA01A9F69F5d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: HITACHI  Product: HUS723030ALS640  Revision: A222 Serial No:
YHJUNUUG
Size: 3000.59GB <3000592982016 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
c5t5000CCA01A9F816Dd0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: HITACHI  Product: HUS723030ALS640  Revision: A222 Serial No:
YHJURD8G
Size: 3000.59GB <3000592982016 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
c2t5000CCA02490F2F9d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: HGST     Product: HUS724040ALS640  Revision: A152 Serial No:
PAJKR6YX
Size: 4000.79GB <4000787030016 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
c2t5000CCA02490F0B1d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: HGST     Product: HUS724040ALS640  Revision: A152 Serial No:
PAJKR27X
Size: 4000.79GB <4000787030016 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
c2t5000CCA0249CB6A9d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: HGST     Product: HUS724040ALS640  Revision: A152 Serial No:
PAJT5TMX
Size: 4000.79GB <4000787030016 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
c2t5000CCA02490F66Dd0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: HGST     Product: HUS724040ALS640  Revision: A152 Serial No:
PAJKRG2X
Size: 4000.79GB <4000787030016 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
c2t5000CCA02490F865d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: HGST     Product: HUS724040ALS640  Revision: A152 Serial No:
PAJKRL4X
Size: 4000.79GB <4000787030016 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
c2t5000CCA02490F0D1d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: HGST     Product: HUS724040ALS640  Revision: A152 Serial No:
PAJKR2HX
Size: 4000.79GB <4000787030016 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
c2t5000CCA02490EE79d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: HGST     Product: HUS724040ALS640  Revision: A152 Serial No:
PAJKPXNX
Size: 4000.79GB <4000787030016 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
c2t5000CCA02490EDB5d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: HGST     Product: HUS724040ALS640  Revision: A152 Serial No:
PAJKPW2X
Size: 4000.79GB <4000787030016 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
c2t5000CCA02490ED7Dd0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: HGST     Product: HUS724040ALS640  Revision: A152 Serial No:
PAJKPVMX
Size: 4000.79GB <4000787030016 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
c2t5000CCA02490EC4Dd0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: HGST     Product: HUS724040ALS640  Revision: A152 Serial No:
PAJKPT5X
Size: 4000.79GB <4000787030016 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
c2t5000CCA0249CB6D9d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: HGST     Product: HUS724040ALS640  Revision: A152 Serial No:
PAJT5U0X
Size: 4000.79GB <4000787030016 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
c2t5000CCA02490F271d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: HGST     Product: HUS724040ALS640  Revision: A152 Serial No:
PAJKR5VX
Size: 4000.79GB <4000787030016 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0

-- 
Derek T. Yarnell
University of Maryland
Institute for Advanced Computer Studies

From venture37 at gmail.com  Sun Jul 12 11:29:15 2015
From: venture37 at gmail.com (Sevan / Venture37)
Date: Sun, 12 Jul 2015 12:29:15 +0100
Subject: [OmniOS-discuss] pkgsrc-current OmniOS 170cea2/i386 2015-07-09 21:35
Message-ID: <CA+U3Mf6yA5VHgr5WdS9dhiEHs1epCmr22V_0KZ-S25soeWwnHw@mail.gmail.com>

System ld.cache was modified to add the GCC lib directory in search
path with crle(1) prior to attempting this bulkbuild.
This resolved the previous breakage with the gettext installed from
the OmniOS IPS repo and allowed us to progress.

pkgsrc bulk build report
========================

OmniOS 170cea2/i386
Compiler: gcc

Build start: 2015-07-09 21:35
Build end:   2015-07-12 07:59

Full report: http://pkgsrc.geeklan.co.uk/reports/current/OmniOS/20150709.2135/meta/report.html
Machine readable version:
http://pkgsrc.geeklan.co.uk/reports/current/OmniOS/20150709.2135/meta/report.bz2

Total number of packages:      16536
  Successfully built:           8961
  Failed to build:               890
  Depending on failed package:  5968
  Explicitly broken or masked:   658
  Depending on masked package:    59

Packages breaking the most other packages

Package                               Breaks Maintainer
-------------------------------------------------------------------------
devel/gobject-introspection             1718 pkgsrc-users at NetBSD.org
sysutils/dbus-glib                      1013 pkgsrc-users at NetBSD.org
lang/ruby200-base                        616 taca at NetBSD.org
lang/ruby21-base                         542 taca at NetBSD.org
lang/ruby193-base                        540 taca at NetBSD.org
lang/ruby22-base                         537 taca at NetBSD.org
math/mpfr                                484 pkgsrc-users at NetBSD.org
audio/libsndfile                         438 pkgsrc-users at NetBSD.org
devel/yasm                               404 shaun at inerd.com
devel/boost-headers                      388 pkgsrc-users at NetBSD.org

Build failures

Package                               Breaks Maintainer
-------------------------------------------------------------------------
archivers/unalz                              pkgsrc-users at NetBSD.org
audio/akode-plugins-oss                      pkgsrc-users at NetBSD.org
audio/akode-plugins-sun                      pkgsrc-users at NetBSD.org
audio/alsa-plugins-oss                       pkgsrc-users at NetBSD.org
audio/amp                                    pkgsrc-users at NetBSD.org
audio/aumix                                  trevor at jpj.net
audio/cam                                    hubertf at NetBSD.org
audio/daapd                                  nathanw at NetBSD.org
audio/dap                                    pkgsrc-users at NetBSD.org
audio/esound                             379 pkgsrc-users at NetBSD.org
audio/freealut                             3 pkgsrc-users at NetBSD.org
audio/gramofile                              pkgsrc-users at NetBSD.org
audio/liba52                              57 pkgsrc-users at NetBSD.org
audio/libao-oss                              pkgsrc-users at NetBSD.org
audio/libao-sun                              pkgsrc-users at NetBSD.org
audio/libdca                              43 shattered at NetBSD.org
audio/libebur128                           1 pkgsrc-users at NetBSD.org
audio/libsndfile                         438 pkgsrc-users at NetBSD.org
audio/libvisual0.2-plugins                   pkgsrc-users at NetBSD.org
audio/maplay                                 pkgsrc-users at NetBSD.org
audio/mixer.app                              ptiJo at noos.fr
audio/mp3blaster                             rxg at NetBSD.org
audio/mpg123                              21 martin at NetBSD.org
audio/mppenc                                 pkgsrc-users at NetBSD.org
audio/nas                                256 markd at NetBSD.org
audio/nosefart                               dgriffi at cs.csubak.edu
audio/nspmod                                 pkgsrc-users at NetBSD.org
audio/ocp                                    shattered at NetBSD.org
audio/pd                                     pkgsrc-users at NetBSD.org
audio/portaudio                            4 pkgsrc-users at NetBSD.org
audio/portaudio-devel                     30 pkgsrc-users at NetBSD.org
audio/rexima                                 pkgsrc-users at NetBSD.org
audio/rplay                                1 pkgsrc-users at NetBSD.org
audio/rtunes                                 pkgsrc-users at NetBSD.org
audio/sidplay                                pkgsrc-users at NetBSD.org
audio/sidplay2                               pkgsrc-users at NetBSD.org
audio/spiralloops                            pkgsrc-users at NetBSD.org
audio/spiralsynth                            pkgsrc-users at NetBSD.org
audio/splay                                  pkgsrc-users at NetBSD.org
audio/tcl-snack                            1 gson at NetBSD.org
audio/tfmxplay                               pkgsrc-users at NetBSD.org
audio/tracker                                pkgsrc-users at NetBSD.org
audio/wmmixer                              2 pr at alles.prima.de
audio/wmsmixer                               pkgsrc-users at NetBSD.org
audio/wsoundserver                         1 pkgsrc-users at NetBSD.org
audio/xcdplayer                              pkgsrc-users at NetBSD.org
audio/xmix                                   pkgsrc-users at NetBSD.org
audio/xmms                                16 pkgsrc-users at NetBSD.org
audio/xsidplay                               pkgsrc-users at NetBSD.org
audio/xwave                                  pkgsrc-users at NetBSD.org
benchmarks/glmark2                           prlw1 at cam.ac.uk
benchmarks/iozone                            pkgsrc-users at NetBSD.org
biology/chemical-mime-data                 1 kamel.derouiche at gmail.com
biology/clustalw                             babafou at babafou.eu.org
biology/mopac                                pkgsrc-users at NetBSD.org
biology/mpqc                                 asau at inbox.ru
cad/qcad-partlibrary                         pkgsrc-users at NetBSD.org
chat/ctrlproxy                               pkgsrc-users at NetBSD.org
chat/gtmess                                  luapzz at gmail.com
chat/i2cbd                                   pkgsrc-users at NetBSD.org
chat/inspircd                                pkgsrc-users at NetBSD.org
chat/licq-core                             2 pkgsrc-users at NetBSD.org
chat/naim                                    pkgsrc-users at NetBSD.org
chat/phone                                   christos at NetBSD.org
chat/quirc                                   pkgsrc-users at NetBSD.org
chat/tkicb                                   pkgsrc-users at NetBSD.org
comms/asterisk                             1 jnemeth at NetBSD.org
comms/asterisk10                             jnemeth at NetBSD.org
comms/asterisk18                             jnemeth at NetBSD.org
comms/dl-ezkit                               pkgsrc-users at NetBSD.org
comms/hylafax                              1 pkgsrc-users at NetBSD.org
comms/libopensync-plugin-file                pkgsrc-users at NetBSD.org
comms/msynctool                              pkgsrc-users at NetBSD.org
comms/tn3270                                 pkgsrc-users at NetBSD.org
converters/p5-MARC-Charset                 1 pkgsrc-users at NetBSD.org
converters/TECkit                         11 minskim at NetBSD.org
cross/binutils                             2 pkgsrc-users at NetBSD.org
cross/cc65                                   pkgsrc-users at NetBSD.org
cross/GenFw                                  jakllsch at NetBSD.org
cross/h8300-hms-binutils                   1 pkgsrc-users at NetBSD.org
cross/icdprog                                pkgsrc-users at NetBSD.org
cross/mingw-binutils                       2 pkgsrc-users at NetBSD.org
cross/nios2-gcc3                             pkgsrc-users at NetBSD.org
cross/nios2-gcc41                            pkgsrc-users at NetBSD.org
cross/uisp                                   pkgsrc-users at NetBSD.org
databases/db5                              3 pkgsrc-users at NetBSD.org
databases/gnats                              pkgsrc-users at NetBSD.org
databases/libgda                           5 pkgsrc-users at NetBSD.org
databases/mariadb55-server                   pkgsrc-users at NetBSD.org
databases/mysql51-client                   3 pkgsrc-users at NetBSD.org
databases/mysql55-server                   3 pkgsrc-users at NetBSD.org
databases/nss-pgsql                          pkgsrc-users at NetBSD.org
databases/odbc-postgresql                    adam at NetBSD.org
databases/pgtcl                              pkgsrc-users at NetBSD.org
databases/pgtclng                            pkgsrc-users at NetBSD.org
databases/postgresql-postgis2                gdt at NetBSD.org
    postgresql90-postgis-2.1.8               gdt at NetBSD.org
databases/postgresql84-uuid                  brook at nmsu.edu
databases/postgresql90-uuid                  brook at nmsu.edu
databases/qdbm-plus                          pkgsrc-users at NetBSD.org
databases/sqsh                               pkgsrc-users at NetBSD.org
databases/sqsh-x11                           pkgsrc-users at NetBSD.org
databases/xsqlmenu                           pkgsrc-users at NetBSD.org
devel/aegis                                  pkgsrc-users at NetBSD.org
devel/afl                                    pkgsrc-users at NetBSD.org
devel/ald                                    pkgsrc-users at NetBSD.org
devel/astyle                                 pkgsrc-users at NetBSD.org
devel/bcc                                    pkgsrc-users at NetBSD.org
devel/bmake                                9 joerg at NetBSD.org
devel/boost-docs                           3 pkgsrc-users at NetBSD.org
devel/boost-headers                      388 pkgsrc-users at NetBSD.org
devel/bouml                                  pkgsrc-users at NetBSD.org
devel/bullet                                 pkgsrc-users at NetBSD.org
devel/cgen                                   ryoon at NetBSD.org
devel/concurrencykit                         pkgsrc-users at NetBSD.org
devel/cook                                 1 groo at NetBSD.org
devel/cssc                                   pkgsrc-users at NetBSD.org
devel/cvs-fast-export                        apb at NetBSD.org
devel/cvsync                                 pkgsrc-users at NetBSD.org
devel/editline                             7 ryoon at NetBSD.org
devel/elf                                    pkgsrc-users at NetBSD.org
devel/elftoolchain                           agc at NetBSD.org
devel/fhist                                  pkgsrc-users at NetBSD.org
devel/fortran-utils                          pkgsrc-users at NetBSD.org
devel/gcvs                                   pkgsrc-users at NetBSD.org
devel/gdb                                    pkgsrc-users at NetBSD.org
devel/gobject-introspection             1718 pkgsrc-users at NetBSD.org
devel/hdf5-c++                               alnsn at NetBSD.org
devel/ht                                     pkgsrc-users at NetBSD.org
devel/hugs-unix                              pkgsrc-users at NetBSD.org
devel/intel2gas                              ccatrian at eml.cc
devel/kyua                                   jmmv at NetBSD.org
devel/ldapsdk                              1 pkgsrc-users at NetBSD.org
devel/libblkid                               pkgsrc-users at NetBSD.org
devel/libdnsres                            1 pkgsrc-users at NetBSD.org
devel/libjit                                 pkgsrc-users at NetBSD.org
devel/liblangtag                           1 jaapb at NetBSD.org
devel/libpgm                                 agc at NetBSD.org
devel/librlog                              2 pkgsrc-users at NetBSD.org
devel/libstash                               pkgsrc-users at NetBSD.org
devel/libtecla                             2 kristerw at NetBSD.org
devel/libusb                              95 drochner at NetBSD.org
devel/libusbx                                pkgsrc-users at NetBSD.org
devel/libxenserver                           sborrill at NetBSD.org
devel/lua-lrexlib-posix                    1 cheusov at NetBSD.org
devel/lua-posix                              ryoon at NetBSD.org
devel/mad-flute                              agc at NetBSD.org
devel/man-pages                              pkgsrc-users at NetBSD.org
devel/mico                                   tonnerre at NetBSD.org
devel/netbsd-iscsi-lib                     2 sborrill at NetBSD.org
devel/npth                                71 ada at netbsdfr.org
devel/nqc                                    nathanw at NetBSD.org
devel/nsis                                   pkgsrc-users at NetBSD.org
devel/ode                                    pkgsrc-users at NetBSD.org
devel/openrcs                                asau at inbox.ru
devel/p5-Coro                              7 pkgsrc-users at NetBSD.org
devel/p5-Data-Alias                        2 pkgsrc-users at NetBSD.org
devel/p5-Devel-FindRef                       pkgsrc-users at NetBSD.org
devel/p5-local-lib                         2 pkgsrc-users at NetBSD.org
devel/p5-RT-Authen-ExternalAuth              tonnerre at NetBSD.org
devel/palmpower                              pkgsrc-users at NetBSD.org
devel/pdcurses                               pkgsrc-users at NetBSD.org
devel/picp                                   dogcow at NetBSD.org
devel/pwlib                                2 pkgsrc-users at NetBSD.org
devel/py-astroid                           1 kamel.derouiche at gmail.com
devel/py-distribute                          pkgsrc-users at NetBSD.org
devel/py-h5py                                jihbed.research at gmail.com
devel/py-protobuf                            khorben at defora.org
devel/py-spwd                                pkgsrc-users at NetBSD.org
devel/quilt                                  jmmv at NetBSD.org
devel/radare2                                pkgsrc-users at NetBSD.org
devel/sfio                                   pkgsrc-users at NetBSD.org
devel/sfslite                                agc at NetBSD.org
devel/st                                     gson at NetBSD.org
devel/sunifdef                               pkgsrc-users at NetBSD.org
devel/syncdir                                schmonz at NetBSD.org
devel/tavrasm                                pkgsrc-users at NetBSD.org
devel/tcl-tclOO                              rodent at NetBSD.org
devel/tet3                                   pkgsrc-users at NetBSD.org
devel/ucommon                              2 pkgsrc-users at NetBSD.org
devel/xdelta3                                pkgsrc-users at NetBSD.org
devel/yasm                               404 shaun at inerd.com
editors/biew                                 pkgsrc-users at NetBSD.org
editors/cooledit                             pkgsrc-users at NetBSD.org
editors/emacs20                            1 dholland at NetBSD.org
editors/emacs23-nox11                        minskim at NetBSD.org
editors/ex                                   pkgsrc-users at NetBSD.org
editors/manedit                              pkgsrc-users at NetBSD.org
editors/mule                                 pkgsrc-users at NetBSD.org
editors/nvi                                2 pkgsrc-users at NetBSD.org
editors/nvi-m17n                             pkgsrc-users at NetBSD.org
editors/obby                               1 pkgsrc-users at NetBSD.org
editors/sam                                1 pkgsrc-users at NetBSD.org
editors/sandy                                pkgsrc-users at NetBSD.org
editors/TeXmacs                              pkgsrc-users at NetBSD.org
editors/tweak                                pkgsrc-users at NetBSD.org
editors/vim-lang                             morr at NetBSD.org
editors/xemacs                               hauke at NetBSD.org
editors/xemacs-current                       pkgsrc-users at NetBSD.org
editors/xemacs-current-nox11               3 pkgsrc-users at NetBSD.org
editors/xemacs-nox11                         hauke at NetBSD.org
emulators/aliados                            ivaniclixx at gmail.com
emulators/BasiliskII                         pkgsrc-users at NetBSD.org
emulators/bochs                              pkgsrc-users at NetBSD.org
emulators/ckmame                             wiz at NetBSD.org
emulators/darcnes                            pkgsrc-users at NetBSD.org
emulators/fmsx                               pkgsrc-users at NetBSD.org
emulators/hercules                           pkgsrc-users at NetBSD.org
emulators/ia64sim                            pkgsrc-users at NetBSD.org
emulators/klh10                              ryoon at NetBSD.org
emulators/libretro-catsfc                    pkgsrc-users at NetBSD.org
emulators/libretro-fceumm                    pkgsrc-users at NetBSD.org
emulators/libretro-gambatte                  pkgsrc-users at NetBSD.org
emulators/libretro-mednafen-pce-fast         pkgsrc-users at NetBSD.org
emulators/libretro-mupen64plus               pkgsrc-users at NetBSD.org
emulators/libretro-nestopia                  snj at NetBSD.org
emulators/libretro-picodrive                 pkgsrc-users at NetBSD.org
emulators/libretro-snes9x-next               pkgsrc-users at NetBSD.org
emulators/libretro-stella                    pkgsrc-users at NetBSD.org
emulators/libretro-vba-next                  pkgsrc-users at NetBSD.org
emulators/lrmi                               pkgsrc-users at NetBSD.org
emulators/p11                                packages at grumpf.hope-2000.org
emulators/palmosemulator                     dsainty at NetBSD.org
emulators/pcemu                              eric at fox.phoenix.az.us
emulators/pearpc                             pkgsrc-users at NetBSD.org
emulators/ski                                cherry at zyx.in
emulators/twin                               pkgsrc-users at NetBSD.org
emulators/ucon64                             pkgsrc-users at NetBSD.org
emulators/x48                                pkgsrc-users at NetBSD.org
emulators/xm7                                tech-pkg-ja at jp.NetBSD.org
emulators/xtrs                               pkgsrc-users at NetBSD.org
filesystems/glusterfs                      1 pkgsrc-users at NetBSD.org
filesystems/libntfs                        3 pkgsrc-users at NetBSD.org
filesystems/openafs                          gendalia at NetBSD.org
filesystems/openafs-devel                    gendalia at NetBSD.org
filesystems/perfuse                        2 manu at NetBSD.org
fonts/mglfonts                             1 fk200329 at fsinet.or.jp
fonts/mminstance                             jonb at NetBSD.org
fonts/oto                                    rxg at NetBSD.org
fonts/tex-kotex-base                       2 minskim at NetBSD.org
games/abuse                                  pkgsrc-users at NetBSD.org
games/amaze                                  pkgsrc-users at NetBSD.org
games/angband-tty                            pkgsrc-users at NetBSD.org
games/angband-x11                            pkgsrc-users at NetBSD.org
games/battalion                              pkgsrc-users at NetBSD.org
games/billardgl                              wojtek at 3miasto.net
games/bridge-hands                           pkgsrc-users at NetBSD.org
games/cbzone                                 pkgsrc-users at NetBSD.org
games/crafty                               2 pkgsrc-users at NetBSD.org
games/crossfire-server                       adam at NetBSD.org
games/flightgear-data                        pkgsrc-users at NetBSD.org
games/freeciv-server                         pkgsrc-users at NetBSD.org
games/frotz                                  seebs at plethora.net
games/fruit                                  pkgsrc-users at NetBSD.org
games/jetpack                                pkgsrc-users at NetBSD.org
games/knightcap                              pkgsrc-users at NetBSD.org
games/liquidwar                              pkgsrc-users at NetBSD.org
games/lnl                                    pkgsrc-users at NetBSD.org
games/moria                                  simonb at NetBSD.org
games/nethack-qt                           1 pooka at NetBSD.org
games/netmaj                                 pkgsrc-users at NetBSD.org
games/netmaze                                jdc at NetBSD.org
games/nighthawk                              pkgsrc-users at NetBSD.org
games/plib                                 3 pkgsrc-users at NetBSD.org
games/qstat                                  pkgsrc-users at NetBSD.org
games/quake                                  pkgsrc-users at NetBSD.org
games/quake6                                 pkgsrc-users at NetBSD.org
games/robotfindskitten                       pkgsrc-users at NetBSD.org
games/svb                                    pkgsrc-users at NetBSD.org
games/tads                                   pkgsrc-users at NetBSD.org
games/thew                                   ilfoglionascosto at netcat.it
games/tyrquake                               snj at NetBSD.org
games/vms-empire                             pkgsrc-users at NetBSD.org
games/xbat                                   pkgsrc-users at NetBSD.org
games/xblast                                 dsainty at NetBSD.org
games/xboard                                 simonb at NetBSD.org
games/xboing                                 pkgsrc-users at NetBSD.org
games/xbreaky                                pkgsrc-users at NetBSD.org
games/xdoom                                  pkgsrc-users at NetBSD.org
games/xemeraldia                             pkgsrc-users at NetBSD.org
games/xevil                                  snj at pobox.com
games/xgospel                                pkgsrc-users at NetBSD.org
games/xjewel                                 jdc at NetBSD.org
games/xjig                                   pkgsrc-users at NetBSD.org
games/xjump                                  pkgsrc-users at NetBSD.org
games/xnibbles                               pkgsrc-users at NetBSD.org
games/xpipeman                               pkgsrc-users at NetBSD.org
games/xscavenger                             pkgsrc-users at NetBSD.org
games/xtux                                   pkgsrc-users at NetBSD.org
geography/gama                               gdt at NetBSD.org
geography/garmin-utils                       gson at NetBSD.org
geography/gipfel                             johannes.hofmann at gmx.de
geography/gpsdrive                           pkgsrc-users at NetBSD.org
geography/kplex                              pkgsrc-users at NetBSD.org
geography/py-obspy                           wen at NetBSD.org
geography/qlandkartem                        jkunz at NetBSD.org
geography/vis5d+                             hdp at nmsu.edu
graphics/autopano-sift-C                   1 pkgsrc-users at NetBSD.org
graphics/cqcam                               pkgsrc-users at NetBSD.org
graphics/deforaos-icon-theme               2 khorben at defora.org
graphics/feh                                 pkgsrc-users at NetBSD.org
graphics/flickrnet                         1 pkgsrc-users at NetBSD.org
graphics/freeimage                         2 pkgsrc-users at NetBSD.org
graphics/gimp-docs-de                      1 pkgsrc-users at NetBSD.org
graphics/gimp-docs-es                      1 pkgsrc-users at NetBSD.org
graphics/gimp-docs-fr                      1 pkgsrc-users at NetBSD.org
graphics/gimp-docs-it                      1 pkgsrc-users at NetBSD.org
graphics/gimp-docs-ja                      1 pkgsrc-users at NetBSD.org
graphics/gimp-docs-ko                      1 pkgsrc-users at NetBSD.org
graphics/gimp-docs-nl                      1 pkgsrc-users at NetBSD.org
graphics/gimp-docs-nn                      1 pkgsrc-users at NetBSD.org
graphics/gimp-docs-ru                      1 pkgsrc-users at NetBSD.org
graphics/gimp-docs-sv                      1 pkgsrc-users at NetBSD.org
graphics/grap                                pkgsrc-users at NetBSD.org
graphics/hp2xx                               dmcmahill at NetBSD.org
graphics/jpegpixi                            gdt at NetBSD.org
graphics/libpgf                            1 pkgsrc-users at NetBSD.org
graphics/netpbm                           44 adam at NetBSD.org
graphics/ns-cult3d                           abs at NetBSD.org
graphics/opendis                             pkgsrc-users at NetBSD.org
graphics/oxygen-icons                     17 markd at NetBSD.org
graphics/py-cairo                        196 leuzi at trash.net
graphics/tesseract                           pkgsrc-users at NetBSD.org
graphics/vcg                                 pkgsrc-users at NetBSD.org
graphics/xmorph                              pkgsrc-users at NetBSD.org
graphics/zphoto                              pkgsrc-users at NetBSD.org
ham/dpbox                                    pkgsrc-users at NetBSD.org
ham/linpsk                                   pkgsrc-users at NetBSD.org
ham/tfkiss                                   pkgsrc-users at NetBSD.org
ham/tlf                                      pkgsrc-users at NetBSD.org
ham/tnt                                      pkgsrc-users at NetBSD.org
inputmethod/anthy                         14 taya at NetBSD.org
inputmethod/multiskkserv                     ryoon at NetBSD.org
inputmethod/sj3-client                     2 pkgsrc-users at NetBSD.org
inputmethod/sj3-server                     1 pkgsrc-users at NetBSD.org
inputmethod/skkfep                           pkgsrc-users at NetBSD.org
inputmethod/unicon-im                      1 rxg at NetBSD.org
inputmethod/xcin                             rxg at NetBSD.org
lang/algol68g                                olafs at cs.ru.nl
lang/baci                                    pkgsrc-users at NetBSD.org
lang/bwbasic                                 jspath at bcpl.net
lang/chicken                                 asau at inbox.ru
lang/cint                                    drochner at NetBSD.org
lang/clang                                28 adam.hoka at gmail.com
lang/classpath                             2 pkgsrc-users at NetBSD.org
lang/cparser                                 pkgsrc-users at NetBSD.org
lang/g95                                     wennmach at NetBSD.org
lang/gcc2                                    pkgsrc-users at NetBSD.org
lang/gcc34                                   pkgsrc-users at NetBSD.org
lang/gcc44                                   sbd at NetBSD.org
lang/gcc47                                 1 sbd at NetBSD.org
lang/gcc48                                 3 sbd at NetBSD.org
lang/gcc49                                 1 pkgsrc-users at NetBSD.org
lang/ghc                                   2 kristerw at NetBSD.org
lang/ghc7                                 84 pho at cielonegro.org
lang/gpc                                     pkgsrc-users at NetBSD.org
lang/gprolog                                 brook at nmsu.edu
lang/heirloom-awk                            cheusov at NetBSD.org
lang/kaffe                                 3 tonio at NetBSD.org
lang/kali                                    pkgsrc-users at NetBSD.org
lang/konoha                                  ryoon at NetBSD.org
lang/ksi                                     pkgsrc-users at NetBSD.org
lang/librep                                2 pkgsrc-users at NetBSD.org
lang/LuaJIT2                               4 alnsn at NetBSD.org
lang/mono-basic                            1 kefren at NetBSD.org
lang/mono2                                 6 kefren at NetBSD.org
lang/nhc98                                   kristerw at NetBSD.org
lang/nodejs                                  filip at joyent.com
lang/objc                                    pkgsrc-users at NetBSD.org
lang/ocaml                                80 jaapb at NetBSD.org
lang/open-cobol-ce                           ryoon at NetBSD.org
lang/ossp-js                                 pkgsrc-users at NetBSD.org
lang/parrot                                2 he at NetBSD.org
lang/pcc-current                             he at NetBSD.org
lang/pfe                                     asau at inbox.ru
lang/php54                               182 pkgsrc-users at NetBSD.org
lang/php55                               131 pkgsrc-users at NetBSD.org
lang/php56                               129 pkgsrc-users at NetBSD.org
lang/picoc                                   agc at NetBSD.org
lang/rexx-imc                                pkgsrc-users at NetBSD.org
lang/ruby18-base                          33 taca at NetBSD.org
lang/ruby193-base                        540 taca at NetBSD.org
lang/ruby200-base                        616 taca at NetBSD.org
lang/ruby21-base                         542 taca at NetBSD.org
lang/ruby22-base                         537 taca at NetBSD.org
lang/sather                                  pkgsrc-users at NetBSD.org
lang/sbcl                                    asau at NetBSD.org
lang/scheme48                                pkgsrc-users at NetBSD.org
lang/scm                                     pkgsrc-users at NetBSD.org
lang/spl                                     pkgsrc-users at NetBSD.org
lang/sr                                      is at NetBSD.org
lang/sun-jre6                              6 pkgsrc-users at NetBSD.org
lang/sun-jre7                             88 pkgsrc-users at NetBSD.org
lang/yap                                     pkgsrc-users at NetBSD.org
mail/cone                                    pkgsrc-users at NetBSD.org
mail/courier-mta                           1 pkgsrc-users at NetBSD.org
mail/cyrus-imapd23                           pkgsrc-users at NetBSD.org
mail/cyrus-imapd24                         1 pkgsrc-users at NetBSD.org
mail/dcc                                     petar at smokva.net
mail/distribute                              pkgsrc-users at NetBSD.org
mail/dnsbl-milter                            pkgsrc-users at NetBSD.org
mail/drac                                    pkgsrc-users at NetBSD.org
mail/elm-me                                  pkgsrc-users at NetBSD.org
mail/ezmlm                                   schmonz at NetBSD.org
mail/ezmlm-idx                             2 schmonz at NetBSD.org
mail/fdm                                     pkgsrc-users at NetBSD.org
mail/fix-mime-charset                        pkgsrc-users at NetBSD.org
mail/isync                                   tonio at NetBSD.org
mail/ja-mh                                   tech-pkg-ja at jp.NetBSD.org
mail/libetpan                             24 pkgsrc-users at NetBSD.org
mail/mailman                               2 pkgsrc-users at NetBSD.org
mail/mailwrapper                             pkgsrc-users at NetBSD.org
mail/metamail                              2 pkgsrc-users at NetBSD.org
mail/milter-regex                            jdarrow at NetBSD.org
mail/mopher                                  petar at smokva.net
mail/mush                                    christos at NetBSD.org
mail/mutt-kz                                 wiz at NetBSD.org
mail/newmail                                 andrew at ugh.net.au
mail/nmzmail                                 rodent at NetBSD.org
mail/opensmtpd                               ftigeot at wolfpond.org
mail/p5-razor-agents                         pkgsrc-users at NetBSD.org
mail/pfqueue                                 pkgsrc-users at NetBSD.org
mail/popa3d                                  pkgsrc-users at NetBSD.org
mail/relay-ctrl                              schmonz at NetBSD.org
mail/smtp-vilter                             marc at msys.ch
mail/smtpfeed                                pkgsrc-users at NetBSD.org
mail/spamass-milter                          gdt at NetBSD.org
mail/sqwebmail                             1 pkgsrc-users at NetBSD.org
mail/thunderbird-l10n                      1 pkgsrc-users at NetBSD.org
mail/xfmail                                  pkgsrc-users at NetBSD.org
math/aamath                                  esg at sdf.lonestar.org
math/eukleides                             1 ryoon at NetBSD.org
math/graphopt                                pkgsrc-users at NetBSD.org
math/mpfr                                484 pkgsrc-users at NetBSD.org
math/muparser                              2 pkgsrc-users at NetBSD.org
math/qalculate                            26 adam at NetBSD.org
math/R-clim.pact                             pkgsrc-users at NetBSD.org
math/R-combinat                            1 pkgsrc-users at NetBSD.org
math/R-GRASS                                 mchittur at cs.nmsu.edu
math/R-PHYLOGR                               pkgsrc-users at NetBSD.org
math/sc                                      eric at fox.phoenix.az.us
math/snns                                    pkgsrc-users at NetBSD.org
math/teapot                                  hauke at NetBSD.org
math/xlispstat                             1 pkgsrc-users at NetBSD.org
math/yorick                                  pkgsrc-users at NetBSD.org
mbone/mdd                                    manu at NetBSD.org
mbone/mdp                                    pkgsrc-users at NetBSD.org
mbone/rat                                    pkgsrc-users at NetBSD.org
mbone/vat                                    pkgsrc-users at NetBSD.org
mbone/vic                                    is at NetBSD.org
mbone/vic-devel                              sborrill at NetBSD.org
misc/chipmunk                                pkgsrc-users at NetBSD.org
misc/dpkg                                    pkgsrc-users at NetBSD.org
misc/dvtm                                    pkgsrc-users at NetBSD.org
misc/fbreader                                pkgsrc-users at NetBSD.org
misc/fd                                      pkgsrc-users at NetBSD.org
misc/hanzim                                  rxg at NetBSD.org
misc/ibutton-pdkit                           damon at brodiefamily.org
misc/libcec                                  pkgsrc-users at NetBSD.org
misc/lile                                    pkgsrc-users at NetBSD.org
misc/open2300                                martin at NetBSD.org
misc/open2300-mysql                          martin at NetBSD.org
misc/rhash                                 1 pkgsrc-users at NetBSD.org
misc/rlwrap                                  pkgsrc-users at NetBSD.org
misc/root                                    drochner at NetBSD.org
misc/rpm                                   1 pkgsrc-users at NetBSD.org
misc/rump                                  1 stacktic at NetBSD.org
misc/sch                                     pkgsrc-users at NetBSD.org
misc/stellarium                              pkgsrc-users at NetBSD.org
misc/team                                    pkgsrc-users at NetBSD.org
misc/toilet                                  schmonz at NetBSD.org
misc/udfclient                               reinoud at 13thmonkey.org
misc/vera                                    pkgsrc-users at NetBSD.org
misc/vfu                                     pkgsrc-users at NetBSD.org
misc/window                                  pkgsrc-users at NetBSD.org
misc/xjdic                                   pkgsrc-users at NetBSD.org
misc/zyGrib                                2 plunky at NetBSD.org
multimedia/dirac                             pkgsrc-users at NetBSD.org
multimedia/dvb-apps                          pkgsrc-users at NetBSD.org
multimedia/dvdbackup                         pkgsrc-users at NetBSD.org
multimedia/fxtv                            2 pkgsrc-users at NetBSD.org
multimedia/ggrab                             pkgsrc-users at NetBSD.org
multimedia/gmediaserver                      pkgsrc-users at NetBSD.org
multimedia/hwdecode-demos                    pkgsrc-users at NetBSD.org
multimedia/libdv-tools                       tech-pkg-ja at jp.NetBSD.org
multimedia/libflash                        1 pkgsrc-users at NetBSD.org
multimedia/libmpeg3                          pkgsrc-users at NetBSD.org
multimedia/openquicktime                     pkgsrc-users at NetBSD.org
multimedia/schroedinger                    1 pkgsrc-users at NetBSD.org
multimedia/tstools                           tech-multimedia at NetBSD.org
multimedia/tvtime                            pkgsrc-users at NetBSD.org
multimedia/xanim                             pkgsrc-users at NetBSD.org
multimedia/xawtv                             pkgsrc-users at NetBSD.org
net/3proxy                                   pkgsrc-users at NetBSD.org
net/aiccu                                    pkgsrc-users at NetBSD.org
net/arpd                                     pkgsrc-users at NetBSD.org
net/arpwatch                                 bouyer at NetBSD.org
net/batchftp                                 pkgsrc-users at NetBSD.org
net/bftpd                                    pkgsrc-users at NetBSD.org
net/bridged                                  pkgsrc-users at NetBSD.org
net/bsddip                                   pkgsrc-users at NetBSD.org
net/chksniff                                 pkgsrc-users at NetBSD.org
net/choparp                                  dbj at NetBSD.org
net/chrony                                   hannken at NetBSD.org
net/citrix_ica                               sborrill at NetBSD.org
net/cntlm                                    pkgsrc-users at NetBSD.org
net/csup                                     imil at gcu.info
net/dante                                    pkgsrc-users at NetBSD.org
net/DarwinStreamingServer                    adam at NetBSD.org
net/dctc                                   1 pkgsrc-users at NetBSD.org
net/delegate                                 pkgsrc-users at NetBSD.org
net/dgd                                      sco at openface.ca
net/dhcpcd                                 3 roy at NetBSD.org
net/dhisd                                    pkgsrc-users at NetBSD.org
net/dnscap                                   agc at NetBSD.org
net/dnsmasq                                  pkgsrc-users at NetBSD.org
net/dnstracer                                pkgsrc-users at NetBSD.org
net/dynipclient                              pkgsrc-users at NetBSD.org
net/entropy                                  pkgsrc-users at NetBSD.org
net/ettercap                                 pkgsrc-users at NetBSD.org
net/flodo                                    pkgsrc-users at NetBSD.org
net/freeDiameter                             pkgsrc-users at NetBSD.org
net/gated                                    pkgsrc-users at NetBSD.org
net/gethost                                  reed at reedmedia.net
net/host                                     pkgsrc-users at NetBSD.org
net/hping3                                   pettai at NetBSD.org
net/iana-etc                                 christos at NetBSD.org
net/IglooFTP                                 rxg at NetBSD.org
net/inadyn                                   pkgsrc-users at NetBSD.org
net/iodine                                   mwilhelmy at users.sourceforge.net
net/isic                                     pkgsrc-users at NetBSD.org
net/istgt                                    sborrill at NetBSD.org
net/jumpgate                                 pkgsrc-users at NetBSD.org
net/kiax                                     reinoud at NetBSD.org
net/kismet                                   pkgsrc-users at NetBSD.org
net/knock                                    adam at NetBSD.org
net/knot                                     pettai at NetBSD.org
net/kphone                                   pkgsrc-users at NetBSD.org
net/latd                                     fredette at theory.lcs.mit.edu
net/lftp                                     pettai at NetBSD.org
net/libnipper                              1 pkgsrc-users at NetBSD.org
net/libsscript                               pkgsrc-users at NetBSD.org
net/libtorrent                             1 tron at NetBSD.org
net/libtrace                                 pkgsrc-users at NetBSD.org
net/lldpd                                    pkgsrc-users at NetBSD.org
net/maradns                                  ddeus.pkgsrc at mailnull.com
net/mcast-tools                              is at NetBSD.org
net/md-whois                                 reed at reedmedia.net
net/microdc2                                 d.den.brok at uni-bonn.de
net/miniupnpd                                pkgsrc-users at NetBSD.org
net/mouse-pppoe                              mouse at NetBSD.org
net/mrstat                                   abs at absd.org
net/mrt                                      pkgsrc-users at NetBSD.org
net/msdl                                     pkgsrc-users at NetBSD.org
net/nagios-plugin-dnsrbl                   1 manu at NetBSD.org
net/nagios-plugin-dotpid                   1 manu at NetBSD.org
net/nagios-plugin-dumpdates                1 manu at NetBSD.org
net/nagios-plugin-milter                   1 manu at NetBSD.org
net/nagios-plugin-raidctl                  1 manu at NetBSD.org
net/nagios-plugin-syncrepl                 1 manu at NetBSD.org
net/nasd                                     pkgsrc-users at NetBSD.org
net/netcat-openbsd                           pkgsrc-users at NetBSD.org
net/nprobe                                   pkgsrc-users at NetBSD.org
net/ns                                       minskim at NetBSD.org
net/nstx                                     pkgsrc-users at NetBSD.org
net/ntp4                                     pkgsrc-users at NetBSD.org
net/ocsync                                   ryoon at NetBSD.org
net/omniNotify                               pkgsrc-users at NetBSD.org
net/openconnect                              pkgsrc-users at NetBSD.org
net/openslp                               36 pkgsrc-users at NetBSD.org
net/openvpn                                1 pkgsrc-users at NetBSD.org
net/openvpn-acct-wtmpx                       manu at NetBSD.org
net/openwbem                                 pkgsrc-users at NetBSD.org
net/ORBit                                    pkgsrc-users at NetBSD.org
net/ortp                                   3 zul at netbsd-fr.org
net/p5-eBay-API                              pkgsrc-users at NetBSD.org
net/p5-FusionInventory-Agent-Task-Network    bouyer at NetBSD.org
net/p5-Net-Interface                         pkgsrc-users at NetBSD.org
net/p5-Net-RawIP                             bouyer at NetBSD.org
net/packit                                   pkgsrc-users at NetBSD.org
net/parpd                                    roy at NetBSD.org
net/pconsole                                 pkgsrc-users at NetBSD.org
net/pim6dd                                   pkgsrc-users at NetBSD.org
net/pim6sd                                   pkgsrc-users at NetBSD.org
net/poptop                                   pkgsrc-users at NetBSD.org
net/portmap                                  reed at reedmedia.net
net/pptp                                     pkgsrc-users at NetBSD.org
net/radiusd-cistron                          pkgsrc-users at NetBSD.org
net/rdist6                                 2 christos at NetBSD.org
net/rp-l2tp                                  pkgsrc-users at NetBSD.org
net/rp-pppoe                                 pkgsrc-users at NetBSD.org
net/samba                                 27 pkgsrc-users at NetBSD.org
net/scamper                                  mjl at luckie.org.nz
net/ser                                      pkgsrc-users at NetBSD.org
net/sharity-light                            pkgsrc-users at NetBSD.org
net/socket++                                 mchittur at cs.nmsu.edu
net/socks4                                   pkgsrc-users at NetBSD.org
net/socks5                                 1 ambrisko at ambrisko.roble.com
net/spegla                                   jnilsson at ludd.luth.se
net/spread                                 2 pkgsrc-users at NetBSD.org
net/sslh                                     pkgsrc-users at NetBSD.org
net/ssmping                                  is at NetBSD.org
net/sup                                      christos at NetBSD.org
net/tcl-scotty                             1 pkgsrc-users at NetBSD.org
net/tcpreplay                                pkgsrc-users at NetBSD.org
net/tightvnc                                 pkgsrc-users at NetBSD.org
net/tinc                                     tonnerre at NetBSD.org
net/trickle                                  pkgsrc-users at NetBSD.org
net/ttt                                      pkgsrc-users at NetBSD.org
net/u6rd                                     kamada at nanohz.org
net/udpcast                                  pkgsrc-users at NetBSD.org
net/unworkable                               agc at NetBSD.org
net/uucp                                     pkgsrc-users at NetBSD.org
net/vnc                                      pkgsrc-users at NetBSD.org
net/vtun                                     pkgsrc-users at NetBSD.org
net/waste                                    pkgsrc-users at NetBSD.org
net/wide-dhcpv6                              pkgsrc-users at NetBSD.org
net/wistumbler                               pkgsrc-users at NetBSD.org
net/wmget                                    pkgsrc-users at NetBSD.org
net/wminet                                   coto at core.de
net/wpa_gui                                  pkgsrc-users at NetBSD.org
net/wpa_supplicant                         2 reed at reedmedia.net
net/wu-ftpd                                  pkgsrc-users at NetBSD.org
net/xl2tpd                                   christos at NetBSD.org
net/xorp                                     pkgsrc-users at NetBSD.org
news/nntpcache                               pkgsrc-users at NetBSD.org
news/tin                                     pkgsrc-users at NetBSD.org
parallel/dqs                                 pkgsrc-users at NetBSD.org
parallel/mpi-ch                            4 asau at inbox.ru
parallel/openmpi                             asau at inbox.ru
parallel/threadingbuildingblocks             asau at inbox.ru
pkgtools/pkg                                 khorben at defora.org
pkgtools/pkg_select                          imil at gcu.info
pkgtools/pkgin                             4 imil at NetBSD.org
print/cups-pdf                               pkgsrc-users at NetBSD.org
print/foomatic4-db                           pkgsrc-users at NetBSD.org
print/foomatic4-filters                    2 pkgsrc-users at NetBSD.org
print/hpijs                                  pkgsrc-users at NetBSD.org
print/LPRng-core                           1 pkgsrc-users at NetBSD.org
print/pdvipsk                              1 pkgsrc-users at NetBSD.org
print/pnm2ppa                                pkgsrc-users at NetBSD.org
print/poppler-qt                           1 reed at reedmedia.net
print/tex-bibtex8                          1 cheusov at NetBSD.org
security/aide06                              pkgsrc-users at NetBSD.org
security/asignify                            youri at NetBSD.org
security/beecrypt                            pkgsrc-users at NetBSD.org
security/dcfldd                              pettai at NetBSD.org
security/dsniff-nox11                      1 pkgsrc-users at NetBSD.org
security/fail2ban                            nils at netbsd.org
security/fwbuilder                           pkgsrc-users at NetBSD.org
security/google-authenticator                jmmv at NetBSD.org
security/hashcash                            pkgsrc-users at NetBSD.org
security/heimdal                           1 pkgsrc-users at NetBSD.org
security/ipv6-toolkit                        agc at NetBSD.org
security/john                                pkgsrc-users at NetBSD.org
security/keepassx                            pkgsrc-users at NetBSD.org
security/kstart                              jakllsch at NetBSD.org
security/lasso                             5 manu at NetBSD.org
security/libbf                               pkgsrc-users at NetBSD.org
security/libgfshare                          agc at NetBSD.org
security/libmultigest                      2 agc at NetBSD.org
security/libnetpgpverify                     agc at NetBSD.org
security/libprelude-perl                     pkgsrc-users at NetBSD.org
security/libpreludedb-perl                   pkgsrc-users at NetBSD.org
security/libsodium                        14 csosstudy at gmail.com
security/lsh                                 pkgsrc-users at NetBSD.org
security/mit-krb5                          4 tez at NetBSD.org
security/mit-krb5-appl                     1 tez at NetBSD.org
security/mozilla-rootcerts-openssl           dholland at NetBSD.org
security/multigest                           agc at NetBSD.org
security/munge                             1 asau at inbox.ru
security/MyPasswordSafe                      pkgsrc-users at NetBSD.org
security/netpgp                              agc at NetBSD.org
security/netpgpverify                        agc at NetBSD.org
security/oath-toolkit                        pettai at NetBSD.org
security/openpam                             joerg at NetBSD.org
security/openssh                             pkgsrc-users at NetBSD.org
security/p5-Net-DNS-SEC                    5 pkgsrc-users at NetBSD.org
security/pakemon                             pkgsrc-users at NetBSD.org
security/pam-mkhomedir                       tonnerre at NetBSD.org
security/pam-pgsql                           pkgsrc-users at NetBSD.org
security/pam-pwauth_suid                     drochner at NetBSD.org
security/pam-radius                          pkgsrc-users at NetBSD.org
security/pam-tacplus                         pettai at NetBSD.org
security/pev                                 khorben at defora.org
security/pgp2                              1 pkgsrc-users at NetBSD.org
security/pkcs11-helper                     1 gdt at ir.bbn.com
security/prelude-correlator                  pkgsrc-users at NetBSD.org
security/prelude-pflogger                    pkgsrc-users at NetBSD.org
security/py-cryptkit                         pkgsrc-users at NetBSD.org
security/py-Kerberos                         pkgsrc-users at NetBSD.org
security/racoon2                             kamada at nanohz.org
security/rainbowcrack                        dhowland at users.sourceforge.net
security/sfs                                 pkgsrc-users at NetBSD.org
security/snoopy                              imil at NetBSD.org
security/srp_client                          pkgsrc-users at NetBSD.org
security/sslsplit                            pkgsrc-users at NetBSD.org
security/steghide                            agc at NetBSD.org
security/yara                              1 pettai at NetBSD.org
security/zkt                                 pettai at NetBSD.org
shells/ast-ksh                               pkgsrc-users at NetBSD.org
shells/es                                    pkgsrc-users at NetBSD.org
shells/fish                                  pkgsrc-users at NetBSD.org
shells/static-tcsh                           kim at NetBSD.org
sysutils/acpica-utils                      5 pkgsrc-users at NetBSD.org
sysutils/amtterm                             pkgsrc-users at NetBSD.org
sysutils/attr                                pkgsrc-users at NetBSD.org
sysutils/bacula                            2 pkgsrc-users at NetBSD.org
sysutils/bacula-clientonly                   pkgsrc-users at NetBSD.org
sysutils/bbsload                             pino at dohd.org
sysutils/bup                                 wiz at NetBSD.org
sysutils/collectd                         20 filip at joyent.com
sysutils/conky                               imil at NetBSD.org
sysutils/dbus-glib                      1013 pkgsrc-users at NetBSD.org
sysutils/direvent                            pkgsrc-users at NetBSD.org
sysutils/dvd+rw-tools                      2 pkgsrc-users at NetBSD.org
sysutils/e2fsprogs                           pkgsrc-users at NetBSD.org
sysutils/entr                                pkgsrc-users at NetBSD.org
sysutils/fakeroot                            pkgsrc-users at NetBSD.org
sysutils/fix4SA110rev2                       abs at NetBSD.org
sysutils/gnometoaster                        pkgsrc-users at NetBSD.org
sysutils/grub                              1 pkgsrc-users at NetBSD.org
sysutils/heirloom-ls                         cheusov at NetBSD.org
sysutils/heirloom-shl                        cheusov at NetBSD.org
sysutils/idesk                               pkgsrc-users at NetBSD.org
sysutils/ipa                               1 simon at comsys.ntu-kpi.kiev.ua
sysutils/lastfss                             manu at NetBSD.org
sysutils/libisoburn                          ryoon at NetBSD.org
sysutils/libvirt                           1 agc at NetBSD.org
sysutils/lockf                               pkgsrc-users at NetBSD.org
sysutils/memconf                             shattered at NetBSD.org
sysutils/menu-cache                          pkgsrc-users at NetBSD.org
sysutils/mesure                              pkgsrc-users at NetBSD.org
sysutils/mrsync                              agc at NetBSD.org
sysutils/mtscan                              pkgsrc-users at NetBSD.org
sysutils/mtx                               3 bouyer at NetBSD.org
sysutils/p5-Sys-Utmp                         pkgsrc-users at NetBSD.org
sysutils/pidof                               pkgsrc-users at NetBSD.org
sysutils/pluged                              pkgsrc-users at NetBSD.org
sysutils/pscpug                              bartosz.kuzma at gmail.com
sysutils/py-attic                            pkgsrc-users at NetBSD.org
sysutils/qlogtools                           pkgsrc-users at NetBSD.org
sysutils/radeontool                          pkgsrc-users at NetBSD.org
sysutils/same                                pkgsrc-users at NetBSD.org
sysutils/scsiinfo                            pkgsrc-users at NetBSD.org
sysutils/sformat                             Andreas.Hallmann at tiscali.de
sysutils/slocate                             pkgsrc-users at NetBSD.org
sysutils/socket                              pkgsrc-users at NetBSD.org
sysutils/strace                              christos at NetBSD.org
sysutils/sysinfo                             pkgsrc-users at NetBSD.org
sysutils/systester                           pkgsrc-users at NetBSD.org
sysutils/tits                                pkgsrc-users at NetBSD.org
sysutils/tkdesk                              pkgsrc-users at NetBSD.org
sysutils/torsmo                              pkgsrc-users at NetBSD.org
sysutils/toshutils                           pkgsrc-users at NetBSD.org
sysutils/tpb                                 soren at NetBSD.org
sysutils/ups-nut                           3 pkgsrc-users at NetBSD.org
sysutils/usbutil                             lennart at augustsson.net
sysutils/vifm                                pkgsrc-users at NetBSD.org
sysutils/webmin                           42 pkgsrc-users at NetBSD.org
sysutils/whowatch                            pkgsrc-users at NetBSD.org
sysutils/wmSMPmon                            joel at carnat.net
sysutils/xbattbar                            pkgsrc-users at NetBSD.org
sysutils/xfm                                 rxg at NetBSD.org
sysutils/xfsprogs                            pkgsrc-users at NetBSD.org
sysutils/xnc                                 pkgsrc-users at NetBSD.org
textproc/aspell-csb                          rodent at NetBSD.org
textproc/aspell-fo                           pkgsrc-users at NetBSD.org
textproc/aspell-is                           rodent at NetBSD.org
textproc/aspell-nb                           pkgsrc-users at NetBSD.org
textproc/biblook                             pkgsrc-users at NetBSD.org
textproc/cabocha                           6 pkgsrc-users at NetBSD.org
textproc/convertlit                        1 pkgsrc-users at NetBSD.org
textproc/dict-dictionaries                   pkgsrc-users at NetBSD.org
textproc/docbook2mdoc                        pkgsrc-users at NetBSD.org
textproc/expatobjc                           pkgsrc-users at NetBSD.org
textproc/grepcidr                            pkgsrc-users at netbsd.org
textproc/GutenMark                           pkgsrc-users at NetBSD.org
textproc/heirloom-doctools                   carsten.kunze at arcor.de
textproc/ispell-en_GB                        pkgsrc-users at NetBSD.org
textproc/ispell-pl                           adam at NetBSD.org
textproc/ja-groff                          2 pkgsrc-users at NetBSD.org
textproc/jsoncpp                           1 pkgsrc-users at NetBSD.org
textproc/libpinyin                         1 pkgsrc-users at NetBSD.org
textproc/libplist                          8 pkgsrc-users at NetBSD.org
textproc/link-grammar                        pkgsrc-users at NetBSD.org
textproc/lq-sp                             1 norm at sandbox.org.uk
textproc/lua-expat                           pkgsrc-users at NetBSD.org
    lua53-expat-1.3.0nb2                     pkgsrc-users at NetBSD.org
textproc/ndtpd                               uebayasi at NetBSD.org
textproc/pod2mdoc                            pkgsrc-users at NetBSD.org
textproc/qsubst                              pkgsrc-users at NetBSD.org
textproc/regxml                              pkgsrc-users at NetBSD.org
textproc/rtfm                                pkgsrc-users at NetBSD.org
textproc/tcl-expat                           pkgsrc-users at NetBSD.org
textproc/texi2mdoc                           pkgsrc-users at NetBSD.org
textproc/unroff                              pkgsrc-users at NetBSD.org
textproc/vis                                 christos at NetBSD.org
textproc/xml2doc                             pkgsrc-users at NetBSD.org
time/emiclock                                minoura at NetBSD.org
time/glclock                                 pkgsrc-users at NetBSD.org
time/gnutime                                 reed at reedmedia.net
time/p5-Date-Extract                       1 ryoon at NetBSD.org
time/phoon                                   is at NetBSD.org
time/vixie-cron                              reed at reedmedia.net
time/xalarm                                  pkgsrc-users at NetBSD.org
time/xchrono                                 pkgsrc-users at NetBSD.org
time/xonclock                                pkgsrc-users at NetBSD.org
wm/3ddesktop                                 pkgsrc-users at NetBSD.org
wm/bspwm                                     youri at NetBSD.org
wm/jwm                                       pkgsrc-users at NetBSD.org
wm/notion                                    wiz at NetBSD.org
wm/pekwm                                     gls at NetBSD.org
wm/piewm                                     pkgsrc-users at NetBSD.org
wm/py-tyle                                   pkgsrc-users at NetBSD.org
wm/spectrwm                                  pkgsrc-users at NetBSD.org
wm/uwm                                       pkgsrc-users at NetBSD.org
wm/wmii                                      pkgsrc-users at NetBSD.org
www/ap-auth-kerb                             markd at NetBSD.org
www/ap2-cband                                pkgsrc-users at NetBSD.org
www/ap2-suphp                                pkgsrc-users at NetBSD.org
www/bozohttpd                              3 mrg at eterna.com.au
www/c-icap-modules                           adam at NetBSD.org
www/cgit                                     pkgsrc-users at NetBSD.org
www/crawl                                    pkgsrc-users at NetBSD.org
www/ffproxy                                  niklas at noxa.de
www/firefox-l10n                             pkgsrc-users at NetBSD.org
www/firefox31-l10n                           pkgsrc-users at NetBSD.org
www/firefox38-l10n                           pkgsrc-users at NetBSD.org
www/h2o                                      pkgsrc-users at NetBSD.org
www/htmlcxx                                1 pkgsrc-users at NetBSD.org
www/kannel                                   pkgsrc-users at NetBSD.org
www/mono-xsp                               2 kefren at NetBSD.org
www/nvu                                      pkgsrc-users at NetBSD.org
www/p5-Catalyst-Engine-HTTP-Prefork          pkgsrc-users at NetBSD.org
www/p5-Catalyst-Plugin-AutoRestart           pkgsrc-users at NetBSD.org
www/p5-HTMLObject                            pkgsrc-users at NetBSD.org
www/p5-libapreq                              pkgsrc-users at NetBSD.org
www/py-cherrypy                            3 pkgsrc-users at NetBSD.org
www/py-django                             72 joerg at NetBSD.org
www/py-django14                              joerg at NetBSD.org
www/py-tornado                             1 imil at NetBSD.org
    py33-tornado-4.2                       1 imil at NetBSD.org
www/qDecoder                                 jypak at users.sourceforge.net
www/screws                                   pkgsrc-users at NetBSD.org
www/sqtop                                    rodent at NetBSD.org
www/squidpurge                               gdt at NetBSD.org
www/swiggle                                  snj at NetBSD.org
www/swill                                    pkgsrc-users at NetBSD.org
www/urlget                                   kim at tac.nyc.ny.us
www/w3c-httpd                                pkgsrc-users at NetBSD.org
www/wml                                      kim at tac.nyc.ny.us
www/www6to4                                  feico at pasta.cs.uit.no
x11/autocutsel                               ryoon at NetBSD.org
x11/dtx11session                             pkgsrc-users at NetBSD.org
x11/efltk                                  1 pkgsrc-users at NetBSD.org
x11/elementary                             2 joerg at NetBSD.org
x11/fbdesk                                   pkgsrc-users at NetBSD.org
x11/filerunner                               pkgsrc-users at NetBSD.org
x11/fltk2                                  1 pkgsrc-users at NetBSD.org
x11/fox                                    2 pkgsrc-users at NetBSD.org
x11/hanterm                                  pkgsrc-users at NetBSD.org
x11/matchbox-panel                         2 pkgsrc-users at NetBSD.org
x11/modular-xorg-server                   58 pkgsrc-users at NetBSD.org
x11/motif                                 69 pkgsrc-users at NetBSD.org
x11/qt4-creator                              fw at submerge.ch
x11/qt4-docs                               1 pkgsrc-users at NetBSD.org
x11/qt5-qtdeclarative                     22 ryoon at NetBSD.org
x11/slock                                    reed at reedmedia.net
x11/tk-Tix                                 1 he at NetBSD.org
x11/wmavgload                                pkgsrc-users at NetBSD.org
x11/wmfstatus                                pkgsrc-users at NetBSD.org
x11/x3270                                    gls at NetBSD.org
x11/XaoS                                     pkgsrc-users at NetBSD.org
x11/xdesktopwaves                            pkgsrc-users at NetBSD.org
x11/xrootconsole                             inittab at unixdev.net
x11/xscreensaver                           7 pkgsrc-users at NetBSD.org
x11/xview-lib                              4 pkgsrc-users at NetBSD.org

From venture37 at gmail.com  Sun Jul 12 11:46:58 2015
From: venture37 at gmail.com (Sevan / Venture37)
Date: Sun, 12 Jul 2015 12:46:58 +0100
Subject: [OmniOS-discuss] pkgsrc-current OmniOS 170cea2/i386 2015-07-09 21:35
Message-ID: <CA+U3Mf6sgawwn=b5O081KVbp+Mz60scN2Qk1fLZTRwBUouMvGw@mail.gmail.com>

System ld.cache was modified to add the GCC lib directory in search
path with crle(1)
This resolved the previous breakage with the gettext installed from
the OmniOS IPS repo and allowed us to progress. Unfortunately, it
looks like there was a permission issue with /var/tmp on the zone I
was using to build so there's a lots of false breakages. I've
corrected the issue and restarted.

pkgsrc bulk build report
========================

OmniOS 170cea2/i386
Compiler: gcc

Build start: 2015-07-09 21:35
Build end:   2015-07-12 07:59

Full report: http://pkgsrc.geeklan.co.uk/reports/current/OmniOS/20150709.2135/meta/report.html
Machine readable version:
http://pkgsrc.geeklan.co.uk/reports/current/OmniOS/20150709.2135/meta/report.bz2

Total number of packages:      16536
  Successfully built:           8961
  Failed to build:               890
  Depending on failed package:  5968
  Explicitly broken or masked:   658
  Depending on masked package:    59

Packages breaking the most other packages

Package                               Breaks Maintainer
-------------------------------------------------------------------------
devel/gobject-introspection             1718 pkgsrc-users at NetBSD.org
sysutils/dbus-glib                      1013 pkgsrc-users at NetBSD.org
lang/ruby200-base                        616 taca at NetBSD.org
lang/ruby21-base                         542 taca at NetBSD.org
lang/ruby193-base                        540 taca at NetBSD.org
lang/ruby22-base                         537 taca at NetBSD.org
math/mpfr                                484 pkgsrc-users at NetBSD.org
audio/libsndfile                         438 pkgsrc-users at NetBSD.org
devel/yasm                               404 shaun at inerd.com
devel/boost-headers                      388 pkgsrc-users at NetBSD.org

Build failures

Package                               Breaks Maintainer
-------------------------------------------------------------------------
archivers/unalz                              pkgsrc-users at NetBSD.org
audio/akode-plugins-oss                      pkgsrc-users at NetBSD.org
audio/akode-plugins-sun                      pkgsrc-users at NetBSD.org
audio/alsa-plugins-oss                       pkgsrc-users at NetBSD.org
audio/amp                                    pkgsrc-users at NetBSD.org
audio/aumix                                  trevor at jpj.net
audio/cam                                    hubertf at NetBSD.org
audio/daapd                                  nathanw at NetBSD.org
audio/dap                                    pkgsrc-users at NetBSD.org
audio/esound                             379 pkgsrc-users at NetBSD.org
audio/freealut                             3 pkgsrc-users at NetBSD.org
audio/gramofile                              pkgsrc-users at NetBSD.org
audio/liba52                              57 pkgsrc-users at NetBSD.org
audio/libao-oss                              pkgsrc-users at NetBSD.org
audio/libao-sun                              pkgsrc-users at NetBSD.org
audio/libdca                              43 shattered at NetBSD.org
audio/libebur128                           1 pkgsrc-users at NetBSD.org
audio/libsndfile                         438 pkgsrc-users at NetBSD.org
audio/libvisual0.2-plugins                   pkgsrc-users at NetBSD.org
audio/maplay                                 pkgsrc-users at NetBSD.org
audio/mixer.app                              ptiJo at noos.fr
audio/mp3blaster                             rxg at NetBSD.org
audio/mpg123                              21 martin at NetBSD.org
audio/mppenc                                 pkgsrc-users at NetBSD.org
audio/nas                                256 markd at NetBSD.org
audio/nosefart                               dgriffi at cs.csubak.edu
audio/nspmod                                 pkgsrc-users at NetBSD.org
audio/ocp                                    shattered at NetBSD.org
audio/pd                                     pkgsrc-users at NetBSD.org
audio/portaudio                            4 pkgsrc-users at NetBSD.org
audio/portaudio-devel                     30 pkgsrc-users at NetBSD.org
audio/rexima                                 pkgsrc-users at NetBSD.org
audio/rplay                                1 pkgsrc-users at NetBSD.org
audio/rtunes                                 pkgsrc-users at NetBSD.org
audio/sidplay                                pkgsrc-users at NetBSD.org
audio/sidplay2                               pkgsrc-users at NetBSD.org
audio/spiralloops                            pkgsrc-users at NetBSD.org
audio/spiralsynth                            pkgsrc-users at NetBSD.org
audio/splay                                  pkgsrc-users at NetBSD.org
audio/tcl-snack                            1 gson at NetBSD.org
audio/tfmxplay                               pkgsrc-users at NetBSD.org
audio/tracker                                pkgsrc-users at NetBSD.org
audio/wmmixer                              2 pr at alles.prima.de
audio/wmsmixer                               pkgsrc-users at NetBSD.org
audio/wsoundserver                         1 pkgsrc-users at NetBSD.org
audio/xcdplayer                              pkgsrc-users at NetBSD.org
audio/xmix                                   pkgsrc-users at NetBSD.org
audio/xmms                                16 pkgsrc-users at NetBSD.org
audio/xsidplay                               pkgsrc-users at NetBSD.org
audio/xwave                                  pkgsrc-users at NetBSD.org
benchmarks/glmark2                           prlw1 at cam.ac.uk
benchmarks/iozone                            pkgsrc-users at NetBSD.org
biology/chemical-mime-data                 1 kamel.derouiche at gmail.com
biology/clustalw                             babafou at babafou.eu.org
biology/mopac                                pkgsrc-users at NetBSD.org
biology/mpqc                                 asau at inbox.ru
cad/qcad-partlibrary                         pkgsrc-users at NetBSD.org
chat/ctrlproxy                               pkgsrc-users at NetBSD.org
chat/gtmess                                  luapzz at gmail.com
chat/i2cbd                                   pkgsrc-users at NetBSD.org
chat/inspircd                                pkgsrc-users at NetBSD.org
chat/licq-core                             2 pkgsrc-users at NetBSD.org
chat/naim                                    pkgsrc-users at NetBSD.org
chat/phone                                   christos at NetBSD.org
chat/quirc                                   pkgsrc-users at NetBSD.org
chat/tkicb                                   pkgsrc-users at NetBSD.org
comms/asterisk                             1 jnemeth at NetBSD.org
comms/asterisk10                             jnemeth at NetBSD.org
comms/asterisk18                             jnemeth at NetBSD.org
comms/dl-ezkit                               pkgsrc-users at NetBSD.org
comms/hylafax                              1 pkgsrc-users at NetBSD.org
comms/libopensync-plugin-file                pkgsrc-users at NetBSD.org
comms/msynctool                              pkgsrc-users at NetBSD.org
comms/tn3270                                 pkgsrc-users at NetBSD.org
converters/p5-MARC-Charset                 1 pkgsrc-users at NetBSD.org
converters/TECkit                         11 minskim at NetBSD.org
cross/binutils                             2 pkgsrc-users at NetBSD.org
cross/cc65                                   pkgsrc-users at NetBSD.org
cross/GenFw                                  jakllsch at NetBSD.org
cross/h8300-hms-binutils                   1 pkgsrc-users at NetBSD.org
cross/icdprog                                pkgsrc-users at NetBSD.org
cross/mingw-binutils                       2 pkgsrc-users at NetBSD.org
cross/nios2-gcc3                             pkgsrc-users at NetBSD.org
cross/nios2-gcc41                            pkgsrc-users at NetBSD.org
cross/uisp                                   pkgsrc-users at NetBSD.org
databases/db5                              3 pkgsrc-users at NetBSD.org
databases/gnats                              pkgsrc-users at NetBSD.org
databases/libgda                           5 pkgsrc-users at NetBSD.org
databases/mariadb55-server                   pkgsrc-users at NetBSD.org
databases/mysql51-client                   3 pkgsrc-users at NetBSD.org
databases/mysql55-server                   3 pkgsrc-users at NetBSD.org
databases/nss-pgsql                          pkgsrc-users at NetBSD.org
databases/odbc-postgresql                    adam at NetBSD.org
databases/pgtcl                              pkgsrc-users at NetBSD.org
databases/pgtclng                            pkgsrc-users at NetBSD.org
databases/postgresql-postgis2                gdt at NetBSD.org
    postgresql90-postgis-2.1.8               gdt at NetBSD.org
databases/postgresql84-uuid                  brook at nmsu.edu
databases/postgresql90-uuid                  brook at nmsu.edu
databases/qdbm-plus                          pkgsrc-users at NetBSD.org
databases/sqsh                               pkgsrc-users at NetBSD.org
databases/sqsh-x11                           pkgsrc-users at NetBSD.org
databases/xsqlmenu                           pkgsrc-users at NetBSD.org
devel/aegis                                  pkgsrc-users at NetBSD.org
devel/afl                                    pkgsrc-users at NetBSD.org
devel/ald                                    pkgsrc-users at NetBSD.org
devel/astyle                                 pkgsrc-users at NetBSD.org
devel/bcc                                    pkgsrc-users at NetBSD.org
devel/bmake                                9 joerg at NetBSD.org
devel/boost-docs                           3 pkgsrc-users at NetBSD.org
devel/boost-headers                      388 pkgsrc-users at NetBSD.org
devel/bouml                                  pkgsrc-users at NetBSD.org
devel/bullet                                 pkgsrc-users at NetBSD.org
devel/cgen                                   ryoon at NetBSD.org
devel/concurrencykit                         pkgsrc-users at NetBSD.org
devel/cook                                 1 groo at NetBSD.org
devel/cssc                                   pkgsrc-users at NetBSD.org
devel/cvs-fast-export                        apb at NetBSD.org
devel/cvsync                                 pkgsrc-users at NetBSD.org
devel/editline                             7 ryoon at NetBSD.org
devel/elf                                    pkgsrc-users at NetBSD.org
devel/elftoolchain                           agc at NetBSD.org
devel/fhist                                  pkgsrc-users at NetBSD.org
devel/fortran-utils                          pkgsrc-users at NetBSD.org
devel/gcvs                                   pkgsrc-users at NetBSD.org
devel/gdb                                    pkgsrc-users at NetBSD.org
devel/gobject-introspection             1718 pkgsrc-users at NetBSD.org
devel/hdf5-c++                               alnsn at NetBSD.org
devel/ht                                     pkgsrc-users at NetBSD.org
devel/hugs-unix                              pkgsrc-users at NetBSD.org
devel/intel2gas                              ccatrian at eml.cc
devel/kyua                                   jmmv at NetBSD.org
devel/ldapsdk                              1 pkgsrc-users at NetBSD.org
devel/libblkid                               pkgsrc-users at NetBSD.org
devel/libdnsres                            1 pkgsrc-users at NetBSD.org
devel/libjit                                 pkgsrc-users at NetBSD.org
devel/liblangtag                           1 jaapb at NetBSD.org
devel/libpgm                                 agc at NetBSD.org
devel/librlog                              2 pkgsrc-users at NetBSD.org
devel/libstash                               pkgsrc-users at NetBSD.org
devel/libtecla                             2 kristerw at NetBSD.org
devel/libusb                              95 drochner at NetBSD.org
devel/libusbx                                pkgsrc-users at NetBSD.org
devel/libxenserver                           sborrill at NetBSD.org
devel/lua-lrexlib-posix                    1 cheusov at NetBSD.org
devel/lua-posix                              ryoon at NetBSD.org
devel/mad-flute                              agc at NetBSD.org
devel/man-pages                              pkgsrc-users at NetBSD.org
devel/mico                                   tonnerre at NetBSD.org
devel/netbsd-iscsi-lib                     2 sborrill at NetBSD.org
devel/npth                                71 ada at netbsdfr.org
devel/nqc                                    nathanw at NetBSD.org
devel/nsis                                   pkgsrc-users at NetBSD.org
devel/ode                                    pkgsrc-users at NetBSD.org
devel/openrcs                                asau at inbox.ru
devel/p5-Coro                              7 pkgsrc-users at NetBSD.org
devel/p5-Data-Alias                        2 pkgsrc-users at NetBSD.org
devel/p5-Devel-FindRef                       pkgsrc-users at NetBSD.org
devel/p5-local-lib                         2 pkgsrc-users at NetBSD.org
devel/p5-RT-Authen-ExternalAuth              tonnerre at NetBSD.org
devel/palmpower                              pkgsrc-users at NetBSD.org
devel/pdcurses                               pkgsrc-users at NetBSD.org
devel/picp                                   dogcow at NetBSD.org
devel/pwlib                                2 pkgsrc-users at NetBSD.org
devel/py-astroid                           1 kamel.derouiche at gmail.com
devel/py-distribute                          pkgsrc-users at NetBSD.org
devel/py-h5py                                jihbed.research at gmail.com
devel/py-protobuf                            khorben at defora.org
devel/py-spwd                                pkgsrc-users at NetBSD.org
devel/quilt                                  jmmv at NetBSD.org
devel/radare2                                pkgsrc-users at NetBSD.org
devel/sfio                                   pkgsrc-users at NetBSD.org
devel/sfslite                                agc at NetBSD.org
devel/st                                     gson at NetBSD.org
devel/sunifdef                               pkgsrc-users at NetBSD.org
devel/syncdir                                schmonz at NetBSD.org
devel/tavrasm                                pkgsrc-users at NetBSD.org
devel/tcl-tclOO                              rodent at NetBSD.org
devel/tet3                                   pkgsrc-users at NetBSD.org
devel/ucommon                              2 pkgsrc-users at NetBSD.org
devel/xdelta3                                pkgsrc-users at NetBSD.org
devel/yasm                               404 shaun at inerd.com
editors/biew                                 pkgsrc-users at NetBSD.org
editors/cooledit                             pkgsrc-users at NetBSD.org
editors/emacs20                            1 dholland at NetBSD.org
editors/emacs23-nox11                        minskim at NetBSD.org
editors/ex                                   pkgsrc-users at NetBSD.org
editors/manedit                              pkgsrc-users at NetBSD.org
editors/mule                                 pkgsrc-users at NetBSD.org
editors/nvi                                2 pkgsrc-users at NetBSD.org
editors/nvi-m17n                             pkgsrc-users at NetBSD.org
editors/obby                               1 pkgsrc-users at NetBSD.org
editors/sam                                1 pkgsrc-users at NetBSD.org
editors/sandy                                pkgsrc-users at NetBSD.org
editors/TeXmacs                              pkgsrc-users at NetBSD.org
editors/tweak                                pkgsrc-users at NetBSD.org
editors/vim-lang                             morr at NetBSD.org
editors/xemacs                               hauke at NetBSD.org
editors/xemacs-current                       pkgsrc-users at NetBSD.org
editors/xemacs-current-nox11               3 pkgsrc-users at NetBSD.org
editors/xemacs-nox11                         hauke at NetBSD.org
emulators/aliados                            ivaniclixx at gmail.com
emulators/BasiliskII                         pkgsrc-users at NetBSD.org
emulators/bochs                              pkgsrc-users at NetBSD.org
emulators/ckmame                             wiz at NetBSD.org
emulators/darcnes                            pkgsrc-users at NetBSD.org
emulators/fmsx                               pkgsrc-users at NetBSD.org
emulators/hercules                           pkgsrc-users at NetBSD.org
emulators/ia64sim                            pkgsrc-users at NetBSD.org
emulators/klh10                              ryoon at NetBSD.org
emulators/libretro-catsfc                    pkgsrc-users at NetBSD.org
emulators/libretro-fceumm                    pkgsrc-users at NetBSD.org
emulators/libretro-gambatte                  pkgsrc-users at NetBSD.org
emulators/libretro-mednafen-pce-fast         pkgsrc-users at NetBSD.org
emulators/libretro-mupen64plus               pkgsrc-users at NetBSD.org
emulators/libretro-nestopia                  snj at NetBSD.org
emulators/libretro-picodrive                 pkgsrc-users at NetBSD.org
emulators/libretro-snes9x-next               pkgsrc-users at NetBSD.org
emulators/libretro-stella                    pkgsrc-users at NetBSD.org
emulators/libretro-vba-next                  pkgsrc-users at NetBSD.org
emulators/lrmi                               pkgsrc-users at NetBSD.org
emulators/p11                                packages at grumpf.hope-2000.org
emulators/palmosemulator                     dsainty at NetBSD.org
emulators/pcemu                              eric at fox.phoenix.az.us
emulators/pearpc                             pkgsrc-users at NetBSD.org
emulators/ski                                cherry at zyx.in
emulators/twin                               pkgsrc-users at NetBSD.org
emulators/ucon64                             pkgsrc-users at NetBSD.org
emulators/x48                                pkgsrc-users at NetBSD.org
emulators/xm7                                tech-pkg-ja at jp.NetBSD.org
emulators/xtrs                               pkgsrc-users at NetBSD.org
filesystems/glusterfs                      1 pkgsrc-users at NetBSD.org
filesystems/libntfs                        3 pkgsrc-users at NetBSD.org
filesystems/openafs                          gendalia at NetBSD.org
filesystems/openafs-devel                    gendalia at NetBSD.org
filesystems/perfuse                        2 manu at NetBSD.org
fonts/mglfonts                             1 fk200329 at fsinet.or.jp
fonts/mminstance                             jonb at NetBSD.org
fonts/oto                                    rxg at NetBSD.org
fonts/tex-kotex-base                       2 minskim at NetBSD.org
games/abuse                                  pkgsrc-users at NetBSD.org
games/amaze                                  pkgsrc-users at NetBSD.org
games/angband-tty                            pkgsrc-users at NetBSD.org
games/angband-x11                            pkgsrc-users at NetBSD.org
games/battalion                              pkgsrc-users at NetBSD.org
games/billardgl                              wojtek at 3miasto.net
games/bridge-hands                           pkgsrc-users at NetBSD.org
games/cbzone                                 pkgsrc-users at NetBSD.org
games/crafty                               2 pkgsrc-users at NetBSD.org
games/crossfire-server                       adam at NetBSD.org
games/flightgear-data                        pkgsrc-users at NetBSD.org
games/freeciv-server                         pkgsrc-users at NetBSD.org
games/frotz                                  seebs at plethora.net
games/fruit                                  pkgsrc-users at NetBSD.org
games/jetpack                                pkgsrc-users at NetBSD.org
games/knightcap                              pkgsrc-users at NetBSD.org
games/liquidwar                              pkgsrc-users at NetBSD.org
games/lnl                                    pkgsrc-users at NetBSD.org
games/moria                                  simonb at NetBSD.org
games/nethack-qt                           1 pooka at NetBSD.org
games/netmaj                                 pkgsrc-users at NetBSD.org
games/netmaze                                jdc at NetBSD.org
games/nighthawk                              pkgsrc-users at NetBSD.org
games/plib                                 3 pkgsrc-users at NetBSD.org
games/qstat                                  pkgsrc-users at NetBSD.org
games/quake                                  pkgsrc-users at NetBSD.org
games/quake6                                 pkgsrc-users at NetBSD.org
games/robotfindskitten                       pkgsrc-users at NetBSD.org
games/svb                                    pkgsrc-users at NetBSD.org
games/tads                                   pkgsrc-users at NetBSD.org
games/thew                                   ilfoglionascosto at netcat.it
games/tyrquake                               snj at NetBSD.org
games/vms-empire                             pkgsrc-users at NetBSD.org
games/xbat                                   pkgsrc-users at NetBSD.org
games/xblast                                 dsainty at NetBSD.org
games/xboard                                 simonb at NetBSD.org
games/xboing                                 pkgsrc-users at NetBSD.org
games/xbreaky                                pkgsrc-users at NetBSD.org
games/xdoom                                  pkgsrc-users at NetBSD.org
games/xemeraldia                             pkgsrc-users at NetBSD.org
games/xevil                                  snj at pobox.com
games/xgospel                                pkgsrc-users at NetBSD.org
games/xjewel                                 jdc at NetBSD.org
games/xjig                                   pkgsrc-users at NetBSD.org
games/xjump                                  pkgsrc-users at NetBSD.org
games/xnibbles                               pkgsrc-users at NetBSD.org
games/xpipeman                               pkgsrc-users at NetBSD.org
games/xscavenger                             pkgsrc-users at NetBSD.org
games/xtux                                   pkgsrc-users at NetBSD.org
geography/gama                               gdt at NetBSD.org
geography/garmin-utils                       gson at NetBSD.org
geography/gipfel                             johannes.hofmann at gmx.de
geography/gpsdrive                           pkgsrc-users at NetBSD.org
geography/kplex                              pkgsrc-users at NetBSD.org
geography/py-obspy                           wen at NetBSD.org
geography/qlandkartem                        jkunz at NetBSD.org
geography/vis5d+                             hdp at nmsu.edu
graphics/autopano-sift-C                   1 pkgsrc-users at NetBSD.org
graphics/cqcam                               pkgsrc-users at NetBSD.org
graphics/deforaos-icon-theme               2 khorben at defora.org
graphics/feh                                 pkgsrc-users at NetBSD.org
graphics/flickrnet                         1 pkgsrc-users at NetBSD.org
graphics/freeimage                         2 pkgsrc-users at NetBSD.org
graphics/gimp-docs-de                      1 pkgsrc-users at NetBSD.org
graphics/gimp-docs-es                      1 pkgsrc-users at NetBSD.org
graphics/gimp-docs-fr                      1 pkgsrc-users at NetBSD.org
graphics/gimp-docs-it                      1 pkgsrc-users at NetBSD.org
graphics/gimp-docs-ja                      1 pkgsrc-users at NetBSD.org
graphics/gimp-docs-ko                      1 pkgsrc-users at NetBSD.org
graphics/gimp-docs-nl                      1 pkgsrc-users at NetBSD.org
graphics/gimp-docs-nn                      1 pkgsrc-users at NetBSD.org
graphics/gimp-docs-ru                      1 pkgsrc-users at NetBSD.org
graphics/gimp-docs-sv                      1 pkgsrc-users at NetBSD.org
graphics/grap                                pkgsrc-users at NetBSD.org
graphics/hp2xx                               dmcmahill at NetBSD.org
graphics/jpegpixi                            gdt at NetBSD.org
graphics/libpgf                            1 pkgsrc-users at NetBSD.org
graphics/netpbm                           44 adam at NetBSD.org
graphics/ns-cult3d                           abs at NetBSD.org
graphics/opendis                             pkgsrc-users at NetBSD.org
graphics/oxygen-icons                     17 markd at NetBSD.org
graphics/py-cairo                        196 leuzi at trash.net
graphics/tesseract                           pkgsrc-users at NetBSD.org
graphics/vcg                                 pkgsrc-users at NetBSD.org
graphics/xmorph                              pkgsrc-users at NetBSD.org
graphics/zphoto                              pkgsrc-users at NetBSD.org
ham/dpbox                                    pkgsrc-users at NetBSD.org
ham/linpsk                                   pkgsrc-users at NetBSD.org
ham/tfkiss                                   pkgsrc-users at NetBSD.org
ham/tlf                                      pkgsrc-users at NetBSD.org
ham/tnt                                      pkgsrc-users at NetBSD.org
inputmethod/anthy                         14 taya at NetBSD.org
inputmethod/multiskkserv                     ryoon at NetBSD.org
inputmethod/sj3-client                     2 pkgsrc-users at NetBSD.org
inputmethod/sj3-server                     1 pkgsrc-users at NetBSD.org
inputmethod/skkfep                           pkgsrc-users at NetBSD.org
inputmethod/unicon-im                      1 rxg at NetBSD.org
inputmethod/xcin                             rxg at NetBSD.org
lang/algol68g                                olafs at cs.ru.nl
lang/baci                                    pkgsrc-users at NetBSD.org
lang/bwbasic                                 jspath at bcpl.net
lang/chicken                                 asau at inbox.ru
lang/cint                                    drochner at NetBSD.org
lang/clang                                28 adam.hoka at gmail.com
lang/classpath                             2 pkgsrc-users at NetBSD.org
lang/cparser                                 pkgsrc-users at NetBSD.org
lang/g95                                     wennmach at NetBSD.org
lang/gcc2                                    pkgsrc-users at NetBSD.org
lang/gcc34                                   pkgsrc-users at NetBSD.org
lang/gcc44                                   sbd at NetBSD.org
lang/gcc47                                 1 sbd at NetBSD.org
lang/gcc48                                 3 sbd at NetBSD.org
lang/gcc49                                 1 pkgsrc-users at NetBSD.org
lang/ghc                                   2 kristerw at NetBSD.org
lang/ghc7                                 84 pho at cielonegro.org
lang/gpc                                     pkgsrc-users at NetBSD.org
lang/gprolog                                 brook at nmsu.edu
lang/heirloom-awk                            cheusov at NetBSD.org
lang/kaffe                                 3 tonio at NetBSD.org
lang/kali                                    pkgsrc-users at NetBSD.org
lang/konoha                                  ryoon at NetBSD.org
lang/ksi                                     pkgsrc-users at NetBSD.org
lang/librep                                2 pkgsrc-users at NetBSD.org
lang/LuaJIT2                               4 alnsn at NetBSD.org
lang/mono-basic                            1 kefren at NetBSD.org
lang/mono2                                 6 kefren at NetBSD.org
lang/nhc98                                   kristerw at NetBSD.org
lang/nodejs                                  filip at joyent.com
lang/objc                                    pkgsrc-users at NetBSD.org
lang/ocaml                                80 jaapb at NetBSD.org
lang/open-cobol-ce                           ryoon at NetBSD.org
lang/ossp-js                                 pkgsrc-users at NetBSD.org
lang/parrot                                2 he at NetBSD.org
lang/pcc-current                             he at NetBSD.org
lang/pfe                                     asau at inbox.ru
lang/php54                               182 pkgsrc-users at NetBSD.org
lang/php55                               131 pkgsrc-users at NetBSD.org
lang/php56                               129 pkgsrc-users at NetBSD.org
lang/picoc                                   agc at NetBSD.org
lang/rexx-imc                                pkgsrc-users at NetBSD.org
lang/ruby18-base                          33 taca at NetBSD.org
lang/ruby193-base                        540 taca at NetBSD.org
lang/ruby200-base                        616 taca at NetBSD.org
lang/ruby21-base                         542 taca at NetBSD.org
lang/ruby22-base                         537 taca at NetBSD.org
lang/sather                                  pkgsrc-users at NetBSD.org
lang/sbcl                                    asau at NetBSD.org
lang/scheme48                                pkgsrc-users at NetBSD.org
lang/scm                                     pkgsrc-users at NetBSD.org
lang/spl                                     pkgsrc-users at NetBSD.org
lang/sr                                      is at NetBSD.org
lang/sun-jre6                              6 pkgsrc-users at NetBSD.org
lang/sun-jre7                             88 pkgsrc-users at NetBSD.org
lang/yap                                     pkgsrc-users at NetBSD.org
mail/cone                                    pkgsrc-users at NetBSD.org
mail/courier-mta                           1 pkgsrc-users at NetBSD.org
mail/cyrus-imapd23                           pkgsrc-users at NetBSD.org
mail/cyrus-imapd24                         1 pkgsrc-users at NetBSD.org
mail/dcc                                     petar at smokva.net
mail/distribute                              pkgsrc-users at NetBSD.org
mail/dnsbl-milter                            pkgsrc-users at NetBSD.org
mail/drac                                    pkgsrc-users at NetBSD.org
mail/elm-me                                  pkgsrc-users at NetBSD.org
mail/ezmlm                                   schmonz at NetBSD.org
mail/ezmlm-idx                             2 schmonz at NetBSD.org
mail/fdm                                     pkgsrc-users at NetBSD.org
mail/fix-mime-charset                        pkgsrc-users at NetBSD.org
mail/isync                                   tonio at NetBSD.org
mail/ja-mh                                   tech-pkg-ja at jp.NetBSD.org
mail/libetpan                             24 pkgsrc-users at NetBSD.org
mail/mailman                               2 pkgsrc-users at NetBSD.org
mail/mailwrapper                             pkgsrc-users at NetBSD.org
mail/metamail                              2 pkgsrc-users at NetBSD.org
mail/milter-regex                            jdarrow at NetBSD.org
mail/mopher                                  petar at smokva.net
mail/mush                                    christos at NetBSD.org
mail/mutt-kz                                 wiz at NetBSD.org
mail/newmail                                 andrew at ugh.net.au
mail/nmzmail                                 rodent at NetBSD.org
mail/opensmtpd                               ftigeot at wolfpond.org
mail/p5-razor-agents                         pkgsrc-users at NetBSD.org
mail/pfqueue                                 pkgsrc-users at NetBSD.org
mail/popa3d                                  pkgsrc-users at NetBSD.org
mail/relay-ctrl                              schmonz at NetBSD.org
mail/smtp-vilter                             marc at msys.ch
mail/smtpfeed                                pkgsrc-users at NetBSD.org
mail/spamass-milter                          gdt at NetBSD.org
mail/sqwebmail                             1 pkgsrc-users at NetBSD.org
mail/thunderbird-l10n                      1 pkgsrc-users at NetBSD.org
mail/xfmail                                  pkgsrc-users at NetBSD.org
math/aamath                                  esg at sdf.lonestar.org
math/eukleides                             1 ryoon at NetBSD.org
math/graphopt                                pkgsrc-users at NetBSD.org
math/mpfr                                484 pkgsrc-users at NetBSD.org
math/muparser                              2 pkgsrc-users at NetBSD.org
math/qalculate                            26 adam at NetBSD.org
math/R-clim.pact                             pkgsrc-users at NetBSD.org
math/R-combinat                            1 pkgsrc-users at NetBSD.org
math/R-GRASS                                 mchittur at cs.nmsu.edu
math/R-PHYLOGR                               pkgsrc-users at NetBSD.org
math/sc                                      eric at fox.phoenix.az.us
math/snns                                    pkgsrc-users at NetBSD.org
math/teapot                                  hauke at NetBSD.org
math/xlispstat                             1 pkgsrc-users at NetBSD.org
math/yorick                                  pkgsrc-users at NetBSD.org
mbone/mdd                                    manu at NetBSD.org
mbone/mdp                                    pkgsrc-users at NetBSD.org
mbone/rat                                    pkgsrc-users at NetBSD.org
mbone/vat                                    pkgsrc-users at NetBSD.org
mbone/vic                                    is at NetBSD.org
mbone/vic-devel                              sborrill at NetBSD.org
misc/chipmunk                                pkgsrc-users at NetBSD.org
misc/dpkg                                    pkgsrc-users at NetBSD.org
misc/dvtm                                    pkgsrc-users at NetBSD.org
misc/fbreader                                pkgsrc-users at NetBSD.org
misc/fd                                      pkgsrc-users at NetBSD.org
misc/hanzim                                  rxg at NetBSD.org
misc/ibutton-pdkit                           damon at brodiefamily.org
misc/libcec                                  pkgsrc-users at NetBSD.org
misc/lile                                    pkgsrc-users at NetBSD.org
misc/open2300                                martin at NetBSD.org
misc/open2300-mysql                          martin at NetBSD.org
misc/rhash                                 1 pkgsrc-users at NetBSD.org
misc/rlwrap                                  pkgsrc-users at NetBSD.org
misc/root                                    drochner at NetBSD.org
misc/rpm                                   1 pkgsrc-users at NetBSD.org
misc/rump                                  1 stacktic at NetBSD.org
misc/sch                                     pkgsrc-users at NetBSD.org
misc/stellarium                              pkgsrc-users at NetBSD.org
misc/team                                    pkgsrc-users at NetBSD.org
misc/toilet                                  schmonz at NetBSD.org
misc/udfclient                               reinoud at 13thmonkey.org
misc/vera                                    pkgsrc-users at NetBSD.org
misc/vfu                                     pkgsrc-users at NetBSD.org
misc/window                                  pkgsrc-users at NetBSD.org
misc/xjdic                                   pkgsrc-users at NetBSD.org
misc/zyGrib                                2 plunky at NetBSD.org
multimedia/dirac                             pkgsrc-users at NetBSD.org
multimedia/dvb-apps                          pkgsrc-users at NetBSD.org
multimedia/dvdbackup                         pkgsrc-users at NetBSD.org
multimedia/fxtv                            2 pkgsrc-users at NetBSD.org
multimedia/ggrab                             pkgsrc-users at NetBSD.org
multimedia/gmediaserver                      pkgsrc-users at NetBSD.org
multimedia/hwdecode-demos                    pkgsrc-users at NetBSD.org
multimedia/libdv-tools                       tech-pkg-ja at jp.NetBSD.org
multimedia/libflash                        1 pkgsrc-users at NetBSD.org
multimedia/libmpeg3                          pkgsrc-users at NetBSD.org
multimedia/openquicktime                     pkgsrc-users at NetBSD.org
multimedia/schroedinger                    1 pkgsrc-users at NetBSD.org
multimedia/tstools                           tech-multimedia at NetBSD.org
multimedia/tvtime                            pkgsrc-users at NetBSD.org
multimedia/xanim                             pkgsrc-users at NetBSD.org
multimedia/xawtv                             pkgsrc-users at NetBSD.org
net/3proxy                                   pkgsrc-users at NetBSD.org
net/aiccu                                    pkgsrc-users at NetBSD.org
net/arpd                                     pkgsrc-users at NetBSD.org
net/arpwatch                                 bouyer at NetBSD.org
net/batchftp                                 pkgsrc-users at NetBSD.org
net/bftpd                                    pkgsrc-users at NetBSD.org
net/bridged                                  pkgsrc-users at NetBSD.org
net/bsddip                                   pkgsrc-users at NetBSD.org
net/chksniff                                 pkgsrc-users at NetBSD.org
net/choparp                                  dbj at NetBSD.org
net/chrony                                   hannken at NetBSD.org
net/citrix_ica                               sborrill at NetBSD.org
net/cntlm                                    pkgsrc-users at NetBSD.org
net/csup                                     imil at gcu.info
net/dante                                    pkgsrc-users at NetBSD.org
net/DarwinStreamingServer                    adam at NetBSD.org
net/dctc                                   1 pkgsrc-users at NetBSD.org
net/delegate                                 pkgsrc-users at NetBSD.org
net/dgd                                      sco at openface.ca
net/dhcpcd                                 3 roy at NetBSD.org
net/dhisd                                    pkgsrc-users at NetBSD.org
net/dnscap                                   agc at NetBSD.org
net/dnsmasq                                  pkgsrc-users at NetBSD.org
net/dnstracer                                pkgsrc-users at NetBSD.org
net/dynipclient                              pkgsrc-users at NetBSD.org
net/entropy                                  pkgsrc-users at NetBSD.org
net/ettercap                                 pkgsrc-users at NetBSD.org
net/flodo                                    pkgsrc-users at NetBSD.org
net/freeDiameter                             pkgsrc-users at NetBSD.org
net/gated                                    pkgsrc-users at NetBSD.org
net/gethost                                  reed at reedmedia.net
net/host                                     pkgsrc-users at NetBSD.org
net/hping3                                   pettai at NetBSD.org
net/iana-etc                                 christos at NetBSD.org
net/IglooFTP                                 rxg at NetBSD.org
net/inadyn                                   pkgsrc-users at NetBSD.org
net/iodine                                   mwilhelmy at users.sourceforge.net
net/isic                                     pkgsrc-users at NetBSD.org
net/istgt                                    sborrill at NetBSD.org
net/jumpgate                                 pkgsrc-users at NetBSD.org
net/kiax                                     reinoud at NetBSD.org
net/kismet                                   pkgsrc-users at NetBSD.org
net/knock                                    adam at NetBSD.org
net/knot                                     pettai at NetBSD.org
net/kphone                                   pkgsrc-users at NetBSD.org
net/latd                                     fredette at theory.lcs.mit.edu
net/lftp                                     pettai at NetBSD.org
net/libnipper                              1 pkgsrc-users at NetBSD.org
net/libsscript                               pkgsrc-users at NetBSD.org
net/libtorrent                             1 tron at NetBSD.org
net/libtrace                                 pkgsrc-users at NetBSD.org
net/lldpd                                    pkgsrc-users at NetBSD.org
net/maradns                                  ddeus.pkgsrc at mailnull.com
net/mcast-tools                              is at NetBSD.org
net/md-whois                                 reed at reedmedia.net
net/microdc2                                 d.den.brok at uni-bonn.de
net/miniupnpd                                pkgsrc-users at NetBSD.org
net/mouse-pppoe                              mouse at NetBSD.org
net/mrstat                                   abs at absd.org
net/mrt                                      pkgsrc-users at NetBSD.org
net/msdl                                     pkgsrc-users at NetBSD.org
net/nagios-plugin-dnsrbl                   1 manu at NetBSD.org
net/nagios-plugin-dotpid                   1 manu at NetBSD.org
net/nagios-plugin-dumpdates                1 manu at NetBSD.org
net/nagios-plugin-milter                   1 manu at NetBSD.org
net/nagios-plugin-raidctl                  1 manu at NetBSD.org
net/nagios-plugin-syncrepl                 1 manu at NetBSD.org
net/nasd                                     pkgsrc-users at NetBSD.org
net/netcat-openbsd                           pkgsrc-users at NetBSD.org
net/nprobe                                   pkgsrc-users at NetBSD.org
net/ns                                       minskim at NetBSD.org
net/nstx                                     pkgsrc-users at NetBSD.org
net/ntp4                                     pkgsrc-users at NetBSD.org
net/ocsync                                   ryoon at NetBSD.org
net/omniNotify                               pkgsrc-users at NetBSD.org
net/openconnect                              pkgsrc-users at NetBSD.org
net/openslp                               36 pkgsrc-users at NetBSD.org
net/openvpn                                1 pkgsrc-users at NetBSD.org
net/openvpn-acct-wtmpx                       manu at NetBSD.org
net/openwbem                                 pkgsrc-users at NetBSD.org
net/ORBit                                    pkgsrc-users at NetBSD.org
net/ortp                                   3 zul at netbsd-fr.org
net/p5-eBay-API                              pkgsrc-users at NetBSD.org
net/p5-FusionInventory-Agent-Task-Network    bouyer at NetBSD.org
net/p5-Net-Interface                         pkgsrc-users at NetBSD.org
net/p5-Net-RawIP                             bouyer at NetBSD.org
net/packit                                   pkgsrc-users at NetBSD.org
net/parpd                                    roy at NetBSD.org
net/pconsole                                 pkgsrc-users at NetBSD.org
net/pim6dd                                   pkgsrc-users at NetBSD.org
net/pim6sd                                   pkgsrc-users at NetBSD.org
net/poptop                                   pkgsrc-users at NetBSD.org
net/portmap                                  reed at reedmedia.net
net/pptp                                     pkgsrc-users at NetBSD.org
net/radiusd-cistron                          pkgsrc-users at NetBSD.org
net/rdist6                                 2 christos at NetBSD.org
net/rp-l2tp                                  pkgsrc-users at NetBSD.org
net/rp-pppoe                                 pkgsrc-users at NetBSD.org
net/samba                                 27 pkgsrc-users at NetBSD.org
net/scamper                                  mjl at luckie.org.nz
net/ser                                      pkgsrc-users at NetBSD.org
net/sharity-light                            pkgsrc-users at NetBSD.org
net/socket++                                 mchittur at cs.nmsu.edu
net/socks4                                   pkgsrc-users at NetBSD.org
net/socks5                                 1 ambrisko at ambrisko.roble.com
net/spegla                                   jnilsson at ludd.luth.se
net/spread                                 2 pkgsrc-users at NetBSD.org
net/sslh                                     pkgsrc-users at NetBSD.org
net/ssmping                                  is at NetBSD.org
net/sup                                      christos at NetBSD.org
net/tcl-scotty                             1 pkgsrc-users at NetBSD.org
net/tcpreplay                                pkgsrc-users at NetBSD.org
net/tightvnc                                 pkgsrc-users at NetBSD.org
net/tinc                                     tonnerre at NetBSD.org
net/trickle                                  pkgsrc-users at NetBSD.org
net/ttt                                      pkgsrc-users at NetBSD.org
net/u6rd                                     kamada at nanohz.org
net/udpcast                                  pkgsrc-users at NetBSD.org
net/unworkable                               agc at NetBSD.org
net/uucp                                     pkgsrc-users at NetBSD.org
net/vnc                                      pkgsrc-users at NetBSD.org
net/vtun                                     pkgsrc-users at NetBSD.org
net/waste                                    pkgsrc-users at NetBSD.org
net/wide-dhcpv6                              pkgsrc-users at NetBSD.org
net/wistumbler                               pkgsrc-users at NetBSD.org
net/wmget                                    pkgsrc-users at NetBSD.org
net/wminet                                   coto at core.de
net/wpa_gui                                  pkgsrc-users at NetBSD.org
net/wpa_supplicant                         2 reed at reedmedia.net
net/wu-ftpd                                  pkgsrc-users at NetBSD.org
net/xl2tpd                                   christos at NetBSD.org
net/xorp                                     pkgsrc-users at NetBSD.org
news/nntpcache                               pkgsrc-users at NetBSD.org
news/tin                                     pkgsrc-users at NetBSD.org
parallel/dqs                                 pkgsrc-users at NetBSD.org
parallel/mpi-ch                            4 asau at inbox.ru
parallel/openmpi                             asau at inbox.ru
parallel/threadingbuildingblocks             asau at inbox.ru
pkgtools/pkg                                 khorben at defora.org
pkgtools/pkg_select                          imil at gcu.info
pkgtools/pkgin                             4 imil at NetBSD.org
print/cups-pdf                               pkgsrc-users at NetBSD.org
print/foomatic4-db                           pkgsrc-users at NetBSD.org
print/foomatic4-filters                    2 pkgsrc-users at NetBSD.org
print/hpijs                                  pkgsrc-users at NetBSD.org
print/LPRng-core                           1 pkgsrc-users at NetBSD.org
print/pdvipsk                              1 pkgsrc-users at NetBSD.org
print/pnm2ppa                                pkgsrc-users at NetBSD.org
print/poppler-qt                           1 reed at reedmedia.net
print/tex-bibtex8                          1 cheusov at NetBSD.org
security/aide06                              pkgsrc-users at NetBSD.org
security/asignify                            youri at NetBSD.org
security/beecrypt                            pkgsrc-users at NetBSD.org
security/dcfldd                              pettai at NetBSD.org
security/dsniff-nox11                      1 pkgsrc-users at NetBSD.org
security/fail2ban                            nils at netbsd.org
security/fwbuilder                           pkgsrc-users at NetBSD.org
security/google-authenticator                jmmv at NetBSD.org
security/hashcash                            pkgsrc-users at NetBSD.org
security/heimdal                           1 pkgsrc-users at NetBSD.org
security/ipv6-toolkit                        agc at NetBSD.org
security/john                                pkgsrc-users at NetBSD.org
security/keepassx                            pkgsrc-users at NetBSD.org
security/kstart                              jakllsch at NetBSD.org
security/lasso                             5 manu at NetBSD.org
security/libbf                               pkgsrc-users at NetBSD.org
security/libgfshare                          agc at NetBSD.org
security/libmultigest                      2 agc at NetBSD.org
security/libnetpgpverify                     agc at NetBSD.org
security/libprelude-perl                     pkgsrc-users at NetBSD.org
security/libpreludedb-perl                   pkgsrc-users at NetBSD.org
security/libsodium                        14 csosstudy at gmail.com
security/lsh                                 pkgsrc-users at NetBSD.org
security/mit-krb5                          4 tez at NetBSD.org
security/mit-krb5-appl                     1 tez at NetBSD.org
security/mozilla-rootcerts-openssl           dholland at NetBSD.org
security/multigest                           agc at NetBSD.org
security/munge                             1 asau at inbox.ru
security/MyPasswordSafe                      pkgsrc-users at NetBSD.org
security/netpgp                              agc at NetBSD.org
security/netpgpverify                        agc at NetBSD.org
security/oath-toolkit                        pettai at NetBSD.org
security/openpam                             joerg at NetBSD.org
security/openssh                             pkgsrc-users at NetBSD.org
security/p5-Net-DNS-SEC                    5 pkgsrc-users at NetBSD.org
security/pakemon                             pkgsrc-users at NetBSD.org
security/pam-mkhomedir                       tonnerre at NetBSD.org
security/pam-pgsql                           pkgsrc-users at NetBSD.org
security/pam-pwauth_suid                     drochner at NetBSD.org
security/pam-radius                          pkgsrc-users at NetBSD.org
security/pam-tacplus                         pettai at NetBSD.org
security/pev                                 khorben at defora.org
security/pgp2                              1 pkgsrc-users at NetBSD.org
security/pkcs11-helper                     1 gdt at ir.bbn.com
security/prelude-correlator                  pkgsrc-users at NetBSD.org
security/prelude-pflogger                    pkgsrc-users at NetBSD.org
security/py-cryptkit                         pkgsrc-users at NetBSD.org
security/py-Kerberos                         pkgsrc-users at NetBSD.org
security/racoon2                             kamada at nanohz.org
security/rainbowcrack                        dhowland at users.sourceforge.net
security/sfs                                 pkgsrc-users at NetBSD.org
security/snoopy                              imil at NetBSD.org
security/srp_client                          pkgsrc-users at NetBSD.org
security/sslsplit                            pkgsrc-users at NetBSD.org
security/steghide                            agc at NetBSD.org
security/yara                              1 pettai at NetBSD.org
security/zkt                                 pettai at NetBSD.org
shells/ast-ksh                               pkgsrc-users at NetBSD.org
shells/es                                    pkgsrc-users at NetBSD.org
shells/fish                                  pkgsrc-users at NetBSD.org
shells/static-tcsh                           kim at NetBSD.org
sysutils/acpica-utils                      5 pkgsrc-users at NetBSD.org
sysutils/amtterm                             pkgsrc-users at NetBSD.org
sysutils/attr                                pkgsrc-users at NetBSD.org
sysutils/bacula                            2 pkgsrc-users at NetBSD.org
sysutils/bacula-clientonly                   pkgsrc-users at NetBSD.org
sysutils/bbsload                             pino at dohd.org
sysutils/bup                                 wiz at NetBSD.org
sysutils/collectd                         20 filip at joyent.com
sysutils/conky                               imil at NetBSD.org
sysutils/dbus-glib                      1013 pkgsrc-users at NetBSD.org
sysutils/direvent                            pkgsrc-users at NetBSD.org
sysutils/dvd+rw-tools                      2 pkgsrc-users at NetBSD.org
sysutils/e2fsprogs                           pkgsrc-users at NetBSD.org
sysutils/entr                                pkgsrc-users at NetBSD.org
sysutils/fakeroot                            pkgsrc-users at NetBSD.org
sysutils/fix4SA110rev2                       abs at NetBSD.org
sysutils/gnometoaster                        pkgsrc-users at NetBSD.org
sysutils/grub                              1 pkgsrc-users at NetBSD.org
sysutils/heirloom-ls                         cheusov at NetBSD.org
sysutils/heirloom-shl                        cheusov at NetBSD.org
sysutils/idesk                               pkgsrc-users at NetBSD.org
sysutils/ipa                               1 simon at comsys.ntu-kpi.kiev.ua
sysutils/lastfss                             manu at NetBSD.org
sysutils/libisoburn                          ryoon at NetBSD.org
sysutils/libvirt                           1 agc at NetBSD.org
sysutils/lockf                               pkgsrc-users at NetBSD.org
sysutils/memconf                             shattered at NetBSD.org
sysutils/menu-cache                          pkgsrc-users at NetBSD.org
sysutils/mesure                              pkgsrc-users at NetBSD.org
sysutils/mrsync                              agc at NetBSD.org
sysutils/mtscan                              pkgsrc-users at NetBSD.org
sysutils/mtx                               3 bouyer at NetBSD.org
sysutils/p5-Sys-Utmp                         pkgsrc-users at NetBSD.org
sysutils/pidof                               pkgsrc-users at NetBSD.org
sysutils/pluged                              pkgsrc-users at NetBSD.org
sysutils/pscpug                              bartosz.kuzma at gmail.com
sysutils/py-attic                            pkgsrc-users at NetBSD.org
sysutils/qlogtools                           pkgsrc-users at NetBSD.org
sysutils/radeontool                          pkgsrc-users at NetBSD.org
sysutils/same                                pkgsrc-users at NetBSD.org
sysutils/scsiinfo                            pkgsrc-users at NetBSD.org
sysutils/sformat                             Andreas.Hallmann at tiscali.de
sysutils/slocate                             pkgsrc-users at NetBSD.org
sysutils/socket                              pkgsrc-users at NetBSD.org
sysutils/strace                              christos at NetBSD.org
sysutils/sysinfo                             pkgsrc-users at NetBSD.org
sysutils/systester                           pkgsrc-users at NetBSD.org
sysutils/tits                                pkgsrc-users at NetBSD.org
sysutils/tkdesk                              pkgsrc-users at NetBSD.org
sysutils/torsmo                              pkgsrc-users at NetBSD.org
sysutils/toshutils                           pkgsrc-users at NetBSD.org
sysutils/tpb                                 soren at NetBSD.org
sysutils/ups-nut                           3 pkgsrc-users at NetBSD.org
sysutils/usbutil                             lennart at augustsson.net
sysutils/vifm                                pkgsrc-users at NetBSD.org
sysutils/webmin                           42 pkgsrc-users at NetBSD.org
sysutils/whowatch                            pkgsrc-users at NetBSD.org
sysutils/wmSMPmon                            joel at carnat.net
sysutils/xbattbar                            pkgsrc-users at NetBSD.org
sysutils/xfm                                 rxg at NetBSD.org
sysutils/xfsprogs                            pkgsrc-users at NetBSD.org
sysutils/xnc                                 pkgsrc-users at NetBSD.org
textproc/aspell-csb                          rodent at NetBSD.org
textproc/aspell-fo                           pkgsrc-users at NetBSD.org
textproc/aspell-is                           rodent at NetBSD.org
textproc/aspell-nb                           pkgsrc-users at NetBSD.org
textproc/biblook                             pkgsrc-users at NetBSD.org
textproc/cabocha                           6 pkgsrc-users at NetBSD.org
textproc/convertlit                        1 pkgsrc-users at NetBSD.org
textproc/dict-dictionaries                   pkgsrc-users at NetBSD.org
textproc/docbook2mdoc                        pkgsrc-users at NetBSD.org
textproc/expatobjc                           pkgsrc-users at NetBSD.org
textproc/grepcidr                            pkgsrc-users at netbsd.org
textproc/GutenMark                           pkgsrc-users at NetBSD.org
textproc/heirloom-doctools                   carsten.kunze at arcor.de
textproc/ispell-en_GB                        pkgsrc-users at NetBSD.org
textproc/ispell-pl                           adam at NetBSD.org
textproc/ja-groff                          2 pkgsrc-users at NetBSD.org
textproc/jsoncpp                           1 pkgsrc-users at NetBSD.org
textproc/libpinyin                         1 pkgsrc-users at NetBSD.org
textproc/libplist                          8 pkgsrc-users at NetBSD.org
textproc/link-grammar                        pkgsrc-users at NetBSD.org
textproc/lq-sp                             1 norm at sandbox.org.uk
textproc/lua-expat                           pkgsrc-users at NetBSD.org
    lua53-expat-1.3.0nb2                     pkgsrc-users at NetBSD.org
textproc/ndtpd                               uebayasi at NetBSD.org
textproc/pod2mdoc                            pkgsrc-users at NetBSD.org
textproc/qsubst                              pkgsrc-users at NetBSD.org
textproc/regxml                              pkgsrc-users at NetBSD.org
textproc/rtfm                                pkgsrc-users at NetBSD.org
textproc/tcl-expat                           pkgsrc-users at NetBSD.org
textproc/texi2mdoc                           pkgsrc-users at NetBSD.org
textproc/unroff                              pkgsrc-users at NetBSD.org
textproc/vis                                 christos at NetBSD.org
textproc/xml2doc                             pkgsrc-users at NetBSD.org
time/emiclock                                minoura at NetBSD.org
time/glclock                                 pkgsrc-users at NetBSD.org
time/gnutime                                 reed at reedmedia.net
time/p5-Date-Extract                       1 ryoon at NetBSD.org
time/phoon                                   is at NetBSD.org
time/vixie-cron                              reed at reedmedia.net
time/xalarm                                  pkgsrc-users at NetBSD.org
time/xchrono                                 pkgsrc-users at NetBSD.org
time/xonclock                                pkgsrc-users at NetBSD.org
wm/3ddesktop                                 pkgsrc-users at NetBSD.org
wm/bspwm                                     youri at NetBSD.org
wm/jwm                                       pkgsrc-users at NetBSD.org
wm/notion                                    wiz at NetBSD.org
wm/pekwm                                     gls at NetBSD.org
wm/piewm                                     pkgsrc-users at NetBSD.org
wm/py-tyle                                   pkgsrc-users at NetBSD.org
wm/spectrwm                                  pkgsrc-users at NetBSD.org
wm/uwm                                       pkgsrc-users at NetBSD.org
wm/wmii                                      pkgsrc-users at NetBSD.org
www/ap-auth-kerb                             markd at NetBSD.org
www/ap2-cband                                pkgsrc-users at NetBSD.org
www/ap2-suphp                                pkgsrc-users at NetBSD.org
www/bozohttpd                              3 mrg at eterna.com.au
www/c-icap-modules                           adam at NetBSD.org
www/cgit                                     pkgsrc-users at NetBSD.org
www/crawl                                    pkgsrc-users at NetBSD.org
www/ffproxy                                  niklas at noxa.de
www/firefox-l10n                             pkgsrc-users at NetBSD.org
www/firefox31-l10n                           pkgsrc-users at NetBSD.org
www/firefox38-l10n                           pkgsrc-users at NetBSD.org
www/h2o                                      pkgsrc-users at NetBSD.org
www/htmlcxx                                1 pkgsrc-users at NetBSD.org
www/kannel                                   pkgsrc-users at NetBSD.org
www/mono-xsp                               2 kefren at NetBSD.org
www/nvu                                      pkgsrc-users at NetBSD.org
www/p5-Catalyst-Engine-HTTP-Prefork          pkgsrc-users at NetBSD.org
www/p5-Catalyst-Plugin-AutoRestart           pkgsrc-users at NetBSD.org
www/p5-HTMLObject                            pkgsrc-users at NetBSD.org
www/p5-libapreq                              pkgsrc-users at NetBSD.org
www/py-cherrypy                            3 pkgsrc-users at NetBSD.org
www/py-django                             72 joerg at NetBSD.org
www/py-django14                              joerg at NetBSD.org
www/py-tornado                             1 imil at NetBSD.org
    py33-tornado-4.2                       1 imil at NetBSD.org
www/qDecoder                                 jypak at users.sourceforge.net
www/screws                                   pkgsrc-users at NetBSD.org
www/sqtop                                    rodent at NetBSD.org
www/squidpurge                               gdt at NetBSD.org
www/swiggle                                  snj at NetBSD.org
www/swill                                    pkgsrc-users at NetBSD.org
www/urlget                                   kim at tac.nyc.ny.us
www/w3c-httpd                                pkgsrc-users at NetBSD.org
www/wml                                      kim at tac.nyc.ny.us
www/www6to4                                  feico at pasta.cs.uit.no
x11/autocutsel                               ryoon at NetBSD.org
x11/dtx11session                             pkgsrc-users at NetBSD.org
x11/efltk                                  1 pkgsrc-users at NetBSD.org
x11/elementary                             2 joerg at NetBSD.org
x11/fbdesk                                   pkgsrc-users at NetBSD.org
x11/filerunner                               pkgsrc-users at NetBSD.org
x11/fltk2                                  1 pkgsrc-users at NetBSD.org
x11/fox                                    2 pkgsrc-users at NetBSD.org
x11/hanterm                                  pkgsrc-users at NetBSD.org
x11/matchbox-panel                         2 pkgsrc-users at NetBSD.org
x11/modular-xorg-server                   58 pkgsrc-users at NetBSD.org
x11/motif                                 69 pkgsrc-users at NetBSD.org
x11/qt4-creator                              fw at submerge.ch
x11/qt4-docs                               1 pkgsrc-users at NetBSD.org
x11/qt5-qtdeclarative                     22 ryoon at NetBSD.org
x11/slock                                    reed at reedmedia.net
x11/tk-Tix                                 1 he at NetBSD.org
x11/wmavgload                                pkgsrc-users at NetBSD.org
x11/wmfstatus                                pkgsrc-users at NetBSD.org
x11/x3270                                    gls at NetBSD.org
x11/XaoS                                     pkgsrc-users at NetBSD.org
x11/xdesktopwaves                            pkgsrc-users at NetBSD.org
x11/xrootconsole                             inittab at unixdev.net
x11/xscreensaver                           7 pkgsrc-users at NetBSD.org
x11/xview-lib                              4 pkgsrc-users at NetBSD.org

From bfriesen at simple.dallas.tx.us  Sun Jul 12 17:58:56 2015
From: bfriesen at simple.dallas.tx.us (Bob Friesenhahn)
Date: Sun, 12 Jul 2015 12:58:56 -0500 (CDT)
Subject: [OmniOS-discuss] ZFS crash/reboot loop
In-Reply-To: <55A1D457.7060207@umiacs.umd.edu>
References: <55A1D457.7060207@umiacs.umd.edu>
Message-ID: <alpine.GSO.2.01.1507121253560.4186@freddy.simplesystems.org>

On Sat, 11 Jul 2015, Derek Yarnell wrote:

> Hi,
>
> We just have had a catastrophic event on one of our OmniOS r14 file
> servers.  In what seems to have been triggered by the weekly scrub of
> its one large zfs pool (~100T) it panics.  This made it basically reboot
> continually and we have installed a second copy of OmniOS r14 in the
> mean time.  We are able to mount the pool readonly and are currently
> securing the data as soon as possible.

The on-going scrub automatically restarts, apparently even in 
read-only mode.  You should 'zpool scrub -s poolname' ASAP after boot 
(if you can) to stop the ongoing scrub.

> ### After mounting in readonly mode
>  pool: zvol00
> state: ONLINE
> status: The pool is formatted using a legacy on-disk format.  The pool can
>        still be used, but some features are unavailable.
> action: Upgrade the pool using 'zpool upgrade'.  Once this is done, the
>        pool will no longer be accessible on software that does not
> support feature
>        flags.
>  scan: scrub in progress since Sat Jul 11 11:00:02 2015
>    2.24G scanned out of 69.5T at 1/s, (scan is slow, no estimated time)
>    0 repaired, 0.00% done

Observe evidence of the re-started scrub.  This may be tickling the 
problem which causes the panic.

The underlying problem needs to be identified and fixed.

Bob
-- 
Bob Friesenhahn
bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/

From derek at umiacs.umd.edu  Sun Jul 12 18:43:11 2015
From: derek at umiacs.umd.edu (Derek Yarnell)
Date: Sun, 12 Jul 2015 14:43:11 -0400
Subject: [OmniOS-discuss] ZFS crash/reboot loop
In-Reply-To: <alpine.GSO.2.01.1507121253560.4186@freddy.simplesystems.org>
References: <55A1D457.7060207@umiacs.umd.edu>
	<alpine.GSO.2.01.1507121253560.4186@freddy.simplesystems.org>
Message-ID: <55A2B53F.2010001@umiacs.umd.edu>

> The on-going scrub automatically restarts, apparently even in read-only
> mode.  You should 'zpool scrub -s poolname' ASAP after boot (if you can)
> to stop the ongoing scrub.

We have tried to stop the scrub but it seems you can not cancel a scrub
when the pool is mounted readonly.

-- 
Derek T. Yarnell
University of Maryland
Institute for Advanced Computer Studies

From alka at hfg-gmuend.de  Sun Jul 12 19:21:07 2015
From: alka at hfg-gmuend.de (=?windows-1252?Q?G=FCnther_Alka?=)
Date: Sun, 12 Jul 2015 21:21:07 +0200
Subject: [OmniOS-discuss] ZFS crash/reboot loop
In-Reply-To: <55A2B53F.2010001@umiacs.umd.edu>
References: <55A1D457.7060207@umiacs.umd.edu>
	<alpine.GSO.2.01.1507121253560.4186@freddy.simplesystems.org>
	<55A2B53F.2010001@umiacs.umd.edu>
Message-ID: <55A2BE23.20503@hfg-gmuend.de>

First action:
If you can mount the pool read-only, update your backup

Then
I would expect that a single bad disk is the reason of the problem on a 
write command. I would first check the system and fault log or 
smartvalues for hints about a bad disk. If there is a suspicious disk, 
remove that and retry a regular import.

If there is no hint
Next what I would try is a pool export. Then create a script that 
imports the pool followed by a scrub cancel. (Hope that the cancel is 
faster than the crash). Then check logs during some pool activity.

If this does not help, I would remove all data disks and bootup.
Then hot-plug disk by disk and check if its detected properly and check 
logs. Your pool remains offline until enough disks come back.
Adding disk by disk and checking logs should help to find a bad disk 
that initiates a crash

Next option is, try a pool import where always one or next disk is 
missing. Until there is no write, missing disks are not a problem with 
ZFS (you may need to clear errors).

Last option:
use another server where you try to import (mainboard, power,  hba or 
backplane problem) remove all disks and do a nondestructive or smart 
test on another machine


Gea

On 12.07.2015 20:43, Derek Yarnell wrote:
>> The on-going scrub automatically restarts, apparently even in read-only
>> mode.  You should 'zpool scrub -s poolname' ASAP after boot (if you can)
>> to stop the ongoing scrub.
> We have tried to stop the scrub but it seems you can not cancel a scrub
> when the pool is mounted readonly.
>



From derek at umiacs.umd.edu  Mon Jul 13 00:26:54 2015
From: derek at umiacs.umd.edu (Derek Yarnell)
Date: Sun, 12 Jul 2015 20:26:54 -0400
Subject: [OmniOS-discuss] ZFS crash/reboot loop
In-Reply-To: <55A2BE23.20503@hfg-gmuend.de>
References: <55A1D457.7060207@umiacs.umd.edu>	<alpine.GSO.2.01.1507121253560.4186@freddy.simplesystems.org>	<55A2B53F.2010001@umiacs.umd.edu>
	<55A2BE23.20503@hfg-gmuend.de>
Message-ID: <55A305CE.8050408@umiacs.umd.edu>

On 7/12/15 3:21 PM, G?nther Alka wrote:
> First action:
> If you can mount the pool read-only, update your backup

We are securing all the non-scratch data currently before messing with
the pool any more.  We had backups as recent as the night before but it
is still going to be faster to pull the current data from the readonly
pool than from backups.

> Then
> I would expect that a single bad disk is the reason of the problem on a
> write command. I would first check the system and fault log or
> smartvalues for hints about a bad disk. If there is a suspicious disk,
> remove that and retry a regular import.

We have pulled all disks individually yesterday to test this exact
theory.  We have hit the mpt_sas disk failure panics before so we had
already tried this.

> If there is no hint
> Next what I would try is a pool export. Then create a script that
> imports the pool followed by a scrub cancel. (Hope that the cancel is
> faster than the crash). Then check logs during some pool activity.

If I have not imported the pool RW can I export the pool?  I thought we
have tried this but I will have to confer.

> If this does not help, I would remove all data disks and bootup.
> Then hot-plug disk by disk and check if its detected properly and check
> logs. Your pool remains offline until enough disks come back.
> Adding disk by disk and checking logs should help to find a bad disk
> that initiates a crash

This is interesting and we will try this once we secure the data.

> Next option is, try a pool import where always one or next disk is
> missing. Until there is no write, missing disks are not a problem with
> ZFS (you may need to clear errors).

Wouldn't this be the same as above hot-plugging disk by disk?

> Last option:
> use another server where you try to import (mainboard, power,  hba or
> backplane problem) remove all disks and do a nondestructive or smart
> test on another machine

Sadly we do not have a spare chassis with 40 slots around to test this.
 I am so far unconvinced that this is a hardware problem though.

We will most likely boot up into linux live CD to run smartctl and see
if it has any information on the disks.

-- 
Derek T. Yarnell
University of Maryland
Institute for Advanced Computer Studies

From richard.elling at richardelling.com  Mon Jul 13 01:18:17 2015
From: richard.elling at richardelling.com (Richard Elling)
Date: Sun, 12 Jul 2015 18:18:17 -0700
Subject: [OmniOS-discuss] ZFS crash/reboot loop
In-Reply-To: <55A305CE.8050408@umiacs.umd.edu>
References: <55A1D457.7060207@umiacs.umd.edu>
	<alpine.GSO.2.01.1507121253560.4186@freddy.simplesystems.org>
	<55A2B53F.2010001@umiacs.umd.edu> <55A2BE23.20503@hfg-gmuend.de>
	<55A305CE.8050408@umiacs.umd.edu>
Message-ID: <10F57376-3A1A-4C2F-B6D7-70A1EDC54C90@richardelling.com>


> On Jul 12, 2015, at 5:26 PM, Derek Yarnell <derek at umiacs.umd.edu> wrote:
> 
> On 7/12/15 3:21 PM, G?nther Alka wrote:
>> First action:
>> If you can mount the pool read-only, update your backup
> 
> We are securing all the non-scratch data currently before messing with
> the pool any more.  We had backups as recent as the night before but it
> is still going to be faster to pull the current data from the readonly
> pool than from backups.
> 
>> Then
>> I would expect that a single bad disk is the reason of the problem on a
>> write command. I would first check the system and fault log or
>> smartvalues for hints about a bad disk. If there is a suspicious disk,
>> remove that and retry a regular import.
> 
> We have pulled all disks individually yesterday to test this exact
> theory.  We have hit the mpt_sas disk failure panics before so we had
> already tried this.

I don't believe this is a bad disk.

Some additional block pointer verification code was added in changeset
f63ab3d5a84a12b474655fc7e700db3efba6c4c9 and likely is the cause
of this assertion. In general, assertion failures are almost always software
problems -- the programmer didn't see what they expected.

Dan, if you're listening, Matt would be the best person to weigh-in on this.
 -- richard

> 
>> If there is no hint
>> Next what I would try is a pool export. Then create a script that
>> imports the pool followed by a scrub cancel. (Hope that the cancel is
>> faster than the crash). Then check logs during some pool activity.
> 
> If I have not imported the pool RW can I export the pool?  I thought we
> have tried this but I will have to confer.
> 
>> If this does not help, I would remove all data disks and bootup.
>> Then hot-plug disk by disk and check if its detected properly and check
>> logs. Your pool remains offline until enough disks come back.
>> Adding disk by disk and checking logs should help to find a bad disk
>> that initiates a crash
> 
> This is interesting and we will try this once we secure the data.
> 
>> Next option is, try a pool import where always one or next disk is
>> missing. Until there is no write, missing disks are not a problem with
>> ZFS (you may need to clear errors).
> 
> Wouldn't this be the same as above hot-plugging disk by disk?
> 
>> Last option:
>> use another server where you try to import (mainboard, power,  hba or
>> backplane problem) remove all disks and do a nondestructive or smart
>> test on another machine
> 
> Sadly we do not have a spare chassis with 40 slots around to test this.
> I am so far unconvinced that this is a hardware problem though.
> 
> We will most likely boot up into linux live CD to run smartctl and see
> if it has any information on the disks.
> 
> -- 
> Derek T. Yarnell
> University of Maryland
> Institute for Advanced Computer Studies
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss


From henson at acm.org  Mon Jul 13 02:30:40 2015
From: henson at acm.org (Paul B. Henson)
Date: Sun, 12 Jul 2015 19:30:40 -0700
Subject: [OmniOS-discuss] ZFS crash/reboot loop
In-Reply-To: <10F57376-3A1A-4C2F-B6D7-70A1EDC54C90@richardelling.com>
References: <55A1D457.7060207@umiacs.umd.edu>
	<alpine.GSO.2.01.1507121253560.4186@freddy.simplesystems.org>
	<55A2B53F.2010001@umiacs.umd.edu> <55A2BE23.20503@hfg-gmuend.de>
	<55A305CE.8050408@umiacs.umd.edu>
	<10F57376-3A1A-4C2F-B6D7-70A1EDC54C90@richardelling.com>
Message-ID: <20150713023039.GJ11967@bender.unx.cpp.edu>

On Sun, Jul 12, 2015 at 06:18:17PM -0700, Richard Elling wrote:

> Some additional block pointer verification code was added in changeset
> f63ab3d5a84a12b474655fc7e700db3efba6c4c9 and likely is the cause
> of this assertion. In general, assertion failures are almost always software
> problems -- the programmer didn't see what they expected.

If this is something that might have been ignored prior to this code
change, maybe they could set aok to avoid panicking when they import the
pool to recover data? Not very familiar with that technique myself but
I've seen it mentioned frequently in cases like this, unless things have
changed since then.


From danmcd at omniti.com  Mon Jul 13 04:51:02 2015
From: danmcd at omniti.com (Dan McDonald)
Date: Mon, 13 Jul 2015 00:51:02 -0400
Subject: [OmniOS-discuss] pkgsrc-current OmniOS 170cea2/i386 2015-07-09
	21:35
In-Reply-To: <CA+U3Mf6yA5VHgr5WdS9dhiEHs1epCmr22V_0KZ-S25soeWwnHw@mail.gmail.com>
References: <CA+U3Mf6yA5VHgr5WdS9dhiEHs1epCmr22V_0KZ-S25soeWwnHw@mail.gmail.com>
Message-ID: <811B3E16-0565-4F36-94F3-AACC0CBB2D59@omniti.com>

<Mucho snippage deleted!>

I also saw you mention this indirectly on twitter.

Generally, the OmniOS release should mention which release.  170cea2 is r151014.  It's good to mention that alongside the uname as that's how most of us lock in on a release.

Is there anything we can fix to help these move along?

Also, you ARE aware that pkgsrc's Jonathan Perkin works for Joyent, and does work to make sure pkgsrc bits build on all illumos distros, right?

Thanks,
Dan


From danmcd at omniti.com  Mon Jul 13 04:55:13 2015
From: danmcd at omniti.com (Dan McDonald)
Date: Mon, 13 Jul 2015 00:55:13 -0400
Subject: [OmniOS-discuss] ZFS crash/reboot loop
In-Reply-To: <10F57376-3A1A-4C2F-B6D7-70A1EDC54C90@richardelling.com>
References: <55A1D457.7060207@umiacs.umd.edu>
	<alpine.GSO.2.01.1507121253560.4186@freddy.simplesystems.org>
	<55A2B53F.2010001@umiacs.umd.edu> <55A2BE23.20503@hfg-gmuend.de>
	<55A305CE.8050408@umiacs.umd.edu>
	<10F57376-3A1A-4C2F-B6D7-70A1EDC54C90@richardelling.com>
Message-ID: <4485A150-518B-4139-9665-610841F13DA8@omniti.com>


> On Jul 12, 2015, at 9:18 PM, Richard Elling <richard.elling at richardelling.com> wrote:
> 
> Dan, if you're listening, Matt would be the best person to weigh-in on this.

Yes he would be, Richard..

The panic in the arc_get_data_buf() paths is similar to older problems we'd seen in r151006.

Derek, do you have a kernel coredump from these?  I know you've been panic-and-reboot-and-panic-ing, but if you can get savecore(1M) to do its thing, having that dump would be useful.

Thanks,
Dan


From chip at innovates.com  Mon Jul 13 11:58:00 2015
From: chip at innovates.com (Schweiss, Chip)
Date: Mon, 13 Jul 2015 06:58:00 -0500
Subject: [OmniOS-discuss] big zfs storage?
In-Reply-To: <CAESZ+_-zWA31os-tw+uvW5+FRCXwM6dt_VxgoNOAtq6fA==d_Q@mail.gmail.com>
References: <559EE5BF.4040900@kateley.com>
	<CALeZrrQ1nVjCFiTPEnDdpvCyHHDw7pm6z3tKH=678gHtR3c5tw@mail.gmail.com>
	<559FF4DE.4040202@kateley.com>
	<CALeZrrTSMhsD0K4Vw1khAVE=XDtThJ21kyGk-5DHvy5A_oBuug@mail.gmail.com>
	<CAESZ+_-zWA31os-tw+uvW5+FRCXwM6dt_VxgoNOAtq6fA==d_Q@mail.gmail.com>
Message-ID: <CALeZrrT10rqrdmrJ87U=Bo=XCd5QR-Z=DxA442nL2dxKvDTrJQ@mail.gmail.com>

Liam,

This report is encouraging.  Please share some details of your
configuration.   What disk failure parameters are have you set?   Which
JBODs and disks are you running?

I have mostly DataON JBODs and a some Supermicro.   DataON has PMC SAS
expanders and Supermicro has LSI, both setups have pretty much the same
behavior with disk failures.   All my servers are Supermicro with LSI HBAs.

If there's a magic combination of hardware and OS config out there that
solves the disk failure panic problem, I will certainly change my builds
going forward.

-Chip

On Fri, Jul 10, 2015 at 1:04 PM, Liam Slusser <lslusser at gmail.com> wrote:

> I have two 800T ZFS systems on OmniOS and a bunch of smaller <50T
> systems.  Things generally work very well.  We loose a disk here and there
> but its never resulted in downtime.  They're all on Dell hardware with LSI
> or Dell PERC controllers.
>
> Putting in smaller disk failure parameters, so disks fail quicker, was a
> big help when something does go wrong with a disk.
>
> thanks,
> liam
>
>
> On Fri, Jul 10, 2015 at 10:31 AM, Schweiss, Chip <chip at innovates.com>
> wrote:
>
>> Unfortunately for the past couple years panics on disk failure has been
>> the norm.   All my production systems are HA with RSF-1, so at least things
>> come back online relatively quick.  There are quite a few open tickets in
>> the Illumos bug tracker related to mpt_sas related panics.
>>
>> Most of the work to fix these problems has been committed in the past
>> year, though problems still exist.  For example, my systems are dual path
>> SAS, however, mpt_sas will panic if you pull a cable instead of dropping a
>> path to the disks.  Dan McDonald is actively working to resolve this.   He
>> is also pushing a bug fix in genunix from Nexenta that appears to fix a lot
>> of the panic problems.   I'll know for sure in a few months after I see a
>> disk or two drop if it truly fixes things.  Hans Rosenfeld at Nexenta is
>> responsible for most of the updates to mpt_sas including support for 3008
>> (12G SAS).
>>
>> I haven't run any 12G SAS yet, but plan to on my next build in a couple
>> months.   This will be about 300TB using an 84 disk JBOD.  All the code
>> from Nexenta to support the 3008 appears to be in Illumos now, and they
>> fully support it so I suspect it's pretty stable now.  From what I
>> understand there may be some 12G performance fixes coming sometime.
>>
>> The fault manager is nice when the system doesn't panic.  When it panics,
>> the fault manger never gets a chance to take action.  It is still the
>> consensus that is is better to run pools without hot spares because there
>> are situations the fault manager will do bad things.   I witnessed this
>> myself when building a system and the fault manger replaced 5 disks in a
>> raidz2 vdev inside 1 minute, trashing the pool.   I haven't completely
>> yield to the "best practice".  I now run one hot spare per pool.  I figure
>> with raidz2, the odds of the fault manager causing something catastrophic
>> is much less possible.
>>
>> -Chip
>>
>>
>>
>> On Fri, Jul 10, 2015 at 11:37 AM, Linda Kateley <lkateley at kateley.com>
>> wrote:
>>
>>>  I have to build and maintain my own system. I usually help others
>>> build(i teach zfs and freenas classes/consulting). I really love fault
>>> management in solaris and miss it. Just thought since it's my system and I
>>> get to choose I would use omni. I have 20+ years using solaris and only 2
>>> on freebsd.
>>>
>>> I like freebsd for how well tuned for zfs oob. I miss the network, v12n
>>> and resource controls in solaris.
>>>
>>> Concerned about panics on disk failure. Is that common?
>>>
>>>
>> linda
>>>
>>>
>>> On 7/9/15 9:30 PM, Schweiss, Chip wrote:
>>>
>>>   Linda,
>>>
>>>  I have 3.5 PB running under OmniOS.  All my systems have LSI 2108 HBAs
>>> which is considered the best choice for HBAs.
>>>
>>> Illumos leaves a bit to be desired with handling faults from disks or
>>> SAS problems, but things under OmniOS have been improving, much thanks to
>>> Dan McDonald and OmniTI.   We have a paid support on all of our production
>>> systems with OmniTI.  Their response and dedication has been very good.
>>> Other than the occasional panic and restart from a disk failure, OmniOS has
>>> been solid.   ZFS of course never has lost a single bit of information.
>>>
>>>  I'd be curious why you're looking to move, have there been specific
>>> problems under BSD or ZoL?  I've been slowly evaluating FreeBSD ZFS, but of
>>> course the skeletons in the closet never seem to come out until you do
>>> something big.
>>>
>>>  -Chip
>>>
>>> On Thu, Jul 9, 2015 at 4:21 PM, Linda Kateley <lkateley at kateley.com>
>>> wrote:
>>>
>>>> Hey is there anyone out there running big zfs on omni?
>>>>
>>>> I have been doing mostly zol and freebsd for the last year but have to
>>>> build a 300+TB box and i want to come back home to roots(solaris). Feeling
>>>> kind of hesitant :) Also, if you had to do over, is there anything you
>>>> would do different.
>>>>
>>>> Also, what is the go to HBA these days? Seems like i saw stable code
>>>> for lsi 3008?
>>>>
>>>> TIA
>>>>
>>>> linda
>>>>
>>>>
>>>> _______________________________________________
>>>> OmniOS-discuss mailing list
>>>> OmniOS-discuss at lists.omniti.com
>>>> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>>>>
>>>
>>>
>>> --
>>> Linda Kateley
>>> Kateley Company
>>> Skype ID-kateleycohttp://kateleyco.com
>>>
>>>
>>
>> _______________________________________________
>> OmniOS-discuss mailing list
>> OmniOS-discuss at lists.omniti.com
>> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150713/7498e90d/attachment.html>

From derek at umiacs.umd.edu  Mon Jul 13 15:25:06 2015
From: derek at umiacs.umd.edu (Derek Yarnell)
Date: Mon, 13 Jul 2015 11:25:06 -0400
Subject: [OmniOS-discuss] ZFS crash/reboot loop
In-Reply-To: <4485A150-518B-4139-9665-610841F13DA8@omniti.com>
References: <55A1D457.7060207@umiacs.umd.edu>
	<alpine.GSO.2.01.1507121253560.4186@freddy.simplesystems.org>
	<55A2B53F.2010001@umiacs.umd.edu> <55A2BE23.20503@hfg-gmuend.de>
	<55A305CE.8050408@umiacs.umd.edu>
	<10F57376-3A1A-4C2F-B6D7-70A1EDC54C90@richardelling.com>
	<4485A150-518B-4139-9665-610841F13DA8@omniti.com>
Message-ID: <55A3D852.3000200@umiacs.umd.edu>

Hi Dan,

Sorry I have not dealt with dumpadm/savecore that much but it looks like
this is what you want.

https://obj.umiacs.umd.edu/derek_support/vmdump.0

Thanks,
derek

On 7/13/15 12:55 AM, Dan McDonald wrote:
> 
>> On Jul 12, 2015, at 9:18 PM, Richard Elling <richard.elling at richardelling.com> wrote:
>>
>> Dan, if you're listening, Matt would be the best person to weigh-in on this.
> 
> Yes he would be, Richard..
> 
> The panic in the arc_get_data_buf() paths is similar to older problems we'd seen in r151006.
> 
> Derek, do you have a kernel coredump from these?  I know you've been panic-and-reboot-and-panic-ing, but if you can get savecore(1M) to do its thing, having that dump would be useful.
> 
> Thanks,
> Dan
> 
> 

-- 
Derek T. Yarnell
University of Maryland
Institute for Advanced Computer Studies

From danmcd at omniti.com  Mon Jul 13 15:29:15 2015
From: danmcd at omniti.com (Dan McDonald)
Date: Mon, 13 Jul 2015 11:29:15 -0400
Subject: [OmniOS-discuss] ZFS crash/reboot loop
In-Reply-To: <55A3D852.3000200@umiacs.umd.edu>
References: <55A1D457.7060207@umiacs.umd.edu>
	<alpine.GSO.2.01.1507121253560.4186@freddy.simplesystems.org>
	<55A2B53F.2010001@umiacs.umd.edu> <55A2BE23.20503@hfg-gmuend.de>
	<55A305CE.8050408@umiacs.umd.edu>
	<10F57376-3A1A-4C2F-B6D7-70A1EDC54C90@richardelling.com>
	<4485A150-518B-4139-9665-610841F13DA8@omniti.com>
	<55A3D852.3000200@umiacs.umd.edu>
Message-ID: <02C09D2F-8B54-41CF-BD08-3045C9E767D4@omniti.com>


> On Jul 13, 2015, at 11:25 AM, Derek Yarnell <derek at umiacs.umd.edu> wrote:
> 
> https://obj.umiacs.umd.edu/derek_support/vmdump.0

Yeah, that's what I'm seeking.  Downloading it now to an r151014 box (you are running r151014 according to the first mail).  My normal '014 box is otherwise indisposed at the moment, so this dump may take a bit longer to analyze.  I can forward it along to the ZFS folks once I've done my initial analysis.

For bugs like these, I usually have to engage the illumos ZFS list.  If anyone here wants to follow along, I'll Cc: you on anything I report to them.

Thanks!
Dan


From danmcd at omniti.com  Mon Jul 13 15:46:42 2015
From: danmcd at omniti.com (Dan McDonald)
Date: Mon, 13 Jul 2015 11:46:42 -0400
Subject: [OmniOS-discuss] ZFS crash/reboot loop
In-Reply-To: <02C09D2F-8B54-41CF-BD08-3045C9E767D4@omniti.com>
References: <55A1D457.7060207@umiacs.umd.edu>
	<alpine.GSO.2.01.1507121253560.4186@freddy.simplesystems.org>
	<55A2B53F.2010001@umiacs.umd.edu> <55A2BE23.20503@hfg-gmuend.de>
	<55A305CE.8050408@umiacs.umd.edu>
	<10F57376-3A1A-4C2F-B6D7-70A1EDC54C90@richardelling.com>
	<4485A150-518B-4139-9665-610841F13DA8@omniti.com>
	<55A3D852.3000200@umiacs.umd.edu>
	<02C09D2F-8B54-41CF-BD08-3045C9E767D4@omniti.com>
Message-ID: <CF83CFA1-CFA3-46FD-8A39-1C882EFD1740@omniti.com>


> On Jul 13, 2015, at 11:29 AM, Dan McDonald <danmcd at omniti.com> wrote:
> 
> 
>> On Jul 13, 2015, at 11:25 AM, Derek Yarnell <derek at umiacs.umd.edu> wrote:
>> 
>> https://obj.umiacs.umd.edu/derek_support/vmdump.0
> 
> Yeah, that's what I'm seeking.  Downloading it now to an r151014 box (you are running r151014 according to the first mail).  My normal '014 box is otherwise indisposed at the moment, so this dump may take a bit longer to analyze.  I can forward it along to the ZFS folks once I've done my initial analysis.
> 
> For bugs like these, I usually have to engage the illumos ZFS list.  If anyone here wants to follow along, I'll Cc: you on anything I report to them.


Okay, it's a VERIFY() failure in zio_buf_alloc().  It's passed a size of 0 by its caller.  Observe this MDB interaction:

> $c
vpanic()
0xfffffffffba8b13d()
zio_buf_alloc+0x49(0)
arc_get_data_buf+0x12b(ffffff0d4071ca98)
arc_buf_alloc+0xd2(ffffff0d4dfec000, 0, 0, 1)
...<SNIP!>


0xffffff0d4071ca98 is an arc_buf_t, read off of disk.  The code in arc_get_data_buf starts with:

static void
arc_get_data_buf(arc_buf_t *buf)
{
        arc_state_t             *state = buf->b_hdr->b_l1hdr.b_state;
        uint64_t                size = buf->b_hdr->b_size;
        arc_buf_contents_t      type = arc_buf_type(buf->b_hdr);


So let's look at that size:

> ffffff0d4071ca98::print arc_buf_t b_hdr |::print arc_buf_hdr_t b_size
b_size = 0
> 

Ouch.  There's your zero.

I'm going to forward this very note to the illumos ZFS list.  I see ONE possible bugfix post-r151014 that might help:

commit 31c46cf23cd1cf4d66390a983dc5072d7d299ba2
Author: Alek Pinchuk <alek at nexenta.com>
Date:   Tue Jun 30 09:44:11 2015 -0700

    6033 arc_adjust() should search MFU lists for oldest buffer when adjusting MFU size
    Reviewed by: Saso Kiselkov <saso.kiselkov at nexenta.com>
    Reviewed by: Xin Li <delphij at delphij.net>
    Reviewed by: Prakash Surya <me at prakashsurya.com>
    Approved by: Matthew Ahrens <mahrens at delphix.com>

It's a small bug, and I shudder to say this, even hot-patchable on a running system if you're desperate.  :)

Thanks,
Dan
d

From derek at umiacs.umd.edu  Mon Jul 13 15:56:57 2015
From: derek at umiacs.umd.edu (Derek Yarnell)
Date: Mon, 13 Jul 2015 11:56:57 -0400
Subject: [OmniOS-discuss] ZFS crash/reboot loop
In-Reply-To: <CF83CFA1-CFA3-46FD-8A39-1C882EFD1740@omniti.com>
References: <55A1D457.7060207@umiacs.umd.edu>
	<alpine.GSO.2.01.1507121253560.4186@freddy.simplesystems.org>
	<55A2B53F.2010001@umiacs.umd.edu> <55A2BE23.20503@hfg-gmuend.de>
	<55A305CE.8050408@umiacs.umd.edu>
	<10F57376-3A1A-4C2F-B6D7-70A1EDC54C90@richardelling.com>
	<4485A150-518B-4139-9665-610841F13DA8@omniti.com>
	<55A3D852.3000200@umiacs.umd.edu>
	<02C09D2F-8B54-41CF-BD08-3045C9E767D4@omniti.com>
	<CF83CFA1-CFA3-46FD-8A39-1C882EFD1740@omniti.com>
Message-ID: <55A3DFC9.5060603@umiacs.umd.edu>

>> ffffff0d4071ca98::print arc_buf_t b_hdr |::print arc_buf_hdr_t b_size
> b_size = 0
>>
> 
> Ouch.  There's your zero.
> 
> I'm going to forward this very note to the illumos ZFS list.  I see ONE possible bugfix post-r151014 that might help:
> 
> commit 31c46cf23cd1cf4d66390a983dc5072d7d299ba2
> Author: Alek Pinchuk <alek at nexenta.com>
> Date:   Tue Jun 30 09:44:11 2015 -0700
> 
>     6033 arc_adjust() should search MFU lists for oldest buffer when adjusting MFU size
>     Reviewed by: Saso Kiselkov <saso.kiselkov at nexenta.com>
>     Reviewed by: Xin Li <delphij at delphij.net>
>     Reviewed by: Prakash Surya <me at prakashsurya.com>
>     Approved by: Matthew Ahrens <mahrens at delphix.com>
> 
> It's a small bug, and I shudder to say this, even hot-patchable on a running system if you're desperate.  :)
> 

Hi Dan,

I don't need to hot patch (cold patch would be fine) so any update that
I can apply and reboot would be fine.  We have a second OmniOS r14 copy
running that we are happy to patch in any way possible to get it mounted rw.

Thanks,
derek

-- 
Derek T. Yarnell
University of Maryland
Institute for Advanced Computer Studies

From danmcd at omniti.com  Mon Jul 13 16:02:26 2015
From: danmcd at omniti.com (Dan McDonald)
Date: Mon, 13 Jul 2015 12:02:26 -0400
Subject: [OmniOS-discuss] ZFS crash/reboot loop
In-Reply-To: <55A3DFC9.5060603@umiacs.umd.edu>
References: <55A1D457.7060207@umiacs.umd.edu>
	<alpine.GSO.2.01.1507121253560.4186@freddy.simplesystems.org>
	<55A2B53F.2010001@umiacs.umd.edu> <55A2BE23.20503@hfg-gmuend.de>
	<55A305CE.8050408@umiacs.umd.edu>
	<10F57376-3A1A-4C2F-B6D7-70A1EDC54C90@richardelling.com>
	<4485A150-518B-4139-9665-610841F13DA8@omniti.com>
	<55A3D852.3000200@umiacs.umd.edu>
	<02C09D2F-8B54-41CF-BD08-3045C9E767D4@omniti.com>
	<CF83CFA1-CFA3-46FD-8A39-1C882EFD1740@omniti.com>
	<55A3DFC9.5060603@umiacs.umd.edu>
Message-ID: <1E616002-B9A3-4C1C-83AB-90F8B18F250B@omniti.com>


> On Jul 13, 2015, at 11:56 AM, Derek Yarnell <derek at umiacs.umd.edu> wrote:
> 
> I don't need to hot patch (cold patch would be fine) so any update that
> I can apply and reboot would be fine.  We have a second OmniOS r14 copy
> running that we are happy to patch in any way possible to get it mounted rw.

IF (and only if) it's the bug I mentioned that's the problem.

I want ZFS experts to take a look as well.  It's on the ZFS list now, so we'll see what happens.  If you're REALLY feeling brave, I can build a replacement ZFS module with 6033 in place for you to try, but I can't promise it'll work.

Dan


From venture37 at gmail.com  Mon Jul 13 21:11:05 2015
From: venture37 at gmail.com (Sevan / Venture37)
Date: Mon, 13 Jul 2015 22:11:05 +0100
Subject: [OmniOS-discuss] pkgsrc-current OmniOS 170cea2/i386 2015-07-09
	21:35
In-Reply-To: <811B3E16-0565-4F36-94F3-AACC0CBB2D59@omniti.com>
References: <CA+U3Mf6yA5VHgr5WdS9dhiEHs1epCmr22V_0KZ-S25soeWwnHw@mail.gmail.com>
	<811B3E16-0565-4F36-94F3-AACC0CBB2D59@omniti.com>
Message-ID: <CA+U3Mf5=vUi+2sDhQ2qXNCLMjUtzMsi=CDcbJHA1gtTbCAEOjg@mail.gmail.com>

Hi Dan,

On 13 July 2015 at 05:51, Dan McDonald <danmcd at omniti.com> wrote:
> <Mucho snippage deleted!>
>
> I also saw you mention this indirectly on twitter.
>
> Generally, the OmniOS release should mention which release.  170cea2 is r151014.  It's good to mention that alongside the uname as that's how most of us lock in on a release.

This was only recently implemented at pkgsrcCon and pending review so
it's not something that's in the pkgsrc tree yet, previously it was
just marked as Solaris 11 and I was manually adding illumos version
info manually which was clumsy. It can be changed, not a problem.
Where can I get the release number (the r151014) from the system? (I'm
currently unable to reach the OmniOS zone I'm using to check for
myself)

> Is there anything we can fix to help these move along?

I'd like to propose a fix for the gettext issue but I've not been able
to get very far with the buildctl tool in omnios-build. It appears to
be incapable of generating a repo or utilising a pre-prepared repo for
me (I used  a different path for the location of my repo if that makes
any difference).
I wanted to evaluate the solution proposed previously in this thread
that is to rebuild gettext with -rpath specified which includes the
lib directory of the GCC with building with so that libgomp can be
found.
Happy to get a pull request in for the change necessary once I've
worked out what's needed.

> Also, you ARE aware that pkgsrc's Jonathan Perkin works for Joyent, and does work to make sure pkgsrc bits build on all illumos distros, right?

Yes I'm aware that Jonathan Perkin works for Joyent on pkgsrc which
runs on SmartOS and should in theory work on the other illumos distros
provided the other distros are in sync with the changes Joyent made to
the source. :)
The gettext issue is not present in SmartOS and is specific to OmniOS.
Building specifically on the various distros is of benefit to pkgsrc
because it builds against another version of toolchain on each distro.
One of the proposed solutions has been to ignore everything & rebuild
from scratch using components in pkgsrc. - This would cause a
substantial delay in bootstrap time get everything built.
Another proposal was to patch pkgsrc so that gettext bundled is
ignored and replaced with the version in pkgsrc - theres an effort to
generate a patch for that.

I personally would rather get the gettext fixed upstream (in OmniOS)
and there doesn't need any work around in the pkgsrc tree or having to
recreate the user land components to work around a single component.



Sevan / Venture37

From mir at miras.org  Mon Jul 13 22:11:09 2015
From: mir at miras.org (Michael Rasmussen)
Date: Tue, 14 Jul 2015 00:11:09 +0200
Subject: [OmniOS-discuss] pkgsrc-current OmniOS 170cea2/i386 2015-07-09
 21:35
In-Reply-To: <CA+U3Mf5=vUi+2sDhQ2qXNCLMjUtzMsi=CDcbJHA1gtTbCAEOjg@mail.gmail.com>
References: <CA+U3Mf6yA5VHgr5WdS9dhiEHs1epCmr22V_0KZ-S25soeWwnHw@mail.gmail.com>
	<811B3E16-0565-4F36-94F3-AACC0CBB2D59@omniti.com>
	<CA+U3Mf5=vUi+2sDhQ2qXNCLMjUtzMsi=CDcbJHA1gtTbCAEOjg@mail.gmail.com>
Message-ID: <20150714001109.7aefa11a@sleipner.datanom.net>

On Mon, 13 Jul 2015 22:11:05 +0100
"Sevan / Venture37" <venture37 at gmail.com> wrote:

> 
> This was only recently implemented at pkgsrcCon and pending review so
> it's not something that's in the pkgsrc tree yet, previously it was
> just marked as Solaris 11 and I was manually adding illumos version
> info manually which was clumsy. It can be changed, not a problem.
> Where can I get the release number (the r151014) from the system? (I'm
> currently unable to reach the OmniOS zone I'm using to check for
> myself)
> 
pkg info kernel |perl -ne 'print "$1\n" if /Branch:\s+\d\.(\d+)/'

or to add 'r'

pkg info kernel |perl -ne 'print "r$1\n" if /Branch:\s+\d\.(\d+)/'

-- 
Hilsen/Regards
Michael Rasmussen

Get my public GnuPG keys:
michael <at> rasmussen <dot> cc
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E
mir <at> datanom <dot> net
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C
mir <at> miras <dot> org
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917
--------------------------------------------------------------
/usr/games/fortune -es says:
No discipline is ever requisite to force attendance upon lectures which
are really worth the attending.
		-- Adam Smith, "The Wealth of Nations"
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 181 bytes
Desc: OpenPGP digital signature
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150714/74dceb11/attachment.bin>

From danmcd at omniti.com  Tue Jul 14 01:14:38 2015
From: danmcd at omniti.com (Dan McDonald)
Date: Mon, 13 Jul 2015 21:14:38 -0400
Subject: [OmniOS-discuss] pkgsrc-current OmniOS 170cea2/i386 2015-07-09
	21:35
In-Reply-To: <CA+U3Mf5=vUi+2sDhQ2qXNCLMjUtzMsi=CDcbJHA1gtTbCAEOjg@mail.gmail.com>
References: <CA+U3Mf6yA5VHgr5WdS9dhiEHs1epCmr22V_0KZ-S25soeWwnHw@mail.gmail.com>
	<811B3E16-0565-4F36-94F3-AACC0CBB2D59@omniti.com>
	<CA+U3Mf5=vUi+2sDhQ2qXNCLMjUtzMsi=CDcbJHA1gtTbCAEOjg@mail.gmail.com>
Message-ID: <A3703B3C-8953-4CB1-BC11-7C898AB6F182@omniti.com>


> On Jul 13, 2015, at 5:11 PM, Sevan / Venture37 <venture37 at gmail.com> wrote:
> 
> Where can I get the release number (the r151014) from the system? (I'm
> currently unable to reach the OmniOS zone I'm using to check for
> myself)

/etc/release.  (Yes, this even works in zones.)

>> Is there anything we can fix to help these move along?
> 
> I'd like to propose a fix for the gettext issue but I've not been able
> to get very far with the buildctl tool in omnios-build. It appears to
> be incapable of generating a repo or utilising a pre-prepared repo for
> me (I used  a different path for the location of my repo if that makes
> any difference).

To rebuild gettext from omnios-build:

	Set the PKGSRVR environment variable to be where your repo lives.  e.g. file:///data/builder/builder.repo/

	cd $PATH_TO/omnios-build/build

	./buildctl build gettext

Say no to pkglint and say yes to publication, verifying that PKGSRVR was set in your environment.

> I wanted to evaluate the solution proposed previously in this thread
> that is to rebuild gettext with -rpath specified which includes the
> lib directory of the GCC with building with so that libgomp can be
> found.
> Happy to get a pull request in for the change necessary once I've
> worked out what's needed.

We take 'em in omnios-build.

> The gettext issue is not present in SmartOS and is specific to OmniOS.

Weird... as I don't see any changes in the libc callers:

bloody(~/ws)[0]% diff -r illumos-{gate,joyent}/usr/src/lib/libc/port/i18n/
bloody(~/ws)[0]% diff -r illumos-{omnios,joyent}/usr/src/lib/libc/port/i18n/
bloody(~/ws)[0]% 


> Building specifically on the various distros is of benefit to pkgsrc
> because it builds against another version of toolchain on each distro.
> One of the proposed solutions has been to ignore everything & rebuild
> from scratch using components in pkgsrc. - This would cause a
> substantial delay in bootstrap time get everything built.
> Another proposal was to patch pkgsrc so that gettext bundled is
> ignored and replaced with the version in pkgsrc - theres an effort to
> generate a patch for that.
> 
> I personally would rather get the gettext fixed upstream (in OmniOS)
> and there doesn't need any work around in the pkgsrc tree or having to
> recreate the user land components to work around a single component.

I'm game for any changes.  Your best bet is to modify $PATH_TO/omnios-build/build/gettext/build.sh.

Dan


From derek at umiacs.umd.edu  Tue Jul 14 01:26:14 2015
From: derek at umiacs.umd.edu (Derek Yarnell)
Date: Mon, 13 Jul 2015 21:26:14 -0400
Subject: [OmniOS-discuss] ZFS crash/reboot loop
In-Reply-To: <1E616002-B9A3-4C1C-83AB-90F8B18F250B@omniti.com>
References: <55A1D457.7060207@umiacs.umd.edu>
	<alpine.GSO.2.01.1507121253560.4186@freddy.simplesystems.org>
	<55A2B53F.2010001@umiacs.umd.edu> <55A2BE23.20503@hfg-gmuend.de>
	<55A305CE.8050408@umiacs.umd.edu>
	<10F57376-3A1A-4C2F-B6D7-70A1EDC54C90@richardelling.com>
	<4485A150-518B-4139-9665-610841F13DA8@omniti.com>
	<55A3D852.3000200@umiacs.umd.edu>
	<02C09D2F-8B54-41CF-BD08-3045C9E767D4@omniti.com>
	<CF83CFA1-CFA3-46FD-8A39-1C882EFD1740@omniti.com>
	<55A3DFC9.5060603@umiacs.umd.edu>
	<1E616002-B9A3-4C1C-83AB-90F8B18F250B@omniti.com>
Message-ID: <55A46536.6070808@umiacs.umd.edu>



On 7/13/15 12:02 PM, Dan McDonald wrote:
> 
>> On Jul 13, 2015, at 11:56 AM, Derek Yarnell <derek at umiacs.umd.edu> wrote:
>>
>> I don't need to hot patch (cold patch would be fine) so any update that
>> I can apply and reboot would be fine.  We have a second OmniOS r14 copy
>> running that we are happy to patch in any way possible to get it mounted rw.
> 
> IF (and only if) it's the bug I mentioned that's the problem.
> 
> I want ZFS experts to take a look as well.  It's on the ZFS list now, so we'll see what happens.  If you're REALLY feeling brave, I can build a replacement ZFS module with 6033 in place for you to try, but I can't promise it'll work.

Hi Dan,

I would be happy to try to test a build with 6033 on it to see if that
is my issue.  We have secured all the critical data and only have
scratch data left.  So at this point I would happy to take a chance to
see if this will fix the issue.

Thanks,
derek

-- 
Derek T. Yarnell
University of Maryland
Institute for Advanced Computer Studies

From danmcd at omniti.com  Tue Jul 14 01:34:09 2015
From: danmcd at omniti.com (Dan McDonald)
Date: Mon, 13 Jul 2015 21:34:09 -0400
Subject: [OmniOS-discuss] For developers - the dmake-in-illumos flag day
Message-ID: <E1484A84-B2B2-4F36-9F9D-D9B4C634E351@omniti.com>

Hello folks who use OmniOS to build illumos-{omnios,gate}!

Y'all have likely seen Rich Lowe's flag day announcement.  The developer/build/make package is now in illumos-gate (and the master branch of illumos-omnios).  Sometime in the next 48 hours I plan to update the bloody IPS repo server with new bits that embrace this flag day.

What used to be developer/build/make will now be built in illumos-omnios, and contain less bits.  To that end, two new packages:

	developer/as

	developer/versioning/sccs

have been created.  "developer/as" is a BIT of a misnomer, as it includes other closed-source binaries as well:

bloody(~)[0]% pkg contents developer/as
PATH
usr/bin/as
usr/lib/amd64
usr/lib/amd64/libtdf.so
usr/lib/amd64/libtdf.so.1
usr/lib/amd64/libxprof.so
usr/lib/amd64/libxprof.so.1
usr/lib/amd64/libxprof_audit.so
usr/lib/amd64/libxprof_audit.so.1
usr/lib/libtdf.so
usr/lib/libtdf.so.1
usr/lib/libxprof.so
usr/lib/libxprof.so.1
usr/lib/libxprof_audit.so
usr/lib/libxprof_audit.so.1
bloody(~)[0]% 

I may choose to split those out further, but for the next bloody build, it'll stay like that.

IF you are attempting to build the world of omnios-build or all of OOod, you will be unpleasantly surprised.  There are chicken & egg problems to overcome.  I've done it for you, and when the next update of bloody happens, you'll be just fine building the new world order of things.

Thanks, and watch this space for an update to bloody.

Dan


From cks at cs.toronto.edu  Tue Jul 14 18:38:42 2015
From: cks at cs.toronto.edu (Chris Siebenmann)
Date: Tue, 14 Jul 2015 14:38:42 -0400
Subject: [OmniOS-discuss] Clues for tracking down why kernel memory isn't
	being released?
Message-ID: <20150714183842.3D5AF7A0255@apps0.cs.toronto.edu>

 We have one ZFS-based NFS fileserver that persistently runs at a very
high level of non-ARC kernel memory usage that never seems to shrink.
On a 128 GB machine, mdb's ::memstat reports 95% memory usage by just
'Kernel' while the ZFS ARC is only at about 21 GB (as reported by
'kstat -m') although c_max should allow it to grow much bigger.

 According to ::kmastat, a *huge* amount of this memory appears to be
vanishing into allocated but not used kmem_alloc_131072 slab buffers:

> ::kmastat
cache                            buf       buf       buf memory      alloc alloc
name                            size    in use     total in use    succeed  fail
------------------------------ ----- --------- --------- ------ ---------- -----
[...]
kmem_alloc_131072               128K         6    613033  74.8G  196862991     0

The next closest slab memory user appears to have much more active buffers
in use:

kmem_va_4096                      4K   6644009  10834464  41.3G   85384529     0

 I assume that under normal circumstances, at least some of the memory
claimed by these free buffers would be released back to the overall
kernel memory pool. Perhaps I'm incorrect here and the kmem_alloc_*
slabs hold on to basically all of the memory they ever grab?

 If I'm correct, does anyone know of a reason why the kmem_alloc_131072
memory here is not being released back to the system? Is there any way
to push OmniOS so that it is released back, eg by allocating and using
a lot of memory at the user level?

 Thanks in advance.

	- cks

From danmcd at omniti.com  Tue Jul 14 18:56:00 2015
From: danmcd at omniti.com (Dan McDonald)
Date: Tue, 14 Jul 2015 14:56:00 -0400
Subject: [OmniOS-discuss] OmniOS bloody update
In-Reply-To: <E1484A84-B2B2-4F36-9F9D-D9B4C634E351@omniti.com>
References: <E1484A84-B2B2-4F36-9F9D-D9B4C634E351@omniti.com>
Message-ID: <E28AA803-91E6-44D4-85CE-D7C752D87907@omniti.com>

As I previously wrote...

> On Jul 13, 2015, at 9:34 PM, Dan McDonald <danmcd at omniti.com> wrote:
> 
> IF you are attempting to build the world of omnios-build or all of OOod, you will be unpleasantly surprised.  There are chicken & egg problems to overcome.  I've done it for you, and when the next update of bloody happens, you'll be just fine building the new world order of things.
> 
> Thanks, and watch this space for an update to bloody.

Here's your update for bloody!  Bloody users can & should "pkg update".  I pushed out all of the bits this time, so you will have to "pkg update pkg" before doing a full blown update.

illumos-omnios was built from master, commit cc0bbb2.  Highlights include:

- dmake is now built in illumos-{gate,omnios}.

- rpcgen doesn't generate absolute #includes.

- ZFS fixes from upstream

- man page corrections


omnios-build was built from master, commit 4be148b.  Highlights include:

- New developer/versioning/sccs and developer/as packages that MUST BE INSTALLED by developers who used to expect those binaries from developer/build/make.

- OpenSSL to 1.0.2d (already released when it was announced, though).

- Fixes to openssh to allow better usage on OmniOS. (Thank you Lauri "lotheac" Tirkkonen)

- Git updated to 2.4.5

- NTP to 4.2.8p3 (also already released when it was announced).


Happy updating!
Dan

p.s. Don't forget about developer/as and developer/versioning/sccs, developers!

From gate03 at landcroft.co.uk  Wed Jul 15 12:10:15 2015
From: gate03 at landcroft.co.uk (Michael Mounteney)
Date: Wed, 15 Jul 2015 22:10:15 +1000
Subject: [OmniOS-discuss] re-tune round-robin reading from a mirror
Message-ID: <20150715221015.49e0abaf@punda-mlia>

Hello list;  is it possible with OmniOS to have a multi-way mirror with
one disk being an SSD and the rest magnetic;  then to tune ZFS to
perform all reads from the SSD?  for the sake of performance.  The
default case is round-robin reading, which is the best if all disks are
of approximately equal performance, especially if they're on separate
controllers.  But SSD changes that.

______________
Michael Mounteney

From jimklimov at cos.ru  Wed Jul 15 15:09:47 2015
From: jimklimov at cos.ru (Jim Klimov)
Date: Wed, 15 Jul 2015 17:09:47 +0200
Subject: [OmniOS-discuss] re-tune round-robin reading from a mirror
In-Reply-To: <20150715221015.49e0abaf@punda-mlia>
References: <20150715221015.49e0abaf@punda-mlia>
Message-ID: <ACB6E6F4-4F24-409D-A432-C61BFA564C43@cos.ru>

15 ???? 2015??. 14:10:15 CEST, Michael Mounteney <gate03 at landcroft.co.uk> ?????:
>Hello list;  is it possible with OmniOS to have a multi-way mirror with
>one disk being an SSD and the rest magnetic;  then to tune ZFS to
>perform all reads from the SSD?  for the sake of performance.  The
>default case is round-robin reading, which is the best if all disks are
>of approximately equal performance, especially if they're on separate
>controllers.  But SSD changes that.
>
>______________
>Michael Mounteney
>_______________________________________________
>OmniOS-discuss mailing list
>OmniOS-discuss at lists.omniti.com
>http://lists.omniti.com/mailman/listinfo/omnios-discuss

When I last asked a few years ago (but IIRC forma mirror of local+iSCSI vdevs), the answer was along the lines that round-robin first considers the available devices. If the faster (ssd, local) device has no queue, it gets the load while the slower device still struggles with the task it has, so on average the faster device serves more io's - but not 100%. Queue depth tuning can also help here.

Jim

--
Typos courtesy of K-9 Mail on my Samsung Android

From chip at innovates.com  Wed Jul 15 15:45:55 2015
From: chip at innovates.com (Schweiss, Chip)
Date: Wed, 15 Jul 2015 10:45:55 -0500
Subject: [OmniOS-discuss] re-tune round-robin reading from a mirror
In-Reply-To: <ACB6E6F4-4F24-409D-A432-C61BFA564C43@cos.ru>
References: <20150715221015.49e0abaf@punda-mlia>
	<ACB6E6F4-4F24-409D-A432-C61BFA564C43@cos.ru>
Message-ID: <CALeZrrQS9fGJm3OY+jbza9SuZ9KfiT=AHpnd-goLk6Rg8fWFtw@mail.gmail.com>

This is a very interesting idea.  It could allow for the creation of a
scratch pool with great ROI.   I have a particular need for an extremely
high read rate pool for data analysis that will leverage the fattest read
optimized SSDs I can get for the dollar.  I was considering this to be
raidz1, but if mirroring with disks would work this way, it could be even
better bang for the buck.

Did you ever do any actual testing with this type of setup?  I'd love to
see some real world performance data.

-Chip

On Wed, Jul 15, 2015 at 10:09 AM, Jim Klimov <jimklimov at cos.ru> wrote:

> 15 ???? 2015 ?. 14:10:15 CEST, Michael Mounteney <gate03 at landcroft.co.uk>
> ?????:
> >Hello list;  is it possible with OmniOS to have a multi-way mirror with
> >one disk being an SSD and the rest magnetic;  then to tune ZFS to
> >perform all reads from the SSD?  for the sake of performance.  The
> >default case is round-robin reading, which is the best if all disks are
> >of approximately equal performance, especially if they're on separate
> >controllers.  But SSD changes that.
> >
> >______________
> >Michael Mounteney
> >_______________________________________________
> >OmniOS-discuss mailing list
> >OmniOS-discuss at lists.omniti.com
> >http://lists.omniti.com/mailman/listinfo/omnios-discuss
>
> When I last asked a few years ago (but IIRC forma mirror of local+iSCSI
> vdevs), the answer was along the lines that round-robin first considers the
> available devices. If the faster (ssd, local) device has no queue, it gets
> the load while the slower device still struggles with the task it has, so
> on average the faster device serves more io's - but not 100%. Queue depth
> tuning can also help here.
>
> Jim
>
> --
> Typos courtesy of K-9 Mail on my Samsung Android
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150715/5db1d3fe/attachment.html>

From danmcd at omniti.com  Wed Jul 15 15:53:37 2015
From: danmcd at omniti.com (Dan McDonald)
Date: Wed, 15 Jul 2015 11:53:37 -0400
Subject: [OmniOS-discuss] re-tune round-robin reading from a mirror
In-Reply-To: <CALeZrrQS9fGJm3OY+jbza9SuZ9KfiT=AHpnd-goLk6Rg8fWFtw@mail.gmail.com>
References: <20150715221015.49e0abaf@punda-mlia>
	<ACB6E6F4-4F24-409D-A432-C61BFA564C43@cos.ru>
	<CALeZrrQS9fGJm3OY+jbza9SuZ9KfiT=AHpnd-goLk6Rg8fWFtw@mail.gmail.com>
Message-ID: <58911DC5-6838-44CD-9344-02FD01125169@omniti.com>


> On Jul 15, 2015, at 11:45 AM, Schweiss, Chip <chip at innovates.com> wrote:
> 
> This is a very interesting idea.  It could allow for the creation of a scratch pool with great ROI.   I have a particular need for an extremely high read rate pool for data analysis that will leverage the fattest read optimized SSDs I can get for the dollar.  I was considering this to be raidz1, but if mirroring with disks would work this way, it could be even better bang for the buck. 
> 
> Did you ever do any actual testing with this type of setup?  I'd love to see some real world performance data.

There's a great (computer) science experiment in here waiting to be run.  Setting up a 1-SSD + N-spinning-rust mirror (given the capacity mismatch, one could theoretically use N-slices of N-spinning-rusts) and then measuring read vs. write, which disks used, etc. would be spectacularly useful.  It would also probably make a nice presentation at the next OpenZFS conference.

Speaking of that, this question should be posed on the OpenZFS mailing list.  It's possible someone in the wider ZFS community has considered this or even run tests already.

Dan


From matthew.lagoe at subrigo.net  Wed Jul 15 20:22:13 2015
From: matthew.lagoe at subrigo.net (Matthew Lagoe)
Date: Wed, 15 Jul 2015 13:22:13 -0700
Subject: [OmniOS-discuss] Zil Device
Message-ID: <007101d0bf3b$f0288f30$d079ad90$@subrigo.net>

Is the zeusram SSD still the big zil device out there or are there other
high performance reliable options that anyone knows of on the market now? I
can't go with like the DDRdrive as its pcie.

Thanks 



From doug at will.to  Wed Jul 15 20:42:01 2015
From: doug at will.to (Doug Hughes)
Date: Wed, 15 Jul 2015 16:42:01 -0400
Subject: [OmniOS-discuss] Zil Device
In-Reply-To: <007101d0bf3b$f0288f30$d079ad90$@subrigo.net>
References: <007101d0bf3b$f0288f30$d079ad90$@subrigo.net>
Message-ID: <b983444d-4ee1-429b-a68f-0886d1e7cdb3.maildroid@localhost>

We have been preferring commodity SSD like Intel 320 (older), intel 710, or currently, Samsung 850 pro. We also use it as boot drive and reserve an 8GB slide for ZIL so that massive synchronous NFS IOPS are manageable.

Sent from my android device.

-----Original Message-----
From: Matthew Lagoe <matthew.lagoe at subrigo.net>
To: omnios-discuss at lists.omniti.com
Sent: Wed, 15 Jul 2015 16:29
Subject: [OmniOS-discuss] Zil Device

Is the zeusram SSD still the big zil device out there or are there other
high performance reliable options that anyone knows of on the market now? I
can't go with like the DDRdrive as its pcie.

Thanks 


_______________________________________________
OmniOS-discuss mailing list
OmniOS-discuss at lists.omniti.com
http://lists.omniti.com/mailman/listinfo/omnios-discuss
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150715/9e24ac38/attachment-0001.html>

From brogyi at gmail.com  Thu Jul 16 16:36:53 2015
From: brogyi at gmail.com (=?windows-1252?Q?Brogy=E1nyi_J=F3zsef?=)
Date: Thu, 16 Jul 2015 18:36:53 +0200
Subject: [OmniOS-discuss] Zil Device
In-Reply-To: <b983444d-4ee1-429b-a68f-0886d1e7cdb3.maildroid@localhost>
References: <007101d0bf3b$f0288f30$d079ad90$@subrigo.net>
	<b983444d-4ee1-429b-a68f-0886d1e7cdb3.maildroid@localhost>
Message-ID: <55A7DDA5.20100@gmail.com>

Hi Doug

Can you write its life time? I don't trust any SSD but I've thinking for 
a while to use as a ZIL+L2ARC.
Could you share with us your experiences? I would be interested in 
server usage. Thanks.


2015.07.15. 22:42 keltez?ssel, Doug Hughes ?rta:
> We have been preferring commodity SSD like Intel 320 (older), intel 
> 710, or currently, Samsung 850 pro. We also use it as boot drive and 
> reserve an 8GB slide for ZIL so that massive synchronous NFS IOPS are 
> manageable.
>
> Sent from my android device.
>
> -----Original Message-----
> From: Matthew Lagoe <matthew.lagoe at subrigo.net>
> To: omnios-discuss at lists.omniti.com
> Sent: Wed, 15 Jul 2015 16:29
> Subject: [OmniOS-discuss] Zil Device
>
> Is the zeusram SSD still the big zil device out there or are there other
> high performance reliable options that anyone knows of on the market 
> now? I
> can't go with like the DDRdrive as its pcie.
>
> Thanks
>
>
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com <mailto:OmniOS-discuss at lists.omniti.com>
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>
>
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150716/78f24641/attachment.html>

From cks at cs.toronto.edu  Thu Jul 16 16:48:14 2015
From: cks at cs.toronto.edu (Chris Siebenmann)
Date: Thu, 16 Jul 2015 12:48:14 -0400
Subject: [OmniOS-discuss] Clues for tracking down why kernel memory
	isn't being released?
In-Reply-To: cks's message of Tue, 14 Jul 2015 14:38:42 -0400.
	<20150714183842.3D5AF7A0255@apps0.cs.toronto.edu>
Message-ID: <20150716164814.25F447A00B6@apps0.cs.toronto.edu>

I wrote:
>  We have one ZFS-based NFS fileserver that persistently runs at a very
> high level of non-ARC kernel memory usage that never seems to shrink.
> On a 128 GB machine, mdb's ::memstat reports 95% memory usage by just
> 'Kernel' while the ZFS ARC is only at about 21 GB (as reported by
> 'kstat -m') although c_max should allow it to grow much bigger.
>
>  According to ::kmastat, a *huge* amount of this memory appears to be
> vanishing into allocated but not used kmem_alloc_131072 slab buffers:
> 
> > ::kmastat
> cache                            buf       buf       buf memory      alloc alloc
> name                            size    in use     total in use    succeed  fail
> ------------------------------ ----- --------- --------- ------ ---------- -----
> [...]
> kmem_alloc_131072               128K         6    613033  74.8G  196862991     0

 It turns out that the explanation for this is relatively simple, as
is the work around. Put simply: the OmniOS kernel does not actually
free up these deallocated cache objects until the system is put under
relatively strong memory pressure. Crucially, *the ZFS ARC does not
create this memory pressure*; I think that you pretty much need a user
level program allocating enough memory in order to trigger it, and I
think the memory growth needs to happen relatively rapidly fast so that
the kernel doesn't reclaim enough memory through lesser means (such as
shrinking the ZFS ARC).

(Specifically, you need to force kmem_reap() to be called. The primary
path for this is if 'freemem' drops under 'lotsfree', which is only a few
hundred MB on many systems. See usr/src/uts/common/os/vm_pageout.c in
the OmniOS source repo.)

 Since our fileservers are purely NFS fileservers and have a basically
static level of user memory usage, they rarely or never rapidly use up
enough memory to trigger this 'allocated but unused' reclaim[*].

 The good news is that it's easy enough these days to eat memory at the
user level (you can do it with modern 64-bit scripting languages like
Python, even at an interactive prompt). The bad news is that when we did
this on the server in question we provoked a significant system stall at
both the NFS server level and even the level of ssh logins and shells;
this is clearly not something that we'd want to automate.

 It's my personal opinion that there should be something in the kernel
that automatically reaps drastically outsized kmem caches after a
while. It's absurd that we've run for weeks with more than 70 GB of RAM
sitting unused and an undersized ZFS ARC because of this.

	- cks
[*: interested parties can see how often cache reaping has been triggered
    with the following 'mdb -k' command:
	::walk kmem_cache | ::printf "%4d %s\n" kmem_cache_t cache_reap cache_name

    Even on this heavily used fileserver, up for 45 days, the reap count
    was *8*.  Many of our other fileservers, with less usage, have reap
    counts of 0.
]

From doug at will.to  Thu Jul 16 16:55:30 2015
From: doug at will.to (Doug Hughes)
Date: Thu, 16 Jul 2015 12:55:30 -0400
Subject: [OmniOS-discuss] Zil Device
In-Reply-To: <55A7DDA5.20100@gmail.com>
References: <007101d0bf3b$f0288f30$d079ad90$@subrigo.net>
	<b983444d-4ee1-429b-a68f-0886d1e7cdb3.maildroid@localhost>
	<55A7DDA5.20100@gmail.com>
Message-ID: <CAOpmc6wmi3QyQVj76kAbkKLpTTabckjfxS5_eLweSh0byHdq6g@mail.gmail.com>

8GB zil on very active server and 100+GB ssd lasts many years. We have yet,
after years of use of various SSDs, to have one fail from wear usage, and
that's with fairly active NFS use.
They usually fail for other reasons.
We started with with Intel X series, which are only 32GB in size, and some
of them are still active, though less active use now. With Samsung 850 pro,
you practically don't have to worry about it, and the price is really good.


On Thu, Jul 16, 2015 at 12:36 PM, Brogy?nyi J?zsef <brogyi at gmail.com> wrote:

>  Hi Doug
>
> Can you write its life time? I don't trust any SSD but I've thinking for a
> while to use as a ZIL+L2ARC.
> Could you share with us your experiences? I would be interested in server
> usage. Thanks.
>
>
>
> 2015.07.15. 22:42 keltez?ssel, Doug Hughes ?rta:
>
> We have been preferring commodity SSD like Intel 320 (older), intel 710,
> or currently, Samsung 850 pro. We also use it as boot drive and reserve an
> 8GB slide for ZIL so that massive synchronous NFS IOPS are manageable.
>
> Sent from my android device.
>
> -----Original Message-----
> From: Matthew Lagoe <matthew.lagoe at subrigo.net>
> <matthew.lagoe at subrigo.net>
> To: omnios-discuss at lists.omniti.com
> Sent: Wed, 15 Jul 2015 16:29
> Subject: [OmniOS-discuss] Zil Device
>
>  Is the zeusram SSD still the big zil device out there or are there other
> high performance reliable options that anyone knows of on the market now?
> I
> can't go with like the DDRdrive as its pcie.
>
> Thanks
>
>
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>
>
> _______________________________________________
> OmniOS-discuss mailing listOmniOS-discuss at lists.omniti.comhttp://lists.omniti.com/mailman/listinfo/omnios-discuss
>
>
>
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150716/66303ecc/attachment.html>

From danmcd at omniti.com  Thu Jul 16 17:05:50 2015
From: danmcd at omniti.com (Dan McDonald)
Date: Thu, 16 Jul 2015 13:05:50 -0400
Subject: [OmniOS-discuss] Clues for tracking down why kernel memory
	isn't being released?
In-Reply-To: <20150716164814.25F447A00B6@apps0.cs.toronto.edu>
References: <20150716164814.25F447A00B6@apps0.cs.toronto.edu>
Message-ID: <23806661-0CF6-46BD-B454-7C14C6BE3DC2@omniti.com>


> On Jul 16, 2015, at 12:48 PM, Chris Siebenmann <cks at cs.toronto.edu> wrote:
> 
> It turns out that the explanation for this is relatively simple, as
> is the work around. Put simply: the OmniOS kernel does not actually
> free up these deallocated cache objects until the system is put under
> relatively strong memory pressure. Crucially, *the ZFS ARC does not
> create this memory pressure*; I think that you pretty much need a user
> level program allocating enough memory in order to trigger it, and I
> think the memory growth needs to happen relatively rapidly fast so that
> the kernel doesn't reclaim enough memory through lesser means (such as
> shrinking the ZFS ARC).
> 
> (Specifically, you need to force kmem_reap() to be called. The primary
> path for this is if 'freemem' drops under 'lotsfree', which is only a few
> hundred MB on many systems. See usr/src/uts/common/os/vm_pageout.c in
> the OmniOS source repo.)

You should bring this up on the illumos developer's list.  This isn't exclusive to OmniOS, and I believe at least on non-OmniOS illumos developer is working on general VM improvements as a side-gig at his day job.

Dan


From matthew.lagoe at subrigo.net  Thu Jul 16 17:22:04 2015
From: matthew.lagoe at subrigo.net (Matthew Lagoe)
Date: Thu, 16 Jul 2015 10:22:04 -0700
Subject: [OmniOS-discuss] Zil Device
In-Reply-To: <55A7DDA5.20100@gmail.com>
References: <007101d0bf3b$f0288f30$d079ad90$@subrigo.net>	<b983444d-4ee1-429b-a68f-0886d1e7cdb3.maildroid@localhost>
	<55A7DDA5.20100@gmail.com>
Message-ID: <00be01d0bfeb$f206dc20$d6149460$@subrigo.net>

The big concern I have is that we previously had a bunch of SSD?s that when
they failed they hung and took down everything


 

From: OmniOS-discuss [mailto:omnios-discuss-bounces at lists.omniti.com] On
Behalf Of Brogy?nyi J?zsef
Sent: Thursday, July 16, 2015 09:37 AM
To: omnios-discuss at lists.omniti.com
Subject: Re: [OmniOS-discuss] Zil Device

 

Hi Doug

Can you write its life time? I don't trust any SSD but I've thinking for a
while to use as a ZIL+L2ARC.
Could you share with us your experiences? I would be interested in server
usage. Thanks.


2015.07.15. 22:42 keltez?ssel, Doug Hughes ?rta:

We have been preferring commodity SSD like Intel 320 (older), intel 710, or
currently, Samsung 850 pro. We also use it as boot drive and reserve an 8GB
slide for ZIL so that massive synchronous NFS IOPS are manageable.

Sent from my android device.

-----Original Message-----
From: Matthew Lagoe  <mailto:matthew.lagoe at subrigo.net>
<matthew.lagoe at subrigo.net>
To: omnios-discuss at lists.omniti.com
Sent: Wed, 15 Jul 2015 16:29
Subject: [OmniOS-discuss] Zil Device

Is the zeusram SSD still the big zil device out there or are there other 
high performance reliable options that anyone knows of on the market now? I 
can't go with like the DDRdrive as its pcie. 

Thanks 


_______________________________________________ 
OmniOS-discuss mailing list 
OmniOS-discuss at lists.omniti.com 
http://lists.omniti.com/mailman/listinfo/omnios-discuss 





_______________________________________________
OmniOS-discuss mailing list
OmniOS-discuss at lists.omniti.com
http://lists.omniti.com/mailman/listinfo/omnios-discuss

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150716/c9b1fc28/attachment-0001.html>

From chip at innovates.com  Thu Jul 16 18:30:42 2015
From: chip at innovates.com (Schweiss, Chip)
Date: Thu, 16 Jul 2015 13:30:42 -0500
Subject: [OmniOS-discuss] Zil Device
In-Reply-To: <CAOpmc6wmi3QyQVj76kAbkKLpTTabckjfxS5_eLweSh0byHdq6g@mail.gmail.com>
References: <007101d0bf3b$f0288f30$d079ad90$@subrigo.net>
	<b983444d-4ee1-429b-a68f-0886d1e7cdb3.maildroid@localhost>
	<55A7DDA5.20100@gmail.com>
	<CAOpmc6wmi3QyQVj76kAbkKLpTTabckjfxS5_eLweSh0byHdq6g@mail.gmail.com>
Message-ID: <CALeZrrQ0mHMNjyH3qZ88UtOXWoyUXuda=hN=qMd5_p4oCdjCmA@mail.gmail.com>

The 850 Pro should never be used as a log device.  It does not have power
fail protection of its ram cache.   You might as well set sync=disabled and
skip using a log device entirely because the 850 Pro is not protecting your
last transactions in case of power failure.

Only SSDs with power failure protection should be considered for log
devices.

That being said, unless your running application that need transaction
consistency such as databases, don't bother with using a log device and set
sync=disabled.

-Chip

On Thu, Jul 16, 2015 at 11:55 AM, Doug Hughes <doug at will.to> wrote:

> 8GB zil on very active server and 100+GB ssd lasts many years. We have
> yet, after years of use of various SSDs, to have one fail from wear usage,
> and that's with fairly active NFS use.
> They usually fail for other reasons.
> We started with with Intel X series, which are only 32GB in size, and some
> of them are still active, though less active use now. With Samsung 850 pro,
> you practically don't have to worry about it, and the price is really good.
>
>
> On Thu, Jul 16, 2015 at 12:36 PM, Brogy?nyi J?zsef <brogyi at gmail.com>
> wrote:
>
>>  Hi Doug
>>
>> Can you write its life time? I don't trust any SSD but I've thinking for
>> a while to use as a ZIL+L2ARC.
>> Could you share with us your experiences? I would be interested in server
>> usage. Thanks.
>>
>>
>>
>> 2015.07.15. 22:42 keltez?ssel, Doug Hughes ?rta:
>>
>> We have been preferring commodity SSD like Intel 320 (older), intel 710,
>> or currently, Samsung 850 pro. We also use it as boot drive and reserve an
>> 8GB slide for ZIL so that massive synchronous NFS IOPS are manageable.
>>
>> Sent from my android device.
>>
>> -----Original Message-----
>> From: Matthew Lagoe <matthew.lagoe at subrigo.net>
>> <matthew.lagoe at subrigo.net>
>> To: omnios-discuss at lists.omniti.com
>> Sent: Wed, 15 Jul 2015 16:29
>> Subject: [OmniOS-discuss] Zil Device
>>
>>  Is the zeusram SSD still the big zil device out there or are there other
>> high performance reliable options that anyone knows of on the market now?
>> I
>> can't go with like the DDRdrive as its pcie.
>>
>> Thanks
>>
>>
>> _______________________________________________
>> OmniOS-discuss mailing list
>> OmniOS-discuss at lists.omniti.com
>> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>>
>>
>> _______________________________________________
>> OmniOS-discuss mailing listOmniOS-discuss at lists.omniti.comhttp://lists.omniti.com/mailman/listinfo/omnios-discuss
>>
>>
>>
>> _______________________________________________
>> OmniOS-discuss mailing list
>> OmniOS-discuss at lists.omniti.com
>> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>>
>>
>
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150716/2d96c190/attachment.html>

From richard.elling at richardelling.com  Thu Jul 16 18:34:37 2015
From: richard.elling at richardelling.com (Richard Elling)
Date: Thu, 16 Jul 2015 11:34:37 -0700
Subject: [OmniOS-discuss] Clues for tracking down why kernel memory
	isn't being released?
In-Reply-To: <20150716164814.25F447A00B6@apps0.cs.toronto.edu>
References: <20150716164814.25F447A00B6@apps0.cs.toronto.edu>
Message-ID: <90379C36-7605-40DD-9975-E7B5961983BD@richardelling.com>


> On Jul 16, 2015, at 9:48 AM, Chris Siebenmann <cks at cs.toronto.edu> wrote:
> 
> I wrote:
>> We have one ZFS-based NFS fileserver that persistently runs at a very
>> high level of non-ARC kernel memory usage that never seems to shrink.
>> On a 128 GB machine, mdb's ::memstat reports 95% memory usage by just
>> 'Kernel' while the ZFS ARC is only at about 21 GB (as reported by
>> 'kstat -m') although c_max should allow it to grow much bigger.
>> 
>> According to ::kmastat, a *huge* amount of this memory appears to be
>> vanishing into allocated but not used kmem_alloc_131072 slab buffers:
>> 
>>> ::kmastat
>> cache                            buf       buf       buf memory      alloc alloc
>> name                            size    in use     total in use    succeed  fail
>> ------------------------------ ----- --------- --------- ------ ---------- -----
>> [...]
>> kmem_alloc_131072               128K         6    613033  74.8G  196862991     0
> 
> It turns out that the explanation for this is relatively simple, as
> is the work around. Put simply: the OmniOS kernel does not actually
> free up these deallocated cache objects until the system is put under
> relatively strong memory pressure. Crucially, *the ZFS ARC does not
> create this memory pressure*; I think that you pretty much need a user
> level program allocating enough memory in order to trigger it, and I
> think the memory growth needs to happen relatively rapidly fast so that
> the kernel doesn't reclaim enough memory through lesser means (such as
> shrinking the ZFS ARC).

I don't think we will get much traction for ZFS pushing applications out of RAM.
There is a nuance here, that can be difficult to resolve.

> 
> (Specifically, you need to force kmem_reap() to be called. The primary
> path for this is if 'freemem' drops under 'lotsfree', which is only a few
> hundred MB on many systems. See usr/src/uts/common/os/vm_pageout.c in
> the OmniOS source repo.)
> 
> Since our fileservers are purely NFS fileservers and have a basically
> static level of user memory usage, they rarely or never rapidly use up
> enough memory to trigger this 'allocated but unused' reclaim[*].
> 
> The good news is that it's easy enough these days to eat memory at the
> user level (you can do it with modern 64-bit scripting languages like
> Python, even at an interactive prompt). The bad news is that when we did
> this on the server in question we provoked a significant system stall at
> both the NFS server level and even the level of ssh logins and shells;
> this is clearly not something that we'd want to automate.
> 
> It's my personal opinion that there should be something in the kernel
> that automatically reaps drastically outsized kmem caches after a
> while. It's absurd that we've run for weeks with more than 70 GB of RAM
> sitting unused and an undersized ZFS ARC because of this.

kmem reaps can be very painful

> 
> 	- cks
> [*: interested parties can see how often cache reaping has been triggered
>    with the following 'mdb -k' command:
> 	::walk kmem_cache | ::printf "%4d %s\n" kmem_cache_t cache_reap cache_name

ugh. How about:
kstat -p :::reap

 -- richard

> 
>    Even on this heavily used fileserver, up for 45 days, the reap count
>    was *8*.  Many of our other fileservers, with less usage, have reap
>    counts of 0.
> ]
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150716/14647018/attachment.html>

From richard.elling at richardelling.com  Thu Jul 16 18:38:07 2015
From: richard.elling at richardelling.com (Richard Elling)
Date: Thu, 16 Jul 2015 11:38:07 -0700
Subject: [OmniOS-discuss] Zil Device
In-Reply-To: <CALeZrrQ0mHMNjyH3qZ88UtOXWoyUXuda=hN=qMd5_p4oCdjCmA@mail.gmail.com>
References: <007101d0bf3b$f0288f30$d079ad90$@subrigo.net>
	<b983444d-4ee1-429b-a68f-0886d1e7cdb3.maildroid@localhost>
	<55A7DDA5.20100@gmail.com>
	<CAOpmc6wmi3QyQVj76kAbkKLpTTabckjfxS5_eLweSh0byHdq6g@mail.gmail.com>
	<CALeZrrQ0mHMNjyH3qZ88UtOXWoyUXuda=hN=qMd5_p4oCdjCmA@mail.gmail.com>
Message-ID: <5F6C56BE-C976-470A-8AB3-4B2772D1B2C0@richardelling.com>


> On Jul 16, 2015, at 11:30 AM, Schweiss, Chip <chip at innovates.com> wrote:
> 
> The 850 Pro should never be used as a log device.  It does not have power fail protection of its ram cache.   You might as well set sync=disabled and skip using a log device entirely because the 850 Pro is not protecting your last transactions in case of power failure.

Chip, are you asserting that the 850 Pro does not honor the cache flush command?
In the bad old days, there were SSDs that were broken for cache flushes, but some of
the most important (to Samsung) OSes, like Windows, rely on cache flushes to work.
ZFS does as well.
 -- richard

> 
> Only SSDs with power failure protection should be considered for log devices.   
> 
> That being said, unless your running application that need transaction consistency such as databases, don't bother with using a log device and set sync=disabled.   
> 
> -Chip
> 
> On Thu, Jul 16, 2015 at 11:55 AM, Doug Hughes <doug at will.to <mailto:doug at will.to>> wrote:
> 8GB zil on very active server and 100+GB ssd lasts many years. We have yet, after years of use of various SSDs, to have one fail from wear usage, and that's with fairly active NFS use.
> They usually fail for other reasons.
> We started with with Intel X series, which are only 32GB in size, and some of them are still active, though less active use now. With Samsung 850 pro, you practically don't have to worry about it, and the price is really good.
> 
> 
> On Thu, Jul 16, 2015 at 12:36 PM, Brogy?nyi J?zsef <brogyi at gmail.com <mailto:brogyi at gmail.com>> wrote:
> Hi Doug
> 
> Can you write its life time? I don't trust any SSD but I've thinking for a while to use as a ZIL+L2ARC.
> Could you share with us your experiences? I would be interested in server usage. Thanks.
> 
> 
> 
> 2015.07.15. 22:42 keltez?ssel, Doug Hughes ?rta:
>> We have been preferring commodity SSD like Intel 320 (older), intel 710, or currently, Samsung 850 pro. We also use it as boot drive and reserve an 8GB slide for ZIL so that massive synchronous NFS IOPS are manageable.
>> 
>> Sent from my android device.
>> 
>> -----Original Message-----
>> From: Matthew Lagoe <matthew.lagoe at subrigo.net> <mailto:matthew.lagoe at subrigo.net>
>> To: omnios-discuss at lists.omniti.com <mailto:omnios-discuss at lists.omniti.com>
>> Sent: Wed, 15 Jul 2015 16:29
>> Subject: [OmniOS-discuss] Zil Device
>> 
>> Is the zeusram SSD still the big zil device out there or are there other 
>> high performance reliable options that anyone knows of on the market now? I 
>> can't go with like the DDRdrive as its pcie. 
>> 
>> Thanks 
>> 
>> 
>> _______________________________________________ 
>> OmniOS-discuss mailing list 
>> OmniOS-discuss at lists.omniti.com <mailto:OmniOS-discuss at lists.omniti.com> 
>> http://lists.omniti.com/mailman/listinfo/omnios-discuss <http://lists.omniti.com/mailman/listinfo/omnios-discuss> 
>> 
>> 
>> _______________________________________________
>> OmniOS-discuss mailing list
>> OmniOS-discuss at lists.omniti.com <mailto:OmniOS-discuss at lists.omniti.com>
>> http://lists.omniti.com/mailman/listinfo/omnios-discuss <http://lists.omniti.com/mailman/listinfo/omnios-discuss>
> 
> 
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com <mailto:OmniOS-discuss at lists.omniti.com>
> http://lists.omniti.com/mailman/listinfo/omnios-discuss <http://lists.omniti.com/mailman/listinfo/omnios-discuss>
> 
> 
> 
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com <mailto:OmniOS-discuss at lists.omniti.com>
> http://lists.omniti.com/mailman/listinfo/omnios-discuss <http://lists.omniti.com/mailman/listinfo/omnios-discuss>
> 
> 
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150716/71dc9b9f/attachment-0001.html>

From cks at cs.toronto.edu  Thu Jul 16 18:40:57 2015
From: cks at cs.toronto.edu (Chris Siebenmann)
Date: Thu, 16 Jul 2015 14:40:57 -0400
Subject: [OmniOS-discuss] Clues for tracking down why kernel memory
	isn't being released?
In-Reply-To: richard.elling's message of Thu, 16 Jul 2015 11:34:37 -0700.
	<90379C36-7605-40DD-9975-E7B5961983BD@richardelling.com>
Message-ID: <20150716184057.B6D677A00B6@apps0.cs.toronto.edu>

> > It turns out that the explanation for this is relatively simple, as
> > is the work around. Put simply: the OmniOS kernel does not actually
> > free up these deallocated cache objects until the system is put
> > under relatively strong memory pressure. Crucially, *the ZFS ARC
> > does not create this memory pressure*; I think that you pretty much
> > need a user level program allocating enough memory in order to
> > trigger it, and I think the memory growth needs to happen relatively
> > rapidly fast so that the kernel doesn't reclaim enough memory
> > through lesser means (such as shrinking the ZFS ARC).
>
> I don't think we will get much traction for ZFS pushing applications
> out of RAM.  There is a nuance here, that can be difficult to resolve.

 I don't think ZFS should push applications out of RAM, but that's not
the issue here. The RAM in question is in the kernel and is known to
be unused; it simply isn't being freed when ZFS would like more ARC
space.  The kmem code even maintains information on the 'working set' of
allocated objects for caches, it just doesn't do anything with it until
a kmem reap triggers.

(Which as Richard mentions can be (very) painful.)

 The ideal situation would be that caches with significant numbers of
fully unused slab objects would have those unused slabs slowly released
back to the kernel's free memory pool. This might happen either just
generally or specifically in response to continued desire from the ARC
for more memory.

> > 	- cks
> > [*: interested parties can see how often cache reaping has been triggered
> >    with the following 'mdb -k' command:
> > 	::walk kmem_cache | ::printf "%4d %s\n" kmem_cache_t cache_reap cache_name
> 
> ugh. How about:
> kstat -p :::reap

 Thanks! That's much better.

(I should get better with kstat.)

	- cks

From illumos at cucumber.demon.co.uk  Thu Jul 16 18:53:18 2015
From: illumos at cucumber.demon.co.uk (Andrew Gabriel)
Date: Thu, 16 Jul 2015 19:53:18 +0100
Subject: [OmniOS-discuss] Zil Device
In-Reply-To: <5F6C56BE-C976-470A-8AB3-4B2772D1B2C0@richardelling.com>
References: <007101d0bf3b$f0288f30$d079ad90$@subrigo.net>	<b983444d-4ee1-429b-a68f-0886d1e7cdb3.maildroid@localhost>	<55A7DDA5.20100@gmail.com>	<CAOpmc6wmi3QyQVj76kAbkKLpTTabckjfxS5_eLweSh0byHdq6g@mail.gmail.com>	<CALeZrrQ0mHMNjyH3qZ88UtOXWoyUXuda=hN=qMd5_p4oCdjCmA@mail.gmail.com>
	<5F6C56BE-C976-470A-8AB3-4B2772D1B2C0@richardelling.com>
Message-ID: <55A7FD9E.4000009@cucumber.demon.co.uk>

On 16/07/2015 19:38, Richard Elling wrote:
>
>> On Jul 16, 2015, at 11:30 AM, Schweiss, Chip <chip at innovates.com 
>> <mailto:chip at innovates.com>> wrote:
>>
>> The 850 Pro should never be used as a log device.  It does not have 
>> power fail protection of its ram cache.   You might as well set 
>> sync=disabled and skip using a log device entirely because the 850 
>> Pro is not protecting your last transactions in case of power failure.
>
> Chip, are you asserting that the 850 Pro does not honor the cache 
> flush command?
> In the bad old days, there were SSDs that were broken for cache 
> flushes, but some of
> the most important (to Samsung) OSes, like Windows, rely on cache 
> flushes to work.
> ZFS does as well.
>  -- richard

Was just in the middle of writing something similar when your reply came in.
Just to add, ZIL devices don't have to have power fail emergency supply 
to work correctly.
What's important is that they don't lie about when they get data to 
non-volatile storage.
Those with power fail protection can in many cases reply when data hits 
their DRAM cache, whereas those without have to wait a bit longer until 
the data hits the flash chips. So the difference tends to impact latency 
of synchronous operations, but in both cases they are very much faster 
than waiting until the data hits the platters of spinning rust.

-- 
Andrew Gabriel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150716/f7d5af19/attachment.html>

From fabio at fabiorabelo.wiki.br  Thu Jul 16 22:03:46 2015
From: fabio at fabiorabelo.wiki.br (=?UTF-8?Q?F=C3=A1bio_Rabelo?=)
Date: Thu, 16 Jul 2015 19:03:46 -0300
Subject: [OmniOS-discuss] Xeon SOC with 10 GB net card
Message-ID: <CAEekY6445O+7Bj7zFzyQCUPFr2UDLCMfQ4g_b7xpDZAfyPZqAA@mail.gmail.com>

Hi to all

Someone knows if the 10 GB network ports in this litle baby works with
OmniOS ?

http://www.supermicro.com/products/motherboard/Xeon/D/X10SDV-8C-TLN4F.cfm

Yes, I am dreaming with a very compact pure SSD storage ;- ))

With one 8 port SAS in the PCIe slot, plus 4 Sata 3 ports in the
motherboard, with a DOM module to boot the system and the remainig Sata 3
to ZIL device ....


F?bio Rabelo
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150716/567e5b22/attachment.html>

From danmcd at omniti.com  Thu Jul 16 22:16:30 2015
From: danmcd at omniti.com (Dan McDonald)
Date: Thu, 16 Jul 2015 18:16:30 -0400
Subject: [OmniOS-discuss] Xeon SOC with 10 GB net card
In-Reply-To: <CAEekY6445O+7Bj7zFzyQCUPFr2UDLCMfQ4g_b7xpDZAfyPZqAA@mail.gmail.com>
References: <CAEekY6445O+7Bj7zFzyQCUPFr2UDLCMfQ4g_b7xpDZAfyPZqAA@mail.gmail.com>
Message-ID: <CA8C19B1-F0D9-4250-9D8E-591DD243C4FC@omniti.com>


> On Jul 16, 2015, at 6:03 PM, F?bio Rabelo <fabio at fabiorabelo.wiki.br> wrote:
> 
> Hi to all
> 
> Someone knows if the 10 GB network ports in this litle baby works with OmniOS ?
> 
> http://www.supermicro.com/products/motherboard/Xeon/D/X10SDV-8C-TLN4F.cfm

It's not clear to me which controller the 10Gig is based upon.  The slideware I see for Xeon D says, "Based on Intel's Robust 10G solution".  To me that screams, "82599 or X540".

Further review suggests we need to update our ixgbe driver with the latest bits from upstream and Intel.  You should ask this question on the illumos developer's list.

Dan


From danmcd at omniti.com  Thu Jul 16 22:22:37 2015
From: danmcd at omniti.com (Dan McDonald)
Date: Thu, 16 Jul 2015 18:22:37 -0400
Subject: [OmniOS-discuss] Xeon SOC with 10 GB net card
In-Reply-To: <CA8C19B1-F0D9-4250-9D8E-591DD243C4FC@omniti.com>
References: <CAEekY6445O+7Bj7zFzyQCUPFr2UDLCMfQ4g_b7xpDZAfyPZqAA@mail.gmail.com>
	<CA8C19B1-F0D9-4250-9D8E-591DD243C4FC@omniti.com>
Message-ID: <7F4E3A55-794E-44C5-91FA-F22E200F1034@omniti.com>


> On Jul 16, 2015, at 6:16 PM, Dan McDonald <danmcd at omniti.com> wrote:
> 
> 
> Further review suggests we need to update our ixgbe driver with the latest bits from upstream and Intel.  You should ask this question on the illumos developer's list.
> 

See illumos bug 6064:

	https://www.illumos.org/issues/6064

For tracking this issue.  Community contributions ALWAYS welcome.  :)

Dan


From daleg at omniti.com  Fri Jul 17 02:54:42 2015
From: daleg at omniti.com (Dale Ghent)
Date: Thu, 16 Jul 2015 22:54:42 -0400
Subject: [OmniOS-discuss] Xeon SOC with 10 GB net card
In-Reply-To: <CA8C19B1-F0D9-4250-9D8E-591DD243C4FC@omniti.com>
References: <CAEekY6445O+7Bj7zFzyQCUPFr2UDLCMfQ4g_b7xpDZAfyPZqAA@mail.gmail.com>
	<CA8C19B1-F0D9-4250-9D8E-591DD243C4FC@omniti.com>
Message-ID: <28B54A53-6887-4EDF-AC4C-2F8EF8700745@omniti.com>


> On Jul 16, 2015, at 6:16 PM, Dan McDonald <danmcd at omniti.com> wrote:
> 
> 
>> On Jul 16, 2015, at 6:03 PM, F?bio Rabelo <fabio at fabiorabelo.wiki.br> wrote:
>> 
>> Hi to all
>> 
>> Someone knows if the 10 GB network ports in this litle baby works with OmniOS ?
>> 
>> http://www.supermicro.com/products/motherboard/Xeon/D/X10SDV-8C-TLN4F.cfm
> 
> It's not clear to me which controller the 10Gig is based upon.  The slideware I see for Xeon D says, "Based on Intel's Robust 10G solution".  To me that screams, "82599 or X540".
> 
> Further review suggests we need to update our ixgbe driver with the latest bits from upstream and Intel.  You should ask this question on the illumos developer's list.

From section 9.2.2.2 of http://www.intel.com/content/dam/www/public/us/en/documents/datasheets/xeon-d-1500-datasheet-vol-4.pdf

the PCI PID of the 10Gb PHY is 0x15ad

According to the PCI ID database, that is a X552/X557-AT 10GBASE-T (https://pci-ids.ucw.cz/read/PC/8086/15ad)

Some cursory googling seems to indicate that the existing ixgbe driver might need some X55x-specific MAC init routines.

/dale
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 494 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150716/0253b421/attachment.bin>

From bfriesen at simple.dallas.tx.us  Fri Jul 17 13:49:43 2015
From: bfriesen at simple.dallas.tx.us (Bob Friesenhahn)
Date: Fri, 17 Jul 2015 08:49:43 -0500 (CDT)
Subject: [OmniOS-discuss] Xeon SOC with 10 GB net card
In-Reply-To: <CAEekY6445O+7Bj7zFzyQCUPFr2UDLCMfQ4g_b7xpDZAfyPZqAA@mail.gmail.com>
References: <CAEekY6445O+7Bj7zFzyQCUPFr2UDLCMfQ4g_b7xpDZAfyPZqAA@mail.gmail.com>
Message-ID: <alpine.GSO.2.01.1507170835100.4186@freddy.simplesystems.org>

On Thu, 16 Jul 2015, F?bio Rabelo wrote:

> Hi to all
> 
> Someone knows if the 10 GB network ports in this litle baby works with OmniOS ?
> 
> http://www.supermicro.com/products/motherboard/Xeon/D/X10SDV-8C-TLN4F.cfm
> 
> Yes, I am dreaming with a very compact pure SSD storage ;- ))
> 
> With one 8 port SAS in the PCIe slot, plus 4 Sata 3 ports in the motherboard, with a DOM module to boot the system and the
> remainig Sata 3 to ZIL device ....

I have one of these on order, which will have 6 hot-swap AHCI/SATA 
SSDs and 2 M.2 AHCI SSDs (Samsung SM951).  One of the M.2 AHCI SSDs is 
supported by an a PCIe adaptor card.  I have instructed my computer 
builder to install OmniOS on it.  It is not clear if this will mostly 
just work (other than 10Gbit interfaces) or be an adventure.

Look for my previous posting on the illumos-discuss mailing list which 
was inappropriately entitled "Does Illumos support Pentum D?".  There 
were some useful responses based on pre-production samples.

The M.2 interface is really interesting since it is based on 4 (or 2 
on some boards) channels of PCIe 3.0 and is able to support AHCI or 
NVME cards without the transfer overhead of SATA.  Illumos is not 
quite there yet for NVME, although a driver has already been going 
through the review process.  In addition to the possible use of M.2 as 
a boot drive, these cards should work really well as ZFS L2 ARC since 
the read performance is extremely high.

Bob
-- 
Bob Friesenhahn
bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/

From mtalbott at lji.org  Fri Jul 17 23:07:09 2015
From: mtalbott at lji.org (Michael Talbott)
Date: Fri, 17 Jul 2015 16:07:09 -0700
Subject: [OmniOS-discuss] Slow Drive Detection and boot-archive
Message-ID: <47D283D6-3021-4C82-AC10-BAA7279EB58E@lji.org>

I have multiple NAS servers I've moved to OmniOS and each of them have 90-180 4T disks. Everything has worked out pretty well for the most part. But I've come into an issue where when I reboot any of them, I'm getting boot-archive service timeouts happening. I found a workaround of increasing the timeout value which brings me to the following. As you can see below in a dmesg output, it's taking the kernel about 8 seconds to detect each of the drives. They're connected via a couple SAS2008 based LSI cards.

Is this normal?
Is there a way to speed that up?

I've fixed my frustrating boot-archive timeout problem by adjusting the timeout value from the default of 60 seconds (I guess that'll work ok on systems with less than 8 drives?) to 8 seconds * 90 drives + a little extra time = 280 seconds (for the 90 drive systems). Which means it takes between 12-24 minutes to boot those machines up.

# svccfg -s boot-archive setprop start/timeout_seconds=280

I figure I can't be the only one. A little googling also revealed: https://www.illumos.org/issues/4614 <https://www.illumos.org/issues/4614>

Jul 17 15:40:15 store2 genunix: [ID 583861 kern.info] sd29 at mpt_sas3: unit-address w50000c0f0401bd43,0: w50000c0f0401bd43,0
Jul 17 15:40:15 store2 genunix: [ID 936769 kern.info] sd29 is /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0401bd43,0
Jul 17 15:40:16 store2 genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0401bd43,0 (sd29) online
Jul 17 15:40:24 store2 genunix: [ID 583861 kern.info] sd30 at mpt_sas3: unit-address w50000c0f045679c3,0: w50000c0f045679c3,0
Jul 17 15:40:24 store2 genunix: [ID 936769 kern.info] sd30 is /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f045679c3,0
Jul 17 15:40:24 store2 genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f045679c3,0 (sd30) online
Jul 17 15:40:33 store2 genunix: [ID 583861 kern.info] sd31 at mpt_sas3: unit-address w50000c0f045712b3,0: w50000c0f045712b3,0
Jul 17 15:40:33 store2 genunix: [ID 936769 kern.info] sd31 is /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f045712b3,0
Jul 17 15:40:33 store2 genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f045712b3,0 (sd31) online
Jul 17 15:40:42 store2 genunix: [ID 583861 kern.info] sd32 at mpt_sas3: unit-address w50000c0f04571497,0: w50000c0f04571497,0
Jul 17 15:40:42 store2 genunix: [ID 936769 kern.info] sd32 is /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f04571497,0
Jul 17 15:40:42 store2 genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f04571497,0 (sd32) online
Jul 17 15:40:50 store2 genunix: [ID 583861 kern.info] sd33 at mpt_sas3: unit-address w50000c0f042ac8eb,0: w50000c0f042ac8eb,0
Jul 17 15:40:50 store2 genunix: [ID 936769 kern.info] sd33 is /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f042ac8eb,0
Jul 17 15:40:50 store2 genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f042ac8eb,0 (sd33) online
Jul 17 15:40:59 store2 genunix: [ID 583861 kern.info] sd34 at mpt_sas3: unit-address w50000c0f04571473,0: w50000c0f04571473,0
Jul 17 15:40:59 store2 genunix: [ID 936769 kern.info] sd34 is /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f04571473,0
Jul 17 15:40:59 store2 genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f04571473,0 (sd34) online
Jul 17 15:41:08 store2 genunix: [ID 583861 kern.info] sd35 at mpt_sas3: unit-address w50000c0f042c636f,0: w50000c0f042c636f,0
Jul 17 15:41:08 store2 genunix: [ID 936769 kern.info] sd35 is /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f042c636f,0
Jul 17 15:41:08 store2 genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f042c636f,0 (sd35) online
Jul 17 15:41:17 store2 genunix: [ID 583861 kern.info] sd36 at mpt_sas3: unit-address w50000c0f0401bf2f,0: w50000c0f0401bf2f,0
Jul 17 15:41:17 store2 genunix: [ID 936769 kern.info] sd36 is /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0401bf2f,0
Jul 17 15:41:17 store2 genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0401bf2f,0 (sd36) online
Jul 17 15:41:25 store2 genunix: [ID 583861 kern.info] sd38 at mpt_sas3: unit-address w50000c0f0401bc1f,0: w50000c0f0401bc1f,0
Jul 17 15:41:25 store2 genunix: [ID 936769 kern.info] sd38 is /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0401bc1f,0
Jul 17 15:41:26 store2 genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0401bc1f,0 (sd38) online


________________________
Michael Talbott
Systems Administrator
La Jolla Institute

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150717/8f4205b0/attachment.html>

From mtalbott at lji.org  Fri Jul 17 23:15:47 2015
From: mtalbott at lji.org (Michael Talbott)
Date: Fri, 17 Jul 2015 16:15:47 -0700
Subject: [OmniOS-discuss] Slow Drive Detection and boot-archive
In-Reply-To: <47D283D6-3021-4C82-AC10-BAA7279EB58E@lji.org>
References: <47D283D6-3021-4C82-AC10-BAA7279EB58E@lji.org>
Message-ID: <95908196-2C9D-4584-8E82-F715DD9A3A14@lji.org>

Just realized my typo. I'm using this on my 90 and 180 drive systems:

# svccfg -s boot-archive setprop start/timeout_seconds=720
# svccfg -s boot-archive setprop start/timeout_seconds=1440

Seems like 8 seconds to detect each drive is pretty excessive.

Any ideas on how to speed that up?


________________________
Michael Talbott
Systems Administrator
La Jolla Institute

> On Jul 17, 2015, at 4:07 PM, Michael Talbott <mtalbott at lji.org> wrote:
> 
> I have multiple NAS servers I've moved to OmniOS and each of them have 90-180 4T disks. Everything has worked out pretty well for the most part. But I've come into an issue where when I reboot any of them, I'm getting boot-archive service timeouts happening. I found a workaround of increasing the timeout value which brings me to the following. As you can see below in a dmesg output, it's taking the kernel about 8 seconds to detect each of the drives. They're connected via a couple SAS2008 based LSI cards.
> 
> Is this normal?
> Is there a way to speed that up?
> 
> I've fixed my frustrating boot-archive timeout problem by adjusting the timeout value from the default of 60 seconds (I guess that'll work ok on systems with less than 8 drives?) to 8 seconds * 90 drives + a little extra time = 280 seconds (for the 90 drive systems). Which means it takes between 12-24 minutes to boot those machines up.
> 
> # svccfg -s boot-archive setprop start/timeout_seconds=280
> 
> I figure I can't be the only one. A little googling also revealed: https://www.illumos.org/issues/4614 <https://www.illumos.org/issues/4614>
> 
> Jul 17 15:40:15 store2 genunix: [ID 583861 kern.info] sd29 at mpt_sas3: unit-address w50000c0f0401bd43,0: w50000c0f0401bd43,0
> Jul 17 15:40:15 store2 genunix: [ID 936769 kern.info] sd29 is /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0401bd43,0
> Jul 17 15:40:16 store2 genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0401bd43,0 (sd29) online
> Jul 17 15:40:24 store2 genunix: [ID 583861 kern.info] sd30 at mpt_sas3: unit-address w50000c0f045679c3,0: w50000c0f045679c3,0
> Jul 17 15:40:24 store2 genunix: [ID 936769 kern.info] sd30 is /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f045679c3,0
> Jul 17 15:40:24 store2 genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f045679c3,0 (sd30) online
> Jul 17 15:40:33 store2 genunix: [ID 583861 kern.info] sd31 at mpt_sas3: unit-address w50000c0f045712b3,0: w50000c0f045712b3,0
> Jul 17 15:40:33 store2 genunix: [ID 936769 kern.info] sd31 is /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f045712b3,0
> Jul 17 15:40:33 store2 genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f045712b3,0 (sd31) online
> Jul 17 15:40:42 store2 genunix: [ID 583861 kern.info] sd32 at mpt_sas3: unit-address w50000c0f04571497,0: w50000c0f04571497,0
> Jul 17 15:40:42 store2 genunix: [ID 936769 kern.info] sd32 is /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f04571497,0
> Jul 17 15:40:42 store2 genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f04571497,0 (sd32) online
> Jul 17 15:40:50 store2 genunix: [ID 583861 kern.info] sd33 at mpt_sas3: unit-address w50000c0f042ac8eb,0: w50000c0f042ac8eb,0
> Jul 17 15:40:50 store2 genunix: [ID 936769 kern.info] sd33 is /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f042ac8eb,0
> Jul 17 15:40:50 store2 genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f042ac8eb,0 (sd33) online
> Jul 17 15:40:59 store2 genunix: [ID 583861 kern.info] sd34 at mpt_sas3: unit-address w50000c0f04571473,0: w50000c0f04571473,0
> Jul 17 15:40:59 store2 genunix: [ID 936769 kern.info] sd34 is /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f04571473,0
> Jul 17 15:40:59 store2 genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f04571473,0 (sd34) online
> Jul 17 15:41:08 store2 genunix: [ID 583861 kern.info] sd35 at mpt_sas3: unit-address w50000c0f042c636f,0: w50000c0f042c636f,0
> Jul 17 15:41:08 store2 genunix: [ID 936769 kern.info] sd35 is /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f042c636f,0
> Jul 17 15:41:08 store2 genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f042c636f,0 (sd35) online
> Jul 17 15:41:17 store2 genunix: [ID 583861 kern.info] sd36 at mpt_sas3: unit-address w50000c0f0401bf2f,0: w50000c0f0401bf2f,0
> Jul 17 15:41:17 store2 genunix: [ID 936769 kern.info] sd36 is /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0401bf2f,0
> Jul 17 15:41:17 store2 genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0401bf2f,0 (sd36) online
> Jul 17 15:41:25 store2 genunix: [ID 583861 kern.info] sd38 at mpt_sas3: unit-address w50000c0f0401bc1f,0: w50000c0f0401bc1f,0
> Jul 17 15:41:25 store2 genunix: [ID 936769 kern.info] sd38 is /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0401bc1f,0
> Jul 17 15:41:26 store2 genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0401bc1f,0 (sd38) online
> 
> 
> ________________________
> Michael Talbott
> Systems Administrator
> La Jolla Institute
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150717/5d22d096/attachment-0001.html>

From venture37 at gmail.com  Sat Jul 18 12:02:28 2015
From: venture37 at gmail.com (Sevan / Venture37)
Date: Sat, 18 Jul 2015 13:02:28 +0100
Subject: [OmniOS-discuss] pkgsrc-current OmniOS 170cea2/i386 2015-07-09
	21:35
In-Reply-To: <A3703B3C-8953-4CB1-BC11-7C898AB6F182@omniti.com>
References: <CA+U3Mf6yA5VHgr5WdS9dhiEHs1epCmr22V_0KZ-S25soeWwnHw@mail.gmail.com>
	<811B3E16-0565-4F36-94F3-AACC0CBB2D59@omniti.com>
	<CA+U3Mf5=vUi+2sDhQ2qXNCLMjUtzMsi=CDcbJHA1gtTbCAEOjg@mail.gmail.com>
	<A3703B3C-8953-4CB1-BC11-7C898AB6F182@omniti.com>
Message-ID: <CA+U3Mf5nyOH6sT0Utn0whQL3iNmCaGVS-=CGuU3VFxVKOX9JJw@mail.gmail.com>

Hi,

On 14 July 2015 at 02:14, Dan McDonald <danmcd at omniti.com> wrote:
> /etc/release.  (Yes, this even works in zones.)

I sent another diff for review using
/usr/bin/awk '{ if (!seen) { print $$3; seen=1 } }' /etc/release
to pull the release info out.
Unfortunately I didn't restart my build which was in progress so the
latest report still contains the previous convention.

>>> Is there anything we can fix to help these move along?
>>
>> I'd like to propose a fix for the gettext issue but I've not been able
>> to get very far with the buildctl tool in omnios-build. It appears to
>> be incapable of generating a repo or utilising a pre-prepared repo for
>> me (I used  a different path for the location of my repo if that makes
>> any difference).
>
> To rebuild gettext from omnios-build:
>
>         Set the PKGSRVR environment variable to be where your repo lives.  e.g. file:///data/builder/builder.repo/
>
>         cd $PATH_TO/omnios-build/build
>
>         ./buildctl build gettext
>
> Say no to pkglint and say yes to publication, verifying that PKGSRVR was set in your environment.

Haven't had a chance to play with this yet, but with just the
temporary work around using crle that was mentioned previously, the
package count is considerably higher

http://mail-index.netbsd.org/pkgsrc-bulk/2015/07/18/msg011844.html

Total number of packages:      16536
  Successfully built:          13398

:)

It was 8961 previously.

> Weird... as I don't see any changes in the libc callers:
>
> bloody(~/ws)[0]% diff -r illumos-{gate,joyent}/usr/src/lib/libc/port/i18n/
> bloody(~/ws)[0]% diff -r illumos-{omnios,joyent}/usr/src/lib/libc/port/i18n/
> bloody(~/ws)[0]%

Would things have been out of sync in the early days of the docker stuff?


> I'm game for any changes.  Your best bet is to modify $PATH_TO/omnios-build/build/gettext/build.sh.

Cool, I'll follow up again soon.


Sevan

From danmcd at omniti.com  Sun Jul 19 19:02:05 2015
From: danmcd at omniti.com (Dan McDonald)
Date: Sun, 19 Jul 2015 15:02:05 -0400
Subject: [OmniOS-discuss] pkgsrc-current OmniOS 170cea2/i386 2015-07-09
	21:35
In-Reply-To: <CA+U3Mf5nyOH6sT0Utn0whQL3iNmCaGVS-=CGuU3VFxVKOX9JJw@mail.gmail.com>
References: <CA+U3Mf6yA5VHgr5WdS9dhiEHs1epCmr22V_0KZ-S25soeWwnHw@mail.gmail.com>
	<811B3E16-0565-4F36-94F3-AACC0CBB2D59@omniti.com>
	<CA+U3Mf5=vUi+2sDhQ2qXNCLMjUtzMsi=CDcbJHA1gtTbCAEOjg@mail.gmail.com>
	<A3703B3C-8953-4CB1-BC11-7C898AB6F182@omniti.com>
	<CA+U3Mf5nyOH6sT0Utn0whQL3iNmCaGVS-=CGuU3VFxVKOX9JJw@mail.gmail.com>
Message-ID: <650A82F9-E07E-44CF-AE9C-4EAA4FFE87F5@omniti.com>


> On Jul 18, 2015, at 8:02 AM, Sevan / Venture37 <venture37 at gmail.com> wrote:
> 
>> 
>> Weird... as I don't see any changes in the libc callers:
>> 
>> bloody(~/ws)[0]% diff -r illumos-{gate,joyent}/usr/src/lib/libc/port/i18n/
>> bloody(~/ws)[0]% diff -r illumos-{omnios,joyent}/usr/src/lib/libc/port/i18n/
>> bloody(~/ws)[0]%
> 
> Would things have been out of sync in the early days of the docker stuff?

Don't think so.

I'd like to know (again if you'd already mentioned it) the precise nature of your workaround that allow more pkgsrc to be built.  You can, of course, just express it as a pull-request to omnios-build too, if you wish.

Dan


From omnios at citrus-it.net  Mon Jul 20 21:50:49 2015
From: omnios at citrus-it.net (Andy Fiddaman)
Date: Mon, 20 Jul 2015 21:50:49 +0000 (UTC)
Subject: [OmniOS-discuss] [developer] HEADS UP -- illumos-gate can now
 be built on OmniOS r151014 or later
In-Reply-To: <alpine.GSO.2.00.1507101329510.19413@erncre.pvgehf-vg.arg>
References: <D8F62323-340B-4334-8BBC-9DE7198ADAC9@omniti.com>
	<alpine.GSO.2.00.1507091040020.2334@erncre.pvgehf-vg.arg>
	<EB7EAE73-93BD-446B-9B9F-3A0DE2C26992@omniti.com>
	<alpine.GSO.2.00.1507101329510.19413@erncre.pvgehf-vg.arg>
Message-ID: <alpine.GSO.2.00.1507202148570.7194@erncre.pvgehf-vg.arg>


I haven't managed to get to the bottom of this.

Could someone please send me a nightly.log from a build of the Illumos
gate on an OmniOS system so I can look for differences?

Any comments on the previous message below appreciated too.

Thanks,

Andy

On Fri, 10 Jul 2015, Andy Fiddaman wrote:

;
; On Thu, 9 Jul 2015, Dan McDonald wrote:
;
; ; Are you missing these files?  That's the explanation off the top of my head.
; ;
; ; Otherwise I'd have to look at the whole nightly.log, and I won't be able to do that at all today, possibly this week.
;
; I am not getting much further with finding the root cause of this. It does
; seem to be a fault in the build process but in that case I don't understand
; why I don't see it when building on OI. I might have to try and recover my
; OI machine to compare.
;
; Here's what I've found so far:
;
; Once nightly is finished, the OpenSSL licence file is present in
; proto/root_i386/licenses/usr/src/common/openssl/LICENSE
;
; and the nightly log says that it failed to build it once then didn't need
; to after that:
;
; build-env# (154) grep openssl/LICEN ~/log/log.2015-07-10.12:47/nightly.log
; dmake: Warning: Don't know how to make target `/root/illumos/illumos-gate/proto/root_i386/licenses/usr/src/common/openssl/LICENSE'
; make: Warning: Don't know how to make target `/root/illumos/illumos-gate/proto/root_i386/licenses/usr/src/common/openssl/LICENSE.descrip'
; `/root/illumos/illumos-gate/proto/root_i386/licenses/usr/src/common/openssl/LICENSE' is up to date.
; `/root/illumos/illumos-gate/proto/root_i386/licenses/usr/src/common/openssl/LICENSE.descrip' is up to date.
; `/root/illumos/illumos-gate/proto/root_i386/licenses/usr/src/common/openssl/LICENSE' is up to date.
; `/root/illumos/illumos-gate/proto/root_i386/licenses/usr/src/common/openssl/LICENSE.descrip' is up to date.
;
; If I run the make command that builds the file by hand, it says it is
; up-to-date:
;
; build-env# (146) pwd
; /root/illumos/illumos-gate/usr/src/pkg
; build-env# (147) make -e -f Makefile.lic \
;     PKGDEBUG= LICROOT=$PARENT_WS/proto/root_i386/licenses \
;     $PARENT_WS/proto/root_i386/licenses/usr/src/common/openssl/LICENSE
; `/root/illumos/illumos-gate/proto/root_i386/licenses/usr/src/common/openssl/LICENSE' is up to date.
;
; but if I remove it and attempt to rebuild it:
;
; build-env# (148) rm /root/illumos/illumos-gate/proto/root_i386/licenses/usr/src/common/openssl/LICENSE
; build-env# (149) make -e -f Makefile.lic \
;     PKGDEBUG= LICROOT=$PARENT_WS/proto/root_i386/licenses \
;     $PARENT_WS/proto/root_i386/licenses/usr/src/common/openssl/LICENSE
; make: Fatal error: Don't know how to make target `/root/illumos/illumos-gate/proto/root_i386/licenses/usr/src/common/openssl/LICENSE'
;
; This is because the stage-licenses target just tries to copy the licence
; file from the usr/src tree and there is no usr/src/common/openssl in the
; gate.
;
; So I understand why it is failing but not yet what process ends up creating
; proto/root_i386/licenses/usr/src/common/openssl/LICENSE nor why I am not
; getting a clean build here. It looks like it might be the order in which
; things are built - it would be interesting to see the output of
; grep openssl/LIC from someone else's nightly.log to compare.
;
; I'll keep digging but any help appreciated,
;
; Thanks,
;
; Andy
;
;
-- 
Citrus IT Limited | +44 (0)870 199 8000 | enquiries at citrus-it.co.uk
Rock House Farm | Green Moor | Wortley | Sheffield | S35 7DQ
Registered in England and Wales | Company number 4899123


From danmcd at omniti.com  Mon Jul 20 21:55:40 2015
From: danmcd at omniti.com (Dan McDonald)
Date: Mon, 20 Jul 2015 17:55:40 -0400
Subject: [OmniOS-discuss] [developer] HEADS UP -- illumos-gate can now
	be built on OmniOS r151014 or later
In-Reply-To: <alpine.GSO.2.00.1507202148570.7194@erncre.pvgehf-vg.arg>
References: <D8F62323-340B-4334-8BBC-9DE7198ADAC9@omniti.com>
	<alpine.GSO.2.00.1507091040020.2334@erncre.pvgehf-vg.arg>
	<EB7EAE73-93BD-446B-9B9F-3A0DE2C26992@omniti.com>
	<alpine.GSO.2.00.1507101329510.19413@erncre.pvgehf-vg.arg>
	<alpine.GSO.2.00.1507202148570.7194@erncre.pvgehf-vg.arg>
Message-ID: <9571A1E5-E02E-48D6-885C-94B27F1D911F@omniti.com>


> On Jul 20, 2015, at 5:50 PM, Andy Fiddaman <omnios at citrus-it.net> wrote:
> 
> 
> I haven't managed to get to the bottom of this.
> 
> Could someone please send me a nightly.log from a build of the Illumos
> gate on an OmniOS system so I can look for differences?

Sure.

I'm attaching:

* illumos-gate.env  -- an .env file for nightly

* mail_msg.txt -- the mail_msg from my latest illumos-gate build

And I'll put nightly.log here:

	http://kebe.com/~danmcd/nightly.log

These are from my r151014 build machine.

Dan
-------------- next part --------------
A non-text attachment was scrubbed...
Name: illumos-gate.env
Type: application/octet-stream
Size: 8486 bytes
Desc: not available
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150720/b31c56f4/attachment.obj>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: mail_msg.txt
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150720/b31c56f4/attachment.txt>

From hakansom at ohsu.edu  Tue Jul 21 01:06:40 2015
From: hakansom at ohsu.edu (Marion Hakanson)
Date: Mon, 20 Jul 2015 18:06:40 -0700
Subject: [OmniOS-discuss] Slow Drive Detection and boot-archive
In-Reply-To: Message from Michael Talbott <mtalbott@lji.org> of "Fri,
	17 Jul 2015 16:15:47 PDT."
	<95908196-2C9D-4584-8E82-F715DD9A3A14@lji.org>
Message-ID: <201507210106.t6L16ee4022209@kyklops.ohsu.edu>

Michael,

I've not seen this;  I do have one system with 120 drives and it
definitely does not have this problem.  A couple with 80+ drives
are also free of this issue, though they are still running OpenIndiana.

One thing I pretty much always do here, is to disable the boot option
in the LSI HBA's config utility (accessible from the during boot after
the BIOS has started up).  I do this because I don't want the BIOS
thinking it can boot from any of the external JBOD disks;  And also
because I've had some system BIOS crashes when they tried to enumerate
too many drives.  But, this all happens at the BIOS level, before the
OS has even started up, so in theory it should not affect what
you are seeing.

Regards,

Marion


================================================================
Subject: Re: [OmniOS-discuss] Slow Drive Detection and boot-archive
From: Michael Talbott <mtalbott at lji.org>
Date: Fri, 17 Jul 2015 16:15:47 -0700
To: omnios-discuss <omnios-discuss at lists.omniti.com>

Just realized my typo. I'm using this on my 90 and 180 drive systems:

# svccfg -s boot-archive setprop start/timeout_seconds=720
# svccfg -s boot-archive setprop start/timeout_seconds=1440

Seems like 8 seconds to detect each drive is pretty excessive.

Any ideas on how to speed that up?


________________________
Michael Talbott
Systems Administrator
La Jolla Institute

> On Jul 17, 2015, at 4:07 PM, Michael Talbott <mtalbott at lji.org> wrote:
> 
> I have multiple NAS servers I've moved to OmniOS and each of them have 90-180 4T disks. Everything has worked out pretty well for the most part. But I've come into an issue where when I reboot any of them, I'm getting boot-archive service timeouts happening. I found a workaround of increasing the timeout value which brings me to the following. As you can see below in a dmesg output, it's taking the kernel about 8 seconds to detect each of the drives. They're connected via a couple SAS2008 based LSI cards.
> 
> Is this normal?
> Is there a way to speed that up?
> 
> I've fixed my frustrating boot-archive timeout problem by adjusting the timeout value from the default of 60 seconds (I guess that'll work ok on systems with less than 8 drives?) to 8 seconds * 90 drives + a little extra time = 280 seconds (for the 90 drive systems). Which means it takes between 12-24 minutes to boot those machines up.
> 
> # svccfg -s boot-archive setprop start/timeout_seconds=280
> 
> I figure I can't be the only one. A little googling also revealed: https://www.illumos.org/issues/4614 <https://www.illumos.org/issues/4614>
> 
> Jul 17 15:40:15 store2 genunix: [ID 583861 kern.info] sd29 at mpt_sas3: unit-address w50000c0f0401bd43,0: w50000c0f0401bd43,0
> Jul 17 15:40:15 store2 genunix: [ID 936769 kern.info] sd29 is /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0401bd43,0
> Jul 17 15:40:16 store2 genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0401bd43,0 (sd29) online
> Jul 17 15:40:24 store2 genunix: [ID 583861 kern.info] sd30 at mpt_sas3: unit-address w50000c0f045679c3,0: w50000c0f045679c3,0
> Jul 17 15:40:24 store2 genunix: [ID 936769 kern.info] sd30 is /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f045679c3,0
> Jul 17 15:40:24 store2 genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f045679c3,0 (sd30) online
> Jul 17 15:40:33 store2 genunix: [ID 583861 kern.info] sd31 at mpt_sas3: unit-address w50000c0f045712b3,0: w50000c0f045712b3,0
> Jul 17 15:40:33 store2 genunix: [ID 936769 kern.info] sd31 is /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f045712b3,0
> Jul 17 15:40:33 store2 genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f045712b3,0 (sd31) online
> Jul 17 15:40:42 store2 genunix: [ID 583861 kern.info] sd32 at mpt_sas3: unit-address w50000c0f04571497,0: w50000c0f04571497,0
> Jul 17 15:40:42 store2 genunix: [ID 936769 kern.info] sd32 is /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f04571497,0
> Jul 17 15:40:42 store2 genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f04571497,0 (sd32) online
> Jul 17 15:40:50 store2 genunix: [ID 583861 kern.info] sd33 at mpt_sas3: unit-address w50000c0f042ac8eb,0: w50000c0f042ac8eb,0
> Jul 17 15:40:50 store2 genunix: [ID 936769 kern.info] sd33 is /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f042ac8eb,0
> Jul 17 15:40:50 store2 genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f042ac8eb,0 (sd33) online
> Jul 17 15:40:59 store2 genunix: [ID 583861 kern.info] sd34 at mpt_sas3: unit-address w50000c0f04571473,0: w50000c0f04571473,0
> Jul 17 15:40:59 store2 genunix: [ID 936769 kern.info] sd34 is /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f04571473,0
> Jul 17 15:40:59 store2 genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f04571473,0 (sd34) online
> Jul 17 15:41:08 store2 genunix: [ID 583861 kern.info] sd35 at mpt_sas3: unit-address w50000c0f042c636f,0: w50000c0f042c636f,0
> Jul 17 15:41:08 store2 genunix: [ID 936769 kern.info] sd35 is /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f042c636f,0
> Jul 17 15:41:08 store2 genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f042c636f,0 (sd35) online
> Jul 17 15:41:17 store2 genunix: [ID 583861 kern.info] sd36 at mpt_sas3: unit-address w50000c0f0401bf2f,0: w50000c0f0401bf2f,0
> Jul 17 15:41:17 store2 genunix: [ID 936769 kern.info] sd36 is /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0401bf2f,0
> Jul 17 15:41:17 store2 genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0401bf2f,0 (sd36) online
> Jul 17 15:41:25 store2 genunix: [ID 583861 kern.info] sd38 at mpt_sas3: unit-address w50000c0f0401bc1f,0: w50000c0f0401bc1f,0
> Jul 17 15:41:25 store2 genunix: [ID 936769 kern.info] sd38 is /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0401bc1f,0
> Jul 17 15:41:26 store2 genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0401bc1f,0 (sd38) online
> 
> 
> ________________________
> Michael Talbott
> Systems Administrator
> La Jolla Institute
> 

_______________________________________________
OmniOS-discuss mailing list
OmniOS-discuss at lists.omniti.com
http://lists.omniti.com/mailman/listinfo/omnios-discuss



From mtalbott at lji.org  Tue Jul 21 02:56:52 2015
From: mtalbott at lji.org (Michael Talbott)
Date: Mon, 20 Jul 2015 19:56:52 -0700
Subject: [OmniOS-discuss] Slow Drive Detection and boot-archive
In-Reply-To: <201507210106.t6L16ee4022209@kyklops.ohsu.edu>
References: <201507210106.t6L16ee4022209@kyklops.ohsu.edu>
Message-ID: <EF194F85-A03F-428E-A056-424B1D4AA23B@lji.org>

Thanks for the reply. The bios for the card is disabled already. The 8 second per drive scan happens after the kernel has already loaded and it is scanning for devices. I wonder if it's due to running newer firmware. I did update the cards to fw v.20.something before I moved to omnios. Is there a particular firmware version on the cards I should run to match OmniOS's drivers?


________________________
Michael Talbott
Systems Administrator
La Jolla Institute

> On Jul 20, 2015, at 6:06 PM, Marion Hakanson <hakansom at ohsu.edu> wrote:
> 
> Michael,
> 
> I've not seen this;  I do have one system with 120 drives and it
> definitely does not have this problem.  A couple with 80+ drives
> are also free of this issue, though they are still running OpenIndiana.
> 
> One thing I pretty much always do here, is to disable the boot option
> in the LSI HBA's config utility (accessible from the during boot after
> the BIOS has started up).  I do this because I don't want the BIOS
> thinking it can boot from any of the external JBOD disks;  And also
> because I've had some system BIOS crashes when they tried to enumerate
> too many drives.  But, this all happens at the BIOS level, before the
> OS has even started up, so in theory it should not affect what
> you are seeing.
> 
> Regards,
> 
> Marion
> 
> 
> ================================================================
> Subject: Re: [OmniOS-discuss] Slow Drive Detection and boot-archive
> From: Michael Talbott <mtalbott at lji.org>
> Date: Fri, 17 Jul 2015 16:15:47 -0700
> To: omnios-discuss <omnios-discuss at lists.omniti.com>
> 
> Just realized my typo. I'm using this on my 90 and 180 drive systems:
> 
> # svccfg -s boot-archive setprop start/timeout_seconds=720
> # svccfg -s boot-archive setprop start/timeout_seconds=1440
> 
> Seems like 8 seconds to detect each drive is pretty excessive.
> 
> Any ideas on how to speed that up?
> 
> 
> ________________________
> Michael Talbott
> Systems Administrator
> La Jolla Institute
> 
>> On Jul 17, 2015, at 4:07 PM, Michael Talbott <mtalbott at lji.org> wrote:
>> 
>> I have multiple NAS servers I've moved to OmniOS and each of them have 90-180 4T disks. Everything has worked out pretty well for the most part. But I've come into an issue where when I reboot any of them, I'm getting boot-archive service timeouts happening. I found a workaround of increasing the timeout value which brings me to the following. As you can see below in a dmesg output, it's taking the kernel about 8 seconds to detect each of the drives. They're connected via a couple SAS2008 based LSI cards.
>> 
>> Is this normal?
>> Is there a way to speed that up?
>> 
>> I've fixed my frustrating boot-archive timeout problem by adjusting the timeout value from the default of 60 seconds (I guess that'll work ok on systems with less than 8 drives?) to 8 seconds * 90 drives + a little extra time = 280 seconds (for the 90 drive systems). Which means it takes between 12-24 minutes to boot those machines up.
>> 
>> # svccfg -s boot-archive setprop start/timeout_seconds=280
>> 
>> I figure I can't be the only one. A little googling also revealed: https://www.illumos.org/issues/4614 <https://www.illumos.org/issues/4614>
>> 
>> Jul 17 15:40:15 store2 genunix: [ID 583861 kern.info] sd29 at mpt_sas3: unit-address w50000c0f0401bd43,0: w50000c0f0401bd43,0
>> Jul 17 15:40:15 store2 genunix: [ID 936769 kern.info] sd29 is /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0401bd43,0
>> Jul 17 15:40:16 store2 genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0401bd43,0 (sd29) online
>> Jul 17 15:40:24 store2 genunix: [ID 583861 kern.info] sd30 at mpt_sas3: unit-address w50000c0f045679c3,0: w50000c0f045679c3,0
>> Jul 17 15:40:24 store2 genunix: [ID 936769 kern.info] sd30 is /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f045679c3,0
>> Jul 17 15:40:24 store2 genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f045679c3,0 (sd30) online
>> Jul 17 15:40:33 store2 genunix: [ID 583861 kern.info] sd31 at mpt_sas3: unit-address w50000c0f045712b3,0: w50000c0f045712b3,0
>> Jul 17 15:40:33 store2 genunix: [ID 936769 kern.info] sd31 is /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f045712b3,0
>> Jul 17 15:40:33 store2 genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f045712b3,0 (sd31) online
>> Jul 17 15:40:42 store2 genunix: [ID 583861 kern.info] sd32 at mpt_sas3: unit-address w50000c0f04571497,0: w50000c0f04571497,0
>> Jul 17 15:40:42 store2 genunix: [ID 936769 kern.info] sd32 is /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f04571497,0
>> Jul 17 15:40:42 store2 genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f04571497,0 (sd32) online
>> Jul 17 15:40:50 store2 genunix: [ID 583861 kern.info] sd33 at mpt_sas3: unit-address w50000c0f042ac8eb,0: w50000c0f042ac8eb,0
>> Jul 17 15:40:50 store2 genunix: [ID 936769 kern.info] sd33 is /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f042ac8eb,0
>> Jul 17 15:40:50 store2 genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f042ac8eb,0 (sd33) online
>> Jul 17 15:40:59 store2 genunix: [ID 583861 kern.info] sd34 at mpt_sas3: unit-address w50000c0f04571473,0: w50000c0f04571473,0
>> Jul 17 15:40:59 store2 genunix: [ID 936769 kern.info] sd34 is /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f04571473,0
>> Jul 17 15:40:59 store2 genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f04571473,0 (sd34) online
>> Jul 17 15:41:08 store2 genunix: [ID 583861 kern.info] sd35 at mpt_sas3: unit-address w50000c0f042c636f,0: w50000c0f042c636f,0
>> Jul 17 15:41:08 store2 genunix: [ID 936769 kern.info] sd35 is /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f042c636f,0
>> Jul 17 15:41:08 store2 genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f042c636f,0 (sd35) online
>> Jul 17 15:41:17 store2 genunix: [ID 583861 kern.info] sd36 at mpt_sas3: unit-address w50000c0f0401bf2f,0: w50000c0f0401bf2f,0
>> Jul 17 15:41:17 store2 genunix: [ID 936769 kern.info] sd36 is /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0401bf2f,0
>> Jul 17 15:41:17 store2 genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0401bf2f,0 (sd36) online
>> Jul 17 15:41:25 store2 genunix: [ID 583861 kern.info] sd38 at mpt_sas3: unit-address w50000c0f0401bc1f,0: w50000c0f0401bc1f,0
>> Jul 17 15:41:25 store2 genunix: [ID 936769 kern.info] sd38 is /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0401bc1f,0
>> Jul 17 15:41:26 store2 genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0401bc1f,0 (sd38) online
>> 
>> 
>> ________________________
>> Michael Talbott
>> Systems Administrator
>> La Jolla Institute
>> 
> 
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
> 
> 


From info at houseofancients.nl  Tue Jul 21 06:27:39 2015
From: info at houseofancients.nl (Floris van Essen ..:: House of Ancients Amstafs ::..)
Date: Tue, 21 Jul 2015 06:27:39 +0000
Subject: [OmniOS-discuss] Slow Drive Detection and boot-archive
In-Reply-To: <EF194F85-A03F-428E-A056-424B1D4AA23B@lji.org>
References: <201507210106.t6L16ee4022209@kyklops.ohsu.edu>
	<EF194F85-A03F-428E-A056-424B1D4AA23B@lji.org>
Message-ID: <356582D1FC91784992ABB4265A16ED489AABAAC8@vEX01.mindstorm-internet.local>

Michael,

I know v20 does cause lots of issue's.
V19 , to the best of my knowledge doesn't contain any, so I would downgrade to v19


Kr,


Floris
-----Oorspronkelijk bericht-----
Van: OmniOS-discuss [mailto:omnios-discuss-bounces at lists.omniti.com] Namens Michael Talbott
Verzonden: dinsdag 21 juli 2015 4:57
Aan: Marion Hakanson <hakansom at ohsu.edu>
CC: omnios-discuss <omnios-discuss at lists.omniti.com>
Onderwerp: Re: [OmniOS-discuss] Slow Drive Detection and boot-archive

Thanks for the reply. The bios for the card is disabled already. The 8 second per drive scan happens after the kernel has already loaded and it is scanning for devices. I wonder if it's due to running newer firmware. I did update the cards to fw v.20.something before I moved to omnios. Is there a particular firmware version on the cards I should run to match OmniOS's drivers?


________________________
Michael Talbott
Systems Administrator
La Jolla Institute

> On Jul 20, 2015, at 6:06 PM, Marion Hakanson <hakansom at ohsu.edu> wrote:
> 
> Michael,
> 
> I've not seen this;  I do have one system with 120 drives and it 
> definitely does not have this problem.  A couple with 80+ drives are 
> also free of this issue, though they are still running OpenIndiana.
> 
> One thing I pretty much always do here, is to disable the boot option 
> in the LSI HBA's config utility (accessible from the during boot after 
> the BIOS has started up).  I do this because I don't want the BIOS 
> thinking it can boot from any of the external JBOD disks;  And also 
> because I've had some system BIOS crashes when they tried to enumerate 
> too many drives.  But, this all happens at the BIOS level, before the 
> OS has even started up, so in theory it should not affect what you are 
> seeing.
> 
> Regards,
> 
> Marion
> 
> 
> ================================================================
> Subject: Re: [OmniOS-discuss] Slow Drive Detection and boot-archive
> From: Michael Talbott <mtalbott at lji.org>
> Date: Fri, 17 Jul 2015 16:15:47 -0700
> To: omnios-discuss <omnios-discuss at lists.omniti.com>
> 
> Just realized my typo. I'm using this on my 90 and 180 drive systems:
> 
> # svccfg -s boot-archive setprop start/timeout_seconds=720 # svccfg -s 
> boot-archive setprop start/timeout_seconds=1440
> 
> Seems like 8 seconds to detect each drive is pretty excessive.
> 
> Any ideas on how to speed that up?
> 
> 
> ________________________
> Michael Talbott
> Systems Administrator
> La Jolla Institute
> 
>> On Jul 17, 2015, at 4:07 PM, Michael Talbott <mtalbott at lji.org> wrote:
>> 
>> I have multiple NAS servers I've moved to OmniOS and each of them have 90-180 4T disks. Everything has worked out pretty well for the most part. But I've come into an issue where when I reboot any of them, I'm getting boot-archive service timeouts happening. I found a workaround of increasing the timeout value which brings me to the following. As you can see below in a dmesg output, it's taking the kernel about 8 seconds to detect each of the drives. They're connected via a couple SAS2008 based LSI cards.
>> 
>> Is this normal?
>> Is there a way to speed that up?
>> 
>> I've fixed my frustrating boot-archive timeout problem by adjusting the timeout value from the default of 60 seconds (I guess that'll work ok on systems with less than 8 drives?) to 8 seconds * 90 drives + a little extra time = 280 seconds (for the 90 drive systems). Which means it takes between 12-24 minutes to boot those machines up.
>> 
>> # svccfg -s boot-archive setprop start/timeout_seconds=280
>> 
>> I figure I can't be the only one. A little googling also revealed: 
>> https://www.illumos.org/issues/4614 
>> <https://www.illumos.org/issues/4614>
>> 
>> Jul 17 15:40:15 store2 genunix: [ID 583861 kern.info] sd29 at 
>> mpt_sas3: unit-address w50000c0f0401bd43,0: w50000c0f0401bd43,0 Jul 
>> 17 15:40:15 store2 genunix: [ID 936769 kern.info] sd29 is 
>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0401bd4
>> 3,0 Jul 17 15:40:16 store2 genunix: [ID 408114 kern.info] 
>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0401bd4
>> 3,0 (sd29) online Jul 17 15:40:24 store2 genunix: [ID 583861 
>> kern.info] sd30 at mpt_sas3: unit-address w50000c0f045679c3,0: 
>> w50000c0f045679c3,0 Jul 17 15:40:24 store2 genunix: [ID 936769 
>> kern.info] sd30 is 
>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f045679c
>> 3,0 Jul 17 15:40:24 store2 genunix: [ID 408114 kern.info] 
>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f045679c
>> 3,0 (sd30) online Jul 17 15:40:33 store2 genunix: [ID 583861 
>> kern.info] sd31 at mpt_sas3: unit-address w50000c0f045712b3,0: 
>> w50000c0f045712b3,0 Jul 17 15:40:33 store2 genunix: [ID 936769 
>> kern.info] sd31 is 
>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f045712b
>> 3,0 Jul 17 15:40:33 store2 genunix: [ID 408114 kern.info] 
>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f045712b
>> 3,0 (sd31) online Jul 17 15:40:42 store2 genunix: [ID 583861 
>> kern.info] sd32 at mpt_sas3: unit-address w50000c0f04571497,0: 
>> w50000c0f04571497,0 Jul 17 15:40:42 store2 genunix: [ID 936769 
>> kern.info] sd32 is 
>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0457149
>> 7,0 Jul 17 15:40:42 store2 genunix: [ID 408114 kern.info] 
>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0457149
>> 7,0 (sd32) online Jul 17 15:40:50 store2 genunix: [ID 583861 
>> kern.info] sd33 at mpt_sas3: unit-address w50000c0f042ac8eb,0: 
>> w50000c0f042ac8eb,0 Jul 17 15:40:50 store2 genunix: [ID 936769 
>> kern.info] sd33 is 
>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f042ac8e
>> b,0 Jul 17 15:40:50 store2 genunix: [ID 408114 kern.info] 
>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f042ac8e
>> b,0 (sd33) online Jul 17 15:40:59 store2 genunix: [ID 583861 
>> kern.info] sd34 at mpt_sas3: unit-address w50000c0f04571473,0: 
>> w50000c0f04571473,0 Jul 17 15:40:59 store2 genunix: [ID 936769 
>> kern.info] sd34 is 
>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0457147
>> 3,0 Jul 17 15:40:59 store2 genunix: [ID 408114 kern.info] 
>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0457147
>> 3,0 (sd34) online Jul 17 15:41:08 store2 genunix: [ID 583861 
>> kern.info] sd35 at mpt_sas3: unit-address w50000c0f042c636f,0: 
>> w50000c0f042c636f,0 Jul 17 15:41:08 store2 genunix: [ID 936769 
>> kern.info] sd35 is 
>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f042c636
>> f,0 Jul 17 15:41:08 store2 genunix: [ID 408114 kern.info] 
>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f042c636
>> f,0 (sd35) online Jul 17 15:41:17 store2 genunix: [ID 583861 
>> kern.info] sd36 at mpt_sas3: unit-address w50000c0f0401bf2f,0: 
>> w50000c0f0401bf2f,0 Jul 17 15:41:17 store2 genunix: [ID 936769 
>> kern.info] sd36 is 
>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0401bf2
>> f,0 Jul 17 15:41:17 store2 genunix: [ID 408114 kern.info] 
>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0401bf2
>> f,0 (sd36) online Jul 17 15:41:25 store2 genunix: [ID 583861 
>> kern.info] sd38 at mpt_sas3: unit-address w50000c0f0401bc1f,0: 
>> w50000c0f0401bc1f,0 Jul 17 15:41:25 store2 genunix: [ID 936769 
>> kern.info] sd38 is 
>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0401bc1
>> f,0 Jul 17 15:41:26 store2 genunix: [ID 408114 kern.info] 
>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0401bc1
>> f,0 (sd38) online
>> 
>> 
>> ________________________
>> Michael Talbott
>> Systems Administrator
>> La Jolla Institute
>> 
> 
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
> 
> 

_______________________________________________
OmniOS-discuss mailing list
OmniOS-discuss at lists.omniti.com
http://lists.omniti.com/mailman/listinfo/omnios-discuss
...:: House of Ancients ::...
American Staffordshire Terriers

+31-628-161-350
+31-614-198-389
Het Perk 48
4903 RB
Oosterhout
Netherlands
www.houseofancients.nl

From johan.kragsterman at capvert.se  Tue Jul 21 08:06:14 2015
From: johan.kragsterman at capvert.se (Johan Kragsterman)
Date: Tue, 21 Jul 2015 10:06:14 +0200
Subject: [OmniOS-discuss] Ang: Re:  Slow Drive Detection and boot-archive
In-Reply-To: <356582D1FC91784992ABB4265A16ED489AABAAC8@vEX01.mindstorm-internet.local>
References: <356582D1FC91784992ABB4265A16ED489AABAAC8@vEX01.mindstorm-internet.local>,
	<201507210106.t6L16ee4022209@kyklops.ohsu.edu>	<EF194F85-A03F-428E-A056-424B1D4AA23B@lji.org>
Message-ID: <OF4498A788.B5D3F3C2-ONC1257E89.002C8473-C1257E89.002C8477@inse.com>


Hi!


-----"OmniOS-discuss" <omnios-discuss-bounces at lists.omniti.com> skrev: -----
Till: Michael Talbott <mtalbott at lji.org>, Marion Hakanson <hakansom at ohsu.edu>
Fr?n: "Floris van Essen ..:: House of Ancients Amstafs ::.." 
S?nt av: "OmniOS-discuss" 
Datum: 2015-07-21 08:29
Kopia: omnios-discuss <omnios-discuss at lists.omniti.com>
?rende: Re: [OmniOS-discuss] Slow Drive Detection and boot-archive

Michael,

I know v20 does cause lots of issue's.
V19 , to the best of my knowledge doesn't contain any, so I would downgrade to v19


Kr,


Floris







Yes, v.20 is known to be poisonous, the v.18 and 19 are known to work well. I don't know of any reports of more recent versions, so I can't comment on those, if there are any.

Johan





-----Oorspronkelijk bericht-----
Van: OmniOS-discuss [mailto:omnios-discuss-bounces at lists.omniti.com] Namens Michael Talbott
Verzonden: dinsdag 21 juli 2015 4:57
Aan: Marion Hakanson <hakansom at ohsu.edu>
CC: omnios-discuss <omnios-discuss at lists.omniti.com>
Onderwerp: Re: [OmniOS-discuss] Slow Drive Detection and boot-archive

Thanks for the reply. The bios for the card is disabled already. The 8 second per drive scan happens after the kernel has already loaded and it is scanning for devices. I wonder if it's due to running newer firmware. I did update the cards to fw v.20.something before I moved to omnios. Is there a particular firmware version on the cards I should run to match OmniOS's drivers?


________________________
Michael Talbott
Systems Administrator
La Jolla Institute

> On Jul 20, 2015, at 6:06 PM, Marion Hakanson <hakansom at ohsu.edu> wrote:
> 
> Michael,
> 
> I've not seen this; ?I do have one system with 120 drives and it 
> definitely does not have this problem. ?A couple with 80+ drives are 
> also free of this issue, though they are still running OpenIndiana.
> 
> One thing I pretty much always do here, is to disable the boot option 
> in the LSI HBA's config utility (accessible from the during boot after 
> the BIOS has started up). ?I do this because I don't want the BIOS 
> thinking it can boot from any of the external JBOD disks; ?And also 
> because I've had some system BIOS crashes when they tried to enumerate 
> too many drives. ?But, this all happens at the BIOS level, before the 
> OS has even started up, so in theory it should not affect what you are 
> seeing.
> 
> Regards,
> 
> Marion
> 
> 
> ================================================================
> Subject: Re: [OmniOS-discuss] Slow Drive Detection and boot-archive
> From: Michael Talbott <mtalbott at lji.org>
> Date: Fri, 17 Jul 2015 16:15:47 -0700
> To: omnios-discuss <omnios-discuss at lists.omniti.com>
> 
> Just realized my typo. I'm using this on my 90 and 180 drive systems:
> 
> # svccfg -s boot-archive setprop start/timeout_seconds=720 # svccfg -s 
> boot-archive setprop start/timeout_seconds=1440
> 
> Seems like 8 seconds to detect each drive is pretty excessive.
> 
> Any ideas on how to speed that up?
> 
> 
> ________________________
> Michael Talbott
> Systems Administrator
> La Jolla Institute
> 
>> On Jul 17, 2015, at 4:07 PM, Michael Talbott <mtalbott at lji.org> wrote:
>> 
>> I have multiple NAS servers I've moved to OmniOS and each of them have 90-180 4T disks. Everything has worked out pretty well for the most part. But I've come into an issue where when I reboot any of them, I'm getting boot-archive service timeouts happening. I found a workaround of increasing the timeout value which brings me to the following. As you can see below in a dmesg output, it's taking the kernel about 8 seconds to detect each of the drives. They're connected via a couple SAS2008 based LSI cards.
>> 
>> Is this normal?
>> Is there a way to speed that up?
>> 
>> I've fixed my frustrating boot-archive timeout problem by adjusting the timeout value from the default of 60 seconds (I guess that'll work ok on systems with less than 8 drives?) to 8 seconds * 90 drives + a little extra time = 280 seconds (for the 90 drive systems). Which means it takes between 12-24 minutes to boot those machines up.
>> 
>> # svccfg -s boot-archive setprop start/timeout_seconds=280
>> 
>> I figure I can't be the only one. A little googling also revealed: 
>> https://www.illumos.org/issues/4614?
>> <https://www.illumos.org/issues/4614>
>> 
>> Jul 17 15:40:15 store2 genunix: [ID 583861 kern.info] sd29 at 
>> mpt_sas3: unit-address w50000c0f0401bd43,0: w50000c0f0401bd43,0 Jul 
>> 17 15:40:15 store2 genunix: [ID 936769 kern.info] sd29 is 
>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0401bd4
>> 3,0 Jul 17 15:40:16 store2 genunix: [ID 408114 kern.info] 
>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0401bd4
>> 3,0 (sd29) online Jul 17 15:40:24 store2 genunix: [ID 583861 
>> kern.info] sd30 at mpt_sas3: unit-address w50000c0f045679c3,0: 
>> w50000c0f045679c3,0 Jul 17 15:40:24 store2 genunix: [ID 936769 
>> kern.info] sd30 is 
>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f045679c
>> 3,0 Jul 17 15:40:24 store2 genunix: [ID 408114 kern.info] 
>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f045679c
>> 3,0 (sd30) online Jul 17 15:40:33 store2 genunix: [ID 583861 
>> kern.info] sd31 at mpt_sas3: unit-address w50000c0f045712b3,0: 
>> w50000c0f045712b3,0 Jul 17 15:40:33 store2 genunix: [ID 936769 
>> kern.info] sd31 is 
>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f045712b
>> 3,0 Jul 17 15:40:33 store2 genunix: [ID 408114 kern.info] 
>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f045712b
>> 3,0 (sd31) online Jul 17 15:40:42 store2 genunix: [ID 583861 
>> kern.info] sd32 at mpt_sas3: unit-address w50000c0f04571497,0: 
>> w50000c0f04571497,0 Jul 17 15:40:42 store2 genunix: [ID 936769 
>> kern.info] sd32 is 
>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0457149
>> 7,0 Jul 17 15:40:42 store2 genunix: [ID 408114 kern.info] 
>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0457149
>> 7,0 (sd32) online Jul 17 15:40:50 store2 genunix: [ID 583861 
>> kern.info] sd33 at mpt_sas3: unit-address w50000c0f042ac8eb,0: 
>> w50000c0f042ac8eb,0 Jul 17 15:40:50 store2 genunix: [ID 936769 
>> kern.info] sd33 is 
>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f042ac8e
>> b,0 Jul 17 15:40:50 store2 genunix: [ID 408114 kern.info] 
>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f042ac8e
>> b,0 (sd33) online Jul 17 15:40:59 store2 genunix: [ID 583861 
>> kern.info] sd34 at mpt_sas3: unit-address w50000c0f04571473,0: 
>> w50000c0f04571473,0 Jul 17 15:40:59 store2 genunix: [ID 936769 
>> kern.info] sd34 is 
>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0457147
>> 3,0 Jul 17 15:40:59 store2 genunix: [ID 408114 kern.info] 
>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0457147
>> 3,0 (sd34) online Jul 17 15:41:08 store2 genunix: [ID 583861 
>> kern.info] sd35 at mpt_sas3: unit-address w50000c0f042c636f,0: 
>> w50000c0f042c636f,0 Jul 17 15:41:08 store2 genunix: [ID 936769 
>> kern.info] sd35 is 
>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f042c636
>> f,0 Jul 17 15:41:08 store2 genunix: [ID 408114 kern.info] 
>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f042c636
>> f,0 (sd35) online Jul 17 15:41:17 store2 genunix: [ID 583861 
>> kern.info] sd36 at mpt_sas3: unit-address w50000c0f0401bf2f,0: 
>> w50000c0f0401bf2f,0 Jul 17 15:41:17 store2 genunix: [ID 936769 
>> kern.info] sd36 is 
>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0401bf2
>> f,0 Jul 17 15:41:17 store2 genunix: [ID 408114 kern.info] 
>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0401bf2
>> f,0 (sd36) online Jul 17 15:41:25 store2 genunix: [ID 583861 
>> kern.info] sd38 at mpt_sas3: unit-address w50000c0f0401bc1f,0: 
>> w50000c0f0401bc1f,0 Jul 17 15:41:25 store2 genunix: [ID 936769 
>> kern.info] sd38 is 
>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0401bc1
>> f,0 Jul 17 15:41:26 store2 genunix: [ID 408114 kern.info] 
>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0401bc1
>> f,0 (sd38) online
>> 
>> 
>> ________________________
>> Michael Talbott
>> Systems Administrator
>> La Jolla Institute
>> 
> 
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
> 
> 

_______________________________________________
OmniOS-discuss mailing list
OmniOS-discuss at lists.omniti.com
http://lists.omniti.com/mailman/listinfo/omnios-discuss
...:: House of Ancients ::...
American Staffordshire Terriers

+31-628-161-350
+31-614-198-389
Het Perk 48
4903 RB
Oosterhout
Netherlands
www.houseofancients.nl
_______________________________________________
OmniOS-discuss mailing list
OmniOS-discuss at lists.omniti.com
http://lists.omniti.com/mailman/listinfo/omnios-discuss




From richard.elling at richardelling.com  Tue Jul 21 14:08:16 2015
From: richard.elling at richardelling.com (Richard Elling)
Date: Tue, 21 Jul 2015 07:08:16 -0700
Subject: [OmniOS-discuss] Slow Drive Detection and boot-archive
In-Reply-To: <EF194F85-A03F-428E-A056-424B1D4AA23B@lji.org>
References: <201507210106.t6L16ee4022209@kyklops.ohsu.edu>
	<EF194F85-A03F-428E-A056-424B1D4AA23B@lji.org>
Message-ID: <630C6FFA-9DA8-4617-AB76-8B4C09EB6C86@richardelling.com>


> On Jul 20, 2015, at 7:56 PM, Michael Talbott <mtalbott at lji.org> wrote:
> 
> Thanks for the reply. The bios for the card is disabled already. The 8 second per drive scan happens after the kernel has already loaded and it is scanning for devices. I wonder if it's due to running newer firmware. I did update the cards to fw v.20.something before I moved to omnios. Is there a particular firmware version on the cards I should run to match OmniOS's drivers?

Google "LSI P20 firmware" for many tales of woe for many different OSes.
Be aware that getting the latest version of firmware from Avago might not be obvious...
the latest version is 20.00.04.00 for Windows.
 -- richard

> 
> 
> ________________________
> Michael Talbott
> Systems Administrator
> La Jolla Institute
> 
>> On Jul 20, 2015, at 6:06 PM, Marion Hakanson <hakansom at ohsu.edu> wrote:
>> 
>> Michael,
>> 
>> I've not seen this;  I do have one system with 120 drives and it
>> definitely does not have this problem.  A couple with 80+ drives
>> are also free of this issue, though they are still running OpenIndiana.
>> 
>> One thing I pretty much always do here, is to disable the boot option
>> in the LSI HBA's config utility (accessible from the during boot after
>> the BIOS has started up).  I do this because I don't want the BIOS
>> thinking it can boot from any of the external JBOD disks;  And also
>> because I've had some system BIOS crashes when they tried to enumerate
>> too many drives.  But, this all happens at the BIOS level, before the
>> OS has even started up, so in theory it should not affect what
>> you are seeing.
>> 
>> Regards,
>> 
>> Marion
>> 
>> 
>> ================================================================
>> Subject: Re: [OmniOS-discuss] Slow Drive Detection and boot-archive
>> From: Michael Talbott <mtalbott at lji.org>
>> Date: Fri, 17 Jul 2015 16:15:47 -0700
>> To: omnios-discuss <omnios-discuss at lists.omniti.com>
>> 
>> Just realized my typo. I'm using this on my 90 and 180 drive systems:
>> 
>> # svccfg -s boot-archive setprop start/timeout_seconds=720
>> # svccfg -s boot-archive setprop start/timeout_seconds=1440
>> 
>> Seems like 8 seconds to detect each drive is pretty excessive.
>> 
>> Any ideas on how to speed that up?
>> 
>> 
>> ________________________
>> Michael Talbott
>> Systems Administrator
>> La Jolla Institute
>> 
>>> On Jul 17, 2015, at 4:07 PM, Michael Talbott <mtalbott at lji.org> wrote:
>>> 
>>> I have multiple NAS servers I've moved to OmniOS and each of them have 90-180 4T disks. Everything has worked out pretty well for the most part. But I've come into an issue where when I reboot any of them, I'm getting boot-archive service timeouts happening. I found a workaround of increasing the timeout value which brings me to the following. As you can see below in a dmesg output, it's taking the kernel about 8 seconds to detect each of the drives. They're connected via a couple SAS2008 based LSI cards.
>>> 
>>> Is this normal?
>>> Is there a way to speed that up?
>>> 
>>> I've fixed my frustrating boot-archive timeout problem by adjusting the timeout value from the default of 60 seconds (I guess that'll work ok on systems with less than 8 drives?) to 8 seconds * 90 drives + a little extra time = 280 seconds (for the 90 drive systems). Which means it takes between 12-24 minutes to boot those machines up.
>>> 
>>> # svccfg -s boot-archive setprop start/timeout_seconds=280
>>> 
>>> I figure I can't be the only one. A little googling also revealed: https://www.illumos.org/issues/4614 <https://www.illumos.org/issues/4614>
>>> 
>>> Jul 17 15:40:15 store2 genunix: [ID 583861 kern.info] sd29 at mpt_sas3: unit-address w50000c0f0401bd43,0: w50000c0f0401bd43,0
>>> Jul 17 15:40:15 store2 genunix: [ID 936769 kern.info] sd29 is /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0401bd43,0
>>> Jul 17 15:40:16 store2 genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0401bd43,0 (sd29) online
>>> Jul 17 15:40:24 store2 genunix: [ID 583861 kern.info] sd30 at mpt_sas3: unit-address w50000c0f045679c3,0: w50000c0f045679c3,0
>>> Jul 17 15:40:24 store2 genunix: [ID 936769 kern.info] sd30 is /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f045679c3,0
>>> Jul 17 15:40:24 store2 genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f045679c3,0 (sd30) online
>>> Jul 17 15:40:33 store2 genunix: [ID 583861 kern.info] sd31 at mpt_sas3: unit-address w50000c0f045712b3,0: w50000c0f045712b3,0
>>> Jul 17 15:40:33 store2 genunix: [ID 936769 kern.info] sd31 is /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f045712b3,0
>>> Jul 17 15:40:33 store2 genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f045712b3,0 (sd31) online
>>> Jul 17 15:40:42 store2 genunix: [ID 583861 kern.info] sd32 at mpt_sas3: unit-address w50000c0f04571497,0: w50000c0f04571497,0
>>> Jul 17 15:40:42 store2 genunix: [ID 936769 kern.info] sd32 is /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f04571497,0
>>> Jul 17 15:40:42 store2 genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f04571497,0 (sd32) online
>>> Jul 17 15:40:50 store2 genunix: [ID 583861 kern.info] sd33 at mpt_sas3: unit-address w50000c0f042ac8eb,0: w50000c0f042ac8eb,0
>>> Jul 17 15:40:50 store2 genunix: [ID 936769 kern.info] sd33 is /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f042ac8eb,0
>>> Jul 17 15:40:50 store2 genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f042ac8eb,0 (sd33) online
>>> Jul 17 15:40:59 store2 genunix: [ID 583861 kern.info] sd34 at mpt_sas3: unit-address w50000c0f04571473,0: w50000c0f04571473,0
>>> Jul 17 15:40:59 store2 genunix: [ID 936769 kern.info] sd34 is /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f04571473,0
>>> Jul 17 15:40:59 store2 genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f04571473,0 (sd34) online
>>> Jul 17 15:41:08 store2 genunix: [ID 583861 kern.info] sd35 at mpt_sas3: unit-address w50000c0f042c636f,0: w50000c0f042c636f,0
>>> Jul 17 15:41:08 store2 genunix: [ID 936769 kern.info] sd35 is /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f042c636f,0
>>> Jul 17 15:41:08 store2 genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f042c636f,0 (sd35) online
>>> Jul 17 15:41:17 store2 genunix: [ID 583861 kern.info] sd36 at mpt_sas3: unit-address w50000c0f0401bf2f,0: w50000c0f0401bf2f,0
>>> Jul 17 15:41:17 store2 genunix: [ID 936769 kern.info] sd36 is /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0401bf2f,0
>>> Jul 17 15:41:17 store2 genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0401bf2f,0 (sd36) online
>>> Jul 17 15:41:25 store2 genunix: [ID 583861 kern.info] sd38 at mpt_sas3: unit-address w50000c0f0401bc1f,0: w50000c0f0401bc1f,0
>>> Jul 17 15:41:25 store2 genunix: [ID 936769 kern.info] sd38 is /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0401bc1f,0
>>> Jul 17 15:41:26 store2 genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0401bc1f,0 (sd38) online
>>> 
>>> 
>>> ________________________
>>> Michael Talbott
>>> Systems Administrator
>>> La Jolla Institute
>>> 
>> 
>> _______________________________________________
>> OmniOS-discuss mailing list
>> OmniOS-discuss at lists.omniti.com
>> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>> 
>> 
> 
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss


From rafibeyli at gmail.com  Tue Jul 21 17:25:42 2015
From: rafibeyli at gmail.com (Hafiz Rafiyev)
Date: Tue, 21 Jul 2015 20:25:42 +0300 (EEST)
Subject: [OmniOS-discuss] DL380 Gen9 H240 hba
Message-ID: <674082797.215043.1437499542057.JavaMail.zimbra@cantekstil.com.tr>

Hello,

anyway using H240 HBA on Proliant DL380 G9?is it supported on illumos or omnios?

Server configured with H240 HBA by customer, supposed  its LSI SAS3008 based:)),but H240  arrived with  PMC(Adaptec) controller.

Regards

Hafiz.



From danmcd at omniti.com  Tue Jul 21 17:44:33 2015
From: danmcd at omniti.com (Dan McDonald)
Date: Tue, 21 Jul 2015 13:44:33 -0400
Subject: [OmniOS-discuss] DL380 Gen9 H240 hba
In-Reply-To: <674082797.215043.1437499542057.JavaMail.zimbra@cantekstil.com.tr>
References: <674082797.215043.1437499542057.JavaMail.zimbra@cantekstil.com.tr>
Message-ID: <52DE4E2C-989F-468F-949E-7DEB5EA94784@omniti.com>

Does the cpqary3 driver help here?  "prtconf -d" output would be helpful.

Dan

Sent from my iPhone (typos, autocorrect, and all)

> On Jul 21, 2015, at 1:25 PM, Hafiz Rafiyev <rafibeyli at gmail.com> wrote:
> 
> Hello,
> 
> anyway using H240 HBA on Proliant DL380 G9?is it supported on illumos or omnios?
> 
> Server configured with H240 HBA by customer, supposed  its LSI SAS3008 based:)),but H240  arrived with  PMC(Adaptec) controller.
> 
> Regards
> 
> Hafiz.
> 
> 
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss

From rafibeyli at gmail.com  Wed Jul 22 09:55:14 2015
From: rafibeyli at gmail.com (Hafiz Rafiyev)
Date: Wed, 22 Jul 2015 12:55:14 +0300 (EEST)
Subject: [OmniOS-discuss] DL380 Gen9 H240 hba
In-Reply-To: <52DE4E2C-989F-468F-949E-7DEB5EA94784@omniti.com>
References: <674082797.215043.1437499542057.JavaMail.zimbra@cantekstil.com.tr>
	<52DE4E2C-989F-468F-949E-7DEB5EA94784@omniti.com>
Message-ID: <2089438292.229624.1437558914407.JavaMail.zimbra@cantekstil.com.tr>

Dan it's HP's Smart Host Bus Adaptor

http://www8.hp.com/h20195/v2/getpdf.aspx/c04375839.pdf?ver=2

prtconf -d
System Configuration:  Oracle Corporation  i86pc
Memory size: 32644 Megabytes
System Peripherals (Software Nodes):

i86pc
    scsi_vhci, instance #0
    pci, instance #0
        pci103c,21ea (pciex8086,2f00) [Intel Corporation Haswell-E DMI2] (driver not attached)
        pci8086,2f02 (pciex8086,2f02) [Intel Corporation Haswell-E PCI Express Root Port 1] (driver not attached)
        pci8086,2f03 (pciex8086,2f03) [Intel Corporation Haswell-E PCI Express Root Port 1] (driver not attached)
        pci8086,2f04 (pciex8086,2f04) [Intel Corporation Haswell-E PCI Express Root Port 2], instance #2
            pci103c,21c7 (pciex103c,3239) [Hewlett-Packard Company unknown device] (driver not attached)
        pci8086,2f05 (pciex8086,2f05) [Intel Corporation Haswell-E PCI Express Root Port 2] (driver not attached)
        pci8086,2f06 (pciex8086,2f06) [Intel Corporation Haswell-E PCI Express Root Port 2], instance #4
            pci103c,17d1 (pciex8086,1521) [Intel Corporation I350 Gigabit Network Connection], instance #0
            pci103c,17d1 (pciex8086,1521) [Intel Corporation I350 Gigabit Network Connection], instance #1
            pci103c,17d1 (pciex8086,1521) [Intel Corporation I350 Gigabit Network Connection], instance #2
            pci103c,17d1 (pciex8086,1521) [Intel Corporation I350 Gigabit Network Connection], instance #3
        pci8086,2f07 (pciex8086,2f07) [Intel Corporation Haswell-E PCI Express Root Port 2] (driver not attached)
        pci8086,2f08 (pciex8086,2f08) [Intel Corporation Haswell-E PCI Express Root Port 3] (driver not attached)
        pci8086,2f09 (pciex8086,2f09) [Intel Corporation Haswell-E PCI Express Root Port 3] (driver not attached)
        pci8086,2f0a (pciex8086,2f0a) [Intel Corporation Haswell-E PCI Express Root Port 3] (driver not attached)
        pci8086,2f0b (pciex8086,2f0b) [Intel Corporation Haswell-E PCI Express Root Port 3] (driver not attached)
        pci103c,21ea (pciex8086,2f20) [Intel Corporation Haswell-E DMA Channel 0] (driver not attached)
        pci103c,21ea (pciex8086,2f21) [Intel Corporation Haswell-E DMA Channel 1] (driver not attached)
        pci103c,21ea (pciex8086,2f22) [Intel Corporation Haswell-E DMA Channel 2] (driver not attached)
        pci103c,21ea (pciex8086,2f23) [Intel Corporation Haswell-E DMA Channel 3] (driver not attached)
        pci103c,21ea (pciex8086,2f24) [Intel Corporation Haswell-E DMA Channel 4] (driver not attached)
        pci103c,21ea (pciex8086,2f25) [Intel Corporation Haswell-E DMA Channel 5] (driver not attached)
        pci103c,21ea (pciex8086,2f26) [Intel Corporation Haswell-E DMA Channel 6] (driver not attached)
        pci103c,21ea (pciex8086,2f27) [Intel Corporation Haswell-E DMA Channel 7] (driver not attached)
        pci103c,21ea (pciex8086,2f28) [Intel Corporation Haswell-E Address Map, VTd_Misc, System Management] (driver not attached)
        pci103c,21ea (pciex8086,2f29) [Intel Corporation Haswell-E Hot Plug] (driver not attached)
        pci103c,21ea (pciex8086,2f2a) [Intel Corporation Haswell-E RAS, Control Status and Global Errors] (driver not attached)
        pci103c,21ea (pciex8086,2f2c) [Intel Corporation Haswell-E I/O Apic] (driver not attached)
        pci103c,8030 (pciex8086,8d7c) [Intel Corporation Wellsburg SPSR] (driver not attached)
        pci103c,8030 (pciex8086,8d2d) [Intel Corporation Wellsburg USB Enhanced Host Controller #2], instance #0
            hub, instance #0
                device, instance #0
                    keyboard, instance #1
                    input, instance #2
                mouse, instance #0
        pci8086,8d10 (pciex8086,8d10) [Intel Corporation Wellsburg PCI Express Root Port #1] (driver not attached)
        pci8086,8d14 (pciex8086,8d14) [Intel Corporation Wellsburg PCI Express Root Port #3], instance #11
            pci103c,3381 (pciex103c,3306) [Hewlett-Packard Company Integrated Lights-Out Standard Slave Instrumentation & System Support] (driver not attached)
            display (pciex102b,533) [Matrox Electronics Systems Ltd. MGA G200EH], instance #0
            pci103c,3381 (pciex103c,3307) [Hewlett-Packard Company Integrated Lights-Out Standard Management Processor Support and Messaging] (driver not attached)
            pci103c,3381 (pciex103c,3300) [Hewlett-Packard Company Integrated Lights-Out Standard Virtual USB Controller], instance #0
        pci8086,8d18 (pciex8086,8d18) [Intel Corporation Wellsburg PCI Express Root Port #5], instance #12
            pci103c,22be (pciex14e4,1657) [Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet PCIe] (driver not attached)
            pci103c,22be (pciex14e4,1657) [Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet PCIe] (driver not attached)
            pci103c,22be (pciex14e4,1657) [Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet PCIe] (driver not attached)
            pci103c,22be (pciex14e4,1657) [Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet PCIe] (driver not attached)
        pci8086,8d1c (pciex8086,8d1c) [Intel Corporation Wellsburg PCI Express Root Port #7] (driver not attached)
        pci8086,8d1e (pciex8086,8d1e) [Intel Corporation Wellsburg PCI Express Root Port #8] (driver not attached)
        pci103c,8030 (pciex8086,8d26) [Intel Corporation Wellsburg USB Enhanced Host Controller #1], instance #1
            hub, instance #1
                hub, instance #2
        isa (pciex8086,8d44) [Intel Corporation Wellsburg LPC Controller], instance #0
            pit_beep, instance #0
        pci103c,8030 (pciex8086,8d02) [Intel Corporation Wellsburg 6-Port SATA Controller [AHCI mode]], instance #0
            disk, instance #0
            disk, instance #1
        pci103c,8030 (pciex8086,8d22) [Intel Corporation Wellsburg SMBus Controller] (driver not attached)
    ioapics (driver not attached)
        ioapic, instance #0 (driver not attached)
        ioapic, instance #1 (driver not attached)
    pci, instance #0
        pci8086,2f80 (pci8086,2f80) [Intel Corporation Haswell-E QPI Link 0] (driver not attached)
        pci8086,2f83 (pci8086,2f83) [Intel Corporation Haswell-E QPI Link 0] (driver not attached)
        pci103c,21ea (pci8086,2f81) [Intel Corporation Haswell-E R3 QPI Link 0 & 1 Monitoring] (driver not attached)
        pci103c,21ea (pci8086,2f36) [Intel Corporation Haswell-E R3 QPI Link 0 & 1 Monitoring] (driver not attached)
        pci103c,21ea (pci8086,2f37) [Intel Corporation Haswell-E R3 QPI Link 0 & 1 Monitoring] (driver not attached)
        pci103c,21ea (pci8086,2fe0) [Intel Corporation Haswell-E Unicast Registers] (driver not attached)
        pci103c,21ea (pci8086,2fe1) [Intel Corporation Haswell-E Unicast Registers] (driver not attached)
        pci103c,21ea (pci8086,2fe2) [Intel Corporation Haswell-E Unicast Registers] (driver not attached)
        pci103c,21ea (pci8086,2fe3) [Intel Corporation Haswell-E Unicast Registers] (driver not attached)
        pci103c,21ea (pci8086,2fe4) [Intel Corporation Haswell-E Unicast Registers] (driver not attached)
        pci103c,21ea (pci8086,2fe5) [Intel Corporation Haswell-E Unicast Registers] (driver not attached)
        pci8086,2ff8 (pci8086,2ff8) [Intel Corporation Haswell-E Buffered Ring Agent] (driver not attached)
        pci8086,2ff9 (pci8086,2ff9) [Intel Corporation Haswell-E Buffered Ring Agent] (driver not attached)
        pci103c,21ea (pci8086,2ffc) [Intel Corporation Haswell-E System Address Decoder & Broadcast Registers] (driver not attached)
        pci103c,21ea (pci8086,2ffd) [Intel Corporation Haswell-E System Address Decoder & Broadcast Registers] (driver not attached)
        pci103c,21ea (pci8086,2ffe) [Intel Corporation Haswell-E System Address Decoder & Broadcast Registers] (driver not attached)
        pci103c,21ea (pci8086,2f1d) [Intel Corporation Haswell-E PCIe Ring Interface] (driver not attached)
        pci103c,21ea (pci8086,2f34) [Intel Corporation Haswell-E PCIe Ring Interface] (driver not attached)
        pci103c,21ea (pci8086,2f1e) [Intel Corporation Haswell-E Scratchpad & Semaphore Registers] (driver not attached)
        pci103c,21ea (pci8086,2f7d) [Intel Corporation Haswell-E Scratchpad & Semaphore Registers] (driver not attached)
        pci103c,21ea (pci8086,2f1f) [Intel Corporation Haswell-E Scratchpad & Semaphore Registers] (driver not attached)
        pci103c,21ea (pci8086,2fa0) [Intel Corporation Haswell-E Home Agent 0] (driver not attached)
        pci103c,21ea (pci8086,2f30) [Intel Corporation Haswell-E Home Agent 0] (driver not attached)
        pci103c,21ea (pci8086,2f70) [Intel Corporation Haswell-E Home Agent 0 Debug] (driver not attached)
        pci103c,21ea (pci8086,2fa8) [Intel Corporation Haswell-E Integrated Memory Controller 0 Target Address, Thermal & RAS Registers] (driver not attached)
        pci103c,21ea (pci8086,2f71) [Intel Corporation Haswell-E Integrated Memory Controller 0 Target Address, Thermal & RAS Registers] (driver not attached)
        pci103c,21ea (pci8086,2faa) [Intel Corporation Haswell-E Integrated Memory Controller 0 Channel Target Address Decoder] (driver not attached)
        pci103c,21ea (pci8086,2fab) [Intel Corporation Haswell-E Integrated Memory Controller 0 Channel Target Address Decoder] (driver not attached)
        pci103c,21ea (pci8086,2fac) [Intel Corporation Haswell-E Integrated Memory Controller 0 Channel Target Address Decoder] (driver not attached)
        pci103c,21ea (pci8086,2fad) [Intel Corporation Haswell-E Integrated Memory Controller 0 Channel Target Address Decoder] (driver not attached)
        pci8086,2fae (pci8086,2fae) [Intel Corporation Haswell-E DDRIO Channel 0/1 Broadcast] (driver not attached)
        pci8086,2faf (pci8086,2faf) [Intel Corporation Haswell-E DDRIO Global Broadcast] (driver not attached)
        pci103c,21ea (pci8086,2fb0) [Intel Corporation Haswell-E Integrated Memory Controller 0 Channel 0 Thermal Control] (driver not attached)
        pci103c,21ea (pci8086,2fb1) [Intel Corporation Haswell-E Integrated Memory Controller 0 Channel 1 Thermal Control] (driver not attached)
        pci103c,21ea (pci8086,2fb2) [Intel Corporation Haswell-E Integrated Memory Controller 0 Channel 0 ERROR Registers] (driver not attached)
        pci103c,21ea (pci8086,2fb3) [Intel Corporation Haswell-E Integrated Memory Controller 0 Channel 1 ERROR Registers] (driver not attached)
        pci8086,2fbe (pci8086,2fbe) [Intel Corporation Haswell-E DDRIO (VMSE) 0 & 1] (driver not attached)
        pci8086,2fbf (pci8086,2fbf) [Intel Corporation Haswell-E DDRIO (VMSE) 0 & 1] (driver not attached)
        pci103c,21ea (pci8086,2fb4) [Intel Corporation Haswell-E Integrated Memory Controller 0 Channel 2 Thermal Control] (driver not attached)
        pci103c,21ea (pci8086,2fb5) [Intel Corporation Haswell-E Integrated Memory Controller 0 Channel 3 Thermal Control] (driver not attached)
        pci103c,21ea (pci8086,2fb6) [Intel Corporation Haswell-E Integrated Memory Controller 0 Channel 2 ERROR Registers] (driver not attached)
        pci103c,21ea (pci8086,2fb7) [Intel Corporation Haswell-E Integrated Memory Controller 0 Channel 3 ERROR Registers] (driver not attached)
        pci8086,2f68 (pci8086,2f68) [Intel Corporation Haswell-E Integrated Memory Controller 1 Target Address, Thermal & RAS Registers] (driver not attached)
        pci8086,2f6e (pci8086,2f6e) [Intel Corporation Haswell-E DDRIO Channel 2/3 Broadcast] (driver not attached)
        pci8086,2f6f (pci8086,2f6f) [Intel Corporation Haswell-E DDRIO Global Broadcast] (driver not attached)
        pci8086,2fd0 (pci8086,2fd0) [Intel Corporation Haswell-E Integrated Memory Controller 1 Channel 0 Thermal Control] (driver not attached)
        pci8086,2fb8 (pci8086,2fb8) [Intel Corporation Haswell-E DDRIO (VMSE) 2 & 3] (driver not attached)
        pci8086,2fb9 (pci8086,2fb9) [Intel Corporation Haswell-E DDRIO (VMSE) 2 & 3] (driver not attached)
        pci8086,2fba (pci8086,2fba) [Intel Corporation Haswell-E DDRIO (VMSE) 2 & 3] (driver not attached)
        pci8086,2fbb (pci8086,2fbb) [Intel Corporation Haswell-E DDRIO (VMSE) 2 & 3] (driver not attached)
        pci103c,21ea (pci8086,2f98) [Intel Corporation Haswell-E Power Control Unit] (driver not attached)
        pci103c,21ea (pci8086,2f99) [Intel Corporation Haswell-E Power Control Unit] (driver not attached)
        pci103c,21ea (pci8086,2f9a) [Intel Corporation Haswell-E Power Control Unit] (driver not attached)
        pci103c,21ea (pci8086,2fc0) [Intel Corporation Haswell-E Power Control Unit] (driver not attached)
        pci103c,21ea (pci8086,2f9c) [Intel Corporation Haswell-E Power Control Unit] (driver not attached)
        pci8086,2f88 (pci8086,2f88) [Intel Corporation Haswell-E VCU] (driver not attached)
        pci8086,2f8a (pci8086,2f8a) [Intel Corporation Haswell-E VCU] (driver not attached)
    pci, instance #1
        pci8086,2f01 (pciex8086,2f01) [Intel Corporation Haswell-E PCI Express Root Port 0] (driver not attached)
        pci8086,2f02 (pciex8086,2f02) [Intel Corporation Haswell-E PCI Express Root Port 1] (driver not attached)
        pci8086,2f03 (pciex8086,2f03) [Intel Corporation Haswell-E PCI Express Root Port 1] (driver not attached)
        pci8086,2f04 (pciex8086,2f04) [Intel Corporation Haswell-E PCI Express Root Port 2] (driver not attached)
        pci8086,2f05 (pciex8086,2f05) [Intel Corporation Haswell-E PCI Express Root Port 2] (driver not attached)
        pci8086,2f06 (pciex8086,2f06) [Intel Corporation Haswell-E PCI Express Root Port 2] (driver not attached)
        pci8086,2f07 (pciex8086,2f07) [Intel Corporation Haswell-E PCI Express Root Port 2] (driver not attached)
        pci8086,2f08 (pciex8086,2f08) [Intel Corporation Haswell-E PCI Express Root Port 3] (driver not attached)
        pci8086,2f09 (pciex8086,2f09) [Intel Corporation Haswell-E PCI Express Root Port 3] (driver not attached)
        pci8086,2f0a (pciex8086,2f0a) [Intel Corporation Haswell-E PCI Express Root Port 3] (driver not attached)
        pci8086,2f0b (pciex8086,2f0b) [Intel Corporation Haswell-E PCI Express Root Port 3] (driver not attached)
        pci103c,21ea (pciex8086,2f20) [Intel Corporation Haswell-E DMA Channel 0] (driver not attached)
        pci103c,21ea (pciex8086,2f21) [Intel Corporation Haswell-E DMA Channel 1] (driver not attached)
        pci103c,21ea (pciex8086,2f22) [Intel Corporation Haswell-E DMA Channel 2] (driver not attached)
        pci103c,21ea (pciex8086,2f23) [Intel Corporation Haswell-E DMA Channel 3] (driver not attached)
        pci103c,21ea (pciex8086,2f24) [Intel Corporation Haswell-E DMA Channel 4] (driver not attached)
        pci103c,21ea (pciex8086,2f25) [Intel Corporation Haswell-E DMA Channel 5] (driver not attached)
        pci103c,21ea (pciex8086,2f26) [Intel Corporation Haswell-E DMA Channel 6] (driver not attached)
        pci103c,21ea (pciex8086,2f27) [Intel Corporation Haswell-E DMA Channel 7] (driver not attached)
        pci103c,21ea (pciex8086,2f28) [Intel Corporation Haswell-E Address Map, VTd_Misc, System Management] (driver not attached)
        pci103c,21ea (pciex8086,2f29) [Intel Corporation Haswell-E Hot Plug] (driver not attached)
        pci103c,21ea (pciex8086,2f2a) [Intel Corporation Haswell-E RAS, Control Status and Global Errors] (driver not attached)
        pci103c,21ea (pciex8086,2f2c) [Intel Corporation Haswell-E I/O Apic] (driver not attached)
    fw, instance #0
        sb, instance #1
            socket, instance #2
            socket, instance #3
    used-resources (driver not attached)
    fcoe, instance #0
    aoe, instance #0
    iscsi, instance #0
    pseudo, instance #0
    options, instance #0
    xsvc, instance #0
    cpus, instance #0
        cpu (driver not attached)


----- Original Message -----
From: "Dan McDonald" <danmcd at omniti.com>
To: "Hafiz Rafibeyli" <rafibeyli at gmail.com>
Cc: "omnios-discuss" <omnios-discuss at lists.omniti.com>
Sent: Tuesday, 21 July, 2015 20:44:33
Subject: Re: [OmniOS-discuss] DL380 Gen9 H240 hba

Does the cpqary3 driver help here?  "prtconf -d" output would be helpful.

Dan

Sent from my iPhone (typos, autocorrect, and all)

> On Jul 21, 2015, at 1:25 PM, Hafiz Rafiyev <rafibeyli at gmail.com> wrote:
> 
> Hello,
> 
> anyway using H240 HBA on Proliant DL380 G9?is it supported on illumos or omnios?
> 
> Server configured with H240 HBA by customer, supposed  its LSI SAS3008 based:)),but H240  arrived with  PMC(Adaptec) controller.
> 
> Regards
> 
> Hafiz.
> 
> 
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss

From danmcd at omniti.com  Wed Jul 22 14:18:25 2015
From: danmcd at omniti.com (Dan McDonald)
Date: Wed, 22 Jul 2015 10:18:25 -0400
Subject: [OmniOS-discuss] DL380 Gen9 H240 hba
In-Reply-To: <2089438292.229624.1437558914407.JavaMail.zimbra@cantekstil.com.tr>
References: <674082797.215043.1437499542057.JavaMail.zimbra@cantekstil.com.tr>
	<52DE4E2C-989F-468F-949E-7DEB5EA94784@omniti.com>
	<2089438292.229624.1437558914407.JavaMail.zimbra@cantekstil.com.tr>
Message-ID: <984F8D94-FFD7-42FA-9932-2B18324BB60B@omniti.com>


> On Jul 22, 2015, at 5:55 AM, Hafiz Rafiyev <rafibeyli at gmail.com> wrote:
> 
> Dan it's HP's Smart Host Bus Adaptor

Yep, aka:

>            pci103c,21c7 (pciex103c,3239) [Hewlett-Packard Company unknown device] (driver not attached)

That guy.  If you look, there are PCI number *close* to it in /etc/driver_aliases for "cpqary3".

I wonder if HP needs to update the driver or not?  I'm going to forward this note to the illumos developer list and ask.

Dan



From martinschki at mac.com  Thu Jul 23 06:51:49 2015
From: martinschki at mac.com (Martin Kloboucnik)
Date: Thu, 23 Jul 2015 08:51:49 +0200
Subject: [OmniOS-discuss] vmware issue with mirror
Message-ID: <261F69C4-5DD6-41E0-8993-508C8ED8ACC9@mac.com>

Hello,

I?m running  "OmniOS 5.11   omnios-8d266aa  2013.05.04? as my storage system with Napp-it on top on an ESXi server.
I have OmniOS on a virtual disk (vmdk) and have a second disk mounted with an identical disk, also basically a vmdk file.
From there , more specific from one pool then I start my VM machines
This was working flawless until recently?

Omnios didn?t boot anymore and I managed to snap a screenshot of the boot process?

~~~~~~~~~~~~~~~~~~~~~~
SunOS Release 5.11 Version omnios-b281e50 64-bit
Copyright?..
NOTICE: Cannot read the pool lable frpm ?/pci at 0,0/pci15ad, 1976 at 10/sd at 1,0:a?
NOTICE: spa_import_rootpool: error 5
Cannot mount on /pci at 0,0/pci15ad, 1976 at 10/sd at 1,0:a fstype zfs

panic[cpu0]/thread=fffffffffbc2fac0: vfs_mountroot: cannot mount root

Warning - stack not written to the dump buffer
fffffffffbc722f0 genunix:vfs_mountroot+373 ()
fffffffffbc72330 genunix:main+138 ()
fffffffffbc72340 unix:_locore_start+90 ()

skipping system dump - no dump device configured
rebooting?
~~~~~~~~~~~~~~~~~~~~~~




Now here are my two questions:

1. Anyone any idea why I get this message during boot?

2. Would it make sense to install a fresh stable Omnios and import all my pools? It?s only 3 mirrored data pools?
	I have to say that I?m a little scared of doing this since I had once an episode after updating omnios that my pools went ?bad? after the upgrade. So I downgraded to where I am now and never touched it anymore but maybe it is time to move on?


systemspec:
Supermicro X9SCL
Intel 1240v2
32GB RAM
2 M1015 controller flashed with IT Firmware
NEW: two EVO 850 SSD?s for omnios mounted in ESXi
2 320GB HDD
2 4TB HDD
2 3TB HDD



I appreciate your help
Thanks
Martin



From danmcd at omniti.com  Thu Jul 23 13:49:37 2015
From: danmcd at omniti.com (Dan McDonald)
Date: Thu, 23 Jul 2015 09:49:37 -0400
Subject: [OmniOS-discuss] vmware issue with mirror
In-Reply-To: <261F69C4-5DD6-41E0-8993-508C8ED8ACC9@mac.com>
References: <261F69C4-5DD6-41E0-8993-508C8ED8ACC9@mac.com>
Message-ID: <CFA037FF-F25D-4A8A-AA3A-044E384BFE29@omniti.com>


> On Jul 23, 2015, at 2:51 AM, Martin Kloboucnik <martinschki at mac.com> wrote:
> 
> 
> I?m running  "OmniOS 5.11   omnios-8d266aa

You claim this, but then your boot banner says:

> SunOS Release 5.11 Version omnios-b281e50 64-bit

That.

> Copyright?..
> NOTICE: Cannot read the pool lable frpm ?/pci at 0,0/pci15ad, 1976 at 10/sd at 1,0:a?
> NOTICE: spa_import_rootpool: error 5
> Cannot mount on /pci at 0,0/pci15ad, 1976 at 10/sd at 1,0:a fstype zfs

The disk mentioned in the path above (one of your rpool disks) is somehow broken.  "error 5" is EIO, which ZFS returns if it has a problem reading the disk (pardon the redundancy).


I'd recommend doing a fresh install (of OmniOS r151014) and importing your data pools.  You may need to use -f for import, but since you keep data separate from the rpool, I don't see a huge problem doing this.

Dan


From martinschki at mac.com  Thu Jul 23 13:56:16 2015
From: martinschki at mac.com (Martin Kloboucnik)
Date: Thu, 23 Jul 2015 15:56:16 +0200
Subject: [OmniOS-discuss] vmware issue with mirror
In-Reply-To: <CFA037FF-F25D-4A8A-AA3A-044E384BFE29@omniti.com>
References: <261F69C4-5DD6-41E0-8993-508C8ED8ACC9@mac.com>
	<CFA037FF-F25D-4A8A-AA3A-044E384BFE29@omniti.com>
Message-ID: <DDA9D17D-B174-4485-8196-B2B091FD8314@mac.com>

Dan,

I?m confused too?.
shell says: omnios-8d266aa
boot screen says: omnios-b281e50

but thanks anyway for your answer
Martin



> On 23.07.2015, at 15:49, Dan McDonald <danmcd at omniti.com> wrote:
> 
> 
>> On Jul 23, 2015, at 2:51 AM, Martin Kloboucnik <martinschki at mac.com> wrote:
>> 
>> 
>> I?m running  "OmniOS 5.11   omnios-8d266aa
> 
> You claim this, but then your boot banner says:
> 
>> SunOS Release 5.11 Version omnios-b281e50 64-bit
> 
> That.
> 
>> Copyright?..
>> NOTICE: Cannot read the pool lable frpm ?/pci at 0,0/pci15ad, 1976 at 10/sd at 1,0:a?
>> NOTICE: spa_import_rootpool: error 5
>> Cannot mount on /pci at 0,0/pci15ad, 1976 at 10/sd at 1,0:a fstype zfs
> 
> The disk mentioned in the path above (one of your rpool disks) is somehow broken.  "error 5" is EIO, which ZFS returns if it has a problem reading the disk (pardon the redundancy).
> 
> 
> I'd recommend doing a fresh install (of OmniOS r151014) and importing your data pools.  You may need to use -f for import, but since you keep data separate from the rpool, I don't see a huge problem doing this.
> 
> Dan
> 


From cal-s at blue-bolt.com  Fri Jul 24 09:33:12 2015
From: cal-s at blue-bolt.com (Cal Sawyer)
Date: Fri, 24 Jul 2015 10:33:12 +0100
Subject: [OmniOS-discuss] root pool on HP SmartArray: any cons?
In-Reply-To: <mailman.240.1437558918.63303.omnios-discuss@lists.omniti.com>
References: <mailman.240.1437558918.63303.omnios-discuss@lists.omniti.com>
Message-ID: <55B20658.3070103@blue-bolt.com>

Hi

I'm building a new head on an HP dl360 G6 running r151014 that will 
front a large storage pool.  The storage pool itself will be connected 
(properly) via an LSI 9207 HBA

I've installed OmniOS's root pool on top of the SmartArray's p410i 
mirrored LV mainly because it was there already and i like to use 
hardware RAID whenever i can

Are there any issues associated with running the root pool on top of 
hardware RAID?

thanks!

Cal Sawyer | Systems Engineer | BlueBolt Ltd


From danmcd at omniti.com  Fri Jul 24 13:00:01 2015
From: danmcd at omniti.com (Dan McDonald)
Date: Fri, 24 Jul 2015 09:00:01 -0400
Subject: [OmniOS-discuss] root pool on HP SmartArray: any cons?
In-Reply-To: <55B20658.3070103@blue-bolt.com>
References: <mailman.240.1437558918.63303.omnios-discuss@lists.omniti.com>
	<55B20658.3070103@blue-bolt.com>
Message-ID: <A3CFD9B0-CC31-4E1C-8E5A-72CFFF208169@omniti.com>


> On Jul 24, 2015, at 5:33 AM, Cal Sawyer <cal-s at blue-bolt.com> wrote:
> 
> Hi
> 
> I'm building a new head on an HP dl360 G6 running r151014 that will front a large storage pool.  The storage pool itself will be connected (properly) via an LSI 9207 HBA
> 
> I've installed OmniOS's root pool on top of the SmartArray's p410i mirrored LV mainly because it was there already and i like to use hardware RAID whenever i can

Okay, G6 is supported.  I'd almost read that as "G9", which isn't yet, but should be coming.

> Are there any issues associated with running the root pool on top of hardware RAID?

You really are *reducing* your reliability by trusting hardware RAID.  If a disk fails with hw-RAID, you're beholding to the (closed source) board firmware to do the right thing and recover.  A disk fails on ZFS, you can detach it, remove it, replace it, and resilver it without any machine downtime.

We discourage our paying support customers from using hardware RAID in any situation.

Dan


From stephan.budach at JVM.DE  Fri Jul 24 14:40:32 2015
From: stephan.budach at JVM.DE (Stephan Budach)
Date: Fri, 24 Jul 2015 16:40:32 +0200
Subject: [OmniOS-discuss] root pool on HP SmartArray: any cons?
In-Reply-To: <A3CFD9B0-CC31-4E1C-8E5A-72CFFF208169@omniti.com>
References: <mailman.240.1437558918.63303.omnios-discuss@lists.omniti.com>	<55B20658.3070103@blue-bolt.com>
	<A3CFD9B0-CC31-4E1C-8E5A-72CFFF208169@omniti.com>
Message-ID: <55B24E60.5030705@jvm.de>

Am 24.07.15 um 15:00 schrieb Dan McDonald:
>> On Jul 24, 2015, at 5:33 AM, Cal Sawyer <cal-s at blue-bolt.com> wrote:
>>
>> Hi
>>
>> I'm building a new head on an HP dl360 G6 running r151014 that will front a large storage pool.  The storage pool itself will be connected (properly) via an LSI 9207 HBA
>>
>> I've installed OmniOS's root pool on top of the SmartArray's p410i mirrored LV mainly because it was there already and i like to use hardware RAID whenever i can
> Okay, G6 is supported.  I'd almost read that as "G9", which isn't yet, but should be coming.
>
>> Are there any issues associated with running the root pool on top of hardware RAID?
> You really are *reducing* your reliability by trusting hardware RAID.  If a disk fails with hw-RAID, you're beholding to the (closed source) board firmware to do the right thing and recover.  A disk fails on ZFS, you can detach it, remove it, replace it, and resilver it without any machine downtime.
>
> We discourage our paying support customers from using hardware RAID in any situation.
>
> Dan
>
Don't you have to run a single disk on these HP raids as raid0 anyway? 
That is, doesn't the controller firmware create some kind of logical 
volume on it anyway? It may be better than trusting the whole hw-raid1, 
but likely only marginally, no?

Cheers,
budy

From danmcd at omniti.com  Fri Jul 24 18:44:48 2015
From: danmcd at omniti.com (Dan McDonald)
Date: Fri, 24 Jul 2015 14:44:48 -0400
Subject: [OmniOS-discuss] New bloody update - now with vioif and vioif on
	installer media
Message-ID: <1A3D1B20-9C33-4514-9956-5303F6BD320A@omniti.com>

Hello folks!

I've updated OmniOS bloody on the repo server, AND on the media side as well.

The big highlight for this update is the addition of vioif, a virtual network device, that enables OmniOS to be an OpenStack/Linux KVM guest.  This should allow the installation of an OmniOS bloody ISO as an OpenStack guest.  I'd appreciate any additional testing with this, though we have tried this on a customer OpenStack deployment.

omnios-build is at masterrevision b42fd1f for this update, and illumos-omnios is at master 3d8d739, so uname -v will show omnios-3d8d739.

Other changes besides vioif support since last time:

- To enable vioif, disabling directed EOIs (end of interrupts) on Linux KVM.

- zfs recv now prefetches indirect blocks.

- dmake is now part of illumos-omnios.  Developers should also install the "sccs" and "developer/as" packages.  (Actually in last repo update, but relevant for new CD/USB/Kayak installs).

- "ssh-keygen -A" is now the default usage for host installs.  Thanks to Lauri "lotheac" Tirkkonen for this.

Happy updating!
Dan


From mtalbott at lji.org  Fri Jul 24 22:03:05 2015
From: mtalbott at lji.org (Michael Talbott)
Date: Fri, 24 Jul 2015 15:03:05 -0700
Subject: [OmniOS-discuss] Slow Drive Detection and boot-archive
In-Reply-To: <356582D1FC91784992ABB4265A16ED489AABAAC8@vEX01.mindstorm-internet.local>
References: <201507210106.t6L16ee4022209@kyklops.ohsu.edu>
	<EF194F85-A03F-428E-A056-424B1D4AA23B@lji.org>
	<356582D1FC91784992ABB4265A16ED489AABAAC8@vEX01.mindstorm-internet.local>
Message-ID: <C1265D7C-AB4E-47E3-B0B5-C3FE8B578D1D@lji.org>

Hi,

I've downgraded the cards (LSI 9211-8e) to v.19 and disabled their boot bios. But I'm still getting the 8 second per drive delay after the kernel loads. Any other ideas?


________________________
Michael Talbott
Systems Administrator
La Jolla Institute

> On Jul 20, 2015, at 11:27 PM, Floris van Essen ..:: House of Ancients Amstafs ::.. <info at houseofancients.nl> wrote:
> 
> Michael,
> 
> I know v20 does cause lots of issue's.
> V19 , to the best of my knowledge doesn't contain any, so I would downgrade to v19
> 
> 
> Kr,
> 
> 
> Floris
> -----Oorspronkelijk bericht-----
> Van: OmniOS-discuss [mailto:omnios-discuss-bounces at lists.omniti.com] Namens Michael Talbott
> Verzonden: dinsdag 21 juli 2015 4:57
> Aan: Marion Hakanson <hakansom at ohsu.edu>
> CC: omnios-discuss <omnios-discuss at lists.omniti.com>
> Onderwerp: Re: [OmniOS-discuss] Slow Drive Detection and boot-archive
> 
> Thanks for the reply. The bios for the card is disabled already. The 8 second per drive scan happens after the kernel has already loaded and it is scanning for devices. I wonder if it's due to running newer firmware. I did update the cards to fw v.20.something before I moved to omnios. Is there a particular firmware version on the cards I should run to match OmniOS's drivers?
> 
> 
> ________________________
> Michael Talbott
> Systems Administrator
> La Jolla Institute
> 
>> On Jul 20, 2015, at 6:06 PM, Marion Hakanson <hakansom at ohsu.edu> wrote:
>> 
>> Michael,
>> 
>> I've not seen this;  I do have one system with 120 drives and it 
>> definitely does not have this problem.  A couple with 80+ drives are 
>> also free of this issue, though they are still running OpenIndiana.
>> 
>> One thing I pretty much always do here, is to disable the boot option 
>> in the LSI HBA's config utility (accessible from the during boot after 
>> the BIOS has started up).  I do this because I don't want the BIOS 
>> thinking it can boot from any of the external JBOD disks;  And also 
>> because I've had some system BIOS crashes when they tried to enumerate 
>> too many drives.  But, this all happens at the BIOS level, before the 
>> OS has even started up, so in theory it should not affect what you are 
>> seeing.
>> 
>> Regards,
>> 
>> Marion
>> 
>> 
>> ================================================================
>> Subject: Re: [OmniOS-discuss] Slow Drive Detection and boot-archive
>> From: Michael Talbott <mtalbott at lji.org>
>> Date: Fri, 17 Jul 2015 16:15:47 -0700
>> To: omnios-discuss <omnios-discuss at lists.omniti.com>
>> 
>> Just realized my typo. I'm using this on my 90 and 180 drive systems:
>> 
>> # svccfg -s boot-archive setprop start/timeout_seconds=720 # svccfg -s 
>> boot-archive setprop start/timeout_seconds=1440
>> 
>> Seems like 8 seconds to detect each drive is pretty excessive.
>> 
>> Any ideas on how to speed that up?
>> 
>> 
>> ________________________
>> Michael Talbott
>> Systems Administrator
>> La Jolla Institute
>> 
>>> On Jul 17, 2015, at 4:07 PM, Michael Talbott <mtalbott at lji.org> wrote:
>>> 
>>> I have multiple NAS servers I've moved to OmniOS and each of them have 90-180 4T disks. Everything has worked out pretty well for the most part. But I've come into an issue where when I reboot any of them, I'm getting boot-archive service timeouts happening. I found a workaround of increasing the timeout value which brings me to the following. As you can see below in a dmesg output, it's taking the kernel about 8 seconds to detect each of the drives. They're connected via a couple SAS2008 based LSI cards.
>>> 
>>> Is this normal?
>>> Is there a way to speed that up?
>>> 
>>> I've fixed my frustrating boot-archive timeout problem by adjusting the timeout value from the default of 60 seconds (I guess that'll work ok on systems with less than 8 drives?) to 8 seconds * 90 drives + a little extra time = 280 seconds (for the 90 drive systems). Which means it takes between 12-24 minutes to boot those machines up.
>>> 
>>> # svccfg -s boot-archive setprop start/timeout_seconds=280
>>> 
>>> I figure I can't be the only one. A little googling also revealed: 
>>> https://www.illumos.org/issues/4614 
>>> <https://www.illumos.org/issues/4614>
>>> 
>>> Jul 17 15:40:15 store2 genunix: [ID 583861 kern.info] sd29 at 
>>> mpt_sas3: unit-address w50000c0f0401bd43,0: w50000c0f0401bd43,0 Jul 
>>> 17 15:40:15 store2 genunix: [ID 936769 kern.info] sd29 is 
>>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0401bd4
>>> 3,0 Jul 17 15:40:16 store2 genunix: [ID 408114 kern.info] 
>>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0401bd4
>>> 3,0 (sd29) online Jul 17 15:40:24 store2 genunix: [ID 583861 
>>> kern.info] sd30 at mpt_sas3: unit-address w50000c0f045679c3,0: 
>>> w50000c0f045679c3,0 Jul 17 15:40:24 store2 genunix: [ID 936769 
>>> kern.info] sd30 is 
>>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f045679c
>>> 3,0 Jul 17 15:40:24 store2 genunix: [ID 408114 kern.info] 
>>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f045679c
>>> 3,0 (sd30) online Jul 17 15:40:33 store2 genunix: [ID 583861 
>>> kern.info] sd31 at mpt_sas3: unit-address w50000c0f045712b3,0: 
>>> w50000c0f045712b3,0 Jul 17 15:40:33 store2 genunix: [ID 936769 
>>> kern.info] sd31 is 
>>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f045712b
>>> 3,0 Jul 17 15:40:33 store2 genunix: [ID 408114 kern.info] 
>>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f045712b
>>> 3,0 (sd31) online Jul 17 15:40:42 store2 genunix: [ID 583861 
>>> kern.info] sd32 at mpt_sas3: unit-address w50000c0f04571497,0: 
>>> w50000c0f04571497,0 Jul 17 15:40:42 store2 genunix: [ID 936769 
>>> kern.info] sd32 is 
>>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0457149
>>> 7,0 Jul 17 15:40:42 store2 genunix: [ID 408114 kern.info] 
>>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0457149
>>> 7,0 (sd32) online Jul 17 15:40:50 store2 genunix: [ID 583861 
>>> kern.info] sd33 at mpt_sas3: unit-address w50000c0f042ac8eb,0: 
>>> w50000c0f042ac8eb,0 Jul 17 15:40:50 store2 genunix: [ID 936769 
>>> kern.info] sd33 is 
>>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f042ac8e
>>> b,0 Jul 17 15:40:50 store2 genunix: [ID 408114 kern.info] 
>>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f042ac8e
>>> b,0 (sd33) online Jul 17 15:40:59 store2 genunix: [ID 583861 
>>> kern.info] sd34 at mpt_sas3: unit-address w50000c0f04571473,0: 
>>> w50000c0f04571473,0 Jul 17 15:40:59 store2 genunix: [ID 936769 
>>> kern.info] sd34 is 
>>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0457147
>>> 3,0 Jul 17 15:40:59 store2 genunix: [ID 408114 kern.info] 
>>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0457147
>>> 3,0 (sd34) online Jul 17 15:41:08 store2 genunix: [ID 583861 
>>> kern.info] sd35 at mpt_sas3: unit-address w50000c0f042c636f,0: 
>>> w50000c0f042c636f,0 Jul 17 15:41:08 store2 genunix: [ID 936769 
>>> kern.info] sd35 is 
>>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f042c636
>>> f,0 Jul 17 15:41:08 store2 genunix: [ID 408114 kern.info] 
>>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f042c636
>>> f,0 (sd35) online Jul 17 15:41:17 store2 genunix: [ID 583861 
>>> kern.info] sd36 at mpt_sas3: unit-address w50000c0f0401bf2f,0: 
>>> w50000c0f0401bf2f,0 Jul 17 15:41:17 store2 genunix: [ID 936769 
>>> kern.info] sd36 is 
>>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0401bf2
>>> f,0 Jul 17 15:41:17 store2 genunix: [ID 408114 kern.info] 
>>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0401bf2
>>> f,0 (sd36) online Jul 17 15:41:25 store2 genunix: [ID 583861 
>>> kern.info] sd38 at mpt_sas3: unit-address w50000c0f0401bc1f,0: 
>>> w50000c0f0401bc1f,0 Jul 17 15:41:25 store2 genunix: [ID 936769 
>>> kern.info] sd38 is 
>>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0401bc1
>>> f,0 Jul 17 15:41:26 store2 genunix: [ID 408114 kern.info] 
>>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0401bc1
>>> f,0 (sd38) online
>>> 
>>> 
>>> ________________________
>>> Michael Talbott
>>> Systems Administrator
>>> La Jolla Institute
>>> 
>> 
>> _______________________________________________
>> OmniOS-discuss mailing list
>> OmniOS-discuss at lists.omniti.com
>> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>> 
>> 
> 
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
> ...:: House of Ancients ::...
> American Staffordshire Terriers
> 
> +31-628-161-350
> +31-614-198-389
> Het Perk 48
> 4903 RB
> Oosterhout
> Netherlands
> www.houseofancients.nl


From venture37 at gmail.com  Sat Jul 25 15:26:08 2015
From: venture37 at gmail.com (Sevan / Venture37)
Date: Sat, 25 Jul 2015 16:26:08 +0100
Subject: [OmniOS-discuss] pkgsrc-current OmniOS 170cea2/i386 2015-07-09
	21:35
In-Reply-To: <A3703B3C-8953-4CB1-BC11-7C898AB6F182@omniti.com>
References: <CA+U3Mf6yA5VHgr5WdS9dhiEHs1epCmr22V_0KZ-S25soeWwnHw@mail.gmail.com>
	<811B3E16-0565-4F36-94F3-AACC0CBB2D59@omniti.com>
	<CA+U3Mf5=vUi+2sDhQ2qXNCLMjUtzMsi=CDcbJHA1gtTbCAEOjg@mail.gmail.com>
	<A3703B3C-8953-4CB1-BC11-7C898AB6F182@omniti.com>
Message-ID: <CA+U3Mf5Vm96oo=CBRWmWm94fquM3hYNEkVy895wXGdiQ5=QiwQ@mail.gmail.com>

On 14 July 2015 at 02:14, Dan McDonald <danmcd at omniti.com> wrote:
>
>> On Jul 13, 2015, at 5:11 PM, Sevan / Venture37 <venture37 at gmail.com> wrote:
>>
>> Where can I get the release number (the r151014) from the system? (I'm
>> currently unable to reach the OmniOS zone I'm using to check for
>> myself)
>
> /etc/release.  (Yes, this even works in zones.)

Release info should be reported correctly based on the info in
/etc/release from now on with the following commit.
http://cvsweb.netbsd.org/bsdweb.cgi/pkgsrc/mk/bsd.prefs.mk?rev=1.367&content-type=text/x-cvsweb-markup&only_with_tag=MAIN


Sevan / Venture37

From venture37 at gmail.com  Sat Jul 25 15:30:10 2015
From: venture37 at gmail.com (Sevan / Venture37)
Date: Sat, 25 Jul 2015 16:30:10 +0100
Subject: [OmniOS-discuss] pkgsrc-current OmniOS 170cea2/i386 2015-07-09
	21:35
In-Reply-To: <650A82F9-E07E-44CF-AE9C-4EAA4FFE87F5@omniti.com>
References: <CA+U3Mf6yA5VHgr5WdS9dhiEHs1epCmr22V_0KZ-S25soeWwnHw@mail.gmail.com>
	<811B3E16-0565-4F36-94F3-AACC0CBB2D59@omniti.com>
	<CA+U3Mf5=vUi+2sDhQ2qXNCLMjUtzMsi=CDcbJHA1gtTbCAEOjg@mail.gmail.com>
	<A3703B3C-8953-4CB1-BC11-7C898AB6F182@omniti.com>
	<CA+U3Mf5nyOH6sT0Utn0whQL3iNmCaGVS-=CGuU3VFxVKOX9JJw@mail.gmail.com>
	<650A82F9-E07E-44CF-AE9C-4EAA4FFE87F5@omniti.com>
Message-ID: <CA+U3Mf4dzgzoqziS41DXwJb9TsfYe+D-3J7RDmceNzECpvHwQw@mail.gmail.com>

On 19 July 2015 at 20:02, Dan McDonald <danmcd at omniti.com> wrote:
> I'd like to know (again if you'd already mentioned it) the precise nature of your workaround that allow more pkgsrc to be built.  You can, of course, just express it as a pull-request to omnios-build too, if you wish.

Appologies for the late response, I've not spent any time on finding a
solution to provide a pull request.
The temporary workaround that I made to my system, just to get over
the hurdle with gettext was to add the GCC lib directory to the ld
search path.

on r151014:

$ crle

Configuration file [version 4]: /var/ld/ld.config
  Platform:     32-bit LSB 80386
  Default Library Path (ELF):   /lib:/usr/lib:/opt/gcc-4.8.1/lib
  Trusted Directories (ELF):    /lib/secure:/usr/lib/secure  (system default)

Command line:
  crle -c /var/ld/ld.config -l /lib:/usr/lib:/opt/gcc-4.8.1/lib

From danmcd at omniti.com  Mon Jul 27 14:26:35 2015
From: danmcd at omniti.com (Dan McDonald)
Date: Mon, 27 Jul 2015 10:26:35 -0400
Subject: [OmniOS-discuss] pkgsrc-current OmniOS 170cea2/i386 2015-07-09
	21:35
In-Reply-To: <CA+U3Mf4dzgzoqziS41DXwJb9TsfYe+D-3J7RDmceNzECpvHwQw@mail.gmail.com>
References: <CA+U3Mf6yA5VHgr5WdS9dhiEHs1epCmr22V_0KZ-S25soeWwnHw@mail.gmail.com>
	<811B3E16-0565-4F36-94F3-AACC0CBB2D59@omniti.com>
	<CA+U3Mf5=vUi+2sDhQ2qXNCLMjUtzMsi=CDcbJHA1gtTbCAEOjg@mail.gmail.com>
	<A3703B3C-8953-4CB1-BC11-7C898AB6F182@omniti.com>
	<CA+U3Mf5nyOH6sT0Utn0whQL3iNmCaGVS-=CGuU3VFxVKOX9JJw@mail.gmail.com>
	<650A82F9-E07E-44CF-AE9C-4EAA4FFE87F5@omniti.com>
	<CA+U3Mf4dzgzoqziS41DXwJb9TsfYe+D-3J7RDmceNzECpvHwQw@mail.gmail.com>
Message-ID: <3CB02866-6686-4D80-869E-7D5ED0A6D088@omniti.com>


> On Jul 25, 2015, at 11:30 AM, Sevan / Venture37 <venture37 at gmail.com> wrote:
> 
> 
> The temporary workaround that I made to my system, just to get over
> the hurdle with gettext was to add the GCC lib directory to the ld
> search path.
> 
> on r151014:
> 
> $ crle
> 
> Configuration file [version 4]: /var/ld/ld.config
>  Platform:     32-bit LSB 80386
>  Default Library Path (ELF):   /lib:/usr/lib:/opt/gcc-4.8.1/lib


Ahhh yes.

We default to no /opt/gcc-* because of KYSTY (keep your sh*t to yourself).

Earlier you said:

> I wanted to evaluate the solution proposed previously in this thread
> that is to rebuild gettext with -rpath specified which includes the
> lib directory of the GCC with building with so that libgomp can be
> found.


Where is libgomp pulled from in gettext?  I checked all the binaries from the gnu-gettext package, and none seek libgomp, unless they do so by ldload().

Dan


From danmcd at omniti.com  Mon Jul 27 19:16:18 2015
From: danmcd at omniti.com (Dan McDonald)
Date: Mon, 27 Jul 2015 15:16:18 -0400
Subject: [OmniOS-discuss] Anyone try the new bloody ISO on OpenStack/KVM or
	Linux/KVM
Message-ID: <6FA64693-C9BD-4680-9922-48823EA34DA1@omniti.com>

Subject says it all.  I'm looking for additional testing datapoints if any are available above/beyond our own.  ESPECIALLY because of the vioif support now in bloody.

Thanks,
Dan


From mir at miras.org  Mon Jul 27 19:25:54 2015
From: mir at miras.org (Michael Rasmussen)
Date: Mon, 27 Jul 2015 21:25:54 +0200
Subject: [OmniOS-discuss] Anyone try the new bloody ISO on OpenStack/KVM
 or Linux/KVM
In-Reply-To: <6FA64693-C9BD-4680-9922-48823EA34DA1@omniti.com>
References: <6FA64693-C9BD-4680-9922-48823EA34DA1@omniti.com>
Message-ID: <20150727212554.26cd0c97@sleipner.datanom.net>

On Mon, 27 Jul 2015 15:16:18 -0400
Dan McDonald <danmcd at omniti.com> wrote:

> Subject says it all.  I'm looking for additional testing datapoints if any are available above/beyond our own.  ESPECIALLY because of the vioif support now in bloody.
> 
I am downloading now to test under Linux-kvm (proxmox).

Run basic network and vlan tests (Linux Bridge and Open VSwitch).

-- 
Hilsen/Regards
Michael Rasmussen

Get my public GnuPG keys:
michael <at> rasmussen <dot> cc
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E
mir <at> datanom <dot> net
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C
mir <at> miras <dot> org
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917
--------------------------------------------------------------
/usr/games/fortune -es says:
Vulcans worship peace above all.
		-- McCoy, "Return to Tomorrow", stardate 4768.3
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 181 bytes
Desc: OpenPGP digital signature
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150727/e98d2782/attachment-0001.bin>

From peter.tribble at gmail.com  Mon Jul 27 20:28:43 2015
From: peter.tribble at gmail.com (Peter Tribble)
Date: Mon, 27 Jul 2015 21:28:43 +0100
Subject: [OmniOS-discuss] Anyone try the new bloody ISO on OpenStack/KVM
 or Linux/KVM
In-Reply-To: <6FA64693-C9BD-4680-9922-48823EA34DA1@omniti.com>
References: <6FA64693-C9BD-4680-9922-48823EA34DA1@omniti.com>
Message-ID: <CAEgYsbHe=mfo8yPir+3dyTk6yiN6cA1Y9zJqze4ZBXkBO=Aqvg@mail.gmail.com>

Dan,

Subject says it all.  I'm looking for additional testing datapoints if any
> are available above/beyond our own.  ESPECIALLY because of the vioif
> support now in bloody.
>

I tried this on a UK supplier, Bytemark, who support installation of any OS
you can get running on KVM.

It's not entirely convincing.

Booting the iso with the VM in compatibility mode (so you see an ide drive
and
an e1000g0 network) works just fine.

Booting the iso with the VM in virtio mode hangs. The first time it got to
'Probing for device nodes ...' and hung for as long as I had patience.
(10-20
minutes or more, certainly much longer than you would expect.)

My second attempt I also get the hang, but do see a message from vioif.


[image: Inline image 1]


-- 
-Peter Tribble
http://www.petertribble.co.uk/ - http://ptribble.blogspot.com/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150727/9f16a28a/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: vioif2.png
Type: image/png
Size: 6783 bytes
Desc: not available
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150727/9f16a28a/attachment.png>

From danmcd at omniti.com  Mon Jul 27 20:35:02 2015
From: danmcd at omniti.com (Dan McDonald)
Date: Mon, 27 Jul 2015 16:35:02 -0400
Subject: [OmniOS-discuss] Anyone try the new bloody ISO on OpenStack/KVM
	or Linux/KVM
In-Reply-To: <CAEgYsbHe=mfo8yPir+3dyTk6yiN6cA1Y9zJqze4ZBXkBO=Aqvg@mail.gmail.com>
References: <6FA64693-C9BD-4680-9922-48823EA34DA1@omniti.com>
	<CAEgYsbHe=mfo8yPir+3dyTk6yiN6cA1Y9zJqze4ZBXkBO=Aqvg@mail.gmail.com>
Message-ID: <64943F3D-1647-4587-81EC-7E5272D85026@omniti.com>


> On Jul 27, 2015, at 4:28 PM, Peter Tribble <peter.tribble at gmail.com> wrote:
> 
> My second attempt I also get the hang, but do see a message from vioif.

The installer has problems detecting vioblk devices.  This is a Known Problem (TM).

The vioif detecting something, however, is encouraging.

Thanks!
Dan


From danmcd at omniti.com  Mon Jul 27 20:36:57 2015
From: danmcd at omniti.com (Dan McDonald)
Date: Mon, 27 Jul 2015 16:36:57 -0400
Subject: [OmniOS-discuss] Anyone try the new bloody ISO on OpenStack/KVM
	or Linux/KVM
In-Reply-To: <64943F3D-1647-4587-81EC-7E5272D85026@omniti.com>
References: <6FA64693-C9BD-4680-9922-48823EA34DA1@omniti.com>
	<CAEgYsbHe=mfo8yPir+3dyTk6yiN6cA1Y9zJqze4ZBXkBO=Aqvg@mail.gmail.com>
	<64943F3D-1647-4587-81EC-7E5272D85026@omniti.com>
Message-ID: <F3C032B9-3FBA-4C17-B4E2-73157862E9B3@omniti.com>


> On Jul 27, 2015, at 4:35 PM, Dan McDonald <danmcd at omniti.com> wrote:
> 
> 
> The installer has problems detecting vioblk devices.  This is a Known Problem (TM).
> 
> The vioif detecting something, however, is encouraging.

If I understand correctly, one can now:

	- Use vioif on an install

BUT

	- Still need to install onto an ide drive, then do the stupid ZFS tricks to get it over onto a vioblk boot drive.

So if you can boot with IDE, but with vioif, you should be able to move networking bits.

Dan


From mir at miras.org  Mon Jul 27 20:37:42 2015
From: mir at miras.org (Michael Rasmussen)
Date: Mon, 27 Jul 2015 22:37:42 +0200
Subject: [OmniOS-discuss] Anyone try the new bloody ISO on OpenStack/KVM
 or Linux/KVM
In-Reply-To: <6FA64693-C9BD-4680-9922-48823EA34DA1@omniti.com>
References: <6FA64693-C9BD-4680-9922-48823EA34DA1@omniti.com>
Message-ID: <20150727223742.53f364e0@sleipner.datanom.net>

On Mon, 27 Jul 2015 15:16:18 -0400
Dan McDonald <danmcd at omniti.com> wrote:

> Subject says it all.  I'm looking for additional testing datapoints if any are available above/beyond our own.  ESPECIALLY because of the vioif support now in bloody.
> 
First test using plain nics (eg. no vlan tagging) works flawlessly. See
attached screenshots.

Hypervisor running on RHEL 7.1 kernel (3.10). Later or tomorrow I will
try a Hypervisor running on Ubuntu-14.04.2-LTS (server 3.19)

PS. any date for seeing virtio-blk or virtio-scsi in Omnios? Only being
able to use emulated IDE or SATA sort off reduces performance.

-- 
Hilsen/Regards
Michael Rasmussen

Get my public GnuPG keys:
michael <at> rasmussen <dot> cc
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E
mir <at> datanom <dot> net
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C
mir <at> miras <dot> org
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917
--------------------------------------------------------------
/usr/games/fortune -es says:
Anti-trust laws should be approached with exactly that attitude.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: dladm_show-phys.png
Type: image/png
Size: 4643 bytes
Desc: not available
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150727/5f693008/attachment-0003.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ipadm_create-addr.png
Type: image/png
Size: 5146 bytes
Desc: not available
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150727/5f693008/attachment-0004.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: netstat_-rn_-finet.png
Type: image/png
Size: 3540 bytes
Desc: not available
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150727/5f693008/attachment-0005.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 181 bytes
Desc: OpenPGP digital signature
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150727/5f693008/attachment-0001.bin>

From danmcd at omniti.com  Mon Jul 27 20:46:15 2015
From: danmcd at omniti.com (Dan McDonald)
Date: Mon, 27 Jul 2015 16:46:15 -0400
Subject: [OmniOS-discuss] Anyone try the new bloody ISO on OpenStack/KVM
	or Linux/KVM
In-Reply-To: <20150727223742.53f364e0@sleipner.datanom.net>
References: <6FA64693-C9BD-4680-9922-48823EA34DA1@omniti.com>
	<20150727223742.53f364e0@sleipner.datanom.net>
Message-ID: <75D8FA0A-9E90-4FD8-9EF1-BE42C394F3FD@omniti.com>


> On Jul 27, 2015, at 4:37 PM, Michael Rasmussen <mir at miras.org> wrote:
> 
> PS. any date for seeing virtio-blk or virtio-scsi in Omnios? Only being
> able to use emulated IDE or SATA sort off reduces performance.

vioblk is already in.  The problem is, the goddamned installer (which needs some love) doesn't cope with any blkdev devices.  I mentioned this earlier in this thread.

Thanks for the reality checks, Michael!
Dan



From mir at miras.org  Mon Jul 27 20:59:43 2015
From: mir at miras.org (Michael Rasmussen)
Date: Mon, 27 Jul 2015 22:59:43 +0200
Subject: [OmniOS-discuss] Anyone try the new bloody ISO on OpenStack/KVM
 or Linux/KVM
In-Reply-To: <20150727223742.53f364e0@sleipner.datanom.net>
References: <6FA64693-C9BD-4680-9922-48823EA34DA1@omniti.com>
	<20150727223742.53f364e0@sleipner.datanom.net>
Message-ID: <20150727225943.0e9f16ed@sleipner.datanom.net>

On Mon, 27 Jul 2015 22:37:42 +0200
Michael Rasmussen <mir at miras.org> wrote:

> On Mon, 27 Jul 2015 15:16:18 -0400
> Dan McDonald <danmcd at omniti.com> wrote:
> 
> > Subject says it all.  I'm looking for additional testing datapoints if any are available above/beyond our own.  ESPECIALLY because of the vioif support now in bloody.
> > 
> 
> Hypervisor running on RHEL 7.1 kernel (3.10). Later or tomorrow I will
> try a Hypervisor running on Ubuntu-14.04.2-LTS (server 3.19)
> 
Next test using vlan tagging (same kernel and hypervisor) also works
flawlessly. See attached screenshot.

-- 
Hilsen/Regards
Michael Rasmussen

Get my public GnuPG keys:
michael <at> rasmussen <dot> cc
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E
mir <at> datanom <dot> net
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C
mir <at> miras <dot> org
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917
--------------------------------------------------------------
/usr/games/fortune -es says:
You'll always be,
What you always were,
Which has nothing to do with,
All to do, with her.
		-- Company
-------------- next part --------------
A non-text attachment was scrubbed...
Name: dladm_create-vlan.png
Type: image/png
Size: 6383 bytes
Desc: not available
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150727/2af9f27d/attachment.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 181 bytes
Desc: OpenPGP digital signature
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150727/2af9f27d/attachment.bin>

From danmcd at omniti.com  Mon Jul 27 21:04:02 2015
From: danmcd at omniti.com (Dan McDonald)
Date: Mon, 27 Jul 2015 17:04:02 -0400
Subject: [OmniOS-discuss] r151014 update --> vioif, KVM, and one NFS/RPC fix
Message-ID: <13D348EE-7649-47EB-90B1-61AB6A12E821@omniti.com>

Hello folks!

I've pushed out an update for r151014, our Long-Term Support release.  It's a reboot-your-system update, because to insure no mismatched modules, I rebuilt all of illumos-omnios.

New in this update are:

- illumos-omnios built from branch r151014, commit 7648372.

- vioif backported from bloody.

- KVM hosts with broken directed-EOI are now worked-around, for better vioif guest support.

- Flow control in rpcmod to allow NFS servers to better keep up with fast networks.  (Thanks to Nexenta for the original fix, and to U. of Toronto's Chris Siebenmann for field-testing on his own '014 boxes.)

- /opt/onbld/bin/onu now weakens the signature policy on the ONU-generated BE for easier local building.

- omnios-build built from branch r151014, commit b738989.

- Small updates to KVM.

- vioif now will be included by default in "entire" and in release media.

I did mention release media, didn't I?  The ISO, USB, and Kayak media are now updated as well.

These media include vioif in the boot media, so you can better install or migrate as a guest on OpenStack/KVM or Linux/KVM hosts (though you will, unfortunately, still have to deal with the installer's inability to detect vioblk drives).

Happy updating!
Dan


From mir at miras.org  Mon Jul 27 21:20:53 2015
From: mir at miras.org (Michael Rasmussen)
Date: Mon, 27 Jul 2015 23:20:53 +0200
Subject: [OmniOS-discuss] Anyone try the new bloody ISO on OpenStack/KVM
 or Linux/KVM
In-Reply-To: <75D8FA0A-9E90-4FD8-9EF1-BE42C394F3FD@omniti.com>
References: <6FA64693-C9BD-4680-9922-48823EA34DA1@omniti.com>
	<20150727223742.53f364e0@sleipner.datanom.net>
	<75D8FA0A-9E90-4FD8-9EF1-BE42C394F3FD@omniti.com>
Message-ID: <20150727232053.22e36626@sleipner.datanom.net>

On Mon, 27 Jul 2015 16:46:15 -0400
Dan McDonald <danmcd at omniti.com> wrote:

> 
> vioblk is already in.  The problem is, the goddamned installer (which needs some love) doesn't cope with any blkdev devices.  I mentioned this earlier in this thread.
> 
You mean the:
1) install on IDE/SATA
2) poweroff
3) Attach disk as virtio
4) boot from install CD and go to shell
5) zpool import -f rpool
6) zpool export rpool
7) reboot and enjoy;-)

-- 
Hilsen/Regards
Michael Rasmussen

Get my public GnuPG keys:
michael <at> rasmussen <dot> cc
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E
mir <at> datanom <dot> net
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C
mir <at> miras <dot> org
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917
--------------------------------------------------------------
/usr/games/fortune -es says:
interlard - vt., to intersperse; diversify
		-- Webster's New World Dictionary Of The American
Language
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 181 bytes
Desc: OpenPGP digital signature
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150727/1fc97e8b/attachment.bin>

From mir at miras.org  Mon Jul 27 22:13:10 2015
From: mir at miras.org (Michael Rasmussen)
Date: Tue, 28 Jul 2015 00:13:10 +0200
Subject: [OmniOS-discuss] Anyone try the new bloody ISO on OpenStack/KVM
 or Linux/KVM
In-Reply-To: <6FA64693-C9BD-4680-9922-48823EA34DA1@omniti.com>
References: <6FA64693-C9BD-4680-9922-48823EA34DA1@omniti.com>
Message-ID: <20150728001310.44595ad2@sleipner.datanom.net>

On Mon, 27 Jul 2015 15:16:18 -0400
Dan McDonald <danmcd at omniti.com> wrote:

> Subject says it all.  I'm looking for additional testing datapoints if any are available above/beyond our own.  ESPECIALLY because of the vioif support now in bloody.
> 
The speed is a bit disappointing though:
Client connecting to 192.168.3.154, TCP port 5001
TCP window size: 85.0 KByte (default)
------------------------------------------------------------
[  3] local 192.168.3.60 port 55395 connected with 192.168.3.154 port
5001 [ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec   749 MBytes   628 Mbits/sec

Anything needed to be tuned?

Compare this to a Debian Stable also running virtio-net:
------------------------------------------------------------
Client connecting to balder, TCP port 5001
TCP window size:  325 KByte (default)
------------------------------------------------------------
[  3] local 192.168.2.79 port 46249 connected with 192.168.2.2 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec  1.10 GBytes   941 Mbits/sec

-- 
Hilsen/Regards
Michael Rasmussen

Get my public GnuPG keys:
michael <at> rasmussen <dot> cc
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E
mir <at> datanom <dot> net
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C
mir <at> miras <dot> org
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917
--------------------------------------------------------------
/usr/games/fortune -es says:
<Davide> how bout a policy policing policy with a policy for changing
the police policing policy
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 181 bytes
Desc: OpenPGP digital signature
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150728/4a22d994/attachment-0001.bin>

From chip at innovates.com  Wed Jul 29 14:51:03 2015
From: chip at innovates.com (Schweiss, Chip)
Date: Wed, 29 Jul 2015 09:51:03 -0500
Subject: [OmniOS-discuss] Slow Drive Detection and boot-archive
In-Reply-To: <C1265D7C-AB4E-47E3-B0B5-C3FE8B578D1D@lji.org>
References: <201507210106.t6L16ee4022209@kyklops.ohsu.edu>
	<EF194F85-A03F-428E-A056-424B1D4AA23B@lji.org>
	<356582D1FC91784992ABB4265A16ED489AABAAC8@vEX01.mindstorm-internet.local>
	<C1265D7C-AB4E-47E3-B0B5-C3FE8B578D1D@lji.org>
Message-ID: <CALeZrrSTovc2ixmNfvrfh_YsK9KEwpqPDj_2zSM+3FC+=kh+Tg@mail.gmail.com>

On Fri, Jul 24, 2015 at 5:03 PM, Michael Talbott <mtalbott at lji.org> wrote:

> Hi,
>
> I've downgraded the cards (LSI 9211-8e) to v.19 and disabled their boot
> bios. But I'm still getting the 8 second per drive delay after the kernel
> loads. Any other ideas?
>
>
8 seconds is way too long.   What JBODs and disks are you using?   Could it
be they are powered off and the delay in waiting for the power on command
to complete?   This could be accelerated by using lsiutils to send them all
power on commands first.

While I still consider it slow, however, my OmniOS systems with  LSI HBAs
discover about 2 disks per second.   With systems with LOTS of disk all
multipathed it still stacks up to a long time to discover them all.

-Chip


>
> ________________________
> Michael Talbott
> Systems Administrator
> La Jolla Institute
>
> > On Jul 20, 2015, at 11:27 PM, Floris van Essen ..:: House of Ancients
> Amstafs ::.. <info at houseofancients.nl> wrote:
> >
> > Michael,
> >
> > I know v20 does cause lots of issue's.
> > V19 , to the best of my knowledge doesn't contain any, so I would
> downgrade to v19
> >
> >
> > Kr,
> >
> >
> > Floris
> > -----Oorspronkelijk bericht-----
> > Van: OmniOS-discuss [mailto:omnios-discuss-bounces at lists.omniti.com]
> Namens Michael Talbott
> > Verzonden: dinsdag 21 juli 2015 4:57
> > Aan: Marion Hakanson <hakansom at ohsu.edu>
> > CC: omnios-discuss <omnios-discuss at lists.omniti.com>
> > Onderwerp: Re: [OmniOS-discuss] Slow Drive Detection and boot-archive
> >
> > Thanks for the reply. The bios for the card is disabled already. The 8
> second per drive scan happens after the kernel has already loaded and it is
> scanning for devices. I wonder if it's due to running newer firmware. I did
> update the cards to fw v.20.something before I moved to omnios. Is there a
> particular firmware version on the cards I should run to match OmniOS's
> drivers?
> >
> >
> > ________________________
> > Michael Talbott
> > Systems Administrator
> > La Jolla Institute
> >
> >> On Jul 20, 2015, at 6:06 PM, Marion Hakanson <hakansom at ohsu.edu> wrote:
> >>
> >> Michael,
> >>
> >> I've not seen this;  I do have one system with 120 drives and it
> >> definitely does not have this problem.  A couple with 80+ drives are
> >> also free of this issue, though they are still running OpenIndiana.
> >>
> >> One thing I pretty much always do here, is to disable the boot option
> >> in the LSI HBA's config utility (accessible from the during boot after
> >> the BIOS has started up).  I do this because I don't want the BIOS
> >> thinking it can boot from any of the external JBOD disks;  And also
> >> because I've had some system BIOS crashes when they tried to enumerate
> >> too many drives.  But, this all happens at the BIOS level, before the
> >> OS has even started up, so in theory it should not affect what you are
> >> seeing.
> >>
> >> Regards,
> >>
> >> Marion
> >>
> >>
> >> ================================================================
> >> Subject: Re: [OmniOS-discuss] Slow Drive Detection and boot-archive
> >> From: Michael Talbott <mtalbott at lji.org>
> >> Date: Fri, 17 Jul 2015 16:15:47 -0700
> >> To: omnios-discuss <omnios-discuss at lists.omniti.com>
> >>
> >> Just realized my typo. I'm using this on my 90 and 180 drive systems:
> >>
> >> # svccfg -s boot-archive setprop start/timeout_seconds=720 # svccfg -s
> >> boot-archive setprop start/timeout_seconds=1440
> >>
> >> Seems like 8 seconds to detect each drive is pretty excessive.
> >>
> >> Any ideas on how to speed that up?
> >>
> >>
> >> ________________________
> >> Michael Talbott
> >> Systems Administrator
> >> La Jolla Institute
> >>
> >>> On Jul 17, 2015, at 4:07 PM, Michael Talbott <mtalbott at lji.org> wrote:
> >>>
> >>> I have multiple NAS servers I've moved to OmniOS and each of them have
> 90-180 4T disks. Everything has worked out pretty well for the most part.
> But I've come into an issue where when I reboot any of them, I'm getting
> boot-archive service timeouts happening. I found a workaround of increasing
> the timeout value which brings me to the following. As you can see below in
> a dmesg output, it's taking the kernel about 8 seconds to detect each of
> the drives. They're connected via a couple SAS2008 based LSI cards.
> >>>
> >>> Is this normal?
> >>> Is there a way to speed that up?
> >>>
> >>> I've fixed my frustrating boot-archive timeout problem by adjusting
> the timeout value from the default of 60 seconds (I guess that'll work ok
> on systems with less than 8 drives?) to 8 seconds * 90 drives + a little
> extra time = 280 seconds (for the 90 drive systems). Which means it takes
> between 12-24 minutes to boot those machines up.
> >>>
> >>> # svccfg -s boot-archive setprop start/timeout_seconds=280
> >>>
> >>> I figure I can't be the only one. A little googling also revealed:
> >>> https://www.illumos.org/issues/4614
> >>> <https://www.illumos.org/issues/4614>
> >>>
> >>> Jul 17 15:40:15 store2 genunix: [ID 583861 kern.info] sd29 at
> >>> mpt_sas3: unit-address w50000c0f0401bd43,0: w50000c0f0401bd43,0 Jul
> >>> 17 15:40:15 store2 genunix: [ID 936769 kern.info] sd29 is
> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0401bd4
> >>> 3,0 Jul 17 15:40:16 store2 genunix: [ID 408114 kern.info]
> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0401bd4
> >>> 3,0 (sd29) online Jul 17 15:40:24 store2 genunix: [ID 583861
> >>> kern.info] sd30 at mpt_sas3: unit-address w50000c0f045679c3,0:
> >>> w50000c0f045679c3,0 Jul 17 15:40:24 store2 genunix: [ID 936769
> >>> kern.info] sd30 is
> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f045679c
> >>> 3,0 Jul 17 15:40:24 store2 genunix: [ID 408114 kern.info]
> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f045679c
> >>> 3,0 (sd30) online Jul 17 15:40:33 store2 genunix: [ID 583861
> >>> kern.info] sd31 at mpt_sas3: unit-address w50000c0f045712b3,0:
> >>> w50000c0f045712b3,0 Jul 17 15:40:33 store2 genunix: [ID 936769
> >>> kern.info] sd31 is
> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f045712b
> >>> 3,0 Jul 17 15:40:33 store2 genunix: [ID 408114 kern.info]
> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f045712b
> >>> 3,0 (sd31) online Jul 17 15:40:42 store2 genunix: [ID 583861
> >>> kern.info] sd32 at mpt_sas3: unit-address w50000c0f04571497,0:
> >>> w50000c0f04571497,0 Jul 17 15:40:42 store2 genunix: [ID 936769
> >>> kern.info] sd32 is
> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0457149
> >>> 7,0 Jul 17 15:40:42 store2 genunix: [ID 408114 kern.info]
> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0457149
> >>> 7,0 (sd32) online Jul 17 15:40:50 store2 genunix: [ID 583861
> >>> kern.info] sd33 at mpt_sas3: unit-address w50000c0f042ac8eb,0:
> >>> w50000c0f042ac8eb,0 Jul 17 15:40:50 store2 genunix: [ID 936769
> >>> kern.info] sd33 is
> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f042ac8e
> >>> b,0 Jul 17 15:40:50 store2 genunix: [ID 408114 kern.info]
> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f042ac8e
> >>> b,0 (sd33) online Jul 17 15:40:59 store2 genunix: [ID 583861
> >>> kern.info] sd34 at mpt_sas3: unit-address w50000c0f04571473,0:
> >>> w50000c0f04571473,0 Jul 17 15:40:59 store2 genunix: [ID 936769
> >>> kern.info] sd34 is
> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0457147
> >>> 3,0 Jul 17 15:40:59 store2 genunix: [ID 408114 kern.info]
> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0457147
> >>> 3,0 (sd34) online Jul 17 15:41:08 store2 genunix: [ID 583861
> >>> kern.info] sd35 at mpt_sas3: unit-address w50000c0f042c636f,0:
> >>> w50000c0f042c636f,0 Jul 17 15:41:08 store2 genunix: [ID 936769
> >>> kern.info] sd35 is
> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f042c636
> >>> f,0 Jul 17 15:41:08 store2 genunix: [ID 408114 kern.info]
> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f042c636
> >>> f,0 (sd35) online Jul 17 15:41:17 store2 genunix: [ID 583861
> >>> kern.info] sd36 at mpt_sas3: unit-address w50000c0f0401bf2f,0:
> >>> w50000c0f0401bf2f,0 Jul 17 15:41:17 store2 genunix: [ID 936769
> >>> kern.info] sd36 is
> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0401bf2
> >>> f,0 Jul 17 15:41:17 store2 genunix: [ID 408114 kern.info]
> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0401bf2
> >>> f,0 (sd36) online Jul 17 15:41:25 store2 genunix: [ID 583861
> >>> kern.info] sd38 at mpt_sas3: unit-address w50000c0f0401bc1f,0:
> >>> w50000c0f0401bc1f,0 Jul 17 15:41:25 store2 genunix: [ID 936769
> >>> kern.info] sd38 is
> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0401bc1
> >>> f,0 Jul 17 15:41:26 store2 genunix: [ID 408114 kern.info]
> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0401bc1
> >>> f,0 (sd38) online
> >>>
> >>>
> >>> ________________________
> >>> Michael Talbott
> >>> Systems Administrator
> >>> La Jolla Institute
> >>>
> >>
> >> _______________________________________________
> >> OmniOS-discuss mailing list
> >> OmniOS-discuss at lists.omniti.com
> >> http://lists.omniti.com/mailman/listinfo/omnios-discuss
> >>
> >>
> >
> > _______________________________________________
> > OmniOS-discuss mailing list
> > OmniOS-discuss at lists.omniti.com
> > http://lists.omniti.com/mailman/listinfo/omnios-discuss
> > ...:: House of Ancients ::...
> > American Staffordshire Terriers
> >
> > +31-628-161-350
> > +31-614-198-389
> > Het Perk 48
> > 4903 RB
> > Oosterhout
> > Netherlands
> > www.houseofancients.nl
>
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150729/d9ec3699/attachment.html>

From sergey57 at gmail.com  Wed Jul 29 17:40:28 2015
From: sergey57 at gmail.com (sergey ivanov)
Date: Wed, 29 Jul 2015 13:40:28 -0400
Subject: [OmniOS-discuss] question about service firewall rules
Message-ID: <CAAv=BDGzLvpPqGdzfEX5CSwZ4Qwbowtj46k1Npfy=Bwa0NhPUw@mail.gmail.com>

Hi,
I want to restrict ssh logins to my OmniOS boxes to particular subnets.
I am trying to do the following:
---
# svccfg -s ssh setprop firewall_config/policy = astring: allow
# svccfg -s ssh setprop firewall_config/apply_to = astring:
network:192.168.1.0/24
# svccfg -s ssh setprop firewall_config/apply_to = astring:  host:128.8.128.117
# svcadm refresh ssh
# svcadm refresh ipfilter
---
It works, but when I want to restart service ssh, it goes to
maintenance mode with log lines in
/var/svc/log/network-ipfilter\:default.log telling:
---
[ Wed Jul 29 15:11:14 UTC 2015 /lib/svc/method/ipfilter:
svc:/network/ssh:default has invalid ipf configuration. ]
[ Wed Jul 29 15:11:14 UTC 2015 /lib/svc/method/ipfilter: placing
svc:/network/ssh:default in maintenance. ]
---
Everything returns to working mode by disabling both ssh and ipfilter
services and reenabling them. Is it known problem, or I am doing
something wrong?
-- 
Regards,
Sergey Ivanov | sergey57 at gmail.com
http://www.linkedin.com/pub/sergey-ivanov/8/270/a09

From mtalbott at lji.org  Wed Jul 29 17:48:35 2015
From: mtalbott at lji.org (Michael Talbott)
Date: Wed, 29 Jul 2015 10:48:35 -0700
Subject: [OmniOS-discuss] Slow Drive Detection and boot-archive
In-Reply-To: <CALeZrrSTovc2ixmNfvrfh_YsK9KEwpqPDj_2zSM+3FC+=kh+Tg@mail.gmail.com>
References: <201507210106.t6L16ee4022209@kyklops.ohsu.edu>
	<EF194F85-A03F-428E-A056-424B1D4AA23B@lji.org>
	<356582D1FC91784992ABB4265A16ED489AABAAC8@vEX01.mindstorm-internet.local>
	<C1265D7C-AB4E-47E3-B0B5-C3FE8B578D1D@lji.org>
	<CALeZrrSTovc2ixmNfvrfh_YsK9KEwpqPDj_2zSM+3FC+=kh+Tg@mail.gmail.com>
Message-ID: <303EF982-24E7-4E61-9B78-7D40B77A3530@lji.org>

Here's the specs of that server.

Fujitsu RX300S8
 - http://www.fujitsu.com/fts/products/computing/servers/primergy/rack/rx300/ <http://www.fujitsu.com/fts/products/computing/servers/primergy/rack/rx300/>
128G ECC DDR3 1600 RAM
2 x Intel(R) Xeon(R) CPU E5-2630 v2 @ 2.60GHz
2 x LSI 9200-8e
2 x 10Gb Intel NICs
2 x SuperMicro 847E26-RJBOD1 45 bay JBOD enclosures
 - http://www.supermicro.com/products/chassis/4U/847/SC847E26-RJBOD1.cfm <http://www.supermicro.com/products/chassis/4U/847/SC847E26-RJBOD1.cfm>

The enclosures are not currently set up for multipathing. The front and rear backplane each have a single independent SAS connection to one of the LSI 9200s.

The two enclosures are fully loaded with 45 x 4TB WD4001FYYG-01SL3 drives each (90 total).
http://www.newegg.com/Product/Product.aspx?Item=N82E16822236353 <http://www.newegg.com/Product/Product.aspx?Item=N82E16822236353>

Booting the server up in Ubuntu or CentOS does not have that 8 second delay. Each drive is found in a fraction of a second (activity LEDs on the enclosure flash on and off really quick as the drives are scanned). On OmniOS, the drives seem to be scanned in the same order, but, instead of it spending a fraction of a second on each drive, it spends 8 seconds on 1 drive (led of only one drive rapidly flashing during that process) before moving on to the next x 90 drives.

Is there anything I can do to get more verbosity in the boot messages that might just reveal the root issue?

Any suggestions appreciated.

Thanks

________________________
Michael Talbott
Systems Administrator
La Jolla Institute

> On Jul 29, 2015, at 7:51 AM, Schweiss, Chip <chip at innovates.com> wrote:
> 
> 
> 
> On Fri, Jul 24, 2015 at 5:03 PM, Michael Talbott <mtalbott at lji.org <mailto:mtalbott at lji.org>> wrote:
> Hi,
> 
> I've downgraded the cards (LSI 9211-8e) to v.19 and disabled their boot bios. But I'm still getting the 8 second per drive delay after the kernel loads. Any other ideas?
> 
> 
> 8 seconds is way too long.   What JBODs and disks are you using?   Could it be they are powered off and the delay in waiting for the power on command to complete?   This could be accelerated by using lsiutils to send them all power on commands first.
> 
> While I still consider it slow, however, my OmniOS systems with  LSI HBAs discover about 2 disks per second.   With systems with LOTS of disk all multipathed it still stacks up to a long time to discover them all.
> 
> -Chip
>  
> 
> ________________________
> Michael Talbott
> Systems Administrator
> La Jolla Institute
> 
> > On Jul 20, 2015, at 11:27 PM, Floris van Essen ..:: House of Ancients Amstafs ::.. <info at houseofancients.nl <mailto:info at houseofancients.nl>> wrote:
> >
> > Michael,
> >
> > I know v20 does cause lots of issue's.
> > V19 , to the best of my knowledge doesn't contain any, so I would downgrade to v19
> >
> >
> > Kr,
> >
> >
> > Floris
> > -----Oorspronkelijk bericht-----
> > Van: OmniOS-discuss [mailto:omnios-discuss-bounces at lists.omniti.com <mailto:omnios-discuss-bounces at lists.omniti.com>] Namens Michael Talbott
> > Verzonden: dinsdag 21 juli 2015 4:57
> > Aan: Marion Hakanson <hakansom at ohsu.edu <mailto:hakansom at ohsu.edu>>
> > CC: omnios-discuss <omnios-discuss at lists.omniti.com <mailto:omnios-discuss at lists.omniti.com>>
> > Onderwerp: Re: [OmniOS-discuss] Slow Drive Detection and boot-archive
> >
> > Thanks for the reply. The bios for the card is disabled already. The 8 second per drive scan happens after the kernel has already loaded and it is scanning for devices. I wonder if it's due to running newer firmware. I did update the cards to fw v.20.something before I moved to omnios. Is there a particular firmware version on the cards I should run to match OmniOS's drivers?
> >
> >
> > ________________________
> > Michael Talbott
> > Systems Administrator
> > La Jolla Institute
> >
> >> On Jul 20, 2015, at 6:06 PM, Marion Hakanson <hakansom at ohsu.edu <mailto:hakansom at ohsu.edu>> wrote:
> >>
> >> Michael,
> >>
> >> I've not seen this;  I do have one system with 120 drives and it
> >> definitely does not have this problem.  A couple with 80+ drives are
> >> also free of this issue, though they are still running OpenIndiana.
> >>
> >> One thing I pretty much always do here, is to disable the boot option
> >> in the LSI HBA's config utility (accessible from the during boot after
> >> the BIOS has started up).  I do this because I don't want the BIOS
> >> thinking it can boot from any of the external JBOD disks;  And also
> >> because I've had some system BIOS crashes when they tried to enumerate
> >> too many drives.  But, this all happens at the BIOS level, before the
> >> OS has even started up, so in theory it should not affect what you are
> >> seeing.
> >>
> >> Regards,
> >>
> >> Marion
> >>
> >>
> >> ================================================================
> >> Subject: Re: [OmniOS-discuss] Slow Drive Detection and boot-archive
> >> From: Michael Talbott <mtalbott at lji.org <mailto:mtalbott at lji.org>>
> >> Date: Fri, 17 Jul 2015 16:15:47 -0700
> >> To: omnios-discuss <omnios-discuss at lists.omniti.com <mailto:omnios-discuss at lists.omniti.com>>
> >>
> >> Just realized my typo. I'm using this on my 90 and 180 drive systems:
> >>
> >> # svccfg -s boot-archive setprop start/timeout_seconds=720 # svccfg -s
> >> boot-archive setprop start/timeout_seconds=1440
> >>
> >> Seems like 8 seconds to detect each drive is pretty excessive.
> >>
> >> Any ideas on how to speed that up?
> >>
> >>
> >> ________________________
> >> Michael Talbott
> >> Systems Administrator
> >> La Jolla Institute
> >>
> >>> On Jul 17, 2015, at 4:07 PM, Michael Talbott <mtalbott at lji.org <mailto:mtalbott at lji.org>> wrote:
> >>>
> >>> I have multiple NAS servers I've moved to OmniOS and each of them have 90-180 4T disks. Everything has worked out pretty well for the most part. But I've come into an issue where when I reboot any of them, I'm getting boot-archive service timeouts happening. I found a workaround of increasing the timeout value which brings me to the following. As you can see below in a dmesg output, it's taking the kernel about 8 seconds to detect each of the drives. They're connected via a couple SAS2008 based LSI cards.
> >>>
> >>> Is this normal?
> >>> Is there a way to speed that up?
> >>>
> >>> I've fixed my frustrating boot-archive timeout problem by adjusting the timeout value from the default of 60 seconds (I guess that'll work ok on systems with less than 8 drives?) to 8 seconds * 90 drives + a little extra time = 280 seconds (for the 90 drive systems). Which means it takes between 12-24 minutes to boot those machines up.
> >>>
> >>> # svccfg -s boot-archive setprop start/timeout_seconds=280
> >>>
> >>> I figure I can't be the only one. A little googling also revealed:
> >>> https://www.illumos.org/issues/4614 <https://www.illumos.org/issues/4614>
> >>> <https://www.illumos.org/issues/4614 <https://www.illumos.org/issues/4614>>
> >>>
> >>> Jul 17 15:40:15 store2 genunix: [ID 583861 kern.info <http://kern.info/>] sd29 at
> >>> mpt_sas3: unit-address w50000c0f0401bd43,0: w50000c0f0401bd43,0 Jul
> >>> 17 15:40:15 store2 genunix: [ID 936769 kern.info <http://kern.info/>] sd29 is
> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0401bd4
> >>> 3,0 Jul 17 15:40:16 store2 genunix: [ID 408114 kern.info <http://kern.info/>]
> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0401bd4
> >>> 3,0 (sd29) online Jul 17 15:40:24 store2 genunix: [ID 583861
> >>> kern.info <http://kern.info/>] sd30 at mpt_sas3: unit-address w50000c0f045679c3,0:
> >>> w50000c0f045679c3,0 Jul 17 15:40:24 store2 genunix: [ID 936769
> >>> kern.info <http://kern.info/>] sd30 is
> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f045679c
> >>> 3,0 Jul 17 15:40:24 store2 genunix: [ID 408114 kern.info <http://kern.info/>]
> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f045679c
> >>> 3,0 (sd30) online Jul 17 15:40:33 store2 genunix: [ID 583861
> >>> kern.info <http://kern.info/>] sd31 at mpt_sas3: unit-address w50000c0f045712b3,0:
> >>> w50000c0f045712b3,0 Jul 17 15:40:33 store2 genunix: [ID 936769
> >>> kern.info <http://kern.info/>] sd31 is
> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f045712b
> >>> 3,0 Jul 17 15:40:33 store2 genunix: [ID 408114 kern.info <http://kern.info/>]
> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f045712b
> >>> 3,0 (sd31) online Jul 17 15:40:42 store2 genunix: [ID 583861
> >>> kern.info <http://kern.info/>] sd32 at mpt_sas3: unit-address w50000c0f04571497,0:
> >>> w50000c0f04571497,0 Jul 17 15:40:42 store2 genunix: [ID 936769
> >>> kern.info <http://kern.info/>] sd32 is
> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0457149
> >>> 7,0 Jul 17 15:40:42 store2 genunix: [ID 408114 kern.info <http://kern.info/>]
> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0457149
> >>> 7,0 (sd32) online Jul 17 15:40:50 store2 genunix: [ID 583861
> >>> kern.info <http://kern.info/>] sd33 at mpt_sas3: unit-address w50000c0f042ac8eb,0:
> >>> w50000c0f042ac8eb,0 Jul 17 15:40:50 store2 genunix: [ID 936769
> >>> kern.info <http://kern.info/>] sd33 is
> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f042ac8e
> >>> b,0 Jul 17 15:40:50 store2 genunix: [ID 408114 kern.info <http://kern.info/>]
> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f042ac8e
> >>> b,0 (sd33) online Jul 17 15:40:59 store2 genunix: [ID 583861
> >>> kern.info <http://kern.info/>] sd34 at mpt_sas3: unit-address w50000c0f04571473,0:
> >>> w50000c0f04571473,0 Jul 17 15:40:59 store2 genunix: [ID 936769
> >>> kern.info <http://kern.info/>] sd34 is
> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0457147
> >>> 3,0 Jul 17 15:40:59 store2 genunix: [ID 408114 kern.info <http://kern.info/>]
> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0457147
> >>> 3,0 (sd34) online Jul 17 15:41:08 store2 genunix: [ID 583861
> >>> kern.info <http://kern.info/>] sd35 at mpt_sas3: unit-address w50000c0f042c636f,0:
> >>> w50000c0f042c636f,0 Jul 17 15:41:08 store2 genunix: [ID 936769
> >>> kern.info <http://kern.info/>] sd35 is
> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f042c636
> >>> f,0 Jul 17 15:41:08 store2 genunix: [ID 408114 kern.info <http://kern.info/>]
> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f042c636
> >>> f,0 (sd35) online Jul 17 15:41:17 store2 genunix: [ID 583861
> >>> kern.info <http://kern.info/>] sd36 at mpt_sas3: unit-address w50000c0f0401bf2f,0:
> >>> w50000c0f0401bf2f,0 Jul 17 15:41:17 store2 genunix: [ID 936769
> >>> kern.info <http://kern.info/>] sd36 is
> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0401bf2
> >>> f,0 Jul 17 15:41:17 store2 genunix: [ID 408114 kern.info <http://kern.info/>]
> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0401bf2
> >>> f,0 (sd36) online Jul 17 15:41:25 store2 genunix: [ID 583861
> >>> kern.info <http://kern.info/>] sd38 at mpt_sas3: unit-address w50000c0f0401bc1f,0:
> >>> w50000c0f0401bc1f,0 Jul 17 15:41:25 store2 genunix: [ID 936769
> >>> kern.info <http://kern.info/>] sd38 is
> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0401bc1
> >>> f,0 Jul 17 15:41:26 store2 genunix: [ID 408114 kern.info <http://kern.info/>]
> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0401bc1
> >>> f,0 (sd38) online
> >>>
> >>>
> >>> ________________________
> >>> Michael Talbott
> >>> Systems Administrator
> >>> La Jolla Institute
> >>>
> >>
> >> _______________________________________________
> >> OmniOS-discuss mailing list
> >> OmniOS-discuss at lists.omniti.com <mailto:OmniOS-discuss at lists.omniti.com>
> >> http://lists.omniti.com/mailman/listinfo/omnios-discuss <http://lists.omniti.com/mailman/listinfo/omnios-discuss>
> >>
> >>
> >
> > _______________________________________________
> > OmniOS-discuss mailing list
> > OmniOS-discuss at lists.omniti.com <mailto:OmniOS-discuss at lists.omniti.com>
> > http://lists.omniti.com/mailman/listinfo/omnios-discuss <http://lists.omniti.com/mailman/listinfo/omnios-discuss>
> > ...:: House of Ancients ::...
> > American Staffordshire Terriers
> >
> > +31-628-161-350 <tel:%2B31-628-161-350>
> > +31-614-198-389 <tel:%2B31-614-198-389>
> > Het Perk 48
> > 4903 RB
> > Oosterhout
> > Netherlands
> > www.houseofancients.nl <http://www.houseofancients.nl/>
> 
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com <mailto:OmniOS-discuss at lists.omniti.com>
> http://lists.omniti.com/mailman/listinfo/omnios-discuss <http://lists.omniti.com/mailman/listinfo/omnios-discuss>
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150729/1d39014d/attachment-0001.html>

From mtalbott at lji.org  Wed Jul 29 17:55:46 2015
From: mtalbott at lji.org (Michael Talbott)
Date: Wed, 29 Jul 2015 10:55:46 -0700
Subject: [OmniOS-discuss] question about service firewall rules
In-Reply-To: <CAAv=BDGzLvpPqGdzfEX5CSwZ4Qwbowtj46k1Npfy=Bwa0NhPUw@mail.gmail.com>
References: <CAAv=BDGzLvpPqGdzfEX5CSwZ4Qwbowtj46k1Npfy=Bwa0NhPUw@mail.gmail.com>
Message-ID: <00647C02-5815-46D3-9818-488608D14DE4@lji.org>

Hi,

Granted this won't fix the firewall issue, but, I think you can get the results your looking for just by editing /etc/ssh/sshd_config and use "AllowUsers username at 192.168.1.0/24"

From sshd_config's man page:

       AllowUsers
           This keyword can be followed by a number of user names, separated
           by spaces. If specified, login is allowed only for user names that
           match one of the patterns.  Asterisk (*) and question mark (?) can
           be used as wildcards in the patterns. Only user names are valid; a
           numerical user ID is not recognized. By default login is allowed
           regardless of the user name.

           If a specified pattern takes the form user at host then user and host
           are checked separately, restricting logins to particular users from
           particular hosts.


________________________
Michael Talbott
Systems Administrator
La Jolla Institute

> On Jul 29, 2015, at 10:40 AM, sergey ivanov <sergey57 at gmail.com> wrote:
> 
> Hi,
> I want to restrict ssh logins to my OmniOS boxes to particular subnets.
> I am trying to do the following:
> ---
> # svccfg -s ssh setprop firewall_config/policy = astring: allow
> # svccfg -s ssh setprop firewall_config/apply_to = astring:
> network:192.168.1.0/24
> # svccfg -s ssh setprop firewall_config/apply_to = astring:  host:128.8.128.117
> # svcadm refresh ssh
> # svcadm refresh ipfilter
> ---
> It works, but when I want to restart service ssh, it goes to
> maintenance mode with log lines in
> /var/svc/log/network-ipfilter\:default.log telling:
> ---
> [ Wed Jul 29 15:11:14 UTC 2015 /lib/svc/method/ipfilter:
> svc:/network/ssh:default has invalid ipf configuration. ]
> [ Wed Jul 29 15:11:14 UTC 2015 /lib/svc/method/ipfilter: placing
> svc:/network/ssh:default in maintenance. ]
> ---
> Everything returns to working mode by disabling both ssh and ipfilter
> services and reenabling them. Is it known problem, or I am doing
> something wrong?
> -- 
> Regards,
> Sergey Ivanov | sergey57 at gmail.com
> http://www.linkedin.com/pub/sergey-ivanov/8/270/a09
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss


From KBruene at simmonsperrine.com  Wed Jul 29 18:54:23 2015
From: KBruene at simmonsperrine.com (Kyle Bruene)
Date: Wed, 29 Jul 2015 18:54:23 +0000
Subject: [OmniOS-discuss] LX Branded Zones
Message-ID: <202C92988C5CF249BD3F9F21B2B199CB33EFE69F@SPMAIL1.spae.local>

Are there any plans to make LX branded zones available in OmniOS, or will this remain a SmartOS-only Illumos feature?


From danmcd at omniti.com  Wed Jul 29 19:09:12 2015
From: danmcd at omniti.com (Dan McDonald)
Date: Wed, 29 Jul 2015 15:09:12 -0400
Subject: [OmniOS-discuss] LX Branded Zones
In-Reply-To: <202C92988C5CF249BD3F9F21B2B199CB33EFE69F@SPMAIL1.spae.local>
References: <202C92988C5CF249BD3F9F21B2B199CB33EFE69F@SPMAIL1.spae.local>
Message-ID: <CF4D34D8-5AE7-4BFA-904C-F66146445FAD@omniti.com>


> On Jul 29, 2015, at 2:54 PM, Kyle Bruene <KBruene at simmonsperrine.com> wrote:
> 
> Are there any plans to make LX branded zones available in OmniOS, or will this remain a SmartOS-only Illumos feature?

There are plans.  Not sure of all of their details yet, but there are plans.  I can only say with certainty that they will NOT be in r151016 (the next stable).

The largest consideration is the admin model.  We don't have vmadm(1M) in OmniOS, and to do so would mean porting over large chunks of SmartOS and/or SmartDC, which I'd rather not do.  So we need to consider what a zonecfg(1M) model for LX zone would look like.  Admins in the audience should think about this as well.

Dan


From chip at innovates.com  Wed Jul 29 20:50:36 2015
From: chip at innovates.com (Schweiss, Chip)
Date: Wed, 29 Jul 2015 15:50:36 -0500
Subject: [OmniOS-discuss] Slow Drive Detection and boot-archive
In-Reply-To: <303EF982-24E7-4E61-9B78-7D40B77A3530@lji.org>
References: <201507210106.t6L16ee4022209@kyklops.ohsu.edu>
	<EF194F85-A03F-428E-A056-424B1D4AA23B@lji.org>
	<356582D1FC91784992ABB4265A16ED489AABAAC8@vEX01.mindstorm-internet.local>
	<C1265D7C-AB4E-47E3-B0B5-C3FE8B578D1D@lji.org>
	<CALeZrrSTovc2ixmNfvrfh_YsK9KEwpqPDj_2zSM+3FC+=kh+Tg@mail.gmail.com>
	<303EF982-24E7-4E61-9B78-7D40B77A3530@lji.org>
Message-ID: <CALeZrrR2k-pBwyTp6pe5USEDPbTPzu2Fnxs=yivh44c5+GrVzg@mail.gmail.com>

I have an OmniOS box with all the same hardware except the server and hard
disks.  I would wager this something to do with the WD disks and something
different happening in the init.

This is a stab in the dark, but try adding "power-condition:false" in
/kernel/drv/sd.conf for the WD disks.

-Chip



On Wed, Jul 29, 2015 at 12:48 PM, Michael Talbott <mtalbott at lji.org> wrote:

> Here's the specs of that server.
>
> Fujitsu RX300S8
>  -
> http://www.fujitsu.com/fts/products/computing/servers/primergy/rack/rx300/
> 128G ECC DDR3 1600 RAM
> 2 x Intel(R) Xeon(R) CPU E5-2630 v2 @ 2.60GHz
> 2 x LSI 9200-8e
> 2 x 10Gb Intel NICs
> 2 x SuperMicro 847E26-RJBOD1 45 bay JBOD enclosures
>  - http://www.supermicro.com/products/chassis/4U/847/SC847E26-RJBOD1.cfm
>
> The enclosures are not currently set up for multipathing. The front and
> rear backplane each have a single independent SAS connection to one of the
> LSI 9200s.
>
> The two enclosures are fully loaded with 45 x 4TB WD4001FYYG-01SL3 drives
> each (90 total).
> http://www.newegg.com/Product/Product.aspx?Item=N82E16822236353
>
> Booting the server up in Ubuntu or CentOS does not have that 8 second
> delay. Each drive is found in a fraction of a second (activity LEDs on the
> enclosure flash on and off really quick as the drives are scanned). On
> OmniOS, the drives seem to be scanned in the same order, but, instead of it
> spending a fraction of a second on each drive, it spends 8 seconds on 1
> drive (led of only one drive rapidly flashing during that process) before
> moving on to the next x 90 drives.
>
> Is there anything I can do to get more verbosity in the boot messages that
> might just reveal the root issue?
>
> Any suggestions appreciated.
>
> Thanks
>
> ________________________
> Michael Talbott
> Systems Administrator
> La Jolla Institute
>
> On Jul 29, 2015, at 7:51 AM, Schweiss, Chip <chip at innovates.com> wrote:
>
>
>
> On Fri, Jul 24, 2015 at 5:03 PM, Michael Talbott <mtalbott at lji.org> wrote:
>
>> Hi,
>>
>> I've downgraded the cards (LSI 9211-8e) to v.19 and disabled their boot
>> bios. But I'm still getting the 8 second per drive delay after the kernel
>> loads. Any other ideas?
>>
>>
> 8 seconds is way too long.   What JBODs and disks are you using?   Could
> it be they are powered off and the delay in waiting for the power on
> command to complete?   This could be accelerated by using lsiutils to send
> them all power on commands first.
>
> While I still consider it slow, however, my OmniOS systems with  LSI HBAs
> discover about 2 disks per second.   With systems with LOTS of disk all
> multipathed it still stacks up to a long time to discover them all.
>
> -Chip
>
>
>>
>> ________________________
>> Michael Talbott
>> Systems Administrator
>> La Jolla Institute
>>
>> > On Jul 20, 2015, at 11:27 PM, Floris van Essen ..:: House of Ancients
>> Amstafs ::.. <info at houseofancients.nl> wrote:
>> >
>> > Michael,
>> >
>> > I know v20 does cause lots of issue's.
>> > V19 , to the best of my knowledge doesn't contain any, so I would
>> downgrade to v19
>> >
>> >
>> > Kr,
>> >
>> >
>> > Floris
>> > -----Oorspronkelijk bericht-----
>> > Van: OmniOS-discuss [mailto:omnios-discuss-bounces at lists.omniti.com]
>> Namens Michael Talbott
>> > Verzonden: dinsdag 21 juli 2015 4:57
>> > Aan: Marion Hakanson <hakansom at ohsu.edu>
>> > CC: omnios-discuss <omnios-discuss at lists.omniti.com>
>> > Onderwerp: Re: [OmniOS-discuss] Slow Drive Detection and boot-archive
>> >
>> > Thanks for the reply. The bios for the card is disabled already. The 8
>> second per drive scan happens after the kernel has already loaded and it is
>> scanning for devices. I wonder if it's due to running newer firmware. I did
>> update the cards to fw v.20.something before I moved to omnios. Is there a
>> particular firmware version on the cards I should run to match OmniOS's
>> drivers?
>> >
>> >
>> > ________________________
>> > Michael Talbott
>> > Systems Administrator
>> > La Jolla Institute
>> >
>> >> On Jul 20, 2015, at 6:06 PM, Marion Hakanson <hakansom at ohsu.edu>
>> wrote:
>> >>
>> >> Michael,
>> >>
>> >> I've not seen this;  I do have one system with 120 drives and it
>> >> definitely does not have this problem.  A couple with 80+ drives are
>> >> also free of this issue, though they are still running OpenIndiana.
>> >>
>> >> One thing I pretty much always do here, is to disable the boot option
>> >> in the LSI HBA's config utility (accessible from the during boot after
>> >> the BIOS has started up).  I do this because I don't want the BIOS
>> >> thinking it can boot from any of the external JBOD disks;  And also
>> >> because I've had some system BIOS crashes when they tried to enumerate
>> >> too many drives.  But, this all happens at the BIOS level, before the
>> >> OS has even started up, so in theory it should not affect what you are
>> >> seeing.
>> >>
>> >> Regards,
>> >>
>> >> Marion
>> >>
>> >>
>> >> ================================================================
>> >> Subject: Re: [OmniOS-discuss] Slow Drive Detection and boot-archive
>> >> From: Michael Talbott <mtalbott at lji.org>
>> >> Date: Fri, 17 Jul 2015 16:15:47 -0700
>> >> To: omnios-discuss <omnios-discuss at lists.omniti.com>
>> >>
>> >> Just realized my typo. I'm using this on my 90 and 180 drive systems:
>> >>
>> >> # svccfg -s boot-archive setprop start/timeout_seconds=720 # svccfg -s
>> >> boot-archive setprop start/timeout_seconds=1440
>> >>
>> >> Seems like 8 seconds to detect each drive is pretty excessive.
>> >>
>> >> Any ideas on how to speed that up?
>> >>
>> >>
>> >> ________________________
>> >> Michael Talbott
>> >> Systems Administrator
>> >> La Jolla Institute
>> >>
>> >>> On Jul 17, 2015, at 4:07 PM, Michael Talbott <mtalbott at lji.org>
>> wrote:
>> >>>
>> >>> I have multiple NAS servers I've moved to OmniOS and each of them
>> have 90-180 4T disks. Everything has worked out pretty well for the most
>> part. But I've come into an issue where when I reboot any of them, I'm
>> getting boot-archive service timeouts happening. I found a workaround of
>> increasing the timeout value which brings me to the following. As you can
>> see below in a dmesg output, it's taking the kernel about 8 seconds to
>> detect each of the drives. They're connected via a couple SAS2008 based LSI
>> cards.
>> >>>
>> >>> Is this normal?
>> >>> Is there a way to speed that up?
>> >>>
>> >>> I've fixed my frustrating boot-archive timeout problem by adjusting
>> the timeout value from the default of 60 seconds (I guess that'll work ok
>> on systems with less than 8 drives?) to 8 seconds * 90 drives + a little
>> extra time = 280 seconds (for the 90 drive systems). Which means it takes
>> between 12-24 minutes to boot those machines up.
>> >>>
>> >>> # svccfg -s boot-archive setprop start/timeout_seconds=280
>> >>>
>> >>> I figure I can't be the only one. A little googling also revealed:
>> >>> https://www.illumos.org/issues/4614
>> >>> <https://www.illumos.org/issues/4614>
>> >>>
>> >>> Jul 17 15:40:15 store2 genunix: [ID 583861 kern.info] sd29 at
>> >>> mpt_sas3: unit-address w50000c0f0401bd43,0: w50000c0f0401bd43,0 Jul
>> >>> 17 15:40:15 store2 genunix: [ID 936769 kern.info] sd29 is
>> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0401bd4
>> >>> 3,0 Jul 17 15:40:16 store2 genunix: [ID 408114 kern.info]
>> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0401bd4
>> >>> 3,0 (sd29) online Jul 17 15:40:24 store2 genunix: [ID 583861
>> >>> kern.info] sd30 at mpt_sas3: unit-address w50000c0f045679c3,0:
>> >>> w50000c0f045679c3,0 Jul 17 15:40:24 store2 genunix: [ID 936769
>> >>> kern.info] sd30 is
>> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f045679c
>> >>> 3,0 Jul 17 15:40:24 store2 genunix: [ID 408114 kern.info]
>> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f045679c
>> >>> 3,0 (sd30) online Jul 17 15:40:33 store2 genunix: [ID 583861
>> >>> kern.info] sd31 at mpt_sas3: unit-address w50000c0f045712b3,0:
>> >>> w50000c0f045712b3,0 Jul 17 15:40:33 store2 genunix: [ID 936769
>> >>> kern.info] sd31 is
>> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f045712b
>> >>> 3,0 Jul 17 15:40:33 store2 genunix: [ID 408114 kern.info]
>> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f045712b
>> >>> 3,0 (sd31) online Jul 17 15:40:42 store2 genunix: [ID 583861
>> >>> kern.info] sd32 at mpt_sas3: unit-address w50000c0f04571497,0:
>> >>> w50000c0f04571497,0 Jul 17 15:40:42 store2 genunix: [ID 936769
>> >>> kern.info] sd32 is
>> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0457149
>> >>> 7,0 Jul 17 15:40:42 store2 genunix: [ID 408114 kern.info]
>> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0457149
>> >>> 7,0 (sd32) online Jul 17 15:40:50 store2 genunix: [ID 583861
>> >>> kern.info] sd33 at mpt_sas3: unit-address w50000c0f042ac8eb,0:
>> >>> w50000c0f042ac8eb,0 Jul 17 15:40:50 store2 genunix: [ID 936769
>> >>> kern.info] sd33 is
>> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f042ac8e
>> >>> b,0 Jul 17 15:40:50 store2 genunix: [ID 408114 kern.info]
>> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f042ac8e
>> >>> b,0 (sd33) online Jul 17 15:40:59 store2 genunix: [ID 583861
>> >>> kern.info] sd34 at mpt_sas3: unit-address w50000c0f04571473,0:
>> >>> w50000c0f04571473,0 Jul 17 15:40:59 store2 genunix: [ID 936769
>> >>> kern.info] sd34 is
>> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0457147
>> >>> 3,0 Jul 17 15:40:59 store2 genunix: [ID 408114 kern.info]
>> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0457147
>> >>> 3,0 (sd34) online Jul 17 15:41:08 store2 genunix: [ID 583861
>> >>> kern.info] sd35 at mpt_sas3: unit-address w50000c0f042c636f,0:
>> >>> w50000c0f042c636f,0 Jul 17 15:41:08 store2 genunix: [ID 936769
>> >>> kern.info] sd35 is
>> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f042c636
>> >>> f,0 Jul 17 15:41:08 store2 genunix: [ID 408114 kern.info]
>> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f042c636
>> >>> f,0 (sd35) online Jul 17 15:41:17 store2 genunix: [ID 583861
>> >>> kern.info] sd36 at mpt_sas3: unit-address w50000c0f0401bf2f,0:
>> >>> w50000c0f0401bf2f,0 Jul 17 15:41:17 store2 genunix: [ID 936769
>> >>> kern.info] sd36 is
>> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0401bf2
>> >>> f,0 Jul 17 15:41:17 store2 genunix: [ID 408114 kern.info]
>> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0401bf2
>> >>> f,0 (sd36) online Jul 17 15:41:25 store2 genunix: [ID 583861
>> >>> kern.info] sd38 at mpt_sas3: unit-address w50000c0f0401bc1f,0:
>> >>> w50000c0f0401bc1f,0 Jul 17 15:41:25 store2 genunix: [ID 936769
>> >>> kern.info] sd38 is
>> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0401bc1
>> >>> f,0 Jul 17 15:41:26 store2 genunix: [ID 408114 kern.info]
>> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0401bc1
>> >>> f,0 (sd38) online
>> >>>
>> >>>
>> >>> ________________________
>> >>> Michael Talbott
>> >>> Systems Administrator
>> >>> La Jolla Institute
>> >>>
>> >>
>> >> _______________________________________________
>> >> OmniOS-discuss mailing list
>> >> OmniOS-discuss at lists.omniti.com
>> >> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>> >>
>> >>
>> >
>> > _______________________________________________
>> > OmniOS-discuss mailing list
>> > OmniOS-discuss at lists.omniti.com
>> > http://lists.omniti.com/mailman/listinfo/omnios-discuss
>> > ...:: House of Ancients ::...
>> > American Staffordshire Terriers
>> >
>> > +31-628-161-350
>> > +31-614-198-389
>> > Het Perk 48
>> > 4903 RB
>> > Oosterhout
>> > Netherlands
>> > www.houseofancients.nl
>>
>> _______________________________________________
>> OmniOS-discuss mailing list
>> OmniOS-discuss at lists.omniti.com
>> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150729/4c7d376e/attachment-0001.html>

From mtalbott at lji.org  Wed Jul 29 22:02:17 2015
From: mtalbott at lji.org (Michael Talbott)
Date: Wed, 29 Jul 2015 15:02:17 -0700
Subject: [OmniOS-discuss] Slow Drive Detection and boot-archive
In-Reply-To: <CALeZrrR2k-pBwyTp6pe5USEDPbTPzu2Fnxs=yivh44c5+GrVzg@mail.gmail.com>
References: <201507210106.t6L16ee4022209@kyklops.ohsu.edu>
	<EF194F85-A03F-428E-A056-424B1D4AA23B@lji.org>
	<356582D1FC91784992ABB4265A16ED489AABAAC8@vEX01.mindstorm-internet.local>
	<C1265D7C-AB4E-47E3-B0B5-C3FE8B578D1D@lji.org>
	<CALeZrrSTovc2ixmNfvrfh_YsK9KEwpqPDj_2zSM+3FC+=kh+Tg@mail.gmail.com>
	<303EF982-24E7-4E61-9B78-7D40B77A3530@lji.org>
	<CALeZrrR2k-pBwyTp6pe5USEDPbTPzu2Fnxs=yivh44c5+GrVzg@mail.gmail.com>
Message-ID: <B7C02F14-FD48-4C22-BB6E-996686F2B516@lji.org>

Gave that a shot. No dice. Still getting the 8 second lag. It reminds me of raid cards that do staggered spinups that sequentially spin up 1 drive at a time. Only, this is happening after the kernel loads and of course, the LSI 9200s are flashed in IT mode with v.19 firmware and bios disabled.


Jul 29 14:57:12 store2 genunix: [ID 583861 kern.info] sd10 at mpt_sas2: unit-address w50000c0f0401c20f,0: w50000c0f0401c20f,0
Jul 29 14:57:12 store2 genunix: [ID 936769 kern.info] sd10 is /pci at 0,0/pci8086,e04 at 2/pci1000,3080 at 0/iport at f0/disk at w50000c0f0401c20f,0
Jul 29 14:57:12 store2 genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,e04 at 2/pci1000,3080 at 0/iport at f0/disk at w50000c0f0401c20f,0 (sd10) online
Jul 29 14:57:20 store2 genunix: [ID 583861 kern.info] sd11 at mpt_sas2: unit-address w50000c0f040075db,0: w50000c0f040075db,0
Jul 29 14:57:20 store2 genunix: [ID 936769 kern.info] sd11 is /pci at 0,0/pci8086,e04 at 2/pci1000,3080 at 0/iport at f0/disk at w50000c0f040075db,0
Jul 29 14:57:21 store2 genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,e04 at 2/pci1000,3080 at 0/iport at f0/disk at w50000c0f040075db,0 (sd11) online
Jul 29 14:57:29 store2 genunix: [ID 583861 kern.info] sd12 at mpt_sas2: unit-address w50000c0f042c684b,0: w50000c0f042c684b,0
Jul 29 14:57:29 store2 genunix: [ID 936769 kern.info] sd12 is /pci at 0,0/pci8086,e04 at 2/pci1000,3080 at 0/iport at f0/disk at w50000c0f042c684b,0
Jul 29 14:57:29 store2 genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,e04 at 2/pci1000,3080 at 0/iport at f0/disk at w50000c0f042c684b,0 (sd12) online
Jul 29 14:57:38 store2 genunix: [ID 583861 kern.info] sd13 at mpt_sas2: unit-address w50000c0f0457149f,0: w50000c0f0457149f,0
Jul 29 14:57:38 store2 genunix: [ID 936769 kern.info] sd13 is /pci at 0,0/pci8086,e04 at 2/pci1000,3080 at 0/iport at f0/disk at w50000c0f0457149f,0
Jul 29 14:57:38 store2 genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,e04 at 2/pci1000,3080 at 0/iport at f0/disk at w50000c0f0457149f,0 (sd13) online
Jul 29 14:57:47 store2 genunix: [ID 583861 kern.info] sd14 at mpt_sas2: unit-address w50000c0f042b1c6f,0: w50000c0f042b1c6f,0
Jul 29 14:57:47 store2 genunix: [ID 936769 kern.info] sd14 is /pci at 0,0/pci8086,e04 at 2/pci1000,3080 at 0/iport at f0/disk at w50000c0f042b1c6f,0
Jul 29 14:57:47 store2 genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,e04 at 2/pci1000,3080 at 0/iport at f0/disk at w50000c0f042b1c6f,0 (sd14) online


________________________
Michael Talbott
Systems Administrator
La Jolla Institute

> On Jul 29, 2015, at 1:50 PM, Schweiss, Chip <chip at innovates.com> wrote:
> 
> I have an OmniOS box with all the same hardware except the server and hard disks.  I would wager this something to do with the WD disks and something different happening in the init.
> 
> This is a stab in the dark, but try adding "power-condition:false" in /kernel/drv/sd.conf for the WD disks. 
> 
> -Chip
> 
> 
> 
> On Wed, Jul 29, 2015 at 12:48 PM, Michael Talbott <mtalbott at lji.org <mailto:mtalbott at lji.org>> wrote:
> Here's the specs of that server.
> 
> Fujitsu RX300S8
>  - http://www.fujitsu.com/fts/products/computing/servers/primergy/rack/rx300/ <http://www.fujitsu.com/fts/products/computing/servers/primergy/rack/rx300/>
> 128G ECC DDR3 1600 RAM
> 2 x Intel(R) Xeon(R) CPU E5-2630 v2 @ 2.60GHz
> 2 x LSI 9200-8e
> 2 x 10Gb Intel NICs
> 2 x SuperMicro 847E26-RJBOD1 45 bay JBOD enclosures
>  - http://www.supermicro.com/products/chassis/4U/847/SC847E26-RJBOD1.cfm <http://www.supermicro.com/products/chassis/4U/847/SC847E26-RJBOD1.cfm>
> 
> The enclosures are not currently set up for multipathing. The front and rear backplane each have a single independent SAS connection to one of the LSI 9200s.
> 
> The two enclosures are fully loaded with 45 x 4TB WD4001FYYG-01SL3 drives each (90 total).
> http://www.newegg.com/Product/Product.aspx?Item=N82E16822236353 <http://www.newegg.com/Product/Product.aspx?Item=N82E16822236353>
> 
> Booting the server up in Ubuntu or CentOS does not have that 8 second delay. Each drive is found in a fraction of a second (activity LEDs on the enclosure flash on and off really quick as the drives are scanned). On OmniOS, the drives seem to be scanned in the same order, but, instead of it spending a fraction of a second on each drive, it spends 8 seconds on 1 drive (led of only one drive rapidly flashing during that process) before moving on to the next x 90 drives.
> 
> Is there anything I can do to get more verbosity in the boot messages that might just reveal the root issue?
> 
> Any suggestions appreciated.
> 
> Thanks
> 
> ________________________
> Michael Talbott
> Systems Administrator
> La Jolla Institute
> 
>> On Jul 29, 2015, at 7:51 AM, Schweiss, Chip <chip at innovates.com <mailto:chip at innovates.com>> wrote:
>> 
>> 
>> 
>> On Fri, Jul 24, 2015 at 5:03 PM, Michael Talbott <mtalbott at lji.org <mailto:mtalbott at lji.org>> wrote:
>> Hi,
>> 
>> I've downgraded the cards (LSI 9211-8e) to v.19 and disabled their boot bios. But I'm still getting the 8 second per drive delay after the kernel loads. Any other ideas?
>> 
>> 
>> 8 seconds is way too long.   What JBODs and disks are you using?   Could it be they are powered off and the delay in waiting for the power on command to complete?   This could be accelerated by using lsiutils to send them all power on commands first.
>> 
>> While I still consider it slow, however, my OmniOS systems with  LSI HBAs discover about 2 disks per second.   With systems with LOTS of disk all multipathed it still stacks up to a long time to discover them all.
>> 
>> -Chip
>>  
>> 
>> ________________________
>> Michael Talbott
>> Systems Administrator
>> La Jolla Institute
>> 
>> > On Jul 20, 2015, at 11:27 PM, Floris van Essen ..:: House of Ancients Amstafs ::.. <info at houseofancients.nl <mailto:info at houseofancients.nl>> wrote:
>> >
>> > Michael,
>> >
>> > I know v20 does cause lots of issue's.
>> > V19 , to the best of my knowledge doesn't contain any, so I would downgrade to v19
>> >
>> >
>> > Kr,
>> >
>> >
>> > Floris
>> > -----Oorspronkelijk bericht-----
>> > Van: OmniOS-discuss [mailto:omnios-discuss-bounces at lists.omniti.com <mailto:omnios-discuss-bounces at lists.omniti.com>] Namens Michael Talbott
>> > Verzonden: dinsdag 21 juli 2015 4:57
>> > Aan: Marion Hakanson <hakansom at ohsu.edu <mailto:hakansom at ohsu.edu>>
>> > CC: omnios-discuss <omnios-discuss at lists.omniti.com <mailto:omnios-discuss at lists.omniti.com>>
>> > Onderwerp: Re: [OmniOS-discuss] Slow Drive Detection and boot-archive
>> >
>> > Thanks for the reply. The bios for the card is disabled already. The 8 second per drive scan happens after the kernel has already loaded and it is scanning for devices. I wonder if it's due to running newer firmware. I did update the cards to fw v.20.something before I moved to omnios. Is there a particular firmware version on the cards I should run to match OmniOS's drivers?
>> >
>> >
>> > ________________________
>> > Michael Talbott
>> > Systems Administrator
>> > La Jolla Institute
>> >
>> >> On Jul 20, 2015, at 6:06 PM, Marion Hakanson <hakansom at ohsu.edu <mailto:hakansom at ohsu.edu>> wrote:
>> >>
>> >> Michael,
>> >>
>> >> I've not seen this;  I do have one system with 120 drives and it
>> >> definitely does not have this problem.  A couple with 80+ drives are
>> >> also free of this issue, though they are still running OpenIndiana.
>> >>
>> >> One thing I pretty much always do here, is to disable the boot option
>> >> in the LSI HBA's config utility (accessible from the during boot after
>> >> the BIOS has started up).  I do this because I don't want the BIOS
>> >> thinking it can boot from any of the external JBOD disks;  And also
>> >> because I've had some system BIOS crashes when they tried to enumerate
>> >> too many drives.  But, this all happens at the BIOS level, before the
>> >> OS has even started up, so in theory it should not affect what you are
>> >> seeing.
>> >>
>> >> Regards,
>> >>
>> >> Marion
>> >>
>> >>
>> >> ================================================================
>> >> Subject: Re: [OmniOS-discuss] Slow Drive Detection and boot-archive
>> >> From: Michael Talbott <mtalbott at lji.org <mailto:mtalbott at lji.org>>
>> >> Date: Fri, 17 Jul 2015 16:15:47 -0700
>> >> To: omnios-discuss <omnios-discuss at lists.omniti.com <mailto:omnios-discuss at lists.omniti.com>>
>> >>
>> >> Just realized my typo. I'm using this on my 90 and 180 drive systems:
>> >>
>> >> # svccfg -s boot-archive setprop start/timeout_seconds=720 # svccfg -s
>> >> boot-archive setprop start/timeout_seconds=1440
>> >>
>> >> Seems like 8 seconds to detect each drive is pretty excessive.
>> >>
>> >> Any ideas on how to speed that up?
>> >>
>> >>
>> >> ________________________
>> >> Michael Talbott
>> >> Systems Administrator
>> >> La Jolla Institute
>> >>
>> >>> On Jul 17, 2015, at 4:07 PM, Michael Talbott <mtalbott at lji.org <mailto:mtalbott at lji.org>> wrote:
>> >>>
>> >>> I have multiple NAS servers I've moved to OmniOS and each of them have 90-180 4T disks. Everything has worked out pretty well for the most part. But I've come into an issue where when I reboot any of them, I'm getting boot-archive service timeouts happening. I found a workaround of increasing the timeout value which brings me to the following. As you can see below in a dmesg output, it's taking the kernel about 8 seconds to detect each of the drives. They're connected via a couple SAS2008 based LSI cards.
>> >>>
>> >>> Is this normal?
>> >>> Is there a way to speed that up?
>> >>>
>> >>> I've fixed my frustrating boot-archive timeout problem by adjusting the timeout value from the default of 60 seconds (I guess that'll work ok on systems with less than 8 drives?) to 8 seconds * 90 drives + a little extra time = 280 seconds (for the 90 drive systems). Which means it takes between 12-24 minutes to boot those machines up.
>> >>>
>> >>> # svccfg -s boot-archive setprop start/timeout_seconds=280
>> >>>
>> >>> I figure I can't be the only one. A little googling also revealed:
>> >>> https://www.illumos.org/issues/4614 <https://www.illumos.org/issues/4614>
>> >>> <https://www.illumos.org/issues/4614 <https://www.illumos.org/issues/4614>>
>> >>>
>> >>> Jul 17 15:40:15 store2 genunix: [ID 583861 kern.info <http://kern.info/>] sd29 at
>> >>> mpt_sas3: unit-address w50000c0f0401bd43,0: w50000c0f0401bd43,0 Jul
>> >>> 17 15:40:15 store2 genunix: [ID 936769 kern.info <http://kern.info/>] sd29 is
>> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0401bd4
>> >>> 3,0 Jul 17 15:40:16 store2 genunix: [ID 408114 kern.info <http://kern.info/>]
>> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0401bd4
>> >>> 3,0 (sd29) online Jul 17 15:40:24 store2 genunix: [ID 583861
>> >>> kern.info <http://kern.info/>] sd30 at mpt_sas3: unit-address w50000c0f045679c3,0:
>> >>> w50000c0f045679c3,0 Jul 17 15:40:24 store2 genunix: [ID 936769
>> >>> kern.info <http://kern.info/>] sd30 is
>> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f045679c
>> >>> 3,0 Jul 17 15:40:24 store2 genunix: [ID 408114 kern.info <http://kern.info/>]
>> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f045679c
>> >>> 3,0 (sd30) online Jul 17 15:40:33 store2 genunix: [ID 583861
>> >>> kern.info <http://kern.info/>] sd31 at mpt_sas3: unit-address w50000c0f045712b3,0:
>> >>> w50000c0f045712b3,0 Jul 17 15:40:33 store2 genunix: [ID 936769
>> >>> kern.info <http://kern.info/>] sd31 is
>> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f045712b
>> >>> 3,0 Jul 17 15:40:33 store2 genunix: [ID 408114 kern.info <http://kern.info/>]
>> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f045712b
>> >>> 3,0 (sd31) online Jul 17 15:40:42 store2 genunix: [ID 583861
>> >>> kern.info <http://kern.info/>] sd32 at mpt_sas3: unit-address w50000c0f04571497,0:
>> >>> w50000c0f04571497,0 Jul 17 15:40:42 store2 genunix: [ID 936769
>> >>> kern.info <http://kern.info/>] sd32 is
>> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0457149
>> >>> 7,0 Jul 17 15:40:42 store2 genunix: [ID 408114 kern.info <http://kern.info/>]
>> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0457149
>> >>> 7,0 (sd32) online Jul 17 15:40:50 store2 genunix: [ID 583861
>> >>> kern.info <http://kern.info/>] sd33 at mpt_sas3: unit-address w50000c0f042ac8eb,0:
>> >>> w50000c0f042ac8eb,0 Jul 17 15:40:50 store2 genunix: [ID 936769
>> >>> kern.info <http://kern.info/>] sd33 is
>> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f042ac8e
>> >>> b,0 Jul 17 15:40:50 store2 genunix: [ID 408114 kern.info <http://kern.info/>]
>> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f042ac8e
>> >>> b,0 (sd33) online Jul 17 15:40:59 store2 genunix: [ID 583861
>> >>> kern.info <http://kern.info/>] sd34 at mpt_sas3: unit-address w50000c0f04571473,0:
>> >>> w50000c0f04571473,0 Jul 17 15:40:59 store2 genunix: [ID 936769
>> >>> kern.info <http://kern.info/>] sd34 is
>> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0457147
>> >>> 3,0 Jul 17 15:40:59 store2 genunix: [ID 408114 kern.info <http://kern.info/>]
>> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0457147
>> >>> 3,0 (sd34) online Jul 17 15:41:08 store2 genunix: [ID 583861
>> >>> kern.info <http://kern.info/>] sd35 at mpt_sas3: unit-address w50000c0f042c636f,0:
>> >>> w50000c0f042c636f,0 Jul 17 15:41:08 store2 genunix: [ID 936769
>> >>> kern.info <http://kern.info/>] sd35 is
>> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f042c636
>> >>> f,0 Jul 17 15:41:08 store2 genunix: [ID 408114 kern.info <http://kern.info/>]
>> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f042c636
>> >>> f,0 (sd35) online Jul 17 15:41:17 store2 genunix: [ID 583861
>> >>> kern.info <http://kern.info/>] sd36 at mpt_sas3: unit-address w50000c0f0401bf2f,0:
>> >>> w50000c0f0401bf2f,0 Jul 17 15:41:17 store2 genunix: [ID 936769
>> >>> kern.info <http://kern.info/>] sd36 is
>> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0401bf2
>> >>> f,0 Jul 17 15:41:17 store2 genunix: [ID 408114 kern.info <http://kern.info/>]
>> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0401bf2
>> >>> f,0 (sd36) online Jul 17 15:41:25 store2 genunix: [ID 583861
>> >>> kern.info <http://kern.info/>] sd38 at mpt_sas3: unit-address w50000c0f0401bc1f,0:
>> >>> w50000c0f0401bc1f,0 Jul 17 15:41:25 store2 genunix: [ID 936769
>> >>> kern.info <http://kern.info/>] sd38 is
>> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0401bc1
>> >>> f,0 Jul 17 15:41:26 store2 genunix: [ID 408114 kern.info <http://kern.info/>]
>> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f/disk at w50000c0f0401bc1
>> >>> f,0 (sd38) online
>> >>>
>> >>>
>> >>> ________________________
>> >>> Michael Talbott
>> >>> Systems Administrator
>> >>> La Jolla Institute
>> >>>
>> >>
>> >> _______________________________________________
>> >> OmniOS-discuss mailing list
>> >> OmniOS-discuss at lists.omniti.com <mailto:OmniOS-discuss at lists.omniti.com>
>> >> http://lists.omniti.com/mailman/listinfo/omnios-discuss <http://lists.omniti.com/mailman/listinfo/omnios-discuss>
>> >>
>> >>
>> >
>> > _______________________________________________
>> > OmniOS-discuss mailing list
>> > OmniOS-discuss at lists.omniti.com <mailto:OmniOS-discuss at lists.omniti.com>
>> > http://lists.omniti.com/mailman/listinfo/omnios-discuss <http://lists.omniti.com/mailman/listinfo/omnios-discuss>
>> > ...:: House of Ancients ::...
>> > American Staffordshire Terriers
>> >
>> > +31-628-161-350 <tel:%2B31-628-161-350>
>> > +31-614-198-389 <tel:%2B31-614-198-389>
>> > Het Perk 48
>> > 4903 RB
>> > Oosterhout
>> > Netherlands
>> > www.houseofancients.nl <http://www.houseofancients.nl/>
>> 
>> _______________________________________________
>> OmniOS-discuss mailing list
>> OmniOS-discuss at lists.omniti.com <mailto:OmniOS-discuss at lists.omniti.com>
>> http://lists.omniti.com/mailman/listinfo/omnios-discuss <http://lists.omniti.com/mailman/listinfo/omnios-discuss>
>> 
> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150729/eab97739/attachment-0001.html>

From chip at innovates.com  Wed Jul 29 22:07:52 2015
From: chip at innovates.com (Schweiss, Chip)
Date: Wed, 29 Jul 2015 17:07:52 -0500
Subject: [OmniOS-discuss] Slow Drive Detection and boot-archive
In-Reply-To: <B7C02F14-FD48-4C22-BB6E-996686F2B516@lji.org>
References: <201507210106.t6L16ee4022209@kyklops.ohsu.edu>
	<EF194F85-A03F-428E-A056-424B1D4AA23B@lji.org>
	<356582D1FC91784992ABB4265A16ED489AABAAC8@vEX01.mindstorm-internet.local>
	<C1265D7C-AB4E-47E3-B0B5-C3FE8B578D1D@lji.org>
	<CALeZrrSTovc2ixmNfvrfh_YsK9KEwpqPDj_2zSM+3FC+=kh+Tg@mail.gmail.com>
	<303EF982-24E7-4E61-9B78-7D40B77A3530@lji.org>
	<CALeZrrR2k-pBwyTp6pe5USEDPbTPzu2Fnxs=yivh44c5+GrVzg@mail.gmail.com>
	<B7C02F14-FD48-4C22-BB6E-996686F2B516@lji.org>
Message-ID: <CALeZrrTkXmneQFSSGRWR0zXnMtCsGRui30obs95QnaiTjr7rsw@mail.gmail.com>

The only other thing that come to mind is that you mentioned you have only
a single SAS path to these disks.   Have you disabled multipath?  (stmsboot
-d)


-Chip

On Wed, Jul 29, 2015 at 5:02 PM, Michael Talbott <mtalbott at lji.org> wrote:

> Gave that a shot. No dice. Still getting the 8 second lag. It reminds me
> of raid cards that do staggered spinups that sequentially spin up 1 drive
> at a time. Only, this is happening after the kernel loads and of course,
> the LSI 9200s are flashed in IT mode with v.19 firmware and bios disabled.
>
>
> Jul 29 14:57:12 store2 genunix: [ID 583861 kern.info] sd10 at mpt_sas2:
> unit-address w50000c0f0401c20f,0: w50000c0f0401c20f,0
> Jul 29 14:57:12 store2 genunix: [ID 936769 kern.info] sd10 is /pci at 0
> ,0/pci8086,e04 at 2/pci1000,3080 at 0/iport at f0/disk at w50000c0f0401c20f,0
> Jul 29 14:57:12 store2 genunix: [ID 408114 kern.info] /pci at 0
> ,0/pci8086,e04 at 2/pci1000,3080 at 0/iport at f0/disk at w50000c0f0401c20f,0 (sd10)
> online
> Jul 29 14:57:20 store2 genunix: [ID 583861 kern.info] sd11 at mpt_sas2:
> unit-address w50000c0f040075db,0: w50000c0f040075db,0
> Jul 29 14:57:20 store2 genunix: [ID 936769 kern.info] sd11 is /pci at 0
> ,0/pci8086,e04 at 2/pci1000,3080 at 0/iport at f0/disk at w50000c0f040075db,0
> Jul 29 14:57:21 store2 genunix: [ID 408114 kern.info] /pci at 0
> ,0/pci8086,e04 at 2/pci1000,3080 at 0/iport at f0/disk at w50000c0f040075db,0 (sd11)
> online
> Jul 29 14:57:29 store2 genunix: [ID 583861 kern.info] sd12 at mpt_sas2:
> unit-address w50000c0f042c684b,0: w50000c0f042c684b,0
> Jul 29 14:57:29 store2 genunix: [ID 936769 kern.info] sd12 is /pci at 0
> ,0/pci8086,e04 at 2/pci1000,3080 at 0/iport at f0/disk at w50000c0f042c684b,0
> Jul 29 14:57:29 store2 genunix: [ID 408114 kern.info] /pci at 0
> ,0/pci8086,e04 at 2/pci1000,3080 at 0/iport at f0/disk at w50000c0f042c684b,0 (sd12)
> online
> Jul 29 14:57:38 store2 genunix: [ID 583861 kern.info] sd13 at mpt_sas2:
> unit-address w50000c0f0457149f,0: w50000c0f0457149f,0
> Jul 29 14:57:38 store2 genunix: [ID 936769 kern.info] sd13 is /pci at 0
> ,0/pci8086,e04 at 2/pci1000,3080 at 0/iport at f0/disk at w50000c0f0457149f,0
> Jul 29 14:57:38 store2 genunix: [ID 408114 kern.info] /pci at 0
> ,0/pci8086,e04 at 2/pci1000,3080 at 0/iport at f0/disk at w50000c0f0457149f,0 (sd13)
> online
> Jul 29 14:57:47 store2 genunix: [ID 583861 kern.info] sd14 at mpt_sas2:
> unit-address w50000c0f042b1c6f,0: w50000c0f042b1c6f,0
> Jul 29 14:57:47 store2 genunix: [ID 936769 kern.info] sd14 is /pci at 0
> ,0/pci8086,e04 at 2/pci1000,3080 at 0/iport at f0/disk at w50000c0f042b1c6f,0
> Jul 29 14:57:47 store2 genunix: [ID 408114 kern.info] /pci at 0
> ,0/pci8086,e04 at 2/pci1000,3080 at 0/iport at f0/disk at w50000c0f042b1c6f,0 (sd14)
> online
>
>
> ________________________
> Michael Talbott
> Systems Administrator
> La Jolla Institute
>
> On Jul 29, 2015, at 1:50 PM, Schweiss, Chip <chip at innovates.com> wrote:
>
> I have an OmniOS box with all the same hardware except the server and hard
> disks.  I would wager this something to do with the WD disks and something
> different happening in the init.
>
> This is a stab in the dark, but try adding "power-condition:false" in
> /kernel/drv/sd.conf for the WD disks.
>
> -Chip
>
>
>
> On Wed, Jul 29, 2015 at 12:48 PM, Michael Talbott <mtalbott at lji.org>
> wrote:
>
>> Here's the specs of that server.
>>
>> Fujitsu RX300S8
>>  -
>> http://www.fujitsu.com/fts/products/computing/servers/primergy/rack/rx300/
>> 128G ECC DDR3 1600 RAM
>> 2 x Intel(R) Xeon(R) CPU E5-2630 v2 @ 2.60GHz
>> 2 x LSI 9200-8e
>> 2 x 10Gb Intel NICs
>> 2 x SuperMicro 847E26-RJBOD1 45 bay JBOD enclosures
>>  - http://www.supermicro.com/products/chassis/4U/847/SC847E26-RJBOD1.cfm
>>
>> The enclosures are not currently set up for multipathing. The front and
>> rear backplane each have a single independent SAS connection to one of the
>> LSI 9200s.
>>
>> The two enclosures are fully loaded with 45 x 4TB WD4001FYYG-01SL3 drives
>> each (90 total).
>> http://www.newegg.com/Product/Product.aspx?Item=N82E16822236353
>>
>> Booting the server up in Ubuntu or CentOS does not have that 8 second
>> delay. Each drive is found in a fraction of a second (activity LEDs on the
>> enclosure flash on and off really quick as the drives are scanned). On
>> OmniOS, the drives seem to be scanned in the same order, but, instead of it
>> spending a fraction of a second on each drive, it spends 8 seconds on 1
>> drive (led of only one drive rapidly flashing during that process) before
>> moving on to the next x 90 drives.
>>
>> Is there anything I can do to get more verbosity in the boot messages
>> that might just reveal the root issue?
>>
>> Any suggestions appreciated.
>>
>> Thanks
>>
>> ________________________
>> Michael Talbott
>> Systems Administrator
>> La Jolla Institute
>>
>> On Jul 29, 2015, at 7:51 AM, Schweiss, Chip <chip at innovates.com> wrote:
>>
>>
>>
>> On Fri, Jul 24, 2015 at 5:03 PM, Michael Talbott <mtalbott at lji.org>
>> wrote:
>>
>>> Hi,
>>>
>>> I've downgraded the cards (LSI 9211-8e) to v.19 and disabled their boot
>>> bios. But I'm still getting the 8 second per drive delay after the kernel
>>> loads. Any other ideas?
>>>
>>>
>> 8 seconds is way too long.   What JBODs and disks are you using?   Could
>> it be they are powered off and the delay in waiting for the power on
>> command to complete?   This could be accelerated by using lsiutils to send
>> them all power on commands first.
>>
>> While I still consider it slow, however, my OmniOS systems with  LSI HBAs
>> discover about 2 disks per second.   With systems with LOTS of disk all
>> multipathed it still stacks up to a long time to discover them all.
>>
>> -Chip
>>
>>
>>>
>>> ________________________
>>> Michael Talbott
>>> Systems Administrator
>>> La Jolla Institute
>>>
>>> > On Jul 20, 2015, at 11:27 PM, Floris van Essen ..:: House of Ancients
>>> Amstafs ::.. <info at houseofancients.nl> wrote:
>>> >
>>> > Michael,
>>> >
>>> > I know v20 does cause lots of issue's.
>>> > V19 , to the best of my knowledge doesn't contain any, so I would
>>> downgrade to v19
>>> >
>>> >
>>> > Kr,
>>> >
>>> >
>>> > Floris
>>> > -----Oorspronkelijk bericht-----
>>> > Van: OmniOS-discuss [mailto:omnios-discuss-bounces at lists.omniti.com]
>>> Namens Michael Talbott
>>> > Verzonden: dinsdag 21 juli 2015 4:57
>>> > Aan: Marion Hakanson <hakansom at ohsu.edu>
>>> > CC: omnios-discuss <omnios-discuss at lists.omniti.com>
>>> > Onderwerp: Re: [OmniOS-discuss] Slow Drive Detection and boot-archive
>>> >
>>> > Thanks for the reply. The bios for the card is disabled already. The 8
>>> second per drive scan happens after the kernel has already loaded and it is
>>> scanning for devices. I wonder if it's due to running newer firmware. I did
>>> update the cards to fw v.20.something before I moved to omnios. Is there a
>>> particular firmware version on the cards I should run to match OmniOS's
>>> drivers?
>>> >
>>> >
>>> > ________________________
>>> > Michael Talbott
>>> > Systems Administrator
>>> > La Jolla Institute
>>> >
>>> >> On Jul 20, 2015, at 6:06 PM, Marion Hakanson <hakansom at ohsu.edu>
>>> wrote:
>>> >>
>>> >> Michael,
>>> >>
>>> >> I've not seen this;  I do have one system with 120 drives and it
>>> >> definitely does not have this problem.  A couple with 80+ drives are
>>> >> also free of this issue, though they are still running OpenIndiana.
>>> >>
>>> >> One thing I pretty much always do here, is to disable the boot option
>>> >> in the LSI HBA's config utility (accessible from the during boot after
>>> >> the BIOS has started up).  I do this because I don't want the BIOS
>>> >> thinking it can boot from any of the external JBOD disks;  And also
>>> >> because I've had some system BIOS crashes when they tried to enumerate
>>> >> too many drives.  But, this all happens at the BIOS level, before the
>>> >> OS has even started up, so in theory it should not affect what you are
>>> >> seeing.
>>> >>
>>> >> Regards,
>>> >>
>>> >> Marion
>>> >>
>>> >>
>>> >> ================================================================
>>> >> Subject: Re: [OmniOS-discuss] Slow Drive Detection and boot-archive
>>> >> From: Michael Talbott <mtalbott at lji.org>
>>> >> Date: Fri, 17 Jul 2015 16:15:47 -0700
>>> >> To: omnios-discuss <omnios-discuss at lists.omniti.com>
>>> >>
>>> >> Just realized my typo. I'm using this on my 90 and 180 drive systems:
>>> >>
>>> >> # svccfg -s boot-archive setprop start/timeout_seconds=720 # svccfg -s
>>> >> boot-archive setprop start/timeout_seconds=1440
>>> >>
>>> >> Seems like 8 seconds to detect each drive is pretty excessive.
>>> >>
>>> >> Any ideas on how to speed that up?
>>> >>
>>> >>
>>> >> ________________________
>>> >> Michael Talbott
>>> >> Systems Administrator
>>> >> La Jolla Institute
>>> >>
>>> >>> On Jul 17, 2015, at 4:07 PM, Michael Talbott <mtalbott at lji.org>
>>> wrote:
>>> >>>
>>> >>> I have multiple NAS servers I've moved to OmniOS and each of them
>>> have 90-180 4T disks. Everything has worked out pretty well for the most
>>> part. But I've come into an issue where when I reboot any of them, I'm
>>> getting boot-archive service timeouts happening. I found a workaround of
>>> increasing the timeout value which brings me to the following. As you can
>>> see below in a dmesg output, it's taking the kernel about 8 seconds to
>>> detect each of the drives. They're connected via a couple SAS2008 based LSI
>>> cards.
>>> >>>
>>> >>> Is this normal?
>>> >>> Is there a way to speed that up?
>>> >>>
>>> >>> I've fixed my frustrating boot-archive timeout problem by adjusting
>>> the timeout value from the default of 60 seconds (I guess that'll work ok
>>> on systems with less than 8 drives?) to 8 seconds * 90 drives + a little
>>> extra time = 280 seconds (for the 90 drive systems). Which means it takes
>>> between 12-24 minutes to boot those machines up.
>>> >>>
>>> >>> # svccfg -s boot-archive setprop start/timeout_seconds=280
>>> >>>
>>> >>> I figure I can't be the only one. A little googling also revealed:
>>> >>> https://www.illumos.org/issues/4614
>>> >>> <https://www.illumos.org/issues/4614>
>>> >>>
>>> >>> Jul 17 15:40:15 store2 genunix: [ID 583861 kern.info] sd29 at
>>> >>> mpt_sas3: unit-address w50000c0f0401bd43,0: w50000c0f0401bd43,0 Jul
>>> >>> 17 15:40:15 store2 genunix: [ID 936769 kern.info] sd29 is
>>> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f
>>> /disk at w50000c0f0401bd4
>>> >>> 3,0 Jul 17 15:40:16 store2 genunix: [ID 408114 kern.info]
>>> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f
>>> /disk at w50000c0f0401bd4
>>> >>> 3,0 (sd29) online Jul 17 15:40:24 store2 genunix: [ID 583861
>>> >>> kern.info] sd30 at mpt_sas3: unit-address w50000c0f045679c3,0:
>>> >>> w50000c0f045679c3,0 Jul 17 15:40:24 store2 genunix: [ID 936769
>>> >>> kern.info] sd30 is
>>> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f
>>> /disk at w50000c0f045679c
>>> >>> 3,0 Jul 17 15:40:24 store2 genunix: [ID 408114 kern.info]
>>> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f
>>> /disk at w50000c0f045679c
>>> >>> 3,0 (sd30) online Jul 17 15:40:33 store2 genunix: [ID 583861
>>> >>> kern.info] sd31 at mpt_sas3: unit-address w50000c0f045712b3,0:
>>> >>> w50000c0f045712b3,0 Jul 17 15:40:33 store2 genunix: [ID 936769
>>> >>> kern.info] sd31 is
>>> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f
>>> /disk at w50000c0f045712b
>>> >>> 3,0 Jul 17 15:40:33 store2 genunix: [ID 408114 kern.info]
>>> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f
>>> /disk at w50000c0f045712b
>>> >>> 3,0 (sd31) online Jul 17 15:40:42 store2 genunix: [ID 583861
>>> >>> kern.info] sd32 at mpt_sas3: unit-address w50000c0f04571497,0:
>>> >>> w50000c0f04571497,0 Jul 17 15:40:42 store2 genunix: [ID 936769
>>> >>> kern.info] sd32 is
>>> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f
>>> /disk at w50000c0f0457149
>>> >>> 7,0 Jul 17 15:40:42 store2 genunix: [ID 408114 kern.info]
>>> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f
>>> /disk at w50000c0f0457149
>>> >>> 7,0 (sd32) online Jul 17 15:40:50 store2 genunix: [ID 583861
>>> >>> kern.info] sd33 at mpt_sas3: unit-address w50000c0f042ac8eb,0:
>>> >>> w50000c0f042ac8eb,0 Jul 17 15:40:50 store2 genunix: [ID 936769
>>> >>> kern.info] sd33 is
>>> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f
>>> /disk at w50000c0f042ac8e
>>> >>> b,0 Jul 17 15:40:50 store2 genunix: [ID 408114 kern.info]
>>> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f
>>> /disk at w50000c0f042ac8e
>>> >>> b,0 (sd33) online Jul 17 15:40:59 store2 genunix: [ID 583861
>>> >>> kern.info] sd34 at mpt_sas3: unit-address w50000c0f04571473,0:
>>> >>> w50000c0f04571473,0 Jul 17 15:40:59 store2 genunix: [ID 936769
>>> >>> kern.info] sd34 is
>>> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f
>>> /disk at w50000c0f0457147
>>> >>> 3,0 Jul 17 15:40:59 store2 genunix: [ID 408114 kern.info]
>>> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f
>>> /disk at w50000c0f0457147
>>> >>> 3,0 (sd34) online Jul 17 15:41:08 store2 genunix: [ID 583861
>>> >>> kern.info] sd35 at mpt_sas3: unit-address w50000c0f042c636f,0:
>>> >>> w50000c0f042c636f,0 Jul 17 15:41:08 store2 genunix: [ID 936769
>>> >>> kern.info] sd35 is
>>> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f
>>> /disk at w50000c0f042c636
>>> >>> f,0 Jul 17 15:41:08 store2 genunix: [ID 408114 kern.info]
>>> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f
>>> /disk at w50000c0f042c636
>>> >>> f,0 (sd35) online Jul 17 15:41:17 store2 genunix: [ID 583861
>>> >>> kern.info] sd36 at mpt_sas3: unit-address w50000c0f0401bf2f,0:
>>> >>> w50000c0f0401bf2f,0 Jul 17 15:41:17 store2 genunix: [ID 936769
>>> >>> kern.info] sd36 is
>>> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f
>>> /disk at w50000c0f0401bf2
>>> >>> f,0 Jul 17 15:41:17 store2 genunix: [ID 408114 kern.info]
>>> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f
>>> /disk at w50000c0f0401bf2
>>> >>> f,0 (sd36) online Jul 17 15:41:25 store2 genunix: [ID 583861
>>> >>> kern.info] sd38 at mpt_sas3: unit-address w50000c0f0401bc1f,0:
>>> >>> w50000c0f0401bc1f,0 Jul 17 15:41:25 store2 genunix: [ID 936769
>>> >>> kern.info] sd38 is
>>> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f
>>> /disk at w50000c0f0401bc1
>>> >>> f,0 Jul 17 15:41:26 store2 genunix: [ID 408114 kern.info]
>>> >>> /pci at 0,0/pci8086,e06 at 2,2/pci1000,3080 at 0/iport at f
>>> /disk at w50000c0f0401bc1
>>> >>> f,0 (sd38) online
>>> >>>
>>> >>>
>>> >>> ________________________
>>> >>> Michael Talbott
>>> >>> Systems Administrator
>>> >>> La Jolla Institute
>>> >>>
>>> >>
>>> >> _______________________________________________
>>> >> OmniOS-discuss mailing list
>>> >> OmniOS-discuss at lists.omniti.com
>>> >> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>>> >>
>>> >>
>>> >
>>> > _______________________________________________
>>> > OmniOS-discuss mailing list
>>> > OmniOS-discuss at lists.omniti.com
>>> > http://lists.omniti.com/mailman/listinfo/omnios-discuss
>>> > ...:: House of Ancients ::...
>>> > American Staffordshire Terriers
>>> >
>>> > +31-628-161-350
>>> > +31-614-198-389
>>> > Het Perk 48
>>> > 4903 RB
>>> > Oosterhout
>>> > Netherlands
>>> > www.houseofancients.nl
>>>
>>> _______________________________________________
>>> OmniOS-discuss mailing list
>>> OmniOS-discuss at lists.omniti.com
>>> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>>>
>>
>>
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150729/8802befc/attachment-0001.html>

From mir at miras.org  Wed Jul 29 23:03:56 2015
From: mir at miras.org (Michael Rasmussen)
Date: Thu, 30 Jul 2015 01:03:56 +0200
Subject: [OmniOS-discuss] Slow Drive Detection and boot-archive
In-Reply-To: <CALeZrrTkXmneQFSSGRWR0zXnMtCsGRui30obs95QnaiTjr7rsw@mail.gmail.com>
References: <201507210106.t6L16ee4022209@kyklops.ohsu.edu>
	<EF194F85-A03F-428E-A056-424B1D4AA23B@lji.org>
	<356582D1FC91784992ABB4265A16ED489AABAAC8@vEX01.mindstorm-internet.local>
	<C1265D7C-AB4E-47E3-B0B5-C3FE8B578D1D@lji.org>
	<CALeZrrSTovc2ixmNfvrfh_YsK9KEwpqPDj_2zSM+3FC+=kh+Tg@mail.gmail.com>
	<303EF982-24E7-4E61-9B78-7D40B77A3530@lji.org>
	<CALeZrrR2k-pBwyTp6pe5USEDPbTPzu2Fnxs=yivh44c5+GrVzg@mail.gmail.com>
	<B7C02F14-FD48-4C22-BB6E-996686F2B516@lji.org>
	<CALeZrrTkXmneQFSSGRWR0zXnMtCsGRui30obs95QnaiTjr7rsw@mail.gmail.com>
Message-ID: <20150730010356.42399fa6@sleipner.datanom.net>

On Wed, 29 Jul 2015 17:07:52 -0500
"Schweiss, Chip" <chip at innovates.com> wrote:

> The only other thing that come to mind is that you mentioned you have only
> a single SAS path to these disks.   Have you disabled multipath?  (stmsboot
> -d)
> 
What about driver firmware?
http://www.dell.com/support/home/us/en/19/Drivers/DriversDetails?driverId=CKTR9

-- 
Hilsen/Regards
Michael Rasmussen

Get my public GnuPG keys:
michael <at> rasmussen <dot> cc
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E
mir <at> datanom <dot> net
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C
mir <at> miras <dot> org
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917
--------------------------------------------------------------
/usr/games/fortune -es says:
Specifications subject to change without notice.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 181 bytes
Desc: OpenPGP digital signature
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150730/52d1b39e/attachment.bin>

From mtalbott at lji.org  Thu Jul 30 00:23:14 2015
From: mtalbott at lji.org (Michael Talbott)
Date: Wed, 29 Jul 2015 17:23:14 -0700
Subject: [OmniOS-discuss] Slow Drive Detection and boot-archive
In-Reply-To: <20150730010356.42399fa6@sleipner.datanom.net>
References: <201507210106.t6L16ee4022209@kyklops.ohsu.edu>
	<EF194F85-A03F-428E-A056-424B1D4AA23B@lji.org>
	<356582D1FC91784992ABB4265A16ED489AABAAC8@vEX01.mindstorm-internet.local>
	<C1265D7C-AB4E-47E3-B0B5-C3FE8B578D1D@lji.org>
	<CALeZrrSTovc2ixmNfvrfh_YsK9KEwpqPDj_2zSM+3FC+=kh+Tg@mail.gmail.com>
	<303EF982-24E7-4E61-9B78-7D40B77A3530@lji.org>
	<CALeZrrR2k-pBwyTp6pe5USEDPbTPzu2Fnxs=yivh44c5+GrVzg@mail.gmail.com>
	<B7C02F14-FD48-4C22-BB6E-996686F2B516@lji.org>
	<CALeZrrTkXmneQFSSGRWR0zXnMtCsGRui30obs95QnaiTjr7rsw@mail.gmail.com>
	<20150730010356.42399fa6@sleipner.datanom.net>
Message-ID: <5055E053-3883-48D3-A813-29224318975E@lji.org>

After some more trial and error, disabling/enabling multipath did not help.

But after a little googling, I ran across this: https://syneto.eu/knowledgebase/cannot-use-some-models-of-enterprise-wd-or-seagate-disks/
I guess these guys use the Illumos kernel too and it became obvious I didn't use the right syntax in sd.conf with your previous suggestion.

so I added 
sd-config-list= "WD      WD4001FYYG","power-condition:false";
to /kernel/drv/sd.conf

and it seems to have cut my cold boot time in half :) Still, 4-5 seconds per drive seems pretty high for device detection, but it a vast improvement from than 8-9 seconds per drive.

Jul 29 17:05:25 store2 genunix: [ID 583861 kern.info] sd15 at mpt_sas2: unit-address w50000c0f042ba1bf,0: w50000c0f042ba1bf,0
Jul 29 17:05:25 store2 genunix: [ID 936769 kern.info] sd15 is /pci at 0,0/pci8086,e04 at 2/pci1000,3080 at 0/iport at f0/disk at w50000c0f042ba1bf,0
Jul 29 17:05:25 store2 genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,e04 at 2/pci1000,3080 at 0/iport at f0/disk at w50000c0f042ba1bf,0 (sd15) online
Jul 29 17:05:30 store2 genunix: [ID 583861 kern.info] sd17 at mpt_sas2: unit-address w50000c0f01c06d37,0: w50000c0f01c06d37,0
Jul 29 17:05:30 store2 genunix: [ID 936769 kern.info] sd17 is /pci at 0,0/pci8086,e04 at 2/pci1000,3080 at 0/iport at f0/disk at w50000c0f01c06d37,0
Jul 29 17:05:30 store2 genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,e04 at 2/pci1000,3080 at 0/iport at f0/disk at w50000c0f01c06d37,0 (sd17) online
Jul 29 17:05:34 store2 genunix: [ID 583861 kern.info] sd18 at mpt_sas2: unit-address w50000c0f045717af,0: w50000c0f045717af,0
Jul 29 17:05:34 store2 genunix: [ID 936769 kern.info] sd18 is /pci at 0,0/pci8086,e04 at 2/pci1000,3080 at 0/iport at f0/disk at w50000c0f045717af,0
Jul 29 17:05:34 store2 genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,e04 at 2/pci1000,3080 at 0/iport at f0/disk at w50000c0f045717af,0 (sd18) online
Jul 29 17:05:38 store2 genunix: [ID 583861 kern.info] sd19 at mpt_sas2: unit-address w50000c0f011a48fb,0: w50000c0f011a48fb,0
Jul 29 17:05:38 store2 genunix: [ID 936769 kern.info] sd19 is /pci at 0,0/pci8086,e04 at 2/pci1000,3080 at 0/iport at f0/disk at w50000c0f011a48fb,0
Jul 29 17:05:38 store2 genunix: [ID 408114 kern.info] /pci at 0,0/pci8086,e04 at 2/pci1000,3080 at 0/iport at f0/disk at w50000c0f011a48fb,0 (sd19) online

Thanks.



________________________
Michael Talbott
Systems Administrator
La Jolla Institute

> On Jul 29, 2015, at 4:03 PM, Michael Rasmussen <mir at miras.org> wrote:
> 
> On Wed, 29 Jul 2015 17:07:52 -0500
> "Schweiss, Chip" <chip at innovates.com> wrote:
> 
>> The only other thing that come to mind is that you mentioned you have only
>> a single SAS path to these disks.   Have you disabled multipath?  (stmsboot
>> -d)
>> 
> What about driver firmware?
> http://www.dell.com/support/home/us/en/19/Drivers/DriversDetails?driverId=CKTR9
> 
> -- 
> Hilsen/Regards
> Michael Rasmussen
> 
> Get my public GnuPG keys:
> michael <at> rasmussen <dot> cc
> http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E
> mir <at> datanom <dot> net
> http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C
> mir <at> miras <dot> org
> http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917
> --------------------------------------------------------------
> /usr/games/fortune -es says:
> Specifications subject to change without notice.
> _______________________________________________
> OmniOS-discuss mailing list
> OmniOS-discuss at lists.omniti.com
> http://lists.omniti.com/mailman/listinfo/omnios-discuss


From lotheac at iki.fi  Thu Jul 30 12:17:52 2015
From: lotheac at iki.fi (Lauri Tirkkonen)
Date: Thu, 30 Jul 2015 15:17:52 +0300
Subject: [OmniOS-discuss] pkg.niksula.hut.fi packages now signed
Message-ID: <20150730121752.GA16540@gutsman.lotheac.fi>

A heads up to anyone using our unofficial package repo at
pkg.niksula.hut.fi: we've started signing our packages and have
retroactively signed all our previously published packages.

Unfortunately pkg(1) refuses to install signed-but-not-verifiable
packages by default, even though it has no problems if the signatures
are not present. This means that any users of our repo will need to
configure their publishers to either trust our signing certificate or to
configure the per-publisher signature policy to ignore.

Here's a quick guide: http://pkg.niksula.hut.fi/static/index.html

-- 
Lauri Tirkkonen | lotheac @ IRCnet

From cal-s at blue-bolt.com  Fri Jul 31 08:48:34 2015
From: cal-s at blue-bolt.com (Cal Sawyer)
Date: Fri, 31 Jul 2015 09:48:34 +0100
Subject: [OmniOS-discuss] Missing readline module in r151014
In-Reply-To: <mailman.341.1438207694.63303.omnios-discuss@lists.omniti.com>
References: <mailman.341.1438207694.63303.omnios-discuss@lists.omniti.com>
Message-ID: <55BB3662.1060301@blue-bolt.com>

Hi

Setting up a new server on r151014 and found that a python script that 
ran on r151010 fails with this old python-chestnut:

    Traceback (most recent call last):
       File "/root/diskmap.py", line 9, in <module>
         import subprocess, re, os, sys, readline, cmd, pickle, glob
    ImportError: No module named readline


It appears that 014's python package no longer includes 
lib-dynload/[64/]readline.so. Searching on python packages containing 
readline.so turns up

pkg:/omniti/runtime/python-26 at 2.6.7-0.151008   ms.omniti.com
pkg:/omniti/runtime/python-27 at 2.7.6-0.151006   ms.omniti.com
pkg:/omniti/runtime/python-27 at 2.7.6-0.151008   ms.omniti.com
pkg:/omniti/runtime/python-34 at 3.4.0-0.151006   ms.omniti.com
pkg:/omniti/runtime/python-34 at 3.4.0-0.151010   ms.omniti.com
pkg:/omniti/runtime/python-34 at 3.4.3-0.151014   ms.omniti.com

yet

 > python -V
Python 2.6.8

No python 2.x.x for 014 (and 010) at all in omniti and only 3.x.x? Am i 
missing something? I would have expected to see 2.6.8-0.151014 as that's 
what is installed

Cal Sawyer | Systems Engineer | BlueBolt Ltd

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20150731/d1c92c89/attachment.html>

From danmcd at omniti.com  Fri Jul 31 15:18:14 2015
From: danmcd at omniti.com (Dan McDonald)
Date: Fri, 31 Jul 2015 11:18:14 -0400
Subject: [OmniOS-discuss] Missing readline module in r151014
In-Reply-To: <55BB3662.1060301@blue-bolt.com>
References: <mailman.341.1438207694.63303.omnios-discuss@lists.omniti.com>
	<55BB3662.1060301@blue-bolt.com>
Message-ID: <84AEBE40-A189-4229-8227-9A3792ED271F@omniti.com>


> On Jul 31, 2015, at 4:48 AM, Cal Sawyer <cal-s at blue-bolt.com> wrote:
> 
> Hi
> 
> Setting up a new server on r151014 and found that a python script that ran on r151010 fails with this old python-chestnut:
> Traceback (most recent call last):
>   File "/root/diskmap.py", line 9, in <module>
>     import subprocess, re, os, sys, readline, cmd, pickle, glob
> ImportError: No module named readline
> 
> It appears that 014's python package no longer includes lib-dynload/[64/]readline.so. Searching on python packages containing readline.so turns up

012's doesn't either.

When 012 updated to readline6.3, it broke the building of that python module.  Oddly enough, this never halted the python build process... it just kept going.  Not sure why that's the case, but that's the problem.

I'm working on a fix right now.  Please stay tuned.

Dan


From danmcd at omniti.com  Fri Jul 31 15:32:04 2015
From: danmcd at omniti.com (Dan McDonald)
Date: Fri, 31 Jul 2015 11:32:04 -0400
Subject: [OmniOS-discuss] Missing readline module in r151014
In-Reply-To: <84AEBE40-A189-4229-8227-9A3792ED271F@omniti.com>
References: <mailman.341.1438207694.63303.omnios-discuss@lists.omniti.com>
	<55BB3662.1060301@blue-bolt.com>
	<84AEBE40-A189-4229-8227-9A3792ED271F@omniti.com>
Message-ID: <EA85118D-CC16-4919-A3FE-D66F4CE25E7C@omniti.com>


> On Jul 31, 2015, at 11:18 AM, Dan McDonald <danmcd at omniti.com> wrote:
> 
> I'm working on a fix right now.  Please stay tuned.

I have a tentative fix.  Cal --> Please contact me unicast, since you're the ideal person to test the fix.

Thanks,
Dan


From danmcd at omniti.com  Fri Jul 31 16:47:49 2015
From: danmcd at omniti.com (Dan McDonald)
Date: Fri, 31 Jul 2015 12:47:49 -0400
Subject: [OmniOS-discuss] Missing readline module in r151014
In-Reply-To: <EA85118D-CC16-4919-A3FE-D66F4CE25E7C@omniti.com>
References: <mailman.341.1438207694.63303.omnios-discuss@lists.omniti.com>
	<55BB3662.1060301@blue-bolt.com>
	<84AEBE40-A189-4229-8227-9A3792ED271F@omniti.com>
	<EA85118D-CC16-4919-A3FE-D66F4CE25E7C@omniti.com>
Message-ID: <BD7587A8-6386-434A-BB0B-AF2C48CD03A3@omniti.com>


> On Jul 31, 2015, at 11:32 AM, Dan McDonald <danmcd at omniti.com> wrote:
> 
> I have a tentative fix.  Cal --> Please contact me unicast, since you're the ideal person to test the fix.

Thanks to Cal for confirming the tactical value of this fix:

commit 02f0aad7c7cc88c6048c13d18e573839caabe038
Author: Dan McDonald <danmcd at omniti.com>
Date:   Fri Jul 31 15:25:59 2015 +0000

    Stock OmniOS Python needs to cope with readline 6.3


It's in master now. I'll be running it on bloody locally to make sure nothing else goes fubar.  Then I will backport it to r151014 and update runtime/python-26 with this as well.

The bug was introduced in r151012, when I bumped readline to 6.3.  Why python's build didn't just fail when readline wouldn't build is beyond me, but it took this long to find it out.  I'm sorry I didn't find it sooner.

Thanks,
Dan


From hakansom at ohsu.edu  Fri Jul 31 20:19:44 2015
From: hakansom at ohsu.edu (Marion Hakanson)
Date: Fri, 31 Jul 2015 13:19:44 -0700
Subject: [OmniOS-discuss] LX Branded Zones
In-Reply-To: Message from Dan McDonald <danmcd@omniti.com> of "Wed,
	29 Jul 2015 15:09:12 EDT."
	<CF4D34D8-5AE7-4BFA-904C-F66146445FAD@omniti.com>
Message-ID: <201507312019.t6VKJiZ1008125@kyklops.ohsu.edu>

Dan,

I tested out Brand-Z zones back in the days before they were removed,
and as I recall there was just a "brand:" field in the zonecfg, where
you placed something (an "lx"?) value for such zones.  Maybe you won't
need to reinvent that wheel particular wheel?

Regards,

Marion


==============================================================
Subject: Re: [OmniOS-discuss] LX Branded Zones
From: Dan McDonald <danmcd at omniti.com>
Date: Wed, 29 Jul 2015 15:09:12 -0400
To: Kyle Bruene <KBruene at simmonsperrine.com>
Cc: "omnios-discuss at lists.omniti.com" <omnios-discuss at lists.omniti.com>

> On Jul 29, 2015, at 2:54 PM, Kyle Bruene <KBruene at simmonsperrine.com> wrote:
> 
> Are there any plans to make LX branded zones available in OmniOS, or will this remain a SmartOS-only Illumos feature?

There are plans.  Not sure of all of their details yet, but there are plans.  
I can only say with certainty that they will NOT be in r151016 (the next 
stable).

The largest consideration is the admin model.  We don't have vmadm(1M) in 
OmniOS, and to do so would mean porting over large chunks of SmartOS and/or 
SmartDC, which I'd rather not do.  So we need to consider what a zonecfg(1M) 
model for LX zone would look like.  Admins in the audience should think about 
this as well.

Dan



From danmcd at omniti.com  Fri Jul 31 20:44:58 2015
From: danmcd at omniti.com (Dan McDonald)
Date: Fri, 31 Jul 2015 16:44:58 -0400
Subject: [OmniOS-discuss] LX Branded Zones
In-Reply-To: <201507312019.t6VKJiZ1008125@kyklops.ohsu.edu>
References: <201507312019.t6VKJiZ1008125@kyklops.ohsu.edu>
Message-ID: <6C5CEF76-0158-4EB0-B67D-FAB3720B86FF@omniti.com>


> On Jul 31, 2015, at 4:19 PM, Marion Hakanson <hakansom at ohsu.edu> wrote:
> 
> Maybe you won't
> need to reinvent that wheel particular wheel?

I will need to reinvent that wheel because of installation.  Where do I get bits to do "zoneadm -z lx-zone install" from, for example?

I like the idea of pre-built supported images (which are ZFS snapshots, or modified ZFS snapshot for vmadm(1M) in SmartOS).  I'm imagining a few additional zone properties for an lx zone.  (While I have that patient open, I may also do a zoneadm/zonecfg-style "KVM" zone as well, depending...)

Thanks!
Dan


From hakansom at ohsu.edu  Fri Jul 31 21:01:23 2015
From: hakansom at ohsu.edu (Marion Hakanson)
Date: Fri, 31 Jul 2015 14:01:23 -0700
Subject: [OmniOS-discuss] LX Branded Zones
In-Reply-To: Message from Dan McDonald <danmcd@omniti.com> of "Fri,
	31 Jul 2015 16:44:58 EDT."
	<6C5CEF76-0158-4EB0-B67D-FAB3720B86FF@omniti.com>
Message-ID: <201507312101.t6VL1NDl008281@kyklops.ohsu.edu>

danmcd at omniti.com said:
> I will need to reinvent that wheel because of installation.  Where do I get
> bits to do "zoneadm -z lx-zone install" from, for example?
> 
> I like the idea of pre-built supported images (which are ZFS snapshots, or
> modified ZFS snapshot for vmadm(1M) in SmartOS).  I'm imagining a few
> additional zone properties for an lx zone.  (While I have that patient open,
> I may also do a zoneadm/zonecfg-style "KVM" zone as well, depending...) 

It's been a long time since I did this, but I recall that there were
pre-built Linux images that you pointed to at install time.

I still have a Solaris-10_x86 branded zone that runs on an OpenIndiana
machine.  I think it was necessary to create a CPIO image (or maybe a
flar image?) of a Solaris-10_x86 system (metal) in order to create
that zone.  Again, recalling from long-paged-out neurons here....

But the image management was done outside of zonecfg/zoneadm.

Regards,

Marion



