# [OmniOS-discuss] [developer] NVMe Performance

Josh Coombs jcoombs at staff.gwi.net
Sat Apr 16 02:24:44 UTC 2016

- Previous message: [OmniOS-discuss] [developer] NVMe Performance
- Next message: [OmniOS-discuss] [developer] NVMe Performance
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

    On Fri, Apr 15, 2016 at 9:26 PM, Richard Yao <ryao at gentoo.org> wrote:

    >
    > The first is to make sure that ZFS uses proper alignment on the device.
    > According to what I learned via Google searches, the Intel DC P3600
    > supports both 512-byte sectors and 4096-byte sectors, but is low leveled
    > formatted to 512-byte sectors by default. You could run fio to see how the
    > random IO performance differs on 512-byte IOs at 512-byte formatting vs 4KB
    > IOs at 4KB formatting, but I expect that you will find it performs best in
    > the 4KB case like Intel's enterprise SATA SSDs do. If the 512-byte random
    > IO performance was notable, Intel would have advertised it, but they did
    > not do that:
    >
    >
    > http://www.intel.com/content/dam/www/public/us/en/documents/product-specifications/ssd-dc-p3600-spec.pdf
    >
    > http://www.cadalyst.com/%5Blevel-1-with-primary-path%5D/how-configure-oracle-redo-intel-pcie-ssd-dc-p3700-23534
    >
    > So, I played around with this.  Intel's isdct tool will let you secure
    erase the P3600 and set it up as a 4k sector device, or a 512, with a few
    other options as well.  I have to re-look but it might support 8k sectors
    too.  Unfortunately the NVMe driver doesn't play well with the SSD
    formatted for anything other than 512 byte sectors.  I noted my findings in
    Illumos bug #6912.

    I need to look at how Illumos partitions the devices if you just feed zpool
    the device rather than a partition, I didn't look to see if it was aligning
    things correctly or not on it's own.

    The second is that it is possible to increase IOPS beyond Intel's
    > specifications by doing a secure erase, giving SLOG a tiny 4KB aligned
    > partition and leaving the rest of the device unused. Intel's numbers are
    > for steady state performance where almost every flash page is dirty. If you
    > leave a significant number of pages clean (i.e. unused following a secure
    > erase), the drive should perform better than what Intel claims by virtue of
    > the internal book keeping and garbage collection having to do less. Anandtech
    > has benchmarks numbers showing this effect on older consumer SSDs on
    > Windows in a comparison with the Intel DC S3700:
    >

    Using isdct I have mine set to 50% over-provisioning, so they show up as
    200GB devices now.  As noted in bug 6912 you have to secure erase after
    changing that setting or the NVMe driver REALLY gets unhappy.

    Josh C
    -------------- next part --------------
    An HTML attachment was scrubbed...
    URL: <http://lists.omniti.com/pipermail/omnios-discuss/attachments/20160415/4a51164f/attachment.html>

- Previous message: [OmniOS-discuss] [developer] NVMe Performance
- Next message: [OmniOS-discuss] [developer] NVMe Performance
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

More information about the OmniOS-discuss mailing list