NEX-4794 Write Back Cache sync and async writes: adjust routing according to watermark limits
Reviewed by: Alex Aizman <alex.aizman@nexenta.com>
Reviewed by: Alek Pinchuk <alek.pinchuk@nexenta.com>
5269 zpool import slow
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: George Wilson <george@delphix.com>
Reviewed by: Dan McDonald <danmcd@omniti.com>
Approved by: Dan McDonald <danmcd@omniti.com>

*** 351,365 ****
   *    In this mode, if we need to commit the write later, then the block
   *    is immediately written into the file system (using dmu_sync),
   *    and a pointer to the block is put into the log record.
   *    When the txg commits the block is linked in.
   *    This saves additionally writing the data into the log record.
!  *    There are a few requirements for this to occur:
!  *      - write is greater than zfs/zvol_immediate_write_sz
!  *      - not using slogs (as slogs are assumed to always be faster
!  *        than writing into the main pool)
!  *      - the write occupies only one block
   * WR_COPIED:
   *    If we know we'll immediately be committing the
   *    transaction (FSYNC or FDSYNC), the we allocate a larger
   *    log record here for the data and copy the data in.
   * WR_NEED_COPY:
--- 351,366 ----
   *    In this mode, if we need to commit the write later, then the block
   *    is immediately written into the file system (using dmu_sync),
   *    and a pointer to the block is put into the log record.
   *    When the txg commits the block is linked in.
   *    This saves additionally writing the data into the log record.
!  *    There are a few requirements for this to occur. In general, WR_INDIRECT
!  *    with the subsequent dmu_sync-ing of the data directly into the filesystem
!  *    is used in the two following separate cases:
!  *      - logbias = THROUGHPUT mode
!  *      - writeback cache (via special vdev) when there is no separate slog
!  *    For specific details and configurable tunables - see zfs_log_write().
   * WR_COPIED:
   *    If we know we'll immediately be committing the
   *    transaction (FSYNC or FDSYNC), the we allocate a larger
   *    log record here for the data and copy the data in.
   * WR_NEED_COPY:
*** 417,427 ****
  extern void     zil_itx_assign(zilog_t *zilog, itx_t *itx, dmu_tx_t *tx);
  
  extern void     zil_commit(zilog_t *zilog, uint64_t oid);
  extern void     zil_commit_impl(zilog_t *zilog, uint64_t oid);
  
! extern int      zil_reset(const char *osname, void *txarg);
  extern int      zil_claim(struct dsl_pool *dp,
      struct dsl_dataset *ds, void *txarg);
  extern int      zil_check_log_chain(struct dsl_pool *dp,
      struct dsl_dataset *ds, void *tx);
  extern void     zil_sync(zilog_t *zilog, dmu_tx_t *tx);
--- 418,428 ----
  extern void     zil_itx_assign(zilog_t *zilog, itx_t *itx, dmu_tx_t *tx);
  
  extern void     zil_commit(zilog_t *zilog, uint64_t oid);
  extern void     zil_commit_impl(zilog_t *zilog, uint64_t oid);
  
! extern int      zil_vdev_offline(const char *osname, void *txarg);
  extern int      zil_claim(struct dsl_pool *dp,
      struct dsl_dataset *ds, void *txarg);
  extern int      zil_check_log_chain(struct dsl_pool *dp,
      struct dsl_dataset *ds, void *tx);
  extern void     zil_sync(zilog_t *zilog, dmu_tx_t *tx);