Print this page
NEX-4856 SMB2 kstats don't correctly count compound requests
Reviewed by: Gordon Ross <gordon.ross@nexenta.com>
Reviewed by: Evan Layton <evan.layton@nexenta.com>
NEX-19944 SMB2 server should require signed Validate Negotiate requests
Reviewed by: Gordon Ross <gordon.ross@nexenta.com>
Reviewed by: Evan Layton <evan.layton@nexenta.com>
NEX-17589 Get "too high" smbd error when copy big file to cifs share (redo)
Reviewed by: Matt Barden <matt.barden@nexenta.com>
Reviewed by: Evan Layton <evan.layton@nexenta.com>
NEX-16943 network outages with thread stuck in smb2_scoreboard_cancel
Reviewed by: Matt Barden <matt.barden@nexenta.com>
Reviewed by: Evan Layton <evan.layton@nexenta.com>
NEX-9808 SMB3 persistent handles
Reviewed by: Matt Barden <matt.barden@nexenta.com>
Reviewed by: Evan Layton <evan.layton@nexenta.com>
NEX-15578 SMB2 durable handle redesign
Reviewed by: Matt Barden <matt.barden@nexenta.com>
Reviewed by: Evan Layton <evan.layton@nexenta.com>
NEX-5665 SMB2 oplock leases
Reviewed by: Matt Barden <matt.barden@nexenta.com>
Reviewed by: Evan Layton <evan.layton@nexenta.com>
Reviewed by: Roman Strashkin <roman.strashkin@nexenta.com>
NEX-15069 smtorture smb2.create.blob is failed
Reviewed by: Matt Barden <matt.barden@nexenta.com>
Reviewed by: Evan Layton <evan.layton@nexenta.com>
NEX-15555 SMB2 async redesign
NEX-15061 smtorture smb2.lock.cancel.cancel is failed
Reviewed by: Matt Barden <matt.barden@nexenta.com>
Reviewed by: Evan Layton <evan.layton@nexenta.com>
Reviewed by: Roman Strashkin <roman.strashkin@nexenta.com>
Also follow-up change to:
 NEX-1643 dtrace provider for smbsrv (remove "done2" probes,
 which don't make sense with the new async design)
NEX-9808 SMB3 persistent handles
Reviewed by: Matt Barden <matt.barden@nexenta.com>
Reviewed by: Evan Layton <evan.layton@nexenta.com>
NEX-15578 SMB2 durable handle redesign
Reviewed by: Matt Barden <matt.barden@nexenta.com>
Reviewed by: Evan Layton <evan.layton@nexenta.com>
NEX-5665 SMB2 oplock leases
Reviewed by: Matt Barden <matt.barden@nexenta.com>
Reviewed by: Evan Layton <evan.layton@nexenta.com>
Reviewed by: Roman Strashkin <roman.strashkin@nexenta.com>
NEX-15069 smtorture smb2.create.blob is failed
Reviewed by: Matt Barden <matt.barden@nexenta.com>
Reviewed by: Evan Layton <evan.layton@nexenta.com>
NEX-15555 SMB2 async redesign
NEX-15061 smtorture smb2.lock.cancel.cancel is failed
Reviewed by: Matt Barden <matt.barden@nexenta.com>
Reviewed by: Evan Layton <evan.layton@nexenta.com>
Reviewed by: Roman Strashkin <roman.strashkin@nexenta.com>
Also follow-up change to:
 NEX-1643 dtrace provider for smbsrv (remove "done2" probes,
 which don't make sense with the new async design)
NEX-1643 dtrace provider for smbsrv
Reviewed by: Evan Layton <evan.layton@nexenta.com>
Reviewed by: Matt Barden <matt.barden@nexenta.com>
NEX-9864 Some SMB cancel races remain after NEX-5845 (smb3 nit)
NEX-9864 Some SMB cancel races remain after NEX-5845
Revert (part of) "NEX-5845 rework SMB immediate cancel"
reverts (part of) commit 7a5da69f6d42b17ebcc95ca3d02925d07a01343e.
Reviewed by: Matt Barden <matt.barden@nexenta.com>
Reviewed by: Evan Layton <evan.layton@nexenta.com>
NEX-5273 SMB 3 Encryption
Reviewed by: Gordon Ross <gordon.ross@nexenta.com>
Reviewed by: Evan Layton <evan.layton@nexenta.com>
Reviewed by: Roman Strashkin <roman.strashkin@nexenta.com>
NEX-5844 want SMB2 ioctl FSCTL_SRV_COPYCHUNK
NEX-6124 smb_fsop_read/write should allow file != sr->fid_ofile
NEX-6125 smbtorture invalid response with smb2.ioctl
Reviewed by: Evan Layton <evan.layton@nexenta.com>
Reviewed by: Matt Barden <matt.barden@nexenta.com>
NEX-5845 rework SMB immediate cancel
Reviewed by: Matt Barden <matt.barden@nexenta.com>
Reviewed by: Kevin Crowe <kevin.crowe@nexenta.com>
NEX-5586 SMB2 ofiles need real Persistent IDs
NEX-5313 SMB2 oplock break notification should use TID=0
Reviewed by: Gordon Ross <gwr@nexenta.com>
NEX-5560 smb2 should use 64-bit server-global uids
Reviewed by: Gordon Ross <gwr@nexenta.com>
NEX-3906 Prefer that SMB change notify not tie up a worker thread
NEX-5278 SMB notify should buffer per file handle
Reviewed by: Kevin Crowe <kevin.crowe@nexenta.com>
Reviewed by: Matt Barden <Matt.Barden@nexenta.com>
NEX-5152 immediate SMB cancel may fail
Reviewed by: Kevin Crowe <kevin.crowe@nexenta.com>
Reviewed by: Matt Barden <Matt.Barden@nexenta.com>
NEX-5175 want SMB statistics separately for reads, writes, other
Reviewed by: Gordon Ross <gwr@nexenta.com>
Reviewed by: Matt Barden <Matt.Barden@nexenta.com>
NEX-5198 smb2 kernel kstats incorrect for received data
Reviewed by: Kevin Crowe <kevin.crowe@nexenta.com>
NEX-5133 Deleting directory over CIFS SMB2 fails after visiting in explorer
Reviewed by: Kevin Crowe <kevin.crowe@nexenta.com>
Reviewed by: Matt Barden <Matt.Barden@nexenta.com>
NEX-4313 want iops, bandwidth, and latency kstats for smb
Portions contributed by: Gordon Ross <gwr@nexenta.com>
Reviewed by: Matt Barden <Matt.Barden@nexenta.com>
Reviewed by: Gordon Ross <gordon.ross@nexenta.com>
NEX-4598 SMB2 credit shortage with Mac client
Reviewed by: Bayard Bell <bayard.bell@nexenta.com>
Reviewed by: Kevin Crowe <kevin.crowe@nexenta.com>
Reviewed by: Matt Barden <Matt.Barden@nexenta.com>
NEX-4391 improve smb cancel support
Reviewed by: Matt Barden <Matt.Barden@nexenta.com>
NEX-4147 Cancelled SMB2 requests sometimes have no response (missed things)
NEX-4147 Cancelled SMB2 requests sometimes have no response
NEX-4157 Improve SMB2 compound request handler
Reviewed by: Matt Barden <Matt.Barden@nexenta.com>
NEX-1999 SMB2 panic - missing tree connection
NEX-3377 Want reply header set earlier
Reviewed by: Dan Fields <dan.fields@nexenta.com>
Reviewed by: Kevin Crowe <kevin.crowe@nexenta.com>
Reviewed by: Matt Barden <Matt.Barden@nexenta.com>
NEX-3310 smbstat misreports change notify
Reviewed by: Alek Pinchuk <alek.pinchuk@nexenta.com>
Reviewed by: Dan Fields <dan.fields@nexenta.com>
NEX-3080 SMB1 signing problem with Kerberos auth.
Reviewed by: Bayard Bell <bayard.bell@nexenta.com>
Reviewed by: Dan Fields <dan.fields@nexenta.com>
Reviewed by: Kevin Crowe <kevin.crowe@nexenta.com>
Reviewed by: Matt Barden <Matt.Barden@nexenta.com>
NEX-2869 SMB2 signing fails for multi-user clients like Citrix RDS
NEX-2975 SMB2 Cancel may fail
Reviewed by: Kevin Crowe <kevin.crowe@nexenta.com>
Reviewed by: Tony Nguyen <tony.nguyen@nexenta.com>
NEX-2781 SMB2 credit handling needs work
NEX-2353 Codenomicon: SMB2 TC # 448950 - PANIC in SMB2.Compounded-commands...
NEX-2344 Codenomicon: SMB2 TC: 458439 Panic in SMB2.Compounded-commands
SMB-55 SMB2 signing
SMB-122 smbd core dumps in smbd_dc_update / smb_log
SMB-117 Win7 fails to open security properties
SMB-110 panic mapping a share from Nexentastor to the windows 2012 R2 client
SMB-109 Codenomicon: SMB TC: 409480 - Panic with SMB2_FIND request
SMB-79 Codenomicon: SMB2 TC: 4978 - Panic in smb_latency_add_sample
SMB-11 SMB2 message parse & dispatch
SMB-12 SMB2 Negotiate Protocol
SMB-13 SMB2 Session Setup
SMB-14 SMB2 Logoff
SMB-15 SMB2 Tree Connect
SMB-16 SMB2 Tree Disconnect
SMB-17 SMB2 Create
SMB-18 SMB2 Close
SMB-19 SMB2 Flush
SMB-20 SMB2 Read
SMB-21 SMB2 Write
SMB-22 SMB2 Lock/Unlock
SMB-23 SMB2 Ioctl
SMB-24 SMB2 Cancel
SMB-25 SMB2 Echo
SMB-26 SMB2 Query Dir
SMB-27 SMB2 Change Notify
SMB-28 SMB2 Query Info
SMB-29 SMB2 Set Info
SMB-30 SMB2 Oplocks
SMB-53 SMB2 Create Context options
(SMB2 code review cleanup 1, 2, 3)
        
*** 8,74 ****
   * source.  A copy of the CDDL is also available via the Internet at
   * http://www.illumos.org/license/CDDL.
   */
  
  /*
!  * Copyright 2015 Nexenta Systems, Inc.  All rights reserved.
   */
  
  
  #include <smbsrv/smb2_kproto.h>
  #include <smbsrv/smb_kstat.h>
  #include <smbsrv/smb2.h>
  
! /*
!  * Saved state for a command that "goes async".  When a compound request
!  * contains a command that may block indefinitely, the compound reply is
!  * composed with an "interim response" for that command, and information
!  * needed to actually dispatch that command is saved on a list of "async"
!  * commands for this compound request.  After the compound reply is sent,
!  * the list of async commands is processed, and those may block as long
!  * as they need to without affecting the initial compound request.
!  *
!  * Now interestingly, this "async" mechanism is not used with the full
!  * range of asynchrony that one might imagine.  The design of async
!  * request processing can be drastically simplified if we can assume
!  * that there's no need to run more than one async command at a time.
!  * With that simplifying assumption, we can continue using the current
!  * "one worker thread per request message" model, which has very simple
!  * locking rules etc.  The same worker thread that handles the initial
!  * compound request can handle the list of async requests.
!  *
!  * As it turns out, SMB2 clients do not try to use more than one "async"
!  * command in a compound.  If they were to do so, the [MS-SMB2] spec.
!  * allows us to decline additional async requests with an error.
!  *
!  * smb_async_req_t is the struct used to save an "async" request on
!  * the list of requests that had an interim reply in the initial
!  * compound reply.  This includes everything needed to restart
!  * processing at the async command.
!  */
  
- typedef struct smb2_async_req {
- 
-         smb_sdrc_t              (*ar_func)(smb_request_t *);
- 
-         int ar_cmd_hdr;         /* smb2_cmd_hdr offset */
-         int ar_cmd_len;         /* length from hdr */
- 
-         /*
-          * SMB2 header fields.
-          */
-         uint16_t                ar_cmd_code;
-         uint16_t                ar_uid;
-         uint16_t                ar_tid;
-         uint32_t                ar_pid;
-         uint32_t                ar_hdr_flags;
-         uint64_t                ar_messageid;
- } smb2_async_req_t;
- 
- void smb2sr_do_async(smb_request_t *);
  smb_sdrc_t smb2_invalid_cmd(smb_request_t *);
  static void smb2_tq_work(void *);
  
  static const smb_disp_entry_t const
  smb2_disp_table[SMB2__NCMDS] = {
  
          /* text-name, pre, func, post, cmd-code, dialect, flags */
--- 8,31 ----
   * source.  A copy of the CDDL is also available via the Internet at
   * http://www.illumos.org/license/CDDL.
   */
  
  /*
!  * Copyright 2019 Nexenta Systems, Inc.  All rights reserved.
   */
  
  
  #include <smbsrv/smb2_kproto.h>
  #include <smbsrv/smb_kstat.h>
  #include <smbsrv/smb2.h>
  
! #define SMB2_ASYNCID(sr) (sr->smb2_messageid ^ (1ULL << 62))
  
  smb_sdrc_t smb2_invalid_cmd(smb_request_t *);
  static void smb2_tq_work(void *);
+ static void smb2sr_run_postwork(smb_request_t *);
+ static int smb3_decrypt_msg(smb_request_t *);
  
  static const smb_disp_entry_t const
  smb2_disp_table[SMB2__NCMDS] = {
  
          /* text-name, pre, func, post, cmd-code, dialect, flags */
*** 111,129 ****
              smb2_lock, NULL, 0, 0 },
  
          {  "smb2_ioctl", NULL,
              smb2_ioctl, NULL, 0, 0 },
  
-         /*
-          * Note: Cancel gets the "invalid command" handler because
-          * that's always handled directly in the reader.  We should
-          * never get to the function using this table, but note:
-          * We CAN get here if a nasty client adds cancel to some
-          * compound message, which is a protocol violation.
-          */
          {  "smb2_cancel", NULL,
!             smb2_invalid_cmd, NULL, 0, 0 },
  
          {  "smb2_echo", NULL,
              smb2_echo, NULL, 0, 0,
              SDDF_SUPPRESS_UID | SDDF_SUPPRESS_TID },
  
--- 68,80 ----
              smb2_lock, NULL, 0, 0 },
  
          {  "smb2_ioctl", NULL,
              smb2_ioctl, NULL, 0, 0 },
  
          {  "smb2_cancel", NULL,
!             smb2_cancel, NULL, 0, 0,
!             SDDF_SUPPRESS_UID | SDDF_SUPPRESS_TID },
  
          {  "smb2_echo", NULL,
              smb2_echo, NULL, 0, 0,
              SDDF_SUPPRESS_UID | SDDF_SUPPRESS_TID },
  
*** 173,219 ****
   * will drop the session.
   */
  int
  smb2sr_newrq(smb_request_t *sr)
  {
          uint32_t magic;
!         uint16_t command;
!         int rc;
  
!         magic = LE_IN32(sr->sr_request_buf);
!         if (magic != SMB2_PROTOCOL_MAGIC) {
!                 smb_request_free(sr);
!                 /* will drop the connection */
!                 return (EPROTO);
          }
  
          /*
!          * Execute Cancel requests immediately, (here in the
!          * reader thread) so they won't wait for any other
!          * commands we might already have in the task queue.
!          * Cancel also skips signature verification and
!          * does not consume a sequence number.
!          * [MS-SMB2] 3.2.4.24 Cancellation...
           */
!         command = LE_IN16((uint8_t *)sr->sr_request_buf + 12);
!         if (command == SMB2_CANCEL) {
!                 rc = smb2sr_newrq_cancel(sr);
                  smb_request_free(sr);
                  return (rc);
          }
  
          /*
           * Submit the request to the task queue, which calls
           * smb2_tq_work when the workload permits.
           */
          sr->sr_time_submitted = gethrtime();
          sr->sr_state = SMB_REQ_STATE_SUBMITTED;
          smb_srqueue_waitq_enter(sr->session->s_srqueue);
          (void) taskq_dispatch(sr->sr_server->sv_worker_pool,
              smb2_tq_work, sr, TQ_SLEEP);
- 
          return (0);
  }
  
  static void
  smb2_tq_work(void *arg)
  {
--- 124,233 ----
   * will drop the session.
   */
  int
  smb2sr_newrq(smb_request_t *sr)
  {
+         struct mbuf_chain *mbc = &sr->command;
          uint32_t magic;
!         int rc, skip;
  
!         if (smb_mbc_peek(mbc, 0, "l", &magic) != 0)
!                 goto drop;
! 
!         /* 0xFD S M B */
!         if (magic == SMB3_ENCRYPTED_MAGIC) {
!                 if (smb3_decrypt_msg(sr) != 0)
!                         goto drop;
!                 /*
!                  * Should now be looking at an un-encrypted
!                  * SMB2 message header.
!                  */
!                 if (smb_mbc_peek(mbc, 0, "l", &magic) != 0)
!                         goto drop;
          }
  
+         if (magic != SMB2_PROTOCOL_MAGIC)
+                 goto drop;
+ 
          /*
!          * Walk the SMB2 commands in this compound message and
!          * keep track of the range of message IDs it uses.
           */
!         for (;;) {
!                 if (smb2_decode_header(sr) != 0)
!                         goto drop;
! 
!                 /*
!                  * Cancel requests are special:  They refer to
!                  * an earlier message ID (or an async. ID),
!                  * never a new ID, and are never compounded.
!                  * This is intentionally not "goto drop"
!                  * because rc may be zero (success).
!                  */
!                 if (sr->smb2_cmd_code == SMB2_CANCEL) {
!                         rc = smb2_newrq_cancel(sr);
                          smb_request_free(sr);
                          return (rc);
                  }
  
                  /*
+                  * Keep track of the total credits in this compound
+                  * and the first (real) message ID (not: 0, -1)
+                  * While we're looking, verify that all (real) IDs
+                  * are (first <= ID < (first + msg_credits))
+                  */
+                 if (sr->smb2_credit_charge == 0)
+                         sr->smb2_credit_charge = 1;
+                 sr->smb2_total_credits += sr->smb2_credit_charge;
+ 
+                 if (sr->smb2_messageid != 0 &&
+                     sr->smb2_messageid != UINT64_MAX) {
+ 
+                         if (sr->smb2_first_msgid == 0)
+                                 sr->smb2_first_msgid = sr->smb2_messageid;
+ 
+                         if (sr->smb2_messageid < sr->smb2_first_msgid ||
+                             sr->smb2_messageid >= (sr->smb2_first_msgid +
+                             sr->smb2_total_credits)) {
+                                 long long id = (long long) sr->smb2_messageid;
+                                 cmn_err(CE_WARN, "clnt %s msg ID 0x%llx "
+                                     "out of sequence in compound",
+                                     sr->session->ip_addr_str, id);
+                         }
+                 }
+ 
+                 /* Normal loop exit on next == zero */
+                 if (sr->smb2_next_command == 0)
+                         break;
+ 
+                 /* Abundance of caution... */
+                 if (sr->smb2_next_command < SMB2_HDR_SIZE)
+                         goto drop;
+ 
+                 /* Advance to the next header. */
+                 skip = sr->smb2_next_command - SMB2_HDR_SIZE;
+                 if (MBC_ROOM_FOR(mbc, skip) == 0)
+                         goto drop;
+                 mbc->chain_offset += skip;
+         }
+         /* Rewind back to the top. */
+         mbc->chain_offset = 0;
+ 
+         /*
           * Submit the request to the task queue, which calls
           * smb2_tq_work when the workload permits.
           */
          sr->sr_time_submitted = gethrtime();
          sr->sr_state = SMB_REQ_STATE_SUBMITTED;
          smb_srqueue_waitq_enter(sr->session->s_srqueue);
          (void) taskq_dispatch(sr->sr_server->sv_worker_pool,
              smb2_tq_work, sr, TQ_SLEEP);
          return (0);
+ 
+ drop:
+         smb_request_free(sr);
+         return (-1);
  }
  
  static void
  smb2_tq_work(void *arg)
  {
*** 227,246 ****
          smb_srqueue_waitq_to_runq(srq);
          sr->sr_worker = curthread;
          sr->sr_time_active = gethrtime();
  
          /*
!          * In contrast with SMB1, SMB2 must _always_ dispatch to
!          * the handler function, because cancelled requests need
!          * an error reply (NT_STATUS_CANCELLED).
           */
          smb2sr_work(sr);
  
          smb_srqueue_runq_exit(srq);
  }
  
  /*
   * smb2sr_work
   *
   * This function processes each SMB command in the current request
   * (which may be a compound request) building a reply containing
   * SMB reply messages, one-to-one with the SMB commands.  Some SMB
--- 241,456 ----
          smb_srqueue_waitq_to_runq(srq);
          sr->sr_worker = curthread;
          sr->sr_time_active = gethrtime();
  
          /*
!          * Always dispatch to the work function, because cancelled
!          * requests need an error reply (NT_STATUS_CANCELLED).
           */
+         mutex_enter(&sr->sr_mutex);
+         if (sr->sr_state == SMB_REQ_STATE_SUBMITTED)
+                 sr->sr_state = SMB_REQ_STATE_ACTIVE;
+         mutex_exit(&sr->sr_mutex);
+ 
          smb2sr_work(sr);
  
          smb_srqueue_runq_exit(srq);
  }
  
+ static int
+ smb3_decrypt_msg(smb_request_t *sr)
+ {
+         int save_offset;
+ 
+         if (sr->session->dialect < SMB_VERS_3_0) {
+                 cmn_err(CE_WARN, "encrypted message in SMB 2.x");
+                 return (-1);
+         }
+ 
+         sr->encrypted = B_TRUE;
+         save_offset = sr->command.chain_offset;
+         if (smb3_decode_tform_header(sr) != 0) {
+                 cmn_err(CE_WARN, "bad transform header");
+                 return (-1);
+         }
+         sr->command.chain_offset = save_offset;
+ 
+         sr->tform_ssn = smb_session_lookup_ssnid(sr->session,
+             sr->smb3_tform_ssnid);
+         if (sr->tform_ssn == NULL) {
+                 cmn_err(CE_WARN, "transform header: session not found");
+                 return (-1);
+         }
+ 
+         if (smb3_decrypt_sr(sr) != 0) {
+                 cmn_err(CE_WARN, "smb3 decryption failed");
+                 return (-1);
+         }
+ 
+         return (0);
+ }
+ 
  /*
+  * SMB2 credits determine how many simultaneous commands the
+  * client may issue, and bounds the range of message IDs those
+  * commands may use.  With multi-credit support, commands may
+  * use ranges of message IDs, where the credits used by each
+  * command are proportional to their data transfer size.
+  *
+  * Every command may request an increase or decrease of
+  * the currently granted credits, based on the difference
+  * between the credit request and the credit charge.
+  * [MS-SMB2] 3.3.1.2 Algorithm for the Granting of Credits
+  *
+  * Most commands have credit_request=1, credit_charge=1,
+  * which keeps the credit grant unchanged.
+  *
+  * All we're really doing here (for now) is reducing the
+  * credit_response if the client requests a credit increase
+  * that would take their credit over the maximum, and
+  * limiting the decrease so they don't run out of credits.
+  *
+  * Later, this could do something dynamic based on load.
+  *
+  * One other non-obvious bit about credits: We keep the
+  * session s_max_credits low until the 1st authentication,
+  * at which point we'll set the normal maximum_credits.
+  * Some clients ask for more credits with session setup,
+  * and we need to handle that requested increase _after_
+  * the command-specific handler returns so it won't be
+  * restricted to the lower (pre-auth) limit.
+  */
+ static inline void
+ smb2_credit_decrease(smb_request_t *sr)
+ {
+         smb_session_t *session = sr->session;
+         uint16_t cur, d;
+ 
+         mutex_enter(&session->s_credits_mutex);
+         cur = session->s_cur_credits;
+ 
+         /* Handle credit decrease. */
+         d = sr->smb2_credit_charge - sr->smb2_credit_request;
+         cur -= d;
+         if (cur & 0x8000) {
+                 /*
+                  * underflow (bad credit charge or request)
+                  * leave credits unchanged (response=charge)
+                  */
+                 cur = session->s_cur_credits;
+                 sr->smb2_credit_response = sr->smb2_credit_charge;
+                 DTRACE_PROBE1(smb2__credit__neg, smb_request_t *, sr);
+         }
+ 
+         /*
+          * The server MUST ensure that the number of credits
+          * held by the client is never reduced to zero.
+          * [MS-SMB2] 3.3.1.2
+          */
+         if (cur == 0) {
+                 cur = 1;
+                 sr->smb2_credit_response += 1;
+                 DTRACE_PROBE1(smb2__credit__min, smb_request_t *, sr);
+         }
+ 
+         DTRACE_PROBE3(smb2__credit__decrease,
+             smb_request_t *, sr, int, (int)cur,
+             int, (int)session->s_cur_credits);
+ 
+         session->s_cur_credits = cur;
+         mutex_exit(&session->s_credits_mutex);
+ }
+ 
+ /*
+  * Second half of SMB2 credit handling (increases)
+  */
+ static inline void
+ smb2_credit_increase(smb_request_t *sr)
+ {
+         smb_session_t *session = sr->session;
+         uint16_t cur, d;
+ 
+         mutex_enter(&session->s_credits_mutex);
+         cur = session->s_cur_credits;
+ 
+         /* Handle credit increase. */
+         d = sr->smb2_credit_request - sr->smb2_credit_charge;
+         cur += d;
+ 
+         /*
+          * If new credits would be above max,
+          * reduce the credit grant.
+          */
+         if (cur > session->s_max_credits) {
+                 d = cur - session->s_max_credits;
+                 cur = session->s_max_credits;
+                 sr->smb2_credit_response -= d;
+                 DTRACE_PROBE1(smb2__credit__max, smb_request_t, sr);
+         }
+ 
+         DTRACE_PROBE3(smb2__credit__increase,
+             smb_request_t *, sr, int, (int)cur,
+             int, (int)session->s_cur_credits);
+ 
+         session->s_cur_credits = cur;
+         mutex_exit(&session->s_credits_mutex);
+ }
+ 
+ /*
+  * Record some statistics:  latency, rx bytes, tx bytes
+  * per:  server, session & kshare.
+  */
+ static inline void
+ smb2_record_stats(smb_request_t *sr, smb_disp_stats_t *sds, boolean_t tx_only)
+ {
+         hrtime_t        dt;
+         int64_t         rxb;
+         int64_t         txb;
+         smb_disp_stats_t        *client_sds;
+         smb_disp_stats_t        *share_sds;
+         int                     cmd_type;
+         smb_session_t *session = sr->session;
+ 
+         if (sr->smb2_cmd_code == SMB2_READ) {
+                 cmd_type = SMBSRV_CLSH_READ;
+         } else if (sr->smb2_cmd_code == SMB2_WRITE) {
+                 cmd_type = SMBSRV_CLSH_WRITE;
+         } else {
+                 cmd_type = SMBSRV_CLSH_OTHER;
+         }
+ 
+         dt = gethrtime() - sr->sr_time_start;
+         rxb = (int64_t)(sr->command.chain_offset - sr->smb2_cmd_hdr);
+         txb = (int64_t)(sr->reply.chain_offset - sr->smb2_reply_hdr);
+ 
+         if (!tx_only) {
+                 smb_server_inc_req(sr->sr_server);
+                 smb_latency_add_sample(&sds->sdt_lat, dt);
+                 atomic_add_64(&sds->sdt_rxb, rxb);
+         }
+         atomic_add_64(&sds->sdt_txb, txb);
+ 
+         client_sds = &session->s_stats[cmd_type];
+         if (!tx_only) {
+                 smb_latency_add_sample(&client_sds->sdt_lat, dt);
+                 atomic_add_64(&client_sds->sdt_rxb, rxb);
+         }
+         atomic_add_64(&client_sds->sdt_txb, txb);
+ 
+         if ((sr->tid_tree != NULL) &&
+             (sr->tid_tree->t_kshare != NULL)) {
+                 share_sds =
+                     &sr->tid_tree->t_kshare->shr_stats[cmd_type];
+                 if (!tx_only) {
+                         smb_latency_add_sample(&share_sds->sdt_lat, dt);
+                         atomic_add_64(&share_sds->sdt_rxb, rxb);
+                 }
+                 atomic_add_64(&share_sds->sdt_txb, txb);
+         }
+ }
+ 
+ /*
   * smb2sr_work
   *
   * This function processes each SMB command in the current request
   * (which may be a compound request) building a reply containing
   * SMB reply messages, one-to-one with the SMB commands.  Some SMB
*** 257,267 ****
   * we need to keep track of the bounds of the current request
   * and reply.  For the request, this uses an MBC_SHADOW_CHAIN
   * that begins at smb2_cmd_hdr.  The reply is appended to the
   * sr->reply chain starting at smb2_reply_hdr.
   *
!  * This function must always free the smb request.
   */
  void
  smb2sr_work(struct smb_request *sr)
  {
          const smb_disp_entry_t  *sdd;
--- 467,478 ----
   * we need to keep track of the bounds of the current request
   * and reply.  For the request, this uses an MBC_SHADOW_CHAIN
   * that begins at smb2_cmd_hdr.  The reply is appended to the
   * sr->reply chain starting at smb2_reply_hdr.
   *
!  * This function must always free the smb request, or arrange
!  * for it to be completed and free'd later (if SDRC_SR_KEPT).
   */
  void
  smb2sr_work(struct smb_request *sr)
  {
          const smb_disp_entry_t  *sdd;
*** 273,327 ****
          boolean_t               disconnect = B_FALSE;
          boolean_t               related;
  
          session = sr->session;
  
          ASSERT(sr->tid_tree == 0);
          ASSERT(sr->uid_user == 0);
          ASSERT(sr->fid_ofile == 0);
          sr->smb_fid = (uint16_t)-1;
          sr->smb2_status = 0;
  
          /* temporary until we identify a user */
          sr->user_cr = zone_kcred();
  
-         mutex_enter(&sr->sr_mutex);
-         switch (sr->sr_state) {
-         case SMB_REQ_STATE_SUBMITTED:
-         case SMB_REQ_STATE_CLEANED_UP:
-                 sr->sr_state = SMB_REQ_STATE_ACTIVE;
-                 break;
-         default:
-                 ASSERT(0);
-                 /* FALLTHROUGH */
-         case SMB_REQ_STATE_CANCELED:
-                 sr->smb2_status = NT_STATUS_CANCELLED;
-                 break;
-         }
-         mutex_exit(&sr->sr_mutex);
- 
  cmd_start:
          /*
!          * Decode the request header
           *
-          * Most problems with decoding will result in the error
-          * STATUS_INVALID_PARAMETER.  If the decoding problem
-          * prevents continuing, we'll close the connection.
-          * [MS-SMB2] 3.3.5.2.6 Handling Incorrectly Formatted...
-          *
           * We treat some status codes as if "sticky", meaning
           * once they're set after some command handler returns,
           * all remaining commands get this status without even
!          * calling the command-specific handler. The cancelled
!          * status is used above, and insufficient_resources is
!          * used when smb2sr_go_async declines to "go async".
!          * Otherwise initialize to zero (success).
           */
          if (sr->smb2_status != NT_STATUS_CANCELLED &&
              sr->smb2_status != NT_STATUS_INSUFFICIENT_RESOURCES)
                  sr->smb2_status = 0;
  
          sr->smb2_cmd_hdr = sr->command.chain_offset;
          if ((rc = smb2_decode_header(sr)) != 0) {
                  cmn_err(CE_WARN, "clnt %s bad SMB2 header",
                      session->ip_addr_str);
                  disconnect = B_TRUE;
--- 484,526 ----
          boolean_t               disconnect = B_FALSE;
          boolean_t               related;
  
          session = sr->session;
  
+         ASSERT(sr->smb2_async == B_FALSE);
          ASSERT(sr->tid_tree == 0);
          ASSERT(sr->uid_user == 0);
          ASSERT(sr->fid_ofile == 0);
          sr->smb_fid = (uint16_t)-1;
          sr->smb2_status = 0;
  
          /* temporary until we identify a user */
          sr->user_cr = zone_kcred();
  
  cmd_start:
          /*
!          * Note that we don't check sr_state here and abort the
!          * compound if cancelled (etc.) because some SMB2 command
!          * handlers need to do work even when cancelled.
           *
           * We treat some status codes as if "sticky", meaning
           * once they're set after some command handler returns,
           * all remaining commands get this status without even
!          * calling the command-specific handler.
           */
          if (sr->smb2_status != NT_STATUS_CANCELLED &&
              sr->smb2_status != NT_STATUS_INSUFFICIENT_RESOURCES)
                  sr->smb2_status = 0;
  
+         /*
+          * Decode the request header
+          *
+          * Most problems with decoding will result in the error
+          * STATUS_INVALID_PARAMETER.  If the decoding problem
+          * prevents continuing, we'll close the connection.
+          * [MS-SMB2] 3.3.5.2.6 Handling Incorrectly Formatted...
+          */
          sr->smb2_cmd_hdr = sr->command.chain_offset;
          if ((rc = smb2_decode_header(sr)) != 0) {
                  cmn_err(CE_WARN, "clnt %s bad SMB2 header",
                      session->ip_addr_str);
                  disconnect = B_TRUE;
*** 337,346 ****
--- 536,554 ----
                      session->ip_addr_str);
                  disconnect = B_TRUE;
                  goto cleanup;
          }
          related = (sr->smb2_hdr_flags & SMB2_FLAGS_RELATED_OPERATIONS);
+         sr->smb2_hdr_flags |= SMB2_FLAGS_SERVER_TO_REDIR;
+         if (sr->smb2_hdr_flags & SMB2_FLAGS_ASYNC_COMMAND) {
+                 /* Probably an async cancel. */
+                 DTRACE_PROBE1(smb2__dispatch__async, smb_request_t *, sr);
+         } else if (sr->smb2_async) {
+                 /* Previous command in compound went async. */
+                 sr->smb2_hdr_flags |= SMB2_FLAGS_ASYNC_COMMAND;
+                 sr->smb2_async_id = SMB2_ASYNCID(sr);
+         }
  
          /*
           * In case we bail out with an error before we get to the
           * section that computes the credit grant, initialize the
           * response header fields so that credits won't change.
*** 349,363 ****
          if (sr->smb2_credit_charge == 0)
                  sr->smb2_credit_charge = 1;
          sr->smb2_credit_response = sr->smb2_credit_charge;
  
          /*
!          * Reserve space for the reply header, and save the offset.
!          * The reply header will be overwritten later.  If we have
!          * already exhausted the output space, then this client is
!          * trying something funny.  Log it and kill 'em.
           */
          sr->smb2_reply_hdr = sr->reply.chain_offset;
          if ((rc = smb2_encode_header(sr, B_FALSE)) != 0) {
                  cmn_err(CE_WARN, "clnt %s excessive reply",
                      session->ip_addr_str);
                  disconnect = B_TRUE;
--- 557,577 ----
          if (sr->smb2_credit_charge == 0)
                  sr->smb2_credit_charge = 1;
          sr->smb2_credit_response = sr->smb2_credit_charge;
  
          /*
!          * Write a tentative reply header.
!          *
!          * We could just leave this blank, but if we're using the
!          * mdb module feature that extracts packets, it's useful
!          * to have the header mostly correct here.
!          *
!          * If we have already exhausted the output space, then the
!          * client is trying something funny.  Log it and kill 'em.
           */
+         sr->smb2_next_reply = 0;
+         ASSERT((sr->reply.chain_offset & 7) == 0);
          sr->smb2_reply_hdr = sr->reply.chain_offset;
          if ((rc = smb2_encode_header(sr, B_FALSE)) != 0) {
                  cmn_err(CE_WARN, "clnt %s excessive reply",
                      session->ip_addr_str);
                  disconnect = B_TRUE;
*** 393,402 ****
--- 607,625 ----
           */
          (void) MBC_SHADOW_CHAIN(&sr->smb_data, &sr->command,
              sr->smb2_cmd_hdr, msg_len);
  
          /*
+          * We will consume the data for this request from smb_data.
+          * That effectively consumes msg_len bytes from sr->command
+          * but doesn't update its chain_offset, so we need to update
+          * that here to make later received bytes accounting work.
+          */
+         sr->command.chain_offset = sr->smb2_cmd_hdr + msg_len;
+         ASSERT(sr->command.chain_offset <= sr->command.max_bytes);
+ 
+         /*
           * Validate the commmand code, get dispatch table entries.
           * [MS-SMB2] 3.3.5.2.6 Handling Incorrectly Formatted...
           *
           * The last slot in the dispatch table is used to handle
           * invalid commands.  Same for statistics.
*** 421,431 ****
                  /*
                   * Drop user, tree, file; carefully ordered to
                   * avoid dangling references: file, tree, user
                   */
                  if (sr->fid_ofile != NULL) {
-                         smb_ofile_request_complete(sr->fid_ofile);
                          smb_ofile_release(sr->fid_ofile);
                          sr->fid_ofile = NULL;
                  }
                  if (sr->tid_tree != NULL) {
                          smb_tree_release(sr->tid_tree);
--- 644,653 ----
*** 455,482 ****
                          if (sr->uid_user == NULL) {
                                  smb2sr_put_error(sr,
                                      NT_STATUS_INVALID_PARAMETER);
                                  goto cmd_done;
                          }
!                         sr->smb_uid = sr->uid_user->u_uid;
                  } else {
                          /*
                           * Lookup the UID
                           * [MS-SMB2] 3.3.5.2 Verifying the Session
                           */
                          ASSERT(sr->uid_user == NULL);
!                         sr->uid_user = smb_session_lookup_uid(session,
!                             sr->smb_uid);
                          if (sr->uid_user == NULL) {
                                  smb2sr_put_error(sr,
                                      NT_STATUS_USER_SESSION_DELETED);
                                  goto cmd_done;
                          }
                          sr->user_cr = smb_user_getcred(sr->uid_user);
                  }
                  ASSERT(sr->uid_user != NULL);
          }
  
          if ((sdd->sdt_flags & SDDF_SUPPRESS_TID) == 0) {
                  /*
                   * This command requires a tree connection.
                   */
--- 677,758 ----
                          if (sr->uid_user == NULL) {
                                  smb2sr_put_error(sr,
                                      NT_STATUS_INVALID_PARAMETER);
                                  goto cmd_done;
                          }
!                         sr->smb2_ssnid = sr->uid_user->u_ssnid;
                  } else {
                          /*
                           * Lookup the UID
                           * [MS-SMB2] 3.3.5.2 Verifying the Session
                           */
                          ASSERT(sr->uid_user == NULL);
!                         /*
!                          * [MS-SMB2] 3.3.5.2.7 Handling Compounded Requests
!                          *
!                          * If this is an encrypted compound request,
!                          * ensure that the ssnid in the request
!                          * is the same as the tform ssnid if this
!                          * message is not related.
!                          *
!                          * The reasons this is done seem to apply equally
!                          * to uncompounded requests, so we apply it to all.
!                          */
! 
!                         if (sr->encrypted &&
!                             sr->smb2_ssnid != sr->smb3_tform_ssnid) {
!                                 disconnect = B_TRUE;
!                                 goto cleanup; /* just do this for now */
!                         }
! 
!                         sr->uid_user = smb_session_lookup_ssnid(session,
!                             sr->smb2_ssnid);
                          if (sr->uid_user == NULL) {
                                  smb2sr_put_error(sr,
                                      NT_STATUS_USER_SESSION_DELETED);
                                  goto cmd_done;
                          }
+ 
+                         /*
+                          * [MS-SMB2] 3.3.5.2.9 Verifying the Session
+                          *
+                          * If we're talking 3.x,
+                          * RejectUnencryptedAccess is TRUE,
+                          * Session.EncryptData is TRUE,
+                          * and the message wasn't encrypted,
+                          * return ACCESS_DENIED.
+                          *
+                          * Note that Session.EncryptData can only be TRUE when
+                          * we're talking 3.x.
+                          */
+ 
+                         if (sr->uid_user->u_encrypt ==
+                             SMB_CONFIG_REQUIRED &&
+                             !sr->encrypted) {
+                                 smb2sr_put_error(sr,
+                                     NT_STATUS_ACCESS_DENIED);
+                                 goto cmd_done;
+                         }
+ 
                          sr->user_cr = smb_user_getcred(sr->uid_user);
                  }
                  ASSERT(sr->uid_user != NULL);
+ 
+                 /*
+                  * Encrypt if:
+                  * - The cmd is not SESSION_SETUP or NEGOTIATE; AND
+                  * - Session.EncryptData is TRUE
+                  *
+                  * Those commands suppress UID, so they can't be the cmd here.
+                  */
+                 if (sr->uid_user->u_encrypt != SMB_CONFIG_DISABLED &&
+                     sr->tform_ssn == NULL) {
+                         smb_user_hold_internal(sr->uid_user);
+                         sr->tform_ssn = sr->uid_user;
+                         sr->smb3_tform_ssnid = sr->smb2_ssnid;
                  }
+         }
  
          if ((sdd->sdt_flags & SDDF_SUPPRESS_TID) == 0) {
                  /*
                   * This command requires a tree connection.
                   */
*** 502,514 ****
--- 778,831 ----
                          if (sr->tid_tree == NULL) {
                                  smb2sr_put_error(sr,
                                      NT_STATUS_NETWORK_NAME_DELETED);
                                  goto cmd_done;
                          }
+ 
+                         /*
+                          * [MS-SMB2] 3.3.5.2.11 Verifying the Tree Connect
+                          *
+                          * If we support 3.x, RejectUnencryptedAccess is TRUE,
+                          * if Tcon.EncryptData is TRUE or
+                          * global EncryptData is TRUE and
+                          * the message wasn't encrypted, or
+                          * if Tcon.EncryptData is TRUE or
+                          * global EncryptData is TRUE or
+                          * the request was encrypted and
+                          * the connection doesn't support encryption,
+                          * return ACCESS_DENIED.
+                          *
+                          * If RejectUnencryptedAccess is TRUE, we force
+                          * max_protocol to at least 3.0. Additionally,
+                          * if the tree requires encryption, we don't care
+                          * what we support, we still enforce encryption.
+                          */
+                         if (sr->tid_tree->t_encrypt == SMB_CONFIG_REQUIRED &&
+                             (!sr->encrypted ||
+                             (session->srv_cap & SMB2_CAP_ENCRYPTION) == 0)) {
+                                 smb2sr_put_error(sr,
+                                     NT_STATUS_ACCESS_DENIED);
+                                 goto cmd_done;
                          }
+                 }
                  ASSERT(sr->tid_tree != NULL);
+ 
+                 /*
+                  * Encrypt if:
+                  * - The cmd is not TREE_CONNECT; AND
+                  * - Tree.EncryptData is TRUE
+                  *
+                  * TREE_CONNECT suppresses TID, so that can't be the cmd here.
+                  * NOTE: assumes we can't have a tree without a user
+                  */
+                 if (sr->tid_tree->t_encrypt != SMB_CONFIG_DISABLED &&
+                     sr->tform_ssn == NULL) {
+                         smb_user_hold_internal(sr->uid_user);
+                         sr->tform_ssn = sr->uid_user;
+                         sr->smb3_tform_ssnid = sr->smb2_ssnid;
                  }
+         }
  
          /*
           * SMB2 signature verification, two parts:
           * (a) Require SMB2_FLAGS_SIGNED (for most request types)
           * (b) If SMB2_FLAGS_SIGNED is set, check the signature.
*** 525,549 ****
  
          /*
           * The SDDF_SUPPRESS_UID dispatch is set for requests that
           * don't need a UID (user).  These also don't require a
           * signature check here.
           */
          if ((sdd->sdt_flags & SDDF_SUPPRESS_UID) == 0 &&
!             sr->uid_user != NULL &&
!             (sr->uid_user->u_sign_flags & SMB_SIGNING_CHECK) != 0) {
                  /*
!                  * This request type should be signed, and
!                  * we're configured to require signatures.
                   */
!                 if ((sr->smb2_hdr_flags & SMB2_FLAGS_SIGNED) == 0) {
                          smb2sr_put_error(sr, NT_STATUS_ACCESS_DENIED);
                          goto cmd_done;
                  }
!                 rc = smb2_sign_check_request(sr);
!                 if (rc != 0) {
!                         DTRACE_PROBE1(smb2__sign__check, smb_request_t, sr);
                          smb2sr_put_error(sr, NT_STATUS_ACCESS_DENIED);
                          goto cmd_done;
                  }
          }
  
--- 842,874 ----
  
          /*
           * The SDDF_SUPPRESS_UID dispatch is set for requests that
           * don't need a UID (user).  These also don't require a
           * signature check here.
+          *
+          * [MS-SMB2] 3.3.5.2.4 Verifying the Signature
+          *
+          * If the packet was successfully decrypted, the message
+          * signature has already been verified, so we can skip this.
           */
          if ((sdd->sdt_flags & SDDF_SUPPRESS_UID) == 0 &&
!             !sr->encrypted && sr->uid_user != NULL &&
!             (sr->uid_user->u_sign_flags & SMB_SIGNING_ENABLED) != 0) {
                  /*
!                  * If the request is signed, check the signature.
!                  * Otherwise, if signing is required, deny access.
                   */
!                 if ((sr->smb2_hdr_flags & SMB2_FLAGS_SIGNED) != 0) {
!                         rc = smb2_sign_check_request(sr);
!                         if (rc != 0) {
!                                 DTRACE_PROBE1(smb2__sign__check,
!                                     smb_request_t *, sr);
                                  smb2sr_put_error(sr, NT_STATUS_ACCESS_DENIED);
                                  goto cmd_done;
                          }
!                 } else if (
!                     (sr->uid_user->u_sign_flags & SMB_SIGNING_CHECK) != 0) {
                          smb2sr_put_error(sr, NT_STATUS_ACCESS_DENIED);
                          goto cmd_done;
                  }
          }
  
*** 555,632 ****
           * function called next.
           */
          sr->smb_data.chain_offset = sr->smb2_cmd_hdr + SMB2_HDR_SIZE;
  
          /*
!          * SMB2 credits determine how many simultaneous commands the
!          * client may issue, and bounds the range of message IDs those
!          * commands may use.  With multi-credit support, commands may
!          * use ranges of message IDs, where the credits used by each
!          * command are proportional to their data transfer size.
           *
!          * Every command may request an increase or decrease of
!          * the currently granted credits, based on the difference
!          * between the credit request and the credit charge.
!          * [MS-SMB2] 3.3.1.2 Algorithm for the Granting of Credits
!          *
!          * Most commands have credit_request=1, credit_charge=1,
!          * which keeps the credit grant unchanged.
!          *
!          * All we're really doing here (for now) is reducing the
!          * credit_response if the client requests a credit increase
!          * that would take their credit over the maximum, and
!          * limiting the decrease so they don't run out of credits.
!          *
!          * Later, this could do something dynamic based on load.
!          *
!          * One other non-obvious bit about credits: We keep the
!          * session s_max_credits low until the 1st authentication,
!          * at which point we'll set the normal maximum_credits.
!          * Some clients ask for more credits with session setup,
!          * and we need to handle that requested increase _after_
!          * the command-specific handler returns so it won't be
!          * restricted to the lower (pre-auth) limit.
           */
          sr->smb2_credit_response = sr->smb2_credit_request;
          if (sr->smb2_credit_request < sr->smb2_credit_charge) {
!                 uint16_t cur, d;
! 
!                 mutex_enter(&session->s_credits_mutex);
!                 cur = session->s_cur_credits;
! 
!                 /* Handle credit decrease. */
!                 d = sr->smb2_credit_charge - sr->smb2_credit_request;
!                 cur -= d;
!                 if (cur & 0x8000) {
!                         /*
!                          * underflow (bad credit charge or request)
!                          * leave credits unchanged (response=charge)
!                          */
!                         cur = session->s_cur_credits;
!                         sr->smb2_credit_response = sr->smb2_credit_charge;
!                         DTRACE_PROBE1(smb2__credit__neg, smb_request_t, sr);
                  }
- 
-                 /*
-                  * The server MUST ensure that the number of credits
-                  * held by the client is never reduced to zero.
-                  * [MS-SMB2] 3.3.1.2
-                  */
-                 if (cur == 0) {
-                         cur = 1;
-                         sr->smb2_credit_response += 1;
-                         DTRACE_PROBE1(smb2__credit__min, smb_request_t, sr);
                  }
  
-                 DTRACE_PROBE3(smb2__credit__decrease,
-                     smb_request_t, sr, int, (int)cur,
-                     int, (int)session->s_cur_credits);
- 
-                 session->s_cur_credits = cur;
-                 mutex_exit(&session->s_credits_mutex);
-         }
- 
          /*
           * The real work: call the SMB2 command handler
           * (except for "sticky" smb2_status - see above)
           */
          sr->sr_time_start = gethrtime();
--- 880,901 ----
           * function called next.
           */
          sr->smb_data.chain_offset = sr->smb2_cmd_hdr + SMB2_HDR_SIZE;
  
          /*
!          * Credit adjustments (decrease)
           *
!          * If we've gone async, credit adjustments were done
!          * when we sent the interim reply.
           */
+         if (!sr->smb2_async) {
                  sr->smb2_credit_response = sr->smb2_credit_request;
                  if (sr->smb2_credit_request < sr->smb2_credit_charge) {
!                         smb2_credit_decrease(sr);
                  }
          }
  
          /*
           * The real work: call the SMB2 command handler
           * (except for "sticky" smb2_status - see above)
           */
          sr->sr_time_start = gethrtime();
*** 633,698 ****
          rc = SDRC_SUCCESS;
          if (sr->smb2_status == 0) {
                  /* NB: not using pre_op */
                  rc = (*sdd->sdt_function)(sr);
                  /* NB: not using post_op */
          }
  
-         MBC_FLUSH(&sr->raw_data);
- 
          /*
!          * Second half of SMB2 credit handling (increases)
           */
!         if (sr->smb2_credit_request > sr->smb2_credit_charge) {
!                 uint16_t cur, d;
  
!                 mutex_enter(&session->s_credits_mutex);
!                 cur = session->s_cur_credits;
  
-                 /* Handle credit increase. */
-                 d = sr->smb2_credit_request - sr->smb2_credit_charge;
-                 cur += d;
- 
                  /*
!                  * If new credits would be above max,
!                  * reduce the credit grant.
                   */
!                 if (cur > session->s_max_credits) {
!                         d = cur - session->s_max_credits;
!                         cur = session->s_max_credits;
!                         sr->smb2_credit_response -= d;
!                         DTRACE_PROBE1(smb2__credit__max, smb_request_t, sr);
                  }
- 
-                 DTRACE_PROBE3(smb2__credit__increase,
-                     smb_request_t, sr, int, (int)cur,
-                     int, (int)session->s_cur_credits);
- 
-                 session->s_cur_credits = cur;
-                 mutex_exit(&session->s_credits_mutex);
          }
  
  cmd_done:
-         /*
-          * Pad the reply to align(8) if necessary.
-          */
-         if (sr->reply.chain_offset & 7) {
-                 int padsz = 8 - (sr->reply.chain_offset & 7);
-                 (void) smb_mbc_encodef(&sr->reply, "#.", padsz);
-         }
-         ASSERT((sr->reply.chain_offset & 7) == 0);
- 
-         /*
-          * Record some statistics: latency, rx bytes, tx bytes.
-          */
-         smb_latency_add_sample(&sds->sdt_lat,
-             gethrtime() - sr->sr_time_start);
-         atomic_add_64(&sds->sdt_rxb,
-             (int64_t)(sr->command.chain_offset - sr->smb2_cmd_hdr));
-         atomic_add_64(&sds->sdt_txb,
-             (int64_t)(sr->reply.chain_offset - sr->smb2_reply_hdr));
- 
          switch (rc) {
          case SDRC_SUCCESS:
                  break;
          default:
                  /*
--- 902,935 ----
          rc = SDRC_SUCCESS;
          if (sr->smb2_status == 0) {
                  /* NB: not using pre_op */
                  rc = (*sdd->sdt_function)(sr);
                  /* NB: not using post_op */
+         } else {
+                 smb2sr_put_error(sr, sr->smb2_status);
          }
  
          /*
!          * When the sdt_function returns SDRC_SR_KEPT, it means
!          * this SR may have been passed to another thread so we
!          * MUST NOT touch it anymore.
           */
!         if (rc == SDRC_SR_KEPT)
!                 return;
  
!         MBC_FLUSH(&sr->raw_data);
  
          /*
!          * Credit adjustments (increase)
           */
!         if (!sr->smb2_async) {
!                 if (sr->smb2_credit_request > sr->smb2_credit_charge) {
!                         smb2_credit_increase(sr);
                  }
          }
  
  cmd_done:
          switch (rc) {
          case SDRC_SUCCESS:
                  break;
          default:
                  /*
*** 704,997 ****
                   */
  #ifdef  DEBUG
                  cmn_err(CE_NOTE, "handler for %u returned 0x%x",
                      sr->smb2_cmd_code, rc);
  #endif
!                 /* FALLTHROUGH */
          case SDRC_ERROR:
                  if (sr->smb2_status == 0)
!                         sr->smb2_status = NT_STATUS_INTERNAL_ERROR;
                  break;
          case SDRC_DROP_VC:
                  disconnect = B_TRUE;
                  goto cleanup;
          }
  
          /*
           * If there's a next command, figure out where it starts,
!          * and fill in the next command offset for the reply.
!          * Note: We sanity checked smb2_next_command above
!          * (the offset to the next command).  Similarly set
!          * smb2_next_reply as the offset to the next reply.
           */
          if (sr->smb2_next_command != 0) {
                  sr->command.chain_offset =
                      sr->smb2_cmd_hdr + sr->smb2_next_command;
                  sr->smb2_next_reply =
                      sr->reply.chain_offset - sr->smb2_reply_hdr;
          } else {
!                 sr->smb2_next_reply = 0;
          }
  
          /*
!          * Overwrite the SMB2 header for the response of
!          * this command (possibly part of a compound).
!          * encode_header adds: SMB2_FLAGS_SERVER_TO_REDIR
           */
          (void) smb2_encode_header(sr, B_TRUE);
  
!         if (sr->smb2_hdr_flags & SMB2_FLAGS_SIGNED)
                  smb2_sign_reply(sr);
  
-         if (sr->smb2_next_command != 0)
-                 goto cmd_start;
- 
          /*
!          * We've done all the commands in this compound.
!          * Send it out.
           */
          smb2_send_reply(sr);
  
          /*
!          * If any of the requests "went async", process those now.
!          * The async. function "keeps" this sr, changing its state
!          * to completed and calling smb_request_free().
           */
!         if (sr->sr_async_req != NULL) {
!                 smb2sr_do_async(sr);
!                 return;
          }
  
! cleanup:
!         if (disconnect) {
!                 smb_rwx_rwenter(&session->s_lock, RW_WRITER);
!                 switch (session->s_state) {
!                 case SMB_SESSION_STATE_DISCONNECTED:
!                 case SMB_SESSION_STATE_TERMINATED:
!                         break;
!                 default:
!                         smb_soshutdown(session->sock);
!                         session->s_state = SMB_SESSION_STATE_DISCONNECTED;
!                         break;
!                 }
!                 smb_rwx_rwexit(&session->s_lock);
!         }
  
          mutex_enter(&sr->sr_mutex);
          sr->sr_state = SMB_REQ_STATE_COMPLETED;
          mutex_exit(&sr->sr_mutex);
  
          smb_request_free(sr);
  }
  
  /*
!  * Dispatch an async request using saved information.
!  * See smb2sr_save_async and [MS-SMB2] 3.3.4.2
   *
!  * This is sort of a "lite" version of smb2sr_work.  Initialize the
!  * command and reply areas as they were when the command-speicific
!  * handler started (in case it needs to decode anything again).
!  * Call the async function, which builds the command-specific part
!  * of the response.  Finally, send the response and free the sr.
   */
! void
! smb2sr_do_async(smb_request_t *sr)
  {
!         const smb_disp_entry_t  *sdd;
          smb_disp_stats_t        *sds;
!         smb2_async_req_t        *ar;
!         int rc = 0;
  
          /*
!          * Restore what smb2_decode_header found.
!          * (In lieu of decoding it again.)
           */
!         ar = sr->sr_async_req;
!         sr->smb2_cmd_hdr   = ar->ar_cmd_hdr;
!         sr->smb2_cmd_code  = ar->ar_cmd_code;
!         sr->smb2_hdr_flags = ar->ar_hdr_flags;
!         sr->smb2_async_id  = (uintptr_t)ar;
!         sr->smb2_messageid = ar->ar_messageid;
!         sr->smb_pid = ar->ar_pid;
!         sr->smb_tid = ar->ar_tid;
!         sr->smb_uid = ar->ar_uid;
!         sr->smb2_status = 0;
  
          /*
!          * Async requests don't grant credits, because any credits
!          * should have gone out with the interim reply.
!          * An async reply goes alone (no next reply).
           */
!         sr->smb2_credit_response = 0;
!         sr->smb2_next_reply = 0;
  
          /*
!          * Setup input mbuf_chain
           */
!         ASSERT(ar->ar_cmd_len >= SMB2_HDR_SIZE);
!         (void) MBC_SHADOW_CHAIN(&sr->smb_data, &sr->command,
!             sr->smb2_cmd_hdr + SMB2_HDR_SIZE,
!             ar->ar_cmd_len - SMB2_HDR_SIZE);
  
          /*
!          * Setup output mbuf_chain
           */
!         MBC_FLUSH(&sr->reply);
!         sr->smb2_reply_hdr = sr->reply.chain_offset;
!         (void) smb2_encode_header(sr, B_FALSE);
  
!         VERIFY3U(sr->smb2_cmd_code, <, SMB2_INVALID_CMD);
!         sdd = &smb2_disp_table[sr->smb2_cmd_code];
!         sds = sr->session->s_server->sv_disp_stats2;
!         sds = &sds[sr->smb2_cmd_code];
  
          /*
!          * Keep the UID, TID, ofile we have.
           */
!         if ((sdd->sdt_flags & SDDF_SUPPRESS_UID) == 0 &&
!             sr->uid_user == NULL) {
!                 smb2sr_put_error(sr, NT_STATUS_USER_SESSION_DELETED);
!                 goto cmd_done;
          }
!         if ((sdd->sdt_flags & SDDF_SUPPRESS_TID) == 0 &&
!             sr->tid_tree == NULL) {
!                 smb2sr_put_error(sr, NT_STATUS_NETWORK_NAME_DELETED);
!                 goto cmd_done;
          }
  
          /*
!          * Signature already verified
!          * Credits handled...
           *
!          * Just call the async handler function.
           */
!         rc = ar->ar_func(sr);
!         if (rc != 0 && sr->smb2_status == 0)
!                 sr->smb2_status = NT_STATUS_INTERNAL_ERROR;
  
- cmd_done:
          /*
!          * Pad the reply to align(8) if necessary.
           */
!         if (sr->reply.chain_offset & 7) {
!                 int padsz = 8 - (sr->reply.chain_offset & 7);
!                 (void) smb_mbc_encodef(&sr->reply, "#.", padsz);
          }
-         ASSERT((sr->reply.chain_offset & 7) == 0);
  
          /*
!          * Record some statistics: (just tx bytes here)
           */
!         atomic_add_64(&sds->sdt_txb,
!             (int64_t)(sr->reply.chain_offset - sr->smb2_reply_hdr));
  
          /*
!          * Overwrite the SMB2 header for the response of
!          * this command (possibly part of a compound).
!          * The call adds: SMB2_FLAGS_SERVER_TO_REDIR
           */
!         (void) smb2_encode_header(sr, B_TRUE);
  
!         if (sr->smb2_hdr_flags & SMB2_FLAGS_SIGNED)
!                 smb2_sign_reply(sr);
  
          smb2_send_reply(sr);
  
          /*
!          * Done.  Unlink and free.
           */
!         sr->sr_async_req = NULL;
!         kmem_free(ar, sizeof (*ar));
  
!         mutex_enter(&sr->sr_mutex);
!         sr->sr_state = SMB_REQ_STATE_COMPLETED;
!         mutex_exit(&sr->sr_mutex);
  
!         smb_request_free(sr);
! }
  
! /*
!  * In preparation for sending an "interim response", save
!  * all the state we'll need to run an async command later,
!  * and assign an "async id" for this (now async) command.
!  * See [MS-SMB2] 3.3.4.2
!  *
!  * If more than one request in a compound request tries to
!  * "go async", we can "say no".  See [MS-SMB2] 3.3.4.2
!  *      If an operation would require asynchronous processing
!  *      but resources are constrained, the server MAY choose to
!  *      fail that operation with STATUS_INSUFFICIENT_RESOURCES.
!  *
!  * For simplicity, we further restrict the cases where we're
!  * willing to "go async", and only allow the last command in a
!  * compound to "go async".  It happens that this is the only
!  * case where we're actually asked to go async anyway. This
!  * simplification also means there can be at most one command
!  * in a compound that "goes async" (the last one).
!  *
!  * If we agree to "go async", this should return STATUS_PENDING.
!  * Otherwise return STATUS_INSUFFICIENT_RESOURCES for this and
!  * all requests following this request.  (See the comments re.
!  * "sticky" smb2_status values in smb2sr_work).
!  *
!  * Note: the Async ID we assign here is arbitrary, and need only
!  * be unique among pending async responses on this connection, so
!  * this just uses an object address as the Async ID.
!  *
!  * Also, the assigned worker is the ONLY thread using this
!  * async request object (sr_async_req) so no locking.
   */
! uint32_t
! smb2sr_go_async(smb_request_t *sr,
!         smb_sdrc_t (*async_func)(smb_request_t *))
  {
!         smb2_async_req_t *ar;
  
!         if (sr->smb2_next_command != 0)
!                 return (NT_STATUS_INSUFFICIENT_RESOURCES);
  
!         ASSERT(sr->sr_async_req == NULL);
!         ar = kmem_zalloc(sizeof (*ar), KM_SLEEP);
  
          /*
!          * Place an interim response in the compound reply.
!          *
!          * Turn on the "async" flag for both the (synchronous)
!          * interim response and the (later) async response,
!          * by storing that in flags before coping into ar.
!          *
!          * The "related" flag should always be off for the
!          * async part because we're no longer operating on a
!          * sequence of commands when we execute that.
           */
!         sr->smb2_hdr_flags |= SMB2_FLAGS_ASYNC_COMMAND;
!         sr->smb2_async_id = (uintptr_t)ar;
  
!         ar->ar_func = async_func;
!         ar->ar_cmd_hdr = sr->smb2_cmd_hdr;
!         ar->ar_cmd_len = sr->smb_data.max_bytes - sr->smb2_cmd_hdr;
  
!         ar->ar_cmd_code = sr->smb2_cmd_code;
!         ar->ar_hdr_flags = sr->smb2_hdr_flags &
!             ~SMB2_FLAGS_RELATED_OPERATIONS;
!         ar->ar_messageid = sr->smb2_messageid;
!         ar->ar_pid = sr->smb_pid;
!         ar->ar_tid = sr->smb_tid;
!         ar->ar_uid = sr->smb_uid;
  
!         sr->sr_async_req = ar;
  
!         /* Interim responses are NOT signed. */
!         sr->smb2_hdr_flags &= ~SMB2_FLAGS_SIGNED;
! 
!         return (NT_STATUS_PENDING);
  }
  
  int
  smb2_decode_header(smb_request_t *sr)
  {
-         uint64_t ssnid;
          uint32_t pid, tid;
          uint16_t hdr_len;
          int rc;
  
          rc = smb_mbc_decodef(
--- 941,1401 ----
                   */
  #ifdef  DEBUG
                  cmn_err(CE_NOTE, "handler for %u returned 0x%x",
                      sr->smb2_cmd_code, rc);
  #endif
!                 sr->smb2_status = NT_STATUS_INTERNAL_ERROR;
!                 break;
          case SDRC_ERROR:
+                 /*
+                  * Many command handlers return SDRC_ERROR for any
+                  * problems decoding the request, and don't bother
+                  * setting smb2_status.  For those cases, the best
+                  * status return would be "invalid parameter".
+                  */
                  if (sr->smb2_status == 0)
!                         sr->smb2_status = NT_STATUS_INVALID_PARAMETER;
                  break;
          case SDRC_DROP_VC:
                  disconnect = B_TRUE;
                  goto cleanup;
+ 
+         case SDRC_NO_REPLY:
+                 /* will free sr */
+                 goto cleanup;
          }
  
          /*
+          * Pad the reply to align(8) if there will be another.
+          * (We don't compound async replies.)
+          */
+         if (!sr->smb2_async && sr->smb2_next_command != 0)
+                 (void) smb_mbc_put_align(&sr->reply, 8);
+ 
+         /*
+          * Record some statistics.  Uses:
+          *   rxb = command.chain_offset - smb2_cmd_hdr;
+          *   txb = reply.chain_offset - smb2_reply_hdr;
+          * which at this point represent the current cmd/reply.
+          *
+          * Note: If async, this does txb only, and
+          * skips the smb_latency_add_sample() calls.
+          */
+         smb2_record_stats(sr, sds, sr->smb2_async);
+ 
+         /*
           * If there's a next command, figure out where it starts,
!          * and fill in the next header offset for the reply.
!          * Note: We sanity checked smb2_next_command above.
           */
          if (sr->smb2_next_command != 0) {
                  sr->command.chain_offset =
                      sr->smb2_cmd_hdr + sr->smb2_next_command;
                  sr->smb2_next_reply =
                      sr->reply.chain_offset - sr->smb2_reply_hdr;
          } else {
!                 ASSERT(sr->smb2_next_reply == 0);
          }
  
          /*
!          * Overwrite the (now final) SMB2 header for this response.
           */
          (void) smb2_encode_header(sr, B_TRUE);
  
!         /* Don't sign if we're going to encrypt */
!         if (sr->tform_ssn == NULL &&
!             (sr->smb2_hdr_flags & SMB2_FLAGS_SIGNED) != 0)
                  smb2_sign_reply(sr);
  
          /*
!          * Non-async runs the whole compound before send.
!          * When we've gone async, send each individually.
           */
+         if (!sr->smb2_async && sr->smb2_next_command != 0)
+                 goto cmd_start;
          smb2_send_reply(sr);
+         if (sr->smb2_async && sr->smb2_next_command != 0) {
+                 MBC_FLUSH(&sr->reply);  /* New reply buffer. */
+                 ASSERT(sr->reply.max_bytes == sr->session->reply_max_bytes);
+                 goto cmd_start;
+         }
  
+ cleanup:
+         if (disconnect)
+                 smb_session_disconnect(session);
+ 
          /*
!          * If we have a durable handle, and this operation
!          * updated the nvlist, write it out.
           */
!         if (sr->dh_nvl_dirty) {
!                 sr->dh_nvl_dirty = B_FALSE;
!                 smb2_dh_update_nvfile(sr);
          }
  
!         /*
!          * Do "postwork" for oplock (and maybe other things)
!          */
!         if (sr->sr_postwork != NULL)
!                 smb2sr_run_postwork(sr);
  
          mutex_enter(&sr->sr_mutex);
          sr->sr_state = SMB_REQ_STATE_COMPLETED;
          mutex_exit(&sr->sr_mutex);
  
          smb_request_free(sr);
  }
  
  /*
!  * Build interim responses for the current and all following
!  * requests in this compound, then send the compound response,
!  * leaving the SR state so that smb2sr_work() can continue its
!  * processing of this compound in "async mode".
   *
!  * If we agree to "go async", this should return STATUS_SUCCESS.
!  * Otherwise return STATUS_INSUFFICIENT_RESOURCES for this and
!  * all requests following this request.  (See the comments re.
!  * "sticky" smb2_status values in smb2sr_work).
!  *
!  * Note: the Async ID we assign here is arbitrary, and need only
!  * be unique among pending async responses on this connection, so
!  * this just uses a modified messageID, which is already unique.
!  *
!  * Credits:  All credit changes should happen via the interim
!  * responses, so we have to manage credits here.  After this
!  * returns to smb2sr_work, the final replies for all these
!  * commands will have smb2_credit_response = smb2_credit_charge
!  * (meaning no further changes to the clients' credits).
   */
! uint32_t
! smb2sr_go_async(smb_request_t *sr)
  {
!         smb_session_t *session;
          smb_disp_stats_t *sds;
!         uint16_t cmd_idx;
!         int32_t saved_com_offset;
!         uint32_t saved_cmd_hdr;
!         uint16_t saved_cred_resp;
!         uint32_t saved_hdr_flags;
!         uint32_t saved_reply_hdr;
!         uint32_t msg_len;
!         boolean_t disconnect = B_FALSE;
  
+         if (sr->smb2_async) {
+                 /* already went async in some previous cmd. */
+                 return (NT_STATUS_SUCCESS);
+         }
+         sr->smb2_async = B_TRUE;
+ 
+         /* The "server" session always runs async. */
+         session = sr->session;
+         if (session->sock == NULL)
+                 return (NT_STATUS_SUCCESS);
+ 
+         sds = NULL;
+         saved_com_offset = sr->command.chain_offset;
+         saved_cmd_hdr = sr->smb2_cmd_hdr;
+         saved_cred_resp = sr->smb2_credit_response;
+         saved_hdr_flags = sr->smb2_hdr_flags;
+         saved_reply_hdr = sr->smb2_reply_hdr;
+ 
          /*
!          * The command-specific handler should not yet have put any
!          * data in the reply except for the (place holder) header.
           */
!         if (sr->reply.chain_offset != sr->smb2_reply_hdr + SMB2_HDR_SIZE) {
!                 ASSERT3U(sr->reply.chain_offset, ==,
!                     sr->smb2_reply_hdr + SMB2_HDR_SIZE);
!                 return (NT_STATUS_INTERNAL_ERROR);
!         }
  
          /*
!          * Rewind to the start of the current header in both the
!          * command and reply bufers, so the loop below can just
!          * decode/encode just in every pass.  This means the
!          * current command header is decoded again, but that
!          * avoids having to special-case the first loop pass.
           */
!         sr->command.chain_offset = sr->smb2_cmd_hdr;
!         sr->reply.chain_offset = sr->smb2_reply_hdr;
  
          /*
!          * This command processing loop is a simplified version of
!          * smb2sr_work() that just puts an "interim response" for
!          * every command in the compound (NT_STATUS_PENDING).
           */
! cmd_start:
!         sr->smb2_status = NT_STATUS_PENDING;
  
          /*
!          * Decode the request header
           */
!         sr->smb2_cmd_hdr = sr->command.chain_offset;
!         if ((smb2_decode_header(sr)) != 0) {
!                 cmn_err(CE_WARN, "clnt %s bad SMB2 header",
!                     session->ip_addr_str);
!                 disconnect = B_TRUE;
!                 goto cleanup;
!         }
!         sr->smb2_hdr_flags |=  (SMB2_FLAGS_SERVER_TO_REDIR |
!                                 SMB2_FLAGS_ASYNC_COMMAND);
!         sr->smb2_async_id = SMB2_ASYNCID(sr);
  
!         /*
!          * In case we bail out...
!          */
!         if (sr->smb2_credit_charge == 0)
!                 sr->smb2_credit_charge = 1;
!         sr->smb2_credit_response = sr->smb2_credit_charge;
  
          /*
!          * Write a tentative reply header.
           */
!         sr->smb2_next_reply = 0;
!         ASSERT((sr->reply.chain_offset & 7) == 0);
!         sr->smb2_reply_hdr = sr->reply.chain_offset;
!         if ((smb2_encode_header(sr, B_FALSE)) != 0) {
!                 cmn_err(CE_WARN, "clnt %s excessive reply",
!                     session->ip_addr_str);
!                 disconnect = B_TRUE;
!                 goto cleanup;
          }
! 
!         /*
!          * Figure out the length of data...
!          */
!         if (sr->smb2_next_command != 0) {
!                 /* [MS-SMB2] says this is 8-byte aligned */
!                 msg_len = sr->smb2_next_command;
!                 if ((msg_len & 7) != 0 || (msg_len < SMB2_HDR_SIZE) ||
!                     ((sr->smb2_cmd_hdr + msg_len) > sr->command.max_bytes)) {
!                         cmn_err(CE_WARN, "clnt %s bad SMB2 next cmd",
!                             session->ip_addr_str);
!                         disconnect = B_TRUE;
!                         goto cleanup;
                  }
+         } else {
+                 msg_len = sr->command.max_bytes - sr->smb2_cmd_hdr;
+         }
  
          /*
!          * We just skip any data, so no shadow chain etc.
!          */
!         sr->command.chain_offset = sr->smb2_cmd_hdr + msg_len;
!         ASSERT(sr->command.chain_offset <= sr->command.max_bytes);
! 
!         /*
!          * Validate the commmand code...
!          */
!         if (sr->smb2_cmd_code < SMB2_INVALID_CMD)
!                 cmd_idx = sr->smb2_cmd_code;
!         else
!                 cmd_idx = SMB2_INVALID_CMD;
!         sds = &session->s_server->sv_disp_stats2[cmd_idx];
! 
!         /*
!          * Don't change (user, tree, file) because we want them
!          * exactly as they were when we entered.  That also means
!          * we may not have the right user in sr->uid_user for
!          * signature checks, so leave that until smb2sr_work
!          * runs these commands "for real".  Therefore, here
!          * we behave as if: (sr->uid_user == NULL)
!          */
!         sr->smb2_hdr_flags &= ~SMB2_FLAGS_SIGNED;
! 
!         /*
!          * Credit adjustments (decrease)
           *
!          * NOTE: interim responses are not signed.
!          * Any attacker can modify the credit grant
!          * in the response. Because of this property,
!          * it is no worse to assume the credit charge and grant
!          * are sane without verifying the signature,
!          * and that saves us a whole lot of work.
!          * If the credits WERE modified, we'll find out
!          * when we verify the signature later,
!          * which nullifies any changes caused here.
!          *
!          * Skip this on the first command, because the
!          * credit decrease was done by the caller.
           */
!         if (sr->smb2_cmd_hdr != saved_cmd_hdr) {
!                 sr->smb2_credit_response = sr->smb2_credit_request;
!                 if (sr->smb2_credit_request < sr->smb2_credit_charge) {
!                         smb2_credit_decrease(sr);
!                 }
!         }
  
          /*
!          * The real work: ... (would be here)
           */
!         smb2sr_put_error(sr, sr->smb2_status);
! 
!         /*
!          * Credit adjustments (increase)
!          */
!         if (sr->smb2_credit_request > sr->smb2_credit_charge) {
!                 smb2_credit_increase(sr);
          }
  
+         /* cmd_done: label */
+ 
          /*
!          * Pad the reply to align(8) if there will be another.
!          * This (interim) reply uses compounding.
           */
!         if (sr->smb2_next_command != 0)
!                 (void) smb_mbc_put_align(&sr->reply, 8);
  
          /*
!          * Record some statistics.  Uses:
!          *   rxb = command.chain_offset - smb2_cmd_hdr;
!          *   txb = reply.chain_offset - smb2_reply_hdr;
!          * which at this point represent the current cmd/reply.
!          *
!          * Note: We're doing smb_latency_add_sample() for all
!          * remaining commands NOW, which means we won't include
!          * the async part of their work in latency statistics.
!          * That's intentional, as the async part of a command
!          * would otherwise skew our latency statistics.
           */
!         smb2_record_stats(sr, sds, B_FALSE);
  
!         /*
!          * If there's a next command, figure out where it starts,
!          * and fill in the next header offset for the reply.
!          * Note: We sanity checked smb2_next_command above.
!          */
!         if (sr->smb2_next_command != 0) {
!                 sr->command.chain_offset =
!                     sr->smb2_cmd_hdr + sr->smb2_next_command;
!                 sr->smb2_next_reply =
!                     sr->reply.chain_offset - sr->smb2_reply_hdr;
!         } else {
!                 ASSERT(sr->smb2_next_reply == 0);
!         }
  
+         /*
+          * Overwrite the (now final) SMB2 header for this response.
+          */
+         (void) smb2_encode_header(sr, B_TRUE);
+ 
+         /*
+          * Process whole compound before sending.
+          */
+         if (sr->smb2_next_command != 0)
+                 goto cmd_start;
          smb2_send_reply(sr);
  
+         ASSERT(!disconnect);
+ 
+ cleanup:
          /*
!          * Restore caller's command processing state.
           */
!         sr->smb2_cmd_hdr = saved_cmd_hdr;
!         sr->command.chain_offset = saved_cmd_hdr;
!         (void) smb2_decode_header(sr);
!         sr->command.chain_offset = saved_com_offset;
  
!         sr->smb2_credit_response = saved_cred_resp;
!         sr->smb2_hdr_flags = saved_hdr_flags;
!         sr->smb2_status = NT_STATUS_SUCCESS;
  
!         /*
!          * In here, the "disconnect" flag just means we had an
!          * error decoding or encoding something.  Rather than
!          * actually disconnect here, let's assume whatever
!          * problem we encountered will be seen by the caller
!          * as they continue processing the compound, and just
!          * restore everything and return an error.
!          */
!         if (disconnect) {
!                 sr->smb2_async = B_FALSE;
!                 sr->smb2_reply_hdr = saved_reply_hdr;
!                 sr->reply.chain_offset = sr->smb2_reply_hdr;
!                 (void) smb2_encode_header(sr, B_FALSE);
!                 return (NT_STATUS_INVALID_PARAMETER);
!         }
  
!         /*
!          * The compound reply buffer we sent is now gone.
!          * Setup a new reply buffer for the caller.
           */
!         sr->smb2_hdr_flags |= SMB2_FLAGS_ASYNC_COMMAND;
!         sr->smb2_async_id = SMB2_ASYNCID(sr);
!         sr->smb2_next_reply = 0;
!         MBC_FLUSH(&sr->reply);
!         ASSERT(sr->reply.max_bytes == sr->session->reply_max_bytes);
!         ASSERT(sr->reply.chain_offset == 0);
!         sr->smb2_reply_hdr = 0;
!         (void) smb2_encode_header(sr, B_FALSE);
! 
!         return (NT_STATUS_SUCCESS);
! }
! 
! int
! smb3_decode_tform_header(smb_request_t *sr)
  {
!         uint16_t flags;
!         int rc;
!         uint32_t protocolid;
  
!         rc = smb_mbc_decodef(
!             &sr->command, "l16c16cl..wq",
!             &protocolid,        /*  l  */
!             sr->smb2_sig,       /* 16c */
!             sr->nonce,  /* 16c */
!             &sr->msgsize,       /* l */
!             /* reserved   .. */
!             &flags,             /* w */
!             &sr->smb3_tform_ssnid); /* q */
!         if (rc)
!                 return (rc);
  
!         ASSERT3U(protocolid, ==, SMB3_ENCRYPTED_MAGIC);
  
+         if (flags != 1) {
+ #ifdef DEBUG
+                 cmn_err(CE_NOTE, "flags field not 1: %x", flags);
+ #endif
+                 return (-1);
+         }
+ 
          /*
!          * MsgSize is the amount of data the client tell us to decrypt.
!          * Make sure this value is not too big and not too small.
           */
!         if (sr->msgsize < SMB2_HDR_SIZE ||
!             sr->msgsize > sr->session->cmd_max_bytes ||
!             sr->msgsize > sr->command.max_bytes - SMB3_TFORM_HDR_SIZE)
!                 return (-1);
  
!         return (rc);
! }
  
! int
! smb3_encode_tform_header(smb_request_t *sr, struct mbuf_chain *mbc)
! {
!         int rc;
  
!         /* Signature and Nonce are added in smb3_encrypt_sr */
!         rc = smb_mbc_encodef(
!             mbc, "l32.lwwq",
!             SMB3_ENCRYPTED_MAGIC, /* l */
!             /* signature(16), nonce(16) 32. */
!             sr->msgsize,        /* l */
!             0, /* reserved         w */
!             1, /* flags            w */
!             sr->smb3_tform_ssnid); /* q */
  
!         return (rc);
  }
  
  int
  smb2_decode_header(smb_request_t *sr)
  {
          uint32_t pid, tid;
          uint16_t hdr_len;
          int rc;
  
          rc = smb_mbc_decodef(
*** 1005,1028 ****
              &sr->smb2_hdr_flags,        /* l */
              &sr->smb2_next_command,     /* l */
              &sr->smb2_messageid,        /* q */
              &pid,                       /* l */
              &tid,                       /* l */
!             &ssnid,                     /* q */
              sr->smb2_sig);              /* 16c */
          if (rc)
                  return (rc);
  
          if (hdr_len != SMB2_HDR_SIZE)
                  return (-1);
  
-         sr->smb_uid = (uint16_t)ssnid;  /* XXX wide UIDs */
- 
          if (sr->smb2_hdr_flags & SMB2_FLAGS_ASYNC_COMMAND) {
                  sr->smb2_async_id = pid |
                      ((uint64_t)tid) << 32;
          } else {
                  sr->smb_pid = pid;
                  sr->smb_tid = (uint16_t)tid; /* XXX wide TIDs */
          }
  
          return (rc);
--- 1409,1433 ----
              &sr->smb2_hdr_flags,        /* l */
              &sr->smb2_next_command,     /* l */
              &sr->smb2_messageid,        /* q */
              &pid,                       /* l */
              &tid,                       /* l */
!             &sr->smb2_ssnid,            /* q */
              sr->smb2_sig);              /* 16c */
          if (rc)
                  return (rc);
  
          if (hdr_len != SMB2_HDR_SIZE)
                  return (-1);
  
          if (sr->smb2_hdr_flags & SMB2_FLAGS_ASYNC_COMMAND) {
                  sr->smb2_async_id = pid |
                      ((uint64_t)tid) << 32;
+                 sr->smb_pid = 0;
+                 sr->smb_tid = 0;
          } else {
+                 sr->smb2_async_id = 0;
                  sr->smb_pid = pid;
                  sr->smb_tid = (uint16_t)tid; /* XXX wide TIDs */
          }
  
          return (rc);
*** 1029,1050 ****
  }
  
  int
  smb2_encode_header(smb_request_t *sr, boolean_t overwrite)
  {
-         uint64_t ssnid = sr->smb_uid;
          uint64_t pid_tid_aid; /* pid+tid, or async id */
-         uint32_t reply_hdr_flags;
          int rc;
  
          if (sr->smb2_hdr_flags & SMB2_FLAGS_ASYNC_COMMAND) {
                  pid_tid_aid = sr->smb2_async_id;
          } else {
                  pid_tid_aid = sr->smb_pid |
                      ((uint64_t)sr->smb_tid) << 32;
          }
-         reply_hdr_flags = sr->smb2_hdr_flags | SMB2_FLAGS_SERVER_TO_REDIR;
  
          if (overwrite) {
                  rc = smb_mbc_poke(&sr->reply,
                      sr->smb2_reply_hdr,
                      "Nwwlwwllqqq16c",
--- 1434,1452 ----
*** 1051,1091 ****
                      SMB2_HDR_SIZE,              /* w */
                      sr->smb2_credit_charge,     /* w */
                      sr->smb2_status,            /* l */
                      sr->smb2_cmd_code,          /* w */
                      sr->smb2_credit_response,   /* w */
!                     reply_hdr_flags,            /* l */
                      sr->smb2_next_reply,        /* l */
                      sr->smb2_messageid,         /* q */
                      pid_tid_aid,                /* q */
!                     ssnid,                      /* q */
                      sr->smb2_sig);              /* 16c */
          } else {
                  rc = smb_mbc_encodef(&sr->reply,
                      "Nwwlwwllqqq16c",
                      SMB2_HDR_SIZE,              /* w */
                      sr->smb2_credit_charge,     /* w */
                      sr->smb2_status,            /* l */
                      sr->smb2_cmd_code,          /* w */
                      sr->smb2_credit_response,   /* w */
!                     reply_hdr_flags,            /* l */
                      sr->smb2_next_reply,        /* l */
                      sr->smb2_messageid,         /* q */
                      pid_tid_aid,                /* q */
!                     ssnid,                      /* q */
                      sr->smb2_sig);              /* 16c */
          }
  
          return (rc);
  }
  
  void
  smb2_send_reply(smb_request_t *sr)
  {
  
          if (smb_session_send(sr->session, 0, &sr->reply) == 0)
                  sr->reply.chain = 0;
  }
  
  /*
   * This wrapper function exists to help catch calls to smbsr_status()
   * (which is SMB1-specific) in common code.  See smbsr_status().
--- 1453,1546 ----
                      SMB2_HDR_SIZE,              /* w */
                      sr->smb2_credit_charge,     /* w */
                      sr->smb2_status,            /* l */
                      sr->smb2_cmd_code,          /* w */
                      sr->smb2_credit_response,   /* w */
!                     sr->smb2_hdr_flags,         /* l */
                      sr->smb2_next_reply,        /* l */
                      sr->smb2_messageid,         /* q */
                      pid_tid_aid,                /* q */
!                     sr->smb2_ssnid,             /* q */
                      sr->smb2_sig);              /* 16c */
          } else {
                  rc = smb_mbc_encodef(&sr->reply,
                      "Nwwlwwllqqq16c",
                      SMB2_HDR_SIZE,              /* w */
                      sr->smb2_credit_charge,     /* w */
                      sr->smb2_status,            /* l */
                      sr->smb2_cmd_code,          /* w */
                      sr->smb2_credit_response,   /* w */
!                     sr->smb2_hdr_flags,         /* l */
                      sr->smb2_next_reply,        /* l */
                      sr->smb2_messageid,         /* q */
                      pid_tid_aid,                /* q */
!                     sr->smb2_ssnid,             /* q */
                      sr->smb2_sig);              /* 16c */
          }
  
          return (rc);
  }
  
  void
  smb2_send_reply(smb_request_t *sr)
  {
+         struct mbuf_chain enc_reply;
+         smb_session_t *session = sr->session;
+         void *tmpbuf;
+         size_t buflen;
+         struct mbuf_chain tmp;
  
+         /*
+          * [MS-SMB2] 3.3.4.1.4 Encrypting the Message
+          *
+          * When the connection supports encryption and the dialect
+          * is 3.x, encrypt if:
+          * - The request was encrypted OR
+          * - The cmd is not SESSION_SETUP or NEGOTIATE AND
+          * -- Session.EncryptData is TRUE OR
+          * -- The cmd is not TREE_CONNECT AND
+          * --- Tree.EncryptData is TRUE
+          *
+          * This boils down to sr->tform_ssn != NULL, and the rest
+          * is enforced when tform_ssn is set.
+          */
+ 
+         if ((session->capabilities & SMB2_CAP_ENCRYPTION) == 0 ||
+             sr->tform_ssn == NULL) {
                  if (smb_session_send(sr->session, 0, &sr->reply) == 0)
                          sr->reply.chain = 0;
+                 return;
+         }
+ 
+         sr->msgsize = sr->reply.chain_offset;
+         (void) MBC_SHADOW_CHAIN(&tmp, &sr->reply,
+             0, sr->msgsize);
+ 
+         buflen = SMB3_TFORM_HDR_SIZE + sr->msgsize;
+ 
+         /* taken from smb_request_init_command_mbuf */
+         tmpbuf = kmem_alloc(buflen, KM_SLEEP);
+         MBC_ATTACH_BUF(&enc_reply, tmpbuf, buflen);
+         enc_reply.flags = 0;
+         enc_reply.shadow_of = NULL;
+ 
+         if (smb3_encode_tform_header(sr, &enc_reply) != 0) {
+                 cmn_err(CE_WARN, "couldn't encode transform header");
+                 goto errout;
+         }
+         if (smb3_encrypt_sr(sr, &tmp, &enc_reply) != 0) {
+                 cmn_err(CE_WARN, "smb3 encryption failed");
+                 goto errout;
+         }
+ 
+         if (smb_session_send(sr->session, 0, &enc_reply) == 0)
+                 enc_reply.chain = 0;
+         return;
+ 
+ errout:
+         kmem_free(tmpbuf, buflen);
+         smb_session_disconnect(sr->session);
  }
  
  /*
   * This wrapper function exists to help catch calls to smbsr_status()
   * (which is SMB1-specific) in common code.  See smbsr_status().
*** 1189,1199 ****
           */
          if (sr->fid_ofile == NULL) {
                  sr->smb_fid = (uint16_t)fid->temporal;
                  sr->fid_ofile = smb_ofile_lookup_by_fid(sr, sr->smb_fid);
          }
!         if (sr->fid_ofile == NULL)
                  return (NT_STATUS_FILE_CLOSED);
  
          return (0);
  }
  
--- 1644,1655 ----
           */
          if (sr->fid_ofile == NULL) {
                  sr->smb_fid = (uint16_t)fid->temporal;
                  sr->fid_ofile = smb_ofile_lookup_by_fid(sr, sr->smb_fid);
          }
!         if (sr->fid_ofile == NULL ||
!             sr->fid_ofile->f_persistid != fid->persistent)
                  return (NT_STATUS_FILE_CLOSED);
  
          return (0);
  }
  
*** 1265,1269 ****
--- 1721,1789 ----
                          sds[i].sdt_lat.ly_d_sum = 0;
                          mutex_exit(&sds[i].sdt_lat.ly_mutex);
                  }
          }
  }
+ 
+ /*
+  * Append new_sr to the postwork queue.  sr->smb2_cmd_code encodes
+  * the action that should be run by this sr.
+  *
+  * This queue is rarely used (and normally empty) so we're OK
+  * using a simple "walk to tail and insert" here.
+  */
+ void
+ smb2sr_append_postwork(smb_request_t *top_sr, smb_request_t *new_sr)
+ {
+         smb_request_t *last_sr;
+ 
+         ASSERT(top_sr->session->dialect >= SMB_VERS_2_BASE);
+ 
+         last_sr = top_sr;
+         while (last_sr->sr_postwork != NULL)
+                 last_sr = last_sr->sr_postwork;
+ 
+         last_sr->sr_postwork = new_sr;
+ }
+ 
+ /*
+  * Run any "post work" that was appended to the main SR while it
+  * was running.  This is called after the request has been sent
+  * for the main SR, and used in cases i.e. the oplock code, where
+  * we need to send something to the client only _after_ the main
+  * sr request has gone out.
+  */
+ static void
+ smb2sr_run_postwork(smb_request_t *top_sr)
+ {
+         smb_request_t *post_sr; /* the one we're running */
+         smb_request_t *next_sr;
+ 
+         while ((post_sr = top_sr->sr_postwork) != NULL) {
+                 next_sr = post_sr->sr_postwork;
+                 top_sr->sr_postwork = next_sr;
+                 post_sr->sr_postwork = NULL;
+ 
+                 post_sr->sr_worker = top_sr->sr_worker;
+                 post_sr->sr_state = SMB_REQ_STATE_ACTIVE;
+ 
+                 switch (post_sr->smb2_cmd_code) {
+                 case SMB2_OPLOCK_BREAK:
+                         smb_oplock_send_brk(post_sr);
+                         break;
+                 default:
+                         ASSERT(0);
+                 }
+ 
+                 /*
+                  * If we have a durable handle, and this operation
+                  * updated the nvlist, write it out.
+                  */
+                 if (post_sr->dh_nvl_dirty) {
+                         post_sr->dh_nvl_dirty = B_FALSE;
+                         smb2_dh_update_nvfile(post_sr);
+                 }
+ 
+                 post_sr->sr_state = SMB_REQ_STATE_COMPLETED;
+                 smb_request_free(post_sr);
+         }
+ }