Print this page
NEX-6832 fcsm module's debug level default should be 0 (cstyle fix)
Reviewed by: Rob Gittins <rob.gittins@nexenta.com>
NEX-7503 backport illumos #7307
Reviewed by: Saso Kiselkov <saso.kiselkov@nexenta.com>
Reviewed by: Roman Strashkin <roman.strashkin@nexenta.com>
Reviewed by: Yuri Pankov <yuri.pankov@nexenta.com>
Reviewed by: Steve Peng <steve.peng@nexenta.com>
NEX-7048 COMSTAR MODE_SENSE support is broken
Reviewed by: Rob Gittins <rob.gittins@nexenta.com>
Reviewed by: Steve Peng <steve.peng@nexenta.com>
Reviewed by: Evan Layton <evan.layton@nexenta.com>
NEX-6018 Return of the walking dead idm_refcnt_wait_ref comstar threads
Reviewed by: Rick McNeal <rick.mcneal@nexenta.com>
Reviewed by: Evan Layton <evan.layton@nexenta.com>
NEX-5428 Backout the 5.0 changes
NEX-2937 Continuous write_same starves all other commands
Reviewed by: Rick McNeal <rick.mcneal@nexenta.com>
Reviewed by: Steve Peng <steve.peng@nexenta.com>
NEX-3508 CLONE - Port NEX-2946 Add UNMAP/TRIM functionality to ZFS and illumos
Reviewed by: Josef Sipek <josef.sipek@nexenta.com>
Reviewed by: Alek Pinchuk <alek.pinchuk@nexenta.com>
Conflicts:
usr/src/uts/common/io/scsi/targets/sd.c
usr/src/uts/common/sys/scsi/targets/sddef.h
NEX-3217 Panic running benchmark at ESX VM
NEX-3204 Panic doing FC rescan from ESXi 5.5u1 with VAAI enabled
Reviewed by: Rick McNeal <rick.mcneal@nexenta.com>
Reviewed by: Tony Nguyen <tony.nguyen@nexenta.com>
NEX-2613 There should be a tunable to enable/disable SCSI UNMAP in NexentaStor
Reviewed by: Steve Peng <steve.peng@nexenta.com>
Reviewed by: Jean McCormack <jean.mccormack@nexenta.com>
NEX-1825 LUN's not discovered with ALUA - Commands sent down standby path
Reviewed by: Rob Gittins <rob.gittins@nexenta.com>
Reviewed by: Steve Peng <steve.peng@nexenta.com>
NEX-3171 VAAI Disable not yet included in 5.0
Reviewed by: Rick McNeal <rick.mcneal@nexenta.com>
Reviewed by: Tony Nguyen <tony.nguyen@nexenta.com>
Reviewed by: Jean McCormack <jean.mccormack@nexenta.com>
NEX-3111 Comstar does not pass cstyle and hdrchk
Reviewed by: Jean McCormack <jean.mccormack@nexenta.com>
Reviewed by: Rick McNeal <rick.mcneal@nexenta.com>
Reviewed by: Tony Nguyen <tony.nguyen@nexenta.com>
NEX-3023 Panics and hangs when using write_same and compare_and_write
Review by: Bayard Bell <bayard.bell@nexenta.com>
Review by: Rick McNeal <rick.mcneal@nexenta.com>
Review by: Jean McCormack <jean.mccormack@nexenta.com>
Approved by: Jean McCormack <jean.mccormack@nexenta.com>
Related bug: NEX-2723 Kernel panic in xfer_completion code for write_same (0x93) and compare_and_write (0x89)
NEX-2378 RW_LOCK_HELD assertion blown for sl_access_state_lock in sbd_flush_data_cache()
NEX-2178 Multi-block transfers on memory constrained systems for write_same (0x93) and compare_and_write (0x89) cause memory corruption
NEX-2105 assertion failed: (scmd->flags & SBD_SCSI_CMD_TRANS_DATA) && scmd->trans_data != NULL, file: ../../common/io/comstar/lu/stmf_sbd/sbd_scsi.c, line: 2447
SUP-761 sbd_flush_data_cache() call against closed zvol results in NULL pointer deref in zil_commit() call further down the stack
SUP-761 sbd_flush_data_cache() call against closed zvol results in NULL pointer deref in zil_commit() call further down the stack
NEX-1965 Page fault at netbios_first_level_name_decode+0xbb
Support simultaneous compare_and_write operations for VAAI
Bug IDs SUP-505
SUP-1768
SUP-1928
Code Reviewers:
Sarah Jelinek
Jeffry Molanus
Albert Lee
Harold Shaw
SUP-782 COMSTAR UNMAP support should limit number of LBAs per operation
NEX-988 itask_lu_[read|write]_time was inadvertently removed by the Illumos 3862 fix
OS-69 Open source VAAI
re #13117, rb4251 EEE Setting for I350 on by default
re #12981, rbYYYY Panic due to possible race between LU coming to ready state and COMSTAR (stmf) sbd task
re #12375 rb4141 Create ALUA Support on NexentaStor; Failover causes loss of storage
re #7936 rb3706 Support for COMSTAR/OEM
re #8002 rb3706 Allow setting iSCSI vendor ID via stmf_sbd.conf
re #11454 rb3750 Fix inconsistent vid/pid in stmf
re #8499 rb3117 Unreleased ATS lock possibly caused by concurrent svmotions
8226 nza-kernel needs to be buildable by itself
re #6919, rb2433 ATS not dropping locks upon completion
Re #6790 backspace should perform delete on console
VAAI (XXX ATS support for COMSTAR, YYY Block-copy support for COMSTAR)
| Split |
Close |
| Expand all |
| Collapse all |
--- old/usr/src/uts/common/io/comstar/lu/stmf_sbd/sbd_scsi.c
+++ new/usr/src/uts/common/io/comstar/lu/stmf_sbd/sbd_scsi.c
1 1 /*
2 2 * CDDL HEADER START
3 3 *
4 4 * The contents of this file are subject to the terms of the
5 5 * Common Development and Distribution License (the "License").
6 6 * You may not use this file except in compliance with the License.
7 7 *
8 8 * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE
9 9 * or http://www.opensolaris.org/os/licensing.
10 10 * See the License for the specific language governing permissions
|
↓ open down ↓ |
10 lines elided |
↑ open up ↑ |
11 11 * and limitations under the License.
12 12 *
13 13 * When distributing Covered Code, include this CDDL HEADER in each
14 14 * file and include the License file at usr/src/OPENSOLARIS.LICENSE.
15 15 * If applicable, add the following below this CDDL HEADER, with the
16 16 * fields enclosed by brackets "[]" replaced with your own identifying
17 17 * information: Portions Copyright [yyyy] [name of copyright owner]
18 18 *
19 19 * CDDL HEADER END
20 20 */
21 +
21 22 /*
22 23 * Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved.
23 - * Copyright 2011 Nexenta Systems, Inc. All rights reserved.
24 24 * Copyright (c) 2013 by Delphix. All rights reserved.
25 + * Copyright 2016 Nexenta Systems, Inc. All rights reserved.
25 26 */
26 27
27 28 #include <sys/conf.h>
28 29 #include <sys/file.h>
29 30 #include <sys/ddi.h>
30 31 #include <sys/sunddi.h>
31 32 #include <sys/modctl.h>
32 33 #include <sys/scsi/scsi.h>
33 34 #include <sys/scsi/impl/scsi_reset_notify.h>
34 35 #include <sys/scsi/generic/mode.h>
35 36 #include <sys/disp.h>
36 37 #include <sys/byteorder.h>
37 38 #include <sys/atomic.h>
38 39 #include <sys/sdt.h>
39 40 #include <sys/dkio.h>
41 +#include <sys/dkioc_free_util.h>
40 42
41 43 #include <sys/stmf.h>
42 44 #include <sys/lpif.h>
43 45 #include <sys/portif.h>
44 46 #include <sys/stmf_ioctl.h>
45 47 #include <sys/stmf_sbd_ioctl.h>
46 48
47 49 #include "stmf_sbd.h"
48 50 #include "sbd_impl.h"
49 51
50 52 #define SCSI2_CONFLICT_FREE_CMDS(cdb) ( \
51 53 /* ----------------------- */ \
52 54 /* Refer Both */ \
53 55 /* SPC-2 (rev 20) Table 10 */ \
54 56 /* SPC-3 (rev 23) Table 31 */ \
55 57 /* ----------------------- */ \
56 58 ((cdb[0]) == SCMD_INQUIRY) || \
57 59 ((cdb[0]) == SCMD_LOG_SENSE_G1) || \
58 60 ((cdb[0]) == SCMD_RELEASE) || \
59 61 ((cdb[0]) == SCMD_RELEASE_G1) || \
60 62 ((cdb[0]) == SCMD_REPORT_LUNS) || \
61 63 ((cdb[0]) == SCMD_REQUEST_SENSE) || \
62 64 /* PREVENT ALLOW MEDIUM REMOVAL with prevent == 0 */ \
63 65 ((((cdb[0]) == SCMD_DOORLOCK) && (((cdb[4]) & 0x3) == 0))) || \
64 66 /* SERVICE ACTION IN with READ MEDIA SERIAL NUMBER (0x01) */ \
65 67 (((cdb[0]) == SCMD_SVC_ACTION_IN_G5) && ( \
66 68 ((cdb[1]) & 0x1F) == 0x01)) || \
67 69 /* MAINTENANCE IN with service actions REPORT ALIASES (0x0Bh) */ \
68 70 /* REPORT DEVICE IDENTIFIER (0x05) REPORT PRIORITY (0x0Eh) */ \
69 71 /* REPORT TARGET PORT GROUPS (0x0A) REPORT TIMESTAMP (0x0F) */ \
70 72 (((cdb[0]) == SCMD_MAINTENANCE_IN) && ( \
71 73 (((cdb[1]) & 0x1F) == 0x0B) || \
72 74 (((cdb[1]) & 0x1F) == 0x05) || \
73 75 (((cdb[1]) & 0x1F) == 0x0E) || \
74 76 (((cdb[1]) & 0x1F) == 0x0A) || \
75 77 (((cdb[1]) & 0x1F) == 0x0F))) || \
76 78 /* ----------------------- */ \
77 79 /* SBC-3 (rev 17) Table 3 */ \
78 80 /* ----------------------- */ \
|
↓ open down ↓ |
29 lines elided |
↑ open up ↑ |
79 81 /* READ CAPACITY(10) */ \
80 82 ((cdb[0]) == SCMD_READ_CAPACITY) || \
81 83 /* READ CAPACITY(16) */ \
82 84 (((cdb[0]) == SCMD_SVC_ACTION_IN_G4) && ( \
83 85 ((cdb[1]) & 0x1F) == 0x10)) || \
84 86 /* START STOP UNIT with START bit 0 and POWER CONDITION 0 */ \
85 87 (((cdb[0]) == SCMD_START_STOP) && ( \
86 88 (((cdb[4]) & 0xF0) == 0) && (((cdb[4]) & 0x01) == 0))))
87 89 /* End of SCSI2_CONFLICT_FREE_CMDS */
88 90
91 +uint8_t HardwareAcceleratedInit = 1;
92 +uint8_t sbd_unmap_enable = 1; /* allow unmap by default */
93 +
94 +/*
95 + * An /etc/system tunable which specifies the maximum number of LBAs supported
96 + * in a single UNMAP operation. Default is 0x002000 blocks or 4MB in size.
97 + */
98 +int stmf_sbd_unmap_max_nblks = 0x002000;
99 +
100 +/*
101 + * An /etc/system tunable which indicates if READ ops can run on the standby
102 + * path or return an error.
103 + */
104 +int stmf_standby_fail_reads = 0;
105 +
89 106 stmf_status_t sbd_lu_reset_state(stmf_lu_t *lu);
90 107 static void sbd_handle_sync_cache(struct scsi_task *task,
91 108 struct stmf_data_buf *initial_dbuf);
92 109 void sbd_handle_read_xfer_completion(struct scsi_task *task,
93 110 sbd_cmd_t *scmd, struct stmf_data_buf *dbuf);
94 111 void sbd_handle_short_write_xfer_completion(scsi_task_t *task,
95 112 stmf_data_buf_t *dbuf);
96 -void sbd_handle_short_write_transfers(scsi_task_t *task,
97 - stmf_data_buf_t *dbuf, uint32_t cdb_xfer_size);
98 113 void sbd_handle_mode_select_xfer(scsi_task_t *task, uint8_t *buf,
99 114 uint32_t buflen);
100 115 void sbd_handle_mode_select(scsi_task_t *task, stmf_data_buf_t *dbuf);
101 116 void sbd_handle_identifying_info(scsi_task_t *task, stmf_data_buf_t *dbuf);
102 117
103 118 static void sbd_handle_unmap_xfer(scsi_task_t *task, uint8_t *buf,
104 119 uint32_t buflen);
105 120 static void sbd_handle_unmap(scsi_task_t *task, stmf_data_buf_t *dbuf);
106 121
107 122 extern void sbd_pgr_initialize_it(scsi_task_t *, sbd_it_data_t *);
108 -extern int sbd_pgr_reservation_conflict(scsi_task_t *);
123 +extern int sbd_pgr_reservation_conflict(scsi_task_t *, struct sbd_lu *sl);
109 124 extern void sbd_pgr_reset(sbd_lu_t *);
110 125 extern void sbd_pgr_remove_it_handle(sbd_lu_t *, sbd_it_data_t *);
111 126 extern void sbd_handle_pgr_in_cmd(scsi_task_t *, stmf_data_buf_t *);
112 127 extern void sbd_handle_pgr_out_cmd(scsi_task_t *, stmf_data_buf_t *);
113 128 extern void sbd_handle_pgr_out_data(scsi_task_t *, stmf_data_buf_t *);
114 129 void sbd_do_sgl_write_xfer(struct scsi_task *task, sbd_cmd_t *scmd,
115 130 int first_xfer);
116 131 static void sbd_handle_write_same(scsi_task_t *task,
117 132 struct stmf_data_buf *initial_dbuf);
118 133 static void sbd_do_write_same_xfer(struct scsi_task *task, sbd_cmd_t *scmd,
119 134 struct stmf_data_buf *dbuf, uint8_t dbuf_reusable);
120 135 static void sbd_handle_write_same_xfer_completion(struct scsi_task *task,
121 136 sbd_cmd_t *scmd, struct stmf_data_buf *dbuf, uint8_t dbuf_reusable);
137 +
122 138 /*
123 139 * IMPORTANT NOTE:
124 140 * =================
125 141 * The whole world here is based on the assumption that everything within
126 142 * a scsi task executes in a single threaded manner, even the aborts.
127 143 * Dont ever change that. There wont be any performance gain but there
128 144 * will be tons of race conditions.
129 145 */
130 146
131 147 void
132 148 sbd_do_read_xfer(struct scsi_task *task, sbd_cmd_t *scmd,
133 - struct stmf_data_buf *dbuf)
149 + struct stmf_data_buf *dbuf)
134 150 {
135 151 sbd_lu_t *sl = (sbd_lu_t *)task->task_lu->lu_provider_private;
136 152 uint64_t laddr;
137 153 uint32_t len, buflen, iolen;
138 154 int ndx;
139 155 int bufs_to_take;
140 156
141 157 /* Lets try not to hog all the buffers the port has. */
142 158 bufs_to_take = ((task->task_max_nbufs > 2) &&
143 159 (task->task_cmd_xfer_length < (32 * 1024))) ? 2 :
144 160 task->task_max_nbufs;
145 161
146 - len = scmd->len > dbuf->db_buf_size ? dbuf->db_buf_size : scmd->len;
162 + len = ATOMIC32_GET(scmd->len) > dbuf->db_buf_size ?
163 + dbuf->db_buf_size : ATOMIC32_GET(scmd->len);
147 164 laddr = scmd->addr + scmd->current_ro;
148 165
149 166 for (buflen = 0, ndx = 0; (buflen < len) &&
150 167 (ndx < dbuf->db_sglist_length); ndx++) {
151 168 iolen = min(len - buflen, dbuf->db_sglist[ndx].seg_length);
152 169 if (iolen == 0)
153 170 break;
154 171 if (sbd_data_read(sl, task, laddr, (uint64_t)iolen,
155 172 dbuf->db_sglist[ndx].seg_addr) != STMF_SUCCESS) {
156 173 scmd->flags |= SBD_SCSI_CMD_XFER_FAIL;
157 174 /* Do not need to do xfer anymore, just complete it */
158 175 dbuf->db_data_size = 0;
159 176 dbuf->db_xfer_status = STMF_SUCCESS;
|
↓ open down ↓ |
3 lines elided |
↑ open up ↑ |
160 177 sbd_handle_read_xfer_completion(task, scmd, dbuf);
161 178 return;
162 179 }
163 180 buflen += iolen;
164 181 laddr += (uint64_t)iolen;
165 182 }
166 183 dbuf->db_relative_offset = scmd->current_ro;
167 184 dbuf->db_data_size = buflen;
168 185 dbuf->db_flags = DB_DIRECTION_TO_RPORT;
169 186 (void) stmf_xfer_data(task, dbuf, 0);
170 - scmd->len -= buflen;
187 + atomic_add_32(&scmd->len, -buflen);
171 188 scmd->current_ro += buflen;
172 - if (scmd->len && (scmd->nbufs < bufs_to_take)) {
189 + if (ATOMIC32_GET(scmd->len) &&
190 + (ATOMIC8_GET(scmd->nbufs) < bufs_to_take)) {
173 191 uint32_t maxsize, minsize, old_minsize;
174 192
175 - maxsize = (scmd->len > (128*1024)) ? 128*1024 : scmd->len;
193 + maxsize = (ATOMIC32_GET(scmd->len) > (128*1024)) ? 128*1024 :
194 + ATOMIC32_GET(scmd->len);
176 195 minsize = maxsize >> 2;
177 196 do {
178 197 /*
179 198 * A bad port implementation can keep on failing the
180 199 * the request but keep on sending us a false
181 200 * minsize.
182 201 */
183 202 old_minsize = minsize;
184 203 dbuf = stmf_alloc_dbuf(task, maxsize, &minsize, 0);
185 204 } while ((dbuf == NULL) && (old_minsize > minsize) &&
186 205 (minsize >= 512));
187 206 if (dbuf == NULL) {
188 207 return;
189 208 }
190 - scmd->nbufs++;
209 + atomic_inc_8(&scmd->nbufs);
191 210 sbd_do_read_xfer(task, scmd, dbuf);
192 211 }
193 212 }
194 213
195 214 /*
196 215 * sbd_zcopy: Bail-out switch for reduced copy path.
197 216 *
198 217 * 0 - read & write off
199 218 * 1 - read & write on
200 219 * 2 - only read on
201 220 * 4 - only write on
202 221 */
203 222 int sbd_zcopy = 1; /* enable zcopy read & write path */
204 223 uint32_t sbd_max_xfer_len = 0; /* Valid if non-zero */
205 224 uint32_t sbd_1st_xfer_len = 0; /* Valid if non-zero */
206 225 uint32_t sbd_copy_threshold = 0; /* Valid if non-zero */
207 226
208 227 static void
209 228 sbd_do_sgl_read_xfer(struct scsi_task *task, sbd_cmd_t *scmd, int first_xfer)
210 229 {
|
↓ open down ↓ |
10 lines elided |
↑ open up ↑ |
211 230 sbd_lu_t *sl = (sbd_lu_t *)task->task_lu->lu_provider_private;
212 231 sbd_zvol_io_t *zvio;
213 232 int ret, final_xfer;
214 233 uint64_t offset;
215 234 uint32_t xfer_len, max_len, first_len;
216 235 stmf_status_t xstat;
217 236 stmf_data_buf_t *dbuf;
218 237 uint_t nblks;
219 238 uint64_t blksize = sl->sl_blksize;
220 239 size_t db_private_sz;
240 + hrtime_t xfer_start;
221 241 uintptr_t pad;
222 242
223 243 ASSERT(rw_read_held(&sl->sl_access_state_lock));
224 244 ASSERT((sl->sl_flags & SL_MEDIA_LOADED) != 0);
225 245
226 246 /*
227 247 * Calculate the limits on xfer_len to the minimum of :
228 248 * - task limit
229 249 * - lun limit
230 250 * - sbd global limit if set
231 251 * - first xfer limit if set
232 252 *
233 253 * First, protect against silly over-ride value
234 254 */
235 255 if (sbd_max_xfer_len && ((sbd_max_xfer_len % DEV_BSIZE) != 0)) {
236 256 cmn_err(CE_WARN, "sbd_max_xfer_len invalid %d, resetting\n",
237 257 sbd_max_xfer_len);
238 258 sbd_max_xfer_len = 0;
239 259 }
240 260 if (sbd_1st_xfer_len && ((sbd_1st_xfer_len % DEV_BSIZE) != 0)) {
241 261 cmn_err(CE_WARN, "sbd_1st_xfer_len invalid %d, resetting\n",
242 262 sbd_1st_xfer_len);
243 263 sbd_1st_xfer_len = 0;
244 264 }
245 265
246 266 max_len = MIN(task->task_max_xfer_len, sl->sl_max_xfer_len);
247 267 if (sbd_max_xfer_len)
248 268 max_len = MIN(max_len, sbd_max_xfer_len);
249 269 /*
250 270 * Special case the first xfer if hints are set.
251 271 */
|
↓ open down ↓ |
21 lines elided |
↑ open up ↑ |
252 272 if (first_xfer && (sbd_1st_xfer_len || task->task_1st_xfer_len)) {
253 273 /* global over-ride has precedence */
254 274 if (sbd_1st_xfer_len)
255 275 first_len = sbd_1st_xfer_len;
256 276 else
257 277 first_len = task->task_1st_xfer_len;
258 278 } else {
259 279 first_len = 0;
260 280 }
261 281
262 - while (scmd->len && scmd->nbufs < task->task_max_nbufs) {
282 + while (ATOMIC32_GET(scmd->len) &&
283 + ATOMIC8_GET(scmd->nbufs) < task->task_max_nbufs) {
263 284
264 - xfer_len = MIN(max_len, scmd->len);
285 + xfer_len = MIN(max_len, ATOMIC32_GET(scmd->len));
265 286 if (first_len) {
266 287 xfer_len = MIN(xfer_len, first_len);
267 288 first_len = 0;
268 289 }
269 - if (scmd->len == xfer_len) {
290 + if (ATOMIC32_GET(scmd->len) == xfer_len) {
270 291 final_xfer = 1;
271 292 } else {
272 293 /*
273 294 * Attempt to end xfer on a block boundary.
274 295 * The only way this does not happen is if the
275 296 * xfer_len is small enough to stay contained
276 297 * within the same block.
277 298 */
278 299 uint64_t xfer_offset, xfer_aligned_end;
279 300
280 301 final_xfer = 0;
281 302 xfer_offset = scmd->addr + scmd->current_ro;
282 303 xfer_aligned_end =
283 304 P2ALIGN(xfer_offset+xfer_len, blksize);
284 305 if (xfer_aligned_end > xfer_offset)
285 306 xfer_len = xfer_aligned_end - xfer_offset;
286 307 }
287 308 /*
288 309 * Allocate object to track the read and reserve
289 310 * enough space for scatter/gather list.
290 311 */
291 312 offset = scmd->addr + scmd->current_ro;
292 313 nblks = sbd_zvol_numsegs(sl, offset, xfer_len);
293 314
294 315 db_private_sz = sizeof (*zvio) + sizeof (uintptr_t) /* PAD */ +
295 316 (nblks * sizeof (stmf_sglist_ent_t));
296 317 dbuf = stmf_alloc(STMF_STRUCT_DATA_BUF, db_private_sz,
297 318 AF_DONTZERO);
298 319 /*
299 320 * Setup the dbuf
300 321 *
301 322 * XXX Framework does not handle variable length sglists
302 323 * properly, so setup db_lu_private and db_port_private
303 324 * fields here. db_stmf_private is properly set for
304 325 * calls to stmf_free.
305 326 */
306 327 if (dbuf->db_port_private == NULL) {
307 328 /*
308 329 * XXX Framework assigns space to PP after db_sglist[0]
309 330 */
310 331 cmn_err(CE_PANIC, "db_port_private == NULL");
311 332 }
312 333 pad = (uintptr_t)&dbuf->db_sglist[nblks];
313 334 dbuf->db_lu_private = (void *)P2ROUNDUP(pad, sizeof (pad));
314 335 dbuf->db_port_private = NULL;
315 336 dbuf->db_buf_size = xfer_len;
316 337 dbuf->db_data_size = xfer_len;
317 338 dbuf->db_relative_offset = scmd->current_ro;
318 339 dbuf->db_sglist_length = (uint16_t)nblks;
319 340 dbuf->db_xfer_status = 0;
320 341 dbuf->db_handle = 0;
321 342
322 343 dbuf->db_flags = (DB_DONT_CACHE | DB_DONT_REUSE |
323 344 DB_DIRECTION_TO_RPORT | DB_LU_DATA_BUF);
324 345 if (final_xfer)
325 346 dbuf->db_flags |= DB_SEND_STATUS_GOOD;
|
↓ open down ↓ |
46 lines elided |
↑ open up ↑ |
326 347
327 348 zvio = dbuf->db_lu_private;
328 349 /* Need absolute offset for zvol access */
329 350 zvio->zvio_offset = offset;
330 351 zvio->zvio_flags = ZVIO_SYNC;
331 352
332 353 /*
333 354 * Accounting for start of read.
334 355 * Note there is no buffer address for the probe yet.
335 356 */
357 + xfer_start = gethrtime();
336 358 DTRACE_PROBE5(backing__store__read__start, sbd_lu_t *, sl,
337 359 uint8_t *, NULL, uint64_t, xfer_len,
338 360 uint64_t, offset, scsi_task_t *, task);
339 361
340 362 ret = sbd_zvol_alloc_read_bufs(sl, dbuf);
341 363
364 + stmf_lu_xfer_done(task, B_TRUE /* read */,
365 + (gethrtime() - xfer_start));
342 366 DTRACE_PROBE6(backing__store__read__end, sbd_lu_t *, sl,
343 367 uint8_t *, NULL, uint64_t, xfer_len,
344 368 uint64_t, offset, int, ret, scsi_task_t *, task);
345 369
346 370 if (ret != 0) {
347 371 /*
348 372 * Read failure from the backend.
349 373 */
350 374 stmf_free(dbuf);
351 - if (scmd->nbufs == 0) {
375 + if (ATOMIC8_GET(scmd->nbufs) == 0) {
352 376 /* nothing queued, just finish */
353 377 scmd->flags &= ~SBD_SCSI_CMD_ACTIVE;
378 + sbd_ats_remove_by_task(task);
354 379 stmf_scsilib_send_status(task, STATUS_CHECK,
355 380 STMF_SAA_READ_ERROR);
356 381 rw_exit(&sl->sl_access_state_lock);
357 382 } else {
358 383 /* process failure when other dbufs finish */
359 384 scmd->flags |= SBD_SCSI_CMD_XFER_FAIL;
360 385 }
361 386 return;
362 387 }
363 388
364 -
365 389 /*
366 390 * Allow PP to do setup
367 391 */
368 392 xstat = stmf_setup_dbuf(task, dbuf, 0);
369 393 if (xstat != STMF_SUCCESS) {
370 394 /*
371 395 * This could happen if the driver cannot get the
372 396 * DDI resources it needs for this request.
373 397 * If other dbufs are queued, try again when the next
374 398 * one completes, otherwise give up.
375 399 */
376 400 sbd_zvol_rele_read_bufs(sl, dbuf);
377 401 stmf_free(dbuf);
378 - if (scmd->nbufs > 0) {
402 + if (ATOMIC8_GET(scmd->nbufs) > 0) {
379 403 /* completion of previous dbuf will retry */
380 404 return;
381 405 }
382 406 /*
383 407 * Done with this command.
384 408 */
385 409 scmd->flags &= ~SBD_SCSI_CMD_ACTIVE;
410 + sbd_ats_remove_by_task(task);
386 411 if (first_xfer)
387 412 stmf_scsilib_send_status(task, STATUS_QFULL, 0);
388 413 else
389 414 stmf_scsilib_send_status(task, STATUS_CHECK,
390 415 STMF_SAA_READ_ERROR);
391 416 rw_exit(&sl->sl_access_state_lock);
392 417 return;
393 418 }
394 419 /*
395 420 * dbuf is now queued on task
396 421 */
397 - scmd->nbufs++;
422 + atomic_inc_8(&scmd->nbufs);
398 423
399 424 /* XXX leave this in for FW? */
400 425 DTRACE_PROBE4(sbd__xfer, struct scsi_task *, task,
401 426 struct stmf_data_buf *, dbuf, uint64_t, offset,
402 427 uint32_t, xfer_len);
403 428 /*
404 429 * Do not pass STMF_IOF_LU_DONE so that the zvol
405 430 * state can be released in the completion callback.
406 431 */
407 432 xstat = stmf_xfer_data(task, dbuf, 0);
408 433 switch (xstat) {
|
↓ open down ↓ |
1 lines elided |
↑ open up ↑ |
409 434 case STMF_SUCCESS:
410 435 break;
411 436 case STMF_BUSY:
412 437 /*
413 438 * The dbuf is queued on the task, but unknown
414 439 * to the PP, thus no completion will occur.
415 440 */
416 441 sbd_zvol_rele_read_bufs(sl, dbuf);
417 442 stmf_teardown_dbuf(task, dbuf);
418 443 stmf_free(dbuf);
419 - scmd->nbufs--;
420 - if (scmd->nbufs > 0) {
444 + atomic_dec_8(&scmd->nbufs);
445 + if (ATOMIC8_GET(scmd->nbufs) > 0) {
421 446 /* completion of previous dbuf will retry */
422 447 return;
423 448 }
424 449 /*
425 450 * Done with this command.
426 451 */
427 452 rw_exit(&sl->sl_access_state_lock);
428 453 scmd->flags &= ~SBD_SCSI_CMD_ACTIVE;
454 + sbd_ats_remove_by_task(task);
429 455 if (first_xfer)
430 456 stmf_scsilib_send_status(task, STATUS_QFULL, 0);
431 457 else
432 458 stmf_scsilib_send_status(task, STATUS_CHECK,
433 459 STMF_SAA_READ_ERROR);
434 460 return;
435 461 case STMF_ABORTED:
436 462 /*
437 463 * Completion from task_done will cleanup
438 464 */
439 465 scmd->flags &= ~SBD_SCSI_CMD_ACTIVE;
466 + sbd_ats_remove_by_task(task);
440 467 return;
441 468 }
442 469 /*
443 470 * Update the xfer progress.
444 471 */
445 472 ASSERT(scmd->len >= xfer_len);
446 - scmd->len -= xfer_len;
473 + atomic_add_32(&scmd->len, -xfer_len);
447 474 scmd->current_ro += xfer_len;
448 475 }
449 476 }
450 477
451 478 void
452 479 sbd_handle_read_xfer_completion(struct scsi_task *task, sbd_cmd_t *scmd,
453 - struct stmf_data_buf *dbuf)
480 + struct stmf_data_buf *dbuf)
454 481 {
455 482 if (dbuf->db_xfer_status != STMF_SUCCESS) {
456 483 stmf_abort(STMF_QUEUE_TASK_ABORT, task,
457 484 dbuf->db_xfer_status, NULL);
458 485 return;
459 486 }
460 487 task->task_nbytes_transferred += dbuf->db_data_size;
461 - if (scmd->len == 0 || scmd->flags & SBD_SCSI_CMD_XFER_FAIL) {
488 + if (ATOMIC32_GET(scmd->len) == 0 ||
489 + scmd->flags & SBD_SCSI_CMD_XFER_FAIL) {
462 490 stmf_free_dbuf(task, dbuf);
463 - scmd->nbufs--;
464 - if (scmd->nbufs)
491 + atomic_dec_8(&scmd->nbufs);
492 + if (ATOMIC8_GET(scmd->nbufs))
465 493 return; /* wait for all buffers to complete */
466 494 scmd->flags &= ~SBD_SCSI_CMD_ACTIVE;
495 + sbd_ats_remove_by_task(task);
467 496 if (scmd->flags & SBD_SCSI_CMD_XFER_FAIL)
468 497 stmf_scsilib_send_status(task, STATUS_CHECK,
469 498 STMF_SAA_READ_ERROR);
470 499 else
471 500 stmf_scsilib_send_status(task, STATUS_GOOD, 0);
472 501 return;
473 502 }
474 503 if (dbuf->db_flags & DB_DONT_REUSE) {
475 504 /* allocate new dbuf */
476 505 uint32_t maxsize, minsize, old_minsize;
477 506 stmf_free_dbuf(task, dbuf);
478 507
479 - maxsize = (scmd->len > (128*1024)) ? 128*1024 : scmd->len;
508 + maxsize = (ATOMIC32_GET(scmd->len) > (128*1024)) ?
509 + 128*1024 : ATOMIC32_GET(scmd->len);
480 510 minsize = maxsize >> 2;
481 511 do {
482 512 old_minsize = minsize;
483 513 dbuf = stmf_alloc_dbuf(task, maxsize, &minsize, 0);
484 514 } while ((dbuf == NULL) && (old_minsize > minsize) &&
485 515 (minsize >= 512));
486 516 if (dbuf == NULL) {
487 - scmd->nbufs --;
488 - if (scmd->nbufs == 0) {
517 + atomic_dec_8(&scmd->nbufs);
518 + if (ATOMIC8_GET(scmd->nbufs) == 0) {
489 519 stmf_abort(STMF_QUEUE_TASK_ABORT, task,
490 520 STMF_ALLOC_FAILURE, NULL);
491 521 }
492 522 return;
493 523 }
494 524 }
495 525 sbd_do_read_xfer(task, scmd, dbuf);
496 526 }
497 527
498 528 /*
499 529 * This routine must release the DMU resources and free the dbuf
500 530 * in all cases. If this is the final dbuf of the task, then drop
501 531 * the reader lock on the LU state. If there are no errors and more
502 532 * work to do, then queue more xfer operations.
503 533 */
504 534 void
505 535 sbd_handle_sgl_read_xfer_completion(struct scsi_task *task, sbd_cmd_t *scmd,
506 - struct stmf_data_buf *dbuf)
536 + struct stmf_data_buf *dbuf)
507 537 {
508 538 sbd_lu_t *sl = (sbd_lu_t *)task->task_lu->lu_provider_private;
509 539 stmf_status_t xfer_status;
510 540 uint32_t data_size;
511 541 int scmd_err;
512 542
513 543 ASSERT(dbuf->db_lu_private);
514 544 ASSERT(scmd->cmd_type == SBD_CMD_SCSI_READ);
515 545
516 - scmd->nbufs--; /* account for this dbuf */
546 + atomic_dec_8(&scmd->nbufs); /* account for this dbuf */
517 547 /*
518 548 * Release the DMU resources.
519 549 */
520 550 sbd_zvol_rele_read_bufs(sl, dbuf);
521 551 /*
522 552 * Release the dbuf after retrieving needed fields.
523 553 */
524 554 xfer_status = dbuf->db_xfer_status;
525 555 data_size = dbuf->db_data_size;
526 556 stmf_teardown_dbuf(task, dbuf);
527 557 stmf_free(dbuf);
528 558 /*
529 559 * Release the state lock if this is the last completion.
530 560 * If this is the last dbuf on task and all data has been
531 561 * transferred or an error encountered, then no more dbufs
532 562 * will be queued.
533 563 */
534 564 scmd_err = (((scmd->flags & SBD_SCSI_CMD_ACTIVE) == 0) ||
535 565 (scmd->flags & SBD_SCSI_CMD_XFER_FAIL) ||
536 566 (xfer_status != STMF_SUCCESS));
537 - if (scmd->nbufs == 0 && (scmd->len == 0 || scmd_err)) {
567 + if ((ATOMIC8_GET(scmd->nbufs) == 0) &&
568 + (ATOMIC32_GET(scmd->len) == 0 || scmd_err)) {
538 569 /* all DMU state has been released */
539 570 rw_exit(&sl->sl_access_state_lock);
540 571 }
541 572
542 573 /*
543 574 * If there have been no errors, either complete the task
544 575 * or issue more data xfer operations.
545 576 */
546 577 if (!scmd_err) {
547 578 /*
548 579 * This chunk completed successfully
549 580 */
550 581 task->task_nbytes_transferred += data_size;
551 - if (scmd->nbufs == 0 && scmd->len == 0) {
582 + if (ATOMIC8_GET(scmd->nbufs) == 0 &&
583 + ATOMIC32_GET(scmd->len) == 0) {
552 584 /*
553 585 * This command completed successfully
554 586 *
555 587 * Status was sent along with data, so no status
556 588 * completion will occur. Tell stmf we are done.
557 589 */
558 590 scmd->flags &= ~SBD_SCSI_CMD_ACTIVE;
591 + sbd_ats_remove_by_task(task);
559 592 stmf_task_lu_done(task);
560 593 return;
561 594 }
562 595 /*
563 596 * Start more xfers
564 597 */
565 598 sbd_do_sgl_read_xfer(task, scmd, 0);
566 599 return;
567 600 }
568 601 /*
569 602 * Sort out the failure
570 603 */
571 604 if (scmd->flags & SBD_SCSI_CMD_ACTIVE) {
572 605 /*
573 606 * If a previous error occurred, leave the command active
574 607 * and wait for the last completion to send the status check.
575 608 */
576 609 if (scmd->flags & SBD_SCSI_CMD_XFER_FAIL) {
577 - if (scmd->nbufs == 0) {
610 + if (ATOMIC8_GET(scmd->nbufs) == 0) {
578 611 scmd->flags &= ~SBD_SCSI_CMD_ACTIVE;
612 + sbd_ats_remove_by_task(task);
579 613 stmf_scsilib_send_status(task, STATUS_CHECK,
580 614 STMF_SAA_READ_ERROR);
581 615 }
582 616 return;
583 617 }
584 618 /*
585 619 * Must have been a failure on current dbuf
586 620 */
587 621 ASSERT(xfer_status != STMF_SUCCESS);
622 +
623 + /*
624 + * Actually this is a bug. stmf abort should have reset the
625 + * active flag but since its been there for some time.
626 + * I wont change it.
627 + */
588 628 scmd->flags &= ~SBD_SCSI_CMD_ACTIVE;
629 + sbd_ats_remove_by_task(task);
589 630 stmf_abort(STMF_QUEUE_TASK_ABORT, task, xfer_status, NULL);
590 631 }
591 632 }
592 633
593 634 void
594 635 sbd_handle_sgl_write_xfer_completion(struct scsi_task *task, sbd_cmd_t *scmd,
595 - struct stmf_data_buf *dbuf)
636 + struct stmf_data_buf *dbuf)
596 637 {
597 638 sbd_zvol_io_t *zvio = dbuf->db_lu_private;
598 639 sbd_lu_t *sl = (sbd_lu_t *)task->task_lu->lu_provider_private;
599 640 int ret;
600 641 int scmd_err, scmd_xfer_done;
601 642 stmf_status_t xfer_status = dbuf->db_xfer_status;
602 643 uint32_t data_size = dbuf->db_data_size;
644 + hrtime_t xfer_start;
603 645
604 646 ASSERT(zvio);
605 647
606 648 /*
607 649 * Allow PP to free up resources before releasing the write bufs
608 650 * as writing to the backend could take some time.
609 651 */
610 652 stmf_teardown_dbuf(task, dbuf);
611 653
612 - scmd->nbufs--; /* account for this dbuf */
654 + atomic_dec_8(&scmd->nbufs); /* account for this dbuf */
613 655 /*
614 656 * All data was queued and this is the last completion,
615 657 * but there could still be an error.
616 658 */
617 - scmd_xfer_done = (scmd->len == 0 && scmd->nbufs == 0);
659 + scmd_xfer_done = (ATOMIC32_GET(scmd->len) == 0 &&
660 + (ATOMIC8_GET(scmd->nbufs) == 0));
618 661 scmd_err = (((scmd->flags & SBD_SCSI_CMD_ACTIVE) == 0) ||
619 662 (scmd->flags & SBD_SCSI_CMD_XFER_FAIL) ||
620 663 (xfer_status != STMF_SUCCESS));
621 664
665 + xfer_start = gethrtime();
622 666 DTRACE_PROBE5(backing__store__write__start, sbd_lu_t *, sl,
623 667 uint8_t *, NULL, uint64_t, data_size,
624 668 uint64_t, zvio->zvio_offset, scsi_task_t *, task);
625 669
626 670 if (scmd_err) {
627 671 /* just return the write buffers */
628 672 sbd_zvol_rele_write_bufs_abort(sl, dbuf);
629 673 ret = 0;
630 674 } else {
631 675 if (scmd_xfer_done)
632 676 zvio->zvio_flags = ZVIO_COMMIT;
633 677 else
634 678 zvio->zvio_flags = 0;
635 679 /* write the data */
636 680 ret = sbd_zvol_rele_write_bufs(sl, dbuf);
637 681 }
638 682
683 + stmf_lu_xfer_done(task, B_FALSE /* write */,
684 + (gethrtime() - xfer_start));
639 685 DTRACE_PROBE6(backing__store__write__end, sbd_lu_t *, sl,
640 686 uint8_t *, NULL, uint64_t, data_size,
641 687 uint64_t, zvio->zvio_offset, int, ret, scsi_task_t *, task);
642 688
643 689 if (ret != 0) {
644 690 /* update the error flag */
645 691 scmd->flags |= SBD_SCSI_CMD_XFER_FAIL;
646 692 scmd_err = 1;
647 693 }
648 694
649 695 /* Release the dbuf */
650 696 stmf_free(dbuf);
651 697
652 698 /*
653 699 * Release the state lock if this is the last completion.
654 700 * If this is the last dbuf on task and all data has been
655 701 * transferred or an error encountered, then no more dbufs
656 702 * will be queued.
657 703 */
658 - if (scmd->nbufs == 0 && (scmd->len == 0 || scmd_err)) {
704 + if ((ATOMIC8_GET(scmd->nbufs) == 0) &&
705 + (ATOMIC32_GET(scmd->len) == 0 || scmd_err)) {
659 706 /* all DMU state has been released */
660 707 rw_exit(&sl->sl_access_state_lock);
661 708 }
662 709 /*
663 710 * If there have been no errors, either complete the task
664 711 * or issue more data xfer operations.
665 712 */
666 713 if (!scmd_err) {
667 714 /* This chunk completed successfully */
668 715 task->task_nbytes_transferred += data_size;
669 716 if (scmd_xfer_done) {
670 717 /* This command completed successfully */
671 718 scmd->flags &= ~SBD_SCSI_CMD_ACTIVE;
719 + sbd_ats_remove_by_task(task);
672 720 if ((scmd->flags & SBD_SCSI_CMD_SYNC_WRITE) &&
673 721 (sbd_flush_data_cache(sl, 0) != SBD_SUCCESS)) {
674 722 stmf_scsilib_send_status(task, STATUS_CHECK,
675 723 STMF_SAA_WRITE_ERROR);
676 724 } else {
677 725 stmf_scsilib_send_status(task, STATUS_GOOD, 0);
678 726 }
679 727 return;
680 728 }
681 729 /*
682 730 * Start more xfers
683 731 */
684 732 sbd_do_sgl_write_xfer(task, scmd, 0);
685 733 return;
686 734 }
687 735 /*
688 736 * Sort out the failure
689 737 */
690 738 if (scmd->flags & SBD_SCSI_CMD_ACTIVE) {
691 739 if (scmd->flags & SBD_SCSI_CMD_XFER_FAIL) {
692 - if (scmd->nbufs == 0) {
740 + if (ATOMIC8_GET(scmd->nbufs) == 0) {
693 741 scmd->flags &= ~SBD_SCSI_CMD_ACTIVE;
742 + sbd_ats_remove_by_task(task);
694 743 stmf_scsilib_send_status(task, STATUS_CHECK,
695 744 STMF_SAA_WRITE_ERROR);
696 745 }
697 746 /*
698 747 * Leave the command active until last dbuf completes.
699 748 */
700 749 return;
701 750 }
702 751 scmd->flags &= ~SBD_SCSI_CMD_ACTIVE;
752 + sbd_ats_remove_by_task(task);
703 753 ASSERT(xfer_status != STMF_SUCCESS);
704 754 stmf_abort(STMF_QUEUE_TASK_ABORT, task, xfer_status, NULL);
705 755 }
706 756 }
707 757
708 758 /*
709 759 * Handle a copy operation using the zvol interface.
710 760 *
711 761 * Similar to the sbd_data_read/write path, except it goes directly through
712 762 * the zvol interfaces. It can pass a port provider sglist in the
713 763 * form of uio which is lost through the vn_rdwr path.
714 764 *
715 765 * Returns:
716 766 * STMF_SUCCESS - request handled
717 767 * STMF_FAILURE - request not handled, caller must deal with error
|
↓ open down ↓ |
5 lines elided |
↑ open up ↑ |
718 768 */
719 769 static stmf_status_t
720 770 sbd_copy_rdwr(scsi_task_t *task, uint64_t laddr, stmf_data_buf_t *dbuf,
721 771 int cmd, int commit)
722 772 {
723 773 sbd_lu_t *sl = task->task_lu->lu_provider_private;
724 774 struct uio uio;
725 775 struct iovec *iov, *tiov, iov1[8];
726 776 uint32_t len, resid;
727 777 int ret, i, iovcnt, flags;
778 + hrtime_t xfer_start;
728 779 boolean_t is_read;
729 780
730 781 ASSERT(cmd == SBD_CMD_SCSI_READ || cmd == SBD_CMD_SCSI_WRITE);
731 782
732 783 is_read = (cmd == SBD_CMD_SCSI_READ) ? B_TRUE : B_FALSE;
733 784 iovcnt = dbuf->db_sglist_length;
734 785 /* use the stack for small iovecs */
735 786 if (iovcnt > 8) {
736 787 iov = kmem_alloc(iovcnt * sizeof (*iov), KM_SLEEP);
737 788 } else {
738 789 iov = &iov1[0];
739 790 }
740 791
741 792 /* Convert dbuf sglist to iovec format */
742 793 len = dbuf->db_data_size;
743 794 resid = len;
744 795 tiov = iov;
745 796 for (i = 0; i < iovcnt; i++) {
746 797 tiov->iov_base = (caddr_t)dbuf->db_sglist[i].seg_addr;
747 798 tiov->iov_len = MIN(resid, dbuf->db_sglist[i].seg_length);
748 799 resid -= tiov->iov_len;
749 800 tiov++;
750 801 }
751 802 if (resid != 0) {
752 803 cmn_err(CE_WARN, "inconsistant sglist rem %d", resid);
753 804 if (iov != &iov1[0])
754 805 kmem_free(iov, iovcnt * sizeof (*iov));
|
↓ open down ↓ |
17 lines elided |
↑ open up ↑ |
755 806 return (STMF_FAILURE);
756 807 }
757 808 /* Setup the uio struct */
758 809 uio.uio_iov = iov;
759 810 uio.uio_iovcnt = iovcnt;
760 811 uio.uio_loffset = laddr;
761 812 uio.uio_segflg = (short)UIO_SYSSPACE;
762 813 uio.uio_resid = (uint64_t)len;
763 814 uio.uio_llimit = RLIM64_INFINITY;
764 815
816 + xfer_start = gethrtime();
765 817 if (is_read == B_TRUE) {
766 818 uio.uio_fmode = FREAD;
767 819 uio.uio_extflg = UIO_COPY_CACHED;
768 820 DTRACE_PROBE5(backing__store__read__start, sbd_lu_t *, sl,
769 821 uint8_t *, NULL, uint64_t, len, uint64_t, laddr,
770 822 scsi_task_t *, task);
771 823
772 824 /* Fetch the data */
773 825 ret = sbd_zvol_copy_read(sl, &uio);
774 826
775 827 DTRACE_PROBE6(backing__store__read__end, sbd_lu_t *, sl,
776 828 uint8_t *, NULL, uint64_t, len, uint64_t, laddr, int, ret,
777 829 scsi_task_t *, task);
778 830 } else {
779 831 uio.uio_fmode = FWRITE;
780 832 uio.uio_extflg = UIO_COPY_DEFAULT;
781 833 DTRACE_PROBE5(backing__store__write__start, sbd_lu_t *, sl,
782 834 uint8_t *, NULL, uint64_t, len, uint64_t, laddr,
|
↓ open down ↓ |
8 lines elided |
↑ open up ↑ |
783 835 scsi_task_t *, task);
784 836
785 837 flags = (commit) ? ZVIO_COMMIT : 0;
786 838 /* Write the data */
787 839 ret = sbd_zvol_copy_write(sl, &uio, flags);
788 840
789 841 DTRACE_PROBE6(backing__store__write__end, sbd_lu_t *, sl,
790 842 uint8_t *, NULL, uint64_t, len, uint64_t, laddr, int, ret,
791 843 scsi_task_t *, task);
792 844 }
845 + /* finalize accounting */
846 + stmf_lu_xfer_done(task, is_read, (gethrtime() - xfer_start));
793 847
794 848 if (iov != &iov1[0])
795 849 kmem_free(iov, iovcnt * sizeof (*iov));
796 850 if (ret != 0) {
797 851 /* Backend I/O error */
798 852 return (STMF_FAILURE);
799 853 }
800 854 return (STMF_SUCCESS);
801 855 }
802 856
803 857 void
804 858 sbd_handle_read(struct scsi_task *task, struct stmf_data_buf *initial_dbuf)
805 859 {
806 860 uint64_t lba, laddr;
861 + uint64_t blkcount;
807 862 uint32_t len;
808 863 uint8_t op = task->task_cdb[0];
809 864 sbd_lu_t *sl = (sbd_lu_t *)task->task_lu->lu_provider_private;
810 865 sbd_cmd_t *scmd;
811 866 stmf_data_buf_t *dbuf;
812 867 int fast_path;
868 + boolean_t fua_bit = B_FALSE;
813 869
870 + /*
871 + * Check to see if the command is READ(10), READ(12), or READ(16).
872 + * If it is then check for bit 3 being set to indicate if Forced
873 + * Unit Access is being requested. If so, we'll bypass the use of
874 + * DMA buffers to simplify support of this feature.
875 + */
876 + if (((op == SCMD_READ_G1) || (op == SCMD_READ_G4) ||
877 + (op == SCMD_READ_G5)) &&
878 + (task->task_cdb[1] & BIT_3)) {
879 + fua_bit = B_TRUE;
880 + }
814 881 if (op == SCMD_READ) {
815 882 lba = READ_SCSI21(&task->task_cdb[1], uint64_t);
816 883 len = (uint32_t)task->task_cdb[4];
817 884
818 885 if (len == 0) {
819 886 len = 256;
820 887 }
821 888 } else if (op == SCMD_READ_G1) {
822 889 lba = READ_SCSI32(&task->task_cdb[2], uint64_t);
823 890 len = READ_SCSI16(&task->task_cdb[7], uint32_t);
824 891 } else if (op == SCMD_READ_G5) {
825 892 lba = READ_SCSI32(&task->task_cdb[2], uint64_t);
826 893 len = READ_SCSI32(&task->task_cdb[6], uint32_t);
|
↓ open down ↓ |
3 lines elided |
↑ open up ↑ |
827 894 } else if (op == SCMD_READ_G4) {
828 895 lba = READ_SCSI64(&task->task_cdb[2], uint64_t);
829 896 len = READ_SCSI32(&task->task_cdb[10], uint32_t);
830 897 } else {
831 898 stmf_scsilib_send_status(task, STATUS_CHECK,
832 899 STMF_SAA_INVALID_OPCODE);
833 900 return;
834 901 }
835 902
836 903 laddr = lba << sl->sl_data_blocksize_shift;
904 + blkcount = len;
837 905 len <<= sl->sl_data_blocksize_shift;
838 906
839 907 if ((laddr + (uint64_t)len) > sl->sl_lu_size) {
840 908 stmf_scsilib_send_status(task, STATUS_CHECK,
841 909 STMF_SAA_LBA_OUT_OF_RANGE);
842 910 return;
843 911 }
844 912
845 913 task->task_cmd_xfer_length = len;
846 914 if (task->task_additional_flags & TASK_AF_NO_EXPECTED_XFER_LENGTH) {
847 915 task->task_expected_xfer_length = len;
848 916 }
849 917
850 918 if (len != task->task_expected_xfer_length) {
851 919 fast_path = 0;
852 920 len = (len > task->task_expected_xfer_length) ?
|
↓ open down ↓ |
6 lines elided |
↑ open up ↑ |
853 921 task->task_expected_xfer_length : len;
854 922 } else {
855 923 fast_path = 1;
856 924 }
857 925
858 926 if (len == 0) {
859 927 stmf_scsilib_send_status(task, STATUS_GOOD, 0);
860 928 return;
861 929 }
862 930
931 + if (sbd_ats_handling_before_io(task, sl, lba, blkcount) !=
932 + SBD_SUCCESS) {
933 + if (stmf_task_poll_lu(task, 10) != STMF_SUCCESS) {
934 + stmf_scsilib_send_status(task, STATUS_BUSY, 0);
935 + }
936 + return;
937 + }
863 938 /*
864 939 * Determine if this read can directly use DMU buffers.
865 940 */
866 941 if (sbd_zcopy & (2|1) && /* Debug switch */
867 942 initial_dbuf == NULL && /* No PP buffer passed in */
868 943 sl->sl_flags & SL_CALL_ZVOL && /* zvol backing store */
869 944 (task->task_additional_flags &
870 - TASK_AF_ACCEPT_LU_DBUF)) /* PP allows it */
871 - {
945 + TASK_AF_ACCEPT_LU_DBUF) && /* PP allows it */
946 + !fua_bit) {
872 947 /*
873 948 * Reduced copy path
874 949 */
875 950 uint32_t copy_threshold, minsize;
876 951 int ret;
877 952
878 953 /*
879 954 * The sl_access_state_lock will be held shared
880 955 * for the entire request and released when all
881 956 * dbufs have completed.
882 957 */
883 958 rw_enter(&sl->sl_access_state_lock, RW_READER);
884 959 if ((sl->sl_flags & SL_MEDIA_LOADED) == 0) {
885 960 rw_exit(&sl->sl_access_state_lock);
961 + sbd_ats_remove_by_task(task);
886 962 stmf_scsilib_send_status(task, STATUS_CHECK,
887 963 STMF_SAA_READ_ERROR);
888 964 return;
889 965 }
890 966
891 967 /*
892 968 * Check if setup is more expensive than copying the data.
893 969 *
894 970 * Use the global over-ride sbd_zcopy_threshold if set.
895 971 */
896 972 copy_threshold = (sbd_copy_threshold > 0) ?
897 973 sbd_copy_threshold : task->task_copy_threshold;
898 974 minsize = len;
899 975 if (len < copy_threshold &&
900 976 (dbuf = stmf_alloc_dbuf(task, len, &minsize, 0)) != 0) {
901 977
902 978 ret = sbd_copy_rdwr(task, laddr, dbuf,
903 979 SBD_CMD_SCSI_READ, 0);
904 980 /* done with the backend */
905 981 rw_exit(&sl->sl_access_state_lock);
982 + sbd_ats_remove_by_task(task);
906 983 if (ret != 0) {
907 984 /* backend error */
908 985 stmf_scsilib_send_status(task, STATUS_CHECK,
909 986 STMF_SAA_READ_ERROR);
910 987 } else {
911 988 /* send along good data */
912 989 dbuf->db_relative_offset = 0;
913 990 dbuf->db_data_size = len;
914 991 dbuf->db_flags = DB_SEND_STATUS_GOOD |
915 992 DB_DIRECTION_TO_RPORT;
916 993 /* XXX keep for FW? */
917 994 DTRACE_PROBE4(sbd__xfer,
918 995 struct scsi_task *, task,
919 996 struct stmf_data_buf *, dbuf,
920 997 uint64_t, laddr, uint32_t, len);
921 998 (void) stmf_xfer_data(task, dbuf,
922 999 STMF_IOF_LU_DONE);
923 1000 }
924 1001 return;
925 1002 }
926 1003
927 1004 /* committed to reduced copy */
|
↓ open down ↓ |
12 lines elided |
↑ open up ↑ |
928 1005 if (task->task_lu_private) {
929 1006 scmd = (sbd_cmd_t *)task->task_lu_private;
930 1007 } else {
931 1008 scmd = (sbd_cmd_t *)kmem_alloc(sizeof (sbd_cmd_t),
932 1009 KM_SLEEP);
933 1010 task->task_lu_private = scmd;
934 1011 }
935 1012 /*
936 1013 * Setup scmd to track read progress.
937 1014 */
938 - scmd->flags = SBD_SCSI_CMD_ACTIVE;
1015 + scmd->flags = SBD_SCSI_CMD_ACTIVE | SBD_SCSI_CMD_ATS_RELATED;
939 1016 scmd->cmd_type = SBD_CMD_SCSI_READ;
940 1017 scmd->nbufs = 0;
941 1018 scmd->addr = laddr;
942 1019 scmd->len = len;
943 1020 scmd->current_ro = 0;
944 -
945 1021 /*
946 1022 * Kick-off the read.
947 1023 */
948 1024 sbd_do_sgl_read_xfer(task, scmd, 1);
949 1025 return;
950 1026 }
951 1027
952 1028 if (initial_dbuf == NULL) {
953 1029 uint32_t maxsize, minsize, old_minsize;
954 1030
955 1031 maxsize = (len > (128*1024)) ? 128*1024 : len;
956 1032 minsize = maxsize >> 2;
957 1033 do {
958 1034 old_minsize = minsize;
959 1035 initial_dbuf = stmf_alloc_dbuf(task, maxsize,
960 1036 &minsize, 0);
961 1037 } while ((initial_dbuf == NULL) && (old_minsize > minsize) &&
962 1038 (minsize >= 512));
963 1039 if (initial_dbuf == NULL) {
1040 + sbd_ats_remove_by_task(task);
964 1041 stmf_scsilib_send_status(task, STATUS_QFULL, 0);
965 1042 return;
966 1043 }
967 1044 }
968 1045 dbuf = initial_dbuf;
969 1046
970 1047 if ((dbuf->db_buf_size >= len) && fast_path &&
971 1048 (dbuf->db_sglist_length == 1)) {
972 1049 if (sbd_data_read(sl, task, laddr, (uint64_t)len,
973 1050 dbuf->db_sglist[0].seg_addr) == STMF_SUCCESS) {
974 1051 dbuf->db_relative_offset = 0;
975 1052 dbuf->db_data_size = len;
976 1053 dbuf->db_flags = DB_SEND_STATUS_GOOD |
|
↓ open down ↓ |
3 lines elided |
↑ open up ↑ |
977 1054 DB_DIRECTION_TO_RPORT;
978 1055 /* XXX keep for FW? */
979 1056 DTRACE_PROBE4(sbd__xfer, struct scsi_task *, task,
980 1057 struct stmf_data_buf *, dbuf,
981 1058 uint64_t, laddr, uint32_t, len);
982 1059 (void) stmf_xfer_data(task, dbuf, STMF_IOF_LU_DONE);
983 1060 } else {
984 1061 stmf_scsilib_send_status(task, STATUS_CHECK,
985 1062 STMF_SAA_READ_ERROR);
986 1063 }
1064 + sbd_ats_remove_by_task(task);
987 1065 return;
988 1066 }
989 1067
990 1068 if (task->task_lu_private) {
991 1069 scmd = (sbd_cmd_t *)task->task_lu_private;
992 1070 } else {
993 1071 scmd = (sbd_cmd_t *)kmem_alloc(sizeof (sbd_cmd_t), KM_SLEEP);
994 1072 task->task_lu_private = scmd;
995 1073 }
996 - scmd->flags = SBD_SCSI_CMD_ACTIVE;
1074 + scmd->flags = SBD_SCSI_CMD_ACTIVE | SBD_SCSI_CMD_ATS_RELATED;
997 1075 scmd->cmd_type = SBD_CMD_SCSI_READ;
998 1076 scmd->nbufs = 1;
999 1077 scmd->addr = laddr;
1000 1078 scmd->len = len;
1001 1079 scmd->current_ro = 0;
1002 1080
1003 1081 sbd_do_read_xfer(task, scmd, dbuf);
1004 1082 }
1005 1083
1006 1084 void
1007 1085 sbd_do_write_xfer(struct scsi_task *task, sbd_cmd_t *scmd,
1008 1086 struct stmf_data_buf *dbuf, uint8_t dbuf_reusable)
1009 1087 {
1010 1088 uint32_t len;
1011 1089 int bufs_to_take;
1012 1090
1013 - if (scmd->len == 0) {
1091 + if (ATOMIC32_GET(scmd->len) == 0) {
1014 1092 goto DO_WRITE_XFER_DONE;
1015 1093 }
1016 1094
1017 1095 /* Lets try not to hog all the buffers the port has. */
1018 1096 bufs_to_take = ((task->task_max_nbufs > 2) &&
1019 1097 (task->task_cmd_xfer_length < (32 * 1024))) ? 2 :
1020 1098 task->task_max_nbufs;
1021 1099
1022 1100 if ((dbuf != NULL) &&
1023 1101 ((dbuf->db_flags & DB_DONT_REUSE) || (dbuf_reusable == 0))) {
1024 1102 /* free current dbuf and allocate a new one */
1025 1103 stmf_free_dbuf(task, dbuf);
1026 1104 dbuf = NULL;
1027 1105 }
1028 - if (scmd->nbufs >= bufs_to_take) {
1106 + if (ATOMIC8_GET(scmd->nbufs) >= bufs_to_take) {
1029 1107 goto DO_WRITE_XFER_DONE;
1030 1108 }
1031 1109 if (dbuf == NULL) {
1032 1110 uint32_t maxsize, minsize, old_minsize;
1033 1111
1034 - maxsize = (scmd->len > (128*1024)) ? 128*1024 :
1035 - scmd->len;
1112 + maxsize = (ATOMIC32_GET(scmd->len) > (128*1024)) ? 128*1024 :
1113 + ATOMIC32_GET(scmd->len);
1036 1114 minsize = maxsize >> 2;
1037 1115 do {
1038 1116 old_minsize = minsize;
1039 1117 dbuf = stmf_alloc_dbuf(task, maxsize, &minsize, 0);
1040 1118 } while ((dbuf == NULL) && (old_minsize > minsize) &&
1041 1119 (minsize >= 512));
1042 1120 if (dbuf == NULL) {
1043 - if (scmd->nbufs == 0) {
1121 + if (ATOMIC8_GET(scmd->nbufs) == 0) {
1044 1122 stmf_abort(STMF_QUEUE_TASK_ABORT, task,
1045 1123 STMF_ALLOC_FAILURE, NULL);
1046 1124 }
1047 1125 return;
1048 1126 }
1049 1127 }
1050 1128
1051 - len = scmd->len > dbuf->db_buf_size ? dbuf->db_buf_size :
1052 - scmd->len;
1129 + len = ATOMIC32_GET(scmd->len) > dbuf->db_buf_size ? dbuf->db_buf_size :
1130 + ATOMIC32_GET(scmd->len);
1053 1131
1054 1132 dbuf->db_relative_offset = scmd->current_ro;
1055 1133 dbuf->db_data_size = len;
1056 1134 dbuf->db_flags = DB_DIRECTION_FROM_RPORT;
1057 1135 (void) stmf_xfer_data(task, dbuf, 0);
1058 - scmd->nbufs++; /* outstanding port xfers and bufs used */
1059 - scmd->len -= len;
1136 + /* outstanding port xfers and bufs used */
1137 + atomic_inc_8(&scmd->nbufs);
1138 + atomic_add_32(&scmd->len, -len);
1060 1139 scmd->current_ro += len;
1061 1140
1062 - if ((scmd->len != 0) && (scmd->nbufs < bufs_to_take)) {
1141 + if ((ATOMIC32_GET(scmd->len) != 0) &&
1142 + (ATOMIC8_GET(scmd->nbufs) < bufs_to_take)) {
1063 1143 sbd_do_write_xfer(task, scmd, NULL, 0);
1064 1144 }
1065 1145 return;
1066 1146
1067 1147 DO_WRITE_XFER_DONE:
1068 1148 if (dbuf != NULL) {
1069 1149 stmf_free_dbuf(task, dbuf);
1070 1150 }
1071 1151 }
1072 1152
1073 1153 void
1074 1154 sbd_do_sgl_write_xfer(struct scsi_task *task, sbd_cmd_t *scmd, int first_xfer)
1075 1155 {
1076 1156 sbd_lu_t *sl = (sbd_lu_t *)task->task_lu->lu_provider_private;
1077 1157 sbd_zvol_io_t *zvio;
1078 1158 int ret;
1079 1159 uint32_t xfer_len, max_len, first_len;
1080 1160 stmf_status_t xstat;
1081 1161 stmf_data_buf_t *dbuf;
1082 1162 uint_t nblks;
1083 1163 uint64_t blksize = sl->sl_blksize;
1084 1164 uint64_t offset;
1085 1165 size_t db_private_sz;
1086 1166 uintptr_t pad;
1087 1167
1088 1168 ASSERT(rw_read_held(&sl->sl_access_state_lock));
1089 1169 ASSERT((sl->sl_flags & SL_MEDIA_LOADED) != 0);
1090 1170
1091 1171 /*
1092 1172 * Calculate the limits on xfer_len to the minimum of :
1093 1173 * - task limit
1094 1174 * - lun limit
1095 1175 * - sbd global limit if set
1096 1176 * - first xfer limit if set
1097 1177 *
1098 1178 * First, protect against silly over-ride value
1099 1179 */
1100 1180 if (sbd_max_xfer_len && ((sbd_max_xfer_len % DEV_BSIZE) != 0)) {
1101 1181 cmn_err(CE_WARN, "sbd_max_xfer_len invalid %d, resetting\n",
1102 1182 sbd_max_xfer_len);
1103 1183 sbd_max_xfer_len = 0;
1104 1184 }
1105 1185 if (sbd_1st_xfer_len && ((sbd_1st_xfer_len % DEV_BSIZE) != 0)) {
1106 1186 cmn_err(CE_WARN, "sbd_1st_xfer_len invalid %d, resetting\n",
1107 1187 sbd_1st_xfer_len);
1108 1188 sbd_1st_xfer_len = 0;
1109 1189 }
1110 1190
1111 1191 max_len = MIN(task->task_max_xfer_len, sl->sl_max_xfer_len);
1112 1192 if (sbd_max_xfer_len)
1113 1193 max_len = MIN(max_len, sbd_max_xfer_len);
1114 1194 /*
1115 1195 * Special case the first xfer if hints are set.
1116 1196 */
1117 1197 if (first_xfer && (sbd_1st_xfer_len || task->task_1st_xfer_len)) {
|
↓ open down ↓ |
45 lines elided |
↑ open up ↑ |
1118 1198 /* global over-ride has precedence */
1119 1199 if (sbd_1st_xfer_len)
1120 1200 first_len = sbd_1st_xfer_len;
1121 1201 else
1122 1202 first_len = task->task_1st_xfer_len;
1123 1203 } else {
1124 1204 first_len = 0;
1125 1205 }
1126 1206
1127 1207
1128 - while (scmd->len && scmd->nbufs < task->task_max_nbufs) {
1129 -
1130 - xfer_len = MIN(max_len, scmd->len);
1208 + while (ATOMIC32_GET(scmd->len) &&
1209 + ATOMIC8_GET(scmd->nbufs) < task->task_max_nbufs) {
1210 + xfer_len = MIN(max_len, ATOMIC32_GET(scmd->len));
1131 1211 if (first_len) {
1132 1212 xfer_len = MIN(xfer_len, first_len);
1133 1213 first_len = 0;
1134 1214 }
1135 - if (xfer_len < scmd->len) {
1215 + if (xfer_len < ATOMIC32_GET(scmd->len)) {
1136 1216 /*
1137 1217 * Attempt to end xfer on a block boundary.
1138 1218 * The only way this does not happen is if the
1139 1219 * xfer_len is small enough to stay contained
1140 1220 * within the same block.
1141 1221 */
1142 1222 uint64_t xfer_offset, xfer_aligned_end;
1143 1223
1144 1224 xfer_offset = scmd->addr + scmd->current_ro;
1145 1225 xfer_aligned_end =
1146 1226 P2ALIGN(xfer_offset+xfer_len, blksize);
1147 1227 if (xfer_aligned_end > xfer_offset)
1148 1228 xfer_len = xfer_aligned_end - xfer_offset;
1149 1229 }
1150 1230 /*
1151 1231 * Allocate object to track the write and reserve
1152 1232 * enough space for scatter/gather list.
1153 1233 */
1154 1234 offset = scmd->addr + scmd->current_ro;
1155 1235 nblks = sbd_zvol_numsegs(sl, offset, xfer_len);
1156 1236 db_private_sz = sizeof (*zvio) + sizeof (uintptr_t) /* PAD */ +
1157 1237 (nblks * sizeof (stmf_sglist_ent_t));
1158 1238 dbuf = stmf_alloc(STMF_STRUCT_DATA_BUF, db_private_sz,
1159 1239 AF_DONTZERO);
1160 1240
1161 1241 /*
1162 1242 * Setup the dbuf
1163 1243 *
1164 1244 * XXX Framework does not handle variable length sglists
1165 1245 * properly, so setup db_lu_private and db_port_private
1166 1246 * fields here. db_stmf_private is properly set for
1167 1247 * calls to stmf_free.
1168 1248 */
1169 1249 if (dbuf->db_port_private == NULL) {
1170 1250 /*
1171 1251 * XXX Framework assigns space to PP after db_sglist[0]
1172 1252 */
1173 1253 cmn_err(CE_PANIC, "db_port_private == NULL");
1174 1254 }
1175 1255 pad = (uintptr_t)&dbuf->db_sglist[nblks];
1176 1256 dbuf->db_lu_private = (void *)P2ROUNDUP(pad, sizeof (pad));
1177 1257 dbuf->db_port_private = NULL;
1178 1258 dbuf->db_buf_size = xfer_len;
1179 1259 dbuf->db_data_size = xfer_len;
1180 1260 dbuf->db_relative_offset = scmd->current_ro;
1181 1261 dbuf->db_sglist_length = (uint16_t)nblks;
1182 1262 dbuf->db_xfer_status = 0;
1183 1263 dbuf->db_handle = 0;
1184 1264 dbuf->db_flags = (DB_DONT_CACHE | DB_DONT_REUSE |
1185 1265 DB_DIRECTION_FROM_RPORT | DB_LU_DATA_BUF);
1186 1266
1187 1267 zvio = dbuf->db_lu_private;
1188 1268 zvio->zvio_offset = offset;
|
↓ open down ↓ |
43 lines elided |
↑ open up ↑ |
1189 1269
1190 1270 /* get the buffers */
1191 1271 ret = sbd_zvol_alloc_write_bufs(sl, dbuf);
1192 1272 if (ret != 0) {
1193 1273 /*
1194 1274 * Could not allocate buffers from the backend;
1195 1275 * treat it like an IO error.
1196 1276 */
1197 1277 stmf_free(dbuf);
1198 1278 scmd->flags |= SBD_SCSI_CMD_XFER_FAIL;
1199 - if (scmd->nbufs == 0) {
1279 + if (ATOMIC8_GET(scmd->nbufs) == 0) {
1200 1280 /*
1201 1281 * Nothing queued, so no completions coming
1202 1282 */
1283 + sbd_ats_remove_by_task(task);
1203 1284 stmf_scsilib_send_status(task, STATUS_CHECK,
1204 1285 STMF_SAA_WRITE_ERROR);
1205 1286 rw_exit(&sl->sl_access_state_lock);
1206 1287 }
1207 1288 /*
1208 1289 * Completions of previous buffers will cleanup.
1209 1290 */
1210 1291 return;
1211 1292 }
1212 1293
1213 1294 /*
1214 1295 * Allow PP to do setup
1215 1296 */
|
↓ open down ↓ |
3 lines elided |
↑ open up ↑ |
1216 1297 xstat = stmf_setup_dbuf(task, dbuf, 0);
1217 1298 if (xstat != STMF_SUCCESS) {
1218 1299 /*
1219 1300 * This could happen if the driver cannot get the
1220 1301 * DDI resources it needs for this request.
1221 1302 * If other dbufs are queued, try again when the next
1222 1303 * one completes, otherwise give up.
1223 1304 */
1224 1305 sbd_zvol_rele_write_bufs_abort(sl, dbuf);
1225 1306 stmf_free(dbuf);
1226 - if (scmd->nbufs > 0) {
1307 + if (ATOMIC8_GET(scmd->nbufs) > 0) {
1227 1308 /* completion of previous dbuf will retry */
1228 1309 return;
1229 1310 }
1230 1311 /*
1231 1312 * Done with this command.
1232 1313 */
1233 1314 scmd->flags &= ~SBD_SCSI_CMD_ACTIVE;
1315 + sbd_ats_remove_by_task(task);
1234 1316 if (first_xfer)
1235 1317 stmf_scsilib_send_status(task, STATUS_QFULL, 0);
1236 1318 else
1237 1319 stmf_scsilib_send_status(task, STATUS_CHECK,
1238 1320 STMF_SAA_WRITE_ERROR);
1239 1321 rw_exit(&sl->sl_access_state_lock);
1240 1322 return;
1241 1323 }
1242 1324
1243 1325 /*
1244 1326 * dbuf is now queued on task
1245 1327 */
1246 - scmd->nbufs++;
1328 + atomic_inc_8(&scmd->nbufs);
1247 1329
1248 1330 xstat = stmf_xfer_data(task, dbuf, 0);
1249 1331 switch (xstat) {
1250 1332 case STMF_SUCCESS:
1251 1333 break;
1252 1334 case STMF_BUSY:
1253 1335 /*
1254 1336 * The dbuf is queued on the task, but unknown
1255 1337 * to the PP, thus no completion will occur.
1256 1338 */
1257 1339 sbd_zvol_rele_write_bufs_abort(sl, dbuf);
1258 1340 stmf_teardown_dbuf(task, dbuf);
1259 1341 stmf_free(dbuf);
1260 - scmd->nbufs--;
1261 - if (scmd->nbufs > 0) {
1342 + atomic_dec_8(&scmd->nbufs);
1343 + if (ATOMIC8_GET(scmd->nbufs) > 0) {
1262 1344 /* completion of previous dbuf will retry */
1263 1345 return;
1264 1346 }
1265 1347 /*
1266 1348 * Done with this command.
1267 1349 */
1268 1350 scmd->flags &= ~SBD_SCSI_CMD_ACTIVE;
1351 + sbd_ats_remove_by_task(task);
1269 1352 if (first_xfer)
1270 1353 stmf_scsilib_send_status(task, STATUS_QFULL, 0);
1271 1354 else
1272 1355 stmf_scsilib_send_status(task, STATUS_CHECK,
1273 1356 STMF_SAA_WRITE_ERROR);
1274 1357 rw_exit(&sl->sl_access_state_lock);
1275 1358 return;
1276 1359 case STMF_ABORTED:
1277 1360 /*
1278 1361 * Completion code will cleanup.
1279 1362 */
1280 1363 scmd->flags |= SBD_SCSI_CMD_XFER_FAIL;
1281 1364 return;
1282 1365 }
1283 1366 /*
1284 1367 * Update the xfer progress.
1285 1368 */
1286 - scmd->len -= xfer_len;
1369 + atomic_add_32(&scmd->len, -xfer_len);
1287 1370 scmd->current_ro += xfer_len;
1288 1371 }
1289 1372 }
1290 1373
1291 1374 void
1292 1375 sbd_handle_write_xfer_completion(struct scsi_task *task, sbd_cmd_t *scmd,
1293 1376 struct stmf_data_buf *dbuf, uint8_t dbuf_reusable)
1294 1377 {
1295 1378 sbd_lu_t *sl = (sbd_lu_t *)task->task_lu->lu_provider_private;
1296 1379 uint64_t laddr;
1297 1380 uint32_t buflen, iolen;
1298 1381 int ndx;
1382 + uint8_t op = task->task_cdb[0];
1383 + boolean_t fua_bit = B_FALSE;
1299 1384
1300 - if (scmd->nbufs > 0) {
1385 + if (ATOMIC8_GET(scmd->nbufs) > 0) {
1301 1386 /*
1302 1387 * Decrement the count to indicate the port xfer
1303 1388 * into the dbuf has completed even though the buf is
1304 1389 * still in use here in the LU provider.
1305 1390 */
1306 - scmd->nbufs--;
1391 + atomic_dec_8(&scmd->nbufs);
1307 1392 }
1308 1393
1309 1394 if (dbuf->db_xfer_status != STMF_SUCCESS) {
1395 + sbd_ats_remove_by_task(task);
1310 1396 stmf_abort(STMF_QUEUE_TASK_ABORT, task,
1311 1397 dbuf->db_xfer_status, NULL);
1312 1398 return;
1313 1399 }
1314 1400
1315 1401 if (scmd->flags & SBD_SCSI_CMD_XFER_FAIL) {
1316 1402 goto WRITE_XFER_DONE;
1317 1403 }
1318 1404
1319 - if (scmd->len != 0) {
1405 + if (ATOMIC32_GET(scmd->len) != 0) {
1320 1406 /*
1321 1407 * Initiate the next port xfer to occur in parallel
1322 1408 * with writing this buf.
1323 1409 */
1324 1410 sbd_do_write_xfer(task, scmd, NULL, 0);
1325 1411 }
1326 1412
1413 + /*
1414 + * Check to see if the command is WRITE(10), WRITE(12), or WRITE(16).
1415 + * If it is then check for bit 3 being set to indicate if Forced
1416 + * Unit Access is being requested. If so, we'll bypass the direct
1417 + * call and handle it in sbd_data_write().
1418 + */
1419 + if (((op == SCMD_WRITE_G1) || (op == SCMD_WRITE_G4) ||
1420 + (op == SCMD_WRITE_G5)) && (task->task_cdb[1] & BIT_3)) {
1421 + fua_bit = B_TRUE;
1422 + }
1327 1423 laddr = scmd->addr + dbuf->db_relative_offset;
1328 1424
1329 1425 /*
1330 1426 * If this is going to a zvol, use the direct call to
1331 1427 * sbd_zvol_copy_{read,write}. The direct call interface is
1332 1428 * restricted to PPs that accept sglists, but that is not required.
1333 1429 */
1334 1430 if (sl->sl_flags & SL_CALL_ZVOL &&
1335 1431 (task->task_additional_flags & TASK_AF_ACCEPT_LU_DBUF) &&
1336 - (sbd_zcopy & (4|1))) {
1432 + (sbd_zcopy & (4|1)) && !fua_bit) {
1337 1433 int commit;
1338 1434
1339 - commit = (scmd->len == 0 && scmd->nbufs == 0);
1340 - if (sbd_copy_rdwr(task, laddr, dbuf, SBD_CMD_SCSI_WRITE,
1435 + commit = (ATOMIC32_GET(scmd->len) == 0 &&
1436 + ATOMIC8_GET(scmd->nbufs) == 0);
1437 + rw_enter(&sl->sl_access_state_lock, RW_READER);
1438 + if ((sl->sl_flags & SL_MEDIA_LOADED) == 0 ||
1439 + sbd_copy_rdwr(task, laddr, dbuf, SBD_CMD_SCSI_WRITE,
1341 1440 commit) != STMF_SUCCESS)
1342 1441 scmd->flags |= SBD_SCSI_CMD_XFER_FAIL;
1442 + rw_exit(&sl->sl_access_state_lock);
1343 1443 buflen = dbuf->db_data_size;
1344 1444 } else {
1345 1445 for (buflen = 0, ndx = 0; (buflen < dbuf->db_data_size) &&
1346 1446 (ndx < dbuf->db_sglist_length); ndx++) {
1347 1447 iolen = min(dbuf->db_data_size - buflen,
1348 1448 dbuf->db_sglist[ndx].seg_length);
1349 1449 if (iolen == 0)
1350 1450 break;
1351 1451 if (sbd_data_write(sl, task, laddr, (uint64_t)iolen,
1352 1452 dbuf->db_sglist[ndx].seg_addr) != STMF_SUCCESS) {
1353 1453 scmd->flags |= SBD_SCSI_CMD_XFER_FAIL;
1354 1454 break;
1355 1455 }
1356 1456 buflen += iolen;
1357 1457 laddr += (uint64_t)iolen;
1358 1458 }
1359 1459 }
1360 1460 task->task_nbytes_transferred += buflen;
1361 1461 WRITE_XFER_DONE:
1362 - if (scmd->len == 0 || scmd->flags & SBD_SCSI_CMD_XFER_FAIL) {
1462 + if (ATOMIC32_GET(scmd->len) == 0 ||
1463 + scmd->flags & SBD_SCSI_CMD_XFER_FAIL) {
1363 1464 stmf_free_dbuf(task, dbuf);
1364 - if (scmd->nbufs)
1465 + if (ATOMIC8_GET(scmd->nbufs))
1365 1466 return; /* wait for all buffers to complete */
1366 1467 scmd->flags &= ~SBD_SCSI_CMD_ACTIVE;
1468 + sbd_ats_remove_by_task(task);
1367 1469 if (scmd->flags & SBD_SCSI_CMD_XFER_FAIL) {
1368 1470 stmf_scsilib_send_status(task, STATUS_CHECK,
1369 1471 STMF_SAA_WRITE_ERROR);
1370 1472 } else {
1371 1473 /*
1372 1474 * If SYNC_WRITE flag is on then we need to flush
1373 1475 * cache before sending status.
1374 1476 * Note: this may be a no-op because of how
1375 1477 * SL_WRITEBACK_CACHE_DISABLE and
1376 1478 * SL_FLUSH_ON_DISABLED_WRITECACHE are set, but not
1377 1479 * worth code complexity of checking those in this code
1378 1480 * path, SBD_SCSI_CMD_SYNC_WRITE is rarely set.
1379 1481 */
1380 1482 if ((scmd->flags & SBD_SCSI_CMD_SYNC_WRITE) &&
1381 1483 (sbd_flush_data_cache(sl, 0) != SBD_SUCCESS)) {
1382 1484 stmf_scsilib_send_status(task, STATUS_CHECK,
1383 1485 STMF_SAA_WRITE_ERROR);
1384 1486 } else {
1385 1487 stmf_scsilib_send_status(task, STATUS_GOOD, 0);
1386 1488 }
1387 1489 }
1388 1490 return;
1389 1491 }
1390 1492 sbd_do_write_xfer(task, scmd, dbuf, dbuf_reusable);
1391 1493 }
1392 1494
1393 1495 /*
1394 1496 * Return true if copy avoidance is beneficial.
1395 1497 */
1396 1498 static int
1397 1499 sbd_zcopy_write_useful(scsi_task_t *task, uint64_t laddr, uint32_t len,
1398 1500 uint64_t blksize)
1399 1501 {
1400 1502 /*
1401 1503 * If there is a global copy threshold over-ride, use it.
1402 1504 * Otherwise use the PP value with the caveat that at least
1403 1505 * 1/2 the data must avoid being copied to be useful.
1404 1506 */
1405 1507 if (sbd_copy_threshold > 0) {
1406 1508 return (len >= sbd_copy_threshold);
1407 1509 } else {
1408 1510 uint64_t no_copy_span;
1409 1511
1410 1512 /* sub-blocksize writes always copy */
1411 1513 if (len < task->task_copy_threshold || len < blksize)
1412 1514 return (0);
1413 1515 /*
1414 1516 * Calculate amount of data that will avoid the copy path.
1415 1517 * The calculation is only valid if len >= blksize.
1416 1518 */
1417 1519 no_copy_span = P2ALIGN(laddr+len, blksize) -
1418 1520 P2ROUNDUP(laddr, blksize);
1419 1521 return (no_copy_span >= len/2);
1420 1522 }
1421 1523 }
|
↓ open down ↓ |
45 lines elided |
↑ open up ↑ |
1422 1524
1423 1525 void
1424 1526 sbd_handle_write(struct scsi_task *task, struct stmf_data_buf *initial_dbuf)
1425 1527 {
1426 1528 uint64_t lba, laddr;
1427 1529 uint32_t len;
1428 1530 uint8_t op = task->task_cdb[0], do_immediate_data = 0;
1429 1531 sbd_lu_t *sl = (sbd_lu_t *)task->task_lu->lu_provider_private;
1430 1532 sbd_cmd_t *scmd;
1431 1533 stmf_data_buf_t *dbuf;
1534 + uint64_t blkcount;
1432 1535 uint8_t sync_wr_flag = 0;
1536 + boolean_t fua_bit = B_FALSE;
1433 1537
1434 1538 if (sl->sl_flags & SL_WRITE_PROTECTED) {
1435 1539 stmf_scsilib_send_status(task, STATUS_CHECK,
1436 1540 STMF_SAA_WRITE_PROTECTED);
1437 1541 return;
1438 1542 }
1543 + /*
1544 + * Check to see if the command is WRITE(10), WRITE(12), or WRITE(16).
1545 + * If it is then check for bit 3 being set to indicate if Forced
1546 + * Unit Access is being requested. If so, we'll bypass the fast path
1547 + * code to simplify support of this feature.
1548 + */
1549 + if (((op == SCMD_WRITE_G1) || (op == SCMD_WRITE_G4) ||
1550 + (op == SCMD_WRITE_G5)) && (task->task_cdb[1] & BIT_3)) {
1551 + fua_bit = B_TRUE;
1552 + }
1439 1553 if (op == SCMD_WRITE) {
1440 1554 lba = READ_SCSI21(&task->task_cdb[1], uint64_t);
1441 1555 len = (uint32_t)task->task_cdb[4];
1442 1556
1443 1557 if (len == 0) {
1444 1558 len = 256;
1445 1559 }
1446 1560 } else if (op == SCMD_WRITE_G1) {
1447 1561 lba = READ_SCSI32(&task->task_cdb[2], uint64_t);
1448 1562 len = READ_SCSI16(&task->task_cdb[7], uint32_t);
1449 1563 } else if (op == SCMD_WRITE_G5) {
1450 1564 lba = READ_SCSI32(&task->task_cdb[2], uint64_t);
1451 1565 len = READ_SCSI32(&task->task_cdb[6], uint32_t);
1452 1566 } else if (op == SCMD_WRITE_G4) {
1453 1567 lba = READ_SCSI64(&task->task_cdb[2], uint64_t);
1454 1568 len = READ_SCSI32(&task->task_cdb[10], uint32_t);
1455 1569 } else if (op == SCMD_WRITE_VERIFY) {
1456 1570 lba = READ_SCSI32(&task->task_cdb[2], uint64_t);
1457 1571 len = READ_SCSI16(&task->task_cdb[7], uint32_t);
1458 1572 sync_wr_flag = SBD_SCSI_CMD_SYNC_WRITE;
1459 1573 } else if (op == SCMD_WRITE_VERIFY_G5) {
1460 1574 lba = READ_SCSI32(&task->task_cdb[2], uint64_t);
1461 1575 len = READ_SCSI32(&task->task_cdb[6], uint32_t);
1462 1576 sync_wr_flag = SBD_SCSI_CMD_SYNC_WRITE;
1463 1577 } else if (op == SCMD_WRITE_VERIFY_G4) {
|
↓ open down ↓ |
15 lines elided |
↑ open up ↑ |
1464 1578 lba = READ_SCSI64(&task->task_cdb[2], uint64_t);
1465 1579 len = READ_SCSI32(&task->task_cdb[10], uint32_t);
1466 1580 sync_wr_flag = SBD_SCSI_CMD_SYNC_WRITE;
1467 1581 } else {
1468 1582 stmf_scsilib_send_status(task, STATUS_CHECK,
1469 1583 STMF_SAA_INVALID_OPCODE);
1470 1584 return;
1471 1585 }
1472 1586
1473 1587 laddr = lba << sl->sl_data_blocksize_shift;
1588 + blkcount = len;
1474 1589 len <<= sl->sl_data_blocksize_shift;
1475 1590
1476 1591 if ((laddr + (uint64_t)len) > sl->sl_lu_size) {
1477 1592 stmf_scsilib_send_status(task, STATUS_CHECK,
1478 1593 STMF_SAA_LBA_OUT_OF_RANGE);
1479 1594 return;
1480 1595 }
1481 1596
1482 1597 task->task_cmd_xfer_length = len;
1483 1598 if (task->task_additional_flags & TASK_AF_NO_EXPECTED_XFER_LENGTH) {
1484 1599 task->task_expected_xfer_length = len;
|
↓ open down ↓ |
1 lines elided |
↑ open up ↑ |
1485 1600 }
1486 1601
1487 1602 len = (len > task->task_expected_xfer_length) ?
1488 1603 task->task_expected_xfer_length : len;
1489 1604
1490 1605 if (len == 0) {
1491 1606 stmf_scsilib_send_status(task, STATUS_GOOD, 0);
1492 1607 return;
1493 1608 }
1494 1609
1610 + if (sbd_ats_handling_before_io(task, sl, lba, blkcount) !=
1611 + SBD_SUCCESS) {
1612 + if (stmf_task_poll_lu(task, 10) != STMF_SUCCESS) {
1613 + stmf_scsilib_send_status(task, STATUS_BUSY, 0);
1614 + }
1615 + return;
1616 + }
1617 +
1495 1618 if (sbd_zcopy & (4|1) && /* Debug switch */
1496 1619 initial_dbuf == NULL && /* No PP buf passed in */
1497 1620 sl->sl_flags & SL_CALL_ZVOL && /* zvol backing store */
1498 1621 (task->task_additional_flags &
1499 1622 TASK_AF_ACCEPT_LU_DBUF) && /* PP allows it */
1500 - sbd_zcopy_write_useful(task, laddr, len, sl->sl_blksize)) {
1623 + sbd_zcopy_write_useful(task, laddr, len, sl->sl_blksize) &&
1624 + !fua_bit) {
1501 1625
1502 1626 /*
1503 1627 * XXX Note that disallowing initial_dbuf will eliminate
1504 1628 * iSCSI from participating. For small writes, that is
1505 1629 * probably ok. For large writes, it may be best to just
1506 1630 * copy the data from the initial dbuf and use zcopy for
1507 1631 * the rest.
1508 1632 */
1509 1633 rw_enter(&sl->sl_access_state_lock, RW_READER);
1510 1634 if ((sl->sl_flags & SL_MEDIA_LOADED) == 0) {
1511 1635 rw_exit(&sl->sl_access_state_lock);
1636 + sbd_ats_remove_by_task(task);
1512 1637 stmf_scsilib_send_status(task, STATUS_CHECK,
1513 1638 STMF_SAA_READ_ERROR);
1514 1639 return;
1515 1640 }
1516 1641 /*
1517 1642 * Setup scmd to track the write progress.
1518 1643 */
1519 1644 if (task->task_lu_private) {
1520 1645 scmd = (sbd_cmd_t *)task->task_lu_private;
1521 1646 } else {
1522 1647 scmd = (sbd_cmd_t *)kmem_alloc(sizeof (sbd_cmd_t),
1523 1648 KM_SLEEP);
1524 1649 task->task_lu_private = scmd;
1525 1650 }
1526 - scmd->flags = SBD_SCSI_CMD_ACTIVE | sync_wr_flag;
1651 + scmd->flags = SBD_SCSI_CMD_ACTIVE | SBD_SCSI_CMD_ATS_RELATED |
1652 + sync_wr_flag;
1527 1653 scmd->cmd_type = SBD_CMD_SCSI_WRITE;
1528 1654 scmd->nbufs = 0;
1529 1655 scmd->addr = laddr;
1530 1656 scmd->len = len;
1531 1657 scmd->current_ro = 0;
1532 1658 sbd_do_sgl_write_xfer(task, scmd, 1);
1533 1659 return;
1534 1660 }
1535 1661
1536 1662 if ((initial_dbuf != NULL) && (task->task_flags & TF_INITIAL_BURST)) {
1537 1663 if (initial_dbuf->db_data_size > len) {
1538 1664 if (initial_dbuf->db_data_size >
1539 1665 task->task_expected_xfer_length) {
1540 1666 /* protocol error */
1541 1667 stmf_abort(STMF_QUEUE_TASK_ABORT, task,
1542 1668 STMF_INVALID_ARG, NULL);
1543 1669 return;
1544 1670 }
1545 1671 initial_dbuf->db_data_size = len;
1546 1672 }
|
↓ open down ↓ |
10 lines elided |
↑ open up ↑ |
1547 1673 do_immediate_data = 1;
1548 1674 }
1549 1675 dbuf = initial_dbuf;
1550 1676
1551 1677 if (task->task_lu_private) {
1552 1678 scmd = (sbd_cmd_t *)task->task_lu_private;
1553 1679 } else {
1554 1680 scmd = (sbd_cmd_t *)kmem_alloc(sizeof (sbd_cmd_t), KM_SLEEP);
1555 1681 task->task_lu_private = scmd;
1556 1682 }
1557 - scmd->flags = SBD_SCSI_CMD_ACTIVE | sync_wr_flag;
1683 + scmd->flags = SBD_SCSI_CMD_ACTIVE | SBD_SCSI_CMD_ATS_RELATED |
1684 + sync_wr_flag;
1558 1685 scmd->cmd_type = SBD_CMD_SCSI_WRITE;
1559 1686 scmd->nbufs = 0;
1560 1687 scmd->addr = laddr;
1561 1688 scmd->len = len;
1562 1689 scmd->current_ro = 0;
1563 1690
1564 1691 if (do_immediate_data) {
1565 1692 /*
1566 1693 * Account for data passed in this write command
1567 1694 */
1568 1695 (void) stmf_xfer_data(task, dbuf, STMF_IOF_STATS_ONLY);
1569 - scmd->len -= dbuf->db_data_size;
1696 + atomic_add_32(&scmd->len, -dbuf->db_data_size);
1570 1697 scmd->current_ro += dbuf->db_data_size;
1571 1698 dbuf->db_xfer_status = STMF_SUCCESS;
1572 1699 sbd_handle_write_xfer_completion(task, scmd, dbuf, 0);
1573 1700 } else {
1574 1701 sbd_do_write_xfer(task, scmd, dbuf, 0);
1575 1702 }
1576 1703 }
1577 1704
1578 1705 /*
1579 1706 * Utility routine to handle small non performance data transfers to the
1580 1707 * initiators. dbuf is an initial data buf (if any), 'p' points to a data
1581 1708 * buffer which is source of data for transfer, cdb_xfer_size is the
1582 1709 * transfer size based on CDB, cmd_xfer_size is the actual amount of data
1583 1710 * which this command would transfer (the size of data pointed to by 'p').
1584 1711 */
1585 1712 void
1586 1713 sbd_handle_short_read_transfers(scsi_task_t *task, stmf_data_buf_t *dbuf,
1587 1714 uint8_t *p, uint32_t cdb_xfer_size, uint32_t cmd_xfer_size)
1588 1715 {
1589 1716 uint32_t bufsize, ndx;
1590 1717 sbd_cmd_t *scmd;
1591 1718
1592 1719 cmd_xfer_size = min(cmd_xfer_size, cdb_xfer_size);
1593 1720
1594 1721 task->task_cmd_xfer_length = cmd_xfer_size;
1595 1722 if (task->task_additional_flags & TASK_AF_NO_EXPECTED_XFER_LENGTH) {
1596 1723 task->task_expected_xfer_length = cmd_xfer_size;
1597 1724 } else {
1598 1725 cmd_xfer_size = min(cmd_xfer_size,
1599 1726 task->task_expected_xfer_length);
1600 1727 }
1601 1728
1602 1729 if (cmd_xfer_size == 0) {
1603 1730 stmf_scsilib_send_status(task, STATUS_CHECK,
1604 1731 STMF_SAA_INVALID_FIELD_IN_CDB);
1605 1732 return;
1606 1733 }
1607 1734 if (dbuf == NULL) {
1608 1735 uint32_t minsize = cmd_xfer_size;
1609 1736
1610 1737 dbuf = stmf_alloc_dbuf(task, cmd_xfer_size, &minsize, 0);
1611 1738 }
1612 1739 if (dbuf == NULL) {
1613 1740 stmf_scsilib_send_status(task, STATUS_QFULL, 0);
1614 1741 return;
1615 1742 }
1616 1743
1617 1744 for (bufsize = 0, ndx = 0; bufsize < cmd_xfer_size; ndx++) {
1618 1745 uint8_t *d;
1619 1746 uint32_t s;
1620 1747
1621 1748 d = dbuf->db_sglist[ndx].seg_addr;
1622 1749 s = min((cmd_xfer_size - bufsize),
1623 1750 dbuf->db_sglist[ndx].seg_length);
1624 1751 bcopy(p+bufsize, d, s);
1625 1752 bufsize += s;
1626 1753 }
1627 1754 dbuf->db_relative_offset = 0;
1628 1755 dbuf->db_data_size = cmd_xfer_size;
1629 1756 dbuf->db_flags = DB_DIRECTION_TO_RPORT;
1630 1757
1631 1758 if (task->task_lu_private == NULL) {
1632 1759 task->task_lu_private =
1633 1760 kmem_alloc(sizeof (sbd_cmd_t), KM_SLEEP);
|
↓ open down ↓ |
54 lines elided |
↑ open up ↑ |
1634 1761 }
1635 1762 scmd = (sbd_cmd_t *)task->task_lu_private;
1636 1763
1637 1764 scmd->cmd_type = SBD_CMD_SMALL_READ;
1638 1765 scmd->flags = SBD_SCSI_CMD_ACTIVE;
1639 1766 (void) stmf_xfer_data(task, dbuf, 0);
1640 1767 }
1641 1768
1642 1769 void
1643 1770 sbd_handle_short_read_xfer_completion(struct scsi_task *task, sbd_cmd_t *scmd,
1644 - struct stmf_data_buf *dbuf)
1771 + struct stmf_data_buf *dbuf)
1645 1772 {
1646 1773 if (dbuf->db_xfer_status != STMF_SUCCESS) {
1647 1774 stmf_abort(STMF_QUEUE_TASK_ABORT, task,
1648 1775 dbuf->db_xfer_status, NULL);
1649 1776 return;
1650 1777 }
1651 1778 task->task_nbytes_transferred = dbuf->db_data_size;
1652 1779 scmd->flags &= ~SBD_SCSI_CMD_ACTIVE;
1653 1780 stmf_scsilib_send_status(task, STATUS_GOOD, 0);
1654 1781 }
1655 1782
1656 1783 void
1657 1784 sbd_handle_short_write_transfers(scsi_task_t *task,
1658 1785 stmf_data_buf_t *dbuf, uint32_t cdb_xfer_size)
1659 1786 {
1660 1787 sbd_cmd_t *scmd;
1661 1788
1662 1789 task->task_cmd_xfer_length = cdb_xfer_size;
1663 1790 if (task->task_additional_flags & TASK_AF_NO_EXPECTED_XFER_LENGTH) {
1664 1791 task->task_expected_xfer_length = cdb_xfer_size;
1665 1792 } else {
1666 1793 cdb_xfer_size = min(cdb_xfer_size,
1667 1794 task->task_expected_xfer_length);
1668 1795 }
1669 1796
1670 1797 if (cdb_xfer_size == 0) {
1671 1798 stmf_scsilib_send_status(task, STATUS_CHECK,
1672 1799 STMF_SAA_INVALID_FIELD_IN_CDB);
1673 1800 return;
1674 1801 }
1675 1802 if (task->task_lu_private == NULL) {
1676 1803 task->task_lu_private = kmem_zalloc(sizeof (sbd_cmd_t),
1677 1804 KM_SLEEP);
1678 1805 } else {
1679 1806 bzero(task->task_lu_private, sizeof (sbd_cmd_t));
1680 1807 }
1681 1808 scmd = (sbd_cmd_t *)task->task_lu_private;
1682 1809 scmd->cmd_type = SBD_CMD_SMALL_WRITE;
1683 1810 scmd->flags = SBD_SCSI_CMD_ACTIVE;
1684 1811 scmd->len = cdb_xfer_size;
1685 1812 if (dbuf == NULL) {
1686 1813 uint32_t minsize = cdb_xfer_size;
1687 1814
1688 1815 dbuf = stmf_alloc_dbuf(task, cdb_xfer_size, &minsize, 0);
1689 1816 if (dbuf == NULL) {
1690 1817 stmf_abort(STMF_QUEUE_TASK_ABORT, task,
1691 1818 STMF_ALLOC_FAILURE, NULL);
1692 1819 return;
1693 1820 }
1694 1821 dbuf->db_data_size = cdb_xfer_size;
1695 1822 dbuf->db_relative_offset = 0;
1696 1823 dbuf->db_flags = DB_DIRECTION_FROM_RPORT;
1697 1824 (void) stmf_xfer_data(task, dbuf, 0);
1698 1825 } else {
1699 1826 if (dbuf->db_data_size < cdb_xfer_size) {
1700 1827 stmf_abort(STMF_QUEUE_TASK_ABORT, task,
1701 1828 STMF_ABORTED, NULL);
1702 1829 return;
1703 1830 }
1704 1831 dbuf->db_data_size = cdb_xfer_size;
1705 1832 sbd_handle_short_write_xfer_completion(task, dbuf);
1706 1833 }
1707 1834 }
1708 1835
1709 1836 void
1710 1837 sbd_handle_short_write_xfer_completion(scsi_task_t *task,
1711 1838 stmf_data_buf_t *dbuf)
1712 1839 {
1713 1840 sbd_cmd_t *scmd;
1714 1841 stmf_status_t st_ret;
1715 1842 sbd_lu_t *sl = (sbd_lu_t *)task->task_lu->lu_provider_private;
1716 1843
1717 1844 /*
1718 1845 * For now lets assume we will get only one sglist element
1719 1846 * for short writes. If that ever changes, we should allocate
1720 1847 * a local buffer and copy all the sg elements to one linear space.
1721 1848 */
1722 1849 if ((dbuf->db_xfer_status != STMF_SUCCESS) ||
1723 1850 (dbuf->db_sglist_length > 1)) {
1724 1851 stmf_abort(STMF_QUEUE_TASK_ABORT, task,
1725 1852 dbuf->db_xfer_status, NULL);
1726 1853 return;
1727 1854 }
1728 1855
1729 1856 task->task_nbytes_transferred = dbuf->db_data_size;
1730 1857 scmd = (sbd_cmd_t *)task->task_lu_private;
1731 1858 scmd->flags &= ~SBD_SCSI_CMD_ACTIVE;
1732 1859
1733 1860 /* Lets find out who to call */
1734 1861 switch (task->task_cdb[0]) {
1735 1862 case SCMD_MODE_SELECT:
1736 1863 case SCMD_MODE_SELECT_G1:
1737 1864 if (sl->sl_access_state == SBD_LU_STANDBY) {
1738 1865 st_ret = stmf_proxy_scsi_cmd(task, dbuf);
1739 1866 if (st_ret != STMF_SUCCESS) {
1740 1867 stmf_scsilib_send_status(task, STATUS_CHECK,
1741 1868 STMF_SAA_LU_NO_ACCESS_UNAVAIL);
|
↓ open down ↓ |
87 lines elided |
↑ open up ↑ |
1742 1869 }
1743 1870 } else {
1744 1871 sbd_handle_mode_select_xfer(task,
1745 1872 dbuf->db_sglist[0].seg_addr, dbuf->db_data_size);
1746 1873 }
1747 1874 break;
1748 1875 case SCMD_UNMAP:
1749 1876 sbd_handle_unmap_xfer(task,
1750 1877 dbuf->db_sglist[0].seg_addr, dbuf->db_data_size);
1751 1878 break;
1879 + case SCMD_EXTENDED_COPY:
1880 + sbd_handle_xcopy_xfer(task, dbuf->db_sglist[0].seg_addr);
1881 + break;
1752 1882 case SCMD_PERSISTENT_RESERVE_OUT:
1753 1883 if (sl->sl_access_state == SBD_LU_STANDBY) {
1754 1884 st_ret = stmf_proxy_scsi_cmd(task, dbuf);
1755 1885 if (st_ret != STMF_SUCCESS) {
1756 1886 stmf_scsilib_send_status(task, STATUS_CHECK,
1757 1887 STMF_SAA_LU_NO_ACCESS_UNAVAIL);
1758 1888 }
1759 1889 } else {
1760 1890 sbd_handle_pgr_out_data(task, dbuf);
1761 1891 }
1762 1892 break;
1763 1893 default:
1764 1894 /* This should never happen */
1765 1895 stmf_abort(STMF_QUEUE_TASK_ABORT, task,
1766 1896 STMF_ABORTED, NULL);
1767 1897 }
1768 1898 }
1769 1899
1770 1900 void
1771 1901 sbd_handle_read_capacity(struct scsi_task *task,
1772 1902 struct stmf_data_buf *initial_dbuf)
1773 1903 {
1774 1904 sbd_lu_t *sl = (sbd_lu_t *)task->task_lu->lu_provider_private;
1775 1905 uint32_t cdb_len;
1776 1906 uint8_t p[32];
1777 1907 uint64_t s;
1778 1908 uint16_t blksize;
1779 1909
1780 1910 s = sl->sl_lu_size >> sl->sl_data_blocksize_shift;
1781 1911 s--;
1782 1912 blksize = ((uint16_t)1) << sl->sl_data_blocksize_shift;
1783 1913
1784 1914 switch (task->task_cdb[0]) {
1785 1915 case SCMD_READ_CAPACITY:
1786 1916 if (s & 0xffffffff00000000ull) {
1787 1917 p[0] = p[1] = p[2] = p[3] = 0xFF;
1788 1918 } else {
1789 1919 p[0] = (s >> 24) & 0xff;
1790 1920 p[1] = (s >> 16) & 0xff;
1791 1921 p[2] = (s >> 8) & 0xff;
1792 1922 p[3] = s & 0xff;
1793 1923 }
1794 1924 p[4] = 0; p[5] = 0;
1795 1925 p[6] = (blksize >> 8) & 0xff;
1796 1926 p[7] = blksize & 0xff;
1797 1927 sbd_handle_short_read_transfers(task, initial_dbuf, p, 8, 8);
1798 1928 break;
1799 1929
1800 1930 case SCMD_SVC_ACTION_IN_G4:
1801 1931 cdb_len = READ_SCSI32(&task->task_cdb[10], uint32_t);
1802 1932 bzero(p, 32);
1803 1933 p[0] = (s >> 56) & 0xff;
1804 1934 p[1] = (s >> 48) & 0xff;
1805 1935 p[2] = (s >> 40) & 0xff;
1806 1936 p[3] = (s >> 32) & 0xff;
1807 1937 p[4] = (s >> 24) & 0xff;
1808 1938 p[5] = (s >> 16) & 0xff;
1809 1939 p[6] = (s >> 8) & 0xff;
1810 1940 p[7] = s & 0xff;
1811 1941 p[10] = (blksize >> 8) & 0xff;
1812 1942 p[11] = blksize & 0xff;
1813 1943 if (sl->sl_flags & SL_UNMAP_ENABLED) {
1814 1944 p[14] = 0x80;
1815 1945 }
1816 1946 sbd_handle_short_read_transfers(task, initial_dbuf, p,
1817 1947 cdb_len, 32);
1818 1948 break;
1819 1949 }
1820 1950 }
1821 1951
1822 1952 void
1823 1953 sbd_calc_geometry(uint64_t s, uint16_t blksize, uint8_t *nsectors,
1824 1954 uint8_t *nheads, uint32_t *ncyl)
1825 1955 {
1826 1956 if (s < (4ull * 1024ull * 1024ull * 1024ull)) {
1827 1957 *nsectors = 32;
1828 1958 *nheads = 8;
1829 1959 } else {
1830 1960 *nsectors = 254;
1831 1961 *nheads = 254;
1832 1962 }
1833 1963 *ncyl = s / ((uint64_t)blksize * (uint64_t)(*nsectors) *
1834 1964 (uint64_t)(*nheads));
1835 1965 }
|
↓ open down ↓ |
74 lines elided |
↑ open up ↑ |
1836 1966
1837 1967 void
1838 1968 sbd_handle_mode_sense(struct scsi_task *task,
1839 1969 struct stmf_data_buf *initial_dbuf, uint8_t *buf)
1840 1970 {
1841 1971 sbd_lu_t *sl = (sbd_lu_t *)task->task_lu->lu_provider_private;
1842 1972 uint32_t cmd_size, n;
1843 1973 uint8_t *cdb;
1844 1974 uint32_t ncyl;
1845 1975 uint8_t nsectors, nheads;
1846 - uint8_t page, ctrl, header_size, pc_valid;
1976 + uint8_t page, ctrl, header_size;
1847 1977 uint16_t nbytes;
1848 1978 uint8_t *p;
1849 1979 uint64_t s = sl->sl_lu_size;
1850 1980 uint32_t dev_spec_param_offset;
1851 1981
1852 1982 p = buf; /* buf is assumed to be zeroed out and large enough */
1853 1983 n = 0;
1854 1984 cdb = &task->task_cdb[0];
1855 1985 page = cdb[2] & 0x3F;
1856 1986 ctrl = (cdb[2] >> 6) & 3;
1857 - cmd_size = (cdb[0] == SCMD_MODE_SENSE) ? cdb[4] :
1858 - READ_SCSI16(&cdb[7], uint32_t);
1859 1987
1860 1988 if (cdb[0] == SCMD_MODE_SENSE) {
1989 + cmd_size = cdb[4];
1861 1990 header_size = 4;
1862 1991 dev_spec_param_offset = 2;
1863 1992 } else {
1993 + cmd_size = READ_SCSI16(&cdb[7], uint32_t);
1864 1994 header_size = 8;
1865 1995 dev_spec_param_offset = 3;
1866 1996 }
1867 1997
1868 1998 /* Now validate the command */
1869 - if ((cdb[2] == 0) || (page == MODEPAGE_ALLPAGES) || (page == 0x08) ||
1870 - (page == 0x0A) || (page == 0x03) || (page == 0x04)) {
1871 - pc_valid = 1;
1872 - } else {
1873 - pc_valid = 0;
1874 - }
1875 - if ((cmd_size < header_size) || (pc_valid == 0)) {
1999 + if ((cdb[2] != 0) && (page != MODEPAGE_ALLPAGES) &&
2000 + (page != MODEPAGE_CACHING) && (page != MODEPAGE_CTRL_MODE) &&
2001 + (page != MODEPAGE_FORMAT) && (page != MODEPAGE_GEOMETRY)) {
1876 2002 stmf_scsilib_send_status(task, STATUS_CHECK,
1877 2003 STMF_SAA_INVALID_FIELD_IN_CDB);
1878 2004 return;
1879 2005 }
1880 2006
1881 2007 /* We will update the length in the mode header at the end */
1882 2008
1883 2009 /* Block dev device specific param in mode param header has wp bit */
1884 2010 if (sl->sl_flags & SL_WRITE_PROTECTED) {
1885 2011 p[n + dev_spec_param_offset] = BIT_7;
1886 2012 }
1887 2013 n += header_size;
1888 2014 /* We are not going to return any block descriptor */
1889 2015
1890 2016 nbytes = ((uint16_t)1) << sl->sl_data_blocksize_shift;
1891 2017 sbd_calc_geometry(s, nbytes, &nsectors, &nheads, &ncyl);
1892 2018
1893 - if ((page == 0x03) || (page == MODEPAGE_ALLPAGES)) {
2019 + if ((page == MODEPAGE_FORMAT) || (page == MODEPAGE_ALLPAGES)) {
1894 2020 p[n] = 0x03;
1895 2021 p[n+1] = 0x16;
1896 2022 if (ctrl != 1) {
1897 2023 p[n + 11] = nsectors;
1898 2024 p[n + 12] = nbytes >> 8;
1899 2025 p[n + 13] = nbytes & 0xff;
1900 2026 p[n + 20] = 0x80;
1901 2027 }
1902 2028 n += 24;
1903 2029 }
1904 - if ((page == 0x04) || (page == MODEPAGE_ALLPAGES)) {
2030 + if ((page == MODEPAGE_GEOMETRY) || (page == MODEPAGE_ALLPAGES)) {
1905 2031 p[n] = 0x04;
1906 2032 p[n + 1] = 0x16;
1907 2033 if (ctrl != 1) {
1908 2034 p[n + 2] = ncyl >> 16;
1909 2035 p[n + 3] = ncyl >> 8;
1910 2036 p[n + 4] = ncyl & 0xff;
1911 2037 p[n + 5] = nheads;
1912 2038 p[n + 20] = 0x15;
1913 2039 p[n + 21] = 0x18;
1914 2040 }
1915 2041 n += 24;
1916 2042 }
1917 2043 if ((page == MODEPAGE_CACHING) || (page == MODEPAGE_ALLPAGES)) {
1918 2044 struct mode_caching *mode_caching_page;
1919 2045
1920 2046 mode_caching_page = (struct mode_caching *)&p[n];
1921 2047
1922 2048 mode_caching_page->mode_page.code = MODEPAGE_CACHING;
1923 2049 mode_caching_page->mode_page.ps = 1; /* A saveable page */
1924 2050 mode_caching_page->mode_page.length = 0x12;
1925 2051
1926 2052 switch (ctrl) {
1927 2053 case (0):
1928 2054 /* Current */
1929 2055 if ((sl->sl_flags & SL_WRITEBACK_CACHE_DISABLE) == 0) {
1930 2056 mode_caching_page->wce = 1;
1931 2057 }
1932 2058 break;
1933 2059
1934 2060 case (1):
1935 2061 /* Changeable */
1936 2062 if ((sl->sl_flags &
1937 2063 SL_WRITEBACK_CACHE_SET_UNSUPPORTED) == 0) {
1938 2064 mode_caching_page->wce = 1;
1939 2065 }
1940 2066 break;
1941 2067
1942 2068 default:
1943 2069 if ((sl->sl_flags &
1944 2070 SL_SAVED_WRITE_CACHE_DISABLE) == 0) {
1945 2071 mode_caching_page->wce = 1;
1946 2072 }
1947 2073 break;
1948 2074 }
1949 2075 n += (sizeof (struct mode_page) +
1950 2076 mode_caching_page->mode_page.length);
1951 2077 }
1952 2078 if ((page == MODEPAGE_CTRL_MODE) || (page == MODEPAGE_ALLPAGES)) {
1953 2079 struct mode_control_scsi3 *mode_control_page;
1954 2080
1955 2081 mode_control_page = (struct mode_control_scsi3 *)&p[n];
1956 2082
1957 2083 mode_control_page->mode_page.code = MODEPAGE_CTRL_MODE;
1958 2084 mode_control_page->mode_page.length =
1959 2085 PAGELENGTH_MODE_CONTROL_SCSI3;
1960 2086 if (ctrl != 1) {
1961 2087 /* If not looking for changeable values, report this. */
1962 2088 mode_control_page->que_mod = CTRL_QMOD_UNRESTRICT;
1963 2089 }
1964 2090 n += (sizeof (struct mode_page) +
1965 2091 mode_control_page->mode_page.length);
1966 2092 }
1967 2093
1968 2094 if (cdb[0] == SCMD_MODE_SENSE) {
|
↓ open down ↓ |
54 lines elided |
↑ open up ↑ |
1969 2095 if (n > 255) {
1970 2096 stmf_scsilib_send_status(task, STATUS_CHECK,
1971 2097 STMF_SAA_INVALID_FIELD_IN_CDB);
1972 2098 return;
1973 2099 }
1974 2100 /*
1975 2101 * Mode parameter header length doesn't include the number
1976 2102 * of bytes in the length field, so adjust the count.
1977 2103 * Byte count minus header length field size.
1978 2104 */
1979 - buf[0] = (n - 1) & 0xff;
2105 + buf[0] = (n - header_size) & 0xff;
1980 2106 } else {
1981 2107 /* Byte count minus header length field size. */
1982 - buf[1] = (n - 2) & 0xff;
1983 - buf[0] = ((n - 2) >> 8) & 0xff;
2108 + buf[1] = (n - header_size) & 0xff;
2109 + buf[0] = ((n - header_size) >> 8) & 0xff;
1984 2110 }
1985 2111
1986 2112 sbd_handle_short_read_transfers(task, initial_dbuf, buf,
1987 2113 cmd_size, n);
1988 2114 }
1989 2115
1990 2116 void
1991 2117 sbd_handle_mode_select(scsi_task_t *task, stmf_data_buf_t *dbuf)
1992 2118 {
1993 2119 uint32_t cmd_xfer_len;
1994 2120
1995 2121 if (task->task_cdb[0] == SCMD_MODE_SELECT) {
1996 2122 cmd_xfer_len = (uint32_t)task->task_cdb[4];
1997 2123 } else {
1998 2124 cmd_xfer_len = READ_SCSI16(&task->task_cdb[7], uint32_t);
1999 2125 }
2000 2126
2001 2127 if ((task->task_cdb[1] & 0xFE) != 0x10) {
2002 2128 stmf_scsilib_send_status(task, STATUS_CHECK,
2003 2129 STMF_SAA_INVALID_FIELD_IN_CDB);
2004 2130 return;
2005 2131 }
2006 2132
2007 2133 if (cmd_xfer_len == 0) {
2008 2134 /* zero byte mode selects are allowed */
2009 2135 stmf_scsilib_send_status(task, STATUS_GOOD, 0);
2010 2136 return;
2011 2137 }
2012 2138
2013 2139 sbd_handle_short_write_transfers(task, dbuf, cmd_xfer_len);
2014 2140 }
2015 2141
2016 2142 void
2017 2143 sbd_handle_mode_select_xfer(scsi_task_t *task, uint8_t *buf, uint32_t buflen)
2018 2144 {
2019 2145 sbd_lu_t *sl = (sbd_lu_t *)task->task_lu->lu_provider_private;
2020 2146 sbd_it_data_t *it;
2021 2147 int hdr_len, bd_len;
2022 2148 sbd_status_t sret;
2023 2149 int i;
2024 2150
2025 2151 if (task->task_cdb[0] == SCMD_MODE_SELECT) {
2026 2152 hdr_len = 4;
2027 2153 } else {
2028 2154 hdr_len = 8;
2029 2155 }
2030 2156
2031 2157 if (buflen < hdr_len)
2032 2158 goto mode_sel_param_len_err;
2033 2159
2034 2160 bd_len = hdr_len == 4 ? buf[3] : READ_SCSI16(&buf[6], int);
2035 2161
2036 2162 if (buflen < (hdr_len + bd_len + 2))
2037 2163 goto mode_sel_param_len_err;
2038 2164
2039 2165 buf += hdr_len + bd_len;
2040 2166 buflen -= hdr_len + bd_len;
2041 2167
2042 2168 if ((buf[0] != 8) || (buflen != ((uint32_t)buf[1] + 2))) {
2043 2169 goto mode_sel_param_len_err;
2044 2170 }
2045 2171
2046 2172 if (buf[2] & 0xFB) {
2047 2173 goto mode_sel_param_field_err;
2048 2174 }
2049 2175
2050 2176 for (i = 3; i < (buf[1] + 2); i++) {
2051 2177 if (buf[i]) {
2052 2178 goto mode_sel_param_field_err;
2053 2179 }
2054 2180 }
2055 2181
2056 2182 sret = SBD_SUCCESS;
2057 2183
2058 2184 /* All good. Lets handle the write cache change, if any */
2059 2185 if (buf[2] & BIT_2) {
2060 2186 sret = sbd_wcd_set(0, sl);
2061 2187 } else {
2062 2188 sret = sbd_wcd_set(1, sl);
2063 2189 }
2064 2190
2065 2191 if (sret != SBD_SUCCESS) {
2066 2192 stmf_scsilib_send_status(task, STATUS_CHECK,
2067 2193 STMF_SAA_WRITE_ERROR);
2068 2194 return;
2069 2195 }
2070 2196
2071 2197 /* set on the device passed, now set the flags */
2072 2198 mutex_enter(&sl->sl_lock);
2073 2199 if (buf[2] & BIT_2) {
2074 2200 sl->sl_flags &= ~SL_WRITEBACK_CACHE_DISABLE;
2075 2201 } else {
2076 2202 sl->sl_flags |= SL_WRITEBACK_CACHE_DISABLE;
2077 2203 }
2078 2204
2079 2205 for (it = sl->sl_it_list; it != NULL; it = it->sbd_it_next) {
2080 2206 if (it == task->task_lu_itl_handle)
2081 2207 continue;
2082 2208 it->sbd_it_ua_conditions |= SBD_UA_MODE_PARAMETERS_CHANGED;
2083 2209 }
2084 2210
2085 2211 if (task->task_cdb[1] & 1) {
2086 2212 if (buf[2] & BIT_2) {
2087 2213 sl->sl_flags &= ~SL_SAVED_WRITE_CACHE_DISABLE;
2088 2214 } else {
2089 2215 sl->sl_flags |= SL_SAVED_WRITE_CACHE_DISABLE;
2090 2216 }
2091 2217 mutex_exit(&sl->sl_lock);
2092 2218 sret = sbd_write_lu_info(sl);
2093 2219 } else {
2094 2220 mutex_exit(&sl->sl_lock);
2095 2221 }
2096 2222 if (sret == SBD_SUCCESS) {
2097 2223 stmf_scsilib_send_status(task, STATUS_GOOD, 0);
2098 2224 } else {
2099 2225 stmf_scsilib_send_status(task, STATUS_CHECK,
2100 2226 STMF_SAA_WRITE_ERROR);
2101 2227 }
2102 2228 return;
2103 2229
2104 2230 mode_sel_param_len_err:
2105 2231 stmf_scsilib_send_status(task, STATUS_CHECK,
2106 2232 STMF_SAA_PARAM_LIST_LENGTH_ERROR);
2107 2233 return;
2108 2234 mode_sel_param_field_err:
2109 2235 stmf_scsilib_send_status(task, STATUS_CHECK,
2110 2236 STMF_SAA_INVALID_FIELD_IN_PARAM_LIST);
2111 2237 }
2112 2238
2113 2239 /*
2114 2240 * Command support added from SPC-4 r24
2115 2241 * Supports info type 0, 2, 127
2116 2242 */
2117 2243 void
2118 2244 sbd_handle_identifying_info(struct scsi_task *task,
2119 2245 stmf_data_buf_t *initial_dbuf)
2120 2246 {
2121 2247 sbd_lu_t *sl = (sbd_lu_t *)task->task_lu->lu_provider_private;
2122 2248 uint8_t *cdb;
2123 2249 uint32_t cmd_size;
2124 2250 uint32_t param_len;
2125 2251 uint32_t xfer_size;
2126 2252 uint8_t info_type;
2127 2253 uint8_t *buf, *p;
2128 2254
2129 2255 cdb = &task->task_cdb[0];
2130 2256 cmd_size = READ_SCSI32(&cdb[6], uint32_t);
2131 2257 info_type = cdb[10]>>1;
2132 2258
2133 2259 /* Validate the command */
2134 2260 if (cmd_size < 4) {
2135 2261 stmf_scsilib_send_status(task, STATUS_CHECK,
2136 2262 STMF_SAA_INVALID_FIELD_IN_CDB);
2137 2263 return;
2138 2264 }
2139 2265
2140 2266 p = buf = kmem_zalloc(260, KM_SLEEP);
2141 2267
2142 2268 switch (info_type) {
2143 2269 case 0:
2144 2270 /*
2145 2271 * No value is supplied but this info type
2146 2272 * is mandatory.
2147 2273 */
2148 2274 xfer_size = 4;
2149 2275 break;
2150 2276 case 2:
2151 2277 mutex_enter(&sl->sl_lock);
2152 2278 param_len = strlcpy((char *)(p+4), sl->sl_alias, 256);
2153 2279 mutex_exit(&sl->sl_lock);
2154 2280 /* text info must be null terminated */
2155 2281 if (++param_len > 256)
2156 2282 param_len = 256;
2157 2283 SCSI_WRITE16(p+2, param_len);
2158 2284 xfer_size = param_len + 4;
2159 2285 break;
2160 2286 case 127:
2161 2287 /* 0 and 2 descriptor supported */
2162 2288 SCSI_WRITE16(p+2, 8); /* set param length */
2163 2289 p += 8;
2164 2290 *p = 4; /* set type to 2 (7 hi bits) */
2165 2291 p += 2;
2166 2292 SCSI_WRITE16(p, 256); /* 256 max length */
2167 2293 xfer_size = 12;
2168 2294 break;
2169 2295 default:
2170 2296 stmf_scsilib_send_status(task, STATUS_CHECK,
2171 2297 STMF_SAA_INVALID_FIELD_IN_CDB);
2172 2298 kmem_free(buf, 260);
2173 2299 return;
2174 2300 }
2175 2301 sbd_handle_short_read_transfers(task, initial_dbuf, buf,
2176 2302 cmd_size, xfer_size);
2177 2303 kmem_free(buf, 260);
2178 2304 }
2179 2305
2180 2306 /*
2181 2307 * This function parse through a string, passed to it as a pointer to a string,
|
↓ open down ↓ |
188 lines elided |
↑ open up ↑ |
2182 2308 * by adjusting the pointer to the first non-space character and returns
2183 2309 * the count/length of the first bunch of non-space characters. Multiple
2184 2310 * Management URLs are stored as a space delimited string in sl_mgmt_url
2185 2311 * field of sbd_lu_t. This function is used to retrieve one url at a time.
2186 2312 *
2187 2313 * i/p : pointer to pointer to a url string
2188 2314 * o/p : Adjust the pointer to the url to the first non white character
2189 2315 * and returns the length of the URL
2190 2316 */
2191 2317 uint16_t
2192 -sbd_parse_mgmt_url(char **url_addr) {
2318 +sbd_parse_mgmt_url(char **url_addr)
2319 +{
2193 2320 uint16_t url_length = 0;
2194 2321 char *url;
2195 2322 url = *url_addr;
2196 2323
2197 2324 while (*url != '\0') {
2198 2325 if (*url == ' ' || *url == '\t' || *url == '\n') {
2199 2326 (*url_addr)++;
2200 2327 url = *url_addr;
2201 2328 } else {
2202 2329 break;
2203 2330 }
2204 2331 }
2205 2332
2206 2333 while (*url != '\0') {
2207 2334 if (*url == ' ' || *url == '\t' ||
2208 2335 *url == '\n' || *url == '\0') {
2209 2336 break;
2210 2337 }
2211 2338 url++;
2212 2339 url_length++;
2213 2340 }
2214 2341 return (url_length);
2215 2342 }
2216 2343
2217 2344 /* Try to make this the size of a kmem allocation cache. */
2218 2345 static uint_t sbd_write_same_optimal_chunk = 128 * 1024;
2219 2346
2220 2347 static sbd_status_t
2221 2348 sbd_write_same_data(struct scsi_task *task, sbd_cmd_t *scmd)
2222 2349 {
2223 2350 sbd_lu_t *sl = (sbd_lu_t *)task->task_lu->lu_provider_private;
2224 2351 uint64_t addr, len, sz_done;
2225 2352 uint32_t big_buf_size, xfer_size, off;
2226 2353 uint8_t *big_buf;
2227 2354 sbd_status_t ret;
2228 2355
2229 2356 if (task->task_cdb[0] == SCMD_WRITE_SAME_G1) {
2230 2357 addr = READ_SCSI32(&task->task_cdb[2], uint64_t);
2231 2358 len = READ_SCSI16(&task->task_cdb[7], uint64_t);
2232 2359 } else {
2233 2360 addr = READ_SCSI64(&task->task_cdb[2], uint64_t);
2234 2361 len = READ_SCSI32(&task->task_cdb[10], uint64_t);
2235 2362 }
2236 2363 addr <<= sl->sl_data_blocksize_shift;
2237 2364 len <<= sl->sl_data_blocksize_shift;
2238 2365
2239 2366 /*
2240 2367 * Reminders:
2241 2368 * "len" is total size of what we wish to "write same".
2242 2369 *
2243 2370 * xfer_size will be scmd->trans_data_len, which is the length
2244 2371 * of the pattern we wish to replicate over "len". We replicate
2245 2372 * "xfer_size" of pattern over "len".
2246 2373 *
2247 2374 * big_buf_size is set to an ideal actual-write size for an output
2248 2375 * operation. It may be the same as "len". If it's not, it should
2249 2376 * be an exact multiple of "xfer_size" so we don't get pattern
2250 2377 * breakage until the very end of "len".
2251 2378 */
2252 2379 big_buf_size = len > sbd_write_same_optimal_chunk ?
2253 2380 sbd_write_same_optimal_chunk : (uint32_t)len;
2254 2381 xfer_size = scmd->trans_data_len;
2255 2382
2256 2383 /*
2257 2384 * All transfers should be an integral multiple of the sector size.
2258 2385 */
2259 2386 ASSERT((big_buf_size % xfer_size) == 0);
2260 2387
2261 2388 /*
2262 2389 * Don't sleep for the allocation, and don't make the system
2263 2390 * reclaim memory. Trade higher I/Os if in a low-memory situation.
2264 2391 */
2265 2392 big_buf = kmem_alloc(big_buf_size, KM_NOSLEEP | KM_NORMALPRI);
2266 2393
2267 2394 if (big_buf == NULL) {
2268 2395 /*
2269 2396 * Just send it in terms of of the transmitted data. This
2270 2397 * will be very slow.
2271 2398 */
2272 2399 DTRACE_PROBE1(write__same__low__memory, uint64_t, big_buf_size);
2273 2400 big_buf = scmd->trans_data;
2274 2401 big_buf_size = scmd->trans_data_len;
2275 2402 } else {
2276 2403 /*
2277 2404 * We already ASSERT()ed big_buf_size is an integral multiple
2278 2405 * of xfer_size.
2279 2406 */
2280 2407 for (off = 0; off < big_buf_size; off += xfer_size)
2281 2408 bcopy(scmd->trans_data, big_buf + off, xfer_size);
2282 2409 }
2283 2410
2284 2411 /* Do the actual I/O. Recycle xfer_size now to be write size. */
2285 2412 DTRACE_PROBE1(write__same__io__begin, uint64_t, len);
2286 2413 for (sz_done = 0; sz_done < len; sz_done += (uint64_t)xfer_size) {
2287 2414 xfer_size = ((big_buf_size + sz_done) <= len) ? big_buf_size :
2288 2415 len - sz_done;
2289 2416 ret = sbd_data_write(sl, task, addr + sz_done,
2290 2417 (uint64_t)xfer_size, big_buf);
2291 2418 if (ret != SBD_SUCCESS)
2292 2419 break;
|
↓ open down ↓ |
90 lines elided |
↑ open up ↑ |
2293 2420 }
2294 2421 DTRACE_PROBE2(write__same__io__end, uint64_t, len, uint64_t, sz_done);
2295 2422
2296 2423 if (big_buf != scmd->trans_data)
2297 2424 kmem_free(big_buf, big_buf_size);
2298 2425
2299 2426 return (ret);
2300 2427 }
2301 2428
2302 2429 static void
2430 +sbd_write_same_release_resources(struct scsi_task *task)
2431 +{
2432 + sbd_cmd_t *scmd = (sbd_cmd_t *)task->task_lu_private;
2433 +
2434 + if (scmd->nbufs == 0XFF)
2435 + cmn_err(CE_WARN, "%s invalid buffer count %x",
2436 + __func__, scmd->nbufs);
2437 + if ((scmd->trans_data_len != 0) && (scmd->trans_data != NULL))
2438 + kmem_free(scmd->trans_data, scmd->trans_data_len);
2439 + scmd->trans_data = NULL;
2440 + scmd->trans_data_len = 0;
2441 + scmd->flags &= ~SBD_SCSI_CMD_TRANS_DATA;
2442 +}
2443 +
2444 +static void
2303 2445 sbd_handle_write_same_xfer_completion(struct scsi_task *task, sbd_cmd_t *scmd,
2304 2446 struct stmf_data_buf *dbuf, uint8_t dbuf_reusable)
2305 2447 {
2306 2448 uint64_t laddr;
2307 2449 uint32_t buflen, iolen;
2308 2450 int ndx, ret;
2309 2451
2452 + if (ATOMIC8_GET(scmd->nbufs) > 0) {
2453 + atomic_dec_8(&scmd->nbufs);
2454 + }
2455 +
2310 2456 if (dbuf->db_xfer_status != STMF_SUCCESS) {
2457 + sbd_write_same_release_resources(task);
2458 + sbd_ats_remove_by_task(task);
2311 2459 stmf_abort(STMF_QUEUE_TASK_ABORT, task,
2312 2460 dbuf->db_xfer_status, NULL);
2313 2461 return;
2314 2462 }
2315 2463
2316 2464 if (scmd->flags & SBD_SCSI_CMD_XFER_FAIL) {
2317 2465 goto write_same_xfer_done;
2318 2466 }
2319 2467
2320 - if (scmd->len != 0) {
2468 + /* if this is a unnessary callback just return */
2469 + if (((scmd->flags & SBD_SCSI_CMD_TRANS_DATA) == 0) ||
2470 + ((scmd->flags & SBD_SCSI_CMD_ACTIVE) == 0) ||
2471 + (scmd->trans_data == NULL)) {
2472 + sbd_ats_remove_by_task(task);
2473 + scmd->flags &= ~SBD_SCSI_CMD_ACTIVE;
2474 + return;
2475 + }
2476 +
2477 + if (ATOMIC32_GET(scmd->len) != 0) {
2321 2478 /*
2322 2479 * Initiate the next port xfer to occur in parallel
2323 2480 * with writing this buf.
2324 2481 */
2325 2482 sbd_do_write_same_xfer(task, scmd, NULL, 0);
2326 2483 }
2327 2484
2328 2485 laddr = dbuf->db_relative_offset;
2329 2486
2330 2487 for (buflen = 0, ndx = 0; (buflen < dbuf->db_data_size) &&
2331 2488 (ndx < dbuf->db_sglist_length); ndx++) {
2332 2489 iolen = min(dbuf->db_data_size - buflen,
2333 2490 dbuf->db_sglist[ndx].seg_length);
|
↓ open down ↓ |
3 lines elided |
↑ open up ↑ |
2334 2491 if (iolen == 0)
2335 2492 break;
2336 2493 bcopy(dbuf->db_sglist[ndx].seg_addr, &scmd->trans_data[laddr],
2337 2494 iolen);
2338 2495 buflen += iolen;
2339 2496 laddr += (uint64_t)iolen;
2340 2497 }
2341 2498 task->task_nbytes_transferred += buflen;
2342 2499
2343 2500 write_same_xfer_done:
2344 - if (scmd->len == 0 || scmd->flags & SBD_SCSI_CMD_XFER_FAIL) {
2501 + if (ATOMIC32_GET(scmd->len) == 0 ||
2502 + scmd->flags & SBD_SCSI_CMD_XFER_FAIL) {
2345 2503 stmf_free_dbuf(task, dbuf);
2504 + if (ATOMIC8_GET(scmd->nbufs) > 0)
2505 + return;
2346 2506 scmd->flags &= ~SBD_SCSI_CMD_ACTIVE;
2347 2507 if (scmd->flags & SBD_SCSI_CMD_XFER_FAIL) {
2508 + sbd_ats_remove_by_task(task);
2509 + sbd_write_same_release_resources(task);
2348 2510 stmf_scsilib_send_status(task, STATUS_CHECK,
2349 2511 STMF_SAA_WRITE_ERROR);
2350 2512 } else {
2351 2513 ret = sbd_write_same_data(task, scmd);
2514 + sbd_ats_remove_by_task(task);
2515 + sbd_write_same_release_resources(task);
2352 2516 if (ret != SBD_SUCCESS) {
2353 2517 stmf_scsilib_send_status(task, STATUS_CHECK,
2354 2518 STMF_SAA_WRITE_ERROR);
2355 2519 } else {
2356 2520 stmf_scsilib_send_status(task, STATUS_GOOD, 0);
2357 2521 }
2358 2522 }
2359 - /*
2360 - * Only way we should get here is via handle_write_same(),
2361 - * and that should make the following assertion always pass.
2362 - */
2363 - ASSERT((scmd->flags & SBD_SCSI_CMD_TRANS_DATA) &&
2364 - scmd->trans_data != NULL);
2365 - kmem_free(scmd->trans_data, scmd->trans_data_len);
2366 - scmd->flags &= ~SBD_SCSI_CMD_TRANS_DATA;
2367 2523 return;
2368 2524 }
2369 2525 sbd_do_write_same_xfer(task, scmd, dbuf, dbuf_reusable);
2370 2526 }
2371 2527
2372 2528 static void
2373 2529 sbd_do_write_same_xfer(struct scsi_task *task, sbd_cmd_t *scmd,
2374 2530 struct stmf_data_buf *dbuf, uint8_t dbuf_reusable)
2375 2531 {
2376 2532 uint32_t len;
2377 2533
2378 - if (scmd->len == 0) {
2534 + if (ATOMIC32_GET(scmd->len) == 0) {
2379 2535 if (dbuf != NULL)
2380 2536 stmf_free_dbuf(task, dbuf);
2381 2537 return;
2382 2538 }
2383 2539
2384 2540 if ((dbuf != NULL) &&
2385 2541 ((dbuf->db_flags & DB_DONT_REUSE) || (dbuf_reusable == 0))) {
2386 2542 /* free current dbuf and allocate a new one */
2387 2543 stmf_free_dbuf(task, dbuf);
2388 2544 dbuf = NULL;
2389 2545 }
2390 2546 if (dbuf == NULL) {
2391 2547 uint32_t maxsize, minsize, old_minsize;
2392 2548
2393 - maxsize = (scmd->len > (128*1024)) ? 128*1024 :
2394 - scmd->len;
2549 + maxsize = (ATOMIC32_GET(scmd->len) > (128*1024)) ? 128*1024 :
2550 + ATOMIC32_GET(scmd->len);
2395 2551 minsize = maxsize >> 2;
2396 2552 do {
2397 2553 old_minsize = minsize;
2398 2554 dbuf = stmf_alloc_dbuf(task, maxsize, &minsize, 0);
2399 2555 } while ((dbuf == NULL) && (old_minsize > minsize) &&
2400 2556 (minsize >= 512));
2401 2557 if (dbuf == NULL) {
2402 - if (scmd->nbufs == 0) {
2558 + sbd_ats_remove_by_task(task);
2559 + sbd_write_same_release_resources(task);
2560 + if (ATOMIC8_GET(scmd->nbufs) == 0) {
2403 2561 stmf_abort(STMF_QUEUE_TASK_ABORT, task,
2404 2562 STMF_ALLOC_FAILURE, NULL);
2405 2563 }
2406 2564 return;
2407 2565 }
2408 2566 }
2409 2567
2410 - len = scmd->len > dbuf->db_buf_size ? dbuf->db_buf_size :
2411 - scmd->len;
2568 + len = ATOMIC32_GET(scmd->len) > dbuf->db_buf_size ? dbuf->db_buf_size :
2569 + ATOMIC32_GET(scmd->len);
2412 2570
2413 2571 dbuf->db_relative_offset = scmd->current_ro;
2414 2572 dbuf->db_data_size = len;
2415 2573 dbuf->db_flags = DB_DIRECTION_FROM_RPORT;
2416 2574 (void) stmf_xfer_data(task, dbuf, 0);
2417 - scmd->nbufs++; /* outstanding port xfers and bufs used */
2418 - scmd->len -= len;
2575 + /* outstanding port xfers and bufs used */
2576 + atomic_inc_8(&scmd->nbufs);
2577 + atomic_add_32(&scmd->len, -len);
2419 2578 scmd->current_ro += len;
2420 2579 }
2421 2580
2422 2581 static void
2423 2582 sbd_handle_write_same(scsi_task_t *task, struct stmf_data_buf *initial_dbuf)
2424 2583 {
2425 2584 sbd_lu_t *sl = (sbd_lu_t *)task->task_lu->lu_provider_private;
2426 2585 uint64_t addr, len;
2427 2586 sbd_cmd_t *scmd;
2428 2587 stmf_data_buf_t *dbuf;
2429 2588 uint8_t unmap;
2430 2589 uint8_t do_immediate_data = 0;
2431 2590
2591 + if (HardwareAcceleratedInit == 0) {
2592 + stmf_scsilib_send_status(task, STATUS_CHECK,
2593 + STMF_SAA_INVALID_OPCODE);
2594 + return;
2595 + }
2596 +
2432 2597 task->task_cmd_xfer_length = 0;
2433 2598 if (task->task_additional_flags &
2434 2599 TASK_AF_NO_EXPECTED_XFER_LENGTH) {
2435 2600 task->task_expected_xfer_length = 0;
2436 2601 }
2437 2602 if (sl->sl_flags & SL_WRITE_PROTECTED) {
2438 2603 stmf_scsilib_send_status(task, STATUS_CHECK,
2439 2604 STMF_SAA_WRITE_PROTECTED);
2440 2605 return;
2441 2606 }
2442 2607 if (task->task_cdb[1] & 0xF7) {
2443 2608 stmf_scsilib_send_status(task, STATUS_CHECK,
2444 2609 STMF_SAA_INVALID_FIELD_IN_CDB);
2445 2610 return;
2446 2611 }
2447 2612 unmap = task->task_cdb[1] & 0x08;
2613 +
2448 2614 if (unmap && ((sl->sl_flags & SL_UNMAP_ENABLED) == 0)) {
2449 2615 stmf_scsilib_send_status(task, STATUS_CHECK,
2450 2616 STMF_SAA_INVALID_FIELD_IN_CDB);
2451 2617 return;
2452 2618 }
2619 +
2453 2620 if (task->task_cdb[0] == SCMD_WRITE_SAME_G1) {
2454 2621 addr = READ_SCSI32(&task->task_cdb[2], uint64_t);
2455 2622 len = READ_SCSI16(&task->task_cdb[7], uint64_t);
2456 2623 } else {
2457 2624 addr = READ_SCSI64(&task->task_cdb[2], uint64_t);
2458 2625 len = READ_SCSI32(&task->task_cdb[10], uint64_t);
2459 2626 }
2627 +
2460 2628 if (len == 0) {
2461 2629 stmf_scsilib_send_status(task, STATUS_CHECK,
2462 2630 STMF_SAA_INVALID_FIELD_IN_CDB);
2463 2631 return;
2464 2632 }
2633 +
2634 + if (sbd_ats_handling_before_io(task, sl, addr, len) !=
2635 + SBD_SUCCESS) {
2636 + if (stmf_task_poll_lu(task, 10) != STMF_SUCCESS)
2637 + stmf_scsilib_send_status(task, STATUS_BUSY, 0);
2638 + return;
2639 + }
2640 +
2465 2641 addr <<= sl->sl_data_blocksize_shift;
2466 2642 len <<= sl->sl_data_blocksize_shift;
2467 2643
2468 2644 /* Check if the command is for the unmap function */
2469 2645 if (unmap) {
2470 - if (sbd_unmap(sl, addr, len) != 0) {
2646 + dkioc_free_list_t *dfl = kmem_zalloc(DFL_SZ(1), KM_SLEEP);
2647 +
2648 + dfl->dfl_num_exts = 1;
2649 + dfl->dfl_exts[0].dfle_start = addr;
2650 + dfl->dfl_exts[0].dfle_length = len;
2651 + if (sbd_unmap(sl, dfl) != 0) {
2471 2652 stmf_scsilib_send_status(task, STATUS_CHECK,
2472 2653 STMF_SAA_LBA_OUT_OF_RANGE);
2473 2654 } else {
2474 2655 stmf_scsilib_send_status(task, STATUS_GOOD, 0);
2475 2656 }
2657 + dfl_free(dfl);
2476 2658 return;
2477 2659 }
2478 2660
2479 2661 /* Write same function */
2480 2662
2481 2663 task->task_cmd_xfer_length = 1 << sl->sl_data_blocksize_shift;
2482 2664 if (task->task_additional_flags &
2483 2665 TASK_AF_NO_EXPECTED_XFER_LENGTH) {
2484 2666 task->task_expected_xfer_length = task->task_cmd_xfer_length;
2485 2667 }
2486 2668 if ((addr + len) > sl->sl_lu_size) {
2669 + sbd_ats_remove_by_task(task);
2487 2670 stmf_scsilib_send_status(task, STATUS_CHECK,
2488 2671 STMF_SAA_LBA_OUT_OF_RANGE);
2489 2672 return;
2490 2673 }
2491 2674
2492 2675 /* For rest of this I/O the transfer length is 1 block */
2493 2676 len = ((uint64_t)1) << sl->sl_data_blocksize_shift;
2494 2677
2495 2678 /* Some basic checks */
2496 2679 if ((len == 0) || (len != task->task_expected_xfer_length)) {
2680 + sbd_ats_remove_by_task(task);
2497 2681 stmf_scsilib_send_status(task, STATUS_CHECK,
2498 2682 STMF_SAA_INVALID_FIELD_IN_CDB);
2499 2683 return;
2500 2684 }
2501 2685
2502 2686
2503 2687 if ((initial_dbuf != NULL) && (task->task_flags & TF_INITIAL_BURST)) {
2504 2688 if (initial_dbuf->db_data_size > len) {
2505 2689 if (initial_dbuf->db_data_size >
2506 2690 task->task_expected_xfer_length) {
2507 2691 /* protocol error */
2692 + sbd_ats_remove_by_task(task);
2508 2693 stmf_abort(STMF_QUEUE_TASK_ABORT, task,
2509 2694 STMF_INVALID_ARG, NULL);
2510 2695 return;
2511 2696 }
2512 2697 initial_dbuf->db_data_size = (uint32_t)len;
2513 2698 }
2514 2699 do_immediate_data = 1;
2515 2700 }
2516 2701 dbuf = initial_dbuf;
2517 2702
2518 2703 if (task->task_lu_private) {
2519 2704 scmd = (sbd_cmd_t *)task->task_lu_private;
2520 2705 } else {
2521 2706 scmd = (sbd_cmd_t *)kmem_alloc(sizeof (sbd_cmd_t), KM_SLEEP);
2522 2707 task->task_lu_private = scmd;
2523 2708 }
2524 - scmd->flags = SBD_SCSI_CMD_ACTIVE | SBD_SCSI_CMD_TRANS_DATA;
2709 + scmd->flags = SBD_SCSI_CMD_ACTIVE | SBD_SCSI_CMD_TRANS_DATA |
2710 + SBD_SCSI_CMD_ATS_RELATED;
2525 2711 scmd->cmd_type = SBD_CMD_SCSI_WRITE;
2526 2712 scmd->nbufs = 0;
2527 2713 scmd->len = (uint32_t)len;
2528 2714 scmd->trans_data_len = (uint32_t)len;
2529 2715 scmd->trans_data = kmem_alloc((size_t)len, KM_SLEEP);
2530 2716 scmd->current_ro = 0;
2531 2717
2532 2718 if (do_immediate_data) {
2533 2719 /*
2534 2720 * Account for data passed in this write command
2535 2721 */
2536 2722 (void) stmf_xfer_data(task, dbuf, STMF_IOF_STATS_ONLY);
2537 - scmd->len -= dbuf->db_data_size;
2723 + atomic_add_32(&scmd->len, -dbuf->db_data_size);
2538 2724 scmd->current_ro += dbuf->db_data_size;
2539 2725 dbuf->db_xfer_status = STMF_SUCCESS;
2540 2726 sbd_handle_write_same_xfer_completion(task, scmd, dbuf, 0);
2541 2727 } else {
2542 2728 sbd_do_write_same_xfer(task, scmd, dbuf, 0);
2543 2729 }
2544 2730 }
2545 2731
2546 2732 static void
2547 2733 sbd_handle_unmap(scsi_task_t *task, stmf_data_buf_t *dbuf)
2548 2734 {
2735 + sbd_lu_t *sl = (sbd_lu_t *)task->task_lu->lu_provider_private;
2549 2736 uint32_t cmd_xfer_len;
2550 2737
2738 + if (sbd_unmap_enable == 0) {
2739 + stmf_scsilib_send_status(task, STATUS_CHECK,
2740 + STMF_SAA_INVALID_OPCODE);
2741 + return;
2742 + }
2743 +
2744 + if (sl->sl_flags & SL_WRITE_PROTECTED) {
2745 + stmf_scsilib_send_status(task, STATUS_CHECK,
2746 + STMF_SAA_WRITE_PROTECTED);
2747 + return;
2748 + }
2551 2749 cmd_xfer_len = READ_SCSI16(&task->task_cdb[7], uint32_t);
2552 2750
2553 2751 if (task->task_cdb[1] & 1) {
2554 2752 stmf_scsilib_send_status(task, STATUS_CHECK,
2555 2753 STMF_SAA_INVALID_FIELD_IN_CDB);
2556 2754 return;
2557 2755 }
2558 2756
2559 2757 if (cmd_xfer_len == 0) {
2560 2758 task->task_cmd_xfer_length = 0;
2561 2759 if (task->task_additional_flags &
2562 2760 TASK_AF_NO_EXPECTED_XFER_LENGTH) {
2563 2761 task->task_expected_xfer_length = 0;
2564 2762 }
2565 2763 stmf_scsilib_send_status(task, STATUS_GOOD, 0);
2566 2764 return;
2567 2765 }
2568 2766
|
↓ open down ↓ |
8 lines elided |
↑ open up ↑ |
2569 2767 sbd_handle_short_write_transfers(task, dbuf, cmd_xfer_len);
2570 2768 }
2571 2769
2572 2770 static void
2573 2771 sbd_handle_unmap_xfer(scsi_task_t *task, uint8_t *buf, uint32_t buflen)
2574 2772 {
2575 2773 sbd_lu_t *sl = (sbd_lu_t *)task->task_lu->lu_provider_private;
2576 2774 uint32_t ulen, dlen, num_desc;
2577 2775 uint64_t addr, len;
2578 2776 uint8_t *p;
2777 + dkioc_free_list_t *dfl;
2579 2778 int ret;
2779 + int i;
2580 2780
2581 2781 if (buflen < 24) {
2582 2782 stmf_scsilib_send_status(task, STATUS_CHECK,
2583 2783 STMF_SAA_INVALID_FIELD_IN_CDB);
2584 2784 return;
2585 2785 }
2586 2786 ulen = READ_SCSI16(buf, uint32_t);
2587 2787 dlen = READ_SCSI16(buf + 2, uint32_t);
2588 2788 num_desc = dlen >> 4;
2589 2789 if (((ulen + 2) != buflen) || ((dlen + 8) != buflen) || (dlen & 0xf) ||
2590 2790 (num_desc == 0)) {
2591 2791 stmf_scsilib_send_status(task, STATUS_CHECK,
2592 2792 STMF_SAA_INVALID_FIELD_IN_CDB);
2593 2793 return;
2594 2794 }
2595 2795
2596 - for (p = buf + 8; num_desc; num_desc--, p += 16) {
2796 + dfl = kmem_zalloc(DFL_SZ(num_desc), KM_SLEEP);
2797 + dfl->dfl_num_exts = num_desc;
2798 + /*
2799 + * This should use ATS locking but that was disabled by the
2800 + * changes to ZFS top take advantage of TRIM in SSDs.
2801 + *
2802 + * Since the entire list is passed to ZFS in one list ATS
2803 + * locking is not done. This may be detectable, and if it is
2804 + * then the entire list needs to be locked and then after the
2805 + * unmap completes the entire list must be unlocked
2806 + */
2807 + for (p = buf + 8, i = 0; num_desc; num_desc--, p += 16, i++) {
2597 2808 addr = READ_SCSI64(p, uint64_t);
2598 - addr <<= sl->sl_data_blocksize_shift;
2599 2809 len = READ_SCSI32(p+8, uint64_t);
2810 + addr <<= sl->sl_data_blocksize_shift;
2600 2811 len <<= sl->sl_data_blocksize_shift;
2601 - ret = sbd_unmap(sl, addr, len);
2602 - if (ret != 0) {
2603 - stmf_scsilib_send_status(task, STATUS_CHECK,
2604 - STMF_SAA_LBA_OUT_OF_RANGE);
2605 - return;
2606 - }
2812 +
2813 + /* Prepare a list of extents to unmap */
2814 + dfl->dfl_exts[i].dfle_start = addr;
2815 + dfl->dfl_exts[i].dfle_length = len;
2816 +
2817 + /* release the overlap */
2607 2818 }
2819 + ASSERT(i == dfl->dfl_num_exts);
2608 2820
2609 -unmap_done:
2821 + /* Finally execute the unmap operations in a single step */
2822 + ret = sbd_unmap(sl, dfl);
2823 + dfl_free(dfl);
2824 + if (ret != 0) {
2825 + stmf_scsilib_send_status(task, STATUS_CHECK,
2826 + STMF_SAA_LBA_OUT_OF_RANGE);
2827 + return;
2828 + }
2829 +
2610 2830 stmf_scsilib_send_status(task, STATUS_GOOD, 0);
2611 2831 }
2612 2832
2613 2833 void
2614 2834 sbd_handle_inquiry(struct scsi_task *task, struct stmf_data_buf *initial_dbuf)
2615 2835 {
2616 2836 sbd_lu_t *sl = (sbd_lu_t *)task->task_lu->lu_provider_private;
2617 2837 uint8_t *cdbp = (uint8_t *)&task->task_cdb[0];
2618 2838 uint8_t *p;
2619 2839 uint8_t byte0;
2620 2840 uint8_t page_length;
2621 2841 uint16_t bsize = 512;
2622 2842 uint16_t cmd_size;
2623 2843 uint32_t xfer_size = 4;
2624 2844 uint32_t mgmt_url_size = 0;
2625 2845 uint8_t exp;
2626 2846 uint64_t s;
2627 2847 char *mgmt_url = NULL;
2628 2848
2629 2849
2630 2850 byte0 = DTYPE_DIRECT;
2631 2851 /*
2632 2852 * Basic protocol checks.
2633 2853 */
2634 2854
2635 2855 if ((((cdbp[1] & 1) == 0) && cdbp[2]) || cdbp[5]) {
2636 2856 stmf_scsilib_send_status(task, STATUS_CHECK,
2637 2857 STMF_SAA_INVALID_FIELD_IN_CDB);
2638 2858 return;
2639 2859 }
2640 2860
2641 2861 /*
2642 2862 * Zero byte allocation length is not an error. Just
2643 2863 * return success.
2644 2864 */
2645 2865
2646 2866 cmd_size = (((uint16_t)cdbp[3]) << 8) | cdbp[4];
2647 2867
2648 2868 if (cmd_size == 0) {
2649 2869 task->task_cmd_xfer_length = 0;
2650 2870 if (task->task_additional_flags &
2651 2871 TASK_AF_NO_EXPECTED_XFER_LENGTH) {
2652 2872 task->task_expected_xfer_length = 0;
2653 2873 }
2654 2874 stmf_scsilib_send_status(task, STATUS_GOOD, 0);
2655 2875 return;
2656 2876 }
2657 2877
2658 2878 /*
2659 2879 * Standard inquiry
2660 2880 */
2661 2881
2662 2882 if ((cdbp[1] & 1) == 0) {
2663 2883 int i;
2664 2884 struct scsi_inquiry *inq;
2665 2885
2666 2886 p = (uint8_t *)kmem_zalloc(bsize, KM_SLEEP);
2667 2887 inq = (struct scsi_inquiry *)p;
2668 2888
2669 2889 page_length = 69;
|
↓ open down ↓ |
50 lines elided |
↑ open up ↑ |
2670 2890 xfer_size = page_length + 5;
2671 2891
2672 2892 inq->inq_dtype = DTYPE_DIRECT;
2673 2893 inq->inq_ansi = 5; /* SPC-3 */
2674 2894 inq->inq_hisup = 1;
2675 2895 inq->inq_rdf = 2; /* Response data format for SPC-3 */
2676 2896 inq->inq_len = page_length;
2677 2897
2678 2898 inq->inq_tpgs = TPGS_FAILOVER_IMPLICIT;
2679 2899 inq->inq_cmdque = 1;
2900 + inq->inq_3pc = 1;
2680 2901
2681 2902 if (sl->sl_flags & SL_VID_VALID) {
2682 2903 bcopy(sl->sl_vendor_id, inq->inq_vid, 8);
2683 2904 } else {
2684 2905 bcopy(sbd_vendor_id, inq->inq_vid, 8);
2685 2906 }
2686 2907
2687 2908 if (sl->sl_flags & SL_PID_VALID) {
2688 2909 bcopy(sl->sl_product_id, inq->inq_pid, 16);
2689 2910 } else {
2690 2911 bcopy(sbd_product_id, inq->inq_pid, 16);
2691 2912 }
2692 2913
2693 2914 if (sl->sl_flags & SL_REV_VALID) {
2694 2915 bcopy(sl->sl_revision, inq->inq_revision, 4);
2695 2916 } else {
2696 2917 bcopy(sbd_revision, inq->inq_revision, 4);
2697 2918 }
2698 2919
2699 2920 /* Adding Version Descriptors */
2700 2921 i = 0;
2701 2922 /* SAM-3 no version */
2702 2923 inq->inq_vd[i].inq_vd_msb = 0x00;
2703 2924 inq->inq_vd[i].inq_vd_lsb = 0x60;
2704 2925 i++;
2705 2926
2706 2927 /* transport */
2707 2928 switch (task->task_lport->lport_id->protocol_id) {
2708 2929 case PROTOCOL_FIBRE_CHANNEL:
2709 2930 inq->inq_vd[i].inq_vd_msb = 0x09;
2710 2931 inq->inq_vd[i].inq_vd_lsb = 0x00;
2711 2932 i++;
2712 2933 break;
2713 2934
2714 2935 case PROTOCOL_PARALLEL_SCSI:
2715 2936 case PROTOCOL_SSA:
2716 2937 case PROTOCOL_IEEE_1394:
2717 2938 /* Currently no claims of conformance */
2718 2939 break;
2719 2940
2720 2941 case PROTOCOL_SRP:
2721 2942 inq->inq_vd[i].inq_vd_msb = 0x09;
2722 2943 inq->inq_vd[i].inq_vd_lsb = 0x40;
2723 2944 i++;
2724 2945 break;
2725 2946
2726 2947 case PROTOCOL_iSCSI:
2727 2948 inq->inq_vd[i].inq_vd_msb = 0x09;
2728 2949 inq->inq_vd[i].inq_vd_lsb = 0x60;
2729 2950 i++;
2730 2951 break;
2731 2952
2732 2953 case PROTOCOL_SAS:
2733 2954 case PROTOCOL_ADT:
2734 2955 case PROTOCOL_ATAPI:
2735 2956 default:
2736 2957 /* Currently no claims of conformance */
2737 2958 break;
2738 2959 }
2739 2960
2740 2961 /* SPC-3 no version */
2741 2962 inq->inq_vd[i].inq_vd_msb = 0x03;
2742 2963 inq->inq_vd[i].inq_vd_lsb = 0x00;
2743 2964 i++;
2744 2965
2745 2966 /* SBC-2 no version */
2746 2967 inq->inq_vd[i].inq_vd_msb = 0x03;
2747 2968 inq->inq_vd[i].inq_vd_lsb = 0x20;
2748 2969
2749 2970 sbd_handle_short_read_transfers(task, initial_dbuf, p, cmd_size,
2750 2971 min(cmd_size, xfer_size));
2751 2972 kmem_free(p, bsize);
2752 2973
2753 2974 return;
2754 2975 }
2755 2976
2756 2977 rw_enter(&sbd_global_prop_lock, RW_READER);
2757 2978 if (sl->sl_mgmt_url) {
2758 2979 mgmt_url_size = strlen(sl->sl_mgmt_url);
2759 2980 mgmt_url = sl->sl_mgmt_url;
2760 2981 } else if (sbd_mgmt_url) {
2761 2982 mgmt_url_size = strlen(sbd_mgmt_url);
2762 2983 mgmt_url = sbd_mgmt_url;
2763 2984 }
2764 2985
2765 2986 /*
2766 2987 * EVPD handling
2767 2988 */
|
↓ open down ↓ |
78 lines elided |
↑ open up ↑ |
2768 2989
2769 2990 /* Default 512 bytes may not be enough, increase bsize if necessary */
2770 2991 if (cdbp[2] == 0x83 || cdbp[2] == 0x85) {
2771 2992 if (bsize < cmd_size)
2772 2993 bsize = cmd_size;
2773 2994 }
2774 2995 p = (uint8_t *)kmem_zalloc(bsize, KM_SLEEP);
2775 2996
2776 2997 switch (cdbp[2]) {
2777 2998 case 0x00:
2778 - page_length = 4 + (mgmt_url_size ? 1 : 0);
2999 + page_length = 5 + (mgmt_url_size ? 1 : 0);
3000 +
2779 3001 if (sl->sl_flags & SL_UNMAP_ENABLED)
2780 - page_length += 2;
3002 + page_length += 1;
2781 3003
2782 3004 p[0] = byte0;
2783 3005 p[3] = page_length;
2784 3006 /* Supported VPD pages in ascending order */
3007 + /* CSTYLED */
2785 3008 {
2786 3009 uint8_t i = 5;
2787 3010
2788 3011 p[i++] = 0x80;
2789 3012 p[i++] = 0x83;
2790 3013 if (mgmt_url_size != 0)
2791 3014 p[i++] = 0x85;
2792 3015 p[i++] = 0x86;
3016 + p[i++] = 0xb0;
2793 3017 if (sl->sl_flags & SL_UNMAP_ENABLED) {
2794 - p[i++] = 0xb0;
2795 3018 p[i++] = 0xb2;
2796 3019 }
2797 3020 }
2798 3021 xfer_size = page_length + 4;
2799 3022 break;
2800 3023
2801 3024 case 0x80:
2802 3025 if (sl->sl_serial_no_size) {
2803 3026 page_length = sl->sl_serial_no_size;
2804 3027 bcopy(sl->sl_serial_no, p + 4, sl->sl_serial_no_size);
2805 3028 } else {
2806 3029 /* if no serial num is specified set 4 spaces */
2807 3030 page_length = 4;
2808 3031 bcopy(" ", p + 4, 4);
2809 3032 }
2810 3033 p[0] = byte0;
2811 3034 p[1] = 0x80;
2812 3035 p[3] = page_length;
2813 3036 xfer_size = page_length + 4;
2814 3037 break;
2815 3038
2816 3039 case 0x83:
|
↓ open down ↓ |
12 lines elided |
↑ open up ↑ |
2817 3040 xfer_size = stmf_scsilib_prepare_vpd_page83(task, p,
2818 3041 bsize, byte0, STMF_VPD_LU_ID|STMF_VPD_TARGET_ID|
2819 3042 STMF_VPD_TP_GROUP|STMF_VPD_RELATIVE_TP_ID);
2820 3043 break;
2821 3044
2822 3045 case 0x85:
2823 3046 if (mgmt_url_size == 0) {
2824 3047 stmf_scsilib_send_status(task, STATUS_CHECK,
2825 3048 STMF_SAA_INVALID_FIELD_IN_CDB);
2826 3049 goto err_done;
2827 - }
3050 + } /* CSTYLED */
2828 3051 {
2829 3052 uint16_t idx, newidx, sz, url_size;
2830 3053 char *url;
2831 3054
2832 3055 p[0] = byte0;
2833 3056 p[1] = 0x85;
2834 3057
2835 3058 idx = 4;
2836 3059 url = mgmt_url;
2837 3060 url_size = sbd_parse_mgmt_url(&url);
2838 3061 /* Creating Network Service Descriptors */
2839 3062 while (url_size != 0) {
2840 3063 /* Null terminated and 4 Byte aligned */
2841 3064 sz = url_size + 1;
2842 3065 sz += (sz % 4) ? 4 - (sz % 4) : 0;
2843 3066 newidx = idx + sz + 4;
2844 3067
2845 3068 if (newidx < bsize) {
2846 3069 /*
2847 3070 * SPC-3r23 : Table 320 (Sec 7.6.5)
2848 3071 * (Network service descriptor format
2849 3072 *
2850 3073 * Note: Hard coding service type as
2851 3074 * "Storage Configuration Service".
2852 3075 */
2853 3076 p[idx] = 1;
2854 3077 SCSI_WRITE16(p + idx + 2, sz);
2855 3078 bcopy(url, p + idx + 4, url_size);
2856 3079 xfer_size = newidx + 4;
2857 3080 }
2858 3081 idx = newidx;
2859 3082
2860 3083 /* skip to next mgmt url if any */
2861 3084 url += url_size;
2862 3085 url_size = sbd_parse_mgmt_url(&url);
2863 3086 }
2864 3087
2865 3088 /* Total descriptor length */
2866 3089 SCSI_WRITE16(p + 2, idx - 4);
2867 3090 break;
2868 3091 }
2869 3092
2870 3093 case 0x86:
2871 3094 page_length = 0x3c;
2872 3095
2873 3096 p[0] = byte0;
2874 3097 p[1] = 0x86; /* Page 86 response */
2875 3098 p[3] = page_length;
2876 3099
2877 3100 /*
|
↓ open down ↓ |
40 lines elided |
↑ open up ↑ |
2878 3101 * Bits 0, 1, and 2 will need to be updated
2879 3102 * to reflect the queue tag handling if/when
2880 3103 * that is implemented. For now, we're going
2881 3104 * to claim support only for Simple TA.
2882 3105 */
2883 3106 p[5] = 1;
2884 3107 xfer_size = page_length + 4;
2885 3108 break;
2886 3109
2887 3110 case 0xb0:
2888 - if ((sl->sl_flags & SL_UNMAP_ENABLED) == 0) {
2889 - stmf_scsilib_send_status(task, STATUS_CHECK,
2890 - STMF_SAA_INVALID_FIELD_IN_CDB);
2891 - goto err_done;
2892 - }
2893 3111 page_length = 0x3c;
2894 3112 p[0] = byte0;
2895 3113 p[1] = 0xb0;
2896 3114 p[3] = page_length;
2897 - p[20] = p[21] = p[22] = p[23] = 0xFF;
2898 - p[24] = p[25] = p[26] = p[27] = 0xFF;
3115 + p[4] = 1;
3116 + p[5] = sbd_ats_max_nblks();
3117 + if (sl->sl_flags & SL_UNMAP_ENABLED && sbd_unmap_enable) {
3118 + p[20] = (stmf_sbd_unmap_max_nblks >> 24) & 0xff;
3119 + p[21] = (stmf_sbd_unmap_max_nblks >> 16) & 0xff;
3120 + p[22] = (stmf_sbd_unmap_max_nblks >> 8) & 0xff;
3121 + p[23] = stmf_sbd_unmap_max_nblks & 0xff;
3122 +
3123 + p[24] = 0;
3124 + p[25] = 0;
3125 + p[26] = 0;
3126 + p[27] = 0xFF;
3127 + }
2899 3128 xfer_size = page_length + 4;
2900 3129 break;
2901 3130
2902 3131 case 0xb2:
2903 3132 if ((sl->sl_flags & SL_UNMAP_ENABLED) == 0) {
2904 3133 stmf_scsilib_send_status(task, STATUS_CHECK,
2905 3134 STMF_SAA_INVALID_FIELD_IN_CDB);
2906 3135 goto err_done;
2907 3136 }
2908 3137 page_length = 4;
2909 3138 p[0] = byte0;
|
↓ open down ↓ |
1 lines elided |
↑ open up ↑ |
2910 3139 p[1] = 0xb2;
2911 3140 p[3] = page_length;
2912 3141
2913 3142 exp = (uint8_t)sl->sl_data_blocksize_shift;
2914 3143 s = sl->sl_lu_size >> sl->sl_data_blocksize_shift;
2915 3144 while (s & ((uint64_t)0xFFFFFFFF80000000ull)) {
2916 3145 s >>= 1;
2917 3146 exp++;
2918 3147 }
2919 3148 p[4] = exp;
2920 - p[5] = 0xc0;
3149 + p[5] = 0xc0; /* Logical provisioning UNMAP and WRITE SAME */
2921 3150 xfer_size = page_length + 4;
2922 3151 break;
2923 3152
2924 3153 default:
2925 3154 stmf_scsilib_send_status(task, STATUS_CHECK,
2926 3155 STMF_SAA_INVALID_FIELD_IN_CDB);
2927 3156 goto err_done;
2928 3157 }
2929 3158
2930 3159 sbd_handle_short_read_transfers(task, initial_dbuf, p, cmd_size,
2931 3160 min(cmd_size, xfer_size));
2932 3161 err_done:
2933 3162 kmem_free(p, bsize);
2934 3163 rw_exit(&sbd_global_prop_lock);
2935 3164 }
2936 3165
2937 3166 stmf_status_t
2938 3167 sbd_task_alloc(struct scsi_task *task)
2939 3168 {
2940 3169 if ((task->task_lu_private =
2941 - kmem_alloc(sizeof (sbd_cmd_t), KM_NOSLEEP)) != NULL) {
3170 + kmem_zalloc(sizeof (sbd_cmd_t), KM_NOSLEEP)) != NULL) {
2942 3171 sbd_cmd_t *scmd = (sbd_cmd_t *)task->task_lu_private;
2943 3172 scmd->flags = 0;
2944 3173 return (STMF_SUCCESS);
2945 3174 }
2946 3175 return (STMF_ALLOC_FAILURE);
2947 3176 }
2948 3177
2949 3178 void
2950 3179 sbd_remove_it_handle(sbd_lu_t *sl, sbd_it_data_t *it)
2951 3180 {
2952 3181 sbd_it_data_t **ppit;
2953 3182
2954 3183 sbd_pgr_remove_it_handle(sl, it);
2955 3184 mutex_enter(&sl->sl_lock);
2956 3185 for (ppit = &sl->sl_it_list; *ppit != NULL;
2957 3186 ppit = &((*ppit)->sbd_it_next)) {
2958 3187 if ((*ppit) == it) {
2959 3188 *ppit = it->sbd_it_next;
2960 3189 break;
2961 3190 }
2962 3191 }
2963 3192 mutex_exit(&sl->sl_lock);
2964 3193
2965 3194 DTRACE_PROBE2(itl__nexus__end, stmf_lu_t *, sl->sl_lu,
2966 3195 sbd_it_data_t *, it);
2967 3196
2968 3197 kmem_free(it, sizeof (*it));
2969 3198 }
2970 3199
2971 3200 void
2972 3201 sbd_check_and_clear_scsi2_reservation(sbd_lu_t *sl, sbd_it_data_t *it)
2973 3202 {
2974 3203 mutex_enter(&sl->sl_lock);
2975 3204 if ((sl->sl_flags & SL_LU_HAS_SCSI2_RESERVATION) == 0) {
2976 3205 /* If we dont have any reservations, just get out. */
2977 3206 mutex_exit(&sl->sl_lock);
2978 3207 return;
2979 3208 }
2980 3209
2981 3210 if (it == NULL) {
2982 3211 /* Find the I_T nexus which is holding the reservation. */
2983 3212 for (it = sl->sl_it_list; it != NULL; it = it->sbd_it_next) {
2984 3213 if (it->sbd_it_flags & SBD_IT_HAS_SCSI2_RESERVATION) {
2985 3214 ASSERT(it->sbd_it_session_id ==
2986 3215 sl->sl_rs_owner_session_id);
2987 3216 break;
2988 3217 }
2989 3218 }
2990 3219 ASSERT(it != NULL);
2991 3220 } else {
2992 3221 /*
2993 3222 * We were passed an I_T nexus. If this nexus does not hold
2994 3223 * the reservation, do nothing. This is why this function is
2995 3224 * called "check_and_clear".
2996 3225 */
|
↓ open down ↓ |
45 lines elided |
↑ open up ↑ |
2997 3226 if ((it->sbd_it_flags & SBD_IT_HAS_SCSI2_RESERVATION) == 0) {
2998 3227 mutex_exit(&sl->sl_lock);
2999 3228 return;
3000 3229 }
3001 3230 }
3002 3231 it->sbd_it_flags &= ~SBD_IT_HAS_SCSI2_RESERVATION;
3003 3232 sl->sl_flags &= ~SL_LU_HAS_SCSI2_RESERVATION;
3004 3233 mutex_exit(&sl->sl_lock);
3005 3234 }
3006 3235
3236 +/*
3237 + * Given a LU and a task, check if the task is causing reservation
3238 + * conflict. Returns 1 in case of conflict, 0 otherwise.
3239 + * Note that the LU might not be the same LU as in the task but the
3240 + * caller makes sure that the LU can be accessed.
3241 + */
3242 +int
3243 +sbd_check_reservation_conflict(struct sbd_lu *sl, struct scsi_task *task)
3244 +{
3245 + sbd_it_data_t *it;
3007 3246
3247 + it = task->task_lu_itl_handle;
3248 + ASSERT(it);
3249 + if (sl->sl_access_state == SBD_LU_ACTIVE) {
3250 + if (SBD_PGR_RSVD(sl->sl_pgr)) {
3251 + if (sbd_pgr_reservation_conflict(task, sl)) {
3252 + return (1);
3253 + }
3254 + } else if ((sl->sl_flags & SL_LU_HAS_SCSI2_RESERVATION) &&
3255 + ((it->sbd_it_flags & SBD_IT_HAS_SCSI2_RESERVATION) == 0)) {
3256 + if (!(SCSI2_CONFLICT_FREE_CMDS(task->task_cdb))) {
3257 + return (1);
3258 + }
3259 + }
3260 + }
3008 3261
3262 + return (0);
3263 +}
3264 +
3265 +/*
3266 + * Keep in mind that sbd_new_task can be called multiple times for the same
3267 + * task because of us calling stmf_task_poll_lu resulting in a call to
3268 + * sbd_task_poll().
3269 + */
3009 3270 void
3010 3271 sbd_new_task(struct scsi_task *task, struct stmf_data_buf *initial_dbuf)
3011 3272 {
3012 3273 sbd_lu_t *sl = (sbd_lu_t *)task->task_lu->lu_provider_private;
3013 3274 sbd_it_data_t *it;
3014 3275 uint8_t cdb0, cdb1;
3015 3276 stmf_status_t st_ret;
3016 3277
3017 3278 if ((it = task->task_lu_itl_handle) == NULL) {
3018 3279 mutex_enter(&sl->sl_lock);
3019 3280 for (it = sl->sl_it_list; it != NULL; it = it->sbd_it_next) {
3020 3281 if (it->sbd_it_session_id ==
3021 3282 task->task_session->ss_session_id) {
3022 3283 mutex_exit(&sl->sl_lock);
3023 3284 stmf_scsilib_send_status(task, STATUS_BUSY, 0);
3024 3285 return;
3025 3286 }
3026 3287 }
3027 3288 it = (sbd_it_data_t *)kmem_zalloc(sizeof (*it), KM_NOSLEEP);
3028 3289 if (it == NULL) {
3029 3290 mutex_exit(&sl->sl_lock);
3030 3291 stmf_scsilib_send_status(task, STATUS_BUSY, 0);
3031 3292 return;
3032 3293 }
3033 3294 it->sbd_it_session_id = task->task_session->ss_session_id;
3034 3295 bcopy(task->task_lun_no, it->sbd_it_lun, 8);
3035 3296 it->sbd_it_next = sl->sl_it_list;
3036 3297 sl->sl_it_list = it;
3037 3298 mutex_exit(&sl->sl_lock);
3038 3299
3039 3300 DTRACE_PROBE1(itl__nexus__start, scsi_task *, task);
3040 3301
3041 3302 sbd_pgr_initialize_it(task, it);
3042 3303 if (stmf_register_itl_handle(task->task_lu, task->task_lun_no,
3043 3304 task->task_session, it->sbd_it_session_id, it)
3044 3305 != STMF_SUCCESS) {
3045 3306 sbd_remove_it_handle(sl, it);
3046 3307 stmf_scsilib_send_status(task, STATUS_BUSY, 0);
3047 3308 return;
3048 3309 }
3049 3310 task->task_lu_itl_handle = it;
3050 3311 if (sl->sl_access_state != SBD_LU_STANDBY) {
3051 3312 it->sbd_it_ua_conditions = SBD_UA_POR;
3052 3313 }
3053 3314 } else if (it->sbd_it_flags & SBD_IT_PGR_CHECK_FLAG) {
3054 3315 mutex_enter(&sl->sl_lock);
3055 3316 it->sbd_it_flags &= ~SBD_IT_PGR_CHECK_FLAG;
3056 3317 mutex_exit(&sl->sl_lock);
3057 3318 sbd_pgr_initialize_it(task, it);
3058 3319 }
3059 3320
3060 3321 if (task->task_mgmt_function) {
3061 3322 stmf_scsilib_handle_task_mgmt(task);
|
↓ open down ↓ |
43 lines elided |
↑ open up ↑ |
3062 3323 return;
3063 3324 }
3064 3325
3065 3326 /*
3066 3327 * if we're transitioning between access
3067 3328 * states, return NOT READY
3068 3329 */
3069 3330 if (sl->sl_access_state == SBD_LU_TRANSITION_TO_STANDBY ||
3070 3331 sl->sl_access_state == SBD_LU_TRANSITION_TO_ACTIVE) {
3071 3332 stmf_scsilib_send_status(task, STATUS_CHECK,
3072 - STMF_SAA_LU_NO_ACCESS_UNAVAIL);
3333 + STMF_SAA_LU_NO_ACCESS_TRANSITION);
3073 3334 return;
3074 3335 }
3075 3336
3076 - /* Checking ua conditions as per SAM3R14 5.3.2 specified order */
3077 - if ((it->sbd_it_ua_conditions) && (task->task_cdb[0] != SCMD_INQUIRY)) {
3337 + cdb0 = task->task_cdb[0];
3338 + cdb1 = task->task_cdb[1];
3339 + /*
3340 + * Special case for different versions of Windows.
3341 + * 1) Windows 2012 and VMWare will fail to discover LU's if a READ
3342 + * operation sent down the standby path returns an error. By default
3343 + * standby_fail_reads will be set to 0.
3344 + * 2) Windows 2008 R2 has a severe performace problem if READ ops
3345 + * aren't rejected on the standby path. 2008 sends commands
3346 + * down the standby path which then must be proxied over to the
3347 + * active node and back.
3348 + */
3349 + if ((sl->sl_access_state == SBD_LU_STANDBY) &&
3350 + stmf_standby_fail_reads &&
3351 + (cdb0 == SCMD_READ || cdb0 == SCMD_READ_G1 ||
3352 + cdb0 == SCMD_READ_G4 || cdb0 == SCMD_READ_G5)) {
3353 + stmf_scsilib_send_status(task, STATUS_CHECK,
3354 + STMF_SAA_LU_NO_ACCESS_STANDBY);
3355 + return;
3356 + }
3357 +
3358 + /*
3359 + * Don't go further if cmd is unsupported in standby mode
3360 + */
3361 + if (sl->sl_access_state == SBD_LU_STANDBY) {
3362 + if (cdb0 != SCMD_INQUIRY &&
3363 + cdb0 != SCMD_MODE_SENSE &&
3364 + cdb0 != SCMD_MODE_SENSE_G1 &&
3365 + cdb0 != SCMD_MODE_SELECT &&
3366 + cdb0 != SCMD_MODE_SELECT_G1 &&
3367 + cdb0 != SCMD_RESERVE &&
3368 + cdb0 != SCMD_RELEASE &&
3369 + cdb0 != SCMD_PERSISTENT_RESERVE_OUT &&
3370 + cdb0 != SCMD_PERSISTENT_RESERVE_IN &&
3371 + cdb0 != SCMD_REQUEST_SENSE &&
3372 + cdb0 != SCMD_READ_CAPACITY &&
3373 + cdb0 != SCMD_TEST_UNIT_READY &&
3374 + cdb0 != SCMD_START_STOP &&
3375 + cdb0 != SCMD_READ &&
3376 + cdb0 != SCMD_READ_G1 &&
3377 + cdb0 != SCMD_READ_G4 &&
3378 + cdb0 != SCMD_READ_G5 &&
3379 + !(cdb0 == SCMD_SVC_ACTION_IN_G4 &&
3380 + cdb1 == SSVC_ACTION_READ_CAPACITY_G4) &&
3381 + !(cdb0 == SCMD_MAINTENANCE_IN &&
3382 + (cdb1 & 0x1F) == 0x05) &&
3383 + !(cdb0 == SCMD_MAINTENANCE_IN &&
3384 + (cdb1 & 0x1F) == 0x0A)) {
3385 + stmf_scsilib_send_status(task, STATUS_CHECK,
3386 + STMF_SAA_LU_NO_ACCESS_STANDBY);
3387 + return;
3388 + }
3389 + }
3390 +
3391 + /*
3392 + * Checking ua conditions as per SAM3R14 5.3.2 specified order. During
3393 + * MPIO/ALUA failover, cmds come in through local ports and proxy port
3394 + * port provider (i.e. pppt), we want to report unit attention to
3395 + * only local cmds since initiators (Windows MPIO/DSM) would continue
3396 + * sending I/O to the target that reported unit attention.
3397 + */
3398 + if ((it->sbd_it_ua_conditions) &&
3399 + !(task->task_additional_flags & TASK_AF_PPPT_TASK) &&
3400 + (task->task_cdb[0] != SCMD_INQUIRY)) {
3078 3401 uint32_t saa = 0;
3079 3402
3080 3403 mutex_enter(&sl->sl_lock);
3081 3404 if (it->sbd_it_ua_conditions & SBD_UA_POR) {
3082 3405 it->sbd_it_ua_conditions &= ~SBD_UA_POR;
3083 3406 saa = STMF_SAA_POR;
3407 + } else if (it->sbd_it_ua_conditions &
3408 + SBD_UA_ASYMMETRIC_ACCESS_CHANGED) {
3409 + it->sbd_it_ua_conditions &=
3410 + ~SBD_UA_ASYMMETRIC_ACCESS_CHANGED;
3411 + saa = STMF_SAA_ASYMMETRIC_ACCESS_CHANGED;
3084 3412 }
3085 3413 mutex_exit(&sl->sl_lock);
3086 3414 if (saa) {
3087 3415 stmf_scsilib_send_status(task, STATUS_CHECK, saa);
3088 3416 return;
3089 3417 }
3090 3418 }
3091 3419
3092 3420 /* Reservation conflict checks */
3093 - if (sl->sl_access_state == SBD_LU_ACTIVE) {
3094 - if (SBD_PGR_RSVD(sl->sl_pgr)) {
3095 - if (sbd_pgr_reservation_conflict(task)) {
3096 - stmf_scsilib_send_status(task,
3097 - STATUS_RESERVATION_CONFLICT, 0);
3098 - return;
3099 - }
3100 - } else if ((sl->sl_flags & SL_LU_HAS_SCSI2_RESERVATION) &&
3101 - ((it->sbd_it_flags & SBD_IT_HAS_SCSI2_RESERVATION) == 0)) {
3102 - if (!(SCSI2_CONFLICT_FREE_CMDS(task->task_cdb))) {
3103 - stmf_scsilib_send_status(task,
3104 - STATUS_RESERVATION_CONFLICT, 0);
3105 - return;
3106 - }
3107 - }
3421 + if (sbd_check_reservation_conflict(sl, task)) {
3422 + stmf_scsilib_send_status(task,
3423 + STATUS_RESERVATION_CONFLICT, 0);
3424 + return;
3108 3425 }
3109 3426
3110 3427 /* Rest of the ua conndition checks */
3111 3428 if ((it->sbd_it_ua_conditions) && (task->task_cdb[0] != SCMD_INQUIRY)) {
3112 3429 uint32_t saa = 0;
3113 3430
3114 3431 mutex_enter(&sl->sl_lock);
3115 3432 if (it->sbd_it_ua_conditions & SBD_UA_CAPACITY_CHANGED) {
3116 3433 it->sbd_it_ua_conditions &= ~SBD_UA_CAPACITY_CHANGED;
3117 3434 if ((task->task_cdb[0] == SCMD_READ_CAPACITY) ||
3118 3435 ((task->task_cdb[0] == SCMD_SVC_ACTION_IN_G4) &&
3119 3436 (task->task_cdb[1] ==
3120 3437 SSVC_ACTION_READ_CAPACITY_G4))) {
3121 3438 saa = 0;
|
↓ open down ↓ |
4 lines elided |
↑ open up ↑ |
3122 3439 } else {
3123 3440 saa = STMF_SAA_CAPACITY_DATA_HAS_CHANGED;
3124 3441 }
3125 3442 } else if (it->sbd_it_ua_conditions &
3126 3443 SBD_UA_MODE_PARAMETERS_CHANGED) {
3127 3444 it->sbd_it_ua_conditions &=
3128 3445 ~SBD_UA_MODE_PARAMETERS_CHANGED;
3129 3446 saa = STMF_SAA_MODE_PARAMETERS_CHANGED;
3130 3447 } else if (it->sbd_it_ua_conditions &
3131 3448 SBD_UA_ASYMMETRIC_ACCESS_CHANGED) {
3132 - it->sbd_it_ua_conditions &=
3133 - ~SBD_UA_ASYMMETRIC_ACCESS_CHANGED;
3134 - saa = STMF_SAA_ASYMMETRIC_ACCESS_CHANGED;
3449 + saa = 0;
3450 + } else if (it->sbd_it_ua_conditions & SBD_UA_POR) {
3451 + saa = 0;
3135 3452 } else if (it->sbd_it_ua_conditions &
3136 3453 SBD_UA_ACCESS_STATE_TRANSITION) {
3137 3454 it->sbd_it_ua_conditions &=
3138 3455 ~SBD_UA_ACCESS_STATE_TRANSITION;
3139 3456 saa = STMF_SAA_LU_NO_ACCESS_TRANSITION;
3140 3457 } else {
3141 3458 it->sbd_it_ua_conditions = 0;
3142 3459 saa = 0;
3143 3460 }
3144 3461 mutex_exit(&sl->sl_lock);
3145 3462 if (saa) {
3146 3463 stmf_scsilib_send_status(task, STATUS_CHECK, saa);
3147 3464 return;
3148 3465 }
3149 3466 }
3150 3467
3151 - cdb0 = task->task_cdb[0];
3152 - cdb1 = task->task_cdb[1];
3153 -
3154 3468 if (sl->sl_access_state == SBD_LU_STANDBY) {
3155 - if (cdb0 != SCMD_INQUIRY &&
3156 - cdb0 != SCMD_MODE_SENSE &&
3157 - cdb0 != SCMD_MODE_SENSE_G1 &&
3158 - cdb0 != SCMD_MODE_SELECT &&
3159 - cdb0 != SCMD_MODE_SELECT_G1 &&
3160 - cdb0 != SCMD_RESERVE &&
3161 - cdb0 != SCMD_RELEASE &&
3162 - cdb0 != SCMD_PERSISTENT_RESERVE_OUT &&
3163 - cdb0 != SCMD_PERSISTENT_RESERVE_IN &&
3164 - cdb0 != SCMD_REQUEST_SENSE &&
3165 - cdb0 != SCMD_READ_CAPACITY &&
3166 - cdb0 != SCMD_TEST_UNIT_READY &&
3167 - cdb0 != SCMD_START_STOP &&
3168 - cdb0 != SCMD_READ &&
3169 - cdb0 != SCMD_READ_G1 &&
3170 - cdb0 != SCMD_READ_G4 &&
3171 - cdb0 != SCMD_READ_G5 &&
3172 - !(cdb0 == SCMD_SVC_ACTION_IN_G4 &&
3173 - cdb1 == SSVC_ACTION_READ_CAPACITY_G4) &&
3174 - !(cdb0 == SCMD_MAINTENANCE_IN &&
3175 - (cdb1 & 0x1F) == 0x05) &&
3176 - !(cdb0 == SCMD_MAINTENANCE_IN &&
3177 - (cdb1 & 0x1F) == 0x0A)) {
3178 - stmf_scsilib_send_status(task, STATUS_CHECK,
3179 - STMF_SAA_LU_NO_ACCESS_STANDBY);
3180 - return;
3181 - }
3182 -
3183 3469 /*
3184 3470 * is this a short write?
3185 3471 * if so, we'll need to wait until we have the buffer
3186 3472 * before proxying the command
3187 3473 */
3188 3474 switch (cdb0) {
3189 3475 case SCMD_MODE_SELECT:
3190 3476 case SCMD_MODE_SELECT_G1:
3191 3477 case SCMD_PERSISTENT_RESERVE_OUT:
3192 3478 break;
3193 3479 default:
3194 3480 st_ret = stmf_proxy_scsi_cmd(task,
3195 3481 initial_dbuf);
3196 3482 if (st_ret != STMF_SUCCESS) {
3197 3483 stmf_scsilib_send_status(task,
3198 3484 STATUS_CHECK,
3199 3485 STMF_SAA_LU_NO_ACCESS_UNAVAIL);
3200 3486 }
3201 3487 return;
3202 3488 }
3203 3489 }
3204 3490
3205 3491 cdb0 = task->task_cdb[0] & 0x1F;
3206 3492
3207 3493 if ((cdb0 == SCMD_READ) || (cdb0 == SCMD_WRITE)) {
3208 3494 if (task->task_additional_flags & TASK_AF_PORT_LOAD_HIGH) {
3209 3495 stmf_scsilib_send_status(task, STATUS_QFULL, 0);
3210 3496 return;
3211 3497 }
3212 3498 if (cdb0 == SCMD_READ) {
3213 3499 sbd_handle_read(task, initial_dbuf);
3214 3500 return;
3215 3501 }
3216 3502 sbd_handle_write(task, initial_dbuf);
3217 3503 return;
3218 3504 }
3219 3505
3220 3506 cdb0 = task->task_cdb[0];
3221 3507 cdb1 = task->task_cdb[1];
3222 3508
3223 3509 if (cdb0 == SCMD_INQUIRY) { /* Inquiry */
3224 3510 sbd_handle_inquiry(task, initial_dbuf);
3225 3511 return;
3226 3512 }
3227 3513
3228 3514 if (cdb0 == SCMD_PERSISTENT_RESERVE_OUT) {
3229 3515 sbd_handle_pgr_out_cmd(task, initial_dbuf);
3230 3516 return;
3231 3517 }
3232 3518
3233 3519 if (cdb0 == SCMD_PERSISTENT_RESERVE_IN) {
3234 3520 sbd_handle_pgr_in_cmd(task, initial_dbuf);
3235 3521 return;
3236 3522 }
3237 3523
3238 3524 if (cdb0 == SCMD_RELEASE) {
3239 3525 if (cdb1) {
3240 3526 stmf_scsilib_send_status(task, STATUS_CHECK,
3241 3527 STMF_SAA_INVALID_FIELD_IN_CDB);
3242 3528 return;
3243 3529 }
3244 3530
3245 3531 mutex_enter(&sl->sl_lock);
3246 3532 if (sl->sl_flags & SL_LU_HAS_SCSI2_RESERVATION) {
3247 3533 /* If not owner don't release it, just return good */
3248 3534 if (it->sbd_it_session_id !=
3249 3535 sl->sl_rs_owner_session_id) {
3250 3536 mutex_exit(&sl->sl_lock);
3251 3537 stmf_scsilib_send_status(task, STATUS_GOOD, 0);
3252 3538 return;
3253 3539 }
3254 3540 }
3255 3541 sl->sl_flags &= ~SL_LU_HAS_SCSI2_RESERVATION;
3256 3542 it->sbd_it_flags &= ~SBD_IT_HAS_SCSI2_RESERVATION;
3257 3543 mutex_exit(&sl->sl_lock);
3258 3544 stmf_scsilib_send_status(task, STATUS_GOOD, 0);
3259 3545 return;
3260 3546 }
3261 3547
3262 3548 if (cdb0 == SCMD_RESERVE) {
3263 3549 if (cdb1) {
3264 3550 stmf_scsilib_send_status(task, STATUS_CHECK,
3265 3551 STMF_SAA_INVALID_FIELD_IN_CDB);
3266 3552 return;
3267 3553 }
3268 3554
3269 3555 mutex_enter(&sl->sl_lock);
3270 3556 if (sl->sl_flags & SL_LU_HAS_SCSI2_RESERVATION) {
3271 3557 /* If not owner, return conflict status */
3272 3558 if (it->sbd_it_session_id !=
3273 3559 sl->sl_rs_owner_session_id) {
3274 3560 mutex_exit(&sl->sl_lock);
3275 3561 stmf_scsilib_send_status(task,
3276 3562 STATUS_RESERVATION_CONFLICT, 0);
3277 3563 return;
3278 3564 }
3279 3565 }
3280 3566 sl->sl_flags |= SL_LU_HAS_SCSI2_RESERVATION;
3281 3567 it->sbd_it_flags |= SBD_IT_HAS_SCSI2_RESERVATION;
3282 3568 sl->sl_rs_owner_session_id = it->sbd_it_session_id;
3283 3569 mutex_exit(&sl->sl_lock);
3284 3570 stmf_scsilib_send_status(task, STATUS_GOOD, 0);
3285 3571 return;
3286 3572 }
3287 3573
3288 3574 if (cdb0 == SCMD_REQUEST_SENSE) {
3289 3575 /*
3290 3576 * LU provider needs to store unretrieved sense data
3291 3577 * (e.g. after power-on/reset). For now, we'll just
3292 3578 * return good status with no sense.
3293 3579 */
3294 3580
3295 3581 if ((cdb1 & ~1) || task->task_cdb[2] || task->task_cdb[3] ||
3296 3582 task->task_cdb[5]) {
3297 3583 stmf_scsilib_send_status(task, STATUS_CHECK,
3298 3584 STMF_SAA_INVALID_FIELD_IN_CDB);
3299 3585 } else {
3300 3586 stmf_scsilib_send_status(task, STATUS_GOOD, 0);
3301 3587 }
3302 3588
3303 3589 return;
3304 3590 }
3305 3591
3306 3592 /* Report Target Port Groups */
3307 3593 if ((cdb0 == SCMD_MAINTENANCE_IN) &&
3308 3594 ((cdb1 & 0x1F) == 0x0A)) {
3309 3595 stmf_scsilib_handle_report_tpgs(task, initial_dbuf);
3310 3596 return;
3311 3597 }
3312 3598
3313 3599 /* Report Identifying Information */
3314 3600 if ((cdb0 == SCMD_MAINTENANCE_IN) &&
3315 3601 ((cdb1 & 0x1F) == 0x05)) {
3316 3602 sbd_handle_identifying_info(task, initial_dbuf);
3317 3603 return;
3318 3604 }
3319 3605
3320 3606 if (cdb0 == SCMD_START_STOP) { /* Start stop */
3321 3607 task->task_cmd_xfer_length = 0;
3322 3608 if (task->task_cdb[4] & 0xFC) {
3323 3609 stmf_scsilib_send_status(task, STATUS_CHECK,
3324 3610 STMF_SAA_INVALID_FIELD_IN_CDB);
3325 3611 return;
3326 3612 }
3327 3613 if (task->task_cdb[4] & 2) {
3328 3614 stmf_scsilib_send_status(task, STATUS_CHECK,
3329 3615 STMF_SAA_INVALID_FIELD_IN_CDB);
3330 3616 } else {
3331 3617 stmf_scsilib_send_status(task, STATUS_GOOD, 0);
3332 3618 }
3333 3619 return;
3334 3620
3335 3621 }
3336 3622
3337 3623 if ((cdb0 == SCMD_MODE_SENSE) || (cdb0 == SCMD_MODE_SENSE_G1)) {
3338 3624 uint8_t *p;
3339 3625 p = kmem_zalloc(512, KM_SLEEP);
3340 3626 sbd_handle_mode_sense(task, initial_dbuf, p);
3341 3627 kmem_free(p, 512);
3342 3628 return;
3343 3629 }
3344 3630
3345 3631 if ((cdb0 == SCMD_MODE_SELECT) || (cdb0 == SCMD_MODE_SELECT_G1)) {
3346 3632 sbd_handle_mode_select(task, initial_dbuf);
3347 3633 return;
3348 3634 }
3349 3635
|
↓ open down ↓ |
157 lines elided |
↑ open up ↑ |
3350 3636 if ((cdb0 == SCMD_UNMAP) && (sl->sl_flags & SL_UNMAP_ENABLED)) {
3351 3637 sbd_handle_unmap(task, initial_dbuf);
3352 3638 return;
3353 3639 }
3354 3640
3355 3641 if ((cdb0 == SCMD_WRITE_SAME_G4) || (cdb0 == SCMD_WRITE_SAME_G1)) {
3356 3642 sbd_handle_write_same(task, initial_dbuf);
3357 3643 return;
3358 3644 }
3359 3645
3646 + if (cdb0 == SCMD_COMPARE_AND_WRITE) {
3647 + sbd_handle_ats(task, initial_dbuf);
3648 + return;
3649 + }
3650 +
3651 + if (cdb0 == SCMD_EXTENDED_COPY) {
3652 + sbd_handle_xcopy(task, initial_dbuf);
3653 + return;
3654 + }
3655 +
3656 + if (cdb0 == SCMD_RECV_COPY_RESULTS) {
3657 + sbd_handle_recv_copy_results(task, initial_dbuf);
3658 + return;
3659 + }
3660 +
3360 3661 if (cdb0 == SCMD_TEST_UNIT_READY) { /* Test unit ready */
3361 3662 task->task_cmd_xfer_length = 0;
3362 3663 stmf_scsilib_send_status(task, STATUS_GOOD, 0);
3363 3664 return;
3364 3665 }
3365 3666
3366 3667 if (cdb0 == SCMD_READ_CAPACITY) { /* Read Capacity */
3367 3668 sbd_handle_read_capacity(task, initial_dbuf);
3368 3669 return;
3369 3670 }
3370 3671
3371 3672 if (cdb0 == SCMD_SVC_ACTION_IN_G4) { /* Read Capacity or read long */
3372 3673 if (cdb1 == SSVC_ACTION_READ_CAPACITY_G4) {
3373 3674 sbd_handle_read_capacity(task, initial_dbuf);
3374 3675 return;
3375 3676 /*
3376 3677 * } else if (cdb1 == SSVC_ACTION_READ_LONG_G4) {
3377 3678 * sbd_handle_read(task, initial_dbuf);
3378 3679 * return;
3379 3680 */
3380 3681 }
3381 3682 }
3382 3683
3383 3684 /*
3384 3685 * if (cdb0 == SCMD_SVC_ACTION_OUT_G4) {
3385 3686 * if (cdb1 == SSVC_ACTION_WRITE_LONG_G4) {
3386 3687 * sbd_handle_write(task, initial_dbuf);
3387 3688 * return;
3388 3689 * }
3389 3690 * }
3390 3691 */
3391 3692
3392 3693 if (cdb0 == SCMD_VERIFY) {
3393 3694 /*
3394 3695 * Something more likely needs to be done here.
3395 3696 */
3396 3697 task->task_cmd_xfer_length = 0;
3397 3698 stmf_scsilib_send_status(task, STATUS_GOOD, 0);
3398 3699 return;
3399 3700 }
3400 3701
3401 3702 if (cdb0 == SCMD_SYNCHRONIZE_CACHE ||
3402 3703 cdb0 == SCMD_SYNCHRONIZE_CACHE_G4) {
3403 3704 sbd_handle_sync_cache(task, initial_dbuf);
3404 3705 return;
3405 3706 }
3406 3707
3407 3708 /*
3408 3709 * Write and Verify use the same path as write, but don't clutter the
3409 3710 * performance path above with checking for write_verify opcodes. We
3410 3711 * rely on zfs's integrity checks for the "Verify" part of Write &
3411 3712 * Verify. (Even if we did a read to "verify" we'd merely be reading
3412 3713 * cache, not actual media.)
3413 3714 * Therefore we
3414 3715 * a) only support this if sbd_is_zvol, and
3415 3716 * b) run the IO through the normal write path with a forced
3416 3717 * sbd_flush_data_cache at the end.
3417 3718 */
3418 3719
3419 3720 if ((sl->sl_flags & SL_ZFS_META) && (
3420 3721 cdb0 == SCMD_WRITE_VERIFY ||
3421 3722 cdb0 == SCMD_WRITE_VERIFY_G4 ||
3422 3723 cdb0 == SCMD_WRITE_VERIFY_G5)) {
3423 3724 sbd_handle_write(task, initial_dbuf);
3424 3725 return;
3425 3726 }
3426 3727
3427 3728 stmf_scsilib_send_status(task, STATUS_CHECK, STMF_SAA_INVALID_OPCODE);
3428 3729 }
3429 3730
3430 3731 void
3431 3732 sbd_dbuf_xfer_done(struct scsi_task *task, struct stmf_data_buf *dbuf)
3432 3733 {
3433 3734 sbd_cmd_t *scmd = (sbd_cmd_t *)task->task_lu_private;
3434 3735
3435 3736 if (dbuf->db_flags & DB_LU_DATA_BUF) {
3436 3737 /*
3437 3738 * Buffers passed in from the LU always complete
3438 3739 * even if the task is no longer active.
3439 3740 */
3440 3741 ASSERT(task->task_additional_flags & TASK_AF_ACCEPT_LU_DBUF);
3441 3742 ASSERT(scmd);
3442 3743 switch (scmd->cmd_type) {
3443 3744 case (SBD_CMD_SCSI_READ):
3444 3745 sbd_handle_sgl_read_xfer_completion(task, scmd, dbuf);
3445 3746 break;
3446 3747 case (SBD_CMD_SCSI_WRITE):
3447 3748 sbd_handle_sgl_write_xfer_completion(task, scmd, dbuf);
3448 3749 break;
3449 3750 default:
3450 3751 cmn_err(CE_PANIC, "Unknown cmd type, task = %p",
3451 3752 (void *)task);
3452 3753 break;
3453 3754 }
3454 3755 return;
3455 3756 }
|
↓ open down ↓ |
86 lines elided |
↑ open up ↑ |
3456 3757
3457 3758 if ((scmd == NULL) || ((scmd->flags & SBD_SCSI_CMD_ACTIVE) == 0))
3458 3759 return;
3459 3760
3460 3761 switch (scmd->cmd_type) {
3461 3762 case (SBD_CMD_SCSI_READ):
3462 3763 sbd_handle_read_xfer_completion(task, scmd, dbuf);
3463 3764 break;
3464 3765
3465 3766 case (SBD_CMD_SCSI_WRITE):
3466 - if ((task->task_cdb[0] == SCMD_WRITE_SAME_G1) ||
3467 - (task->task_cdb[0] == SCMD_WRITE_SAME_G4)) {
3767 + switch (task->task_cdb[0]) {
3768 + case SCMD_WRITE_SAME_G1:
3769 + case SCMD_WRITE_SAME_G4:
3468 3770 sbd_handle_write_same_xfer_completion(task, scmd, dbuf,
3469 3771 1);
3470 - } else {
3772 + break;
3773 + case SCMD_COMPARE_AND_WRITE:
3774 + sbd_handle_ats_xfer_completion(task, scmd, dbuf, 1);
3775 + break;
3776 + default:
3471 3777 sbd_handle_write_xfer_completion(task, scmd, dbuf, 1);
3778 + /* FALLTHRU */
3472 3779 }
3473 3780 break;
3474 3781
3475 3782 case (SBD_CMD_SMALL_READ):
3476 3783 sbd_handle_short_read_xfer_completion(task, scmd, dbuf);
3477 3784 break;
3478 3785
3479 3786 case (SBD_CMD_SMALL_WRITE):
3480 3787 sbd_handle_short_write_xfer_completion(task, dbuf);
3481 3788 break;
3482 3789
3483 3790 default:
3484 3791 cmn_err(CE_PANIC, "Unknown cmd type, task = %p", (void *)task);
3485 3792 break;
3486 3793 }
3487 3794 }
3488 3795
3489 3796 /* ARGSUSED */
3490 3797 void
3491 3798 sbd_send_status_done(struct scsi_task *task)
3492 3799 {
3493 3800 cmn_err(CE_PANIC,
3494 3801 "sbd_send_status_done: this should not have been called");
3495 3802 }
3496 3803
3497 3804 void
3498 3805 sbd_task_free(struct scsi_task *task)
3499 3806 {
3500 3807 if (task->task_lu_private) {
3501 3808 sbd_cmd_t *scmd = (sbd_cmd_t *)task->task_lu_private;
3502 3809 if (scmd->flags & SBD_SCSI_CMD_ACTIVE) {
3503 3810 cmn_err(CE_PANIC, "cmd is active, task = %p",
3504 3811 (void *)task);
3505 3812 }
3506 3813 kmem_free(scmd, sizeof (sbd_cmd_t));
3507 3814 }
3508 3815 }
3509 3816
3510 3817 /*
3511 3818 * Aborts are synchronus w.r.t. I/O AND
3512 3819 * All the I/O which SBD does is synchronous AND
3513 3820 * Everything within a task is single threaded.
3514 3821 * IT MEANS
3515 3822 * If this function is called, we are doing nothing with this task
3516 3823 * inside of sbd module.
3517 3824 */
3518 3825 /* ARGSUSED */
3519 3826 stmf_status_t
3520 3827 sbd_abort(struct stmf_lu *lu, int abort_cmd, void *arg, uint32_t flags)
3521 3828 {
3522 3829 sbd_lu_t *sl = (sbd_lu_t *)lu->lu_provider_private;
3523 3830 scsi_task_t *task;
3524 3831
3525 3832 if (abort_cmd == STMF_LU_RESET_STATE) {
3526 3833 return (sbd_lu_reset_state(lu));
|
↓ open down ↓ |
45 lines elided |
↑ open up ↑ |
3527 3834 }
3528 3835
3529 3836 if (abort_cmd == STMF_LU_ITL_HANDLE_REMOVED) {
3530 3837 sbd_check_and_clear_scsi2_reservation(sl, (sbd_it_data_t *)arg);
3531 3838 sbd_remove_it_handle(sl, (sbd_it_data_t *)arg);
3532 3839 return (STMF_SUCCESS);
3533 3840 }
3534 3841
3535 3842 ASSERT(abort_cmd == STMF_LU_ABORT_TASK);
3536 3843 task = (scsi_task_t *)arg;
3844 + sbd_ats_remove_by_task(task);
3537 3845 if (task->task_lu_private) {
3538 3846 sbd_cmd_t *scmd = (sbd_cmd_t *)task->task_lu_private;
3539 3847
3540 3848 if (scmd->flags & SBD_SCSI_CMD_ACTIVE) {
3541 3849 if (scmd->flags & SBD_SCSI_CMD_TRANS_DATA) {
3542 3850 kmem_free(scmd->trans_data,
3543 3851 scmd->trans_data_len);
3544 3852 scmd->flags &= ~SBD_SCSI_CMD_TRANS_DATA;
3545 3853 }
3546 3854 scmd->flags &= ~SBD_SCSI_CMD_ACTIVE;
3547 3855 return (STMF_ABORT_SUCCESS);
3548 3856 }
3549 3857 }
3550 3858
3551 3859 return (STMF_NOT_FOUND);
3552 3860 }
3553 3861
3862 +void
3863 +sbd_task_poll(struct scsi_task *task)
3864 +{
3865 + stmf_data_buf_t *initial_dbuf;
3866 +
3867 + initial_dbuf = stmf_handle_to_buf(task, 0);
3868 + sbd_new_task(task, initial_dbuf);
3869 +}
3870 +
3554 3871 /*
3555 3872 * This function is called during task clean-up if the
3556 3873 * DB_LU_FLAG is set on the dbuf. This should only be called for
3557 3874 * abort processing after sbd_abort has been called for the task.
3558 3875 */
3559 3876 void
3560 3877 sbd_dbuf_free(struct scsi_task *task, struct stmf_data_buf *dbuf)
3561 3878 {
3562 3879 sbd_cmd_t *scmd = (sbd_cmd_t *)task->task_lu_private;
3563 3880 sbd_lu_t *sl = (sbd_lu_t *)task->task_lu->lu_provider_private;
3564 3881
3565 3882 ASSERT(dbuf->db_lu_private);
3566 - ASSERT(scmd && scmd->nbufs > 0);
3883 + ASSERT(scmd && ATOMIC8_GET(scmd->nbufs) > 0);
3567 3884 ASSERT((scmd->flags & SBD_SCSI_CMD_ACTIVE) == 0);
3568 3885 ASSERT(dbuf->db_flags & DB_LU_DATA_BUF);
3569 3886 ASSERT(task->task_additional_flags & TASK_AF_ACCEPT_LU_DBUF);
3570 3887 ASSERT((curthread->t_flag & T_INTR_THREAD) == 0);
3571 3888
3572 3889 if (scmd->cmd_type == SBD_CMD_SCSI_READ) {
3573 3890 sbd_zvol_rele_read_bufs(sl, dbuf);
3574 3891 } else if (scmd->cmd_type == SBD_CMD_SCSI_WRITE) {
3575 3892 sbd_zvol_rele_write_bufs_abort(sl, dbuf);
3576 3893 } else {
3577 3894 cmn_err(CE_PANIC, "Unknown cmd type %d, task = %p",
3578 3895 scmd->cmd_type, (void *)task);
3579 3896 }
3580 - if (--scmd->nbufs == 0)
3897 + if (atomic_dec_8_nv(&scmd->nbufs) == 0)
3581 3898 rw_exit(&sl->sl_access_state_lock);
3582 3899 stmf_teardown_dbuf(task, dbuf);
3583 3900 stmf_free(dbuf);
3584 3901 }
3585 3902
3586 3903 /* ARGSUSED */
3587 3904 void
3588 3905 sbd_ctl(struct stmf_lu *lu, int cmd, void *arg)
3589 3906 {
3590 3907 sbd_lu_t *sl = (sbd_lu_t *)lu->lu_provider_private;
3591 3908 stmf_change_status_t st;
3592 3909
3593 3910 ASSERT((cmd == STMF_CMD_LU_ONLINE) ||
3594 3911 (cmd == STMF_CMD_LU_OFFLINE) ||
3595 3912 (cmd == STMF_ACK_LU_ONLINE_COMPLETE) ||
3596 3913 (cmd == STMF_ACK_LU_OFFLINE_COMPLETE));
3597 3914
3598 3915 st.st_completion_status = STMF_SUCCESS;
3599 3916 st.st_additional_info = NULL;
3600 3917
3601 3918 switch (cmd) {
3602 3919 case STMF_CMD_LU_ONLINE:
3603 3920 if (sl->sl_state == STMF_STATE_ONLINE)
3604 3921 st.st_completion_status = STMF_ALREADY;
3605 3922 else if (sl->sl_state != STMF_STATE_OFFLINE)
3606 3923 st.st_completion_status = STMF_FAILURE;
3607 3924 if (st.st_completion_status == STMF_SUCCESS) {
3608 3925 sl->sl_state = STMF_STATE_ONLINE;
3609 3926 sl->sl_state_not_acked = 1;
3610 3927 }
3611 3928 (void) stmf_ctl(STMF_CMD_LU_ONLINE_COMPLETE, lu, &st);
3612 3929 break;
3613 3930
3614 3931 case STMF_CMD_LU_OFFLINE:
3615 3932 if (sl->sl_state == STMF_STATE_OFFLINE)
3616 3933 st.st_completion_status = STMF_ALREADY;
3617 3934 else if (sl->sl_state != STMF_STATE_ONLINE)
3618 3935 st.st_completion_status = STMF_FAILURE;
3619 3936 if (st.st_completion_status == STMF_SUCCESS) {
3620 3937 sl->sl_flags &= ~(SL_MEDIUM_REMOVAL_PREVENTED |
3621 3938 SL_LU_HAS_SCSI2_RESERVATION);
3622 3939 sl->sl_state = STMF_STATE_OFFLINE;
3623 3940 sl->sl_state_not_acked = 1;
3624 3941 sbd_pgr_reset(sl);
3625 3942 }
3626 3943 (void) stmf_ctl(STMF_CMD_LU_OFFLINE_COMPLETE, lu, &st);
3627 3944 break;
3628 3945
3629 3946 case STMF_ACK_LU_ONLINE_COMPLETE:
3630 3947 /* Fallthrough */
3631 3948 case STMF_ACK_LU_OFFLINE_COMPLETE:
3632 3949 sl->sl_state_not_acked = 0;
3633 3950 break;
3634 3951
3635 3952 }
3636 3953 }
3637 3954
3638 3955 /* ARGSUSED */
3639 3956 stmf_status_t
3640 3957 sbd_info(uint32_t cmd, stmf_lu_t *lu, void *arg, uint8_t *buf,
3641 3958 uint32_t *bufsizep)
3642 3959 {
3643 3960 return (STMF_NOT_SUPPORTED);
3644 3961 }
3645 3962
3646 3963 stmf_status_t
3647 3964 sbd_lu_reset_state(stmf_lu_t *lu)
3648 3965 {
3649 3966 sbd_lu_t *sl = (sbd_lu_t *)lu->lu_provider_private;
3650 3967
3651 3968 mutex_enter(&sl->sl_lock);
3652 3969 if (sl->sl_flags & SL_SAVED_WRITE_CACHE_DISABLE) {
3653 3970 sl->sl_flags |= SL_WRITEBACK_CACHE_DISABLE;
3654 3971 mutex_exit(&sl->sl_lock);
3655 3972 if (sl->sl_access_state == SBD_LU_ACTIVE) {
3656 3973 (void) sbd_wcd_set(1, sl);
3657 3974 }
3658 3975 } else {
3659 3976 sl->sl_flags &= ~SL_WRITEBACK_CACHE_DISABLE;
3660 3977 mutex_exit(&sl->sl_lock);
3661 3978 if (sl->sl_access_state == SBD_LU_ACTIVE) {
3662 3979 (void) sbd_wcd_set(0, sl);
3663 3980 }
3664 3981 }
3665 3982 sbd_pgr_reset(sl);
3666 3983 sbd_check_and_clear_scsi2_reservation(sl, NULL);
|
↓ open down ↓ |
76 lines elided |
↑ open up ↑ |
3667 3984 if (stmf_deregister_all_lu_itl_handles(lu) != STMF_SUCCESS) {
3668 3985 return (STMF_FAILURE);
3669 3986 }
3670 3987 return (STMF_SUCCESS);
3671 3988 }
3672 3989
3673 3990 sbd_status_t
3674 3991 sbd_flush_data_cache(sbd_lu_t *sl, int fsync_done)
3675 3992 {
3676 3993 int r = 0;
3677 - int ret;
3994 + sbd_status_t ret;
3678 3995
3996 + rw_enter(&sl->sl_access_state_lock, RW_READER);
3997 + if ((sl->sl_flags & SL_MEDIA_LOADED) == 0) {
3998 + ret = SBD_FILEIO_FAILURE;
3999 + goto flush_fail;
4000 + }
3679 4001 if (fsync_done)
3680 4002 goto over_fsync;
3681 4003 if ((sl->sl_data_vtype == VREG) || (sl->sl_data_vtype == VBLK)) {
3682 - if (VOP_FSYNC(sl->sl_data_vp, FSYNC, kcred, NULL))
3683 - return (SBD_FAILURE);
4004 + if (VOP_FSYNC(sl->sl_data_vp, FSYNC, kcred, NULL)) {
4005 + ret = SBD_FAILURE;
4006 + goto flush_fail;
4007 + }
3684 4008 }
3685 4009 over_fsync:
3686 4010 if (((sl->sl_data_vtype == VCHR) || (sl->sl_data_vtype == VBLK)) &&
3687 4011 ((sl->sl_flags & SL_NO_DATA_DKIOFLUSH) == 0)) {
3688 4012 ret = VOP_IOCTL(sl->sl_data_vp, DKIOCFLUSHWRITECACHE, NULL,
3689 4013 FKIOCTL, kcred, &r, NULL);
3690 4014 if ((ret == ENOTTY) || (ret == ENOTSUP)) {
3691 4015 mutex_enter(&sl->sl_lock);
3692 4016 sl->sl_flags |= SL_NO_DATA_DKIOFLUSH;
3693 4017 mutex_exit(&sl->sl_lock);
3694 - } else if (ret != 0) {
3695 - return (SBD_FAILURE);
4018 + } else {
4019 + ret = (ret != 0) ? SBD_FAILURE : SBD_SUCCESS;
3696 4020 }
3697 4021 }
4022 +flush_fail:
4023 + rw_exit(&sl->sl_access_state_lock);
3698 4024
3699 - return (SBD_SUCCESS);
4025 + return (ret);
3700 4026 }
3701 4027
3702 4028 /* ARGSUSED */
3703 4029 static void
3704 4030 sbd_handle_sync_cache(struct scsi_task *task,
3705 4031 struct stmf_data_buf *initial_dbuf)
3706 4032 {
3707 4033 sbd_lu_t *sl = (sbd_lu_t *)task->task_lu->lu_provider_private;
3708 4034 uint64_t lba, laddr;
3709 4035 sbd_status_t sret;
3710 4036 uint32_t len;
3711 4037 int is_g4 = 0;
3712 4038 int immed;
3713 4039
3714 4040 task->task_cmd_xfer_length = 0;
3715 4041 /*
3716 4042 * Determine if this is a 10 or 16 byte CDB
3717 4043 */
3718 4044
3719 4045 if (task->task_cdb[0] == SCMD_SYNCHRONIZE_CACHE_G4)
3720 4046 is_g4 = 1;
3721 4047
3722 4048 /*
3723 4049 * Determine other requested parameters
3724 4050 *
3725 4051 * We don't have a non-volatile cache, so don't care about SYNC_NV.
3726 4052 * Do not support the IMMED bit.
3727 4053 */
3728 4054
3729 4055 immed = (task->task_cdb[1] & 0x02);
3730 4056
3731 4057 if (immed) {
3732 4058 stmf_scsilib_send_status(task, STATUS_CHECK,
3733 4059 STMF_SAA_INVALID_FIELD_IN_CDB);
3734 4060 return;
3735 4061 }
3736 4062
3737 4063 /*
3738 4064 * Check to be sure we're not being asked to sync an LBA
3739 4065 * that is out of range. While checking, verify reserved fields.
3740 4066 */
3741 4067
3742 4068 if (is_g4) {
3743 4069 if ((task->task_cdb[1] & 0xf9) || task->task_cdb[14] ||
3744 4070 task->task_cdb[15]) {
3745 4071 stmf_scsilib_send_status(task, STATUS_CHECK,
3746 4072 STMF_SAA_INVALID_FIELD_IN_CDB);
3747 4073 return;
3748 4074 }
3749 4075
3750 4076 lba = READ_SCSI64(&task->task_cdb[2], uint64_t);
3751 4077 len = READ_SCSI32(&task->task_cdb[10], uint32_t);
3752 4078 } else {
3753 4079 if ((task->task_cdb[1] & 0xf9) || task->task_cdb[6] ||
3754 4080 task->task_cdb[9]) {
3755 4081 stmf_scsilib_send_status(task, STATUS_CHECK,
3756 4082 STMF_SAA_INVALID_FIELD_IN_CDB);
3757 4083 return;
3758 4084 }
3759 4085
3760 4086 lba = READ_SCSI32(&task->task_cdb[2], uint64_t);
3761 4087 len = READ_SCSI16(&task->task_cdb[7], uint32_t);
3762 4088 }
3763 4089
3764 4090 laddr = lba << sl->sl_data_blocksize_shift;
3765 4091 len <<= sl->sl_data_blocksize_shift;
3766 4092
3767 4093 if ((laddr + (uint64_t)len) > sl->sl_lu_size) {
3768 4094 stmf_scsilib_send_status(task, STATUS_CHECK,
3769 4095 STMF_SAA_LBA_OUT_OF_RANGE);
3770 4096 return;
3771 4097 }
3772 4098
3773 4099 sret = sbd_flush_data_cache(sl, 0);
3774 4100 if (sret != SBD_SUCCESS) {
3775 4101 stmf_scsilib_send_status(task, STATUS_CHECK,
3776 4102 STMF_SAA_WRITE_ERROR);
3777 4103 return;
3778 4104 }
3779 4105
3780 4106 stmf_scsilib_send_status(task, STATUS_GOOD, 0);
3781 4107 }
|
↓ open down ↓ |
72 lines elided |
↑ open up ↑ |
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX