Print this page
6274 MAC tries to use aggr rings from downed links
Reviewed by: Bryan Cantrill <bryan@joyent.com>
Reviewed by: Patrick Mooney <patrick.mooney@joyent.com>
Reviewed by: Dan McDonald <danmcd@omniti.com>
Approved by: Richard Lowe <richlowe@richlowe.net>
| Split |
Close |
| Expand all |
| Collapse all |
--- old/usr/src/uts/common/io/mac/mac.c
+++ new/usr/src/uts/common/io/mac/mac.c
1 1 /*
2 2 * CDDL HEADER START
3 3 *
4 4 * The contents of this file are subject to the terms of the
5 5 * Common Development and Distribution License (the "License").
6 6 * You may not use this file except in compliance with the License.
7 7 *
8 8 * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE
9 9 * or http://www.opensolaris.org/os/licensing.
10 10 * See the License for the specific language governing permissions
11 11 * and limitations under the License.
12 12 *
13 13 * When distributing Covered Code, include this CDDL HEADER in each
|
↓ open down ↓ |
13 lines elided |
↑ open up ↑ |
14 14 * file and include the License file at usr/src/OPENSOLARIS.LICENSE.
15 15 * If applicable, add the following below this CDDL HEADER, with the
16 16 * fields enclosed by brackets "[]" replaced with your own identifying
17 17 * information: Portions Copyright [yyyy] [name of copyright owner]
18 18 *
19 19 * CDDL HEADER END
20 20 */
21 21
22 22 /*
23 23 * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved.
24 - * Copyright (c) 2014, Joyent, Inc. All rights reserved.
24 + * Copyright 2015 Joyent, Inc.
25 25 */
26 26
27 27 /*
28 28 * MAC Services Module
29 29 *
30 30 * The GLDv3 framework locking - The MAC layer
31 31 * --------------------------------------------
32 32 *
33 33 * The MAC layer is central to the GLD framework and can provide the locking
34 34 * framework needed for itself and for the use of MAC clients. MAC end points
35 35 * are fairly disjoint and don't share a lot of state. So a coarse grained
36 36 * multi-threading scheme is to single thread all create/modify/delete or set
37 37 * type of control operations on a per mac end point while allowing data threads
38 38 * concurrently.
39 39 *
40 40 * Control operations (set) that modify a mac end point are always serialized on
41 41 * a per mac end point basis, We have at most 1 such thread per mac end point
42 42 * at a time.
43 43 *
44 44 * All other operations that are not serialized are essentially multi-threaded.
45 45 * For example a control operation (get) like getting statistics which may not
46 46 * care about reading values atomically or data threads sending or receiving
47 47 * data. Mostly these type of operations don't modify the control state. Any
48 48 * state these operations care about are protected using traditional locks.
49 49 *
50 50 * The perimeter only serializes serial operations. It does not imply there
51 51 * aren't any other concurrent operations. However a serialized operation may
52 52 * sometimes need to make sure it is the only thread. In this case it needs
53 53 * to use reference counting mechanisms to cv_wait until any current data
54 54 * threads are done.
55 55 *
56 56 * The mac layer itself does not hold any locks across a call to another layer.
57 57 * The perimeter is however held across a down call to the driver to make the
58 58 * whole control operation atomic with respect to other control operations.
59 59 * Also the data path and get type control operations may proceed concurrently.
60 60 * These operations synchronize with the single serial operation on a given mac
61 61 * end point using regular locks. The perimeter ensures that conflicting
62 62 * operations like say a mac_multicast_add and a mac_multicast_remove on the
63 63 * same mac end point don't interfere with each other and also ensures that the
64 64 * changes in the mac layer and the call to the underlying driver to say add a
65 65 * multicast address are done atomically without interference from a thread
66 66 * trying to delete the same address.
67 67 *
68 68 * For example, consider
69 69 * mac_multicst_add()
70 70 * {
71 71 * mac_perimeter_enter(); serialize all control operations
72 72 *
73 73 * grab list lock protect against access by data threads
74 74 * add to list
75 75 * drop list lock
76 76 *
77 77 * call driver's mi_multicst
78 78 *
79 79 * mac_perimeter_exit();
80 80 * }
81 81 *
82 82 * To lessen the number of serialization locks and simplify the lock hierarchy,
83 83 * we serialize all the control operations on a per mac end point by using a
84 84 * single serialization lock called the perimeter. We allow recursive entry into
85 85 * the perimeter to facilitate use of this mechanism by both the mac client and
86 86 * the MAC layer itself.
87 87 *
88 88 * MAC client means an entity that does an operation on a mac handle
89 89 * obtained from a mac_open/mac_client_open. Similarly MAC driver means
90 90 * an entity that does an operation on a mac handle obtained from a
91 91 * mac_register. An entity could be both client and driver but on different
92 92 * handles eg. aggr. and should only make the corresponding mac interface calls
93 93 * i.e. mac driver interface or mac client interface as appropriate for that
94 94 * mac handle.
95 95 *
96 96 * General rules.
97 97 * -------------
98 98 *
99 99 * R1. The lock order of upcall threads is natually opposite to downcall
100 100 * threads. Hence upcalls must not hold any locks across layers for fear of
101 101 * recursive lock enter and lock order violation. This applies to all layers.
102 102 *
103 103 * R2. The perimeter is just another lock. Since it is held in the down
104 104 * direction, acquiring the perimeter in an upcall is prohibited as it would
105 105 * cause a deadlock. This applies to all layers.
106 106 *
107 107 * Note that upcalls that need to grab the mac perimeter (for example
108 108 * mac_notify upcalls) can still achieve that by posting the request to a
109 109 * thread, which can then grab all the required perimeters and locks in the
110 110 * right global order. Note that in the above example the mac layer iself
111 111 * won't grab the mac perimeter in the mac_notify upcall, instead the upcall
112 112 * to the client must do that. Please see the aggr code for an example.
113 113 *
114 114 * MAC client rules
115 115 * ----------------
116 116 *
117 117 * R3. A MAC client may use the MAC provided perimeter facility to serialize
118 118 * control operations on a per mac end point. It does this by by acquring
119 119 * and holding the perimeter across a sequence of calls to the mac layer.
120 120 * This ensures atomicity across the entire block of mac calls. In this
121 121 * model the MAC client must not hold any client locks across the calls to
122 122 * the mac layer. This model is the preferred solution.
123 123 *
124 124 * R4. However if a MAC client has a lot of global state across all mac end
125 125 * points the per mac end point serialization may not be sufficient. In this
126 126 * case the client may choose to use global locks or use its own serialization.
127 127 * To avoid deadlocks, these client layer locks held across the mac calls
128 128 * in the control path must never be acquired by the data path for the reason
129 129 * mentioned below.
130 130 *
131 131 * (Assume that a control operation that holds a client lock blocks in the
132 132 * mac layer waiting for upcall reference counts to drop to zero. If an upcall
133 133 * data thread that holds this reference count, tries to acquire the same
134 134 * client lock subsequently it will deadlock).
135 135 *
136 136 * A MAC client may follow either the R3 model or the R4 model, but can't
137 137 * mix both. In the former, the hierarchy is Perim -> client locks, but in
138 138 * the latter it is client locks -> Perim.
139 139 *
140 140 * R5. MAC clients must make MAC calls (excluding data calls) in a cv_wait'able
141 141 * context since they may block while trying to acquire the perimeter.
142 142 * In addition some calls may block waiting for upcall refcnts to come down to
143 143 * zero.
144 144 *
145 145 * R6. MAC clients must make sure that they are single threaded and all threads
146 146 * from the top (in particular data threads) have finished before calling
147 147 * mac_client_close. The MAC framework does not track the number of client
148 148 * threads using the mac client handle. Also mac clients must make sure
149 149 * they have undone all the control operations before calling mac_client_close.
150 150 * For example mac_unicast_remove/mac_multicast_remove to undo the corresponding
151 151 * mac_unicast_add/mac_multicast_add.
152 152 *
153 153 * MAC framework rules
154 154 * -------------------
155 155 *
156 156 * R7. The mac layer itself must not hold any mac layer locks (except the mac
157 157 * perimeter) across a call to any other layer from the mac layer. The call to
158 158 * any other layer could be via mi_* entry points, classifier entry points into
159 159 * the driver or via upcall pointers into layers above. The mac perimeter may
160 160 * be acquired or held only in the down direction, for e.g. when calling into
161 161 * a mi_* driver enty point to provide atomicity of the operation.
162 162 *
163 163 * R8. Since it is not guaranteed (see R14) that drivers won't hold locks across
164 164 * mac driver interfaces, the MAC layer must provide a cut out for control
165 165 * interfaces like upcall notifications and start them in a separate thread.
166 166 *
167 167 * R9. Note that locking order also implies a plumbing order. For example
168 168 * VNICs are allowed to be created over aggrs, but not vice-versa. An attempt
169 169 * to plumb in any other order must be failed at mac_open time, otherwise it
170 170 * could lead to deadlocks due to inverse locking order.
171 171 *
172 172 * R10. MAC driver interfaces must not block since the driver could call them
173 173 * in interrupt context.
174 174 *
175 175 * R11. Walkers must preferably not hold any locks while calling walker
176 176 * callbacks. Instead these can operate on reference counts. In simple
177 177 * callbacks it may be ok to hold a lock and call the callbacks, but this is
178 178 * harder to maintain in the general case of arbitrary callbacks.
179 179 *
180 180 * R12. The MAC layer must protect upcall notification callbacks using reference
181 181 * counts rather than holding locks across the callbacks.
182 182 *
183 183 * R13. Given the variety of drivers, it is preferable if the MAC layer can make
184 184 * sure that any pointers (such as mac ring pointers) it passes to the driver
185 185 * remain valid until mac unregister time. Currently the mac layer achieves
186 186 * this by using generation numbers for rings and freeing the mac rings only
187 187 * at unregister time. The MAC layer must provide a layer of indirection and
188 188 * must not expose underlying driver rings or driver data structures/pointers
189 189 * directly to MAC clients.
190 190 *
191 191 * MAC driver rules
192 192 * ----------------
193 193 *
194 194 * R14. It would be preferable if MAC drivers don't hold any locks across any
195 195 * mac call. However at a minimum they must not hold any locks across data
196 196 * upcalls. They must also make sure that all references to mac data structures
197 197 * are cleaned up and that it is single threaded at mac_unregister time.
198 198 *
199 199 * R15. MAC driver interfaces don't block and so the action may be done
200 200 * asynchronously in a separate thread as for example handling notifications.
201 201 * The driver must not assume that the action is complete when the call
202 202 * returns.
203 203 *
204 204 * R16. Drivers must maintain a generation number per Rx ring, and pass it
205 205 * back to mac_rx_ring(); They are expected to increment the generation
206 206 * number whenever the ring's stop routine is invoked.
207 207 * See comments in mac_rx_ring();
208 208 *
209 209 * R17 Similarly mi_stop is another synchronization point and the driver must
210 210 * ensure that all upcalls are done and there won't be any future upcall
211 211 * before returning from mi_stop.
212 212 *
213 213 * R18. The driver may assume that all set/modify control operations via
214 214 * the mi_* entry points are single threaded on a per mac end point.
215 215 *
216 216 * Lock and Perimeter hierarchy scenarios
217 217 * ---------------------------------------
218 218 *
219 219 * i_mac_impl_lock -> mi_rw_lock -> srs_lock -> s_ring_lock[i_mac_tx_srs_notify]
220 220 *
221 221 * ft_lock -> fe_lock [mac_flow_lookup]
222 222 *
223 223 * mi_rw_lock -> fe_lock [mac_bcast_send]
224 224 *
225 225 * srs_lock -> mac_bw_lock [mac_rx_srs_drain_bw]
226 226 *
227 227 * cpu_lock -> mac_srs_g_lock -> srs_lock -> s_ring_lock [mac_walk_srs_and_bind]
228 228 *
229 229 * i_dls_devnet_lock -> mac layer locks [dls_devnet_rename]
230 230 *
231 231 * Perimeters are ordered P1 -> P2 -> P3 from top to bottom in order of mac
232 232 * client to driver. In the case of clients that explictly use the mac provided
233 233 * perimeter mechanism for its serialization, the hierarchy is
234 234 * Perimeter -> mac layer locks, since the client never holds any locks across
235 235 * the mac calls. In the case of clients that use its own locks the hierarchy
236 236 * is Client locks -> Mac Perim -> Mac layer locks. The client never explicitly
237 237 * calls mac_perim_enter/exit in this case.
238 238 *
239 239 * Subflow creation rules
240 240 * ---------------------------
241 241 * o In case of a user specified cpulist present on underlying link and flows,
242 242 * the flows cpulist must be a subset of the underlying link.
243 243 * o In case of a user specified fanout mode present on link and flow, the
244 244 * subflow fanout count has to be less than or equal to that of the
245 245 * underlying link. The cpu-bindings for the subflows will be a subset of
246 246 * the underlying link.
247 247 * o In case if no cpulist specified on both underlying link and flow, the
248 248 * underlying link relies on a MAC tunable to provide out of box fanout.
249 249 * The subflow will have no cpulist (the subflow will be unbound)
250 250 * o In case if no cpulist is specified on the underlying link, a subflow can
251 251 * carry either a user-specified cpulist or fanout count. The cpu-bindings
252 252 * for the subflow will not adhere to restriction that they need to be subset
253 253 * of the underlying link.
254 254 * o In case where the underlying link is carrying either a user specified
255 255 * cpulist or fanout mode and for a unspecified subflow, the subflow will be
256 256 * created unbound.
257 257 * o While creating unbound subflows, bandwidth mode changes attempt to
258 258 * figure a right fanout count. In such cases the fanout count will override
259 259 * the unbound cpu-binding behavior.
260 260 * o In addition to this, while cycling between flow and link properties, we
261 261 * impose a restriction that if a link property has a subflow with
262 262 * user-specified attributes, we will not allow changing the link property.
263 263 * The administrator needs to reset all the user specified properties for the
264 264 * subflows before attempting a link property change.
265 265 * Some of the above rules can be overridden by specifying additional command
266 266 * line options while creating or modifying link or subflow properties.
267 267 */
268 268
269 269 #include <sys/types.h>
270 270 #include <sys/conf.h>
271 271 #include <sys/id_space.h>
272 272 #include <sys/esunddi.h>
273 273 #include <sys/stat.h>
274 274 #include <sys/mkdev.h>
275 275 #include <sys/stream.h>
276 276 #include <sys/strsun.h>
277 277 #include <sys/strsubr.h>
278 278 #include <sys/dlpi.h>
279 279 #include <sys/list.h>
280 280 #include <sys/modhash.h>
281 281 #include <sys/mac_provider.h>
282 282 #include <sys/mac_client_impl.h>
283 283 #include <sys/mac_soft_ring.h>
284 284 #include <sys/mac_stat.h>
285 285 #include <sys/mac_impl.h>
286 286 #include <sys/mac.h>
287 287 #include <sys/dls.h>
288 288 #include <sys/dld.h>
289 289 #include <sys/modctl.h>
290 290 #include <sys/fs/dv_node.h>
291 291 #include <sys/thread.h>
292 292 #include <sys/proc.h>
293 293 #include <sys/callb.h>
294 294 #include <sys/cpuvar.h>
295 295 #include <sys/atomic.h>
296 296 #include <sys/bitmap.h>
297 297 #include <sys/sdt.h>
298 298 #include <sys/mac_flow.h>
299 299 #include <sys/ddi_intr_impl.h>
300 300 #include <sys/disp.h>
301 301 #include <sys/sdt.h>
302 302 #include <sys/vnic.h>
303 303 #include <sys/vnic_impl.h>
304 304 #include <sys/vlan.h>
305 305 #include <inet/ip.h>
306 306 #include <inet/ip6.h>
307 307 #include <sys/exacct.h>
308 308 #include <sys/exacct_impl.h>
309 309 #include <inet/nd.h>
310 310 #include <sys/ethernet.h>
311 311 #include <sys/pool.h>
312 312 #include <sys/pool_pset.h>
313 313 #include <sys/cpupart.h>
314 314 #include <inet/wifi_ioctl.h>
315 315 #include <net/wpa.h>
316 316
317 317 #define IMPL_HASHSZ 67 /* prime */
318 318
319 319 kmem_cache_t *i_mac_impl_cachep;
320 320 mod_hash_t *i_mac_impl_hash;
321 321 krwlock_t i_mac_impl_lock;
322 322 uint_t i_mac_impl_count;
323 323 static kmem_cache_t *mac_ring_cache;
324 324 static id_space_t *minor_ids;
325 325 static uint32_t minor_count;
326 326 static pool_event_cb_t mac_pool_event_reg;
327 327
328 328 /*
329 329 * Logging stuff. Perhaps mac_logging_interval could be broken into
330 330 * mac_flow_log_interval and mac_link_log_interval if we want to be
331 331 * able to schedule them differently.
332 332 */
333 333 uint_t mac_logging_interval;
334 334 boolean_t mac_flow_log_enable;
335 335 boolean_t mac_link_log_enable;
336 336 timeout_id_t mac_logging_timer;
337 337
338 338 /* for debugging, see MAC_DBG_PRT() in mac_impl.h */
339 339 int mac_dbg = 0;
340 340
341 341 #define MACTYPE_KMODDIR "mac"
342 342 #define MACTYPE_HASHSZ 67
343 343 static mod_hash_t *i_mactype_hash;
344 344 /*
345 345 * i_mactype_lock synchronizes threads that obtain references to mactype_t
346 346 * structures through i_mactype_getplugin().
347 347 */
348 348 static kmutex_t i_mactype_lock;
349 349
350 350 /*
351 351 * mac_tx_percpu_cnt
352 352 *
353 353 * Number of per cpu locks per mac_client_impl_t. Used by the transmit side
354 354 * in mac_tx to reduce lock contention. This is sized at boot time in mac_init.
355 355 * mac_tx_percpu_cnt_max is settable in /etc/system and must be a power of 2.
356 356 * Per cpu locks may be disabled by setting mac_tx_percpu_cnt_max to 1.
357 357 */
358 358 int mac_tx_percpu_cnt;
359 359 int mac_tx_percpu_cnt_max = 128;
360 360
361 361 /*
362 362 * Call back functions for the bridge module. These are guaranteed to be valid
363 363 * when holding a reference on a link or when holding mip->mi_bridge_lock and
364 364 * mi_bridge_link is non-NULL.
365 365 */
366 366 mac_bridge_tx_t mac_bridge_tx_cb;
367 367 mac_bridge_rx_t mac_bridge_rx_cb;
368 368 mac_bridge_ref_t mac_bridge_ref_cb;
369 369 mac_bridge_ls_t mac_bridge_ls_cb;
370 370
371 371 static int i_mac_constructor(void *, void *, int);
372 372 static void i_mac_destructor(void *, void *);
373 373 static int i_mac_ring_ctor(void *, void *, int);
374 374 static void i_mac_ring_dtor(void *, void *);
375 375 static mblk_t *mac_rx_classify(mac_impl_t *, mac_resource_handle_t, mblk_t *);
376 376 void mac_tx_client_flush(mac_client_impl_t *);
377 377 void mac_tx_client_block(mac_client_impl_t *);
378 378 static void mac_rx_ring_quiesce(mac_ring_t *, uint_t);
379 379 static int mac_start_group_and_rings(mac_group_t *);
380 380 static void mac_stop_group_and_rings(mac_group_t *);
381 381 static void mac_pool_event_cb(pool_event_t, int, void *);
382 382
383 383 typedef struct netinfo_s {
384 384 list_node_t ni_link;
385 385 void *ni_record;
386 386 int ni_size;
387 387 int ni_type;
388 388 } netinfo_t;
389 389
390 390 /*
391 391 * Module initialization functions.
392 392 */
393 393
394 394 void
395 395 mac_init(void)
396 396 {
397 397 mac_tx_percpu_cnt = ((boot_max_ncpus == -1) ? max_ncpus :
398 398 boot_max_ncpus);
399 399
400 400 /* Upper bound is mac_tx_percpu_cnt_max */
401 401 if (mac_tx_percpu_cnt > mac_tx_percpu_cnt_max)
402 402 mac_tx_percpu_cnt = mac_tx_percpu_cnt_max;
403 403
404 404 if (mac_tx_percpu_cnt < 1) {
405 405 /* Someone set max_tx_percpu_cnt_max to 0 or less */
406 406 mac_tx_percpu_cnt = 1;
407 407 }
408 408
409 409 ASSERT(mac_tx_percpu_cnt >= 1);
410 410 mac_tx_percpu_cnt = (1 << highbit(mac_tx_percpu_cnt - 1));
411 411 /*
412 412 * Make it of the form 2**N - 1 in the range
413 413 * [0 .. mac_tx_percpu_cnt_max - 1]
414 414 */
415 415 mac_tx_percpu_cnt--;
416 416
417 417 i_mac_impl_cachep = kmem_cache_create("mac_impl_cache",
418 418 sizeof (mac_impl_t), 0, i_mac_constructor, i_mac_destructor,
419 419 NULL, NULL, NULL, 0);
420 420 ASSERT(i_mac_impl_cachep != NULL);
421 421
422 422 mac_ring_cache = kmem_cache_create("mac_ring_cache",
423 423 sizeof (mac_ring_t), 0, i_mac_ring_ctor, i_mac_ring_dtor, NULL,
424 424 NULL, NULL, 0);
425 425 ASSERT(mac_ring_cache != NULL);
426 426
427 427 i_mac_impl_hash = mod_hash_create_extended("mac_impl_hash",
428 428 IMPL_HASHSZ, mod_hash_null_keydtor, mod_hash_null_valdtor,
429 429 mod_hash_bystr, NULL, mod_hash_strkey_cmp, KM_SLEEP);
430 430 rw_init(&i_mac_impl_lock, NULL, RW_DEFAULT, NULL);
431 431
432 432 mac_flow_init();
433 433 mac_soft_ring_init();
434 434 mac_bcast_init();
435 435 mac_client_init();
436 436
437 437 i_mac_impl_count = 0;
438 438
439 439 i_mactype_hash = mod_hash_create_extended("mactype_hash",
440 440 MACTYPE_HASHSZ,
441 441 mod_hash_null_keydtor, mod_hash_null_valdtor,
442 442 mod_hash_bystr, NULL, mod_hash_strkey_cmp, KM_SLEEP);
443 443
444 444 /*
445 445 * Allocate an id space to manage minor numbers. The range of the
446 446 * space will be from MAC_MAX_MINOR+1 to MAC_PRIVATE_MINOR-1. This
447 447 * leaves half of the 32-bit minors available for driver private use.
448 448 */
449 449 minor_ids = id_space_create("mac_minor_ids", MAC_MAX_MINOR+1,
450 450 MAC_PRIVATE_MINOR-1);
451 451 ASSERT(minor_ids != NULL);
452 452 minor_count = 0;
453 453
454 454 /* Let's default to 20 seconds */
455 455 mac_logging_interval = 20;
456 456 mac_flow_log_enable = B_FALSE;
457 457 mac_link_log_enable = B_FALSE;
458 458 mac_logging_timer = 0;
459 459
460 460 /* Register to be notified of noteworthy pools events */
461 461 mac_pool_event_reg.pec_func = mac_pool_event_cb;
462 462 mac_pool_event_reg.pec_arg = NULL;
463 463 pool_event_cb_register(&mac_pool_event_reg);
464 464 }
465 465
466 466 int
467 467 mac_fini(void)
468 468 {
469 469
470 470 if (i_mac_impl_count > 0 || minor_count > 0)
471 471 return (EBUSY);
472 472
473 473 pool_event_cb_unregister(&mac_pool_event_reg);
474 474
475 475 id_space_destroy(minor_ids);
476 476 mac_flow_fini();
477 477
478 478 mod_hash_destroy_hash(i_mac_impl_hash);
479 479 rw_destroy(&i_mac_impl_lock);
480 480
481 481 mac_client_fini();
482 482 kmem_cache_destroy(mac_ring_cache);
483 483
484 484 mod_hash_destroy_hash(i_mactype_hash);
485 485 mac_soft_ring_finish();
486 486
487 487
488 488 return (0);
489 489 }
490 490
491 491 /*
492 492 * Initialize a GLDv3 driver's device ops. A driver that manages its own ops
493 493 * (e.g. softmac) may pass in a NULL ops argument.
494 494 */
495 495 void
496 496 mac_init_ops(struct dev_ops *ops, const char *name)
497 497 {
498 498 major_t major = ddi_name_to_major((char *)name);
499 499
500 500 /*
501 501 * By returning on error below, we are not letting the driver continue
502 502 * in an undefined context. The mac_register() function will faill if
503 503 * DN_GLDV3_DRIVER isn't set.
504 504 */
505 505 if (major == DDI_MAJOR_T_NONE)
506 506 return;
507 507 LOCK_DEV_OPS(&devnamesp[major].dn_lock);
508 508 devnamesp[major].dn_flags |= (DN_GLDV3_DRIVER | DN_NETWORK_DRIVER);
509 509 UNLOCK_DEV_OPS(&devnamesp[major].dn_lock);
510 510 if (ops != NULL)
511 511 dld_init_ops(ops, name);
512 512 }
513 513
514 514 void
515 515 mac_fini_ops(struct dev_ops *ops)
516 516 {
517 517 dld_fini_ops(ops);
518 518 }
519 519
520 520 /*ARGSUSED*/
521 521 static int
522 522 i_mac_constructor(void *buf, void *arg, int kmflag)
523 523 {
524 524 mac_impl_t *mip = buf;
525 525
526 526 bzero(buf, sizeof (mac_impl_t));
527 527
528 528 mip->mi_linkstate = LINK_STATE_UNKNOWN;
529 529
530 530 rw_init(&mip->mi_rw_lock, NULL, RW_DRIVER, NULL);
531 531 mutex_init(&mip->mi_notify_lock, NULL, MUTEX_DRIVER, NULL);
532 532 mutex_init(&mip->mi_promisc_lock, NULL, MUTEX_DRIVER, NULL);
533 533 mutex_init(&mip->mi_ring_lock, NULL, MUTEX_DEFAULT, NULL);
534 534
535 535 mip->mi_notify_cb_info.mcbi_lockp = &mip->mi_notify_lock;
536 536 cv_init(&mip->mi_notify_cb_info.mcbi_cv, NULL, CV_DRIVER, NULL);
537 537 mip->mi_promisc_cb_info.mcbi_lockp = &mip->mi_promisc_lock;
538 538 cv_init(&mip->mi_promisc_cb_info.mcbi_cv, NULL, CV_DRIVER, NULL);
539 539
540 540 mutex_init(&mip->mi_bridge_lock, NULL, MUTEX_DEFAULT, NULL);
541 541
542 542 return (0);
543 543 }
544 544
545 545 /*ARGSUSED*/
546 546 static void
547 547 i_mac_destructor(void *buf, void *arg)
548 548 {
549 549 mac_impl_t *mip = buf;
550 550 mac_cb_info_t *mcbi;
551 551
552 552 ASSERT(mip->mi_ref == 0);
553 553 ASSERT(mip->mi_active == 0);
554 554 ASSERT(mip->mi_linkstate == LINK_STATE_UNKNOWN);
555 555 ASSERT(mip->mi_devpromisc == 0);
556 556 ASSERT(mip->mi_ksp == NULL);
557 557 ASSERT(mip->mi_kstat_count == 0);
558 558 ASSERT(mip->mi_nclients == 0);
559 559 ASSERT(mip->mi_nactiveclients == 0);
560 560 ASSERT(mip->mi_single_active_client == NULL);
561 561 ASSERT(mip->mi_state_flags == 0);
562 562 ASSERT(mip->mi_factory_addr == NULL);
563 563 ASSERT(mip->mi_factory_addr_num == 0);
564 564 ASSERT(mip->mi_default_tx_ring == NULL);
565 565
566 566 mcbi = &mip->mi_notify_cb_info;
567 567 ASSERT(mcbi->mcbi_del_cnt == 0 && mcbi->mcbi_walker_cnt == 0);
568 568 ASSERT(mip->mi_notify_bits == 0);
569 569 ASSERT(mip->mi_notify_thread == NULL);
570 570 ASSERT(mcbi->mcbi_lockp == &mip->mi_notify_lock);
571 571 mcbi->mcbi_lockp = NULL;
572 572
573 573 mcbi = &mip->mi_promisc_cb_info;
574 574 ASSERT(mcbi->mcbi_del_cnt == 0 && mip->mi_promisc_list == NULL);
575 575 ASSERT(mip->mi_promisc_list == NULL);
576 576 ASSERT(mcbi->mcbi_lockp == &mip->mi_promisc_lock);
577 577 mcbi->mcbi_lockp = NULL;
578 578
579 579 ASSERT(mip->mi_bcast_ngrps == 0 && mip->mi_bcast_grp == NULL);
580 580 ASSERT(mip->mi_perim_owner == NULL && mip->mi_perim_ocnt == 0);
581 581
582 582 rw_destroy(&mip->mi_rw_lock);
583 583
584 584 mutex_destroy(&mip->mi_promisc_lock);
585 585 cv_destroy(&mip->mi_promisc_cb_info.mcbi_cv);
586 586 mutex_destroy(&mip->mi_notify_lock);
587 587 cv_destroy(&mip->mi_notify_cb_info.mcbi_cv);
588 588 mutex_destroy(&mip->mi_ring_lock);
589 589
590 590 ASSERT(mip->mi_bridge_link == NULL);
591 591 }
592 592
593 593 /* ARGSUSED */
594 594 static int
595 595 i_mac_ring_ctor(void *buf, void *arg, int kmflag)
596 596 {
597 597 mac_ring_t *ring = (mac_ring_t *)buf;
598 598
599 599 bzero(ring, sizeof (mac_ring_t));
600 600 cv_init(&ring->mr_cv, NULL, CV_DEFAULT, NULL);
601 601 mutex_init(&ring->mr_lock, NULL, MUTEX_DEFAULT, NULL);
602 602 ring->mr_state = MR_FREE;
603 603 return (0);
604 604 }
605 605
606 606 /* ARGSUSED */
607 607 static void
608 608 i_mac_ring_dtor(void *buf, void *arg)
609 609 {
610 610 mac_ring_t *ring = (mac_ring_t *)buf;
611 611
612 612 cv_destroy(&ring->mr_cv);
613 613 mutex_destroy(&ring->mr_lock);
614 614 }
615 615
616 616 /*
617 617 * Common functions to do mac callback addition and deletion. Currently this is
618 618 * used by promisc callbacks and notify callbacks. List addition and deletion
619 619 * need to take care of list walkers. List walkers in general, can't hold list
620 620 * locks and make upcall callbacks due to potential lock order and recursive
621 621 * reentry issues. Instead list walkers increment the list walker count to mark
622 622 * the presence of a walker thread. Addition can be carefully done to ensure
623 623 * that the list walker always sees either the old list or the new list.
624 624 * However the deletion can't be done while the walker is active, instead the
625 625 * deleting thread simply marks the entry as logically deleted. The last walker
626 626 * physically deletes and frees up the logically deleted entries when the walk
627 627 * is complete.
628 628 */
629 629 void
630 630 mac_callback_add(mac_cb_info_t *mcbi, mac_cb_t **mcb_head,
631 631 mac_cb_t *mcb_elem)
632 632 {
633 633 mac_cb_t *p;
634 634 mac_cb_t **pp;
635 635
636 636 /* Verify it is not already in the list */
637 637 for (pp = mcb_head; (p = *pp) != NULL; pp = &p->mcb_nextp) {
638 638 if (p == mcb_elem)
639 639 break;
640 640 }
641 641 VERIFY(p == NULL);
642 642
643 643 /*
644 644 * Add it to the head of the callback list. The membar ensures that
645 645 * the following list pointer manipulations reach global visibility
646 646 * in exactly the program order below.
647 647 */
648 648 ASSERT(MUTEX_HELD(mcbi->mcbi_lockp));
649 649
650 650 mcb_elem->mcb_nextp = *mcb_head;
651 651 membar_producer();
652 652 *mcb_head = mcb_elem;
653 653 }
654 654
655 655 /*
656 656 * Mark the entry as logically deleted. If there aren't any walkers unlink
657 657 * from the list. In either case return the corresponding status.
658 658 */
659 659 boolean_t
660 660 mac_callback_remove(mac_cb_info_t *mcbi, mac_cb_t **mcb_head,
661 661 mac_cb_t *mcb_elem)
662 662 {
663 663 mac_cb_t *p;
664 664 mac_cb_t **pp;
665 665
666 666 ASSERT(MUTEX_HELD(mcbi->mcbi_lockp));
667 667 /*
668 668 * Search the callback list for the entry to be removed
669 669 */
670 670 for (pp = mcb_head; (p = *pp) != NULL; pp = &p->mcb_nextp) {
671 671 if (p == mcb_elem)
672 672 break;
673 673 }
674 674 VERIFY(p != NULL);
675 675
676 676 /*
677 677 * If there are walkers just mark it as deleted and the last walker
678 678 * will remove from the list and free it.
679 679 */
680 680 if (mcbi->mcbi_walker_cnt != 0) {
681 681 p->mcb_flags |= MCB_CONDEMNED;
682 682 mcbi->mcbi_del_cnt++;
683 683 return (B_FALSE);
684 684 }
685 685
686 686 ASSERT(mcbi->mcbi_del_cnt == 0);
687 687 *pp = p->mcb_nextp;
688 688 p->mcb_nextp = NULL;
689 689 return (B_TRUE);
690 690 }
691 691
692 692 /*
693 693 * Wait for all pending callback removals to be completed
694 694 */
695 695 void
696 696 mac_callback_remove_wait(mac_cb_info_t *mcbi)
697 697 {
698 698 ASSERT(MUTEX_HELD(mcbi->mcbi_lockp));
699 699 while (mcbi->mcbi_del_cnt != 0) {
700 700 DTRACE_PROBE1(need_wait, mac_cb_info_t *, mcbi);
701 701 cv_wait(&mcbi->mcbi_cv, mcbi->mcbi_lockp);
702 702 }
703 703 }
704 704
705 705 /*
706 706 * The last mac callback walker does the cleanup. Walk the list and unlik
707 707 * all the logically deleted entries and construct a temporary list of
708 708 * removed entries. Return the list of removed entries to the caller.
709 709 */
710 710 mac_cb_t *
711 711 mac_callback_walker_cleanup(mac_cb_info_t *mcbi, mac_cb_t **mcb_head)
712 712 {
713 713 mac_cb_t *p;
714 714 mac_cb_t **pp;
715 715 mac_cb_t *rmlist = NULL; /* List of removed elements */
716 716 int cnt = 0;
717 717
718 718 ASSERT(MUTEX_HELD(mcbi->mcbi_lockp));
719 719 ASSERT(mcbi->mcbi_del_cnt != 0 && mcbi->mcbi_walker_cnt == 0);
720 720
721 721 pp = mcb_head;
722 722 while (*pp != NULL) {
723 723 if ((*pp)->mcb_flags & MCB_CONDEMNED) {
724 724 p = *pp;
725 725 *pp = p->mcb_nextp;
726 726 p->mcb_nextp = rmlist;
727 727 rmlist = p;
728 728 cnt++;
729 729 continue;
730 730 }
731 731 pp = &(*pp)->mcb_nextp;
732 732 }
733 733
734 734 ASSERT(mcbi->mcbi_del_cnt == cnt);
735 735 mcbi->mcbi_del_cnt = 0;
736 736 return (rmlist);
737 737 }
738 738
739 739 boolean_t
740 740 mac_callback_lookup(mac_cb_t **mcb_headp, mac_cb_t *mcb_elem)
741 741 {
742 742 mac_cb_t *mcb;
743 743
744 744 /* Verify it is not already in the list */
745 745 for (mcb = *mcb_headp; mcb != NULL; mcb = mcb->mcb_nextp) {
746 746 if (mcb == mcb_elem)
747 747 return (B_TRUE);
748 748 }
749 749
750 750 return (B_FALSE);
751 751 }
752 752
753 753 boolean_t
754 754 mac_callback_find(mac_cb_info_t *mcbi, mac_cb_t **mcb_headp, mac_cb_t *mcb_elem)
755 755 {
756 756 boolean_t found;
757 757
758 758 mutex_enter(mcbi->mcbi_lockp);
759 759 found = mac_callback_lookup(mcb_headp, mcb_elem);
760 760 mutex_exit(mcbi->mcbi_lockp);
761 761
762 762 return (found);
763 763 }
764 764
765 765 /* Free the list of removed callbacks */
766 766 void
767 767 mac_callback_free(mac_cb_t *rmlist)
768 768 {
769 769 mac_cb_t *mcb;
770 770 mac_cb_t *mcb_next;
771 771
772 772 for (mcb = rmlist; mcb != NULL; mcb = mcb_next) {
773 773 mcb_next = mcb->mcb_nextp;
774 774 kmem_free(mcb->mcb_objp, mcb->mcb_objsize);
775 775 }
776 776 }
777 777
778 778 /*
779 779 * The promisc callbacks are in 2 lists, one off the 'mip' and another off the
780 780 * 'mcip' threaded by mpi_mi_link and mpi_mci_link respectively. However there
781 781 * is only a single shared total walker count, and an entry can't be physically
782 782 * unlinked if a walker is active on either list. The last walker does this
783 783 * cleanup of logically deleted entries.
784 784 */
785 785 void
786 786 i_mac_promisc_walker_cleanup(mac_impl_t *mip)
787 787 {
788 788 mac_cb_t *rmlist;
789 789 mac_cb_t *mcb;
790 790 mac_cb_t *mcb_next;
791 791 mac_promisc_impl_t *mpip;
792 792
793 793 /*
794 794 * Construct a temporary list of deleted callbacks by walking the
795 795 * the mi_promisc_list. Then for each entry in the temporary list,
796 796 * remove it from the mci_promisc_list and free the entry.
797 797 */
798 798 rmlist = mac_callback_walker_cleanup(&mip->mi_promisc_cb_info,
799 799 &mip->mi_promisc_list);
800 800
801 801 for (mcb = rmlist; mcb != NULL; mcb = mcb_next) {
802 802 mcb_next = mcb->mcb_nextp;
803 803 mpip = (mac_promisc_impl_t *)mcb->mcb_objp;
804 804 VERIFY(mac_callback_remove(&mip->mi_promisc_cb_info,
805 805 &mpip->mpi_mcip->mci_promisc_list, &mpip->mpi_mci_link));
806 806 mcb->mcb_flags = 0;
807 807 mcb->mcb_nextp = NULL;
808 808 kmem_cache_free(mac_promisc_impl_cache, mpip);
809 809 }
810 810 }
811 811
812 812 void
813 813 i_mac_notify(mac_impl_t *mip, mac_notify_type_t type)
814 814 {
815 815 mac_cb_info_t *mcbi;
816 816
817 817 /*
818 818 * Signal the notify thread even after mi_ref has become zero and
819 819 * mi_disabled is set. The synchronization with the notify thread
820 820 * happens in mac_unregister and that implies the driver must make
821 821 * sure it is single-threaded (with respect to mac calls) and that
822 822 * all pending mac calls have returned before it calls mac_unregister
823 823 */
824 824 rw_enter(&i_mac_impl_lock, RW_READER);
825 825 if (mip->mi_state_flags & MIS_DISABLED)
826 826 goto exit;
827 827
828 828 /*
829 829 * Guard against incorrect notifications. (Running a newer
830 830 * mac client against an older implementation?)
831 831 */
832 832 if (type >= MAC_NNOTE)
833 833 goto exit;
834 834
835 835 mcbi = &mip->mi_notify_cb_info;
836 836 mutex_enter(mcbi->mcbi_lockp);
837 837 mip->mi_notify_bits |= (1 << type);
838 838 cv_broadcast(&mcbi->mcbi_cv);
839 839 mutex_exit(mcbi->mcbi_lockp);
840 840
841 841 exit:
842 842 rw_exit(&i_mac_impl_lock);
843 843 }
844 844
845 845 /*
846 846 * Mac serialization primitives. Please see the block comment at the
847 847 * top of the file.
848 848 */
849 849 void
850 850 i_mac_perim_enter(mac_impl_t *mip)
851 851 {
852 852 mac_client_impl_t *mcip;
853 853
854 854 if (mip->mi_state_flags & MIS_IS_VNIC) {
855 855 /*
856 856 * This is a VNIC. Return the lower mac since that is what
857 857 * we want to serialize on.
858 858 */
859 859 mcip = mac_vnic_lower(mip);
860 860 mip = mcip->mci_mip;
861 861 }
862 862
863 863 mutex_enter(&mip->mi_perim_lock);
864 864 if (mip->mi_perim_owner == curthread) {
865 865 mip->mi_perim_ocnt++;
866 866 mutex_exit(&mip->mi_perim_lock);
867 867 return;
868 868 }
869 869
870 870 while (mip->mi_perim_owner != NULL)
871 871 cv_wait(&mip->mi_perim_cv, &mip->mi_perim_lock);
872 872
873 873 mip->mi_perim_owner = curthread;
874 874 ASSERT(mip->mi_perim_ocnt == 0);
875 875 mip->mi_perim_ocnt++;
876 876 #ifdef DEBUG
877 877 mip->mi_perim_stack_depth = getpcstack(mip->mi_perim_stack,
878 878 MAC_PERIM_STACK_DEPTH);
879 879 #endif
880 880 mutex_exit(&mip->mi_perim_lock);
881 881 }
882 882
883 883 int
884 884 i_mac_perim_enter_nowait(mac_impl_t *mip)
885 885 {
886 886 /*
887 887 * The vnic is a special case, since the serialization is done based
888 888 * on the lower mac. If the lower mac is busy, it does not imply the
889 889 * vnic can't be unregistered. But in the case of other drivers,
890 890 * a busy perimeter or open mac handles implies that the mac is busy
891 891 * and can't be unregistered.
892 892 */
893 893 if (mip->mi_state_flags & MIS_IS_VNIC) {
894 894 i_mac_perim_enter(mip);
895 895 return (0);
896 896 }
897 897
898 898 mutex_enter(&mip->mi_perim_lock);
899 899 if (mip->mi_perim_owner != NULL) {
900 900 mutex_exit(&mip->mi_perim_lock);
901 901 return (EBUSY);
902 902 }
903 903 ASSERT(mip->mi_perim_ocnt == 0);
904 904 mip->mi_perim_owner = curthread;
905 905 mip->mi_perim_ocnt++;
906 906 mutex_exit(&mip->mi_perim_lock);
907 907
908 908 return (0);
909 909 }
910 910
911 911 void
912 912 i_mac_perim_exit(mac_impl_t *mip)
913 913 {
914 914 mac_client_impl_t *mcip;
915 915
916 916 if (mip->mi_state_flags & MIS_IS_VNIC) {
917 917 /*
918 918 * This is a VNIC. Return the lower mac since that is what
919 919 * we want to serialize on.
920 920 */
921 921 mcip = mac_vnic_lower(mip);
922 922 mip = mcip->mci_mip;
923 923 }
924 924
925 925 ASSERT(mip->mi_perim_owner == curthread && mip->mi_perim_ocnt != 0);
926 926
927 927 mutex_enter(&mip->mi_perim_lock);
928 928 if (--mip->mi_perim_ocnt == 0) {
929 929 mip->mi_perim_owner = NULL;
930 930 cv_signal(&mip->mi_perim_cv);
931 931 }
932 932 mutex_exit(&mip->mi_perim_lock);
933 933 }
934 934
935 935 /*
936 936 * Returns whether the current thread holds the mac perimeter. Used in making
937 937 * assertions.
938 938 */
939 939 boolean_t
940 940 mac_perim_held(mac_handle_t mh)
941 941 {
942 942 mac_impl_t *mip = (mac_impl_t *)mh;
943 943 mac_client_impl_t *mcip;
944 944
945 945 if (mip->mi_state_flags & MIS_IS_VNIC) {
946 946 /*
947 947 * This is a VNIC. Return the lower mac since that is what
948 948 * we want to serialize on.
949 949 */
950 950 mcip = mac_vnic_lower(mip);
951 951 mip = mcip->mci_mip;
952 952 }
953 953 return (mip->mi_perim_owner == curthread);
954 954 }
955 955
956 956 /*
957 957 * mac client interfaces to enter the mac perimeter of a mac end point, given
958 958 * its mac handle, or macname or linkid.
959 959 */
960 960 void
961 961 mac_perim_enter_by_mh(mac_handle_t mh, mac_perim_handle_t *mphp)
962 962 {
963 963 mac_impl_t *mip = (mac_impl_t *)mh;
964 964
965 965 i_mac_perim_enter(mip);
966 966 /*
967 967 * The mac_perim_handle_t returned encodes the 'mip' and whether a
968 968 * mac_open has been done internally while entering the perimeter.
969 969 * This information is used in mac_perim_exit
970 970 */
971 971 MAC_ENCODE_MPH(*mphp, mip, 0);
972 972 }
973 973
974 974 int
975 975 mac_perim_enter_by_macname(const char *name, mac_perim_handle_t *mphp)
976 976 {
977 977 int err;
978 978 mac_handle_t mh;
979 979
980 980 if ((err = mac_open(name, &mh)) != 0)
981 981 return (err);
982 982
983 983 mac_perim_enter_by_mh(mh, mphp);
984 984 MAC_ENCODE_MPH(*mphp, mh, 1);
985 985 return (0);
986 986 }
987 987
988 988 int
989 989 mac_perim_enter_by_linkid(datalink_id_t linkid, mac_perim_handle_t *mphp)
990 990 {
991 991 int err;
992 992 mac_handle_t mh;
993 993
994 994 if ((err = mac_open_by_linkid(linkid, &mh)) != 0)
995 995 return (err);
996 996
997 997 mac_perim_enter_by_mh(mh, mphp);
998 998 MAC_ENCODE_MPH(*mphp, mh, 1);
999 999 return (0);
1000 1000 }
1001 1001
1002 1002 void
1003 1003 mac_perim_exit(mac_perim_handle_t mph)
1004 1004 {
1005 1005 mac_impl_t *mip;
1006 1006 boolean_t need_close;
1007 1007
1008 1008 MAC_DECODE_MPH(mph, mip, need_close);
1009 1009 i_mac_perim_exit(mip);
1010 1010 if (need_close)
1011 1011 mac_close((mac_handle_t)mip);
1012 1012 }
1013 1013
1014 1014 int
1015 1015 mac_hold(const char *macname, mac_impl_t **pmip)
1016 1016 {
1017 1017 mac_impl_t *mip;
1018 1018 int err;
1019 1019
1020 1020 /*
1021 1021 * Check the device name length to make sure it won't overflow our
1022 1022 * buffer.
1023 1023 */
1024 1024 if (strlen(macname) >= MAXNAMELEN)
1025 1025 return (EINVAL);
1026 1026
1027 1027 /*
1028 1028 * Look up its entry in the global hash table.
1029 1029 */
1030 1030 rw_enter(&i_mac_impl_lock, RW_WRITER);
1031 1031 err = mod_hash_find(i_mac_impl_hash, (mod_hash_key_t)macname,
1032 1032 (mod_hash_val_t *)&mip);
1033 1033
1034 1034 if (err != 0) {
1035 1035 rw_exit(&i_mac_impl_lock);
1036 1036 return (ENOENT);
1037 1037 }
1038 1038
1039 1039 if (mip->mi_state_flags & MIS_DISABLED) {
1040 1040 rw_exit(&i_mac_impl_lock);
1041 1041 return (ENOENT);
1042 1042 }
1043 1043
1044 1044 if (mip->mi_state_flags & MIS_EXCLUSIVE_HELD) {
1045 1045 rw_exit(&i_mac_impl_lock);
1046 1046 return (EBUSY);
1047 1047 }
1048 1048
1049 1049 mip->mi_ref++;
1050 1050 rw_exit(&i_mac_impl_lock);
1051 1051
1052 1052 *pmip = mip;
1053 1053 return (0);
1054 1054 }
1055 1055
1056 1056 void
1057 1057 mac_rele(mac_impl_t *mip)
1058 1058 {
1059 1059 rw_enter(&i_mac_impl_lock, RW_WRITER);
1060 1060 ASSERT(mip->mi_ref != 0);
1061 1061 if (--mip->mi_ref == 0) {
1062 1062 ASSERT(mip->mi_nactiveclients == 0 &&
1063 1063 !(mip->mi_state_flags & MIS_EXCLUSIVE));
1064 1064 }
1065 1065 rw_exit(&i_mac_impl_lock);
1066 1066 }
1067 1067
1068 1068 /*
1069 1069 * Private GLDv3 function to start a MAC instance.
1070 1070 */
1071 1071 int
1072 1072 mac_start(mac_handle_t mh)
1073 1073 {
1074 1074 mac_impl_t *mip = (mac_impl_t *)mh;
1075 1075 int err = 0;
1076 1076 mac_group_t *defgrp;
1077 1077
1078 1078 ASSERT(MAC_PERIM_HELD((mac_handle_t)mip));
1079 1079 ASSERT(mip->mi_start != NULL);
1080 1080
1081 1081 /*
1082 1082 * Check whether the device is already started.
1083 1083 */
1084 1084 if (mip->mi_active++ == 0) {
1085 1085 mac_ring_t *ring = NULL;
1086 1086
1087 1087 /*
1088 1088 * Start the device.
1089 1089 */
1090 1090 err = mip->mi_start(mip->mi_driver);
1091 1091 if (err != 0) {
1092 1092 mip->mi_active--;
1093 1093 return (err);
1094 1094 }
1095 1095
1096 1096 /*
1097 1097 * Start the default tx ring.
1098 1098 */
1099 1099 if (mip->mi_default_tx_ring != NULL) {
1100 1100
1101 1101 ring = (mac_ring_t *)mip->mi_default_tx_ring;
1102 1102 if (ring->mr_state != MR_INUSE) {
1103 1103 err = mac_start_ring(ring);
1104 1104 if (err != 0) {
1105 1105 mip->mi_active--;
1106 1106 return (err);
1107 1107 }
1108 1108 }
1109 1109 }
1110 1110
1111 1111 if ((defgrp = MAC_DEFAULT_RX_GROUP(mip)) != NULL) {
1112 1112 /*
1113 1113 * Start the default ring, since it will be needed
1114 1114 * to receive broadcast and multicast traffic for
1115 1115 * both primary and non-primary MAC clients.
1116 1116 */
1117 1117 ASSERT(defgrp->mrg_state == MAC_GROUP_STATE_REGISTERED);
1118 1118 err = mac_start_group_and_rings(defgrp);
1119 1119 if (err != 0) {
1120 1120 mip->mi_active--;
1121 1121 if ((ring != NULL) &&
1122 1122 (ring->mr_state == MR_INUSE))
1123 1123 mac_stop_ring(ring);
1124 1124 return (err);
1125 1125 }
1126 1126 mac_set_group_state(defgrp, MAC_GROUP_STATE_SHARED);
1127 1127 }
1128 1128 }
1129 1129
1130 1130 return (err);
1131 1131 }
1132 1132
1133 1133 /*
1134 1134 * Private GLDv3 function to stop a MAC instance.
1135 1135 */
1136 1136 void
1137 1137 mac_stop(mac_handle_t mh)
1138 1138 {
1139 1139 mac_impl_t *mip = (mac_impl_t *)mh;
1140 1140 mac_group_t *grp;
1141 1141
1142 1142 ASSERT(mip->mi_stop != NULL);
1143 1143 ASSERT(MAC_PERIM_HELD((mac_handle_t)mip));
1144 1144
1145 1145 /*
1146 1146 * Check whether the device is still needed.
1147 1147 */
1148 1148 ASSERT(mip->mi_active != 0);
1149 1149 if (--mip->mi_active == 0) {
1150 1150 if ((grp = MAC_DEFAULT_RX_GROUP(mip)) != NULL) {
1151 1151 /*
1152 1152 * There should be no more active clients since the
1153 1153 * MAC is being stopped. Stop the default RX group
1154 1154 * and transition it back to registered state.
1155 1155 *
1156 1156 * When clients are torn down, the groups
1157 1157 * are release via mac_release_rx_group which
1158 1158 * knows the the default group is always in
1159 1159 * started mode since broadcast uses it. So
1160 1160 * we can assert that their are no clients
1161 1161 * (since mac_bcast_add doesn't register itself
1162 1162 * as a client) and group is in SHARED state.
1163 1163 */
1164 1164 ASSERT(grp->mrg_state == MAC_GROUP_STATE_SHARED);
1165 1165 ASSERT(MAC_GROUP_NO_CLIENT(grp) &&
1166 1166 mip->mi_nactiveclients == 0);
1167 1167 mac_stop_group_and_rings(grp);
1168 1168 mac_set_group_state(grp, MAC_GROUP_STATE_REGISTERED);
1169 1169 }
1170 1170
1171 1171 if (mip->mi_default_tx_ring != NULL) {
1172 1172 mac_ring_t *ring;
1173 1173
1174 1174 ring = (mac_ring_t *)mip->mi_default_tx_ring;
1175 1175 if (ring->mr_state == MR_INUSE) {
1176 1176 mac_stop_ring(ring);
1177 1177 ring->mr_flag = 0;
1178 1178 }
1179 1179 }
1180 1180
1181 1181 /*
1182 1182 * Stop the device.
1183 1183 */
1184 1184 mip->mi_stop(mip->mi_driver);
1185 1185 }
1186 1186 }
1187 1187
1188 1188 int
1189 1189 i_mac_promisc_set(mac_impl_t *mip, boolean_t on)
1190 1190 {
1191 1191 int err = 0;
1192 1192
1193 1193 ASSERT(MAC_PERIM_HELD((mac_handle_t)mip));
1194 1194 ASSERT(mip->mi_setpromisc != NULL);
1195 1195
1196 1196 if (on) {
1197 1197 /*
1198 1198 * Enable promiscuous mode on the device if not yet enabled.
1199 1199 */
1200 1200 if (mip->mi_devpromisc++ == 0) {
1201 1201 err = mip->mi_setpromisc(mip->mi_driver, B_TRUE);
1202 1202 if (err != 0) {
1203 1203 mip->mi_devpromisc--;
1204 1204 return (err);
1205 1205 }
1206 1206 i_mac_notify(mip, MAC_NOTE_DEVPROMISC);
1207 1207 }
1208 1208 } else {
1209 1209 if (mip->mi_devpromisc == 0)
1210 1210 return (EPROTO);
1211 1211
1212 1212 /*
1213 1213 * Disable promiscuous mode on the device if this is the last
1214 1214 * enabling.
1215 1215 */
1216 1216 if (--mip->mi_devpromisc == 0) {
1217 1217 err = mip->mi_setpromisc(mip->mi_driver, B_FALSE);
1218 1218 if (err != 0) {
1219 1219 mip->mi_devpromisc++;
1220 1220 return (err);
1221 1221 }
1222 1222 i_mac_notify(mip, MAC_NOTE_DEVPROMISC);
1223 1223 }
1224 1224 }
1225 1225
1226 1226 return (0);
1227 1227 }
1228 1228
1229 1229 /*
1230 1230 * The promiscuity state can change any time. If the caller needs to take
1231 1231 * actions that are atomic with the promiscuity state, then the caller needs
1232 1232 * to bracket the entire sequence with mac_perim_enter/exit
1233 1233 */
1234 1234 boolean_t
1235 1235 mac_promisc_get(mac_handle_t mh)
1236 1236 {
1237 1237 mac_impl_t *mip = (mac_impl_t *)mh;
1238 1238
1239 1239 /*
1240 1240 * Return the current promiscuity.
1241 1241 */
1242 1242 return (mip->mi_devpromisc != 0);
1243 1243 }
1244 1244
1245 1245 /*
1246 1246 * Invoked at MAC instance attach time to initialize the list
1247 1247 * of factory MAC addresses supported by a MAC instance. This function
1248 1248 * builds a local cache in the mac_impl_t for the MAC addresses
1249 1249 * supported by the underlying hardware. The MAC clients themselves
1250 1250 * use the mac_addr_factory*() functions to query and reserve
1251 1251 * factory MAC addresses.
1252 1252 */
1253 1253 void
1254 1254 mac_addr_factory_init(mac_impl_t *mip)
1255 1255 {
1256 1256 mac_capab_multifactaddr_t capab;
1257 1257 uint8_t *addr;
1258 1258 int i;
1259 1259
1260 1260 /*
1261 1261 * First round to see how many factory MAC addresses are available.
1262 1262 */
1263 1263 bzero(&capab, sizeof (capab));
1264 1264 if (!i_mac_capab_get((mac_handle_t)mip, MAC_CAPAB_MULTIFACTADDR,
1265 1265 &capab) || (capab.mcm_naddr == 0)) {
1266 1266 /*
1267 1267 * The MAC instance doesn't support multiple factory
1268 1268 * MAC addresses, we're done here.
1269 1269 */
1270 1270 return;
1271 1271 }
1272 1272
1273 1273 /*
1274 1274 * Allocate the space and get all the factory addresses.
1275 1275 */
1276 1276 addr = kmem_alloc(capab.mcm_naddr * MAXMACADDRLEN, KM_SLEEP);
1277 1277 capab.mcm_getaddr(mip->mi_driver, capab.mcm_naddr, addr);
1278 1278
1279 1279 mip->mi_factory_addr_num = capab.mcm_naddr;
1280 1280 mip->mi_factory_addr = kmem_zalloc(mip->mi_factory_addr_num *
1281 1281 sizeof (mac_factory_addr_t), KM_SLEEP);
1282 1282
1283 1283 for (i = 0; i < capab.mcm_naddr; i++) {
1284 1284 bcopy(addr + i * MAXMACADDRLEN,
1285 1285 mip->mi_factory_addr[i].mfa_addr,
1286 1286 mip->mi_type->mt_addr_length);
1287 1287 mip->mi_factory_addr[i].mfa_in_use = B_FALSE;
1288 1288 }
1289 1289
1290 1290 kmem_free(addr, capab.mcm_naddr * MAXMACADDRLEN);
1291 1291 }
1292 1292
1293 1293 void
1294 1294 mac_addr_factory_fini(mac_impl_t *mip)
1295 1295 {
1296 1296 if (mip->mi_factory_addr == NULL) {
1297 1297 ASSERT(mip->mi_factory_addr_num == 0);
1298 1298 return;
1299 1299 }
1300 1300
1301 1301 kmem_free(mip->mi_factory_addr, mip->mi_factory_addr_num *
1302 1302 sizeof (mac_factory_addr_t));
1303 1303
1304 1304 mip->mi_factory_addr = NULL;
1305 1305 mip->mi_factory_addr_num = 0;
1306 1306 }
1307 1307
1308 1308 /*
1309 1309 * Reserve a factory MAC address. If *slot is set to -1, the function
1310 1310 * attempts to reserve any of the available factory MAC addresses and
1311 1311 * returns the reserved slot id. If no slots are available, the function
1312 1312 * returns ENOSPC. If *slot is not set to -1, the function reserves
1313 1313 * the specified slot if it is available, or returns EBUSY is the slot
1314 1314 * is already used. Returns ENOTSUP if the underlying MAC does not
1315 1315 * support multiple factory addresses. If the slot number is not -1 but
1316 1316 * is invalid, returns EINVAL.
1317 1317 */
1318 1318 int
1319 1319 mac_addr_factory_reserve(mac_client_handle_t mch, int *slot)
1320 1320 {
1321 1321 mac_client_impl_t *mcip = (mac_client_impl_t *)mch;
1322 1322 mac_impl_t *mip = mcip->mci_mip;
1323 1323 int i, ret = 0;
1324 1324
1325 1325 i_mac_perim_enter(mip);
1326 1326 /*
1327 1327 * Protect against concurrent readers that may need a self-consistent
1328 1328 * view of the factory addresses
1329 1329 */
1330 1330 rw_enter(&mip->mi_rw_lock, RW_WRITER);
1331 1331
1332 1332 if (mip->mi_factory_addr_num == 0) {
1333 1333 ret = ENOTSUP;
1334 1334 goto bail;
1335 1335 }
1336 1336
1337 1337 if (*slot != -1) {
1338 1338 /* check the specified slot */
1339 1339 if (*slot < 1 || *slot > mip->mi_factory_addr_num) {
1340 1340 ret = EINVAL;
1341 1341 goto bail;
1342 1342 }
1343 1343 if (mip->mi_factory_addr[*slot-1].mfa_in_use) {
1344 1344 ret = EBUSY;
1345 1345 goto bail;
1346 1346 }
1347 1347 } else {
1348 1348 /* pick the next available slot */
1349 1349 for (i = 0; i < mip->mi_factory_addr_num; i++) {
1350 1350 if (!mip->mi_factory_addr[i].mfa_in_use)
1351 1351 break;
1352 1352 }
1353 1353
1354 1354 if (i == mip->mi_factory_addr_num) {
1355 1355 ret = ENOSPC;
1356 1356 goto bail;
1357 1357 }
1358 1358 *slot = i+1;
1359 1359 }
1360 1360
1361 1361 mip->mi_factory_addr[*slot-1].mfa_in_use = B_TRUE;
1362 1362 mip->mi_factory_addr[*slot-1].mfa_client = mcip;
1363 1363
1364 1364 bail:
1365 1365 rw_exit(&mip->mi_rw_lock);
1366 1366 i_mac_perim_exit(mip);
1367 1367 return (ret);
1368 1368 }
1369 1369
1370 1370 /*
1371 1371 * Release the specified factory MAC address slot.
1372 1372 */
1373 1373 void
1374 1374 mac_addr_factory_release(mac_client_handle_t mch, uint_t slot)
1375 1375 {
1376 1376 mac_client_impl_t *mcip = (mac_client_impl_t *)mch;
1377 1377 mac_impl_t *mip = mcip->mci_mip;
1378 1378
1379 1379 i_mac_perim_enter(mip);
1380 1380 /*
1381 1381 * Protect against concurrent readers that may need a self-consistent
1382 1382 * view of the factory addresses
1383 1383 */
1384 1384 rw_enter(&mip->mi_rw_lock, RW_WRITER);
1385 1385
1386 1386 ASSERT(slot > 0 && slot <= mip->mi_factory_addr_num);
1387 1387 ASSERT(mip->mi_factory_addr[slot-1].mfa_in_use);
1388 1388
1389 1389 mip->mi_factory_addr[slot-1].mfa_in_use = B_FALSE;
1390 1390
1391 1391 rw_exit(&mip->mi_rw_lock);
1392 1392 i_mac_perim_exit(mip);
1393 1393 }
1394 1394
1395 1395 /*
1396 1396 * Stores in mac_addr the value of the specified MAC address. Returns
1397 1397 * 0 on success, or EINVAL if the slot number is not valid for the MAC.
1398 1398 * The caller must provide a string of at least MAXNAMELEN bytes.
1399 1399 */
1400 1400 void
1401 1401 mac_addr_factory_value(mac_handle_t mh, int slot, uchar_t *mac_addr,
1402 1402 uint_t *addr_len, char *client_name, boolean_t *in_use_arg)
1403 1403 {
1404 1404 mac_impl_t *mip = (mac_impl_t *)mh;
1405 1405 boolean_t in_use;
1406 1406
1407 1407 ASSERT(slot > 0 && slot <= mip->mi_factory_addr_num);
1408 1408
1409 1409 /*
1410 1410 * Readers need to hold mi_rw_lock. Writers need to hold mac perimeter
1411 1411 * and mi_rw_lock
1412 1412 */
1413 1413 rw_enter(&mip->mi_rw_lock, RW_READER);
1414 1414 bcopy(mip->mi_factory_addr[slot-1].mfa_addr, mac_addr, MAXMACADDRLEN);
1415 1415 *addr_len = mip->mi_type->mt_addr_length;
1416 1416 in_use = mip->mi_factory_addr[slot-1].mfa_in_use;
1417 1417 if (in_use && client_name != NULL) {
1418 1418 bcopy(mip->mi_factory_addr[slot-1].mfa_client->mci_name,
1419 1419 client_name, MAXNAMELEN);
1420 1420 }
1421 1421 if (in_use_arg != NULL)
1422 1422 *in_use_arg = in_use;
1423 1423 rw_exit(&mip->mi_rw_lock);
1424 1424 }
1425 1425
1426 1426 /*
1427 1427 * Returns the number of factory MAC addresses (in addition to the
1428 1428 * primary MAC address), 0 if the underlying MAC doesn't support
1429 1429 * that feature.
1430 1430 */
1431 1431 uint_t
1432 1432 mac_addr_factory_num(mac_handle_t mh)
1433 1433 {
1434 1434 mac_impl_t *mip = (mac_impl_t *)mh;
1435 1435
1436 1436 return (mip->mi_factory_addr_num);
1437 1437 }
1438 1438
1439 1439
1440 1440 void
1441 1441 mac_rx_group_unmark(mac_group_t *grp, uint_t flag)
1442 1442 {
1443 1443 mac_ring_t *ring;
1444 1444
1445 1445 for (ring = grp->mrg_rings; ring != NULL; ring = ring->mr_next)
1446 1446 ring->mr_flag &= ~flag;
1447 1447 }
1448 1448
1449 1449 /*
1450 1450 * The following mac_hwrings_xxx() functions are private mac client functions
1451 1451 * used by the aggr driver to access and control the underlying HW Rx group
1452 1452 * and rings. In this case, the aggr driver has exclusive control of the
1453 1453 * underlying HW Rx group/rings, it calls the following functions to
1454 1454 * start/stop the HW Rx rings, disable/enable polling, add/remove mac'
1455 1455 * addresses, or set up the Rx callback.
1456 1456 */
1457 1457 /* ARGSUSED */
1458 1458 static void
1459 1459 mac_hwrings_rx_process(void *arg, mac_resource_handle_t srs,
1460 1460 mblk_t *mp_chain, boolean_t loopback)
1461 1461 {
1462 1462 mac_soft_ring_set_t *mac_srs = (mac_soft_ring_set_t *)srs;
1463 1463 mac_srs_rx_t *srs_rx = &mac_srs->srs_rx;
1464 1464 mac_direct_rx_t proc;
1465 1465 void *arg1;
1466 1466 mac_resource_handle_t arg2;
1467 1467
1468 1468 proc = srs_rx->sr_func;
1469 1469 arg1 = srs_rx->sr_arg1;
1470 1470 arg2 = mac_srs->srs_mrh;
1471 1471
1472 1472 proc(arg1, arg2, mp_chain, NULL);
1473 1473 }
1474 1474
1475 1475 /*
1476 1476 * This function is called to get the list of HW rings that are reserved by
1477 1477 * an exclusive mac client.
1478 1478 *
1479 1479 * Return value: the number of HW rings.
1480 1480 */
1481 1481 int
1482 1482 mac_hwrings_get(mac_client_handle_t mch, mac_group_handle_t *hwgh,
1483 1483 mac_ring_handle_t *hwrh, mac_ring_type_t rtype)
1484 1484 {
1485 1485 mac_client_impl_t *mcip = (mac_client_impl_t *)mch;
1486 1486 flow_entry_t *flent = mcip->mci_flent;
1487 1487 mac_group_t *grp;
1488 1488 mac_ring_t *ring;
1489 1489 int cnt = 0;
1490 1490
1491 1491 if (rtype == MAC_RING_TYPE_RX) {
1492 1492 grp = flent->fe_rx_ring_group;
1493 1493 } else if (rtype == MAC_RING_TYPE_TX) {
1494 1494 grp = flent->fe_tx_ring_group;
1495 1495 } else {
1496 1496 ASSERT(B_FALSE);
1497 1497 return (-1);
1498 1498 }
1499 1499 /*
1500 1500 * The mac client did not reserve any RX group, return directly.
1501 1501 * This is probably because the underlying MAC does not support
1502 1502 * any groups.
1503 1503 */
1504 1504 if (hwgh != NULL)
1505 1505 *hwgh = NULL;
1506 1506 if (grp == NULL)
1507 1507 return (0);
1508 1508 /*
1509 1509 * This group must be reserved by this mac client.
1510 1510 */
1511 1511 ASSERT((grp->mrg_state == MAC_GROUP_STATE_RESERVED) &&
1512 1512 (mcip == MAC_GROUP_ONLY_CLIENT(grp)));
1513 1513
1514 1514 for (ring = grp->mrg_rings; ring != NULL; ring = ring->mr_next, cnt++) {
1515 1515 ASSERT(cnt < MAX_RINGS_PER_GROUP);
1516 1516 hwrh[cnt] = (mac_ring_handle_t)ring;
1517 1517 }
1518 1518 if (hwgh != NULL)
1519 1519 *hwgh = (mac_group_handle_t)grp;
1520 1520
1521 1521 return (cnt);
1522 1522 }
1523 1523
1524 1524 /*
1525 1525 * This function is called to get info about Tx/Rx rings.
1526 1526 *
1527 1527 * Return value: returns uint_t which will have various bits set
1528 1528 * that indicates different properties of the ring.
1529 1529 */
1530 1530 uint_t
1531 1531 mac_hwring_getinfo(mac_ring_handle_t rh)
1532 1532 {
1533 1533 mac_ring_t *ring = (mac_ring_t *)rh;
1534 1534 mac_ring_info_t *info = &ring->mr_info;
1535 1535
1536 1536 return (info->mri_flags);
1537 1537 }
1538 1538
1539 1539 /*
1540 1540 * Export ddi interrupt handles from the HW ring to the pseudo ring and
1541 1541 * setup the RX callback of the mac client which exclusively controls
1542 1542 * HW ring.
1543 1543 */
1544 1544 void
1545 1545 mac_hwring_setup(mac_ring_handle_t hwrh, mac_resource_handle_t prh,
1546 1546 mac_ring_handle_t pseudo_rh)
1547 1547 {
1548 1548 mac_ring_t *hw_ring = (mac_ring_t *)hwrh;
1549 1549 mac_ring_t *pseudo_ring;
1550 1550 mac_soft_ring_set_t *mac_srs = hw_ring->mr_srs;
1551 1551
1552 1552 if (pseudo_rh != NULL) {
1553 1553 pseudo_ring = (mac_ring_t *)pseudo_rh;
1554 1554 /* Export the ddi handles to pseudo ring */
1555 1555 pseudo_ring->mr_info.mri_intr.mi_ddi_handle =
1556 1556 hw_ring->mr_info.mri_intr.mi_ddi_handle;
1557 1557 pseudo_ring->mr_info.mri_intr.mi_ddi_shared =
1558 1558 hw_ring->mr_info.mri_intr.mi_ddi_shared;
1559 1559 /*
1560 1560 * Save a pointer to pseudo ring in the hw ring. If
1561 1561 * interrupt handle changes, the hw ring will be
1562 1562 * notified of the change (see mac_ring_intr_set())
1563 1563 * and the appropriate change has to be made to
1564 1564 * the pseudo ring that has exported the ddi handle.
1565 1565 */
1566 1566 hw_ring->mr_prh = pseudo_rh;
1567 1567 }
1568 1568
1569 1569 if (hw_ring->mr_type == MAC_RING_TYPE_RX) {
1570 1570 ASSERT(!(mac_srs->srs_type & SRST_TX));
1571 1571 mac_srs->srs_mrh = prh;
1572 1572 mac_srs->srs_rx.sr_lower_proc = mac_hwrings_rx_process;
1573 1573 }
1574 1574 }
1575 1575
1576 1576 void
1577 1577 mac_hwring_teardown(mac_ring_handle_t hwrh)
1578 1578 {
1579 1579 mac_ring_t *hw_ring = (mac_ring_t *)hwrh;
1580 1580 mac_soft_ring_set_t *mac_srs;
1581 1581
1582 1582 if (hw_ring == NULL)
1583 1583 return;
1584 1584 hw_ring->mr_prh = NULL;
1585 1585 if (hw_ring->mr_type == MAC_RING_TYPE_RX) {
1586 1586 mac_srs = hw_ring->mr_srs;
1587 1587 ASSERT(!(mac_srs->srs_type & SRST_TX));
1588 1588 mac_srs->srs_rx.sr_lower_proc = mac_rx_srs_process;
1589 1589 mac_srs->srs_mrh = NULL;
1590 1590 }
1591 1591 }
1592 1592
1593 1593 int
1594 1594 mac_hwring_disable_intr(mac_ring_handle_t rh)
1595 1595 {
1596 1596 mac_ring_t *rr_ring = (mac_ring_t *)rh;
1597 1597 mac_intr_t *intr = &rr_ring->mr_info.mri_intr;
1598 1598
1599 1599 return (intr->mi_disable(intr->mi_handle));
1600 1600 }
1601 1601
1602 1602 int
1603 1603 mac_hwring_enable_intr(mac_ring_handle_t rh)
1604 1604 {
1605 1605 mac_ring_t *rr_ring = (mac_ring_t *)rh;
1606 1606 mac_intr_t *intr = &rr_ring->mr_info.mri_intr;
1607 1607
1608 1608 return (intr->mi_enable(intr->mi_handle));
1609 1609 }
1610 1610
1611 1611 int
1612 1612 mac_hwring_start(mac_ring_handle_t rh)
1613 1613 {
1614 1614 mac_ring_t *rr_ring = (mac_ring_t *)rh;
1615 1615
1616 1616 MAC_RING_UNMARK(rr_ring, MR_QUIESCE);
1617 1617 return (0);
1618 1618 }
1619 1619
1620 1620 void
1621 1621 mac_hwring_stop(mac_ring_handle_t rh)
1622 1622 {
1623 1623 mac_ring_t *rr_ring = (mac_ring_t *)rh;
1624 1624
1625 1625 mac_rx_ring_quiesce(rr_ring, MR_QUIESCE);
1626 1626 }
1627 1627
1628 1628 mblk_t *
1629 1629 mac_hwring_poll(mac_ring_handle_t rh, int bytes_to_pickup)
1630 1630 {
1631 1631 mac_ring_t *rr_ring = (mac_ring_t *)rh;
1632 1632 mac_ring_info_t *info = &rr_ring->mr_info;
1633 1633
1634 1634 return (info->mri_poll(info->mri_driver, bytes_to_pickup));
1635 1635 }
1636 1636
1637 1637 /*
1638 1638 * Send packets through a selected tx ring.
1639 1639 */
1640 1640 mblk_t *
1641 1641 mac_hwring_tx(mac_ring_handle_t rh, mblk_t *mp)
1642 1642 {
1643 1643 mac_ring_t *ring = (mac_ring_t *)rh;
1644 1644 mac_ring_info_t *info = &ring->mr_info;
1645 1645
1646 1646 ASSERT(ring->mr_type == MAC_RING_TYPE_TX &&
1647 1647 ring->mr_state >= MR_INUSE);
1648 1648 return (info->mri_tx(info->mri_driver, mp));
1649 1649 }
1650 1650
1651 1651 /*
1652 1652 * Query stats for a particular rx/tx ring
1653 1653 */
1654 1654 int
1655 1655 mac_hwring_getstat(mac_ring_handle_t rh, uint_t stat, uint64_t *val)
1656 1656 {
1657 1657 mac_ring_t *ring = (mac_ring_t *)rh;
1658 1658 mac_ring_info_t *info = &ring->mr_info;
1659 1659
1660 1660 return (info->mri_stat(info->mri_driver, stat, val));
1661 1661 }
1662 1662
1663 1663 /*
1664 1664 * Private function that is only used by aggr to send packets through
1665 1665 * a port/Tx ring. Since aggr exposes a pseudo Tx ring even for ports
1666 1666 * that does not expose Tx rings, aggr_ring_tx() entry point needs
1667 1667 * access to mac_impl_t to send packets through m_tx() entry point.
1668 1668 * It accomplishes this by calling mac_hwring_send_priv() function.
1669 1669 */
|
↓ open down ↓ |
1635 lines elided |
↑ open up ↑ |
1670 1670 mblk_t *
1671 1671 mac_hwring_send_priv(mac_client_handle_t mch, mac_ring_handle_t rh, mblk_t *mp)
1672 1672 {
1673 1673 mac_client_impl_t *mcip = (mac_client_impl_t *)mch;
1674 1674 mac_impl_t *mip = mcip->mci_mip;
1675 1675
1676 1676 MAC_TX(mip, rh, mp, mcip);
1677 1677 return (mp);
1678 1678 }
1679 1679
1680 +/*
1681 + * Private function that is only used by aggr to update the default transmission
1682 + * ring. Because aggr exposes a pseudo Tx ring even for ports that may
1683 + * temporarily be down, it may need to update the default ring that is used by
1684 + * MAC such that it refers to a link that can actively be used to send traffic.
1685 + * Note that this is different from the case where the port has been removed
1686 + * from the group. In those cases, all of the rings will be torn down because
1687 + * the ring will no longer exist. It's important to give aggr a case where the
1688 + * rings can still exist such that it may be able to continue to send LACP PDUs
1689 + * to potentially restore the link.
1690 + *
1691 + * Finally, we explicitly don't do anything if the ring hasn't been enabled yet.
1692 + * This is to help out aggr which doesn't really know the internal state that
1693 + * MAC does about the rings and can't know that it's not quite ready for use
1694 + * yet.
1695 + */
1696 +void
1697 +mac_hwring_set_default(mac_handle_t mh, mac_ring_handle_t rh)
1698 +{
1699 + mac_impl_t *mip = (mac_impl_t *)mh;
1700 + mac_ring_t *ring = (mac_ring_t *)rh;
1701 +
1702 + ASSERT(MAC_PERIM_HELD(mh));
1703 + VERIFY(mip->mi_state_flags & MIS_IS_AGGR);
1704 +
1705 + if (ring->mr_state != MR_INUSE)
1706 + return;
1707 +
1708 + mip->mi_default_tx_ring = rh;
1709 +}
1710 +
1680 1711 int
1681 1712 mac_hwgroup_addmac(mac_group_handle_t gh, const uint8_t *addr)
1682 1713 {
1683 1714 mac_group_t *group = (mac_group_t *)gh;
1684 1715
1685 1716 return (mac_group_addmac(group, addr));
1686 1717 }
1687 1718
1688 1719 int
1689 1720 mac_hwgroup_remmac(mac_group_handle_t gh, const uint8_t *addr)
1690 1721 {
1691 1722 mac_group_t *group = (mac_group_t *)gh;
1692 1723
1693 1724 return (mac_group_remmac(group, addr));
1694 1725 }
1695 1726
1696 1727 /*
1697 1728 * Set the RX group to be shared/reserved. Note that the group must be
1698 1729 * started/stopped outside of this function.
1699 1730 */
1700 1731 void
1701 1732 mac_set_group_state(mac_group_t *grp, mac_group_state_t state)
1702 1733 {
1703 1734 /*
1704 1735 * If there is no change in the group state, just return.
1705 1736 */
1706 1737 if (grp->mrg_state == state)
1707 1738 return;
1708 1739
1709 1740 switch (state) {
1710 1741 case MAC_GROUP_STATE_RESERVED:
1711 1742 /*
1712 1743 * Successfully reserved the group.
1713 1744 *
1714 1745 * Given that there is an exclusive client controlling this
1715 1746 * group, we enable the group level polling when available,
1716 1747 * so that SRSs get to turn on/off individual rings they's
1717 1748 * assigned to.
1718 1749 */
1719 1750 ASSERT(MAC_PERIM_HELD(grp->mrg_mh));
1720 1751
1721 1752 if (grp->mrg_type == MAC_RING_TYPE_RX &&
1722 1753 GROUP_INTR_DISABLE_FUNC(grp) != NULL) {
1723 1754 GROUP_INTR_DISABLE_FUNC(grp)(GROUP_INTR_HANDLE(grp));
1724 1755 }
1725 1756 break;
1726 1757
1727 1758 case MAC_GROUP_STATE_SHARED:
1728 1759 /*
1729 1760 * Set all rings of this group to software classified.
1730 1761 * If the group has an overriding interrupt, then re-enable it.
1731 1762 */
1732 1763 ASSERT(MAC_PERIM_HELD(grp->mrg_mh));
1733 1764
1734 1765 if (grp->mrg_type == MAC_RING_TYPE_RX &&
1735 1766 GROUP_INTR_ENABLE_FUNC(grp) != NULL) {
1736 1767 GROUP_INTR_ENABLE_FUNC(grp)(GROUP_INTR_HANDLE(grp));
1737 1768 }
1738 1769 /* The ring is not available for reservations any more */
1739 1770 break;
1740 1771
1741 1772 case MAC_GROUP_STATE_REGISTERED:
1742 1773 /* Also callable from mac_register, perim is not held */
1743 1774 break;
1744 1775
1745 1776 default:
1746 1777 ASSERT(B_FALSE);
1747 1778 break;
1748 1779 }
1749 1780
1750 1781 grp->mrg_state = state;
1751 1782 }
1752 1783
1753 1784 /*
1754 1785 * Quiesce future hardware classified packets for the specified Rx ring
1755 1786 */
1756 1787 static void
1757 1788 mac_rx_ring_quiesce(mac_ring_t *rx_ring, uint_t ring_flag)
1758 1789 {
1759 1790 ASSERT(rx_ring->mr_classify_type == MAC_HW_CLASSIFIER);
1760 1791 ASSERT(ring_flag == MR_CONDEMNED || ring_flag == MR_QUIESCE);
1761 1792
1762 1793 mutex_enter(&rx_ring->mr_lock);
1763 1794 rx_ring->mr_flag |= ring_flag;
1764 1795 while (rx_ring->mr_refcnt != 0)
1765 1796 cv_wait(&rx_ring->mr_cv, &rx_ring->mr_lock);
1766 1797 mutex_exit(&rx_ring->mr_lock);
1767 1798 }
1768 1799
1769 1800 /*
1770 1801 * Please see mac_tx for details about the per cpu locking scheme
1771 1802 */
1772 1803 static void
1773 1804 mac_tx_lock_all(mac_client_impl_t *mcip)
1774 1805 {
1775 1806 int i;
1776 1807
1777 1808 for (i = 0; i <= mac_tx_percpu_cnt; i++)
1778 1809 mutex_enter(&mcip->mci_tx_pcpu[i].pcpu_tx_lock);
1779 1810 }
1780 1811
1781 1812 static void
1782 1813 mac_tx_unlock_all(mac_client_impl_t *mcip)
1783 1814 {
1784 1815 int i;
1785 1816
1786 1817 for (i = mac_tx_percpu_cnt; i >= 0; i--)
1787 1818 mutex_exit(&mcip->mci_tx_pcpu[i].pcpu_tx_lock);
1788 1819 }
1789 1820
1790 1821 static void
1791 1822 mac_tx_unlock_allbutzero(mac_client_impl_t *mcip)
1792 1823 {
1793 1824 int i;
1794 1825
1795 1826 for (i = mac_tx_percpu_cnt; i > 0; i--)
1796 1827 mutex_exit(&mcip->mci_tx_pcpu[i].pcpu_tx_lock);
1797 1828 }
1798 1829
1799 1830 static int
1800 1831 mac_tx_sum_refcnt(mac_client_impl_t *mcip)
1801 1832 {
1802 1833 int i;
1803 1834 int refcnt = 0;
1804 1835
1805 1836 for (i = 0; i <= mac_tx_percpu_cnt; i++)
1806 1837 refcnt += mcip->mci_tx_pcpu[i].pcpu_tx_refcnt;
1807 1838
1808 1839 return (refcnt);
1809 1840 }
1810 1841
1811 1842 /*
1812 1843 * Stop future Tx packets coming down from the client in preparation for
1813 1844 * quiescing the Tx side. This is needed for dynamic reclaim and reassignment
1814 1845 * of rings between clients
1815 1846 */
1816 1847 void
1817 1848 mac_tx_client_block(mac_client_impl_t *mcip)
1818 1849 {
1819 1850 mac_tx_lock_all(mcip);
1820 1851 mcip->mci_tx_flag |= MCI_TX_QUIESCE;
1821 1852 while (mac_tx_sum_refcnt(mcip) != 0) {
1822 1853 mac_tx_unlock_allbutzero(mcip);
1823 1854 cv_wait(&mcip->mci_tx_cv, &mcip->mci_tx_pcpu[0].pcpu_tx_lock);
1824 1855 mutex_exit(&mcip->mci_tx_pcpu[0].pcpu_tx_lock);
1825 1856 mac_tx_lock_all(mcip);
1826 1857 }
1827 1858 mac_tx_unlock_all(mcip);
1828 1859 }
1829 1860
1830 1861 void
1831 1862 mac_tx_client_unblock(mac_client_impl_t *mcip)
1832 1863 {
1833 1864 mac_tx_lock_all(mcip);
1834 1865 mcip->mci_tx_flag &= ~MCI_TX_QUIESCE;
1835 1866 mac_tx_unlock_all(mcip);
1836 1867 /*
1837 1868 * We may fail to disable flow control for the last MAC_NOTE_TX
1838 1869 * notification because the MAC client is quiesced. Send the
1839 1870 * notification again.
1840 1871 */
1841 1872 i_mac_notify(mcip->mci_mip, MAC_NOTE_TX);
1842 1873 }
1843 1874
1844 1875 /*
1845 1876 * Wait for an SRS to quiesce. The SRS worker will signal us when the
1846 1877 * quiesce is done.
1847 1878 */
1848 1879 static void
1849 1880 mac_srs_quiesce_wait(mac_soft_ring_set_t *srs, uint_t srs_flag)
1850 1881 {
1851 1882 mutex_enter(&srs->srs_lock);
1852 1883 while (!(srs->srs_state & srs_flag))
1853 1884 cv_wait(&srs->srs_quiesce_done_cv, &srs->srs_lock);
1854 1885 mutex_exit(&srs->srs_lock);
1855 1886 }
1856 1887
1857 1888 /*
1858 1889 * Quiescing an Rx SRS is achieved by the following sequence. The protocol
1859 1890 * works bottom up by cutting off packet flow from the bottommost point in the
1860 1891 * mac, then the SRS, and then the soft rings. There are 2 use cases of this
1861 1892 * mechanism. One is a temporary quiesce of the SRS, such as say while changing
1862 1893 * the Rx callbacks. Another use case is Rx SRS teardown. In the former case
1863 1894 * the QUIESCE prefix/suffix is used and in the latter the CONDEMNED is used
1864 1895 * for the SRS and MR flags. In the former case the threads pause waiting for
1865 1896 * a restart, while in the latter case the threads exit. The Tx SRS teardown
1866 1897 * is also mostly similar to the above.
1867 1898 *
1868 1899 * 1. Stop future hardware classified packets at the lowest level in the mac.
1869 1900 * Remove any hardware classification rule (CONDEMNED case) and mark the
1870 1901 * rings as CONDEMNED or QUIESCE as appropriate. This prevents the mr_refcnt
1871 1902 * from increasing. Upcalls from the driver that come through hardware
1872 1903 * classification will be dropped in mac_rx from now on. Then we wait for
1873 1904 * the mr_refcnt to drop to zero. When the mr_refcnt reaches zero we are
1874 1905 * sure there aren't any upcall threads from the driver through hardware
1875 1906 * classification. In the case of SRS teardown we also remove the
1876 1907 * classification rule in the driver.
1877 1908 *
1878 1909 * 2. Stop future software classified packets by marking the flow entry with
1879 1910 * FE_QUIESCE or FE_CONDEMNED as appropriate which prevents the refcnt from
1880 1911 * increasing. We also remove the flow entry from the table in the latter
1881 1912 * case. Then wait for the fe_refcnt to reach an appropriate quiescent value
1882 1913 * that indicates there aren't any active threads using that flow entry.
1883 1914 *
1884 1915 * 3. Quiesce the SRS and softrings by signaling the SRS. The SRS poll thread,
1885 1916 * SRS worker thread, and the soft ring threads are quiesced in sequence
1886 1917 * with the SRS worker thread serving as a master controller. This
1887 1918 * mechansim is explained in mac_srs_worker_quiesce().
1888 1919 *
1889 1920 * The restart mechanism to reactivate the SRS and softrings is explained
1890 1921 * in mac_srs_worker_restart(). Here we just signal the SRS worker to start the
1891 1922 * restart sequence.
1892 1923 */
1893 1924 void
1894 1925 mac_rx_srs_quiesce(mac_soft_ring_set_t *srs, uint_t srs_quiesce_flag)
1895 1926 {
1896 1927 flow_entry_t *flent = srs->srs_flent;
1897 1928 uint_t mr_flag, srs_done_flag;
1898 1929
1899 1930 ASSERT(MAC_PERIM_HELD((mac_handle_t)FLENT_TO_MIP(flent)));
1900 1931 ASSERT(!(srs->srs_type & SRST_TX));
1901 1932
1902 1933 if (srs_quiesce_flag == SRS_CONDEMNED) {
1903 1934 mr_flag = MR_CONDEMNED;
1904 1935 srs_done_flag = SRS_CONDEMNED_DONE;
1905 1936 if (srs->srs_type & SRST_CLIENT_POLL_ENABLED)
1906 1937 mac_srs_client_poll_disable(srs->srs_mcip, srs);
1907 1938 } else {
1908 1939 ASSERT(srs_quiesce_flag == SRS_QUIESCE);
1909 1940 mr_flag = MR_QUIESCE;
1910 1941 srs_done_flag = SRS_QUIESCE_DONE;
1911 1942 if (srs->srs_type & SRST_CLIENT_POLL_ENABLED)
1912 1943 mac_srs_client_poll_quiesce(srs->srs_mcip, srs);
1913 1944 }
1914 1945
1915 1946 if (srs->srs_ring != NULL) {
1916 1947 mac_rx_ring_quiesce(srs->srs_ring, mr_flag);
1917 1948 } else {
1918 1949 /*
1919 1950 * SRS is driven by software classification. In case
1920 1951 * of CONDEMNED, the top level teardown functions will
1921 1952 * deal with flow removal.
1922 1953 */
1923 1954 if (srs_quiesce_flag != SRS_CONDEMNED) {
1924 1955 FLOW_MARK(flent, FE_QUIESCE);
1925 1956 mac_flow_wait(flent, FLOW_DRIVER_UPCALL);
1926 1957 }
1927 1958 }
1928 1959
1929 1960 /*
1930 1961 * Signal the SRS to quiesce itself, and then cv_wait for the
1931 1962 * SRS quiesce to complete. The SRS worker thread will wake us
1932 1963 * up when the quiesce is complete
1933 1964 */
1934 1965 mac_srs_signal(srs, srs_quiesce_flag);
1935 1966 mac_srs_quiesce_wait(srs, srs_done_flag);
1936 1967 }
1937 1968
1938 1969 /*
1939 1970 * Remove an SRS.
1940 1971 */
1941 1972 void
1942 1973 mac_rx_srs_remove(mac_soft_ring_set_t *srs)
1943 1974 {
1944 1975 flow_entry_t *flent = srs->srs_flent;
1945 1976 int i;
1946 1977
1947 1978 mac_rx_srs_quiesce(srs, SRS_CONDEMNED);
1948 1979 /*
1949 1980 * Locate and remove our entry in the fe_rx_srs[] array, and
1950 1981 * adjust the fe_rx_srs array entries and array count by
1951 1982 * moving the last entry into the vacated spot.
1952 1983 */
1953 1984 mutex_enter(&flent->fe_lock);
1954 1985 for (i = 0; i < flent->fe_rx_srs_cnt; i++) {
1955 1986 if (flent->fe_rx_srs[i] == srs)
1956 1987 break;
1957 1988 }
1958 1989
1959 1990 ASSERT(i != 0 && i < flent->fe_rx_srs_cnt);
1960 1991 if (i != flent->fe_rx_srs_cnt - 1) {
1961 1992 flent->fe_rx_srs[i] =
1962 1993 flent->fe_rx_srs[flent->fe_rx_srs_cnt - 1];
1963 1994 i = flent->fe_rx_srs_cnt - 1;
1964 1995 }
1965 1996
1966 1997 flent->fe_rx_srs[i] = NULL;
1967 1998 flent->fe_rx_srs_cnt--;
1968 1999 mutex_exit(&flent->fe_lock);
1969 2000
1970 2001 mac_srs_free(srs);
1971 2002 }
1972 2003
1973 2004 static void
1974 2005 mac_srs_clear_flag(mac_soft_ring_set_t *srs, uint_t flag)
1975 2006 {
1976 2007 mutex_enter(&srs->srs_lock);
1977 2008 srs->srs_state &= ~flag;
1978 2009 mutex_exit(&srs->srs_lock);
1979 2010 }
1980 2011
1981 2012 void
1982 2013 mac_rx_srs_restart(mac_soft_ring_set_t *srs)
1983 2014 {
1984 2015 flow_entry_t *flent = srs->srs_flent;
1985 2016 mac_ring_t *mr;
1986 2017
1987 2018 ASSERT(MAC_PERIM_HELD((mac_handle_t)FLENT_TO_MIP(flent)));
1988 2019 ASSERT((srs->srs_type & SRST_TX) == 0);
1989 2020
1990 2021 /*
1991 2022 * This handles a change in the number of SRSs between the quiesce and
1992 2023 * and restart operation of a flow.
1993 2024 */
1994 2025 if (!SRS_QUIESCED(srs))
1995 2026 return;
1996 2027
1997 2028 /*
1998 2029 * Signal the SRS to restart itself. Wait for the restart to complete
1999 2030 * Note that we only restart the SRS if it is not marked as
2000 2031 * permanently quiesced.
2001 2032 */
2002 2033 if (!SRS_QUIESCED_PERMANENT(srs)) {
2003 2034 mac_srs_signal(srs, SRS_RESTART);
2004 2035 mac_srs_quiesce_wait(srs, SRS_RESTART_DONE);
2005 2036 mac_srs_clear_flag(srs, SRS_RESTART_DONE);
2006 2037
2007 2038 mac_srs_client_poll_restart(srs->srs_mcip, srs);
2008 2039 }
2009 2040
2010 2041 /* Finally clear the flags to let the packets in */
2011 2042 mr = srs->srs_ring;
2012 2043 if (mr != NULL) {
2013 2044 MAC_RING_UNMARK(mr, MR_QUIESCE);
2014 2045 /* In case the ring was stopped, safely restart it */
2015 2046 if (mr->mr_state != MR_INUSE)
2016 2047 (void) mac_start_ring(mr);
2017 2048 } else {
2018 2049 FLOW_UNMARK(flent, FE_QUIESCE);
2019 2050 }
2020 2051 }
2021 2052
2022 2053 /*
2023 2054 * Temporary quiesce of a flow and associated Rx SRS.
2024 2055 * Please see block comment above mac_rx_classify_flow_rem.
2025 2056 */
2026 2057 /* ARGSUSED */
2027 2058 int
2028 2059 mac_rx_classify_flow_quiesce(flow_entry_t *flent, void *arg)
2029 2060 {
2030 2061 int i;
2031 2062
2032 2063 for (i = 0; i < flent->fe_rx_srs_cnt; i++) {
2033 2064 mac_rx_srs_quiesce((mac_soft_ring_set_t *)flent->fe_rx_srs[i],
2034 2065 SRS_QUIESCE);
2035 2066 }
2036 2067 return (0);
2037 2068 }
2038 2069
2039 2070 /*
2040 2071 * Restart a flow and associated Rx SRS that has been quiesced temporarily
2041 2072 * Please see block comment above mac_rx_classify_flow_rem
2042 2073 */
2043 2074 /* ARGSUSED */
2044 2075 int
2045 2076 mac_rx_classify_flow_restart(flow_entry_t *flent, void *arg)
2046 2077 {
2047 2078 int i;
2048 2079
2049 2080 for (i = 0; i < flent->fe_rx_srs_cnt; i++)
2050 2081 mac_rx_srs_restart((mac_soft_ring_set_t *)flent->fe_rx_srs[i]);
2051 2082
2052 2083 return (0);
2053 2084 }
2054 2085
2055 2086 void
2056 2087 mac_srs_perm_quiesce(mac_client_handle_t mch, boolean_t on)
2057 2088 {
2058 2089 mac_client_impl_t *mcip = (mac_client_impl_t *)mch;
2059 2090 flow_entry_t *flent = mcip->mci_flent;
2060 2091 mac_impl_t *mip = mcip->mci_mip;
2061 2092 mac_soft_ring_set_t *mac_srs;
2062 2093 int i;
2063 2094
2064 2095 ASSERT(MAC_PERIM_HELD((mac_handle_t)mip));
2065 2096
2066 2097 if (flent == NULL)
2067 2098 return;
2068 2099
2069 2100 for (i = 0; i < flent->fe_rx_srs_cnt; i++) {
2070 2101 mac_srs = flent->fe_rx_srs[i];
2071 2102 mutex_enter(&mac_srs->srs_lock);
2072 2103 if (on)
2073 2104 mac_srs->srs_state |= SRS_QUIESCE_PERM;
2074 2105 else
2075 2106 mac_srs->srs_state &= ~SRS_QUIESCE_PERM;
2076 2107 mutex_exit(&mac_srs->srs_lock);
2077 2108 }
2078 2109 }
2079 2110
2080 2111 void
2081 2112 mac_rx_client_quiesce(mac_client_handle_t mch)
2082 2113 {
2083 2114 mac_client_impl_t *mcip = (mac_client_impl_t *)mch;
2084 2115 mac_impl_t *mip = mcip->mci_mip;
2085 2116
2086 2117 ASSERT(MAC_PERIM_HELD((mac_handle_t)mip));
2087 2118
2088 2119 if (MCIP_DATAPATH_SETUP(mcip)) {
2089 2120 (void) mac_rx_classify_flow_quiesce(mcip->mci_flent,
2090 2121 NULL);
2091 2122 (void) mac_flow_walk_nolock(mcip->mci_subflow_tab,
2092 2123 mac_rx_classify_flow_quiesce, NULL);
2093 2124 }
2094 2125 }
2095 2126
2096 2127 void
2097 2128 mac_rx_client_restart(mac_client_handle_t mch)
2098 2129 {
2099 2130 mac_client_impl_t *mcip = (mac_client_impl_t *)mch;
2100 2131 mac_impl_t *mip = mcip->mci_mip;
2101 2132
2102 2133 ASSERT(MAC_PERIM_HELD((mac_handle_t)mip));
2103 2134
2104 2135 if (MCIP_DATAPATH_SETUP(mcip)) {
2105 2136 (void) mac_rx_classify_flow_restart(mcip->mci_flent, NULL);
2106 2137 (void) mac_flow_walk_nolock(mcip->mci_subflow_tab,
2107 2138 mac_rx_classify_flow_restart, NULL);
2108 2139 }
2109 2140 }
2110 2141
2111 2142 /*
2112 2143 * This function only quiesces the Tx SRS and softring worker threads. Callers
2113 2144 * need to make sure that there aren't any mac client threads doing current or
2114 2145 * future transmits in the mac before calling this function.
2115 2146 */
2116 2147 void
2117 2148 mac_tx_srs_quiesce(mac_soft_ring_set_t *srs, uint_t srs_quiesce_flag)
2118 2149 {
2119 2150 mac_client_impl_t *mcip = srs->srs_mcip;
2120 2151
2121 2152 ASSERT(MAC_PERIM_HELD((mac_handle_t)mcip->mci_mip));
2122 2153
2123 2154 ASSERT(srs->srs_type & SRST_TX);
2124 2155 ASSERT(srs_quiesce_flag == SRS_CONDEMNED ||
2125 2156 srs_quiesce_flag == SRS_QUIESCE);
2126 2157
2127 2158 /*
2128 2159 * Signal the SRS to quiesce itself, and then cv_wait for the
2129 2160 * SRS quiesce to complete. The SRS worker thread will wake us
2130 2161 * up when the quiesce is complete
2131 2162 */
2132 2163 mac_srs_signal(srs, srs_quiesce_flag);
2133 2164 mac_srs_quiesce_wait(srs, srs_quiesce_flag == SRS_QUIESCE ?
2134 2165 SRS_QUIESCE_DONE : SRS_CONDEMNED_DONE);
2135 2166 }
2136 2167
2137 2168 void
2138 2169 mac_tx_srs_restart(mac_soft_ring_set_t *srs)
2139 2170 {
2140 2171 /*
2141 2172 * Resizing the fanout could result in creation of new SRSs.
2142 2173 * They may not necessarily be in the quiesced state in which
2143 2174 * case it need be restarted
2144 2175 */
2145 2176 if (!SRS_QUIESCED(srs))
2146 2177 return;
2147 2178
2148 2179 mac_srs_signal(srs, SRS_RESTART);
2149 2180 mac_srs_quiesce_wait(srs, SRS_RESTART_DONE);
2150 2181 mac_srs_clear_flag(srs, SRS_RESTART_DONE);
2151 2182 }
2152 2183
2153 2184 /*
2154 2185 * Temporary quiesce of a flow and associated Rx SRS.
2155 2186 * Please see block comment above mac_rx_srs_quiesce
2156 2187 */
2157 2188 /* ARGSUSED */
2158 2189 int
2159 2190 mac_tx_flow_quiesce(flow_entry_t *flent, void *arg)
2160 2191 {
2161 2192 /*
2162 2193 * The fe_tx_srs is null for a subflow on an interface that is
2163 2194 * not plumbed
2164 2195 */
2165 2196 if (flent->fe_tx_srs != NULL)
2166 2197 mac_tx_srs_quiesce(flent->fe_tx_srs, SRS_QUIESCE);
2167 2198 return (0);
2168 2199 }
2169 2200
2170 2201 /* ARGSUSED */
2171 2202 int
2172 2203 mac_tx_flow_restart(flow_entry_t *flent, void *arg)
2173 2204 {
2174 2205 /*
2175 2206 * The fe_tx_srs is null for a subflow on an interface that is
2176 2207 * not plumbed
2177 2208 */
2178 2209 if (flent->fe_tx_srs != NULL)
2179 2210 mac_tx_srs_restart(flent->fe_tx_srs);
2180 2211 return (0);
2181 2212 }
2182 2213
2183 2214 static void
2184 2215 i_mac_tx_client_quiesce(mac_client_handle_t mch, uint_t srs_quiesce_flag)
2185 2216 {
2186 2217 mac_client_impl_t *mcip = (mac_client_impl_t *)mch;
2187 2218
2188 2219 ASSERT(MAC_PERIM_HELD((mac_handle_t)mcip->mci_mip));
2189 2220
2190 2221 mac_tx_client_block(mcip);
2191 2222 if (MCIP_TX_SRS(mcip) != NULL) {
2192 2223 mac_tx_srs_quiesce(MCIP_TX_SRS(mcip), srs_quiesce_flag);
2193 2224 (void) mac_flow_walk_nolock(mcip->mci_subflow_tab,
2194 2225 mac_tx_flow_quiesce, NULL);
2195 2226 }
2196 2227 }
2197 2228
2198 2229 void
2199 2230 mac_tx_client_quiesce(mac_client_handle_t mch)
2200 2231 {
2201 2232 i_mac_tx_client_quiesce(mch, SRS_QUIESCE);
2202 2233 }
2203 2234
2204 2235 void
2205 2236 mac_tx_client_condemn(mac_client_handle_t mch)
2206 2237 {
2207 2238 i_mac_tx_client_quiesce(mch, SRS_CONDEMNED);
2208 2239 }
2209 2240
2210 2241 void
2211 2242 mac_tx_client_restart(mac_client_handle_t mch)
2212 2243 {
2213 2244 mac_client_impl_t *mcip = (mac_client_impl_t *)mch;
2214 2245
2215 2246 ASSERT(MAC_PERIM_HELD((mac_handle_t)mcip->mci_mip));
2216 2247
2217 2248 mac_tx_client_unblock(mcip);
2218 2249 if (MCIP_TX_SRS(mcip) != NULL) {
2219 2250 mac_tx_srs_restart(MCIP_TX_SRS(mcip));
2220 2251 (void) mac_flow_walk_nolock(mcip->mci_subflow_tab,
2221 2252 mac_tx_flow_restart, NULL);
2222 2253 }
2223 2254 }
2224 2255
2225 2256 void
2226 2257 mac_tx_client_flush(mac_client_impl_t *mcip)
2227 2258 {
2228 2259 ASSERT(MAC_PERIM_HELD((mac_handle_t)mcip->mci_mip));
2229 2260
2230 2261 mac_tx_client_quiesce((mac_client_handle_t)mcip);
2231 2262 mac_tx_client_restart((mac_client_handle_t)mcip);
2232 2263 }
2233 2264
2234 2265 void
2235 2266 mac_client_quiesce(mac_client_impl_t *mcip)
2236 2267 {
2237 2268 mac_rx_client_quiesce((mac_client_handle_t)mcip);
2238 2269 mac_tx_client_quiesce((mac_client_handle_t)mcip);
2239 2270 }
2240 2271
2241 2272 void
2242 2273 mac_client_restart(mac_client_impl_t *mcip)
2243 2274 {
2244 2275 mac_rx_client_restart((mac_client_handle_t)mcip);
2245 2276 mac_tx_client_restart((mac_client_handle_t)mcip);
2246 2277 }
2247 2278
2248 2279 /*
2249 2280 * Allocate a minor number.
2250 2281 */
2251 2282 minor_t
2252 2283 mac_minor_hold(boolean_t sleep)
2253 2284 {
2254 2285 minor_t minor;
2255 2286
2256 2287 /*
2257 2288 * Grab a value from the arena.
2258 2289 */
2259 2290 atomic_inc_32(&minor_count);
2260 2291
2261 2292 if (sleep)
2262 2293 minor = (uint_t)id_alloc(minor_ids);
2263 2294 else
2264 2295 minor = (uint_t)id_alloc_nosleep(minor_ids);
2265 2296
2266 2297 if (minor == 0) {
2267 2298 atomic_dec_32(&minor_count);
2268 2299 return (0);
2269 2300 }
2270 2301
2271 2302 return (minor);
2272 2303 }
2273 2304
2274 2305 /*
2275 2306 * Release a previously allocated minor number.
2276 2307 */
2277 2308 void
2278 2309 mac_minor_rele(minor_t minor)
2279 2310 {
2280 2311 /*
2281 2312 * Return the value to the arena.
2282 2313 */
2283 2314 id_free(minor_ids, minor);
2284 2315 atomic_dec_32(&minor_count);
2285 2316 }
2286 2317
2287 2318 uint32_t
2288 2319 mac_no_notification(mac_handle_t mh)
2289 2320 {
2290 2321 mac_impl_t *mip = (mac_impl_t *)mh;
2291 2322
2292 2323 return (((mip->mi_state_flags & MIS_LEGACY) != 0) ?
2293 2324 mip->mi_capab_legacy.ml_unsup_note : 0);
2294 2325 }
2295 2326
2296 2327 /*
2297 2328 * Prevent any new opens of this mac in preparation for unregister
2298 2329 */
2299 2330 int
2300 2331 i_mac_disable(mac_impl_t *mip)
2301 2332 {
2302 2333 mac_client_impl_t *mcip;
2303 2334
2304 2335 rw_enter(&i_mac_impl_lock, RW_WRITER);
2305 2336 if (mip->mi_state_flags & MIS_DISABLED) {
2306 2337 /* Already disabled, return success */
2307 2338 rw_exit(&i_mac_impl_lock);
2308 2339 return (0);
2309 2340 }
2310 2341 /*
2311 2342 * See if there are any other references to this mac_t (e.g., VLAN's).
2312 2343 * If so return failure. If all the other checks below pass, then
2313 2344 * set mi_disabled atomically under the i_mac_impl_lock to prevent
2314 2345 * any new VLAN's from being created or new mac client opens of this
2315 2346 * mac end point.
2316 2347 */
2317 2348 if (mip->mi_ref > 0) {
2318 2349 rw_exit(&i_mac_impl_lock);
2319 2350 return (EBUSY);
2320 2351 }
2321 2352
2322 2353 /*
2323 2354 * mac clients must delete all multicast groups they join before
2324 2355 * closing. bcast groups are reference counted, the last client
2325 2356 * to delete the group will wait till the group is physically
2326 2357 * deleted. Since all clients have closed this mac end point
2327 2358 * mi_bcast_ngrps must be zero at this point
2328 2359 */
2329 2360 ASSERT(mip->mi_bcast_ngrps == 0);
2330 2361
2331 2362 /*
2332 2363 * Don't let go of this if it has some flows.
2333 2364 * All other code guarantees no flows are added to a disabled
2334 2365 * mac, therefore it is sufficient to check for the flow table
2335 2366 * only here.
2336 2367 */
2337 2368 mcip = mac_primary_client_handle(mip);
2338 2369 if ((mcip != NULL) && mac_link_has_flows((mac_client_handle_t)mcip)) {
2339 2370 rw_exit(&i_mac_impl_lock);
2340 2371 return (ENOTEMPTY);
2341 2372 }
2342 2373
2343 2374 mip->mi_state_flags |= MIS_DISABLED;
2344 2375 rw_exit(&i_mac_impl_lock);
2345 2376 return (0);
2346 2377 }
2347 2378
2348 2379 int
2349 2380 mac_disable_nowait(mac_handle_t mh)
2350 2381 {
2351 2382 mac_impl_t *mip = (mac_impl_t *)mh;
2352 2383 int err;
2353 2384
2354 2385 if ((err = i_mac_perim_enter_nowait(mip)) != 0)
2355 2386 return (err);
2356 2387 err = i_mac_disable(mip);
2357 2388 i_mac_perim_exit(mip);
2358 2389 return (err);
2359 2390 }
2360 2391
2361 2392 int
2362 2393 mac_disable(mac_handle_t mh)
2363 2394 {
2364 2395 mac_impl_t *mip = (mac_impl_t *)mh;
2365 2396 int err;
2366 2397
2367 2398 i_mac_perim_enter(mip);
2368 2399 err = i_mac_disable(mip);
2369 2400 i_mac_perim_exit(mip);
2370 2401
2371 2402 /*
2372 2403 * Clean up notification thread and wait for it to exit.
2373 2404 */
2374 2405 if (err == 0)
2375 2406 i_mac_notify_exit(mip);
2376 2407
2377 2408 return (err);
2378 2409 }
2379 2410
2380 2411 /*
2381 2412 * Called when the MAC instance has a non empty flow table, to de-multiplex
2382 2413 * incoming packets to the right flow.
2383 2414 * The MAC's rw lock is assumed held as a READER.
2384 2415 */
2385 2416 /* ARGSUSED */
2386 2417 static mblk_t *
2387 2418 mac_rx_classify(mac_impl_t *mip, mac_resource_handle_t mrh, mblk_t *mp)
2388 2419 {
2389 2420 flow_entry_t *flent = NULL;
2390 2421 uint_t flags = FLOW_INBOUND;
2391 2422 int err;
2392 2423
2393 2424 /*
2394 2425 * If the mac is a port of an aggregation, pass FLOW_IGNORE_VLAN
2395 2426 * to mac_flow_lookup() so that the VLAN packets can be successfully
2396 2427 * passed to the non-VLAN aggregation flows.
2397 2428 *
2398 2429 * Note that there is possibly a race between this and
2399 2430 * mac_unicast_remove/add() and VLAN packets could be incorrectly
2400 2431 * classified to non-VLAN flows of non-aggregation mac clients. These
2401 2432 * VLAN packets will be then filtered out by the mac module.
2402 2433 */
2403 2434 if ((mip->mi_state_flags & MIS_EXCLUSIVE) != 0)
2404 2435 flags |= FLOW_IGNORE_VLAN;
2405 2436
2406 2437 err = mac_flow_lookup(mip->mi_flow_tab, mp, flags, &flent);
2407 2438 if (err != 0) {
2408 2439 /* no registered receive function */
2409 2440 return (mp);
2410 2441 } else {
2411 2442 mac_client_impl_t *mcip;
2412 2443
2413 2444 /*
2414 2445 * This flent might just be an additional one on the MAC client,
2415 2446 * i.e. for classification purposes (different fdesc), however
2416 2447 * the resources, SRS et. al., are in the mci_flent, so if
2417 2448 * this isn't the mci_flent, we need to get it.
2418 2449 */
2419 2450 if ((mcip = flent->fe_mcip) != NULL &&
2420 2451 mcip->mci_flent != flent) {
2421 2452 FLOW_REFRELE(flent);
2422 2453 flent = mcip->mci_flent;
2423 2454 FLOW_TRY_REFHOLD(flent, err);
2424 2455 if (err != 0)
2425 2456 return (mp);
2426 2457 }
2427 2458 (flent->fe_cb_fn)(flent->fe_cb_arg1, flent->fe_cb_arg2, mp,
2428 2459 B_FALSE);
2429 2460 FLOW_REFRELE(flent);
2430 2461 }
2431 2462 return (NULL);
2432 2463 }
2433 2464
2434 2465 mblk_t *
2435 2466 mac_rx_flow(mac_handle_t mh, mac_resource_handle_t mrh, mblk_t *mp_chain)
2436 2467 {
2437 2468 mac_impl_t *mip = (mac_impl_t *)mh;
2438 2469 mblk_t *bp, *bp1, **bpp, *list = NULL;
2439 2470
2440 2471 /*
2441 2472 * We walk the chain and attempt to classify each packet.
2442 2473 * The packets that couldn't be classified will be returned
2443 2474 * back to the caller.
2444 2475 */
2445 2476 bp = mp_chain;
2446 2477 bpp = &list;
2447 2478 while (bp != NULL) {
2448 2479 bp1 = bp;
2449 2480 bp = bp->b_next;
2450 2481 bp1->b_next = NULL;
2451 2482
2452 2483 if (mac_rx_classify(mip, mrh, bp1) != NULL) {
2453 2484 *bpp = bp1;
2454 2485 bpp = &bp1->b_next;
2455 2486 }
2456 2487 }
2457 2488 return (list);
2458 2489 }
2459 2490
2460 2491 static int
2461 2492 mac_tx_flow_srs_wakeup(flow_entry_t *flent, void *arg)
2462 2493 {
2463 2494 mac_ring_handle_t ring = arg;
2464 2495
2465 2496 if (flent->fe_tx_srs)
2466 2497 mac_tx_srs_wakeup(flent->fe_tx_srs, ring);
2467 2498 return (0);
2468 2499 }
2469 2500
2470 2501 void
2471 2502 i_mac_tx_srs_notify(mac_impl_t *mip, mac_ring_handle_t ring)
2472 2503 {
2473 2504 mac_client_impl_t *cclient;
2474 2505 mac_soft_ring_set_t *mac_srs;
2475 2506
2476 2507 /*
2477 2508 * After grabbing the mi_rw_lock, the list of clients can't change.
2478 2509 * If there are any clients mi_disabled must be B_FALSE and can't
2479 2510 * get set since there are clients. If there aren't any clients we
2480 2511 * don't do anything. In any case the mip has to be valid. The driver
2481 2512 * must make sure that it goes single threaded (with respect to mac
2482 2513 * calls) and wait for all pending mac calls to finish before calling
2483 2514 * mac_unregister.
2484 2515 */
2485 2516 rw_enter(&i_mac_impl_lock, RW_READER);
2486 2517 if (mip->mi_state_flags & MIS_DISABLED) {
2487 2518 rw_exit(&i_mac_impl_lock);
2488 2519 return;
2489 2520 }
2490 2521
2491 2522 /*
2492 2523 * Get MAC tx srs from walking mac_client_handle list.
2493 2524 */
2494 2525 rw_enter(&mip->mi_rw_lock, RW_READER);
2495 2526 for (cclient = mip->mi_clients_list; cclient != NULL;
2496 2527 cclient = cclient->mci_client_next) {
2497 2528 if ((mac_srs = MCIP_TX_SRS(cclient)) != NULL) {
2498 2529 mac_tx_srs_wakeup(mac_srs, ring);
2499 2530 } else {
2500 2531 /*
2501 2532 * Aggr opens underlying ports in exclusive mode
2502 2533 * and registers flow control callbacks using
2503 2534 * mac_tx_client_notify(). When opened in
2504 2535 * exclusive mode, Tx SRS won't be created
2505 2536 * during mac_unicast_add().
2506 2537 */
2507 2538 if (cclient->mci_state_flags & MCIS_EXCLUSIVE) {
2508 2539 mac_tx_invoke_callbacks(cclient,
2509 2540 (mac_tx_cookie_t)ring);
2510 2541 }
2511 2542 }
2512 2543 (void) mac_flow_walk(cclient->mci_subflow_tab,
2513 2544 mac_tx_flow_srs_wakeup, ring);
2514 2545 }
2515 2546 rw_exit(&mip->mi_rw_lock);
2516 2547 rw_exit(&i_mac_impl_lock);
2517 2548 }
2518 2549
2519 2550 /* ARGSUSED */
2520 2551 void
2521 2552 mac_multicast_refresh(mac_handle_t mh, mac_multicst_t refresh, void *arg,
2522 2553 boolean_t add)
2523 2554 {
2524 2555 mac_impl_t *mip = (mac_impl_t *)mh;
2525 2556
2526 2557 i_mac_perim_enter((mac_impl_t *)mh);
2527 2558 /*
2528 2559 * If no specific refresh function was given then default to the
2529 2560 * driver's m_multicst entry point.
2530 2561 */
2531 2562 if (refresh == NULL) {
2532 2563 refresh = mip->mi_multicst;
2533 2564 arg = mip->mi_driver;
2534 2565 }
2535 2566
2536 2567 mac_bcast_refresh(mip, refresh, arg, add);
2537 2568 i_mac_perim_exit((mac_impl_t *)mh);
2538 2569 }
2539 2570
2540 2571 void
2541 2572 mac_promisc_refresh(mac_handle_t mh, mac_setpromisc_t refresh, void *arg)
2542 2573 {
2543 2574 mac_impl_t *mip = (mac_impl_t *)mh;
2544 2575
2545 2576 /*
2546 2577 * If no specific refresh function was given then default to the
2547 2578 * driver's m_promisc entry point.
2548 2579 */
2549 2580 if (refresh == NULL) {
2550 2581 refresh = mip->mi_setpromisc;
2551 2582 arg = mip->mi_driver;
2552 2583 }
2553 2584 ASSERT(refresh != NULL);
2554 2585
2555 2586 /*
2556 2587 * Call the refresh function with the current promiscuity.
2557 2588 */
2558 2589 refresh(arg, (mip->mi_devpromisc != 0));
2559 2590 }
2560 2591
2561 2592 /*
2562 2593 * The mac client requests that the mac not to change its margin size to
2563 2594 * be less than the specified value. If "current" is B_TRUE, then the client
2564 2595 * requests the mac not to change its margin size to be smaller than the
2565 2596 * current size. Further, return the current margin size value in this case.
2566 2597 *
2567 2598 * We keep every requested size in an ordered list from largest to smallest.
2568 2599 */
2569 2600 int
2570 2601 mac_margin_add(mac_handle_t mh, uint32_t *marginp, boolean_t current)
2571 2602 {
2572 2603 mac_impl_t *mip = (mac_impl_t *)mh;
2573 2604 mac_margin_req_t **pp, *p;
2574 2605 int err = 0;
2575 2606
2576 2607 rw_enter(&(mip->mi_rw_lock), RW_WRITER);
2577 2608 if (current)
2578 2609 *marginp = mip->mi_margin;
2579 2610
2580 2611 /*
2581 2612 * If the current margin value cannot satisfy the margin requested,
2582 2613 * return ENOTSUP directly.
2583 2614 */
2584 2615 if (*marginp > mip->mi_margin) {
2585 2616 err = ENOTSUP;
2586 2617 goto done;
2587 2618 }
2588 2619
2589 2620 /*
2590 2621 * Check whether the given margin is already in the list. If so,
2591 2622 * bump the reference count.
2592 2623 */
2593 2624 for (pp = &mip->mi_mmrp; (p = *pp) != NULL; pp = &p->mmr_nextp) {
2594 2625 if (p->mmr_margin == *marginp) {
2595 2626 /*
2596 2627 * The margin requested is already in the list,
2597 2628 * so just bump the reference count.
2598 2629 */
2599 2630 p->mmr_ref++;
2600 2631 goto done;
2601 2632 }
2602 2633 if (p->mmr_margin < *marginp)
2603 2634 break;
2604 2635 }
2605 2636
2606 2637
2607 2638 p = kmem_zalloc(sizeof (mac_margin_req_t), KM_SLEEP);
2608 2639 p->mmr_margin = *marginp;
2609 2640 p->mmr_ref++;
2610 2641 p->mmr_nextp = *pp;
2611 2642 *pp = p;
2612 2643
2613 2644 done:
2614 2645 rw_exit(&(mip->mi_rw_lock));
2615 2646 return (err);
2616 2647 }
2617 2648
2618 2649 /*
2619 2650 * The mac client requests to cancel its previous mac_margin_add() request.
2620 2651 * We remove the requested margin size from the list.
2621 2652 */
2622 2653 int
2623 2654 mac_margin_remove(mac_handle_t mh, uint32_t margin)
2624 2655 {
2625 2656 mac_impl_t *mip = (mac_impl_t *)mh;
2626 2657 mac_margin_req_t **pp, *p;
2627 2658 int err = 0;
2628 2659
2629 2660 rw_enter(&(mip->mi_rw_lock), RW_WRITER);
2630 2661 /*
2631 2662 * Find the entry in the list for the given margin.
2632 2663 */
2633 2664 for (pp = &(mip->mi_mmrp); (p = *pp) != NULL; pp = &(p->mmr_nextp)) {
2634 2665 if (p->mmr_margin == margin) {
2635 2666 if (--p->mmr_ref == 0)
2636 2667 break;
2637 2668
2638 2669 /*
2639 2670 * There is still a reference to this address so
2640 2671 * there's nothing more to do.
2641 2672 */
2642 2673 goto done;
2643 2674 }
2644 2675 }
2645 2676
2646 2677 /*
2647 2678 * We did not find an entry for the given margin.
2648 2679 */
2649 2680 if (p == NULL) {
2650 2681 err = ENOENT;
2651 2682 goto done;
2652 2683 }
2653 2684
2654 2685 ASSERT(p->mmr_ref == 0);
2655 2686
2656 2687 /*
2657 2688 * Remove it from the list.
2658 2689 */
2659 2690 *pp = p->mmr_nextp;
2660 2691 kmem_free(p, sizeof (mac_margin_req_t));
2661 2692 done:
2662 2693 rw_exit(&(mip->mi_rw_lock));
2663 2694 return (err);
2664 2695 }
2665 2696
2666 2697 boolean_t
2667 2698 mac_margin_update(mac_handle_t mh, uint32_t margin)
2668 2699 {
2669 2700 mac_impl_t *mip = (mac_impl_t *)mh;
2670 2701 uint32_t margin_needed = 0;
2671 2702
2672 2703 rw_enter(&(mip->mi_rw_lock), RW_WRITER);
2673 2704
2674 2705 if (mip->mi_mmrp != NULL)
2675 2706 margin_needed = mip->mi_mmrp->mmr_margin;
2676 2707
2677 2708 if (margin_needed <= margin)
2678 2709 mip->mi_margin = margin;
2679 2710
2680 2711 rw_exit(&(mip->mi_rw_lock));
2681 2712
2682 2713 if (margin_needed <= margin)
2683 2714 i_mac_notify(mip, MAC_NOTE_MARGIN);
2684 2715
2685 2716 return (margin_needed <= margin);
2686 2717 }
2687 2718
2688 2719 /*
2689 2720 * MAC clients use this interface to request that a MAC device not change its
2690 2721 * MTU below the specified amount. At this time, that amount must be within the
2691 2722 * range of the device's current minimum and the device's current maximum. eg. a
2692 2723 * client cannot request a 3000 byte MTU when the device's MTU is currently
2693 2724 * 2000.
2694 2725 *
2695 2726 * If "current" is set to B_TRUE, then the request is to simply to reserve the
2696 2727 * current underlying mac's maximum for this mac client and return it in mtup.
2697 2728 */
2698 2729 int
2699 2730 mac_mtu_add(mac_handle_t mh, uint32_t *mtup, boolean_t current)
2700 2731 {
2701 2732 mac_impl_t *mip = (mac_impl_t *)mh;
2702 2733 mac_mtu_req_t *prev, *cur;
2703 2734 mac_propval_range_t mpr;
2704 2735 int err;
2705 2736
2706 2737 i_mac_perim_enter(mip);
2707 2738 rw_enter(&mip->mi_rw_lock, RW_WRITER);
2708 2739
2709 2740 if (current == B_TRUE)
2710 2741 *mtup = mip->mi_sdu_max;
2711 2742 mpr.mpr_count = 1;
2712 2743 err = mac_prop_info(mh, MAC_PROP_MTU, "mtu", NULL, 0, &mpr, NULL);
2713 2744 if (err != 0) {
2714 2745 rw_exit(&mip->mi_rw_lock);
2715 2746 i_mac_perim_exit(mip);
2716 2747 return (err);
2717 2748 }
2718 2749
2719 2750 if (*mtup > mip->mi_sdu_max ||
2720 2751 *mtup < mpr.mpr_range_uint32[0].mpur_min) {
2721 2752 rw_exit(&mip->mi_rw_lock);
2722 2753 i_mac_perim_exit(mip);
2723 2754 return (ENOTSUP);
2724 2755 }
2725 2756
2726 2757 prev = NULL;
2727 2758 for (cur = mip->mi_mtrp; cur != NULL; cur = cur->mtr_nextp) {
2728 2759 if (*mtup == cur->mtr_mtu) {
2729 2760 cur->mtr_ref++;
2730 2761 rw_exit(&mip->mi_rw_lock);
2731 2762 i_mac_perim_exit(mip);
2732 2763 return (0);
2733 2764 }
2734 2765
2735 2766 if (*mtup > cur->mtr_mtu)
2736 2767 break;
2737 2768
2738 2769 prev = cur;
2739 2770 }
2740 2771
2741 2772 cur = kmem_alloc(sizeof (mac_mtu_req_t), KM_SLEEP);
2742 2773 cur->mtr_mtu = *mtup;
2743 2774 cur->mtr_ref = 1;
2744 2775 if (prev != NULL) {
2745 2776 cur->mtr_nextp = prev->mtr_nextp;
2746 2777 prev->mtr_nextp = cur;
2747 2778 } else {
2748 2779 cur->mtr_nextp = mip->mi_mtrp;
2749 2780 mip->mi_mtrp = cur;
2750 2781 }
2751 2782
2752 2783 rw_exit(&mip->mi_rw_lock);
2753 2784 i_mac_perim_exit(mip);
2754 2785 return (0);
2755 2786 }
2756 2787
2757 2788 int
2758 2789 mac_mtu_remove(mac_handle_t mh, uint32_t mtu)
2759 2790 {
2760 2791 mac_impl_t *mip = (mac_impl_t *)mh;
2761 2792 mac_mtu_req_t *cur, *prev;
2762 2793
2763 2794 i_mac_perim_enter(mip);
2764 2795 rw_enter(&mip->mi_rw_lock, RW_WRITER);
2765 2796
2766 2797 prev = NULL;
2767 2798 for (cur = mip->mi_mtrp; cur != NULL; cur = cur->mtr_nextp) {
2768 2799 if (cur->mtr_mtu == mtu) {
2769 2800 ASSERT(cur->mtr_ref > 0);
2770 2801 cur->mtr_ref--;
2771 2802 if (cur->mtr_ref == 0) {
2772 2803 if (prev == NULL) {
2773 2804 mip->mi_mtrp = cur->mtr_nextp;
2774 2805 } else {
2775 2806 prev->mtr_nextp = cur->mtr_nextp;
2776 2807 }
2777 2808 kmem_free(cur, sizeof (mac_mtu_req_t));
2778 2809 }
2779 2810 rw_exit(&mip->mi_rw_lock);
2780 2811 i_mac_perim_exit(mip);
2781 2812 return (0);
2782 2813 }
2783 2814
2784 2815 prev = cur;
2785 2816 }
2786 2817
2787 2818 rw_exit(&mip->mi_rw_lock);
2788 2819 i_mac_perim_exit(mip);
2789 2820 return (ENOENT);
2790 2821 }
2791 2822
2792 2823 /*
2793 2824 * MAC Type Plugin functions.
2794 2825 */
2795 2826
2796 2827 mactype_t *
2797 2828 mactype_getplugin(const char *pname)
2798 2829 {
2799 2830 mactype_t *mtype = NULL;
2800 2831 boolean_t tried_modload = B_FALSE;
2801 2832
2802 2833 mutex_enter(&i_mactype_lock);
2803 2834
2804 2835 find_registered_mactype:
2805 2836 if (mod_hash_find(i_mactype_hash, (mod_hash_key_t)pname,
2806 2837 (mod_hash_val_t *)&mtype) != 0) {
2807 2838 if (!tried_modload) {
2808 2839 /*
2809 2840 * If the plugin has not yet been loaded, then
2810 2841 * attempt to load it now. If modload() succeeds,
2811 2842 * the plugin should have registered using
2812 2843 * mactype_register(), in which case we can go back
2813 2844 * and attempt to find it again.
2814 2845 */
2815 2846 if (modload(MACTYPE_KMODDIR, (char *)pname) != -1) {
2816 2847 tried_modload = B_TRUE;
2817 2848 goto find_registered_mactype;
2818 2849 }
2819 2850 }
2820 2851 } else {
2821 2852 /*
2822 2853 * Note that there's no danger that the plugin we've loaded
2823 2854 * could be unloaded between the modload() step and the
2824 2855 * reference count bump here, as we're holding
2825 2856 * i_mactype_lock, which mactype_unregister() also holds.
2826 2857 */
2827 2858 atomic_inc_32(&mtype->mt_ref);
2828 2859 }
2829 2860
2830 2861 mutex_exit(&i_mactype_lock);
2831 2862 return (mtype);
2832 2863 }
2833 2864
2834 2865 mactype_register_t *
2835 2866 mactype_alloc(uint_t mactype_version)
2836 2867 {
2837 2868 mactype_register_t *mtrp;
2838 2869
2839 2870 /*
2840 2871 * Make sure there isn't a version mismatch between the plugin and
2841 2872 * the framework. In the future, if multiple versions are
2842 2873 * supported, this check could become more sophisticated.
2843 2874 */
2844 2875 if (mactype_version != MACTYPE_VERSION)
2845 2876 return (NULL);
2846 2877
2847 2878 mtrp = kmem_zalloc(sizeof (mactype_register_t), KM_SLEEP);
2848 2879 mtrp->mtr_version = mactype_version;
2849 2880 return (mtrp);
2850 2881 }
2851 2882
2852 2883 void
2853 2884 mactype_free(mactype_register_t *mtrp)
2854 2885 {
2855 2886 kmem_free(mtrp, sizeof (mactype_register_t));
2856 2887 }
2857 2888
2858 2889 int
2859 2890 mactype_register(mactype_register_t *mtrp)
2860 2891 {
2861 2892 mactype_t *mtp;
2862 2893 mactype_ops_t *ops = mtrp->mtr_ops;
2863 2894
2864 2895 /* Do some sanity checking before we register this MAC type. */
2865 2896 if (mtrp->mtr_ident == NULL || ops == NULL)
2866 2897 return (EINVAL);
2867 2898
2868 2899 /*
2869 2900 * Verify that all mandatory callbacks are set in the ops
2870 2901 * vector.
2871 2902 */
2872 2903 if (ops->mtops_unicst_verify == NULL ||
2873 2904 ops->mtops_multicst_verify == NULL ||
2874 2905 ops->mtops_sap_verify == NULL ||
2875 2906 ops->mtops_header == NULL ||
2876 2907 ops->mtops_header_info == NULL) {
2877 2908 return (EINVAL);
2878 2909 }
2879 2910
2880 2911 mtp = kmem_zalloc(sizeof (*mtp), KM_SLEEP);
2881 2912 mtp->mt_ident = mtrp->mtr_ident;
2882 2913 mtp->mt_ops = *ops;
2883 2914 mtp->mt_type = mtrp->mtr_mactype;
2884 2915 mtp->mt_nativetype = mtrp->mtr_nativetype;
2885 2916 mtp->mt_addr_length = mtrp->mtr_addrlen;
2886 2917 if (mtrp->mtr_brdcst_addr != NULL) {
2887 2918 mtp->mt_brdcst_addr = kmem_alloc(mtrp->mtr_addrlen, KM_SLEEP);
2888 2919 bcopy(mtrp->mtr_brdcst_addr, mtp->mt_brdcst_addr,
2889 2920 mtrp->mtr_addrlen);
2890 2921 }
2891 2922
2892 2923 mtp->mt_stats = mtrp->mtr_stats;
2893 2924 mtp->mt_statcount = mtrp->mtr_statcount;
2894 2925
2895 2926 mtp->mt_mapping = mtrp->mtr_mapping;
2896 2927 mtp->mt_mappingcount = mtrp->mtr_mappingcount;
2897 2928
2898 2929 if (mod_hash_insert(i_mactype_hash,
2899 2930 (mod_hash_key_t)mtp->mt_ident, (mod_hash_val_t)mtp) != 0) {
2900 2931 kmem_free(mtp->mt_brdcst_addr, mtp->mt_addr_length);
2901 2932 kmem_free(mtp, sizeof (*mtp));
2902 2933 return (EEXIST);
2903 2934 }
2904 2935 return (0);
2905 2936 }
2906 2937
2907 2938 int
2908 2939 mactype_unregister(const char *ident)
2909 2940 {
2910 2941 mactype_t *mtp;
2911 2942 mod_hash_val_t val;
2912 2943 int err;
2913 2944
2914 2945 /*
2915 2946 * Let's not allow MAC drivers to use this plugin while we're
2916 2947 * trying to unregister it. Holding i_mactype_lock also prevents a
2917 2948 * plugin from unregistering while a MAC driver is attempting to
2918 2949 * hold a reference to it in i_mactype_getplugin().
2919 2950 */
2920 2951 mutex_enter(&i_mactype_lock);
2921 2952
2922 2953 if ((err = mod_hash_find(i_mactype_hash, (mod_hash_key_t)ident,
2923 2954 (mod_hash_val_t *)&mtp)) != 0) {
2924 2955 /* A plugin is trying to unregister, but it never registered. */
2925 2956 err = ENXIO;
2926 2957 goto done;
2927 2958 }
2928 2959
2929 2960 if (mtp->mt_ref != 0) {
2930 2961 err = EBUSY;
2931 2962 goto done;
2932 2963 }
2933 2964
2934 2965 err = mod_hash_remove(i_mactype_hash, (mod_hash_key_t)ident, &val);
2935 2966 ASSERT(err == 0);
2936 2967 if (err != 0) {
2937 2968 /* This should never happen, thus the ASSERT() above. */
2938 2969 err = EINVAL;
2939 2970 goto done;
2940 2971 }
2941 2972 ASSERT(mtp == (mactype_t *)val);
2942 2973
2943 2974 if (mtp->mt_brdcst_addr != NULL)
2944 2975 kmem_free(mtp->mt_brdcst_addr, mtp->mt_addr_length);
2945 2976 kmem_free(mtp, sizeof (mactype_t));
2946 2977 done:
2947 2978 mutex_exit(&i_mactype_lock);
2948 2979 return (err);
2949 2980 }
2950 2981
2951 2982 /*
2952 2983 * Checks the size of the value size specified for a property as
2953 2984 * part of a property operation. Returns B_TRUE if the size is
2954 2985 * correct, B_FALSE otherwise.
2955 2986 */
2956 2987 boolean_t
2957 2988 mac_prop_check_size(mac_prop_id_t id, uint_t valsize, boolean_t is_range)
2958 2989 {
2959 2990 uint_t minsize = 0;
2960 2991
2961 2992 if (is_range)
2962 2993 return (valsize >= sizeof (mac_propval_range_t));
2963 2994
2964 2995 switch (id) {
2965 2996 case MAC_PROP_ZONE:
2966 2997 minsize = sizeof (dld_ioc_zid_t);
2967 2998 break;
2968 2999 case MAC_PROP_AUTOPUSH:
2969 3000 if (valsize != 0)
2970 3001 minsize = sizeof (struct dlautopush);
2971 3002 break;
2972 3003 case MAC_PROP_TAGMODE:
2973 3004 minsize = sizeof (link_tagmode_t);
2974 3005 break;
2975 3006 case MAC_PROP_RESOURCE:
2976 3007 case MAC_PROP_RESOURCE_EFF:
2977 3008 minsize = sizeof (mac_resource_props_t);
2978 3009 break;
2979 3010 case MAC_PROP_DUPLEX:
2980 3011 minsize = sizeof (link_duplex_t);
2981 3012 break;
2982 3013 case MAC_PROP_SPEED:
2983 3014 minsize = sizeof (uint64_t);
2984 3015 break;
2985 3016 case MAC_PROP_STATUS:
2986 3017 minsize = sizeof (link_state_t);
2987 3018 break;
2988 3019 case MAC_PROP_AUTONEG:
2989 3020 case MAC_PROP_EN_AUTONEG:
2990 3021 minsize = sizeof (uint8_t);
2991 3022 break;
2992 3023 case MAC_PROP_MTU:
2993 3024 case MAC_PROP_LLIMIT:
2994 3025 case MAC_PROP_LDECAY:
2995 3026 minsize = sizeof (uint32_t);
2996 3027 break;
2997 3028 case MAC_PROP_FLOWCTRL:
2998 3029 minsize = sizeof (link_flowctrl_t);
2999 3030 break;
3000 3031 case MAC_PROP_ADV_10GFDX_CAP:
3001 3032 case MAC_PROP_EN_10GFDX_CAP:
3002 3033 case MAC_PROP_ADV_1000HDX_CAP:
3003 3034 case MAC_PROP_EN_1000HDX_CAP:
3004 3035 case MAC_PROP_ADV_100FDX_CAP:
3005 3036 case MAC_PROP_EN_100FDX_CAP:
3006 3037 case MAC_PROP_ADV_100HDX_CAP:
3007 3038 case MAC_PROP_EN_100HDX_CAP:
3008 3039 case MAC_PROP_ADV_10FDX_CAP:
3009 3040 case MAC_PROP_EN_10FDX_CAP:
3010 3041 case MAC_PROP_ADV_10HDX_CAP:
3011 3042 case MAC_PROP_EN_10HDX_CAP:
3012 3043 case MAC_PROP_ADV_100T4_CAP:
3013 3044 case MAC_PROP_EN_100T4_CAP:
3014 3045 minsize = sizeof (uint8_t);
3015 3046 break;
3016 3047 case MAC_PROP_PVID:
3017 3048 minsize = sizeof (uint16_t);
3018 3049 break;
3019 3050 case MAC_PROP_IPTUN_HOPLIMIT:
3020 3051 minsize = sizeof (uint32_t);
3021 3052 break;
3022 3053 case MAC_PROP_IPTUN_ENCAPLIMIT:
3023 3054 minsize = sizeof (uint32_t);
3024 3055 break;
3025 3056 case MAC_PROP_MAX_TX_RINGS_AVAIL:
3026 3057 case MAC_PROP_MAX_RX_RINGS_AVAIL:
3027 3058 case MAC_PROP_MAX_RXHWCLNT_AVAIL:
3028 3059 case MAC_PROP_MAX_TXHWCLNT_AVAIL:
3029 3060 minsize = sizeof (uint_t);
3030 3061 break;
3031 3062 case MAC_PROP_WL_ESSID:
3032 3063 minsize = sizeof (wl_linkstatus_t);
3033 3064 break;
3034 3065 case MAC_PROP_WL_BSSID:
3035 3066 minsize = sizeof (wl_bssid_t);
3036 3067 break;
3037 3068 case MAC_PROP_WL_BSSTYPE:
3038 3069 minsize = sizeof (wl_bss_type_t);
3039 3070 break;
3040 3071 case MAC_PROP_WL_LINKSTATUS:
3041 3072 minsize = sizeof (wl_linkstatus_t);
3042 3073 break;
3043 3074 case MAC_PROP_WL_DESIRED_RATES:
3044 3075 minsize = sizeof (wl_rates_t);
3045 3076 break;
3046 3077 case MAC_PROP_WL_SUPPORTED_RATES:
3047 3078 minsize = sizeof (wl_rates_t);
3048 3079 break;
3049 3080 case MAC_PROP_WL_AUTH_MODE:
3050 3081 minsize = sizeof (wl_authmode_t);
3051 3082 break;
3052 3083 case MAC_PROP_WL_ENCRYPTION:
3053 3084 minsize = sizeof (wl_encryption_t);
3054 3085 break;
3055 3086 case MAC_PROP_WL_RSSI:
3056 3087 minsize = sizeof (wl_rssi_t);
3057 3088 break;
3058 3089 case MAC_PROP_WL_PHY_CONFIG:
3059 3090 minsize = sizeof (wl_phy_conf_t);
3060 3091 break;
3061 3092 case MAC_PROP_WL_CAPABILITY:
3062 3093 minsize = sizeof (wl_capability_t);
3063 3094 break;
3064 3095 case MAC_PROP_WL_WPA:
3065 3096 minsize = sizeof (wl_wpa_t);
3066 3097 break;
3067 3098 case MAC_PROP_WL_SCANRESULTS:
3068 3099 minsize = sizeof (wl_wpa_ess_t);
3069 3100 break;
3070 3101 case MAC_PROP_WL_POWER_MODE:
3071 3102 minsize = sizeof (wl_ps_mode_t);
3072 3103 break;
3073 3104 case MAC_PROP_WL_RADIO:
3074 3105 minsize = sizeof (wl_radio_t);
3075 3106 break;
3076 3107 case MAC_PROP_WL_ESS_LIST:
3077 3108 minsize = sizeof (wl_ess_list_t);
3078 3109 break;
3079 3110 case MAC_PROP_WL_KEY_TAB:
3080 3111 minsize = sizeof (wl_wep_key_tab_t);
3081 3112 break;
3082 3113 case MAC_PROP_WL_CREATE_IBSS:
3083 3114 minsize = sizeof (wl_create_ibss_t);
3084 3115 break;
3085 3116 case MAC_PROP_WL_SETOPTIE:
3086 3117 minsize = sizeof (wl_wpa_ie_t);
3087 3118 break;
3088 3119 case MAC_PROP_WL_DELKEY:
3089 3120 minsize = sizeof (wl_del_key_t);
3090 3121 break;
3091 3122 case MAC_PROP_WL_KEY:
3092 3123 minsize = sizeof (wl_key_t);
3093 3124 break;
3094 3125 case MAC_PROP_WL_MLME:
3095 3126 minsize = sizeof (wl_mlme_t);
3096 3127 break;
3097 3128 }
3098 3129
3099 3130 return (valsize >= minsize);
3100 3131 }
3101 3132
3102 3133 /*
3103 3134 * mac_set_prop() sets MAC or hardware driver properties:
3104 3135 *
3105 3136 * - MAC-managed properties such as resource properties include maxbw,
3106 3137 * priority, and cpu binding list, as well as the default port VID
3107 3138 * used by bridging. These properties are consumed by the MAC layer
3108 3139 * itself and not passed down to the driver. For resource control
3109 3140 * properties, this function invokes mac_set_resources() which will
3110 3141 * cache the property value in mac_impl_t and may call
3111 3142 * mac_client_set_resource() to update property value of the primary
3112 3143 * mac client, if it exists.
3113 3144 *
3114 3145 * - Properties which act on the hardware and must be passed to the
3115 3146 * driver, such as MTU, through the driver's mc_setprop() entry point.
3116 3147 */
3117 3148 int
3118 3149 mac_set_prop(mac_handle_t mh, mac_prop_id_t id, char *name, void *val,
3119 3150 uint_t valsize)
3120 3151 {
3121 3152 int err = ENOTSUP;
3122 3153 mac_impl_t *mip = (mac_impl_t *)mh;
3123 3154
3124 3155 ASSERT(MAC_PERIM_HELD(mh));
3125 3156
3126 3157 switch (id) {
3127 3158 case MAC_PROP_RESOURCE: {
3128 3159 mac_resource_props_t *mrp;
3129 3160
3130 3161 /* call mac_set_resources() for MAC properties */
3131 3162 ASSERT(valsize >= sizeof (mac_resource_props_t));
3132 3163 mrp = kmem_zalloc(sizeof (*mrp), KM_SLEEP);
3133 3164 bcopy(val, mrp, sizeof (*mrp));
3134 3165 err = mac_set_resources(mh, mrp);
3135 3166 kmem_free(mrp, sizeof (*mrp));
3136 3167 break;
3137 3168 }
3138 3169
3139 3170 case MAC_PROP_PVID:
3140 3171 ASSERT(valsize >= sizeof (uint16_t));
3141 3172 if (mip->mi_state_flags & MIS_IS_VNIC)
3142 3173 return (EINVAL);
3143 3174 err = mac_set_pvid(mh, *(uint16_t *)val);
3144 3175 break;
3145 3176
3146 3177 case MAC_PROP_MTU: {
3147 3178 uint32_t mtu;
3148 3179
3149 3180 ASSERT(valsize >= sizeof (uint32_t));
3150 3181 bcopy(val, &mtu, sizeof (mtu));
3151 3182 err = mac_set_mtu(mh, mtu, NULL);
3152 3183 break;
3153 3184 }
3154 3185
3155 3186 case MAC_PROP_LLIMIT:
3156 3187 case MAC_PROP_LDECAY: {
3157 3188 uint32_t learnval;
3158 3189
3159 3190 if (valsize < sizeof (learnval) ||
3160 3191 (mip->mi_state_flags & MIS_IS_VNIC))
3161 3192 return (EINVAL);
3162 3193 bcopy(val, &learnval, sizeof (learnval));
3163 3194 if (learnval == 0 && id == MAC_PROP_LDECAY)
3164 3195 return (EINVAL);
3165 3196 if (id == MAC_PROP_LLIMIT)
3166 3197 mip->mi_llimit = learnval;
3167 3198 else
3168 3199 mip->mi_ldecay = learnval;
3169 3200 err = 0;
3170 3201 break;
3171 3202 }
3172 3203
3173 3204 default:
3174 3205 /* For other driver properties, call driver's callback */
3175 3206 if (mip->mi_callbacks->mc_callbacks & MC_SETPROP) {
3176 3207 err = mip->mi_callbacks->mc_setprop(mip->mi_driver,
3177 3208 name, id, valsize, val);
3178 3209 }
3179 3210 }
3180 3211 return (err);
3181 3212 }
3182 3213
3183 3214 /*
3184 3215 * mac_get_prop() gets MAC or device driver properties.
3185 3216 *
3186 3217 * If the property is a driver property, mac_get_prop() calls driver's callback
3187 3218 * entry point to get it.
3188 3219 * If the property is a MAC property, mac_get_prop() invokes mac_get_resources()
3189 3220 * which returns the cached value in mac_impl_t.
3190 3221 */
3191 3222 int
3192 3223 mac_get_prop(mac_handle_t mh, mac_prop_id_t id, char *name, void *val,
3193 3224 uint_t valsize)
3194 3225 {
3195 3226 int err = ENOTSUP;
3196 3227 mac_impl_t *mip = (mac_impl_t *)mh;
3197 3228 uint_t rings;
3198 3229 uint_t vlinks;
3199 3230
3200 3231 bzero(val, valsize);
3201 3232
3202 3233 switch (id) {
3203 3234 case MAC_PROP_RESOURCE: {
3204 3235 mac_resource_props_t *mrp;
3205 3236
3206 3237 /* If mac property, read from cache */
3207 3238 ASSERT(valsize >= sizeof (mac_resource_props_t));
3208 3239 mrp = kmem_zalloc(sizeof (*mrp), KM_SLEEP);
3209 3240 mac_get_resources(mh, mrp);
3210 3241 bcopy(mrp, val, sizeof (*mrp));
3211 3242 kmem_free(mrp, sizeof (*mrp));
3212 3243 return (0);
3213 3244 }
3214 3245 case MAC_PROP_RESOURCE_EFF: {
3215 3246 mac_resource_props_t *mrp;
3216 3247
3217 3248 /* If mac effective property, read from client */
3218 3249 ASSERT(valsize >= sizeof (mac_resource_props_t));
3219 3250 mrp = kmem_zalloc(sizeof (*mrp), KM_SLEEP);
3220 3251 mac_get_effective_resources(mh, mrp);
3221 3252 bcopy(mrp, val, sizeof (*mrp));
3222 3253 kmem_free(mrp, sizeof (*mrp));
3223 3254 return (0);
3224 3255 }
3225 3256
3226 3257 case MAC_PROP_PVID:
3227 3258 ASSERT(valsize >= sizeof (uint16_t));
3228 3259 if (mip->mi_state_flags & MIS_IS_VNIC)
3229 3260 return (EINVAL);
3230 3261 *(uint16_t *)val = mac_get_pvid(mh);
3231 3262 return (0);
3232 3263
3233 3264 case MAC_PROP_LLIMIT:
3234 3265 case MAC_PROP_LDECAY:
3235 3266 ASSERT(valsize >= sizeof (uint32_t));
3236 3267 if (mip->mi_state_flags & MIS_IS_VNIC)
3237 3268 return (EINVAL);
3238 3269 if (id == MAC_PROP_LLIMIT)
3239 3270 bcopy(&mip->mi_llimit, val, sizeof (mip->mi_llimit));
3240 3271 else
3241 3272 bcopy(&mip->mi_ldecay, val, sizeof (mip->mi_ldecay));
3242 3273 return (0);
3243 3274
3244 3275 case MAC_PROP_MTU: {
3245 3276 uint32_t sdu;
3246 3277
3247 3278 ASSERT(valsize >= sizeof (uint32_t));
3248 3279 mac_sdu_get2(mh, NULL, &sdu, NULL);
3249 3280 bcopy(&sdu, val, sizeof (sdu));
3250 3281
3251 3282 return (0);
3252 3283 }
3253 3284 case MAC_PROP_STATUS: {
3254 3285 link_state_t link_state;
3255 3286
3256 3287 if (valsize < sizeof (link_state))
3257 3288 return (EINVAL);
3258 3289 link_state = mac_link_get(mh);
3259 3290 bcopy(&link_state, val, sizeof (link_state));
3260 3291
3261 3292 return (0);
3262 3293 }
3263 3294
3264 3295 case MAC_PROP_MAX_RX_RINGS_AVAIL:
3265 3296 case MAC_PROP_MAX_TX_RINGS_AVAIL:
3266 3297 ASSERT(valsize >= sizeof (uint_t));
3267 3298 rings = id == MAC_PROP_MAX_RX_RINGS_AVAIL ?
3268 3299 mac_rxavail_get(mh) : mac_txavail_get(mh);
3269 3300 bcopy(&rings, val, sizeof (uint_t));
3270 3301 return (0);
3271 3302
3272 3303 case MAC_PROP_MAX_RXHWCLNT_AVAIL:
3273 3304 case MAC_PROP_MAX_TXHWCLNT_AVAIL:
3274 3305 ASSERT(valsize >= sizeof (uint_t));
3275 3306 vlinks = id == MAC_PROP_MAX_RXHWCLNT_AVAIL ?
3276 3307 mac_rxhwlnksavail_get(mh) : mac_txhwlnksavail_get(mh);
3277 3308 bcopy(&vlinks, val, sizeof (uint_t));
3278 3309 return (0);
3279 3310
3280 3311 case MAC_PROP_RXRINGSRANGE:
3281 3312 case MAC_PROP_TXRINGSRANGE:
3282 3313 /*
3283 3314 * The value for these properties are returned through
3284 3315 * the MAC_PROP_RESOURCE property.
3285 3316 */
3286 3317 return (0);
3287 3318
3288 3319 default:
3289 3320 break;
3290 3321
3291 3322 }
3292 3323
3293 3324 /* If driver property, request from driver */
3294 3325 if (mip->mi_callbacks->mc_callbacks & MC_GETPROP) {
3295 3326 err = mip->mi_callbacks->mc_getprop(mip->mi_driver, name, id,
3296 3327 valsize, val);
3297 3328 }
3298 3329
3299 3330 return (err);
3300 3331 }
3301 3332
3302 3333 /*
3303 3334 * Helper function to initialize the range structure for use in
3304 3335 * mac_get_prop. If the type can be other than uint32, we can
3305 3336 * pass that as an arg.
3306 3337 */
3307 3338 static void
3308 3339 _mac_set_range(mac_propval_range_t *range, uint32_t min, uint32_t max)
3309 3340 {
3310 3341 range->mpr_count = 1;
3311 3342 range->mpr_type = MAC_PROPVAL_UINT32;
3312 3343 range->mpr_range_uint32[0].mpur_min = min;
3313 3344 range->mpr_range_uint32[0].mpur_max = max;
3314 3345 }
3315 3346
3316 3347 /*
3317 3348 * Returns information about the specified property, such as default
3318 3349 * values or permissions.
3319 3350 */
3320 3351 int
3321 3352 mac_prop_info(mac_handle_t mh, mac_prop_id_t id, char *name,
3322 3353 void *default_val, uint_t default_size, mac_propval_range_t *range,
3323 3354 uint_t *perm)
3324 3355 {
3325 3356 mac_prop_info_state_t state;
3326 3357 mac_impl_t *mip = (mac_impl_t *)mh;
3327 3358 uint_t max;
3328 3359
3329 3360 /*
3330 3361 * A property is read/write by default unless the driver says
3331 3362 * otherwise.
3332 3363 */
3333 3364 if (perm != NULL)
3334 3365 *perm = MAC_PROP_PERM_RW;
3335 3366
3336 3367 if (default_val != NULL)
3337 3368 bzero(default_val, default_size);
3338 3369
3339 3370 /*
3340 3371 * First, handle framework properties for which we don't need to
3341 3372 * involve the driver.
3342 3373 */
3343 3374 switch (id) {
3344 3375 case MAC_PROP_RESOURCE:
3345 3376 case MAC_PROP_PVID:
3346 3377 case MAC_PROP_LLIMIT:
3347 3378 case MAC_PROP_LDECAY:
3348 3379 return (0);
3349 3380
3350 3381 case MAC_PROP_MAX_RX_RINGS_AVAIL:
3351 3382 case MAC_PROP_MAX_TX_RINGS_AVAIL:
3352 3383 case MAC_PROP_MAX_RXHWCLNT_AVAIL:
3353 3384 case MAC_PROP_MAX_TXHWCLNT_AVAIL:
3354 3385 if (perm != NULL)
3355 3386 *perm = MAC_PROP_PERM_READ;
3356 3387 return (0);
3357 3388
3358 3389 case MAC_PROP_RXRINGSRANGE:
3359 3390 case MAC_PROP_TXRINGSRANGE:
3360 3391 /*
3361 3392 * Currently, we support range for RX and TX rings properties.
3362 3393 * When we extend this support to maxbw, cpus and priority,
3363 3394 * we should move this to mac_get_resources.
3364 3395 * There is no default value for RX or TX rings.
3365 3396 */
3366 3397 if ((mip->mi_state_flags & MIS_IS_VNIC) &&
3367 3398 mac_is_vnic_primary(mh)) {
3368 3399 /*
3369 3400 * We don't support setting rings for a VLAN
3370 3401 * data link because it shares its ring with the
3371 3402 * primary MAC client.
3372 3403 */
3373 3404 if (perm != NULL)
3374 3405 *perm = MAC_PROP_PERM_READ;
3375 3406 if (range != NULL)
3376 3407 range->mpr_count = 0;
3377 3408 } else if (range != NULL) {
3378 3409 if (mip->mi_state_flags & MIS_IS_VNIC)
3379 3410 mh = mac_get_lower_mac_handle(mh);
3380 3411 mip = (mac_impl_t *)mh;
3381 3412 if ((id == MAC_PROP_RXRINGSRANGE &&
3382 3413 mip->mi_rx_group_type == MAC_GROUP_TYPE_STATIC) ||
3383 3414 (id == MAC_PROP_TXRINGSRANGE &&
3384 3415 mip->mi_tx_group_type == MAC_GROUP_TYPE_STATIC)) {
3385 3416 if (id == MAC_PROP_RXRINGSRANGE) {
3386 3417 if ((mac_rxhwlnksavail_get(mh) +
3387 3418 mac_rxhwlnksrsvd_get(mh)) <= 1) {
3388 3419 /*
3389 3420 * doesn't support groups or
3390 3421 * rings
3391 3422 */
3392 3423 range->mpr_count = 0;
3393 3424 } else {
3394 3425 /*
3395 3426 * supports specifying groups,
3396 3427 * but not rings
3397 3428 */
3398 3429 _mac_set_range(range, 0, 0);
3399 3430 }
3400 3431 } else {
3401 3432 if ((mac_txhwlnksavail_get(mh) +
3402 3433 mac_txhwlnksrsvd_get(mh)) <= 1) {
3403 3434 /*
3404 3435 * doesn't support groups or
3405 3436 * rings
3406 3437 */
3407 3438 range->mpr_count = 0;
3408 3439 } else {
3409 3440 /*
3410 3441 * supports specifying groups,
3411 3442 * but not rings
3412 3443 */
3413 3444 _mac_set_range(range, 0, 0);
3414 3445 }
3415 3446 }
3416 3447 } else {
3417 3448 max = id == MAC_PROP_RXRINGSRANGE ?
3418 3449 mac_rxavail_get(mh) + mac_rxrsvd_get(mh) :
3419 3450 mac_txavail_get(mh) + mac_txrsvd_get(mh);
3420 3451 if (max <= 1) {
3421 3452 /*
3422 3453 * doesn't support groups or
3423 3454 * rings
3424 3455 */
3425 3456 range->mpr_count = 0;
3426 3457 } else {
3427 3458 /*
3428 3459 * -1 because we have to leave out the
3429 3460 * default ring.
3430 3461 */
3431 3462 _mac_set_range(range, 1, max - 1);
3432 3463 }
3433 3464 }
3434 3465 }
3435 3466 return (0);
3436 3467
3437 3468 case MAC_PROP_STATUS:
3438 3469 if (perm != NULL)
3439 3470 *perm = MAC_PROP_PERM_READ;
3440 3471 return (0);
3441 3472 }
3442 3473
3443 3474 /*
3444 3475 * Get the property info from the driver if it implements the
3445 3476 * property info entry point.
3446 3477 */
3447 3478 bzero(&state, sizeof (state));
3448 3479
3449 3480 if (mip->mi_callbacks->mc_callbacks & MC_PROPINFO) {
3450 3481 state.pr_default = default_val;
3451 3482 state.pr_default_size = default_size;
3452 3483
3453 3484 /*
3454 3485 * The caller specifies the maximum number of ranges
3455 3486 * it can accomodate using mpr_count. We don't touch
3456 3487 * this value until the driver returns from its
3457 3488 * mc_propinfo() callback, and ensure we don't exceed
3458 3489 * this number of range as the driver defines
3459 3490 * supported range from its mc_propinfo().
3460 3491 *
3461 3492 * pr_range_cur_count keeps track of how many ranges
3462 3493 * were defined by the driver from its mc_propinfo()
3463 3494 * entry point.
3464 3495 *
3465 3496 * On exit, the user-specified range mpr_count returns
3466 3497 * the number of ranges specified by the driver on
3467 3498 * success, or the number of ranges it wanted to
3468 3499 * define if that number of ranges could not be
3469 3500 * accomodated by the specified range structure. In
3470 3501 * the latter case, the caller will be able to
3471 3502 * allocate a larger range structure, and query the
3472 3503 * property again.
3473 3504 */
3474 3505 state.pr_range_cur_count = 0;
3475 3506 state.pr_range = range;
3476 3507
3477 3508 mip->mi_callbacks->mc_propinfo(mip->mi_driver, name, id,
3478 3509 (mac_prop_info_handle_t)&state);
3479 3510
3480 3511 if (state.pr_flags & MAC_PROP_INFO_RANGE)
3481 3512 range->mpr_count = state.pr_range_cur_count;
3482 3513
3483 3514 /*
3484 3515 * The operation could fail if the buffer supplied by
3485 3516 * the user was too small for the range or default
3486 3517 * value of the property.
3487 3518 */
3488 3519 if (state.pr_errno != 0)
3489 3520 return (state.pr_errno);
3490 3521
3491 3522 if (perm != NULL && state.pr_flags & MAC_PROP_INFO_PERM)
3492 3523 *perm = state.pr_perm;
3493 3524 }
3494 3525
3495 3526 /*
3496 3527 * The MAC layer may want to provide default values or allowed
3497 3528 * ranges for properties if the driver does not provide a
3498 3529 * property info entry point, or that entry point exists, but
3499 3530 * it did not provide a default value or allowed ranges for
3500 3531 * that property.
3501 3532 */
3502 3533 switch (id) {
3503 3534 case MAC_PROP_MTU: {
3504 3535 uint32_t sdu;
3505 3536
3506 3537 mac_sdu_get2(mh, NULL, &sdu, NULL);
3507 3538
3508 3539 if (range != NULL && !(state.pr_flags &
3509 3540 MAC_PROP_INFO_RANGE)) {
3510 3541 /* MTU range */
3511 3542 _mac_set_range(range, sdu, sdu);
3512 3543 }
3513 3544
3514 3545 if (default_val != NULL && !(state.pr_flags &
3515 3546 MAC_PROP_INFO_DEFAULT)) {
3516 3547 if (mip->mi_info.mi_media == DL_ETHER)
3517 3548 sdu = ETHERMTU;
3518 3549 /* default MTU value */
3519 3550 bcopy(&sdu, default_val, sizeof (sdu));
3520 3551 }
3521 3552 }
3522 3553 }
3523 3554
3524 3555 return (0);
3525 3556 }
3526 3557
3527 3558 int
3528 3559 mac_fastpath_disable(mac_handle_t mh)
3529 3560 {
3530 3561 mac_impl_t *mip = (mac_impl_t *)mh;
3531 3562
3532 3563 if ((mip->mi_state_flags & MIS_LEGACY) == 0)
3533 3564 return (0);
3534 3565
3535 3566 return (mip->mi_capab_legacy.ml_fastpath_disable(mip->mi_driver));
3536 3567 }
3537 3568
3538 3569 void
3539 3570 mac_fastpath_enable(mac_handle_t mh)
3540 3571 {
3541 3572 mac_impl_t *mip = (mac_impl_t *)mh;
3542 3573
3543 3574 if ((mip->mi_state_flags & MIS_LEGACY) == 0)
3544 3575 return;
3545 3576
3546 3577 mip->mi_capab_legacy.ml_fastpath_enable(mip->mi_driver);
3547 3578 }
3548 3579
3549 3580 void
3550 3581 mac_register_priv_prop(mac_impl_t *mip, char **priv_props)
3551 3582 {
3552 3583 uint_t nprops, i;
3553 3584
3554 3585 if (priv_props == NULL)
3555 3586 return;
3556 3587
3557 3588 nprops = 0;
3558 3589 while (priv_props[nprops] != NULL)
3559 3590 nprops++;
3560 3591 if (nprops == 0)
3561 3592 return;
3562 3593
3563 3594
3564 3595 mip->mi_priv_prop = kmem_zalloc(nprops * sizeof (char *), KM_SLEEP);
3565 3596
3566 3597 for (i = 0; i < nprops; i++) {
3567 3598 mip->mi_priv_prop[i] = kmem_zalloc(MAXLINKPROPNAME, KM_SLEEP);
3568 3599 (void) strlcpy(mip->mi_priv_prop[i], priv_props[i],
3569 3600 MAXLINKPROPNAME);
3570 3601 }
3571 3602
3572 3603 mip->mi_priv_prop_count = nprops;
3573 3604 }
3574 3605
3575 3606 void
3576 3607 mac_unregister_priv_prop(mac_impl_t *mip)
3577 3608 {
3578 3609 uint_t i;
3579 3610
3580 3611 if (mip->mi_priv_prop_count == 0) {
3581 3612 ASSERT(mip->mi_priv_prop == NULL);
3582 3613 return;
3583 3614 }
3584 3615
3585 3616 for (i = 0; i < mip->mi_priv_prop_count; i++)
3586 3617 kmem_free(mip->mi_priv_prop[i], MAXLINKPROPNAME);
3587 3618 kmem_free(mip->mi_priv_prop, mip->mi_priv_prop_count *
3588 3619 sizeof (char *));
3589 3620
3590 3621 mip->mi_priv_prop = NULL;
3591 3622 mip->mi_priv_prop_count = 0;
3592 3623 }
3593 3624
3594 3625 /*
3595 3626 * mac_ring_t 'mr' macros. Some rogue drivers may access ring structure
3596 3627 * (by invoking mac_rx()) even after processing mac_stop_ring(). In such
3597 3628 * cases if MAC free's the ring structure after mac_stop_ring(), any
3598 3629 * illegal access to the ring structure coming from the driver will panic
3599 3630 * the system. In order to protect the system from such inadverent access,
3600 3631 * we maintain a cache of rings in the mac_impl_t after they get free'd up.
3601 3632 * When packets are received on free'd up rings, MAC (through the generation
3602 3633 * count mechanism) will drop such packets.
3603 3634 */
3604 3635 static mac_ring_t *
3605 3636 mac_ring_alloc(mac_impl_t *mip)
3606 3637 {
3607 3638 mac_ring_t *ring;
3608 3639
3609 3640 mutex_enter(&mip->mi_ring_lock);
3610 3641 if (mip->mi_ring_freelist != NULL) {
3611 3642 ring = mip->mi_ring_freelist;
3612 3643 mip->mi_ring_freelist = ring->mr_next;
3613 3644 bzero(ring, sizeof (mac_ring_t));
3614 3645 mutex_exit(&mip->mi_ring_lock);
3615 3646 } else {
3616 3647 mutex_exit(&mip->mi_ring_lock);
3617 3648 ring = kmem_cache_alloc(mac_ring_cache, KM_SLEEP);
3618 3649 }
3619 3650 ASSERT((ring != NULL) && (ring->mr_state == MR_FREE));
3620 3651 return (ring);
3621 3652 }
3622 3653
3623 3654 static void
3624 3655 mac_ring_free(mac_impl_t *mip, mac_ring_t *ring)
3625 3656 {
3626 3657 ASSERT(ring->mr_state == MR_FREE);
3627 3658
3628 3659 mutex_enter(&mip->mi_ring_lock);
3629 3660 ring->mr_state = MR_FREE;
3630 3661 ring->mr_flag = 0;
3631 3662 ring->mr_next = mip->mi_ring_freelist;
3632 3663 ring->mr_mip = NULL;
3633 3664 mip->mi_ring_freelist = ring;
3634 3665 mac_ring_stat_delete(ring);
3635 3666 mutex_exit(&mip->mi_ring_lock);
3636 3667 }
3637 3668
3638 3669 static void
3639 3670 mac_ring_freeall(mac_impl_t *mip)
3640 3671 {
3641 3672 mac_ring_t *ring_next;
3642 3673 mutex_enter(&mip->mi_ring_lock);
3643 3674 mac_ring_t *ring = mip->mi_ring_freelist;
3644 3675 while (ring != NULL) {
3645 3676 ring_next = ring->mr_next;
3646 3677 kmem_cache_free(mac_ring_cache, ring);
3647 3678 ring = ring_next;
3648 3679 }
3649 3680 mip->mi_ring_freelist = NULL;
3650 3681 mutex_exit(&mip->mi_ring_lock);
3651 3682 }
3652 3683
3653 3684 int
3654 3685 mac_start_ring(mac_ring_t *ring)
3655 3686 {
3656 3687 int rv = 0;
3657 3688
3658 3689 ASSERT(ring->mr_state == MR_FREE);
3659 3690
3660 3691 if (ring->mr_start != NULL) {
3661 3692 rv = ring->mr_start(ring->mr_driver, ring->mr_gen_num);
3662 3693 if (rv != 0)
3663 3694 return (rv);
3664 3695 }
3665 3696
3666 3697 ring->mr_state = MR_INUSE;
3667 3698 return (rv);
3668 3699 }
3669 3700
3670 3701 void
3671 3702 mac_stop_ring(mac_ring_t *ring)
3672 3703 {
3673 3704 ASSERT(ring->mr_state == MR_INUSE);
3674 3705
3675 3706 if (ring->mr_stop != NULL)
3676 3707 ring->mr_stop(ring->mr_driver);
3677 3708
3678 3709 ring->mr_state = MR_FREE;
3679 3710
3680 3711 /*
3681 3712 * Increment the ring generation number for this ring.
3682 3713 */
3683 3714 ring->mr_gen_num++;
3684 3715 }
3685 3716
3686 3717 int
3687 3718 mac_start_group(mac_group_t *group)
3688 3719 {
3689 3720 int rv = 0;
3690 3721
3691 3722 if (group->mrg_start != NULL)
3692 3723 rv = group->mrg_start(group->mrg_driver);
3693 3724
3694 3725 return (rv);
3695 3726 }
3696 3727
3697 3728 void
3698 3729 mac_stop_group(mac_group_t *group)
3699 3730 {
3700 3731 if (group->mrg_stop != NULL)
3701 3732 group->mrg_stop(group->mrg_driver);
3702 3733 }
3703 3734
3704 3735 /*
3705 3736 * Called from mac_start() on the default Rx group. Broadcast and multicast
3706 3737 * packets are received only on the default group. Hence the default group
3707 3738 * needs to be up even if the primary client is not up, for the other groups
3708 3739 * to be functional. We do this by calling this function at mac_start time
3709 3740 * itself. However the broadcast packets that are received can't make their
3710 3741 * way beyond mac_rx until a mac client creates a broadcast flow.
3711 3742 */
3712 3743 static int
3713 3744 mac_start_group_and_rings(mac_group_t *group)
3714 3745 {
3715 3746 mac_ring_t *ring;
3716 3747 int rv = 0;
3717 3748
3718 3749 ASSERT(group->mrg_state == MAC_GROUP_STATE_REGISTERED);
3719 3750 if ((rv = mac_start_group(group)) != 0)
3720 3751 return (rv);
3721 3752
3722 3753 for (ring = group->mrg_rings; ring != NULL; ring = ring->mr_next) {
3723 3754 ASSERT(ring->mr_state == MR_FREE);
3724 3755 if ((rv = mac_start_ring(ring)) != 0)
3725 3756 goto error;
3726 3757 ring->mr_classify_type = MAC_SW_CLASSIFIER;
3727 3758 }
3728 3759 return (0);
3729 3760
3730 3761 error:
3731 3762 mac_stop_group_and_rings(group);
3732 3763 return (rv);
3733 3764 }
3734 3765
3735 3766 /* Called from mac_stop on the default Rx group */
3736 3767 static void
3737 3768 mac_stop_group_and_rings(mac_group_t *group)
3738 3769 {
3739 3770 mac_ring_t *ring;
3740 3771
3741 3772 for (ring = group->mrg_rings; ring != NULL; ring = ring->mr_next) {
3742 3773 if (ring->mr_state != MR_FREE) {
3743 3774 mac_stop_ring(ring);
3744 3775 ring->mr_flag = 0;
3745 3776 ring->mr_classify_type = MAC_NO_CLASSIFIER;
3746 3777 }
3747 3778 }
3748 3779 mac_stop_group(group);
3749 3780 }
3750 3781
3751 3782
3752 3783 static mac_ring_t *
3753 3784 mac_init_ring(mac_impl_t *mip, mac_group_t *group, int index,
3754 3785 mac_capab_rings_t *cap_rings)
3755 3786 {
3756 3787 mac_ring_t *ring, *rnext;
3757 3788 mac_ring_info_t ring_info;
3758 3789 ddi_intr_handle_t ddi_handle;
3759 3790
3760 3791 ring = mac_ring_alloc(mip);
3761 3792
3762 3793 /* Prepare basic information of ring */
3763 3794
3764 3795 /*
3765 3796 * Ring index is numbered to be unique across a particular device.
3766 3797 * Ring index computation makes following assumptions:
3767 3798 * - For drivers with static grouping (e.g. ixgbe, bge),
3768 3799 * ring index exchanged with the driver (e.g. during mr_rget)
3769 3800 * is unique only across the group the ring belongs to.
3770 3801 * - Drivers with dynamic grouping (e.g. nxge), start
3771 3802 * with single group (mrg_index = 0).
3772 3803 */
3773 3804 ring->mr_index = group->mrg_index * group->mrg_info.mgi_count + index;
3774 3805 ring->mr_type = group->mrg_type;
3775 3806 ring->mr_gh = (mac_group_handle_t)group;
3776 3807
3777 3808 /* Insert the new ring to the list. */
3778 3809 ring->mr_next = group->mrg_rings;
3779 3810 group->mrg_rings = ring;
3780 3811
3781 3812 /* Zero to reuse the info data structure */
3782 3813 bzero(&ring_info, sizeof (ring_info));
3783 3814
3784 3815 /* Query ring information from driver */
3785 3816 cap_rings->mr_rget(mip->mi_driver, group->mrg_type, group->mrg_index,
3786 3817 index, &ring_info, (mac_ring_handle_t)ring);
3787 3818
3788 3819 ring->mr_info = ring_info;
3789 3820
3790 3821 /*
3791 3822 * The interrupt handle could be shared among multiple rings.
3792 3823 * Thus if there is a bunch of rings that are sharing an
3793 3824 * interrupt, then only one ring among the bunch will be made
3794 3825 * available for interrupt re-targeting; the rest will have
3795 3826 * ddi_shared flag set to TRUE and would not be available for
3796 3827 * be interrupt re-targeting.
3797 3828 */
3798 3829 if ((ddi_handle = ring_info.mri_intr.mi_ddi_handle) != NULL) {
3799 3830 rnext = ring->mr_next;
3800 3831 while (rnext != NULL) {
3801 3832 if (rnext->mr_info.mri_intr.mi_ddi_handle ==
3802 3833 ddi_handle) {
3803 3834 /*
3804 3835 * If default ring (mr_index == 0) is part
3805 3836 * of a group of rings sharing an
3806 3837 * interrupt, then set ddi_shared flag for
3807 3838 * the default ring and give another ring
3808 3839 * the chance to be re-targeted.
3809 3840 */
3810 3841 if (rnext->mr_index == 0 &&
3811 3842 !rnext->mr_info.mri_intr.mi_ddi_shared) {
3812 3843 rnext->mr_info.mri_intr.mi_ddi_shared =
3813 3844 B_TRUE;
3814 3845 } else {
3815 3846 ring->mr_info.mri_intr.mi_ddi_shared =
3816 3847 B_TRUE;
3817 3848 }
3818 3849 break;
3819 3850 }
3820 3851 rnext = rnext->mr_next;
3821 3852 }
3822 3853 /*
3823 3854 * If rnext is NULL, then no matching ddi_handle was found.
3824 3855 * Rx rings get registered first. So if this is a Tx ring,
3825 3856 * then go through all the Rx rings and see if there is a
3826 3857 * matching ddi handle.
3827 3858 */
3828 3859 if (rnext == NULL && ring->mr_type == MAC_RING_TYPE_TX) {
3829 3860 mac_compare_ddi_handle(mip->mi_rx_groups,
3830 3861 mip->mi_rx_group_count, ring);
3831 3862 }
3832 3863 }
3833 3864
3834 3865 /* Update ring's status */
3835 3866 ring->mr_state = MR_FREE;
3836 3867 ring->mr_flag = 0;
3837 3868
3838 3869 /* Update the ring count of the group */
3839 3870 group->mrg_cur_count++;
3840 3871
3841 3872 /* Create per ring kstats */
3842 3873 if (ring->mr_stat != NULL) {
3843 3874 ring->mr_mip = mip;
3844 3875 mac_ring_stat_create(ring);
3845 3876 }
3846 3877
3847 3878 return (ring);
3848 3879 }
3849 3880
3850 3881 /*
3851 3882 * Rings are chained together for easy regrouping.
3852 3883 */
3853 3884 static void
3854 3885 mac_init_group(mac_impl_t *mip, mac_group_t *group, int size,
3855 3886 mac_capab_rings_t *cap_rings)
3856 3887 {
3857 3888 int index;
3858 3889
3859 3890 /*
3860 3891 * Initialize all ring members of this group. Size of zero will not
3861 3892 * enter the loop, so it's safe for initializing an empty group.
3862 3893 */
3863 3894 for (index = size - 1; index >= 0; index--)
3864 3895 (void) mac_init_ring(mip, group, index, cap_rings);
3865 3896 }
3866 3897
3867 3898 int
3868 3899 mac_init_rings(mac_impl_t *mip, mac_ring_type_t rtype)
3869 3900 {
3870 3901 mac_capab_rings_t *cap_rings;
3871 3902 mac_group_t *group;
3872 3903 mac_group_t *groups;
3873 3904 mac_group_info_t group_info;
3874 3905 uint_t group_free = 0;
3875 3906 uint_t ring_left;
3876 3907 mac_ring_t *ring;
3877 3908 int g;
3878 3909 int err = 0;
3879 3910 uint_t grpcnt;
3880 3911 boolean_t pseudo_txgrp = B_FALSE;
3881 3912
3882 3913 switch (rtype) {
3883 3914 case MAC_RING_TYPE_RX:
3884 3915 ASSERT(mip->mi_rx_groups == NULL);
3885 3916
3886 3917 cap_rings = &mip->mi_rx_rings_cap;
3887 3918 cap_rings->mr_type = MAC_RING_TYPE_RX;
3888 3919 break;
3889 3920 case MAC_RING_TYPE_TX:
3890 3921 ASSERT(mip->mi_tx_groups == NULL);
3891 3922
3892 3923 cap_rings = &mip->mi_tx_rings_cap;
3893 3924 cap_rings->mr_type = MAC_RING_TYPE_TX;
3894 3925 break;
3895 3926 default:
3896 3927 ASSERT(B_FALSE);
3897 3928 }
3898 3929
3899 3930 if (!i_mac_capab_get((mac_handle_t)mip, MAC_CAPAB_RINGS, cap_rings))
3900 3931 return (0);
3901 3932 grpcnt = cap_rings->mr_gnum;
3902 3933
3903 3934 /*
3904 3935 * If we have multiple TX rings, but only one TX group, we can
3905 3936 * create pseudo TX groups (one per TX ring) in the MAC layer,
3906 3937 * except for an aggr. For an aggr currently we maintain only
3907 3938 * one group with all the rings (for all its ports), going
3908 3939 * forwards we might change this.
3909 3940 */
3910 3941 if (rtype == MAC_RING_TYPE_TX &&
3911 3942 cap_rings->mr_gnum == 0 && cap_rings->mr_rnum > 0 &&
3912 3943 (mip->mi_state_flags & MIS_IS_AGGR) == 0) {
3913 3944 /*
3914 3945 * The -1 here is because we create a default TX group
3915 3946 * with all the rings in it.
3916 3947 */
3917 3948 grpcnt = cap_rings->mr_rnum - 1;
3918 3949 pseudo_txgrp = B_TRUE;
3919 3950 }
3920 3951
3921 3952 /*
3922 3953 * Allocate a contiguous buffer for all groups.
3923 3954 */
3924 3955 groups = kmem_zalloc(sizeof (mac_group_t) * (grpcnt+ 1), KM_SLEEP);
3925 3956
3926 3957 ring_left = cap_rings->mr_rnum;
3927 3958
3928 3959 /*
3929 3960 * Get all ring groups if any, and get their ring members
3930 3961 * if any.
3931 3962 */
3932 3963 for (g = 0; g < grpcnt; g++) {
3933 3964 group = groups + g;
3934 3965
3935 3966 /* Prepare basic information of the group */
3936 3967 group->mrg_index = g;
3937 3968 group->mrg_type = rtype;
3938 3969 group->mrg_state = MAC_GROUP_STATE_UNINIT;
3939 3970 group->mrg_mh = (mac_handle_t)mip;
3940 3971 group->mrg_next = group + 1;
3941 3972
3942 3973 /* Zero to reuse the info data structure */
3943 3974 bzero(&group_info, sizeof (group_info));
3944 3975
3945 3976 if (pseudo_txgrp) {
3946 3977 /*
3947 3978 * This is a pseudo group that we created, apart
3948 3979 * from setting the state there is nothing to be
3949 3980 * done.
3950 3981 */
3951 3982 group->mrg_state = MAC_GROUP_STATE_REGISTERED;
3952 3983 group_free++;
3953 3984 continue;
3954 3985 }
3955 3986 /* Query group information from driver */
3956 3987 cap_rings->mr_gget(mip->mi_driver, rtype, g, &group_info,
3957 3988 (mac_group_handle_t)group);
3958 3989
3959 3990 switch (cap_rings->mr_group_type) {
3960 3991 case MAC_GROUP_TYPE_DYNAMIC:
3961 3992 if (cap_rings->mr_gaddring == NULL ||
3962 3993 cap_rings->mr_gremring == NULL) {
3963 3994 DTRACE_PROBE3(
3964 3995 mac__init__rings_no_addremring,
3965 3996 char *, mip->mi_name,
3966 3997 mac_group_add_ring_t,
3967 3998 cap_rings->mr_gaddring,
3968 3999 mac_group_add_ring_t,
3969 4000 cap_rings->mr_gremring);
3970 4001 err = EINVAL;
3971 4002 goto bail;
3972 4003 }
3973 4004
3974 4005 switch (rtype) {
3975 4006 case MAC_RING_TYPE_RX:
3976 4007 /*
3977 4008 * The first RX group must have non-zero
3978 4009 * rings, and the following groups must
3979 4010 * have zero rings.
3980 4011 */
3981 4012 if (g == 0 && group_info.mgi_count == 0) {
3982 4013 DTRACE_PROBE1(
3983 4014 mac__init__rings__rx__def__zero,
3984 4015 char *, mip->mi_name);
3985 4016 err = EINVAL;
3986 4017 goto bail;
3987 4018 }
3988 4019 if (g > 0 && group_info.mgi_count != 0) {
3989 4020 DTRACE_PROBE3(
3990 4021 mac__init__rings__rx__nonzero,
3991 4022 char *, mip->mi_name,
3992 4023 int, g, int, group_info.mgi_count);
3993 4024 err = EINVAL;
3994 4025 goto bail;
3995 4026 }
3996 4027 break;
3997 4028 case MAC_RING_TYPE_TX:
3998 4029 /*
3999 4030 * All TX ring groups must have zero rings.
4000 4031 */
4001 4032 if (group_info.mgi_count != 0) {
4002 4033 DTRACE_PROBE3(
4003 4034 mac__init__rings__tx__nonzero,
4004 4035 char *, mip->mi_name,
4005 4036 int, g, int, group_info.mgi_count);
4006 4037 err = EINVAL;
4007 4038 goto bail;
4008 4039 }
4009 4040 break;
4010 4041 }
4011 4042 break;
4012 4043 case MAC_GROUP_TYPE_STATIC:
4013 4044 /*
4014 4045 * Note that an empty group is allowed, e.g., an aggr
4015 4046 * would start with an empty group.
4016 4047 */
4017 4048 break;
4018 4049 default:
4019 4050 /* unknown group type */
4020 4051 DTRACE_PROBE2(mac__init__rings__unknown__type,
4021 4052 char *, mip->mi_name,
4022 4053 int, cap_rings->mr_group_type);
4023 4054 err = EINVAL;
4024 4055 goto bail;
4025 4056 }
4026 4057
4027 4058
4028 4059 /*
4029 4060 * Driver must register group->mgi_addmac/remmac() for rx groups
4030 4061 * to support multiple MAC addresses.
4031 4062 */
4032 4063 if (rtype == MAC_RING_TYPE_RX) {
4033 4064 if ((group_info.mgi_addmac == NULL) ||
4034 4065 (group_info.mgi_addmac == NULL)) {
4035 4066 goto bail;
4036 4067 }
4037 4068 }
4038 4069
4039 4070 /* Cache driver-supplied information */
4040 4071 group->mrg_info = group_info;
4041 4072
4042 4073 /* Update the group's status and group count. */
4043 4074 mac_set_group_state(group, MAC_GROUP_STATE_REGISTERED);
4044 4075 group_free++;
4045 4076
4046 4077 group->mrg_rings = NULL;
4047 4078 group->mrg_cur_count = 0;
4048 4079 mac_init_group(mip, group, group_info.mgi_count, cap_rings);
4049 4080 ring_left -= group_info.mgi_count;
4050 4081
4051 4082 /* The current group size should be equal to default value */
4052 4083 ASSERT(group->mrg_cur_count == group_info.mgi_count);
4053 4084 }
4054 4085
4055 4086 /* Build up a dummy group for free resources as a pool */
4056 4087 group = groups + grpcnt;
4057 4088
4058 4089 /* Prepare basic information of the group */
4059 4090 group->mrg_index = -1;
4060 4091 group->mrg_type = rtype;
4061 4092 group->mrg_state = MAC_GROUP_STATE_UNINIT;
4062 4093 group->mrg_mh = (mac_handle_t)mip;
4063 4094 group->mrg_next = NULL;
4064 4095
4065 4096 /*
4066 4097 * If there are ungrouped rings, allocate a continuous buffer for
4067 4098 * remaining resources.
4068 4099 */
4069 4100 if (ring_left != 0) {
4070 4101 group->mrg_rings = NULL;
4071 4102 group->mrg_cur_count = 0;
4072 4103 mac_init_group(mip, group, ring_left, cap_rings);
4073 4104
4074 4105 /* The current group size should be equal to ring_left */
4075 4106 ASSERT(group->mrg_cur_count == ring_left);
4076 4107
4077 4108 ring_left = 0;
4078 4109
4079 4110 /* Update this group's status */
4080 4111 mac_set_group_state(group, MAC_GROUP_STATE_REGISTERED);
4081 4112 } else
4082 4113 group->mrg_rings = NULL;
4083 4114
4084 4115 ASSERT(ring_left == 0);
4085 4116
4086 4117 bail:
4087 4118
4088 4119 /* Cache other important information to finalize the initialization */
4089 4120 switch (rtype) {
4090 4121 case MAC_RING_TYPE_RX:
4091 4122 mip->mi_rx_group_type = cap_rings->mr_group_type;
4092 4123 mip->mi_rx_group_count = cap_rings->mr_gnum;
4093 4124 mip->mi_rx_groups = groups;
4094 4125 mip->mi_rx_donor_grp = groups;
4095 4126 if (mip->mi_rx_group_type == MAC_GROUP_TYPE_DYNAMIC) {
4096 4127 /*
4097 4128 * The default ring is reserved since it is
4098 4129 * used for sending the broadcast etc. packets.
4099 4130 */
4100 4131 mip->mi_rxrings_avail =
4101 4132 mip->mi_rx_groups->mrg_cur_count - 1;
4102 4133 mip->mi_rxrings_rsvd = 1;
4103 4134 }
4104 4135 /*
4105 4136 * The default group cannot be reserved. It is used by
4106 4137 * all the clients that do not have an exclusive group.
4107 4138 */
4108 4139 mip->mi_rxhwclnt_avail = mip->mi_rx_group_count - 1;
4109 4140 mip->mi_rxhwclnt_used = 1;
4110 4141 break;
4111 4142 case MAC_RING_TYPE_TX:
4112 4143 mip->mi_tx_group_type = pseudo_txgrp ? MAC_GROUP_TYPE_DYNAMIC :
4113 4144 cap_rings->mr_group_type;
4114 4145 mip->mi_tx_group_count = grpcnt;
4115 4146 mip->mi_tx_group_free = group_free;
4116 4147 mip->mi_tx_groups = groups;
4117 4148
4118 4149 group = groups + grpcnt;
4119 4150 ring = group->mrg_rings;
4120 4151 /*
4121 4152 * The ring can be NULL in the case of aggr. Aggr will
4122 4153 * have an empty Tx group which will get populated
4123 4154 * later when pseudo Tx rings are added after
4124 4155 * mac_register() is done.
4125 4156 */
4126 4157 if (ring == NULL) {
4127 4158 ASSERT(mip->mi_state_flags & MIS_IS_AGGR);
4128 4159 /*
4129 4160 * pass the group to aggr so it can add Tx
4130 4161 * rings to the group later.
4131 4162 */
4132 4163 cap_rings->mr_gget(mip->mi_driver, rtype, 0, NULL,
4133 4164 (mac_group_handle_t)group);
4134 4165 /*
4135 4166 * Even though there are no rings at this time
4136 4167 * (rings will come later), set the group
4137 4168 * state to registered.
4138 4169 */
4139 4170 group->mrg_state = MAC_GROUP_STATE_REGISTERED;
4140 4171 } else {
4141 4172 /*
4142 4173 * Ring 0 is used as the default one and it could be
4143 4174 * assigned to a client as well.
4144 4175 */
4145 4176 while ((ring->mr_index != 0) && (ring->mr_next != NULL))
4146 4177 ring = ring->mr_next;
4147 4178 ASSERT(ring->mr_index == 0);
4148 4179 mip->mi_default_tx_ring = (mac_ring_handle_t)ring;
4149 4180 }
4150 4181 if (mip->mi_tx_group_type == MAC_GROUP_TYPE_DYNAMIC)
4151 4182 mip->mi_txrings_avail = group->mrg_cur_count - 1;
4152 4183 /*
4153 4184 * The default ring cannot be reserved.
4154 4185 */
4155 4186 mip->mi_txrings_rsvd = 1;
4156 4187 /*
4157 4188 * The default group cannot be reserved. It will be shared
4158 4189 * by clients that do not have an exclusive group.
4159 4190 */
4160 4191 mip->mi_txhwclnt_avail = mip->mi_tx_group_count;
4161 4192 mip->mi_txhwclnt_used = 1;
4162 4193 break;
4163 4194 default:
4164 4195 ASSERT(B_FALSE);
4165 4196 }
4166 4197
4167 4198 if (err != 0)
4168 4199 mac_free_rings(mip, rtype);
4169 4200
4170 4201 return (err);
4171 4202 }
4172 4203
4173 4204 /*
4174 4205 * The ddi interrupt handle could be shared amoung rings. If so, compare
4175 4206 * the new ring's ddi handle with the existing ones and set ddi_shared
4176 4207 * flag.
4177 4208 */
4178 4209 void
4179 4210 mac_compare_ddi_handle(mac_group_t *groups, uint_t grpcnt, mac_ring_t *cring)
4180 4211 {
4181 4212 mac_group_t *group;
4182 4213 mac_ring_t *ring;
4183 4214 ddi_intr_handle_t ddi_handle;
4184 4215 int g;
4185 4216
4186 4217 ddi_handle = cring->mr_info.mri_intr.mi_ddi_handle;
4187 4218 for (g = 0; g < grpcnt; g++) {
4188 4219 group = groups + g;
4189 4220 for (ring = group->mrg_rings; ring != NULL;
4190 4221 ring = ring->mr_next) {
4191 4222 if (ring == cring)
4192 4223 continue;
4193 4224 if (ring->mr_info.mri_intr.mi_ddi_handle ==
4194 4225 ddi_handle) {
4195 4226 if (cring->mr_type == MAC_RING_TYPE_RX &&
4196 4227 ring->mr_index == 0 &&
4197 4228 !ring->mr_info.mri_intr.mi_ddi_shared) {
4198 4229 ring->mr_info.mri_intr.mi_ddi_shared =
4199 4230 B_TRUE;
4200 4231 } else {
4201 4232 cring->mr_info.mri_intr.mi_ddi_shared =
4202 4233 B_TRUE;
4203 4234 }
4204 4235 return;
4205 4236 }
4206 4237 }
4207 4238 }
4208 4239 }
4209 4240
4210 4241 /*
4211 4242 * Called to free all groups of particular type (RX or TX). It's assumed that
4212 4243 * no clients are using these groups.
4213 4244 */
4214 4245 void
4215 4246 mac_free_rings(mac_impl_t *mip, mac_ring_type_t rtype)
4216 4247 {
4217 4248 mac_group_t *group, *groups;
4218 4249 uint_t group_count;
4219 4250
4220 4251 switch (rtype) {
4221 4252 case MAC_RING_TYPE_RX:
4222 4253 if (mip->mi_rx_groups == NULL)
4223 4254 return;
4224 4255
4225 4256 groups = mip->mi_rx_groups;
4226 4257 group_count = mip->mi_rx_group_count;
4227 4258
4228 4259 mip->mi_rx_groups = NULL;
4229 4260 mip->mi_rx_donor_grp = NULL;
4230 4261 mip->mi_rx_group_count = 0;
4231 4262 break;
4232 4263 case MAC_RING_TYPE_TX:
4233 4264 ASSERT(mip->mi_tx_group_count == mip->mi_tx_group_free);
4234 4265
4235 4266 if (mip->mi_tx_groups == NULL)
4236 4267 return;
4237 4268
4238 4269 groups = mip->mi_tx_groups;
4239 4270 group_count = mip->mi_tx_group_count;
4240 4271
4241 4272 mip->mi_tx_groups = NULL;
4242 4273 mip->mi_tx_group_count = 0;
4243 4274 mip->mi_tx_group_free = 0;
4244 4275 mip->mi_default_tx_ring = NULL;
4245 4276 break;
4246 4277 default:
4247 4278 ASSERT(B_FALSE);
4248 4279 }
4249 4280
4250 4281 for (group = groups; group != NULL; group = group->mrg_next) {
4251 4282 mac_ring_t *ring;
4252 4283
4253 4284 if (group->mrg_cur_count == 0)
4254 4285 continue;
4255 4286
4256 4287 ASSERT(group->mrg_rings != NULL);
4257 4288
4258 4289 while ((ring = group->mrg_rings) != NULL) {
4259 4290 group->mrg_rings = ring->mr_next;
4260 4291 mac_ring_free(mip, ring);
4261 4292 }
4262 4293 }
4263 4294
4264 4295 /* Free all the cached rings */
4265 4296 mac_ring_freeall(mip);
4266 4297 /* Free the block of group data strutures */
4267 4298 kmem_free(groups, sizeof (mac_group_t) * (group_count + 1));
4268 4299 }
4269 4300
4270 4301 /*
4271 4302 * Associate a MAC address with a receive group.
4272 4303 *
4273 4304 * The return value of this function should always be checked properly, because
4274 4305 * any type of failure could cause unexpected results. A group can be added
4275 4306 * or removed with a MAC address only after it has been reserved. Ideally,
4276 4307 * a successful reservation always leads to calling mac_group_addmac() to
4277 4308 * steer desired traffic. Failure of adding an unicast MAC address doesn't
4278 4309 * always imply that the group is functioning abnormally.
4279 4310 *
4280 4311 * Currently this function is called everywhere, and it reflects assumptions
4281 4312 * about MAC addresses in the implementation. CR 6735196.
4282 4313 */
4283 4314 int
4284 4315 mac_group_addmac(mac_group_t *group, const uint8_t *addr)
4285 4316 {
4286 4317 ASSERT(group->mrg_type == MAC_RING_TYPE_RX);
4287 4318 ASSERT(group->mrg_info.mgi_addmac != NULL);
4288 4319
4289 4320 return (group->mrg_info.mgi_addmac(group->mrg_info.mgi_driver, addr));
4290 4321 }
4291 4322
4292 4323 /*
4293 4324 * Remove the association between MAC address and receive group.
4294 4325 */
4295 4326 int
4296 4327 mac_group_remmac(mac_group_t *group, const uint8_t *addr)
4297 4328 {
4298 4329 ASSERT(group->mrg_type == MAC_RING_TYPE_RX);
4299 4330 ASSERT(group->mrg_info.mgi_remmac != NULL);
4300 4331
4301 4332 return (group->mrg_info.mgi_remmac(group->mrg_info.mgi_driver, addr));
4302 4333 }
4303 4334
4304 4335 /*
4305 4336 * This is the entry point for packets transmitted through the bridging code.
4306 4337 * If no bridge is in place, MAC_RING_TX transmits using tx ring. The 'rh'
4307 4338 * pointer may be NULL to select the default ring.
4308 4339 */
4309 4340 mblk_t *
4310 4341 mac_bridge_tx(mac_impl_t *mip, mac_ring_handle_t rh, mblk_t *mp)
4311 4342 {
4312 4343 mac_handle_t mh;
4313 4344
4314 4345 /*
4315 4346 * Once we take a reference on the bridge link, the bridge
4316 4347 * module itself can't unload, so the callback pointers are
4317 4348 * stable.
4318 4349 */
4319 4350 mutex_enter(&mip->mi_bridge_lock);
4320 4351 if ((mh = mip->mi_bridge_link) != NULL)
4321 4352 mac_bridge_ref_cb(mh, B_TRUE);
4322 4353 mutex_exit(&mip->mi_bridge_lock);
4323 4354 if (mh == NULL) {
4324 4355 MAC_RING_TX(mip, rh, mp, mp);
4325 4356 } else {
4326 4357 mp = mac_bridge_tx_cb(mh, rh, mp);
4327 4358 mac_bridge_ref_cb(mh, B_FALSE);
4328 4359 }
4329 4360
4330 4361 return (mp);
4331 4362 }
4332 4363
4333 4364 /*
4334 4365 * Find a ring from its index.
4335 4366 */
4336 4367 mac_ring_handle_t
4337 4368 mac_find_ring(mac_group_handle_t gh, int index)
4338 4369 {
4339 4370 mac_group_t *group = (mac_group_t *)gh;
4340 4371 mac_ring_t *ring = group->mrg_rings;
4341 4372
4342 4373 for (ring = group->mrg_rings; ring != NULL; ring = ring->mr_next)
4343 4374 if (ring->mr_index == index)
4344 4375 break;
4345 4376
4346 4377 return ((mac_ring_handle_t)ring);
4347 4378 }
4348 4379 /*
4349 4380 * Add a ring to an existing group.
4350 4381 *
4351 4382 * The ring must be either passed directly (for example if the ring
4352 4383 * movement is initiated by the framework), or specified through a driver
4353 4384 * index (for example when the ring is added by the driver.
4354 4385 *
4355 4386 * The caller needs to call mac_perim_enter() before calling this function.
4356 4387 */
4357 4388 int
4358 4389 i_mac_group_add_ring(mac_group_t *group, mac_ring_t *ring, int index)
4359 4390 {
4360 4391 mac_impl_t *mip = (mac_impl_t *)group->mrg_mh;
4361 4392 mac_capab_rings_t *cap_rings;
4362 4393 boolean_t driver_call = (ring == NULL);
4363 4394 mac_group_type_t group_type;
4364 4395 int ret = 0;
4365 4396 flow_entry_t *flent;
4366 4397
4367 4398 ASSERT(MAC_PERIM_HELD((mac_handle_t)mip));
4368 4399
4369 4400 switch (group->mrg_type) {
4370 4401 case MAC_RING_TYPE_RX:
4371 4402 cap_rings = &mip->mi_rx_rings_cap;
4372 4403 group_type = mip->mi_rx_group_type;
4373 4404 break;
4374 4405 case MAC_RING_TYPE_TX:
4375 4406 cap_rings = &mip->mi_tx_rings_cap;
4376 4407 group_type = mip->mi_tx_group_type;
4377 4408 break;
4378 4409 default:
4379 4410 ASSERT(B_FALSE);
4380 4411 }
4381 4412
4382 4413 /*
4383 4414 * There should be no ring with the same ring index in the target
4384 4415 * group.
4385 4416 */
4386 4417 ASSERT(mac_find_ring((mac_group_handle_t)group,
4387 4418 driver_call ? index : ring->mr_index) == NULL);
4388 4419
4389 4420 if (driver_call) {
4390 4421 /*
4391 4422 * The function is called as a result of a request from
4392 4423 * a driver to add a ring to an existing group, for example
4393 4424 * from the aggregation driver. Allocate a new mac_ring_t
4394 4425 * for that ring.
4395 4426 */
4396 4427 ring = mac_init_ring(mip, group, index, cap_rings);
4397 4428 ASSERT(group->mrg_state > MAC_GROUP_STATE_UNINIT);
4398 4429 } else {
4399 4430 /*
4400 4431 * The function is called as a result of a MAC layer request
4401 4432 * to add a ring to an existing group. In this case the
4402 4433 * ring is being moved between groups, which requires
4403 4434 * the underlying driver to support dynamic grouping,
4404 4435 * and the mac_ring_t already exists.
4405 4436 */
4406 4437 ASSERT(group_type == MAC_GROUP_TYPE_DYNAMIC);
4407 4438 ASSERT(group->mrg_driver == NULL ||
4408 4439 cap_rings->mr_gaddring != NULL);
4409 4440 ASSERT(ring->mr_gh == NULL);
4410 4441 }
4411 4442
4412 4443 /*
4413 4444 * At this point the ring should not be in use, and it should be
4414 4445 * of the right for the target group.
4415 4446 */
4416 4447 ASSERT(ring->mr_state < MR_INUSE);
4417 4448 ASSERT(ring->mr_srs == NULL);
4418 4449 ASSERT(ring->mr_type == group->mrg_type);
4419 4450
4420 4451 if (!driver_call) {
4421 4452 /*
4422 4453 * Add the driver level hardware ring if the process was not
4423 4454 * initiated by the driver, and the target group is not the
4424 4455 * group.
4425 4456 */
4426 4457 if (group->mrg_driver != NULL) {
4427 4458 cap_rings->mr_gaddring(group->mrg_driver,
4428 4459 ring->mr_driver, ring->mr_type);
4429 4460 }
4430 4461
4431 4462 /*
4432 4463 * Insert the ring ahead existing rings.
4433 4464 */
4434 4465 ring->mr_next = group->mrg_rings;
4435 4466 group->mrg_rings = ring;
4436 4467 ring->mr_gh = (mac_group_handle_t)group;
4437 4468 group->mrg_cur_count++;
4438 4469 }
4439 4470
4440 4471 /*
4441 4472 * If the group has not been actively used, we're done.
4442 4473 */
4443 4474 if (group->mrg_index != -1 &&
4444 4475 group->mrg_state < MAC_GROUP_STATE_RESERVED)
4445 4476 return (0);
4446 4477
4447 4478 /*
4448 4479 * Start the ring if needed. Failure causes to undo the grouping action.
4449 4480 */
4450 4481 if (ring->mr_state != MR_INUSE) {
4451 4482 if ((ret = mac_start_ring(ring)) != 0) {
4452 4483 if (!driver_call) {
4453 4484 cap_rings->mr_gremring(group->mrg_driver,
4454 4485 ring->mr_driver, ring->mr_type);
4455 4486 }
4456 4487 group->mrg_cur_count--;
4457 4488 group->mrg_rings = ring->mr_next;
4458 4489
4459 4490 ring->mr_gh = NULL;
4460 4491
4461 4492 if (driver_call)
4462 4493 mac_ring_free(mip, ring);
4463 4494
4464 4495 return (ret);
4465 4496 }
4466 4497 }
4467 4498
4468 4499 /*
4469 4500 * Set up SRS/SR according to the ring type.
4470 4501 */
4471 4502 switch (ring->mr_type) {
4472 4503 case MAC_RING_TYPE_RX:
4473 4504 /*
4474 4505 * Setup SRS on top of the new ring if the group is
4475 4506 * reserved for someones exclusive use.
4476 4507 */
4477 4508 if (group->mrg_state == MAC_GROUP_STATE_RESERVED) {
4478 4509 mac_client_impl_t *mcip;
4479 4510
4480 4511 mcip = MAC_GROUP_ONLY_CLIENT(group);
4481 4512 /*
4482 4513 * Even though this group is reserved we migth still
4483 4514 * have multiple clients, i.e a VLAN shares the
4484 4515 * group with the primary mac client.
4485 4516 */
4486 4517 if (mcip != NULL) {
4487 4518 flent = mcip->mci_flent;
4488 4519 ASSERT(flent->fe_rx_srs_cnt > 0);
4489 4520 mac_rx_srs_group_setup(mcip, flent, SRST_LINK);
4490 4521 mac_fanout_setup(mcip, flent,
4491 4522 MCIP_RESOURCE_PROPS(mcip), mac_rx_deliver,
4492 4523 mcip, NULL, NULL);
4493 4524 } else {
4494 4525 ring->mr_classify_type = MAC_SW_CLASSIFIER;
4495 4526 }
4496 4527 }
4497 4528 break;
4498 4529 case MAC_RING_TYPE_TX:
4499 4530 {
4500 4531 mac_grp_client_t *mgcp = group->mrg_clients;
4501 4532 mac_client_impl_t *mcip;
4502 4533 mac_soft_ring_set_t *mac_srs;
4503 4534 mac_srs_tx_t *tx;
4504 4535
4505 4536 if (MAC_GROUP_NO_CLIENT(group)) {
4506 4537 if (ring->mr_state == MR_INUSE)
4507 4538 mac_stop_ring(ring);
4508 4539 ring->mr_flag = 0;
4509 4540 break;
4510 4541 }
4511 4542 /*
4512 4543 * If the rings are being moved to a group that has
4513 4544 * clients using it, then add the new rings to the
4514 4545 * clients SRS.
4515 4546 */
4516 4547 while (mgcp != NULL) {
4517 4548 boolean_t is_aggr;
4518 4549
4519 4550 mcip = mgcp->mgc_client;
4520 4551 flent = mcip->mci_flent;
4521 4552 is_aggr = (mcip->mci_state_flags & MCIS_IS_AGGR);
4522 4553 mac_srs = MCIP_TX_SRS(mcip);
4523 4554 tx = &mac_srs->srs_tx;
4524 4555 mac_tx_client_quiesce((mac_client_handle_t)mcip);
4525 4556 /*
4526 4557 * If we are growing from 1 to multiple rings.
4527 4558 */
4528 4559 if (tx->st_mode == SRS_TX_BW ||
4529 4560 tx->st_mode == SRS_TX_SERIALIZE ||
4530 4561 tx->st_mode == SRS_TX_DEFAULT) {
4531 4562 mac_ring_t *tx_ring = tx->st_arg2;
4532 4563
4533 4564 tx->st_arg2 = NULL;
4534 4565 mac_tx_srs_stat_recreate(mac_srs, B_TRUE);
4535 4566 mac_tx_srs_add_ring(mac_srs, tx_ring);
4536 4567 if (mac_srs->srs_type & SRST_BW_CONTROL) {
4537 4568 tx->st_mode = is_aggr ? SRS_TX_BW_AGGR :
4538 4569 SRS_TX_BW_FANOUT;
4539 4570 } else {
4540 4571 tx->st_mode = is_aggr ? SRS_TX_AGGR :
4541 4572 SRS_TX_FANOUT;
4542 4573 }
4543 4574 tx->st_func = mac_tx_get_func(tx->st_mode);
4544 4575 }
4545 4576 mac_tx_srs_add_ring(mac_srs, ring);
4546 4577 mac_fanout_setup(mcip, flent, MCIP_RESOURCE_PROPS(mcip),
4547 4578 mac_rx_deliver, mcip, NULL, NULL);
4548 4579 mac_tx_client_restart((mac_client_handle_t)mcip);
4549 4580 mgcp = mgcp->mgc_next;
4550 4581 }
4551 4582 break;
4552 4583 }
4553 4584 default:
4554 4585 ASSERT(B_FALSE);
4555 4586 }
4556 4587 /*
4557 4588 * For aggr, the default ring will be NULL to begin with. If it
4558 4589 * is NULL, then pick the first ring that gets added as the
4559 4590 * default ring. Any ring in an aggregation can be removed at
4560 4591 * any time (by the user action of removing a link) and if the
4561 4592 * current default ring gets removed, then a new one gets
4562 4593 * picked (see i_mac_group_rem_ring()).
4563 4594 */
4564 4595 if (mip->mi_state_flags & MIS_IS_AGGR &&
4565 4596 mip->mi_default_tx_ring == NULL &&
4566 4597 ring->mr_type == MAC_RING_TYPE_TX) {
4567 4598 mip->mi_default_tx_ring = (mac_ring_handle_t)ring;
4568 4599 }
4569 4600
4570 4601 MAC_RING_UNMARK(ring, MR_INCIPIENT);
4571 4602 return (0);
4572 4603 }
4573 4604
4574 4605 /*
4575 4606 * Remove a ring from it's current group. MAC internal function for dynamic
4576 4607 * grouping.
4577 4608 *
4578 4609 * The caller needs to call mac_perim_enter() before calling this function.
4579 4610 */
4580 4611 void
4581 4612 i_mac_group_rem_ring(mac_group_t *group, mac_ring_t *ring,
4582 4613 boolean_t driver_call)
4583 4614 {
4584 4615 mac_impl_t *mip = (mac_impl_t *)group->mrg_mh;
4585 4616 mac_capab_rings_t *cap_rings = NULL;
4586 4617 mac_group_type_t group_type;
4587 4618
4588 4619 ASSERT(MAC_PERIM_HELD((mac_handle_t)mip));
4589 4620
4590 4621 ASSERT(mac_find_ring((mac_group_handle_t)group,
4591 4622 ring->mr_index) == (mac_ring_handle_t)ring);
4592 4623 ASSERT((mac_group_t *)ring->mr_gh == group);
4593 4624 ASSERT(ring->mr_type == group->mrg_type);
4594 4625
4595 4626 if (ring->mr_state == MR_INUSE)
4596 4627 mac_stop_ring(ring);
4597 4628 switch (ring->mr_type) {
4598 4629 case MAC_RING_TYPE_RX:
4599 4630 group_type = mip->mi_rx_group_type;
4600 4631 cap_rings = &mip->mi_rx_rings_cap;
4601 4632
4602 4633 /*
4603 4634 * Only hardware classified packets hold a reference to the
4604 4635 * ring all the way up the Rx path. mac_rx_srs_remove()
4605 4636 * will take care of quiescing the Rx path and removing the
4606 4637 * SRS. The software classified path neither holds a reference
4607 4638 * nor any association with the ring in mac_rx.
4608 4639 */
4609 4640 if (ring->mr_srs != NULL) {
4610 4641 mac_rx_srs_remove(ring->mr_srs);
4611 4642 ring->mr_srs = NULL;
4612 4643 }
4613 4644
4614 4645 break;
4615 4646 case MAC_RING_TYPE_TX:
4616 4647 {
4617 4648 mac_grp_client_t *mgcp;
4618 4649 mac_client_impl_t *mcip;
4619 4650 mac_soft_ring_set_t *mac_srs;
4620 4651 mac_srs_tx_t *tx;
4621 4652 mac_ring_t *rem_ring;
4622 4653 mac_group_t *defgrp;
4623 4654 uint_t ring_info = 0;
4624 4655
4625 4656 /*
4626 4657 * For TX this function is invoked in three
4627 4658 * cases:
4628 4659 *
4629 4660 * 1) In the case of a failure during the
4630 4661 * initial creation of a group when a share is
4631 4662 * associated with a MAC client. So the SRS is not
4632 4663 * yet setup, and will be setup later after the
4633 4664 * group has been reserved and populated.
4634 4665 *
4635 4666 * 2) From mac_release_tx_group() when freeing
4636 4667 * a TX SRS.
4637 4668 *
4638 4669 * 3) In the case of aggr, when a port gets removed,
4639 4670 * the pseudo Tx rings that it exposed gets removed.
4640 4671 *
4641 4672 * In the first two cases the SRS and its soft
4642 4673 * rings are already quiesced.
4643 4674 */
4644 4675 if (driver_call) {
4645 4676 mac_client_impl_t *mcip;
4646 4677 mac_soft_ring_set_t *mac_srs;
4647 4678 mac_soft_ring_t *sringp;
4648 4679 mac_srs_tx_t *srs_tx;
4649 4680
4650 4681 if (mip->mi_state_flags & MIS_IS_AGGR &&
4651 4682 mip->mi_default_tx_ring ==
4652 4683 (mac_ring_handle_t)ring) {
4653 4684 /* pick a new default Tx ring */
4654 4685 mip->mi_default_tx_ring =
4655 4686 (group->mrg_rings != ring) ?
4656 4687 (mac_ring_handle_t)group->mrg_rings :
4657 4688 (mac_ring_handle_t)(ring->mr_next);
4658 4689 }
4659 4690 /* Presently only aggr case comes here */
4660 4691 if (group->mrg_state != MAC_GROUP_STATE_RESERVED)
4661 4692 break;
4662 4693
4663 4694 mcip = MAC_GROUP_ONLY_CLIENT(group);
4664 4695 ASSERT(mcip != NULL);
4665 4696 ASSERT(mcip->mci_state_flags & MCIS_IS_AGGR);
4666 4697 mac_srs = MCIP_TX_SRS(mcip);
4667 4698 ASSERT(mac_srs->srs_tx.st_mode == SRS_TX_AGGR ||
4668 4699 mac_srs->srs_tx.st_mode == SRS_TX_BW_AGGR);
4669 4700 srs_tx = &mac_srs->srs_tx;
4670 4701 /*
4671 4702 * Wakeup any callers blocked on this
4672 4703 * Tx ring due to flow control.
4673 4704 */
4674 4705 sringp = srs_tx->st_soft_rings[ring->mr_index];
4675 4706 ASSERT(sringp != NULL);
4676 4707 mac_tx_invoke_callbacks(mcip, (mac_tx_cookie_t)sringp);
4677 4708 mac_tx_client_quiesce((mac_client_handle_t)mcip);
4678 4709 mac_tx_srs_del_ring(mac_srs, ring);
4679 4710 mac_tx_client_restart((mac_client_handle_t)mcip);
4680 4711 break;
4681 4712 }
4682 4713 ASSERT(ring != (mac_ring_t *)mip->mi_default_tx_ring);
4683 4714 group_type = mip->mi_tx_group_type;
4684 4715 cap_rings = &mip->mi_tx_rings_cap;
4685 4716 /*
4686 4717 * See if we need to take it out of the MAC clients using
4687 4718 * this group
4688 4719 */
4689 4720 if (MAC_GROUP_NO_CLIENT(group))
4690 4721 break;
4691 4722 mgcp = group->mrg_clients;
4692 4723 defgrp = MAC_DEFAULT_TX_GROUP(mip);
4693 4724 while (mgcp != NULL) {
4694 4725 mcip = mgcp->mgc_client;
4695 4726 mac_srs = MCIP_TX_SRS(mcip);
4696 4727 tx = &mac_srs->srs_tx;
4697 4728 mac_tx_client_quiesce((mac_client_handle_t)mcip);
4698 4729 /*
4699 4730 * If we are here when removing rings from the
4700 4731 * defgroup, mac_reserve_tx_ring would have
4701 4732 * already deleted the ring from the MAC
4702 4733 * clients in the group.
4703 4734 */
4704 4735 if (group != defgrp) {
4705 4736 mac_tx_invoke_callbacks(mcip,
4706 4737 (mac_tx_cookie_t)
4707 4738 mac_tx_srs_get_soft_ring(mac_srs, ring));
4708 4739 mac_tx_srs_del_ring(mac_srs, ring);
4709 4740 }
4710 4741 /*
4711 4742 * Additionally, if we are left with only
4712 4743 * one ring in the group after this, we need
4713 4744 * to modify the mode etc. to. (We haven't
4714 4745 * yet taken the ring out, so we check with 2).
4715 4746 */
4716 4747 if (group->mrg_cur_count == 2) {
4717 4748 if (ring->mr_next == NULL)
4718 4749 rem_ring = group->mrg_rings;
4719 4750 else
4720 4751 rem_ring = ring->mr_next;
4721 4752 mac_tx_invoke_callbacks(mcip,
4722 4753 (mac_tx_cookie_t)
4723 4754 mac_tx_srs_get_soft_ring(mac_srs,
4724 4755 rem_ring));
4725 4756 mac_tx_srs_del_ring(mac_srs, rem_ring);
4726 4757 if (rem_ring->mr_state != MR_INUSE) {
4727 4758 (void) mac_start_ring(rem_ring);
4728 4759 }
4729 4760 tx->st_arg2 = (void *)rem_ring;
4730 4761 mac_tx_srs_stat_recreate(mac_srs, B_FALSE);
4731 4762 ring_info = mac_hwring_getinfo(
4732 4763 (mac_ring_handle_t)rem_ring);
4733 4764 /*
4734 4765 * We are shrinking from multiple
4735 4766 * to 1 ring.
4736 4767 */
4737 4768 if (mac_srs->srs_type & SRST_BW_CONTROL) {
4738 4769 tx->st_mode = SRS_TX_BW;
4739 4770 } else if (mac_tx_serialize ||
4740 4771 (ring_info & MAC_RING_TX_SERIALIZE)) {
4741 4772 tx->st_mode = SRS_TX_SERIALIZE;
4742 4773 } else {
4743 4774 tx->st_mode = SRS_TX_DEFAULT;
4744 4775 }
4745 4776 tx->st_func = mac_tx_get_func(tx->st_mode);
4746 4777 }
4747 4778 mac_tx_client_restart((mac_client_handle_t)mcip);
4748 4779 mgcp = mgcp->mgc_next;
4749 4780 }
4750 4781 break;
4751 4782 }
4752 4783 default:
4753 4784 ASSERT(B_FALSE);
4754 4785 }
4755 4786
4756 4787 /*
4757 4788 * Remove the ring from the group.
4758 4789 */
4759 4790 if (ring == group->mrg_rings)
4760 4791 group->mrg_rings = ring->mr_next;
4761 4792 else {
4762 4793 mac_ring_t *pre;
4763 4794
4764 4795 pre = group->mrg_rings;
4765 4796 while (pre->mr_next != ring)
4766 4797 pre = pre->mr_next;
4767 4798 pre->mr_next = ring->mr_next;
4768 4799 }
4769 4800 group->mrg_cur_count--;
4770 4801
4771 4802 if (!driver_call) {
4772 4803 ASSERT(group_type == MAC_GROUP_TYPE_DYNAMIC);
4773 4804 ASSERT(group->mrg_driver == NULL ||
4774 4805 cap_rings->mr_gremring != NULL);
4775 4806
4776 4807 /*
4777 4808 * Remove the driver level hardware ring.
4778 4809 */
4779 4810 if (group->mrg_driver != NULL) {
4780 4811 cap_rings->mr_gremring(group->mrg_driver,
4781 4812 ring->mr_driver, ring->mr_type);
4782 4813 }
4783 4814 }
4784 4815
4785 4816 ring->mr_gh = NULL;
4786 4817 if (driver_call)
4787 4818 mac_ring_free(mip, ring);
4788 4819 else
4789 4820 ring->mr_flag = 0;
4790 4821 }
4791 4822
4792 4823 /*
4793 4824 * Move a ring to the target group. If needed, remove the ring from the group
4794 4825 * that it currently belongs to.
4795 4826 *
4796 4827 * The caller need to enter MAC's perimeter by calling mac_perim_enter().
4797 4828 */
4798 4829 static int
4799 4830 mac_group_mov_ring(mac_impl_t *mip, mac_group_t *d_group, mac_ring_t *ring)
4800 4831 {
4801 4832 mac_group_t *s_group = (mac_group_t *)ring->mr_gh;
4802 4833 int rv;
4803 4834
4804 4835 ASSERT(MAC_PERIM_HELD((mac_handle_t)mip));
4805 4836 ASSERT(d_group != NULL);
4806 4837 ASSERT(s_group->mrg_mh == d_group->mrg_mh);
4807 4838
4808 4839 if (s_group == d_group)
4809 4840 return (0);
4810 4841
4811 4842 /*
4812 4843 * Remove it from current group first.
4813 4844 */
4814 4845 if (s_group != NULL)
4815 4846 i_mac_group_rem_ring(s_group, ring, B_FALSE);
4816 4847
4817 4848 /*
4818 4849 * Add it to the new group.
4819 4850 */
4820 4851 rv = i_mac_group_add_ring(d_group, ring, 0);
4821 4852 if (rv != 0) {
4822 4853 /*
4823 4854 * Failed to add ring back to source group. If
4824 4855 * that fails, the ring is stuck in limbo, log message.
4825 4856 */
4826 4857 if (i_mac_group_add_ring(s_group, ring, 0)) {
4827 4858 cmn_err(CE_WARN, "%s: failed to move ring %p\n",
4828 4859 mip->mi_name, (void *)ring);
4829 4860 }
4830 4861 }
4831 4862
4832 4863 return (rv);
4833 4864 }
4834 4865
4835 4866 /*
4836 4867 * Find a MAC address according to its value.
4837 4868 */
4838 4869 mac_address_t *
4839 4870 mac_find_macaddr(mac_impl_t *mip, uint8_t *mac_addr)
4840 4871 {
4841 4872 mac_address_t *map;
4842 4873
4843 4874 ASSERT(MAC_PERIM_HELD((mac_handle_t)mip));
4844 4875
4845 4876 for (map = mip->mi_addresses; map != NULL; map = map->ma_next) {
4846 4877 if (bcmp(mac_addr, map->ma_addr, map->ma_len) == 0)
4847 4878 break;
4848 4879 }
4849 4880
4850 4881 return (map);
4851 4882 }
4852 4883
4853 4884 /*
4854 4885 * Check whether the MAC address is shared by multiple clients.
4855 4886 */
4856 4887 boolean_t
4857 4888 mac_check_macaddr_shared(mac_address_t *map)
4858 4889 {
4859 4890 ASSERT(MAC_PERIM_HELD((mac_handle_t)map->ma_mip));
4860 4891
4861 4892 return (map->ma_nusers > 1);
4862 4893 }
4863 4894
4864 4895 /*
4865 4896 * Remove the specified MAC address from the MAC address list and free it.
4866 4897 */
4867 4898 static void
4868 4899 mac_free_macaddr(mac_address_t *map)
4869 4900 {
4870 4901 mac_impl_t *mip = map->ma_mip;
4871 4902
4872 4903 ASSERT(MAC_PERIM_HELD((mac_handle_t)mip));
4873 4904 ASSERT(mip->mi_addresses != NULL);
4874 4905
4875 4906 map = mac_find_macaddr(mip, map->ma_addr);
4876 4907
4877 4908 ASSERT(map != NULL);
4878 4909 ASSERT(map->ma_nusers == 0);
4879 4910
4880 4911 if (map == mip->mi_addresses) {
4881 4912 mip->mi_addresses = map->ma_next;
4882 4913 } else {
4883 4914 mac_address_t *pre;
4884 4915
4885 4916 pre = mip->mi_addresses;
4886 4917 while (pre->ma_next != map)
4887 4918 pre = pre->ma_next;
4888 4919 pre->ma_next = map->ma_next;
4889 4920 }
4890 4921
4891 4922 kmem_free(map, sizeof (mac_address_t));
4892 4923 }
4893 4924
4894 4925 /*
4895 4926 * Add a MAC address reference for a client. If the desired MAC address
4896 4927 * exists, add a reference to it. Otherwise, add the new address by adding
4897 4928 * it to a reserved group or setting promiscuous mode. Won't try different
4898 4929 * group is the group is non-NULL, so the caller must explictly share
4899 4930 * default group when needed.
4900 4931 *
4901 4932 * Note, the primary MAC address is initialized at registration time, so
4902 4933 * to add it to default group only need to activate it if its reference
4903 4934 * count is still zero. Also, some drivers may not have advertised RINGS
4904 4935 * capability.
4905 4936 */
4906 4937 int
4907 4938 mac_add_macaddr(mac_impl_t *mip, mac_group_t *group, uint8_t *mac_addr,
4908 4939 boolean_t use_hw)
4909 4940 {
4910 4941 mac_address_t *map;
4911 4942 int err = 0;
4912 4943 boolean_t allocated_map = B_FALSE;
4913 4944
4914 4945 ASSERT(MAC_PERIM_HELD((mac_handle_t)mip));
4915 4946
4916 4947 map = mac_find_macaddr(mip, mac_addr);
4917 4948
4918 4949 /*
4919 4950 * If the new MAC address has not been added. Allocate a new one
4920 4951 * and set it up.
4921 4952 */
4922 4953 if (map == NULL) {
4923 4954 map = kmem_zalloc(sizeof (mac_address_t), KM_SLEEP);
4924 4955 map->ma_len = mip->mi_type->mt_addr_length;
4925 4956 bcopy(mac_addr, map->ma_addr, map->ma_len);
4926 4957 map->ma_nusers = 0;
4927 4958 map->ma_group = group;
4928 4959 map->ma_mip = mip;
4929 4960
4930 4961 /* add the new MAC address to the head of the address list */
4931 4962 map->ma_next = mip->mi_addresses;
4932 4963 mip->mi_addresses = map;
4933 4964
4934 4965 allocated_map = B_TRUE;
4935 4966 }
4936 4967
4937 4968 ASSERT(map->ma_group == NULL || map->ma_group == group);
4938 4969 if (map->ma_group == NULL)
4939 4970 map->ma_group = group;
4940 4971
4941 4972 /*
4942 4973 * If the MAC address is already in use, simply account for the
4943 4974 * new client.
4944 4975 */
4945 4976 if (map->ma_nusers++ > 0)
4946 4977 return (0);
4947 4978
4948 4979 /*
4949 4980 * Activate this MAC address by adding it to the reserved group.
4950 4981 */
4951 4982 if (group != NULL) {
4952 4983 err = mac_group_addmac(group, (const uint8_t *)mac_addr);
4953 4984 if (err == 0) {
4954 4985 map->ma_type = MAC_ADDRESS_TYPE_UNICAST_CLASSIFIED;
4955 4986 return (0);
4956 4987 }
4957 4988 }
4958 4989
4959 4990 /*
4960 4991 * The MAC address addition failed. If the client requires a
4961 4992 * hardware classified MAC address, fail the operation.
4962 4993 */
4963 4994 if (use_hw) {
4964 4995 err = ENOSPC;
4965 4996 goto bail;
4966 4997 }
4967 4998
4968 4999 /*
4969 5000 * Try promiscuous mode.
4970 5001 *
4971 5002 * For drivers that don't advertise RINGS capability, do
4972 5003 * nothing for the primary address.
4973 5004 */
4974 5005 if ((group == NULL) &&
4975 5006 (bcmp(map->ma_addr, mip->mi_addr, map->ma_len) == 0)) {
4976 5007 map->ma_type = MAC_ADDRESS_TYPE_UNICAST_CLASSIFIED;
4977 5008 return (0);
4978 5009 }
4979 5010
4980 5011 /*
4981 5012 * Enable promiscuous mode in order to receive traffic
4982 5013 * to the new MAC address.
4983 5014 */
4984 5015 if ((err = i_mac_promisc_set(mip, B_TRUE)) == 0) {
4985 5016 map->ma_type = MAC_ADDRESS_TYPE_UNICAST_PROMISC;
4986 5017 return (0);
4987 5018 }
4988 5019
4989 5020 /*
4990 5021 * Free the MAC address that could not be added. Don't free
4991 5022 * a pre-existing address, it could have been the entry
4992 5023 * for the primary MAC address which was pre-allocated by
4993 5024 * mac_init_macaddr(), and which must remain on the list.
4994 5025 */
4995 5026 bail:
4996 5027 map->ma_nusers--;
4997 5028 if (allocated_map)
4998 5029 mac_free_macaddr(map);
4999 5030 return (err);
5000 5031 }
5001 5032
5002 5033 /*
5003 5034 * Remove a reference to a MAC address. This may cause to remove the MAC
5004 5035 * address from an associated group or to turn off promiscuous mode.
5005 5036 * The caller needs to handle the failure properly.
5006 5037 */
5007 5038 int
5008 5039 mac_remove_macaddr(mac_address_t *map)
5009 5040 {
5010 5041 mac_impl_t *mip = map->ma_mip;
5011 5042 int err = 0;
5012 5043
5013 5044 ASSERT(MAC_PERIM_HELD((mac_handle_t)mip));
5014 5045
5015 5046 ASSERT(map == mac_find_macaddr(mip, map->ma_addr));
5016 5047
5017 5048 /*
5018 5049 * If it's not the last client using this MAC address, only update
5019 5050 * the MAC clients count.
5020 5051 */
5021 5052 if (--map->ma_nusers > 0)
5022 5053 return (0);
5023 5054
5024 5055 /*
5025 5056 * The MAC address is no longer used by any MAC client, so remove
5026 5057 * it from its associated group, or turn off promiscuous mode
5027 5058 * if it was enabled for the MAC address.
5028 5059 */
5029 5060 switch (map->ma_type) {
5030 5061 case MAC_ADDRESS_TYPE_UNICAST_CLASSIFIED:
5031 5062 /*
5032 5063 * Don't free the preset primary address for drivers that
5033 5064 * don't advertise RINGS capability.
5034 5065 */
5035 5066 if (map->ma_group == NULL)
5036 5067 return (0);
5037 5068
5038 5069 err = mac_group_remmac(map->ma_group, map->ma_addr);
5039 5070 if (err == 0)
5040 5071 map->ma_group = NULL;
5041 5072 break;
5042 5073 case MAC_ADDRESS_TYPE_UNICAST_PROMISC:
5043 5074 err = i_mac_promisc_set(mip, B_FALSE);
5044 5075 break;
5045 5076 default:
5046 5077 ASSERT(B_FALSE);
5047 5078 }
5048 5079
5049 5080 if (err != 0)
5050 5081 return (err);
5051 5082
5052 5083 /*
5053 5084 * We created MAC address for the primary one at registration, so we
5054 5085 * won't free it here. mac_fini_macaddr() will take care of it.
5055 5086 */
5056 5087 if (bcmp(map->ma_addr, mip->mi_addr, map->ma_len) != 0)
5057 5088 mac_free_macaddr(map);
5058 5089
5059 5090 return (0);
5060 5091 }
5061 5092
5062 5093 /*
5063 5094 * Update an existing MAC address. The caller need to make sure that the new
5064 5095 * value has not been used.
5065 5096 */
5066 5097 int
5067 5098 mac_update_macaddr(mac_address_t *map, uint8_t *mac_addr)
5068 5099 {
5069 5100 mac_impl_t *mip = map->ma_mip;
5070 5101 int err = 0;
5071 5102
5072 5103 ASSERT(MAC_PERIM_HELD((mac_handle_t)mip));
5073 5104 ASSERT(mac_find_macaddr(mip, mac_addr) == NULL);
5074 5105
5075 5106 switch (map->ma_type) {
5076 5107 case MAC_ADDRESS_TYPE_UNICAST_CLASSIFIED:
5077 5108 /*
5078 5109 * Update the primary address for drivers that are not
5079 5110 * RINGS capable.
5080 5111 */
5081 5112 if (mip->mi_rx_groups == NULL) {
5082 5113 err = mip->mi_unicst(mip->mi_driver, (const uint8_t *)
5083 5114 mac_addr);
5084 5115 if (err != 0)
5085 5116 return (err);
5086 5117 break;
5087 5118 }
5088 5119
5089 5120 /*
5090 5121 * If this MAC address is not currently in use,
5091 5122 * simply break out and update the value.
5092 5123 */
5093 5124 if (map->ma_nusers == 0)
5094 5125 break;
5095 5126
5096 5127 /*
5097 5128 * Need to replace the MAC address associated with a group.
5098 5129 */
5099 5130 err = mac_group_remmac(map->ma_group, map->ma_addr);
5100 5131 if (err != 0)
5101 5132 return (err);
5102 5133
5103 5134 err = mac_group_addmac(map->ma_group, mac_addr);
5104 5135
5105 5136 /*
5106 5137 * Failure hints hardware error. The MAC layer needs to
5107 5138 * have error notification facility to handle this.
5108 5139 * Now, simply try to restore the value.
5109 5140 */
5110 5141 if (err != 0)
5111 5142 (void) mac_group_addmac(map->ma_group, map->ma_addr);
5112 5143
5113 5144 break;
5114 5145 case MAC_ADDRESS_TYPE_UNICAST_PROMISC:
5115 5146 /*
5116 5147 * Need to do nothing more if in promiscuous mode.
5117 5148 */
5118 5149 break;
5119 5150 default:
5120 5151 ASSERT(B_FALSE);
5121 5152 }
5122 5153
5123 5154 /*
5124 5155 * Successfully replaced the MAC address.
5125 5156 */
5126 5157 if (err == 0)
5127 5158 bcopy(mac_addr, map->ma_addr, map->ma_len);
5128 5159
5129 5160 return (err);
5130 5161 }
5131 5162
5132 5163 /*
5133 5164 * Freshen the MAC address with new value. Its caller must have updated the
5134 5165 * hardware MAC address before calling this function.
5135 5166 * This funcitons is supposed to be used to handle the MAC address change
5136 5167 * notification from underlying drivers.
5137 5168 */
5138 5169 void
5139 5170 mac_freshen_macaddr(mac_address_t *map, uint8_t *mac_addr)
5140 5171 {
5141 5172 mac_impl_t *mip = map->ma_mip;
5142 5173
5143 5174 ASSERT(MAC_PERIM_HELD((mac_handle_t)mip));
5144 5175 ASSERT(mac_find_macaddr(mip, mac_addr) == NULL);
5145 5176
5146 5177 /*
5147 5178 * Freshen the MAC address with new value.
5148 5179 */
5149 5180 bcopy(mac_addr, map->ma_addr, map->ma_len);
5150 5181 bcopy(mac_addr, mip->mi_addr, map->ma_len);
5151 5182
5152 5183 /*
5153 5184 * Update all MAC clients that share this MAC address.
5154 5185 */
5155 5186 mac_unicast_update_clients(mip, map);
5156 5187 }
5157 5188
5158 5189 /*
5159 5190 * Set up the primary MAC address.
5160 5191 */
5161 5192 void
5162 5193 mac_init_macaddr(mac_impl_t *mip)
5163 5194 {
5164 5195 mac_address_t *map;
5165 5196
5166 5197 /*
5167 5198 * The reference count is initialized to zero, until it's really
5168 5199 * activated.
5169 5200 */
5170 5201 map = kmem_zalloc(sizeof (mac_address_t), KM_SLEEP);
5171 5202 map->ma_len = mip->mi_type->mt_addr_length;
5172 5203 bcopy(mip->mi_addr, map->ma_addr, map->ma_len);
5173 5204
5174 5205 /*
5175 5206 * If driver advertises RINGS capability, it shouldn't have initialized
5176 5207 * its primary MAC address. For other drivers, including VNIC, the
5177 5208 * primary address must work after registration.
5178 5209 */
5179 5210 if (mip->mi_rx_groups == NULL)
5180 5211 map->ma_type = MAC_ADDRESS_TYPE_UNICAST_CLASSIFIED;
5181 5212
5182 5213 map->ma_mip = mip;
5183 5214
5184 5215 mip->mi_addresses = map;
5185 5216 }
5186 5217
5187 5218 /*
5188 5219 * Clean up the primary MAC address. Note, only one primary MAC address
5189 5220 * is allowed. All other MAC addresses must have been freed appropriately.
5190 5221 */
5191 5222 void
5192 5223 mac_fini_macaddr(mac_impl_t *mip)
5193 5224 {
5194 5225 mac_address_t *map = mip->mi_addresses;
5195 5226
5196 5227 if (map == NULL)
5197 5228 return;
5198 5229
5199 5230 /*
5200 5231 * If mi_addresses is initialized, there should be exactly one
5201 5232 * entry left on the list with no users.
5202 5233 */
5203 5234 ASSERT(map->ma_nusers == 0);
5204 5235 ASSERT(map->ma_next == NULL);
5205 5236
5206 5237 kmem_free(map, sizeof (mac_address_t));
5207 5238 mip->mi_addresses = NULL;
5208 5239 }
5209 5240
5210 5241 /*
5211 5242 * Logging related functions.
5212 5243 *
5213 5244 * Note that Kernel statistics have been extended to maintain fine
5214 5245 * granularity of statistics viz. hardware lane, software lane, fanout
5215 5246 * stats etc. However, extended accounting continues to support only
5216 5247 * aggregate statistics like before.
5217 5248 */
5218 5249
5219 5250 /* Write the flow description to a netinfo_t record */
5220 5251 static netinfo_t *
5221 5252 mac_write_flow_desc(flow_entry_t *flent, mac_client_impl_t *mcip)
5222 5253 {
5223 5254 netinfo_t *ninfo;
5224 5255 net_desc_t *ndesc;
5225 5256 flow_desc_t *fdesc;
5226 5257 mac_resource_props_t *mrp;
5227 5258
5228 5259 ninfo = kmem_zalloc(sizeof (netinfo_t), KM_NOSLEEP);
5229 5260 if (ninfo == NULL)
5230 5261 return (NULL);
5231 5262 ndesc = kmem_zalloc(sizeof (net_desc_t), KM_NOSLEEP);
5232 5263 if (ndesc == NULL) {
5233 5264 kmem_free(ninfo, sizeof (netinfo_t));
5234 5265 return (NULL);
5235 5266 }
5236 5267
5237 5268 /*
5238 5269 * Grab the fe_lock to see a self-consistent fe_flow_desc.
5239 5270 * Updates to the fe_flow_desc are done under the fe_lock
5240 5271 */
5241 5272 mutex_enter(&flent->fe_lock);
5242 5273 fdesc = &flent->fe_flow_desc;
5243 5274 mrp = &flent->fe_resource_props;
5244 5275
5245 5276 ndesc->nd_name = flent->fe_flow_name;
5246 5277 ndesc->nd_devname = mcip->mci_name;
5247 5278 bcopy(fdesc->fd_src_mac, ndesc->nd_ehost, ETHERADDRL);
5248 5279 bcopy(fdesc->fd_dst_mac, ndesc->nd_edest, ETHERADDRL);
5249 5280 ndesc->nd_sap = htonl(fdesc->fd_sap);
5250 5281 ndesc->nd_isv4 = (uint8_t)fdesc->fd_ipversion == IPV4_VERSION;
5251 5282 ndesc->nd_bw_limit = mrp->mrp_maxbw;
5252 5283 if (ndesc->nd_isv4) {
5253 5284 ndesc->nd_saddr[3] = htonl(fdesc->fd_local_addr.s6_addr32[3]);
5254 5285 ndesc->nd_daddr[3] = htonl(fdesc->fd_remote_addr.s6_addr32[3]);
5255 5286 } else {
5256 5287 bcopy(&fdesc->fd_local_addr, ndesc->nd_saddr, IPV6_ADDR_LEN);
5257 5288 bcopy(&fdesc->fd_remote_addr, ndesc->nd_daddr, IPV6_ADDR_LEN);
5258 5289 }
5259 5290 ndesc->nd_sport = htons(fdesc->fd_local_port);
5260 5291 ndesc->nd_dport = htons(fdesc->fd_remote_port);
5261 5292 ndesc->nd_protocol = (uint8_t)fdesc->fd_protocol;
5262 5293 mutex_exit(&flent->fe_lock);
5263 5294
5264 5295 ninfo->ni_record = ndesc;
5265 5296 ninfo->ni_size = sizeof (net_desc_t);
5266 5297 ninfo->ni_type = EX_NET_FLDESC_REC;
5267 5298
5268 5299 return (ninfo);
5269 5300 }
5270 5301
5271 5302 /* Write the flow statistics to a netinfo_t record */
5272 5303 static netinfo_t *
5273 5304 mac_write_flow_stats(flow_entry_t *flent)
5274 5305 {
5275 5306 netinfo_t *ninfo;
5276 5307 net_stat_t *nstat;
5277 5308 mac_soft_ring_set_t *mac_srs;
5278 5309 mac_rx_stats_t *mac_rx_stat;
5279 5310 mac_tx_stats_t *mac_tx_stat;
5280 5311 int i;
5281 5312
5282 5313 ninfo = kmem_zalloc(sizeof (netinfo_t), KM_NOSLEEP);
5283 5314 if (ninfo == NULL)
5284 5315 return (NULL);
5285 5316 nstat = kmem_zalloc(sizeof (net_stat_t), KM_NOSLEEP);
5286 5317 if (nstat == NULL) {
5287 5318 kmem_free(ninfo, sizeof (netinfo_t));
5288 5319 return (NULL);
5289 5320 }
5290 5321
5291 5322 nstat->ns_name = flent->fe_flow_name;
5292 5323 for (i = 0; i < flent->fe_rx_srs_cnt; i++) {
5293 5324 mac_srs = (mac_soft_ring_set_t *)flent->fe_rx_srs[i];
5294 5325 mac_rx_stat = &mac_srs->srs_rx.sr_stat;
5295 5326
5296 5327 nstat->ns_ibytes += mac_rx_stat->mrs_intrbytes +
5297 5328 mac_rx_stat->mrs_pollbytes + mac_rx_stat->mrs_lclbytes;
5298 5329 nstat->ns_ipackets += mac_rx_stat->mrs_intrcnt +
5299 5330 mac_rx_stat->mrs_pollcnt + mac_rx_stat->mrs_lclcnt;
5300 5331 nstat->ns_oerrors += mac_rx_stat->mrs_ierrors;
5301 5332 }
5302 5333
5303 5334 mac_srs = (mac_soft_ring_set_t *)(flent->fe_tx_srs);
5304 5335 if (mac_srs != NULL) {
5305 5336 mac_tx_stat = &mac_srs->srs_tx.st_stat;
5306 5337
5307 5338 nstat->ns_obytes = mac_tx_stat->mts_obytes;
5308 5339 nstat->ns_opackets = mac_tx_stat->mts_opackets;
5309 5340 nstat->ns_oerrors = mac_tx_stat->mts_oerrors;
5310 5341 }
5311 5342
5312 5343 ninfo->ni_record = nstat;
5313 5344 ninfo->ni_size = sizeof (net_stat_t);
5314 5345 ninfo->ni_type = EX_NET_FLSTAT_REC;
5315 5346
5316 5347 return (ninfo);
5317 5348 }
5318 5349
5319 5350 /* Write the link description to a netinfo_t record */
5320 5351 static netinfo_t *
5321 5352 mac_write_link_desc(mac_client_impl_t *mcip)
5322 5353 {
5323 5354 netinfo_t *ninfo;
5324 5355 net_desc_t *ndesc;
5325 5356 flow_entry_t *flent = mcip->mci_flent;
5326 5357
5327 5358 ninfo = kmem_zalloc(sizeof (netinfo_t), KM_NOSLEEP);
5328 5359 if (ninfo == NULL)
5329 5360 return (NULL);
5330 5361 ndesc = kmem_zalloc(sizeof (net_desc_t), KM_NOSLEEP);
5331 5362 if (ndesc == NULL) {
5332 5363 kmem_free(ninfo, sizeof (netinfo_t));
5333 5364 return (NULL);
5334 5365 }
5335 5366
5336 5367 ndesc->nd_name = mcip->mci_name;
5337 5368 ndesc->nd_devname = mcip->mci_name;
5338 5369 ndesc->nd_isv4 = B_TRUE;
5339 5370 /*
5340 5371 * Grab the fe_lock to see a self-consistent fe_flow_desc.
5341 5372 * Updates to the fe_flow_desc are done under the fe_lock
5342 5373 * after removing the flent from the flow table.
5343 5374 */
5344 5375 mutex_enter(&flent->fe_lock);
5345 5376 bcopy(flent->fe_flow_desc.fd_src_mac, ndesc->nd_ehost, ETHERADDRL);
5346 5377 mutex_exit(&flent->fe_lock);
5347 5378
5348 5379 ninfo->ni_record = ndesc;
5349 5380 ninfo->ni_size = sizeof (net_desc_t);
5350 5381 ninfo->ni_type = EX_NET_LNDESC_REC;
5351 5382
5352 5383 return (ninfo);
5353 5384 }
5354 5385
5355 5386 /* Write the link statistics to a netinfo_t record */
5356 5387 static netinfo_t *
5357 5388 mac_write_link_stats(mac_client_impl_t *mcip)
5358 5389 {
5359 5390 netinfo_t *ninfo;
5360 5391 net_stat_t *nstat;
5361 5392 flow_entry_t *flent;
5362 5393 mac_soft_ring_set_t *mac_srs;
5363 5394 mac_rx_stats_t *mac_rx_stat;
5364 5395 mac_tx_stats_t *mac_tx_stat;
5365 5396 int i;
5366 5397
5367 5398 ninfo = kmem_zalloc(sizeof (netinfo_t), KM_NOSLEEP);
5368 5399 if (ninfo == NULL)
5369 5400 return (NULL);
5370 5401 nstat = kmem_zalloc(sizeof (net_stat_t), KM_NOSLEEP);
5371 5402 if (nstat == NULL) {
5372 5403 kmem_free(ninfo, sizeof (netinfo_t));
5373 5404 return (NULL);
5374 5405 }
5375 5406
5376 5407 nstat->ns_name = mcip->mci_name;
5377 5408 flent = mcip->mci_flent;
5378 5409 if (flent != NULL) {
5379 5410 for (i = 0; i < flent->fe_rx_srs_cnt; i++) {
5380 5411 mac_srs = (mac_soft_ring_set_t *)flent->fe_rx_srs[i];
5381 5412 mac_rx_stat = &mac_srs->srs_rx.sr_stat;
5382 5413
5383 5414 nstat->ns_ibytes += mac_rx_stat->mrs_intrbytes +
5384 5415 mac_rx_stat->mrs_pollbytes +
5385 5416 mac_rx_stat->mrs_lclbytes;
5386 5417 nstat->ns_ipackets += mac_rx_stat->mrs_intrcnt +
5387 5418 mac_rx_stat->mrs_pollcnt + mac_rx_stat->mrs_lclcnt;
5388 5419 nstat->ns_oerrors += mac_rx_stat->mrs_ierrors;
5389 5420 }
5390 5421 }
5391 5422
5392 5423 mac_srs = (mac_soft_ring_set_t *)(mcip->mci_flent->fe_tx_srs);
5393 5424 if (mac_srs != NULL) {
5394 5425 mac_tx_stat = &mac_srs->srs_tx.st_stat;
5395 5426
5396 5427 nstat->ns_obytes = mac_tx_stat->mts_obytes;
5397 5428 nstat->ns_opackets = mac_tx_stat->mts_opackets;
5398 5429 nstat->ns_oerrors = mac_tx_stat->mts_oerrors;
5399 5430 }
5400 5431
5401 5432 ninfo->ni_record = nstat;
5402 5433 ninfo->ni_size = sizeof (net_stat_t);
5403 5434 ninfo->ni_type = EX_NET_LNSTAT_REC;
5404 5435
5405 5436 return (ninfo);
5406 5437 }
5407 5438
5408 5439 typedef struct i_mac_log_state_s {
5409 5440 boolean_t mi_last;
5410 5441 int mi_fenable;
5411 5442 int mi_lenable;
5412 5443 list_t *mi_list;
5413 5444 } i_mac_log_state_t;
5414 5445
5415 5446 /*
5416 5447 * For a given flow, if the description has not been logged before, do it now.
5417 5448 * If it is a VNIC, then we have collected information about it from the MAC
5418 5449 * table, so skip it.
5419 5450 *
5420 5451 * Called through mac_flow_walk_nolock()
5421 5452 *
5422 5453 * Return 0 if successful.
5423 5454 */
5424 5455 static int
5425 5456 mac_log_flowinfo(flow_entry_t *flent, void *arg)
5426 5457 {
5427 5458 mac_client_impl_t *mcip = flent->fe_mcip;
5428 5459 i_mac_log_state_t *lstate = arg;
5429 5460 netinfo_t *ninfo;
5430 5461
5431 5462 if (mcip == NULL)
5432 5463 return (0);
5433 5464
5434 5465 /*
5435 5466 * If the name starts with "vnic", and fe_user_generated is true (to
5436 5467 * exclude the mcast and active flow entries created implicitly for
5437 5468 * a vnic, it is a VNIC flow. i.e. vnic1 is a vnic flow,
5438 5469 * vnic/bge1/mcast1 is not and neither is vnic/bge1/active.
5439 5470 */
5440 5471 if (strncasecmp(flent->fe_flow_name, "vnic", 4) == 0 &&
5441 5472 (flent->fe_type & FLOW_USER) != 0) {
5442 5473 return (0);
5443 5474 }
5444 5475
5445 5476 if (!flent->fe_desc_logged) {
5446 5477 /*
5447 5478 * We don't return error because we want to continue the
5448 5479 * walk in case this is the last walk which means we
5449 5480 * need to reset fe_desc_logged in all the flows.
5450 5481 */
5451 5482 if ((ninfo = mac_write_flow_desc(flent, mcip)) == NULL)
5452 5483 return (0);
5453 5484 list_insert_tail(lstate->mi_list, ninfo);
5454 5485 flent->fe_desc_logged = B_TRUE;
5455 5486 }
5456 5487
5457 5488 /*
5458 5489 * Regardless of the error, we want to proceed in case we have to
5459 5490 * reset fe_desc_logged.
5460 5491 */
5461 5492 ninfo = mac_write_flow_stats(flent);
5462 5493 if (ninfo == NULL)
5463 5494 return (-1);
5464 5495
5465 5496 list_insert_tail(lstate->mi_list, ninfo);
5466 5497
5467 5498 if (mcip != NULL && !(mcip->mci_state_flags & MCIS_DESC_LOGGED))
5468 5499 flent->fe_desc_logged = B_FALSE;
5469 5500
5470 5501 return (0);
5471 5502 }
5472 5503
5473 5504 /*
5474 5505 * Log the description for each mac client of this mac_impl_t, if it
5475 5506 * hasn't already been done. Additionally, log statistics for the link as
5476 5507 * well. Walk the flow table and log information for each flow as well.
5477 5508 * If it is the last walk (mci_last), then we turn off mci_desc_logged (and
5478 5509 * also fe_desc_logged, if flow logging is on) since we want to log the
5479 5510 * description if and when logging is restarted.
5480 5511 *
5481 5512 * Return 0 upon success or -1 upon failure
5482 5513 */
5483 5514 static int
5484 5515 i_mac_impl_log(mac_impl_t *mip, i_mac_log_state_t *lstate)
5485 5516 {
5486 5517 mac_client_impl_t *mcip;
5487 5518 netinfo_t *ninfo;
5488 5519
5489 5520 i_mac_perim_enter(mip);
5490 5521 /*
5491 5522 * Only walk the client list for NIC and etherstub
5492 5523 */
5493 5524 if ((mip->mi_state_flags & MIS_DISABLED) ||
5494 5525 ((mip->mi_state_flags & MIS_IS_VNIC) &&
5495 5526 (mac_get_lower_mac_handle((mac_handle_t)mip) != NULL))) {
5496 5527 i_mac_perim_exit(mip);
5497 5528 return (0);
5498 5529 }
5499 5530
5500 5531 for (mcip = mip->mi_clients_list; mcip != NULL;
5501 5532 mcip = mcip->mci_client_next) {
5502 5533 if (!MCIP_DATAPATH_SETUP(mcip))
5503 5534 continue;
5504 5535 if (lstate->mi_lenable) {
5505 5536 if (!(mcip->mci_state_flags & MCIS_DESC_LOGGED)) {
5506 5537 ninfo = mac_write_link_desc(mcip);
5507 5538 if (ninfo == NULL) {
5508 5539 /*
5509 5540 * We can't terminate it if this is the last
5510 5541 * walk, else there might be some links with
5511 5542 * mi_desc_logged set to true, which means
5512 5543 * their description won't be logged the next
5513 5544 * time logging is started (similarly for the
5514 5545 * flows within such links). We can continue
5515 5546 * without walking the flow table (i.e. to
5516 5547 * set fe_desc_logged to false) because we
5517 5548 * won't have written any flow stuff for this
5518 5549 * link as we haven't logged the link itself.
5519 5550 */
5520 5551 i_mac_perim_exit(mip);
5521 5552 if (lstate->mi_last)
5522 5553 return (0);
5523 5554 else
5524 5555 return (-1);
5525 5556 }
5526 5557 mcip->mci_state_flags |= MCIS_DESC_LOGGED;
5527 5558 list_insert_tail(lstate->mi_list, ninfo);
5528 5559 }
5529 5560 }
5530 5561
5531 5562 ninfo = mac_write_link_stats(mcip);
5532 5563 if (ninfo == NULL && !lstate->mi_last) {
5533 5564 i_mac_perim_exit(mip);
5534 5565 return (-1);
5535 5566 }
5536 5567 list_insert_tail(lstate->mi_list, ninfo);
5537 5568
5538 5569 if (lstate->mi_last)
5539 5570 mcip->mci_state_flags &= ~MCIS_DESC_LOGGED;
5540 5571
5541 5572 if (lstate->mi_fenable) {
5542 5573 if (mcip->mci_subflow_tab != NULL) {
5543 5574 (void) mac_flow_walk_nolock(
5544 5575 mcip->mci_subflow_tab, mac_log_flowinfo,
5545 5576 lstate);
5546 5577 }
5547 5578 }
5548 5579 }
5549 5580 i_mac_perim_exit(mip);
5550 5581 return (0);
5551 5582 }
5552 5583
5553 5584 /*
5554 5585 * modhash walker function to add a mac_impl_t to a list
5555 5586 */
5556 5587 /*ARGSUSED*/
5557 5588 static uint_t
5558 5589 i_mac_impl_list_walker(mod_hash_key_t key, mod_hash_val_t *val, void *arg)
5559 5590 {
5560 5591 list_t *list = (list_t *)arg;
5561 5592 mac_impl_t *mip = (mac_impl_t *)val;
5562 5593
5563 5594 if ((mip->mi_state_flags & MIS_DISABLED) == 0) {
5564 5595 list_insert_tail(list, mip);
5565 5596 mip->mi_ref++;
5566 5597 }
5567 5598
5568 5599 return (MH_WALK_CONTINUE);
5569 5600 }
5570 5601
5571 5602 void
5572 5603 i_mac_log_info(list_t *net_log_list, i_mac_log_state_t *lstate)
5573 5604 {
5574 5605 list_t mac_impl_list;
5575 5606 mac_impl_t *mip;
5576 5607 netinfo_t *ninfo;
5577 5608
5578 5609 /* Create list of mac_impls */
5579 5610 ASSERT(RW_LOCK_HELD(&i_mac_impl_lock));
5580 5611 list_create(&mac_impl_list, sizeof (mac_impl_t), offsetof(mac_impl_t,
5581 5612 mi_node));
5582 5613 mod_hash_walk(i_mac_impl_hash, i_mac_impl_list_walker, &mac_impl_list);
5583 5614 rw_exit(&i_mac_impl_lock);
5584 5615
5585 5616 /* Create log entries for each mac_impl */
5586 5617 for (mip = list_head(&mac_impl_list); mip != NULL;
5587 5618 mip = list_next(&mac_impl_list, mip)) {
5588 5619 if (i_mac_impl_log(mip, lstate) != 0)
5589 5620 continue;
5590 5621 }
5591 5622
5592 5623 /* Remove elements and destroy list of mac_impls */
5593 5624 rw_enter(&i_mac_impl_lock, RW_WRITER);
5594 5625 while ((mip = list_remove_tail(&mac_impl_list)) != NULL) {
5595 5626 mip->mi_ref--;
5596 5627 }
5597 5628 rw_exit(&i_mac_impl_lock);
5598 5629 list_destroy(&mac_impl_list);
5599 5630
5600 5631 /*
5601 5632 * Write log entries to files outside of locks, free associated
5602 5633 * structures, and remove entries from the list.
5603 5634 */
5604 5635 while ((ninfo = list_head(net_log_list)) != NULL) {
5605 5636 (void) exacct_commit_netinfo(ninfo->ni_record, ninfo->ni_type);
5606 5637 list_remove(net_log_list, ninfo);
5607 5638 kmem_free(ninfo->ni_record, ninfo->ni_size);
5608 5639 kmem_free(ninfo, sizeof (*ninfo));
5609 5640 }
5610 5641 list_destroy(net_log_list);
5611 5642 }
5612 5643
5613 5644 /*
5614 5645 * The timer thread that runs every mac_logging_interval seconds and logs
5615 5646 * link and/or flow information.
5616 5647 */
5617 5648 /* ARGSUSED */
5618 5649 void
5619 5650 mac_log_linkinfo(void *arg)
5620 5651 {
5621 5652 i_mac_log_state_t lstate;
5622 5653 list_t net_log_list;
5623 5654
5624 5655 list_create(&net_log_list, sizeof (netinfo_t),
5625 5656 offsetof(netinfo_t, ni_link));
5626 5657
5627 5658 rw_enter(&i_mac_impl_lock, RW_READER);
5628 5659 if (!mac_flow_log_enable && !mac_link_log_enable) {
5629 5660 rw_exit(&i_mac_impl_lock);
5630 5661 return;
5631 5662 }
5632 5663 lstate.mi_fenable = mac_flow_log_enable;
5633 5664 lstate.mi_lenable = mac_link_log_enable;
5634 5665 lstate.mi_last = B_FALSE;
5635 5666 lstate.mi_list = &net_log_list;
5636 5667
5637 5668 /* Write log entries for each mac_impl in the list */
5638 5669 i_mac_log_info(&net_log_list, &lstate);
5639 5670
5640 5671 if (mac_flow_log_enable || mac_link_log_enable) {
5641 5672 mac_logging_timer = timeout(mac_log_linkinfo, NULL,
5642 5673 SEC_TO_TICK(mac_logging_interval));
5643 5674 }
5644 5675 }
5645 5676
5646 5677 typedef struct i_mac_fastpath_state_s {
5647 5678 boolean_t mf_disable;
5648 5679 int mf_err;
5649 5680 } i_mac_fastpath_state_t;
5650 5681
5651 5682 /* modhash walker function to enable or disable fastpath */
5652 5683 /*ARGSUSED*/
5653 5684 static uint_t
5654 5685 i_mac_fastpath_walker(mod_hash_key_t key, mod_hash_val_t *val,
5655 5686 void *arg)
5656 5687 {
5657 5688 i_mac_fastpath_state_t *state = arg;
5658 5689 mac_handle_t mh = (mac_handle_t)val;
5659 5690
5660 5691 if (state->mf_disable)
5661 5692 state->mf_err = mac_fastpath_disable(mh);
5662 5693 else
5663 5694 mac_fastpath_enable(mh);
5664 5695
5665 5696 return (state->mf_err == 0 ? MH_WALK_CONTINUE : MH_WALK_TERMINATE);
5666 5697 }
5667 5698
5668 5699 /*
5669 5700 * Start the logging timer.
5670 5701 */
5671 5702 int
5672 5703 mac_start_logusage(mac_logtype_t type, uint_t interval)
5673 5704 {
5674 5705 i_mac_fastpath_state_t dstate = {B_TRUE, 0};
5675 5706 i_mac_fastpath_state_t estate = {B_FALSE, 0};
5676 5707 int err;
5677 5708
5678 5709 rw_enter(&i_mac_impl_lock, RW_WRITER);
5679 5710 switch (type) {
5680 5711 case MAC_LOGTYPE_FLOW:
5681 5712 if (mac_flow_log_enable) {
5682 5713 rw_exit(&i_mac_impl_lock);
5683 5714 return (0);
5684 5715 }
5685 5716 /* FALLTHRU */
5686 5717 case MAC_LOGTYPE_LINK:
5687 5718 if (mac_link_log_enable) {
5688 5719 rw_exit(&i_mac_impl_lock);
5689 5720 return (0);
5690 5721 }
5691 5722 break;
5692 5723 default:
5693 5724 ASSERT(0);
5694 5725 }
5695 5726
5696 5727 /* Disable fastpath */
5697 5728 mod_hash_walk(i_mac_impl_hash, i_mac_fastpath_walker, &dstate);
5698 5729 if ((err = dstate.mf_err) != 0) {
5699 5730 /* Reenable fastpath */
5700 5731 mod_hash_walk(i_mac_impl_hash, i_mac_fastpath_walker, &estate);
5701 5732 rw_exit(&i_mac_impl_lock);
5702 5733 return (err);
5703 5734 }
5704 5735
5705 5736 switch (type) {
5706 5737 case MAC_LOGTYPE_FLOW:
5707 5738 mac_flow_log_enable = B_TRUE;
5708 5739 /* FALLTHRU */
5709 5740 case MAC_LOGTYPE_LINK:
5710 5741 mac_link_log_enable = B_TRUE;
5711 5742 break;
5712 5743 }
5713 5744
5714 5745 mac_logging_interval = interval;
5715 5746 rw_exit(&i_mac_impl_lock);
5716 5747 mac_log_linkinfo(NULL);
5717 5748 return (0);
5718 5749 }
5719 5750
5720 5751 /*
5721 5752 * Stop the logging timer if both link and flow logging are turned off.
5722 5753 */
5723 5754 void
5724 5755 mac_stop_logusage(mac_logtype_t type)
5725 5756 {
5726 5757 i_mac_log_state_t lstate;
5727 5758 i_mac_fastpath_state_t estate = {B_FALSE, 0};
5728 5759 list_t net_log_list;
5729 5760
5730 5761 list_create(&net_log_list, sizeof (netinfo_t),
5731 5762 offsetof(netinfo_t, ni_link));
5732 5763
5733 5764 rw_enter(&i_mac_impl_lock, RW_WRITER);
5734 5765
5735 5766 lstate.mi_fenable = mac_flow_log_enable;
5736 5767 lstate.mi_lenable = mac_link_log_enable;
5737 5768 lstate.mi_list = &net_log_list;
5738 5769
5739 5770 /* Last walk */
5740 5771 lstate.mi_last = B_TRUE;
5741 5772
5742 5773 switch (type) {
5743 5774 case MAC_LOGTYPE_FLOW:
5744 5775 if (lstate.mi_fenable) {
5745 5776 ASSERT(mac_link_log_enable);
5746 5777 mac_flow_log_enable = B_FALSE;
5747 5778 mac_link_log_enable = B_FALSE;
5748 5779 break;
5749 5780 }
5750 5781 /* FALLTHRU */
5751 5782 case MAC_LOGTYPE_LINK:
5752 5783 if (!lstate.mi_lenable || mac_flow_log_enable) {
5753 5784 rw_exit(&i_mac_impl_lock);
5754 5785 return;
5755 5786 }
5756 5787 mac_link_log_enable = B_FALSE;
5757 5788 break;
5758 5789 default:
5759 5790 ASSERT(0);
5760 5791 }
5761 5792
5762 5793 /* Reenable fastpath */
5763 5794 mod_hash_walk(i_mac_impl_hash, i_mac_fastpath_walker, &estate);
5764 5795
5765 5796 (void) untimeout(mac_logging_timer);
5766 5797 mac_logging_timer = 0;
5767 5798
5768 5799 /* Write log entries for each mac_impl in the list */
5769 5800 i_mac_log_info(&net_log_list, &lstate);
5770 5801 }
5771 5802
5772 5803 /*
5773 5804 * Walk the rx and tx SRS/SRs for a flow and update the priority value.
5774 5805 */
5775 5806 void
5776 5807 mac_flow_update_priority(mac_client_impl_t *mcip, flow_entry_t *flent)
5777 5808 {
5778 5809 pri_t pri;
5779 5810 int count;
5780 5811 mac_soft_ring_set_t *mac_srs;
5781 5812
5782 5813 if (flent->fe_rx_srs_cnt <= 0)
5783 5814 return;
5784 5815
5785 5816 if (((mac_soft_ring_set_t *)flent->fe_rx_srs[0])->srs_type ==
5786 5817 SRST_FLOW) {
5787 5818 pri = FLOW_PRIORITY(mcip->mci_min_pri,
5788 5819 mcip->mci_max_pri,
5789 5820 flent->fe_resource_props.mrp_priority);
5790 5821 } else {
5791 5822 pri = mcip->mci_max_pri;
5792 5823 }
5793 5824
5794 5825 for (count = 0; count < flent->fe_rx_srs_cnt; count++) {
5795 5826 mac_srs = flent->fe_rx_srs[count];
5796 5827 mac_update_srs_priority(mac_srs, pri);
5797 5828 }
5798 5829 /*
5799 5830 * If we have a Tx SRS, we need to modify all the threads associated
5800 5831 * with it.
5801 5832 */
5802 5833 if (flent->fe_tx_srs != NULL)
5803 5834 mac_update_srs_priority(flent->fe_tx_srs, pri);
5804 5835 }
5805 5836
5806 5837 /*
5807 5838 * RX and TX rings are reserved according to different semantics depending
5808 5839 * on the requests from the MAC clients and type of rings:
5809 5840 *
5810 5841 * On the Tx side, by default we reserve individual rings, independently from
5811 5842 * the groups.
5812 5843 *
5813 5844 * On the Rx side, the reservation is at the granularity of the group
5814 5845 * of rings, and used for v12n level 1 only. It has a special case for the
5815 5846 * primary client.
5816 5847 *
5817 5848 * If a share is allocated to a MAC client, we allocate a TX group and an
5818 5849 * RX group to the client, and assign TX rings and RX rings to these
5819 5850 * groups according to information gathered from the driver through
5820 5851 * the share capability.
5821 5852 *
5822 5853 * The foreseable evolution of Rx rings will handle v12n level 2 and higher
5823 5854 * to allocate individual rings out of a group and program the hw classifier
5824 5855 * based on IP address or higher level criteria.
5825 5856 */
5826 5857
5827 5858 /*
5828 5859 * mac_reserve_tx_ring()
5829 5860 * Reserve a unused ring by marking it with MR_INUSE state.
5830 5861 * As reserved, the ring is ready to function.
5831 5862 *
5832 5863 * Notes for Hybrid I/O:
5833 5864 *
5834 5865 * If a specific ring is needed, it is specified through the desired_ring
5835 5866 * argument. Otherwise that argument is set to NULL.
5836 5867 * If the desired ring was previous allocated to another client, this
5837 5868 * function swaps it with a new ring from the group of unassigned rings.
5838 5869 */
5839 5870 mac_ring_t *
5840 5871 mac_reserve_tx_ring(mac_impl_t *mip, mac_ring_t *desired_ring)
5841 5872 {
5842 5873 mac_group_t *group;
5843 5874 mac_grp_client_t *mgcp;
5844 5875 mac_client_impl_t *mcip;
5845 5876 mac_soft_ring_set_t *srs;
5846 5877
5847 5878 ASSERT(MAC_PERIM_HELD((mac_handle_t)mip));
5848 5879
5849 5880 /*
5850 5881 * Find an available ring and start it before changing its status.
5851 5882 * The unassigned rings are at the end of the mi_tx_groups
5852 5883 * array.
5853 5884 */
5854 5885 group = MAC_DEFAULT_TX_GROUP(mip);
5855 5886
5856 5887 /* Can't take the default ring out of the default group */
5857 5888 ASSERT(desired_ring != (mac_ring_t *)mip->mi_default_tx_ring);
5858 5889
5859 5890 if (desired_ring->mr_state == MR_FREE) {
5860 5891 ASSERT(MAC_GROUP_NO_CLIENT(group));
5861 5892 if (mac_start_ring(desired_ring) != 0)
5862 5893 return (NULL);
5863 5894 return (desired_ring);
5864 5895 }
5865 5896 /*
5866 5897 * There are clients using this ring, so let's move the clients
5867 5898 * away from using this ring.
5868 5899 */
5869 5900 for (mgcp = group->mrg_clients; mgcp != NULL; mgcp = mgcp->mgc_next) {
5870 5901 mcip = mgcp->mgc_client;
5871 5902 mac_tx_client_quiesce((mac_client_handle_t)mcip);
5872 5903 srs = MCIP_TX_SRS(mcip);
5873 5904 ASSERT(mac_tx_srs_ring_present(srs, desired_ring));
5874 5905 mac_tx_invoke_callbacks(mcip,
5875 5906 (mac_tx_cookie_t)mac_tx_srs_get_soft_ring(srs,
5876 5907 desired_ring));
5877 5908 mac_tx_srs_del_ring(srs, desired_ring);
5878 5909 mac_tx_client_restart((mac_client_handle_t)mcip);
5879 5910 }
5880 5911 return (desired_ring);
5881 5912 }
5882 5913
5883 5914 /*
5884 5915 * For a reserved group with multiple clients, return the primary client.
5885 5916 */
5886 5917 static mac_client_impl_t *
5887 5918 mac_get_grp_primary(mac_group_t *grp)
5888 5919 {
5889 5920 mac_grp_client_t *mgcp = grp->mrg_clients;
5890 5921 mac_client_impl_t *mcip;
5891 5922
5892 5923 while (mgcp != NULL) {
5893 5924 mcip = mgcp->mgc_client;
5894 5925 if (mcip->mci_flent->fe_type & FLOW_PRIMARY_MAC)
5895 5926 return (mcip);
5896 5927 mgcp = mgcp->mgc_next;
5897 5928 }
5898 5929 return (NULL);
5899 5930 }
5900 5931
5901 5932 /*
5902 5933 * Hybrid I/O specifies the ring that should be given to a share.
5903 5934 * If the ring is already used by clients, then we need to release
5904 5935 * the ring back to the default group so that we can give it to
5905 5936 * the share. This means the clients using this ring now get a
5906 5937 * replacement ring. If there aren't any replacement rings, this
5907 5938 * function returns a failure.
5908 5939 */
5909 5940 static int
5910 5941 mac_reclaim_ring_from_grp(mac_impl_t *mip, mac_ring_type_t ring_type,
5911 5942 mac_ring_t *ring, mac_ring_t **rings, int nrings)
5912 5943 {
5913 5944 mac_group_t *group = (mac_group_t *)ring->mr_gh;
5914 5945 mac_resource_props_t *mrp;
5915 5946 mac_client_impl_t *mcip;
5916 5947 mac_group_t *defgrp;
5917 5948 mac_ring_t *tring;
5918 5949 mac_group_t *tgrp;
5919 5950 int i;
5920 5951 int j;
5921 5952
5922 5953 mcip = MAC_GROUP_ONLY_CLIENT(group);
5923 5954 if (mcip == NULL)
5924 5955 mcip = mac_get_grp_primary(group);
5925 5956 ASSERT(mcip != NULL);
5926 5957 ASSERT(mcip->mci_share == NULL);
5927 5958
5928 5959 mrp = MCIP_RESOURCE_PROPS(mcip);
5929 5960 if (ring_type == MAC_RING_TYPE_RX) {
5930 5961 defgrp = mip->mi_rx_donor_grp;
5931 5962 if ((mrp->mrp_mask & MRP_RX_RINGS) == 0) {
5932 5963 /* Need to put this mac client in the default group */
5933 5964 if (mac_rx_switch_group(mcip, group, defgrp) != 0)
5934 5965 return (ENOSPC);
5935 5966 } else {
5936 5967 /*
5937 5968 * Switch this ring with some other ring from
5938 5969 * the default group.
5939 5970 */
5940 5971 for (tring = defgrp->mrg_rings; tring != NULL;
5941 5972 tring = tring->mr_next) {
5942 5973 if (tring->mr_index == 0)
5943 5974 continue;
5944 5975 for (j = 0; j < nrings; j++) {
5945 5976 if (rings[j] == tring)
5946 5977 break;
5947 5978 }
5948 5979 if (j >= nrings)
5949 5980 break;
5950 5981 }
5951 5982 if (tring == NULL)
5952 5983 return (ENOSPC);
5953 5984 if (mac_group_mov_ring(mip, group, tring) != 0)
5954 5985 return (ENOSPC);
5955 5986 if (mac_group_mov_ring(mip, defgrp, ring) != 0) {
5956 5987 (void) mac_group_mov_ring(mip, defgrp, tring);
5957 5988 return (ENOSPC);
5958 5989 }
5959 5990 }
5960 5991 ASSERT(ring->mr_gh == (mac_group_handle_t)defgrp);
5961 5992 return (0);
5962 5993 }
5963 5994
5964 5995 defgrp = MAC_DEFAULT_TX_GROUP(mip);
5965 5996 if (ring == (mac_ring_t *)mip->mi_default_tx_ring) {
5966 5997 /*
5967 5998 * See if we can get a spare ring to replace the default
5968 5999 * ring.
5969 6000 */
5970 6001 if (defgrp->mrg_cur_count == 1) {
5971 6002 /*
5972 6003 * Need to get a ring from another client, see if
5973 6004 * there are any clients that can be moved to
5974 6005 * the default group, thereby freeing some rings.
5975 6006 */
5976 6007 for (i = 0; i < mip->mi_tx_group_count; i++) {
5977 6008 tgrp = &mip->mi_tx_groups[i];
5978 6009 if (tgrp->mrg_state ==
5979 6010 MAC_GROUP_STATE_REGISTERED) {
5980 6011 continue;
5981 6012 }
5982 6013 mcip = MAC_GROUP_ONLY_CLIENT(tgrp);
5983 6014 if (mcip == NULL)
5984 6015 mcip = mac_get_grp_primary(tgrp);
5985 6016 ASSERT(mcip != NULL);
5986 6017 mrp = MCIP_RESOURCE_PROPS(mcip);
5987 6018 if ((mrp->mrp_mask & MRP_TX_RINGS) == 0) {
5988 6019 ASSERT(tgrp->mrg_cur_count == 1);
5989 6020 /*
5990 6021 * If this ring is part of the
5991 6022 * rings asked by the share we cannot
5992 6023 * use it as the default ring.
5993 6024 */
5994 6025 for (j = 0; j < nrings; j++) {
5995 6026 if (rings[j] == tgrp->mrg_rings)
5996 6027 break;
5997 6028 }
5998 6029 if (j < nrings)
5999 6030 continue;
6000 6031 mac_tx_client_quiesce(
6001 6032 (mac_client_handle_t)mcip);
6002 6033 mac_tx_switch_group(mcip, tgrp,
6003 6034 defgrp);
6004 6035 mac_tx_client_restart(
6005 6036 (mac_client_handle_t)mcip);
6006 6037 break;
6007 6038 }
6008 6039 }
6009 6040 /*
6010 6041 * All the rings are reserved, can't give up the
6011 6042 * default ring.
6012 6043 */
6013 6044 if (defgrp->mrg_cur_count <= 1)
6014 6045 return (ENOSPC);
6015 6046 }
6016 6047 /*
6017 6048 * Swap the default ring with another.
6018 6049 */
6019 6050 for (tring = defgrp->mrg_rings; tring != NULL;
6020 6051 tring = tring->mr_next) {
6021 6052 /*
6022 6053 * If this ring is part of the rings asked by the
6023 6054 * share we cannot use it as the default ring.
6024 6055 */
6025 6056 for (j = 0; j < nrings; j++) {
6026 6057 if (rings[j] == tring)
6027 6058 break;
6028 6059 }
6029 6060 if (j >= nrings)
6030 6061 break;
6031 6062 }
6032 6063 ASSERT(tring != NULL);
6033 6064 mip->mi_default_tx_ring = (mac_ring_handle_t)tring;
6034 6065 return (0);
6035 6066 }
6036 6067 /*
6037 6068 * The Tx ring is with a group reserved by a MAC client. See if
6038 6069 * we can swap it.
6039 6070 */
6040 6071 ASSERT(group->mrg_state == MAC_GROUP_STATE_RESERVED);
6041 6072 mcip = MAC_GROUP_ONLY_CLIENT(group);
6042 6073 if (mcip == NULL)
6043 6074 mcip = mac_get_grp_primary(group);
6044 6075 ASSERT(mcip != NULL);
6045 6076 mrp = MCIP_RESOURCE_PROPS(mcip);
6046 6077 mac_tx_client_quiesce((mac_client_handle_t)mcip);
6047 6078 if ((mrp->mrp_mask & MRP_TX_RINGS) == 0) {
6048 6079 ASSERT(group->mrg_cur_count == 1);
6049 6080 /* Put this mac client in the default group */
6050 6081 mac_tx_switch_group(mcip, group, defgrp);
6051 6082 } else {
6052 6083 /*
6053 6084 * Switch this ring with some other ring from
6054 6085 * the default group.
6055 6086 */
6056 6087 for (tring = defgrp->mrg_rings; tring != NULL;
6057 6088 tring = tring->mr_next) {
6058 6089 if (tring == (mac_ring_t *)mip->mi_default_tx_ring)
6059 6090 continue;
6060 6091 /*
6061 6092 * If this ring is part of the rings asked by the
6062 6093 * share we cannot use it for swapping.
6063 6094 */
6064 6095 for (j = 0; j < nrings; j++) {
6065 6096 if (rings[j] == tring)
6066 6097 break;
6067 6098 }
6068 6099 if (j >= nrings)
6069 6100 break;
6070 6101 }
6071 6102 if (tring == NULL) {
6072 6103 mac_tx_client_restart((mac_client_handle_t)mcip);
6073 6104 return (ENOSPC);
6074 6105 }
6075 6106 if (mac_group_mov_ring(mip, group, tring) != 0) {
6076 6107 mac_tx_client_restart((mac_client_handle_t)mcip);
6077 6108 return (ENOSPC);
6078 6109 }
6079 6110 if (mac_group_mov_ring(mip, defgrp, ring) != 0) {
6080 6111 (void) mac_group_mov_ring(mip, defgrp, tring);
6081 6112 mac_tx_client_restart((mac_client_handle_t)mcip);
6082 6113 return (ENOSPC);
6083 6114 }
6084 6115 }
6085 6116 mac_tx_client_restart((mac_client_handle_t)mcip);
6086 6117 ASSERT(ring->mr_gh == (mac_group_handle_t)defgrp);
6087 6118 return (0);
6088 6119 }
6089 6120
6090 6121 /*
6091 6122 * Populate a zero-ring group with rings. If the share is non-NULL,
6092 6123 * the rings are chosen according to that share.
6093 6124 * Invoked after allocating a new RX or TX group through
6094 6125 * mac_reserve_rx_group() or mac_reserve_tx_group(), respectively.
6095 6126 * Returns zero on success, an errno otherwise.
6096 6127 */
6097 6128 int
6098 6129 i_mac_group_allocate_rings(mac_impl_t *mip, mac_ring_type_t ring_type,
6099 6130 mac_group_t *src_group, mac_group_t *new_group, mac_share_handle_t share,
6100 6131 uint32_t ringcnt)
6101 6132 {
6102 6133 mac_ring_t **rings, *ring;
6103 6134 uint_t nrings;
6104 6135 int rv = 0, i = 0, j;
6105 6136
6106 6137 ASSERT((ring_type == MAC_RING_TYPE_RX &&
6107 6138 mip->mi_rx_group_type == MAC_GROUP_TYPE_DYNAMIC) ||
6108 6139 (ring_type == MAC_RING_TYPE_TX &&
6109 6140 mip->mi_tx_group_type == MAC_GROUP_TYPE_DYNAMIC));
6110 6141
6111 6142 /*
6112 6143 * First find the rings to allocate to the group.
6113 6144 */
6114 6145 if (share != NULL) {
6115 6146 /* get rings through ms_squery() */
6116 6147 mip->mi_share_capab.ms_squery(share, ring_type, NULL, &nrings);
6117 6148 ASSERT(nrings != 0);
6118 6149 rings = kmem_alloc(nrings * sizeof (mac_ring_handle_t),
6119 6150 KM_SLEEP);
6120 6151 mip->mi_share_capab.ms_squery(share, ring_type,
6121 6152 (mac_ring_handle_t *)rings, &nrings);
6122 6153 for (i = 0; i < nrings; i++) {
6123 6154 /*
6124 6155 * If we have given this ring to a non-default
6125 6156 * group, we need to check if we can get this
6126 6157 * ring.
6127 6158 */
6128 6159 ring = rings[i];
6129 6160 if (ring->mr_gh != (mac_group_handle_t)src_group ||
6130 6161 ring == (mac_ring_t *)mip->mi_default_tx_ring) {
6131 6162 if (mac_reclaim_ring_from_grp(mip, ring_type,
6132 6163 ring, rings, nrings) != 0) {
6133 6164 rv = ENOSPC;
6134 6165 goto bail;
6135 6166 }
6136 6167 }
6137 6168 }
6138 6169 } else {
6139 6170 /*
6140 6171 * Pick one ring from default group.
6141 6172 *
6142 6173 * for now pick the second ring which requires the first ring
6143 6174 * at index 0 to stay in the default group, since it is the
6144 6175 * ring which carries the multicast traffic.
6145 6176 * We need a better way for a driver to indicate this,
6146 6177 * for example a per-ring flag.
6147 6178 */
6148 6179 rings = kmem_alloc(ringcnt * sizeof (mac_ring_handle_t),
6149 6180 KM_SLEEP);
6150 6181 for (ring = src_group->mrg_rings; ring != NULL;
6151 6182 ring = ring->mr_next) {
6152 6183 if (ring_type == MAC_RING_TYPE_RX &&
6153 6184 ring->mr_index == 0) {
6154 6185 continue;
6155 6186 }
6156 6187 if (ring_type == MAC_RING_TYPE_TX &&
6157 6188 ring == (mac_ring_t *)mip->mi_default_tx_ring) {
6158 6189 continue;
6159 6190 }
6160 6191 rings[i++] = ring;
6161 6192 if (i == ringcnt)
6162 6193 break;
6163 6194 }
6164 6195 ASSERT(ring != NULL);
6165 6196 nrings = i;
6166 6197 /* Not enough rings as required */
6167 6198 if (nrings != ringcnt) {
6168 6199 rv = ENOSPC;
6169 6200 goto bail;
6170 6201 }
6171 6202 }
6172 6203
6173 6204 switch (ring_type) {
6174 6205 case MAC_RING_TYPE_RX:
6175 6206 if (src_group->mrg_cur_count - nrings < 1) {
6176 6207 /* we ran out of rings */
6177 6208 rv = ENOSPC;
6178 6209 goto bail;
6179 6210 }
6180 6211
6181 6212 /* move receive rings to new group */
6182 6213 for (i = 0; i < nrings; i++) {
6183 6214 rv = mac_group_mov_ring(mip, new_group, rings[i]);
6184 6215 if (rv != 0) {
6185 6216 /* move rings back on failure */
6186 6217 for (j = 0; j < i; j++) {
6187 6218 (void) mac_group_mov_ring(mip,
6188 6219 src_group, rings[j]);
6189 6220 }
6190 6221 goto bail;
6191 6222 }
6192 6223 }
6193 6224 break;
6194 6225
6195 6226 case MAC_RING_TYPE_TX: {
6196 6227 mac_ring_t *tmp_ring;
6197 6228
6198 6229 /* move the TX rings to the new group */
6199 6230 for (i = 0; i < nrings; i++) {
6200 6231 /* get the desired ring */
6201 6232 tmp_ring = mac_reserve_tx_ring(mip, rings[i]);
6202 6233 if (tmp_ring == NULL) {
6203 6234 rv = ENOSPC;
6204 6235 goto bail;
6205 6236 }
6206 6237 ASSERT(tmp_ring == rings[i]);
6207 6238 rv = mac_group_mov_ring(mip, new_group, rings[i]);
6208 6239 if (rv != 0) {
6209 6240 /* cleanup on failure */
6210 6241 for (j = 0; j < i; j++) {
6211 6242 (void) mac_group_mov_ring(mip,
6212 6243 MAC_DEFAULT_TX_GROUP(mip),
6213 6244 rings[j]);
6214 6245 }
6215 6246 goto bail;
6216 6247 }
6217 6248 }
6218 6249 break;
6219 6250 }
6220 6251 }
6221 6252
6222 6253 /* add group to share */
6223 6254 if (share != NULL)
6224 6255 mip->mi_share_capab.ms_sadd(share, new_group->mrg_driver);
6225 6256
6226 6257 bail:
6227 6258 /* free temporary array of rings */
6228 6259 kmem_free(rings, nrings * sizeof (mac_ring_handle_t));
6229 6260
6230 6261 return (rv);
6231 6262 }
6232 6263
6233 6264 void
6234 6265 mac_group_add_client(mac_group_t *grp, mac_client_impl_t *mcip)
6235 6266 {
6236 6267 mac_grp_client_t *mgcp;
6237 6268
6238 6269 for (mgcp = grp->mrg_clients; mgcp != NULL; mgcp = mgcp->mgc_next) {
6239 6270 if (mgcp->mgc_client == mcip)
6240 6271 break;
6241 6272 }
6242 6273
6243 6274 VERIFY(mgcp == NULL);
6244 6275
6245 6276 mgcp = kmem_zalloc(sizeof (mac_grp_client_t), KM_SLEEP);
6246 6277 mgcp->mgc_client = mcip;
6247 6278 mgcp->mgc_next = grp->mrg_clients;
6248 6279 grp->mrg_clients = mgcp;
6249 6280
6250 6281 }
6251 6282
6252 6283 void
6253 6284 mac_group_remove_client(mac_group_t *grp, mac_client_impl_t *mcip)
6254 6285 {
6255 6286 mac_grp_client_t *mgcp, **pprev;
6256 6287
6257 6288 for (pprev = &grp->mrg_clients, mgcp = *pprev; mgcp != NULL;
6258 6289 pprev = &mgcp->mgc_next, mgcp = *pprev) {
6259 6290 if (mgcp->mgc_client == mcip)
6260 6291 break;
6261 6292 }
6262 6293
6263 6294 ASSERT(mgcp != NULL);
6264 6295
6265 6296 *pprev = mgcp->mgc_next;
6266 6297 kmem_free(mgcp, sizeof (mac_grp_client_t));
6267 6298 }
6268 6299
6269 6300 /*
6270 6301 * mac_reserve_rx_group()
6271 6302 *
6272 6303 * Finds an available group and exclusively reserves it for a client.
6273 6304 * The group is chosen to suit the flow's resource controls (bandwidth and
6274 6305 * fanout requirements) and the address type.
6275 6306 * If the requestor is the pimary MAC then return the group with the
6276 6307 * largest number of rings, otherwise the default ring when available.
6277 6308 */
6278 6309 mac_group_t *
6279 6310 mac_reserve_rx_group(mac_client_impl_t *mcip, uint8_t *mac_addr, boolean_t move)
6280 6311 {
6281 6312 mac_share_handle_t share = mcip->mci_share;
6282 6313 mac_impl_t *mip = mcip->mci_mip;
6283 6314 mac_group_t *grp = NULL;
6284 6315 int i;
6285 6316 int err = 0;
6286 6317 mac_address_t *map;
6287 6318 mac_resource_props_t *mrp = MCIP_RESOURCE_PROPS(mcip);
6288 6319 int nrings;
6289 6320 int donor_grp_rcnt;
6290 6321 boolean_t need_exclgrp = B_FALSE;
6291 6322 int need_rings = 0;
6292 6323 mac_group_t *candidate_grp = NULL;
6293 6324 mac_client_impl_t *gclient;
6294 6325 mac_resource_props_t *gmrp;
6295 6326 mac_group_t *donorgrp = NULL;
6296 6327 boolean_t rxhw = mrp->mrp_mask & MRP_RX_RINGS;
6297 6328 boolean_t unspec = mrp->mrp_mask & MRP_RXRINGS_UNSPEC;
6298 6329 boolean_t isprimary;
6299 6330
6300 6331 ASSERT(MAC_PERIM_HELD((mac_handle_t)mip));
6301 6332
6302 6333 isprimary = mcip->mci_flent->fe_type & FLOW_PRIMARY_MAC;
6303 6334
6304 6335 /*
6305 6336 * Check if a group already has this mac address (case of VLANs)
6306 6337 * unless we are moving this MAC client from one group to another.
6307 6338 */
6308 6339 if (!move && (map = mac_find_macaddr(mip, mac_addr)) != NULL) {
6309 6340 if (map->ma_group != NULL)
6310 6341 return (map->ma_group);
6311 6342 }
6312 6343 if (mip->mi_rx_groups == NULL || mip->mi_rx_group_count == 0)
6313 6344 return (NULL);
6314 6345 /*
6315 6346 * If exclusive open, return NULL which will enable the
6316 6347 * caller to use the default group.
6317 6348 */
6318 6349 if (mcip->mci_state_flags & MCIS_EXCLUSIVE)
6319 6350 return (NULL);
6320 6351
6321 6352 /* For dynamic groups default unspecified to 1 */
6322 6353 if (rxhw && unspec &&
6323 6354 mip->mi_rx_group_type == MAC_GROUP_TYPE_DYNAMIC) {
6324 6355 mrp->mrp_nrxrings = 1;
6325 6356 }
6326 6357 /*
6327 6358 * For static grouping we allow only specifying rings=0 and
6328 6359 * unspecified
6329 6360 */
6330 6361 if (rxhw && mrp->mrp_nrxrings > 0 &&
6331 6362 mip->mi_rx_group_type == MAC_GROUP_TYPE_STATIC) {
6332 6363 return (NULL);
6333 6364 }
6334 6365 if (rxhw) {
6335 6366 /*
6336 6367 * We have explicitly asked for a group (with nrxrings,
6337 6368 * if unspec).
6338 6369 */
6339 6370 if (unspec || mrp->mrp_nrxrings > 0) {
6340 6371 need_exclgrp = B_TRUE;
6341 6372 need_rings = mrp->mrp_nrxrings;
6342 6373 } else if (mrp->mrp_nrxrings == 0) {
6343 6374 /*
6344 6375 * We have asked for a software group.
6345 6376 */
6346 6377 return (NULL);
6347 6378 }
6348 6379 } else if (isprimary && mip->mi_nactiveclients == 1 &&
6349 6380 mip->mi_rx_group_type == MAC_GROUP_TYPE_DYNAMIC) {
6350 6381 /*
6351 6382 * If the primary is the only active client on this
6352 6383 * mip and we have not asked for any rings, we give
6353 6384 * it the default group so that the primary gets to
6354 6385 * use all the rings.
6355 6386 */
6356 6387 return (NULL);
6357 6388 }
6358 6389
6359 6390 /* The group that can donate rings */
6360 6391 donorgrp = mip->mi_rx_donor_grp;
6361 6392
6362 6393 /*
6363 6394 * The number of rings that the default group can donate.
6364 6395 * We need to leave at least one ring.
6365 6396 */
6366 6397 donor_grp_rcnt = donorgrp->mrg_cur_count - 1;
6367 6398
6368 6399 /*
6369 6400 * Try to exclusively reserve a RX group.
6370 6401 *
6371 6402 * For flows requiring HW_DEFAULT_RING (unicast flow of the primary
6372 6403 * client), try to reserve the a non-default RX group and give
6373 6404 * it all the rings from the donor group, except the default ring
6374 6405 *
6375 6406 * For flows requiring HW_RING (unicast flow of other clients), try
6376 6407 * to reserve non-default RX group with the specified number of
6377 6408 * rings, if available.
6378 6409 *
6379 6410 * For flows that have not asked for software or hardware ring,
6380 6411 * try to reserve a non-default group with 1 ring, if available.
6381 6412 */
6382 6413 for (i = 1; i < mip->mi_rx_group_count; i++) {
6383 6414 grp = &mip->mi_rx_groups[i];
6384 6415
6385 6416 DTRACE_PROBE3(rx__group__trying, char *, mip->mi_name,
6386 6417 int, grp->mrg_index, mac_group_state_t, grp->mrg_state);
6387 6418
6388 6419 /*
6389 6420 * Check if this group could be a candidate group for
6390 6421 * eviction if we need a group for this MAC client,
6391 6422 * but there aren't any. A candidate group is one
6392 6423 * that didn't ask for an exclusive group, but got
6393 6424 * one and it has enough rings (combined with what
6394 6425 * the donor group can donate) for the new MAC
6395 6426 * client
6396 6427 */
6397 6428 if (grp->mrg_state >= MAC_GROUP_STATE_RESERVED) {
6398 6429 /*
6399 6430 * If the primary/donor group is not the default
6400 6431 * group, don't bother looking for a candidate group.
6401 6432 * If we don't have enough rings we will check
6402 6433 * if the primary group can be vacated.
6403 6434 */
6404 6435 if (candidate_grp == NULL &&
6405 6436 donorgrp == MAC_DEFAULT_RX_GROUP(mip)) {
6406 6437 ASSERT(!MAC_GROUP_NO_CLIENT(grp));
6407 6438 gclient = MAC_GROUP_ONLY_CLIENT(grp);
6408 6439 if (gclient == NULL)
6409 6440 gclient = mac_get_grp_primary(grp);
6410 6441 ASSERT(gclient != NULL);
6411 6442 gmrp = MCIP_RESOURCE_PROPS(gclient);
6412 6443 if (gclient->mci_share == NULL &&
6413 6444 (gmrp->mrp_mask & MRP_RX_RINGS) == 0 &&
6414 6445 (unspec ||
6415 6446 (grp->mrg_cur_count + donor_grp_rcnt >=
6416 6447 need_rings))) {
6417 6448 candidate_grp = grp;
6418 6449 }
6419 6450 }
6420 6451 continue;
6421 6452 }
6422 6453 /*
6423 6454 * This group could already be SHARED by other multicast
6424 6455 * flows on this client. In that case, the group would
6425 6456 * be shared and has already been started.
6426 6457 */
6427 6458 ASSERT(grp->mrg_state != MAC_GROUP_STATE_UNINIT);
6428 6459
6429 6460 if ((grp->mrg_state == MAC_GROUP_STATE_REGISTERED) &&
6430 6461 (mac_start_group(grp) != 0)) {
6431 6462 continue;
6432 6463 }
6433 6464
6434 6465 if (mip->mi_rx_group_type != MAC_GROUP_TYPE_DYNAMIC)
6435 6466 break;
6436 6467 ASSERT(grp->mrg_cur_count == 0);
6437 6468
6438 6469 /*
6439 6470 * Populate the group. Rings should be taken
6440 6471 * from the donor group.
6441 6472 */
6442 6473 nrings = rxhw ? need_rings : isprimary ? donor_grp_rcnt: 1;
6443 6474
6444 6475 /*
6445 6476 * If the donor group can't donate, let's just walk and
6446 6477 * see if someone can vacate a group, so that we have
6447 6478 * enough rings for this, unless we already have
6448 6479 * identified a candiate group..
6449 6480 */
6450 6481 if (nrings <= donor_grp_rcnt) {
6451 6482 err = i_mac_group_allocate_rings(mip, MAC_RING_TYPE_RX,
6452 6483 donorgrp, grp, share, nrings);
6453 6484 if (err == 0) {
6454 6485 /*
6455 6486 * For a share i_mac_group_allocate_rings gets
6456 6487 * the rings from the driver, let's populate
6457 6488 * the property for the client now.
6458 6489 */
6459 6490 if (share != NULL) {
6460 6491 mac_client_set_rings(
6461 6492 (mac_client_handle_t)mcip,
6462 6493 grp->mrg_cur_count, -1);
6463 6494 }
6464 6495 if (mac_is_primary_client(mcip) && !rxhw)
6465 6496 mip->mi_rx_donor_grp = grp;
6466 6497 break;
6467 6498 }
6468 6499 }
6469 6500
6470 6501 DTRACE_PROBE3(rx__group__reserve__alloc__rings, char *,
6471 6502 mip->mi_name, int, grp->mrg_index, int, err);
6472 6503
6473 6504 /*
6474 6505 * It's a dynamic group but the grouping operation
6475 6506 * failed.
6476 6507 */
6477 6508 mac_stop_group(grp);
6478 6509 }
6479 6510 /* We didn't find an exclusive group for this MAC client */
6480 6511 if (i >= mip->mi_rx_group_count) {
6481 6512
6482 6513 if (!need_exclgrp)
6483 6514 return (NULL);
6484 6515
6485 6516 /*
6486 6517 * If we found a candidate group then we switch the
6487 6518 * MAC client from the candidate_group to the default
6488 6519 * group and give the group to this MAC client. If
6489 6520 * we didn't find a candidate_group, check if the
6490 6521 * primary is in its own group and if it can make way
6491 6522 * for this MAC client.
6492 6523 */
6493 6524 if (candidate_grp == NULL &&
6494 6525 donorgrp != MAC_DEFAULT_RX_GROUP(mip) &&
6495 6526 donorgrp->mrg_cur_count >= need_rings) {
6496 6527 candidate_grp = donorgrp;
6497 6528 }
6498 6529 if (candidate_grp != NULL) {
6499 6530 boolean_t prim_grp = B_FALSE;
6500 6531
6501 6532 /*
6502 6533 * Switch the MAC client from the candidate group
6503 6534 * to the default group.. If this group was the
6504 6535 * donor group, then after the switch we need
6505 6536 * to update the donor group too.
6506 6537 */
6507 6538 grp = candidate_grp;
6508 6539 gclient = MAC_GROUP_ONLY_CLIENT(grp);
6509 6540 if (gclient == NULL)
6510 6541 gclient = mac_get_grp_primary(grp);
6511 6542 if (grp == mip->mi_rx_donor_grp)
6512 6543 prim_grp = B_TRUE;
6513 6544 if (mac_rx_switch_group(gclient, grp,
6514 6545 MAC_DEFAULT_RX_GROUP(mip)) != 0) {
6515 6546 return (NULL);
6516 6547 }
6517 6548 if (prim_grp) {
6518 6549 mip->mi_rx_donor_grp =
6519 6550 MAC_DEFAULT_RX_GROUP(mip);
6520 6551 donorgrp = MAC_DEFAULT_RX_GROUP(mip);
6521 6552 }
6522 6553
6523 6554
6524 6555 /*
6525 6556 * Now give this group with the required rings
6526 6557 * to this MAC client.
6527 6558 */
6528 6559 ASSERT(grp->mrg_state == MAC_GROUP_STATE_REGISTERED);
6529 6560 if (mac_start_group(grp) != 0)
6530 6561 return (NULL);
6531 6562
6532 6563 if (mip->mi_rx_group_type != MAC_GROUP_TYPE_DYNAMIC)
6533 6564 return (grp);
6534 6565
6535 6566 donor_grp_rcnt = donorgrp->mrg_cur_count - 1;
6536 6567 ASSERT(grp->mrg_cur_count == 0);
6537 6568 ASSERT(donor_grp_rcnt >= need_rings);
6538 6569 err = i_mac_group_allocate_rings(mip, MAC_RING_TYPE_RX,
6539 6570 donorgrp, grp, share, need_rings);
6540 6571 if (err == 0) {
6541 6572 /*
6542 6573 * For a share i_mac_group_allocate_rings gets
6543 6574 * the rings from the driver, let's populate
6544 6575 * the property for the client now.
6545 6576 */
6546 6577 if (share != NULL) {
6547 6578 mac_client_set_rings(
6548 6579 (mac_client_handle_t)mcip,
6549 6580 grp->mrg_cur_count, -1);
6550 6581 }
6551 6582 DTRACE_PROBE2(rx__group__reserved,
6552 6583 char *, mip->mi_name, int, grp->mrg_index);
6553 6584 return (grp);
6554 6585 }
6555 6586 DTRACE_PROBE3(rx__group__reserve__alloc__rings, char *,
6556 6587 mip->mi_name, int, grp->mrg_index, int, err);
6557 6588 mac_stop_group(grp);
6558 6589 }
6559 6590 return (NULL);
6560 6591 }
6561 6592 ASSERT(grp != NULL);
6562 6593
6563 6594 DTRACE_PROBE2(rx__group__reserved,
6564 6595 char *, mip->mi_name, int, grp->mrg_index);
6565 6596 return (grp);
6566 6597 }
6567 6598
6568 6599 /*
6569 6600 * mac_rx_release_group()
6570 6601 *
6571 6602 * This is called when there are no clients left for the group.
6572 6603 * The group is stopped and marked MAC_GROUP_STATE_REGISTERED,
6573 6604 * and if it is a non default group, the shares are removed and
6574 6605 * all rings are assigned back to default group.
6575 6606 */
6576 6607 void
6577 6608 mac_release_rx_group(mac_client_impl_t *mcip, mac_group_t *group)
6578 6609 {
6579 6610 mac_impl_t *mip = mcip->mci_mip;
6580 6611 mac_ring_t *ring;
6581 6612
6582 6613 ASSERT(group != MAC_DEFAULT_RX_GROUP(mip));
6583 6614
6584 6615 if (mip->mi_rx_donor_grp == group)
6585 6616 mip->mi_rx_donor_grp = MAC_DEFAULT_RX_GROUP(mip);
6586 6617
6587 6618 /*
6588 6619 * This is the case where there are no clients left. Any
6589 6620 * SRS etc on this group have also be quiesced.
6590 6621 */
6591 6622 for (ring = group->mrg_rings; ring != NULL; ring = ring->mr_next) {
6592 6623 if (ring->mr_classify_type == MAC_HW_CLASSIFIER) {
6593 6624 ASSERT(group->mrg_state == MAC_GROUP_STATE_RESERVED);
6594 6625 /*
6595 6626 * Remove the SRS associated with the HW ring.
6596 6627 * As a result, polling will be disabled.
6597 6628 */
6598 6629 ring->mr_srs = NULL;
6599 6630 }
6600 6631 ASSERT(group->mrg_state < MAC_GROUP_STATE_RESERVED ||
6601 6632 ring->mr_state == MR_INUSE);
6602 6633 if (ring->mr_state == MR_INUSE) {
6603 6634 mac_stop_ring(ring);
6604 6635 ring->mr_flag = 0;
6605 6636 }
6606 6637 }
6607 6638
6608 6639 /* remove group from share */
6609 6640 if (mcip->mci_share != NULL) {
6610 6641 mip->mi_share_capab.ms_sremove(mcip->mci_share,
6611 6642 group->mrg_driver);
6612 6643 }
6613 6644
6614 6645 if (mip->mi_rx_group_type == MAC_GROUP_TYPE_DYNAMIC) {
6615 6646 mac_ring_t *ring;
6616 6647
6617 6648 /*
6618 6649 * Rings were dynamically allocated to group.
6619 6650 * Move rings back to default group.
6620 6651 */
6621 6652 while ((ring = group->mrg_rings) != NULL) {
6622 6653 (void) mac_group_mov_ring(mip, mip->mi_rx_donor_grp,
6623 6654 ring);
6624 6655 }
6625 6656 }
6626 6657 mac_stop_group(group);
6627 6658 /*
6628 6659 * Possible improvement: See if we can assign the group just released
6629 6660 * to a another client of the mip
6630 6661 */
6631 6662 }
6632 6663
6633 6664 /*
6634 6665 * When we move the primary's mac address between groups, we need to also
6635 6666 * take all the clients sharing the same mac address along with it (VLANs)
6636 6667 * We remove the mac address for such clients from the group after quiescing
6637 6668 * them. When we add the mac address we restart the client. Note that
6638 6669 * the primary's mac address is removed from the group after all the
6639 6670 * other clients sharing the address are removed. Similarly, the primary's
6640 6671 * mac address is added before all the other client's mac address are
6641 6672 * added. While grp is the group where the clients reside, tgrp is
6642 6673 * the group where the addresses have to be added.
6643 6674 */
6644 6675 static void
6645 6676 mac_rx_move_macaddr_prim(mac_client_impl_t *mcip, mac_group_t *grp,
6646 6677 mac_group_t *tgrp, uint8_t *maddr, boolean_t add)
6647 6678 {
6648 6679 mac_impl_t *mip = mcip->mci_mip;
6649 6680 mac_grp_client_t *mgcp = grp->mrg_clients;
6650 6681 mac_client_impl_t *gmcip;
6651 6682 boolean_t prim;
6652 6683
6653 6684 prim = (mcip->mci_state_flags & MCIS_UNICAST_HW) != 0;
6654 6685
6655 6686 /*
6656 6687 * If the clients are in a non-default group, we just have to
6657 6688 * walk the group's client list. If it is in the default group
6658 6689 * (which will be shared by other clients as well, we need to
6659 6690 * check if the unicast address matches mcip's unicast.
6660 6691 */
6661 6692 while (mgcp != NULL) {
6662 6693 gmcip = mgcp->mgc_client;
6663 6694 if (gmcip != mcip &&
6664 6695 (grp != MAC_DEFAULT_RX_GROUP(mip) ||
6665 6696 mcip->mci_unicast == gmcip->mci_unicast)) {
6666 6697 if (!add) {
6667 6698 mac_rx_client_quiesce(
6668 6699 (mac_client_handle_t)gmcip);
6669 6700 (void) mac_remove_macaddr(mcip->mci_unicast);
6670 6701 } else {
6671 6702 (void) mac_add_macaddr(mip, tgrp, maddr, prim);
6672 6703 mac_rx_client_restart(
6673 6704 (mac_client_handle_t)gmcip);
6674 6705 }
6675 6706 }
6676 6707 mgcp = mgcp->mgc_next;
6677 6708 }
6678 6709 }
6679 6710
6680 6711
6681 6712 /*
6682 6713 * Move the MAC address from fgrp to tgrp. If this is the primary client,
6683 6714 * we need to take any VLANs etc. together too.
6684 6715 */
6685 6716 static int
6686 6717 mac_rx_move_macaddr(mac_client_impl_t *mcip, mac_group_t *fgrp,
6687 6718 mac_group_t *tgrp)
6688 6719 {
6689 6720 mac_impl_t *mip = mcip->mci_mip;
6690 6721 uint8_t maddr[MAXMACADDRLEN];
6691 6722 int err = 0;
6692 6723 boolean_t prim;
6693 6724 boolean_t multiclnt = B_FALSE;
6694 6725
6695 6726 mac_rx_client_quiesce((mac_client_handle_t)mcip);
6696 6727 ASSERT(mcip->mci_unicast != NULL);
6697 6728 bcopy(mcip->mci_unicast->ma_addr, maddr, mcip->mci_unicast->ma_len);
6698 6729
6699 6730 prim = (mcip->mci_state_flags & MCIS_UNICAST_HW) != 0;
6700 6731 if (mcip->mci_unicast->ma_nusers > 1) {
6701 6732 mac_rx_move_macaddr_prim(mcip, fgrp, NULL, maddr, B_FALSE);
6702 6733 multiclnt = B_TRUE;
6703 6734 }
6704 6735 ASSERT(mcip->mci_unicast->ma_nusers == 1);
6705 6736 err = mac_remove_macaddr(mcip->mci_unicast);
6706 6737 if (err != 0) {
6707 6738 mac_rx_client_restart((mac_client_handle_t)mcip);
6708 6739 if (multiclnt) {
6709 6740 mac_rx_move_macaddr_prim(mcip, fgrp, fgrp, maddr,
6710 6741 B_TRUE);
6711 6742 }
6712 6743 return (err);
6713 6744 }
6714 6745 /*
6715 6746 * Program the H/W Classifier first, if this fails we need
6716 6747 * not proceed with the other stuff.
6717 6748 */
6718 6749 if ((err = mac_add_macaddr(mip, tgrp, maddr, prim)) != 0) {
6719 6750 /* Revert back the H/W Classifier */
6720 6751 if ((err = mac_add_macaddr(mip, fgrp, maddr, prim)) != 0) {
6721 6752 /*
6722 6753 * This should not fail now since it worked earlier,
6723 6754 * should we panic?
6724 6755 */
6725 6756 cmn_err(CE_WARN,
6726 6757 "mac_rx_switch_group: switching %p back"
6727 6758 " to group %p failed!!", (void *)mcip,
6728 6759 (void *)fgrp);
6729 6760 }
6730 6761 mac_rx_client_restart((mac_client_handle_t)mcip);
6731 6762 if (multiclnt) {
6732 6763 mac_rx_move_macaddr_prim(mcip, fgrp, fgrp, maddr,
6733 6764 B_TRUE);
6734 6765 }
6735 6766 return (err);
6736 6767 }
6737 6768 mcip->mci_unicast = mac_find_macaddr(mip, maddr);
6738 6769 mac_rx_client_restart((mac_client_handle_t)mcip);
6739 6770 if (multiclnt)
6740 6771 mac_rx_move_macaddr_prim(mcip, fgrp, tgrp, maddr, B_TRUE);
6741 6772 return (err);
6742 6773 }
6743 6774
6744 6775 /*
6745 6776 * Switch the MAC client from one group to another. This means we need
6746 6777 * to remove the MAC address from the group, remove the MAC client,
6747 6778 * teardown the SRSs and revert the group state. Then, we add the client
6748 6779 * to the destination group, set the SRSs, and add the MAC address to the
6749 6780 * group.
6750 6781 */
6751 6782 int
6752 6783 mac_rx_switch_group(mac_client_impl_t *mcip, mac_group_t *fgrp,
6753 6784 mac_group_t *tgrp)
6754 6785 {
6755 6786 int err;
6756 6787 mac_group_state_t next_state;
6757 6788 mac_client_impl_t *group_only_mcip;
6758 6789 mac_client_impl_t *gmcip;
6759 6790 mac_impl_t *mip = mcip->mci_mip;
6760 6791 mac_grp_client_t *mgcp;
6761 6792
6762 6793 ASSERT(fgrp == mcip->mci_flent->fe_rx_ring_group);
6763 6794
6764 6795 if ((err = mac_rx_move_macaddr(mcip, fgrp, tgrp)) != 0)
6765 6796 return (err);
6766 6797
6767 6798 /*
6768 6799 * The group might be reserved, but SRSs may not be set up, e.g.
6769 6800 * primary and its vlans using a reserved group.
6770 6801 */
6771 6802 if (fgrp->mrg_state == MAC_GROUP_STATE_RESERVED &&
6772 6803 MAC_GROUP_ONLY_CLIENT(fgrp) != NULL) {
6773 6804 mac_rx_srs_group_teardown(mcip->mci_flent, B_TRUE);
6774 6805 }
6775 6806 if (fgrp != MAC_DEFAULT_RX_GROUP(mip)) {
6776 6807 mgcp = fgrp->mrg_clients;
6777 6808 while (mgcp != NULL) {
6778 6809 gmcip = mgcp->mgc_client;
6779 6810 mgcp = mgcp->mgc_next;
6780 6811 mac_group_remove_client(fgrp, gmcip);
6781 6812 mac_group_add_client(tgrp, gmcip);
6782 6813 gmcip->mci_flent->fe_rx_ring_group = tgrp;
6783 6814 }
6784 6815 mac_release_rx_group(mcip, fgrp);
6785 6816 ASSERT(MAC_GROUP_NO_CLIENT(fgrp));
6786 6817 mac_set_group_state(fgrp, MAC_GROUP_STATE_REGISTERED);
6787 6818 } else {
6788 6819 mac_group_remove_client(fgrp, mcip);
6789 6820 mac_group_add_client(tgrp, mcip);
6790 6821 mcip->mci_flent->fe_rx_ring_group = tgrp;
6791 6822 /*
6792 6823 * If there are other clients (VLANs) sharing this address
6793 6824 * we should be here only for the primary.
6794 6825 */
6795 6826 if (mcip->mci_unicast->ma_nusers > 1) {
6796 6827 /*
6797 6828 * We need to move all the clients that are using
6798 6829 * this h/w address.
6799 6830 */
6800 6831 mgcp = fgrp->mrg_clients;
6801 6832 while (mgcp != NULL) {
6802 6833 gmcip = mgcp->mgc_client;
6803 6834 mgcp = mgcp->mgc_next;
6804 6835 if (mcip->mci_unicast == gmcip->mci_unicast) {
6805 6836 mac_group_remove_client(fgrp, gmcip);
6806 6837 mac_group_add_client(tgrp, gmcip);
6807 6838 gmcip->mci_flent->fe_rx_ring_group =
6808 6839 tgrp;
6809 6840 }
6810 6841 }
6811 6842 }
6812 6843 /*
6813 6844 * The default group will still take the multicast,
6814 6845 * broadcast traffic etc., so it won't go to
6815 6846 * MAC_GROUP_STATE_REGISTERED.
6816 6847 */
6817 6848 if (fgrp->mrg_state == MAC_GROUP_STATE_RESERVED)
6818 6849 mac_rx_group_unmark(fgrp, MR_CONDEMNED);
6819 6850 mac_set_group_state(fgrp, MAC_GROUP_STATE_SHARED);
6820 6851 }
6821 6852 next_state = mac_group_next_state(tgrp, &group_only_mcip,
6822 6853 MAC_DEFAULT_RX_GROUP(mip), B_TRUE);
6823 6854 mac_set_group_state(tgrp, next_state);
6824 6855 /*
6825 6856 * If the destination group is reserved, setup the SRSs etc.
6826 6857 */
6827 6858 if (tgrp->mrg_state == MAC_GROUP_STATE_RESERVED) {
6828 6859 mac_rx_srs_group_setup(mcip, mcip->mci_flent, SRST_LINK);
6829 6860 mac_fanout_setup(mcip, mcip->mci_flent,
6830 6861 MCIP_RESOURCE_PROPS(mcip), mac_rx_deliver, mcip, NULL,
6831 6862 NULL);
6832 6863 mac_rx_group_unmark(tgrp, MR_INCIPIENT);
6833 6864 } else {
6834 6865 mac_rx_switch_grp_to_sw(tgrp);
6835 6866 }
6836 6867 return (0);
6837 6868 }
6838 6869
6839 6870 /*
6840 6871 * Reserves a TX group for the specified share. Invoked by mac_tx_srs_setup()
6841 6872 * when a share was allocated to the client.
6842 6873 */
6843 6874 mac_group_t *
6844 6875 mac_reserve_tx_group(mac_client_impl_t *mcip, boolean_t move)
6845 6876 {
6846 6877 mac_impl_t *mip = mcip->mci_mip;
6847 6878 mac_group_t *grp = NULL;
6848 6879 int rv;
6849 6880 int i;
6850 6881 int err;
6851 6882 mac_group_t *defgrp;
6852 6883 mac_share_handle_t share = mcip->mci_share;
6853 6884 mac_resource_props_t *mrp = MCIP_RESOURCE_PROPS(mcip);
6854 6885 int nrings;
6855 6886 int defnrings;
6856 6887 boolean_t need_exclgrp = B_FALSE;
6857 6888 int need_rings = 0;
6858 6889 mac_group_t *candidate_grp = NULL;
6859 6890 mac_client_impl_t *gclient;
6860 6891 mac_resource_props_t *gmrp;
6861 6892 boolean_t txhw = mrp->mrp_mask & MRP_TX_RINGS;
6862 6893 boolean_t unspec = mrp->mrp_mask & MRP_TXRINGS_UNSPEC;
6863 6894 boolean_t isprimary;
6864 6895
6865 6896 isprimary = mcip->mci_flent->fe_type & FLOW_PRIMARY_MAC;
6866 6897 /*
6867 6898 * When we come here for a VLAN on the primary (dladm create-vlan),
6868 6899 * we need to pair it along with the primary (to keep it consistent
6869 6900 * with the RX side). So, we check if the primary is already assigned
6870 6901 * to a group and return the group if so. The other way is also
6871 6902 * true, i.e. the VLAN is already created and now we are plumbing
6872 6903 * the primary.
6873 6904 */
6874 6905 if (!move && isprimary) {
6875 6906 for (gclient = mip->mi_clients_list; gclient != NULL;
6876 6907 gclient = gclient->mci_client_next) {
6877 6908 if (gclient->mci_flent->fe_type & FLOW_PRIMARY_MAC &&
6878 6909 gclient->mci_flent->fe_tx_ring_group != NULL) {
6879 6910 return (gclient->mci_flent->fe_tx_ring_group);
6880 6911 }
6881 6912 }
6882 6913 }
6883 6914
6884 6915 if (mip->mi_tx_groups == NULL || mip->mi_tx_group_count == 0)
6885 6916 return (NULL);
6886 6917
6887 6918 /* For dynamic groups, default unspec to 1 */
6888 6919 if (txhw && unspec &&
6889 6920 mip->mi_tx_group_type == MAC_GROUP_TYPE_DYNAMIC) {
6890 6921 mrp->mrp_ntxrings = 1;
6891 6922 }
6892 6923 /*
6893 6924 * For static grouping we allow only specifying rings=0 and
6894 6925 * unspecified
6895 6926 */
6896 6927 if (txhw && mrp->mrp_ntxrings > 0 &&
6897 6928 mip->mi_tx_group_type == MAC_GROUP_TYPE_STATIC) {
6898 6929 return (NULL);
6899 6930 }
6900 6931
6901 6932 if (txhw) {
6902 6933 /*
6903 6934 * We have explicitly asked for a group (with ntxrings,
6904 6935 * if unspec).
6905 6936 */
6906 6937 if (unspec || mrp->mrp_ntxrings > 0) {
6907 6938 need_exclgrp = B_TRUE;
6908 6939 need_rings = mrp->mrp_ntxrings;
6909 6940 } else if (mrp->mrp_ntxrings == 0) {
6910 6941 /*
6911 6942 * We have asked for a software group.
6912 6943 */
6913 6944 return (NULL);
6914 6945 }
6915 6946 }
6916 6947 defgrp = MAC_DEFAULT_TX_GROUP(mip);
6917 6948 /*
6918 6949 * The number of rings that the default group can donate.
6919 6950 * We need to leave at least one ring - the default ring - in
6920 6951 * this group.
6921 6952 */
6922 6953 defnrings = defgrp->mrg_cur_count - 1;
6923 6954
6924 6955 /*
6925 6956 * Primary gets default group unless explicitly told not
6926 6957 * to (i.e. rings > 0).
6927 6958 */
6928 6959 if (isprimary && !need_exclgrp)
6929 6960 return (NULL);
6930 6961
6931 6962 nrings = (mrp->mrp_mask & MRP_TX_RINGS) != 0 ? mrp->mrp_ntxrings : 1;
6932 6963 for (i = 0; i < mip->mi_tx_group_count; i++) {
6933 6964 grp = &mip->mi_tx_groups[i];
6934 6965 if ((grp->mrg_state == MAC_GROUP_STATE_RESERVED) ||
6935 6966 (grp->mrg_state == MAC_GROUP_STATE_UNINIT)) {
6936 6967 /*
6937 6968 * Select a candidate for replacement if we don't
6938 6969 * get an exclusive group. A candidate group is one
6939 6970 * that didn't ask for an exclusive group, but got
6940 6971 * one and it has enough rings (combined with what
6941 6972 * the default group can donate) for the new MAC
6942 6973 * client.
6943 6974 */
6944 6975 if (grp->mrg_state == MAC_GROUP_STATE_RESERVED &&
6945 6976 candidate_grp == NULL) {
6946 6977 gclient = MAC_GROUP_ONLY_CLIENT(grp);
6947 6978 if (gclient == NULL)
6948 6979 gclient = mac_get_grp_primary(grp);
6949 6980 gmrp = MCIP_RESOURCE_PROPS(gclient);
6950 6981 if (gclient->mci_share == NULL &&
6951 6982 (gmrp->mrp_mask & MRP_TX_RINGS) == 0 &&
6952 6983 (unspec ||
6953 6984 (grp->mrg_cur_count + defnrings) >=
6954 6985 need_rings)) {
6955 6986 candidate_grp = grp;
6956 6987 }
6957 6988 }
6958 6989 continue;
6959 6990 }
6960 6991 /*
6961 6992 * If the default can't donate let's just walk and
6962 6993 * see if someone can vacate a group, so that we have
6963 6994 * enough rings for this.
6964 6995 */
6965 6996 if (mip->mi_tx_group_type != MAC_GROUP_TYPE_DYNAMIC ||
6966 6997 nrings <= defnrings) {
6967 6998 if (grp->mrg_state == MAC_GROUP_STATE_REGISTERED) {
6968 6999 rv = mac_start_group(grp);
6969 7000 ASSERT(rv == 0);
6970 7001 }
6971 7002 break;
6972 7003 }
6973 7004 }
6974 7005
6975 7006 /* The default group */
6976 7007 if (i >= mip->mi_tx_group_count) {
6977 7008 /*
6978 7009 * If we need an exclusive group and have identified a
6979 7010 * candidate group we switch the MAC client from the
6980 7011 * candidate group to the default group and give the
6981 7012 * candidate group to this client.
6982 7013 */
6983 7014 if (need_exclgrp && candidate_grp != NULL) {
6984 7015 /*
6985 7016 * Switch the MAC client from the candidate group
6986 7017 * to the default group.
6987 7018 */
6988 7019 grp = candidate_grp;
6989 7020 gclient = MAC_GROUP_ONLY_CLIENT(grp);
6990 7021 if (gclient == NULL)
6991 7022 gclient = mac_get_grp_primary(grp);
6992 7023 mac_tx_client_quiesce((mac_client_handle_t)gclient);
6993 7024 mac_tx_switch_group(gclient, grp, defgrp);
6994 7025 mac_tx_client_restart((mac_client_handle_t)gclient);
6995 7026
6996 7027 /*
6997 7028 * Give the candidate group with the specified number
6998 7029 * of rings to this MAC client.
6999 7030 */
7000 7031 ASSERT(grp->mrg_state == MAC_GROUP_STATE_REGISTERED);
7001 7032 rv = mac_start_group(grp);
7002 7033 ASSERT(rv == 0);
7003 7034
7004 7035 if (mip->mi_tx_group_type != MAC_GROUP_TYPE_DYNAMIC)
7005 7036 return (grp);
7006 7037
7007 7038 ASSERT(grp->mrg_cur_count == 0);
7008 7039 ASSERT(defgrp->mrg_cur_count > need_rings);
7009 7040
7010 7041 err = i_mac_group_allocate_rings(mip, MAC_RING_TYPE_TX,
7011 7042 defgrp, grp, share, need_rings);
7012 7043 if (err == 0) {
7013 7044 /*
7014 7045 * For a share i_mac_group_allocate_rings gets
7015 7046 * the rings from the driver, let's populate
7016 7047 * the property for the client now.
7017 7048 */
7018 7049 if (share != NULL) {
7019 7050 mac_client_set_rings(
7020 7051 (mac_client_handle_t)mcip, -1,
7021 7052 grp->mrg_cur_count);
7022 7053 }
7023 7054 mip->mi_tx_group_free--;
7024 7055 return (grp);
7025 7056 }
7026 7057 DTRACE_PROBE3(tx__group__reserve__alloc__rings, char *,
7027 7058 mip->mi_name, int, grp->mrg_index, int, err);
7028 7059 mac_stop_group(grp);
7029 7060 }
7030 7061 return (NULL);
7031 7062 }
7032 7063 /*
7033 7064 * We got an exclusive group, but it is not dynamic.
7034 7065 */
7035 7066 if (mip->mi_tx_group_type != MAC_GROUP_TYPE_DYNAMIC) {
7036 7067 mip->mi_tx_group_free--;
7037 7068 return (grp);
7038 7069 }
7039 7070
7040 7071 rv = i_mac_group_allocate_rings(mip, MAC_RING_TYPE_TX, defgrp, grp,
7041 7072 share, nrings);
7042 7073 if (rv != 0) {
7043 7074 DTRACE_PROBE3(tx__group__reserve__alloc__rings,
7044 7075 char *, mip->mi_name, int, grp->mrg_index, int, rv);
7045 7076 mac_stop_group(grp);
7046 7077 return (NULL);
7047 7078 }
7048 7079 /*
7049 7080 * For a share i_mac_group_allocate_rings gets the rings from the
7050 7081 * driver, let's populate the property for the client now.
7051 7082 */
7052 7083 if (share != NULL) {
7053 7084 mac_client_set_rings((mac_client_handle_t)mcip, -1,
7054 7085 grp->mrg_cur_count);
7055 7086 }
7056 7087 mip->mi_tx_group_free--;
7057 7088 return (grp);
7058 7089 }
7059 7090
7060 7091 void
7061 7092 mac_release_tx_group(mac_client_impl_t *mcip, mac_group_t *grp)
7062 7093 {
7063 7094 mac_impl_t *mip = mcip->mci_mip;
7064 7095 mac_share_handle_t share = mcip->mci_share;
7065 7096 mac_ring_t *ring;
7066 7097 mac_soft_ring_set_t *srs = MCIP_TX_SRS(mcip);
7067 7098 mac_group_t *defgrp;
7068 7099
7069 7100 defgrp = MAC_DEFAULT_TX_GROUP(mip);
7070 7101 if (srs != NULL) {
7071 7102 if (srs->srs_soft_ring_count > 0) {
7072 7103 for (ring = grp->mrg_rings; ring != NULL;
7073 7104 ring = ring->mr_next) {
7074 7105 ASSERT(mac_tx_srs_ring_present(srs, ring));
7075 7106 mac_tx_invoke_callbacks(mcip,
7076 7107 (mac_tx_cookie_t)
7077 7108 mac_tx_srs_get_soft_ring(srs, ring));
7078 7109 mac_tx_srs_del_ring(srs, ring);
7079 7110 }
7080 7111 } else {
7081 7112 ASSERT(srs->srs_tx.st_arg2 != NULL);
7082 7113 srs->srs_tx.st_arg2 = NULL;
7083 7114 mac_srs_stat_delete(srs);
7084 7115 }
7085 7116 }
7086 7117 if (share != NULL)
7087 7118 mip->mi_share_capab.ms_sremove(share, grp->mrg_driver);
7088 7119
7089 7120 /* move the ring back to the pool */
7090 7121 if (mip->mi_tx_group_type == MAC_GROUP_TYPE_DYNAMIC) {
7091 7122 while ((ring = grp->mrg_rings) != NULL)
7092 7123 (void) mac_group_mov_ring(mip, defgrp, ring);
7093 7124 }
7094 7125 mac_stop_group(grp);
7095 7126 mip->mi_tx_group_free++;
7096 7127 }
7097 7128
7098 7129 /*
7099 7130 * Disassociate a MAC client from a group, i.e go through the rings in the
7100 7131 * group and delete all the soft rings tied to them.
7101 7132 */
7102 7133 static void
7103 7134 mac_tx_dismantle_soft_rings(mac_group_t *fgrp, flow_entry_t *flent)
7104 7135 {
7105 7136 mac_client_impl_t *mcip = flent->fe_mcip;
7106 7137 mac_soft_ring_set_t *tx_srs;
7107 7138 mac_srs_tx_t *tx;
7108 7139 mac_ring_t *ring;
7109 7140
7110 7141 tx_srs = flent->fe_tx_srs;
7111 7142 tx = &tx_srs->srs_tx;
7112 7143
7113 7144 /* Single ring case we haven't created any soft rings */
7114 7145 if (tx->st_mode == SRS_TX_BW || tx->st_mode == SRS_TX_SERIALIZE ||
7115 7146 tx->st_mode == SRS_TX_DEFAULT) {
7116 7147 tx->st_arg2 = NULL;
7117 7148 mac_srs_stat_delete(tx_srs);
7118 7149 /* Fanout case, where we have to dismantle the soft rings */
7119 7150 } else {
7120 7151 for (ring = fgrp->mrg_rings; ring != NULL;
7121 7152 ring = ring->mr_next) {
7122 7153 ASSERT(mac_tx_srs_ring_present(tx_srs, ring));
7123 7154 mac_tx_invoke_callbacks(mcip,
7124 7155 (mac_tx_cookie_t)mac_tx_srs_get_soft_ring(tx_srs,
7125 7156 ring));
7126 7157 mac_tx_srs_del_ring(tx_srs, ring);
7127 7158 }
7128 7159 ASSERT(tx->st_arg2 == NULL);
7129 7160 }
7130 7161 }
7131 7162
7132 7163 /*
7133 7164 * Switch the MAC client from one group to another. This means we need
7134 7165 * to remove the MAC client, teardown the SRSs and revert the group state.
7135 7166 * Then, we add the client to the destination roup, set the SRSs etc.
7136 7167 */
7137 7168 void
7138 7169 mac_tx_switch_group(mac_client_impl_t *mcip, mac_group_t *fgrp,
7139 7170 mac_group_t *tgrp)
7140 7171 {
7141 7172 mac_client_impl_t *group_only_mcip;
7142 7173 mac_impl_t *mip = mcip->mci_mip;
7143 7174 flow_entry_t *flent = mcip->mci_flent;
7144 7175 mac_group_t *defgrp;
7145 7176 mac_grp_client_t *mgcp;
7146 7177 mac_client_impl_t *gmcip;
7147 7178 flow_entry_t *gflent;
7148 7179
7149 7180 defgrp = MAC_DEFAULT_TX_GROUP(mip);
7150 7181 ASSERT(fgrp == flent->fe_tx_ring_group);
7151 7182
7152 7183 if (fgrp == defgrp) {
7153 7184 /*
7154 7185 * If this is the primary we need to find any VLANs on
7155 7186 * the primary and move them too.
7156 7187 */
7157 7188 mac_group_remove_client(fgrp, mcip);
7158 7189 mac_tx_dismantle_soft_rings(fgrp, flent);
7159 7190 if (mcip->mci_unicast->ma_nusers > 1) {
7160 7191 mgcp = fgrp->mrg_clients;
7161 7192 while (mgcp != NULL) {
7162 7193 gmcip = mgcp->mgc_client;
7163 7194 mgcp = mgcp->mgc_next;
7164 7195 if (mcip->mci_unicast != gmcip->mci_unicast)
7165 7196 continue;
7166 7197 mac_tx_client_quiesce(
7167 7198 (mac_client_handle_t)gmcip);
7168 7199
7169 7200 gflent = gmcip->mci_flent;
7170 7201 mac_group_remove_client(fgrp, gmcip);
7171 7202 mac_tx_dismantle_soft_rings(fgrp, gflent);
7172 7203
7173 7204 mac_group_add_client(tgrp, gmcip);
7174 7205 gflent->fe_tx_ring_group = tgrp;
7175 7206 /* We could directly set this to SHARED */
7176 7207 tgrp->mrg_state = mac_group_next_state(tgrp,
7177 7208 &group_only_mcip, defgrp, B_FALSE);
7178 7209
7179 7210 mac_tx_srs_group_setup(gmcip, gflent,
7180 7211 SRST_LINK);
7181 7212 mac_fanout_setup(gmcip, gflent,
7182 7213 MCIP_RESOURCE_PROPS(gmcip), mac_rx_deliver,
7183 7214 gmcip, NULL, NULL);
7184 7215
7185 7216 mac_tx_client_restart(
7186 7217 (mac_client_handle_t)gmcip);
7187 7218 }
7188 7219 }
7189 7220 if (MAC_GROUP_NO_CLIENT(fgrp)) {
7190 7221 mac_ring_t *ring;
7191 7222 int cnt;
7192 7223 int ringcnt;
7193 7224
7194 7225 fgrp->mrg_state = MAC_GROUP_STATE_REGISTERED;
7195 7226 /*
7196 7227 * Additionally, we also need to stop all
7197 7228 * the rings in the default group, except
7198 7229 * the default ring. The reason being
7199 7230 * this group won't be released since it is
7200 7231 * the default group, so the rings won't
7201 7232 * be stopped otherwise.
7202 7233 */
7203 7234 ringcnt = fgrp->mrg_cur_count;
7204 7235 ring = fgrp->mrg_rings;
7205 7236 for (cnt = 0; cnt < ringcnt; cnt++) {
7206 7237 if (ring->mr_state == MR_INUSE &&
7207 7238 ring !=
7208 7239 (mac_ring_t *)mip->mi_default_tx_ring) {
7209 7240 mac_stop_ring(ring);
7210 7241 ring->mr_flag = 0;
7211 7242 }
7212 7243 ring = ring->mr_next;
7213 7244 }
7214 7245 } else if (MAC_GROUP_ONLY_CLIENT(fgrp) != NULL) {
7215 7246 fgrp->mrg_state = MAC_GROUP_STATE_RESERVED;
7216 7247 } else {
7217 7248 ASSERT(fgrp->mrg_state == MAC_GROUP_STATE_SHARED);
7218 7249 }
7219 7250 } else {
7220 7251 /*
7221 7252 * We could have VLANs sharing the non-default group with
7222 7253 * the primary.
7223 7254 */
7224 7255 mgcp = fgrp->mrg_clients;
7225 7256 while (mgcp != NULL) {
7226 7257 gmcip = mgcp->mgc_client;
7227 7258 mgcp = mgcp->mgc_next;
7228 7259 if (gmcip == mcip)
7229 7260 continue;
7230 7261 mac_tx_client_quiesce((mac_client_handle_t)gmcip);
7231 7262 gflent = gmcip->mci_flent;
7232 7263
7233 7264 mac_group_remove_client(fgrp, gmcip);
7234 7265 mac_tx_dismantle_soft_rings(fgrp, gflent);
7235 7266
7236 7267 mac_group_add_client(tgrp, gmcip);
7237 7268 gflent->fe_tx_ring_group = tgrp;
7238 7269 /* We could directly set this to SHARED */
7239 7270 tgrp->mrg_state = mac_group_next_state(tgrp,
7240 7271 &group_only_mcip, defgrp, B_FALSE);
7241 7272 mac_tx_srs_group_setup(gmcip, gflent, SRST_LINK);
7242 7273 mac_fanout_setup(gmcip, gflent,
7243 7274 MCIP_RESOURCE_PROPS(gmcip), mac_rx_deliver,
7244 7275 gmcip, NULL, NULL);
7245 7276
7246 7277 mac_tx_client_restart((mac_client_handle_t)gmcip);
7247 7278 }
7248 7279 mac_group_remove_client(fgrp, mcip);
7249 7280 mac_release_tx_group(mcip, fgrp);
7250 7281 fgrp->mrg_state = MAC_GROUP_STATE_REGISTERED;
7251 7282 }
7252 7283
7253 7284 /* Add it to the tgroup */
7254 7285 mac_group_add_client(tgrp, mcip);
7255 7286 flent->fe_tx_ring_group = tgrp;
7256 7287 tgrp->mrg_state = mac_group_next_state(tgrp, &group_only_mcip,
7257 7288 defgrp, B_FALSE);
7258 7289
7259 7290 mac_tx_srs_group_setup(mcip, flent, SRST_LINK);
7260 7291 mac_fanout_setup(mcip, flent, MCIP_RESOURCE_PROPS(mcip),
7261 7292 mac_rx_deliver, mcip, NULL, NULL);
7262 7293 }
7263 7294
7264 7295 /*
7265 7296 * This is a 1-time control path activity initiated by the client (IP).
7266 7297 * The mac perimeter protects against other simultaneous control activities,
7267 7298 * for example an ioctl that attempts to change the degree of fanout and
7268 7299 * increase or decrease the number of softrings associated with this Tx SRS.
7269 7300 */
7270 7301 static mac_tx_notify_cb_t *
7271 7302 mac_client_tx_notify_add(mac_client_impl_t *mcip,
7272 7303 mac_tx_notify_t notify, void *arg)
7273 7304 {
7274 7305 mac_cb_info_t *mcbi;
7275 7306 mac_tx_notify_cb_t *mtnfp;
7276 7307
7277 7308 ASSERT(MAC_PERIM_HELD((mac_handle_t)mcip->mci_mip));
7278 7309
7279 7310 mtnfp = kmem_zalloc(sizeof (mac_tx_notify_cb_t), KM_SLEEP);
7280 7311 mtnfp->mtnf_fn = notify;
7281 7312 mtnfp->mtnf_arg = arg;
7282 7313 mtnfp->mtnf_link.mcb_objp = mtnfp;
7283 7314 mtnfp->mtnf_link.mcb_objsize = sizeof (mac_tx_notify_cb_t);
7284 7315 mtnfp->mtnf_link.mcb_flags = MCB_TX_NOTIFY_CB_T;
7285 7316
7286 7317 mcbi = &mcip->mci_tx_notify_cb_info;
7287 7318 mutex_enter(mcbi->mcbi_lockp);
7288 7319 mac_callback_add(mcbi, &mcip->mci_tx_notify_cb_list, &mtnfp->mtnf_link);
7289 7320 mutex_exit(mcbi->mcbi_lockp);
7290 7321 return (mtnfp);
7291 7322 }
7292 7323
7293 7324 static void
7294 7325 mac_client_tx_notify_remove(mac_client_impl_t *mcip, mac_tx_notify_cb_t *mtnfp)
7295 7326 {
7296 7327 mac_cb_info_t *mcbi;
7297 7328 mac_cb_t **cblist;
7298 7329
7299 7330 ASSERT(MAC_PERIM_HELD((mac_handle_t)mcip->mci_mip));
7300 7331
7301 7332 if (!mac_callback_find(&mcip->mci_tx_notify_cb_info,
7302 7333 &mcip->mci_tx_notify_cb_list, &mtnfp->mtnf_link)) {
7303 7334 cmn_err(CE_WARN,
7304 7335 "mac_client_tx_notify_remove: callback not "
7305 7336 "found, mcip 0x%p mtnfp 0x%p", (void *)mcip, (void *)mtnfp);
7306 7337 return;
7307 7338 }
7308 7339
7309 7340 mcbi = &mcip->mci_tx_notify_cb_info;
7310 7341 cblist = &mcip->mci_tx_notify_cb_list;
7311 7342 mutex_enter(mcbi->mcbi_lockp);
7312 7343 if (mac_callback_remove(mcbi, cblist, &mtnfp->mtnf_link))
7313 7344 kmem_free(mtnfp, sizeof (mac_tx_notify_cb_t));
7314 7345 else
7315 7346 mac_callback_remove_wait(&mcip->mci_tx_notify_cb_info);
7316 7347 mutex_exit(mcbi->mcbi_lockp);
7317 7348 }
7318 7349
7319 7350 /*
7320 7351 * mac_client_tx_notify():
7321 7352 * call to add and remove flow control callback routine.
7322 7353 */
7323 7354 mac_tx_notify_handle_t
7324 7355 mac_client_tx_notify(mac_client_handle_t mch, mac_tx_notify_t callb_func,
7325 7356 void *ptr)
7326 7357 {
7327 7358 mac_client_impl_t *mcip = (mac_client_impl_t *)mch;
7328 7359 mac_tx_notify_cb_t *mtnfp = NULL;
7329 7360
7330 7361 i_mac_perim_enter(mcip->mci_mip);
7331 7362
7332 7363 if (callb_func != NULL) {
7333 7364 /* Add a notify callback */
7334 7365 mtnfp = mac_client_tx_notify_add(mcip, callb_func, ptr);
7335 7366 } else {
7336 7367 mac_client_tx_notify_remove(mcip, (mac_tx_notify_cb_t *)ptr);
7337 7368 }
7338 7369 i_mac_perim_exit(mcip->mci_mip);
7339 7370
7340 7371 return ((mac_tx_notify_handle_t)mtnfp);
7341 7372 }
7342 7373
7343 7374 void
7344 7375 mac_bridge_vectors(mac_bridge_tx_t txf, mac_bridge_rx_t rxf,
7345 7376 mac_bridge_ref_t reff, mac_bridge_ls_t lsf)
7346 7377 {
7347 7378 mac_bridge_tx_cb = txf;
7348 7379 mac_bridge_rx_cb = rxf;
7349 7380 mac_bridge_ref_cb = reff;
7350 7381 mac_bridge_ls_cb = lsf;
7351 7382 }
7352 7383
7353 7384 int
7354 7385 mac_bridge_set(mac_handle_t mh, mac_handle_t link)
7355 7386 {
7356 7387 mac_impl_t *mip = (mac_impl_t *)mh;
7357 7388 int retv;
7358 7389
7359 7390 mutex_enter(&mip->mi_bridge_lock);
7360 7391 if (mip->mi_bridge_link == NULL) {
7361 7392 mip->mi_bridge_link = link;
7362 7393 retv = 0;
7363 7394 } else {
7364 7395 retv = EBUSY;
7365 7396 }
7366 7397 mutex_exit(&mip->mi_bridge_lock);
7367 7398 if (retv == 0) {
7368 7399 mac_poll_state_change(mh, B_FALSE);
7369 7400 mac_capab_update(mh);
7370 7401 }
7371 7402 return (retv);
7372 7403 }
7373 7404
7374 7405 /*
7375 7406 * Disable bridging on the indicated link.
7376 7407 */
7377 7408 void
7378 7409 mac_bridge_clear(mac_handle_t mh, mac_handle_t link)
7379 7410 {
7380 7411 mac_impl_t *mip = (mac_impl_t *)mh;
7381 7412
7382 7413 mutex_enter(&mip->mi_bridge_lock);
7383 7414 ASSERT(mip->mi_bridge_link == link);
7384 7415 mip->mi_bridge_link = NULL;
7385 7416 mutex_exit(&mip->mi_bridge_lock);
7386 7417 mac_poll_state_change(mh, B_TRUE);
7387 7418 mac_capab_update(mh);
7388 7419 }
7389 7420
7390 7421 void
7391 7422 mac_no_active(mac_handle_t mh)
7392 7423 {
7393 7424 mac_impl_t *mip = (mac_impl_t *)mh;
7394 7425
7395 7426 i_mac_perim_enter(mip);
7396 7427 mip->mi_state_flags |= MIS_NO_ACTIVE;
7397 7428 i_mac_perim_exit(mip);
7398 7429 }
7399 7430
7400 7431 /*
7401 7432 * Walk the primary VLAN clients whenever the primary's rings property
7402 7433 * changes and update the mac_resource_props_t for the VLAN's client.
7403 7434 * We need to do this since we don't support setting these properties
7404 7435 * on the primary's VLAN clients, but the VLAN clients have to
7405 7436 * follow the primary w.r.t the rings property;
7406 7437 */
7407 7438 void
7408 7439 mac_set_prim_vlan_rings(mac_impl_t *mip, mac_resource_props_t *mrp)
7409 7440 {
7410 7441 mac_client_impl_t *vmcip;
7411 7442 mac_resource_props_t *vmrp;
7412 7443
7413 7444 for (vmcip = mip->mi_clients_list; vmcip != NULL;
7414 7445 vmcip = vmcip->mci_client_next) {
7415 7446 if (!(vmcip->mci_flent->fe_type & FLOW_PRIMARY_MAC) ||
7416 7447 mac_client_vid((mac_client_handle_t)vmcip) ==
7417 7448 VLAN_ID_NONE) {
7418 7449 continue;
7419 7450 }
7420 7451 vmrp = MCIP_RESOURCE_PROPS(vmcip);
7421 7452
7422 7453 vmrp->mrp_nrxrings = mrp->mrp_nrxrings;
7423 7454 if (mrp->mrp_mask & MRP_RX_RINGS)
7424 7455 vmrp->mrp_mask |= MRP_RX_RINGS;
7425 7456 else if (vmrp->mrp_mask & MRP_RX_RINGS)
7426 7457 vmrp->mrp_mask &= ~MRP_RX_RINGS;
7427 7458
7428 7459 vmrp->mrp_ntxrings = mrp->mrp_ntxrings;
7429 7460 if (mrp->mrp_mask & MRP_TX_RINGS)
7430 7461 vmrp->mrp_mask |= MRP_TX_RINGS;
7431 7462 else if (vmrp->mrp_mask & MRP_TX_RINGS)
7432 7463 vmrp->mrp_mask &= ~MRP_TX_RINGS;
7433 7464
7434 7465 if (mrp->mrp_mask & MRP_RXRINGS_UNSPEC)
7435 7466 vmrp->mrp_mask |= MRP_RXRINGS_UNSPEC;
7436 7467 else
7437 7468 vmrp->mrp_mask &= ~MRP_RXRINGS_UNSPEC;
7438 7469
7439 7470 if (mrp->mrp_mask & MRP_TXRINGS_UNSPEC)
7440 7471 vmrp->mrp_mask |= MRP_TXRINGS_UNSPEC;
7441 7472 else
7442 7473 vmrp->mrp_mask &= ~MRP_TXRINGS_UNSPEC;
7443 7474 }
7444 7475 }
7445 7476
7446 7477 /*
7447 7478 * We are adding or removing ring(s) from a group. The source for taking
7448 7479 * rings is the default group. The destination for giving rings back is
7449 7480 * the default group.
7450 7481 */
7451 7482 int
7452 7483 mac_group_ring_modify(mac_client_impl_t *mcip, mac_group_t *group,
7453 7484 mac_group_t *defgrp)
7454 7485 {
7455 7486 mac_resource_props_t *mrp = MCIP_RESOURCE_PROPS(mcip);
7456 7487 uint_t modify;
7457 7488 int count;
7458 7489 mac_ring_t *ring;
7459 7490 mac_ring_t *next;
7460 7491 mac_impl_t *mip = mcip->mci_mip;
7461 7492 mac_ring_t **rings;
7462 7493 uint_t ringcnt;
7463 7494 int i = 0;
7464 7495 boolean_t rx_group = group->mrg_type == MAC_RING_TYPE_RX;
7465 7496 int start;
7466 7497 int end;
7467 7498 mac_group_t *tgrp;
7468 7499 int j;
7469 7500 int rv = 0;
7470 7501
7471 7502 /*
7472 7503 * If we are asked for just a group, we give 1 ring, else
7473 7504 * the specified number of rings.
7474 7505 */
7475 7506 if (rx_group) {
7476 7507 ringcnt = (mrp->mrp_mask & MRP_RXRINGS_UNSPEC) ? 1:
7477 7508 mrp->mrp_nrxrings;
7478 7509 } else {
7479 7510 ringcnt = (mrp->mrp_mask & MRP_TXRINGS_UNSPEC) ? 1:
7480 7511 mrp->mrp_ntxrings;
7481 7512 }
7482 7513
7483 7514 /* don't allow modifying rings for a share for now. */
7484 7515 ASSERT(mcip->mci_share == NULL);
7485 7516
7486 7517 if (ringcnt == group->mrg_cur_count)
7487 7518 return (0);
7488 7519
7489 7520 if (group->mrg_cur_count > ringcnt) {
7490 7521 modify = group->mrg_cur_count - ringcnt;
7491 7522 if (rx_group) {
7492 7523 if (mip->mi_rx_donor_grp == group) {
7493 7524 ASSERT(mac_is_primary_client(mcip));
7494 7525 mip->mi_rx_donor_grp = defgrp;
7495 7526 } else {
7496 7527 defgrp = mip->mi_rx_donor_grp;
7497 7528 }
7498 7529 }
7499 7530 ring = group->mrg_rings;
7500 7531 rings = kmem_alloc(modify * sizeof (mac_ring_handle_t),
7501 7532 KM_SLEEP);
7502 7533 j = 0;
7503 7534 for (count = 0; count < modify; count++) {
7504 7535 next = ring->mr_next;
7505 7536 rv = mac_group_mov_ring(mip, defgrp, ring);
7506 7537 if (rv != 0) {
7507 7538 /* cleanup on failure */
7508 7539 for (j = 0; j < count; j++) {
7509 7540 (void) mac_group_mov_ring(mip, group,
7510 7541 rings[j]);
7511 7542 }
7512 7543 break;
7513 7544 }
7514 7545 rings[j++] = ring;
7515 7546 ring = next;
7516 7547 }
7517 7548 kmem_free(rings, modify * sizeof (mac_ring_handle_t));
7518 7549 return (rv);
7519 7550 }
7520 7551 if (ringcnt >= MAX_RINGS_PER_GROUP)
7521 7552 return (EINVAL);
7522 7553
7523 7554 modify = ringcnt - group->mrg_cur_count;
7524 7555
7525 7556 if (rx_group) {
7526 7557 if (group != mip->mi_rx_donor_grp)
7527 7558 defgrp = mip->mi_rx_donor_grp;
7528 7559 else
7529 7560 /*
7530 7561 * This is the donor group with all the remaining
7531 7562 * rings. Default group now gets to be the donor
7532 7563 */
7533 7564 mip->mi_rx_donor_grp = defgrp;
7534 7565 start = 1;
7535 7566 end = mip->mi_rx_group_count;
7536 7567 } else {
7537 7568 start = 0;
7538 7569 end = mip->mi_tx_group_count - 1;
7539 7570 }
7540 7571 /*
7541 7572 * If the default doesn't have any rings, lets see if we can
7542 7573 * take rings given to an h/w client that doesn't need it.
7543 7574 * For now, we just see if there is any one client that can donate
7544 7575 * all the required rings.
7545 7576 */
7546 7577 if (defgrp->mrg_cur_count < (modify + 1)) {
7547 7578 for (i = start; i < end; i++) {
7548 7579 if (rx_group) {
7549 7580 tgrp = &mip->mi_rx_groups[i];
7550 7581 if (tgrp == group || tgrp->mrg_state <
7551 7582 MAC_GROUP_STATE_RESERVED) {
7552 7583 continue;
7553 7584 }
7554 7585 mcip = MAC_GROUP_ONLY_CLIENT(tgrp);
7555 7586 if (mcip == NULL)
7556 7587 mcip = mac_get_grp_primary(tgrp);
7557 7588 ASSERT(mcip != NULL);
7558 7589 mrp = MCIP_RESOURCE_PROPS(mcip);
7559 7590 if ((mrp->mrp_mask & MRP_RX_RINGS) != 0)
7560 7591 continue;
7561 7592 if ((tgrp->mrg_cur_count +
7562 7593 defgrp->mrg_cur_count) < (modify + 1)) {
7563 7594 continue;
7564 7595 }
7565 7596 if (mac_rx_switch_group(mcip, tgrp,
7566 7597 defgrp) != 0) {
7567 7598 return (ENOSPC);
7568 7599 }
7569 7600 } else {
7570 7601 tgrp = &mip->mi_tx_groups[i];
7571 7602 if (tgrp == group || tgrp->mrg_state <
7572 7603 MAC_GROUP_STATE_RESERVED) {
7573 7604 continue;
7574 7605 }
7575 7606 mcip = MAC_GROUP_ONLY_CLIENT(tgrp);
7576 7607 if (mcip == NULL)
7577 7608 mcip = mac_get_grp_primary(tgrp);
7578 7609 mrp = MCIP_RESOURCE_PROPS(mcip);
7579 7610 if ((mrp->mrp_mask & MRP_TX_RINGS) != 0)
7580 7611 continue;
7581 7612 if ((tgrp->mrg_cur_count +
7582 7613 defgrp->mrg_cur_count) < (modify + 1)) {
7583 7614 continue;
7584 7615 }
7585 7616 /* OK, we can switch this to s/w */
7586 7617 mac_tx_client_quiesce(
7587 7618 (mac_client_handle_t)mcip);
7588 7619 mac_tx_switch_group(mcip, tgrp, defgrp);
7589 7620 mac_tx_client_restart(
7590 7621 (mac_client_handle_t)mcip);
7591 7622 }
7592 7623 }
7593 7624 if (defgrp->mrg_cur_count < (modify + 1))
7594 7625 return (ENOSPC);
7595 7626 }
7596 7627 if ((rv = i_mac_group_allocate_rings(mip, group->mrg_type, defgrp,
7597 7628 group, mcip->mci_share, modify)) != 0) {
7598 7629 return (rv);
7599 7630 }
7600 7631 return (0);
7601 7632 }
7602 7633
7603 7634 /*
7604 7635 * Given the poolname in mac_resource_props, find the cpupart
7605 7636 * that is associated with this pool. The cpupart will be used
7606 7637 * later for finding the cpus to be bound to the networking threads.
7607 7638 *
7608 7639 * use_default is set B_TRUE if pools are enabled and pool_default
7609 7640 * is returned. This avoids a 2nd lookup to set the poolname
7610 7641 * for pool-effective.
7611 7642 *
7612 7643 * returns:
7613 7644 *
7614 7645 * NULL - pools are disabled or if the 'cpus' property is set.
7615 7646 * cpupart of pool_default - pools are enabled and the pool
7616 7647 * is not available or poolname is blank
7617 7648 * cpupart of named pool - pools are enabled and the pool
7618 7649 * is available.
7619 7650 */
7620 7651 cpupart_t *
7621 7652 mac_pset_find(mac_resource_props_t *mrp, boolean_t *use_default)
7622 7653 {
7623 7654 pool_t *pool;
7624 7655 cpupart_t *cpupart;
7625 7656
7626 7657 *use_default = B_FALSE;
7627 7658
7628 7659 /* CPUs property is set */
7629 7660 if (mrp->mrp_mask & MRP_CPUS)
7630 7661 return (NULL);
7631 7662
7632 7663 ASSERT(pool_lock_held());
7633 7664
7634 7665 /* Pools are disabled, no pset */
7635 7666 if (pool_state == POOL_DISABLED)
7636 7667 return (NULL);
7637 7668
7638 7669 /* Pools property is set */
7639 7670 if (mrp->mrp_mask & MRP_POOL) {
7640 7671 if ((pool = pool_lookup_pool_by_name(mrp->mrp_pool)) == NULL) {
7641 7672 /* Pool not found */
7642 7673 DTRACE_PROBE1(mac_pset_find_no_pool, char *,
7643 7674 mrp->mrp_pool);
7644 7675 *use_default = B_TRUE;
7645 7676 pool = pool_default;
7646 7677 }
7647 7678 /* Pools property is not set */
7648 7679 } else {
7649 7680 *use_default = B_TRUE;
7650 7681 pool = pool_default;
7651 7682 }
7652 7683
7653 7684 /* Find the CPU pset that corresponds to the pool */
7654 7685 mutex_enter(&cpu_lock);
7655 7686 if ((cpupart = cpupart_find(pool->pool_pset->pset_id)) == NULL) {
7656 7687 DTRACE_PROBE1(mac_find_pset_no_pset, psetid_t,
7657 7688 pool->pool_pset->pset_id);
7658 7689 }
7659 7690 mutex_exit(&cpu_lock);
7660 7691
7661 7692 return (cpupart);
7662 7693 }
7663 7694
7664 7695 void
7665 7696 mac_set_pool_effective(boolean_t use_default, cpupart_t *cpupart,
7666 7697 mac_resource_props_t *mrp, mac_resource_props_t *emrp)
7667 7698 {
7668 7699 ASSERT(pool_lock_held());
7669 7700
7670 7701 if (cpupart != NULL) {
7671 7702 emrp->mrp_mask |= MRP_POOL;
7672 7703 if (use_default) {
7673 7704 (void) strcpy(emrp->mrp_pool,
7674 7705 "pool_default");
7675 7706 } else {
7676 7707 ASSERT(strlen(mrp->mrp_pool) != 0);
7677 7708 (void) strcpy(emrp->mrp_pool,
7678 7709 mrp->mrp_pool);
7679 7710 }
7680 7711 } else {
7681 7712 emrp->mrp_mask &= ~MRP_POOL;
7682 7713 bzero(emrp->mrp_pool, MAXPATHLEN);
7683 7714 }
7684 7715 }
7685 7716
7686 7717 struct mac_pool_arg {
7687 7718 char mpa_poolname[MAXPATHLEN];
7688 7719 pool_event_t mpa_what;
7689 7720 };
7690 7721
7691 7722 /*ARGSUSED*/
7692 7723 static uint_t
7693 7724 mac_pool_link_update(mod_hash_key_t key, mod_hash_val_t *val, void *arg)
7694 7725 {
7695 7726 struct mac_pool_arg *mpa = arg;
7696 7727 mac_impl_t *mip = (mac_impl_t *)val;
7697 7728 mac_client_impl_t *mcip;
7698 7729 mac_resource_props_t *mrp, *emrp;
7699 7730 boolean_t pool_update = B_FALSE;
7700 7731 boolean_t pool_clear = B_FALSE;
7701 7732 boolean_t use_default = B_FALSE;
7702 7733 cpupart_t *cpupart = NULL;
7703 7734
7704 7735 mrp = kmem_zalloc(sizeof (*mrp), KM_SLEEP);
7705 7736 i_mac_perim_enter(mip);
7706 7737 for (mcip = mip->mi_clients_list; mcip != NULL;
7707 7738 mcip = mcip->mci_client_next) {
7708 7739 pool_update = B_FALSE;
7709 7740 pool_clear = B_FALSE;
7710 7741 use_default = B_FALSE;
7711 7742 mac_client_get_resources((mac_client_handle_t)mcip, mrp);
7712 7743 emrp = MCIP_EFFECTIVE_PROPS(mcip);
7713 7744
7714 7745 /*
7715 7746 * When pools are enabled
7716 7747 */
7717 7748 if ((mpa->mpa_what == POOL_E_ENABLE) &&
7718 7749 ((mrp->mrp_mask & MRP_CPUS) == 0)) {
7719 7750 mrp->mrp_mask |= MRP_POOL;
7720 7751 pool_update = B_TRUE;
7721 7752 }
7722 7753
7723 7754 /*
7724 7755 * When pools are disabled
7725 7756 */
7726 7757 if ((mpa->mpa_what == POOL_E_DISABLE) &&
7727 7758 ((mrp->mrp_mask & MRP_CPUS) == 0)) {
7728 7759 mrp->mrp_mask |= MRP_POOL;
7729 7760 pool_clear = B_TRUE;
7730 7761 }
7731 7762
7732 7763 /*
7733 7764 * Look for links with the pool property set and the poolname
7734 7765 * matching the one which is changing.
7735 7766 */
7736 7767 if (strcmp(mrp->mrp_pool, mpa->mpa_poolname) == 0) {
7737 7768 /*
7738 7769 * The pool associated with the link has changed.
7739 7770 */
7740 7771 if (mpa->mpa_what == POOL_E_CHANGE) {
7741 7772 mrp->mrp_mask |= MRP_POOL;
7742 7773 pool_update = B_TRUE;
7743 7774 }
7744 7775 }
7745 7776
7746 7777 /*
7747 7778 * This link is associated with pool_default and
7748 7779 * pool_default has changed.
7749 7780 */
7750 7781 if ((mpa->mpa_what == POOL_E_CHANGE) &&
7751 7782 (strcmp(emrp->mrp_pool, "pool_default") == 0) &&
7752 7783 (strcmp(mpa->mpa_poolname, "pool_default") == 0)) {
7753 7784 mrp->mrp_mask |= MRP_POOL;
7754 7785 pool_update = B_TRUE;
7755 7786 }
7756 7787
7757 7788 /*
7758 7789 * Get new list of cpus for the pool, bind network
7759 7790 * threads to new list of cpus and update resources.
7760 7791 */
7761 7792 if (pool_update) {
7762 7793 if (MCIP_DATAPATH_SETUP(mcip)) {
7763 7794 pool_lock();
7764 7795 cpupart = mac_pset_find(mrp, &use_default);
7765 7796 mac_fanout_setup(mcip, mcip->mci_flent, mrp,
7766 7797 mac_rx_deliver, mcip, NULL, cpupart);
7767 7798 mac_set_pool_effective(use_default, cpupart,
7768 7799 mrp, emrp);
7769 7800 pool_unlock();
7770 7801 }
7771 7802 mac_update_resources(mrp, MCIP_RESOURCE_PROPS(mcip),
7772 7803 B_FALSE);
7773 7804 }
7774 7805
7775 7806 /*
7776 7807 * Clear the effective pool and bind network threads
7777 7808 * to any available CPU.
7778 7809 */
7779 7810 if (pool_clear) {
7780 7811 if (MCIP_DATAPATH_SETUP(mcip)) {
7781 7812 emrp->mrp_mask &= ~MRP_POOL;
7782 7813 bzero(emrp->mrp_pool, MAXPATHLEN);
7783 7814 mac_fanout_setup(mcip, mcip->mci_flent, mrp,
7784 7815 mac_rx_deliver, mcip, NULL, NULL);
7785 7816 }
7786 7817 mac_update_resources(mrp, MCIP_RESOURCE_PROPS(mcip),
7787 7818 B_FALSE);
7788 7819 }
7789 7820 }
7790 7821 i_mac_perim_exit(mip);
7791 7822 kmem_free(mrp, sizeof (*mrp));
7792 7823 return (MH_WALK_CONTINUE);
7793 7824 }
7794 7825
7795 7826 static void
7796 7827 mac_pool_update(void *arg)
7797 7828 {
7798 7829 mod_hash_walk(i_mac_impl_hash, mac_pool_link_update, arg);
7799 7830 kmem_free(arg, sizeof (struct mac_pool_arg));
7800 7831 }
7801 7832
7802 7833 /*
7803 7834 * Callback function to be executed when a noteworthy pool event
7804 7835 * takes place.
7805 7836 */
7806 7837 /* ARGSUSED */
7807 7838 static void
7808 7839 mac_pool_event_cb(pool_event_t what, poolid_t id, void *arg)
7809 7840 {
7810 7841 pool_t *pool;
7811 7842 char *poolname = NULL;
7812 7843 struct mac_pool_arg *mpa;
7813 7844
7814 7845 pool_lock();
7815 7846 mpa = kmem_zalloc(sizeof (struct mac_pool_arg), KM_SLEEP);
7816 7847
7817 7848 switch (what) {
7818 7849 case POOL_E_ENABLE:
7819 7850 case POOL_E_DISABLE:
7820 7851 break;
7821 7852
7822 7853 case POOL_E_CHANGE:
7823 7854 pool = pool_lookup_pool_by_id(id);
7824 7855 if (pool == NULL) {
7825 7856 kmem_free(mpa, sizeof (struct mac_pool_arg));
7826 7857 pool_unlock();
7827 7858 return;
7828 7859 }
7829 7860 pool_get_name(pool, &poolname);
7830 7861 (void) strlcpy(mpa->mpa_poolname, poolname,
7831 7862 sizeof (mpa->mpa_poolname));
7832 7863 break;
7833 7864
7834 7865 default:
7835 7866 kmem_free(mpa, sizeof (struct mac_pool_arg));
7836 7867 pool_unlock();
7837 7868 return;
7838 7869 }
7839 7870 pool_unlock();
7840 7871
7841 7872 mpa->mpa_what = what;
7842 7873
7843 7874 mac_pool_update(mpa);
7844 7875 }
7845 7876
7846 7877 /*
7847 7878 * Set effective rings property. This could be called from datapath_setup/
7848 7879 * datapath_teardown or set-linkprop.
7849 7880 * If the group is reserved we just go ahead and set the effective rings.
7850 7881 * Additionally, for TX this could mean the default group has lost/gained
7851 7882 * some rings, so if the default group is reserved, we need to adjust the
7852 7883 * effective rings for the default group clients. For RX, if we are working
7853 7884 * with the non-default group, we just need * to reset the effective props
7854 7885 * for the default group clients.
7855 7886 */
7856 7887 void
7857 7888 mac_set_rings_effective(mac_client_impl_t *mcip)
7858 7889 {
7859 7890 mac_impl_t *mip = mcip->mci_mip;
7860 7891 mac_group_t *grp;
7861 7892 mac_group_t *defgrp;
7862 7893 flow_entry_t *flent = mcip->mci_flent;
7863 7894 mac_resource_props_t *emrp = MCIP_EFFECTIVE_PROPS(mcip);
7864 7895 mac_grp_client_t *mgcp;
7865 7896 mac_client_impl_t *gmcip;
7866 7897
7867 7898 grp = flent->fe_rx_ring_group;
7868 7899 if (grp != NULL) {
7869 7900 defgrp = MAC_DEFAULT_RX_GROUP(mip);
7870 7901 /*
7871 7902 * If we have reserved a group, set the effective rings
7872 7903 * to the ring count in the group.
7873 7904 */
7874 7905 if (grp->mrg_state == MAC_GROUP_STATE_RESERVED) {
7875 7906 emrp->mrp_mask |= MRP_RX_RINGS;
7876 7907 emrp->mrp_nrxrings = grp->mrg_cur_count;
7877 7908 }
7878 7909
7879 7910 /*
7880 7911 * We go through the clients in the shared group and
7881 7912 * reset the effective properties. It is possible this
7882 7913 * might have already been done for some client (i.e.
7883 7914 * if some client is being moved to a group that is
7884 7915 * already shared). The case where the default group is
7885 7916 * RESERVED is taken care of above (note in the RX side if
7886 7917 * there is a non-default group, the default group is always
7887 7918 * SHARED).
7888 7919 */
7889 7920 if (grp != defgrp || grp->mrg_state == MAC_GROUP_STATE_SHARED) {
7890 7921 if (grp->mrg_state == MAC_GROUP_STATE_SHARED)
7891 7922 mgcp = grp->mrg_clients;
7892 7923 else
7893 7924 mgcp = defgrp->mrg_clients;
7894 7925 while (mgcp != NULL) {
7895 7926 gmcip = mgcp->mgc_client;
7896 7927 emrp = MCIP_EFFECTIVE_PROPS(gmcip);
7897 7928 if (emrp->mrp_mask & MRP_RX_RINGS) {
7898 7929 emrp->mrp_mask &= ~MRP_RX_RINGS;
7899 7930 emrp->mrp_nrxrings = 0;
7900 7931 }
7901 7932 mgcp = mgcp->mgc_next;
7902 7933 }
7903 7934 }
7904 7935 }
7905 7936
7906 7937 /* Now the TX side */
7907 7938 grp = flent->fe_tx_ring_group;
7908 7939 if (grp != NULL) {
7909 7940 defgrp = MAC_DEFAULT_TX_GROUP(mip);
7910 7941
7911 7942 if (grp->mrg_state == MAC_GROUP_STATE_RESERVED) {
7912 7943 emrp->mrp_mask |= MRP_TX_RINGS;
7913 7944 emrp->mrp_ntxrings = grp->mrg_cur_count;
7914 7945 } else if (grp->mrg_state == MAC_GROUP_STATE_SHARED) {
7915 7946 mgcp = grp->mrg_clients;
7916 7947 while (mgcp != NULL) {
7917 7948 gmcip = mgcp->mgc_client;
7918 7949 emrp = MCIP_EFFECTIVE_PROPS(gmcip);
7919 7950 if (emrp->mrp_mask & MRP_TX_RINGS) {
7920 7951 emrp->mrp_mask &= ~MRP_TX_RINGS;
7921 7952 emrp->mrp_ntxrings = 0;
7922 7953 }
7923 7954 mgcp = mgcp->mgc_next;
7924 7955 }
7925 7956 }
7926 7957
7927 7958 /*
7928 7959 * If the group is not the default group and the default
7929 7960 * group is reserved, the ring count in the default group
7930 7961 * might have changed, update it.
7931 7962 */
7932 7963 if (grp != defgrp &&
7933 7964 defgrp->mrg_state == MAC_GROUP_STATE_RESERVED) {
7934 7965 gmcip = MAC_GROUP_ONLY_CLIENT(defgrp);
7935 7966 emrp = MCIP_EFFECTIVE_PROPS(gmcip);
7936 7967 emrp->mrp_ntxrings = defgrp->mrg_cur_count;
7937 7968 }
7938 7969 }
7939 7970 emrp = MCIP_EFFECTIVE_PROPS(mcip);
7940 7971 }
7941 7972
7942 7973 /*
7943 7974 * Check if the primary is in the default group. If so, see if we
7944 7975 * can give it a an exclusive group now that another client is
7945 7976 * being configured. We take the primary out of the default group
7946 7977 * because the multicast/broadcast packets for the all the clients
7947 7978 * will land in the default ring in the default group which means
7948 7979 * any client in the default group, even if it is the only on in
7949 7980 * the group, will lose exclusive access to the rings, hence
7950 7981 * polling.
7951 7982 */
7952 7983 mac_client_impl_t *
7953 7984 mac_check_primary_relocation(mac_client_impl_t *mcip, boolean_t rxhw)
7954 7985 {
7955 7986 mac_impl_t *mip = mcip->mci_mip;
7956 7987 mac_group_t *defgrp = MAC_DEFAULT_RX_GROUP(mip);
7957 7988 flow_entry_t *flent = mcip->mci_flent;
7958 7989 mac_resource_props_t *mrp = MCIP_RESOURCE_PROPS(mcip);
7959 7990 uint8_t *mac_addr;
7960 7991 mac_group_t *ngrp;
7961 7992
7962 7993 /*
7963 7994 * Check if the primary is in the default group, if not
7964 7995 * or if it is explicitly configured to be in the default
7965 7996 * group OR set the RX rings property, return.
7966 7997 */
7967 7998 if (flent->fe_rx_ring_group != defgrp || mrp->mrp_mask & MRP_RX_RINGS)
7968 7999 return (NULL);
7969 8000
7970 8001 /*
7971 8002 * If the new client needs an exclusive group and we
7972 8003 * don't have another for the primary, return.
7973 8004 */
7974 8005 if (rxhw && mip->mi_rxhwclnt_avail < 2)
7975 8006 return (NULL);
7976 8007
7977 8008 mac_addr = flent->fe_flow_desc.fd_dst_mac;
7978 8009 /*
7979 8010 * We call this when we are setting up the datapath for
7980 8011 * the first non-primary.
7981 8012 */
7982 8013 ASSERT(mip->mi_nactiveclients == 2);
7983 8014 /*
7984 8015 * OK, now we have the primary that needs to be relocated.
7985 8016 */
7986 8017 ngrp = mac_reserve_rx_group(mcip, mac_addr, B_TRUE);
7987 8018 if (ngrp == NULL)
7988 8019 return (NULL);
7989 8020 if (mac_rx_switch_group(mcip, defgrp, ngrp) != 0) {
7990 8021 mac_stop_group(ngrp);
7991 8022 return (NULL);
7992 8023 }
7993 8024 return (mcip);
7994 8025 }
|
↓ open down ↓ |
6305 lines elided |
↑ open up ↑ |
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX