Print this page
NEX-20178 Heavy read load using 10G i40e causes network disconnect
MFV illumos-joyent@83a8d0d616db36010b59cc850d1926c0f6a30de1
OS-7457 i40e Tx freezes on zero descriptors
Reviewed by: Robert Mustacchi <rm@joyent.com>
Reviewed by: Rob Johnston <rob.johnston@joyent.com>
Approved by: Robert Mustacchi <rm@joyent.com>
MFV illumos-joyent@0d3f2b61dcfb18edace4fd257054f6fdbe07c99c
OS-7492 i40e Tx freeze when b_cont chain exceeds 8 descriptors
Reviewed by: Robert Mustacchi <rm@joyent.com>
Reviewed by: Rob Johnston <rob.johnston@joyent.com>
Approved by: Robert Mustacchi <rm@joyent.com>
MFV illumos-joyent@b4bede175d4c50ac1b36078a677b69388f6fb59f
OS-7577 initialize FC for i40e
Reviewed by: Robert Mustacchi <rm@joyent.com>
Approved by: Rob Johnston <rob.johnston@joyent.com>
MFV illumos-joyent@83a8d0d616db36010b59cc850d1926c0f6a30de1
OS-7457 i40e Tx freezes on zero descriptors
Reviewed by: Robert Mustacchi <rm@joyent.com>
Reviewed by: Rob Johnston <rob.johnston@joyent.com>
Approved by: Robert Mustacchi <rm@joyent.com>
NEX-19928 i40e: cannot create static IP address
Reviewed by: Cynthia Eastham <cynthia.eastham@nexenta.com>
MFV: illumos-joyent@b93056a35d6d6d301f24bc6631fc57dd2c8992c4
OS-7456 i40e default VSI sometimes lacks implicit L2 filter
Reviewed by: Robert Mustacchi <rm@joyent.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author: Ryan Zezeski <rpz@joyent.com>
MFV: illumos-joyent@61dc3dec4f82a3e13e94609a0a83d5f66c64e760
OS-6846 want i40e multi-group support
OS-7372 i40e_alloc_ring_mem() unwinds when it shouldn't
Reviewed by: Robert Mustacchi <rm@joyent.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author: Ryan Zezeski <rpz@joyent.com>
MFV: illumos-joyent@9e30beee2f0c127bf41868db46257124206e28d6
OS-5225 Want Fortville TSO support
Reviewed by: Ryan Zezeski <rpz@joyent.com>
Reviewed by: Robert Mustacchi <rm@joyent.com>
Approved by: Patrick Mooney <patrick.mooney@joyent.com>
Author: Rob Johnston <rob.johnston@joyent.com>
NEX-7822 40Gb Intel XL710 NIC performance data
Reviewed by: Steve Peng <steve.peng@nexenta.com>
Reviewed by: Evan Layton <evan.layton@nexenta.com>
| Split |
Close |
| Expand all |
| Collapse all |
--- old/usr/src/uts/common/io/i40e/i40e_main.c
+++ new/usr/src/uts/common/io/i40e/i40e_main.c
1 1 /*
2 2 * This file and its contents are supplied under the terms of the
3 3 * Common Development and Distribution License ("CDDL"), version 1.0.
|
↓ open down ↓ |
3 lines elided |
↑ open up ↑ |
4 4 * You may only use this file in accordance with the terms of version
5 5 * 1.0 of the CDDL.
6 6 *
7 7 * A full copy of the text of the CDDL should have accompanied this
8 8 * source. A copy of the CDDL is also available via the Internet at
9 9 * http://www.illumos.org/license/CDDL.
10 10 */
11 11
12 12 /*
13 13 * Copyright 2015 OmniTI Computer Consulting, Inc. All rights reserved.
14 - * Copyright (c) 2017, Joyent, Inc.
14 + * Copyright 2019 Joyent, Inc.
15 15 * Copyright 2017 Tegile Systems, Inc. All rights reserved.
16 16 */
17 17
18 18 /*
19 19 * i40e - Intel 10/40 Gb Ethernet driver
20 20 *
21 21 * The i40e driver is the main software device driver for the Intel 40 Gb family
22 22 * of devices. Note that these devices come in many flavors with both 40 GbE
23 23 * ports and 10 GbE ports. This device is the successor to the 82599 family of
24 24 * devices (ixgbe).
25 25 *
26 26 * Unlike previous generations of Intel 1 GbE and 10 GbE devices, the 40 GbE
27 27 * devices defined in the XL710 controller (previously known as Fortville) are a
28 28 * rather different beast and have a small switch embedded inside of them. In
29 29 * addition, the way that most of the programming is done has been overhauled.
30 30 * As opposed to just using PCIe memory mapped registers, it also has an
31 31 * administrative queue which is used to communicate with firmware running on
32 32 * the chip.
33 33 *
34 34 * Each physical function in the hardware shows up as a device that this driver
35 35 * will bind to. The hardware splits many resources evenly across all of the
36 36 * physical functions present on the device, while other resources are instead
37 37 * shared across the entire card and its up to the device driver to
38 38 * intelligently partition them.
39 39 *
40 40 * ------------
41 41 * Organization
42 42 * ------------
43 43 *
44 44 * This driver is made up of several files which have their own theory
45 45 * statements spread across them. We'll touch on the high level purpose of each
46 46 * file here, and then we'll get into more discussion on how the device is
47 47 * generally modelled with respect to the interfaces in illumos.
48 48 *
49 49 * i40e_gld.c: This file contains all of the bindings to MAC and the networking
50 50 * stack.
51 51 *
52 52 * i40e_intr.c: This file contains all of the interrupt service routines and
53 53 * contains logic to enable and disable interrupts on the hardware.
54 54 * It also contains the logic to map hardware resources such as the
55 55 * rings to and from interrupts and controls their ability to fire.
56 56 *
57 57 * There is a big theory statement on interrupts present there.
58 58 *
59 59 * i40e_main.c: The file that you're currently in. It interfaces with the
60 60 * traditional OS DDI interfaces and is in charge of configuring
61 61 * the device.
62 62 *
63 63 * i40e_osdep.[ch]: These files contain interfaces and definitions needed to
64 64 * work with Intel's common code for the device.
65 65 *
66 66 * i40e_stats.c: This file contains the general work and logic around our
67 67 * kstats. A theory statement on their organization and use of the
68 68 * hardware exists there.
69 69 *
70 70 * i40e_sw.h: This header file contains all of the primary structure definitions
71 71 * and constants that are used across the entire driver.
72 72 *
73 73 * i40e_transceiver.c: This file contains all of the logic for sending and
74 74 * receiving data. It contains all of the ring and DMA
75 75 * allocation logic, as well as, the actual interfaces to
76 76 * send and receive data.
77 77 *
78 78 * A big theory statement on ring management, descriptors,
79 79 * and how it ties into the OS is present there.
80 80 *
81 81 * --------------
82 82 * General Design
83 83 * --------------
84 84 *
85 85 * Before we go too far into the general way we've laid out data structures and
86 86 * the like, it's worth taking some time to explain how the hardware is
87 87 * organized. This organization informs a lot of how we do things at this time
88 88 * in the driver.
89 89 *
90 90 * Each physical device consists of a number of one or more ports, which are
91 91 * considered physical functions in the PCI sense and thus each get enumerated
92 92 * by the system, resulting in an instance being created and attached to. While
93 93 * there are many resources that are unique to each physical function eg.
94 94 * instance of the device, there are many that are shared across all of them.
95 95 * Several resources have an amount reserved for each Virtual Station Interface
96 96 * (VSI) and then a static pool of resources, available for all functions on the
97 97 * card.
98 98 *
99 99 * The most important resource in hardware are its transmit and receive queue
100 100 * pairs (i40e_trqpair_t). These should be thought of as rings in GLDv3
101 101 * parlance. There are a set number of these on each device; however, they are
102 102 * statically partitioned among all of the different physical functions.
103 103 *
104 104 * 'Fortville' (the code name for this device family) is basically a switch. To
105 105 * map MAC addresses and other things to queues, we end up having to create
106 106 * Virtual Station Interfaces (VSIs) and establish forwarding rules that direct
107 107 * traffic to a queue. A VSI owns a collection of queues and has a series of
108 108 * forwarding rules that point to it. One way to think of this is to treat it
109 109 * like MAC does a VNIC. When MAC refers to a group, a collection of rings and
110 110 * classification resources, that is a VSI in i40e.
111 111 *
112 112 * The sets of VSIs is shared across the entire device, though there may be some
113 113 * amount that are reserved to each PF. Because the GLDv3 does not let us change
114 114 * the number of groups dynamically, we instead statically divide this amount
115 115 * evenly between all the functions that exist. In addition, we have the same
116 116 * problem with the mac address forwarding rules. There are a static number that
117 117 * exist shared across all the functions.
118 118 *
119 119 * To handle both of these resources, what we end up doing is going through and
120 120 * determining which functions belong to the same device. Nominally one might do
121 121 * this by having a nexus driver; however, a prime requirement for a nexus
122 122 * driver is identifying the various children and activating them. While it is
123 123 * possible to get this information from NVRAM, we would end up duplicating a
124 124 * lot of the PCI enumeration logic. Really, at the end of the day, the device
125 125 * doesn't give us the traditional identification properties we want from a
126 126 * nexus driver.
127 127 *
128 128 * Instead, we rely on some properties that are guaranteed to be unique. While
129 129 * it might be tempting to leverage the PBA or serial number of the device from
|
↓ open down ↓ |
105 lines elided |
↑ open up ↑ |
130 130 * NVRAM, there is nothing that says that two devices can't be mis-programmed to
131 131 * have the same values in NVRAM. Instead, we uniquely identify a group of
132 132 * functions based on their parent in the /devices tree, their PCI bus and PCI
133 133 * function identifiers. Using either on their own may not be sufficient.
134 134 *
135 135 * For each unique PCI device that we encounter, we'll create a i40e_device_t.
136 136 * From there, because we don't have a good way to tell the GLDv3 about sharing
137 137 * resources between everything, we'll end up just dividing the resources
138 138 * evenly between all of the functions. Longer term, if we don't have to declare
139 139 * to the GLDv3 that these resources are shared, then we'll maintain a pool and
140 - * hae each PF allocate from the pool in the device, thus if only two of four
140 + * have each PF allocate from the pool in the device, thus if only two of four
141 141 * ports are being used, for example, then all of the resources can still be
142 142 * used.
143 143 *
144 144 * -------------------------------------------
145 145 * Transmit and Receive Queue Pair Allocations
146 146 * -------------------------------------------
147 147 *
148 148 * NVRAM ends up assigning each PF its own share of the transmit and receive LAN
149 149 * queue pairs, we have no way of modifying it, only observing it. From there,
150 150 * it's up to us to map these queues to VSIs and VFs. Since we don't support any
151 151 * VFs at this time, we only focus on assignments to VSIs.
152 152 *
153 153 * At the moment, we used a static mapping of transmit/receive queue pairs to a
154 154 * given VSI (eg. rings to a group). Though in the fullness of time, we want to
155 155 * make this something which is fully dynamic and take advantage of documented,
156 156 * but not yet available functionality for adding filters based on VXLAN and
157 157 * other encapsulation technologies.
158 158 *
159 159 * -------------------------------------
160 160 * Broadcast, Multicast, and Promiscuous
161 161 * -------------------------------------
|
↓ open down ↓ |
11 lines elided |
↑ open up ↑ |
162 162 *
163 163 * As part of the GLDv3, we need to make sure that we can handle receiving
164 164 * broadcast and multicast traffic. As well as enabling promiscuous mode when
165 165 * requested. GLDv3 requires that all broadcast and multicast traffic be
166 166 * retrieved by the default group, eg. the first one. This is the same thing as
167 167 * the default VSI.
168 168 *
169 169 * To receieve broadcast traffic, we enable it through the admin queue, rather
170 170 * than use one of our filters for it. For multicast traffic, we reserve a
171 171 * certain number of the hash filters and assign them to a given PF. When we
172 - * exceed those, we then switch to using promicuous mode for multicast traffic.
172 + * exceed those, we then switch to using promiscuous mode for multicast traffic.
173 173 *
174 174 * More specifically, once we exceed the number of filters (indicated because
175 175 * the i40e_t`i40e_resources.ifr_nmcastfilt ==
176 176 * i40e_t`i40e_resources.ifr_nmcastfilt_used), we then instead need to toggle
177 177 * promiscuous mode. If promiscuous mode is toggled then we keep track of the
178 178 * number of MACs added to it by incrementing i40e_t`i40e_mcast_promisc_count.
179 179 * That will stay enabled until that count reaches zero indicating that we have
180 180 * only added multicast addresses that we have a corresponding entry for.
181 181 *
182 182 * Because MAC itself wants to toggle promiscuous mode, which includes both
183 183 * unicast and multicast traffic, we go through and keep track of that
184 184 * ourselves. That is maintained through the use of the i40e_t`i40e_promisc_on
185 185 * member.
186 186 *
187 187 * --------------
188 188 * VSI Management
189 189 * --------------
190 190 *
191 - * At this time, we currently only support a single MAC group, and thus a single
192 - * VSI. This VSI is considered the default VSI and should be the only one that
193 - * exists after a reset. Currently it is stored as the member
194 - * i40e_t`i40e_vsi_id. While this works for the moment and for an initial
195 - * driver, it's not sufficient for the longer-term path of the driver. Instead,
196 - * we'll want to actually have a unique i40e_vsi_t structure which is used
197 - * everywhere. Note that this means that every place that uses the
198 - * i40e_t`i40e_vsi_id will need to be refactored.
191 + * The PFs share 384 VSIs. The firmware creates one VSI per PF by default.
192 + * During chip start we retrieve the SEID of this VSI and assign it as the
193 + * default VSI for our VEB (one VEB per PF). We then add additional VSIs to
194 + * the VEB up to the determined number of rx groups: i40e_t`i40e_num_rx_groups.
195 + * We currently cap this number to I40E_GROUP_MAX to a) make sure all PFs can
196 + * allocate the same number of VSIs, and b) to keep the interrupt multiplexing
197 + * under control. In the future, when we improve the interrupt allocation, we
198 + * may want to revisit this cap to make better use of the available VSIs. The
199 + * VSI allocation and configuration can be found in i40e_chip_start().
199 200 *
200 201 * ----------------
201 202 * Structure Layout
202 203 * ----------------
203 204 *
204 205 * The following images relates the core data structures together. The primary
205 206 * structure in the system is the i40e_t. It itself contains multiple rings,
206 207 * i40e_trqpair_t's which contain the various transmit and receive data. The
207 208 * receive data is stored outside of the i40e_trqpair_t and instead in the
208 209 * i40e_rx_data_t. The i40e_t has a corresponding i40e_device_t which keeps
209 210 * track of per-physical device state. Finally, for every active descriptor,
210 211 * there is a corresponding control block, which is where the
211 212 * i40e_rx_control_block_t and the i40e_tx_control_block_t come from.
212 213 *
213 214 * +-----------------------+ +-----------------------+
214 215 * | Global i40e_t list | | Global Device list |
215 216 * | | +--| |
216 217 * | i40e_glist | | | i40e_dlist |
217 218 * +-----------------------+ | +-----------------------+
218 219 * | v
219 220 * | +------------------------+ +-----------------------+
220 221 * | | Device-wide Structure |----->| Device-wide Structure |--> ...
221 222 * | | i40e_device_t | | i40e_device_t |
222 223 * | | | +-----------------------+
223 224 * | | dev_info_t * ------+--> Parent in devices tree.
224 225 * | | uint_t ------+--> PCI bus number
225 226 * | | uint_t ------+--> PCI device number
226 227 * | | uint_t ------+--> Number of functions
227 228 * | | i40e_switch_rsrcs_t ---+--> Captured total switch resources
228 229 * | | list_t ------+-------------+
229 230 * | +------------------------+ |
230 231 * | ^ |
231 232 * | +--------+ |
232 233 * | | v
|
↓ open down ↓ |
24 lines elided |
↑ open up ↑ |
233 234 * | +---------------------------+ | +-------------------+
234 235 * +->| GLDv3 Device, per PF |-----|-->| GLDv3 Device (PF) |--> ...
235 236 * | i40e_t | | | i40e_t |
236 237 * | **Primary Structure** | | +-------------------+
237 238 * | | |
238 239 * | i40e_device_t * --+-----+
239 240 * | i40e_state_t --+---> Device State
240 241 * | i40e_hw_t --+---> Intel common code structure
241 242 * | mac_handle_t --+---> GLDv3 handle to MAC
242 243 * | ddi_periodic_t --+---> Link activity timer
243 - * | int (vsi_id) --+---> VSI ID, main identifier
244 + * | i40e_vsi_t * --+---> Array of VSIs
244 245 * | i40e_func_rsrc_t --+---> Available hardware resources
245 246 * | i40e_switch_rsrc_t * --+---> Switch resource snapshot
246 247 * | i40e_sdu --+---> Current MTU
247 248 * | i40e_frame_max --+---> Current HW frame size
248 249 * | i40e_uaddr_t * --+---> Array of assigned unicast MACs
249 250 * | i40e_maddr_t * --+---> Array of assigned multicast MACs
250 251 * | i40e_mcast_promisccount --+---> Active multicast state
251 252 * | i40e_promisc_on --+---> Current promiscuous mode state
252 - * | int --+---> Number of transmit/receive pairs
253 + * | uint_t --+---> Number of transmit/receive pairs
254 + * | i40e_rx_group_t * --+---> Array of Rx groups
253 255 * | kstat_t * --+---> PF kstats
254 - * | kstat_t * --+---> VSI kstats
255 256 * | i40e_pf_stats_t --+---> PF kstat backing data
256 - * | i40e_vsi_stats_t --+---> VSI kstat backing data
257 257 * | i40e_trqpair_t * --+---------+
258 258 * +---------------------------+ |
259 259 * |
260 260 * v
261 261 * +-------------------------------+ +-----------------------------+
262 262 * | Transmit/Receive Queue Pair |-------| Transmit/Receive Queue Pair |->...
263 263 * | i40e_trqpair_t | | i40e_trqpair_t |
264 264 * + Ring Data Structure | +-----------------------------+
265 265 * | |
266 266 * | mac_ring_handle_t +--> MAC RX ring handle
267 267 * | mac_ring_handle_t +--> MAC TX ring handle
268 268 * | i40e_rxq_stat_t --+--> RX Queue stats
269 269 * | i40e_txq_stat_t --+--> TX Queue stats
270 270 * | uint32_t (tx ring size) +--> TX Ring Size
271 271 * | uint32_t (tx free list size) +--> TX Free List Size
272 272 * | i40e_dma_buffer_t --------+--> TX Descriptor ring DMA
273 273 * | i40e_tx_desc_t * --------+--> TX descriptor ring
274 274 * | volatile unt32_t * +--> TX Write back head
275 275 * | uint32_t -------+--> TX ring head
276 276 * | uint32_t -------+--> TX ring tail
277 277 * | uint32_t -------+--> Num TX desc free
278 278 * | i40e_tx_control_block_t * --+--> TX control block array ---+
279 279 * | i40e_tx_control_block_t ** --+--> TCB work list ----+
280 280 * | i40e_tx_control_block_t ** --+--> TCB free list ---+
281 281 * | uint32_t -------+--> Free TCB count |
282 282 * | i40e_rx_data_t * -------+--+ v
283 283 * +-------------------------------+ | +---------------------------+
284 284 * | | Per-TX Frame Metadata |
285 285 * | | i40e_tx_control_block_t |
286 286 * +--------------------+ | |
287 287 * | mblk to transmit <--+--- mblk_t * |
288 288 * | type of transmit <--+--- i40e_tx_type_t |
289 289 * | TX DMA handle <--+--- ddi_dma_handle_t |
290 290 * v TX DMA buffer <--+--- i40e_dma_buffer_t |
291 291 * +------------------------------+ +---------------------------+
292 292 * | Core Receive Data |
293 293 * | i40e_rx_data_t |
294 294 * | |
295 295 * | i40e_dma_buffer_t --+--> RX descriptor DMA Data
296 296 * | i40e_rx_desc_t --+--> RX descriptor ring
297 297 * | uint32_t --+--> Next free desc.
298 298 * | i40e_rx_control_block_t * --+--> RX Control Block Array ---+
299 299 * | i40e_rx_control_block_t ** --+--> RCB work list ---+
300 300 * | i40e_rx_control_block_t ** --+--> RCB free list ---+
301 301 * +------------------------------+ |
302 302 * ^ |
303 303 * | +---------------------------+ |
304 304 * | | Per-RX Frame Metadata |<---------------+
305 305 * | | i40e_rx_control_block_t |
306 306 * | | |
307 307 * | | mblk_t * ----+--> Received mblk_t data
308 308 * | | uint32_t ----+--> Reference count
309 309 * | | i40e_dma_buffer_t ----+--> Receive data DMA info
310 310 * | | frtn_t ----+--> mblk free function info
311 311 * +-----+-- i40e_rx_data_t * |
312 312 * +---------------------------+
313 313 *
314 314 * -------------
315 315 * Lock Ordering
316 316 * -------------
317 317 *
318 318 * In order to ensure that we don't deadlock, the following represents the
319 319 * lock order being used. When grabbing locks, follow the following order. Lower
320 320 * numbers are more important. Thus, the i40e_glock which is number 0, must be
321 321 * taken before any other locks in the driver. On the other hand, the
322 322 * i40e_t`i40e_stat_lock, has the highest number because it's the least
323 323 * important lock. Note, that just because one lock is higher than another does
324 324 * not mean that all intermediary locks are required.
325 325 *
326 326 * 0) i40e_glock
327 327 * 1) i40e_t`i40e_general_lock
328 328 *
329 329 * 2) i40e_trqpair_t`itrq_rx_lock
330 330 * 3) i40e_trqpair_t`itrq_tx_lock
331 331 * 4) i40e_t`i40e_rx_pending_lock
332 332 * 5) i40e_trqpair_t`itrq_tcb_lock
333 333 *
334 334 * 6) i40e_t`i40e_stat_lock
335 335 *
336 336 * Rules and expectations:
337 337 *
338 338 * 1) A thread holding locks belong to one PF should not hold locks belonging to
339 339 * a second. If for some reason this becomes necessary, locks should be grabbed
340 340 * based on the list order in the i40e_device_t, which implies that the
341 341 * i40e_glock is held.
342 342 *
343 343 * 2) When grabbing locks between multiple transmit and receive queues, the
344 344 * locks for the lowest number transmit/receive queue should be grabbed first.
345 345 *
346 346 * 3) When grabbing both the transmit and receive lock for a given queue, always
347 347 * grab i40e_trqpair_t`itrq_rx_lock before the i40e_trqpair_t`itrq_tx_lock.
348 348 *
349 349 * 4) The following pairs of locks are not expected to be held at the same time:
350 350 *
351 351 * o i40e_t`i40e_rx_pending_lock and i40e_trqpair_t`itrq_tcb_lock
|
↓ open down ↓ |
85 lines elided |
↑ open up ↑ |
352 352 *
353 353 * -----------
354 354 * Future Work
355 355 * -----------
356 356 *
357 357 * At the moment the i40e_t driver is rather bare bones, allowing us to start
358 358 * getting data flowing and folks using it while we develop additional features.
359 359 * While bugs have been filed to cover this future work, the following gives an
360 360 * overview of expected work:
361 361 *
362 - * o TSO support
363 - * o Multiple group support
364 362 * o DMA binding and breaking up the locking in ring recycling.
365 363 * o Enhanced detection of device errors
366 364 * o Participation in IRM
367 365 * o FMA device reset
368 366 * o Stall detection, temperature error detection, etc.
369 367 * o More dynamic resource pools
370 368 */
371 369
372 370 #include "i40e_sw.h"
373 371
374 -static char i40e_ident[] = "Intel 10/40Gb Ethernet v1.0.1";
372 +static char i40e_ident[] = "Intel 10/40Gb Ethernet v1.0.3";
375 373
376 374 /*
377 375 * The i40e_glock primarily protects the lists below and the i40e_device_t
378 376 * structures.
379 377 */
380 378 static kmutex_t i40e_glock;
381 379 static list_t i40e_glist;
382 380 static list_t i40e_dlist;
383 381
384 382 /*
385 383 * Access attributes for register mapping.
386 384 */
387 385 static ddi_device_acc_attr_t i40e_regs_acc_attr = {
388 386 DDI_DEVICE_ATTR_V1,
389 387 DDI_STRUCTURE_LE_ACC,
390 388 DDI_STRICTORDER_ACC,
391 389 DDI_FLAGERR_ACC
392 390 };
393 391
394 392 /*
395 393 * Logging function for this driver.
396 394 */
397 395 static void
398 396 i40e_dev_err(i40e_t *i40e, int level, boolean_t console, const char *fmt,
399 397 va_list ap)
400 398 {
401 399 char buf[1024];
402 400
403 401 (void) vsnprintf(buf, sizeof (buf), fmt, ap);
404 402
405 403 if (i40e == NULL) {
406 404 cmn_err(level, (console) ? "%s: %s" : "!%s: %s",
407 405 I40E_MODULE_NAME, buf);
408 406 } else {
409 407 dev_err(i40e->i40e_dip, level, (console) ? "%s" : "!%s",
410 408 buf);
411 409 }
412 410 }
413 411
414 412 /*
415 413 * Because there's the stupid trailing-comma problem with the C preprocessor
416 414 * and variable arguments, I need to instantiate these. Pardon the redundant
417 415 * code.
418 416 */
419 417 /*PRINTFLIKE2*/
420 418 void
421 419 i40e_error(i40e_t *i40e, const char *fmt, ...)
422 420 {
423 421 va_list ap;
424 422
425 423 va_start(ap, fmt);
426 424 i40e_dev_err(i40e, CE_WARN, B_FALSE, fmt, ap);
427 425 va_end(ap);
428 426 }
429 427
430 428 /*PRINTFLIKE2*/
431 429 void
432 430 i40e_log(i40e_t *i40e, const char *fmt, ...)
433 431 {
434 432 va_list ap;
435 433
436 434 va_start(ap, fmt);
437 435 i40e_dev_err(i40e, CE_NOTE, B_FALSE, fmt, ap);
438 436 va_end(ap);
439 437 }
440 438
441 439 /*PRINTFLIKE2*/
442 440 void
443 441 i40e_notice(i40e_t *i40e, const char *fmt, ...)
444 442 {
445 443 va_list ap;
446 444
447 445 va_start(ap, fmt);
448 446 i40e_dev_err(i40e, CE_NOTE, B_TRUE, fmt, ap);
449 447 va_end(ap);
450 448 }
451 449
452 450 /*
453 451 * Various parts of the driver need to know if the controller is from the X722
454 452 * family, which has a few additional capabilities and different programming
455 453 * means. We don't consider virtual functions as part of this as they are quite
456 454 * different and will require substantially more work.
457 455 */
458 456 static boolean_t
459 457 i40e_is_x722(i40e_t *i40e)
460 458 {
461 459 return (i40e->i40e_hw_space.mac.type == I40E_MAC_X722);
462 460 }
463 461
464 462 static void
465 463 i40e_device_rele(i40e_t *i40e)
466 464 {
467 465 i40e_device_t *idp = i40e->i40e_device;
468 466
469 467 if (idp == NULL)
470 468 return;
471 469
472 470 mutex_enter(&i40e_glock);
473 471 VERIFY(idp->id_nreg > 0);
474 472 list_remove(&idp->id_i40e_list, i40e);
475 473 idp->id_nreg--;
476 474 if (idp->id_nreg == 0) {
477 475 list_remove(&i40e_dlist, idp);
478 476 list_destroy(&idp->id_i40e_list);
479 477 kmem_free(idp->id_rsrcs, sizeof (i40e_switch_rsrc_t) *
480 478 idp->id_rsrcs_alloc);
481 479 kmem_free(idp, sizeof (i40e_device_t));
482 480 }
483 481 i40e->i40e_device = NULL;
484 482 mutex_exit(&i40e_glock);
485 483 }
486 484
487 485 static i40e_device_t *
488 486 i40e_device_find(i40e_t *i40e, dev_info_t *parent, uint_t bus, uint_t device)
489 487 {
490 488 i40e_device_t *idp;
491 489 mutex_enter(&i40e_glock);
492 490 for (idp = list_head(&i40e_dlist); idp != NULL;
493 491 idp = list_next(&i40e_dlist, idp)) {
494 492 if (idp->id_parent == parent && idp->id_pci_bus == bus &&
495 493 idp->id_pci_device == device) {
496 494 break;
497 495 }
498 496 }
499 497
500 498 if (idp != NULL) {
501 499 VERIFY(idp->id_nreg < idp->id_nfuncs);
502 500 idp->id_nreg++;
503 501 } else {
504 502 i40e_hw_t *hw = &i40e->i40e_hw_space;
505 503 ASSERT(hw->num_ports > 0);
506 504 ASSERT(hw->num_partitions > 0);
507 505
508 506 /*
509 507 * The Intel common code doesn't exactly keep the number of PCI
510 508 * functions. But it calculates it during discovery of
511 509 * partitions and ports. So what we do is undo the calculation
512 510 * that it does originally, as functions are evenly spread
513 511 * across ports in the rare case of partitions.
514 512 */
515 513 idp = kmem_alloc(sizeof (i40e_device_t), KM_SLEEP);
516 514 idp->id_parent = parent;
517 515 idp->id_pci_bus = bus;
518 516 idp->id_pci_device = device;
519 517 idp->id_nfuncs = hw->num_ports * hw->num_partitions;
520 518 idp->id_nreg = 1;
521 519 idp->id_rsrcs_alloc = i40e->i40e_switch_rsrc_alloc;
522 520 idp->id_rsrcs_act = i40e->i40e_switch_rsrc_actual;
523 521 idp->id_rsrcs = kmem_alloc(sizeof (i40e_switch_rsrc_t) *
524 522 idp->id_rsrcs_alloc, KM_SLEEP);
525 523 bcopy(i40e->i40e_switch_rsrcs, idp->id_rsrcs,
526 524 sizeof (i40e_switch_rsrc_t) * idp->id_rsrcs_alloc);
527 525 list_create(&idp->id_i40e_list, sizeof (i40e_t),
528 526 offsetof(i40e_t, i40e_dlink));
529 527
530 528 list_insert_tail(&i40e_dlist, idp);
531 529 }
532 530
533 531 list_insert_tail(&idp->id_i40e_list, i40e);
534 532 mutex_exit(&i40e_glock);
535 533
536 534 return (idp);
537 535 }
538 536
539 537 static void
540 538 i40e_link_state_set(i40e_t *i40e, link_state_t state)
541 539 {
542 540 if (i40e->i40e_link_state == state)
543 541 return;
544 542
545 543 i40e->i40e_link_state = state;
546 544 mac_link_update(i40e->i40e_mac_hdl, i40e->i40e_link_state);
547 545 }
548 546
549 547 /*
550 548 * This is a basic link check routine. Mostly we're using this just to see
551 549 * if we can get any accurate information about the state of the link being
552 550 * up or down, as well as updating the link state, speed, etc. information.
553 551 */
554 552 void
555 553 i40e_link_check(i40e_t *i40e)
556 554 {
557 555 i40e_hw_t *hw = &i40e->i40e_hw_space;
558 556 boolean_t ls;
559 557 int ret;
560 558
561 559 ASSERT(MUTEX_HELD(&i40e->i40e_general_lock));
562 560
563 561 hw->phy.get_link_info = B_TRUE;
564 562 if ((ret = i40e_get_link_status(hw, &ls)) != I40E_SUCCESS) {
565 563 i40e->i40e_s_link_status_errs++;
566 564 i40e->i40e_s_link_status_lasterr = ret;
567 565 return;
568 566 }
569 567
570 568 /*
571 569 * Firmware abstracts all of the mac and phy information for us, so we
572 570 * can use i40e_get_link_status to determine the current state.
573 571 */
574 572 if (ls == B_TRUE) {
575 573 enum i40e_aq_link_speed speed;
576 574
577 575 speed = i40e_get_link_speed(hw);
578 576
579 577 /*
580 578 * Translate from an i40e value to a value in Mbits/s.
581 579 */
582 580 switch (speed) {
583 581 case I40E_LINK_SPEED_100MB:
584 582 i40e->i40e_link_speed = 100;
585 583 break;
586 584 case I40E_LINK_SPEED_1GB:
587 585 i40e->i40e_link_speed = 1000;
588 586 break;
589 587 case I40E_LINK_SPEED_10GB:
590 588 i40e->i40e_link_speed = 10000;
591 589 break;
592 590 case I40E_LINK_SPEED_20GB:
593 591 i40e->i40e_link_speed = 20000;
594 592 break;
595 593 case I40E_LINK_SPEED_40GB:
596 594 i40e->i40e_link_speed = 40000;
597 595 break;
598 596 case I40E_LINK_SPEED_25GB:
599 597 i40e->i40e_link_speed = 25000;
600 598 break;
601 599 default:
602 600 i40e->i40e_link_speed = 0;
603 601 break;
604 602 }
605 603
606 604 /*
607 605 * At this time, hardware does not support half-duplex
608 606 * operation, hence why we don't ask the hardware about our
609 607 * current speed.
610 608 */
611 609 i40e->i40e_link_duplex = LINK_DUPLEX_FULL;
612 610 i40e_link_state_set(i40e, LINK_STATE_UP);
613 611 } else {
614 612 i40e->i40e_link_speed = 0;
615 613 i40e->i40e_link_duplex = 0;
616 614 i40e_link_state_set(i40e, LINK_STATE_DOWN);
617 615 }
618 616 }
619 617
620 618 static void
621 619 i40e_rem_intrs(i40e_t *i40e)
622 620 {
623 621 int i, rc;
624 622
625 623 for (i = 0; i < i40e->i40e_intr_count; i++) {
626 624 rc = ddi_intr_free(i40e->i40e_intr_handles[i]);
627 625 if (rc != DDI_SUCCESS) {
628 626 i40e_log(i40e, "failed to free interrupt %d: %d",
629 627 i, rc);
630 628 }
631 629 }
632 630
633 631 kmem_free(i40e->i40e_intr_handles, i40e->i40e_intr_size);
634 632 i40e->i40e_intr_handles = NULL;
635 633 }
636 634
637 635 static void
638 636 i40e_rem_intr_handlers(i40e_t *i40e)
639 637 {
640 638 int i, rc;
641 639
642 640 for (i = 0; i < i40e->i40e_intr_count; i++) {
643 641 rc = ddi_intr_remove_handler(i40e->i40e_intr_handles[i]);
644 642 if (rc != DDI_SUCCESS) {
645 643 i40e_log(i40e, "failed to remove interrupt %d: %d",
646 644 i, rc);
647 645 }
648 646 }
649 647 }
650 648
651 649 /*
652 650 * illumos Fault Management Architecture (FMA) support.
653 651 */
654 652
655 653 int
656 654 i40e_check_acc_handle(ddi_acc_handle_t handle)
657 655 {
658 656 ddi_fm_error_t de;
659 657
660 658 ddi_fm_acc_err_get(handle, &de, DDI_FME_VERSION);
661 659 ddi_fm_acc_err_clear(handle, DDI_FME_VERSION);
662 660 return (de.fme_status);
663 661 }
664 662
665 663 int
666 664 i40e_check_dma_handle(ddi_dma_handle_t handle)
667 665 {
668 666 ddi_fm_error_t de;
669 667
670 668 ddi_fm_dma_err_get(handle, &de, DDI_FME_VERSION);
671 669 return (de.fme_status);
672 670 }
673 671
674 672 /*
675 673 * Fault service error handling callback function.
676 674 */
677 675 /* ARGSUSED */
678 676 static int
679 677 i40e_fm_error_cb(dev_info_t *dip, ddi_fm_error_t *err, const void *impl_data)
680 678 {
681 679 pci_ereport_post(dip, err, NULL);
682 680 return (err->fme_status);
683 681 }
684 682
685 683 static void
686 684 i40e_fm_init(i40e_t *i40e)
687 685 {
688 686 ddi_iblock_cookie_t iblk;
689 687
690 688 i40e->i40e_fm_capabilities = ddi_prop_get_int(DDI_DEV_T_ANY,
691 689 i40e->i40e_dip, DDI_PROP_DONTPASS, "fm_capable",
692 690 DDI_FM_EREPORT_CAPABLE | DDI_FM_ACCCHK_CAPABLE |
693 691 DDI_FM_DMACHK_CAPABLE | DDI_FM_ERRCB_CAPABLE);
694 692
695 693 if (i40e->i40e_fm_capabilities < 0) {
696 694 i40e->i40e_fm_capabilities = 0;
697 695 } else if (i40e->i40e_fm_capabilities > 0xf) {
698 696 i40e->i40e_fm_capabilities = DDI_FM_EREPORT_CAPABLE |
699 697 DDI_FM_ACCCHK_CAPABLE | DDI_FM_DMACHK_CAPABLE |
700 698 DDI_FM_ERRCB_CAPABLE;
701 699 }
702 700
703 701 /*
704 702 * Only register with IO Fault Services if we have some capability
705 703 */
706 704 if (i40e->i40e_fm_capabilities & DDI_FM_ACCCHK_CAPABLE) {
707 705 i40e_regs_acc_attr.devacc_attr_access = DDI_FLAGERR_ACC;
708 706 } else {
709 707 i40e_regs_acc_attr.devacc_attr_access = DDI_DEFAULT_ACC;
710 708 }
711 709
712 710 if (i40e->i40e_fm_capabilities) {
713 711 ddi_fm_init(i40e->i40e_dip, &i40e->i40e_fm_capabilities, &iblk);
714 712
715 713 if (DDI_FM_EREPORT_CAP(i40e->i40e_fm_capabilities) ||
716 714 DDI_FM_ERRCB_CAP(i40e->i40e_fm_capabilities)) {
717 715 pci_ereport_setup(i40e->i40e_dip);
718 716 }
719 717
720 718 if (DDI_FM_ERRCB_CAP(i40e->i40e_fm_capabilities)) {
721 719 ddi_fm_handler_register(i40e->i40e_dip,
722 720 i40e_fm_error_cb, (void*)i40e);
723 721 }
724 722 }
725 723
726 724 if (i40e->i40e_fm_capabilities & DDI_FM_DMACHK_CAPABLE) {
727 725 i40e_init_dma_attrs(i40e, B_TRUE);
728 726 } else {
729 727 i40e_init_dma_attrs(i40e, B_FALSE);
730 728 }
731 729 }
732 730
733 731 static void
734 732 i40e_fm_fini(i40e_t *i40e)
735 733 {
736 734 if (i40e->i40e_fm_capabilities) {
737 735
738 736 if (DDI_FM_EREPORT_CAP(i40e->i40e_fm_capabilities) ||
739 737 DDI_FM_ERRCB_CAP(i40e->i40e_fm_capabilities))
740 738 pci_ereport_teardown(i40e->i40e_dip);
741 739
742 740 if (DDI_FM_ERRCB_CAP(i40e->i40e_fm_capabilities))
743 741 ddi_fm_handler_unregister(i40e->i40e_dip);
744 742
745 743 ddi_fm_fini(i40e->i40e_dip);
746 744 }
747 745 }
748 746
749 747 void
750 748 i40e_fm_ereport(i40e_t *i40e, char *detail)
751 749 {
752 750 uint64_t ena;
753 751 char buf[FM_MAX_CLASS];
|
↓ open down ↓ |
369 lines elided |
↑ open up ↑ |
754 752
755 753 (void) snprintf(buf, FM_MAX_CLASS, "%s.%s", DDI_FM_DEVICE, detail);
756 754 ena = fm_ena_generate(0, FM_ENA_FMT1);
757 755 if (DDI_FM_EREPORT_CAP(i40e->i40e_fm_capabilities)) {
758 756 ddi_fm_ereport_post(i40e->i40e_dip, buf, ena, DDI_NOSLEEP,
759 757 FM_VERSION, DATA_TYPE_UINT8, FM_EREPORT_VERS0, NULL);
760 758 }
761 759 }
762 760
763 761 /*
764 - * Here we're trying to get the ID of the default VSI. In general, when we come
765 - * through and look at this shortly after attach, we expect there to only be a
766 - * single element present, which is the default VSI. Importantly, each PF seems
767 - * to not see any other devices, in part because of the simple switch mode that
768 - * we're using. If for some reason, we see more artifact, we'll need to revisit
769 - * what we're doing here.
762 + * Here we're trying to set the SEID of the default VSI. In general,
763 + * when we come through and look at this shortly after attach, we
764 + * expect there to only be a single element present, which is the
765 + * default VSI. Importantly, each PF seems to not see any other
766 + * devices, in part because of the simple switch mode that we're
767 + * using. If for some reason, we see more artifacts, we'll need to
768 + * revisit what we're doing here.
770 769 */
771 -static int
772 -i40e_get_vsi_id(i40e_t *i40e)
770 +static boolean_t
771 +i40e_set_def_vsi_seid(i40e_t *i40e)
773 772 {
774 773 i40e_hw_t *hw = &i40e->i40e_hw_space;
775 774 struct i40e_aqc_get_switch_config_resp *sw_config;
776 775 uint8_t aq_buf[I40E_AQ_LARGE_BUF];
777 776 uint16_t next = 0;
778 777 int rc;
779 778
780 779 /* LINTED: E_BAD_PTR_CAST_ALIGN */
781 780 sw_config = (struct i40e_aqc_get_switch_config_resp *)aq_buf;
782 781 rc = i40e_aq_get_switch_config(hw, sw_config, sizeof (aq_buf), &next,
783 782 NULL);
784 783 if (rc != I40E_SUCCESS) {
785 784 i40e_error(i40e, "i40e_aq_get_switch_config() failed %d: %d",
786 785 rc, hw->aq.asq_last_status);
787 - return (-1);
786 + return (B_FALSE);
788 787 }
789 788
790 789 if (LE_16(sw_config->header.num_reported) != 1) {
791 790 i40e_error(i40e, "encountered multiple (%d) switching units "
792 791 "during attach, not proceeding",
793 792 LE_16(sw_config->header.num_reported));
793 + return (B_FALSE);
794 + }
795 +
796 + I40E_DEF_VSI_SEID(i40e) = sw_config->element[0].seid;
797 + return (B_TRUE);
798 +}
799 +
800 +/*
801 + * Get the SEID of the uplink MAC.
802 + */
803 +static int
804 +i40e_get_mac_seid(i40e_t *i40e)
805 +{
806 + i40e_hw_t *hw = &i40e->i40e_hw_space;
807 + struct i40e_aqc_get_switch_config_resp *sw_config;
808 + uint8_t aq_buf[I40E_AQ_LARGE_BUF];
809 + uint16_t next = 0;
810 + int rc;
811 +
812 + /* LINTED: E_BAD_PTR_CAST_ALIGN */
813 + sw_config = (struct i40e_aqc_get_switch_config_resp *)aq_buf;
814 + rc = i40e_aq_get_switch_config(hw, sw_config, sizeof (aq_buf), &next,
815 + NULL);
816 + if (rc != I40E_SUCCESS) {
817 + i40e_error(i40e, "i40e_aq_get_switch_config() failed %d: %d",
818 + rc, hw->aq.asq_last_status);
794 819 return (-1);
795 820 }
796 821
797 - return (sw_config->element[0].seid);
822 + return (LE_16(sw_config->element[0].uplink_seid));
798 823 }
799 824
800 825 /*
801 826 * We need to fill the i40e_hw_t structure with the capabilities of this PF. We
802 827 * must also provide the memory for it; however, we don't need to keep it around
803 828 * to the call to the common code. It takes it and parses it into an internal
804 829 * structure.
805 830 */
806 831 static boolean_t
807 832 i40e_get_hw_capabilities(i40e_t *i40e, i40e_hw_t *hw)
808 833 {
809 834 struct i40e_aqc_list_capabilities_element_resp *buf;
810 835 int rc;
811 836 size_t len;
812 837 uint16_t needed;
813 838 int nelems = I40E_HW_CAP_DEFAULT;
814 839
815 840 len = nelems * sizeof (*buf);
816 841
817 842 for (;;) {
818 843 ASSERT(len > 0);
819 844 buf = kmem_alloc(len, KM_SLEEP);
820 845 rc = i40e_aq_discover_capabilities(hw, buf, len,
821 846 &needed, i40e_aqc_opc_list_func_capabilities, NULL);
822 847 kmem_free(buf, len);
823 848
824 849 if (hw->aq.asq_last_status == I40E_AQ_RC_ENOMEM &&
825 850 nelems == I40E_HW_CAP_DEFAULT) {
826 851 if (nelems == needed) {
827 852 i40e_error(i40e, "Capability discovery failed "
828 853 "due to byzantine common code");
829 854 return (B_FALSE);
830 855 }
831 856 len = needed;
832 857 continue;
833 858 } else if (rc != I40E_SUCCESS ||
834 859 hw->aq.asq_last_status != I40E_AQ_RC_OK) {
835 860 i40e_error(i40e, "Capability discovery failed: %d", rc);
836 861 return (B_FALSE);
837 862 }
838 863
839 864 break;
840 865 }
841 866
842 867 return (B_TRUE);
843 868 }
844 869
845 870 /*
846 871 * Obtain the switch's capabilities as seen by this PF and keep it around for
847 872 * our later use.
848 873 */
849 874 static boolean_t
850 875 i40e_get_switch_resources(i40e_t *i40e)
851 876 {
852 877 i40e_hw_t *hw = &i40e->i40e_hw_space;
853 878 uint8_t cnt = 2;
854 879 uint8_t act;
855 880 size_t size;
856 881 i40e_switch_rsrc_t *buf;
857 882
858 883 for (;;) {
859 884 enum i40e_status_code ret;
860 885 size = cnt * sizeof (i40e_switch_rsrc_t);
861 886 ASSERT(size > 0);
862 887 if (size > UINT16_MAX)
863 888 return (B_FALSE);
864 889 buf = kmem_alloc(size, KM_SLEEP);
865 890
866 891 ret = i40e_aq_get_switch_resource_alloc(hw, &act, buf,
867 892 cnt, NULL);
868 893 if (ret == I40E_ERR_ADMIN_QUEUE_ERROR &&
869 894 hw->aq.asq_last_status == I40E_AQ_RC_EINVAL) {
870 895 kmem_free(buf, size);
871 896 cnt += I40E_SWITCH_CAP_DEFAULT;
872 897 continue;
873 898 } else if (ret != I40E_SUCCESS) {
874 899 kmem_free(buf, size);
875 900 i40e_error(i40e,
876 901 "failed to retrieve switch statistics: %d", ret);
877 902 return (B_FALSE);
878 903 }
879 904
880 905 break;
881 906 }
882 907
883 908 i40e->i40e_switch_rsrc_alloc = cnt;
884 909 i40e->i40e_switch_rsrc_actual = act;
885 910 i40e->i40e_switch_rsrcs = buf;
886 911
887 912 return (B_TRUE);
888 913 }
889 914
890 915 static void
891 916 i40e_cleanup_resources(i40e_t *i40e)
892 917 {
893 918 if (i40e->i40e_uaddrs != NULL) {
894 919 kmem_free(i40e->i40e_uaddrs, sizeof (i40e_uaddr_t) *
895 920 i40e->i40e_resources.ifr_nmacfilt);
896 921 i40e->i40e_uaddrs = NULL;
897 922 }
898 923
899 924 if (i40e->i40e_maddrs != NULL) {
900 925 kmem_free(i40e->i40e_maddrs, sizeof (i40e_maddr_t) *
901 926 i40e->i40e_resources.ifr_nmcastfilt);
902 927 i40e->i40e_maddrs = NULL;
903 928 }
904 929
905 930 if (i40e->i40e_switch_rsrcs != NULL) {
906 931 size_t sz = sizeof (i40e_switch_rsrc_t) *
907 932 i40e->i40e_switch_rsrc_alloc;
908 933 ASSERT(sz > 0);
909 934 kmem_free(i40e->i40e_switch_rsrcs, sz);
910 935 i40e->i40e_switch_rsrcs = NULL;
911 936 }
912 937
913 938 if (i40e->i40e_device != NULL)
914 939 i40e_device_rele(i40e);
915 940 }
916 941
917 942 static boolean_t
918 943 i40e_get_available_resources(i40e_t *i40e)
919 944 {
920 945 dev_info_t *parent;
921 946 uint16_t bus, device, func;
922 947 uint_t nregs;
923 948 int *regs, i;
924 949 i40e_device_t *idp;
925 950 i40e_hw_t *hw = &i40e->i40e_hw_space;
926 951
927 952 parent = ddi_get_parent(i40e->i40e_dip);
928 953
929 954 if (ddi_prop_lookup_int_array(DDI_DEV_T_ANY, i40e->i40e_dip, 0, "reg",
930 955 ®s, &nregs) != DDI_PROP_SUCCESS) {
931 956 return (B_FALSE);
932 957 }
933 958
934 959 if (nregs < 1) {
935 960 ddi_prop_free(regs);
936 961 return (B_FALSE);
937 962 }
938 963
939 964 bus = PCI_REG_BUS_G(regs[0]);
940 965 device = PCI_REG_DEV_G(regs[0]);
941 966 func = PCI_REG_FUNC_G(regs[0]);
942 967 ddi_prop_free(regs);
943 968
944 969 i40e->i40e_hw_space.bus.func = func;
945 970 i40e->i40e_hw_space.bus.device = device;
946 971
947 972 if (i40e_get_switch_resources(i40e) == B_FALSE) {
948 973 return (B_FALSE);
949 974 }
950 975
951 976 /*
952 977 * To calculate the total amount of a resource we have available, we
953 978 * need to add how many our i40e_t thinks it has guaranteed, if any, and
954 979 * then we need to go through and divide the number of available on the
955 980 * device, which was snapshotted before anyone should have allocated
956 981 * anything, and use that to derive how many are available from the
957 982 * pool. Longer term, we may want to turn this into something that's
958 983 * more of a pool-like resource that everything can share (though that
959 984 * may require some more assistance from MAC).
960 985 *
961 986 * Though for transmit and receive queue pairs, we just have to ask
962 987 * firmware instead.
963 988 */
964 989 idp = i40e_device_find(i40e, parent, bus, device);
965 990 i40e->i40e_device = idp;
966 991 i40e->i40e_resources.ifr_nvsis = 0;
967 992 i40e->i40e_resources.ifr_nvsis_used = 0;
968 993 i40e->i40e_resources.ifr_nmacfilt = 0;
969 994 i40e->i40e_resources.ifr_nmacfilt_used = 0;
970 995 i40e->i40e_resources.ifr_nmcastfilt = 0;
971 996 i40e->i40e_resources.ifr_nmcastfilt_used = 0;
972 997
973 998 for (i = 0; i < i40e->i40e_switch_rsrc_actual; i++) {
974 999 i40e_switch_rsrc_t *srp = &i40e->i40e_switch_rsrcs[i];
975 1000
976 1001 switch (srp->resource_type) {
977 1002 case I40E_AQ_RESOURCE_TYPE_VSI:
978 1003 i40e->i40e_resources.ifr_nvsis +=
979 1004 LE_16(srp->guaranteed);
980 1005 i40e->i40e_resources.ifr_nvsis_used = LE_16(srp->used);
981 1006 break;
982 1007 case I40E_AQ_RESOURCE_TYPE_MACADDR:
983 1008 i40e->i40e_resources.ifr_nmacfilt +=
984 1009 LE_16(srp->guaranteed);
985 1010 i40e->i40e_resources.ifr_nmacfilt_used =
986 1011 LE_16(srp->used);
987 1012 break;
988 1013 case I40E_AQ_RESOURCE_TYPE_MULTICAST_HASH:
989 1014 i40e->i40e_resources.ifr_nmcastfilt +=
990 1015 LE_16(srp->guaranteed);
991 1016 i40e->i40e_resources.ifr_nmcastfilt_used =
992 1017 LE_16(srp->used);
993 1018 break;
994 1019 default:
995 1020 break;
996 1021 }
997 1022 }
998 1023
999 1024 for (i = 0; i < idp->id_rsrcs_act; i++) {
1000 1025 i40e_switch_rsrc_t *srp = &i40e->i40e_switch_rsrcs[i];
1001 1026 switch (srp->resource_type) {
1002 1027 case I40E_AQ_RESOURCE_TYPE_VSI:
1003 1028 i40e->i40e_resources.ifr_nvsis +=
1004 1029 LE_16(srp->total_unalloced) / idp->id_nfuncs;
1005 1030 break;
1006 1031 case I40E_AQ_RESOURCE_TYPE_MACADDR:
1007 1032 i40e->i40e_resources.ifr_nmacfilt +=
1008 1033 LE_16(srp->total_unalloced) / idp->id_nfuncs;
1009 1034 break;
1010 1035 case I40E_AQ_RESOURCE_TYPE_MULTICAST_HASH:
1011 1036 i40e->i40e_resources.ifr_nmcastfilt +=
1012 1037 LE_16(srp->total_unalloced) / idp->id_nfuncs;
1013 1038 default:
1014 1039 break;
1015 1040 }
1016 1041 }
1017 1042
1018 1043 i40e->i40e_resources.ifr_nrx_queue = hw->func_caps.num_rx_qp;
1019 1044 i40e->i40e_resources.ifr_ntx_queue = hw->func_caps.num_tx_qp;
1020 1045
1021 1046 i40e->i40e_uaddrs = kmem_zalloc(sizeof (i40e_uaddr_t) *
1022 1047 i40e->i40e_resources.ifr_nmacfilt, KM_SLEEP);
1023 1048 i40e->i40e_maddrs = kmem_zalloc(sizeof (i40e_maddr_t) *
1024 1049 i40e->i40e_resources.ifr_nmcastfilt, KM_SLEEP);
1025 1050
1026 1051 /*
1027 1052 * Initialize these as multicast addresses to indicate it's invalid for
1028 1053 * sanity purposes. Think of it like 0xdeadbeef.
1029 1054 */
1030 1055 for (i = 0; i < i40e->i40e_resources.ifr_nmacfilt; i++)
1031 1056 i40e->i40e_uaddrs[i].iua_mac[0] = 0x01;
1032 1057
1033 1058 return (B_TRUE);
1034 1059 }
1035 1060
1036 1061 static boolean_t
1037 1062 i40e_enable_interrupts(i40e_t *i40e)
1038 1063 {
1039 1064 int i, rc;
1040 1065
1041 1066 if (i40e->i40e_intr_cap & DDI_INTR_FLAG_BLOCK) {
1042 1067 rc = ddi_intr_block_enable(i40e->i40e_intr_handles,
1043 1068 i40e->i40e_intr_count);
1044 1069 if (rc != DDI_SUCCESS) {
1045 1070 i40e_error(i40e, "Interrupt block-enable failed: %d",
1046 1071 rc);
1047 1072 return (B_FALSE);
1048 1073 }
1049 1074 } else {
1050 1075 for (i = 0; i < i40e->i40e_intr_count; i++) {
1051 1076 rc = ddi_intr_enable(i40e->i40e_intr_handles[i]);
1052 1077 if (rc != DDI_SUCCESS) {
1053 1078 i40e_error(i40e,
1054 1079 "Failed to enable interrupt %d: %d", i, rc);
1055 1080 while (--i >= 0) {
1056 1081 (void) ddi_intr_disable(
1057 1082 i40e->i40e_intr_handles[i]);
1058 1083 }
1059 1084 return (B_FALSE);
1060 1085 }
1061 1086 }
1062 1087 }
1063 1088
1064 1089 return (B_TRUE);
1065 1090 }
1066 1091
1067 1092 static boolean_t
1068 1093 i40e_disable_interrupts(i40e_t *i40e)
1069 1094 {
1070 1095 int i, rc;
1071 1096
1072 1097 if (i40e->i40e_intr_cap & DDI_INTR_FLAG_BLOCK) {
1073 1098 rc = ddi_intr_block_disable(i40e->i40e_intr_handles,
1074 1099 i40e->i40e_intr_count);
1075 1100 if (rc != DDI_SUCCESS) {
1076 1101 i40e_error(i40e,
1077 1102 "Interrupt block-disabled failed: %d", rc);
1078 1103 return (B_FALSE);
1079 1104 }
1080 1105 } else {
1081 1106 for (i = 0; i < i40e->i40e_intr_count; i++) {
1082 1107 rc = ddi_intr_disable(i40e->i40e_intr_handles[i]);
1083 1108 if (rc != DDI_SUCCESS) {
1084 1109 i40e_error(i40e,
1085 1110 "Failed to disable interrupt %d: %d",
1086 1111 i, rc);
1087 1112 return (B_FALSE);
1088 1113 }
1089 1114 }
1090 1115 }
|
↓ open down ↓ |
283 lines elided |
↑ open up ↑ |
1091 1116
1092 1117 return (B_TRUE);
1093 1118 }
1094 1119
1095 1120 /*
1096 1121 * Free receive & transmit rings.
1097 1122 */
1098 1123 static void
1099 1124 i40e_free_trqpairs(i40e_t *i40e)
1100 1125 {
1101 - int i;
1102 1126 i40e_trqpair_t *itrq;
1103 1127
1128 + if (i40e->i40e_rx_groups != NULL) {
1129 + kmem_free(i40e->i40e_rx_groups,
1130 + sizeof (i40e_rx_group_t) * i40e->i40e_num_rx_groups);
1131 + i40e->i40e_rx_groups = NULL;
1132 + }
1133 +
1104 1134 if (i40e->i40e_trqpairs != NULL) {
1105 - for (i = 0; i < i40e->i40e_num_trqpairs; i++) {
1135 + for (uint_t i = 0; i < i40e->i40e_num_trqpairs; i++) {
1106 1136 itrq = &i40e->i40e_trqpairs[i];
1107 1137 mutex_destroy(&itrq->itrq_rx_lock);
1108 1138 mutex_destroy(&itrq->itrq_tx_lock);
1109 1139 mutex_destroy(&itrq->itrq_tcb_lock);
1110 1140
1111 1141 /*
1112 1142 * Should have already been cleaned up by start/stop,
1113 1143 * etc.
1114 1144 */
1115 1145 ASSERT(itrq->itrq_txkstat == NULL);
1116 1146 ASSERT(itrq->itrq_rxkstat == NULL);
1117 1147 }
1118 1148
1119 1149 kmem_free(i40e->i40e_trqpairs,
1120 1150 sizeof (i40e_trqpair_t) * i40e->i40e_num_trqpairs);
1121 1151 i40e->i40e_trqpairs = NULL;
1122 1152 }
1123 1153
1124 1154 cv_destroy(&i40e->i40e_rx_pending_cv);
1125 1155 mutex_destroy(&i40e->i40e_rx_pending_lock);
|
↓ open down ↓ |
10 lines elided |
↑ open up ↑ |
1126 1156 mutex_destroy(&i40e->i40e_general_lock);
1127 1157 }
1128 1158
1129 1159 /*
1130 1160 * Allocate transmit and receive rings, as well as other data structures that we
1131 1161 * need.
1132 1162 */
1133 1163 static boolean_t
1134 1164 i40e_alloc_trqpairs(i40e_t *i40e)
1135 1165 {
1136 - int i;
1137 1166 void *mutexpri = DDI_INTR_PRI(i40e->i40e_intr_pri);
1138 1167
1139 1168 /*
1140 1169 * Now that we have the priority for the interrupts, initialize
1141 1170 * all relevant locks.
1142 1171 */
1143 1172 mutex_init(&i40e->i40e_general_lock, NULL, MUTEX_DRIVER, mutexpri);
1144 1173 mutex_init(&i40e->i40e_rx_pending_lock, NULL, MUTEX_DRIVER, mutexpri);
1145 1174 cv_init(&i40e->i40e_rx_pending_cv, NULL, CV_DRIVER, NULL);
1146 1175
1147 1176 i40e->i40e_trqpairs = kmem_zalloc(sizeof (i40e_trqpair_t) *
1148 1177 i40e->i40e_num_trqpairs, KM_SLEEP);
1149 - for (i = 0; i < i40e->i40e_num_trqpairs; i++) {
1178 + for (uint_t i = 0; i < i40e->i40e_num_trqpairs; i++) {
1150 1179 i40e_trqpair_t *itrq = &i40e->i40e_trqpairs[i];
1151 1180
1152 1181 itrq->itrq_i40e = i40e;
1153 1182 mutex_init(&itrq->itrq_rx_lock, NULL, MUTEX_DRIVER, mutexpri);
1154 1183 mutex_init(&itrq->itrq_tx_lock, NULL, MUTEX_DRIVER, mutexpri);
1155 1184 mutex_init(&itrq->itrq_tcb_lock, NULL, MUTEX_DRIVER, mutexpri);
1156 1185 itrq->itrq_index = i;
1157 1186 }
1158 1187
1188 + i40e->i40e_rx_groups = kmem_zalloc(sizeof (i40e_rx_group_t) *
1189 + i40e->i40e_num_rx_groups, KM_SLEEP);
1190 +
1191 + for (uint_t i = 0; i < i40e->i40e_num_rx_groups; i++) {
1192 + i40e_rx_group_t *rxg = &i40e->i40e_rx_groups[i];
1193 +
1194 + rxg->irg_index = i;
1195 + rxg->irg_i40e = i40e;
1196 + }
1197 +
1159 1198 return (B_TRUE);
1160 1199 }
1161 1200
1162 1201
1163 1202
1164 1203 /*
1165 1204 * Unless a .conf file already overrode i40e_t structure values, they will
1166 1205 * be 0, and need to be set in conjunction with the now-available HW report.
1167 - *
1168 - * However, at the moment, we cap all of these resources as we only support a
1169 - * single receive ring and a single group.
1170 1206 */
1171 1207 /* ARGSUSED */
1172 1208 static void
1173 1209 i40e_hw_to_instance(i40e_t *i40e, i40e_hw_t *hw)
1174 1210 {
1175 - if (i40e->i40e_num_trqpairs == 0) {
1176 - i40e->i40e_num_trqpairs = I40E_TRQPAIR_MAX;
1211 + if (i40e->i40e_num_trqpairs_per_vsi == 0) {
1212 + if (i40e_is_x722(i40e)) {
1213 + i40e->i40e_num_trqpairs_per_vsi =
1214 + I40E_722_MAX_TC_QUEUES;
1215 + } else {
1216 + i40e->i40e_num_trqpairs_per_vsi =
1217 + I40E_710_MAX_TC_QUEUES;
1218 + }
1177 1219 }
1178 1220
1179 1221 if (i40e->i40e_num_rx_groups == 0) {
1180 1222 i40e->i40e_num_rx_groups = I40E_GROUP_MAX;
1181 1223 }
1182 1224 }
1183 1225
1184 1226 /*
1185 1227 * Free any resources required by, or setup by, the Intel common code.
1186 1228 */
1187 1229 static void
1188 1230 i40e_common_code_fini(i40e_t *i40e)
1189 1231 {
1190 1232 i40e_hw_t *hw = &i40e->i40e_hw_space;
1191 1233 int rc;
1192 1234
1193 1235 rc = i40e_shutdown_lan_hmc(hw);
1194 1236 if (rc != I40E_SUCCESS)
1195 1237 i40e_error(i40e, "failed to shutdown LAN hmc: %d", rc);
1196 1238
1197 1239 rc = i40e_shutdown_adminq(hw);
1198 1240 if (rc != I40E_SUCCESS)
1199 1241 i40e_error(i40e, "failed to shutdown admin queue: %d", rc);
1200 1242 }
1201 1243
1202 1244 /*
1203 1245 * Initialize and call Intel common-code routines, includes some setup
1204 1246 * the common code expects from the driver. Also prints on failure, so
1205 1247 * the caller doesn't have to.
1206 1248 */
1207 1249 static boolean_t
1208 1250 i40e_common_code_init(i40e_t *i40e, i40e_hw_t *hw)
1209 1251 {
1210 1252 int rc;
1211 1253
1212 1254 i40e_clear_hw(hw);
1213 1255 rc = i40e_pf_reset(hw);
1214 1256 if (rc != 0) {
1215 1257 i40e_error(i40e, "failed to reset hardware: %d", rc);
1216 1258 i40e_fm_ereport(i40e, DDI_FM_DEVICE_NO_RESPONSE);
1217 1259 return (B_FALSE);
1218 1260 }
1219 1261
1220 1262 rc = i40e_init_shared_code(hw);
1221 1263 if (rc != 0) {
1222 1264 i40e_error(i40e, "failed to initialize i40e core: %d", rc);
1223 1265 return (B_FALSE);
1224 1266 }
1225 1267
1226 1268 hw->aq.num_arq_entries = I40E_DEF_ADMINQ_SIZE;
1227 1269 hw->aq.num_asq_entries = I40E_DEF_ADMINQ_SIZE;
1228 1270 hw->aq.arq_buf_size = I40E_ADMINQ_BUFSZ;
1229 1271 hw->aq.asq_buf_size = I40E_ADMINQ_BUFSZ;
1230 1272
1231 1273 rc = i40e_init_adminq(hw);
1232 1274 if (rc != 0) {
1233 1275 i40e_error(i40e, "failed to initialize firmware admin queue: "
1234 1276 "%d, potential firmware version mismatch", rc);
1235 1277 i40e_fm_ereport(i40e, DDI_FM_DEVICE_INVAL_STATE);
1236 1278 return (B_FALSE);
1237 1279 }
1238 1280
1239 1281 if (hw->aq.api_maj_ver == I40E_FW_API_VERSION_MAJOR &&
1240 1282 hw->aq.api_min_ver > I40E_FW_API_VERSION_MINOR) {
1241 1283 i40e_log(i40e, "The driver for the device detected a newer "
1242 1284 "version of the NVM image (%d.%d) than expected (%d.%d).\n"
1243 1285 "Please install the most recent version of the network "
1244 1286 "driver.\n", hw->aq.api_maj_ver, hw->aq.api_min_ver,
1245 1287 I40E_FW_API_VERSION_MAJOR, I40E_FW_API_VERSION_MINOR);
1246 1288 } else if (hw->aq.api_maj_ver < I40E_FW_API_VERSION_MAJOR ||
1247 1289 hw->aq.api_min_ver < (I40E_FW_API_VERSION_MINOR - 1)) {
1248 1290 i40e_log(i40e, "The driver for the device detected an older"
1249 1291 " version of the NVM image (%d.%d) than expected (%d.%d)."
1250 1292 "\nPlease update the NVM image.\n",
1251 1293 hw->aq.api_maj_ver, hw->aq.api_min_ver,
1252 1294 I40E_FW_API_VERSION_MAJOR, I40E_FW_API_VERSION_MINOR - 1);
1253 1295 }
1254 1296
1255 1297 i40e_clear_pxe_mode(hw);
1256 1298
1257 1299 /*
1258 1300 * We need to call this so that the common code can discover
1259 1301 * capabilities of the hardware, which it uses throughout the rest.
1260 1302 */
1261 1303 if (!i40e_get_hw_capabilities(i40e, hw)) {
1262 1304 i40e_error(i40e, "failed to obtain hardware capabilities");
1263 1305 return (B_FALSE);
1264 1306 }
1265 1307
1266 1308 if (i40e_get_available_resources(i40e) == B_FALSE) {
1267 1309 i40e_error(i40e, "failed to obtain hardware resources");
1268 1310 return (B_FALSE);
1269 1311 }
1270 1312
1271 1313 i40e_hw_to_instance(i40e, hw);
1272 1314
1273 1315 rc = i40e_init_lan_hmc(hw, hw->func_caps.num_tx_qp,
1274 1316 hw->func_caps.num_rx_qp, 0, 0);
1275 1317 if (rc != 0) {
1276 1318 i40e_error(i40e, "failed to initialize hardware memory cache: "
1277 1319 "%d", rc);
1278 1320 return (B_FALSE);
1279 1321 }
1280 1322
1281 1323 rc = i40e_configure_lan_hmc(hw, I40E_HMC_MODEL_DIRECT_ONLY);
1282 1324 if (rc != 0) {
1283 1325 i40e_error(i40e, "failed to configure hardware memory cache: "
1284 1326 "%d", rc);
1285 1327 return (B_FALSE);
1286 1328 }
1287 1329
1288 1330 (void) i40e_aq_stop_lldp(hw, TRUE, NULL);
1289 1331
1290 1332 rc = i40e_get_mac_addr(hw, hw->mac.addr);
1291 1333 if (rc != I40E_SUCCESS) {
1292 1334 i40e_error(i40e, "failed to retrieve hardware mac address: %d",
1293 1335 rc);
1294 1336 return (B_FALSE);
1295 1337 }
1296 1338
1297 1339 rc = i40e_validate_mac_addr(hw->mac.addr);
1298 1340 if (rc != 0) {
1299 1341 i40e_error(i40e, "failed to validate internal mac address: "
1300 1342 "%d", rc);
1301 1343 return (B_FALSE);
|
↓ open down ↓ |
115 lines elided |
↑ open up ↑ |
1302 1344 }
1303 1345 bcopy(hw->mac.addr, hw->mac.perm_addr, ETHERADDRL);
1304 1346 if ((rc = i40e_get_port_mac_addr(hw, hw->mac.port_addr)) !=
1305 1347 I40E_SUCCESS) {
1306 1348 i40e_error(i40e, "failed to retrieve port mac address: %d",
1307 1349 rc);
1308 1350 return (B_FALSE);
1309 1351 }
1310 1352
1311 1353 /*
1312 - * We need to obtain the Virtual Station ID (VSI) before we can
1313 - * perform other operations on the device.
1354 + * We need to obtain the Default Virtual Station SEID (VSI)
1355 + * before we can perform other operations on the device.
1314 1356 */
1315 - i40e->i40e_vsi_id = i40e_get_vsi_id(i40e);
1316 - if (i40e->i40e_vsi_id == -1) {
1317 - i40e_error(i40e, "failed to obtain VSI ID");
1357 + if (!i40e_set_def_vsi_seid(i40e)) {
1358 + i40e_error(i40e, "failed to obtain Default VSI SEID");
1318 1359 return (B_FALSE);
1319 1360 }
1320 1361
1321 1362 return (B_TRUE);
1322 1363 }
1323 1364
1324 1365 static void
1325 1366 i40e_unconfigure(dev_info_t *devinfo, i40e_t *i40e)
1326 1367 {
1327 1368 int rc;
1328 1369
1329 1370 if (i40e->i40e_attach_progress & I40E_ATTACH_ENABLE_INTR)
1330 1371 (void) i40e_disable_interrupts(i40e);
1331 1372
1332 1373 if ((i40e->i40e_attach_progress & I40E_ATTACH_LINK_TIMER) &&
1333 1374 i40e->i40e_periodic_id != 0) {
1334 1375 ddi_periodic_delete(i40e->i40e_periodic_id);
1335 1376 i40e->i40e_periodic_id = 0;
1336 1377 }
1337 1378
1338 1379 if (i40e->i40e_attach_progress & I40E_ATTACH_MAC) {
1339 1380 rc = mac_unregister(i40e->i40e_mac_hdl);
1340 1381 if (rc != 0) {
1341 1382 i40e_error(i40e, "failed to unregister from mac: %d",
1342 1383 rc);
1343 1384 }
1344 1385 }
1345 1386
1346 1387 if (i40e->i40e_attach_progress & I40E_ATTACH_STATS) {
1347 1388 i40e_stats_fini(i40e);
1348 1389 }
1349 1390
1350 1391 if (i40e->i40e_attach_progress & I40E_ATTACH_ADD_INTR)
1351 1392 i40e_rem_intr_handlers(i40e);
1352 1393
1353 1394 if (i40e->i40e_attach_progress & I40E_ATTACH_ALLOC_RINGSLOCKS)
1354 1395 i40e_free_trqpairs(i40e);
1355 1396
1356 1397 if (i40e->i40e_attach_progress & I40E_ATTACH_ALLOC_INTR)
1357 1398 i40e_rem_intrs(i40e);
1358 1399
1359 1400 if (i40e->i40e_attach_progress & I40E_ATTACH_COMMON_CODE)
1360 1401 i40e_common_code_fini(i40e);
1361 1402
1362 1403 i40e_cleanup_resources(i40e);
1363 1404
1364 1405 if (i40e->i40e_attach_progress & I40E_ATTACH_PROPS)
1365 1406 (void) ddi_prop_remove_all(devinfo);
1366 1407
1367 1408 if (i40e->i40e_attach_progress & I40E_ATTACH_REGS_MAP &&
1368 1409 i40e->i40e_osdep_space.ios_reg_handle != NULL) {
1369 1410 ddi_regs_map_free(&i40e->i40e_osdep_space.ios_reg_handle);
1370 1411 i40e->i40e_osdep_space.ios_reg_handle = NULL;
1371 1412 }
1372 1413
1373 1414 if ((i40e->i40e_attach_progress & I40E_ATTACH_PCI_CONFIG) &&
1374 1415 i40e->i40e_osdep_space.ios_cfg_handle != NULL) {
1375 1416 pci_config_teardown(&i40e->i40e_osdep_space.ios_cfg_handle);
1376 1417 i40e->i40e_osdep_space.ios_cfg_handle = NULL;
1377 1418 }
1378 1419
1379 1420 if (i40e->i40e_attach_progress & I40E_ATTACH_FM_INIT)
1380 1421 i40e_fm_fini(i40e);
1381 1422
1382 1423 kmem_free(i40e->i40e_aqbuf, I40E_ADMINQ_BUFSZ);
1383 1424 kmem_free(i40e, sizeof (i40e_t));
1384 1425
1385 1426 ddi_set_driver_private(devinfo, NULL);
1386 1427 }
1387 1428
1388 1429 static boolean_t
1389 1430 i40e_final_init(i40e_t *i40e)
1390 1431 {
1391 1432 i40e_hw_t *hw = &i40e->i40e_hw_space;
1392 1433 struct i40e_osdep *osdep = OS_DEP(hw);
1393 1434 uint8_t pbanum[I40E_PBANUM_STRLEN];
1394 1435 enum i40e_status_code irc;
1395 1436 char buf[I40E_DDI_PROP_LEN];
1396 1437
1397 1438 pbanum[0] = '\0';
1398 1439 irc = i40e_read_pba_string(hw, pbanum, sizeof (pbanum));
1399 1440 if (irc != I40E_SUCCESS) {
1400 1441 i40e_log(i40e, "failed to read PBA string: %d", irc);
1401 1442 } else {
1402 1443 (void) ddi_prop_update_string(DDI_DEV_T_NONE, i40e->i40e_dip,
1403 1444 "printed-board-assembly", (char *)pbanum);
1404 1445 }
1405 1446
1406 1447 #ifdef DEBUG
1407 1448 ASSERT(snprintf(NULL, 0, "%d.%d", hw->aq.fw_maj_ver,
1408 1449 hw->aq.fw_min_ver) < sizeof (buf));
1409 1450 ASSERT(snprintf(NULL, 0, "%x", hw->aq.fw_build) < sizeof (buf));
1410 1451 ASSERT(snprintf(NULL, 0, "%d.%d", hw->aq.api_maj_ver,
1411 1452 hw->aq.api_min_ver) < sizeof (buf));
1412 1453 #endif
1413 1454
1414 1455 (void) snprintf(buf, sizeof (buf), "%d.%d", hw->aq.fw_maj_ver,
1415 1456 hw->aq.fw_min_ver);
1416 1457 (void) ddi_prop_update_string(DDI_DEV_T_NONE, i40e->i40e_dip,
1417 1458 "firmware-version", buf);
1418 1459 (void) snprintf(buf, sizeof (buf), "%x", hw->aq.fw_build);
1419 1460 (void) ddi_prop_update_string(DDI_DEV_T_NONE, i40e->i40e_dip,
1420 1461 "firmware-build", buf);
1421 1462 (void) snprintf(buf, sizeof (buf), "%d.%d", hw->aq.api_maj_ver,
1422 1463 hw->aq.api_min_ver);
1423 1464 (void) ddi_prop_update_string(DDI_DEV_T_NONE, i40e->i40e_dip,
1424 1465 "api-version", buf);
1425 1466
1426 1467 if (!i40e_set_hw_bus_info(hw))
1427 1468 return (B_FALSE);
1428 1469
1429 1470 if (i40e_check_acc_handle(osdep->ios_reg_handle) != DDI_FM_OK) {
1430 1471 ddi_fm_service_impact(i40e->i40e_dip, DDI_SERVICE_LOST);
1431 1472 return (B_FALSE);
1432 1473 }
1433 1474
1434 1475 return (B_TRUE);
1435 1476 }
1436 1477
1437 1478 static void
1438 1479 i40e_identify_hardware(i40e_t *i40e)
1439 1480 {
1440 1481 i40e_hw_t *hw = &i40e->i40e_hw_space;
1441 1482 struct i40e_osdep *osdep = &i40e->i40e_osdep_space;
1442 1483
1443 1484 hw->vendor_id = pci_config_get16(osdep->ios_cfg_handle, PCI_CONF_VENID);
1444 1485 hw->device_id = pci_config_get16(osdep->ios_cfg_handle, PCI_CONF_DEVID);
1445 1486 hw->revision_id = pci_config_get8(osdep->ios_cfg_handle,
1446 1487 PCI_CONF_REVID);
1447 1488 hw->subsystem_device_id =
1448 1489 pci_config_get16(osdep->ios_cfg_handle, PCI_CONF_SUBSYSID);
1449 1490 hw->subsystem_vendor_id =
1450 1491 pci_config_get16(osdep->ios_cfg_handle, PCI_CONF_SUBVENID);
1451 1492
1452 1493 /*
1453 1494 * Note that we set the hardware's bus information later on, in
1454 1495 * i40e_get_available_resources(). The common code doesn't seem to
1455 1496 * require that it be set in any ways, it seems to be mostly for
1456 1497 * book-keeping.
1457 1498 */
1458 1499 }
1459 1500
1460 1501 static boolean_t
1461 1502 i40e_regs_map(i40e_t *i40e)
1462 1503 {
1463 1504 dev_info_t *devinfo = i40e->i40e_dip;
1464 1505 i40e_hw_t *hw = &i40e->i40e_hw_space;
1465 1506 struct i40e_osdep *osdep = &i40e->i40e_osdep_space;
1466 1507 off_t memsize;
1467 1508 int ret;
1468 1509
1469 1510 if (ddi_dev_regsize(devinfo, I40E_ADAPTER_REGSET, &memsize) !=
1470 1511 DDI_SUCCESS) {
1471 1512 i40e_error(i40e, "Used invalid register set to map PCIe regs");
1472 1513 return (B_FALSE);
1473 1514 }
1474 1515
1475 1516 if ((ret = ddi_regs_map_setup(devinfo, I40E_ADAPTER_REGSET,
1476 1517 (caddr_t *)&hw->hw_addr, 0, memsize, &i40e_regs_acc_attr,
1477 1518 &osdep->ios_reg_handle)) != DDI_SUCCESS) {
1478 1519 i40e_error(i40e, "failed to map device registers: %d", ret);
1479 1520 return (B_FALSE);
1480 1521 }
1481 1522
1482 1523 osdep->ios_reg_size = memsize;
1483 1524 return (B_TRUE);
1484 1525 }
1485 1526
1486 1527 /*
1487 1528 * Update parameters required when a new MTU has been configured. Calculate the
1488 1529 * maximum frame size, as well as, size our DMA buffers which we size in
1489 1530 * increments of 1K.
1490 1531 */
1491 1532 void
1492 1533 i40e_update_mtu(i40e_t *i40e)
1493 1534 {
1494 1535 uint32_t rx, tx;
1495 1536
1496 1537 i40e->i40e_frame_max = i40e->i40e_sdu +
1497 1538 sizeof (struct ether_vlan_header) + ETHERFCSL;
1498 1539
1499 1540 rx = i40e->i40e_frame_max + I40E_BUF_IPHDR_ALIGNMENT;
1500 1541 i40e->i40e_rx_buf_size = ((rx >> 10) +
1501 1542 ((rx & (((uint32_t)1 << 10) -1)) > 0 ? 1 : 0)) << 10;
1502 1543
1503 1544 tx = i40e->i40e_frame_max;
1504 1545 i40e->i40e_tx_buf_size = ((tx >> 10) +
1505 1546 ((tx & (((uint32_t)1 << 10) -1)) > 0 ? 1 : 0)) << 10;
1506 1547 }
1507 1548
1508 1549 static int
1509 1550 i40e_get_prop(i40e_t *i40e, char *prop, int min, int max, int def)
1510 1551 {
1511 1552 int val;
1512 1553
1513 1554 val = ddi_prop_get_int(DDI_DEV_T_ANY, i40e->i40e_dip, DDI_PROP_DONTPASS,
1514 1555 prop, def);
1515 1556 if (val > max)
1516 1557 val = max;
1517 1558 if (val < min)
1518 1559 val = min;
1519 1560 return (val);
1520 1561 }
1521 1562
1522 1563 static void
1523 1564 i40e_init_properties(i40e_t *i40e)
1524 1565 {
1525 1566 i40e->i40e_sdu = i40e_get_prop(i40e, "default_mtu",
1526 1567 I40E_MIN_MTU, I40E_MAX_MTU, I40E_DEF_MTU);
1527 1568
1528 1569 i40e->i40e_intr_force = i40e_get_prop(i40e, "intr_force",
1529 1570 I40E_INTR_NONE, I40E_INTR_LEGACY, I40E_INTR_NONE);
1530 1571
1531 1572 i40e->i40e_mr_enable = i40e_get_prop(i40e, "mr_enable",
1532 1573 B_FALSE, B_TRUE, B_TRUE);
1533 1574
1534 1575 i40e->i40e_tx_ring_size = i40e_get_prop(i40e, "tx_ring_size",
1535 1576 I40E_MIN_TX_RING_SIZE, I40E_MAX_TX_RING_SIZE,
1536 1577 I40E_DEF_TX_RING_SIZE);
1537 1578 if ((i40e->i40e_tx_ring_size % I40E_DESC_ALIGN) != 0) {
1538 1579 i40e->i40e_tx_ring_size = P2ROUNDUP(i40e->i40e_tx_ring_size,
1539 1580 I40E_DESC_ALIGN);
1540 1581 }
1541 1582
1542 1583 i40e->i40e_tx_block_thresh = i40e_get_prop(i40e, "tx_resched_threshold",
1543 1584 I40E_MIN_TX_BLOCK_THRESH,
1544 1585 i40e->i40e_tx_ring_size - I40E_TX_MAX_COOKIE,
1545 1586 I40E_DEF_TX_BLOCK_THRESH);
1546 1587
1547 1588 i40e->i40e_rx_ring_size = i40e_get_prop(i40e, "rx_ring_size",
1548 1589 I40E_MIN_RX_RING_SIZE, I40E_MAX_RX_RING_SIZE,
1549 1590 I40E_DEF_RX_RING_SIZE);
1550 1591 if ((i40e->i40e_rx_ring_size % I40E_DESC_ALIGN) != 0) {
1551 1592 i40e->i40e_rx_ring_size = P2ROUNDUP(i40e->i40e_rx_ring_size,
|
↓ open down ↓ |
224 lines elided |
↑ open up ↑ |
1552 1593 I40E_DESC_ALIGN);
1553 1594 }
1554 1595
1555 1596 i40e->i40e_rx_limit_per_intr = i40e_get_prop(i40e, "rx_limit_per_intr",
1556 1597 I40E_MIN_RX_LIMIT_PER_INTR, I40E_MAX_RX_LIMIT_PER_INTR,
1557 1598 I40E_DEF_RX_LIMIT_PER_INTR);
1558 1599
1559 1600 i40e->i40e_tx_hcksum_enable = i40e_get_prop(i40e, "tx_hcksum_enable",
1560 1601 B_FALSE, B_TRUE, B_TRUE);
1561 1602
1603 + i40e->i40e_tx_lso_enable = i40e_get_prop(i40e, "tx_lso_enable",
1604 + B_FALSE, B_TRUE, B_TRUE);
1605 +
1562 1606 i40e->i40e_rx_hcksum_enable = i40e_get_prop(i40e, "rx_hcksum_enable",
1563 1607 B_FALSE, B_TRUE, B_TRUE);
1564 1608
1565 1609 i40e->i40e_rx_dma_min = i40e_get_prop(i40e, "rx_dma_threshold",
1566 1610 I40E_MIN_RX_DMA_THRESH, I40E_MAX_RX_DMA_THRESH,
1567 1611 I40E_DEF_RX_DMA_THRESH);
1568 1612
1569 1613 i40e->i40e_tx_dma_min = i40e_get_prop(i40e, "tx_dma_threshold",
1570 1614 I40E_MIN_TX_DMA_THRESH, I40E_MAX_TX_DMA_THRESH,
1571 1615 I40E_DEF_TX_DMA_THRESH);
1572 1616
1573 1617 i40e->i40e_tx_itr = i40e_get_prop(i40e, "tx_intr_throttle",
1574 1618 I40E_MIN_ITR, I40E_MAX_ITR, I40E_DEF_TX_ITR);
1575 1619
1576 1620 i40e->i40e_rx_itr = i40e_get_prop(i40e, "rx_intr_throttle",
1577 1621 I40E_MIN_ITR, I40E_MAX_ITR, I40E_DEF_RX_ITR);
1578 1622
1579 1623 i40e->i40e_other_itr = i40e_get_prop(i40e, "other_intr_throttle",
1580 1624 I40E_MIN_ITR, I40E_MAX_ITR, I40E_DEF_OTHER_ITR);
1581 1625
1582 1626 if (!i40e->i40e_mr_enable) {
1583 1627 i40e->i40e_num_trqpairs = I40E_TRQPAIR_NOMSIX;
1584 1628 i40e->i40e_num_rx_groups = I40E_GROUP_NOMSIX;
1585 1629 }
1586 1630
1587 1631 i40e_update_mtu(i40e);
1588 1632 }
1589 1633
1590 1634 /*
1591 1635 * There are a few constraints on interrupts that we're currently imposing, some
1592 1636 * of which are restrictions from hardware. For a fuller treatment, see
1593 1637 * i40e_intr.c.
1594 1638 *
1595 1639 * Currently, to use MSI-X we require two interrupts be available though in
1596 1640 * theory we should participate in IRM and happily use more interrupts.
1597 1641 *
1598 1642 * Hardware only supports a single MSI being programmed and therefore if we
1599 1643 * don't have MSI-X interrupts available at this time, then we ratchet down the
1600 1644 * number of rings and groups available. Obviously, we only bother with a single
1601 1645 * fixed interrupt.
1602 1646 */
1603 1647 static boolean_t
1604 1648 i40e_alloc_intr_handles(i40e_t *i40e, dev_info_t *devinfo, int intr_type)
1605 1649 {
1606 1650 i40e_hw_t *hw = &i40e->i40e_hw_space;
1607 1651 ddi_acc_handle_t rh = i40e->i40e_osdep_space.ios_reg_handle;
1608 1652 int request, count, actual, rc, min;
1609 1653 uint32_t reg;
1610 1654
1611 1655 switch (intr_type) {
1612 1656 case DDI_INTR_TYPE_FIXED:
1613 1657 case DDI_INTR_TYPE_MSI:
1614 1658 request = 1;
1615 1659 min = 1;
1616 1660 break;
1617 1661 case DDI_INTR_TYPE_MSIX:
1618 1662 min = 2;
1619 1663 if (!i40e->i40e_mr_enable) {
1620 1664 request = 2;
1621 1665 break;
1622 1666 }
1623 1667 reg = I40E_READ_REG(hw, I40E_GLPCI_CNF2);
1624 1668 /*
1625 1669 * Should this read fail, we will drop back to using
1626 1670 * MSI or fixed interrupts.
1627 1671 */
1628 1672 if (i40e_check_acc_handle(rh) != DDI_FM_OK) {
1629 1673 ddi_fm_service_impact(i40e->i40e_dip,
1630 1674 DDI_SERVICE_DEGRADED);
1631 1675 return (B_FALSE);
1632 1676 }
1633 1677 request = (reg & I40E_GLPCI_CNF2_MSI_X_PF_N_MASK) >>
1634 1678 I40E_GLPCI_CNF2_MSI_X_PF_N_SHIFT;
1635 1679 request++; /* the register value is n - 1 */
1636 1680 break;
1637 1681 default:
1638 1682 panic("bad interrupt type passed to i40e_alloc_intr_handles: "
1639 1683 "%d", intr_type);
1640 1684 return (B_FALSE);
1641 1685 }
1642 1686
1643 1687 rc = ddi_intr_get_nintrs(devinfo, intr_type, &count);
1644 1688 if (rc != DDI_SUCCESS || count < min) {
1645 1689 i40e_log(i40e, "Get interrupt number failed, "
1646 1690 "returned %d, count %d", rc, count);
1647 1691 return (B_FALSE);
1648 1692 }
1649 1693
1650 1694 rc = ddi_intr_get_navail(devinfo, intr_type, &count);
1651 1695 if (rc != DDI_SUCCESS || count < min) {
1652 1696 i40e_log(i40e, "Get AVAILABLE interrupt number failed, "
1653 1697 "returned %d, count %d", rc, count);
1654 1698 return (B_FALSE);
1655 1699 }
1656 1700
1657 1701 actual = 0;
1658 1702 i40e->i40e_intr_count = 0;
1659 1703 i40e->i40e_intr_count_max = 0;
1660 1704 i40e->i40e_intr_count_min = 0;
1661 1705
1662 1706 i40e->i40e_intr_size = request * sizeof (ddi_intr_handle_t);
1663 1707 ASSERT(i40e->i40e_intr_size != 0);
1664 1708 i40e->i40e_intr_handles = kmem_alloc(i40e->i40e_intr_size, KM_SLEEP);
1665 1709
1666 1710 rc = ddi_intr_alloc(devinfo, i40e->i40e_intr_handles, intr_type, 0,
1667 1711 min(request, count), &actual, DDI_INTR_ALLOC_NORMAL);
1668 1712 if (rc != DDI_SUCCESS) {
1669 1713 i40e_log(i40e, "Interrupt allocation failed with %d.", rc);
1670 1714 goto alloc_handle_fail;
1671 1715 }
1672 1716
1673 1717 i40e->i40e_intr_count = actual;
1674 1718 i40e->i40e_intr_count_max = request;
1675 1719 i40e->i40e_intr_count_min = min;
1676 1720
1677 1721 if (actual < min) {
1678 1722 i40e_log(i40e, "actual (%d) is less than minimum (%d).",
1679 1723 actual, min);
1680 1724 goto alloc_handle_fail;
1681 1725 }
1682 1726
1683 1727 /*
1684 1728 * Record the priority and capabilities for our first vector. Once
1685 1729 * we have it, that's our priority until detach time. Even if we
1686 1730 * eventually participate in IRM, our priority shouldn't change.
1687 1731 */
1688 1732 rc = ddi_intr_get_pri(i40e->i40e_intr_handles[0], &i40e->i40e_intr_pri);
1689 1733 if (rc != DDI_SUCCESS) {
1690 1734 i40e_log(i40e,
1691 1735 "Getting interrupt priority failed with %d.", rc);
1692 1736 goto alloc_handle_fail;
1693 1737 }
1694 1738
1695 1739 rc = ddi_intr_get_cap(i40e->i40e_intr_handles[0], &i40e->i40e_intr_cap);
1696 1740 if (rc != DDI_SUCCESS) {
1697 1741 i40e_log(i40e,
1698 1742 "Getting interrupt capabilities failed with %d.", rc);
1699 1743 goto alloc_handle_fail;
1700 1744 }
1701 1745
1702 1746 i40e->i40e_intr_type = intr_type;
1703 1747 return (B_TRUE);
1704 1748
1705 1749 alloc_handle_fail:
1706 1750
1707 1751 i40e_rem_intrs(i40e);
1708 1752 return (B_FALSE);
1709 1753 }
1710 1754
1711 1755 static boolean_t
1712 1756 i40e_alloc_intrs(i40e_t *i40e, dev_info_t *devinfo)
1713 1757 {
1714 1758 int intr_types, rc;
1715 1759 uint_t max_trqpairs;
1716 1760
1717 1761 if (i40e_is_x722(i40e)) {
1718 1762 max_trqpairs = I40E_722_MAX_TC_QUEUES;
1719 1763 } else {
1720 1764 max_trqpairs = I40E_710_MAX_TC_QUEUES;
|
↓ open down ↓ |
149 lines elided |
↑ open up ↑ |
1721 1765 }
1722 1766
1723 1767 rc = ddi_intr_get_supported_types(devinfo, &intr_types);
1724 1768 if (rc != DDI_SUCCESS) {
1725 1769 i40e_error(i40e, "failed to get supported interrupt types: %d",
1726 1770 rc);
1727 1771 return (B_FALSE);
1728 1772 }
1729 1773
1730 1774 i40e->i40e_intr_type = 0;
1775 + i40e->i40e_num_rx_groups = I40E_GROUP_MAX;
1731 1776
1777 + /*
1778 + * We need to determine the number of queue pairs per traffic
1779 + * class. We only have one traffic class (TC0), so we'll base
1780 + * this off the number of interrupts provided. Furthermore,
1781 + * since we only use one traffic class, the number of queues
1782 + * per traffic class and per VSI are the same.
1783 + */
1732 1784 if ((intr_types & DDI_INTR_TYPE_MSIX) &&
1733 - i40e->i40e_intr_force <= I40E_INTR_MSIX) {
1734 - if (i40e_alloc_intr_handles(i40e, devinfo,
1735 - DDI_INTR_TYPE_MSIX)) {
1736 - i40e->i40e_num_trqpairs =
1737 - MIN(i40e->i40e_intr_count - 1, max_trqpairs);
1738 - return (B_TRUE);
1739 - }
1785 + (i40e->i40e_intr_force <= I40E_INTR_MSIX) &&
1786 + (i40e_alloc_intr_handles(i40e, devinfo, DDI_INTR_TYPE_MSIX))) {
1787 + uint32_t n;
1788 +
1789 + /*
1790 + * While we want the number of queue pairs to match
1791 + * the number of interrupts, we must keep stay in
1792 + * bounds of the maximum number of queues per traffic
1793 + * class. We subtract one from i40e_intr_count to
1794 + * account for interrupt zero; which is currently
1795 + * restricted to admin queue commands and other
1796 + * interrupt causes.
1797 + */
1798 + n = MIN(i40e->i40e_intr_count - 1, max_trqpairs);
1799 + ASSERT3U(n, >, 0);
1800 +
1801 + /*
1802 + * Round up to the nearest power of two to ensure that
1803 + * the QBASE aligns with the TC size which must be
1804 + * programmed as a power of two. See the queue mapping
1805 + * description in section 7.4.9.5.5.1.
1806 + *
1807 + * If i40e_intr_count - 1 is not a power of two then
1808 + * some queue pairs on the same VSI will have to share
1809 + * an interrupt.
1810 + *
1811 + * We may want to revisit this logic in a future where
1812 + * we have more interrupts and more VSIs. Otherwise,
1813 + * each VSI will use as many interrupts as possible.
1814 + * Using more QPs per VSI means better RSS for each
1815 + * group, but at the same time may require more
1816 + * sharing of interrupts across VSIs. This may be a
1817 + * good candidate for a .conf tunable.
1818 + */
1819 + n = 0x1 << ddi_fls(n);
1820 + i40e->i40e_num_trqpairs_per_vsi = n;
1821 + ASSERT3U(i40e->i40e_num_rx_groups, >, 0);
1822 + i40e->i40e_num_trqpairs = i40e->i40e_num_trqpairs_per_vsi *
1823 + i40e->i40e_num_rx_groups;
1824 + return (B_TRUE);
1740 1825 }
1741 1826
1742 1827 /*
1743 1828 * We only use multiple transmit/receive pairs when MSI-X interrupts are
1744 1829 * available due to the fact that the device basically only supports a
1745 1830 * single MSI interrupt.
1746 1831 */
1747 1832 i40e->i40e_num_trqpairs = I40E_TRQPAIR_NOMSIX;
1833 + i40e->i40e_num_trqpairs_per_vsi = i40e->i40e_num_trqpairs;
1748 1834 i40e->i40e_num_rx_groups = I40E_GROUP_NOMSIX;
1749 1835
1750 1836 if ((intr_types & DDI_INTR_TYPE_MSI) &&
1751 1837 (i40e->i40e_intr_force <= I40E_INTR_MSI)) {
1752 1838 if (i40e_alloc_intr_handles(i40e, devinfo, DDI_INTR_TYPE_MSI))
1753 1839 return (B_TRUE);
1754 1840 }
1755 1841
1756 1842 if (intr_types & DDI_INTR_TYPE_FIXED) {
1757 1843 if (i40e_alloc_intr_handles(i40e, devinfo, DDI_INTR_TYPE_FIXED))
1758 1844 return (B_TRUE);
1759 1845 }
|
↓ open down ↓ |
2 lines elided |
↑ open up ↑ |
1760 1846
1761 1847 return (B_FALSE);
1762 1848 }
1763 1849
1764 1850 /*
1765 1851 * Map different interrupts to MSI-X vectors.
1766 1852 */
1767 1853 static boolean_t
1768 1854 i40e_map_intrs_to_vectors(i40e_t *i40e)
1769 1855 {
1770 - int i;
1771 -
1772 1856 if (i40e->i40e_intr_type != DDI_INTR_TYPE_MSIX) {
1773 1857 return (B_TRUE);
1774 1858 }
1775 1859
1776 1860 /*
1777 - * Each queue pair is mapped to a single interrupt, so transmit
1778 - * and receive interrupts for a given queue share the same vector.
1779 - * The number of queue pairs is one less than the number of interrupt
1780 - * vectors and is assigned the vector one higher than its index.
1781 - * Vector zero is reserved for the admin queue.
1861 + * Each queue pair is mapped to a single interrupt, so
1862 + * transmit and receive interrupts for a given queue share the
1863 + * same vector. Vector zero is reserved for the admin queue.
1782 1864 */
1783 - ASSERT(i40e->i40e_intr_count == i40e->i40e_num_trqpairs + 1);
1865 + for (uint_t i = 0; i < i40e->i40e_num_trqpairs; i++) {
1866 + uint_t vector = i % (i40e->i40e_intr_count - 1);
1784 1867
1785 - for (i = 0; i < i40e->i40e_num_trqpairs; i++) {
1786 - i40e->i40e_trqpairs[i].itrq_rx_intrvec = i + 1;
1787 - i40e->i40e_trqpairs[i].itrq_tx_intrvec = i + 1;
1868 + i40e->i40e_trqpairs[i].itrq_rx_intrvec = vector + 1;
1869 + i40e->i40e_trqpairs[i].itrq_tx_intrvec = vector + 1;
1788 1870 }
1789 1871
1790 1872 return (B_TRUE);
1791 1873 }
1792 1874
1793 1875 static boolean_t
1794 1876 i40e_add_intr_handlers(i40e_t *i40e)
1795 1877 {
1796 1878 int rc, vector;
1797 1879
1798 1880 switch (i40e->i40e_intr_type) {
1799 1881 case DDI_INTR_TYPE_MSIX:
1800 1882 for (vector = 0; vector < i40e->i40e_intr_count; vector++) {
1801 1883 rc = ddi_intr_add_handler(
1802 1884 i40e->i40e_intr_handles[vector],
1803 1885 (ddi_intr_handler_t *)i40e_intr_msix, i40e,
1804 1886 (void *)(uintptr_t)vector);
1805 1887 if (rc != DDI_SUCCESS) {
1806 1888 i40e_log(i40e, "Add interrupt handler (MSI-X) "
1807 1889 "failed: return %d, vector %d", rc, vector);
1808 1890 for (vector--; vector >= 0; vector--) {
1809 1891 (void) ddi_intr_remove_handler(
1810 1892 i40e->i40e_intr_handles[vector]);
1811 1893 }
1812 1894 return (B_FALSE);
1813 1895 }
1814 1896 }
1815 1897 break;
1816 1898 case DDI_INTR_TYPE_MSI:
1817 1899 rc = ddi_intr_add_handler(i40e->i40e_intr_handles[0],
1818 1900 (ddi_intr_handler_t *)i40e_intr_msi, i40e, NULL);
1819 1901 if (rc != DDI_SUCCESS) {
1820 1902 i40e_log(i40e, "Add interrupt handler (MSI) failed: "
1821 1903 "return %d", rc);
1822 1904 return (B_FALSE);
1823 1905 }
1824 1906 break;
1825 1907 case DDI_INTR_TYPE_FIXED:
1826 1908 rc = ddi_intr_add_handler(i40e->i40e_intr_handles[0],
1827 1909 (ddi_intr_handler_t *)i40e_intr_legacy, i40e, NULL);
1828 1910 if (rc != DDI_SUCCESS) {
1829 1911 i40e_log(i40e, "Add interrupt handler (legacy) failed:"
1830 1912 " return %d", rc);
1831 1913 return (B_FALSE);
1832 1914 }
1833 1915 break;
1834 1916 default:
1835 1917 /* Cast to pacify lint */
1836 1918 panic("i40e_intr_type %p contains an unknown type: %d",
1837 1919 (void *)i40e, i40e->i40e_intr_type);
1838 1920 }
1839 1921
1840 1922 return (B_TRUE);
1841 1923 }
1842 1924
1843 1925 /*
1844 1926 * Perform periodic checks. Longer term, we should be thinking about additional
1845 1927 * things here:
1846 1928 *
1847 1929 * o Stall Detection
1848 1930 * o Temperature sensor detection
1849 1931 * o Device resetting
1850 1932 * o Statistics updating to avoid wraparound
1851 1933 */
1852 1934 static void
1853 1935 i40e_timer(void *arg)
1854 1936 {
1855 1937 i40e_t *i40e = arg;
1856 1938
1857 1939 mutex_enter(&i40e->i40e_general_lock);
1858 1940 i40e_link_check(i40e);
1859 1941 mutex_exit(&i40e->i40e_general_lock);
1860 1942 }
1861 1943
1862 1944 /*
1863 1945 * Get the hardware state, and scribble away anything that needs scribbling.
1864 1946 */
1865 1947 static void
1866 1948 i40e_get_hw_state(i40e_t *i40e, i40e_hw_t *hw)
1867 1949 {
1868 1950 int rc;
1869 1951
1870 1952 ASSERT(MUTEX_HELD(&i40e->i40e_general_lock));
1871 1953
1872 1954 (void) i40e_aq_get_link_info(hw, TRUE, NULL, NULL);
1873 1955 i40e_link_check(i40e);
1874 1956
1875 1957 /*
1876 1958 * Try and determine our PHY. Note that we may have to retry to and
1877 1959 * delay to detect fiber correctly.
1878 1960 */
1879 1961 rc = i40e_aq_get_phy_capabilities(hw, B_FALSE, B_TRUE, &i40e->i40e_phy,
1880 1962 NULL);
1881 1963 if (rc == I40E_ERR_UNKNOWN_PHY) {
1882 1964 i40e_msec_delay(200);
1883 1965 rc = i40e_aq_get_phy_capabilities(hw, B_FALSE, B_TRUE,
1884 1966 &i40e->i40e_phy, NULL);
1885 1967 }
1886 1968
1887 1969 if (rc != I40E_SUCCESS) {
1888 1970 if (rc == I40E_ERR_UNKNOWN_PHY) {
1889 1971 i40e_error(i40e, "encountered unknown PHY type, "
1890 1972 "not attaching.");
1891 1973 } else {
1892 1974 i40e_error(i40e, "error getting physical capabilities: "
1893 1975 "%d, %d", rc, hw->aq.asq_last_status);
1894 1976 }
1895 1977 }
1896 1978
1897 1979 rc = i40e_update_link_info(hw);
1898 1980 if (rc != I40E_SUCCESS) {
1899 1981 i40e_error(i40e, "failed to update link information: %d", rc);
1900 1982 }
1901 1983
1902 1984 /*
1903 1985 * In general, we don't want to mask off (as in stop from being a cause)
1904 1986 * any of the interrupts that the phy might be able to generate.
1905 1987 */
1906 1988 rc = i40e_aq_set_phy_int_mask(hw, 0, NULL);
1907 1989 if (rc != I40E_SUCCESS) {
1908 1990 i40e_error(i40e, "failed to update phy link mask: %d", rc);
1909 1991 }
1910 1992 }
1911 1993
1912 1994 /*
1913 1995 * Go through and re-initialize any existing filters that we may have set up for
1914 1996 * this device. Note that we would only expect them to exist if hardware had
1915 1997 * already been initialized and we had just reset it. While we're not
|
↓ open down ↓ |
118 lines elided |
↑ open up ↑ |
1916 1998 * implementing this yet, we're keeping this around for when we add reset
1917 1999 * capabilities, so this isn't forgotten.
1918 2000 */
1919 2001 /* ARGSUSED */
1920 2002 static void
1921 2003 i40e_init_macaddrs(i40e_t *i40e, i40e_hw_t *hw)
1922 2004 {
1923 2005 }
1924 2006
1925 2007 /*
1926 - * Configure the hardware for the Virtual Station Interface (VSI). Currently
1927 - * we only support one, but in the future we could instantiate more than one
1928 - * per attach-point.
2008 + * Set the properties which have common values across all the VSIs.
2009 + * Consult the "Add VSI" command section (7.4.9.5.5.1) for a
2010 + * complete description of these properties.
1929 2011 */
1930 -static boolean_t
1931 -i40e_config_vsi(i40e_t *i40e, i40e_hw_t *hw)
2012 +static void
2013 +i40e_set_shared_vsi_props(i40e_t *i40e,
2014 + struct i40e_aqc_vsi_properties_data *info, uint_t vsi_idx)
1932 2015 {
1933 - struct i40e_vsi_context context;
1934 - int err, tc_queues;
2016 + uint_t tc_queues;
2017 + uint16_t vsi_qp_base;
1935 2018
1936 - bzero(&context, sizeof (struct i40e_vsi_context));
1937 - context.seid = i40e->i40e_vsi_id;
1938 - context.pf_num = hw->pf_id;
1939 - err = i40e_aq_get_vsi_params(hw, &context, NULL);
1940 - if (err != I40E_SUCCESS) {
1941 - i40e_error(i40e, "get VSI params failed with %d", err);
1942 - return (B_FALSE);
1943 - }
2019 + /*
2020 + * It's important that we use bitwise-OR here; callers to this
2021 + * function might enable other sections before calling this
2022 + * function.
2023 + */
2024 + info->valid_sections |= LE_16(I40E_AQ_VSI_PROP_QUEUE_MAP_VALID |
2025 + I40E_AQ_VSI_PROP_VLAN_VALID);
1944 2026
1945 - i40e->i40e_vsi_num = context.vsi_number;
1946 -
1947 2027 /*
1948 - * Set the queue and traffic class bits. Keep it simple for now.
2028 + * Calculate the starting QP index for this VSI. This base is
2029 + * relative to the PF queue space; so a value of 0 for PF#1
2030 + * represents the absolute index PFLAN_QALLOC_FIRSTQ for PF#1.
1949 2031 */
1950 - context.info.valid_sections = I40E_AQ_VSI_PROP_QUEUE_MAP_VALID;
1951 - context.info.mapping_flags = I40E_AQ_VSI_QUE_MAP_CONTIG;
1952 - context.info.queue_mapping[0] = I40E_ASSIGN_ALL_QUEUES;
2032 + vsi_qp_base = vsi_idx * i40e->i40e_num_trqpairs_per_vsi;
2033 + info->mapping_flags = LE_16(I40E_AQ_VSI_QUE_MAP_CONTIG);
2034 + info->queue_mapping[0] =
2035 + LE_16((vsi_qp_base << I40E_AQ_VSI_QUEUE_SHIFT) &
2036 + I40E_AQ_VSI_QUEUE_MASK);
1953 2037
1954 2038 /*
1955 - * tc_queues determines the size of the traffic class, where the size is
1956 - * 2^^tc_queues to a maximum of 64 for the X710 and 128 for the X722.
2039 + * tc_queues determines the size of the traffic class, where
2040 + * the size is 2^^tc_queues to a maximum of 64 for the X710
2041 + * and 128 for the X722.
1957 2042 *
1958 2043 * Some examples:
1959 - * i40e_num_trqpairs == 1 => tc_queues = 0, 2^^0 = 1.
1960 - * i40e_num_trqpairs == 7 => tc_queues = 3, 2^^3 = 8.
1961 - * i40e_num_trqpairs == 8 => tc_queues = 3, 2^^3 = 8.
1962 - * i40e_num_trqpairs == 9 => tc_queues = 4, 2^^4 = 16.
1963 - * i40e_num_trqpairs == 17 => tc_queues = 5, 2^^5 = 32.
1964 - * i40e_num_trqpairs == 64 => tc_queues = 6, 2^^6 = 64.
2044 + * i40e_num_trqpairs_per_vsi == 1 => tc_queues = 0, 2^^0 = 1.
2045 + * i40e_num_trqpairs_per_vsi == 7 => tc_queues = 3, 2^^3 = 8.
2046 + * i40e_num_trqpairs_per_vsi == 8 => tc_queues = 3, 2^^3 = 8.
2047 + * i40e_num_trqpairs_per_vsi == 9 => tc_queues = 4, 2^^4 = 16.
2048 + * i40e_num_trqpairs_per_vsi == 17 => tc_queues = 5, 2^^5 = 32.
2049 + * i40e_num_trqpairs_per_vsi == 64 => tc_queues = 6, 2^^6 = 64.
1965 2050 */
1966 - tc_queues = ddi_fls(i40e->i40e_num_trqpairs - 1);
2051 + tc_queues = ddi_fls(i40e->i40e_num_trqpairs_per_vsi - 1);
1967 2052
1968 - context.info.tc_mapping[0] = ((0 << I40E_AQ_VSI_TC_QUE_OFFSET_SHIFT) &
1969 - I40E_AQ_VSI_TC_QUE_OFFSET_MASK) |
1970 - ((tc_queues << I40E_AQ_VSI_TC_QUE_NUMBER_SHIFT) &
1971 - I40E_AQ_VSI_TC_QUE_NUMBER_MASK);
2053 + /*
2054 + * The TC queue mapping is in relation to the VSI queue space.
2055 + * Since we are only using one traffic class (TC0) we always
2056 + * start at queue offset 0.
2057 + */
2058 + info->tc_mapping[0] =
2059 + LE_16(((0 << I40E_AQ_VSI_TC_QUE_OFFSET_SHIFT) &
2060 + I40E_AQ_VSI_TC_QUE_OFFSET_MASK) |
2061 + ((tc_queues << I40E_AQ_VSI_TC_QUE_NUMBER_SHIFT) &
2062 + I40E_AQ_VSI_TC_QUE_NUMBER_MASK));
1972 2063
1973 - context.info.valid_sections |= I40E_AQ_VSI_PROP_VLAN_VALID;
1974 - context.info.port_vlan_flags = I40E_AQ_VSI_PVLAN_MODE_ALL |
2064 + /*
2065 + * I40E_AQ_VSI_PVLAN_MODE_ALL ("VLAN driver insertion mode")
2066 + *
2067 + * Allow tagged and untagged packets to be sent to this
2068 + * VSI from the host.
2069 + *
2070 + * I40E_AQ_VSI_PVLAN_EMOD_NOTHING ("VLAN and UP expose mode")
2071 + *
2072 + * Leave the tag on the frame and place no VLAN
2073 + * information in the descriptor. We want this mode
2074 + * because our MAC layer will take care of the VLAN tag,
2075 + * if there is one.
2076 + */
2077 + info->port_vlan_flags = I40E_AQ_VSI_PVLAN_MODE_ALL |
1975 2078 I40E_AQ_VSI_PVLAN_EMOD_NOTHING;
2079 +}
1976 2080
1977 - context.flags = LE16_TO_CPU(I40E_AQ_VSI_TYPE_PF);
2081 +/*
2082 + * Delete the VSI at this index, if one exists. We assume there is no
2083 + * action we can take if this command fails but to log the failure.
2084 + */
2085 +static void
2086 +i40e_delete_vsi(i40e_t *i40e, uint_t idx)
2087 +{
2088 + i40e_hw_t *hw = &i40e->i40e_hw_space;
2089 + uint16_t seid = i40e->i40e_vsis[idx].iv_seid;
1978 2090
1979 - i40e->i40e_vsi_stat_id = LE16_TO_CPU(context.info.stat_counter_idx);
1980 - if (i40e_stat_vsi_init(i40e) == B_FALSE)
1981 - return (B_FALSE);
2091 + if (seid != 0) {
2092 + int rc;
1982 2093
1983 - err = i40e_aq_update_vsi_params(hw, &context, NULL);
1984 - if (err != I40E_SUCCESS) {
1985 - i40e_error(i40e, "Update VSI params failed with %d", err);
2094 + rc = i40e_aq_delete_element(hw, seid, NULL);
2095 +
2096 + if (rc != I40E_SUCCESS) {
2097 + i40e_error(i40e, "Failed to delete VSI %d: %d",
2098 + rc, hw->aq.asq_last_status);
2099 + }
2100 +
2101 + i40e->i40e_vsis[idx].iv_seid = 0;
2102 + }
2103 +}
2104 +
2105 +/*
2106 + * Add a new VSI.
2107 + */
2108 +static boolean_t
2109 +i40e_add_vsi(i40e_t *i40e, i40e_hw_t *hw, uint_t idx)
2110 +{
2111 + struct i40e_vsi_context ctx;
2112 + i40e_rx_group_t *rxg;
2113 + int rc;
2114 +
2115 + /*
2116 + * The default VSI is created by the controller. This function
2117 + * creates new, non-defualt VSIs only.
2118 + */
2119 + ASSERT3U(idx, !=, 0);
2120 +
2121 + bzero(&ctx, sizeof (struct i40e_vsi_context));
2122 + ctx.uplink_seid = i40e->i40e_veb_seid;
2123 + ctx.pf_num = hw->pf_id;
2124 + ctx.flags = I40E_AQ_VSI_TYPE_PF;
2125 + ctx.connection_type = I40E_AQ_VSI_CONN_TYPE_NORMAL;
2126 + i40e_set_shared_vsi_props(i40e, &ctx.info, idx);
2127 +
2128 + rc = i40e_aq_add_vsi(hw, &ctx, NULL);
2129 + if (rc != I40E_SUCCESS) {
2130 + i40e_error(i40e, "i40e_aq_add_vsi() failed %d: %d", rc,
2131 + hw->aq.asq_last_status);
1986 2132 return (B_FALSE);
1987 2133 }
1988 2134
2135 + rxg = &i40e->i40e_rx_groups[idx];
2136 + rxg->irg_vsi_seid = ctx.seid;
2137 + i40e->i40e_vsis[idx].iv_number = ctx.vsi_number;
2138 + i40e->i40e_vsis[idx].iv_seid = ctx.seid;
2139 + i40e->i40e_vsis[idx].iv_stats_id = LE_16(ctx.info.stat_counter_idx);
1989 2140
2141 + if (i40e_stat_vsi_init(i40e, idx) == B_FALSE)
2142 + return (B_FALSE);
2143 +
1990 2144 return (B_TRUE);
1991 2145 }
1992 2146
1993 2147 /*
1994 - * Configure the RSS key. For the X710 controller family, this is set on a
1995 - * per-PF basis via registers. For the X722, this is done on a per-VSI basis
1996 - * through the admin queue.
2148 + * Configure the hardware for the Default Virtual Station Interface (VSI).
1997 2149 */
1998 2150 static boolean_t
1999 -i40e_config_rss_key(i40e_t *i40e, i40e_hw_t *hw)
2151 +i40e_config_def_vsi(i40e_t *i40e, i40e_hw_t *hw)
2000 2152 {
2001 - uint32_t seed[I40E_PFQF_HKEY_MAX_INDEX + 1];
2153 + struct i40e_vsi_context ctx;
2154 + i40e_rx_group_t *def_rxg;
2155 + int err;
2156 + struct i40e_aqc_remove_macvlan_element_data filt;
2002 2157
2003 - (void) random_get_pseudo_bytes((uint8_t *)seed, sizeof (seed));
2158 + bzero(&ctx, sizeof (struct i40e_vsi_context));
2159 + ctx.seid = I40E_DEF_VSI_SEID(i40e);
2160 + ctx.pf_num = hw->pf_id;
2161 + err = i40e_aq_get_vsi_params(hw, &ctx, NULL);
2162 + if (err != I40E_SUCCESS) {
2163 + i40e_error(i40e, "get VSI params failed with %d", err);
2164 + return (B_FALSE);
2165 + }
2004 2166
2005 - if (i40e_is_x722(i40e)) {
2167 + ctx.info.valid_sections = 0;
2168 + i40e->i40e_vsis[0].iv_number = ctx.vsi_number;
2169 + i40e->i40e_vsis[0].iv_stats_id = LE_16(ctx.info.stat_counter_idx);
2170 + if (i40e_stat_vsi_init(i40e, 0) == B_FALSE)
2171 + return (B_FALSE);
2172 +
2173 + i40e_set_shared_vsi_props(i40e, &ctx.info, I40E_DEF_VSI_IDX);
2174 +
2175 + err = i40e_aq_update_vsi_params(hw, &ctx, NULL);
2176 + if (err != I40E_SUCCESS) {
2177 + i40e_error(i40e, "Update VSI params failed with %d", err);
2178 + return (B_FALSE);
2179 + }
2180 +
2181 + def_rxg = &i40e->i40e_rx_groups[0];
2182 + def_rxg->irg_vsi_seid = I40E_DEF_VSI_SEID(i40e);
2183 +
2184 + /*
2185 + * We have seen three different behaviors in regards to the
2186 + * Default VSI and its implicit L2 MAC+VLAN filter.
2187 + *
2188 + * 1. It has an implicit filter for the factory MAC address
2189 + * and this filter counts against 'ifr_nmacfilt_used'.
2190 + *
2191 + * 2. It has an implicit filter for the factory MAC address
2192 + * and this filter DOES NOT count against 'ifr_nmacfilt_used'.
2193 + *
2194 + * 3. It DOES NOT have an implicit filter.
2195 + *
2196 + * All three of these cases are accounted for below. If we
2197 + * fail to remove the L2 filter (ENOENT) then we assume there
2198 + * wasn't one. Otherwise, if we successfully remove the
2199 + * filter, we make sure to update the 'ifr_nmacfilt_used'
2200 + * count accordingly.
2201 + *
2202 + * We remove this filter to prevent duplicate delivery of
2203 + * packets destined for the primary MAC address as DLS will
2204 + * create the same filter on a non-default VSI for the primary
2205 + * MAC client.
2206 + *
2207 + * If you change the following code please test it across as
2208 + * many X700 series controllers and firmware revisions as you
2209 + * can.
2210 + */
2211 + bzero(&filt, sizeof (filt));
2212 + bcopy(hw->mac.port_addr, filt.mac_addr, ETHERADDRL);
2213 + filt.flags = I40E_AQC_MACVLAN_DEL_PERFECT_MATCH;
2214 + filt.vlan_tag = 0;
2215 +
2216 + ASSERT3U(i40e->i40e_resources.ifr_nmacfilt_used, <=, 1);
2217 + i40e_log(i40e, "Num L2 filters: %u",
2218 + i40e->i40e_resources.ifr_nmacfilt_used);
2219 +
2220 + err = i40e_aq_remove_macvlan(hw, I40E_DEF_VSI_SEID(i40e), &filt, 1,
2221 + NULL);
2222 + if (err == I40E_SUCCESS) {
2223 + i40e_log(i40e,
2224 + "Removed L2 filter from Default VSI with SEID %u",
2225 + I40E_DEF_VSI_SEID(i40e));
2226 + } else if (hw->aq.asq_last_status == ENOENT) {
2227 + i40e_log(i40e,
2228 + "No L2 filter for Default VSI with SEID %u",
2229 + I40E_DEF_VSI_SEID(i40e));
2230 + } else {
2231 + i40e_error(i40e, "Failed to remove L2 filter from"
2232 + " Default VSI with SEID %u: %d (%d)",
2233 + I40E_DEF_VSI_SEID(i40e), err, hw->aq.asq_last_status);
2234 +
2235 + return (B_FALSE);
2236 + }
2237 +
2238 + /*
2239 + * As mentioned above, the controller created an implicit L2
2240 + * filter for the primary MAC. We want to remove both the
2241 + * filter and decrement the filter count. However, not all
2242 + * controllers count this implicit filter against the total
2243 + * MAC filter count. So here we are making sure it is either
2244 + * one or zero. If it is one, then we know it is for the
2245 + * implicit filter and we should decrement since we just
2246 + * removed the filter above. If it is zero then we know the
2247 + * controller that does not count the implicit filter, and it
2248 + * was enough to just remove it; we leave the count alone.
2249 + * But if it is neither, then we have never seen a controller
2250 + * like this before and we should fail to attach.
2251 + *
2252 + * It is unfortunate that this code must exist but the
2253 + * behavior of this implicit L2 filter and its corresponding
2254 + * count were dicovered through empirical testing. The
2255 + * programming manuals hint at this filter but do not
2256 + * explicitly call out the exact behavior.
2257 + */
2258 + if (i40e->i40e_resources.ifr_nmacfilt_used == 1) {
2259 + i40e->i40e_resources.ifr_nmacfilt_used--;
2260 + } else {
2261 + if (i40e->i40e_resources.ifr_nmacfilt_used != 0) {
2262 + i40e_error(i40e, "Unexpected L2 filter count: %u"
2263 + " (expected 0)",
2264 + i40e->i40e_resources.ifr_nmacfilt_used);
2265 + return (B_FALSE);
2266 + }
2267 + }
2268 +
2269 + return (B_TRUE);
2270 +}
2271 +
2272 +static boolean_t
2273 +i40e_config_rss_key_x722(i40e_t *i40e, i40e_hw_t *hw)
2274 +{
2275 + for (uint_t i = 0; i < i40e->i40e_num_rx_groups; i++) {
2276 + uint32_t seed[I40E_PFQF_HKEY_MAX_INDEX + 1];
2006 2277 struct i40e_aqc_get_set_rss_key_data key;
2007 - const char *u8seed = (char *)seed;
2278 + const char *u8seed;
2008 2279 enum i40e_status_code status;
2280 + uint16_t vsi_number = i40e->i40e_vsis[i].iv_number;
2009 2281
2282 + (void) random_get_pseudo_bytes((uint8_t *)seed, sizeof (seed));
2283 + u8seed = (char *)seed;
2284 +
2010 2285 CTASSERT(sizeof (key) >= (sizeof (key.standard_rss_key) +
2011 2286 sizeof (key.extended_hash_key)));
2012 2287
2013 2288 bcopy(u8seed, key.standard_rss_key,
2014 2289 sizeof (key.standard_rss_key));
2015 2290 bcopy(&u8seed[sizeof (key.standard_rss_key)],
2016 2291 key.extended_hash_key, sizeof (key.extended_hash_key));
2017 2292
2018 - status = i40e_aq_set_rss_key(hw, i40e->i40e_vsi_num, &key);
2293 + ASSERT3U(vsi_number, !=, 0);
2294 + status = i40e_aq_set_rss_key(hw, vsi_number, &key);
2295 +
2019 2296 if (status != I40E_SUCCESS) {
2020 - i40e_error(i40e, "failed to set rss key: %d", status);
2297 + i40e_error(i40e, "failed to set RSS key for VSI %u: %d",
2298 + vsi_number, status);
2021 2299 return (B_FALSE);
2022 2300 }
2301 + }
2302 +
2303 + return (B_TRUE);
2304 +}
2305 +
2306 +/*
2307 + * Configure the RSS key. For the X710 controller family, this is set on a
2308 + * per-PF basis via registers. For the X722, this is done on a per-VSI basis
2309 + * through the admin queue.
2310 + */
2311 +static boolean_t
2312 +i40e_config_rss_key(i40e_t *i40e, i40e_hw_t *hw)
2313 +{
2314 + if (i40e_is_x722(i40e)) {
2315 + if (!i40e_config_rss_key_x722(i40e, hw))
2316 + return (B_FALSE);
2023 2317 } else {
2024 - uint_t i;
2025 - for (i = 0; i <= I40E_PFQF_HKEY_MAX_INDEX; i++)
2318 + uint32_t seed[I40E_PFQF_HKEY_MAX_INDEX + 1];
2319 +
2320 + (void) random_get_pseudo_bytes((uint8_t *)seed, sizeof (seed));
2321 + for (uint_t i = 0; i <= I40E_PFQF_HKEY_MAX_INDEX; i++)
2026 2322 i40e_write_rx_ctl(hw, I40E_PFQF_HKEY(i), seed[i]);
2027 2323 }
2028 2324
2029 2325 return (B_TRUE);
2030 2326 }
2031 2327
2032 2328 /*
2033 2329 * Populate the LUT. The size of each entry in the LUT depends on the controller
2034 2330 * family, with the X722 using a known 7-bit width. On the X710 controller, this
2035 2331 * is programmed through its control registers where as on the X722 this is
2036 2332 * configured through the admin queue. Also of note, the X722 allows the LUT to
2037 - * be set on a per-PF or VSI basis. At this time, as we only have a single VSI,
2038 - * we use the PF setting as it is the primary VSI.
2333 + * be set on a per-PF or VSI basis. At this time we use the PF setting. If we
2334 + * decide to use the per-VSI LUT in the future, then we will need to modify the
2335 + * i40e_add_vsi() function to set the RSS LUT bits in the queueing section.
2039 2336 *
2040 2337 * We populate the LUT in a round robin fashion with the rx queue indices from 0
2041 - * to i40e_num_trqpairs - 1.
2338 + * to i40e_num_trqpairs_per_vsi - 1.
2042 2339 */
2043 2340 static boolean_t
2044 2341 i40e_config_rss_hlut(i40e_t *i40e, i40e_hw_t *hw)
2045 2342 {
2046 2343 uint32_t *hlut;
2047 2344 uint8_t lut_mask;
2048 2345 uint_t i;
2049 2346 boolean_t ret = B_FALSE;
2050 2347
2051 2348 /*
2052 2349 * We always configure the PF with a table size of 512 bytes in
2053 2350 * i40e_chip_start().
2054 2351 */
2055 2352 hlut = kmem_alloc(I40E_HLUT_TABLE_SIZE, KM_NOSLEEP);
2056 2353 if (hlut == NULL) {
2057 2354 i40e_error(i40e, "i40e_config_rss() buffer allocation failed");
2058 2355 return (B_FALSE);
2059 2356 }
2060 2357
|
↓ open down ↓ |
9 lines elided |
↑ open up ↑ |
2061 2358 /*
2062 2359 * The width of the X722 is apparently defined to be 7 bits, regardless
2063 2360 * of the capability.
2064 2361 */
2065 2362 if (i40e_is_x722(i40e)) {
2066 2363 lut_mask = (1 << 7) - 1;
2067 2364 } else {
2068 2365 lut_mask = (1 << hw->func_caps.rss_table_entry_width) - 1;
2069 2366 }
2070 2367
2071 - for (i = 0; i < I40E_HLUT_TABLE_SIZE; i++)
2072 - ((uint8_t *)hlut)[i] = (i % i40e->i40e_num_trqpairs) & lut_mask;
2368 + for (i = 0; i < I40E_HLUT_TABLE_SIZE; i++) {
2369 + ((uint8_t *)hlut)[i] =
2370 + (i % i40e->i40e_num_trqpairs_per_vsi) & lut_mask;
2371 + }
2073 2372
2074 2373 if (i40e_is_x722(i40e)) {
2075 2374 enum i40e_status_code status;
2076 - status = i40e_aq_set_rss_lut(hw, i40e->i40e_vsi_num, B_TRUE,
2077 - (uint8_t *)hlut, I40E_HLUT_TABLE_SIZE);
2375 +
2376 + status = i40e_aq_set_rss_lut(hw, 0, B_TRUE, (uint8_t *)hlut,
2377 + I40E_HLUT_TABLE_SIZE);
2378 +
2078 2379 if (status != I40E_SUCCESS) {
2079 - i40e_error(i40e, "failed to set RSS LUT: %d", status);
2380 + i40e_error(i40e, "failed to set RSS LUT %d: %d",
2381 + status, hw->aq.asq_last_status);
2080 2382 goto out;
2081 2383 }
2082 2384 } else {
2083 2385 for (i = 0; i < I40E_HLUT_TABLE_SIZE >> 2; i++) {
2084 2386 I40E_WRITE_REG(hw, I40E_PFQF_HLUT(i), hlut[i]);
2085 2387 }
2086 2388 }
2087 2389 ret = B_TRUE;
2088 2390 out:
2089 2391 kmem_free(hlut, I40E_HLUT_TABLE_SIZE);
2090 2392 return (ret);
2091 2393 }
2092 2394
2093 2395 /*
2094 2396 * Set up RSS.
2095 2397 * 1. Seed the hash key.
2096 2398 * 2. Enable PCTYPEs for the hash filter.
2097 2399 * 3. Populate the LUT.
2098 2400 */
2099 2401 static boolean_t
2100 2402 i40e_config_rss(i40e_t *i40e, i40e_hw_t *hw)
2101 2403 {
2102 2404 uint64_t hena;
2103 2405
2104 2406 /*
2105 2407 * 1. Seed the hash key
2106 2408 */
2107 2409 if (!i40e_config_rss_key(i40e, hw))
2108 2410 return (B_FALSE);
2109 2411
2110 2412 /*
2111 2413 * 2. Configure PCTYPES
2112 2414 */
2113 2415 hena = (1ULL << I40E_FILTER_PCTYPE_NONF_IPV4_OTHER) |
2114 2416 (1ULL << I40E_FILTER_PCTYPE_NONF_IPV4_TCP) |
2115 2417 (1ULL << I40E_FILTER_PCTYPE_NONF_IPV4_SCTP) |
2116 2418 (1ULL << I40E_FILTER_PCTYPE_NONF_IPV4_UDP) |
2117 2419 (1ULL << I40E_FILTER_PCTYPE_FRAG_IPV4) |
2118 2420 (1ULL << I40E_FILTER_PCTYPE_NONF_IPV6_OTHER) |
2119 2421 (1ULL << I40E_FILTER_PCTYPE_NONF_IPV6_TCP) |
2120 2422 (1ULL << I40E_FILTER_PCTYPE_NONF_IPV6_SCTP) |
2121 2423 (1ULL << I40E_FILTER_PCTYPE_NONF_IPV6_UDP) |
2122 2424 (1ULL << I40E_FILTER_PCTYPE_FRAG_IPV6) |
2123 2425 (1ULL << I40E_FILTER_PCTYPE_L2_PAYLOAD);
2124 2426
2125 2427 /*
2126 2428 * Add additional types supported by the X722 controller.
2127 2429 */
2128 2430 if (i40e_is_x722(i40e)) {
2129 2431 hena |= (1ULL << I40E_FILTER_PCTYPE_NONF_UNICAST_IPV4_UDP) |
2130 2432 (1ULL << I40E_FILTER_PCTYPE_NONF_MULTICAST_IPV4_UDP) |
2131 2433 (1ULL << I40E_FILTER_PCTYPE_NONF_IPV4_TCP_SYN_NO_ACK) |
2132 2434 (1ULL << I40E_FILTER_PCTYPE_NONF_UNICAST_IPV6_UDP) |
2133 2435 (1ULL << I40E_FILTER_PCTYPE_NONF_MULTICAST_IPV6_UDP) |
2134 2436 (1ULL << I40E_FILTER_PCTYPE_NONF_IPV6_TCP_SYN_NO_ACK);
2135 2437 }
2136 2438
2137 2439 i40e_write_rx_ctl(hw, I40E_PFQF_HENA(0), (uint32_t)hena);
2138 2440 i40e_write_rx_ctl(hw, I40E_PFQF_HENA(1), (uint32_t)(hena >> 32));
2139 2441
2140 2442 /*
2141 2443 * 3. Populate LUT
2142 2444 */
2143 2445 return (i40e_config_rss_hlut(i40e, hw));
2144 2446 }
|
↓ open down ↓ |
55 lines elided |
↑ open up ↑ |
2145 2447
2146 2448 /*
2147 2449 * Wrapper to kick the chipset on.
2148 2450 */
2149 2451 static boolean_t
2150 2452 i40e_chip_start(i40e_t *i40e)
2151 2453 {
2152 2454 i40e_hw_t *hw = &i40e->i40e_hw_space;
2153 2455 struct i40e_filter_control_settings filter;
2154 2456 int rc;
2457 + uint8_t err;
2155 2458
2156 2459 if (((hw->aq.fw_maj_ver == 4) && (hw->aq.fw_min_ver < 33)) ||
2157 2460 (hw->aq.fw_maj_ver < 4)) {
2158 2461 i40e_msec_delay(75);
2159 2462 if (i40e_aq_set_link_restart_an(hw, TRUE, NULL) !=
2160 2463 I40E_SUCCESS) {
2161 2464 i40e_error(i40e, "failed to restart link: admin queue "
2162 2465 "error: %d", hw->aq.asq_last_status);
2163 2466 return (B_FALSE);
2164 2467 }
2165 2468 }
2166 2469
2167 2470 /* Determine hardware state */
2168 2471 i40e_get_hw_state(i40e, hw);
2169 2472
2473 + /* For now, we always disable Ethernet Flow Control. */
2474 + hw->fc.requested_mode = I40E_FC_NONE;
2475 + rc = i40e_set_fc(hw, &err, B_TRUE);
2476 + if (rc != I40E_SUCCESS) {
2477 + i40e_error(i40e, "Setting flow control failed, returned %d"
2478 + " with error: 0x%x", rc, err);
2479 + return (B_FALSE);
2480 + }
2481 +
2170 2482 /* Initialize mac addresses. */
2171 2483 i40e_init_macaddrs(i40e, hw);
2172 2484
2173 2485 /*
2174 2486 * Set up the filter control. If the hash lut size is changed from
2175 2487 * I40E_HASH_LUT_SIZE_512 then I40E_HLUT_TABLE_SIZE and
2176 2488 * i40e_config_rss_hlut() will need to be updated.
2177 2489 */
2178 2490 bzero(&filter, sizeof (filter));
2179 2491 filter.enable_ethtype = TRUE;
2180 2492 filter.enable_macvlan = TRUE;
|
↓ open down ↓ |
1 lines elided |
↑ open up ↑ |
2181 2493 filter.hash_lut_size = I40E_HASH_LUT_SIZE_512;
2182 2494
2183 2495 rc = i40e_set_filter_control(hw, &filter);
2184 2496 if (rc != I40E_SUCCESS) {
2185 2497 i40e_error(i40e, "i40e_set_filter_control() returned %d", rc);
2186 2498 return (B_FALSE);
2187 2499 }
2188 2500
2189 2501 i40e_intr_chip_init(i40e);
2190 2502
2191 - if (!i40e_config_vsi(i40e, hw))
2503 + rc = i40e_get_mac_seid(i40e);
2504 + if (rc == -1) {
2505 + i40e_error(i40e, "failed to obtain MAC Uplink SEID");
2192 2506 return (B_FALSE);
2507 + }
2508 + i40e->i40e_mac_seid = (uint16_t)rc;
2193 2509
2510 + /*
2511 + * Create a VEB in order to support multiple VSIs. Each VSI
2512 + * functions as a MAC group. This call sets the PF's MAC as
2513 + * the uplink port and the PF's default VSI as the default
2514 + * downlink port.
2515 + */
2516 + rc = i40e_aq_add_veb(hw, i40e->i40e_mac_seid, I40E_DEF_VSI_SEID(i40e),
2517 + 0x1, B_TRUE, &i40e->i40e_veb_seid, B_FALSE, NULL);
2518 + if (rc != I40E_SUCCESS) {
2519 + i40e_error(i40e, "i40e_aq_add_veb() failed %d: %d", rc,
2520 + hw->aq.asq_last_status);
2521 + return (B_FALSE);
2522 + }
2523 +
2524 + if (!i40e_config_def_vsi(i40e, hw))
2525 + return (B_FALSE);
2526 +
2527 + for (uint_t i = 1; i < i40e->i40e_num_rx_groups; i++) {
2528 + if (!i40e_add_vsi(i40e, hw, i))
2529 + return (B_FALSE);
2530 + }
2531 +
2194 2532 if (!i40e_config_rss(i40e, hw))
2195 2533 return (B_FALSE);
2196 2534
2197 2535 i40e_flush(hw);
2198 2536
2199 2537 return (B_TRUE);
2200 2538 }
2201 2539
2202 2540 /*
2203 2541 * Take care of tearing down the rx ring. See 8.3.3.1.2 for more information.
2204 2542 */
2205 2543 static void
2206 2544 i40e_shutdown_rx_rings(i40e_t *i40e)
2207 2545 {
2208 2546 int i;
2209 2547 uint32_t reg;
2210 2548
2211 2549 i40e_hw_t *hw = &i40e->i40e_hw_space;
2212 2550
2213 2551 /*
2214 2552 * Step 1. The interrupt linked list (see i40e_intr.c for more
2215 2553 * information) should have already been cleared before calling this
2216 2554 * function.
2217 2555 */
2218 2556 #ifdef DEBUG
2219 2557 if (i40e->i40e_intr_type == DDI_INTR_TYPE_MSIX) {
2220 2558 for (i = 1; i < i40e->i40e_intr_count; i++) {
2221 2559 reg = I40E_READ_REG(hw, I40E_PFINT_LNKLSTN(i - 1));
2222 2560 VERIFY3U(reg, ==, I40E_QUEUE_TYPE_EOL);
2223 2561 }
2224 2562 } else {
2225 2563 reg = I40E_READ_REG(hw, I40E_PFINT_LNKLST0);
2226 2564 VERIFY3U(reg, ==, I40E_QUEUE_TYPE_EOL);
2227 2565 }
2228 2566
2229 2567 #endif /* DEBUG */
2230 2568
2231 2569 for (i = 0; i < i40e->i40e_num_trqpairs; i++) {
2232 2570 /*
2233 2571 * Step 1. Request the queue by clearing QENA_REQ. It may not be
2234 2572 * set due to unwinding from failures and a partially enabled
2235 2573 * ring set.
2236 2574 */
2237 2575 reg = I40E_READ_REG(hw, I40E_QRX_ENA(i));
2238 2576 if (!(reg & I40E_QRX_ENA_QENA_REQ_MASK))
2239 2577 continue;
2240 2578 VERIFY((reg & I40E_QRX_ENA_QENA_REQ_MASK) ==
2241 2579 I40E_QRX_ENA_QENA_REQ_MASK);
2242 2580 reg &= ~I40E_QRX_ENA_QENA_REQ_MASK;
2243 2581 I40E_WRITE_REG(hw, I40E_QRX_ENA(i), reg);
2244 2582 }
2245 2583
2246 2584 /*
2247 2585 * Step 2. Wait for the disable to take, by having QENA_STAT in the FPM
2248 2586 * be cleared. Note that we could still receive data in the queue during
2249 2587 * this time. We don't actually wait for this now and instead defer this
2250 2588 * to i40e_shutdown_rings_wait(), after we've interleaved disabling the
2251 2589 * TX queues as well.
2252 2590 */
2253 2591 }
2254 2592
2255 2593 static void
2256 2594 i40e_shutdown_tx_rings(i40e_t *i40e)
2257 2595 {
2258 2596 int i;
2259 2597 uint32_t reg;
2260 2598
2261 2599 i40e_hw_t *hw = &i40e->i40e_hw_space;
2262 2600
2263 2601 /*
2264 2602 * Step 1. The interrupt linked list should already have been cleared.
2265 2603 */
2266 2604 #ifdef DEBUG
2267 2605 if (i40e->i40e_intr_type == DDI_INTR_TYPE_MSIX) {
2268 2606 for (i = 1; i < i40e->i40e_intr_count; i++) {
2269 2607 reg = I40E_READ_REG(hw, I40E_PFINT_LNKLSTN(i - 1));
2270 2608 VERIFY3U(reg, ==, I40E_QUEUE_TYPE_EOL);
2271 2609 }
2272 2610 } else {
2273 2611 reg = I40E_READ_REG(hw, I40E_PFINT_LNKLST0);
2274 2612 VERIFY3U(reg, ==, I40E_QUEUE_TYPE_EOL);
2275 2613
2276 2614 }
2277 2615 #endif /* DEBUG */
2278 2616
2279 2617 for (i = 0; i < i40e->i40e_num_trqpairs; i++) {
2280 2618 /*
2281 2619 * Step 2. Set the SET_QDIS flag for every queue.
2282 2620 */
2283 2621 i40e_pre_tx_queue_cfg(hw, i, B_FALSE);
2284 2622 }
2285 2623
2286 2624 /*
2287 2625 * Step 3. Wait at least 400 usec (can be done once for all queues).
2288 2626 */
2289 2627 drv_usecwait(500);
2290 2628
2291 2629 for (i = 0; i < i40e->i40e_num_trqpairs; i++) {
2292 2630 /*
2293 2631 * Step 4. Clear the QENA_REQ flag which tells hardware to
2294 2632 * quiesce. If QENA_REQ is not already set then that means that
2295 2633 * we likely already tried to disable this queue.
2296 2634 */
2297 2635 reg = I40E_READ_REG(hw, I40E_QTX_ENA(i));
2298 2636 if (!(reg & I40E_QTX_ENA_QENA_REQ_MASK))
2299 2637 continue;
2300 2638 reg &= ~I40E_QTX_ENA_QENA_REQ_MASK;
2301 2639 I40E_WRITE_REG(hw, I40E_QTX_ENA(i), reg);
2302 2640 }
2303 2641
2304 2642 /*
2305 2643 * Step 5. Wait for all drains to finish. This will be done by the
2306 2644 * hardware removing the QENA_STAT flag from the queue. Rather than
2307 2645 * waiting here, we interleave it with all the others in
2308 2646 * i40e_shutdown_rings_wait().
2309 2647 */
2310 2648 }
2311 2649
2312 2650 /*
2313 2651 * Wait for all the rings to be shut down. e.g. Steps 2 and 5 from the above
2314 2652 * functions.
2315 2653 */
2316 2654 static boolean_t
2317 2655 i40e_shutdown_rings_wait(i40e_t *i40e)
2318 2656 {
2319 2657 int i, try;
2320 2658 i40e_hw_t *hw = &i40e->i40e_hw_space;
2321 2659
2322 2660 for (i = 0; i < i40e->i40e_num_trqpairs; i++) {
2323 2661 uint32_t reg;
2324 2662
2325 2663 for (try = 0; try < I40E_RING_WAIT_NTRIES; try++) {
2326 2664 reg = I40E_READ_REG(hw, I40E_QRX_ENA(i));
2327 2665 if ((reg & I40E_QRX_ENA_QENA_STAT_MASK) == 0)
2328 2666 break;
2329 2667 i40e_msec_delay(I40E_RING_WAIT_PAUSE);
2330 2668 }
2331 2669
2332 2670 if ((reg & I40E_QRX_ENA_QENA_STAT_MASK) != 0) {
2333 2671 i40e_error(i40e, "timed out disabling rx queue %d",
2334 2672 i);
2335 2673 return (B_FALSE);
2336 2674 }
2337 2675
2338 2676 for (try = 0; try < I40E_RING_WAIT_NTRIES; try++) {
2339 2677 reg = I40E_READ_REG(hw, I40E_QTX_ENA(i));
2340 2678 if ((reg & I40E_QTX_ENA_QENA_STAT_MASK) == 0)
2341 2679 break;
2342 2680 i40e_msec_delay(I40E_RING_WAIT_PAUSE);
2343 2681 }
2344 2682
2345 2683 if ((reg & I40E_QTX_ENA_QENA_STAT_MASK) != 0) {
2346 2684 i40e_error(i40e, "timed out disabling tx queue %d",
2347 2685 i);
2348 2686 return (B_FALSE);
2349 2687 }
2350 2688 }
2351 2689
2352 2690 return (B_TRUE);
2353 2691 }
2354 2692
2355 2693 static boolean_t
2356 2694 i40e_shutdown_rings(i40e_t *i40e)
2357 2695 {
2358 2696 i40e_shutdown_rx_rings(i40e);
2359 2697 i40e_shutdown_tx_rings(i40e);
2360 2698 return (i40e_shutdown_rings_wait(i40e));
2361 2699 }
2362 2700
2363 2701 static void
2364 2702 i40e_setup_rx_descs(i40e_trqpair_t *itrq)
2365 2703 {
2366 2704 int i;
2367 2705 i40e_rx_data_t *rxd = itrq->itrq_rxdata;
2368 2706
2369 2707 for (i = 0; i < rxd->rxd_ring_size; i++) {
2370 2708 i40e_rx_control_block_t *rcb;
2371 2709 i40e_rx_desc_t *rdesc;
2372 2710
2373 2711 rcb = rxd->rxd_work_list[i];
2374 2712 rdesc = &rxd->rxd_desc_ring[i];
2375 2713
2376 2714 rdesc->read.pkt_addr =
2377 2715 CPU_TO_LE64((uintptr_t)rcb->rcb_dma.dmab_dma_address);
2378 2716 rdesc->read.hdr_addr = 0;
2379 2717 }
2380 2718 }
2381 2719
2382 2720 static boolean_t
2383 2721 i40e_setup_rx_hmc(i40e_trqpair_t *itrq)
2384 2722 {
2385 2723 i40e_rx_data_t *rxd = itrq->itrq_rxdata;
2386 2724 i40e_t *i40e = itrq->itrq_i40e;
2387 2725 i40e_hw_t *hw = &i40e->i40e_hw_space;
2388 2726
2389 2727 struct i40e_hmc_obj_rxq rctx;
2390 2728 int err;
2391 2729
2392 2730 bzero(&rctx, sizeof (struct i40e_hmc_obj_rxq));
2393 2731 rctx.base = rxd->rxd_desc_area.dmab_dma_address /
2394 2732 I40E_HMC_RX_CTX_UNIT;
2395 2733 rctx.qlen = rxd->rxd_ring_size;
2396 2734 VERIFY(i40e->i40e_rx_buf_size >= I40E_HMC_RX_DBUFF_MIN);
2397 2735 VERIFY(i40e->i40e_rx_buf_size <= I40E_HMC_RX_DBUFF_MAX);
2398 2736 rctx.dbuff = i40e->i40e_rx_buf_size >> I40E_RXQ_CTX_DBUFF_SHIFT;
2399 2737 rctx.hbuff = 0 >> I40E_RXQ_CTX_HBUFF_SHIFT;
2400 2738 rctx.dtype = I40E_HMC_RX_DTYPE_NOSPLIT;
2401 2739 rctx.dsize = I40E_HMC_RX_DSIZE_32BYTE;
2402 2740 rctx.crcstrip = I40E_HMC_RX_CRCSTRIP_ENABLE;
2403 2741 rctx.fc_ena = I40E_HMC_RX_FC_DISABLE;
2404 2742 rctx.l2tsel = I40E_HMC_RX_L2TAGORDER;
2405 2743 rctx.hsplit_0 = I40E_HMC_RX_HDRSPLIT_DISABLE;
2406 2744 rctx.hsplit_1 = I40E_HMC_RX_HDRSPLIT_DISABLE;
2407 2745 rctx.showiv = I40E_HMC_RX_INVLAN_DONTSTRIP;
2408 2746 rctx.rxmax = i40e->i40e_frame_max;
2409 2747 rctx.tphrdesc_ena = I40E_HMC_RX_TPH_DISABLE;
2410 2748 rctx.tphwdesc_ena = I40E_HMC_RX_TPH_DISABLE;
2411 2749 rctx.tphdata_ena = I40E_HMC_RX_TPH_DISABLE;
2412 2750 rctx.tphhead_ena = I40E_HMC_RX_TPH_DISABLE;
2413 2751 rctx.lrxqthresh = I40E_HMC_RX_LOWRXQ_NOINTR;
2414 2752
2415 2753 /*
2416 2754 * This must be set to 0x1, see Table 8-12 in section 8.3.3.2.2.
2417 2755 */
2418 2756 rctx.prefena = I40E_HMC_RX_PREFENA;
2419 2757
2420 2758 err = i40e_clear_lan_rx_queue_context(hw, itrq->itrq_index);
2421 2759 if (err != I40E_SUCCESS) {
2422 2760 i40e_error(i40e, "failed to clear rx queue %d context: %d",
2423 2761 itrq->itrq_index, err);
2424 2762 return (B_FALSE);
2425 2763 }
2426 2764
2427 2765 err = i40e_set_lan_rx_queue_context(hw, itrq->itrq_index, &rctx);
2428 2766 if (err != I40E_SUCCESS) {
2429 2767 i40e_error(i40e, "failed to set rx queue %d context: %d",
2430 2768 itrq->itrq_index, err);
2431 2769 return (B_FALSE);
2432 2770 }
2433 2771
2434 2772 return (B_TRUE);
2435 2773 }
2436 2774
2437 2775 /*
2438 2776 * Take care of setting up the descriptor rings and actually programming the
2439 2777 * device. See 8.3.3.1.1 for the full list of steps we need to do to enable the
2440 2778 * rx rings.
2441 2779 */
2442 2780 static boolean_t
2443 2781 i40e_setup_rx_rings(i40e_t *i40e)
2444 2782 {
2445 2783 int i;
2446 2784 i40e_hw_t *hw = &i40e->i40e_hw_space;
2447 2785
2448 2786 for (i = 0; i < i40e->i40e_num_trqpairs; i++) {
2449 2787 i40e_trqpair_t *itrq = &i40e->i40e_trqpairs[i];
2450 2788 i40e_rx_data_t *rxd = itrq->itrq_rxdata;
2451 2789 uint32_t reg;
2452 2790
2453 2791 /*
2454 2792 * Step 1. Program all receive ring descriptors.
2455 2793 */
2456 2794 i40e_setup_rx_descs(itrq);
2457 2795
2458 2796 /*
2459 2797 * Step 2. Program the queue's FPM/HMC context.
2460 2798 */
2461 2799 if (i40e_setup_rx_hmc(itrq) == B_FALSE)
2462 2800 return (B_FALSE);
2463 2801
2464 2802 /*
2465 2803 * Step 3. Clear the queue's tail pointer and set it to the end
2466 2804 * of the space.
2467 2805 */
2468 2806 I40E_WRITE_REG(hw, I40E_QRX_TAIL(i), 0);
2469 2807 I40E_WRITE_REG(hw, I40E_QRX_TAIL(i), rxd->rxd_ring_size - 1);
2470 2808
2471 2809 /*
2472 2810 * Step 4. Enable the queue via the QENA_REQ.
2473 2811 */
2474 2812 reg = I40E_READ_REG(hw, I40E_QRX_ENA(i));
2475 2813 VERIFY0(reg & (I40E_QRX_ENA_QENA_REQ_MASK |
2476 2814 I40E_QRX_ENA_QENA_STAT_MASK));
2477 2815 reg |= I40E_QRX_ENA_QENA_REQ_MASK;
2478 2816 I40E_WRITE_REG(hw, I40E_QRX_ENA(i), reg);
2479 2817 }
2480 2818
2481 2819 /*
2482 2820 * Note, we wait for every queue to be enabled before we start checking.
2483 2821 * This will hopefully cause most queues to be enabled at this point.
2484 2822 */
2485 2823 for (i = 0; i < i40e->i40e_num_trqpairs; i++) {
2486 2824 uint32_t j, reg;
2487 2825
2488 2826 /*
2489 2827 * Step 5. Verify that QENA_STAT has been set. It's promised
2490 2828 * that this should occur within about 10 us, but like other
2491 2829 * systems, we give the card a bit more time.
2492 2830 */
2493 2831 for (j = 0; j < I40E_RING_WAIT_NTRIES; j++) {
2494 2832 reg = I40E_READ_REG(hw, I40E_QRX_ENA(i));
2495 2833
2496 2834 if (reg & I40E_QRX_ENA_QENA_STAT_MASK)
2497 2835 break;
2498 2836 i40e_msec_delay(I40E_RING_WAIT_PAUSE);
2499 2837 }
2500 2838
2501 2839 if ((reg & I40E_QRX_ENA_QENA_STAT_MASK) == 0) {
2502 2840 i40e_error(i40e, "failed to enable rx queue %d, timed "
2503 2841 "out.", i);
2504 2842 return (B_FALSE);
2505 2843 }
2506 2844 }
2507 2845
2508 2846 return (B_TRUE);
2509 2847 }
2510 2848
2511 2849 static boolean_t
2512 2850 i40e_setup_tx_hmc(i40e_trqpair_t *itrq)
2513 2851 {
2514 2852 i40e_t *i40e = itrq->itrq_i40e;
2515 2853 i40e_hw_t *hw = &i40e->i40e_hw_space;
2516 2854
2517 2855 struct i40e_hmc_obj_txq tctx;
2518 2856 struct i40e_vsi_context context;
2519 2857 int err;
2520 2858
2521 2859 bzero(&tctx, sizeof (struct i40e_hmc_obj_txq));
2522 2860 tctx.new_context = I40E_HMC_TX_NEW_CONTEXT;
2523 2861 tctx.base = itrq->itrq_desc_area.dmab_dma_address /
2524 2862 I40E_HMC_TX_CTX_UNIT;
2525 2863 tctx.fc_ena = I40E_HMC_TX_FC_DISABLE;
2526 2864 tctx.timesync_ena = I40E_HMC_TX_TS_DISABLE;
2527 2865 tctx.fd_ena = I40E_HMC_TX_FD_DISABLE;
2528 2866 tctx.alt_vlan_ena = I40E_HMC_TX_ALT_VLAN_DISABLE;
2529 2867 tctx.head_wb_ena = I40E_HMC_TX_WB_ENABLE;
2530 2868 tctx.qlen = itrq->itrq_tx_ring_size;
2531 2869 tctx.tphrdesc_ena = I40E_HMC_TX_TPH_DISABLE;
2532 2870 tctx.tphrpacket_ena = I40E_HMC_TX_TPH_DISABLE;
2533 2871 tctx.tphwdesc_ena = I40E_HMC_TX_TPH_DISABLE;
2534 2872 tctx.head_wb_addr = itrq->itrq_desc_area.dmab_dma_address +
2535 2873 sizeof (i40e_tx_desc_t) * itrq->itrq_tx_ring_size;
2536 2874
2537 2875 /*
2538 2876 * This field isn't actually documented, like crc, but it suggests that
2539 2877 * it should be zeroed. We leave both of these here because of that for
2540 2878 * now. We should check with Intel on why these are here even.
2541 2879 */
|
↓ open down ↓ |
338 lines elided |
↑ open up ↑ |
2542 2880 tctx.crc = 0;
2543 2881 tctx.rdylist_act = 0;
2544 2882
2545 2883 /*
2546 2884 * We're supposed to assign the rdylist field with the value of the
2547 2885 * traffic class index for the first device. We query the VSI parameters
2548 2886 * again to get what the handle is. Note that every queue is always
2549 2887 * assigned to traffic class zero, because we don't actually use them.
2550 2888 */
2551 2889 bzero(&context, sizeof (struct i40e_vsi_context));
2552 - context.seid = i40e->i40e_vsi_id;
2890 + context.seid = I40E_DEF_VSI_SEID(i40e);
2553 2891 context.pf_num = hw->pf_id;
2554 2892 err = i40e_aq_get_vsi_params(hw, &context, NULL);
2555 2893 if (err != I40E_SUCCESS) {
2556 2894 i40e_error(i40e, "get VSI params failed with %d", err);
2557 2895 return (B_FALSE);
2558 2896 }
2559 2897 tctx.rdylist = LE_16(context.info.qs_handle[0]);
2560 2898
2561 2899 err = i40e_clear_lan_tx_queue_context(hw, itrq->itrq_index);
2562 2900 if (err != I40E_SUCCESS) {
2563 2901 i40e_error(i40e, "failed to clear tx queue %d context: %d",
2564 2902 itrq->itrq_index, err);
2565 2903 return (B_FALSE);
2566 2904 }
2567 2905
2568 2906 err = i40e_set_lan_tx_queue_context(hw, itrq->itrq_index, &tctx);
2569 2907 if (err != I40E_SUCCESS) {
2570 2908 i40e_error(i40e, "failed to set tx queue %d context: %d",
2571 2909 itrq->itrq_index, err);
2572 2910 return (B_FALSE);
2573 2911 }
2574 2912
2575 2913 return (B_TRUE);
2576 2914 }
2577 2915
2578 2916 /*
2579 2917 * Take care of setting up the descriptor rings and actually programming the
2580 2918 * device. See 8.4.3.1.1 for what we need to do here.
2581 2919 */
2582 2920 static boolean_t
2583 2921 i40e_setup_tx_rings(i40e_t *i40e)
2584 2922 {
2585 2923 int i;
2586 2924 i40e_hw_t *hw = &i40e->i40e_hw_space;
2587 2925
2588 2926 for (i = 0; i < i40e->i40e_num_trqpairs; i++) {
2589 2927 i40e_trqpair_t *itrq = &i40e->i40e_trqpairs[i];
2590 2928 uint32_t reg;
2591 2929
2592 2930 /*
2593 2931 * Step 1. Clear the queue disable flag and verify that the
2594 2932 * index is set correctly.
2595 2933 */
2596 2934 i40e_pre_tx_queue_cfg(hw, i, B_TRUE);
2597 2935
2598 2936 /*
2599 2937 * Step 2. Prepare the queue's FPM/HMC context.
2600 2938 */
2601 2939 if (i40e_setup_tx_hmc(itrq) == B_FALSE)
2602 2940 return (B_FALSE);
2603 2941
2604 2942 /*
2605 2943 * Step 3. Verify that it's clear that this PF owns this queue.
2606 2944 */
2607 2945 reg = I40E_QTX_CTL_PF_QUEUE;
2608 2946 reg |= (hw->pf_id << I40E_QTX_CTL_PF_INDX_SHIFT) &
2609 2947 I40E_QTX_CTL_PF_INDX_MASK;
2610 2948 I40E_WRITE_REG(hw, I40E_QTX_CTL(itrq->itrq_index), reg);
2611 2949 i40e_flush(hw);
2612 2950
2613 2951 /*
2614 2952 * Step 4. Set the QENA_REQ flag.
2615 2953 */
2616 2954 reg = I40E_READ_REG(hw, I40E_QTX_ENA(i));
2617 2955 VERIFY0(reg & (I40E_QTX_ENA_QENA_REQ_MASK |
2618 2956 I40E_QTX_ENA_QENA_STAT_MASK));
2619 2957 reg |= I40E_QTX_ENA_QENA_REQ_MASK;
2620 2958 I40E_WRITE_REG(hw, I40E_QTX_ENA(i), reg);
2621 2959 }
2622 2960
2623 2961 /*
2624 2962 * Note, we wait for every queue to be enabled before we start checking.
2625 2963 * This will hopefully cause most queues to be enabled at this point.
2626 2964 */
2627 2965 for (i = 0; i < i40e->i40e_num_trqpairs; i++) {
2628 2966 uint32_t j, reg;
2629 2967
2630 2968 /*
2631 2969 * Step 5. Verify that QENA_STAT has been set. It's promised
2632 2970 * that this should occur within about 10 us, but like BSD,
2633 2971 * we'll try for up to 100 ms for this queue.
2634 2972 */
2635 2973 for (j = 0; j < I40E_RING_WAIT_NTRIES; j++) {
2636 2974 reg = I40E_READ_REG(hw, I40E_QTX_ENA(i));
2637 2975
2638 2976 if (reg & I40E_QTX_ENA_QENA_STAT_MASK)
2639 2977 break;
2640 2978 i40e_msec_delay(I40E_RING_WAIT_PAUSE);
2641 2979 }
2642 2980
2643 2981 if ((reg & I40E_QTX_ENA_QENA_STAT_MASK) == 0) {
2644 2982 i40e_error(i40e, "failed to enable tx queue %d, timed "
2645 2983 "out", i);
|
↓ open down ↓ |
83 lines elided |
↑ open up ↑ |
2646 2984 return (B_FALSE);
2647 2985 }
2648 2986 }
2649 2987
2650 2988 return (B_TRUE);
2651 2989 }
2652 2990
2653 2991 void
2654 2992 i40e_stop(i40e_t *i40e, boolean_t free_allocations)
2655 2993 {
2656 - int i;
2994 + uint_t i;
2995 + i40e_hw_t *hw = &i40e->i40e_hw_space;
2657 2996
2658 2997 ASSERT(MUTEX_HELD(&i40e->i40e_general_lock));
2659 2998
2660 2999 /*
2661 3000 * Shutdown and drain the tx and rx pipeline. We do this using the
2662 3001 * following steps.
2663 3002 *
2664 3003 * 1) Shutdown interrupts to all the queues (trying to keep the admin
2665 3004 * queue alive).
2666 3005 *
2667 3006 * 2) Remove all of the interrupt tx and rx causes by setting the
2668 3007 * interrupt linked lists to zero.
2669 3008 *
2670 3009 * 2) Shutdown the tx and rx rings. Because i40e_shutdown_rings() should
2671 3010 * wait for all the queues to be disabled, once we reach that point
2672 3011 * it should be safe to free associated data.
2673 3012 *
2674 3013 * 4) Wait 50ms after all that is done. This ensures that the rings are
2675 3014 * ready for programming again and we don't have to think about this
2676 3015 * in other parts of the driver.
2677 3016 *
2678 3017 * 5) Disable remaining chip interrupts, (admin queue, etc.)
2679 3018 *
2680 3019 * 6) Verify that FM is happy with all the register accesses we
2681 3020 * performed.
|
↓ open down ↓ |
15 lines elided |
↑ open up ↑ |
2682 3021 */
2683 3022 i40e_intr_io_disable_all(i40e);
2684 3023 i40e_intr_io_clear_cause(i40e);
2685 3024
2686 3025 if (i40e_shutdown_rings(i40e) == B_FALSE) {
2687 3026 ddi_fm_service_impact(i40e->i40e_dip, DDI_SERVICE_LOST);
2688 3027 }
2689 3028
2690 3029 delay(50 * drv_usectohz(1000));
2691 3030
3031 + /*
3032 + * We don't delete the default VSI because it replaces the VEB
3033 + * after VEB deletion (see the "Delete Element" section).
3034 + * Furthermore, since the default VSI is provided by the
3035 + * firmware, we never attempt to delete it.
3036 + */
3037 + for (i = 1; i < i40e->i40e_num_rx_groups; i++) {
3038 + i40e_delete_vsi(i40e, i);
3039 + }
3040 +
3041 + if (i40e->i40e_veb_seid != 0) {
3042 + int rc = i40e_aq_delete_element(hw, i40e->i40e_veb_seid, NULL);
3043 +
3044 + if (rc != I40E_SUCCESS) {
3045 + i40e_error(i40e, "Failed to delete VEB %d: %d", rc,
3046 + hw->aq.asq_last_status);
3047 + }
3048 +
3049 + i40e->i40e_veb_seid = 0;
3050 + }
3051 +
2692 3052 i40e_intr_chip_fini(i40e);
2693 3053
2694 3054 for (i = 0; i < i40e->i40e_num_trqpairs; i++) {
2695 3055 mutex_enter(&i40e->i40e_trqpairs[i].itrq_rx_lock);
2696 3056 mutex_enter(&i40e->i40e_trqpairs[i].itrq_tx_lock);
2697 3057 }
2698 3058
2699 3059 /*
2700 3060 * We should consider refactoring this to be part of the ring start /
2701 3061 * stop routines at some point.
2702 3062 */
2703 3063 for (i = 0; i < i40e->i40e_num_trqpairs; i++) {
2704 3064 i40e_stats_trqpair_fini(&i40e->i40e_trqpairs[i]);
2705 3065 }
2706 3066
2707 3067 if (i40e_check_acc_handle(i40e->i40e_osdep_space.ios_cfg_handle) !=
2708 3068 DDI_FM_OK) {
2709 3069 ddi_fm_service_impact(i40e->i40e_dip, DDI_SERVICE_LOST);
2710 3070 }
|
↓ open down ↓ |
9 lines elided |
↑ open up ↑ |
2711 3071
2712 3072 for (i = 0; i < i40e->i40e_num_trqpairs; i++) {
2713 3073 i40e_tx_cleanup_ring(&i40e->i40e_trqpairs[i]);
2714 3074 }
2715 3075
2716 3076 for (i = 0; i < i40e->i40e_num_trqpairs; i++) {
2717 3077 mutex_exit(&i40e->i40e_trqpairs[i].itrq_rx_lock);
2718 3078 mutex_exit(&i40e->i40e_trqpairs[i].itrq_tx_lock);
2719 3079 }
2720 3080
2721 - i40e_stat_vsi_fini(i40e);
3081 + for (i = 0; i < i40e->i40e_num_rx_groups; i++) {
3082 + i40e_stat_vsi_fini(i40e, i);
3083 + }
2722 3084
2723 3085 i40e->i40e_link_speed = 0;
2724 3086 i40e->i40e_link_duplex = 0;
2725 3087 i40e_link_state_set(i40e, LINK_STATE_UNKNOWN);
2726 3088
2727 3089 if (free_allocations) {
2728 3090 i40e_free_ring_mem(i40e, B_FALSE);
2729 3091 }
2730 3092 }
2731 3093
2732 3094 boolean_t
2733 3095 i40e_start(i40e_t *i40e, boolean_t alloc)
2734 3096 {
2735 3097 i40e_hw_t *hw = &i40e->i40e_hw_space;
2736 3098 boolean_t rc = B_TRUE;
2737 3099 int i, err;
2738 3100
2739 3101 ASSERT(MUTEX_HELD(&i40e->i40e_general_lock));
2740 3102
2741 3103 if (alloc) {
2742 3104 if (i40e_alloc_ring_mem(i40e) == B_FALSE) {
2743 3105 i40e_error(i40e,
2744 3106 "Failed to allocate ring memory");
2745 3107 return (B_FALSE);
2746 3108 }
2747 3109 }
2748 3110
2749 3111 /*
2750 3112 * This should get refactored to be part of ring start and stop at
2751 3113 * some point, along with most of the logic here.
2752 3114 */
2753 3115 for (i = 0; i < i40e->i40e_num_trqpairs; i++) {
2754 3116 if (i40e_stats_trqpair_init(&i40e->i40e_trqpairs[i]) ==
2755 3117 B_FALSE) {
2756 3118 int j;
2757 3119
2758 3120 for (j = 0; j < i; j++) {
2759 3121 i40e_trqpair_t *itrq = &i40e->i40e_trqpairs[j];
2760 3122 i40e_stats_trqpair_fini(itrq);
2761 3123 }
2762 3124 return (B_FALSE);
2763 3125 }
2764 3126 }
2765 3127
2766 3128 if (!i40e_chip_start(i40e)) {
2767 3129 i40e_fm_ereport(i40e, DDI_FM_DEVICE_INVAL_STATE);
2768 3130 rc = B_FALSE;
2769 3131 goto done;
2770 3132 }
2771 3133
2772 3134 if (i40e_setup_rx_rings(i40e) == B_FALSE) {
2773 3135 rc = B_FALSE;
2774 3136 goto done;
2775 3137 }
|
↓ open down ↓ |
44 lines elided |
↑ open up ↑ |
2776 3138
2777 3139 if (i40e_setup_tx_rings(i40e) == B_FALSE) {
2778 3140 rc = B_FALSE;
2779 3141 goto done;
2780 3142 }
2781 3143
2782 3144 /*
2783 3145 * Enable broadcast traffic; however, do not enable multicast traffic.
2784 3146 * That's handle exclusively through MAC's mc_multicst routines.
2785 3147 */
2786 - err = i40e_aq_set_vsi_broadcast(hw, i40e->i40e_vsi_id, B_TRUE, NULL);
3148 + err = i40e_aq_set_vsi_broadcast(hw, I40E_DEF_VSI_SEID(i40e), B_TRUE,
3149 + NULL);
2787 3150 if (err != I40E_SUCCESS) {
2788 3151 i40e_error(i40e, "failed to set default VSI: %d", err);
2789 3152 rc = B_FALSE;
2790 3153 goto done;
2791 3154 }
2792 3155
2793 3156 err = i40e_aq_set_mac_config(hw, i40e->i40e_frame_max, B_TRUE, 0, NULL);
2794 3157 if (err != I40E_SUCCESS) {
2795 3158 i40e_error(i40e, "failed to set MAC config: %d", err);
2796 3159 rc = B_FALSE;
2797 3160 goto done;
2798 3161 }
2799 3162
2800 3163 /*
2801 3164 * Finally, make sure that we're happy from an FM perspective.
2802 3165 */
2803 3166 if (i40e_check_acc_handle(i40e->i40e_osdep_space.ios_reg_handle) !=
2804 3167 DDI_FM_OK) {
2805 3168 rc = B_FALSE;
2806 3169 goto done;
2807 3170 }
2808 3171
2809 3172 /* Clear state bits prior to final interrupt enabling. */
2810 3173 atomic_and_32(&i40e->i40e_state,
2811 3174 ~(I40E_ERROR | I40E_STALL | I40E_OVERTEMP));
2812 3175
2813 3176 i40e_intr_io_enable_all(i40e);
2814 3177
2815 3178 done:
2816 3179 if (rc == B_FALSE) {
2817 3180 i40e_stop(i40e, B_FALSE);
2818 3181 if (alloc == B_TRUE) {
2819 3182 i40e_free_ring_mem(i40e, B_TRUE);
2820 3183 }
2821 3184 ddi_fm_service_impact(i40e->i40e_dip, DDI_SERVICE_LOST);
2822 3185 }
2823 3186
2824 3187 return (rc);
2825 3188 }
2826 3189
2827 3190 /*
2828 3191 * We may have loaned up descriptors to the stack. As such, if we still have
2829 3192 * them outstanding, then we will not continue with detach.
2830 3193 */
2831 3194 static boolean_t
2832 3195 i40e_drain_rx(i40e_t *i40e)
2833 3196 {
2834 3197 mutex_enter(&i40e->i40e_rx_pending_lock);
2835 3198 while (i40e->i40e_rx_pending > 0) {
2836 3199 if (cv_reltimedwait(&i40e->i40e_rx_pending_cv,
2837 3200 &i40e->i40e_rx_pending_lock,
2838 3201 drv_usectohz(I40E_DRAIN_RX_WAIT), TR_CLOCK_TICK) == -1) {
2839 3202 mutex_exit(&i40e->i40e_rx_pending_lock);
2840 3203 return (B_FALSE);
2841 3204 }
2842 3205 }
2843 3206 mutex_exit(&i40e->i40e_rx_pending_lock);
2844 3207
2845 3208 return (B_TRUE);
2846 3209 }
2847 3210
2848 3211 static int
2849 3212 i40e_attach(dev_info_t *devinfo, ddi_attach_cmd_t cmd)
2850 3213 {
2851 3214 i40e_t *i40e;
2852 3215 struct i40e_osdep *osdep;
2853 3216 i40e_hw_t *hw;
2854 3217 int instance;
2855 3218
2856 3219 if (cmd != DDI_ATTACH)
2857 3220 return (DDI_FAILURE);
2858 3221
2859 3222 instance = ddi_get_instance(devinfo);
2860 3223 i40e = kmem_zalloc(sizeof (i40e_t), KM_SLEEP);
2861 3224
2862 3225 i40e->i40e_aqbuf = kmem_zalloc(I40E_ADMINQ_BUFSZ, KM_SLEEP);
2863 3226 i40e->i40e_instance = instance;
2864 3227 i40e->i40e_dip = devinfo;
2865 3228
2866 3229 hw = &i40e->i40e_hw_space;
2867 3230 osdep = &i40e->i40e_osdep_space;
2868 3231 hw->back = osdep;
2869 3232 osdep->ios_i40e = i40e;
2870 3233
2871 3234 ddi_set_driver_private(devinfo, i40e);
2872 3235
2873 3236 i40e_fm_init(i40e);
2874 3237 i40e->i40e_attach_progress |= I40E_ATTACH_FM_INIT;
2875 3238
2876 3239 if (pci_config_setup(devinfo, &osdep->ios_cfg_handle) != DDI_SUCCESS) {
2877 3240 i40e_error(i40e, "Failed to map PCI configurations.");
2878 3241 goto attach_fail;
2879 3242 }
2880 3243 i40e->i40e_attach_progress |= I40E_ATTACH_PCI_CONFIG;
2881 3244
2882 3245 i40e_identify_hardware(i40e);
2883 3246
2884 3247 if (!i40e_regs_map(i40e)) {
2885 3248 i40e_error(i40e, "Failed to map device registers.");
2886 3249 goto attach_fail;
2887 3250 }
2888 3251 i40e->i40e_attach_progress |= I40E_ATTACH_REGS_MAP;
2889 3252
2890 3253 i40e_init_properties(i40e);
2891 3254 i40e->i40e_attach_progress |= I40E_ATTACH_PROPS;
2892 3255
2893 3256 if (!i40e_common_code_init(i40e, hw))
2894 3257 goto attach_fail;
2895 3258 i40e->i40e_attach_progress |= I40E_ATTACH_COMMON_CODE;
2896 3259
2897 3260 /*
2898 3261 * When we participate in IRM, we should make sure that we register
2899 3262 * ourselves with it before callbacks.
2900 3263 */
2901 3264 if (!i40e_alloc_intrs(i40e, devinfo)) {
2902 3265 i40e_error(i40e, "Failed to allocate interrupts.");
2903 3266 goto attach_fail;
2904 3267 }
2905 3268 i40e->i40e_attach_progress |= I40E_ATTACH_ALLOC_INTR;
2906 3269
2907 3270 if (!i40e_alloc_trqpairs(i40e)) {
2908 3271 i40e_error(i40e,
2909 3272 "Failed to allocate receive & transmit rings.");
2910 3273 goto attach_fail;
2911 3274 }
2912 3275 i40e->i40e_attach_progress |= I40E_ATTACH_ALLOC_RINGSLOCKS;
2913 3276
2914 3277 if (!i40e_map_intrs_to_vectors(i40e)) {
2915 3278 i40e_error(i40e, "Failed to map interrupts to vectors.");
2916 3279 goto attach_fail;
2917 3280 }
2918 3281
2919 3282 if (!i40e_add_intr_handlers(i40e)) {
2920 3283 i40e_error(i40e, "Failed to add the interrupt handlers.");
2921 3284 goto attach_fail;
2922 3285 }
2923 3286 i40e->i40e_attach_progress |= I40E_ATTACH_ADD_INTR;
2924 3287
2925 3288 if (!i40e_final_init(i40e)) {
2926 3289 i40e_error(i40e, "Final initialization failed.");
2927 3290 goto attach_fail;
2928 3291 }
2929 3292 i40e->i40e_attach_progress |= I40E_ATTACH_INIT;
2930 3293
2931 3294 if (i40e_check_acc_handle(i40e->i40e_osdep_space.ios_cfg_handle) !=
2932 3295 DDI_FM_OK) {
2933 3296 ddi_fm_service_impact(i40e->i40e_dip, DDI_SERVICE_LOST);
2934 3297 goto attach_fail;
2935 3298 }
2936 3299
2937 3300 if (!i40e_stats_init(i40e)) {
2938 3301 i40e_error(i40e, "Stats initialization failed.");
2939 3302 goto attach_fail;
2940 3303 }
2941 3304 i40e->i40e_attach_progress |= I40E_ATTACH_STATS;
2942 3305
2943 3306 if (!i40e_register_mac(i40e)) {
2944 3307 i40e_error(i40e, "Failed to register to MAC/GLDv3");
2945 3308 goto attach_fail;
2946 3309 }
2947 3310 i40e->i40e_attach_progress |= I40E_ATTACH_MAC;
2948 3311
2949 3312 i40e->i40e_periodic_id = ddi_periodic_add(i40e_timer, i40e,
2950 3313 I40E_CYCLIC_PERIOD, DDI_IPL_0);
2951 3314 if (i40e->i40e_periodic_id == 0) {
2952 3315 i40e_error(i40e, "Failed to add the link-check timer");
2953 3316 goto attach_fail;
2954 3317 }
2955 3318 i40e->i40e_attach_progress |= I40E_ATTACH_LINK_TIMER;
2956 3319
2957 3320 if (!i40e_enable_interrupts(i40e)) {
2958 3321 i40e_error(i40e, "Failed to enable DDI interrupts");
2959 3322 goto attach_fail;
2960 3323 }
2961 3324 i40e->i40e_attach_progress |= I40E_ATTACH_ENABLE_INTR;
2962 3325
2963 3326 atomic_or_32(&i40e->i40e_state, I40E_INITIALIZED);
2964 3327
2965 3328 mutex_enter(&i40e_glock);
2966 3329 list_insert_tail(&i40e_glist, i40e);
2967 3330 mutex_exit(&i40e_glock);
2968 3331
2969 3332 return (DDI_SUCCESS);
2970 3333
2971 3334 attach_fail:
2972 3335 i40e_unconfigure(devinfo, i40e);
2973 3336 return (DDI_FAILURE);
2974 3337 }
2975 3338
2976 3339 static int
2977 3340 i40e_detach(dev_info_t *devinfo, ddi_detach_cmd_t cmd)
2978 3341 {
2979 3342 i40e_t *i40e;
2980 3343
2981 3344 if (cmd != DDI_DETACH)
2982 3345 return (DDI_FAILURE);
2983 3346
2984 3347 i40e = (i40e_t *)ddi_get_driver_private(devinfo);
2985 3348 if (i40e == NULL) {
2986 3349 i40e_log(NULL, "i40e_detach() called with no i40e pointer!");
2987 3350 return (DDI_FAILURE);
2988 3351 }
2989 3352
2990 3353 if (i40e_drain_rx(i40e) == B_FALSE) {
2991 3354 i40e_log(i40e, "timed out draining DMA resources, %d buffers "
2992 3355 "remain", i40e->i40e_rx_pending);
2993 3356 return (DDI_FAILURE);
2994 3357 }
2995 3358
2996 3359 mutex_enter(&i40e_glock);
2997 3360 list_remove(&i40e_glist, i40e);
2998 3361 mutex_exit(&i40e_glock);
2999 3362
3000 3363 i40e_unconfigure(devinfo, i40e);
3001 3364
3002 3365 return (DDI_SUCCESS);
3003 3366 }
3004 3367
3005 3368 static struct cb_ops i40e_cb_ops = {
3006 3369 nulldev, /* cb_open */
3007 3370 nulldev, /* cb_close */
3008 3371 nodev, /* cb_strategy */
3009 3372 nodev, /* cb_print */
3010 3373 nodev, /* cb_dump */
3011 3374 nodev, /* cb_read */
3012 3375 nodev, /* cb_write */
3013 3376 nodev, /* cb_ioctl */
3014 3377 nodev, /* cb_devmap */
3015 3378 nodev, /* cb_mmap */
3016 3379 nodev, /* cb_segmap */
3017 3380 nochpoll, /* cb_chpoll */
3018 3381 ddi_prop_op, /* cb_prop_op */
3019 3382 NULL, /* cb_stream */
3020 3383 D_MP | D_HOTPLUG, /* cb_flag */
3021 3384 CB_REV, /* cb_rev */
3022 3385 nodev, /* cb_aread */
3023 3386 nodev /* cb_awrite */
3024 3387 };
3025 3388
3026 3389 static struct dev_ops i40e_dev_ops = {
3027 3390 DEVO_REV, /* devo_rev */
3028 3391 0, /* devo_refcnt */
3029 3392 NULL, /* devo_getinfo */
3030 3393 nulldev, /* devo_identify */
3031 3394 nulldev, /* devo_probe */
3032 3395 i40e_attach, /* devo_attach */
3033 3396 i40e_detach, /* devo_detach */
3034 3397 nodev, /* devo_reset */
3035 3398 &i40e_cb_ops, /* devo_cb_ops */
3036 3399 NULL, /* devo_bus_ops */
3037 3400 ddi_power, /* devo_power */
3038 3401 ddi_quiesce_not_supported /* devo_quiesce */
3039 3402 };
3040 3403
3041 3404 static struct modldrv i40e_modldrv = {
3042 3405 &mod_driverops,
3043 3406 i40e_ident,
3044 3407 &i40e_dev_ops
3045 3408 };
3046 3409
3047 3410 static struct modlinkage i40e_modlinkage = {
3048 3411 MODREV_1,
3049 3412 &i40e_modldrv,
3050 3413 NULL
3051 3414 };
3052 3415
3053 3416 /*
3054 3417 * Module Initialization Functions.
3055 3418 */
3056 3419 int
3057 3420 _init(void)
3058 3421 {
3059 3422 int status;
3060 3423
3061 3424 list_create(&i40e_glist, sizeof (i40e_t), offsetof(i40e_t, i40e_glink));
3062 3425 list_create(&i40e_dlist, sizeof (i40e_device_t),
3063 3426 offsetof(i40e_device_t, id_link));
3064 3427 mutex_init(&i40e_glock, NULL, MUTEX_DRIVER, NULL);
3065 3428 mac_init_ops(&i40e_dev_ops, I40E_MODULE_NAME);
3066 3429
3067 3430 status = mod_install(&i40e_modlinkage);
3068 3431 if (status != DDI_SUCCESS) {
3069 3432 mac_fini_ops(&i40e_dev_ops);
3070 3433 mutex_destroy(&i40e_glock);
3071 3434 list_destroy(&i40e_dlist);
3072 3435 list_destroy(&i40e_glist);
3073 3436 }
3074 3437
3075 3438 return (status);
3076 3439 }
3077 3440
3078 3441 int
3079 3442 _info(struct modinfo *modinfop)
3080 3443 {
3081 3444 return (mod_info(&i40e_modlinkage, modinfop));
3082 3445 }
3083 3446
3084 3447 int
3085 3448 _fini(void)
3086 3449 {
3087 3450 int status;
3088 3451
3089 3452 status = mod_remove(&i40e_modlinkage);
3090 3453 if (status == DDI_SUCCESS) {
3091 3454 mac_fini_ops(&i40e_dev_ops);
3092 3455 mutex_destroy(&i40e_glock);
3093 3456 list_destroy(&i40e_dlist);
3094 3457 list_destroy(&i40e_glist);
3095 3458 }
3096 3459
3097 3460 return (status);
3098 3461 }
|
↓ open down ↓ |
302 lines elided |
↑ open up ↑ |
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX