Print this page
NEX-16819 loader UEFI support
Includes work by Toomas Soome <tsoome@me.com>
Upstream commits:
loader: pxe receive cleanup
9475 libefi: Do not return only if ReceiveFilter
installboot: should support efi system partition
8931 boot1.efi: scan all display modes rather than
loader: spinconsole updates
loader: gfx experiment to try GOP Blt() function.
sha1 build test
loader: add sha1 hash calculation
common/sha1: update for loader build
loader: biosdisk rework
uts: 32-bit kernel FB needs mapping in low memory
uts: add diag-device
uts: boot console mirror with diag-device
uts: enable very early console on ttya
kmdb: add diag-device as input/output device
uts: test VGA memory exclusion from mapping
uts: clear boot mapping and protect boot pages test
uts: add dboot map debug printf
uts: need to release FB pages in release_bootstrap()
uts: add screenmap ioctl
uts: update sys/queue.h
loader: add illumos uts/common to include path
loader: tem/gfx font cleanup
loader: vbe checks
uts: gfx_private set KD_TEXT when KD_RESETTEXT is
uts: gfx 8-bit update
loader: gfx 8-bit fix
loader: always set media size from partition.
uts: MB2 support for 32-bit kernel
loader: x86 should have tem 80x25
uts: x86 should have tem 80x25
uts: font update
loader: font update
uts: tem attributes
loader: tem.c comment added
uts: use font module
loader: add font module
loader: build rules for new font setup
uts: gfx_private update for new font structure
uts: early boot update for new font structure
uts: font update
uts: font build rules update for new fonts
uts: tem update to new font structure
loader: module.c needs to include tem_impl.h
uts: gfx_private 8x16 font rework
uts: make font_lookup public
loader: font rework
uts: font rework
9259 libefi: efi_alloc_and_read should check for PMBR
uts: tem utf-8 support
loader: implement tem utf-8 support
loader: tem should be able to display UTF-8
7784 uts: console input should support utf-8
7796 uts: ldterm default to utf-8
uts: do not reset serial console
uts: set up colors even if tem is not console
uts: add type for early boot properties
uts: gfx_private experiment with drm and vga
uts: gfx_private should use setmode drm callback.
uts: identify FB types and set up gfx_private based
loader: replace gop and vesa with framebuffer
uts: boot needs simple tem to support mdb
uts: boot_keyboard should emit esc sequences for
uts: gfx_private FB showuld be written by line
kmdb: set terminal window size
uts: gfx_private needs to keep track of early boot FB
pnglite: move pnglite to usr/src/common
loader: gfx_fb
ficl-sys: add gfx primitives
loader: add illumos.png logo
ficl: add fb-putimage
loader: add png support
loader: add alpha blending for gfx_fb
loader: use term-drawrect for menu frame
ficl: add simple gfx words
uts: provide fb_info via fbgattr dev_specific array.
uts: gfx_private add alpha blending
uts: update sys/ascii.h
uts: tem OSC support (incomplete)
uts: implement env module support and use data from
uts: tem get colors from early boot data
loader: use crc32 from libstand (libz)
loader: optimize for size
loader: pass tem info to the environment
loader: import tem for loader console
loader: UEFI loader needs to set ISADIR based on
loader: need UEFI32 support
8918 loader.efi: add vesa edid support
uts: tem_safe_pix_clear_prom_output() should only
uts: tem_safe_pix_clear_entire_screen() should use
uts: tem_safe_check_first_time() should query cursor
uts: tem implement cls callback & visual_io v4
uts: gfx_vgatext use block cursor for vgatext
uts: gfx_private implement cls callback & visual_io
uts: gfx_private bitmap framebuffer implementation
uts: early start frame buffer console support
uts: font functions should check the input char
uts: font rendering should support 16/24/32bit depths
uts: use smallest font as fallback default.
uts: update terminal dimensions based on selected
7834 uts: vgatext should use gfx_private
uts: add spacing property to 8859-1.bdf
terminfo: add underline for sun-color
terminfo: sun-color has 16 colors
uts: add font load callback type
loader: do not repeat int13 calls with error 0x20 and
8905 loader: add skein/edonr support
8904 common/crypto: make skein and edonr loader
Reviewed by: Yuri Pankov <yuri.pankov@nexenta.com>
Reviewed by: Sanjay Nadkarni <sanjay.nadkarni@nexenta.com>
Reviewed by: Evan Layton <evan.layton@nexenta.com>
Revert "NEX-16819 loader UEFI support"
This reverts commit ec06b9fc617b99234e538bf2e7e4d02a24993e0c.
Reverting due to failures in the zfs-tests and the sharefs-tests
NEX-16819 loader UEFI support
Includes work by Toomas Soome <tsoome@me.com>
Upstream commits:
loader: pxe receive cleanup
9475 libefi: Do not return only if ReceiveFilter
installboot: should support efi system partition
8931 boot1.efi: scan all display modes rather than
loader: spinconsole updates
loader: gfx experiment to try GOP Blt() function.
sha1 build test
loader: add sha1 hash calculation
common/sha1: update for loader build
loader: biosdisk rework
uts: 32-bit kernel FB needs mapping in low memory
uts: add diag-device
uts: boot console mirror with diag-device
uts: enable very early console on ttya
kmdb: add diag-device as input/output device
uts: test VGA memory exclusion from mapping
uts: clear boot mapping and protect boot pages test
uts: add dboot map debug printf
uts: need to release FB pages in release_bootstrap()
uts: add screenmap ioctl
uts: update sys/queue.h
loader: add illumos uts/common to include path
loader: tem/gfx font cleanup
loader: vbe checks
uts: gfx_private set KD_TEXT when KD_RESETTEXT is
uts: gfx 8-bit update
loader: gfx 8-bit fix
loader: always set media size from partition.
uts: MB2 support for 32-bit kernel
loader: x86 should have tem 80x25
uts: x86 should have tem 80x25
uts: font update
loader: font update
uts: tem attributes
loader: tem.c comment added
uts: use font module
loader: add font module
loader: build rules for new font setup
uts: gfx_private update for new font structure
uts: early boot update for new font structure
uts: font update
uts: font build rules update for new fonts
uts: tem update to new font structure
loader: module.c needs to include tem_impl.h
uts: gfx_private 8x16 font rework
uts: make font_lookup public
loader: font rework
uts: font rework
libefi: efi_alloc_and_read should check for PMBR
uts: tem utf-8 support
loader: implement tem utf-8 support
loader: tem should be able to display UTF-8
7784 uts: console input should support utf-8
7796 uts: ldterm default to utf-8
uts: do not reset serial console
uts: set up colors even if tem is not console
uts: add type for early boot properties
uts: gfx_private experiment with drm and vga
uts: gfx_private should use setmode drm callback.
uts: identify FB types and set up gfx_private based
loader: replace gop and vesa with framebuffer
uts: boot needs simple tem to support mdb
uts: boot_keyboard should emit esc sequences for
uts: gfx_private FB showuld be written by line
kmdb: set terminal window size
uts: gfx_private needs to keep track of early boot FB
pnglite: move pnglite to usr/src/common
loader: gfx_fb
ficl-sys: add gfx primitives
loader: add illumos.png logo
ficl: add fb-putimage
loader: add png support
loader: add alpha blending for gfx_fb
loader: use term-drawrect for menu frame
ficl: add simple gfx words
uts: provide fb_info via fbgattr dev_specific array.
uts: gfx_private add alpha blending
uts: update sys/ascii.h
uts: tem OSC support (incomplete)
uts: implement env module support and use data from
uts: tem get colors from early boot data
loader: use crc32 from libstand (libz)
loader: optimize for size
loader: pass tem info to the environment
loader: import tem for loader console
loader: UEFI loader needs to set ISADIR based on
loader: need UEFI32 support
8918 loader.efi: add vesa edid support
uts: tem_safe_pix_clear_prom_output() should only
uts: tem_safe_pix_clear_entire_screen() should use
uts: tem_safe_check_first_time() should query cursor
uts: tem implement cls callback & visual_io v4
uts: gfx_vgatext use block cursor for vgatext
uts: gfx_private implement cls callback & visual_io
uts: gfx_private bitmap framebuffer implementation
uts: early start frame buffer console support
uts: font functions should check the input char
uts: font rendering should support 16/24/32bit depths
uts: use smallest font as fallback default.
uts: update terminal dimensions based on selected
7834 uts: vgatext should use gfx_private
uts: add spacing property to 8859-1.bdf
terminfo: add underline for sun-color
terminfo: sun-color has 16 colors
uts: add font load callback type
loader: do not repeat int13 calls with error 0x20 and
8905 loader: add skein/edonr support
8904 common/crypto: make skein and edonr loader
Reviewed by: Yuri Pankov <yuri.pankov@nexenta.com>
Reviewed by: Sanjay Nadkarni <sanjay.nadkarni@nexenta.com>
Reviewed by: Evan Layton <evan.layton@nexenta.com>
| Split |
Close |
| Expand all |
| Collapse all |
--- old/usr/src/uts/i86pc/os/ddi_impl.c
+++ new/usr/src/uts/i86pc/os/ddi_impl.c
1 1 /*
2 2 * CDDL HEADER START
3 3 *
4 4 * The contents of this file are subject to the terms of the
5 5 * Common Development and Distribution License (the "License").
6 6 * You may not use this file except in compliance with the License.
7 7 *
8 8 * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE
9 9 * or http://www.opensolaris.org/os/licensing.
10 10 * See the License for the specific language governing permissions
11 11 * and limitations under the License.
12 12 *
13 13 * When distributing Covered Code, include this CDDL HEADER in each
14 14 * file and include the License file at usr/src/OPENSOLARIS.LICENSE.
15 15 * If applicable, add the following below this CDDL HEADER, with the
16 16 * fields enclosed by brackets "[]" replaced with your own identifying
17 17 * information: Portions Copyright [yyyy] [name of copyright owner]
18 18 *
19 19 * CDDL HEADER END
20 20 */
21 21
22 22 /*
23 23 * Copyright (c) 1992, 2010, Oracle and/or its affiliates. All rights reserved.
24 24 * Copyright 2012 Garrett D'Amore <garrett@damore.org>
25 25 * Copyright 2014 Pluribus Networks, Inc.
26 26 * Copyright 2016 Nexenta Systems, Inc.
27 27 */
28 28
29 29 /*
30 30 * PC specific DDI implementation
31 31 */
32 32 #include <sys/types.h>
33 33 #include <sys/autoconf.h>
34 34 #include <sys/avintr.h>
35 35 #include <sys/bootconf.h>
36 36 #include <sys/conf.h>
37 37 #include <sys/cpuvar.h>
38 38 #include <sys/ddi_impldefs.h>
39 39 #include <sys/ddi_subrdefs.h>
40 40 #include <sys/ethernet.h>
41 41 #include <sys/fp.h>
42 42 #include <sys/instance.h>
43 43 #include <sys/kmem.h>
44 44 #include <sys/machsystm.h>
45 45 #include <sys/modctl.h>
46 46 #include <sys/promif.h>
47 47 #include <sys/prom_plat.h>
48 48 #include <sys/sunndi.h>
49 49 #include <sys/ndi_impldefs.h>
50 50 #include <sys/ddi_impldefs.h>
51 51 #include <sys/sysmacros.h>
52 52 #include <sys/systeminfo.h>
53 53 #include <sys/utsname.h>
54 54 #include <sys/atomic.h>
55 55 #include <sys/spl.h>
56 56 #include <sys/archsystm.h>
57 57 #include <vm/seg_kmem.h>
58 58 #include <sys/ontrap.h>
59 59 #include <sys/fm/protocol.h>
60 60 #include <sys/ramdisk.h>
|
↓ open down ↓ |
60 lines elided |
↑ open up ↑ |
61 61 #include <sys/sunndi.h>
62 62 #include <sys/vmem.h>
63 63 #include <sys/pci_impl.h>
64 64 #if defined(__xpv)
65 65 #include <sys/hypervisor.h>
66 66 #endif
67 67 #include <sys/mach_intr.h>
68 68 #include <vm/hat_i86.h>
69 69 #include <sys/x86_archext.h>
70 70 #include <sys/avl.h>
71 +#include <sys/font.h>
71 72
72 73 /*
73 74 * DDI Boot Configuration
74 75 */
75 76
76 77 /*
77 78 * Platform drivers on this platform
78 79 */
79 80 char *platform_module_list[] = {
80 81 "acpippm",
81 82 "ppm",
82 83 (char *)0
83 84 };
84 85
85 86 /* pci bus resource maps */
86 87 struct pci_bus_resource *pci_bus_res;
87 88
88 89 size_t dma_max_copybuf_size = 0x101000; /* 1M + 4K */
89 90
90 91 uint64_t ramdisk_start, ramdisk_end;
91 92
92 93 int pseudo_isa = 0;
93 94
94 95 /*
95 96 * Forward declarations
96 97 */
97 98 static int getlongprop_buf();
98 99 static void get_boot_properties(void);
99 100 static void impl_bus_initialprobe(void);
100 101 static void impl_bus_reprobe(void);
101 102
102 103 static int poke_mem(peekpoke_ctlops_t *in_args);
103 104 static int peek_mem(peekpoke_ctlops_t *in_args);
104 105
105 106 static int kmem_override_cache_attrs(caddr_t, size_t, uint_t);
106 107
107 108 #if defined(__amd64) && !defined(__xpv)
108 109 extern void immu_init(void);
109 110 #endif
110 111
111 112 /*
112 113 * We use an AVL tree to store contiguous address allocations made with the
113 114 * kalloca() routine, so that we can return the size to free with kfreea().
114 115 * Note that in the future it would be vastly faster if we could eliminate
115 116 * this lookup by insisting that all callers keep track of their own sizes,
116 117 * just as for kmem_alloc().
117 118 */
118 119 struct ctgas {
119 120 avl_node_t ctg_link;
120 121 void *ctg_addr;
121 122 size_t ctg_size;
122 123 };
123 124
124 125 static avl_tree_t ctgtree;
125 126
126 127 static kmutex_t ctgmutex;
127 128 #define CTGLOCK() mutex_enter(&ctgmutex)
128 129 #define CTGUNLOCK() mutex_exit(&ctgmutex)
129 130
130 131 /*
131 132 * Minimum pfn value of page_t's put on the free list. This is to simplify
132 133 * support of ddi dma memory requests which specify small, non-zero addr_lo
133 134 * values.
134 135 *
135 136 * The default value of 2, which corresponds to the only known non-zero addr_lo
136 137 * value used, means a single page will be sacrificed (pfn typically starts
137 138 * at 1). ddiphysmin can be set to 0 to disable. It cannot be set above 0x100
138 139 * otherwise mp startup panics.
139 140 */
140 141 pfn_t ddiphysmin = 2;
141 142
142 143 static void
143 144 check_driver_disable(void)
144 145 {
145 146 int proplen = 128;
146 147 char *prop_name;
147 148 char *drv_name, *propval;
148 149 major_t major;
149 150
150 151 prop_name = kmem_alloc(proplen, KM_SLEEP);
151 152 for (major = 0; major < devcnt; major++) {
152 153 drv_name = ddi_major_to_name(major);
153 154 if (drv_name == NULL)
154 155 continue;
155 156 (void) snprintf(prop_name, proplen, "disable-%s", drv_name);
156 157 if (ddi_prop_lookup_string(DDI_DEV_T_ANY, ddi_root_node(),
157 158 DDI_PROP_DONTPASS, prop_name, &propval) == DDI_SUCCESS) {
158 159 if (strcmp(propval, "true") == 0) {
159 160 devnamesp[major].dn_flags |= DN_DRIVER_REMOVED;
160 161 cmn_err(CE_NOTE, "driver %s disabled",
161 162 drv_name);
162 163 }
163 164 ddi_prop_free(propval);
164 165 }
165 166 }
166 167 kmem_free(prop_name, proplen);
167 168 }
168 169
169 170
170 171 /*
171 172 * Configure the hardware on the system.
172 173 * Called before the rootfs is mounted
173 174 */
174 175 void
175 176 configure(void)
176 177 {
177 178 extern void i_ddi_init_root();
178 179
179 180 #if defined(__i386)
180 181 extern int fpu_pentium_fdivbug;
181 182 #endif /* __i386 */
182 183 extern int fpu_ignored;
183 184
184 185 /*
185 186 * Determine if an FPU is attached
186 187 */
187 188
188 189 fpu_probe();
189 190
190 191 #if defined(__i386)
191 192 if (fpu_pentium_fdivbug) {
192 193 printf("\
193 194 FP hardware exhibits Pentium floating point divide problem\n");
194 195 }
195 196 #endif /* __i386 */
196 197
197 198 if (fpu_ignored) {
198 199 printf("FP hardware will not be used\n");
199 200 } else if (!fpu_exists) {
200 201 printf("No FPU in configuration\n");
201 202 }
202 203
203 204 /*
204 205 * Initialize devices on the machine.
205 206 * Uses configuration tree built by the PROMs to determine what
206 207 * is present, and builds a tree of prototype dev_info nodes
207 208 * corresponding to the hardware which identified itself.
208 209 */
209 210
210 211 /*
211 212 * Initialize root node.
212 213 */
213 214 i_ddi_init_root();
214 215
215 216 /* reprogram devices not set up by firmware (BIOS) */
216 217 impl_bus_reprobe();
217 218
218 219 #if defined(__amd64) && !defined(__xpv)
219 220 /*
220 221 * Setup but don't startup the IOMMU
221 222 * Startup happens later via a direct call
222 223 * to IOMMU code by boot code.
223 224 * At this point, all PCI bus renumbering
224 225 * is done, so safe to init the IMMU
225 226 * AKA Intel IOMMU.
226 227 */
227 228 immu_init();
228 229 #endif
229 230
230 231 /*
231 232 * attach the isa nexus to get ACPI resource usage
232 233 * isa is "kind of" a pseudo node
233 234 */
234 235 #if defined(__xpv)
235 236 if (DOMAIN_IS_INITDOMAIN(xen_info)) {
236 237 if (pseudo_isa)
237 238 (void) i_ddi_attach_pseudo_node("isa");
238 239 else
239 240 (void) i_ddi_attach_hw_nodes("isa");
240 241 }
241 242 #else
242 243 if (pseudo_isa)
243 244 (void) i_ddi_attach_pseudo_node("isa");
244 245 else
245 246 (void) i_ddi_attach_hw_nodes("isa");
246 247 #endif
247 248 }
248 249
249 250 /*
250 251 * The "status" property indicates the operational status of a device.
251 252 * If this property is present, the value is a string indicating the
252 253 * status of the device as follows:
253 254 *
254 255 * "okay" operational.
255 256 * "disabled" not operational, but might become operational.
256 257 * "fail" not operational because a fault has been detected,
257 258 * and it is unlikely that the device will become
258 259 * operational without repair. no additional details
259 260 * are available.
260 261 * "fail-xxx" not operational because a fault has been detected,
261 262 * and it is unlikely that the device will become
262 263 * operational without repair. "xxx" is additional
263 264 * human-readable information about the particular
264 265 * fault condition that was detected.
265 266 *
266 267 * The absence of this property means that the operational status is
267 268 * unknown or okay.
268 269 *
269 270 * This routine checks the status property of the specified device node
270 271 * and returns 0 if the operational status indicates failure, and 1 otherwise.
271 272 *
272 273 * The property may exist on plug-in cards the existed before IEEE 1275-1994.
273 274 * And, in that case, the property may not even be a string. So we carefully
274 275 * check for the value "fail", in the beginning of the string, noting
275 276 * the property length.
276 277 */
277 278 int
278 279 status_okay(int id, char *buf, int buflen)
279 280 {
280 281 char status_buf[OBP_MAXPROPNAME];
281 282 char *bufp = buf;
282 283 int len = buflen;
283 284 int proplen;
284 285 static const char *status = "status";
285 286 static const char *fail = "fail";
286 287 int fail_len = (int)strlen(fail);
287 288
288 289 /*
289 290 * Get the proplen ... if it's smaller than "fail",
290 291 * or doesn't exist ... then we don't care, since
291 292 * the value can't begin with the char string "fail".
292 293 *
293 294 * NB: proplen, if it's a string, includes the NULL in the
294 295 * the size of the property, and fail_len does not.
295 296 */
296 297 proplen = prom_getproplen((pnode_t)id, (caddr_t)status);
297 298 if (proplen <= fail_len) /* nonexistant or uninteresting len */
298 299 return (1);
299 300
300 301 /*
301 302 * if a buffer was provided, use it
302 303 */
303 304 if ((buf == (char *)NULL) || (buflen <= 0)) {
304 305 bufp = status_buf;
305 306 len = sizeof (status_buf);
306 307 }
307 308 *bufp = (char)0;
308 309
309 310 /*
310 311 * Get the property into the buffer, to the extent of the buffer,
311 312 * and in case the buffer is smaller than the property size,
312 313 * NULL terminate the buffer. (This handles the case where
313 314 * a buffer was passed in and the caller wants to print the
314 315 * value, but the buffer was too small).
315 316 */
316 317 (void) prom_bounded_getprop((pnode_t)id, (caddr_t)status,
317 318 (caddr_t)bufp, len);
318 319 *(bufp + len - 1) = (char)0;
319 320
320 321 /*
321 322 * If the value begins with the char string "fail",
322 323 * then it means the node is failed. We don't care
323 324 * about any other values. We assume the node is ok
324 325 * although it might be 'disabled'.
325 326 */
326 327 if (strncmp(bufp, fail, fail_len) == 0)
327 328 return (0);
328 329
329 330 return (1);
330 331 }
331 332
332 333 /*
333 334 * Check the status of the device node passed as an argument.
334 335 *
335 336 * if ((status is OKAY) || (status is DISABLED))
336 337 * return DDI_SUCCESS
337 338 * else
338 339 * print a warning and return DDI_FAILURE
339 340 */
340 341 /*ARGSUSED1*/
341 342 int
342 343 check_status(int id, char *name, dev_info_t *parent)
343 344 {
344 345 char status_buf[64];
345 346 char devtype_buf[OBP_MAXPROPNAME];
346 347 int retval = DDI_FAILURE;
347 348
348 349 /*
349 350 * is the status okay?
350 351 */
351 352 if (status_okay(id, status_buf, sizeof (status_buf)))
352 353 return (DDI_SUCCESS);
353 354
354 355 /*
355 356 * a status property indicating bad memory will be associated
356 357 * with a node which has a "device_type" property with a value of
357 358 * "memory-controller". in this situation, return DDI_SUCCESS
358 359 */
359 360 if (getlongprop_buf(id, OBP_DEVICETYPE, devtype_buf,
360 361 sizeof (devtype_buf)) > 0) {
361 362 if (strcmp(devtype_buf, "memory-controller") == 0)
362 363 retval = DDI_SUCCESS;
363 364 }
364 365
365 366 /*
366 367 * print the status property information
367 368 */
368 369 cmn_err(CE_WARN, "status '%s' for '%s'", status_buf, name);
369 370 return (retval);
370 371 }
371 372
372 373 /*ARGSUSED*/
373 374 uint_t
374 375 softlevel1(caddr_t arg1, caddr_t arg2)
375 376 {
376 377 softint();
377 378 return (1);
378 379 }
379 380
380 381 /*
381 382 * Allow for implementation specific correction of PROM property values.
382 383 */
383 384
384 385 /*ARGSUSED*/
385 386 void
386 387 impl_fix_props(dev_info_t *dip, dev_info_t *ch_dip, char *name, int len,
387 388 caddr_t buffer)
388 389 {
389 390 /*
390 391 * There are no adjustments needed in this implementation.
391 392 */
392 393 }
393 394
394 395 static int
395 396 getlongprop_buf(int id, char *name, char *buf, int maxlen)
396 397 {
397 398 int size;
398 399
399 400 size = prom_getproplen((pnode_t)id, name);
400 401 if (size <= 0 || (size > maxlen - 1))
401 402 return (-1);
402 403
403 404 if (-1 == prom_getprop((pnode_t)id, name, buf))
404 405 return (-1);
405 406
406 407 if (strcmp("name", name) == 0) {
407 408 if (buf[size - 1] != '\0') {
408 409 buf[size] = '\0';
409 410 size += 1;
410 411 }
411 412 }
412 413
413 414 return (size);
414 415 }
415 416
416 417 static int
417 418 get_prop_int_array(dev_info_t *di, char *pname, int **pval, uint_t *plen)
418 419 {
419 420 int ret;
420 421
421 422 if ((ret = ddi_prop_lookup_int_array(DDI_DEV_T_ANY, di,
422 423 DDI_PROP_DONTPASS, pname, pval, plen))
423 424 == DDI_PROP_SUCCESS) {
424 425 *plen = (*plen) * (sizeof (int));
425 426 }
426 427 return (ret);
427 428 }
428 429
429 430
430 431 /*
431 432 * Node Configuration
432 433 */
433 434
434 435 struct prop_ispec {
435 436 uint_t pri, vec;
436 437 };
437 438
438 439 /*
439 440 * For the x86, we're prepared to claim that the interrupt string
440 441 * is in the form of a list of <ipl,vec> specifications.
441 442 */
442 443
443 444 #define VEC_MIN 1
444 445 #define VEC_MAX 255
445 446
446 447 static int
447 448 impl_xlate_intrs(dev_info_t *child, int *in,
448 449 struct ddi_parent_private_data *pdptr)
449 450 {
450 451 size_t size;
451 452 int n;
452 453 struct intrspec *new;
453 454 caddr_t got_prop;
454 455 int *inpri;
455 456 int got_len;
456 457 extern int ignore_hardware_nodes; /* force flag from ddi_impl.c */
457 458
458 459 static char bad_intr_fmt[] =
459 460 "bad interrupt spec from %s%d - ipl %d, irq %d\n";
460 461
461 462 /*
462 463 * determine if the driver is expecting the new style "interrupts"
463 464 * property which just contains the IRQ, or the old style which
464 465 * contains pairs of <IPL,IRQ>. if it is the new style, we always
465 466 * assign IPL 5 unless an "interrupt-priorities" property exists.
466 467 * in that case, the "interrupt-priorities" property contains the
467 468 * IPL values that match, one for one, the IRQ values in the
468 469 * "interrupts" property.
469 470 */
470 471 inpri = NULL;
471 472 if ((ddi_getprop(DDI_DEV_T_ANY, child, DDI_PROP_DONTPASS,
472 473 "ignore-hardware-nodes", -1) != -1) || ignore_hardware_nodes) {
473 474 /* the old style "interrupts" property... */
474 475
475 476 /*
476 477 * The list consists of <ipl,vec> elements
477 478 */
478 479 if ((n = (*in++ >> 1)) < 1)
479 480 return (DDI_FAILURE);
480 481
481 482 pdptr->par_nintr = n;
482 483 size = n * sizeof (struct intrspec);
483 484 new = pdptr->par_intr = kmem_zalloc(size, KM_SLEEP);
484 485
485 486 while (n--) {
486 487 int level = *in++;
487 488 int vec = *in++;
488 489
489 490 if (level < 1 || level > MAXIPL ||
490 491 vec < VEC_MIN || vec > VEC_MAX) {
491 492 cmn_err(CE_CONT, bad_intr_fmt,
492 493 DEVI(child)->devi_name,
493 494 DEVI(child)->devi_instance, level, vec);
494 495 goto broken;
495 496 }
496 497 new->intrspec_pri = level;
497 498 if (vec != 2)
498 499 new->intrspec_vec = vec;
499 500 else
500 501 /*
501 502 * irq 2 on the PC bus is tied to irq 9
502 503 * on ISA, EISA and MicroChannel
503 504 */
504 505 new->intrspec_vec = 9;
505 506 new++;
506 507 }
507 508
508 509 return (DDI_SUCCESS);
509 510 } else {
510 511 /* the new style "interrupts" property... */
511 512
512 513 /*
513 514 * The list consists of <vec> elements
514 515 */
515 516 if ((n = (*in++)) < 1)
516 517 return (DDI_FAILURE);
517 518
518 519 pdptr->par_nintr = n;
519 520 size = n * sizeof (struct intrspec);
520 521 new = pdptr->par_intr = kmem_zalloc(size, KM_SLEEP);
521 522
522 523 /* XXX check for "interrupt-priorities" property... */
523 524 if (ddi_getlongprop(DDI_DEV_T_ANY, child, DDI_PROP_DONTPASS,
524 525 "interrupt-priorities", (caddr_t)&got_prop, &got_len)
525 526 == DDI_PROP_SUCCESS) {
526 527 if (n != (got_len / sizeof (int))) {
527 528 cmn_err(CE_CONT,
528 529 "bad interrupt-priorities length"
529 530 " from %s%d: expected %d, got %d\n",
530 531 DEVI(child)->devi_name,
531 532 DEVI(child)->devi_instance, n,
532 533 (int)(got_len / sizeof (int)));
533 534 goto broken;
534 535 }
535 536 inpri = (int *)got_prop;
536 537 }
537 538
538 539 while (n--) {
539 540 int level;
540 541 int vec = *in++;
541 542
542 543 if (inpri == NULL)
543 544 level = 5;
544 545 else
545 546 level = *inpri++;
546 547
547 548 if (level < 1 || level > MAXIPL ||
548 549 vec < VEC_MIN || vec > VEC_MAX) {
549 550 cmn_err(CE_CONT, bad_intr_fmt,
550 551 DEVI(child)->devi_name,
551 552 DEVI(child)->devi_instance, level, vec);
552 553 goto broken;
553 554 }
554 555 new->intrspec_pri = level;
555 556 if (vec != 2)
556 557 new->intrspec_vec = vec;
557 558 else
558 559 /*
559 560 * irq 2 on the PC bus is tied to irq 9
560 561 * on ISA, EISA and MicroChannel
561 562 */
562 563 new->intrspec_vec = 9;
563 564 new++;
564 565 }
565 566
566 567 if (inpri != NULL)
567 568 kmem_free(got_prop, got_len);
568 569 return (DDI_SUCCESS);
569 570 }
570 571
571 572 broken:
572 573 kmem_free(pdptr->par_intr, size);
573 574 pdptr->par_intr = NULL;
574 575 pdptr->par_nintr = 0;
575 576 if (inpri != NULL)
576 577 kmem_free(got_prop, got_len);
577 578
578 579 return (DDI_FAILURE);
579 580 }
580 581
581 582 /*
582 583 * Create a ddi_parent_private_data structure from the ddi properties of
583 584 * the dev_info node.
584 585 *
585 586 * The "reg" and either an "intr" or "interrupts" properties are required
586 587 * if the driver wishes to create mappings or field interrupts on behalf
587 588 * of the device.
588 589 *
589 590 * The "reg" property is assumed to be a list of at least one triple
590 591 *
591 592 * <bustype, address, size>*1
592 593 *
593 594 * The "intr" property is assumed to be a list of at least one duple
594 595 *
595 596 * <SPARC ipl, vector#>*1
596 597 *
597 598 * The "interrupts" property is assumed to be a list of at least one
598 599 * n-tuples that describes the interrupt capabilities of the bus the device
599 600 * is connected to. For SBus, this looks like
600 601 *
601 602 * <SBus-level>*1
602 603 *
603 604 * (This property obsoletes the 'intr' property).
604 605 *
605 606 * The "ranges" property is optional.
606 607 */
607 608 void
608 609 make_ddi_ppd(dev_info_t *child, struct ddi_parent_private_data **ppd)
609 610 {
610 611 struct ddi_parent_private_data *pdptr;
611 612 int n;
612 613 int *reg_prop, *rng_prop, *intr_prop, *irupts_prop;
613 614 uint_t reg_len, rng_len, intr_len, irupts_len;
614 615
615 616 *ppd = pdptr = kmem_zalloc(sizeof (*pdptr), KM_SLEEP);
616 617
617 618 /*
618 619 * Handle the 'reg' property.
619 620 */
620 621 if ((get_prop_int_array(child, "reg", ®_prop, ®_len) ==
621 622 DDI_PROP_SUCCESS) && (reg_len != 0)) {
622 623 pdptr->par_nreg = reg_len / (int)sizeof (struct regspec);
623 624 pdptr->par_reg = (struct regspec *)reg_prop;
624 625 }
625 626
626 627 /*
627 628 * See if I have a range (adding one where needed - this
628 629 * means to add one for sbus node in sun4c, when romvec > 0,
629 630 * if no range is already defined in the PROM node.
630 631 * (Currently no sun4c PROMS define range properties,
631 632 * but they should and may in the future.) For the SBus
632 633 * node, the range is defined by the SBus reg property.
633 634 */
634 635 if (get_prop_int_array(child, "ranges", &rng_prop, &rng_len)
635 636 == DDI_PROP_SUCCESS) {
636 637 pdptr->par_nrng = rng_len / (int)(sizeof (struct rangespec));
637 638 pdptr->par_rng = (struct rangespec *)rng_prop;
638 639 }
639 640
640 641 /*
641 642 * Handle the 'intr' and 'interrupts' properties
642 643 */
643 644
644 645 /*
645 646 * For backwards compatibility
646 647 * we first look for the 'intr' property for the device.
647 648 */
648 649 if (get_prop_int_array(child, "intr", &intr_prop, &intr_len)
649 650 != DDI_PROP_SUCCESS) {
650 651 intr_len = 0;
651 652 }
652 653
653 654 /*
654 655 * If we're to support bus adapters and future platforms cleanly,
655 656 * we need to support the generalized 'interrupts' property.
656 657 */
657 658 if (get_prop_int_array(child, "interrupts", &irupts_prop,
658 659 &irupts_len) != DDI_PROP_SUCCESS) {
659 660 irupts_len = 0;
660 661 } else if (intr_len != 0) {
661 662 /*
662 663 * If both 'intr' and 'interrupts' are defined,
663 664 * then 'interrupts' wins and we toss the 'intr' away.
664 665 */
665 666 ddi_prop_free((void *)intr_prop);
666 667 intr_len = 0;
667 668 }
668 669
669 670 if (intr_len != 0) {
670 671
671 672 /*
672 673 * Translate the 'intr' property into an array
673 674 * an array of struct intrspec's. There's not really
674 675 * very much to do here except copy what's out there.
675 676 */
676 677
677 678 struct intrspec *new;
678 679 struct prop_ispec *l;
679 680
680 681 n = pdptr->par_nintr = intr_len / sizeof (struct prop_ispec);
681 682 l = (struct prop_ispec *)intr_prop;
682 683 pdptr->par_intr =
683 684 new = kmem_zalloc(n * sizeof (struct intrspec), KM_SLEEP);
684 685 while (n--) {
685 686 new->intrspec_pri = l->pri;
686 687 new->intrspec_vec = l->vec;
687 688 new++;
688 689 l++;
689 690 }
690 691 ddi_prop_free((void *)intr_prop);
691 692
692 693 } else if ((n = irupts_len) != 0) {
693 694 size_t size;
694 695 int *out;
695 696
696 697 /*
697 698 * Translate the 'interrupts' property into an array
698 699 * of intrspecs for the rest of the DDI framework to
699 700 * toy with. Only our ancestors really know how to
700 701 * do this, so ask 'em. We massage the 'interrupts'
701 702 * property so that it is pre-pended by a count of
702 703 * the number of integers in the argument.
703 704 */
704 705 size = sizeof (int) + n;
705 706 out = kmem_alloc(size, KM_SLEEP);
706 707 *out = n / sizeof (int);
707 708 bcopy(irupts_prop, out + 1, (size_t)n);
708 709 ddi_prop_free((void *)irupts_prop);
709 710 if (impl_xlate_intrs(child, out, pdptr) != DDI_SUCCESS) {
710 711 cmn_err(CE_CONT,
711 712 "Unable to translate 'interrupts' for %s%d\n",
712 713 DEVI(child)->devi_binding_name,
713 714 DEVI(child)->devi_instance);
714 715 }
715 716 kmem_free(out, size);
716 717 }
717 718 }
718 719
719 720 /*
720 721 * Name a child
721 722 */
722 723 static int
723 724 impl_sunbus_name_child(dev_info_t *child, char *name, int namelen)
724 725 {
725 726 /*
726 727 * Fill in parent-private data and this function returns to us
727 728 * an indication if it used "registers" to fill in the data.
728 729 */
729 730 if (ddi_get_parent_data(child) == NULL) {
730 731 struct ddi_parent_private_data *pdptr;
731 732 make_ddi_ppd(child, &pdptr);
732 733 ddi_set_parent_data(child, pdptr);
733 734 }
734 735
735 736 name[0] = '\0';
736 737 if (sparc_pd_getnreg(child) > 0) {
737 738 (void) snprintf(name, namelen, "%x,%x",
738 739 (uint_t)sparc_pd_getreg(child, 0)->regspec_bustype,
739 740 (uint_t)sparc_pd_getreg(child, 0)->regspec_addr);
740 741 }
741 742
742 743 return (DDI_SUCCESS);
743 744 }
744 745
745 746 /*
746 747 * Called from the bus_ctl op of sunbus (sbus, obio, etc) nexus drivers
747 748 * to implement the DDI_CTLOPS_INITCHILD operation. That is, it names
748 749 * the children of sun busses based on the reg spec.
749 750 *
750 751 * Handles the following properties (in make_ddi_ppd):
751 752 * Property value
752 753 * Name type
753 754 * reg register spec
754 755 * intr old-form interrupt spec
755 756 * interrupts new (bus-oriented) interrupt spec
756 757 * ranges range spec
757 758 */
758 759 int
759 760 impl_ddi_sunbus_initchild(dev_info_t *child)
760 761 {
761 762 char name[MAXNAMELEN];
762 763 void impl_ddi_sunbus_removechild(dev_info_t *);
763 764
764 765 /*
765 766 * Name the child, also makes parent private data
766 767 */
767 768 (void) impl_sunbus_name_child(child, name, MAXNAMELEN);
768 769 ddi_set_name_addr(child, name);
769 770
770 771 /*
771 772 * Attempt to merge a .conf node; if successful, remove the
772 773 * .conf node.
773 774 */
774 775 if ((ndi_dev_is_persistent_node(child) == 0) &&
775 776 (ndi_merge_node(child, impl_sunbus_name_child) == DDI_SUCCESS)) {
776 777 /*
777 778 * Return failure to remove node
778 779 */
779 780 impl_ddi_sunbus_removechild(child);
780 781 return (DDI_FAILURE);
781 782 }
782 783 return (DDI_SUCCESS);
783 784 }
784 785
785 786 void
786 787 impl_free_ddi_ppd(dev_info_t *dip)
787 788 {
788 789 struct ddi_parent_private_data *pdptr;
789 790 size_t n;
790 791
791 792 if ((pdptr = ddi_get_parent_data(dip)) == NULL)
792 793 return;
793 794
794 795 if ((n = (size_t)pdptr->par_nintr) != 0)
795 796 /*
796 797 * Note that kmem_free is used here (instead of
797 798 * ddi_prop_free) because the contents of the
798 799 * property were placed into a separate buffer and
799 800 * mucked with a bit before being stored in par_intr.
800 801 * The actual return value from the prop lookup
801 802 * was freed with ddi_prop_free previously.
802 803 */
803 804 kmem_free(pdptr->par_intr, n * sizeof (struct intrspec));
804 805
805 806 if ((n = (size_t)pdptr->par_nrng) != 0)
806 807 ddi_prop_free((void *)pdptr->par_rng);
807 808
808 809 if ((n = pdptr->par_nreg) != 0)
809 810 ddi_prop_free((void *)pdptr->par_reg);
810 811
811 812 kmem_free(pdptr, sizeof (*pdptr));
812 813 ddi_set_parent_data(dip, NULL);
813 814 }
814 815
815 816 void
816 817 impl_ddi_sunbus_removechild(dev_info_t *dip)
817 818 {
818 819 impl_free_ddi_ppd(dip);
819 820 ddi_set_name_addr(dip, NULL);
820 821 /*
821 822 * Strip the node to properly convert it back to prototype form
822 823 */
823 824 impl_rem_dev_props(dip);
824 825 }
825 826
826 827 /*
827 828 * DDI Interrupt
828 829 */
829 830
830 831 /*
831 832 * turn this on to force isa, eisa, and mca device to ignore the new
832 833 * hardware nodes in the device tree (normally turned on only for
833 834 * drivers that need it by setting the property "ignore-hardware-nodes"
834 835 * in their driver.conf file).
835 836 *
836 837 * 7/31/96 -- Turned off globally. Leaving variable in for the moment
837 838 * as safety valve.
838 839 */
839 840 int ignore_hardware_nodes = 0;
840 841
841 842 /*
842 843 * Local data
843 844 */
844 845 static struct impl_bus_promops *impl_busp;
845 846
846 847
847 848 /*
848 849 * New DDI interrupt framework
849 850 */
850 851
851 852 /*
852 853 * i_ddi_intr_ops:
853 854 *
854 855 * This is the interrupt operator function wrapper for the bus function
855 856 * bus_intr_op.
856 857 */
857 858 int
858 859 i_ddi_intr_ops(dev_info_t *dip, dev_info_t *rdip, ddi_intr_op_t op,
859 860 ddi_intr_handle_impl_t *hdlp, void * result)
860 861 {
861 862 dev_info_t *pdip = (dev_info_t *)DEVI(dip)->devi_parent;
862 863 int ret = DDI_FAILURE;
863 864
864 865 /* request parent to process this interrupt op */
865 866 if (NEXUS_HAS_INTR_OP(pdip))
866 867 ret = (*(DEVI(pdip)->devi_ops->devo_bus_ops->bus_intr_op))(
867 868 pdip, rdip, op, hdlp, result);
868 869 else
869 870 cmn_err(CE_WARN, "Failed to process interrupt "
870 871 "for %s%d due to down-rev nexus driver %s%d",
871 872 ddi_get_name(rdip), ddi_get_instance(rdip),
872 873 ddi_get_name(pdip), ddi_get_instance(pdip));
873 874 return (ret);
874 875 }
875 876
876 877 /*
877 878 * i_ddi_add_softint - allocate and add a soft interrupt to the system
878 879 */
879 880 int
880 881 i_ddi_add_softint(ddi_softint_hdl_impl_t *hdlp)
881 882 {
882 883 int ret;
883 884
884 885 /* add soft interrupt handler */
885 886 ret = add_avsoftintr((void *)hdlp, hdlp->ih_pri, hdlp->ih_cb_func,
886 887 DEVI(hdlp->ih_dip)->devi_name, hdlp->ih_cb_arg1, hdlp->ih_cb_arg2);
887 888 return (ret ? DDI_SUCCESS : DDI_FAILURE);
888 889 }
889 890
890 891
891 892 void
892 893 i_ddi_remove_softint(ddi_softint_hdl_impl_t *hdlp)
893 894 {
894 895 (void) rem_avsoftintr((void *)hdlp, hdlp->ih_pri, hdlp->ih_cb_func);
895 896 }
896 897
897 898
898 899 extern void (*setsoftint)(int, struct av_softinfo *);
899 900 extern boolean_t av_check_softint_pending(struct av_softinfo *, boolean_t);
900 901
901 902 int
902 903 i_ddi_trigger_softint(ddi_softint_hdl_impl_t *hdlp, void *arg2)
903 904 {
904 905 if (av_check_softint_pending(hdlp->ih_pending, B_FALSE))
905 906 return (DDI_EPENDING);
906 907
907 908 update_avsoftintr_args((void *)hdlp, hdlp->ih_pri, arg2);
908 909
909 910 (*setsoftint)(hdlp->ih_pri, hdlp->ih_pending);
910 911 return (DDI_SUCCESS);
911 912 }
912 913
913 914 /*
914 915 * i_ddi_set_softint_pri:
915 916 *
916 917 * The way this works is that it first tries to add a softint vector
917 918 * at the new priority in hdlp. If that succeeds; then it removes the
918 919 * existing softint vector at the old priority.
919 920 */
920 921 int
921 922 i_ddi_set_softint_pri(ddi_softint_hdl_impl_t *hdlp, uint_t old_pri)
922 923 {
923 924 int ret;
924 925
925 926 /*
926 927 * If a softint is pending at the old priority then fail the request.
927 928 */
928 929 if (av_check_softint_pending(hdlp->ih_pending, B_TRUE))
929 930 return (DDI_FAILURE);
930 931
931 932 ret = av_softint_movepri((void *)hdlp, old_pri);
932 933 return (ret ? DDI_SUCCESS : DDI_FAILURE);
933 934 }
934 935
935 936 void
936 937 i_ddi_alloc_intr_phdl(ddi_intr_handle_impl_t *hdlp)
937 938 {
938 939 hdlp->ih_private = (void *)kmem_zalloc(sizeof (ihdl_plat_t), KM_SLEEP);
939 940 }
940 941
941 942 void
942 943 i_ddi_free_intr_phdl(ddi_intr_handle_impl_t *hdlp)
943 944 {
944 945 kmem_free(hdlp->ih_private, sizeof (ihdl_plat_t));
945 946 hdlp->ih_private = NULL;
946 947 }
947 948
948 949 int
949 950 i_ddi_get_intx_nintrs(dev_info_t *dip)
950 951 {
951 952 struct ddi_parent_private_data *pdp;
952 953
953 954 if ((pdp = ddi_get_parent_data(dip)) == NULL)
954 955 return (0);
955 956
956 957 return (pdp->par_nintr);
957 958 }
958 959
959 960 /*
960 961 * DDI Memory/DMA
961 962 */
962 963
963 964 /*
964 965 * Support for allocating DMAable memory to implement
965 966 * ddi_dma_mem_alloc(9F) interface.
966 967 */
967 968
968 969 #define KA_ALIGN_SHIFT 7
969 970 #define KA_ALIGN (1 << KA_ALIGN_SHIFT)
970 971 #define KA_NCACHE (PAGESHIFT + 1 - KA_ALIGN_SHIFT)
971 972
972 973 /*
973 974 * Dummy DMA attribute template for kmem_io[].kmem_io_attr. We only
974 975 * care about addr_lo, addr_hi, and align. addr_hi will be dynamically set.
975 976 */
976 977
977 978 static ddi_dma_attr_t kmem_io_attr = {
978 979 DMA_ATTR_V0,
979 980 0x0000000000000000ULL, /* dma_attr_addr_lo */
980 981 0x0000000000000000ULL, /* dma_attr_addr_hi */
981 982 0x00ffffff,
982 983 0x1000, /* dma_attr_align */
983 984 1, 1, 0xffffffffULL, 0xffffffffULL, 0x1, 1, 0
984 985 };
985 986
986 987 /* kmem io memory ranges and indices */
987 988 enum {
988 989 IO_4P, IO_64G, IO_4G, IO_2G, IO_1G, IO_512M,
989 990 IO_256M, IO_128M, IO_64M, IO_32M, IO_16M, MAX_MEM_RANGES
990 991 };
991 992
992 993 static struct {
993 994 vmem_t *kmem_io_arena;
994 995 kmem_cache_t *kmem_io_cache[KA_NCACHE];
995 996 ddi_dma_attr_t kmem_io_attr;
996 997 } kmem_io[MAX_MEM_RANGES];
997 998
998 999 static int kmem_io_idx; /* index of first populated kmem_io[] */
999 1000
1000 1001 static page_t *
1001 1002 page_create_io_wrapper(void *addr, size_t len, int vmflag, void *arg)
1002 1003 {
1003 1004 extern page_t *page_create_io(vnode_t *, u_offset_t, uint_t,
1004 1005 uint_t, struct as *, caddr_t, ddi_dma_attr_t *);
1005 1006
1006 1007 return (page_create_io(&kvp, (u_offset_t)(uintptr_t)addr, len,
1007 1008 PG_EXCL | ((vmflag & VM_NOSLEEP) ? 0 : PG_WAIT), &kas, addr, arg));
1008 1009 }
1009 1010
1010 1011 #ifdef __xpv
1011 1012 static void
1012 1013 segkmem_free_io(vmem_t *vmp, void * ptr, size_t size)
1013 1014 {
1014 1015 extern void page_destroy_io(page_t *);
1015 1016 segkmem_xfree(vmp, ptr, size, page_destroy_io);
1016 1017 }
1017 1018 #endif
1018 1019
1019 1020 static void *
1020 1021 segkmem_alloc_io_4P(vmem_t *vmp, size_t size, int vmflag)
1021 1022 {
1022 1023 return (segkmem_xalloc(vmp, NULL, size, vmflag, 0,
1023 1024 page_create_io_wrapper, &kmem_io[IO_4P].kmem_io_attr));
1024 1025 }
1025 1026
1026 1027 static void *
1027 1028 segkmem_alloc_io_64G(vmem_t *vmp, size_t size, int vmflag)
1028 1029 {
1029 1030 return (segkmem_xalloc(vmp, NULL, size, vmflag, 0,
1030 1031 page_create_io_wrapper, &kmem_io[IO_64G].kmem_io_attr));
1031 1032 }
1032 1033
1033 1034 static void *
1034 1035 segkmem_alloc_io_4G(vmem_t *vmp, size_t size, int vmflag)
1035 1036 {
1036 1037 return (segkmem_xalloc(vmp, NULL, size, vmflag, 0,
1037 1038 page_create_io_wrapper, &kmem_io[IO_4G].kmem_io_attr));
1038 1039 }
1039 1040
1040 1041 static void *
1041 1042 segkmem_alloc_io_2G(vmem_t *vmp, size_t size, int vmflag)
1042 1043 {
1043 1044 return (segkmem_xalloc(vmp, NULL, size, vmflag, 0,
1044 1045 page_create_io_wrapper, &kmem_io[IO_2G].kmem_io_attr));
1045 1046 }
1046 1047
1047 1048 static void *
1048 1049 segkmem_alloc_io_1G(vmem_t *vmp, size_t size, int vmflag)
1049 1050 {
1050 1051 return (segkmem_xalloc(vmp, NULL, size, vmflag, 0,
1051 1052 page_create_io_wrapper, &kmem_io[IO_1G].kmem_io_attr));
1052 1053 }
1053 1054
1054 1055 static void *
1055 1056 segkmem_alloc_io_512M(vmem_t *vmp, size_t size, int vmflag)
1056 1057 {
1057 1058 return (segkmem_xalloc(vmp, NULL, size, vmflag, 0,
1058 1059 page_create_io_wrapper, &kmem_io[IO_512M].kmem_io_attr));
1059 1060 }
1060 1061
1061 1062 static void *
1062 1063 segkmem_alloc_io_256M(vmem_t *vmp, size_t size, int vmflag)
1063 1064 {
1064 1065 return (segkmem_xalloc(vmp, NULL, size, vmflag, 0,
1065 1066 page_create_io_wrapper, &kmem_io[IO_256M].kmem_io_attr));
1066 1067 }
1067 1068
1068 1069 static void *
1069 1070 segkmem_alloc_io_128M(vmem_t *vmp, size_t size, int vmflag)
1070 1071 {
1071 1072 return (segkmem_xalloc(vmp, NULL, size, vmflag, 0,
1072 1073 page_create_io_wrapper, &kmem_io[IO_128M].kmem_io_attr));
1073 1074 }
1074 1075
1075 1076 static void *
1076 1077 segkmem_alloc_io_64M(vmem_t *vmp, size_t size, int vmflag)
1077 1078 {
1078 1079 return (segkmem_xalloc(vmp, NULL, size, vmflag, 0,
1079 1080 page_create_io_wrapper, &kmem_io[IO_64M].kmem_io_attr));
1080 1081 }
1081 1082
1082 1083 static void *
1083 1084 segkmem_alloc_io_32M(vmem_t *vmp, size_t size, int vmflag)
1084 1085 {
1085 1086 return (segkmem_xalloc(vmp, NULL, size, vmflag, 0,
1086 1087 page_create_io_wrapper, &kmem_io[IO_32M].kmem_io_attr));
1087 1088 }
1088 1089
1089 1090 static void *
1090 1091 segkmem_alloc_io_16M(vmem_t *vmp, size_t size, int vmflag)
1091 1092 {
1092 1093 return (segkmem_xalloc(vmp, NULL, size, vmflag, 0,
1093 1094 page_create_io_wrapper, &kmem_io[IO_16M].kmem_io_attr));
1094 1095 }
1095 1096
1096 1097 struct {
1097 1098 uint64_t io_limit;
1098 1099 char *io_name;
1099 1100 void *(*io_alloc)(vmem_t *, size_t, int);
1100 1101 int io_initial; /* kmem_io_init during startup */
1101 1102 } io_arena_params[MAX_MEM_RANGES] = {
1102 1103 {0x000fffffffffffffULL, "kmem_io_4P", segkmem_alloc_io_4P, 1},
1103 1104 {0x0000000fffffffffULL, "kmem_io_64G", segkmem_alloc_io_64G, 0},
1104 1105 {0x00000000ffffffffULL, "kmem_io_4G", segkmem_alloc_io_4G, 1},
1105 1106 {0x000000007fffffffULL, "kmem_io_2G", segkmem_alloc_io_2G, 1},
1106 1107 {0x000000003fffffffULL, "kmem_io_1G", segkmem_alloc_io_1G, 0},
1107 1108 {0x000000001fffffffULL, "kmem_io_512M", segkmem_alloc_io_512M, 0},
1108 1109 {0x000000000fffffffULL, "kmem_io_256M", segkmem_alloc_io_256M, 0},
1109 1110 {0x0000000007ffffffULL, "kmem_io_128M", segkmem_alloc_io_128M, 0},
1110 1111 {0x0000000003ffffffULL, "kmem_io_64M", segkmem_alloc_io_64M, 0},
1111 1112 {0x0000000001ffffffULL, "kmem_io_32M", segkmem_alloc_io_32M, 0},
1112 1113 {0x0000000000ffffffULL, "kmem_io_16M", segkmem_alloc_io_16M, 1}
1113 1114 };
1114 1115
1115 1116 void
1116 1117 kmem_io_init(int a)
1117 1118 {
1118 1119 int c;
1119 1120 char name[40];
1120 1121
1121 1122 kmem_io[a].kmem_io_arena = vmem_create(io_arena_params[a].io_name,
1122 1123 NULL, 0, PAGESIZE, io_arena_params[a].io_alloc,
1123 1124 #ifdef __xpv
1124 1125 segkmem_free_io,
1125 1126 #else
1126 1127 segkmem_free,
1127 1128 #endif
1128 1129 heap_arena, 0, VM_SLEEP);
1129 1130
1130 1131 for (c = 0; c < KA_NCACHE; c++) {
1131 1132 size_t size = KA_ALIGN << c;
1132 1133 (void) sprintf(name, "%s_%lu",
1133 1134 io_arena_params[a].io_name, size);
1134 1135 kmem_io[a].kmem_io_cache[c] = kmem_cache_create(name,
1135 1136 size, size, NULL, NULL, NULL, NULL,
1136 1137 kmem_io[a].kmem_io_arena, 0);
1137 1138 }
1138 1139 }
1139 1140
1140 1141 /*
1141 1142 * Return the index of the highest memory range for addr.
1142 1143 */
1143 1144 static int
1144 1145 kmem_io_index(uint64_t addr)
1145 1146 {
1146 1147 int n;
1147 1148
1148 1149 for (n = kmem_io_idx; n < MAX_MEM_RANGES; n++) {
1149 1150 if (kmem_io[n].kmem_io_attr.dma_attr_addr_hi <= addr) {
1150 1151 if (kmem_io[n].kmem_io_arena == NULL)
1151 1152 kmem_io_init(n);
1152 1153 return (n);
1153 1154 }
1154 1155 }
1155 1156 panic("kmem_io_index: invalid addr - must be at least 16m");
1156 1157
1157 1158 /*NOTREACHED*/
1158 1159 }
1159 1160
1160 1161 /*
1161 1162 * Return the index of the next kmem_io populated memory range
1162 1163 * after curindex.
1163 1164 */
1164 1165 static int
1165 1166 kmem_io_index_next(int curindex)
1166 1167 {
1167 1168 int n;
1168 1169
1169 1170 for (n = curindex + 1; n < MAX_MEM_RANGES; n++) {
1170 1171 if (kmem_io[n].kmem_io_arena)
1171 1172 return (n);
1172 1173 }
1173 1174 return (-1);
1174 1175 }
1175 1176
1176 1177 /*
1177 1178 * allow kmem to be mapped in with different PTE cache attribute settings.
1178 1179 * Used by i_ddi_mem_alloc()
1179 1180 */
1180 1181 int
1181 1182 kmem_override_cache_attrs(caddr_t kva, size_t size, uint_t order)
1182 1183 {
1183 1184 uint_t hat_flags;
1184 1185 caddr_t kva_end;
1185 1186 uint_t hat_attr;
1186 1187 pfn_t pfn;
1187 1188
1188 1189 if (hat_getattr(kas.a_hat, kva, &hat_attr) == -1) {
1189 1190 return (-1);
1190 1191 }
1191 1192
1192 1193 hat_attr &= ~HAT_ORDER_MASK;
1193 1194 hat_attr |= order | HAT_NOSYNC;
1194 1195 hat_flags = HAT_LOAD_LOCK;
1195 1196
1196 1197 kva_end = (caddr_t)(((uintptr_t)kva + size + PAGEOFFSET) &
1197 1198 (uintptr_t)PAGEMASK);
1198 1199 kva = (caddr_t)((uintptr_t)kva & (uintptr_t)PAGEMASK);
1199 1200
1200 1201 while (kva < kva_end) {
1201 1202 pfn = hat_getpfnum(kas.a_hat, kva);
1202 1203 hat_unload(kas.a_hat, kva, PAGESIZE, HAT_UNLOAD_UNLOCK);
1203 1204 hat_devload(kas.a_hat, kva, PAGESIZE, pfn, hat_attr, hat_flags);
1204 1205 kva += MMU_PAGESIZE;
1205 1206 }
1206 1207
1207 1208 return (0);
1208 1209 }
1209 1210
1210 1211 static int
1211 1212 ctgcompare(const void *a1, const void *a2)
1212 1213 {
1213 1214 /* we just want to compare virtual addresses */
1214 1215 a1 = ((struct ctgas *)a1)->ctg_addr;
1215 1216 a2 = ((struct ctgas *)a2)->ctg_addr;
1216 1217 return (a1 == a2 ? 0 : (a1 < a2 ? -1 : 1));
1217 1218 }
1218 1219
1219 1220 void
1220 1221 ka_init(void)
1221 1222 {
1222 1223 int a;
1223 1224 paddr_t maxphysaddr;
1224 1225 #if !defined(__xpv)
1225 1226 extern pfn_t physmax;
1226 1227
1227 1228 maxphysaddr = mmu_ptob((paddr_t)physmax) + MMU_PAGEOFFSET;
1228 1229 #else
1229 1230 maxphysaddr = mmu_ptob((paddr_t)HYPERVISOR_memory_op(
1230 1231 XENMEM_maximum_ram_page, NULL)) + MMU_PAGEOFFSET;
1231 1232 #endif
1232 1233
1233 1234 ASSERT(maxphysaddr <= io_arena_params[0].io_limit);
1234 1235
1235 1236 for (a = 0; a < MAX_MEM_RANGES; a++) {
1236 1237 if (maxphysaddr >= io_arena_params[a + 1].io_limit) {
1237 1238 if (maxphysaddr > io_arena_params[a + 1].io_limit)
1238 1239 io_arena_params[a].io_limit = maxphysaddr;
1239 1240 else
1240 1241 a++;
1241 1242 break;
1242 1243 }
1243 1244 }
1244 1245 kmem_io_idx = a;
1245 1246
1246 1247 for (; a < MAX_MEM_RANGES; a++) {
1247 1248 kmem_io[a].kmem_io_attr = kmem_io_attr;
1248 1249 kmem_io[a].kmem_io_attr.dma_attr_addr_hi =
1249 1250 io_arena_params[a].io_limit;
1250 1251 /*
1251 1252 * initialize kmem_io[] arena/cache corresponding to
1252 1253 * maxphysaddr and to the "common" io memory ranges that
1253 1254 * have io_initial set to a non-zero value.
1254 1255 */
1255 1256 if (io_arena_params[a].io_initial || a == kmem_io_idx)
1256 1257 kmem_io_init(a);
1257 1258 }
1258 1259
1259 1260 /* initialize ctgtree */
1260 1261 avl_create(&ctgtree, ctgcompare, sizeof (struct ctgas),
1261 1262 offsetof(struct ctgas, ctg_link));
1262 1263 }
1263 1264
1264 1265 /*
1265 1266 * put contig address/size
1266 1267 */
1267 1268 static void *
1268 1269 putctgas(void *addr, size_t size)
1269 1270 {
1270 1271 struct ctgas *ctgp;
1271 1272 if ((ctgp = kmem_zalloc(sizeof (*ctgp), KM_NOSLEEP)) != NULL) {
1272 1273 ctgp->ctg_addr = addr;
1273 1274 ctgp->ctg_size = size;
1274 1275 CTGLOCK();
1275 1276 avl_add(&ctgtree, ctgp);
1276 1277 CTGUNLOCK();
1277 1278 }
1278 1279 return (ctgp);
1279 1280 }
1280 1281
1281 1282 /*
1282 1283 * get contig size by addr
1283 1284 */
1284 1285 static size_t
1285 1286 getctgsz(void *addr)
1286 1287 {
1287 1288 struct ctgas *ctgp;
1288 1289 struct ctgas find;
1289 1290 size_t sz = 0;
1290 1291
1291 1292 find.ctg_addr = addr;
1292 1293 CTGLOCK();
1293 1294 if ((ctgp = avl_find(&ctgtree, &find, NULL)) != NULL) {
1294 1295 avl_remove(&ctgtree, ctgp);
1295 1296 }
1296 1297 CTGUNLOCK();
1297 1298
1298 1299 if (ctgp != NULL) {
1299 1300 sz = ctgp->ctg_size;
1300 1301 kmem_free(ctgp, sizeof (*ctgp));
1301 1302 }
1302 1303
1303 1304 return (sz);
1304 1305 }
1305 1306
1306 1307 /*
1307 1308 * contig_alloc:
1308 1309 *
1309 1310 * allocates contiguous memory to satisfy the 'size' and dma attributes
1310 1311 * specified in 'attr'.
1311 1312 *
1312 1313 * Not all of memory need to be physically contiguous if the
1313 1314 * scatter-gather list length is greater than 1.
1314 1315 */
1315 1316
1316 1317 /*ARGSUSED*/
1317 1318 void *
1318 1319 contig_alloc(size_t size, ddi_dma_attr_t *attr, uintptr_t align, int cansleep)
1319 1320 {
1320 1321 pgcnt_t pgcnt = btopr(size);
1321 1322 size_t asize = pgcnt * PAGESIZE;
1322 1323 page_t *ppl;
1323 1324 int pflag;
1324 1325 void *addr;
1325 1326
1326 1327 extern page_t *page_create_io(vnode_t *, u_offset_t, uint_t,
1327 1328 uint_t, struct as *, caddr_t, ddi_dma_attr_t *);
1328 1329
1329 1330 /* segkmem_xalloc */
1330 1331
1331 1332 if (align <= PAGESIZE)
1332 1333 addr = vmem_alloc(heap_arena, asize,
1333 1334 (cansleep) ? VM_SLEEP : VM_NOSLEEP);
1334 1335 else
1335 1336 addr = vmem_xalloc(heap_arena, asize, align, 0, 0, NULL, NULL,
1336 1337 (cansleep) ? VM_SLEEP : VM_NOSLEEP);
1337 1338 if (addr) {
1338 1339 ASSERT(!((uintptr_t)addr & (align - 1)));
1339 1340
1340 1341 if (page_resv(pgcnt, (cansleep) ? KM_SLEEP : KM_NOSLEEP) == 0) {
1341 1342 vmem_free(heap_arena, addr, asize);
1342 1343 return (NULL);
1343 1344 }
1344 1345 pflag = PG_EXCL;
1345 1346
1346 1347 if (cansleep)
1347 1348 pflag |= PG_WAIT;
1348 1349
1349 1350 /* 4k req gets from freelists rather than pfn search */
1350 1351 if (pgcnt > 1 || align > PAGESIZE)
1351 1352 pflag |= PG_PHYSCONTIG;
1352 1353
1353 1354 ppl = page_create_io(&kvp, (u_offset_t)(uintptr_t)addr,
1354 1355 asize, pflag, &kas, (caddr_t)addr, attr);
1355 1356
1356 1357 if (!ppl) {
1357 1358 vmem_free(heap_arena, addr, asize);
1358 1359 page_unresv(pgcnt);
1359 1360 return (NULL);
1360 1361 }
1361 1362
1362 1363 while (ppl != NULL) {
1363 1364 page_t *pp = ppl;
1364 1365 page_sub(&ppl, pp);
1365 1366 ASSERT(page_iolock_assert(pp));
1366 1367 page_io_unlock(pp);
1367 1368 page_downgrade(pp);
1368 1369 hat_memload(kas.a_hat, (caddr_t)(uintptr_t)pp->p_offset,
1369 1370 pp, (PROT_ALL & ~PROT_USER) |
1370 1371 HAT_NOSYNC, HAT_LOAD_LOCK);
1371 1372 }
1372 1373 }
1373 1374 return (addr);
1374 1375 }
1375 1376
1376 1377 void
1377 1378 contig_free(void *addr, size_t size)
1378 1379 {
1379 1380 pgcnt_t pgcnt = btopr(size);
1380 1381 size_t asize = pgcnt * PAGESIZE;
1381 1382 caddr_t a, ea;
1382 1383 page_t *pp;
1383 1384
1384 1385 hat_unload(kas.a_hat, addr, asize, HAT_UNLOAD_UNLOCK);
1385 1386
1386 1387 for (a = addr, ea = a + asize; a < ea; a += PAGESIZE) {
1387 1388 pp = page_find(&kvp, (u_offset_t)(uintptr_t)a);
1388 1389 if (!pp)
1389 1390 panic("contig_free: contig pp not found");
1390 1391
1391 1392 if (!page_tryupgrade(pp)) {
1392 1393 page_unlock(pp);
1393 1394 pp = page_lookup(&kvp,
1394 1395 (u_offset_t)(uintptr_t)a, SE_EXCL);
1395 1396 if (pp == NULL)
1396 1397 panic("contig_free: page freed");
1397 1398 }
1398 1399 page_destroy(pp, 0);
1399 1400 }
1400 1401
1401 1402 page_unresv(pgcnt);
1402 1403 vmem_free(heap_arena, addr, asize);
1403 1404 }
1404 1405
1405 1406 /*
1406 1407 * Allocate from the system, aligned on a specific boundary.
1407 1408 * The alignment, if non-zero, must be a power of 2.
1408 1409 */
1409 1410 static void *
1410 1411 kalloca(size_t size, size_t align, int cansleep, int physcontig,
1411 1412 ddi_dma_attr_t *attr)
1412 1413 {
1413 1414 size_t *addr, *raddr, rsize;
1414 1415 size_t hdrsize = 4 * sizeof (size_t); /* must be power of 2 */
1415 1416 int a, i, c;
1416 1417 vmem_t *vmp;
1417 1418 kmem_cache_t *cp = NULL;
1418 1419
1419 1420 if (attr->dma_attr_addr_lo > mmu_ptob((uint64_t)ddiphysmin))
1420 1421 return (NULL);
1421 1422
1422 1423 align = MAX(align, hdrsize);
1423 1424 ASSERT((align & (align - 1)) == 0);
1424 1425
1425 1426 /*
1426 1427 * All of our allocators guarantee 16-byte alignment, so we don't
1427 1428 * need to reserve additional space for the header.
1428 1429 * To simplify picking the correct kmem_io_cache, we round up to
1429 1430 * a multiple of KA_ALIGN.
1430 1431 */
1431 1432 rsize = P2ROUNDUP_TYPED(size + align, KA_ALIGN, size_t);
1432 1433
1433 1434 if (physcontig && rsize > PAGESIZE) {
1434 1435 if (addr = contig_alloc(size, attr, align, cansleep)) {
1435 1436 if (!putctgas(addr, size))
1436 1437 contig_free(addr, size);
1437 1438 else
1438 1439 return (addr);
1439 1440 }
1440 1441 return (NULL);
1441 1442 }
1442 1443
1443 1444 a = kmem_io_index(attr->dma_attr_addr_hi);
1444 1445
1445 1446 if (rsize > PAGESIZE) {
1446 1447 vmp = kmem_io[a].kmem_io_arena;
1447 1448 raddr = vmem_alloc(vmp, rsize,
1448 1449 (cansleep) ? VM_SLEEP : VM_NOSLEEP);
1449 1450 } else {
1450 1451 c = highbit((rsize >> KA_ALIGN_SHIFT) - 1);
1451 1452 cp = kmem_io[a].kmem_io_cache[c];
1452 1453 raddr = kmem_cache_alloc(cp, (cansleep) ? KM_SLEEP :
1453 1454 KM_NOSLEEP);
1454 1455 }
1455 1456
1456 1457 if (raddr == NULL) {
1457 1458 int na;
1458 1459
1459 1460 ASSERT(cansleep == 0);
1460 1461 if (rsize > PAGESIZE)
1461 1462 return (NULL);
1462 1463 /*
1463 1464 * System does not have memory in the requested range.
1464 1465 * Try smaller kmem io ranges and larger cache sizes
1465 1466 * to see if there might be memory available in
1466 1467 * these other caches.
1467 1468 */
1468 1469
1469 1470 for (na = kmem_io_index_next(a); na >= 0;
1470 1471 na = kmem_io_index_next(na)) {
1471 1472 ASSERT(kmem_io[na].kmem_io_arena);
1472 1473 cp = kmem_io[na].kmem_io_cache[c];
1473 1474 raddr = kmem_cache_alloc(cp, KM_NOSLEEP);
1474 1475 if (raddr)
1475 1476 goto kallocdone;
1476 1477 }
1477 1478 /* now try the larger kmem io cache sizes */
1478 1479 for (na = a; na >= 0; na = kmem_io_index_next(na)) {
1479 1480 for (i = c + 1; i < KA_NCACHE; i++) {
1480 1481 cp = kmem_io[na].kmem_io_cache[i];
1481 1482 raddr = kmem_cache_alloc(cp, KM_NOSLEEP);
1482 1483 if (raddr)
1483 1484 goto kallocdone;
1484 1485 }
1485 1486 }
1486 1487 return (NULL);
1487 1488 }
1488 1489
1489 1490 kallocdone:
1490 1491 ASSERT(!P2BOUNDARY((uintptr_t)raddr, rsize, PAGESIZE) ||
1491 1492 rsize > PAGESIZE);
1492 1493
1493 1494 addr = (size_t *)P2ROUNDUP((uintptr_t)raddr + hdrsize, align);
1494 1495 ASSERT((uintptr_t)addr + size - (uintptr_t)raddr <= rsize);
1495 1496
1496 1497 addr[-4] = (size_t)cp;
1497 1498 addr[-3] = (size_t)vmp;
1498 1499 addr[-2] = (size_t)raddr;
1499 1500 addr[-1] = rsize;
1500 1501
1501 1502 return (addr);
1502 1503 }
1503 1504
1504 1505 static void
1505 1506 kfreea(void *addr)
1506 1507 {
1507 1508 size_t size;
1508 1509
1509 1510 if (!((uintptr_t)addr & PAGEOFFSET) && (size = getctgsz(addr))) {
1510 1511 contig_free(addr, size);
1511 1512 } else {
1512 1513 size_t *saddr = addr;
1513 1514 if (saddr[-4] == 0)
1514 1515 vmem_free((vmem_t *)saddr[-3], (void *)saddr[-2],
1515 1516 saddr[-1]);
1516 1517 else
1517 1518 kmem_cache_free((kmem_cache_t *)saddr[-4],
1518 1519 (void *)saddr[-2]);
1519 1520 }
1520 1521 }
1521 1522
1522 1523 /*ARGSUSED*/
1523 1524 void
1524 1525 i_ddi_devacc_to_hatacc(ddi_device_acc_attr_t *devaccp, uint_t *hataccp)
1525 1526 {
1526 1527 }
1527 1528
1528 1529 /*
1529 1530 * Check if the specified cache attribute is supported on the platform.
1530 1531 * This function must be called before i_ddi_cacheattr_to_hatacc().
1531 1532 */
1532 1533 boolean_t
1533 1534 i_ddi_check_cache_attr(uint_t flags)
1534 1535 {
1535 1536 /*
1536 1537 * The cache attributes are mutually exclusive. Any combination of
1537 1538 * the attributes leads to a failure.
1538 1539 */
1539 1540 uint_t cache_attr = IOMEM_CACHE_ATTR(flags);
1540 1541 if ((cache_attr != 0) && !ISP2(cache_attr))
1541 1542 return (B_FALSE);
1542 1543
1543 1544 /* All cache attributes are supported on X86/X64 */
1544 1545 if (cache_attr & (IOMEM_DATA_UNCACHED | IOMEM_DATA_CACHED |
1545 1546 IOMEM_DATA_UC_WR_COMBINE))
1546 1547 return (B_TRUE);
1547 1548
1548 1549 /* undefined attributes */
1549 1550 return (B_FALSE);
1550 1551 }
1551 1552
1552 1553 /* set HAT cache attributes from the cache attributes */
1553 1554 void
1554 1555 i_ddi_cacheattr_to_hatacc(uint_t flags, uint_t *hataccp)
1555 1556 {
1556 1557 uint_t cache_attr = IOMEM_CACHE_ATTR(flags);
1557 1558 static char *fname = "i_ddi_cacheattr_to_hatacc";
1558 1559
1559 1560 /*
1560 1561 * If write-combining is not supported, then it falls back
1561 1562 * to uncacheable.
1562 1563 */
1563 1564 if (cache_attr == IOMEM_DATA_UC_WR_COMBINE &&
1564 1565 !is_x86_feature(x86_featureset, X86FSET_PAT))
1565 1566 cache_attr = IOMEM_DATA_UNCACHED;
1566 1567
1567 1568 /*
1568 1569 * set HAT attrs according to the cache attrs.
1569 1570 */
1570 1571 switch (cache_attr) {
1571 1572 case IOMEM_DATA_UNCACHED:
1572 1573 *hataccp &= ~HAT_ORDER_MASK;
1573 1574 *hataccp |= (HAT_STRICTORDER | HAT_PLAT_NOCACHE);
1574 1575 break;
1575 1576 case IOMEM_DATA_UC_WR_COMBINE:
1576 1577 *hataccp &= ~HAT_ORDER_MASK;
1577 1578 *hataccp |= (HAT_MERGING_OK | HAT_PLAT_NOCACHE);
1578 1579 break;
1579 1580 case IOMEM_DATA_CACHED:
1580 1581 *hataccp &= ~HAT_ORDER_MASK;
1581 1582 *hataccp |= HAT_UNORDERED_OK;
1582 1583 break;
1583 1584 /*
1584 1585 * This case must not occur because the cache attribute is scrutinized
1585 1586 * before this function is called.
1586 1587 */
1587 1588 default:
1588 1589 /*
1589 1590 * set cacheable to hat attrs.
1590 1591 */
1591 1592 *hataccp &= ~HAT_ORDER_MASK;
1592 1593 *hataccp |= HAT_UNORDERED_OK;
1593 1594 cmn_err(CE_WARN, "%s: cache_attr=0x%x is ignored.",
1594 1595 fname, cache_attr);
1595 1596 }
1596 1597 }
1597 1598
1598 1599 /*
1599 1600 * This should actually be called i_ddi_dma_mem_alloc. There should
1600 1601 * also be an i_ddi_pio_mem_alloc. i_ddi_dma_mem_alloc should call
1601 1602 * through the device tree with the DDI_CTLOPS_DMA_ALIGN ctl ops to
1602 1603 * get alignment requirements for DMA memory. i_ddi_pio_mem_alloc
1603 1604 * should use DDI_CTLOPS_PIO_ALIGN. Since we only have i_ddi_mem_alloc
1604 1605 * so far which is used for both, DMA and PIO, we have to use the DMA
1605 1606 * ctl ops to make everybody happy.
1606 1607 */
1607 1608 /*ARGSUSED*/
1608 1609 int
1609 1610 i_ddi_mem_alloc(dev_info_t *dip, ddi_dma_attr_t *attr,
1610 1611 size_t length, int cansleep, int flags,
1611 1612 ddi_device_acc_attr_t *accattrp, caddr_t *kaddrp,
1612 1613 size_t *real_length, ddi_acc_hdl_t *ap)
1613 1614 {
1614 1615 caddr_t a;
1615 1616 int iomin;
1616 1617 ddi_acc_impl_t *iap;
1617 1618 int physcontig = 0;
1618 1619 pgcnt_t npages;
1619 1620 pgcnt_t minctg;
1620 1621 uint_t order;
1621 1622 int e;
1622 1623
1623 1624 /*
1624 1625 * Check legality of arguments
1625 1626 */
1626 1627 if (length == 0 || kaddrp == NULL || attr == NULL) {
1627 1628 return (DDI_FAILURE);
1628 1629 }
1629 1630
1630 1631 if (attr->dma_attr_minxfer == 0 || attr->dma_attr_align == 0 ||
1631 1632 !ISP2(attr->dma_attr_align) || !ISP2(attr->dma_attr_minxfer)) {
1632 1633 return (DDI_FAILURE);
1633 1634 }
1634 1635
1635 1636 /*
1636 1637 * figure out most restrictive alignment requirement
1637 1638 */
1638 1639 iomin = attr->dma_attr_minxfer;
1639 1640 iomin = maxbit(iomin, attr->dma_attr_align);
1640 1641 if (iomin == 0)
1641 1642 return (DDI_FAILURE);
1642 1643
1643 1644 ASSERT((iomin & (iomin - 1)) == 0);
1644 1645
1645 1646 /*
1646 1647 * if we allocate memory with IOMEM_DATA_UNCACHED or
1647 1648 * IOMEM_DATA_UC_WR_COMBINE, make sure we allocate a page aligned
1648 1649 * memory that ends on a page boundry.
1649 1650 * Don't want to have to different cache mappings to the same
1650 1651 * physical page.
1651 1652 */
1652 1653 if (OVERRIDE_CACHE_ATTR(flags)) {
1653 1654 iomin = (iomin + MMU_PAGEOFFSET) & MMU_PAGEMASK;
1654 1655 length = (length + MMU_PAGEOFFSET) & (size_t)MMU_PAGEMASK;
1655 1656 }
1656 1657
1657 1658 /*
1658 1659 * Determine if we need to satisfy the request for physically
1659 1660 * contiguous memory or alignments larger than pagesize.
1660 1661 */
1661 1662 npages = btopr(length + attr->dma_attr_align);
1662 1663 minctg = howmany(npages, attr->dma_attr_sgllen);
1663 1664
1664 1665 if (minctg > 1) {
1665 1666 uint64_t pfnseg = attr->dma_attr_seg >> PAGESHIFT;
1666 1667 /*
1667 1668 * verify that the minimum contig requirement for the
1668 1669 * actual length does not cross segment boundary.
1669 1670 */
1670 1671 length = P2ROUNDUP_TYPED(length, attr->dma_attr_minxfer,
1671 1672 size_t);
1672 1673 npages = btopr(length);
1673 1674 minctg = howmany(npages, attr->dma_attr_sgllen);
1674 1675 if (minctg > pfnseg + 1)
1675 1676 return (DDI_FAILURE);
1676 1677 physcontig = 1;
1677 1678 } else {
1678 1679 length = P2ROUNDUP_TYPED(length, iomin, size_t);
1679 1680 }
1680 1681
1681 1682 /*
1682 1683 * Allocate the requested amount from the system.
1683 1684 */
1684 1685 a = kalloca(length, iomin, cansleep, physcontig, attr);
1685 1686
1686 1687 if ((*kaddrp = a) == NULL)
1687 1688 return (DDI_FAILURE);
1688 1689
1689 1690 /*
1690 1691 * if we to modify the cache attributes, go back and muck with the
1691 1692 * mappings.
1692 1693 */
1693 1694 if (OVERRIDE_CACHE_ATTR(flags)) {
1694 1695 order = 0;
1695 1696 i_ddi_cacheattr_to_hatacc(flags, &order);
1696 1697 e = kmem_override_cache_attrs(a, length, order);
1697 1698 if (e != 0) {
1698 1699 kfreea(a);
1699 1700 return (DDI_FAILURE);
1700 1701 }
1701 1702 }
1702 1703
1703 1704 if (real_length) {
1704 1705 *real_length = length;
1705 1706 }
1706 1707 if (ap) {
1707 1708 /*
1708 1709 * initialize access handle
1709 1710 */
1710 1711 iap = (ddi_acc_impl_t *)ap->ah_platform_private;
1711 1712 iap->ahi_acc_attr |= DDI_ACCATTR_CPU_VADDR;
1712 1713 impl_acc_hdl_init(ap);
1713 1714 }
1714 1715
1715 1716 return (DDI_SUCCESS);
1716 1717 }
1717 1718
1718 1719 /* ARGSUSED */
1719 1720 void
1720 1721 i_ddi_mem_free(caddr_t kaddr, ddi_acc_hdl_t *ap)
1721 1722 {
1722 1723 if (ap != NULL) {
1723 1724 /*
1724 1725 * if we modified the cache attributes on alloc, go back and
1725 1726 * fix them since this memory could be returned to the
1726 1727 * general pool.
1727 1728 */
1728 1729 if (OVERRIDE_CACHE_ATTR(ap->ah_xfermodes)) {
1729 1730 uint_t order = 0;
1730 1731 int e;
1731 1732 i_ddi_cacheattr_to_hatacc(IOMEM_DATA_CACHED, &order);
1732 1733 e = kmem_override_cache_attrs(kaddr, ap->ah_len, order);
1733 1734 if (e != 0) {
1734 1735 cmn_err(CE_WARN, "i_ddi_mem_free() failed to "
1735 1736 "override cache attrs, memory leaked\n");
1736 1737 return;
1737 1738 }
1738 1739 }
1739 1740 }
1740 1741 kfreea(kaddr);
1741 1742 }
1742 1743
1743 1744 /*
1744 1745 * Access Barriers
1745 1746 *
1746 1747 */
1747 1748 /*ARGSUSED*/
1748 1749 int
1749 1750 i_ddi_ontrap(ddi_acc_handle_t hp)
1750 1751 {
1751 1752 return (DDI_FAILURE);
1752 1753 }
1753 1754
1754 1755 /*ARGSUSED*/
1755 1756 void
1756 1757 i_ddi_notrap(ddi_acc_handle_t hp)
1757 1758 {
1758 1759 }
1759 1760
1760 1761
1761 1762 /*
1762 1763 * Misc Functions
1763 1764 */
1764 1765
1765 1766 /*
1766 1767 * Implementation instance override functions
1767 1768 *
1768 1769 * No override on i86pc
1769 1770 */
1770 1771 /*ARGSUSED*/
1771 1772 uint_t
1772 1773 impl_assign_instance(dev_info_t *dip)
1773 1774 {
1774 1775 return ((uint_t)-1);
1775 1776 }
1776 1777
1777 1778 /*ARGSUSED*/
1778 1779 int
1779 1780 impl_keep_instance(dev_info_t *dip)
1780 1781 {
1781 1782
1782 1783 #if defined(__xpv)
1783 1784 /*
1784 1785 * Do not persist instance numbers assigned to devices in dom0
1785 1786 */
1786 1787 dev_info_t *pdip;
1787 1788 if (DOMAIN_IS_INITDOMAIN(xen_info)) {
1788 1789 if (((pdip = ddi_get_parent(dip)) != NULL) &&
1789 1790 (strcmp(ddi_get_name(pdip), "xpvd") == 0))
1790 1791 return (DDI_SUCCESS);
1791 1792 }
1792 1793 #endif
1793 1794 return (DDI_FAILURE);
1794 1795 }
1795 1796
1796 1797 /*ARGSUSED*/
1797 1798 int
1798 1799 impl_free_instance(dev_info_t *dip)
1799 1800 {
1800 1801 return (DDI_FAILURE);
1801 1802 }
1802 1803
1803 1804 /*ARGSUSED*/
1804 1805 int
1805 1806 impl_check_cpu(dev_info_t *devi)
1806 1807 {
1807 1808 return (DDI_SUCCESS);
1808 1809 }
1809 1810
1810 1811 /*
1811 1812 * Referenced in common/cpr_driver.c: Power off machine.
1812 1813 * Don't know how to power off i86pc.
1813 1814 */
1814 1815 void
1815 1816 arch_power_down()
1816 1817 {}
1817 1818
1818 1819 /*
1819 1820 * Copy name to property_name, since name
1820 1821 * is in the low address range below kernelbase.
1821 1822 */
1822 1823 static void
1823 1824 copy_boot_str(const char *boot_str, char *kern_str, int len)
1824 1825 {
1825 1826 int i = 0;
1826 1827
1827 1828 while (i < len - 1 && boot_str[i] != '\0') {
1828 1829 kern_str[i] = boot_str[i];
1829 1830 i++;
1830 1831 }
1831 1832
1832 1833 kern_str[i] = 0; /* null terminate */
1833 1834 if (boot_str[i] != '\0')
|
↓ open down ↓ |
1753 lines elided |
↑ open up ↑ |
1834 1835 cmn_err(CE_WARN,
1835 1836 "boot property string is truncated to %s", kern_str);
1836 1837 }
1837 1838
1838 1839 static void
1839 1840 get_boot_properties(void)
1840 1841 {
1841 1842 extern char hw_provider[];
1842 1843 dev_info_t *devi;
1843 1844 char *name;
1844 - int length;
1845 + int length, flags;
1845 1846 char property_name[50], property_val[50];
1846 1847 void *bop_staging_area;
1847 1848
1848 1849 bop_staging_area = kmem_zalloc(MMU_PAGESIZE, KM_NOSLEEP);
1849 1850
1850 1851 /*
1851 1852 * Import "root" properties from the boot.
1852 1853 *
1853 1854 * We do this by invoking BOP_NEXTPROP until the list
1854 1855 * is completely copied in.
1855 1856 */
1856 1857
1857 1858 devi = ddi_root_node();
1858 1859 for (name = BOP_NEXTPROP(bootops, ""); /* get first */
1859 1860 name; /* NULL => DONE */
1860 1861 name = BOP_NEXTPROP(bootops, name)) { /* get next */
1861 1862
1862 1863 /* copy string to memory above kernelbase */
1863 1864 copy_boot_str(name, property_name, 50);
1864 1865
|
↓ open down ↓ |
10 lines elided |
↑ open up ↑ |
1865 1866 /*
1866 1867 * Skip vga properties. They will be picked up later
1867 1868 * by get_vga_properties.
1868 1869 */
1869 1870 if (strcmp(property_name, "display-edif-block") == 0 ||
1870 1871 strcmp(property_name, "display-edif-id") == 0) {
1871 1872 continue;
1872 1873 }
1873 1874
1874 1875 length = BOP_GETPROPLEN(bootops, property_name);
1875 - if (length == 0)
1876 + if (length < 0)
1876 1877 continue;
1877 1878 if (length > MMU_PAGESIZE) {
1878 1879 cmn_err(CE_NOTE,
1879 1880 "boot property %s longer than 0x%x, ignored\n",
1880 1881 property_name, MMU_PAGESIZE);
1881 1882 continue;
1882 1883 }
1883 1884 BOP_GETPROP(bootops, property_name, bop_staging_area);
1885 + flags = do_bsys_getproptype(bootops, property_name);
1884 1886
1885 1887 /*
1886 1888 * special properties:
1887 1889 * si-machine, si-hw-provider
1888 1890 * goes to kernel data structures.
1889 1891 * bios-boot-device and stdout
1890 1892 * goes to hardware property list so it may show up
1891 1893 * in the prtconf -vp output. This is needed by
1892 1894 * Install/Upgrade. Once we fix install upgrade,
1893 1895 * this can be taken out.
1894 1896 */
1895 1897 if (strcmp(name, "si-machine") == 0) {
1896 1898 (void) strncpy(utsname.machine, bop_staging_area,
1897 1899 SYS_NMLN);
1898 - utsname.machine[SYS_NMLN - 1] = (char)NULL;
1899 - } else if (strcmp(name, "si-hw-provider") == 0) {
1900 + utsname.machine[SYS_NMLN - 1] = '\0';
1901 + continue;
1902 + }
1903 + if (strcmp(name, "si-hw-provider") == 0) {
1900 1904 (void) strncpy(hw_provider, bop_staging_area, SYS_NMLN);
1901 - hw_provider[SYS_NMLN - 1] = (char)NULL;
1902 - } else if (strcmp(name, "bios-boot-device") == 0) {
1905 + hw_provider[SYS_NMLN - 1] = '\0';
1906 + continue;
1907 + }
1908 + if (strcmp(name, "bios-boot-device") == 0) {
1903 1909 copy_boot_str(bop_staging_area, property_val, 50);
1904 1910 (void) ndi_prop_update_string(DDI_DEV_T_NONE, devi,
1905 1911 property_name, property_val);
1906 - } else if (strcmp(name, "acpi-root-tab") == 0) {
1907 - (void) ndi_prop_update_int64(DDI_DEV_T_NONE, devi,
1908 - property_name, *((int64_t *)bop_staging_area));
1909 - } else if (strcmp(name, "smbios-address") == 0) {
1910 - (void) ndi_prop_update_int64(DDI_DEV_T_NONE, devi,
1911 - property_name, *((int64_t *)bop_staging_area));
1912 - } else if (strcmp(name, "efi-systab") == 0) {
1913 - (void) ndi_prop_update_int64(DDI_DEV_T_NONE, devi,
1914 - property_name, *((int64_t *)bop_staging_area));
1915 - } else if (strcmp(name, "efi-systype") == 0) {
1916 - copy_boot_str(bop_staging_area, property_val, 50);
1917 - (void) ndi_prop_update_string(DDI_DEV_T_NONE, devi,
1918 - property_name, property_val);
1919 - } else if (strcmp(name, "stdout") == 0) {
1912 + continue;
1913 + }
1914 + if (strcmp(name, "stdout") == 0) {
1920 1915 (void) ndi_prop_update_int(DDI_DEV_T_NONE, devi,
1921 1916 property_name, *((int *)bop_staging_area));
1922 - } else if (strcmp(name, "boot-args") == 0) {
1923 - copy_boot_str(bop_staging_area, property_val, 50);
1917 + continue;
1918 + }
1919 +
1920 + /* Boolean property */
1921 + if (length == 0) {
1922 + (void) e_ddi_prop_create(DDI_DEV_T_NONE, devi,
1923 + DDI_PROP_CANSLEEP, property_name, NULL, 0);
1924 + continue;
1925 + }
1926 +
1927 + /* Now anything else based on type. */
1928 + switch (flags) {
1929 + case DDI_PROP_TYPE_INT:
1930 + if (length == sizeof (int)) {
1931 + (void) e_ddi_prop_update_int(DDI_DEV_T_NONE,
1932 + devi, property_name,
1933 + *((int *)bop_staging_area));
1934 + } else {
1935 + (void) e_ddi_prop_update_int_array(
1936 + DDI_DEV_T_NONE, devi, property_name,
1937 + bop_staging_area, length / sizeof (int));
1938 + }
1939 + break;
1940 + case DDI_PROP_TYPE_STRING:
1924 1941 (void) e_ddi_prop_update_string(DDI_DEV_T_NONE, devi,
1925 - property_name, property_val);
1926 - } else if (strcmp(name, "bootargs") == 0) {
1927 - copy_boot_str(bop_staging_area, property_val, 50);
1928 - (void) e_ddi_prop_update_string(DDI_DEV_T_NONE, devi,
1929 - property_name, property_val);
1930 - } else if (strcmp(name, "bootp-response") == 0) {
1942 + property_name, bop_staging_area);
1943 + break;
1944 + case DDI_PROP_TYPE_BYTE:
1931 1945 (void) e_ddi_prop_update_byte_array(DDI_DEV_T_NONE,
1932 1946 devi, property_name, bop_staging_area, length);
1933 - } else if (strcmp(name, "ramdisk_start") == 0) {
1934 - (void) e_ddi_prop_update_int64(DDI_DEV_T_NONE, devi,
1935 - property_name, *((int64_t *)bop_staging_area));
1936 - } else if (strcmp(name, "ramdisk_end") == 0) {
1937 - (void) e_ddi_prop_update_int64(DDI_DEV_T_NONE, devi,
1938 - property_name, *((int64_t *)bop_staging_area));
1939 - } else if (strncmp(name, "module-addr-", 12) == 0) {
1940 - (void) e_ddi_prop_update_int64(DDI_DEV_T_NONE, devi,
1941 - property_name, *((int64_t *)bop_staging_area));
1942 - } else if (strncmp(name, "module-size-", 12) == 0) {
1943 - (void) e_ddi_prop_update_int64(DDI_DEV_T_NONE, devi,
1944 - property_name, *((int64_t *)bop_staging_area));
1945 - } else {
1947 + break;
1948 + case DDI_PROP_TYPE_INT64:
1949 + if (length == sizeof (uint64_t)) {
1950 + (void) e_ddi_prop_update_int64(DDI_DEV_T_NONE,
1951 + devi, property_name,
1952 + *((uint64_t *)bop_staging_area));
1953 + } else {
1954 + (void) e_ddi_prop_update_int64_array(
1955 + DDI_DEV_T_NONE, devi, property_name,
1956 + bop_staging_area,
1957 + length / sizeof (uint64_t));
1958 + }
1959 + break;
1960 + default:
1946 1961 /* Property type unknown, use old prop interface */
1947 1962 (void) e_ddi_prop_create(DDI_DEV_T_NONE, devi,
1948 1963 DDI_PROP_CANSLEEP, property_name, bop_staging_area,
1949 1964 length);
1950 1965 }
1951 1966 }
1952 1967
1953 1968 kmem_free(bop_staging_area, MMU_PAGESIZE);
1954 1969 }
1955 1970
1956 1971 static void
1957 1972 get_vga_properties(void)
1958 1973 {
1959 1974 dev_info_t *devi;
1960 1975 major_t major;
1961 1976 char *name;
1962 1977 int length;
1963 1978 char property_val[50];
1964 1979 void *bop_staging_area;
1965 1980
1966 1981 /*
1967 1982 * XXXX Hack Allert!
1968 1983 * There really needs to be a better way for identifying various
1969 1984 * console framebuffers and their related issues. Till then,
1970 1985 * check for this one as a replacement to vgatext.
1971 1986 */
1972 1987 major = ddi_name_to_major("ragexl");
1973 1988 if (major == (major_t)-1) {
1974 1989 major = ddi_name_to_major("vgatext");
1975 1990 if (major == (major_t)-1)
1976 1991 return;
1977 1992 }
1978 1993 devi = devnamesp[major].dn_head;
1979 1994 if (devi == NULL)
1980 1995 return;
1981 1996
1982 1997 bop_staging_area = kmem_zalloc(MMU_PAGESIZE, KM_SLEEP);
1983 1998
1984 1999 /*
1985 2000 * Import "vga" properties from the boot.
1986 2001 */
1987 2002 name = "display-edif-block";
1988 2003 length = BOP_GETPROPLEN(bootops, name);
1989 2004 if (length > 0 && length < MMU_PAGESIZE) {
1990 2005 BOP_GETPROP(bootops, name, bop_staging_area);
1991 2006 (void) ndi_prop_update_byte_array(DDI_DEV_T_NONE,
1992 2007 devi, name, bop_staging_area, length);
1993 2008 }
1994 2009
1995 2010 /*
1996 2011 * kdmconfig is also looking for display-type and
1997 2012 * video-adapter-type. We default to color and svga.
1998 2013 *
1999 2014 * Could it be "monochrome", "vga"?
2000 2015 * Nah, you've got to come to the 21st century...
2001 2016 * And you can set monitor type manually in kdmconfig
2002 2017 * if you are really an old junky.
2003 2018 */
2004 2019 (void) ndi_prop_update_string(DDI_DEV_T_NONE,
2005 2020 devi, "display-type", "color");
2006 2021 (void) ndi_prop_update_string(DDI_DEV_T_NONE,
2007 2022 devi, "video-adapter-type", "svga");
2008 2023
2009 2024 name = "display-edif-id";
2010 2025 length = BOP_GETPROPLEN(bootops, name);
|
↓ open down ↓ |
55 lines elided |
↑ open up ↑ |
2011 2026 if (length > 0 && length < MMU_PAGESIZE) {
2012 2027 BOP_GETPROP(bootops, name, bop_staging_area);
2013 2028 copy_boot_str(bop_staging_area, property_val, length);
2014 2029 (void) ndi_prop_update_string(DDI_DEV_T_NONE,
2015 2030 devi, name, property_val);
2016 2031 }
2017 2032
2018 2033 kmem_free(bop_staging_area, MMU_PAGESIZE);
2019 2034 }
2020 2035
2036 +/*
2037 + * Copy console font to kernel memory. The temporary font setup
2038 + * to use font module was done in early console setup, using low
2039 + * memory and data from font module. Now we need to allocate
2040 + * kernel memory and copy data over, so the low memory can be freed.
2041 + * We can have at most one entry in font list from early boot.
2042 + */
2043 +static void
2044 +get_console_font(void)
2045 +{
2046 + struct fontlist *fp, *fl;
2047 + bitmap_data_t *bd;
2048 + struct font *fd, *tmp;
2049 + int i;
2021 2050
2051 + if (STAILQ_EMPTY(&fonts))
2052 + return;
2053 +
2054 + fl = STAILQ_FIRST(&fonts);
2055 + STAILQ_REMOVE_HEAD(&fonts, font_next);
2056 + fp = kmem_zalloc(sizeof (*fp), KM_SLEEP);
2057 + bd = kmem_zalloc(sizeof (*bd), KM_SLEEP);
2058 + fd = kmem_zalloc(sizeof (*fd), KM_SLEEP);
2059 +
2060 + fp->font_name = NULL;
2061 + fp->font_flags = FONT_BOOT;
2062 + fp->font_data = bd;
2063 +
2064 + bd->width = fl->font_data->width;
2065 + bd->height = fl->font_data->height;
2066 + bd->uncompressed_size = fl->font_data->uncompressed_size;
2067 + bd->font = fd;
2068 +
2069 + tmp = fl->font_data->font;
2070 + fd->vf_width = tmp->vf_width;
2071 + fd->vf_height = tmp->vf_height;
2072 + for (i = 0; i < VFNT_MAPS; i++) {
2073 + if (tmp->vf_map_count[i] == 0)
2074 + continue;
2075 + fd->vf_map_count[i] = tmp->vf_map_count[i];
2076 + fd->vf_map[i] = kmem_alloc(fd->vf_map_count[i] *
2077 + sizeof (*fd->vf_map[i]), KM_SLEEP);
2078 + bcopy(tmp->vf_map[i], fd->vf_map[i], fd->vf_map_count[i] *
2079 + sizeof (*fd->vf_map[i]));
2080 + }
2081 + fd->vf_bytes = kmem_alloc(bd->uncompressed_size, KM_SLEEP);
2082 + bcopy(tmp->vf_bytes, fd->vf_bytes, bd->uncompressed_size);
2083 + STAILQ_INSERT_HEAD(&fonts, fp, font_next);
2084 +}
2085 +
2022 2086 /*
2023 2087 * This is temporary, but absolutely necessary. If we are being
2024 2088 * booted with a device tree created by the DevConf project's bootconf
2025 2089 * program, then we have device information nodes that reflect
2026 2090 * reality. At this point in time in the Solaris release schedule, the
2027 2091 * kernel drivers aren't prepared for reality. They still depend on their
2028 2092 * own ad-hoc interpretations of the properties created when their .conf
2029 2093 * files were interpreted. These drivers use an "ignore-hardware-nodes"
2030 2094 * property to prevent them from using the nodes passed up from the bootconf
2031 2095 * device tree.
2032 2096 *
2033 2097 * Trying to assemble root file system drivers as we are booting from
2034 2098 * devconf will fail if the kernel driver is basing its name_addr's on the
2035 2099 * psuedo-node device info while the bootpath passed up from bootconf is using
2036 2100 * reality-based name_addrs. We help the boot along in this case by
2037 2101 * looking at the pre-bootconf bootpath and determining if we would have
2038 2102 * successfully matched if that had been the bootpath we had chosen.
2039 2103 *
2040 2104 * Note that we only even perform this extra check if we've booted
2041 2105 * using bootconf's 1275 compliant bootpath, this is the boot device, and
2042 2106 * we're trying to match the name_addr specified in the 1275 bootpath.
2043 2107 */
2044 2108
2045 2109 #define MAXCOMPONENTLEN 32
2046 2110
2047 2111 int
2048 2112 x86_old_bootpath_name_addr_match(dev_info_t *cdip, char *caddr, char *naddr)
2049 2113 {
2050 2114 /*
2051 2115 * There are multiple criteria to be met before we can even
2052 2116 * consider allowing a name_addr match here.
2053 2117 *
2054 2118 * 1) We must have been booted such that the bootconf program
2055 2119 * created device tree nodes and properties. This can be
2056 2120 * determined by examining the 'bootpath' property. This
2057 2121 * property will be a non-null string iff bootconf was
2058 2122 * involved in the boot.
2059 2123 *
2060 2124 * 2) The module that we want to match must be the boot device.
2061 2125 *
2062 2126 * 3) The instance of the module we are thinking of letting be
2063 2127 * our match must be ignoring hardware nodes.
2064 2128 *
2065 2129 * 4) The name_addr we want to match must be the name_addr
2066 2130 * specified in the 1275 bootpath.
2067 2131 */
2068 2132 static char bootdev_module[MAXCOMPONENTLEN];
2069 2133 static char bootdev_oldmod[MAXCOMPONENTLEN];
2070 2134 static char bootdev_newaddr[MAXCOMPONENTLEN];
2071 2135 static char bootdev_oldaddr[MAXCOMPONENTLEN];
2072 2136 static int quickexit;
2073 2137
2074 2138 char *daddr;
2075 2139 int dlen;
2076 2140
2077 2141 char *lkupname;
2078 2142 int rv = DDI_FAILURE;
2079 2143
2080 2144 if ((ddi_getlongprop(DDI_DEV_T_ANY, cdip, DDI_PROP_DONTPASS,
2081 2145 "devconf-addr", (caddr_t)&daddr, &dlen) == DDI_PROP_SUCCESS) &&
2082 2146 (ddi_getprop(DDI_DEV_T_ANY, cdip, DDI_PROP_DONTPASS,
2083 2147 "ignore-hardware-nodes", -1) != -1)) {
2084 2148 if (strcmp(daddr, caddr) == 0) {
2085 2149 return (DDI_SUCCESS);
2086 2150 }
2087 2151 }
2088 2152
2089 2153 if (quickexit)
2090 2154 return (rv);
2091 2155
2092 2156 if (bootdev_module[0] == '\0') {
2093 2157 char *addrp, *eoaddrp;
2094 2158 char *busp, *modp, *atp;
2095 2159 char *bp1275, *bp;
2096 2160 int bp1275len, bplen;
2097 2161
2098 2162 bp1275 = bp = addrp = eoaddrp = busp = modp = atp = NULL;
2099 2163
2100 2164 if (ddi_getlongprop(DDI_DEV_T_ANY,
2101 2165 ddi_root_node(), 0, "bootpath",
2102 2166 (caddr_t)&bp1275, &bp1275len) != DDI_PROP_SUCCESS ||
2103 2167 bp1275len <= 1) {
2104 2168 /*
2105 2169 * We didn't boot from bootconf so we never need to
2106 2170 * do any special matches.
2107 2171 */
2108 2172 quickexit = 1;
2109 2173 if (bp1275)
2110 2174 kmem_free(bp1275, bp1275len);
2111 2175 return (rv);
2112 2176 }
2113 2177
2114 2178 if (ddi_getlongprop(DDI_DEV_T_ANY,
2115 2179 ddi_root_node(), 0, "boot-path",
2116 2180 (caddr_t)&bp, &bplen) != DDI_PROP_SUCCESS || bplen <= 1) {
2117 2181 /*
2118 2182 * No fallback position for matching. This is
2119 2183 * certainly unexpected, but we'll handle it
2120 2184 * just in case.
2121 2185 */
2122 2186 quickexit = 1;
2123 2187 kmem_free(bp1275, bp1275len);
2124 2188 if (bp)
2125 2189 kmem_free(bp, bplen);
2126 2190 return (rv);
2127 2191 }
2128 2192
2129 2193 /*
2130 2194 * Determine boot device module and 1275 name_addr
2131 2195 *
2132 2196 * bootpath assumed to be of the form /bus/module@name_addr
2133 2197 */
2134 2198 if (busp = strchr(bp1275, '/')) {
2135 2199 if (modp = strchr(busp + 1, '/')) {
2136 2200 if (atp = strchr(modp + 1, '@')) {
2137 2201 *atp = '\0';
2138 2202 addrp = atp + 1;
2139 2203 if (eoaddrp = strchr(addrp, '/'))
2140 2204 *eoaddrp = '\0';
2141 2205 }
2142 2206 }
2143 2207 }
2144 2208
2145 2209 if (modp && addrp) {
2146 2210 (void) strncpy(bootdev_module, modp + 1,
2147 2211 MAXCOMPONENTLEN);
2148 2212 bootdev_module[MAXCOMPONENTLEN - 1] = '\0';
2149 2213
2150 2214 (void) strncpy(bootdev_newaddr, addrp, MAXCOMPONENTLEN);
2151 2215 bootdev_newaddr[MAXCOMPONENTLEN - 1] = '\0';
2152 2216 } else {
2153 2217 quickexit = 1;
2154 2218 kmem_free(bp1275, bp1275len);
2155 2219 kmem_free(bp, bplen);
2156 2220 return (rv);
2157 2221 }
2158 2222
2159 2223 /*
2160 2224 * Determine fallback name_addr
2161 2225 *
2162 2226 * 10/3/96 - Also save fallback module name because it
2163 2227 * might actually be different than the current module
2164 2228 * name. E.G., ISA pnp drivers have new names.
2165 2229 *
2166 2230 * bootpath assumed to be of the form /bus/module@name_addr
2167 2231 */
2168 2232 addrp = NULL;
2169 2233 if (busp = strchr(bp, '/')) {
2170 2234 if (modp = strchr(busp + 1, '/')) {
2171 2235 if (atp = strchr(modp + 1, '@')) {
2172 2236 *atp = '\0';
2173 2237 addrp = atp + 1;
2174 2238 if (eoaddrp = strchr(addrp, '/'))
2175 2239 *eoaddrp = '\0';
2176 2240 }
2177 2241 }
2178 2242 }
2179 2243
2180 2244 if (modp && addrp) {
2181 2245 (void) strncpy(bootdev_oldmod, modp + 1,
2182 2246 MAXCOMPONENTLEN);
2183 2247 bootdev_module[MAXCOMPONENTLEN - 1] = '\0';
2184 2248
2185 2249 (void) strncpy(bootdev_oldaddr, addrp, MAXCOMPONENTLEN);
2186 2250 bootdev_oldaddr[MAXCOMPONENTLEN - 1] = '\0';
2187 2251 }
2188 2252
2189 2253 /* Free up the bootpath storage now that we're done with it. */
2190 2254 kmem_free(bp1275, bp1275len);
2191 2255 kmem_free(bp, bplen);
2192 2256
2193 2257 if (bootdev_oldaddr[0] == '\0') {
2194 2258 quickexit = 1;
2195 2259 return (rv);
2196 2260 }
2197 2261 }
2198 2262
2199 2263 if (((lkupname = ddi_get_name(cdip)) != NULL) &&
2200 2264 (strcmp(bootdev_module, lkupname) == 0 ||
2201 2265 strcmp(bootdev_oldmod, lkupname) == 0) &&
2202 2266 ((ddi_getprop(DDI_DEV_T_ANY, cdip, DDI_PROP_DONTPASS,
2203 2267 "ignore-hardware-nodes", -1) != -1) ||
2204 2268 ignore_hardware_nodes) &&
2205 2269 strcmp(bootdev_newaddr, caddr) == 0 &&
2206 2270 strcmp(bootdev_oldaddr, naddr) == 0) {
2207 2271 rv = DDI_SUCCESS;
2208 2272 }
2209 2273
2210 2274 return (rv);
2211 2275 }
2212 2276
2213 2277 /*
2214 2278 * Perform a copy from a memory mapped device (whose devinfo pointer is devi)
2215 2279 * separately mapped at devaddr in the kernel to a kernel buffer at kaddr.
2216 2280 */
2217 2281 /*ARGSUSED*/
2218 2282 int
2219 2283 e_ddi_copyfromdev(dev_info_t *devi,
2220 2284 off_t off, const void *devaddr, void *kaddr, size_t len)
2221 2285 {
2222 2286 bcopy(devaddr, kaddr, len);
2223 2287 return (0);
2224 2288 }
2225 2289
2226 2290 /*
2227 2291 * Perform a copy to a memory mapped device (whose devinfo pointer is devi)
2228 2292 * separately mapped at devaddr in the kernel from a kernel buffer at kaddr.
2229 2293 */
2230 2294 /*ARGSUSED*/
2231 2295 int
2232 2296 e_ddi_copytodev(dev_info_t *devi,
2233 2297 off_t off, const void *kaddr, void *devaddr, size_t len)
2234 2298 {
2235 2299 bcopy(kaddr, devaddr, len);
2236 2300 return (0);
2237 2301 }
2238 2302
2239 2303
2240 2304 static int
2241 2305 poke_mem(peekpoke_ctlops_t *in_args)
2242 2306 {
2243 2307 int err = DDI_SUCCESS;
2244 2308 on_trap_data_t otd;
2245 2309
2246 2310 /* Set up protected environment. */
2247 2311 if (!on_trap(&otd, OT_DATA_ACCESS)) {
2248 2312 switch (in_args->size) {
2249 2313 case sizeof (uint8_t):
2250 2314 *(uint8_t *)(in_args->dev_addr) =
2251 2315 *(uint8_t *)in_args->host_addr;
2252 2316 break;
2253 2317
2254 2318 case sizeof (uint16_t):
2255 2319 *(uint16_t *)(in_args->dev_addr) =
2256 2320 *(uint16_t *)in_args->host_addr;
2257 2321 break;
2258 2322
2259 2323 case sizeof (uint32_t):
2260 2324 *(uint32_t *)(in_args->dev_addr) =
2261 2325 *(uint32_t *)in_args->host_addr;
2262 2326 break;
2263 2327
2264 2328 case sizeof (uint64_t):
2265 2329 *(uint64_t *)(in_args->dev_addr) =
2266 2330 *(uint64_t *)in_args->host_addr;
2267 2331 break;
2268 2332
2269 2333 default:
2270 2334 err = DDI_FAILURE;
2271 2335 break;
2272 2336 }
2273 2337 } else
2274 2338 err = DDI_FAILURE;
2275 2339
2276 2340 /* Take down protected environment. */
2277 2341 no_trap();
2278 2342
2279 2343 return (err);
2280 2344 }
2281 2345
2282 2346
2283 2347 static int
2284 2348 peek_mem(peekpoke_ctlops_t *in_args)
2285 2349 {
2286 2350 int err = DDI_SUCCESS;
2287 2351 on_trap_data_t otd;
2288 2352
2289 2353 if (!on_trap(&otd, OT_DATA_ACCESS)) {
2290 2354 switch (in_args->size) {
2291 2355 case sizeof (uint8_t):
2292 2356 *(uint8_t *)in_args->host_addr =
2293 2357 *(uint8_t *)in_args->dev_addr;
2294 2358 break;
2295 2359
2296 2360 case sizeof (uint16_t):
2297 2361 *(uint16_t *)in_args->host_addr =
2298 2362 *(uint16_t *)in_args->dev_addr;
2299 2363 break;
2300 2364
2301 2365 case sizeof (uint32_t):
2302 2366 *(uint32_t *)in_args->host_addr =
2303 2367 *(uint32_t *)in_args->dev_addr;
2304 2368 break;
2305 2369
2306 2370 case sizeof (uint64_t):
2307 2371 *(uint64_t *)in_args->host_addr =
2308 2372 *(uint64_t *)in_args->dev_addr;
2309 2373 break;
2310 2374
2311 2375 default:
2312 2376 err = DDI_FAILURE;
2313 2377 break;
2314 2378 }
2315 2379 } else
2316 2380 err = DDI_FAILURE;
2317 2381
2318 2382 no_trap();
2319 2383 return (err);
2320 2384 }
2321 2385
2322 2386
2323 2387 /*
2324 2388 * This is called only to process peek/poke when the DIP is NULL.
2325 2389 * Assume that this is for memory, as nexi take care of device safe accesses.
2326 2390 */
2327 2391 int
2328 2392 peekpoke_mem(ddi_ctl_enum_t cmd, peekpoke_ctlops_t *in_args)
2329 2393 {
2330 2394 return (cmd == DDI_CTLOPS_PEEK ? peek_mem(in_args) : poke_mem(in_args));
2331 2395 }
2332 2396
2333 2397 /*
2334 2398 * we've just done a cautious put/get. Check if it was successful by
2335 2399 * calling pci_ereport_post() on all puts and for any gets that return -1
2336 2400 */
2337 2401 static int
2338 2402 pci_peekpoke_check_fma(dev_info_t *dip, void *arg, ddi_ctl_enum_t ctlop,
2339 2403 void (*scan)(dev_info_t *, ddi_fm_error_t *))
2340 2404 {
2341 2405 int rval = DDI_SUCCESS;
2342 2406 peekpoke_ctlops_t *in_args = (peekpoke_ctlops_t *)arg;
2343 2407 ddi_fm_error_t de;
2344 2408 ddi_acc_impl_t *hp = (ddi_acc_impl_t *)in_args->handle;
2345 2409 ddi_acc_hdl_t *hdlp = (ddi_acc_hdl_t *)in_args->handle;
2346 2410 int check_err = 0;
2347 2411 int repcount = in_args->repcount;
2348 2412
2349 2413 if (ctlop == DDI_CTLOPS_POKE &&
2350 2414 hdlp->ah_acc.devacc_attr_access != DDI_CAUTIOUS_ACC)
2351 2415 return (DDI_SUCCESS);
2352 2416
2353 2417 if (ctlop == DDI_CTLOPS_PEEK &&
2354 2418 hdlp->ah_acc.devacc_attr_access != DDI_CAUTIOUS_ACC) {
2355 2419 for (; repcount; repcount--) {
2356 2420 switch (in_args->size) {
2357 2421 case sizeof (uint8_t):
2358 2422 if (*(uint8_t *)in_args->host_addr == 0xff)
2359 2423 check_err = 1;
2360 2424 break;
2361 2425 case sizeof (uint16_t):
2362 2426 if (*(uint16_t *)in_args->host_addr == 0xffff)
2363 2427 check_err = 1;
2364 2428 break;
2365 2429 case sizeof (uint32_t):
2366 2430 if (*(uint32_t *)in_args->host_addr ==
2367 2431 0xffffffff)
2368 2432 check_err = 1;
2369 2433 break;
2370 2434 case sizeof (uint64_t):
2371 2435 if (*(uint64_t *)in_args->host_addr ==
2372 2436 0xffffffffffffffff)
2373 2437 check_err = 1;
2374 2438 break;
2375 2439 }
2376 2440 }
2377 2441 if (check_err == 0)
2378 2442 return (DDI_SUCCESS);
2379 2443 }
2380 2444 /*
2381 2445 * for a cautious put or get or a non-cautious get that returned -1 call
2382 2446 * io framework to see if there really was an error
2383 2447 */
2384 2448 bzero(&de, sizeof (ddi_fm_error_t));
2385 2449 de.fme_version = DDI_FME_VERSION;
2386 2450 de.fme_ena = fm_ena_generate(0, FM_ENA_FMT1);
2387 2451 if (hdlp->ah_acc.devacc_attr_access == DDI_CAUTIOUS_ACC) {
2388 2452 de.fme_flag = DDI_FM_ERR_EXPECTED;
2389 2453 de.fme_acc_handle = in_args->handle;
2390 2454 } else if (hdlp->ah_acc.devacc_attr_access == DDI_DEFAULT_ACC) {
2391 2455 /*
2392 2456 * We only get here with DDI_DEFAULT_ACC for config space gets.
2393 2457 * Non-hardened drivers may be probing the hardware and
2394 2458 * expecting -1 returned. So need to treat errors on
2395 2459 * DDI_DEFAULT_ACC as DDI_FM_ERR_EXPECTED.
2396 2460 */
2397 2461 de.fme_flag = DDI_FM_ERR_EXPECTED;
2398 2462 de.fme_acc_handle = in_args->handle;
2399 2463 } else {
2400 2464 /*
2401 2465 * Hardened driver doing protected accesses shouldn't
2402 2466 * get errors unless there's a hardware problem. Treat
2403 2467 * as nonfatal if there's an error, but set UNEXPECTED
2404 2468 * so we raise ereports on any errors and potentially
2405 2469 * fault the device
2406 2470 */
2407 2471 de.fme_flag = DDI_FM_ERR_UNEXPECTED;
2408 2472 }
2409 2473 (void) scan(dip, &de);
2410 2474 if (hdlp->ah_acc.devacc_attr_access != DDI_DEFAULT_ACC &&
2411 2475 de.fme_status != DDI_FM_OK) {
2412 2476 ndi_err_t *errp = (ndi_err_t *)hp->ahi_err;
2413 2477 rval = DDI_FAILURE;
2414 2478 errp->err_ena = de.fme_ena;
2415 2479 errp->err_expected = de.fme_flag;
2416 2480 errp->err_status = DDI_FM_NONFATAL;
2417 2481 }
2418 2482 return (rval);
2419 2483 }
2420 2484
2421 2485 /*
2422 2486 * pci_peekpoke_check_nofma() is for when an error occurs on a register access
2423 2487 * during pci_ereport_post(). We can't call pci_ereport_post() again or we'd
2424 2488 * recurse, so assume all puts are OK and gets have failed if they return -1
2425 2489 */
2426 2490 static int
2427 2491 pci_peekpoke_check_nofma(void *arg, ddi_ctl_enum_t ctlop)
2428 2492 {
2429 2493 int rval = DDI_SUCCESS;
2430 2494 peekpoke_ctlops_t *in_args = (peekpoke_ctlops_t *)arg;
2431 2495 ddi_acc_impl_t *hp = (ddi_acc_impl_t *)in_args->handle;
2432 2496 ddi_acc_hdl_t *hdlp = (ddi_acc_hdl_t *)in_args->handle;
2433 2497 int repcount = in_args->repcount;
2434 2498
2435 2499 if (ctlop == DDI_CTLOPS_POKE)
2436 2500 return (rval);
2437 2501
2438 2502 for (; repcount; repcount--) {
2439 2503 switch (in_args->size) {
2440 2504 case sizeof (uint8_t):
2441 2505 if (*(uint8_t *)in_args->host_addr == 0xff)
2442 2506 rval = DDI_FAILURE;
2443 2507 break;
2444 2508 case sizeof (uint16_t):
2445 2509 if (*(uint16_t *)in_args->host_addr == 0xffff)
2446 2510 rval = DDI_FAILURE;
2447 2511 break;
2448 2512 case sizeof (uint32_t):
2449 2513 if (*(uint32_t *)in_args->host_addr == 0xffffffff)
2450 2514 rval = DDI_FAILURE;
2451 2515 break;
2452 2516 case sizeof (uint64_t):
2453 2517 if (*(uint64_t *)in_args->host_addr ==
2454 2518 0xffffffffffffffff)
2455 2519 rval = DDI_FAILURE;
2456 2520 break;
2457 2521 }
2458 2522 }
2459 2523 if (hdlp->ah_acc.devacc_attr_access != DDI_DEFAULT_ACC &&
2460 2524 rval == DDI_FAILURE) {
2461 2525 ndi_err_t *errp = (ndi_err_t *)hp->ahi_err;
2462 2526 errp->err_ena = fm_ena_generate(0, FM_ENA_FMT1);
2463 2527 errp->err_expected = DDI_FM_ERR_UNEXPECTED;
2464 2528 errp->err_status = DDI_FM_NONFATAL;
2465 2529 }
2466 2530 return (rval);
2467 2531 }
2468 2532
2469 2533 int
2470 2534 pci_peekpoke_check(dev_info_t *dip, dev_info_t *rdip,
2471 2535 ddi_ctl_enum_t ctlop, void *arg, void *result,
2472 2536 int (*handler)(dev_info_t *, dev_info_t *, ddi_ctl_enum_t, void *,
2473 2537 void *), kmutex_t *err_mutexp, kmutex_t *peek_poke_mutexp,
2474 2538 void (*scan)(dev_info_t *, ddi_fm_error_t *))
2475 2539 {
2476 2540 int rval;
2477 2541 peekpoke_ctlops_t *in_args = (peekpoke_ctlops_t *)arg;
2478 2542 ddi_acc_impl_t *hp = (ddi_acc_impl_t *)in_args->handle;
2479 2543
2480 2544 /*
2481 2545 * this function only supports cautious accesses, not peeks/pokes
2482 2546 * which don't have a handle
2483 2547 */
2484 2548 if (hp == NULL)
2485 2549 return (DDI_FAILURE);
2486 2550
2487 2551 if (hp->ahi_acc_attr & DDI_ACCATTR_CONFIG_SPACE) {
2488 2552 if (!mutex_tryenter(err_mutexp)) {
2489 2553 /*
2490 2554 * As this may be a recursive call from within
2491 2555 * pci_ereport_post() we can't wait for the mutexes.
2492 2556 * Fortunately we know someone is already calling
2493 2557 * pci_ereport_post() which will handle the error bits
2494 2558 * for us, and as this is a config space access we can
2495 2559 * just do the access and check return value for -1
2496 2560 * using pci_peekpoke_check_nofma().
2497 2561 */
2498 2562 rval = handler(dip, rdip, ctlop, arg, result);
2499 2563 if (rval == DDI_SUCCESS)
2500 2564 rval = pci_peekpoke_check_nofma(arg, ctlop);
2501 2565 return (rval);
2502 2566 }
2503 2567 /*
2504 2568 * This can't be a recursive call. Drop the err_mutex and get
2505 2569 * both mutexes in the right order. If an error hasn't already
2506 2570 * been detected by the ontrap code, use pci_peekpoke_check_fma
2507 2571 * which will call pci_ereport_post() to check error status.
2508 2572 */
2509 2573 mutex_exit(err_mutexp);
2510 2574 }
2511 2575 mutex_enter(peek_poke_mutexp);
2512 2576 rval = handler(dip, rdip, ctlop, arg, result);
2513 2577 if (rval == DDI_SUCCESS) {
2514 2578 mutex_enter(err_mutexp);
2515 2579 rval = pci_peekpoke_check_fma(dip, arg, ctlop, scan);
2516 2580 mutex_exit(err_mutexp);
2517 2581 }
2518 2582 mutex_exit(peek_poke_mutexp);
2519 2583 return (rval);
2520 2584 }
2521 2585
2522 2586 void
2523 2587 impl_setup_ddi(void)
2524 2588 {
2525 2589 #if !defined(__xpv)
2526 2590 extern void startup_bios_disk(void);
2527 2591 extern int post_fastreboot;
2528 2592 #endif
2529 2593 dev_info_t *xdip, *isa_dip;
2530 2594 rd_existing_t rd_mem_prop;
2531 2595 int err;
2532 2596
2533 2597 ndi_devi_alloc_sleep(ddi_root_node(), "ramdisk",
2534 2598 (pnode_t)DEVI_SID_NODEID, &xdip);
2535 2599
2536 2600 (void) BOP_GETPROP(bootops,
2537 2601 "ramdisk_start", (void *)&ramdisk_start);
2538 2602 (void) BOP_GETPROP(bootops,
2539 2603 "ramdisk_end", (void *)&ramdisk_end);
2540 2604
2541 2605 #ifdef __xpv
2542 2606 ramdisk_start -= ONE_GIG;
2543 2607 ramdisk_end -= ONE_GIG;
2544 2608 #endif
2545 2609 rd_mem_prop.phys = ramdisk_start;
2546 2610 rd_mem_prop.size = ramdisk_end - ramdisk_start + 1;
2547 2611
2548 2612 (void) ndi_prop_update_byte_array(DDI_DEV_T_NONE, xdip,
2549 2613 RD_EXISTING_PROP_NAME, (uchar_t *)&rd_mem_prop,
2550 2614 sizeof (rd_mem_prop));
2551 2615 err = ndi_devi_bind_driver(xdip, 0);
2552 2616 ASSERT(err == 0);
2553 2617
2554 2618 /* isa node */
2555 2619 if (pseudo_isa) {
2556 2620 ndi_devi_alloc_sleep(ddi_root_node(), "isa",
2557 2621 (pnode_t)DEVI_SID_NODEID, &isa_dip);
2558 2622 (void) ndi_prop_update_string(DDI_DEV_T_NONE, isa_dip,
2559 2623 "device_type", "isa");
2560 2624 (void) ndi_prop_update_string(DDI_DEV_T_NONE, isa_dip,
2561 2625 "bus-type", "isa");
2562 2626 (void) ndi_devi_bind_driver(isa_dip, 0);
|
↓ open down ↓ |
531 lines elided |
↑ open up ↑ |
2563 2627 }
2564 2628
2565 2629 /*
2566 2630 * Read in the properties from the boot.
2567 2631 */
2568 2632 get_boot_properties();
2569 2633
2570 2634 /* not framebuffer should be enumerated, if present */
2571 2635 get_vga_properties();
2572 2636
2637 + /* Copy console font if provided by boot. */
2638 + get_console_font();
2639 +
2573 2640 /*
2574 2641 * Check for administratively disabled drivers.
2575 2642 */
2576 2643 check_driver_disable();
2577 2644
2578 2645 #if !defined(__xpv)
2579 2646 if (!post_fastreboot && BOP_GETPROPLEN(bootops, "efi-systab") < 0)
2580 2647 startup_bios_disk();
2581 2648 #endif
2582 2649 /* do bus dependent probes. */
2583 2650 impl_bus_initialprobe();
2584 2651 }
2585 2652
2586 2653 dev_t
2587 2654 getrootdev(void)
2588 2655 {
2589 2656 /*
2590 2657 * Usually rootfs.bo_name is initialized by the
2591 2658 * the bootpath property from bootenv.rc, but
2592 2659 * defaults to "/ramdisk:a" otherwise.
2593 2660 */
2594 2661 return (ddi_pathname_to_dev_t(rootfs.bo_name));
2595 2662 }
2596 2663
2597 2664 static struct bus_probe {
2598 2665 struct bus_probe *next;
2599 2666 void (*probe)(int);
2600 2667 } *bus_probes;
2601 2668
2602 2669 void
2603 2670 impl_bus_add_probe(void (*func)(int))
2604 2671 {
2605 2672 struct bus_probe *probe;
2606 2673 struct bus_probe *lastprobe = NULL;
2607 2674
2608 2675 probe = kmem_alloc(sizeof (*probe), KM_SLEEP);
2609 2676 probe->probe = func;
2610 2677 probe->next = NULL;
2611 2678
2612 2679 if (!bus_probes) {
2613 2680 bus_probes = probe;
2614 2681 return;
2615 2682 }
2616 2683
2617 2684 lastprobe = bus_probes;
2618 2685 while (lastprobe->next)
2619 2686 lastprobe = lastprobe->next;
2620 2687 lastprobe->next = probe;
2621 2688 }
2622 2689
2623 2690 /*ARGSUSED*/
2624 2691 void
2625 2692 impl_bus_delete_probe(void (*func)(int))
2626 2693 {
2627 2694 struct bus_probe *prev = NULL;
2628 2695 struct bus_probe *probe = bus_probes;
2629 2696
2630 2697 while (probe) {
2631 2698 if (probe->probe == func)
2632 2699 break;
2633 2700 prev = probe;
2634 2701 probe = probe->next;
2635 2702 }
2636 2703
2637 2704 if (probe == NULL)
2638 2705 return;
2639 2706
2640 2707 if (prev)
2641 2708 prev->next = probe->next;
2642 2709 else
2643 2710 bus_probes = probe->next;
2644 2711
2645 2712 kmem_free(probe, sizeof (struct bus_probe));
2646 2713 }
2647 2714
2648 2715 /*
2649 2716 * impl_bus_initialprobe
2650 2717 * Modload the prom simulator, then let it probe to verify existence
2651 2718 * and type of PCI support.
2652 2719 */
2653 2720 static void
2654 2721 impl_bus_initialprobe(void)
2655 2722 {
2656 2723 struct bus_probe *probe;
2657 2724
2658 2725 /* load modules to install bus probes */
2659 2726 #if defined(__xpv)
2660 2727 if (DOMAIN_IS_INITDOMAIN(xen_info)) {
2661 2728 if (modload("misc", "pci_autoconfig") < 0) {
2662 2729 panic("failed to load misc/pci_autoconfig");
2663 2730 }
2664 2731
2665 2732 if (modload("drv", "isa") < 0)
2666 2733 panic("failed to load drv/isa");
2667 2734 }
2668 2735
2669 2736 (void) modload("misc", "xpv_autoconfig");
2670 2737 #else
2671 2738 if (modload("misc", "pci_autoconfig") < 0) {
2672 2739 panic("failed to load misc/pci_autoconfig");
2673 2740 }
2674 2741
2675 2742 (void) modload("misc", "acpidev");
2676 2743
2677 2744 if (modload("drv", "isa") < 0)
2678 2745 panic("failed to load drv/isa");
2679 2746 #endif
2680 2747
2681 2748 probe = bus_probes;
2682 2749 while (probe) {
2683 2750 /* run the probe functions */
2684 2751 (*probe->probe)(0);
2685 2752 probe = probe->next;
2686 2753 }
2687 2754 }
2688 2755
2689 2756 /*
2690 2757 * impl_bus_reprobe
2691 2758 * Reprogram devices not set up by firmware.
2692 2759 */
2693 2760 static void
2694 2761 impl_bus_reprobe(void)
2695 2762 {
2696 2763 struct bus_probe *probe;
2697 2764
2698 2765 probe = bus_probes;
2699 2766 while (probe) {
2700 2767 /* run the probe function */
2701 2768 (*probe->probe)(1);
2702 2769 probe = probe->next;
2703 2770 }
2704 2771 }
2705 2772
2706 2773
2707 2774 /*
2708 2775 * The following functions ready a cautious request to go up to the nexus
2709 2776 * driver. It is up to the nexus driver to decide how to process the request.
2710 2777 * It may choose to call i_ddi_do_caut_get/put in this file, or do it
2711 2778 * differently.
2712 2779 */
2713 2780
2714 2781 static void
2715 2782 i_ddi_caut_getput_ctlops(ddi_acc_impl_t *hp, uint64_t host_addr,
2716 2783 uint64_t dev_addr, size_t size, size_t repcount, uint_t flags,
2717 2784 ddi_ctl_enum_t cmd)
2718 2785 {
2719 2786 peekpoke_ctlops_t cautacc_ctlops_arg;
2720 2787
2721 2788 cautacc_ctlops_arg.size = size;
2722 2789 cautacc_ctlops_arg.dev_addr = dev_addr;
2723 2790 cautacc_ctlops_arg.host_addr = host_addr;
2724 2791 cautacc_ctlops_arg.handle = (ddi_acc_handle_t)hp;
2725 2792 cautacc_ctlops_arg.repcount = repcount;
2726 2793 cautacc_ctlops_arg.flags = flags;
2727 2794
2728 2795 (void) ddi_ctlops(hp->ahi_common.ah_dip, hp->ahi_common.ah_dip, cmd,
2729 2796 &cautacc_ctlops_arg, NULL);
2730 2797 }
2731 2798
2732 2799 uint8_t
2733 2800 i_ddi_caut_get8(ddi_acc_impl_t *hp, uint8_t *addr)
2734 2801 {
2735 2802 uint8_t value;
2736 2803 i_ddi_caut_getput_ctlops(hp, (uintptr_t)&value, (uintptr_t)addr,
2737 2804 sizeof (uint8_t), 1, 0, DDI_CTLOPS_PEEK);
2738 2805
2739 2806 return (value);
2740 2807 }
2741 2808
2742 2809 uint16_t
2743 2810 i_ddi_caut_get16(ddi_acc_impl_t *hp, uint16_t *addr)
2744 2811 {
2745 2812 uint16_t value;
2746 2813 i_ddi_caut_getput_ctlops(hp, (uintptr_t)&value, (uintptr_t)addr,
2747 2814 sizeof (uint16_t), 1, 0, DDI_CTLOPS_PEEK);
2748 2815
2749 2816 return (value);
2750 2817 }
2751 2818
2752 2819 uint32_t
2753 2820 i_ddi_caut_get32(ddi_acc_impl_t *hp, uint32_t *addr)
2754 2821 {
2755 2822 uint32_t value;
2756 2823 i_ddi_caut_getput_ctlops(hp, (uintptr_t)&value, (uintptr_t)addr,
2757 2824 sizeof (uint32_t), 1, 0, DDI_CTLOPS_PEEK);
2758 2825
2759 2826 return (value);
2760 2827 }
2761 2828
2762 2829 uint64_t
2763 2830 i_ddi_caut_get64(ddi_acc_impl_t *hp, uint64_t *addr)
2764 2831 {
2765 2832 uint64_t value;
2766 2833 i_ddi_caut_getput_ctlops(hp, (uintptr_t)&value, (uintptr_t)addr,
2767 2834 sizeof (uint64_t), 1, 0, DDI_CTLOPS_PEEK);
2768 2835
2769 2836 return (value);
2770 2837 }
2771 2838
2772 2839 void
2773 2840 i_ddi_caut_put8(ddi_acc_impl_t *hp, uint8_t *addr, uint8_t value)
2774 2841 {
2775 2842 i_ddi_caut_getput_ctlops(hp, (uintptr_t)&value, (uintptr_t)addr,
2776 2843 sizeof (uint8_t), 1, 0, DDI_CTLOPS_POKE);
2777 2844 }
2778 2845
2779 2846 void
2780 2847 i_ddi_caut_put16(ddi_acc_impl_t *hp, uint16_t *addr, uint16_t value)
2781 2848 {
2782 2849 i_ddi_caut_getput_ctlops(hp, (uintptr_t)&value, (uintptr_t)addr,
2783 2850 sizeof (uint16_t), 1, 0, DDI_CTLOPS_POKE);
2784 2851 }
2785 2852
2786 2853 void
2787 2854 i_ddi_caut_put32(ddi_acc_impl_t *hp, uint32_t *addr, uint32_t value)
2788 2855 {
2789 2856 i_ddi_caut_getput_ctlops(hp, (uintptr_t)&value, (uintptr_t)addr,
2790 2857 sizeof (uint32_t), 1, 0, DDI_CTLOPS_POKE);
2791 2858 }
2792 2859
2793 2860 void
2794 2861 i_ddi_caut_put64(ddi_acc_impl_t *hp, uint64_t *addr, uint64_t value)
2795 2862 {
2796 2863 i_ddi_caut_getput_ctlops(hp, (uintptr_t)&value, (uintptr_t)addr,
2797 2864 sizeof (uint64_t), 1, 0, DDI_CTLOPS_POKE);
2798 2865 }
2799 2866
2800 2867 void
2801 2868 i_ddi_caut_rep_get8(ddi_acc_impl_t *hp, uint8_t *host_addr, uint8_t *dev_addr,
2802 2869 size_t repcount, uint_t flags)
2803 2870 {
2804 2871 i_ddi_caut_getput_ctlops(hp, (uintptr_t)host_addr, (uintptr_t)dev_addr,
2805 2872 sizeof (uint8_t), repcount, flags, DDI_CTLOPS_PEEK);
2806 2873 }
2807 2874
2808 2875 void
2809 2876 i_ddi_caut_rep_get16(ddi_acc_impl_t *hp, uint16_t *host_addr,
2810 2877 uint16_t *dev_addr, size_t repcount, uint_t flags)
2811 2878 {
2812 2879 i_ddi_caut_getput_ctlops(hp, (uintptr_t)host_addr, (uintptr_t)dev_addr,
2813 2880 sizeof (uint16_t), repcount, flags, DDI_CTLOPS_PEEK);
2814 2881 }
2815 2882
2816 2883 void
2817 2884 i_ddi_caut_rep_get32(ddi_acc_impl_t *hp, uint32_t *host_addr,
2818 2885 uint32_t *dev_addr, size_t repcount, uint_t flags)
2819 2886 {
2820 2887 i_ddi_caut_getput_ctlops(hp, (uintptr_t)host_addr, (uintptr_t)dev_addr,
2821 2888 sizeof (uint32_t), repcount, flags, DDI_CTLOPS_PEEK);
2822 2889 }
2823 2890
2824 2891 void
2825 2892 i_ddi_caut_rep_get64(ddi_acc_impl_t *hp, uint64_t *host_addr,
2826 2893 uint64_t *dev_addr, size_t repcount, uint_t flags)
2827 2894 {
2828 2895 i_ddi_caut_getput_ctlops(hp, (uintptr_t)host_addr, (uintptr_t)dev_addr,
2829 2896 sizeof (uint64_t), repcount, flags, DDI_CTLOPS_PEEK);
2830 2897 }
2831 2898
2832 2899 void
2833 2900 i_ddi_caut_rep_put8(ddi_acc_impl_t *hp, uint8_t *host_addr, uint8_t *dev_addr,
2834 2901 size_t repcount, uint_t flags)
2835 2902 {
2836 2903 i_ddi_caut_getput_ctlops(hp, (uintptr_t)host_addr, (uintptr_t)dev_addr,
2837 2904 sizeof (uint8_t), repcount, flags, DDI_CTLOPS_POKE);
2838 2905 }
2839 2906
2840 2907 void
2841 2908 i_ddi_caut_rep_put16(ddi_acc_impl_t *hp, uint16_t *host_addr,
2842 2909 uint16_t *dev_addr, size_t repcount, uint_t flags)
2843 2910 {
2844 2911 i_ddi_caut_getput_ctlops(hp, (uintptr_t)host_addr, (uintptr_t)dev_addr,
2845 2912 sizeof (uint16_t), repcount, flags, DDI_CTLOPS_POKE);
2846 2913 }
2847 2914
2848 2915 void
2849 2916 i_ddi_caut_rep_put32(ddi_acc_impl_t *hp, uint32_t *host_addr,
2850 2917 uint32_t *dev_addr, size_t repcount, uint_t flags)
2851 2918 {
2852 2919 i_ddi_caut_getput_ctlops(hp, (uintptr_t)host_addr, (uintptr_t)dev_addr,
2853 2920 sizeof (uint32_t), repcount, flags, DDI_CTLOPS_POKE);
2854 2921 }
2855 2922
2856 2923 void
2857 2924 i_ddi_caut_rep_put64(ddi_acc_impl_t *hp, uint64_t *host_addr,
2858 2925 uint64_t *dev_addr, size_t repcount, uint_t flags)
2859 2926 {
2860 2927 i_ddi_caut_getput_ctlops(hp, (uintptr_t)host_addr, (uintptr_t)dev_addr,
2861 2928 sizeof (uint64_t), repcount, flags, DDI_CTLOPS_POKE);
2862 2929 }
2863 2930
2864 2931 boolean_t
2865 2932 i_ddi_copybuf_required(ddi_dma_attr_t *attrp)
2866 2933 {
2867 2934 uint64_t hi_pa;
2868 2935
2869 2936 hi_pa = ((uint64_t)physmax + 1ull) << PAGESHIFT;
2870 2937 if (attrp->dma_attr_addr_hi < hi_pa) {
2871 2938 return (B_TRUE);
2872 2939 }
2873 2940
2874 2941 return (B_FALSE);
2875 2942 }
2876 2943
2877 2944 size_t
2878 2945 i_ddi_copybuf_size()
2879 2946 {
2880 2947 return (dma_max_copybuf_size);
2881 2948 }
2882 2949
2883 2950 /*
2884 2951 * i_ddi_dma_max()
2885 2952 * returns the maximum DMA size which can be performed in a single DMA
2886 2953 * window taking into account the devices DMA contraints (attrp), the
2887 2954 * maximum copy buffer size (if applicable), and the worse case buffer
2888 2955 * fragmentation.
2889 2956 */
2890 2957 /*ARGSUSED*/
2891 2958 uint32_t
2892 2959 i_ddi_dma_max(dev_info_t *dip, ddi_dma_attr_t *attrp)
2893 2960 {
2894 2961 uint64_t maxxfer;
2895 2962
2896 2963
2897 2964 /*
2898 2965 * take the min of maxxfer and the the worse case fragementation
2899 2966 * (e.g. every cookie <= 1 page)
2900 2967 */
2901 2968 maxxfer = MIN(attrp->dma_attr_maxxfer,
2902 2969 ((uint64_t)(attrp->dma_attr_sgllen - 1) << PAGESHIFT));
2903 2970
2904 2971 /*
2905 2972 * If the DMA engine can't reach all off memory, we also need to take
2906 2973 * the max size of the copybuf into consideration.
2907 2974 */
2908 2975 if (i_ddi_copybuf_required(attrp)) {
2909 2976 maxxfer = MIN(i_ddi_copybuf_size(), maxxfer);
2910 2977 }
2911 2978
2912 2979 /*
2913 2980 * we only return a 32-bit value. Make sure it's not -1. Round to a
2914 2981 * page so it won't be mistaken for an error value during debug.
2915 2982 */
2916 2983 if (maxxfer >= 0xFFFFFFFF) {
2917 2984 maxxfer = 0xFFFFF000;
2918 2985 }
2919 2986
2920 2987 /*
2921 2988 * make sure the value we return is a whole multiple of the
2922 2989 * granlarity.
2923 2990 */
2924 2991 if (attrp->dma_attr_granular > 1) {
2925 2992 maxxfer = maxxfer - (maxxfer % attrp->dma_attr_granular);
2926 2993 }
2927 2994
2928 2995 return ((uint32_t)maxxfer);
2929 2996 }
2930 2997
2931 2998 /*ARGSUSED*/
2932 2999 void
2933 3000 translate_devid(dev_info_t *dip)
2934 3001 {
2935 3002 }
2936 3003
2937 3004 pfn_t
2938 3005 i_ddi_paddr_to_pfn(paddr_t paddr)
2939 3006 {
2940 3007 pfn_t pfn;
2941 3008
2942 3009 #ifdef __xpv
2943 3010 if (DOMAIN_IS_INITDOMAIN(xen_info)) {
2944 3011 pfn = xen_assign_pfn(mmu_btop(paddr));
2945 3012 } else {
2946 3013 pfn = mmu_btop(paddr);
2947 3014 }
2948 3015 #else
2949 3016 pfn = mmu_btop(paddr);
2950 3017 #endif
2951 3018
2952 3019 return (pfn);
2953 3020 }
|
↓ open down ↓ |
371 lines elided |
↑ open up ↑ |
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX