1 .\"
2 .\" CDDL HEADER START
3 .\"
4 .\" The contents of this file are subject to the terms of the
5 .\" Common Development and Distribution License (the "License").
6 .\" You may not use this file except in compliance with the License.
7 .\"
8 .\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE
9 .\" or http://www.opensolaris.org/os/licensing.
10 .\" See the License for the specific language governing permissions
11 .\" and limitations under the License.
12 .\"
13 .\" When distributing Covered Code, include this CDDL HEADER in each
14 .\" file and include the License file at usr/src/OPENSOLARIS.LICENSE.
15 .\" If applicable, add the following below this CDDL HEADER, with the
16 .\" fields enclosed by brackets "[]" replaced with your own identifying
17 .\" information: Portions Copyright [yyyy] [name of copyright owner]
18 .\"
19 .\" CDDL HEADER END
20 .\"
21 .\"
22 .\" Copyright (c) 2007, Sun Microsystems, Inc. All Rights Reserved.
23 .\" Copyright (c) 2012, 2017 by Delphix. All rights reserved.
24 .\" Copyright 2017 Nexenta Systems, Inc.
25 .\" Copyright (c) 2017 Datto Inc.
26 .\" Copyright (c) 2017 George Melikov. All Rights Reserved.
27 .\"
28 .Dd April 27, 2018
29 .Dt ZPOOL 1M
30 .Os
31 .Sh NAME
32 .Nm zpool
33 .Nd configure ZFS storage pools
34 .Sh SYNOPSIS
35 .Nm
36 .Fl \?
37 .Nm
38 .Cm add
39 .Op Fl fn
40 .Ar pool vdev Ns ...
41 .Nm
42 .Cm attach
43 .Op Fl f
44 .Ar pool device new_device
45 .Nm
46 .Cm checkpoint
47 .Op Fl d, -discard
48 .Ar pool
49 .Nm
50 .Cm clear
51 .Ar pool
52 .Op Ar device
53 .Nm
54 .Cm create
55 .Op Fl dfn
56 .Op Fl B
57 .Op Fl m Ar mountpoint
58 .Oo Fl o Ar property Ns = Ns Ar value Oc Ns ...
59 .Oo Fl O Ar file-system-property Ns = Ns Ar value Oc Ns ...
60 .Op Fl R Ar root
61 .Op Fl t Ar tempname
62 .Ar pool vdev Ns ...
63 .Nm
64 .Cm destroy
65 .Op Fl f
66 .Ar pool
67 .Nm
68 .Cm detach
69 .Ar pool device
70 .Nm
71 .Cm export
72 .Op Fl f
73 .Ar pool Ns ...
74 .Nm
75 .Cm get
76 .Op Fl Hp
77 .Op Fl o Ar field Ns Oo , Ns Ar field Oc Ns ...
78 .Sy all Ns | Ns Ar property Ns Oo , Ns Ar property Oc Ns ...
79 .Ar pool Ns ...
80 .Nm
81 .Cm history
82 .Op Fl il
83 .Oo Ar pool Oc Ns ...
84 .Nm
85 .Cm import
86 .Op Fl D
87 .Op Fl d Ar dir
88 .Nm
89 .Cm import
90 .Fl a
91 .Op Fl DfmN
92 .Op Fl F Op Fl n
93 .Op Fl c Ar cachefile Ns | Ns Fl d Ar dir
94 .Op Fl o Ar mntopts
95 .Oo Fl o Ar property Ns = Ns Ar value Oc Ns ...
96 .Op Fl R Ar root
97 .Nm
98 .Cm import
99 .Op Fl Dfmt
100 .Op Fl F Op Fl n
101 .Op Fl -rewind-to-checkpoint
102 .Op Fl c Ar cachefile Ns | Ns Fl d Ar dir
103 .Op Fl o Ar mntopts
104 .Oo Fl o Ar property Ns = Ns Ar value Oc Ns ...
105 .Op Fl R Ar root
106 .Ar pool Ns | Ns Ar id
107 .Op Ar newpool
108 .Nm
109 .Cm initialize
110 .Op Fl cs
111 .Ar pool
112 .Op Ar device Ns ...
113 .Nm
114 .Cm iostat
115 .Op Fl v
116 .Op Fl T Sy u Ns | Ns Sy d
117 .Oo Ar pool Oc Ns ...
118 .Op Ar interval Op Ar count
119 .Nm
120 .Cm labelclear
121 .Op Fl f
122 .Ar device
123 .Nm
124 .Cm list
125 .Op Fl Hpv
126 .Op Fl o Ar property Ns Oo , Ns Ar property Oc Ns ...
127 .Op Fl T Sy u Ns | Ns Sy d
128 .Oo Ar pool Oc Ns ...
129 .Op Ar interval Op Ar count
130 .Nm
131 .Cm offline
132 .Op Fl t
133 .Ar pool Ar device Ns ...
134 .Nm
135 .Cm online
136 .Op Fl e
137 .Ar pool Ar device Ns ...
138 .Nm
139 .Cm reguid
140 .Ar pool
141 .Nm
142 .Cm reopen
143 .Ar pool
144 .Nm
145 .Cm remove
146 .Op Fl np
147 .Ar pool Ar device Ns ...
148 .Nm
149 .Cm remove
150 .Fl s
151 .Ar pool
152 .Nm
153 .Cm replace
154 .Op Fl f
155 .Ar pool Ar device Op Ar new_device
156 .Nm
157 .Cm scrub
158 .Op Fl s | Fl p
159 .Ar pool Ns ...
160 .Nm
161 .Cm set
162 .Ar property Ns = Ns Ar value
163 .Ar pool
164 .Nm
165 .Cm split
166 .Op Fl n
167 .Oo Fl o Ar property Ns = Ns Ar value Oc Ns ...
168 .Op Fl R Ar root
169 .Ar pool newpool
170 .Nm
171 .Cm status
172 .Op Fl Dvx
173 .Op Fl T Sy u Ns | Ns Sy d
174 .Oo Ar pool Oc Ns ...
175 .Op Ar interval Op Ar count
176 .Nm
177 .Cm upgrade
178 .Nm
179 .Cm upgrade
180 .Fl v
181 .Nm
182 .Cm upgrade
183 .Op Fl V Ar version
184 .Fl a Ns | Ns Ar pool Ns ...
185 .Sh DESCRIPTION
186 The
187 .Nm
188 command configures ZFS storage pools.
189 A storage pool is a collection of devices that provides physical storage and
190 data replication for ZFS datasets.
191 All datasets within a storage pool share the same space.
192 See
193 .Xr zfs 1M
194 for information on managing datasets.
195 .Ss Virtual Devices (vdevs)
196 A "virtual device" describes a single device or a collection of devices
197 organized according to certain performance and fault characteristics.
198 The following virtual devices are supported:
199 .Bl -tag -width Ds
200 .It Sy disk
201 A block device, typically located under
202 .Pa /dev/dsk .
203 ZFS can use individual slices or partitions, though the recommended mode of
204 operation is to use whole disks.
205 A disk can be specified by a full path, or it can be a shorthand name
206 .Po the relative portion of the path under
207 .Pa /dev/dsk
208 .Pc .
209 A whole disk can be specified by omitting the slice or partition designation.
210 For example,
211 .Pa c0t0d0
212 is equivalent to
213 .Pa /dev/dsk/c0t0d0s2 .
214 When given a whole disk, ZFS automatically labels the disk, if necessary.
215 .It Sy file
216 A regular file.
217 The use of files as a backing store is strongly discouraged.
218 It is designed primarily for experimental purposes, as the fault tolerance of a
219 file is only as good as the file system of which it is a part.
220 A file must be specified by a full path.
221 .It Sy mirror
222 A mirror of two or more devices.
223 Data is replicated in an identical fashion across all components of a mirror.
224 A mirror with N disks of size X can hold X bytes and can withstand (N-1) devices
225 failing before data integrity is compromised.
226 .It Sy raidz , raidz1 , raidz2 , raidz3
227 A variation on RAID-5 that allows for better distribution of parity and
228 eliminates the RAID-5
229 .Qq write hole
230 .Pq in which data and parity become inconsistent after a power loss .
231 Data and parity is striped across all disks within a raidz group.
232 .Pp
233 A raidz group can have single-, double-, or triple-parity, meaning that the
234 raidz group can sustain one, two, or three failures, respectively, without
235 losing any data.
236 The
237 .Sy raidz1
238 vdev type specifies a single-parity raidz group; the
239 .Sy raidz2
240 vdev type specifies a double-parity raidz group; and the
241 .Sy raidz3
242 vdev type specifies a triple-parity raidz group.
243 The
244 .Sy raidz
245 vdev type is an alias for
246 .Sy raidz1 .
247 .Pp
248 A raidz group with N disks of size X with P parity disks can hold approximately
249 (N-P)*X bytes and can withstand P device(s) failing before data integrity is
250 compromised.
251 The minimum number of devices in a raidz group is one more than the number of
252 parity disks.
253 The recommended number is between 3 and 9 to help increase performance.
254 .It Sy spare
255 A special pseudo-vdev which keeps track of available hot spares for a pool.
256 For more information, see the
257 .Sx Hot Spares
258 section.
259 .It Sy log
260 A separate intent log device.
261 If more than one log device is specified, then writes are load-balanced between
262 devices.
263 Log devices can be mirrored.
264 However, raidz vdev types are not supported for the intent log.
265 For more information, see the
266 .Sx Intent Log
267 section.
268 .It Sy cache
269 A device used to cache storage pool data.
270 A cache device cannot be configured as a mirror or raidz group.
271 For more information, see the
272 .Sx Cache Devices
273 section.
274 .El
275 .Pp
276 Virtual devices cannot be nested, so a mirror or raidz virtual device can only
277 contain files or disks.
278 Mirrors of mirrors
279 .Pq or other combinations
280 are not allowed.
281 .Pp
282 A pool can have any number of virtual devices at the top of the configuration
283 .Po known as
284 .Qq root vdevs
285 .Pc .
286 Data is dynamically distributed across all top-level devices to balance data
287 among devices.
288 As new virtual devices are added, ZFS automatically places data on the newly
289 available devices.
290 .Pp
291 Virtual devices are specified one at a time on the command line, separated by
292 whitespace.
293 The keywords
294 .Sy mirror
295 and
296 .Sy raidz
297 are used to distinguish where a group ends and another begins.
298 For example, the following creates two root vdevs, each a mirror of two disks:
299 .Bd -literal
300 # zpool create mypool mirror c0t0d0 c0t1d0 mirror c1t0d0 c1t1d0
301 .Ed
302 .Ss Device Failure and Recovery
303 ZFS supports a rich set of mechanisms for handling device failure and data
304 corruption.
305 All metadata and data is checksummed, and ZFS automatically repairs bad data
306 from a good copy when corruption is detected.
307 .Pp
308 In order to take advantage of these features, a pool must make use of some form
309 of redundancy, using either mirrored or raidz groups.
310 While ZFS supports running in a non-redundant configuration, where each root
311 vdev is simply a disk or file, this is strongly discouraged.
312 A single case of bit corruption can render some or all of your data unavailable.
313 .Pp
314 A pool's health status is described by one of three states: online, degraded,
315 or faulted.
316 An online pool has all devices operating normally.
317 A degraded pool is one in which one or more devices have failed, but the data is
318 still available due to a redundant configuration.
319 A faulted pool has corrupted metadata, or one or more faulted devices, and
320 insufficient replicas to continue functioning.
321 .Pp
322 The health of the top-level vdev, such as mirror or raidz device, is
323 potentially impacted by the state of its associated vdevs, or component
324 devices.
325 A top-level vdev or component device is in one of the following states:
326 .Bl -tag -width "DEGRADED"
327 .It Sy DEGRADED
328 One or more top-level vdevs is in the degraded state because one or more
329 component devices are offline.
330 Sufficient replicas exist to continue functioning.
331 .Pp
332 One or more component devices is in the degraded or faulted state, but
333 sufficient replicas exist to continue functioning.
334 The underlying conditions are as follows:
335 .Bl -bullet
336 .It
337 The number of checksum errors exceeds acceptable levels and the device is
338 degraded as an indication that something may be wrong.
339 ZFS continues to use the device as necessary.
340 .It
341 The number of I/O errors exceeds acceptable levels.
342 The device could not be marked as faulted because there are insufficient
343 replicas to continue functioning.
344 .El
345 .It Sy FAULTED
346 One or more top-level vdevs is in the faulted state because one or more
347 component devices are offline.
348 Insufficient replicas exist to continue functioning.
349 .Pp
350 One or more component devices is in the faulted state, and insufficient
351 replicas exist to continue functioning.
352 The underlying conditions are as follows:
353 .Bl -bullet
354 .It
355 The device could be opened, but the contents did not match expected values.
356 .It
357 The number of I/O errors exceeds acceptable levels and the device is faulted to
358 prevent further use of the device.
359 .El
360 .It Sy OFFLINE
361 The device was explicitly taken offline by the
362 .Nm zpool Cm offline
363 command.
364 .It Sy ONLINE
365 The device is online and functioning.
366 .It Sy REMOVED
367 The device was physically removed while the system was running.
368 Device removal detection is hardware-dependent and may not be supported on all
369 platforms.
370 .It Sy UNAVAIL
371 The device could not be opened.
372 If a pool is imported when a device was unavailable, then the device will be
373 identified by a unique identifier instead of its path since the path was never
374 correct in the first place.
375 .El
376 .Pp
377 If a device is removed and later re-attached to the system, ZFS attempts
378 to put the device online automatically.
379 Device attach detection is hardware-dependent and might not be supported on all
380 platforms.
381 .Ss Hot Spares
382 ZFS allows devices to be associated with pools as
383 .Qq hot spares .
384 These devices are not actively used in the pool, but when an active device
385 fails, it is automatically replaced by a hot spare.
386 To create a pool with hot spares, specify a
387 .Sy spare
388 vdev with any number of devices.
389 For example,
390 .Bd -literal
391 # zpool create pool mirror c0d0 c1d0 spare c2d0 c3d0
392 .Ed
393 .Pp
394 Spares can be shared across multiple pools, and can be added with the
395 .Nm zpool Cm add
396 command and removed with the
397 .Nm zpool Cm remove
398 command.
399 Once a spare replacement is initiated, a new
400 .Sy spare
401 vdev is created within the configuration that will remain there until the
402 original device is replaced.
403 At this point, the hot spare becomes available again if another device fails.
404 .Pp
405 If a pool has a shared spare that is currently being used, the pool can not be
406 exported since other pools may use this shared spare, which may lead to
407 potential data corruption.
408 .Pp
409 An in-progress spare replacement can be cancelled by detaching the hot spare.
410 If the original faulted device is detached, then the hot spare assumes its
411 place in the configuration, and is removed from the spare list of all active
412 pools.
413 .Pp
414 Spares cannot replace log devices.
415 .Ss Intent Log
416 The ZFS Intent Log (ZIL) satisfies POSIX requirements for synchronous
417 transactions.
418 For instance, databases often require their transactions to be on stable storage
419 devices when returning from a system call.
420 NFS and other applications can also use
421 .Xr fsync 3C
422 to ensure data stability.
423 By default, the intent log is allocated from blocks within the main pool.
424 However, it might be possible to get better performance using separate intent
425 log devices such as NVRAM or a dedicated disk.
426 For example:
427 .Bd -literal
428 # zpool create pool c0d0 c1d0 log c2d0
429 .Ed
430 .Pp
431 Multiple log devices can also be specified, and they can be mirrored.
432 See the
433 .Sx EXAMPLES
434 section for an example of mirroring multiple log devices.
435 .Pp
436 Log devices can be added, replaced, attached, detached, and imported and
437 exported as part of the larger pool.
438 Mirrored devices can be removed by specifying the top-level mirror vdev.
439 .Ss Cache Devices
440 Devices can be added to a storage pool as
441 .Qq cache devices .
442 These devices provide an additional layer of caching between main memory and
443 disk.
444 For read-heavy workloads, where the working set size is much larger than what
445 can be cached in main memory, using cache devices allow much more of this
446 working set to be served from low latency media.
447 Using cache devices provides the greatest performance improvement for random
448 read-workloads of mostly static content.
449 .Pp
450 To create a pool with cache devices, specify a
451 .Sy cache
452 vdev with any number of devices.
453 For example:
454 .Bd -literal
455 # zpool create pool c0d0 c1d0 cache c2d0 c3d0
456 .Ed
457 .Pp
458 Cache devices cannot be mirrored or part of a raidz configuration.
459 If a read error is encountered on a cache device, that read I/O is reissued to
460 the original storage pool device, which might be part of a mirrored or raidz
461 configuration.
462 .Pp
463 The content of the cache devices is considered volatile, as is the case with
464 other system caches.
465 .Ss Pool checkpoint
466 Before starting critical procedures that include destructive actions (e.g
467 .Nm zfs Cm destroy
468 ), an administrator can checkpoint the pool's state and in the case of a
469 mistake or failure, rewind the entire pool back to the checkpoint.
470 Otherwise, the checkpoint can be discarded when the procedure has completed
471 successfully.
472 .Pp
473 A pool checkpoint can be thought of as a pool-wide snapshot and should be used
474 with care as it contains every part of the pool's state, from properties to vdev
475 configuration.
476 Thus, while a pool has a checkpoint certain operations are not allowed.
477 Specifically, vdev removal/attach/detach, mirror splitting, and
478 changing the pool's guid.
479 Adding a new vdev is supported but in the case of a rewind it will have to be
480 added again.
481 Finally, users of this feature should keep in mind that scrubs in a pool that
482 has a checkpoint do not repair checkpointed data.
483 .Pp
484 To create a checkpoint for a pool:
485 .Bd -literal
486 # zpool checkpoint pool
487 .Ed
488 .Pp
489 To later rewind to its checkpointed state, you need to first export it and
490 then rewind it during import:
491 .Bd -literal
492 # zpool export pool
493 # zpool import --rewind-to-checkpoint pool
494 .Ed
495 .Pp
496 To discard the checkpoint from a pool:
497 .Bd -literal
498 # zpool checkpoint -d pool
499 .Ed
500 .Pp
501 Dataset reservations (controlled by the
502 .Nm reservation
503 or
504 .Nm refreservation
505 zfs properties) may be unenforceable while a checkpoint exists, because the
506 checkpoint is allowed to consume the dataset's reservation.
507 Finally, data that is part of the checkpoint but has been freed in the
508 current state of the pool won't be scanned during a scrub.
509 .Ss Properties
510 Each pool has several properties associated with it.
511 Some properties are read-only statistics while others are configurable and
512 change the behavior of the pool.
513 .Pp
514 The following are read-only properties:
515 .Bl -tag -width Ds
516 .It Cm allocated
517 Amount of storage space used within the pool.
518 .It Sy bootsize
519 The size of the system boot partition.
520 This property can only be set at pool creation time and is read-only once pool
521 is created.
522 Setting this property implies using the
523 .Fl B
524 option.
525 .It Sy capacity
526 Percentage of pool space used.
527 This property can also be referred to by its shortened column name,
528 .Sy cap .
529 .It Sy expandsize
530 Amount of uninitialized space within the pool or device that can be used to
531 increase the total capacity of the pool.
532 Uninitialized space consists of any space on an EFI labeled vdev which has not
533 been brought online
534 .Po e.g, using
535 .Nm zpool Cm online Fl e
536 .Pc .
537 This space occurs when a LUN is dynamically expanded.
538 .It Sy fragmentation
539 The amount of fragmentation in the pool.
540 .It Sy free
541 The amount of free space available in the pool.
542 .It Sy freeing
543 After a file system or snapshot is destroyed, the space it was using is
544 returned to the pool asynchronously.
545 .Sy freeing
546 is the amount of space remaining to be reclaimed.
547 Over time
548 .Sy freeing
549 will decrease while
550 .Sy free
551 increases.
552 .It Sy health
553 The current health of the pool.
554 Health can be one of
555 .Sy ONLINE , DEGRADED , FAULTED , OFFLINE, REMOVED , UNAVAIL .
556 .It Sy guid
557 A unique identifier for the pool.
558 .It Sy size
559 Total size of the storage pool.
560 .It Sy unsupported@ Ns Em feature_guid
561 Information about unsupported features that are enabled on the pool.
562 See
563 .Xr zpool-features 5
564 for details.
565 .El
566 .Pp
567 The space usage properties report actual physical space available to the
568 storage pool.
569 The physical space can be different from the total amount of space that any
570 contained datasets can actually use.
571 The amount of space used in a raidz configuration depends on the characteristics
572 of the data being written.
573 In addition, ZFS reserves some space for internal accounting that the
574 .Xr zfs 1M
575 command takes into account, but the
576 .Nm
577 command does not.
578 For non-full pools of a reasonable size, these effects should be invisible.
579 For small pools, or pools that are close to being completely full, these
580 discrepancies may become more noticeable.
581 .Pp
582 The following property can be set at creation time and import time:
583 .Bl -tag -width Ds
584 .It Sy altroot
585 Alternate root directory.
586 If set, this directory is prepended to any mount points within the pool.
587 This can be used when examining an unknown pool where the mount points cannot be
588 trusted, or in an alternate boot environment, where the typical paths are not
589 valid.
590 .Sy altroot
591 is not a persistent property.
592 It is valid only while the system is up.
593 Setting
594 .Sy altroot
595 defaults to using
596 .Sy cachefile Ns = Ns Sy none ,
597 though this may be overridden using an explicit setting.
598 .El
599 .Pp
600 The following property can be set only at import time:
601 .Bl -tag -width Ds
602 .It Sy readonly Ns = Ns Sy on Ns | Ns Sy off
603 If set to
604 .Sy on ,
605 the pool will be imported in read-only mode.
606 This property can also be referred to by its shortened column name,
607 .Sy rdonly .
608 .El
609 .Pp
610 The following properties can be set at creation time and import time, and later
611 changed with the
612 .Nm zpool Cm set
613 command:
614 .Bl -tag -width Ds
615 .It Sy autoexpand Ns = Ns Sy on Ns | Ns Sy off
616 Controls automatic pool expansion when the underlying LUN is grown.
617 If set to
618 .Sy on ,
619 the pool will be resized according to the size of the expanded device.
620 If the device is part of a mirror or raidz then all devices within that
621 mirror/raidz group must be expanded before the new space is made available to
622 the pool.
623 The default behavior is
624 .Sy off .
625 This property can also be referred to by its shortened column name,
626 .Sy expand .
627 .It Sy autoreplace Ns = Ns Sy on Ns | Ns Sy off
628 Controls automatic device replacement.
629 If set to
630 .Sy off ,
631 device replacement must be initiated by the administrator by using the
632 .Nm zpool Cm replace
633 command.
634 If set to
635 .Sy on ,
636 any new device, found in the same physical location as a device that previously
637 belonged to the pool, is automatically formatted and replaced.
638 The default behavior is
639 .Sy off .
640 This property can also be referred to by its shortened column name,
641 .Sy replace .
642 .It Sy bootfs Ns = Ns Ar pool Ns / Ns Ar dataset
643 Identifies the default bootable dataset for the root pool.
644 This property is expected to be set mainly by the installation and upgrade
645 programs.
646 .It Sy cachefile Ns = Ns Ar path Ns | Ns Sy none
647 Controls the location of where the pool configuration is cached.
648 Discovering all pools on system startup requires a cached copy of the
649 configuration data that is stored on the root file system.
650 All pools in this cache are automatically imported when the system boots.
651 Some environments, such as install and clustering, need to cache this
652 information in a different location so that pools are not automatically
653 imported.
654 Setting this property caches the pool configuration in a different location that
655 can later be imported with
656 .Nm zpool Cm import Fl c .
657 Setting it to the special value
658 .Sy none
659 creates a temporary pool that is never cached, and the special value
660 .Qq
661 .Pq empty string
662 uses the default location.
663 .Pp
664 Multiple pools can share the same cache file.
665 Because the kernel destroys and recreates this file when pools are added and
666 removed, care should be taken when attempting to access this file.
667 When the last pool using a
668 .Sy cachefile
669 is exported or destroyed, the file is removed.
670 .It Sy comment Ns = Ns Ar text
671 A text string consisting of printable ASCII characters that will be stored
672 such that it is available even if the pool becomes faulted.
673 An administrator can provide additional information about a pool using this
674 property.
675 .It Sy dedupditto Ns = Ns Ar number
676 Threshold for the number of block ditto copies.
677 If the reference count for a deduplicated block increases above this number, a
678 new ditto copy of this block is automatically stored.
679 The default setting is
680 .Sy 0
681 which causes no ditto copies to be created for deduplicated blocks.
682 The minimum legal nonzero setting is
683 .Sy 100 .
684 .It Sy delegation Ns = Ns Sy on Ns | Ns Sy off
685 Controls whether a non-privileged user is granted access based on the dataset
686 permissions defined on the dataset.
687 See
688 .Xr zfs 1M
689 for more information on ZFS delegated administration.
690 .It Sy failmode Ns = Ns Sy wait Ns | Ns Sy continue Ns | Ns Sy panic
691 Controls the system behavior in the event of catastrophic pool failure.
692 This condition is typically a result of a loss of connectivity to the underlying
693 storage device(s) or a failure of all devices within the pool.
694 The behavior of such an event is determined as follows:
695 .Bl -tag -width "continue"
696 .It Sy wait
697 Blocks all I/O access until the device connectivity is recovered and the errors
698 are cleared.
699 This is the default behavior.
700 .It Sy continue
701 Returns
702 .Er EIO
703 to any new write I/O requests but allows reads to any of the remaining healthy
704 devices.
705 Any write requests that have yet to be committed to disk would be blocked.
706 .It Sy panic
707 Prints out a message to the console and generates a system crash dump.
708 .El
709 .It Sy feature@ Ns Ar feature_name Ns = Ns Sy enabled
710 The value of this property is the current state of
711 .Ar feature_name .
712 The only valid value when setting this property is
713 .Sy enabled
714 which moves
715 .Ar feature_name
716 to the enabled state.
717 See
718 .Xr zpool-features 5
719 for details on feature states.
720 .It Sy listsnapshots Ns = Ns Sy on Ns | Ns Sy off
721 Controls whether information about snapshots associated with this pool is
722 output when
723 .Nm zfs Cm list
724 is run without the
725 .Fl t
726 option.
727 The default value is
728 .Sy off .
729 This property can also be referred to by its shortened name,
730 .Sy listsnaps .
731 .It Sy version Ns = Ns Ar version
732 The current on-disk version of the pool.
733 This can be increased, but never decreased.
734 The preferred method of updating pools is with the
735 .Nm zpool Cm upgrade
736 command, though this property can be used when a specific version is needed for
737 backwards compatibility.
738 Once feature flags are enabled on a pool this property will no longer have a
739 value.
740 .El
741 .Ss Subcommands
742 All subcommands that modify state are logged persistently to the pool in their
743 original form.
744 .Pp
745 The
746 .Nm
747 command provides subcommands to create and destroy storage pools, add capacity
748 to storage pools, and provide information about the storage pools.
749 The following subcommands are supported:
750 .Bl -tag -width Ds
751 .It Xo
752 .Nm
753 .Fl \?
754 .Xc
755 Displays a help message.
756 .It Xo
757 .Nm
758 .Cm add
759 .Op Fl fn
760 .Ar pool vdev Ns ...
761 .Xc
762 Adds the specified virtual devices to the given pool.
763 The
764 .Ar vdev
765 specification is described in the
766 .Sx Virtual Devices
767 section.
768 The behavior of the
769 .Fl f
770 option, and the device checks performed are described in the
771 .Nm zpool Cm create
772 subcommand.
773 .Bl -tag -width Ds
774 .It Fl f
775 Forces use of
776 .Ar vdev Ns s ,
777 even if they appear in use or specify a conflicting replication level.
778 Not all devices can be overridden in this manner.
779 .It Fl n
780 Displays the configuration that would be used without actually adding the
781 .Ar vdev Ns s .
782 The actual pool creation can still fail due to insufficient privileges or
783 device sharing.
784 .El
785 .It Xo
786 .Nm
787 .Cm attach
788 .Op Fl f
789 .Ar pool device new_device
790 .Xc
791 Attaches
792 .Ar new_device
793 to the existing
794 .Ar device .
795 The existing device cannot be part of a raidz configuration.
796 If
797 .Ar device
798 is not currently part of a mirrored configuration,
799 .Ar device
800 automatically transforms into a two-way mirror of
801 .Ar device
802 and
803 .Ar new_device .
804 If
805 .Ar device
806 is part of a two-way mirror, attaching
807 .Ar new_device
808 creates a three-way mirror, and so on.
809 In either case,
810 .Ar new_device
811 begins to resilver immediately.
812 .Bl -tag -width Ds
813 .It Fl f
814 Forces use of
815 .Ar new_device ,
816 even if its appears to be in use.
817 Not all devices can be overridden in this manner.
818 .El
819 .It Xo
820 .Nm
821 .Cm checkpoint
822 .Op Fl d, -discard
823 .Ar pool
824 .Xc
825 Checkpoints the current state of
826 .Ar pool
827 , which can be later restored by
828 .Nm zpool Cm import --rewind-to-checkpoint .
829 The existence of a checkpoint in a pool prohibits the following
830 .Nm zpool
831 commands:
832 .Cm remove ,
833 .Cm attach ,
834 .Cm detach ,
835 .Cm split ,
836 and
837 .Cm reguid .
838 In addition, it may break reservation boundaries if the pool lacks free
839 space.
840 The
841 .Nm zpool Cm status
842 command indicates the existence of a checkpoint or the progress of discarding a
843 checkpoint from a pool.
844 The
845 .Nm zpool Cm list
846 command reports how much space the checkpoint takes from the pool.
847 .Bl -tag -width Ds
848 .It Fl d, -discard
849 Discards an existing checkpoint from
850 .Ar pool .
851 .El
852 .It Xo
853 .Nm
854 .Cm clear
855 .Ar pool
856 .Op Ar device
857 .Xc
858 Clears device errors in a pool.
859 If no arguments are specified, all device errors within the pool are cleared.
860 If one or more devices is specified, only those errors associated with the
861 specified device or devices are cleared.
862 .It Xo
863 .Nm
864 .Cm create
865 .Op Fl dfn
866 .Op Fl B
867 .Op Fl m Ar mountpoint
868 .Oo Fl o Ar property Ns = Ns Ar value Oc Ns ...
869 .Oo Fl O Ar file-system-property Ns = Ns Ar value Oc Ns ...
870 .Op Fl R Ar root
871 .Op Fl t Ar tempname
872 .Ar pool vdev Ns ...
873 .Xc
874 Creates a new storage pool containing the virtual devices specified on the
875 command line.
876 The pool name must begin with a letter, and can only contain
877 alphanumeric characters as well as underscore
878 .Pq Qq Sy _ ,
879 dash
880 .Pq Qq Sy - ,
881 and period
882 .Pq Qq Sy \&. .
883 The pool names
884 .Sy mirror ,
885 .Sy raidz ,
886 .Sy spare
887 and
888 .Sy log
889 are reserved, as are names beginning with the pattern
890 .Sy c[0-9] .
891 The
892 .Ar vdev
893 specification is described in the
894 .Sx Virtual Devices
895 section.
896 .Pp
897 The command verifies that each device specified is accessible and not currently
898 in use by another subsystem.
899 There are some uses, such as being currently mounted, or specified as the
900 dedicated dump device, that prevents a device from ever being used by ZFS.
901 Other uses, such as having a preexisting UFS file system, can be overridden with
902 the
903 .Fl f
904 option.
905 .Pp
906 The command also checks that the replication strategy for the pool is
907 consistent.
908 An attempt to combine redundant and non-redundant storage in a single pool, or
909 to mix disks and files, results in an error unless
910 .Fl f
911 is specified.
912 The use of differently sized devices within a single raidz or mirror group is
913 also flagged as an error unless
914 .Fl f
915 is specified.
916 .Pp
917 Unless the
918 .Fl R
919 option is specified, the default mount point is
920 .Pa / Ns Ar pool .
921 The mount point must not exist or must be empty, or else the root dataset
922 cannot be mounted.
923 This can be overridden with the
924 .Fl m
925 option.
926 .Pp
927 By default all supported features are enabled on the new pool unless the
928 .Fl d
929 option is specified.
930 .Bl -tag -width Ds
931 .It Fl B
932 Create whole disk pool with EFI System partition to support booting system
933 with UEFI firmware.
934 Default size is 256MB.
935 To create boot partition with custom size, set the
936 .Sy bootsize
937 property with the
938 .Fl o
939 option.
940 See the
941 .Sx Properties
942 section for details.
943 .It Fl d
944 Do not enable any features on the new pool.
945 Individual features can be enabled by setting their corresponding properties to
946 .Sy enabled
947 with the
948 .Fl o
949 option.
950 See
951 .Xr zpool-features 5
952 for details about feature properties.
953 .It Fl f
954 Forces use of
955 .Ar vdev Ns s ,
956 even if they appear in use or specify a conflicting replication level.
957 Not all devices can be overridden in this manner.
958 .It Fl m Ar mountpoint
959 Sets the mount point for the root dataset.
960 The default mount point is
961 .Pa /pool
962 or
963 .Pa altroot/pool
964 if
965 .Ar altroot
966 is specified.
967 The mount point must be an absolute path,
968 .Sy legacy ,
969 or
970 .Sy none .
971 For more information on dataset mount points, see
972 .Xr zfs 1M .
973 .It Fl n
974 Displays the configuration that would be used without actually creating the
975 pool.
976 The actual pool creation can still fail due to insufficient privileges or
977 device sharing.
978 .It Fl o Ar property Ns = Ns Ar value
979 Sets the given pool properties.
980 See the
981 .Sx Properties
982 section for a list of valid properties that can be set.
983 .It Fl O Ar file-system-property Ns = Ns Ar value
984 Sets the given file system properties in the root file system of the pool.
985 See the
986 .Sx Properties
987 section of
988 .Xr zfs 1M
989 for a list of valid properties that can be set.
990 .It Fl R Ar root
991 Equivalent to
992 .Fl o Sy cachefile Ns = Ns Sy none Fl o Sy altroot Ns = Ns Ar root
993 .It Fl t Ar tempname
994 Sets the in-core pool name to
995 .Pa tempname
996 while the on-disk name will be the name specified as the pool name
997 .Pa pool .
998 This will set the default cachefile property to
999 .Sy none.
1000 This is intended to handle name space collisions when creating pools
1001 for other systems, such as virtual machines or physical machines
1002 whose pools live on network block devices.
1003 .El
1004 .It Xo
1005 .Nm
1006 .Cm destroy
1007 .Op Fl f
1008 .Ar pool
1009 .Xc
1010 Destroys the given pool, freeing up any devices for other use.
1011 This command tries to unmount any active datasets before destroying the pool.
1012 .Bl -tag -width Ds
1013 .It Fl f
1014 Forces any active datasets contained within the pool to be unmounted.
1015 .El
1016 .It Xo
1017 .Nm
1018 .Cm detach
1019 .Ar pool device
1020 .Xc
1021 Detaches
1022 .Ar device
1023 from a mirror.
1024 The operation is refused if there are no other valid replicas of the data.
1025 .It Xo
1026 .Nm
1027 .Cm export
1028 .Op Fl f
1029 .Ar pool Ns ...
1030 .Xc
1031 Exports the given pools from the system.
1032 All devices are marked as exported, but are still considered in use by other
1033 subsystems.
1034 The devices can be moved between systems
1035 .Pq even those of different endianness
1036 and imported as long as a sufficient number of devices are present.
1037 .Pp
1038 Before exporting the pool, all datasets within the pool are unmounted.
1039 A pool can not be exported if it has a shared spare that is currently being
1040 used.
1041 .Pp
1042 For pools to be portable, you must give the
1043 .Nm
1044 command whole disks, not just slices, so that ZFS can label the disks with
1045 portable EFI labels.
1046 Otherwise, disk drivers on platforms of different endianness will not recognize
1047 the disks.
1048 .Bl -tag -width Ds
1049 .It Fl f
1050 Forcefully unmount all datasets, using the
1051 .Nm unmount Fl f
1052 command.
1053 .Pp
1054 This command will forcefully export the pool even if it has a shared spare that
1055 is currently being used.
1056 This may lead to potential data corruption.
1057 .El
1058 .It Xo
1059 .Nm
1060 .Cm get
1061 .Op Fl Hp
1062 .Op Fl o Ar field Ns Oo , Ns Ar field Oc Ns ...
1063 .Sy all Ns | Ns Ar property Ns Oo , Ns Ar property Oc Ns ...
1064 .Ar pool Ns ...
1065 .Xc
1066 Retrieves the given list of properties
1067 .Po
1068 or all properties if
1069 .Sy all
1070 is used
1071 .Pc
1072 for the specified storage pool(s).
1073 These properties are displayed with the following fields:
1074 .Bd -literal
1075 name Name of storage pool
1076 property Property name
1077 value Property value
1078 source Property source, either 'default' or 'local'.
1079 .Ed
1080 .Pp
1081 See the
1082 .Sx Properties
1083 section for more information on the available pool properties.
1084 .Bl -tag -width Ds
1085 .It Fl H
1086 Scripted mode.
1087 Do not display headers, and separate fields by a single tab instead of arbitrary
1088 space.
1089 .It Fl o Ar field
1090 A comma-separated list of columns to display.
1091 .Sy name Ns \&, Ns Sy property Ns \&, Ns Sy value Ns \&, Ns Sy source
1092 is the default value.
1093 .It Fl p
1094 Display numbers in parsable (exact) values.
1095 .El
1096 .It Xo
1097 .Nm
1098 .Cm history
1099 .Op Fl il
1100 .Oo Ar pool Oc Ns ...
1101 .Xc
1102 Displays the command history of the specified pool(s) or all pools if no pool is
1103 specified.
1104 .Bl -tag -width Ds
1105 .It Fl i
1106 Displays internally logged ZFS events in addition to user initiated events.
1107 .It Fl l
1108 Displays log records in long format, which in addition to standard format
1109 includes, the user name, the hostname, and the zone in which the operation was
1110 performed.
1111 .El
1112 .It Xo
1113 .Nm
1114 .Cm import
1115 .Op Fl D
1116 .Op Fl d Ar dir
1117 .Xc
1118 Lists pools available to import.
1119 If the
1120 .Fl d
1121 option is not specified, this command searches for devices in
1122 .Pa /dev/dsk .
1123 The
1124 .Fl d
1125 option can be specified multiple times, and all directories are searched.
1126 If the device appears to be part of an exported pool, this command displays a
1127 summary of the pool with the name of the pool, a numeric identifier, as well as
1128 the vdev layout and current health of the device for each device or file.
1129 Destroyed pools, pools that were previously destroyed with the
1130 .Nm zpool Cm destroy
1131 command, are not listed unless the
1132 .Fl D
1133 option is specified.
1134 .Pp
1135 The numeric identifier is unique, and can be used instead of the pool name when
1136 multiple exported pools of the same name are available.
1137 .Bl -tag -width Ds
1138 .It Fl c Ar cachefile
1139 Reads configuration from the given
1140 .Ar cachefile
1141 that was created with the
1142 .Sy cachefile
1143 pool property.
1144 This
1145 .Ar cachefile
1146 is used instead of searching for devices.
1147 .It Fl d Ar dir
1148 Searches for devices or files in
1149 .Ar dir .
1150 The
1151 .Fl d
1152 option can be specified multiple times.
1153 .It Fl D
1154 Lists destroyed pools only.
1155 .El
1156 .It Xo
1157 .Nm
1158 .Cm import
1159 .Fl a
1160 .Op Fl DfmN
1161 .Op Fl F Op Fl n
1162 .Op Fl c Ar cachefile Ns | Ns Fl d Ar dir
1163 .Op Fl o Ar mntopts
1164 .Oo Fl o Ar property Ns = Ns Ar value Oc Ns ...
1165 .Op Fl R Ar root
1166 .Xc
1167 Imports all pools found in the search directories.
1168 Identical to the previous command, except that all pools with a sufficient
1169 number of devices available are imported.
1170 Destroyed pools, pools that were previously destroyed with the
1171 .Nm zpool Cm destroy
1172 command, will not be imported unless the
1173 .Fl D
1174 option is specified.
1175 .Bl -tag -width Ds
1176 .It Fl a
1177 Searches for and imports all pools found.
1178 .It Fl c Ar cachefile
1179 Reads configuration from the given
1180 .Ar cachefile
1181 that was created with the
1182 .Sy cachefile
1183 pool property.
1184 This
1185 .Ar cachefile
1186 is used instead of searching for devices.
1187 .It Fl d Ar dir
1188 Searches for devices or files in
1189 .Ar dir .
1190 The
1191 .Fl d
1192 option can be specified multiple times.
1193 This option is incompatible with the
1194 .Fl c
1195 option.
1196 .It Fl D
1197 Imports destroyed pools only.
1198 The
1199 .Fl f
1200 option is also required.
1201 .It Fl f
1202 Forces import, even if the pool appears to be potentially active.
1203 .It Fl F
1204 Recovery mode for a non-importable pool.
1205 Attempt to return the pool to an importable state by discarding the last few
1206 transactions.
1207 Not all damaged pools can be recovered by using this option.
1208 If successful, the data from the discarded transactions is irretrievably lost.
1209 This option is ignored if the pool is importable or already imported.
1210 .It Fl m
1211 Allows a pool to import when there is a missing log device.
1212 Recent transactions can be lost because the log device will be discarded.
1213 .It Fl n
1214 Used with the
1215 .Fl F
1216 recovery option.
1217 Determines whether a non-importable pool can be made importable again, but does
1218 not actually perform the pool recovery.
1219 For more details about pool recovery mode, see the
1220 .Fl F
1221 option, above.
1222 .It Fl N
1223 Import the pool without mounting any file systems.
1224 .It Fl o Ar mntopts
1225 Comma-separated list of mount options to use when mounting datasets within the
1226 pool.
1227 See
1228 .Xr zfs 1M
1229 for a description of dataset properties and mount options.
1230 .It Fl o Ar property Ns = Ns Ar value
1231 Sets the specified property on the imported pool.
1232 See the
1233 .Sx Properties
1234 section for more information on the available pool properties.
1235 .It Fl R Ar root
1236 Sets the
1237 .Sy cachefile
1238 property to
1239 .Sy none
1240 and the
1241 .Sy altroot
1242 property to
1243 .Ar root .
1244 .El
1245 .It Xo
1246 .Nm
1247 .Cm import
1248 .Op Fl Dfmt
1249 .Op Fl F Op Fl n
1250 .Op Fl -rewind-to-checkpoint
1251 .Op Fl c Ar cachefile Ns | Ns Fl d Ar dir
1252 .Op Fl o Ar mntopts
1253 .Oo Fl o Ar property Ns = Ns Ar value Oc Ns ...
1254 .Op Fl R Ar root
1255 .Ar pool Ns | Ns Ar id
1256 .Op Ar newpool
1257 .Xc
1258 Imports a specific pool.
1259 A pool can be identified by its name or the numeric identifier.
1260 If
1261 .Ar newpool
1262 is specified, the pool is imported using the name
1263 .Ar newpool .
1264 Otherwise, it is imported with the same name as its exported name.
1265 .Pp
1266 If a device is removed from a system without running
1267 .Nm zpool Cm export
1268 first, the device appears as potentially active.
1269 It cannot be determined if this was a failed export, or whether the device is
1270 really in use from another host.
1271 To import a pool in this state, the
1272 .Fl f
1273 option is required.
1274 .Bl -tag -width Ds
1275 .It Fl c Ar cachefile
1276 Reads configuration from the given
1277 .Ar cachefile
1278 that was created with the
1279 .Sy cachefile
1280 pool property.
1281 This
1282 .Ar cachefile
1283 is used instead of searching for devices.
1284 .It Fl d Ar dir
1285 Searches for devices or files in
1286 .Ar dir .
1287 The
1288 .Fl d
1289 option can be specified multiple times.
1290 This option is incompatible with the
1291 .Fl c
1292 option.
1293 .It Fl D
1294 Imports destroyed pool.
1295 The
1296 .Fl f
1297 option is also required.
1298 .It Fl f
1299 Forces import, even if the pool appears to be potentially active.
1300 .It Fl F
1301 Recovery mode for a non-importable pool.
1302 Attempt to return the pool to an importable state by discarding the last few
1303 transactions.
1304 Not all damaged pools can be recovered by using this option.
1305 If successful, the data from the discarded transactions is irretrievably lost.
1306 This option is ignored if the pool is importable or already imported.
1307 .It Fl m
1308 Allows a pool to import when there is a missing log device.
1309 Recent transactions can be lost because the log device will be discarded.
1310 .It Fl n
1311 Used with the
1312 .Fl F
1313 recovery option.
1314 Determines whether a non-importable pool can be made importable again, but does
1315 not actually perform the pool recovery.
1316 For more details about pool recovery mode, see the
1317 .Fl F
1318 option, above.
1319 .It Fl o Ar mntopts
1320 Comma-separated list of mount options to use when mounting datasets within the
1321 pool.
1322 See
1323 .Xr zfs 1M
1324 for a description of dataset properties and mount options.
1325 .It Fl o Ar property Ns = Ns Ar value
1326 Sets the specified property on the imported pool.
1327 See the
1328 .Sx Properties
1329 section for more information on the available pool properties.
1330 .It Fl R Ar root
1331 Sets the
1332 .Sy cachefile
1333 property to
1334 .Sy none
1335 and the
1336 .Sy altroot
1337 property to
1338 .Ar root .
1339 .It Fl t
1340 Used with
1341 .Ar newpool .
1342 Specifies that
1343 .Ar newpool
1344 is temporary.
1345 Temporary pool names last until export.
1346 Ensures that the original pool name will be used in all label updates and
1347 therefore is retained upon export.
1348 Will also set
1349 .Sy cachefile
1350 property to
1351 .Sy none
1352 when not explicitly specified.
1353 .It Fl -rewind-to-checkpoint
1354 Rewinds pool to the checkpointed state.
1355 Once the pool is imported with this flag there is no way to undo the rewind.
1356 All changes and data that were written after the checkpoint are lost!
1357 The only exception is when the
1358 .Sy readonly
1359 mounting option is enabled.
1360 In this case, the checkpointed state of the pool is opened and an
1361 administrator can see how the pool would look like if they were
1362 to fully rewind.
1363 .El
1364 .It Xo
1365 .Nm
1366 .Cm initialize
1367 .Op Fl cs
1368 .Ar pool
1369 .Op Ar device Ns ...
1370 .Xc
1371 Begins initializing by writing to all unallocated regions on the specified
1372 devices, or all eligible devices in the pool if no individual devices are
1373 specified.
1374 Only leaf data or log devices may be initialized.
1375 .Bl -tag -width Ds
1376 .It Fl c, -cancel
1377 Cancel initializing on the specified devices, or all eligible devices if none
1378 are specified.
1379 If one or more target devices are invalid or are not currently being
1380 initialized, the command will fail and no cancellation will occur on any device.
1381 .It Fl s -suspend
1382 Suspend initializing on the specified devices, or all eligible devices if none
1383 are specified.
1384 If one or more target devices are invalid or are not currently being
1385 initialized, the command will fail and no suspension will occur on any device.
1386 Initializing can then be resumed by running
1387 .Nm zpool Cm initialize
1388 with no flags on the relevant target devices.
1389 .El
1390 .It Xo
1391 .Nm
1392 .Cm iostat
1393 .Op Fl v
1394 .Op Fl T Sy u Ns | Ns Sy d
1395 .Oo Ar pool Oc Ns ...
1396 .Op Ar interval Op Ar count
1397 .Xc
1398 Displays I/O statistics for the given pools.
1399 When given an
1400 .Ar interval ,
1401 the statistics are printed every
1402 .Ar interval
1403 seconds until ^C is pressed.
1404 If no
1405 .Ar pool Ns s
1406 are specified, statistics for every pool in the system is shown.
1407 If
1408 .Ar count
1409 is specified, the command exits after
1410 .Ar count
1411 reports are printed.
1412 .Bl -tag -width Ds
1413 .It Fl T Sy u Ns | Ns Sy d
1414 Display a time stamp.
1415 Specify
1416 .Sy u
1417 for a printed representation of the internal representation of time.
1418 See
1419 .Xr time 2 .
1420 Specify
1421 .Sy d
1422 for standard date format.
1423 See
1424 .Xr date 1 .
1425 .It Fl v
1426 Verbose statistics Reports usage statistics for individual vdevs within the
1427 pool, in addition to the pool-wide statistics.
1428 .El
1429 .It Xo
1430 .Nm
1431 .Cm labelclear
1432 .Op Fl f
1433 .Ar device
1434 .Xc
1435 Removes ZFS label information from the specified
1436 .Ar device .
1437 The
1438 .Ar device
1439 must not be part of an active pool configuration.
1440 .Bl -tag -width Ds
1441 .It Fl f
1442 Treat exported or foreign devices as inactive.
1443 .El
1444 .It Xo
1445 .Nm
1446 .Cm list
1447 .Op Fl Hpv
1448 .Op Fl o Ar property Ns Oo , Ns Ar property Oc Ns ...
1449 .Op Fl T Sy u Ns | Ns Sy d
1450 .Oo Ar pool Oc Ns ...
1451 .Op Ar interval Op Ar count
1452 .Xc
1453 Lists the given pools along with a health status and space usage.
1454 If no
1455 .Ar pool Ns s
1456 are specified, all pools in the system are listed.
1457 When given an
1458 .Ar interval ,
1459 the information is printed every
1460 .Ar interval
1461 seconds until ^C is pressed.
1462 If
1463 .Ar count
1464 is specified, the command exits after
1465 .Ar count
1466 reports are printed.
1467 .Bl -tag -width Ds
1468 .It Fl H
1469 Scripted mode.
1470 Do not display headers, and separate fields by a single tab instead of arbitrary
1471 space.
1472 .It Fl o Ar property
1473 Comma-separated list of properties to display.
1474 See the
1475 .Sx Properties
1476 section for a list of valid properties.
1477 The default list is
1478 .Cm name , size , allocated , free , checkpoint, expandsize , fragmentation , capacity ,
1479 .Cm dedupratio , health , altroot .
1480 .It Fl p
1481 Display numbers in parsable
1482 .Pq exact
1483 values.
1484 .It Fl T Sy u Ns | Ns Sy d
1485 Display a time stamp.
1486 Specify
1487 .Fl u
1488 for a printed representation of the internal representation of time.
1489 See
1490 .Xr time 2 .
1491 Specify
1492 .Fl d
1493 for standard date format.
1494 See
1495 .Xr date 1 .
1496 .It Fl v
1497 Verbose statistics.
1498 Reports usage statistics for individual vdevs within the pool, in addition to
1499 the pool-wise statistics.
1500 .El
1501 .It Xo
1502 .Nm
1503 .Cm offline
1504 .Op Fl t
1505 .Ar pool Ar device Ns ...
1506 .Xc
1507 Takes the specified physical device offline.
1508 While the
1509 .Ar device
1510 is offline, no attempt is made to read or write to the device.
1511 This command is not applicable to spares.
1512 .Bl -tag -width Ds
1513 .It Fl t
1514 Temporary.
1515 Upon reboot, the specified physical device reverts to its previous state.
1516 .El
1517 .It Xo
1518 .Nm
1519 .Cm online
1520 .Op Fl e
1521 .Ar pool Ar device Ns ...
1522 .Xc
1523 Brings the specified physical device online.
1524 This command is not applicable to spares.
1525 .Bl -tag -width Ds
1526 .It Fl e
1527 Expand the device to use all available space.
1528 If the device is part of a mirror or raidz then all devices must be expanded
1529 before the new space will become available to the pool.
1530 .El
1531 .It Xo
1532 .Nm
1533 .Cm reguid
1534 .Ar pool
1535 .Xc
1536 Generates a new unique identifier for the pool.
1537 You must ensure that all devices in this pool are online and healthy before
1538 performing this action.
1539 .It Xo
1540 .Nm
1541 .Cm reopen
1542 .Ar pool
1543 .Xc
1544 Reopen all the vdevs associated with the pool.
1545 .It Xo
1546 .Nm
1547 .Cm remove
1548 .Op Fl np
1549 .Ar pool Ar device Ns ...
1550 .Xc
1551 Removes the specified device from the pool.
1552 This command currently only supports removing hot spares, cache, log
1553 devices and mirrored top-level vdevs (mirror of leaf devices); but not raidz.
1554 .sp
1555 Removing a top-level vdev reduces the total amount of space in the storage pool.
1556 The specified device will be evacuated by copying all allocated space from it to
1557 the other devices in the pool.
1558 In this case, the
1559 .Nm zpool Cm remove
1560 command initiates the removal and returns, while the evacuation continues in
1561 the background.
1562 The removal progress can be monitored with
1563 .Nm zpool Cm status.
1564 This feature must be enabled to be used, see
1565 .Xr zpool-features 5
1566 .Pp
1567 A mirrored top-level device (log or data) can be removed by specifying the top-level mirror for the
1568 same.
1569 Non-log devices or data devices that are part of a mirrored configuration can be removed using
1570 the
1571 .Nm zpool Cm detach
1572 command.
1573 .Bl -tag -width Ds
1574 .It Fl n
1575 Do not actually perform the removal ("no-op").
1576 Instead, print the estimated amount of memory that will be used by the
1577 mapping table after the removal completes.
1578 This is nonzero only for top-level vdevs.
1579 .El
1580 .Bl -tag -width Ds
1581 .It Fl p
1582 Used in conjunction with the
1583 .Fl n
1584 flag, displays numbers as parsable (exact) values.
1585 .El
1586 .It Xo
1587 .Nm
1588 .Cm remove
1589 .Fl s
1590 .Ar pool
1591 .Xc
1592 Stops and cancels an in-progress removal of a top-level vdev.
1593 .It Xo
1594 .Nm
1595 .Cm replace
1596 .Op Fl f
1597 .Ar pool Ar device Op Ar new_device
1598 .Xc
1599 Replaces
1600 .Ar old_device
1601 with
1602 .Ar new_device .
1603 This is equivalent to attaching
1604 .Ar new_device ,
1605 waiting for it to resilver, and then detaching
1606 .Ar old_device .
1607 .Pp
1608 The size of
1609 .Ar new_device
1610 must be greater than or equal to the minimum size of all the devices in a mirror
1611 or raidz configuration.
1612 .Pp
1613 .Ar new_device
1614 is required if the pool is not redundant.
1615 If
1616 .Ar new_device
1617 is not specified, it defaults to
1618 .Ar old_device .
1619 This form of replacement is useful after an existing disk has failed and has
1620 been physically replaced.
1621 In this case, the new disk may have the same
1622 .Pa /dev/dsk
1623 path as the old device, even though it is actually a different disk.
1624 ZFS recognizes this.
1625 .Bl -tag -width Ds
1626 .It Fl f
1627 Forces use of
1628 .Ar new_device ,
1629 even if its appears to be in use.
1630 Not all devices can be overridden in this manner.
1631 .El
1632 .It Xo
1633 .Nm
1634 .Cm scrub
1635 .Op Fl s | Fl p
1636 .Ar pool Ns ...
1637 .Xc
1638 Begins a scrub or resumes a paused scrub.
1639 The scrub examines all data in the specified pools to verify that it checksums
1640 correctly.
1641 For replicated
1642 .Pq mirror or raidz
1643 devices, ZFS automatically repairs any damage discovered during the scrub.
1644 The
1645 .Nm zpool Cm status
1646 command reports the progress of the scrub and summarizes the results of the
1647 scrub upon completion.
1648 .Pp
1649 Scrubbing and resilvering are very similar operations.
1650 The difference is that resilvering only examines data that ZFS knows to be out
1651 of date
1652 .Po
1653 for example, when attaching a new device to a mirror or replacing an existing
1654 device
1655 .Pc ,
1656 whereas scrubbing examines all data to discover silent errors due to hardware
1657 faults or disk failure.
1658 .Pp
1659 Because scrubbing and resilvering are I/O-intensive operations, ZFS only allows
1660 one at a time.
1661 If a scrub is paused, the
1662 .Nm zpool Cm scrub
1663 resumes it.
1664 If a resilver is in progress, ZFS does not allow a scrub to be started until the
1665 resilver completes.
1666 .Bl -tag -width Ds
1667 .It Fl s
1668 Stop scrubbing.
1669 .El
1670 .Bl -tag -width Ds
1671 .It Fl p
1672 Pause scrubbing.
1673 Scrub pause state and progress are periodically synced to disk.
1674 If the system is restarted or pool is exported during a paused scrub,
1675 even after import, scrub will remain paused until it is resumed.
1676 Once resumed the scrub will pick up from the place where it was last
1677 checkpointed to disk.
1678 To resume a paused scrub issue
1679 .Nm zpool Cm scrub
1680 again.
1681 .El
1682 .It Xo
1683 .Nm
1684 .Cm set
1685 .Ar property Ns = Ns Ar value
1686 .Ar pool
1687 .Xc
1688 Sets the given property on the specified pool.
1689 See the
1690 .Sx Properties
1691 section for more information on what properties can be set and acceptable
1692 values.
1693 .It Xo
1694 .Nm
1695 .Cm split
1696 .Op Fl n
1697 .Oo Fl o Ar property Ns = Ns Ar value Oc Ns ...
1698 .Op Fl R Ar root
1699 .Ar pool newpool
1700 .Xc
1701 Splits devices off
1702 .Ar pool
1703 creating
1704 .Ar newpool .
1705 All vdevs in
1706 .Ar pool
1707 must be mirrors.
1708 At the time of the split,
1709 .Ar newpool
1710 will be a replica of
1711 .Ar pool .
1712 .Bl -tag -width Ds
1713 .It Fl n
1714 Do dry run, do not actually perform the split.
1715 Print out the expected configuration of
1716 .Ar newpool .
1717 .It Fl o Ar property Ns = Ns Ar value
1718 Sets the specified property for
1719 .Ar newpool .
1720 See the
1721 .Sx Properties
1722 section for more information on the available pool properties.
1723 .It Fl R Ar root
1724 Set
1725 .Sy altroot
1726 for
1727 .Ar newpool
1728 to
1729 .Ar root
1730 and automatically import it.
1731 .El
1732 .It Xo
1733 .Nm
1734 .Cm status
1735 .Op Fl Dvx
1736 .Op Fl T Sy u Ns | Ns Sy d
1737 .Oo Ar pool Oc Ns ...
1738 .Op Ar interval Op Ar count
1739 .Xc
1740 Displays the detailed health status for the given pools.
1741 If no
1742 .Ar pool
1743 is specified, then the status of each pool in the system is displayed.
1744 For more information on pool and device health, see the
1745 .Sx Device Failure and Recovery
1746 section.
1747 .Pp
1748 If a scrub or resilver is in progress, this command reports the percentage done
1749 and the estimated time to completion.
1750 Both of these are only approximate, because the amount of data in the pool and
1751 the other workloads on the system can change.
1752 .Bl -tag -width Ds
1753 .It Fl D
1754 Display a histogram of deduplication statistics, showing the allocated
1755 .Pq physically present on disk
1756 and referenced
1757 .Pq logically referenced in the pool
1758 block counts and sizes by reference count.
1759 .It Fl T Sy u Ns | Ns Sy d
1760 Display a time stamp.
1761 Specify
1762 .Fl u
1763 for a printed representation of the internal representation of time.
1764 See
1765 .Xr time 2 .
1766 Specify
1767 .Fl d
1768 for standard date format.
1769 See
1770 .Xr date 1 .
1771 .It Fl v
1772 Displays verbose data error information, printing out a complete list of all
1773 data errors since the last complete pool scrub.
1774 .It Fl x
1775 Only display status for pools that are exhibiting errors or are otherwise
1776 unavailable.
1777 Warnings about pools not using the latest on-disk format will not be included.
1778 .El
1779 .It Xo
1780 .Nm
1781 .Cm upgrade
1782 .Xc
1783 Displays pools which do not have all supported features enabled and pools
1784 formatted using a legacy ZFS version number.
1785 These pools can continue to be used, but some features may not be available.
1786 Use
1787 .Nm zpool Cm upgrade Fl a
1788 to enable all features on all pools.
1789 .It Xo
1790 .Nm
1791 .Cm upgrade
1792 .Fl v
1793 .Xc
1794 Displays legacy ZFS versions supported by the current software.
1795 See
1796 .Xr zpool-features 5
1797 for a description of feature flags features supported by the current software.
1798 .It Xo
1799 .Nm
1800 .Cm upgrade
1801 .Op Fl V Ar version
1802 .Fl a Ns | Ns Ar pool Ns ...
1803 .Xc
1804 Enables all supported features on the given pool.
1805 Once this is done, the pool will no longer be accessible on systems that do not
1806 support feature flags.
1807 See
1808 .Xr zpool-features 5
1809 for details on compatibility with systems that support feature flags, but do not
1810 support all features enabled on the pool.
1811 .Bl -tag -width Ds
1812 .It Fl a
1813 Enables all supported features on all pools.
1814 .It Fl V Ar version
1815 Upgrade to the specified legacy version.
1816 If the
1817 .Fl V
1818 flag is specified, no features will be enabled on the pool.
1819 This option can only be used to increase the version number up to the last
1820 supported legacy version number.
1821 .El
1822 .El
1823 .Sh EXIT STATUS
1824 The following exit values are returned:
1825 .Bl -tag -width Ds
1826 .It Sy 0
1827 Successful completion.
1828 .It Sy 1
1829 An error occurred.
1830 .It Sy 2
1831 Invalid command line options were specified.
1832 .El
1833 .Sh EXAMPLES
1834 .Bl -tag -width Ds
1835 .It Sy Example 1 No Creating a RAID-Z Storage Pool
1836 The following command creates a pool with a single raidz root vdev that
1837 consists of six disks.
1838 .Bd -literal
1839 # zpool create tank raidz c0t0d0 c0t1d0 c0t2d0 c0t3d0 c0t4d0 c0t5d0
1840 .Ed
1841 .It Sy Example 2 No Creating a Mirrored Storage Pool
1842 The following command creates a pool with two mirrors, where each mirror
1843 contains two disks.
1844 .Bd -literal
1845 # zpool create tank mirror c0t0d0 c0t1d0 mirror c0t2d0 c0t3d0
1846 .Ed
1847 .It Sy Example 3 No Creating a ZFS Storage Pool by Using Slices
1848 The following command creates an unmirrored pool using two disk slices.
1849 .Bd -literal
1850 # zpool create tank /dev/dsk/c0t0d0s1 c0t1d0s4
1851 .Ed
1852 .It Sy Example 4 No Creating a ZFS Storage Pool by Using Files
1853 The following command creates an unmirrored pool using files.
1854 While not recommended, a pool based on files can be useful for experimental
1855 purposes.
1856 .Bd -literal
1857 # zpool create tank /path/to/file/a /path/to/file/b
1858 .Ed
1859 .It Sy Example 5 No Adding a Mirror to a ZFS Storage Pool
1860 The following command adds two mirrored disks to the pool
1861 .Em tank ,
1862 assuming the pool is already made up of two-way mirrors.
1863 The additional space is immediately available to any datasets within the pool.
1864 .Bd -literal
1865 # zpool add tank mirror c1t0d0 c1t1d0
1866 .Ed
1867 .It Sy Example 6 No Listing Available ZFS Storage Pools
1868 The following command lists all available pools on the system.
1869 In this case, the pool
1870 .Em zion
1871 is faulted due to a missing device.
1872 The results from this command are similar to the following:
1873 .Bd -literal
1874 # zpool list
1875 NAME SIZE ALLOC FREE FRAG EXPANDSZ CAP DEDUP HEALTH ALTROOT
1876 rpool 19.9G 8.43G 11.4G 33% - 42% 1.00x ONLINE -
1877 tank 61.5G 20.0G 41.5G 48% - 32% 1.00x ONLINE -
1878 zion - - - - - - - FAULTED -
1879 .Ed
1880 .It Sy Example 7 No Destroying a ZFS Storage Pool
1881 The following command destroys the pool
1882 .Em tank
1883 and any datasets contained within.
1884 .Bd -literal
1885 # zpool destroy -f tank
1886 .Ed
1887 .It Sy Example 8 No Exporting a ZFS Storage Pool
1888 The following command exports the devices in pool
1889 .Em tank
1890 so that they can be relocated or later imported.
1891 .Bd -literal
1892 # zpool export tank
1893 .Ed
1894 .It Sy Example 9 No Importing a ZFS Storage Pool
1895 The following command displays available pools, and then imports the pool
1896 .Em tank
1897 for use on the system.
1898 The results from this command are similar to the following:
1899 .Bd -literal
1900 # zpool import
1901 pool: tank
1902 id: 15451357997522795478
1903 state: ONLINE
1904 action: The pool can be imported using its name or numeric identifier.
1905 config:
1906
1907 tank ONLINE
1908 mirror ONLINE
1909 c1t2d0 ONLINE
1910 c1t3d0 ONLINE
1911
1912 # zpool import tank
1913 .Ed
1914 .It Sy Example 10 No Upgrading All ZFS Storage Pools to the Current Version
1915 The following command upgrades all ZFS Storage pools to the current version of
1916 the software.
1917 .Bd -literal
1918 # zpool upgrade -a
1919 This system is currently running ZFS version 2.
1920 .Ed
1921 .It Sy Example 11 No Managing Hot Spares
1922 The following command creates a new pool with an available hot spare:
1923 .Bd -literal
1924 # zpool create tank mirror c0t0d0 c0t1d0 spare c0t2d0
1925 .Ed
1926 .Pp
1927 If one of the disks were to fail, the pool would be reduced to the degraded
1928 state.
1929 The failed device can be replaced using the following command:
1930 .Bd -literal
1931 # zpool replace tank c0t0d0 c0t3d0
1932 .Ed
1933 .Pp
1934 Once the data has been resilvered, the spare is automatically removed and is
1935 made available for use should another device fail.
1936 The hot spare can be permanently removed from the pool using the following
1937 command:
1938 .Bd -literal
1939 # zpool remove tank c0t2d0
1940 .Ed
1941 .It Sy Example 12 No Creating a ZFS Pool with Mirrored Separate Intent Logs
1942 The following command creates a ZFS storage pool consisting of two, two-way
1943 mirrors and mirrored log devices:
1944 .Bd -literal
1945 # zpool create pool mirror c0d0 c1d0 mirror c2d0 c3d0 log mirror \e
1946 c4d0 c5d0
1947 .Ed
1948 .It Sy Example 13 No Adding Cache Devices to a ZFS Pool
1949 The following command adds two disks for use as cache devices to a ZFS storage
1950 pool:
1951 .Bd -literal
1952 # zpool add pool cache c2d0 c3d0
1953 .Ed
1954 .Pp
1955 Once added, the cache devices gradually fill with content from main memory.
1956 Depending on the size of your cache devices, it could take over an hour for
1957 them to fill.
1958 Capacity and reads can be monitored using the
1959 .Cm iostat
1960 option as follows:
1961 .Bd -literal
1962 # zpool iostat -v pool 5
1963 .Ed
1964 .It Sy Example 14 No Removing a Mirrored top-level (Log or Data) Device
1965 The following commands remove the mirrored log device
1966 .Sy mirror-2
1967 and mirrored top-level data device
1968 .Sy mirror-1 .
1969 .Pp
1970 Given this configuration:
1971 .Bd -literal
1972 pool: tank
1973 state: ONLINE
1974 scrub: none requested
1975 config:
1976
1977 NAME STATE READ WRITE CKSUM
1978 tank ONLINE 0 0 0
1979 mirror-0 ONLINE 0 0 0
1980 c6t0d0 ONLINE 0 0 0
1981 c6t1d0 ONLINE 0 0 0
1982 mirror-1 ONLINE 0 0 0
1983 c6t2d0 ONLINE 0 0 0
1984 c6t3d0 ONLINE 0 0 0
1985 logs
1986 mirror-2 ONLINE 0 0 0
1987 c4t0d0 ONLINE 0 0 0
1988 c4t1d0 ONLINE 0 0 0
1989 .Ed
1990 .Pp
1991 The command to remove the mirrored log
1992 .Sy mirror-2
1993 is:
1994 .Bd -literal
1995 # zpool remove tank mirror-2
1996 .Ed
1997 .Pp
1998 The command to remove the mirrored data
1999 .Sy mirror-1
2000 is:
2001 .Bd -literal
2002 # zpool remove tank mirror-1
2003 .Ed
2004 .It Sy Example 15 No Displaying expanded space on a device
2005 The following command displays the detailed information for the pool
2006 .Em data .
2007 This pool is comprised of a single raidz vdev where one of its devices
2008 increased its capacity by 10GB.
2009 In this example, the pool will not be able to utilize this extra capacity until
2010 all the devices under the raidz vdev have been expanded.
2011 .Bd -literal
2012 # zpool list -v data
2013 NAME SIZE ALLOC FREE FRAG EXPANDSZ CAP DEDUP HEALTH ALTROOT
2014 data 23.9G 14.6G 9.30G 48% - 61% 1.00x ONLINE -
2015 raidz1 23.9G 14.6G 9.30G 48% -
2016 c1t1d0 - - - - -
2017 c1t2d0 - - - - 10G
2018 c1t3d0 - - - - -
2019 .Ed
2020 .El
2021 .Sh INTERFACE STABILITY
2022 .Sy Evolving
2023 .Sh SEE ALSO
2024 .Xr zfs 1M ,
2025 .Xr attributes 5 ,
2026 .Xr zpool-features 5