1 '\" te
2 .\" Copyright (c) 2013, 2016 by Delphix. All rights reserved.
3 .\" Copyright (c) 2013 by Saso Kiselkov. All rights reserved.
4 .\" Copyright (c) 2014, Joyent, Inc. All rights reserved.
5 .\" Copyright (c) 2014 Integros [integros.com]
6 .\" Copyright (c) 2016, Nexenta Systems, Inc. All rights reserved.
7 .\" The contents of this file are subject to the terms of the Common Development
8 .\" and Distribution License (the "License"). You may not use this file except
9 .\" in compliance with the License. You can obtain a copy of the license at
10 .\" usr/src/OPENSOLARIS.LICENSE or http://www.opensolaris.org/os/licensing.
11 .\"
12 .\" See the License for the specific language governing permissions and
13 .\" limitations under the License. When distributing Covered Code, include this
14 .\" CDDL HEADER in each file and include the License file at
15 .\" usr/src/OPENSOLARIS.LICENSE. If applicable, add the following below this
16 .\" CDDL HEADER, with the fields enclosed by brackets "[]" replaced with your
17 .\" own identifying information:
18 .\" Portions Copyright [yyyy] [name of copyright owner]
19 .TH ZPOOL-FEATURES 5 "Mar 25, 2016"
20 .SH NAME
21 zpool\-features \- ZFS pool feature descriptions
22 .SH DESCRIPTION
23 .LP
24 ZFS pool on\-disk format versions are specified via "features" which replace
25 the old on\-disk format numbers (the last supported on\-disk format number is
26 28). To enable a feature on a pool use the \fBupgrade\fR subcommand of the
27 \fBzpool\fR(1M) command, or set the \fBfeature@\fR\fIfeature_name\fR property
28 to \fBenabled\fR.
29 .sp
30 .LP
31 The pool format does not affect file system version compatibility or the ability
32 to send file systems between pools.
33 .sp
34 .LP
35 Since most features can be enabled independently of each other the on\-disk
36 format of the pool is specified by the set of all features marked as
37 \fBactive\fR on the pool. If the pool was created by another software version
38 this set may include unsupported features.
39 .SS "Identifying features"
40 .LP
41 Every feature has a guid of the form \fIcom.example:feature_name\fR. The reverse
42 DNS name ensures that the feature's guid is unique across all ZFS
43 implementations. When unsupported features are encountered on a pool they will
44 be identified by their guids. Refer to the documentation for the ZFS
45 implementation that created the pool for information about those features.
46 .sp
47 .LP
48 Each supported feature also has a short name. By convention a feature's short
49 name is the portion of its guid which follows the ':' (e.g.
50 \fIcom.example:feature_name\fR would have the short name \fIfeature_name\fR),
51 however a feature's short name may differ across ZFS implementations if
52 following the convention would result in name conflicts.
53 .SS "Feature states"
54 .LP
55 Features can be in one of three states:
56 .sp
57 .ne 2
58 .na
59 \fB\fBactive\fR\fR
60 .ad
61 .RS 12n
62 This feature's on\-disk format changes are in effect on the pool. Support for
63 this feature is required to import the pool in read\-write mode. If this
64 feature is not read-only compatible, support is also required to import the pool
65 in read\-only mode (see "Read\-only compatibility").
66 .RE
67
68 .sp
69 .ne 2
70 .na
71 \fB\fBenabled\fR\fR
72 .ad
73 .RS 12n
74 An administrator has marked this feature as enabled on the pool, but the
75 feature's on\-disk format changes have not been made yet. The pool can still be
76 imported by software that does not support this feature, but changes may be made
77 to the on\-disk format at any time which will move the feature to the
78 \fBactive\fR state. Some features may support returning to the \fBenabled\fR
79 state after becoming \fBactive\fR. See feature\-specific documentation for
80 details.
81 .RE
82
83 .sp
84 .ne 2
85 .na
86 \fBdisabled\fR
87 .ad
88 .RS 12n
89 This feature's on\-disk format changes have not been made and will not be made
90 unless an administrator moves the feature to the \fBenabled\fR state. Features
91 cannot be disabled once they have been enabled.
92 .RE
93
94 .sp
95 .LP
96 The state of supported features is exposed through pool properties of the form
97 \fIfeature@short_name\fR.
98 .SS "Read\-only compatibility"
99 .LP
100 Some features may make on\-disk format changes that do not interfere with other
101 software's ability to read from the pool. These features are referred to as
102 "read\-only compatible". If all unsupported features on a pool are read\-only
103 compatible, the pool can be imported in read\-only mode by setting the
104 \fBreadonly\fR property during import (see \fBzpool\fR(1M) for details on
105 importing pools).
106 .SS "Unsupported features"
107 .LP
108 For each unsupported feature enabled on an imported pool a pool property
109 named \fIunsupported@feature_guid\fR will indicate why the import was allowed
110 despite the unsupported feature. Possible values for this property are:
111
112 .sp
113 .ne 2
114 .na
115 \fB\fBinactive\fR\fR
116 .ad
117 .RS 12n
118 The feature is in the \fBenabled\fR state and therefore the pool's on\-disk
119 format is still compatible with software that does not support this feature.
120 .RE
121
122 .sp
123 .ne 2
124 .na
125 \fB\fBreadonly\fR\fR
126 .ad
127 .RS 12n
128 The feature is read\-only compatible and the pool has been imported in
129 read\-only mode.
130 .RE
131
132 .SS "Feature dependencies"
133 .LP
134 Some features depend on other features being enabled in order to function
135 properly. Enabling a feature will automatically enable any features it
136 depends on.
137 .SH FEATURES
138 .LP
139 The following features are supported on this system:
140 .sp
141 .ne 2
142 .na
143 \fB\fBasync_destroy\fR\fR
144 .ad
145 .RS 4n
146 .TS
147 l l .
148 GUID com.delphix:async_destroy
149 READ\-ONLY COMPATIBLE yes
150 DEPENDENCIES none
151 .TE
152
153 Destroying a file system requires traversing all of its data in order to
154 return its used space to the pool. Without \fBasync_destroy\fR the file system
155 is not fully removed until all space has been reclaimed. If the destroy
156 operation is interrupted by a reboot or power outage the next attempt to open
157 the pool will need to complete the destroy operation synchronously.
158
159 When \fBasync_destroy\fR is enabled the file system's data will be reclaimed
160 by a background process, allowing the destroy operation to complete without
161 traversing the entire file system. The background process is able to resume
162 interrupted destroys after the pool has been opened, eliminating the need
163 to finish interrupted destroys as part of the open operation. The amount
164 of space remaining to be reclaimed by the background process is available
165 through the \fBfreeing\fR property.
166
167 This feature is only \fBactive\fR while \fBfreeing\fR is non\-zero.
168 .RE
169
170 .sp
171 .ne 2
172 .na
173 \fB\fBempty_bpobj\fR\fR
174 .ad
175 .RS 4n
176 .TS
177 l l .
178 GUID com.delphix:empty_bpobj
179 READ\-ONLY COMPATIBLE yes
180 DEPENDENCIES none
181 .TE
182
183 This feature increases the performance of creating and using a large
184 number of snapshots of a single filesystem or volume, and also reduces
185 the disk space required.
186
187 When there are many snapshots, each snapshot uses many Block Pointer
188 Objects (bpobj's) to track blocks associated with that snapshot.
189 However, in common use cases, most of these bpobj's are empty. This
190 feature allows us to create each bpobj on-demand, thus eliminating the
191 empty bpobjs.
192
193 This feature is \fBactive\fR while there are any filesystems, volumes,
194 or snapshots which were created after enabling this feature.
195 .RE
196
197 .sp
198 .ne 2
199 .na
200 \fB\fBfilesystem_limits\fR\fR
201 .ad
202 .RS 4n
203 .TS
204 l l .
205 GUID com.joyent:filesystem_limits
206 READ\-ONLY COMPATIBLE yes
207 DEPENDENCIES extensible_dataset
208 .TE
209
210 This feature enables filesystem and snapshot limits. These limits can be used
211 to control how many filesystems and/or snapshots can be created at the point in
212 the tree on which the limits are set.
213
214 This feature is \fBactive\fR once either of the limit properties has been
215 set on a dataset. Once activated the feature is never deactivated.
216 .RE
217
218 .sp
219 .ne 2
220 .na
221 \fB\fBlz4_compress\fR\fR
222 .ad
223 .RS 4n
224 .TS
225 l l .
226 GUID org.illumos:lz4_compress
227 READ\-ONLY COMPATIBLE no
228 DEPENDENCIES none
229 .TE
230
231 \fBlz4\fR is a high-performance real-time compression algorithm that
232 features significantly faster compression and decompression as well as a
233 higher compression ratio than the older \fBlzjb\fR compression.
234 Typically, \fBlz4\fR compression is approximately 50% faster on
235 compressible data and 200% faster on incompressible data than
236 \fBlzjb\fR. It is also approximately 80% faster on decompression, while
237 giving approximately 10% better compression ratio.
238
239 When the \fBlz4_compress\fR feature is set to \fBenabled\fR, the
240 administrator can turn on \fBlz4\fR compression on any dataset on the
241 pool using the \fBzfs\fR(1M) command. Also, all newly written metadata
242 will be compressed with \fBlz4\fR algorithm. Since this feature is not
243 read-only compatible, this operation will render the pool unimportable
244 on systems without support for the \fBlz4_compress\fR feature. Booting
245 off of \fBlz4\fR-compressed root pools is supported.
246
247 This feature becomes \fBactive\fR as soon as it is enabled and will
248 never return to being \fBenabled\fR.
249 .RE
250
251 .sp
252 .ne 2
253 .na
254 \fB\fBspacemap_histogram\fR\fR
255 .ad
256 .RS 4n
257 .TS
258 l l .
259 GUID com.delphix:spacemap_histogram
260 READ\-ONLY COMPATIBLE yes
261 DEPENDENCIES none
262 .TE
263
264 This features allows ZFS to maintain more information about how free space
265 is organized within the pool. If this feature is \fBenabled\fR, ZFS will
266 set this feature to \fBactive\fR when a new space map object is created or
267 an existing space map is upgraded to the new format. Once the feature is
268 \fBactive\fR, it will remain in that state until the pool is destroyed.
269 .RE
270
271 .sp
272 .ne 2
273 .na
274 \fB\fBmulti_vdev_crash_dump\fR\fR
275 .ad
276 .RS 4n
277 .TS
278 l l .
279 GUID com.joyent:multi_vdev_crash_dump
280 READ\-ONLY COMPATIBLE no
281 DEPENDENCIES none
282 .TE
283
284 This feature allows a dump device to be configured with a pool comprised
285 of multiple vdevs. Those vdevs may be arranged in any mirrored or raidz
286 configuration.
287
288 When the \fBmulti_vdev_crash_dump\fR feature is set to \fBenabled\fR,
289 the administrator can use the \fBdumpadm\fR(1M) command to configure a
290 dump device on a pool comprised of multiple vdevs.
291 .RE
292
293 .sp
294 .ne 2
295 .na
296 \fB\fBextensible_dataset\fR\fR
297 .ad
298 .RS 4n
299 .TS
300 l l .
301 GUID com.delphix:extensible_dataset
302 READ\-ONLY COMPATIBLE no
303 DEPENDENCIES none
304 .TE
305
306 This feature allows more flexible use of internal ZFS data structures,
307 and exists for other features to depend on.
308
309 This feature will be \fBactive\fR when the first dependent feature uses it,
310 and will be returned to the \fBenabled\fR state when all datasets that use
311 this feature are destroyed.
312
313 .RE
314
315 .sp
316 .ne 2
317 .na
318 \fB\fBbookmarks\fR\fR
319 .ad
320 .RS 4n
321 .TS
322 l l .
323 GUID com.delphix:bookmarks
324 READ\-ONLY COMPATIBLE yes
325 DEPENDENCIES extensible_dataset
326 .TE
327
328 This feature enables use of the \fBzfs bookmark\fR subcommand.
329
330 This feature is \fBactive\fR while any bookmarks exist in the pool.
331 All bookmarks in the pool can be listed by running
332 \fBzfs list -t bookmark -r \fIpoolname\fR\fR.
333
334 .RE
335
336 .sp
337 .ne 2
338 .na
339 \fB\fBenabled_txg\fR\fR
340 .ad
341 .RS 4n
342 .TS
343 l l .
344 GUID com.delphix:enabled_txg
345 READ\-ONLY COMPATIBLE yes
346 DEPENDENCIES none
347 .TE
348
349 Once this feature is enabled ZFS records the transaction group number
350 in which new features are enabled. This has no user-visible impact,
351 but other features may depend on this feature.
352
353 This feature becomes \fBactive\fR as soon as it is enabled and will
354 never return to being \fBenabled\fR.
355
356 .RE
357
358 .sp
359 .ne 2
360 .na
361 \fB\fBhole_birth\fR\fR
362 .ad
363 .RS 4n
364 .TS
365 l l .
366 GUID com.delphix:hole_birth
367 READ\-ONLY COMPATIBLE no
368 DEPENDENCIES enabled_txg
369 .TE
370
371 This feature improves performance of incremental sends ("zfs send -i")
372 and receives for objects with many holes. The most common case of
373 hole-filled objects is zvols.
374
375 An incremental send stream from snapshot \fBA\fR to snapshot \fBB\fR
376 contains information about every block that changed between \fBA\fR and
377 \fBB\fR. Blocks which did not change between those snapshots can be
378 identified and omitted from the stream using a piece of metadata called
379 the 'block birth time', but birth times are not recorded for holes (blocks
380 filled only with zeroes). Since holes created after \fBA\fR cannot be
381 distinguished from holes created before \fBA\fR, information about every
382 hole in the entire filesystem or zvol is included in the send stream.
383
384 For workloads where holes are rare this is not a problem. However, when
385 incrementally replicating filesystems or zvols with many holes (for
386 example a zvol formatted with another filesystem) a lot of time will
387 be spent sending and receiving unnecessary information about holes that
388 already exist on the receiving side.
389
390 Once the \fBhole_birth\fR feature has been enabled the block birth times
391 of all new holes will be recorded. Incremental sends between snapshots
392 created after this feature is enabled will use this new metadata to avoid
393 sending information about holes that already exist on the receiving side.
394
395 This feature becomes \fBactive\fR as soon as it is enabled and will
396 never return to being \fBenabled\fR.
397
398 .RE
399
400 .sp
401 .ne 2
402 .na
403 \fB\fBembedded_data\fR\fR
404 .ad
405 .RS 4n
406 .TS
407 l l .
408 GUID com.delphix:embedded_data
409 READ\-ONLY COMPATIBLE no
410 DEPENDENCIES none
411 .TE
412
413 This feature improves the performance and compression ratio of
414 highly-compressible blocks. Blocks whose contents can compress to 112 bytes
415 or smaller can take advantage of this feature.
416
417 When this feature is enabled, the contents of highly-compressible blocks are
418 stored in the block "pointer" itself (a misnomer in this case, as it contains
419 the compresseed data, rather than a pointer to its location on disk). Thus
420 the space of the block (one sector, typically 512 bytes or 4KB) is saved,
421 and no additional i/o is needed to read and write the data block.
422
423 This feature becomes \fBactive\fR as soon as it is enabled and will
424 never return to being \fBenabled\fR.
425
426 .RE
427
428 .sp
429 .ne 2
430 .na
431 \fB\fBdevice_removal\fR\fR
432 .ad
433 .RS 4n
434 .TS
435 l l .
436 GUID com.delphix:device_removal
437 READ\-ONLY COMPATIBLE no
438 DEPENDENCIES none
439 .TE
440
441 This feature enables the "zpool remove" subcommand to remove top-level
442 vdevs, evacuating them to reduce the total size of the pool.
443
444 This feature becomes \fBactive\fR when the "zpool remove" command is used
445 on a top-level vdev, and will never return to being \fBenabled\fR.
446
447 .RE
448 .sp
449 .ne 2
450 .na
451 \fB\fBobsolete_counts\fR\fR
452 .ad
453 .RS 4n
454 .TS
455 l l .
456 GUID com.delphix:obsolete_counts
457 READ\-ONLY COMPATIBLE yes
458 DEPENDENCIES device_removal
459 .TE
460
461 This feature is an enhancement of device_removal, which will over time
462 reduce the memory used to track removed devices. When indirect blocks
463 are freed or remapped, we note that their part of the indirect mapping
464 is "obsolete", i.e. no longer needed. See also the \fBzfs remap\fR
465 subcommand in \fBzfs\fR(1M).
466
467 This feature becomes \fBactive\fR when the "zpool remove" command is
468 used on a top-level vdev, and will never return to being \fBenabled\fR.
469
470 .RE
471 .sp
472 .ne 2
473 .na
474 \fB\fBlarge_blocks\fR\fR
475 .ad
476 .RS 4n
477 .TS
478 l l .
479 GUID org.open-zfs:large_block
480 READ\-ONLY COMPATIBLE no
481 DEPENDENCIES extensible_dataset
482 .TE
483
484 The \fBlarge_block\fR feature allows the record size on a dataset to be
485 set larger than 128KB.
486
487 This feature becomes \fBactive\fR once a \fBrecordsize\fR property has been
488 set larger than 128KB, and will return to being \fBenabled\fR once all
489 filesystems that have ever had their recordsize larger than 128KB are destroyed.
490
491 .RE
492
493 .sp
494 .ne 2
495 .na
496 \fB\fBvdev_properties\fR\fR
497 .ad
498 .RS 4n
499 .TS
500 l l .
501 GUID com.nexenta:vdev_properties
502 READ\-ONLY COMPATIBLE yes
503 DEPENDENCIES none
504 .TE
505
506 This feature indicates that the pool includes on-disk format changes that
507 support persistent vdev-specific properties. This feature will be
508 \fBactive\fR when the first vdev-specific property is set.
509
510 .RE
511
512 .sp
513 .ne 2
514 .na
515 \fB\fBcos_properties\fR\fR
516 .ad
517 .RS 4n
518 .TS
519 l l .
520 GUID com.nexenta:cos_properties
521 READ\-ONLY COMPATIBLE yes
522 DEPENDENCIES com.nexenta:vdev_properties
523 .TE
524
525 This feature indicates that the pool includes on-disk format changes that
526 support persistent Class of Storage (CoS) properties. Such properties can
527 be associated with a collection of device that belong to a common class
528 from storage management standpoint. This feature will be
529 \fBactive\fR when the first CoS property is set.
530
531 .RE
532
533 .sp
534 .ne 2
535 .na
536 \fB\fBmeta_devices\fR\fR
537 .ad
538 .RS 4n
539 .TS
540 l l .
541 GUID com.nexenta:meta_devices
542 READ\-ONLY COMPATIBLE yes
543 DEPENDENCIES none
544 .TE
545
546 This feature flag indicates presence of special-vdev in the pool. Special
547 vdev is used to speed-up read and write operations and can be used to store
548 ZFS metadata and/or write log (ZIL). In addition, special vdev can function
549 as a writeback cache (WBC) within the pool, accelerating ZFS writes via
550 underlying fast media (typically, write-optimized SSD).
551
552 Note that unlike the first two functions the WBC function is configurable
553 on a per-dataset tree basis.
554
555 .RE
556
557 .sp
558 .ne 2
559 .na
560 \fB\fBwbc\fR\fR
561 .ad
562 .RS 4n
563 .TS
564 l l .
565 GUID com.nexenta:wbc
566 READ\-ONLY COMPATIBLE no
567 DEPENDENCIES com.nexenta:meta_devices
568 .TE
569
570 When enabled, this feature indicates that the pool supports writeback caching.
571 The latter can be activated on a specific filesystem (and all its children) or
572 a volume (zvol) within the pool by setting the corresponding \fBwbc_mode\fR
573 property to 'on'. This feature will show up as \fBdisabled\fR if the pool does
574 not contain special-vdev.
575
576 .RE
577
578 .sp
579 .ne 2
580 .na
581 \fB\fBsha512\fR\fR
582 .ad
583 .RS 4n
584 .TS
585 l l .
586 GUID org.illumos:sha512
587 READ\-ONLY COMPATIBLE no
588 DEPENDENCIES extensible_dataset
589 .TE
590
591 This feature enables the use of the SHA-512/256 truncated hash algorithm
592 (FIPS 180-4) for checksum and dedup. The native 64-bit arithmetic of
593 SHA-512 provides an approximate 50% performance boost over SHA-256 on
594 64-bit hardware and is thus a good minimum-change replacement candidate
595 for systems where hash performance is important, but these systems
596 cannot for whatever reason utilize the faster \fBskein\fR and
597 \fBedonr\fR algorithms.
598
599 When the \fBsha512\fR feature is set to \fBenabled\fR, the administrator
600 can turn on the \fBsha512\fR checksum on any dataset using the
601 \fBzfs set checksum=sha512\fR command. This feature becomes
602 \fBactive\fR once a \fBchecksum\fR property has been set to \fBsha512\fR,
603 and will return to being \fBenabled\fR once all filesystems that have
604 ever had their checksum set to \fBsha512\fR are destroyed.
605
606 Booting off of pools utilizing SHA-512/256 is supported (provided that
607 the updated GRUB stage2 module is installed).
608
609 .RE
610
611 .sp
612 .ne 2
613 .na
614 \fB\fBskein\fR\fR
615 .ad
616 .RS 4n
617 .TS
618 l l .
619 GUID org.illumos:skein
620 READ\-ONLY COMPATIBLE no
621 DEPENDENCIES extensible_dataset
622 .TE
623
624 This feature enables the use of the Skein hash algorithm for checksum
625 and dedup. Skein is a high-performance secure hash algorithm that was a
626 finalist in the NIST SHA-3 competition. It provides a very high security
627 margin and high performance on 64-bit hardware (80% faster than
628 SHA-256). This implementation also utilizes the new salted checksumming
629 functionality in ZFS, which means that the checksum is pre-seeded with a
630 secret 256-bit random key (stored on the pool) before being fed the data
631 block to be checksummed. Thus the produced checksums are unique to a
632 given pool, preventing hash collision attacks on systems with dedup.
633
634 When the \fBskein\fR feature is set to \fBenabled\fR, the administrator
635 can turn on the \fBskein\fR checksum on any dataset using the
636 \fBzfs set checksum=skein\fR command. This feature becomes
637 \fBactive\fR once a \fBchecksum\fR property has been set to \fBskein\fR,
638 and will return to being \fBenabled\fR once all filesystems that have
639 ever had their checksum set to \fBskein\fR are destroyed.
640
641 Booting off of pools using \fBskein\fR is \fBNOT\fR supported
642 -- any attempt to enable \fBskein\fR on a root pool will fail with an
643 error.
644
645 .RE
646
647 .sp
648 .ne 2
649 .na
650 \fB\fBedonr\fR\fR
651 .ad
652 .RS 4n
653 .TS
654 l l .
655 GUID org.illumos:edonr
656 READ\-ONLY COMPATIBLE no
657 DEPENDENCIES extensible_dataset
658 .TE
659
660 This feature enables the use of the Edon-R hash algorithm for checksum,
661 including for nopwrite (if compression is also enabled, an overwrite of
662 a block whose checksum matches the data being written will be ignored).
663 In an abundance of caution, Edon-R can not be used with dedup
664 (without verification).
665
666 Edon-R is a very high-performance hash algorithm that was part
667 of the NIST SHA-3 competition. It provides extremely high hash
668 performance (over 350% faster than SHA-256), but was not selected
669 because of its unsuitability as a general purpose secure hash algorithm.
670 This implementation utilizes the new salted checksumming functionality
671 in ZFS, which means that the checksum is pre-seeded with a secret
672 256-bit random key (stored on the pool) before being fed the data block
673 to be checksummed. Thus the produced checksums are unique to a given
674 pool.
675
676 When the \fBedonr\fR feature is set to \fBenabled\fR, the administrator
677 can turn on the \fBedonr\fR checksum on any dataset using the
678 \fBzfs set checksum=edonr\fR command. This feature becomes
679 \fBactive\fR once a \fBchecksum\fR property has been set to \fBedonr\fR,
680 and will return to being \fBenabled\fR once all filesystems that have
681 ever had their checksum set to \fBedonr\fR are destroyed.
682
683 Booting off of pools using \fBedonr\fR is \fBNOT\fR supported
684 -- any attempt to enable \fBedonr\fR on a root pool will fail with an
685 error.
686
687 .SH "SEE ALSO"
688 \fBzfs\fR(1M), \fBzpool\fR(1M)