Print this page
NEX-15090 Due to mismerge "zfs send" command cannot generate compressed stream
Reviewed by: Yuri Pankov <yuri.pankov@nexenta.com>
NEX-15090 Due to mismerge "zfs send" command cannot generate compressed stream
Reviewed by: Yuri Pankov <yuri.pankov@nexenta.com>
NEX-9732 Infinite loop in zfs receive (lint fix)
NEX-9732 Infinite loop in zfs receive
NEX-10194 replication Assertion failed: 0 == nvlist_lookup_string(nvfs, "name", &fsname)
NEX-5300 ZFS receive failed: no error
NEX-10201 Replication successfully completed but zfs receive exit with status 1 and without error message
Reviewed by: Eugene Khudyakoff <eugene.khudyakoff@nexenta.com>
Reviewed by: Dan Fields <dan.fields@nexenta.com>
Reviewed by: Steve Peng <steve.peng@nexenta.com>
Reviewed by: Roman Strashkin <roman.strashkin@nexenta.com>
Reviewed by: Cynthia Eastham <cynthia.eastham@nexenta.com>
NEX-1520 Incorrect behavior of ZFS recv in case of incremental recursive replication to a cloned FS
Reviewed by: Alex Aizman <alex.aizman@nexenta.com>
Reviewed by: Saso Kiselkov <saso.kiselkov@nexenta.com>
Reviewed by: Roman Strashkin <roman.strashkin@nexenta.com>
NEX-6115 Segmentation Fault on zfs recv -x property command
Reviewed by: Roman Strashkin <roman.strashkin@nexenta.com>
Reviewed by: Alek Pinchuk <alek.pinchuk@nexenta.com>
Reviewed by: Alex Aizman <alex.aizman@nexenta.com>
Reviewed by: Yuri Pankov <yuri.pankov@nexenta.com>
NEX-5928 KRRP: Integrate illumos/openzfs resume-token, to resume replication from a given synced offset
Reviewed by: Alek Pinchuk <alek.pinchuk@nexenta.com>
Reviewed by: Alexey Komarov <alexey.komarov@nexenta.com>
Reviewed by: Alex Aizman <alex.aizman@nexenta.com>
NEX-5728 Autosync Destination retention policy not being honoured
Reviewed by: Roman Strashkin <roman.strashkin@nexenta.com>
Reviewed by: Sanjay Nadkarni <sanjay.nadkarni@nexenta.com>
NEX-5795 Rename 'wrc' as 'wbc' in the source and in the tech docs
Reviewed by: Alex Aizman <alex.aizman@nexenta.com>
Reviewed by: Sanjay Nadkarni <sanjay.nadkarni@nexenta.com>
Reviewed by: Alek Pinchuk <alek.pinchuk@nexenta.com>
NEX-5239 Recursive auto-sync may get broken if new child datasets are added between the job's runs
Reviewed by: Alek Pinchuk <alek.pinchuk@nexenta.com>
Reviewed by: Roman Strashkin <roman.strashkin@nexenta.com>
NEX-5272 KRRP: replicate snapshot properties
Reviewed by: Sanjay Nadkarni <sanjay.nadkarni@nexenta.com>
Reviewed by: Alexey Komarov <alexey.komarov@nexenta.com>
Reviewed by: Alex Aizman <alex.aizman@nexenta.com>
NEX-5270 WBC: Incorrect error message when trying to 'zfs recv' into wrcached dataset
Reviewed by: Alek Pinchuk <alek.pinchuk@nexenta.com>
Reviewed by: Sanjay Nadkarni <sanjay.nadkarni@nexenta.com>
4986 receiving replication stream fails if any snapshot exceeds refquota
Reviewed by: John Kennedy <john.kennedy@delphix.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Approved by: Gordon Ross <gordon.ross@nexenta.com>
6298 zfs_create_008_neg and zpool_create_023_neg need to be updated for large block support
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: John Kennedy <john.kennedy@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
2605 want to resume interrupted zfs send
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Paul Dagnelie <pcd@delphix.com>
Reviewed by: Richard Elling <Richard.Elling@RichardElling.com>
Reviewed by: Xin Li <delphij@freebsd.org>
Reviewed by: Arne Jansen <sensille@gmx.net>
Approved by: Dan McDonald <danmcd@omniti.com>
NEX-4582 update wrc test cases for allow to use write back cache per tree of datasets
Reviewed by: Steve Peng <steve.peng@nexenta.com>
Reviewed by: Alex Aizman <alex.aizman@nexenta.com>
5960 zfs recv should prefetch indirect blocks
5925 zfs receive -o origin=
Reviewed by: Prakash Surya <prakash.surya@delphix.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
5746 more checksumming in zfs send
Reviewed by: Christopher Siden <christopher.siden@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Bayard Bell <buffer.g.overflow@gmail.com>
Approved by: Albert Lee <trisk@omniti.com>
5764 "zfs send -nv" directs output to stderr
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Paul Dagnelie <paul.dagnelie@delphix.com>
Reviewed by: Basil Crow <basil.crow@delphix.com>
Reviewed by: Steven Hartland <killing@multiplay.co.uk>
Reviewed by: Bayard Bell <buffer.g.overflow@gmail.com>
Approved by: Dan McDonald <danmcd@omniti.com>
NEX-4476 WRC: Allow to use write back cache per tree of datasets
Reviewed by: Alek Pinchuk <alek.pinchuk@nexenta.com>
Reviewed by: Alex Aizman <alex.aizman@nexenta.com>
Revert "NEX-4476 WRC: Allow to use write back cache per tree of datasets"
This reverts commit fe97b74444278a6f36fec93179133641296312da.
NEX-4476 WRC: Allow to use write back cache per tree of datasets
Reviewed by: Alek Pinchuk <alek.pinchuk@nexenta.com>
Reviewed by: Alex Aizman <alex.aizman@nexenta.com>
NEX-1456 Part 2, port FreeBSD patch - new zfs recv options support
Bug 10481 - cstyle/lint cleanup
Bug 10481 - Dry run option in 'zfs send' isn't the same as in NexentaStor 3.1
@@ -21,10 +21,11 @@
/*
* Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved.
* Copyright (c) 2011, 2015 by Delphix. All rights reserved.
* Copyright (c) 2012, Joyent, Inc. All rights reserved.
+ * Copyright 2017 Nexenta Systems, Inc. All rights reserved.
* Copyright (c) 2012 Pawel Jakub Dawidek. All rights reserved.
* Copyright (c) 2013 Steven Hartland. All rights reserved.
* Copyright 2015, OmniTI Computer Consulting, Inc. All rights reserved.
* Copyright (c) 2014 Integros [integros.com]
* Copyright 2016 Igor Kozhukhov <ikozhukhov@gmail.com>
@@ -46,13 +47,15 @@
#include <time.h>
#include <libzfs.h>
#include <libzfs_core.h>
+#include "zfs_errno.h"
#include "zfs_namecheck.h"
#include "zfs_prop.h"
#include "zfs_fletcher.h"
+#include "zfs_sendrecv.h"
#include "libzfs_impl.h"
#include <zlib.h>
#include <sha2.h>
#include <sys/zio_checksum.h>
#include <sys/ddt.h>
@@ -59,20 +62,22 @@
/* in libzfs_dataset.c */
extern void zfs_setprop_error(libzfs_handle_t *, zfs_prop_t, int, char *);
static int zfs_receive_impl(libzfs_handle_t *, const char *, const char *,
- recvflags_t *, int, const char *, nvlist_t *, avl_tree_t *, char **, int,
- uint64_t *, const char *);
+ recvflags_t *, int, nvlist_t *, nvlist_t *, const char *, nvlist_t *,
+ avl_tree_t *, char **, int, uint64_t *, const char *);
static int guid_to_name(libzfs_handle_t *, const char *,
uint64_t, boolean_t, char *);
static const zio_cksum_t zero_cksum = { 0 };
typedef struct dedup_arg {
int inputfd;
int outputfd;
+ uint64_t dedup_data_sz;
+ boolean_t sendsize;
libzfs_handle_t *dedup_hdl;
} dedup_arg_t;
typedef struct progress_arg {
zfs_handle_t *pa_zhp;
@@ -185,15 +190,19 @@
ddt_hash_append(hdl, ddt, ddepp, cs, prop, dr);
return (B_FALSE);
}
static int
-dump_record(dmu_replay_record_t *drr, void *payload, int payload_len,
- zio_cksum_t *zc, int outfd)
+dump_record(dedup_arg_t *dda, dmu_replay_record_t *drr, void *payload,
+ int payload_len, zio_cksum_t *zc, int outfd)
{
ASSERT3U(offsetof(dmu_replay_record_t, drr_u.drr_checksum.drr_checksum),
==, sizeof (dmu_replay_record_t) - sizeof (zio_cksum_t));
+ if (dda != NULL) {
+ dda->dedup_data_sz +=
+ sizeof (dmu_replay_record_t) + payload_len;
+ }
(void) fletcher_4_incremental_native(drr,
offsetof(dmu_replay_record_t, drr_u.drr_checksum.drr_checksum), zc);
if (drr->drr_type != DRR_BEGIN) {
ASSERT(ZIO_CHECKSUM_IS_ZERO(&drr->drr_u.
drr_checksum.drr_checksum));
@@ -300,11 +309,11 @@
}
(void) ssread(buf, sz, ofp);
if (ferror(stdin))
perror("fread");
}
- if (dump_record(drr, buf, sz, &stream_cksum,
+ if (dump_record(dda, drr, buf, sz, &stream_cksum,
outfd) != 0)
goto out;
break;
}
@@ -311,11 +320,11 @@
case DRR_END:
{
struct drr_end *drre = &drr->drr_u.drr_end;
/* use the recalculated checksum */
drre->drr_checksum = stream_cksum;
- if (dump_record(drr, NULL, 0, &stream_cksum,
+ if (dump_record(dda, drr, NULL, 0, &stream_cksum,
outfd) != 0)
goto out;
break;
}
@@ -325,11 +334,11 @@
if (drro->drr_bonuslen > 0) {
(void) ssread(buf,
P2ROUNDUP((uint64_t)drro->drr_bonuslen, 8),
ofp);
}
- if (dump_record(drr, buf,
+ if (dump_record(dda, drr, buf,
P2ROUNDUP((uint64_t)drro->drr_bonuslen, 8),
&stream_cksum, outfd) != 0)
goto out;
break;
}
@@ -336,19 +345,19 @@
case DRR_SPILL:
{
struct drr_spill *drrs = &drr->drr_u.drr_spill;
(void) ssread(buf, drrs->drr_length, ofp);
- if (dump_record(drr, buf, drrs->drr_length,
+ if (dump_record(dda, drr, buf, drrs->drr_length,
&stream_cksum, outfd) != 0)
goto out;
break;
}
case DRR_FREEOBJECTS:
{
- if (dump_record(drr, NULL, 0, &stream_cksum,
+ if (dump_record(dda, drr, NULL, 0, &stream_cksum,
outfd) != 0)
goto out;
break;
}
@@ -418,16 +427,16 @@
wbr_drrr->drr_key.ddk_cksum =
drrw->drr_key.ddk_cksum;
wbr_drrr->drr_key.ddk_prop =
drrw->drr_key.ddk_prop;
- if (dump_record(&wbr_drr, NULL, 0,
+ if (dump_record(dda, &wbr_drr, NULL, 0,
&stream_cksum, outfd) != 0)
goto out;
} else {
/* block not previously seen */
- if (dump_record(drr, buf, payload_size,
+ if (dump_record(dda, drr, buf, payload_size,
&stream_cksum, outfd) != 0)
goto out;
}
break;
}
@@ -436,20 +445,20 @@
{
struct drr_write_embedded *drrwe =
&drr->drr_u.drr_write_embedded;
(void) ssread(buf,
P2ROUNDUP((uint64_t)drrwe->drr_psize, 8), ofp);
- if (dump_record(drr, buf,
+ if (dump_record(dda, drr, buf,
P2ROUNDUP((uint64_t)drrwe->drr_psize, 8),
&stream_cksum, outfd) != 0)
goto out;
break;
}
case DRR_FREE:
{
- if (dump_record(drr, NULL, 0, &stream_cksum,
+ if (dump_record(dda, drr, NULL, 0, &stream_cksum,
outfd) != 0)
goto out;
break;
}
@@ -468,121 +477,10 @@
return (NULL);
}
/*
- * Routines for dealing with the AVL tree of fs-nvlists
- */
-typedef struct fsavl_node {
- avl_node_t fn_node;
- nvlist_t *fn_nvfs;
- char *fn_snapname;
- uint64_t fn_guid;
-} fsavl_node_t;
-
-static int
-fsavl_compare(const void *arg1, const void *arg2)
-{
- const fsavl_node_t *fn1 = arg1;
- const fsavl_node_t *fn2 = arg2;
-
- if (fn1->fn_guid > fn2->fn_guid)
- return (+1);
- else if (fn1->fn_guid < fn2->fn_guid)
- return (-1);
- else
- return (0);
-}
-
-/*
- * Given the GUID of a snapshot, find its containing filesystem and
- * (optionally) name.
- */
-static nvlist_t *
-fsavl_find(avl_tree_t *avl, uint64_t snapguid, char **snapname)
-{
- fsavl_node_t fn_find;
- fsavl_node_t *fn;
-
- fn_find.fn_guid = snapguid;
-
- fn = avl_find(avl, &fn_find, NULL);
- if (fn) {
- if (snapname)
- *snapname = fn->fn_snapname;
- return (fn->fn_nvfs);
- }
- return (NULL);
-}
-
-static void
-fsavl_destroy(avl_tree_t *avl)
-{
- fsavl_node_t *fn;
- void *cookie;
-
- if (avl == NULL)
- return;
-
- cookie = NULL;
- while ((fn = avl_destroy_nodes(avl, &cookie)) != NULL)
- free(fn);
- avl_destroy(avl);
- free(avl);
-}
-
-/*
- * Given an nvlist, produce an avl tree of snapshots, ordered by guid
- */
-static avl_tree_t *
-fsavl_create(nvlist_t *fss)
-{
- avl_tree_t *fsavl;
- nvpair_t *fselem = NULL;
-
- if ((fsavl = malloc(sizeof (avl_tree_t))) == NULL)
- return (NULL);
-
- avl_create(fsavl, fsavl_compare, sizeof (fsavl_node_t),
- offsetof(fsavl_node_t, fn_node));
-
- while ((fselem = nvlist_next_nvpair(fss, fselem)) != NULL) {
- nvlist_t *nvfs, *snaps;
- nvpair_t *snapelem = NULL;
-
- VERIFY(0 == nvpair_value_nvlist(fselem, &nvfs));
- VERIFY(0 == nvlist_lookup_nvlist(nvfs, "snaps", &snaps));
-
- while ((snapelem =
- nvlist_next_nvpair(snaps, snapelem)) != NULL) {
- fsavl_node_t *fn;
- uint64_t guid;
-
- VERIFY(0 == nvpair_value_uint64(snapelem, &guid));
- if ((fn = malloc(sizeof (fsavl_node_t))) == NULL) {
- fsavl_destroy(fsavl);
- return (NULL);
- }
- fn->fn_nvfs = nvfs;
- fn->fn_snapname = nvpair_name(snapelem);
- fn->fn_guid = guid;
-
- /*
- * Note: if there are multiple snaps with the
- * same GUID, we ignore all but one.
- */
- if (avl_find(fsavl, fn, NULL) == NULL)
- avl_add(fsavl, fn);
- else
- free(fn);
- }
- }
-
- return (fsavl);
-}
-
-/*
* Routines for dealing with the giant nvlist of fs-nvlists, etc.
*/
typedef struct send_data {
/*
* assigned inside every recursive call,
@@ -622,11 +520,12 @@
*
* "props" -> { name -> value (only if set here) }
* "snaps" -> { name (lastname) -> number (guid) }
* "snapprops" -> { name (lastname) -> { name -> value } }
*
- * "origin" -> number (guid) (if clone)
+ * "origin" -> number (guid of origin snapshot) (if clone)
+ * "origin_fsname" -> string (full name of origin file system)
* "sent" -> boolean (not on-disk)
* }
* }
* }
*
@@ -685,10 +584,17 @@
while ((elem = nvlist_next_nvpair(zhp->zfs_props, elem)) != NULL) {
char *propname = nvpair_name(elem);
zfs_prop_t prop = zfs_name_to_prop(propname);
nvlist_t *propnv;
+ /*
+ * This property make sense only to this dataset,
+ * so no reasons to include it into stream
+ */
+ if (prop == ZFS_PROP_WBC_MODE)
+ continue;
+
if (!zfs_prop_user(propname)) {
/*
* Realistically, this should never happen. However,
* we want the ability to add DSL properties without
* needing to make incompatible version changes. We
@@ -832,18 +738,25 @@
VERIFY(0 == nvlist_add_string(nvfs, "name", zhp->zfs_name));
VERIFY(0 == nvlist_add_uint64(nvfs, "parentfromsnap",
sd->parent_fromsnap_guid));
if (zhp->zfs_dmustats.dds_origin[0]) {
+ char origin_fsname[ZFS_MAX_DATASET_NAME_LEN];
zfs_handle_t *origin = zfs_open(zhp->zfs_hdl,
zhp->zfs_dmustats.dds_origin, ZFS_TYPE_SNAPSHOT);
if (origin == NULL) {
rv = -1;
goto out;
}
VERIFY(0 == nvlist_add_uint64(nvfs, "origin",
origin->zfs_dmustats.dds_guid));
+ zfs_close(origin);
+ (void) strlcpy(origin_fsname, zhp->zfs_dmustats.dds_origin,
+ sizeof (origin_fsname));
+ *strchr(origin_fsname, '@') = '\0';
+ VERIFY(0 == nvlist_add_string(nvfs, "origin_fsname",
+ origin_fsname));
}
/* iterate over props */
VERIFY(0 == nvlist_alloc(&nv, NV_UNIQUE_NAME, 0));
send_iterate_prop(zhp, nv);
@@ -852,11 +765,11 @@
/* iterate over snaps, and set sd->parent_fromsnap_guid */
sd->parent_fromsnap_guid = 0;
VERIFY(0 == nvlist_alloc(&sd->parent_snaps, NV_UNIQUE_NAME, 0));
VERIFY(0 == nvlist_alloc(&sd->snapprops, NV_UNIQUE_NAME, 0));
- (void) zfs_iter_snapshots(zhp, B_FALSE, send_iterate_snap, sd);
+ (void) zfs_iter_snapshots_sorted(zhp, send_iterate_snap, sd);
VERIFY(0 == nvlist_add_nvlist(nvfs, "snaps", sd->parent_snaps));
VERIFY(0 == nvlist_add_nvlist(nvfs, "snapprops", sd->snapprops));
nvlist_free(sd->parent_snaps);
nvlist_free(sd->snapprops);
@@ -905,11 +818,11 @@
*avlp = NULL;
*nvlp = NULL;
return (error);
}
- if (avlp != NULL && (*avlp = fsavl_create(sd.fss)) == NULL) {
+ if (avlp != NULL && fsavl_create(sd.fss, avlp) != 0) {
nvlist_free(sd.fss);
*nvlp = NULL;
return (EZFS_NOMEM);
}
@@ -925,12 +838,15 @@
const char *fromsnap;
const char *tosnap;
char prevsnap[ZFS_MAX_DATASET_NAME_LEN];
uint64_t prevsnap_obj;
boolean_t seenfrom, seento, replicate, doall, fromorigin;
- boolean_t verbose, dryrun, parsable, progress, embed_data, std_out;
+ boolean_t verbose, dryrun, dedup, parsable, progress, embed_data, std_out;
boolean_t large_block, compress;
+ boolean_t sendsize;
+ uint32_t hdr_send_sz;
+ uint64_t send_sz;
int outfd;
boolean_t err;
nvlist_t *fss;
nvlist_t *snapholds;
avl_tree_t *fsavl;
@@ -1008,11 +924,11 @@
* NULL) to the file descriptor specified by outfd.
*/
static int
dump_ioctl(zfs_handle_t *zhp, const char *fromsnap, uint64_t fromsnap_obj,
boolean_t fromorigin, int outfd, enum lzc_send_flags flags,
- nvlist_t *debugnv)
+ nvlist_t *debugnv, boolean_t sendsize, uint64_t *sendcounter)
{
zfs_cmd_t zc = { 0 };
libzfs_handle_t *hdl = zhp->zfs_hdl;
nvlist_t *thisdbg;
@@ -1023,10 +939,12 @@
zc.zc_cookie = outfd;
zc.zc_obj = fromorigin;
zc.zc_sendobj = zfs_prop_get_int(zhp, ZFS_PROP_OBJSETID);
zc.zc_fromobj = fromsnap_obj;
zc.zc_flags = flags;
+ zc.zc_sendsize = sendsize;
+ zc.zc_sendcounter = 0;
VERIFY(0 == nvlist_alloc(&thisdbg, NV_UNIQUE_NAME, 0));
if (fromsnap && fromsnap[0] != '\0') {
VERIFY(0 == nvlist_add_string(thisdbg,
"fromsnap", fromsnap));
@@ -1076,10 +994,11 @@
default:
return (zfs_standard_error(hdl, errno, errbuf));
}
}
+ *sendcounter = (uint64_t)zc.zc_sendcounter;
if (debugnv)
VERIFY(0 == nvlist_add_nvlist(debugnv, zhp->zfs_name, thisdbg));
nvlist_free(thisdbg);
return (0);
@@ -1273,10 +1192,11 @@
gather_holds(zhp, sdd);
fromorigin = sdd->prevsnap[0] == '\0' &&
(sdd->fromorigin || sdd->replicate);
+ /* print out to-from and approximate size in verbose mode */
if (sdd->verbose) {
uint64_t size = 0;
(void) estimate_ioctl(zhp, sdd->prevsnap_obj,
fromorigin, flags, &size);
@@ -1285,15 +1205,18 @@
size, sdd->parsable);
sdd->size += size;
}
if (!sdd->dryrun) {
+ uint64_t sendcounter = 0;
+ boolean_t track_progress = (sdd->progress && !sdd->sendsize);
+ boolean_t sendsize = B_FALSE;
/*
* If progress reporting is requested, spawn a new thread to
* poll ZFS_IOC_SEND_PROGRESS at a regular interval.
*/
- if (sdd->progress) {
+ if (track_progress) {
pa.pa_zhp = zhp;
pa.pa_fd = sdd->outfd;
pa.pa_parsable = sdd->parsable;
if ((err = pthread_create(&tid, NULL,
@@ -1301,14 +1224,29 @@
zfs_close(zhp);
return (err);
}
}
+ /*
+ * We need to reset the sendsize flag being sent to
+ * kernel if sdd->dedup is set. With dedup, the file
+ * descriptor sent to kernel is one end of the pipe,
+ * and we would want the data back in the pipe for
+ * cksummer() to calculate the exact size of the dedup-ed
+ * stream. So reset the sendsize flag such that
+ * kernel writes to the pipe.
+ */
+
+ sendsize = sdd->dedup ? B_FALSE : sdd->sendsize;
+
err = dump_ioctl(zhp, sdd->prevsnap, sdd->prevsnap_obj,
- fromorigin, sdd->outfd, flags, sdd->debugnv);
+ fromorigin, sdd->outfd, flags, sdd->debugnv,
+ sendsize, &sendcounter);
- if (sdd->progress) {
+ sdd->send_sz += sendcounter;
+
+ if (track_progress) {
(void) pthread_cancel(tid);
(void) pthread_join(tid, NULL);
}
}
@@ -1489,82 +1427,48 @@
}
nvlist_t *
zfs_send_resume_token_to_nvlist(libzfs_handle_t *hdl, const char *token)
{
- unsigned int version;
- int nread;
- unsigned long long checksum, packed_len;
+ nvlist_t *nvl = NULL;
+ int error;
- /*
- * Decode token header, which is:
- * <token version>-<checksum of payload>-<uncompressed payload length>
- * Note that the only supported token version is 1.
- */
- nread = sscanf(token, "%u-%llx-%llx-",
- &version, &checksum, &packed_len);
- if (nread != 3) {
+ error = zfs_send_resume_token_to_nvlist_impl(token, &nvl);
+ switch (error) {
+ case EINVAL:
zfs_error_aux(hdl, dgettext(TEXT_DOMAIN,
"resume token is corrupt (invalid format)"));
- return (NULL);
- }
-
- if (version != ZFS_SEND_RESUME_TOKEN_VERSION) {
+ break;
+ case ENOTSUP:
zfs_error_aux(hdl, dgettext(TEXT_DOMAIN,
- "resume token is corrupt (invalid version %u)"),
- version);
- return (NULL);
- }
-
- /* convert hexadecimal representation to binary */
- token = strrchr(token, '-') + 1;
- int len = strlen(token) / 2;
- unsigned char *compressed = zfs_alloc(hdl, len);
- for (int i = 0; i < len; i++) {
- nread = sscanf(token + i * 2, "%2hhx", compressed + i);
- if (nread != 1) {
- free(compressed);
+ "resume token is corrupt (invalid version)"));
+ break;
+ case EBADMSG:
zfs_error_aux(hdl, dgettext(TEXT_DOMAIN,
"resume token is corrupt "
"(payload is not hex-encoded)"));
- return (NULL);
- }
- }
-
- /* verify checksum */
- zio_cksum_t cksum;
- fletcher_4_native(compressed, len, NULL, &cksum);
- if (cksum.zc_word[0] != checksum) {
- free(compressed);
+ break;
+ case ECKSUM:
zfs_error_aux(hdl, dgettext(TEXT_DOMAIN,
"resume token is corrupt (incorrect checksum)"));
- return (NULL);
- }
-
- /* uncompress */
- void *packed = zfs_alloc(hdl, packed_len);
- uLongf packed_len_long = packed_len;
- if (uncompress(packed, &packed_len_long, compressed, len) != Z_OK ||
- packed_len_long != packed_len) {
- free(packed);
- free(compressed);
+ break;
+ case ENOSR:
zfs_error_aux(hdl, dgettext(TEXT_DOMAIN,
"resume token is corrupt (decompression failed)"));
- return (NULL);
- }
-
- /* unpack nvlist */
- nvlist_t *nv;
- int error = nvlist_unpack(packed, packed_len, &nv, KM_SLEEP);
- free(packed);
- free(compressed);
- if (error != 0) {
+ break;
+ case ENODATA:
zfs_error_aux(hdl, dgettext(TEXT_DOMAIN,
"resume token is corrupt (nvlist_unpack failed)"));
- return (NULL);
- }
- return (nv);
+ break;
+ case ENOMEM:
+ (void) no_memory(hdl);
+ break;
+ default:
+ break;
+ };
+
+ return (nvl);
}
int
zfs_send_resume(libzfs_handle_t *hdl, sendflags_t *flags, int outfd,
const char *resume_token)
@@ -1575,11 +1479,10 @@
uint64_t resumeobj, resumeoff, toguid, fromguid, bytes;
zfs_handle_t *zhp;
int error = 0;
char name[ZFS_MAX_DATASET_NAME_LEN];
enum lzc_send_flags lzc_flags = 0;
- FILE *fout = (flags->verbose && flags->dryrun) ? stdout : stderr;
(void) snprintf(errbuf, sizeof (errbuf), dgettext(TEXT_DOMAIN,
"cannot resume send"));
nvlist_t *resume_nvl =
@@ -1590,13 +1493,13 @@
* zfs_send_resume_token_to_nvlist
*/
return (zfs_error(hdl, EZFS_FAULT, errbuf));
}
if (flags->verbose) {
- (void) fprintf(fout, dgettext(TEXT_DOMAIN,
+ (void) fprintf(stderr, dgettext(TEXT_DOMAIN,
"resume token contents:\n"));
- nvlist_print(fout, resume_nvl);
+ nvlist_print(stderr, resume_nvl);
}
if (nvlist_lookup_string(resume_nvl, "toname", &toname) != 0 ||
nvlist_lookup_uint64(resume_nvl, "object", &resumeobj) != 0 ||
nvlist_lookup_uint64(resume_nvl, "offset", &resumeoff) != 0 ||
@@ -1649,11 +1552,11 @@
uint64_t size = 0;
error = lzc_send_space(zhp->zfs_name, fromname,
lzc_flags, &size);
if (error == 0)
size = MAX(0, (int64_t)(size - bytes));
- send_print_verbose(fout, zhp->zfs_name, fromname,
+ send_print_verbose(stderr, zhp->zfs_name, fromname,
size, flags->parsable);
}
if (!flags->dryrun) {
progress_arg_t pa = { 0 };
@@ -1779,10 +1682,11 @@
errbuf));
}
dda.outputfd = outfd;
dda.inputfd = pipefd[1];
dda.dedup_hdl = zhp->zfs_hdl;
+ dda.sendsize = flags->sendsize;
if ((err = pthread_create(&tid, NULL, cksummer, &dda)) != 0) {
(void) close(pipefd[0]);
(void) close(pipefd[1]);
zfs_error_aux(zhp->zfs_hdl, strerror(errno));
return (zfs_error(zhp->zfs_hdl,
@@ -1837,20 +1741,21 @@
(void) snprintf(drr.drr_u.drr_begin.drr_toname,
sizeof (drr.drr_u.drr_begin.drr_toname),
"%s@%s", zhp->zfs_name, tosnap);
drr.drr_payloadlen = buflen;
- err = dump_record(&drr, packbuf, buflen, &zc, outfd);
+ err = dump_record(NULL, &drr, packbuf, buflen, &zc, outfd);
free(packbuf);
if (err != 0)
goto stderr_out;
/* write end record */
bzero(&drr, sizeof (drr));
drr.drr_type = DRR_END;
drr.drr_u.drr_end.drr_checksum = zc;
err = write(outfd, &drr, sizeof (drr));
+ sdd.hdr_send_sz += sizeof (drr);
if (err == -1) {
err = errno;
goto stderr_out;
}
@@ -1869,10 +1774,12 @@
sdd.doall = flags->doall;
sdd.fromorigin = flags->fromorigin;
sdd.fss = fss;
sdd.fsavl = fsavl;
sdd.verbose = flags->verbose;
+ sdd.dedup = flags->dedup;
+ sdd.sendsize = flags->sendsize;
sdd.parsable = flags->parsable;
sdd.progress = flags->progress;
sdd.dryrun = flags->dryrun;
sdd.large_block = flags->largeblock;
sdd.embed_data = flags->embed_data;
@@ -1907,11 +1814,11 @@
sdd.snapholds = fnvlist_alloc();
} else {
sdd.cleanup_fd = -1;
sdd.snapholds = NULL;
}
- if (flags->verbose || sdd.snapholds != NULL) {
+ if ((flags->verbose && !flags->sendsize) || sdd.snapholds != NULL) {
/*
* Do a verbose no-op dry run to get all the verbose output
* or to gather snapshot hold's before generating any data,
* then do a non-verbose real run to generate the streams.
*/
@@ -1967,10 +1874,11 @@
if (tid != 0) {
if (err != 0)
(void) pthread_cancel(tid);
(void) close(pipefd[0]);
(void) pthread_join(tid, NULL);
+ sdd.send_sz = dda.dedup_data_sz;
}
if (sdd.cleanup_fd != -1) {
VERIFY(0 == close(sdd.cleanup_fd));
sdd.cleanup_fd = -1;
@@ -1987,12 +1895,33 @@
drr.drr_type = DRR_END;
if (write(outfd, &drr, sizeof (drr)) == -1) {
return (zfs_standard_error(zhp->zfs_hdl,
errno, errbuf));
}
+ sdd.hdr_send_sz += sizeof (drr);
}
+ if (flags->sendsize) {
+ if (flags->verbose) {
+ (void) fprintf(stderr,
+ "Send stream header size (bytes): %u\n",
+ sdd.hdr_send_sz);
+ (void) fprintf(stderr,
+ "Send stream data size (bytes): %llu\n",
+ (longlong_t)sdd.send_sz);
+ (void) fprintf(stderr,
+ "Total send stream size (bytes): %llu\n",
+ (longlong_t)(sdd.send_sz +
+ (uint64_t)sdd.hdr_send_sz));
+ } else {
+ (void) fprintf(stderr,
+ "Total send stream size (bytes): %llu\n",
+ (longlong_t)(sdd.send_sz +
+ (uint64_t)sdd.hdr_send_sz));
+ }
+ }
+
return (err || sdd.err);
stderr_out:
err = zfs_standard_error(zhp->zfs_hdl, err, errbuf);
err_out:
@@ -2136,10 +2065,13 @@
zfs_cmd_t zc = { 0 };
int err;
prop_changelist_t *clp;
zfs_handle_t *zhp;
+ if (!zfs_dataset_exists(hdl, name, ZFS_TYPE_DATASET))
+ return (ENOENT);
+
zhp = zfs_open(hdl, name, ZFS_TYPE_DATASET);
if (zhp == NULL)
return (-1);
clp = changelist_gather(zhp, ZFS_PROP_NAME, 0,
flags->force ? MS_FORCE : 0);
@@ -2153,20 +2085,30 @@
zc.zc_objset_type = DMU_OST_ZFS;
(void) strlcpy(zc.zc_name, name, sizeof (zc.zc_name));
if (tryname) {
(void) strcpy(newname, tryname);
-
(void) strlcpy(zc.zc_value, tryname, sizeof (zc.zc_value));
+ err = ioctl(hdl->libzfs_fd, ZFS_IOC_RENAME, &zc);
if (flags->verbose) {
- (void) printf("attempting rename %s to %s\n",
+ char errbuf[1024];
+ (void) snprintf(errbuf, sizeof (errbuf),
+ dgettext(TEXT_DOMAIN,
+ "attempting to rename '%s' to '%s': "),
zc.zc_name, zc.zc_value);
+ if (err == 0) {
+ (void) fprintf(stderr, dgettext(TEXT_DOMAIN,
+ "%s: success\n"), errbuf);
+ } else {
+ (void) zfs_standard_error(hdl, errno, errbuf);
}
- err = ioctl(hdl->libzfs_fd, ZFS_IOC_RENAME, &zc);
+ }
+
if (err == 0)
changelist_rename(clp, name, tryname);
+
} else {
err = ENOENT;
}
if (err != 0 && strncmp(name + baselen, "recv-", 5) != 0) {
@@ -2173,28 +2115,30 @@
seq++;
(void) snprintf(newname, ZFS_MAX_DATASET_NAME_LEN,
"%.*srecv-%u-%u", baselen, name, getpid(), seq);
(void) strlcpy(zc.zc_value, newname, sizeof (zc.zc_value));
+ err = ioctl(hdl->libzfs_fd, ZFS_IOC_RENAME, &zc);
if (flags->verbose) {
- (void) printf("failed - trying rename %s to %s\n",
+ char errbuf[1024];
+ (void) snprintf(errbuf, sizeof (errbuf),
+ dgettext(TEXT_DOMAIN,
+ "attempting to temporarily rename '%s' to '%s': "),
zc.zc_name, zc.zc_value);
+ if (err == 0) {
+ (void) fprintf(stderr, dgettext(TEXT_DOMAIN,
+ "%s: success\n"), errbuf);
+ } else {
+ (void) zfs_standard_error(hdl, errno, errbuf);
}
- err = ioctl(hdl->libzfs_fd, ZFS_IOC_RENAME, &zc);
+ }
+
if (err == 0)
changelist_rename(clp, name, newname);
- if (err && flags->verbose) {
- (void) printf("failed (%u) - "
- "will try again on next pass\n", errno);
- }
+
err = EAGAIN;
- } else if (flags->verbose) {
- if (err == 0)
- (void) printf("success\n");
- else
- (void) printf("failed (%u)\n", errno);
}
(void) changelist_postfix(clp);
changelist_free(clp);
@@ -2210,10 +2154,13 @@
prop_changelist_t *clp;
zfs_handle_t *zhp;
boolean_t defer = B_FALSE;
int spa_version;
+ if (!zfs_dataset_exists(hdl, name, ZFS_TYPE_DATASET))
+ return (ENOENT);
+
zhp = zfs_open(hdl, name, ZFS_TYPE_DATASET);
if (zhp == NULL)
return (-1);
clp = changelist_gather(zhp, ZFS_PROP_NAME, 0,
flags->force ? MS_FORCE : 0);
@@ -2229,20 +2176,27 @@
return (err);
zc.zc_objset_type = DMU_OST_ZFS;
zc.zc_defer_destroy = defer;
(void) strlcpy(zc.zc_name, name, sizeof (zc.zc_name));
-
- if (flags->verbose)
- (void) printf("attempting destroy %s\n", zc.zc_name);
err = ioctl(hdl->libzfs_fd, ZFS_IOC_DESTROY, &zc);
+
+ if (flags->verbose) {
+ char errbuf[1024];
+ (void) snprintf(errbuf, sizeof (errbuf), dgettext(TEXT_DOMAIN,
+ "attempting to destroy '%s'"), zc.zc_name);
if (err == 0) {
- if (flags->verbose)
- (void) printf("success\n");
- changelist_remove(clp, zc.zc_name);
+ (void) fprintf(stderr, dgettext(TEXT_DOMAIN,
+ "%s: success\n"), errbuf);
+ } else {
+ (void) zfs_standard_error(hdl, errno, errbuf);
}
+ }
+ if (err == 0)
+ changelist_remove(clp, zc.zc_name);
+
(void) changelist_postfix(clp);
changelist_free(clp);
/*
* Deferred destroy might destroy the snapshot or only mark it to be
@@ -2345,65 +2299,56 @@
return (ENOENT);
}
/*
- * Return +1 if guid1 is before guid2, 0 if they are the same, and -1 if
- * guid1 is after guid2.
+ * Returns a value:
+ * +1 - promote is reqired
+ * 0 - promote is not required
+ * -1 - an error is occured
*/
static int
-created_before(libzfs_handle_t *hdl, avl_tree_t *avl,
+check_promote(libzfs_handle_t *hdl, avl_tree_t *avl,
uint64_t guid1, uint64_t guid2)
{
nvlist_t *nvfs;
char *fsname, *snapname;
- char buf[ZFS_MAX_DATASET_NAME_LEN];
- int rv;
- zfs_handle_t *guid1hdl, *guid2hdl;
uint64_t create1, create2;
+ /* the local dataset is not cloned */
if (guid2 == 0)
return (0);
+
+ /* the stream dataset is not cloned */
if (guid1 == 0)
return (1);
nvfs = fsavl_find(avl, guid1, &snapname);
+ if (nvfs == NULL)
+ return (0);
VERIFY(0 == nvlist_lookup_string(nvfs, "name", &fsname));
- (void) snprintf(buf, sizeof (buf), "%s@%s", fsname, snapname);
- guid1hdl = zfs_open(hdl, buf, ZFS_TYPE_SNAPSHOT);
- if (guid1hdl == NULL)
- return (-1);
+ create1 = get_snap_txg(hdl, fsname, snapname);
nvfs = fsavl_find(avl, guid2, &snapname);
+ if (nvfs == NULL)
+ return (0);
VERIFY(0 == nvlist_lookup_string(nvfs, "name", &fsname));
- (void) snprintf(buf, sizeof (buf), "%s@%s", fsname, snapname);
- guid2hdl = zfs_open(hdl, buf, ZFS_TYPE_SNAPSHOT);
- if (guid2hdl == NULL) {
- zfs_close(guid1hdl);
+ create2 = get_snap_txg(hdl, fsname, snapname);
+
+ if (create1 == 0 || create2 == 0)
return (-1);
- }
- create1 = zfs_prop_get_int(guid1hdl, ZFS_PROP_CREATETXG);
- create2 = zfs_prop_get_int(guid2hdl, ZFS_PROP_CREATETXG);
-
if (create1 < create2)
- rv = -1;
- else if (create1 > create2)
- rv = +1;
- else
- rv = 0;
+ return (1);
- zfs_close(guid1hdl);
- zfs_close(guid2hdl);
-
- return (rv);
+ return (0);
}
static int
recv_incremental_replication(libzfs_handle_t *hdl, const char *tofs,
recvflags_t *flags, nvlist_t *stream_nv, avl_tree_t *stream_avl,
- nvlist_t *renamed)
+ nvlist_t *renamed, nvlist_t *limitds)
{
nvlist_t *local_nv;
avl_tree_t *local_avl;
nvpair_t *fselem, *nextfselem;
char *fromsnap;
@@ -2438,10 +2383,12 @@
uint64_t fromguid = 0;
uint64_t originguid = 0;
uint64_t stream_originguid = 0;
uint64_t parent_fromsnap_guid, stream_parent_fromsnap_guid;
char *fsname, *stream_fsname;
+ boolean_t stream_fs_exists = B_FALSE;
+ boolean_t stream_originfs_exists = B_FALSE;
nextfselem = nvlist_next_nvpair(local_nv, fselem);
VERIFY(0 == nvpair_value_nvlist(fselem, &nvfs));
VERIFY(0 == nvlist_lookup_nvlist(nvfs, "snaps", &snaps));
@@ -2448,97 +2395,180 @@
VERIFY(0 == nvlist_lookup_string(nvfs, "name", &fsname));
VERIFY(0 == nvlist_lookup_uint64(nvfs, "parentfromsnap",
&parent_fromsnap_guid));
(void) nvlist_lookup_uint64(nvfs, "origin", &originguid);
+ if (!nvlist_empty(limitds) && !nvlist_exists(limitds, fsname)) {
+ if (flags->verbose) {
+ (void) fprintf(stderr, dgettext(TEXT_DOMAIN,
+ "skip receiving for excluded '%s'\n"),
+ fsname);
+ }
+ continue;
+ }
+
/*
* First find the stream's fs, so we can check for
* a different origin (due to "zfs promote")
+ * and for preserving snapshots on the receiving side
*/
for (snapelem = nvlist_next_nvpair(snaps, NULL);
- snapelem; snapelem = nvlist_next_nvpair(snaps, snapelem)) {
- uint64_t thisguid;
+ snapelem != NULL; snapelem = nextsnapelem) {
+ uint64_t snapguid;
- VERIFY(0 == nvpair_value_uint64(snapelem, &thisguid));
- stream_nvfs = fsavl_find(stream_avl, thisguid, NULL);
+ nextsnapelem = nvlist_next_nvpair(snaps, snapelem);
+ VERIFY(0 == nvpair_value_uint64(snapelem, &snapguid));
+ stream_nvfs = fsavl_find(stream_avl, snapguid, NULL);
- if (stream_nvfs != NULL)
+ if (stream_nvfs != NULL) {
+ stream_fs_exists = B_TRUE;
break;
}
+ }
+ /* Check the stream's fs for origin snapshot */
+ if (stream_fs_exists && originguid != 0) {
+ nvlist_t *stream_snaps;
+
+ VERIFY(0 == nvlist_lookup_nvlist(stream_nvfs, "snaps",
+ &stream_snaps));
+
+ for (snapelem = nvlist_next_nvpair(stream_snaps, NULL);
+ snapelem != NULL; snapelem = nextsnapelem) {
+ uint64_t stream_snapguid;
+
+ nextsnapelem = nvlist_next_nvpair(stream_snaps,
+ snapelem);
+ VERIFY(0 == nvpair_value_uint64(snapelem,
+ &stream_snapguid));
+
+ if (stream_snapguid == originguid) {
+ stream_originfs_exists = B_TRUE;
+ break;
+ }
+ }
+ }
+
/* check for promote */
(void) nvlist_lookup_uint64(stream_nvfs, "origin",
&stream_originguid);
- if (stream_nvfs && originguid != stream_originguid) {
- switch (created_before(hdl, local_avl,
+ if (originguid != stream_originguid && stream_originfs_exists) {
+ switch (check_promote(hdl, local_avl,
stream_originguid, originguid)) {
+ case 0:
+ break;
case 1: {
/* promote it! */
zfs_cmd_t zc = { 0 };
- nvlist_t *origin_nvfs;
char *origin_fsname;
- if (flags->verbose)
- (void) printf("promoting %s\n", fsname);
-
- origin_nvfs = fsavl_find(local_avl, originguid,
- NULL);
- VERIFY(0 == nvlist_lookup_string(origin_nvfs,
- "name", &origin_fsname));
+ VERIFY(0 == nvlist_lookup_string(nvfs,
+ "origin_fsname", &origin_fsname));
(void) strlcpy(zc.zc_value, origin_fsname,
sizeof (zc.zc_value));
(void) strlcpy(zc.zc_name, fsname,
sizeof (zc.zc_name));
error = zfs_ioctl(hdl, ZFS_IOC_PROMOTE, &zc);
- if (error == 0)
- progress = B_TRUE;
- break;
+
+ if (flags->verbose) {
+ char errbuf[1024];
+ (void) snprintf(errbuf, sizeof (errbuf),
+ dgettext(TEXT_DOMAIN,
+ "attempting to promote '%s': "),
+ zc.zc_name);
+ if (error == 0) {
+ (void) fprintf(stderr,
+ dgettext(TEXT_DOMAIN,
+ "%s: success\n"), errbuf);
+ } else {
+ (void) zfs_standard_error(hdl,
+ errno, errbuf);
}
- default:
- break;
- case -1:
- fsavl_destroy(local_avl);
- nvlist_free(local_nv);
- return (-1);
}
+
+ if (error == 0)
+ progress = B_TRUE;
+
/*
* We had/have the wrong origin, therefore our
* list of snapshots is wrong. Need to handle
* them on the next pass.
*/
+
needagain = B_TRUE;
- continue;
+ goto out;
}
+ default:
+ progress = B_FALSE;
+ needagain = B_FALSE;
+ goto out;
+ }
+ }
for (snapelem = nvlist_next_nvpair(snaps, NULL);
- snapelem; snapelem = nextsnapelem) {
- uint64_t thisguid;
+ snapelem != NULL; snapelem = nextsnapelem) {
+ uint64_t snapguid;
char *stream_snapname;
nvlist_t *found, *props;
nextsnapelem = nvlist_next_nvpair(snaps, snapelem);
- VERIFY(0 == nvpair_value_uint64(snapelem, &thisguid));
- found = fsavl_find(stream_avl, thisguid,
+ VERIFY(0 == nvpair_value_uint64(snapelem, &snapguid));
+ found = fsavl_find(stream_avl, snapguid,
&stream_snapname);
/* check for delete */
if (found == NULL) {
char name[ZFS_MAX_DATASET_NAME_LEN];
+ /*
+ * Conventional force-receive (-F) behavior
+ * combines two different steps:
+ * 1. rollback the destination dataset to the
+ * most recent received snapshot
+ * 2. destroy all those destination snapshots
+ * that are not present at the source
+ * The keepsnap flag allows to effectively
+ * separate 1 from 2 and perform forced receive
+ * while still maintaining the destination
+ * snapshots as per the corresponding snapshot
+ * retention policy (at the destination).
+ */
+
+ /*
+ * When -F (force-receive) is not specified we
+ * always keep snapshots at the destination
+ * (i.e., this has always been zfs conventional
+ * behavior). See also 'keepsnap' comment below
+ */
if (!flags->force)
continue;
+ /*
+ * keepsnap flag modifies the conventional
+ * force-receive behavior not to destroy
+ * destination snapshots that are not present
+ * at the replication source
+ */
+ if (flags->keepsnap && stream_fs_exists)
+ continue;
+
+ /*
+ * Destroy destination snapshots that do
+ * not exist at the replication source
+ */
(void) snprintf(name, sizeof (name), "%s@%s",
fsname, nvpair_name(snapelem));
error = recv_destroy(hdl, name,
strlen(fsname)+1, newname, flags);
- if (error)
- needagain = B_TRUE;
- else
+
+ if (error == 0)
progress = B_TRUE;
+ else
+ needagain = B_TRUE;
+
continue;
}
stream_nvfs = found;
@@ -2569,44 +2599,49 @@
(void) snprintf(tryname, sizeof (name), "%s@%s",
fsname, stream_snapname);
error = recv_rename(hdl, name, tryname,
strlen(fsname)+1, newname, flags);
- if (error)
- needagain = B_TRUE;
- else
+
+ if (error == 0)
progress = B_TRUE;
+ else
+ needagain = B_TRUE;
}
if (strcmp(stream_snapname, fromsnap) == 0)
- fromguid = thisguid;
+ fromguid = snapguid;
}
/* check for delete */
if (stream_nvfs == NULL) {
if (!flags->force)
continue;
error = recv_destroy(hdl, fsname, strlen(tofs)+1,
newname, flags);
- if (error)
- needagain = B_TRUE;
- else
+
+ switch (error) {
+ case 0:
progress = B_TRUE;
- continue;
+ break;
+ case EAGAIN:
+ progress = B_TRUE;
+ needagain = B_TRUE;
+ goto out;
+ default:
+ needagain = B_TRUE;
+ break;
}
- if (fromguid == 0) {
- if (flags->verbose) {
- (void) printf("local fs %s does not have "
- "fromsnap (%s in stream); must have "
- "been deleted locally; ignoring\n",
- fsname, fromsnap);
- }
continue;
}
+ /* skip destroyed or re-created datasets */
+ if (fromguid == 0)
+ continue;
+
VERIFY(0 == nvlist_lookup_string(stream_nvfs,
"name", &stream_fsname));
VERIFY(0 == nvlist_lookup_uint64(stream_nvfs,
"parentfromsnap", &stream_parent_fromsnap_guid));
@@ -2643,12 +2678,14 @@
(void) snprintf(tryname, sizeof (tryname),
"%s%s", pname, strrchr(stream_fsname, '/'));
} else {
tryname[0] = '\0';
if (flags->verbose) {
- (void) printf("local fs %s new parent "
- "not found\n", fsname);
+ (void) fprintf(stderr,
+ dgettext(TEXT_DOMAIN,
+ "parent dataset not found for "
+ "local dataset '%s'\n"), fsname);
}
}
newname[0] = '\0';
@@ -2658,34 +2695,37 @@
if (renamed != NULL && newname[0] != '\0') {
VERIFY(0 == nvlist_add_boolean(renamed,
newname));
}
- if (error)
- needagain = B_TRUE;
- else
+ if (error == 0)
progress = B_TRUE;
+ else
+ needagain = B_TRUE;
}
}
+out:
fsavl_destroy(local_avl);
nvlist_free(local_nv);
if (needagain && progress) {
/* do another pass to fix up temporary names */
if (flags->verbose)
- (void) printf("another pass:\n");
+ (void) fprintf(stderr, dgettext(TEXT_DOMAIN,
+ "another pass for promote, destroy and rename:\n"));
goto again;
}
return (needagain);
}
static int
zfs_receive_package(libzfs_handle_t *hdl, int fd, const char *destname,
- recvflags_t *flags, dmu_replay_record_t *drr, zio_cksum_t *zc,
- char **top_zfs, int cleanup_fd, uint64_t *action_handlep)
+ recvflags_t *flags, nvlist_t *exprops, nvlist_t *limitds,
+ dmu_replay_record_t *drr, zio_cksum_t *zc, char **top_zfs, int cleanup_fd,
+ uint64_t *action_handlep)
{
nvlist_t *stream_nv = NULL;
avl_tree_t *stream_avl = NULL;
char *fromsnap = NULL;
char *sendsnap = NULL;
@@ -2762,14 +2802,19 @@
if (drr->drr_payloadlen != 0) {
nvlist_t *stream_fss;
VERIFY(0 == nvlist_lookup_nvlist(stream_nv, "fss",
&stream_fss));
- if ((stream_avl = fsavl_create(stream_fss)) == NULL) {
+ error = fsavl_create(stream_fss, &stream_avl);
+ if (error != 0) {
zfs_error_aux(hdl, dgettext(TEXT_DOMAIN,
"couldn't allocate avl tree"));
+ if (error == ENOMEM)
error = zfs_error(hdl, EZFS_NOMEM, errbuf);
+ else
+ error = zfs_error(hdl, EZFS_BADSTREAM, errbuf);
+
goto out;
}
if (fromsnap != NULL && recursive) {
nvlist_t *renamed = NULL;
@@ -2802,11 +2847,11 @@
VERIFY(0 == nvlist_alloc(&renamed,
NV_UNIQUE_NAME, 0));
}
softerr = recv_incremental_replication(hdl, tofs, flags,
- stream_nv, stream_avl, renamed);
+ stream_nv, stream_avl, renamed, limitds);
/* Unmount renamed filesystems before receiving. */
while ((pair = nvlist_next_nvpair(renamed,
pair)) != NULL) {
zfs_handle_t *zhp;
@@ -2856,12 +2901,12 @@
* Note, if we fail due to already having this guid,
* zfs_receive_one() will take care of it (ie,
* recv_skip() and return 0).
*/
error = zfs_receive_impl(hdl, destname, NULL, flags, fd,
- sendfs, stream_nv, stream_avl, top_zfs, cleanup_fd,
- action_handlep, sendsnap);
+ exprops, limitds, sendfs, stream_nv, stream_avl, top_zfs,
+ cleanup_fd, action_handlep, sendsnap);
if (error == ENODATA) {
error = 0;
break;
}
anyerr |= error;
@@ -2871,11 +2916,11 @@
/*
* Now that we have the fs's they sent us, try the
* renames again.
*/
softerr = recv_incremental_replication(hdl, tofs, flags,
- stream_nv, stream_avl, NULL);
+ stream_nv, stream_avl, NULL, limitds);
}
out:
fsavl_destroy(stream_avl);
nvlist_free(stream_nv);
@@ -3021,34 +3066,131 @@
}
zfs_close(zhp);
}
/*
+ * Calculate a list of properties for the current dataset taking into account
+ * stream properties (props) and the properties specified on the command line
+ * using -x and/or -o options (exprops)
+ *
+ * This calculation:
+ * - Removes excluded properties (booleans)
+ * - Changes the values of overridden properties (strings)
+ *
+ */
+static int
+props_override(char *dsname, nvlist_t *props, nvlist_t *exprops,
+ nvlist_t **merged_propsp, recvflags_t *flags, libzfs_handle_t *hdl,
+ zfs_type_t type, uint64_t zoned, zfs_handle_t *zhp,
+ zpool_handle_t *zpool_hdl, const char *errbuf)
+{
+ nvlist_t *goprops, *gxprops, *merged_props, *vprops;
+ nvpair_t *pair;
+ int ret = 0;
+
+ if (nvlist_empty(props) || nvlist_empty(exprops))
+ return (0); /* No properties */
+
+ if (nvlist_dup(props, &merged_props, 0) != 0)
+ return (-1);
+
+ VERIFY(nvlist_alloc(&goprops, NV_UNIQUE_NAME, 0) == 0);
+ VERIFY(nvlist_alloc(&gxprops, NV_UNIQUE_NAME, 0) == 0);
+
+ /* build lists to process in order */
+ for (pair = nvlist_next_nvpair(exprops, NULL); pair != NULL;
+ pair = nvlist_next_nvpair(exprops, pair)) {
+ const char *propname = nvpair_name(pair);
+ switch (nvpair_type(pair)) {
+ case DATA_TYPE_BOOLEAN:
+ VERIFY0(nvlist_add_nvpair(gxprops, pair));
+ break;
+ case DATA_TYPE_STRING:
+ VERIFY0(nvlist_add_nvpair(goprops, pair));
+ break;
+ default:
+ (void) fprintf(stderr, dgettext(TEXT_DOMAIN,
+ "property '%s' must be a string or boolean"),
+ propname);
+ /* should never happen, so assert */
+ assert(B_FALSE);
+ }
+ }
+
+ /* convert override properties e.g. strings to native */
+ if ((vprops = zfs_valid_proplist(hdl, type, goprops, zoned, zhp,
+ zpool_hdl, errbuf)) == NULL)
+ goto error;
+
+ nvlist_free(goprops);
+ goprops = vprops;
+
+ /* override / set properties */
+ for (nvpair_t *pair = nvlist_next_nvpair(goprops, NULL); pair != NULL;
+ pair = nvlist_next_nvpair(goprops, pair)) {
+ const char *pname = nvpair_name(pair);
+ if (!nvlist_exists(gxprops, pname)) {
+ if (flags->verbose) {
+ (void) printf("%s %s property from %s\n",
+ nvlist_exists(merged_props, pname) ?
+ "overriding" : "setting", pname, dsname);
+ }
+ VERIFY0(nvlist_add_nvpair(merged_props, pair));
+ }
+ }
+
+ /* exclude properties */
+ for (nvpair_t *pair = nvlist_next_nvpair(gxprops, NULL); pair != NULL;
+ pair = nvlist_next_nvpair(gxprops, pair)) {
+ const char *pname = nvpair_name(pair);
+ if (nvlist_exists(merged_props, pname)) {
+ if (flags->verbose) {
+ (void) printf("excluding %s property "
+ "from %s\n", pname, dsname);
+ }
+ VERIFY0(nvlist_remove_all(merged_props, pname));
+ }
+ }
+
+ *merged_propsp = merged_props;
+
+error:
+ if (0 != ret)
+ nvlist_free(merged_props);
+ nvlist_free(goprops);
+ nvlist_free(gxprops);
+
+ return (ret);
+}
+
+/*
* Restores a backup of tosnap from the file descriptor specified by infd.
*/
static int
zfs_receive_one(libzfs_handle_t *hdl, int infd, const char *tosnap,
- const char *originsnap, recvflags_t *flags, dmu_replay_record_t *drr,
- dmu_replay_record_t *drr_noswap, const char *sendfs, nvlist_t *stream_nv,
- avl_tree_t *stream_avl, char **top_zfs, int cleanup_fd,
- uint64_t *action_handlep, const char *finalsnap)
+ const char *originsnap, recvflags_t *flags, nvlist_t *exprops,
+ nvlist_t *limitds, dmu_replay_record_t *drr,
+ dmu_replay_record_t *drr_noswap,
+ const char *sendfs, nvlist_t *stream_nv, avl_tree_t *stream_avl,
+ char **top_zfs, int cleanup_fd, uint64_t *action_handlep, const char *finalsnap)
{
zfs_cmd_t zc = { 0 };
time_t begin_time;
int ioctl_err, ioctl_errno, err;
char *cp;
struct drr_begin *drrb = &drr->drr_u.drr_begin;
+ char dsname[ZFS_MAX_DATASET_NAME_LEN];
char errbuf[1024];
char prop_errbuf[1024];
const char *chopprefix;
boolean_t newfs = B_FALSE;
boolean_t stream_wantsnewfs;
uint64_t parent_snapguid = 0;
prop_changelist_t *clp = NULL;
- nvlist_t *snapprops_nvlist = NULL;
+ nvlist_t *snapprops_nvlist = NULL, *props = NULL, *merged_props = NULL;
zprop_errflags_t prop_errflags;
- boolean_t recursive;
+ boolean_t recursive, skip;
char *snapname = NULL;
begin_time = time(NULL);
(void) snprintf(errbuf, sizeof (errbuf), dgettext(TEXT_DOMAIN,
@@ -3056,14 +3198,13 @@
recursive = (nvlist_lookup_boolean(stream_nv, "not_recursive") ==
ENOENT);
if (stream_avl != NULL) {
+ nvlist_t *snapprops;
nvlist_t *fs = fsavl_find(stream_avl, drrb->drr_toguid,
&snapname);
- nvlist_t *props;
- int ret;
(void) nvlist_lookup_uint64(fs, "parentfromsnap",
&parent_snapguid);
err = nvlist_lookup_nvlist(fs, "props", &props);
if (err)
@@ -3071,21 +3212,20 @@
if (flags->canmountoff) {
VERIFY(0 == nvlist_add_uint64(props,
zfs_prop_to_name(ZFS_PROP_CANMOUNT), 0));
}
- ret = zcmd_write_src_nvlist(hdl, &zc, props);
- if (err)
+
+ if (err) {
nvlist_free(props);
+ props = NULL;
+ }
- if (0 == nvlist_lookup_nvlist(fs, "snapprops", &props)) {
- VERIFY(0 == nvlist_lookup_nvlist(props,
+ if (0 == nvlist_lookup_nvlist(fs, "snapprops", &snapprops)) {
+ VERIFY(0 == nvlist_lookup_nvlist(snapprops,
snapname, &snapprops_nvlist));
}
-
- if (ret != 0)
- return (-1);
}
cp = NULL;
/*
@@ -3190,10 +3330,13 @@
}
if (flags->verbose)
(void) printf("found clone origin %s\n", zc.zc_string);
}
+ (void) strcpy(dsname, drrb->drr_toname);
+ *strchr(dsname, '@') = '\0';
+
boolean_t resuming = DMU_GET_FEATUREFLAGS(drrb->drr_versioninfo) &
DMU_BACKUP_FEATURE_RESUMING;
stream_wantsnewfs = (drrb->drr_fromguid == NULL ||
(drrb->drr_flags & DRR_FLAG_CLONE) || originsnap) && !resuming;
@@ -3314,10 +3457,20 @@
zcmd_free_nvlists(&zc);
return (-1);
}
}
+ /* convert override properties e.g. strings to native */
+ if (!nvlist_empty(exprops) && props_override(dsname, props,
+ exprops, &merged_props, flags, hdl, zhp->zfs_type,
+ zfs_prop_get_int(zhp, ZFS_PROP_ZONED), zhp, zhp->zpool_hdl,
+ errbuf) != 0) {
+ zfs_close(zhp);
+ zcmd_free_nvlists(&zc);
+ return (-1);
+ }
+
/*
* If we are resuming a newfs, set newfs here so that we will
* mount it if the recv succeeds this time. We can tell
* that it was a newfs on the first recv because the fs
* itself will be inconsistent (if the fs existed when we
@@ -3355,34 +3508,81 @@
zcmd_free_nvlists(&zc);
return (zfs_error(hdl, EZFS_BADRESTORE, errbuf));
}
newfs = B_TRUE;
+
+ if (!nvlist_empty(exprops)) {
+ /* Create an override set of properties if needed */
+ uint64_t zoned = 0;
+ char zp_name[MAXNAMELEN];
+ zpool_handle_t *zp_handle;
+ if (flags->isprefix && !flags->istail &&
+ !flags->dryrun) {
+ /* Check if we're zoned or not */
+ if (check_parents(hdl, zc.zc_value, &zoned,
+ B_FALSE, NULL) != 0) {
+ zcmd_free_nvlists(&zc);
+ return (-1);
}
+ }
+ (void) strlcpy(zp_name, zc.zc_name, sizeof (zp_name));
+ cp = strchr(zp_name, '/');
+ if (cp != NULL)
+ *cp = '\0';
+ zp_handle = zpool_open(hdl, zp_name);
+ if (zp_handle != NULL &&
+ props_override(dsname, props,exprops,
+ &merged_props, flags, hdl, ZFS_TYPE_DATASET,
+ zoned, NULL, zp_handle, errbuf) != 0) {
+ zcmd_free_nvlists(&zc);
+ zpool_close(zp_handle);
+ return (-1);
+ }
+ if (zp_handle != NULL)
+ zpool_close(zp_handle);
+ }
+ }
+
zc.zc_begin_record = *drr_noswap;
zc.zc_cookie = infd;
zc.zc_guid = flags->force;
zc.zc_resumable = flags->resumable;
+ skip = !nvlist_empty(limitds) && !nvlist_exists(limitds, dsname);
if (flags->verbose) {
(void) printf("%s %s stream of %s into %s\n",
- flags->dryrun ? "would receive" : "receiving",
+ skip ? (flags->dryrun ? "would skip" : "skipping") :
+ (flags->dryrun ? "would receive" : "receiving"),
drrb->drr_fromguid ? "incremental" : "full",
drrb->drr_toname, zc.zc_value);
(void) fflush(stdout);
}
- if (flags->dryrun) {
+ if (flags->dryrun || skip) {
zcmd_free_nvlists(&zc);
return (recv_skip(hdl, infd, flags->byteswap));
}
zc.zc_nvlist_dst = (uint64_t)(uintptr_t)prop_errbuf;
zc.zc_nvlist_dst_size = sizeof (prop_errbuf);
zc.zc_cleanup_fd = cleanup_fd;
zc.zc_action_handle = *action_handlep;
+ /*
+ * if we ended up overriding props, use the merged ones,
+ * otherwise use the ones that we got from the send stream
+ */
+ if (merged_props) {
+ if (zcmd_write_src_nvlist(hdl, &zc, merged_props) != 0) {
+ nvlist_free(merged_props);
+ return (-1);
+ }
+ nvlist_free(merged_props);
+ } else if (props && zcmd_write_src_nvlist(hdl, &zc, props) != 0)
+ return (-1);
+
err = ioctl_err = zfs_ioctl(hdl, ZFS_IOC_RECV, &zc);
ioctl_errno = errno;
prop_errflags = (zprop_errflags_t)zc.zc_obj;
if (err == 0) {
@@ -3524,10 +3724,15 @@
case EDQUOT:
zfs_error_aux(hdl, dgettext(TEXT_DOMAIN,
"destination %s space quota exceeded"), zc.zc_name);
(void) zfs_error(hdl, EZFS_NOSPC, errbuf);
break;
+ case EKZFS_WBCNOTSUP:
+ zfs_error_aux(hdl, dgettext(TEXT_DOMAIN,
+ "write back cached datasets do not support recv"));
+ (void) zfs_error(hdl, EZFS_WBCNOTSUP, errbuf);
+ break;
default:
(void) zfs_standard_error(hdl, ioctl_errno, errbuf);
}
}
@@ -3600,11 +3805,12 @@
return (0);
}
static int
zfs_receive_impl(libzfs_handle_t *hdl, const char *tosnap,
- const char *originsnap, recvflags_t *flags, int infd, const char *sendfs,
+ const char *originsnap, recvflags_t *flags,
+ int infd, nvlist_t *exprops, nvlist_t *limitds, const char *sendfs,
nvlist_t *stream_nv, avl_tree_t *stream_avl, char **top_zfs, int cleanup_fd,
uint64_t *action_handlep, const char *finalsnap)
{
int err;
dmu_replay_record_t drr, drr_noswap;
@@ -3703,17 +3909,18 @@
*cp = '\0';
sendfs = nonpackage_sendfs;
VERIFY(finalsnap == NULL);
}
return (zfs_receive_one(hdl, infd, tosnap, originsnap, flags,
- &drr, &drr_noswap, sendfs, stream_nv, stream_avl, top_zfs,
- cleanup_fd, action_handlep, finalsnap));
+ exprops, limitds, &drr, &drr_noswap, sendfs, stream_nv,
+ stream_avl, top_zfs, cleanup_fd, action_handlep, finalsnap));
} else {
assert(DMU_GET_STREAM_HDRTYPE(drrb->drr_versioninfo) ==
DMU_COMPOUNDSTREAM);
- return (zfs_receive_package(hdl, infd, tosnap, flags, &drr,
- &zcksum, top_zfs, cleanup_fd, action_handlep));
+ return (zfs_receive_package(hdl, infd, tosnap, flags, exprops,
+ limitds, &drr, &zcksum, top_zfs, cleanup_fd,
+ action_handlep));
}
}
/*
* Restores a backup of tosnap from the file descriptor specified by infd.
@@ -3721,29 +3928,30 @@
* destroyed/renamed/promoted, -1 if some things couldn't be received.
* (-1 will override -2, if -1 and the resumable flag was specified the
* transfer can be resumed if the sending side supports it).
*/
int
-zfs_receive(libzfs_handle_t *hdl, const char *tosnap, nvlist_t *props,
- recvflags_t *flags, int infd, avl_tree_t *stream_avl)
+zfs_receive(libzfs_handle_t *hdl, const char *tosnap, recvflags_t *flags,
+ int infd, nvlist_t *exprops, nvlist_t *limitds, avl_tree_t *stream_avl)
{
char *top_zfs = NULL;
int err;
int cleanup_fd;
uint64_t action_handle = 0;
char *originsnap = NULL;
- if (props) {
- err = nvlist_lookup_string(props, "origin", &originsnap);
+ if (exprops) {
+ err = nvlist_lookup_string(exprops, "origin", &originsnap);
if (err && err != ENOENT)
return (err);
}
cleanup_fd = open(ZFS_DEV, O_RDWR|O_EXCL);
VERIFY(cleanup_fd >= 0);
- err = zfs_receive_impl(hdl, tosnap, originsnap, flags, infd, NULL, NULL,
- stream_avl, &top_zfs, cleanup_fd, &action_handle, NULL);
+ err = zfs_receive_impl(hdl, tosnap, originsnap, flags, infd, exprops,
+ limitds, NULL, NULL, stream_avl, &top_zfs, cleanup_fd,
+ &action_handle, NULL);
VERIFY(0 == close(cleanup_fd));
if (err == 0 && !flags->nomount && top_zfs) {
zfs_handle_t *zhp;