Features/trash : Combined patches for trash translator

This is the combined patch set for supporting trash feature.
http://www.gluster.org/community/documentation/index.php/Features/Trash

Current patch includes the following features:
* volume set options for enabling trash globally and
  exclusively for internal operations like self-heal
  and re-balance
* volume set options for setting the eliminate
  path, trash directory path and maximum trashable
  file size.
* test script for checking the functionality of the
  feature
* brief documentation on different aspects of trash
  feature.

Change-Id: Ic7486982dcd6e295d1eba0f4d5ee6d33bf1b4cb3
BUG: 1132465
Signed-off-by: Anoop C S <achiraya@redhat.com>
Signed-off-by: Jiffin Tony Thottan <jthottan@redhat.com>
Reviewed-on: http://review.gluster.org/8312
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
This commit is contained in:
Anoop C S 2015-02-27 15:14:08 +05:30 committed by Vijay Bellur
parent 393cdb2613
commit 0ef870741a
12 changed files with 2423 additions and 764 deletions

View File

@ -144,6 +144,8 @@ AC_CONFIG_FILES([Makefile
xlators/features/protect/src/Makefile
xlators/features/gfid-access/Makefile
xlators/features/gfid-access/src/Makefile
xlators/features/trash/Makefile
xlators/features/trash/src/Makefile
xlators/features/snapview-server/Makefile
xlators/features/snapview-server/src/Makefile
xlators/features/snapview-client/Makefile

80
doc/xlators/trash.md Normal file
View File

@ -0,0 +1,80 @@
Trash Translator
=================
Trash translator will allow users to access deleted or truncated files. Every brick will maintain a hidden .trashcan directory , which will be used to store the files deleted or truncated from the respective brick .The aggreagate of all those .trashcan directory can be accesed from the mount point.In order to avoid name collisions , a time stamp is appended to the original file name while it is being moved to trash directory.
##Implications and Usage
Apart from the primary use-case of accessing files deleted or truncated by user , the trash translator can be helpful for internal operations such as self-heal and rebalance . During self-heal and rebalance it is possible to lose crucial data.In those circumstances the trash translator can assist in recovery of the lost data. The trash translator is designed to intercept unlink, truncate and ftruncate fops, store a copy of the current file in the trash directory, and then perform the fop on the original file. For the internal operations , the files are stored under 'internal_op' folder inside trash directory.
##Volume Options
1. *gluster volume set &lt;VOLNAME> features.trash &lt;on | off>*
This command can be used to enable trash translator in a volume. If set to on, trash directory will be created in every brick inside the volume during volume start command. By default translator is loaded during volume start but remains non-functional. Disabling trash with the help of this option will not remove the trash directory or even its contents from the volume.
2. *gluster volume set &lt;VOLNAME> features.trash-dir &lt;name>*
This command is used to reconfigure the trash directory to a user specified name. The argument is a valid directory name. Directory will be created inside every brick under this name. If not specified by the user, the trash translator will create the trash directory with the default name “.trashcan”. This can be used only when trash-translator is on.
3. *gluster volume set &lt;VOLNAME> features.trash-max-filesize &lt;size>*
This command can be used to filter files entering trash directory based on their size. Files above trash_max_filesize are deleted/truncated directly. Value for size may be followed by mutliplicative suffixes KB (=1024), MB (=1024*1024 and GB. Default size is set to 5MB. As of now any value specified higher than 1GB will be changed to 1GB at the maximum level.
4. *gluster volume set &lt;VOLNAME> features.trash-eliminate-path &lt;path1> [ , &lt;path2> , . . . ]*
This command can be used to set the eliminate pattern for the trash translator. Files residing under this pattern will not be moved to trash directory during deletion/truncation. Path must be a valid one present in volume.
5. *gluster volume set &lt;VOLNAME> features.trash-internal-op &lt;on | off>*
This command can be used to enable trash for internal operations like self-heal and re-balance. By default set to off.
##Testing
Following steps give illustrates a simple scenario of deletion of file from directory
1. Create a distributed volume with two bricks and start it.
gluster volume create test rhs:/home/brick
gluster volume start test
2. Enable trash translator
gluster volume set test feature.trash on
3. Mount glusterfs client as follows.
mount -t glusterfs rhs:test /mnt
4. Create a directory and file in the mount.
mkdir mnt/dir
echo abc > mnt/dir/file
5. Delete the file from the mount.
rm mnt/dir/file -rf
6. Checkout inside the trash directory.
ls mnt/.trashcan
We can find the deleted file inside the trash directory with timestamp appending on its filename.
For example,
[root@rh-host ~]#mount -t glusterfs rh-host:/test /mnt/test
[root@rh-host ~]#mkdir /mnt/test/abc
[root@rh-host ~]#touch /mnt/test/abc/file
[root@rh-host ~]#rm /mnt/test/abc/filer
remove regular empty file /mnt/test/abc/file? y
[root@rh-host ~]#ls /mnt/test/abc
[root@rh-host ~]#
[root@rh-host ~]#ls /mnt/test/.trashcan/abc/
file2014-08-21_123400
##Points to be remembered
[1] As soon as the volume is started, trash directory will be created inside the volume and will be visible through mount. Disabling trash will not have any impact on its visibilty from the mount.
[2] Eventhough deletion of trash-directory is not permitted, currently residing trash contents will be removed on issuing delete on it and only an empty trash-directory exists.
##Known issues
[1] Since trash translator resides on the server side, DHT translator is unaware of rename and truncate operations being done by this translator which will eventually moves the files to trash directory. Unless and until a complete-path-based lookup comes on trashed files, those may not be visible from the mount.

218
tests/features/trash.t Executable file
View File

@ -0,0 +1,218 @@
#!/bin/bash
. $(dirname $0)/../include.rc
. $(dirname $0)/../volume.rc
cleanup
test_mount() {
glusterfs -s $H0 --volfile-id $V0 $M0 --attribute-timeout=0
test -d $M0/.trashcan
}
start_vol() {
$CLI volume start $V0
test_mount
}
stop_vol() {
umount $M0
$CLI volume stop $V0
}
create_files() {
echo 'Hi' > $1
echo 'Hai' > $2
}
file_exists() {
test -e $B0/${V0}1/$1 -o -e $B0/${V0}2/$1
test -e $B0/${V0}1/$2 -o -e $B0/${V0}2/$2
}
unlink_op() {
rm -f $M0/$1
ls $M0/.trashcan/1/2/3 &> /dev/null
sleep 2
test ! -e $M0/$1
test -e $M0/.trashcan/$1*
# remove from trashcan
rm -f $M0/.trashcan/$1*
test ! -e $M0/.trashcan/$1*
}
truncate_op() {
truncate -s 2 $M0/$1
ls $M0/.trashcan/1/2/3 &> /dev/null
sleep 2
test -e $M0/$1
test $(du -b $M0/$1 | awk '{print $1}') -eq 2 &>/dev/null
test -e $M0/.trashcan/$1*
test $(du -b $M0/.trashcan/$1*|awk '{print $1}') -eq $2 &>/dev/null
# truncate from trashcan
truncate -s 1 $M0/.trashcan/$1*
test $(ls $M0/.trashcan/$1* | wc -l) -eq 1
}
# testing glusterd [1-3]
TEST glusterd
TEST pidof glusterd
TEST $CLI volume info
# creating distributed volume [4]
TEST $CLI volume create $V0 $H0:$B0/${V0}{1,2}
# checking volume status [5-7]
EXPECT "$V0" volinfo_field $V0 'Volume Name'
EXPECT 'Created' volinfo_field $V0 'Status'
EXPECT '2' brick_count $V0
# test without enabling trash translator [8-10]
TEST $CLI volume start $V0
TEST glusterfs -s $H0 --volfile-id $V0 $M0 --attribute-timeout=0
TEST [ -d $M0/.trashcan ]
# test on enabling trash translator [11-12]
TEST $CLI volume set $V0 features.trash on
EXPECT 'on' volinfo_field $V0 'features.trash'
# files directly under mount point [13]
create_files $M0/file1 $M0/file2
TEST file_exists file1 file2
# perform unlink [14]
TEST unlink_op file1
# perform truncate [15]
TEST truncate_op file2 4
# create files directory hierarchy and check [16]
mkdir $M0/1/2/3 -p
create_files $M0/1/2/3/foo1 $M0/1/2/3/foo2
TEST file_exists 1/2/3/foo1 1/2/3/foo2
# perform unlink [17]
TEST unlink_op 1/2/3/foo1
# perform truncate [18]
TEST truncate_op 1/2/3/foo2 4
# create a directory for eliminate pattern
mkdir $M0/a
# set the eliminate pattern [19-20]
TEST $CLI volume set $V0 features.trash-eliminate-path /a
EXPECT '/a' volinfo_field $V0 'features.trash-eliminate-path'
# create two files and check [21]
create_files $M0/a/test1 $M0/a/test2
TEST file_exists a/test1 a/test2
# remove from eliminate pattern [22]
rm -f $M0/a/test1
TEST [ ! -e $M0/.trashcan/a/test1* ]
# truncate from eliminate path [23-25]
truncate -s 2 $M0/a/test2
TEST [ -e $M0/a/test2 ]
TEST [ `du -b $M0/a/test2 | awk '{print $1}'` -eq 2 ]
TEST [ ! -e $M0/.trashcan/a/test2* ]
# set internal op on [26-27]
TEST $CLI volume set $V0 features.trash-internal-op on
EXPECT 'on' volinfo_field $V0 'features.trash-internal-op'
# again create two files and check [28]
create_files $M0/inop1 $M0/inop2
TEST file_exists inop1 inop2
# perform unlink [29]
TEST unlink_op inop1
# perform truncate [30]
TEST truncate_op inop2 4
# remove one brick and restart the volume [31-33]
TEST $CLI volume remove-brick $V0 $H0:$B0/${V0}2 force
TEST stop_vol
TEST start_vol
# again create two files and check [34]
create_files $M0/rebal1 $M0/rebal2
TEST file_exists rebal1 rebal2
# add one brick [35-36]
TEST $CLI volume add-brick $V0 $H0:$B0/${V0}3
TEST [ -d $B0/${V0}3 ]
# perform rebalance [37]
TEST $CLI volume rebalance $V0 start force
sleep 3
# check whether rebalance was succesful [38-40]
TEST [ -e $B0/${V0}3/rebal2 ]
TEST [ -e $B0/${V0}1/.trashcan/internal_op/rebal2* ]
TEST stop_vol
# create a replicated volume [41]
TEST $CLI volume create $V1 replica 2 $H0:$B0/${V1}{1,2}
# checking volume status [42-45]
EXPECT "$V1" volinfo_field $V1 'Volume Name'
EXPECT 'Replicate' volinfo_field $V1 'Type'
EXPECT 'Created' volinfo_field $V1 'Status'
EXPECT '2' brick_count $V1
# enable trash with options and start the replicate volume by disabling automatic self-heal [46-52]
TEST $CLI volume set $V1 features.trash on
TEST $CLI volume set $V1 features.trash-internal-op on
TEST $CLI volume set $V1 cluster.self-heal-daemon off
EXPECT 'on' volinfo_field $V1 'features.trash'
EXPECT 'on' volinfo_field $V1 'features.trash-internal-op'
EXPECT 'off' volinfo_field $V1 'cluster.self-heal-daemon'
TEST $CLI volume start $V1
# mount and check for trash directory [53]
glusterfs -s $H0 --volfile-id $V1 $M1 --attribute-timeout=0
TEST [ -d $M1/.trashcan/internal_op ]
# create a file and check [54]
touch $M1/self
TEST [ -e $B0/${V1}1/self -a -e $B0/${V1}2/self ]
# kill one brick and delete the file from mount point [55]
kill `ps aux| grep glusterfsd | awk '{print $2}' | head -1`
sleep 2
rm -f $M1/self
TEST [ -e $M1/.trashcan/self* ]
# force start the volume and trigger the self-heal manually [56]
TEST $CLI volume start $V1 force
sleep 3
# check for the removed file in trashcan [57]
TEST [ -e $B0/${V1}1/.trashcan/internal_op/self* -o -e $B0/${V1}2/.trashcan/internal_op/self* ]
# check renaming of trash directory through cli [58-62]
TEST $CLI volume set $V0 trash-dir abc
TEST $CLI volume start $V0
TEST glusterfs -s $H0 --volfile-id $V0 $M0 --attribute-timeout=0
TEST [ -e $M0/abc -a ! -e $M0/.trashcan ]
TEST [ -e $B0/${V0}1/abc/internal_op/rebal2* ]
sleep 2
# ensure that rename and delete operation on trash directory fails [63-65]
rm -rf $M0/abc/internal_op
TEST [ -e $M0/abc/internal_op ]
rm -rf $M0/abc/
TEST [ -e $M0/abc ]
mv $M0/abc $M0/trash
TEST [ -e $M0/abc ]
cleanup

View File

@ -1,4 +1,4 @@
SUBDIRS = locks quota read-only mac-compat quiesce marker index barrier \
protect compress changelog gfid-access $(GLUPY_SUBDIR) qemu-block snapview-client snapview-server # trash path-converter # filter
protect compress changelog gfid-access $(GLUPY_SUBDIR) qemu-block snapview-client snapview-server trash # path-converter # filter
CLEANFILES =

View File

@ -1,5 +1,5 @@
xlator_LTLIBRARIES = trash.la
xlatordir = $(libdir)/glusterfs/$(PACKAGE_VERSION)/xlator/testing/features
xlatordir = $(libdir)/glusterfs/$(PACKAGE_VERSION)/xlator/features
trash_la_LDFLAGS = -module -avoid-version

View File

@ -15,7 +15,8 @@
enum gf_trash_mem_types_ {
gf_trash_mt_trash_private_t = gf_common_mt_end + 1,
gf_trash_mt_char,
gf_trash_mt_trash_elim_pattern_t,
gf_trash_mt_uuid,
gf_trash_mt_trash_elim_path,
gf_trash_mt_end
};
#endif

File diff suppressed because it is too large Load Diff

View File

@ -37,7 +37,6 @@
#define GF_ALLOWED_MAX_FILE_SIZE (1 * GF_UNIT_GB)
#endif
struct trash_struct {
fd_t *fd; /* for the fd of existing file */
fd_t *newfd; /* for the newly created file */
@ -54,26 +53,31 @@ struct trash_struct {
};
typedef struct trash_struct trash_local_t;
struct _trash_elim_pattern;
typedef struct _trash_elim_pattern {
struct _trash_elim_pattern *next;
char *pattern;
} trash_elim_pattern_t;
struct _trash_elim_path {
struct _trash_elim_path *next;
char *path;
};
typedef struct _trash_elim_path trash_elim_path;
struct trash_priv {
char *trash_dir;
trash_elim_pattern_t *eliminate;
char *oldtrash_dir;
char *newtrash_dir;
char *brick_path;
trash_elim_path *eliminate;
size_t max_trash_file_size;
gf_boolean_t state;
gf_boolean_t internal;
inode_t *trash_inode;
inode_table_t *trash_itable;
};
typedef struct trash_priv trash_private_t;
#define TRASH_STACK_UNWIND(op, frame, params ...) do { \
trash_local_t *__local = NULL; \
__local = frame->local; \
frame->local = NULL; \
STACK_UNWIND_STRICT (op, frame, params); \
trash_local_wipe (__local); \
} while (0)
#define TRASH_STACK_UNWIND(op, frame, params ...) do { \
trash_local_t *__local = NULL; \
__local = frame->local; \
frame->local = NULL; \
STACK_UNWIND_STRICT (op, frame, params); \
trash_local_wipe (__local); \
} while (0)
#endif /* __TRASH_H__ */

View File

@ -647,11 +647,14 @@ glusterd_op_stage_set_volume (dict_t *dict, char **op_errstr)
char *key = NULL;
char *key_fixed = NULL;
char *value = NULL;
char *val_dup = NULL;
char str[100] = {0, };
char *trash_path = NULL;
int count = 0;
int dict_count = 0;
char errstr[2048] = {0, };
glusterd_volinfo_t *volinfo = NULL;
glusterd_brickinfo_t *brickinfo = NULL;
dict_t *val_dict = NULL;
gf_boolean_t global_opt = _gf_false;
glusterd_volinfo_t *voliter = NULL;
@ -663,7 +666,9 @@ glusterd_op_stage_set_volume (dict_t *dict, char **op_errstr)
uint32_t local_key_op_version = 0;
gf_boolean_t origin_glusterd = _gf_true;
gf_boolean_t check_op_version = _gf_true;
gf_boolean_t trash_enabled = _gf_false;
gf_boolean_t all_vol = _gf_false;
struct stat stbuf = {0, };
GF_ASSERT (dict);
this = THIS;
@ -940,6 +945,76 @@ glusterd_op_stage_set_volume (dict_t *dict, char **op_errstr)
if (glusterd_check_globaloption (key))
global_opt = _gf_true;
if (volinfo) {
ret = glusterd_volinfo_get (volinfo,
VKEY_FEATURES_TRASH, &val_dup);
if (val_dup) {
ret = gf_string2boolean (val_dup,
&trash_enabled);
if (ret)
goto out;
}
}
if (!strcmp(key, "features.trash-dir") && trash_enabled) {
if (strchr (value, '/')) {
snprintf (errstr, sizeof (errstr),
"Path is not allowed as option");
gf_log (this->name, GF_LOG_ERROR,
"Unable to set the options in 'volume "
"set': %s", errstr);
ret = -1;
goto out;
}
list_for_each_entry (brickinfo, &volinfo->bricks,
brick_list) {
/* Check for local brick */
if (!uuid_compare (brickinfo->uuid, MY_UUID)) {
trash_path = gf_strdup (brickinfo->path);
strcat(trash_path, "/");
strcat(trash_path, value);
/* Checks whether a directory with
given option exists or not */
if (!stat(trash_path, &stbuf)) {
snprintf (errstr, sizeof (errstr),
"Path %s exists", value);
gf_log (this->name, GF_LOG_ERROR,
"Unable to set the options in "
"'volume set': %s", errstr);
ret = -1;
goto out;
} else {
gf_log (this->name, GF_LOG_DEBUG,
"Directory with given name "
"does not exists, continuing");
}
if (volinfo->status == GLUSTERD_STATUS_STARTED
&& brickinfo->status != GF_BRICK_STARTED) {
/* If volume is in started state , checks
whether bricks are online */
snprintf (errstr, sizeof (errstr),
"One or more bricks are down");
gf_log (this->name, GF_LOG_ERROR,
"Unable to set the options in "
"'volume set': %s", errstr);
ret = -1;
goto out;
}
}
}
} else if (!strcmp(key, "features.trash-dir") && !trash_enabled) {
snprintf (errstr, sizeof (errstr),
"Trash translator is not enabled. Use "
"volume set %s trash on", volname);
gf_log (this->name, GF_LOG_ERROR,
"Unable to set the options in 'volume "
"set': %s", errstr);
ret = -1;
goto out;
}
ret = dict_set_str (val_dict, key, value);
if (ret) {
@ -1015,6 +1090,9 @@ out:
if (val_dict)
dict_unref (val_dict);
if (trash_path)
GF_FREE (trash_path);
GF_FREE (key_fixed);
if (errstr[0] != '\0')
*op_errstr = gf_strdup (errstr);

View File

@ -1421,6 +1421,7 @@ brick_graph_add_posix (volgen_graph_t *graph, glusterd_volinfo_t *volinfo,
{
int ret = -1;
gf_boolean_t quota_enabled = _gf_true;
gf_boolean_t trash_enabled = _gf_false;
gf_boolean_t pgfid_feat = _gf_false;
char *value = NULL;
xlator_t *xl = NULL;
@ -1435,6 +1436,13 @@ brick_graph_add_posix (volgen_graph_t *graph, glusterd_volinfo_t *volinfo,
goto out;
}
ret = glusterd_volinfo_get (volinfo, VKEY_FEATURES_TRASH, &value);
if (value) {
ret = gf_string2boolean (value, &trash_enabled);
if (ret)
goto out;
}
ret = glusterd_volinfo_get (volinfo,
"update-link-count-parent",
&value);
@ -1459,13 +1467,36 @@ brick_graph_add_posix (volgen_graph_t *graph, glusterd_volinfo_t *volinfo,
if (ret)
goto out;
if (quota_enabled || pgfid_feat)
if (quota_enabled || pgfid_feat || trash_enabled)
xlator_set_option (xl, "update-link-count-parent",
"on");
out:
return ret;
}
static int
brick_graph_add_trash (volgen_graph_t *graph, glusterd_volinfo_t *volinfo,
dict_t *set_dict, glusterd_brickinfo_t *brickinfo)
{
int ret = -1;
xlator_t *xl = NULL;
xl = volgen_graph_add (graph, "features/trash", volinfo->volname);
if (!xl)
goto out;
ret = xlator_set_option (xl, "trash-dir", ".trashcan");
if (ret)
goto out;
ret = xlator_set_option (xl, "brick-path", brickinfo->path);
if (ret)
goto out;
ret = xlator_set_option (xl, "trash-internal-op", "off");
if (ret)
goto out;
out:
return ret;
}
static int
brick_graph_add_bd (volgen_graph_t *graph, glusterd_volinfo_t *volinfo,
dict_t *set_dict, glusterd_brickinfo_t *brickinfo)
@ -2018,6 +2049,7 @@ static volgen_brick_xlator_t server_graph_table[] = {
{brick_graph_add_acl, "acl"},
{brick_graph_add_changelog, "changelog"},
{brick_graph_add_bd, "bd"},
{brick_graph_add_trash, "trash"},
{brick_graph_add_posix, "posix"},
};

View File

@ -32,6 +32,7 @@
#define VKEY_MARKER_XTIME_FORCE GEOREP".ignore-pid-check"
#define VKEY_CHANGELOG "changelog.changelog"
#define VKEY_FEATURES_QUOTA "features.quota"
#define VKEY_FEATURES_TRASH "features.trash"
#define AUTH_ALLOW_MAP_KEY "auth.allow"
#define AUTH_REJECT_MAP_KEY "auth.reject"

View File

@ -1641,6 +1641,28 @@ struct volopt_map_entry glusterd_volopt_map[] = {
.voltype = "mgmt/glusterd",
.op_version = GD_OP_VERSION_3_6_0,
},
/*Trash translator options */
{ .key = "features.trash",
.voltype = "features/trash",
.op_version = GD_OP_VERSION_3_7_0,
},
{ .key = "features.trash-dir",
.voltype = "features/trash",
.op_version = GD_OP_VERSION_3_7_0,
},
{ .key = "features.trash-eliminate-path",
.voltype = "features/trash",
.op_version = GD_OP_VERSION_3_7_0,
},
{ .key = "features.trash-max-filesize",
.voltype = "features/trash",
.op_version = GD_OP_VERSION_3_7_0,
},
{ .key = "features.trash-internal-op",
.voltype = "features/trash",
.op_version = GD_OP_VERSION_3_7_0,
},
{ .key = "locks.trace",
.voltype = "features/locks",
.type = NO_DOC,