313 Commits

Author SHA1 Message Date
Varun Shastry
2a4c48b4ed cluster/dht: update ctx-time only if we receive the new iatt
1. Used local->postparent(contains merged iatt of all succesful calls) instead
of postparent for dht ctx time update.

2. dht_inode_ctx_time_update avoided in case of opret -1.

Change-Id: Ie04a7842a41c241f911b6a3f76267b996d27fb43
BUG: 881013
Signed-off-by: Varun Shastry <vshastry@redhat.com>
Reviewed-on: http://review.gluster.org/4338
Reviewed-by: Shishir Gowda <sgowda@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2013-01-17 00:26:57 -08:00
shishir gowda
5267406e5b cluster/dht: Add "afr.readdir-failover=off" option the rebalance process
By failing over readdir (default behaviour), rebalance could get duplicate
files, as readdir would re-read from offset 0. Rebalance should not attempt
to migrate these files again.

Additionally, we need to handle these cases as failure in rebalance crawl.

No test case provided, as we cannot determine the read child in afr.

Change-Id: If07508b4f92dacc17e0f695b48a866c7c66004be
BUG: 859387
Signed-off-by: shishir gowda <sgowda@redhat.com>
Reviewed-on: http://review.gluster.org/4300
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2012-12-16 19:45:53 -08:00
shishir gowda
448b59e60b cluster/distribute: re-set layouts to prevent overlaps
When subvols-per-directory option is used, with bricks addition/removal
the layouts might get distributed to other subvols, which were not part
of the layout before. We need to clean up layouts on old subvolumes, to
prevent overlaps.

Also, we need to make sure if layout-cnt is never less
than subvolume-cnt.

Change-Id: I00994a092ca0c99aedcc41bd9412d43460f88a04
BUG: 884455
Signed-off-by: shishir gowda <sgowda@redhat.com>
Reviewed-on: http://review.gluster.org/4281
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2012-12-11 16:05:50 -08:00
Jeff Darcy
86b01a278b dht: support auto-NUFA option
Many people have asked for behavior like the old NUFA, which builds and
seems to run but was previously impossible to enable/configure in a
standard way.  This change allows NUFA to be enabled instead of DHT from
the command line, with automatic selection of the local subvolume on each
host.

Change-Id: I0065938db3922361fd450a6c1919a4cbbf6f202e
BUG: 882278
Signed-off-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-on: http://review.gluster.org/4234
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2012-12-04 14:42:14 -08:00
Raghavendra Bhat
48a1738467 fix memory leaks
* write-behind: free the inode context in wb_forget
* distribute: in readdirp callback put the allocated context to the inode
* distribute: check if the layout is NULL before accessing it in layout_unref

Change-Id: I7698f81b85b99d06bf6b01fc1a6e51e1593b5e27
BUG: 790709
Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com>
Reviewed-on: http://review.gluster.org/4250
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2012-12-04 12:14:28 -08:00
shishir gowda
bde0e14263 cluster/dht: fail fix-layout if any of the subvol is down
If any subvolume is down, and a layout is re-written and hash
values change, entry names in the downed subvol can be reused
in the other subvol which got the same hash range. when the
downed subvol is brought back up, duplicate entried might appear

Also separated handling of ENOSPC and ENOTCONN error.

Change-Id: I5ed93990425a4cee70df2dab7c7c119fdc87ad56
BUG: 860663
Signed-off-by: shishir gowda <sgowda@redhat.com>
Reviewed-on: http://review.gluster.org/4000
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2012-11-29 22:11:06 -08:00
shishir gowda
01cd59506e cluster/dht: Heal dir uid/gid
Identify mismatching uid/gid in lookup, and trigger a syncop
heal. uid/gid of subvol with latest ctime is trusted (local->prebuf).

Change-Id: Ib5c4bc438e7f4b1f33080e73593f40f400e997f0
BUG: 862967
Signed-off-by: shishir gowda <sgowda@redhat.com>
Reviewed-on: http://review.gluster.org/3964
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2012-11-29 13:56:11 -08:00
shishir gowda
c85a3eee54 cluster/dht: send ACCESS call on dir to first_up_subvol if cached is down
Change-Id: I4f518a969bbe3a11075e7c9ae10bd21bf059d5f3
BUG: 867253
Signed-off-by: shishir gowda <sgowda@redhat.com>
Reviewed-on: http://review.gluster.org/4240
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
2012-11-29 02:26:18 -08:00
Raghavendra
da0972232e cluster/distribute: send getxattr on LOCKINFO to only cached subvolumes.
lk is sent to only cached subvolume. Hence there is no point in
sending LOCKINFO to other children (even in case of directories).

Change-Id: Ia20fc358dfa84cee9a52d1f613564ff6f25aa0c9
BUG: 808400
Signed-off-by: Raghavendra <raghavendra@gluster.com>
Reviewed-on: http://review.gluster.org/4123
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2012-11-27 22:43:57 -08:00
Pranith Kumar K
76a4afec6e libglusterfs: Implement float percentage
Change-Id: Ia7ea63471f0bbd74686873f5f6f183475880f1a0
BUG: 839595
Signed-off-by: Pranith Kumar K <pranithk@gluster.com>
Reviewed-on: http://review.gluster.org/4162
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2012-11-23 23:13:39 -08:00
Raghavendra Bhat
f90ca96f54 cluster/dht: dump the layout information of directories only
testcase:
The changes are for removing gf_log from statedump related sections in dht and
using pthread_mutex_trylock in statedump sections. Changes are internal. So
tests were done by attaching gdb to the process and executing by manually
changing the values of some of the pointers.

Change-Id: I41fa76c1812b462cb76f5bbf2fd14de080e73895
BUG: 843822
Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com>
Reviewed-on: http://review.gluster.org/4117
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2012-11-19 02:58:06 -08:00
Venky Shankar
59dfcf1557 cluster/dht: ignore empty ->hashed_subvol during lookup
->hashed_subvol is not valid (== NULL) when the subvolume
the entity hashes to is down. For directories, we need not
rely on ->hashed_subvol as we aggregate information from all
subvolumes. So, during lookup, NULL ->hashed_subvol is ingored
but logged.

Change-Id: I306e4e274fe29d60ff028add4a6c3bcd67b2f314
Signed-off-by: Venky Shankar <vshankar@redhat.com>
BUG: 856459
Reviewed-on: http://review.gluster.org/4046
Reviewed-by: Anand Avati <avati@redhat.com>
Tested-by: Anand Avati <avati@redhat.com>
2012-10-17 00:16:05 -07:00
shishir gowda
53e49e8592 cluster/distribute: Always return the latest time in struct iatt.
save the a/c/mtime in inode_ctx, and dht_inode_ctx_update
checks the passed iatte, and updates the stat's time,
and inode_ctx's time accordingly. For preparent times, only
the iatt stat to be returned is updated, not the ctx.

With this, update, WIPE is removed, as we would always be passing
back the latest mtime, and hence cache times will be relevant.

TODO-handle rename WIPE calls

Change-Id: I8e4c738cd830f3fafeef789c9181f9c242ac96a2
BUG: 857791
Signed-off-by: shishir gowda <sgowda@redhat.com>
Reviewed-on: http://review.gluster.org/3737
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2012-10-16 14:27:15 -07:00
Jeff Darcy
58e6296fa2 build: split CPPFLAGS from CFLAGS
Automake provides a separate variable for preprocessor flags
(*_CPPFLAGS). They are already uses in a few places, so make it
consistent and use it everywhere. Note that cflags obtained from
pkg-config often are cppflags, which is why LIBXML2_CFLAGS moves with
into AM_CPPFLAGS, for example.

Change-Id: I15feed1d18b2ca497371271c4b5876d5ec6289dd
BUG: 862082
Original-author: Jan Engelhardt <jengelh@inai.de>
Signed-off-by: Jan Engelhardt <jengelh@inai.de>
Signed-off-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-on: http://review.gluster.org/4029
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2012-10-03 12:26:45 -07:00
Jeff Darcy
1ecbb7ca68 build: remove useless explicit -fPIC -shared from
CFLAGS

libtool will automatically add "-fPIC" to the compiler command line as
needed, so there is no need to specify it separately.

"-shared" is normally a linker flag and has an odd effect when used with
libtool --mode=compile, namely that it inhibits production of static
objects. For that however, using AC_DISABLE_STATIC is a lot simpler.

Change-Id: Ic4cba0fad18ffd985cf07f8d6951a976ae59a48f
BUG: 862082
Original-author: Jan Engelhardt <jengelh@inai.de>
Signed-off-by: Jan Engelhardt <jengelh@inai.de>
Signed-off-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-on: http://review.gluster.org/4027
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2012-10-03 12:24:59 -07:00
Jeff Darcy
04371377f2 build: remove -nostartfiles flag
The "-nostartfiles" is a discouraged option and is documented to
potentially result in undesired behavior. Since I see no reason why it
should be in glusterfs, remove it.

Change-Id: I56f2b08874516ebad91447b2583ca2fb776bb7ab
BUG: 862082
Original-author: Jan Engelhardt <jengelh@inai.de>
Signed-off-by: Jan Engelhardt <jengelh@inai.de>
Signed-off-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-on: http://review.gluster.org/4018
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2012-10-02 13:11:43 -07:00
Jeff Darcy
9059a76c67 build: consolidate common compilation flags into one variable
Some -D flags are present in all files, so collect them.
This adds -D${GF_HOST_OS} to some compiler command lines,
but this should not be a problem.

Change-Id: I1aeb346143d4984c9cc4f2750c465ce09af1e6ca
BUG: 862082
Original-author: Jan Engelhardt <jengelh@inai.de>
Signed-off-by: Jan Engelhardt <jengelh@inai.de>
Signed-off-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-on: http://review.gluster.org/4013
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2012-10-01 16:44:50 -07:00
Varun Shastry
1ee65fe16f Fixed some general typing errors.
Eg: changed recieved to received

Change-Id: I360fcb99c97c8a0222e373fee20ea2fccfb938db
BUG: 860543
Signed-off-by: Varun Shastry <vshastry@redhat.com>
Reviewed-on: http://review.gluster.org/3998
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2012-09-27 11:20:20 -07:00
Anand Avati
4f87fd0ae2 dht: improve dht_fix_layout_of_directory for better re-assignment
Jeff Darcy wrote:
> AFAICT, the fix-layout code doesn't do the same rotation that the
> new-directory code does. Therefore, the new bricks always claim
> completely predictable hash ranges for every directory, leading to
> either a 0-1-2-3 pattern or a 1-0-2-3 pattern.  In other words, a
> file whose hash falls into the second quarter of the range will always
> be assigned to brick 2, and a file whose hash falls into the fourth
> quarter will always be assigned to brick 3.  The rest will be split
> according to the original pattern.  Put still another way, instead of
> same-named files in different directories being spread across N bricks,
> they might be spread across only two bricks (bad) or totally
> concentrated on one brick (worse) regardless of N.

The current dht_fix_layout_of_directory() code, in an attempt to
maximize overlap of new layout with existing layout (to minimize
movement of data) fails to do a good job of randomizing new assignment
even when it could do a better job. In an example where we expand
from 2 nodes to 4 nodes, the current possibilities are limited in the
following way -

(theoretical hash range: 00 - 99)

OLD 1
-----
server1: 00 - 49
server2: 50 - 99

NEW 1
-----
server1: 00 - 24
server2: 50 - 74
server3: 25 - 49
server4: 75 - 99

OLD 2
-----
server1: 50 - 99
server2: 00 - 49

NEW 2
------
server1: 50 - 74
server2: 00 - 24
server3: 25 - 49
server4: 75 - 99

The above shows that when add-brick from 2 bricks to 4 bricks, server3
and server4 always get the _same_ hash range no matter what the original
hash range assignment was.

The fix in this patch is first do the standard new directory assignment
to a directory (with rotation etc.) and then do the reassignment to
maximize overlap. This way newly added servers still get random ranges
and existing servers have a probability of getting either of the quarters
which were part of its half previously. The same principles hold for
all add-brick from M to M+N.

Change-Id: I0cbbf3bfa334645728072d66aaaa80120d0b295f
BUG: 853258
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/3883
Tested-by: Gluster Build System <jenkins@build.gluster.com>
2012-09-12 14:29:51 -07:00
Amar Tumballi
c78919ce37 cluster/dht: handle percent option for 'min-free-disk'
* with the init option cleanups, setting of 'conf->disk_unit'
  was reset, which made it not set the '%' in the option.

* bring a global check, which makes the option assume its
  percent, as long as value is < 100.

Change-Id: I00bd1395a309cdc596a2b2b80304c6d98696a24a
Signed-off-by: Amar Tumballi <amarts@redhat.com>
BUG: 852889
Reviewed-on: http://review.gluster.org/3918
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2012-09-07 16:48:51 -07:00
Amar Tumballi
fb6c8f8b4e cluster/distribute: remove gf_log() from statedump functions
Change-Id: I83cccab6819d6a74e96c2717ca539fa1568cac89
Signed-off-by: Amar Tumballi <amarts@redhat.com>
BUG: 843822
Reviewed-on: http://review.gluster.org/3912
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2012-09-06 13:22:30 -07:00
Amar Tumballi
d6c99b6134 libglusterfs/dict: make 'dict_t' a opaque object
* ie, don't dereference dict_t pointer, instead use APIs everywhere
* other than dict_t only 'data_t' should be the valid export from dict.h

* added 'dict_foreach_fnmatch()' API
* changed dict_lookup() to use data_t, instead of data_pair_t

Change-Id: I400bb0dd55519a7c5d2a107e67c8e7a7207228dc
Signed-off-by: Amar Tumballi <amarts@redhat.com>
BUG: 850917
Reviewed-on: http://review.gluster.org/3829
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2012-09-06 00:34:15 -07:00
shishir gowda
e442b07f1d dht/rebalance: set the correct ownership on the dst file.
Currently, the dst file created has root:root ownership, till
migration is completed. During this phase, open fails on the dst
file if uid/gid is non-root.
Setting the dst_file to the correct ownership fixes the issue

Change-Id: Icfec89eb10dc866cdee38dab17695fe21174ef99
BUG: 852361
Signed-off-by: shishir gowda <sgowda@redhat.com>
Reviewed-on: http://review.gluster.org/3861
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2012-08-28 23:15:59 -07:00
Amar Tumballi
2f2e3bfb5e syncop: handle 'dataonly' flag in syncop_fsync()
* and also in syncop_readv(), don't look at _cbk args if op_ret
  is < 0.

Change-Id: I3ab2982bc6d186e75b6adb74c8981e4ff7058bbe
Signed-off-by: Amar Tumballi <amarts@redhat.com>
BUG: 839950
Reviewed-on: http://review.gluster.org/3828
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2012-08-20 00:16:01 -07:00
Jim Meyering
b757819289 cluster/dht: don't leak upon GF_REALLOC failure
Change-Id: I7dfabcc2981df5c5a1e1a54c3135400a60626cd1
BUG: 846755
Signed-off-by: Jim Meyering <meyering@redhat.com>
Reviewed-on: http://review.gluster.com/3798
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2012-08-19 10:14:31 -07:00
shishir gowda
e404e9b81f cluster/dht: Optimize readdirp calls in DHT
Bring in option which is supported by posix xlator
to filter out directory's entries from being returned.
DHT would now request non-first subvols to filter out
directory entries.

dht xlator-option readdir-optimize will enable this
optimization

Change-Id: I35224bc81c9657f54f952efac02790276c35ded5
BUG: 838199
Signed-off-by: shishir gowda <sgowda@redhat.com>
Reviewed-on: http://review.gluster.com/3772
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2012-08-13 23:24:04 -07:00
shishir gowda
7cce8c843e cluster/distribute: Suppress user xattr mismatch log message
Changing the log-level to DEBUG.
Xattr mismatch can occur when parallel setxattr's race, or when
one of the bricks was down. A subsequent setxattr will fix the
condition when all the subvols are up. In this case, the 'user.swift'
xattr used by ufo was out of sync, but did not cause any other error.

Change-Id: I6fdff78869b8ff72c305bbe122033e6c1d9d3cff
BUG: 838197
Signed-off-by: shishir gowda <sgowda@redhat.com>
Reviewed-on: http://review.gluster.com/3722
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Reviewed-by: Mohammed Junaid <junaid@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2012-07-25 08:20:35 -07:00
Anand Avati
a4e11fd67e glusterfs_ctx_t: un-globalize the filesystem context
So far there has been a global glusterfs_ctx_t object which
represents the running instance of the filesystem (client or server).
It contains the various graphs, connection to the management daemon
over which new graphs are obtained, calls stacks issued on this
filesystem, and a bunch of such things.

With the introduction of libgfapi, it is no more true that there will
be only one filesystem context in a process. Applications can
be written to use libgfapi and obtain serveral instances of different
filesystems/volumes in the same process.

This involves messy untangling of assumptions inside libglusterfs that
there would only be one global glusterfs_ctx_t and offload that
assumption to glusterfsd/ and cli/ (where it is true).

Change-Id: Ifd7d1259428c26076140a5764a2dc7361694139c
BUG: 839950
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.com/3678
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
2012-07-17 23:04:39 -07:00
Jim Meyering
7820b2c1f8 remove useless if-before-free (and free-like) functions
See comments in http://bugzilla.redhat.com/839925 for
the code to perform this change.

Signed-off-by: Jim Meyering <meyering@redhat.com>
BUG: 839925
Change-Id: I10e4ecff16c3749fe17c2831c516737e08a3205a
Reviewed-on: http://review.gluster.com/3661
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2012-07-13 14:03:42 -07:00
shishir gowda
5a5148f07a cluster/dht: Remove dht dependency on glusterfsd-mgmt
glusterfs_ctx->notify can be used by any xlator to talk to
glusterfsd-mgmt.
Note- This is for any rpc communication initiated by the xlator,
      and not from glusterd.

Change-Id: Ic0e4af106fe1e98d797ca621facda8839b87598a
BUG: 835757
Signed-off-by: shishir gowda <sgowda@redhat.com>
Reviewed-on: http://review.gluster.com/3618
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2012-06-29 17:25:46 -07:00
Amar Tumballi
6eee473eba core: coverity fixes (mostly resource leak fixes)
currently working on obvious resource leak reports in coverity

Change-Id: I261f4c578987b16da399ab5a504ad0fda0b176b1
Signed-off-by: Amar Tumballi <amarts@redhat.com>
BUG: 789278
Reviewed-on: http://review.gluster.com/3265
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2012-06-05 10:45:10 -07:00
Raghavendra Bhat
4a92900e5f cluster/dht: set conf->defrag to NULL after freeing the defrag structure
Also no need to free the xlator object after rebalance is over, as the process
is about to be killed.

Change-Id: I6973e43c0353b5de61c0b39e52a22c618be361f4
BUG: 826584
Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com>
Reviewed-on: http://review.gluster.com/3495
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2012-05-31 17:51:27 -07:00
Amar Tumballi
833a222544 cluster/distribute: set the inode layout in readdirp_cbk() for files
with this, inode-linking it in readdirp_cbk will be neater.

Change-Id: Ie2cd646438f851e1755e9b6a3fc9898059bee359
Signed-off-by: Amar Tumballi <amar@gluster.com>
BUG: 816140
Reviewed-on: http://review.gluster.com/2717
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2012-05-31 17:48:12 -07:00
Jeff Darcy
e066a5fea7 distribute: support user-specified layouts.
The new type is DHT_HASH_TYPE_DM_USER=1 (on disk in network byte order) and
we treat it the same as DHT_HASH_TYPE_DM except that we don't stomp on it
during rebalance.

Change-Id: I893571a9b89577acdea2fe868915b18d3663fd77
BUG: 807312
Signed-off-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-on: http://review.gluster.com/3004
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2012-05-31 17:13:29 -07:00
shishir gowda
9f81eb0b24 glusterd/rebalance: Display run-time of rebalance process
Added run-time value field to cli output of rebalance/remove-brick.

A new cluster/distribute boolean option rebalance-stats when set to
ON, time taken for migration of each file is logged.

With rebalance-stats OFF (default), rebalance logs will only have
entries showing time spent in each directory.

Change-Id: I02a8918621120068cd71ffaf2999d30b3a2d10a2
BUG: 821987
Signed-off-by: shishir gowda <shishirng@gluster.com>
Reviewed-on: http://review.gluster.com/3303
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2012-05-24 00:03:36 -07:00
Amar Tumballi
bb8a0664ef distribute: use global synctask 'env' instead of local
creating a local synctask_env can lead to creating of many more
syncop threads than required. The current syncop logic can handle
the scale-up/scale-down of threads depending on the load. Hence,
its neater to use global synctask env.

Change-Id: Id46f963a0190c0154513317ae03323db155ac15a
Signed-off-by: Amar Tumballi <amarts@redhat.com>
BUG: 823774
Reviewed-on: http://review.gluster.com/3412
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2012-05-23 00:43:19 -07:00
shishir gowda
4a4477ee29 cluster/dht: Handle ENOENT failure in dht_rmdir_opendir_cbk
We should not treat ENOENT as a failure in rmdir.

Change-Id: I1570d2be2bbafe7fc61ca39b8f7f68ee60d2c707
BUG: 806761
Signed-off-by: shishir gowda <shishirng@gluster.com>
Reviewed-on: http://review.gluster.com/3327
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2012-05-14 17:25:12 -07:00
Kaleb KEITHLEY
7319017fae license: dual license under GPLV2 and LGPLV3+
Note that the license was not changed in any of the following:
	.../argp-standalone/...
        .../booster/...
        .../cli/...
        .../contrib/...
        .../extras/...
        .../glusterfsd/...
        .../glusterfs-hadoop/...
        .../mod_clusterfs/...
        .../scheduler/...
        .../swift/...

The license was not changed in any of the non-building xlators. The
license was not changed in any of the xlators that seemed — to me — to
be clearly server-side only, e.g. protocol/server

Note too that copyright was changed along with the license; I did
not change the copyright in files where the license did not change.

If you find any errors or ommissions please don't hesitate to let me know.

The complete list of files with the license change is:
	libglusterfs/src/byte-order.h
	libglusterfs/src/call-stub.c
	libglusterfs/src/call-stub.h
	libglusterfs/src/checksum.c
	libglusterfs/src/checksum.h
	libglusterfs/src/circ-buff.c
	libglusterfs/src/circ-buff.h
	libglusterfs/src/common-utils.c
	libglusterfs/src/common-utils.h
	libglusterfs/src/compat-errno.c
	libglusterfs/src/compat-errno.h
	libglusterfs/src/compat.c
	libglusterfs/src/compat.h
	libglusterfs/src/daemon.c
	libglusterfs/src/daemon.h
	libglusterfs/src/defaults.c
	libglusterfs/src/defaults.h
	libglusterfs/src/dict.c
	libglusterfs/src/dict.h
	libglusterfs/src/event-history.c
	libglusterfs/src/event-history.h
	libglusterfs/src/event.c
	libglusterfs/src/event.h
	libglusterfs/src/fd-lk.c
	libglusterfs/src/fd-lk.h
	libglusterfs/src/fd.c
	libglusterfs/src/fd.h
	libglusterfs/src/gf-dirent.c
	libglusterfs/src/gf-dirent.h
	libglusterfs/src/globals.c
	libglusterfs/src/globals.h
	libglusterfs/src/glusterfs.h
	libglusterfs/src/graph-print.c
	libglusterfs/src/graph-utils.h
	libglusterfs/src/graph.c
	libglusterfs/src/hashfn.c
	libglusterfs/src/hashfn.h
	libglusterfs/src/iatt.h
	libglusterfs/src/inode.c
	libglusterfs/src/inode.h
	libglusterfs/src/iobuf.c
	libglusterfs/src/iobuf.h
	libglusterfs/src/latency.c
	libglusterfs/src/latency.h
	libglusterfs/src/list.h
	libglusterfs/src/lkowner.h
	libglusterfs/src/locking.h
	libglusterfs/src/logging.c
	libglusterfs/src/logging.h
	libglusterfs/src/mem-pool.c
	libglusterfs/src/mem-pool.h
	libglusterfs/src/mem-types.h
	libglusterfs/src/options.c
	libglusterfs/src/options.h
	libglusterfs/src/rbthash.c
	libglusterfs/src/rbthash.h
	libglusterfs/src/run.c
	libglusterfs/src/run.h
	libglusterfs/src/scheduler.c
	libglusterfs/src/scheduler.h
	libglusterfs/src/stack.c
	libglusterfs/src/stack.h
	libglusterfs/src/statedump.c
	libglusterfs/src/statedump.h
	libglusterfs/src/syncop.c
	libglusterfs/src/syncop.h
	libglusterfs/src/syscall.c
	libglusterfs/src/syscall.h
	libglusterfs/src/timer.c
	libglusterfs/src/timer.h
	libglusterfs/src/trie.c
	libglusterfs/src/trie.h
	libglusterfs/src/xlator.c
	libglusterfs/src/xlator.h
	libglusterfsclient/src/libglusterfsclient-dentry.c
	libglusterfsclient/src/libglusterfsclient-internals.h
	libglusterfsclient/src/libglusterfsclient.c
	libglusterfsclient/src/libglusterfsclient.h
	rpc/rpc-lib/src/auth-glusterfs.c
	rpc/rpc-lib/src/auth-null.c
	rpc/rpc-lib/src/auth-unix.c
	rpc/rpc-lib/src/protocol-common.h
	rpc/rpc-lib/src/rpc-clnt.c
	rpc/rpc-lib/src/rpc-clnt.h
	rpc/rpc-lib/src/rpc-transport.c
	rpc/rpc-lib/src/rpc-transport.h
	rpc/rpc-lib/src/rpcsvc-auth.c
	rpc/rpc-lib/src/rpcsvc-common.h
	rpc/rpc-lib/src/rpcsvc.c
	rpc/rpc-lib/src/rpcsvc.h
	rpc/rpc-lib/src/xdr-common.h
	rpc/rpc-lib/src/xdr-rpc.c
	rpc/rpc-lib/src/xdr-rpc.h
	rpc/rpc-lib/src/xdr-rpcclnt.c
	rpc/rpc-lib/src/xdr-rpcclnt.h
	rpc/rpc-transport/rdma/src/name.c
	rpc/rpc-transport/rdma/src/name.h
	rpc/rpc-transport/rdma/src/rdma.c
	rpc/rpc-transport/rdma/src/rdma.h
	rpc/rpc-transport/socket/src/name.c
	rpc/rpc-transport/socket/src/name.h
	rpc/rpc-transport/socket/src/socket.c
	rpc/rpc-transport/socket/src/socket.h
	xlators/cluster/afr/src/afr-common.c
	xlators/cluster/afr/src/afr-dir-read.c
	xlators/cluster/afr/src/afr-dir-read.h
	xlators/cluster/afr/src/afr-dir-write.c
	xlators/cluster/afr/src/afr-dir-write.h
	xlators/cluster/afr/src/afr-inode-read.c
	xlators/cluster/afr/src/afr-inode-read.h
	xlators/cluster/afr/src/afr-inode-write.c
	xlators/cluster/afr/src/afr-inode-write.h
	xlators/cluster/afr/src/afr-lk-common.c
	xlators/cluster/afr/src/afr-mem-types.h
	xlators/cluster/afr/src/afr-open.c
	xlators/cluster/afr/src/afr-self-heal-algorithm.c
	xlators/cluster/afr/src/afr-self-heal-algorithm.h
	xlators/cluster/afr/src/afr-self-heal-common.c
	xlators/cluster/afr/src/afr-self-heal-common.h
	xlators/cluster/afr/src/afr-self-heal-data.c
	xlators/cluster/afr/src/afr-self-heal-entry.c
	xlators/cluster/afr/src/afr-self-heal-metadata.c
	xlators/cluster/afr/src/afr-self-heal.h
	xlators/cluster/afr/src/afr-self-heald.c
	xlators/cluster/afr/src/afr-self-heald.h
	xlators/cluster/afr/src/afr-transaction.c
	xlators/cluster/afr/src/afr-transaction.h
	xlators/cluster/afr/src/afr.c
	xlators/cluster/afr/src/afr.h
	xlators/cluster/afr/src/pump.c
	xlators/cluster/afr/src/pump.h
	xlators/cluster/dht/src/dht-common.c
	xlators/cluster/dht/src/dht-common.h
	xlators/cluster/dht/src/dht-diskusage.c
	xlators/cluster/dht/src/dht-hashfn.c
	xlators/cluster/dht/src/dht-helper.c
	xlators/cluster/dht/src/dht-inode-read.c
	xlators/cluster/dht/src/dht-inode-write.c
	xlators/cluster/dht/src/dht-layout.c
	xlators/cluster/dht/src/dht-linkfile.c
	xlators/cluster/dht/src/dht-mem-types.h
	xlators/cluster/dht/src/dht-rebalance.c
	xlators/cluster/dht/src/dht-rename.c
	xlators/cluster/dht/src/dht-selfheal.c
	xlators/cluster/dht/src/dht.c
	xlators/cluster/dht/src/nufa.c
	xlators/cluster/dht/src/switch.c
	xlators/cluster/stripe/src/stripe-helpers.c
	xlators/cluster/stripe/src/stripe-mem-types.h
	xlators/cluster/stripe/src/stripe.c
	xlators/cluster/stripe/src/stripe.h
	xlators/features/index/src/index-mem-types.h ¹
	xlators/features/index/src/index.c ¹
	xlators/features/index/src/index.h ¹
	xlators/performance/io-cache/src/io-cache.c
	xlators/performance/io-cache/src/io-cache.h
	xlators/performance/io-cache/src/ioc-inode.c
	xlators/performance/io-cache/src/ioc-mem-types.h
	xlators/performance/io-cache/src/page.c
	xlators/performance/io-threads/src/io-threads.c
	xlators/performance/io-threads/src/io-threads.h
	xlators/performance/io-threads/src/iot-mem-types.h
	xlators/performance/md-cache/src/md-cache-mem-types.h
	xlators/performance/md-cache/src/md-cache.c
	xlators/performance/quick-read/src/quick-read-mem-types.h
	xlators/performance/quick-read/src/quick-read.c
	xlators/performance/quick-read/src/quick-read.h
	xlators/performance/read-ahead/src/page.c
	xlators/performance/read-ahead/src/read-ahead-mem-types.h
	xlators/performance/read-ahead/src/read-ahead.c
	xlators/performance/read-ahead/src/read-ahead.h
	xlators/performance/symlink-cache/src/symlink-cache.c
	xlators/performance/write-behind/src/write-behind-mem-types.h
	xlators/performance/write-behind/src/write-behind.c
	xlators/protocol/auth/addr/src/addr.c ¹
	xlators/protocol/auth/login/src/login.c ¹
	xlators/protocol/client/src/client-callback.c
	xlators/protocol/client/src/client-handshake.c
	xlators/protocol/client/src/client-helpers.c
	xlators/protocol/client/src/client-lk.c
	xlators/protocol/client/src/client-mem-types.h
	xlators/protocol/client/src/client.c
	xlators/protocol/client/src/client.h
	xlators/protocol/client/src/client3_1-fops.c

¹ Copyright only, license reverted to original

Change-Id: If560e826c61b6b26f8b9af7bed6e4bcbaeba31a8
BUG: 820551
Signed-off-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Reviewed-on: http://review.gluster.com/3304
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vijay@gluster.com>
2012-05-10 22:06:16 -07:00
shishir gowda
8af87a7ff4 cluster/dht: rmdir should succeed last on the hashed subvol
mkdir and rename both first happen on the hashed subvol.
Rmdir should succeed, only if all non hashed subvol succeed,
and finally removal from hashed_subvol succeeds.

if hashed_subvol is unavailable, then rmdir should be sent
on all subvols

Change-Id: I1bf5d5fc25393d0e2fde13ae403cf867a01f5ec7
BUG: 773187
Signed-off-by: shishir gowda <shishirng@gluster.com>
Reviewed-on: http://review.gluster.com/3202
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2012-05-08 18:32:56 -07:00
shishir gowda
8be3e656aa glusterd/rebalance: Remove state dependency on rpc event
Status is updated by event_notify handshake by the rebalance process.
If there is disconnect, before rebalance process, only then the
state is moved to failure.

Change-Id: I334055397c3d38688dbef653826bc07ac47a1911
BUG: 813167
Signed-off-by: shishir gowda <shishirng@gluster.com>
Reviewed-on: http://review.gluster.com/3165
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
2012-04-25 04:06:11 -07:00
shishir gowda
8f2eba00fd rebalance: handshake_event_notify to make fsd talk to glusterd
Event_notify can be used by others to communicate with glusterd.
A cbk event is also added for future use.

req has a op, and dict.
rsp has op_ret, op_errno, and dict.

With this, rebalance process can update the status before
exiting.

Signed-off-by: shishir gowda <shishirng@gluster.com>
Change-Id: If5c0ec00514eb3a109a790b2ea273317611e4562
BUG: 807126
Reviewed-on: http://review.gluster.com/3013
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Reviewed-by: Vijay Bellur <vijay@gluster.com>
2012-04-25 04:05:35 -07:00
shylesh
19c2d6b3d6 dht: Log message fix gfid's to gfids
Change-Id: I9ad67b6ded05992d1b56ca843f685c4fe2b15b71
BUG: 815186
Signed-off-by: shylesh <shmohan@redhat.com>
Reviewed-on: http://review.gluster.com/3217
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
2012-04-24 09:41:30 -07:00
Amar Tumballi
29f2de478c core: coverity issues fixed
this is not a complete set of issues getting fixed. Will
address other issues in another patch.

Change-Id: Ib01c7b11b205078cc4d0b3f11610751e32d14b69
Signed-off-by: Amar Tumballi <amarts@redhat.com>
BUG: 789278
Reviewed-on: http://review.gluster.com/3145
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
2012-04-23 14:52:57 -07:00
Raghavendra G
fc024df2b6 cluster/distribute: handle dht_layout_normalize errors differently in
dht_lookup_dir_cbk and dht_discover_complete.

return value from dht_layout_normalize is interpreted as follows:

ret > 0 - number of subvolumes returned ENOENT errors.
ret = 0 - no anomalies found.
ret < 0 - there are holes/overlaps.

We need to handle errors differently in dht_discover_cbk because,
if a subvolume is newly added, the directory will not be present on it
and we cannot create a directory there since a nameless lookup has
resulted execution in this code-path. Hence if directory is guaranteed
to be present in atleast one subvolume and there are no holes/overlaps
in the layout, lookup can be treated as successful.

there is no change in behaviour dht_lookup_dir_cbk caused by this
patch.

Change-Id: I9b0b510f1c7de187be95a47da6c9ec57b5e38d1d
BUG: 802233
Signed-off-by: Raghavendra G <raghavendra@gluster.com>
Reviewed-on: http://review.gluster.com/3069
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2012-04-23 10:41:03 -07:00
shishir gowda
678efee048 cluster/dht: Remove un-used pathinfo code for link files
Currently, we do not return LINK: attributes for a file
in pathinfo requests.

Change-Id: If8667bd6af06de3d11c37dd0a09726dfe1d2e330
BUG: 795289
Signed-off-by: shishir gowda <shishirng@gluster.com>
Reviewed-on: http://review.gluster.com/3175
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
2012-04-18 23:42:30 -07:00
shishir gowda
c0eec503b7 glusterd/rebalance: Start process with xlator option client-pid -3
This would prevent updation of xtime by the marker.
Set frame->root->pid while creating sync-task. Removing the command
option, as it is in-effective

Change-Id: Ifaa587df67a72ec0571c7d0043de8048e2be12a8
BUG: 812287
Signed-off-by: shishir gowda <shishirng@gluster.com>
Reviewed-on: http://review.gluster.com/3180
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Reviewed-by: Vijay Bellur <vijay@gluster.com>
2012-04-18 09:30:41 -07:00
shishir gowda
c20f501d06 cluster/dht: Handle failures in getxattr_cbk
Change-Id: Ifbce2d69f0fdd0c45cba92b6fc2dffe13c60de28
BUG: 810106
Signed-off-by: shishir gowda <shishirng@gluster.com>
Reviewed-on: http://review.gluster.com/3156
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
2012-04-17 06:26:42 -07:00
shishir gowda
369873fbea cluster/dht: Revert to calling ftruncate
Additionally, change lookup failure error to DEBUG level.

Change-Id: I6eaf8e92a9cd34eba125a1e0e183bc3b6dc71945
BUG: 812769
Signed-off-by: shishir gowda <shishirng@gluster.com>
Reviewed-on: http://review.gluster.com/3155
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
2012-04-16 11:19:18 -07:00
shishir gowda
b19a7fee17 dht/rebalance: Display migration failure count
Change-Id: Ib2f8be2c98eae1b4df5d2090c1f1eec8ab264a35
BUG: 811923
Signed-off-by: shishir gowda <shishirng@gluster.com>
Reviewed-on: http://review.gluster.com/3135
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Reviewed-by: Vijay Bellur <vijay@gluster.com>
2012-04-13 00:52:09 -07:00
shishir gowda
d80846bee0 dht/rebalance: Handle ASSERT_ON_CHILD_DOWN gracefully
In rebalance process, instead of terminating, send a stop
event. The migration in question will either complete or
be handled as error.
Also, handle few syncop calls as errors, instead of just
logging them.

Change-Id: If6ed54474cb0f1fe0e28a5765e6d90966740dfde
BUG: 811444
Signed-off-by: shishir gowda <shishirng@gluster.com>
Reviewed-on: http://review.gluster.com/3120
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Reviewed-by: Vijay Bellur <vijay@gluster.com>
2012-04-13 00:42:25 -07:00