Commit Graph

196 Commits

Author SHA1 Message Date
Pranith Kumar K
273a42a421 cluster/afr: Club missing entry, missing gfid self-heals
Problem:
gfid-self-heal always assigns the gfid(GFID-1) it gets from lookup.
Between the time of lookup to triggering the gfid-self-heal the
entry could have changed. Now lets say there is a case where
one of the files of the replica subolumes already has a gfid
(GFID-2) and the other does not. In that case healing should
happen with GFID-2 instead of GFID-1.

Fix:
Missing-entry-self-heal already handles all these cases. So removed
separate handling of gfid-self-heal.

Change-Id: Ie96261e9036c8f3cb4cad89347f9bf7b681cdc1a
BUG: 767585
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/2670
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2013-05-07 08:35:47 -07:00
Krutika Dhananjay
6556b5cf40 glusterd: delete "volume-name" from dict before processing the next option
Change-Id: Ib78963c1f43a66dab50b443742979c7c4e4cbc23
BUG: 958790
Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com>
Reviewed-on: http://review.gluster.org/4940
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2013-05-02 10:58:20 -07:00
Vijay Bellur
7967e3e5cf cluster/afr: Avoid self-healing extended attribute used by SELinux.
Since removexattr() fails to remove "security.selinux" in a system 
where SELinux is enforcing, xattr self-healing fails. 
As a consequence of this, user extended attributes are not being healed.
Added a check in afr to prune SELinux xattr from the dictionary 
used for removing xattrs from the sink.

Minor changes in tests and md-cache as well.

Signed-off-by: Vijay Bellur <vbellur@redhat.com>
Change-Id: I854bfc0098dde812ce2afe64b125ee40c04bdeb1
BUG: 957877
Reviewed-on: http://review.gluster.org/4905
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2013-04-30 20:11:37 -07:00
Krishnan Parthasarathi
f75be775a9 tests: Modified test to use remove-brick instead of 'start' variant
remove-brick start doesn't remove the brick from the volume immediately.
It would wait until migration of data to other bricks are complete. Even
when there is no data to be migrated, one can expect a finite delay from
the time of remove-brick start command's exit and removal of brick(s).
This may cause subsequent checks on brick count to fail in a
non-deterministic manner.

Also, renamed the test file name to reflect bug-id corresponding to
community release.

Change-Id: Ic43f011e251640decb68e46f4a10e0824ade0ac9
BUG: 878004
Signed-off-by: Krishnan Parthasarathi <kparthas@redhat.com>
Reviewed-on: http://review.gluster.org/4885
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
2013-04-30 00:32:07 -07:00
Niels de Vos
e6b6d1235d cli: add a command 'gluster pool list [--xml]'
* unlike 'gluster peer status', which lists only info about peers,
  this command lists localhost also in the list, so the sorted
  output from all the nodes should match.

* made the output script friendly by keeping it one output per line.

Change-Id: I853656753b35c617debbcceecbb71c8d6dd3c334
BUG: 764638
Original-review: http://review.gluster.org/4221
Original-author: Amar Tumballi <amarts@redhat.com>
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: http://review.gluster.org/4862
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2013-04-26 04:28:06 -07:00
Kaleb S. KEITHLEY
5e0116109e build: sync glusterfs.spec.in with Fedora glusterfs.spec
BUG: 950083
Change-Id: I96aeb8fbe8b79bbc058ff9a45167d822abb576ed
Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
Reviewed-on: http://review.gluster.org/4876
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
2013-04-26 02:38:07 -07:00
Rajesh Amaravathi
054c1d7eb3 nfs: change default nfs port to 2049
This change makes it possible to mount glusterfs volumes
without specifying vers=3 option.

Change-Id: If5a974e2bdfd2adbeac3d82af774310cdf30f988
BUG: 832939
Signed-off-by: Rajesh Amaravathi <rajesh@redhat.com>
Reviewed-on: http://review.gluster.org/4840
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
2013-04-24 02:06:02 -07:00
Krishnan Parthasarathi
1fc9b98a4e tests: Fix volume-status test script
* The functions now return the aggregated return value instead of just echoing it.
* Fixed the shd test loop to use the proper variable
* Added 'wait's for nfs and shd services to come online.

BUG: 764966
Change-Id: Iedbb5be0d5e50108ed4847d58625ccb093c22251
Original-author: Kaushal M <kaushal@redhat.com>
Signed-off-by: Krishnan Parthasarathi <kparthas@redhat.com>
Reviewed-on: http://review.gluster.org/4856
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
2013-04-19 01:55:36 -07:00
Krutika Dhananjay
9d74f78141 glusterd: validate performance.nfs.* option values during volume set stage
PROBLEM:

performance.nfs.* option values (which are of type boolean) are
not validated during the stage phase of 'volume set'.
The result - nfs graph generation fails during commit phase,
AFTER the option and its (invalid) value have been placed in
volinfo->dict.

CAUSE:

nfsperfxl_option_handler() - the function that validates the values of
performance.nfs.* options - never receives the (key,value) pair that
needs to be set, for validation during 'volume set' stage.

FIX:

In build_nfs_graph(), copy the (mod_)dict containing the (option,value)
parameters into set_dict before attempting to build the client graph
for the volume on which the operation is being performed.

Of course, an easier way out would be to simply do a 'volume reset' and
pretend nothing wrong happened!

Change-Id: I56b17d0239d58a9e0b7798933a3c8451e2675b69
BUG: 949930
Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com>
Reviewed-on: http://review.gluster.org/4814
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2013-04-18 06:53:29 -07:00
Krishnan Parthasarathi
dcebed550e glusterd: Avoided deadlock in single node cluster, glusterd restart
In a single node cluster, it is possible to deadlock on the "big
lock", while restarting bricks. In glusterd_restart_bricks, we perform a
glusterd_brick_connect, where we release the big lock in anticipation
that glusterd_brick_rpc_notify could run in the same C stack (and
deadlocking). So, in the restart code path, we could unlock before we
have performed a lock on the big lock.

To fix this, we need to take the big lock in the
glusterd_launch_synctask 'thread' as well.

Change-Id: I1abea1ca82b55c784b8a810a8194f254b32b1dcc
BUG: 948686
Signed-off-by: Krishnan Parthasarathi <kparthas@redhat.com>
Reviewed-on: http://review.gluster.org/4837
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
2013-04-16 10:42:51 -07:00
Krishnan Parthasarathi
f34343d375 glusterd: big lock - a coarse-grained locking to prevent races
There are primarily three lists that are part of glusterd process,
that are concurrently accessed. Namely, priv->volumes, priv->peers
and volinfo->bricks_list.

Big-lock approach
-----------------
WHAT IS IT?
Big lock is a coarse-grained lock which protects all three
lists, mentioned above, from racy access.

HOW DOES IT WORK?
At any given point in time, glusterd's thread(s) are in execution
_iff_ there is a preceding, inbound network event. Of course, the
sigwaiter thread and timer thread are exceptions.
A network event is an external trigger to glusterd, via the epoll
thread, in the form of POLLIN and POLLERR.
As long as we take the big-lock at all such entry points and yield
it when we are done, we are guaranteed that all the network events,
accessing the global lists, are serialised.

This amounts to holding the big lock at
- all the handlers of all the actors in glusterd. (POLLIN)
- all the cbks in glusterd. (POLLIN)
- rpc_notify (DISCONNECT event), if we access/modify
  one of the three lists. (POLLERR)

In the case of synctask'ized volume operations, we must remember that,
if we held the big lock for the entire duration of the handler,
we may block other non-synctask rpc actors from executing.
For eg, volume-start would block in PMAP SIGNIN, if done incorrectly.
To prevent this, we need to yield the big lock, when we yield the
synctask, and reacquire on waking up of the synctask.

Change-Id: Ib929f9905b55fb6c3fc27fefb497a26dba058e4f
BUG: 948686
Signed-off-by: Krishnan Parthasarathi <kparthas@redhat.com>
Reviewed-on: http://review.gluster.org/4784
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
2013-04-12 13:47:46 -07:00
Anand Avati
a216f5f446 tests: fix further issues with bug-874498.t
The failure of bug-874498.t seems to be a "bug" in glustershd.
The situation seems to be when both subvolumes of a replica are
"local" to glustershd, and in such cases glustershd is sensitive
to the order in which the subvols come up.

The core of the issue itself is that, without the patch (#4784),
self-heal daemon completes the processing of index and no entries
are left inside the xattrop index after a few seconds of volume
start force. However with the patch, the stale "backing file"
(against which index performs link()) is left. The likely reason
is that an "INDEX" based crawl is not happening against the subvol
when this patch is applied.

Before #4784 patch, the order in which subvols came up was :

  [2013-04-09 22:55:35.117679] I [client-handshake.c:1456:client_setvolume_cbk] 0-patchy-client-0: Connected to 10.3.129.13:49156, attached to remote volume '/d/backends/brick1'.
  ...
  [2013-04-09 22:55:35.118399] I [client-handshake.c:1456:client_setvolume_cbk] 0-patchy-client-1: Connected to 10.3.129.13:49157, attached to remote volume '/d/backends/brick2'.

However, with the patch, the order is reversed:

  [2013-04-09 22:53:34.945370] I [client-handshake.c:1456:client_setvolume_cbk] 0-patchy-client-1: Connected to 10.3.129.13:49153, attached to remote volume '/d/backends/brick2'.
  ...
  [2013-04-09 22:53:34.950966] I [client-handshake.c:1456:client_setvolume_cbk] 0-patchy-client-0: Connected to 10.3.129.13:49152, attached to remote volume '/d/backends/brick1'.

The index in brick2 has the list of files/gfid to heal. It appears
to be the case that when brick1 is the first subvol to be detected
as coming up, somehow an INDEX based crawl is clearing all the
index entries in brick2, but if brick2 comes up as the first subvol,
then the backing file is left stale.

Also, doing a "gluster volume heal full" seems to leave out stale
backing files too. As the crawl is performed on the namespace and
the backing file is never encountered there to get cleared out.

So the interim (possibly permanent) fix is to have the script issue
a regular self-heal command (and not a "full" one).

The failure of the script itself is non-critical. The data files are
all healed, and it is just the backing file which is left behind. The
stale backing file too gets cleared in the next index based healing,
either triggered manually or after 10mins.

Change-Id: I5deb79652ef449b7e88684311e804a8a2aa4725d
BUG: 874498
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/4798
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
2013-04-09 21:43:42 -07:00
Krutika Dhananjay
c9800f0986 glusterd: changes in 'volume create' behaviour
This patch incorporates all the changes suggested on the behaviour of
'volume create' command in http://review.gluster.org/#change,4214
(comment #14, to be precise).

Change-Id: Iaac524a59738b177415595b18aa8a136090d3d25
BUG: 948729
Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com>
Reviewed-on: http://review.gluster.org/4740
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2013-04-09 17:39:30 -07:00
Anand Avati
f364d542aa tests: fix dependency on sleep in bug-874498.t
With the introduction of http://review.gluster.org/4784, there are
delays which breaks bug-874498.t which wrongly depends on healing
to finish within 2 seconds.

Fix this by using 'EXPECT_WITHIN 60' instead of sleep 2.

Change-Id: I2716d156c977614c719665a5e1f159dabf2878b5
BUG: 874498
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/4796
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
2013-04-09 12:55:19 -07:00
shishir gowda
9c1d7cebbc cluster/distribute: Ignore non-participating subvols for layout checks
When subvols-per-directory is < available subvols, then there are layouts
which are not populated. This leads to incorrect identification of holes or
overlaps. We need to ignore layouts, which have err == 0, and start == stop.
In the current scenario (start == stop == 0).

Additionally, in layout-merge, treat missing xattrs as err = 0. In case of
missing layouts, anomalies will reset them.

For any other valid subvoles, err != 0 in case of layouts being zeroed out.

Also reverted back dht_selfheal_dir_xattr, which does layout calculation only
on subvols which have errors.

Change-Id: I9f57062722c9e8a26285e10675c31a78921115a1
BUG: 921408
Signed-off-by: shishir gowda <sgowda@redhat.com>
Reviewed-on: http://review.gluster.org/4668
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
2013-04-09 12:49:40 -07:00
Jeff Darcy
1ca50941d6 tests: fix spurious regression test failures
Change-Id: I752aeb8e25f43281d2f5cf33d0ff5aeae49687e7
BUG: 764966
Signed-off-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-on: http://review.gluster.org/4794
Reviewed-by: Anand Avati <avati@redhat.com>
Tested-by: Anand Avati <avati@redhat.com>
2013-04-08 19:19:54 -07:00
Vijay Bellur
108475decf cli: Address a double free with volume info.
Crash is observed when volume info is performed on a
non-exisiting volume name and the output format is xml.

Change-Id: I88aa5d9dc954b1352f5cc3b5b38742c832bc1bb8
BUG: 949298
Signed-off-by: Vijay Bellur <vbellur@redhat.com>
Reviewed-on: http://review.gluster.org/4785
Reviewed-by: Kaushal M <kaushal@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2013-04-08 14:22:28 -07:00
Vijay Bellur
e0fac0b68f glusterfsd: Cleanup temporary files from /tmp
For each gluster{d,fs,fsd} start, one or more temporary
file(s) created in /tmp were not being unlinked. This
patch cleans that up.

Modified a typo in an unrelated log message as well.

Change-Id: I3dec2a2ca40c7d6828eb238ec9cd08b6072cf0dd
BUG: 949327
Signed-off-by: Vijay Bellur <vbellur@redhat.com>
Reviewed-on: http://review.gluster.org/4786
Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com>
Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
2013-04-08 14:21:39 -07:00
Venky Shankar
24ee79345f storage/posix: introduce node-uuid-pathinfo
enabling this option has an effect on pathinfo xattr
request returning <node-uuid>:<path> instead of the
default - which is <hostname>:<path>.

Change-Id: Ice1b38abf8e5df1568bab6d79ec0d53dfa520332
BUG: 765380
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.org/4567
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2013-04-05 14:40:53 -07:00
Raghavendra Talur
fff8008c6c tests: Remove grep process entries from pidgrep
Problem:
We were picking process with lowest pid from ps|grep
result. However, lowest pid need not be oldest process
as recycling of PIDs can take place.

Solution:
Removed grep process entries from ps entries using 
grep -v grep.

Change-Id: I2b9687a05a34cf6358f773183770d69a3fb9eb10
BUG: 858488
Signed-off-by: Raghavendra Talur <rtalur@redhat.com>
Reviewed-on: http://review.gluster.org/4765
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2013-04-04 00:47:42 -07:00
Pranith Kumar K
b38d98e706 Tests: Change rebalance status verification
According to the comment at the following URL
https://bugzilla.redhat.com/show_bug.cgi?id=916226#c2
"success:" can come even before rebalance is completed.
Changed it to check for "completed" instead.

Change-Id: Ibe9d3b75493240f30261ac2a1280f32ef32886da
BUG: 916226
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/4614
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2013-04-02 19:05:36 -07:00
Rajesh Amaravathi
f3a9e390af rpc/nfs: cleanup legacy code of general options
Removing the code which handles "general" options.
Since it is no longer possible to set general options which
apply for all volumes by default, this was redundant.

This cleanup of general options code also solves a bug wherein
with nfs.addr-namelookup on, nfs.rpc-auth-reject wouldn't work
on ip addresses

Change-Id: Iba066e32f9a0255287c322ef85ad1d04b325d739
BUG: 921072
Signed-off-by: Rajesh Amaravathi <rajesh@redhat.com>
Reviewed-on: http://review.gluster.org/4691
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
2013-04-02 16:05:24 -07:00
Venky Shankar
af939370ad cluster/afr: sync xattrs removed on source to sink(s)
xattrs are first removed from sink followed by setting
source xattrs.

Change-Id: I181cb5b785b667bbfc6e40787a2183a8f45de06b
BUG: 906646
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.org/4656
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
2013-04-02 10:29:38 -07:00
Raghavendra Bhat
d5c3024f9f rpc: disable root-squash dynamically upon volume set command
Change-Id: I2ba9ca339ffbe07cb74833165a46a941225b623d
BUG: 927616
Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com>
Reviewed-on: http://review.gluster.org/4722
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2013-04-01 02:29:39 -07:00
Pranith Kumar K
ca6a3d1e39 cluster/afr: piggyback and fsync resume changes
1) pre_op_piggyback should always be decremented.
2) Move fsync resume to just after post_op.
3) fsync stub should be created from afr's local
   not from the final response.

Change-Id: I220bb532eb03bea584292f4dd2e816ad0c3e0cf7
BUG: 927146
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/4741
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2013-03-28 08:49:55 -07:00
Anand Avati
ca10fdc81a cluster/afr: ensure DATA operations are made durable before POST-OP
The changelogging scheme of AFR stores information about the state
of all replicas in all replicas (in the extended attribute of the
respective files on each server) in the form of 'pending counts'
of operations (effectively "dirty flags"). These xattrs are blindly
trusted while performing self-heal, and therefore utmost care has
to be taken while updating and maintaing them.

The most critical updation is the clearing of the pending counts
corresponding to the *other* server in the changelog of a given
server. Before clearing the pending count, we need durability
guarantee of the write which was performed on the other server.

To obtain such a guarantee, it may be necessary to explicitly
introduce an fsync() phase (if the file itself wasn't already
opened with O_SYNC).

This patch introduces the detection of unstable stable writes on
a file and issues explicit fsync() on the servers before performing
the POST-OP clearing of pending flags.

Change-Id: I2171b86a74ec91e40e5877eef0a4e7379578ecf7
BUG: 927146
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/4721
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
2013-03-27 22:15:53 -07:00
Krishnan Parthasarathi
87d03fa7f4 glusterd: Removed fd leaks in glusterfs_start utility function
PROBLEM:
    The FILE* associated with the pidfile was leaked if
    pmap_registry_search on the brickinfo' path failed.

FIX:
    Eliminates the use of the FILE* that was leaked. Uses
    glusterd_is_service_running utility function in place
    of the earlier attempt to check for the same.

Change-Id: I94082bd5a94b8a6340f8cc11726d3264e364efe6
BUG: 916549
Signed-off-by: Krishnan Parthasarathi <kparthas@redhat.com>
Reviewed-on: http://review.gluster.org/4596
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2013-03-25 14:50:53 -07:00
Krishnan Parthasarathi
7149682695 glusterfsd: Fixed fd leak due to use of tmpfile()
Change-Id: I3c2dc070ebe967100170e39f3545acacc6016d61
BUG: 924075
Signed-off-by: Krishnan Parthasarathi <kparthas@redhat.com>
Reviewed-on: http://review.gluster.org/4703
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
2013-03-22 13:34:29 -07:00
Jeff Darcy
76bc5d1b2d dht: make DHT xattr names configurable
This is necessary to support "DHT over DHT" configurations, so that the
upper and lower instances of DHT don't step all over each other.  Why
would we even consider such a thing?  Because it gives us the ability to
do data tiering and rack-aware placement, either by themselves or as
complements to other functionality such as erasure codes or
deduplication which save space but cost performance.  By setting up the
top-level DHT to place data into one of several lower-level DHT pools
based on policy instead of pure elastic hashing, we get better
performance for 90% of accesses and better storage efficiency for 90% of
data, all for relatively low effort.

Change-Id: I72e65c29edfc80babf39f7a2a00090f4588c4070
BUG: 924265
Signed-off-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-on: http://review.gluster.org/4694
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2013-03-21 15:04:31 -07:00
Pranith Kumar K
66d5d91465 performance/io-threads: Fix range-check for least-rate-limit
The issue could be fixed with .validate=GF_OPT_VALIDATE_MIN. But
adding max value is more robust.

Change-Id: Ia69c6f86855dbd34a26e20391e77bfa0f796a200
BUG: 923573
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/4698
Reviewed-by: Brian Foster <bfoster@redhat.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
2013-03-21 00:05:19 -07:00
Pranith Kumar K
35660032d6 cluster/afr: Preserve mtime in self-heal
Problem:
Data self-heal may choose sink iatt to set mtimes.
This happens because after syncing of data is done
self-heal does one more xattrops/fstat to determine
sources sinks to set the inode-ctx. Since this is done
after data syncing and erase of xattrs, old source and
old sink are now sources, but the mtimes of them differ.
Old code just takes the first source from the list and
update mtimes, which could be sink before the self-heal
started.

Fix:
Set mtime from 'sources before syncing'.

Change-Id: Id769e1b99aa4f041eaee775f64cbf2c57b799723
BUG: 918437
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/4658
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
2013-03-12 08:08:52 -07:00
Krutika Dhananjay
e125e2ae61 glusterd: Mark vol as deleted by renaming voldir before cleaning up the store
PROBLEM:

During 'volume delete', when glusterd fails to erase all information
about a volume from the backend store (for instance because rmdir()
failed on non-empty directories), not only does volume delete fail on
that node, but also subsequent attempts to restart glusterd fail
because the volume store is left in an inconsistent state.

FIX:

Rename the volume directory path to a new location
<working-dir>/trash/<volume-id>.deleted, and then go on to clean up its
contents. The volume is considered deleted once rename() succeeds,
irrespective of whether the cleanup succeeds or not.

Change-Id: Iaf18e1684f0b101808bd5e1cd53a5d55790541a8
BUG: 889630
Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com>
Reviewed-on: http://review.gluster.org/4639
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Reviewed-by: Kaushal M <kaushal@redhat.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
2013-03-11 14:07:19 -07:00
Varun Shastry
bc4350423a features/quota: regression tests on general functionality of quota
Change-Id: Ie8fd9d21f3a8644ce4d6f6cb887c101451cec10c
BUG: 919998
Signed-off-by: Varun Shastry <vshastry@redhat.com>
Reviewed-on: http://review.gluster.org/4657
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
2013-03-11 14:00:33 -07:00
Lalatendu Mohanty
b1755efdff tests:Adding remove-brick test to volume.t
Change-Id: Ibef0c151ef6518136cd5f4c39ec5e89f06ff4133
BUG: 764966
Signed-off-by: Lalatendu Mohanty <lmohanty@redhat.com>
Reviewed-on: http://review.gluster.org/4650
Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
2013-03-08 21:46:20 -08:00
shishir gowda
140e9756a5 cluster/distribute: Fix layout overlaps due to spread-count in selfheal path
We needed to zero out the layout range, before we re-calculate the range.
When spread-count is issued, we would end up with stale ranges in the layout.

Replaced dht_selfheal_dir_xattr with dht_fix_dir_xattr, which correctly resets
the un-used (after re-cal) layouts.

Change-Id: I1a900d15df07335f59356bd23182ccec34381ab2
BUG: 884455
Signed-off-by: shishir gowda <sgowda@redhat.com>
Reviewed-on: http://review.gluster.org/4647
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
2013-03-07 19:51:22 -08:00
Avra Sengupta
1ffe690f56 Modified validation parameters for owner-uid and owner-gid.
owner-uid and owner-gid will not receive negative values anymore.

Change-Id: I82741d3d01b29e448294b2ec093fb70d22a5c77e
BUG: 912297
Signed-off-by: Avra Sengupta <asengupt@redhat.com>
Reviewed-on: http://review.gluster.org/4581
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2013-03-01 14:52:32 -08:00
Pranith Kumar K
9dac72481b cluster/afr: Turn on eager-lock for fd DATA transactions
Problem:
With the present implementation, eager-lock is issued for
any fd fop. eager-lock is being transferred to metadata
transactions. But the lk-owner is set to local->fd address
only for DATA transactions, but for METADATA transactions
it is frame->root. Because of this unlock on the eager-lock fails
and rebalance hangs.

Fix:
Enable eager-lock for fd DATA transactions

Change-Id: If30df7486a0b2f5e4150d3259d1261f81473ce8a
BUG: 916226
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/4588
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2013-03-01 14:50:58 -08:00
Raghavendra G
d4269b0000 performance/write-behind: Add test case for fd being marked bad
after write failures.

Change-Id: I5a3c27be0ad77b7f2660f45977f9f1f8f34ae31c
BUG: 765473
Signed-off-by: Raghavendra G <raghavendra@gluster.com>
Reviewed-on: http://review.gluster.org/4560
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2013-02-28 15:21:59 -08:00
Avra Sengupta
549231dda9 glusterd: Added the validation function for subvols-per-directory
Change-Id: Ie2259023b9001311a2032792639c3093054f6750
BUG: 896431
Signed-off-by: Avra Sengupta <asengupt@redhat.com>
Reviewed-on: http://review.gluster.org/4552
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
2013-02-28 11:31:28 -08:00
Avra Sengupta
4579609e33 glusterd: Added validation function for stripe-block-size.
Change-Id: I050d01b01eac46550aa435da7d96a972e0393d35
BUG: 770655
Signed-off-by: Avra Sengupta <asengupt@redhat.com>
Reviewed-on: http://review.gluster.org/4561
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
2013-02-27 21:36:55 -08:00
Vijay Bellur
dc8d19148a nfs/server: Fix multiple crashes in acl handling code.
Change-Id: I9b39a485c8b98d9eabe6153487f4dfbd26f8af13
BUG: 915280
Signed-off-by: Vijay Bellur <vbellur@redhat.com>
Reviewed-on: http://review.gluster.org/4578
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2013-02-26 11:18:24 -08:00
Anand Avati
89ea458316 tests/cluster.rc: support for virtual multi-server glusterd tests
Since http://review.gluster.org/4556 glusterd is capable of running
many instances of itself on a single system. This patch exploits
that feature and enhances the regression test framework to expose
handy primitives so that test cases may be written to test glusterd
in a cluster.

Usage:

1. Include "$(dirname)/../cluster.rc" to get access to the extensions

2. Call launch_cluster $N where $N is the count of virtual servers

Calling launch_cluster, starts $N glusterds which bind to $N different
IPs and dynamically defines these primitives:

 - Variables $H1 .. $Hn assigned to hostnames of each "server".

 - Variables $CLI_1 .. $CLI_n assigned as commands to run CLI commands
   on the corresponding N'th server.

 - Variables $B1 .. $Bn assigned to the backend directories on each
   "server".

 - Function kill_glusterd, which accepts a parameter - index number of
   glusterd to be killed.

 - Variables $glusterd_1 .. $glusterd_n assigned to the command lines
   to restart the corresponding glusterd, if it was previously killed.

The current set of primitives and functions were implemented with the goal
of satisfying ./tests/bugs/bug-913555.t. The API will be made richer as
we add more cluster test cases

Change-Id: Ieb13ed9f4a72ac0321db0ca0844c7b294145bb32
BUG: 913555
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/4566
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
2013-02-26 09:07:13 -08:00
shishir gowda
164c9586ae cluster/distribute: Preserve file size during rebalance migration
If holes are encountered, then we do not write these to the dst,
which sometimes causes file size to be lesser than src. Data is not
corrupted, as when non-zero reads are received, we do write that data.

Calling a truncrate to give file size to prevent it from being
truncated to less than src in case the file end has holes.

Thanks to Brian Foster for providing the test case

Change-Id: I3cdd143b63ec8d797273d76189dff8b05eb9e551
BUG: 915554
Signed-off-by: shishir gowda <sgowda@redhat.com>
Reviewed-on: http://review.gluster.org/4574
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2013-02-25 23:01:28 -08:00
Avra Sengupta
3f1ef648b9 libglusterfs: Fixed valid_ipv4_address and valid_host_name
In valid_ipv4_address(), strtok_r was ignoring consecutive dots,
and was marking any address with consecutive dots as valid. Now
such addresses will be marked as invalid.

In valid_host_name(), any name ending with any special characters
was marked as valid. Now such hostnames will be marked as invalid.

Change-Id: If9b625d9040a14bde8c176fe36e337cb483e284a
BUG: 822830
Signed-off-by: Avra Sengupta <asengupt@redhat.com>
Reviewed-on: http://review.gluster.org/4579
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
2013-02-25 19:52:25 -08:00
Pranith Kumar K
15d4ae7da9 cluster/afr: Don't queue transactions during open-fd fix
Before Anonymous fds are available, afr had to queue up
transactions if the file is not opened on one of its
subvolumes. This happens until the attempt to open the
file either succeeds or fails. These attempts happen
until the file is successfully opened on the subvolume.
Now client xlator uses anonymous fds to perform the fops
if the fd used for the fop is not 'opened'.
Fops will be successful even when the file is not opened
so there is no need to queue up the transactions anymore in afr.
Open is attempted on the subvolume where it is not
opened independent of the fop.

Change-Id: Id1a4b4ebe6f89f9efe8f6a8247918b91247d0819
BUG: 913051
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/4568
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2013-02-22 21:03:03 -08:00
Avra Sengupta
8897d08e6a glusterd: Added validation function for quota-timeout.
Change-Id: I7f82f4217a41e0fe41272e6ef82925e1fe97fcd5
BUG: 765230
Signed-off-by: Avra Sengupta <asengupt@redhat.com>
Reviewed-on: http://review.gluster.org/4557
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
2013-02-22 12:13:10 -08:00
Raghavendra G
9f8a8df814 tests: move common funtion definitions to include.rc
Change-Id: I3526557d0a675692e94edb71b7d92c9202e6afa4
BUG: 765473
Signed-off-by: Raghavendra G <raghavendra@gluster.com>
Reviewed-on: http://review.gluster.org/4559
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2013-02-22 12:12:26 -08:00
Anand Avati
91052d8b56 tests/fileio.rc: library for file descriptor based IO in tests
There are situations in test scripts where we want to keep open file
descriptors while performing other commands. Bash has abilities
to manage file descriptors by numbers, but the syntax is a little
brain damaging.

This library provides wrappers around it to abstract away bash's
syntax and also provides a helper function to pick a free file
descriptor on the fly.

The APIs are pretty self explanatory.

Change-Id: I82f1d1957646dd6c468d9e85c90ec30c978c7ad6
BUG: 764966
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/4565
Reviewed-by: Amar Tumballi <amarts@redhat.com>
2013-02-21 23:26:57 -08:00
Jeff Darcy
1dbe9a05fe glusterd: allow multiple instances of glusterd on one machine
This is needed to support automated testing of cluster-communication
features such as probing and quorum.  In order to use this, you need to
do the following preparatory steps.

* Copy /var/lib/glusterd to another directory for each virtual host

* Ensure that each virtual host has a different UUID in its glusterd.info

Now you can start each copy of glusterd with the following xlator-options.

* management.transport.socket.bind-address=$ip_address

* management.working-directory=$unique_working_directory

You can use 127.x.y.z addresses for binding without needing to assign
them to interfaces explicitly.  Note that you must use addresses, not
names, because of some stuff in the socket code that's not worth fixing
just for this usage, but after that you can use names in /etc/hosts
instead.

At this point you can issue CLI commands to a specific glusterd using
the --remote-host option.  So far probe, volume create/start/stop,
mount, and basic I/O all seem to work as expected with multiple
instances.

Change-Id: I1beabb44cff8763d2774bc208b2ffcda27c1a550
BUG: 913555
Signed-off-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-on: http://review.gluster.org/4556
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2013-02-21 17:27:56 -08:00
Krutika Dhananjay
673287ae4d glusterd: use gf_strdup() in place of strdup()
Change-Id: Idee71019dbc6eeaa0a808d671b29d6f3038a1a89
BUG: 913487
Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com>
Reviewed-on: http://review.gluster.org/4562
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2013-02-21 06:26:42 -08:00