5815 Commits

Author SHA1 Message Date
Amar Tumballi
52ce8fc0a8 geo-rep: retry in case of ENOENT errors in entry creations
Change-Id: I8961633a7371c941a3feee44c949d5c934eca998
Original-Author: Venky Shankar <vshankar@redhat.com>
Signed-off-by: Amar Tumballi <amarts@redhat.com>
BUG: 847839
Reviewed-on: http://review.gluster.org/5933
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Tested-by: Venky Shankar <vshankar@redhat.com>
2013-09-17 03:42:19 -07:00
Vijay Bellur
58423e6f14 mount/fuse: Implement forget in cbks for fuse.
With the introduction of inode_ctx_set in fuse as part of  2991503d014,
forget cbk gets called for fuse xlator. Though nothing needs to be done
inf forget_cbk, excessive log messages of the following kind are observed:

[2013-09-16 06:09:50.758063] W [defaults.c:1331:default_forget]
(-->/usr/local/lib/glusterfs/3git/xlator/mount/fuse.so(+0xa1f2) [0x7f51432781f2]
(-->/usr/local/lib/libglusterfs.so.0(inode_unref+0x3c) [0x7f5144e5
816c] (-->/usr/local/lib/libglusterfs.so.0(+0x2d061) [0x7f5144e58061]))) 0-fuse:
xlator does not implement forget_cbk

This patch prevents such log messages from being seen.
Signed-off-by: Vijay Bellur <vbellur@redhat.com>
BUG: 979910
Change-Id: Ie5874138f46822b10ff4213bd1134d78330ec460
Reviewed-on: http://review.gluster.org/5932
Reviewed-by: Anand Avati <avati@redhat.com>
Tested-by: Anand Avati <avati@redhat.com>
2013-09-16 15:41:46 -07:00
Kaushal M
91cd0eae2c cli,glusterd: Task parameters in xml output
This patch introduces task parameters for the asynchronus task shown in
volume status. The parameters are only given for xml output. The
parameters shown currently are,
- source and destination bricks for replace-brick tasks
  ......
        <tasks>
          <task>
            <type>Replace brick</type>
            <id>3d1a1005-9d2e-4ae0-bd62-577bc1d333a3</id>
            <status>1</status>
            <params>
              <srcBrick>archm:/export/test4</srcBrick>
              <dstBrick>archm:/export/test-replace1</dstBrick>
            </params>
          </task>
        </tasks>
  ......
- list of bricks being removed for remove-brick tasks
  ......
        <tasks>
          <task>
            <type>Remove brick</type>
            <id>901c20ca-0da2-41de-8669-5f0caca6b846</id>
            <status>1</status>
            <params>
              <brick>archm:/export/test2</brick>
              <brick>archm:/export/test3</brick>
            </params>
          </task>
        </tasks>
  ......

The changes for non-xml output will be done in a subsequent patch.

Change-Id: I322afe2f83ed8adeddb99f7962c25911204dc204
BUG: 916577
Signed-off-by: Kaushal M <kaushal@redhat.com>
Reviewed-on: http://review.gluster.org/5771
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
Tested-by: Vijay Bellur <vbellur@redhat.com>
2013-09-13 03:38:04 -07:00
Niels de Vos
0b60850874 qemu-block: fix building from distribution tarball when glib2-devel is installed
Building RPMs from a 'make dist' tarball fails when qemu-block is
enabled. Enabling is done automatically when the glib2 development files
are available (enabled by ./configure).

Manual building with:

  $ ./autogen.sh && ./configure && make dist && rpmbuild -ta *.tar.gz

Building in mock works fine, glib2-devel is not installed by default so
the qemu-block xlator gets disabled. This change also adds glib2-devel
to the BuildRequires in the glusterfs.spec file, causing the qemu-block
xlator to be built by default, and included in the glusterfs RPM.

Change-Id: Ibb73628772586d9e07bbfde7a8ff2fc973489086
BUG: 986775
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: http://review.gluster.org/5896
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2013-09-12 13:45:40 -07:00
Niels de Vos
df12400012 tests/basic/rpm.t: run mock builds inside the current working directory
When running multiple mock builds at the same time, the space
requirements under /var/tmp increases as well. Not all systems (like
build.gluster.org) have enough free space in /var/tmp to host two
mock-roots, which causes building to fail randomly.

Change-Id: Iff1064d01893fb35ef59650d4373105068b5d3f7
BUG: 1006269
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: http://review.gluster.org/5899
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2013-09-12 13:45:14 -07:00
Aravinda VK
cd7951aa31 cli: Add statusStr xml tag to task list and rebalance/remove brick status
New xml tag statusStr added to following gluster cli commands
gluster volume status all --xml (For Task status)
gluster volume rebalance <VOLNAME> status --xml
gluster volume remove-brick <VOLNAME> <BRICK1..> status --xml

Example(volume status all):
<task>
    <type>Rebalance</type>
    <id>82d8d122-8738-4144-8507-d93fc98b61df</id>
    <status>3</status>
    <statusStr>completed</statusStr>
</task>

Example(volume rebalance <VOL> status)
<node>
    <nodeName>localhost</nodeName>
    <files>0</files>
    <size>0</size>
    <lookups>0</lookups>
    <failures>0</failures>
    <status>3</status>
    <statusStr>completed</statusStr>
</node>

Also modified task status as string instead of showing number
in gluster volume status all

Example:
Status of volume: gv1
Gluster process                                         Port    Online  Pid
------------------------------------------------------------------------------
Brick sumne.sumne:/gfs/b1                               49154   Y       15489
Brick sumne.sumne:/gfs/b2                               49155   Y       15493
NFS Server on localhost                                 N/A     N       15913

           Task                                      ID         Status
           ----                                      --         ------
      Rebalance    82d8d122-8738-4144-8507-d93fc98b61df      completed

BUG: 1003521
Change-Id: Ib283016af4c18132fb13fb33d44075782d77823c
Signed-off-by: Aravinda VK <avishwan@redhat.com>
Reviewed-on: http://review.gluster.org/5739
Reviewed-by: Kaushal M <kaushal@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
2013-09-12 10:57:22 -07:00
Vijay Bellur
643533c77f mgmt/glusterd: Update sub_count on remove brick
Change-Id: I7c17de39da03c6b2764790581e097936da406695
BUG: 1002556
Signed-off-by: Vijay Bellur <vbellur@redhat.com>
Reviewed-on: http://review.gluster.org/5893
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2013-09-11 22:51:36 -07:00
Ravishankar N
7cfb26296f doc: Steps to recover from split-brain
BUG: 1006776
Change-Id: I6f4bffe992758b2889df9ebfe117834dabe5c1cf
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Signed-off-by: Ravishankar N <ravishankar@redhat.com>
Reviewed-on: http://review.gluster.org/4216
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Tested-by: Anand Avati <avati@redhat.com>
2013-09-11 12:56:21 -07:00
Pranith Kumar K
015f1fd479 rpc/rpc-lib: rpcsvc should reply when rpc_err is set
Problem:
When requests are received on a connection before setvolume
is done, creating frame from the requests fail because there is
no association of the transport with the conn(i.e. xl_private).
xl_private is set only on set_volume. In such cases error response
is not sent from server xlator to that request because of which
operations on mount point are hanging.

Fix:
Set actor return value to RPCSVC_ACTOR_ERROR so that response is sent
even in these cases.

Change-Id: I74d7bc6849fde6c734008d67c1f4bc9d9f7a84f9
BUG: 1006367
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/5892
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2013-09-11 12:32:33 -07:00
Kaushal M
7d9bc0d214 cli: Fix 'status all' xml output when volumes are not started
CLI now only outputs one XML document for 'status all' only containing
those volumes which are started.

BUG: 1004218
Change-Id: Id4130fe59b3b74475d8bd1cc8134ac59a28f1b7e
Signed-off-by: Kaushal M <kaushal@redhat.com>
Reviewed-on: http://review.gluster.org/5773
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
2013-09-11 09:28:45 -07:00
Kaleb S. KEITHLEY
4c4aaa5abf tests/basic/rpm.t: run mock builds in parallel
save about six minutes per regression by building in parallel

Change-Id: Iaf6fa545e20b0322b10e9e4ab7cc3d4254a0ece5
BUG: 1006269
Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
Reviewed-on: http://review.gluster.org/5845
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
2013-09-10 13:39:42 -07:00
Kaushal M
c05cd5a392 glusterd: Allow bumping down a peer's op-version during probe
Earlier, a peer running a higher op-version couldn't be probed into a
cluster running at a lower op-version. This created issues when trying
to expand an upgraded cluster. This patch changes this behaviour.

The cluster no longer rejects a peer being probed if its op-version is
higher than the cluster op-version. The peer will reduce its op-version
if it doesn't have any volumes. If the peer contains volumes and needs
to reduce its op-version, it fails the handshake and the probe fails.

Change-Id: I12c6c873922799e1557b7184e956baea643d0dea
BUG: 1005038
Signed-off-by: Kaushal M <kaushal@redhat.com>
Reviewed-on: http://review.gluster.org/5715
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2013-09-10 09:07:56 -07:00
Avra Sengupta
367e9b922c glusterd: Added missing MY_UUID conversion in gsync staging
Change-Id: Ia4bf607e044d50636fb0a599a2ff91059f5b17aa
BUG: 1006177
Signed-off-by: Avra Sengupta <asengupt@redhat.com>
Reviewed-on: http://review.gluster.org/5887
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2013-09-10 03:41:16 -07:00
Harshavardhana
1f7469164a glusterfsd: use-readdirp w/o arguments should not turn off readdirp
`use-readdirp` has an optional argument in argp -
specifying just `--use-readdirp` command line should not
'turn off' readdirp, since that undermines the meaning
of such an argument.

Change-Id: I965d87e29bd0d61997d9be96fa698e270a2ee173
BUG: 983477
Signed-off-by: Harshavardhana <harsha@harshavardhana.net>
Reviewed-on: http://review.gluster.org/5851
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2013-09-09 21:41:41 -07:00
Anand Avati
ebcf1c8ddb cluster/dht: assign layout onto missing directories too
The current self-healing algorithm is ignoring missing directories
for assigning new layout. When lookup() is racing against mkdir()
or when self-healing a half-done mkdir(), the layout assignment split
must happen based on the final number of directories, and not the
currently existing number of directories (because we finish mkdir()
of missing directories before hash layout assignment).

Without this fix, concurrent mkdir() and lookup() will step on
each others feet, create a messed up layout on disk, and end up
with different in-memory layouts.

Once two clients have different in-memory layouts, creation of
subdirectory will not arbitrate on the same hashed subvolume and will
result in GFID mismatch of the sub-directory.

Change-Id: Ia47acad67c265060405984c822b4d37512b9dbb3
BUG: 907072
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/5849
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Reviewed-by: Peter Portante <pportant@redhat.com>
Tested-by: Peter Portante <pportant@redhat.com>
2013-09-09 14:58:09 -07:00
Niels de Vos
d3e533fe33 rpm: fix "warning: File listed twice: .../glusterd.info"
Commit a482e422 added glusterd.info to the %files section of the
glusterfs-server package. However, this file was listed already. This
causes the following warning when building the RPMs:

    warning: File listed twice: /var/lib/glusterd/glusterd.info

Merging the attributes for /var/lib/glusterd/glusterd.info into one line
prevents this warning.

Change-Id: I0d518ec186a8725dc4c5cba00b60da83fdadf103
BUG: 1005161
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: http://review.gluster.org/5836
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2013-09-08 09:49:31 -07:00
Anand Avati
2347a1a759 parser: make the parser thread safe.
The volfile parser thread safe by guarding the parsing phase
in a mutex. Thread safety becomes a problem when there are multiple
glfs_t objects created by gfapi and all of them potentially parse
the respective volfiles at the same time.

Change-Id: I4376019c4956994b72397ab36e6ac3ce849797ec
BUG: 1004519
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/5790
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@gluster.org>
2013-09-06 17:54:10 -07:00
Poornima
1955f7dc84 gfapi: store the open/create flags in fd_t object
The flags passed on to open and create calls were not being saved
in the fd_t object, hence the fd migration was failing.

Change-Id: I486bb818477fe4c393d64a711534a082162a0e53
BUG: 1005159
Signed-off-by: Poornima <pgurusid@redhat.com>
Reviewed-on: http://review.gluster.org/5837
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2013-09-06 15:56:57 -07:00
Harshavardhana
b610f1be7c glusterfsd: Round robin DNS should not be relied upon with
config service availability for clients.

Backupvolfile server as it stands is slow and prone to errors
with mount script and its combination with RRDNS. Instead in
theory it should use all the available nodes in 'trusted pool'
by default (Right now we don't have a mechanism in place for
this)

Nevertheless this patch provides a scenario where a list of
volfile-server can be provided on command as shown below

-----------------------------------------------------------------
$ glusterfs -s server1 .. -s serverN --volfile-id=<volname> \
      <mount_point>
-----------------------------------------------------------------
                   OR
-----------------------------------------------------------------
$ mount -t glusterfs -obackup-volfile-servers=<server2>: \
      <server3>:...:<serverN> <server1>:/<volname> <mount_point>
-----------------------------------------------------------------

Here ':' is used as a separator for mount script parsing

Now these will be remembered and recursively attempted for
fetching vol-file until exhausted. This would ensure that the
clients get 'volume' configs in a consistent manner avoiding the
need to poll through RRDNS.

Change-Id: If808bb8a52e6034c61574cdae3ac4e7e83513a40
BUG: 986429
Signed-off-by: Harshavardhana <harsha@harshavardhana.net>
Reviewed-on: http://review.gluster.org/5400
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2013-09-06 15:56:34 -07:00
Vijay Bellur
94fdc3dd79 Revert "tests/basic/rpm.t: run mock builds in parallel"
This reverts commit 91e8103794349ebdd578b77987a089456a59c52b

Reverting as regression tests are failing after this merge.

Change-Id: I9dcd430b2f9a431c3c01deab1a205aed297216a3
Reviewed-on: http://review.gluster.org/5842
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
Tested-by: Vijay Bellur <vbellur@redhat.com>
2013-09-06 06:49:49 -07:00
Kaleb S. KEITHLEY
91e8103794 tests/basic/rpm.t: run mock builds in parallel
save save about five minutes per regression

Change-Id: I75d91cfef63347ca756b9e1fedcb697e9537edc0
Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
Reviewed-on: http://review.gluster.org/5828
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2013-09-05 18:09:18 -07:00
Anand Avati
2a1f445d30 gfapi: apply an upper bound on nested symlink resolution
In case of nested symlink resolution, implement an upper
bound on the number of such nested levels the resolver will
descend. This limit is arbitrary, and set to 2048 nested
levels.

Change-Id: I264e5bd60d317eda97f4e6f49bd5d8694f8664a9
BUG: 1004100
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/5768
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Raghavendra Talur <rtalur@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2013-09-05 05:26:30 -07:00
Anand Avati
4b317e64ca glusterfs, gfapi: fix symbol clash
The callback structures in both protocol/client and glusterfsd,
gfapi used the same name for the actor table - gluster_cbk_actors.

CBKs are required only for the management connection, and the
actors of protocol/client are NOP functions. This supposed-to-be
NOP function dispatch tabble is actually ending up pointing to
the actor table of glusterfsd or gfapi.

These functions, even though set wrongly, are not even expected
to be called through the protocol/client callback path. Glusterd
however sends the FETCHSPEC (and other) notify callbacks to *all*
connected clients unconditionally, and there is a small period
of time when protocol/client is connected to glusterd for
PORTMAP query. If the FETCHSPEC callback notify is issued in
this window of time, we end up calling the wrong actor in the
client side resulting in a crash.

Change-Id: I605ff7df64c7faf4607369bbf275aedec28e1778
BUG: 1004091
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/5767
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
2013-09-05 05:24:42 -07:00
Venky Shankar
774e6d162d gsyncd / geo-rep: use the correct virtual xattr to collect 'gfid'
Change-Id: Ifea32ad1b2695f1668428fae6b263014bf320b70
BUG: 996379
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.org/5589
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
2013-09-05 02:33:34 -07:00
Avra Sengupta
3b8b184afe gfid-access: Error logs for ga_newfile_parse_args
Change-Id: I7aab98a70793bee936272f0b795f4c22c3733dd2
BUG: 1001055
Signed-off-by: Avra Sengupta <asengupt@redhat.com>
Reviewed-on: http://review.gluster.org/5705
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Tested-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
2013-09-05 02:33:18 -07:00
Venky Shankar
85c2b2de12 gsyncd / geo-rep: distributify slave
commit fbb8fd92 introduced slave distributification but had
some problems (monitor would crash upon gsyncd start). This
patch fixes the issue and makes code more pythonic ;)

Change-Id: I2cbf5669d81966046a4aeeb4a6ad11a947aa8f09
BUG: 1003807
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Tested-by: Amar Tumballi <amarts@redhat.com>
Reviewed-on: http://review.gluster.org/5761
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Avra Sengupta <asengupt@redhat.com>
Tested-by: Avra Sengupta <asengupt@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2013-09-04 20:55:49 -07:00
Venky Shankar
717d2db0c0 gsyncd / geo-rep: Fix "regular file" overloading link()
... missing entry2pb() call before going ahead with create.

Change-Id: I48de4df3f0ea6c789c9eeb10cd332ab461f3c868
BUG: 1003805
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Tested-by: Amar Tumballi <amarts@redhat.com>
Reviewed-on: http://review.gluster.org/5760
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Avra Sengupta <asengupt@redhat.com>
Tested-by: Avra Sengupta <asengupt@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2013-09-04 20:55:33 -07:00
Venky Shankar
eebd4e1485 geo-replication: fix the logic of choosing the remote node to sync
Change-Id: Ie15636357d89e94b6bfad0e168b1fcad53508c47
BUG: 1003807
Signed-off-by: Amar Tumballi <amarts@redhat.com>
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.org/5759
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Avra Sengupta <asengupt@redhat.com>
Tested-by: Avra Sengupta <asengupt@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2013-09-04 20:54:57 -07:00
Venky Shankar
b2e8a4517d gsyncd / geo-rep: maximum number of retries per changelog
With entry retries MAX_RETRIES is the threashold for the number
of times we retry a changelog. After hitting this limit, the
changelog is marked as processed after warning for incompleteness
in the log.

Change-Id: I777b71b4cdaa63335d9c936f2d64e5c0264a0f26
BUG: 1003803
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Tested-by: Amar Tumballi <amarts@redhat.com>
Reviewed-on: http://review.gluster.org/5758
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Avra Sengupta <asengupt@redhat.com>
Tested-by: Avra Sengupta <asengupt@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2013-09-04 20:54:38 -07:00
Venky Shankar
e0fa0bb1b1 gsyncd / geo-rep: fix hardlink creation on slave
Change-Id: I20fbd518bf519cbf2362b97aeb8be7c3b105087a
BUG: 1003805
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Tested-by: Amar Tumballi <amarts@redhat.com>
Reviewed-on: http://review.gluster.org/5757
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Avra Sengupta <asengupt@redhat.com>
Tested-by: Avra Sengupta <asengupt@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2013-09-04 20:54:21 -07:00
Venky Shankar
f201b46d1b gsyncd / geo-rep: handle ENOENT during readlink()
Change-Id: I8611122fda63ea496ad4491473ecabe96bf19cfa
BUG: 1003803
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Tested-by: Amar Tumballi <amarts@redhat.com>
Reviewed-on: http://review.gluster.org/5756
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Avra Sengupta <asengupt@redhat.com>
Tested-by: Avra Sengupta <asengupt@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2013-09-04 20:53:43 -07:00
Venky Shankar
8910136c09 gsyncd / geo-rep: overload rename with an unlink
This is certainly a "hack" for cases when after a rename
the gfid is unlinked (which is now pointed by a different
pargfid/basename). This was osbserved when a file edited
(uisng vi/vim) on the glusterfs mount point which has a
geo-replication session running.

Change-Id: I0a5d9f0f549aa4403671fa1f11e95510142e5720
BUG: 1003800
Signed-off-by: Avra Sengupta <asengupt@redhat.com>
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Tested-by: Amar Tumballi <amarts@redhat.com>
Reviewed-on: http://review.gluster.org/5755
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2013-09-04 20:52:02 -07:00
Venky Shankar
aa292ab99c gsyncd / geo-rep: fix "regular file" overloading rename
entry operation on the slave was using source parent gfid
and basename when renames are overloaded to use regular file
creation. This patch fixes the issue by using the destination
parent gfid and basename for these cases.

Change-Id: I1a4e8df7f07905224ce44ef5abd6f180234285ab
BUG: 1003800
Tested-by: Amar Tumballi <amarts@redhat.com>
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.org/5754
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Avra Sengupta <asengupt@redhat.com>
Tested-by: Avra Sengupta <asengupt@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2013-09-04 20:49:50 -07:00
Venky Shankar
2f305eaed9 gsyncd / geo-rep: "disjoint" cascading geo-replication sessions
Slave's xtime is now stored on the master itself (and that too only on
the root), which implies it cannot be propogated to the cascaded slave.
Thus the intermediate master now makes use of it's own volume information
to propogate volume-mark and xtime.

On starting Geo-Replication "geo-replication.ignore-pid-check" marker
option is enabled, which is an override for the client-pid check in
marker. This options triggers marker update only for geo-replication
auxillary mount (client-pid == -1). Since gsyncd not does setxattr()
directly on the bricks, this option won't trigger a chain of spurious
metadata updates that would need to be processed by gsyncd.

Change-Id: If50c5ef275dfb6b4ff4fd35be2565587e2fdf3e1
BUG: 996371
Original Author: Venky Shankar <vshankar@redhat.com>
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.org/5592
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Avra Sengupta <asengupt@redhat.com>
Tested-by: Avra Sengupta <asengupt@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2013-09-04 20:49:12 -07:00
Venky Shankar
a6a74c926b features/marker: force xtime updates (configurable) for client-pid = -1
This is required by Geo-Replication that does auxillary mount
with client-pid as -1 (which has special treatment at specific
places in GlusterFS), to trigger xtime updates on the intermediate
master in a cascading setup.

Marker too had a check to "not" mark updates for geo-replication's
auxillary mounts. With the new geo-replication design, xtimes are
not set by the master on the slave for all entities. Due to this
cascading setups were broken.

This patch introduces "geo-replication.ignore-pid-check" option
as a "override" for the client-pid check for gsyncd's client-pid.
When this options is enabled, marker start "marking" even if the
updates are from the special client.

Geo-Replication on the detection of itself being an intermediate
master, enables this option.

Change-Id: I9f7140edd12fef5480595ee0f93f35b94cdb8345
BUG: 996371
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.org/5591
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Avra Sengupta <asengupt@redhat.com>
Tested-by: Avra Sengupta <asengupt@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2013-09-04 20:43:03 -07:00
Venky Shankar
9ccdba413f gsyncd / geo-rep: introduce "recovery" mode
This mode introduces and optimized recovery of master,
used during failover-failback cases. Files are only
transferred if they are identified as changed on the slave.

A normal failback mechanism would be just to reverse the
direction of geo-replication syncing. Doing this is the
correct way but could be time consuming. "Recovery" mode
optimizes this by identifying files (as skipping them) duing
the one-shot crawl.

NOTE: Use this mode when you are _absolutely_ sure that the
namespace on the master is intact, ie. there is number of files
in master is equal to the number of files in the slave, but the
slave was written to (after a failover) when master was brought
back up.

- gluster volume geo <MV> <SLAVE>::<SV> config special_sync_mode recover
- gluster volume geo <MV> <SLAVE>::<SV> start

Change-Id: Idb698949e4e86698b15d04a35d33095562f51714
BUG: 994462
Original Author: Venky Shankar <vshankar@redhat.com>
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.org/5515
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Avra Sengupta <asengupt@redhat.com>
Tested-by: Avra Sengupta <asengupt@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2013-09-04 20:42:31 -07:00
Venky Shankar
07bb611867 gsyncd / geo-rep: remove deprecated code!
This includes "xtime" base crawling and syncing,
"Blind" and "Wrapup" mixins for failover/failback.

Change-Id: I9bc8695dc1c19b1261332f64621305dff2929b32
BUG: 994461
Original Author: Venky Shankar <vshankar@redhat.com>
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.org/5514
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Avra Sengupta <asengupt@redhat.com>
Tested-by: Avra Sengupta <asengupt@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2013-09-04 20:32:15 -07:00
Venky Shankar
60431cecae gsyncd / geo-rep: fix regular file creation on rename
On creation of a regular file on the slave (this happens
when an unlink()/rmdir() operation is performed on master
after a rename() or a link().

This patch fixes the regular file creation on the slave when
the above mentioned series of operation occurs. entry_ops()
on the slave for a rename() was missing the stat() information.

Change-Id: Idb80c6b3e27c1797dade1113ac8ce920db9c740c
BUG: 994353
Original Author: Venky Shankar <vshankar@redhat.com>
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.org/5509
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Avra Sengupta <asengupt@redhat.com>
Tested-by: Avra Sengupta <asengupt@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2013-09-04 20:28:12 -07:00
Venky Shankar
fe98c2902f glusterd/cli: Geo-Replication "status detail" cmd
Provides detailed status info in the following format

                    MASTER <master-vol>  SLAVE <slave-vol>

NODE   HEALTH   UPTIME  FILES SYNCD  FILES PENDING  BYTES PENDING  DELETES PENDING
-----------------------------------------------------------------------------------

This patch introdues "status detail" command to show crawl related
information in CLI. These values are "pulled" from gsyncd when
"status detail" is executed.

Change-Id: I1fdaf7180eacce054a864d34971dc160bd7301e1
BUG: 990420
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.org/5590
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Avra Sengupta <asengupt@redhat.com>
Tested-by: Avra Sengupta <asengupt@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2013-09-04 19:50:49 -07:00
Venky Shankar
603119c57c gsyncd / geo-rep: Introduce basic crawl instrumentation
This patch extends the persistent instrumentation work done by
Aravinda (@avishwa), by introducing a handfull of instrumentation
variables for crawl. These variables are "pulled up" by glusterd
in the event of a geo-replication status cli command and looks
something like below:

"Uptime=00:21:10;FilesSyned=2982;FilesPending=0;BytesPending=0;DeletesPending=0;"

"FilesPending", "BytesPending" and "DeletesPending" are short-lived
variables that are non-zero when a changelog is being processes (ie.
when an active sync in ongoing). After a successfull changelog process
"FilesPending" is summed up into "FilesSynced". The three short-lived
variabled are then reset to zero and the data is persisted

Additionally this patch also reverts some of the changes made for
BZ #986929 (those were not needed).

Change-Id: I948f1a0884ca71bc5e5bcfdc017d16c8c54fc30b
BUG: 990420
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.org/5441
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2013-09-04 19:50:09 -07:00
Venky Shankar
f0b92a45e5 gsyncd / geo-rep: periodically set slave xtime on slave
setting the slave xtime on the slave (after each changelog/xsync)
crawl helps in two things:
  * effective recover of master (failover/failback)
  * cascading setup - instances when the session from intermediate
                      master session is stopped, data is put on
                      the master -> slave sesssion and then the
                      cascading session is started again.

Change-Id: Ifae10a6ac09dc0d17707c3b5a3090bcf1efec8b6
BUG: 990900
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.org/5451
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Avra Sengupta <asengupt@redhat.com>
Tested-by: Avra Sengupta <asengupt@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2013-09-04 19:48:49 -07:00
Venky Shankar
1a642df1e9 glusterd: Added op-version checks to geo-rep commands.
Added op-version checks to all geo-rep commands. Min
op-version should be 2.

Change-Id: I942d897404e11e4d53123409731ba5cd252668fe
BUG: 847839
Signed-off-by: Avra Sengupta <asengupt@redhat.com>
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.org/5732
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2013-09-04 19:35:00 -07:00
Venky Shankar
81d32caa26 glusterd/gverify: Check for passwordless ssh in gverify.
Change-Id: I8c2d398114ad4534bcc052f9a5be8bbb2e7e2582
BUG: 999531
Signed-off-by: Avra Sengupta <asengupt@redhat.com>
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.org/5677
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2013-09-04 19:34:02 -07:00
Venky Shankar
fc6938613c glusterd: Allowing root@hostname::slave georep sessions to be created.
non-root@hostname::slave-vol geo-rep sessions are not supported.
only hostname and root@hostname sessions are supported, and are
treated as the same.

Change-Id: I87551e1bd4ff4e0e6520c34eb3d944587cc65476
BUG: 998933
Signed-off-by: Avra Sengupta <asengupt@redhat.com>
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.org/5659
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2013-09-04 19:32:12 -07:00
Venky Shankar
3bfa0dde13 glusterd/gverify.sh: Stops session being created with invalid slave details
create force will fail with proper message, if the ip is not
reachable, or is unable to fetch slave details.

Change-Id: I44a3ba777b37702ffd0e48e9cb46c51e293327d4
BUG: 988314
Signed-off-by: Avra Sengupta <asengupt@redhat.com>
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.org/5516
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2013-09-04 19:29:17 -07:00
Venky Shankar
59d37265ac gverify/glusterd : Added check to warn the user of existing files in slave volume
Change-Id: I41c9fb44613386189d9f3c090729f4ded8e4dea8
BUG: 990997
Signed-off-by: Avra Sengupta <asengupt@redhat.com>
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.org/5453
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2013-09-04 19:28:35 -07:00
Venky Shankar
c7cc5252a3 glusterd: Saving geo-rep session details in a more specific path
Now saving the session details in
/var/lib/glusterd/geo-replication/<mastervol>_<slaveip>_<slavevol>
repo to distinguish between two master-slave sessions where the
slavename is same across two different clusters.

Change-Id: I57c93f55cc9bd4fe2bffe579028aaf5e4335b223
BUG: 991501
Signed-off-by: Avra Sengupta <asengupt@redhat.com>
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Reviewed-on: http://review.gluster.org/5488
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2013-09-04 19:27:19 -07:00
Harshavardhana
fa095c2497 geo-replication: Use a md5 based unique control path
A hostname fqdn can be of length 255 according to RFC1123
------------------------->
/usr/include/bits/posix1_lim.h:#define _POSIX_HOST_NAME_MAX  255
<-------------------------
On linux this length is 64
------------------------->
/usr/include/bits/local_lim.h:#define HOST_NAME_MAX 64
<-------------------------

When a given hostname is > 45 (characters) - SSH fails with

-------------------------->
"ControlPath too long for Unix domain socket".
<--------------------------

Indicating that the total length of ControlPath which is
on linux should be 108

------------------------->
/usr/include/linux/un.h:#define UNIX_PATH_MAX   108
<-------------------------

This leads to "faulty" geo-replication status.

This patch brings in a new file called manifest which carries
given a geo-rep session some unique information - with which
a unique `md5` is generated in a 32length digest, this ensures
that we don't exceed UNIX_PATH_MAX limitations instead we use
a conservative approach and still be able to provide a unique
socket path.

Change-Id: I3a6a27d605d751a86e7c82eace4561d9b0134fe1
BUG: 990330
Signed-off-by: Harshavardhana <harsha@harshavardhana.net>
Reviewed-on: http://review.gluster.org/5681
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Csaba Henk <csaba@redhat.com>
2013-09-04 12:29:04 -07:00
Brian Foster
3b23f9acae performance/readdir-ahead: introduce directory read-ahead translator
This is a translator to improve the performance of typical,
sequential directory reads (i.e., ls). readdir-ahead begins
preloading the contents of a directory on open and serves readdir
requests from the preloaded content. readdir-ahead is currently
implemented to only handle the single threaded directory read
case.

readdir-ahead is currently disabled by default. It can be enabled
with the following command:

	gluster volume set <volname> readdir-ahead on

The following are results of a getdents test on a single brick
volume.

Test info:

- Single VM, gluster client/server.
- Volume mounted with native client using --gid-timeout=2.
- getdents on single directory with 100k 0-byte files.

Test results:

- !readdir-ahead

read 3120080 bytes from offset 0
3 MiB, 4348 ops, 0:00:07.00 (416.590 KiB/sec and 594.4737 ops/sec)

- readdir-ahead

read 3120080 bytes from offset 0
3 MiB, 4348 ops, 0:00:03.00 (820.116 KiB/sec and 1170.3043 ops/sec)

BUG: 980517
Change-Id: Ieceb9e1eb47d1d5b5af8da2bf03839537364653f
Signed-off-by: Brian Foster <bfoster@redhat.com>
Reviewed-on: http://review.gluster.org/4519
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2013-09-04 09:04:15 -07:00
shishir gowda
98df12f040 mgmt/glusterd: Regenerate client volfiles during upgrade
Change-Id: I1442bc1d115a9c6ecf139a0ca9da74d07e0fe928
BUG: 1003855
Signed-off-by: shishir gowda <sgowda@redhat.com>
Reviewed-on: http://review.gluster.org/5764
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Reviewed-by: Kaushal M <kaushal@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2013-09-03 21:39:42 -07:00