Commit Graph

12678 Commits

Author SHA1 Message Date
Atin Mukherjee
c2723c57d2 logging: Fix GF_LOG_OCCASSIONALLY API
GF_LOG_OCCASSIONALLY doesn't log on the first instance rather at every
42nd iterations which isn't effective as in some cases we might not have
the code flow hitting the same log for as many as 42 times and we'd end
up suppressing the log.

Updates: bz#1679904
Change-Id: Iee293281d25a652b64df111d59b13de4efce06fa
Signed-off-by: Atin Mukherjee <amukherj@redhat.com>
(cherry picked from commit d0d3e10d44366c68fc153e48b229e72a4aa26e61)
2019-04-16 10:49:22 +00:00
Atin Mukherjee
55c5e2ecc7 glusterd: fix txn-id mem leak
This commit ensures the following:
1. Don't send commit op request to the remote nodes when gluster v
status all is executed as for the status all transaction the local
commit gets the name of the volumes and remote commit ops are
technically a no-op. So no need for additional rpc requests.
2. In op state machine flow, if the transaction is in staged state and
op_info.skip_locking is true, then no need to set the txn id in the
priv->glusterd_txn_opinfo dictionary which never gets freed.

Fixes: bz#1694610
Change-Id: Ib6a9300ea29633f501abac2ba53fb72ff648c822
Signed-off-by: Atin Mukherjee <amukherj@redhat.com>
(cherry picked from commit 34e010d64905b7387de57840d3fb16a326853c9b)
2019-04-16 10:49:09 +00:00
Ravishankar N
5946db166a afr: add client-pid to all gf_event() calls
client-pid for glustershd is GF_CLIENT_PID_SELF_HEALD
client-pid for glfsheal is GF_CLIENT_PID_GLFS_HEALD

updates: bz#1693155
Change-Id: Ib3a863af160ff48c822a5e6b0c27c575c9887470
Signed-off-by: Ravishankar N <ravishankar@redhat.com>
(cherry picked from commit 8016d51a3bbd410b0b927ed66be50a09574b7982)
2019-04-16 10:48:40 +00:00
Kaleb S. KEITHLEY
27a96f1f34 rpclib: slow floating point math and libm
In release-6 rpc/rpc-lib (libgfrpc) added the function
get_rightmost_set_bit() which calls log2(3), a call that takes
a floating point parameter and returns a floating point.

It's used thusly:
    right_most_unset_bit = get_rightmost_set_bit(...);

(So is it really the right-most unset bit, or the right-most set bit?)

It's unclear to me whether this is in the data path or not. If it is,
it's rather scary to think about integer-to-float and float-to-integer
conversions and slow calls to libm functions in the data path.

gcc and clang have __builtin_ctz() which returns the same result as
get_rightmost_set_bit(), and does it substantially faster. Approx
20M iterations of get_rightmost_set_bit() took ~33sec of wall clock
time on my devel machine, while 20M iterations of __builtin_ctz()
took < 9sec; get_rightmost_set_bit() is 3x slower than __builtin_ctz().

And as a side benefit, we can again eliminate the need to link libgfrpc
with libm.

Change-Id: If9e7e80874577c52223f8125b385fc930de20699
fixes: bz#1692957
Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
2019-04-16 10:48:06 +00:00
Ashish Pandey
c5bc21ebbc cluster/ec: Fix handling of heal info cases without locks
When we use heal info command, it takes lot of time as in
some cases it takes lock on entries to find out if the
entry actually needs heal or not.

There are some cases where we can avoid these locks and
can conclude if the entry needs heal or not.

1 - We do a lookup (without lock) on an entry, which we found in
.glusterfs/indices/xattrop, and find that lock count is
zero. Now if the file contains dirty bit set on all or any
brick, we can say that this entry needs heal.

2 - If the lock count is one and dirty is greater than 1,
then it also means that some fop had left the dirty bit set
which made the dirty count of current fop (which has taken lock)
more than one. At this point also we can definitely say that
this entry needs heal.

This patch is modifying code to take into consideration above two
points.
It is also changing code to not to call ec_heal_inspect if ec_heal_do
was called from client side heal. Client side heal triggeres heal
only when it is sure that it requires heal.

[We have changed the code to not to call heal for lookup]

updates bz#1697764
Change-Id: I7f09f0ecd12f65a353297aefd57026fd2bebdf9c
Signed-off-by: Ashish Pandey <aspandey@redhat.com>
(cherry picked from commit da47caf2405c08c9abafc4a55525a8b2c2dd5bb8)
2019-04-09 05:27:52 +00:00
Kotresh HR
381e7603d9 geo-rep: Fix syncing multiple rename of symlink
Problem:
Geo-rep fails to sync rename of symlink if it's
renamed multiple times if creation and rename
happened successively

Worker crash at slave:
Traceback (most recent call last):
  File "/usr/libexec/glusterfs/python/syncdaemon/repce.py",  in worker
    res = getattr(self.obj, rmeth)(*in_data[2:])
  File "/usr/libexec/glusterfs/python/syncdaemon/resource.py", in entry_ops
    [ESTALE, EINVAL, EBUSY])
  File "/usr/libexec/glusterfs/python/syncdaemon/syncdutils.py", in errno_wrap
    return call(*arg)
  File "/usr/libexec/glusterfs/python/syncdaemon/libcxattr.py", in lsetxattr
    cls.raise_oserr()
  File "/usr/libexec/glusterfs/python/syncdaemon/libcxattr.py", in raise_oserr
    raise OSError(errn, os.strerror(errn))
OSError: [Errno 12] Cannot allocate memory

Geo-rep Behaviour:
1. SYMLINK doesn't record target path in changelog.
   So while syncing SYMLINK, readlink is done on
   master to get target path.

2. Geo-rep will create destination if source is not
   present while syncing RENAME. Hence while syncing
   RENAME of SYMLINK, target path is collected from
   destination.

Cause:
If symlink is created and renamed multiple times, creation of
symlink is ignored, as it's no longer present on master at
that path. While symlink is renamed multiple times at master,
when syncing first RENAME of SYMLINK, both source and destination
is not present, hence target path is not known.  In this case,
while creating destination directly at slave,  regular file
attributes were encoded into blob instead of symlink,
causing failure in gfid-access translator while decoding
blob.

Solution:
While syncing of RENAME of SYMLINK, when target is not known
and when src and destination is not present on the master,
don't create destination. Ignore the rename. It's ok to ignore.
If it's unliked, it's fine.  If it's renamed to something else,
it will be synced then.

Backport of:
> Change-Id: Ibdfa495513b7c05b5370ab0b89c69a6802338d87
> BUG: bz#1693648
> Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit 877af725b3e35b548d6d7aeec5adb21721d8bf8b)

Change-Id: Ibdfa495513b7c05b5370ab0b89c69a6802338d87
fixes: bz#1694002
Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit 877af725b3e35b548d6d7aeec5adb21721d8bf8b)
2019-04-03 04:31:06 +00:00
Soumya Koduri
491ff40a7a gfapi: Unblock epoll thread for upcall processing
With commit#ad35193,we have made changes to offload
processing upcall notifications to synctask so as not
to block epoll threads. However seems like the issue wasnt
fully addressed.

In "glfs_cbk_upcall_data" -> "synctask_new1" after creating synctask
if there is no callback defined, the thread waits on synctask_join
till the syncfn is finished. So that way even with those changes,
epoll threads are blocked till the upcalls are processed.

Hence the right fix now is to define a callback function for that
synctask "glfs_cbk_upcall_syncop" so as to unblock epoll/notify threads
completely and the upcall processing can happen in parallel by synctask
threads.

Change-Id: I4d8645e3588fab2c3ca534e0112773aaab68a5dd
fixes: bz#1694561
Signed-off-by: Soumya Koduri <skoduri@redhat.com>
(cherry picked from commit 4a03a71c6171f6e8382664d9d29857d06ef37741)
2019-04-03 04:30:53 +00:00
Poornima G
dbfff66092 client-rpc: Fix the payload being sent on the wire
The fops allocate 3 kind of payload(buffer) in the client xlator:
- fop payload, this is the buffer allocated by the write and put fop
- rsphdr paylod, this is the buffer required by the reply cbk of
  some fops like lookup, readdir.
- rsp_paylod, this is the buffer required by the reply cbk of fops like
  readv etc.

Currently, in the lookup and readdir fop the rsphdr is sent as payload,
hence the allocated rsphdr buffer is also sent on the wire, increasing
the bandwidth consumption on the wire.

With this patch, the issue is fixed.

Fixes: bz#1692101
Change-Id: Ie8158921f4db319e60ad5f52d851fa5c9d4a269b
Signed-off-by: Poornima G <pgurusid@redhat.com>
2019-03-29 15:23:52 +00:00
Ravishankar N
4e7afab231 gfapi: add function to set client-pid
This api offers the ability to set the pid of a client to a particular
value, identical to how gluster fuse clients provide the --client-pid
option. This is an internal API to be used by gluster processes only. See
https://lists.gluster.org/pipermail/gluster-devel/2019-March/055925.html
for more details. Currently glfsheal is the only proposed consumer.

updates: bz#1693155
Change-Id: I0620be2127d79d69cdd57cffb29bba44e6e5da1f
Signed-off-by: Ravishankar N <ravishankar@redhat.com>
(cherry picked from commit 08d502c3b05c6f7831bb4cc764bd458b405a99b1)
2019-03-29 11:08:32 +00:00
Yaniv Kaul
cba59f6cb7 server.c: fix Coverity CID 1399758
1399758 Dereference before null check

It was introduced @ commit 67f48bfcc16a38052e6c9ae7c25e69b03b8ae008

updates: bz#1691187
> updates: bz#789278
> Signed-off-by: Yaniv Kaul <ykaul@redhat.com>

> Change-Id: I1424b008b240691fe2a8924e31c708d0fb4f362d
> (cherry picked from commit 8aff9cc5c6277ef7dacfb89f1392b7c2eda9b825)

Change-Id: Ie2160fb9ae9cdeacf845e849da7f6001b3b6b10b
2019-03-21 04:57:14 +00:00
ShyamsundarR
3fadf5cc41 doc: Final version of release-6 release notes
Fixes: bz#1672818
Change-Id: I6a98985a7f25bc2b85af5bd85f4be3ffac7d619d
Signed-off-by: ShyamsundarR <srangana@redhat.com>
2019-03-19 10:17:57 -04:00
Kotresh HR
7e90a3b592 release-notes/6.0: Add ctime feature changes in release notes
Change-Id: I3a305b9eb292a450c83de5628ceeadcb0a44afc7
updates: bz#1672818
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Signed-off-by: ShyamsundarR <srangana@redhat.com>
2019-03-19 09:51:05 -04:00
Amar Tumballi
1398fa8045 release-notes: add status of gd2 and a highlights section
Change-Id: Idd4fd07b3fcdd7810c242eff0dcca9072048a43e
fixes: bz#1672818
Signed-off-by: Amar Tumballi <amarts@redhat.com>
Signed-off-by: ShyamsundarR <srangana@redhat.com>
2019-03-19 09:51:05 -04:00
Mohit Agrawal
c7a582818d glusterfsd: Brick is getting crash at the time of startup
Problem: Brick is getting crash because graph was not activated
         at the time of accessing server_conf

Solution: To avoid the crash check ctx->active before processing
          a request

> Change-Id: Ib112e0eace19189e45f430abdac5511c026bed47
> fixes: bz#1687705
>(cherry picked from commit 67f48bfcc16a38052e6c9ae7c25e69b03b8ae008)
> (Reviewed on upstream link https://review.gluster.org/#/c/glusterfs/+/22339/)

Change-Id: I1367c564f04edbad145575b811c67522cc318851
fixes: bz#1688218
Signed-off-by: Mohit Agrawal <moagrawal@redhat.com>
2019-03-13 11:18:23 +00:00
Ashish Pandey
3bcf210a5a rpm: add thin-arbiter package
Discussion on thin arbiter volume -
https://github.com/gluster/glusterfs/issues/352#issuecomment-350981148

Main idea of having this rpm package is to deploy thin-arbiter
without glusterd and other commands on a node, and all we need
on that tie-breaker node is to run a single glusterfs command.
Also note that, no other glusterfs installation needs
thin-arbiter.so.

Make sure RPM contains sample vol file, which can work by default,
and a script to configure that volfile, along with translator image.

Change-Id: Ibace758373d8a991b6a19b2ecc60c93b2f8fc489
updates: bz#1672818
Signed-off-by: Amar Tumballi <amarts@redhat.com>
Signed-off-by: Ashish Pandey <aspandey@redhat.com>
(cherry picked from commit ca9bef7f1538beb570fcb190ff94f86f0b8ba38a)
2019-03-13 01:48:26 +00:00
Anoop C S
bda2feeaf2 doc: Update release notes for Samba integration
Change-Id: I2a246111522bd3cb0c9b228ed3387e72b0ac20a4
updates: bz#1672818
Signed-off-by: Anoop C S <anoopcs@redhat.com>
2019-03-12 20:56:57 +00:00
Aravinda VK
05fce4e668 eventsapi: Fix error while handling GlusterCmdException
`GlusterCmdException` was wrongly accessed instead of accessing
`GlusterCmdException.message`.

Fixes: bz#1687248
Change-Id: I35ec1b05726050bfd8761e05ad9b9e47917dc0c6
Signed-off-by: Aravinda VK <avishwan@redhat.com>
(cherry picked from commit 27f6375df009c8c4798b72aeafce79456007d21f)
2019-03-12 20:53:37 +00:00
Mohit Agrawal
36c75523c1 glusterfsd: Multiple shd processes are spawned on brick_mux environment
Problem: Multiple shd processes are spawned while starting volumes
         in the loop on brick_mux environment.glusterd spawn a process
         based on a pidfile and shd daemon is taking some time to
         update pid in pidfile due to that glusterd is not able to
         get shd pid

Solution: Commit cd249f4cb7 changed
          the code to update pidfile in parent for any gluster daemon
          after getting the status of forking child in parent.To resolve
          the same correct the condition update pidfile in parent only
          for glusterd and for rest of the daemon pidfile is updated in
          child

> Change-Id: Ifd14797fa949562594a285ec82d58384ad717e81
> fixes: bz#1684404
> (Cherry pick from commit 66986594a9023c49e61b32769b7e6b260b600626)
> (Reviewed on upstream link https://review.gluster.org/#/c/glusterfs/+/22290/)

Change-Id: I9a68064d2da1acd0ec54b4071a9995ece0c3320c
fixes: bz#1683880
Signed-off-by: Mohit Agrawal <moagrawal@redhat.com>
2019-03-12 20:52:59 +00:00
karthik-us
9b58cfc83c cluster/afr: Send truncate on arbiter brick from SHD
Problem:
In an arbiter volume configuration SHD will not send any writes onto the arbiter
brick even if there is data pending marker for the arbiter brick. If we have a
arbiter setup on the geo-rep master and there are data pending markers for the files
on arbiter brick, SHD will not mark any data changelog during healing. While syncing
the data from master to slave, if the arbiter-brick is considered as ACTIVE, then
there is a chance that slave will miss out some data. If the arbiter brick is being
newly added or replaced there is a chance of slave missing all the data during sync.

Fix:
If there is data pending marker for the arbiter brick, send truncate on the arbiter
brick during heal, so that it will record truncate as the data transaction in changelog.

Change-Id: I3242ba6cea6da495c418ef860d9c3359c5459dec
fixes: bz#1687672
Signed-off-by: karthik-us <ksubrahm@redhat.com>
2019-03-12 20:51:47 +00:00
Mohit Agrawal
0d0b54a9de glusterd: glusterd memory leak while running "gluster v profile" in a loop
Problem: glusterd has memory leak while running "gluster v profile"
         in a loop

Solution: Resolve leak code path to avoid leak

> Change-Id: Id608703ff6d0ad34ed8f921a5d25544e24cfadcd
> fixes: bz#1685414
> (Cherry pick from commit 9374484917466dff4688d96ff7faa0de1c804a6c)
> (Reviewed on link https://review.gluster.org/#/c/glusterfs/+/22301/)

Change-Id: I1ca118265f97b188f94b3d5cff649ec36cb18ca0
fixes: bz#1685771
Signed-off-by: Mohit Agrawal <moagrawal@redhat.com>
2019-03-12 20:51:03 +00:00
Kaleb S. KEITHLEY
47ed784e37 packaging: rdma on s390x, unnecessary ldconfig scriptlets
s390x has rdma since around fedora27, and in RHEL7 since July 2016.
see https://src.fedoraproject.org/rpms/glusterfs/pull-request/5

Reported-By: Igor Gnatenko
Reported-By: Dan Horák <dan@danny.cz>

master branch Change-Ids:
> Change-Id: Idad29425495f9c69bfd84b53b009a607abb418f0
> Change-Id: I914f36290e96a387cdd64ee633e75c2940ec1e56

Change-Id: I7747700e982b365533ebd031164708b9ad97f18a
fixes: bz#1686875
Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
2019-03-08 10:19:43 -05:00
Sanju Rakonde
7964a320f3 core: make compute_cksum function op_version compatible
Problem: commit 5a152a changed the mechansim of computing the
checksum. In heterogeneous cluster, peers are running into
rejected state because we have different cksum computation
mechansims in upgraded and non-upgraded nodes.

Solution: add a check for op-version so that all the nodes
in the cluster follow the same mechanism for computing the
cksum.

fixes: bz#1684029

> Change-Id: I1508f000e8c9895588b6011b8b6cc0eda7102193
> BUG: bz#1685120
> Signed-off-by: Sanju Rakonde <srakonde@redhat.com>
> (cherry picked from commit 073444b693b7a91c42963512e0fdafb57ad46670)

Change-Id: I1508f000e8c9895588b6011b8b6cc0eda7102193
2019-03-08 14:45:41 +00:00
Amar Tumballi
9c441360ac dict: handle STR_OLD data type in xdr conversions
Currently a dict conversion on wire for 3.x protocol happens using
`dict_unserialize()`, which sets the type of data as STR_OLD. But the
new protocol doesn't send it over the wire as its not considered as a
valid format in new processes.

But considering we deal with old and new protocol when we do a rolling
upgrade, it will allow us to get all the information properly with new
protocol.

Credits: Krutika Dhananjay

Fixes: bz#1686364
Change-Id: I165c0021fb195b399790b9cf14a7416ae75ec84f
Signed-off-by: Amar Tumballi <amarts@redhat.com>
2019-03-08 14:08:40 +00:00
Raghavendra Gowdappa
ae24a2c390 performance/readdir-ahead: fix deadlock
This deadlock happens while processing dentry corresponding to current
directory (.) in rda_fill_readdirp. In this case following order is
followed:

LOCK(directory_fd_ctx->lock);
  rda_inode_ctx_get_iatt -> LOCK(directory_inode->lock);

However, in rda_mark_inode_dirty following lock order is followed:
LOCK(directory_inode->lock);
  LOCK(directory_fd_ctx->lock);

these two codepaths when executed concurrently resulted in a deadlock.

Current patch fixes this by removing locking directory inode and
fd-ctx in rda_fill_readdirp. This is fine as directory inode's stat
won't change due to writes to files within directory.

Change-Id: Ic93a67a0dac8229bb0d79582e526a512e6f2569c
Signed-off-by: Raghavendra Gowdappa <rgowdapp@redhat.com>
Fixes: bz#1686399
2019-03-08 14:08:29 +00:00
Susant Palai
8444c1ea54 io-threads: Prioritize fops with NO_ROOT_SQUASH pid
There was 30% regression observed in mkdir path with commit
b139bc58eb. On analysis it is found
that io-threads xlator deprioritzes fops with all -ve pid.

Some context in to the no-root-squash pid requirement:
DHT xlator does some of the internal fops with root privileges. This is
needed so that operations like layout healing should not be abandoned
because a non root user is operating.  If root-squash option is enabled
the layout set operation looses its root privilege as server xlator
converts the uid and pid to random numbers. Hence, the above mentioned
commit converted pid to GF_CLIENT_PID_NO_ROOT_SQUASH to continue fops
as root.

Combining the above I am proposing not to deprioritize fops with
no-root-squash pid.

> Change-Id: I54d056c01b25729304a77f9242fbaff39c5672ba
> fixes: bz#1676430
> Signed-off-by: Susant Palai <spalai@redhat.com>
(cherry picked from commit f5c3b1727f55ffaa3dcdb3c3a09b968ebb45dbb2)

Change-Id: I54d056c01b25729304a77f9242fbaff39c5672ba
fixes: bz#1676429
Signed-off-by: Susant Palai <spalai@redhat.com>
2019-03-06 03:16:53 +00:00
hujianfei
ea78c029a7 glusterfsd: Do not process PROFILE_NFS_INFO if graph is not ready
Otherwise, gnfs will crash in following situation.
Also see commit 2f9e555f.

Reproducible Steps:
1. kill gnfs process
2. service glusterd restart;gluster volume profile [vol] info nfs

dump trace info:
/lib64/libglusterfs.so.0(_gf_msg_backtrace_nomem+0xc2)[0x7fcf5cb6a872]
/lib64/libglusterfs.so.0(gf_print_trace+0x324)[0x7fcf5cb743a4]
/lib64/libc.so.6(+0x35670)[0x7fcf5b1d5670]
/usr/sbin/glusterfs(glusterfs_handle_nfs_profile+0x114)[0x7fcf5d066474]
/lib64/libglusterfs.so.0(synctask_wrap+0x12)[0x7fcf5cba1502]
/lib64/libc.so.6(+0x47110)[0x7fcf5b1e7110]

Fixes: bz#1684777

Change-Id: Id68edb3e4646c39544e0b4c90b5e0a9083b37b0d
Signed-off-by: hujianfei <hujianfei@cmss.chinamobile.com>
(cherry picked from commit 09394856e1197db4933e7e814589a3340eb739cd)
2019-03-04 03:38:08 +00:00
Kaleb S. KEITHLEY
06a65016f1 glusterfind: revert shebangs to #!/usr/bin/python3
Using #!/usr/bin/env python is illegal in Fedora and RHEL packaging.
(Debian and SUSE packaging will complain about it too but don't
enforce it like Fedora packaging does. Yet.)

We went through a giant exercise fixing these once already.

The python needs to be python2/python3 clean. There is a build
scriptlet that converts the shebangs back to /usr/bin/python on rhel7.

Change-Id: If208a2557f7e5d727cda98c27905e249cf485d5b
fixes: bz#1683716
Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
2019-03-02 11:54:35 +00:00
Milind Changire
4cb1d6d94a socket: socket event handlers now return void
Problem:
Returning any value from socket event handlers to the event sub-system
doesn't make sense since event sub-system cannot handle socket
sub-system errors.

Solution:
Change return type of all socket event handlers to 'void'

mainline:
> Change-Id: I70dc2c57f12b7ea2fae41120f71aa0d7fe0b2b6f
> Fixes: bz#1651246
> Signed-off-by: Milind Changire <mchangir@redhat.com>
> Reviewed-on: https://review.gluster.org/c/glusterfs/+/22221

Change-Id: I70dc2c57f12b7ea2fae41120f71aa0d7fe0b2b6f
Fixes: bz#1683900
Signed-off-by: Milind Changire <mchangir@redhat.com>
(cherry picked from commit 776ba851c6ee6c265253d44cf1d6e4e3d4a21772)
2019-03-02 11:54:24 +00:00
Arjun Sharma
cd6795c5e8 Updating the glfs_setattr and glfs_fsetattr public APIs to use glfs_stat
Change-Id: I0e86c8222c88b4c71087ec287ba81f8353d70822
updates: #389
Signed-off-by: Arjun <arjsharm@redhat.com>
2019-02-28 12:00:30 +05:30
Sanju Rakonde
73b8d6c8d1 glusterd: remove experimental xlator options from glusterd-volume-set.c
experimental xlators have been removed from the codebase. But we
missed to remove the options related to experimental xlators from
the codebase. This patch removes those options.

fixes: bz#1683506
Change-Id: I3fa7e14c6cd8ebde5cebc8d2b0cb2409bf37c1ae
Signed-off-by: Sanju Rakonde <srakonde@redhat.com>
(cherry picked from commit 5cddd4d758014fe116d9c130632eada2ecded88c)
2019-02-27 03:39:55 +00:00
Aravinda VK
77f327862a eventsapi: Fix Python3 compatibility issues
- Fixed Relative import and non-package import related issues.
- socketserver import issues fix
- Renamed installed directory name to `gfevents` from `events`(To
  avoid any issues with other global libs)

Fixes: bz#1683008
Change-Id: I3dc38bc92b23387a6dfbcc0ab8283178235bf756
Signed-off-by: Aravinda VK <avishwan@redhat.com>
(cherry picked from commit cd68f7b88b9a2c9a4e4ff9fca61517384e54130a)
2019-02-26 14:11:41 +00:00
Niels de Vos
3d7cd3d0f5 build: do not install service related files when building --without-server
It seems that glusterfs-6 disables building the server (./configure
--without-server) on el6 and hence the installed but unpackaged files
should not get installed in the first place.

Backport of commit 30f358bb6b69b8c31da550608a4996a84e8b420b:
> Change-Id: I7bea08202dd59ac309de8bc861da920ed3600bc6
> Fixes: bz#1680587
> Signed-off-by: Niels de Vos <ndevos@redhat.com>

Change-Id: I7bea08202dd59ac309de8bc861da920ed3600bc6
Fixes: bz#1680586
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2019-02-26 08:38:53 +00:00
Amar Tumballi
0419d9ab93 fuse: reflect the actual default for lru-limit option
in both `--help` text and man page

updates: bz#1679998
Change-Id: I9aa9367c6863ac8e2403255280697c9e6be26cf0
Signed-off-by: Amar Tumballi <amarts@redhat.com>
2019-02-25 15:26:08 +00:00
David Spisla
c360210685 md-cache: Adapt integer data types to avoid integer overflow
The "struct iatt" in iatt.h is using int64_t types for storing
the atime, mtime and ctime. Therefore the struct 'struct md_cache' in
md-cache.c should also use this types to avoid an integer overflow.

This can happen e.g. if someone uses a very high default-retention-period
in the WORM-Xlator.

Change-Id: I605268d300ab622b9c8ab30e459dc00d9340aad1
fixes: bz#1680020
Signed-off-by: David Spisla <david.spisla@iternity.com>
(cherry picked from commit 15423e14f16dd1a15ee5e5cbbdbdd370e57ed59f)
2019-02-25 15:25:57 +00:00
Niels de Vos
fce763a6fc build: include gluster/__init__.* files for python packaging
The gluster/__init__.py file was originaly part of the glupy
installation. This file is required to have the python-gluster package
function correctly, it is expected to provide the 'gluster' namespace
for other python packages (like from the libgfapi-python project).

Because glupy does not exist anymore, this file is now added to in a new
extras/python directory.

Backport of commit b235a56394c09bb1d7308a620053dbe098d15967:
> Change-Id: I14fe959778ee3344d7d54ba342c7928a4d8080a2
> Fixes: c3fcff9ccb
> Fixes: 8293d21280
> Updates: bz#1642810
> Signed-off-by: Niels de Vos <ndevos@redhat.com>

Change-Id: I04e1e0dc4022b53c02053d5452f305b78e56c389
Updates: bz#1680585
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2019-02-25 11:16:43 +00:00
Sahina Bose
5801a0e30e packaging: Obsoleting glusterfs-gnfs for upgrade
Master branch reference
>link : https://review.gluster.org/#/c/glusterfs/+/22190/
>fixes: bz#1672711
>Change-Id: Iad7194e788a8eeecd617614e9f8a1fe3264a384d
>Signed-off-by: Sahina Bose <sabose@redhat.com>
>Signed-off-by: Jiffin Tony Thottan <jthottan@redhat.com>
>(cherry picked from commit d3608bd26cbc3e47fc0e60e85bd9f66515e050b1)

fixes: bz#1679965
Change-Id: Iad7194e788a8eeecd617614e9f8a1fe3264a384d
Signed-off-by: Jiffin Tony Thottan <jthottan@redhat.com>
2019-02-22 11:23:49 +00:00
ShyamsundarR
e6fabf8c33 doc: Added initial release notes for release-6
Change-Id: I31fff344045344c501536b207c864e90ed34bf20
updates: bz#1672818
Signed-off-by: ShyamsundarR <srangana@redhat.com>
2019-02-22 04:12:45 +00:00
N Balachandran
951abf5c53 cluster/dht: Request linkto xattrs in dht_rmdir opendir
If parallel-readdir is enabled, the rda xlator is loaded
below dht in the graph and proactively lists and caches
entries when an opendir is performed. dht_rmdir checks if
the directory being deleted contains stale linkto files by
performing a readdirp on its child subvols. However, as
the entries are actually read in during the opendir operation
which does not request the linkto xattr,no linkto xattrs are
present for the entries causing dht to incorrectly identify
them as data files and fail the rmdir operation with ENOTEMPTY.
DHT now always adds the linkto xattr in the list of xattrs
requested in the opendir.

Change-Id: I0711198e66c59146282eb8b88084170bedfb4018
fixes: bz#1679004
Signed-off-by: N Balachandran <nbalacha@redhat.com>
2019-02-22 03:34:36 +00:00
Kinglong Mee
5c9b2dfb4d dht: fix double extra unref of inode at heal path
The loc_wipe is done in the _out_ section, inode_unref(loc.parent) here
casues a double extra unref of loc.parent.

> Change-Id: I2dc809328d3d34bf7b02c7df9a4f97788af511e6
> Signed-off-by: Kinglong Mee <mijinlong@open-fs.com>
(cherry-pick of https://review.gluster.org/#/c/glusterfs/+/21998/)

Change-Id: I2dc809328d3d34bf7b02c7df9a4f97788af511e6
updates: bz#1679275
Signed-off-by: Susant Palai <spalai@redhat.com>
2019-02-22 03:34:06 +00:00
Raghavendra Gowdappa
bff265f8bc performance/write-behind: fix use-after-free in readdirp
Two issues were found:
1. in wb_readdirp_cbk, inode should unrefed after wb_inode is
unlocked. Otherwise, inode and hence the context wb_inode can be freed
by the type we try to unlock wb_inode
2. wb_readdirp_mark_end iterates over a list of wb_inodes of children
of a directory. But inodes could've been freed and hence the list
might be corrupted. To fix take a reference on inode before adding it
to invalidate_list of parent.

Change-Id: I911b0e0b2060f7f41ded0b05db11af6f9b7c09c5
Signed-off-by: Raghavendra Gowdappa <rgowdapp@redhat.com>
Updates: bz#1678570
(cherry picked from commit 64cc4458918e8c8bfdeb114da0a6501b2b98491a)
2019-02-22 03:33:07 +00:00
Raghavendra Gowdappa
1f8b2912c8 performance/write-behind: handle call-stub leaks
Change-Id: I7be9a5f48dcad1b136c479c58b1dca1e0488166d
Signed-off-by: Raghavendra Gowdappa <rgowdapp@redhat.com>
Fixes: bz#1678570
(cherry picked from commit 6175cb10cd5f59f3c7ae4100bc78f359b68ca3e9)
2019-02-19 06:23:34 +00:00
N Balachandran
8d4b8e2644 cluster/dht: Fix lookup selfheal and rmdir race
A race between the lookup selfheal and rmdir can cause
directories to be healed only on non-hashed subvols.
This can prevent the directory from being listed from
the mount point and in turn causes rm -rf to fail with
ENOTEMPTY.
Fix: Update the layout information correctly and reduce
the call count only after processing the response.

Change-Id: I812779aaf3d7bcf24aab1cb158cb6ed50d212451
fixes: bz#1677260
Signed-off-by: N Balachandran <nbalacha@redhat.com>
2019-02-18 14:45:30 +00:00
Raghavendra Gowdappa
2026d24667 performance/md-cache: introduce an option to control invalidation of inodes
Explicit invalidation by calling inode_invalidate is necessary when
same (meta)data is shared/access across multiple mounts. Without an
explicit inode_invalidate call, caches in the mount which didn't
witness writes wouldn't be aware of changes as writes wouldn't have
passed through them. However, if (meta)data is not shared, all
relevant I/O goes through the cache of single mount and hence is
coherent with (meta)data on bricks always. So, explicit inode
invalidation can be disabled for this case which gives a huge
performance boost for workloads that write data and then immediately
read the data they just wrote. Note that otherwise, local writes
(which pass through the cache) will change ctime and cause unnecessary
invalidations.

The name of the option that controls this behavior is
"performance.global-cache-invalidation". This option is global and it
purges caches both in glusterfs and kernel stack for native FUSE
mounts. For non-native FUSE mounts, it purges cache only from
glusterfs stack. This option is effective only when
performance.stat-prefetch is on.

Note that there is a similar option "performance.cache-invalidation",
but the scope of that option is limited to quick-read and md-cache.

Change-Id: I462bb4b65ff9aae1f6ba76f50b1f2f94fb10323b
Signed-off-by: Raghavendra Gowdappa <rgowdapp@redhat.com>
updates: bz#1674364
(cherry picked from commit 2b5aa4489de2017a03bcb6ec8986286f0c76a670)
2019-02-18 14:44:27 +00:00
Raghavendra Gowdappa
beca73e04d mount/fuse: fix bug related to --auto-invalidation in mount script
When "auto-invalidation" option was not specified for mount script,
glusterfs cmdline ended with "--auto-invalidation=" option. This patch
fixes that bug in mount script.

Thanks to Amar for reporting it.

Change-Id: Ie5cd4c6ffb3ac644d9d2b032035f914a935d05a8
Signed-off-by: Raghavendra Gowdappa <rgowdapp@redhat.com>
updates: bz#1674364
2019-02-18 14:44:27 +00:00
Atin Mukherjee
e571233df3 glusterd: improve logging
glusterd_resolve_all_bricks failure log should highlight the brick
identifier.

Change-Id: I035b4650ef6a14bb1e1221d3bad1c40f9d43dbdd
fixes: bz#1673972
Signed-off-by: Atin Mukherjee <amukherj@redhat.com>
(cherry picked from commit 12af2067a82e37079e76723d3e25ba1c72ca078a)
2019-02-11 16:24:39 +00:00
ShyamsundarR
352e203266 api: Update all future API versions to rel-6
As release 6 is branched, all future APIs now become 6.0

This change implements the same.

Change-Id: I6db368b4dc8585278ec11d4a411adcd04635de53
Updates: bz#1672818
Signed-off-by: ShyamsundarR <srangana@redhat.com>
(cherry picked from commit ddcb3c7438472072daa6fbe3f3b87067b6d7bc54)
2019-02-08 15:47:36 +00:00
Nigel Babu
8f6973e326 Bump up timeout for tests on AWS
Fixes: bz#1673267
Change-Id: I2b9be45f199f6436b858536c6f49be85902217f0
Signed-off-by: Nigel Babu <nigelb@redhat.com>
2019-02-07 13:27:53 +05:30
ShyamsundarR
c396ba1826 Update rfc.sh for release-6
Signed-off-by: ShyamsundarR <srangana@redhat.com>
2019-02-05 19:59:13 -05:00
Sanju Rakonde
90922d20f5 glusterd: get-state command should not fail if any brick is gone bad
Problem: get-state command will error out, if any of the underlying
brick(s) of volume(s) in the cluster go bad.

It is expected that get-state command should not error out, but
should generate an output successfully.

Solution: In glusterd_get_state(), a statfs call is made on the
brick path for every bricks of the volumes to calculate the total
and free memory available. If any of statfs call fails on any
brick, we should not error out and should report total memory and free
memory of that brick as 0.

This patch also handles a statfs failure scenario in
glusterd_store_retrieve_bricks().

fixes: bz#1672205

Change-Id: Ia9e8a1d8843b65949d72fd6809bd21d39b31ad83
Signed-off-by: Sanju Rakonde <srakonde@redhat.com>
2019-02-05 14:40:25 +00:00
Kotresh HR
a9b51f2d1f geo-rep: Fix configparser import issue
'configparser' is backported to python2 and can
be installed using pip (pip install configparser).
So trying to import 'configparser' first and later
'ConfigParser' can cause issues w.r.t unicode strings.

Always try importing 'ConfigParser' first and then
'configparser'. This solves python2/python3 compat
issues.

Change-Id: I2a87c3fc46476296b8cb547338f35723518751cc
fixes: bz#1671637
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2019-02-05 04:19:59 +00:00