12689 Commits

Author SHA1 Message Date
Pranith Kumar K
7ec3a8527f cluster/afr: Remove local from owners_list on failure of lock-acquisition
When eager-lock lock acquisition fails because of say network failures, the
local is not being removed from owners_list, this leads to accumulation of
waiting frames and the application will hang because the waiting frames are
under the assumption that another transaction is in the process of acquiring
lock because owner-list is not empty. Handled this case as well in this patch.
Added asserts to make it easier to find these problems in future.

fixes bz#1699731
Change-Id: I3101393265e9827755725b1f2d94a93d8709e923
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
2019-04-16 11:29:03 +00:00
Mohit Agrawal
d2de3f6639 core: Log level changes do not effect on running client process
Problem: commit c34e4161f3cb6539ec83a9020f3d27eb4759a975 set log-level
         per xlator during reconfigure only for a brick process not for
         the client process.

Solution: 1) Change per xlator log-level only if brick_mux is enabled.To make sure
             about brick multiplex introudce a flag brick_mux at ctx->cmd_args.

Note: There are two other changes done with this patch
      1) Ignore client-log-level option to attach a brick with
         already running brick if brick_mux is enabled
      2) Add a log to print pid of the running process to make easier
         debugging

> Change-Id: I39e85de778e150d0685cd9a79425ce8b4783f9c9
> Signed-off-by: Mohit Agrawal <moagrawal@redhat.com>
> Fixes: bz#1696046
> (Cherry picked from commit 798aadbe51a9a02dd98a0f861cc239ecf7c8ed57)
> (Reviewed on upstream link https://review.gluster.org/#/c/glusterfs/+/22495/)

Change-Id: If91682830f894ab8f6857f19dcb1797fc15ca64c
Fixes: bz#1699715
Signed-off-by: Mohit Agrawal <moagrawal@redhat.com>
2019-04-16 10:59:36 +00:00
Kotresh HR
80d2dae631 posix/ctime: Fix stat(time attributes) inconsistency during readdirp
Problem:
   Creation of tar file on gluster volume throws warning
'file changed as we read it'

Cause:
   During readdirp, for few of the files whose inode is not
present, time attributes were served from backend. This caused
the ctime of few files to be different between before readdir
and after readdir by tar.

Solution:
  If ctime feature is enabled and inode is not present, don't
serve the time attributes from backend file, serve it from xattr.

Backport of:
> Patch: https://review.gluster.org/22540
> BUG: 1698078
> Change-Id: I427ef865f97399475faf5aa6ca495f7e317603ae
> Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit c56f102da21c5b69e656a055aaf736281596284d)

fixes: bz#1699703
Change-Id: I427ef865f97399475faf5aa6ca495f7e317603ae
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2019-04-16 10:57:12 +00:00
Kinglong Mee
5f51159463 ec: fix truncate lock to cover the write in tuncate clean
ec_truncate_clean does writing under the lock granted for truncate,
but the lock is calculated by ec_adjust_offset_up, so that,
the write in ec_truncate_clean is out of lock.

Updates: bz#1699499
Change-Id: Idbe1fd48d26afe49c36b77db9f12e0907f5a4134
Signed-off-by: Kinglong Mee <kinglongmee@gmail.com>
(cherry picked from commit 0e1223491e964096384edfae5032ed0d50d028ad)
2019-04-16 10:57:05 +00:00
Mohit Agrawal
88ecd64604 core: Brick is not able to detach successfully in brick_mux environment
Problem: In brick_mux environment, while volumes are stopped in a
         loop bricks are not detached successfully. Brick's are not
         detached because xprtrefcnt has not become 0 for detached brick.
         At the time of initiating brick detach process server_notify
         saves xprtrefcnt on detach brick and once counter has become
         0 then server_rpc_notify spawn a server_graph_janitor_threads
         for cleanup brick resources.xprtrefcnt has not become 0 because
         socket framework is not working due to assigning 0 as a fd for socket.
         In commit dc25d2c1eeace91669052e3cecc083896e7329b2
         there was a change in changelog fini to close htime_fd if htime_fd is not
         negative, by default htime_fd is 0 so it close 0 also.

Solution: Initialize htime_fd to -1 after just allocate changelog_priv
          by GF_CALLOC

> Fixes: bz#1699025
> Change-Id: I5f7ca62a0eb1c0510c3e9b880d6ab8af8d736a25
> Signed-off-by: Mohit Agrawal <moagrawal@redhat.com>
> (cherry picked from commit b777d83001d8006420b6c7d2d88fe68950aa7e00)

Change-Id: I7a2b6fc2d36405d51990376333e093661be48475
Fixes: bz#1699714
Signed-off-by: Mohit Agrawal <moagrawal@redhat.com>
2019-04-16 10:55:51 +00:00
Mohit Agrawal
08278e8823 build: glusterfs build is failing on RHEL-6
Problem: glusterfs build is throwing error undefined
         reference to `dlclose' on RHEL 6

Solution: Add LIB_DL link in Makefile.am to resolve the same

> Fixes: bz#1696512
> Change-Id: I58019ca9e29d569d8e6df282b8ab178ad540843b
> Signed-off-by: Mohit Agrawal <moagrawa@redhat.com>
> (cherry picked from commit 064aad721c249d63fb89686b32e5d15de50e2f8c)

Change-Id: I4f68553b501c283e2066ddc64e204db40552ee73
Fixes: bz#1699713
Signed-off-by: Mohit Agrawal <moagrawa@redhat.com>
2019-04-16 10:55:43 +00:00
karthik-us
6bd52e5108 cluster/afr: Thin-arbiter SHD fixes
This patch address post-merge review comments for commit
5784a00f997212d34bd52b2303e20c097240d91c

Change-Id: I7ed954664a2ae8e1091d23ee3ceb9c66e83bfeac
fixes: bz#1699319
Signed-off-by: karthik-us <ksubrahm@redhat.com>
2019-04-16 10:53:04 +00:00
Pranith Kumar K
fbba6e397f protocol/client: Do not fallback to anon-fd if fd is not open
If an open comes on a file when a brick is down and after the brick comes up,
a fop comes on the fd, client xlator would still wind the fop on anon-fd
leading to wrong behavior of the fops in some cases.

Example:
If lk fop is issued on the fd just after the brick is up in the scenario above,
lk fop will be sent on anon-fd instead of failing it on that client xlator.
This lock will never be freed upon close of the fd as flush on anon-fd is
invalid and is not wound below server xlator.

As a fix, failing the fop unless the fd has FALLBACK_TO_ANON_FD flag.

Change-Id: I77692d056660b2858e323bdabdfe0a381807cccc
fixes bz#1699198
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
(cherry picked from commit 92ae26ae8039847e38c738ef98835a14be9d4296)
2019-04-16 10:52:32 +00:00
Ravishankar N
74db82dd5d afr: thin-arbiter read txn fixes
- Fixes afr_ta_read_txn() to handle inode refresh failures.
code-path.
- Fixes a double free issue of dict.

Note: This patch address post-merge review comments for commit
69532c141be160b3fea03c1579ae4ac13018dcdf

fixes: bz#1693992
Change-Id: Id5299b45b68569d47df6b73755918237a1592cb4
Signed-off-by: Ravishankar N <ravishankar@redhat.com>
(cherry picked from commit 500bd0014128e6727e83b6cb77e8ac94304b8f4a)
2019-04-16 10:51:51 +00:00
Ashish Pandey
f792fd01aa cluster/ec: Don't enqueue an entry if it is already healing
Problem:
1 - heal-wait-qlength is by default 128. If shd is disabled
and we need to heal files, client side heal is needed.
If we access these files that will trigger the heal.
However, it has been observed that a file will be enqueued
multiple times in the heal wait queue, which in turn causes
queue to be filled and prevent other files to be enqueued.

2 - While a file is going through healing and a write fop from
mount comes on that file, it sends write on all the bricks including
healing one. At the end it updates version and size on all the
bricks. However, it does not unset dirty flag on all the bricks,
even if this write fop was successful on all the bricks.
After healing completion this dirty flag remain set and never
gets cleaned up if SHD is disabled.

Solution:
1 - If an entry is already in queue or going through heal process,
don't enqueue next client side request to heal the same file.

2 - Unset dirty on all the bricks at the end if fop has succeeded on
all the bricks even if some of the bricks are going through heal.

Change-Id: Ia61ffe230c6502ce6cb934425d55e2f40dd1a727
updates: bz#1693223
Signed-off-by: Ashish Pandey <aspandey@redhat.com>
(cherry picked from commit 313dcefe7a62bd16cd794040df068f9bec9c6927)
2019-04-16 10:50:49 +00:00
Atin Mukherjee
aca7ec21ed glusterd: load ctime in the client graph only if it's not turned off
Considering ctime is a client side feature, we can't blindly load ctime
xlator into the client graph if it's explicitly turned off, that'd
result into backward compatibility issue where an old client can't mount
a volume configured on a server which is having ctime feature.

Fixes: bz#1698471
Change-Id: I6ae7b96d056073aa6746de9a449cf319786d45cc
Signed-off-by: Atin Mukherjee <amukherj@redhat.com>
(cherry picked from commit efbf8abcc3bc729a90d4a7b57dc515f1df8a5863)
2019-04-16 10:50:21 +00:00
Atin Mukherjee
c2723c57d2 logging: Fix GF_LOG_OCCASSIONALLY API
GF_LOG_OCCASSIONALLY doesn't log on the first instance rather at every
42nd iterations which isn't effective as in some cases we might not have
the code flow hitting the same log for as many as 42 times and we'd end
up suppressing the log.

Updates: bz#1679904
Change-Id: Iee293281d25a652b64df111d59b13de4efce06fa
Signed-off-by: Atin Mukherjee <amukherj@redhat.com>
(cherry picked from commit d0d3e10d44366c68fc153e48b229e72a4aa26e61)
2019-04-16 10:49:22 +00:00
Atin Mukherjee
55c5e2ecc7 glusterd: fix txn-id mem leak
This commit ensures the following:
1. Don't send commit op request to the remote nodes when gluster v
status all is executed as for the status all transaction the local
commit gets the name of the volumes and remote commit ops are
technically a no-op. So no need for additional rpc requests.
2. In op state machine flow, if the transaction is in staged state and
op_info.skip_locking is true, then no need to set the txn id in the
priv->glusterd_txn_opinfo dictionary which never gets freed.

Fixes: bz#1694610
Change-Id: Ib6a9300ea29633f501abac2ba53fb72ff648c822
Signed-off-by: Atin Mukherjee <amukherj@redhat.com>
(cherry picked from commit 34e010d64905b7387de57840d3fb16a326853c9b)
2019-04-16 10:49:09 +00:00
Ravishankar N
5946db166a afr: add client-pid to all gf_event() calls
client-pid for glustershd is GF_CLIENT_PID_SELF_HEALD
client-pid for glfsheal is GF_CLIENT_PID_GLFS_HEALD

updates: bz#1693155
Change-Id: Ib3a863af160ff48c822a5e6b0c27c575c9887470
Signed-off-by: Ravishankar N <ravishankar@redhat.com>
(cherry picked from commit 8016d51a3bbd410b0b927ed66be50a09574b7982)
2019-04-16 10:48:40 +00:00
Kaleb S. KEITHLEY
27a96f1f34 rpclib: slow floating point math and libm
In release-6 rpc/rpc-lib (libgfrpc) added the function
get_rightmost_set_bit() which calls log2(3), a call that takes
a floating point parameter and returns a floating point.

It's used thusly:
    right_most_unset_bit = get_rightmost_set_bit(...);

(So is it really the right-most unset bit, or the right-most set bit?)

It's unclear to me whether this is in the data path or not. If it is,
it's rather scary to think about integer-to-float and float-to-integer
conversions and slow calls to libm functions in the data path.

gcc and clang have __builtin_ctz() which returns the same result as
get_rightmost_set_bit(), and does it substantially faster. Approx
20M iterations of get_rightmost_set_bit() took ~33sec of wall clock
time on my devel machine, while 20M iterations of __builtin_ctz()
took < 9sec; get_rightmost_set_bit() is 3x slower than __builtin_ctz().

And as a side benefit, we can again eliminate the need to link libgfrpc
with libm.

Change-Id: If9e7e80874577c52223f8125b385fc930de20699
fixes: bz#1692957
Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
2019-04-16 10:48:06 +00:00
Ashish Pandey
c5bc21ebbc cluster/ec: Fix handling of heal info cases without locks
When we use heal info command, it takes lot of time as in
some cases it takes lock on entries to find out if the
entry actually needs heal or not.

There are some cases where we can avoid these locks and
can conclude if the entry needs heal or not.

1 - We do a lookup (without lock) on an entry, which we found in
.glusterfs/indices/xattrop, and find that lock count is
zero. Now if the file contains dirty bit set on all or any
brick, we can say that this entry needs heal.

2 - If the lock count is one and dirty is greater than 1,
then it also means that some fop had left the dirty bit set
which made the dirty count of current fop (which has taken lock)
more than one. At this point also we can definitely say that
this entry needs heal.

This patch is modifying code to take into consideration above two
points.
It is also changing code to not to call ec_heal_inspect if ec_heal_do
was called from client side heal. Client side heal triggeres heal
only when it is sure that it requires heal.

[We have changed the code to not to call heal for lookup]

updates bz#1697764
Change-Id: I7f09f0ecd12f65a353297aefd57026fd2bebdf9c
Signed-off-by: Ashish Pandey <aspandey@redhat.com>
(cherry picked from commit da47caf2405c08c9abafc4a55525a8b2c2dd5bb8)
2019-04-09 05:27:52 +00:00
Kotresh HR
381e7603d9 geo-rep: Fix syncing multiple rename of symlink
Problem:
Geo-rep fails to sync rename of symlink if it's
renamed multiple times if creation and rename
happened successively

Worker crash at slave:
Traceback (most recent call last):
  File "/usr/libexec/glusterfs/python/syncdaemon/repce.py",  in worker
    res = getattr(self.obj, rmeth)(*in_data[2:])
  File "/usr/libexec/glusterfs/python/syncdaemon/resource.py", in entry_ops
    [ESTALE, EINVAL, EBUSY])
  File "/usr/libexec/glusterfs/python/syncdaemon/syncdutils.py", in errno_wrap
    return call(*arg)
  File "/usr/libexec/glusterfs/python/syncdaemon/libcxattr.py", in lsetxattr
    cls.raise_oserr()
  File "/usr/libexec/glusterfs/python/syncdaemon/libcxattr.py", in raise_oserr
    raise OSError(errn, os.strerror(errn))
OSError: [Errno 12] Cannot allocate memory

Geo-rep Behaviour:
1. SYMLINK doesn't record target path in changelog.
   So while syncing SYMLINK, readlink is done on
   master to get target path.

2. Geo-rep will create destination if source is not
   present while syncing RENAME. Hence while syncing
   RENAME of SYMLINK, target path is collected from
   destination.

Cause:
If symlink is created and renamed multiple times, creation of
symlink is ignored, as it's no longer present on master at
that path. While symlink is renamed multiple times at master,
when syncing first RENAME of SYMLINK, both source and destination
is not present, hence target path is not known.  In this case,
while creating destination directly at slave,  regular file
attributes were encoded into blob instead of symlink,
causing failure in gfid-access translator while decoding
blob.

Solution:
While syncing of RENAME of SYMLINK, when target is not known
and when src and destination is not present on the master,
don't create destination. Ignore the rename. It's ok to ignore.
If it's unliked, it's fine.  If it's renamed to something else,
it will be synced then.

Backport of:
> Change-Id: Ibdfa495513b7c05b5370ab0b89c69a6802338d87
> BUG: bz#1693648
> Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit 877af725b3e35b548d6d7aeec5adb21721d8bf8b)

Change-Id: Ibdfa495513b7c05b5370ab0b89c69a6802338d87
fixes: bz#1694002
Signed-off-by: Kotresh HR <khiremat@redhat.com>
(cherry picked from commit 877af725b3e35b548d6d7aeec5adb21721d8bf8b)
2019-04-03 04:31:06 +00:00
Soumya Koduri
491ff40a7a gfapi: Unblock epoll thread for upcall processing
With commit#ad35193,we have made changes to offload
processing upcall notifications to synctask so as not
to block epoll threads. However seems like the issue wasnt
fully addressed.

In "glfs_cbk_upcall_data" -> "synctask_new1" after creating synctask
if there is no callback defined, the thread waits on synctask_join
till the syncfn is finished. So that way even with those changes,
epoll threads are blocked till the upcalls are processed.

Hence the right fix now is to define a callback function for that
synctask "glfs_cbk_upcall_syncop" so as to unblock epoll/notify threads
completely and the upcall processing can happen in parallel by synctask
threads.

Change-Id: I4d8645e3588fab2c3ca534e0112773aaab68a5dd
fixes: bz#1694561
Signed-off-by: Soumya Koduri <skoduri@redhat.com>
(cherry picked from commit 4a03a71c6171f6e8382664d9d29857d06ef37741)
2019-04-03 04:30:53 +00:00
Poornima G
dbfff66092 client-rpc: Fix the payload being sent on the wire
The fops allocate 3 kind of payload(buffer) in the client xlator:
- fop payload, this is the buffer allocated by the write and put fop
- rsphdr paylod, this is the buffer required by the reply cbk of
  some fops like lookup, readdir.
- rsp_paylod, this is the buffer required by the reply cbk of fops like
  readv etc.

Currently, in the lookup and readdir fop the rsphdr is sent as payload,
hence the allocated rsphdr buffer is also sent on the wire, increasing
the bandwidth consumption on the wire.

With this patch, the issue is fixed.

Fixes: bz#1692101
Change-Id: Ie8158921f4db319e60ad5f52d851fa5c9d4a269b
Signed-off-by: Poornima G <pgurusid@redhat.com>
2019-03-29 15:23:52 +00:00
Ravishankar N
4e7afab231 gfapi: add function to set client-pid
This api offers the ability to set the pid of a client to a particular
value, identical to how gluster fuse clients provide the --client-pid
option. This is an internal API to be used by gluster processes only. See
https://lists.gluster.org/pipermail/gluster-devel/2019-March/055925.html
for more details. Currently glfsheal is the only proposed consumer.

updates: bz#1693155
Change-Id: I0620be2127d79d69cdd57cffb29bba44e6e5da1f
Signed-off-by: Ravishankar N <ravishankar@redhat.com>
(cherry picked from commit 08d502c3b05c6f7831bb4cc764bd458b405a99b1)
2019-03-29 11:08:32 +00:00
Yaniv Kaul
cba59f6cb7 server.c: fix Coverity CID 1399758
1399758 Dereference before null check

It was introduced @ commit 67f48bfcc16a38052e6c9ae7c25e69b03b8ae008

updates: bz#1691187
> updates: bz#789278
> Signed-off-by: Yaniv Kaul <ykaul@redhat.com>

> Change-Id: I1424b008b240691fe2a8924e31c708d0fb4f362d
> (cherry picked from commit 8aff9cc5c6277ef7dacfb89f1392b7c2eda9b825)

Change-Id: Ie2160fb9ae9cdeacf845e849da7f6001b3b6b10b
2019-03-21 04:57:14 +00:00
ShyamsundarR
3fadf5cc41 doc: Final version of release-6 release notes
Fixes: bz#1672818
Change-Id: I6a98985a7f25bc2b85af5bd85f4be3ffac7d619d
Signed-off-by: ShyamsundarR <srangana@redhat.com>
2019-03-19 10:17:57 -04:00
Kotresh HR
7e90a3b592 release-notes/6.0: Add ctime feature changes in release notes
Change-Id: I3a305b9eb292a450c83de5628ceeadcb0a44afc7
updates: bz#1672818
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Signed-off-by: ShyamsundarR <srangana@redhat.com>
2019-03-19 09:51:05 -04:00
Amar Tumballi
1398fa8045 release-notes: add status of gd2 and a highlights section
Change-Id: Idd4fd07b3fcdd7810c242eff0dcca9072048a43e
fixes: bz#1672818
Signed-off-by: Amar Tumballi <amarts@redhat.com>
Signed-off-by: ShyamsundarR <srangana@redhat.com>
2019-03-19 09:51:05 -04:00
Mohit Agrawal
c7a582818d glusterfsd: Brick is getting crash at the time of startup
Problem: Brick is getting crash because graph was not activated
         at the time of accessing server_conf

Solution: To avoid the crash check ctx->active before processing
          a request

> Change-Id: Ib112e0eace19189e45f430abdac5511c026bed47
> fixes: bz#1687705
>(cherry picked from commit 67f48bfcc16a38052e6c9ae7c25e69b03b8ae008)
> (Reviewed on upstream link https://review.gluster.org/#/c/glusterfs/+/22339/)

Change-Id: I1367c564f04edbad145575b811c67522cc318851
fixes: bz#1688218
Signed-off-by: Mohit Agrawal <moagrawal@redhat.com>
2019-03-13 11:18:23 +00:00
Ashish Pandey
3bcf210a5a rpm: add thin-arbiter package
Discussion on thin arbiter volume -
https://github.com/gluster/glusterfs/issues/352#issuecomment-350981148

Main idea of having this rpm package is to deploy thin-arbiter
without glusterd and other commands on a node, and all we need
on that tie-breaker node is to run a single glusterfs command.
Also note that, no other glusterfs installation needs
thin-arbiter.so.

Make sure RPM contains sample vol file, which can work by default,
and a script to configure that volfile, along with translator image.

Change-Id: Ibace758373d8a991b6a19b2ecc60c93b2f8fc489
updates: bz#1672818
Signed-off-by: Amar Tumballi <amarts@redhat.com>
Signed-off-by: Ashish Pandey <aspandey@redhat.com>
(cherry picked from commit ca9bef7f1538beb570fcb190ff94f86f0b8ba38a)
2019-03-13 01:48:26 +00:00
Anoop C S
bda2feeaf2 doc: Update release notes for Samba integration
Change-Id: I2a246111522bd3cb0c9b228ed3387e72b0ac20a4
updates: bz#1672818
Signed-off-by: Anoop C S <anoopcs@redhat.com>
2019-03-12 20:56:57 +00:00
Aravinda VK
05fce4e668 eventsapi: Fix error while handling GlusterCmdException
`GlusterCmdException` was wrongly accessed instead of accessing
`GlusterCmdException.message`.

Fixes: bz#1687248
Change-Id: I35ec1b05726050bfd8761e05ad9b9e47917dc0c6
Signed-off-by: Aravinda VK <avishwan@redhat.com>
(cherry picked from commit 27f6375df009c8c4798b72aeafce79456007d21f)
2019-03-12 20:53:37 +00:00
Mohit Agrawal
36c75523c1 glusterfsd: Multiple shd processes are spawned on brick_mux environment
Problem: Multiple shd processes are spawned while starting volumes
         in the loop on brick_mux environment.glusterd spawn a process
         based on a pidfile and shd daemon is taking some time to
         update pid in pidfile due to that glusterd is not able to
         get shd pid

Solution: Commit cd249f4cb783f8d79e79468c455732669e835a4f changed
          the code to update pidfile in parent for any gluster daemon
          after getting the status of forking child in parent.To resolve
          the same correct the condition update pidfile in parent only
          for glusterd and for rest of the daemon pidfile is updated in
          child

> Change-Id: Ifd14797fa949562594a285ec82d58384ad717e81
> fixes: bz#1684404
> (Cherry pick from commit 66986594a9023c49e61b32769b7e6b260b600626)
> (Reviewed on upstream link https://review.gluster.org/#/c/glusterfs/+/22290/)

Change-Id: I9a68064d2da1acd0ec54b4071a9995ece0c3320c
fixes: bz#1683880
Signed-off-by: Mohit Agrawal <moagrawal@redhat.com>
2019-03-12 20:52:59 +00:00
karthik-us
9b58cfc83c cluster/afr: Send truncate on arbiter brick from SHD
Problem:
In an arbiter volume configuration SHD will not send any writes onto the arbiter
brick even if there is data pending marker for the arbiter brick. If we have a
arbiter setup on the geo-rep master and there are data pending markers for the files
on arbiter brick, SHD will not mark any data changelog during healing. While syncing
the data from master to slave, if the arbiter-brick is considered as ACTIVE, then
there is a chance that slave will miss out some data. If the arbiter brick is being
newly added or replaced there is a chance of slave missing all the data during sync.

Fix:
If there is data pending marker for the arbiter brick, send truncate on the arbiter
brick during heal, so that it will record truncate as the data transaction in changelog.

Change-Id: I3242ba6cea6da495c418ef860d9c3359c5459dec
fixes: bz#1687672
Signed-off-by: karthik-us <ksubrahm@redhat.com>
2019-03-12 20:51:47 +00:00
Mohit Agrawal
0d0b54a9de glusterd: glusterd memory leak while running "gluster v profile" in a loop
Problem: glusterd has memory leak while running "gluster v profile"
         in a loop

Solution: Resolve leak code path to avoid leak

> Change-Id: Id608703ff6d0ad34ed8f921a5d25544e24cfadcd
> fixes: bz#1685414
> (Cherry pick from commit 9374484917466dff4688d96ff7faa0de1c804a6c)
> (Reviewed on link https://review.gluster.org/#/c/glusterfs/+/22301/)

Change-Id: I1ca118265f97b188f94b3d5cff649ec36cb18ca0
fixes: bz#1685771
Signed-off-by: Mohit Agrawal <moagrawal@redhat.com>
2019-03-12 20:51:03 +00:00
Kaleb S. KEITHLEY
47ed784e37 packaging: rdma on s390x, unnecessary ldconfig scriptlets
s390x has rdma since around fedora27, and in RHEL7 since July 2016.
see https://src.fedoraproject.org/rpms/glusterfs/pull-request/5

Reported-By: Igor Gnatenko
Reported-By: Dan Horák <dan@danny.cz>

master branch Change-Ids:
> Change-Id: Idad29425495f9c69bfd84b53b009a607abb418f0
> Change-Id: I914f36290e96a387cdd64ee633e75c2940ec1e56

Change-Id: I7747700e982b365533ebd031164708b9ad97f18a
fixes: bz#1686875
Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
2019-03-08 10:19:43 -05:00
Sanju Rakonde
7964a320f3 core: make compute_cksum function op_version compatible
Problem: commit 5a152a changed the mechansim of computing the
checksum. In heterogeneous cluster, peers are running into
rejected state because we have different cksum computation
mechansims in upgraded and non-upgraded nodes.

Solution: add a check for op-version so that all the nodes
in the cluster follow the same mechanism for computing the
cksum.

fixes: bz#1684029

> Change-Id: I1508f000e8c9895588b6011b8b6cc0eda7102193
> BUG: bz#1685120
> Signed-off-by: Sanju Rakonde <srakonde@redhat.com>
> (cherry picked from commit 073444b693b7a91c42963512e0fdafb57ad46670)

Change-Id: I1508f000e8c9895588b6011b8b6cc0eda7102193
2019-03-08 14:45:41 +00:00
Amar Tumballi
9c441360ac dict: handle STR_OLD data type in xdr conversions
Currently a dict conversion on wire for 3.x protocol happens using
`dict_unserialize()`, which sets the type of data as STR_OLD. But the
new protocol doesn't send it over the wire as its not considered as a
valid format in new processes.

But considering we deal with old and new protocol when we do a rolling
upgrade, it will allow us to get all the information properly with new
protocol.

Credits: Krutika Dhananjay

Fixes: bz#1686364
Change-Id: I165c0021fb195b399790b9cf14a7416ae75ec84f
Signed-off-by: Amar Tumballi <amarts@redhat.com>
2019-03-08 14:08:40 +00:00
Raghavendra Gowdappa
ae24a2c390 performance/readdir-ahead: fix deadlock
This deadlock happens while processing dentry corresponding to current
directory (.) in rda_fill_readdirp. In this case following order is
followed:

LOCK(directory_fd_ctx->lock);
  rda_inode_ctx_get_iatt -> LOCK(directory_inode->lock);

However, in rda_mark_inode_dirty following lock order is followed:
LOCK(directory_inode->lock);
  LOCK(directory_fd_ctx->lock);

these two codepaths when executed concurrently resulted in a deadlock.

Current patch fixes this by removing locking directory inode and
fd-ctx in rda_fill_readdirp. This is fine as directory inode's stat
won't change due to writes to files within directory.

Change-Id: Ic93a67a0dac8229bb0d79582e526a512e6f2569c
Signed-off-by: Raghavendra Gowdappa <rgowdapp@redhat.com>
Fixes: bz#1686399
2019-03-08 14:08:29 +00:00
Susant Palai
8444c1ea54 io-threads: Prioritize fops with NO_ROOT_SQUASH pid
There was 30% regression observed in mkdir path with commit
b139bc58eb504adf5ef81658896c9283ae21f390. On analysis it is found
that io-threads xlator deprioritzes fops with all -ve pid.

Some context in to the no-root-squash pid requirement:
DHT xlator does some of the internal fops with root privileges. This is
needed so that operations like layout healing should not be abandoned
because a non root user is operating.  If root-squash option is enabled
the layout set operation looses its root privilege as server xlator
converts the uid and pid to random numbers. Hence, the above mentioned
commit converted pid to GF_CLIENT_PID_NO_ROOT_SQUASH to continue fops
as root.

Combining the above I am proposing not to deprioritize fops with
no-root-squash pid.

> Change-Id: I54d056c01b25729304a77f9242fbaff39c5672ba
> fixes: bz#1676430
> Signed-off-by: Susant Palai <spalai@redhat.com>
(cherry picked from commit f5c3b1727f55ffaa3dcdb3c3a09b968ebb45dbb2)

Change-Id: I54d056c01b25729304a77f9242fbaff39c5672ba
fixes: bz#1676429
Signed-off-by: Susant Palai <spalai@redhat.com>
2019-03-06 03:16:53 +00:00
hujianfei
ea78c029a7 glusterfsd: Do not process PROFILE_NFS_INFO if graph is not ready
Otherwise, gnfs will crash in following situation.
Also see commit 2f9e555f.

Reproducible Steps:
1. kill gnfs process
2. service glusterd restart;gluster volume profile [vol] info nfs

dump trace info:
/lib64/libglusterfs.so.0(_gf_msg_backtrace_nomem+0xc2)[0x7fcf5cb6a872]
/lib64/libglusterfs.so.0(gf_print_trace+0x324)[0x7fcf5cb743a4]
/lib64/libc.so.6(+0x35670)[0x7fcf5b1d5670]
/usr/sbin/glusterfs(glusterfs_handle_nfs_profile+0x114)[0x7fcf5d066474]
/lib64/libglusterfs.so.0(synctask_wrap+0x12)[0x7fcf5cba1502]
/lib64/libc.so.6(+0x47110)[0x7fcf5b1e7110]

Fixes: bz#1684777

Change-Id: Id68edb3e4646c39544e0b4c90b5e0a9083b37b0d
Signed-off-by: hujianfei <hujianfei@cmss.chinamobile.com>
(cherry picked from commit 09394856e1197db4933e7e814589a3340eb739cd)
2019-03-04 03:38:08 +00:00
Kaleb S. KEITHLEY
06a65016f1 glusterfind: revert shebangs to #!/usr/bin/python3
Using #!/usr/bin/env python is illegal in Fedora and RHEL packaging.
(Debian and SUSE packaging will complain about it too but don't
enforce it like Fedora packaging does. Yet.)

We went through a giant exercise fixing these once already.

The python needs to be python2/python3 clean. There is a build
scriptlet that converts the shebangs back to /usr/bin/python on rhel7.

Change-Id: If208a2557f7e5d727cda98c27905e249cf485d5b
fixes: bz#1683716
Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
2019-03-02 11:54:35 +00:00
Milind Changire
4cb1d6d94a socket: socket event handlers now return void
Problem:
Returning any value from socket event handlers to the event sub-system
doesn't make sense since event sub-system cannot handle socket
sub-system errors.

Solution:
Change return type of all socket event handlers to 'void'

mainline:
> Change-Id: I70dc2c57f12b7ea2fae41120f71aa0d7fe0b2b6f
> Fixes: bz#1651246
> Signed-off-by: Milind Changire <mchangir@redhat.com>
> Reviewed-on: https://review.gluster.org/c/glusterfs/+/22221

Change-Id: I70dc2c57f12b7ea2fae41120f71aa0d7fe0b2b6f
Fixes: bz#1683900
Signed-off-by: Milind Changire <mchangir@redhat.com>
(cherry picked from commit 776ba851c6ee6c265253d44cf1d6e4e3d4a21772)
2019-03-02 11:54:24 +00:00
Arjun Sharma
cd6795c5e8 Updating the glfs_setattr and glfs_fsetattr public APIs to use glfs_stat
Change-Id: I0e86c8222c88b4c71087ec287ba81f8353d70822
updates: #389
Signed-off-by: Arjun <arjsharm@redhat.com>
2019-02-28 12:00:30 +05:30
Sanju Rakonde
73b8d6c8d1 glusterd: remove experimental xlator options from glusterd-volume-set.c
experimental xlators have been removed from the codebase. But we
missed to remove the options related to experimental xlators from
the codebase. This patch removes those options.

fixes: bz#1683506
Change-Id: I3fa7e14c6cd8ebde5cebc8d2b0cb2409bf37c1ae
Signed-off-by: Sanju Rakonde <srakonde@redhat.com>
(cherry picked from commit 5cddd4d758014fe116d9c130632eada2ecded88c)
2019-02-27 03:39:55 +00:00
Aravinda VK
77f327862a eventsapi: Fix Python3 compatibility issues
- Fixed Relative import and non-package import related issues.
- socketserver import issues fix
- Renamed installed directory name to `gfevents` from `events`(To
  avoid any issues with other global libs)

Fixes: bz#1683008
Change-Id: I3dc38bc92b23387a6dfbcc0ab8283178235bf756
Signed-off-by: Aravinda VK <avishwan@redhat.com>
(cherry picked from commit cd68f7b88b9a2c9a4e4ff9fca61517384e54130a)
2019-02-26 14:11:41 +00:00
Niels de Vos
3d7cd3d0f5 build: do not install service related files when building --without-server
It seems that glusterfs-6 disables building the server (./configure
--without-server) on el6 and hence the installed but unpackaged files
should not get installed in the first place.

Backport of commit 30f358bb6b69b8c31da550608a4996a84e8b420b:
> Change-Id: I7bea08202dd59ac309de8bc861da920ed3600bc6
> Fixes: bz#1680587
> Signed-off-by: Niels de Vos <ndevos@redhat.com>

Change-Id: I7bea08202dd59ac309de8bc861da920ed3600bc6
Fixes: bz#1680586
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2019-02-26 08:38:53 +00:00
Amar Tumballi
0419d9ab93 fuse: reflect the actual default for lru-limit option
in both `--help` text and man page

updates: bz#1679998
Change-Id: I9aa9367c6863ac8e2403255280697c9e6be26cf0
Signed-off-by: Amar Tumballi <amarts@redhat.com>
2019-02-25 15:26:08 +00:00
David Spisla
c360210685 md-cache: Adapt integer data types to avoid integer overflow
The "struct iatt" in iatt.h is using int64_t types for storing
the atime, mtime and ctime. Therefore the struct 'struct md_cache' in
md-cache.c should also use this types to avoid an integer overflow.

This can happen e.g. if someone uses a very high default-retention-period
in the WORM-Xlator.

Change-Id: I605268d300ab622b9c8ab30e459dc00d9340aad1
fixes: bz#1680020
Signed-off-by: David Spisla <david.spisla@iternity.com>
(cherry picked from commit 15423e14f16dd1a15ee5e5cbbdbdd370e57ed59f)
2019-02-25 15:25:57 +00:00
Niels de Vos
fce763a6fc build: include gluster/__init__.* files for python packaging
The gluster/__init__.py file was originaly part of the glupy
installation. This file is required to have the python-gluster package
function correctly, it is expected to provide the 'gluster' namespace
for other python packages (like from the libgfapi-python project).

Because glupy does not exist anymore, this file is now added to in a new
extras/python directory.

Backport of commit b235a56394c09bb1d7308a620053dbe098d15967:
> Change-Id: I14fe959778ee3344d7d54ba342c7928a4d8080a2
> Fixes: c3fcff9ccbfcec1be242fd5cf210c9995586b078
> Fixes: 8293d21280fd6ddfc9bb54068cf87794fc6be207
> Updates: bz#1642810
> Signed-off-by: Niels de Vos <ndevos@redhat.com>

Change-Id: I04e1e0dc4022b53c02053d5452f305b78e56c389
Updates: bz#1680585
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2019-02-25 11:16:43 +00:00
Sahina Bose
5801a0e30e packaging: Obsoleting glusterfs-gnfs for upgrade
Master branch reference
>link : https://review.gluster.org/#/c/glusterfs/+/22190/
>fixes: bz#1672711
>Change-Id: Iad7194e788a8eeecd617614e9f8a1fe3264a384d
>Signed-off-by: Sahina Bose <sabose@redhat.com>
>Signed-off-by: Jiffin Tony Thottan <jthottan@redhat.com>
>(cherry picked from commit d3608bd26cbc3e47fc0e60e85bd9f66515e050b1)

fixes: bz#1679965
Change-Id: Iad7194e788a8eeecd617614e9f8a1fe3264a384d
Signed-off-by: Jiffin Tony Thottan <jthottan@redhat.com>
2019-02-22 11:23:49 +00:00
ShyamsundarR
e6fabf8c33 doc: Added initial release notes for release-6
Change-Id: I31fff344045344c501536b207c864e90ed34bf20
updates: bz#1672818
Signed-off-by: ShyamsundarR <srangana@redhat.com>
2019-02-22 04:12:45 +00:00
N Balachandran
951abf5c53 cluster/dht: Request linkto xattrs in dht_rmdir opendir
If parallel-readdir is enabled, the rda xlator is loaded
below dht in the graph and proactively lists and caches
entries when an opendir is performed. dht_rmdir checks if
the directory being deleted contains stale linkto files by
performing a readdirp on its child subvols. However, as
the entries are actually read in during the opendir operation
which does not request the linkto xattr,no linkto xattrs are
present for the entries causing dht to incorrectly identify
them as data files and fail the rmdir operation with ENOTEMPTY.
DHT now always adds the linkto xattr in the list of xattrs
requested in the opendir.

Change-Id: I0711198e66c59146282eb8b88084170bedfb4018
fixes: bz#1679004
Signed-off-by: N Balachandran <nbalacha@redhat.com>
2019-02-22 03:34:36 +00:00
Kinglong Mee
5c9b2dfb4d dht: fix double extra unref of inode at heal path
The loc_wipe is done in the _out_ section, inode_unref(loc.parent) here
casues a double extra unref of loc.parent.

> Change-Id: I2dc809328d3d34bf7b02c7df9a4f97788af511e6
> Signed-off-by: Kinglong Mee <mijinlong@open-fs.com>
(cherry-pick of https://review.gluster.org/#/c/glusterfs/+/21998/)

Change-Id: I2dc809328d3d34bf7b02c7df9a4f97788af511e6
updates: bz#1679275
Signed-off-by: Susant Palai <spalai@redhat.com>
2019-02-22 03:34:06 +00:00