1586 Commits

Author SHA1 Message Date
Mohit Agrawal
430484c92a glusterfs: Use gcc builtin ATOMIC operator to increase/decreate refcount.
Problem: In glusterfs code base we call mutex_lock/unlock to take
         reference/dereference for a object.Sometime it could be
         reason for lock contention also.

Solution: There is no need to use mutex to increase/decrease ref
          counter, instead of using mutex use gcc builtin ATOMIC
          operation.

Test:   I have not observed yet how much performance gain after apply
        this patch specific to glusterfs but i have tested same
        with below small program(mutex and atomic both) and
        get good difference.

static int numOuterLoops;
static void *
threadFunc(void *arg)
{
    int j;

    for (j = 0; j < numOuterLoops; j++) {
            __atomic_add_fetch (&glob, 1,__ATOMIC_ACQ_REL);
    }
    return NULL;
}

int
main(int argc, char *argv[])
{
    int opt, s, j;
    int numThreads;
    pthread_t *thread;
    int verbose;
    int64_t n = 0;

    if (argc < 2 ) {
     printf(" Please provide 2 args Num of threads && Outer Loop\n");
     exit (-1);
    }
    numThreads = atoi(argv[1]);
    numOuterLoops = atoi (argv[2]);

    if (1) {
        printf("\tthreads: %d; outer loops: %d;\n",
                numThreads, numOuterLoops);
    }

    thread = calloc(numThreads, sizeof(pthread_t));
    if (thread == NULL) {
        printf ("calloc error so exit\n");
        exit (-1);
    }

    __atomic_store (&glob, &n, __ATOMIC_RELEASE);
    for (j = 0; j < numThreads; j++) {
        s = pthread_create(&thread[j], NULL, threadFunc, NULL);
        if (s != 0) {
            printf ("pthread_create failed so exit\n");
            exit (-1);
        }
    }

    for (j = 0; j < numThreads; j++) {
        s = pthread_join(thread[j], NULL);
        if (s != 0) {
            printf ("pthread_join failed so exit\n");
            exit (-1);
        }
    }
    printf("glob value is %ld\n",__atomic_load_n (&glob,__ATOMIC_RELAXED));

    exit(0);
}

   time ./thr_count 800 800000
   threads: 800; outer loops: 800000;
   glob value is 640000000

real	1m10.288s
user	0m57.269s
sys	3m31.565s

time ./thr_count_atomic 800 800000
     threads: 800; outer loops: 800000;
glob value is 640000000

real	0m20.313s
user	1m20.558s
sys	0m0.028

Change-Id: Ie5030a52ea264875e002e108dd4b207b15ab7cc7
Signed-off-by: Mohit Agrawal <moagrawa@redhat.com>
2017-12-12 09:05:56 +00:00
N Balachandran
47d01546a1 core/memacct: save allocs in mem_acct_rec list
With configure --enable-debug, add all object allocations
to a list in the corresponding mem_acct_rec. This
allows us to see all objects of a particular type
and allows for additional debugging in case of memory
leaks.

This is not compiled in by default and must be explicitly
enabled. It is intended to be used by developers.

Change-Id: I7cf2dbeadecf994423d7e7591e85f18d2575cce8
BUG: 1522662
Signed-off-by: N Balachandran <nbalacha@redhat.com>
2017-12-07 14:57:27 +05:30
Richard Wareing
47d7e7441f Fixes gNFSd gf_update_latency crashes
Summary:
- Per title, does a bounds check on the frame->op and bails from the
  function if it's invalid preventing the crash

Test Plan: Prove tests

Reviewers: dph, jackl

Reviewed By: jackl

FB-commit-id: e67cc15

Change-Id: If1a5a9c0630573d4a6615050a9114ccf532551c7
BUG: 1522847
Signed-off-by: Kevin Vigor <kvigor@fb.com>
Reviewed-on: https://review.gluster.org/16847
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
Smoke: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Shreyas Siravara <sshreyas@fb.com>
2017-12-07 03:44:08 +00:00
Max Rijevski
b5184d6e75 performance/io-threads: Reduce the number of timing calls in iot_worker
Summary:
- Reduce the amount of unnecessary timing calls
in iot_worker servicing.
- The current logic is unnecessarily accurate and
hurts performance for many small FOPS.

Change-Id: I6db4f1ad9a48d9d474bb251a2204969061021954
BUG: 1522950
Signed-off-by: Shreyas Siravara <sshreyas@fb.com>
Reviewed-on: http://review.gluster.org/16081
Smoke: Gluster Build System <jenkins@build.gluster.org>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Kevin Vigor <kvigor@fb.com>
2017-12-07 02:41:24 +00:00
Zhang Huan
28202631fe libglusterfs: specify ctx in gf_log_set_loglevel
specify ctx in gf_log_set_loglevel, instead of getting it from a thread
specific variable.

Change-Id: I498f826e8e32231235a6b0005026a27c327727fd
BUG: 1521213
Signed-off-by: Zhang Huan <zhanghuan@open-fs.com>
2017-12-06 23:45:23 +00:00
Amar Tumballi
7f2e67d40d metrics: provide options to dump metrics from xlators
* Introduce xlator methods to allow dumping of metrics
* Separate options to get the metrics dumped in a path

Updates #168

Change-Id: I7df80df33b71d6f449f03c2332665b4a45f6ddf2
Signed-off-by: Amar Tumballi <amarts@redhat.com>
2017-12-06 19:59:22 +00:00
Susant Palai
efad782603 rio/everywhere: add icreate/namelink fop
icreate creates inode, while namelink links the basename to it's
parent gfid.

For now mkdir is the primary user of these fops. Better distribution is
acheived by creating the inode on ,(say) mds1 and linking the basename to it's
parent gfid on mds2. The inode serves readdirp, stat etc.

More details about the fops are present at:
https://review.gluster.org/#/c/13395/3/design/DHT2/DHT2_Icreate_Namelink_Notes.md

This backport of three patches from experimental branch.
1- https://review.gluster.org/#/c/18085/
2- https://review.gluster.org/#/c/18086/
3- https://review.gluster.org/#/c/18094/

Updates gluster/glusterfs#243
Change-Id: I1bd3d5a441a3cfab1acfeb52f15c6c867d362592
Signed-off-by: Susant Palai <spalai@redhat.com>
2017-12-05 21:23:57 +00:00
Poornima G
dc1258bfe4 libglusterfs: Add put fop
Problem: It had been a longtime request to implement put fop
in gluster. put fop in gluster may not have the exact sementics
of HTTP PUT, but can be easily extended to do so. The subsequent
patches, will contain more semantics on the put fop and its
guarentees.

Why compound fop framework is not used for put?
Compound fop framework currently doesn't allow compounding of
entry fop and inode fops, i.e. fops on multiple inodes cannot be
combined in compound fop.

Updates #353
Change-Id: Idb7891b3e056d46d570bb7e31bad1b6a28656ada
Signed-off-by: Poornima G <pgurusid@redhat.com>
2017-12-05 14:21:01 +00:00
Zhang Huan
aba2b775bd libglusterfs: use rwlock for fdtable
To resolve a fd from client requests, need to take a mutex lock for the
fdtable to do the lookup. When a client is busy doing read and write,
the mutex lock could introduce contention. Therefore, use rwlock instead
of mutex to reduce the contention.

Change-Id: Ic833aed738a178a7ea1abafed7eb13814989d28c
BUG: 1518582
Signed-off-by: Zhang Huan <zhanghuan@open-fs.com>
2017-11-30 12:34:25 +00:00
Amar Tumballi
5b4b25c697 xlator: provide a xlator_api_t structure to include all exported options
each translator from now on can have just 1 symbol exported called
'xlator_api', which has all the required fields in it.

Updates: #164

Change-Id: I48d54f5ec59fee842b1d55877e3ac5e9ec9b6bdd
Signed-off-by: Amar Tumballi <amarts@redhat.com>
2017-11-30 09:16:13 +00:00
Pranith Kumar K
9af6fb54d8 cluster-syncop: Address comments in 3ad68df725ac32f83b5ea7c0976e2327e7037c8c
Change-Id: I325f718c6c440076c9d9dcd5ad1a0c6bde5393b1
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
2017-11-29 09:17:19 +00:00
Csaba Henk
f3a8953e6a libglusterfs: fix the call_stack_set_group() function
- call_stack_set_group() will take the ownership of passed
  buffer from caller;
- to indicate the change, its signature is changed from
  including the buffer directly to take a pointer to it;
- either the content of the buffer is copied to the
  groups_small embedded buffer of the call stack, or
  the buffer is set as groups_large member of the call
  stack;
- the groups member of the call stack is set to,
  respectively, groups_small or groups_large, according
  to the memory management conventions of the call stack;
- the buffer address is overwritten with junk to effectively
  prevent the caller from using it further on.

Also move call_stack_set_group to stack.c from stack.h
to prevent "defined but not used [-Wunused-function]"
warnings (not using it anymore in call_stack_alloc_group()
implementation, which saved us from this so far).

protocol/server: refactor gid_resolve()

In gid_resolve there are two cases:
either the gid_cache_lookup() call returns
a value or not. The result is caputured in
the agl variable, and throughout the function,
each particular stage of the implementation
comes with an agl and a no-agl variant.

In most cases this is explicitly indicated
via an

   if (agl) {
      ...
   } else {
      ...
   }

but some of this branching are expressed via
goto constructs (obfuscating the fact we stated
above, that is, each particular stage having
an agl/no-agl variant).

In the current refactor, we bring the agl
conditional to the top, and present the
agl/non-agl implementations sequentially.

Also we take the opportunity to clean up and
fix the agl case:
- remove the spurious
    gl.gl_list = agl->gl_list;
  setting, as gl is not used in the agl caae
- populate the group list of call stack from
  agl, fixing thus referred BUG.

Also fixes BUG: 1513920

Change-Id: I61f4574ba21969f7661b9ff0c9dce202b874025d
BUG: 1513928
Signed-off-by: Csaba Henk <csaba@redhat.com>
2017-11-24 16:54:44 +00:00
Pranith Kumar K
038d8f994d libglusterfs: Handle FS errors gracefully
Problem:
FS sometimes doesn't give the expected return values. We need our common
functions to guard against this.
Example BUG: https://bugzilla.redhat.com/show_bug.cgi?id=864401

Fix:
When the return value is not as per specification, change the return value
to -1 and errno to EIO

BUG: 1469487
Change-Id: I14739ab2e5ae225b1a91438b87f8928af56f2934
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
2017-11-22 10:07:36 +00:00
Pranith Kumar K
3ad68df725 cluster-syncop: Implement tiebreaker inodelk/entrylk
In this implementation, inodelk/entrylk will be tried for the subvols
given with trylock. In this attempt if all locks are obtained, then
inodelk is successful, otherwise, if it gets success on the first
available subvolume, then it will go for blocking lock, where as other
subvolumes will not try and this acts as tie-breaker.

Updates gluster/glusterfs#354
Change-Id: Ia2521b9ccb81a42bd6104ab21f610f761ba2b801
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
2017-11-22 07:47:49 +00:00
Soumya Koduri
7100c0ec02 timer: Fix possible race during cleanup
As mentioned in bug1509189, there is a possible race
between gf_timer_cancel(), gf_timer_proc() and
gf_timer_registry_destroy() leading to use_after_free.

Problem:

1) gf_timer_proc() is called, locks reg, and gets an event.
It unlocks reg, and calls the callback.

2) Meanwhile gf_timer_registry_destroy() is called, and removes
reg from ctx, and joins on gf_timer_proc().

3) gf_timer_call_cancel() is called on the event being
processed.  It cannot find reg (since it's been removed from reg),
so it frees event.

4) the callback returns into gf_timer_proc(), and it tries to free
event, but it's already free, so double free.

Solution:
The fix is to bail out in gf_timer_cancel() when registry
is not found. The logic behind this is that, gf_timer_cancel()
is called only on any existing event. That means there was a valid
registry earlier while creating that event. And the only reason
we cannot find that registry now is that it must have got set to
NULL when context cleanup is started.
Since gf_timer_proc() takes care of releasing all the remaining
events active on that registry, it seems safe to bail out
in gf_timer_cancel().

Change-Id: Ia9b088533141c3bb335eff2fe06b52d1575bb34f
BUG: 1509189
Reported-by: Daniel Gryniewicz <dang@redhat.com>
Signed-off-by: Soumya Koduri <skoduri@redhat.com>
2017-11-21 08:56:24 +00:00
Subha sree Mohankumar
c6e8738797 libglusterfs:checked return coverity fix
Problem:Calling "gf_thread_create" without checking return value.

Fix:The return value is saved and checked if gf_thread_create fails.

Change-Id: Ibdaac1c90a1a8369e92ade50825598b041063da8
BUG: 789278
Signed-off-by: Subha sree Mohankumar <smohanku@redhat.com>
2017-11-21 06:34:50 +00:00
Poornima G
c810ea1e2b rpc : Change the way client uuid is built
Problem:
Today the main users of client uuid are protocol layers, locks, leases.
Protocol layers requires each client uuid to be unique, even across
connects and disconnects. Locks and leases on the server side also use
the same client uid which changes across file migrations. Which makes the graph
switch and file migration tedious for locks and leases.

file migration across bricks becomes difficult as client uuid for the same
client, is different on the other brick.

The exact set of issues exists for leases as well.

Solution would be to introduce a constant in the client-uid string which
the locks and leases can use to identify the owner client across bricks.

Client uuid currently:
%s(ctx uuid)-%s(protocol client name)-%d(graph id)%s(setvolume count/reconnect count)

Proposed Client uuid:
"CTX_ID:%s-GRAPH_ID:%d-PID:%d-HOST:%s-PC_NAME:%s-RECON_NO:%s"
-  CTX_ID: This is will be constant per client.
-  GRAPH_ID, PID, HOST, PC_NAME(protocol client name), RECON_NO(setvolume count)
remains the same.

Change-Id: Ia81d57a9693207cd325d7b26aee4593fcbd6482c
BUG: 1369028
Signed-off-by: Susant Palai <spalai@redhat.com>
2017-11-20 08:20:35 +00:00
Mohit Agrawal
959f5267fa dict: Fix several coverity issues in dict
This patch fixes issues 230,592,593,110,63 from [1]

[1] https://download.gluster.org/pub/gluster/glusterfs/static-analysis/master/glusterfs-coverity/2017-10-30-9aa574a5/html/

Note: Resolve FORWARD_NULL coverity issue in glusterfs_ctx_new is
      also fixed with this patch.

BUG: 789278
Change-Id: Ic4199a144a14cc9ead7366fb1c9699197141bc86
Signed-off-by: Mohit Agrawal <moagrawa@redhat.com>
2017-11-15 14:37:31 +00:00
Kaushal M
306344a1cc libglusterfs/options: Add new member 'setkey' to xlator options
The 'setkey' will be used as the key by GD2 when setting the option
during volgen. 'setkey' also supports using varstrings.

This is mainly to be used for options, which use a different key for
'volume set' and in volfiles. For eg. the 'auth.*' options of
protocol/server.

The protocol/server xlator has been updated to make use of this for the
auth.allow and auth.reject options.

Updates #302

Change-Id: I1fd2fd69625c9db48595bd3f494c221625255169
Signed-off-by: Amar Tumballi <amarts@redhat.com>
2017-11-15 10:01:34 +00:00
Xavier Hernandez
3f8d118e48 libglusterfs/atomic: Improved atomic support
This patch solves a detection problem in configure.ac that prevented
that compilation detects builtin __atomic or __sync functions.

It also adds more atomic types and support for other atomic functions.

An special case has been added to support 64-bit atomics on 32-bit
systems. The solution is to fallback to the mutex solution only for
64-bit atomics, but smaller atomic types will still take advantage
of builtins if available.

Change-Id: I6b9afc7cd6e66b28a33278715583552872278801
BUG: 1510397
Signed-off-by: Xavier Hernandez <jahernan@redhat.com>
2017-11-14 05:22:00 +00:00
Subha sree Mohankumar
f6ab03d27c libglusterfs/gfdb:coverity issue fix
Coverity Id:719,748,761

problem : The value of "ret" is overwritten in init_db and
add_conection_node

Change-Id: Iade8ca8d61c5e25e8c311b1375219f5f61d51bc3
BUG: 789278
Signed-off-by: Subha sree Mohankumar <smohanku@redhat.com>
2017-11-13 16:34:39 +00:00
Subha sree Mohankumar
6343b16f53 libglusterfs:UNUSED VALUE coverity fix
Problem : Overwriting previous write to "ret" with value "EVENT_SEND_OK"
before itr can be used.

Fix: The value of ret is used in out.

Change-Id: I2cdb32e441c85c94de30de89a7a4121fd54d1acd
BUG: 789278
Signed-off-by: Subha sree Mohankumar <smohanku@redhat.com>
2017-11-13 16:34:39 +00:00
Girjesh Rajoria
0821a57bd2 Coverity Issue: PW.INCLUDE_RECURSION in several files
Coverity ID: 407, 408, 409, 410, 411, 412, 413, 414, 415, 416, 417,
418, 419, 423, 424, 425, 426, 427, 428, 429, 436, 437, 438, 439,
440, 441, 442, 443

Issue: Event include_recursion

Removed redundant, recursive includes from the files.

Change-Id: I920776b1fa089a2d4917ca722d0075a9239911a7
BUG: 789278
Signed-off-by: Girjesh Rajoria <grajoria@redhat.com>
2017-11-09 13:21:11 +00:00
Amar Tumballi
b79f296941 xlator: add more metrics per fops
Make sure to handle these counters in STACK_WIND/UNWIND macro, and
keep the counters as part of xlator_t structure itself, to provide
infra to monitoring.

Updates #137

Change-Id: Ib54d45e2321c2b095dac5810c37e6cdffe1f71b7
Signed-off-by: Amar Tumballi <amarts@redhat.com>
2017-11-08 08:33:22 +00:00
Jeff Darcy
54bf1a9d46 libglusterfs: fix (some) silliness in timer.c
This addresses two issues. First is that we currently keep
traversing the timer list even when it's no longer possible for
us to find a timer that's ready to fire. Second is that we sleep
a full second even when the next timer is due sooner.

Change-Id: I178a460d0176dbb45f972c62ee94c6df66d92ca5
Signed-off-by: Jeff Darcy <jdarcy@fb.com>
2017-11-07 20:18:43 +00:00
Amar Tumballi
125fc934e7 stack: change gettimeofday() to clock_gettime()
For achieving the above, needed below changes too.

* more sanity into how 'frame->op' is assigned.
* infra to have 'stats' as separate section in 'xlator_t' structure

Updates #137

Change-Id: I36679bf9577f3ed00a695b4e7d92870dcb3db8e1
Signed-off-by: Amar Tumballi <amarts@redhat.com>
2017-11-06 14:23:49 +00:00
Xavier Hernandez
0dcd5b2fee cluster/ec: create eager-lock option for non-regular files
A new option is added to allow independent configuration of eager
locking for regular files and non-regular files.

Change-Id: I8f80e46d36d8551011132b15c0fac549b7fb1c60
BUG: 1502610
Signed-off-by: Xavier Hernandez <jahernan@redhat.com>
2017-11-05 03:41:46 +00:00
Amar Tumballi
bf5c5841c4 global options: add a sample option to handle
Fixes #303

Change-Id: Icdaa804711c43c65b9684f2649437aae1b5c1ed5
Signed-off-by: Amar Tumballi <amarts@redhat.com>
2017-11-03 14:40:52 +05:30
Amar Tumballi
cf62283467 Add framework for global xlator in graph
Updates #303

Change-Id: Id0b9050c93ea87532dc80b4fda650c5663d285bd
Signed-off-by: Amar Tumballi <amarts@redhat.com>
2017-11-03 07:26:41 +00:00
Jeff Darcy
8e973d3ab9 core: make gf_boolean_t a C99 bool instead of an enum
This reduces the space used from four bytes to one, and allows
new code to use familiar C99 types/values interoperably with our
old cruft. It does *not* change current declarations or code;
that will be left for a separate - much larger - patch.

Updates: #80

Change-Id: I5baedd17d3fb05b38f0d8b8bb9dd62824475842e
Signed-off-by: Jeff Darcy <jdarcy@fb.com>
2017-11-03 05:04:11 +00:00
Kevin Vigor
1260ee53b1 gluster: IPv6 single stack support
Summary:
- This diff changes all locations in the code to prefer inet6 family
  instead of inet.  This will allow change GlusterFS to operate
  via IPv6 instead of IPv4 for all internal operations while still
  being able to serve (FUSE or NFS) clients via IPv4.
- The changes apply to NFS as well.
- This diff ports D1892990, D1897341 & D1896522 to the 3.8 branch.

Test Plan: Prove tests!

Reviewers: dph, rwareing

Signed-off-by: Shreyas Siravara <sshreyas@fb.com>

Change-Id: I34fdaaeb33c194782255625e00616faf75d60c33
BUG: 1406898
Reviewed-on-3.8-fb: http://review.gluster.org/16059
Reviewed-by: Shreyas Siravara <sshreyas@fb.com>
Tested-by: Shreyas Siravara <sshreyas@fb.com>
2017-10-24 09:28:08 +00:00
Poornima G
77271e9c1e gfproxyd: Let glusterd manage gfproxy daemon
Updates: #242
BUG: 1428063
Change-Id: Iaaf2edf99b2ecc75f6d30762c752a6d445c1c826
Signed-off-by: Poornima G <pgurusid@redhat.com>
2017-10-18 14:15:05 +00:00
Amar Tumballi
e848410ab6 stack.h: make sure 'oldgroups' is never null
If 'oldgroups' is null, it can lead to a crash.

Change-Id: I7a4f71256b07dc0ca3b988cad3694c21b150c99e
BUG: 789278
Signed-off-by: Amar Tumballi <amarts@redhat.com>
2017-10-18 13:55:24 +00:00
Gaurav Yadav
614904fa7a glusterd : introduce timer in mgmt_v3_lock
Problem:
In a multinode environment, if two of the op-sm transactions
are initiated on one of the receiver nodes at the same time,
there might be a possibility that glusterd  may end up in
stale lock.

Solution:
During mgmt_v3_lock a registration is made to  gf_timer_call_after
which release the lock after certain period of time

Change-Id: I16cc2e5186a2e8a5e35eca2468b031811e093843
BUG: 1499004
Signed-off-by: Gaurav Yadav <gyadav@redhat.com>
2017-10-17 15:44:49 +00:00
Niels de Vos
a413c6353d build: make it possible to build cleanly 2x in a row
'make clean' does not cleanup everything, and some of the files get
cleaned too eagerly. Several files are being packaged in a 'make dist'
tarball, that get rebuild each time anyway.

Specifically, this change prevents
 - libglusterfs/src/generator.pyc from laying around
 - keeping rpc/xdr/gen/*.x symlinks
 - modifying tests/basic/{fuse,gfapi}/Makefile each run
 - including tests/env.rc and events/src/eventtypes.py in the tarball

Change-Id: I774dd1abf3a9d3b6a89b938cf6ee7d7792c59a82
BUG: 1501317
Reported-by: Patrick Matthäi <pmatthaei@debian.org>
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2017-10-13 16:28:55 +00:00
Amar Tumballi
a3767c4ceb protocol-auth: use the proper validation method
Currently, server protocol's init and glusterd's option
validation methods are different, causing an issue. They
should be same for having consistent behavior

Updates #175

Change-Id: Ibbf9a18c7192b2d77f9b7675ae7da9b8d2fe5de4
Signed-off-by: Amar Tumballi <amarts@redhat.com>
2017-10-12 10:36:26 +00:00
Ravishankar N
452b9124f4 glusterd : fix client io-threads option for replicate volumes
Problem:
Commit ff075a3d6f9b142911d25c27fd209838782bfff0 disabled loading
client-io-threads for replicate volumes (it was set to on by default in
commit e068c1997314046658dd502e9118dab32decf879) due to performance
issues but in doing so, inadvertently failed to load the xlator even if
the user explicitly enabled the option using the volume set command.
This was despite returning returning sucess for the volume set.

Fix:
Modify the check in perfxl_option_handler() and add checks in volume
create/add-brick/remove-brick code paths, tying it all to
GD_OP_VERSION_3_12_2.

Change-Id: Ib612973a999a7da818cc926f5c2601b1f0794fcf
BUG: 1498570
Signed-off-by: Ravishankar N <ravishankar@redhat.com>
2017-10-09 12:19:16 +05:30
Sunil Kumar Acharya
12cfc1983f cluster/ec: Improve performance with xattrop update
Existing EC code updates the xattr on the subvolume
in a sequential pattern resulting in very poor performance.

With this fix EC now updates the xattr on the subvolume
in parallel which improves the xattr update performance.

BUG: 1445663
Change-Id: I3fc40d66db0b88875ca96a9fa01002ba386c0486
Signed-off-by: Sunil Kumar Acharya <sheggodu@redhat.com>
2017-10-06 01:42:06 +00:00
karthik-us
601b6547f2 storage/posix: Adding implementation for posix_do_futimes
Adding the implementation for the posix_do_futimes function which is
not complete in the current implementation and giving the ENOSYS error.

Change-Id: I9cfc95a7ea293b0a2df8efd4ac80d0120b3120e4
BUG: 1350406
Signed-off-by: karthik-us <ksubrahm@redhat.com>
2017-10-05 12:57:35 +00:00
Mohammed Azhar Padariyakam
306164fb41 libglusterfs: Coverity Fix CONSTANT_EXPRESSION_RESULT in gf_string2int64
Issue : "l <= 9223372036854775807L" is always true regardless of the values
of its operands. This occurs as the logical operand of "if".

Solution : Remove the comparison which always turns out to be true

Fix : The if-condition was removed and the body inside the same was retained.

Change-Id: Iba94d7f4f2dee85a180d10cdb7f7235b406cc400
BUG: 789278
Signed-off-by: Mohammed Azhar Padariyakam <mpadariy@redhat.com>
2017-09-29 16:31:27 +00:00
Girjesh Rajoria
e994fbc825 libglusterfs: Coverity Issue NEGATIVE_RETURNS
Issue: Event negative_return_fn: Function "dup(handle->fd)" returns a
negative number. Event negative_returns: "dup(handle->fd)" is passed
to a parameter that cannot be negative.

With this change value of dup(handle->fd) is stored in duped_fd & if
condition checks the value of duped_fd is non-negative.

Change-Id: I563d717108016d740ffa64fbe0929eb1e08c8f33
BUG: 789278
Signed-off-by: Girjesh Rajoria <grajoria@redhat.com>
2017-09-29 12:37:26 +00:00
Akarsha Rai
9af20af096 core: Verify pool pointer before destroying it
Problem: Current code is not checking whether the pool pointer is null or not.

Solution: Updated the code to verify pool pointer.

Bug: 1496675
Change-Id: Ie1f2de4e4204fde15d2b1e3a966ea4c9e7b41534
Signed-off-by: Akarsha Rai <akrai@redhat.com>
2017-09-29 12:36:23 +00:00
Sanju Rakonde
2c124b5256 stack.h:PW.INCLUDE_RECURSION
The headerfile globals.h is recursively adding itself.
( globals.h -> xlator.h -> stack.h -> globals.h).
We are finding the source files which are including the header
file globals.h and removing the inclusion line.

I used git grep -l stack.h | xargs git grep globals.h --
to find out the files and removed the header file from all files
except libglusterfs/src/xlator.h and libglusterfs/src/Makefile.am

When I try to remove header file from libglusterfs/src/xlator.h
I'm getting some errors. In libglusterfs/src/Makefile.am it is
required for building RPMs.

Change-Id: I537218c09ade6d7ea51717768b26563a247daf60
BUG: 789278
Signed-off-by: Sanju Rakonde <srakonde@redhat.com>
2017-09-29 07:46:53 +00:00
Kamal Mohanan
77f89a2eba libglusterfs: fix a coverity error of type - PW.INCLUDE_RECURSION
Problem: xlator.h which is included at inode.h:32 includes itself.

Solution: Delete the include statement.

Change-Id: I304bbef5293a2fef0584773a1f8e61e838d7cbf1
BUG: 789278
Signed-off-by: Kamal Mohanan <kmohanan@redhat.com>
2017-09-27 13:29:33 +00:00
Krutika Dhananjay
956d43d6e8 mount/fuse: Make event-history feature configurable
... and disable it by default.

This is because having it disabled seems to improve performance.
This could be due to the lock contention by the different epoll threads
on the circular buff lock in the fop cbks just before writing their response
to /dev/fuse.

Just to provide some data - wrt ovirt-gluster hyperconverged
environment, I saw an increase in IOPs by 12K with event-history
disabled for randrom read workload.

Usage:
mount -t glusterfs -o event-history=on $HOSTNAME:$VOLNAME $MOUNTPOINT
OR
glusterfs --event-history=on --volfile-server=$HOSTNAME --volfile-id=$VOLNAME $MOUNTPOINT

Change-Id: Ia533788d309c78688a315dc8cd04d30fad9e9485
BUG: 1467614
Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com>
2017-09-24 06:22:07 +00:00
Anoop C S
db3fe245a9 libglusterfs: Fix openSUSE build error
GlusterFS failed during make on openSUSE Tumbleweed with the following
error:

Making all in fdl
Making all in src
  CC       logdump.o
  CC       recon.o
  CC       fdl.lo
  CC       librecon.o
  CC       libfdl.o
  CCLD     gf_logdump
  CCLD     gf_recon
/usr/lib64/gcc/x86_64-suse-linux/7/../../../../lib64/libfl.so: undefined reference to `yylex'
collect2: error: ld returned 1 exit status
make[5]: *** [Makefile:618: gf_logdump] Error 1
make[5]: *** Waiting for unfinished jobs....
/usr/lib64/gcc/x86_64-suse-linux/7/../../../../lib64/libfl.so: undefined reference to `yylex'
collect2: error: ld returned 1 exit status

Reading through autoconf manual[1](see AC_PROG_LEX) reveals that LEXLIB
is automatically set to appropriate value for the system. The reference
to LEXLIB in automake file caused the above mentioned error on openSUSE.
In particular, we do not bother about LEXLIB hereafter.

[1] https://www.gnu.org/software/autoconf/manual/autoconf.html#Particular-Programs

Change-Id: I9bfce80c9654b2e3bfb393b08c25e8ad3d79e449
BUG: 1493133
Signed-off-by: Anoop C S <anoopcs@redhat.com>
2017-09-21 10:22:37 +00:00
Csaba Henk
b290247741 libglusterfs: fix run.c demo mode
run.c can be compiled into a standalone object file
or a demo program with the appropriate defines for
development purposes. This functionality was broken.

We fix it and also clean it up:
- call it "demo mode" not "test mode" as tests should
  come with verification of the results of the invocation
  which is not being done
- add comments to the source explaining the feature
- provide more comprehensive output in the demo program

This change does not affect standard compilation and
usage of run.c functionality.

BUG: 1454590
Change-Id: I1ea618e3262bf6a4d9f79f6b59209438d5163244
Signed-off-by: Csaba Henk <csaba@redhat.com>
Reviewed-on: https://review.gluster.org/17363
Smoke: Gluster Build System <jenkins@build.gluster.org>
Tested-by: Amar Tumballi <amarts@redhat.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
2017-09-07 10:09:50 +00:00
Michael Scherer
6826905213 Fix warning on FreeBSD, show by -Wformat-extra-args
The warning is specific to FreeBSD, since both Linux and NetBSD
do trigger a different codepath.

Change-Id: I3d2b374b6a39804942af076d7e0d130a76f869a2
BUG: 1488808
Signed-off-by: Michael Scherer <misc@redhat.com>
Reviewed-on: https://review.gluster.org/18214
Smoke: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Tested-by: Niels de Vos <ndevos@redhat.com>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
2017-09-07 09:24:25 +00:00
Raghavendra G
b1b4999757 event/epoll: don't call handler for events received after a pollerr
we register socket with EPOLLONESHOT, which means it has to be
explicitly added back through epoll_ctl to receive more
events. Normally we do this once the handler completes processing of
current event. But event_select_on_epoll is one asynchronous codepath
where socket can be added back for polling while an event on the same
socket is being processed. event_select_on_epoll has a check whether
an event is being processed in the form of slot->in_handler. But this
check is not sufficient enough to prevent parallel events as
slot->in_handler is not atomically incremented with respect to
reception of the event. This means following imaginary sequence of
events can happen:

* epoll_wait returns with a POLLERR - say POLLERR1 - on a socket
  (sock1) associated with slot s1. socket_event_handle_pollerr is yet
  to be invoked.
* an event_select_on called from __socket_ioq_churn which was called
  in request/reply/msg submission codepath (as opposed to
  __socket_ioq_churn called as part of POLLOUT handling - we cannot
  receive a POLLOUT due to EPOLLONESHOT) adds back sock1 for polling.
* since sock1 was added back for polling in step 2 and our polling is
  level-triggered, another thread picks up another POLLERR event - say
  POLLERR2. socket_event_handler is invoked as part of processing
  POLLERR2 and it completes execution setting priv->sock to -1.
* event_unregister_epoll called as part of __socket_reset due to
  POLLERR1 would receive fd as -1 resulting in assert failure.

Also, since the first pollerr event has done rpc_transport_unref,
subsequent parallel events (not just pollerr, but other events too)
could be acting on a freed up transport too.

Change-Id: I5db755068e7890ec755b59f7a35a57da110339eb
BUG: 1486134
Signed-off-by: Raghavendra G <rgowdapp@redhat.com>
Reviewed-on: https://review.gluster.org/18129
Smoke: Gluster Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: mohammed rafi  kc <rkavunga@redhat.com>
2017-09-06 06:20:42 +00:00
Niels de Vos
3737ed53ca mempool: fix code when GF_DISABLE_MEMPOOL is defined
Problem: Run-time crash is observed when attempting to memset() a zero
length buffer.

Solution: When GF_DISABLE_MEMPOOL is set, mem_get() gets translated to a
GF_MALLOC(). The size of the allocation does not need to relate to the
available (but uninitialized) global memory pools. It is fine to
allocate the exact amount of memory that was configured when the
mem-pool was created.

Change-Id: Iea0bff974bb771623a34d7a940e10cb0db0f90e1
BUG: 1481199
Reported-by: Milind Changire <mchangir@redhat.com>
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: https://review.gluster.org/18034
Smoke: Gluster Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
2017-09-02 02:21:15 +00:00