837 Commits

Author SHA1 Message Date
ShyamsundarR
bfe2b5e153 clang: Fix various missing checks for empty list
When using list_for_each_entry(_safe) functions, care needs
to be taken that the list passed in are not empty, as these
functions are not empty list safe.

clag scan reported various points where this this pattern
could be caught, and this patch fixes the same.

Additionally the following changes are present in this patch,
- Added an explicit op_ret setting in error case in the
macro MAKE_INODE_HANDLE to address another clang issue reported
- Minor refactoring of some functions in quota code, to address
possible allocation failures in certain functions (which in turn
cause possible empty lists to be passed around)

Change-Id: I1e761a8d218708f714effb56fa643df2a3ea2cc7
Updates: bz#1622665
Signed-off-by: ShyamsundarR <srangana@redhat.com>
2018-12-14 04:33:15 +00:00
Raghavendra Bhat
7dadea15c5 copy_file_range support in GlusterFS
* libglusterfs changes to add new fop

    * Fuse changes:
      - Changes in fuse bridge xlator to receive and send responses

    * posix changes to perform the op on the backend filesystem

    * protocol and rpc changes for sending and receiving the fop

    * gfapi changes for performing the fop

    * tools: glfs-copy-file-range tool for testing copy_file_range fop

      - Although, copy_file_range support has been added to the upstream
	    fuse kernel module, no release has been made yet of a kernel
        which contains the support. It is expected to come in the
        upcoming release of linux-4.20

        So, as of now, executing copy_file_range fop on a fused based
        filesystem results in fuse kernel module sending read on the
	    source fd and write on the destination fd.

	    Therefore a small gfapi based tool has been written to be able
        test the copy_file_range fop. This tool is similar (in functionality)
	    to the example program given in copy_file_range man page.

	    So, running regular copy_file_range on a fuse mount point and
	    running gfapi based glfs-copy-file-range tool gives some idea about
	    how fast, the copy_file_range (or reflink) can be.

	    On the local machine this was the result obtained.

	    mount -t glusterfs workstation:new /mnt/glusterfs
	    [root@workstation ~]# cd /mnt/glusterfs/
	    [root@workstation glusterfs]# ls
	    file
	    [root@workstation glusterfs]# cd
	    [root@workstation ~]# time /tmp/a.out /mnt/glusterfs/file /mnt/glusterfs/new
	    real  0m6.495s
	    user  0m0.000s
	    sys   0m1.439s
	    [root@workstation ~]# time glfs-copy-file-range $(hostname) new /tmp/glfs.log /file /rrr
	    OPEN_SRC: opening /file is success
	    OPEN_DST: opening /rrr is success
	    FSTAT_SRC: fstat on /rrr is success
	    copy_file_range successful

        real  0m0.309s
        user  0m0.039s
        sys   0m0.017s

        This tool needs following arguments
         1) hostname
         2) volume name
         3) log file path
         4) source file path (relative to the gluster volume root)
         5) destination file path (relative to the gluster volume root)

        "glfs-copy-file-range <hostname> <volume> <log file path> <source> <destination>"

      - Added a testcase as well to run glfs-copy-file-range tool

    * io-stats changes to capture the fop for profiling

    * NOTE:

      - Added conditional check to see whether the copy_file_range syscall
        is available or not. If not, then return ENOSYS.

      - Added conditional check for kernel minor version in fuse_kernel.h
        and fuse-bridge while referring to copy_file_range. And the kernel
        minor version is kept as it is. i.e. 24. Increment it in future
        when there is a kernel release which contains the support for
        copy_file_range fop in fuse kernel module.

    * The document which contains a writeup on this enhancement can be found at
      https://docs.google.com/document/d/1BSILbXr_knynNwxSyyu503JoTz5QFM_4suNIh2WwrSc/edit

Change-Id: I280069c814dd21ce6ec3be00a884fc24ab692367
updates: #536
Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com>
2018-12-12 15:56:55 +00:00
Amar Tumballi
340e58f9b3 all: add xlator_api to many translators
Fixes: #164
Change-Id: I93ad6f0232a1dc534df099059f69951e1339086f
Signed-off-by: Amar Tumballi <amarts@redhat.com>
2018-12-06 07:54:28 +00:00
ShyamsundarR
20ef211cfa libglusterfs: Move devel headers under glusterfs directory
libglusterfs devel package headers are referenced in code using
include semantics for a program, this while it works can be better
especially when dealing with out of tree xlator builds or in
general out of tree devel package usage.

Towards this, the following changes are done,
- moved all devel headers under a glusterfs directory
- Included these headers using system header notation <> in all
code outside of libglusterfs
- Included these headers using own program notation "" within
libglusterfs

This change although big, is just moving around the headers and
making it correct when including these headers from other sources.

This helps us correctly include libglusterfs includes without
namespace conflicts.

Change-Id: Id2a98854e671a7ee5d73be44da5ba1a74252423b
Updates: bz#1193929
Signed-off-by: ShyamsundarR <srangana@redhat.com>
2018-12-05 21:47:04 +00:00
Xie Changlong
ad446dabb8 protocol/server: support server.all-squash
We still use gnfs on our side, so do a little work to support
server.all-squash. Just like server.root-squash, it's also a
volume wide option. Also see bz#1285126

$ gluster volume set <VOLNAME> server.all-squash on

Note: If you enable server.root-squash and server.all-squash
at the same time, only server.all-squash works. Please refer
to following table

+---------------+-----------------+---------------------------+
|               |all_squash       | no_all_squash             |
+-------------------------------------------------------------+
|               |                 |anonuid/anongid for root   |
|root_squash    |anonuid/anongid  |useruid/usergid for no-root|
+-------------------------------------------------------------+
|no_root_squash |anonuid/anongid  |useruid/usergid            |
+-------------------------------------------------------------+

Updates bz#1285126
Signed-off-by: Xie Changlong <xiechanglong@cmss.chinamobile.com>
Signed-off-by: Xue Chuanyu <xuechuanyu@cmss.chinamobile.com>
Change-Id: Iea043318fe6e9a75fa92b396737985062a26b47e
2018-12-05 21:45:49 +00:00
Milind Changire
748e993d1f rpc: bump up server.event-threads
Problem:
A single event-thread causes performance issues in the system.

Solution:
Bump up event-threads to 2 to make the system more performant.
This helps in making the system more responsive and helps avoid the
ping-timer-expiry problem as well. However, setting the event-threads
to 2 is not the only thing required to avoid ping-timer-expiry issues.

Change-Id: Idb0fd49e078db3bd5085dd083b0cdc77b59ddb00
fixes: bz#1653277
Signed-off-by: Milind Changire <mchangir@redhat.com>
2018-12-04 06:38:55 +00:00
Mohit Agrawal
46c15ea8fa server: Resolve memory leak path in server_init
Problem: 1) server_init does not cleanup allocate resources
            while it is failed before return error
         2) dict leak at the time of graph destroying

Solution: 1) free resources in case of server_init is failed
          2) Take dict_ref of graph xlator before destroying
             the graph to avoid leak

Change-Id: I9e31e156b9ed6bebe622745a8be0e470774e3d15
fixes: bz#1654917
Signed-off-by: Mohit Agrawal <moagrawal@redhat.com>
2018-12-03 11:34:35 +00:00
Yaniv Kaul
98a672f504 Multiple xlator .h files: remove unused private gf_* memory types.
It seems there were quite a few unused enums (that in turn
cause unndeeded memory allocation) in some xlators.
I've removed them, hopefully not causing any damage.

Compile-tested only!

updates: bz#1193929
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>

Change-Id: I8252bd763dc1506e2d922496d896cd2fc0886ea7
2018-11-30 11:51:18 +00:00
Raghavendra Gowdappa
95e380eca1 rpcsvc: provide each request handler thread its own queue
A single global per program queue is contended by all request handler
threads and event threads. This can lead to high contention. So,
reduce the contention by providing each request handler thread its own
private queue.

Thanks to "Manoj Pillai"<mpillai@redhat.com> for the idea of pairing a
single queue with a fixed request-handler-thread and event-thread,
which brought down the performance regression due to overhead of
queuing significantly.

Thanks to "Xavi Hernandez"<xhernandez@redhat.com> for discussion on
how to communicate the event-thread death to request-handler-thread.

Thanks to "Karan Sandha"<ksandha@redhat.com> for voluntarily running
the perf benchmarks to qualify that performance regression introduced
by ping-timer-fixes is fixed with this patch and patiently running
many iterations of regression tests while RCAing the issue.

Thanks to "Milind Changire"<mchangir@redhat.com> for patiently running
the many iterations of perf benchmarking tests while RCAing the
regression caused by ping-timer-expiry fixes.

Change-Id: I578c3fc67713f4234bd3abbec5d3fbba19059ea5
Fixes: bz#1644629
Signed-off-by: Raghavendra Gowdappa <rgowdapp@redhat.com>
2018-11-29 01:19:12 +00:00
Raghavendra Gowdappa
18b6d7ce7d libglusterfs: rename macros roof and floor to not conflict with math.h
Change-Id: I666eeb63ebd000711b3f793b948d4e0c04b1a242
Signed-off-by: Raghavendra Gowdappa <rgowdapp@redhat.com>
Updates: bz#1644629
2018-11-28 15:11:59 +00:00
Xavi Hernandez
a0fdc9202c core: create a constant for default network timeout
A new constant named GF_NETWORK_TIMEOUT has been defined and all
references to the hard-coded timeout of 42 seconds have been
replaced with this constant.

Change-Id: Id30f5ce4f1230f9288d9e300538624bcf1a6da27
fixes: bz#1652852
Signed-off-by: Xavi Hernandez <xhernandez@redhat.com>
2018-11-23 11:05:02 +01:00
Shwetha Acharya
ada43dc22d protocol/client: unchecked return value
Problem: In client_process_response_v2, value returned from
function client_post_common_dict is not checked for errors
before being used.

Solution: Added a check condition to resolve the issue.

CID: 1390020

Change-Id: I4d297f33c8dd332ae5f6f21667a4871133b2b570
updates: bz#789278
Signed-off-by: Shwetha Acharya <sacharya@redhat.com>
2018-11-20 17:29:31 +05:30
Amar Tumballi
74e8328d3f all: fix the format string exceptions
Currently, there are possibilities in few places, where a user-controlled
(like filename, program parameter etc) string can be passed as 'fmt' for
printf(), which can lead to segfault, if the user's string contains '%s',
'%d' in it.

While fixing it, makes sense to make the explicit check for such issues
across the codebase, by making the format call properly.

Fixes: CVE-2018-14661

Fixes: bz#1644763
Change-Id: Ib547293f2d9eb618594cbff0df3b9c800e88bde4
Signed-off-by: Amar Tumballi <amarts@redhat.com>
2018-11-05 18:50:59 +00:00
Amar Tumballi
2effe3b0d3 protocol: remove the option 'verify-volfile-checksum'
'getspec' operation is not used between 'client' and 'server' ever since
we have off-loaded volfile management to glusterd, ie, at least 7 years.

No reason to keep the dead code! The removed option had no meaning,
as glusterd didn't provide a way to set (or unset) this option. So,
no regression should be observed from any of the existing glusterfs
deployment, supported or unsupported.

Updates: CVE-2018-14653

Updates: bz#1644756
Change-Id: I4a2e0f673c5bcd4644976a61dbd2d37003a428eb
Signed-off-by: Amar Tumballi <amarts@redhat.com>
2018-11-05 18:50:40 +00:00
Amar Tumballi
5441fb4196 server: don't allow '/' in basename
Server stack needs to have all the sort of validation, assuming
clients can be compromized. It is possible for a compromized
client to send basenames with paths with '/', and with that
create files without permission on server. By sanitizing the basename,
and not allowing anything other than actual directory as the parent
for any entry creation, we can mitigate the effects of clients
not able to exploit the server.

Fixes: CVE-2018-14651

Fixes: bz#1644755
Change-Id: I5dc0da0da2713452ff2b65ac2ddbccf1a267dc20
Signed-off-by: Amar Tumballi <amarts@redhat.com>
2018-11-05 18:50:10 +00:00
Amar Tumballi
0f817c7795 server-protocol: set the frame type to TYPE_FOP
This will allow proper printing of exact 'fop' type to be logged in
string, not number, during backtraces.

Considering this was not done on brick processes, we have no easy
way to glance and understand which fops were pending.

What gets changed:

After a crash, most of the core-dumps logged were of the form:
```
pending frames:
frame : type(0) op(18)
frame : type(0) op(18)
frame : type(0) op(28)
```
would change to
```
pending frames:
frame : type(1) op(SETXATTR)
frame : type(1) op(SETXATTR)
frame : type(1) op(READDIR)
```

updates: bz#1639599
Change-Id: I0e3d2a8dee9cfde7ed0112a948f5213f546efb80
Signed-off-by: Amar Tumballi <amarts@redhat.com>
2018-10-29 08:56:46 +00:00
Arjun Sharma
e1a9c23b83 protocol/server: added check for positve value in server-handshake.c
This patch fixes CID 1124651

Change-Id: I6f33954f08cfdd7cb4236f9a81ec7980f81d19e7
updates: bz#789278
Signed-off-by: Arjun <arjsharm@redhat.com>
2018-10-21 05:56:22 +00:00
Yaniv Kaul
46689e34b9 servers-helpers.c: reduce clang warnings on dead initialization.
Assign a value to a parameter on its definition.
Removes tens of clang warnings.

Compile-tested only!

updates: bz#1193929
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>

Change-Id: I828bacb4033111af6162772a554cc5c6b2820169
2018-10-16 10:45:45 +00:00
Xavi Hernandez
2d96ce8faa all: fix warnings on non 64-bits architectures
When compiling in other architectures there appear many warnings. Some
of them are actual problems that prevent gluster to work correctly on
those architectures.

Change-Id: Icdc7107a2bc2da662903c51910beddb84bdf03c0
fixes: bz#1632717
Signed-off-by: Xavi Hernandez <xhernandez@redhat.com>
2018-10-10 05:50:29 +00:00
Milind Changire
3108fb24e7 rpc: coverity fixes
CID: [1] 1394646 Unchecked return value from library
CID: [2] 1394633 Unused value
CID:     1382443 Sleeping while holding a lock [This is intentional]

[1] https://scan6.coverity.com/reports.htm#v40014/p10714/fileInstanceId=86159112&defectInstanceId=26360786&mergedDefectId=1394646
[2] https://scan6.coverity.com/reports.htm#v40014/p10714/fileInstanceId=86159365&defectInstanceId=26360919&mergedDefectId=1394633

Change-Id: I03086f7a9672c9f50a2bc44cdbce0006c887357b
updates: bz#789278
Signed-off-by: Milind Changire <mchangir@redhat.com>
2018-10-09 02:25:51 +00:00
Mohit Agrawal
5bc4594dab core: glusterfsd keeping fd open in index xlator
Problem: Current resource cleanup sequence is not
         perfect while brick mux is enabled

Solution: 1) Destroying xprt after cleanup all fd associated
             with a client
          2) Before call fini for brick xlators ensure no stub
             should be running on a brick

Change-Id: I86195785e428f57d3ef0da3e4061021fafacd435
fixes: bz#1631357
Signed-off-by: Mohit Agrawal <moagrawal@redhat.com>
2018-10-08 15:45:47 +00:00
Shwetha K Acharya
f2137d4c84 protocol/server: NULL pointer dereferencing clang fix
Problem: Access to field fop_length results in null pointer dereferencing.

Solution: Added condition checks fix the issue.

Change-Id: Id408e3ac62ea9574f0cd9aecce5434add09eb7d0
Updates: bz#1622665
Signed-off-by: Shwetha K Acharya <sacharya@redhat.com>
2018-09-17 12:23:09 +00:00
Kaleb S. KEITHLEY
f15e948da3 build: cleanup xlator link, --no-undefined, libuuid
While attempting to build a (pre-)5.0 of glusterfs on Ubuntu
bionic and cosmic, it became apparent that there are some gremlins
hiding in the combination of the xlator export-symbols, the newish
addition of -Wl,--no-undefined, and the new switch to libuuid from
the old contrib/uuid.

Note: even though Fedora 28 (and later) and Ubuntu bionic (and
later) have the same nominal version of libtool, the Fedora version
appears to do a better job of recursing through dependencies to
determine the libraries to link with.

Examination of the build logs showed that despite appearing to work
on Fedora, not all xlators and shared libs were linked with -Wl,
--no-undefined, and -luuid. And in the case of the gnfs xlator, it
was not only not linked with -Wl,--no-undefined but alsos not linked
with -lgfxdr and -lgfrpc.

Added GF_XLATOR_LDFLAGS, similar to GF_XLATOR_DEFAULT_LDFLAGS.
GF_XLATOR_DEFAULT_LDFLAGS is for xlators that export/expose the
default or common set of symbols. GF_XLATOR_LDFLAGS is for those
remaining xlators that export/expose non-default symbols, e.g. dht
and glupy. This removes the need in the future to add things like
$(UUID_LIBS) to every xlator's Makefile.am. Just add it to
GF_XLATOR_LDFLAGS and GF_XLATOR_DEFAULT_LDFLAGS in configure.ac
and you're done.

This patch was tested on Fedora 28 (build, rpmbuild), Fedora
Rawhide/30 (rpmbuild), RHEL8 (rpmbuild), CentOS7 (rpmbuild), Fedora
koji --scratch build for f30/rawhide, and a Launchpad build for
Ubuntu cosmic/18.10.

Change-Id: Ieca104fa5c5d3c094e701c8ca4a73754dd0292b0
updates: bz#1193929
Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
2018-09-12 14:38:13 +00:00
Gluster Ant
e16868dede Land part 2 of clang-format changes
Change-Id: Ia84cc24c8924e6d22d02ac15f611c10e26db99b4
Signed-off-by: Nigel Babu <nigelb@redhat.com>
2018-09-12 17:52:45 +05:30
Gluster Ant
45a71c0548 Land clang-format changes
Change-Id: I6f5d8140a06f3c1b2d196849299f8d483028d33b
2018-09-12 17:22:48 +05:30
Amar Tumballi
35f86ce462 protocol: don't use alloca
current implementation of alloca can cause issues when strings larger
than the allocated buffer is passed to the xdr. Hence it makes sense
to allow XDR decode functions to deal with memory allocations, which
we can free later.

Fixes: bz#1625097

Change-Id: I3a05553f5702de9575c244649ca0e5ac9abaac94
Signed-off-by: Amar Tumballi <amarts@redhat.com>
2018-09-05 09:23:53 +00:00
Amar Tumballi
9ae986f18c server-protocol: don't allow '../' path in 'name'
This will prevent any arbitrary file creation through glusterfs
by modifying the client bits.

Also check for the similar flaw inside posix too, so we prevent any
changes in layers in-between.

Fixes: bz#1625095

Signed-off-by: Amar Tumballi <amarts@redhat.com>
Change-Id: Id9fe0ef6e86459e8ed85ab947d977f058c5ae06e
2018-09-05 09:23:26 +00:00
Amar Tumballi
c25eee58c9 xlators: add classification flag to some
Add classification to those translators which has `xlator_api_t`
already defined and used.

Updates: #430
Change-Id: I9d2772cb2c4ed4ab06aaa546500cf3b7d00bddac
Signed-off-by: Amar Tumballi <amarts@redhat.com>
2018-09-04 09:16:36 +00:00
Yaniv Kaul
81cbbfd1d8 Multiple files: calloc -> malloc
xlators/storage/posix/src/posix-inode-fd-ops.c:
xlators/storage/posix/src/posix-helpers.c:
xlators/storage/bd/src/bd.c:
xlators/protocol/client/src/client-lk.c:
xlators/performance/quick-read/src/quick-read.c:
xlators/performance/io-cache/src/page.c
xlators/nfs/server/src/nfs3-helpers.c
xlators/nfs/server/src/nfs-fops.c
xlators/nfs/server/src/mount3udp_svc.c
xlators/nfs/server/src/mount3.c
xlators/mount/fuse/src/fuse-helpers.c
xlators/mount/fuse/src/fuse-bridge.c
xlators/mgmt/glusterd/src/glusterd-utils.c
xlators/mgmt/glusterd/src/glusterd-syncop.h
xlators/mgmt/glusterd/src/glusterd-snapshot.c
xlators/mgmt/glusterd/src/glusterd-rpc-ops.c
xlators/mgmt/glusterd/src/glusterd-replace-brick.c
xlators/mgmt/glusterd/src/glusterd-op-sm.c
xlators/mgmt/glusterd/src/glusterd-mgmt.c
xlators/meta/src/subvolumes-dir.c
xlators/meta/src/graph-dir.c
xlators/features/trash/src/trash.c
xlators/features/shard/src/shard.h
xlators/features/shard/src/shard.c
xlators/features/marker/src/marker-quota.c
xlators/features/locks/src/common.c
xlators/features/leases/src/leases-internal.c
xlators/features/gfid-access/src/gfid-access.c
xlators/features/cloudsync/src/cloudsync-plugins/src/cloudsyncs3/src/libcloudsyncs3.c
xlators/features/bit-rot/src/bitd/bit-rot.c
xlators/features/bit-rot/src/bitd/bit-rot-scrub.c
bxlators/encryption/crypt/src/metadata.c
xlators/encryption/crypt/src/crypt.c
xlators/performance/md-cache/src/md-cache.c:

Move to GF_MALLOC() instead of GF_CALLOC() when possible

It doesn't make sense to calloc (allocate and clear) memory
when the code right away fills that memory with data.
It may be optimized by the compiler, or have a microscopic
performance improvement.

In some cases, also changed allocation size to be sizeof some
struct or type instead of a pointer - easier to read.
In some cases, removed redundant strlen() calls by saving the result
into a variable.

1. Only done for the straightforward cases. There's room for improvement.
2. Please review carefully, especially for string allocation, with the
terminating NULL string.

Only compile-tested!

.. and allocate memory as much as needed.

xlators/nfs/server/src/mount3.c :

Don't blindly allocate PATH_MAX, but strlen() the string and allocate
appropriately.
Also, align error messges.

updates: bz#1193929
Original-Author: Yaniv Kaul <ykaul@redhat.com>
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
Change-Id: Ibda6f33dd180b7f7694f20a12af1e9576fe197f5
2018-09-04 04:58:14 +00:00
Yaniv Kaul
058d215174 multiple xlators: move from strlen() to sizeof()
xlators/performance/nl-cache/src/nl-cache.c
xlators/performance/md-cache/src/md-cache.c
xlators/protocol/server/src/authenticate.c
xlators/storage/bd/src/bd-helper.c

For const strings, just do compile time size calc instead of runtime.

Compile-tested only!

Change-Id: I9b98940a38d85321a69436a1871930da367b918a
updates: bz#1193929
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
2018-08-31 06:13:54 +00:00
Amar Tumballi
d3b1456c52 clang-scan: fix multiple issues
* Buffer overflow issue in glusterfsd
* Null argument passed to function expecting non-null (event-epoll)
* Make sure the op_ret value is set in macro (posix)

Updates: bz#1622665

Change-Id: I32b378fc40a5e3ee800c0dfbc13335d44c9db9ac
Signed-off-by: Amar Tumballi <amarts@redhat.com>
2018-08-31 01:26:06 +00:00
Amar Tumballi
6a2f83caad coverity: multiple fixes
CID: 1390477, 1124827

updates: bz#789278
Change-Id: I41060d131aec6e58e7267ac8531b29a70f8c4359
Signed-off-by: Amar Tumballi <amarts@redhat.com>
2018-08-31 01:25:32 +00:00
Niels de Vos
44e40404ed build: add --enable-asan configure options
Introduce a `./configure --enable-asan` to build with
`-fsanitize=address -fno-omit-frame-pointer` options. This uses the
libasan.so shared library, so that needs to be available.

While running builds with the ASAN options, several linker issues
surfaced and these have been addressed with this change as well.
Building with --enable-asan has been tested on Fedora 28.

Change-Id: I428a9da70dd8f7d0056cfbe5c398619a571469b2
Updates: #492
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2018-08-30 05:41:26 +00:00
Bhumika Goyal
0ebaa9c663 protocol: coverity fixes
Fixes CID: 1389388 1389320 1274113 1388881 1388623 1124801 1124795

Change-Id: Ia72abc0560c959b0298f42e25abdfc5523755569
updates: bz#789278
Signed-off-by: Bhumika Goyal <bgoyal@redhat.com>
2018-08-22 06:45:43 +00:00
Bhumika Goyal
6fa531c57c xlators: protocol: Fix deferencing pointer after free coverity issues
The pointer of type struct iobuf * is getting dereferenced after
getting freed by iobuf_unref function. Therefore, move this function
after all the dereferences of this pointer type.
Also, it is useful coding standard to have iobuf_unref just after
iobref_add. So, move iobref_add too.

Occurences found using Coccinelle script:
@@
identifier rsphdr_iobuf;
expression E;
identifier func;
@@
*iobuf_unref(rsphdr_iobuf);
...
*E = func(rsphdr_iobuf);

Fixes CID: 1390517, 1390278, 1388666, 1356588, 1356587 at [1].
and also some more occurences which were found using the above script but not
caught by Coverity.
[1]. https://scan6.coverity.com/reports.htm#v42388/p10714/fileInstanceId=84384920&defectInstanceId=25600709&mergedDefectId=1388666

Change-Id: I579e9d12698f14e9e24bc926c6efef16bac5c06c
updates: bz#789278
Signed-off-by: Bhumika Goyal <bgoyal@redhat.com>
2018-08-21 05:01:08 +00:00
Yaniv Kaul
b2b6ab8eff All: remove memset() before sprintf()
It's not needed.
There's a good chance the compiler is smart enough to remove it
anyway, but it can't hurt - I hope.

Compile-tested only!

Change-Id: Id7c054e146ba630227affa591007803f3046416b
updates: bz#1193929
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
2018-08-14 05:15:04 +00:00
Niels de Vos
35cfc01fa2 build: rename event.h to gf-event.h
Newer FreeBSD versions (noticed with 10.3-RELEASE) provide a event.h
file that on occasion gets included instead of the libglusterfs file.
When this happens, 'struct event_pool' will not be defined and building
will fail with errors like:

    autoscale-threads.c:18:55: error: incomplete definition of type 'struct event_pool'
            int                      thread_count   = pool->eventthreadcount;
                                                      ~~~~^
    autoscale-threads.c:17:16: note: forward declaration of 'struct event_pool'
            struct event_pool       *pool           = ctx->event_pool;
                   ^

This problem is caused by 'pkg-config --cflags uuid' that adds
/usr/local/include to the GF_CPPFLAGS. The use of libuuid is preferred
so that the contrib/uuid/ directory can be removed.

By renaming event.h to gf-event.h there is no conflict between the
different event.h files anymore and compiling on FreeBSD works without
issues.

Change-Id: Ie69f6b8a4f8f8e9630d39a86693eb74674f0f763
Updates: bz#1607319
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2018-07-27 17:37:32 +00:00
Yaniv Kaul
621138ce76 All: run codespell on the code and fix issues.
Please review, it's not always just the comments that were fixed.
I've had to revert of course all calls to creat() that were changed
to create() ...

Only compile-tested!

Change-Id: I7d02e82d9766e272a7fd9cc68e51901d69e5aab5
updates: bz#1193929
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
2018-07-22 14:40:16 +00:00
Prashanth Pai
af6c6429f9 server: Set ssl-allow option in options table and rename ID
This change explicitly adds 'ssl-allow' options to the server xlator's
options table so that glusterd2 can see it as a settable option. This
change also marks 'auth.allow' and 'auth.reject' options as a settable.

Glusterd2 doesn't maintain a separate volume options table. Glusterd2
dynamically loads shared objects of xlators to read their option table
and other information. Glusterd2 reads 'xlator_api_t' if available. If
that's not available, it falls back to reading just the options table
directly.

In glusterd2, volume set operations are performed by users on keys of
the format <xlator>.<option-name>. Glusterd2 uses xlator name set in
'xlator_api_t.identifier'. If that's not present it will use the shared
object's file name as xlator name. Hence, it is important for
'xlator_api_t.identifier' to be set properly, and in this case, the
proper value is "server". This name shall be used by users as prefix
while setting volume options implemented in server xlator. The name will
also be used in volfile.

A user in glusterd2 can authorize a client over TLS as follows:

$ glustercli volume set <volname> server.ssl-allow <client1-CN>[,<clientN-CN>]

gd2 References:
https://github.com/gluster/glusterd2/issues/971
https://github.com/gluster/glusterd2/issues/214
https://github.com/gluster/glusterd2/pull/967

Updates: bz#1193929
Change-Id: I59ef58acb8d51917e6365a83be03e79ae7c5ad17
Signed-off-by: Prashanth Pai <ppai@redhat.com>
2018-07-11 04:08:29 +00:00
Mohit Agrawal
dab12ce093 glusterfs: Brick process is crash at the time of call server_first_lookup
Problem: Brick process is getting crash while executing test case
         tests/bugs/core/bug-1432542-mpx-restart-crash.t

Solution: At the time of initiating connection with brick process by
          client brick process call server_setvolume.If cleanup thread
          has set cleanup_starting flag after check flag by server_setvolume
          then a brick process can crash at the time of calling lookup
          on brick root.To avoid crash check cleanup_starting
          flag before just call server_first_lookup

BUG: 1597627
Change-Id: I12542c124c76429184df34a04c1eae1a30052ca7
fixes: bz#1597627
Signed-off-by: Mohit Agrawal <moagrawa@redhat.com>
Note: To test the patch executing test case
      tests/bugs/core/bug-1432542-mpx-restart-crash.t in a loop
      around 100 times
2018-07-09 03:22:49 +00:00
Amar Tumballi
06cdc95609 protocol/client: handle the fdctx_destroy properly with different versions
while adding the new version of RPC, this part was not handled properly

Updates: bz#1193929
Change-Id: If4cc4c2db075221b9ed731bacb7cc035f7007c5b
Signed-off-by: Amar Tumballi <amarts@redhat.com>
2018-07-05 22:55:43 +00:00
Michael Adam
a2b65d01f5 client: remove the "connecting" state - it's not used
The "connecting" state is not used anywhere really.
It's only being set and printed. So remove it.

Change-Id: I11fc8b0bdcda5a812d065543aa447d39957d3b38
fixes: bz#1583583
Signed-off-by: Michael Adam <obnox@samba.org>
2018-06-21 05:37:09 +00:00
Krutika Dhananjay
337f5815a4 protocol/client: Remove code duplication
client_submit_vec_request() which is used by WRITEV, and PUT
and client_submit_request() used by the rest of the fops have
almost similar code. However, there have been some more checks
- such as whether setvolume was successful or not, and one more
that is send-gid-specific - that have been missed out in the vectored
version of the function.

This patch fixes this code duplication.

Change-Id: I363a28eeead6219cb1009dc831538153e8bd7d40
fixes: bz#1591580
Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com>
2018-06-15 10:16:29 +05:30
Pranith Kumar K
fd5b48ea0a protocol/server: Fix xdata leak in seek fop
Change-Id: I6125283ed22c04564f0b77bb7a50579a83e02eb0
fixes: bz#1589691
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
2018-06-11 12:50:54 +00:00
Raghavendra G
3894f4262d rpc/clnt: Don't let consumers manage "connected" state
The state management of "connected" in rpc is ad-hoc as far as the
responsibility goes. Note that there is nothing wrong with
functionality itself. rpc layer manages this state in disconnect
codepath and has exposed an api to manage this one from
consumers. Note that rpc layer never sets "connected" to true by
itself, which forces the consumers to use this api to get a working
rpc connection. The situation is best captured from a comment in code
from Jeff Darcy in glusterfsd/src/gf-attach.c:

-/*
- * In a sane world, the generic RPC layer would be capable of tracking
- * connection status by itself, with no help from us.  It might invoke our
- * callback if we had registered one, but only to provide information.  Sadly,
- * we don't live in that world.  Instead, the callback *must* exist and *must*
- * call rpc_clnt_{set,unset}_connected, because that's the only way those
- * fields get set (with RPC both above and below us on the stack).  If we don't
- * do that, then rpc_clnt_submit doesn't think we're connected even when we
- * are.  It calls the socket code to reconnect, but the socket code tracks this
- * stuff in a sane way so it knows we're connected and returns EINPROGRESS.
- * Then we're stuck, connected but unable to use the connection.  To make it
- * work, we define and register this trivial callback.
- */

Also, consumers of rpc know about state of connection only through the
notifications sent by rpc-clnt. So, consumers don't have any extra
information to manage the state and hence letting them manage the
state is counter intuitive. This patch cleans that up and instead
moves the responsibility of state management of rpc layer into
itself.

Change-Id: I31e641a60795fc480ca753917f4b2579f1e05094
Signed-off-by: Raghavendra G <rgowdapp@redhat.com>
Fixes: bz#1585585
2018-06-04 12:57:42 +05:30
Raghavendra G
430e4872db protocol/client: Don't send fops till SETVOLUME is complete
An earlier commit set conf->connected just after rpc layer sends
RPC_CLNT_CONNECT event. However, success of socket level connection
connection doesn't indicate brick stack is ready to receive fops, as
an handshake has to be done b/w client and server after
RPC_CLNT_CONNECT event. Any fop sent to brick in the window between,
* protocol/client receiving RPC_CLNT_CONNECT event
* protocol/client receiving a successful setvolume response

can end up accessing an uninitialized brick stack. So, set
conf->connected only after a successful SETVOLUME.

Change-Id: I139a03d2da6b0d95a0d68391fcf54b00e749decf
fixes: bz#1583937
Signed-off-by: Raghavendra G <rgowdapp@redhat.com>
2018-05-31 01:52:33 +00:00
Amar Tumballi
7ac79fb033 client/protocol: fix the log level for removexattr_cbk
noticed that server protocol actually logs all the errors for
removexattr as INFO, instead of WARNING like client, and hence,
doesn't create a confusion in user.

updates: bz#1576418
Change-Id: Ia6681e9ee433fda3c77a4509906c78333396e339
Signed-off-by: Amar Tumballi <amarts@redhat.com>
2018-05-17 04:05:49 +00:00
Mohit Agrawal
8e2289b1e7 glusterfs: Resolve brick crashes at the time of inode_unref
Problem: Sometimes brick process is getting crash at the time
         of calling inode_unref in fd_destroy

Solution: Brick process is getting crash because inode is already
          free by xlator_mem_cleanup call by server_rpc_notify.To
          resolve the same move code specific to call transport_unref
          in last in free_state.

BUG: 1577574
Change-Id: Ia517c230d68af4e929b6b753e4c374a26c39dc1a
fixes: bz#1577574
Signed-off-by: Mohit Agrawal <moagrawa@redhat.com>
2018-05-15 02:37:55 +00:00
hari gowtham
1ef9270c09 Glusterfsd: brick crash during get-state
The xprt's dereferencing wasn't checked before using it for the
strcmp, which caused the segfault and crashed the brick process.

fix: Check every deferenced variable before using it.

Change-Id: I7f705d1c88a124e8219bb877156fadb17ecf11c3
fixes: bz#1575864
Signed-off-by: hari gowtham <hgowtham@redhat.com>
2018-05-11 10:54:48 +00:00
Amar Tumballi
52d9b8bcd5 protocol/server: don't assume there would be a volfile id
Earlier glusterfs never had an assumption someone would start it with
right arguments, and brick processes would be spawned by a management
layer. It just assume the role based on the volfile. Other than
volfile, no other arguments should be technically mandatory for
working of glusterfs. With this patch, that assumption holds true.

Updates: github issue # 352

A note on why this particular issue for this basic sanity?

As per the design of thin-arbiter/tie-breaker, it can be started
independently on any machine, without need of glusterd. So, similar
to 'glusterd', we should be able to spawn a process with any translator
without options/volume id etc.

fixes: bz#1569399
Change-Id: I5c0650fe0bfde35ad94ccba60e63f6cdcd1ae5ff
Signed-off-by: Amar Tumballi <amarts@redhat.com>
2018-05-08 21:16:24 +00:00