377 Commits

Author SHA1 Message Date
Soumya Koduri
491ff40a7a gfapi: Unblock epoll thread for upcall processing
With commit#ad35193,we have made changes to offload
processing upcall notifications to synctask so as not
to block epoll threads. However seems like the issue wasnt
fully addressed.

In "glfs_cbk_upcall_data" -> "synctask_new1" after creating synctask
if there is no callback defined, the thread waits on synctask_join
till the syncfn is finished. So that way even with those changes,
epoll threads are blocked till the upcalls are processed.

Hence the right fix now is to define a callback function for that
synctask "glfs_cbk_upcall_syncop" so as to unblock epoll/notify threads
completely and the upcall processing can happen in parallel by synctask
threads.

Change-Id: I4d8645e3588fab2c3ca534e0112773aaab68a5dd
fixes: bz#1694561
Signed-off-by: Soumya Koduri <skoduri@redhat.com>
(cherry picked from commit 4a03a71c6171f6e8382664d9d29857d06ef37741)
2019-04-03 04:30:53 +00:00
Ravishankar N
4e7afab231 gfapi: add function to set client-pid
This api offers the ability to set the pid of a client to a particular
value, identical to how gluster fuse clients provide the --client-pid
option. This is an internal API to be used by gluster processes only. See
https://lists.gluster.org/pipermail/gluster-devel/2019-March/055925.html
for more details. Currently glfsheal is the only proposed consumer.

updates: bz#1693155
Change-Id: I0620be2127d79d69cdd57cffb29bba44e6e5da1f
Signed-off-by: Ravishankar N <ravishankar@redhat.com>
(cherry picked from commit 08d502c3b05c6f7831bb4cc764bd458b405a99b1)
2019-03-29 11:08:32 +00:00
Arjun Sharma
cd6795c5e8 Updating the glfs_setattr and glfs_fsetattr public APIs to use glfs_stat
Change-Id: I0e86c8222c88b4c71087ec287ba81f8353d70822
updates: #389
Signed-off-by: Arjun <arjsharm@redhat.com>
2019-02-28 12:00:30 +05:30
ShyamsundarR
352e203266 api: Update all future API versions to rel-6
As release 6 is branched, all future APIs now become 6.0

This change implements the same.

Change-Id: I6db368b4dc8585278ec11d4a411adcd04635de53
Updates: bz#1672818
Signed-off-by: ShyamsundarR <srangana@redhat.com>
(cherry picked from commit ddcb3c7438472072daa6fbe3f3b87067b6d7bc54)
2019-02-08 15:47:36 +00:00
Kaleb S. KEITHLEY
4aee035cd2 api: bad GFAPI_4.1.6 block
missing global: line, tabs not spaces

Change-Id: Icdbc23b4e4cd608da1d764e81757201c4b1269a6
updates: bz#1193929
Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
2019-01-30 07:01:09 +00:00
Milind Changire
b6c417785e rpc: use address-family option from vol file
This patch helps enable IPv6 connections in the cluster.
The default address-family is IPv4 without using this option explicitly.

When address-family is set to "inet6" in the /etc/glusterfs/glusterd.vol
file, the mount command-line also needs to have
-o xlator-option="transport.address-family=inet6" added to it.

This option also gets added to the brick command-line.
Snapshot and gfapi use-cases should also use this option to pass in the
inet6 address-family.

Change-Id: I97db91021af27bacb6d7578e33ea4817f66d7270
fixes: bz#1635863
Signed-off-by: Milind Changire <mchangir@redhat.com>
2019-01-22 13:47:19 +00:00
Arjun
213f31bf9e gfapi : making glfs_setattr/glfs_fsetattr as public
Initially glfs_fsetattr and glfs_setattr, both functions accepted iatt as arguements
but now they accept stat and later in the function the stat is being converted to iatt
so that it can be passed to syncop_fsetattr/syncop_setattr.

Change-Id: I41a9e0124785a32ca19ef4d492c5ed5002e66ede
updates: #389
Signed-off-by: Arjun Sharma <arjsharm@redhat.com>
2019-01-16 06:24:45 +00:00
Mohit Agrawal
211521f039 core: Resolve dict_leak at the time of destroying graph
Problem: In gluster code some of the places it call's get_new_dict
         to create a dictionary without taking reference so at the time
         of dict_unref it has become a leak

Solution: To resolve the same call dict_new instead of get_new_dict

updates bz#1650403
Change-Id: I3ccbbf5af07079a4fa09aad2cd0458c8625b2f06
Signed-off-by: Mohit Agrawal <moagrawal@redhat.com>
2019-01-14 04:01:54 +00:00
ShyamsundarR
1a3c2a48e7 gfapi: update returned/callback pre/post attributes to glfs_stat
Change-Id: Ie0fe971e694101aa011d66aa496d0644669c2c5a
Updates: #389
Signed-off-by: Kinglong Mee <mijinlong@open-fs.com>
Signed-off-by: ShyamsundarR <srangana@redhat.com>
2019-01-07 14:43:27 +00:00
ShyamsundarR
c31f1c232a gfapi: new api glfs_statx as linux's statx
Change-Id: I44dd6ceef0954ae7fc13f920e84d81bbd3f6a774
Updates: #389
Signed-off-by: Kinglong Mee <mijinlong@open-fs.com>
Signed-off-by: ShyamsundarR <srangana@redhat.com>
2019-01-07 14:43:27 +00:00
Kinglong Mee
f99f51ca87 glfs-fops.c: fix the bad string length for snprintf
Snprintf reserves one byte for the ending '\0'.
A NAME_MAX bytes d_name is truncated to NAME_MAX - 1 bytes.

Change-Id: Ic884d18cee24360e55ddb896dc587b0b74ef97fe
updates: bz#1193929
Signed-off-by: Kinglong Mee <mijinlong@open-fs.com>
2019-01-07 09:20:21 +00:00
Soumya Koduri
d9a8ccd354 gfapi: nit cleanup related to releasing fs->mutex lock
This is follow-up patch to the comment received for
 - https://review.gluster.org/#/c/glusterfs/+/21882

We need not hold the fs->mutex lock to log error message.

Change-Id: I29d2ea2e6cfecc3dd94982bd48f4bc9f11cc3aac
fixes: bz#1660577
Signed-off-by: Soumya Koduri <skoduri@redhat.com>
2018-12-31 16:10:04 +00:00
Sunny Kumar
1ea203d6b6 libgfapi : fix coverity issue
This patch fixes coverity issue in api/src/glfs-fops.c.

CID: 1389247, 1389296, 1389369, 1389392.
All coverity defects are of type Mixing enum types (MIXED_ENUMS).

updates: bz#789278

Change-Id: I007bb317ed5f0b8ddaf94a93b3a4d02b1e74cb8d
Signed-off-by: Sunny Kumar <sunkumar@redhat.com>
2018-12-31 06:51:07 +00:00
Soumya Koduri
8fe3c6107a gfapi: Access fs->oldvolfile under mutex lock
In some cases (for eg., when there are multiple
RPC_CLNT_CONNECT notifications), multiple threads may fetch
volfile and try to update it in 'fs' object simultaneously.
Hence protect those variables' access under fs->mutex lock.

Change-Id: Idaee9548560db32d83f4c04ebb1f375fee7864a9
fixes: bz#1660577
Signed-off-by: Soumya Koduri <skoduri@redhat.com>
2018-12-25 17:36:01 +00:00
ShyamsundarR
bfe2b5e153 clang: Fix various missing checks for empty list
When using list_for_each_entry(_safe) functions, care needs
to be taken that the list passed in are not empty, as these
functions are not empty list safe.

clag scan reported various points where this this pattern
could be caught, and this patch fixes the same.

Additionally the following changes are present in this patch,
- Added an explicit op_ret setting in error case in the
macro MAKE_INODE_HANDLE to address another clang issue reported
- Minor refactoring of some functions in quota code, to address
possible allocation failures in certain functions (which in turn
cause possible empty lists to be passed around)

Change-Id: I1e761a8d218708f714effb56fa643df2a3ea2cc7
Updates: bz#1622665
Signed-off-by: ShyamsundarR <srangana@redhat.com>
2018-12-14 04:33:15 +00:00
Amar Tumballi
af7e957b49 xlator: make 'xlator_api' mandatory
* Remove the options to load old symbol.
* keep only 'xlator_api' symbol from being exported using xlator.sym
* add xlator_api to all the xlators where its missing

NOTE: This covers all the xlators which has at least a test case
to validate its loading. If there is a translator, which doesn't
have any test, then we should probably remove that from codebase.

fixes: #164
Change-Id: Ibcdc8c9844cda6b4463d907a15813745d14c1ebb
Signed-off-by: Amar Tumballi <amarts@redhat.com>
2018-12-13 14:41:50 +05:30
Raghavendra Bhat
7dadea15c5 copy_file_range support in GlusterFS
* libglusterfs changes to add new fop

    * Fuse changes:
      - Changes in fuse bridge xlator to receive and send responses

    * posix changes to perform the op on the backend filesystem

    * protocol and rpc changes for sending and receiving the fop

    * gfapi changes for performing the fop

    * tools: glfs-copy-file-range tool for testing copy_file_range fop

      - Although, copy_file_range support has been added to the upstream
	    fuse kernel module, no release has been made yet of a kernel
        which contains the support. It is expected to come in the
        upcoming release of linux-4.20

        So, as of now, executing copy_file_range fop on a fused based
        filesystem results in fuse kernel module sending read on the
	    source fd and write on the destination fd.

	    Therefore a small gfapi based tool has been written to be able
        test the copy_file_range fop. This tool is similar (in functionality)
	    to the example program given in copy_file_range man page.

	    So, running regular copy_file_range on a fuse mount point and
	    running gfapi based glfs-copy-file-range tool gives some idea about
	    how fast, the copy_file_range (or reflink) can be.

	    On the local machine this was the result obtained.

	    mount -t glusterfs workstation:new /mnt/glusterfs
	    [root@workstation ~]# cd /mnt/glusterfs/
	    [root@workstation glusterfs]# ls
	    file
	    [root@workstation glusterfs]# cd
	    [root@workstation ~]# time /tmp/a.out /mnt/glusterfs/file /mnt/glusterfs/new
	    real  0m6.495s
	    user  0m0.000s
	    sys   0m1.439s
	    [root@workstation ~]# time glfs-copy-file-range $(hostname) new /tmp/glfs.log /file /rrr
	    OPEN_SRC: opening /file is success
	    OPEN_DST: opening /rrr is success
	    FSTAT_SRC: fstat on /rrr is success
	    copy_file_range successful

        real  0m0.309s
        user  0m0.039s
        sys   0m0.017s

        This tool needs following arguments
         1) hostname
         2) volume name
         3) log file path
         4) source file path (relative to the gluster volume root)
         5) destination file path (relative to the gluster volume root)

        "glfs-copy-file-range <hostname> <volume> <log file path> <source> <destination>"

      - Added a testcase as well to run glfs-copy-file-range tool

    * io-stats changes to capture the fop for profiling

    * NOTE:

      - Added conditional check to see whether the copy_file_range syscall
        is available or not. If not, then return ENOSYS.

      - Added conditional check for kernel minor version in fuse_kernel.h
        and fuse-bridge while referring to copy_file_range. And the kernel
        minor version is kept as it is. i.e. 24. Increment it in future
        when there is a kernel release which contains the support for
        copy_file_range fop in fuse kernel module.

    * The document which contains a writeup on this enhancement can be found at
      https://docs.google.com/document/d/1BSILbXr_knynNwxSyyu503JoTz5QFM_4suNIh2WwrSc/edit

Change-Id: I280069c814dd21ce6ec3be00a884fc24ab692367
updates: #536
Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com>
2018-12-12 15:56:55 +00:00
ShyamsundarR
20ef211cfa libglusterfs: Move devel headers under glusterfs directory
libglusterfs devel package headers are referenced in code using
include semantics for a program, this while it works can be better
especially when dealing with out of tree xlator builds or in
general out of tree devel package usage.

Towards this, the following changes are done,
- moved all devel headers under a glusterfs directory
- Included these headers using system header notation <> in all
code outside of libglusterfs
- Included these headers using own program notation "" within
libglusterfs

This change although big, is just moving around the headers and
making it correct when including these headers from other sources.

This helps us correctly include libglusterfs includes without
namespace conflicts.

Change-Id: Id2a98854e671a7ee5d73be44da5ba1a74252423b
Updates: bz#1193929
Signed-off-by: ShyamsundarR <srangana@redhat.com>
2018-12-05 21:47:04 +00:00
Soumya Koduri
ad35193718 gfapi: Offload callback notifications to synctask
Upcall notifications are received from server via epoll
and same thread is used to forward these notifications
to the application. This may lead to deadlock and hang
in the following scenario.

Consider if as part of handling these callbacks,
application has to do some operations which involve
sending I/Os to gfapi stack which inturn have to wait for
epoll threads to receive repsonse. Thus this may lead to
deadlock if all the epoll threads are waiting to complete
these callback notifications.

To address it, instead of using epoll thread itself,
make use of synctask to send those notificaitons to the
application.

Change-Id: If614e0d09246e4279b9d1f40d883a32a39c8fd90
updates: bz#1648768
Signed-off-by: Soumya Koduri <skoduri@redhat.com>
2018-11-26 04:20:05 +00:00
Soumya Koduri
04be5463b2 gfapi: Send fop_attr dict as part of syncop_open
Leaseid (stored in thread locals) is sent to server via xdata.
This dict variable is set but not passed as argument in glfs_h_open().
Fixed the same.

Change-Id: Idd2f8a0ec184b4b6b1ad1e6e5d75df551c36a96d
updates: bz#1648768
Signed-off-by: Soumya Koduri <skoduri@redhat.com>
2018-11-17 09:16:13 +00:00
Kinglong Mee
6d4cd8ce6c gfapi: fix bad dict setting of lease-id
lease_id is a 16 bits opaque data, copying it by gf_strdup is wrong.

Invalid read of size 2
   at 0x483FA2F: memmove (vg_replace_strmem.c:1270)
   by 0xE2EF6FB: ??? (in /usr/lib64/libtirpc.so.3.0.0)
   by 0xE2EE047: xdr_opaque (in /usr/lib64/libtirpc.so.3.0.0)
   by 0x107A97DC: xdr_gfx_value (glusterfs4-xdr.c:207)
   by 0x107A98C0: xdr_gfx_dict_pair (glusterfs4-xdr.c:321)
   by 0xE2EF35E: xdr_array (in /usr/lib64/libtirpc.so.3.0.0)
   by 0x107A9A89: xdr_gfx_dict (glusterfs4-xdr.c:335)
   by 0x107AA97B: xdr_gfx_write_req (glusterfs4-xdr.c:897)
   by 0x107A181E: xdr_serialize_generic (xdr-generic.c:25)
   by 0x231044A2: client_submit_request (client.c:205)
   by 0x2314D3C1: client4_0_writev (client-rpc-fops_v2.c:3863)
   by 0x230FD5FA: client_writev (client.c:956)
 Address 0xad659e18 is 72 bytes inside a block of size 73 alloc'd
   at 0x483880B: malloc (vg_replace_malloc.c:299)
   by 0x106BA7EC: __gf_malloc (mem-pool.c:136)
   by 0x1064521E: gf_strndup (mem-pool.h:166)
   by 0x1064521E: gf_strdup (mem-pool.h:183)
   by 0x1064521E: get_fop_attr_thrd_key (glfs.c:627)
   by 0x1064D8E9: glfs_pwritev@@GFAPI_3.4.0 (glfs-fops.c:1154)
   by 0x10610C0C: glusterfs_write2 (handle.c:2092)
   by 0x54D30C: mdcache_write2 (mdcache_file.c:647)
   by 0x48A3FC: nfs4_write (nfs4_op_write.c:459)
   by 0x48A44D: nfs4_op_write (nfs4_op_write.c:487)
   by 0x4634F5: nfs4_Compound (nfs4_Compound.c:947)
   by 0x460155: nfs_rpc_process_request (nfs_worker_thread.c:1329)
   by 0x4608A3: nfs_rpc_valid_NFS (nfs_worker_thread.c:1539)
   by 0x488F12F: svc_vc_decode (svc_vc.c:825)

Updates bz#1647651
Change-Id: Ib9fff55c897bc43c15036a869888e763df133757
Signed-off-by: Kinglong Mee <mijinlong@open-fs.com>
2018-11-08 15:12:22 +00:00
Raghavendra Gowdappa
6257276d9d api: fill out attribute information if not valid
translators like readdir-ahead selectively retain entry information of
iatt (gfid and type) when rest of the iatt is invalidated (for write
invalidating ia_size, (m)(c)times etc). Fuse-bridge uses this
information and sends only entry information in readdirplus
response. However such option doesn't exist in gfapi. This patch
modifies gfapi to populate the stat by forcing an extra lookup.

Thanks to Shyamsundar Ranganathan <srangana@redhat.com> and Prashanth
Pai <ppai@redhat.com> for tests.

Change-Id: Ieb5f8fc76359c327627b7d8420aaf20810e53000
Fixes: bz#1630804
Signed-off-by: Raghavendra Gowdappa <rgowdapp@redhat.com>
Signed-off-by: Soumya Koduri <skoduri@redhat.com>
2018-10-17 09:44:30 -04:00
Kinglong Mee
f50625e7b0 libgfapi: Send path with "." for the root when loc.parent is set
fixes: bz#1582516
Change-Id: Ia9649d7c3c0930dfec656d364cf6e9a98c774dd2
Signed-off-by: Kinglong Mee <mijinlong@open-fs.com>
2018-10-16 06:42:47 +00:00
Soumya Koduri
a303cd1acc gfapi: Bug fixes in leases processing code-path
This patch fixes below issues in gfapi lease code-path
* 'glfs_setfsleasid' should allow NULL input to be
   able to reset leaseid
* Applications should be allowed to (un)register for
  upcall notifications of type GLFS_EVENT_LEASE_RECALL
* APIs added to read contents of GLFS_EVENT_LEASE_RECALL
  argument which is of type "struct glfs_upcall_lease"

Change-Id: I3320ddf235cc82fad561e13b9457ebd64db6c76b
updates: #350
Signed-off-by: Soumya Koduri <skoduri@redhat.com>
2018-10-16 06:34:21 +00:00
Kinglong Mee
27d3d481bd gfapi: fix crash of using uninitialized fs->ctx
0  0x00007fb3db3a2ee4 in pub_glfs_fini (fs=0x7f8977d63f00) at glfs.c:1236
1  0x00007fb3db3a2065 in pub_glfs_new (volname=0x7f80de4d4d40 "openfs1")
    at glfs.c:784
2  0x00007fb3db5cf089 in glusterfs_get_fs (params=...,
    up_ops=up_ops@entry=0x7fb3ca643130)
    at /usr/src/debug/nfs-ganesha/src/FSAL/FSAL_GLUSTER/export.c:889
3  0x00007fb3db5cf99a in glusterfs_create_export (
    fsal_hdl=0x7fb3db7e2490 <GlusterFS+112>, parse_node=0x7fb3ca6387d0,
    err_type=<optimized out>, up_ops=0x7fb3ca643130)
    at /usr/src/debug/nfs-ganesha/src/FSAL/FSAL_GLUSTER/export.c:1011
4  0x00007fb3e11c485f in mdcache_fsal_create_export (
    sub_fsal=0x7fb3db7e2490 <GlusterFS+112>,
    parse_node=parse_node@entry=0x7fb3ca6387d0,
    err_type=err_type@entry=0x7fb3c0bef140, super_up_ops=<optimized out>)
    at /usr/src/debug/nfs-ganesha/src/FSAL/Stackable_FSALs/FSAL_MDCACHE/mdcache_main.c:281

(gdb) p errno
$1 = 12

Change-Id: I3dd5b84b52962ceb0b5d4f9b4f475bf4aa724292
updates: bz#1626313
Signed-off-by: Kinglong Mee <mijinlong@open-fs.com>
2018-10-10 17:07:48 +00:00
Xavi Hernandez
2d96ce8faa all: fix warnings on non 64-bits architectures
When compiling in other architectures there appear many warnings. Some
of them are actual problems that prevent gluster to work correctly on
those architectures.

Change-Id: Icdc7107a2bc2da662903c51910beddb84bdf03c0
fixes: bz#1632717
Signed-off-by: Xavi Hernandez <xhernandez@redhat.com>
2018-10-10 05:50:29 +00:00
Kinglong Mee
6b273c1644 libgfapi: fix use after freed of clnt when dispatching events
Avoid dispatching events to mgmt after freed,
unreference mgmt after the event_dispatch_destroy.

Change-Id: I5b762b37901de70a955661df0aff95bf055ba4ea
updates: bz#1626313
Signed-off-by: Kinglong Mee <mijinlong@open-fs.com>
2018-09-26 05:44:34 -04:00
Kaleb S. KEITHLEY
b64d08b56f api: old and new versioned symbols must appear in each section of gfapi.map
It's probably not clear in .../doc/developer-guide/gfapi-symbol-versions.md
that this is necessary, but it is.

Change-Id: I0a8a74e3ecf474c93583f5ebc97507c4277542b0
updates: bz#1193929
Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
2018-09-25 11:36:47 +00:00
ShyamsundarR
b4504eadd8 gfapi: Cleanup alias file
2 extra symbols were in the alias file, that are not exported
this is cleaned up with this patch.

Change-Id: I1ab54b9fb6b0d455884fbbfd89820c60bb861e6f
Updates: bz#1193929
Signed-off-by: ShyamsundarR <srangana@redhat.com>
2018-09-17 03:32:36 +00:00
Gluster Ant
e16868dede Land part 2 of clang-format changes
Change-Id: Ia84cc24c8924e6d22d02ac15f611c10e26db99b4
Signed-off-by: Nigel Babu <nigelb@redhat.com>
2018-09-12 17:52:45 +05:30
Gluster Ant
45a71c0548 Land clang-format changes
Change-Id: I6f5d8140a06f3c1b2d196849299f8d483028d33b
2018-09-12 17:22:48 +05:30
Kaleb S. KEITHLEY
af0d5a9b53 core: python3
see https://review.gluster.org/#/c/19788/,
    https://review.gluster.org/#/c/19871/,
    https://review.gluster.org/#/c/19952/,
    https://review.gluster.org/#/c/20104/,
    https://review.gluster.org/#/c/20162/,
    https://review.gluster.org/#/c/20185/,
    https://review.gluster.org/#/c/20207/,
    https://review.gluster.org/#/c/20227/,
    https://review.gluster.org/#/c/20307/,
    https://review.gluster.org/#/c/20320/,
    https://review.gluster.org/#/c/20332/,
    https://review.gluster.org/#/c/20364/,
    https://review.gluster.org/#/c/20441/, and
    https://review.gluster.org/#/c/20484

shebangs changed from /usr/bin/python2 to /usr/bin/python3.
(Reminder, various distribution packaging guidelines require use
of explicit python version and don't allow '#!/usr/bin/env python',
regardless of how handy that idiom may be.)

glusterfs.spec(.in) package python{2,3}-gluster and python2 or
python3 dependencies as appropriate.

configure(.ac):
+ test for and use python2 or python3 as appropriate. If build
  machine has python2 and python3, use python3. Override by
  setting PYTHON=/usr/bin/python2 when running configure.
+ PYTHONDEV_CPPFLAGS from python[23]-config --includes is a
  better match to the original python sysconfig.get_python_inc().
  All those other extraneous flags breaks the build.
+ Only change the shebangs once. Changing them over and over
  again, e.g., during a `make glusterrpms` in extras/LinuxRPM
  just sends make (is it really make that's looping?) into an
  infinite loop. If you figure out why, let me know.
+ Oldest python2 is python2.6 on CentOS 6 and Debian 8 (Jessie).
  Everything else has 2.7 or 3.x
+ logic from https://review.gluster.org/c/glusterfs/+/21050, which
  needs to be removed/merged after that patch is merged.

Builds on CentOS 6, CentOS 7, Fedora 28, Fedora rawhide, and the
mysterious RHEL > 7.

Change-Id: Idae21d3b6f58b32372e1daa0d234e491e563198f
updates: #411
Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
2018-09-03 09:14:44 +00:00
Niels de Vos
44e40404ed build: add --enable-asan configure options
Introduce a `./configure --enable-asan` to build with
`-fsanitize=address -fno-omit-frame-pointer` options. This uses the
libasan.so shared library, so that needs to be available.

While running builds with the ASAN options, several linker issues
surfaced and these have been addressed with this change as well.
Building with --enable-asan has been tested on Fedora 28.

Change-Id: I428a9da70dd8f7d0056cfbe5c398619a571469b2
Updates: #492
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2018-08-30 05:41:26 +00:00
Yaniv Kaul
036327d9e9 glfs-fops.c, glfs.c: strncpy() -> sprintf(), reduce strlen()'s
strncpy may not be very efficient for short strings copied into
a large buffer: If the length of src is less than n,
strncpy() writes additional null bytes to dest to ensure
that a total of n bytes are written.

Instead, use snprintf().

Also, save the result of strlen() and re-use it when possible.

Compile-tested only!

Change-Id: I4ecfb359cf0efaafeab245a8138f526b21613231
updates: bz#1193929
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
2018-08-24 16:26:36 +00:00
ShyamsundarR
faf736cb30 coverity: Multiple coverity fixes for issues with HIGH severity
glfs-fops.c
1391414 Uninitialized pointer read
List head needed initialization

glusterfsd-mgmt.c
graph.c
1382431 Buffer not null terminated
1382417 Dereference before null check
1382347 Buffer not null terminated
Cleaned usage of volfile_checksum member of gf_volfile_t struct
across the code base.

glusterd-tier.c
1382426 Resource leak
1370955 Dereference before null check
The function fixed needs more work, but with tier almost being
deprecated, addressed some parts of the reported coverity issues
as appropriate.

Tested using the following test cases:
./tests/basic/tier/new-tier-cmds.t
./tests/basic/tier/tier.t
./tests/basic/tier/bug-1214222-directories_missing_after_attach_tier.t
./tests/basic/tier/tier_lookup_heal.t
./tests/basic/tier/tier-heald.t
./tests/basic/tier/tier-snapshot.t
./tests/features/glfs-lease.t

Change-Id: I396f1c34bb112bb22d2745ed279e1a4850cac4af
Updates: bz#789278
Signed-off-by: ShyamsundarR <srangana@redhat.com>
2018-08-22 18:13:12 +00:00
Yaniv Kaul
ec8174353a api/src/glfs.c: move to GF_MALLOC() instead of GF_CALLOC()
It doesn't make sense to calloc (allocate and clear) memory
when the code right away fills that memory with data.
It may be optimized by the compiler, or have a microscopic
performance improvement.

Please review carefully, especially for string allocation, with the
terminating NULL string.

Only compile-tested!

Change-Id: Ieeb86ff5d99176250f57c0a28c215c709b83fc80
updates: bz#1193929
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
2018-08-22 08:56:46 +00:00
Kinglong Mee
da7c26ff1b gfapi: Use inode_forget in case of unlink/rename objects
After unlink/rename, the nonexistent inode should be remove from inode table,
that's better than put to lru list.

Change-Id: I5a40df188f3f42a2c864cfb4404b22b288fdcc12
fixes: bz#1612750
Signed-off-by: Kinglong Mee <mijinlong@open-fs.com>
2018-08-21 04:49:46 +00:00
Jiffin Tony Thottan
febee007bb gfapi : Handle the path == "" glfs_resolve_at
Currently there is no check for path = "" in glfs_resolve_at.
So if application sends an empty path, then the function resolves
into the parent inode which is incorrect. Plus modified possible
of "path" with "origpath" in the same function.

Change-Id: Ie5ff9ce4b771607b7dbb3fe00704fe670421792a
fixes: bz#1610236
Signed-off-by: Jiffin Tony Thottan <jthottan@redhat.com>
2018-08-13 11:44:55 +00:00
ShyamsundarR
60f1aeb08d coverity: Fix remaining SECURE_TEMP issues reported
Two pending SECURE_TEMP issues still exist in the coverity
reports, these are fixed by this patch.

In both instances (where functions actually seem to be
duplicates of each other) the need was for a FILE * and
not an fd. Applied the same pattern in both places as in
other parts of the code where mkstemp was used and later
a FILE * was created from the resulting fd for use.

Coverity report: https://download.gluster.org/pub/gluster/
  glusterfs/static-analysis/master/glusterfs-coverity/
  2018-07-30-4d3c62e7/html/

Issues numbered: 382, 383 (named SECURE_TEMP)

Further added tmpfile to the blacklist, so that future code
changes do not add the same, into symbol-check.sh.

Also corrected shellcheck errors in symbol-check.sh as a
result of updating the same.

Updates: bz#789278

Change-Id: I1d572a16ca5b5df2f597aeaa5f454fad34c8296e
Signed-off-by: ShyamsundarR <srangana@redhat.com>
2018-08-03 08:37:15 +00:00
Zhang Huan
75300258c4 libgfapi: fix memory leak on old volume files
Fix missing free of fs->oldvolfile. This patch uses standard
calloc/realloc to allocate memory for vol file. As by the time fs struct
is destroyed, the THIS->ctx is already gone, that causes invalid memory
access.

Change-Id: I72ae19c76eb16e61f077714f7e52405fee721997
fixes: bz#1607689
Signed-off-by: Zhang Huan <zhanghuan@open-fs.com>
2018-07-28 07:21:12 +00:00
Zhang Huan
a6b5ec5cb5 libgfapi: add missing unref of mgmt client in glfs_fini
Change-Id: I89cdf14cb9d822eaf7c01cf0b0220b423eb5b705
fixes: bz#1607689
Signed-off-by: Zhang Huan <zhanghuan@open-fs.com>
2018-07-28 07:21:12 +00:00
Niels de Vos
35cfc01fa2 build: rename event.h to gf-event.h
Newer FreeBSD versions (noticed with 10.3-RELEASE) provide a event.h
file that on occasion gets included instead of the libglusterfs file.
When this happens, 'struct event_pool' will not be defined and building
will fail with errors like:

    autoscale-threads.c:18:55: error: incomplete definition of type 'struct event_pool'
            int                      thread_count   = pool->eventthreadcount;
                                                      ~~~~^
    autoscale-threads.c:17:16: note: forward declaration of 'struct event_pool'
            struct event_pool       *pool           = ctx->event_pool;
                   ^

This problem is caused by 'pkg-config --cflags uuid' that adds
/usr/local/include to the GF_CPPFLAGS. The use of libuuid is preferred
so that the contrib/uuid/ directory can be removed.

By renaming event.h to gf-event.h there is no conflict between the
different event.h files anymore and compiling on FreeBSD works without
issues.

Change-Id: Ie69f6b8a4f8f8e9630d39a86693eb74674f0f763
Updates: bz#1607319
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2018-07-27 17:37:32 +00:00
Yaniv Kaul
621138ce76 All: run codespell on the code and fix issues.
Please review, it's not always just the comments that were fixed.
I've had to revert of course all calls to creat() that were changed
to create() ...

Only compile-tested!

Change-Id: I7d02e82d9766e272a7fd9cc68e51901d69e5aab5
updates: bz#1193929
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
2018-07-22 14:40:16 +00:00
Jiffin Tony Thottan
2a3090028a gfapi : Coverity Fixes
*** CID 1391419:  Control flow issues  (DEADCODE)
glusterfs/api/src/glfs-fops.c: 5224 in glfs_recall_lease_upcall()

*** CID 1391414:  Memory - illegal accesses  (UNINIT)
glusterfs/glusterfs/api/src/glfs-fops.c: 5137 in glfs_recall_lease_fd()

Fixes: #789278
Change-Id: Ie16103e7e321724d3cf1bdcb815c2e2467b9a366
Signed-off-by: Jiffin Tony Thottan <jthottan@redhat.com>
2018-06-14 14:50:16 +00:00
Kaleb S. KEITHLEY
e86c58b473 api: cleanup headers
fix inconsistent use in headers of:
  struct glfs and glfs_t
  struct glfs_fd and glfs_fd_t
  struct glfs_object and glfs_object_t

add typedefs to headers for various struct glfs_foos, and use them, e.g.:
  struct glfs_upcall and glfs_upcall_t

fix inconsistent type naming for 'enum glfs_lock_mode_t'. (All other
enum decls are 'enum glfs_foo' not 'enum_glfs_foo_t'), and add
typedefs for all enums, and use them.

style, replace tabs with spaces and follow return value style, i.e.:
  int
  pub_glfs_foo()
instead of:
  int pub_glfs_foo()

Update copyright dates, substantial new work has been done since 2012

Change-Id: I0a9654486ea132efde7977b655799fb37887b1d5
Updates: bz#1193929
Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
2018-06-11 13:10:42 +00:00
Raghavendra G
3894f4262d rpc/clnt: Don't let consumers manage "connected" state
The state management of "connected" in rpc is ad-hoc as far as the
responsibility goes. Note that there is nothing wrong with
functionality itself. rpc layer manages this state in disconnect
codepath and has exposed an api to manage this one from
consumers. Note that rpc layer never sets "connected" to true by
itself, which forces the consumers to use this api to get a working
rpc connection. The situation is best captured from a comment in code
from Jeff Darcy in glusterfsd/src/gf-attach.c:

-/*
- * In a sane world, the generic RPC layer would be capable of tracking
- * connection status by itself, with no help from us.  It might invoke our
- * callback if we had registered one, but only to provide information.  Sadly,
- * we don't live in that world.  Instead, the callback *must* exist and *must*
- * call rpc_clnt_{set,unset}_connected, because that's the only way those
- * fields get set (with RPC both above and below us on the stack).  If we don't
- * do that, then rpc_clnt_submit doesn't think we're connected even when we
- * are.  It calls the socket code to reconnect, but the socket code tracks this
- * stuff in a sane way so it knows we're connected and returns EINPROGRESS.
- * Then we're stuck, connected but unable to use the connection.  To make it
- * work, we define and register this trivial callback.
- */

Also, consumers of rpc know about state of connection only through the
notifications sent by rpc-clnt. So, consumers don't have any extra
information to manage the state and hence letting them manage the
state is counter intuitive. This patch cleans that up and instead
moves the responsibility of state management of rpc layer into
itself.

Change-Id: I31e641a60795fc480ca753917f4b2579f1e05094
Signed-off-by: Raghavendra G <rgowdapp@redhat.com>
Fixes: bz#1585585
2018-06-04 12:57:42 +05:30
Kaleb S. KEITHLEY
202d27c530 core/various: python3 compat, prepare for python2 -> python3
see https://review.gluster.org/#/c/19788/ and
    https://review.gluster.org/#/c/19871/

Selected small fixes from 2to3 utility. Specifically apply, basestring,
funcattrs, idioms, numliterals, set_literal, types, urllib, zip

Note: these 2to3 fixes report no changes are necessary: exec, execfile,
exitfunc, filter, getcwdu, intern, itertools, metaclass, methodattrs, ne,
next, nonzero, operator, paren, raw_input, reduce, reload, renames, repr,
standarderror, sys_exc, throw, tuple_params, xreadlines.

Any 2to3 fixes not in the above two lists have more extensive changes
which will follow in separate patches.

most unicode changes suggested by 2to3 will need to be applied at the
same time as changing the shebangs from python2 to python3. Prashanth
notes that unicode strings in py2 need 'u' prefix; unicode strings in
py3 3.0, 3.1, and 3.2 a 'u' prefix will throw an error, but in py3 3.3+
it is legal (or just ignored). All Linux dists we care about have 3.3
or later so we can leave 'u' prefixes on unicode strings.

Change-Id: I49bba2f328b0ee24b9a8115a7183be979981563e
updates: #411
Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
2018-05-30 03:37:46 +00:00
Kotresh HR
3d38e4e47f libgfapi: Fix lookup on root
Lookup on root was sending "/" as the path.
This will break the basename calculation in
loc_copy and hence lookup on root was failing
if the loc_copy was involved in the stack.

With ctime, a first lookup on root initiates
a metadata self heal because of ctime xattr
not being same on all afr subvolumes. This
results in loc_copy and hence the failure
of lookup.

Fix would be to send path with "." for the root.

fixes: bz#1582516
Change-Id: Iafe4b99f249a4f5034ad34c1d30590de0e35aa0d
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2018-05-28 04:57:22 +00:00
Kaleb S. KEITHLEY
dcccfeb887 api: missing __THROW on pub function decls
missing __THROW on pub function decls needed for C++

Change-Id: Ia0ff09d311741e4cdc8ef171e07bbd56c24b9260
updates: bz#1193929
Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
2018-05-25 08:18:50 +00:00
Susant Palai
fc17daf2e6 core: make glfs_iobuf_copy() consumable for general purpose.
Currently plugins for cloudsync will be using it to write back data
downloaded from remote store/cloud.

Change-Id: I59f10bebed21b19568c94cbf29e3d536d5570749
Updates: #387
Signed-off-by: Susant Palai <spalai@redhat.com>
2018-05-24 09:32:09 +00:00