1208 Commits

Author SHA1 Message Date
Humble Devassy Chirammal
fb6858b475 inode: 'this' has been set unwantedly.
Problem:
  CC       libglusterfs_la-inode.lo
inode.c: In function 'inode_table_destroy':
inode.c:1630:19: warning: variable 'this' set
but not used [-Wunused-but-set-variable]
         xlator_t *this = NULL;

Change-Id: If4b37ab896ee0a309826d4be48c6599d6ec2710b
Signed-off-by: Humble Devassy Chirammal <hchiramm@redhat.com>
Reviewed-on: http://review.gluster.org/9846
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anoop C S <achiraya@redhat.com>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Reviewed-by: Poornima G <pgurusid@redhat.com>
2015-03-17 02:02:09 -07:00
Humble Devassy Chirammal
38e342ca4a xlator: avoiding possibility of a crash if (xl->ctx) is NULL.
Change-Id: I41acd9970bef04bb16cd4d8532a84a95d5fb642a
BUG: 1199003
Signed-off-by: Humble Devassy Chirammal <hchiramm@redhat.com>.
Reviewed-on: http://review.gluster.org/9810
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2015-03-15 20:01:23 -07:00
Niels de Vos
aa66b8404f gNFS: Export / Netgroup authentication on Gluster NFS mount
* Parses linux style export file/netgroups file into a structure that
  can be lookedup.

* This parser turns each line into a structure called an "export
  directory". Each of these has a dictionary of hosts and netgroups
  which can be looked up during the mount authentication process.
  (See Change-Id Ic060aac and I7e6aa6bc)

* A string beginning withan '@' is treated as a netgroup and a string
  beginning without an @ is a host.
  (See Change-Id Ie04800d)

* This parser does not currently support all the options in the man page
  ('man exports'), but we can easily add them.

BUG: 1143880
URL: http://www.gluster.org/community/documentation/index.php/Features/Exports_Netgroups_Authentication
Change-Id: I181e8c1814d6ef3cae5b4d88353622734f0c0f0b
Original-author: Shreyas Siravara <shreyas.siravara@gmail.com>
CC: Richard Wareing <rwareing@fb.com>
CC: Jiffin Tony Thottan <jthottan@redhat.com>
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: http://review.gluster.org/8758
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2015-03-15 07:01:38 -07:00
Niels de Vos
1cb3b1abed nfs: more fine grained authentication for the MOUNT protocol
The /etc/exports format for NFS-exports (see Change-Id I7e6aa6b) allows
a more fine grained control over the authentication. This change adds
the functions and structures that will be used in by Change-Id I181e8c1.

BUG: 1143880
Change-Id: Ic060aac7c52d91e08519b222ba46383c94665ce7
Original-author: Shreyas Siravara <shreyas.siravara@gmail.com>
CC: Richard Wareing <rwareing@fb.com>
CC: Jiffin Tony Thottan <jthottan@redhat.com>
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: http://review.gluster.org/9362
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2015-03-15 01:37:22 -07:00
vmallika
9f71b25f34 syncop: assign lk_owner for the newly created frame
syncop_inodelk doesn't work properly as lk_owner is not set
in the frame created by 'synctask_create'.
There is a possibility that more than one thread can acquire inode lock
with syncop_inodelk

Change-Id: I8193edb0d24b3a6e3a3f6a0c5d7ab5a1be8e7daf
BUG: 1188636
Signed-off-by: vmallika <vmallika@redhat.com>
Reviewed-on: http://review.gluster.org/9858
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com>
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2015-03-14 20:18:04 -07:00
Poornima G
e64415ad9d libglusterfs: Replace pipe2 with pipe.
pipe2() doesn't works on Linux kernel version < 2.6.27 and 
glibc < version 2.9. Hence replacing it with pipe(),
so that the build will not fail on Centos5.

Change-Id: If17aed0d51466cd7528cf8dde0edfa28b68139e5
BUG: 1200255
Signed-off-by: Poornima G <pgurusid@redhat.com>
Reviewed-on: http://review.gluster.org/9844
Reviewed-by: Raghavendra Talur <rtalur@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com>
2015-03-10 19:20:52 -07:00
Niels de Vos
72dc1025dc posix: add ACL translation for the GF_POSIX_ACL_*_KEY xattr
Adding support for two virtual extended attributes that are used for
converting a binary POSIX ACL to a POSIX.1e long ACL text format. This
makes it possible to transfer the ACL over the network to a different OS
which can convert the POSIX.1e text format to its native structures.

The following xattrs are sent over RPC in SETXATTR/GETXATTR procedures,
and contain the POSIX.1e long ACL text format:

- glusterfs.posix.acl: maps to ACL_TYPE_ACCESS
- glusterfs.posix.default_acl: maps to ACL_TYPE_DEFAULT

acl_from_text() (from libacl) converts the text format into an acl_t
structure. This structure is then used by acl_set_file() to set the ACL
in the filesystem.

libacl-devel is needed for linking against libacl, so it has been added
to the BuildRequires in the .spec.

NetBSD does not support POSIX ACLs. Trying to get/set POSIX ACLs on a
storage server running NetBSD, an error will be returned with errno set
to ENOTSUP. Faking support, but not enforcing ACLs seems wrong to me.

URL: http://www.gluster.org/community/documentation/index.php/Features/Improved_POSIX_ACLs
BUG: 1185654
Change-Id: Ic5eb73d69190d3492df2f711d0436775eeea7de3
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: http://review.gluster.org/9627
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: soumya k <skoduri@redhat.com>
Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com>
2015-03-09 13:53:00 -07:00
Jiffin Tony Thottan
83eab209d8 core : using gluster-like memory allocation for parse-utility feature
Change-Id: I58dc7e0dc8d4ac4e10795e0536fcd0e1722116ed
BUG: 1143880
Signed-off-by: Jiffin Tony Thottan <jthottan@redhat.com>
Reviewed-on: http://review.gluster.org/9830
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
2015-03-09 10:42:46 -07:00
Emmanuel Dreyfus
3361bab038 glfsheal: Avoid infinite loop on exit
Make sure we do not get stuck looping forever in event_dispatch_destroy()
by limiting the retries when waiting for other threads, and by giving up
when writing to other thread fails.

This fixes regression tests hanging forever on NetBSD.

BUG: 1129939
Change-Id: I4459cfb1ab7294e8c15a21b592e0154c22abae07
Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org>
Reviewed-on: http://review.gluster.org/9825
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2015-03-09 10:42:02 -07:00
Pranith Kumar K
a25cdf135f Use common loc-touchup in fuse/server/gfapi
Change-Id: Id41fb29480bb6d22c34469339163da05b98c1a98
BUG: 1115907
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/8226
Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2015-03-08 08:24:39 -07:00
Niels de Vos
bc2e58a436 testing: Switch to cmocka the successor of cmockery2
This uses https://cmocka.org/ as the unit testing framework.

With this change, unit testing is made optional as well. We assume there
is no cmocka available while building. cmocka will be enabled by default
later on. For now, to build with cmocka run:

    $ ./configure --enable-cmocka

This change is based on the work of Andreas (replacing cmockery2 with
cmocka) and Kaleb (make cmockery2 an optional build dependency).

The only modifications I made, are additional #defines in unittest.h for
making sure the unit tests function as expected.

Change-Id: Iea4cbcdaf09996b49ffcf3680c76731459cb197e
BUG: 1067059
Merged-change: http://review.gluster.org/9762/
Signed-off-by: Andreas Schneider <asn@samba.org>
Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Change-Id: Ia2e955481c102d5dce17695a9205395a6030e985
Reviewed-on: http://review.gluster.org/9738
Tested-by: Gluster Build System <jenkins@build.gluster.com>
2015-03-05 13:51:37 -08:00
Poornima G
a42db19a5d libglusterfs: Change iobuf_pool_destroy to destroy all the arenas.
Change-Id: If5e9d4ce98f845d3b52565ac62970959e663497f
BUG: 1093594
Signed-off-by: Poornima G <pgurusid@redhat.com>
Reviewed-on: http://review.gluster.org/9699
Reviewed-by: Raghavendra Talur <rtalur@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Rajesh Joseph <rjoseph@redhat.com>
Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
2015-03-04 09:15:11 -08:00
Niels de Vos
95d5e60afb core: add generic parser utility
This generic parser will get used for parsing the netgroups and exports
files for the Gluster/NFS server. The parsing of netgroups shows how the
parser can be used (see Change-Id Ie04800d4).

BUG: 1143880
Change-Id: Id4cf2b0189ef5799c06868d211d3fcd9c8608c08
Original-author: Shreyas Siravara <shreyas.siravara@gmail.com>
CC: Richard Wareing <rwareing@fb.com>
CC: Jiffin Tony Thottan <jthottan@redhat.com>
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: http://review.gluster.org/9359
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2015-03-03 12:31:38 -08:00
Poornima G
2acfbcf34e event_pool: Add the code to destroy the poller threads and event pool gracefully.
Change-Id: I49b6ceebb45773620c318fb5d20b81623db75ab6
BUG: 1093594
Signed-off-by: Poornima G <pgurusid@redhat.com>
Reviewed-on: http://review.gluster.org/9691
Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
2015-03-03 06:45:21 -08:00
Poornima G
fc54f75ea4 libglusterfs: Add functions for xlator and graph cleanup.
Change-Id: If341e3c0a559aa5bbca9c1263a241c6592c59706
BUG: 1093594
Signed-off-by: Poornima G <pgurusid@redhat.com>
Reviewed-on: http://review.gluster.org/9696
Reviewed-by: Rajesh Joseph <rjoseph@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2015-03-02 20:34:38 -08:00
Poornima G
7e416b6d00 libglusterfs: Add timer thread destroy code.
Change-Id: Iafbbbfd9319751742b3c79419e1dd8e2958fee07
BUG: 1093594
Signed-off-by: Poornima G <pgurusid@redhat.com>
Reviewed-on: http://review.gluster.org/9701
Reviewed-by: Rajesh Joseph <rjoseph@redhat.com>
Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2015-03-02 20:31:56 -08:00
Poornima G
84e6eac265 syncop: Add the code to destroy the synenv processors and
syncenv structures

Change-Id: I28020eb2fc08d886cd7c05ff96daf7ebb4264ffe
BUG: 1093594
Signed-off-by: Poornima G <pgurusid@redhat.com>
Reviewed-on: http://review.gluster.org/9693
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2015-03-02 20:30:53 -08:00
Poornima G
5e12c658d6 libglusterfs: Add inode table cleanup APIs.
Took the inode context free code from the patch
http://review.gluster.org/#/c/4775/18/libglusterfs/src/inode.c

Change-Id: I05fc025763fe4ce61dc61503de27ec1d3a203e50
BUG: 1093594
Signed-off-by: Poornima G <pgurusid@redhat.com>
Reviewed-on: http://review.gluster.org/9700
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
2015-03-02 20:28:48 -08:00
Humble Devassy Chirammal
7c44ac371c logging: Adding msg id segment for common errors.
By this reservation, we are assigning some space for common errors
like dict_{get,set},memory accounting..etc.

Change-Id: Iee0f65b3dc4e00819f344bed01989352a4f8a87b
BUG: 1194640.
Signed-off-by: Humble Devassy Chirammal <hchiramm@redhat.com>
Reviewed-on: http://review.gluster.org/9752
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Tested-by: N Balachandran <nbalacha@redhat.com>
Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
2015-03-02 10:13:33 -08:00
Poornima G
60cff8ab4f epoll: Fix a leak in slot allocation.
When the slots_used in a table becomes 0, the table will not
get reused, leading to a leak.

This patch fixes the leak.

Change-Id: Ib86826d287368174ea7ebe0d0d64b2dec574634e
BUG: 1093594
Signed-off-by: Poornima G <pgurusid@redhat.com>
Reviewed-on: http://review.gluster.org/9725
Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com>
Reviewed-by: Prashanth Pai <ppai@redhat.com>
Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2015-03-02 00:56:07 -08:00
Shyam
c48cbccfaf epoll: Fix broken RPC throttling due to MT epoll
The RPC throttle which kicks in by setting the poll-in event on a
socket to false, is broken with the MT epoll commit. This is due
to the event handler of poll-in attempting to read as much out of
the socket till it receives an EAGAIN. Which may never happen and
hence we would be processing far more RPCs that we want to.

This is being fixed by changing the epoll from ET to LT, and
reading request by request, so that we honor the throttle.

The downside is that we do not drain the socket, but go back to
epoll_wait before reading the next request, but when kicking in
throttle, we need to anyway and so a busy connection would degrade
to LT anyway to maintain the throttle. As a result this change
should not cause deviation in the performance much for busy
connections.

Change-Id: I522d284d2d0f40e1812ab4c1a453c8aec666464c
BUG: 1192114
Signed-off-by: Shyam <srangana@redhat.com>
Reviewed-on: http://review.gluster.org/9726
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
Tested-by: Raghavendra G <rgowdapp@redhat.com>
2015-03-01 22:50:07 -08:00
Krishnan Parthasarathi
b117d4d84b socket: allow only one epoll thread to read msg fragments
__socket_read_reply function releases sock priv->lock briefly for
notifying higher layers of message's xid. This could result in other
epoll threads that are processing events on this socket to read further
fragments of the same message. This may lead to incorrect fragment
processing and result in a crash.

Change-Id: I915665b2e54ca16f2ad65970e51bf76c65d954a4
BUG: 1197118
Signed-off-by: Krishnan Parthasarathi <kparthas@redhat.com>
Signed-off-by: Shyam <srangana@redhat.com>
Reviewed-on: http://review.gluster.org/9742
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
Tested-by: Raghavendra G <rgowdapp@redhat.com>
2015-02-27 21:16:11 -08:00
Pranith Kumar K
d5624b2d4b libglusterfs: Moved common functions as utils in syncop/common-utils
These will be used by both afr and ec. Moved syncop_dirfd, syncop_ftw,
syncop_dir_scan functions also into syncop-utils.c

Change-Id: I467253c74a346e1e292d36a8c1a035775c3aa670
BUG: 1177601
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/9740
Reviewed-by: Krutika Dhananjay <kdhananj@redhat.com>
Reviewed-by: Anuradha Talur <atalur@redhat.com>
Reviewed-by: Ravishankar N <ravishankar@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2015-02-27 01:28:35 -08:00
Anuradha
9112806b02 cluster/afr : provide split-brain info by using getxattr
This patch is one part to enable users analyze and resolve
split-brain.

Problem : To know if a file is in data/metadata split-brain

Solution : Performing "getfattr -n afr.split-brain-status
<path-to-file>" from the mount provides this information.
Also provides the list of afr children to analyse to
get more information.

Change-Id: I4d9b429794759a906371416cb84c84a212e2c7b9
BUG: 1191396
Signed-off-by: Anuradha <atalur@redhat.com>
Reviewed-on: http://review.gluster.org/9633
Reviewed-by: Ravishankar N <ravishankar@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Krutika Dhananjay <kdhananj@redhat.com>
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
Tested-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
2015-02-25 22:32:16 -08:00
Pranith Kumar K
5d59402af8 features/index: Add option to track specific xattrs for xattrop64
This enables trusted.ec.dirty to be tracked in index

Change-Id: Ief1619110859f6f9ccee3da229f0688b73e2124b
BUG: 1177601
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/9602
Reviewed-by: Krutika Dhananjay <kdhananj@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
2015-02-24 21:57:53 -08:00
Soumya Koduri
bf22868f82 libglusterfs: Added support to set 'frame->root->lkowner'
This support can be used by the clients using SYNCOP framework,
to pass unique owners for various locks taken on a file, so that
the glusterfs-server can treat them as being locks from different owners.

Change-Id: Ie88014053af40fc7913ad6c1f7730d54cc44ddab
BUG: 1186713
Signed-off-by: Soumya Koduri <skoduri@redhat.com>
Reviewed-on: http://review.gluster.org/9482
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
2015-02-23 04:51:43 -08:00
Nithya Balachandran
f1c4ce0e22 Storage/posix : Adding error checks in path formation
Renaming directories can cause the size of the buffer
required for posix_handle_path to increase between the
first call, which calculates the size, and the second call
which forms the path in the buffer allocated based on
the size calculated in the first call.

The path created in the second call overflows the
allocated buffer and overwrites the stack causing the
brick process to crash.

The fix adds a buffer size check to prevent the buffer
overflow. It also checks and returns an error if the
posix_handle_path call is unable to form the path instead
of working on the incomplete path, which is likely to cause
subsequent calls using the path to fail with ELOOP.

Preventing buffer overflow and handling errors
BUG: 1113960

Change-Id: If3d3c1952e297ad14f121f05f90a35baf42923aa
Signed-off-by: Nithya Balachandran <nbalacha@redhat.com>
Reviewed-on: http://review.gluster.org/9289
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com>
2015-02-18 22:55:08 -08:00
Mohammed Rafi KC
abcb2017b0 rdma: pre-register iobuf_pool with rdma devices.
registring buffers with rdma device is a time consuming
operation. So performing registration in code path will
decrease the performance.
Using a pre registered memory will give a bettor performance,
ie, register iobuf_pool during rdma initialization. For
dynamically created arena, we can register with all the
device.

Change-Id: Ic79183e2efd014c43faf5911fdb6d5cfbcee64ca
BUG: 1187456
Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com>
Reviewed-on: http://review.gluster.org/9506
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
Tested-by: Raghavendra G <rgowdapp@redhat.com>
2015-02-17 20:09:54 -08:00
Mohammed Rafi KC
b8ed8da853 rdma: post multiple work request in a single call.
ibv_post-send will allow to send multiple work request
in a single call posting as linked list.

So if the payload count > 1, we can perform the data
operation in a single call to ibv_post_send.

Change-Id: Ib2e485cbbe6887919109e73e17d4fab595d5e65e
BUG: 1164079
Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com>
Reviewed-on: http://review.gluster.org/9327
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
Tested-by: Raghavendra G <rgowdapp@redhat.com>
2015-02-12 05:03:50 -08:00
Krishnan Parthasarathi
f18a3f30bb protocol/client: sequence CHILD_UP, CHILD_DOWN etc notifications
... from all bricks in the volume

This patch is important in the context of MT epoll. With MT epoll,
notification events from client xlators could reach cluster xlators like
afr, dht, ec, stripe etc. in different orders.

For e.g, In a distributed replicate volume of 2 bricks, namely Brick1
and Brick2, the following network events are observed by a mount
process.

- connection to Brick1 is broken.
- connection to Brick1 has been restored.

- connection to Brick2 is broken.
- connection to Brick2 has been restored.

Without establishing a total ordering of events, we can't guarantee that
cluster xlators like afr, dht perceive them in the same order.  While we
would expect afr (say) to perceive it as only one of Brick1 and Brick2
going down at any given time, it is possible for the notification of
Brick2 going offline to race with the notification of Brick1 coming back
online.

Change-Id: I78f5a52bfb05593335d0e9ad53ebfff98995593d
BUG: 1104462
Signed-off-by: Krishnan Parthasarathi <kparthas@redhat.com>
Reviewed-on: http://review.gluster.org/9591
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2015-02-07 13:25:26 -08:00
Shyam
a7f5893c92 epoll: Adding the ability to configure epoll threads
Add the ability to configure the number of event threads
for various gluster services.

Currently with the multi thread epoll patch, it is possible
to have more than one thread waiting on socket activity and
processing the same. This thread count is currently static,
which this commit makes dynamic.

The current services which use IO path, i.e brick processes,
any client process (nfs, FUSE, gfapi, heal,
rebalance, etc.a), gain 2 set parameters to control the number
of threads that are processing events. These settings are,
  - client.event-threads <n>
  - server.event-threads <n>

The client setting affects the client graph consumers, and the
server setting affects the brick processes. These are processed
and inited/reconfigured using the client/server protocol xlators.

Other services (say glusterd) would need to extend similar
configuration settings to take advantage of multi threaded event
processing.

At present glusterd is not enabled with this commit, as it does not
stand to gain from this multi-threading (as I understand it).

Change-Id: Id8422fc57a9f95a135158eb6477ccf9d3c9ea4d9
BUG: 1104462
Signed-off-by: Shyam <srangana@redhat.com>
Reviewed-on: http://review.gluster.org/9488
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2015-02-07 13:23:03 -08:00
Vijaikumar M
c61074400a epoll: edge triggered and multi-threaded epoll
- edge triggered (oneshot) polling with epoll
- pick one event to avoid multiple events getting picked up by same
  thread
  and so get better distribution of events against multiple threads
- wire support for multiple poll threads to epoll_wait in parallel
- evdata to store absolute index and not hint for epoll
- store index and gen of slot instead of fd and index hint
- perform fd close asynchronously inside event.c for multithread safety
- poll is still single threaded

Change-Id: I536851dda0ab224c5d5a1b130a571397c9cace8f
BUG: 1104462
Signed-off-by: Anand Avati <avati@redhat.com>
Signed-off-by: Vijaikumar M <vmallika@redhat.com>
Signed-off-by: Jeff Darcy <jdarcy@redhat.com>
Signed-off-by: Shyam <srangana@redhat.com>
Reviewed-on: http://review.gluster.org/3842
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2015-02-07 13:17:30 -08:00
Pranith Kumar K
2efb36047a syncop: Provide syncop_ftw and syncop_dir_scan utils
ftw provides file tree walk.
dir_scan does just a readdir not readdirp.

Also changed Afr's self-heal-daemon's crawling functions to use this.
These utils will be used by ec in future to do proactive/full healing.

Change-Id: I05715ddb789592c1b79a71e98f1e8cc29aac5c26
BUG: 1177601
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/9485
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Ravishankar N <ravishankar@redhat.com>
Reviewed-by: Krutika Dhananjay <kdhananj@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2015-02-06 14:04:32 -08:00
GauravKumarGarg
e44d4e1357 DHT: cluster.min-free-disk option should validate correctly
PROBLEM:
Previously gluster accepting input value as a percentage which is out of range
[0-100] and accepting input value as a size (unit is byte) which is fractional
for option cluster.min-free-disk.

FIX:
Now with this change it will refer to correct validation function
and it will accept value that is in range [0-100] for input value as a
percentage and unsigned integer value for input as a size (unit in byte)
for option cluster.min-free-disk.

Change-Id: Iee1962a100542e146276cfc8a4068abddee2bf2d
BUG: 1163108
Signed-off-by: Gaurav Kumar Garg <ggarg@redhat.com>
Reviewed-on: http://review.gluster.org/9104
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2015-02-04 22:02:42 -08:00
Anand
3a332a834a libglusterfs : Corrected functions calls.
Problem  : There was mismatch between arguments and parameters in some functions
(ex:glusterfs_uuid_buf_get,glusterfs_lkowner_buf_get). It could lead to stack overflow issues .

Fix      : Arguments are removed during calling these function.

Change-Id: Icb41bd4119502d192d9cc7242d385ebe62cdb51a
BUG: 1180424
Signed-off-by: Anand <anekkunt@redhat.com>
Reviewed-on: http://review.gluster.org/9427
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Gaurav Kumar Garg <ggarg@redhat.com>
Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2015-02-04 07:43:07 -08:00
Vijay Bellur
88136b53f5 libglusterfs: Avoid initializing per process globals more than once.
gfapi consumers can invoke glusters_globals_init() multiple times
through glfs_new(). This will result in re-initialization of already
inited variables and non deterministic behavior. To avoid this, a
new function gf_globals_init_once() has been added. The invocation
of this function is guarded through pthread_once(), thereby ensuring
single initialization of per process globals.

Change-Id: I0ecde02ee49e0c7379c2eb0f1c879d89774ec82f
BUG: 1184366
Signed-off-by: Vijay Bellur <vbellur@redhat.com>
Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com>
Reviewed-on: http://review.gluster.org/9430
Tested-by: Gluster Build System <jenkins@build.gluster.com>
2015-01-28 06:06:19 -08:00
Raghavendra Bhat
c8a6904396 uss: disable memory accounting for the snapshot daemon
* Bring in option to disable memory accounting for a glusterfs process
  This reverses the changes done by the commit
  7fba3a88f1ced610eca0c23516a1e720d75160cd.

* Change the key from "memory-accounting" to "no-memory-accounting", as by
  default all the glusterfs process enable memory accounting now. So to
  disable memory accounting for some process, "no-mem-accounting" argument has
  to be passed.

Change-Id: I39c7cefb0fe764ea3e48f4e73e1305b084c5f497
BUG: 1184366
Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com>
Reviewed-on: http://review.gluster.org/9469
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2015-01-28 06:05:24 -08:00
Pranith Kumar K
7510d8edf4 mgmt/glusterd: Implement Volume heal enable/disable
For volumes with replicate, disperse xlators, self-heal daemon should do
healing. This patch provides enable/disable functionality for the xlators to be
part of self-heal-daemon. Replicate already had this functionality with
'gluster volume set cluster.self-heal-daemon on/off'. But this patch makes it
uniform for both types of volumes. Internally it still does 'volume set' based
on the volume type.

Change-Id: Ie0f3799b74c2afef9ac658ef3d50dce3e8072b29
BUG: 1177601
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/9358
Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Xavier Hernandez <xhernandez@datalab.es>
Tested-by: Krishnan Parthasarathi <kparthas@redhat.com>
2015-01-20 02:24:24 -08:00
Aravinda VK
dec4700c66 features/changelog: Cleanup .processing and .current directory
On changelog_register cleanup .processing, .history/.processing,
.current and .history/.current from the working directory.

Moved glusterd_recursive_rmdir and glusterd_for_each_entry to common
place(libglusterfs) and renamed as recursive_rmdir and
GF_FOR_EACH_ENTRY_IN_DIR respectively

BUG: 1162057
Change-Id: I1f98468a344cead039026762a805437b2f9e507b
Signed-off-by: Aravinda VK <avishwan@redhat.com>
Reviewed-on: http://review.gluster.org/9082
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Tested-by: Venky Shankar <vshankar@redhat.com>
2015-01-18 04:30:42 -08:00
Emmanuel Dreyfus
2466f4debb NetBSD portability fix: recover errno on runner error
Some time ago we introduced F_CLOSEM to efficiently close unused
file descriptors when using a runner. But since it also close the
file descriptor used to send back errno to glusterd, it got unable
to detect an error on execve().

Fix this by backing out F_CLOSEM usage.

BUG: 1129939
Change-Id: I40d3255555145e04e8feafaa2ff4e5fb1570e9a2
Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org>
Reviewed-on: http://review.gluster.org/9447
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2015-01-16 01:05:14 -08:00
Ravishankar N
8beaf169e3 cluster/afr: split-brain resolution CLI
Extend the AFR heal command to include automated split-brain resolution.

This patch [3/3] is the final patch for afr automated split-brain resolution
implementation.

"gluster volume heal <VOLNAME> [full | statistics [heal-count [replica
<HOSTNAME:BRICKNAME>]] |info [healed | heal-failed | split-brain]| split-brain
{bigger-file <FILE> |source-brick <HOSTNAME:BRICKNAME> [<FILE>]}]"

The new additions being:
1.gluster volume heal <VOLNAME> split-brain bigger-file <FILE>
Locates the replica containing the FILE, selects bigger-file as source and
completes heal.

2.gluster volume heal <VOLNAME> split-brain source-brick <HOSTNAME:BRICKNAME>
<FILE>
Selects <FILE> present in <HOSTNAME:BRICKNAME> as source and completes heal.

3.gluster volume heal <VOLNAME> split-brain <HOSTNAME:BRICKNAME>
Selects all split-brained files in <HOSTNAME:BRICKNAME> as source and completes
heal.

Note: <FILE> can be either the full file name as seen from the root of the
volume (or) the gfid-string representation of the file, which sometimes gets
displayed in the heal info command's output.

Entry/gfid split-brain resolution is not supported.

Example can be found in the test case.

Change-Id: I4649733922d406f14f28ee9033a5cb627b9538b3
BUG: 1136769
Signed-off-by: Ravishankar N <ravishankar@redhat.com>
Reviewed-on: http://review.gluster.org/9377
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
Tested-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
2015-01-15 01:28:37 -08:00
Ravishankar N
c4ab37c02e libglusterfs: change signature of syncop_(f)getxattr
Pass xdata dict to syncop_(f)getxattr calls.

This patch [1/3] is required as a part of afr automated split-brain resolution
implementation.

Change-Id: I3970b3dd6daf64681a031e37f8e9afb14fb3d668
BUG: 1136769
Signed-off-by: Ravishankar N <ravishankar@redhat.com>
Reviewed-on: http://review.gluster.org/9375
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2015-01-05 22:00:09 -08:00
Kotresh HR
5e7e12694e feature/changelog: Virtual xattr to trigger explicit sync in geo-rep.
A virtual xattr "glusterfs.geo-rep.trigger-sync" is provided
in glusterfs through changelog translator. Geo-rep triggers
a explicit data sync on setting this xattr on a file.
Changelog captures a DATA entry on file's gfid on setting this
virtual xattr on a file. This is supported only for files. It
doesn't support directories.

Usage: setfattr -n glusterfs.geo-rep.trigger-sync <file-path>

Change-Id: Ia689326ac2dcb31035ffbecad2c548eda4eb9245
BUG: 1176934
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Reviewed-on: http://review.gluster.org/9337
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
Tested-by: Venky Shankar <vshankar@redhat.com>
2014-12-29 09:08:05 -08:00
Anuradha
85427a23c2 cluster/afr : Change in volume heal info command
gluster volume heal <volname> info command
will now also display if the files listed (in the output
of the command) are in split-brain or possibly being
healed.

This patch also fixes build warning that occurs.
Change-Id: I1fc92e62137f23b2b9ddf6e05819cee6230741d1
BUG: 1163804
Signed-off-by: Anuradha <atalur@redhat.com>
Reviewed-on: http://review.gluster.org/9119
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
Tested-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
2014-12-23 07:09:23 -08:00
Emmanuel Dreyfus
e398f99d9a telldir()/seekdir() portability fixes
POSIX says that an offset obtained from telldir() can only be used
on the same DIR *. Linux is abls to reuse the offset accross
closedir()/opendir() for a given directory, but this is not portable
and such a behavior should be fixed.

An incomplete fix for the posix xlator was merged in
http://review.gluster.com/8926
This change set completes it.

- Perform the same fix index xlator.
- Use appropriate casts and variable types so that 32 bit signed
  offsets obtained by telldir() do not get clobbered when copied into
  64 bit signed types.
- modify glfs-heal.c and afr-self-heald.c so that they do not use
  anonymous fd, since this will cause closedir()/opendir() between
  each syncop_readdir(). On failure we fallback to anonymous fs
  only for Linux so that we can cope with updated client vs not
  updated brick.
- Avoid sending an EINVAL when the client request for the EOF offset.
  Here we fix an error in previous fix for posix xlator: since we
  fill each directory entry with the offset of the next entry, we
  must consider as EOF the offset of the last entry, and not the
  value of telldir() after we read it.
- Add checks in regression tests that we do not hit cases where
  offsets fed to seekdir() are wrong. Introduce log_newer() shell
  function to check for messages produced by the current script.

This fix gather changes from http://review.gluster.org/9047
and http://review.gluster.org/8936 making them obsolete.

BUG: 1129939
Change-Id: I59fb7f06a872c4f98987105792d648141c258c6a
Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org>
Reviewed-on: http://review.gluster.org/9071
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com>
Tested-by: Raghavendra Bhat <raghavendra@redhat.com>
2014-12-17 03:48:22 -08:00
Niels de Vos
9ca8003921 client_t: fix for potential NULL pointer dereference
In case an error occurs, 'client' is free'd. The log message just before
exiting the function should therefore not use the structure anymore.

BUG: 789278
Change-Id: I0848328b29585057cd037a5972c4e5f06a7f978b
CID: 1226165
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: http://review.gluster.org/9262
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
2014-12-12 01:19:05 -08:00
Krutika Dhananjay
ad1ed80060 cluster/afr: Associate the inode returned by inode_link() with corresponding entry
Change-Id: Ic4436a64075a2615a2293cdfdf2ba6622827cafa
BUG: 1129939
Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com>
Reviewed-on: http://review.gluster.org/9254
Reviewed-by: Emmanuel Dreyfus <manu@netbsd.org>
Tested-by: Emmanuel Dreyfus <manu@netbsd.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
2014-12-09 20:15:21 -08:00
Anoop C S
cd6ffa93dc libgfapi: Wait for GF_EVENT_CHILD_DOWN in glfs_fini()
Whenever glfs_fini() is being called, currently no
check is made inside the function to determine whether
the child is already down or not. This patch will wait
for GF_EVENT_CHILD_DOWN for the active subvol and
then exits.

TBD:
Apart from the active subvol, wait for other CHILD_DOWN
events generated through operations like volume set in
future.

Change-Id: I81c64ac07b463bfed48bf306f9e8f46ba0f0a76f
BUG: 1153610
Signed-off-by: Anoop C S <achiraya@redhat.com>
Reviewed-on: http://review.gluster.org/9060
Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
2014-12-08 01:54:45 -08:00
Poornima Gurusiddaiah
8f07191a4a dict: Remove the redundant hash calculation when the hash size is 1
Currently the dict is created with hash size 1, i.e. there is
only one hash bucket and the calculation of hash decomes redundant.

Change-Id: Id70aea0d798902494ebb6d82955d97d591bc73d2
BUG: 789278
Signed-off-by: Poornima Gurusiddaiah <pgurusid@redhat.com>
Reviewed-on: http://review.gluster.org/8211
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Raghavendra Talur <rtalur@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2014-11-26 00:10:15 -08:00
Pranith Kumar K
ff9ffab7b4 dict: Write dict_foreach_* in form of dict_foreach_match
Change-Id: Iaa3454f7f3b6516660b1976bea63e39ea7795f8f
BUG: 1164051
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/9121
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
2014-11-25 09:50:30 -08:00