533 Commits

Author SHA1 Message Date
Pranith Kumar K
1350aecc7e protocol/client: Print correct error message
Change-Id: Ic68626c4a205cd78b60831aa7bd838b6d8824fa1
BUG: 796195
Signed-off-by: Pranith Kumar K <pranithk@gluster.com>
Reviewed-on: http://review.gluster.com/2800
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
2012-02-22 07:09:32 -08:00
Amar Tumballi
67104b716a mempool: adjustments in pool sizes
* while creating 'rpc_clnt', the caller knows what would be the ideal
  load on it, so an extra argument to set some pool sizes

* while creating 'rpcsvc', the caller knows what would be the ideal
  load of it, so an extra argument to set request pool size

* cli memory footprint is reduced

Change-Id: Ie245216525b450e3373ef55b654b4cd30741347f
Signed-off-by: Amar Tumballi <amarts@redhat.com>
BUG: 765336
Reviewed-on: http://review.gluster.com/2784
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vijay@gluster.com>
2012-02-22 04:23:48 -08:00
Amar Tumballi
0ef7e763c8 core: utilize mempool for frame->local allocations
in each translator, which uses 'frame->local', we are using
GF_CALLOC/GF_FREE, which would be costly considering the
number of allocation happening in a lifetime of 'fop'. It
would be good to utilize the mem pool framework for xlator's
local structures, so there is no allocation overhead.

Change-Id: Ida6e65039a24d9c219b380aa1c3559f36046dc94
Signed-off-by: Amar Tumballi <amar@gluster.com>
BUG: 765336
Reviewed-on: http://review.gluster.com/2772
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2012-02-21 02:42:09 -08:00
Amar Tumballi
bca46fd46b rpc/clnt: handle PARENT_DOWN event appropriately
Change-Id: I4644e944bad4d240d16de47786b9fa277333dba4
BUG: 767862
Signed-off-by: Raghavendra G <raghavendra@gluster.com>
Signed-off-by: Amar Tumballi <amarts@redhat.com>
Reviewed-on: http://review.gluster.com/2735
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vijay@gluster.com>
2012-02-21 01:42:42 -08:00
Mohammed Junaid
f764516c2e protocol/client,server: fcntl lock self healing.
Currently(with out this patch), on a disconnect the server cleans up
the transport which inturn closes the fd's and releases the locks acquired on
those fd's by that client. On a reconnect, client just reopens the fd's but
doesn't reacquire the locks. The application that had previously acquired
the locks still is under the assumption that it is the owner of those locks
which might have been granted to other clients(if they request) by the server
leading to data corruption.

This patch allows the client to reacquire the fcntl locks (held on the fd's)
during client-server handshake.

* The server identifies the client via process-uuid-xl (which is a combination
  of uuid and client-protocol name, it is assumed to be unique) and lk-version
  number.

* The client maintains a list of process-uuid-xl, lk-version pair for each
  accepted connection. On a connect, the server traverses the list for a
  matching pair, if a matching pair is not found the the server returns
  lk-version with value 0, else it returns the lk-version it has in store.

* On a disconnect, the server and client enter grace period, and on the
  completion of the grace period, the client bumps up its lk-version number
  (which means, it will reacquire the locks the next time) and the server will
  distroy the connection. If reconnection happens within the grace period, the
  server will find the matching (process-uuid-xl, lk-version) pair in its list
  which guarantees that the fd's and there corresponding locks are still valid
  for this client.

Configurable options:
  To set grace-timeout, the following options are
    option server.grace-timeout value
    option client.grace-timeout value

  To enable or disable the lk-heal,
    option lk-heal [on|off]

gluster volume set command can be used to configurable options
Change-Id: Id677ef1087b300d649f278b8b2aa0d94eae85ed2
BUG: 795386
Signed-off-by: Mohammed Junaid <junaid@redhat.com>
Reviewed-on: http://review.gluster.com/2766
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vijay@gluster.com>
2012-02-20 04:45:31 -08:00
Rajesh Amaravathi
6827f56dfc protocol/client: delete locks only for non-anonymous fds
delete_granted_lock_owners () is not called for anonymous
fds since they are not involved in locking

Change-Id: Icdc7818f98f5371232ba276ed442704ef69e6b0e
BUG: 787365
Signed-off-by: Rajesh Amaravathi <rajesh@redhat.com>
Reviewed-on: http://review.gluster.com/2754
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Reviewed-by: Vijay Bellur <vijay@gluster.com>
2012-02-20 00:49:44 -08:00
Amar Tumballi
c0b8e886ca iobuf: use 'iobuf_get2()' to get variable sized buffers
added 'TODO' in places where it is missing.

Change-Id: Ia802c94e3bb76930f7c88c990f078525be5459f5
Signed-off-by: Amar Tumballi <amar@gluster.com>
BUG: 765264
Reviewed-on: http://review.gluster.com/388
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vijay@gluster.com>
2012-02-20 00:46:27 -08:00
Rajesh Amaravathi
975933a25d glusterd: auth allow enhancements
* PROBLEM:

  When address-based authentication is enabled on a volume,
  the gNfs server, self-heal daemon (shd), and other operations
  such as quota, rebalance, replace-brick and geo-replication
  either stop working or the services are not started if all
  the peers' ipv{4,6} addresses or hostnames are not added in
  the "set auth.allow" operation, breaking the functionality
  of several operations.

  E.g:
    volume vol in a cluster of two peers:
    /mnt/brick1 in 192.168.1.4
    /mnt/brick2 in 192.168.1.5

    option auth.allow 192.168.1.6
    (allow connection requests only from 192.168.1.6)

    This will disrupt the nfs servers on 192.168.1.{4,5}.
    brick server processes reject connection requests from both
    nfs servers (on 4,5), because the peer addresses are not in
    the auth.allow list.

    Same holds true for local mounts (on peer machines),
    self-heal daemon, and other operations which perform
    a glusterfs mount on one of the peers.

* SOLUTION:

  Login-based authentication (username/password pairs,
  henceforth referred to as "keys") for gluster services and
  operations.

  These *per-volume* keys can be used to by-pass the addr-based
  authentication, provided none of the peers' addresses are put
  in the auth.reject list, to enable gluster services like gNfs,
  self-heal daemon and internal operations on volumes when
  auth.allow option is exercised.

* IMPLEMENTATION:

  1. Glusterd generates keys for each volume and stores it in
     memory as well as in respective volfiles.
     A new TRUSTED-FUSE volfile is generated which is
     fuse volfile + keys in protocol/client,
     and is named trusted-<volname>-fuse.vol.
     This is used by all local mounts. ANY local mount (on any peer)
     is granted the trusted-fuse volfile instead of fuse volfile
     via getspec. non-local mounts are NOT granted the trusted fuse
     volfile.

  2. The keys generated for the volume is written to each server
     volfile telling servers to allow users with these keys.

  3. NFS, self-heal daemon and replace-brick volfiles are updated
     with the volume's authentication keys.

  4. The keys are NOT written to fuse volfiles for obvious reasons.

  5. The ownership of volfiles and logfiles is restricted to root users.

  6. Merging two identical definitions of peer_info_t in auth/addr
     and rpc-lib, throwing away the one in auth/addr.

  7. Code cleanup in numerous places as appropriate.

* IMPORTANT NOTES:

  1. One SHOULD NOT put any of the peer addresses in the auth.reject
     list if one wants any of the glusterd services and features
     such as gNfs, self-heal, rebalance, geo-rep and quota.

  2. If one wants to use username/password based authentication
     to volumes, one shall append to the server, nfs and shd volfiles,
     the keys one wants to use for authentication, *while_retaining
     those_generated_by_glusterd*.
     See doc/authentication.txt file for details.

Change-Id: Ie0331d625ad000d63090e2d622fe1728fbfcc453
BUG: 789942
Signed-off-by: Rajesh Amaravathi <rajesh@redhat.com>
Reviewed-on: http://review.gluster.com/2733
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vijay@gluster.com>
2012-02-20 00:30:54 -08:00
Amar Tumballi
7fa06c4ce1 protocol: remove the 'path<>' from rename() and link()
missed it in the previous round of cleanup, path is completely
useless in resolve function.

Change-Id: I1aef0f5276afb77dfacfcc0c337ac80b4fcacc55
Signed-off-by: Amar Tumballi <amar@gluster.com>
BUG: 790298
Reviewed-on: http://review.gluster.com/2756
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2012-02-16 22:49:15 -08:00
Rajesh Amaravathi
1fbed11ead nfs: avoid logging invalid fdctx in case of anonymous fds
if get_fd_ctx fails (as in case of anonymous fds), overwhelming
amount of entries are seen in the nfs log, causing dd and other
heavy i/o operations to become unresponsive. this patch logs an
invalid fdctx only if it is not an anonymous fd.

Change-Id: I4e917d150d6a053af77d47a94a2f1c2633acadb5
BUG: 787365
Signed-off-by: Rajesh Amaravathi <rajesh@redhat.com>
Reviewed-on: http://review.gluster.com/2747
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
2012-02-14 10:25:30 -08:00
Amar Tumballi
5f002b5418 protocol xdr: remove 'path<>'
client was sending 'path' on wire, which gets ignored on server
side, and also doesn't get freed up, which causes memory leak.

also with not having path on wire, the xdr size on wire most of the
time can remain constant, which helps in allocating RDMA buffers.

Change-Id: Ie0d36a670be60b02fd1e925c6f977b1a71def5cd
BUG: 790298
Signed-off-by: Amar Tumballi <amar@gluster.com>
Reviewed-on: http://review.gluster.com/2744
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2012-02-14 04:34:06 -08:00
Amar Tumballi
0e6df0100e core: add an extra flag to readv()/writev() API
needed to implement a proper handling of open flag alterations
using fcntl() on fd.

Change-Id: Ic280d5db6f1dc0418d5c439abb8db1d3ac21ced0
Signed-off-by: Amar Tumballi <amar@gluster.com>
BUG: 782265
Reviewed-on: http://review.gluster.com/2723
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2012-02-14 01:09:02 -08:00
Amar Tumballi
e17ac220e4 protocol: code cleanup
make dict serialize and unserialization code a macro

Change-Id: I459c77c6c1f54118c6c94390162670f4159b9690
BUG: 764890
Signed-off-by: Amar Tumballi <amar@gluster.com>
Reviewed-on: http://review.gluster.com/2742
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2012-02-14 01:00:26 -08:00
Amar Tumballi
0272f7de41 protocol/client: assign the right value to 'conf' before de-refing it
variable assignment was done after it was actually getting de-referenced.
moved the assignment few lines up.

Change-Id: Id65e3e2d3dfe071e1c5b14c32488647070398ae4
BUG: 787117
Signed-off-by: Amar Tumballi <amar@gluster.com>
Reviewed-on: http://review.gluster.com/2712
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
2012-02-07 04:21:44 -08:00
Kaushal M
b708b18b83 cli, protocol/server : improve validation for the option auth.(allow/reject)
cli now checks validity of address list given for 'volume set auth.*'

Server xlator checks addresses supplied to auth.(allow/reject) option
including wildcards for correctness in case volfile is manually edited.

Original patch done by shylesh@gluster.com
Original patch  is at http://patches.gluster.com/patch/7566/

Change-Id: Icf52d6eeef64d6632b15aa90a379fadacdf74fef
BUG: 764197
Signed-off-by: Kaushal M <kaushal@redhat.com>
Reviewed-on: http://review.gluster.com/306
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-by: Vijay Bellur <vijay@gluster.com>
2012-02-05 22:19:54 -08:00
Kaushal M
08041c3212 protocol/client : prevent client from reconnecting when server
authentication fails

This prevents the client from trying to reconnect on server
authentication failure. Reconnecting on authentcation failure causes hung
mounts on unauthorised clients. This patch fixes this problem.

Also, mount.glusterfs script unmounts mount-point on mount failure to
prevent hung mounts.

Change-Id: I5615074d27948077bad491a38cecae1b7f5159fb
BUG: 765240
Signed-off-by: Kaushal M <kaushal@redhat.com>
Reviewed-on: http://review.gluster.com/398
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Amar Tumballi <amar@gluster.com>
2012-02-03 07:41:51 -08:00
Kaushal M
623919a78a cli: Extend "volume status" with statedump info
This patch enhances and extends the "volume status" command with information
obtained from the statedump of the bricks of volumes.

Adds new status types : clients, inode, fd, mem, callpool
The new syntax of "volume status" is,
 #gluster volume status [all|{<volname> [<brickname>]
                         [misc-details|clients|inode|fd|mem|callpool]}]

Change-Id: I8d019718465bbc3de727653a839de7238f45da5c
BUG: 765495
Signed-off-by: Kaushal M <kaushal@redhat.com>
Reviewed-on: http://review.gluster.com/2637
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Krishnan Parthasarathi <kp@gluster.com>
2012-01-27 04:20:04 -08:00
Vijay Bellur
6b0cd9fc84 protocol/client: Pass the right arguments to CLIENT_GET_REMOTE_FD
Change-Id: I04f984f20964650a38009bba7711d2757151ade5
BUG: 762935
Signed-off-by: Vijay Bellur <vijay@gluster.com>
Reviewed-on: http://review.gluster.com/2694
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@gluster.com>
2012-01-26 05:23:33 -08:00
Raghavendra Bhat
fd61cd598b protocol/client: if the remote_fd is -1, then unwind instead of sending the call to server
For calls with remote_fd set to -1, client xlator is sending the call to the
server which results in server not resolving it and thus fd being NULL. Locks
xlator when tries to get the inode context using the fd it segfaults. To avoid
it unwind the call in the client xlator if the remote_fd is -1.

Change-Id: Ic34a49fdf1012dd371f4b194703c0be74f29bda2
BUG: 784187
Signed-off-by: Raghavendra Bhat <raghavendrabhat@gluster.com>
Reviewed-on: http://review.gluster.com/2684
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-by: Krishnan Parthasarathi <kp@gluster.com>
Reviewed-by: Anand Avati <avati@gluster.com>
2012-01-25 08:08:38 -08:00
Amar Tumballi
d7ecaaa1ed core: add 'fremovexattr()' fop
so operations can be done on fd for extended attribute removal

Change-Id: Ie026f1b53793aeb4ae33e96ea5408c7a97f34bf6
Signed-off-by: Amar Tumballi <amar@gluster.com>
BUG: 766571
Reviewed-on: http://review.gluster.com/778
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@gluster.com>
2012-01-25 02:24:20 -08:00
Amar Tumballi
cf8486cbef core: get xattrs also as part of readdirp
readdirp_req() call sends a dict_t * as an argument, which
contains all the xattr keys for which the entries got in
readdirp_rsp() are having xattr value filled dictionary.

Change-Id: I8b7e1290740ea3e884e67d19156ce849227167c0
Signed-off-by: Amar Tumballi <amar@gluster.com>
BUG: 765785
Reviewed-on: http://review.gluster.com/771
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@gluster.com>
2012-01-25 02:03:44 -08:00
Amar Tumballi
b02afc6d00 core: change lk-owner as a 1k buffer
so, NLM can send the lk-owner field directly to the locks translators,
while doing the same effort, also enabled sending maximum of 500 aux gid
over protocol.

Change-Id: I87c2514392748416f7ffe21d5154faad2e413969
Signed-off-by: Amar Tumballi <amar@gluster.com>
BUG: 767229
Reviewed-on: http://review.gluster.com/779
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@gluster.com>
2012-01-24 20:14:17 -08:00
Csaba Henk
4e92c58d27 rpc: extend actors with flag signing if privilege is required
Currently we allow the following RPC messages for unprivileged users:
GLUSTER_CLI_GETWD, GLUSTER_CLI_MOUNT, GLUSTER_CLI_UMOUNT

Change-Id: I05414f3ca7cbe47de45c5e5cfba1537efc774e6c
BUG: 781256
Signed-off-by: Csaba Henk <csaba@gluster.com>
Reviewed-on: http://review.gluster.com/2641
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@gluster.com>
2012-01-21 05:33:19 -08:00
Anand Avati
7e1f8e3bac core: GFID filehandle based backend and anonymous FDs
1. What
--------
This change introduces an infrastructure change in the filesystem
which lets filesystem operation address objects (inodes) just by its
GFID. Thus far GFID has been a unique identifier of a user-visible
inode. But in terms of addressability the only mechanism thus far has
been the backend filesystem path, which could be derived from the
GFID only if it was cached in the inode table along with the entire set
of dentry ancestry leading up to the root.

This change essentially decouples addressability from the namespace. It
is no more necessary to be aware of the parent directory to address a
file or directory.

2. Why
-------
The biggest use case for such a feature is NFS for generating
persistent filehandles. So far the technique for generating filehandles
in NFS has been to encode path components so that the appropriate
inode_t can be repopulated into the inode table by means of a recursive
lookup of each component top-down.

Another use case is the ability to perform more intelligent self-healing
and rebalancing of inodes with hardlinks and also to detect renames.

A derived feature from GFID filehandles is anonymous FDs. An anonymous FD
is an internal USABLE "fd_t" which does not map to a user opened file
descriptor or to an internal ->open()'d fd. The ability to address a file
by the GFID eliminates the need to have a persistent ->open()'d fd for the
purpose of avoiding the namespace. This improves NFS read/write performance
significantly eliminating open/close calls and also fixes some of today's
limitations (like keeping an FD open longer than necessary resulting
in disk space leakage)

3. How
-------

At each storage/posix translator level, every file is hardlinked inside
a hidden .glusterfs directory (under the top level export) with the name
as the ascii-encoded standard UUID format string. For reasons of performance
and scalability there is a two-tier classification of those hardlinks
under directories with the initial parts of the UUID string as the directory
names.

For directories (which cannot be hardlinked), the approach is to use a symlink
which dereferences the parent GFID path along with basename of the directory.
The parent GFID dereference will in turn be a dereference of the grandparent
with the parent's basename, and so on recursively up to the root export.

4. Development
---------------

4a. To leverage the ability to address an inode by its GFID, the technique is
to perform a "nameless lookup". This means, to populate a loc_t structure as:

loc_t {
   pargfid: NULL
   parent: NULL
   name: NULL
   path: NULL
   gfid: GFID to be looked up [out parameter]
   inode: inode_new () result [in parameter]
}

and performing such lookup will return in its callback an inode_t
populated with the right contexts and a struct iatt which can be
used to perform an inode_link () on the inode (without a parent and
basename). The inode will now be hashed and linked in the inode table
and findable via inode_find().

A fundamental change moving forward is that the primary fields in a
loc_t structure are now going to be (pargfid, name) and (gfid) depending
on the kind of FOP. So far path had been the primary field for operations.
The remaining fields only serve as hints/helpers.

4b. If read/write is to be performed on an inode_t, the approach so far
has been to: fd_create(), STACK_WIND(open, fd), fd_bind (in callback) and
then perform STACK_WIND(read, fd) etc. With anonymous fds now you can do
fd_anonymous (inode), STACK_WIND (read, fd). This results in great boost
in performance in the inbuilt NFS server.

5. Misc
-------
The inode_ctx_put[2] has been renamed to inode_ctx_set[2] to be consistent
with the rest of the codebase.

Change-Id: Ie4629edf6bd32a595f4d7f01e90c0a01f16fb12f
BUG: 781318
Reviewed-on: http://review.gluster.com/669
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@gluster.com>
2012-01-20 05:03:42 -08:00
Pranith Kumar K
bddb3a014a protocol/server: Do connection cleanup if reply fails
We observed that after the first connection cleanup happens on
DISCONNECT the lock calls in transit are granted or added in
blocked locks queue. These locks were never cleaned up after that
because no unlock would come up on that connection. This would
leave references on that transport so it would never be destroyed.
Now, the connection cleanup happens whenever the reply
submission fails.

Also cleaned up the old code which is not used any more.

Change-Id: Ie4fe6f388ed18d9c907cf8ae06b0b7fd0601a660
BUG: 765430
Signed-off-by: Pranith Kumar K <pranithk@gluster.com>
Reviewed-on: http://review.gluster.com/809
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-by: Vijay Bellur <vijay@gluster.com>
2011-12-22 05:13:08 -08:00
Jeff Darcy
2a3e74e7a6 Fix local==NULL crash in wb_sync_cbk during disconnect.
Change-Id: I26dc48a85756e189b1ef5cfef1658f9c2aed2157
BUG: 767359
Signed-off-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-on: http://review.gluster.com/784
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Amar Tumballi <amar@gluster.com>
Reviewed-by: Vijay Bellur <vijay@gluster.com>
2011-12-19 21:40:32 -08:00
Pranith Kumar K
ebe0cedb07 protocol/client: Be strict about gfids in fop req
Change-Id: I7508ab3a93329bb6a679801fddfcd0e5b0c7c134
BUG: 765198
Signed-off-by: Pranith Kumar K <pranithk@gluster.com>
Reviewed-on: http://review.gluster.com/770
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-by: Vijay Bellur <vijay@gluster.com>
2011-12-15 02:50:00 -08:00
Harshavardhana
51a78ad316 glusterfs: An effort to fix all the spell mistakes and typo
in the entire glusterfs codebase.

This patch fixes many of spell mistakes and typo in the entire
glusterfs codebase and all supported modules.

Change-Id: I83238a41aa08118df3cf4d1d605505dd3cda35a1
BUG: 3809
Signed-off-by: Harshavardhana <fharshav@redhat.com>
Reviewed-on: http://review.gluster.com/731
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Amar Tumballi <amar@gluster.com>
Reviewed-by: Vijay Bellur <vijay@gluster.com>
2011-11-16 21:08:56 -08:00
Amar Tumballi
14f4e02382 core: remove 'ino' variable from 'inode_t' structure
Change-Id: I0f078d1753db65d2f2e0380d1b0450c114cf40dd
BUG: 3518
Reviewed-on: http://review.gluster.com/522
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vijay@gluster.com>
2011-11-16 03:44:32 -08:00
Sachidananda Urs
5e14417b47 Move some of the messages to log level `TRACE'.
Change-Id: I46133b5e2218b9d810251b3dadadd8f157ab07d7
BUG: 3761
Reviewed-on: http://review.gluster.com/643
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Amar Tumballi <amar@gluster.com>
2011-11-15 23:30:22 -08:00
Amar Tumballi
18071a240b libglusterfs/iobuf: increase the iobref's iobuf array size
earlier it was hardcoded to 8, now increased the size to 16.
also return the exact error code in client_submit_vec_request(),
so there will be no missing frames in case of errors.

Change-Id: I82a6ee681a543b673a7cf1a0b9c5ade2a7175ebe
BUG: 3679
Reviewed-on: http://review.gluster.com/555
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@gluster.com>
2011-10-10 03:57:10 -07:00
Raghavendra Bhat
9ef8eabae2 statedump: do not print the inode number in the statedump
Since gfid is used to uniquely identify a inode, in the statedump
    printing inode number is not necessary. Its suffecient if the gfid
    of the inode is printed. And do not print the the inodelks, entrylks
    and posixlks if the lock count is 0.

Change-Id: Idac115fbce3a5684a0f02f8f5f20b194df8fb27f
BUG: 3476
Reviewed-on: http://review.gluster.com/530
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Amar Tumballi <amar@gluster.com>
2011-10-01 05:55:22 -07:00
Raghavendra Bhat
a8270c537c protocol/server: check for the fd being NULL and unwind
Change-Id: I400e515431cf739fe0b2f90840359496a2b529d2
BUG: 3158
Reviewed-on: http://review.gluster.com/528
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Shishir Gowda <shishirng@gluster.com>
2011-09-29 09:06:56 -07:00
Kaushal M
45172a5415 cli : new volume statedump command
Changes:
        1. Add a new 'volume statedump' command, that performs statedumps of
        all the bricks in the volume and saves them in a specified location.
        2. Add new server option 'server.statedump-path'.
        3. Remove multiple function definitions in glusterd.h

Statedump Information:

The 'volume statedump' command performs statedumps on all the bricks in
a given volume. The syntax of the command is,
        gluster volume statedump <VOLNAME> [type]......

Types include,
        * all
        * mem
        * iobuf
        * callpool
        * priv
        * fd
        * inode
Defaults to 'all' when no type is specified.

The statedump files are created by default in /tmp directory of the
server on which the bricks are present.
This path can be changed by setting the 'server.statedump-path' option.

The statedump files will be named as,
        <brick-name>.<pid of brick process>.dump

Change-Id: I01c0e1a8aad490da818e086d89f292bd2ed06fd4
BUG: 1964
Reviewed-on: http://review.gluster.com/321
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Amar Tumballi <amar@gluster.com>
2011-09-27 06:45:10 -07:00
Amar Tumballi
b274b8792c glusterfs protocol: handshake to log the version of the peer
* As RPC program's name is just used for logging, we now have
  'PACKAGE_VERSION' part of the string, which gets logged in client
  side.
* From client, we send the PACKAGE_VERSION in handshake dictionary,
  which gets logged on serverside handshake.

The change doesn't break any compatibility between client or server
as it would only enhance the logging part of handshake.

Change-Id: Ie7f498af2f5d3f97be37c8d982061cb6021883ce
BUG: 3589
Reviewed-on: http://review.gluster.com/467
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vijay@gluster.com>
2011-09-21 04:28:43 -07:00
Rajesh Amaravathi
909f80181c protocol/client: minor log enhancements
minor changes to the log enhancements of bug 3473.

Change-Id: Id38d29db5a744e0ab7342d10ead6d16866228062
BUG: 3473
Reviewed-on: http://review.gluster.com/452
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Amar Tumballi <amar@gluster.com>
2011-09-19 01:53:28 -07:00
Rajesh Amaravathi
35bbb34a3f protocol/client: log enhancements
* print paths wherever it is possible to log, to help debugging.
* bring uniformity in log level.

Change-Id: I2fa85b629de5dd0f0057ed96cba08ecb0ff1a798
BUG: 3473
Reviewed-on: http://review.gluster.com/328
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Amar Tumballi <amar@gluster.com>
2011-09-18 22:28:09 -07:00
Amar Tumballi
a07bb18c8a distribute rebalance: handle the open file migration
Complexity involved: To migrate a file with open fd, we have to
notify the other client process which has the open fd, and make
sure the write()s happening on that fd is properly synced to the
migrated file. Once the migration is complete, the client
process which has open-fd should get notified and it should
start performing all the operations on the new subvolume,
instead of earlier cached volume.

How to solve the notification part:

We can overload the 'postbuf' attribute in the _cbk() function to
understand if a file is 'under-migration' or 'migration-complete'
state. (This will be something similar to deciding whether a file
is DHT-linkfile by its 'mode').

Overall change includes below mentioned major changes:

1. dht_linkfile is decided by only 2 factors (mode(01000),
   xattr(trusted.glusterfs.dht.linkto)), instead of earlier
   3 factors (size==0)

2. in linkfile self-heal part (in 'dht_lookup_everywhere_cbk()'),
   don't delete a linkfile if there is a open-fd on it. It means,
   there may be a migration in progress.

3. if a file's revalidate fails with ENOENT, it may be due to file
   migration, and hence need a lookup_everywhere()

4. There will be 2 phases of file-migration.

-> Phase 1: Migration in progress
   * The source data file will have SGID and STICKY bit set in its mode.
   * The source data file will have a 'linkto' xattr pointing the
     destination.
   * Destination file will have mode set to '01000', and 'linkto' xattr
     set to itself.

-> Phase 2: File migration Complete
   * The source data file will have mode '01000', and will be 'truncated'
     to size 0.
   * The destination file will have inherited mode from the source. (without
     sgid and sticky bit) and its 'linkto' attribute will be removed.

4. Changes in distribute to work smoothly with a file which is in migration /
   got migrated.

The 'fops' are divided into 3 categories, inode-read, inode-write and others.
inode-read fops need to handle only 'phase 2' notification, where as, the
inode-write fops need to handle both 'phase 1' and phase2. The inode-write
operations will be done on source file, and if any of 'file-migration' procedures
are detected in _cbk(), then the operations should be performed on the destination
too.

when a phase-2 is detected, then the inode-ctx itself should be changed to represent
a new layout.

With these changes, the open file migration will work smoothly with multiple clients.

Change-Id: I512408463814e650f34c62ed009bf2101d016fd6
BUG: 3071
Reviewed-on: http://review.gluster.com/209
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vijay@gluster.com>
2011-09-12 23:57:38 -07:00
Raghavendra Bhat
225dd7da89 protocol/client: avoid code duplication in fd based operations
Change-Id: I012f78bac8ba82333628c59ef51d5e5f43d05ac7
BUG: 3158
Reviewed-on: http://review.gluster.com/329
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Amar Tumballi <amar@gluster.com>
Reviewed-by: Vijay Bellur <vijay@gluster.com>
2011-09-11 23:24:38 -07:00
Jeff Darcy
694ef54978 Eliminate many "var set but not used" warnings with newer gcc.
This fixes ~200 such warnings, but leaves three categories untouched.

(1) Rpcgen code.

(2) Macros which set variables in the outer (calling function) scope.

(3) Variables which are set via function calls which may have side effects.

Change-Id: I6554555f78ed26134251504b038da7e94adacbcd
BUG: 2550
Reviewed-on: http://review.gluster.com/371
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@gluster.com>
2011-09-07 23:48:01 -07:00
Amar Tumballi
f0f3b040df modify to the way we used XDR definitions files (.x files)
Earlier:
step 1: copy the existing <xdr>.x files to /tmp
step 2: generate '.[ch]' files using 'rpcgen <xdr>.x'
step 3: check diff with the to the existing files, add only your part
        of changes back to the original file. (ignore other changes).
step 4: there is another file to write wrapper functions to convert
        structures to/from XDR buffers, update it with your new structure.
step 5: use these wrapper functions in the newly written procedures.
step 6: commit :-|

Now:
step 1: update (mostly adding only) the <xdr>.x file
step 2: run '<path-to-src>/extras/generate-xdr-files.sh <xdr>.x' command
step 3: implement rpc procedure to handle the request/response.
step 4: commit :-)

Change-Id: I219f9159fc980438c86e847c6b030be96e595ea2
BUG: 3488
Reviewed-on: http://review.gluster.com/341
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@gluster.com>
2011-09-07 10:48:52 -07:00
Amar Tumballi
f78c8253d7 glusterfs protocol: bring in variable sized iobuf support
is a step towards reducing glusterfs memory footprint. should also
help a bit in overall performance.

Change-Id: I074d5813602b2c960d59562e792b3dc6e43d2f42
BUG: 3475
Reviewed-on: http://review.gluster.com/322
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@gluster.com>
2011-09-07 10:48:29 -07:00
Vijay Bellur
08e8c96686 protocol/client: Changes to be benign to replace-brick
Change-Id: Ic227781760a5f6dbf8aad69a19f90e45d4aaec13
BUG: 3415
Reviewed-on: http://review.gluster.com/288
Reviewed-by: Krishnan Parthasarathi <kp@gluster.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
2011-08-21 06:28:18 -07:00
Anand Avati
dd2c197c1d option validation: further fixes
fixes in option handling changes

Change-Id: I0a44cdb088e3f08cd43d583a580736d0903fa88c
BUG: 3415
Reviewed-on: http://review.gluster.com/261
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@gluster.com>
2011-08-19 03:06:19 -07:00
Anand Avati
d2849bd349 xlator options: revamp xlator option validation/reconfigure code
- move option handling to options.c (new file)
- remove duplication of option validation code
- remove duplication of gf_log / sprintf
- get rid of xlator_t->validate_options
- get rid of option validation in rpc-transport
- get rid of validate_options() in every xlator
- use xlator_volume_option_get to clean up many functions
- introduce primitives to init/reconfigure option types

Change-Id: I51798af72c8dc0a2b9e017424036eb3667dfc7ff
BUG: 3415
Reviewed-on: http://review.gluster.com/235
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@gluster.com>
2011-08-18 23:49:03 -07:00
Pranith Kumar K
b58dc65f2a Change Copyright current year
Change-Id: I2d10f2be44f518f496427f257988f1858e888084
BUG: 3348
Reviewed-on: http://review.gluster.com/200
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@gluster.com>
2011-08-10 10:57:19 -07:00
krishna
1b5a860f15 pass xlator pointer to rpcsvc_init() so that it can init svc->mydata to xlator
Change-Id: Icfd95cc67400c16a951d6a9f922fbdc07f40c5b6
BUG: 3314
Reviewed-on: http://review.gluster.com/180
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vijay@gluster.com>
2011-08-09 02:12:03 -07:00
Pranith Kumar K
0cf100b58c LICENSE: s/GNU Affero General Public/GNU General Public/
Change-Id: I3914467611e573cccee0d22df93920cf1b2eb79f
BUG: 3348
Reviewed-on: http://review.gluster.com/182
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@gluster.com>
2011-08-06 06:33:52 -07:00
shishir gowda
76acac485b IOBUF: Bring in variable iobuf support
Rebasing Raghavendra's patch to master

Change-Id: Ie7a0c9a7b8e73cfe48c4573a5153460d5126208e
BUG: 2472
Reviewed-on: http://review.gluster.com/12
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Amar Tumballi <amar@gluster.com>
Reviewed-by: Anand Avati <avati@gluster.com>
2011-07-31 23:15:15 -07:00
krishna
6c7a89321a RPC unify code change
Change-Id: Ibe18a2a63fd023ac57652c4dfc8ac8a69d983b83
BUG: 3112
Signed-off-by: krishna <krishna@gluster.com>
Reviewed-on: http://review.gluster.com/116
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Amar Tumballi <amar@gluster.com>
2011-07-29 03:37:07 -07:00