79 Commits

Author SHA1 Message Date
M. Mohan Kumar
c8fef37c5d glusterfs: zerofill support
Add support for a new ZEROFILL fop. Zerofill writes zeroes to a file in
the specified range. This fop will be useful when a whole file needs to
be initialized with zero (could be useful for zero filled VM disk image
provisioning or  during scrubbing of VM disk images).

Client/application can issue this FOP for zeroing out. Gluster server
will zero out required range of bytes ie server offloaded zeroing. In
the absence of this fop,  client/application has to repetitively issue
write (zero) fop to the server, which is very inefficient method because
of the overheads involved in RPC calls  and acknowledgements.

WRITESAME is a  SCSI T10 command that takes a block of data as input and
writes the same data to other blocks and this write is handled
completely within the storage and hence is known as offload . Linux ,now
has support for SCSI WRITESAME command which is exposed to the user in
the form of BLKZEROOUT ioctl.  BD Xlator can exploit BLKZEROOUT ioctl to
implement this fop. Thus zeroing out operations can be completely
offloaded to the storage device , making it highly efficient.

The fop takes two arguments offset and size. It zeroes out 'size' number
of bytes in an opened file starting from 'offset' position.

This patch adds zerofill support to the following areas:
	- libglusterfs
	- io-stats
	- performance/md-cache,open-behind
	- quota
	- cluster/afr,dht,stripe
	- rpc/xdr
	- protocol/client,server
	- io-threads
	- marker
	- storage/posix
	- libgfapi

Client applications can exloit this fop by using glfs_zerofill introduced in
libgfapi.FUSE support to this fop has not been added as there is no system call
for this fop.

Changes from previous version 3:
* Removed redundant memory failure log messages

Changes from previous version 2:
* Rebased and fixed build error

Changes from previous version 1:
* Rebased for latest master

TODO :
     * Add zerofill support to trace xlator
     * Expose zerofill capability as part of gluster volume info

Here is a performance comparison of server offloaded zeofill vs zeroing
out using repeated writes.

[root@llmvm02 remote]# time ./offloaded aakash-test log 20

real	3m34.155s
user	0m0.018s
sys	0m0.040s
[root@llmvm02 remote]# time ./manually aakash-test log 20

real	4m23.043s
user	0m2.197s
sys	0m14.457s
[root@llmvm02 remote]# time ./offloaded aakash-test log 25;

real	4m28.363s
user	0m0.021s
sys	0m0.025s
[root@llmvm02 remote]# time ./manually aakash-test log 25

real	5m34.278s
user	0m2.957s
sys	0m18.808s

The argument log is a file which we want to set for logging purpose and
the third argument is size in GB .

As we can see there is a performance improvement of around 20% with this
fop.

Change-Id: I081159f5f7edde0ddb78169fb4c21c776ec91a18
BUG: 1028673
Signed-off-by: Aakash Lal Das <aakash@linux.vnet.ibm.com>
Signed-off-by: M. Mohan Kumar <mohan@in.ibm.com>
Reviewed-on: http://review.gluster.org/5327
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2013-11-10 21:25:49 -08:00
Anand Avati
c80794079a gfapi: remove unnecessary call to glfs_resolve_base()
Calling glfs_resolve_base() on the root inode for every resolver
invocation is unnecessary and wasteful.

Here are the results from running a test program which performs
path based operations (creates and deletes 1000 files):

Without patch:
[root@blackbox ~]# sync
[root@blackbox ~]# time ./a.out 1

real    0m4.314s
user    0m1.923s
sys     0m1.144s
[root@blackbox ~]# sync
[root@blackbox ~]# time ./a.out 1

real    0m4.383s
user    0m1.940s
sys     0m1.177s
[root@blackbox ~]# sync
[root@blackbox ~]# time ./a.out 1

real    0m4.339s
user    0m1.863s
sys     0m1.129s

With patch:

[root@blackbox ~]# sync
[root@blackbox ~]# time ./a.out 1

real    0m3.005s
user    0m1.162s
sys     0m0.816s
[root@blackbox ~]# sync
[root@blackbox ~]# time ./a.out 1

real    0m3.188s
user    0m1.222s
sys     0m0.867s
[root@blackbox ~]# sync
[root@blackbox ~]# time ./a.out 1

real    0m2.999s
user    0m1.131s
sys     0m0.832s

Change-Id: Id160a24f44b4dccfcfce99a6f69ddb8938523cd5
BUG: 953694
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/6131
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Raghavendra Talur <rtalur@redhat.com>
Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2013-11-07 23:55:59 -08:00
Lalatendu Mohanty
f42a764924 libgfapi: fixing tabs in glfsxmp.c with whitespace
Just replacing tabs with whitespace. No code changes present
in this patch

Change-Id: Ic3f64f5a52b0ab5e5c9fe9f45113901dec751e68
Signed-off-by: Lalatendu Mohanty <lmohanty@redhat.com>
Reviewed-on: http://review.gluster.org/6148
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
Reviewed-by: Santosh Pradhan <spradhan@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2013-10-27 22:50:11 -07:00
Krutika Dhananjay
93c49c6e8c gfapi: fix mem-types initialisation
... and also fill the body of mem_acct_init().

Change-Id: I457d0629c20f999dd2f50e710c75b1c7575b6c19
BUG: 1021904
Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com>
Reviewed-on: http://review.gluster.org/6134
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2013-10-24 13:56:13 -07:00
Anand Avati
7ebcf96c94 gfapi: wake migration waiters after migration
Wake call threads which are waiting on migration to complete with
a cond_broadcast. Else if any call which arrives right when migration
is attempted will end up hanging indefinitely.

Change-Id: I7df5298f93998d9a54fb12c16654e62333018ece
BUG: 953694
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/6111
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com>
Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
2013-10-22 09:33:39 -07:00
Krishnan Parthasarathi
dfa08fe058 gfapi: fix return value of glfs_fini
Change-Id: I5b90c7602334226a978bbdae2f9516e8701b403f
BUG: 1004519
Signed-off-by: Krishnan Parthasarathi <kparthas@redhat.com>
Reviewed-on: http://review.gluster.org/6092
Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2013-10-17 12:49:28 -07:00
Lalatendu Mohanty
656db3a8e0 libgfapi: Fix to compilation failure/warnings for examples/glfsxmp.c
The comilation of glfsxmp.c i.e. "make glfsxmp" was failing
with "undefined reference to `clock_gettime'" error.

Fixed all compilation warnings for unused variables and
wrong formart specifier in printf e.g. : "format ‘%d’ expects
argument of type ‘int’, but argument 3 has type ‘__syscall_slong_t’
[-Wformat]"

Also added the compilation steps for glfsxmp.c in the
README file

Change-Id: I399ec7c9ad7b6412463c89099d63922caef6749e
Signed-off-by: Lalatendu Mohanty <lmohanty@redhat.com>
Reviewed-on: http://review.gluster.org/6085
Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
Reviewed-by: Santosh Pradhan <spradhan@redhat.com>
Tested-by: Shyamsundar Ranganathan <srangana@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2013-10-16 00:55:50 -07:00
R.Shyamsundar
d573f170cf gfapi: object handle based API extensions
There is an ongoing effort to integrate NFS Ganesha (
https://github.com/nfs-ganesha/nfs-ganesha/wiki ) with GlusterFS as one of
the file system back ends.

Towards this we need extensions to gfapi that can handle object based
operations. Meaning, instead of using full paths or relative paths from
cwd, it is required that we can work with APIs, like the *at POSIX
variants, to be able to create, lookup, open etc. files and directories.
Hence the objects are the files or directories themselves and we give out
handles to these objects that can be used for further operations.

This code drop is an initial implementation of the proposed APIs.

The new APIs are implemented as glfs_h_XXX variants in the file
glfs-handleops.c to mirror glfs-fops.c style. The code leverages holding
onto inode references and doling these out as opaque/cookie type objects to
the callers, to enable them to be used as handles in other operations.

An fd based approach was considered, but due to the extra footprint that
the fd structure and its counterparts would incur, this was dropped to take
the approach of holding inode references themselves.

Tested by extending glfsxmp.c to invoke and exercise the added APIs, and
further tested with a reference integration of the same as an FSAL with NFS
Ganesha.

Change-Id: I23629c99e905b54070fa2e6565147812e5f3fa5d
BUG: 1016000
Signed-off-by: R.Shyamsundar <srangana@redhat.com>
Reviewed-on: http://review.gluster.org/5936
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2013-10-11 12:16:46 -07:00
Poornima
d3558ae5c4 gfapi: Fix iobuf leaks in gfapi
In glfs_readv, if syncop_readv() fails with return value <= 0
the iobref was not being unrefd which would cause iobuf leaks.

Change-Id: I9850ae149e53cf75ba26f8b5f4c5446cce4b5991
BUG: 1018176
Signed-off-by: Poornima <pgurusid@redhat.com>
Reviewed-on: http://review.gluster.org/6074
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2013-10-11 12:05:55 -07:00
Raghavendra Bhat
dcfe4ab475 glusterfsd, libgfapi: destroy the temporary graphs constructed for comparison
* The new and the oldgraphs which have been constructed whenever there is
  a volfile change (either reconfigure of the existing graph or creating
  a new graph) for comparison should be freed. Otherwise frequent graph
  changes will lead to huge memory leak

Change-Id: I4faddb1aa9393b34cd2de6732e537a60f600026a
BUG: 948178
Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com>
Reviewed-on: http://review.gluster.org/5388
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2013-10-03 21:32:43 -07:00
Niels de Vos
c499ef86a2 gfapi.py: support dynamic loading of versioned libraries
Currently gfapi.py only loads libraries by filename ending in ".so".  On
an installed system without development packages, the <lib>.so filenames
are not available. ctypes.util.find_library() can be used to detect the
files dynamically.

In addition to this, also fixing some minor indention errors and package
the library into the Python site-packages path. Python applications and
libraries can now access libgfapi through 'from glusterfs import gfapi'.

Change-Id: I71e38dabd3ade5dcf24813bf2fc25cda91b571c6
BUG: 1005146
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: http://review.gluster.org/5835
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
2013-10-02 13:04:03 -07:00
Anand Avati
84fa8af38d core: block unused signals in created threads
Block all signal except those which are set for explicit handling
in glusterfs_signals_setup(). Since thread spawning code in
libglusterfs and xlators can get called from application threads
when used through libgfapi, it is necessary to do this blocking.

Change-Id: Ia320f80521a83d2edcda50b9ad414583a0175281
BUG: 1011662
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/5995
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2013-09-25 01:33:16 -07:00
Vijay Bellur
aa971f439d logging: Expose set_log_file_path() in libglusterfs.
This patch also changes the behavior of glfs_set_logging().
If logfile argument is not provided to glfs_set_logging(),
libgfapi uses set_log_file_path() to create a logfile.

Change-Id: I49ec66c7f16f5604ff2f7cf7b365b08a05b5460d
BUG: 764890
Signed-off-by: Vijay Bellur <vbellur@redhat.com>
Reviewed-on: http://review.gluster.org/5910
Reviewed-by: Anand Avati <avati@redhat.com>
Tested-by: Anand Avati <avati@redhat.com>
2013-09-24 12:00:54 -07:00
Anand Avati
d7f7510f21 gfapi: implement a minimial glfs_fini()
At the very least, we should PARENT_DOWN on the currently active
graph and disconnect ourselves from glusterd.

Further cleanups underway.

Change-Id: I9276686a84b0975b5ce272b4cbec1b80920d5c5c
BUG: 1004519
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/5788
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
2013-09-17 11:42:12 -07:00
Anand Avati
8eb3898578 gfapi: use native STACK_WIND for read _async() calls
There is little value in using synctask wrappers for async IO
requests, as STACK_WIND is asynchronous by nature already.

Skip going through synctask for read/write async calls.

Change-Id: Ifde331d7c97e0f33426da6ef4377c5ba70dddc06
BUG: 1009134
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/5897
2013-09-17 11:41:42 -07:00
Poornima
1955f7dc84 gfapi: store the open/create flags in fd_t object
The flags passed on to open and create calls were not being saved
in the fd_t object, hence the fd migration was failing.

Change-Id: I486bb818477fe4c393d64a711534a082162a0e53
BUG: 1005159
Signed-off-by: Poornima <pgurusid@redhat.com>
Reviewed-on: http://review.gluster.org/5837
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2013-09-06 15:56:57 -07:00
Anand Avati
2a1f445d30 gfapi: apply an upper bound on nested symlink resolution
In case of nested symlink resolution, implement an upper
bound on the number of such nested levels the resolver will
descend. This limit is arbitrary, and set to 2048 nested
levels.

Change-Id: I264e5bd60d317eda97f4e6f49bd5d8694f8664a9
BUG: 1004100
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/5768
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Raghavendra Talur <rtalur@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2013-09-05 05:26:30 -07:00
Anand Avati
4b317e64ca glusterfs, gfapi: fix symbol clash
The callback structures in both protocol/client and glusterfsd,
gfapi used the same name for the actor table - gluster_cbk_actors.

CBKs are required only for the management connection, and the
actors of protocol/client are NOP functions. This supposed-to-be
NOP function dispatch tabble is actually ending up pointing to
the actor table of glusterfsd or gfapi.

These functions, even though set wrongly, are not even expected
to be called through the protocol/client callback path. Glusterd
however sends the FETCHSPEC (and other) notify callbacks to *all*
connected clients unconditionally, and there is a small period
of time when protocol/client is connected to glusterd for
PORTMAP query. If the FETCHSPEC callback notify is issued in
this window of time, we end up calling the wrong actor in the
client side resulting in a crash.

Change-Id: I605ff7df64c7faf4607369bbf275aedec28e1778
BUG: 1004091
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/5767
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
2013-09-05 05:24:42 -07:00
Raghavendra Talur
479087c1e6 api: Add null check for active_subvol in glfs_close.
Problem:
In glfs_close, NULL value returned by glfs_active_subvol was not
being checked and was causing a crash.

Change-Id: I6eefa60e7b07dee251b98932b1d08a5c2981d3d7
BUG: 1002511
Signed-off-by: Raghavendra Talur <rtalur@redhat.com>
Reviewed-on: http://review.gluster.org/5733
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2013-08-29 14:11:50 -07:00
Raghavendra Talur
2ec4b22bb0 api: Fill loc.path and loc.gfid before syncop_open.
syncop_open was crashing because of NULL dereference.
This fixes that.

Change-Id: I4bc48fac2a6c0b15c806cbbb4ae45c67891ab7ed
BUG: 1002577
Signed-off-by: Raghavendra Talur <rtalur@redhat.com>
Reviewed-on: http://review.gluster.org/5734
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2013-08-29 13:52:28 -07:00
Anand Avati
bbcdbd8c36 synctask: minor enhancements
- Enhance syncenv_new() to accept scaling parameters of syncproc.
  Previously the scaling parameters were hardcoded and decided at
  compile time.

- New API synctask_create() which returns the created synctask. This
  is similar to synctask_new which only returned the status of whether
  a synctask could be created or not.

  The meaning of NULL cbk in synctask_create() means the task is
  "joinable". Until synctask_join() is called on such a synctask,
  the task is not reaped and resources are not destroyed. The
  task would be in a zombie state after synctask_fn returns and
  before synctask_join() is called.

Change-Id: I368ec9037de9510d2ba951f0aad86aaf18d9a6b6
BUG: 986775
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/5365
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Brian Foster <bfoster@redhat.com>
2013-08-28 15:52:24 -07:00
Vijay Bellur
cd25375415 api: Set errno before invoking glfs_init_done()
Change-Id: I3f4d3cfaf7078e126881111eb840d854b0b6466d
BUG: 860212
Signed-off-by: Vijay Bellur <vbellur@redhat.com>
Reviewed-on: http://review.gluster.org/5704
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2013-08-25 13:03:24 -07:00
Anand Avati
d3bbbbb158 gfapi: Add description of glfs_io_cbk function as a comment
No code change, just add a comment describing the callback function
type.

Change-Id: I3a5c5b3d3fe042b4f535963dd477806832c15704
BUG: 839950
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/5696
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
Tested-by: Vijay Bellur <vbellur@redhat.com>
2013-08-23 03:37:23 -07:00
Anand Avati
a78f7dcddb libgfapi : Fix for hardlink creation on SMB mount
Previously the inode of the hard link was left uninitialised hence
glfs_loc_link() used to fail with invalid parameter.Since inode is
same for both the files in hard link creation,   updated the inode
field of hard link to be same as that of the original file.

Original-author: susant palai <spalai@redhat.com>
Change-Id: Ifd6439867d982524a7b48bf3f6add6e844b3c4c1
BUG: 996063
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/5576
Tested-by: susant palai <spalai@redhat.com>
Reviewed-by: Raghavendra Talur <rtalur@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2013-08-13 08:59:52 -07:00
Bala.FA
4e63eafaed log: set ident to openlog
at syslog side, log message is identified by its properties like
programname, pid, etc.  brick/mount processes need to be identified
uniquely as they are different process of gluterfsd/glusterfs.  At
rsyslog side, log separated by programname/app-name with pid works but
bit hard to identify them in long run which process is for what
brick/mount.

This patch fixes by setting identity string at openlog() which sets
programname/app-name as similar to old style log file prefixed by
gluster, glusterd, glusterfs or glusterfsd

Change-Id: Ia05068943fa67ae1663aaded1444cf84ea648db8
BUG: 928648
Signed-off-by: Bala.FA <barumuga@redhat.com>
Reviewed-on: http://review.gluster.org/5541
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
2013-08-13 07:07:43 -07:00
Tiziano Müller
16061c0d0f Fix broken parallel install
There is an automake bug when using _LTLIBRARIES, breaking parallel
`make install` (see http://debbugs.gnu.org/cgi/bugreport.cgi?bug=7328)

Change-Id: I06aea1dc4c3e6cb0ae9aee5a04dd61881c8b6a7e
BUG: 981888
Signed-off-by: Tiziano Müller <tiziano.mueller@stepping-stone.ch>
Reviewed-on: http://review.gluster.org/5448
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2013-07-31 19:32:19 -07:00
Anand Avati
dfedfec359 gfapi: perform open() on pre-existing inode in glfs_creat()
Performing syncop_create() on a pre-existing file can result
in opening the linkfile if DHT has a linkfile for the file.
This is because dht_create() will perform the op on the
hashed_subvol() and overwrite the layout in dht_create_cbk.

Change-Id: I7a0db56921ec9fc3e278e0418db3b967f81e5598
BUG: 990410
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/5447
Reviewed-by: Raghavendra Talur <rtalur@redhat.com>
Tested-by: Raghavendra Talur <rtalur@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
2013-07-31 13:29:35 -07:00
Anand Avati
21d032bf05 gfapi: fix glfs_readlink()
glfs_readlink() is supposed to memcpy the buffer pointer
returned by syncop_readlink(). Fix it.

Change-Id: I5936b07abbd93cf02b354233dc60f6623e30a38b
BUG: 953694
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/5439
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Raghavendra Talur <rtalur@redhat.com>
2013-07-31 06:23:37 -07:00
Vijay Bellur
0bbdfa3d40 api: Avoid a NULL deref in glfs_open().
Change-Id: I23e9fabcb8a430bcd44595ca31460c54d7bd4a39
BUG: 953694
Signed-off-by: Vijay Bellur <vbellur@redhat.com>
Reviewed-on: http://review.gluster.org/5426
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2013-07-30 19:50:50 -07:00
Anand Avati
aa70c0bba0 gfapi: handle failures in graph switches
A graph switch failure can potentially result in NULL returned
for glfs_active_subvol(). Handle this gracefully in glfs_subvol_done()

Change-Id: I6adbd878c354b9ed2314fc71cea4671792efa0a4
BUG: 953694
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/5407
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2013-07-28 08:20:20 -07:00
Raghavendra Talur
da50f61174 gfapi: fix for not able to rename folders
If oldname is being renamed to newname, we need to
check for type of newname only if newname exists.

Change-Id: I068a283f9ffe67fcd5e8754d6bf052a2339efbf2
BUG: 953694
Signed-off-by: Raghavendra Talur <rtalur@redhat.com>
Reviewed-on: http://review.gluster.org/5333
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2013-07-16 15:58:48 -07:00
Anand Avati
d7e80787fc gfapi: fix read past EOF in glfs_preadv()
When syncop_preadv() returns 0, we are returning @size as-is
(without modifying user passed buffer). This results in
junk data being read when read offset is at or beyond EOF.

Change-Id: I8aa9c957359f9a52833b944b452f501b6f5538d3
BUG: 953694
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/5236
Reviewed-by: Raghavendra Talur <rtalur@redhat.com>
Reviewed-by: José Rivera <jrivera@redhat.com>
Tested-by: José Rivera <jrivera@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2013-06-20 01:45:50 -07:00
Brian Foster
cede8eb3d0 gfapi: async discard support
From: Bharata B Rao <bharata@linux.vnet.ibm.com>

Add glfs_discard_async() to meet QEMU's discard requirements.

BUG: 963678
Change-Id: I97723107fca88dd96cddc08cd67a286645a37922
Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
Signed-off-by: Brian Foster <bfoster@redhat.com>
Reviewed-on: http://review.gluster.org/5144
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2013-06-13 14:38:57 -07:00
Brian Foster
17f2871724 glusterfs: discard (hole punch) support
Add support for the DISCARD file operation. Discard punches a hole
in a file in the provided range. Block de-allocation is implemented
via fallocate() (as requested via fuse and passed on to the brick
fs) but a separate fop is created within gluster to emphasize the
fact that discard changes file data (the discarded region is
replaced with zeroes) and must invalidate caches where appropriate.

BUG: 963678
Change-Id: I34633a0bfff2187afeab4292a15f3cc9adf261af
Signed-off-by: Brian Foster <bfoster@redhat.com>
Reviewed-on: http://review.gluster.org/5090
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2013-06-13 14:37:43 -07:00
Brian Foster
d1ccc4e400 gluster: add fallocate fop support
Implement support for the fallocate file operation. fallocate
allocates blocks for a particular inode such that future writes
to the associated region of the file are guaranteed not to fail
with ENOSPC.

This patch adds fallocate support to the following areas:

	- libglusterfs
	- mount/fuse
	- io-stats
	- performance/md-cache,open-behind
	- quota
	- cluster/afr,dht,stripe
	- rpc/xdr
	- protocol/client,server
	- io-threads
	- marker
	- storage/posix
	- libgfapi

BUG: 949242
Change-Id: Ice8e61351f9d6115c5df68768bc844abbf0ce8bd
Signed-off-by: Brian Foster <bfoster@redhat.com>
Reviewed-on: http://review.gluster.org/4969
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2013-06-13 14:37:21 -07:00
Anand Avati
83e4a149bd api: Fix api.so and libgfapi.so build time dependency
Not all versions of autoconf handle dependencies expressed
as "A: B". Use the xxx_DEPENDENCIES variable to make it
version independent

Change-Id: Ia0b13399d0bf10995d9845e04cd9eb3ef6518dbe
BUG: 953694
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/5118
Reviewed-by: Raghavendra Talur <rtalur@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
2013-05-30 23:44:28 -07:00
Anand Avati
7646169f20 gfapi: add new API glfs_dup() to copy a file descriptor
Duplicate a glfs_fd file descriptor. Inherit the internal
fd_t with a reference.

Change-Id: Ib30e9a46b608b9f78202957f4dab6ba6265a9ec0
BUG: 953694
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/5109
Reviewed-by: Raghavendra Talur <rtalur@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
2013-05-29 21:40:54 -07:00
Anand Avati
deaf7574a6 api/Makefile: link api.so against libgfapi (and dependencies)
Samba opens vfs_gluster.so with dlopen(..,RTLD_LOCAL). This makes
the symobls in the shared module and the library it exclusively
links against (specifically gfapi) not resolvable from the .so
files which are opened in turn within (like xlator .so files)
unless explicitly linked against the dependent libraries.

This is not otherwise required in case of glusterfsd as the main
executable symbols are always available for resolution from
dlopen()'ed shared objects, making Samba's vfs_glusterfs
requirement unique, but luckily not incompatible.

Change-Id: I894c521daade7d46ec5bf07793f2eea02ebc9700
BUG: 953694
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/5105
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2013-05-29 05:27:33 -07:00
Anand Avati
1a5e8638a1 gfapi: handle graph switch (cwd, fds, locks)
- Migrate open fds to new graph
- Migrate locks held in open fd to new fd
- Refresh CWD, so resolution of relative paths happens on refreshed
  inode (on new graph).

Change-Id: I4b18083b9b290107ebda1f917fc85b635ab72fb4
BUG: 953694
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/5059
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Raghavendra G <raghavendra@gluster.com>
2013-05-28 00:08:57 -07:00
Anand Avati
2a22449720 gfapi: ignore NULL and -1 parameters in glfs_set_logging()
Permit changing either logfile or level by specifying the other
as -1 or NULL respectively.

Change-Id: I7df05db7b613c57e0106a5cce846ae26d2a160d4
BUG: 953694
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/5085
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
2013-05-25 11:40:07 -07:00
Anand Avati
ed11e88b77 gfapi: provide stable st_dev in stat structure
Samba depends on st_dev being unique and stable for different
filesystems (it uses st_dev+st_ino as key to store metadata of
a file).

Change-Id: Ia022fabadfb5ef3fc9724c1b1ca86f1a20f98af9
BUG: 953694
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/5045
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
2013-05-25 11:39:20 -07:00
Anand Avati
9b645be90f gfapi: link inodes in relevant entry FOPs
Do not let inode linking to happen only in lookup(). While
that works, it is inefficient.

Change-Id: I51bbfb6255ec4324ab17ff00566375f49d120c06
BUG: 953694
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/4931
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
2013-05-25 11:36:21 -07:00
Anand Avati
e9ac9d4d5c gfapi: optimistic resolution with ESTALE detection
Enhance the path resolver to be optimistic. Instead of performing
a wasteful ->lookup() fop for every component in every API call,
build in speculation logic to consider the previous resolution
result temporarily, and if it results in ESTALE either later in
the path resolution or in the FOP, then retry resolution with
->lookup() at every level and retry the FOP.

This is the same optimistic resolution/retry logic Linux kernel
resolver is proposing to use.

Change-Id: Iecbc25248754caf0915cd7205910563ff88ce5e7
BUG: 953694
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/4928
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2013-05-25 11:35:06 -07:00
Anand Avati
54b9cd3df3 gfapi: add new API glfs_set_xlator_option()
This is the equivalent of specifying --xlator-option command line
argument to glusterfsd.

Change-Id: I1171dd518a85db4bd456fa0790f450e889a956cd
BUG: 953694
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/4927
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2013-05-02 23:20:49 -07:00
Anand Avati
1bcb1c4d52 gfapi: POSIX locking support
Change-Id: I37d9e1fb4a715094876be6af3856c1b4cf398021
BUG: 953694
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/4881
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2013-04-24 01:06:05 -07:00
Anand Avati
d37b2198dc gfapi: implement readdirplus_r support
This call is used by Samba VFS.

Change-Id: Ib371502ad5a71b3b0e993f6b30e3dfb1f839c020
BUG: 953694
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/4879
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2013-04-24 01:05:56 -07:00
Anand Avati
fdde66da06 gfapi: handle root (/) resolution gracefully for realpath()
Resolving path of / is crucial for realpath() to work properly.

Change-Id: I8a1b482beba37f01afbd20446d9e6f6125f2ec34
BUG: 953694
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/4872
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2013-04-23 05:33:46 -07:00
Anand Avati
a1db18cf7a gfapi: support for chdir() and family
Add support for chdir, fchdir, getcwd, realpath equivalents in
GFAPI. These are required for the Samba VFS plugin.

Change-Id: I91d2db9146994403c98961c489c7640c51d5add2
BUG: 953694
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/4853
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2013-04-19 04:41:35 -07:00
Anand Avati
90d77fbdd2 gfapi: dict_unref() xattr_req in fop finish instead of dict_destroy()
The current way of calling dict_destroy() at the end of an API fop assumes
that xattr_req is not stored/ref'd by any translators in the stack. However
when translators like DHT store xattr_req in dht_local_t with a dict_ref()
and perform dict_unref() in the unwind path, things get subject to a race.

The race is between the woken up thread (by syncop_wake) i.e, the gfapi
invoking thread and the thread where the FOP was unwound. As the C stack
of STACK_UNWIND unwinds back, dht_local_unref() gets invoked within the
DHT_STACK_UNWIND macro. This thread attempts dict_unref, which would
be "safe" if it wins the race against the gfapi invoking thread. However
if the gfapi invoking thread wins the race, it will perform dict_destroy()
first and therefore make dict_unref() within dht_local_unref perform
a double free.

This is the embarrassing on-screen bug which showed up in a roomful of people
during the gluster dev summit demo of qemu/libgfapi integration.

Change-Id: I284c93de87cdc128d5801f42c84aa87f753090d4
BUG: 839950
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/4644
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
2013-03-07 07:03:26 -08:00
Vijay Bellur
fc501a57bc Better mechanism to handle memory accounting
Memory accounting will now be enabled if:

1) Any glusterfs process is spawned with argument --mem-accounting.
2) DEBUG is defined.

Change-Id: I3345e114127a57ce61916be0e2c4e0049a4c3432
BUG: 834465
Signed-off-by: Vijay Bellur <vbellur@redhat.com>
Reviewed-on: http://review.gluster.org/4523
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2013-02-17 16:55:37 -08:00