IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
glfs_free releases different types of data depends on memory type.
Drop the depends of memory type of memory accounting,
new macro GLFS_CALLOC/GLFS_MALLOC/GLFS_REALLOC/GLFS_FREE are added
to support assign release function dynamically, it adds a separate
memory header named glfs_mem_header for gfapi.
Updates: #312
Change-Id: Ie608e5227cbaa05d3f4681a515e83a50d5b17c3f
Signed-off-by: Kinglong Mee <mijinlong@open-fs.com>
Reviewed-on: https://review.gluster.org/18092
Smoke: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Tested-by: Niels de Vos <ndevos@redhat.com>
Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Count the active allocations per 'struct mem_pool'. These are the
objects that the calling component allocated and free'd in the memory
pool for this specific type. Having this count in the statedump will
make it easy to find memory leaks.
Updates: #307
Change-Id: I797fabab86f104e49338c00e449a7d0b0d270004
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: https://review.gluster.org/18074
Smoke: Gluster Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
In order to generate statedumps per glusterfs_ctx_t, it is needed to
place all the memory pools in a structure that the context can reach.
The 'struct mem_pool' has been extended with a 'list_head owner' that is
linked with the glusterfs_ctx_t->mempool_list.
All callers of mem_pool_new() have been updated to pass the current
glusterfs_ctx_t along. This context is needed to add the new memory pool
to the list and for grabbing the ctx->lock while updating the
glusterfs_ctx_t->mempool_list.
Updates: #307
Change-Id: Ia9384424d8d1630ef3efc9d5d523bf739c356c6e
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: https://review.gluster.org/18075
Smoke: Gluster Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
The new fields are required to enable equivalent volume set and volgen
features, and some more additional features in GD2. GD2 does not use a
hard-coded volume options map like GD1, but builds such by reading the
options tables directly from the xlators.
The new fields being introduced into the volume options struct include
the following,
- op-version - version(s) the option was introduced in
- deprecated - version(s) the option was deprecated in
- flags - flags for the option (settable, client, global, force, doc etc.)
- tags - descriptive tags that apply to this option, can be used to
group options
- validate_fn - custom option validation function
Enums for currently available flags have also been defined. To avoid a
naming clashes, the flag enums in GD1 have been renamed.
Updates #302
Change-Id: Ic7e08aef9e051beb47e8dc17d7f7be211aed308a
Signed-off-by: Kaushal M <kaushal@redhat.com>
Reviewed-on: https://review.gluster.org/18059
Smoke: Gluster Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
This renames the current 'struct mem_pool' to 'struct mem_pool_shared'.
The mem_pool_shared is globally allocated and not specific for
particular objects.
A new 'struct mem_pool' gets allocated when mem_pool_new() is called. It
points to the mem_pool_shared that handles the actual allocation
requests. The 'struct mem_pool' is only used for accounting of the
objects that the caller requested and free'd.
All of these changes will be used to collect all the memory pools a
glusterfs_ctx_t is consuming, so that statedumps can be collected per
context.
Updates: #307
Change-Id: I6355d3f0251c928e0bbfc71be3431307c6f3a3da
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: https://review.gluster.org/18073
Smoke: Gluster Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
With all the new 'struct mem_pool' infrastructure in place, it is now
possible to fetch details about the memory pools that a glusterfs_ctx_t
uses.
This only captures the information from 'struct mem_pool', and not from
the global 'struct mem_pool_shared' or the pool_sweeper thread. The
current details help with detecting memory leaks.
Updates: #307
Change-Id: Idbc5ba136df50863e1e380b448061509896f2c23
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: https://review.gluster.org/18076
Smoke: Gluster Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
qr_content_extract used dict_get to get the value of
the GF_CONTENT_KEY key. dict_get does not ref the data
before returning it so QR could be acting on freed memory
if another thread deletes the key before then.
This patch also fixes a race in dict_get_with_ref.
Fix: Use dict_get_with_ref to retrieve the file contents.
Change-Id: Ib1a7a70bb92eed7e70747ec530e0b3edc53127ec
BUG: 1484709
Signed-off-by: N Balachandran <nbalacha@redhat.com>
Reviewed-on: https://review.gluster.org/18115
Smoke: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Tested-by: Raghavendra G <rgowdapp@redhat.com>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
Problem:
On start of glusterd service, glusterd fetch data from store, while
parsing data from store if peers file consists of blank line glusterd
fails to start.
Fix:
With this fix while parsing peers file glusterd will skip blank lines
if it contains any.
Signed-off-by: Gaurav Yadav <gyadav@redhat.com>
Change-Id: I53cd65a54de5f57baef292b2118b70ffb7f99388
BUG: 1482906
Reviewed-on: https://review.gluster.org/18066
Tested-by: Gaurav Yadav <gyadav@redhat.com>
Smoke: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Prashanth Pai <ppai@redhat.com>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Problem: currently we can't identify which process is running and
how many instances of it are available.
Fix: name the process when its spawned and send it to the server
and save it in the client_t
The processes that abide by this change from this patch are:
1) fuse mount,
2) rebalance,
3) selfheal,
4) tier,
5) quota,
6) snapshot,
7) brick.
8) gfapi (by default. gfapi.<processname> if processname is found)
Note: fuse gets a process name as native-fuse-client by default.
If the user gives a name for the fuse and spawns it, it will be of
this type --process-name native-fuse-client.<name_specified>.
This can be made use by the process like aux mount done by quota,
geo-rep, etc by adding another option in the aux mount " -o
process-name=gsync_mount"
Updates: #178
Signed-off-by: hari gowtham <hgowtham@redhat.com>
Change-Id: Ie4d02257216839338043737691753bab9a974d5e
Reviewed-on: https://review.gluster.org/17957
Smoke: Gluster Build System <jenkins@build.gluster.org>
Tested-by: hari gowtham <hari.gowtham005@gmail.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
Reviewed-by: Aravinda VK <avishwan@redhat.com>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Currently Gluster keeps process pid information of all the daemons
and brick processes in Gluster configuration file directory
(ie., /var/lib/glusterd/*).
These pid files should be seperate from configuration files.
Deletion of the configuration file directory might result into serious problems.
Also, /var/run/gluster is the default placeholder directory for pid files.
So, with this fix Gluster will keep all process pid information of all
processes in /var/run/gluster/* directory.
Change-Id: Idb09e3fccb6a7355fbac1df31082637c8d7ab5b4
BUG: 1258561
Signed-off-by: Gaurav Kumar Garg <ggarg@redhat.com>
Signed-off-by: Saravanakumar Arumugam <sarumuga@redhat.com>
Reviewed-on: https://review.gluster.org/13580
Tested-by: MOHIT AGRAWAL <moagrawa@redhat.com>
Smoke: Gluster Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
Issue:
The caller of glfs_setxattr sends a buffer to set as the value.
We create a dict in which the pointer to the value is set.
Underlying layers like md-cache take a ref on this dict to store
the value for a longer time. But the moment setxattr is complete,
the caller of glfs_setxattr can free the value memory.
Solution:
memcpy the setxattr value to the gluster buffer.
Change-Id: I58753fe702e8b7d0f6c4f058714c65d0ad5d7a0a
BUG: 1477488
Signed-off-by: Poornima G <pgurusid@redhat.com>
Reviewed-on: https://review.gluster.org/17967
Reviewed-by: soumya k <skoduri@redhat.com>
Smoke: Gluster Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
This commit adds support to the get-state CLI to capture details
on geo-replication session as obtained in
`gluster volume geo-replication status detail` in its output.
Fixes: #291
Change-Id: I2fbcba70bfdaf439522637234805545194777ed4
Signed-off-by: Samikshan Bairagya <samikshan@gmail.com>
Reviewed-on: https://review.gluster.org/17941
Smoke: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Shubhendu Tripathi <shtripat@redhat.com>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
Changes:
1. Take subdir mount option in client (mount.gluster / glusterfsd)
2. Pass the subdir mount to server-handshake (from client-handshake)
3. Handle subdir-mount dir's lookup in server-first-lookup and handle
all fops resolution accordingly with proper gfid of subdir
4. Change the auth/addr module to handle the multiple subdir entries
in option, and valid parsing.
How to use the feature:
`# mount -t glusterfs $hostname:/$volname/$subdir /$mount_point`
Or
`# mount -t glusterfs $hostname:/$volname -osubdir_mount=$subdir /$mount_point`
Option can be set like:
`# gluster volume set <volname> auth.allow "/subdir1(192.168.1.*),/(192.168.10.*),/subdir2(192.168.8.*)"`
Updates #175
Change-Id: I7ea57f76ddbe6c3862cfe02e13f89e8a39719e11
Signed-off-by: Amar Tumballi <amarts@redhat.com>
Reviewed-on: https://review.gluster.org/17141
Smoke: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
What it really does is skip irrelevant entries like . and .. until
we're at an entry we might actually care about. Renamed to
GF_SKIP_IRRELEVANT_ENTRIES accordingly.
Change-Id: If0464451a8243c29c0a93b4c6f0f0eda2fade44c
Signed-off-by: Jeff Darcy <jdarcy@fb.com>
Reviewed-on: https://review.gluster.org/17901
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Smoke: Gluster Build System <jenkins@build.gluster.org>
Tested-by: Jeff Darcy <jeff@pl.atyp.us>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
It is possible that pthread_getspecific() returns a non-NULL value in
case the pthread_key_t is not initialized. The behaviour for
pthread_getspecific() is not defined in this case. This can happen when
applications use mem-pools from libglusterfs.so, but did not call
mem_pools_init_early().
By tracking the status of the mem-pools initialization, it is now
possible to prevent calling pthread_getspecific() in case the
pthread_key_t is not initialized. In future, we might want to exend this
more to faciliate debugging.
Reported-by: Kotresh HR <khiremat@redhat.com>
Tested-by: Jiffin Tony Thottan <jthottan@redhat.com>
Change-Id: I6255419fe05792dc78b1eaff55bc008fc5ff3933
Fixes: 1e8e62640 ("mem-pool: initialize pthread_key_t pool_key in mem_pool_init_early()")
BUG: 1475255
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: https://review.gluster.org/17899
Smoke: Gluster Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: jiffin tony Thottan <jthottan@redhat.com>
Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
The gfid2path infra stores the "pargfid/bname" as on xattr
value for each non directory entry. Hardlinks would have a
separate xattr. This xattr key is internal and is not
exposed to applications. A virtual xattr is exposed for
the applications to fetch the path from gfid.
Internal xattr:
trusted.gfid2path.<xxhash>
Virtual xattr:
glusterfs.gfidtopath
getfattr -h -n glusterfs.gfidtopath /<aux-mnt>/.gfid/<gfid>
If there are hardlinks, it returns all the paths separated
by ':'.
A volume set option is introduced to change the delimiter
to required string of max length 7.
gluster vol set gfid2path-separator ":::"
Updates: #139
Change-Id: Ie3b0c3fd8bd5333c4a27410011e608333918c02a
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Reviewed-on: https://review.gluster.org/17785
Smoke: Gluster Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Krutika Dhananjay <kdhananj@redhat.com>
There is a case where Gluster/NFS needs to resolve a path outside of the
nfs-xlator itself. While resolving the path to fetch the GFID for
creating the NFS-filehandle, gfapi may set an inode-ctx through
glfs_resolve_at(). This inode-ctx is linked with the global_xlator.
Because the global_xlator does not have any cbks, loc_wipe() will cause
a segfault when it calls inode_unref() and xl->cbks->forget(). It is
assumed that all xlators have a cbks symbol, otherwise loading of the
xlator will fail. The global_xlator is not loaded in the same way, so
there is no failure noticed when the instance is created. By adding an
empty `struct xlator_cbks`, the global_xlator behaves similat to other
xlators that do not implement all callbacks.
I would have preferred to keep the inode-ctx setting through
glfs_resolve_at() contained within Gluster/NFS. Unfortunately
Gluster/NFS also uses the inode-ctx, and is not prepared to see the
values that glfs_resolve_at() stores there.
This problem is not easily reproducible because it involves mounting
over WebNFS (like Solaris 10 can do). The segfault will also not be
immediate, unless the following is done:
1. create a subdir on a volume
2. mount the volume/subdir over WebNFS
3. unmount the volume/subdir
4. mount the root of the volume
5. delete the subdir on the volume -> segfault of Gluster/NFS
Change-Id: I2bd71d033e97edc07ba93b2d4ada558f65d68999
BUG: 1468291
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: https://review.gluster.org/17897
Smoke: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Reviewed-by: jiffin tony Thottan <jthottan@redhat.com>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
gfid2path xattr is an internal xattr and should not be
allowed to modify by other applications via gluster
mount. This patch blocks the same.
Updates: #139
Change-Id: Id2cb29797ee1bd77e0e0d2203a47469fd7203355
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Reviewed-on: https://review.gluster.org/17744
Smoke: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Prashanth Pai <ppai@redhat.com>
Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com>
Reviewed-by: Aravinda VK <avishwan@redhat.com>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
gcc 7 (default in Fedora 26) complains about the roof() and floor()
macros:
stripe.c: In function 'stripe_truncate':
stripe.c:701:49: warning: '*' in boolean context, suggest '&&' instead [-Wint-in-bool-context]
tmp_offset = roof(offset, fctx->stripe_size *
../../../../libglusterfs/src/common-utils.h:55:35: note: in definition of macro 'roof'
#define roof(a,b) ((((a)+(b)-1)/((b)?(b):1))*(b))
^
stripe.c:704:50: warning: '*' in boolean context, suggest '&&' instead [-Wint-in-bool-context]
tmp_offset = floor(offset, fctx->stripe_size *
../../../../libglusterfs/src/common-utils.h:56:28: note: in definition of macro 'floor'
#define floor(a,b) (((a)/((b)?(b):1))*(b))
^
The calculations done in stripe_truncate() look safe enough, but gcc
does not seem to like the passing the int/size_t to the `((b)?(b):1)`
compact if-statement, so use `b != 0` for the test.
Change-Id: If9fa4b8e86ba4b2ace61b1e05a5c28050fe4a7d3
Updates: #259
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: https://review.gluster.org/17842
Smoke: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Reviewed-by: Raghavendra Talur <rtalur@redhat.com>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
GCC 7 warns about the following
crypt.c:3685:31: warning: ?: using integer constants in boolean context,
the expression will always evaluate to ‘true’ [-Wint-in-bool-context]
local->op_ret < 0 ? -1 : 0,
../../../../libglusterfs/src/stack.h:342:36: note: in definition of
macro ‘STACK_UNWIND_STRICT’
} else if (op_ret == 0) {
\
^~~~~~
}
This is because args to pre-processor are lazily evaluated and operator
precedence for == is higher than ?:
Change-Id: I2c2ffb08bc3731ad0e17796047e01cb10771dbcf
Updates: #259
Reported-by: Raghavendra Talur <rtalur@redhat.com>
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: https://review.gluster.org/17827
Reviewed-by: Prashanth Pai <ppai@redhat.com>
Smoke: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Raghavendra Talur <rtalur@redhat.com>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
When using a minimal gfapi application that only initializes a small
graph (sink, shard and meta xlators) the following memory leaks are
reported by Valgrind:
HEAP SUMMARY:
in use at exit: 322,976 bytes in 75 blocks
total heap usage: 684 allocs, 609 frees, 2,092,116 bytes allocated
With this change, the mem-pools are cleaned up on calling of
mem_pools_fini() and the objects in the pool are free'd.
HEAP SUMMARY:
in use at exit: 315,265 bytes in 58 blocks
total heap usage: 684 allocs, 626 frees, 2,092,079 bytes allocated
This information was gathered with `./run-xlator.sh features/shard` that
comes with `gfapi-load-volfile` from gluster-debug-tools.
While working on the free'ing of the per_thread_pool_list_t structures,
it became apparent that GF_CALLOC() in mem_get_pool_list() gets
redirected to a standard calloc() without prepending the Gluster
specific memory header. This is because mem_pools_init() gets called
before THIS->ctx is valid, so it is not possible to check if memory
accounting is enabled or not. Because of this, the GF_CALLOC() call in
mem_get_pool_list() has been replaced by CALLOC() to prevent potential
mismatches between the allocation/free'ing of per_thread_pool_list_t
structures.
Change-Id: Id6f558816f399b0c613d74df36deac2300b6dd98
BUG: 1470170
URL: https://github.com/gluster/gluster-debug-tools
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: https://review.gluster.org/17768
Smoke: Gluster Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
Reviewed-by: soumya k <skoduri@redhat.com>
It is not possible to call pthread_key_delete for the pool_key that is
intialized in the constructor for the memory pools. This makes it
difficult to do a full cleanup of all the resources in mem_pools_fini().
For this, the initialization of pool_key should be moved to
mem_pool_init().
However, the glusterfsd binary has a rather complex initialization
procedure. The memory pools need to get initialized partially to get
mem_get() functionality working. But, the pool_sweeper thread can get
killed in case it is started before glusterfsd deamonizes.
In order to solve this, mem_pools_init() is split into two pieces:
1. mem_pools_init_early() for initializing the basic structures
2. mem_pools_init_late() to start the pool_sweeper thread
With the split of mem_pools_init(), and placing the pthread_key_create()
in mem_pools_init_early(), it is now possible to correctly cleanup the
pool_key with pthread_key_delete() in mem_pools_fini().
It seems that there was no memory pool initialization in the CLI. This
has been added as well now. Without it, the CLI will not be able to call
mem_get() successfully which results in a hang of the process.
Change-Id: I1de0153dfe600fd79eac7468cc070e4bd35e71dd
BUG: 1470170
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: https://review.gluster.org/17779
Smoke: Gluster Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
Implementation of these two functions becomes easier by using gf_fop_list[]
array. So implemented that and removed usage of these functions.
BUG: 1472250
Change-Id: I8a592913f9eeb02d965708bcf28a637588ed4988
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: https://review.gluster.org/17812
Reviewed-by: Niels de Vos <ndevos@redhat.com>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Smoke: Gluster Build System <jenkins@build.gluster.org>
Problem:
Bulk xattr removal doesn't check if the xattrs that are coming in xdata
have gfid/volume-id xattrs, so there is potential for bulkremovexattr
removing gfid/volume-id.
I also observed that bulkremovexattr is not available for fremovexattr.
Fix:
Do proper checks in bulk removexattr to remove gfid/volume-id.
Refactor [f]removexattr to reduce the differences.
BUG: 1470489
Change-Id: Ia845b31846a149500111c0996646e648f72cdce6
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: https://review.gluster.org/17765
Smoke: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Anuradha Talur <atalur@redhat.com>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Reviewed-by: Krutika Dhananjay <kdhananj@redhat.com>
Only the old (removed) implementation of mem-pools provided access to
the ctx->mempool_list to faciliate state-dumps. The usage of the
mempool_list has been disabled with "#if defined(OLD_MEM_POOLS)", but a
few occurences were missed.
Change-Id: I912fb63830efc06247eb0c6551d198271ee57a86
BUG: 1470170
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: https://review.gluster.org/17778
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Smoke: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Fix fuse ctx memory leak in case an error occurs and the cleanup path
is different than usual. Also fix a memory leak in logging if
eh_save_history() fails.
Change-Id: I7ec967c807b0ed91184e5b958be70702215c46c9
BUG: 1470220
Signed-off-by: Danny Couture <couture.danny@gmail.com>
Reviewed-on: https://review.gluster.org/17759
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Smoke: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: N Balachandran <nbalacha@redhat.com>
Reviewed-by: Prashanth Pai <ppai@redhat.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Tested-by: Amar Tumballi <amarts@redhat.com>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
clean up things that I tripped over doing other changes.
1) fix mishmash of random spacing in struct decls in glusterfs.h.
Not technically a problem, just ugly to look at.
2) replace open-coded strings constants with existing #define
constants. A disaster waiting to happen.
3) Use sys_access() instead of sys_stat() or sys_lstat() to test
simple existence of file. Why copy dozens of bytes from kernel to
user space that aren't going to be used by anything? There are
probably more instances like these.
Change-Id: I28089bef4cc93d5e4e4213045fb1a2649d110f82
Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
Reviewed-on: https://review.gluster.org/17769
Smoke: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Prashanth Pai <ppai@redhat.com>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
Problem:
For allowing parallel writes we shouldn't depend on ia_size to be same for
all the bricks in each write_cbk(). But we need to make sure backend size
is correct on all the bricks and no crashes/manual modifications happened.
Fix:
At the time of get_size_version() we do 1 check to make sure size of the file
is same across the bricks. From then on the FOPs will give the status of the
fop, so we rely on this information to keep which bricks are good/bad.
Updates #251
Change-Id: I1df645347e2e9f2e09cfa4411b6cc305d7f4e4e5
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: https://review.gluster.org/17741
Smoke: Gluster Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Xavier Hernandez <xhernandez@datalab.es>
With this infra, a new xattr is stored on each entry
creation as below.
trusted.gfid2path.<xxhash> = <pargfid>/<basename>
If there are hardlinks, multiple xattrs would be present.
Fops which are impacted:
create, mknod, link, symlink, rename, unlink
Option to enable:
gluster vol set <VOLNAME> storage.gfid2path on
Updates: #139
Change-Id: I369974cd16703c45ee87f82e6c2ff5a987a6cc6a
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Reviewed-on: https://review.gluster.org/17488
Smoke: Gluster Build System <jenkins@build.gluster.org>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Aravinda VK <avishwan@redhat.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
We are storing the entire volfile and using this to check
volfile change. With brick multiplexing there will be lot
of graphs per process which will increase the memory foot
print of the process. So instead of storing the entire
graph we could use sha256 and we can compare the hash to
see whether volfile change happened or not.
Also with Brick multiplexing, the direct comparison of vol
file is not correct. There are two problems.
Problem 1:
We are currently storing one single graph (the last
updated volfile) whereas, what we need is the entire
graph with all atttached bricks.
If we fix this issue, we have second problem
Problem 2:
With multiplexing we have a graph that contains multiple
bricks. But what we are checking as part of the reconfigure
is, comparing the entire graph with one single graph,
which will always fail.
Solution:
We create list in glusterfs_ctx_t that stores sha256 hash
of individual brick graphs. When a graph changes happens
we compare the stored hash and the current hash. If the
hash matches, then no need for reconfigure. Otherwise we
first do the reconfigure and then update the hash.
For now, gfapi has not changed this way. Meaning when gfapi
volfile fetch or reconfigure happens, we still store the
entire graph and compare, each memory.
This is fine, because libgfapi will not load brick graphs.
But changing the libgfapi will make the code similar in
both glusterfsd-mgmt and api. Also it helps to reduce some
memory.
Change-Id: I9df917a771a52b95622ab8f63af34ec390163a77
BUG: 1467986
Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com>
Reviewed-on: https://review.gluster.org/17709
Smoke: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
The change consists of two parts: make sure it doesn't happen (in
glfs.c), and make it harmless if it does (in mem-pool.c).
Change-Id: Icb7dda7a45dd3d1ade2ee3991bb6a22c8ec88424
BUG: 1468863
Signed-off-by: Jeff Darcy <jdarcy@fb.com>
Reviewed-on: https://review.gluster.org/17728
Tested-by: Jeff Darcy <jeff@pl.atyp.us>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Smoke: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
When glusterfs wants to retrieve the list of auxiliary gids
of a user, it typically allocates a sufficiently big gid_t
array on stack and calls getgrouplist(3) with it. However,
"sufficiently big" means to be of maximum supported gid list
size, which in GlusterFS is GF_MAX_AUX_GROUPS = 64k.
That means a 64k * sizeof(gid_t) = 256k allocation, which is
big enough to overflow the stack in certain cases.
A further observation is that stack allocation of the gid list
brings no gain, as in all cases the content of the gid list
eventually gets copied over to a heap allocated buffer.
So we add a convenience wrapper of getgrouplist to libglusterfs
called gf_getgrouplist which calls getgrouplist with a sufficiently
big heap allocated buffer (it takes care of the allocation too).
We are porting all the getgrouplist invocations to gf_getgrouplist
and thus eliminate the huge stack allocation.
BUG: 1464327
Change-Id: Icea76d0d74dcf2f87d26cb299acc771ca3b32d2b
Signed-off-by: Csaba Henk <csaba@redhat.com>
Reviewed-on: https://review.gluster.org/17706
Smoke: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
This also makes mem_pools_init and mem_pools_fini re-callable, so GFAPI
can go through infinite init/fini cycles if they want to. Not saying
that's a good idea, but at least it's safe.
Change-Id: I617913410bcff54568b802cb653f48bdd533bd65
Signed-off-by: Jeff Darcy <jdarcy@fb.com>
Reviewed-on: https://review.gluster.org/17662
Smoke: Gluster Build System <jenkins@build.gluster.org>
Tested-by: Jeff Darcy <jeff@pl.atyp.us>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
The most common pattern, both in our code and elsewhere, is this:
struct _xyz {
...
};
typedef struct _xyz xyz_t;
These exceptions - especially call_frame/call_stack - have been slowing
down code navigation for years. By converging on a single pattern,
navigating from xyz_t in code to the actual definition of struct _xyz
(i.e. without having to visit the typedef first) might even be
automatable.
Change-Id: I0e5dd1f51f98e000173c62ef4ddc5b21d9ec44ed
Signed-off-by: Jeff Darcy <jdarcy@fb.com>
Reviewed-on: https://review.gluster.org/17650
Smoke: Gluster Build System <jenkins@build.gluster.org>
Tested-by: Jeff Darcy <jeff@pl.atyp.us>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
xxhash is a faster non-cryptographic hash.
https://github.com/Cyan4973/xxHash
Release Taken: "xxHash v0.6.2"
--------------
Files added:
contrib/xxhash/xxhash.c
contrib/xxhash/xxhash.h
contrib/xxhash/xxhsum.c
Modifications to source:
------------------------
Following functions and data types got 'GF_' prefix
as below to avoid any form of name collisions in future.
---- Functions ----
GF_XXH_versionNumber
GF_XXH32
GF_XXH32_createState
GF_XXH32_freeState
GF_XXH32_copyState
GF_XXH32_reset
GF_XXH32_update
GF_XXH32_digest
GF_XXH32_canonicalFromHash
GF_XXH32_hashFromCanonical
GF_XXH64
GF_XXH64_createState
GF_XXH64_freeState
GF_XXH64_copyState
GF_XXH64_reset
GF_XXH64_update
GF_XXH64_digest
GF_XXH64_canonicalFromHash
GF_XXH64_hashFromCanonical
---- Data Types ----
GF_XXH_errorcode
GF_XXH32_state_t*
GF_XXH32_canonical_t*
GF_XXH32_hash_t
GF_XXH64_state_t*
GF_XXH64_canonical_t*
GF_XXH64_hash_t
It is linked with libglusterfs.so. A wrapper
funtion is also added for the easy usage in
common-utils.c.
xxhash can be used for the all the usecases where
a faster non-cryptographic hash is required.
gfid to path infra would be using this for now.
NOTE:
----
The gluster coding guidelines check is ignored
as maintaining it further would be difficult.
Updates: #253
Change-Id: Ib143f90d91d4ee99864a10246d5983e92900173b
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Reviewed-on: https://review.gluster.org/17641
Smoke: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
When we build GlusterFS with GF_DISBLE_MEMPOOL, it is failed due to macro
condition in mem-pool.c:mem_get().
Change-Id: I03fe804f93d761ea3bfdc3b20f0253a03350a68f
BUG: 1465214
Signed-off-by: Ji-Hyeon Gim <potatogim@potatogim.net>
Reviewed-on: https://review.gluster.org/17633
Smoke: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
Reviewed-by: jiffin tony Thottan <jthottan@redhat.com>
Tested-by: Ji-Hyeon Gim
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Afr has introduced a new key GF_XATTR_LIST_NODE_UUIDS_KEY,
through which rebalance will figure out its local subvolumes.(Reference
bugid=1463250)
key: GF_XATTR_NODE_UUID_KEY will continue to serve it's old
purpose of returning the first afr chiild.
test: prove tests/basic/distribute/rebal-all-nodes-migrate.t
Change-Id: I4d602feda2a05b29d2210c712a07a4ac6b8bc112
BUG: 1463648
Signed-off-by: Susant Palai <spalai@redhat.com>
Signed-off-by: N Balachandran <nbalacha@redhat.com>
Reviewed-on: https://review.gluster.org/17595
Smoke: Gluster Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
Problem:
The change in EC to return list of node uuids for
GF_XATTR_NODE_UUID_KEY was causing problems with
geo-rep.
Fix:
This patch will allow to get the single node uuid
as it was doing before with the key
"GF_XATTR_NODE_UUID_KEY", and will also allow to get
the list of node uuids by using a new key
"GF_XATTR_LIST_NODE_UUIDS_KEY". This will solve
the problem with geo-rep and any other features which
were depending on this.
BUG: 1462790
Change-Id: I2d9214a9658d4a41a3d6de08600884d2bda5f3eb
Signed-off-by: Sunil Kumar Acharya <sheggodu@redhat.com>
Reviewed-on: https://review.gluster.org/17594
Smoke: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Xavier Hernandez <xhernandez@datalab.es>
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
The patch https://review.gluster.org/#/c/17177 resolves "." and ".."
to corrosponding inodes and names before sending the request to the
backend server. But this will only work if inode and its parent is
linked properly. Incase of nameless lookup(applications like ganesha)
the inode of parent can be NULL(only gfid is send). So this patch will
resolve "." and ".." only if proper parent is available
Change-Id: I4c50258b0d896dabf000a547ab180b57df308a0b
BUG: 1460514
Signed-off-by: Jiffin Tony Thottan <jthottan@redhat.com>
Reviewed-on: https://review.gluster.org/17502
Smoke: Gluster Build System <jenkins@build.gluster.org>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Poornima G <pgurusid@redhat.com>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: soumya k <skoduri@redhat.com>
Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
Problem:
The change in afr to return list of node uuids was causing problems
with geo-rep.
Fix:
This patch will allow to get the single node uuid as it was doing
before with the key "GF_XATTR_NODE_UUID_KEY", and will also allow
to get the list of node uuids by using a new key
"GF_XATTR_LIST_NODE_UUIDS_KEY". This will solve the problem with
geo-rep and any other feature which were depending on this.
Change-Id: I09885dac6dfca127be94b708470c8c2941356f9a
BUG: 1462790
Signed-off-by: karthik-us <ksubrahm@redhat.com>
Reviewed-on: https://review.gluster.org/17576
Smoke: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Ravishankar N <ravishankar@redhat.com>
Reviewed-by: Kotresh HR <khiremat@redhat.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
... as opposed to hardcoding it to "json" always.
Change-Id: I5e79473a514373145ad764f24bb6219a6983a4c6
BUG: 1458197
Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com>
Reviewed-on: https://review.gluster.org/17451
Smoke: Gluster Build System <jenkins@build.gluster.org>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
problem:
When we call listen from protocol/server, we are giving a
hard coded valie of 10 if it is not manually given.
With multiplexing, especially when glusterd restarts all
clients may try to connect to the server at a time.
Which will result in overflowing the queue, and kernel
will complain about the errors.
Solution:
This patch will introduce a volume set command to make backlog
value as a configurable. This patch also changes the default
values for backlog from 10 to 128. This changes is only applicable
for sockets listening from protocol.
Example:
gluster volume set <volname> transport.listen-backlog 1024
Note: 1 Brick has to be restarted to get this value in effect
2 This changes won't be reflected in glusterd, or other
xlators which calls listen. If you need, you have to
add this option to the volfile.
Change-Id: I0c5a2bbf28b5db612f9979e7560e05dd82b41477
BUG: 1456405
Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com>
Reviewed-on: https://review.gluster.org/17411
Smoke: Gluster Build System <jenkins@build.gluster.org>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
Reviewed-by: Raghavendra Talur <rtalur@redhat.com>
Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
when you take the 'statedump', it shows the output like below
-----
[dict]
max-number-of-dict-pairs=13
total-pairs-used=41613
total-dict-used=12629
average-pairs-per-dict=3
------
Updates #220
Change-Id: I71a7eda3a3cd23edf4483234f22f983923bbb081
Signed-off-by: Amar Tumballi <amarts@redhat.com>
Reviewed-on: https://review.gluster.org/4035
Smoke: Gluster Build System <jenkins@build.gluster.org>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
