IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
It is not possible to call pthread_key_delete for the pool_key that is
intialized in the constructor for the memory pools. This makes it
difficult to do a full cleanup of all the resources in mem_pools_fini().
For this, the initialization of pool_key should be moved to
mem_pool_init().
However, the glusterfsd binary has a rather complex initialization
procedure. The memory pools need to get initialized partially to get
mem_get() functionality working. But, the pool_sweeper thread can get
killed in case it is started before glusterfsd deamonizes.
In order to solve this, mem_pools_init() is split into two pieces:
1. mem_pools_init_early() for initializing the basic structures
2. mem_pools_init_late() to start the pool_sweeper thread
With the split of mem_pools_init(), and placing the pthread_key_create()
in mem_pools_init_early(), it is now possible to correctly cleanup the
pool_key with pthread_key_delete() in mem_pools_fini().
It seems that there was no memory pool initialization in the CLI. This
has been added as well now. Without it, the CLI will not be able to call
mem_get() successfully which results in a hang of the process.
Change-Id: I1de0153dfe600fd79eac7468cc070e4bd35e71dd
BUG: 1470170
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: https://review.gluster.org/17779
Smoke: Gluster Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
When brick-multiplexing is enabled, and
"cluster.max-bricks-per-process" isn't explicitly set, multiplexing
happens without any limit set. But the default value set for that
tunable is 1, which is confusing. This commit sets the default
value to 0, and prevents the user from being able to set this value
to 1 when brick-multiplexing is enbaled. The default value of 0
denotes that brick-multiplexing can happen without any limit on the
number of bricks per process.
Change-Id: I4647f7bf5837d520075dc5c19a6e75bc1bba258b
BUG: 1472417
Signed-off-by: Samikshan Bairagya <samikshan@gmail.com>
Reviewed-on: https://review.gluster.org/17819
Smoke: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Implementation of these two functions becomes easier by using gf_fop_list[]
array. So implemented that and removed usage of these functions.
BUG: 1472250
Change-Id: I8a592913f9eeb02d965708bcf28a637588ed4988
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: https://review.gluster.org/17812
Reviewed-by: Niels de Vos <ndevos@redhat.com>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Smoke: Gluster Build System <jenkins@build.gluster.org>
Makefile generation should include default LD_FLAGS and also include rpc
related paths in include path
Change-Id: I45e1c97b96f08bbfe4663384f4873726febef9f6
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: https://review.gluster.org/17811
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Smoke: Gluster Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Problem:
Currently there is no way for the admin from CLI to resolve gfid
split-brain based on some policy like choice of the brick, mtime
or size.
Fix:
With the existing CLI options based on size, mtime, and choice of
brick, we do lookup on the parent for the specified file. As
part of the lookup, if we find gfid mismatch, we resolve them
based on the policy and return. If the file is not in gfid split-
brain, then we check for the data and metadata split-brain in the
getxattr code path, and resolve if any.
This will work provided absolute path to the file with the CLI
and not with gfid of the file. Hence the source-brick policy
without any file path will also not resolve the gfid split-brain
since it uses the gfid of the files. But it can resolve any other
type of split-brains and skip the gfid mismatch resolution with
the usual error message.
Reverting the change https://review.gluster.org/17290. This patch
resolves the issue.
Fixesgluster/glusterfs#135
Change-Id: Iaeba6fc32f184a34255d03be87cda02773130a09
BUG: 1459530
Signed-off-by: karthik-us <ksubrahm@redhat.com>
Reviewed-on: https://review.gluster.org/17485
Reviewed-by: Ravishankar N <ravishankar@redhat.com>
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Smoke: Gluster Build System <jenkins@build.gluster.org>
Problem:
Killing the bricks(using kill signal) in test scripts will
result in test failures with brick multiplexing enabled.
Solution:
Updated the script to use kill_brick function to bring down
the bricks.
BUG: 1472094
Change-Id: Ibbf1fdc1be660ad3cd93e95af2838c0aae0181af
Signed-off-by: Sunil Kumar Acharya <sheggodu@redhat.com>
Reviewed-on: https://review.gluster.org/17809
Smoke: Gluster Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
Earlier, it was not reported if a function call had arguments on next
line and the args were not aligned to opening bracket. With this change,
it is a check that is displayed in output.
I have not analyzed what all --strict brings in. If it is found to be
more restrictive than required, we may choose to promote this *one* CHK
to WARN and stop using --strict.
Sample:
CHECK: Alignment should match open parenthesis
+ ret = gf_asprintf (&footer,
+ " [%s]", strerror(errnum));
total: 0 warnings, 1 checks, 8 lines checked
Change-Id: I48834087efcae872b69746ba962b0bb1c130b5cd
BUG: 1469179
Signed-off-by: Raghavendra Talur <rtalur@redhat.com>
Reviewed-on: https://review.gluster.org/17737
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Smoke: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Prashanth Pai <ppai@redhat.com>
Reviewed-by: ankitraj
Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
Reviewed-by: Sonal Arora <sarora@redhat.com>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
The IPC test-case is not correct, and can cause segfaults or hangs. The
ipctest.py script calls glfs_ipc() with the `glfs_t` structure, but
should do so with a `glfs_fd_t`.
In addition, the test-case is written in a way that we do not suggest to
use libgfapi. Python scripts are encouraged to use the bindings from the
libgfapi-python project. It would be better to rewrite the test in C so
that there is type-checking while compiling and no additional issues
with portability (see `LD_PRELOAD` note in the `.t` file).
Change-Id: Icb52b5b1585fbee98f2c694547c31df0aa2ba70b
Updates: #269
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: https://review.gluster.org/17786
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Smoke: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
Reviewed-by: Raghavendra Talur <rtalur@redhat.com>
Program
Since poller thread bears the brunt of execution till the request is
handed over to io-threads, poller thread experiencies lock
contention(s) in the control flow till io-threads, which slows it
down. This delay invariably affects reading ping requests from network
and responding to them, resulting in increased ping latencies, which
sometimes results in a ping-timer-expiry on client leading to
disconnect of transport. So, this patch aims to free up poller thread
from executing code of Glusterfs Program. We do this by making
* Glusterfs Program registering itself asking rpcsvc to execute its
actors in its own threads.
* GF-DUMP Program registering itself asking rpcsvc to _NOT_ execute
its actors in its own threads. Otherwise program's ownthreads become
bottleneck in processing ping traffic. This means that poller thread
reads a ping packet, invokes its actor and hands the response msg to
transport queue.
Change-Id: I526268c10bdd5ef93f322a4f95385137550a6a49
Signed-off-by: Raghavendra G <rgowdapp@redhat.com>
BUG: 1421938
Reviewed-on: https://review.gluster.org/17105
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Smoke: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
Problem:
Bulk xattr removal doesn't check if the xattrs that are coming in xdata
have gfid/volume-id xattrs, so there is potential for bulkremovexattr
removing gfid/volume-id.
I also observed that bulkremovexattr is not available for fremovexattr.
Fix:
Do proper checks in bulk removexattr to remove gfid/volume-id.
Refactor [f]removexattr to reduce the differences.
BUG: 1470489
Change-Id: Ia845b31846a149500111c0996646e648f72cdce6
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: https://review.gluster.org/17765
Smoke: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Anuradha Talur <atalur@redhat.com>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Reviewed-by: Krutika Dhananjay <kdhananj@redhat.com>
Currently the "cluster.brick-multiplex" and
"cluster.max-bricks-per-process" options do not show anything in
the description field when gluster volume set help is called. This
commit adds the description fields for these 2 options.
Change-Id: I3d162c61fa2774dd994f046e305d457f0fd43192
BUG: 1471790
Signed-off-by: Samikshan Bairagya <samikshan@gmail.com>
Reviewed-on: https://review.gluster.org/17790
Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
Smoke: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Gaurav Yadav <gyadav@redhat.com>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
init_openssl_mt() wasn't explicitly invoked and was run implicitly before
dlopen() returned as it was tagged as __attribute__ ((constructor)). This
function used to call GF_CALLOC() which wasn't available or initialized
when socket.so is dlopen()ed by an external program. The program used to
crash with SIGSEGV as follows:
0x00007ffff5efe1ad in __gf_calloc (nmemb=41, size=40, type=158, typestr=0x7ffff63eb3d6 "gf_sock_mt_lock_array")
at mem-pool.c:109
0x00007ffff63e6acf in init_openssl_mt () at socket.c:4016
0x00007ffff7de90da in call_init.part () from /lib64/ld-linux-x86-64.so.2
0x00007ffff7de91eb in _dl_init () from /lib64/ld-linux-x86-64.so.2
0x00007ffff7dedde1 in dl_open_worker () from /lib64/ld-linux-x86-64.so.2
0x00007ffff7de8f84 in _dl_catch_error () from /lib64/ld-linux-x86-64.so.2
0x00007ffff7ded339 in _dl_open () from /lib64/ld-linux-x86-64.so.2
This change moves call to init_openssl_mt() from being a constructor function
to the init() function and introduces fini_openssl_mt() which cleans up
resources (called in destructor).
BUG: 1193929
Change-Id: Iab690897ec34e24c33f6b43f8d8d9f8fd75ac607
Signed-off-by: Prashanth Pai <ppai@redhat.com>
Reviewed-on: https://review.gluster.org/17753
Smoke: Gluster Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
Currently the script used by glusterfssharedstorage have dependency over
var-run-gluster-shared_storage. But this service will be present only if
node has rebooted. Also in reboot scenario , there is a chance that this
service can be executed before creating var-run-gluster-shared_storage.
In that case glusterfssharedstorage will get succeed even without mounting
the shared storage
Also the type of glusterfssharedstorage changed to "forking" so that it can
be active(instead of dead) after the successful start.
Change-Id: I1c02cc64946e534d845aa7ec7b72644bbe4d26f9
BUG: 1452527
Signed-off-by: Jiffin Tony Thottan <jthottan@redhat.com>
Reviewed-on: https://review.gluster.org/17658
Smoke: Gluster Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: soumya k <skoduri@redhat.com>
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Only the old (removed) implementation of mem-pools provided access to
the ctx->mempool_list to faciliate state-dumps. The usage of the
mempool_list has been disabled with "#if defined(OLD_MEM_POOLS)", but a
few occurences were missed.
Change-Id: I912fb63830efc06247eb0c6551d198271ee57a86
BUG: 1470170
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: https://review.gluster.org/17778
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Smoke: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
e.g. /usr/libexec/glusterfs or /usr/lib64/glusterfs/$NV/xlator
Change-Id: Ibccb8d27b6ad3d6c335e5bbf39aec03b4fe5c8dd
BUG: 1470768
Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
Reviewed-on: https://review.gluster.org/17770
Smoke: Gluster Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Milind Changire <mchangir@redhat.com>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Fix fuse ctx memory leak in case an error occurs and the cleanup path
is different than usual. Also fix a memory leak in logging if
eh_save_history() fails.
Change-Id: I7ec967c807b0ed91184e5b958be70702215c46c9
BUG: 1470220
Signed-off-by: Danny Couture <couture.danny@gmail.com>
Reviewed-on: https://review.gluster.org/17759
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Smoke: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: N Balachandran <nbalacha@redhat.com>
Reviewed-by: Prashanth Pai <ppai@redhat.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Tested-by: Amar Tumballi <amarts@redhat.com>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
clean up things that I tripped over doing other changes.
1) fix mishmash of random spacing in struct decls in glusterfs.h.
Not technically a problem, just ugly to look at.
2) replace open-coded strings constants with existing #define
constants. A disaster waiting to happen.
3) Use sys_access() instead of sys_stat() or sys_lstat() to test
simple existence of file. Why copy dozens of bytes from kernel to
user space that aren't going to be used by anything? There are
probably more instances like these.
Change-Id: I28089bef4cc93d5e4e4213045fb1a2649d110f82
Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
Reviewed-on: https://review.gluster.org/17769
Smoke: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Prashanth Pai <ppai@redhat.com>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
Problem:
Enabling optimistic changelog on EC volume was not
handling node down scenarios appropriately resulting
in volume data inaccessibility.
Solution:
Update dirty xattr appropriately on good bricks whenever
nodes are down. This would fix the metadata information
as part of heal and thus ensures data accessibility.
BUG: 1468261
Change-Id: I08b0d28df386d9b2b49c3de84b4aac1c729ac057
Signed-off-by: Sunil Kumar Acharya <sheggodu@redhat.com>
Reviewed-on: https://review.gluster.org/17703
Smoke: Gluster Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
Problem:
In a 3 way replica, when the source brick does not have pending xattrs
for the sinks, but the 2 sinks blame each other, metadata heal was not
happpening because we were not setting all non-sources as sinks.
Fix: Mark all non-sources as sinks, like it is done in data and entry
heal.
Change-Id: I534978940f5087302e307fcc810a48ffe898ce08
BUG: 1468279
Signed-off-by: Ravishankar N <ravishankar@redhat.com>
Reviewed-on: https://review.gluster.org/17717
Smoke: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Problem: sometime brick process is getting crash after stop the volume
while brick mux is enabled and no. of volumes are high
Solution: In posix notify at the time close mount_lock dir , dir handle
needs to set NULL to avoid the reuse of same dir handle.
BUG: 1470533
Change-Id: Ifd41c20b3c597317851f91049a7c801949840b16
Signed-off-by: Mohit Agrawal <moagrawa@redhat.com>
Reviewed-on: https://review.gluster.org/17767
Smoke: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
Problem:
For allowing parallel writes we shouldn't depend on ia_size to be same for
all the bricks in each write_cbk(). But we need to make sure backend size
is correct on all the bricks and no crashes/manual modifications happened.
Fix:
At the time of get_size_version() we do 1 check to make sure size of the file
is same across the bricks. From then on the FOPs will give the status of the
fop, so we rely on this information to keep which bricks are good/bad.
Updates #251
Change-Id: I1df645347e2e9f2e09cfa4411b6cc305d7f4e4e5
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: https://review.gluster.org/17741
Smoke: Gluster Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Xavier Hernandez <xhernandez@datalab.es>
A brief about how hardlink migration works:
- Different hardlinks (to the same file) may hash to different bricks,
but their cached subvol will be same. Rebalance picks up the first hardlink,
calculates it's hash(call it TARGET) and set the hashed subvolume as an xattr
on the data file.
- Now all the hardlinks those come after this will fetch that xattr and will
create linkto files on TARGET (all linkto files for the hardlinks will be hardlink
to each other on TARGET).
- When number of hardlinks on source is equal to the number of hardlinks on
TARGET, the data migration will happen.
RACE:1
Since rebalance is multi-threaded, the first lookup (which decides where the TARGET
subvol should be), can be called by two hardlink migration parallely and they may end
up creating linkto files on two different TARGET subvols. Hence, hardlinks won't be
migrated.
Fix: Rely on the xattr response of lookup inside gf_defrag_handle_hardlink since it
is executed under synclock.
RACE:2
The linkto files on TARGET can be created by other clients also if they are doing
lookup on the hardlinks. Consider a scenario where you have 100 hardlinks. When
rebalance is migrating 99th hardlink, as a result of continuous lookups from other
client, linkcount on TARGET is equal to source linkcount. Rebalance will migrate data
on the 99th hardlink itself. On 100th hardlink migration, hardlink will have TARGET as
cached subvolume. If it's hash is also the same, then a migration will be triggered from
TARGET to TARGET leading to data loss.
Fix: Make sure before the final data migration, source is not same as destination.
RACE:3
Since a hardlink can be migrating to a non-hashed subvolume, a lookup from other
client or even the rebalance it self, might delete the linkto file on TARGET leading
to hardlinks never getting migrated.
This will be addressed in a different patch in future.
Change-Id: If0f6852f0e662384ee3875a2ac9d19ac4a6cea98
BUG: 1469964
Signed-off-by: Susant Palai <spalai@redhat.com>
Reviewed-on: https://review.gluster.org/17755
Smoke: Gluster Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: N Balachandran <nbalacha@redhat.com>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
Commit 7039243e187 adds a call to mem_pools_init() so that the memory
pool cleanup thread ("sweeper") is started. However, now it is possible
that users of gfapi can not cleanup this thread because glfs_new() can
return NULL, but the sweeper is still running.
In case glfs_fs_new() fails, mem_pools_fini() needs to be called as
well. This seems more correct than calling mem_pools_init() after
glfs_fs_new(), and this makes using memory pools possible *really* early
in the gfapi initialization.
Change-Id: I1f2fb25cc33e227b3c33ce9d1b03f67bc27e981a
Fixes: 7039243e187 ("gfapi: add mem_pools_init and mem_pools_fini calls")
BUG: 1468863
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: https://review.gluster.org/17734
Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Smoke: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: soumya k <skoduri@redhat.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
If the "read-only" volume option is set, it would
make the volume "read-only". But it also makes it
read-only to gluster internal clients such as
gsyncd, self heal, bitd, rebalance etc. In which
case, all the internal operations would fail. This
patch allows internal clients to read and write
when "read-only" option is set.
Change-Id: I8110e8d9eac8def403bb29f235000ddc79eaa433
BUG: 1430608
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Reviewed-on: https://review.gluster.org/16855
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Smoke: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Karthik U S <ksubrahm@redhat.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
If the target of a file migration was changed because
of min-free-disk limits, the dst_fd was closed but the
clean_dst flag was not set to false. If the file could
not be created on the new target for some reason, the
ftruncate call to clean up the dst was sent on the now
invalid fd causing the process to deadlock.
Change-Id: I5bfa80f519b04567413d84229cf62d143c6e2f04
BUG: 1469029
Signed-off-by: N Balachandran <nbalacha@redhat.com>
Reviewed-on: https://review.gluster.org/17735
Smoke: Gluster Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
There is a another race between the cached subvol
being updated in the inode_ctx and the fd being opened on
the target.
1. fop1 -> fd1 -> subvol0
2. file migrated from subvol0 to subvol1 and cached_subvol
changed to subvol1 in inode_ctx
3. fop2 -> fd1 -> subvol1 [takes new cached subvol]
4. fop2 -> checks fd ctx (fd not open on subvol1) -> opens fd1 on subvol1
5. fop1 -> checks fd ctx (fd not open on subvol0)
-> tries to open fd1 on subvol0 -> fails with "No such file on directory".
Fix:
If dht_fd_open_on_dst fails with ENOENT or ESTALE, wind to old subvol
and let the phase1/phase2 checks handle it.
Change-Id: I34f8011574a8b72e3bcfe03b0cc4f024b352f225
BUG: 1465075
Signed-off-by: N Balachandran <nbalacha@redhat.com>
Reviewed-on: https://review.gluster.org/17731
Smoke: Gluster Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
With this infra, a new xattr is stored on each entry
creation as below.
trusted.gfid2path.<xxhash> = <pargfid>/<basename>
If there are hardlinks, multiple xattrs would be present.
Fops which are impacted:
create, mknod, link, symlink, rename, unlink
Option to enable:
gluster vol set <VOLNAME> storage.gfid2path on
Updates: #139
Change-Id: I369974cd16703c45ee87f82e6c2ff5a987a6cc6a
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Reviewed-on: https://review.gluster.org/17488
Smoke: Gluster Build System <jenkins@build.gluster.org>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Aravinda VK <avishwan@redhat.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
The earlier approach of using the number of files
to determine when the rebalance would complete did
not work well when file sizes differed widely.
The new approach now gets the total data size and
uses that information to determine how long
the rebalance is expected to take.
Change-Id: I84e80a0893efab72ff06130e4596fa71c9c8c868
BUG: 1467209
Signed-off-by: N Balachandran <nbalacha@redhat.com>
Reviewed-on: https://review.gluster.org/17668
Smoke: Gluster Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: MOHIT AGRAWAL <moagrawa@redhat.com>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
Since https://review.gluster.org/#/c/17452, the statistics are appended
to the same file instead of overwritten over the previous stats. This
was causing the .t to fail since it checks for only the presence of a
non zero aggr.fop.write.count assuming the latest statistics will
overwrite the previous ones.
Fix it by checking for that the latest value of aggr.fop.write.count is
non zero.
Change-Id: I858011f343966a5d1c19d66dcc64b8cd26315df7
BUG: 1468432
Signed-off-by: Ravishankar N <ravishankar@redhat.com>
Reviewed-on: https://review.gluster.org/17721
Smoke: Gluster Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Reviewed-by: Raghavendra Talur <rtalur@redhat.com>
In the internal testing that was done, stat-prefetch did help
reduce the number of stats coming from qemu hitting the disk,
and thereby improved performance.
Change-Id: Icf1ce62ecf4e96b97e1946a77b30434157a7786a
BUG: 1468191
Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com>
Reviewed-on: https://review.gluster.org/17713
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Smoke: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
We are storing the entire volfile and using this to check
volfile change. With brick multiplexing there will be lot
of graphs per process which will increase the memory foot
print of the process. So instead of storing the entire
graph we could use sha256 and we can compare the hash to
see whether volfile change happened or not.
Also with Brick multiplexing, the direct comparison of vol
file is not correct. There are two problems.
Problem 1:
We are currently storing one single graph (the last
updated volfile) whereas, what we need is the entire
graph with all atttached bricks.
If we fix this issue, we have second problem
Problem 2:
With multiplexing we have a graph that contains multiple
bricks. But what we are checking as part of the reconfigure
is, comparing the entire graph with one single graph,
which will always fail.
Solution:
We create list in glusterfs_ctx_t that stores sha256 hash
of individual brick graphs. When a graph changes happens
we compare the stored hash and the current hash. If the
hash matches, then no need for reconfigure. Otherwise we
first do the reconfigure and then update the hash.
For now, gfapi has not changed this way. Meaning when gfapi
volfile fetch or reconfigure happens, we still store the
entire graph and compare, each memory.
This is fine, because libgfapi will not load brick graphs.
But changing the libgfapi will make the code similar in
both glusterfsd-mgmt and api. Also it helps to reduce some
memory.
Change-Id: I9df917a771a52b95622ab8f63af34ec390163a77
BUG: 1467986
Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com>
Reviewed-on: https://review.gluster.org/17709
Smoke: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
This commit introduces a new global option that can be set to limit
the number of multiplexed bricks in one process.
Usage:
`# gluster volume set all cluster.max-bricks-per-process <value>`
If this option is not set then multiplexing will happen for now
with no limitations set; i.e. a brick process will have as many
bricks multiplexed to it as possible. In other words the current
multiplexing behaviour won't change if this option isn't set to
any value.
This commit also introduces a brick process instance that contains
information about brick processes, like the number of bricks
handled by the process (which is 1 in non-multiplexing cases), list
of bricks, and port number which also serves as an unique identifier
for each brick process instance. The brick process list is
maintained in 'glusterd_conf_t'.
Updates: #151
Change-Id: Ib987d14ab0a4f6034dac01b73a4b2839f7b0b695
Signed-off-by: Samikshan Bairagya <samikshan@gmail.com>
Reviewed-on: https://review.gluster.org/17469
Smoke: Gluster Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
Solaris 10 uses WebNFS and not the MOUNT protocol. All permission checks
for allowing/denying clients to mount are done through the MNT handlers.
These handlers will not give out a filehandle to the NFS-client when
mounting is denied. This prevents clients from successful mounting.
However, over WebNFS a well known 'root-filehandle' is used directly
with the NFSv3 protocol.
When WebNFS was used, no permission checks (the "nfs.export-dir" option)
were applied. Now the WebNFS mount-handler in Gluster/NFS calls the
mnt3_parse_dir_exports() function that takes care of the permission
checking.
BUG: 1468291
Change-Id: Ic9dfd092473ba9c1c7b5fa38401cf9c0aa8395bb
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: https://review.gluster.org/17718
Smoke: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: soumya k <skoduri@redhat.com>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
When blocking locks are used, a new frame is allocated that is used to
send the notification to the client once once the lock becomes
available. In all other cases, the frame that contains the request from
the client will be used for the reply.
Because there was no way to track the different clients with their
requests (captured in the call-state), the call-state could be free'd
before the notification was sent to the client. This caused a
use-after-free of the call-state and could trigger segfaults of the
Gluster/NFS server or incorrect replies on (un)lock requests.
By introducing a nlm4_notify_args structure, the call-state and frame
can be tracked better. This prevents the possibility of segfaulting when
the call-state is used after being free'd.
BUG: 1467313
Change-Id: I285d2bc552f509e5145653b7a50afcff827cd612
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: https://review.gluster.org/17700
Smoke: Gluster Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Reviewed-by: jiffin tony Thottan <jthottan@redhat.com>
In order to track down a potential use-after-free of the
nfs3_call_state_t structure in the NLM component, add reference counting
where teh structure is used. This should prevent premature free'ing of
the structure.
Change-Id: Ib1f13b0463ab1e012b7b49a623c91f0f3e73e1fb
BUG: 1467313
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: https://review.gluster.org/17699
Reviewed-by: jiffin tony Thottan <jthottan@redhat.com>
Smoke: Gluster Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
When a reply on an NLM-procedure gets stuck, the NFS-client will resend
the request. This can happen through a re-connect in case the connection
was terminated (long delay in the reply on the initial request). Once
that happens, not all NLM-procedures are handled correctly.
Testing this is difficult and time-consuming. There still may be
problems with certain operations, but this definitely makes it behave
much better than before.
The problem occured due to a problem in EC, change-id I18a782903ba
addressed the root cause.
Change-Id: I23b385568e27232951fa3fbd7198a0e5d775a8c2
BUG: 1467313
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: https://review.gluster.org/17698
Smoke: Gluster Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
we were taking unref on wrong dictionary which results
in wrong memory access.
Change-Id: Ic25a6c209ecd72c9056dfcb79fabcfc650dd3c1e
BUG: 1467513
Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com>
Reviewed-on: https://review.gluster.org/17691
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Smoke: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
.snaps directory is a virtual direcotory, that doesn't
exist on the backend. Even though it is a special dentry,
it doesn't have a dedicated inode. So the inode number is
always random. Which means it will get different inode
number when reboot happens on snapd process.
Now with windows client the show-direcotry feature requires
a lookup on the .snpas direcoty post readdirp on root.
If the snapd restarted after a lookup, then subsequent lookup
will fail, because linked inode will be stale.
This patch will do a revalidate lookup with a new inode.
Change-Id: If97c07ecb307cefe7c86be8ebd05e28cbf678d1f
BUG: 1467513
Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com>
Reviewed-on: https://review.gluster.org/17690
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com>
Smoke: Gluster Build System <jenkins@build.gluster.org>
protocol/server expects a child up event to successfully
configure the graph. In the actual brick graph, posix is
the one who decide to initiate the notification to the parent
that the child is up.
But in snapd graph there is no posix, hence the child up
notification was missing.
Ideally each xlator should initiate the child up event whenever
it see's that this is the last child xlator.
Change-Id: Icccdb9fe920c265cadaf9f91c040a0831b4b78fc
BUG: 1467513
Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com>
Reviewed-on: https://review.gluster.org/17689
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Smoke: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com>
The change consists of two parts: make sure it doesn't happen (in
glfs.c), and make it harmless if it does (in mem-pool.c).
Change-Id: Icb7dda7a45dd3d1ade2ee3991bb6a22c8ec88424
BUG: 1468863
Signed-off-by: Jeff Darcy <jdarcy@fb.com>
Reviewed-on: https://review.gluster.org/17728
Tested-by: Jeff Darcy <jeff@pl.atyp.us>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Smoke: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
Problem:
4 + 2 EC volume configuration.
If untar of linux is going on and we kill a brick,
indices will be created for the files/dir which need
to be healed. ec_shd_index_sweep spawns threads to
scan these entries and start heal. If in the middle
of this we kill one more brick, we end up in a
situation where we can not heal an entry as there
are only "ec->fragment" number of bricks are UP.
However, the scan will be continued and it will
trigger the heal for those entries.
Solution:
When a heal is triggered for an entry, check if it
*CAN* be healed or not. If not come out with ENOTCONN.
Change-Id: I305be7701c289f36bd7bde22491b71074771424f
BUG: 1464359
Signed-off-by: Ashish Pandey <aspandey@redhat.com>
Reviewed-on: https://review.gluster.org/17692
Smoke: Gluster Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
Reviewed-by: Sunil Kumar Acharya <sheggodu@redhat.com>
Reviewed-by: Xavier Hernandez <xhernandez@datalab.es>
Disabling this test right now so that master is green again. This patch
https://review.gluster.org/#/c/17721/ will actually fix the test. This
patch will make master green again unblocking other patches to land onto
master.
Change-Id: I77d177ce92eb6edcf5326b27a0f7fdbefdec007b
Signed-off-by: Nigel Babu <nigelb@redhat.com>
BUG: 1468432
Reviewed-on: https://review.gluster.org/17723
When glusterfs wants to retrieve the list of auxiliary gids
of a user, it typically allocates a sufficiently big gid_t
array on stack and calls getgrouplist(3) with it. However,
"sufficiently big" means to be of maximum supported gid list
size, which in GlusterFS is GF_MAX_AUX_GROUPS = 64k.
That means a 64k * sizeof(gid_t) = 256k allocation, which is
big enough to overflow the stack in certain cases.
A further observation is that stack allocation of the gid list
brings no gain, as in all cases the content of the gid list
eventually gets copied over to a heap allocated buffer.
So we add a convenience wrapper of getgrouplist to libglusterfs
called gf_getgrouplist which calls getgrouplist with a sufficiently
big heap allocated buffer (it takes care of the allocation too).
We are porting all the getgrouplist invocations to gf_getgrouplist
and thus eliminate the huge stack allocation.
BUG: 1464327
Change-Id: Icea76d0d74dcf2f87d26cb299acc771ca3b32d2b
Signed-off-by: Csaba Henk <csaba@redhat.com>
Reviewed-on: https://review.gluster.org/17706
Smoke: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
... instead of overwriting stats from the previous interval.
This is so that consumers of this feature do not have to be worried
about monitoring when each 'ios-dump-interval' has passed and back up
the resultant stats file well before the next interval has expired.
Change-Id: Ide897237bf4d38e5d759f09911f7d9c817019edf
BUG: 1458197
Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com>
Reviewed-on: https://review.gluster.org/17452
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Smoke: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Amar Tumballi <amarts@redhat.com>
Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
