IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
Note 1) we're not supposed to be using #!/usr/bin/env python, see
https://fedoraproject.org/wiki/Packaging:Guidelines?rd=Packaging/Guidelines#Shebang_lines
Note 2) we're also not supposed to be using "!/usr/bin/python,
see https://fedoraproject.org/wiki/Changes/Avoid_usr_bin_python_in_RPM_Build#Quick_Opt-Out
The previous patch (https://review.gluster.org/19767) tried to do too
much in one patch, so it was abandoned.
This patch does two things:
1) minor cleanup of configure(.ac) to explicitly use python2
2) change all the shebang lines to #!/usr/bin/python2 and add them
where they were missing based on warnings emitted during rpmbuild.
In a follow-up patch python2 will eventually be changed to python3.
Before that python2-isms (e.g. print, string.join(), etc.) need to be
converted to python3. Some of those can be rewritten in version agnostic
python. E.g. print statements become print() with "from __future_ import
print_function". The python 2to3 utility will be used for some of those.
Also Aravinda has given guidance in the comments to the first patch for
changes.
updates: #411
Change-Id: I471730962b2526022115a1fc33629fb078b74338
Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
dht_opendir should wind the open to all subvols
whether or not local->subvols is set. This is
because dht_readdirp winds the calls to all subvols.
Change-Id: I67a96b06dad14a08967c3721301e88555aa01017
updates: bz#1564198
Signed-off-by: N Balachandran <nbalacha@redhat.com>
Add pass-through option in performance traslators. Set the option in
GF_OPTION_INIT() and GF_OPTION_RECONF()
Updates: #304
Change-Id: If1537450147d154905831e36f7162a32866d7ad6
Signed-off-by: Varsha Rao <varao@redhat.com>
Problem: storage.reserve option is not working correctly while
disk space is allocate throguh fallocate
Solution: In posix_disk_space_check_thread_proc after every 5 sec interval
it calls posix_disk_space_check to monitor disk space and set the
flag in posix priv.In 5 sec timestamp user can create big file with
fallocate that can reach posix reserve limit and no error is shown on
terminal even limit has reached.
To resolve the same call posix_disk_space for every fallocate fop
instead to call by a thread after 5 second
BUG: 1560411
Signed-off-by: Mohit Agrawal <moagrawa@redhat.com>
Change-Id: I39ba9390e2e6d084eedbf3bcf45cd6d708591577
In context of Cloudsync:
In scenarios where a data modification fop e.g. a write landed in
POSIX thinking that the file is local, while the file was actually
remote, can be dangerous. Ofcourse we don’t want to take inodelk
for every read/write operation to check the archival status or
coordinate with an upload or a download of a file. To avoid inodelk,
we will check the status of the file in POSIX it self, before we
resume the fop. This helps us avoiding any races mentioned above.
Now e.g. if a write reached POSIX for a file which was actually remote,
it can check the status of the file and will get to know that the file
is remote. It can error out with this status “remote” and cloudsync
xlator will retry the same operation, once it finished downloading the
file.
This patch includes the setxattr changes to do the post processing of
upload i.e. truncate and setting the remote xattr
"trusted.glusterfs.cs.remote" to indicate the file is REMOTE
Each file will have no xattr if the file is LOCAL, one remote xattr if
the file is REMOTE and a combination of REMOTE and DOWNLOADING xattr if
the file is getting downloaded. There is healing logic of these xattrs
to recover from crash inconsitencies.
Fixes: #387
Change-Id: Ie93c2d41aa8d6a798a39bdbef9d1669f057e5fdb
Signed-off-by: Susant Palai <spalai@redhat.com>
Various synchronization present in dht_rename while handling
directories and files is necessary only if we have more than only one
child.
Change-Id: Ie21ad419125504ca2f391b1ae2e5c1d166fee247
fixes: bz#1563511
Signed-off-by: Raghavendra G <rgowdapp@redhat.com>
spec-files:
https://review.gluster.org/#/c/18854/
Overview:
* Cloudsync maintains three file states in it's inode-ctx i.e
1 - LOCAL,
2 - REMOTE,
3 - DOWNLOADING.
* A data modifying fop is allowed only if the state is LOCAL.
If the state is REMOTE or DOWNLOADING, client will download
or wait for the download to finish initiated by other client.
* Multiple download and upload from different clients are synchronized
by inodelk.
* In POSIX a state check is done (part of different commit)before
allowing the fop to continue. If the state is remote/downloading the
fop is unwound with EREMOTE. The client will then download the file
and continue with the fop again.
* Basic Algo for fop (let's say write fop):
- If LOCAL -> resume fop
- If REMOTE ->
- INODELK
- STAT (this gets state and heal the state if needed)
- DOWNLOAD
- resume fop
Note:
* Developers will need to write plugins for download, based on the
remote store they choose. In phase-1, support will be added for
one remote store per volume. In future, more options for multiple
remote stores will be explored.
TODOs:
- Implement stat/lookup/readdirp to return size info from xattr
- Make plugins configurable
- Implement unlink fop
- Add metrics collection
- Add sharding support
Design Contributions:
Aravinda V K <avishwan@redhat.com>
Amar Tumballi <amarts@redhat.com>
Ram Ankireddypalle <areddy@commvault.com>
Susant Palai <spalai@redhat.com>
updates: #387
Change-Id: Iddf711ee7ab4e946ae3e472ff62791a7b85e6d4b
Signed-off-by: Susant Palai <spalai@redhat.com>
Problem:
On node reboot, when glusterd starts volumes, a setup with a large
number of bricks might cause SYN Flooding and connections to be dropped
if the connections are not accepted quickly enough.
Solution:
accept() the connection and rearm the listener socket early to receive
more connection requests as soon as possible.
Change-Id: Ibed421e50284c3f7a8fcdb4de7ac86cf53d4b74e
fixes: bz#1564600
Signed-off-by: Milind Changire <mchangir@redhat.com>
Update the existing admin guide link as it is incorrect.
Change-Id: I05669192623aeac287dfa9002caa0f390ea79499
Updates: bz#1193929
Signed-off-by: Varsha Rao <varao@redhat.com>
duplicatation of exported functions in gfapi.map. Only the newest one
is needed. Both the legacy and current symbols are exported.
glfs_io_cbk34 typedef should not be in a public header file. The old
application was compiled with the original glfs_io_cbk. Outside of
libgfapi, nothing now uses/needs this old typedef, move it into the
C file that needs it.
Similarly glfs_realpath34() decl should not be in glfs.h. Period. Old
applications were compiled with the then glfs_realpath() decl and
linked with glfs_realpath@@GFAPI_3_4.0. New applications should only
call glfs_realpath() and it will be linked to the new/current
glfs_realpath().
Change-Id: Icd5b0c9e9b68f0c133f14447b09ace35f33dbab2
fixes: bz#1564235
Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
gluster-block project needs a dependency check to see if all the bricks
are online before bringing up the relevant gluster-block services. While
the patch https://review.gluster.org/#/c/19785/ attempts to write the
script but brick should be only marked as online only when the
pmap_signin is completed.
While this is perfectly fine for non brick multiplexing, but with brick
multiplexing this patch still doesn't eliminate the race completely as
the attach_req call is asynchrnous and glusterd immediately marks the
port as registerd.
Change-Id: I81db54b88f7315e1b24e0234beebe00de6429f9d
Fixes: bz#1563273
Signed-off-by: Atin Mukherjee <amukherj@redhat.com>
Problem:
We seem to be winding the FOP if pre-op did not succeed on quorum bricks
and then failing the FOP with EROFS since the fop did not meet quorum.
This essentially masks the actual error due to which pre-op failed. (See
BZ).
Fix:
Skip FOP phase if pre-op quorum is not met and go to post-op.
Fixes: 1561129
Change-Id: Ie58a41e8fa1ad79aa06093706e96db8eef61b6d9
fixes: bz#1561129
Signed-off-by: Ravishankar N <ravishankar@redhat.com>
glusterd maintains a boolean flag 'port_registered' which is used to determine
if a brick has completed its portmap sign in process. This flag is (re)set in
pmap_sigin and pmap_signout events. In case of brick multiplexing this flag is
the identifier to determine if the very first brick with which the process is
spawned up has completed its sign in process. However in case of glusterd
restart when a brick is already identified as running, glusterd does a
pmap_registry_bind to ensure its portmap table is updated but this flag isn't
which is fine in case of non brick multiplex case but causes an issue if
the very first brick which came as part of process is replaced and then
the subsequent brick attach will fail. One of the way to validate this
is to create and start a volume, remove the first brick and then
add-brick a new one. Add-brick operation will take a very long time and
post that the volume status will show all other brick status apart from
the new brick as down.
Solution is to set brickinfo->port_registered to true for all the
running bricks when brick multiplexing is enabled.
Change-Id: Ib0662d99d0fa66b1538947fd96b43f1cbc04e4ff
Fixes: bz#1560957
Signed-off-by: Atin Mukherjee <amukherj@redhat.com>
Lookup-optimize has been shown to improve create
performance. The code has been in the project for several
years and is considered stable.
Enabling this by default in order to test this in the
upstream regression runs.
Change-Id: Iab792979ee34f0af4713931e0b5b399c23f65313
updates: bz#1557435
BUG: 1557435
Signed-off-by: N Balachandran <nbalacha@redhat.com>
For transactions where there's no volname involved (eg : gluster v
status), the originator node initiates with staging phase and what that
means in op-sm there's no unlock event triggered which resulted into a
txn_opinfo dictionary leak.
Credits : cynthia.zhou@nokia-sbell.com
Change-Id: I92fffbc2e8e1b010f489060f461be78aa2b86615
Fixes: bz#1550339
Signed-off-by: Atin Mukherjee <amukherj@redhat.com>
On shd, we shouldn't treat any brick down based
on latency, otherwise self-heal will never happen
fixes: bz#1562717
Change-Id: Ica07fcc4fae91a6bfd9c9a670e2be464704d94b7
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
We are setting mgmt_v3_timer->timer to NULL after mgmt_v3_timer is deleted
which is unnecessary. So removing the statement.
This issue is caught while running glusterd with ASAN.
Change-Id: Ied1f91590a2c64ec1af36d4de9c3febd6cf94bb9
Fixes: bz#1562907
Signed-off-by: Sanju Rakonde <srakonde@redhat.com>
The volume status files are located in the glusterd-workdir, not under
/etc (sysconfdir).
BUG: 1234873
Change-Id: Id7f7c83261bb4b5ac2fc104dcd6cb198d6a930aa
Signed-off-by: Niels de Vos <ndevos@redhat.com>
patch https://review.gluster.org/19692 breaks gluster on systems with
IPv6 enabled but don't have IPv6 reverse DNS. Also it defaulted to
enabling ipv6-default regardless of whether --with-ipv6-default or
--without-ipv6-default were specified in the options to configure.
(Also the patch was merged without review.)
Prefer libtirpc over glibc rpc.
On newer linux with tirpc and without glibc rpc use tirpc (obviously)
on less new linux with both tirpc and glibc rpc default to use tirpc,
unless --without-tirpc is specified, in which case use glibc rpc
On less new linux without tirpc fall back to glib rpc (obviously)
ipv6-default requires libtirpc. It is off by default. It must be
explicitly enabled with --with-ipv6-default. If --with-ipv6-default is
specified, but tirpc is not available, disable it and issue a warning
Change-Id: Ib96a230fafb83ec83a71948fe55af1215a7a6ffa
BUG: 1562052
Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
Set the levels for DHT options based on
https://review.gluster.org/#/c/19466/
Change-Id: I51b31a706a0b9517404e83224c89de145fd5d7e1
updates: #430
Signed-off-by: N Balachandran <nbalacha@redhat.com>
Usage: Use 'reader-thread-count=<NUM>' as command line option to
set the thread count at the time of mounting the volume.
Next task is to make these threads auto-scale based on the load,
instead of having the user remount the volume everytime to change
the thread count.
Updates #412
Change-Id: I94aa1505e5ae6a133683d473e0e4e0edd139b76b
Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com>
With lookup-optimize enabled, gf_defrag_settle_hash in rebalance
sometimes flips the on-disk layout on volume root post the
migration of all files in the directory.
This is sometimes seen when attempting to fix the layout of a
directory multiple times before calling gf_defrag_settle_hash.
dht_fix_layout_of_directory generates a new layout in memory but
updates it in the inode ctx before it is set on disk. The layout
may be different the second time around due to
dht_selfheal_layout_maximize_overlap. If the layout is then not
written to the disk, the inode now contains the wrong layout.
gf_defrag_settle_hash does not check the correctness of the layout
in the inode before updating the commit-hash and writing it to the
disk thus changing the layout of the directory.
Change-Id: Ie1407d92982518f2a0c40ec70ad370b34a87b4d4
updates: bz#1557435
Signed-off-by: N Balachandran <nbalacha@redhat.com>
This reverts commit a60fc2ddc03134fb23c5ed5c0bcb195e1649416b.
This commit was causing multiple tests to time out when brick
multiplexing is enabled. With further debugging, it's found that even
though the volume stop transaction is converted into mgmt_v3 to allow
the remote nodes to follow the synctask framework to process the command,
there are other callers of glusterd_brick_stop () which are not synctask
based.
Change-Id: I7aee687abc6bfeaa70c7447031f55ed4ccd64693
updates: bz#1545048
Updates: #363
This new value (3) will try to wind read requests to the child of AFR
having the least amount of pending requests in its queue.
Change-Id: If6bda2aac9bf7aec3fc39622f78659313c4b6508
Signed-off-by: Ravishankar N <ravishankar@redhat.com>
The xattr trusted.glusterfs.list-node-uuids was only sent to a single
subvolume. This was returning null uuids from the other subvolumes as
if they were down.
This fix forces that xattr to be requested from all subvolumes.
Change-Id: If62eb39a6857258923ba625e153d4ad79018ea2f
fixes: bz#1561406
Signed-off-by: Xavi Hernandez <xhernandez@redhat.com>
Looking at the man page show that "Snapshot command" wasn't aligned
with the other section titles.
Change-Id: I24bdb2e3728e03862fee57710cfe34b0607fe09a
BUG: 1507230
Signed-off-by: Michael Scherer <misc@redhat.com>
log message describe the actual test
Change-Id: I1ea7300a6b186032a65236492d6d2a6eef0ab983
fixes: bz#1560441
Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
When a saved frame is to be forced unwind, there is no need to pass an
empty iovector without any data pointed to.
Change-Id: I6e858fb38644326e22239b83272b15db656035e5
BUG: 1523122
Signed-off-by: Zhang Huan <zhanghuan@open-fs.com>
xdr_replymsg is called to decode reply message, and it returns failure
if the message is corrupted. However, retrieving return value from
the global errno is 0 even xdr_replymsg fails.
Fix this issue by simply returning a negative value if call to
xdr_replymsg fails.
Change-Id: I2b9a1dc97652fbb6cf6568ea617f120713784a55
BUG: 1523122
Signed-off-by: Zhang Huan <zhanghuan@open-fs.com>
Problem: There's a race between the last glusterfs_handle_terminate()
response sent to glusterd and the kill that happens immediately if the
terminated brick is the last brick.
Solution: When it is a last brick for the brick process, instead of glusterfsd
killing itself, glusterd will kill the process in case of brick multiplexing.
And also changing gf_attach utility accordingly.
Change-Id: I386c19ca592536daa71294a13d9fc89a26d7e8c0
fixes: bz#1545048
BUG: 1545048
Signed-off-by: Sanju Rakonde <srakonde@redhat.com>
ENOSPC returned by a file migration is no longer
considered a rebalance failure.
Change-Id: I21cf3a8acdc827bc478e138d6cb5db649d53a28c
fixes: bz#1553598
Signed-off-by: N Balachandran <nbalacha@redhat.com>
Problem: if a lookup is done on a newly added brick for a path on which limit
has been reached, the lookup fails to heal the directory tree due to quota.
Solution: Tag the lookup as an internal fop and ignore it in quota.
Since marking internal fop does not usually give enough contextual information.
Introducing new flags to pass the contextual info.
Adding dict_check_flag and dict_set_flag to aid flag operations.
A flag is a single bit in a bit array (currently limited to 256 bits).
Change-Id: Ifb6a68bcaffedd425dd0f01f7db24edd5394c095
fixes: bz#1505355
BUG: 1505355
Signed-off-by: Sanoj Unnikrishnan <sunnikri@redhat.com>
commit fef9293 changed network.inode-lru-limit from 50000 to 200000 in
nl-cache group profile but the test wasn't changed to reflect it
accordingly.
Change-Id: Ibb5fb0a387f160f6b726246b161a9a7b33135755
fixes: bz#1560589
Signed-off-by: Atin Mukherjee <amukherj@redhat.com>
The problem of lost stderr was introduced in
commit feea851fad4f89b48bfe89fe3b75250cc7bd6501.
Change-Id: Ic98f9bc9682ae3bd9c3ebea3855667fc8ba2843d
BUG: 1559130
Signed-off-by: Niklas Hambüchen <mail@nh2.me>
inode table size is currently set to 200000. Hence the need of change in
testcase which was expecting the old value 50000.
Change-Id: I8e44b1d0a2da1e8100bebd25f48bb36e2897b4f8
fixes: bz#1560393
Signed-off-by: Susant Palai <spalai@redhat.com>
By default server inode table size is 16K, when upcall is enabled,
there is going to be too many forgets sent on inodes as the brick can
hold only 16K inodes in memory, so we increased this to 50K. This is
still less than the client inode table size. We have seen performance
improvement when server inode table size is set to 200000(almost as
client inode table size). Hence changing the value to 200000.
Increasing this increases the memory consumption by <1MB.
BUG: 1559235
Change-Id: I931db965cd34bf33094328541bd5a633b3357805
Signed-off-by: Poornima G <pgurusid@redhat.com>
