IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
The current implementation of iobuf_pool has two problems:
- prealloc of 12.5MB memory, this limits the scale factor of the gluster
processes due to RAM requirements
- lock contention, as the current implementation has one global
iobuf_pool lock. Credits for debugging and addressing the same goes to
Krutika Dhananjay <kdhananj@redhat.com>. Issue: #410
Hence changing the iobuf implementation to use per thread mem pool.
This may theoritically appear to cause perf dip as there is no preallocation.
But per thread mem pool will not have significant perf impact as the last
allocated memory is kept alive for subsequent allocs, for some time.
The worst case would be if iobufs requested are of random sizes each time.
The best case is, if we get iobuf request of the same size. From the perf
tests, this patch did not seem to cause any perf decrease.
Note that, with this patch, the rdma performance is going to degrade
drastically. In one of the previous patchsets we had fixes to not
degrade rdma perf, but rdma is not supported and also not tested [1].
Hence the decision was to not have code in rdma that is not tested
and not supported.
[1] https://lists.gluster.org/pipermail/gluster-users.old/2018-July/034400.html
Updates: #325
Change-Id: Ic2ef3bd498f9250dea25f25ba0c01fde19584b27
Signed-off-by: Poornima G <pgurusid@redhat.com>
The inode LRU mechanism is moot in fuse xlator (ie. there is no
limit for the LRU list), as fuse inodes are referenced from
kernel context, and thus they can only be dropped on request of
the kernel. This might results in a high number of passive
inodes which are useless for the glusterfs client, causing a
significant memory overhead.
This change tries to remedy this by extending the LRU semantics
and allowing to set a finite limit on the fuse inode LRU.
A brief history of problem:
When gluster's inode table was designed, fuse didn't have any
'invalidate' method, which means, userspace application could
never ask kernel to send a 'forget()' fop, instead had to wait
for kernel to send it based on kernel's parameters. Inode table
remembers the number of times kernel has cached the inode based
on the 'nlookup' parameter. And 'nlookup' field is not used by
no other entry points (like server-protocol, gfapi etc).
Hence the inode_table of fuse module always has to have lru-limit
as '0', which means no limit. GlusterFS always had to keep all
inodes in memory as kernel would have had a reference to it.
Again, the reason for this is, kernel's glusterfs inode reference
was pointer of 'inode_t' structure in glusterfs. As it is a
pointer, we could never free it (to prevent segfault, or memory
corruption).
Solution:
In the inode table, handle the prune case of inodes with 'nlookup'
differently, and call a 'invalidator' method, which in this case is
fuse_invalidate(), and it sends the request to kernel for getting
the forget request.
When the kernel sends the forget, it means, it has dropped all
the reference to the inode, and it will send the forget with the
'nlookup' parameter too. We just need to make sure to reduce the
'nlookup' value we have when we get forget. That automatically
cause the relevant prune to happen.
Credits: Csaba Henk, Xavier Hernandez, Raghavendra Gowdappa, Nithya B
fixes: bz#1560969
Change-Id: Ifee0737b23b12b1426c224ec5b8f591f487d83a2
Signed-off-by: Amar Tumballi <amarts@redhat.com>
String functions like sprintf/snprintf/vprintf etc terminates the string with a
null character. This makes meset redundant and also incurs some performance
penalty.
Also added a comment about using GF_MALLOC instead of GF_CALLOC for allocating
buffer for strings, targeted by such functions.
Fixes: bz#1193929
Change-Id: I4f40d4fd5342b70983936119606c6c9f352c303c
Signed-off-by: Susant Palai <spalai@redhat.com>
Added the missing options and their description.
fixes: bz#1652911
Change-Id: Id9135953bc6d32e645f004e1c77449899ca3d67c
Signed-off-by: root <Shwetha Acharya>
Keeping this in glusterfs Repo for glusterfs specific improvements.
There would be a link to this page, and some more similar pages from
docs.gluster.org
updates: bz#1193929
Change-Id: Ie98670cc32e62951f1b1880b1ae39f5cc2b45329
Signed-off-by: Amar Tumballi <amarts@redhat.com>
We add dummy interrupt handling for the FLUSH
fuse message. It can be enabled by the
"--fuse-flush-handle-interrupt" hidden command line
option, or "-ofuse-flush-handle-interrupt=yes"
mount option.
It serves no other than diagnostic & demonstational
purposes -- to exercise the interrupt handling framework
a bit and to give an usage example.
Documentation is also provided that showcases interrupt
handling via FLUSH.
Change-Id: I522f1e798501d06b74ac3592a5f73c1ab0590c60
updates: #465
Signed-off-by: Csaba Henk <csaba@redhat.com>
Also update the required documents and scripts to enable clang-format
Change-Id: I73aae6db06c2f732a1779d59a73bc05e28beafba
updates: bz#1564149
Signed-off-by: Amar Tumballi <amarts@redhat.com>
This helps to traverse the docs well as README.md is rendered
by default in github pages, and would help to provide better indexed
document for new users.
updates: bz#1193929
Change-Id: I4130dfd16aed924ecab17f96326cedb86892d6a6
Signed-off-by: Amar Tumballi <amarts@redhat.com>
Having the links in README makes people follow it if they browse
through github.
Updates: bz#1193929
Change-Id: I580332a41d9b52858aa9796a74a563ff57a60b4b
Signed-off-by: Amar Tumballi <amarts@redhat.com>
Detailing how the commit message should be is very important for imposing
some structure, and good practises. Also it helps new developers to do the
right things.
This is mostly copied from the OpenStack commit message reference, as it
was overlapping with most of the things we follow in GlusterFS. Made
minor changes to make it suitable for Gluster project.
Updates: bz#1193929
Change-Id: I4dcd7ff366edfc6c1727a287f32bcf7270c86b11
Signed-off-by: Amar Tumballi <amarts@redhat.com>
`doc/xlator-classification.md` talks about the reasoning and expectations
Reviewers are expected to check the 'category' of new
option / translator added in the codebase, and make sure the flag
is always properly set. It helps to keep the 'expectation' proper
on the codebase.
updates: #430
Change-Id: I2bfc9934a5f6eed77fcc3e20364046242decc82c
Signed-off-by: Amar Tumballi <amarts@redhat.com>
[sh]$ gcc --version
gcc (GCC) 7.3.1 20180303 (Red Hat 7.3.1-5)
Warnings were of the type below:
xlators/mgmt/glusterd/src/glusterd-store.c:3285:33:
warning: ‘/options’ directive output may be truncated writing 8 bytes into a region of size between 1 and 4096 [-Wformat-truncation=]
snprintf (path, len, "%s/options", conf->workdir);
^~~~~~~~
xlators/mgmt/glusterd/src/glusterd-store.c:1280:39:
warning: ‘/snaps/’ directive output may be truncated writing 7 bytes into a region of size between 1 and 4096 [-Wformat-truncation=]
snprintf (snap_fpath, len, "%s/snaps/%s/%s", priv->workdir,
^~~~~~~
* Also changed some places where there was issues with key size
* Made sure all the 'char buf[SOMESIZE] = {0,};' are changed to 'char buf[SOMESIZE] = "";`
- In the files I changed
* Also edited coding standard to reflect that.
updates: bz#1193929
Change-Id: I04c652624ac63199cea2077e46b3a5def37c3689
Signed-off-by: Amar Tumballi <amarts@redhat.com>
This is a recommendation for users, and reviewers can take a point from this.
Updates: bz#1193929
Change-Id: Idcd778e42a886fd79b549da4927149a07573a20b
Signed-off-by: Amar Tumballi <amarts@redhat.com>
Keeping two copies of the files means, one would be out-of-date soon,
and users would always be confused about which one is the source of
truth.
Updates: bz#1193929
Change-Id: I568149732fdb9d282ccd583640eee9b9056963fd
Signed-off-by: Amar Tumballi <amarts@redhat.com>
This is part of the effort to provide consistent time
across distribute and replica set for time attributes
(ctime, atime, mtime) of the object. This patch contains
the APIs to set and get the attributes from on disk
and in inode context.
Credits: Rafi KC <rkavunga@redhat.com>
Updates: #208
Change-Id: I5d3cba53eef90ac252cb8299c0da42ebab3bde9f
Signed-off-by: Kotresh HR <khiremat@redhat.com>
- Added kernel-writeback-cache command line and xlator
option for requesting utilisation of the writeback
cache of the kernel in FUSE_INIT (see [1]).
- Added attr-times-granularity command line and xlator
option via which granularity of the {a,m,c}time in
stat (attr) data that we support can be indicated to
kernel. This is a means to avoid divergence of the
attr times between kernel and userspace that could
occur with writeback-cache, while still maintaining
maximum time precision the FUSE server is capable of
(see [2]).
- Handling FATTR_CTIME flag in FUSE_SETATTR that
indicates presence of ctime in setattr payload.
Currently we cannot associate arbitrary ctimes to
files on backend, so we just touch them to update
their ctimes to current time. Having ctimes in setattr
payload is also a side effect of writeback cache
(see [3] and [4]).
[1]: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=4d99ff8,
"fuse: Turn writeback cache on"
[2]: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=e27c9d3,
"fuse: fuse: add time_gran to INIT_OUT"
[3]: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=1e18bda,
"fuse: add .write_inode"
[4]: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=ab9e13f,
"fuse: allow ctime flushing to userspace"
Updates: #435
Change-Id: Id174c8e0c815c4456c35f8c53e41a6a507d91855
Signed-off-by: Csaba Henk <csaba@redhat.com>
The libgfapi-perl provides declarations and linkage for the Gluster
gfapi C library with FFI for many Perl mongers
In addition, gogfapi URI link is replaced with GitHub because Forge is
dead.
Change-Id: I773e78beb201b48ca3fde0dc72d04b64dc9697d6
Signed-off-by: Ji-Hyeon Gim <potatogim@potatogim.net>
Updates: #447
Update the existing admin guide link as it is incorrect.
Change-Id: I05669192623aeac287dfa9002caa0f390ea79499
Updates: bz#1193929
Signed-off-by: Varsha Rao <varao@redhat.com>
Looking at the man page show that "Snapshot command" wasn't aligned
with the other section titles.
Change-Id: I24bdb2e3728e03862fee57710cfe34b0607fe09a
BUG: 1507230
Signed-off-by: Michael Scherer <misc@redhat.com>
So far the --direct-io-mode option has been presented
as of being Boolean valued. That is however not exact,
as a third behavior is chosen if the option is not
specified.
We accept now the "auto" value as an explicit choice
for the default heuristics, and indicate in the
descriptions of the option (which occur in commandline
help and in the gluterfs / mount.glusterfs man pages)
that auto is the default.
The default heuristics was briefly described in the
commandline help. We are getting rid of that, because:
- it's not the right place to provide such details;
- there is no guarantee of keeping the current heuristics
so it might go out of sync with reality;
- that is already the case to some degree, because the
description did not take into account that the default
heuristics varies between platforms (on Mac, it's just
"off"), and that xlators can also prescribe direct I/O
for the file of their choice (see change
I3fe3312cd96baa4eecfe1247ab7255b4f455f049).
Change-Id: Ia83479c0c67fe66b7fc2e0e8db5b7792d9f44b28
Signed-off-by: Csaba Henk <csaba@redhat.com>
With Gluster 4.0 we will not provide the server components for EL6 and
older. At one point Gluster 4.x will get GlusterD2, which requires
Golang tools in the distribution. EL6 does not contain these at the
moment.
With this change, it is possible to `./configure --without-server` which
prevents building glusterd and the xlators for the bricks. Building RPMs
can pass `--without server` and the glusterfs-server sub-package will
not be created.
Change-Id: I97f5ccf9f2c76e60d9af83915fc59fae57ad6d25
BUG: 1074947
Signed-off-by: Niels de Vos <ndevos@redhat.com>
With configure --enable-debug, add all object allocations
to a list in the corresponding mem_acct_rec. This
allows us to see all objects of a particular type
and allows for additional debugging in case of memory
leaks.
This is not compiled in by default and must be explicitly
enabled. It is intended to be used by developers.
Change-Id: I7cf2dbeadecf994423d7e7591e85f18d2575cce8
BUG: 1522662
Signed-off-by: N Balachandran <nbalacha@redhat.com>
The option 'hourly' is supported for 'scrub-frequency'
but man page was not reflecting it. Fixed the same.
Change-Id: Ida0929c805f6f5332c5dcd94cda9ecf2f4eee20e
BUG: 1503510
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Changes:
1. Take subdir mount option in client (mount.gluster / glusterfsd)
2. Pass the subdir mount to server-handshake (from client-handshake)
3. Handle subdir-mount dir's lookup in server-first-lookup and handle
all fops resolution accordingly with proper gfid of subdir
4. Change the auth/addr module to handle the multiple subdir entries
in option, and valid parsing.
How to use the feature:
`# mount -t glusterfs $hostname:/$volname/$subdir /$mount_point`
Or
`# mount -t glusterfs $hostname:/$volname -osubdir_mount=$subdir /$mount_point`
Option can be set like:
`# gluster volume set <volname> auth.allow "/subdir1(192.168.1.*),/(192.168.10.*),/subdir2(192.168.8.*)"`
Updates #175
Change-Id: I7ea57f76ddbe6c3862cfe02e13f89e8a39719e11
Signed-off-by: Amar Tumballi <amarts@redhat.com>
Reviewed-on: https://review.gluster.org/17141
Smoke: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
This commit makes the get-state CLI capable to returning the values
for all volume options for all volumes. This is similar to what you
get when you issue a `gluster volume get <volname> all` command.
This is the new usage for the get-state CLI:
# gluster get-state [<daemon>] [[odir </path/to/output/dir/>] \
[file <filename>]] [detail|volumeoptions]
Fixes: #277
Change-Id: Ice52d936a5a389c6fa0ba5ab32416a65cdfde46d
Signed-off-by: Samikshan Bairagya <samikshan@gmail.com>
Reviewed-on: https://review.gluster.org/17858
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
Reviewed-by: Gaurav Yadav <gyadav@redhat.com>
Smoke: Gluster Build System <jenkins@build.gluster.org>
Also give hints on where to look for relevant structures.
Fixes#140
Change-Id: If29f67fb6fd39379045fe774559934fb13cbede8
Signed-off-by: Amar Tumballi <amarts@redhat.com>
Reviewed-on: https://review.gluster.org/16925
Smoke: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
Tested-by: Raghavendra G <rgowdapp@redhat.com>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
This new xlator does not allocate any resources on init(). This makes it
a good option to use for debugging xlator releated resources leaks on
fini().
By putting the sink xlator as single xlator in a .vol file, and loading
it through gfapi, we can investigate the resource leaks that are
happening through gfapi (and the Gluster core). By extending the .vol
file with additional xlators, it is possible to analyze resource leaks
of single xlators.
Change-Id: Idb5faa861b623dd5b2a988b181e669b0d52c2a0e
BUG: 1425623
Fixes: #176
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: https://review.gluster.org/16806
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Smoke: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
This commit optionally adds client details corresponding to the
locally running bricks to the get-state output. Since getting
the client details involves sending RPC requests to the respective
local bricks, this is a relatively more costly operation. These
client details would be added to the get-state output only if the
get-state command is invoked with the 'detail' option.
This commit therefore also changes the get-state CLI usage. The
modified usage is as follows:
# gluster get-state [<daemon>] [[odir </path/to/output/dir/>] \
[file <filename>]] [detail]
Change-Id: I42cd4ef160f9e96d55a08a10d32c8ba44e4cd3d8
BUG: 1431183
Signed-off-by: Samikshan Bairagya <samikshan@gmail.com>
Reviewed-on: https://review.gluster.org/17003
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
Smoke: Gluster Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
A new document explaining how ec encoding/decoding is implemented
has been added to the developer guide.
Change-Id: I344493477bf42b935ed75f7cf0574d54dc1b9509
Signed-off-by: Xavier Hernandez <xhernandez@datalab.es>
Reviewed-on: https://review.gluster.org/15637
Smoke: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Ashish Pandey <aspandey@redhat.com>
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
Tested-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
With this, we will be able to trigger statedumps on remote Gluster
clients, mainly targetted for applications using libgfapi.
Design:
SIGUSR signal is the most comman way of taking a statedump in Gluster.
But it cannot be used for libgfapi based processes, as the process
loading the library might have already consumed SIGUSR signal. Hence
going by the command way.
One has to issue a Gluster command to initiate a statedump on the
libgfapi based client. The command takes hostname and PID as an
argument. All the glusterds in the cluster, check if they are connected
to the specified hostname, and send an RPC request to all the connected
clients from that hostname (via the mgmt connection).
URL: http://review.gluster.org/16357
Change-Id: Icbe4d2f026b32a2c7d5535e1bfb2cdaaff042e91
BUG: 1169302
Signed-off-by: Poornima G <pgurusid@redhat.com>
[ndevos: minor fixes and split patch in smaller pieces]
Reviewed-on: https://review.gluster.org/9228
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Tested-by: Niels de Vos <ndevos@redhat.com>
Smoke: Gluster Build System <jenkins@build.gluster.org>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Samikshan Bairagya <samikshan@gmail.com>
Debugging inode ref leaks is very difficult as there is no info
except for the ref count on the inode. Hence this patch is first step
towards debugging inode ref leaks. With this patch, in the statedump
we get additional info that tells the ref count taken by each xlator
on the inode.
Change-Id: I7802f7e7b13c04eb4d41fdf52d5469fd2c2a185a
BUG: 1325531
Signed-off-by: Poornima G <pgurusid@redhat.com>
Reviewed-on: http://review.gluster.org/13736
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
Smoke: Gluster Build System <jenkins@build.gluster.org>
This reverts commit c6fdc2f819b265e018d858232293407835fe2c6c.
Initially I thought gluster --xml taking into the shell prompt is not a
correct behavior, however if we execute any commands from that prompt we
get xml output which is the original intention. This patch reverts the
change and put back --xml option for gluster in man page.
Change-Id: If77d0c0bf27f8cfc799973003c860d9bd8cd6407
BUG: 1360670
Signed-off-by: Atin Mukherjee <amukherj@redhat.com>
Reviewed-on: http://review.gluster.org/16157
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: MOHIT AGRAWAL <moagrawa@redhat.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
Smoke: Gluster Build System <jenkins@build.gluster.org>
gluster --xml will take into the interactive prompt which is not an
intended message here.
Change-Id: Iea5b094a1b1b429d211008a3c7459b703c1a0a12
BUG: 1360670
Signed-off-by: Atin Mukherjee <amukherj@redhat.com>
Reviewed-on: http://review.gluster.org/16142
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
Smoke: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
Currently there is no existing CLI that can be used to get the
local state representation of the cluster as maintained in glusterd
in a readable as well as parseable format.
The CLI added has the following usage:
# gluster get-state [daemon] [odir <path/to/output/dir>] [file <filename>]
This would dump data points that reflect the local state
representation of the cluster as maintained in glusterd (no other
daemons are supported as of now) to a file inside the specified
output directory. The default output directory and filename is
/var/run/gluster and glusterd_state_<timestamp> respectively. The
option for specifying the daemon name leaves room to add support for
other daemons in the future. Following are the data points captured
as of now to represent the state from the local glusterd pov:
* Peer:
- Primary hostname
- uuid
- state
- connection status
- List of hostnames
* Volumes:
- name, id, transport type, status
- counts: bricks, snap, subvol, stripe, arbiter, disperse,
redundancy
- snapd status
- quorum status
- tiering related information
- rebalance status
- replace bricks status
- snapshots
* Bricks:
- Path, hostname (for all bricks these info will be shown)
- port, rdma port, status, mount options, filesystem type and
signed in status for bricks running locally.
* Services:
- name, online status for initialised services
* Others:
- Base port, last allocated port
- op-version
- MYUUID
Change-Id: I4a45cc5407ab92d8afdbbd2098ece851f7e3d618
BUG: 1353156
Signed-off-by: Samikshan Bairagya <samikshan@gmail.com>
Reviewed-on: http://review.gluster.org/14873
Reviewed-by: Avra Sengupta <asengupt@redhat.com>
Smoke: Gluster Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
