60df33ab0b
Problem: Geo-rep sync hangs when tarssh is used as sync engine at heavy workload. Analysis and Root cause: It's found out that the tar process was hung. When debugged further, it's found out that stderr buffer of tar process on master was full i.e., 64k. When the buffer was copied to a file from /proc/pid/fd/2, the hang is resolved. This can happen when files picked by tar process to sync doesn't exist on master anymore. If this count increases around 1k, the stderr buffer is filled up. Fix: The tar process is executed using Popen with stderr as PIPE. The final execution is something like below. tar | ssh <args> root@slave tar --overwrite -xf - -C <path> It was waiting on ssh process first using communicate() and then tar. Note that communicate() reads stdout and stderr. So when stderr of tar process is filled up, there is no one to read until untar via ssh is completed. This can't happen and leads to deadlock. Hence we should be waiting on both process parallely, so that stderr is read on both processes. Backport of: > Patch: https://review.gluster.org/22684/ > Change-Id: I609c7cc5c07e210c504771115b4d551a2e891adf > BUG: 1707728 > Signed-off-by: Kotresh HR <khiremat@redhat.com> Change-Id: I609c7cc5c07e210c504771115b4d551a2e891adf fixes: bz#1709738 Signed-off-by: Kotresh HR <khiremat@redhat.com>
Gluster
Gluster is a software defined distributed storage that can scale to several petabytes. It provides interfaces for object, block and file storage.
Development
Contributions to gluster in the form of patches and new feature additions can be made by following steps outlined at Developers Guide.
Documentation
The Gluster documentation can be found at Gluster Docs.
Deployment
Quick instructions to build and install can be found in INSTALL file.
Testing
GlusterFS source contains some functional tests under tests/ directory. All
these tests are run against every patch submitted for review. If you want your
patch to be tested, please add a .t test file as part of your patch submission.
You can also submit a patch to only add a .t file for the test case you are
aware of.
To run these tests, on your test-machine, just run ./run-tests.sh. Don't run
this on a machine where you have 'production' glusterfs is running, as it would
blindly kill all gluster processes in each runs.
If you are sending a patch, and want to validate one or few specific tests, then run a single test by running the below command.
bash# /bin/bash ${path_to_gluster}/tests/basic/rpc-coverage.t
You can also use prove tool if available in your machine, as follows.
bash# prove -vmfe '/bin/bash' ${path_to_gluster}/tests/basic/rpc-coverage.t
Maintainers
The list of Gluster maintainers is available in MAINTAINERS file.
License
Gluster is dual licensed under GPLV2 and LGPLV3+.
Please visit the Gluster Home Page to find out more about Gluster.