IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
The current code assumes cmockery2 is installed in default paths.
Use PKG_MODULES_CHECK to find it using pkg-config if it is not. If
not found by pkg-config, try AC_CHECK_LIB.
There are also some build flag adjustement so that local overrides
do not loose the required -I flags.
This includes and enhance http://review.gluster.org/8340/
BUG: 764655
Change-Id: Ide9f77d1e70afe3c1c5c57ae2b93127af6a425f9
Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org>
Reviewed-on: http://review.gluster.org/8365
Reviewed-by: Harshavardhana <harsha@harshavardhana.net>
Tested-by: Harshavardhana <harsha@harshavardhana.net>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
This patch will allow for developers to create unit tests for
their code. Documentation has been added to the patch and
is available here:
doc/hacker-guide/en-US/markdown/unittest.md
Also, unit tests are run when RPM is created.
This patch is a replacement for http://review.gluster.org/#/c/7281
which removed unit test infrastucture from the repo due to multiple
conflicts. Cmockery2 is now available in Fedora and EPEL, and soon
to be available in Debian and Ubuntu. For all other operating
systems, please install from the source:
https://github.com/lpabon/cmockery2
BUG: 1067059
Change-Id: I1b36cb1f56fd10916f9bf535e8ad080a3358289f
Signed-off-by: Luis Pabón <lpabon@redhat.com>
Reviewed-on: http://review.gluster.org/7538
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
python binding of libgfapi is evolving via libgfapi-python
project. The example available here confuses end users,
so removing it from this path.
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
Bug Id: 1119328
> Reviewed-on: http://review.gluster.org/7920
> Reviewed-by: Humble Devassy Chirammal <humble.devassy@gmail.com>
> Tested-by: Humble Devassy Chirammal <humble.devassy@gmail.com>
> Reviewed-by: Thiago da Silva <thiago@redhat.com>
> Reviewed-by: Prashanth Pai <ppai@redhat.com>
> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
Change-Id: I17b3aa0f0505342496019ce012cca21d84184027
Reviewed-on: http://review.gluster.org/8237
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
- Provides a working Gluster Management Daemon, CLI
- Provides a working GlusterFS server, GlusterNFS server
- Provides a working GlusterFS client
- execinfo port from FreeBSD is moved into ./contrib/libexecinfo
for ease of portability on NetBSD. (FreeBSD 10 and OSX provide
execinfo natively)
- More portability cleanups for Darwin, FreeBSD and NetBSD
- Provides a new rc script for FreeBSD
Change-Id: I8dff336f97479ca5a7f9b8c6b730051c0f8ac46f
BUG: 1111774
Original-Author: Mike Ma <mikemandarine@gmail.com>
Signed-off-by: Harshavardhana <harsha@harshavardhana.net>
Reviewed-on: http://review.gluster.org/8141
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
python binding of libgfapi is evolving via libgfapi-python
project. The example available here confuses end users,
so removing it from this path.
Change-Id: I90a0a29972f5230b8b440e8941aaca8a62e497a6
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
Reviewed-on: http://review.gluster.org/7920
Reviewed-by: Humble Devassy Chirammal <humble.devassy@gmail.com>
Tested-by: Humble Devassy Chirammal <humble.devassy@gmail.com>
Reviewed-by: Thiago da Silva <thiago@redhat.com>
Reviewed-by: Prashanth Pai <ppai@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
The current unused implementation for message-ids in the logs depends on
automatically generated files. The generated files are not included in
the distributed tarball. This causes issues when distributions build
packages, they need to re-run ./autogen.sh to create the needed files.
I thought of including the generated files in the distribution tarball.
However, the contents of these files are not actively used, so it seems
to make more sense to drop it all together. These functions were the
only users of libintl and gettext too, so dropped the requirement
checking from configure.ac.
A replacement for the message-id logging framework is in progress. Any
changes that this patch makes, can be reverted in the submission of
patches for the new framework.
Reference: http://thread.gmane.org/gmane.comp.file-systems.gluster.devel/6212
Change-Id: Iea82dd3910944a5c6be3ee393806eccabd575e11
BUG: 1038391
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: http://review.gluster.org/7714
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
- Shell scripts: == is specific to bash and ksh. Use = instead.
- Shell scripts: use sh instead of bash if bash functionnality is not used
- Shell scripts: ${var/search/replace} is specific to bash
- sed: The -i option is specific to GNU sed.
- Makefiles: $< outside of generic rules only work in GNU make.
- xdrproc_t() is not universally defined as variadic. Do not specify third
argument if it is not used
- NetBSD FUSE specific: only include <perfuse.h> in FUSE client code,
it harms in other locations
- configure: Search for gettext() in libintl as NetBSD stores it there
- Like MacOS X, NetBSD has unmount(2) and not umount(2) (un vs u)
Some other build issues previously included in this change were
removed:
- __THROW macro, addressed in http://review.gluster.com/#/c/7757/
- getmntent() compat shared with MacOS X, in http://review.gluster.com/#/c/7722/
This patchset adds warning fixes for mount_glusterfs
BUG: 764655
Change-Id: I2f1faf8ff96362d3e2baf237b943df619011f1f4
Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org>
Reviewed-on: http://review.gluster.org/7783
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Harshavardhana <harsha@harshavardhana.net>
Defined new APIs in the libgfapi module, given a glfs object,
* to send handshake RPC call to glusterd process to fetch UUID of the volume
* store it in the glusterfs_context linked to the glfs object.
* to parse UUID from its cannonical string format into 16-byte array
before sending it to the libgfapi users.
Defined a RPC call in glusterd which can be used to query volume related
info by other processes using 'clnt_handshake_procs'.
Note - Currently this RPC call to glusterd process is used only to fetch UUID.
But it can be extended to get other volume related structures as well.
In addition to the above, defined a new variable to keep track of such handshake
RPCs still in progress to make sure all the corresponding RPC callbacks have been
processed before libgfapi returns the glfs object initialized.
Also bumping up the GFAPI current version number since there is a new API
"glfs_get_volume_id" defined and exposed by libgfapi as part of these changes.
Change-Id: I303f76d7177d32d25bdb301b1dbcf5cd73f42807
BUG: 1090363
Signed-off-by: Soumya Koduri <skoduri@redhat.com>
Reviewed-on: http://review.gluster.org/7218
Reviewed-by: Anand Avati <avati@redhat.com>
Reviewed-by: Harshavardhana <harsha@harshavardhana.net>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
Optimization flags has led to segfaults at wrong locations
throughout gluster code for versions < 10.9
Assuming this to be a compiler bug, disable optimization
flags
Change-Id: Ia2dc983dc5bb06935f03b68c07688ce41255d7da
BUG: 1089172
Signed-off-by: Harshavardhana <harsha@harshavardhana.net>
Reviewed-on: http://review.gluster.org/7656
Reviewed-by: Justin Clift <justin@gluster.org>
Tested-by: Justin Clift <justin@gluster.org>
The meta translator exposes details about glusterfs itself
in the form of a virtual namespace.
Loading the translator on the client side creates the
meta virtual view under $mntpoint/.meta by default. The
directory is not listed (even with ls -a) and can be
accessed by doing a "cd /mnt/.meta"
Change-Id: I5ffdf39203841a9562a8280a1f79dc76d4dded5d
BUG: 1089216
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/7509
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Harshavardhana <harsha@harshavardhana.net>
1. gluster volume set nfs-ganesha.enable ON/OFF
If the option is set to ON, the volume field in the nfs-ganesha configuartion file is
edited. Gluster-nfs is disabled on that volume and the volume is exported using
nfs-ganesha.
2.gluster volume set nfs-ganesha.host IP
This is used to provide the IP of the nfs-ganesha host.
Note : nfs-ganesha.host MUST be set before using nfs-ganesha.enable ON
The switch from gluster-nfs to nfs-ganesha is mostly done by the hook-scripts
in the post phase of the 'set' option. As a result, gluster volume reset does not
function as it is expected to. By default, nfs-ganesha will be set to off but the
process will not be killed.
Hence, a few changes have to be made post 'reset' option as well. Those changes
also have been added.
Change-Id: I7fdc14ee49d1724af96eda33c6a3ec08b1020788
BUG: 1092283
Signed-off-by: Meghana <mmadhusu@redhat.com>
Reviewed-on: http://review.gluster.org/7321
Reviewed-by: Raghavendra Talur <rtalur@redhat.com>
Reviewed-by: Santosh Pradhan <spradhan@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
Avoid modifying autogenerated files and keeping them in
repository - autogenerate them on demand from ".x" files
Change-Id: I2cdb1fe9b99768ceb80a8cb100fa00bd1d8fe2c6
BUG: 1090807
Signed-off-by: Harshavardhana <harsha@harshavardhana.net>
Reviewed-on: http://review.gluster.org/7526
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
This patch still configures argp, i.e. creates Makefile in
/contrib/argp-standalone, for the `make dist`, but there's no reason
to compile it when it's not going to be linked into anything.
Change-Id: Id0fcb717b5821d8a4c176e6274339f46b3a4a249
BUG: 1086773
Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
Reviewed-on: http://review.gluster.org/7440
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Harshavardhana <harsha@harshavardhana.net>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
This patch brings version and release number generated at build time
using git tags or fixed content from VERSION file.
With git tag, version/release number are got from output of
'git describe --tags --match "v[0-9]*"' command.
This behavior can be overriden by having VERSION file with fixed
version/release. The VERSION file should have text describing version
and release for example something like 'v3.4.0-1'
For testing this patch, its required to remove autom4te.cache
directory to avoid seeing previously set version.
BUG: 1074919
Change-Id: I8f68172e8b389b0ba0846e9adb4b597e67a909aa
Signed-off-by: Bala.FA <barumuga@redhat.com>
Reviewed-on: http://review.gluster.org/7164
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
While we wait for cmockery2 to be available from Fedora,
we can remove cmockery2 from the repo.
BUG: 1077011
Change-Id: I75d462c607cd376a5d838ea83f4d12eb59757e73
Signed-off-by: Luis Pabon <lpabon@redhat.com>
Reviewed-on: http://review.gluster.org/7281
Reviewed-by: Justin Clift <justin@gluster.org>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Harshavardhana <harsha@harshavardhana.net>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
Moved the hook-scripts to dirs corresponding to event.
Also, edited S30samba-set.sh to with changes in
other two hook scripts to bring uniformity.
Change-Id: Ib98c0a18d4bb5d0ba4b881b2e19b40d72580b1c2
BUG: 1073468
Signed-off-by: Raghavendra Talur <rtalur@redhat.com>
Reviewed-on: http://review.gluster.org/7201
Reviewed-by: Poornima G <pgurusid@redhat.com>
Reviewed-by: Bala FA <barumuga@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
This patch will allow for developers to create unit tests for
their code. Documentation has been added to the patch and
is available here:
doc/hacker-guide/en-US/markdown/unittest.md
Also, unit tests are run when RPM is created.
BUG: 1067059
Change-Id: I95cf8bb0354d4ca4ed4476a0f2385436a17d2369
Signed-off-by: Vijay Bellur <vbellur@redhat.com>
Signed-off-by: Luis Pabon <lpabon@redhat.com>
Reviewed-on: http://review.gluster.org/7145
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Rajesh Joseph <rjoseph@redhat.com>
Reviewed-by: Justin Clift <justin@gluster.org>
Tested-by: Justin Clift <justin@gluster.org>
Problem:
Geo-replication uses volume info xml output.
While installing glusterfs through build+install from source,
and if the libxml2-devel package is not present in the system,
build will succeed with just warning and XML output : no.
If user fails to observe this and doesn't know that
geo-rep is dependent on xml output, user won't be able use
geo-rep.
Solution:
If libxml2-devel is unavailable, and if geo-rep is enabled error out in configure,
otherwise just warn with XML_OUTPUT = no.
If the xml-output is disabled while configuring, error out if
geo-rep is enabled, otherwise just warn.
Change-Id: Icde5d0c1e5f3b869e7db6ed6a046bdc49b450fcb
BUG: 1066997
Signed-off-by: Vijaykumar Koppad <vkoppad@redhat.com>
Reviewed-on: http://review.gluster.org/7131
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
The configure script was discarding the passed CFLAGS inadvertently.
This caused failures while building rpms on epel7 and fedora-{19,20}, as
the hardened CFLAGS passed by rpmbuild on these distros were discarded
which lead to the failure of rpmbuild.
Thanks to Neils de Vos for helping track this down.
Change-Id: I5b2aa5d449d13bf67df6c19cda178bddfe6c3652
BUG: 1066385
Signed-off-by: Kaushal M <kaushal@redhat.com>
Reviewed-on: http://review.gluster.org/7029
Reviewed-by: Justin Clift <jclift@redhat.com>
Tested-by: Justin Clift <jclift@redhat.com>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Tested-by: Niels de Vos <ndevos@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
The CFLAGS were being polluted by the python test for glupy. The
test also sets CFLAGS -NDEBUG even when passing --enable-debug to
configure.
To fix it, the patch now saves the necessary python flags into
new makefile flags which can now be used by the glupy Makefile.am.
BUG: 1063541
Change-Id: I8546a64b8ad4fe9318a97fb6a2f0c36087be1b29
Signed-off-by: Luis Pabon <lpabon@redhat.com>
Reviewed-on: http://review.gluster.org/6957
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Harshavardhana <harsha@harshavardhana.net>
Tested-by: Harshavardhana <harsha@harshavardhana.net>
------------------------------------->
"In essence, every time you make a change to the library and
release it, the C:R:A should change. A new library should
start with 0:0:0. Each time you change the public interface
(i.e., your installed header files), you should increment the
CURRENT number."
So the version which was 3 became 5 when discard API was added
and became 6 when zerofill was added. As per the above description,
API addition will result in a change to the installed header (glfs.h)
and hence warrants a change in CURRENT number ? Should you have
changed the version to 6.0.0 instead of 0.0.6 ?
<------------------------------------
Thanks Bharata B Rao <bharata.rao@gmail.com> for pointing this out
Change-Id: I0aa980a6713fbd28922a94c5debb170a89aa3512
BUG: 862082
Signed-off-by: Harshavardhana <harsha@harshavardhana.net>
Reviewed-on: http://review.gluster.org/6790
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Bharata B Rao <bharata.rao@gmail.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
According to libtool three individual numbers stand for
CURRENT:REVISION:AGE, or C:R:A for short. The libtool
script typically tacks these three numbers onto the end
of the name of the .so file it creates. The formula for
calculating the file numbers on Linux and Solaris is
/path/to/library/<library_name>.(C - A).(A).(R)
As you release new versions of your library, you will
update the library's C:R:A. Although the rules for changing
these version numbers can quickly become confusing, a few
simple tips should help keep you on track. The libtool
documentation goes into greater depth.
In essence, every time you make a change to the library and
release it, the C:R:A should change. A new library should start
with 0:0:0. Each time you change the public interface
(i.e., your installed header files), you should increment the
CURRENT number. This is called your interface number. The main
use of this interface number is to tag successive revisions
of your API.
The AGE number is how many consecutive versions of the API the
current implementation supports. Thus if the CURRENT library
API is the sixth published version of the interface and it is
also binary compatible with the fourth and fifth versions
(i.e., the last two), the C:R:A might be 6:0:2. When you break
binary compatibility, you need to set AGE to 0 and of course
increment CURRENT.
The REVISION marks a change in the source code of the library
that doesn't affect the interface-for example, a minor bug fix.
Anytime you increment CURRENT, you should set REVISION back to 0.
Change-Id: Id72e74c1642c804fea6f93ec109135c7c16f1810
BUG: 862082
Signed-off-by: Harshavardhana <harsha@harshavardhana.net>
Reviewed-on: http://review.gluster.org/5645
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
POSIX does not says wether link(2) on symlink should link on
symlink itself or on target. Linux use symlink, most other
systems use target. Using linkat(2) allows the behavior to be
specified, so that the behavior is portable.
Also fix configure test for NetBSD linkata(2), which ceased to work.
BUG: 764655
Change-Id: Iccd27ac076b7a74e40dcbaa1c4762fd3ad59da5f
Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org>
Reviewed-on: http://review.gluster.org/6539
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
Issue:
Quota directory limit configuration is stored in the xattrs. When a new brick
is added these 'limit-set' xattrs have to be created to the directory in the
new brick. This is done by the dht directory healing when the directory is
created in the new brick. Since 'root' directory is already created DHT doesn't
heal the limit-set xattr root.
Solution:
When the add-brick command is issued run the below hook script to heal the
'limit-set' xattr. The hook script does the following only if limit is
configured on root.
1. Create an auxiliary mount.
2. getxattr 'limit-set' on the root
3. setxattr the same value on the root
But this script needs the volume to be started to make the auxiliary mount.
To handle the case when the add-brick is issued when the volume was stopped,
symlink is created by the 'master' script to the corresponding location and
these two are by default disabled.
So, a 'master' script is added in the add-brick/pre. When add-brick command is
issued, it enables one of the scripts mentioned above based on the condition,
if volume is started - enable add-brick/post script
else - enable start/post script
After the actual script completes its job, it disables itself.
Note:
The enabling and disabling of the script is based on the glusterd's logic, that
it only runs the scripts which starts its name with 'S'. So,
Enable - symlink the file to 'S'*
Disable - unlink the symlink.
Change-Id: I2d3947a4d686c54417ec95f530af3bdd3444f4e2
BUG: 969461
Signed-off-by: Varun Shastry <vshastry@redhat.com>
Reviewed-on: http://review.gluster.org/6104
Reviewed-by: Brian Foster <bfoster@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
The value was getting stomped by the Python stuff (for glupy) after we
had supposedly dealt with enable_debug, so we were getting stray -O2
options that hamper debugging.
Change-Id: Iacd616071c83b92018b597b6144ab565bd5fc66f
Signed-off-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-on: http://review.gluster.org/6362
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
If gettext() is not found in libc, look it up in libintl (this is where
NetBSD has it)
BUG: 764655
Change-Id: Ifba8681b8603ead5d0b8587b71457250982077e1
Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org>
Reviewed-on: http://review.gluster.org/6287
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
.. in the systems with non-trusted server
This new functionality can be useful in various cloud technologies.
It is implemented via a special encryption/crypt translator,which
works on the client side and performs encryption and authentication;
1. Class of supported algorithms
The crypt translator can support any atomic symmetric block cipher
algorithms (which require to pad plain/cipher text before performing
encryption/decryption transform (see glossary in atom.c for
definitions). In particular, it can support algorithms with the EOF
issue (which require to pad the end of file by extra-data).
Crypt translator performs translations
user -> (offset, size) -> (aligned-offset, padded-size) ->server
(and backward), and resolves individual FOPs (write(), truncate(),
etc) to read-modify-write sequences.
A volume can contain files encrypted by different algorithms of the
mentioned class. To change some option value just reconfigure the
volume.
Currently only one algorithm is supported: AES_XTS.
Example of algorithms, which can not be supported by the crypt
translator:
1. Asymmetric block cipher algorithms, which inflate data, e.g. RSA;
2. Symmetric block cipher algorithms with inline MACs for data
authentication.
2. Implementation notes.
a) Atomic algorithms
Since any process in a stackable file system manipulates with local
data (which can be obsoleted by local data of another process), any
atomic cipher algorithm without proper support can lead to non-POSIX
behavior. To resolve the "collisions" we introduce locks: before
performing FOP->read(), FOP->write(), etc. the process should first
lock the file.
b) Algorithms with EOF issue
Such algorithms require to pad the end of file with some extra-data.
Without proper support this will result in losing information about
real file size. Keeping a track of real file size is a responsibility
of the crypt translator. A special extended attribute with the name
"trusted.glusterfs.crypt.att.size" is used for this purpose. All files
contained in bricks of encrypted volume do have "padded" sizes.
3. Non-trusted servers and
Metadata authentication
We assume that server, where user's data is stored on is non-trusted.
It means that the server can be subjected to various attacks directed
to reveal user's encrypted personal data. We provide protection
against such attacks.
Every encrypted file has specific private attributes (cipher algorithm
id, atom size, etc), which are packed to a string (so-called "format
string") and stored as a special extended attribute with the name
"trusted.glusterfs.crypt.att.cfmt". We protect the string from
tampering. This protection is mandatory, hardcoded and is always on.
Without such protection various attacks (based on extending the scope
of per-file secret keys) are possible.
Our authentication method has been developed in tight collaboration
with Red Hat security team and is implemented as "metadata loader of
version 1" (see file metadata.c). This method is NIST-compliant and is
based on checking 8-byte per-hardlink MACs created(updated) by
FOP->create(), FOP->link(), FOP->unlink(), FOP->rename() by the
following unique entities:
. file (hardlink) name;
. verified file's object id (gfid).
Every time, before manipulating with a file, we check it's MACs at
FOP->open() time. Some FOPs don't require a file to be opened (e.g.
FOP->truncate()). In such cases the crypt translator opens the file
mandatory.
4. Generating keys
Unique per-file keys are derived by NIST-compliant methods from the
a) parent key;
b) unique verified object-id of the file (gfid);
Per-volume master key, provided by user at mount time is in the root
of this "tree of keys".
Those keys are used to:
1) encrypt/decrypt file data;
2) encrypt/decrypt file metadata;
3) create per-file and per-link MACs for metadata authentication.
5. Instructions
Getting started with crypt translator
Example:
1) Create a volume "myvol" and enable encryption:
# gluster volume create myvol pepelac:/vols/xvol
# gluster volume set myvol encryption on
2) Set location (absolute pathname) of your master key:
# gluster volume set myvol encryption.master-key /home/me/mykey
3) Set other options to override default options, if needed.
Start the volume.
4) On the client side make sure that the file /home/me/mykey exists
and contains proper per-volume master key (that is 256-bit AES
key). This key has to be in hex form, i.e. should be represented
by 64 symbols from the set {'0', ..., '9', 'a', ..., 'f'}.
The key should start at the beginning of the file. All symbols at
offsets >= 64 are ignored.
5) Mount the volume "myvol" on the client side:
# glusterfs --volfile-server=pepelac --volfile-id=myvol /mnt
After successful mount the file which contains master key may be
removed. NOTE: Keeping the master key between mount sessions is in
user's competence.
**********************************************************************
WARNING! Losing the master key will make content of all regular files
inaccessible. Mount with improper master key allows to access content
of directories: file names are not encrypted.
**********************************************************************
6. Options of crypt translator
1) "master-key": specifies location (absolute pathname) of the file
which contains per-volume master key. There is no default location
for master key.
2) "data-key-size": specifies size of per-file key for data encryption
Possible values:
. "256" default value
. "512"
3) "block-size": specifies atom size. Possible values:
. "512"
. "1024"
. "2048"
. "4096" default value;
7. Test cases
Any workload, which involves the following file operations:
->create();
->open();
->readv();
->writev();
->truncate();
->ftruncate();
->link();
->unlink();
->rename();
->readdirp().
8. TODOs:
1) Currently size of IOs issued by crypt translator is restricted
by block_size (4K by default). We can use larger IOs to improve
performance.
Change-Id: I2601fe95c5c4dc5b22308a53d0cbdc071d5e5cee
BUG: 1030058
Signed-off-by: Edward Shishkin <edward@redhat.com>
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/4667
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Current BD xlator (block backend) has a few limitations such as
* Creation of directories not supported
* Supports only single brick
* Does not use extended attributes (and client gfid) like posix xlator
* Creation of special files (symbolic links, device nodes etc) not
supported
Basic limitation of not allowing directory creation is blocking
oVirt/VDSM to consume BD xlator as part of Gluster domain since VDSM
creates multi-level directories when GlusterFS is used as storage
backend for storing VM images.
To overcome these limitations a new BD xlator with following
improvements is suggested.
* New hybrid BD xlator that handles both regular files and block device
files
* The volume will have both POSIX and BD bricks. Regular files are
created on POSIX bricks, block devices are created on the BD brick (VG)
* BD xlator leverages exiting POSIX xlator for most POSIX calls and
hence sits above the POSIX xlator
* Block device file is differentiated from regular file by an extended
attribute
* The xattr 'user.glusterfs.bd' (BD_XATTR) plays a role in mapping a
posix file to Logical Volume (LV).
* When a client sends a request to set BD_XATTR on a posix file, a new
LV is created and mapped to posix file. So every block device will
have a representative file in POSIX brick with 'user.glusterfs.bd'
(BD_XATTR) set.
* Here after all operations on this file results in LV related
operations.
For example opening a file that has BD_XATTR set results in opening
the LV block device, reading results in reading the corresponding LV
block device.
When BD xlator gets request to set BD_XATTR via setxattr call, it
creates a LV and information about this LV is placed in the xattr of the
posix file. xattr "user.glusterfs.bd" used to identify that posix file
is mapped to BD.
Usage:
Server side:
[root@host1 ~]# gluster volume create bdvol host1:/storage/vg1_info?vg1 host2:/storage/vg2_info?vg2
It creates a distributed gluster volume 'bdvol' with Volume Group vg1
using posix brick /storage/vg1_info in host1 and Volume Group vg2 using
/storage/vg2_info in host2.
[root@host1 ~]# gluster volume start bdvol
Client side:
[root@node ~]# mount -t glusterfs host1:/bdvol /media
[root@node ~]# touch /media/posix
It creates regular posix file 'posix' in either host1:/vg1 or host2:/vg2 brick
[root@node ~]# mkdir /media/image
[root@node ~]# touch /media/image/lv1
It also creates regular posix file 'lv1' in either host1:/vg1 or
host2:/vg2 brick
[root@node ~]# setfattr -n "user.glusterfs.bd" -v "lv" /media/image/lv1
[root@node ~]#
Above setxattr results in creating a new LV in corresponding brick's VG
and it sets 'user.glusterfs.bd' with value 'lv:<default-extent-size'
[root@node ~]# truncate -s5G /media/image/lv1
It results in resizig LV 'lv1'to 5G
New BD xlator code is placed in xlators/storage/bd directory.
Also add volume-uuid to the VG so that same VG can't be used for other
bricks/volumes. After deleting a gluster volume, one has to manually
remove the associated tag using vgchange <vg-name> --deltag
<trusted.glusterfs.volume-id:<volume-id>>
Changes from previous version V5:
* Removed support for delayed deleting of LVs
Changes from previous version V4:
* Consolidated the patches
* Removed usage of BD_XATTR_SIZE and consolidated it in BD_XATTR.
Changes from previous version V3:
* Added support in FUSE to support full/linked clone
* Added support to merge snapshots and provide information about origin
* bd_map xlator removed
* iatt structure used in inode_ctx. iatt is cached and updated during
fsync/flush
* aio support
* Type and capabilities of volume are exported through getxattr
Changes from version 2:
* Used inode_context for caching BD size and to check if loc/fd is BD or
not.
* Added GlusterFS server offloaded copy and snapshot through setfattr
FOP. As part of this libgfapi is modified.
* BD xlator supports stripe
* During unlinking if a LV file is already opened, its added to delete
list and bd_del_thread tries to delete from this list when a last
reference to that file is closed.
Changes from previous version:
* gfid is used as name of LV
* ? is used to specify VG name for creating BD volume in volume
create, add-brick. gluster volume create volname host:/path?vg
* open-behind issue is fixed
* A replicate brick can be added dynamically and LVs from source brick
are replicated to destination brick
* A distribute brick can be added dynamically and rebalance operation
distributes existing LVs/files to the new brick
* Thin provisioning support added.
* bd_map xlator support retained
* setfattr -n user.glusterfs.bd -v "lv" creates a regular LV and
setfattr -n user.glusterfs.bd -v "thin" creates thin LV
* Capability and backend information added to gluster volume info (and
--xml) so
that management tools can exploit BD xlator.
* tracing support for bd xlator added
TODO:
* Add support to display snapshots for a given LV
* Display posix filename for list-origin instead of gfid
Change-Id: I00d32dfbab3b7c806e0841515c86c3aa519332f2
BUG: 1028672
Signed-off-by: M. Mohan Kumar <mohan@in.ibm.com>
Reviewed-on: http://review.gluster.org/4809
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
* When a writev call occurs, the client compresses the data before
sending it to server. On the server, compressed data is decompressed.
Similarly, when a readv call occurs, the server compresses the data
before sending it to client. On the client, the compressed data is
decompressed. Thus the amount of data sent over the wire is minimized.
* Compression/Decompression is done using Zlib library.
* During normal operation, this is the format of data sent over wire :
<compressed-data> + trailer(8)
The trailer contains the CRC32 checksum and length of original
uncompressed data. This is used for validation.
HOW TO USE
----------
Turning on compression xlator:
gluster volume set <vol_name> compress on
Configurable options:
gluster volume set <vol_name> compress.compression-level 8
gluster volume set <vol_name> compress.min-size 50
Change-Id: Ib7a66b6f1f70fe002b7c513588cdf75c69370805
BUG: 923540
Original-author : Venky Shankar <vshankar@redhat.com>
Signed-off-by: Venky Shankar <vshankar@redhat.com>
Signed-off-by: Prashanth Pai <nullpai@gmail.com>
Signed-off-by: Prashanth Pai <ppai@redhat.com>
Reviewed-on: http://review.gluster.org/3251
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
gettimeofday() returns the current wall clock time and timezone.
Using these functions in order to measure the passage of time
(how long an operation took) therefore seems like a no-brainer.
This time suffer's from some limitations:
a. They have a low resolution: “High-performance” timing by
definition, requires clock resolutions into the microseconds
or better.
b. They can jump forwards and backwards in time: Computer
clocks all tick at slightly different rates, which causes
the time to drift. Most systems have NTP enabled which
periodically adjusts the system clock to keep them in sync
with “actual” time. The adjustment can cause the clock to
suddenly jump forward (artificially inflating your timing
numbers) or jump backwards (causing your timing calculations
to go negative or hugely positive). In such cases timer
thread could go into an infinite loop.
From 'man gettimeofday':
----------
..
..
The time returned by gettimeofday() is affected by discontinuous
jumps in the system time (e.g., if the system administrator manually
changes the system time). If you need a monotonically increasing
clock, see clock_gettime(2).
..
..
----------
Rationale:
For calculating interval timing for Timer thread, all that’s
needed should be clock as a simple counter that increments
at a stable rate.
This is necessary to avoid the jumps which are caused by using
"wall time", this counter must be monotonic that can never
“tick” backwards, ever.
Change-Id: I701d31e71a85a73d21a6c5cd15583e7a5a645eeb
BUG: 1017993
Signed-off-by: Harshavardhana <harsha@harshavardhana.net>
Reviewed-on: http://review.gluster.org/6070
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
Currently gfapi.py only loads libraries by filename ending in ".so". On
an installed system without development packages, the <lib>.so filenames
are not available. ctypes.util.find_library() can be used to detect the
files dynamically.
In addition to this, also fixing some minor indention errors and package
the library into the Python site-packages path. Python applications and
libraries can now access libgfapi through 'from glusterfs import gfapi'.
Change-Id: I71e38dabd3ade5dcf24813bf2fc25cda91b571c6
BUG: 1005146
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: http://review.gluster.org/5835
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
make run-tests.sh "location independent" and replace 'make install'
with cpio in glusterfs.spec.in
Change-Id: I140473c7f558e1e0af93a863b79098ced516a76b
BUG: 764966
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/5986
Reviewed-by: Harshavardhana <harsha@harshavardhana.net>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
As of today regression tests are an in-house breed, by making
it a new package and distributing it ensures larger set of
people use it and contribute to it. This can also be used
by any consumer/user to build their own environment for glusterfs
regression testing which is today limited only to 'upstream'
'glusterfs' releases and build.gluster.org
Change-Id: I4f7e9fd1c49982dcf0d788ef6a83ffe895a956ac
BUG: 764966
Signed-off-by: Harshavardhana <harsha@harshavardhana.net>
Reviewed-on: http://review.gluster.org/5674
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
AM_PATH_XML2 needs check for its existence using
aclocal macros - to avoid problems like below when
libxml2 development libraries are not installed
--------------------------------------------------
... GlusterFS autogen ...
Generate gf-error-codes.h ...
`gf-error-codes.h' -> `libglusterfs/src/gf-error-codes.h'
Running aclocal...
configure.ac:524: warning: macro `AM_PATH_XML2' not found in library
Running autoheader...
Running libtoolize...
Running autoconf...
configure.ac:524: error: possibly undefined macro: AM_PATH_XML2
If this token and others are legitimate, please use m4_pattern_allow.
See the Autoconf documentation.
Running automake...
--------------------------------------------------
Change-Id: Ife463c34c45babc1c4c0ed6e8128b5c43419b9b9
BUG: 947226
Signed-off-by: Harshavardhana <harsha@harshavardhana.net>
Reviewed-on: http://review.gluster.org/5962
Reviewed-by: Kaushal M <kaushal@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
This is a translator to improve the performance of typical,
sequential directory reads (i.e., ls). readdir-ahead begins
preloading the contents of a directory on open and serves readdir
requests from the preloaded content. readdir-ahead is currently
implemented to only handle the single threaded directory read
case.
readdir-ahead is currently disabled by default. It can be enabled
with the following command:
gluster volume set <volname> readdir-ahead on
The following are results of a getdents test on a single brick
volume.
Test info:
- Single VM, gluster client/server.
- Volume mounted with native client using --gid-timeout=2.
- getdents on single directory with 100k 0-byte files.
Test results:
- !readdir-ahead
read 3120080 bytes from offset 0
3 MiB, 4348 ops, 0:00:07.00 (416.590 KiB/sec and 594.4737 ops/sec)
- readdir-ahead
read 3120080 bytes from offset 0
3 MiB, 4348 ops, 0:00:03.00 (820.116 KiB/sec and 1170.3043 ops/sec)
BUG: 980517
Change-Id: Ieceb9e1eb47d1d5b5af8da2bf03839537364653f
Signed-off-by: Brian Foster <bfoster@redhat.com>
Reviewed-on: http://review.gluster.org/4519
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
