glusterfs/configure.ac

1674 lines
58 KiB
Plaintext
Raw Normal View History

dnl Copyright (c) 2006-2016 Red Hat, Inc. <http://www.redhat.com>
dnl This file is part of GlusterFS.
2009-02-18 17:36:07 +05:30
dnl
dnl This file is licensed to you under your choice of the GNU Lesser
dnl General Public License, version 3 or any later version (LGPLv3 or
dnl later), or the GNU General Public License, version 2 (GPLv2), in all
dnl cases as published by the Free Software Foundation.
2009-02-18 17:36:07 +05:30
AC_INIT([glusterfs],
[m4_esyscmd([build-aux/pkg-version --version])],
[gluster-users@gluster.org],,[https://github.com/gluster/glusterfs.git])
AC_SUBST([PACKAGE_RELEASE],
[m4_esyscmd([build-aux/pkg-version --release])])
2009-02-18 17:36:07 +05:30
AM_INIT_AUTOMAKE(tar-pax)
# Removes warnings when using automake 1.14 around (...but option 'subdir-objects' is disabled )
#but libglusterfs fails to build with contrib (Then are not set up that way?)
#AM_INIT_AUTOMAKE([subdir-objects])
2009-02-18 17:36:07 +05:30
m4_ifdef([AM_SILENT_RULES], [AM_SILENT_RULES(yes)])
if make --help 2>&1 | grep -q no-print-directory; then
AM_MAKEFLAGS="$AM_MAKEFLAGS --no-print-directory";
fi
build: add site.h as a place to put environment-specific defines Most people consume Gluster in one of two ways: * From packages provided by their OS/distribution vendor * By building themselves from source For the first group it doesn't matter whether configuration is done in a configure script, via command-line options to that configure script, or in a header file. All of these end up as edits to some file under the packager's control, which is then run through their tools and process (e.g. rpmbuild) to create the packages that users will install. For the second group, convenience matters. Such users might not even have a script wrapped around the configure process, and editing one line in a header file is a lot easier than editing several in the configure script. This also prevents a messy profusion of configure options, dozens of which might need to be added to support a single such user's preferences. This comes back around as greater simplicity for packagers as well. This patch defines site.h as the header file for options and parameters that someone building the code for themselves might want to tweak. The project can ship one version to reflect the developers' guess at the best defaults for most users, and sophisticated users with unusual needs can override many options at once just by maintaining their own version of that file. Everybody wins. Further guidelines for how to determine whether an option should go in configure.ac or site.h are explained within site.h itself. Fixes #201 Change-Id: I5b8fb518d42450737423c4c1f43ebeb3130b4ff6 Signed-off-by: Jeff Darcy <jdarcy@fb.com> Reviewed-on: https://review.gluster.org/17206 Tested-by: Jeff Darcy <jeff@pl.atyp.us> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Amar Tumballi <amarts@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org>
2017-05-08 12:55:49 -04:00
AC_CONFIG_HEADERS([config.h site.h])
2009-02-18 17:36:07 +05:30
AC_CONFIG_FILES([Makefile
libglusterfs/Makefile
libglusterfs/src/Makefile
Adding Libgfdb to GlusterFS ************************************************************************* Libgfdb | ************************************************************************* Libgfdb provides abstract mechanism to record extra/rich metadata required for data maintenance, such as data tiering/classification. It provides consumer with API for recording and querying, keeping the consumer abstracted from the data store used beneath for storing data. It works in a plug-and-play model, where data stores can be plugged-in. Presently we have plugin for Sqlite3. In the future will provide recording and querying performance optimizer. In the current implementation the schema of metadata is fixed. Schema: ~~~~~~ GF_FILE_TB Table: ~~~~~~~~~~~~~~~~~ This table has one entry per file inode. It holds the metadata required to make decisions in data maintenance. GF_ID (Primary key) : File GFID (Universal Unique IDentifier in the namespace) W_SEC, W_MSEC : Write wind time in sec & micro-sec UW_SEC, UW_MSEC : Write un-wind time in sec & micro-sec W_READ_SEC, W_READ_MSEC : Read wind time in sec & micro-sec UW_READ_SEC, UW_READ_MSEC : Read un-wind time in sec & micro-sec WRITE_FREQ_CNTR INTEGER : Write Frequency Counter READ_FREQ_CNTR INTEGER : Read Frequency Counter GF_FLINK_TABLE: ~~~~~~~~~~~~~~ This table has all the hardlinks to a file inode. GF_ID : File GFID (Composite Primary Key)``| GF_PID : Parent Directory GFID (Composite Primary Key) |-> Primary Key FNAME : File Base Name (Composite Primary Key)__| FPATH : File Full Path (Its redundant for now, this will go) W_DEL_FLAG : This Flag is used for crash consistancy, when a link is unlinked. i.e Set to 1 during unlink wind and during unwind this record is deleted LINK_UPDATE : This Flag is used when a link is changed i.e rename. Set to 1 when rename wind and set to 0 in rename unwind Libgfdb API: ~~~~~~~~~~~ Refer libglusterfs/src/gfdb/gfdb_data_store.h Change-Id: I2e9fbab3878ce630a7f41221ef61017dc43db11f BUG: 1194753 Signed-off-by: Joseph Fernandes <josferna@redhat.com> Signed-off-by: Dan Lambright <dlambrig@redhat.com> Signed-off-by: Joseph Fernandes <josferna@redhat.com> Reviewed-on: http://review.gluster.org/9683 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2015-02-18 19:45:23 +05:30
libglusterfs/src/gfdb/Makefile
geo-replication/src/peer_gsec_create
geo-rep: mountbroker user management Non root geo-replication setup is now simplified. This patch provides cli for mountbroker user and options management To set Options, gluster system:: execute mountbroker opt <KEY> <VALUE> # for example, gluster system:: execute mountbroker opt mountbroker-root /var/mountbroker-root gluster system:: execute mountbroker opt geo-replication-log-group geogroup gluster system:: execute mountbroker opt rpc-auth-allow-insecure on To remove option, gluster system:: execute mountbroker optdel <KEY> # for example, gluster system:: execute mountbroker optdel geo-replication-log-group To add/edit user, gluster system:: execute mountbroker user <USERNAME> <VOLUMES> # for example gluster system:: execute mountbroker user geoaccount slavevol1,slavevol2 To remove user, gluster system:: execute mountbroker userdel <USERNAME> # for example gluster system:: execute mountbroker userdel geoaccount For info, gluster system:: execute mountbroker info gluster system:: execute mountbroker -j info For JSON output add -j after mountbroker, for example, gluster system:: execute mountbroker -j user geoaccount slavevol1,slavevol2 PS: Each peer prints its own JSON output, aggregator required from consumer side BUG: 1136312 Change-Id: Ie52210c0bcc91ac2ffd3ba58988222ffca62b47f Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/9398 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: darshan n <dnarayan@redhat.com> Reviewed-by: Kotresh HR <khiremat@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2015-01-06 18:20:45 +05:30
geo-replication/src/peer_mountbroker
geo-replication/src/peer_mountbroker.py
geo-replication/src/peer_georep-sshkey.py
feature/glusterfind: A tool to find incremental changes Documentation is available in patch: http://review.gluster.org/#/c/9800/ A tool which helps to get list of modified files or list of all files in GlusterFS Volume using Changelog or find command. Usage ===== glusterfind --help Create: ------- glusterfind create --help The tool creates status file $GLUSTERD_WORKDIR/SESSION/VOLUME/status and records current timestamp to initiate the session. This timestamp will be used as start time for next runs. As part of create also generates ssh key and distributes to all peers. and enables build.pgfid and changelog using volume set command. Pre: ---- glusterfind pre --help This command is used to generate the list of files modified after session creation time or after last run. To get list of all files/dirs in Volume, run pre command with `--full` argument. The tool gets all nodes details using gluster volume info and runs node agent for each brick in respective nodes via ssh command. Once these node agents generate the output file, tool copies to local using scp. Merges all the output files to generate the final output file. Post: ----- glusterfind post --help After consuming the list, this sub command is called to update the session time based on pre command status file. List: ----- glusterfind list --help To view all the sessions Delete: ------- glusterfind delete --help Delete session. Known Issues ------------ 1. Deleted files will not get listed, since we can't convert GFID to Path if file/dir is deleted. 2. Only new name will get listed if Renamed. 3. All hardlinks will get listed. Change-Id: I82991feb0aea85cb6ec035fddbf80a2b276e86b0 BUG: 1193893 Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/9682 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Venky Shankar <vshankar@redhat.com> Reviewed-by: Prashanth Pai <ppai@redhat.com> Reviewed-by: Kotresh HR <khiremat@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2015-02-18 19:07:23 +05:30
extras/peer_add_secret_pub
geo-replication/syncdaemon/conf.py
geo-replication/gsyncd.conf
snapshot/eventsapi: Integrate snapshot events with eventsapi 1. EVENT_SNAPSHOT_CREATED : snapshot_name=snap1 volume_name=test_vol snapshot_uuid=26dd6c52-6021-40b1-a507-001a80401d70 2. EVENT_SNAPSHOT_CREATE_FAILED : snapshot_name=snap1 volume_name=test_vol error=Snapshot snap1 already exists 3. EVENT_SNAPSHOT_ACTIVATED : snapshot_name=snap1 snapshot_uuid=26dd6c52-6021-40b1-a507-001a80401d70 4. EVENT_SNAPSHOT_ACTIVATE_FAILED: snapshot_name=snap1 error=Snapshot snap1 is already activated. 5. EVENT_SNAPSHOT_DEACTIVATED : snapshot_name=snap1 snapshot_uuid=26dd6c52-6021-40b1-a507-001a80401d70 6. EVENT_SNAPSHOT_DEACTIVATE_FAILED : snapshot_name=snap3 error=Snapshot (snap3) does not exist. 7. EVENT_SNAPSHOT_SOFT_LIMIT_REACHED : volume_name=test_vol volume_id=2ace2616-5591-4b9b-be2a-38592dda5758 8. EVENT_SNAPSHOT_HARD_LIMIT_REACHED : volume_name=test_vol volume_id=2ace2616-5591-4b9b-be2a-38592dda5758 9. EVENT_SNAPSHOT_RESTORED : snapshot_name=snap1 volume_name=test_vol snapshot_uuid=3a840ec5-08da-4f2b-850d-1d5539a5d14d 10. EVENT_SNAPSHOT_RESTORE_FAILED : snapshot_name=snap10 error=Snapshot (snap10) does not exist 11. EVENT_SNAPSHOT_DELETED : snapshot_name=snap1 snapshot_uuid=d9ff3d4f-f579-4345-a4da-4f9353f0950c 12. EVENT_SNAPSHOT_DELETE_FAILED : snapshot_name=snap2 error=Snapshot (snap2) does not exist 13. EVENT_SNAPSHOT_CLONED : clone_uuid=93ba9f06-cb9c-4ace-aa52-2616e7f31022 snapshot_name=snap1 clone_name=clone2 14. EVENT_SNAPSHOT_CLONE_FAILED : snapshot_name=snap1 clone_name=clone2 error=Volume with name:clone2 already exists 15. EVENT_SNAPSHOT_CONFIG_UPDATED : auto-delete=enable config_type=system_config config_type=volume_config hard_limit=100 16. EVENT_SNAPSHOT_CONFIG_UPDATE_FAILED : error=Invalid snap-max-soft-limit 110. Expected range 1 - 100 17. EVENT_SNAPSHOT_SCHEDULER_INITIALISED : status=Success 18. EVENT_SNAPSHOT_SCHEDULER_INIT_FAILED 19. EVENT_SNAPSHOT_SCHEDULER_ENABLED : status=Successfuly Enabled 20. EVENT_SNAPSHOT_SCHEDULER_ENABLE_FAILED : error=Snapshot scheduler is already enabled. 21. EVENT_SNAPSHOT_SCHEDULER_SCHEDULE_ADDED : status=Successfuly added job job1 22. EVENT_SNAPSHOT_SCHEDULER_SCHEDULE_ADD_FAILED : status=Failed to add job job1 error=The job already exists. 23. EVENT_SNAPSHOT_SCHEDULER_SCHEDULE_EDITED : status=Successfuly edited job job1 24. EVENT_SNAPSHOT_SCHEDULER_SCHEDULE_EDIT_FAILED : status=Failed to edit job job2 error=The job cannot be found. 25. EVENT_SNAPSHOT_SCHEDULER_SCHEDULE_DELETED : status=Successfuly deleted job job1 26. EVENT_SNAPSHOT_SCHEDULER_SCHEDULE_DELETE_FAILED : status=Failed to delete job job1 error=The job cannot be found. 27. EVENT_SNAPSHOT_SCHEDULER_DISABLED : status=Successfuly Disabled 28. EVENT_SNAPSHOT_SCHEDULER_DISABLE_FAILED : error=Snapshot scheduler is already disabled. Change-Id: I3479cc3fb7af3c76ded67cf289f99547d0a55d21 BUG: 1370567 Signed-off-by: Avra Sengupta <asengupt@redhat.com> Reviewed-on: http://review.gluster.org/15329 Tested-by: Aravinda VK <avishwan@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Rajesh Joseph <rjoseph@redhat.com>
2016-08-26 14:05:07 +05:30
extras/snap_scheduler/conf.py
glusterfsd/Makefile
glusterfsd/src/Makefile
rpc/Makefile
rpc/rpc-lib/Makefile
rpc/rpc-lib/src/Makefile
rpc/rpc-transport/Makefile
rpc/rpc-transport/socket/Makefile
rpc/rpc-transport/socket/src/Makefile
rpc/rpc-transport/rdma/Makefile
rpc/rpc-transport/rdma/src/Makefile
rpc/xdr/Makefile
rpc/xdr/src/Makefile
build: libgfxdr.so calls GF_FREE(), needs to link with -lglusterfs The previous change to remove the xdrgen script exposed (or created) a recursive build dependency: libglusterfs needs the generated headers, and libgfxdr should be linked with libglusterfs for GF_FREE/__gf_free. (Much grumbling about libglusterfs being the kitchen sink of gluster elided. This would not be necessary if there were two or more libs, a gluster "runtime" library with common gluster code shared by the xlators and daemons, and a utility library with things like the rbtree, memory allocation, and whatnot.) So. Link at build time or link at runtime? For truth-and-beauty, link with libglusterfs.so at build time. Without truth-and-beauty, don't link with libglusterfs and rely on the other things that link with libglusterfs to provide resolution of __gf_free(). Truth-and-beauty it is. But how to generate the headers first, then build libglusterfs, then come back and build libgfxdr? Autotools is a maze of twisty passages, all different. Things that work with gnu make on linux don't work with the BSD make. Finally I hit on this solution. Add a shadow directory where make only generates the headers, then build libglusterfs using the generated headers, and finally build libgfxdr and link with libglusterfs. See original BZ 1330604 change http://review.gluster.org/14085 Change-Id: Iede8a30e3103176cb8f0b054885f30fcb352492b BUG: 1429696 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com> Reviewed-on: https://review.gluster.org/16873 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Niels de Vos <ndevos@redhat.com>
2017-03-08 14:44:50 -05:00
rpc/xdr/gen/Makefile
xlators/Makefile
xlators/meta/Makefile
xlators/meta/src/Makefile
xlators/mount/Makefile
xlators/mount/fuse/Makefile
xlators/mount/fuse/src/Makefile
xlators/mount/fuse/utils/mount.glusterfs
xlators/mount/fuse/utils/mount_glusterfs
xlators/mount/fuse/utils/Makefile
xlators/storage/Makefile
xlators/storage/posix/Makefile
xlators/storage/posix/src/Makefile
bd: posix/multi-brick support to BD xlator Current BD xlator (block backend) has a few limitations such as * Creation of directories not supported * Supports only single brick * Does not use extended attributes (and client gfid) like posix xlator * Creation of special files (symbolic links, device nodes etc) not supported Basic limitation of not allowing directory creation is blocking oVirt/VDSM to consume BD xlator as part of Gluster domain since VDSM creates multi-level directories when GlusterFS is used as storage backend for storing VM images. To overcome these limitations a new BD xlator with following improvements is suggested. * New hybrid BD xlator that handles both regular files and block device files * The volume will have both POSIX and BD bricks. Regular files are created on POSIX bricks, block devices are created on the BD brick (VG) * BD xlator leverages exiting POSIX xlator for most POSIX calls and hence sits above the POSIX xlator * Block device file is differentiated from regular file by an extended attribute * The xattr 'user.glusterfs.bd' (BD_XATTR) plays a role in mapping a posix file to Logical Volume (LV). * When a client sends a request to set BD_XATTR on a posix file, a new LV is created and mapped to posix file. So every block device will have a representative file in POSIX brick with 'user.glusterfs.bd' (BD_XATTR) set. * Here after all operations on this file results in LV related operations. For example opening a file that has BD_XATTR set results in opening the LV block device, reading results in reading the corresponding LV block device. When BD xlator gets request to set BD_XATTR via setxattr call, it creates a LV and information about this LV is placed in the xattr of the posix file. xattr "user.glusterfs.bd" used to identify that posix file is mapped to BD. Usage: Server side: [root@host1 ~]# gluster volume create bdvol host1:/storage/vg1_info?vg1 host2:/storage/vg2_info?vg2 It creates a distributed gluster volume 'bdvol' with Volume Group vg1 using posix brick /storage/vg1_info in host1 and Volume Group vg2 using /storage/vg2_info in host2. [root@host1 ~]# gluster volume start bdvol Client side: [root@node ~]# mount -t glusterfs host1:/bdvol /media [root@node ~]# touch /media/posix It creates regular posix file 'posix' in either host1:/vg1 or host2:/vg2 brick [root@node ~]# mkdir /media/image [root@node ~]# touch /media/image/lv1 It also creates regular posix file 'lv1' in either host1:/vg1 or host2:/vg2 brick [root@node ~]# setfattr -n "user.glusterfs.bd" -v "lv" /media/image/lv1 [root@node ~]# Above setxattr results in creating a new LV in corresponding brick's VG and it sets 'user.glusterfs.bd' with value 'lv:<default-extent-size' [root@node ~]# truncate -s5G /media/image/lv1 It results in resizig LV 'lv1'to 5G New BD xlator code is placed in xlators/storage/bd directory. Also add volume-uuid to the VG so that same VG can't be used for other bricks/volumes. After deleting a gluster volume, one has to manually remove the associated tag using vgchange <vg-name> --deltag <trusted.glusterfs.volume-id:<volume-id>> Changes from previous version V5: * Removed support for delayed deleting of LVs Changes from previous version V4: * Consolidated the patches * Removed usage of BD_XATTR_SIZE and consolidated it in BD_XATTR. Changes from previous version V3: * Added support in FUSE to support full/linked clone * Added support to merge snapshots and provide information about origin * bd_map xlator removed * iatt structure used in inode_ctx. iatt is cached and updated during fsync/flush * aio support * Type and capabilities of volume are exported through getxattr Changes from version 2: * Used inode_context for caching BD size and to check if loc/fd is BD or not. * Added GlusterFS server offloaded copy and snapshot through setfattr FOP. As part of this libgfapi is modified. * BD xlator supports stripe * During unlinking if a LV file is already opened, its added to delete list and bd_del_thread tries to delete from this list when a last reference to that file is closed. Changes from previous version: * gfid is used as name of LV * ? is used to specify VG name for creating BD volume in volume create, add-brick. gluster volume create volname host:/path?vg * open-behind issue is fixed * A replicate brick can be added dynamically and LVs from source brick are replicated to destination brick * A distribute brick can be added dynamically and rebalance operation distributes existing LVs/files to the new brick * Thin provisioning support added. * bd_map xlator support retained * setfattr -n user.glusterfs.bd -v "lv" creates a regular LV and setfattr -n user.glusterfs.bd -v "thin" creates thin LV * Capability and backend information added to gluster volume info (and --xml) so that management tools can exploit BD xlator. * tracing support for bd xlator added TODO: * Add support to display snapshots for a given LV * Display posix filename for list-origin instead of gfid Change-Id: I00d32dfbab3b7c806e0841515c86c3aa519332f2 BUG: 1028672 Signed-off-by: M. Mohan Kumar <mohan@in.ibm.com> Reviewed-on: http://review.gluster.org/4809 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Anand Avati <avati@redhat.com>
2013-11-13 22:44:42 +05:30
xlators/storage/bd/Makefile
xlators/storage/bd/src/Makefile
xlators/cluster/Makefile
xlators/cluster/afr/Makefile
xlators/cluster/afr/src/Makefile
xlators/cluster/stripe/Makefile
xlators/cluster/stripe/src/Makefile
xlators/cluster/dht/Makefile
xlators/cluster/dht/src/Makefile
xlators/cluster/ec/Makefile
xlators/cluster/ec/src/Makefile
xlators/performance/Makefile
xlators/performance/write-behind/Makefile
xlators/performance/write-behind/src/Makefile
xlators/performance/read-ahead/Makefile
xlators/performance/read-ahead/src/Makefile
xlators/performance/readdir-ahead/Makefile
xlators/performance/readdir-ahead/src/Makefile
xlators/performance/io-threads/Makefile
xlators/performance/io-threads/src/Makefile
xlators/performance/io-cache/Makefile
xlators/performance/io-cache/src/Makefile
xlators/performance/symlink-cache/Makefile
xlators/performance/symlink-cache/src/Makefile
xlators/performance/quick-read/Makefile
xlators/performance/quick-read/src/Makefile
xlators/performance/open-behind/Makefile
xlators/performance/open-behind/src/Makefile
xlators/performance/md-cache/Makefile
xlators/performance/md-cache/src/Makefile
xlators/performance/decompounder/Makefile
xlators/performance/decompounder/src/Makefile
xlators/performance/nl-cache/Makefile
xlators/performance/nl-cache/src/Makefile
xlators/debug/Makefile
xlators/debug/sink/Makefile
xlators/debug/sink/src/Makefile
xlators/debug/trace/Makefile
xlators/debug/trace/src/Makefile
xlators/debug/error-gen/Makefile
xlators/debug/error-gen/src/Makefile
debug/delay-gen: Implement delay-generation feature Background: I was working on a customer issue where the disks were responding some times after seconds. It was becoming very difficult to recreate the issues in our labs, so had to come up with this feature. Requirements: We need an xlator which can delay x% of ops for y micro seconds. We should be able to enable delays for specific fops. This feature is modeled after error-gen. Most of the logic is borrowed from that xlator. This is a minimum implementation of the feature which satisfied the requirements I had. May be in future with more requirements and understanding of the problem further we can improve upon this implementation. Here are the commands and what they do: Enable delay-gen: (This is similar to how err-gen is enabled on the brick side) - gluster volume set <volname> delay-gen posix Set the percentage of fops that need to be delayed - gluster volume set <volname> delay-gen.delay-percentage 50 Default is 10% Set the delay in micro seconds - gluster volume set <volname> delay-gen.delay-duration 500000 Default is 100000 Set comma separated fops to be delayed - gluster v set r2 delay-gen.enable read,write Default is all fops. Fixes #257 Change-Id: Ib547bd39cc024c9cdb63754d21e3aa62fc9d6473 Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> Reviewed-on: https://review.gluster.org/17591 Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
2017-06-20 15:24:33 +05:30
xlators/debug/delay-gen/Makefile
xlators/debug/delay-gen/src/Makefile
xlators/debug/io-stats/Makefile
xlators/debug/io-stats/src/Makefile
xlators/protocol/Makefile
xlators/protocol/auth/Makefile
xlators/protocol/auth/addr/Makefile
xlators/protocol/auth/addr/src/Makefile
xlators/protocol/auth/login/Makefile
xlators/protocol/auth/login/src/Makefile
xlators/protocol/client/Makefile
xlators/protocol/client/src/Makefile
xlators/protocol/server/Makefile
xlators/protocol/server/src/Makefile
xlators/features/Makefile
xlators/features/arbiter/Makefile
xlators/features/arbiter/src/Makefile
xlators/experimental/fdl/Makefile
xlators/experimental/fdl/src/Makefile
xlators/features/changelog/Makefile
xlators/features/changelog/src/Makefile
xlators/features/changelog/lib/Makefile
xlators/features/changelog/lib/src/Makefile
xlators/features/changetimerecorder/Makefile
xlators/features/changetimerecorder/src/Makefile
xlators/features/glupy/Makefile
xlators/features/glupy/examples/Makefile
xlators/features/glupy/src/Makefile
xlators/features/glupy/src/setup.py
xlators/features/glupy/src/__init__.py
xlators/features/glupy/src/glupy/Makefile
xlators/features/locks/Makefile
xlators/features/locks/src/Makefile
xlators/features/quota/Makefile
xlators/features/quota/src/Makefile
xlators/features/marker/Makefile
xlators/features/marker/src/Makefile
xlators/features/selinux/Makefile
xlators/features/selinux/src/Makefile
dentry fop serializer: added new server side xlator for dentry fop serialization Problems addressed by this xlator : [1]. To prevent race between parallel mkdir,mkdir and lookup etc. Fops like mkdir/create, lookup, rename, unlink, link that happen on a particular dentry must be serialized to ensure atomicity. Another possible case can be a fresh lookup to find existance of a path whose gfid is not set yet. Further, storage/posix employs a ctime based heuristic 'is_fresh_file' (interval time is less than 1 second of current time) to check fresh-ness of file. With serialization of these two fops (lookup & mkdir), we eliminate the race altogether. [2]. Staleness of dentries This causes exponential increase in traversal time for any inode in the subtree of the directory pointed by stale dentry. Cause : Stale dentry is created because of following two operations: a. dentry creation due to inode_link, done during operations like lookup, mkdir, create, mknod, symlink, create and b. dentry unlinking due to various operations like rmdir, rename, unlink. The reason is __inode_link uses __is_dentry_cyclic, which explores all possible path to avoid cyclic link formation during inode linkage. __is_dentry_cyclic explores stale-dentry(ies) and its all ancestors which is increases traversing time exponentially. Implementation : To acheive this all fops on dentry must take entry locks before they proceed, once they have acquired locks, they perform the fop and then release the lock. Some documentation from email conversation: [1] http://www.gluster.org/pipermail/gluster-devel/2015-December/047314.html [2] http://www.gluster.org/pipermail/gluster-devel/2015-August/046428.html With this patch, the feature is optional, enable it by running: `gluster volume set $volname features.sdfs enable` Also the feature is tested for a month without issues in the experiemental branch for all the regression. Change-Id: I6e80ba3cabfa6facd5dda63bd482b9bf18b6b79b Fixes: #397 BUG: 1304962 Signed-off-by: Sakshi Bansal <sabansal@redhat.com> Signed-off-by: Amar Tumballi <amarts@redhat.com> Signed-off-by: Sunny Kumar <sunkumar@redhat.com>
2018-01-22 14:38:17 +05:30
xlators/features/sdfs/Makefile
xlators/features/sdfs/src/Makefile
xlators/features/read-only/Makefile
xlators/features/read-only/src/Makefile
xlators/features/compress/Makefile
xlators/features/compress/src/Makefile
xlators/features/quiesce/Makefile
xlators/features/quiesce/src/Makefile
xlators/features/barrier/Makefile
xlators/features/barrier/src/Makefile
xlators/features/index/Makefile
xlators/features/index/src/Makefile
xlators/features/gfid-access/Makefile
xlators/features/gfid-access/src/Makefile
xlators/features/trash/Makefile
xlators/features/trash/src/Makefile
xlators/features/snapview-server/Makefile
xlators/features/snapview-server/src/Makefile
xlators/features/snapview-client/Makefile
xlators/features/snapview-client/src/Makefile
xlators/features/upcall/Makefile
xlators/features/upcall/src/Makefile
xlators/features/shard/Makefile
xlators/features/shard/src/Makefile
xlators/features/bit-rot/Makefile
xlators/features/bit-rot/src/Makefile
xlators/features/bit-rot/src/stub/Makefile
features/bit-rot: Implementation of bit-rot xlator This is the "Signer" -- responsible for signing files with their checksums upon last file descriptor close (last release()). The event notification facility provided by the changelog xlator is made use of. Moreover, checksums are as of now SHA256 hash of the object data and is the only available hash at this point of time. Therefore, there is no special "what hash to use" type check, although it's does not take much to add various hashing algorithms to sign objects with. Signatures are stored in extended attributes of the objects along with the the type of hashing used to calculate the signature. This makes thing future proof when other hash types are added. The signature infrastructure is provided by bitrot stub: a little piece of code that sits over the POSIX xlator providing interfaces to "get or set" objects signature and it's staleness. Since objects are signed upon receiving release() notification, pre-existing data which are "never" modified would never be signed. To counter this, an initial crawler thread is spawned The crawler scans the entire brick for objects that are unsigned or "missed" signing due to the server going offline (node reboots, crashes, etc..) and triggers an explicit sign. This would also sign objects when bit-rot is enabled for a volume and/or after upgrade. Change-Id: I1d9a98bee6cad1c39c35c53c8fb0fc4bad2bf67b BUG: 1170075 Original-Author: Raghavendra Bhat <raghavendra@redhat.com> Signed-off-by: Venky Shankar <vshankar@redhat.com> Reviewed-on: http://review.gluster.org/9711 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2015-02-15 15:05:19 +05:30
xlators/features/bit-rot/src/bitd/Makefile
xlators/features/leases/Makefile
xlators/features/leases/src/Makefile
xlators/playground/Makefile
xlators/playground/template/Makefile
xlators/playground/template/src/Makefile
xlators/encryption/Makefile
xlators/encryption/rot-13/Makefile
xlators/encryption/rot-13/src/Makefile
Transparent data encryption and metadata authentication .. in the systems with non-trusted server This new functionality can be useful in various cloud technologies. It is implemented via a special encryption/crypt translator,which works on the client side and performs encryption and authentication; 1. Class of supported algorithms The crypt translator can support any atomic symmetric block cipher algorithms (which require to pad plain/cipher text before performing encryption/decryption transform (see glossary in atom.c for definitions). In particular, it can support algorithms with the EOF issue (which require to pad the end of file by extra-data). Crypt translator performs translations user -> (offset, size) -> (aligned-offset, padded-size) ->server (and backward), and resolves individual FOPs (write(), truncate(), etc) to read-modify-write sequences. A volume can contain files encrypted by different algorithms of the mentioned class. To change some option value just reconfigure the volume. Currently only one algorithm is supported: AES_XTS. Example of algorithms, which can not be supported by the crypt translator: 1. Asymmetric block cipher algorithms, which inflate data, e.g. RSA; 2. Symmetric block cipher algorithms with inline MACs for data authentication. 2. Implementation notes. a) Atomic algorithms Since any process in a stackable file system manipulates with local data (which can be obsoleted by local data of another process), any atomic cipher algorithm without proper support can lead to non-POSIX behavior. To resolve the "collisions" we introduce locks: before performing FOP->read(), FOP->write(), etc. the process should first lock the file. b) Algorithms with EOF issue Such algorithms require to pad the end of file with some extra-data. Without proper support this will result in losing information about real file size. Keeping a track of real file size is a responsibility of the crypt translator. A special extended attribute with the name "trusted.glusterfs.crypt.att.size" is used for this purpose. All files contained in bricks of encrypted volume do have "padded" sizes. 3. Non-trusted servers and Metadata authentication We assume that server, where user's data is stored on is non-trusted. It means that the server can be subjected to various attacks directed to reveal user's encrypted personal data. We provide protection against such attacks. Every encrypted file has specific private attributes (cipher algorithm id, atom size, etc), which are packed to a string (so-called "format string") and stored as a special extended attribute with the name "trusted.glusterfs.crypt.att.cfmt". We protect the string from tampering. This protection is mandatory, hardcoded and is always on. Without such protection various attacks (based on extending the scope of per-file secret keys) are possible. Our authentication method has been developed in tight collaboration with Red Hat security team and is implemented as "metadata loader of version 1" (see file metadata.c). This method is NIST-compliant and is based on checking 8-byte per-hardlink MACs created(updated) by FOP->create(), FOP->link(), FOP->unlink(), FOP->rename() by the following unique entities: . file (hardlink) name; . verified file's object id (gfid). Every time, before manipulating with a file, we check it's MACs at FOP->open() time. Some FOPs don't require a file to be opened (e.g. FOP->truncate()). In such cases the crypt translator opens the file mandatory. 4. Generating keys Unique per-file keys are derived by NIST-compliant methods from the a) parent key; b) unique verified object-id of the file (gfid); Per-volume master key, provided by user at mount time is in the root of this "tree of keys". Those keys are used to: 1) encrypt/decrypt file data; 2) encrypt/decrypt file metadata; 3) create per-file and per-link MACs for metadata authentication. 5. Instructions Getting started with crypt translator Example: 1) Create a volume "myvol" and enable encryption: # gluster volume create myvol pepelac:/vols/xvol # gluster volume set myvol encryption on 2) Set location (absolute pathname) of your master key: # gluster volume set myvol encryption.master-key /home/me/mykey 3) Set other options to override default options, if needed. Start the volume. 4) On the client side make sure that the file /home/me/mykey exists and contains proper per-volume master key (that is 256-bit AES key). This key has to be in hex form, i.e. should be represented by 64 symbols from the set {'0', ..., '9', 'a', ..., 'f'}. The key should start at the beginning of the file. All symbols at offsets >= 64 are ignored. 5) Mount the volume "myvol" on the client side: # glusterfs --volfile-server=pepelac --volfile-id=myvol /mnt After successful mount the file which contains master key may be removed. NOTE: Keeping the master key between mount sessions is in user's competence. ********************************************************************** WARNING! Losing the master key will make content of all regular files inaccessible. Mount with improper master key allows to access content of directories: file names are not encrypted. ********************************************************************** 6. Options of crypt translator 1) "master-key": specifies location (absolute pathname) of the file which contains per-volume master key. There is no default location for master key. 2) "data-key-size": specifies size of per-file key for data encryption Possible values: . "256" default value . "512" 3) "block-size": specifies atom size. Possible values: . "512" . "1024" . "2048" . "4096" default value; 7. Test cases Any workload, which involves the following file operations: ->create(); ->open(); ->readv(); ->writev(); ->truncate(); ->ftruncate(); ->link(); ->unlink(); ->rename(); ->readdirp(). 8. TODOs: 1) Currently size of IOs issued by crypt translator is restricted by block_size (4K by default). We can use larger IOs to improve performance. Change-Id: I2601fe95c5c4dc5b22308a53d0cbdc071d5e5cee BUG: 1030058 Signed-off-by: Edward Shishkin <edward@redhat.com> Signed-off-by: Anand Avati <avati@redhat.com> Reviewed-on: http://review.gluster.org/4667 Tested-by: Gluster Build System <jenkins@build.gluster.com>
2013-03-13 21:56:46 +01:00
xlators/encryption/crypt/Makefile
xlators/encryption/crypt/src/Makefile
xlators/system/Makefile
xlators/system/posix-acl/Makefile
xlators/system/posix-acl/src/Makefile
xlators/nfs/Makefile
xlators/nfs/server/Makefile
xlators/nfs/server/src/Makefile
xlators/mgmt/Makefile
xlators/mgmt/glusterd/Makefile
xlators/mgmt/glusterd/src/Makefile
xlators/experimental/Makefile
xlators/experimental/jbr-client/Makefile
xlators/experimental/jbr-client/src/Makefile
xlators/experimental/jbr-server/Makefile
xlators/experimental/jbr-server/src/Makefile
xlators/experimental/dht2/Makefile
xlators/experimental/dht2/dht2-client/Makefile
xlators/experimental/dht2/dht2-client/src/Makefile
xlators/experimental/dht2/dht2-server/Makefile
xlators/experimental/dht2/dht2-server/src/Makefile
xlators/experimental/posix2/Makefile
xlators/experimental/posix2/ds/Makefile
xlators/experimental/posix2/ds/src/Makefile
xlators/experimental/posix2/mds/Makefile
xlators/experimental/posix2/mds/src/Makefile
xlators/experimental/posix2/common/Makefile
xlators/experimental/posix2/common/src/Makefile
cli/Makefile
cli/src/Makefile
doc/Makefile
extras/Makefile
extras/glusterd.vol
extras/cliutils/Makefile
extras/init.d/Makefile
extras/init.d/glusterd.plist
extras/init.d/glusterd-Debian
extras/init.d/glusterd-Redhat
extras/init.d/glusterd-FreeBSD
extras/init.d/glusterd-SuSE
extras/init.d/glustereventsd-Debian
extras/init.d/glustereventsd-Redhat
extras/init.d/glustereventsd-FreeBSD
extras/systemd/Makefile
extras/systemd/glusterd.service
eventsapi: Gluster Eventing Feature implementation [Depends on http://review.gluster.org/14627] Design is available in `glusterfs-specs`, A change from the design is support of webhook instead of Websockets as discussed in the design http://review.gluster.org/13115 Since Websocket support depends on REST APIs, I will add Websocket support once REST APIs patch gets merged Usage: Run following command to start/stop Eventsapi server in all Peers, which will collect the notifications from any Gluster daemon and emits to configured client. gluster-eventsapi start|stop|restart|reload Status of running services can be checked using, gluster-eventsapi status Events listener is a HTTP(S) server which listens to events emited by the Gluster. Create a HTTP Server to listen on POST and register that URL using, gluster-eventsapi webhook-add <URL> [--bearer-token <TOKEN>] For example, if HTTP Server running in `http://192.168.122.188:9000` then add that URL using, gluster-eventsapi webhook-add http://192.168.122.188:9000 If it expects a Token then specify it using `--bearer-token` or `-t` We can also test Webhook if all peer nodes can send message or not using, gluster-eventsapi webhook-test <URL> [--bearer-token <TOKEN>] Configurations can be viewed/updated using, gluster-eventsapi config-get [--name] gluster-eventsapi config-set <NAME> <VALUE> gluster-eventsapi config-reset <NAME|all> If any one peer node was down during config-set/reset or webhook modifications, Run sync command from good node when a peer node comes back. Automatic update is not yet implemented. gluster-eventsapi sync Basic Events Client(HTTP Server) is included with the code, Start running the client with required port and start listening to the events. /usr/share/glusterfs/scripts/eventsdash.py --port 8080 Default port is 9000, if no port is specified, once it started running then configure gluster-eventsapi to send events to that client. Eventsapi Client can be outside of the Cluster, it can be run event on Windows. But only requirement is the client URL should be accessible by all peer nodes.(Or ngrok(https://ngrok.com) like tools can be used) Events implemented with this patch, - Volume Create - Volume Start - Volume Stop - Volume Delete - Peer Attach - Peer Detach It is easy to add/support more events, since it touches Gluster cmd code and to avoid merge conflicts I will add support for more events once this patch merges. BUG: 1334044 Change-Id: I316827ac9dd1443454df7deffe4f54835f7f6a08 Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/14248 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
2016-05-05 18:34:41 +05:30
extras/systemd/glustereventsd.service
extras/systemd/glusterfssharedstorage.service
extras/run-gluster.tmpfiles
extras/benchmarking/Makefile
extras/hook-scripts/Makefile
extras/ocf/Makefile
extras/ocf/glusterd
extras/ocf/volume
extras/LinuxRPM/Makefile
extras/geo-rep/Makefile
extras/geo-rep/schedule_georep.py
extras/firewalld/Makefile
features/quota: Add the quota config xattr to newly added brick Issue: Quota directory limit configuration is stored in the xattrs. When a new brick is added these 'limit-set' xattrs have to be created to the directory in the new brick. This is done by the dht directory healing when the directory is created in the new brick. Since 'root' directory is already created DHT doesn't heal the limit-set xattr root. Solution: When the add-brick command is issued run the below hook script to heal the 'limit-set' xattr. The hook script does the following only if limit is configured on root. 1. Create an auxiliary mount. 2. getxattr 'limit-set' on the root 3. setxattr the same value on the root But this script needs the volume to be started to make the auxiliary mount. To handle the case when the add-brick is issued when the volume was stopped, symlink is created by the 'master' script to the corresponding location and these two are by default disabled. So, a 'master' script is added in the add-brick/pre. When add-brick command is issued, it enables one of the scripts mentioned above based on the condition, if volume is started - enable add-brick/post script else - enable start/post script After the actual script completes its job, it disables itself. Note: The enabling and disabling of the script is based on the glusterd's logic, that it only runs the scripts which starts its name with 'S'. So, Enable - symlink the file to 'S'* Disable - unlink the symlink. Change-Id: I2d3947a4d686c54417ec95f530af3bdd3444f4e2 BUG: 969461 Signed-off-by: Varun Shastry <vshastry@redhat.com> Reviewed-on: http://review.gluster.org/6104 Reviewed-by: Brian Foster <bfoster@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Anand Avati <avati@redhat.com>
2013-10-15 17:25:51 +05:30
extras/hook-scripts/add-brick/Makefile
extras/hook-scripts/add-brick/pre/Makefile
extras/hook-scripts/add-brick/post/Makefile
extras/hook-scripts/create/Makefile
extras/hook-scripts/create/post/Makefile
extras/hook-scripts/delete/Makefile
extras/hook-scripts/delete/pre/Makefile
extras/hook-scripts/start/Makefile
extras/hook-scripts/start/post/Makefile
extras/hook-scripts/set/Makefile
extras/hook-scripts/set/post/Makefile
extras/hook-scripts/stop/Makefile
extras/hook-scripts/stop/pre/Makefile
extras/hook-scripts/reset/Makefile
extras/hook-scripts/reset/post/Makefile
extras/hook-scripts/reset/pre/Makefile
extras/snap_scheduler/Makefile
eventsapi: Gluster Eventing Feature implementation [Depends on http://review.gluster.org/14627] Design is available in `glusterfs-specs`, A change from the design is support of webhook instead of Websockets as discussed in the design http://review.gluster.org/13115 Since Websocket support depends on REST APIs, I will add Websocket support once REST APIs patch gets merged Usage: Run following command to start/stop Eventsapi server in all Peers, which will collect the notifications from any Gluster daemon and emits to configured client. gluster-eventsapi start|stop|restart|reload Status of running services can be checked using, gluster-eventsapi status Events listener is a HTTP(S) server which listens to events emited by the Gluster. Create a HTTP Server to listen on POST and register that URL using, gluster-eventsapi webhook-add <URL> [--bearer-token <TOKEN>] For example, if HTTP Server running in `http://192.168.122.188:9000` then add that URL using, gluster-eventsapi webhook-add http://192.168.122.188:9000 If it expects a Token then specify it using `--bearer-token` or `-t` We can also test Webhook if all peer nodes can send message or not using, gluster-eventsapi webhook-test <URL> [--bearer-token <TOKEN>] Configurations can be viewed/updated using, gluster-eventsapi config-get [--name] gluster-eventsapi config-set <NAME> <VALUE> gluster-eventsapi config-reset <NAME|all> If any one peer node was down during config-set/reset or webhook modifications, Run sync command from good node when a peer node comes back. Automatic update is not yet implemented. gluster-eventsapi sync Basic Events Client(HTTP Server) is included with the code, Start running the client with required port and start listening to the events. /usr/share/glusterfs/scripts/eventsdash.py --port 8080 Default port is 9000, if no port is specified, once it started running then configure gluster-eventsapi to send events to that client. Eventsapi Client can be outside of the Cluster, it can be run event on Windows. But only requirement is the client URL should be accessible by all peer nodes.(Or ngrok(https://ngrok.com) like tools can be used) Events implemented with this patch, - Volume Create - Volume Start - Volume Stop - Volume Delete - Peer Attach - Peer Detach It is easy to add/support more events, since it touches Gluster cmd code and to avoid merge conflicts I will add support for more events once this patch merges. BUG: 1334044 Change-Id: I316827ac9dd1443454df7deffe4f54835f7f6a08 Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/14248 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
2016-05-05 18:34:41 +05:30
events/Makefile
events/src/Makefile
events/src/eventsapiconf.py
events/tools/Makefile
contrib/fuse-util/Makefile
contrib/umountd/Makefile
contrib/uuid/uuid_types.h
glusterfs-api.pc
features/changelog: changelog translator This is the initial version of the Changelog Translator. What is it ----------- Goal is to capture changes performed on a GlusterFS volume. The translator needs to be loaded on the server (bricks) and captures changes in a plain text file inside a configured directory path (controlled by "changelog-dir", should be somewhere in <export>/.glusterfs/changelog by default). Changes are classified into 3 types: - Data: : TYPE-I - Metadata : TYPE-II - Entry : TYPE-III Changelog file is rolled over after a certain time interval (defauls to 60 seconds) after which a changelog is started. The thing to be noted here is that for a time interval (time slice) multiple changes for an inode are recorded only once (ie. say for 100+ writes on an inode that happens within the time slice has only a single corresponding entry in the changelog file). That way we do not bloat up the changelog and also save lots of writes. Changelog Format ----------------- TYPE-I and TYPE-II changes have the gfid on the entity on which the operation happened. TYPE-III being a entry op requires the parent gfid and the basename. Changelog format has been kept to a minimal and it's upto the consumers to do the heavy loading of figuring out deletes, renames etc.. A single changelog file records all three types of changes, with each change starting with an identifier ("D": DATA, "M": METADATA and "E": ENTRY). Option is provided for the encoding type (See TUNABLES). Consumers ---------- The only consumer as of today would be geo-replication, although backup utilities, self-heal, bit-rot detection could be possible consumers in the future. CLI ---- By default, change-logging is disabled (the translator is present in the server graph but does nothing). When enabled (via cli) each brick starts to log the changes. There are a set of tunable that can be used to change the translators behaviour: - enable/disable changelog (disabled by default) gluster volume set <volume> changelog {on|off} - set the logging directory (<brick>/.glusterfs/changelogs is the default) gluster volume set <volume> changelog-dir /path/to/dir - select encoding type (binary (default) or ascii) gluster volume set <volume> encoding {binary|ascii} - change the rollover time for the logs (60 secs by default) gluster volume set <volume> rollover-time <secs> - when secs > 0, changelog file is not open()'d with O_SYNC flag - and fsync is trigerred periodically every <secs> seconds. gluster volume set <volume> fsync-interval <secs> features/changelog: changelog consumer library (libgfchangelog) A shared library is provided for the consumer of the changelogs for easy acess via APIs. Application can link against this library and request for changelog updates. Conversion of binary logs to human-readable ascii format is also taken care by the library which keeps a copy of the changelog in application provided working directory. Change-Id: I75575fb7f1c53d2bec3dba1a329ea7bb3c628497 BUG: 847839 Original Author: Venky Shankar <vshankar@redhat.com> Signed-off-by: Avra Sengupta <asengupt@redhat.com> Reviewed-on: http://review.gluster.org/5127 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2013-06-04 14:20:58 +05:30
libgfchangelog.pc
Adding Libgfdb to GlusterFS ************************************************************************* Libgfdb | ************************************************************************* Libgfdb provides abstract mechanism to record extra/rich metadata required for data maintenance, such as data tiering/classification. It provides consumer with API for recording and querying, keeping the consumer abstracted from the data store used beneath for storing data. It works in a plug-and-play model, where data stores can be plugged-in. Presently we have plugin for Sqlite3. In the future will provide recording and querying performance optimizer. In the current implementation the schema of metadata is fixed. Schema: ~~~~~~ GF_FILE_TB Table: ~~~~~~~~~~~~~~~~~ This table has one entry per file inode. It holds the metadata required to make decisions in data maintenance. GF_ID (Primary key) : File GFID (Universal Unique IDentifier in the namespace) W_SEC, W_MSEC : Write wind time in sec & micro-sec UW_SEC, UW_MSEC : Write un-wind time in sec & micro-sec W_READ_SEC, W_READ_MSEC : Read wind time in sec & micro-sec UW_READ_SEC, UW_READ_MSEC : Read un-wind time in sec & micro-sec WRITE_FREQ_CNTR INTEGER : Write Frequency Counter READ_FREQ_CNTR INTEGER : Read Frequency Counter GF_FLINK_TABLE: ~~~~~~~~~~~~~~ This table has all the hardlinks to a file inode. GF_ID : File GFID (Composite Primary Key)``| GF_PID : Parent Directory GFID (Composite Primary Key) |-> Primary Key FNAME : File Base Name (Composite Primary Key)__| FPATH : File Full Path (Its redundant for now, this will go) W_DEL_FLAG : This Flag is used for crash consistancy, when a link is unlinked. i.e Set to 1 during unlink wind and during unwind this record is deleted LINK_UPDATE : This Flag is used when a link is changed i.e rename. Set to 1 when rename wind and set to 0 in rename unwind Libgfdb API: ~~~~~~~~~~~ Refer libglusterfs/src/gfdb/gfdb_data_store.h Change-Id: I2e9fbab3878ce630a7f41221ef61017dc43db11f BUG: 1194753 Signed-off-by: Joseph Fernandes <josferna@redhat.com> Signed-off-by: Dan Lambright <dlambrig@redhat.com> Signed-off-by: Joseph Fernandes <josferna@redhat.com> Reviewed-on: http://review.gluster.org/9683 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2015-02-18 19:45:23 +05:30
libgfdb.pc
api/Makefile
api/src/Makefile
api/examples/Makefile
geo-replication/Makefile
geo-replication/src/Makefile
geo-replication/syncdaemon/Makefile
tools/Makefile
tools/gfind_missing_files/Makefile
heal/Makefile
heal/src/Makefile
feature/glusterfind: A tool to find incremental changes Documentation is available in patch: http://review.gluster.org/#/c/9800/ A tool which helps to get list of modified files or list of all files in GlusterFS Volume using Changelog or find command. Usage ===== glusterfind --help Create: ------- glusterfind create --help The tool creates status file $GLUSTERD_WORKDIR/SESSION/VOLUME/status and records current timestamp to initiate the session. This timestamp will be used as start time for next runs. As part of create also generates ssh key and distributes to all peers. and enables build.pgfid and changelog using volume set command. Pre: ---- glusterfind pre --help This command is used to generate the list of files modified after session creation time or after last run. To get list of all files/dirs in Volume, run pre command with `--full` argument. The tool gets all nodes details using gluster volume info and runs node agent for each brick in respective nodes via ssh command. Once these node agents generate the output file, tool copies to local using scp. Merges all the output files to generate the final output file. Post: ----- glusterfind post --help After consuming the list, this sub command is called to update the session time based on pre command status file. List: ----- glusterfind list --help To view all the sessions Delete: ------- glusterfind delete --help Delete session. Known Issues ------------ 1. Deleted files will not get listed, since we can't convert GFID to Path if file/dir is deleted. 2. Only new name will get listed if Renamed. 3. All hardlinks will get listed. Change-Id: I82991feb0aea85cb6ec035fddbf80a2b276e86b0 BUG: 1193893 Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/9682 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Venky Shankar <vshankar@redhat.com> Reviewed-by: Prashanth Pai <ppai@redhat.com> Reviewed-by: Kotresh HR <khiremat@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2015-02-18 19:07:23 +05:30
glusterfs.spec
tools/glusterfind/src/tool.conf
tools/glusterfind/glusterfind
tools/glusterfind/Makefile
tools/glusterfind/src/Makefile
tools/setgfid2path/Makefile
tools/setgfid2path/src/Makefile])
2009-02-18 17:36:07 +05:30
AC_CANONICAL_HOST
AC_PROG_CC
AC_DISABLE_STATIC
2009-02-18 17:36:07 +05:30
AC_PROG_LIBTOOL
AC_SUBST([shrext_cmds])
2009-02-18 17:36:07 +05:30
AC_CHECK_PROG([RPCGEN], [rpcgen], [yes], [no])
if test "x$RPCGEN" = "xno"; then
AC_MSG_ERROR([`rpcgen` not found, glusterfs needs `rpcgen` exiting..])
fi
# Initialize CFLAGS before usage
AC_ARG_ENABLE([debug],
AC_HELP_STRING([--enable-debug],
[Enable debug build options.]))
if test "x$enable_debug" = "xyes"; then
BUILD_DEBUG=yes
CFLAGS="${CFLAGS} -g -rdynamic -O0 -DDEBUG"
else
BUILD_DEBUG=no
CFLAGS="${CFLAGS} -g -rdynamic"
Halo Replication feature for AFR translator Summary: Halo Geo-replication is a feature which allows Gluster or NFS clients to write locally to their region (as defined by a latency "halo" or threshold if you like), and have their writes asynchronously propagate from their origin to the rest of the cluster. Clients can also write synchronously to the cluster simply by specifying a halo-latency which is very large (e.g. 10seconds) which will include all bricks. In other words, it allows clients to decide at mount time if they desire synchronous or asynchronous IO into a cluster and the cluster can support both of these modes to any number of clients simultaneously. There are a few new volume options due to this feature: halo-shd-latency: The threshold below which self-heal daemons will consider children (bricks) connected. halo-nfsd-latency: The threshold below which NFS daemons will consider children (bricks) connected. halo-latency: The threshold below which all other clients will consider children (bricks) connected. halo-min-replicas: The minimum number of replicas which are to be enforced regardless of latency specified in the above 3 options. If the number of children falls below this threshold the next best (chosen by latency) shall be swapped in. New FUSE mount options: halo-latency & halo-min-replicas: As descripted above. This feature combined with multi-threaded SHD support (D1271745) results in some pretty cool geo-replication possibilities. Operational Notes: - Global consistency is gaurenteed for synchronous clients, this is provided by the existing entry-locking mechanism. - Asynchronous clients on the other hand and merely consistent to their region. Writes & deletes will be protected via entry-locks as usual preventing concurrent writes into files which are undergoing replication. Read operations on the other hand should never block. - Writes are allowed from _any_ region and propagated from the origin to all other regions. The take away from this is care should be taken to ensure multiple writers do not write the same files resulting in a gfid split-brain which will require resolution via split-brain policies (majority, mtime & size). Recommended method for preventing this is using the nfs-auth feature to define which region for each share has RW permissions, tiers not in the origin region should have RO perms. TODO: - Synchronous clients (including the SHD) should choose clients from their own region as preferred sources for reads. Most of the plumbing is in place for this via the child_latency array. - Better GFID split brain handling & better dent type split brain handling (i.e. create a trash can and move the offending files into it). - Tagging in addition to latency as a means of defining which children you wish to synchronously write to Test Plan: - The usual suspects, clang, gcc w/ address sanitizer & valgrind - Prove tests Reviewers: jackl, dph, cjh, meyering Reviewed By: meyering Subscribers: ethanr Differential Revision: https://phabricator.fb.com/D1272053 Tasks: 4117827 Change-Id: I694a9ab429722da538da171ec528406e77b5e6d1 BUG: 1428061 Signed-off-by: Kevin Vigor <kvigor@fb.com> Reviewed-on: http://review.gluster.org/16099 Reviewed-on: https://review.gluster.org/16177 Tested-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
2017-03-21 08:23:25 -07:00
fi
AC_ARG_WITH([libtirpc],
[AC_HELP_STRING([--without-libtirpc], [Use legacy glibc RPC.])],
[], [with_libtirpc=yes])
AC_ARG_WITH([ipv6-default],
AC_HELP_STRING([--with-ipv6-default], [Set IPv6 as default.]),
[with_ipv6_default=$with_libtirpc], [with_ipv6_default=no])
if test "x$ac_cv_file__etc_redhat_release" = "xyes"; then
if rpm -qa centos-release | grep centos; then
if rpm -q centos-release | grep "release-6"; then
with_ipv6_default=no
fi
fi
fi
AC_ARG_ENABLE([privport_tracking],
AC_HELP_STRING([--disable-privport_tracking],
[Disable internal tracking of privileged ports.]))
TRACK_PRIVPORTS="yes"
if test x"$enable_privport_tracking" = x"no"; then
TRACK_PRIVPORTS="no"
AC_DEFINE(GF_DISABLE_PRIVPORT_TRACKING, 1,
[Disable internal tracking of privileged ports.])
fi
case $host_os in
darwin*)
if ! test "`/usr/bin/sw_vers | grep ProductVersion: | cut -f 2 | cut -d. -f2`" -ge 7; then
AC_MSG_ERROR([You need at least OS X 10.7 (Lion) to build Glusterfs])
fi
# OSX version lesser than 9 has llvm/clang optimization issues which leads to various segfaults
if test "`/usr/bin/sw_vers | grep ProductVersion: | cut -f 2 | cut -d. -f2`" -lt 9; then
CFLAGS="${CFLAGS} -g -O0 -DDEBUG"
fi
;;
esac
xlator: do not call dlclose() when debugging Valgrind can not show the symbols if a .so after calling dlclose(). The unhelpful ??? in the output gets resolved properly with this change: ==25170== 344 bytes in 1 blocks are definitely lost in loss record 233 of 324 ==25170== at 0x4C29975: calloc (vg_replace_malloc.c:711) ==25170== by 0x52C7C0B: __gf_calloc (mem-pool.c:117) ==25170== by 0x12B0638A: ??? ==25170== by 0x528FCE6: __xlator_init (xlator.c:472) ==25170== by 0x528FE16: xlator_init (xlator.c:498) ==25170== by 0x52DA8D6: glusterfs_graph_init (graph.c:321) ==25170== by 0x52DB587: glusterfs_graph_activate (graph.c:695) ==25170== by 0x5046407: glfs_process_volfp (glfs-mgmt.c:79) ==25170== by 0x5043B9E: glfs_volumes_init (glfs.c:281) ==25170== by 0x5044FEC: glfs_init_common (glfs.c:986) ==25170== by 0x50451A7: glfs_init@@GFAPI_3.4.0 (glfs.c:1031) By not calling dlclose(), the dynamically loaded .so is still available upon program exit, and Valgrind is able to resolve the symbols. This will add an additional leak, so dlclose() is called for normal builds, but skipped when configuring with "./configure --enable-valgrind" or passing the "run-with-valgrind" xlator option. URL: http://valgrind.org/docs/manual/faq.html#faq.unhelpful Change-Id: I2044e21b1b8fcce32ad1a817fdd795218f967731 BUG: 1425623 Signed-off-by: Niels de Vos <ndevos@redhat.com> Reviewed-on: https://review.gluster.org/16809 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Samikshan Bairagya <samikshan@gmail.com> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
2017-02-27 22:37:00 -08:00
# --enable-valgrind prevents calling dlclose(), this leaks memory
AC_ARG_ENABLE([valgrind],
AC_HELP_STRING([--enable-valgrind],
[Enable valgrind for resource leak debugging.]))
if test "x$enable_valgrind" = "xyes"; then
AC_DEFINE(RUN_WITH_VALGRIND, 1, [define if all processes should run under valgrind])
fi
AC_ARG_WITH([previous-options],
[AS_HELP_STRING([--with-previous-options],
[read config.status for configure options])
],
[ if test -r ./config.status && \
args=$(grep 'ac_cs_config=' config.status | \
sed -e 's/.*"\(.*\)".*/\1/'| sed -e "s/'//g") ; then
echo "###"
echo "### Rerunning as '$0 $args'"
echo "###"
exec $0 $args
fi
])
AC_ARG_WITH(pkgconfigdir,
[ --with-pkgconfigdir=DIR pkgconfig file in DIR @<:@LIBDIR/pkgconfig@:>@],
[pkgconfigdir=$withval],
[pkgconfigdir='${libdir}/pkgconfig'])
AC_SUBST(pkgconfigdir)
AC_ARG_WITH(mountutildir,
[ --with-mountutildir=DIR mount helper utility in DIR @<:@/sbin@:>@],
[mountutildir=$withval],
[mountutildir='/sbin'])
AC_SUBST(mountutildir)
AC_ARG_WITH(systemddir,
[ --with-systemddir=DIR systemd service files in DIR @<:@PREFIX/lib/systemd/system@:>@],
[systemddir=$withval],
[systemddir='${prefix}/lib/systemd/system'])
AC_SUBST(systemddir)
AM_CONDITIONAL([USE_SYSTEMD], test [ -d '/usr/lib/systemd/system' ])
AC_ARG_WITH(initdir,
[ --with-initdir=DIR init.d scripts in DIR @<:@/etc/init.d@:>@],
[initdir=$withval],
[initdir='/etc/init.d'])
AC_SUBST(initdir)
AC_ARG_WITH(launchddir,
[ --with-launchddir=DIR launchd services in DIR @<:@/Library/LaunchDaemons@:>@],
[launchddir=$withval],
[launchddir='/Library/LaunchDaemons'])
AC_SUBST(launchddir)
AC_ARG_WITH(tmpfilesdir,
AC_HELP_STRING([--with-tmpfilesdir=DIR],
[tmpfiles config in DIR, disabled by default]),
[tmpfilesdir=$withval],
[tmpfilesdir=''])
AC_SUBST(tmpfilesdir)
Add OCF compliant resource agents for glusterd and volumes These resource agents plug glusterd into Open Cluster Framework (OCF) compliant cluster resource managers, like Pacemaker. The glusterd RA is fairly trivial; it simply manages the glusterd daemon like any upstart or systemd job would, except that Pacemaker can do it in a cluster-aware fashion. The volume RA is a bit more involved; It starts a volume and monitors individual brick's daemons in a cluster aware fashion, recovering bricks when their processes fail. Note that this does NOT imply people would deploy GlusterFS servers in pairs, or anything of that nature. Pacemaker has the ability to deploy cluster resources as clones, meaning glusterd and volumes would be configured as follows in a Pacemaker cluster: primitive p_glusterd ocf:glusterfs:glusterd \ op monitor interval="30" primitive p_volume_demo ocf:glusterfs:volume \ params volname="demo" \ op monitor interval="10" clone cl_glusterd p_glusterd \ meta interleave="true" clone cl_volume_demo p_volume_demo \ meta interleave="true" ordered="true" colocation c_volume_on_glusterd inf: cl_volume_demo cl_glusterd order o_glusterd_before_volume 0: cl_glusterd cl_volume_demo The cluster status then looks as follows (in a 4-node cluster; note the configuration above could be applied, unchanged, to a cluster of any number of nodes): ============ Last updated: Fri Mar 30 10:54:50 2012 Last change: Thu Mar 29 17:20:17 2012 via crmd on gluster02.h Stack: openais Current DC: gluster03.h - partition with quorum Version: 1.1.6-3.el6-a02c0f19a00c1eb2527ad38f146ebc0834814558 4 Nodes configured, 4 expected votes 8 Resources configured. ============ Online: [ gluster02.h gluster03.h gluster04.h gluster01.h ] Clone Set: cl_glusterd [p_glusterd] Started: [ gluster02.h gluster03.h gluster04.h gluster01.h ] Clone Set: cl_volume_demo [p_volume_demo] Started: [ gluster01.h gluster02.h gluster03.h gluster04.h ] This is also a way of providing automatic glusterd and brick recovery in systems where neither upstart nor systemd are available. Change-Id: Ied46657bdfd2dd72dc97cf41b0eb7adcecacd18f BUG: 869559 Signed-off-by: Florian Haas <florian@hastexo.com> Reviewed-on: http://review.gluster.org/3043 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Anand Avati <avati@redhat.com>
2012-02-20 16:25:43 +01:00
AC_ARG_WITH([ocf],
[AS_HELP_STRING([--without-ocf], [build OCF-compliant cluster resource agents])],
Add OCF compliant resource agents for glusterd and volumes These resource agents plug glusterd into Open Cluster Framework (OCF) compliant cluster resource managers, like Pacemaker. The glusterd RA is fairly trivial; it simply manages the glusterd daemon like any upstart or systemd job would, except that Pacemaker can do it in a cluster-aware fashion. The volume RA is a bit more involved; It starts a volume and monitors individual brick's daemons in a cluster aware fashion, recovering bricks when their processes fail. Note that this does NOT imply people would deploy GlusterFS servers in pairs, or anything of that nature. Pacemaker has the ability to deploy cluster resources as clones, meaning glusterd and volumes would be configured as follows in a Pacemaker cluster: primitive p_glusterd ocf:glusterfs:glusterd \ op monitor interval="30" primitive p_volume_demo ocf:glusterfs:volume \ params volname="demo" \ op monitor interval="10" clone cl_glusterd p_glusterd \ meta interleave="true" clone cl_volume_demo p_volume_demo \ meta interleave="true" ordered="true" colocation c_volume_on_glusterd inf: cl_volume_demo cl_glusterd order o_glusterd_before_volume 0: cl_glusterd cl_volume_demo The cluster status then looks as follows (in a 4-node cluster; note the configuration above could be applied, unchanged, to a cluster of any number of nodes): ============ Last updated: Fri Mar 30 10:54:50 2012 Last change: Thu Mar 29 17:20:17 2012 via crmd on gluster02.h Stack: openais Current DC: gluster03.h - partition with quorum Version: 1.1.6-3.el6-a02c0f19a00c1eb2527ad38f146ebc0834814558 4 Nodes configured, 4 expected votes 8 Resources configured. ============ Online: [ gluster02.h gluster03.h gluster04.h gluster01.h ] Clone Set: cl_glusterd [p_glusterd] Started: [ gluster02.h gluster03.h gluster04.h gluster01.h ] Clone Set: cl_volume_demo [p_volume_demo] Started: [ gluster01.h gluster02.h gluster03.h gluster04.h ] This is also a way of providing automatic glusterd and brick recovery in systems where neither upstart nor systemd are available. Change-Id: Ied46657bdfd2dd72dc97cf41b0eb7adcecacd18f BUG: 869559 Signed-off-by: Florian Haas <florian@hastexo.com> Reviewed-on: http://review.gluster.org/3043 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Anand Avati <avati@redhat.com>
2012-02-20 16:25:43 +01:00
,
[OCF_SUBDIR='ocf'],
)
AC_SUBST(OCF_SUBDIR)
Add OCF compliant resource agents for glusterd and volumes These resource agents plug glusterd into Open Cluster Framework (OCF) compliant cluster resource managers, like Pacemaker. The glusterd RA is fairly trivial; it simply manages the glusterd daemon like any upstart or systemd job would, except that Pacemaker can do it in a cluster-aware fashion. The volume RA is a bit more involved; It starts a volume and monitors individual brick's daemons in a cluster aware fashion, recovering bricks when their processes fail. Note that this does NOT imply people would deploy GlusterFS servers in pairs, or anything of that nature. Pacemaker has the ability to deploy cluster resources as clones, meaning glusterd and volumes would be configured as follows in a Pacemaker cluster: primitive p_glusterd ocf:glusterfs:glusterd \ op monitor interval="30" primitive p_volume_demo ocf:glusterfs:volume \ params volname="demo" \ op monitor interval="10" clone cl_glusterd p_glusterd \ meta interleave="true" clone cl_volume_demo p_volume_demo \ meta interleave="true" ordered="true" colocation c_volume_on_glusterd inf: cl_volume_demo cl_glusterd order o_glusterd_before_volume 0: cl_glusterd cl_volume_demo The cluster status then looks as follows (in a 4-node cluster; note the configuration above could be applied, unchanged, to a cluster of any number of nodes): ============ Last updated: Fri Mar 30 10:54:50 2012 Last change: Thu Mar 29 17:20:17 2012 via crmd on gluster02.h Stack: openais Current DC: gluster03.h - partition with quorum Version: 1.1.6-3.el6-a02c0f19a00c1eb2527ad38f146ebc0834814558 4 Nodes configured, 4 expected votes 8 Resources configured. ============ Online: [ gluster02.h gluster03.h gluster04.h gluster01.h ] Clone Set: cl_glusterd [p_glusterd] Started: [ gluster02.h gluster03.h gluster04.h gluster01.h ] Clone Set: cl_volume_demo [p_volume_demo] Started: [ gluster01.h gluster02.h gluster03.h gluster04.h ] This is also a way of providing automatic glusterd and brick recovery in systems where neither upstart nor systemd are available. Change-Id: Ied46657bdfd2dd72dc97cf41b0eb7adcecacd18f BUG: 869559 Signed-off-by: Florian Haas <florian@hastexo.com> Reviewed-on: http://review.gluster.org/3043 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Anand Avati <avati@redhat.com>
2012-02-20 16:25:43 +01:00
2009-02-18 17:36:07 +05:30
# LEX needs a check
AC_PROG_LEX
if test "x${LEX}" != "xflex" -a "x${FLEX}" != "xlex"; then
AC_MSG_ERROR([Flex or lex required to build glusterfs.])
fi
dnl
dnl Word sizes...
dnl
AC_CHECK_SIZEOF(short)
AC_CHECK_SIZEOF(int)
AC_CHECK_SIZEOF(long)
AC_CHECK_SIZEOF(long long)
SIZEOF_SHORT=$ac_cv_sizeof_short
SIZEOF_INT=$ac_cv_sizeof_int
SIZEOF_LONG=$ac_cv_sizeof_long
SIZEOF_LONG_LONG=$ac_cv_sizeof_long_long
AC_SUBST(SIZEOF_SHORT)
AC_SUBST(SIZEOF_INT)
AC_SUBST(SIZEOF_LONG)
AC_SUBST(SIZEOF_LONG_LONG)
2009-02-18 17:36:07 +05:30
# YACC needs a check
AC_PROG_YACC
if test "x${YACC}" = "xbyacc" -o "x${YACC}" = "xyacc" -o "x${YACC}" = "x"; then
AC_MSG_ERROR([GNU Bison required to build glusterfs.])
fi
AC_CHECK_TOOL([LD],[ld])
Replace GPLV3 MD5 with OpenSSL MD5 Ric asked me to look at replacing the GPL licensed MD5 code with something better, i.e. perhaps faster, and with a less restrictive license, etc. So I took a couple hour holiday from working on wrapping up the client_t and did this. OpenSSL (nee SSLeay) is released under the OpenSSL license, a BSD/MIT style license. OpenSSL (libcrypto.so) is used on Linux, OS X and *BSD, Open Solaris, etc. IOW it's universally available on the platforms we care about. It's written by Eric Young (eay), now at EMC/RSA, and I can say from experience that the OpenSSL implementation of MD5 (at least) is every bit as fast as RSA's proprietary implementation (primarily because the implementations are very, very similar.) The last time I surveyed MD5 implementations I found they're all pretty much the same speed. I changed the APIs (and ABIs) for the strong and weak checksums. Strictly speaking I didn't need to do that. They're only called on short strings of data, i.e. pathnames, so using int32_t and uint32_t is ostensibly okay. My change is arguably a better, more general API for this sort of thing. It's also what bit me when gerrit/jenkins validation failed due to glusterfs segv-ing. (I didn't pay close enough attention to the implementation of the weak checksum. But it forced me to learn what gerrit/jenkins are doing and going forward I can do better testing before submitting to gerrit.) Now resubmitting with a BZ Change-Id: I545fade1604e74fc68399894550229bd57a5e0df BUG: 807718 Signed-off-by: Kaleb KEITHLEY <kkeithle@redhat.com> Reviewed-on: http://review.gluster.com/3019 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
2012-03-27 11:14:23 -04:00
AC_CHECK_LIB([crypto], [MD5], , AC_MSG_ERROR([OpenSSL crypto library is required to build glusterfs]))
2009-02-18 17:36:07 +05:30
AC_CHECK_LIB([pthread], [pthread_mutex_init], , AC_MSG_ERROR([Posix threads library is required to build glusterfs]))
Replace GPLV3 MD5 with OpenSSL MD5 Ric asked me to look at replacing the GPL licensed MD5 code with something better, i.e. perhaps faster, and with a less restrictive license, etc. So I took a couple hour holiday from working on wrapping up the client_t and did this. OpenSSL (nee SSLeay) is released under the OpenSSL license, a BSD/MIT style license. OpenSSL (libcrypto.so) is used on Linux, OS X and *BSD, Open Solaris, etc. IOW it's universally available on the platforms we care about. It's written by Eric Young (eay), now at EMC/RSA, and I can say from experience that the OpenSSL implementation of MD5 (at least) is every bit as fast as RSA's proprietary implementation (primarily because the implementations are very, very similar.) The last time I surveyed MD5 implementations I found they're all pretty much the same speed. I changed the APIs (and ABIs) for the strong and weak checksums. Strictly speaking I didn't need to do that. They're only called on short strings of data, i.e. pathnames, so using int32_t and uint32_t is ostensibly okay. My change is arguably a better, more general API for this sort of thing. It's also what bit me when gerrit/jenkins validation failed due to glusterfs segv-ing. (I didn't pay close enough attention to the implementation of the weak checksum. But it forced me to learn what gerrit/jenkins are doing and going forward I can do better testing before submitting to gerrit.) Now resubmitting with a BZ Change-Id: I545fade1604e74fc68399894550229bd57a5e0df BUG: 807718 Signed-off-by: Kaleb KEITHLEY <kkeithle@redhat.com> Reviewed-on: http://review.gluster.com/3019 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
2012-03-27 11:14:23 -04:00
2009-02-18 17:36:07 +05:30
AC_CHECK_FUNC([dlopen], [has_dlopen=yes], AC_CHECK_LIB([dl], [dlopen], , AC_MSG_ERROR([Dynamic linking library required to build glusterfs])))
AC_CHECK_LIB([readline], [rl_do_undo], [RL_UNDO="yes"], [RL_UNDO="no"])
2009-02-18 17:36:07 +05:30
AC_CHECK_LIB([intl], [gettext])
2009-02-18 17:36:07 +05:30
AC_CHECK_HEADERS([sys/xattr.h])
AC_CHECK_HEADERS([sys/ioctl.h], AC_DEFINE(HAVE_IOCTL_IN_SYS_IOCTL_H, 1, [have sys/ioctl.h]))
2009-02-18 17:36:07 +05:30
AC_CHECK_HEADERS([sys/extattr.h])
AC_CHECK_HEADERS([openssl/dh.h])
AC_CHECK_HEADERS([openssl/ecdh.h])
libglusterfs/rot-buffs: rotational buffers This patch introduces rotational buffers aiming at the classic multiple producer and multiple consumer problem. A fixed set of buffer list is allocated during initialization, where each list consist of a list of buffers. Each buffer is an iovec pointing to a memory region of fixed allocation size. Multiple producers write data to these buffers. A buffer list starts with a single buffer (iovec) and allocates more when required (although this can be preallocatd in multiples of k). rot-buffs allow multiple producers to write data parallely with a bit of extra cost of taking locks. Therefore, it's much suited for large writes. Multiple producers are allowed to write in the buffer parallely by "reserving" write space for selected number of bytes and returning pointer to the start of the reserved area. The write size is selected by the producer before it starts the write (which is often known). Therefore, the write itself need not be serialized -- just the space reservation needs to be done safely. The other part is when a consumer kicks in to consume what has been produced. At this point, a buffer list switch is performed. The "current" buffer list pointer is safely pointed to the next available buffer list. New writes are now directed to the just switched buffer list (the old buffer list is now considered out of rotation). Note that the old buffer still may have producers in progress (pending writes), so the consumer has to wait till the writers are drained. Currently this is the slow path for producers (write completion) and needs to be improved. Currently, there is special handling for cases where the number of consumers match (or exceed) the number of producers, which could result in writer starvation. In this scenario, when a consumers requests a buffer list for consumption, a check is performed for writer starvation and consumption is denied until at least another buffer list is ready of the producer for writes, i.e., one (or more) consumer(s) completed, thereby putting the buffer list back in rotation. [ NOTE: I've not performance tested this producer-consumer model yet. It's being used in changelog for event notification. The list of buffers (iovecs) are directly passed to RPC layer. ] Change-Id: I88d235522b05ab82509aba861374a2312bff57f2 BUG: 1170075 Signed-off-by: Venky Shankar <vshankar@redhat.com> Reviewed-on: http://review.gluster.org/9706 Tested-by: Vijay Bellur <vbellur@redhat.com>
2015-02-03 19:03:30 +05:30
dnl Math library
AC_CHECK_LIB([m], [pow], [MATH_LIB='-lm'], [MATH_LIB=''])
libglusterfs/rot-buffs: rotational buffers This patch introduces rotational buffers aiming at the classic multiple producer and multiple consumer problem. A fixed set of buffer list is allocated during initialization, where each list consist of a list of buffers. Each buffer is an iovec pointing to a memory region of fixed allocation size. Multiple producers write data to these buffers. A buffer list starts with a single buffer (iovec) and allocates more when required (although this can be preallocatd in multiples of k). rot-buffs allow multiple producers to write data parallely with a bit of extra cost of taking locks. Therefore, it's much suited for large writes. Multiple producers are allowed to write in the buffer parallely by "reserving" write space for selected number of bytes and returning pointer to the start of the reserved area. The write size is selected by the producer before it starts the write (which is often known). Therefore, the write itself need not be serialized -- just the space reservation needs to be done safely. The other part is when a consumer kicks in to consume what has been produced. At this point, a buffer list switch is performed. The "current" buffer list pointer is safely pointed to the next available buffer list. New writes are now directed to the just switched buffer list (the old buffer list is now considered out of rotation). Note that the old buffer still may have producers in progress (pending writes), so the consumer has to wait till the writers are drained. Currently this is the slow path for producers (write completion) and needs to be improved. Currently, there is special handling for cases where the number of consumers match (or exceed) the number of producers, which could result in writer starvation. In this scenario, when a consumers requests a buffer list for consumption, a check is performed for writer starvation and consumption is denied until at least another buffer list is ready of the producer for writes, i.e., one (or more) consumer(s) completed, thereby putting the buffer list back in rotation. [ NOTE: I've not performance tested this producer-consumer model yet. It's being used in changelog for event notification. The list of buffers (iovecs) are directly passed to RPC layer. ] Change-Id: I88d235522b05ab82509aba861374a2312bff57f2 BUG: 1170075 Signed-off-by: Venky Shankar <vshankar@redhat.com> Reviewed-on: http://review.gluster.org/9706 Tested-by: Vijay Bellur <vbellur@redhat.com>
2015-02-03 19:03:30 +05:30
AC_SUBST(MATH_LIB)
dnl use libuuid.so or fall-back to contrib/uuid
PKG_CHECK_MODULES([UUID], [uuid],
[have_uuid=yes
AC_DEFINE(HAVE_LIBUUID, 1, [have libuuid.so])
PKGCONFIG_UUID=uuid],
[have_uuid=no
UUID_CFLAGS='-I$(CONTRIBDIR)/uuid'])
AM_CONDITIONAL([HAVE_LIBUUID], [test x$have_uuid = xyes])
dnl older version of libuuid (from e2fsprogs) require including uuid/uuid.h
saved_CFLAGS=${CFLAGS}
CFLAGS="${CFLAGS} ${UUID_CFLAGS}"
AC_CHECK_HEADER([uuid.h], [], [AC_CHECK_HEADER([uuid/uuid.h])],
[[#if HAVE_UUID_H
#include <uuid.h>
#endif
]])
CFLAGS=${saved_CFLAGS}
if test "x$ac_cv_header_uuid_uuid_h" = "xyes"; then
UUID_CFLAGS="${UUID_CFLAGS} -I$(pkg-config --variable=includedir uuid)/uuid"
fi
dnl libglusterfs needs uuid.h, practically everything depends on it
GF_CPPFLAGS="${GF_CPPFLAGS} ${UUID_CFLAGS}"
dnl PKGCONFIG_UUID is used for the dependency in *.pc.in files
AC_SUBST(PKGCONFIG_UUID)
dnl NetBSD does not support POSIX ACLs :-(
case $host_os in
*netbsd* | darwin*)
AC_MSG_WARN([platform does not support POSIX ACLs... disabling them])
ACL_LIBS=''
USE_POSIX_ACLS='0'
BUILD_POSIX_ACLS='no'
;;
*)
AC_CHECK_HEADERS([sys/acl.h], ,
AC_MSG_ERROR([Support for POSIX ACLs is required]))
USE_POSIX_ACLS='1'
BUILD_POSIX_ACLS='yes'
case $host_os in
linux*)
ACL_LIBS='-lacl'
;;
solaris*)
ACL_LIBS='-lsec'
;;
*freebsd*)
ACL_LIBS='-lc'
;;
darwin*)
ACL_LIBS='-lc'
;;
esac
if test "x${ACL_LIBS}" = "x-lacl"; then
AC_CHECK_HEADERS([acl/libacl.h], , AC_MSG_ERROR([libacl is required for building on ${host_os}]))
fi
;;
esac
AC_SUBST(ACL_LIBS)
AC_SUBST(USE_POSIX_ACLS)
# libglusterfs/checksum
Replace GPLV3 MD5 with OpenSSL MD5 Ric asked me to look at replacing the GPL licensed MD5 code with something better, i.e. perhaps faster, and with a less restrictive license, etc. So I took a couple hour holiday from working on wrapping up the client_t and did this. OpenSSL (nee SSLeay) is released under the OpenSSL license, a BSD/MIT style license. OpenSSL (libcrypto.so) is used on Linux, OS X and *BSD, Open Solaris, etc. IOW it's universally available on the platforms we care about. It's written by Eric Young (eay), now at EMC/RSA, and I can say from experience that the OpenSSL implementation of MD5 (at least) is every bit as fast as RSA's proprietary implementation (primarily because the implementations are very, very similar.) The last time I surveyed MD5 implementations I found they're all pretty much the same speed. I changed the APIs (and ABIs) for the strong and weak checksums. Strictly speaking I didn't need to do that. They're only called on short strings of data, i.e. pathnames, so using int32_t and uint32_t is ostensibly okay. My change is arguably a better, more general API for this sort of thing. It's also what bit me when gerrit/jenkins validation failed due to glusterfs segv-ing. (I didn't pay close enough attention to the implementation of the weak checksum. But it forced me to learn what gerrit/jenkins are doing and going forward I can do better testing before submitting to gerrit.) Now resubmitting with a BZ Change-Id: I545fade1604e74fc68399894550229bd57a5e0df BUG: 807718 Signed-off-by: Kaleb KEITHLEY <kkeithle@redhat.com> Reviewed-on: http://review.gluster.com/3019 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
2012-03-27 11:14:23 -04:00
AC_CHECK_HEADERS([openssl/md5.h])
AC_CHECK_LIB([z], [adler32], [ZLIB_LIBS="-lz"], AC_MSG_ERROR([zlib is required to build glusterfs]))
AC_SUBST(ZLIB_LIBS)
Replace GPLV3 MD5 with OpenSSL MD5 Ric asked me to look at replacing the GPL licensed MD5 code with something better, i.e. perhaps faster, and with a less restrictive license, etc. So I took a couple hour holiday from working on wrapping up the client_t and did this. OpenSSL (nee SSLeay) is released under the OpenSSL license, a BSD/MIT style license. OpenSSL (libcrypto.so) is used on Linux, OS X and *BSD, Open Solaris, etc. IOW it's universally available on the platforms we care about. It's written by Eric Young (eay), now at EMC/RSA, and I can say from experience that the OpenSSL implementation of MD5 (at least) is every bit as fast as RSA's proprietary implementation (primarily because the implementations are very, very similar.) The last time I surveyed MD5 implementations I found they're all pretty much the same speed. I changed the APIs (and ABIs) for the strong and weak checksums. Strictly speaking I didn't need to do that. They're only called on short strings of data, i.e. pathnames, so using int32_t and uint32_t is ostensibly okay. My change is arguably a better, more general API for this sort of thing. It's also what bit me when gerrit/jenkins validation failed due to glusterfs segv-ing. (I didn't pay close enough attention to the implementation of the weak checksum. But it forced me to learn what gerrit/jenkins are doing and going forward I can do better testing before submitting to gerrit.) Now resubmitting with a BZ Change-Id: I545fade1604e74fc68399894550229bd57a5e0df BUG: 807718 Signed-off-by: Kaleb KEITHLEY <kkeithle@redhat.com> Reviewed-on: http://review.gluster.com/3019 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
2012-03-27 11:14:23 -04:00
AC_CHECK_HEADERS([linux/falloc.h])
AC_CHECK_HEADERS([linux/oom.h], AC_DEFINE(HAVE_LINUX_OOM_H, 1, [have linux/oom.h]))
2009-02-18 17:36:07 +05:30
dnl Mac OS X does not have spinlocks
AC_CHECK_FUNC([pthread_spin_init], [have_spinlock=yes])
if test "x${have_spinlock}" = "xyes"; then
AC_DEFINE(HAVE_SPINLOCK, 1, [define if found spinlock])
fi
AC_SUBST(HAVE_SPINLOCK)
dnl some os may not have GNU defined strnlen function
AC_CHECK_FUNC([strnlen], [have_strnlen=yes])
if test "x${have_strnlen}" = "xyes"; then
AC_DEFINE(HAVE_STRNLEN, 1, [define if found strnlen])
fi
AC_SUBST(HAVE_STRNLEN)
AC_CHECK_FUNC([setfsuid], [have_setfsuid=yes])
AC_CHECK_FUNC([setfsgid], [have_setfsgid=yes])
if test "x${have_setfsuid}" = "xyes" -a "x${have_setfsgid}" = "xyes"; then
AC_DEFINE(HAVE_SET_FSID, 1, [define if found setfsuid setfsgid])
fi
dnl test umount2 function
AC_CHECK_FUNC([umount2], [have_umount2=yes])
if test "x${have_umount2}" = "xyes"; then
AC_DEFINE(HAVE_UMOUNT2, 1, [define if found umount2])
fi
2009-02-18 17:36:07 +05:30
dnl Check Python Availability
have_python=no
AM_PATH_PYTHON(,, [:])
if test "$PYTHON" != ":"; then
have_python=yes
fi
dnl Check if version matches that we require
PYTHONDEV_CPPFLAGS=
PYTHONDEV_LDFLAGS=
BUILD_PYTHON_SITE_PACKAGES=
BUILD_PYTHON_INC=
BUILD_PYTHON_LIB=
have_python2=no
have_Python_h=no
if echo $PYTHON_VERSION | grep -q ^2; then
have_python2=yes
dnl Use pkg-config to get runtime search patch missing from ${PYTHON}-config
dnl Just do "true" on failure so that configure does not bail out
PKG_CHECK_MODULES([PYTHON], "python-$PYTHON_VERSION",,true)
PYTHONDEV_CPPFLAGS="`${PYTHON}-config --cflags`"
dnl Edit out the flags that are not required or are conflicting
PYTHONDEV_CPPFLAGS=`echo ${PYTHONDEV_CPPFLAGS} | sed -e 's/-Wp,-D_FORTIFY_SOURCE=[[0-9]]//g'`
dnl Find python libs at user configured libdir and also "lib" under prefix
PYTHONDEV_LDFLAGS="${PYTHON_LIBS} -L`${PYTHON}-config --prefix`/lib -L`${PYTHON}-config --prefix`/$libdir `${PYTHON}-config --ldflags`"
BUILD_PYTHON_SITE_PACKAGES=${pythondir}
BUILD_PYTHON_INC=`$PYTHON -c "import sys; from distutils import sysconfig; sys.stdout.write(sysconfig.get_python_inc())" 2>/dev/null`
BUILD_PYTHON_LIB=python$PYTHON_VERSION
dnl Now check for python header using the include path obtained above
AC_CHECK_HEADERS([${BUILD_PYTHON_INC}/Python.h],[have_Python_h=yes],[])
fi
AC_SUBST(PYTHONDEV_CPPFLAGS)
AC_SUBST(PYTHONDEV_LDFLAGS)
AC_SUBST(BUILD_PYTHON_SITE_PACKAGES)
AC_SUBST(BUILD_PYTHON_INC)
AC_SUBST(BUILD_PYTHON_LIB)
2009-02-18 17:36:07 +05:30
# FUSE section
AC_ARG_ENABLE([fuse-client],
AC_HELP_STRING([--disable-fuse-client],
[Do not build the fuse client. NOTE: you cannot mount glusterfs without the client]))
2009-02-18 17:36:07 +05:30
BUILD_FUSE_CLIENT=no
if test "x$enable_fuse_client" != "xno"; then
2009-02-18 17:36:07 +05:30
FUSE_CLIENT_SUBDIR=fuse
BUILD_FUSE_CLIENT="yes"
fi
bd: posix/multi-brick support to BD xlator Current BD xlator (block backend) has a few limitations such as * Creation of directories not supported * Supports only single brick * Does not use extended attributes (and client gfid) like posix xlator * Creation of special files (symbolic links, device nodes etc) not supported Basic limitation of not allowing directory creation is blocking oVirt/VDSM to consume BD xlator as part of Gluster domain since VDSM creates multi-level directories when GlusterFS is used as storage backend for storing VM images. To overcome these limitations a new BD xlator with following improvements is suggested. * New hybrid BD xlator that handles both regular files and block device files * The volume will have both POSIX and BD bricks. Regular files are created on POSIX bricks, block devices are created on the BD brick (VG) * BD xlator leverages exiting POSIX xlator for most POSIX calls and hence sits above the POSIX xlator * Block device file is differentiated from regular file by an extended attribute * The xattr 'user.glusterfs.bd' (BD_XATTR) plays a role in mapping a posix file to Logical Volume (LV). * When a client sends a request to set BD_XATTR on a posix file, a new LV is created and mapped to posix file. So every block device will have a representative file in POSIX brick with 'user.glusterfs.bd' (BD_XATTR) set. * Here after all operations on this file results in LV related operations. For example opening a file that has BD_XATTR set results in opening the LV block device, reading results in reading the corresponding LV block device. When BD xlator gets request to set BD_XATTR via setxattr call, it creates a LV and information about this LV is placed in the xattr of the posix file. xattr "user.glusterfs.bd" used to identify that posix file is mapped to BD. Usage: Server side: [root@host1 ~]# gluster volume create bdvol host1:/storage/vg1_info?vg1 host2:/storage/vg2_info?vg2 It creates a distributed gluster volume 'bdvol' with Volume Group vg1 using posix brick /storage/vg1_info in host1 and Volume Group vg2 using /storage/vg2_info in host2. [root@host1 ~]# gluster volume start bdvol Client side: [root@node ~]# mount -t glusterfs host1:/bdvol /media [root@node ~]# touch /media/posix It creates regular posix file 'posix' in either host1:/vg1 or host2:/vg2 brick [root@node ~]# mkdir /media/image [root@node ~]# touch /media/image/lv1 It also creates regular posix file 'lv1' in either host1:/vg1 or host2:/vg2 brick [root@node ~]# setfattr -n "user.glusterfs.bd" -v "lv" /media/image/lv1 [root@node ~]# Above setxattr results in creating a new LV in corresponding brick's VG and it sets 'user.glusterfs.bd' with value 'lv:<default-extent-size' [root@node ~]# truncate -s5G /media/image/lv1 It results in resizig LV 'lv1'to 5G New BD xlator code is placed in xlators/storage/bd directory. Also add volume-uuid to the VG so that same VG can't be used for other bricks/volumes. After deleting a gluster volume, one has to manually remove the associated tag using vgchange <vg-name> --deltag <trusted.glusterfs.volume-id:<volume-id>> Changes from previous version V5: * Removed support for delayed deleting of LVs Changes from previous version V4: * Consolidated the patches * Removed usage of BD_XATTR_SIZE and consolidated it in BD_XATTR. Changes from previous version V3: * Added support in FUSE to support full/linked clone * Added support to merge snapshots and provide information about origin * bd_map xlator removed * iatt structure used in inode_ctx. iatt is cached and updated during fsync/flush * aio support * Type and capabilities of volume are exported through getxattr Changes from version 2: * Used inode_context for caching BD size and to check if loc/fd is BD or not. * Added GlusterFS server offloaded copy and snapshot through setfattr FOP. As part of this libgfapi is modified. * BD xlator supports stripe * During unlinking if a LV file is already opened, its added to delete list and bd_del_thread tries to delete from this list when a last reference to that file is closed. Changes from previous version: * gfid is used as name of LV * ? is used to specify VG name for creating BD volume in volume create, add-brick. gluster volume create volname host:/path?vg * open-behind issue is fixed * A replicate brick can be added dynamically and LVs from source brick are replicated to destination brick * A distribute brick can be added dynamically and rebalance operation distributes existing LVs/files to the new brick * Thin provisioning support added. * bd_map xlator support retained * setfattr -n user.glusterfs.bd -v "lv" creates a regular LV and setfattr -n user.glusterfs.bd -v "thin" creates thin LV * Capability and backend information added to gluster volume info (and --xml) so that management tools can exploit BD xlator. * tracing support for bd xlator added TODO: * Add support to display snapshots for a given LV * Display posix filename for list-origin instead of gfid Change-Id: I00d32dfbab3b7c806e0841515c86c3aa519332f2 BUG: 1028672 Signed-off-by: M. Mohan Kumar <mohan@in.ibm.com> Reviewed-on: http://review.gluster.org/4809 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Anand Avati <avati@redhat.com>
2013-11-13 22:44:42 +05:30
AC_ARG_ENABLE([bd-xlator],
AC_HELP_STRING([--enable-bd-xlator], [Build BD xlator]))
if test "x$enable_bd_xlator" != "xno"; then
AC_CHECK_LIB([lvm2app],
[lvm_init,lvm_lv_from_name],
[HAVE_BD_LIB="yes"],
[HAVE_BD_LIB="no"])
if test "x$HAVE_BD_LIB" = "xyes"; then
# lvm_lv_from_name() has been made public with lvm2-2.02.79
AC_CHECK_DECLS(
[lvm_lv_from_name],
[NEED_LVM_LV_FROM_NAME_DECL="no"],
[NEED_LVM_LV_FROM_NAME_DECL="yes"],
[[#include <lvm2app.h>]])
fi
fi
if test "x$enable_bd_xlator" = "xyes" -a "x$HAVE_BD_LIB" = "xno"; then
echo "BD xlator requested but required lvm2 development library not found."
exit 1
fi
BUILD_BD_XLATOR=no
if test "x${enable-bd-xlator}" != "xno" -a "x${HAVE_BD_LIB}" = "xyes"; then
BUILD_BD_XLATOR=yes
AC_DEFINE(HAVE_BD_XLATOR, 1, [define if lvm2app library found and bd xlator
enabled])
if test "x$NEED_LVM_LV_FROM_NAME_DECL" = "xyes"; then
AC_DEFINE(NEED_LVM_LV_FROM_NAME_DECL, 1, [defined if lvm_lv_from_name()
was not found in the lvm2app.h header, but can be linked])
fi
fi
AM_CONDITIONAL([ENABLE_BD_XLATOR], [test x$BUILD_BD_XLATOR = xyes])
dnl check for old openssl
AC_CHECK_LIB([crypto], CRYPTO_THREADID_set_callback, [AC_DEFINE([HAVE_CRYPTO_THREADID], [1], [use new OpenSSL functions])])
AC_CHECK_LIB([ssl], TLS_method, [HAVE_OPENSSL_1_1="yes"], [HAVE_OPENSSL_1_1="no"])
if test "x$HAVE_OPENSSL_1_1" = "xyes"; then
AC_DEFINE([HAVE_TLS_METHOD], [1], [Using OpenSSL-1.1 TLS_method])
else
AC_CHECK_LIB([ssl], TLSv1_2_method, [AC_DEFINE([HAVE_TLSV1_2_METHOD], [1], [Using OpenSSL-1.0 TLSv1_2_method])])
fi
Transparent data encryption and metadata authentication .. in the systems with non-trusted server This new functionality can be useful in various cloud technologies. It is implemented via a special encryption/crypt translator,which works on the client side and performs encryption and authentication; 1. Class of supported algorithms The crypt translator can support any atomic symmetric block cipher algorithms (which require to pad plain/cipher text before performing encryption/decryption transform (see glossary in atom.c for definitions). In particular, it can support algorithms with the EOF issue (which require to pad the end of file by extra-data). Crypt translator performs translations user -> (offset, size) -> (aligned-offset, padded-size) ->server (and backward), and resolves individual FOPs (write(), truncate(), etc) to read-modify-write sequences. A volume can contain files encrypted by different algorithms of the mentioned class. To change some option value just reconfigure the volume. Currently only one algorithm is supported: AES_XTS. Example of algorithms, which can not be supported by the crypt translator: 1. Asymmetric block cipher algorithms, which inflate data, e.g. RSA; 2. Symmetric block cipher algorithms with inline MACs for data authentication. 2. Implementation notes. a) Atomic algorithms Since any process in a stackable file system manipulates with local data (which can be obsoleted by local data of another process), any atomic cipher algorithm without proper support can lead to non-POSIX behavior. To resolve the "collisions" we introduce locks: before performing FOP->read(), FOP->write(), etc. the process should first lock the file. b) Algorithms with EOF issue Such algorithms require to pad the end of file with some extra-data. Without proper support this will result in losing information about real file size. Keeping a track of real file size is a responsibility of the crypt translator. A special extended attribute with the name "trusted.glusterfs.crypt.att.size" is used for this purpose. All files contained in bricks of encrypted volume do have "padded" sizes. 3. Non-trusted servers and Metadata authentication We assume that server, where user's data is stored on is non-trusted. It means that the server can be subjected to various attacks directed to reveal user's encrypted personal data. We provide protection against such attacks. Every encrypted file has specific private attributes (cipher algorithm id, atom size, etc), which are packed to a string (so-called "format string") and stored as a special extended attribute with the name "trusted.glusterfs.crypt.att.cfmt". We protect the string from tampering. This protection is mandatory, hardcoded and is always on. Without such protection various attacks (based on extending the scope of per-file secret keys) are possible. Our authentication method has been developed in tight collaboration with Red Hat security team and is implemented as "metadata loader of version 1" (see file metadata.c). This method is NIST-compliant and is based on checking 8-byte per-hardlink MACs created(updated) by FOP->create(), FOP->link(), FOP->unlink(), FOP->rename() by the following unique entities: . file (hardlink) name; . verified file's object id (gfid). Every time, before manipulating with a file, we check it's MACs at FOP->open() time. Some FOPs don't require a file to be opened (e.g. FOP->truncate()). In such cases the crypt translator opens the file mandatory. 4. Generating keys Unique per-file keys are derived by NIST-compliant methods from the a) parent key; b) unique verified object-id of the file (gfid); Per-volume master key, provided by user at mount time is in the root of this "tree of keys". Those keys are used to: 1) encrypt/decrypt file data; 2) encrypt/decrypt file metadata; 3) create per-file and per-link MACs for metadata authentication. 5. Instructions Getting started with crypt translator Example: 1) Create a volume "myvol" and enable encryption: # gluster volume create myvol pepelac:/vols/xvol # gluster volume set myvol encryption on 2) Set location (absolute pathname) of your master key: # gluster volume set myvol encryption.master-key /home/me/mykey 3) Set other options to override default options, if needed. Start the volume. 4) On the client side make sure that the file /home/me/mykey exists and contains proper per-volume master key (that is 256-bit AES key). This key has to be in hex form, i.e. should be represented by 64 symbols from the set {'0', ..., '9', 'a', ..., 'f'}. The key should start at the beginning of the file. All symbols at offsets >= 64 are ignored. 5) Mount the volume "myvol" on the client side: # glusterfs --volfile-server=pepelac --volfile-id=myvol /mnt After successful mount the file which contains master key may be removed. NOTE: Keeping the master key between mount sessions is in user's competence. ********************************************************************** WARNING! Losing the master key will make content of all regular files inaccessible. Mount with improper master key allows to access content of directories: file names are not encrypted. ********************************************************************** 6. Options of crypt translator 1) "master-key": specifies location (absolute pathname) of the file which contains per-volume master key. There is no default location for master key. 2) "data-key-size": specifies size of per-file key for data encryption Possible values: . "256" default value . "512" 3) "block-size": specifies atom size. Possible values: . "512" . "1024" . "2048" . "4096" default value; 7. Test cases Any workload, which involves the following file operations: ->create(); ->open(); ->readv(); ->writev(); ->truncate(); ->ftruncate(); ->link(); ->unlink(); ->rename(); ->readdirp(). 8. TODOs: 1) Currently size of IOs issued by crypt translator is restricted by block_size (4K by default). We can use larger IOs to improve performance. Change-Id: I2601fe95c5c4dc5b22308a53d0cbdc071d5e5cee BUG: 1030058 Signed-off-by: Edward Shishkin <edward@redhat.com> Signed-off-by: Anand Avati <avati@redhat.com> Reviewed-on: http://review.gluster.org/4667 Tested-by: Gluster Build System <jenkins@build.gluster.com>
2013-03-13 21:56:46 +01:00
# start encryption/crypt section
AC_CHECK_HEADERS([openssl/cmac.h], [have_cmac_h=yes], [have_cmac_h=no])
AC_ARG_ENABLE([crypt-xlator],
Adding Libgfdb to GlusterFS ************************************************************************* Libgfdb | ************************************************************************* Libgfdb provides abstract mechanism to record extra/rich metadata required for data maintenance, such as data tiering/classification. It provides consumer with API for recording and querying, keeping the consumer abstracted from the data store used beneath for storing data. It works in a plug-and-play model, where data stores can be plugged-in. Presently we have plugin for Sqlite3. In the future will provide recording and querying performance optimizer. In the current implementation the schema of metadata is fixed. Schema: ~~~~~~ GF_FILE_TB Table: ~~~~~~~~~~~~~~~~~ This table has one entry per file inode. It holds the metadata required to make decisions in data maintenance. GF_ID (Primary key) : File GFID (Universal Unique IDentifier in the namespace) W_SEC, W_MSEC : Write wind time in sec & micro-sec UW_SEC, UW_MSEC : Write un-wind time in sec & micro-sec W_READ_SEC, W_READ_MSEC : Read wind time in sec & micro-sec UW_READ_SEC, UW_READ_MSEC : Read un-wind time in sec & micro-sec WRITE_FREQ_CNTR INTEGER : Write Frequency Counter READ_FREQ_CNTR INTEGER : Read Frequency Counter GF_FLINK_TABLE: ~~~~~~~~~~~~~~ This table has all the hardlinks to a file inode. GF_ID : File GFID (Composite Primary Key)``| GF_PID : Parent Directory GFID (Composite Primary Key) |-> Primary Key FNAME : File Base Name (Composite Primary Key)__| FPATH : File Full Path (Its redundant for now, this will go) W_DEL_FLAG : This Flag is used for crash consistancy, when a link is unlinked. i.e Set to 1 during unlink wind and during unwind this record is deleted LINK_UPDATE : This Flag is used when a link is changed i.e rename. Set to 1 when rename wind and set to 0 in rename unwind Libgfdb API: ~~~~~~~~~~~ Refer libglusterfs/src/gfdb/gfdb_data_store.h Change-Id: I2e9fbab3878ce630a7f41221ef61017dc43db11f BUG: 1194753 Signed-off-by: Joseph Fernandes <josferna@redhat.com> Signed-off-by: Dan Lambright <dlambrig@redhat.com> Signed-off-by: Joseph Fernandes <josferna@redhat.com> Reviewed-on: http://review.gluster.org/9683 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2015-02-18 19:45:23 +05:30
AC_HELP_STRING([--enable-crypt-xlator], [Build crypt encryption xlator]))
Transparent data encryption and metadata authentication .. in the systems with non-trusted server This new functionality can be useful in various cloud technologies. It is implemented via a special encryption/crypt translator,which works on the client side and performs encryption and authentication; 1. Class of supported algorithms The crypt translator can support any atomic symmetric block cipher algorithms (which require to pad plain/cipher text before performing encryption/decryption transform (see glossary in atom.c for definitions). In particular, it can support algorithms with the EOF issue (which require to pad the end of file by extra-data). Crypt translator performs translations user -> (offset, size) -> (aligned-offset, padded-size) ->server (and backward), and resolves individual FOPs (write(), truncate(), etc) to read-modify-write sequences. A volume can contain files encrypted by different algorithms of the mentioned class. To change some option value just reconfigure the volume. Currently only one algorithm is supported: AES_XTS. Example of algorithms, which can not be supported by the crypt translator: 1. Asymmetric block cipher algorithms, which inflate data, e.g. RSA; 2. Symmetric block cipher algorithms with inline MACs for data authentication. 2. Implementation notes. a) Atomic algorithms Since any process in a stackable file system manipulates with local data (which can be obsoleted by local data of another process), any atomic cipher algorithm without proper support can lead to non-POSIX behavior. To resolve the "collisions" we introduce locks: before performing FOP->read(), FOP->write(), etc. the process should first lock the file. b) Algorithms with EOF issue Such algorithms require to pad the end of file with some extra-data. Without proper support this will result in losing information about real file size. Keeping a track of real file size is a responsibility of the crypt translator. A special extended attribute with the name "trusted.glusterfs.crypt.att.size" is used for this purpose. All files contained in bricks of encrypted volume do have "padded" sizes. 3. Non-trusted servers and Metadata authentication We assume that server, where user's data is stored on is non-trusted. It means that the server can be subjected to various attacks directed to reveal user's encrypted personal data. We provide protection against such attacks. Every encrypted file has specific private attributes (cipher algorithm id, atom size, etc), which are packed to a string (so-called "format string") and stored as a special extended attribute with the name "trusted.glusterfs.crypt.att.cfmt". We protect the string from tampering. This protection is mandatory, hardcoded and is always on. Without such protection various attacks (based on extending the scope of per-file secret keys) are possible. Our authentication method has been developed in tight collaboration with Red Hat security team and is implemented as "metadata loader of version 1" (see file metadata.c). This method is NIST-compliant and is based on checking 8-byte per-hardlink MACs created(updated) by FOP->create(), FOP->link(), FOP->unlink(), FOP->rename() by the following unique entities: . file (hardlink) name; . verified file's object id (gfid). Every time, before manipulating with a file, we check it's MACs at FOP->open() time. Some FOPs don't require a file to be opened (e.g. FOP->truncate()). In such cases the crypt translator opens the file mandatory. 4. Generating keys Unique per-file keys are derived by NIST-compliant methods from the a) parent key; b) unique verified object-id of the file (gfid); Per-volume master key, provided by user at mount time is in the root of this "tree of keys". Those keys are used to: 1) encrypt/decrypt file data; 2) encrypt/decrypt file metadata; 3) create per-file and per-link MACs for metadata authentication. 5. Instructions Getting started with crypt translator Example: 1) Create a volume "myvol" and enable encryption: # gluster volume create myvol pepelac:/vols/xvol # gluster volume set myvol encryption on 2) Set location (absolute pathname) of your master key: # gluster volume set myvol encryption.master-key /home/me/mykey 3) Set other options to override default options, if needed. Start the volume. 4) On the client side make sure that the file /home/me/mykey exists and contains proper per-volume master key (that is 256-bit AES key). This key has to be in hex form, i.e. should be represented by 64 symbols from the set {'0', ..., '9', 'a', ..., 'f'}. The key should start at the beginning of the file. All symbols at offsets >= 64 are ignored. 5) Mount the volume "myvol" on the client side: # glusterfs --volfile-server=pepelac --volfile-id=myvol /mnt After successful mount the file which contains master key may be removed. NOTE: Keeping the master key between mount sessions is in user's competence. ********************************************************************** WARNING! Losing the master key will make content of all regular files inaccessible. Mount with improper master key allows to access content of directories: file names are not encrypted. ********************************************************************** 6. Options of crypt translator 1) "master-key": specifies location (absolute pathname) of the file which contains per-volume master key. There is no default location for master key. 2) "data-key-size": specifies size of per-file key for data encryption Possible values: . "256" default value . "512" 3) "block-size": specifies atom size. Possible values: . "512" . "1024" . "2048" . "4096" default value; 7. Test cases Any workload, which involves the following file operations: ->create(); ->open(); ->readv(); ->writev(); ->truncate(); ->ftruncate(); ->link(); ->unlink(); ->rename(); ->readdirp(). 8. TODOs: 1) Currently size of IOs issued by crypt translator is restricted by block_size (4K by default). We can use larger IOs to improve performance. Change-Id: I2601fe95c5c4dc5b22308a53d0cbdc071d5e5cee BUG: 1030058 Signed-off-by: Edward Shishkin <edward@redhat.com> Signed-off-by: Anand Avati <avati@redhat.com> Reviewed-on: http://review.gluster.org/4667 Tested-by: Gluster Build System <jenkins@build.gluster.com>
2013-03-13 21:56:46 +01:00
if test "x$enable_crypt_xlator" = "xyes" -a "x$have_cmac_h" = "xno"; then
AC_MSG_ERROR([Encryption xlator requires OpenSSL with cmac.h])
Transparent data encryption and metadata authentication .. in the systems with non-trusted server This new functionality can be useful in various cloud technologies. It is implemented via a special encryption/crypt translator,which works on the client side and performs encryption and authentication; 1. Class of supported algorithms The crypt translator can support any atomic symmetric block cipher algorithms (which require to pad plain/cipher text before performing encryption/decryption transform (see glossary in atom.c for definitions). In particular, it can support algorithms with the EOF issue (which require to pad the end of file by extra-data). Crypt translator performs translations user -> (offset, size) -> (aligned-offset, padded-size) ->server (and backward), and resolves individual FOPs (write(), truncate(), etc) to read-modify-write sequences. A volume can contain files encrypted by different algorithms of the mentioned class. To change some option value just reconfigure the volume. Currently only one algorithm is supported: AES_XTS. Example of algorithms, which can not be supported by the crypt translator: 1. Asymmetric block cipher algorithms, which inflate data, e.g. RSA; 2. Symmetric block cipher algorithms with inline MACs for data authentication. 2. Implementation notes. a) Atomic algorithms Since any process in a stackable file system manipulates with local data (which can be obsoleted by local data of another process), any atomic cipher algorithm without proper support can lead to non-POSIX behavior. To resolve the "collisions" we introduce locks: before performing FOP->read(), FOP->write(), etc. the process should first lock the file. b) Algorithms with EOF issue Such algorithms require to pad the end of file with some extra-data. Without proper support this will result in losing information about real file size. Keeping a track of real file size is a responsibility of the crypt translator. A special extended attribute with the name "trusted.glusterfs.crypt.att.size" is used for this purpose. All files contained in bricks of encrypted volume do have "padded" sizes. 3. Non-trusted servers and Metadata authentication We assume that server, where user's data is stored on is non-trusted. It means that the server can be subjected to various attacks directed to reveal user's encrypted personal data. We provide protection against such attacks. Every encrypted file has specific private attributes (cipher algorithm id, atom size, etc), which are packed to a string (so-called "format string") and stored as a special extended attribute with the name "trusted.glusterfs.crypt.att.cfmt". We protect the string from tampering. This protection is mandatory, hardcoded and is always on. Without such protection various attacks (based on extending the scope of per-file secret keys) are possible. Our authentication method has been developed in tight collaboration with Red Hat security team and is implemented as "metadata loader of version 1" (see file metadata.c). This method is NIST-compliant and is based on checking 8-byte per-hardlink MACs created(updated) by FOP->create(), FOP->link(), FOP->unlink(), FOP->rename() by the following unique entities: . file (hardlink) name; . verified file's object id (gfid). Every time, before manipulating with a file, we check it's MACs at FOP->open() time. Some FOPs don't require a file to be opened (e.g. FOP->truncate()). In such cases the crypt translator opens the file mandatory. 4. Generating keys Unique per-file keys are derived by NIST-compliant methods from the a) parent key; b) unique verified object-id of the file (gfid); Per-volume master key, provided by user at mount time is in the root of this "tree of keys". Those keys are used to: 1) encrypt/decrypt file data; 2) encrypt/decrypt file metadata; 3) create per-file and per-link MACs for metadata authentication. 5. Instructions Getting started with crypt translator Example: 1) Create a volume "myvol" and enable encryption: # gluster volume create myvol pepelac:/vols/xvol # gluster volume set myvol encryption on 2) Set location (absolute pathname) of your master key: # gluster volume set myvol encryption.master-key /home/me/mykey 3) Set other options to override default options, if needed. Start the volume. 4) On the client side make sure that the file /home/me/mykey exists and contains proper per-volume master key (that is 256-bit AES key). This key has to be in hex form, i.e. should be represented by 64 symbols from the set {'0', ..., '9', 'a', ..., 'f'}. The key should start at the beginning of the file. All symbols at offsets >= 64 are ignored. 5) Mount the volume "myvol" on the client side: # glusterfs --volfile-server=pepelac --volfile-id=myvol /mnt After successful mount the file which contains master key may be removed. NOTE: Keeping the master key between mount sessions is in user's competence. ********************************************************************** WARNING! Losing the master key will make content of all regular files inaccessible. Mount with improper master key allows to access content of directories: file names are not encrypted. ********************************************************************** 6. Options of crypt translator 1) "master-key": specifies location (absolute pathname) of the file which contains per-volume master key. There is no default location for master key. 2) "data-key-size": specifies size of per-file key for data encryption Possible values: . "256" default value . "512" 3) "block-size": specifies atom size. Possible values: . "512" . "1024" . "2048" . "4096" default value; 7. Test cases Any workload, which involves the following file operations: ->create(); ->open(); ->readv(); ->writev(); ->truncate(); ->ftruncate(); ->link(); ->unlink(); ->rename(); ->readdirp(). 8. TODOs: 1) Currently size of IOs issued by crypt translator is restricted by block_size (4K by default). We can use larger IOs to improve performance. Change-Id: I2601fe95c5c4dc5b22308a53d0cbdc071d5e5cee BUG: 1030058 Signed-off-by: Edward Shishkin <edward@redhat.com> Signed-off-by: Anand Avati <avati@redhat.com> Reviewed-on: http://review.gluster.org/4667 Tested-by: Gluster Build System <jenkins@build.gluster.com>
2013-03-13 21:56:46 +01:00
fi
BUILD_CRYPT_XLATOR=no
if test "x$enable_crypt_xlator" != "xno" -a "x$have_cmac_h" = "xyes"; then
BUILD_CRYPT_XLATOR=yes
AC_DEFINE(HAVE_CRYPT_XLATOR, 1, [enable building crypt encryption xlator])
fi
AM_CONDITIONAL([ENABLE_CRYPT_XLATOR], [test x$BUILD_CRYPT_XLATOR = xyes])
2009-02-18 17:36:07 +05:30
AC_SUBST(FUSE_CLIENT_SUBDIR)
# end FUSE section
2009-08-11 18:26:11 -07:00
# FUSERMOUNT section
AC_ARG_ENABLE([fusermount],
AC_HELP_STRING([--disable-fusermount],
[Use system's fusermount]))
2009-08-11 18:26:11 -07:00
BUILD_FUSERMOUNT="yes"
if test "x$enable_fusermount" = "xno"; then
BUILD_FUSERMOUNT="no"
else
AC_DEFINE(GF_FUSERMOUNT, 1, [Use our own fusermount])
FUSERMOUNT_SUBDIR="contrib/fuse-util"
2009-08-11 18:26:11 -07:00
fi
AC_SUBST(FUSERMOUNT_SUBDIR)
#end FUSERMOUNT section
2009-02-18 17:36:07 +05:30
# EPOLL section
AC_ARG_ENABLE([epoll],
AC_HELP_STRING([--disable-epoll],
[Use poll instead of epoll.]))
2009-02-18 17:36:07 +05:30
BUILD_EPOLL=no
if test "x$enable_epoll" != "xno"; then
AC_CHECK_HEADERS([sys/epoll.h],
[BUILD_EPOLL=yes],
[BUILD_EPOLL=no])
2009-02-18 17:36:07 +05:30
fi
# end EPOLL section
# IBVERBS section
AC_ARG_ENABLE([ibverbs],
AC_HELP_STRING([--disable-ibverbs],
[Do not build the ibverbs transport]))
2009-02-18 17:36:07 +05:30
if test "x$enable_ibverbs" != "xno"; then
AC_CHECK_LIB([ibverbs],
[ibv_get_device_list],
[HAVE_LIBIBVERBS="yes"],
[HAVE_LIBIBVERBS="no"])
AC_CHECK_LIB([rdmacm], [rdma_create_id], [HAVE_RDMACM="yes"], [HAVE_RDMACM="no"])
if test "x$HAVE_RDMACM" = "xyes" ; then
AC_CHECK_DECLS(
[RDMA_OPTION_ID_REUSEADDR],
[],
[AC_ERROR([Need at least version 1.0.15 of librdmacm])],
[[#include <rdma/rdma_cma.h>]])
fi
2009-02-18 17:36:07 +05:30
fi
if test "x$enable_ibverbs" = "xyes"; then
if test "x$HAVE_LIBIBVERBS" = "xno"; then
echo "ibverbs-transport requested, but libibverbs is not present."
exit 1
fi
if test "x$HAVE_RDMACM" = "xno"; then
echo "ibverbs-transport requested, but librdmacm is not present."
exit 1
fi
2009-02-18 17:36:07 +05:30
fi
BUILD_RDMA=no
2009-02-18 17:36:07 +05:30
BUILD_IBVERBS=no
if test "x$enable_ibverbs" != "xno" -a "x$HAVE_LIBIBVERBS" = "xyes" -a "x$HAVE_RDMACM" = "xyes"; then
2009-02-18 17:36:07 +05:30
IBVERBS_SUBDIR=ib-verbs
BUILD_IBVERBS=yes
RDMA_SUBDIR=rdma
BUILD_RDMA=yes
2009-02-18 17:36:07 +05:30
fi
AC_SUBST(IBVERBS_SUBDIR)
AC_SUBST(RDMA_SUBDIR)
2009-02-18 17:36:07 +05:30
# end IBVERBS section
# SYNCDAEMON section
AC_ARG_ENABLE([georeplication],
AC_HELP_STRING([--disable-georeplication],
[Do not install georeplication components]))
BUILD_SYNCDAEMON=no
case $host_os in
linux*)
#do nothing
;;
netbsd*)
#do nothing
;;
*)
#disabling geo replication for non-linux platforms
enable_georeplication=no
;;
esac
SYNCDAEMON_COMPILE=0
if test "x$enable_georeplication" != "xno"; then
SYNCDAEMON_SUBDIR=geo-replication
SYNCDAEMON_COMPILE=1
BUILD_SYNCDAEMON="yes"
AM_PATH_PYTHON([2.4])
echo -n "checking if python is python 2.x... "
if echo $PYTHON_VERSION | grep ^2; then
:
else
echo no
AC_MSG_ERROR([only python 2.x is supported])
fi
echo -n "checking if python has ctypes support... "
if "$PYTHON" -c 'import ctypes' 2>/dev/null; then
echo yes
else
echo no
AC_MSG_ERROR([python does not have ctypes support])
fi
fi
AC_SUBST(SYNCDAEMON_COMPILE)
AC_SUBST(SYNCDAEMON_SUBDIR)
# end SYNCDAEMON section
# only install scripts from extras/geo-rep when enabled
if test "x$enable_georeplication" != "xno"; then
GEOREP_EXTRAS_SUBDIR=geo-rep
fi
AC_SUBST(GEOREP_EXTRAS_SUBDIR)
AM_CONDITIONAL(USE_GEOREP, test "x$enable_georeplication" != "xno")
eventsapi: Gluster Eventing Feature implementation [Depends on http://review.gluster.org/14627] Design is available in `glusterfs-specs`, A change from the design is support of webhook instead of Websockets as discussed in the design http://review.gluster.org/13115 Since Websocket support depends on REST APIs, I will add Websocket support once REST APIs patch gets merged Usage: Run following command to start/stop Eventsapi server in all Peers, which will collect the notifications from any Gluster daemon and emits to configured client. gluster-eventsapi start|stop|restart|reload Status of running services can be checked using, gluster-eventsapi status Events listener is a HTTP(S) server which listens to events emited by the Gluster. Create a HTTP Server to listen on POST and register that URL using, gluster-eventsapi webhook-add <URL> [--bearer-token <TOKEN>] For example, if HTTP Server running in `http://192.168.122.188:9000` then add that URL using, gluster-eventsapi webhook-add http://192.168.122.188:9000 If it expects a Token then specify it using `--bearer-token` or `-t` We can also test Webhook if all peer nodes can send message or not using, gluster-eventsapi webhook-test <URL> [--bearer-token <TOKEN>] Configurations can be viewed/updated using, gluster-eventsapi config-get [--name] gluster-eventsapi config-set <NAME> <VALUE> gluster-eventsapi config-reset <NAME|all> If any one peer node was down during config-set/reset or webhook modifications, Run sync command from good node when a peer node comes back. Automatic update is not yet implemented. gluster-eventsapi sync Basic Events Client(HTTP Server) is included with the code, Start running the client with required port and start listening to the events. /usr/share/glusterfs/scripts/eventsdash.py --port 8080 Default port is 9000, if no port is specified, once it started running then configure gluster-eventsapi to send events to that client. Eventsapi Client can be outside of the Cluster, it can be run event on Windows. But only requirement is the client URL should be accessible by all peer nodes.(Or ngrok(https://ngrok.com) like tools can be used) Events implemented with this patch, - Volume Create - Volume Start - Volume Stop - Volume Delete - Peer Attach - Peer Detach It is easy to add/support more events, since it touches Gluster cmd code and to avoid merge conflicts I will add support for more events once this patch merges. BUG: 1334044 Change-Id: I316827ac9dd1443454df7deffe4f54835f7f6a08 Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/14248 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
2016-05-05 18:34:41 +05:30
# Events section
AC_ARG_ENABLE([events],
AC_HELP_STRING([--disable-events],
[Do not install Events components]))
BUILD_EVENTS=no
EVENTS_ENABLED=0
EVENTS_SUBDIR=
if test "x$enable_events" != "xno"; then
EVENTS_SUBDIR=events
EVENTS_ENABLED=1
BUILD_EVENTS="yes"
if test "x$have_python2" = "xno"; then
if test "x$enable_events" = "xyes"; then
AC_MSG_ERROR([python 2.x packages required. exiting..])
fi
AC_MSG_WARN([python 2.x not found, disabling events])
EVENTS_SUBDIR=
EVENTS_ENABLED=0
BUILD_EVENTS="no"
else
AC_DEFINE(USE_EVENTS, 1, [define if events enabled])
fi
fi
AC_SUBST(EVENTS_ENABLED)
AC_SUBST(EVENTS_SUBDIR)
AM_CONDITIONAL([BUILD_EVENTS], [test x$BUILD_EVENTS = xyes])
# end Events section
# CDC xlator - check if libz is present if so enable HAVE_LIB_Z
BUILD_CDC=yes
PKG_CHECK_MODULES([ZLIB], [zlib >= 1.2.0],,
[AC_CHECK_LIB([z], [deflate], [ZLIB_LIBS="-lz"],
[BUILD_CDC=no])])
echo -n "features requiring zlib enabled: "
if test "x$BUILD_CDC" = "xyes" ; then
echo "yes"
AC_DEFINE(HAVE_LIB_Z, 1, [define if zlib is present])
else
echo "no"
fi
AC_SUBST(ZLIB_CFLAGS)
AC_SUBST(ZLIB_LIBS)
# end CDC xlator secion
#start firewalld section
BUILD_FIREWALLD="no"
AC_ARG_ENABLE([firewalld],
AC_HELP_STRING([--enable-firewalld],
[enable installation configuration for firewalld]),
[BUILD_FIREWALLD="${enableval}"], [BUILD_FIREWALLD="no"])
if test "x${BUILD_FIREWALLD}" = "xyes"; then
if !(test -d /usr/lib/firewalld/services 1>/dev/null 2>&1) ; then
BUILD_FIREWALLD="no (firewalld not installed)"
fi
fi
AM_CONDITIONAL([USE_FIREWALLD],test ["x${BUILD_FIREWALLD}" = "xyes"])
#endof firewald section
Adding Libgfdb to GlusterFS ************************************************************************* Libgfdb | ************************************************************************* Libgfdb provides abstract mechanism to record extra/rich metadata required for data maintenance, such as data tiering/classification. It provides consumer with API for recording and querying, keeping the consumer abstracted from the data store used beneath for storing data. It works in a plug-and-play model, where data stores can be plugged-in. Presently we have plugin for Sqlite3. In the future will provide recording and querying performance optimizer. In the current implementation the schema of metadata is fixed. Schema: ~~~~~~ GF_FILE_TB Table: ~~~~~~~~~~~~~~~~~ This table has one entry per file inode. It holds the metadata required to make decisions in data maintenance. GF_ID (Primary key) : File GFID (Universal Unique IDentifier in the namespace) W_SEC, W_MSEC : Write wind time in sec & micro-sec UW_SEC, UW_MSEC : Write un-wind time in sec & micro-sec W_READ_SEC, W_READ_MSEC : Read wind time in sec & micro-sec UW_READ_SEC, UW_READ_MSEC : Read un-wind time in sec & micro-sec WRITE_FREQ_CNTR INTEGER : Write Frequency Counter READ_FREQ_CNTR INTEGER : Read Frequency Counter GF_FLINK_TABLE: ~~~~~~~~~~~~~~ This table has all the hardlinks to a file inode. GF_ID : File GFID (Composite Primary Key)``| GF_PID : Parent Directory GFID (Composite Primary Key) |-> Primary Key FNAME : File Base Name (Composite Primary Key)__| FPATH : File Full Path (Its redundant for now, this will go) W_DEL_FLAG : This Flag is used for crash consistancy, when a link is unlinked. i.e Set to 1 during unlink wind and during unwind this record is deleted LINK_UPDATE : This Flag is used when a link is changed i.e rename. Set to 1 when rename wind and set to 0 in rename unwind Libgfdb API: ~~~~~~~~~~~ Refer libglusterfs/src/gfdb/gfdb_data_store.h Change-Id: I2e9fbab3878ce630a7f41221ef61017dc43db11f BUG: 1194753 Signed-off-by: Joseph Fernandes <josferna@redhat.com> Signed-off-by: Dan Lambright <dlambrig@redhat.com> Signed-off-by: Joseph Fernandes <josferna@redhat.com> Reviewed-on: http://review.gluster.org/9683 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2015-02-18 19:45:23 +05:30
# Data tiering requires sqlite
AC_ARG_ENABLE([tiering],
AC_HELP_STRING([--disable-tiering],
[Disable data classification/tiering]),
[BUILD_GFDB="${enableval}"], [BUILD_GFDB="yes"])
case $host_os in
darwin*)
SQLITE_LIBS="-lsqlite3"
AC_CHECK_HEADERS([sqlite3.h], AC_DEFINE(USE_GFDB, 1))
;;
*)
if test "x${BUILD_GFDB}" = "xyes"; then
PKG_CHECK_MODULES([SQLITE], [sqlite3],
AC_DEFINE(USE_GFDB, 1),
AC_MSG_ERROR([pass --disable-tiering to build without sqlite]))
else
AC_DEFINE(USE_GFDB, 0, [no sqlite, gfdb is disabled])
fi
;;
esac
Adding Libgfdb to GlusterFS ************************************************************************* Libgfdb | ************************************************************************* Libgfdb provides abstract mechanism to record extra/rich metadata required for data maintenance, such as data tiering/classification. It provides consumer with API for recording and querying, keeping the consumer abstracted from the data store used beneath for storing data. It works in a plug-and-play model, where data stores can be plugged-in. Presently we have plugin for Sqlite3. In the future will provide recording and querying performance optimizer. In the current implementation the schema of metadata is fixed. Schema: ~~~~~~ GF_FILE_TB Table: ~~~~~~~~~~~~~~~~~ This table has one entry per file inode. It holds the metadata required to make decisions in data maintenance. GF_ID (Primary key) : File GFID (Universal Unique IDentifier in the namespace) W_SEC, W_MSEC : Write wind time in sec & micro-sec UW_SEC, UW_MSEC : Write un-wind time in sec & micro-sec W_READ_SEC, W_READ_MSEC : Read wind time in sec & micro-sec UW_READ_SEC, UW_READ_MSEC : Read un-wind time in sec & micro-sec WRITE_FREQ_CNTR INTEGER : Write Frequency Counter READ_FREQ_CNTR INTEGER : Read Frequency Counter GF_FLINK_TABLE: ~~~~~~~~~~~~~~ This table has all the hardlinks to a file inode. GF_ID : File GFID (Composite Primary Key)``| GF_PID : Parent Directory GFID (Composite Primary Key) |-> Primary Key FNAME : File Base Name (Composite Primary Key)__| FPATH : File Full Path (Its redundant for now, this will go) W_DEL_FLAG : This Flag is used for crash consistancy, when a link is unlinked. i.e Set to 1 during unlink wind and during unwind this record is deleted LINK_UPDATE : This Flag is used when a link is changed i.e rename. Set to 1 when rename wind and set to 0 in rename unwind Libgfdb API: ~~~~~~~~~~~ Refer libglusterfs/src/gfdb/gfdb_data_store.h Change-Id: I2e9fbab3878ce630a7f41221ef61017dc43db11f BUG: 1194753 Signed-off-by: Joseph Fernandes <josferna@redhat.com> Signed-off-by: Dan Lambright <dlambrig@redhat.com> Signed-off-by: Joseph Fernandes <josferna@redhat.com> Reviewed-on: http://review.gluster.org/9683 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2015-02-18 19:45:23 +05:30
AC_SUBST(SQLITE_CFLAGS)
AC_SUBST(SQLITE_LIBS)
AM_CONDITIONAL(BUILD_GFDB, test "x${BUILD_GFDB}" = "xyes")
AM_CONDITIONAL(USE_GFDB, test "x${BUILD_GFDB}" = "xyes")
# xml-output
AC_ARG_ENABLE([xml-output],
AC_HELP_STRING([--disable-xml-output],
[Disable the xml output]))
BUILD_XML_OUTPUT="yes"
if test "x$enable_xml_output" != "xno"; then
#check if libxml is present if so enable HAVE_LIB_XML
m4_ifdef([AM_PATH_XML2],[AM_PATH_XML2([2.6.19])], [no_xml=yes])
if test "x${no_xml}" = "x"; then
AC_DEFINE([HAVE_LIB_XML], [1], [Define to 1 if using libxml2.])
else
Adding Libgfdb to GlusterFS ************************************************************************* Libgfdb | ************************************************************************* Libgfdb provides abstract mechanism to record extra/rich metadata required for data maintenance, such as data tiering/classification. It provides consumer with API for recording and querying, keeping the consumer abstracted from the data store used beneath for storing data. It works in a plug-and-play model, where data stores can be plugged-in. Presently we have plugin for Sqlite3. In the future will provide recording and querying performance optimizer. In the current implementation the schema of metadata is fixed. Schema: ~~~~~~ GF_FILE_TB Table: ~~~~~~~~~~~~~~~~~ This table has one entry per file inode. It holds the metadata required to make decisions in data maintenance. GF_ID (Primary key) : File GFID (Universal Unique IDentifier in the namespace) W_SEC, W_MSEC : Write wind time in sec & micro-sec UW_SEC, UW_MSEC : Write un-wind time in sec & micro-sec W_READ_SEC, W_READ_MSEC : Read wind time in sec & micro-sec UW_READ_SEC, UW_READ_MSEC : Read un-wind time in sec & micro-sec WRITE_FREQ_CNTR INTEGER : Write Frequency Counter READ_FREQ_CNTR INTEGER : Read Frequency Counter GF_FLINK_TABLE: ~~~~~~~~~~~~~~ This table has all the hardlinks to a file inode. GF_ID : File GFID (Composite Primary Key)``| GF_PID : Parent Directory GFID (Composite Primary Key) |-> Primary Key FNAME : File Base Name (Composite Primary Key)__| FPATH : File Full Path (Its redundant for now, this will go) W_DEL_FLAG : This Flag is used for crash consistancy, when a link is unlinked. i.e Set to 1 during unlink wind and during unwind this record is deleted LINK_UPDATE : This Flag is used when a link is changed i.e rename. Set to 1 when rename wind and set to 0 in rename unwind Libgfdb API: ~~~~~~~~~~~ Refer libglusterfs/src/gfdb/gfdb_data_store.h Change-Id: I2e9fbab3878ce630a7f41221ef61017dc43db11f BUG: 1194753 Signed-off-by: Joseph Fernandes <josferna@redhat.com> Signed-off-by: Dan Lambright <dlambrig@redhat.com> Signed-off-by: Joseph Fernandes <josferna@redhat.com> Reviewed-on: http://review.gluster.org/9683 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2015-02-18 19:45:23 +05:30
if test "x$enable_georeplication" != "xno"; then
AC_MSG_ERROR([libxml2 devel libraries not found])
Adding Libgfdb to GlusterFS ************************************************************************* Libgfdb | ************************************************************************* Libgfdb provides abstract mechanism to record extra/rich metadata required for data maintenance, such as data tiering/classification. It provides consumer with API for recording and querying, keeping the consumer abstracted from the data store used beneath for storing data. It works in a plug-and-play model, where data stores can be plugged-in. Presently we have plugin for Sqlite3. In the future will provide recording and querying performance optimizer. In the current implementation the schema of metadata is fixed. Schema: ~~~~~~ GF_FILE_TB Table: ~~~~~~~~~~~~~~~~~ This table has one entry per file inode. It holds the metadata required to make decisions in data maintenance. GF_ID (Primary key) : File GFID (Universal Unique IDentifier in the namespace) W_SEC, W_MSEC : Write wind time in sec & micro-sec UW_SEC, UW_MSEC : Write un-wind time in sec & micro-sec W_READ_SEC, W_READ_MSEC : Read wind time in sec & micro-sec UW_READ_SEC, UW_READ_MSEC : Read un-wind time in sec & micro-sec WRITE_FREQ_CNTR INTEGER : Write Frequency Counter READ_FREQ_CNTR INTEGER : Read Frequency Counter GF_FLINK_TABLE: ~~~~~~~~~~~~~~ This table has all the hardlinks to a file inode. GF_ID : File GFID (Composite Primary Key)``| GF_PID : Parent Directory GFID (Composite Primary Key) |-> Primary Key FNAME : File Base Name (Composite Primary Key)__| FPATH : File Full Path (Its redundant for now, this will go) W_DEL_FLAG : This Flag is used for crash consistancy, when a link is unlinked. i.e Set to 1 during unlink wind and during unwind this record is deleted LINK_UPDATE : This Flag is used when a link is changed i.e rename. Set to 1 when rename wind and set to 0 in rename unwind Libgfdb API: ~~~~~~~~~~~ Refer libglusterfs/src/gfdb/gfdb_data_store.h Change-Id: I2e9fbab3878ce630a7f41221ef61017dc43db11f BUG: 1194753 Signed-off-by: Joseph Fernandes <josferna@redhat.com> Signed-off-by: Dan Lambright <dlambrig@redhat.com> Signed-off-by: Joseph Fernandes <josferna@redhat.com> Reviewed-on: http://review.gluster.org/9683 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2015-02-18 19:45:23 +05:30
else
AC_MSG_WARN([libxml2 devel libraries not found disabling XML support])
BUILD_XML_OUTPUT="no"
Adding Libgfdb to GlusterFS ************************************************************************* Libgfdb | ************************************************************************* Libgfdb provides abstract mechanism to record extra/rich metadata required for data maintenance, such as data tiering/classification. It provides consumer with API for recording and querying, keeping the consumer abstracted from the data store used beneath for storing data. It works in a plug-and-play model, where data stores can be plugged-in. Presently we have plugin for Sqlite3. In the future will provide recording and querying performance optimizer. In the current implementation the schema of metadata is fixed. Schema: ~~~~~~ GF_FILE_TB Table: ~~~~~~~~~~~~~~~~~ This table has one entry per file inode. It holds the metadata required to make decisions in data maintenance. GF_ID (Primary key) : File GFID (Universal Unique IDentifier in the namespace) W_SEC, W_MSEC : Write wind time in sec & micro-sec UW_SEC, UW_MSEC : Write un-wind time in sec & micro-sec W_READ_SEC, W_READ_MSEC : Read wind time in sec & micro-sec UW_READ_SEC, UW_READ_MSEC : Read un-wind time in sec & micro-sec WRITE_FREQ_CNTR INTEGER : Write Frequency Counter READ_FREQ_CNTR INTEGER : Read Frequency Counter GF_FLINK_TABLE: ~~~~~~~~~~~~~~ This table has all the hardlinks to a file inode. GF_ID : File GFID (Composite Primary Key)``| GF_PID : Parent Directory GFID (Composite Primary Key) |-> Primary Key FNAME : File Base Name (Composite Primary Key)__| FPATH : File Full Path (Its redundant for now, this will go) W_DEL_FLAG : This Flag is used for crash consistancy, when a link is unlinked. i.e Set to 1 during unlink wind and during unwind this record is deleted LINK_UPDATE : This Flag is used when a link is changed i.e rename. Set to 1 when rename wind and set to 0 in rename unwind Libgfdb API: ~~~~~~~~~~~ Refer libglusterfs/src/gfdb/gfdb_data_store.h Change-Id: I2e9fbab3878ce630a7f41221ef61017dc43db11f BUG: 1194753 Signed-off-by: Joseph Fernandes <josferna@redhat.com> Signed-off-by: Dan Lambright <dlambrig@redhat.com> Signed-off-by: Joseph Fernandes <josferna@redhat.com> Reviewed-on: http://review.gluster.org/9683 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2015-02-18 19:45:23 +05:30
fi
fi
else
if test "x$enable_georeplication" != "xno"; then
AC_MSG_ERROR([geo-replication requires xml output])
fi
BUILD_XML_OUTPUT="no"
2013-03-25 08:22:16 -04:00
fi
# end of xml-output
dnl SELinux feature enablement
case $host_os in
linux*)
AC_ARG_ENABLE([selinux],
AC_HELP_STRING([--disable-selinux],
[Disable SELinux features]),
[USE_SELINUX="${enableval}"], [USE_SELINUX="yes"])
;;
*)
USE_SELINUX=no
;;
esac
AM_CONDITIONAL(USE_SELINUX, test "x${USE_SELINUX}" = "xyes")
dnl end of SELinux feature enablement
AC_CHECK_HEADERS([execinfo.h], [have_backtrace=yes])
2009-02-18 17:36:07 +05:30
if test "x${have_backtrace}" = "xyes"; then
AC_DEFINE(HAVE_BACKTRACE, 1, [define if found backtrace])
fi
AC_SUBST(HAVE_BACKTRACE)
if test "x${have_backtrace}" != "xyes"; then
AC_TRY_COMPILE([#include <math.h>], [double x=0.0; x=ceil(0.0);],
[],
AC_MSG_ERROR([need math library for libexecinfo]))
fi
2009-02-18 17:36:07 +05:30
dnl glusterfs prints memory usage to stderr by sending it SIGUSR1
AC_CHECK_FUNC([malloc_stats], [have_malloc_stats=yes])
if test "x${have_malloc_stats}" = "xyes"; then
AC_DEFINE(HAVE_MALLOC_STATS, 1, [define if found malloc_stats])
fi
AC_SUBST(HAVE_MALLOC_STATS)
dnl Linux, Solaris, Cygwin
AC_CHECK_MEMBERS([struct stat.st_atim.tv_nsec])
dnl FreeBSD, NetBSD
AC_CHECK_MEMBERS([struct stat.st_atimespec.tv_nsec])
case $host_os in
*netbsd*)
CFLAGS="${CFLAGS} -D_INCOMPLETE_XOPEN_C063 -DCONFIG_MACHINE_BSWAP_H"
;;
esac
AC_CHECK_FUNC([linkat], [have_linkat=yes])
if test "x${have_linkat}" = "xyes"; then
AC_DEFINE(HAVE_LINKAT, 1, [define if found linkat])
fi
AC_SUBST(HAVE_LINKAT)
2009-02-18 17:36:07 +05:30
libglusterfs: Add monotonic clocking counter for timer thread gettimeofday() returns the current wall clock time and timezone. Using these functions in order to measure the passage of time (how long an operation took) therefore seems like a no-brainer. This time suffer's from some limitations: a. They have a low resolution: “High-performance” timing by definition, requires clock resolutions into the microseconds or better. b. They can jump forwards and backwards in time: Computer clocks all tick at slightly different rates, which causes the time to drift. Most systems have NTP enabled which periodically adjusts the system clock to keep them in sync with “actual” time. The adjustment can cause the clock to suddenly jump forward (artificially inflating your timing numbers) or jump backwards (causing your timing calculations to go negative or hugely positive). In such cases timer thread could go into an infinite loop. From 'man gettimeofday': ---------- .. .. The time returned by gettimeofday() is affected by discontinuous jumps in the system time (e.g., if the system administrator manually changes the system time). If you need a monotonically increasing clock, see clock_gettime(2). .. .. ---------- Rationale: For calculating interval timing for Timer thread, all that’s needed should be clock as a simple counter that increments at a stable rate. This is necessary to avoid the jumps which are caused by using "wall time", this counter must be monotonic that can never “tick” backwards, ever. Change-Id: I701d31e71a85a73d21a6c5cd15583e7a5a645eeb BUG: 1017993 Signed-off-by: Harshavardhana <harsha@harshavardhana.net> Reviewed-on: http://review.gluster.org/6070 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Anand Avati <avati@redhat.com>
2013-10-10 04:19:16 -07:00
dnl check for Monotonic clock
AC_CHECK_LIB([rt], [clock_gettime], ,
AC_MSG_WARN([System doesn't have monotonic clock using contrib]))
libglusterfs: Add monotonic clocking counter for timer thread gettimeofday() returns the current wall clock time and timezone. Using these functions in order to measure the passage of time (how long an operation took) therefore seems like a no-brainer. This time suffer's from some limitations: a. They have a low resolution: “High-performance” timing by definition, requires clock resolutions into the microseconds or better. b. They can jump forwards and backwards in time: Computer clocks all tick at slightly different rates, which causes the time to drift. Most systems have NTP enabled which periodically adjusts the system clock to keep them in sync with “actual” time. The adjustment can cause the clock to suddenly jump forward (artificially inflating your timing numbers) or jump backwards (causing your timing calculations to go negative or hugely positive). In such cases timer thread could go into an infinite loop. From 'man gettimeofday': ---------- .. .. The time returned by gettimeofday() is affected by discontinuous jumps in the system time (e.g., if the system administrator manually changes the system time). If you need a monotonically increasing clock, see clock_gettime(2). .. .. ---------- Rationale: For calculating interval timing for Timer thread, all that’s needed should be clock as a simple counter that increments at a stable rate. This is necessary to avoid the jumps which are caused by using "wall time", this counter must be monotonic that can never “tick” backwards, ever. Change-Id: I701d31e71a85a73d21a6c5cd15583e7a5a645eeb BUG: 1017993 Signed-off-by: Harshavardhana <harsha@harshavardhana.net> Reviewed-on: http://review.gluster.org/6070 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Anand Avati <avati@redhat.com>
2013-10-10 04:19:16 -07:00
2009-02-18 17:36:07 +05:30
dnl Check for argp
AC_CHECK_HEADER([argp.h], AC_DEFINE(HAVE_ARGP, 1, [have argp]))
libglusterfs: Add monotonic clocking counter for timer thread gettimeofday() returns the current wall clock time and timezone. Using these functions in order to measure the passage of time (how long an operation took) therefore seems like a no-brainer. This time suffer's from some limitations: a. They have a low resolution: “High-performance” timing by definition, requires clock resolutions into the microseconds or better. b. They can jump forwards and backwards in time: Computer clocks all tick at slightly different rates, which causes the time to drift. Most systems have NTP enabled which periodically adjusts the system clock to keep them in sync with “actual” time. The adjustment can cause the clock to suddenly jump forward (artificially inflating your timing numbers) or jump backwards (causing your timing calculations to go negative or hugely positive). In such cases timer thread could go into an infinite loop. From 'man gettimeofday': ---------- .. .. The time returned by gettimeofday() is affected by discontinuous jumps in the system time (e.g., if the system administrator manually changes the system time). If you need a monotonically increasing clock, see clock_gettime(2). .. .. ---------- Rationale: For calculating interval timing for Timer thread, all that’s needed should be clock as a simple counter that increments at a stable rate. This is necessary to avoid the jumps which are caused by using "wall time", this counter must be monotonic that can never “tick” backwards, ever. Change-Id: I701d31e71a85a73d21a6c5cd15583e7a5a645eeb BUG: 1017993 Signed-off-by: Harshavardhana <harsha@harshavardhana.net> Reviewed-on: http://review.gluster.org/6070 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Anand Avati <avati@redhat.com>
2013-10-10 04:19:16 -07:00
2009-02-18 17:36:07 +05:30
BUILD_ARGP_STANDALONE=no
if test "x${ac_cv_header_argp_h}" = "xno"; then
AC_CONFIG_SUBDIRS(contrib/argp-standalone)
2009-02-18 17:36:07 +05:30
BUILD_ARGP_STANDALONE=yes
ARGP_STANDALONE_CPPFLAGS='-I${top_srcdir}/contrib/argp-standalone'
ARGP_STANDALONE_LDADD='${top_builddir}/contrib/argp-standalone/libargp.a'
ARGP_STANDALONE_DIR='${top_builddir}/contrib/argp-standalone'
2009-02-18 17:36:07 +05:30
fi
dnl libglusterfs needs argp.h, practically everything depends on it
GF_CPPFLAGS="${GF_CPPFLAGS} ${ARGP_STANDALONE_CPPFLAGS}"
2009-02-18 17:36:07 +05:30
AC_SUBST(ARGP_STANDALONE_CPPFLAGS)
AC_SUBST(ARGP_STANDALONE_LDADD)
AC_SUBST(ARGP_STANDALONE_DIR)
2009-02-18 17:36:07 +05:30
dnl Check for atomic operation support
AC_MSG_CHECKING([for gcc __atomic builtins])
AC_TRY_LINK([], [int v; __atomic_load_n(&v, __ATOMIC_ACQUIRE);],
[have_atomic_builtins=yes], [have_atomic_builtins=no])
if test "x${have_atomic_builtins}" = "xyes"; then
AC_DEFINE(HAVE_ATOMIC_BUILTINS, 1, [define if __atomic_*() builtins are available])
fi
AC_SUBST(HAVE_ATOMIC_BUILTINS)
AC_MSG_RESULT([$have_atomic_builtins])
dnl __sync_*() will not be needed if __atomic_*() is available
AC_MSG_CHECKING([for gcc __sync builtins])
AC_TRY_LINK([], [__sync_synchronize();],
[have_sync_builtins=yes], [have_sync_builtins=no])
if test "x${have_sync_builtins}" = "xyes"; then
AC_DEFINE(HAVE_SYNC_BUILTINS, 1, [define if __sync_*() builtins are available])
fi
AC_SUBST(HAVE_SYNC_BUILTINS)
AC_MSG_RESULT([$have_sync_builtins])
2009-02-18 17:36:07 +05:30
AC_CHECK_HEADER([malloc.h], AC_DEFINE(HAVE_MALLOC_H, 1, [have malloc.h]))
AC_CHECK_FUNC([llistxattr], [have_llistxattr=yes])
if test "x${have_llistxattr}" = "xyes"; then
AC_DEFINE(HAVE_LLISTXATTR, 1, [define if llistxattr exists])
fi
AC_CHECK_FUNC([fdatasync], [have_fdatasync=no])
2009-02-18 17:36:07 +05:30
if test "x${have_fdatasync}" = "xyes"; then
AC_DEFINE(HAVE_FDATASYNC, 1, [define if fdatasync exists])
fi
AC_CHECK_FUNC([fallocate], [have_fallocate=yes])
if test "x${have_fallocate}" = "xyes"; then
AC_DEFINE(HAVE_FALLOCATE, 1, [define if fallocate exists])
fi
AC_CHECK_FUNC([posix_fallocate], [have_posix_fallocate=yes])
if test "x${have_posix_fallocate}" = "xyes"; then
AC_DEFINE(HAVE_POSIX_FALLOCATE, 1, [define if posix_fallocate exists])
fi
BUILD_NANOSECOND_TIMESTAMPS=no
AC_CHECK_FUNC([utimensat], [have_utimensat=yes])
if test "x${have_utimensat}" = "xyes"; then
BUILD_NANOSECOND_TIMESTAMPS=yes
AC_DEFINE(HAVE_UTIMENSAT, 1, [define if utimensat exists])
fi
OLD_CFLAGS=${CFLAGS}
CFLAGS="-D_GNU_SOURCE"
AC_CHECK_DECL([SEEK_HOLE], , , [#include <unistd.h>])
if test "x${ac_cv_have_decl_SEEK_HOLE}" = "xyes"; then
AC_DEFINE(HAVE_SEEK_HOLE, 1, [define if SEEK_HOLE is available])
fi
CFLAGS=${OLD_CFLAGS}
# Check the distribution where you are compiling glusterfs on
GF_DISTRIBUTION=
AC_CHECK_FILE([/etc/debian_version])
AC_CHECK_FILE([/etc/SuSE-release])
AC_CHECK_FILE([/etc/redhat-release])
if test "x$ac_cv_file__etc_debian_version" = "xyes"; then
GF_DISTRIBUTION=Debian
fi
if test "x$ac_cv_file__etc_SuSE_release" = "xyes"; then
GF_DISTRIBUTION=SuSE
fi
if test "x$ac_cv_file__etc_redhat_release" = "xyes"; then
GF_DISTRIBUTION=Redhat
fi
AC_SUBST(GF_DISTRIBUTION)
2009-02-18 17:36:07 +05:30
GF_HOST_OS=""
GF_LDFLAGS="-rdynamic"
DISABLE_LIBTIRPC=no
dnl include tirpc for IPv6 builds
if test "x$with_libtirpc" = "xyes" || test "x$with_ipv6_default" = "xyes" ; then
PKG_CHECK_MODULES([TIRPC], [libtirpc],
[GF_CFLAGS="$GF_CFLAGS $TIRPC_CFLAGS"; GF_LDFLAGS="$GF_LDFLAGS $TIRPC_LIBS";],
[DISABLE_LIBTIRPC=yes])
fi
if test "x$DISABLE_LIBTIRPC" = "xyes" ; then
with_libtirpc=no; with_ipv6_default=no
AC_CHECK_HEADERS([rpc/rpc.h],[
AC_MSG_WARN([
---------------------------------------------------------------------------------
libtirpc or ipv6-default were enabled (by default) but libtirpc-devel is not
installed. Disabling libtirpc and ipv6-default and using legacy glibc rpc headers
This is a transistional warning message. Eventually it will be an error message
---------------------------------------------------------------------------------])],[
AC_MSG_ERROR([
---------------------------------------------------------------------------------
libtirpc or ipv6-default were enabled (by default) but libtirpc-devel is not
installed and there were no legacy glibc rpc headers and library to fall back to.
---------------------------------------------------------------------------------])])
fi
if test "x$with_ipv6_default" = "xyes" ; then
GF_CFLAGS="$GF_CFLAGS -DIPV6_DEFAULT"
fi
dnl check for gcc -Werror=format-security
saved_CFLAGS=$CFLAGS
CFLAGS="-Wformat -Werror=format-security"
AC_MSG_CHECKING([whether $CC accepts -Werror=format-security])
AC_COMPILE_IFELSE([AC_LANG_PROGRAM()], [cc_werror_format_security=yes], [cc_werror_format_security=no])
echo $cc_werror_format_security
if test "x$cc_werror_format_security" = "xno"; then
CFLAGS="$saved_CFLAGS"
else
CFLAGS="$saved_CFLAGS $CFLAGS"
GF_CFLAGS="$GF_CFLAGS $CFLAGS"
fi
dnl check for gcc -Werror=implicit-function-declaration
saved_CFLAGS=$CFLAGS
saved_GF_CFLAGS=$GF_CFLAGS
CFLAGS="-Werror=implicit-function-declaration"
GF_CFLAGS="-Werror=implicit-function-declaration"
AC_MSG_CHECKING([whether $CC accepts -Werror=implicit-function-declaration])
AC_COMPILE_IFELSE([AC_LANG_PROGRAM()], [cc_werror_implicit=yes], [cc_werror_implicit=no])
echo $cc_werror_implicit
if test "x$cc_werror_implicit" = "xno"; then
CFLAGS="$saved_CFLAGS"
GF_CFLAGS="$saved_GF_CFLAGS"
else
CFLAGS="$saved_CFLAGS $CFLAGS"
GF_CFLAGS="$saved_GF_CFLAGS $GF_CFLAGS"
fi
dnl clang is mostly GCC-compatible, but its version is much lower,
dnl so we have to check for it.
AC_MSG_CHECKING([if compiling with clang])
AC_COMPILE_IFELSE(
[AC_LANG_PROGRAM([], [[
#ifndef __clang__
not clang
#endif
]])],
[CLANG=yes], [CLANG=no])
AC_MSG_RESULT([$CLANG])
if test "x$CLANG" = "xyes"; then
GF_CFLAGS="${GF_CFLAGS} -Wno-gnu"
fi
if test "x$ac_cv_header_execinfo_h" = "xno"; then
# The reason is that __builtin_frame_address(n) for n > 0 seems
# to just crash on most platforms when -fomit-stack-pointer is
# specified, which seems to be the default for many platforms on
# -O2. The documentation says that __builtin_frame_address()
# should return NULL in case it can't get the frame, but it
# seems to crash instead.
# execinfo.c in ./contrib/libexecinfo uses __builtin_frame_address(n)
# for providing cross platform backtrace*() functions.
if test "x$CLANG" = "xno"; then
GF_CFLAGS="${GF_CFLAGS} -fno-omit-frame-pointer"
fi
fi
feature/glusterfind: A tool to find incremental changes Documentation is available in patch: http://review.gluster.org/#/c/9800/ A tool which helps to get list of modified files or list of all files in GlusterFS Volume using Changelog or find command. Usage ===== glusterfind --help Create: ------- glusterfind create --help The tool creates status file $GLUSTERD_WORKDIR/SESSION/VOLUME/status and records current timestamp to initiate the session. This timestamp will be used as start time for next runs. As part of create also generates ssh key and distributes to all peers. and enables build.pgfid and changelog using volume set command. Pre: ---- glusterfind pre --help This command is used to generate the list of files modified after session creation time or after last run. To get list of all files/dirs in Volume, run pre command with `--full` argument. The tool gets all nodes details using gluster volume info and runs node agent for each brick in respective nodes via ssh command. Once these node agents generate the output file, tool copies to local using scp. Merges all the output files to generate the final output file. Post: ----- glusterfind post --help After consuming the list, this sub command is called to update the session time based on pre command status file. List: ----- glusterfind list --help To view all the sessions Delete: ------- glusterfind delete --help Delete session. Known Issues ------------ 1. Deleted files will not get listed, since we can't convert GFID to Path if file/dir is deleted. 2. Only new name will get listed if Renamed. 3. All hardlinks will get listed. Change-Id: I82991feb0aea85cb6ec035fddbf80a2b276e86b0 BUG: 1193893 Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/9682 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Venky Shankar <vshankar@redhat.com> Reviewed-by: Prashanth Pai <ppai@redhat.com> Reviewed-by: Kotresh HR <khiremat@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2015-02-18 19:07:23 +05:30
old_prefix=$prefix
if test "x$prefix" = xNONE; then
prefix=$ac_default_prefix
feature/glusterfind: A tool to find incremental changes Documentation is available in patch: http://review.gluster.org/#/c/9800/ A tool which helps to get list of modified files or list of all files in GlusterFS Volume using Changelog or find command. Usage ===== glusterfind --help Create: ------- glusterfind create --help The tool creates status file $GLUSTERD_WORKDIR/SESSION/VOLUME/status and records current timestamp to initiate the session. This timestamp will be used as start time for next runs. As part of create also generates ssh key and distributes to all peers. and enables build.pgfid and changelog using volume set command. Pre: ---- glusterfind pre --help This command is used to generate the list of files modified after session creation time or after last run. To get list of all files/dirs in Volume, run pre command with `--full` argument. The tool gets all nodes details using gluster volume info and runs node agent for each brick in respective nodes via ssh command. Once these node agents generate the output file, tool copies to local using scp. Merges all the output files to generate the final output file. Post: ----- glusterfind post --help After consuming the list, this sub command is called to update the session time based on pre command status file. List: ----- glusterfind list --help To view all the sessions Delete: ------- glusterfind delete --help Delete session. Known Issues ------------ 1. Deleted files will not get listed, since we can't convert GFID to Path if file/dir is deleted. 2. Only new name will get listed if Renamed. 3. All hardlinks will get listed. Change-Id: I82991feb0aea85cb6ec035fddbf80a2b276e86b0 BUG: 1193893 Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/9682 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Venky Shankar <vshankar@redhat.com> Reviewed-by: Prashanth Pai <ppai@redhat.com> Reviewed-by: Kotresh HR <khiremat@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2015-02-18 19:07:23 +05:30
fi
old_exec_prefix=$exec_prefix
if test "x$exec_prefix" = xNONE; then
exec_prefix="$(eval echo $prefix)"
fi
GLUSTERFS_LIBEXECDIR="$(eval echo $libexecdir)/glusterfs"
GLUSTERFSD_MISCDIR="$(eval echo $prefix)/var/lib/misc/glusterfsd"
feature/glusterfind: A tool to find incremental changes Documentation is available in patch: http://review.gluster.org/#/c/9800/ A tool which helps to get list of modified files or list of all files in GlusterFS Volume using Changelog or find command. Usage ===== glusterfind --help Create: ------- glusterfind create --help The tool creates status file $GLUSTERD_WORKDIR/SESSION/VOLUME/status and records current timestamp to initiate the session. This timestamp will be used as start time for next runs. As part of create also generates ssh key and distributes to all peers. and enables build.pgfid and changelog using volume set command. Pre: ---- glusterfind pre --help This command is used to generate the list of files modified after session creation time or after last run. To get list of all files/dirs in Volume, run pre command with `--full` argument. The tool gets all nodes details using gluster volume info and runs node agent for each brick in respective nodes via ssh command. Once these node agents generate the output file, tool copies to local using scp. Merges all the output files to generate the final output file. Post: ----- glusterfind post --help After consuming the list, this sub command is called to update the session time based on pre command status file. List: ----- glusterfind list --help To view all the sessions Delete: ------- glusterfind delete --help Delete session. Known Issues ------------ 1. Deleted files will not get listed, since we can't convert GFID to Path if file/dir is deleted. 2. Only new name will get listed if Renamed. 3. All hardlinks will get listed. Change-Id: I82991feb0aea85cb6ec035fddbf80a2b276e86b0 BUG: 1193893 Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/9682 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Venky Shankar <vshankar@redhat.com> Reviewed-by: Prashanth Pai <ppai@redhat.com> Reviewed-by: Kotresh HR <khiremat@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2015-02-18 19:07:23 +05:30
prefix=$old_prefix
exec_prefix=$old_exec_prefix
feature/glusterfind: A tool to find incremental changes Documentation is available in patch: http://review.gluster.org/#/c/9800/ A tool which helps to get list of modified files or list of all files in GlusterFS Volume using Changelog or find command. Usage ===== glusterfind --help Create: ------- glusterfind create --help The tool creates status file $GLUSTERD_WORKDIR/SESSION/VOLUME/status and records current timestamp to initiate the session. This timestamp will be used as start time for next runs. As part of create also generates ssh key and distributes to all peers. and enables build.pgfid and changelog using volume set command. Pre: ---- glusterfind pre --help This command is used to generate the list of files modified after session creation time or after last run. To get list of all files/dirs in Volume, run pre command with `--full` argument. The tool gets all nodes details using gluster volume info and runs node agent for each brick in respective nodes via ssh command. Once these node agents generate the output file, tool copies to local using scp. Merges all the output files to generate the final output file. Post: ----- glusterfind post --help After consuming the list, this sub command is called to update the session time based on pre command status file. List: ----- glusterfind list --help To view all the sessions Delete: ------- glusterfind delete --help Delete session. Known Issues ------------ 1. Deleted files will not get listed, since we can't convert GFID to Path if file/dir is deleted. 2. Only new name will get listed if Renamed. 3. All hardlinks will get listed. Change-Id: I82991feb0aea85cb6ec035fddbf80a2b276e86b0 BUG: 1193893 Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/9682 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Venky Shankar <vshankar@redhat.com> Reviewed-by: Prashanth Pai <ppai@redhat.com> Reviewed-by: Kotresh HR <khiremat@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2015-02-18 19:07:23 +05:30
### Dirty hacky stuff to make LOCALSTATEDIR work
if test "x$prefix" = xNONE; then
test $localstatedir = '${prefix}/var' && localstatedir=$ac_default_prefix/var
localstatedir=/var
fi
localstatedir="$(eval echo ${localstatedir})"
LOCALSTATEDIR=$localstatedir
geo-rep: mountbroker user management Non root geo-replication setup is now simplified. This patch provides cli for mountbroker user and options management To set Options, gluster system:: execute mountbroker opt <KEY> <VALUE> # for example, gluster system:: execute mountbroker opt mountbroker-root /var/mountbroker-root gluster system:: execute mountbroker opt geo-replication-log-group geogroup gluster system:: execute mountbroker opt rpc-auth-allow-insecure on To remove option, gluster system:: execute mountbroker optdel <KEY> # for example, gluster system:: execute mountbroker optdel geo-replication-log-group To add/edit user, gluster system:: execute mountbroker user <USERNAME> <VOLUMES> # for example gluster system:: execute mountbroker user geoaccount slavevol1,slavevol2 To remove user, gluster system:: execute mountbroker userdel <USERNAME> # for example gluster system:: execute mountbroker userdel geoaccount For info, gluster system:: execute mountbroker info gluster system:: execute mountbroker -j info For JSON output add -j after mountbroker, for example, gluster system:: execute mountbroker -j user geoaccount slavevol1,slavevol2 PS: Each peer prints its own JSON output, aggregator required from consumer side BUG: 1136312 Change-Id: Ie52210c0bcc91ac2ffd3ba58988222ffca62b47f Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/9398 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: darshan n <dnarayan@redhat.com> Reviewed-by: Kotresh HR <khiremat@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2015-01-06 18:20:45 +05:30
old_prefix=$prefix
if test "x$prefix" = xNONE; then
prefix=$ac_default_prefix
fi
GLUSTERD_VOLFILE="$(eval echo ${sysconfdir})/glusterfs/glusterd.vol"
geo-rep: mountbroker user management Non root geo-replication setup is now simplified. This patch provides cli for mountbroker user and options management To set Options, gluster system:: execute mountbroker opt <KEY> <VALUE> # for example, gluster system:: execute mountbroker opt mountbroker-root /var/mountbroker-root gluster system:: execute mountbroker opt geo-replication-log-group geogroup gluster system:: execute mountbroker opt rpc-auth-allow-insecure on To remove option, gluster system:: execute mountbroker optdel <KEY> # for example, gluster system:: execute mountbroker optdel geo-replication-log-group To add/edit user, gluster system:: execute mountbroker user <USERNAME> <VOLUMES> # for example gluster system:: execute mountbroker user geoaccount slavevol1,slavevol2 To remove user, gluster system:: execute mountbroker userdel <USERNAME> # for example gluster system:: execute mountbroker userdel geoaccount For info, gluster system:: execute mountbroker info gluster system:: execute mountbroker -j info For JSON output add -j after mountbroker, for example, gluster system:: execute mountbroker -j user geoaccount slavevol1,slavevol2 PS: Each peer prints its own JSON output, aggregator required from consumer side BUG: 1136312 Change-Id: Ie52210c0bcc91ac2ffd3ba58988222ffca62b47f Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/9398 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: darshan n <dnarayan@redhat.com> Reviewed-by: Kotresh HR <khiremat@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2015-01-06 18:20:45 +05:30
prefix=$old_prefix
2009-02-18 17:36:07 +05:30
case $host_os in
linux*)
GF_HOST_OS="GF_LINUX_HOST_OS"
GF_FUSE_CFLAGS="-DFUSERMOUNT_DIR=\\\"\$(bindir)\\\""
GLUSTERD_WORKDIR="${LOCALSTATEDIR}/lib/glusterd"
;;
2009-02-18 17:36:07 +05:30
solaris*)
GF_HOST_OS="GF_SOLARIS_HOST_OS"
GF_CFLAGS="${GF_CFLAGS} -D_REENTRANT -D_POSIX_PTHREAD_SEMANTICS -m64"
BUILD_FUSE_CLIENT=no
FUSE_CLIENT_SUBDIR=""
GLUSTERD_WORKDIR="${LOCALSTATEDIR}/lib/glusterd"
;;
*netbsd*)
GF_HOST_OS="GF_BSD_HOST_OS"
GF_CFLAGS="${GF_CFLAGS} -D_INCOMPLETE_XOPEN_C063"
GF_CFLAGS="${GF_CFLAGS} -DTHREAD_UNSAFE_BASENAME"
GF_CFLAGS="${GF_CFLAGS} -DTHREAD_UNSAFE_DIRNAME"
GF_FUSE_CFLAGS="-DFUSERMOUNT_DIR=\\\"\$(sbindir)\\\""
GF_LDADD="${ARGP_STANDALONE_LDADD}"
if test "x$ac_cv_header_execinfo_h" = "xyes"; then
GF_LDFLAGS="-lexecinfo"
fi
GF_FUSE_LDADD="-lperfuse"
BUILD_FUSE_CLIENT=yes
LEXLIB=""
BUILD_FUSERMOUNT=no
FUSERMOUNT_SUBDIR=""
GLUSTERD_WORKDIR="${LOCALSTATEDIR}/db/glusterd"
;;
*freebsd*)
2009-02-18 17:36:07 +05:30
GF_HOST_OS="GF_BSD_HOST_OS"
GF_CFLAGS="${GF_CFLAGS} ${ARGP_STANDALONE_CPPFLAGS} -O0"
GF_CFLAGS="${GF_CFLAGS} -DTHREAD_UNSAFE_BASENAME"
GF_CFLAGS="${GF_CFLAGS} -DTHREAD_UNSAFE_DIRNAME"
GF_CFLAGS="${GF_CFLAGS} -D_LIBGEN_H_"
GF_CFLAGS="${GF_CFLAGS} -DO_DSYNC=0"
GF_CFLAGS="${GF_CFLAGS} -Dxdr_quad_t=xdr_longlong_t"
GF_CFLAGS="${GF_CFLAGS} -Dxdr_u_quad_t=xdr_u_longlong_t"
GF_FUSE_CFLAGS="-DFUSERMOUNT_DIR=\\\"\$(sbindir)\\\""
GF_LDADD="${ARGP_STANDALONE_LDADD}"
if test "x$ac_cv_header_execinfo_h" = "xyes"; then
GF_LDFLAGS="-lexecinfo"
fi
BUILD_FUSE_CLIENT=yes
BUILD_FUSERMOUNT=no
FUSERMOUNT_SUBDIR=""
GLUSTERD_WORKDIR="${LOCALSTATEDIR}/db/glusterd"
;;
2009-02-18 17:36:07 +05:30
darwin*)
GF_HOST_OS="GF_DARWIN_HOST_OS"
LIBTOOL=glibtool
GF_CFLAGS="${GF_CFLAGS} ${ARGP_STANDALONE_CPPFLAGS} "
GF_CFLAGS="${GF_CFLAGS} -D_REENTRANT -D_XOPEN_SOURCE "
GF_CFLAGS="${GF_CFLAGS} -D_DARWIN_USE_64_BIT_INODE "
GF_CFLAGS="${GF_CFLAGS} -DTHREAD_UNSAFE_BASENAME"
GF_CFLAGS="${GF_CFLAGS} -DTHREAD_UNSAFE_DIRNAME"
GF_LDADD="${ARGP_STANDALONE_LDADD}"
GF_LDFLAGS=""
GF_FUSE_CFLAGS="-I\$(CONTRIBDIR)/macfuse"
BUILD_FUSERMOUNT="no"
FUSERMOUNT_SUBDIR=""
GLUSTERD_WORKDIR="${LOCALSTATEDIR}/db/glusterd"
;;
2009-02-18 17:36:07 +05:30
esac
api: versioned symbols in libgfapi.so for compatibility Use versioned symbols to keep libgfapi at libgfapi.so.0.0.0 Revisited to address broken build on Mac OS X See http://review.gluster.org/9036 Rebased to include http://review.gluster.org/#/c/9376/ (glfs_resolve()) but note that gerrit's "Rebase Change" couldn't do it. N.B. noticed that glfs_get_volumeid() decl in glfs.h was missing the __THROW, added it. On systems using ELF and the GNU toolchain, symbol versions are created with a .symver asm operand in the .c source file. Clang is claimed to be compatible with gcc, so we'll pretend for now that this also works with clang. On Mac OS X, aliases are created with __asm "magic" in the .h header file. In the normal case, when both the decl and defn match, that's all that's needed. In our case though the decl and defn don't match --- we have, e.g. a defn such as 'int glfs_foo(...)' and the corresponding decl is 'int pub_glfs_foo(...)'. To make this work we create the necessary aliases in the library at link time with the -alias_list link option. Note that this results in there being pairs of symbols in the .dylib, e.g. _pub_glfs_foo and _glfs_foo$GFAPI_3.4.0. We could use another link option, -unexported_symbols_list to elide the _pub_glfs_* symbols. (And we probably should.) Linux symbol versioning was essentially copied from Solaris; in general I would expect this to "just work" on Solaris, but until someone tries we don't really know. Change-Id: Icb96a3c2d80be7b6d7a6849bb9168f03a947f47c BUG: 1160709 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com> Reviewed-on: http://review.gluster.org/9143 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Niels de Vos <ndevos@redhat.com> Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
2014-11-18 11:08:16 -05:00
case $host_os in
darwin*)
GFAPI_EXTRA_LDFLAGS='-Wl,-alias_list,$(top_srcdir)/api/src/gfapi.aliases'
;;
*)
GFAPI_EXTRA_LDFLAGS='-Wl,--version-script=$(top_srcdir)/api/src/gfapi.map'
;;
esac
# Default value for sbindir
prefix_temp=$prefix
exec_prefix_temp=$exec_prefix
test "${prefix}" = "NONE" && prefix="${ac_default_prefix}"
test "${exec_prefix}" = "NONE" && exec_prefix='${prefix}'
sbintemp="${sbindir}"
eval sbintemp=\"${sbintemp}\"
eval sbintemp=\"${sbintemp}\"
SBIN_DIR=${sbintemp}
eventsapi: Gluster Eventing Feature implementation [Depends on http://review.gluster.org/14627] Design is available in `glusterfs-specs`, A change from the design is support of webhook instead of Websockets as discussed in the design http://review.gluster.org/13115 Since Websocket support depends on REST APIs, I will add Websocket support once REST APIs patch gets merged Usage: Run following command to start/stop Eventsapi server in all Peers, which will collect the notifications from any Gluster daemon and emits to configured client. gluster-eventsapi start|stop|restart|reload Status of running services can be checked using, gluster-eventsapi status Events listener is a HTTP(S) server which listens to events emited by the Gluster. Create a HTTP Server to listen on POST and register that URL using, gluster-eventsapi webhook-add <URL> [--bearer-token <TOKEN>] For example, if HTTP Server running in `http://192.168.122.188:9000` then add that URL using, gluster-eventsapi webhook-add http://192.168.122.188:9000 If it expects a Token then specify it using `--bearer-token` or `-t` We can also test Webhook if all peer nodes can send message or not using, gluster-eventsapi webhook-test <URL> [--bearer-token <TOKEN>] Configurations can be viewed/updated using, gluster-eventsapi config-get [--name] gluster-eventsapi config-set <NAME> <VALUE> gluster-eventsapi config-reset <NAME|all> If any one peer node was down during config-set/reset or webhook modifications, Run sync command from good node when a peer node comes back. Automatic update is not yet implemented. gluster-eventsapi sync Basic Events Client(HTTP Server) is included with the code, Start running the client with required port and start listening to the events. /usr/share/glusterfs/scripts/eventsdash.py --port 8080 Default port is 9000, if no port is specified, once it started running then configure gluster-eventsapi to send events to that client. Eventsapi Client can be outside of the Cluster, it can be run event on Windows. But only requirement is the client URL should be accessible by all peer nodes.(Or ngrok(https://ngrok.com) like tools can be used) Events implemented with this patch, - Volume Create - Volume Start - Volume Stop - Volume Delete - Peer Attach - Peer Detach It is easy to add/support more events, since it touches Gluster cmd code and to avoid merge conflicts I will add support for more events once this patch merges. BUG: 1334044 Change-Id: I316827ac9dd1443454df7deffe4f54835f7f6a08 Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/14248 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
2016-05-05 18:34:41 +05:30
sysconfdirtemp="${sysconfdir}"
eval sysconfdirtemp=\"${sysconfdirtemp}\"
SYSCONF_DIR=${sysconfdirtemp}
# Eval two times to expand fully. First eval replaces $exec_prefix into $prefix
# Second eval will expand $prefix
build_python_site_packages_temp="${BUILD_PYTHON_SITE_PACKAGES}"
eval build_python_site_packages_temp=\"${build_python_site_packages_temp}\"
eval build_python_site_packages_temp=\"${build_python_site_packages_temp}\"
BUILD_PYTHON_SITE_PACKAGES_EXPANDED=${build_python_site_packages_temp}
prefix=$prefix_temp
exec_prefix=$exec_prefix_temp
AC_SUBST(SBIN_DIR)
eventsapi: Gluster Eventing Feature implementation [Depends on http://review.gluster.org/14627] Design is available in `glusterfs-specs`, A change from the design is support of webhook instead of Websockets as discussed in the design http://review.gluster.org/13115 Since Websocket support depends on REST APIs, I will add Websocket support once REST APIs patch gets merged Usage: Run following command to start/stop Eventsapi server in all Peers, which will collect the notifications from any Gluster daemon and emits to configured client. gluster-eventsapi start|stop|restart|reload Status of running services can be checked using, gluster-eventsapi status Events listener is a HTTP(S) server which listens to events emited by the Gluster. Create a HTTP Server to listen on POST and register that URL using, gluster-eventsapi webhook-add <URL> [--bearer-token <TOKEN>] For example, if HTTP Server running in `http://192.168.122.188:9000` then add that URL using, gluster-eventsapi webhook-add http://192.168.122.188:9000 If it expects a Token then specify it using `--bearer-token` or `-t` We can also test Webhook if all peer nodes can send message or not using, gluster-eventsapi webhook-test <URL> [--bearer-token <TOKEN>] Configurations can be viewed/updated using, gluster-eventsapi config-get [--name] gluster-eventsapi config-set <NAME> <VALUE> gluster-eventsapi config-reset <NAME|all> If any one peer node was down during config-set/reset or webhook modifications, Run sync command from good node when a peer node comes back. Automatic update is not yet implemented. gluster-eventsapi sync Basic Events Client(HTTP Server) is included with the code, Start running the client with required port and start listening to the events. /usr/share/glusterfs/scripts/eventsdash.py --port 8080 Default port is 9000, if no port is specified, once it started running then configure gluster-eventsapi to send events to that client. Eventsapi Client can be outside of the Cluster, it can be run event on Windows. But only requirement is the client URL should be accessible by all peer nodes.(Or ngrok(https://ngrok.com) like tools can be used) Events implemented with this patch, - Volume Create - Volume Start - Volume Stop - Volume Delete - Peer Attach - Peer Detach It is easy to add/support more events, since it touches Gluster cmd code and to avoid merge conflicts I will add support for more events once this patch merges. BUG: 1334044 Change-Id: I316827ac9dd1443454df7deffe4f54835f7f6a08 Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/14248 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
2016-05-05 18:34:41 +05:30
AC_SUBST(SYSCONF_DIR)
AC_SUBST(BUILD_PYTHON_SITE_PACKAGES_EXPANDED)
# lazy umount emulation
UMOUNTD_SUBDIR=""
if test "x${GF_HOST_OS}" != "xGF_LINUX_HOST_OS" ; then
UMOUNTD_SUBDIR="contrib/umountd"
fi
AC_SUBST(UMOUNTD_SUBDIR)
# enable debug section
AC_ARG_ENABLE([debug],
AC_HELP_STRING([--enable-debug],
[Enable debug build options.]))
AC_ARG_ENABLE([mempool],
AC_HELP_STRING([--disable-mempool],
[Disable the Gluster memory pooler.]))
USE_MEMPOOL="yes"
if test "x$enable_mempool" = "xno"; then
USE_MEMPOOL="no"
AC_DEFINE(GF_DISABLE_MEMPOOL, 1, [Disable the Gluster memory pooler.])
fi
# syslog section
AC_ARG_ENABLE([syslog],
AC_HELP_STRING([--disable-syslog],
[Disable syslog for logging]))
USE_SYSLOG="yes"
if test "x$enable_syslog" != "xno"; then
AC_DEFINE(GF_USE_SYSLOG, 1, [Use syslog for logging])
else
USE_SYSLOG="no"
fi
AM_CONDITIONAL([ENABLE_SYSLOG], [test x$USE_SYSLOG = xyes])
#end syslog section
BUILD_READLINE=no
AC_CHECK_LIB([readline -lcurses],[readline],[RLLIBS="-lreadline -lcurses"])
AC_CHECK_LIB([readline -ltermcap],[readline],[RLLIBS="-lreadline -ltermcap"])
AC_CHECK_LIB([readline -lncurses],[readline],[RLLIBS="-lreadline -lncurses"])
if test "x$RLLIBS" != "x"; then
if test "x$RL_UNDO" = "xyes"; then
AC_DEFINE(HAVE_READLINE, 1, [readline enabled CLI])
BUILD_READLINE=yes
else
BUILD_READLINE="no (present but missing undo)"
fi
fi
BUILD_LIBAIO=no
AC_CHECK_LIB([aio],[io_setup],[LIBAIO="-laio"])
if test "x$LIBAIO" != "x"; then
AC_DEFINE(HAVE_LIBAIO, 1, [libaio based POSIX enabled])
BUILD_LIBAIO=yes
fi
dnl glupy section
BUILD_GLUPY=no
AC_ARG_ENABLE([glupy], AS_HELP_STRING([--enable-glupy], [build glupy]))
if test "x$enable_glupy" != "xno"; then enable_glupy=yes; fi
if test "x$enable_glupy" = "xyes"; then
GLUPY_SUBDIR=glupy
GLUPY_SUBDIR_MAKEFILE=xlators/features/glupy/Makefile
GLUPY_SUBDIR_SRC_MAKEFILE=xlators/features/glupy/src/Makefile
if test "x$have_python2" = "xyes" -a "x$have_Python_h" = "xyes"; then
case $host_os in
darwin*)
BUILD_GLUPY=no
;;
*)
BUILD_GLUPY=yes
;;
esac
else
AC_MSG_WARN([
---------------------------------------------------------------------------------
cannot build glupy. python 2.x and python-devel/python-dev package are required.
---------------------------------------------------------------------------------])
fi
if test "x$BUILD_GLUPY" = "xyes"; then
echo "building glupy with -isystem $BUILD_PYTHON_INC -l $BUILD_PYTHON_LIB"
AC_SUBST(GLUPY_SUBDIR)
AC_SUBST(GLUPY_SUBDIR_MAKEFILE)
AC_SUBST(GLUPY_SUBDIR_SRC_MAKEFILE)
fi
fi
dnl end glupy section
dnl gnfs section
BUILD_GNFS="no"
AC_ARG_ENABLE([gnfs],
AC_HELP_STRING([--enable-gnfs],
[Enable legacy gnfs server xlator.]))
if test "x$enable_gnfs" = "xyes"; then
BUILD_GNFS="yes"
fi
AM_CONDITIONAL([BUILD_GNFS], [test x$BUILD_GNFS = xyes])
dnl end gnfs section
dnl Check for userspace-rcu
PKG_CHECK_MODULES([URCU], [liburcu-bp], [],
[AC_CHECK_HEADERS([urcu-bp.h],
[URCU_LIBS='-lurcu-bp'],
AC_MSG_ERROR([liburcu-bp not found]))])
PKG_CHECK_MODULES([URCU_CDS], [liburcu-cds >= 0.8], [],
[PKG_CHECK_MODULES([URCU_CDS], [liburcu-cds >= 0.7],
[AC_DEFINE(URCU_OLD, 1, [Define if liburcu 0.6 or 0.7 is found])],
[AC_CHECK_HEADERS([urcu/cds.h],
[AC_DEFINE(URCU_OLD, 1, [Define if liburcu 0.6 or 0.7 is found])
URCU_CDS_LIBS='-lurcu-cds'],
[AC_MSG_ERROR([liburcu-cds not found])])])])
BUILD_UNITTEST="no"
AC_ARG_ENABLE([cmocka],
AC_HELP_STRING([--enable-cmocka],
[Enable cmocka build options.]))
if test "x$enable_cmocka" = "xyes"; then
BUILD_UNITTEST="yes"
PKG_CHECK_MODULES([UNITTEST], [cmocka >= 1.0.1], [BUILD_UNITTEST="yes"],
[AC_MSG_ERROR([cmocka library is required to build glusterfs])]
)
fi
AM_CONDITIONAL([UNITTEST], [test x$BUILD_UNITTEST = xyes])
dnl Define UNIT_TESTING only for building cmocka binaries.
UNITTEST_CFLAGS="${UNITTEST_CFLAGS} -DUNIT_TESTING=1"
dnl Add cmocka for unit tests
case $host_os in
freebsd*)
dnl remove --coverage on FreeBSD due to a known llvm packaging bug
UNITTEST_CFLAGS="${UNITTEST_CPPFLAGS} ${UNITTEST_CFLAGS} -g -DDEBUG -O0"
UNITTEST_LDFLAGS="${UNITTEST_LIBS} ${UNITTEST_LDFLAGS}"
;;
*)
UNITTEST_CFLAGS="${UNITTEST_CPPFLAGS} ${UNITTEST_CFLAGS} -g -DDEBUG -O0 --coverage"
UNITTEST_LDFLAGS="${UNITTEST_LIBS} ${UNITTEST_LDFLAGS}"
;;
esac
AC_SUBST(UNITTEST_CFLAGS)
AC_SUBST(UNITTEST_LDFLAGS)
AC_SUBST(CFLAGS)
# end enable debug section
# experimental section
AC_ARG_ENABLE([experimental],
AC_HELP_STRING([--disable-experimental],
[Disable building experimental xlators]))
BUILD_EXPERIMENTAL="yes"
if test "x$enable_experimental" = "xno"; then
BUILD_EXPERIMENTAL="no"
fi
AM_CONDITIONAL([ENABLE_EXPERIMENTAL], [test x$BUILD_EXPERIMENTAL = xyes])
#end experimental section
# EC dynamic code generation section
EC_DYNAMIC_SUPPORT="none"
EC_DYNAMIC_ARCH="none"
AC_ARG_ENABLE([ec-dynamic],
AC_HELP_STRING([--disable-ec-dynamic],
[Disable all dynamic code generation extensions for EC module]))
AC_ARG_ENABLE([ec-dynamic-intel],
AC_HELP_STRING([--disable-ec-dynamic-intel],
[Disable all INTEL dynamic code generation extensions for EC module]))
AC_ARG_ENABLE([ec-dynamic-arm],
AC_HELP_STRING([--disable-ec-dynamic-arm],
[Disable all ARM dynamic code generation extensions for EC module]))
AC_ARG_ENABLE([ec-dynamic-x64],
AC_HELP_STRING([--disable-ec-dynamic-x64],
[Disable dynamic INTEL x64 code generation for EC module]))
AC_ARG_ENABLE([ec-dynamic-sse],
AC_HELP_STRING([--disable-ec-dynamic-sse],
[Disable dynamic INTEL SSE code generation for EC module]))
AC_ARG_ENABLE([ec-dynamic-avx],
AC_HELP_STRING([--disable-ec-dynamic-avx],
[Disable dynamic INTEL AVX code generation for EC module]))
AC_ARG_ENABLE([ec-dynamic-neon],
AC_HELP_STRING([--disable-ec-dynamic-neon],
[Disable dynamic ARM NEON code generation for EC module]))
if test "x$enable_ec_dynamic" != "xno"; then
case $host in
x86_64*)
if test "x$enable_ec_dynamic_intel" != "xno"; then
if test "x$enable_ec_dynamic_x64" != "xno"; then
EC_DYNAMIC_SUPPORT="$EC_DYNAMIC_SUPPORT x64"
AC_DEFINE(USE_EC_DYNAMIC_X64, 1, [Defined if using dynamic INTEL x64 code])
fi
if test "x$enable_ec_dynamic_sse" != "xno"; then
EC_DYNAMIC_SUPPORT="$EC_DYNAMIC_SUPPORT sse"
AC_DEFINE(USE_EC_DYNAMIC_SSE, 1, [Defined if using dynamic INTEL SSE code])
fi
if test "x$enable_ec_dynamic_avx" != "xno"; then
EC_DYNAMIC_SUPPORT="$EC_DYNAMIC_SUPPORT avx"
AC_DEFINE(USE_EC_DYNAMIC_AVX, 1, [Defined if using dynamic INTEL AVX code])
fi
if test "x$EC_DYNAMIC_SUPPORT" != "xnone"; then
EC_DYNAMIC_ARCH="intel"
fi
fi
;;
arm*)
if test "x$enable_ec_dynamic_arm" != "xno"; then
if test "x$enable_ec_dynamic_neon" != "xno"; then
EC_DYNAMIC_SUPPORT="$EC_DYNAMIC_SUPPORT neon"
AC_DEFINE(USE_EC_DYNAMIC_NEON, 1, [Defined if using dynamic ARM NEON code])
fi
if test "x$EC_DYNAMIC_SUPPORT" != "xnone"; then
EC_DYNAMIC_ARCH="arm"
fi
fi
;;
esac
EC_DYNAMIC_SUPPORT="${EC_DYNAMIC_SUPPORT#none }"
fi
AM_CONDITIONAL([ENABLE_EC_DYNAMIC_INTEL], [test "x$EC_DYNAMIC_ARCH" = "xintel"])
AM_CONDITIONAL([ENABLE_EC_DYNAMIC_ARM], [test "x$EC_DYNAMIC_ARCH" = "xarm"])
AM_CONDITIONAL([ENABLE_EC_DYNAMIC_X64], [test "x${EC_DYNAMIC_SUPPORT##*x64*}" = "x"])
AM_CONDITIONAL([ENABLE_EC_DYNAMIC_SSE], [test "x${EC_DYNAMIC_SUPPORT##*sse*}" = "x"])
AM_CONDITIONAL([ENABLE_EC_DYNAMIC_AVX], [test "x${EC_DYNAMIC_SUPPORT##*avx*}" = "x"])
AM_CONDITIONAL([ENABLE_EC_DYNAMIC_NEON], [test "x${EC_DYNAMIC_SUPPORT##*neon*}" = "x"])
AC_SUBST(USE_EC_DYNAMIC_X64)
AC_SUBST(USE_EC_DYNAMIC_SSE)
AC_SUBST(USE_EC_DYNAMIC_AVX)
AC_SUBST(USE_EC_DYNAMIC_NEON)
# end EC dynamic code generation section
dnl libglusterfs.so uses math functions
GF_LDADD="${GF_LDADD} ${MATH_LIB}"
GF_XLATOR_DEFAULT_LDFLAGS='-avoid-version -export-symbols $(top_srcdir)/xlators/xlator.sym'
2009-02-18 17:36:07 +05:30
AC_SUBST(GF_HOST_OS)
AC_SUBST(GF_CFLAGS)
AC_SUBST(GF_LDFLAGS)
AC_SUBST(GF_LDADD)
AC_SUBST(GF_FUSE_LDADD)
AC_SUBST(GF_FUSE_CFLAGS)
AC_SUBST(RLLIBS)
AC_SUBST(LIBAIO)
AC_SUBST(AM_MAKEFLAGS)
AC_SUBST(AM_LIBTOOLFLAGS)
AC_SUBST(GF_XLATOR_DEFAULT_LDFLAGS)
2009-02-18 17:36:07 +05:30
2009-08-11 18:26:11 -07:00
CONTRIBDIR='$(top_srcdir)/contrib'
AC_SUBST(CONTRIBDIR)
GF_CPPDEFINES='-D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -D$(GF_HOST_OS)'
build: add site.h as a place to put environment-specific defines Most people consume Gluster in one of two ways: * From packages provided by their OS/distribution vendor * By building themselves from source For the first group it doesn't matter whether configuration is done in a configure script, via command-line options to that configure script, or in a header file. All of these end up as edits to some file under the packager's control, which is then run through their tools and process (e.g. rpmbuild) to create the packages that users will install. For the second group, convenience matters. Such users might not even have a script wrapped around the configure process, and editing one line in a header file is a lot easier than editing several in the configure script. This also prevents a messy profusion of configure options, dozens of which might need to be added to support a single such user's preferences. This comes back around as greater simplicity for packagers as well. This patch defines site.h as the header file for options and parameters that someone building the code for themselves might want to tweak. The project can ship one version to reflect the developers' guess at the best defaults for most users, and sophisticated users with unusual needs can override many options at once just by maintaining their own version of that file. Everybody wins. Further guidelines for how to determine whether an option should go in configure.ac or site.h are explained within site.h itself. Fixes #201 Change-Id: I5b8fb518d42450737423c4c1f43ebeb3130b4ff6 Signed-off-by: Jeff Darcy <jdarcy@fb.com> Reviewed-on: https://review.gluster.org/17206 Tested-by: Jeff Darcy <jeff@pl.atyp.us> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Amar Tumballi <amarts@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org>
2017-05-08 12:55:49 -04:00
GF_CPPINCLUDES='-include $(top_builddir)/config.h -include $(top_builddir)/site.h -I$(top_srcdir)/libglusterfs/src -I$(top_builddir)/libglusterfs/src'
GF_CPPFLAGS="$GF_CPPFLAGS $GF_CPPDEFINES $GF_CPPINCLUDES"
AC_SUBST([GF_CPPFLAGS])
AM_CONDITIONAL([GF_LINUX_HOST_OS], test "${GF_HOST_OS}" = "GF_LINUX_HOST_OS")
Replace GPLV3 MD5 with OpenSSL MD5 Ric asked me to look at replacing the GPL licensed MD5 code with something better, i.e. perhaps faster, and with a less restrictive license, etc. So I took a couple hour holiday from working on wrapping up the client_t and did this. OpenSSL (nee SSLeay) is released under the OpenSSL license, a BSD/MIT style license. OpenSSL (libcrypto.so) is used on Linux, OS X and *BSD, Open Solaris, etc. IOW it's universally available on the platforms we care about. It's written by Eric Young (eay), now at EMC/RSA, and I can say from experience that the OpenSSL implementation of MD5 (at least) is every bit as fast as RSA's proprietary implementation (primarily because the implementations are very, very similar.) The last time I surveyed MD5 implementations I found they're all pretty much the same speed. I changed the APIs (and ABIs) for the strong and weak checksums. Strictly speaking I didn't need to do that. They're only called on short strings of data, i.e. pathnames, so using int32_t and uint32_t is ostensibly okay. My change is arguably a better, more general API for this sort of thing. It's also what bit me when gerrit/jenkins validation failed due to glusterfs segv-ing. (I didn't pay close enough attention to the implementation of the weak checksum. But it forced me to learn what gerrit/jenkins are doing and going forward I can do better testing before submitting to gerrit.) Now resubmitting with a BZ Change-Id: I545fade1604e74fc68399894550229bd57a5e0df BUG: 807718 Signed-off-by: Kaleb KEITHLEY <kkeithle@redhat.com> Reviewed-on: http://review.gluster.com/3019 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
2012-03-27 11:14:23 -04:00
AM_CONDITIONAL([GF_DARWIN_HOST_OS], test "${GF_HOST_OS}" = "GF_DARWIN_HOST_OS")
AM_CONDITIONAL([GF_BSD_HOST_OS], test "${GF_HOST_OS}" = "GF_BSD_HOST_OS")
2009-02-18 17:36:07 +05:30
AC_SUBST(GLUSTERD_WORKDIR)
AM_CONDITIONAL([GF_INSTALL_GLUSTERD_WORKDIR], test ! -d ${GLUSTERD_WORKDIR} && test -d ${sysconfdir}/glusterd )
geo-rep: mountbroker user management Non root geo-replication setup is now simplified. This patch provides cli for mountbroker user and options management To set Options, gluster system:: execute mountbroker opt <KEY> <VALUE> # for example, gluster system:: execute mountbroker opt mountbroker-root /var/mountbroker-root gluster system:: execute mountbroker opt geo-replication-log-group geogroup gluster system:: execute mountbroker opt rpc-auth-allow-insecure on To remove option, gluster system:: execute mountbroker optdel <KEY> # for example, gluster system:: execute mountbroker optdel geo-replication-log-group To add/edit user, gluster system:: execute mountbroker user <USERNAME> <VOLUMES> # for example gluster system:: execute mountbroker user geoaccount slavevol1,slavevol2 To remove user, gluster system:: execute mountbroker userdel <USERNAME> # for example gluster system:: execute mountbroker userdel geoaccount For info, gluster system:: execute mountbroker info gluster system:: execute mountbroker -j info For JSON output add -j after mountbroker, for example, gluster system:: execute mountbroker -j user geoaccount slavevol1,slavevol2 PS: Each peer prints its own JSON output, aggregator required from consumer side BUG: 1136312 Change-Id: Ie52210c0bcc91ac2ffd3ba58988222ffca62b47f Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/9398 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: darshan n <dnarayan@redhat.com> Reviewed-by: Kotresh HR <khiremat@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2015-01-06 18:20:45 +05:30
AC_SUBST(GLUSTERD_VOLFILE)
feature/glusterfind: A tool to find incremental changes Documentation is available in patch: http://review.gluster.org/#/c/9800/ A tool which helps to get list of modified files or list of all files in GlusterFS Volume using Changelog or find command. Usage ===== glusterfind --help Create: ------- glusterfind create --help The tool creates status file $GLUSTERD_WORKDIR/SESSION/VOLUME/status and records current timestamp to initiate the session. This timestamp will be used as start time for next runs. As part of create also generates ssh key and distributes to all peers. and enables build.pgfid and changelog using volume set command. Pre: ---- glusterfind pre --help This command is used to generate the list of files modified after session creation time or after last run. To get list of all files/dirs in Volume, run pre command with `--full` argument. The tool gets all nodes details using gluster volume info and runs node agent for each brick in respective nodes via ssh command. Once these node agents generate the output file, tool copies to local using scp. Merges all the output files to generate the final output file. Post: ----- glusterfind post --help After consuming the list, this sub command is called to update the session time based on pre command status file. List: ----- glusterfind list --help To view all the sessions Delete: ------- glusterfind delete --help Delete session. Known Issues ------------ 1. Deleted files will not get listed, since we can't convert GFID to Path if file/dir is deleted. 2. Only new name will get listed if Renamed. 3. All hardlinks will get listed. Change-Id: I82991feb0aea85cb6ec035fddbf80a2b276e86b0 BUG: 1193893 Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/9682 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Venky Shankar <vshankar@redhat.com> Reviewed-by: Prashanth Pai <ppai@redhat.com> Reviewed-by: Kotresh HR <khiremat@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2015-02-18 19:07:23 +05:30
AC_SUBST(GLUSTERFS_LIBEXECDIR)
AC_SUBST(GLUSTERFSD_MISCDIR)
build: Start using library versioning for various libraries According to libtool three individual numbers stand for CURRENT:REVISION:AGE, or C:R:A for short. The libtool script typically tacks these three numbers onto the end of the name of the .so file it creates. The formula for calculating the file numbers on Linux and Solaris is /path/to/library/<library_name>.(C - A).(A).(R) As you release new versions of your library, you will update the library's C:R:A. Although the rules for changing these version numbers can quickly become confusing, a few simple tips should help keep you on track. The libtool documentation goes into greater depth. In essence, every time you make a change to the library and release it, the C:R:A should change. A new library should start with 0:0:0. Each time you change the public interface (i.e., your installed header files), you should increment the CURRENT number. This is called your interface number. The main use of this interface number is to tag successive revisions of your API. The AGE number is how many consecutive versions of the API the current implementation supports. Thus if the CURRENT library API is the sixth published version of the interface and it is also binary compatible with the fourth and fifth versions (i.e., the last two), the C:R:A might be 6:0:2. When you break binary compatibility, you need to set AGE to 0 and of course increment CURRENT. The REVISION marks a change in the source code of the library that doesn't affect the interface-for example, a minor bug fix. Anytime you increment CURRENT, you should set REVISION back to 0. Change-Id: Id72e74c1642c804fea6f93ec109135c7c16f1810 BUG: 862082 Signed-off-by: Harshavardhana <harsha@harshavardhana.net> Reviewed-on: http://review.gluster.org/5645 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Niels de Vos <ndevos@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2013-08-17 13:01:23 -07:00
dnl pkg-config versioning
dnl
dnl Once we released gluster-api.pc with version=7. Since then we undid the
dnl library versioning and replaced it with symbol-versioning. The current
dnl libgfapi.so has version 0, but the symbols have the version from the main
dnl package at the time they were added.
dnl
dnl Because other packages (like samba) use the pkg-config version, we can not
dnl drop it, or decrease the version easily. The simplest solution is to keep
dnl the version=7 and add sub-digits for the actual package/symbol versions.
GFAPI_VERSION="7."${PACKAGE_VERSION}
build: Start using library versioning for various libraries According to libtool three individual numbers stand for CURRENT:REVISION:AGE, or C:R:A for short. The libtool script typically tacks these three numbers onto the end of the name of the .so file it creates. The formula for calculating the file numbers on Linux and Solaris is /path/to/library/<library_name>.(C - A).(A).(R) As you release new versions of your library, you will update the library's C:R:A. Although the rules for changing these version numbers can quickly become confusing, a few simple tips should help keep you on track. The libtool documentation goes into greater depth. In essence, every time you make a change to the library and release it, the C:R:A should change. A new library should start with 0:0:0. Each time you change the public interface (i.e., your installed header files), you should increment the CURRENT number. This is called your interface number. The main use of this interface number is to tag successive revisions of your API. The AGE number is how many consecutive versions of the API the current implementation supports. Thus if the CURRENT library API is the sixth published version of the interface and it is also binary compatible with the fourth and fifth versions (i.e., the last two), the C:R:A might be 6:0:2. When you break binary compatibility, you need to set AGE to 0 and of course increment CURRENT. The REVISION marks a change in the source code of the library that doesn't affect the interface-for example, a minor bug fix. Anytime you increment CURRENT, you should set REVISION back to 0. Change-Id: Id72e74c1642c804fea6f93ec109135c7c16f1810 BUG: 862082 Signed-off-by: Harshavardhana <harsha@harshavardhana.net> Reviewed-on: http://review.gluster.org/5645 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Niels de Vos <ndevos@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2013-08-17 13:01:23 -07:00
LIBGFCHANGELOG_VERSION="0.0.1"
AC_SUBST(GFAPI_VERSION)
AC_SUBST(LIBGFCHANGELOG_VERSION)
Adding Libgfdb to GlusterFS ************************************************************************* Libgfdb | ************************************************************************* Libgfdb provides abstract mechanism to record extra/rich metadata required for data maintenance, such as data tiering/classification. It provides consumer with API for recording and querying, keeping the consumer abstracted from the data store used beneath for storing data. It works in a plug-and-play model, where data stores can be plugged-in. Presently we have plugin for Sqlite3. In the future will provide recording and querying performance optimizer. In the current implementation the schema of metadata is fixed. Schema: ~~~~~~ GF_FILE_TB Table: ~~~~~~~~~~~~~~~~~ This table has one entry per file inode. It holds the metadata required to make decisions in data maintenance. GF_ID (Primary key) : File GFID (Universal Unique IDentifier in the namespace) W_SEC, W_MSEC : Write wind time in sec & micro-sec UW_SEC, UW_MSEC : Write un-wind time in sec & micro-sec W_READ_SEC, W_READ_MSEC : Read wind time in sec & micro-sec UW_READ_SEC, UW_READ_MSEC : Read un-wind time in sec & micro-sec WRITE_FREQ_CNTR INTEGER : Write Frequency Counter READ_FREQ_CNTR INTEGER : Read Frequency Counter GF_FLINK_TABLE: ~~~~~~~~~~~~~~ This table has all the hardlinks to a file inode. GF_ID : File GFID (Composite Primary Key)``| GF_PID : Parent Directory GFID (Composite Primary Key) |-> Primary Key FNAME : File Base Name (Composite Primary Key)__| FPATH : File Full Path (Its redundant for now, this will go) W_DEL_FLAG : This Flag is used for crash consistancy, when a link is unlinked. i.e Set to 1 during unlink wind and during unwind this record is deleted LINK_UPDATE : This Flag is used when a link is changed i.e rename. Set to 1 when rename wind and set to 0 in rename unwind Libgfdb API: ~~~~~~~~~~~ Refer libglusterfs/src/gfdb/gfdb_data_store.h Change-Id: I2e9fbab3878ce630a7f41221ef61017dc43db11f BUG: 1194753 Signed-off-by: Joseph Fernandes <josferna@redhat.com> Signed-off-by: Dan Lambright <dlambrig@redhat.com> Signed-off-by: Joseph Fernandes <josferna@redhat.com> Reviewed-on: http://review.gluster.org/9683 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2015-02-18 19:45:23 +05:30
LIBGFDB_VERSION="0.0.1"
AC_SUBST(LIBGFDB_VERSION)
build: Start using library versioning for various libraries According to libtool three individual numbers stand for CURRENT:REVISION:AGE, or C:R:A for short. The libtool script typically tacks these three numbers onto the end of the name of the .so file it creates. The formula for calculating the file numbers on Linux and Solaris is /path/to/library/<library_name>.(C - A).(A).(R) As you release new versions of your library, you will update the library's C:R:A. Although the rules for changing these version numbers can quickly become confusing, a few simple tips should help keep you on track. The libtool documentation goes into greater depth. In essence, every time you make a change to the library and release it, the C:R:A should change. A new library should start with 0:0:0. Each time you change the public interface (i.e., your installed header files), you should increment the CURRENT number. This is called your interface number. The main use of this interface number is to tag successive revisions of your API. The AGE number is how many consecutive versions of the API the current implementation supports. Thus if the CURRENT library API is the sixth published version of the interface and it is also binary compatible with the fourth and fifth versions (i.e., the last two), the C:R:A might be 6:0:2. When you break binary compatibility, you need to set AGE to 0 and of course increment CURRENT. The REVISION marks a change in the source code of the library that doesn't affect the interface-for example, a minor bug fix. Anytime you increment CURRENT, you should set REVISION back to 0. Change-Id: Id72e74c1642c804fea6f93ec109135c7c16f1810 BUG: 862082 Signed-off-by: Harshavardhana <harsha@harshavardhana.net> Reviewed-on: http://review.gluster.org/5645 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Niels de Vos <ndevos@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2013-08-17 13:01:23 -07:00
dnl libtool versioning
LIBGFXDR_LT_VERSION="0:1:0"
LIBGFRPC_LT_VERSION="0:1:0"
LIBGLUSTERFS_LT_VERSION="0:1:0"
LIBGFCHANGELOG_LT_VERSION="0:1:0"
2014-11-03 16:07:30 -05:00
GFAPI_LT_VERSION="0:0:0"
build: Start using library versioning for various libraries According to libtool three individual numbers stand for CURRENT:REVISION:AGE, or C:R:A for short. The libtool script typically tacks these three numbers onto the end of the name of the .so file it creates. The formula for calculating the file numbers on Linux and Solaris is /path/to/library/<library_name>.(C - A).(A).(R) As you release new versions of your library, you will update the library's C:R:A. Although the rules for changing these version numbers can quickly become confusing, a few simple tips should help keep you on track. The libtool documentation goes into greater depth. In essence, every time you make a change to the library and release it, the C:R:A should change. A new library should start with 0:0:0. Each time you change the public interface (i.e., your installed header files), you should increment the CURRENT number. This is called your interface number. The main use of this interface number is to tag successive revisions of your API. The AGE number is how many consecutive versions of the API the current implementation supports. Thus if the CURRENT library API is the sixth published version of the interface and it is also binary compatible with the fourth and fifth versions (i.e., the last two), the C:R:A might be 6:0:2. When you break binary compatibility, you need to set AGE to 0 and of course increment CURRENT. The REVISION marks a change in the source code of the library that doesn't affect the interface-for example, a minor bug fix. Anytime you increment CURRENT, you should set REVISION back to 0. Change-Id: Id72e74c1642c804fea6f93ec109135c7c16f1810 BUG: 862082 Signed-off-by: Harshavardhana <harsha@harshavardhana.net> Reviewed-on: http://review.gluster.org/5645 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Niels de Vos <ndevos@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2013-08-17 13:01:23 -07:00
AC_SUBST(LIBGFXDR_LT_VERSION)
AC_SUBST(LIBGFRPC_LT_VERSION)
AC_SUBST(LIBGLUSTERFS_LT_VERSION)
AC_SUBST(LIBGFCHANGELOG_LT_VERSION)
AC_SUBST(GFAPI_LT_VERSION)
api: versioned symbols in libgfapi.so for compatibility Use versioned symbols to keep libgfapi at libgfapi.so.0.0.0 Revisited to address broken build on Mac OS X See http://review.gluster.org/9036 Rebased to include http://review.gluster.org/#/c/9376/ (glfs_resolve()) but note that gerrit's "Rebase Change" couldn't do it. N.B. noticed that glfs_get_volumeid() decl in glfs.h was missing the __THROW, added it. On systems using ELF and the GNU toolchain, symbol versions are created with a .symver asm operand in the .c source file. Clang is claimed to be compatible with gcc, so we'll pretend for now that this also works with clang. On Mac OS X, aliases are created with __asm "magic" in the .h header file. In the normal case, when both the decl and defn match, that's all that's needed. In our case though the decl and defn don't match --- we have, e.g. a defn such as 'int glfs_foo(...)' and the corresponding decl is 'int pub_glfs_foo(...)'. To make this work we create the necessary aliases in the library at link time with the -alias_list link option. Note that this results in there being pairs of symbols in the .dylib, e.g. _pub_glfs_foo and _glfs_foo$GFAPI_3.4.0. We could use another link option, -unexported_symbols_list to elide the _pub_glfs_* symbols. (And we probably should.) Linux symbol versioning was essentially copied from Solaris; in general I would expect this to "just work" on Solaris, but until someone tries we don't really know. Change-Id: Icb96a3c2d80be7b6d7a6849bb9168f03a947f47c BUG: 1160709 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com> Reviewed-on: http://review.gluster.org/9143 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Niels de Vos <ndevos@redhat.com> Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
2014-11-18 11:08:16 -05:00
AC_SUBST(GFAPI_EXTRA_LDFLAGS)
build: Start using library versioning for various libraries According to libtool three individual numbers stand for CURRENT:REVISION:AGE, or C:R:A for short. The libtool script typically tacks these three numbers onto the end of the name of the .so file it creates. The formula for calculating the file numbers on Linux and Solaris is /path/to/library/<library_name>.(C - A).(A).(R) As you release new versions of your library, you will update the library's C:R:A. Although the rules for changing these version numbers can quickly become confusing, a few simple tips should help keep you on track. The libtool documentation goes into greater depth. In essence, every time you make a change to the library and release it, the C:R:A should change. A new library should start with 0:0:0. Each time you change the public interface (i.e., your installed header files), you should increment the CURRENT number. This is called your interface number. The main use of this interface number is to tag successive revisions of your API. The AGE number is how many consecutive versions of the API the current implementation supports. Thus if the CURRENT library API is the sixth published version of the interface and it is also binary compatible with the fourth and fifth versions (i.e., the last two), the C:R:A might be 6:0:2. When you break binary compatibility, you need to set AGE to 0 and of course increment CURRENT. The REVISION marks a change in the source code of the library that doesn't affect the interface-for example, a minor bug fix. Anytime you increment CURRENT, you should set REVISION back to 0. Change-Id: Id72e74c1642c804fea6f93ec109135c7c16f1810 BUG: 862082 Signed-off-by: Harshavardhana <harsha@harshavardhana.net> Reviewed-on: http://review.gluster.org/5645 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Niels de Vos <ndevos@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2013-08-17 13:01:23 -07:00
GFAPI_LIBS="${ACL_LIBS}"
AC_SUBST(GFAPI_LIBS)
dnl this change necessary for run-tests.sh
AC_CONFIG_FILES([tests/env.rc],[ln -s ${ac_abs_builddir}/env.rc ${ac_abs_srcdir}/env.rc 2>/dev/null])
2009-02-18 17:36:07 +05:30
AC_OUTPUT
echo
echo "GlusterFS configure summary"
echo "==========================="
echo "FUSE client : $BUILD_FUSE_CLIENT"
echo "Infiniband verbs : $BUILD_IBVERBS"
echo "epoll IO multiplex : $BUILD_EPOLL"
echo "argp-standalone : $BUILD_ARGP_STANDALONE"
echo "fusermount : $BUILD_FUSERMOUNT"
echo "readline : $BUILD_READLINE"
echo "georeplication : $BUILD_SYNCDAEMON"
echo "Linux-AIO : $BUILD_LIBAIO"
echo "Enable Debug : $BUILD_DEBUG"
bd: posix/multi-brick support to BD xlator Current BD xlator (block backend) has a few limitations such as * Creation of directories not supported * Supports only single brick * Does not use extended attributes (and client gfid) like posix xlator * Creation of special files (symbolic links, device nodes etc) not supported Basic limitation of not allowing directory creation is blocking oVirt/VDSM to consume BD xlator as part of Gluster domain since VDSM creates multi-level directories when GlusterFS is used as storage backend for storing VM images. To overcome these limitations a new BD xlator with following improvements is suggested. * New hybrid BD xlator that handles both regular files and block device files * The volume will have both POSIX and BD bricks. Regular files are created on POSIX bricks, block devices are created on the BD brick (VG) * BD xlator leverages exiting POSIX xlator for most POSIX calls and hence sits above the POSIX xlator * Block device file is differentiated from regular file by an extended attribute * The xattr 'user.glusterfs.bd' (BD_XATTR) plays a role in mapping a posix file to Logical Volume (LV). * When a client sends a request to set BD_XATTR on a posix file, a new LV is created and mapped to posix file. So every block device will have a representative file in POSIX brick with 'user.glusterfs.bd' (BD_XATTR) set. * Here after all operations on this file results in LV related operations. For example opening a file that has BD_XATTR set results in opening the LV block device, reading results in reading the corresponding LV block device. When BD xlator gets request to set BD_XATTR via setxattr call, it creates a LV and information about this LV is placed in the xattr of the posix file. xattr "user.glusterfs.bd" used to identify that posix file is mapped to BD. Usage: Server side: [root@host1 ~]# gluster volume create bdvol host1:/storage/vg1_info?vg1 host2:/storage/vg2_info?vg2 It creates a distributed gluster volume 'bdvol' with Volume Group vg1 using posix brick /storage/vg1_info in host1 and Volume Group vg2 using /storage/vg2_info in host2. [root@host1 ~]# gluster volume start bdvol Client side: [root@node ~]# mount -t glusterfs host1:/bdvol /media [root@node ~]# touch /media/posix It creates regular posix file 'posix' in either host1:/vg1 or host2:/vg2 brick [root@node ~]# mkdir /media/image [root@node ~]# touch /media/image/lv1 It also creates regular posix file 'lv1' in either host1:/vg1 or host2:/vg2 brick [root@node ~]# setfattr -n "user.glusterfs.bd" -v "lv" /media/image/lv1 [root@node ~]# Above setxattr results in creating a new LV in corresponding brick's VG and it sets 'user.glusterfs.bd' with value 'lv:<default-extent-size' [root@node ~]# truncate -s5G /media/image/lv1 It results in resizig LV 'lv1'to 5G New BD xlator code is placed in xlators/storage/bd directory. Also add volume-uuid to the VG so that same VG can't be used for other bricks/volumes. After deleting a gluster volume, one has to manually remove the associated tag using vgchange <vg-name> --deltag <trusted.glusterfs.volume-id:<volume-id>> Changes from previous version V5: * Removed support for delayed deleting of LVs Changes from previous version V4: * Consolidated the patches * Removed usage of BD_XATTR_SIZE and consolidated it in BD_XATTR. Changes from previous version V3: * Added support in FUSE to support full/linked clone * Added support to merge snapshots and provide information about origin * bd_map xlator removed * iatt structure used in inode_ctx. iatt is cached and updated during fsync/flush * aio support * Type and capabilities of volume are exported through getxattr Changes from version 2: * Used inode_context for caching BD size and to check if loc/fd is BD or not. * Added GlusterFS server offloaded copy and snapshot through setfattr FOP. As part of this libgfapi is modified. * BD xlator supports stripe * During unlinking if a LV file is already opened, its added to delete list and bd_del_thread tries to delete from this list when a last reference to that file is closed. Changes from previous version: * gfid is used as name of LV * ? is used to specify VG name for creating BD volume in volume create, add-brick. gluster volume create volname host:/path?vg * open-behind issue is fixed * A replicate brick can be added dynamically and LVs from source brick are replicated to destination brick * A distribute brick can be added dynamically and rebalance operation distributes existing LVs/files to the new brick * Thin provisioning support added. * bd_map xlator support retained * setfattr -n user.glusterfs.bd -v "lv" creates a regular LV and setfattr -n user.glusterfs.bd -v "thin" creates thin LV * Capability and backend information added to gluster volume info (and --xml) so that management tools can exploit BD xlator. * tracing support for bd xlator added TODO: * Add support to display snapshots for a given LV * Display posix filename for list-origin instead of gfid Change-Id: I00d32dfbab3b7c806e0841515c86c3aa519332f2 BUG: 1028672 Signed-off-by: M. Mohan Kumar <mohan@in.ibm.com> Reviewed-on: http://review.gluster.org/4809 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Anand Avati <avati@redhat.com>
2013-11-13 22:44:42 +05:30
echo "Block Device xlator : $BUILD_BD_XLATOR"
echo "glupy : $BUILD_GLUPY"
echo "Use syslog : $USE_SYSLOG"
echo "XML output : $BUILD_XML_OUTPUT"
Transparent data encryption and metadata authentication .. in the systems with non-trusted server This new functionality can be useful in various cloud technologies. It is implemented via a special encryption/crypt translator,which works on the client side and performs encryption and authentication; 1. Class of supported algorithms The crypt translator can support any atomic symmetric block cipher algorithms (which require to pad plain/cipher text before performing encryption/decryption transform (see glossary in atom.c for definitions). In particular, it can support algorithms with the EOF issue (which require to pad the end of file by extra-data). Crypt translator performs translations user -> (offset, size) -> (aligned-offset, padded-size) ->server (and backward), and resolves individual FOPs (write(), truncate(), etc) to read-modify-write sequences. A volume can contain files encrypted by different algorithms of the mentioned class. To change some option value just reconfigure the volume. Currently only one algorithm is supported: AES_XTS. Example of algorithms, which can not be supported by the crypt translator: 1. Asymmetric block cipher algorithms, which inflate data, e.g. RSA; 2. Symmetric block cipher algorithms with inline MACs for data authentication. 2. Implementation notes. a) Atomic algorithms Since any process in a stackable file system manipulates with local data (which can be obsoleted by local data of another process), any atomic cipher algorithm without proper support can lead to non-POSIX behavior. To resolve the "collisions" we introduce locks: before performing FOP->read(), FOP->write(), etc. the process should first lock the file. b) Algorithms with EOF issue Such algorithms require to pad the end of file with some extra-data. Without proper support this will result in losing information about real file size. Keeping a track of real file size is a responsibility of the crypt translator. A special extended attribute with the name "trusted.glusterfs.crypt.att.size" is used for this purpose. All files contained in bricks of encrypted volume do have "padded" sizes. 3. Non-trusted servers and Metadata authentication We assume that server, where user's data is stored on is non-trusted. It means that the server can be subjected to various attacks directed to reveal user's encrypted personal data. We provide protection against such attacks. Every encrypted file has specific private attributes (cipher algorithm id, atom size, etc), which are packed to a string (so-called "format string") and stored as a special extended attribute with the name "trusted.glusterfs.crypt.att.cfmt". We protect the string from tampering. This protection is mandatory, hardcoded and is always on. Without such protection various attacks (based on extending the scope of per-file secret keys) are possible. Our authentication method has been developed in tight collaboration with Red Hat security team and is implemented as "metadata loader of version 1" (see file metadata.c). This method is NIST-compliant and is based on checking 8-byte per-hardlink MACs created(updated) by FOP->create(), FOP->link(), FOP->unlink(), FOP->rename() by the following unique entities: . file (hardlink) name; . verified file's object id (gfid). Every time, before manipulating with a file, we check it's MACs at FOP->open() time. Some FOPs don't require a file to be opened (e.g. FOP->truncate()). In such cases the crypt translator opens the file mandatory. 4. Generating keys Unique per-file keys are derived by NIST-compliant methods from the a) parent key; b) unique verified object-id of the file (gfid); Per-volume master key, provided by user at mount time is in the root of this "tree of keys". Those keys are used to: 1) encrypt/decrypt file data; 2) encrypt/decrypt file metadata; 3) create per-file and per-link MACs for metadata authentication. 5. Instructions Getting started with crypt translator Example: 1) Create a volume "myvol" and enable encryption: # gluster volume create myvol pepelac:/vols/xvol # gluster volume set myvol encryption on 2) Set location (absolute pathname) of your master key: # gluster volume set myvol encryption.master-key /home/me/mykey 3) Set other options to override default options, if needed. Start the volume. 4) On the client side make sure that the file /home/me/mykey exists and contains proper per-volume master key (that is 256-bit AES key). This key has to be in hex form, i.e. should be represented by 64 symbols from the set {'0', ..., '9', 'a', ..., 'f'}. The key should start at the beginning of the file. All symbols at offsets >= 64 are ignored. 5) Mount the volume "myvol" on the client side: # glusterfs --volfile-server=pepelac --volfile-id=myvol /mnt After successful mount the file which contains master key may be removed. NOTE: Keeping the master key between mount sessions is in user's competence. ********************************************************************** WARNING! Losing the master key will make content of all regular files inaccessible. Mount with improper master key allows to access content of directories: file names are not encrypted. ********************************************************************** 6. Options of crypt translator 1) "master-key": specifies location (absolute pathname) of the file which contains per-volume master key. There is no default location for master key. 2) "data-key-size": specifies size of per-file key for data encryption Possible values: . "256" default value . "512" 3) "block-size": specifies atom size. Possible values: . "512" . "1024" . "2048" . "4096" default value; 7. Test cases Any workload, which involves the following file operations: ->create(); ->open(); ->readv(); ->writev(); ->truncate(); ->ftruncate(); ->link(); ->unlink(); ->rename(); ->readdirp(). 8. TODOs: 1) Currently size of IOs issued by crypt translator is restricted by block_size (4K by default). We can use larger IOs to improve performance. Change-Id: I2601fe95c5c4dc5b22308a53d0cbdc071d5e5cee BUG: 1030058 Signed-off-by: Edward Shishkin <edward@redhat.com> Signed-off-by: Anand Avati <avati@redhat.com> Reviewed-on: http://review.gluster.org/4667 Tested-by: Gluster Build System <jenkins@build.gluster.com>
2013-03-13 21:56:46 +01:00
echo "Encryption xlator : $BUILD_CRYPT_XLATOR"
echo "Unit Tests : $BUILD_UNITTEST"
echo "Track priv ports : $TRACK_PRIVPORTS"
echo "POSIX ACLs : $BUILD_POSIX_ACLS"
Adding Libgfdb to GlusterFS ************************************************************************* Libgfdb | ************************************************************************* Libgfdb provides abstract mechanism to record extra/rich metadata required for data maintenance, such as data tiering/classification. It provides consumer with API for recording and querying, keeping the consumer abstracted from the data store used beneath for storing data. It works in a plug-and-play model, where data stores can be plugged-in. Presently we have plugin for Sqlite3. In the future will provide recording and querying performance optimizer. In the current implementation the schema of metadata is fixed. Schema: ~~~~~~ GF_FILE_TB Table: ~~~~~~~~~~~~~~~~~ This table has one entry per file inode. It holds the metadata required to make decisions in data maintenance. GF_ID (Primary key) : File GFID (Universal Unique IDentifier in the namespace) W_SEC, W_MSEC : Write wind time in sec & micro-sec UW_SEC, UW_MSEC : Write un-wind time in sec & micro-sec W_READ_SEC, W_READ_MSEC : Read wind time in sec & micro-sec UW_READ_SEC, UW_READ_MSEC : Read un-wind time in sec & micro-sec WRITE_FREQ_CNTR INTEGER : Write Frequency Counter READ_FREQ_CNTR INTEGER : Read Frequency Counter GF_FLINK_TABLE: ~~~~~~~~~~~~~~ This table has all the hardlinks to a file inode. GF_ID : File GFID (Composite Primary Key)``| GF_PID : Parent Directory GFID (Composite Primary Key) |-> Primary Key FNAME : File Base Name (Composite Primary Key)__| FPATH : File Full Path (Its redundant for now, this will go) W_DEL_FLAG : This Flag is used for crash consistancy, when a link is unlinked. i.e Set to 1 during unlink wind and during unwind this record is deleted LINK_UPDATE : This Flag is used when a link is changed i.e rename. Set to 1 when rename wind and set to 0 in rename unwind Libgfdb API: ~~~~~~~~~~~ Refer libglusterfs/src/gfdb/gfdb_data_store.h Change-Id: I2e9fbab3878ce630a7f41221ef61017dc43db11f BUG: 1194753 Signed-off-by: Joseph Fernandes <josferna@redhat.com> Signed-off-by: Dan Lambright <dlambrig@redhat.com> Signed-off-by: Joseph Fernandes <josferna@redhat.com> Reviewed-on: http://review.gluster.org/9683 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2015-02-18 19:45:23 +05:30
echo "Data Classification : $BUILD_GFDB"
echo "SELinux features : $USE_SELINUX"
echo "firewalld-config : $BUILD_FIREWALLD"
echo "Experimental xlators : $BUILD_EXPERIMENTAL"
eventsapi: Gluster Eventing Feature implementation [Depends on http://review.gluster.org/14627] Design is available in `glusterfs-specs`, A change from the design is support of webhook instead of Websockets as discussed in the design http://review.gluster.org/13115 Since Websocket support depends on REST APIs, I will add Websocket support once REST APIs patch gets merged Usage: Run following command to start/stop Eventsapi server in all Peers, which will collect the notifications from any Gluster daemon and emits to configured client. gluster-eventsapi start|stop|restart|reload Status of running services can be checked using, gluster-eventsapi status Events listener is a HTTP(S) server which listens to events emited by the Gluster. Create a HTTP Server to listen on POST and register that URL using, gluster-eventsapi webhook-add <URL> [--bearer-token <TOKEN>] For example, if HTTP Server running in `http://192.168.122.188:9000` then add that URL using, gluster-eventsapi webhook-add http://192.168.122.188:9000 If it expects a Token then specify it using `--bearer-token` or `-t` We can also test Webhook if all peer nodes can send message or not using, gluster-eventsapi webhook-test <URL> [--bearer-token <TOKEN>] Configurations can be viewed/updated using, gluster-eventsapi config-get [--name] gluster-eventsapi config-set <NAME> <VALUE> gluster-eventsapi config-reset <NAME|all> If any one peer node was down during config-set/reset or webhook modifications, Run sync command from good node when a peer node comes back. Automatic update is not yet implemented. gluster-eventsapi sync Basic Events Client(HTTP Server) is included with the code, Start running the client with required port and start listening to the events. /usr/share/glusterfs/scripts/eventsdash.py --port 8080 Default port is 9000, if no port is specified, once it started running then configure gluster-eventsapi to send events to that client. Eventsapi Client can be outside of the Cluster, it can be run event on Windows. But only requirement is the client URL should be accessible by all peer nodes.(Or ngrok(https://ngrok.com) like tools can be used) Events implemented with this patch, - Volume Create - Volume Start - Volume Stop - Volume Delete - Peer Attach - Peer Detach It is easy to add/support more events, since it touches Gluster cmd code and to avoid merge conflicts I will add support for more events once this patch merges. BUG: 1334044 Change-Id: I316827ac9dd1443454df7deffe4f54835f7f6a08 Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: http://review.gluster.org/14248 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
2016-05-05 18:34:41 +05:30
echo "Events : $BUILD_EVENTS"
echo "EC dynamic support : $EC_DYNAMIC_SUPPORT"
echo "Use memory pools : $USE_MEMPOOL"
echo "Nanosecond m/atimes : $BUILD_NANOSECOND_TIMESTAMPS"
echo "Legacy gNFS server : $BUILD_GNFS"
echo "IPV6 default : $with_ipv6_default"
echo "Use TIRPC : $with_libtirpc"
2009-02-18 17:36:07 +05:30
echo