373 Commits

Author SHA1 Message Date
Avra Sengupta
a181282d14 tests/cluster.rc: Moving the glusterd logs for cluster testcases
Moving the glusterd logs for cluster testcases from /d/backends to
logdir.

Change-Id: I6f22ca0c2faa87b0774e718c6c26d6547afd7c51
BUG: 1102593
Signed-off-by: Avra Sengupta <asengupt@redhat.com>
Reviewed-on: http://review.gluster.org/7924
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2014-05-29 10:50:13 -07:00
Raghavendra Bhat
cc0378d39f user servicable snapshots
Change-Id: Idbf27dbe088e646a8ab81cedc5818413795895ea
Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com>
Signed-off-by: Anand Subramanian <anands@redhat.com>
Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com>
Reviewed-on: http://review.gluster.org/7700
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2014-05-29 09:25:46 -07:00
Susant Palai
c9d53e53af Glusterd/Rebalance: Update rebalance status properly in
node_state.info

credit: kaushal@redhat.com
        spalai@redhat.com

Change-Id: I08d0771e2168a4a6ebd473e8a937b8b2eda1341a
BUG: 1075087
Signed-off-by: Susant Palai <spalai@redhat.com>
Reviewed-on: http://review.gluster.org/7214
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Kaushal M <kaushal@redhat.com>
2014-05-26 23:08:59 -07:00
Niels de Vos
79de5b6377 tests/rpm: always run ./autogen.sh to create missing files
In some occasions 'install-sh' seems to be missing in the 'make dist'
tarball when ./autogen.sh has not been run (skipped when 'configure'
exists). With this changes, 'autogen.sh' is always run so that missing
files should get added to the tarball.

Also write the logs from mock to a known location, and copy them to the
'/var/log/' directory that Jenkins archives after a regression test
failure. This makes it easier to find build issues that mock detects.

Change-Id: I8d0cf1afef61ebab0137aa0d20521e75a35ddbdd
BUG: 1038391
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: http://review.gluster.org/7786
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2014-05-21 22:21:49 -07:00
Pranith Kumar K
0a77eb1e45 tests: s/timeout/EXPECT_WITHIN/
Also fixed nfs.rc so that regression build works on my fedora VM

Change-Id: Ife36343bf1a590430e24065b9bcdf5bed3ae546d
BUG: 1092850
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/7837
Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
2014-05-21 19:53:46 -07:00
Pranith Kumar K
5315df7644 Tests: Umount nfs mount before volume stop
Change-Id: Ia1b98ee56bfa562c3f97f1649699f49917ebbfc7
BUG: 1092850
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/7813
Reviewed-by: Krutika Dhananjay <kdhananj@redhat.com>
Reviewed-by: Humble Devassy Chirammal <humble.devassy@gmail.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
2014-05-20 20:58:39 -07:00
Pranith Kumar K
7cd32c18a1 tests: Use uniform timeouts
Change-Id: I479ab941b3b2da3b16f624400fbd300f08326268
BUG: 1092850
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/7799
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2014-05-19 21:00:46 -07:00
Pranith Kumar K
05a5ffe972 tests: Wait for nfs export to be available
Change-Id: I59a5e0cb78f2b670761a65272b8ab1d7bdb3668a
BUG: 1092850
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/7773
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2014-05-16 03:28:16 -07:00
Atin Mukherjee
e64dd0a358 glusterd : barrier enable/disable should fail if already enabled/disabled
In barrier notify function, if we fail to set the barrier option execution goes
to default_notify which returns 0 and command returns success.

Fix : We need not call the default_notify function when handling
GF_EVENT_TRANSLATOR_OP in barrier xlator's notify.

Change-Id: Ia2c361b43cca7791c29829d69dcd6fc7923102f6
BUG: 1092841
Signed-off-by: Atin Mukherjee <amukherj@redhat.com>
Reviewed-on: http://review.gluster.org/7609
Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Sachin Pandit <spandit@redhat.com>
Reviewed-by: Kaushal M <kaushal@redhat.com>
2014-05-12 04:37:03 -07:00
Niels de Vos
2fd499d148 rpc: implement server.manage-gids for group resolving on the bricks
The new volume option 'server.manage-gids' can be enabled in
environments where a user belongs to more than the current absolute
maximum of 93 groups. This option triggers the following behavior:

1. The AUTH_GLUSTERFS structure sent by GlusterFS clients (fuse, nfs or
   libgfapi) will contain only one (1) auxiliary group, instead of
   a full list. This reduces network usage and prevents problems in
   encoding the AUTH_GLUSTERFS structure which should fit in 400 bytes.
2. The single group in the RPC Calls received by the server is replaced
   by resolving the groups server-side. Permission checks and similar in
   lower xlators are applied against the full list of groups where the
   user belongs to, and not the single auxiliary group that the client
   sent.

Change-Id: I9e540de13e3022f8b63ff893ecba511129a47b91
BUG: 1053579
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: http://review.gluster.org/7501
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Santosh Pradhan <spradhan@redhat.com>
Reviewed-by: Harshavardhana <harsha@harshavardhana.net>
Reviewed-by: Anand Avati <avati@redhat.com>
2014-05-09 12:22:39 -07:00
Harshavardhana
47c33dd271 tests: min-free-disk test was incomplete
- use '%' when we mean it for clarity
- in bash we need to evaluate counter decrements

Change-Id: Ibd17126945e8a335fa2671d658a2e0c71049fd1e
BUG: 874554
Signed-off-by: Harshavardhana <harsha@harshavardhana.net>
Reviewed-on: http://review.gluster.org/7687
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Raghavendra Talur <rtalur@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2014-05-09 11:16:26 -07:00
Raghavendra Bhat
26a4b5c98e mgmt/glusterd: delete oldest snapshot upon exceeding soft-limit
Change-Id: I2d6ebae3ced1910f2dee43eeb9fc430e9f31073f
BUG: 1061685
Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com>
Reviewed-on: http://review.gluster.org/7587
Reviewed-by: Rajesh Joseph <rjoseph@redhat.com>
Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com>
Tested-by: Krishnan Parthasarathi <kparthas@redhat.com>
2014-05-08 23:10:16 -07:00
Pranith Kumar K
1b042296dd cluster/afr: Remove stale index in self-heal codepath
Change-Id: I635fc0fa955b33590f1c5b4dfec22d591ea8575c
BUG: 1032894
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/6592
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2014-05-08 11:25:17 -07:00
Avra Sengupta
e80daecc3d glusterd: Use a calloc-ed copy of txn_id for glusterd_do_replace_brick
As glusterd_do_replace_brick() is spawned through gf_timer_call_after(),
by the time it's called the event is freed, and the txn_id is lost.
Hence using a calloc-ed copy, which will be freed as a part of rb_ctx
dict.

Change-Id: I3e309fe1a7ba96ad1d1ce01f4d2aa18178f59244
BUG: 1095097
Signed-off-by: Avra Sengupta <asengupt@redhat.com>
Reviewed-on: http://review.gluster.org/7686
Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Kaushal M <kaushal@redhat.com>
2014-05-08 03:44:41 -07:00
Harshavardhana
a05c579f1c meta: print in json for stack/frames, cmdline and version
- Follow formatting rules based on RFC4627 -
  http://www.ietf.org/rfc/rfc4627.txt
- Add checks for json in regression test meta.t

Change-Id: I480d32ce042b202d3ed8939623c629a03b458551
BUG: 1089216
Signed-off-by: Harshavardhana <harsha@harshavardhana.net>
Reviewed-on: http://review.gluster.org/7653
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2014-05-05 17:29:59 -07:00
Anand Avati
8160399a36 meta: (re-)Implement Meta translator
The meta translator exposes details about glusterfs itself
in the form of a virtual namespace.

Loading the translator on the client side creates the
meta virtual view under $mntpoint/.meta by default. The
directory is not listed (even with ls -a) and can be
accessed by doing a "cd /mnt/.meta"

Change-Id: I5ffdf39203841a9562a8280a1f79dc76d4dded5d
BUG: 1089216
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/7509
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Harshavardhana <harsha@harshavardhana.net>
2014-05-05 17:28:55 -07:00
Atin Mukherjee
e73fc9939a glusterd : Volname, brickpath & volfpath length validation
While creating a volume and adding a brick validation for _POSIX_PATH_MAX is
done on absolute pathname instead of relative pathname due to which a brickpath
having less than _POSIX_PATH_MAX may also fail the validation if the directory
length is greater than (_POSIX_PATH_MAX -strlen(brickpath/volume name).

Also this fix addresses one cli response message correction which says the
volume file is too long instead of brick path is too long (when brickpath
length validation doesn't fail and vol file length validation fails.)

It is also important to note that with the current design of volfile naming, it
can not be guranteed that volname and brickpath can have max of _POSIX_PATH_MAX
characters.

Change-Id: I1283d1f9dea96ae797620002c8723719f26a866d
BUG: 1085330
Signed-off-by: Atin Mukherjee <amukherj@redhat.com>
Reviewed-on: http://review.gluster.org/7420
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2014-05-03 08:20:06 -07:00
Xavier Hernandez
c7838fbd6a storage/posix: do not dereference gfid symlinks before posix_handle_mkdir_hashes()
Whenever a new directory is created, its corresponding gfid file must
also be created. This was done first calling MAKE_HANDLE_PATH() to get
the path of the gfid file, then calling posix_handle_mkdir_hashes() to
create the parent directories of the gfid, and finally creating the
soft-link.

In normal circumstances, the gfid we want to create won't exist and
MAKE_HANDLE_PATH() will return a simple path to the new gfid. However if
the volume is damaged and a self-heal is running, it is possible that we
try to create an already existing gfid. In this case, MAKE_HANDLE_PATH()
will return a path to the directory instead of the path to the gfid.

To solve this problem, every time a path to a gfid is needed, a call to
MAKE_HANDLE_ABSPATH() is made instead of the call to MAKE_HANDLE_PATH().

Change-Id: Ic319cc38c170434db8e86e2f89f0b8c28c0d611a
BUG: 859581
Signed-off-by: Xavier Hernandez <xhernandez@datalab.es>
Reviewed-on: http://review.gluster.org/5075
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2014-05-02 09:34:45 -07:00
ggarg
dd5e318e02 glusterd: Differentiate rebalance status and remove-brick status messages
previously when user triggred 'gluster volume remove-brick VOLNAME
BRICK start' then command' gluster volume rebalance <volname> status'
showing output even user has not triggred "rebalance start" and when
user triggred 'gluster volume rebalance <volname> start' then command
'gluster volume remove-brick VOLNAME BRICK status' showing output even
user has not run rebalance start and remove brick start.

regression test failed in previous patch. file test/dht.rc and
test/bug/bug-973073 edited to avoid regression test failure.

now with this fix it will differentiate rebalance and remove-brick
status messages.

Signed-off-by: ggarg <ggarg@redhat.com>

Change-Id: I7f92ad247863b9f5fbc0887cc2ead07754bcfb4f
BUG: 1089668
Reviewed-on: http://review.gluster.org/7517
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
Reviewed-by: Humble Devassy Chirammal <humble.devassy@gmail.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2014-05-02 09:31:45 -07:00
Joseph Fernandes
0f56f0ce2d glusterd/snapshot: Activation and De-activation of snapshot
Previously, snapshots by default were activated on creation and there was
no option to activate or deactivate them on demand.

This will allow the user to activate and deactivate on demand.
The CLI goes as follows

1) Activate the snap using a command "gluster snapshot activate <snapname> [force]"
2) Deactivate the snap using a command "gluster snapshot deactivate <snapname>"

Note: Even now the snapshot will be activated during creation.

Change-Id: I0946d800780f26c63fa1fcaf29aabc900140448f
BUG: 1061685
Signed-off-by: Joseph Fernandes <josferna@redhat.com>
Reviewed-on: http://review.gluster.org/7476
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijaikumar Mallikarjuna <vmallika@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2014-05-02 09:27:46 -07:00
Joseph Fernandes
f63fbca754 [glusterd/snapshot] snapshot create force option
Implement force option in snapshot create i.e

1) Creation of snapshot fails if the original volume
bricks are down
2) With a force option creation of snapshot will continue
even if the original volume bricks are down.

This was the fix for bugs 1089527 and 1083502

Change-Id: I8de0242adf8ee0af00db9fa8701d86fabc12e7fc
BUG: 1090042
Signed-off-by: Joseph Fernandes <josferna@redhat.com>
Reviewed-on: http://review.gluster.org/7520
Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2014-04-30 23:27:23 -07:00
Sachin Pandit
e2034a84d2 snapshot/config : Fix for bug which states gluster snapshot config
command should only accept the decimal numeric value.

Syntax : gluster snapshot config [volname]
                                 [snap-max-hard-limit <count>]
                                 [snap-max-soft-limit <percentage>]

Problem : Snapshot config used to consider the alphanumeric value
staring with digit as an integer (Example: "9abc" is converted to "9").

Solution : Refined the code to check if the entered value is numeric.

This patch also fixes some of the minor problems related to snapshot
config.
1) Output correction in gluster snapshot config snap-max-soft-limit.
2) setting the soft limit to greater than 100% displays that "Invalid
snap-max-soft-limit 0". The error message used to display "zero" in
the output, Changed this to display relevant value.
3) Setting greater than allowed snap-max-hard-limit output needs to
have space in between.

Change-Id: Ie7c7045722fe57b2b3c50c873664b67c28eb3853
BUG: 1087203
Signed-off-by: Sachin Pandit <spandit@redhat.com>
Reviewed-on: http://review.gluster.org/7457
Reviewed-by: Vijaikumar Mallikarjuna <vmallika@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2014-04-29 22:02:01 -07:00
Varun Shastry
b00ef83d2b feature/quota: Logging corrections
This patch solves the inconsistent quota usage logging when soft limit reached.

Change-Id: I47e7f1e65ed4b8306a999a20cc8f6b1772d47627
BUG: 1087198
Signed-off-by: Varun Shastry <vshastry@redhat.com>
Reviewed-on: http://review.gluster.org/7451
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2014-04-29 09:46:07 -07:00
ggarg
a3c12fdabe cli: Error out when character "." is available in volume name
previously it was failed without showing any information why it is failed.

Now with this fix, when "." or any non alphanumeric character
present in volume name, it will give error messages

Change-Id: I17e8e69c08345c4d760f3ba333fe841e754bc9c8
BUG: 921215
Signed-off-by: ggarg <ggarg@redhat.com>
Reviewed-on: http://review.gluster.org/7364
Reviewed-by: Humble Devassy Chirammal <humble.devassy@gmail.com>
Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
2014-04-28 08:42:10 -07:00
Niels de Vos
d2cdc392ac Update references to the maillinglist to gluster-devel@gluster.org
gluster-devel@nongnu.org has moved to gluster-devel@gluster.org. All
occurrences in the current (non legacy) documentation and code have been
adjusted.

Change-Id: I053162e633f7ea14fd3eed239ded017df165147c
BUG: 1091705
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: http://review.gluster.org/7573
Reviewed-by: Justin Clift <justin@gluster.org>
Reviewed-by: Anand Avati <avati@redhat.com>
Tested-by: Anand Avati <avati@redhat.com>
2014-04-27 21:29:36 -07:00
Harshavardhana
a3cb38e3ed build: MacOSX Porting fixes
git@forge.gluster.org:~schafdog/glusterfs-core/osx-glusterfs

Working functionality on MacOSX

 - GlusterD (management daemon)
 - GlusterCLI (management cli)
 - GlusterFS FUSE (using OSXFUSE)
 - GlusterNFS (without NLM - issues with rpc.statd)

Change-Id: I20193d3f8904388e47344e523b3787dbeab044ac
BUG: 1089172
Signed-off-by: Harshavardhana <harsha@harshavardhana.net>
Signed-off-by: Dennis Schafroth <dennis@schafroth.com>
Tested-by: Harshavardhana <harsha@harshavardhana.net>
Tested-by: Dennis Schafroth <dennis@schafroth.com>
Reviewed-on: http://review.gluster.org/7503
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2014-04-24 14:41:48 -07:00
Jeff Darcy
9c13471b10 tests: print info about failed tests
It's a bit frustrating to run a test normally (e.g. during development),
see a failure, and then have to re-run it with DEBUG=1 to see what
actually failed.  This tiny patch prints out the command line of the
test that just failed, even if DEBUG=0 (in fact only if DEBUG=0 since
otherwise it would be redundant).

Change-Id: Icffa096d0bcc6f35176f0e47d9d1bc538698ca2c
Signed-off-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-on: http://review.gluster.org/7537
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2014-04-23 20:49:13 -07:00
Santosh Kumar Pradhan
00e247ee44 gNFS: Support wildcard in RPC auth allow/reject
RFE: Support wildcard in "nfs.rpc-auth-allow" and
"nfs.rpc-auth-reject". e.g.
  *.redhat.com
  192.168.1[1-5].*
  192.168.1[1-5].*, *.redhat.com, 192.168.21.9

  Along with wildcard, support for subnetwork or IP range e.g.
  192.168.10.23/24

The option will be validated for following categories:
1) Anonymous i.e. "*"
2) Wildcard pattern i.e. string containing any ('*', '?', '[')
3) IPv4 address
4) IPv6 address
5) FQDN
6) subnetwork or IPv4 range

Currently this does not support IPv6 subnetwork.

Change-Id: Iac8caf5e490c8174d61111dad47fd547d4f67bf4
BUG: 1086097
Signed-off-by: Santosh Kumar Pradhan <spradhan@redhat.com>
Reviewed-on: http://review.gluster.org/7485
Reviewed-by: Poornima G <pgurusid@redhat.com>
Reviewed-by: Harshavardhana <harsha@harshavardhana.net>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2014-04-22 23:40:26 -07:00
Avra Sengupta
29bccc2ed1 gluster: GlusterFS Volume Snapshot Feature
This is the initial patch for the Snapshot feature. Current patch
includes following features:
* Snapshot create
* Snapshot delete
* Snapshot restore
* Snapshot list
* Snapshot info
* Snapshot status
* Snapshot config

Change-Id: I2f46920c0d61c515f6a60e0f8b46fff886d9f6a9
BUG: 1061685
Signed-off-by: shishir gowda <sgowda@redhat.com>
Signed-off-by: Sachin Pandit <spandit@redhat.com>
Signed-off-by: Vijaikumar M <vmallika@redhat.com>
Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com>
Signed-off-by: Rajesh Joseph <rjoseph@redhat.com>
Signed-off-by: Joseph Fernandes <josferna@redhat.com>
Signed-off-by: Avra Sengupta <asengupt@redhat.com>
Reviewed-on: http://review.gluster.org/7128
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2014-04-11 16:29:17 -07:00
Niels de Vos
8235de1898 rpc: warn and truncate grouplist if RPC/AUTH can not hold everything
The GlusterFS protocol currently uses AUTH_GLUSTERFS_V2 in the RPC/AUTH
header. This header contains the uid, gid and auxiliary groups of the
user/process that accesses the Gluster Volume.

The AUTH_GLUSTERFS_V2 structure allows up to 65535 auxiliary groups to
be passed on. Unfortunately, the RPC/AUTH header is limited to 400 bytes
by the RPC specification: http://tools.ietf.org/html/rfc5531#section-8.2

In order to not cause complete failures on the client-side when trying
to encode a AUTH_GLUSTERFS_V2 that would result in more than 400 bytes,
we can calculate the expected size of the other elements:

    1 | pid
    1 | uid
    1 | gid
    1 | groups_len
   XX | groups_val (GF_MAX_AUX_GROUPS=65535)
    1 | lk_owner_len
   YY | lk_owner_val (GF_MAX_LOCK_OWNER_LEN=1024)
  ----+-------------------------------------------
    5 | total xdr-units

  one XDR-unit is defined as BYTES_PER_XDR_UNIT = 4 bytes
  MAX_AUTH_BYTES = 400 is the maximum, this is 100 xdr-units.
  XX + YY can be 95 to fill the 100 xdr-units.

  Note that the on-wire protocol has tighter requirements than the
  internal structures. It is possible for xlators to use more groups and
  a bigger lk_owner than that can be sent by a GlusterFS-client.

This change prevents overflows when allocating the RPC/AUTH header. Two
new macros are introduced to calculate the number of groups that fit in
the RPC/AUTH header, when taking the size of the lk_owner in account. In
case the list of groups exceeds the maximum possible, only the first
groups are passed over the RPC/GlusterFS protocol to the bricks.
A warning is added to the logs, so that most system administrators will
get informed.

The reducing of the number of groups is not a new inventions. The
RPC/AUTH header (AUTH_SYS or AUTH_UNIX) that NFS uses has a limit of 16
groups. Most, if not all, NFS-clients will reduce any bigger number of
groups to 16. (nfs.server-aux-gids can be used to workaround the limit
of 16 groups, but the Gluster NFS-server will be limited to a maximum of
93 groups, or fewer in case the lk_owner structure contains more items.)

Change-Id: I8410e59d0fd246d601b54b961d3ae9cb5a858c10
BUG: 1053579
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: http://review.gluster.org/7202
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Harshavardhana <harsha@harshavardhana.net>
Reviewed-by: Santosh Pradhan <spradhan@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2014-04-08 10:50:52 -07:00
Justin Clift
aef305334c tests: Increase bug-865825.t wait time for self-heal daemon
BUG: 1084653
Change-Id: I057bbd2e50803344552314b32d2d0e6240bf9604
Signed-off-by: Justin Clift <justin@gluster.org>
Reviewed-on: http://review.gluster.org/7404
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2014-04-07 22:52:11 -07:00
Ravishankar N
0c20b17c09 tests/afr: self-heal
Basic functional tests related to self-heal.

arequal-checksum.c is taken from
https://github.com/raghavendrabhat/arequal after consent from all
authors.

Change-Id: I43facc31c61375f4dbe58bbb46238e15df5c9011
BUG: 1080759
Signed-off-by: Ravishankar N <ravishankar@redhat.com>
Reviewed-on: http://review.gluster.org/7357
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2014-04-07 09:49:16 -07:00
Ravishankar N
49fbc578ef tests/afr: select correct read-child for entry OPs.
Change-Id: If375c937579a18d603ed70232130a4664060e9d6
BUG: 1080759
Signed-off-by: Ravishankar N <ravishankar@redhat.com>
Reviewed-on: http://review.gluster.org/7344
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2014-04-04 06:35:27 -07:00
Justin Clift
27bc6a07b4 tests: Adjust test 767095 to cope with long hostnames
BUG: 1084147
Change-Id: Ie1ff8852a501690e681072c54620d305b5e20d6a
Signed-off-by: Justin Clift <justin@gluster.org>
Reviewed-on: http://review.gluster.org/7395
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2014-04-04 06:29:17 -07:00
Atin Mukherjee
5dedef81b6 cli: remove-brick no longer defaults to commit-force
Problem : When gluster volume remove-brick is executed with out any option, it
defaults to force commit which results in data loss.

Fix : remove-brick can not be executed with out explicit option, user needs to
provide the option in the command line else the command will throw back an usage
error.

Earlier usage : volume remove-brick <VOLNAME> [replica <COUNT>] <BRICK> ...
                [start|stop|status|commit|force]

Current usage : volume remove-brick <VOLNAME> [replica <COUNT>] <BRICK> ...
                <start|stop|status|commit|force>

Change-Id: I2a49131f782a6c0dcd03b4dc8ebe5907999b0b49
BUG: 1077682
Signed-off-by: Atin Mukherjee <amukherj@redhat.com>
Reviewed-on: http://review.gluster.org/7292
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Shyamsundar Ranganathan <sam.somari@gmail.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2014-04-02 12:04:11 -07:00
Varun Shastry
1c1b8269d9 tests/quota: Wait till the rebalance is complete
Change-Id: Ia6f0c81fb1542ce1de965a69a61535691df056c3
BUG: 1077159
Signed-off-by: Varun Shastry <vshastry@redhat.com>
Reviewed-on: http://review.gluster.org/7380
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Justin Clift <justin@gluster.org>
Tested-by: Justin Clift <justin@gluster.org>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2014-04-02 07:53:15 -07:00
Ravishankar N
e7dcc7f824 tests/afr: gfid mismatch test
Change-Id: I12bae9c4035d5b28292e8085a5b600a3e22abaf4
BUG: 1080759
Signed-off-by: Ravishankar N <ravishankar@redhat.com>
Reviewed-on: http://review.gluster.org/7342
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
2014-03-30 05:31:53 -07:00
Ravishankar N
36c7f83415 tests/afr: select correct read-child for data OPs.
Change-Id: If84bc489b6c45bde3bdb858da5f1600cea78c8a5
BUG: 1080759
Signed-off-by: Ravishankar N <ravishankar@redhat.com>
Reviewed-on: http://review.gluster.org/7345
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2014-03-30 01:59:18 -07:00
Susant Palai
9a3de81fe5 DHT/Rebalance : Hard link Migration Failure
Probelm : __is_file_migratable used to return ENOTSUP
for all the cases. Hence, it will add to the failure
count. And the remove-brick status will show failure
for all the files.

Solution : Added 'ret = -2' to gf_defrag_handle_hardlink to
be deemed as success. Otherwise dht_migrate_file will try to
migrate each of the hard link, which not intended.

Change-Id: Iff74f6634fb64e4b91fc5d016e87ff1290b7a0d6
BUG: 1066798
Signed-off-by: Susant Palai <spalai@redhat.com>
Reviewed-on: http://review.gluster.org/7124
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2014-03-30 00:55:43 -07:00
Pranith Kumar K
283ae136d4 tests: Stale file lookup test
Change-Id: I6edfc5b7ee42677e92d9cff6a7180692d20e9310
BUG: 1080759
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/7341
Reviewed-by: Ravishankar N <ravishankar@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2014-03-28 19:21:06 -07:00
ShyamsundarR
31e34cfd72 log: enhance gluster log format with message ID and standardize errno reporting
Currently there are quite a slew of logs in Gluster that do not
	lend themselves to trivial analysis by various tools that help
	collect	and monitor logs, due to the textual nature of the logs.

	This FEAT is to make this better by giving logs message IDs so
	that the tools do not have to do complex log parsing to break
	it down to problem areas and suggest troubleshooting options.

	With this patch, a new set of logging APIs are introduced that
	take additionally a message ID and an error number, so as to
	print the message ID and the descriptive string for the error.

	New APIs:
		- gf_msg, gf_msg_debug/trace, gf_msg_nomem, gf_msg_callingfn

	These APIs follow the functionality of the previous gf_log*
	counterparts, and hence are 1:1 replacements, with the delta
	that, gf_msg, gf_msg_callingfn take additional parameters as
	specified above.

	Defining the log messages:
	Each invocation of gf_msg/gf_msg_callingfn, should provide an ID
	and an errnum (if available). Towards this, a common message id
	file is provided, which contains defines to various messages and
	their respective strings. As other messages are changed to the
	new infrastructure APIs, it is intended that this file is edited
	to add these messages as well.

	Framework enhanced:
	The logging framework is also enhanced to be able to support
	different logging backends in the future. Hence new configuration
	options for logging framework and logging formats are introduced.

	Backward compatibility:
	Currently the framework supports logging in the traditional
	format, with the inclusion of an error string based on the errnum
	passed in. Hence the shift to these new APIs would retain the log
	file names, locations, and format with the exception of an
	additional error string where applicable.

	Testing done:
	Tested the new APIs with different messages in normal code paths
	Tested with configurations set to gluster logs (syslog pending)
	Tested nomem variants, inducing the message in normal code paths
	Tested ident generation for normal code paths (other paths
		pending)
	Tested with sample gfapi program for gfapi messages
	Test code is stripped from the commit

	Pending work (not to be addressed in this patch (future)):
	- Logging framework should be configurable
	- Logging format should be configurable
	- Once all messages move to the new APIs deprecate/delete older
	  APIs to prevent misuse/abuse using the same
	- Repeated log messages should be suppressed (as a configurable
	  option)
	- Logging framework assumes that only one init is possible, but
	  there is no protection around the same (in existing code)
	- gf_log_fini is not invoked anywhere and does very little
	  cleanup (in existing code)
	- DOxygen comments to message id headers for each message

Change-Id: Ia043fda99a1c6cf7817517ef9e279bfcf35dcc24
BUG: 1075611
Signed-off-by: ShyamsundarR <srangana@redhat.com>
Reviewed-on: http://review.gluster.org/6547
Reviewed-by: Krutika Dhananjay <kdhananj@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2014-03-28 04:53:37 -07:00
Pranith Kumar K
21c282ef31 cluster/afr: Sparse file self-heal canges
- Fix boundary condition for offset
- Honour data-self-heal-algorithm option
- Added tests for sparse file self-healing

Change-Id: I14bb1c9d04118a3df4072f962fc8f2f197391d95
BUG: 1080707
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/7339
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2014-03-26 22:43:14 -07:00
Susant Palai
53194718bb Glusterd/Remove-brick: Reconfigure the nfs server volfile upon remove-brick start
Problem : For remove-brick start operation  all client volfiles are reconfigured
except nfs server volfile. Hence, even after layout is fixed by the rebalance
process, the nfs clients dont see the change and go on creating directories and
files in the decommissioned brick which leads to data loss after remove-brick
commit.

Solution : Reconfigure the nfs server volfile for remove-brick start

credit: kaushal@redhat.com
        spalai@redhat.com

Change-Id: Ib8cd8b45a9e1f888d5e00dff65cdf77c1613a2af
BUG: 1070734
Signed-off-by: Susant Palai <spalai@redhat.com>
Reviewed-on: http://review.gluster.org/7162
Reviewed-by: Kaushal M <kaushal@redhat.com>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2014-03-22 05:28:35 -07:00
Anand Avati
6d3739292b cluster/afr: refactor
- Remove client side self-healing completely (opendir, openfd, lookup)
- Re-work readdir-failover to work reliably in case of NFS
- Remove unused/dead lock recovery code
- Consistently use xdata in both calls and callbacks in all FOPs
- Per-inode event generation, used to force inode ctx refresh
- Implement dirty flag support (in place of pending counts)
- Eliminate inode ctx structure, use read subvol bits + event_generation
- Implement inode ctx refreshing based on event generation
- Provide backward compatibility in transactions
- remove unused variables and functions
- make code more consistent in style and pattern
- regularize and clean up inode-write transaction code
- regularize and clean up dir-write transaction code
- regularize and clean up common FOPs
- reorganize transaction framework code
- skip setting xattrs in pending dict if nothing is pending
- re-write self-healing code using syncops
- re-write simpler self-heal-daemon

Change-Id: I1e4080c9796c8a2815c2dab4be3073f389d614a8
BUG: 1021686
Signed-off-by: Anand Avati <avati@redhat.com>
Reviewed-on: http://review.gluster.org/6010
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2014-03-22 05:25:57 -07:00
Edward Shishkin
2010fb2bc6 Added test case for crypt translator
Change-Id: I7a28a1285c19c3279c2f71e9d9914cf14c761858
BUG: 1030058
Signed-off-by: Edward Shishkin <edward@redhat.com>
Reviewed-on: http://review.gluster.org/6504
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2014-02-27 22:33:32 -08:00
Niels de Vos
b0515e2a4a write-behind: track filesize when doing extending writes
A program that calls mmap() on a newly created sparse file, may receive
a SIGBUS signal. If SIGBUS is not handled, a segmentation fault will
occur and the program will exit.

A bug in the write-behind translator can cause the creation of a sparse
file created with open(), seek(), write() to be cached. The last write()
may not be sent to the server, until write-behind deems this necessary.

* open(.., O_TRUNC, ...)/creat() the file, it is 0 bytes big
* seek() into the file, use offset 31
* write() 1 byte to the file
* the range from byte 0-30 are unwritten so called 'sparse'

The following illustration tries to capture this:

    Legend:
    [ = start of file
    _ = unallocated/unwritten bytes
    # = allocated bytes in the file
    ] = end of file

    [_______________#]
     |              |
     '- byte 0      '- byte 31

Without this change, reading from byte 0-30 will return an error, and
reading the same area through an mmap()'d pointer will trigger a SIGBUS.
Reading from this range did not trigger the outstanding write() to be
flushed. The brick that receives the read() (translated over the network
from mmap()) does not know that the file has been extended, and returns
-EINVAL. This error gets transported back from the brick to the
glusterfs-fuse client, and translated by the Linux kernel/VFS into
SIGBUS triggered by mmap().

In order to solve this, a new attribute to the wb_inode structure is
introduced; the current size of the file. All FOPs that can modify the
size, are expected to update wb_inode->size. This makes it possible for
extending writes with an offset bigger than EOF to mark the unwritten
area as modified/pending.

Change-Id: If5ba6646732e6be26568541ea9b12852a5d0b988
BUG: 1058663
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: http://review.gluster.org/6835
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2014-02-27 21:56:48 -08:00
Prashanth Pai
f69e855116 feature/compress: Validate option and enable doc
* Validate network.compression option
* Enable descriptions of xlator configurable options
* Improve indentation in code
* Make network.compression.mode not configurable by user.
  This is similar to "iam-self-heal-daemon" option in AFR xlator.

Fixes BUGs: 1065658, 1065640, 1065655

Change-Id: I99d82b574ee0e5c8c2baf5f5d52dbf8d015d330a
BUG: 1065640
Signed-off-by: Prashanth Pai <ppai@redhat.com>
Reviewed-on: http://review.gluster.org/7024
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2014-02-26 09:18:35 -08:00
Krishnan Parthasarathi
a78dfebb73 add build-gfid option to enable pgfid tracking ...
.. for inode to pathname mapping

Change-Id: I0486d85b02e86d739fc1d8ea16d118fb666abf60
BUG: 1064863
Signed-off-by: Krishnan Parthasarathi <kparthas@redhat.com>
Reviewed-on: http://review.gluster.org/6989
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2014-02-14 04:18:26 -08:00
Ravishankar N
8148dc2eab storage/posix: perform chmod after chown.
Problem:
When a replica brick is added to a volume, set-user-ID and set-group-ID
permission bits of files are not set correctly in the new brick. The issue
is in the posix_setattr() call where we do a chmod followed by a chown.

But according to the man pages for chown:
When the owner or group of an executable file are changed by an unprivileged
user the S_ISUID and S_ISGID mode bits are cleared.  POSIX does not specify
whether this also  should  happen  when  root does the chown().

Fix:
Swap the chmod and chown calls in posix_setattr()

Change-Id: I094e47a995c210d2fdbc23ae7a5718286e7a9cf8
BUG: 1058797
Signed-off-by: Ravishankar N <ravishankar@redhat.com>
Reviewed-on: http://review.gluster.org/6862
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
2014-02-11 22:13:29 -08:00
Raghavendra Bhat
28209283a6 protocol/server: do not do root-squashing for trusted clients
* As of now clients mounting within the storage pool using that machine's
  ip/hostname are trusted clients (i.e clients local to the glusterd).

* Be careful when the request itself comes in as nfsnobody (ex: posix tests).
  So move the squashing part to protocol/server when it creates a new frame
  for the request, instead of auth part of rpc layer.

* For nfs servers do root-squashing without checking if it is trusted client,
  as all the nfs servers would be running within the storage pool, hence will
  be trusted clients for the bricks.

* Provide one more option for mounting which actually says root-squash
  should/should not happen. This value is given priority only for the trusted
  clients. For non trusted clients, the volume option takes the priority. But
  for trusted clients if root-squash should not happen, then they have to be
  mounted with root-squash=no option. (This is done because by default
  blocking root-squashing for the trusted clients will cause problems for smb
  and UFO clients for which the requests have to be squashed if the option is
  enabled).

* For geo-replication and defrag clients do not do root-squashing.

* Introduce a new option in open-behind for doing read after successful open.

Change-Id: I8a8359840313dffc34824f3ea80a9c48375067f0
BUG: 954057
Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com>
Reviewed-on: http://review.gluster.org/4863
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vbellur@redhat.com>
2014-02-10 23:32:05 -08:00