508 Commits

Author SHA1 Message Date
Piotr Sarna
383ca240a7 common, mount: Add NFS4 acls generated from mode
This commit makes NFS4 ACL handler generate an entry from POSIX mode
if access control lists are not present in a file.
Thanks to this change, using nfs4_setfacl tool to append ACL entries
will work correctly.

This commit comes with a test that checks whether nfs4_setacl
and nfs4_getacl work fine.

Fixes #644

Change-Id: I8dbaeceb76d770744e9cd5b316604009e58f83af
2018-06-26 16:59:13 +02:00
Hazeman
96c2d9f64b tests: Add ec2 conversion test
This commit adds test for validating
name conversion to ec2 format.

Change-Id: Ie61609c56f023b98d1f37b9570c736d0ca6f5311
2018-06-20 14:57:08 +02:00
Hazeman
e76c38623c chunkserver: Convert old EC chunks to new format
This commit converts old EC chunks to new format. The procedure is following

1. For data parts chunk name is converted so that _ec_ is replaced to _ec2_.
2. Parity parts for slice type with parity count < 5 are converted
   the same way as data parts.
3. Parity parts for slice type with parity count >= 5 are removed and must
   be later regenerated by master server.

Change-Id: I5b1e65ee3b034eacd08154cfac28355073fee85a
2018-06-18 12:19:42 +02:00
Hazeman
a3361dcf8b tests: Fix timeouts
This commit fixes timeouts in following tests:
  * test_mapall
  * test_cs_failure_during_xor_write
  * test_simultaneous_write_read

Change-Id: I994beeb37c316cc181c58b7bda201340832a9b1f
2018-06-18 11:56:03 +02:00
Hazeman
97b7a6fc99 tests: Add test for simultaneous write and read
This commit adds test that writes to file
and at the same time tries to read it.

Change-Id: I7790752b0d7fc02659b37980b59890c1b83b7ade
2018-06-14 12:11:53 +02:00
Hazeman
7085958ee6 mount, fuse: Add mfsmount3
This commit adds mfsmount3 executable linked to FUSE3 library.

Change-Id: I327e8afbf1370c4f53c09b7538e5cc35558cac5b
2018-06-13 16:07:26 +02:00
Hazeman
4af83dbb53 mount: Migrate to fuse3
This commit migrates lizardfs to fuse3.

Change-Id: I693405713049fabdcfac8167eb43a13d539700ed
2018-06-12 14:04:24 +02:00
Hazeman
96fee8023c tests: Fix problems in tests
This commit fixes issues in following tests
  * test_backwards_changelog_compatibility - race condition
  * test_legacy_old_lizardfs_upgrade - compact_vector with new compilers
  * ShortSystemTests.test_shadow_reject - issue with nc behaviour

Change-Id: If6a47e781a3370532d12bead11ae820832f0be23
2018-06-12 13:45:41 +02:00
Hazeman
722153f13e tests: Fix timeout in test_pread_eio_in_chunkserver_all_disks
This commit fixes timeout in test test_pread_eio_in_chunkserver_all_disks.

Change-Id: I6a813de902974e2f191167ffdb8a788fd133b686
2018-06-11 11:50:05 +02:00
Hazeman
4cd375043d tests: Fix timeouts
This commit fixes timeouts in tests test_multiple_truncates
and test_pread_eio_in_chunkserver_all_disks.

Change-Id: I580ab98f3193b8a35993397cf1fa498a6ea70127
2018-06-08 13:21:24 +02:00
Hazeman
3cd2ed9ce3 test: Fix timeouts and xor|ec_goal_with_labels
This commit changes timeouts in few tests so
they can work with vagrant. Also there are fixes
in tests xor|ec_goal_with_labels to take into account
changes made in commit 3174.

Change-Id: I798d994fdd75e558b1d0d5cfea1ea4a75e6b8101
2018-06-01 14:14:15 +02:00
Hazeman
dee80645ca tests: Add valgrind suppress entry
This commit adds entry to valgrind supress list.
In mount valgrind reports leak of stacks allocated for working threads.

Change-Id: I4bd3aed18c8586b955586dc7a2f05a023b6a5938
2018-05-23 11:31:03 +02:00
Hazeman
2dc77f8efd tests: Increase CPU limit for chunk loop
This commit increases CPU limit for chunk loop processing.
Also timeouts in test chunk_type_conversion are increased.

Change-Id: If5308203619d7527cb06faa29fb02cf248fc6daf
2018-05-22 14:43:23 +02:00
Hazeman
b2c6bd3abb tests: Add test for nfs-ganesha
This commit adds test for nfs-ganesha plugin.

Change-Id: Ieaf34bd037cbdae9afafdcf26c9d1d48b10f7279
2017-12-13 13:44:44 +01:00
Hazeman
7854e69714 test: Remove race condition in test admin_magic_recalculate_metadata_checksum
This commit improves timing issues in test
test_admin_magic_recalculate_metadata_checksum.

Change-Id: I7b15f11d7262ae00fe2f0642490c10813216fdda
2017-11-27 11:17:43 +01:00
Piotr Sarna
36e1fcb999 tests: Update HDD_TEST_FREQ value in tests
This commit changes HDD_TEST_FREQ value from undocumented 0
to new semantics to turn off chunk testing.

Change-Id: I7facbb5e6cd3916b94fb2b497e4fab529af8c88a
2017-11-23 13:46:32 +02:00
Piotr Sarna
cf11fe9a4e master: Prevent cross-label replication for safe chunks
This commit prevents chunks from being replicated across their labels
if they are not endangered.

Closes #612

Change-Id: I72ecb3a49e400d56517d21f1a5af1267b0d99a82
2017-11-22 13:17:06 +01:00
Hazeman
470c8dd636 tests: Remove explicit calls to LizardFS binaries
This commit replaces explicit calls to programs mfschunkserver and mfsmaster
with wrapper functions.

Change-Id: I2f090029c53c6f29a55c1a036449b688abeb5902
2017-11-03 10:43:41 +01:00
Piotr Sarna
f78b04501a tests: Remove ramdisk usage from test_build_lizardfs
This commit drops USE_RAMDISK=YES option from a test that builds lizardfs
from source, because it's space-consuming. LizardFS source with external
dependencies takes up a lot of space, as does compiling it all.
2GB ramdisk proved to be not enough to handle this test, as it keeps
failing with ENOSPC recently. There is no valid reason for this test
to require a ramdisk, so it is removed. Now this test can be run on
test machines even if their ramdisk is <= 2GB in size.

Change-Id: I4b820937237bfd51f02a846d047d4484791a7413
2017-10-27 10:53:19 +02:00
Piotr Sarna
dcfe5d8eeb chunkserver, tests: Restore correct format in prefetching xor test
This commit fixes incorrect formatting of debug output which
broke prefetching_xor_stripes test from ShortSystemTests suite.

Change-Id: I7b02878fe9ae5338fcc01b1c5013d133765b47a0
2017-10-27 10:49:51 +02:00
Hazeman
a453ba4116 tests: Add test for find between writes
This commit adds tests for case when find (or ls) is used
between writes to file.

Change-Id: Ie740e76c11ad73d6382bdb850321fbcff75f7303
2017-10-11 10:11:39 +02:00
Piotr Sarna
dd215ee016 all: Modify logging back-end
This commit changes logging back-end to spdlog library.
It also enables adding custom log files with custom priorities.

Change-Id: If51213204724b23f608df6c5c77ea318956699f5
2017-10-09 14:18:35 +02:00
Hazeman
198a936335 tests: Fix tests
This commit fixes following tests

test_build_lizardfs - increase timeout
test_polonaise - increase timeout and moved to LongSystemTests
test_legacy_new_mount_with_old_lizardfs - change version of old
                                          lizardfs to 3.11.0

Change-Id: I7862f7f661420bc82b3497f726ba9ed3f24d0c71
2017-10-09 14:09:10 +02:00
Hazeman
d3d108ff9e nfs-ganesha: Add nfs-ganesha plugin
This commit adds LizardFS FSAL plugin for nfs-ganesha.

Change-Id: Idfa38f83a5dbc61bb208090029013a66e51896ed
2017-09-15 13:30:41 +02:00
Piotr Sarna
d23ccb33e6 tests: Add a test for truncating ec with not enough copies
This commit checks if truncating ec/xor with not enough copies
doesn't change chunk version, which would make the chunk unusable
after enough chunkservers are launched.

This test fails due to a bug in master, properly fixed in the next commit.

Change-Id: I81316ff9c337eeac1d61bcb498eceb46d8b6a77f
2017-09-08 15:34:03 +02:00
Hazeman
a1a69f53bc tests: Add test for RichACL
This commit adds test using test suite from RichACL tools.

Change-Id: Iad37bc568aaf94e17a659c441f9dbc30844dd1a7
2017-09-05 12:19:01 +02:00
Hazeman
38c9ad34b0 tests: Add connectathon NFS test suite
This commit adds test running connectathon test suite for NFS on LizardFS.

Change-Id: I1759ddbcbd93a3026be6b4a90ba4fa90e1095020
2017-07-21 15:39:56 +02:00
Piotr Sarna
ada4f2628b mount: Add C API skeleton
This commit complements C++ API with corresponding C API
and a shared library liblizardfs-client.so.
It is still a partial implementation and needs to be finished.

Change-Id: I8dce435966ff46a09c9acc2683e596d32170988b
2017-06-23 11:50:55 +02:00
Piotr Sarna
6ce6e26877 common, protocol: Move error codes to separate header
This commit moves error code declarations from main protocol header
to a separate file. It also removes MFS_ERROR_STRINGS macro.

Change-Id: I9c6fecc82664b2c5d093076553c6eef4ce4d10ee
2017-06-20 11:47:17 +02:00
Hazeman
dff3328764 tests: Fix tests using metadata_print
This commit fixes timing issue in metadata_print which was caused
by new directory/attribute cache.

Change-Id: I4b5bf0dfd3498afa85f2eecb7e861d652850d3c5
2017-05-09 15:14:16 +02:00
Hazeman
a725a3032f master: Extend defective files tools
This commit extends defective files tools with
information about structure inconsistency.

Change-Id: Ib47ed7bde466e41ae7a190971c4f4e59fce7ac26
2017-05-09 11:27:17 +02:00
Hazeman
a8b0abf6e5 tests: Add test for defective files tool
This commit adds test for defective files tool.

Change-Id: I3ae41f52fa7691e00f107f7c99a7100fdd33ad87
2017-05-08 16:50:13 +02:00
Fretek
75132dc2f8 tests: Update redundancy level test to new logic
This commit updates redudndancy level test to new logic.
Also limits ioretries from 30 to 3 to speed test up.

Change-Id: I096277700be86af275a6ef9fb69f044d4ddd9e3b
2017-05-08 15:57:33 +02:00
Konrad Lipinski
a3f9c7f1fe tests: Fix test_lizardfs26_upgrade
This commit fixes test_lizardfs26_upgrade by adding a 2.6.0 patch
analogous to the 3.9.4 one.

Change-Id: If37bb500784a11407e0acef22e23a59ae3a9a656
2017-05-04 14:13:21 +02:00
Maksymilian Paszkiewicz
33c1d28728 master: Add minimal extra parts option for writing
This commit adds an option to set minimal number of
extra (above required) chunk parts available in order to
successfully write a chunk.
In case of a goal which by definition has less extra
chunk parts than given minimum, this option has no effect.

Closes #227

Change-Id: I1839e5e306c1dc6311d6f2ab0b85881cfc60bf94
2017-04-21 13:50:16 +02:00
Hazeman
f68e389bdf mount: Upgrade to new directory cache
This commit replaces old directory cache with new one
based on class DirEntryCache.

Change-Id: I71dd4bb6ecc5a35622c0159aba17973f4aef5ffb
2017-04-21 11:53:29 +02:00
Hazeman
9d4253af6e common, main: Move event loop functions to common library
This commit moves event loop functions to common library.

Change-Id: Ibca352a098e549904e8a9914f0b642d1ff7cf7af
2017-03-20 15:26:05 +01:00
Maksymilian Paszkiewicz
d39f4ee70b master, tools: Add recursive remove operation
This commit adds new tool which is used for removing
files and directories recursively.
It is advised to use this command once tools for stopping tasks
are available.

Closes #254

Change-Id: I2f1043a1b707c2db66ed4d37de08eb9a51935eee
2017-03-17 11:32:04 +01:00
Piotr Sarna
af022df836 tests: Add secondary groups permission test
This commit adds a test which checks secondary groups permissions.
All test users belong to main 'lizardfstest' group.

Change-Id: Ib7f91546c97fd99a3ae871fe5f9037edc43eddda
2017-02-15 15:22:24 +01:00
Hazeman
4ea86a4ba8 common, mount: Remove PosixAclXattr class
This commit removes PosixAclXattr class and
replaces its uses with AccessControlList class.

Change-Id: I76c09c8ade338e778a74cb95876fe3ae60e2750e
2017-02-15 15:21:22 +01:00
Piotr Sarna
4b7873f722 tests: Add compliance with new versioning
This commit updates tests to acknowledge new versioning with -devel suffix.

Change-Id: Idc75739402c762cbd50e1a1c1eb22b31336438b9
2017-02-10 14:38:12 +01:00
Maksymilian Paszkiewicz
1a3b73ea4d master, tools: Add correct-only flag to filerepair
This commit adds a functionality to filerepair utility:
with correct-only flag, a chunk will be restored from previous versions
if possible, but it will never be erased.

Closes #404

Change-Id: I5e26b99c4106470f923a32e5d0f6726144fb0243
2017-01-04 14:37:56 +01:00
klemens
13b2cda9e8 all: Fix spelling mistakes
This commit fixes typos found by misspell_fixer:
https://github.com/ka7/misspell_fixer

Closes #508

Change-Id: I412d3d462d6892a75e6f5559312594dc6afca182
2017-01-04 13:59:11 +01:00
Piotr Sarna
842ca832e3 tests: Update prefetching test for readahead
This commit makes xor prefetching test compatible with client-side readahead.

Change-Id: Ie531a4ee6177ce7be49357efd10f668a2f48f8e1
2016-10-25 16:54:42 +02:00
Piotr Sarna
bcef0a566a mount: Add readahead
This commit enables readahead in LizardFS client.
Consecutive reads result in client requesting more data than necessary
in order to reduce number of operations. Current implementation of FUSE
library restricts in-kernel readahead window size to 256K, which is far
too small for distributed systems.

Change-Id: I31ccff8ec91c8c3c4f4de202bd7dce5f6444f2ab
2016-10-25 16:05:21 +02:00
Piotr Sarna
e3b599fd63 tests: Fix paths in mount
This commit fixes path incosistencies created by previous commit,
which enabled keeping core dumps of mount processes.

Change-Id: I602848aed3d83d2d9dc6e6e135be77a39caef462
2016-10-25 13:40:02 +02:00
Piotr Sarna
9069bbd331 tests: Allow keeping core dumps
This commit makes test environment try to preserve core dumps
which occurred during tests.

Change-Id: I62dae64cfedcdd6b9efa065d16e910e2742fbf25
2016-10-21 16:45:52 +02:00
Piotr Sarna
2079e435f5 tests: Amend rebalance tests
This commits amends timeouts and parameters of custom rebalance tests,
because more responsive implementation of chunk loop made these tests
unlikely to pass.

Change-Id: I957436cd9fc463ac40f2161fd71cf02b5a8c9170
2016-09-06 13:10:11 +02:00
Hazeman
0313d84784 tests: Add read/write test during scan
This commit adds read and write test when some of the chunkservers
are still in scan mode (not fully operational).

Change-Id: I81c3d310f382c45606963ca72ebc92444ad09289
2016-08-31 16:00:46 +02:00
Piotr Sarna
782313e186 tests: Add more erasure chunks tests
This commits adds handling erasure code goals to short system tests.

Change-Id: Iefae22f8ed003b657e3f9d80489093fdc2d80e0c
2016-08-31 14:25:59 +02:00