IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
Fix a couple more issues that kabi found.
- Add some error messages in failure cases
- s/malloc/zalloc/
- use vg->vgmem for lv names instead of vg->cmd->mem
Skip decoding of DM flags when device is removed.
We currently need DM flags only for add|change events. So forking
dmsetup process for removed devices is a waste of CPU time.
Udev is already quite slow, so make it just a tiny bit faster.
Add config option to enable crc checking of VG structures.
Currently it's disabled by default.
For the internal test-suite this check it is enabled.
Note: In the case the internal error is detected, debug build with
compile option DEBUG_ENFORCE_POOL_LOCKING helps to catch the source
of the problem.
Use debug pool locking functionality. So the command could check,
whether the memory in the pool has not been modified.
For lv_postoder() instead of unlocking and locking for every changed
struct status member do it once when entering and leaving function.
(mprotect would trap each such memory access).
Currently lv_postoder() does not modify other part of vg structure
then status flags of each LV with flags that are reverted back to
its original state after function exit.
Adding debuging functionality to lock and unlock memory pool.
2 ways to debug code:
crc - is default checksum/hash of the locked pool.
It gets slower when the pool is larger - so the check is only
made when VG is finaly released and it has been used more then
once.Thus the result is rather informative.
mprotect - quite fast all the time - but requires more memory and
currently it is using posix_memalign() - this could be
later modified to use dm_malloc() and align internally.
Tool segfaults when locked memory is modified and core
could be examined for faulty code section (backtrace).
Only fast memory pools could use mprotect for now -
so such debug builds cannot be combined with DEBUG_POOL.
Extend vginfo cache with cached VG structure. So if the same metadata
are use, skip mda decoding in the case, the same data are in use.
This helps for operations like activation of all LVs in one VG,
where same data were decoded giving the same output result.
Patch adds 1-to-1 connection between volume_group and lvmcache_vginfo.
Move the free_vg() to vg.c and replace free_vg with release_vg
and make the _free_vg internal.
Patch is needed for sharing VG in vginfo cache so the release_vg function name
is a better fit here.
As this flag could not have been set by the current code - removing it.
Note: because of the wrong code logic this call:
lvmcache_update_vg(correct_vg, correct_vg->status & PRECOMMITTED &
(inconsistent ? INCONSISTENT_VG : 0));
had always passed '0' - now after flag removal it's passing
PRECOMMITTED flag in - this present functinal change in this patch.
To match the original functionality - 0 had to be always passed.
More testing is needed here.
Compiler complaining that meta_lv could be used uninitialized. (Not true
because it is protected by 'clear_metadata'.) I switched to using 'lv->vg',
as it makes no difference to vg_[write|commit].
(here clvmd crashed in the middle of operation),
lock is not removed from cache - here is one example:
locking/cluster_locking.c:497 Locking VG V_vg_test UN (VG) (0x6)
locking/cluster_locking.c:113 Error writing data to clvmd: Broken pipe
locking/locking.c:399 <backtrace>
locking/locking.c:461 <backtrace>
Internal error: Volume Group vg_test was not unlocked
Code should always remove lock info from lvmcache and update counters
on unlock, even if unlock fails.
Today, we use "suppress_messages" flag (set internally in init_locking fn based
on 'ignorelockingfailure() && getenv("LVM_SUPPRESS_LOCKING_FAILURE_MESSAGES")'.
This way, we can suppress high level messages like "File-based locking
initialisation failed" or "Internal cluster locking initialisation failed".
However, each locking has its own sequence of initialization steps and these
could log some errors as well. It's quite misleading for the user to see such
errors and warnings if the "--sysinit" is used (and so the ignorelockingfailure
&& LVM_SUPPRESS_LOCKING_FAILURE_MESSAGES environment variable). Errors and
warnings from these intermediary steps should be suppressed as well if requested.
This patch propagates the "suppress_messages" flag deeper into locking init
functions. I've also added these flags for other locking types for consistency,
though it's not actually used for no_locking and readonly_locking.
Last usage was removed in Petr's commit related to VG mda repair fix
where relaxed check starts to ignore inconsistencies coming from
PVs that are marked MISSING - thus removing unused variable.
Defer the test of the function return value after the string memory is released.
Otherwise in this error path the string would present memory leak.
(Thought in this case we are already out of memory...)
Implementation described in doc/lvm2-raid.txt.
Basic support includes:
- ability to create RAID 1/4/5/6 arrays
- ability to delete RAID arrays
- ability to display RAID arrays
Notable missing features (not included in this patch):
- ability to clean-up/repair failures
- ability to convert RAID segment types
- ability to monitor RAID segment types
This should be set by default! Normally we have "activation/udev_sync = 1"
in lvm.conf (example.conf.in). But if we use lvm2 without any config file
(or without a definition within '--config' option) the DEFAULT_UDEV_SYNC
is used instead. Together with verify_udev_operations=0 (when we rely on
udev fully), this can cause races as the node could be missing when needed.
(See also https://bugzilla.redhat.com/show_bug.cgi?id=723144)
Systemd preloads file descriptors for us and passes them in for
newly spawned daemon when using on-demand fifo (or socket)
based activation.
This patch adds checks for file descriptors preloaded by
systemd and uses them instead of opening the FIFOs again
to properly support on-demand FIFO-based activation.
(We'll change FIFOs to sockets soon - but still this
part of the code will stay almost the same.)
The filename to adjust the oom score was changed in 2.6.36.
We should use oom_score_adj instead of oom_adj (which is still
there under /proc, but it's scheduled for removal in August 2012).
New oom_score_adj uses a range from -1000 (OOM_SCORE_ADJ_MIN,
disable oom killing) to 1000 (OOM_SCORE_ADJ_MAX).
Clvmd detects modifed config file before it takes lv_lock.
If the config file is changed rapidly - the change was ignored within
a seocnd ranged. This patch adds also compare of file size.
So change like some flag for 0 to 1 would pass unnoticed - but
it's quick fix for failing test suite.
FIXME: Implement inotify solution.
very reasonable amount of parallel access, although the hash tables may become
a point of contention under heavy loads. Nevertheless, there should be orders
of magnitude less contention on the hash table locks than we currently have on
block device scanning.
where the naming is left completely on lvm.
(Commited code has been different version of test).
So here it should be able to figure out new free name and create a new LV.
device that has been brought back from the dead: this sometimes fails with
clvmd (the cache is updated "too soon"). Instead, force a pvscan and rely on an
up-to-date cache as usual.
If MD linear device has set rounding (overload chunk size attribute),
the pvcreate command prints this warning:
/dev/md0 sysfs attr level not in expected format: linear
teardown sequence. (Previously the snapshot was deactivated while its
origin was active and before its removal was committed to disk, so
restarting after a crash at the point would leave corruption.)
When some target is passing empty parameters to some dm target,
report this as an internal error to better catch some broken
table construction (some mirror conversions seem to be doing
this for now).
Since some test may leave devices in suspend mode which would require
carefull order of resume operation - use '-f' to replace them with
error targets
For disable_dev - when 'error' target is used for open count - treat
return code as ok (|| true) to avoid breaking futher test processing.
The conditional is not just unnecessary, it would have been wrong. The code
is suppose to be checking if the 'splitmirrors_ARG' is negative, but it
instead is checking 'mirrors_ARG'. Rather than changing the argument being
checked, I've pulled the check entirely because 'splitmirrors_ARG' is already
guarenteed to not be negative by virtue of the fact that it is a 'int_arg'.
Negative values will be caught in _process_command_line().
When an LVM mirror is up-converted (an additional image added), it creates
a temporary mirror stack. The lower-level mirror in the stack that is
created was not being activated exclusively - violating the exclusive nature
of the original mirror. We now check for exclusive activation of a mirror
before converting it, and if found, we ensure that the temporary mirror
is also exclusively activated.
Mirrors used to be created by first creating a linear device and then adding
the other images plus the log. Now mirrors are created by creating all the
images in one go and then adding the log separately. The new way ran into
the condition that cluster mirrors cannot change the log type (in the case
of creation, from core -> disk) while the mirror is not active. (It isn't
active because it is in the process of being created.) The reason this
condition is in place is because a remote node may have the mirror active, and
we don't want to alter the log underneath it.
What we really needed was a way of checking if the mirror was active remotely
but not locally, and in that case do not allow a change of the log. I've added
this check, and cluster mirrors can now be created again.
This fn calls rm_dev_node directly - an exceptional case. It needs to check
the DM_UDEV_DISABLE_LIBRARY_FALLBACK flag directly (it's called in dm_task_run
normally where it's checked already).
We've used udev fallback code till now to check whether udev
created/removed the entries in /dev correctly and if not,
a repair was done (giving a warning messagea about that).
This patch adds a possibility to enable this additional check
and subsequent fallback only when required (debugging purposes
mostly) and trust udev completely.
So let's disable the fallback code by default and add a new
configuration option "activation/udev_fallback".
(The original code for creating the nodes will still be used
in case the device directory that is set in lvm.conf differs
from the one that udev uses and also when activation/udev_rules
is set to 0 - otherwise we would end up with no nodes/symlinks
at all)
It's useful to keep the partial flag cached - so just move the call
for vg_mark_partil_lvs() into import_vg_from_config_tree() so it gets
evaluated before it goes through the lvmcache.
This patch should not present any functional change.
Note: It is rather temporal solution - proper place is probably inside the
'read' call back - but needs some more discussion.
For now using this minor hack.
As the ACTIVATE_EXCL could be set only in clvmd code - there is no
use for this test in lv_add_mirrors() function only called from
tools context.
FIXME: Add cluster test case for this.
To avoid modification of 'read-only' volume group structure
add a new structure to pass local data around the code for LV
activation.
As origin_only is one such flag - replace this parameter with new
struct lv_activate_opts.
More parameters might eventually become part of lv_activate_opts.
transient error), stemming from the following sequence of events:
1) devices fail IO, triggering repair
2) dmeventd starts fixing up the mirror
3) during the downconversion, a new metadata version is written
--> the devices come back online here
4) the mirror device suspend/resume is called to update DM tables
5) during the suspend/resume cycle, *pre*-commit metadata is read;
however, since the failed devices are now back online, we get back
inconsistent set of precommit metadata and the whole operation fails
The patch relaxes the check that fails in step 5 above, namely by ignoring
inconsistencies coming from PVs that are marked MISSING.
This was missing in liblvm and it caused all udev-related operations to
not take effect when using liblvm, e.g. obtaining the list of devices from udev
db instead of scanning the whole /dev which also recreated the .cache as a side
effect. This was also the case with udisks-lvm-pv-export prober which is run
from within udev rules whenever the CHANGE event is fired.
and use this for the LVM critical section logic. Also report an error if
code tries to load a table while any device is known to be in the
suspended state.
(If the variety of problems these changes are showing up can't be fixed
before the next release, the error messages can be reduced to debug
level.)
are affected by the move. (Currently it's possible for I/O to become
trapped between suspended devices amongst other problems.
The current fix was selected so as to minimise the testing surface. I
hope eventually to replace it with a cleaner one that extends the
deptree code.
Some lvconvert scenarios still suffer from related problems.
Patch adds check for stripe not only in direct
LV segment but also in mirror image segment.
This prevents bugs like:
# lvcreate -i2 -l10 -n lv vg_test
# lvconvert -m1 -i1 vg_test/lv
# lvreduce -f -l1 vg_test/lv
WARNING: Reducing active logical volume to 4.00 MiB
THIS MAY DESTROY YOUR DATA (filesystem etc.)
Reducing logical volume lv to 4.00 MiB
Segment extent reduction 9 not divisible by #stripes 2
Logical volume lv successfully resized
# lvremove -f vg_test
Segment extent reduction 1 not divisible by #stripes 2
LV segment lv:0-4294967295 is incorrectly listed as being used by LV lv_mimage_0
Internal error: LV segments corrupted in lv_mimage_0.
Currently some operation with striped mirrors lead
to corrupted metadata, this patch just add detection of such
situation.
Example:
# lvcreate -i2 -l10 -n lvs vg_test
# lvconvert -m1 vg_test/lvs
# lvreduce -f -l1 vg_test/lvs
Reducing logical volume lvs to 4.00 MiB
Segment extent reduction 9not divisible by #stripes 2
Logical volume lvs successfully resized
# lvremove vg_test/lvs
Segment extent reduction 1not divisible by #stripes 2
LV segment lvs:0-4294967295 is incorrectly listed as being used by LV lvs_mimage_0
Internal error: LV segments corrupted in lvs_mimage_0.
We should never remove more extents than requested by user,
so round up to next stripe boundary during lvreduce.
Also this fixes round to zero sized LV bug:
# lvcreate -i2 -I 64k -l10 -n lvs vg_test
# lvreduce -f -l1 vg_test/lvs
Rounding size (1 extents) down to stripe boundary size for segment (0 extents)
WARNING: Reducing active logical volume to 0
THIS MAY DESTROY YOUR DATA (filesystem etc.)
Reducing logical volume lvs to 0
Failed to suspend lvs
There's a possibility someone will use the '/' in the hostname. Since we
generate a temporary file name (path) including the hostname, any '/' would
be ambiguous.
We can always set such hostname using 'sethostname' from unistd.h. But the
'hostname' command already includes the check and removes the '/' char.
However, some old versions still allow that.
See: https://bugzilla.redhat.com/show_bug.cgi?id=711445.
Since this is only a temporary name and the possibility of this error is
quite negligible, we don't need any complex escape sequence here, just a
simple char replace.
Before, we used vg_write_lock_held call to determnine the way a device is
opened. Unfortunately, this opened many devices in RW mode when it was not
really necessary. With the OPTIONS+="watch" rule used in the udev rules,
this could fire numerous events while closing such devices (and it caused
useless scans from within udev rules in return).
A common bug we hit with this was with the lvremove command which was unable
to remove the LV since it was being opened from within the udev rules. This
patch should minimize such situations (at least with respect to LVM handling
of devices).
Though there's still a possibility someone will open a device 'outside' in
parallel and fire the event based on the watch rule when closing a device
once opened for RW.
allocates these buffers in such way it adds memory page for each such buffer
and size of unlock memory check will mismatch by 1 or 2 pages.
This happens when we print or read lines without '\n' so these buffers are
used. To avoid this extra allocation, use setvbuf to set these bufffers ahead.
Signed-off-by: Zdenek Kabelac <zkabelac@redhat.com>
Reviewed-by: Milan Broz <mbroz@redhat.com>
Reviewed-by: Petr Rockai <prockai@redhat.com>
Also, add a new 'obtain_device_list_from_udev' setting to lvm.conf with which
we can turn this feature on or off if needed.
If set, the cache of block device nodes with all associated symlinks
will be constructed out of the existing udev database content.
This avoids using and opening any inapplicable non-block devices or
subdirectories found in the device directory. This setting is applied
to udev-managed device directory only, other directories will be scanned
fully. LVM2 needs to be compiled with udev support for this setting to
take effect. N.B. Any device node or symlink not managed by udev in
udev directory will be ignored with this setting on.
This fails on lenny buildslave for some reason.
For now disable the vgimportclone part of the test until proper fix.
Let the first part of the test still run though, which shows pvs working
with duplicate pvs.
Related to rhbz 697959.
This test fails prior to these two commits:
commit af112eb2c9
Author: Zdenek Kabelac <zkabelac@redhat.com>
Date: Thu Apr 21 13:15:26 2011 +0000
Skip check for NULL before dm_free
dm_free makes this test itself.
commit 91419c3e86
Author: Zdenek Kabelac <zkabelac@redhat.com>
Date: Thu Apr 21 13:13:40 2011 +0000
Fix use of released vgname and vgid
Avoid using of already released memory when duplicated MDA is found.
As get_pv_from_vg_by_id() may call lvmcache_label_scan() use the local copy
of the vgname and vgid on the stack as vginfo may dissapear and code was
then accessing garbage in memory.
i.e. pvs /dev/loop0
(when /dev/loop0 and /dev/loop1 has same MDA content)
Invalid read of size 1
at 0x523C986: dm_hash_lookup (hash.c:325)
by 0x440C8C: vginfo_from_vgname (lvmcache.c:399)
by 0x4605C0: _create_vg_text_instance (format-text.c:1882)
by 0x46140D: _text_create_text_instance (format-text.c:2243)
by 0x47EB49: _vg_read (metadata.c:2887)
by 0x47FBD8: vg_read_internal (metadata.c:3231)
by 0x477594: get_pv_from_vg_by_id (metadata.c:344)
by 0x45F07A: _get_pv_if_in_vg (format-text.c:1400)
by 0x45F0B9: _populate_pv_fields (format-text.c:1414)
by 0x45F40F: _text_pv_read (format-text.c:1493)
by 0x480431: _pv_read (metadata.c:3500)
by 0x4802B2: pv_read (metadata.c:3462)
Address 0x652ab80 is 0 bytes inside a block of size 4 free'd
at 0x4C2756E: free (vg_replace_malloc.c:366)
by 0x442277: _free_vginfo (lvmcache.c:963)
by 0x44235E: _drop_vginfo (lvmcache.c:992)
by 0x442B23: _lvmcache_update_vgname (lvmcache.c:1165)
by 0x443449: lvmcache_update_vgname_and_id (lvmcache.c:1358)
by 0x443C07: lvmcache_add (lvmcache.c:1492)
by 0x46588C: _text_read (text_label.c:271)
by 0x466A65: label_read (label.c:289)
by 0x4413FC: lvmcache_label_scan (lvmcache.c:635)
by 0x4605AD: _create_vg_text_instance (format-text.c:1881)
by 0x46140D: _text_create_text_instance (format-text.c:2243)
by 0x47EB49: _vg_read (metadata.c:2887)
Add testing script
pv_manip.c to properly account for case when pe_start=0 and the first
physical extent is to be released (currently skip the first extent to
avoid discarding the PV label).
My previous patch fixed incorrect error check for dm_snprintf.
However in this particular case - dm_snprintf has been used differently -
just like strncpy + setting last char with '\0' - so the code had to return
error - because the buffer was to short for whole string.
Patch replaces it with real strncpy.
Also test for alloca() failure is removed - as the program behaviour
is rather undefined in this case - it never returns NULL.
lvseg properties for lvm2app, 'devices' and 'seg_pe_ranges'.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
Reviewed-by: Petr Rockai <prockai@redhat.com>
dm_pool_free incorrectly. This check-in fixes that incorrect usage.
I've also added a WHATS_NEW line to reflect the changes I made to allow
lv_extend to operate on 0 length intrinsically layered LVs (i.e mirrors
and RAID). I forgot that in the last commit.
allows us to allocate all images of a mirror (or RAID array) at one
time during create.
The current mirror implementation still requires a separate allocation
for the log, however.
Actually, we can call vg_set_fid(vg, NULL) instead of calling
destroy_instance for all PV structs and a VG struct - it's the same
code we already have in the vg_set_fid.
Also, allow exactly the same fid to be set again for the same PV/VG
Before, this could end up with the fid destroyed because we destroyed
existing fid first and then we used the new one and we didn't care
whether existing one == new one by chance.
As now the FID management is more complex, the code inside free_vg needs
to access some parts of memory pools which were not needed before.
For this - makes the order of unlock_and_free_vg() unconditional.
Keek using unlock_and_free_vg() API function.
For properly working VG locking mechanism only the alphabeting locking
orderer needs to be preserved.
TODO: there could be few more code parts simplified when we 'officially'
support of referencies between different memory pools.
Instead of searching linear list of all LVs, PVs - use created hash tables
also for quick mapping between LV.
(Note - for small number of PVs or LVs the overhead of the hash is bigger).
TODO: Use hash tables in volume_group structure directly.
Instead of regenerating config tree and parsing same data again,
check whether export_vg_to_buffer does not produce same string as
the one already cached - in this case keep it, otherwise throw cached
content away.
For the code simplicity calling _free_cached_vgmetadata() with
vgmetadata == NULL as the function handles this itself.
Note: sometimes export_vg_to_buffer() generates almost the same data
with just different time stamp, but for the patch simplicity,
data are reparsed in this case.
This patch currently helps for vgrefresh.
Align strdup char* allocation just on 2 bytes.
It looks like wasting space to align strings on 8 bytes.
(Could be even 1byte - but for hashing it might eventually get better
perfomance - but probably hardly measurable).
TODO: check on various architectures it's not making any problems.
Isn't usually perfomance critical - but log_error is used i.e.for debuging,
this code noticable slows down the processing.
Added 512KB limit to avoid memory exhastions in case of some endless loop.
TODO: use _lvm_errmsg buffer only when lvm2api needs it.
Avoid locking sum testing with valgrind compilation.
Make memory unaccessible in the valgrind for dm_pool_abadon_object.
Valgrind hinting should not be needed in _free_chunk for dm_free.
'a small step' towards cleaner shutdown sequence.
Normally clvmd doens't care about unreleased memory on exit -
but for valgrind testing it's better to have them cleaned all.
So - few things are left on exit path - this patch starts to remove
just some of them.
1. lvm_thread_fs is made as a thread which could be joined on exit()
2. memory allocated to local_clien_head list is released.
(this part is somewhat more complex if the proper reaction is
needed - and as it requires some heavier code moving - it will
be resolved later.
Could be reached via few of our lvm2 test cases:
==11501== Invalid read of size 8
==11501== at 0x49B2E0: _area_length (import-extents.c:204)
==11501== by 0x49B40C: _read_linear (import-extents.c:222)
==11501== by 0x49B952: _build_segments (import-extents.c:323)
==11501== by 0x49B9A0: _build_all_segments (import-extents.c:334)
==11501== by 0x49BB4C: import_extents (import-extents.c:364)
==11501== by 0x497655: _format1_vg_read (format1.c:217)
==11501== by 0x47E43E: _vg_read (metadata.c:2901)
cut from t-vgcvgbackup-usage.sh
--
pvcreate -M1 $(cat DEVICES)
vgcreate -M1 -c n $vg $(cat DEVICES)
lvcreate -l1 -n $lv1 $vg $dev1
--
Idea of the fix is rather defensive - to allocate one extra element
to 'map' array which is then used in _area_length() - where the
loop checks, whether next map entry is continuous.
By placing there always one extra zero entry -
we fix the read of unallocated memory, and we make sure the data would
not make a continous block.
FIXME: there could be a problem if some special broken lvm1 data would be imported.
As the format1 is currently not really used - leave it for future fix
and use this small hotfix for now.
With recent update of dm_report_field_string() API call to accept
completely const objects - we no longer need loose constness here
and keep it forwarding.
Invalid primary_vginfo was supposed to move all its lvmcache_infos to
orphan_vginfo - however it has called _drop_vginfo() inside the loop
that released primary_vginfo itself - thus made the loop using released
memory.
Use _vginfo_detach_info() instead and call _drop_vginfo after
th loop is finished.
Valgrind trace it should fix:
Invalid read of size 8
at 0x41E960: _lvmcache_update_vgname (lvmcache.c:1229)
by 0x41EF86: lvmcache_update_vgname_and_id (lvmcache.c:1360)
by 0x441393: _text_read (text_label.c:329)
by 0x442221: label_read (label.c:289)
by 0x41CF92: lvmcache_label_scan (lvmcache.c:635)
by 0x45B303: _vg_read_by_vgid (metadata.c:3342)
by 0x45B4A6: lv_from_lvid (metadata.c:3381)
by 0x41B555: lv_activation_filter (activate.c:1346)
by 0x415868: do_activate_lv (lvm-functions.c:343)
by 0x415E8C: do_lock_lv (lvm-functions.c:532)
by 0x40FD5F: do_command (clvmd-command.c:120)
by 0x413D7B: process_local_command (clvmd.c:1686)
Address 0x63eba10 is 16 bytes inside a block of size 160 free'd
at 0x4C2756E: free (vg_replace_malloc.c:366)
by 0x41DE70: _free_vginfo (lvmcache.c:980)
by 0x41DEDA: _drop_vginfo (lvmcache.c:998)
by 0x41E854: _lvmcache_update_vgname (lvmcache.c:1238)
by 0x41EF86: lvmcache_update_vgname_and_id (lvmcache.c:1360)
by 0x441393: _text_read (text_label.c:329)
by 0x442221: label_read (label.c:289)
by 0x41CF92: lvmcache_label_scan (lvmcache.c:635)
by 0x45B303: _vg_read_by_vgid (metadata.c:3342)
by 0x45B4A6: lv_from_lvid (metadata.c:3381)
by 0x41B555: lv_activation_filter (activate.c:1346)
by 0x415868: do_activate_lv (lvm-functions.c:343)
problematic line:
dm_list_iterate_items_safe(info2, info3, &primary_vginfo->infos)
Fix 2 more functions sending cluster messages to avoid passing uninitilised bytes
and compensate 1 extra byte attached to the message from the clvm_header.args[1]
member variable.
If _move_lv_segments is passed a 'lv_from' that does not yet
have any segments, it will screw things up because the code
that does the segment copy assumes there is at least one
segment. See copy code here:
lv_to->segments = lv_from->segments;
lv_to->segments.n->p = &lv_to->segments;
lv_to->segments.p->n = &lv_to->segments;
If 'segments' is an empty list, the first statement copies over
the values, but the next two reset those values to point to the
other LV's list structure. 'lv_to' now appears to have one
segment, but it is really an ill-set pointer.
When I see 'seg_is_mirrored', I expect the argument to be an lv_segment.
In this case, it is lvcreate_params. Both structures, have a 'segtype'
entry which the macro dereferences. However, it just seems easier to
understand if we do 'segtype_is_mirrored' instead.
other cases, the code could wind up removing wrong number of mirrors. In yet
other cases, we could remove the right number of mirrors, but fail to respect
the removal preferences (i.e. keep an image that was requested to be removed
while removing an image that was requested to be kept). Under some
circumstances, remove_mirror_images could also get stuck in an infinite loop.
This patch should fix all of the above undesirable behaviours.
Signed-off-by: Petr Rockai <prockai@redhat.com>
Reviewed-by: Jonathan Brassow <jbrassow@redhat.com>
LVM doesn't behave correctly if running as non-root user,
there is warning when it detects it.
Despite this, it produces many error messages, saying nothing.
See https://bugzilla.redhat.com/show_bug.cgi?id=620571
This patch fixes two things:
1) Removes eror message from device_is_usable() which has no
information value anyway (real warning is printed inside it).
2) it fixes device-mapper initialization, if we support
core dm module autoload and device node is present, it should
fail early and not try recreate existing and correct node.
(non-root == permission denied here)
N.B. In future code should support user roles, some more
drastic checks in code are probably contraproductive now.
Attach \0 for proper char* display - otherwise somewhat random message could
be displayed in debug more and read of unpredictable read of uninitilized
memory values could happen.
As code uses strncpy(system_id, NAME_LEN) and doesn't set '\0'
Fix it by always allocating NAME_LEN + 1 buffer size and with zalloc
we always get '\0' as the last byte.
This bug may trigger some unexpected behavior of the string operation
code - depends on the pool allocator.
FIXME: refactor this code to alloc_vg.
We have 3 components and traling '\0' so allocate proper room for all of them.
Problem was nicely hidden by allocation from pool and allocation aligment
offset - so to trigger real problem with this one is actually hard.
Return value of readlink limits valid string size.
Characters after returned size present some garbage to printf.
Fix it by placing '\0' on the return size value.
Missing free_vg on error_path in lvmcache_get_vg fn. Call destroy_instance
only if the fid is not part of the vg in backup_read_vg fn (otherwise it's
part of the VG we're returning and we definitely don't want to destroy it!).
This is necessary for proper format instance ref_count support. We iterate
over vg->pvs and vg->removed_pvs list and the ref_count is decremented and
then it is destroyed if not referenced anymore.
Since format instances will use own memory pool, it's necessary to properly
deallocate it. For now, only fid is deallocated. The PV structure itself
still uses cmd mempool mostly, but anytime we'd like to add a mempool
in the struct physical_volume, we can just rename this fn to free_pv and
add the code (like we have free_vg fn for VGs).
This is essential for proper format instance ref_count support. We must
use these functions to set the fid everywhere from now on, even the NULL
value!
We'd like to use the fid mempool for text_context that is stored
in the instance (we used cmd mempool before, so the order of
initialisation was not a matter, but now it is since we need to
create the fid mempool first which happens in create_instance fn).
The text_context initialisation is not needed anywhere outside the
create_instance fn so move it there.
Format instances can be created anytime on demand and it contains
metadata area information mostly (at least for now, but in the future,
we may store more things here to update/edit in a PV/VG). In case we
have lots of metadata areas, memory consumption will rise. Using cmd
context mempool is not quite optimal here because it is destroyed too
late. So let's use a separate mempool for format instances.
Reference counting is used because fids could be shared, e.g. each PV
has either a PV-based fid or VG-based fid. If it's VG-based, each PV has
a shared fid with the VG - a reference to VG's fid.
Makes the code more readable and has a smaller number of memory
accesses thus it's small optimisation as well.
For _get_token() optimize number parsing. Check for '.' char only
if it's not a digit. Move pointer incrementation into one place.
For _eat_space() check only p->te for '\0' in skipping of comment line.
Avoid check for '\0' when we know it is space. Also master while loop
doesn't need checking p->tb for '\0'. We just need to check p->tb
isn't already at the end of buffer. This could give 'extra' loop cycle
if we are already there - but safes memory access in every other case.
Add _lv_postorder_vg() - for calling _lv_postorder() for every LV from VG.
We use this in 2 places - vg_mark_partial_lvs() and vg_validate()
so make it as a one function.
Benefit here is - to use only one cleanup code and avoid
potentially duplicate scans of same LVs.
Copy this file as '.gdbinit' to your home directory or your working
directory. It adds the following commands to gdb:
- first_seg
- lv_status
- lv_status_r
- lv_is_mirrored
- seg_item
- seg_status
- segs_using_this_lv
You can get a list of these user-defined commands by typing:
(gdb) help user-defined
You can get more information on each command by typing:
(gdb) help <command>
Accelerate validation loop by using lvname, lvid, pvid hash tables.
Also merge pvl loop into one cycle now - no need to scan the list twice.
List scan is stopped when dm_hash_insert fails.
The error message with loop_counter1 is no longer provided - however
the message has been misleading anyway.
dm_hash binary functions takes void* key - so there is no need to cast
pointers to char* (also the hash key does not have trailing '\0').
This is slight API change, but presents no change for the API user side
it just allows to write code easier as the casting could be removed.
Create new function alloc_vg() to allocate VG structure.
It takes pool_name (for easier debugging).
and also take vg_name to futher simplify code.
Move remainder of _build_vg_from_pds to _pool_vg_read
and use vg memory pool for import functions.
(it's been using smem -> fid mempool -> cmd mempool)
(FIXME: remove mempool parameter for import functions and use vg).
Move remainder of the _build_vg to _format1_vg_read
Fixing few issues:
struct clvm_header contains 'char args[1]' - so adding '+ 1' here
for message length calculation is 1 byte off.
Message with last byte uninitialized is then passed to write function.
Update also related arglen.
Initialise xid and clintid to 0.
Memory allocation is checked for NULL
When the ->params string is empty - memory access is made on the byte
before allocated buffer (catched by valgrind) - in the case it would
constain 0x20 - it would even overwrite this buffer.
So fix by checking len > 0 before doing such access.
Also slightly optimise this loop from repeated strlen call.
lvseg_segtype_dup used memory pool vg memory pool for strind duplication.
However this one gets released before reporting happens so the command like:
pvs -o segtype
prints data from already released memory pool. Thanks to the fact there
is not much allocation happing after the VG is released, the memory
stays unmodified and correct result is printed.
Fix adds support for mempool passed parameter (like other similar
query commands) and uses dm_report memory pool for string duplication.
Failure on pipe kills lv command and produces memory stacktrace.
Add 'true' for empty grep/sed to avoid generating stacktrace.
This should help with long time (7min) for this test on Lenny testing maching
(mimage was filler with stacktrace and each word was used as LV name).
This lvconvert is now able to complete operation automatically without question.
So remove 'echo y |' - it's breaking pipe on some bash versions.
i.e.:
+ echo y
./t-lvconvert-mirror-basic.sh: line 104: echo: write error: Broken pipe
while command has completed this operation succesfully
While STRIPE_SIZE_LIMIT * 2 is basically UINT_MAX, 32bit integer
value can already overflow durin arg size parsing.
(This really happens in test where --stripesize 4294967291 is used,
in s390x uint overflow and this test is ineffective.)
- returned char not needed to be explicitly const
- warn if pipe() fails in clvmd (more fixes here needed for error paths...)
- assign (and ignore) read() output in drain buffer
Currently the test are not properly working with new code.
Use settings for the previous behaviour.
FIXME: update tests to properly test new allocation policies.
We allow writing non-orphan PVs only for resize now. The "orphan PV" assert
in pv_write fn uses the "allow_non_orphan" parameter to control this assert.
However, we should find a more elaborate solution so we can remove this
restriction altogether (pv_write together with vg_write is not atomic, we
need to find a safe mechanism so there's an easy revert possible in case of
an error).
There is a lot to test.
Two new config settings added that are intended to make the code behave
closely to the way it did before - worth a try if you find problems.
Add a small fix that preserves pe_start for lvm1 PVs when being converted.
(this fix needs to be replaced with something more clever, but let's have this working now)
If the PV is already part of the VG (so the pv->fid == vg->fid), it makes no
sense to attach the mdas information from PV to a VG. Instead, we read new
PV metadata information from cache and attach it to the VG fid.
This function also sets a reference to a new VG format instance for all PVs
that are part of the VG so the PV-VG interconnection is consistent after the
change.
Add supporting functions to work with the format instance and metadata area
structures stored within the format instance. Add support for simple indexing
of metadata areas using PV id and mda order (for on-disk PV only for now, we
can extend the indexing even for other mdas if needed - we only need to define
a proper key for the index).
As the kernel seems to be doing weird things during
mlock -> munlock - allow 1 page locking difference without
warning - and log just debug message for a 1 page difference.
Allocation happens outside critical section probably during
log_warn printing.
Should make tests passing for now.
Fixing some const warnings - with API change in:
int vg_extend(struct volume_group *vg, int pv_count, const char *const *pv_names,
Change is needed - as lvm2api expects const behaviour here.
So vg_extend() is doing local strdup for unescaping.
skip_dev_dir return const char* from const char* vg_name.
Rest of the patch is cleanup of related warnings.
Also using dm_report_filed_string() API change to simplify
casting in _string_disp and _lvname_disp.
As dm_report_field_string() doesn't modify content of data pointer,
it can be marked as const.
It's slight API change - but doesn't require any change on the user side
and supports wider range of arguments without const casting.
(i.e. we may use as paramater const lv struct this way: &lv->name)
New strategy for memory locking to decrease the number of call to
to un/lock memory when processing critical lvm functions.
Introducing functions for critical section.
Inside the critical section - memory is always locked.
When leaving the critical section, the memory stays locked
until memlock_unlock() is called - this happens with
sync_local_dev_names() and sync_dev_names() function call.
memlock_reset() is needed to reset locking numbers after fork
(polldaemon).
The patch itself is mostly rename:
memlock_inc -> critical_section_inc
memlock_dec -> critical_section_dec
memlock -> critical_section
Daemons (clmvd, dmevent) are using memlock_daemon_inc&dec
(mlockall()) thus they will never release or relock memory they've
already locked memory.
Macros sync_local_dev_names() and sync_dev_names() are functions.
It's better for debugging - and also we do not need to add memlock.h
to locking.h header (for memlock_unlock() prototyp).
Add configurable option to define minimal size of
of block device usable as a PV.
pv_min_size() is added to lvm-globals and it's being
initialized through _process_config.
Macro PV_MIN_SIZE is unused and removed.
New define DEFAULT_PV_MIN_SIZE_KB is added to lvm-global
and unlike PV_MIN_SIZE it uses KB units.
Should help users with various slow devices attached to the system,
which cannot be easily filtered out (like FDD on /dev/sdX):
https://bugzilla.redhat.com/show_bug.cgi?id=644578
results in clvmd deadlock
When a logical volume is activated exclusively in a cluster, the
local (non-cluster-aware) target is used. However, when creating
a snapshot on the exclusive LV, the resulting suspend/resume fails
to load the appropriate device-mapper table - instead loading the
cluster-aware target.
This patch adds an 'exclusive' parameter to the pertinent resume
functions to allow for the right target type to be loaded.
As functions compiled within this define are apparently stil part of the public API,
(though lvm2 code is never using them unless this define is used for compilation),
keep functions available in the code for now -> revert.
Fix regresion from 2.02.75 speedup - so currently crc32 is a little bit
more complicated on big-endian CPU as the uint32_t needs to be shifted
on here.
Make configurable default behaviour how to deal with device node creates.
With udev system natural options should be 'resume'.
For older systems where user expect there is node in /dev/mapper immediately
after dmsetup create --notable - use 'create'
FIXME:
Code needs fixing passing this flag through udev cookie.
activated.
In order to achieve this, we need to be able to query whether
the origin is active exclusively (a condition of being able to
add an exclusive snapshot).
Once we are able to query the exclusive activation of an LV, we
can safely create/activate the snapshot.
A change to 'hold_lock' was also made so that a request to aquire
a WRITE lock did not replace an EX lock, which is already a form
of write lock.
Add new function dm_task_set_add_node() to select between 2 types
of node creation in device directory.
DM_ADD_NODE_ON_RESUME is now default and ensures node is created on
resume. Old original behavior is accessible with DM_ADD_NODE_ON_CREATE.
In this case node would be created during dmsetup create --notable.
For the user 2 new options for dmsetup create are added:
[{--addnodeonresume | --addnodeoncreate }]
Properly working node creation on resume is needed for proper operation
stacking and ability to correctly check in which state the device should
after whole udev transation.
Remove temporaly added fs_unlock() calls to fix clmvd usablity.
Now when the message passing is properly working - they are no longer needed.
Simplify no_locking check for VG unlock - as message is always send
for all targets - clustered & non-clustered.
Thanks to CLVMD_CMD_SYNC_NAMES propagation fix the message passing started
to work. So starts to send a message before the VG is unlocked.
Removing also implicit sync in VG unlock from clmvd as now the message
is delievered and processed in do_command().
Also add support for this new message into external locking
and mask this event from further processing.
With the ability to stack many operations in one udev transaction -
in same cases we are adding and removing same device at the same time
(i.e. deactivate followed by activate).
This leads to a problem of checking stacked operations:
i.e. remove /dev/node1 followed by create /dev/node1
If the node creation is handled with udev - there is a problem as
stacked operation gives warning about existing node1 and will try to
remove it - while next operation needs to recreate it.
Current code removes all previous stacked operation if the fs op is
FS_DEL - patch adds similar behavior for FS_ADD - it will try to
remove any 'delete' operation if udev is in use.
For FS_RENAME operation it seems to be more complex. But as we
are always stacking FS_READ_AHEAD after FS_ADD operation -
should be safe to remove all previous operation on the node
when udev is running.
Code does same checking for stacking libdm and liblvm operations.
As a very simple optimization counters were added for each stacked ops
type to avoid unneeded list scans if some operation does not exists in
the list.
Enable skipping of fs_unlock() (udev sync) if only DEL operations are staked.
as we do not use lv_info for already deleted nodes.
This is better way how to fix clustered synchronization with udev.
As the code for message passing needs fixed - put currently
fs_unlock() after every active/deactive command in clvmd to
ensure nodes are properly created in time.
It would be most useful to add "dmsetup ls --tree" to the commands run.
This command helps in answering the question "which devices are actually
underneath a given LV?"
Although the info is available with other existing dmsetup commands,
adding this command gives a much clearer summary of complex setups.
Here's an example of an LVM mirror, with mirror images on partitions
created on top of multipath devices. The multipath devices are on
simple block devices. As you can see, it is easy to see the stacking
from the "dmsetup ls --tree" output:
vgmpathtest-lvmpathmir (253:14)
├─vgmpathtest-lvmpathmir_mimage_1 (253:13)
│ └─mpath5p1 (253:5)
│ └─mpath5 (253:2)
│ ├─ (8:16)
│ └─ (8:0)
├─vgmpathtest-lvmpathmir_mimage_0 (253:12)
│ └─mpath6p1 (253:6)
│ └─mpath6 (253:3)
│ ├─ (8:48)
│ └─ (8:32)
└─vgmpathtest-lvmpathmir_mlog (253:11)
└─mpath7 (253:4)
├─ (8:80)
└─ (8:64)
VolGroup00-LogVol01 (253:1)
└─ (202:2)
vgtest-lvmir (253:10)
├─vgtest-lvmir_mimage_1 (253:9)
│ └─ (7:1)
├─vgtest-lvmir_mimage_0 (253:8)
│ └─ (7:0)
└─vgtest-lvmir_mlog (253:7)
└─ (7:3)
VolGroup00-LogVol00 (253:0)
└─ (202:2)
But it is much harder to see the stacking with only the commands today
("dmsetup info", "dmsetup status", and "dmsetup table"). We could
piece together the stacking from "dmsetup table" but it requires
further processing (take output from "dmsetup info to get
map name to major/minor, then parse "dmsetup table", etc).
There was no effect from having this wrong yet, because the
tree of callers only ever cared about the answer to the first
condition (!response), which determines whether a lock is
held or not. Correct responses, however, are needed soon.
Instead of implicitly syncing udev operation in clustered and
file locking code - call synchronization directly in lock_vol() when
the operation unlocks VG
The problem is missing implicit fs_unlock() in the no_locking code.
This is used with --sysinit on read-only filesystem locking dir.
In this case vgchange -ay could exit before all udev nodes are properly
synchronised and may cause problems with accessing such node right after
vgchange --sysinint command is finished.
Add test case for vgchange --sysinit.
As the option 'set -e -o pipefail' is very sensite on pipe breaking
stop using '-q' for grep commands.
Otherwise this command (with large enough table) would fail:
dmsetup table | egrep -q
with exit code 141 (128 + SIGPIPE)
As Peter suggested, he prefers to keep '-o pipefail' - so make sure all
piped commands will read the whole output and will not exit too early.
This is to avoid any scanning and processing of DM devices while they are in
suspended state (e.g. a rename while the device is suspended - a CHANGE event
is generated!). Otherwise, any scanning in the rules could end up with locking
the calling process until the device is resumed and so we don't receive a
notification about udev rules completion until then (and that effectively
locks out the process awaiting the notification!).
However, we still keep 'disk' and any 'subsystem' related udev rules running.
We trust these and these should check themselves whether a device is suspended
or not, not trying to run any scanning if it is.
strncpy (which check each byte for \0) is not need as we always copy
the length size - so using memcpy is a bit cheaper.
Add missing log_error message for failed allocation.
Fix assert abort of dmsetup (when compiled with pool debug)
dmsetup splitname --nameprefixes --noheadings --rows gvg-a2
Move pool begin in the inner loop - otherwise it would using
already 'ended' pool object.
return lockspace reference (even if lockspace already exists)
and thus increases DLM lockspace count. It means that after
clvmd restart the lockspace is still in use.
(The only way to clean environment to enable clean cluster
shutdown is call "dlm_tool leave clvmd" several times.)
Because only one clvmd can run in time, we can use simpler logic,
try to open lockspace with dlm_open_lockspace() and only if it fails
try to create new one. This way the lockspace reference doesn not
increase.
Very easily reproducible with "clvmd -S" command.
Patch also fixes return code when clvmd_restart fails and fixes
double free if debug option was specified during restart.
Fixes https://bugzilla.redhat.com/show_bug.cgi?id=612862
Improve condition within lock_vol so we are not calling extra unlock
if the volume just has been deactivated.
Patch uses lck_type and replaces negative 'and' condition to more
readable 'or' condition.
Few missing strace traces added.
As sync_local_dev_names() cannot be called within activation context,
add new parametr which allows to select if the sync call is needed
before executing new command.
Instead of checking $LOCAL_CLVMD set during some 'aux' execution which
doesn't seem to be propagated to this shell - check for existance of pid
file of clvmd process - so this test is skipped in singlenode cluster test.
Harness seems to be able to busyloop in while cycle and not moving forward
for certain buffer - so check whethere there was some progress.
This fix allows to continue after failed cluster test.
Fix gcc warning for hiding global variable 's' -> sig.
Stop calling fs_unlock() from lv_de/activate().
Start using internal lvm fs cookie for dm_tree.
Stop directly calling dm_udev_wait() and
dm_tree_set/get_cookie() from activate code -
it's now called through fs_unlock() function.
Add lvm_do_fs_unlock()
Call fs_unlock() when unlocking vg where implicit unlock solves the
problem also for cluster - thus no extra command for clustering
environment is required - only lvm_do_fs_unlock() function is added
to call lvm's fs_unlock() while holding lvm_lock mutex in clvmd.
Add fs_unlock() also to set_lv() so the command waits until devices
are ready for regular open (i.e. wiping its begining).
Move fs_unlock() prototype to activation.h to keep fs.h private
in lib/activate dir and not expose other functions from this header.
Change function import_vg_from_buffer() to import_vg_from_config_tree().
Instead of creating config tree inside the function allow config tree to
be passed as parameter - usable later for caching.
If some allocation for peristent filter fails its memory reference
was lost, fix it by calling filter's destructor.
Fix log_error messages for failing allocation.
Variable 'ret' assigned from _do_event() was actually not used and replaced with next
assignment without any read of the returned value.
Code is reformated - so the error path is put in the if() branch and normal
code is put after the 'if' together with FIXME comment.
FIXME lowprio: logging needs to be fixed in this code,
- multiple log_errors are printed, stacks are missing...
a mirror image (or removing a log while adding a mirror). Advise the
user to use two separate commands instead.
This issue become especially problematic when PVs are specified, as they
tend to mean different things when adding vs removing. In a command that
mixes adding and removing, it is impossible to decern exactly what the
user wants.
This change prevents bug 603912.
- somewhat neater, more consistent and more readable output
- possible to set any lvm.conf value: aux lvmconf "section/key = value"
- LVM_TEST_NODEBUG to suppress the (lengthy) "## DEBUG" output
- back-substitution on test output ($TESTDIR/$PREFIX -> @TESTDIR@/@PREFIX@)
- support code moved from test/ to test/lib/ --> less clutter
Checking for vg being != NULL in this place is not needed.
Pointer vg is already dereferced in this function above this code line.
Also this internal function _read_pv is always called with valid 'vg' pointer.
Call for pthread_join() does not set errno value even though return values
looks like that. For now assign errno from return value and still use
strerror() to print some error message as this seems to be commonly used.
Add also log_sys_error() message for error close of local pipe.
Add checks for clonning allocation a fail-out when something is
not allocated correctly.
Also move var declaration to the begining of the function
and fix log_error messages.
As 'const' types are also passed to macro dm_list_struct_base -
keep offset calculation with const char pointers.
Fixes several gcc constness warnings.
As const segment_type or const format_type are never released
use their non-const version and remove const downcast from dm_free calls.
This change fixes many gcc warnings we were getting from them.
Change API interface to accept even completely const array patterns.
This should present no change for libdm users and allows to pass
pattern arrays without cast to const char **.
To have better control were the config tree could be modified use more
const pointers and very carefully downcast them back to non-const
(for config tree merge).
As ternary operator has lower priority then add operation, this check
was not doing what seemed to be expected.
So enclose the test in braces and check for NULL in *buf.
We need to be sure that /var/run and /var/lock is always there.
(E.g. these two directories could be using tmpfs which then loose
all the content after reboot.)
Detect existence of new SELinux selabel interface during configure.
Use new dm_prepare_selinux_context instead of dm_set_selinux_context.
We should set the SELinux context before the actual file system object creation.
The new dm_prepare_selinux_context function sets this using the selabel_lookup
fn in conjuction with the setfscreatecon fn. If selinux/label.h interface
(that should be a part of the selinux library) is not found during configure,
we fallback to the original matchpathcon function instead.
Set cmd->independent_metadata_areas if metadata/dirs or disk_areas in use.
- Identify and record this state.
Don't skip full scan when independent mdas are present even if memlock is set.
- Clusters and OOM aren't supported, so no problem doing the proper scans.
Avoid revalidating the label cache immediately after scanning.
- A simple optimisation.
Support scanning for a single VG in independent mdas.
- Not used by the fix but I left it in anyway as later patches might use it.
This reset of vgmem pointer causes access of already released memory.
(_vg_make_handle allocates vg from vgmem pool itself - which is a bit tricky)
Interestingly this memory fault was missed by our test suite.
Add log_error message for lv_info failure and exit from futher
processing.
Replace 'leg' occurence in debug message with 'image' which
is used in other messages.
Patch adds extra check for lv_name not being NULL.
Test avoids unneeded strlen call for this case.
Otherwise there is no functional change as test would fail on
size_t comparation even for NULL lv_name (thus there is no risk
of NULL dereference when taking 'true' if branch.
Add test for NULL before passing uuid as src argument to memcpy.
As memcpy function is declared as function not accepting NULL.
Though we pass NULL only with zero length so this patch presents
no functional change to the code.
Add test for NULL from dm_poll_create.
Reorder dm_pool_destroy() before file close and add label out:.
Avoid leaking file descriptor if the allocation fails.
Nicely hidden memory leak in outf macro error path.
This macro is using out_text() and does automagical return_0.
That would leak tag_buffer allocated memory.
As there was same code for tags output - create _out_tags() function.
Set vg to NULL after releasing it as the following memlock() test may
lead to goto for the second call of vg_release() with the already
released vg pointer.
LCK_CACHE is defined as 0x100 so it cannot be passed through
unsigned char parameter - remove it from the sprintf code.
If the LCK_CLUSTER should be printed here - lot of code need
to be reworked - so adding FIXME comment.
was lacking the (vgmem) pool. We now create that pool. There is at least one
more such VG (_dummy_vg) which is pool-less. I am not sure what is the right
way to go about this, but this is currently necessary to fix a segfault
introduced by using vgmem in the reporter in Dave's lvseg lvm2app patches.
Signed-off-by: Petr Rockai <prockai@redhat.com>
Similar to 'get' property internal functions.
Add specific 'set' function for vg_mda_copies.
Signed-off-by: Dave Wysochanski <wysochanski@pobox.com>
Reviewed-by: Petr Rockai <prockai@redhat.com>
Fix for the last commit as $MOUNTED is not only used as bool flag,
but also store mounted location for remount - so parsing output
from mount differently then from /proc/mounts.
Prefix calls of 'tunefs' tools with LANG=C to be sure we always do get
some nonlocalized strings.
Avoid using forced 'resize2fs' for cleanly unmounted filesystems and
run regular fsck -f for this case as required by resize2fs.
'fsadm check' uses date difference for extX filesystems between
the last mount and last check of 'fsck -f' execution and if the mount
was later run 'fsck' with -f so resize2fs is happy and user does not
need to pass '-f' flag.
As util-linux package seems to give all the time different names,
try harder to figure out, where is the given lv possible mounted
and scan /proc/mounts and if not found there, test also 'mount' output.
/dev/dm-xxx
/dev/mapper/vg-lv
/dev/vg/lv
All of them could be used different combination in /proc/mount and mount output.
Patch fixes regression for older systems where new detection code failed to
find valid combination.
Fixing warning introduced by 'include make.tmpl' commit.
Produced this warning:
Makefile:29: warning: overriding commands for target `distclean'
../make.tmpl:366: warning: ignoring old commands for target `distclean'
Updated patch from Florian Haas from Linux-HA project.
User needs to 'configure --enable-ocf' to get file installed
by 'make install' target by default.
User can also use 'make install_ocf' to get only ocf files installed.
With disabled (default) ocf support - no ocf files are installed.
FIXME: ocf installation path needs to be kept in sync with pacemaker.
find better way and possible also better location.
Patch updates exec_cmd() and adds 3rd parameter with pointer for
status value, so caller might examine returned status code.
If the passed pointer is NULL, behavior is unmodified.
Patch allows to confinue with lvresize if the failure from fsadm check is
caused by mounted filesystem as many of filesystem resize tools do support
online filesystem resize. (originally user had to use flag '-n' to bypass
this filesystem check)
Return status code 3 for fsadm check of mounted filesystem - used later with
lvresize update patch to better support online filesystem resize.
Also makes a more consistent user interruption and returns status code 2
in this case.
Fix memory leak of field_id in _output_field function.
There's been a patch added recently to use dynamic allocation for metadata
tags buffer to remove the 4k limit (for writing metadata out). However, when
using reporting commands like vgs and lvs, we still need to fix libdm reporting
functions themselves to support such long outputs. So the buffer used in those
reporting functions is dynamic now.
The patch also includes a fix for field_id memory leak which was found in
the _output_field function.
The management threads (main_loop, the socket thread) could close a single fd
twice in a row sometimes. At least one other thread can be running at the same
time as the threads doing the double close. That one running thread also
happens to do some IO (namely, open /proc/devices, read from it, close it). If
there was enough "demand" for the local socket, this could happen:
- a connection to clvmd is about to finish, let's say the fd is 13 (it often
happens to be in my test script, don't ask why)
- the local_sock thread calls close(13)
- the lvm thread calls open("/proc/devices"...) and gets 13
- the main_loop thread calls close(13) [OOPS!]
- new connection arrives, and is accept'd by a (new) local_sock thread
- the accept gives an fd of 13 (since it's the lowest free fd at this point)
- the lvm thread gets around to read from it's /proc/devices handle... 13,
again
- the lvm thread hangs forever trying to read from the socket instead of
/proc/devices
Signed-off-by: Petr Rockai <prockai@redhat.com>
Reviewed-by: Milan Broz <mbroz@redhat.com>
Function pull_stateo() checks for NULL 'buf' - but return for this error
path was missing. cmirror code never calls this function with NULL 'buf',
so this fix has no effect on current code base, but makes clang happier.
It's quite new feature which is not supported by older compilers.
So until some better macros are introduced into LVM code - hotfix current
compilation problems and compile this code only for __clang__ defining compilers.
Simultaneous -a and --refresh is not valid.
poll+monitor are valid together with or without -ay* (but not with -an*)
No longer print polling results summary if no LVs in the VG were polled.
Add a generic LV property function to lvm2app, similar to VG function.
Return lvm_property_value and require caller to check 'is_valid' flag
and lvm_errno() for API error.
Add a generic PV property function to lvm2app, similar to VG function.
Return lvm_property_value and require caller to check 'is_valid' flag
before using the value. If 'is_valid' is not set, then lvm_errno()
should be used to obtain the specific error.
Add a generic VG property function to lvm2app. Call the internal library
vg_get_property() function. Strings are dup'd internally.
Rework lvm_vg_get_property to return lvm_property_value and require caller
to check 'is_valid' flag. If !is_valid, the caller can check lvm_errno()
for the specific error.
Create a 'get_property' function, local to lvm2app, that factors out
most of the common code that copies the components of lvm_property_type
into lvm_property_value. This allows for a 1-line function for each
of the generic property functions exported by lvm2app.
Makes clang happier as it covers all code paths and avoids NULL pointer
dereference through the 'com' pointer (which is NULL by default static
initialisation).
Reported by clang as: Argument with 'nonnull' attribute passed null
Reuse the result of the last strchr() call - make sure, 'st' point is not
null for the next strchr() call.
We cast (char*) to (uint32_t*) that changes alignment requierements.
For our case the code has been correct as alloca() returns properly
aligned buffer, however this patch make it cleaner and more readable
and avoids warning generation.
A merged snapshot's DM device is made to use the "error" target as part
of lvm's transaction to merge a snapshot. This snapshot merge use-case
aside, any device using the error target shouldn't be scanned.
Based on review comments, rename a few fields in lvm_property_type.
In particular, change 'is_writeable' to 'is_settable', which is
more intuitive to the intent of the bitfield (a 'set' function
exists for this field/property). Also, remove the char array
for 'id' - unnecessary as we can just use the string passed in
to do the strcmp. Finally rename the union members from n_val
to 'integer' and 's_val' to 'string'.
The signalling code (pthread_cond_signal/pthread_cond_wait) in the
pre_and_post_thread was using the wait mutex (see man pthread_cond_wait)
incorrectly, and this could cause clvmd to deadlock when timing was
right. Detailed explanation of the problem follows.
There is a single mutex around (L for Lock, U for Unlock), a signal (S) and a
wait (W). C for pthread_create. Time flows from left to right, each arrow is a
thread.
So first the "naive" scenario, with no mutex (PPT = pre_and_post_thread, MCT =
main clvmd thread; well actually the thread that does read_from_local_sock). I
will also use X, for a moment when MCT actually waits for something to happen
that PPT was supposed to do.
MCT -----C ------S--X-----S----X----------------------S------XXXXXXXXX
| everything OK up to this --> <-- point...
PPT -----WWW-----WWWW------------------------------WWWWWWWWWWWWW
Ok, so pthread API actually does not let you use W/S like that. It goes out of
its way to tell you that you need a mutex to protect the W so that the above
cannot happen. *But* if you are creative and just lock around the W's and S's,
this happens:
MCT ----C-----LSU----X-----LSU----X------------LSU------XXXXXXX
|
PPT ---LWWWU-------LWWWWU-----------------------LWWWWWWWWW
Ooops. Nothing changed (the above is what actually was done by clvmd before
this satch). So let's do it differently, holding L locked *all* the time in
PPT, unless we are actually in W (this is something that the pthread API does
itself, see the man page).
MCT ----C-----LSU------X---LSU---X-----LLLLLLLSU----X----
| (and they live happily ever after)
PPT L---WWWWW---------WWWW----------------W----------
So W actually ensures that L is unlocked *atomically* together with entering
the wait. That means that unless PPT is actually waiting, it cannot be
signalled by MCT. So if MCT happens to signal it too soon (it wasn't waiting
yet), it (MCT) will be blocked on the mutex (L), until PPT is actually ready to
do something.
to lvm.conf in the activation section: 'snapshot_autoextend_threshold' and
'snapshot_autoextend_percent', that define how to handle automatic snapshot
extension. The former defines when the snapshot should be extended: when its
space usage exceeds this many percent. The latter defines how much extra space
should be allocated for the snapshot, in percent of its current size.
Reorder linked libraries so we better support --as-needed linker flag used
by some distributions (i.e. Gentoo).
Patch suggested by Diego Elio Pettenò <flameeyes <at> gmail.com>
Problem:
When both legs of a mirrored log fail, neither the log nor the parent
mirror can proceed. The repair code must be careful to replace the
log with an error target before operating on the parent - otherwise,
the parent can get stuck trying to suspend because it can't push through
any writes. The steps to replace the log device with an error target
were incomplete and resulted in the replacement not happening at all!
The code originally had all the necessary logic to complete the
replacement task, but was pulled out in a effort to clean-up that
section of code, while fixing another bug:
<offending commit msg>
In addition, I added following three changes.
- Removed tmp_orphan_lvs handling procedure
It seems that _delete_lv() can handle detached_log_lv properly
without adding mirror legs in mirrored log to tmp_orphan_lvs.
Therefore, I removed the procedure.
- Removed vg_write()/vg_commit()
Metadata is saved by vg_write()/vg_commit() just after detached_log_lv
is handled. Therefore, I removed vg_write()/vg_commit().
</offending commit msg>
http://sources.redhat.com/cgi-bin/cvsweb.cgi/LVM2/lib/metadata/mirror.c?cvsroot=lvm2&f=h#rev1.130
I've reverted the "clean-up" changes associated with that fix, but not what
that commit was actually fixing.
Signed-off-by: Jonathan Brassow <jbrassow@redhat.com>
Reviewed-by: Petr Rockai <prockai@redhat.com>
page.
Add ->target_name to segtype_handler to allow a more specific target
name to be returned based on the state of the segment.
Result of trying to merge a snapshot using a kernel that doesn't have
the snapshot-merge target:
Before:
# lvconvert --merge vg/snap
Can't expand LV lv: snapshot target support missing from kernel?
Failed to suspend origin lv
After:
# lvconvert --merge vg/snap
Can't process LV lv: snapshot-merge target support missing from kernel?
Failed to suspend origin lv
Unable to merge LV "snap" into it's origin.
Use _even_rand() function instead of floor() in _bitset_with_random_bits().
floor() function is missing in dietlibc (on architectures other than x86).
Moreover using floor() to clip rand results does not assure even result
distribution. _even_rand() uses integer arithmetic only and is designed to
return evenly distributed results.
> Looks OK to me. It took a while to decipher what is the exact meaning of
> the loop in _even_rand (to a non-pseudorandomness-expert) but I am
> fairly comfortable with it now. If I understand this correctly, it
> rejects numbers that come from an "incomplete" slice of the RAND_MAX
> space (considering the number space [0, RAND_MAX] is divided into some
> "max"-sized slices and at most a single smaller slice, between [n*max,
> RAND_MAX] for suitable n -- numbers from this last slice are discarded
> because they could distort the distribution in favour of smaller
> numbers).
Signed-off-by: Przemyslaw Iskra <sparky <at> pld-linux.org>
Reviewed-by: Petr Rockai <prockai <at> redhat.com>
re-add a physical volume that has gone missing previously, due to a transient
device failure, without re-initialising it.
Signed-off-by: Petr Rockai <prockai@redhat.com>
Reviewed-by: Alasdair Kergon <agk@redhat.com>
Try to distinguish between the case of using interactive shell and non
interactive running - different combinations of '-y' and '-p' option
needs to be used for fsck.
Update the way how fsadm detects mounted filesystem.
With udev /dev/dm-XXX paths are now returned - but mount or /proc/mounts
prints names in form of /dev/mapper/vg-lv - so the match was not found.
Fixex RHBZ #638050.
Current solution uses same trick as mount and detects vg-lv name through
/sys where available - this should be reasonable safe.
Instead of calling mount without parameter to get actual mount table,
switch to use /proc/mounts directly.
Fix missing 'dry' execution of lvresize - fixing problem where resize
command were 'dry-run' executed - but lvresize has been executed for real.
Also adapt code slightly to support better recursive execution of fsadm
through lvresize call.
Under certain conditions it was possible to break (^C) fsadm before actually
resizing filesystem, but lvresize which executed fsadm will think resize
was succesful and shrinks partitions with unresized filesystem on it.
Fix by returning error (1) for this case - this stops lvresize from futher
proceding in resize operation.
In other LVM memory structures such as volume_group, the field
used to store flags is called "status", and on-disk fields are called
'flags', so rename the one inside metadata_area to be consistent.
Not only is it more consistent with existing code but is cleaner
to say "the status of this mda is ignored".
Background for this patch - prajnoha pinged me on IRC this morning
about a fix he was working on related to metadataignore when
metadata/dirs was set. I was reviewing my patches from this year
and realized the 'flags' field was probably not the best choice
when I originally did the metadataignore patches.
Current lvm1 allocation code seems to not properly
map segments on missing PVs.
For now disable this functionality.
(It never worked and previous commit just introduced segfault here.)
So the partial mode in lvm1 can only process missing PVs
with no LV segments only.
Also do not use random PV UUID for missing part but use fixed
string derived from VG UUID (to not confuse clvmd tests).
We need to use a similar function for pv and lv properties, so just make
a generic _get_property() function that contains most of the required
functionality. Also, add a check to ensure the field name matches the
object passed in by re-using report_type_t enum. For pv properties,
the report_type might be either PVS or LABEL.
In addition, add 'const' to 'get' functions object parameter, but not
'set' functions. Add _not_implemented_set() and _not_implemented_get()
functions.
Add 'get' functions based on generic macros for VG, PV, and LV.
Add 'get' functions for vg string fields, vg_name, vg_fmt, vg_sysid,
vg_uuid, vg_attr, and vg_tags, and all numeric fields.
Add supporting functions for vg_name, vg_fmt, vg_system_id.
Append "_dup" to end of supporting functions to make clear the strings
are dup'd and to avoid namespace conflict with vg_name.
Add supporting functions for pv_uuid, vg_uuid, and lv_uuid.
Call new function id_format_and_copy. Use 'const' where appropriate.
Add "_dup" suffix to indicate memory is being allocated.
Call {pv|vg|lv}_uuid_dup from lvm2app uuid functions.
This patch addresses code review request to simplify creation of 'attr'
strings. The simplification is done in this separate patch to more
easily review and ensure the simplification is done without error.
Move the creating of the 'attr' strings into a common function so
they can be called from the 'disp' functions as well as the new
'get' property functions.
Add "_dup" suffix to indicate memory is allocated.
Refactor pvstatus_disp to take pv argument and call pv_attr_dup().
This patch is similar to the other patches for pv and vg
functionality, and separates lv functionality into separate
files, concentrating on reporting fields and simple functions.
The metadata.[ch] files are very large. This patch makes a first
attempt at separating out pv functions and data, particularly
related to the reporting fields calculations.
More code could be moved here but for now I'm stopping at reporting
functions 'get' / 'set' functions.
The metadata.[ch] files are very large. This patch makes a first
attempt at separating out vg functions and data, particularly
related to the reporting fields calculations.
Read complete content of /proc/self/maps into one buffer without
realocation in the middle of reading and before doing any m/unlock
operation with these lines - as some of them gets change.
With previous implementation we've read some mappings twice ([stack])
If some lvm1 device is missing, lvm fails on all operations
# vgcfgbackup -f bck -P vg_test
Partial mode. Incomplete volume groups will be activated read-only.
3 PV(s) found for VG vg_test: expected 4
PV segment VG free_count mismatch: 152599 != 228909
PV segment VG extent_count mismatch: 152600 != 228910
Internal error: PV segments corrupted in vg_test.
Volume group "vg_test" not found
Allow loading of lvm1 partial VG by allocating "new" missing PV,
which covers lost space. Also this fake mising PV inform code
that it is partial VG.
https://bugzilla.redhat.com/show_bug.cgi?id=501390
Revert to old glibc behaviour for vsnprintf used in emit_to_buffer fn.
Otherwise, the check that follows would be wrong for new glibc versions.
This caused the rh bug #633033 to be undetected and pass throught the check,
corrupting the metadata!
In certain configurations, we're not under a VG rw lock while trying to write
a new archive file with VG metadata. A common example is using "vgs" while
having the content of backup and archive directories empty. The code scans the
content of these directories and tries to determine the final index that should
be used in archive name. Since we're not under a lock, we can get into a race
while choosing the index which could end up showing errors about not being able
to rename to final archive name. Let's add random number suffix to these archive
file names so we can avoid the race.
For example, when using '--config "backup { ... }"' line, the values from
lvm.conf (or default values) should be overridden. This patch adds
reinitialisation of archive and backup handling on toolcontext refresh
which makes these settings to be applied.
can be opprobriously slow if created with '--nosync'.
One of the ways cluster mirrors coordinate I/O and recovery
amoung the different machines is by the use of the log
function 'is_remote_recovering()' which lets nodes know if
a region they wish to perform a write on is currently being
recovered on another node. If the region is being recovered,
the I/O is delayed.
The 'is_remote_recovering' routine has been optimized to
avoid the deluge of requests that would be issued to the
userspace log server by maintaining a marker of how far
the recovery has gotten. It can then immediately return
'not recovering' if the region being inquired about is
less than this mark. Additionally, if the region of
concern is greater than the mark, the function will
limit the number of transmissions to userspace by assuming
the region /is/ being recovered when skipping the
transmission. This limits the amount of processing
and updates the mark in 1/4 sec time steps.
This patch fixes a problem where 'the mark' is not being
updated because of faulty logic in the userspace log
daemon. When '--nosync' is used to create a cluster
mirror, the userspace log daemon never has a chance
to update the mark in the normal way. The fix is to set
the mark to "complete" if the mirror was created with
the --nosync flag.
For now force it in lvm.conf (otherwise it fails on older systems like RHEL5
where are these options disabled by default).
FIXME: it should test and detect both versions.
may never complete.
If you convert from a linear to a mirror and then convert that
mirror back to linear /while/ the previous (up)convert is
taking place, the mirror polling process will never complete.
This is because the function that polls the mirror for
completion doesn't check if it is still polling a mirror and
the copy_percent that it gets back from the linear device is
certainly never 100%.
The fix is simply to check if the daemon is still looking at
a mirror device - if not, return PROGRESS_CHECK_FAILED.
The user sees the following output from the first (up)convert
if someone else sneaks in and does a down-convert shortly
after their convert:
[root@bp-01 ~]# lvconvert -m1 vg/lv
vg/lv: Converted: 43.4%
ABORTING: Mirror percentage check failed.
to block when a mirror under a snapshot suffers a failure.
The problem has to do with label scanning. When a mirror suffers
a failure, the kernel blocks I/O to prevent corruption. When
LVM attempts to repair the mirror, it scans the devices on the
system for LVM labels. While mirrors are skipped during this
scanning process, snapshot-origins are not. When the origin is
scanned, it kicks up I/O to the mirror (which is blocked)
underneath - causing the label scan (an thus the repair operation)
to hang.
This patch simply bypasses snapshot-origin devices when doing
labels scans (while ignore_suspended_devices() is set). This
fixes the issue.
introduced in commit b16b4d92a7
"Improve various log messages."
fixes a lot of
../include/metadata.h:148: warning: type qualifiers ignored on function return type
set appropriate Required-Start and Required-Stop at configure time.
Reorder the checks for user selected cluster managers to match auto
detected ones, to be consistent in the output.
Add special case for qdiskd that´s started after cman/lock_gulmd for
RHEL-4/RHEL-5.
If pvmove crashed and metadata contains pvmove LV
but without miorrored segments, pvmove --abort
will not repair the situation (and finish wth success!).
Fix it by allowing metadata update if aborting
(thus removing pvmove LV) even if no moved LVs detected.
(Tested on real metadata provided by an lvm user:-)
detected alignment.
NOTE: lvm2 doesn't detect MD 1.2 metadata (now the default on RHEL6) so
for now I'm forcing 1.0 metadata. This was needed to be able to reuse
the existing loop devices but recreate the md device with different
raid0 striping.
Add "devices/default_data_alignment" to lvm.conf to control the internal
default that LVM2 uses: 0==64k, 1==1MB, 2==2MB, etc.
If --dataalignment (or lvm.conf's "devices/data_alignment") is specified
then it is always used to align the start of the data area. This means
the md_chunk_alignment and data_alignment_detection are disabled if set.
(Same now applies to pvcreate --dataalignmentoffset, the specified value
will be used instead of the result from data_alignment_offset_detection)
set_pe_align() still looks to use the determined default alignment
(based on lvm.conf's default_data_alignment) if the default is a
multiple of the MD or topology detected values.
Add 'get' functions based on the simple macro function definition for a
numeric property.
Add 'get' functions for the following: _vg_extent_count_get,
_vg_free_count_get, _max_lv_get, _max_pv_get, _pv_count_get,
_lv_count_get, _snap_count_get, _vg_seqno_get, _vg_size_get,
_vg_free_get, vg_mda_*.
For size functions, multiply by SECTOR_SIZE to return the value in bytes.
Extend the existing reporting infrastructure definitions and structures
to include a 'get' and 'set' function for each field. We will provide
a 'get' and 'set' function for each of these fields, which will be utilized
by exported lvm2app functions.
Define a default _not_implemented 'get' and 'set' function that just sets
an errno and returns 0. Future patches will actually implement the
specific 'get' and 'set' functions for each property. For read-only
properties, only the 'get' function will be implemented.
Define vg_get_property() function to query a property. We will call
this from a lvm2app function.
Rather than hard code the size of the field, use a #define, so we can re-use.
The #define will be needed in a future patch when we extend the reporting
infrastructure to have 'get' and 'set' functions for each field, allowing
lvm2app functions which query any report field. In order to provide a
generic lookup based on the field id, we will define a type containing this
field id, and thus, we will need to re-use the length of this string as
it's defined inside libdevmapper.h.
The 'id' entries in columns.h are the report field names. Since these are
unique, we'd like to use them in generation of 'get' / 'set' functions.
As a step towards using them for this purpose, remove the explicit double
quotes and use the macro '#' character to add the double quotes back when
placing them into the '_fields' array 'id' member.
Add a 'flags' field to columns.h, and set it to 0 by default.
Define FIELD_MODIFIABLE flag to indicate whether a 'set' function exists
to change the field's value.
In all top vg read functions only LCK_VG_READ/WRITE can be used.
All other vg lock definitions are low-level backend machinery.
Moreover, LCK_WRITE cannot be tested through bitmask.
This patch fixes these mistakes.
For _recover_vg() we do not need lock_flags, it can be only
two of above and we always upgrading to LCK_VG_WRITE lock there.
(N.B. that code is racy)
There is no functional change in code (despite wrong masking
it produces correct bits:-)
One shiny day we should use libblkid here. But now using LUKS is
very common together with LVM and pvcreate destroys LUKS completely.
So for user's convenience, try to detect LUKS signature and allow abort.
This is not only undocumented but is is also in violation with --help
documentation.
Using --yes without --force is useful in pvcreate when it detects
old signature.
pvcreate detects MD and swap signature.
The logic hidden there is not only documented but it is also
user unfriendly. Who invented this logic should run pvcreate
on its own critical MD device to see why;-)
This patch
- creates one function instead of duplication code
- asks if user want to overwrite signature
- allows aborting (!)
(Please note that writing LVM signatute without wiping old
is wrong, it confuses blkid, MD will not work anyway and
swap and LUKS is broken too.)
We can't rely on the fact that udev should prepare the node with right major
and minor number to trigger the module autoloading. We have to take into
account that the node could be missing or it could exist with improper
major and minor number assigned (e.g. from previous kernel versions in
an environment with static nodes and without udev). Make any corrections
if needed!
The lvm repair issues I believe are the superficial symptoms of this
bug - there are worse issues that are not as clearly seen. From my
inline comments:
* If the mirror was successfully recovered, we want to always
* force every machine to write to all devices - otherwise,
* corruption will occur. Here's how:
* Node1 suffers a failure and marks a region out-of-sync
* Node2 attempts a write, gets by is_remote_recovering,
* and queries the sync status of the region - finding
* it out-of-sync.
* Node2 thinks the write should be a nosync write, but it
* hasn't suffered the drive failure that Node1 has yet.
* It then issues a generic_make_request directly to
* the primary image only - which is exactly the device
* that has suffered the failure.
* Node2 suffers a lost write - which completely bypasses the
* mirror layer because it had gone through generic_m_r.
* The file system will likely explode at this point due to
* I/O errors. If it wasn't the primary that failed, it is
* easily possible in this case to issue writes to just one
* of the remaining images - also leaving the mirror inconsistent.
*
* We let in_sync() return 1 in a cluster regardless of what is
* in the bitmap once recovery has successfully completed on a
* mirror. This ensures the mirroring code will continue to
* attempt to write to all mirror images. The worst that can
* happen for reads is that additional read attempts may be
* taken.
Ignore snapshots when performing mirror recovery beneath an origin.
Pass LCK_ORIGIN_ONLY flag around cluster.
Add suspend_lv_origin and resume_lv_origin using LCK_ORIGIN_ONLY.
DM devices were not handled properly on nodes in a cluster that were not
where the splitmirrors command was issued. This was happening because
suspend_lv/resume_lv were being used in a place where activate_lv should
have been used.
When the suspend/resume are issued on (effectively) new LVs, their
'resource' (UUID) is not located in the lv_hash. Thus, both operations
turn into no-ops. You can see this from the output of clvmd from one
of the remote nodes:
<snip>
do_suspend_lv, lock not already held
<snip>
do_resume_lv, lock not already held
'activate_lv' enjoins the other nodes in the cluster to process the lock
and activate the new LV. clvmd output from remote node as follows:
do_lock_lv: resource 'zMseY7CBuO3Ty09vXlplPAHzD0Y0CovjrTdv0R1VcwggMwPdYhutHErRcwm5Nd2S', cmd = 0x19 LCK_LV_ACTIVATE (READ|LV|NONBLOCK), flags = 0x84 (DMEVENTD_MONITOR ), memlock = 1
sync_lock: 'zMseY7CBuO3Ty09vXlplPAHzD0Y0CovjrTdv0R1VcwggMwPdYhutHErRcwm5Nd2S' mode:1 flags=1
sync_lock: returning lkid 27b0001
Signed-off-by: Jonathan Brassow <jbrassow@redhat.com>
Reviewed-by: Petr Rockai <prockai@redhat.com>
This can happen with older rules (without support for synthesized events)
that are still part of initrd while using new udev rules in the system itself.
The consequence was that new udev rules incorrectly assumed that not having
DM_UDEV_PRIMARY_SOURCE_FLAG set always means the uevent is synthesized and
inappropriate (device is still not properly activated) and so it should be
ignored. However, initrd is not updated automatically while updating the
libdevmapper/udev rules in the system and so we end up with the rules not
detecting and setting crucial parts in the initrd environment and the rules
in the system that rely on the information that should have been stored in
udev db (which is incorrect in this configuration, of course).
The overall consequence is that the update of libdevmapper/lvm2 without
regenerating the initrd could end up with a boot failure! Ignoring the event
means removing any existing symlinks in /dev!
To fix this, increase udev rules version to make a difference. So from now on,
mark rules without proper support for synthesized events as
DM_UDEV_RULES_VSN="1" and 2 (or higher) if that support is included.
We still need to detect this one! We're not so strict with CHANGE events as
with the ADD events while applying filters in the rules so this one would
pass and it would process the rules prematurely (because it appears *before*
the actual CHANGE event used when resuming a DM device while setting read-only
state at the same time).
clvmd daemon itself does the right thing when invoked as non-root, by
returning 4.
The patch removes the use daemon function from
/etc/rc.d/init.d/functions that´s unnecessary and has th bad habit to
mask the return codes from the real daemon.
Add a simple and generic check to see if clvmd is executed by root or not.
Our stop/reload/restart paths in the init script are complex and not all
the tools involved in the process are guaranteed to return 4 if executed
by non-root against a process that´s running as root (for example kill
-TERM will return -1 and parsing the output to catch the error is
suboptimal at best).
https://bugzilla.redhat.com/show_bug.cgi?id=553381
The new standard in the storage industry is to default alignment of data
areas to 1MB. fdisk, parted, and mdadm have all been updated to this
default.
Update LVM to align the PV's data area start (pe_start) to 1MB. This
provides a more useful default than the previous default of 64K (which
generally ended up being a 192K pe_start once the first metadata area
was created).
Before this patch:
# pvs -o name,vg_mda_size,pe_start
PV VMdaSize 1st PE
/dev/sdd 188.00k 192.00k
After this patch:
# pvs -o name,vg_mda_size,pe_start
PV VMdaSize 1st PE
/dev/sdd 1020.00k 1.00m
The heuristic for setting the default alignment for LVM data areas is:
- If the default value (1MB) is a multiple of the detected alignment
then just use the default.
- Otherwise, use the detected value.
In practice this means we'll almost always use 1MB -- that is unless:
- the alignment was explicitly specified with --dataalignment
- or MD's full stripe width, or the {minimum,optimal}_io_size exceeds
1MB
- or the specified/detected value is not a power-of-2
Introduce --norestorefile to allow user to override the new requirement.
This can also be overridden with "devices/require_restorefile_with_uuid"
in lvm.conf -- however the default is 1.
Signed-off-by: Mike Snitzer <snitzer@redhat.com>
We can already detect MD devices internally. But when using MD partitions,
these have "block extended major" (blkext) assigned (259). Blkext major
is also used in general, so we need to check whether the original device
is an MD device actually.
An incorrect fix on July 13, 2010 for an annoyance has caused a regression.
The offending check-in was part of the 2.02.71 release of LVM. That
check-in caused any PVs specified on the command line to be ignored when
performing a mirror split.
This patch reverses the aforementioned check-in (solving the regressions)
and posits a new solution to the list reversal problem. The original
problem was that we would always take the lowest mimage LVs from a mirror
when performing a split, but what we really want is to take the highest
mimage LVs. This patch accomplishes that by working through the list in
reverse order - choosing the higher numbered mimages first. (This also
reduces the amount of processing necessary.)
Signed-off-by: Jonathan Brassow <jbrassow@redhat.com>
Reviewed-by: Takahiro Yasui <takahiro.yasui@hds.com>
corruption bug in cmirror. 'dm_bit' is only ever used as a boolean operation
within LVM, but it can return a range of values. If the bit is set, a power of
2 is returned. If the bit is unset, 0 is returned.
'log_test_bit' (a function in the cluster mirror log daemon code) has switched
to using the dm bit operations in rhel6. There are two places in the daemon
code where 'log_test_bit' is not used merely as a boolean, but rather the
return value is used as the return value for the log functions 'is_clean' and
'in_sync' - having assumed that 'dm_bit' was returning 0 or 1 only.
One place the 'in_sync' function is utilized is in 'dm_rh_get_state' - a
function that informs the mirroring code how to treat I/O and which devices to
read/write from. 'dm_rh_get_state' was checking if the return value of
'in_sync' was 1 to determine if the region was DM_RH_CLEAN. Since 'dm_bit'
(and by extension 'log_test_bit' and 'in_sync') was returning powers of 2,
DM_RH_CLEAN was rarely being reported as it should have been. Thinking the
region was out-of-sync, the mirroring code would write only to the primary
device. When the primary device was failed, all of those writes were lost -
leaving the entire mirror corrupted.
udev_sync feature requires semaphores (part of System V IPC) to be configured
in kernel (CONFIG_SYSVIPC). Check whether it is supported and if not, give
a warning message and disable udev synchronisation code automatically to
avoid any further error states and associated problems.
One should use the kernel with System V IPC support enabled or libdevmapper
with udev_sync feature disabled.
all but one mirror leg.
<patch header>
To handle a double failure of a mirrored log, Jon's two patches are
commited, however, lvconvert command can't still handle an error
when mirror leg and mirrored log got failure at the same time.
[Patch]: Handle both devices of a mirrored log failing (bug 607347)
posted: https://www.redhat.com/archives/lvm-devel/2010-July/msg00009.html
commit: https://www.redhat.com/archives/lvm-devel/2010-July/msg00027.html
[Patch]: Handle both devices of a mirrored log failing (bug 607347) -
additional fix
posted: https://www.redhat.com/archives/lvm-devel/2010-July/msg00093.html
commit: https://www.redhat.com/archives/lvm-devel/2010-July/msg00101.html
In the second patch, the target type of mirrored log is replaced with
error target when remove_log is set to 1, but this procedure should be
also used in other cases such as the number of mirror leg is 1. This
patch relocates the procedure to the main path.
In addition, I added following three changes.
- Removed tmp_orphan_lvs handling procedure
It seems that _delete_lv() can handle detached_log_lv properly
without adding mirror legs in mirrored log to tmp_orphan_lvs.
Therefore, I removed the procedure.
- Removed vg_write()/vg_commit()
Metadata is saved by vg_write()/vg_commit() just after detached_log_lv
is handled. Therefore, I removed vg_write()/vg_commit().
- With Jon's second patch, we think that we don't have to call
remove_mirror_log() in _lv_update_mirrored_log() because will be
handled remove_mirror_images() in _lvconvert_mirrors_repaire().
</patch header>
Signed-off-by: Takahiro Yasui <takahiro.yasui@hds.com>
Reviewed-by: Petr Rockai <prockai@redhat.com>
Signed-off-by: Jonathan Brassow <jbrassow@redhat.com>
The cluster log daemon (cmirrord) is not multi-threaded and
can handle only one request at a time. When a log is stacked
on top of a mirror (which itself contains a 'core' log), it
creates a situation that cannot be solved without threading.
When the top level mirror issues a "resume", the log daemon
attempts to read from the log device to retrieve the log
state. However, the log is a mirror which, before issuing
the read, attempts to determine the 'sync' status of the
region of the mirror which is to be read. This sync status
request cannot be completed by the daemon because it is
blocked on a read I/O to the very mirror requesting the
sync status.
With mirror_log_fault_policy of 'remove' and mirror_image_fault_policy
of 'allocate', the log type of the mirror volume is converted from
'disk' or 'mirrored' to 'core' when all mirror legs but one in a mirror
volume broke.
Keep new_log_count as a number of valid log devices by using log_count
variable for a temporary usage in the first phase of error recovery
in _lvconvert_mirrors_repair().
Signed-off-by: Takahiro Yasui <takahiro.yasui@hds.com>
Reviewed-by: Petr Rockai <prockai@redhat.com>
There was missing "revert" call in _create_and_load_v4 fn while the preparation
for table load ends up with failure in create/load/resume sequence. Otherwise
we could end up with a device being created, but not table-loaded nor resumed.
Even though the table is not loaded and the device is not resumed at this
stage, we still need to synchronize with udev when calling the revert
"remove" ioctl - there's still a remove uevent generated! The "revert"
code does exactly that.
CMIRRORD_PIDFILE is not defined. This makes the build fail.
Therefore, we need to conditionalize the check for cmirrord
based on if CMIRRORD_PIDFILE is defined.
mirrors, we must also check that the log daemon (cmirrord) is running.
The log module can be auto-loaded, but the daemon cannot be
"auto-started". Failing to check for the daemon produces cryptic
messages that customers have a hard time deciphering. (The system
messages do report that the log daemon is not running, but people
don't seem to find this message easily.)
Here are examples of what is printed when the module is available,
but the log daemon has not been started.
[root@bp-01 LVM2]# lvcreate -m1 -l1 -n lv vg
Shared cluster mirrors are not available.
[root@bp-01 LVM2]# lvcreate -m1 -l1 -n lv vg -v
Setting logging type to disk
Finding volume group "vg"
Archiving volume group "vg" metadata (seqno 3).
Creating logical volume lv
Executing: /sbin/modprobe dm-log-userspace
Cluster mirror log daemon is not running
Shared cluster mirrors are not available.
Creating volume group backup "/etc/lvm/backup/vg" (seqno 4).
When splitting off mirror images from a mirror, we always take
LVs from the end of a list. For example, if the mirror sub-devices
are lv_mimage_[012], we should select lv_mimage_2 if splitting off
one image. However, lv_mimage_0 was being selected instead.
The problem came from calling '_move_removable_mimages_to_end'
when it was unnecessary to do so. When the user /does/ specify
specific devices to be removed, this function properly moved the
appropriate LVs to the end of the list for extraction. However,
if the user /doesn't/ give any specific PVs, the function should
do nothing. '_move_removable_mimages_to_end' was keying off of
whether 'removable_pvs' was NULL or not and this value was
improperly being populated with the set of all available PVs.
This was causing '_move_removable_mimages_to_end' to completely
reverse the list, which in turn caused us to extract the
hithertofore front-of-the-list LVs.
An unhandled condition allowed the command to terminate
cleanly without a warning. Added a check for the
'--splitmirrors' argument to allow execution to the lower
level function that has the check to see if the user is
trying to split a linear device. You should now see a
message if you try to use --splitmirrors on a linear device.
The main problem with these bugs was that the newly split
off LV was not being suspended properly. This meant that
the memlock count was not being balanced, the DM devices
were not being renamed, and some DM devices which should
have been removed were not.
I've also renamed some of the variables and added comments
to make things clearer as to what is going on. (I can break
this patch in two if it means easier review.)
pvchange: Add --metadataignore description
vgchange: Fix minor formatting
pvcreate: Update metadataignore description to refer to pvchange
lvm.conf: Refer to pvcreate and pvchange for metadata options.
Switch dmeventd to use dm_create_lockfile and drop duplicate code.
Allow clvmd pidfile to be configurable.
Switch cmirrord and clvmd to use dm_create_lockfile.
This should bring less confusion when there are some settings left and
people just forgot about it and then they run into problems. These messages
should give them a hint of what's really going on.
even though there was no log. A simple run through the in-tree test
suite would have caught this. :(
- if (lv_is_mirrored(detached_log_lv) &&
+ if (detached_log_lv && lv_is_mirrored(detached_log_lv) &&
Also, made some cosmetic changes suggested by kabi after my last check-in
(e.g. s/return 0/return_0/ and adding an error message).
A previous check-in added logic to handle the case where both images
of a mirrored log failed. It solved the problem by simply removing
the log entirely - leaving the parent mirror with a 'core' log. This
worked for most cases. However, if there was a small delay between
the failures of the two mirrored log devices, the mirror would hang,
LVM would hang, and no additional LVM commands could be issued.
When the first leg of the log fails, it signals the need for repair.
Before 'lvconvert --repair' is run by dmeventd, the second leg fails.
'lvconvert' would see both devices as failed and try to remove the
log entirely. When it came time to suspend the parent mirror to
update the configuration, the suspend would hang because it couldn't
get any I/O through the mirrored log, which was plugged waiting for
corrective action. The solution is to replace the log with an error
target to clear any pending writes before removing it. This allows
the parent mirror to suspend and make the proper changes.
Pass metadataignore through PV creation / setup paths.
As a result of this cleanup, we can remove the unnecessary setting
of mda_ignore bits inside pvcreate_single(), after call to pv_create.
For now, just set metadataignore to '0' in some places. This is
equivalent to the prior functionality, although the 0 is given
by the caller not hardcoded in _mda_setup() call.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
Indent updates.
Use AC_HELP_STRING for help string.
Start help string with lower letter.
Add [] around some default values i.e. [TYPE=internal].
Skip some "" around shell assigment when not needed.
Fix typo --with-device-gid=UID string.
When using vgmetadatacopies value other than "umanaged" (0), prompt
the user if the usage of --metadataignore would change the value of
vgmetadatacopies. The main 2 cases are:
1) pvchange --metadataignore
2) vgextend --metadataignore
We leave the prompt check in the tools, and do not change anything
if the user says 'n'.
Examples:
vgextend --metadataignore y vgtest /dev/loop0
Setting metadataignore will override preferred number of copies of VG vgtest metadata.
Are you sure? [y/n]: y
No physical volume label read from /dev/loop0
Physical volume "/dev/loop0" successfully created
Volume group "vgtest" successfully extended
pvchange --metadataignore y /dev/loop3
Setting metadataignore on /dev/loop3 will override preferred number of copies of VG vgtest metadata.
Are you sure? [y/n]: y
WARNING: Changing preferred number of copies of VG vgtest metadata from 3 to 2
Physical volume "/dev/loop3" changed
1 physical volume changed / 0 physical volumes not changed
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
Test the auto-repair capability when we fail committing to an mda
on a new pv adding to a vg. This test should fail until we fix
the auto-repair in this case.
For now, this is just a precaution. Normally, all the other (non-dm) rules
should check DM_UDEV_DISABLE_OTHER_RULES_FLAG and therefore avoid setting
any inotify watches as well. But let's make sure.
Support for final assignment of the "nowatch" rule (the use of ":=") will
appear in next udev release, v160. This should also work in previous udev
versions but the setting won't be sealed so any further OPTIONS="watch" will
always prevail there.
We may want to add more specific "nowatch" rules later if needed.
- If a PV contained empty mdas, the auto-recovery code was not kicking in.
- The 'inconsistent' state was getting lost when metadata was cached so
recovery didn't kick in. But leave the behaviour alone when using
precommitted metadata because of a warning in a confusing FIXME.
In my testing, pvs and vgs didn't repair inconsistent metadata like they
used to do. (How many other tools fail similarly now?)
And there should be no need to cache inconsistent metadata because it is
supposed to get repaired under the protection of a write lock immediately it is
discovered.
This code is in need of a redesign based on first principles.
I still see bugs in this code and this commit is risky.
Rather than attempting to remove all the images of a mirrored
log volume via remove_mirror_images, simply remove the log
if all its devices have failed.
Taka was the first to report that there is still an outstanding
issue with handling this case. I've managed to reproduce it
only very rarely, and am still working on identifying the problem.
Failing to handle the problem rarely is better than not handling
the scenario at all, so I'm checking this in.
Moreover, in current mirror handling, when it calls activate
on removed but suspended detached log this counter drops below zero
and confuses debug log.
When a mirror is being downconverted in a cluster, a series of suspends and
resumes is executed.
With the change to using UUIDs in dev_manager instead of names, the behaviour
has changed with regards to including an _mlog in the deptree of a logical
volume. In the old (pre-UUID-enabled) code, the _mlog would appear in a deptree
of any volume purely based on a name match: a linear volume foo would include
foo_mlog in its dependencies if that happened to exist. This behaviour was
fixed and the mlog is now only included for mirrors.
By a coincidence, this mlog bug had been hiding a different bug in clvmd. When
a mirror is being dismantled (and converted to a linear volume), it is first
suspended as a whole, then later resumed in parts. Nevertheless, the overall
memlock balance is maintained in this operation. The problem kicks in, because
even though the mirror log was suspended as part of the mirror, when the
dismantled mirror is resumed again, it is no longer a mirror and therefore the
mirror log stays suspended. This would not be a problem in itself, since
_delete_lv (from metadata/mirror.c) is called on it subsequently, which does an
activate/deactivate cycle and removes the LV. The activate/deactivate cycle
correctly prompts clvmd to resume the device: however, in doing this, it will
issue an unpaired resume operation (the suspend that caused the mirror log to
be suspended is paired with resuming the dismantled mirror later). We have
concluded that the path in clvmd should never affect memlock_count, since there
should never be an unmatched explicit suspend preceding this resume.
Allow metadataignore flag to be passed in to pvcreate.
Ideally, more refactoring of the mda allocation / initialization
is warranted, but for now, we just add another parameter to 'add_mda'
to take an existing mda ignored flag. We need to do this or pv_write
loses the state of the mda 'ignored' flag before copying and writing
to disk.
Print device name when setting or clearing metadata ignore bit.
Example:
label/label.c:160 /dev/loop2: lvm2 label detected
cache/lvmcache.c:1136 lvmcache: /dev/loop2: now in VG #orphans_lvm2 (#orphans_lvm2)
metadata/metadata.c:4142 Setting mda ignored flag for metadata_locn /dev/loop2.
format_text/text_label.c:318 Skipping mda with ignored flag on device /dev/loop2 at offset 4096
Logging isn't ideal, especially for mda_set_ignore. Ideally we'd
like to display the device name and offset in this case but this
requires a bit more work and a per-format 'mda_description' function
pointer definition (we don't have access to mda_context in
metadata.c).
In preparation to call this from both pvcreate as well as pvchange,
move the guts of metadataignore into a library function.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
There's an intermittent failure with vgcfgbackup that seems to have been
introduced with the metadataignore / vgmetadatacopies patchset.
Intermittent failures are often the result of uninitialized data,
so this patch calls zalloc in a few places it might matter.
Update example.conf to describe vgmetadatacopies. Provide an
explanation for the '0' ("unmanaged") value.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
Allowing an 'all' and 'unmanaged' value is more intuitive, and
provides a simple way for users to get back to original LVM behavior
of metadata written to all PVs in the volume group.
If the user requests "--vgmetadatacopies unmanaged", this instructs
LVM not to manage the ignore bits to achieve a specific number of
metadata copies in the volume group. The user is free to use
"pvchange --metadataignore" to control the mdas on a per-PV basis.
If the user requests "--vgmetadatacopies all", this instructs LVM
to do 2 things: 1) clear all ignore bits, and 2) set the "unmanaged"
policy going forward.
Internally, we use the special MAX_UINT32 value to indicate 'all'.
This 'just' works since it's the largest value possible for the
field and so all 'ignore' bits on all mdas in the VG will get
cleared inside _vg_metadata_balance(). However, after we've
called the _vg_metadata_balance function, we check for the special
'all' value, and if set, we write the "unmanaged" value into the
metadata. As such, the 'all' value is never written to disk.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
Now that we have both --pvmetadatacopies and --vgmetadatacopies,
we need to make sure --metadatacopies gets interpreted correctly.
For pv commands, --metadatacopies should imply --pvmetadatacopies,
and for vg commands, --vgmetadatacopies.
Note: this will change the behavior of vgcreate with --metadatacopies
to be a synonym for --vgmetadatacopies. Previously, --metadatacopies
would apply to any PVs given with vgcreate that needed an implicit
pvcreate. As a result, one small change is needed to one of the nightly
tests - t-vgcreate-usage.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
When vgmerge is called we move the mdas from the source to the
destination. With metadata balancing we now have another mda
list, fid->metadata_areas_ignored, so move the mdas on this list
as well.
This patch should not matter as the code is written today. However
we include it for completeness in the case that _vgmerge_single()
is refactored and/or moved into a library function.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
The check in vg_split_mdas will trigger an error if the 'from' vg
list is empty. However, this might be ok in some instances now
that we have ignored mdas. Relax this check so an error is triggered
only in the case where there's truly no more mdas in the 'from'
vg.
One example of where this makes a difference is with vgreduce.
If we try to vgreduce a PV with un-ignored mdas, this should trigger
the balancing function to un-ignore mdas on another PV in the VG.
However, we don't get to vg_write() before we fail because this
list size check fails, and we see an error message indicating:
"Cannot remove final metadata area ..."
Another example is with vgsplit into a new VG, where the PVs
being moved contain all ignored mdas. We must move the mdas on
fid->metadata_areas_ignored from 'vg_from' to 'vg_to'.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
The vgextend path calls add_pv_to_vg(). Inside add_pv_to_vg(),
we must ensure we pass the correct mdas list into pv_setup(), as
copies of mdas are placed on the vg->fid list. If we don't place
the mdas on the correct vg->fid list, the various counts may be
incorrect and the metadata balance algorithm will not work when
called from vg_write() path.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
Allow parsing of --vgmetadatacopies for vgcreate. Accept
--metadatacopies as a synonym for --vgmetadatacopies.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
When a user explicitly sets a new mda ignore value for a PV, we
should update vg_mda_copies accordingly. When the VG is written
out, the user would not want the new ignore state to get lost as
a result of the vg_mda_copies value and logic in the vg_write
path.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
Compare the value of the newly added vg_mda_copies field
(--vgmetadatacopies parameter) with the current count of
in-use mdas and ignoring or unignoring mdas as necessary to
get to the target count. Also, as a safety check before
returning, ensure we have at least one mda enabled.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
Update logic in vgchange to handle --vgmetadatacopies, allow
--metadatacopies as a synonym to --vgmetadatacopies,
and add these parameters to args.h and commands.h
Forbit both --vgmetadatacopies and --metadatacopies as only
one allowed.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
This patch adds the ability to read/write the vg->mda_copies values
from/to the vg metadata.
If we read the VG metadata and this field does not exist, we set
mda_copies to the default value of 0. Later in the code, we use
this special '0' value to indicate a disable of metadata balancing.
This should preserve existing LVM behavior and ensure metadata balancing
can be turned off should the need arise.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
This patch adds the get and partially implemented set function.
The 'set' function should probably ignore or un-ignore metadata areas
based on new values.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
Add a field to struct volume_group to later implement metadata
balancing:
- mda_copies: target # of non-ignored mdas in the VG; default 0 (do
not control pv 'ignore mdas' bit.
This patch just adds the parameter to the structures with the default
values but does not modify any commands. Should be no functional change.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
This patch adds a vgmetadatacopies parameter for metadata balancing.
This parameter provides a simple way for users to create a policy for
placing metadata on PVs automatically by LVM. The behavior is implemented
inside LVM by managing the 'ignore' mda bits. We chose the name
'vgmetadatacopies' as this is a natural extension to the existing parameter
'pvmetadatacopies' / 'metadatacopies' in pvcreate.
This is a first step at VG parameter based metadata balancing. Most users
will probably want to state that they want a certain number of PVs to contain
metadata, and they may be less concerned about a specific number of metadata
copies in the volume group. However, for default values (pvmetadatacopies
is 1 by default), the number of metadatacopies in the volume group, and the
number of PVs with metadata are the same. In the future we could add
vgmetadatacopiespvs to define more specifically the number of pvs in the
VG that contain metadata, but for now we start with this parameter.
Another possible future extension would be to define a specific pv tag
to mark the set of PVs that should be used for metadata balancing. This
tag based approach could be used in conjunction with 'vgmetadatacopies'.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
Arrange mdas so mdas that are to be ignored come first. This is an
optimization that ensures consistency on disk for the longest period of time.
This was noted by agk in review of the v4 patchset of pvchange-based mda
balance.
Note the following example for an explanation of the background:
Assume the initial state on disk is as follows:
PV0 (v1, non-ignored)
PV1 (v1, non-ignored)
PV2 (v1, non-ignored)
PV3 (v1, non-ignored)
If we did not sort the list, we would have a commit sequence something like
this:
PV0 (v2, non-ignored)
PV1 (v2, ignored)
PV2 (v2, ignored)
PV3 (v2, non-ignored)
After the commit of PV0's mdas, we'd have an on-disk state like this:
PV0 (v2, non-ignored)
PV1 (v1, non-ignored)
PV2 (v1, non-ignored)
PV3 (v1, non-ignored)
This is an inconsistent state of the disk. If the machine fails, the next
time it was brought back up, the auto-correct mechanism in vg_read would
update the metadata on PV1-PV3. However, if possible we try to avoid
inconsistent on-disk states. Clearly, because we did not sort, we have
a greater chance of on-disk inconsistency - from the time the commit of
PV0 is complete until the time PV3 is complete.
We could improve the amount of time the on-disk state is consistent by simply
sorting the commit order as follows:
PV1 (v2, ignored)
PV2 (v2, ignored)
PV0 (v2, non-ignored)
PV3 (v2, non-ignored)
Thus, after the first PV is committed (in this case PV1), on-disk we would
have:
PV0 (v1, non-ignored)
PV1 (v2, ignored)
PV2 (v1, non-ignored)
PV3 (v1, non-ignored)
This is clearly a consistent state. PV1 will be read but the mda will be
ignored. All other PVs contain v1 metadata, and no auto-correct will be
required. In fact, if we commit all PVs with ignored mdas first, we'll
only have an inconsistent state when we start writing non-ignored PVs,
and thus the chances we'll get an inconsistent state on disk is much
less with the sorted method.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
When we are constructing the vg, we may need to adjust the list of
metadata_areas if there are ignored mdas. At label read time, we
do not read the metadata of ignored mdas, and as a result, they do
not get placed on vg->fid->metadata_areas inside _text_create_text_instance
since lvmcache does not have these areas attached to vginfo->infos.
However, when we're checking the pvids inside _vg_read, after having
read another metadata area from another PV, we do have the opportunity
to update the metadata_area and metadata_areas_ignored lists based
on the read metadata_area. We need accurate mda lists for the reporting
functions that count the ignored mdas, as well as general correctness
of mda balancing.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
With the addition of ignored mdas, we replace all checks for an empty
mda list with a new function to look for either an empty mda list or
ignored mdas.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
Add a helper function to consolidate checking for an empty mdas list
or ignored mdas. Ignored mdas should behave almost identically to
an empty mda list - the metadata areas should not be read or written
to. This function will make it easier to implement metadata balancing
and easier to track pvs with an empty mda list or ignored mdas.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
We implement ignore of an mda at label_read time by checking for
the ignore bit, and then skipping the reading of the vgname and
other information in the metadata. This will have an effect similar
to a PV found with no mdas. Thus, it will look like an orphan in the
cache until we scan the rest of the system and find a PV with
metadata, and the mda will not be on the vg->fid->metadata_areas
list so no read/writes will be done to the metadata area.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
This patch just modifies pvchange to call the underlying ignore
functions for mdas. Ensure special cases do not reflect changes
in metadata (PVs with 0 mdas, setting ignored when already ignored,
clearing ignored when not ignored).
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
Define a new pvs field, pv_mda_used_count, and a new vgs field,
vg_mda_used_count to match the existing pv_mda_count and vg_mda_count.
These new fields count the number of mdas that have the 'ignored' bit
clear (they are in use on the PV / VG). Also define various supporting
functions to implement the counting as well as setting the ignored
flag and determining if an mda is ignored. These high level functions
call into the lower level location independent mda ignore functions
defined by earlier patches.
Note that counting ignored mdas in a vg requires traversing both lists
and checking for the ignored bit on the mda. The count of 'ignored'
mdas then is defined by having the bit set, not by which list the mda
is on. The list does determine whether LVM actually does read/write to
the mda, though we must count the bits in order to return accurate numbers
for the various counts. Also, pv_mda_set_ignored must search both vg
lists for ignored mda. If the state changes and needs to be committed
to disk, the ignored mda will be on the non-ignored list.
Note also in pv_mda_set_ignored(), we must properly manage the mda lists.
If we change the ignored state of an mda, we must change any mdas on
vg->fid->metadata_areas that correspond to this pv. Also, we may
need to allocate a copy of the mda, as is done when fid->metadata_areas
is populated from _vg_read(), if we are un-ignoring an ignored mda.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
Add a second mda list, metadata_areas_ignored to fid, and a couple
functions, fid_add_mda() and fid_add_mdas() to help manage the lists.
These functions are needed to properly count the ignored mdas and
manage the lists attached to the 'fid' and ultimately the 'vg'.
Ensure metadata_areas_ignored is initialized in other formats, even
if the list is never used.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
Because of the way mdas are handled internally, where a PV in a VG
has mdas on both info->mdas and vg->fid->metadata_areas list, we
need a location independent copy constructor for struct
metadata_area. Break up the existing format-text specific copy
constructor into a format independent piece and a format dependent
piece.
This function is necessary to properly implement pv_set_mda_ignored().
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
Reviewed-by: Alasdair G Kergon <agk@redhat.com>
A metadata_area is defined independent of the location. One downside
is that there is no obvious mapping from a pv to an mda. For a PV in
a VG, we need a way to start with a PV and end up with an MDA, if we
are to manage mdas starting with a device/pv. This function provides
us a way to go down the list of PVs on a VG, and identify which ones
match a particular PV.
I'm not entirely happy with this approach, but it does fit into the
existing structures in a reasonable way.
An alternative solution might be to refactor the VG - PV interface such
that mdas are a list tied to a PV. However, this seemed a bit tricky since
a PV does not come into existence until after the list of mdas is
constructed (see _vg_read() - we create a 'fid' and attach mdas to it,
then we go through them and attach pvs).
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
Reviewed-by: Alasdair G Kergon <agk@redhat.com>
We'd like to pass in mda_header to vgname_from_mda(). In order to
do this, we need to call raw_read_mda_header() from text_label.c,
_text_read(), which gets called from the label_read() path, and
peers into the metadata and update vginfo cache. We should check
the disable bit here, and if set, not peer into the vg metadata,
thus reducing the I/O to disk.
In the process, move vgname_from_mda() to layout.h, since the fn
only gets called from format_text code, and we need the mda_header
definition from the private layout.h.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
This refactoring moves the device open/close up one level to the caller of
_vg_read_raw_area(). Should be no functional change and facilitate future
changes related to metadata balancing.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
First we add a 'flags' field to the location independent
metadata_area structure, and a MDA_IGNORE flag. The
mda_is_ignored and mda_set_ignored functions are added to
manage the flag. Adding the flag and functions gives a
library interface to ignore metadata areas independent of
the underlying location (disk, file, etc). The location
specific read/write functions must then handle the specifics
of what this flag means to the location.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
Reviewed-by: Alasdair G Kergon <agk@redhat.com>
Adding a flag to the 'rlocn' structure in the mda header of the
text format allows us to flip a bit to ignore an area on disk that
stores the metadata via the text format specific mda_header.
This patch defines the flag and access functions to manage the flag.
Other patches will manage the ignore on a format-independent basis,
by using a flag in the metadata_area structure.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
Future patches will make use of a specific flag in the on-disk 'raw_locn'
structure to enable/disable metadata areas, and facilitate metadata
balancing.
Note that 'filler' is always set to '0' (see add_mda() - memset),
so use of this area as a non-zero flags field is a safe way to
provide future code features.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
The same region size is used for both mirror volume and mirrored
log volume, but when the physical extent size is bigger than region size,
the size of mirror leg for mirrored log is smaller than the region size
and lvcreate command fails.
This patch adjusts a region size of mirrored log to a smaller value of
region size or physical extent size.
[This patch ensures that the region_size of the mirrored log does not
exceed the size of the mirrored log itself, which would violate the
kernel constraint: (region_size <= ti->len).]
Signed-off-by: Takahiro Yasui <takahiro.yasui@hds.com>
Signed-off-by: Jonathan Brassow <jbrassow@redhat.com>
Preload libc.mo file for localized lvm before taking memory lock - this way
we prevent disk access for some error paths in libdm, that prints localized
errno messages while they are still in memory locked state.
the failure of a device that contained both a image of
a mirror and an image of the mirrored log. The order
of the handling of those faults was important (and
wrong), this patch corrects that.
Patch-From: Takahiro Yasui <tyasui@redhat.com>
We can use DM_UDEV_PRIMARY_SOURCE_FLAG to identify the spurious events
and use it as an indication that the device has already been activated before
(and hence we can find this property in udev database).
WARNING: This change requires udev startup script to preserve udev database
from initrd. All the information stored there during activation of devices
is important for the initial "udevadm trigger --action=add" call that is
used in udev startup script. If not done this way, udev startup script needs
to define DM_UDEV_PRIMARY_SOURCE_FLAG=1 property for any ADD events it uses.
- s/Active clustred VG/clustered VG/ (only LV can be active)
- print only active LVs (not all) in status command
(In the lvdisplay form /dev/vg/lv.)
For now, still use awk (already used in clustered_vgs).
https://bugzilla.redhat.com/show_bug.cgi?id=598495
converting from 2-way to 3-way mirror (collapse_mirrored_lv)
was calling '_remove_mirror_images' with the 'remove_log'
parameter set. When the code was put in to fix 599898 to
honor log parameters during conversion, this argument was
suddenly being honored. Thus, when someone would convert from
a 2-way to 3-way mirror, the log would get removed.
'collapse_mirrored_lv' should not be calling '_remove_mirror_images'
with 'remove_log' set.
to 3-way mirror. When conversion operations are performed on
these types of mirrors, log options can be confused/ignored.
In the case of a converting 3-way mirror, we have a top-level
2-way corelog mirror whose legs are 1) a 2-way disk-log mirror
and 2) a linear device. If we wish to convert this 3-way mirror
to a 2-way mirror, the linear device is removed and the extra
top layer is eliminated. If we also wished to convert the disk
log to a core log in the same step, ambiguity creeps in. It is
somewhat obvious what the user wants - a 2-way mirror with a
corelog. However, looking at the top level mirror before
compression, it seems that the mirror already has a core log.
This is why the operation seemed to fail.
This patch simply re-evaluates what mirrored_seg points to after
a compression and then considers the log argument.
This is a fix for bug 599898.
Because execve stops the command loop,
we never receive response (only socket close) for clvmd -S,
so waiting for response here makes no sense.
But if the calling process (clvmd -S) exits too early, connection
is closed from client side, clvmd takes this as an error and
never run restart code.
Ugly hack(TM).
When using clustered mirrors, we need device nodes to be created during
processing of device tree, not at its end like we normally do (we need to
access the nodes in cmirror prematurely). Therefore we use a new flag called
"immediate_dev_node" stored in deptree's load_properties struct to instruct the
device tree processing code to immediately synchronize with udev and flush all
stacked node operations so the nodes are prepared for use.
For now, the immediate_dev_node is used for clustered mirrors during
processing the dm_tree_preload_children code only. We can add more later if
needed.
linux/kdev_t.h even though it wasn't needed. Strangely, it seems
to be causing problems on various architectures (i686) in the
function daemons/cmirrord/functions.c:disk_status_info()->sprintf.
I'm not sure why this is a problem since none of the macros in
kdev_t.h are used in that code, but it certainly doesn't hurt to
pull an unnecessary header and it seems to fix the problem.
Code is mixing up internal DLM and LVM definitions of lock
modes and flags.
OpenAIS and singlenode locking do not depend on DLM but
code currently cannot be compiled without libdlm.h!
LCK_* flags is LVM abstraction, used through all the code.
Only low-level backend (clvmd-cman etc) should use DLM definitions,
also this code should do all needed conversions.
Because there are two DLM flags used in generic code
(NOQUEUE, CONVERT) we define it similar way like lock modes.
(So all needed binary-compatible flags are on one place in locking.h)
(Further code cleaning still needed, though:-)
Introduce lvm_exec_prefix with resolved exec_prefix.
(using same ac_default_prefix as for CLVMD_PATH)
Use lvm_exec_prefix instead of dmeventd_prefix (fixes missing ac_default_prefix)
Note: This patch is rather hot-fix as currently generate code
does not create correct code for make exec_prefix=
- allocate environment dynamically (still missing some limit?)
- try to recover, if destroy failed (do not destroy lvm here) and free memory
- check strdup() return codes
- report failure to log
- do not print NULL in exclusive lock loop
A kernel patch is on its way for 2.6.35 adding support for dm-mod module
autoload. Udev v155 and higher is able to read static node information given
in modules.devname (extracted by depmod before) and will create such nodes
at its start. The first access to such node will load the module automatically
(directly in kernel) before the actual read/write operation is processed.
Activate only the first replicator-dev LV, that activates all other
related LVs from Replicator. In case of error during this activation,
it will not retry again for other heads (less confusing error log).
length(array) is specific to GNU awk and doesn't work in mawk.
Use a return value of "split" function to indicate array size, this is
supported in both gawk and mawk.
This patch fixes the following errors during "make install" when mawk is
installed as a default awk.
mawk: scripts/relpath.awk: line 25: illegal reference to array from
mawk: scripts/relpath.awk: line 25: illegal reference to array to
mawk: scripts/relpath.awk: line 27: illegal reference to array from
mawk: scripts/relpath.awk: line 32: illegal reference to array to
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
Adding function _add_partial_replicator_to_dtree() to create
partial tree for Replicator target.
Using dm_tree_node_set_presuspend_node() for Replicator.
As for _process_one_vg() we need similar retry loop for
process_each_lv_in_vg(). This patch retries to process
failed LVs with reopened VGs.
Patch does not add any extra repeated invocations if there is not
found any missing VG during LV processing.
Patch modifes behavior of _process_one_vg().
In the first pass vg_read() collectis for replicator sorted list of
additional VGs during lock_vol().
If any other VG is needed by the replicator and it is not yet opened
then next iteration loop is taken with all collected VGs.
Flag vg->cmd_missing_vgs detects missing VGs.
Introduce struct cmd_vg to store information about needed
volume group name, vgid, flags and the pointer to opened VG.
Keep VGs list in alphabetical order for locking order.
Introduce functions:
cmd_vg_add() add new cmd_vg entry.
cmd_vg_lookup() search cmd_vgs for vg_name.
cmd_vg_read() open VGs in cmd_vgs list.
cmd_vg_release() close VGs in reversed order.
Add pointer to linked list of opened VGs. List temporarily keeps
the information about needed or locked and opened VGs for replicator target.
Also add cmd_missing_vgs flag information for quick check and
also for possible continuos process_each_lv() usage where we need
to detect whether failure has been caused by missing VG or
some other reason.
Adding configure.in support for Replicators.
Adding basic lib lvm support for Replicators.
Adding flags REPLICATOR and REPLICATOR_LOG.
Adding segments SEG_REPLICATOR and SEG_REPLICATOR_DEV.
Adding basic methods for handling replicator metadata.
For deactivation of Replicator check in advance that all heads
have open_count == 0. For this presuspend_node is used as all
head nodes are linking this control node.
Introducing dm_tree_node_set_presuspend_node() for presuspending child
node (i.e. replicator control target) before deactivation of parent node
(i.e. replicator-dev target).
This patch presents no functional change to current dtree - only
replicator target currently sets presuspend node for dev nodes.
Patch adds failed_lvnames to the list of parameters for process_each_lv_in_vg().
If the list is not NULL it will be filled with LV names of failing LVs
during function execution.
Application could later reiterate only on failed LVs.
lvm2app forces applications to start with a volume group name,
open the volume group, then operate on individual pvs. In some
cases the application may want to start with a device name rather
than the volume group name. Today, if an application wants to
do this, it must iterate through all the volume groups to find
the volume group that the specific device is attached to.
These new interfaces allow the application to avoid such overhead.
Bump the lvm2app version number to 3.
Earlier patches added some infrastructure to lookup a vgname from
a pvname. We now can cleanup some of the pvchange and other code
by requiring callers that want to modify some pv property:
1) lookup the vgname by the pvname
2) use the vgname to obtain a vg handle
3) get the pv handle from the vg handle
This should work going forward and be a much cleaner interface,
as we move away from pvs as standalone objects.
Some commands start with a pvname, but we'd like to force users to
start with a vg handle to obtain a pv handle. Our best option seems
to be providing a way to look up the vgname from the pvname, and then
require them to use vg_read/vg_open.
In addition to the pvname lookup function, this patch also provides a
lookup by pvid. The lookup by pvid can be used in conjunction with
lvmcache_get_pvids to process all pvs in the system.
The pvid find function first calls lvmcache_vgname_from_pvid, which may
cause the label to be read if it is not in the cache. If the vgname is
returned is an orphan, we then check to see if there are metadata areas,
and if not, we scan every PV on the system by calling scan_vgs_for_pvs().
In most cases we should not need to do this, and by using the info->mdas
count, we avoid calling pv_read() as prior code did. So this patch is a
bit cleaner and should allow us to refactor more of the pv code.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
are active mirrors or snapshots.
We don't have the mechanisms in place to change the device-mapper
tables for those targets that have behavioral differences between
cluster and single machine instances. Allowing users to change
the attribute but not changing the target's behavior can lead to
data corruption.
The following bugs are fixed/avoided by this patch:
235123 - vgchange -c [ny] do not change target types when necessary
289331 - RFE: switching from cluster domain to local domain needs to deactivate volume somehow
289541 - when changing from local to cluster, volumes can not appear to be deactivated
This should avoid various races between dmeventd on multiple nodes
in cluster where one node already repairing device and another
run full scan and locks the device.
the device cache file is dumped both in vgscan and clvmd process.
Unfortunately, clvmd calls lvmcache_label_scan,
it properly destroys persistent filter, but during
persistent_filter_dump it merges old cache content back!
This causes that change in filters is not properly propagated
into device cache after vgscan on cluster.
(Only new devices are added.)
https://bugzilla.redhat.com/show_bug.cgi?id=591861
to a file using log/file, with log/overwrite set and dump this file in
STACKTRACE. The overall effect is that only the command that ran last before
the failure has been triggered will get its debug output logged. This is
similar to how we treat coredumps.
lvconvert testing is now in its own test, t-lvconvert-mirror-basic ... it
doesn't do anything fancy but it does run lvconvert through a lot of
combinations.
I have also merged the remaining t-mirror-lvconvert tests into
t-lvconvert-mirror and abolished the former. The latter will be split again
later into more thematic divisions. (The previous split was rather arbitrary,
may I even say random...)
Use Requires.private: instead of Libs.private:
Use UDEV_PC and SELINUX_PC for Require.private:
It looks like usage of Requires.private is prefered from Libs.private.
However pkg-config documentation is really poor here. But here is
short outcome:
There is a difference in Libs.private: and Requires.private: where
we specify libselinux instead of -lselinux -lsepol.
We leave resolving of query like 'pkg-config --libs --static devmapper'
on taking proper selinux and udev libs to their .pc files instead of
hardcoding them into our .pc file which is might give incorrect answer.
- i.e. dependency of libselinux package might change and we may return
wrong list of linked libraries.
http://bugs.freedesktop.org/show_bug.cgi?id=4738http://err.no/personal/blog/tech/2008-03-25-18-07_pkg-config,_sonames_and_Requires.private
A shortcut for --ignorelockingfailure, --ignoremonitoring, --poll n options
and LVM_SUPPRESS_LOCKING_FAILURE_MESSAGES environment variable used all at
once in initialisation scripts (e.g. rc.sysinit or initrd).
Target install_dm_plugin installs files to libdir/device-mapper.
Target install_lvm2_plugin installs files to libdir/lvm2.
Both targets creates relative links to libdir to keep the code
compatible with current dlopen handling.
Once we will be able to read plugins from subdir, links
could be removed.
(with table provided).
This remove ioctl generates udev events like any other hence it needs to be
synchronized properly as well. Also, add dm task type in debug log when
setting a cookie (for better debugging).
fails, the test will carry on but will issue a warning. The harness detects
such warnings from tests and marks tests that passed with warnings with a
special status.
We can use it even in read-only environment where a try to initialise
file-based locking fails (not to mention other processing related with
lvm2 init). Simply, we want to output the version only, nothing else.
And this should always work.
is active and while it is in-active.
+for i in $(seq 0 4); do
+ for j in $(seq 0 4); do
+ for k in core disk mirrored; do
+ for l in core disk mirrored; do
The testing code still needs some improvement. I'd like to add
the ability to test specifying the PVs to be added/removed during
a convert. It will also be important to test partial PV
specification during down converts (i.e. request to remove more
mirror images than we have provided PVs for).
This rule appeared in udev v152 and it helps us to support spurious events
where we didn't have any flags set (events originated in udevadm trigger
or the watch rule). These flags are important to direct the rule application.
Now, with the help of this rule, we can regenerate old udev db content.
To implement this correctly, we need to flag all proper DM udev events with
DM_UDEV_PRIMARY_SOURCE_FLAG. That happens automatically for all ioctls
generating events originated in libdevmapper.
being able to remove more images from a mirror than the
number of PVs directly specified for removal.
The effort to fix bug 581611 corrected a bug that was unnoticed
at the time. The loop in _remove_mirror_images that looks over
the specified PVs was allowing devices that were previously
counted and moved to the end of the list to be double-counted.
This resulted in the number of devices needed for removal always
being satisfied - even if the user did not specify enough PVs
for removal to satisfy the request. When 581611 was fixed, this
double-counting no longer took place and the result was to remove
only the minimum of the number of PVs specified or the number
that was asked to be removed.
By simply always setting 'new_area_count' (as used to be done
only in the else statement), we return to the previous behavior.
Indeed, this is exactly what the double-counting was allowing
to happen before the fix of 581611.
Allow lv_remove_with_dependencies() to know the top-level LV that was
requested to be removed (otherwise it recurses and we lose context).
A merging snapshot cannot be removed directly but the associated origin
can be. Disallow removal of a merging snapshot unless the associated
origin is also being removed.
(process_each_lv_in_vg) provides it.
Also, removing this check from _lvs_single now allows displaying hidden
LVs that are specifically named on the command line.
There's no need for foreign udev rules to touch LVM reserved devices
(snapshot, pvmove, _mlog, _mimage, _vorigin) even if they happen to
be visible. The same applies for /dev/disk content - no need to create
any content for these devices (and so no need to run any "blkid" etc.).
This also prevents setting any inotify "watch" from udev rules on such
devices that is a source of race conditions (the rules need to honor
DM_UDEV_DISABLE_OTHER_RULES_FLAG for this to work though).
1) Test that the primary mirror image cannot be removed while
the mirror set is sync'ing.
2) Test that you cannot start a second mirror up-convert while
one is already in progress.
The trouble is that if the sync/conversion finishes before the
tests occur, the tests will fail by why of success where there
should have been failure. This means the sync/conversion must
happen very quickly, but this is possible because the test
mirrors we are creating are so small.
In order to decrease the likelyhood of these test failing (or
more correctly, failing to test the right thing), I've increase
the size of the mirrors. It will still be remotely possible that
the tests will fail (by way of failing to test the right thing).
If this continues to happen, more involved mechanisms will need
to be put in place. (Perhaps these will still be created, but
this change should be a remedy until that time.)
These two old-test/regex utils are usable for testing output of regex
processing at core level. For getting them usable configure need to create
Makefile. This is currently disable by default.
Reintroduce split teardown (teardown() calls teardown_devs()) because
t-topology-support.sh only needs the teardown_devs() subset of the full
teardown() between each iteration of the topology tests -- in particular
the $TESTDIR must not get removed between each topology test iteration.
prepare_loop() must return if prepare_scsi_debug_dev() already
established $LOOP.
Also fix (and simplify) the unsafe scsi-debug device discovery in
prepare_scsi_debug_dev().
Ensure we can create devices for use in tests before running the real
tests. This may in various cases, and may involve machine configuration
rather than a failure in some specific test.
For example, if a test is run with LVM_TEST_DIR=/tmp, selinux is enabled,
and the default security context is set to "<<none>>" for /tmp, all the
tests will fail, unable to create devices, since dmsetup will fail, a
result of machpathcon() returning an error code.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
This version number change reflects the memory handling change
for string-based pv/vg/lv string based attributes.
In addition, when adding support for tags, I forgot to increase
the version number.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
Everywhere else in the API the caller can rely on lvm2app taking care of
memory allocation and free, so make the 'name' and 'uuid' properties of a
vg/lv/pv use the vg handle to allocate memory.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
We should write metadata into next position in the ring buffer while calling
vgrename and vgcfgrestore. At this code level (_vg_write_raw), we were not able
to determine if this is a rename or not. If yes, then accompanying VG structure
passed here has a new name set, not the old one.
When looking for a location where to put metadata next, we were given a NULL
value because of failed VG name comparison (in _find_vg_rlocn) between the
name in existing metadata and metadata we're just about to write.
This resets the position in the ring buffer, overwriting any existing metadata
(and also incorrectly updates the cache to "orphan" afterwards).
This patch just adds old_name item in struct volume_group that we can check and use
if necessary and detect renames at lower layers as well.
The same applies for vgcfgrestore, but here we're using a special value of
old_name, an empty string, to disable the check with existing metadata totally.
Internally, we used DM names instead of UUIDs while processing event
handlers. This caused problems while trying to vgrename a VG with active LVs
where the names are being changed and so the devices were not found then.
The patch also contains a little bit of refactoring, moving "build_dlid" code
found in dev_manager.c to "build_dm_uuid", now in lvm-string.c (so we have
build_dm_uuid and build_dm_name at one place).
lvm2app needs a link back to the vg in order to use the vg handle for
memory allocations as well as other things. This patch adds the field
to struct physical_volume, and sets pv->vg when reading a vg from disk or
extending a vg by using the helper function previously added,
add_pvl_to_vgs(). Moves and renames are handled with separate code
inside move_pv() and vgmerge(). Add pv->vg check to vg_validate().
A NULL value in pv->vg signifies membership in the orphan VG.
Note though in the case of pv_read() on a device with metadatacopies == 0,
more devices may need to be read for an authoritative answer.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
Now that we have library functions to add/delete a pv from the vg->pvs
list, call them from everywhere.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
Add a delete function to manage the vg->pvs list.
NOTE: It may be possible to do further cleanup to these add/del functions
by passing a 'pv' as input instead of 'pv_list'. The pv_list is used for
functions which do allocations (lvcreate) while other places in the code
just manage a list of 'pv' (e.g. import functions, vgextend, etc).
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
Move the increment of vg->pv_count next to the place where we add to
vg->pvs. It looks safe to do this since the only caller of import_pool_vg()
calls import_pool_pvs() immediately afterward, and there is no way
import_pool_vg() can fail (always returns 1). However, if there's a
memory allocation failure inside import_pool_pvs(), we will end up with
a different count in vg->pv_count that with the original code. In any
case, vg->pv_count should be as close to dm_list_size(&vg->pvs) as
possible, as is the case everywhere else in the code.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
The dm_list * parameter is unnecessary since we are passing in 'vg'
and the only caller of import_pool_pvs() passes '&vg->pvs' in the
dm_list * parameter. Just use vg->pvs directly in the function.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
{core|disk|mirrored}. However, under the options section they are
described as {core|disk} - even though the 'mirrored' argument is
described.
s/'{core|disk}'/'{core|disk|mirrored}'/
Fix unwanted modification of $(top_builddir)/make.tmpl.
Using dependency rules to install rules for udev.
There is minor problem, with concurent usage of builddir
and srcdir could lead to missuse of 10-dm.rules which
could be found in VPATH from different builddir.
However current solution uses intermediate target so
the generated 10-dm.rules exists only for short period of time
during make install execution.
Patch is inspired by Debian's extra patch.
- removes OWNER & GROUP make vars they are parts of INSTALL command.
- adds INSTALL_PROGRAM for executable, uses $(INSTALL)
- adds INSTALL_DATA for non-executable data, uses ($INSTALL)
- adds INSTALL_WDATA for writable non-executable data, uses ($INSTALL)
- adds configure option --enable-write_install - to support
installatin of writable files used by distribution
- replaces usage of ifeq @LIB_SUFFIX@ with $(LIB_SUFFIX)
- installs .a files from static builds without executable flag
- installs .a files to $(usrlibdir) instead of $(libdir)
- installs all static binaries to $(staticdir)
- create .so links for devel package in $(usrlibdir) instead of
$(libdir)
- makes .so and .so.LIB_VERSION files within builddir
- removes VERSIONED_SHLIB and created versioned LIB_SHARED automagicaly
- install LIB_SHARED via install_lib_shared target
- install plugins via install_lib_shared_plugin target
- prints whole 'install' command during installation instead of less
informative "Installing $(something) $(somewhere)"
- install multiple man pages with one INSTALL command
- use DISTCLEAN_TARGETS instead of creating multiple distclean targets
Usage of VPATH makes troubles when used within $(builddir).
Not only source files are being found through VPATH,
but targets as well. (make --debug=v)
Thus if user builds the code in $(srcdir) and also in some $(builddir)
he gets mangled results as some generated files (i.e. .export.sym)
are 'reused' from $(srcdir) instead of $(builddir).
This patch switches to use vpath were we could explicitly name
suffixes that should be looked via vpath - we must take care,
we do not generate files with these suffixes:
.c, .in, .po, .exported_symbols
A user specifying duplicate paths on the cmdline of vgcreate will
get a message similar to the following:
vgcreate vgtest2 /dev/loop3 /dev/loop5
Found duplicate PV jk1lXsKzwyOKlXq6bhaFFKMQQ06oPgu8: using /dev/loop5 not /dev/loop3
Found duplicate PV jk1lXsKzwyOKlXq6bhaFFKMQQ06oPgu8: using /dev/loop3 not /dev/loop5
Internal error: Duplicate PV id jk1lXs-Kzwy-OKlX-q6bh-aFFK-MQQ0-6oPgu8 detected for /dev/loop3 in vgtest2.
This is caught by vg_validate(), but it would be good to find
this condition earlier in the vgcreate code. add_pv_to_vg()
currently checks by pvname, but does not look for duplcate pvids.
This patch adds the check for duplicate pvids and results in new
error output as follows:
vgcreate vgtest2 /dev/loop3 /dev/loop5
Found duplicate PV jk1lXsKzwyOKlXq6bhaFFKMQQ06oPgu8: using /dev/loop5 not /dev/loop3
Found duplicate PV jk1lXsKzwyOKlXq6bhaFFKMQQ06oPgu8: using /dev/loop3 not /dev/loop5
Physical volume '/dev/loop5 (jk1lXs-Kzwy-OKlX-q6bh-aFFK-MQQ0-6oPgu8)' listed more than once.
Unable to add physical volume '/dev/loop5' to volume group 'vgtest2'.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
When moving parts of striped LVs, pvmove wouldn't care about leaving you with
two stripes on the same disk. Now --alloc anywhere is needed for that.
(Tried and gave up on two alternative approaches before the one committed here.)
Small refactor of main places in the code where a pv is added to a
vg into a small function which adds the pv to the list and updates
the vg counts.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
Refactor adding to the vg->pvs list and incrementing the count, which
will allow further refactoring. Should be no functional change.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
Simple refactor to mov code that updates the vg extent counts from a
single pv's counts close to the code that adds a pv to vg->pvs and
updates vg->pv_count. No functional change.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
In add_pv_to_vg(), we should only add the pv to vg->pvs after all
internal checks have passed. The check for vg->extent_count exeeding
maximum was after we added the pv to the list, so this function could
return a state of vg->pvs that did not reflect other parameters such
as vg->pv_count.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
to check for presence of this module and avoid using --frames
option for genhtml in this case.
Fix arg list for AC_PATH_PROG for lcov and genhtml.
(detecting empty LCOV and GENHTML string in Makefiles).
Patch fixes generation of coverage files for dmeventd and adds support for clvmd.
Path names are stripped, so the the html looks better.
Frames 'previews' is enabled for generated pages.
Using top_srcdir was wrong here - though we still can't easily use builddir.
Requiers using shell variables before execution of binaries build outside
of srcdir.
Because we have now strong rule for lock ordering:
- VG locks must be taken in alphabetical order
- ORPHAN locks must be the last
vgs_locked() is now not needed.
This fixes problem with orphan locking, e.g.
vgremove VG1 | vgremove VG2
lock(VG1) | lock(VG2)
lock(ORPHAN) | lock(ORPHAN) -> fail, non-blocking
https://bugzilla.redhat.com/show_bug.cgi?id=578413
(More similar places in code.)
Physical segments were still allocated from global
command context mempool.
This leads to very high memory usage when
activating large VG (vgchange).
(Memory usage was about 2G when >3000LVs).
Fix it by properly using vg->vgmem private pool,
so all the memory is released early.
New memory pool parameter is needed here for pv_split_segment
function.
Also fix the same problem in some minor allocations
(vg description, lv segment split).
In addition to previous patch, we really do not need
to search for segment which was just allocated in
split request.
Make pv_split_segment function return newly allocated
(split) segment also.
(So after this patch, there is only one user
of slow find_peg_by_pe).
The function find_peg_by_pe is incredibly inefficient
for Pvs with many segments.
In shiny future there should be binary (or interval) tree
instead of sorted linked list (volunteers?).
Anyway, for now, we can use dirty trick here to optimise this case:
- Allocations are usually applied from the beginning
of PV (we have no alloocation policy which allocates areas
"backwards")
- The only user of find_peg_by_pe is pv_split_segment()
call. In *most* cases it need to split *last* PV segment.
So if we search sorted pv segment list backwards, we
hit the requested segment immediatelly.
This patch applies this tiny change.
(and saves >30% of processing time when >3000LVs segments are on one PV!)
To discourage using this inefficient function from other code,
it is moved to pv_manip.c and used static for now:-)
vg_validate call is an adept to optimisation, it is very
ineeficient and slow.
Anyway, we should call it only before writing data to disk.
The call in lvmcache was just temporary validation,
we realy do not need to revalidate cached metadata
every time.
(Actually, I added that there just to prove that cache works
properly and forgot to remove it.)
Patch removes it from lvmcache completely, this can hit only
internal bug in export function (and this bug must
be detected in any vg_write call anyway before).
The _read_vg uses already hash for PVs to optimise
reading of large VGs and avoiding repeated PV list traversing.
Use the same aproach to speed up parsing VG with many LVs.
If dmeventd runs with -d flag, it doesn't fork into backgroud.
The command kill(getppid(), SIGTERM) attempts to kill the parent dmeventd
process, however, if there is no parent, it kills whatever process spawned
dmeventd. In case of debugging with gdb, the parent is gdb, thus
kill(getppid(), SIGTERM) kills the debugger.
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
we are running the repair manually. If we don't ignore, then dmeventd
and the manually run repair can collide. (We should still get clean
results in such a case, but it makes it harder to validate the test
results.)
Code moves initilization of stats values to _memlock_maps().
For dmeventd we need to use mlockall() - so avoid reading config value
and go with _use_mlockall code path.
Patch assumes dmeventd uses C locales!
Patch needs the call or memlock_inc_daemon() before memlock_inc()
(which is our common use case).
Some minor code cleanup patch for _un/_lock_mem_if_needed().
If the error path of _register_for_event() calls _free_thread_status()
_lib_put() call is missing.
To make thing simpler move this _lib_put() into common error path code.
As the header file <sys/mman.h> was not included in dmeventd.c
thus missed definition of MCL_CURRENT so this patch only makes
it obvious we were not locking memory here.
This patch has no functional change.
Later part of this patch set handles mlockall() via memlock_inc_daemon().
clvmd does not propagate DMEVENTD_MONITOR_IGNORE.
Update get_activation_monitoring_mode() to check if the VG that the
LV is being activated in is clustered. If so, skip it.
Any get_activation_monitoring_mode() error will cause the associated LV
(or VG) to be skipped during activation. Both vgchange_single() and
lvchange_single(), which call get_activation_monitoring_mode(), are
called by their respective process_each_..() method.
in clvmd, dmevend, man, tests.
Don't include dependency files for clow and cscope.out targets
Improve dependency tracking for dmeventd and liblvm2cmd sources.
to obtain sources. Create make.tmpl target for
simplier generation of cflow files with the help of
CFLOW_LIST, CFLOW_LIST_TARGET, CFLOW_TARGET.
Still cflow usage is not perfect.
Move daemons/ and lib/ subtargets to their Makefiles so we don't get
double cleanup error during execution of distclean target.
Instead of duplicating clean target inside distclean target,
just use it as a subtarget and avoid add duplicating code.
This check-in enables the 'mirrored' log type. It can be specified
by using the '--mirrorlog' option as follows:
#> lvcreate -m1 --mirrorlog mirrored -L 5G -n lv vg
I've also included a couple updates to the testsuite. These updates
include tests for the new log type, and some fixes to some of the
*lvconvert* tests.
clvmd's do_lock_lv() already properly controls dmeventd monitoring based
on LCK_DMEVENTD_MONITOR_MODE in lock_flags -- though one small fix was
needed for this to work: _lock_for_cluster() must treat
dmeventd_monitor_mode()'s return as a tri-state value.
Also cleanup do_lock_lv() to:
- explicitly init_dmeventd_monitor() based on LCK_DMEVENTD_MONITOR_MODE
- no longer reset init_dmeventd_monitor() to default at the end of
do_lock_lv() -- it is unnecessary
This is the next preparatory step towards better --alloc anywhere
support and is not intended to break anything that currently works so
please report any problems - segfaults, bogus data in the new debug
messages, or if the code now chooses bizarre allocation layouts.
. Add "monitoring" option to "activation" section of lvm.conf
. Have clvmd consult the lvm.conf "activation/monitoring" too.
. Introduce toollib.c:get_activation_monitoring_mode().
. Error out when both --monitor and --ignoremonitoring are provided.
. Add --monitor and --ignoremonitoring support to lvcreate. Update
lvcreate man page accordingly.
. Clarify that '--monitor' controls the start and stop of monitoring in
the {vg,lv}change man pages.
Signed-off-by: Mike Snitzer <snitzer@redhat.com>
This prevents some confusion when libudev was not found so udev_sync was disabled
automatically. Configure was successful though giving only a tiny warning.
Also, if "dmsetup udevcreatecookie" is used, never return 0x000000 as a result if
udev is not running and keep the output blank.
We need to know whether we should wait for any uevent or not when
using udev_sync. A kernel patch was posted recently that changed the
way uevents are sent on dm device resume - it is sent only if the
device has been suspended before. There's also a new DM_UEVENT_GENERATED_FLAG
in the ioctl to notify userspace whether the event was generated.
If the uevent was not generated (e.g. the situation where the device is
*not* suspended and we call a resume), we just call dm_udev_complete
explicitly from within libdevmapper itself to prevent infinite waiting
while trying to synchronise with udev processing.
Prevent lvresize from being able to resize internal LVs: mirror legs
(*_mimage_*), mirror log (*_mlog), snapshot placeholder LVs (snapshot*)
and others. Resizing these would leads to unexpected metadata and
sometimes crashes (in case of growing snapshot*).
When we pv_read() a device that has an orphan vgname, we might need to scan
the system to be sure this is true. However, if the PV has mdas, there's
no way possible for it to have an orphan vgname unless it is a true orphan.
Some areas of the code were optimized to take advantage of this fact, while
others were not (we would still do the expensive scan if a device had mdas
but had an orphan VG).
This patch unifies the code so that every place we are operating on such
a PV, we skip the expensive scan if there are mdas.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
Acked-by: Petr Rockai <prockai@redhat.com>
Acked-by: Alasdair G Kergon <agk@redhat.com>
This option should be configurable, but for now
do not set it at all.
(lvm2app is used in udisks probers and there
cac cause several nasty races when trying to update
lvmcache during rescan.)
If user try to vgcreate or vgextend non-existent VG,
these messages appears:
# vgcreate xxx /dev/xxx
Internal error: Volume Group xxx was not unlocked
Device /dev/xxx not found (or ignored by filtering).
Unable to add physical volume '/dev/xxx' to volume group 'xxx'.
Internal error: Attempt to unlock unlocked VG xxx.
(the same with existing VG and non-existing PV & vgextend)
# vgextend vg_test /dev/xxx
...
It is caused because code tries to "refresh" cache if
md filter is switched on using cache destroy.
But we can change filters and rescan even without this
machinery now, just use refresh_filters
(and reset md filter afterwards).
(Patch also discovers cache alias bug in vgsplit test,
fix it by using better filter line.)
(VDSO on 32bit is VSyscall on 64bit)
It seems it could be locked on 64bit kernels running 32bit binaries,
but it makes troubles on real 32bit machines where mlock() returns
error when trying to lock such map area. (0xffffe000)
Behavior of mlockall() seems to be similar.
This patch adds a new implementation of locking function instead
of mlockall() that may lock way too much memory (>100MB).
New function instead uses mlock() system call and selectively locks
memory areas from /proc/self/maps trying to avoid locking areas
unused during lock-ed state.
Patch also adds struct cmd_context to all memlock() calls to have
access to configuration.
For backward compatibility functionality of mlockall()
is preserved with "activation/use_mlockall" flag.
As a simple check, locking and unlocking counts the amount of memory
and compares whether values are matching.
For static builds dependency for SELinux libs is not handled by 'ar'.
Till better solution is found, for static builds STATIC_LIBS is used.
Patch updates SELinux detection to use 3rd & 4th parameter for Success/Fail.
Also removes detection of pthread from this check as we know which
version of libdevmapper we are going to link with lvm after merge.
SELinux header check moved to the SELinux test code.
Create new substituted variable PTHREAD_LIBS and link this library
only with tools/libs which really needs it - i.e. dmeventd.
Check for libpthread only for builds with clvmd or dmeventd.
Remove variable LIB_PTHREAD
Modify linking of readline library. Create new substituted varible
READLINE_LIBS - readline library is linked ONLY with tools that really use
it - i.e. lvm. (Static lvm does not use readlin).
Previous behaviour put this library into the variable LIBS and thus
linked it with all created object files of lvm project (i.e. plugins...).
READLINE detection is simplified.
Termcap library is linked in only if readline library doesn't have its own
dependency (i.e. old distributions).
The kernel's blk_stack_limits() function may flag a device as
'misaligned'. If it does the alignment_offset will be -1.
Update set_pe_align_offset() to accommodate this corner case.
- increase timeout to 30 secs (on Chrissie request)
- source both cluster and clvmd for options (like all the other cluster
init scripts)
- add clustered_vgs and _lvs commodity fns
- move rh_status* fns at the top, so they can be reused
- heavily cleanup start and stop fns from redundant code and unnecessary
loops
- improve output from different operations
- make the init script lsb compliant
- don´t force kill of the daemon, send only a TERM signal and then wait
for it to exit
- Resolves rhbz#533247
lvm2 devices have always UUID set even if imported from lvm1 metadata.
Patch removes name argument from dev_manager_info call and converts
all activation related calls to use query by UUID.
Also it simplifies mknode call (which is the only user on mknodes parameter).
Fix add/remove tag function headers.
Fix a lot of little problems with doxygen comments.
Clarify the basic objects and their handles, and place functions with their
appropriate handles/objects.
All this cleanup moves automatic documentation of lvm2app much closer to being
useful as official documentation. In the future I will add some examples
and plan to build the examples as part of the unit tests.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
Add lvm2app functions to manage LV tags.
For lvm_lv_get_tags(), we return a list of tags, similar to other
functions that return lists. An empty list is returned if there
are no tags. NULL is returned if there is a problem obtaining
the list of tags.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
Add lvm2app functions to manage VG tags.
For lvm_vg_get_tags(), we return a list of tags, similar to other
functions that return lists. An empty list is returned if there
are no VG tags. NULL is returned if there is a problem obtaining
the list of tags.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
Add a supporting function to copy a list of internal tags to lvm2app list.
We need to put this here because of the lvm_str_list_t type which we export
in lvm2app.h. If we didn't export this type, we could put this in the
internal library and use struct str_list.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
We need to allocate memory for the tag and copy the tag value before we
add it to the list of tags. We could put this inside lvm2app since the
tools keep their memory around until vg_write/vg_commit is called, but
we put it inside the internal library to minimize code in lvm2app.
We need to copy the tag passed in by the caller to ensure the lifetime of
the memory until the {vg|lv} handle is released.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
Similar refactoring to vgchange - pull out common parts and put into
library function for reuse. Should be no functional change.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
Pull out common code to be called from tools as well as lvm2app.
Leave archive() at tool level so we can use from vgcreate
as well as vgchange. Should be no functional change.
- add stack macro in vgchange
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
Add a merging snapshot to the deptree, using the "error" target, rather
than avoid adding it entirely. This allows proper cleanup of the -cow
device without having to rename the -cow to use the origin's name as a
prefix.
Move the preloading of the origin LV, after a merge, from
lv_remove_single() to vg_remove_snapshot(). Having vg_remove_snapshot()
preload the origin allows the -cow device to be released so that it can
be removed via deactivate_lv(). lv_remove_single()'s deactivate_lv()
reliably removes the -cow device because the associated snapshot LV,
that is to be removed when a snapshot-merge completes, is always added
to the deptree (and kernel -- via "error" target).
Now when the snapshot LV is removed both the -cow and -real devices
get removed using uuid rather than device name. This paves the way
for us to switch over to info-by-uuid queries.
Signed-off-by: Mike Snitzer <snitzer@redhat.com>
There's a tiny period of time when the _mimage device is visible during
downconversion from mirror to linear. Since it is visible, we need to
create the symlinks, otherwise warning messages will be issued about udev
not creating those symlinks. We have to rely on udev flags completely.
- add DM_UDEV_DISABLE_LIBRARY_FALLBACK udev flag to rely on udev only
- export dm_udev_create_cookie function to create new cookies on demand
- add --udevcookie, udevcreatecookie and udevreleasecookie for dmsetup
(to support "udev transactions" where one cookie value can be used for
several dmsetup calls)
- don't use DM_UDEV_DISABLE_CHECKING env. var. anymore and set the state
automatically (based on udev and libdevmapper dev path comparison)
Internally we store sizes in sectors, but lvm2app exports sizes
in bytes. We could get fancier and allow units configuration but
this fix should do for now.
Fixes rhbz561422.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
We unfortunately don't yet _know_, in dev_manager_snapshot_percent(), if
a snapshot-merge target is active (activation is deferred if dev is
open); so we can't short-circuit origin devices based purely on existing
LVM LV attributes.
Set 'fail_if_percent_unsupported' in dev_manager_snapshot_percent() for
a merging origin LV, otherwise passing unsupported LV types to _percent
will lead to a default successful return with percent_range as
PERCENT_100.
For a merging origin, PERCENT_100 will result in a polldaemon that runs
infinitely (because completion is PERCENT_0).
When activating a merging origin it is valid, and expected, to not have
a node in the deptree for both the origin and its merging snapshot. The
_cached_info() caller is only concerned with whether a device is open.
If there isn't a node in the tree the associated device is definitely
not open.
This change was deferred to help ease the review of previous refactoring
related to using process_each_lv() for lvconvert's merge support. Not
that doing so _really_ helped but...
Signed-off-by: Mike Snitzer <snitzer@redhat.com>
Switch lvconvert's --merge code over to using process_each_lv(). Doing
so adds support for a single 'lvconvert --merge' to start merging
multiple LVs (which includes @tag expansion).
Add 'lvconvert --merge @tag' testing to test/t-snapshot-merge.sh
Adjust man/lvconvert.8.in to reflect these expanded capabilities.
The lvconvert.c implementation requires rereading the VG each iteration
of process_each_lv(). Otherwise a stale VG instance associated with
the LV passed to lvconvert_single_merge() would result in stale VG
metadata being written back out to disk. This overwrote new metadata
that was written when a previous snapshot LV finished merging (via
lvconvert_poll). This is only an issue when merging multiple LVs that
share the same VG (a single VG is typical for most LVM configurations on
system disks).
In the end this new support is very useful for performing a "system
rollback" that requires multiple snapshot LVs be merged to their
respective origin LV.
The yum-utils 'fs-snapshot' plugin tags all snapshot LVs that it creates
with a common 'snapshot_tag' that is unique to the yum transaction.
Rolling back a yum transaction, that created LVM snapshots with the tag
'yum_20100129133223', is as simple as:
lvconvert --merge @yum_20100129133223
Signed-off-by: Mike Snitzer <snitzer@redhat.com>
refactoring.
Document the need to cleanup the "name" args passed around polldaemon,
lvconvert and pvmove. It is quite a mess.
Annotate the unused nature of the existing poll_fns->get_copy_vg
methods' 'uuid' arg.
depending on if the mirror has a 'core' or 'disk' log. When there
is a disk log, the new leg is added by stacking a new mirror on
top of the old (one leg is the old mirror and the other leg is the newly
added device). When the log is a 'core' log, the new leg is simply added
to the existing mirror and all the devices are re-synced.
The logic that handles collapsing the stacked 'disk' log mirror was
having the effect of causing 'core' logged mirrors to begin resync'ing
for a second time. I have used the 'CONVERTING' flag to indicate that
a mirror is converting by way of stacking. This is no longer set for
up-converting core logs. The final 'collapse' logic can safely be skipped
for 'core' log mirrors - getting rid of the second resync.
Signed-off-by: Jonathan Brassow <jbrassow@redhat.com>
where we should not expose internal VG names/uuids (the ones with "#" prefix )through the
interface. Otherwise, we could end up with library users opening internal VGs which will
initiate locking mechanism that won't be cleaned up properly.
"#orphans_{lvm1, lvm2, pool}" names are treated in a special way, they are truncated first
to "orphans" and this is used as a part of the lock name then (e.g. while calling lvm_vg_open()).
When library user calls lvm_vg_close(), the original name "orphans_{lvm1, lvm2, pool}"
is used directly and therefore no unlock occurs.
We should exclude internal VG names and uuids in the lists provided by lvmcache:
lvmcache_get_vgids() and lvmcache_get_vgnames().
Allow the number of logical extents to be expressed (for a snapshot) as
a percentage of the total space in the Origin Logical Volume with the
suffix %ORIGIN.
Update the relevant man pages accordingly. Eliminate inconsistencies
between the man pages and tools/commands.h
Signed-off-by: Mike Snitzer <snitzer@redhat.com>
*_safe. This had the effect of segfaulting the log daemon when
converting a mirror from one log type to another.
Signed-off-by: Jonathan Brassow <jbrassow@redhat.com>
When activation of pvmove mirror fails on cluster, some nodes
still possibly succeeded in activation.
- Explicitly deactivate that mirror to be sure
- properly pair suspend/resume calls to not cause memory lock problems in clvmd
Code cannot simply call _finish_pvmove on cluster in this situation, because
changed LVs are suspended twice (causing memory inbalance) and also temporary
mirror is activated when it is not expected (and we know that it failed already).
Patch prepares special function which remove temporary mirror references from
metadata and then resumes changed LVs.
In dev_manager_info 0 means error and 1 info is returned,
not that device exists (that value is part of info struct).
Fix query by uuid only (no name) which returns 0 when device
does not exist.
Support "wait before testing" using '+' in pvmove and lvconvert
interval. Doing so overrides the new default of sleeping after checking
the LV's progress.
Sleeping before checking progress can lead to extraneous polldaemons
being left running. These polldaemons would have otherwise exited had
they checked before sleeping. Checking progress before sleeping helps
workaround the subtly unreliable nature of "finished" state checking
in _percent_run.
Update test/t-mirror-names.sh to use '+' when providing its lvconvert
interval.
more descriptive message if locking fails instead of
"Locking type -1 initialisation failed."
Use read-only locking instead of misleading ignorelocking option
in message.
All this seems to do is provide a memory leak so remove it.
The only caller of _alloc_pv() later explicitly sets
pv->vg_name = fmt->orphan_vg_name so clearly this allocation
should be removed. I also saw no where in the code where
strncpy was used to assign pv->vg_name - only direct assignments
and strdup's.
DSO is currently not dl_close-ing pluing during it is unregister handling,
so clear structure and related counter, so there are no memory problems.
Futher fixes are needed.
merge completes. This narrows the scope of this "hack" (which still
needs a proper fix within the deptree).
This stops dmeventd from trying to access snapshot devices that were
already removed.
LVM2 test (rather than using the traditional loop device).
prepare_scsi_debug_dev currently assumes exclussive access to the
scsi_debug module. Any script that tries to use prepare_scsi_debug_dev
when scsi_debug is unavailable or already loaded into the kernel will be
skipped.
t-topology-support.sh shows how prepare_scsi_debug_dev function can be
used repeatedly (within a script) to test LVM2 ontop of a ramdisk-based
SCSI device w/ arbitrary scsi_debug features.
isn't already available (in /proc/mdstat).
switch to requiring 2.6.33 for the alignment_offset tests; 2.6.{31,32}
alignment_offset values aren't reliable. 2.6.33 _should_ have mkp's
alignment_offset fixes but so far it doesn't (as of 2.6.33-rc4).
For mirror repair (and similar tasks) it can happen that full
device rescan is issued from clvmd.
Because code can be in the middle of repair (calling suspend)
clvmd should never try to scan suspended devices
(otherwise it causes deadlock).
Also code must not change ignore_suspended_device flag when
doing refresh_filters (called from lvmcache scan code).
'const'. Be consistent with its use (and dev_manager_snapshot_percent()).
Pass 'lv' from dev_manager_snapshot_percent() to _percent() to
_percent_run(). _percent_run() always dereferenced 'lv' (when
initializing segh) even though it may have been NULL (as was the case
until now for dev_manager_snapshot_percent()).
If a "snapshot-origin" LV (snapshot-merge whose merge was deferred
becuase it was open) was passed to _percent_run() it would always return
100%.
Update _percent_run() to NOT return PERCENT_100 et. al. if
->target_percent() wasn't ever called and supplied 'lv' is a merging
origin. A default return of 100% does not work for snapshot-merge.
Also tweak a related lvconvert log_error() to include "Aborting merge."
bitmap tracking was switched from the e2fsprogs implementation to
the device-mapper implementation (dm_bitset_t). The latter has a
leading uin32_t field designed to hold the number of bits that are
being tracked. The code was not properly handling this change in
all places. Specifically, when getting the bitmap to/from disk.
Endian adjustments will likely need to be made on the accounting
field as well, since bitmaps are passed between machines on
start-up.
Signed-off-by: Jonathan Brassow <jbrassow@redhat.com>
that were necessary to be passed on to userspace.
The cluster mirror table (log portion only) used to look like this:
clustered-disk <parm_count> <disk> <region_size> <uuid> \
[[no]sync] [block_on_error]
Now it looks like this:
userspace <parm_count> <uuid> clustered-disk <disk> <region_size> \
[[no]sync]
So, there is one extra argument in the latter case - this was
unaccounted for.
Signed-off-by: Jonathan Brassow <jbrassow@redhat.com>
Eliminate 'merging_snapshot' from 'struct logical_volume' and just use
'snapshot' for origin lv's reference to the merging snapshot; also set
MERGING in the origin lv's status.
If either the origin or snapshot that is to be merged is open the merge
will not start; only the merge metadata will be written. The merge will
start on the next activation of the origin (or via lvchange --refresh)
IFF both the origin and snapshot are closed.
Merge on activate is particularly important if we want to merge over a
mounted filesystem that cannot be unmounted (until next boot) --- for
example root.
snapshots are suspended, new origin is created, snapshots are resumed, new
origin is resumed. So it allocates memory while suspended.
To fix it, move vg_commit after suspend_lv, so that the suspend code will
treat it as precommitted vg and will preload new origin prior to suspend.
NOTE: agk doesn't like this "hack"; need to revisit and fix
This is useful for when the snapshot is still active and merging hasn't
started yet; it shows a merge is pending. Once merging starts the
merging snapshot will be hidden but can still be displayed with 'lvs -a'
Report snapshot origin with merging snapshot as 'O' instead of 'o':
Before merge starts this shows that a merge is pending. While merging
the snapshot will be hidden, 'O' enables a user to see that there is a
snapshot merging.
"snapshot-merge" target based on whether the LV is a merging snapshot.
When activating a snapshot-merge target do not attempt to monitor the
LV for events; the polldaemon will monitor the snapshot as it is
merged.
Allow "snapshot-merge" target's usage to be parsed via standard
"snapshot" methods.
NOTE: follow on fixes to the _percent_run change are still needed
Introduces new libdevmapper function dm_tree_node_add_snapshot_merge_target
Verifies that the kernel (dm-snapshot) provides the 'snapshot-merge'
target.
Activate origin LV as snapshot-merge target. Using snapshot-origin
target would be pointless because the origin contains volatile data
while a merge is in progress.
Because snapshot-merge target is activated in place of the
snapshot-origin target it must be resumed after all other snapshots
(just like snapshot-origin does) --- otherwise small window for data
corruption would exist.
Ideally the merging snapshot would not be activated at all but if it is
to be activated (because snapshot was already active) it _must_ be done
after the snapshot-merge. This insures that DM's snapshot-merge target
will perform exception handover in the proper order (new->resume before
old->resume). DM's snapshot-merge does support handover if the reverse
sequence is used (old->resume before new->resume) but DM will fail to
resume the old snapshot; leaving it suspended.
To insure the proper activation sequence dm_tree_activate_children() was
updated to accommodate an additional 'activation_priority' level. All
regular snapshots are 0, snapshot-merge is 1, and merging snapshot is 2.
Make 'merging_snapshot' pointer that points from the origin to the
segment that represents the merging snapshot.
Import/export 'merging_store' metadata.
Do not allow creating snapshots while another snapshot is merging.
Snapshot created in this state would certainly contain invalid data.
NOTE: patches at the end of this series will remove 'merging_snapshot'
and will introduce helpful wrappers and cleanups.
This spurious 'break' has been here since this code was first committed
in June 2005 and stopped the algorithm behaving as described in the
comment above it and rendered the variable 'already_found_one' useless.
1. Found bug in 'redundant log' implementation that caused
problems when converting a linear that spanned multiple
devices to a mirror (wasn't checking for NULL value of
provided parameter in _alloc_parallel_area)
2. Testsuite was failing to perform tests when 'not' modifier
was used. This allowed a couple issues to slip through.
Added a 'not_sh' modifier that negates tests performed by
functions defined in the shell source file.
3. Was initializing a variable to far down, which cause
previously set value to be overridden. (This was the
result of the collision of the "redundant log" and
lvconvert fix patches.)
successfully created it must _exit() once it completes.
Update _become_daemon() to differentiate between a failed fork() and a
successful fork().
Added lvm_return_code() to lvmcmdline.[ch]
Upon successful fork(), _become_daemon() must assert that the locks that
are currently held belong to the parent, not the child. All of the
child's internal state saying 'this process holds a lock' has to be
reset.
A proper lvmcache_locking_reset() should follow later.
date: 2010/01/07 20:42:55; author: jbrassow; state: Exp; lines: +11 -0
The patch fixes some lvconvert issues (WRT mirror <-> mirror).
1) 'exisiting_mirrors' and 'lp->mirrors' where taken to be in 'n-1'
notation (i.e a 2-way mirror is '1' and a linear is '0'), but the
variables were in 'n' notation.
2) After adding the redundant mirror log support, I was calculating
log_count by looking at the mirror log LV, but didn't take into
account the fact that there could be no mirror log!
Signed-off-by: Jonathan Brassow <jbrassow@redhat.com>
Sometimes it is really needed to switch off udev checking and the warnings we show when
we detect that udev has not done its job right - the messages like "Udev should have done
this and that. Falling back to direct node creation/removal. " etc.
This would be especially handy while setting DM_DEV_DIR env var that could be set to a
different location than standard /dev (udev can't create nodes/symlinks out of that one
directory that is configured into udevd). The exact same situation happens while we're
running our tests.
It is pretty much the same as reducing the number of
mirror legs, but we just don't delete them afterwards.
The following command line interface is enforced:
prompt> lvconvert --splitmirror <n> -n <name> <VG>/<LV>
where 'n' is the number of images to split off, and
where 'name' is the name of the newly split off logical volume.
If more than one leg is split off, a new mirror will be the
result. The newly split off mirror will have a 'core' log.
Example:
[root@bp-01 LVM2]# !lvs
lvs -a -o name,copy_percent,devices
LV Copy% Devices
lv 100.00 lv_mimage_0(0),lv_mimage_1(0),lv_mimage_2(0),lv_mimage_3(0)
[lv_mimage_0] /dev/sdb1(0)
[lv_mimage_1] /dev/sdc1(0)
[lv_mimage_2] /dev/sdd1(0)
[lv_mimage_3] /dev/sde1(0)
[lv_mlog] /dev/sdi1(0)
[root@bp-01 LVM2]# lvconvert --splitmirrors 2 --name split vg/lv /dev/sd[ce]1
Logical volume lv converted.
[root@bp-01 LVM2]# !lvs
lvs -a -o name,copy_percent,devices
LV Copy% Devices
lv 100.00 lv_mimage_0(0),lv_mimage_2(0)
[lv_mimage_0] /dev/sdb1(0)
[lv_mimage_2] /dev/sdd1(0)
[lv_mlog] /dev/sdi1(0)
split 100.00 split_mimage_0(0),split_mimage_1(0)
[split_mimage_0] /dev/sde1(0)
[split_mimage_1] /dev/sdc1(0)
It can be seen that '--splitmirror <n>' is exactly the same
as '--mirrors -<n>' (note the minus sign), except there is the
additional notion to keep the image being detached from the
mirror instead of just throwing it away.
Signed-off-by: Jonathan Brassow <jbrassow@redhat.com>
Made .update_metadata optional in 'struct poll_functions' definitions;
eliminated _update_lvconvert_mirror() stub.
Tweak a mirror-specific error message in the generic polldaemon code.
Signed-off-by: Mike Snitzer <snitzer@redhat.com>
The default log option for a mirror is 'disk'. If the log
type is not explicitly stated on the command line when
converting from an X-way mirror to a Y-way mirror, 'disk'
is chosen. So, if you have a 'core' log mirror and you
convert, your result will contain a 'disk' log.
This patch remembers what the old log type was. If the
user is merely trying to switch the number of mirror
images, the log type is now kept the same.
There is one historical behaviour I left in place...
If you have a 2-way, core-log mirror and you use lvconvert to
specify you want a 2-way mirror - without specifying the
log type - you will get a 2-way, disk-log mirror.
Signed-off-by: Jonathan Brassow <jbrassow@redhat.com>
Informal-IRC-ACK-by: agk
The logic was that lvconvert repair volumes, marking
PV as MISSING and following vgreduce --removemissing
removes these missing devices.
Previously dmeventd mirror DSO removed all LV and PV
from VG by simply relying on
vgreduce --removemissing --force.
Now, there are two subsequent calls:
lvconvert --repair --use-policies
vgreduce --removemissing
So the VG is locked twice, opening space for all races
between other running lvm processes. If the PV reappears
with old metadata on it (so the winner performs autorepair,
if locking VG for update) the situation is even worse.
Patch simply adds removemissing PV functionality into
lvconcert BUT ONLY if running with --repair and --use-policies
and removing only these empty missing PVs which are
involved in repair.
(This combination is expected to run only from dmeventd.)
Version >= 1.8.0 of the DM snapshot target appends metadata sectors used
to a snapshot's status. This patch allows LVM2 to accurately determine
if the snapshot store is empty. Knowing when a snapshot store is empty
is important in the context of snapshot-merge (means merge is complete).
Also update LVM2 to be aware of the possibility for "Merge failed" in
the snapshot-merge target's status.
Signed-off-by: Mike Snitzer <snitzer@redhat.com>
dm_tree_activate_children() callers.
Otherwise resume_lv and its variants can fail silently.
Catching these failures is especially important now that dm targets like
crypt and snapshot-merge can fail in .preresume
Signed-off-by: Mike Snitzer <snitzer@redhat.com>
the background polldaemon is allowed to start. It can be used
standalone or in conjunction with --refresh or --available y.
Control over when the background polldaemon starts will be particularly
important for snapshot-merge of a root filesystem.
Dracut will be updated to activate all LVs with: --poll n
The lvm2-monitor initscript will start polling with: --poll y
NOTE: Because we currently have no way of knowing if a background
polldaemon is active for a given LV the following limitations exist and
have been deemed acceptable:
1) it is not possible to stop an active polldaemon; so the lvm2-monitor
initscript doesn't stop running polldaemon(s)
2) redundant polldaemon instances will be started for all specified LVs
if vgchange or lvchange are repeatedly used with '--poll y'
Signed-off-by: Mike Snitzer <snitzer@redhat.com>
This patch tries to correctly track changes in lvmcache related to commit/revert.
For vg_commit: if there is cached precommitted metadata, after successfull commit
these metadata must be tracked as committed.
For vg_revert: remote nodes must drop precommitted metadata and its flag in lvmcache.
(N.B. Patch do not touch LV locks here in any way.)
All this machinery is needed to properly solve remote node cache invalidaton which
cause several problems recently observed.
Lock mode is int masked by LCK_TYPE_MASK, always.
Patch also remove uneccessary masking lock flag on sender side,
if masking is needed, it is don on client side already.
- Add drop_precommitted flag to force drop precommitted metadata
- add lvmcache_commit_metadata() which upgrades precommitted metadata in cache
No functional change in this patch - just preparation for following change.
And decode flags in humar readable form in client.
And clean some trailing whitespaces.
No functional change in this patch (only debugging messages changed).
The use_precommitted flag indicates, that we want to use precommitted metadata
(used in suspend call to preload table with precommitted data).
But if there are no such data, committed metadata are read but the cache
still contains that precommitted flag.
(The problem is that later possible drop_metadata call will not invalidate
device in cache.)
The wrong precommitted state is stored in on remote nodes during normal
suspend/resume cycle _without_ vg_write/commit.
Use the PRECOMMITTED status flag here instead (which is always set if using
precommited metadata here).
If renaming snapshot with virtual origin, the origin is renamed too.
But the code must resume LVs in reverse order to properly
pair memlock (in cluster locking).
(The resume of snapshot resumes origin too and later resume
is ignored otherwise.)
(So the tests can run under cluster locking and do not require
cluster mirror or snapshots.)
Add vgscan before block device readahead change
(flush long running process - clvmd - dev cache.)
When PV device reappears with old metadata, it is
always updated to new version byt atutomatic metadata
repair.
Remove missing flag if device is empty.
If device contains allocated extents, issue warning that
user must remove volumes and re-add this PV before
manipulating with this volume.
This partially solves bug 547842 when one PV (log) is failed,
dmeventd removes that device and later this device reappears and
is wrongly added into VG marked missing.
The memlock_inc() fix is wrong, memlock count is not
propagated to long living process (clvmd) and just
it underflow there.
Also suspend is needed to pre-load precommited metadata
on other nodes (remapping to error taget in this case).
With explicit suspend we generate lock request and code
can update memlock count.
(Infinitely "locked" memory caused that fs_unlock() was not
called properly and on cluster nodes remains
old links in /dev/mapper for not active devices.)
(N.B. failing of suspend call here is not handled as fatal
error - the LV is going to be removed later anyway.)
The new recovery code first tries to repair LV and then removes failed PV
from VG. It means that during operation there can be VG with PV missing,
and vg_read code handles it like not consistent VG.
We already allows returning "inconsistent" commited metadata,
for mirror repair we need this for precommited too.
(The suspend call prepares precommited metadata to inactive table on
other cluster nodes.)
"Inconsistent" here means - correct metadata, just with some metadata areas
not found (obviously on missing or failed PVs).
The LV locks make sense only for clustered LVs.
Properly check cluster flag and never issue cluster lock here.
There are several places in code, where it is already checked, this
patch add this check to all needed calls.
In previous code the lock behaviour was inconsistent,
for example, the pre/post callback can take lock even for local volume,
but deactivate call do not released this lock and it remains held forever.
The local LV lock request now just let run the underlying activation code
on local node, the same process like in local locking.
(Again, this is important for new mirror repair calls, here for local
mirrors but with cluster locking enabled.)
This is unnoticed regression from commit 31672ff60e
The pre/post callback need to convert lock always, local node
is going to modify metadata in this case, it it fails conversion,
the call is ignored.
Also it fixes bug when the lock is not yet held, we cannot set LKF_CONVERT
in this case, it will fail because this lock do not exist.
Note that the automatic conversion is still disabled in activate
call, so the original fix (reactivation of exlusive LV) should
be still in place.
(Code already not fail if unlocking not locked resource.)
This is needed in pre/post lock_lv call, where we can
request the same lock on local node becuase of suspend call.
- do_command and lock_vg expect flags (no change here)
Bug fixes:
- lock_vg should check for NONBLOCK on lock_cmd, flags have this bit masked-out
- do_pre/post_command expect do not mask flag at all, this causes that
the code inside is never run! (see following patches, these functions
expect plain command without flags)
Patch should not cause any problems, only real change is
removing LCK_LOCAL bit from lock type flag, it is never used there.
(LCK_LOCAL is part arg[1] bits anyway.)
If there is problem deactivate LV and
_init_mirror_log is called with remove_on_failure = 1,
remove the newly created log LV from metadata.
(This can happen if there is active device with the same name
but different UUID.)
The main reason for this "workaround" patch is to
- do not keep _mlog volume in metadata, so user can repeat the action
- print better error message describing the real problem
# lvcreate -m 2 -n lv1 -l 1 --nosync vg_bar
WARNING: New mirror won't be synchronised. Don't read what you didn't write!
/dev/vg_bar/lv1_mlog: not found: device not cleared
Aborting. Failed to wipe mirror log.
Error locking on node bar-01: Input/output error
Unable to deactivate mirror log LV. Manual intervention required.
Failed to create mirror log.
# lvcreate -m 2 -n lv1 -l 1 --nosync vg_bar
WARNING: New mirror won't be synchronised. Don't read what you didn't write!
Aborting. Unable to deactivate mirror log.
Failed to initialise mirror log.
I see "Deactivated" message when I activate and "Activated" message when
I deactivate. The code uses "activate" as boolean but it can be any one
of the enum values from CHANGE_AY, CHANGE_AN, CHANGE_AE, etc.
Signed-off-by: Malahal Naineni <malahal@us.ibm.com>
Acked-by: Dave Wysochanski <dwysocha@redhat.com>
There's a new change udev event generated since kernel 2.6.32 that
notifies userspace about a change in read-only attribute for block
devices (with DISK_RO=1 environment variable set).
We need to detect this and disable the rule application so the
meaning of this change event is not interchanged with the regular
change event used while resuming/renaming DM devices.
If there's anybody awaiting this notification in foreign rules,
he can still check for this env var and do the appropriate actions
separately.
At this point they probably do not matter but going forward they
may - depends on future patches for replicator, etc. I think
these probably got missed because they were 'flags' so I changed
the name to 'status' to be consistent. So the on-disk
things 'flags' and the in structure 'status' (bits).
NOTE: WHATS_NEW already has entry for this in current release.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
Acked-by: Mike Snitzer <snitzer@redhat.com>
pvmove suspends all moved LVs + pvmoveX mirrored LV itself.
This suspends even underlying pvmoveX and following explicit
suspend call is just noop.
But in resume the pvmoveX volume is no longer underlying
device for moved LVs, so it performs full resume with memlock
decrease.
Code must call memlock_inc() if suspend is requested, volume
is already suspended and error is not requested.
These are no longer used by anyone. The dm_list defines are all in
libdevmapper.h and libdm/datastruct/list.c contains any function definitions.
There is some code in "old-tests" that still use this but this code is not
being maintained.
Thanks to Zdenek for spotting this.
The physical_volume, volume_group, logical_volume and lv_segment
structures' 'status' member is now uint64_t.
The alignment of these structures was also audited to remove holes. The
movement of some members in 'volume_group' and 'lv_segment' eliminates
holes. The 'physical_volume' structure still has one 4-byte hole after
'pe_size'; the other structures no longer have any holes. Each
structures' size has not changed.
If the vg_read() returned error, no lock was taken,
so always call vg_release().
Otherwise this can happen because of missing FAILED_*:
# vgchange -a y x --ignorelockingfailure
Volume group "x" not found
Internal error: Attempt to unlock unlocked VG x
The sysfs filter initialise hash of available devices using
scan of /sys/block. We need to refresh even this hash
when performing full scan otherwise the newly appeared
device could be rejected, because there is no entry
in sysfs filter.
This easily could happen when attaching new device
to cluster node. (Only force refresh of context
in clvmd -R works here now).
Unfortunately consequences of this are much worse,
missing device part on that node is replaced with missing segment
(even when no partial arg is selected) and this directly
lead to data corruption.
See https://bugzilla.redhat.com/show_bug.cgi?id=538515
Simply fix it by refreshing device filters in lvmcache
before performing the full device scan.
(on one node a storage connection failed):
# vgchange -a y vg_bar ; echo $?
Error locking on node bar-02: Refusing activation of partial LV lv1. Use --partial to override.
1 logical volume(s) in volume group "vg_bar" now active
0
So activation fails on one node, error is correctly printed but
status code is wrong.
This patch fixes the top level (vgchange) to return proper code
(and print # of activated LVs).
(lvchange returns error properly here.)
(This affects only cluster locking because only cluster
locking module set LCK_PRE_MEMLOCK.)
With currect code you get
# vgchange -a n
Internal error: _memlock_count has dropped below 0.
when using cluster locking.
It is caused by _unlock_memory calls here
if ((flags & (LCK_SCOPE_MASK | LCK_TYPE_MASK)) == LCK_LV_RESUME)
memlock_dec();
Unfortunately it is also (wrongly) called in immediate unlock
(when LCK_HOLD is not set) from lock_vol
(LCK_UNLOCK is misinterpreted as LCK_LV_RESUME).
Avoid this by comparing original flags and provide memlock
code type of operation (suspend/resume).
All hidden (not visible) volumes should be activated through
other visible volumes.
(There are already exceptions like snapshot, mirror log and image,
which should be cleaned one day...)
This solves problems for future types of hidden volumes,
which can have special meaning and must not be activated implicitly
(e.g. key store volume).
This provides better support for environments where udev rules are installed
but udev_sync is not compiled in (however, using udev_sync is highly
recommended). It also provides consistent and expected functionality even
when '--noudevsync' option is used.
There is still requirement for kernel >= 2.6.31 for the flags to work though
(it uses DM cookies to pass the flags into the kernel and set them in udev
event environment that we can read in udev rules).
'last_rule' option has been removed from udev (version >= 147).
From now on, we require foreign rules to check and honor
ENV{DM_UDEV_DISABLE_OTHER_RULES_FLAG} instead. Foreign
rules should be skipped totally when this flag is set.
- fix missing unlocking of VG
lvcreate -l 100%PVS -n lv1 vg_test
Please specify physical volume(s) with %PVS
Internal error: Volume Group vg_test was not unlocked
- if no PVS specified, use all available
Fix segfault if %PVS in lvresize without PVs list.
indented metadata lines.
Macro outnl() is using exported out_newline() instead of direct
call f->fn(), that required the visibility of the internal
struct formatter.
Rename fill_default_pvcreate_params to pvcreate_params_set_defaults.
Rename pvcreate_validate_restore_params to pvcreate_restore_params_validate.
Rename pvcreate_validate_params to pvcreate_params_validate.
- add copyright notice for 10-dm.rules.in,
- set DM_UDEV_DISABLE_{DISK, OTHER}_RULES_FLAG in 11-dm-lvm.rules directly
for inappropriate and non-top-level subdevices in case we use older kernels
where DM_COOKIE is not used (and therefore there are no flags passed from
the LVM process itself). This applies for older kernels (version < 2.6.31),
- remove unnecessary filters in 95-dm-notify.rules - the DM_COOKIE env var
itself is set for change/remove udev events and for DM devices only so
there's no need to double-check this.
Similar to other vg_set_* functions, we create a vg_set_clustered() function
which does a few checks and sets a flag. This is where we check for
any limitations of clusters.
The DRBD uses underlying device so code should prefer top
device if duplicate is found.
Patch also introduce
dev_subsystem_part_major and dev_subsytem_name
functions to easily handle all these replication susbystems
and not hardcode md_major call.
See https://bugzilla.redhat.com/show_bug.cgi?id=530881
for full problem description.
Option --all is only partially documented currently, so document in all
commands. Also make {pv|vg|lv}{display|s} man pages consistent with help
output. Remove ununsed 'disk_ARG' parameter. Leave --trustcache out of
the man page output. Update --units argument to show all possible units.
- we have these levels when the udev rules are processed:
10-dm.rules --> [11-dm-<subsystem>.rules] --> [12-dm-permissions.rules] -->
13-dm-disk.rules --> [...all the other foreign rules...] --> 95-dm-notify.rules
- each level can be disabled now by
DM_UDEV_DISABLE_{DM, SUBSYSTEM, DISK, OTHER}_RULES_FLAG
- add DM_UDEV_DISABLE_DM_RULES_FLAG to disable 10-dm.rules
- add DM_UDEV_DISABLE_OTHER_RULES_FLAG to disable all the other (non-dm) rules.
We cutoff these rules by using the 'last_rule', so this one should really be
used with great care and in well-founded situations. We use this for lvm's
hidden and layer devices now.
- add a parameter for add_dev_node, rm_dev_node and rename_dev_node so it's
possible to switch on/off udev checks
- use DM_UDEV_DISABLE_DM_RULES_FLAG and DM_UDEV_DISABLE_SUBSYSTEM_RULES_FLAG
if there's no cookie set and we have resume, remove and rename ioctl.
This could happen when someone uses the libdevmapper that is compiled with
udev_sync but the software does not make use of it. This way we can switch
off the rules and fallback to libdevmapper node creation so there's no
udev/libdevmapper race.
- remove default permissions set in 95-dm-notify.rules (and add a hint in 12-dm-permissions.rules to set it by the user directly)
- add multipath DM_ACTION=="PATH_FAILED" filter
- remove unnecessary filters in the headers of the rules (we can simply use DM_UDEV_RULES_VSN instead)
- fix symlink priorities in /dev/disk/ (snapshot volumes have low priority for FS UUID symlinks so it will not overwrite symlinks for the origin)
[root@xxxx-01 ~]# lvconvert -m 1 --corelog VG/cmirror
Unable to convert the log of inactive cluster mirror cmirror
I've tried to clean-up the message a little more, so the name
of the mirror stands out more while preserving the sense that
it's not a problem with the specific device, but the fact that
it is inactive that is causing the problem.
New msg:
Unable to convert the log of an inactive cluster mirror, cmirror
Per discussion on lvm-devel mailing list and part of debian patch set,
don't set defaults for owner and group, since nobody seems to use them, and
still allow override.
Going forward, we would like to allow users to specify the total
number of metadatacopies in a VG rather than on a per-PV basis. In
order to facilitate that, introduce --pvmetadatacopes to replace
--metadatacopies everywhere. We still allow --metadatacopies for
pv commands, but require --pvmetadatacopies for vg commands.
Eventually we will introduce --vgmetadatacopies. Once we do that,
we should either deprecate --metadatacopies or make it a synonym based
on the command (pvmetadatacopies for pv commands, and vgmetadatacopies
for vg commands). The latter option would likely just require a simple
'strncpy' check against cmd->command->name to qualify the merge_synonym
call.
Update nightly tests to cover the pvmetadatacopies synonym.
Note that this patch is the result of an eariler review comment for
the implicit pvcreate patches. Should apply cleanly on top of the
implicit pvcreate patches (I applied after patch 10/10 in that series).
NOTE: This patch will require --pvmetadatacopies for vgconvert as
--metadatacopies is no longer accepted.
Going forward, we would like to allow users to specify the total
number of metadatacopies in a VG rather than on a per-PV basis. In
order to facilitate that, introduce --pvmetadatacopes to replace
--metadatacopies everywhere. We still allow --metadatacopies for
pv commands, but require --pvmetadatacopies for vg commands.
Eventually we will introduce --vgmetadatacopies. Once we do that,
we should either deprecate --metadatacopies or make it a synonym based
on the command (pvmetadatacopies for pv commands, and vgmetadatacopies
for vg commands). The latter option would likely just require a simple
'strncpy' check against cmd->command->name to qualify the merge_synonym
call.
Update nightly tests to cover the pvmetadatacopies synonym.
Note that this patch is the result of an eariler review comment for
the implicit pvcreate patches. Should apply cleanly on top of the
implicit pvcreate patches (I applied after patch 10/10 in that series).
NOTE: This patch will require --pvmetadatacopies for vgconvert as
--metadatacopies is no longer accepted.
vgcreate and vgextend now implicitly call pvcreate on devices not
currently initialized for LVM use. Previously these commands would
fail with an error, so clarify the new behavior in the man page.
Adds implicit pvcreate support when calling vgcreate or vgextend with
device paths that are not yet PVs. This changes the behavior of vgcreate
and vgextend from failing with an error message to implicitly pvcreating.
Split pvcreate_validate_params into recovery and non-recovery parameters.
This is necessary so we can call the non-recovery validate function from
vgextend / vgcreate. Note in the pvcreate tool case, we must call the
recovery validation function first (see treatment of pe_start and --zero),
and that we add a call to fill_default_pvcreate_params before the validation
functions.
We need defaults for pvcreate_params at a higher level - this will
allow us to use a common function from the tools to take defaults,
then fill in any non-defaults from the commandline.
Future patches will refactor vgcreate/vgextend to call this function
if one or more pvcreate parameters are given on the commandline.
Another refactoring for implicit pvcreate support. We need to get
the pvcreate parameters somehow to the vg_extend routine. Options
seemed to be:
1. Attach the parameters to struct volume_group. I personally
did not like this idea in most cases, though one could make an
agrument why it might be ok at least for some of the parameters
(e.g. metadatacopies).
2. Pass them in to the extend routine. This second route seemed
to be the best approach given the constraints.
Future patches will parse the command line and fill in the actual
values for the pvcreate_single call.
Should be no functional change.
Should be no functional change. If this parameter is set to NULL, just fail
the extend if the device is not already a PV. If non-NULL, try pvcreate_single
before failing. Note that pvcreate_single() handles the log_error in case
of failure so we just return 0 if pvcreate_single() fails.
is granted at one mode and an attempt to convert it wthout the LCK_CONVERT
flag set then it will return errno=EBUSY.
This fixes a pretty bad bug in which an LV could be activated exclusively on
one node and lvchange -ay on another would convert it to shared!
It might break some things in other areas, but I doubt it.
Now uppercase letters imply Si units, so use lowercase everywhere.
We could stay with uppercase, but then we'd have to deal with rounding, etc.
Also, some output / error messages change slightly (instead of "GB" we're
now saying "GiB").
One test enhancement might be to add some new tests for the units changes
but for now let's just get the test back to passing.
lv_deactivate now returns always success, because tree deactivation
functions (see dm_tree_deactivate_children) always returns success.
Because code should return failure in lv_deactivate at least,
fix it by checking for device existence after real deactivation call.
(After discussion this was prefered solution to dm tree function rewrite
which affects snapshots and mirrors.)
Add configure --enable-units-compat to set si_unit_consistency off by default.
Use standard output units for 'PE Size' and 'Stripe size' in pv/lvdisplay.
confuses me, so I've added a comment at the top of the function to
remind me of this.
I also found that 'mirror_emit_segment_line' was returning 0 (return_0)
on failure /and/ success. It now returns 1 for success and 0 for failure -
just like '_emit_areas_line' and its calling function, '_emit_segment_line'.
Clean up VG_RESIZEABLE flag by creating vg_is_resizeable().
Update comment - we no longer have ALLOW_RESIZEABLE.
Also use vg_is_exported() in one place missed by earlier patch.
Should be no functional change.
Remove the checks for vg_read_error() in most of the tools callback
functions and instead make the check in _process_one_vg() more general.
In all but vgcfgbackup, we do not want to proceed if we get any error
from vg_read(). In vgcfgbackup's case, we may proceed if the backup
is to proceed with inconsistent VGs. This is a special case though,
and we mark it with the READ_ALLOW_INCONSISTENT flag passed to
process_each_vg (and subsequently to _process_one_vg).
NOTE: More cleanup is needed in the vg_read_error() path cases.
This patch is a start.
This patch is all just cleanup and no other patch depends on it.
Replace explicit dereference and check with vg_is_exported().
Update a few copyrights and remove unnecessary whitespace.
Should be no functional change.
Of the vgs field vg_attr, a few of the most likely to be used attributes
are clustered, exported, and partial. This patch adds the following 3
functions:
uint64_t lvm_vg_is_clustered(const vg_t vg)
uint64_t lvm_vg_is_exported(const vg_t vg)
uint64_t lvm_vg_is_partial(const vg_t vg)
We don't have to call dm_cookie_supported with dm_udev_get_sync_support
this way. Also, it's necessary for the current code to work properly on
systems without cookie support (older kernels).
- add DM_UDEV_RULES_VSN to provide a variable to be checked for in the other
rules (e.g. to check that DM rules are actually installed, we can alternate
functionality in the other rules based on this information, also we have
versioning support for the rules)
- set proper sbin path for dmsetup and blkid, /sbin first, then /usr/sbin.
This is necessary for anaconda to work properly.
- add 'last_rule' for cryptsetup's temporary devices (symlinks in /dev/mapper
only)
The test/api directory TARGET line will be reserved for non-interactive
unit tests. Building the interactive test can still be done with "make test"
from the test/api dir.
When clogd was renamed to cmirrord, somehow git got the remove of the old
files but not the add of the new files. This patch adds the new files.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
The test/api directory TARGET line will be reserved for non-interactive
unit tests. Building the interactive test can still be done with "make test"
from the test/api dir.
Now that we've refactored the internal library functions that do the
vg_remove, we can handle the deferred commit of a lvm_vg_remove() inside
lvm_vg_write(). This makes the VG create/remove API more consistent in
terms of disk commits - they now both require an lvm_vg_write() to commit
the create or remove to disk.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
Author: Dave Wysochanski <dwysocha@redhat.com>
Now that we've split vg_remove_single into two routines, in the first routine
that only manipulates memory, we move the PVs from the vg->pvs list to the
vg->removed_pvs list. Then later, we iterate through this list to write the
removed PVs to disk, which removes them from the volume group and places them
into the internal ORPHAN VG.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
Author: Dave Wysochanski <dwysocha@redhat.com>
Split vg_remove_single into vg_remove_check (mandatory checks before
vgremove) and vg_remove (do actual remove by committing to disk).
In liblvm, we'd like to provide an consistent API that allows multiple
changes in memory, then let lvm_vg_write() control the commit to disk. In
some cases (for example, lvresize calls fsadm) this may not be possible.
However, since we are using an object model and dividing things into small
operations, the most logical model seems to be the lvm_vg_write model, and
handling the special cases as they arrive. So as best as possible
we move towards this end.
A possible optimization would be to consolidate vg_remove (committing)
code with vgreduce code. A second possible optimization is making vgreduce
of the last device equivalent to vgremove. Today, lvm_vg_reduce fails if
vgreduce is called with the last device, but from an object model perspective
we could view this as equivalent to vgremove and allow it. My gut feel is
we do not want to do this though.
Author: Dave Wysochanski <dwysocha@redhat.com>
Later patches should consolidate the vgremove / vgreduce functions but for
now let's clarify what vg_remove actually does by changing the name.
Author: Dave Wysochanski <dwysocha@redhat.com>
Add a new constraint that vgname locks must be obtained in
alphabetical order. At this point, we have test coverage for
the 3 commands affected - vgsplit, vgmerge, and vgrename.
Tests have been updated to cover these commands.
Going forward any command or library call that must obtain
more than one vgname lock must do so in alphabetical order.
Future patches will update lvm2app to enforce this ordering.
Author: Dave Wysochanski <dwysocha@redhat.com>
These functions are really identical but for clarity I made them separate
functions in this patch.
Should be no functional change.
Author: Dave Wysochanski <dwysocha@redhat.com>
If the destination vgname comes before the source vgname, we must open the
destination first because of the locking rules. Thus, do a strcmp and set
the flag based on the comparison.
Author: Dave Wysochanski <dwysocha@redhat.com>
Slight functional change. If we open the destination first, we cannot
know the 'fmt'. In this case we use the default metadata type unless
the user has specified -M on the cmdline. If not, in most cases this
is fine since we use the LVM2 default metadata type. However, if the
user is specifying a non-default metadata type (e.g. lvm1) and the order
of the names is such that we have to open the destination (vg_to) first,
we have a problem. So in this case, we require the use of -M and vgsplit
will fail with an error if not. I've updated the man page to recommend
the usage of -M in this case.
Author: Dave Wysochanski <dwysocha@redhat.com>
Move the creating/opening of the destination vg into its own function so later
we can reorder the source / destination vg opening based on the alphabetical
lock order rule.
Should be no functional change but code is a bit tricky.
Author: Dave Wysochanski <dwysocha@redhat.com>
Introduce 'lock_vg_from_first' flag to retain which vg was locked first.
Should be no functional change.
Author: Dave Wysochanski <dwysocha@redhat.com>
This will aid in future refactorings and allow for us to reorder the source
and destination vg based on alphabetical names.
Should be no functional change.
Author: Dave Wysochanski <dwysocha@redhat.com>
interface it should be using, it can still be overriden with -I.
If corosync isn't running or there is no information then the usual
checking will happen.
This code only builds if corosync is available.
# pvcreate -u udwxr7-BoKY-EeKM-r033-xK6o-4og7-F13sGi /dev/sdc
uuid udwxr7BoKYEeKMr033xK6o4og7F13sGi|��� already in use on "/dev/sdb1"
is now
# pvcreate -u udwxr7-BoKY-EeKM-r033-xK6o-4og7-F13sGi /dev/sdc
uuid udwxr7-BoKY-EeKM-r033-xK6o-4og7-F13sGi already in use on "/dev/sdb1"
Eliminate busy loop during pvcreate of a "normal" partition.
_md_sysfs_attribute_snprintf() would busy loop if the device it was
given was not a blkext-based MD partition.
Rather than being cute with a busy-loop prone 'goto check_md_major' in
_md_sysfs_attribute_snprintf(): explicitly check if the provided device
is a blkext-based partition (blkext_major()); and then check that the
get_primary_dev() determined parent is an MD device (md_major()).
The device-mapper mirror CTR table has been changing over time. This has
now been corrected to handle the old and new methods for invoking the
'block_on_errors' and 'cluster' features. (The code that does this was
accidentally committed in the previous check-in. This check-in finishes
the job.)
This check-in includes the touch-ups, make file changes, copyrights,
and other necessities to include the cluster log daemon into the
build system.
[autoconf still needs to be run to generate the 'configure' and
'Makefile' files.]
Eliminate dependency on outside library, since the same functionality
exists in our tree.
[It is important that the bitops work in the same way, as the bitmaps
must remain backwards compatible. I haven't tested every architecture,
but the x86* archs work. My test involved using the old ext2fsprogs
bitops, memcpy'ing the bits over to the LVM bitset array and ensuring
that only the bits set via the old methods were set.]
This patch update pv_t handle to be consistent with lvm_t - define as a pointer
to internal struct physical_volume.
Author: Dave Wysochanski <dwysocha@redhat.com>
This patch update lv_t handle to be consistent with lvm_t - define as a pointer
to internal struct logical_volume.
Author: Dave Wysochanski <dwysocha@redhat.com>
This patch update vg_t handle to be consistent with lvm_t - define as a pointer
to internal struct volume_group.
Author: Dave Wysochanski <dwysocha@redhat.com>
Full changes
- Fix vgextend error path when lock_vol(VG_ORPHANS) fails
- Move lock_vol(VG_ORPHANS) before archive(vg) - safe & simpler error paths
- Remove legacy comment/code that no longer applies
Found in review - Milan Broz <mbroz@redhat.com>
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
The changes to remove LCK_NONBLOCK from the LVM locks broke clvmd because the
code was clearly wrong but working anyway! The constant was being masked rather
than the variable that was supposed to match against it.
Rename private _primary_dev() to a public get_primary_dev() and reuse it
to allow retrieval of the MD sysfs attributes (raid level, etc) for MD
partitions.
Signed-off-by: Mike Snitzer <snitzer@redhat.com>
Improve lib/device/device.c:_primary_dev()'s ability to look up the
primary device associated with all partitions; including blkext
(e.g. partitions directly on MD). The same will also work for obscure
sysfs paths; e.g.: paths with mangled names like the HP cciss driver
uses: /sys/block/cciss!c0d0/cciss!c0d0p1/
Signed-off-by: Mike Snitzer <snitzer@redhat.com>
Adds 'data_alignment_detection' config option to the devices section of
lvm.conf. If your kernel provides topology information in sysfs (linux
>= 2.6.31) for the Physical Volume, the start of data area will be
aligned on a multiple of the ’minimum_io_size’ or ’optimal_io_size’
exposed in sysfs.
minimum_io_size is used if optimal_io_size is undefined (0). If both
md_chunk_alignment and data_alignment_detection are enabled the result
of data_alignment_detection is used.
Signed-off-by: Mike Snitzer <snitzer@redhat.com>
If the pvcreate --dataalignmentoffset option is not specified the start
of a PV's aligned data area will be shifted by the associated
'alignment_offset' exposed in sysfs (unless
devices/data_alignment_offset_detection is disabled in lvm.conf).
Signed-off-by: Mike Snitzer <snitzer@redhat.com>
Documented which use-cases force the reinstatement of the nuanced
handling of pe_start. As soon as orphan PVs are eliminated much of this
will no longer be a concern ('preserve_pe_start' can be reenabled in
.pv_setup).
Added defensive 'if (pv->pe_align)' check in _text_pv_write()'s pe_start
loop.
If pv_setup was given a non-zero pe_start it would short-circuit
establishing a default pv->pe_align. pv->pe_align=0 would result
in a divide by zero in _mda_setup(). 'vgconvert -M2 $vgname' hit this.
.pv_write still properly preserves pe_start if it was supplied.
Adds pe_align_offset to 'struct physical_volume'; is initialized with
set_pe_align_offset(). After pe_start is established pe_align_offset is
added to it.
Signed-off-by: Mike Snitzer <snitzer@redhat.com>
areas.
This preserved pe_start would quickly be readjusted to follow the first
mda anyway. An example use-case that hit this code path is: running
pvcreate on an already existing PV _without_ a preceeding pvremove.
Signed-off-by: Mike Snitzer <snitzer@redhat.com>
Without this fix rounding the end of the first mda to a pe_align
boundary could silently exceed the disk_size.
Final 'if (start1 + mda_size1 > disk_size)' block serves as a safety
net.
Signed-off-by: Mike Snitzer <snitzer@redhat.com>
Document existing pe_start policy.
Fix issue in _text_pv_setup() where existing pe_start case could have
the pv->pe_start set to pv->pe_align even though pe_start shouldn't ever
change.
vgconvert and pvcreate have a facility to preserve the existing start
of the on-disk data extents, known as pe_start.
They indicate this by passing the existing value to the pvsetup function
which must preserve it.
This patch avoids one particular case where the value could get
changed incorrectly now that the alignment settings are configurable.
Signed-off-by: Mike Snitzer <snitzer@redhat.com>
A patch to the kernel, adding the 'luid' field to dm_ulog_request,
will allow us to properly identify log instances. We will now
be able to definitively identify which logs are to be removed/
suspended/resumed. This replaces the old faulty behavior of
assuming the logs were the same if they had the same UUID and
incrementing/decrementing a reference count.
For now, a simple way to enforce the read/write semantics is to just save the
open mode of the VG. If the caller uses lvm_vg_create, the mode is write.
The caller using lvm_vg_open can use either read or write to open the VG.
Once we have this, we enforce the permissions on each API call and don't allow
a caller to modify a VG that has not been opened properly.
This may be better combined with the locking mode, but I view that as future
cleanup, past this initial release. The intial release should enforce the
basic object semantics though, as described in the lvm.h file.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
Adding the ability to get the seqno is important for an application to
determine if something has changed in a VG. Otherwise, the only way to
know is to open the VG with write permission and hold the handle.
This addresses a a large amount of Alasdair's review. Subsequent patches
will address remaining issues.
Addressed:
// FIXME Mention that's also required on error.
// FIXME Be consistent in terminology. It's called "system_dir" then last sentence says "system directory setting". Is it referring to "system_dir" there or something else?
// FIXME Mention it frees all resources and cannot be used subsequently?
// FIXME What does "any system configuration" mean?
// FIXME Expand on that explanation a bit, now that we know what the other fns look like.
// FIXME Not sure about that - it needs to scan sometimes. "will not" or "might not" ?
// FIXME: That's a FIXME in the code!!!
// FIXME What does "copied" mean in this context???
// FIXME Say what struct the returned struct dm_list is a list of...
// FIXME "This API" ? This function creates an object in memory?
// FIXME This function commits the Volume Group object referenced by the VG handle to disk?
// FIXME Where is "Name" defined? Absolute pathname?
Outstanding:
// FIXME Version function first? No structs or handles needed for that.
// FIXME Sort out this alignment. "Set an" directly below "system_dir" looks awful. Indent differently? More blank lines?
// FIXME Check how doxygen processes this. E.g. "return: LVM handle. You must use lvm_error() to check there were no errors and confirm that the handle is valid for passing to other functions."
// FIXME Find a better name. lvm_init.
// FIXME Consider renaming according to the new name for lvm_create.
// FIXME Please can we use dm_malloc throughout?
Allowing the caller to override the LVM configuration with an API will
enable them to use things such as device filters.
While very flexible, there is some danger to this API in that it will
make it harder to debug setups that have a changing config and deduce
what might have happened. At some point we may want to limit the scope
of this API but for now it is as open as the --config option to lvm commands.
Update exported symbols. When I renamed lvm_reload_config to lvm_config_reload
I forgot to rename so I renamed that one here.
This I believe is the last liblvm API for now. ;-)
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
These lower-priority interfaces are not currently implemented in liblvm
but are on the TODO list in the near term.
Author: Thomas Woerner <twoerner@redhat.com>
Acked-by: Dave Wysochanski <dwysocha@redhat.com>
Extend lvm_vg_write to remove pvs removed in lvm_vg_reduce. The lvm
volume_group internal structure removed_pvs is used for that. The list is
empty afterwards.
Add new test for vg->pvs emptyness to lvm_vg_write to prevent to write empty
vgs. This is needed because of lvm_vg_reduce and lv_vg_create, which creates
empty vgs.
Signed-off-by: Thomas Woerner <twoerner@redhat.com>
Acked-by: Dave Wysochanski <dwysocha@redhat.com>
Author: Dave Wysochanski <dwysocha@redhat.com>
This function behaves a little bit different than vg_reduce_single, because
it allowes to remove even the latest pv. This has been done to be consistent
to lvm_vg_create, which creates an empty vg.
removed_pvs has been added to the volume_group struct. vg_reduce adds remove
pvs to this list to be able to commit the changes for the pvs in lvm_vg_comm
in liblvm2app.
Initialize removed_pvs list in format-specific volume_group constructors.
Ideally, we should have a base constructor here that initializes the general
non-format specific members of struct volume_group. But until then, there
are multiple places to initialize these members. Maybe a better patch would
be a base constructor patch for struct volume_group. That is more work
though.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
Signed-off-by: Thomas Woerner <twoerner@redhat.com>
Author: Dave Wysochanski <dwysocha@redhat.com>
The two liblvm functions that return a list of vgnames and vguuids use
cmd->mem to allocate the list. Make it clear to the caller that this
memory will be freed when the LVM handle is freed.
Clean up and clarify the return value of the functions. In the
case of a memory allocation error, add a couple log_errnos to the internal
code, and make it clear that memory allocation returns a NULL pointer.
If there are no VGs in the system, the list returned is an empty list.
Make a note of the fact that currently we return hidden VG names, how
these can be detected (always start with "#"), and that they should
not be used.
Author: Dave Wysochanski <dwysocha@redhat.com>
The general naming scheme for most liblvm APIs is:
lvm_<object>_<action>
As there are likely to be other things to do on the lvm 'config' object
(i.e. lvm_config_set_device_filter), we should use consistent naming.
Author: Dave Wysochanski <dwysocha@redhat.com>
Limited implementation but other types of activation should probably have
separate calls. We also currently do not handle pvmoves or lvconverts.
Author: Dave Wysochanski <dwysocha@redhat.com>
For now, liblvm will return -1 (fail) / 0 (success) or
NULL (fail) / non-NULL (success). Upon failure, lvm_errno and
lvm_errmsg should be used to determine the precise error.
Author: Dave Wysochanski <dwysocha@redhat.com>
For now, we use the following scheme.
For APIs that return an int, success is 0, fail is -1.
APIs that return handles, success is non-NULL, fail is NULL.
At this early stage, liblvm error handling mechanism is subject to change,
but for now we go with this simple scheme consistent with system
programming.
Author: Dave Wysochanski <dwysocha@redhat.com>
Hard to divide into smaller patches because of the moving around and
commenting, so just put in the new file diffs. We will review and can
update as necessary.
Author: Dave Wysochanski <dwysocha@redhat.com>
Add a very simple version of lvm_vg_remove_lv.
Since we currently can only create linear LVs, this simple remove function
is adequate. We must refactor lvremove_single a bit.
Author: Dave Wysochanski <dwysocha@redhat.com>
Add the most straightforward 'get' functions required for anaconda.
These are the ones that return simple uint64_t values.
The other more complex ones involve the lv_attr bits. These will
come in a separate patch series since each lv_attr bit will be returned
in a separate API instred of returning the string and requiring the
user to parse it.
Author: Dave Wysochanski <dwysocha@redhat.com>
For liblvm 'get' functions, we should share code with the reporting functions.
This means we need common code to return the values for the fields.
In this patch we refactor a few of the fields needed in liblvm.
Unfortunately, for the simple fields that do derefernces of structure
members (for example, vg_extent_count), we cannot call the common function
from the reporting infrastructure without more refactoring. The reason is
that the dereference of the simple fields is done deep inside the reporting
code (to get the generic "data" pointer), and the display function is a
generic 'size32' function. We can fix these issues later with more
refactoring.
Should be no functional change and the testsuite should cover any possible
regressions. The only fields in the report affected by this patch are:
vg_size, vg_free, and pv_mda_count.
Author: Dave Wysochanski <dwysocha@redhat.com>
After some refactorings, we can now move the bulk of _lvcreate into the
internal library, and we can call from liblvm. In the future, we should
refactor lv_create_single further, probably by segtype, to reduce the
size of struct lvcreate_params. For now this is a reasonable refactor
and allows us to re-use the function from liblvm.
Author: Dave Wysochanski <dwysocha@redhat.com>
The main _lvcreate function should deal with extents - the 'size' parameter
is just an intermediate step.
Should be no functional change.
Author: Dave Wysochanski <dwysocha@redhat.com>
Create a new structure, lvcreate_cmdline_params, to store parameters only
relevant for the cmdline, not the library call to lvcreate.
Should be no functional change.
Author: Dave Wysochanski <dwysocha@redhat.com>
Move extents calculation adjustments into their own local functions
right after we read the vg. This calculation really is not part of
the LV create function but is rather an adjustment to the parameters
based on what is given on the cmdline. So we move it outside the main
_lvcreate.
Should be no functional change.
Author: Dave Wysochanski <dwysocha@redhat.com>
A couple simple refactorings of _lvcreate - should be no functional change.
Move tags_ARG parsing into _lvcreate_params. Also use lp->voriginsize
instread of arg_count(). These refactorings make it easier to move the
bulk of _lvcreate into the library.
Author: Dave Wysochanski <dwysocha@redhat.com>
Although the tools do not currently do this, we update lvm_vg_extend
to do an implicit pvcreate on an uninitialized device. The tools will
soon be refactored to do this as well, but more work is needed in the
tools. For now we update lvm_vg_extend since this is the behavior
required by liblvm.
With this change, the simple liblvm unit test, test/api/vgtest.c
should pass whether or not the device is initialized.
Author: Dave Wysochanski <dwysocha@redhat.com>
The implicit pvcreate require either moving the ORPHAN_VG lock outside
pvcreate_single or somehow having the function know or detect whether
the ORPHAN_VG lock is already held.
Author: Dave Wysochanski <dwysocha@redhat.com>
Passing NULL for pvcreate parameters gives you default parameters for
pvcreate_single.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
Author: Dave Wysochanski <dwysocha@redhat.com>
In preparation for implicit pvcreate during vgcreate / vgextend,
move bulk of pvcreate logic inside library.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
Author: Dave Wysochanski <dwysocha@redhat.com>
We must hold the VG_ORPHAN lock until we commit to disk. Otherwise,
we risk a race condition on vgcreate / vgextend. Reverts the following
commit:
commit 72a41480ba
Author: Dave Wysochanski <dwysocha@redhat.com>
Date: Fri Jul 10 20:09:21 2009 +0000
Move orphan lock obtain/release inside vg_extend().
With this change we now have vgcreate/vgextend liblvm functions.
Note that this changes the lock order of the following functions as the
orphan lock is now obtained first. With our policy of non-blocking
second locks, this should not be a problem.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
The lvm_list_vg_{names|ids} functions do not do a scan so we provide
a liblvm function that does a scan.
Author: Dave Wysochanski <dwysocha@redhat.com>
These functions provide the capability of enumerating all vgnames and
vgids in the system. They do not do a scan of the system.
Author: Dave Wysochanski <dwysocha@redhat.com>
Caller must free the memory of the uuid / name returned.
This may not be the best memory management policy since it may lead to
memory leaks if the caller has code like this:
if (!lvm_vg_get_name(vg))
Maybe we don't care - if we do we can use pools tied to handles later
or some other scheme.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
Acked-by: Thomas Woerner <twoerner@redhat.com>
- Use vgmem pool to allocate a list of lvm_*_list structs
- Allocate a new list each call (list may have changed since last call)
- Add to liblvm's exported symbols
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
Acked-by: Thomas Woerner <twoerner@redhat.com>
- pv_t, vg_t, lv_t
- include libdevmapper.h: needed for struct dm_list
These list structures will be needed in later APIs to return a list of
handles to one object, given another object. For example, lvm_vg_list_lvs()
will return a list of LV handles (lv_t's) given a VG handle (vg_t). We
need a structure to do this so we define the LV structure, as well as the
other structures at this point.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
Some of the error interface is still TBD. Rather than exporting a lot
of codes, etc, just use a simple pass / fail. The allows our unit test
to not segfault if trying to create a VG that already exists.
lvm_vg_open() calls internal vg_read() function which is the entry point
for reading an existing VG. In addition to the mode, we include a 'flags'
parameter for future extensions.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
Reverts some of my 'cleanup' from last night. For now we will use pass/fail
on API calls (either 'int' return or NULL/non-NULL handle), then use
lvm_errno() to get more specific errors.
Author: Dave Wysochanski <dwysocha@redhat.com>
made to compensate for the changes in the kernel-side component
that recently went upstream. (Things like: renamed structures,
removal of structure fields, and changes to arguments passed
between userspace and kernel.)
These messages are unnecessary in the set functions. We check for this
condition and print a message in the vgchange tool but not the library
functions.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
When converting to the new liblvm functions, the vgcreate code path
changed to create a new vg, then set values. As a result of this
change, and the fact that we give a user a message if they try to
set the same value of a VG attribute (extent_size, alloc_policy, etc),
you'll see these 2 extraneous "is already" messages with vgcreate:
tools/lvm vgcreate vg2 /dev/loop2
Physical extent size of VG vg2 is already 4.00 MB
Volume group allocation policy is already normal
Volume group "vg2" successfully created
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
Author: Dave Wysochanski <dwysocha@redhat.com>
Since we are using errno values, we should use '0' as a default value
which indicates a non-error, rather than defining some made-up default
value that is not defined in errno. If we need to deviate from errno
values, this will most likely indicate a flaw in our design.
Add prototypes for lvm_errno and lvm_errmsg inside lvm.h and provide
a basic description of their function. This fixes a couple compile
warnings.
Author: Dave Wysochanski <dwysocha@redhat.com>
We provide a lock type that behaves like no_locking, but is not
clustered. Moreover, it also forbids any write locks. This magically (and
consistently) prevents use of clustered VGs, or changing local VGs with
--ignorelockingfailure. As a bonus, we can remove the special hacks in a few
places. Of course, people looking for trouble can always set their locking_type
to 0 to override.
In _process_one_vg, we should never proceed if the VG read fails with certain
conditions. If we cannot allocate or construct the volume_group structure,
we should not proceed - this is true regardless of the tool calling the
iterator. In other cases, when the volume group structure is constructed but
there is some error (PVs missing, metadata corrupted, etc), some tools may
want to process the VG while others may not.
Author: Dave Wysochanski <dwysocha@redhat.com>
In vg_backup_single, we should error out if we vg_read_error(vg) and the
error code we received was anything other than FAILED_INCONSISTENT.
Original code contained an error because C operator precedence.
Note - this was part of the vg_read() so no WHATS_NEW entry neceesary.
Author: Dave Wysochanski <dwysocha@redhat.com>
liblvm unit test case uses the following APIs:
- lvm_create, lvm_destroy
- lvm_vg_create, lvm_vg_extend, lvm_vg_set_extent_size, lvm_vg_write,
lvm_vg_remove, lvm_vg_close
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
Acked-by: Alasdair G Kergon <agk@redhat.com>
Add the following VG APIs to liblvm/.exported_symbols:
-lvm_reload_config
-lvm_vg_create
-lvm_vg_extend
-lvm_vg_set_extent_size
-lvm_vg_write
-lvm_vg_close
-lvm_vg_remove
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
Acked-by: Alasdair G Kergon <agk@redhat.com>
Add some liblvm APIs for VGs. Most of these APIs simply call into the internal
liblvm library. Ideally we should call the liblvm functions directly from
the tools. However, until we convert more of the code to liblvm functions,
things like the cmd_context will get in the way. For now just implement the
liblvm functions as wrappers around the internal functions, with a little
error checking and return code handling. We put all these vg APIs into a
new file, lvm_vg.c
The following APIs are implemented:
lvm_vg_create, lvm_vg_extend, lvm_vg_set_extent_size, lvm_vg_write,
lvm_vg_remove, lvm_vg_close.
Still TODO:
- cleanup error handling by using lvm_errno() and related APIs
- cleanup naming / clarify which functions commit to disk vs not
- implement more 'set' functions
- decide on 'set' / 'change' nomenclature
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
Acked-by: Alasdair G Kergon <agk@redhat.com>
This needs initialized to non-NULL before using the archive() call.
Normally this is set to the cmdline string when lvm is called from a tool.
We could think about using it in another way, as a potential audit trail
of liblvm calls, or just leave it set to the default "liblvm", similar to
what clvmd does. For now, just set it to "liblvm".
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
Acked-by: Alasdair G Kergon <agk@redhat.com>
Define the 5 main liblvm objects to be the pv, vg, lv, lvseg, and pvseg.
We need handles defined to all these objects in order for liblvm to be
equivalent to the reporting commands pvs, vgs, and lvs.
- move vg_t, lv_t, and pv_t from metadata-exported.h into lvm.h
- move lv_segment and pv_segment forward declarations into lvm.h
- add lvseg_t and pvseg_t to lvm.h
NOTE: We currently have an inconsistency in handle definitions.
lvm_t is defined as a pointer, while these other handles are just
structures. We should pick one scheme and be consistent - perhaps
define all handles as pointers (this is what I've seen elsewhere).
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
Acked-by: Alasdair G Kergon <agk@redhat.com>
The checks for RESIZEABLE_VG should now be inside the various functions that
have to do such operations.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
Acked-by: Alasdair G Kergon <agk@redhat.com>
Remove READ_REQUIRE_RESIZEABLE flag from vgsplit similar to the removal from
vgextend. Move the check inside the functions that actually move pvs from
one vg structure to another. Should be no functional change.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
Acked-by: Alasdair G Kergon <agk@redhat.com>
In the future we may export these functions or something like them in liblvm
For now this helps in cleaning up the checks for RESIZEABLE since we can
use the internal library function vg_bad_status_bits.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
Acked-by: Alasdair G Kergon <agk@redhat.com>
Move the check for the RESIZEABLE flag inside the vg_extend function.
When we consolidated the vg locking, reading, and status flag checking,
we tied the check for the RESIZEABLE flag to the vg_read() call. The problem
with this is you cannot know what other APIs the application my or may not
call after a vg_read() call. Thus the READ_REQUIRE_RESIZEABLE flag is not
really ideal - ideally we should be checking for this flag on a specific
operation, not inside the vg_read() call. This patch moves one check inside
the library.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
Acked-by: Alasdair G Kergon <agk@redhat.com>
xlate64 produces unsigned long long type, but PRIu64 is defined
to accept argument unsigned long type (on 64-bit machines).
On existing machines, both types have the same size, so it works,
but it is still wrong and produces a warning.
Fix it by using a cast to uint64_t --- according to the standard,
PRIu64 argument matches type uint64_t.
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
Orphan lock is now obtained second and released first, and all tools
are consistent in this regard.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
With this change we now have vgcreate/vgextend liblvm functions.
Note that this changes the lock order of the following functions as the
orphan lock is now obtained first. With our policy of non-blocking
second locks, this should not be a problem.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
Move the vg orphan lock inside vg_remove_single, now a complete liblvm
function. Note that this changes the order of the locks - originally
VG_ORPHAN was obtained first, then the vgname lock. With the current
policy of non-blocking second locks, this could mean we get a failure
obtaining the orphan lock. In the case of a vg with lvs being removed,
this could result in the lvs being removed but not the vg. Such a
scenario could have happened prior though with a different failure.
Other tools were examined for side-effects, and no major problems
were noted.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
Move check for active LVs outside of library function. The vgremove
liblvm function function will fail if there are active LVs. It will
be the application's responsibility to check this condition and remove
the LVs individually before calling vgremove. Note also that we've
duplicated the EXPORTED_VG check in vgremove_single (tools) and
vg_remove_single (library). Duplication seemed the only option here
since we don't want to do the automatic removal of LVs (in the tools)
if the vg is exported, and we still need to protect the library call
from removal if the vg is exported.
We still need to deal with the ORPHAN lock but vg_remove_single is now
very close to our liblvm function.
TODO: Refactor lvremove in a similar way.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
If there is syntax error in metadata, it now prints messages
like:
Couldn't read 'start_extent' for segment 'extent_count'.
Couldn't read all logical volumes for volume group vg_test.
The segment specification is wrong and confusing.
Patch fixes it by introducing "parent" member in config_node which
points to parent section and config_parent_name function, which
provides pointer to node section name.
Also it adds several LV references where possible.
vg_t *vg_create(struct cmd_context *cmd, const char *vg_name);
This is the first step towards the API called to create a VG.
Call vg_lock_newname() inside this function. Use _vg_make_handle()
where possible.
Now we have 2 ways to construct a volume group:
1) vg_read: Used when constructing an existing VG from disks
2) vg_create: Used when constructing a new VG
Both of these interfaces obtain a lock, and return a vg_t *.
The usage of _vg_make_handle() inside vg_create() doesn't fit
perfectly but it's ok for now. Needs some cleanup though and I've
noted "FIXME" in the code.
Add the new vg_create() plus vg 'set' functions for non-default
VG parameters in the following tools:
- vgcreate: Fairly straightforward refactoring. We just moved
vg_lock_newname inside vg_create so we check the return via
vg_read_error.
- vgsplit: The refactoring here is a bit more tricky. Originally
we called vg_lock_newname and depending on the error code, we either
read the existing vg or created the new one. Now vg_create()
calls vg_lock_newname, so we first try to create the VG. If this
fails with FAILED_EXIST, we can then do the vg_read. If the
create succeeds, we check the input parameters and set any new
values on the VG.
TODO in future patches:
1. The VG_ORPHAN lock needs some thought. We may want to treat
this as any other VG, and require the application to obtain a handle
and pass it to other API calls (for example, vg_extend). Or,
we may find that hiding the VG_ORPHAN lock inside other APIs is
the way to go. I thought of placing the VG_ORPHAN lock inside
vg_create() and tying it to the vg handle, but was not certain
this was the right approach.
2. Cleanup error paths. Integrate vg_read_error() with vg_create and
vg_read* error codes and/or the new error APIs.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
NOTE: vg_set_alloc_policy() returns success if you try to set a value that
is already stored. The behavior of vgchange is the same though - it fails.
There is a fixme noted in the code about this inconsistency, which should
be resolved if possible.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
In liblvm, we will reserve the word 'change' to mean an API that
both sets one or more values, and commits to disk. This will be
consistent with the LVM commandline. The existing vg_change_pesize()
function does not commit to disk, but just changes the extent_size
and ensures all internal structures are updated. This logic should
be contained in a function that sets the value.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
It would be nice to have one function that does all the validation
and setting of the VG's pesize. However, currently some checks
are in the higher-level function _vgchange_pesize(), and some
checks are in the lower function vg_change_pesize().
This patch moves most of the higher-level checks inside
vg_change_pesize. In one case a failure return code is
changed from ECMD_FAILED to EINVALID_CMD_LINE.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
Remove unneeded LOCK_NONBLOCKING from vg_read() API and tools that
use it. We no longer need this flag anywhere since we now automatically
set LCK_NONBLOCK inside lock_vol() if vgs_locked().
For further details, see:
commit d52b3fd3fe
Author: Dave Wysochanski <dwysocha@redhat.com>
Date: Wed May 13 13:02:52 2009 +0000
Remove NON_BLOCKING lock flag from tools and set a policy to auto-set.
As a simplification to the tools and further liblvm, this patch pushes
the setting of NON_BLOCKING lock flag inside the lock_vol() call.
The policy we set is if any existing VGs are currently locked, we
set the NON_BLOCKING flag.
At some point it may make sense to add this flag back if we get an
RFE from a liblvm user, but for now let's keep it as simple as
possible.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
Remove READ_CHECK_EXISTENCE and vg_might_exist().
This flag and API is no longer used now that we have a separate
API to check for existence.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
Remove unneeded LOCK_KEEP from vg_read() interface.
Update comment to clarify cases where _vg_lock_and_read() may return
with an error but the lock held. Would be nice to make the vg_read()
interface consistent with regards to lock held and error behavior.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
Remove LOCK_KEEP and READ_CHECK_EXISTENCE from vgsplit.
These flags are no longer necessary. We now check for existence
in a differnet function, and it is not necessary to keep the lock.
Removing these flags simplifies the new vg_read() interface.
After this patch, we can fully remove LOCK_KEEP.
READ_CHECK_EXISTENCE needs a bit more work before full removal.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
Update units_to_bytes() to support (S)ectors: 500 bytes.
- 500 byte (S)ectors is of questionable value but it adds to consistency
if a user happens to use --units S. This seems better than an error.
Updated test/t-covercmd.sh to test --units [hS]
Document the units that can be displayed via --units uniformly.
- (p)etabytes and (e)xabytes were missing in pvs, vgs and lvs man pages.
Made lvreduce man page "... in units of megabytes." consistent (with the
lvextend and lvresize man pages).
Fix vg_read() error paths to properly release upon vg_read_error().
Note that in the iterator paths (process_each_*()), we release
inside the iterator so no individual cleanup is needed. However there
are a number of other places we missed the cleanup. Proper cleanup
when vg_read_error() is true should be calling vg_release(vg), since
there should be no locks held if we get an error (except in certain
special cases, which IMO we should work to remove from the code).
Unfortunately the testsuite is unable to detect these types of memory
leaks. Most of them can be easily seen if you try an operation
(e.g. lvcreate) with a volume group that does not exist. Error
message looks like this:
Volume group "vg2" not found
You have a memory leak (not released memory pool):
[0x1975eb8]
You have a memory leak (not released memory pool):
[0x1975eb8]
Author: Dave Wysochanski <dwysocha@redhat.com>
Sun May 3 12:54:28 CEST 2009 Petr Rockai <me@mornfall.net>
* Convert vgrename to vg_read_for_update.
Rebased 6/26/2009 - Dave W.
Author: Petr Rockai <prockai@redhat.com>
Committer: Dave Wysochanski <dwysocha@redhat.com>
Sun May 3 12:32:30 CEST 2009 Petr Rockai <me@mornfall.net>
* Rework the toollib interface (process_each_*) on top of new vg_read.
Rebased 6/26/09 by Dave W.
- Add skipping message to process_each_lv
- Remove inconsistent_t.
Sun May 3 11:40:51 CEST 2009 Petr Rockai <me@mornfall.net>
* Convert the straight instances of vg_lock_and_read to new vg_read(_for_update).
Rebased 6/26/09 by Dave W.
Sun May 3 11:40:51 CEST 2009 Petr Rockai <me@mornfall.net>
* Convert the straight instances of vg_lock_and_read to new vg_read(_for_update).
Author: Petr Rockai <prockai@redhat.com>
Committer: Dave Wysochanski <dwysocha@redhat.com>
Is an application uses query and set major:minor
to device, it should not fallback to default major by default.
Add new function whoich allows that (and use it in lvm2).
- validate the specified device is a PV and that it is in a VG
- automatically enable DEBUG (-d) if >= 4 -v instances were supplied
- preserve TMP_LVM_SYSTEM_DIR if it contains an lvm.conf and -d was
specified
- fix handling of special-case where PV is listed as "unknown device"
- more descriptive error when a PV is missing ("unknown device")
- unset LVM_SYSTEM_DIR if it was not originally set
- skip final vgscan if no changes were made
http://bugzilla.redhat.com/show_bug.cgi?id=500861
- Update list of fields/columns for each command (a few missing).
- Update list order to match "-o help" output (easier to verify field list)
- Add "{pv|vg|lv}_all" description.
- Move "-o help" sentence above column/field list.
New sample man page for lvs (pvs and vgs are similar):
-o, --options
Comma-separated ordered list of columns. Precede the list with ’+’ to append to the
default selection of columns instead of replacing it.
Use -o lv_all to select all logical volume columns, and -o seg_all to select all logical
segment columns.
Use -o help to view the full list of columns available.
Column names include: lv_uuid, lv_name, lv_attr, lv_major, lv_minor, lv_read_ahead,
lv_kernel_major, lv_kernel_minor, lv_kernel_read_ahead, lv_size, seg_count, origin, ori-
gin_size, snap_percent, copy_percent, move_pv, convert_lv, lv_tags, mirror_log, modules,
segtype, stripes, stripesize, regionsize, chunksize, seg_start, seg_start_pe, seg_size,
seg_tags, seg_pe_ranges, devices.
With --segments, any "seg_" prefixes are optional; otherwise any "lv_" prefixes are
optional. Columns mentioned in vgs (8) can also be chosen.
The lv_attr bits are:
E.g.
# vgscan
Parse error at byte 2360 (line 54): expected a value
Failed to load config file /etc/lvm/lvm.conf
You have a memory leak (not released memory pool):
[0x818c788] library (12 bytes)
...
status flag after reading the volume group. Thus, no need to set the flag
in vg_read() or clear it later before calling _vg_bad_status_bits().
Also, add back in the !lockingfailed() as part of the CLUSTERED flag check.
It's unclear why it was removed when the check was moved from
_vg_bad_status_bits() to inside _vg_lock_and_read().
There was an open question about the last check in the 'if' stmt for
lockingfailed() with a previous patch submitted. However, I would
defer that right now as it is a separate item and this patch should
be no functional change by including the !lockingfailed().
Petr acked this patch on 5/26 I just forgot to check it in.
Acked-by: Petr Rockai <prockai@redhat.com>
- it can support multiple segments, but note that
to work properly, correct IV (initialization vector)
offset parameter must be set properly.
Because most usage of IV start offset is when we join
several crypto segments together (so iv_offset is the segment
start offset), DM_CRYPT_IV_DEFAULT is defined to simplify
the process.
Function accepts the string in cipher agrument (already
including chainmode and iv type; chainmode and iv parameters are NULL
in this case) or user can provide split parameters which will
join into dm-crypt cipher specification "cipher-chainmode-iv".
All these parameters must be supplied in correct dm-crypt format.
Various tools need to check for existence of a VG before doing something
(vgsplit, vgrename, vgcreate). Currently we don't have an interface to
check for existence, but the existence check is part of the vg_read* call(s).
This patch is an attempt to pull out some of that functionality into a
separate function, and hopefully simplify our vg_read interface, and
move those patches along.
vg_lock_newname() is only concerned about checking whether a vg exists in
the system. Unfortunately, we cannot just scan the system, but we must first
obtain a lock. Since we are reserving a vgname, we take a WRITE lock on
the vgname. Once obtained, we scan the system to ensure the name does
not exist. The return codes and behavior is in the function header.
You might think of this function as similar to an open() call with
O_CREAT and O_EXCL flags (returns failure with -EEXIST if file already
exists).
NOTE: I think including the word "lock" in the function name is important,
as it clearly states the function obtains a lock and makes the code more
readable, especially when it comes to cleanup / unlocking. The ultimate
function name is somewhat open for debate though so later we may rename.
- PPC uses 64k page, some caculations are wrong in tests
- file utility is buggy on PPC and cannot detect swap, use blkid instead
- read ahead is quietly rounded down according to page size
(for now use some common divisor value in test)
Because preload of table for snapshot can produce snapshot
metadata (in kernel cow header) read.
Code should suspend origin first to avoid possible deadlock
when preloading (thus calling snapshot in-kernel constructor)
for origin with suspended cow device.
(fixes previous commit)
Several commands calls process_each_vg() and in provided
callback it explicitly recovers VG if inconsistent.
(vgchange, vgconvert, vgscan)
It means that old VG is released and reread but the function
above (process_one_vg) tries to unlock and release old VG.
Patch moves the repair logic into _process_one_vg() function.
It always tries to read vg (even inconsistent) and then decides
what to do according new defined parameter.
Also patch unifies inconsistent error messages.
The only slight change if for vgremove command, where
it now tries to repair VG before it removes if force arg is given.
(It works similar way before, just the order of operation changed).
When some parts of lvm are built as shared libraries (for example with
--with-snapshots=shared), the 'make' command does not build these parts.
The shared parts are built with 'make install' command.
This bug can be seen if you go to 'lib' subdirectory and type 'make'.
If you type 'make', the shared libraries are not built, if you type
'make all', the shared libraries are built.
The reason for the bug is the line $(SUBDIRS): $(LIB_STATIC)
If make is executed without any arguments, it makes the first target
in the Makefile. If the first target is '$(SUBDIRS): $(LIB_STATIC)',
it only builds static libraries.
This patch moves '$(SUBDIRS): $(LIB_STATIC)' after
include $(top_srcdir)/make.tmpl. make.tmpl contains the 'all' target
as its first target, so 'make' will be equivalent to 'make all' and
shared libraries will be build with 'make' command.
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
When mirror convert polling is started (mainly as backgound process,
in lvchange -a y or in lvconvert itself) it tries to read VG
and LV identified by its name.
Unfortunatelly, the VG can have already different LV under the same name,
and various more or less funny things can happen (note that
_finish_lvconvert_mirror suspends the volume for example).
(the typical example is our testing script which continuously recreates
LVs under the same name in the same VG.)
This patch adds optional uuid parameter which helps to properly
select the monitoring object. For lvconvert polling it is set to LV UUID
and both _get_lvconvert_vg and _get_lvconvert_lv uses it to read proper VG/LV.
(In the pvmove case it is NULL, here we poll for physical volume name).
If there is no free area for log, code should break the loop.
(Otherwise it uses uninitializes areas later.)
Easily reproducible using lvconvert --repair
- kill device with log
- run lvconvert --repair vg/lv (with no PV usable for log)
During vgreduce is failed mirror image replaced with error segment,
this segmant type has always area_count == 0.
Current code expects that there is at least one area with device,
patch fixes it by additional check (fixes segfault during vgreduce).
Also do not calculate readahead in every lv_info call, we only need
to cache PV readahead before activation calls which locks memory.
Use lvconvert --repair in dmeventd mirror DSO.
for now.
It replaces bad behaviour in one set of circumstances with bad behaviour
in a different set. We think the behaviour needs to be more configurable.
When we are stacking LV over device, which has for some reason
increased read_ahead (e.g. MD RAID), the read_ahead hint
for libdevmapper is wrong (it is zero).
If the calculated read_ahead hint is zero, patch uses read_ahead of underlying device
(if first segment is PV) when setting DM_READ_AHEAD_MINIMUM_FLAG.
Because we are using dev-cache, it also store this value to cache for future use
(if several LVs are over one PV, BLKRAGET is called only once for underlying device.)
This should fix all the reamining problems with readahead mismatch reported
for DM over MD configurations (and similar cases).
Current code, when need to ensure that volume is not
active on remote node, it need to try to exclusive
activate volume.
Patch adds simple clvmd command which queries all nodes
for lock for given resource.
The lock type is returned in reply in text.
(But code currently uses CR and EX modes only.)
This means two things:
1) Non-mirrored LVs will be no longer affected by mirror monitoring. (Before,
if you had a LV that went partially missing on a VG where a mirror leg failed,
this LV would be removed automatically by dmeventd... Probably not an
unrecoverable dataloss bug, but still quite unpleasant.)
2) If enough parallel PV space is available at the time of the mirror failure,
the failed devices will be automatically replaced using this spare space. Which
(and whether) free space may be used is still not configurable, but is a
planned feature. Since it is relatively easy to undo the action by converting
the mirror manually, I don't consider this to be a showstopper. In fact, I
think the compromise is much better than what we have now.
pvmove now keep suspended devices if temporary mirror creation fails.
We can try to restore previous state if it is first attempt to activate
pvmove (code basically run the same code as --abort automatically).
Currently code uses pv_dev_name() for hash when getting internal
"pvX" name.
This produce corrupted metadata if PVs are missing, pv->dev
is NULL and all these missing devices returns one name
(using "unknown device" for all missing devices as hash key).
We can temporarily violate max_lv during mirror conversion etc.
(If the operation fails, orphan mirror images are visible to administrator
for manual remove for example. Not that this should ever happen:-)
Force limit only for lvcreate (and vg merge) command.
Patch also adds simple max_lv tests into testsuite
The vg->lv_count parameter now includes always number of visible
logical volumes.
Note that virtual snapshot volume (snapshotX) is never visible,
but it is stored in metadata with visible flag.
link_lv_to_vg and unlink_lv_from_vg are the only functions
for adding/removing logical volume from volume group.
Only these function should manipulate with vg->lvs list.
Later patch initializes lv->vg after the LV structure is prepared,
so pass through cmd context and do not use vg->cmd here.
Also move LV id calculation (which uses lv->vg too).
Also properly free memory pool if operation fails.
The snapshot segment (snapshotX) is created twice
during the text metadata segment processing.
This can cause temporary violation of max_lv count.
Simplify the code, snapshot segment is properly initialized
in init_snapshot_seg function now and do not need to be replaced
by vg_add_snapshot call.
The vg_add_snapshot() is now usefull only for adding new
snapshot and it shares the same initialization function.
The snapshot name is always generated, name paramater can be
removed from function call.
As a simplification to the tools and further liblvm, this patch pushes
the setting of NON_BLOCKING lock flag inside the lock_vol() call.
The policy we set is if any existing VGs are currently locked, we
set the NON_BLOCKING flag.
Should be no functional change.
The seg variable is temporary variable for list iterator,
code cannot expect that after iteration it remains NULL
(it contains non-NULL pointer here id list is empty).
Patch fixes first_seg function so it now correctly returns NULL
for empty segment list.
Buildsystem support device-mapper only install,
but generic install tagret includes both dm+lvm2.
For distribution which uses separate install_device-mapper,
there is no way how to install lvm2 only
(so after installing lvm2 for packaging purposes
built system must remove installed device-mapper files).
Fix it by allowing lvm2_install target, similarily like
install_cluster for clvmd.
(install = install_device-mapper + install_lvm2)
The dataalign value must always be aligned according
to MDA area.
The currect code checks if calculated value collides with
MDA area but not if the value is so small that it is
located before MDA starts.
Unfortunatelly there can be also MDA in the end of the device.
The patch adds simple check to avoid this miscalculation.
Patch expects that first MDA always starts on <= pagesize boundary
(this is true for all allowed label sector parameters).
Add lvs origin_size field.
Fix linux configure --enable-debug to exclude -O2.
Still a few rough edges, but hopefully usable now:
lvcreate -s vg1 -L 100M --virtualoriginsize 1T
Run backup of metadata on remote nodes in the
same place like local node - when calling backup().
Introduce backup_locally() which calls only
local backup if needed.
Remote backup is now trigerred by LCK_VG_BACKUP flag
combination (special VG lock).
This lock type will call check_current_backup()
(including backup_locally() call) and updates
metadata on all nodes.
(Patch fixes non-functional remote backup,
current call during VG lock never triggers.)
The backup() call store metadata from memory.
But in cluster backup() call performs
remote nodes metadata backup and it reads data from disk.
For metadata backup consistency,
patch moves all backup() calls after vg_commit.
(Moreover, some tools already do that this way.)
- Rename unlock_all to destroy_lvhash,
this function is called in cluster shutdown
unlocks everything and clean up allocated info space.
- Tidy lv_hash_lock use
.
Except adding free(lvi) in lv_has destructror
there is no functional change.
If user requests report attribute from PVSEG type
and PV is orphan (or all devices is set), the report
is empty.
Try for example (when only orphan PV are present)
#pvs
#pvs -o +devices
# pvs /dev/sdb1
PV VG Fmt Attr PSize PFree
/dev/sdb1 lvm2 -- 46.58G 46.58G
# pvs -o +devices /dev/sdb1
(no output)
The problem is caused by empty pv->segments list.
Fix it by providing fake segment here (similar to fake structures
in _pvsegs_sub_single() calls.
# pvs -a -o devices
Volume group name (null) has invalid characters
Skipping volume group (null)
...
_pvsegs_sub_single creates fake vg, we need to check
that pv is real here.
Since now, all code reading volume group is responsible for releasing
the memory allocated by calling vg_release(vg).
(For simplicity of use, vg_releae can be called for vg == NULL,
the same logic like free(NULL)).
Also providing simple macro for unlocking & releasing in one step,
tools usualy uses this approach.
The global memory pool (cmd->mem) should be used only for global
physical volume operations.
This patch have to be applied with all subsequent patches to complete
memory pool per vg logic.
Using separate memory pool has quite bit memory saving impact when
using large VGs, this is mainly needed when we have to use
preallocated and locked memory (and should not overflow from that
memory space).
The all_pvs list, used in vg_read, should make its own private
copy of pv structures, otherwise (when vg will use its own pool)
it will point to released memory pool.
The same applies for get_pvs() call.
Patch adds pv_list copy helper and adds explicit memory pool
parameter into _copy_pv.
(Please note that all these helper functions cannot guarantee that
vg related fields are valid - proper vg read & lock must be used
if it is requested.)
Currently PV commands, which performs full device scan, repeatly
re-reads PVs and scans for all devices.
This behaviour can lead to OOM for large VG.
This patch allows using internal metadata cache for pvs & pvdisplay,
so the commands scan the PVs only once.
(We have to use VG_GLOBAL otherwise cache is invalidated on every
VG unlock in process_single PV call.)
If the vg in process_each_segment_in_pv is NULL, the pv struct
can be incomplete (for example lv_segs are not copied in get_pvs()
call).
We need use the new pv from just read-in volume group.
(The same code is in pvdisplay already.)
In libdm/Makefile.in, we need to cleanup the symlink properly.
Adding to CLEAN_TARGETS seemed like the simplest way to do this
in the current build framework. We could redo dependencies for
VERSIONED_SHLIB, but for now just add to CLEAN_TARGETS.
For scripts/Makefile.in, we should be adding to DISTCLEAN_TARGETS.
The generic rule in make.tmpl.in takes care of the cleanup.
Signed-off-by: Dave Wysochanski <dwysocha@redhat.com>
Author: Takahiro Yasui <tyasui@redhat.com>
By gnu coding stds, 'distclean' should remove all files generated
by ./configure in addition to what 'clean' does.
Author: Takahiro Yasui <tyasui@redhat.com>
Patch fixes these problems:
- during the snapshot creation process, it needs create 2 LVs,
one is cow, second becomes snapshot.
If the code fails in vg_add_snapshot, code lvcreate will not remove
LV cow volume.
- if max_lv is set and VG contains snapshot, it can happen that
during the activation lv_count is temporarily increased over the limit
and VG metadata are not properly processed
see https://bugzilla.redhat.com/show_bug.cgi?id=490298
- vgcfgrestore alows restore with max_lv set to lower valuer that actual
LV count. This later leads to situation that max_lv is completely ignored.
- vgck doesn't call vg_validate(). It should at least try:-)
Signed-off-by: Milan Broz <mbroz@redhat.com>
We would like to declare our handles pv_t, vg_t, and lv_t in
the external library header lvm.h. However, these are already
defined in metadata-exported.h for the use of some of the
in-progress liblvm APIs. Thus, we cannot both define
them in lvm.h and include metadata-exported.h in the external
library C files. We could use preprocessor tricks (#ifndef)
but for now we just avoid the include.
The original liblvm.a has been moved to liblvm-internal.a.
We now use liblvm.a for the new application library and build
it inside liblvm directory.
Change dependencies so tools depend on liblvm application library,
and application library depends on liblvm internal.
(for example when CLVMD_CMD_LOCK_VG for is called during vgscan).
If clvmd calls LV lock, it calls
/* clean the pool for another command */
dm_pool_empty(cmd->mem);
to clean up memory pool after command.
Unfortunately, do_refresh_cache() do not call this
and because during it operation it allocates some memory,
the pool increases.
Also do_refresh_cache should use lvm_lock
(it manipulates with lvm internal data).
The same applies for lvm_backup command.
Signed-off-by: Milan Broz <mbroz@redhat.com>
if rlocn not defined (there is no metadata area).
In most cases it fails in validate_name(),
unfortunately there are situatuions, when
validate_name is ok and later code fails with
checksum error.
Reproducer:
# dd if=/dev/zero of=/dev/loop0
# pvcreate --metadatasize 637k /dev/loop0
Physical volume "/dev/loop0" successfully created
# pvs /dev/loop0
/dev/loop0: Checksum error
PV VG Fmt Attr PSize PFree
/dev/loop0 lvm2 -- 1.00M 1.00M
Signed-off-by: Milan Broz <mbroz@redhat.com>
-
Using argv[] list in exec_cmd() to allow more params for external commands.
Fsadm does not allow checking mounted filesystem.
Fsadm no longer accepts 'any other key' as 'no' answer to y/n.
Fsadm improved handling of command line options.
New structure lvm (used as an alias to cmd_context), new type definition lvm_t
for the lvm handle. Added functions lvm_create, lvm_destroy and
lvm_reload_config using the new handle.
Modified test/api/test.c:
Use new lvm.h header file and lvm_t handle.
Removed lib/lvm2.h
Author: Thomas Woerner <twoerner@redhat.com>
This patch is not fully tested and leaves some related bugs unfixed.
Intended behaviour of the code now:
pe_start in the lvm2 format PV label header is set only by pvcreate (or
vgconvert -M2) and then preserved in *all* operations thereafter.
In some specialist cases, after the PV is added to a VG, the pe_start
field in the VG metadata may hold a different value and if so, it
overrides the other one for as long as the PV is in such a VG.
Currently, the field storing the size of the data area in the PV label
header always holds 0. As it only has meaning in the context of a
volume group, it is calculated whenever the PV is added to a VG (and can
be derived from extent_size and pe_count in the VG metadata).
When reporting explicitly label attributes (pv_uuid for example), we do not
need to read metadata.
This patch separate the label fileds and removes scan_vgs_for_pvs
in process_each_pv() if not needed.
(There should be no user visible change in output.)
In libdm, we only ever use 'fields', while the tools use 'options' and
'fields' interchangeably.
Ideally it would be good to use 'fields' consistently everywhere.
However, 'options' most likely comes from the tool commandline '-o' and
'--options' which cannot be changed.
We display '0' for these fields now in this case. Ideally these values are
undefined for an orphan PV but today there is no way to specify undefined
with display functions such as _size64_disp().
"test" never prints anything. Therefore, "return $(test ...)" is equivalent to
just "return;" which means success in sh (same as return 0). We can however,
thanks to set -e, use "test foo = bar" as an assertion.
PS: test a == b is invalid syntax. It is either = or -eq: = is textual and -eq
is numeric comparison.
For example in LVM2, "pv_all" gives all PV fields.
"seg_all" gives all LV segment fields.
"all" gives all fields of the final report type. I think this is more
useful than just adding the current prefix.
So "lvs -o seg_all" gives all the LV segment fields, whilst
"lvs --segments -o all" adds in LV and VG fields too.
"lvs -o all -O vg_name" has report type LVS+VGS so includes all LV and all
VG fields.
Reports the size of the smallest metadata area in a PV or a VG.
Useful to confirm pvcreate --metadatasize or pvmetadatasize setting in
/etc/lvm/lvm.conf file.
NOTE: Actual value in these fields will most always differ from that
given in pvcreate options due to rounding and alignment effects.
There is a rudimentary make file in place so people can build by hand
from 'LVM2/daemons/clogd'. It is not hooked into the main build system
yet. I am checking this in to provide people better access to the
source code.
There is still work to be done to make better use of existing code in
the LVM repository. (list.h could be removed in favor of existing list
implementations, for example. Logging might also be removed in favor
of what is already in the tree.)
I will probably defer updating WHATS_NEW_DM until this code is linked
into the main build system (unless otherwise instructed).
Checks added for DM device names to allow only names < DM_NAME_LEN,
otherwise a part of lengthy name would be silently ignored and could
cause confusion while using dmsetup. Also, the name should not contain
'/' character, if it is used in context of creating a new device
or renaming the existing one (because we do not consider full path
to devices, they do not exist in filesystem yet) and appropriate error
messages are shown.
pvcreate $DEV
vgcreate -s 1k vg_test $DEV
lvcreate -l 1 -n lv1 vg_test
..
/dev/vg_test/lv1: write failed after 1024 of 4096 at 0: No space left on device
Just check for maximum write size in set_lv.
It fails for 1k PE now.
Patch adds log_region_size into allocation habdle struct
and use it in _alloc_parallel_area() for proper log size calculation
instead of hardcoded 1 extent - which can fail.
Reproducer for incorrect log size calculation:
DEV=/dev/sd[bcd]
pvcreate $DEV
vgcreate -s 1k vg_test $DEV
lvcreate -m1 -L 12M -n mirr vg_test
https://bugzilla.redhat.com/show_bug.cgi?id=477040
The log size calculation is mostly copied from kernel code.
Check for major/minor collision is added in _add_dev_to_dtree()
where we already read info by uuid,
so in the case of requesting major/minor it queries device-mapper
by major/minor for device availability.
Fixes https://bugzilla.redhat.com/show_bug.cgi?id=204992
Very simple / crude method of removing 'is_static' from initialization.
Why should we require an application tell us whether it is linked
statically or dynamically to libLVM? If the application is linked
statically, but libraries exist and dlopen() calls succeed, why
do we care if it's statically linked?
This allows us to remove one argument from create_toolcontext() and
moves it closer to a generic library init function.
In the arg_*() functions, we just use _the_args() directly.
For now we leave the first parameter to these
arg_*() functions (struct cmd_context *) because
of the number of files involved in removing the
parameter.
In preparation for removing cmd->args.
IMO, it makes more sense to put these accessor functions
in the same location as the static array _the_args.
Next patch will update arg_* functions to use _the_args[]
directly and remove cmd->args.
Problem is dm_report_init() may return NULL and subsequent call to
dm_report_set_output_field_name_prefix() doesn't handle NULL value.
Example:
pvs --nameprefixes --rows --unquoted --noheadings -opv_name,fred
Logical Volume Fields
---------------------
lv_uuid - Unique identifier
lv_name - Name. LVs created for internal use are enclosed in brackets.
...
Physical Volume Segment Fields
------------------------------
pvseg_start - Physical Extent number of start of segment.
pvseg_size - Number of extents in segment.
Unrecognised field: fred
Segmentation fault
Move init_full_scan_done(0) and init_mirror_in_sync(0) from init_lvm()
after call to create_toolcontext() to _init_globals(), called from bottom
of create_toolcontext(). No functional change.
Author: Dave Wysochanski <dwysocha@redhat.com>
Acked-by: James Cameron <james.cameron@hp.com>
Acked-by: Alasdair G Kergon <agk@redhat.com>
init_formats() sets up the command formats, and currently sets cmd->fmt_backup
but does not set cmd->fmt to a default value. This seems incorrect so we
set it to cmd->default_settings.fmt before returning.
The call we remove here may set cmd->fmt based on a command line setting.
But it is safe to remove this, because the only caller of init_lvm() that
cares about the cmdline override is the cmdline tools (clvmd does not care),
called from lvm2_main(). After lvm2_main() calls init_lvm(), it later calls
lvm_run_command(). In lvm_run_command(), we have a call to _apply_settings(),
which has the identical assignment of cmd->fmt that this patch removes.
This is very obvious - _init_logging() makes the identical init_msg_prefix()
and init_cmd_name() calls with cmd->default_settings so these calls are
clearly redundant after calling create_toolcontext().
Very similar argument to removal of init_debug() and other calls.
create_toolcontext() calls _process_config() which sets
cmd->default_settings.activation, then calls
set_activation(cmd->default_settings.activation). Later, create_toolcontext()
sets cmd->current_settings = cmd->default_settings. So these calls
set_activation(cmd->current_settings.activation) are redundant.
Identical argument to previous patch which removed archive_enable() calls.
We add a new parameter to backup_init() which sets the enable value based
on the cmd->default_settings.backup value. This value was used to set
cmd->current_settings.backup, used in the removed backup_enable() call.
_init_backup() calls archive_init(), which originally set 'enabled' to
a hardcoded '1' value. This seems incorrect based on my read of other
areas of the code so here we add a 'enabled' paramter to archive_init().
We pass in cmd->default_settings.archive, which is obtained from the
config tree. Later in create_toolcontext, cmd->current_settings is
set to cmd->default_settings. The archive_enable() call we remove
here was using cmd->current_settings to set the 'archive' enable
value. The final value of cmd->archive_params->enabled should thus
be equivalent to the original code.
This one we actually need to move. _init_logging() is called from
create_toolcontext(), which makes this call:
/* Test mode */
cmd->default_settings.test =
find_config_tree_int(cmd, "global/test", 0);
But it does not call init_test(). So we need an init_test() somewhere.
The most logical place is to put it inside _init_logging(), since this
is where the config value is read and default_settings are set. Placing
the init_test() call here matches what is done with other variables and
seems to make sense.
This variable is set at the top of create_toolcontext() to 0.
Nothing later in create_toolcontext() changes the value.
In init_lvm(), nothing between create_toolcontext() call and this assignment
changes the value. Thus, the assignment is redundant.
The rationale for removing init_verbose() call is very similar to removing
init_debug() call. create_toolcontext() calls _init_logging() which
makes these calls:
/* Verbose level for tty output */
cmd->default_settings.verbose =
find_config_tree_int(cmd, "log/verbose", DEFAULT_VERBOSE);
init_verbose(cmd->default_settings.verbose + VERBOSE_BASE_LEVEL);
And being that create_toolcontext() copies default_settings into
current_settings at the bottom, the init_verbose() call we are removing:
init_verbose(cmd->current_settings.verbose + VERBOSE_BASE_LEVEL);
is redundant.
We can safely remove because create_toolcontext() calls _init_logging(),
which makes these calls:
/* Debug level for log file output */
cmd->default_settings.debug =
find_config_tree_int(cmd, "log/level", DEFAULT_LOGLEVEL);
init_debug(cmd->default_settings.debug);
Then at the bottom of create_toolcontext() we do this:
cmd->current_settings = cmd->default_settings;
So the call we are removing from init_lvm() functions (clvmd and lvmcmdline):
init_debug(cmd->current_settings.debug);
Just sets the value of debug based on 'cmd->current_settings.debug'.
Since cmd->current_settings is equivalent to cmd->default_settings, and
init_debug() was called with cmd->default_settings, the call we remove is
redundant.
for losetup, break out of the loop when successful setup of loop device,
and only look at 7 loop devices (default loop module setting)
for blockdev, use old option if new one is not available
with the second vgcreate overwriting the first.
Obtain lock before calling vg_create(), which checks for existence of vgname
and fails if it already exists.
Prior to this patch, "lvremove -f vgname" would fail if vgname contained
one or more snapshot LVs. Now this passes, but has a side-effect.
If you issue "lvremove vgname" where vgname contains one or more snaps,
you will get an extra "y/n" prompt to remove the same snapshot.
Example:
$ lvs
LV VG Attr LSize Origin Snap% Move Log Copy% Convert
lvsnap vgtest swi-a- 16.00M lvtest 0.05
lvtest vgtest owi-a- 64.00M
$ lvremove vgtest
Do you really want to remove active logical volume "lvsnap"? [y/n]: n
Logical volume "lvsnap" not removed
Do you really want to remove active logical volume "lvsnap"? [y/n]: n
Logical volume "lvsnap" not removed
Command failed with status code 5.
Fixing this will most likely require modification of the iterator
function, process_each_lvs_in_vg() to iterate over snaps in some
cases (e.g. lvs, vgdisplay -v) but not in others (lvremove).
(Avoids having same mirror table loaded twice concurrently by first
using a 'zero' table to set the size of the device so when mirror
table is preloaded it doesn't have to be activated immediately.)
snapshot DSO unregistered itself when snapshot changed state to invalid.
This can cause a race (and several timeouts), when for example another snapshot
is added and in the middle of operation (suspend/resume) the monitoring thread
unregister itself.
Fix it by keeping the snapshot monitored after invalidation - just reset
threshold to not really print any messages to syslog.
If the PV has two metadata areas, second one is located at the end of the device.
Do not allow resize of PV or second metadata area can be overwritten.
(The check was active only for orphan PVs.)
Fixes problem when after downconvert to lvm1 VG is broken:
# lvcreate -n lv1 -l 4 vg_test
Invalid LV in extent map (PV /dev/sdb1, PE 0, LV 0, LE 0)
...
- add lvcreate rejects repeated invocation test
- fix pvs metadata test for partial failure test
- add pvchange reject --addtag to lvm1 pv test
(All fixes by Jaroslav Stava)
Original code would create "*.so" symbolic links if there were no actual
files ending in "so". The second iteration would then cause an error
in the test logs.
Function _text_pv_write doesn't use memory pool but static buffer,
call dm_pool_free in error path in _raw_write_mda_header is wrong.
Move pool free only to path where is the memory pool used.
Do not override the default action of AC_CHECK_LIB([readline],...
(i.e., leave the ACTION-IF-FOUND parameter blank) so that the
subsequent check for rl_completion_matches can use -lreadline.
Also, replace AC_CHECK_FUNC+AC_DEFINE with an equivalent AC_CHECK_FUNCS call.
Failure to check for label_write() return code caused the following test
to indicate it passed when it really failed:
pvcreate rejects labelsector > 1000000000000
The "status" field is treated as it ever has been, unknown flags there are
treated as fatal metadata errors. However, in the "flags" field, any unknown
flags will be ignored and silently dropped. This improves
backward-compatibility possibilities. (Any versions without support for this
new "flag" field will drop the field altogether, which is same as ignoring all
the flags there.)
failed to link against liblvm2cmd.
Dmeventd DSOs *require* lvm2cmd to be linked in.
For the future:
1) AC_SUBST does not create Makefile variables, only @foo@-style substitutions
2) When using `test', whitespace around `=' is essential:
test a=b is true, as is test a=a
The warning is bogus and is only seen on certain versions of gcc.
However using the enum does seem to clarify the intent of the code - only
3 possible md minor superblock versions.
Related compiler warning:
device/dev-md.c:53: warning: 'sb_offset' may be used uninitialized in this function
* configure.in (LVM2CMD_LIB): Define if --enable-cmdlib.
* dmeventd/mirror/Makefile.in (CLDFLAGS): Use $(LVM2CMD_LIB) rather
than hard-coding -llvm2cmd.
* dmeventd/snapshot/Makefile.in (CLDFLAGS): Likewise.
* configure.in: Define READLINE_SUPPORT not when processing
--enable-readline or --disable-readline, but rather only after
determining that readline support is desired and the readline
library is available/usable.
Related compiler warning:
log/log.c:242: warning: declaration of 'error_message_produced' shadows a global declaration
../include/log.h:98: warning: shadowed declaration is here
The new error checking code caught some commands that were returning '0' as
an exit status for success. This is incorrect and resulted in a benign error
message displayed (see below). As of today, all commands should return a
value defined in lib/commands/errors.h (1-5). This results in an exit code of
0 on success, or > 0 on failure (as stated in the lvm.8 man page).
Before change:
1. Make sure no PVs are on the system
2. Run 'pvs'
Command failed with status code 0.
After change:
<no output>
Specific test case:
1. pvcreate /dev/loop1; vgcreate vg1 /dev/loop1; lvcreate -L 64M -n lv1 vg1
2. vgremove vg1 (will prompt user)
3. CTRL-C
Code will exit with:
Do you really want to remove volume group "vg2" containing 2 logical volumes? [y/n]:
Volume group "vg2" not removed
Command failed with status code 5.
Internal error: Volume Group vg2 was not unlocked
Device '/dev/loop1' has been left open.
After change:
Do you really want to remove volume group "vg2" containing 2 logical volumes? [y/n]:
Volume group "vg2" not removed
Command failed with status code 5.
which is not used since the switch away from async version saLck
. num_nodes should equal to member_list_entries, i.e.
joined_list_entires is 0 when a node leaves the group.
Thanks to Xinwei Hu for the patch.
It does 2 things.
1. The cpg_deliver_callback make a compare between target_nodeid and our_nodeid.
It turns out openais set target_nodeid to 0 sometimes. for broadcasting ? I change the behavior so that lvm will process_remote also on target_nodeid == 0
2. The joined_list passed to cpg_confchg_callback doesn't include the already exist nodes in the group, which leads to an incomplete node_hash. I simply add all other nodes in member_list to node_hash also.
Thanks to Xinwei Hu for this patch.
Handles non-clustered as well as clustered. For clustered,
the best we can do is try exclusive local activation. If this
succeeds, we know it is not active elsewhere in the cluster.
Otherwise, we assume it is active elsewhere.
This bug has been around for a long time as far as I can tell.
Without this fix, a vgsplit would unconditionally move the
'hidden/internal' snapshot LVs, and result in corrupted metadata
in the following case:
vg1: contains lv1, lv1snap, both on pvset1
vg1: contains lv2, on pvset2
"vgsplit vg1 vg2 pvset2"
would result in "snapshot0" hidden LV being moved to vg2, and
the origin and cow being left in vg1. The tools detect the
corruption in vg2, but not in vg1.
When vg_lock_and_read() calls were added, they were done so incorrectly for
the destination VG (vg_to). This resulted in the VG lock not obtained when
a new VG was the destination (vg_lock_and_read() would fail in the vg_read()
clause, which would then release the lock before returning NULL), and could
result in corrupted destination VG.
The fix was to put back the original lock_vol() and vg_read() calls for 'vg_to'.
The failure of vg_read() indicates "vg does not exist", and we key off that
to determine whether we are dealing with a new or existing VG as the
destination.
The first two error messages were also the result of the incorrect
vg_lock_and_read() calls:
Volume group "new" not found
cluster request failed: Invalid argument
New volume group "new" successfully split from "vg"
Fixes https://bugzilla.redhat.com/show_bug.cgi?id=438249
BEFORE:
tools/lvm lvresize -l +4 vg22/lv1linear
Volume group "vg22" not found
Volume group vg22 doesn't exist
AFTER:
tools/lvm lvresize -l +4 vg22/lv1linear
Volume group "vg22" not found
- Add validation on pv_count, lv_count, and snap_count after split
NOTE: Some of these counts are misleading. If you compare "lvs" output
with these counts you will be left scratching your head what a "logical volume"
really is. ;-)
Fix unfilled paramater passed to fsadm from lvresize
Update fsadm to call lvresize if the partition size differs (with option -l)
Fix fsadm to support vg/lv name (like the rest of lv-tools)
Fix 'make check' to use DMDIR to check DM_DEV_DIR support in dmsetup.
Add basic test cases for mirrored LV.
Add basic test cases for lvconvert mirror.
Add basic test cases for pvmove.
Jun'ichi Nomura <j-nomura@ce.jp.nec.com>
Add new vgsplit and vgmerge tests.
Dave Wysochanski <dwysocha@redhat.com>
* test/t-000-basic.sh: Invoke initial test of lvm with its "version"
argument, so that the behavior of the tool doesn't depend on whether
readline was enabled at configure time.
Author: Jun'ichi Nomura <j-nomura@ce.jp.nec.com>
Committer: Jim Meyering <meyering@redhat.com>
Fix missing VG unlocks in some pvchange error paths.
Add some missing validation of VG names.
Rename validate_vg_name() to validate_new_vg_name().
Change orphan lock to VG_ORPHANS.
Change format1 to use ORPHAN as orphan VG name.
This makes the tests more reproducible and helps isolate
them from any existing LVM set-up.
* test/Makefile.in (abs_builddir): Define.
(init.sh): Emit definition of abs_builddir.
* test/lvm-utils.sh (unsafe_losetup_): Keep only the portable,
iterative approach.
(dmsetup_has_dm_devdir_support_): New function.
(init_root_dir_): New function.
Invoke init_root_dir_ for all but the first test.
* test/test-lib.sh (this_test_): Adapt to test-name change.
Invoke lvm-utils.sh much later (after tmpdir creation), and
only if the current test is not being skipped.
Remove useless abs_top_srcdir definition.
Rename t0->test_dir_rand_.
* test/t-lvcreate-pvtags.sh: Skip this test if the available
version of dmsetup is not new enough.
Use global, $G_dev_, rather than hard-coded "/dev".
* test/t-lvcreate-usage.sh: Make --verbose output more useful.
Author: Jim Meyering <jim@meyering.net>
Committer: Jim Meyering <meyering@redhat.com>
* dmsetup/dmsetup.c (DEV_PATH): Remove definition.
(parse_loop_device_name): Add parameter: dev_dir.
Declare the "dev" parameter to be "const".
Use dev_dir, not DEV_PATH. Handle the case in which dev_dir
does not end in a "/".
(_get_abspath): Declare "path" parameter "const", to match.
(_process_losetup_switches): Add parameter: dev_dir.
Pass dev_dir to parse_loop_device_name.
(_process_switches): Add parameter: dev_dir.
Pass dev_dir to _process_losetup_switches.
(main): Set dev_dir from the DM_DEV_DIR envvar, else to "/dev".
Call dm_set_dev_dir.
* lib/libdm-common.c (dm_set_dev_dir): Rewrite to be careful
about boundary conditions, now that dev_dir may be tainted.
* man/dmsetup.8: Mention $DM_DEV_DIR.
Author: Jim Meyering <meyering@redhat.com>
DMSETUP=: to disable dmsetup checks (but let the script run
nevertheless); warn the user if this is the case
b) put the non-root and dmsetup warnings both at start and end of
output
Print just one line:
Use `COMMAND --help' for more information.
after "real" diagnostic(s), rather than all of the usage lines.
Otherwise, the 30-40+ lines of --help output could obscure the real diagnostic.
Author: Jim Meyering <jim@meyering.net>
* test/Makefile.in (srcdir, top_srcdir): Use @srcdir@, etc.
(top_builddir, abs_srcdir, abs_top_builddir, abs_top_srcdir): Likewise.
(so_name): Remove definition.
(.bin-dir-stamp): No longer create symlink in $(DMDIR) tree.
Prompted by suggestions from Alasdair Kergon.
* test/t1000-lvcreate-usage.sh (cleanup_): Redirect to a file,
rather than to /dev/null.
Change wording of some test titles.
Suggestions from Alasdair Kergon.
* test/Makefile.in: Add a copyright notice.
* test/lvm-utils.sh: Likewise.
* test/mkdtemp: Likewise.
* test/t0000-basic.sh: Likewise.
* test/t1000-lvcreate-usage.sh: Likewise.
* test/t3000-lvcreate-pvtags.sh: Likewise.
* test/t4000-pv-range-overflow.sh: Likewise.
* test/test-lib.sh: Likewise.
Author: Jim Meyering <jim@meyering.net>
* test/t1000-lvcreate-usage.sh: New tests.
* test/Makefile.in (T): Add it.
Derived from test cases by Dave Wysochanski.
Author: Jim Meyering <jim@meyering.net>
* test/Makefile.in (so_name): Use @DMDIR@.
(.bin-dir-stamp): Create symlink only if @DMDIR@ is nonempty.
(lvm-wrapper): Emit LD_LIBRARY_PATH setting only if @DMDIR@ is nonempty.
Based on a patch from Jun'ichi Nomura.
Author: Jim Meyering <jim@meyering.net>
* configure.in: Convert a relative dmdir directory name to the required
absolute form, e.g. in ./configure --with-dmdir=../device-mapper
Suggestion from Jun'ichi Nomura.
* configure: Regenerate.
Author: Jim Meyering <jim@meyering.net>
* Makefile.in (check): New target.
* configure.in (AC_CONFIG_FILES): Add test/Makefile.
* configure: Regenerate.
* test/.gitignore: New file.
* test/Makefile.in: New file.
* test/lvm-utils.sh: New script.
* test/mkdtemp (die, rand_bytes, mkdtemp): New script.
* test/t0000-basic.sh: New tests.
* test/t3000-lvcreate-pvtags.sh: New, failing test.
Derived from a script by Jun'ichi Nomura.
* test/t4000-pv-range-overflow.sh: New test.
* test/test-lib.sh: Testing framework, based on the one from git.
Author: Jim Meyering <jim@meyering.net>
* tools/toollib.c (xstrtouint32): New function.
(_parse_pes): Use xstrtouint32; don't cast strtoul's unsigned
long to uint32_t. Detect overflow.
Author: Jim Meyering <jim@meyering.net>
* lib/device/dev-io.c (dev_open_flags):
Use log_sys_error after failed stat to report strerror(errno).
Use a slightly different diagnostic to report mismatched device number.
(MIRROR_NOTSYNCED) is added to the LVM metadata. This flag is
not cleared when converting to linear. Subsequently, if you
up-convert the linear to a mirror, the flag remains - even though
an up-convert will always force a complete resync.
Move yes_no_prompt() into library (display.c, display.h).
Fixup includes as a result of movement of prior two functions.
Fixup force_t enum to be more descriptive.
log type. Previously, we had a '--corelog' argument that
would change the default type from 'disk' to 'core'. I
think that creates too much confusion - especially when
doing conversions on mirrors.
The new argument '--log' takes either "disk" or "core"
as a parameter. This could be expanded in the future
for additional logging types as well.
Examples:
# Creating a 2-way mirror
$> lvcreate -m1 ... # implicitly use default disk logging
$> lvcreate -m1 --log disk ... # explicit disk logging
$> lvcreate -m1 --log core ... # specify core logging
$> lvcreate -m1 --corelog ... # old way still works
# Conversion examples
$> lvconvert --log core ... # convert to core logging
$> lvconvert --log disk ... # convert to disk logging
$> lvconvert -mX --corelog ... # old way still works
$> lvconvert -mX ... # old way of converting to disk logging still works
Changes are reflected in the man pages.
* lib/misc/lvm-file.c (lvm_fclose): New function.
* lib/misc/lvm-file.h (lvm_fclose): Declare it.
* lib/config/config.c (write_config_file): Use the new function to detect
and diagnose unlikely write failure.
* lib/filters/filter-persistent.c (persistent_filter_dump): Likewise.
* lib/format_text/archive.c (archive_vg): Likewise.
* lib/format_text/format-text.c (_vg_write_file): Likewise.
* lib/log/log.c (fin_log): Similar, but use dm_fclose directly.
Include "\n" at end of each fprintf format string.
* dmeventd/dmeventd.c (_set_oom_adj): When writing to /proc/self/oom_adj,
detect failure even if it's hidden behind ferror. [Using dm_fclose's
extra ferror test here is probably not needed, since the amount written
is nowhere near BUFSIZ, but use it regardless, for consistency. ]
* lib/fs/libdevmapper.c (do_suspend): Detect fclose failure when
writing to suspend.
* lib/misc/lvm-file.h (is_same_inode): Define.
* lib/filters/filter-persistent.c (persistent_filter_dump): Use is_same_inode
in place of a direct st_ino-only comparison.
* lib/locking/file_locking.c (_release_lock, _lock_file): Likewise.
config/config.c:493: warning: format '%d' expects type 'int', but argument 5 has type 'long int'
Modified original patch from Jim Meyering <jim@meyering.net>
There are two fixes other than improving variable names and updating code
layout etc.
The loop counter is incremented by area_len instead of area_len * stripes;
the 3rd _check_stripe parameter is no longer multiplied by number of stripes.
Use loop to iterate through the now-ordered policy list in _allocate().
Check for failure to allocate just the mirror log.
Introduce calc_area_multiple().
Support mirror log allocation when there is only one PV: area_count now 0.
(See lvm-devel list archives for further details.)
e.g. lvcreate -l 100%FREE to create an LV using all available space.
lvextend -l 50%LV to increase an LV by 50% of its existing size.
lvcreate -l 20%VG to create an LV using 20% of the total VG size.
Include mirror log (untested) in _for_each_pv() processing.
Use MIRROR_LOG_SIZE constant.
Remove struct seg_pvs from _for_each_pv() for generalisation.
Avoid adding duplicates to list of parallel PVs to avoid.
event-driven model. Without changes to the way the cache gets updated, the
option is currently unreliable without a global lock to prevent any lvm2
commands from running concurrently.
Add --config for overriding most config file settings from cmdline.
Quote arguments when printing command line.
Remove linefeed from 'initialising logging' message.
Add 'Completed' debug message.
Don't attempt library exit after reloading config files.
Always compile with libdevmapper, even if device-mapper is disabled.
Fix some memory leaks in error paths found by coverity.
Use C99 struct initialisers.
Move DEFS into configure.h.
Clean-ups to remove miscellaneous compiler warnings.
[Some activation-related features will stop working for a while now.
Some types of activation are getting split into two steps, with the
first step using the precommitted metadata.]
The daemon side of this is mostly the same as the patch I sent out. To select
a timeout period different than the default and to get the timeout period,
I added two library calls, dm_set_event_timeout() and dm_get_event_timeout().
If people are against them, the other option is to tack extra arguments onto
dm_regiser_for_event() and dm_get_registered_device(). I also added a
-t option to dmevent, so people can try out timeouts.
log types. This means the threaded_syslog type is no longer valid. A new
fxn multilog_async is available to toggle between the two modes. If an
app is compiled without pthreads and tries to use async logging, no logging
will occur while async is enabled.
dmeventd has been modified to use the new code
I'm not positive I like the way the async_logger code calls the log fxn,
but it works for now. Suggestions for other ways to do it would be helpful
- don't echo after an 'action' call - action does the echo itself
- use vgdisplay/vgs to determine which VGs are marked clustered and only
deactivate those VGs (unless the LVM_VGS var is set in
/etc/sysconfig/cluster)
- multilog_add_type, multilog_del_type, multilog_custom, and
multilog_init_verbose all have different arguments.
- Primary change is that caller only passes in config info, and the
lib keeps track of state internally. No more exporting of
'struct log_data'.
- Custom callers now only get the custom data pointer passed into their
log fxn (that is set with multilog_custom)
- Added basic README that describes libmultilog
# dmevent -l
Also, changed the behaviour of dm_get_registered_device(), so that it doesn't
change the pointer you passed in without freeing the memory on a non-next call,
and doesn't free your pointer without setting it to NULL on a failed next call.
multilog_add_type()/multilog_del_type cycles correctly.
o fixed segfault in multilog_add_type()
o fixed test-multilog.c
o cleaned up libmultilog (list macros, indentation, braces, comments)
- add event_nr to thread_status struct and set appropriately so that the
thread actually waits for an event
- essentially make error_detected return true. Let the DSOs determine
how to interpret the status info
o more tweaks to libmultilog calls - the api isn't set in stone yet, so
don't get too comfortable.
o not sure the dmeventd in device-mapper/dmeventd works - i've been using
the one in lib/event/
o currently both daemons are set to log only to syslog
o changed
int dm_get_next_registered_device(char **dso_name, char **device,
enum event_type *events);
to
int dm_get_registered_device(char **dso_name, char **device,
enum event_type *events, int next)
so that the daemon is able to retrive the next one of the list without
running into locking issues.
o changed dmevent.c to use dm_get_registered_device()
o couldn't test this yet because of the comms issues
(daemon exits in do_process_request())
Improve reporting of node-specific locking errors so you'll get
somthing a little more helpfiul than "host is down" - it will now tell
you /which/ host it thinks is down.
./configure --with-clvmd
wil do this by default. Or you can choose which you want with
./configure --with-clvmd=gulm or
./configure --with-clvmd=cman
When clvmd with both included is run, it will automatically detect the cluster
manager in use.
Additional verbosity level -vvvv includes line numbers and backtraces.
Verbose messages now go to stderr not stdout.
Close any stray file descriptors before starting.
Refine partitionable checks for certain device types.
Allow devices/types to override built-ins.
and further reduce the number of ioctl calls made.
o Metadata area struct change.
o Make config file accessible to activation functions & get stripe_filler
from it.
o Allow kernel to return snapshot status as a fraction or a percentage.
checks clearer (incl. variable renaming); using a flag to indicate when
output data doesn't fit into supplied buffer instead of returning an error etc.
Clear many compiler warnings (i386) & associated bugs - hopefully without
introducing too many new bugs:-) (Same exercise required for other archs.)
Default compilation has optimisation - or else use ./configure --enable-debug
to back this out so you can do that commit, let me know. Also, if there's
an issue with the error message that's displayed, just change it in tools.h.
This causes a "device-mapper driver/module not loaded?" error message to
be displayed for the commands that require dm-mod, if the tools can't get
the driver version. It's not done for commands that don't require dm-mod.
This should clear up some problems people have had attempting to use lvm2
without rtfm'ing.
allocation policy. This can currently take one of three values:
typedef enum {
ALLOC_NEXT_FREE,
ALLOC_STRICT,
ALLOC_CONTIGUOUS
} alloc_policy_t;
Notice that 'SIMPLE' has turned into the slightly more meaningful NEXT_FREE.
ii) Put code into display.[hc] for converting one of these enums to a
text representation and back again.
ii) Updated the text format so this also has the alloc_policy field.
o Various other kernel side tidy-ups.
o Version number changes so we have the option of adding new ioctl commands
in future without affecting the use of existing ones should you later
revert to an older kernel but not revert the userspace library/tools.
o Better separation of kernel/userspace elements in the build process to
prepare for independent distribution of the kernel driver.
preparsed status info, shove it all into a string, and then parse it
again to get the info back out (which is what i was doing before)
o basically that's it...i like this *much* better than the previous
method and i think it makes the _status fxn more flexible if we need
to use it to get other info out.
o Not sure if the code in dev_manager is really optimal, but it works..
will look at adjusting it a bit now.
o I *think* it works right when one snapshot if full but others aren't,
but I haven't really been able to test it because the full snapshot
somehow resets itself and weird things start happening to the system...
o There is still a bit missing
+ all are missing the {PV,VG,LV} # - that is not applicable in LVM2
+ pvdisplay doesn't show how many LVs are contained on it
+ much of the snapshot information isn't available for lvdisplay
o Look at the code for other potiential FIXMEs :)
Lots of changes/very little testing so far => there'll be bugs!
Use 'vgcreate -M text' to create a volume group with its metadata stored
in text files. Text format metadata changes should be reasonably atomic,
with a (basic) automatic recovery mechanism if the system crashes while a
change is in progress.
Add a metadata section to lvm.conf to specify multiple directories if
you want (recommended) to keep multiple copies of the metadata (eg on
different filesystems).
e.g. metadata {
dirs = ["/etc/lvm/metadata1","/usr/local/lvm/metadata2"]
}
Plenty of refinements still in the pipeline.
Patrick, can you see if this fixes your cluster syncing problem please ?
If so I'll make it so it only syncs if you have actually written to the
device.
from lock_vol() - otherwise it now attempts to acquire the lock and then
immediately releases it.
o Extend the id field in struct logical_volume to hold VG uuid + LV uuid
for format1. This unique lvid can be used directly when calling lock_vol().
o Add the VG uuid to vgcache to make VG uuid lookups possible. (Another
step towards using them instead of VG names internally.)
o #defines for common lock flag combinations
o Try out hyphens instead of colons in device-mapper names - does this
make messages containing filenames easier to read?
o rewrote activate.c to use dev-manager, I'm sure these two will merge
at some point.
o Rename is broken ATM
o dev-manager puts the calls through to fs.c for layers that have the
'visible' flag set.
o Use first unused lv_number when creating new LV
o Use lv_number for refs to snapshots
o Update persistent minor logic after the lvcreate restructure
o Reset all parameters before use in lvcreate.
and add severe warning if it's used to make a device seem bigger than
it really is. This is not an option people should be using as it
breaks metadata integrity.
o Use uint64_t throughout (rather than unsigned long long)
o Convert a few messages that contain pathnames into the more common form:
pathname: message
I'm taking a different route from LVM1 here in that snapshots are a
seperate entity from the logical volumes, I think of them as an
application of an LV (or two lvs rather). As such there is a list of
snapshots held against the vg, and there is *not* a SNAPSHOT, or
SHAPSHOT_ORG flag in lv->status.
all since it only supports vg_write. It has been replaced with:
int archive_vg(struct volume_group *vg,
const char *dir,
const char *desc,
uint32_t retain_days,
uint32_t min_archive);
which is now called directly by tools/archive.c
o Text format now has a description and time field at the top level.
o archiving and backup set the description appropriately. eg,
for an archive:
description = "Created *before* executing 'lvextend test_vg/lvol0 -l +1'."
creation_time = 1013166332
for a backup:
description = "Created *after* executing 'lvextend test_vg/lvol0 -l +1'."
creation_time = 1013166332
This is preparing the way for a simple vgcfgundo command.
slightly different from the current LVM1 method.
lvcreate --persistent y --minor 10 (to specify when created)
lvchange --persistent n (to turn off)
lvchange --persistent y --minor 11 (to change)
--persistent uses a new LV status flag stored on disk
minor number is stored on disk the same way as LVM1 does
(but major number stored is 0; any LVM1 major/minor setting gets lost)
lvchange -ay --minor 12 (to activate using minor 12, regardless of the
on-disk setting, which doesn't get changed)
--minor == -m
--persistent == -M
missing from a VG. (Linear targets use the device-mapper 'error' target
which returns ioerror; striped targets use '/dev/ioerror' for now - which must
already exist e.g. as a sufficiently large block device version of /dev/zero).
by allocating the data block with an additional dbg_malloc.
o Added an assertion to check that no one is requesting alternate
alignment for memory allocated from pool. I can't see us needing this
for LVM2.
Otherwise LVM1 decides the PV structure is corrupt.
But do we need to keep both pv->pe_size and vg->extent_size
in internal metadata or can we generate pvd->pe_size when writing out
a PV that belongs to a VG?
This should be a rare occurrence so the aim is to recover if it's
straightforward to do so, otherwise just to abort the operation.
If people *knowingly* change device names, they should always run vgscan
afterwards.
A few bytes of memory gets leaked inside a pool each time an alias
has to be discarded - it's not worth restructuring the code to reuse it.
More of LVM2 needs updating to pass device objects (or uuids) about
instead of pathnames so that resolution of pathname->object only happens
once per operation.
dev_cache_get() should now always return the *current* device at the path given
dev_name_confirmed() replaces dev_name() whenever it's important to
know that name for the device is still current (ie when opening it).
If the cache doesn't know a current name, the function fails.
dev_open() guarantees that the file descriptor returned is for the dev_t
of the device structure it was passed.
o When opening device, return error if its cached name is incorrect (eg if
it's changed since the cache was generated). This prevents use until
the cache is rebuilt (eg with vgscan). Doesn't catch every case.
struct pv_list {
struct list list;
struct physical_volume pv;
};
to
struct pv_list {
struct list list;
struct physical_volume *pv;
};
o New function in toollib 'create_pv_list', which creates a list of pv's
from a given command line array of pv's.
o Changed lvcreate/extend to use this (fixes lvextend [pv list] bug).
Kernel driver has a version number (stored in kernel/VERSION).
The first two components of this (0.94) give the version number of the
ioctl interface. This number must be changed whenever a change is
made to the ioctl interface that breaks backwards compatibility.
The library has a version number (stored in VERSION) which is
used for linking.
The first and/or second component of this must be changed whenever
a change is made to the library API that breaks backwards
compatibility.
onto a new device). uuid specified must not already exist on the system.
o More message tidying.
o When checking for label, only read PV metadata.
o Add ataraid. [Needs moving into config/defaults files.]
o updated vgcfgrestore args
o change _check_for_open_devices only to check devices present in the hash
table instead of using dev_iter which triggers a full scan even when only
displaying command line help
Supply offset to start of variable data area (so struct size can change
without breaking backward compatibility)
Add command that just returns the driver version
o roll vgcache back to agk's implementation, we'll revisit this as part
of the cluster integration.
o change the extra_info field in a label to be a void *
is active in the device-mapper.
o Many operations can be carried out regardless of whether the VG is
active or not.
o vgscan does not activate anything - use vgchange.
o Change lvrename to support renaming of active LVs.
o Remove '//' appearing in some pathnames.
o Dummy lv_check_segments() for compilation.
I've split the old autobackup function into two seperate areas:
'archiving' is performed *before* a vg configuration is changed. This
produces a numbered backup in /etc/lvm/archive.
A 'backup' is performed *after* a vg change. So the directory /etc/lvm/backup
will hold the a copy of the current configuration.
Current version of LVM2 instead relies on /usr/include/libdevmapper.h
which gets installed by the device mapper package.
(Should this location now be configurable?)
at the top of the file.
o Changed completion_matches -> rl_completion_matches, and added some consts.
This will probably break things on pre readline 4.2 systems.
o There is now a _default_debug, and _default_verbose level, when
using lvm interactively -vv and -dd switches just effect the current
command.
o Added a --quiet switch which sets both verbose and debug to zero.
o Introduced the LVM_SYSTEM_DIR variable.
This makes more sense because the persistent cache, and backup directories
are config specific.
eg, I use /etc/lvm for running my real LV's
but I have another directory /dev/lvm_loops that contains a config
that allows only loopback devices, I use this for testing.
o You must list long args with no short option (eg. --version) at the
front of the args.h file.
o If an argument has no short option, set the short option in args.h to '\0'
o The index into the 'the_args' var is now stored as the option value
for getopt, iff there is no short opt.
A substantial speed-up - particularly in readline mode.
If the hints turn out to be wrong, the relevant parts get thrown away.
vgscan destroys it totally. In both cases it then rebuilds itself as
required.
- The iterator can find labels by string and also appropriate version number (==,
<= or any) if you want.
- Add labels_match() call that compares the two labels and returns an error if
they do not match.
- Write labels in sector 1 & last rather than 2 & last as per Joe.
o Changed pv_map.c to maintain the list of free areas in size order, which
is more helpful to the allocators. If you want to allocate a bit of an
area call consume_area(area, size), this will adjust the area if there's
some space left and shuffle it to the correct place in the list.
Not tested.
old dmfs-lv.c thats gone.
o Dropped out support for multiple tables in line with ioctl interface
o Some reordering to better support the userland library
o Updated to 2.4.16
I'm fairly happy with the way that this is working now, so the next job is
to start the integration with the ioctl interface so there is a single
common dm.[ch] and selectable interfaces (fs or ioctl).
Further improvements can be made even now, but I hope to wait until we've
got this going and integrated and the libdm parts working as well before
investigating other avenues.
a seperate chunk of memory from dbg_malloc for each pool_alloc. This
will allow the bounds checking code in dbg_malloc to do it's stuff.
o The normal implementation moved to pool-fast.c
o pool.c now just contains a #ifdef and includes the appropriate .c file.
Alasdair, could you make sure that gcc -MM get's passed all the
CFLAGS please, otherwise the dependencies get calculated incorrectly.
logical volumes. It includes:
format1 changes.
metadata.h changes.
lv_manip.c changed (striped allocation still not done though).
activate.c changes.
Nothing has been near a compiler as yet.
Alasdair can you look at changing display.c to use to output the mappings
in a more segment oriented format please ?
I haven't put the span list into struct physical_volume to represent allocated
extents. I think the burden of maintaining it for things like lv_extend may
out weigh it's uses.
o Changed disk-rep to use these
o if NDEBUG is not defined the dev_cache will check for open devices on
teardown.
I was hoping this would speed things up. But I'm still getting:
reti:/home/joe/sistina/LVM2/tools# time ./lvm vgchange -a n
Volume group vg0 successfully changed
real 0m5.751s
user 0m0.060s
sys 0m0.070s
even though I have only 1 device with the vg on it passing the filters.
N.B. This means that you have to take very great care in the event that
you want to access the dcache tree from in kernel
o Added extra field to allow out of memory conditions to result in the
correct error code. (This hasn't received a lot of testing...)
I've ditched the final project (which would have cleared my whole list)
since its got other complications which I don't have time to fix right
now. Still as Meatloaf says, two out of three ain't bad!
o Full signed arguments to lvreduce/lvextend
o Consistent lv_number/pe map use
o Populate pv->pe_allocated
o Fixes for allocation/writing of multiple LVs
o Created dmfs.h as a private header for the filesystem code
o Using seq_file.[ch] written by Al Viro as a generic mechanism for /proc
style files which have one record per line. We use a slight modification
here, so if you are using a recent -ac kernel you'll need to replace the
existing seq_file.[ch] with the ones here and do a bit of editing to make
it work. I'll submit the changes to Al Viro shortly as they are very
small and I think make sense generally.
o Using fail_writepage()
o Init code for filesystem now all in dmfs-super.c
o Some common code reduction amoung the dmfs-*.c files
o Auto allocation of major device number (default). You can specify a
particular major by using a module argument. If built in then you don't
get this option at the moment but it could be added if required.
o Hotplug support
o General tidying
o Updated projects.txt file
o Patches updated to 2.4.14
Please add to/edit this file as you think of new ideas or discover bugs. The
items in it are in no particular order. They are also only ideas and hence may
never get implemented depending on whether they turn out to be good ideas or
not.
filters in order.
eg,
f = composite_filter_create(2, regex_filter, persistent_filter);
ownership of the filters passes, they will be destroyed when f's
destroy method is called.
devices {
# first match is final, eg. /dev/ide/cdrom
# get's rejected due to the first pattern
filter=["r/cdrom/", # don't touch the music !
"a/hd[a-d][0-9]+/",
"a/ide/",
"a/sd/",
"a/md/",
"a|loop/[0-9]+|", # accept devfs style loop back
"r/loop/", # and reject old style
"a/dasd/",
"a/dac960/",
"a/nbd/",
"a/ida/",
"a/cciss/",
"a/ubd/",
"r/.*/"] # reject all others
}
Alasdair this is ready to roll into the tools now.
and builds a *very* efficient engine that will tell you which regex a string
matches with only a single pass through the string. To be used in the config
file when specifying devices.
o Anchor's aren't supported yet (^ and $) but that won't take long.
o Also when we get some realistic config files we may want to consider adding an
extra level of indirection to the dfa state in order to compress the table.
It all depends on how large typical tables get.
Things to note:
o Changes to the dm-*.c files have been kept as small as possible during
the development of the new fs interface and there are a few places where
the new code does odd things to give the original code what it wants. These
places will gradually go away during the next few days once we are sure the
new code is sound.
o I've spent most of my testing time looking at the parser since thats where
a lot of the changes are, I've not checked the actual I/O very much, but
then that code hasn't changed at all.
o The print operation in the target type operations is there to help in
debugging and will go away eventually
o There are some other printk's which will also go away once we are sure that
things are working correctly.
o I've tagged the old code with PRE_DMFS if you want to use that until this is
stable.
o There are no kernel patches for this yet (will fix after lunch... :-)
o Makefile needs some changes
o need to EXPORT_SYMBOL(deny_write_access); in ksyms.c
How to use the new interface ?
mount -t dmfs dmfs /mnt/dm
cd /mnt/dm
mkdir fish fish/tank
cd fish/tank
cat ~/my.table > table
cd ..
ln -s tank ACTIVE
Creates a logical volume called fish and activates a table called tank, if
there is a problem doing the link, look in /mnt/dm/fish/tank/errors to see
what is wrong.
If you see any odd things happening, let me know right away as I'm sure there'll
be one or two things that slipped through my testing.
o Error file routines (initial idea)
o Various fixes for bugs
o Tidy a few things
o Added a bit of debugging code ready for when this gets tested
o get_exclusive_write_access() function which will get moved into namei.c
I hope (and rewritten accordingly), should this become the final version
used.
Still a few more areas need thinking about, but in general much closer now I
think. Last area to sort out before testing is the symlink code which is
pretty close now... just a few more checks needed and the actual calls to
the core code.
o Fixed to work with highmem
o Added dmfs private inode struct for lv and table directories
o Fixed a number of errors/typos
o Status file read returns 0 so we can leave this until we've actually got
something to report in this now.
o New locking on tables.... still some issues to be worked out here but
closer now I think.
o Now use mapping of table directory to hold pages rather than mapping of
table file inode. Need to write a note to myself to fix issues with the
file length at the same time....
Well thats enough for tonight I think. The error file will be part of
tomorrows work.
o Redo write logic for table file
o Relax rules for symlink content by removing the rewriting function
Well I probably won't get a chance to work on this tomorrow, so this is my
changeset to date.
non-obvious, its time to simplify :-)
o Moving towards a simpler and more obviously correct interface
o Removed some fs operations in directories representing volumes
o Changed some file names
o Made things cleaner
more changes to follow...
o Changed dm_remove() to accept a struct mapped_device argument rather than
a name
o We no longer have to look up devices by name, the dcache handles that
nicely for us
o Fixed a bug where we were freeing a structure before we'd finished with
it.
o The name field in struct mapped_device is now only used in a very few
places in dm.c and will be replaced in future with a back reference to
the dentry rather than keeping the name in two places.
o dmfs-dir.c becomes dmfs-lv.c
o dmfs-file.c becomes dmfs-table.c
o A few tweeks and updates
The main reason for the slow progress on these files (which are not yet used
by the device mapper) is that we are working out what this interface should
look like as we go along.
Once this has evolved a bit further and in a state where it can be used we'll
announce it on the lists for further comment.
o Only one list of block devices for all tables
o Locking to ensure that block devices only get opened once
o Block device handling is now in dm-blkdev.c
o We open block devices when we create the tables and hold them open until
the table is destroyed (this prevents the module for the device being
unloaded after table parsing and before the table is used)
o We compute the hardsect size when the table is created rather than when
someone requests it.
Still to fix/change:
o Probably want to hash the device lists in dm-blkdev.c and also remove refs
to struct dm_bdev outside this file.
o Need to ensure that hardsect_size doesn't change when new tables are
swapped in (maybe this ought to be a per volume parameter and the tables
will only parse if they match the value for the volume?).
Things are changing fast here, so if you want a stable version of thic code
try checking out yesterdays.
o Moved the linear target into its own module (not really because it needs to
be there, but because its useful to have a simple example so people can see
what we are doing)
Btw, this needs testing properly.
to up_write() and down_write() etc so that you can see what kind of a lock
it is (otherwise it could be anything.. semaphore, spinlock, spinlock_bh,
spinlock_irq, br_lock, etc.)
Mount the dm-fs filesystem on /device-mapper (will fix later). mkdir
to create a device, inside that directory every file you create is a table
file. If there are errors <table>.err will appear automagically. Mv a table
file to ACTIVE to activeate the device. I'm not happy with mv being the
binding command, symlink would be better.
devices based on /proc/devices
+ The dev_mgr structure now has a 256 element char array that is
initially all 0s
+ When a match is found, the array element corresponding to the major
number of the match is set to a non-zero value
+ to check for a match, all one has to do is check that the array
element at the major number in question is non-zero.
o I'm wondering if we should do this with bitwise operators instead? Does
anyone expect the major numbers to grow larger than 8-bits?
o I don't like it, but I'm committing it so I can go back and laugh at
myself later
o I have a (hopefully) better idea that i'll try to commit yet today.
o I'm working on caching the /proc/devices entries now, and should have
that in by the end of today or early tomorrow.
o There will be much cleanup involved with that...
# lvmdbusd relies on command log report to inspect LVM command's execution status
report_command_log=1
# display only outermost LVM shell-related log that lvmdbusd inspects first after LVM command execution (it calls 'lastlog' for more detailed log afterwards if needed)
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
