1
0
mirror of git://sourceware.org/git/lvm2.git synced 2024-12-22 17:35:59 +03:00
Commit Graph

510 Commits

Author SHA1 Message Date
Zdenek Kabelac
fdd76da33d cov: drop uneeded header files 2018-10-15 17:49:44 +02:00
Zdenek Kabelac
b1ff52ca14 cov: check dev_close_immediate
Function can report log_error() on fail path.
2018-10-15 17:49:44 +02:00
Zdenek Kabelac
eb566e034f cov: add check for positive value
As pgsize parameter for _init_free_list() can't be negative,
report problem in case for any reason we would get negative number.
2018-10-15 17:49:44 +02:00
Zdenek Kabelac
9b85ecb85b cov: fix memleak on bcache io error path
Drop allocated IO.

merge free bache
2018-10-15 17:49:44 +02:00
Joe Thornber
3255e384db [bcache] Remove unused 'hash' field from blocks.
We use a radix tree these days rather than a hash table.
2018-09-11 13:17:29 +01:00
David Teigland
fade9ca3b6 bcache: reduce MAX_IO to 256
This is the number of concurrent async io requests that
the scan layer will submit to the bcache layer.  There
will be an open fd for each of these, so it is best to
keep this well below the default limit for max open files
(1024), otherwise lvm may get EMFILE from open(2) when
there are around 1024 devices to scan on the system.
2018-08-24 14:55:12 -05:00
Zdenek Kabelac
c8b4f9414c dev_io: no discard in testmode
When lvm2 command is executed in test mode, discard ioctl is skipped.
This may cause even data-loose in case, issuing discard for released
areas was enabled and user 'tested'  lvreduce.
2018-07-09 00:19:30 +02:00
Marian Csontos
a14f21bf1d bcache: Fix null pointer dereferencing 2018-06-26 17:04:18 +02:00
Zdenek Kabelac
c728d88e11 build: include configure.h
It's important to consistenly include  configure.h as the 1st. header.
It containts #defines influencing behavior of other included header
files.
2018-06-22 23:11:44 +02:00
David Teigland
15826214f9 Remove code for using files as devices
It appears this has not been used in a long time,
and it seems to have no point since loop devices exist.
2018-06-21 09:33:21 -05:00
David Teigland
42f7caf1c2 scan: work around udev problems by avoiding open RDWR
udev creates a train wreck of events if we open devices
with RDWR.  Until we can fix/disable/scrap udev, work around
this by opening RDONLY and then closing/reopening RDWR when
a write is needed.  This invalidates the bcache blocks for
the device before writing so it can trigger unnecessary
rereading.
2018-06-20 14:08:12 -05:00
David Teigland
f85a010a6b bcache: remove extraneous error message
an error from io_submit is already recognized by
the caller like errors during completion.
2018-06-18 12:02:22 -05:00
David Teigland
328303d4d4 Remove unused device error counting 2018-06-15 14:04:39 -05:00
David Teigland
3fd75d1bcd scan: use full md filter when md 1.0 devices are present
The md filter can operate in two native modes:
- normal: reads only the start of each device
- full: reads both the start and end of each device

md 1.0 devices place the superblock at the end of the device,
so components of this version will only be identified and
excluded when lvm uses the full md filter.

Previously, the full md filter was only used in commands
that could write to the device.  Now, the full md filter
is also applied when there is an md 1.0 device present
on the system.  This means the 'pvs' command can avoid
displaying md 1.0 components (at the cost of doubling
the i/o to every device on the system.)

(The md filter can operate in a third mode, using udev,
but this is disabled by default because there have been
problems with reliability of the info returned from udev.)
2018-06-15 12:21:25 -05:00
David Teigland
8eab37593e Add cmd arg to more functions
so that it can be used in the filter code
2018-06-15 11:03:55 -05:00
Joe Thornber
d5da55ed85 device_mapper: remove dbg_malloc.
I wrote dbg_malloc before we had valgrind.  These days there's just
no need.
2018-06-08 13:40:53 +01:00
Joe Thornber
286c1ba336 device_mapper: rename libdevmapper.h -> all.h
I'm paranoid a file will include the global one in /usr/include
by accident.
2018-06-08 12:31:45 +01:00
David Teigland
1539e51721 devices: clean up io error messages
Remove the io error message from bcache.c since it is not
very useful without the device path.

Make the io error messages from dev_read_bytes/dev_write_bytes
more user friendly.
2018-06-07 16:17:04 +01:00
Joe Thornber
dbba1e9b93 Merge branch 'master' into 2018-05-11-fork-libdm 2018-06-01 13:04:12 +01:00
Joe Thornber
d4d39d0f90 Merge branch 'master' into 2018-05-30-bcache-radix-tree 2018-05-31 16:36:04 +01:00
David Teigland
6d14d5d16b scan: removed failed paths for devices
Drop a device path when the scan fails to open it.
2018-05-30 09:05:18 -05:00
Joe Thornber
7635df8cce bcache: switch to storing blocks in a radix tree.
Rather than a hash table.  This will make invalidate_fd() more
efficient since we can iterate just those blocks that are on
a particular dev.
2018-05-30 14:17:26 +01:00
David Teigland
28c8e95d19 scan: refresh paths and retry open
If scanning fails to open any devices, refresh the
device paths in dev cache, and retry the opens.
2018-05-25 13:09:07 -05:00
David Teigland
3c9ed33f83 scan: move warnings about duplicate devices
We have been warning about duplicate devices (and disabling lvmetad)
immediately when the dup was detected (during label_scan).  Move the
warnings (and the disabling) to happen later, after label_scan is
finished.

This lets us avoid an unwanted warning message about duplicates
in the special case were md components are eliminated during the
duplicate device resolution.
2018-05-21 16:48:02 -05:00
Joe Thornber
5052970da3 bcache: Don't call sysconf for every io 2018-05-17 10:05:10 +01:00
Alex Bennée
c6ca81a38d bcache: don't use PAGE_SIZE compile const
PAGE_SIZE is not a compile time constant. Use sysconf instead like
elsewhere in the code.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2018-05-17 10:38:16 +02:00
Joe Thornber
89fdc0b588 Merge branch 'master' into 2018-05-11-fork-libdm 2018-05-16 13:43:02 +01:00
Joe Thornber
ccc35e2647 device-mapper: Fork libdm internally.
The device-mapper directory now holds a copy of libdm source.  At
the moment this code is identical to libdm.  Over time code will
migrate out to appropriate places (see doc/refactoring.txt).

The libdm directory still exists, and contains the source for the
libdevmapper shared library, which we will continue to ship (though
not neccessarily update).

All code using libdm should now use the version in device-mapper.
2018-05-16 13:00:50 +01:00
Joe Thornber
e296f784c9 Merge branch 'master' of git://sourceware.org/git/lvm2 2018-05-16 10:11:58 +01:00
Joe Thornber
df2acbbb97 bcache: nr_ios_pending wasn't being incremented
... but it was being decremented on completion.  Which meant
it wrapped, and no prefetches were ever issued after the
first completion.
2018-05-16 10:09:17 +01:00
Joe Thornber
7f97c7ea9a build: Don't generate symlinks in include/ dir
As we start refactoring the code to break dependencies (see doc/refactoring.txt),
I want us to use full paths in the includes (eg, #include "base/data-struct/list.h").
This makes it more obvious when we're breaking abstraction boundaries, eg, including a file in
metadata/ from base/
2018-05-14 10:30:20 +01:00
Zdenek Kabelac
ac768a9d2b bcache: do not use libdm header files
Logging for libdm differs from lvm logging - keep using consisten
logging function calls.
2018-05-12 18:18:23 +02:00
David Teigland
09fcc8eaa8 scan: ignore duplicates that are md component devs
md devices using an older superblock version have
superblocks at the end of the md device.  For commands
that skip reading the end of devices during filtering,
the md component devs will be scanned, and will appear
as duplicate PVs to the original md device.  Remove
these md components from the list of unused duplicate
devices, so they are treated as if they had been
ignored during filtering.  This avoids the restrictions
that are placed on using PVs with duplicates.
2018-05-11 15:52:22 -05:00
David Teigland
73578e36fa dev_cache: remove the lvmcache check when closing fd
This is no longer used since devices are not held
open in dev_cache.
2018-05-11 14:30:10 -05:00
David Teigland
3e3cb22f2a dev_cache: fix close in utility functions
All these functions are now used as utilities,
e.g. for ioctl (not for io), and need to
open/close the device each time they are called.
(Many of the opens can probably be eliminated by
just using the bcache fd for the ioctl.)
2018-05-11 14:25:08 -05:00
David Teigland
b5d9914628 devs: recognize md devices in subsystem check
If md components appear as duplicate PVs, let the
existing subsystem check recognize the md device.
2018-05-11 14:00:19 -05:00
David Teigland
ccab54677c dev_cache: fix close in dev_get_block_size 2018-05-11 13:53:19 -05:00
David Teigland
bbb8040456 dev_cache: drop open_list
devices are now held open only in bcache,
so drop the dev_cache list of open devices
which is unused.
2018-05-11 12:47:56 -05:00
Joe Thornber
3b02b35c3e Merge branch 'master' of git+ssh://sourceware.org/git/lvm2 2018-05-11 05:39:27 +01:00
Joe Thornber
5f780813f2 bcache/sync io engine: handle short ios 2018-05-11 05:37:47 +01:00
David Teigland
57bb46c5e7 filter: use bcache for filter reads
Filters are still applied before any device reading or
the label scan, but any filter checks that want to read
the device are skipped and the device is flagged.

After bcache is populated, but before lvm looks for
devices (i.e. before label scan), the filters are
reapplied to the devices that were flagged above.
The filters will then find the data they need in
bcache.
2018-05-10 16:03:19 -05:00
Joe Thornber
ae50374811 bcache: Add sync io engine
Something to fall back to when testing.
2018-05-10 14:29:26 +01:00
Joe Thornber
67b80e2d9d bcache: knock out err param.
Dave used this for debugging.  Not needed in general.
2018-05-10 13:26:08 +01:00
Joe Thornber
1c5c99afce bcache-utils: bcache_set_bytes() 2018-05-09 11:05:29 +01:00
Joe Thornber
dfc320f5b8 bcache-utils: rewrite
They take care to avoid redundant reads now.
2018-05-03 11:36:29 +01:00
Joe Thornber
2688aafefb bcache: rename bcache_write_zeroes() -> bcache_zero_bytes()
Now matches the other util functions:

bcache_{prefetch,read,write,zero}_bytes()
2018-05-03 10:21:14 +01:00
Joe Thornber
8b755f1e04 bcache: rewrite bcache_write_zeros()
It now uses GF_ZERO to avoid reading blocks that are going to be
completely zeroed.
2018-05-03 10:14:56 +01:00
Joe Thornber
dc30d4b2f2 bcache: switch off_t -> uint64_t
We always want it to be 64bit
2018-05-03 09:37:43 +01:00
Joe Thornber
efad84ebc2 bcache: Move the utils to a separate file.
This makes it clearer that they don't access the cache internals.
2018-05-03 09:34:41 +01:00
Joe Thornber
b3c41bce3d bcache: add bcache_block_sectors() query fn 2018-05-03 09:33:55 +01:00
Joe Thornber
65912ce44d bcache: add a comment 2018-05-03 09:21:10 +01:00
Joe Thornber
90d0ff6636 bcache: reorder includes in .c file too 2018-05-02 19:45:06 +01:00
Joe Thornber
8fd300f7df device/bcache: reorder includes 2018-05-02 18:59:43 +01:00
David Teigland
24e7745d7a devices: ignore lvm1 and pool devices 2018-05-01 15:18:47 -05:00
Joe Thornber
bfc61a9543 bcache: squash some warnings on rhel6 2018-05-01 13:21:53 +01:00
Joe Thornber
f564e78d98 bcache: rewrite bcache_{write,zero}_bytes
These are utility functions so should only use the public interface.

Also write_bytes was flushing, which will kill performance.
2018-05-01 12:07:33 +01:00
Joe Thornber
e890c37704 [bcache] Some work on bcache_invalidate()
bcache_invalidate() now returns a bool to indicate success.  If fails
if the block is currently held, or the block is dirty and writeback
fails.

Added a bunch of unit tests for the invalidate functions.

Fixed some bugs to do with invalidating errored blocks.
2018-04-27 10:56:13 +01:00
Joe Thornber
1c97fda425 [bcache] get all unit tests passing again 2018-04-26 13:13:27 +01:00
Zdenek Kabelac
fcdac700f9 gcc: remove duplicate typedef 2018-04-23 22:42:18 +02:00
David Teigland
c0973e70a5 dev_cache: clean up scan
Pull out all of the twisted logic and simply call dev_cache_scan
at the start of the command prior to label scan.
2018-04-20 11:22:48 -05:00
David Teigland
6d05859862 bcache: let caller see an error 2018-04-20 11:22:48 -05:00
David Teigland
570c6239ee bcache: fix error handling
The error handling code wasn't working, but it
appears that just removing it is what we need.
The doesn't really need any different behavior
related to bcache blocks on an io error, it just
wants to know if there was an error.
2018-04-20 11:22:47 -05:00
David Teigland
4331182964 bcache: add some error messages for debugging 2018-04-20 11:22:47 -05:00
David Teigland
e49b114f7e bcache: use wrappers for bcache read write in lvm
Using a wrapper makes it easier to disable bcache if needed.
2018-04-20 11:22:47 -05:00
David Teigland
8065492046 bcache: do all writes through bcache 2018-04-20 11:22:47 -05:00
David Teigland
8b26a007b1 misc bcache fixes from ejt 2018-04-20 11:22:47 -05:00
David Teigland
6c67c7557c scan: use separate fd for bcache
Create a new dev->bcache_fd that the scanning code owns
and is in charge of opening/closing.  This prevents other
parts of lvm code (which do various open/close) from
interfering with the bcache fd.  A number of dev_open
and dev_close are removed from the reading path since
the read path now uses the bcache.

With that in place, open(O_EXCL) for pvcreate/pvremove
can then be fixed.  That wouldn't work previously because
of other open fds.
2018-04-20 11:22:46 -05:00
David Teigland
a7cb76ae94 scan: use bcache for label scan and vg read
New label_scan function populates bcache for each device
on the system.

The two read paths are updated to get data from bcache.

The bcache is not yet used for writing.  bcache blocks
for a device are invalidated when the device is written.
2018-04-20 11:19:24 -05:00
David Teigland
93fc937429 [device/bcache] bcache_read_bytes should put blocks 2018-04-20 11:12:50 -05:00
David Teigland
7be54bd687 [device/bcache] fix min() function 2018-04-20 11:12:50 -05:00
David Teigland
d9e6298edb [device/bcache] fix missing max_io fn in bcache async engine 2018-04-20 11:12:50 -05:00
Joe Thornber
dc8034f5eb [device/bcache] more work on bcache 2018-04-20 11:12:50 -05:00
Joe Thornber
6a57ed17a2 [device/bcache] add bcache_prefetch_bytes() and bcache_read_bytes()
Not tested yet.
2018-04-20 11:12:50 -05:00
Joe Thornber
467adfa082 [device/bcache] More tests and some bug fixes 2018-04-20 11:12:50 -05:00
Joe Thornber
19647d1cd4 [device/bcache] fix bug in _alloc_block 2018-04-20 11:12:50 -05:00
Joe Thornber
1563b93691 [device/bcache] Add bcache_max_prefetches()
Ignore prefetches if max io is in flight.
2018-04-20 11:12:50 -05:00
Joe Thornber
c4c4acfd42 [device/bcache] Add a couple of invalidate methods 2018-04-20 11:12:50 -05:00
Joe Thornber
0f0eb04edb [device/bcache] some more work on bcache 2018-04-20 11:12:50 -05:00
Joe Thornber
46867a45d2 [device/bcache] stub a unit test 2018-04-20 11:12:50 -05:00
Joe Thornber
da7e13ef88 [lib/device/bcache] Tweaks after Kabi's review 2018-04-20 11:10:45 -05:00
Joe Thornber
acb42ec465 [device/bcache] Initial code drop.
Compiles.  Not written tests yet.
2018-04-20 11:10:45 -05:00
Joe Thornber
00f1b208a1 [io paths] Unpick agk's aio stuff 2018-04-20 11:03:58 -05:00
Zdenek Kabelac
e878c3fc32 cleanup: correct casting 2018-04-20 12:17:01 +02:00
Zdenek Kabelac
f2d0eefa77 coverity: make use of defined variable
Since we declare 'r', let's use the value for something.
2018-03-17 23:33:58 +01:00
Zdenek Kabelac
285413b502 cleanup: missing dots and indent 2018-03-15 11:01:04 +01:00
Zdenek Kabelac
70ad633638 devcache: add reason and always log_error
With these read errors it's useful to know the reason.
Also avoid to log error just once so we know exactly
how many times we did failing read.

On the other hand reduce repeated log_error() on code 'backtrace'
path and change severity of message to just log_debug() so the
actual read error is printed once for one read.
2018-03-15 10:50:28 +01:00
Zdenek Kabelac
b6e7a0b490 cleanup: more usage of dm_strncpy
Use existing wrapper function arournd  strncpy + buf[] = 0;
2018-03-06 15:40:34 +01:00
Zdenek Kabelac
6b48868cf0 io: keep 64b arithmetic
Widen to 64b arithmetic from start.
2018-02-28 21:05:18 +01:00
Alasdair G Kergon
d6cabbbc53 device: Fix basic async I/O error handling 2018-02-08 20:19:21 +00:00
Alasdair G Kergon
3e29c80122 device: Queue any aio beyond defined limits. 2018-02-08 20:15:37 +00:00
Alasdair G Kergon
db41fe6c5d lvmcache: Use asynchronous I/O when scanning devices. 2018-02-08 20:15:29 +00:00
Alasdair G Kergon
8c7bbcfb0f device: Basic config and setup to support async I/O. 2018-02-08 20:15:14 +00:00
Alasdair G Kergon
7a9af3cd0e device: Add flag to indicate that a code path can support AIO
Until the whole source supports AIO, library code can check for
AIO_SUPPORTED_CODE_PATH to determine whether or not it is OK
to use AIO.
2018-02-06 01:11:00 +00:00
Alasdair G Kergon
e869a52cc4 callbacks: Miscellaneous fixes for recent changes 2018-02-06 01:09:39 +00:00
Zdenek Kabelac
a1cfef9f26 dev_io: fix writes for unaligned buffers
Actually the removed code is necessary - since not all writes are
getting alligned buffer - older compilers seems to be not able
to create 4K aligned buffers on stack - this the aligning code still
need to be present for write path.
2018-01-23 13:36:12 +01:00
Zdenek Kabelac
6e9148e7ab debug: drop DEBUG_MEM path
Memory is not allocated so no DEBUG_MEM part is needed.
2018-01-23 11:45:18 +01:00
Alasdair G Kergon
9194610f42 device: Add ioflags parameter to transfer additional state.
Flags are set on the initial I/O and passed to any callbacks that
may in turn issue further I/O using the inherited flags.
2018-01-21 21:10:23 +00:00
Alasdair G Kergon
c26458339e device: Move buffer allocation nearer to the I/O.
Don't allocate memory until it's needed - later we'll add
some of the I/O to an internal queue instead of issuing it
immediately.
2018-01-16 01:12:08 +00:00
Alasdair G Kergon
081902b4c1 device: Merge _dev_read and dev_read_callback. 2018-01-16 00:41:42 +00:00
Alasdair G Kergon
b825987b2f device: Rearrange _aligned_io(). 2018-01-15 20:10:54 +00:00