iv/linux
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

- Split dm-bufio's rw_semaphore and rbtree. Offers improvements to

Browse Source 
dm-bufio's locking to allow increased concurrent IO -- particularly
   for read access for buffers already in dm-bufio's cache.
 
 - Also split dm-bio-prison-v1's spinlock and rbtree with comparable
   aim at improving concurrent IO (for the DM thinp target).
 
 - Both the dm-bufio and dm-bio-prison-v1 scaling of the number of
   locks and rbtrees used are managed by dm_num_hash_locks(). And the
   hash function used by both is dm_hash_locks_index().
 
 - Allow DM targets to require DISCARD, WRITE_ZEROES and SECURE_ERASE
   to be split at the target specified boundary (in terms of
   max_discard_sectors, max_write_zeroes_sectors and
   max_secure_erase_sectors respectively).
 
 - DM verity error handling fix for check_at_most_once on FEC.
 
 - Update DM verity target to emit audit events on verification failure
   and more.
 
 - DM core ->io_hints improvements needed in support of new discard
   support that is added to the DM "zero" and "error" targets.
 
 - Fix missing kmem_cache_destroy() call in initialization error path
   of both the DM integrity and DM clone targets.
 
 - A couple fixes for DM flakey, also add "error_reads" feature.
 
 - Fix DM core's resume to not lock FS when the DM map is NULL;
   otherwise initial table load can race with FS mount that takes
   superblock's ->s_umount rw_semaphore.
 
 - Various small improvements to both DM core and DM targets.
 -----BEGIN PGP SIGNATURE-----
 
 iQEzBAABCAAdFiEEJfWUX4UqZ4x1O2wixSPxCi2dA1oFAmRGtWwACgkQxSPxCi2d
 A1pBqgf/W7op3/PdXBI+tlb7j05MEvMaZx0vz3l+qF36SMglaP1yZLZPiU9MCX2V
 Sm2t4p7VEn5gAxvmzqa0/pLINC7u/m1IW9O6y3qdOEFAgwJ2st+/yaDqgguN5kiA
 uOzecyDfR7n0WU5rkaO2EUneO7MrYweLR3IROFNFNHndl4bVJOafDcOJvmsI4YYe
 5PIMHb+AGND+O2lIVOvSiPD6e85trcRWkr2X6DUYlllV3XEaBLke5MP1OAp+o/Y5
 MFPfznnuiEvcFAzsBoDebC5j7RBQjHw12Bp8ltZV1ZFbdvluw9q1GD2/uyR5UolV
 jmerZXKThV7lRJYqilUmt74Rxl2JSg==
 =zPkM
 -----END PGP SIGNATURE-----

Merge tag 'for-6.4/dm-changes' of git://git.kernel.org/pub/scm/linux/kernel/git/device-mapper/linux-dm

Pull device mapper updates from Mike Snitzer:

 - Split dm-bufio's rw_semaphore and rbtree. Offers improvements to
   dm-bufio's locking to allow increased concurrent IO -- particularly
   for read access for buffers already in dm-bufio's cache.

 - Also split dm-bio-prison-v1's spinlock and rbtree with comparable aim
   at improving concurrent IO (for the DM thinp target).

 - Both the dm-bufio and dm-bio-prison-v1 scaling of the number of locks
   and rbtrees used are managed by dm_num_hash_locks(). And the hash
   function used by both is dm_hash_locks_index().

 - Allow DM targets to require DISCARD, WRITE_ZEROES and SECURE_ERASE to
   be split at the target specified boundary (in terms of
   max_discard_sectors, max_write_zeroes_sectors and
   max_secure_erase_sectors respectively).

 - DM verity error handling fix for check_at_most_once on FEC.

 - Update DM verity target to emit audit events on verification failure
   and more.

 - DM core ->io_hints improvements needed in support of new discard
   support that is added to the DM "zero" and "error" targets.

 - Fix missing kmem_cache_destroy() call in initialization error path of
   both the DM integrity and DM clone targets.

 - A couple fixes for DM flakey, also add "error_reads" feature.

 - Fix DM core's resume to not lock FS when the DM map is NULL;
   otherwise initial table load can race with FS mount that takes
   superblock's ->s_umount rw_semaphore.

 - Various small improvements to both DM core and DM targets.

* tag 'for-6.4/dm-changes' of git://git.kernel.org/pub/scm/linux/kernel/git/device-mapper/linux-dm: (40 commits)
  dm: don't lock fs when the map is NULL in process of resume
  dm flakey: add an "error_reads" option
  dm flakey: remove trailing space in the table line
  dm flakey: fix a crash with invalid table line
  dm ioctl: fix nested locking in table_clear() to remove deadlock concern
  dm: unexport dm_get_queue_limits()
  dm: allow targets to require splitting WRITE_ZEROES and SECURE_ERASE
  dm: add helper macro for simple DM target module init and exit
  dm raid: remove unused d variable
  dm: remove unnecessary (void*) conversions
  dm mirror: add DMERR message if alloc_workqueue fails
  dm: push error reporting down to dm_register_target()
  dm integrity: call kmem_cache_destroy() in dm_integrity_init() error path
  dm clone: call kmem_cache_destroy() in dm_clone_init() error path
  dm error: add discard support
  dm zero: add discard support
  dm table: allow targets without devices to set ->io_hints
  dm verity: emit audit events on verification failure and more
  dm verity: fix error handling for check_at_most_once on FEC
  dm: improve hash_locks sizing and hash function
  ...
This commit is contained in:
Linus Torvalds 2023-04-26 13:05:21 -07:00
commit 48dc810012
41 changed files with 1776 additions and 1083 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
Documentation/admin-guide/device-mapper/dm-flakey.rst
View File

@ -39,6 +39,10 @@ Optional feature parameters:
If no feature parameters are present, during the periods of
unreliability, all I/O returns errors.
error_reads:
All read I/O is failed with an error signalled.
Write I/O is handled correctly.
drop_writes:
All write I/O is silently ignored.
Read I/O is handled correctly.

94
drivers/md/dm-bio-prison-v1.c
View File

@ -18,10 +18,15 @@
#define MIN_CELLS 1024
struct dm_bio_prison {
struct prison_region {
spinlock_t lock;
struct rb_root cells;
struct rb_root cell;
} ____cacheline_aligned_in_smp;
struct dm_bio_prison {
mempool_t cell_pool;
unsigned int num_locks;
struct prison_region regions[];
};
static struct kmem_cache *_cell_cache;
@ -34,13 +39,20 @@ static struct kmem_cache *_cell_cache;
*/
struct dm_bio_prison *dm_bio_prison_create(void)
{
struct dm_bio_prison *prison = kzalloc(sizeof(*prison), GFP_KERNEL);
int ret;
unsigned int i, num_locks;
struct dm_bio_prison *prison;
num_locks = dm_num_hash_locks();
prison = kzalloc(struct_size(prison, regions, num_locks), GFP_KERNEL);
if (!prison)
return NULL;
prison->num_locks = num_locks;
spin_lock_init(&prison->lock);
for (i = 0; i < prison->num_locks; i++) {
spin_lock_init(&prison->regions[i].lock);
prison->regions[i].cell = RB_ROOT;
}
ret = mempool_init_slab_pool(&prison->cell_pool, MIN_CELLS, _cell_cache);
if (ret) {
@ -48,8 +60,6 @@ struct dm_bio_prison *dm_bio_prison_create(void)
return NULL;
}
prison->cells = RB_ROOT;
return prison;
}
EXPORT_SYMBOL_GPL(dm_bio_prison_create);
@ -107,14 +117,32 @@ static int cmp_keys(struct dm_cell_key *lhs,
return 0;
}
static int __bio_detain(struct dm_bio_prison *prison,
static inline unsigned int lock_nr(struct dm_cell_key *key, unsigned int num_locks)
{
return dm_hash_locks_index((key->block_begin >> BIO_PRISON_MAX_RANGE_SHIFT),
num_locks);
}
bool dm_cell_key_has_valid_range(struct dm_cell_key *key)
{
if (WARN_ON_ONCE(key->block_end - key->block_begin > BIO_PRISON_MAX_RANGE))
return false;
if (WARN_ON_ONCE((key->block_begin >> BIO_PRISON_MAX_RANGE_SHIFT) !=
(key->block_end - 1) >> BIO_PRISON_MAX_RANGE_SHIFT))
return false;
return true;
}
EXPORT_SYMBOL(dm_cell_key_has_valid_range);
static int __bio_detain(struct rb_root *root,
struct dm_cell_key *key,
struct bio *inmate,
struct dm_bio_prison_cell *cell_prealloc,
struct dm_bio_prison_cell **cell_result)
{
int r;
struct rb_node **new = &prison->cells.rb_node, *parent = NULL;
struct rb_node **new = &root->rb_node, *parent = NULL;
while (*new) {
struct dm_bio_prison_cell *cell =
@ -139,7 +167,7 @@ static int __bio_detain(struct dm_bio_prison *prison,
*cell_result = cell_prealloc;
rb_link_node(&cell_prealloc->node, parent, new);
rb_insert_color(&cell_prealloc->node, &prison->cells);
rb_insert_color(&cell_prealloc->node, root);
return 0;
}
@ -151,10 +179,11 @@ static int bio_detain(struct dm_bio_prison *prison,
struct dm_bio_prison_cell **cell_result)
{
int r;
unsigned l = lock_nr(key, prison->num_locks);
spin_lock_irq(&prison->lock);
r = __bio_detain(prison, key, inmate, cell_prealloc, cell_result);
spin_unlock_irq(&prison->lock);
spin_lock_irq(&prison->regions[l].lock);
r = __bio_detain(&prison->regions[l].cell, key, inmate, cell_prealloc, cell_result);
spin_unlock_irq(&prison->regions[l].lock);
return r;
}
@ -181,11 +210,11 @@ EXPORT_SYMBOL_GPL(dm_get_cell);
/*
* @inmates must have been initialised prior to this call
*/
static void __cell_release(struct dm_bio_prison *prison,
static void __cell_release(struct rb_root *root,
struct dm_bio_prison_cell *cell,
struct bio_list *inmates)
{
rb_erase(&cell->node, &prison->cells);
rb_erase(&cell->node, root);
if (inmates) {
if (cell->holder)
@ -198,20 +227,22 @@ void dm_cell_release(struct dm_bio_prison *prison,
struct dm_bio_prison_cell *cell,
struct bio_list *bios)
{
spin_lock_irq(&prison->lock);
__cell_release(prison, cell, bios);
spin_unlock_irq(&prison->lock);
unsigned l = lock_nr(&cell->key, prison->num_locks);
spin_lock_irq(&prison->regions[l].lock);
__cell_release(&prison->regions[l].cell, cell, bios);
spin_unlock_irq(&prison->regions[l].lock);
}
EXPORT_SYMBOL_GPL(dm_cell_release);
/*
* Sometimes we don't want the holder, just the additional bios.
*/
static void __cell_release_no_holder(struct dm_bio_prison *prison,
static void __cell_release_no_holder(struct rb_root *root,
struct dm_bio_prison_cell *cell,
struct bio_list *inmates)
{
rb_erase(&cell->node, &prison->cells);
rb_erase(&cell->node, root);
bio_list_merge(inmates, &cell->bios);
}
@ -219,11 +250,12 @@ void dm_cell_release_no_holder(struct dm_bio_prison *prison,
struct dm_bio_prison_cell *cell,
struct bio_list *inmates)
{
unsigned l = lock_nr(&cell->key, prison->num_locks);
unsigned long flags;
spin_lock_irqsave(&prison->lock, flags);
__cell_release_no_holder(prison, cell, inmates);
spin_unlock_irqrestore(&prison->lock, flags);
spin_lock_irqsave(&prison->regions[l].lock, flags);
__cell_release_no_holder(&prison->regions[l].cell, cell, inmates);
spin_unlock_irqrestore(&prison->regions[l].lock, flags);
}
EXPORT_SYMBOL_GPL(dm_cell_release_no_holder);
@ -248,18 +280,19 @@ void dm_cell_visit_release(struct dm_bio_prison *prison,
void *context,
struct dm_bio_prison_cell *cell)
{
spin_lock_irq(&prison->lock);
unsigned l = lock_nr(&cell->key, prison->num_locks);
spin_lock_irq(&prison->regions[l].lock);
visit_fn(context, cell);
rb_erase(&cell->node, &prison->cells);
spin_unlock_irq(&prison->lock);
rb_erase(&cell->node, &prison->regions[l].cell);
spin_unlock_irq(&prison->regions[l].lock);
}
EXPORT_SYMBOL_GPL(dm_cell_visit_release);
static int __promote_or_release(struct dm_bio_prison *prison,
static int __promote_or_release(struct rb_root *root,
struct dm_bio_prison_cell *cell)
{
if (bio_list_empty(&cell->bios)) {
rb_erase(&cell->node, &prison->cells);
rb_erase(&cell->node, root);
return 1;
}
@ -271,10 +304,11 @@ int dm_cell_promote_or_release(struct dm_bio_prison *prison,
struct dm_bio_prison_cell *cell)
{
int r;
unsigned l = lock_nr(&cell->key, prison->num_locks);
spin_lock_irq(&prison->lock);
r = __promote_or_release(prison, cell);
spin_unlock_irq(&prison->lock);
spin_lock_irq(&prison->regions[l].lock);
r = __promote_or_release(&prison->regions[l].cell, cell);
spin_unlock_irq(&prison->regions[l].lock);
return r;
}

15
drivers/md/dm-bio-prison-v1.h
View File

@ -34,6 +34,16 @@ struct dm_cell_key {
dm_block_t block_begin, block_end;
};
/*
* The range of a key (block_end - block_begin) must not
* exceed BIO_PRISON_MAX_RANGE. Also the range must not
* cross a similarly sized boundary.
*
* Must be a power of 2.
*/
#define BIO_PRISON_MAX_RANGE 1024
#define BIO_PRISON_MAX_RANGE_SHIFT 10
/*
* Treat this as opaque, only in header so callers can manage allocation
* themselves.
@ -73,6 +83,11 @@ int dm_get_cell(struct dm_bio_prison *prison,
struct dm_bio_prison_cell *cell_prealloc,
struct dm_bio_prison_cell **cell_result);
/*
* Returns false if key is beyond BIO_PRISON_MAX_RANGE or spans a boundary.
*/
bool dm_cell_key_has_valid_range(struct dm_cell_key *key);
/*
* An atomic op that combines retrieving or creating a cell, and adding a
* bio to it.

1986
drivers/md/dm-bufio.c
View File

File diff suppressed because it is too large Load Diff

1
drivers/md/dm-cache-target.c
View File

@ -3459,7 +3459,6 @@ static int __init dm_cache_init(void)
r = dm_register_target(&cache_target);
if (r) {
DMERR("cache target registration failed: %d", r);
kmem_cache_destroy(migration_cache);
return r;
}

2
drivers/md/dm-clone-target.c
View File

@ -2204,7 +2204,7 @@ static int __init dm_clone_init(void)
r = dm_register_target(&clone_target);
if (r < 0) {
DMERR("Failed to register clone target");
kmem_cache_destroy(_hydration_cache);
return r;
}

20
drivers/md/dm-crypt.c
View File

@ -3659,25 +3659,7 @@ static struct target_type crypt_target = {
.iterate_devices = crypt_iterate_devices,
.io_hints = crypt_io_hints,
};
static int __init dm_crypt_init(void)
{
int r;
r = dm_register_target(&crypt_target);
if (r < 0)
DMERR("register failed %d", r);
return r;
}
static void __exit dm_crypt_exit(void)
{
dm_unregister_target(&crypt_target);
}
module_init(dm_crypt_init);
module_exit(dm_crypt_exit);
module_dm(crypt);
MODULE_AUTHOR("Jana Saout <jana@saout.de>");
MODULE_DESCRIPTION(DM_NAME " target for transparent encryption / decryption");

26
drivers/md/dm-delay.c
View File

@ -367,31 +367,7 @@ static struct target_type delay_target = {
.status = delay_status,
.iterate_devices = delay_iterate_devices,
};
static int __init dm_delay_init(void)
{
int r;
r = dm_register_target(&delay_target);
if (r < 0) {
DMERR("register failed %d", r);
goto bad_register;
}
return 0;
bad_register:
return r;
}
static void __exit dm_delay_exit(void)
{
dm_unregister_target(&delay_target);
}
/* Module hooks */
module_init(dm_delay_init);
module_exit(dm_delay_exit);
module_dm(delay);
MODULE_DESCRIPTION(DM_NAME " delay target");
MODULE_AUTHOR("Heinz Mauelshagen <mauelshagen@redhat.com>");

19
drivers/md/dm-dust.c
View File

@ -570,24 +570,7 @@ static struct target_type dust_target = {
.status = dust_status,
.prepare_ioctl = dust_prepare_ioctl,
};
static int __init dm_dust_init(void)
{
int r = dm_register_target(&dust_target);
if (r < 0)
DMERR("dm_register_target failed %d", r);
return r;
}
static void __exit dm_dust_exit(void)
{
dm_unregister_target(&dust_target);
}
module_init(dm_dust_init);
module_exit(dm_dust_exit);
module_dm(dust);
MODULE_DESCRIPTION(DM_NAME " dust test target");
MODULE_AUTHOR("Bryan Gurney <dm-devel@redhat.com>");

19
drivers/md/dm-ebs-target.c
View File

@ -452,24 +452,7 @@ static struct target_type ebs_target = {
.prepare_ioctl = ebs_prepare_ioctl,
.iterate_devices = ebs_iterate_devices,
};
static int __init dm_ebs_init(void)
{
int r = dm_register_target(&ebs_target);
if (r < 0)
DMERR("register failed %d", r);
return r;
}
static void dm_ebs_exit(void)
{
dm_unregister_target(&ebs_target);
}
module_init(dm_ebs_init);
module_exit(dm_ebs_exit);
module_dm(ebs);
MODULE_AUTHOR("Heinz Mauelshagen <dm-devel@redhat.com>");
MODULE_DESCRIPTION(DM_NAME " emulated block size target");

22
drivers/md/dm-era-target.c
View File

@ -1753,27 +1753,7 @@ static struct target_type era_target = {
.iterate_devices = era_iterate_devices,
.io_hints = era_io_hints
};
static int __init dm_era_init(void)
{
int r;
r = dm_register_target(&era_target);
if (r) {
DMERR("era target registration failed: %d", r);
return r;
}
return 0;
}
static void __exit dm_era_exit(void)
{
dm_unregister_target(&era_target);
}
module_init(dm_era_init);
module_exit(dm_era_exit);
module_dm(era);
MODULE_DESCRIPTION(DM_NAME " era target");
MODULE_AUTHOR("Joe Thornber <ejt@redhat.com>");

71
drivers/md/dm-flakey.c
View File

@ -37,6 +37,7 @@ struct flakey_c {
};
enum feature_flag_bits {
ERROR_READS,
DROP_WRITES,
ERROR_WRITES
};
@ -53,7 +54,7 @@ static int parse_features(struct dm_arg_set *as, struct flakey_c *fc,
const char *arg_name;
static const struct dm_arg _args[] = {
{0, 6, "Invalid number of feature args"},
{0, 7, "Invalid number of feature args"},
{1, UINT_MAX, "Invalid corrupt bio byte"},
{0, 255, "Invalid corrupt value to write into bio byte (0-255)"},
{0, UINT_MAX, "Invalid corrupt bio flags mask"},
@ -76,6 +77,17 @@ static int parse_features(struct dm_arg_set *as, struct flakey_c *fc,
return -EINVAL;
}
/*
* error_reads
*/
if (!strcasecmp(arg_name, "error_reads")) {
if (test_and_set_bit(ERROR_READS, &fc->flags)) {
ti->error = "Feature error_reads duplicated";
return -EINVAL;
}
continue;
}
/*
* drop_writes
*/
@ -125,9 +137,9 @@ static int parse_features(struct dm_arg_set *as, struct flakey_c *fc,
* Direction r or w?
*/
arg_name = dm_shift_arg(as);
if (!strcasecmp(arg_name, "w"))
if (arg_name && !strcasecmp(arg_name, "w"))
fc->corrupt_bio_rw = WRITE;
else if (!strcasecmp(arg_name, "r"))
else if (arg_name && !strcasecmp(arg_name, "r"))
fc->corrupt_bio_rw = READ;
else {
ti->error = "Invalid corrupt bio direction (r or w)";
@ -171,6 +183,12 @@ static int parse_features(struct dm_arg_set *as, struct flakey_c *fc,
return -EINVAL;
}
if (!fc->corrupt_bio_byte && !test_bit(ERROR_READS, &fc->flags) &&
!test_bit(DROP_WRITES, &fc->flags) && !test_bit(ERROR_WRITES, &fc->flags)) {
set_bit(ERROR_WRITES, &fc->flags);
set_bit(ERROR_READS, &fc->flags);
}
return 0;
}
@ -346,8 +364,7 @@ static int flakey_map(struct dm_target *ti, struct bio *bio)
* Otherwise, flakey_end_io() will decide if the reads should be modified.
*/
if (bio_data_dir(bio) == READ) {
if (!fc->corrupt_bio_byte && !test_bit(DROP_WRITES, &fc->flags) &&
!test_bit(ERROR_WRITES, &fc->flags))
if (test_bit(ERROR_READS, &fc->flags))
return DM_MAPIO_KILL;
goto map_bio;
}
@ -373,11 +390,6 @@ static int flakey_map(struct dm_target *ti, struct bio *bio)
}
goto map_bio;
}
/*
* By default, error all I/O.
*/
return DM_MAPIO_KILL;
}
map_bio:
@ -404,8 +416,8 @@ static int flakey_end_io(struct dm_target *ti, struct bio *bio,
*/
corrupt_bio_data(bio, fc);
}
} else if (!test_bit(DROP_WRITES, &fc->flags) &&
!test_bit(ERROR_WRITES, &fc->flags)) {
}
if (test_bit(ERROR_READS, &fc->flags)) {
/*
* Error read during the down_interval if drop_writes
* and error_writes were not configured.
@ -422,7 +434,7 @@ static void flakey_status(struct dm_target *ti, status_type_t type,
{
unsigned int sz = 0;
struct flakey_c *fc = ti->private;
unsigned int drop_writes, error_writes;
unsigned int error_reads, drop_writes, error_writes;
switch (type) {
case STATUSTYPE_INFO:
@ -430,21 +442,24 @@ static void flakey_status(struct dm_target *ti, status_type_t type,
break;
case STATUSTYPE_TABLE:
DMEMIT("%s %llu %u %u ", fc->dev->name,
DMEMIT("%s %llu %u %u", fc->dev->name,
(unsigned long long)fc->start, fc->up_interval,
fc->down_interval);
error_reads = test_bit(ERROR_READS, &fc->flags);
drop_writes = test_bit(DROP_WRITES, &fc->flags);
error_writes = test_bit(ERROR_WRITES, &fc->flags);
DMEMIT("%u ", drop_writes + error_writes + (fc->corrupt_bio_byte > 0) * 5);
DMEMIT(" %u", error_reads + drop_writes + error_writes + (fc->corrupt_bio_byte > 0) * 5);
if (error_reads)
DMEMIT(" error_reads");
if (drop_writes)
DMEMIT("drop_writes ");
DMEMIT(" drop_writes");
else if (error_writes)
DMEMIT("error_writes ");
DMEMIT(" error_writes");
if (fc->corrupt_bio_byte)
DMEMIT("corrupt_bio_byte %u %c %u %u ",
DMEMIT(" corrupt_bio_byte %u %c %u %u",
fc->corrupt_bio_byte,
(fc->corrupt_bio_rw == WRITE) ? 'w' : 'r',
fc->corrupt_bio_value, fc->corrupt_bio_flags);
@ -506,25 +521,7 @@ static struct target_type flakey_target = {
.prepare_ioctl = flakey_prepare_ioctl,
.iterate_devices = flakey_iterate_devices,
};
static int __init dm_flakey_init(void)
{
int r = dm_register_target(&flakey_target);
if (r < 0)
DMERR("register failed %d", r);
return r;
}
static void __exit dm_flakey_exit(void)
{
dm_unregister_target(&flakey_target);
}
/* Module hooks */
module_init(dm_flakey_init);
module_exit(dm_flakey_exit);
module_dm(flakey);
MODULE_DESCRIPTION(DM_NAME " flakey target");
MODULE_AUTHOR("Joe Thornber <dm-devel@redhat.com>");

15
drivers/md/dm-integrity.c
View File

@ -3118,7 +3118,7 @@ static int dm_integrity_reboot(struct notifier_block *n, unsigned long code, voi
static void dm_integrity_postsuspend(struct dm_target *ti)
{
struct dm_integrity_c *ic = (struct dm_integrity_c *)ti->private;
struct dm_integrity_c *ic = ti->private;
int r;
WARN_ON(unregister_reboot_notifier(&ic->reboot_notifier));
@ -3167,7 +3167,7 @@ static void dm_integrity_postsuspend(struct dm_target *ti)
static void dm_integrity_resume(struct dm_target *ti)
{
struct dm_integrity_c *ic = (struct dm_integrity_c *)ti->private;
struct dm_integrity_c *ic = ti->private;
__u64 old_provided_data_sectors = le64_to_cpu(ic->sb->provided_data_sectors);
int r;
@ -3290,7 +3290,7 @@ static void dm_integrity_resume(struct dm_target *ti)
static void dm_integrity_status(struct dm_target *ti, status_type_t type,
unsigned int status_flags, char *result, unsigned int maxlen)
{
struct dm_integrity_c *ic = (struct dm_integrity_c *)ti->private;
struct dm_integrity_c *ic = ti->private;
unsigned int arg_count;
size_t sz = 0;
@ -4703,11 +4703,12 @@ static int __init dm_integrity_init(void)
}
r = dm_register_target(&integrity_target);
if (r < 0) {
kmem_cache_destroy(journal_io_cache);
return r;
}
if (r < 0)
DMERR("register failed %d", r);
return r;
return 0;
}
static void __exit dm_integrity_exit(void)

4
drivers/md/dm-io.c
View File

@ -187,7 +187,7 @@ static void list_get_page(struct dpages *dp,
struct page **p, unsigned long *len, unsigned int *offset)
{
unsigned int o = dp->context_u;
struct page_list *pl = (struct page_list *) dp->context_ptr;
struct page_list *pl = dp->context_ptr;
*p = pl->page;
*len = PAGE_SIZE - o;
@ -196,7 +196,7 @@ static void list_get_page(struct dpages *dp,
static void list_next_page(struct dpages *dp)
{
struct page_list *pl = (struct page_list *) dp->context_ptr;
struct page_list *pl = dp->context_ptr;
dp->context_ptr = pl->next;
dp->context_u = 0;

12
drivers/md/dm-ioctl.c
View File

@ -1168,10 +1168,13 @@ static int do_resume(struct dm_ioctl *param)
/* Do we need to load a new map ? */
if (new_map) {
sector_t old_size, new_size;
int srcu_idx;
/* Suspend if it isn't already suspended */
if (param->flags & DM_SKIP_LOCKFS_FLAG)
old_map = dm_get_live_table(md, &srcu_idx);
if ((param->flags & DM_SKIP_LOCKFS_FLAG) || !old_map)
suspend_flags &= ~DM_SUSPEND_LOCKFS_FLAG;
dm_put_live_table(md, srcu_idx);
if (param->flags & DM_NOFLUSH_FLAG)
suspend_flags |= DM_SUSPEND_NOFLUSH_FLAG;
if (!dm_suspended_md(md))
@ -1556,11 +1559,12 @@ static int table_clear(struct file *filp, struct dm_ioctl *param, size_t param_s
has_new_map = true;
}
param->flags &= ~DM_INACTIVE_PRESENT_FLAG;
__dev_status(hc->md, param);
md = hc->md;
up_write(&_hash_lock);
param->flags &= ~DM_INACTIVE_PRESENT_FLAG;
__dev_status(md, param);
if (old_map) {
dm_sync_table(md);
dm_table_destroy(old_map);

4
drivers/md/dm-kcopyd.c
View File

@ -519,7 +519,7 @@ static int run_complete_job(struct kcopyd_job *job)
static void complete_io(unsigned long error, void *context)
{
struct kcopyd_job *job = (struct kcopyd_job *) context;
struct kcopyd_job *job = context;
struct dm_kcopyd_client *kc = job->kc;
io_job_finish(kc->throttle);
@ -696,7 +696,7 @@ static void segment_complete(int read_err, unsigned long write_err,
/* FIXME: tidy this function */
sector_t progress = 0;
sector_t count = 0;
struct kcopyd_job *sub_job = (struct kcopyd_job *) context;
struct kcopyd_job *sub_job = context;
struct kcopyd_job *job = sub_job->master_job;
struct dm_kcopyd_client *kc = job->kc;

6
drivers/md/dm-linear.c
View File

@ -72,7 +72,7 @@ bad:
static void linear_dtr(struct dm_target *ti)
{
struct linear_c *lc = (struct linear_c *) ti->private;
struct linear_c *lc = ti->private;
dm_put_device(ti, lc->dev);
kfree(lc);
@ -98,7 +98,7 @@ static int linear_map(struct dm_target *ti, struct bio *bio)
static void linear_status(struct dm_target *ti, status_type_t type,
unsigned int status_flags, char *result, unsigned int maxlen)
{
struct linear_c *lc = (struct linear_c *) ti->private;
struct linear_c *lc = ti->private;
size_t sz = 0;
switch (type) {
@ -120,7 +120,7 @@ static void linear_status(struct dm_target *ti, status_type_t type,
static int linear_prepare_ioctl(struct dm_target *ti, struct block_device **bdev)
{
struct linear_c *lc = (struct linear_c *) ti->private;
struct linear_c *lc = ti->private;
struct dm_dev *dev = lc->dev;
*bdev = dev->bdev;

21
drivers/md/dm-log-writes.c
View File

@ -429,7 +429,7 @@ static inline sector_t logdev_last_sector(struct log_writes_c *lc)
static int log_writes_kthread(void *arg)
{
struct log_writes_c *lc = (struct log_writes_c *)arg;
struct log_writes_c *lc = arg;
sector_t sector = 0;
while (!kthread_should_stop()) {
@ -937,24 +937,7 @@ static struct target_type log_writes_target = {
.dax_zero_page_range = log_writes_dax_zero_page_range,
.dax_recovery_write = log_writes_dax_recovery_write,
};
static int __init dm_log_writes_init(void)
{
int r = dm_register_target(&log_writes_target);
if (r < 0)
DMERR("register failed %d", r);
return r;
}
static void __exit dm_log_writes_exit(void)
{
dm_unregister_target(&log_writes_target);
}
module_init(dm_log_writes_init);
module_exit(dm_log_writes_exit);
module_dm(log_writes);
MODULE_DESCRIPTION(DM_NAME " log writes target");
MODULE_AUTHOR("Josef Bacik <jbacik@fb.com>");

24
drivers/md/dm-log.c
View File

@ -530,7 +530,7 @@ static void destroy_log_context(struct log_c *lc)
static void core_dtr(struct dm_dirty_log *log)
{
struct log_c *lc = (struct log_c *) log->context;
struct log_c *lc = log->context;
vfree(lc->clean_bits);
destroy_log_context(lc);
@ -569,7 +569,7 @@ static int disk_ctr(struct dm_dirty_log *log, struct dm_target *ti,
static void disk_dtr(struct dm_dirty_log *log)
{
struct log_c *lc = (struct log_c *) log->context;
struct log_c *lc = log->context;
dm_put_device(lc->ti, lc->log_dev);
vfree(lc->disk_header);
@ -590,7 +590,7 @@ static int disk_resume(struct dm_dirty_log *log)
{
int r;
unsigned int i;
struct log_c *lc = (struct log_c *) log->context;
struct log_c *lc = log->context;
size_t size = lc->bitset_uint32_count * sizeof(uint32_t);
/* read the disk header */
@ -652,14 +652,14 @@ static int disk_resume(struct dm_dirty_log *log)
static uint32_t core_get_region_size(struct dm_dirty_log *log)
{
struct log_c *lc = (struct log_c *) log->context;
struct log_c *lc = log->context;
return lc->region_size;
}
static int core_resume(struct dm_dirty_log *log)
{
struct log_c *lc = (struct log_c *) log->context;
struct log_c *lc = log->context;
lc->sync_search = 0;
return 0;
@ -667,14 +667,14 @@ static int core_resume(struct dm_dirty_log *log)
static int core_is_clean(struct dm_dirty_log *log, region_t region)
{
struct log_c *lc = (struct log_c *) log->context;
struct log_c *lc = log->context;
return log_test_bit(lc->clean_bits, region);
}
static int core_in_sync(struct dm_dirty_log *log, region_t region, int block)
{
struct log_c *lc = (struct log_c *) log->context;
struct log_c *lc = log->context;
return log_test_bit(lc->sync_bits, region);
}
@ -727,14 +727,14 @@ static int disk_flush(struct dm_dirty_log *log)
static void core_mark_region(struct dm_dirty_log *log, region_t region)
{
struct log_c *lc = (struct log_c *) log->context;
struct log_c *lc = log->context;
log_clear_bit(lc, lc->clean_bits, region);
}
static void core_clear_region(struct dm_dirty_log *log, region_t region)
{
struct log_c *lc = (struct log_c *) log->context;
struct log_c *lc = log->context;
if (likely(!lc->flush_failed))
log_set_bit(lc, lc->clean_bits, region);
@ -742,7 +742,7 @@ static void core_clear_region(struct dm_dirty_log *log, region_t region)
static int core_get_resync_work(struct dm_dirty_log *log, region_t *region)
{
struct log_c *lc = (struct log_c *) log->context;
struct log_c *lc = log->context;
if (lc->sync_search >= lc->region_count)
return 0;
@ -765,7 +765,7 @@ static int core_get_resync_work(struct dm_dirty_log *log, region_t *region)
static void core_set_region_sync(struct dm_dirty_log *log, region_t region,
int in_sync)
{
struct log_c *lc = (struct log_c *) log->context;
struct log_c *lc = log->context;
log_clear_bit(lc, lc->recovering_bits, region);
if (in_sync) {
@ -779,7 +779,7 @@ static void core_set_region_sync(struct dm_dirty_log *log, region_t region,
static region_t core_get_sync_count(struct dm_dirty_log *log)
{
struct log_c *lc = (struct log_c *) log->context;
struct log_c *lc = log->context;
return lc->sync_count;
}

5
drivers/md/dm-mpath.c
View File

@ -2235,11 +2235,8 @@ static int __init dm_multipath_init(void)
}
r = dm_register_target(&multipath_target);
if (r < 0) {
DMERR("request-based register failed %d", r);
r = -EINVAL;
if (r < 0)
goto bad_register_target;
}
return 0;

22
drivers/md/dm-raid.c
View File

@ -2209,7 +2209,6 @@ static int super_load(struct md_rdev *rdev, struct md_rdev *refdev)
static int super_init_validation(struct raid_set *rs, struct md_rdev *rdev)
{
int role;
unsigned int d;
struct mddev *mddev = &rs->md;
uint64_t events_sb;
uint64_t failed_devices[DISKS_ARRAY_ELEMS];
@ -2324,7 +2323,6 @@ static int super_init_validation(struct raid_set *rs, struct md_rdev *rdev)
* to provide capacity for redundancy or during reshape
* to add capacity to grow the raid set.
*/
d = 0;
rdev_for_each(r, mddev) {
if (test_bit(Journal, &rdev->flags))
continue;
@ -2340,8 +2338,6 @@ static int super_init_validation(struct raid_set *rs, struct md_rdev *rdev)
if (test_bit(FirstUse, &r->flags))
rebuild_and_new++;
}
d++;
}
if (new_devs == rs->raid_disks || !rebuilds) {
@ -4081,23 +4077,7 @@ static struct target_type raid_target = {
.preresume = raid_preresume,
.resume = raid_resume,
};
static int __init dm_raid_init(void)
{
DMINFO("Loading target version %u.%u.%u",
raid_target.version[0],
raid_target.version[1],
raid_target.version[2]);
return dm_register_target(&raid_target);
}
static void __exit dm_raid_exit(void)
{
dm_unregister_target(&raid_target);
}
module_init(dm_raid_init);
module_exit(dm_raid_exit);
module_dm(raid);
module_param(devices_handle_discard_safely, bool, 0644);
MODULE_PARM_DESC(devices_handle_discard_safely,

24
drivers/md/dm-raid1.c
View File

@ -604,7 +604,7 @@ static void do_reads(struct mirror_set *ms, struct bio_list *reads)
static void write_callback(unsigned long error, void *context)
{
unsigned int i;
struct bio *bio = (struct bio *) context;
struct bio *bio = context;
struct mirror_set *ms;
int should_wake = 0;
unsigned long flags;
@ -1180,7 +1180,7 @@ err_free_context:
static void mirror_dtr(struct dm_target *ti)
{
struct mirror_set *ms = (struct mirror_set *) ti->private;
struct mirror_set *ms = ti->private;
del_timer_sync(&ms->timer);
flush_workqueue(ms->kmirrord_wq);
@ -1246,7 +1246,7 @@ static int mirror_end_io(struct dm_target *ti, struct bio *bio,
blk_status_t *error)
{
int rw = bio_data_dir(bio);
struct mirror_set *ms = (struct mirror_set *) ti->private;
struct mirror_set *ms = ti->private;
struct mirror *m = NULL;
struct dm_bio_details *bd = NULL;
struct dm_raid1_bio_record *bio_record =
@ -1311,7 +1311,7 @@ out:
static void mirror_presuspend(struct dm_target *ti)
{
struct mirror_set *ms = (struct mirror_set *) ti->private;
struct mirror_set *ms = ti->private;
struct dm_dirty_log *log = dm_rh_dirty_log(ms->rh);
struct bio_list holds;
@ -1407,7 +1407,7 @@ static void mirror_status(struct dm_target *ti, status_type_t type,
{
unsigned int m, sz = 0;
int num_feature_args = 0;
struct mirror_set *ms = (struct mirror_set *) ti->private;
struct mirror_set *ms = ti->private;
struct dm_dirty_log *log = dm_rh_dirty_log(ms->rh);
char buffer[MAX_NR_MIRRORS + 1];
@ -1498,23 +1498,21 @@ static struct target_type mirror_target = {
static int __init dm_mirror_init(void)
{
int r = -ENOMEM;
int r;
dm_raid1_wq = alloc_workqueue("dm_raid1_wq", 0, 0);
if (!dm_raid1_wq)
goto bad_target;
if (!dm_raid1_wq) {
DMERR("Failed to alloc workqueue");
return -ENOMEM;
}
r = dm_register_target(&mirror_target);
if (r < 0) {
destroy_workqueue(dm_raid1_wq);
goto bad_target;
return r;
}
return 0;
bad_target:
DMERR("Failed to register mirror target");
return r;
}
static void __exit dm_mirror_exit(void)

2
drivers/md/dm-snap-persistent.c
View File

@ -567,7 +567,7 @@ ret_destroy_bufio:
static struct pstore *get_info(struct dm_exception_store *store)
{
return (struct pstore *) store->context;
return store->context;
}
static void persistent_usage(struct dm_exception_store *store,

12
drivers/md/dm-snap.c
View File

@ -2815,22 +2815,16 @@ static int __init dm_snapshot_init(void)
}
r = dm_register_target(&snapshot_target);
if (r < 0) {
DMERR("snapshot target register failed %d", r);
if (r < 0)
goto bad_register_snapshot_target;
}
r = dm_register_target(&origin_target);
if (r < 0) {
DMERR("Origin target register failed %d", r);
if (r < 0)
goto bad_register_origin_target;
}
r = dm_register_target(&merge_target);
if (r < 0) {
DMERR("Merge target register failed %d", r);
if (r < 0)
goto bad_register_merge_target;
}
return 0;

4
drivers/md/dm-stripe.c
View File

@ -189,7 +189,7 @@ static int stripe_ctr(struct dm_target *ti, unsigned int argc, char **argv)
static void stripe_dtr(struct dm_target *ti)
{
unsigned int i;
struct stripe_c *sc = (struct stripe_c *) ti->private;
struct stripe_c *sc = ti->private;
for (i = 0; i < sc->stripes; i++)
dm_put_device(ti, sc->stripe[i].dev);
@ -360,7 +360,7 @@ static size_t stripe_dax_recovery_write(struct dm_target *ti, pgoff_t pgoff,
static void stripe_status(struct dm_target *ti, status_type_t type,
unsigned int status_flags, char *result, unsigned int maxlen)
{
struct stripe_c *sc = (struct stripe_c *) ti->private;
struct stripe_c *sc = ti->private;
unsigned int sz = 0;
unsigned int i;

20
drivers/md/dm-switch.c
View File

@ -565,25 +565,7 @@ static struct target_type switch_target = {
.prepare_ioctl = switch_prepare_ioctl,
.iterate_devices = switch_iterate_devices,
};
static int __init dm_switch_init(void)
{
int r;
r = dm_register_target(&switch_target);
if (r < 0)
DMERR("dm_register_target() failed %d", r);
return r;
}
static void __exit dm_switch_exit(void)
{
dm_unregister_target(&switch_target);
}
module_init(dm_switch_init);
module_exit(dm_switch_exit);
module_dm(switch);
MODULE_DESCRIPTION(DM_NAME " dynamic path switching target");
MODULE_AUTHOR("Kevin D. O'Kelley <Kevin_OKelley@dell.com>");

6
drivers/md/dm-table.c
View File

@ -1661,8 +1661,12 @@ int dm_calculate_queue_limits(struct dm_table *t,
blk_set_stacking_limits(&ti_limits);
if (!ti->type->iterate_devices)
if (!ti->type->iterate_devices) {
/* Set I/O hints portion of queue limits */
if (ti->type->io_hints)
ti->type->io_hints(ti, &ti_limits);
goto combine_limits;
}
/*
* Combine queue limits of all the devices this target uses.

20
drivers/md/dm-target.c
View File

@ -85,12 +85,15 @@ int dm_register_target(struct target_type *tt)
int rv = 0;
down_write(&_lock);
if (__find_target_type(tt->name))
if (__find_target_type(tt->name)) {
DMERR("%s: '%s' target already registered",
__func__, tt->name);
rv = -EEXIST;
else
} else {
list_add(&tt->list, &_targets);
}
up_write(&_lock);
return rv;
}
EXPORT_SYMBOL(dm_register_target);
@ -119,6 +122,7 @@ static int io_err_ctr(struct dm_target *tt, unsigned int argc, char **args)
* Return error for discards instead of -EOPNOTSUPP
*/
tt->num_discard_bios = 1;
tt->discards_supported = true;
return 0;
}
@ -145,6 +149,13 @@ static void io_err_release_clone_rq(struct request *clone,
{
}
static void io_err_io_hints(struct dm_target *ti, struct queue_limits *limits)
{
limits->max_discard_sectors = UINT_MAX;
limits->max_hw_discard_sectors = UINT_MAX;
limits->discard_granularity = 512;
}
static long io_err_dax_direct_access(struct dm_target *ti, pgoff_t pgoff,
long nr_pages, enum dax_access_mode mode, void **kaddr,
pfn_t *pfn)
@ -154,13 +165,14 @@ static long io_err_dax_direct_access(struct dm_target *ti, pgoff_t pgoff,
static struct target_type error_target = {
.name = "error",
.version = {1, 5, 0},
.version = {1, 6, 0},
.features = DM_TARGET_WILDCARD,
.ctr = io_err_ctr,
.dtr = io_err_dtr,
.map = io_err_map,
.clone_and_map_rq = io_err_clone_and_map_rq,
.release_clone_rq = io_err_release_clone_rq,
.io_hints = io_err_io_hints,
.direct_access = io_err_dax_direct_access,
};

125
drivers/md/dm-thin.c
View File

@ -118,25 +118,27 @@ enum lock_space {
PHYSICAL
};
static void build_key(struct dm_thin_device *td, enum lock_space ls,
static bool build_key(struct dm_thin_device *td, enum lock_space ls,
dm_block_t b, dm_block_t e, struct dm_cell_key *key)
{
key->virtual = (ls == VIRTUAL);
key->dev = dm_thin_dev_id(td);
key->block_begin = b;
key->block_end = e;
return dm_cell_key_has_valid_range(key);
}
static void build_data_key(struct dm_thin_device *td, dm_block_t b,
struct dm_cell_key *key)
{
build_key(td, PHYSICAL, b, b + 1llu, key);
(void) build_key(td, PHYSICAL, b, b + 1llu, key);
}
static void build_virtual_key(struct dm_thin_device *td, dm_block_t b,
struct dm_cell_key *key)
{
build_key(td, VIRTUAL, b, b + 1llu, key);
(void) build_key(td, VIRTUAL, b, b + 1llu, key);
}
/*----------------------------------------------------------------*/
@ -883,15 +885,17 @@ static void cell_defer_no_holder(struct thin_c *tc, struct dm_bio_prison_cell *c
{
struct pool *pool = tc->pool;
unsigned long flags;
int has_work;
struct bio_list bios;
spin_lock_irqsave(&tc->lock, flags);
cell_release_no_holder(pool, cell, &tc->deferred_bio_list);
has_work = !bio_list_empty(&tc->deferred_bio_list);
spin_unlock_irqrestore(&tc->lock, flags);
bio_list_init(&bios);
cell_release_no_holder(pool, cell, &bios);
if (has_work)
if (!bio_list_empty(&bios)) {
spin_lock_irqsave(&tc->lock, flags);
bio_list_merge(&tc->deferred_bio_list, &bios);
spin_unlock_irqrestore(&tc->lock, flags);
wake_worker(pool);
}
}
static void thin_defer_bio(struct thin_c *tc, struct bio *bio);
@ -1672,54 +1676,70 @@ static void break_up_discard_bio(struct thin_c *tc, dm_block_t begin, dm_block_t
struct dm_cell_key data_key;
struct dm_bio_prison_cell *data_cell;
struct dm_thin_new_mapping *m;
dm_block_t virt_begin, virt_end, data_begin;
dm_block_t virt_begin, virt_end, data_begin, data_end;
dm_block_t len, next_boundary;
while (begin != end) {
r = ensure_next_mapping(pool);
if (r)
/* we did our best */
return;
r = dm_thin_find_mapped_range(tc->td, begin, end, &virt_begin, &virt_end,
&data_begin, &maybe_shared);
if (r)
if (r) {
/*
* Silently fail, letting any mappings we've
* created complete.
*/
break;
build_key(tc->td, PHYSICAL, data_begin, data_begin + (virt_end - virt_begin), &data_key);
if (bio_detain(tc->pool, &data_key, NULL, &data_cell)) {
/* contention, we'll give up with this range */
begin = virt_end;
continue;
}
/*
* IO may still be going to the destination block. We must
* quiesce before we can do the removal.
*/
m = get_next_mapping(pool);
m->tc = tc;
m->maybe_shared = maybe_shared;
m->virt_begin = virt_begin;
m->virt_end = virt_end;
m->data_block = data_begin;
m->cell = data_cell;
m->bio = bio;
data_end = data_begin + (virt_end - virt_begin);
/*
* The parent bio must not complete before sub discard bios are
* chained to it (see end_discard's bio_chain)!
*
* This per-mapping bi_remaining increment is paired with
* the implicit decrement that occurs via bio_endio() in
* end_discard().
* Make sure the data region obeys the bio prison restrictions.
*/
bio_inc_remaining(bio);
if (!dm_deferred_set_add_work(pool->all_io_ds, &m->list))
pool->process_prepared_discard(m);
while (data_begin < data_end) {
r = ensure_next_mapping(pool);
if (r)
return; /* we did our best */
next_boundary = ((data_begin >> BIO_PRISON_MAX_RANGE_SHIFT) + 1)
<< BIO_PRISON_MAX_RANGE_SHIFT;
len = min_t(sector_t, data_end - data_begin, next_boundary - data_begin);
/* This key is certainly within range given the above splitting */
(void) build_key(tc->td, PHYSICAL, data_begin, data_begin + len, &data_key);
if (bio_detain(tc->pool, &data_key, NULL, &data_cell)) {
/* contention, we'll give up with this range */
data_begin += len;
continue;
}
/*
* IO may still be going to the destination block. We must
* quiesce before we can do the removal.
*/
m = get_next_mapping(pool);
m->tc = tc;
m->maybe_shared = maybe_shared;
m->virt_begin = virt_begin;
m->virt_end = virt_begin + len;
m->data_block = data_begin;
m->cell = data_cell;
m->bio = bio;
/*
* The parent bio must not complete before sub discard bios are
* chained to it (see end_discard's bio_chain)!
*
* This per-mapping bi_remaining increment is paired with
* the implicit decrement that occurs via bio_endio() in
* end_discard().
*/
bio_inc_remaining(bio);
if (!dm_deferred_set_add_work(pool->all_io_ds, &m->list))
pool->process_prepared_discard(m);
virt_begin += len;
data_begin += len;
}
begin = virt_end;
}
@ -1761,8 +1781,13 @@ static void process_discard_bio(struct thin_c *tc, struct bio *bio)
return;
}
build_key(tc->td, VIRTUAL, begin, end, &virt_key);
if (bio_detain(tc->pool, &virt_key, bio, &virt_cell))
if (unlikely(!build_key(tc->td, VIRTUAL, begin, end, &virt_key))) {
DMERR_LIMIT("Discard doesn't respect bio prison limits");
bio_endio(bio);
return;
}
if (bio_detain(tc->pool, &virt_key, bio, &virt_cell)) {
/*
* Potential starvation issue: We're relying on the
* fs/application being well behaved, and not trying to
@ -1771,6 +1796,7 @@ static void process_discard_bio(struct thin_c *tc, struct bio *bio)
* cell will never be granted.
*/
return;
}
tc->pool->process_discard_cell(tc, virt_cell);
}
@ -3378,13 +3404,13 @@ static int pool_ctr(struct dm_target *ti, unsigned int argc, char **argv)
*/
if (pf.discard_enabled && pf.discard_passdown) {
ti->num_discard_bios = 1;
/*
* Setting 'discards_supported' circumvents the normal
* stacking of discard limits (this keeps the pool and
* thin devices' discard limits consistent).
*/
ti->discards_supported = true;
ti->max_discard_granularity = true;
}
ti->private = pt;
@ -4094,7 +4120,7 @@ static struct target_type pool_target = {
.name = "thin-pool",
.features = DM_TARGET_SINGLETON | DM_TARGET_ALWAYS_WRITEABLE |
DM_TARGET_IMMUTABLE,
.version = {1, 22, 0},
.version = {1, 23, 0},
.module = THIS_MODULE,
.ctr = pool_ctr,
.dtr = pool_dtr,
@ -4259,6 +4285,7 @@ static int thin_ctr(struct dm_target *ti, unsigned int argc, char **argv)
if (tc->pool->pf.discard_enabled) {
ti->discards_supported = true;
ti->num_discard_bios = 1;
ti->max_discard_granularity = true;
}
mutex_unlock(&dm_thin_pool_table.mutex);
@ -4474,12 +4501,12 @@ static void thin_io_hints(struct dm_target *ti, struct queue_limits *limits)
return;
limits->discard_granularity = pool->sectors_per_block << SECTOR_SHIFT;
limits->max_discard_sectors = 2048 * 1024 * 16; /* 16G */
limits->max_discard_sectors = pool->sectors_per_block * BIO_PRISON_MAX_RANGE;
}
static struct target_type thin_target = {
.name = "thin",
.version = {1, 22, 0},
.version = {1, 23, 0},
.module = THIS_MODULE,
.ctr = thin_ctr,
.dtr = thin_dtr,

14
drivers/md/dm-unstripe.c
View File

@ -192,19 +192,7 @@ static struct target_type unstripe_target = {
.iterate_devices = unstripe_iterate_devices,
.io_hints = unstripe_io_hints,
};
static int __init dm_unstripe_init(void)
{
return dm_register_target(&unstripe_target);
}
static void __exit dm_unstripe_exit(void)
{
dm_unregister_target(&unstripe_target);
}
module_init(dm_unstripe_init);
module_exit(dm_unstripe_exit);
module_dm(unstripe);
MODULE_DESCRIPTION(DM_NAME " unstriped target");
MODULE_ALIAS("dm-unstriped");

4
drivers/md/dm-verity-fec.c
View File

@ -567,14 +567,14 @@ out:
static void *fec_rs_alloc(gfp_t gfp_mask, void *pool_data)
{
struct dm_verity *v = (struct dm_verity *)pool_data;
struct dm_verity *v = pool_data;
return init_rs_gfp(8, 0x11d, 0, 1, v->fec->roots, gfp_mask);
}
static void fec_rs_free(void *element, void *pool_data)
{
struct rs_control *rs = (struct rs_control *)element;
struct rs_control *rs = element;
if (rs)
free_rs(rs);

42
drivers/md/dm-verity-target.c
View File

@ -16,6 +16,7 @@
#include "dm-verity.h"
#include "dm-verity-fec.h"
#include "dm-verity-verify-sig.h"
#include "dm-audit.h"
#include <linux/module.h>
#include <linux/reboot.h>
#include <linux/scatterlist.h>
@ -248,8 +249,10 @@ static int verity_handle_err(struct dm_verity *v, enum verity_block_type type,
DMERR_LIMIT("%s: %s block %llu is corrupted", v->data_dev->name,
type_str, block);
if (v->corrupted_errs == DM_VERITY_MAX_CORRUPTED_ERRS)
if (v->corrupted_errs == DM_VERITY_MAX_CORRUPTED_ERRS) {
DMERR("%s: reached maximum errors", v->data_dev->name);
dm_audit_log_target(DM_MSG_PREFIX, "max-corrupted-errors", v->ti, 0);
}
snprintf(verity_env, DM_VERITY_ENV_LENGTH, "%s=%d,%llu",
DM_VERITY_ENV_VAR_NAME, type, block);
@ -340,6 +343,11 @@ static int verity_verify_level(struct dm_verity *v, struct dm_verity_io *io,
else if (verity_handle_err(v,
DM_VERITY_BLOCK_TYPE_METADATA,
hash_block)) {
struct bio *bio =
dm_bio_from_per_bio_data(io,
v->ti->per_io_data_size);
dm_audit_log_bio(DM_MSG_PREFIX, "verify-metadata", bio,
block, 0);
r = -EIO;
goto release_ret_r;
}
@ -523,7 +531,7 @@ static int verity_verify_io(struct dm_verity_io *io)
sector_t cur_block = io->block + b;
struct ahash_request *req = verity_io_hash_req(v, io);
if (v->validated_blocks &&
if (v->validated_blocks && bio->bi_status == BLK_STS_OK &&
likely(test_bit(cur_block, v->validated_blocks))) {
verity_bv_skip_block(v, io, iter);
continue;
@ -590,8 +598,11 @@ static int verity_verify_io(struct dm_verity_io *io)
return -EIO;
}
if (verity_handle_err(v, DM_VERITY_BLOCK_TYPE_DATA,
cur_block))
cur_block)) {
dm_audit_log_bio(DM_MSG_PREFIX, "verify-data",
bio, cur_block, 0);
return -EIO;
}
}
}
@ -975,6 +986,8 @@ static void verity_dtr(struct dm_target *ti)
static_branch_dec(&use_tasklet_enabled);
kfree(v);
dm_audit_log_dtr(DM_MSG_PREFIX, ti, 1);
}
static int verity_alloc_most_once(struct dm_verity *v)
@ -1429,11 +1442,14 @@ static int verity_ctr(struct dm_target *ti, unsigned int argc, char **argv)
verity_verify_sig_opts_cleanup(&verify_args);
dm_audit_log_ctr(DM_MSG_PREFIX, ti, 1);
return 0;
bad:
verity_verify_sig_opts_cleanup(&verify_args);
dm_audit_log_ctr(DM_MSG_PREFIX, ti, 0);
verity_dtr(ti);
return r;
@ -1498,25 +1514,7 @@ static struct target_type verity_target = {
.iterate_devices = verity_iterate_devices,
.io_hints = verity_io_hints,
};
static int __init dm_verity_init(void)
{
int r;
r = dm_register_target(&verity_target);
if (r < 0)
DMERR("register failed %d", r);
return r;
}
static void __exit dm_verity_exit(void)
{
dm_unregister_target(&verity_target);
}
module_init(dm_verity_init);
module_exit(dm_verity_exit);
module_dm(verity);
MODULE_AUTHOR("Mikulas Patocka <mpatocka@redhat.com>");
MODULE_AUTHOR("Mandeep Baines <msb@chromium.org>");

22
drivers/md/dm-writecache.c
View File

@ -2773,27 +2773,7 @@ static struct target_type writecache_target = {
.iterate_devices = writecache_iterate_devices,
.io_hints = writecache_io_hints,
};
static int __init dm_writecache_init(void)
{
int r;
r = dm_register_target(&writecache_target);
if (r < 0) {
DMERR("register failed %d", r);
return r;
}
return 0;
}
static void __exit dm_writecache_exit(void)
{
dm_unregister_target(&writecache_target);
}
module_init(dm_writecache_init);
module_exit(dm_writecache_exit);
module_dm(writecache);
MODULE_DESCRIPTION(DM_NAME " writecache target");
MODULE_AUTHOR("Mikulas Patocka <dm-devel@redhat.com>");

31
drivers/md/dm-zero.c
View File

@ -27,6 +27,7 @@ static int zero_ctr(struct dm_target *ti, unsigned int argc, char **argv)
* Silently drop discards, avoiding -EOPNOTSUPP.
*/
ti->num_discard_bios = 1;
ti->discards_supported = true;
return 0;
}
@ -45,6 +46,7 @@ static int zero_map(struct dm_target *ti, struct bio *bio)
zero_fill_bio(bio);
break;
case REQ_OP_WRITE:
case REQ_OP_DISCARD:
/* writes get silently dropped */
break;
default:
@ -57,32 +59,23 @@ static int zero_map(struct dm_target *ti, struct bio *bio)
return DM_MAPIO_SUBMITTED;
}
static void zero_io_hints(struct dm_target *ti, struct queue_limits *limits)
{
limits->max_discard_sectors = UINT_MAX;
limits->max_hw_discard_sectors = UINT_MAX;
limits->discard_granularity = 512;
}
static struct target_type zero_target = {
.name = "zero",
.version = {1, 1, 0},
.version = {1, 2, 0},
.features = DM_TARGET_NOWAIT,
.module = THIS_MODULE,
.ctr = zero_ctr,
.map = zero_map,
.io_hints = zero_io_hints,
};
static int __init dm_zero_init(void)
{
int r = dm_register_target(&zero_target);
if (r < 0)
DMERR("register failed %d", r);
return r;
}
static void __exit dm_zero_exit(void)
{
dm_unregister_target(&zero_target);
}
module_init(dm_zero_init)
module_exit(dm_zero_exit)
module_dm(zero);
MODULE_AUTHOR("Jana Saout <jana@saout.de>");
MODULE_DESCRIPTION(DM_NAME " dummy target returning zeros");

6
drivers/md/dm-zoned-metadata.c
View File

@ -1701,7 +1701,7 @@ static int dmz_load_mapping(struct dmz_metadata *zmd)
if (IS_ERR(dmap_mblk))
return PTR_ERR(dmap_mblk);
zmd->map_mblk[i] = dmap_mblk;
dmap = (struct dmz_map *) dmap_mblk->data;
dmap = dmap_mblk->data;
i++;
e = 0;
}
@ -1832,7 +1832,7 @@ static void dmz_set_chunk_mapping(struct dmz_metadata *zmd, unsigned int chunk,
unsigned int dzone_id, unsigned int bzone_id)
{
struct dmz_mblock *dmap_mblk = zmd->map_mblk[chunk >> DMZ_MAP_ENTRIES_SHIFT];
struct dmz_map *dmap = (struct dmz_map *) dmap_mblk->data;
struct dmz_map *dmap = dmap_mblk->data;
int map_idx = chunk & DMZ_MAP_ENTRIES_MASK;
dmap[map_idx].dzone_id = cpu_to_le32(dzone_id);
@ -2045,7 +2045,7 @@ struct dm_zone *dmz_get_chunk_mapping(struct dmz_metadata *zmd,
unsigned int chunk, enum req_op op)
{
struct dmz_mblock *dmap_mblk = zmd->map_mblk[chunk >> DMZ_MAP_ENTRIES_SHIFT];
struct dmz_map *dmap = (struct dmz_map *) dmap_mblk->data;
struct dmz_map *dmap = dmap_mblk->data;
int dmap_idx = chunk & DMZ_MAP_ENTRIES_MASK;
unsigned int dzone_id;
struct dm_zone *dzone = NULL;

16
drivers/md/dm-zoned-target.c
View File

@ -1138,7 +1138,7 @@ static int dmz_message(struct dm_target *ti, unsigned int argc, char **argv,
return r;
}
static struct target_type dmz_type = {
static struct target_type zoned_target = {
.name = "zoned",
.version = {2, 0, 0},
.features = DM_TARGET_SINGLETON | DM_TARGET_MIXED_ZONED_MODEL,
@ -1154,19 +1154,7 @@ static struct target_type dmz_type = {
.status = dmz_status,
.message = dmz_message,
};
static int __init dmz_init(void)
{
return dm_register_target(&dmz_type);
}
static void __exit dmz_exit(void)
{
dm_unregister_target(&dmz_type);
}
module_init(dmz_init);
module_exit(dmz_exit);
module_dm(zoned);
MODULE_DESCRIPTION(DM_NAME " target for zoned block devices");
MODULE_AUTHOR("Damien Le Moal <damien.lemoal@wdc.com>");

47
drivers/md/dm.c
View File

@ -1072,6 +1072,15 @@ static void dm_io_dec_pending(struct dm_io *io, blk_status_t error)
__dm_io_dec_pending(io);
}
/*
* The queue_limits are only valid as long as you have a reference
* count on 'md'. But _not_ imposing verification to avoid atomic_read(),
*/
static inline struct queue_limits *dm_get_queue_limits(struct mapped_device *md)
{
return &md->queue->limits;
}
void disable_discard(struct mapped_device *md)
{
struct queue_limits *limits = dm_get_queue_limits(md);
@ -1162,7 +1171,8 @@ static inline sector_t max_io_len_target_boundary(struct dm_target *ti,
return ti->len - target_offset;
}
static sector_t max_io_len(struct dm_target *ti, sector_t sector)
static sector_t __max_io_len(struct dm_target *ti, sector_t sector,
unsigned int max_granularity)
{
sector_t target_offset = dm_target_offset(ti, sector);
sector_t len = max_io_len_target_boundary(ti, target_offset);
@ -1173,11 +1183,16 @@ static sector_t max_io_len(struct dm_target *ti, sector_t sector)
* explains why stacked chunk_sectors based splitting via
* bio_split_to_limits() isn't possible here.
*/
if (!ti->max_io_len)
if (!max_granularity)
return len;
return min_t(sector_t, len,
min(queue_max_sectors(ti->table->md->queue),
blk_chunk_sectors_left(target_offset, ti->max_io_len)));
blk_chunk_sectors_left(target_offset, max_granularity)));
}
static inline sector_t max_io_len(struct dm_target *ti, sector_t sector)
{
return __max_io_len(ti, sector, ti->max_io_len);
}
int dm_set_target_max_io_len(struct dm_target *ti, sector_t len)
@ -1565,12 +1580,13 @@ static void __send_empty_flush(struct clone_info *ci)
}
static void __send_changing_extent_only(struct clone_info *ci, struct dm_target *ti,
unsigned int num_bios)
unsigned int num_bios,
unsigned int max_granularity)
{
unsigned int len, bios;
len = min_t(sector_t, ci->sector_count,
max_io_len_target_boundary(ti, dm_target_offset(ti, ci->sector)));
__max_io_len(ti, ci->sector, max_granularity));
atomic_add(num_bios, &ci->io->io_count);
bios = __send_duplicate_bios(ci, ti, num_bios, &len);
@ -1606,16 +1622,24 @@ static blk_status_t __process_abnormal_io(struct clone_info *ci,
struct dm_target *ti)
{
unsigned int num_bios = 0;
unsigned int max_granularity = 0;
struct queue_limits *limits = dm_get_queue_limits(ti->table->md);
switch (bio_op(ci->bio)) {
case REQ_OP_DISCARD:
num_bios = ti->num_discard_bios;
if (ti->max_discard_granularity)
max_granularity = limits->max_discard_sectors;
break;
case REQ_OP_SECURE_ERASE:
num_bios = ti->num_secure_erase_bios;
if (ti->max_secure_erase_granularity)
max_granularity = limits->max_secure_erase_sectors;
break;
case REQ_OP_WRITE_ZEROES:
num_bios = ti->num_write_zeroes_bios;
if (ti->max_write_zeroes_granularity)
max_granularity = limits->max_write_zeroes_sectors;
break;
default:
break;
@ -1630,7 +1654,7 @@ static blk_status_t __process_abnormal_io(struct clone_info *ci,
if (unlikely(!num_bios))
return BLK_STS_NOTSUPP;
__send_changing_extent_only(ci, ti, num_bios);
__send_changing_extent_only(ci, ti, num_bios, max_granularity);
return BLK_STS_OK;
}
@ -2296,17 +2320,6 @@ struct target_type *dm_get_immutable_target_type(struct mapped_device *md)
return md->immutable_target_type;
}
/*
* The queue_limits are only valid as long as you have a reference
* count on 'md'.
*/
struct queue_limits *dm_get_queue_limits(struct mapped_device *md)
{
BUG_ON(!atomic_read(&md->holders));
return &md->queue->limits;
}
EXPORT_SYMBOL_GPL(dm_get_queue_limits);
/*
* Setup the DM device's queue based on md's type
*/

22
drivers/md/dm.h
View File

@ -20,6 +20,7 @@
#include <linux/completion.h>
#include <linux/kobject.h>
#include <linux/refcount.h>
#include <linux/log2.h>
#include "dm-stats.h"
@ -228,4 +229,25 @@ void dm_free_md_mempools(struct dm_md_mempools *pools);
*/
unsigned int dm_get_reserved_bio_based_ios(void);
#define DM_HASH_LOCKS_MAX 64
static inline unsigned int dm_num_hash_locks(void)
{
unsigned int num_locks = roundup_pow_of_two(num_online_cpus()) << 1;
return min_t(unsigned int, num_locks, DM_HASH_LOCKS_MAX);
}
#define DM_HASH_LOCKS_MULT 4294967291ULL
#define DM_HASH_LOCKS_SHIFT 6
static inline unsigned int dm_hash_locks_index(sector_t block,
unsigned int num_locks)
{
sector_t h1 = (block * DM_HASH_LOCKS_MULT) >> DM_HASH_LOCKS_SHIFT;
sector_t h2 = h1 >> DM_HASH_LOCKS_SHIFT;
return (h1 ^ h2) & (num_locks - 1);
}
#endif

40
include/linux/device-mapper.h
View File

@ -358,6 +358,24 @@ struct dm_target {
*/
bool discards_supported:1;
/*
* Set if this target requires that discards be split on
* 'max_discard_sectors' boundaries.
*/
bool max_discard_granularity:1;
/*
* Set if this target requires that secure_erases be split on
* 'max_secure_erase_sectors' boundaries.
*/
bool max_secure_erase_granularity:1;
/*
* Set if this target requires that write_zeroes be split on
* 'max_write_zeroes_sectors' boundaries.
*/
bool max_write_zeroes_granularity:1;
/*
* Set if we need to limit the number of in-flight bios when swapping.
*/
@ -512,8 +530,6 @@ int __init dm_early_create(struct dm_ioctl *dmi,
struct dm_target_spec **spec_array,
char **target_params_array);
struct queue_limits *dm_get_queue_limits(struct mapped_device *md);
/*
* Geometry functions.
*/
@ -625,6 +641,26 @@ void dm_destroy_crypto_profile(struct blk_crypto_profile *profile);
DMEMIT("target_name=%s,target_version=%u.%u.%u", \
(y)->name, (y)->version[0], (y)->version[1], (y)->version[2])
/**
* module_dm() - Helper macro for DM targets that don't do anything
* special in their module_init and module_exit.
* Each module may only use this macro once, and calling it replaces
* module_init() and module_exit().
*
* @name: DM target's name
*/
#define module_dm(name) \
static int __init dm_##name##_init(void) \
{ \
return dm_register_target(&(name##_target)); \
} \
module_init(dm_##name##_init) \
static void __exit dm_##name##_exit(void) \
{ \
dm_unregister_target(&(name##_target)); \
} \
module_exit(dm_##name##_exit)
/*
* Definitions of return values from target end_io function.
*/

6
include/linux/dm-bufio.h
View File

@ -130,12 +130,6 @@ int dm_bufio_issue_flush(struct dm_bufio_client *c);
*/
int dm_bufio_issue_discard(struct dm_bufio_client *c, sector_t block, sector_t count);
/*
* Like dm_bufio_release but also move the buffer to the new
* block. dm_bufio_write_dirty_buffers is needed to commit the new block.
*/
void dm_bufio_release_move(struct dm_buffer *b, sector_t new_block);
/*
* Free the given buffer.
* This is just a hint, if the buffer is in use or dirty, this function

4
include/uapi/linux/dm-ioctl.h
View File

@ -286,9 +286,9 @@ enum {
#define DM_DEV_SET_GEOMETRY _IOWR(DM_IOCTL, DM_DEV_SET_GEOMETRY_CMD, struct dm_ioctl)
#define DM_VERSION_MAJOR 4
#define DM_VERSION_MINOR 47
#define DM_VERSION_MINOR 48
#define DM_VERSION_PATCHLEVEL 0
#define DM_VERSION_EXTRA "-ioctl (2022-07-28)"
#define DM_VERSION_EXTRA "-ioctl (2023-03-01)"
/* Status bits */
#define DM_READONLY_FLAG (1 << 0) /* In/Out */