linux/block
Jan Kara b6e68ee825 blk-mq: Improve performance of non-mq IO schedulers with multiple HW queues
Currently when non-mq aware IO scheduler (BFQ, mq-deadline) is used for
a queue with multiple HW queues, the performance it rather bad. The
problem is that these IO schedulers use queue-wide locking and their
dispatch function does not respect the hctx it is passed in and returns
any request it finds appropriate. Thus locality of request access is
broken and dispatch from multiple CPUs just contends on IO scheduler
locks. For these IO schedulers there's little point in dispatching from
multiple CPUs. Instead dispatch always only from a single CPU to limit
contention.

Below is a comparison of dbench runs on XFS filesystem where the storage
is a raid card with 64 HW queues and to it attached a single rotating
disk. BFQ is used as IO scheduler:

      clients           MQ                     SQ             MQ-Patched
Amean 1      39.12 (0.00%)       43.29 * -10.67%*       36.09 *   7.74%*
Amean 2     128.58 (0.00%)      101.30 *  21.22%*       96.14 *  25.23%*
Amean 4     577.42 (0.00%)      494.47 *  14.37%*      508.49 *  11.94%*
Amean 8     610.95 (0.00%)      363.86 *  40.44%*      362.12 *  40.73%*
Amean 16    391.78 (0.00%)      261.49 *  33.25%*      282.94 *  27.78%*
Amean 32    324.64 (0.00%)      267.71 *  17.54%*      233.00 *  28.23%*
Amean 64    295.04 (0.00%)      253.02 *  14.24%*      242.37 *  17.85%*
Amean 512 10281.61 (0.00%)    10211.16 *   0.69%*    10447.53 *  -1.61%*

Numbers are times so lower is better. MQ is stock 5.10-rc6 kernel. SQ is
the same kernel with megaraid_sas.host_tagset_enable=0 so that the card
advertises just a single HW queue. MQ-Patched is a kernel with this
patch applied.

You can see multiple hardware queues heavily hurt performance in
combination with BFQ. The patch restores the performance.

Signed-off-by: Jan Kara <jack@suse.cz>
Reviewed-by: Ming Lei <ming.lei@redhat.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2021-01-24 18:19:46 -07:00
..
partitions block: use an xarray for disk->part_tbl 2021-01-24 18:17:20 -07:00
badblocks.c treewide: Use fallthrough pseudo-keyword 2020-08-23 17:36:59 -05:00
bfq-cgroup.c bfq: fix blkio cgroup leakage v4 2020-08-18 07:48:08 -07:00
bfq-iosched.c Revert "blk-mq, elevator: Count requests per hctx to improve performance" 2021-01-24 18:19:46 -07:00
bfq-iosched.h bfq: fix blkio cgroup leakage v4 2020-08-18 07:48:08 -07:00
bfq-wf2q.c bfq: fix blkio cgroup leakage v4 2020-08-18 07:48:08 -07:00
bio-integrity.c block: store a block_device pointer in struct bio 2021-01-24 18:17:20 -07:00
bio.c block: store a block_device pointer in struct bio 2021-01-24 18:17:20 -07:00
blk-cgroup-rwstat.c blk-cgroup: separate out blkg_rwstat under CONFIG_BLK_CGROUP_RWSTAT 2019-11-07 12:28:13 -07:00
blk-cgroup-rwstat.h blk-cgroup: separate out blkg_rwstat under CONFIG_BLK_CGROUP_RWSTAT 2019-11-07 12:28:13 -07:00
blk-cgroup.c block: store a block_device pointer in struct bio 2021-01-24 18:17:20 -07:00
blk-core.c blk-mq: use ->bi_bdev for I/O accounting 2021-01-24 18:17:20 -07:00
blk-crypto-fallback.c block: store a block_device pointer in struct bio 2021-01-24 18:17:20 -07:00
blk-crypto-internal.h block: make blk_crypto_rq_bio_prep() able to fail 2020-10-05 10:47:43 -06:00
blk-crypto.c block: store a block_device pointer in struct bio 2021-01-24 18:17:20 -07:00
blk-exec.c block: add a blk_account_io_merge_bio helper 2020-05-27 05:21:23 -06:00
blk-flush.c for-5.11/block-2020-12-14 2020-12-16 12:57:51 -08:00
blk-integrity.c block: remove the unused blk_integrity_merge_bio export 2020-10-06 07:29:53 -06:00
blk-ioc.c block: remove retry loop in ioc_release_fn() 2020-07-16 10:22:15 -06:00
blk-iocost.c blk-iocost: fix NULL iocg deref from racing against initialization 2021-01-05 11:33:32 -07:00
blk-iolatency.c block: Remove redundant 'return' statement 2020-10-08 07:59:48 -06:00
blk-lib.c block: move the start_sect field to struct block_device 2020-12-01 14:53:40 -07:00
blk-map.c block: fix bmd->is_null_mapped initialization 2020-09-23 09:18:39 -06:00
blk-merge.c block: store a block_device pointer in struct bio 2021-01-24 18:17:20 -07:00
blk-mq-cpumap.c blk-mq: remove the calling of local_memory_node() 2020-10-20 07:08:17 -06:00
blk-mq-debugfs-zoned.c block: Cleanup license notice 2019-01-17 21:21:40 -07:00
blk-mq-debugfs.c block-5.11-2021-01-10 2021-01-10 12:53:08 -08:00
blk-mq-debugfs.h blk-mq: no need to check return value of debugfs_create functions 2019-06-13 03:00:30 -06:00
blk-mq-pci.c block: Fix blk_mq_*_map_queues() kernel-doc headers 2019-05-31 15:12:34 -06:00
blk-mq-rdma.c block: Fix blk_mq_*_map_queues() kernel-doc headers 2019-05-31 15:12:34 -06:00
blk-mq-sched.c block: remove the request_queue to argument request based tracepoints 2020-12-04 09:42:00 -07:00
blk-mq-sched.h block-5.10-2020-10-12 2020-10-13 12:12:44 -07:00
blk-mq-sysfs.c blk-mq: move cancel of hctx->run_work to the front of blk_exit_queue 2020-10-09 12:46:28 -06:00
blk-mq-tag.c block-mq: fix comments in blk_mq_queue_tag_busy_iter 2020-09-29 08:11:00 -06:00
blk-mq-tag.h blk-mq: Relocate hctx_may_queue() 2020-09-03 15:20:47 -06:00
blk-mq-virtio.c blk-mq: Fix typo in comment 2020-03-17 20:55:21 +01:00
blk-mq.c blk-mq: Improve performance of non-mq IO schedulers with multiple HW queues 2021-01-24 18:19:46 -07:00
blk-mq.h blk-mq: update arg in comment of blk_mq_map_queue 2020-12-12 11:13:41 -07:00
blk-pm.c scsi: block: Fix a race in the runtime power management code 2020-12-09 11:41:41 -05:00
blk-pm.h scsi: block: Do not accept any requests while suspended 2020-12-09 11:41:42 -05:00
blk-rq-qos.c Revert "blk-rq-qos: remove redundant finish_wait to rq_qos_wait." 2020-07-15 09:33:37 -06:00
blk-rq-qos.h blk-rq-qos: fix first node deletion of rq_qos_del() 2019-10-15 10:13:13 -06:00
blk-settings.c block: use an xarray for disk->part_tbl 2021-01-24 18:17:20 -07:00
blk-stat.c blk-stat: make q->stats->lock irqsafe 2020-09-01 16:48:46 -06:00
blk-stat.h block: deactivate blk_stat timer in wbt_disable_default() 2018-12-12 06:47:51 -07:00
blk-sysfs.c blk-mq: move cancel of hctx->run_work to the front of blk_exit_queue 2020-10-09 12:46:28 -06:00
blk-throttle.c block: store a block_device pointer in struct bio 2021-01-24 18:17:20 -07:00
blk-timeout.c block: blk-timeout: delete duplicated word 2020-07-31 16:29:47 -06:00
blk-wbt.c block: wbt: Remove unnecessary invoking of wbt_update_limits in wbt_init 2020-11-30 15:01:18 -07:00
blk-wbt.h blk-wbt: remove wbt_update_limits 2020-05-29 16:30:39 -06:00
blk-zoned.c block: Improve blk_revalidate_disk_zones() checks 2020-12-07 17:34:21 -07:00
blk.h block: use an xarray for disk->part_tbl 2021-01-24 18:17:20 -07:00
bounce.c block: store a block_device pointer in struct bio 2021-01-24 18:17:20 -07:00
bsg-lib.c block: drop double zeroing 2020-09-23 09:18:13 -06:00
bsg.c compat_ioctl: bsg: add handler 2020-01-03 09:33:21 +01:00
cmdline-parser.c
elevator.c block: fix comment and add lockdep assert 2020-10-09 12:34:06 -06:00
genhd.c block: use an xarray for disk->part_tbl 2021-01-24 18:17:20 -07:00
ioctl.c block: move the policy field to struct block_device 2020-12-01 14:53:40 -07:00
ioprio.c block: grant IOPRIO_CLASS_RT to CAP_SYS_NICE 2020-09-01 19:38:33 -06:00
Kconfig blk-wbt: Remove obsolete multiqueue I/O scheduling comment 2020-09-01 16:49:26 -06:00
Kconfig.iosched treewide: replace '---help---' in Kconfig files with 'help' 2020-06-14 01:57:21 +09:00
keyslot-manager.c block/keyslot-manager: prevent crash when num_slots=1 2020-11-20 11:52:52 -07:00
kyber-iosched.c blk-mq: Improve performance of non-mq IO schedulers with multiple HW queues 2021-01-24 18:19:46 -07:00
Makefile blk-mq: merge blk-softirq.c into blk-mq.c 2020-06-24 09:15:56 -06:00
mq-deadline.c Revert "blk-mq, elevator: Count requests per hctx to improve performance" 2021-01-24 18:19:46 -07:00
opal_proto.h block: sed-opal: Change the check condition for regular session validity 2020-03-12 08:00:10 -06:00
scsi_ioctl.c drivers-5.10-2020-10-12 2020-10-13 13:04:41 -07:00
sed-opal.c block: sed-opal: Change the check condition for regular session validity 2020-03-12 08:00:10 -06:00
t10-pi.c block: Allow t10-pi to be modular 2020-01-06 20:59:04 -07:00