iv/linux
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
linux/kernel
History
Chao Gao 26f827e095 dma-direct: avoid redundant memory sync for swiotlb 
commit 9e02977bfad006af328add9434c8bffa40e053bb upstream.

When we looked into FIO performance with swiotlb enabled in VM, we found
swiotlb_bounce() is always called one more time than expected for each DMA
read request.

It turns out that the bounce buffer is copied to original DMA buffer twice
after the completion of a DMA request (one is done by in
dma_direct_sync_single_for_cpu(), the other by swiotlb_tbl_unmap_single()).
But the content in bounce buffer actually doesn't change between the two
rounds of copy. So, one round of copy is redundant.

Pass DMA_ATTR_SKIP_CPU_SYNC flag to swiotlb_tbl_unmap_single() to
skip the memory copy in it.

This fix increases FIO 64KB sequential read throughput in a guest with
swiotlb=force by 5.6%.

Fixes: 55897af63091 ("dma-direct: merge swiotlb_dma_ops into the dma_direct code")
Reported-by: Wang Zhaoyang1 <zhaoyang1.wang@intel.com>
Reported-by: Gao Liang <liang.gao@intel.com>
Signed-off-by: Chao Gao <chao.gao@intel.com>
Reviewed-by: Kevin Tian <kevin.tian@intel.com>
Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2022-04-20 09:23:30 +02:00
..
bpf
bpf: Adjust BPF stack helper functions to accommodate skip > 0
2022-04-08 14:40:43 +02:00
cgroup
cgroup: Use open-time credentials for process migraton perm checks
2022-04-13 21:01:10 +02:00
configs
…
debug
kdb: Fix the putarea helper function
2022-04-08 14:40:29 +02:00
dma
dma-direct: avoid redundant memory sync for swiotlb
2022-04-20 09:23:30 +02:00
entry
KVM: rseq: Update rseq when processing NOTIFY_RESUME on xfer to KVM guest
2021-10-06 15:55:49 +02:00
events
perf/core: Fix address filter parser for multiple filters
2022-04-08 14:40:03 +02:00
gcov
gcov: re-fix clang-11+ support
2021-04-14 08:41:58 +02:00
irq
genirq/affinity: Consider that CPUs on nodes can be unbalanced
2022-04-20 09:23:29 +02:00
kcsan
kcsan: Fix debugfs initcall return type
2021-05-26 12:06:54 +02:00
livepatch
livepatch: Fix build failure on 32 bits processors
2022-04-08 14:40:15 +02:00
locking
locking/lockdep: Iterate lock_classes directly when reading lockdep files
2022-04-08 14:40:32 +02:00
power
PM: suspend: fix return value of __setup handler
2022-04-08 14:40:01 +02:00
printk
printk: fix return value of printk.devkmsg __setup handler
2022-04-08 14:40:08 +02:00
rcu
rcu: Don't deboost before reporting expedited quiescent state
2022-03-28 09:57:10 +02:00
sched
sched/core: Export pelt_thermal_tp
2022-04-08 14:40:03 +02:00
time
timers: Fix warning condition in __run_timers()
2022-04-20 09:23:30 +02:00
trace
tracing: Ensure trace buffer is at least 4096 bytes large
2022-03-16 14:16:00 +01:00
.gitignore
kbuild: update config_data.gz only when the content of .config is changed
2021-05-11 14:47:37 +02:00
acct.c
kernel: acct.c: fix some kernel-doc nits
2020-10-16 11:11:19 -07:00
async.c
Revert "module, async: async_synchronize_full() on module init iff async is used"
2022-02-23 12:01:00 +01:00
audit_fsnotify.c
fsnotify: generalize handle_inode_event()
2020-12-30 11:54:18 +01:00
audit_tree.c
audit: move put_tree() to avoid trim_trees refcount underflow and UAF
2021-09-03 10:09:31 +02:00
audit_watch.c
fsnotify: generalize handle_inode_event()
2020-12-30 11:54:18 +01:00
audit.c
audit: improve audit queue handling when "audit=1" on cmdline
2022-02-08 18:30:34 +01:00
audit.h
audit: log AUDIT_TIME_* records only from rules
2022-04-08 14:40:00 +02:00
auditfilter.c
treewide: Use fallthrough pseudo-keyword
2020-08-23 17:36:59 -05:00
auditsc.c
audit: log AUDIT_TIME_* records only from rules
2022-04-08 14:40:00 +02:00
backtracetest.c
treewide: Replace DECLARE_TASKLET() with DECLARE_TASKLET_OLD()
2020-07-30 11:15:58 -07:00
bounds.c
…
capability.c
LSM: Signal to SafeSetID when setting group IDs
2020-10-13 09:17:34 -07:00
compat.c
treewide: Use fallthrough pseudo-keyword
2020-08-23 17:36:59 -05:00
configs.c
…
context_tracking.c
…
cpu_pm.c
PM: cpu: Make notifier chain use a raw_spinlock_t
2021-09-15 09:50:40 +02:00
cpu.c
sched/scs: Reset task stack state in bringup_cpu()
2021-12-01 09:19:08 +01:00
crash_core.c
crash_core, vmcoreinfo: append 'SECTION_SIZE_BITS' to vmcoreinfo
2021-06-23 14:42:52 +02:00
crash_dump.c
…
cred.c
Revert "Add a reference to ucounts for each cred"
2021-09-08 08:49:00 +02:00
delayacct.c
…
dma.c
…
exec_domain.c
…
exit.c
kernel/io_uring: cancel io_uring before task works
2021-01-30 13:55:18 +01:00
extable.c
…
fail_function.c
fail_function: Remove a redundant mutex unlock
2020-11-19 11:58:16 -08:00
fork.c
copy_process(): Move fd_install() out of sighand->siglock critical section
2022-02-23 12:01:08 +01:00
freezer.c
Revert "kernel: freezer should treat PF_IO_WORKER like PF_KTHREAD for freezing"
2021-04-07 15:00:14 +02:00
futex.c
mm, futex: fix shared futex pgoff on shmem huge page
2021-06-30 08:47:29 -04:00
gen_kheaders.sh
…
groups.c
LSM: Signal to SafeSetID when setting group IDs
2020-10-13 09:17:34 -07:00
hung_task.c
kernel/hung_task.c: make type annotations consistent
2020-11-02 12:14:19 -08:00
iomem.c
…
irq_work.c
…
jump_label.c
jump_label: Fix jump_label_text_reserved() vs __init
2021-07-20 16:05:58 +02:00
kallsyms.c
treewide: Convert macro and uses of __section(foo) to __section("foo")
2020-10-25 14:51:49 -07:00
kcmp.c
exec: Transform exec_update_mutex into a rw_semaphore
2021-01-09 13:46:24 +01:00
Kconfig.freezer
…
Kconfig.hz
…
Kconfig.locks
…
Kconfig.preempt
…
kcov.c
kcov: make some symbols static
2020-08-12 10:58:02 -07:00
kexec_core.c
kernel: kexec: remove the lock operation of system_transition_mutex
2021-02-03 23:28:37 +01:00
kexec_elf.c
…
kexec_file.c
kernel: kexec_file: fix error return code of kexec_calculate_store_digests()
2021-05-19 10:13:09 +02:00
kexec_internal.h
…
kexec.c
LSM: Introduce kernel_post_load_data() hook
2020-10-05 13:37:03 +02:00
kheaders.c
…
kmod.c
kmod: remove redundant "be an" in the comment
2020-08-12 10:58:01 -07:00
kprobes.c
kprobes: Limit max data_size of the kretprobe instances
2021-12-08 09:03:20 +01:00
ksysfs.c
…
kthread.c
kthread: Fix PF_KTHREAD vs to_kthread() race
2021-09-03 10:09:31 +02:00
latencytop.c
…
Makefile
kbuild: update config_data.gz only when the content of .config is changed
2021-05-11 14:47:37 +02:00
module_signature.c
module: harden ELF info handling
2021-03-25 09:04:11 +01:00
module_signing.c
module: harden ELF info handling
2021-03-25 09:04:11 +01:00
module-internal.h
…
module.c
Revert "module, async: async_synchronize_full() on module init iff async is used"
2022-02-23 12:01:00 +01:00
notifier.c
notifier: Fix broken error handling pattern
2020-09-01 09:58:03 +02:00
nsproxy.c
…
padata.c
padata: fix possible padata_works_lock deadlock
2020-09-04 17:51:55 +10:00
panic.c
panic: don't dump stack twice on warn
2020-11-14 11:26:04 -08:00
params.c
params: Replace zero-length array with flexible-array member
2020-10-29 17:22:59 -05:00
pid_namespace.c
memcg: enable accounting for pids in nested pid namespaces
2021-09-18 13:40:36 +02:00
pid.c
exec: Transform exec_update_mutex into a rw_semaphore
2021-01-09 13:46:24 +01:00
profile.c
profiling: fix shift-out-of-bounds bugs
2021-09-26 14:08:58 +02:00
ptrace.c
ptrace: Check PTRACE_O_SUSPEND_SECCOMP permission on PTRACE_SEIZE
2022-04-08 14:39:50 +02:00
range.c
kernel.h: split out min()/max() et al. helpers
2020-10-16 11:11:19 -07:00
reboot.c
reboot: fix overflow parsing reboot cpu number
2020-11-14 11:26:03 -08:00
regset.c
regset: kill ->get()
2020-07-27 14:31:12 -04:00
relay.c
kernel/relay.c: drop unneeded initialization
2020-10-16 11:11:22 -07:00
resource.c
kernel/resource: make walk_mem_res() find all busy IORESOURCE_MEM resources
2021-05-19 10:13:09 +02:00
rseq.c
rseq: Remove broken uapi field layout on 32-bit little endian
2022-04-08 14:40:03 +02:00
scftorture.c
scftorture: Add cond_resched() to test loop
2020-08-24 18:38:38 -07:00
scs.c
mm: memcontrol: account kernel stack per node
2020-08-07 11:33:25 -07:00
seccomp.c
seccomp: Fix setting loaded filter count during TSYNC
2021-08-18 08:59:06 +02:00
signal.c
signal: Remove the bogus sigkill_pending in ptrace_stop
2021-11-18 14:03:47 +01:00
smp.c
smp: Fix offline cpu check in flush_smp_call_function_queue()
2022-04-20 09:23:29 +02:00
smpboot.c
sched/core: Initialize the idle task with preemption disabled
2021-07-14 16:55:50 +02:00
smpboot.h
…
softirq.c
softirq: Add debug check to __raise_softirq_irqoff()
2020-09-16 15:18:56 +02:00
stackleak.c
gcc-plugins/stackleak: Use noinstr in favor of notrace
2022-02-23 12:01:00 +01:00
stacktrace.c
stacktrace: Remove reliable argument from arch_stack_walk() callback
2020-09-18 14:24:16 +01:00
static_call.c
static_call: Fix unused variable warn w/o MODULE
2021-09-08 08:49:00 +02:00
stop_machine.c
stop_machine, rcu: Mark functions as notrace
2020-10-26 12:12:27 +01:00
sys_ni.c
mm/madvise: introduce process_madvise() syscall: an external memory hinting API
2020-10-18 09:27:10 -07:00
sys.c
prctl: allow to setup brk for et_dyn executables
2021-09-26 14:08:57 +02:00
sysctl-test.c
…
sysctl.c
x86/speculation: Include unprivileged eBPF status in Spectre v2 mitigation reporting
2022-03-11 12:11:49 +01:00
task_work.c
task_work: cleanup notification modes
2020-10-17 15:05:30 -06:00
taskstats.c
taskstats: move specifying netlink policy back to ops
2020-10-02 19:11:12 -07:00
test_kprobes.c
…
torture.c
…
tracepoint.c
tracepoint: Use rcu get state and cond sync for static call updates
2021-09-03 10:09:30 +02:00
tsacct.c
taskstats: Cleanup the use of task->exit_code
2022-01-27 10:54:33 +01:00
ucount.c
Revert "Add a reference to ucounts for each cred"
2021-09-08 08:49:00 +02:00
uid16.c
…
uid16.h
…
umh.c
usermodehelper: reset umask to default before executing user process
2020-10-06 10:31:52 -07:00
up.c
smp: Fix smp_call_function_single_async prototype
2021-05-14 09:50:46 +02:00
user_namespace.c
Revert "Add a reference to ucounts for each cred"
2021-09-08 08:49:00 +02:00
user-return-notifier.c
…
user.c
…
usermode_driver.c
bpf: Fix umd memory leak in copy_process()
2021-03-30 14:32:03 +02:00
utsname_sysctl.c
…
utsname.c
…
watch_queue.c
watch_queue: Free the page array when watch_queue is dismantled
2022-04-08 14:40:41 +02:00
watchdog_hld.c
…
watchdog.c
watchdog: fix barriers when printing backtraces from all CPUs
2021-05-19 10:13:00 +02:00
workqueue_internal.h
…
workqueue.c
workqueue: Fix unbind_workers() VS wq_worker_running() race
2022-01-16 09:14:22 +01:00