linux/kernel/sched
Shrikanth Hegde be3a51e68f sched/fair: Add EAS checks before updating root_domain::overutilized
root_domain::overutilized is only used for EAS(energy aware scheduler)
to decide whether to do load balance or not. It is not used if EAS
not possible.

Currently enqueue_task_fair and task_tick_fair accesses, sometime updates
this field. In update_sd_lb_stats it is updated often. This causes cache
contention due to true sharing and burns a lot of cycles. ::overload and
::overutilized are part of the same cacheline. Updating it often invalidates
the cacheline. That causes access to ::overload to slow down due to
false sharing. Hence add EAS check before accessing/updating this field.
EAS check is optimized at compile time or it is a static branch.
Hence it shouldn't cost much.

With the patch, both enqueue_task_fair and newidle_balance don't show
up as hot routines in perf profile.

  6.8-rc4:
  7.18%  swapper          [kernel.vmlinux]              [k] enqueue_task_fair
  6.78%  s                [kernel.vmlinux]              [k] newidle_balance

  +patch:
  0.14%  swapper          [kernel.vmlinux]              [k] enqueue_task_fair
  0.00%  swapper          [kernel.vmlinux]              [k] newidle_balance

While at it: trace_sched_overutilized_tp expect that second argument to
be bool. So do a int to bool conversion for that.

Fixes: 2802bf3cd9 ("sched/fair: Add over-utilization/tipping point indicator")
Signed-off-by: Shrikanth Hegde <sshegde@linux.ibm.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Reviewed-by: Qais Yousef <qyousef@layalina.io>
Reviewed-by: Srikar Dronamraju <srikar@linux.ibm.com>
Reviewed-by: Vincent Guittot <vincent.guittot@linaro.org>
Link: https://lore.kernel.org/r/20240307085725.444486-2-sshegde@linux.ibm.com
2024-03-26 08:58:59 +01:00
..
autogroup.c sched/all: Change all BUG_ON() instances in the scheduler to WARN_ON_ONCE() 2022-08-12 11:25:10 +02:00
autogroup.h
build_policy.c
build_utility.c sched/headers: Remove duplicate header inclusions 2023-10-03 21:27:55 +02:00
clock.c Locking changes for v6.5: 2023-06-27 14:14:30 -07:00
completion.c sched: add a few helpers to wake up tasks on the current cpu 2023-07-17 16:08:08 -07:00
core_sched.c sched: Rename task_running() to task_on_cpu() 2022-09-07 21:53:47 +02:00
core.c Linux 6.9-rc1 2024-03-25 11:32:29 +01:00
cpuacct.c
cpudeadline.c sched/topology: Consolidate and clean up access to a CPU's max compute capacity 2023-10-09 12:59:48 +02:00
cpudeadline.h
cpufreq_schedutil.c sched/fair: Fix frequency selection for non-invariant case 2024-01-16 10:41:25 +01:00
cpufreq.c
cpupri.c sched/rt: Fix live lock between select_fallback_rq() and RT push 2023-09-28 22:58:13 +02:00
cpupri.h
cputime.c cputime: remove cputime_to_nsecs fallback 2022-12-27 12:52:17 +01:00
deadline.c sched/deadline: Introduce deadline servers 2023-11-15 09:57:51 +01:00
debug.c sched/fair: Simplify util_est 2023-12-23 15:59:58 +01:00
fair.c sched/fair: Add EAS checks before updating root_domain::overutilized 2024-03-26 08:58:59 +01:00
features.h sched/fair: Remove SCHED_FEAT(UTIL_EST_FASTUP, true) 2023-12-23 15:59:56 +01:00
idle.c Core x86 changes for v6.9: 2024-03-11 19:53:15 -07:00
isolation.c
loadavg.c sched/balancing: Rename scheduler_tick() => sched_tick() 2024-03-12 11:59:59 +01:00
Makefile
membarrier.c RISC-V Patches for the 6.9 Merge Window 2024-03-22 10:41:13 -07:00
pelt.c sched/balancing: Fix a couple of outdated function names in comments 2024-03-12 12:00:01 +01:00
pelt.h sched/fair: Simplify util_est 2023-12-23 15:59:58 +01:00
psi.c sched/psi: Update poll => rtpoll in relevant comments 2023-10-16 13:42:49 +02:00
rt.c sched: Unify runtime accounting across classes 2023-11-15 09:57:48 +01:00
sched-pelt.h
sched.h sched/topology: Remove root_domain::max_cpu_capacity 2024-03-25 12:09:56 +01:00
smp.h sched, smp: Trace smp callback causing an IPI 2023-03-24 11:01:29 +01:00
stats.c sched/debug: Increase SCHEDSTAT_VERSION to 16 2024-03-12 11:03:40 +01:00
stats.h sched/psi: Use task->psi_flags to clear in CPU migration 2022-10-30 10:12:15 +01:00
stop_task.c sched: Unify runtime accounting across classes 2023-11-15 09:57:48 +01:00
swait.c sched: add a few helpers to wake up tasks on the current cpu 2023-07-17 16:08:08 -07:00
topology.c sched/topology: Remove root_domain::max_cpu_capacity 2024-03-25 12:09:56 +01:00
wait_bit.c wait_on_bit: add an acquire memory barrier 2022-08-26 09:30:25 -07:00
wait.c sched: remove wait bookmarks 2023-10-18 14:34:18 -07:00