linux/kernel/sched
Vincent Guittot 98b0d89022 sched/pelt: Relax the sync of util_sum with util_avg
Rick reported performance regressions in bugzilla because of cpu frequency
being lower than before:
    https://bugzilla.kernel.org/show_bug.cgi?id=215045

He bisected the problem to:
commit 1c35b07e6d ("sched/fair: Ensure _sum and _avg values stay consistent")

This commit forces util_sum to be synced with the new util_avg after
removing the contribution of a task and before the next periodic sync. By
doing so util_sum is rounded to its lower bound and might lost up to
LOAD_AVG_MAX-1 of accumulated contribution which has not yet been
reflected in util_avg.

Instead of always setting util_sum to the low bound of util_avg, which can
significantly lower the utilization of root cfs_rq after propagating the
change down into the hierarchy, we revert the change of util_sum and
propagate the difference.

In addition, we also check that cfs's util_sum always stays above the
lower bound for a given util_avg as it has been observed that
sched_entity's util_sum is sometimes above cfs one.

Fixes: 1c35b07e6d ("sched/fair: Ensure _sum and _avg values stay consistent")
Reported-by: Rick Yiu <rickyiu@google.com>
Signed-off-by: Vincent Guittot <vincent.guittot@linaro.org>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Dietmar Eggemann <dietmar.eggemann@arm.com>
Tested-by: Sachin Sant <sachinp@linux.ibm.com>
Link: https://lkml.kernel.org/r/20220111134659.24961-2-vincent.guittot@linaro.org
2022-01-18 12:09:58 +01:00
..
autogroup.c sched/fair: Prevent dead task groups from regaining cfs_rq's 2021-11-11 13:09:33 +01:00
autogroup.h
clock.c sched: Fix various typos 2021-03-22 00:11:52 +01:00
completion.c
core_sched.c sched/core: Forced idle accounting 2021-11-17 14:49:00 +01:00
core.c Peter Zijlstra says: 2022-01-11 17:24:45 -08:00
cpuacct.c sched/cpuacct: Make user/system times in cpuacct.stat more precise 2021-11-23 09:55:22 +01:00
cpudeadline.c sched,rt: Use the full cpumask for balancing 2020-11-10 18:39:00 +01:00
cpudeadline.h
cpufreq_schedutil.c sched/fair: Replace CFS internal cpu_util() with cpu_util_cfs() 2021-12-11 09:10:00 +01:00
cpufreq.c
cpupri.c sched: Fix various typos 2021-03-22 00:11:52 +01:00
cpupri.h sched/cpupri: Add CPUPRI_HIGHER 2020-10-29 11:00:30 +01:00
cputime.c Peter Zijlstra says: 2022-01-11 17:14:59 -08:00
deadline.c sched/dl: Support schedstats for deadline sched class 2021-10-05 15:51:53 +02:00
debug.c sched/core: Forced idle accounting 2021-11-17 14:49:00 +01:00
fair.c sched/pelt: Relax the sync of util_sum with util_avg 2022-01-18 12:09:58 +01:00
features.h sched: Disable TTWU_QUEUE on RT 2021-10-05 15:52:12 +02:00
idle.c sched/idle: Make the idle timer expire in hard interrupt context 2021-09-09 10:36:16 +02:00
isolation.c sched/isolation: Reconcile rcu_nocbs= and nohz_full= 2021-05-13 14:12:47 +02:00
loadavg.c sched: Make multiple runqueue task counters 32-bit 2021-05-12 21:34:17 +02:00
Makefile sched, kcsan: Enable memory barrier instrumentation 2021-12-09 16:42:28 -08:00
membarrier.c sched/membarrier: fix missing local execution of ipi_sync_rq_state() 2021-03-06 12:40:21 +01:00
pelt.c sched: Fix various typos 2021-03-22 00:11:52 +01:00
pelt.h sched/pelt: Relax the sync of util_sum with util_avg 2022-01-18 12:09:58 +01:00
psi.c psi: Fix uaf issue when psi trigger is destroyed while being polled 2022-01-18 12:09:57 +01:00
rt.c sched/rt: Try to restart rt period timer when rt runtime exceeded 2021-12-07 15:14:10 +01:00
sched-pelt.h
sched.h sched/fair: Replace CFS internal cpu_util() with cpu_util_cfs() 2021-12-11 09:10:00 +01:00
smp.h
stats.c sched: Introduce task block time in schedstats 2021-10-05 15:51:48 +02:00
stats.h psi: Fix PSI_MEM_FULL state when tasks are in memstall and doing reclaim 2021-11-17 14:49:00 +01:00
stop_task.c sched: Make struct sched_statistics independent of fair sched class 2021-10-05 15:51:45 +02:00
swait.c
topology.c Merge branch 'akpm' (patches from Andrew) 2021-11-06 14:08:17 -07:00
wait_bit.c
wait.c wait: add wake_up_pollfree() 2021-12-09 10:49:56 -08:00