linux/tools/perf/tests/shell
Veronika Molnarova e8b86f0311 perf test stat_bpf_counter.sh: Stabilize the test results
The test has been failing for some time when two separate runs of
perf benchmarks are recorded for cycles events and their counts are
compared, while once the recording was done with option --bpf-counters
and once without it. It is expected that the count of the samples
should be within a certain range, firstly the difference was set to be
within 10%, which was then later raised to 20%. However, the test case
keeps failing on certain architectures as recording the provided
benchmark can produce completely different counts based on the
current load of the system.

Sampling two separate runs on intel-eaglestream-spr-13 of "perf stat
--no-big-num -e cycles -- perf bench sched messaging -g 1 -l 100 -t":

 Performance counter stats for 'perf bench sched messaging -g 1 -l 100 -t':

         396782898      cycles

       0.010051983 seconds time elapsed

       0.008664000 seconds user
       0.097058000 seconds sys

 Performance counter stats for 'perf bench sched messaging -g 1 -l 100 -t':

        1431133032      cycles

       0.021803714 seconds time elapsed

       0.023377000 seconds user
       0.349918000 seconds sys

, which is ranging from 400mil to 1400mil samples.

Instead of recording the cycles use instructions event, which provides
more stable values. At the same time change the tested workload to one
of the provided testing workloads by perf that is not based on a
scheduler, which can provide another dependency on the current load.

Sampling instructions event with the new workload provide much more
stable results on intel-eaglestream-spr-13 of "perf stat --no-big-num
-e instructions -- perf test -w brstack":

 Performance counter stats for 'perf test -w brstack':

          64584494      instructions

       0.009173945 seconds time elapsed

       0.007262000 seconds user
       0.002071000 seconds sys

 Performance counter stats for 'perf test -w brstack':

          64672669      instructions

       0.008888135 seconds time elapsed

       0.005018000 seconds user
       0.004018000 seconds sys

Signed-off-by: Veronika Molnarova <vmolnaro@redhat.com>
Acked-by: Namhyung Kim <namhyung@kernel.org>
Cc: mpetlan@redhat.com
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
Link: https://lore.kernel.org/r/20240625092001.10909-1-vmolnaro@redhat.com
2024-06-26 11:10:58 -07:00
..
base_probe tools/perf: Handle perftool-testsuite_probe testcases fail when kernel debuginfo is not present 2024-06-20 10:05:04 -07:00
common perf testsuite: Add common output checking helpers 2024-02-16 11:49:36 -08:00
coresight perf tests coresight: Remove unused variables 2023-11-27 11:35:43 -03:00
lib perf test: Stat output per thread of just the parent process 2024-03-21 13:54:39 -03:00
annotate.sh perf test: Speed up test case 70 annotate basic tests 2024-06-07 13:06:44 -07:00
buildid.sh perf tests shell: Fixed shellcheck warnings 2023-06-13 23:40:34 -03:00
daemon.sh perf test shell daemon: Make signal test less racy 2024-01-26 10:51:49 -03:00
diff.sh perf test shell diff: Skip test if test_loop symbol is missing in the perf binary 2023-12-06 13:01:36 -03:00
list.sh perf test: Workaround debug output in list test 2024-01-26 10:51:49 -03:00
lock_contention.sh perf tests: Ignore shellcheck warning in lock_contention 2023-10-17 12:40:51 -07:00
perftool-testsuite_probe.sh perf testsuite: Add test case for perf probe 2024-02-16 11:49:22 -08:00
pipe_test.sh perf tests: Skip pipe test if noploop symbol is missing 2023-11-27 15:40:00 -03:00
probe_vfs_getname.sh tests/shell: Fix shellcheck SC1090 to handle the location of sourced files 2023-09-26 21:47:12 -07:00
record_bpf_filter.sh perf test shell record_bpf_filter: Skip 6.2 kernel 2023-08-29 14:16:14 -03:00
record_offcpu.sh perf tests offcpu: Adjust test case perf record offcpu profiling tests for s390 2023-11-09 13:49:33 -03:00
record_sideband.sh perf tests: Fix shellcheck warning in record_sideband.sh 2023-10-17 12:40:51 -07:00
record.sh perf tests: Skip record test if test_loop symbol is missing 2023-11-27 15:40:16 -03:00
record+probe_libc_inet_pton.sh perf test record+probe_libc_inet_pton: Fix call chain match on powerpc 2023-11-29 17:59:36 -03:00
record+script_probe_vfs_getname.sh tests/shell: Fix shellcheck SC1090 to handle the location of sourced files 2023-09-26 21:47:12 -07:00
record+zstd_comp_decomp.sh perf tests record+zstd_comp_decomp: Fix the shellcheck warnings about word splitting/quoting 2023-08-03 17:01:27 -03:00
script.sh perf scripts python: Add a script to run instances of 'perf script' in parallel 2024-04-26 22:07:19 -03:00
stat_all_metricgroups.sh perf tests: Fix shellcheck warning in stat_all_metricgroups 2023-10-17 12:40:51 -07:00
stat_all_metrics.sh perf tests stat_all_metrics: Fix shellcheck warning SC2076 2023-06-13 23:40:33 -03:00
stat_all_pfm.sh perf test: Add test of libpfm4 events 2023-06-12 15:57:53 -03:00
stat_all_pmu.sh perf tools: Fix spelling mistake "parametrized" -> "parameterized" 2023-11-09 13:49:32 -03:00
stat_bpf_counters_cgrp.sh perf test stat_bpf_counters_cgrp: Enhance perf stat cgroup BPF counter test 2023-08-29 14:16:15 -03:00
stat_bpf_counters.sh perf test stat_bpf_counter.sh: Stabilize the test results 2024-06-26 11:10:58 -07:00
stat_metrics_values.sh perf test: Simplify metric value validation test final report 2024-02-01 22:16:37 -08:00
stat.sh perf test: Add cputype testing to perf stat 2023-05-15 09:18:43 -03:00
stat+csv_output.sh perf stat: Support per-cluster aggregation 2024-02-09 14:59:53 -08:00
stat+csv_summary.sh tests/shell: Fix shellcheck issues in tests/shell/stat+shadow_stat.sh tetscase 2023-09-26 21:47:12 -07:00
stat+json_output.sh perf test: Stat output per thread of just the parent process 2024-03-21 13:54:39 -03:00
stat+shadow_stat.sh tests/shell: Fix shellcheck issues in tests/shell/stat+shadow_stat.sh tetscase 2023-09-26 21:47:12 -07:00
stat+std_output.sh perf stat: Support per-cluster aggregation 2024-02-09 14:59:53 -08:00
test_arm_callgraph_fp.sh perf test: Make test_arm_callgraph_fp.sh more robust 2024-06-24 14:42:59 -07:00
test_arm_coresight.sh perf test shell arm_coresight: Increase buffer size for Coresight basic tests 2024-04-18 22:22:51 -03:00
test_arm_spe_fork.sh perf tests: Address signal case issues detected via shellcheck 2023-08-03 17:01:25 -03:00
test_arm_spe.sh perf tests test_arm_spe: Address shellcheck warnings about signal name case 2023-06-13 23:40:33 -03:00
test_brstack.sh perf tests: Skip branch stack sampling test if brstack_bench symbol is missing 2023-11-27 15:40:22 -03:00
test_data_symbol.sh perf tests: Skip data symbol test if buf1 symbol is missing 2023-11-27 15:40:28 -03:00
test_intel_pt.sh tests/shell: Fix shellcheck SC1090 to handle the location of sourced files 2023-09-26 21:47:12 -07:00
test_java_symbol.sh perf test java symbol: Remove needless debuginfod queries 2023-05-10 12:54:53 -03:00
test_perf_data_converter_json.sh perf test: Use common python setup library 2023-12-05 15:48:52 -03:00
test_task_analyzer.sh perf tests task_analyzer: Check perf build options for libtraceevent support 2023-08-03 17:01:25 -03:00
test_uprobe_from_different_cu.sh perf test: Check output of the probe ... --funcs command 2024-06-25 11:06:19 -07:00
trace+probe_vfs_getname.sh tests/shell: Fix shellcheck SC1090 to handle the location of sourced files 2023-09-26 21:47:12 -07:00