linux/tools/perf/tests
Namhyung Kim 4d60e83dfc perf test: Skip metrics w/o event name in stat STD output linter
This test checks if the output of perf stat to match event names and
metrics.  So it wants the output lines to have both event name and
metric.  Otherwise it should skip the line.

On AMD machines, the instruction event has two metrics and they are printed
in separate lines.  It makes the line without event name like below:

  # perf stat -a sleep 1

   Performance counter stats for 'system wide':

           64,383.34 msec cpu-clock                  #   64.048 CPUs utilized
              14,526      context-switches           #  225.617 /sec
                 112      cpu-migrations             #    1.740 /sec
                 190      page-faults                #    2.951 /sec
         807,558,652      cycles                     #    0.013 GHz                         (83.30%)
          69,809,799      stalled-cycles-frontend    #    8.64% frontend cycles idle        (83.30%)
         196,983,266      stalled-cycles-backend     #   24.39% backend cycles idle         (83.30%)
         424,876,008      instructions               #    0.53  insn per cycle
 (here) --->                                  #    0.46  stalled cycles per insn     (83.30%)
          97,788,321      branches                   #    1.519 M/sec                       (83.34%)
           4,147,377      branch-misses              #    4.24% of all branches             (83.46%)

         1.005241409 seconds time elapsed

Also modern Intel machines have TopDown metrics which also don't have
event names.

  # perf stat -a sleep 1

   Performance counter stats for 'system wide':

            8,015.39 msec cpu-clock                        #    7.996 CPUs utilized
               5,823      context-switches                 #  726.477 /sec
                 189      cpu-migrations                   #   23.580 /sec
                 139      page-faults                      #   17.342 /sec
         435,139,308      cycles                           #    0.054 GHz
         193,891,345      instructions                     #    0.45  insn per cycle
          42,773,028      branches                         #    5.336 M/sec
           2,298,113      branch-misses                    #    5.37% of all branches
                          TopdownL1                 #     25.5 %  tma_backend_bound
              /-->                                  #      7.9 %  tma_bad_speculation
    (here) --+                                      #     55.7 %  tma_frontend_bound
              \-->                                  #     10.9 %  tma_retiring

         1.002395924 seconds time elapsed

There is a check to skip TopdownL1 and TopdownL2 specifically but it
does not cover every affected lines.

So there is another check to skip the line if it has nothing on the left
side of # sign.  Well.. it seems ok but that's not enough too.

When aggregation mode (like --per-socket or --per-thread) is used, it
adds some prefix (e.g. CPU socket, task name and PID) in the output
line.  So the test code ignores them to normalize result.

A problem can happen for per-thread mode when task name contains one or
more spaces.  It'd only ignore the first part of the task name, and it
thinks there's something more in the line so it would not skip.

  # perf stat -a --perf-thread sleep 1
  ...
            perf-21276                  #     70.2 %  tma_backend_bound
            perf-21276                  #      3.9 %  tma_bad_speculation
            perf-21276                  #     10.5 %  tma_frontend_bound
            perf-21276                  #     15.3 %  tma_retiring
	    ^^^^^^^^^^
	    (ignored)

         my task-21328                  #     70.2 %  tma_backend_bound
         my task-21328                  #      3.9 %  tma_bad_speculation
         my task-21328                  #     10.5 %  tma_frontend_bound
         my task-21328                  #     15.3 %  tma_retiring
	 ^^
     (ignored)

So I think it should look at the metric names instead.  Add skip_metric
to hold the list of names to skip.  It would contain 'stalled cycles per
insn' and metrics started by 'tma_'.

Fixes: 99a04a48f2 ("perf test: Add test case for the standard 'perf stat' output")
Acked-by: Ian Rogers <irogers@google.com>
Cc: Kan Liang <kan.liang@linux.intel.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Arnaldo Carvalho de Melo <acme@kernel.org>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Ingo Molnar <mingo@kernel.org>
Link: https://lore.kernel.org/r/20230623230139.985594-2-namhyung@kernel.org
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
2023-06-23 21:35:45 -07:00
..
attr perf test attr: Update no event/metric expectations 2023-05-19 10:36:25 -03:00
shell perf test: Skip metrics w/o event name in stat STD output linter 2023-06-23 21:35:45 -07:00
workloads perf test workload thloop: Make count increments atomic 2023-01-19 13:42:06 -03:00
.gitignore
api-io.c perf tests api-io: Use zfree() to reduce chances of use after free 2023-04-12 10:14:58 -03:00
attr.c perf pmus: Remove perf_pmus__has_hybrid 2023-05-27 09:42:38 -03:00
attr.py perf test attr: Fix python SafeConfigParser() deprecation warning 2023-05-19 10:36:35 -03:00
backward-ring-buffer.c perf test: Rename struct test to test_suite 2021-11-13 10:32:22 -03:00
bitmap.c perf tools: Use dedicated non-atomic clear/set bit helpers 2022-12-05 09:29:06 -03:00
bp_account.c perf test: Fix variable length array undefined behavior in bp_account 2022-06-19 10:41:43 -03:00
bp_signal_overflow.c perf test: BP tests, remove is_supported use 2021-11-13 18:11:50 -03:00
bp_signal.c perf test: BP tests, remove is_supported use 2021-11-13 18:11:50 -03:00
bpf-script-example.c perf test: Switch basic bpf filtering test to use syscall tracepoint 2023-01-23 09:58:01 -03:00
bpf-script-test-kbuild.c
bpf-script-test-prologue.c perf tests bpf prologue: Fix bpf-script-test-prologue test compile issue with clang 2023-01-09 11:57:47 -03:00
bpf-script-test-relocation.c
bpf.c perf evlist: Remove nr_groups 2023-03-13 17:42:27 -03:00
Build perf test: Add Symbols test 2023-01-22 18:09:56 -03:00
builtin-test-list.c perf test: Refactor shell tests allowing subdirs 2022-08-13 15:13:20 -03:00
builtin-test-list.h perf test: Refactor shell tests allowing subdirs 2022-08-13 15:13:20 -03:00
builtin-test.c perf tests: Organize cpu_map tests into a single suite 2023-05-26 22:47:22 -03:00
clang.c perf test: Remove skip_if_fail 2021-11-13 18:11:50 -03:00
code-reading.c perf machine: Make delete_threads part of machine__exit 2023-06-12 15:57:53 -03:00
cpumap.c perf cpumap: Add equal function 2023-05-27 09:36:44 -03:00
demangle-java-test.c perf test: Rename struct test to test_suite 2021-11-13 10:32:22 -03:00
demangle-ocaml-test.c perf test: Rename struct test to test_suite 2021-11-13 10:32:22 -03:00
dlfilter-test.c perf tools: Fix usage of the verbose variable 2022-12-20 15:16:33 -03:00
dso-data.c perf test: Rename struct test to test_suite 2021-11-13 10:32:22 -03:00
dwarf-unwind.c perf machine: Make delete_threads part of machine__exit 2023-06-12 15:57:53 -03:00
event_groups.c perf pmu: Separate pmu and pmus 2023-05-27 09:41:39 -03:00
event_update.c perf events: Prefer union over variable length array 2022-10-04 08:55:21 -03:00
event-times.c perf tools: Do not pass NULL to parse_events() 2022-08-10 14:30:09 -03:00
evsel-roundtrip-name.c perf test: Roundtrip name, don't assume 1 event per name 2023-05-15 09:12:13 -03:00
evsel-tp-sched.c perf test: Rename struct test to test_suite 2021-11-13 10:32:22 -03:00
expand-cgroup.c perf evsel: Introduce evsel__name_is() method to check if the evsel name is equal to a given string 2023-04-24 14:28:11 -03:00
expr.c perf expr: Make the evaluation of & and | logical and lazy 2023-06-05 15:56:14 -03:00
fdarray.c perf test: Rename struct test to test_suite 2021-11-13 10:32:22 -03:00
genelf.c perf test: Rename struct test to test_suite 2021-11-13 10:32:22 -03:00
hists_common.c perf thread: Add accessor functions for thread 2023-06-12 15:57:53 -03:00
hists_common.h
hists_cumulate.c perf addr_location: Add init/exit/copy functions 2023-06-12 15:57:53 -03:00
hists_filter.c perf addr_location: Add init/exit/copy functions 2023-06-12 15:57:53 -03:00
hists_link.c perf thread: Add reference count checking 2023-06-12 15:57:53 -03:00
hists_output.c perf addr_location: Add init/exit/copy functions 2023-06-12 15:57:53 -03:00
is_printable_array.c perf test: Rename struct test to test_suite 2021-11-13 10:32:22 -03:00
keep-tracking.c perf tools: Do not pass NULL to parse_events() 2022-08-10 14:30:09 -03:00
kmod-path.c perf test: Rename struct test to test_suite 2021-11-13 10:32:22 -03:00
llvm.c perf: Stop using bpf_object__open_buffer() API 2022-01-24 20:44:09 -08:00
llvm.h
make perf tests: Make x86 new instructions test optional at build time 2023-06-13 23:40:32 -03:00
maps.c perf maps: Make delete static, always use put 2023-06-12 15:57:53 -03:00
mem2node.c perf tools: Use dedicated non-atomic clear/set bit helpers 2022-12-05 09:29:06 -03:00
mem.c perf test: Rename struct test to test_suite 2021-11-13 10:32:22 -03:00
mmap-basic.c perf tests: Add missing event.h include 2022-10-31 11:07:08 -03:00
mmap-thread-lookup.c perf machine: Make delete_threads part of machine__exit 2023-06-12 15:57:53 -03:00
openat-syscall-all-cpus.c perf tests: Avoid pthread.h inclusion 2022-10-04 08:55:19 -03:00
openat-syscall-tp-fields.c perf tools: Move 'struct perf_sample' to a separate header file to disentangle headers 2022-10-31 11:06:41 -03:00
openat-syscall.c perf thread_map: Reduce exposure of libperf internal API 2022-11-16 12:17:15 -03:00
parse-events.c perf parse: Allow config terms with breakpoints 2023-06-12 15:57:53 -03:00
parse-metric.c perf pmus: Remove perf_pmus__has_hybrid 2023-05-27 09:42:38 -03:00
parse-no-sample-id-all.c perf tools: Move 'struct perf_sample' to a separate header file to disentangle headers 2022-10-31 11:06:41 -03:00
pe-file-parsing.c perf test: Rename struct test to test_suite 2021-11-13 10:32:22 -03:00
pe-file.c
pe-file.exe
pe-file.exe.debug
perf-hooks.c perf test: Rename struct test to test_suite 2021-11-13 10:32:22 -03:00
perf-record.c perf tests: Add missing event.h include 2022-10-31 11:07:08 -03:00
perf-targz-src-pkg perf thread: Add accessor functions for thread 2023-06-12 15:57:53 -03:00
perf-time-to-tsc.c perf tools: Move 'struct perf_sample' to a separate header file to disentangle headers 2022-10-31 11:06:41 -03:00
pfm.c perf evlist: Remove nr_groups 2023-03-13 17:42:27 -03:00
pmu-events.c perf pmus: Allow just core PMU scanning 2023-05-27 09:42:00 -03:00
pmu.c perf test pmu: Avoid 2 static path arrays 2023-05-28 10:25:45 -03:00
python-use.c perf test python: Put perf python at start of sys.path 2023-05-27 09:35:14 -03:00
sample-parsing.c perf script: Show branch speculation info 2023-02-02 17:18:31 -03:00
sdt.c perf test: Rename struct test to test_suite 2021-11-13 10:32:22 -03:00
sigtrap.c perf test: Skip sigtrap test on old kernels 2022-10-04 08:55:22 -03:00
stat.c perf cpumap: Give CPUs their own type 2022-01-12 14:28:23 -03:00
sw-clock.c perf tools: Move 'struct perf_sample' to a separate header file to disentangle headers 2022-10-31 11:06:41 -03:00
switch-tracking.c perf pmus: Remove perf_pmus__has_hybrid 2023-05-27 09:42:38 -03:00
symbols.c perf machine: Make delete_threads part of machine__exit 2023-06-12 15:57:53 -03:00
task-exit.c perf test: Rename struct test to test_suite 2021-11-13 10:32:22 -03:00
tests.h perf tests: Organize cpu_map tests into a single suite 2023-05-26 22:47:22 -03:00
thread-map.c perf thread_map: Reduce exposure of libperf internal API 2022-11-16 12:17:15 -03:00
thread-maps-share.c perf thread: Add accessor functions for thread 2023-06-12 15:57:53 -03:00
time-utils-test.c perf test: Rename struct test to test_suite 2021-11-13 10:32:22 -03:00
topology.c perf pmus: Remove perf_pmus__has_hybrid 2023-05-27 09:42:38 -03:00
unit_number__scnprintf.c perf test: Rename struct test to test_suite 2021-11-13 10:32:22 -03:00
vmlinux-kallsyms.c perf map: Add set_ methods for map->{start,end,pgoff,pgoff,reloc,erange_warned,dso,map_ip,unmap_ip,priv} 2023-04-19 12:54:41 -03:00
wp.c perf test: Skip watchpoint tests if no watchpoints available 2022-11-23 10:32:53 -03:00