linux/tools
Ryan Roberts 05f1edac80 selftests/mm: run all tests from run_vmtests.sh
It is very unclear to me how one is supposed to run all the mm selftests
consistently and get clear results.

Most of the test programs are launched by both run_vmtests.sh and
run_kselftest.sh:

  hugepage-mmap
  hugepage-shm
  map_hugetlb
  hugepage-mremap
  hugepage-vmemmap
  hugetlb-madvise
  map_fixed_noreplace
  gup_test
  gup_longterm
  uffd-unit-tests
  uffd-stress
  compaction_test
  on-fault-limit
  map_populate
  mlock-random-test
  mlock2-tests
  mrelease_test
  mremap_test
  thuge-gen
  virtual_address_range
  va_high_addr_switch
  mremap_dontunmap
  hmm-tests
  madv_populate
  memfd_secret
  ksm_tests
  ksm_functional_tests
  soft-dirty
  cow

However, of this set, when launched by run_vmtests.sh, some of the
programs are invoked multiple times with different arguments. When
invoked by run_kselftest.sh, they are invoked without arguments (and as
a consequence, some fail immediately).

Some test programs are only launched by run_vmtests.sh:

  test_vmalloc.sh

And some test programs and only launched by run_kselftest.sh:

  khugepaged
  migration
  mkdirty
  transhuge-stress
  split_huge_page_test
  mdwe_test
  write_to_hugetlbfs

Furthermore, run_vmtests.sh is invoked by run_kselftest.sh, so in this
case all the test programs invoked by both scripts are run twice!

Needless to say, this is a bit of a mess. In the absence of fully
understanding the history here, it looks to me like the best solution is
to launch ALL test programs from run_vmtests.sh, and ONLY invoke
run_vmtests.sh from run_kselftest.sh. This way, we get full control over
the parameters, each program is only invoked the intended number of
times, and regardless of which script is used, the same tests get run in
the same way.

The only drawback is that if using run_kselftest.sh, it's top-level tap
result reporting reports only a single test and it fails if any of the
contained tests fail. I don't see this as a big deal though since we
still see all the nested reporting from multiple layers. The other issue
with this is that all of run_vmtests.sh must execute within a single
kselftest timeout period, so let's increase that to something more
suitable.

In the Makefile, TEST_GEN_PROGS will compile and install the tests and
will add them to the list of tests that run_kselftest.sh will run.
TEST_GEN_FILES will compile and install the tests but will not add them
to the test list. So let's move all the programs from TEST_GEN_PROGS to
TEST_GEN_FILES so that they are built but not executed by
run_kselftest.sh. Note that run_vmtests.sh is added to TEST_PROGS, which
means it ends up in the test list. (the lack of "_GEN" means it won't be
compiled, but simply copied).

Link: https://lkml.kernel.org/r/20230724082522.1202616-9-ryan.roberts@arm.com
Signed-off-by: Ryan Roberts <ryan.roberts@arm.com>
Acked-by: David Hildenbrand <david@redhat.com>
Acked-by: Peter Xu <peterx@redhat.com>
Cc: Florent Revest <revest@chromium.org>
Cc: Jérôme Glisse <jglisse@redhat.com>
Cc: John Hubbard <jhubbard@nvidia.com>
Cc: Mark Brown <broonie@kernel.org>
Cc: Shuah Khan <shuah@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
2023-08-18 10:12:43 -07:00
..
accounting
arch tools headers arm64: Sync arm64's cputype.h with the kernel sources 2023-07-14 10:36:16 -03:00
bootconfig
bpf
build perf build: Fix broken feature check for libtracefs due to external lib changes 2023-07-11 13:17:05 -03:00
certs
cgroup
counter
debugging
edid
firewire
firmware
gpio
hv
iio
include selftests: line buffer test program's stdout 2023-08-18 10:12:42 -07:00
io_uring
kvm/kvm_stat
laptop
leds
lib libsubcmd: Avoid SEGV/use-after-free when commands aren't excluded 2023-07-11 16:44:12 -03:00
memory-model
mm
net/ynl tools: ynl-gen: fix parse multi-attr enum attribute 2023-07-26 13:38:23 -07:00
objtool objtool: initialize all of struct elf 2023-07-10 09:52:28 +02:00
pci
pcmcia
perf perf test task_exit: No need for a cycles event to check if we get an PERF_RECORD_EXIT 2023-07-17 10:27:44 -03:00
power platform-drivers-x86 for v6.5-1 2023-06-30 14:50:00 -07:00
rcu
scripts
spi
testing selftests/mm: run all tests from run_vmtests.sh 2023-08-18 10:12:43 -07:00
thermal
time
tracing rtla/timerlat_hist: Add timerlat user-space support 2023-06-13 16:41:14 -04:00
usb
verification
virtio tools/virtio: fix build break for aarch64 2023-06-27 10:47:08 -04:00
wmi
workqueue
Makefile