Commit Graph

38 Commits

Author SHA1 Message Date
Daniel Latypov
a127b154a8 kunit: tool: allow filtering test cases via glob
Commit 1d71307a6f ("kunit: add unit test for filtering suites by
names") introduced the ability to filter which suites we run via glob.

This change extends it so we can also filter individual test cases
inside of suites as well.

This is quite useful when, e.g.
* trying to run just the tests cases you've just added or are working on
* trying to debug issues with test hermeticity

Examples:
$ ./tools/testing/kunit/kunit.py run --kunitconfig=lib/kunit '*exec*.parse*'
...
============================================================
======== [PASSED] kunit_executor_test ========
[PASSED] parse_filter_test
============================================================
Testing complete. 1 tests run. 0 failed. 0 crashed.

$ ./tools/testing/kunit/kunit.py run --kunitconfig=lib/kunit '*.no_matching_tests'
...
[ERROR] no tests run!

Signed-off-by: Daniel Latypov <dlatypov@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2021-10-19 14:18:49 -06:00
Daniel Latypov
6a499c9c42 kunit: tool: make --raw_output support only showing kunit output
--raw_output is nice, but it would be nicer if could show only output
after KUnit tests have started.

So change the flag to allow specifying a string ('kunit').
Make it so `--raw_output` alone will default to `--raw_output=all` and
have the same original behavior.

Drop the small kunit_parser.raw_output() function since it feels wrong
to put it in "kunit_parser.py" when the point of it is to not parse
anything.

E.g.

$ ./tools/testing/kunit/kunit.py run --raw_output=kunit
...
[15:24:07] Starting KUnit Kernel ...
TAP version 14
1..1
    # Subtest: example
    1..3
    # example_simple_test: initializing
    ok 1 - example_simple_test
    # example_skip_test: initializing
    # example_skip_test: You should not see a line below.
    ok 2 - example_skip_test # SKIP this test should be skipped
    # example_mark_skipped_test: initializing
    # example_mark_skipped_test: You should see a line below.
    # example_mark_skipped_test: You should see this line.
    ok 3 - example_mark_skipped_test # SKIP this test should be skipped
ok 1 - example
[15:24:10] Elapsed time: 6.487s total, 0.001s configuring, 3.510s building, 0.000s running

Signed-off-by: Daniel Latypov <dlatypov@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2021-08-13 13:32:01 -06:00
Daniel Latypov
6cb51a1874 kunit: tool: add --kernel_args to allow setting module params
kunit.py currently does not make it possible for users to specify module
parameters (/kernel arguments more generally) unless one directly tweaks
the kunit.py code itself.

This hasn't mattered much so far, but this would make it easier to port
existing tests that expose module parameters over to KUnit and/or let
current KUnit tests take advantage of them.

Tested using an kunit internal parameter:
$ ./tools/testing/kunit/kunit.py run --kunitconfig=lib/kunit \
    --kernel_args=kunit.filter_glob=kunit_status
...
Testing complete. 2 tests run. 0 failed. 0 crashed. 0 skipped.

Signed-off-by: Daniel Latypov <dlatypov@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2021-08-13 13:28:47 -06:00
Daniel Latypov
a72fa63043 Documentation: kunit: drop obsolete note about uml_abort for coverage
Commit b6d5799b0b ("kunit: Add 'kunit_shutdown' option") changes KUnit
to call kernel_halt() by default when done testing.

This fixes the issue with not having .gcda files due to not calling
atexit() handlers, and therefore we can stop recommending people
manually tweak UML code.

The need to use older versions of GCC (<=6) remains however, due to
linktime issues, same as before. Note: There also might still be issues
with .gcda files as well in newer versions.

Signed-off-by: Daniel Latypov <dlatypov@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2021-07-12 13:54:12 -06:00
Linus Torvalds
a48ad6e7a3 linux-kselftest-kunit-fixes-5.14-rc1
This KUnit update for Linux 5.14-rc1 consists of fixes and features:
 
 -- add support for skipped tests
 -- introduce kunit_kmalloc_array/kunit_kcalloc() helpers
 -- add gnu_printf specifiers
 -- add kunit_shutdown
 -- add unit test for filtering suites by names
 -- convert lib/test_list_sort.c to use KUnit
 -- code organization moving default config to tools/testing/kunit
 -- refactor of internal parser input handling
 -- cleanups and updates to documentation
 -- code cleanup related to casts
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCgAdFiEEPZKym/RZuOCGeA/kCwJExA0NQxwFAmDfNHgACgkQCwJExA0N
 QxzU5g/+ORbcbE5jhumo82xYUCGTScgIqPllNO000Vk9xydMqwx6tpY5puV0ig2z
 5X7pxKEQTnype58yzY5mS1p336ryIx3q+AwEownxTW3YurXo2naQ59ZPjEvlAV+E
 h+DMYLzEjrxzcyJATw02uC9YLUZ3w6FPJfiXViIv93YYrtcnM0u6JpwG0yfBlI4v
 4KKB2Xu4K7T90C9/ADFYFKX3mjXQl5fQwvIdtA7wS90Cgq52LKp2mvg1XEiZE5d+
 0dkTZ4Zo8TxxHt665o7vfnUjQQNmh45iGlW65wONxfAPb8BPoneGjVKWQTnN0Hor
 W+93ZPbMuFMSSKJuoHY9U7sP5VySKvaiIYaGdi6prnZZu0zUabKnLZ6FOy7kEdfs
 v09ulCBTVLslixVgNcp/kD9T+G/SXwCF5YBMAiMDQ0GNfUqlFtBkEA3gd44KwMI0
 KwCcOgUSiaCkqyzOz/VeQsu/nhA5jdMO0KjiAs7Z3e7r7O/qKFs/ll7hZgDNCWSC
 q8eIrcBkSL0EGgXR1iZ4AtGm8op6KKd4ACBM8NdtTyoGFl1npZOgZnHoIsy35G9K
 9mhc7eXSoaDGqy9dONL1Tc8Neg7qLTXQNp2radqsnAAgNPUrJuC7+8YC+DdIsjBH
 W7OyMjpfbwPws5rP4CS+JdwL+nQprKXZvFIhWGYhkDK44MbOngw=
 =5QAv
 -----END PGP SIGNATURE-----

Merge tag 'linux-kselftest-kunit-fixes-5.14-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest

Pull KUnit update from Shuah Khan:
 "Fixes and features:

   - add support for skipped tests

   - introduce kunit_kmalloc_array/kunit_kcalloc() helpers

   - add gnu_printf specifiers

   - add kunit_shutdown

   - add unit test for filtering suites by names

   - convert lib/test_list_sort.c to use KUnit

   - code organization moving default config to tools/testing/kunit

   - refactor of internal parser input handling

   - cleanups and updates to documentation

   - code cleanup related to casts"

* tag 'linux-kselftest-kunit-fixes-5.14-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest: (29 commits)
  kunit: add unit test for filtering suites by names
  kasan: test: make use of kunit_skip()
  kunit: test: Add example tests which are always skipped
  kunit: tool: Support skipped tests in kunit_tool
  kunit: Support skipped tests
  thunderbolt: test: Reinstate a few casts of bitfields
  kunit: tool: internal refactor of parser input handling
  lib/test: convert lib/test_list_sort.c to use KUnit
  kunit: introduce kunit_kmalloc_array/kunit_kcalloc() helpers
  kunit: Remove the unused all_tests.config
  kunit: Move default config from arch/um -> tools/testing/kunit
  kunit: arch/um/configs: Enable KUNIT_ALL_TESTS by default
  kunit: Add gnu_printf specifiers
  lib/cmdline_kunit: Remove a cast which are no-longer required
  kernel/sysctl-test: Remove some casts which are no-longer required
  thunderbolt: test: Remove some casts which are no longer required
  mmc: sdhci-of-aspeed: Remove some unnecessary casts from KUnit tests
  iio: Remove a cast in iio-test-format which is no longer required
  device property: Remove some casts in property-entry-test
  Documentation: kunit: Clean up some string casts in examples
  ...
2021-07-02 12:58:26 -07:00
David Gow
d9d6b8225e kunit: Move default config from arch/um -> tools/testing/kunit
The default .kunitconfig file is currently kept in
arch/um/configs/kunit_defconfig, but -- with the impending QEMU patch
-- will no-longer be exclusively used for UML-based kernels.

Move it alongside the other KUnit configs in
tools/testing/kunit/configs, and give it a name which matches the
existing all_tests.config and broken_on_uml.config files.

Also update the Getting Started documentation to point to the new file.

Signed-off-by: David Gow <davidgow@google.com>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2021-06-23 17:49:17 -06:00
David Gow
99a8e899fa Documentation: kunit: Clean up some string casts in examples
As the type checking is no longer excessively strict, get rid of the
unsightly (char*) casts -- and comment discussing them -- from the KUnit
usage page.

Signed-off-by: David Gow <davidgow@google.com>
Reviewed-by: Daniel Latypov <dlatypov@google.com>
Acked-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2021-06-23 16:36:29 -06:00
Mauro Carvalho Chehab
2ec73ece4e docs: dev-tools: kunit: don't use a table for docs name
We'll be replacing :doc:`foo` references to
Documentation/foo.rst. Yet, here it happens inside a table.
Doing a search-and-replace would break it.

Yet, as there's no good reason to use a table there,
let's just convert it into a list.

Reviewed-by: David Gow <davidgow@google.com>
Acked-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2021-06-23 14:19:19 -06:00
Mauro Carvalho Chehab
654a5bd0ea docs: dev-tools: kunit: avoid using ReST :doc:foo markup
The :doc:`foo` tag is auto-generated via automarkup.py.
So, use the filename at the sources, instead of :doc:`foo`.

Reviewed-by: David Gow <davidgow@google.com>
Acked-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/6fde409079959a95b62b9b2692503608d7ff0dbd.1623824363.git.mchehab+huawei@kernel.org
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2021-06-17 13:24:37 -06:00
Mauro Carvalho Chehab
9129faf904 docs: dev-tools: kunit: don't use a table for docs name
We'll be replacing :doc:`foo` references to
Documentation/foo.rst. Yet, here it happens inside a table.
Doing a search-and-replace would break it.

Yet, as there's no good reason to use a table there,
let's just convert it into a list.

Reviewed-by: David Gow <davidgow@google.com>
Acked-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/07d3a8ccafbb6345d6e78fb090290859e84361a1.1623824363.git.mchehab+huawei@kernel.org
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2021-06-17 13:24:36 -06:00
Brendan Higgins
12ca7a893d Documentation: kunit: document support for QEMU in kunit_tool
Document QEMU support, what it does, and how to use it in kunit_tool.

Signed-off-by: Brendan Higgins <brendanhiggins@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2021-06-11 16:05:12 -06:00
David Gow
d7eab3df8f Documentation: kunit: Update kunit_tool page
The kunit_tool documentation page was pretty minimal, and a bit
outdated. Update it and flesh it out a bit.

In particular,
- Mention that .kunitconfig is now in the build directory
- Describe the use of --kunitconfig to specify a different config
  framgent
- Mention the split functionality (i.e., commands other than 'run')
- Describe --raw_output and kunit.py parse
- Mention the globbing support
- Provide a quick overview of other options, including --build_dir and
  --alltests

Note that this does overlap a little with the new running_tips page. I
don't think it's a problem having both: this page is supposed to be a
bit more of a reference, rather than a list of useful tips, so the fact
that they both describe the same features isn't a problem.

Signed-off-by: David Gow <davidgow@google.com>
Reviewed-by: Daniel Latypov <dlatypov@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2021-05-21 13:25:56 -06:00
Daniel Latypov
11dbc62a73 Documentation: kunit: add tips for running KUnit
This is long overdue.

There are several things that aren't nailed down (in-tree
.kunitconfig's), or partially broken (GCOV on UML), but having them
documented, warts and all, is better than having nothing.

This covers a bunch of the more recent features
* kunit_filter_glob
* kunit.py run --kunitconfig
* slightly more detail on building tests as modules
* CONFIG_KUNIT_DEBUGFS

By my count, the only headline features now not mentioned are the KASAN
integration and KernelCI json output support (kunit.py run --json).

And then it also discusses how to get code coverage reports under UML
and non-UML since this is a question people have repeatedly asked.

Non-UML coverage collection is no different from normal, but we should
probably explicitly call this out.

As for UML, I was able to get it working again with two small hacks.*
E.g. with CONFIG_KUNIT=y && CONFIG_KUNIT_ALL_TESTS=y
  Overall coverage rate:
    lines......: 15.1% (18294 of 120776 lines)
    functions..: 16.8% (1860 of 11050 functions)

Note: this doesn't document --alltests since this is not stable yet.
Hopefully being run more frequently as part of KernelCI will help...

*Using gcc/gcov-6 and not using uml_abort() in os_dump_core().
I've documented these hacks in "Notes" but left TODOs for
brendanhiggins@google.com who tracked down the runtime issue in GCC.
To be clear: these are not issues specific to KUnit, but rather to UML.

Signed-off-by: Daniel Latypov <dlatypov@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2021-05-21 13:25:45 -06:00
Daniel Latypov
de2fcb3e62 Documentation: kunit: add tips for using current->kunit_test
As of commit 359a376081 ("kunit: support failure from dynamic analysis
tools"), we can use current->kunit_test to find the current kunit test.

Mention this in tips.rst and give an example of how this can be used in
conjunction with `test->priv` to pass around state and specifically
implement something like mocking.
There's a lot more we could go into on that topic, but given that
example is already longer than every other "tip" on this page, we just
point to the API docs and leave filling in the blanks as an exercise to
the reader.

Also give an example of kunit_fail_current_test().

Signed-off-by: Daniel Latypov <dlatypov@google.com>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2021-04-07 16:40:37 -06:00
Daniel Latypov
7c2b108cbe Documentation: kunit: add tips.rst for small examples
./usage.rst contains fairly long examples and explanations of things
like how to fake a class and how to use parameterized tests (and how you
could do table-driven tests yourself).

It's not exactly necessary information, so we add a new page with more
digestible tips like "use kunit_kzalloc() instead of kzalloc() so you
don't have to worry about calling kfree() yourself" and the like.

Change start.rst to point users to this new page first and let them know
that usage.rst is more of optional further reading.

Signed-off-by: Daniel Latypov <dlatypov@google.com>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2021-02-08 15:41:11 -07:00
Daniel Latypov
c9ef2d3e3f KUnit: Docs: make start.rst example Kconfig follow style.rst
The primary change is that we want to encourage people to respect
KUNIT_ALL_TESTS to make it easy to run all the relevant tests for a
given config.

Signed-off-by: Daniel Latypov <dlatypov@google.com>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2021-02-08 15:40:12 -07:00
Daniel Latypov
8db50be262 Documentation: kunit: include example of a parameterized test
Commit fadb08e7c7 ("kunit: Support for Parameterized Testing")
introduced support but lacks documentation for how to use it.

This patch builds on commit 1f0e943df6 ("Documentation: kunit: provide
guidance for testing many inputs") to show a minimal example of the new
feature.

Signed-off-by: Daniel Latypov <dlatypov@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Tested-by: Brendan Higgins <brendanhiggins@google.com>
Acked-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2021-01-15 17:47:39 -07:00
Daniel Latypov
1f0e943df6 Documentation: kunit: provide guidance for testing many inputs
usage.rst goes into a detailed section about faking out classes, but
currently lacks wording about how one might idiomatically test a range
of inputs.

Add a new chapter for "Common Patterns" and group "Isolating behvaior"
and this new section under there.

Give an example of how one might test a hash function via macros/helper
funcs and a table-driven test and very briefly discuss pros and cons.

Also highlight the KUNIT_EXPECT_*_MSG() variants (that aren't mentioned
elsewhere [1]) which are particularly useful in these situations.

It is also criminally underused at the moment, only appearing in 2
tests (both written by people involved in KUnit).

[1] not even on
https://www.kernel.org/doc/html/latest/dev-tools/kunit/api/test.html

Signed-off-by: Daniel Latypov <dlatypov@google.com>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2020-12-01 15:30:06 -07:00
Randy Dunlap
873ddeb881 KUnit: Docs: usage: wording fixes
Fix minor grammar and punctutation glitches.
Hyphenate "architecture-specific" instances.

Signed-off-by: Randy Dunlap <rdunlap@infradead.org>
Cc: David Gow <davidgow@google.com>
Cc: linux-kselftest@vger.kernel.org
Cc: kunit-dev@googlegroups.com
Cc: Shuah Khan <shuah@kernel.org>
Cc: Shuah Khan <skhan@linuxfoundation.org>
Cc: Brendan Higgins <brendanhiggins@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2020-11-10 13:40:30 -07:00
Randy Dunlap
1f4dde5712 KUnit: Docs: style: fix some Kconfig example issues
Fix the Kconfig example to be closer to Kconfig coding style.

Also add punctuation and a trailing slash ('/') to a sub-directory
name -- this is how the text mostly appears in other Kconfig files.

Signed-off-by: Randy Dunlap <rdunlap@infradead.org>
Cc: David Gow <davidgow@google.com>
Cc: linux-kselftest@vger.kernel.org
Cc: kunit-dev@googlegroups.com
Cc: Shuah Khan <shuah@kernel.org>
Cc: Shuah Khan <skhan@linuxfoundation.org>
Cc: Brendan Higgins <brendanhiggins@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2020-11-10 13:39:26 -07:00
Randy Dunlap
f7766424cf KUnit: Docs: fix a wording typo
Fix a wording typo (keyboard glitch).

Signed-off-by: Randy Dunlap <rdunlap@infradead.org>
Cc: David Gow <davidgow@google.com>
Cc: linux-kselftest@vger.kernel.org
Cc: kunit-dev@googlegroups.com
Cc: Shuah Khan <shuah@kernel.org>
Cc: Shuah Khan <skhan@linuxfoundation.org>
Cc: Brendan Higgins <brendanhiggins@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2020-11-10 13:36:36 -07:00
SeongJae Park
f0b62039bf Documentation: kunit: Update Kconfig parts for KUNIT's module support
If 'CONFIG_KUNIT=m', letting kunit tests that do not support loadable
module build depends on 'KUNIT' instead of 'KUNIT=y' result in compile
errors.  This commit updates the document for this.

Fixes: 9fe124bf1b ("kunit: allow kunit to be loaded as a module")
Signed-off-by: SeongJae Park <sjpark@amazon.de>
Reviewed-by: David Gow <davidgow@google.com>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2020-10-26 13:24:46 -06:00
Brendan Higgins
a82763e63e Documentation: kunit: add a brief blurb about kunit_test_suite
Add a brief blurb saying how and when the kunit_test_suite() macro
works to the usage documentation.

Signed-off-by: Brendan Higgins <brendanhiggins@google.com>
Reviewed-by: Stephen Boyd <sboyd@kernel.org>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2020-10-09 14:37:55 -06:00
David Gow
2a41fc52c2 Documentation: kunit: Add naming guidelines
As discussed in [1], KUnit tests have hitherto not had a particularly
consistent naming scheme. This adds documentation outlining how tests
and test suites should be named, including how those names should be
used in Kconfig entries and filenames.

[1]:
https://lore.kernel.org/linux-kselftest/202006141005.BA19A9D3@keescook/t/#u

Signed-off-by: David Gow <davidgow@google.com>
Reviewed-by: Kees Cook <keescook@chromium.org>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Reviewed-by: Marco Elver <elver@google.com>
Reviewed-by: Tim Bird <tim.bird@sony.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2020-10-09 14:33:17 -06:00
Linus Torvalds
53e5504bdb linux-kselftest-kunit-5.9-rc1
This Kunit update for Linux 5.9-rc1 consists of:
 
 - Adds a generic kunit_resource API extending it to support
   resources that are passed in to kunit in addition kunit
   allocated resources. In addition, KUnit resources are now
   refcounted to avoid passed in resources being released while
   in use by kunit.
 
 - Add support for named resources.
 
 - Important bug fixes from Brendan Higgins and Will Chen
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCgAdFiEEPZKym/RZuOCGeA/kCwJExA0NQxwFAl8pyI4ACgkQCwJExA0N
 QxwQmQ//YXvS1IMt4yLQ7iTrhSK2CavzPi/zX+1bgXj7RfEPrdolF12tcHf7b/Ef
 UwhG/PTEE6VkBWFuprleGk0fvVW9h13K+2RkN6wkx8rmTMunMhqmMIRj4neGQWQH
 LV9bYcc5HBH36jbSnJYMFDJdsATq3mmPMeknzkJf/qZgWxczBLo3pUHPNRuhZWq7
 MaRUutpo1f3oSroxhcY8gouRjjPq+yUCip9EeQfdZc/LV9tqjwhjfo+zRKAY3jGf
 0KL00wv4NMTNBSgLkxXFrTcyP28Y84MZB+wE3ikyE5a5wzzvMXuEtZhJk9a9q2ZJ
 tZI+ICCjnTa+E3pN93F2w7cwyrQysxNH1tpK0qv1hGXLRuxOGCJ993AeUtIWjnyV
 CYh/OGK9o92pRuTmoJnMNk9em1WF45YTtvhzpadPiBGb84nbxYSAlrDmm/4xC89p
 7PPpkkVxFReMiu1fIyXOV++y33YFMDtoWe/Qn3pLBRpDDs/XWdK0eT4dVWx1/ttl
 BdtbAxXafApVI5lGk1qPoGC1MO9XvaAidQP6PULD/rhw2Ww6rtlvwGVPffx/omkn
 zgzPTxH1rBehzKPcEaorQqqR6yGXxb4v+gJi1Tg8qRE51Lw619dfyZlJO7axp2h6
 7RO3BOVrIkwLP9XT2NPlL6t+RqbQXezGoZjigLmBRlCSvs1lbLs=
 =kIIk
 -----END PGP SIGNATURE-----

Merge tag 'linux-kselftest-kunit-5.9-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest

Pull kunit updates from Shuah Khan:

 - Add a generic kunit_resource API extending it to support resources
   that are passed in to kunit in addition kunit allocated resources. In
   addition, KUnit resources are now refcounted to avoid passed in
   resources being released while in use by kunit.

 - Add support for named resources.

 - Important bug fixes from Brendan Higgins and Will Chen

* tag 'linux-kselftest-kunit-5.9-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest:
  kunit: tool: fix improper treatment of file location
  kunit: tool: fix broken default args in unit tests
  kunit: capture stderr on all make subprocess calls
  Documentation: kunit: Remove references to --defconfig
  kunit: add support for named resources
  kunit: generalize kunit_resource API beyond allocated resources
2020-08-05 10:07:39 -07:00
David Gow
39f65da092 Documentation: kunit: Remove references to --defconfig
The --defconfig option in kunit_tool was removed in [1], but the getting
started and kunit_tool documentation still encouraged its use. Update
those documents to reflect that it's no-longer required, and is the
default behaviour if no .kunitconfig is found.

Also update a couple of places where .kunitconfig is still referred to
as kunitconfig (this was changed in [2]).

[1]:
https://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest.git/commit/?h=kunit-fixes&id=9bdf64b35117cc10813d24e1842cd8ee40ecbf19
[2]:
https://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest.git/commit/?h=kunit-fixes&id=14ee5cfd4512ee3d1e0047d8751450dcc6544070

Signed-off-by: David Gow <davidgow@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2020-07-17 14:08:43 -06:00
David Gow
c63d2dd7e1 Documentation: kunit: Add some troubleshooting tips to the FAQ
Add an FAQ entry to the KUnit documentation with some tips for
troubleshooting KUnit and kunit_tool.

These suggestions largely came from an email thread:
https://lore.kernel.org/linux-kselftest/41db8bbd-3ba0-8bde-7352-083bf4b947ff@intel.com/T/#m23213d4e156db6d59b0b460a9014950f5ff6eb03

Signed-off-by: David Gow <davidgow@google.com>
Reviewed-by: Alan Maguire <alan.maguire@oracle.com>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2020-06-26 14:29:55 -06:00
Lothar Rubusch
c4714b0045 Documentation: test.h - fix warnings
Fix warnings at 'make htmldocs', and formatting issues in the resulting
documentation.

- test.h: Fix annotation in kernel-doc parameter description.

- Documentation/*.rst: Fixing formatting issues, and a duplicate label
  issue due to usage of sphinx.ext.autosectionlabel and identical labels
  within one document (sphinx warning)

Signed-off-by: Lothar Rubusch <l.rubusch@gmail.com>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2020-05-22 14:33:59 -06:00
Alan Maguire
32526908fe kunit: update documentation to describe debugfs representation
Documentation should describe debugfs layout and semantics.

Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Reviewed-by: Frank Rowand <frank.rowand@sony.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2020-03-26 14:09:16 -06:00
David Gow
0d5792c9bc Documentation: kunit: Make the KUnit documentation less UML-specific
Remove some of the outmoded "Why KUnit" rationale, and move some
UML-specific information to the kunit_tool page. Also update the Getting
Started guide to mention running tests without the kunit_tool wrapper.

Signed-off-by: David Gow <davidgow@google.com>
Reviewed-by: Frank Rowand <frank.rowand@sony.com>
Acked-by: Shuah Khan <skhan@linuxfoundation.org>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2020-03-25 16:39:13 -06:00
Brendan Higgins
e20d8e81a0 Documentation: kunit: fixed sphinx error in code block
Fix a missing newline in a code block that was causing a warning:

Documentation/dev-tools/kunit/usage.rst:553: WARNING: Error in "code-block" directive:
maximum 1 argument(s) allowed, 3 supplied.

.. code-block:: bash
        modprobe example-test

Signed-off-by: Brendan Higgins <brendanhiggins@google.com>
Reviewed-by: Alan Maguire <alan.maguire@oracle.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2020-02-10 17:16:42 -07:00
Alan Maguire
6ae2bfd3df kunit: update documentation to describe module-based build
Documentation should describe how to build kunit and tests as
modules.

Co-developed-by: Knut Omang <knut.omang@oracle.com>
Signed-off-by: Knut Omang <knut.omang@oracle.com>
Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
Reviewed-by: Stephen Boyd <swboyd@chromium.org>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2020-01-09 16:44:10 -07:00
SeongJae Park
14ee5cfd45 kunit: Rename 'kunitconfig' to '.kunitconfig'
This commit renames 'kunitconfig' to '.kunitconfig' so that it can be
automatically ignored by git and do not disturb people who want to type
'kernel/' by pressing only the 'k' and then 'tab' key.

Signed-off-by: SeongJae Park <sjpark@amazon.de>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Tested-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2019-12-23 10:52:41 -07:00
SeongJae Park
1c68cbadf0 docs/kunit/start: Use in-tree 'kunit_defconfig'
The kunit doc suggests users to get the default `kunitconfig` from an
external git tree.  However, the file is already located under the
`arch/um/configs/` of the kernel tree.  Because the local file is easier
to access and maintain, this commit updates the doc to use it.

Signed-off-by: SeongJae Park <sjpark@amazon.de>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Tested-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2019-12-23 10:52:40 -07:00
Brendan Higgins
99e51aa8f7 Documentation: kunit: add documentation for kunit_tool
Add documentation for the Python script used to build, run, and collect
results from the kernel known as kunit_tool. kunit_tool
(tools/testing/kunit/kunit.py) was already added in previous commits.

Signed-off-by: Brendan Higgins <brendanhiggins@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Cc: Randy Dunlap <rdunlap@infradead.org>
Acked-by: Randy Dunlap <rdunlap@infradead.org>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2019-12-09 11:22:01 -07:00
Brendan Higgins
e7d7ad0e41 Documentation: kunit: fix typos and gramatical errors
Fix typos and gramatical errors in the Getting Started and Usage guide
for KUnit.

Reported-by: Randy Dunlap <rdunlap@infradead.org>
Link: https://patchwork.kernel.org/patch/11156481/
Reported-by: Rinat Ibragimov <ibragimovrinat@mail.ru>
Link: https://github.com/google/kunit-docs/issues/1
Signed-off-by: Brendan Higgins <brendanhiggins@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Acked-by: Randy Dunlap <rdunlap@infradead.org>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2019-12-09 11:21:22 -07:00
SeongJae Park
d460623c5f Documentation: kunit: Fix verification command
kunit wrapper script ('kunit.py') receives a sub-command (only 'run' for
now) as its argument.  If no sub-command is given, it prints help
message and just quit.  However, an example command in the kunit
documentation for a verification of kunit is missing the sub-command.
This commit fixes the example.

Signed-off-by: SeongJae Park <sj38.park@gmail.com>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2019-10-07 17:02:13 -06:00
Brendan Higgins
c23a283b53 Documentation: kunit: add documentation for KUnit
Add documentation for KUnit, the Linux kernel unit testing framework.
- Add intro and usage guide for KUnit
- Add API reference

Signed-off-by: Felix Guo <felixguoxiuping@gmail.com>
Signed-off-by: Brendan Higgins <brendanhiggins@google.com>
Cc: Jonathan Corbet <corbet@lwn.net>
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-by: Logan Gunthorpe <logang@deltatee.com>
Reviewed-by: Stephen Boyd <sboyd@kernel.org>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2019-09-30 17:35:01 -06:00