19 Oct, 2020
1 commit
-
…/kernel/git/shuah/linux-kselftest
Pull more Kunit updates from Shuah Khan:
- add Kunit to kernel_init() and remove KUnit from init calls entirely.
This addresses the concern that Kunit would not work correctly during
late init phase.- add a linker section where KUnit can put references to its test
suites.This is the first step in transitioning to dispatching all KUnit
tests from a centralized executor rather than having each as its own
separate late_initcall.- add a centralized executor to dispatch tests rather than relying on
late_initcall to schedule each test suite separately. Centralized
execution is for built-in tests only; modules will execute tests when
loaded.- convert bitfield test to use KUnit framework
- Documentation updates for naming guidelines and how
kunit_test_suite() works.- add test plan to KUnit TAP format
* tag 'linux-kselftest-kunit-5.10-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest:
lib: kunit: Fix compilation test when using TEST_BIT_FIELD_COMPILE
lib: kunit: add bitfield test conversion to KUnit
Documentation: kunit: add a brief blurb about kunit_test_suite
kunit: test: add test plan to KUnit TAP format
init: main: add KUnit to kernel init
kunit: test: create a single centralized executor for all tests
vmlinux.lds.h: add linker section for KUnit test suites
Documentation: kunit: Add naming guidelines
14 Oct, 2020
1 commit
-
Integrate KASAN into KUnit testing framework.
- Fail tests when KASAN reports an error that is not expected
- Use KUNIT_EXPECT_KASAN_FAIL to expect a KASAN error in KASAN
tests
- Expected KASAN reports pass tests and are still printed when run
without kunit_tool (kunit_tool still bypasses the report due to the
test passing)
- KUnit struct in current task used to keep track of the current
test from KASAN codeMake use of "[PATCH v3 kunit-next 1/2] kunit: generalize kunit_resource
API beyond allocated resources" and "[PATCH v3 kunit-next 2/2] kunit: add
support for named resources" from Alan Maguire [1]- A named resource is added to a test when a KASAN report is
expected
- This resource contains a struct for kasan_data containing
booleans representing if a KASAN report is expected and if a
KASAN report is found[1] (https://lore.kernel.org/linux-kselftest/1583251361-12748-1-git-send-email-alan.maguire@oracle.com/T/#t)
Signed-off-by: Patricia Alfonso
Signed-off-by: David Gow
Signed-off-by: Andrew Morton
Tested-by: Andrey Konovalov
Reviewed-by: Andrey Konovalov
Reviewed-by: Dmitry Vyukov
Acked-by: Brendan Higgins
Cc: Andrey Ryabinin
Cc: Ingo Molnar
Cc: Juri Lelli
Cc: Peter Zijlstra
Cc: Shuah Khan
Cc: Vincent Guittot
Link: https://lkml.kernel.org/r/20200915035828.570483-3-davidgow@google.com
Link: https://lkml.kernel.org/r/20200910070331.3358048-3-davidgow@google.com
Signed-off-by: Linus Torvalds
10 Oct, 2020
3 commits
-
TAP 14 allows an optional test plan to be emitted before the start of
the start of testing[1]; this is valuable because it makes it possible
for a test harness to detect whether the number of tests run matches the
number of tests expected to be run, ensuring that no tests silently
failed.Link[1]: https://github.com/isaacs/testanything.github.io/blob/tap14/tap-version-14-specification.md#the-plan
Signed-off-by: Brendan Higgins
Reviewed-by: Stephen Boyd
Signed-off-by: Shuah Khan -
Although we have not seen any actual examples where KUnit doesn't work
because it runs in the late init phase of the kernel, it has been a
concern for some time that this could potentially be an issue in the
future. So, remove KUnit from init calls entirely, instead call directly
from kernel_init() so that KUnit runs after late init.Co-developed-by: Alan Maguire
Signed-off-by: Alan Maguire
Signed-off-by: Brendan Higgins
Reviewed-by: Stephen Boyd
Reviewed-by: Kees Cook
Reviewed-by: Luis Chamberlain
Signed-off-by: Shuah Khan -
Add a centralized executor to dispatch tests rather than relying on
late_initcall to schedule each test suite separately. Centralized
execution is for built-in tests only; modules will execute tests when
loaded.Signed-off-by: Alan Maguire
Co-developed-by: Iurii Zaikin
Signed-off-by: Iurii Zaikin
Co-developed-by: Brendan Higgins
Signed-off-by: Brendan Higgins
Reviewed-by: Stephen Boyd
Reviewed-by: Kees Cook
Signed-off-by: Shuah Khan
27 Jun, 2020
2 commits
-
The kunit resources API allows for custom initialization and
cleanup code (init/fini); here a new resource add function sets
the "struct kunit_resource" "name" field, and calls the standard
add function. Having a simple way to name resources is
useful in cases such as multithreaded tests where a set of
resources are shared among threads; a pointer to the
"struct kunit *" test state then is all that is needed to
retrieve and use named resources. Support is provided to add,
find and destroy named resources; the latter two are simply
wrappers that use a "match-by-name" callback.If an attempt to add a resource with a name that already exists
is made kunit_add_named_resource() will return -EEXIST.Signed-off-by: Alan Maguire
Reviewed-by: Brendan Higgins
Signed-off-by: Shuah Khan -
In its original form, the kunit resources API - consisting the
struct kunit_resource and associated functions - was focused on
adding allocated resources during test operation that would be
automatically cleaned up on test completion.The recent RFC patch proposing converting KASAN tests to KUnit [1]
showed another potential model - where outside of test context,
but with a pointer to the test state, we wish to access/update
test-related data, but expressly want to avoid allocations.It turns out we can generalize the kunit_resource to support
static resources where the struct kunit_resource * is passed
in and initialized for us. As part of this work, we also
change the "allocation" field to the more general "data" name,
as instead of associating an allocation, we can associate a
pointer to static data. Static data is distinguished by a NULL
free functions. A test is added to cover using kunit_add_resource()
with a static resource and data.Finally we also make use of the kernel's krefcount interfaces
to manage reference counting of KUnit resources. The motivation
for this is simple; if we have kernel threads accessing and
using resources (say via kunit_find_resource()) we need to
ensure we do not remove said resources (or indeed free them
if they were dynamically allocated) until the reference count
reaches zero. A new function - kunit_put_resource() - is
added to handle this, and it should be called after a
thread using kunit_find_resource() is finished with the
retrieved resource.We ensure that the functions needed to look up, use and
drop reference count are "static inline"-defined so that
they can be used by builtin code as well as modules in
the case that KUnit is built as a module.A cosmetic change here also; I've tried moving to
kunit_[action]_resource() as the format of function names
for consistency and readability.[1] https://lkml.org/lkml/2020/2/26/1286
Signed-off-by: Alan Maguire
Reviewed-by: Brendan Higgins
Signed-off-by: Shuah Khan
02 Jun, 2020
2 commits
-
This makes it easier to enable all KUnit fragments.
Adding 'if !KUNIT_ALL_TESTS' so individual tests can not be turned off.
Therefore if KUNIT_ALL_TESTS is enabled that will hide the prompt in
menuconfig.Reviewed-by: David Gow
Signed-off-by: Anders Roxell
Signed-off-by: Shuah Khan -
Make it easier to enable all KUnit fragments. This is useful for kernel
devs or testers, so its easy to get all KUnit tests enabled and if new
gets added they will be enabled as well. Fragments that has to be
builtin will be missed if CONFIG_KUNIT_ALL_TESTS is set as a module.Signed-off-by: Anders Roxell
Reviewed-by: Brendan Higgins
Signed-off-by: Shuah Khan
24 Apr, 2020
1 commit
-
Add missing newline, as otherwise flushing of the final summary message
to the console log can be delayed.Fixes: e2219db280e3 ("kunit: add debugfs /sys/kernel/debug/kunit//results display")
Signed-off-by: Marco Elver
Tested-by: David Gow
Reviewed-by: Alan Maguire
Acked-by: Brendan Higgins
Signed-off-by: Shuah Khan
27 Mar, 2020
3 commits
-
Introduce KUNIT_SUBTEST_INDENT macro which corresponds to 4-space
indentation and KUNIT_SUBSUBTEST_INDENT macro which corresponds to
8-space indentation in line with TAP spec (e.g. see "Subtests"
section of https://node-tap.org/tap-protocol/).Use these macros in place of one or two tabs in strings to clarify
why we are indenting.Suggested-by: Frank Rowand
Signed-off-by: Alan Maguire
Reviewed-by: Frank Rowand
Signed-off-by: Shuah Khan -
the logging test ensures multiple strings logged appear in the
log string associated with the test when CONFIG_KUNIT_DEBUGFS is
enabled.Signed-off-by: Alan Maguire
Reviewed-by: Brendan Higgins
Reviewed-by: Frank Rowand
Signed-off-by: Shuah Khan -
add debugfs support for displaying kunit test suite results; this is
especially useful for module-loaded tests to allow disentangling of
test result display from other dmesg events. debugfs support is
provided if CONFIG_KUNIT_DEBUGFS=y.As well as printk()ing messages, we append them to a per-test log.
Signed-off-by: Alan Maguire
Reviewed-by: Brendan Higgins
Reviewed-by: Frank Rowand
Signed-off-by: Shuah Khan
26 Mar, 2020
1 commit
-
KUnit assertions and expectations will print the values being tested. If
these are pointers (e.g., KUNIT_EXPECT_PTR_EQ(test, a, b)), these
pointers are currently printed with the %pK format specifier, which -- to
prevent information leaks which may compromise, e.g., ASLR -- are often
either hashed or replaced with ____ptrval____ or similar, making debugging
tests difficult.By replacing %pK with %px as Documentation/core-api/printk-formats.rst
suggests, we disable this security feature for KUnit assertions and
expectations, allowing the actual pointer values to be printed. Given
that KUnit is not intended for use in production kernels, and the
pointers are only printed on failing tests, this seems like a worthwhile
tradeoff.Signed-off-by: David Gow
Reviewed-by: Brendan Higgins
Signed-off-by: Shuah Khan
10 Jan, 2020
5 commits
-
Making kunit itself buildable as a module allows for "always-on"
kunit configuration; specifying CONFIG_KUNIT=m means the module
is built but only used when loaded. Kunit test modules will load
kunit.ko as an implicit dependency, so simply running
"modprobe my-kunit-tests" will load the tests along with the kunit
module and run them.Co-developed-by: Knut Omang
Signed-off-by: Knut Omang
Signed-off-by: Alan Maguire
Reviewed-by: Brendan Higgins
Signed-off-by: Shuah Khan -
In discussion of how to handle timeouts, it was noted that if
sysctl_hung_task_timeout_seconds is exceeded for a kunit test,
the test task will be killed and an oops generated. This should
suffice as a means of debugging such timeout issues for now.Hence remove use of sysctl_hung_task_timeout_secs, which has the
added benefit of avoiding the need to export that symbol from
the core kernel.Co-developed-by: Knut Omang
Signed-off-by: Knut Omang
Signed-off-by: Alan Maguire
Reviewed-by: Stephen Boyd
Acked-by: Brendan Higgins
Signed-off-by: Shuah Khan -
As tests are added to kunit, it will become less feasible to execute
all built tests together. By supporting modular tests we provide
a simple way to do selective execution on a running system; specifyingCONFIG_KUNIT=y
CONFIG_KUNIT_EXAMPLE_TEST=m...means we can simply "insmod example-test.ko" to run the tests.
To achieve this we need to do the following:
o export the required symbols in kunit
o string-stream tests utilize non-exported symbols so for now we skip
building them when CONFIG_KUNIT_TEST=m.
o drivers/base/power/qos-test.c contains a few unexported interface
references, namely freq_qos_read_value() and freq_constraints_init().
Both of these could be potentially defined as static inline functions
in include/linux/pm_qos.h, but for now we simply avoid supporting
module build for that test suite.
o support a new way of declaring test suites. Because a module cannot
do multiple late_initcall()s, we provide a kunit_test_suites() macro
to declare multiple suites within the same module at once.
o some test module names would have been too general ("test-test"
and "example-test" for kunit tests, "inode-test" for ext4 tests);
rename these as appropriate ("kunit-test", "kunit-example-test"
and "ext4-inode-test" respectively).Also define kunit_test_suite() via kunit_test_suites()
as callers in other trees may need the old definition.Co-developed-by: Knut Omang
Signed-off-by: Knut Omang
Signed-off-by: Alan Maguire
Reviewed-by: Brendan Higgins
Acked-by: Theodore Ts'o # for ext4 bits
Acked-by: David Gow # For list-test
Reported-by: kbuild test robot
Signed-off-by: Shuah Khan -
Define function as static inline in try-catch-impl.h to allow it to
be used in kunit itself and tests. Also remove unused
kunit_generic_try_catchCo-developed-by: Knut Omang
Signed-off-by: Knut Omang
Signed-off-by: Alan Maguire
Reviewed-by: Brendan Higgins
Tested-by: Brendan Higgins
Signed-off-by: Shuah Khan -
string-stream interfaces are not intended for external use;
move them from include/kunit to lib/kunit accordingly.Co-developed-by: Knut Omang
Signed-off-by: Knut Omang
Signed-off-by: Alan Maguire
Reviewed-by: Brendan Higgins
Tested-by: Brendan Higgins
Signed-off-by: Shuah Khan
01 Oct, 2019
11 commits
-
Previously KUnit assumed that printk would always be present, which is
not a valid assumption to make. Fix that by removing call to
vprintk_emit, and calling printk directly.This fixes a build error[1] reported by Randy.
For context this change comes after much discussion. My first stab[2] at
this was just to make the KUnit logging code compile out; however, it
was agreed that if we were going to use vprintk_emit, then vprintk_emit
should provide a no-op stub, which lead to my second attempt[3]. In
response to me trying to stub out vprintk_emit, Sergey Senozhatsky
suggested a way for me to remove our usage of vprintk_emit, which led to
my third attempt at solving this[4].In my third version of this patch[4], I completely removed vprintk_emit,
as suggested by Sergey; however, there was a bit of debate over whether
Sergey's solution was the best. The debate arose due to Sergey's version
resulting in a checkpatch warning, which resulted in a debate over
correct printk usage. Joe Perches offered an alternative fix which was
somewhat less far reaching than what Sergey had suggested and
importantly relied on continuing to use %pV. Much of the debated
centered around whether %pV should be widely used, and whether Sergey's
version would result in object size bloat. Ultimately, we decided to go
with Sergey's version.Reported-by: Randy Dunlap
Link[1]: https://lore.kernel.org/linux-kselftest/c7229254-0d90-d90e-f3df-5b6d6fc0b51f@infradead.org/
Link[2]: https://lore.kernel.org/linux-kselftest/20190827174932.44177-1-brendanhiggins@google.com/
Link[3]: https://lore.kernel.org/linux-kselftest/20190827234835.234473-1-brendanhiggins@google.com/
Link[4]: https://lore.kernel.org/linux-kselftest/20190828093143.163302-1-brendanhiggins@google.com/
Cc: Stephen Rothwell
Cc: Sergey Senozhatsky
Cc: Joe Perches
Cc: Tim.Bird@sony.com
Signed-off-by: Brendan Higgins
Acked-by: Randy Dunlap # build-tested
Reviewed-by: Petr Mladek
Signed-off-by: Shuah Khan -
Add unit tests for KUnit managed resources. KUnit managed resources
(struct kunit_resource) are resources that are automatically cleaned up
at the end of a KUnit test, similar to the concept of devm_* managed
resources.Signed-off-by: Avinash Kondareddy
Signed-off-by: Brendan Higgins
Reviewed-by: Greg Kroah-Hartman
Reviewed-by: Logan Gunthorpe
Reviewed-by: Stephen Boyd
Signed-off-by: Shuah Khan -
Add support for assertions which are like expectations except the test
terminates if the assertion is not satisfied.The idea with assertions is that you use them to state all the
preconditions for your test. Logically speaking, these are the premises
of the test case, so if a premise isn't true, there is no point in
continuing the test case because there are no conclusions that can be
drawn without the premises. Whereas, the expectation is the thing you
are trying to prove. It is not used universally in x-unit style test
frameworks, but I really like it as a convention. You could still
express the idea of a premise using the above idiom, but I think
KUNIT_ASSERT_* states the intended idea perfectly.Signed-off-by: Brendan Higgins
Reviewed-by: Greg Kroah-Hartman
Reviewed-by: Logan Gunthorpe
Reviewed-by: Stephen Boyd
Signed-off-by: Shuah Khan -
Add KUnit tests for the KUnit test abort mechanism (see preceding
commit). Add tests both for general try catch mechanism as well as
non-architecture specific mechanism.Signed-off-by: Brendan Higgins
Reviewed-by: Greg Kroah-Hartman
Reviewed-by: Logan Gunthorpe
Reviewed-by: Stephen Boyd
Signed-off-by: Shuah Khan -
Add support for aborting/bailing out of test cases, which is needed for
implementing assertions.An assertion is like an expectation, but bails out of the test case
early if the assertion is not met. The idea with assertions is that you
use them to state all the preconditions for your test. Logically
speaking, these are the premises of the test case, so if a premise isn't
true, there is no point in continuing the test case because there are no
conclusions that can be drawn without the premises. Whereas, the
expectation is the thing you are trying to prove.Signed-off-by: Brendan Higgins
Reviewed-by: Greg Kroah-Hartman
Reviewed-by: Logan Gunthorpe
Reviewed-by: Stephen Boyd
Signed-off-by: Shuah Khan -
Add a test for string stream along with a simpler example.
Signed-off-by: Brendan Higgins
Reviewed-by: Greg Kroah-Hartman
Reviewed-by: Logan Gunthorpe
Reviewed-by: Stephen Boyd
Signed-off-by: Shuah Khan -
Add support for expectations, which allow properties to be specified and
then verified in tests.Signed-off-by: Brendan Higgins
Reviewed-by: Greg Kroah-Hartman
Reviewed-by: Logan Gunthorpe
Reviewed-by: Stephen Boyd
Signed-off-by: Shuah Khan -
Add `struct kunit_assert` and friends which provide a structured way to
capture data from an expectation or an assertion (introduced later in
the series) so that it may be printed out in the event of a failure.Signed-off-by: Brendan Higgins
Reviewed-by: Stephen Boyd
Signed-off-by: Shuah Khan -
A number of test features need to do pretty complicated string printing
where it may not be possible to rely on a single preallocated string
with parameters.So provide a library for constructing the string as you go similar to
C++'s std::string. string_stream is really just a string builder,
nothing more.Signed-off-by: Brendan Higgins
Reviewed-by: Greg Kroah-Hartman
Reviewed-by: Logan Gunthorpe
Reviewed-by: Stephen Boyd
Signed-off-by: Shuah Khan -
Create a common API for test managed resources like memory and test
objects. A lot of times a test will want to set up infrastructure to be
used in test cases; this could be anything from just wanting to allocate
some memory to setting up a driver stack; this defines facilities for
creating "test resources" which are managed by the test infrastructure
and are automatically cleaned up at the conclusion of the test.Signed-off-by: Brendan Higgins
Reviewed-by: Greg Kroah-Hartman
Reviewed-by: Logan Gunthorpe
Reviewed-by: Stephen Boyd
Signed-off-by: Shuah Khan -
Add core facilities for defining unit tests; this provides a common way
to define test cases, functions that execute code which is under test
and determine whether the code under test behaves as expected; this also
provides a way to group together related test cases in test suites (here
we call them test_modules).Just define test cases and how to execute them for now; setting
expectations on code will be defined later.Signed-off-by: Brendan Higgins
Reviewed-by: Greg Kroah-Hartman
Reviewed-by: Logan Gunthorpe
Reviewed-by: Luis Chamberlain
Reviewed-by: Stephen Boyd
Signed-off-by: Shuah Khan
