| 582eb3ae | 25-Jul-2023 |
Kemeng Shi <shikemeng@huaweicloud.com> |
kunit: replace KUNIT_TRIGGER_STATIC_STUB maro with KUNIT_STATIC_STUB_REDIRECT
We mix up KUNIT_TRIGGER_STATIC_STUB and KUNIT_STATIC_STUB_REDIRECT in
static_stub header. Just correct KUNIT_TRIGGER_STA
kunit: replace KUNIT_TRIGGER_STATIC_STUB maro with KUNIT_STATIC_STUB_REDIRECT
We mix up KUNIT_TRIGGER_STATIC_STUB and KUNIT_STATIC_STUB_REDIRECT in
static_stub header. Just correct KUNIT_TRIGGER_STATIC_STUB to
KUNIT_STATIC_STUB_REDIRECT which is documented.
Signed-off-by: Kemeng Shi <shikemeng@huaweicloud.com>
Reviewed-by: David Gow <davidgow@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
show more ...
|
| b67abaad | 07-Aug-2023 |
Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com> |
kunit: Allow kunit test modules to use test filtering
External tools, e.g., Intel GPU tools (IGT), support execution of
individual selftests provided by kernel modules. That could be also
applicabl
kunit: Allow kunit test modules to use test filtering
External tools, e.g., Intel GPU tools (IGT), support execution of
individual selftests provided by kernel modules. That could be also
applicable to kunit test modules if they provided test filtering. But
test filtering is now possible only when kunit code is built into the
kernel. Moreover, a filter can be specified only at boot time, then
reboot is required each time a different filter is needed.
Build the test filtering code also when kunit is configured as a module,
expose test filtering functions to other kunit source files, and use them
in kunit module notifier callback functions. Userspace can then reload
the kunit module with a value of the filter_glob parameter tuned to a
specific kunit test module every time it wants to limit the scope of tests
executed on that module load. Make the kunit.filter* parameters visible
in sysfs for user convenience.
v5: Refresh on tpp of attributes filtering fix
v4: Refresh on top of newly applied attributes patches and changes
introdced by new versions of other patches submitted in series with
this one.
v3: Fix CONFIG_GLOB, required by filtering functions, not selected when
building as a module (lkp@intel.com).
v2: Fix new name of a structure moved to kunit namespace not updated
across all uses (lkp@intel.com).
Signed-off-by: Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com>
Reviewed-by: David Gow <davidgow@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
show more ...
|
| 18258c60 | 07-Aug-2023 |
Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com> |
kunit: Make 'list' action available to kunit test modules
Results from kunit tests reported via dmesg may be interleaved with other
kernel messages. When parsing dmesg for modular kunit results in
kunit: Make 'list' action available to kunit test modules
Results from kunit tests reported via dmesg may be interleaved with other
kernel messages. When parsing dmesg for modular kunit results in real
time, external tools, e.g., Intel GPU tools (IGT), may want to insert
their own test name markers into dmesg at the start of each test, before
any kernel message related to that test appears there, so existing upper
level test result parsers have no doubt which test to blame for a specific
kernel message. Unfortunately, kunit reports names of tests only at their
completion (with the exeption of a not standarized "# Subtest: <name>"
header above a test plan of each test suite or parametrized test).
External tools could be able to insert their own "start of the test"
markers with test names included if they new those names in advance.
Test names could be learned from a list if provided by a kunit test
module.
There exists a feature of listing kunit tests without actually executing
them, but it is now limited to configurations with the kunit module built
in and covers only built-in tests, already available at boot time.
Moreover, switching from list to normal mode requires reboot. If that
feature was also available when kunit is built as a module, userspace
could load the module with action=list parameter, load some kunit test
modules they are interested in and learn about the list of tests provided
by those modules, then unload them, reload the kunit module in normal mode
and execute the tests with their lists already known.
Extend kunit module notifier initialization callback with a processing
path for only listing the tests provided by a module if the kunit action
parameter is set to "list" or "list_attr". For user convenience, make the
kunit.action parameter visible in sysfs.
v2: Don't use a different format, use kunit_exec_list_tests() (Rae),
- refresh on top of new attributes patches, handle newly introduced
kunit.action=list_attr case (Rae).
Signed-off-by: Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com>
Cc: Rae Moar <rmoar@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
show more ...
|
| c95e7c05 | 07-Aug-2023 |
Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com> |
kunit: Report the count of test suites in a module
According to KTAP specification[1], results should always start from a
header that provides a TAP protocol version, followed by a test plan with
a
kunit: Report the count of test suites in a module
According to KTAP specification[1], results should always start from a
header that provides a TAP protocol version, followed by a test plan with
a count of items to be executed. That pattern should be followed at each
nesting level. In the current implementation of the top-most, i.e., test
suite level, those rules apply only for test suites built into the kernel,
executed and reported on boot. Results submitted to dmesg from kunit test
modules loaded later are missing those top-level headers.
As a consequence, if a kunit test module provides more than one test suite
then, without the top level test plan, external tools that are parsing
dmesg for kunit test output are not able to tell how many test suites
should be expected and whether to continue parsing after complete output
from the first test suite is collected.
Submit the top-level headers also from the kunit test module notifier
initialization callback.
v3: Fix new name of a structure moved to kunit namespace not updated in
executor_test functions (lkp@intel.com).
v2: Use kunit_exec_run_tests() (Mauro, Rae), but prevent it from
emitting the headers when called on load of non-test modules.
[1] https://docs.kernel.org/dev-tools/ktap.html#
Signed-off-by: Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com>
Cc: Mauro Carvalho Chehab <mchehab@kernel.org>
Cc: Rae Moar <rmoar@google.com>
Reviewed-by: Rae Moar <rmoar@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
show more ...
|
| 529534e8 | 25-Jul-2023 |
Rae Moar <rmoar@google.com> |
kunit: Add ability to filter attributes
Add filtering of test attributes. Users can filter tests using the
module_param called "filter".
Filters are imputed in the format: <attribute_name><operatio
kunit: Add ability to filter attributes
Add filtering of test attributes. Users can filter tests using the
module_param called "filter".
Filters are imputed in the format: <attribute_name><operation><value>
Example: kunit.filter="speed>slow"
Operations include: >, <, >=, <=, !=, and =. These operations will act the
same for attributes of the same type but may not between types.
Note multiple filters can be inputted by separating them with a comma.
Example: kunit.filter="speed=slow, module!=example"
Since both suites and test cases can have attributes, there may be
conflicts. The process of filtering follows these rules:
- Filtering always operates at a per-test level.
- If a test has an attribute set, then the test's value is filtered on.
- Otherwise, the value falls back to the suite's value.
- If neither are set, the attribute has a global "default" value, which
is used.
Filtered tests will not be run or show in output. The tests can instead be
skipped using the configurable option "kunit.filter_action=skip".
Note the default settings for running tests remains unfiltered.
Finally, add "filter" methods for the speed and module attributes to parse
and compare attribute values.
Note this filtering functionality will be added to kunit.py in the next
patch.
Reviewed-by: David Gow <davidgow@google.com>
Signed-off-by: Rae Moar <rmoar@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
show more ...
|
| a00a7270 | 25-Jul-2023 |
Rae Moar <rmoar@google.com> |
kunit: Add module attribute
Add module attribute to the test attribute API. This attribute stores the
module name associated with the test using KBUILD_MODNAME.
The name of a test suite and the mod
kunit: Add module attribute
Add module attribute to the test attribute API. This attribute stores the
module name associated with the test using KBUILD_MODNAME.
The name of a test suite and the module name often do not match. A
reference to the module name associated with the suite could be extremely
helpful in running tests as modules without needing to check the codebase.
This attribute will be printed for each suite.
Reviewed-by: David Gow <davidgow@google.com>
Signed-off-by: Rae Moar <rmoar@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
show more ...
|
| 02c2d0c2 | 25-Jul-2023 |
Rae Moar <rmoar@google.com> |
kunit: Add speed attribute
Add speed attribute to the test attribute API. This attribute will allow
users to mark tests with a category of speed.
Currently the categories of speed proposed are: nor
kunit: Add speed attribute
Add speed attribute to the test attribute API. This attribute will allow
users to mark tests with a category of speed.
Currently the categories of speed proposed are: normal, slow, and very_slow
(outlined in enum kunit_speed). These are outlined in the enum kunit_speed.
The assumed default speed for tests is "normal". This indicates that the
test takes a relatively trivial amount of time (less than 1 second),
regardless of the machine it is running on. Any test slower than this could
be marked as "slow" or "very_slow".
Add the macro KUNIT_CASE_SLOW to set a test as slow, as this is likely a
common use of the attributes API.
Add an example of marking a slow test to kunit-example-test.c.
Reviewed-by: David Gow <davidgow@google.com>
Signed-off-by: Rae Moar <rmoar@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
show more ...
|
| 39e92cb1 | 25-Jul-2023 |
Rae Moar <rmoar@google.com> |
kunit: Add test attributes API structure
Add the basic structure of the test attribute API to KUnit, which can be
used to save and access test associated data.
Add attributes.c and attributes.h to
kunit: Add test attributes API structure
Add the basic structure of the test attribute API to KUnit, which can be
used to save and access test associated data.
Add attributes.c and attributes.h to hold associated structs and functions
for the API.
Create a struct that holds a variety of associated helper functions for
each test attribute. These helper functions will be used to get the
attribute value, convert the value to a string, and filter based on the
value. This struct is flexible by design to allow for attributes of
numerous types and contexts.
Add a method to print test attributes in the format of "# [<test_name if
not suite>.]<attribute_name>: <attribute_value>".
Example for a suite: "# speed: slow"
Example for a test case: "# test_case.speed: very_slow"
Use this method to report attributes in the KTAP output (KTAP spec:
https://docs.kernel.org/dev-tools/ktap.html) and _list_tests output when
kernel's new kunit.action=list_attr option is used. Note this is derivative
of the kunit.action=list option.
In test.h, add fields and associated helper functions to test cases and
suites to hold user-inputted test attributes.
Reviewed-by: David Gow <davidgow@google.com>
Signed-off-by: Rae Moar <rmoar@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
show more ...
|
| 26075518 | 31-May-2023 |
David Gow <davidgow@google.com> |
kunit: Move kunit_abort() call out of kunit_do_failed_assertion()
KUnit aborts the current thread when an assertion fails. Currently, this
is done conditionally as part of the kunit_do_failed_assert
kunit: Move kunit_abort() call out of kunit_do_failed_assertion()
KUnit aborts the current thread when an assertion fails. Currently, this
is done conditionally as part of the kunit_do_failed_assertion()
function, but this hides the kunit_abort() call from the compiler
(particularly if it's in another module). This, in turn, can lead to
both suboptimal code generation (the compiler can't know if
kunit_do_failed_assertion() will return), and to static analysis tools
like smatch giving false positives.
Moving the kunit_abort() call into the macro should give the compiler
and tools a better chance at understanding what's going on. Doing so
requires exporting kunit_abort(), though it's recommended to continue to
use assertions in lieu of aborting directly.
In addition, kunit_abort() and kunit_do_failed_assertion() are renamed
to make it clear they they're intended for internal KUnit use, to:
__kunit_do_failed_assertion() and __kunit_abort()
Suggested-by: Dan Carpenter <dan.carpenter@linaro.org>
Signed-off-by: David Gow <davidgow@google.com>
Reviewed-by: Miguel Ojeda <ojeda@kernel.org>
Reviewed-by: Daniel Latypov <dlatypov@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
show more ...
|
| c042030a | 30-May-2023 |
David Gow <davidgow@google.com> |
kunit: Fix obsolete name in documentation headers (func->action)
The kunit_add_action() and related functions named the kunit_action_t
parameter 'func' in early drafts, which was later renamed to 'a
kunit: Fix obsolete name in documentation headers (func->action)
The kunit_add_action() and related functions named the kunit_action_t
parameter 'func' in early drafts, which was later renamed to 'action'
However, the doc comments were not properly updated.
Fix these to avoid confusion and 'make htmldocs' warnings.
Fixes: b9dce8a1ed3e ("kunit: Add kunit_add_action() to defer a call until test exit")
Reported-by: Stephen Rothwell <sfr@canb.auug.org.au>
Closes: https://lore.kernel.org/lkml/20230530151840.16a56460@canb.auug.org.au/
Signed-off-by: David Gow <davidgow@google.com>
Reviewed-by: Maxime Ripard <mripard@kernel.org>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
show more ...
|
| b1eaa8b2 | 17-May-2023 |
Michal Wajdeczko <michal.wajdeczko@intel.com> |
kunit: Update kunit_print_ok_not_ok function
There is no need use opaque test_or_suite pointer and is_test flag
as we don't use anything from the suite struct. Always expect test
pointer and use NUL
kunit: Update kunit_print_ok_not_ok function
There is no need use opaque test_or_suite pointer and is_test flag
as we don't use anything from the suite struct. Always expect test
pointer and use NULL as indication that provided results are from
the suite so we can treat them differently.
Since results could be from nested tests, like parameterized tests,
add explicit level parameter to properly indent output messages and
thus allow to reuse this function from other places.
While around, remove small code duplication near skip directive.
Signed-off-by: Michal Wajdeczko <michal.wajdeczko@intel.com>
Cc: David Gow <davidgow@google.com>
Cc: Rae Moar <rmoar@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
show more ...
|
| 57e3cded | 24-May-2023 |
David Gow <davidgow@google.com> |
kunit: kmalloc_array: Use kunit_add_action()
The kunit_add_action() function is much simpler and cleaner to use that
the full KUnit resource API for simple things like the
kunit_kmalloc_array() func
kunit: kmalloc_array: Use kunit_add_action()
The kunit_add_action() function is much simpler and cleaner to use that
the full KUnit resource API for simple things like the
kunit_kmalloc_array() functionality.
Replacing it allows us to get rid of a number of helper functions, and
leaves us with no uses of kunit_alloc_resource(), which has some
usability problems and is going to have its behaviour modified in an
upcoming patch.
Note that we need to use kunit_defer_trigger_all() to implement
kunit_kfree().
Reviewed-by: Benjamin Berg <benjamin.berg@intel.com>
Reviewed-by: Maxime Ripard <maxime@cerno.tech>
Tested-by: Maxime Ripard <maxime@cerno.tech>
Signed-off-by: David Gow <davidgow@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
show more ...
|
| b9dce8a1 | 24-May-2023 |
David Gow <davidgow@google.com> |
kunit: Add kunit_add_action() to defer a call until test exit
Many uses of the KUnit resource system are intended to simply defer
calling a function until the test exits (be it due to success or
fai
kunit: Add kunit_add_action() to defer a call until test exit
Many uses of the KUnit resource system are intended to simply defer
calling a function until the test exits (be it due to success or
failure). The existing kunit_alloc_resource() function is often used for
this, but was awkward to use (requiring passing NULL init functions, etc),
and returned a resource without incrementing its reference count, which
-- while okay for this use-case -- could cause problems in others.
Instead, introduce a simple kunit_add_action() API: a simple function
(returning nothing, accepting a single void* argument) can be scheduled
to be called when the test exits. Deferred actions are called in the
opposite order to that which they were registered.
This mimics the devres API, devm_add_action(), and also provides
kunit_remove_action(), to cancel a deferred action, and
kunit_release_action() to trigger one early.
This is implemented as a resource under the hood, so the ordering
between resource cleanup and deferred functions is maintained.
Reviewed-by: Benjamin Berg <benjamin.berg@intel.com>
Reviewed-by: Maxime Ripard <maxime@cerno.tech>
Tested-by: Maxime Ripard <maxime@cerno.tech>
Signed-off-by: David Gow <davidgow@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
show more ...
|
| 7232282d | 27-Mar-2023 |
Heiko Carstens <hca@linux.ibm.com> |
kunit: increase KUNIT_LOG_SIZE to 2048 bytes
The s390 specific test_unwind kunit test has 39 parameterized tests. The
results in debugfs are truncated since the full log doesn't fit into 1500
bytes.
kunit: increase KUNIT_LOG_SIZE to 2048 bytes
The s390 specific test_unwind kunit test has 39 parameterized tests. The
results in debugfs are truncated since the full log doesn't fit into 1500
bytes.
Therefore increase KUNIT_LOG_SIZE to 2048 bytes in a similar way like it
was done recently with commit "kunit: fix bug in debugfs logs of
parameterized tests". With that the whole test result is present.
Reported-by: Alexander Egorenkov <egorenar@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Reviewed-by: Rae Moar <rmoar@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
show more ...
|
| 2c6a96da | 08-Mar-2023 |
Rae Moar <rmoar@google.com> |
kunit: fix bug of extra newline characters in debugfs logs
Fix bug of the extra newline characters in debugfs logs. When a
line is added to debugfs with a newline character at the end,
an extra line
kunit: fix bug of extra newline characters in debugfs logs
Fix bug of the extra newline characters in debugfs logs. When a
line is added to debugfs with a newline character at the end,
an extra line appears in the debugfs log.
This is due to a discrepancy between how the lines are printed and how they
are added to the logs. Remove this discrepancy by checking if a newline
character is present before adding a newline character. This should closely
match the printk behavior.
Add kunit_log_newline_test to provide test coverage for this issue. (Also,
move kunit_log_test above suite definition to remove the unnecessary
declaration prior to the suite definition)
As an example, say we add these two lines to the log:
kunit_log(..., "KTAP version 1\n");
kunit_log(..., "1..1");
The debugfs log before this fix:
KTAP version 1
1..1
The debugfs log after this fix:
KTAP version 1
1..1
Signed-off-by: Rae Moar <rmoar@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
show more ...
|
| e047c5ea | 31-Jan-2023 |
David Gow <davidgow@google.com> |
kunit: Expose 'static stub' API to redirect functions
Add a simple way of redirecting calls to functions by including a
special prologue in the "real" function which checks to see if the
replacement
kunit: Expose 'static stub' API to redirect functions
Add a simple way of redirecting calls to functions by including a
special prologue in the "real" function which checks to see if the
replacement function should be called (and, if so, calls it).
To redirect calls to a function, make the first (non-declaration) line
of the function:
KUNIT_STATIC_STUB_REDIRECT(function_name, [function arguments]);
(This will compile away to nothing if KUnit is not enabled, otherwise it
will check if a redirection is active, call the replacement function,
and return. This check is protected by a static branch, so has very
little overhead when there are no KUnit tests running.)
Calls to the real function can be redirected to a replacement using:
kunit_activate_static_stub(test, real_fn, replacement_fn);
The redirection will only affect calls made from within the kthread of
the current test, and will be automatically disabled when the test
completes. It can also be manually disabled with
kunit_deactivate_static_stub().
The 'example' KUnit test suite has a more complete example.
Co-developed-by: Daniel Latypov <dlatypov@google.com>
Signed-off-by: Daniel Latypov <dlatypov@google.com>
Signed-off-by: David Gow <davidgow@google.com>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
show more ...
|
| 254c7137 | 30-Jan-2023 |
Brendan Higgins <brendan.higgins@linux.dev> |
kunit: fix kunit_test_init_section_suites(...)
Looks like kunit_test_init_section_suites(...) was messed up in a merge
conflict. This fixes it.
kunit_test_init_section_suites(...) was not updated t
kunit: fix kunit_test_init_section_suites(...)
Looks like kunit_test_init_section_suites(...) was messed up in a merge
conflict. This fixes it.
kunit_test_init_section_suites(...) was not updated to avoid the extra
level of indirection when .kunit_test_suites was flattened. Given no-one
was actively using it, this went unnoticed for a long period of time.
Fixes: e5857d396f35 ("kunit: flatten kunit_suite*** to kunit_suite** in .kunit_test_suites")
Signed-off-by: Brendan Higgins <brendan.higgins@linux.dev>
Signed-off-by: David Gow <davidgow@google.com>
Tested-by: Martin Fernandez <martin.fernandez@eclypsium.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
show more ...
|