Bug 207923

Summary: kunit_tool should print an error if UML doesn't start
Product: Tools Reporter: David Gow (davidgow)
Component: KUnitAssignee: Tools/KUnit virtual default assignee (tools_kunit)
Status: RESOLVED CODE_FIX    
Severity: normal    
Priority: P1    
Hardware: All   
OS: Linux   
Kernel Version: 5.6.0 Subsystem:
Regression: No Bisected commit-id:

Description David Gow 2020-05-27 23:18:38 UTC 
At present, kunit_tool simply reports "0 tests run" if UML fails to start, which can be misleading.

It'd be nice if a "Couldn't start kernel", or even a "No KUnit output detected" or similar message could be displayed if the kernel is never able to get to the point of running KUnit tests. "0 tests run" should ideally only show up if an explicit test plan with only 0 tests is present in the log.

See, as an example of where this was confusing: https://lore.kernel.org/linux-kselftest/ef994ce9-ccd8-e9c9-9838-049a62a3a561@intel.com/
Comment 1 David Gow 2021-06-16 02:55:16 UTC 
This currently reports no tests having been run as an error:

[19:13:18] Starting KUnit Kernel ...
[ERROR] no tests run!
[19:13:18] ============================================================
[19:13:18] Testing complete. 0 tests run. 0 failed. 0 crashed.
[19:13:18] Elapsed time: 0.000s

And Python errors are generated instead for cases where the UML kernel cannot start at all, e.g.:
[Stacktrace elided]
FileNotFoundError: [Errno 2] No such file or directory: '.kunit/linux' 

If there are zero tests configured, we report:

[ERROR] could not parse test results!

The raw TAP output in this case is:
TAP version 14
1..0

It seems to me like these two error messages should really be the other way around, but this is otherwise _mostly_ fixed.
Comment 2 David Gow 2021-06-29 22:58:27 UTC 
The error messages will be the right way around once this patch is accepted:
https://lore.kernel.org/linux-kselftest/20210629223541.1512296-1-rmoar@google.com/