This Kselftest update for 4.18-rc1 consists of:
- Work to restructure timers test suite to move PIE out of rtctest from
Alexandre Belloni.
- Several minor spelling and bug fixes.
- New cgroup tests from Roman Gushchin and Mike Rapoport.
- Kselftest framework changes to handle and report skipped tests correctly.
Prior to these changes, framework treated all non-zero return codes from
tests as failures. When tests are skipped with non-zero return code, due
to unmet dependencies and/or unsupported configuration, reporting them as
failed lead to false negatives on the tests that couldn't be run.
- Fixes to test Makefiles to remove unnecessary RUN_TESTS and EMIT_TESTS
overrides and use common defines from lib.mk.
- Fixes to several tests to return correct Kselftest skip code.
- Changes to improve test output.
-----BEGIN PGP SIGNATURE-----
iQIcBAABCAAGBQJbFaYgAAoJEAsCRMQNDUMccHkQAK3Bx9dw7MBR9z4LsT+/0Qr8
sgQp2WivuricX71T0JjqXImWsrloYS+4ChvaFCmvy1xIK29/4TtiXJR/pCcJrsQr
JA3AzurtLVjNGLxKOJqP+ALxy/oc7JYXcmooxL332fmQafNsfazJm4HKWHwNuSJM
2fQ2xdbqWo3LT+pNHX1i7H0JRrudGbPmCufUO0UZra/9oTrlPgL6oC7q7+KG6hLL
wLSQlF88MYXUm/lnNZyHc9aWGkOLmHcaHPXxMiCo/indI7UN+COUix2lOQ7dZ9a1
Paoo7f9GaS0pKfBGEhRc1aKzNOjDZgUPM+VWgKCkx9B2U8iUNwogt/jDh3feqzwJ
7yhha/g5vSLdhoWiFINmc0R+DEJ+OnG6v+G50M+XIYyriQ7zCoe209EOjgw4yhE4
epJL9CofiOt0/n7UGSNvvCzoEOtlBwhpmKzhnBMpYEzD8cykAJKOrUGjRs/URHyC
MOkGxz4ZDOqsJSfsDd7aJ122SfeatwYey5H6GQgCgoP1D//qXRv3T7CVdIcPcFKs
mhZEn0pekreMyz0dSpPHHfKFaNQBvUJ35taDq5r948mL6QN+TUev4mNjpnHTYgSy
U7QdLiRKft3a01GCZedL/+NdMnege2kvtjpQka9Sa08/HN3aqMmYQHiQM101egUD
/jUd/WF8O5/fNMkGOExk
=eVzz
-----END PGP SIGNATURE-----
Merge tag 'linux-kselftest-4.18-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest
Pull Kselftest update from Shuah Khan:
- Work to restructure timers test suite to move PIE out of rtctest from
Alexandre Belloni.
- Several minor spelling and bug fixes.
- New cgroup tests from Roman Gushchin and Mike Rapoport.
- Kselftest framework changes to handle and report skipped tests
correctly.
Prior to these changes, framework treated all non-zero return codes
from tests as failures. When tests are skipped with non-zero return
code, due to unmet dependencies and/or unsupported configuration,
reporting them as failed lead to false negatives on the tests that
couldn't be run.
- Fixes to test Makefiles to remove unnecessary RUN_TESTS and
EMIT_TESTS overrides and use common defines from lib.mk.
- Fixes to several tests to return correct Kselftest skip code.
- Changes to improve test output.
* tag 'linux-kselftest-4.18-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest: (55 commits)
selftests: lib: fix prime_numbers module search and skip logic
selftests: intel_pstate: notification about privilege required to run intel_pstate testing script
selftests: cgroup/memcontrol: add basic test for socket accounting
selftest: intel_pstate: debug support message from aperf.c and return value
kselftest/cgroup: fix variable dereferenced before check warning
selftests/intel_pstate: Enhance table printing
selftests/intel_pstate: Improve test, minor fixes
selftests: cgroup/memcontrol: add basic test for swap controls
selftests: cgroup: add memory controller self-tests
selftests: memfd: split regular and hugetlbfs tests
selftests: net: return Kselftest Skip code for skipped tests
selftests: mqueue: return Kselftest Skip code for skipped tests
selftests: memory-hotplug: return Kselftest Skip code for skipped tests
selftests: memfd: return Kselftest Skip code for skipped tests
selftests: membarrier: return Kselftest Skip code for skipped tests
selftests: media_tests: return Kselftest Skip code for skipped tests
selftests: locking: return Kselftest Skip code for skipped tests
selftests: locking: add Makefile for locking test
selftests: lib: return Kselftest Skip code for skipped tests
selftests: lib: add prime_numbers.sh test to Makefile
...
Here is the big USB pull request for 4.18-rc1.
Lots of stuff here, the highlights are:
- phy driver updates and new additions
- usual set of xhci driver updates
- normal set of musb updates
- gadget driver updates and new controllers
- typec work, it's getting closer to getting fully out of the
staging portion of the tree.
- lots of minor cleanups and bugfixes.
All of these have been in linux-next for a while with no reported
issues.
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
-----BEGIN PGP SIGNATURE-----
iG0EABECAC0WIQT0tgzFv3jCIUoxPcsxR9QN2y37KQUCWxba6w8cZ3JlZ0Brcm9h
aC5jb20ACgkQMUfUDdst+ykumQCg2abWE5LijR0SNJIwX8xk64HLUAMAnAxBZDG3
aB0GyOQd54L+09q4LAdn
=ZCEx
-----END PGP SIGNATURE-----
Merge tag 'usb-4.18-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/usb
Pull USB and PHY updates from Greg KH:
"Here is the big USB pull request for 4.18-rc1.
Lots of stuff here, the highlights are:
- phy driver updates and new additions
- usual set of xhci driver updates
- normal set of musb updates
- gadget driver updates and new controllers
- typec work, it's getting closer to getting fully out of the staging
portion of the tree.
- lots of minor cleanups and bugfixes.
All of these have been in linux-next for a while with no reported
issues"
* tag 'usb-4.18-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/usb: (263 commits)
Revert "xhci: Reset Renesas uPD72020x USB controller for 32-bit DMA issue"
xhci: Add quirk to zero 64bit registers on Renesas PCIe controllers
xhci: Allow more than 32 quirks
usb: xhci: force all memory allocations to node
selftests: add test for USB over IP driver
USB: typec: fsusb302: no need to check return value of debugfs_create_dir()
USB: gadget: udc: s3c2410_udc: no need to check return value of debugfs_create functions
USB: gadget: udc: renesas_usb3: no need to check return value of debugfs_create functions
USB: gadget: udc: pxa27x_udc: no need to check return value of debugfs_create functions
USB: gadget: udc: gr_udc: no need to check return value of debugfs_create functions
USB: gadget: udc: bcm63xx_udc: no need to check return value of debugfs_create functions
USB: udc: atmel_usba_udc: no need to check return value of debugfs_create functions
USB: dwc3: no need to check return value of debugfs_create functions
USB: dwc2: no need to check return value of debugfs_create functions
USB: core: no need to check return value of debugfs_create functions
USB: chipidea: no need to check return value of debugfs_create functions
USB: ehci-hcd: no need to check return value of debugfs_create functions
USB: fhci-hcd: no need to check return value of debugfs_create functions
USB: fotg210-hcd: no need to check return value of debugfs_create functions
USB: imx21-hcd: no need to check return value of debugfs_create functions
...
Pull RCU updates from Ingo Molnar:
- updates to the handling of expedited grace periods
- updates to reduce lock contention in the rcu_node combining tree
[ These are in preparation for the consolidation of RCU-bh,
RCU-preempt, and RCU-sched into a single flavor, which was
requested by Linus in response to a security flaw whose root cause
included confusion between the multiple flavors of RCU ]
- torture-test updates that save their users some time and effort
- miscellaneous fixes
* 'core-rcu-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: (44 commits)
rcu/x86: Provide early rcu_cpu_starting() callback
torture: Make kvm-find-errors.sh find build warnings
rcutorture: Abbreviate kvm.sh summary lines
rcutorture: Print end-of-test state in kvm.sh summary
rcutorture: Print end-of-test state
torture: Fold parse-torture.sh into parse-console.sh
torture: Add a script to edit output from failed runs
rcu: Update list of rcu_future_grace_period() trace events
rcu: Drop early GP request check from rcu_gp_kthread()
rcu: Simplify and inline cpu_needs_another_gp()
rcu: The rcu_gp_cleanup() function does not need cpu_needs_another_gp()
rcu: Make rcu_start_this_gp() check for out-of-range requests
rcu: Add funnel locking to rcu_start_this_gp()
rcu: Make rcu_start_future_gp() caller select grace period
rcu: Inline rcu_start_gp_advanced() into rcu_start_future_gp()
rcu: Clear request other than RCU_GP_FLAG_INIT at GP end
rcu: Cleanup, don't put ->completed into an int
rcu: Switch __rcu_process_callbacks() to rcu_accelerate_cbs()
rcu: Avoid __call_rcu_core() root rcu_node ->lock acquisition
rcu: Make rcu_migrate_callbacks wake GP kthread when needed
...
Add test for USB over IP driver. This test runs several tests on a device
specified in the -b <busid> argument and path to the usbip tools.
usbip_test.sh -b <busid> -p <usbip tools path>
e.g:
cd tools/testing selftests/drivers/usb/usbip
sudo ./usbip_test.sh -b 3-10.2 -p <yoursrctree>/tools/usb/usbip
This test should be run as root and user should build usbip tools before
running the test.
The usbip test isn't included in the Kselftest run as it requires user to
specify a device to run tests on.
Signed-off-by: Shuah Khan (Samsung OSG) <shuah@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
prime_numbers modules search and skip logic removes the module instead
of searching for it. Fix it.
Signed-off-by: Shuah Khan (Samsung OSG) <shuah@kernel.org>
The intel_pstate related testing script need root level privileges
when trying to access certain file for the successful execution of
the script.But this is not the case always like when using evaluation
only mode, which only require user level privilege.
This patch is to notify the user about the privilege the script
demands for the successful execution of the test.
Signed-off-by: Jeffrin Jose T (Rajagiri SET) <ahiliation@gmail.com>
Signed-off-by: Shuah Khan (Samsung OSG) <shuah@kernel.org>
The test verifies that when there is active TCP connection, the
memory.stat.sock and memory.current values are close.
Signed-off-by: Mike Rapoport <rppt@linux.vnet.ibm.com>
Acked-by: Roman Gushchin <guro@fb.com>
Signed-off-by: Shuah Khan (Samsung OSG) <shuah@kernel.org>
Additional message along with an error message which is more
verbose for debug support from aperf.c and updated with the
new return value "KSFT_SKIP".
Signed-off-by: Jeffrin Jose T [Rajagiri SET] <ahiliation@gmail.com>
Signed-off-by: Shuah Khan (Samsung OSG) <shuah@kernel.org>
cg_name(const char *root, const char *name) is always called with
non-empty root and name arguments, so there is no sense in checking
it in the function body (after using in strlen()).
Signed-off-by: Roman Gushchin <guro@fb.com>
Reported-by: Dan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: Shuah Khan (Samsung OSG) <shuah@kernel.org>
Using coreutils' pr, a nicer table is printed out with the
results.
Signed-off-by: Daniel Díaz <daniel.diaz@linaro.org>
Signed-off-by: Shuah Khan (Samsung OSG) <shuah@kernel.org>
A few changes improve the overall usability of the test:
* fix a hard-coded maximum frequency (3300),
* don't adjust the CPU frequency if only evaluating results,
* fix a comparison for multiple frequencies.
A symptom of that last issue looked like this:
./run.sh: line 107: [: too many arguments
./run.sh: line 110: 3099
3099
3100-3100: syntax error in expression (error token is \"3099
3100-3100\")
Because a check will count how many differente frequencies
there are among the CPUs of the system, and after they are
tallied another read is performed, which might produce
different results.
Signed-off-by: Daniel Díaz <daniel.diaz@linaro.org>
Signed-off-by: Shuah Khan (Samsung OSG) <shuah@kernel.org>
The new test verifies that memory.swap.max and memory.swap.current behave
as expected for simple allocation scenarios
Signed-off-by: Mike Rapoport <rppt@linux.vnet.ibm.com>
Acked-by: Tejun Heo <tj@kernel.org>
Acked-by: Roman Gushchin <guro@fb.com>
Signed-off-by: Shuah Khan (Samsung OSG) <shuah@kernel.org>
Cgroups are used for controlling the physical resource distribution
(memory, CPU, io, etc) and often are used as basic building blocks
for large distributed computing systems. Even small differences
in the actual behavior may lead to significant incidents.
The codebase is under the active development, which will unlikely
stop at any time soon. Also it's scattered over different kernel
subsystems, which makes regressions more probable.
Given that, the lack of any tests is crying.
This patch implements some basic tests for the memory controller,
as well as a minimal required framework. It doesn't pretend for a
very good coverage, but pretends to be a starting point.
Hopefully, any following significant changes will include corresponding
tests.
Tests for CPU and io controllers, as well as cgroup core
are next in the todo list.
Signed-off-by: Roman Gushchin <guro@fb.com>
Cc: Tejun Heo <tj@kernel.org>
Cc: Shuah Khan <shuah@kernel.org>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Mike Rapoport <rppt@linux.vnet.ibm.com>
Cc: kernel-team@fb.com
Cc: linux-kselftest@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Acked-by: Tejun Heo <tj@kernel.org>
Signed-off-by: Shuah Khan (Samsung OSG) <shuah@kernel.org>
Split normal memfd and hugetlbfs tests to improve the test reporting.
Remove run_fuse_test.sh and memfd_test from run_tests.sh and add them
to the Makefile.
Add memfd_test to TEST_GEN_PROGS to be run separately.
Rename run_tests.sh to run_hugetlbfs_test.sh
Add run_fuse_test.sh and run_hugetlbfs_test.sh to TEST_PROGS
The report for non-root run wth this change is:
TAP version 13
selftests: memfd: memfd_test
========================================
memfd: CREATE
memfd: BASIC
memfd: SEAL-WRITE
memfd: SEAL-SHRINK
memfd: SEAL-GROW
memfd: SEAL-RESIZE
memfd: SHARE-DUP
memfd: SHARE-MMAP
memfd: SHARE-OPEN
memfd: SHARE-FORK
memfd: SHARE-DUP (shared file-table)
memfd: SHARE-MMAP (shared file-table)
memfd: SHARE-OPEN (shared file-table)
memfd: SHARE-FORK (shared file-table)
memfd: DONE
ok 1..1 selftests: memfd: memfd_test [PASS]
selftests: memfd: run_fuse_test.sh
========================================
opening: ./mnt/memfd
fuse: DONE
ok 1..2 selftests: memfd: run_fuse_test.sh [PASS]
selftests: memfd: run_hugetlbfs_test.sh
========================================
Please run memfd with hugetlbfs test as root
not ok 1..3 selftests: memfd: run_hugetlbfs_test.sh [SKIP]
Signed-off-by: Shuah Khan (Samsung OSG) <shuah@kernel.org>
Reviewed-by: Mike Kravetz <mike.kravetz@oracle.com>
Signed-off-by: Shuah Khan (Samsung OSG) <shuah@kernel.org>
When net test is skipped because of unmet dependencies and/or unsupported
configuration, it returns 0 which is treated as a pass by the Kselftest
framework. This leads to false positive result even when the test could
not be run.
Change it to return kselftest skip code when a test gets skipped to
clearly report that the test could not be run.
Kselftest framework SKIP code is 4 and the framework prints appropriate
messages to indicate that the test is skipped.
Change psock_tpacket to use ksft_exit_skip() when a non-root user runs
the test and add an explicit check for root and a clear message, instead
of failing the test when /sys/power/state file open fails.
Signed-off-by: Shuah Khan (Samsung OSG) <shuah@kernel.org>
When mqueue test is skipped because of unmet dependencies and/or
unsupported configuration, it exits with error which is treated as
a fail by the Kselftest framework. This leads to false negative
result even when the test could not be run.
Change it to return kselftest skip code when a test gets skipped to
clearly report that the test could not be run.
Change it to use ksft_exit_skip() when the test is skipped.
Signed-off-by: Shuah Khan (Samsung OSG) <shuah@kernel.org>
When memory-hotplug test is skipped because of unmet dependencies and/or
unsupported configuration, it returns non-zero value hich is treated as a
fail by the Kselftest framework. This leads to false negative result even
when the test could not be run.
Change it to return kselftest skip code when a test gets skipped to
clearly report that the test could not be run.
Kselftest framework SKIP code is 4 and the framework prints appropriate
messages to indicate that the test is skipped.
Signed-off-by: Shuah Khan (Samsung OSG) <shuah@kernel.org>
When memfd test is skipped because of unmet dependencies and/or unsupported
configuration, it returns non-zero value which is treated as a fail by the
Kselftest framework. This leads to false negative result even when the test
could not be run.
Change it to return kselftest skip code when a test gets skipped to clearly
report that the test could not be run.
Added an explicit check for root user at the start of memfd hugetlbfs test
and return skip code if a non-root user attempts to run it.
In addition, return skip code when not enough huge pages are available to
run the test.
Kselftest framework SKIP code is 4 and the framework prints appropriate
messages to indicate that the test is skipped.
Signed-off-by: Shuah Khan (Samsung OSG) <shuah@kernel.org>
Reviewed-by: Mike Kravetz <mike.kravetz@oracle.com>
Signed-off-by: Shuah Khan (Samsung OSG) <shuah@kernel.org>
When membarrier test is skipped because of unmet dependencies and/or
unsupported configuration, it exits with error which is treated as a
fail by the Kselftest framework. This leads to false negative result
even when the test could not be run.
Change it to return kselftest skip code when a test gets skipped to
clearly report that the test could not be run.
Change it to use ksft_exit_skip() when the test is skipped.
Signed-off-by: Shuah Khan (Samsung OSG) <shuah@kernel.org>
When media_tests test is skipped because of unmet dependencies and/or
unsupported configuration, it exits with error which is treated as a
fail by the Kselftest framework. This leads to false negative result
even when the test could not be run.
Change it to return kselftest skip code when a test gets skipped to
clearly report that the test could not be run.
Change it to use ksft_exit_skip() when the test is skipped.
Signed-off-by: Shuah Khan (Samsung OSG) <shuah@kernel.org>
When locking test is skipped because of unmet dependencies and/or
unsupported configuration, it exits with error which is treated as
a fail by the Kselftest framework. This leads to false negative
result even when the test could not be run.
Change it to return kselftest skip code when a test gets skipped to
clearly report that the test could not be run.
Added an explicit search for ww_mutex module and return skip code if
it isn't found to differentiate between the failure to load the module
condition and module not found condition.
Kselftest framework SKIP code is 4 and the framework prints appropriate
messages to indicate that the test is skipped.
Signed-off-by: Shuah Khan (Samsung OSG) <shuah@kernel.org>
When lib test(s) is skipped because of unmet dependencies and/or
unsupported configuration, it returns non-zero value hich is treated
as a fail by the Kselftest framework. This leads to false negative result
even when the test could not be run.
Change it to return kselftest skip code when a test gets skipped to
clearly report that the test could not be run.
Kselftest framework SKIP code is 4 and the framework prints appropriate
messages to indicate that the test is skipped.
Signed-off-by: Shuah Khan (Samsung OSG) <shuah@kernel.org>
When kvm test is skipped because of unmet dependencies and/or unsupported
configuration, it exits with error which is treated as a fail by the
Kselftest framework. This leads to false negative result even when the test
could not be run.
Change it to return kselftest skip code when a test gets skipped to clearly
report that the test could not be run.
Change it to use ksft_exit_skip() when the test is skipped. In addition,
refine test_assert() message to include strerror() string and add explicit
check for EACCES to cleary identify when test doesn't run when access is
denied to resources required e.g: open /dev/kvm failed, rc: -1 errno: 13
Signed-off-by: Shuah Khan (Samsung OSG) <shuah@kernel.org>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Shuah Khan (Samsung OSG) <shuah@kernel.org>
When kmod test is skipped because of unmet dependencies and/or unsupported
configuration, it returns 0 which is treated as a pass by the Kselftest
framework. This leads to false positive result even when the test could
not be run. It returns fail in some cases when test is skipped. Either way,
it is incorrect and incosnistent reporting.
Change it to return kselftest skip code when a test gets skipped to
clearly report that the test could not be run.
Kselftest framework SKIP code is 4 and the framework prints appropriate
messages to indicate that the test is skipped.
Signed-off-by: Shuah Khan (Samsung OSG) <shuah@kernel.org>
Reviewed-by: Luis R. Rodriguez <mcgrof@kernel.org>
Signed-off-by: Shuah Khan (Samsung OSG) <shuah@kernel.org>
When ipc test is skipped because of unmet dependencies and/or
unsupported configuration, it exits with error which is treated as
a fail by the Kselftest framework. This leads to false negative
result even when the test could not be run.
Change it to return kselftest skip code when a test gets skipped to
clearly report that the test could not be run.
Change it to use ksft_exit_skip() when the test is skipped.
Signed-off-by: Shuah Khan (Samsung OSG) <shuah@kernel.org>
When intel_pstate test is skipped because of unmet dependencies and/or
unsupported configuration, it returns 0 which is treated as a pass
by the Kselftest framework. This leads to false positive result even
when the test could not be run.
Change it to return kselftest skip code when a test gets skipped to
clearly report that the test could not be run.
Kselftest framework SKIP code is 4 and the framework prints appropriate
messages to indicate that the test is skipped.
Signed-off-by: Shuah Khan (Samsung OSG) <shuah@kernel.org>
When gpio test is skipped because of unmet dependencies and/or
unsupported configuration, it exits with error which is treated as
a fail by the Kselftest framework. This leads to false negative
result even when the test could not be run.
Change it to return kselftest skip code when a test gets skipped to
clearly report that the test could not be run.
Kselftest framework SKIP code is 4 and the framework prints appropriate
messages to indicate that the test is skipped.
Signed-off-by: Shuah Khan (Samsung OSG) <shuah@kernel.org>
When firmware test(s) get skipped because of unmet dependencies and/or
unsupported configuration, it returns 0 which is treated as a pass
by the Kselftest framework. This leads to false positive result even
when the test could not be run.
Change it to return kselftest skip code when a test gets skipped to
clearly report that the test could not be run.
Kselftest framework SKIP code is 4 and the framework prints appropriate
messages to indicate that the test is skipped.
Signed-off-by: Shuah Khan (Samsung OSG) <shuah@kernel.org>
Reviewed-by: Luis R. Rodriguez <mcgrof@kernel.org>
Signed-off-by: Shuah Khan (Samsung OSG) <shuah@kernel.org>
When devpts_pts test is skipped because of unmet dependencies and/or
unsupported configuration, it exits with error which is treated as
a fail by the Kselftest framework. This leads to false negative
result even when the test could not be run.
In another case, it returns pass for a skipped test reporting a false
postive.
Change it to return kselftest skip code when a test gets skipped to
clearly report that the test could not be run.
Change it to use ksft_exit_skip() when test is skipped.
Signed-off-by: Shuah Khan (Samsung OSG) <shuah@kernel.org>
Acked-by: Christian Brauner <christian.brauner@ubuntu.com>
Signed-off-by: Shuah Khan (Samsung OSG) <shuah@kernel.org>
When execveat test is skipped because of unmet dependencies and/or
unsupported configuration, it exits with error which is treated as
a fail by the Kselftest framework. This leads to false negative
result even when the test could not be run.
Change it to return kselftest skip code when a test gets skipped to
clearly report that the test could not be run.
Change it to use ksft_exit_skip() when kernel doesn't support execveat.
Signed-off-by: Shuah Khan (Samsung OSG) <shuah@kernel.org>
When efivarfs test is skipped because of unmet dependencies and/or
unsupported configuration, it returns 0 which is treated as a pass
by the Kselftest framework. This leads to false positive result even
when the test could not be run.
Change it to return kselftest skip code when a test gets skipped to
clearly report that the test could not be run.
Kselftest framework SKIP code is 4 and the framework prints appropriate
messages to indicate that the test is skipped.
Signed-off-by: Shuah Khan (Samsung OSG) <shuah@kernel.org>
When cpufreq test is skipped because of unmet dependencies and/or
unsupported configuration, it exits with error which is treated as
a fail by the Kselftest framework. This leads to false negative
result even when the test could not be run.
Change it to return kselftest skip code when a test gets skipped to
clearly report that the test could not be run.
Kselftest framework SKIP code is 4 and the framework prints appropriate
messages to indicate that the test is skipped.
Signed-off-by: Shuah Khan (Samsung OSG) <shuah@kernel.org>
Acked-by: Viresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: Shuah Khan (Samsung OSG) <shuah@kernel.org>
When cpu-on-off-test is skipped because of unmet dependencies and/or
unsupported configuration, it returns 0 which is treated as a pass
by the Kselftest framework. This leads to false positive result even
when the test could not be run.
Change it to return kselftest skip code when a test gets skipped to
clearly report that the test could not be run.
Kselftest framework SKIP code is 4 and the framework prints appropriate
messages to indicate that the test is skipped.
Signed-off-by: Shuah Khan (Samsung OSG) <shuah@kernel.org>
When step_after_suspend_test is skipped because of unmet dependencies
and/or unsupported configuration, it exits with error which is treated
as a fail by the Kselftest framework. This leads to false negative result
even when the test could not be run.
Change it to return kselftest skip code when a test gets skipped to clearly
report that the test could not be run.
Change it to use ksft_exit_skip() when a non-root user runs the test and
add an explicit check for root and a clear message, instead of failing
the test when /sys/power/state file open fails.
Signed-off-by: Shuah Khan (Samsung OSG) <shuah@kernel.org>
When ion test is skipped because of unmet dependencies and/or unsupported
configuration, it returns 0 which is treated as a pass by the Kselftest
framework. This leads to false positive result even when the test could
not be run.
Change it to return kselftest skip code when a test gets skipped to
clearly report that the test could not be run.
Kselftest framework SKIP code is 4 and the framework prints appropriate
messages to indicate that the test is skipped.
Signed-off-by: Shuah Khan (Samsung OSG) <shuah@kernel.org>
Acked-by: Pintu Agarwal <pintu.ping@gmail.com>
Signed-off-by: Shuah Khan (Samsung OSG) <shuah@kernel.org>
Delete RUN_TESTS and EMIT_TESTS overrides and use common defines in
lib.mk. Common defines work after making the change the test to run
with ratio=2 as the default mode to be able to invoke the test without
the "-r 2" argument from the common RUN_TESTS and EMIT_TESTS.
The run_full_tests target now calls the test with "-r 10".
Signed-off-by: Shuah Khan (Samsung OSG) <shuah@kernel.org>
Reviewed-by: Anders Roxell <anders.roxell@linaro.org>
Reviewed-by: Lei.Yang@windriver.com
Signed-off-by: Shuah Khan (Samsung OSG) <shuah@kernel.org>
Delete RUN_TESTS and EMIT_TESTS overrides and use common defines in lib.mk.
The overrides are in place to call mq_open_tests with queue name argument.
The change to delete overrides is coupled with a change to mq_open_tests
to use default queue name when it is called without one.
Signed-off-by: Shuah Khan (Samsung OSG) <shuah@kernel.org>
Reviewed-by: Anders Roxell <anders.roxell@linaro.org>
Signed-off-by: Shuah Khan (Samsung OSG) <shuah@kernel.org>
Delete RUN_TESTS and EMIT_TESTS overrides and use common defines in
lib.mk. Add new run_tests.sh to do the dependency checks the custom
RUN_TESTS did. Common defines work with the run_tests.sh set as the
TEST_PROGS and defining unprivileged-remount-test in TEST_GEN_FILES.
Kselftest framework builds and installs TEST_GEN_FILES and doesn't run
them via RUN_TESTS and include it in EMIT_TESTS. With this change the
new run_tests.sh runs the test after checking dependencies.
This change also adds Skip handling to return kselftest skip code when
test is skipped to clearly identify when the test is skipped instead of
reporting it as failed.
Output with this change:
TAP version 13
selftests: mount: run_tests.sh
========================================
WARN: No /proc/self/uid_map exist, test skipped.
not ok 1..1 selftests: mount: run_tests.sh [SKIP]
Signed-off-by: Shuah Khan (Samsung OSG) <shuah@kernel.org>
Reviewed-by: Anders Roxell <anders.roxell@linaro.org>
Signed-off-by: Shuah Khan (Samsung OSG) <shuah@kernel.org>
Delete RUN_TESTS and EMIT_TESTS overrides and use common defines in
lib.mk. Common defines work just fine and there is no need to define
custom overrides.
Signed-off-by: Shuah Khan (Samsung OSG) <shuah@kernel.org>
Reviewed-by: Anders Roxell <anders.roxell@linaro.org>
Signed-off-by: Shuah Khan (Samsung OSG) <shuah@kernel.org>
Delete RUN_TESTS and EMIT_TESTS overrides and use common defines in
lib.mk. Common defines work just fine and there is no need to define
custom overrides.
Signed-off-by: Shuah Khan (Samsung OSG) <shuah@kernel.org>
KSFT_SKIP points to KSFT_PASS resulting in reporting skipped tests as
Passed, when test programs exit with KSFT_SKIP or call ksft_exit_skip().
If tests are skipped because of unmet dependencies and/or unsupported
configuration, reporting them as passed leads to too many false positives.
Fix it to return a skip code of 4 to clearly differentiate the skipped
tests.
Signed-off-by: Shuah Khan (Samsung OSG) <shuah@kernel.org>
Similar to what RUN_TESTS does, change EMIT_TESTS to check for execute
bit and emit code to print warnings if test isn't executable to the
the run_kselftest.sh.
Signed-off-by: Shuah Khan (Samsung OSG) <shuah@kernel.org>
EMIT_TESTS which is the common function that implements run_tests target,
treats all non-zero return codes from tests as failures. When tests are
skipped with non-zero return code, because of unmet dependencies and/or
unsupported configuration, it reports them as failed. This will lead to
too many false negatives even on the tests that couldn't be run.
EMIT_TESTS is changed to test for SKIP=4 return from tests to enable
the framework for individual tests to return special SKIP code.
Tests will be changed as needed to report SKIP instead FAIL/PASS when
they get skipped.
Currently just the test name is printed in the RUN_TESTS output. For
example, when raw_skew sub-test from timers tests in run, the output
shows just raw_skew. Include main test name when printing sub-test
results.
In addition, remove duplicate strings for printing common information with
a new for the test header information.
With this change run_kelftest.sh output for breakpoints test will be:
TAP version 13
Running tests in breakpoints
========================================
selftests: breakpoints: step_after_suspend_test
not ok 1..1 selftests: breakpoints: step_after_suspend_test [SKIP]
selftests: breakpoints: breakpoint_test
ok 1..2 selftests: breakpoints: breakpoint_test [PASS]
Signed-off-by: Shuah Khan (Samsung OSG) <shuah@kernel.org>
Currently just the test name is printed in the RUN_TESTS output. For
example, when raw_skew sub-test from timers tests in run, the output
shows just raw_skew. Include main test name when printing sub-test
results.
In addition, remove duplicate strings for printing common information
with a new for the test header information.
Before the change:
selftests: raw_skew
========================================
WARNING: ADJ_OFFSET in progress, this will cause inaccurate results
Estimating clock drift: -20.616(est) -20.586(act) [OK]
Pass 0 Fail 0 Xfail 0 Xpass 0 Skip 0 Error 0
1..0
ok 1..7 selftests: raw_skew [PASS]
After the change:
selftests: timers: raw_skew
========================================
WARNING: ADJ_OFFSET in progress, this will cause inaccurate results
Estimating clock drift: -19.794(est) -19.896(act) [OK]
Pass 0 Fail 0 Xfail 0 Xpass 0 Skip 0 Error 0
1..0
ok 1..7 selftests: timers: raw_skew [PASS]
Signed-off-by: Shuah Khan (Samsung OSG) <shuah@kernel.org>
RUN_TESTS function has grown and becoming harder to maintain. Move
the code that runs and tests for returns codes to a new function
and call it from RUN_TESTS.
A new RUN_TEST_PRINT_RESULT is created to simplify RUN_TESTS and make it
easier to add handling for other return codes as needed.
Signed-off-by: Shuah Khan (Samsung OSG) <shuah@kernel.org>
RUN_TESTS which is the common function that implements run_tests target,
treats all non-zero return codes from tests as failures. When tests are
skipped with non-zero return code, because of unmet dependencies and/or
unsupported configuration, it reports them as failed. This will lead to
too many false negatives even on the tests that couldn't be run.
RUN_TESTS is changed to test for SKIP=4 return from tests to enable the
framework for individual tests to return special SKIP code.
Tests will be changed as needed to report SKIP instead FAIL/PASS when
they get skipped.
Signed-off-by: Shuah Khan (Samsung OSG) <shuah@kernel.org>
Refine RUN_TESTS define's output block for summary and non-summary code
to remove duplicate code and make it readable.
cd `dirname $$TEST` > /dev/null; and cd - > /dev/null; are moved
to common code block and indentation fixed.
Signed-off-by: Shuah Khan (Samsung OSG) <shuah@kernel.org>
Rework rtctest to use the test harness to better handle skipping tests
(e.g. when alarms are not available). Also, it now handles timeout so it
will not block expecting an alarm that never comes.
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Shuah Khan (Samsung OSG) <shuah@kernel.org>
