-
Notifications
You must be signed in to change notification settings - Fork 1.4k
Description
NTFC is a new testing tool based on test cases for NuttX from Xiaomi that allows you to automate NuttX tests for various targets (sim, qemu, hw with serial port). All features of currently used in CI tool (tools/ci/testrun) are supported (and much more).
Here is a discussion on mailing list: https://lists.apache.org/thread/618konqxrl38oq7049njgks2y0x5mllq
At the moment, the repositories with NTFC and test cases are on my Github account:
They will eventually go to the Apache Foundation account if the proposal is accepted, but at the moment it's easier for me to develop the project in a private repo. Once NTFC is running stably on NuttX CI, I think it will be a good time to move the repositories to Apache.
Roadmap
Initial plan to add NTFC support in NuttX CI:
-
Modify nuttx and nuttx-apps repo to support NTFC.
github/ci: add support for NTFC nuttx-apps#3291
initial support for automated testing with NTFC #17728 -
Migrate
sim/citestto NTFC and observe if everything works. -
Migrating the rest of
citest:
Migrate the rest of citest configurations to NTFC #17766 -
Add support for
citestfor x86_64 and arm64 if the CI limit allows it. We can also try with other qemu targets. -
Enabling more test cases for
citesttargets
The tool automatically detects which tests can be run based on the provided ELF. We just have to enable the appropriate applications in defconfig. -
Migrate nuttx-ntfc and nuttx-testing repositories to Apache repositories.
Convert this repository into a test case collection for NTFC nuttx-testing#102 -
Official release of NTFC as Python package.
-
Add stable NTFC package to the Docker image.
-
Remove legacy
tools/ci/testrun. -
(SOMEDAY) Run test cases depending on GitHub labels. Running all tests in most cases makes no sense. With NTFC, we can ignore individual test modules and save CI resources.
-
(SOMEDAY) Network tests on CI. We can set up a virtual network (ETH, CAN) in which several nuttx instances communicate with each other and NTFC runs tests.