Network Manager Test Suites
Summary
The NetworkManager project has recently recieved a bunch of new features and other features are under development. For Fedora, NetworkManager is the network configuration solution of choice. The goal of this feature is to prepare several test suites for NetworkManager and related projects and add them to the upstream project.
Owner
- Name: Pavel Šimerda
- Email: psimerda at redhat.com
- Name: Jiří Pírko
- Email: jpirko at redhat.com
- Name: Radek Pazdera
- Email: rpazdera at redhat.com
Current status
- Targeted release: Unknown
- Last updated: 2013-04-04
- Percentage of completion: 20%
Detailed description
There will be several test suites to fulfill specific goals. Together they will cover not only NetworkManager itself but also some of its dependencies.
We are submitting this Feature because:
- NetworkManager is one of the most important parts of the networking stack
- We believe that reliability is as important as the features themselves
- It depends on a Fedora project called LNST
- The test results uncover bugs in other packages including kernel and libnl
- We want to attract community members to contribute more tests
Underlying platform test suite
Routines to access the underlying platform is being be centralized into a single NetworkManager module called nm-platform. The purpose of this test suite is to check the nm-platform module, the underlying libnl library and the kernel. It requires root access to the machine where it is used.
Core behavior test suite
The nm-platform module offers a fake implementation of the underlying system behavior. This can be used to test the behavior of core NetworkManager code. The purpose of the resulting test suite is to carefully test the inner workings of NetworkManager. This test suite won't have any specific requirements and will run as 'make check' test even with an ordinary user account.
LNST-based virtual network test suite
The LNST project can be used to run larger scale integration tests for specific network configurations. The purpose of this test suite is to test the real-world use cases and configurations. It will be the only test suite that will go as far as testing the actual connectivity. It will require LNST testing framework and a virtualization solution. Alternatively, a network of physical hosts can be used.
Benefit to Fedora
Most Fedora installations are being used in networking environments with NetworkManager handling the configuration. Bugs and regressions always affect the usability and reputation of Fedora. We believe that the testsuites will help to avoid it as well as save developers' time spent on manual tests. Fedora will benefit from being able offer reliability and avoid surprising and buggy behaviour.
Scope
We want to include all the tests into NetworkManager's upstream git master. Even though NetworkManager is the main affected project here, various tests will affect various project NetworkManager relies on.
We expect the following projects to recieve feedback from the testsuites:
- Linux kernel
- libnl
- NetworkManager
- dhclient
- dnsmasq
(other project will be added during the work)
How to test
The scope of this feature is the testing itself. To test this feature, run all the testsuites. Some of them are suitable for testing specific hardware and software combinations, especially the “underlying platform test suite”.
User experience
By improving reliability of the network stack, the user experience will improve on all levels.
Dependencies
This feature requires a stable release of LNST suitable for NetworkManager testing. One of the feature owners (Radek) is working on LNST and will help us in this area.
Contingency plan
If test suites are not ready for the target release, NetworkManager will be used just as it was used before. Even partial completion of the feature helps.
Documentation
The test suites are mostly self-documenting but during the work we will try to improve both information on the Fedora Wiki and the upstream documentation.
Release notes
NetworkManager now comes with test suites that greatly improve its reliability and usability in various environments.