The unittest module was introduced in Python 2.1 and has been massively used by developers since then. But some alternative test frameworks were created in the community by people who were frustrated by the weaknesses and limitations of unittest.
The following are the common criticisms that are often made:
- You have to prefix the method names with test.
- You are encouraged to use assertion methods provided in TestCase instead of plain assert statements as existing methods may not cover every use case.
- The framework is hard to extend because it requires massive subclassing of classes or tricks such as decorators.
- Test fixtures are sometimes hard to organize because the setUp and tearDown facilities are tied to the TestCase level, though they run once per test. In other words, if a test fixture concerns many test modules, it is not simple to organize its creation and cleanup.
- It is not convenient to run a test campaign. The default test runner (python -m unittest) indeed provides some test discovery but does not provide enough filtering capabilities. In practice, extra scripts have to be written to collect the tests, aggregate them, and then run them in a convenient way.
A lighter approach is needed to write tests without suffering from the rigidity of a framework that looks too much like its big Java brother, JUnit. Since Python does not require working with a 100% class-based environment, it is preferable to provide a more Pythonic test framework that is not primarily based on subclassing.
A slightly better framework would include the following:
- Provide a simple way to mark any function or any class as a test
- Be extendable through a plugin system
- Provide a complete test fixture environment for all test levels: at whole campaign, at module level, and at single test level
- Provide a test runner based on test discovery with an extensive set of options