Non-functional tests are complementary to the preceding test types since they focus on other important aspects of the system:
- Performance tests: Performance tests ensure that the system works with the intended performance. Performance is sometimes a critical requirement (for example, for games, trading systems, and so on). Performance tests measure the speed at which the system performs some operations.
- Load/stress tests: Load tests, also called stress tests, are related to performance tests; they verify that the system can withstand the pressure when the load or stress on the system increases. For example, an application might work perfectly with a single user but become horrendously slow as the number of concurrent users increases. Again, this might be okay for some systems but problematic for other ones. Load tests perform many operations within a limited time frame and check specific aspects such as CPU usage, memory consumption, and so on.
- Security/vulnerability tests: Security tests verify that your application is not susceptible to specific vulnerabilities. Usually, you'll develop such tests when you fix a vulnerability in your system in order to ensure that it does not resurface. In that sense, security tests are also regression tests.
- Penetration tests: Penetration tests are closely related to security tests. They are performed by security specialists that try to find all the vulnerabilities that they can in your system. Penetration testers are white hat hackers, meaning that they have good intentions; they're trying to help you improve the security of your application and not to take advantage of the vulnerabilities. When vulnerabilities are discovered during penetration tests, they should be fixed and security/regression tests should be added to the application's test suite.
- Compatibility tests: Compatibility tests are needed to ensure that your application is compatible with whatever it is supposed to be, whether that is a specific runtime environment, operating system, and so on. Those tests are executed under different conditions and should verify compliance.
- Usability tests: Usability tests are checks performed by user interface specialists that aim to ensure that your application is usable, user-friendly, and efficient. This is a very specific category of tests that is much harder to automate as there is a lot of room for interpretation.
Again, there are many books covering this subject alone. In this book, given our space constraints, we'll focus only on unit tests and end-to-end tests.