15 INDUSTRY APPROACHES TO TESTING

‘Software never was perfect and won’t get perfect. But is that a licence to create garbage? The missing ingredient is our reluctance to quantify quality.’

– Boris Beizer

In this chapter we examine modern trends in software testing.

AUTOMATED TOOLS

Testing is often a time-intensive process and developers need feedback quickly in order to remediate problems with their code. The rise of automated tools makes this process less painful. Of the many available, common automated tools include IBM Rational Functional Tester, Ranorex, Selenium and TestComplete.

It is beyond the scope of this book to describe all those available and it should be remembered that each tool typically supports multiple programming languages and application types (such as desktop, mobile and web). However, some are particularly geared to certain types of application – for example, Selenium is very often seen as the go-to automation tool used for commercial web applications.

TRENDS

There are many trends in the software development sector that affect a team’s approach to testing their code. Two in particular stand out: test-driven development (TDD) and secure (defensive) coding.

Test-driven development

This form of development practice is presently in vogue and often seen in software development job adverts as a desirable quality in a potential candidate. Rather than creating tests to check existing code, tests are written first (from use cases or user stories) and this shapes how code is then written.

Developers then write new code to pass these tests. Of course, the code may need further refactoring (see Chapter 7), but teams are assured that the test has been successful and therefore the component must have satisfactorily met the design requirements.

In TDD, each component is small and the development cycle is incredibly short, resulting in incremental – but reliable – development where less time is spent on reworking code and debugging.

Secure (defensive) coding

Due to the rise in popularity of network-aware and mobile applications, any modern testing strategy should place a considerable emphasis on security. Devices and their confidential data are obvious targets for cyberattacks, and poorly written applications are often the attack vector used by hackers to gain access.

Although commercial programming has been around for decades, it is only relatively recently that a concerted emphasis has been placed on application security (often abbreviated to ‘AppSec’ in the industry), primarily due to the software-related breaches which have plagued businesses across the world. Weak AppSec is one of the root causes of data breaches and may lead to reduced customer faith, very large fines, legal action and a level of reputational damage from which few organisations truly recover. AppSec is a feature of DevSecOps (see Chapter 14).

The goals of AppSec are to:

Of course, from a software perspective, these tasks are particularly complex because vulnerabilities can occur in several different application types:

Each of these has its own technology challenges and distinctive set of vulnerabilities. As a developer it is important to know about these weaknesses and code accordingly. Table 15.1 lists some common vulnerabilities and their mitigations.

Table 15.1 Common security issues for software developers

Issue type

Description

Mitigation (how to resolve it)

Buffer overflow

A buffer is a temporary memory store, typically connected to an input stream (keyboard) or output stream (screen). Many languages, e.g. C (most classically), do not provide bounds checking, which allows much larger data values (typically strings) to be input into smaller storage locations on the stack. When this occurs, the buffer overflows into adjacent memory locations, which can corrupt other data items, potentially to the malicious user’s advantage, e.g. by installing executable code.

Modern compilers often flag the use of vulnerable functions (i.e. library functions which are deemed insecure due to this potential for abuse). Typically, the functions are initially deprecated (usage is disapproved of) and then eventually may be physically removed from the library. This risks unsuccessful re-compilation of older code but forces the programmer to use more secure options instead. C’s removal of the gets() function is a good example of this.

Injection attacks

There are many different types of injection attack. One of the most common is the SQL injection attack, which occurs when malicious code is introduced into an executable SQL statement, typically through an unvalidated user input. This code then bypasses security, alters back-end databases and can be used to dump secure information to the user’s screen or escalate privileges. Despite being flagged for years by OWASP (Open Web Application Security Project) and being preventable in most modern code frameworks, it still plagues modern web applications; we can only assume this is because of poor coding techniques.

The most common mitigation technique is to use parameterised queries which ‘bind’ values into pre-prepared SQL statements. These bound values, if injected with malicious code, typically fail to execute as they are syntactically incorrect; a bound value can only contain a particular type of value such as an integer or Boolean, not a full expression. Other useful techniques include use of pattern checking, cleaning the inputted string and limiting the permissions on sensitive database operations to particular users.

Object-relational mapping techniques often offer suitable data-cleaning methods which can also remediate this potential weakness – although exploitation is still possible.

Insecure libraries

Many libraries have issues – i.e. vulnerabilities which can easily be exploited by malicious users. For example, the following once well-respected string-related functions in C are often targets of attack and should not be used: gets(), strcpy(), strcat() and strcmp().

Use other functions which are less vulnerable, such as strncpy() (or strlcpy() if available). Some automated deployment tools, such as Jenkins, have ‘dependency check’ plug-ins which can scan for the use of insecure library functions in submitted source code before it is committed to a remote repository.

File opening

A common technique, particularly in web applications, is file path (or directory) traversal. This occurs when a file’s pathname is modified to ‘escape’ a permitted directory on a server to target another vulnerable one, which may contain sensitive data, configuration files, source code and so on.

This is a simple one – never let the user specify the name of a data file to open! Either hardcode the string or allow the user to select from an available list of filenames, obfuscating the physical pathnames.

HOW SECURITY AFFECTS TESTING IN THE MODERN IT INDUSTRY

Historically, security has been relegated to the end of the production pipeline. At this point, whatever security issues exist in the code are likely to be ‘baked in’ – hard to isolate and consequently difficult to fix (see Figure 15.1).

Figure 15.1 Traditional approach to testing

images

A popular term in DevSecOps (see Chapter 14) that emerged around late 2016 is ‘shifting security to the left’ (see Figure 15.2). More accurately, nowadays we can say we are shifting it to the very start of the process, rather than leaving it to the end of the production pipeline (at deployment).

Figure 15.2 Current thinking around testing

images

The effect of shifting security to the left is that vulnerabilities are discovered as early as possible in the production pipeline. These identified issues are typically resolved faster, often involving fewer people, which in turn saves the organisation both time and money. This represents a good return on investment for the organisation; the millions that may have been lost due to preventable data breaches in a deployed application may be reduced to mere hundreds of pounds in earlier security testing when a code defect or vulnerability is flagged and fixed – definitely a worthwhile investment!

The impact of continuous integration and continuous delivery on testing

Agile workflows which incorporate continuous integration and continuous delivery by their very nature strongly encourage the shifting of security to a much earlier position in the production pipeline. Generally, any new additions to the codebase should successfully pass a batch of tests before deployment. The recommended order is usually defined as follows (see Chapter 10 for more about these test types):

  1. unit tests;
  2. integration tests;
  3. regression tests;
  4. user tests.

It should be noted that creating automated testing within a pipeline is not an easy task. It often requires a high level of expertise (and commercial experience) with testing tools and the integration of these by, for example, launching them and capturing their outcomes. Developers should be intrinsically involved in this process, realising that testing is a critical process that should ideally be performed upon every single commit of code to a version-controlled repository.

TIPS FOR EFFECTIVE TESTING

Testing requires practice but there are some quick wins:

Although acceptance tests that validate the software application against the identified business requirements require successful engagement with stakeholder end users who are familiar with the project’s goals, these tests should not represent the first substantive engagement with the customer. Indeed, far from it!

SUMMARY

In this chapter we examined modern trends in testing and how security affects testing in the modern IT industry.

In the next chapter we will examine why developers need to stay focused on the needs of the client and stakeholders throughout the entire development process.