Chapter 1

A. Benefits of Software Quality Engineering within the Organization

Describe the benefits that software quality engineering can have at the organizational level. (Understand)

BODY OF KNOWLEDGE I.A

Quality Defined

S ince this is a book about software quality engineering, it seems appropriate to start with a definition of quality. However, the industry has not, and may never, come to a single definition of the term quality. For example, the ISO/IEC/IEEE Systems and Software Engineering—Vocabulary (ISO/IEC/IEEE 2010) has the following set of definitions for quality:

1. The degree to which a system, component, or process meets specified requirements
2. Ability of a product, service, system, component, or process to meet customer or user needs, expectations, or requirements
3. The totality of characteristics of an entity that bears on its ability to satisfy stated and implied needs
4. Conformity to user expectations, conformity to user requirements, customer satisfaction, reliability, and level of defects present
5. The degree to which a set of inherent characteristics fulfills requirements

Based on his studies of how quality is perceived in various domains (for example, philosophy, economics, marketing, operations management), Garvin (1984) concluded, “Quality is a complex and multifaceted concept.” He describes quality from five different perspectives:

• Manufacturing perspective. This perspective matches how Crosby (1984) defines quality in terms of conformance to the specification (ISO/IEC/IEEE definition 1). Crosby’s point is that an organization does not want a variety of people, throughout the development of a product, making their own judgments about what the stakeholders need or want. A well-written specification is the cornerstone for creating a quality product. Even in agile development where requirements specifications are not formally documented, without good user stories and the subsequent conversations with the stakeholders to determine their requirements, it is difficult to write high quality software. For software, however, this perspective of quality is necessary but may not be sufficient since, according to Wiegers (2013), errors made during the requirements stage account for 40 percent to 50 percent of all defects found in a software project. From another viewpoint, this perspective refers to the ability to manufacture (replicate) a product to that specification over and over within accepted tolerances. Before an organization can adjust / improve its processes, that organization must let them run long enough to understand what is really being produced. Then the organization can design its specifications to reflect the real process capabilities and work to improve those capabilities by removing variance from those processes. While the primary focus of software quality is on design and development activities, this manufacturing and “variance removal” quality perspective reminds software organizations that the replication process or download process can not be completely ignored and must be verified as well.
• User perspective. Juran (1999) cites “fitness for use” as the appropriate measure for quality (ISO/IEC/IEEE definition 2). Software practitioners can probably all relate stories of software products that conformed to their specifications, but did not function adequately when deployed into operations. This perspective of quality considers both the viewpoints of the individual users and their context of use. For example, what a novice user might consider a “quality” user interface might drive a power user to distraction with pop-up help and warning messages that require responses. Also, what is a secure-enough interface for a software database used for personal information at home might be woefully inadequate in a business environment.
• Product perspective. Quality is tied to inherent characteristics of the product (ISO/IEC/IEEE definitions 3 and 5). These characteristics are the quality attributes, also called the “ilities” of the software product. Examples include reliability, usability, accessibility, availability, flexibility, maintainability, portability, installability, adaptability. Of course, they do not all end in “ility.” Correctness, fault tolerance, integrity, efficiency, security, and safety are also examples of quality attributes (see Chapter 11 for more information about quality attributes). The more the software has high levels of these characteristics, the higher its quality is considered to be. The ISO/IEC 25000 Software Engineering—Software Product Quality Requirements and Evaluation (SQuaRE) standards series (transition from the previous ISO/IEC 9126 and 14598 series of standards) provides a reference model and definitions for external and internal quality attributes and quality-in-use attributes. This standards series also provides guidance for specifying requirements, planning and managing, measuring, and evaluating quality attributes.
• Transcendental perspective. Quality is something that can be recognized but not defined (ISO/IEC/IEEE definition 4). As stated by Kan (2003), “to many people, quality is similar to what a federal judge once said about obscenity: ‘I know it when I see it.’” This perspective of quality takes the viewpoint of the individual into consideration. What is “obscenity” to one person may be “art” to another. What one stakeholder considers good software quality may not be high enough quality for another stakeholder. Tom Peters cites the customers’ (stakeholders’) reaction as the only appropriate measure for the quality of a product. This requires that product developers keep in touch with their stakeholders to make certain that the specifications accurately reflect those stakeholders’ real (and possibly changing) needs and that value is being built into the software for those stakeholders.
• Value-based perspective. Quality is dependent on the amount a customer is willing to pay for it. This perspective leads to considerations of “good enough” software quality. Would people be willing to pay twice as much for the word processor for their home computer if it was twice as reliable? Would people willing to pay an extra thousand dollars for a car if it meant more secure (less likely to be hacked) automated accident avoidance or self-parking systems? How much extra are people willing to pay for high-quality software in medical devices or high-quality software in airplane navigation systems? This value-added perspective can be expanded to include other stakeholders besides just customers. High quality software has a positive benefit to cost ratio for one or more stakeholders.

Benefits of Software Quality Engineering

Software quality engineering is the study and systematic application of scientific, technological, economic, social, and practical knowledge, and empirically proven methods, to the analysis and continuous improvement of all stages of the software life cycle to maximize the quality of software processes and practices, and the products they produce.

At its most basic, increasing the quality of the software typically means reducing the number of defects in that software and in the processes that are used to develop / maintain that software. Defects in the software can result from mistakes that occurred during the software development (either in house development or development by a third party), release, and maintenance processes, or from defects in the processes themselves. These mistakes introduce defects into the software work products. Mistakes can also lead to missing, ambiguous, or incorrect requirement defects that result in the development of software that does not match the needs of its stakeholders. The most cost-effective way of handling a defect is to prevent it. In this case, software quality is accomplished through continuous process improvement, increasing staff knowledge and skill, and through other defect prevention techniques that keep defects out of the software. Every defect that is prevented eliminates rework to correct that defect and also eliminates the effort associated with that rework.

If a defect does get interjected into the software, the more quickly the defect is identified and corrected, the less rework effort is typically required to correct it. Eliminating the waste of excessive rework allows organizations to use the saved effort to produce additional value-added software. In this case, software quality is accomplished through techniques that improve defect detection and find the defects earlier.

For example, as illustrated in Figure 1.1 , the cost to fix a defect will typically increase exponentially as it is found later and later in the life cycle. In fact, studies show that it can cost 100-plus times more to fix a requirements defect if it is not found until after the software is released into operations than it would have cost to fix that same defect if it was found in the requirements phase (Kan 2003; Pressman 2015). The main point here is that software development involves a series of dependencies, where each subsequent activity potentially builds on and expands the products of previous activities. For example, when using traditional software development methods, a single requirement could result in four design elements that expand into seven source code modules. All of these have supporting documentation and / or tests. Therefore, if a defect in that requirement is not found until the design phase, costs increase because the requirement would have to be fixed, and the four design elements would have to be investigated and potentially fixed. If the defect is not found until the coding phase, the requirement would have to be fixed, and the four design elements and seven source code modules would have to be investigated and potentially fixed. By the coding phase, test cases (system, integration, and unit) and user documentation may also have been written based on the defective requirement, which would need to be investigated and potentially corrected. If the defect is not found until the testing phases, all of the work products based on that requirement would have to be investigated and potentially corrected, retested, and regression tested.

Figure 1.1 Cost of fixing defects.

Agile methods attack these costs by significantly shortening the development cycle using shortened iterative/incremental development cycles, and through other techniques that shorten defect interjection to correction cycle times. At the end of each iteration, the goal is to have working software that can be demonstrated to the stakeholders, who provide feedback on the correctness and quality of the software.

Both studies and empirical evidence demonstrate that improving software quality benefits the development organization by:

• Reducing development and maintenance costs
• Decreasing process cycle times, which translate into shortened schedules and improvements in time-to-market
• Increasing the productivity/velocity of software practitioners
• Increasing the predictability of costs and schedules

Both defect prevention and defect detection help keep software defects from being delivered into operations. If fewer software defects are delivered, there is a higher probability of failure-free operations. Unlike hardware, software does not wear out with time. If a defect is not encountered during operations, the software performs reliably. Reliable software can increase the effectiveness and efficiency of work being done using that software. Reliable software reduces operational, failure, and maintenance costs to the software’s stakeholders and thus reduces the overall cost of ownership of the software product to its customers / users. Reliable software also reduces software corrective maintenance costs to the organization that developed the software.

Taking a broader view, high-quality software is software that has been specified correctly and that meets its specification. If the software meets the stakeholders’ needs and expectations and is value-added, it is more likely to be used instead of ending up as “shelfware.” If the customers and users receive software that has fewer defects, is more reliable, and performs to their needs and expectations, then those customers and users will be more satisfied with the software. This is illustrated in Figure 1.2 , which depicts Kano’s model of the relationship between stakeholder satisfaction and quality. Kano talks about three types of quality, including:

Figure 1.2 Kano Model (Pyzdek 2000).

• Basic quality: There is a basic level of quality that a stakeholder expects the product to have. This basic quality comes from satisfying the requirements that are assumed by the stakeholder to be part of the product and are typically not explicitly stated or requested during requirements elicitation activities. For example, people buying a new car expect that car to have four tires, a windshield, windshield wipers, and a steering wheel. They will not ask for these items when purchasing a new car; they just expect them to be there. This seems rather obvious for a car—people who build cars know what these basic requirements are and build them into their cars. But this is not necessarily true in software. A software developer may be writing telecommunications software one year and be writing aerospace software the next. The developer knows software, but without a comprehensive knowledge of the customer / user’s business domain, it is easy to miss requirements that are assumed to be part of the product. Most experienced software developers can relate stories where the delivered product met its requirements but not its intended use because one of these assumed requirements was missed. Our stakeholders then hear the “you never told me that was a requirement” refrain while we argue that it is an enhancement that they should be paying all the while they think we have just delivered a car without a windshield. This level of quality does not satisfy the stakeholders. (Note that the entire “basic quality” line is in the dissatisfaction region.) However, absence of quality at this level will quickly increase a stakeholder’s dissatisfaction.
• Expected quality : The “expected quality” line on the graph in Figure 1.2 represents satisfying those requirements that the stakeholder explicitly considers and requests. For example, a buyer will state preferences for the make, model, and options when shopping for a car. The stakeholder will be dissatisfied if this level of quality is not met. The stakeholder will be increasingly satisfied as this quality level increases when more and more of the stated requirements are satisfied by the software.
• Exciting quality: This quality level represents unrequested but innovative requirements. These are requirements that the stakeholders do not know they want, but will love when they see them. For example, when cup holders were first introduced in cars they were positively received. Note that the entire “exciting quality” line is in the satisfaction region. It should be remembered, however, that today’s innovations are tomorrow’s expectations. Now, in fact, most new car purchasers consider a cup holder to be part of the basic requirements for a car. However, care must always be taken to evaluate any innovative quality items to make sure that they are truly value-added to the stakeholders. This analysis should make certain that these innovations do not lead to “gold-plating,” and the overdoing functionality that will lead to the software costing too much; taking too long to market; impacting software performance, safety, security or other needed software attribute and so on.

Increased stakeholder satisfaction can lead to the following benefits for the software development organization:

• Increased market-share, and the ability to charge higher prices without affecting market share
• Increased profitability and/or return-on-investment
• Improvement to the organization’s reputation in the industry
• Increased customer trust, loyalty, and repeat business
• Increased ability to prosper (or at least survive) even in a bad economic times
• Decreased number of customer service request and audits

An increase in the quality of the software can also increase the satisfaction of the software practitioners. For most software engineers, their favorite activity is not burning the midnight oil trying to debug critical problems reported from operations. Producing high-quality products makes the software practitioner’s job easier and less frustrating. Practitioners can also take pride in what they are doing, thus enhancing their feelings of accomplishment and self-worth. Increases in employee satisfaction benefit the organization by increasing productivity/velocity and decreasing employee turnover.