Trust, but verify.
—Ronald Reagan, citing a Russian proverb
Compliance refers to a strategy and a set of activities and artifacts that allow teams to apply Lean-Agile development methods to build systems that have the highest possible quality, while simultaneously assuring they meet any regulatory, industry, or other relevant standards.
Enterprises use SAFe to build some of the world’s largest and most important systems, many of which have unacceptable social or economic costs of failure. Examples of these high-assurance systems include medical devices, automobiles, avionics, banking and financial services, and aerospace and defense. To protect the public safety, these systems are often subject to extensive regulatory or customer oversight and rigorous compliance requirements. In addition, many enterprises are subject to other government regulations (e.g., Sarbanes-Oxley Act, Health Insurance Portability and Accountability Act [HIPAA], Affordable Care Act [ACA], state insurance regulations) that require similar attention and audits to ensure compliance.
Historically, organizations operating under such regulations have relied on comprehensive quality management systems (QMS). Based on phase-gated development models, they’re intended to reduce risk and ensure compliance. Unfortunately, these traditional approaches don’t scale, even when some teams follow Agile practices. Nor do they keep pace with accelerating time-to-market demands. Of greater concern is that even when the higher Cost of Delay (CoD) is accepted, these traditional approaches often do not increase quality or eliminate risk. As Deming notes, “Inspection is too late. The quality good, or bad, is already in the product.”
This chapter offers guidance on how to apply Lean-Agile methods to build these systems faster and better, while also addressing critical compliance requirements.
Traditional waterfall practices often mandate that full system specifications are defined and committed to in detail, up-front, long before all the real system behaviors can be known. Even worse, the sequential nature of phase-gated development produces large batches of work, long cycles between system integration points, and late feedback. In addition, compliance activities are typically deferred until the end of the project, providing little insight into compliance progress.
Collectively, these practices often result in missed deadlines, disappointing business or mission outcomes, lower quality, and substantial (and late) compliance challenges. In contrast, high-assurance Lean-Agile development builds in quality incrementally—early and throughout the development lifecycle. Moreover, it does so while including the very elements and activities necessary to ensure compliance.
To satisfy compliance requirements, organizations must demonstrate that their system meets its intended purpose and has no unintended consequences that might cause harm. They must also develop the objective evidence required to prove that the system conforms to those standards. To that end, organizations that build high-assurance systems define their approved practices, policies, and procedures in a QMS. These systems are intended to ensure that development activities and outcomes comply with all relevant regulations and provide the required documentation to prove it.
Unfortunately, the QMSs of many organizations are still heavily influenced by traditional phase-gated waterfall methods. This seriously inhibits, and can even prevent, the adoption of newer methods, as the older methods are hard-coded into the only approved way of working. As Figure 1 illustrates, SAFe describes an incremental approach to both development and compliance. It means those who want the benefits of Lean-Agile development (faster time-to-market and higher quality, to name two of these benefits) will typically have to evolve a Lean QMS.
Figure 1. A Lean-Agile quality management system improves quality and makes compliance more predictable
The remainder of this chapter provides guidance on Lean-Agile strategies and patterns that develop these high-assurance systems.
Even with a set of robust specifications, engineering teams never have all the answers when development begins. Instead, they have a set of hypotheses that must be tested through a series of short, iterative experiments, which provide validated learning to ideally advance toward the ultimate solution. Figure 2 highlights SAFe’s incremental development approach, comparing Shewhart/Deming’s plan–do–check–adjust (PDCA) learning cycles with a traditional waterfall model.
Figure 2 highlights two important implications for compliance. First, building smaller, working parts of the solution early allows compliance activities to also begin early, removing the large bow wave of performing such actions at the end. Each increment assesses the viability of the current solution, as well as progress toward compliance, providing early feedback on the system’s ultimate fitness for use. Second, specifications are created and evolved over time in small batches, with faster feedback on decisions and the opportunity for continuous review and assessment.
Agile Release Trains (ARTs) are the primary value-delivery organizations in SAFe. Each train requires all the skills necessary to build and release the Solution, including those responsible for quality assurance (QA), security, testing, and Verification and Validation (V&V). (While some regulations require independent, objective assurance, compliance representatives can still participate continuously as members of the ART.) The result is an ART designed in the manner illustrated in Figure 3.
Solution and Product Management ensure that the Solution Intent and backlog properly reflect compliance requirements. Teams also ensure that their work includes appropriate compliance activities.
Built-In Quality is one of SAFe’s four Core Values and a core tenet of the Lean-Agile Mindset. SAFe describes the use of built-in quality practices, including automation to detect compliance and quality problems and, when detected, stopping the entire system to focus everyone on resolving the problem. This philosophy applies systems thinking by ‘optimizing the whole,’ ensuring fast flow across the entire value stream and making quality everyone’s job. From this perspective, quality is a culture, not a job title.
To that end, compliance concerns are also built directly into the development process, and automated wherever possible, as illustrated in Figure 4.
Not all compliance activities can be automated, however, as some regulatory requirements mandate manual reviews, including activities like Failure Mode and Effects Analysis (FMEA) and audits. These activities are simply planned as part of the team backlog. The goal is to conduct these reviews as the solution is being built, reducing the last sign-off activity from a large, extended event to a quick and boring ‘non-event.’
With this approach, the program receives fast feedback on the degree to which the team’s compliance activities are being met and, conversely, how those activities may be impacting team performance. Figure 5 shows the feedback cycle between team activities and the practices defined by the Lean QMS.
Most high-assurance systems require V&V to ensure that two criteria are met:
The system works as designed (verification).
The system meets the needs of the user (validation).
V&V must always occur against a known set of requirements. Otherwise, there is nothing to verify and validate against. As Figure 6 illustrates, SAFe uses solution intent as the repository for existing and emerging requirements and designs.
Traceability from solution intent ensures that the artifacts produced during each Program Increment (PI)—the actual software, hardware components, and other outputs—address regulatory and compliance specifications, providing end-to-end evidence that V&V requirements have been met.
In SAFe, all elements of the requirements have test cases (see Figure 7), which are created at the same time as the functionality. Each increment yields new functionality and, consequently, adds new tests. As the number of tests grows, automation is vital to prevent testing activities from becoming bottlenecks.
By incrementally building the necessary artifacts in the solution content over a series of PIs, SAFe supports continuous verification. Figure 8 illustrates how this process works.
Verification activities are implemented as part of the flow of value delivery (e.g., backlog items or Definition of Done [DoD] as described earlier). While the artifacts will satisfy the objective evidence needed at the end of development, they are created iteratively throughout the life cycle. Validation occurs when Product Owners, Customers, and end users participate in ART planning, demos, and validation of fitness for purpose.
In the example in Figure 8, regulations require design reviews and specify that all actions be recorded and resolved. The ‘design review’ Enabler backlog item offers evidence of the review, and its DoD ensures that actions are recorded and resolved per the Lean QMS. If needed, the actions themselves could be tracked as enabler stories. Regulations may also require that all changes be reviewed, which is addressed by a compulsory peer review DoD for all stories.
SAFe recommends building the integrated solution frequently (or elements of the system for cyber-physical solutions), at least at every iteration for the System Demo. Building and integrating frequently allows continuous validation from User Acceptance Testing (UAT), customers, and end users. For each iteration, the system demo provides objective evidence that the integrations perform as intended and that the entire system has advanced forward while maintaining quality and compliance standards.
SAFe recognizes that although the product development process happens in a predictable cadence (Develop on Cadence), the release process (Release on Demand) may require additional activities:
Validation testing of the final release candidate (e.g., medical trial, flight test)
Review of the objective evidence required before production approval and release
Customer, regulatory, UAT, sign-offs, or other document submissions
Even then, Lean-thinking organizations always strive to fully automate delivery and, where possible, build in automated final release checks as part of a SAFe Continuous Delivery Pipeline and release on demand.
LEARN MORE
[1] Leffingwell, Dean. Agile Software Requirements. Pearson Education, 2011.
[2] “Achieving Regulatory and Industry Standards Compliance with SAFe” [White paper]. http://scaledagileframework.com/achieving-regulatory-and-industry-standards-compliance-with-safe/.
[3] “Achieving Regulatory and Industry Standards Compliance with SAFe” [Webinar]. https://www.youtube.com/watch?v=-7rVOWTHZEw&feature=youtu.be.
