“Welcome to the team, Pat,” said Kim, smiling at the recent graduate. “Let me show you around. As I said during the interview, we’re an XP shop. You may find that things are a little different here than you learned in school.”
“I’m eager to get started,” said Pat. “I took a software engineering course in school, and they taught us about the software development lifecycle. That made a lot of sense. There was a bit about XP, but it sounded like it was mostly about working in pairs and writing tests first. Is that right?”
“Not exactly,” said Kim. “We do use pair programming, and we do write tests first, but there’s much more to XP than that. Why don’t you ask me some questions? I’ll explain how XP is different than what you learned.”
Pat thought for a moment. “Well, one thing I know from my course is that all development methods use the software development lifecycle: analysis, design, coding, and testing [see Figure 3-1]. Which phase are you in right now? Analysis? Design? Or is it coding or testing?”
“Yes!” Kim grinned. She couldn’t help a bit of showmanship.
“I don’t understand. Which is it?”
“All of them. We’re working on analysis, design, coding, and testing. Simultaneously. Oh, and we deploy the software every week, too.”
Pat looked confused. Was she pulling his leg?
Kim laughed. “You’ll see! Let me show you around.
“This is our team room. As you can see, we all sit together in one big workspace. This helps us collaborate more effectively.”
Kim led Pat over to a big whiteboard where a man stood frowning at dozens of index cards. “Brian, I’d like you to meet Pat, our new programmer. Brian is our product manager. What are you working on right now?”
“I’m trying to figure out how we should modify our release plan based on the feedback from the user meeting last week. Our users love what we’ve done so far, but they also have some really good suggestions. I’m trying to decide if their suggestions are worth postponing the feature we were planning to start next week. Our success has made us visible throughout the company, so requests are starting to flood in. I need to figure out how to keep us on track without alienating too many people.”
“Sounds tough,” Kim said. “So, would you say that you’re working on requirements, then?”
“I’m working on making our stakeholders happy,” Brian shrugged, turning back to the whiteboard.
“Don’t mind him,” Kim whispered to Pat as they walked away. “He’s under a lot of pressure right now. This whole project was his idea. It’s already saved the company two and a half million dollars, but now there’s some political stuff going on. Luckily, we programmers don’t have to worry about that. Brian and Rachel take care of it—Rachel’s our project manager.”
“Wait... I thought Brian was the project manager?”
“No, Brian is the product manager. He’s in charge of deciding what we build, with the help of stakeholders and other team members, of course. Rachel is the project manager—she helps things run smoothly. She helps management understand what we’re doing and gets us what we need, like when she convinced Facilities to tear down the cubicle walls and give us this nice open workspace.
“Let me introduce you to some more members of the team,” Kim continued, leading Pat over to two people sitting at a workstation. “This is Mary and Jeff. Mary is a mechanical engineer. She normally works in manufacturing, but we asked her to join us as an on-site customer for this project so she can help us understand the issues they face on the floor. Jeff is one of our testers. He’s particularly good at finding holes in requirements. Guys, this is Pat, our new programmer.”
Pat nodded hello. “I think I recognize what you’re doing. That looks like a requirements document.”
“Sort of,” Jeff replied. “These are our customer tests for this iteration. They help us know if the software’s doing what it’s supposed to.”
“Customer tests?” Pat asked.
Mary spoke up. “They’re really examples. This particular set focuses on placement of inventory in the warehouse. We want the most frequently used inventory to be the easiest to access, but there are other concerns as well. We’re putting in examples of different selections of inventory and how they should be stored.”
“You can see how things are progressing,” Jeff continued. “Here, I’ll test these examples.” He pressed a button on the keyboard, and a copy of the document popped up on the screen. Some sections of the document were green. Others were red.
“You can see that the early examples are green—that means the programmers have those working. These later ones are red because they cover special cases that the programmers haven’t coded yet. And this one here is brand-new. Mary realized there were some edge cases we hadn’t properly considered. You can see that some of these cases are actually OK—they’re green—but some of them need more work. We’re about to tell the programmers about them.”
“Actually, you can go ahead and do that, Jeff,” said Mary, as they heard a muffled curse from the area of the whiteboard. “It sounds like Brian could use my help with the release plan. Nice to meet you, Pat.”
“Come on,” said Jeff. “Kim and I will introduce you to the other programmers.”
“Sure,” said Pat. “But first—this document you were working on. Is it a requirements document or a test document?”
“Both,” Jeff said, with a twinkle in his eye. “And neither. It’s a way to make sure that we get the hard stuff right. Does it really matter what we call it?”
“You seem pretty casual about this,” Pat said. “I did an internship last year and nobody at that company could even think about coding until the requirements and design plans were signed off. And here you are, adding features and revising your requirements right in the middle of coding!”
“It’s just crazy enough to work,” said Jeff.
“In other words,” Kim added, “we used to have formal process gates and signoffs, too. We used to spend days arguing in meetings about the smallest details in our documents. Now, we focus on doing the right things right, not on what documents we’ve signed off. It takes a lot less work. Because we work on everything together, from requirements to delivery, we make fewer mistakes and can figure out problems much more easily.”
“Things were different for me,” Jeff added. “I haven’t been here as long as Kim. In my last company, we didn’t have any structure at all. People just did what they felt was right. That worked OK when we were starting out, but after a few years we started having terrible problems with quality. We were always under the gun to meet deadlines, and we were constantly running into surprises that prevented us from releasing on time. Here, although we’re still able to do what we think is right, there’s enough structure for everyone to understand what’s going on and make constant progress.”
“It’s made our life easier,” Kim said enthusiastically. “We get a lot more done...”
“...and it’s higher quality,” Jeff finished. “You’ve got to watch out for Kim—she’ll never stop raving about how great it is to work together.” He grinned. “She’s right, you know. It is. Now let’s go tell the other programmers about the new examples Mary and I added.”
One of the most astonishing premises of XP is that you can eliminate requirements, design, and testing phases as well as the formal documents that go with them.
This premise is so far off from the way we typically learn to develop software that many people dismiss it as a delusional fantasy. “These XP folks obviously don’t know what they’re talking about,” they say. “Just last month I was on a project that failed due to inadequate requirements and design. We need more requirements, design, and testing, not less!”
That’s true. Software projects do need more requirements, design, and testing—which is why XP teams work on these activities every day. Yes, every day.
You see, XP emphasizes face-to-face collaboration. This is so effective in eliminating communication delays and misunderstandings that the team no longer needs distinct phases. This allows them to work on all activities every day—with simultaneous phases—as shown in Figure 3-2.
Using simultanous phases, an XP team produces deployable software every week. In each iteration, the team analyzes, designs, codes, tests, and deploys a subset of features.
Although this approach doesn’t necessarily mean that the team is more productive,[5] it does mean that the team gets feedback much more frequently. As a result, the team can easily connect successes and failures to their underlying causes. The amount of unproven work is very small, which allows the team to correct some mistakes on the fly, as when coding reveals a design flaw, or when a customer review reveals that a user interface layout is confusing or ugly.
The tight feedback loop also allows XP teams to refine their plans quickly. It’s much easier for a customer to refine a feature idea if she can request it and start to explore a working prototype within a few days. The same principle applies for tests, design, and team policy. Any information you learn in one phase can change the way you think about the rest of the software. If you find a design defect during coding or testing, you can use that knowledge as you continue to analyze requirements and design the system in subsequent iterations.
XP teams perform nearly every software development activity simultaneously. Analysis, design, coding, testing, and even deployment occur with rapid frequency.
That’s a lot to do simultaneously. XP does it by working in iterations: week-long increments of work. Every week, the team does a bit of release planning, a bit of design, a bit of coding, a bit of testing, and so forth. They work on stories: very small features, or parts of features, that have customer value. Every week, the team commits to delivering four to ten stories. Throughout the week, they work on all phases of development for each story. At the end of the week, they deploy their software for internal review. (In some cases, they deploy it to actual customers.)
The following sections show how traditional phase-based activities correspond to an XP iteration.
Every XP team includes several business experts—the on-site customers—who are responsible for making business decisions. The on-site customers point the project in the right direction by clarifying the project vision, creating stories, constructing a release plan, and managing risks. Programmers provide estimates and suggestions, which are blended with customer priorities in a process called the planning game. Together, the team strives to create small, frequent releases that maximize value.
The planning effort is most intense during the first few weeks of the project. During the remainder of the project, customers continue to review and improve the vision and the release plan to account for new opportunities and unexpected events.
In addition to the overall release plan, the team creates a detailed plan for the upcoming week at the beginning of each iteration. The team touches base every day in a brief stand-up meeting, and its informative workspace keeps everyone informed about the project status.
Rather than using an upfront analysis phase to define requirements, on-site customers sit with the team full-time. On-site customers may or may not be real customers depending on the type of project, but they are the people best qualified to determine what the software should do.
On-site customers are responsible for figuring out the requirements for the software. To do so, they use their own knowledge as customers combined with traditional requirements-gathering techniques. When programmers need information, they simply ask. Customers are responsible for organizing their work so they are ready when programmers ask for information. They figure out the general requirements for a story before the programmers estimate it and the detailed requirements before the programmers implement it.
Some requirements are tricky or difficult to understand. Customers formalize these requirements, with the assistance of testers, by creating customer tests: detailed, automatically checked examples. Customers and testers create the customer tests for a story around the same time that programmers implement the story. To assist in communication, programmers use a ubiquitous language in their design and code.
The user interface (UI) look and feel doesn’t benefit from automated customer tests. For the UI, customers work with the team to create sketches of the application screens. In some cases, customers work alongside programmers as they use a UI builder to create a screen. Some teams include an interaction designer who’s responsible for the application’s UI.
XP uses incremental design and architecture to continuously create and improve the design in small steps. This work is driven by test-driven development (TDD), an activity that inextricably weaves together testing, coding, design, and architecture. To support this process, programmers work in pairs, which increases the amount of brainpower brought to bear on each task and ensures that one person in each pair always has time to think about larger design issues.
Programmers are also responsible for managing their development environment. They use a version control system for configuration management and maintain their own automated build. Programmers integrate their code every few hours and ensure that every integration is technically capable of deployment.
To support this effort, programmers also maintain coding standards and share ownership of the code. The team shares a joint aesthetic for the code, and everyone is expected to fix problems in the code regardless of who wrote it.
XP includes a sophisticated suite of testing practices. Each member of the team—programmers, customers, and testers—makes his own contribution to software quality. Well-functioning XP teams produce only a handful of bugs per month in completed work.
Programmers provide the first line of defense with test-driven development. TDD produces automated unit and integration tests. In some cases, programmers may also create end-to-end tests. These tests help ensure that the software does what the programmers intended.
Likewise, customer tests help ensure that the programmers’ intent matches customers’ expectations. Customers review work in progress to ensure that the UI works the way they expect. They also produce examples for programmers to automate that provide examples of tricky business rules.
Finally, testers help the team understand whether their efforts are in fact producing high-quality code. They use exploratory testing to look for surprises and gaps in the software. When the testers find a bug, the team conducts root-cause analysis and considers how to improve their process to prevent similar bugs from occuring in the future. Testers also explore the software’s nonfunctional characteristics, such as performance and stability. Customers then use this information to decide whether to create additional stories.
The team doesn’t perform any manual regression testing. TDD and customer testing leads to a sophisticated suite of automated regression tests. When bugs are found, programmers create automated tests to show that the bugs have been resolved. This suite is sufficient to prevent regressions. Every time programmers integrate (once every few hours), they run the entire suite of regression tests to check if anything has broken.
The team also supports their quality efforts through pair programming, energized work, and iteration slack. These practices enhance the brainpower that each team member has available for creating high-quality software.
XP teams keep their software ready to deploy at the end of any iteration. They deploy the software to internal stakeholders every week in preparation for the weekly iteration demo. Deployment to real customers is scheduled according to business needs.
As long as the team is active, it maintains the software it has released. Depending on the organization, the team may also support its own software (a batman is helpful in this case; see Iteration Planning” in Chapter 8). In other cases, a separate support team may take over. Similarly, when the project ends, the team may hand off maintenance duties to another team. In this case, the team creates documentation and provides training as necessary during its last few weeks.
“Hey, guys, I’d like you to meet Pat, our new programmer,” Kim announced. She, Jeff, and Pat had walked over to a series of large tables. Index cards and pencils were scattered around the tables, and whiteboards covered the walls. Six programmers were working in pairs at three of the workstations.
“Hi, Pat,” the team chorused. Kim introduced everyone.
“Welcome aboard,” said Justin, one of the programmers. He stood up and grabbed a rubber chicken from a nearby desk. “Kevin and I are going to integrate our changes now,” he announced to the rest of the group. A few people nodded abstractly, already intent on their work.
“Mary and I just made some changes to the customer tests,” said Jeff. “Who’s working on the warehouse story?”
“That’s us,” said Justin. “What’s up?”
“We added some examples to cover some new edge cases. I think they’re pretty straightforward, but if you have any questions, let us know.”
“Will do,” Justin replied. “We were about to finish off the last of the business logic for that story anyway. We’ll take a look at it as soon as we’ve checked in.”
“Thanks!” Jeff went off to meet Mary and Brian at the planning board.
“Before you start on your next task, Justin, do you have a few minutes?” Kim asked.
Justin glanced at his pairing partner, Kevin, who was listening in. He gave Justin a thumbs up.
“Sure. We’re just waiting for the build’s tests to finish, anyway.”
“Great,” Kim said. “I’m helping Pat get oriented, and he wanted to know which phase of development we’re working on. Can you explain what you’ve been doing?”
Justin flashed Pat a knowing look. “She’s on the ’simultaneous phases’ kick, huh? I’m sorry.” He laughed. “Just kidding, Kim! It is pretty different.
“I’m sure Kim is dying to hear me say this: yes, we’re doing testing, design, coding, and integration all at once. We deploy every week—because we do internal development, we actually deploy to production—but deployment happens automatically. The real deployment work happens when we update our deployment scripts, which we do as needed throughout the week.”
“Right now Kevin and I are integrating, which is something everybody does every few hours. Before that, we were using test-driven development and refactoring to test, code, and design simultaneously.” Justin pointed to another pair at a workstation. “Jerry and Suri are doing a in-depth review of a story we finished yesterday. Suri is our other tester—she’s very good at exploratory testing.” He looked at the third pair, who were talking intensely and sketching on the whiteboard. “And it looks like Mark and Allison over there are working on a larger design problem. We use incremental design and architecture, so we’re constantly looking for ways to improve our design.”
Allison looked up. “Actually, I think this is something the whole group should see. Can we get everyone’s attention for a moment?”
“Sure,” said Jerry, pushing away from his desk. He glanced at his partner, who nodded. “We needed a break anyway.”
As Allison started sketching on the whiteboard, Pat thought about what he’d seen. Brian was working on planning. Jeff and Mary were working on requirements and tests—no, examples. Kevin and Justin were integrating, and had been testing, designing, and coding a few minutes earlier. Mark and Allison were designing. And Jerry and Suri were doing more testing.
Except that it wasn’t quite that simple. Everything seemed fluid; people kept changing what they were working on. Jeff and Mary had gone backward, from requirements to planning, and all the programmers had jumped from integration and testing to design. He frowned. How was he going to keep track of it all?
“Don’t worry,” said Kim quietly. She had noticed his discomfort. “It’s not as confusing as it seems. Jeff has it right: don’t worry about what to call what we’re doing. We’re all doing the same thing—getting stuff done. We work together to make it happen, and we jump back and forth between all the normal development activities as the situation dictates. If you focus on what we’re delivering rather than trying to figure out what phase of development we’re in, it will all make sense.”
Pat shook his head. “How do you know what you have to do, though? It looks like chaos.”
Kim pointed at a whiteboard filled with index cards. “There’s our plan for this week. Green are finished and white are remaining. It’s Monday morning and we release on Wednesday, so we have about half a week to go. You tell me; what’s our progress?”
Pat looked at the board. About half the cards had a green circle around them. “About halfway through?” he guessed.
Kim beamed. “Perfect! We keep those cards up-to-date. Just keep that in mind for the first week or two and you’ll do fine!”
[5] Productivity is notoriously difficult to study. I’m not aware of any formal research on XP productivity, although anecdotal evidence indicates that agile teams are more productive than traditional teams.