Programmers, Coaches
We deliver on our iteration commitments.
Imagine that the power cable for your workstation is just barely long enough to reach the wall receptacle. You can plug it in if you stretch it taut, but the slightest vibration will cause the plug to pop out of the wall and the power to go off. You’ll lose everything you were working on.
I can’t afford to have my computer losing power at the slightest provocation. My work’s too important for that. In this situation, I would move the computer closer to the outlet so that it could handle some minor bumps. (Then I would tape the cord to the floor so people couldn’t trip over it, install an uninterruptable power supply, and invest in a continuous backup server.)
Your project plans are also too important to be disrupted by the slightest provocation. Like the power cord, they need slack.
The amount of slack you need doesn’t depend on the number of problems you face. It depends on the randomness of problems. If you always experience exactly 20 hours of problems in each iteration, your velocity will automatically compensate. However, if you experience between 20 and 30 hours of problems in each iteration, your velocity will bounce up and down. You need 10 hours of slack to stabilize your velocity and to ensure that you’ll meet your commitments.
These numbers are just for illustration. Instead of measuring the number of hours you spend on problems, take advantage of velocity’s feedback loop (see Estimating” later in this chapter for more about velocity). If your velocity bounces around a lot, stop signing up for more stories than your velocity allows. This will cause your velocity to settle at a lower number that incorporates enough slack for your team. On the other hand, if your velocity is rock solid, try reducing slack by committing to a small extra story next iteration.
One way to introduce slack into your iterations might be to schedule no work on the last day or two of your iteration. This would give you slack, but it would be pretty wasteful. A better approach is to schedule useful, important work that isn’t time-critical—work you can set aside in case of an emergency. Paying down technical debt fits the bill perfectly.
Only the constraint needs slack. The rest of the team organizes their work around the constraint’s schedule, resulting in slack for the entire team. (XP Concepts” in Chapter 3 discusses the Theory of Constraints in more detail.)
In this book, I’ve assumed that programmers are your team’s constraint. If that isn’t true for your team, you will need slack that is appropriate for your constraint. Talk to your mentor (see Find a Mentor” in Chapter 2) about how to modify this advice for your specific situation.
Even the best teams inadvertantly accumulate technical debt. Although you should always make your code as clean as you can, some technical debt will slip by unnoticed.
Rather than doing the bare minimum necessary to keep your head above water, be generous in refactoring and cleaning up technical debt in existing code. Every iteration, look for opportunities to make existing code better. Make this part of your everyday work. Every time I find myself scratching my head over a variable or method name, I change it. If I see some code that’s no longer in use, I delete it.
In addition to these small improvements, look for opportunities to make larger changes. Perhaps the code uses primitives rather than introducing a new type, or perhaps a class needs to have some of its responsibilities extracted into a new class.
I notice technical debt most when I navigate during pair programming. When a problem slows us down and I find myself feeling irritated, that inspires me to suggest that we fix it.
Paying down technical debt directly increases team productivity, so I spend a lot of time on it throughout the iteration. I usually spend about eight hours per week paying down technical debt, but other members of my teams spend only a few hours. A good rule of thumb is to spend 10 percent of the iteration on technical debt.
Perform big refactorings incrementally.
Don’t spend all your time on a single problem. Refactor throughout the iteration—an hour encapsulating a structure here, two hours fixing class responsibilities there. Each refactoring should address a specific, relatively small problem. Sometimes you’ll fix only part of a larger problem—that’s OK as long as it makes the code better. If your team pays down technical debt every week, you’ll have the opportunity to see and fix remaining problems in the future.
As you fix technical debt, focus on fixes that make your current work easier. Don’t go looking for technical debt that’s unrelated to stories you’re currently working on. If you consistently relate your improvements to your current work, you’ll automatically put the most effort into the most frequently modified and most valuable parts of the system.
To keep up with their constantly expanding field, programmers must continually improve their skills. In doing so, they will often learn things that enhance their work on the project.
Dedicated research time is an excellent way to encourage learning and add additional slack into your iterations. To introduce it, set aside half a day for each programmer to conduct self-directed research on a topic of his choice. Be completely hands-off. I recommend only two rules: don’t spend this time on project stories or tasks, and don’t modify any project code.
If you’re concerned about people goofing off, provide lunch the next day and ask that people share what they’ve done in informal peer discussion. This is a good way to share knowledge anyway.
I’ve introduced this technique to several teams, and it’s paid dividends each time. Two weeks after introducing research time at one organization, the product manager told me that research time was the most valuable time the team spent, and suggested that we double it.
Research time works because programmers are typically motivated by a desire to do good work, particularly when they’re self-directed. Most programmers have a natural desire to make their lives easier and to impress their colleagues. As a result, the work done in research time often has a surprisingly high return for the project.
Research time is particularly valuable for XP teams. The continuous drumbeat of iteration deadlines is great for reducing risk and motivating the team to excel, but it can lead to tunnel vision. Dedicated research time gives programmers a chance to widen their ranges, which often leads to insights about how to develop more effectively.
I schedule research time for the morning of the penultimate day of the iteration. This is late enough in the iteration that we can use the time as slack if we need to, but not so late to distract programmers with the upcoming deadline. Mornings are better than afternoons because it’s harder to start on time when production code is occupying your attention.
For research time to be effective, you must focus. Half a day can go by very quickly. It’s easy to think of research time as a catch-all for postponed meetings. Be strict about avoiding interruptions and enlist the help of the project manager. Ignore your email, turn off your IM, and use your web browser only for specific research.
When you first adopt research time, you might have trouble deciding what to work on. Think about what has puzzled you recently. Would you like to learn more about the details of your UI framework? Is there a programming language you’ve wanted to try but your organization doesn’t use? Has real-time networking always fascinated you?
As you do your research, create spike solutions—small standalone programs—that demonstrate what you’ve learned. Avoid trying to make software that’s generally useful; that will reduce the amount of time available to pursue core ideas. If something turns out to deserve further work, create and schedule a story for it.
Research time and paying down technical debt are important tasks that enhance your programmers’ skills and allow them to deliver more quickly. Paying down technical debt, in particular, should be part of every iteration. However, if your iteration commitments are at risk, it’s OK to set these two tasks aside temporarily in order to meet those commitments.
Before starting a big refactoring to pay down technical debt, consider the iteration plan and think about the kinds of problems the team has encountered. If the iteration is going smoothly, go ahead and refactor. If you’ve encountered problems or you’re a little behind schedule, shrug your shoulders and work on meeting your iteration commitment instead. You’ll have another opportunity to fix the problem later. By varying the amount of time you spend paying down technical debt, you can ensure that most iterations come in exactly on time.
Continue refactoring new code as you write it. It’s OK to defer cleaning up existing technical debt temporarily, but incurring new technical debt will hurt your productivity.
Every iteration experiences a few bumps. Varying the time you spend paying down technical debt is your first line of defense. Rarely, though, a problem sneaks past. In this case, if family commitments permit, I voluntarily work a few extra hours. Overtime is unusual in the companies I work with, so an extra hour a day once in a while isn’t a big deal. Another programmer or two will often volunteer to do the same.
Some problems will be too significant to clean up in overtime. In this case, consider cancelling research time in order to address the problem. First, though, take a second look at whether you need to replan the iteration instead (see Iteration Planning” earlier in this chapter). When problems are big, even cancelling research time may not be enough.
Slack is a wonderful tool. It helps you meet your commitments and gives you time to perform important, nonurgent tasks that improve your productivity.
Be careful, though. Although slack is a wonderful way to handle transitory problems, it can also disguise systemic problems. If you rely on slack to finish your stories in every iteration, that time isn’t really slack—it’s required.
To return to the power cord example, suppose that your workstation is on a jiggly desk. Every time you type, the desk jiggles a bit and the power cord pops out. You could add just enough slack to the cord to handle your typing, but that wouldn’t really be slack; it’s necessary to use the computer at all. If anything else happens while you’re typing—say, Bob from marketing gives you a friendly slap on the back—the cord will still pop out.
If you consistently use most of your slack, you’ve overcommitted.
If you work overtime, cancel research time, or don’t pay down any technical debt for two or three iterations in a row, you’ve overcommitted and have no slack. Congratulate yourself for delivering on your commitments anyway. Now add slack by reducing your velocity.
Making enough time for these nonurgent tasks is difficult. With a deadline rushing head-first toward you, it’s difficult to imagine spending any time that doesn’t directly contribute to getting stories out the door. New XP teams especially struggle with this. Don’t give up. Slack is essential to meeting commitments, and that is the key to successful XP.
In my experience, there are two big sources of randomness on XP teams: customer unavailability and technical debt. Both lead to an unpredictable environment, make estimating difficult, and require you to have more slack in order to meet your commitments.
If programmers have to wait for customer answers as they work, you can reduce the need for slack by making customers more available to answer programmer questions. They may not like that—customers are often surprised by the amount of time XP needs from them—but if you explain that it will help improve velocity, they may be more interested in helping.
On the other hand, if programmers often encounter unexpected technical delays, such as surprising design problems, difficulty integrating, or unavailability of a key staging environment, then your need for slack is due to too much technical debt. Fortunately, using your slack to pay down technical debt will automatically reduce the amount of slack you need in the future.
If our commitment is at risk, shouldn’t we stop pair programming or using test-driven development? Meeting our commitment is most important, right?
First, pair programming and test-driven development (TDD) should allow you to deliver more quickly, not more slowly. However, they do have a learning curve—particularly TDD—so it’s true that avoiding TDD might allow you to meet your commitments in the early stages of your project.
However, you shouldn’t use them as slack. Pair programming, test-driven development, and similar practices maintain your capability to deliver high-quality code. If you don’t do them, you will immediately incur technical debt and hurt your productivity. You may meet this iteration’s commitments, but you’ll do so at the expense of the next iteration. If your existing slack options aren’t enough, you need to replan your iteration, as discussed inIteration Planning” earlier in this chapter.
In contrast, paying down technical debt, going home on time, and conducting research enhance your capability to deliver. Using them as slack once in a while won’t hurt.
Research time sounds like professional development. Shouldn’t programmers do that on their own time?
In my experience, research time pays dividends within a few months. It’s worthwhile even without an explicit policy of encouraging professional development.
However, if research time isn’t appropriate for your organization, you can increase the amount of time you spend paying down technical debt instead.
Should we pair program during research time?
You can if you want, but you don’t have to. Treat it as you would any other spike.
Shouldn’t a project have just one buffer at the end of the project rather than a lot of little buffers?
The book Critical Chain [Goldratt 1997] argues for creating a single buffer at the end of a project rather than padding each estimate. It’s good advice, and adding slack to each iteration might seem to contradict that.
However, an XP team makes a commitment to deliver every week. In a sense, an XP team conducts a series of one-week projects, each with a commitment to stakeholders and its own small buffer protecting that commitment. This commitment is necessary for stakeholder trust, to take advantage of feedback, and to overcome the inherently high risk of software projects. Without it, the project would drift off course.
XP avoids Parkinson’s Law (“work expands to fill the time available”) and Student Syndrome (“work is delayed until its deadline”) by performing important but nonurgent tasks in the buffer.
When you incorporate slack into your iterations, you consistently meet your iteration commitments. You rarely need overtime. In addition, by spending so much time paying down technical debt, your code steadily improves, increasing your productivity and making further enhancements easier.
The danger of thinking of these tasks as slack is thinking they aren’t important. They’re actually vital, and a team that doesn’t perform these tasks will slow down over time. They’re just not time-critical like your iteration commitment is. Don’t use slack as an excuse to set aside these tasks indefinitely.
In addition, never incur technical debt in the name of slack. If you can’t meet your iteration commitments while performing standard XP tasks, replan the iteration instead. Practices you should never use as slack include test-driven development, refactoring new code, pair programming, and making sure stories are “done done.”
Finally, don’t use try to use iteration slack to meet release commitments. There isn’t enough iteration slack to make a meaningful difference to your release commitments—in fact, removing slack from your iterations could easily add so much chaos to your process that productivity actually goes down. Use risk management to provide slack for your release commitments.
Slack allows you to be “done done” and meet your iteration commitments. It enables a consistent velocity. It provides an opportunity for extra refactoring. It reduces technical debt and increases team capability. I’m not aware of any alternatives that provide all these benefits.
Rather than including slack at the iteration level, some teams add a safety buffer to every estimate. As [Goldratt 1997] explains, this leads to delays and often doesn’t improve the team’s ability to meet their commitments.
Many organizations, however, choose not to include any slack in their plans. It’s no coincidence that these organizations have trouble meeting commitments. They also waste a lot of time thrashing around as they attempt to achieve the unachievable.
Some teams form a reading group rather than conducting research time. This is an excellent alternative to research time, particularly for teams that are interested in discussing fundamental ideas rather than exploring new technologies.
One way to do so is to take turns reading sections of a book or an article out loud and then discussing them. This approach has the advantage of not requiring advance preparation, which promotes more active participation. One person should act as facilitator and inspire discussion by asking questions about each section.
A reading group can easily spend half a day per week in productive conversation. To take this book as an example, you could probably discuss one or two practice sections per session.
Other teams schedule silver stories for slack. These are less-important stories that the team can set aside if they need extra time. I prefer to pay down technical debt instead because teams often neglect this crucial task.
The difficulty with silver stories is that you need to set aside slack tasks at a moment’s notice. If you do that to a silver story, you’ll leave behind technical debt in the form of half-done code. If you use silver stories, be sure to delete all code related to the story if you have to put it aside.
I’ve also observed that silver stories hurt programmer morale. If the team needs to use some slack, they don’t finish all the stories on the board. Even though the silver stories were bonus stories, it doesn’t feel like that in practice. In contrast, any time spent paying down technical debt directly improves programmers’ quality of life. That’s a win.
Slack: Getting Past Burnout, Busywork, and the Myth of Total Efficiency [DeMarco 2002] provides a compelling case for providing slack throughout the organization.
The Goal and Critical Chain [Goldratt 1992] and [Goldratt 1997] are two business novels that make the case for using slack (or “buffers”), instead of padding estimates, to protect commitments and increase throughput.
“Silicon Valley Patterns Study of Domain-Driven Design” (http://domaindrivendesign.org/discussion/siliconvalleypatterns/index.html) is an interesting transcript of some meetings of a long-running and highly successful reading group.