Figure 2. Recipe for corn muffins. But how long do you cook it?
In the 1970s, the brothers Dreyfus (Hubert and Stuart) began doing their seminal research on how people attain and master skills.
Dreyfus is applicable per skill.
The Dreyfus brothers looked at highly skilled practitioners, including commercial airline pilots and world-renowned chess masters.[6] Their research showed that quite a bit changes as you move from novice to expert. You don’t just “know more” or gain skill. Instead, you experience fundamental differences in how you perceive the world, how you approach problem solving, and the mental models you form and use. How you go about acquiring new skills changes. External factors that help your performance—or hinder it—change as well.
Unlike other models or assessments that rate the whole person, the Dreyfus model is applicable per skill. In other words, it’s a situational model and not a trait or talent model.
You are neither “expert” nor “novice” at all things; rather, you are at one of these stages in some particular skill domain. You might be a novice cook but an expert sky diver, or vice versa. Most nondisabled adults are experts at walking—we do so without planning or thinking. It has become instinct. Most of us are novices at tax preparation. We can get through it given a sufficient number of clear rules to follow, but we really don’t know what’s going on (and wonder why on Earth those rules are so arcane).
The following are the five stages on the journey from novice to expert.
Novices, by definition, have little or no previous experience in this skill area. By “experience,” I mean specifically that performing this skill results in a change of thinking. As a counterexample, consider the case of the developer who claims ten years of experience, but in reality it was one year of experience repeated nine times. That doesn’t count as experience.
Novices are very concerned about their ability to succeed; with little experience to guide them, they really don’t know whether their actions will all turn out OK. Novices don’t particularly want to learn; they just want to accomplish an immediate goal. They do not know how to respond to mistakes and so are fairly vulnerable to confusion when things go awry.
They can, however, be somewhat effective if they are given context-free rules to follow, that is, rules of the form “Whenever X happens, do Y.” In other words, they need a recipe.
Novices need recipes.
This is why call centers work. You can hire a large number of folks who don’t have a lot of experience in the subject matter at hand and let them navigate a decision tree.
A giant computer hardware company might use a script like this:
It’s tedious, but fixed rules such as these can give novices some measure of capability. Of course, novices face the problem of not knowing which rules are most relevant in a given situation. And when something unexpected comes up, they will be completely flummoxed.
As with most people, I am a novice when it comes to doing my taxes. I have little experience; despite having filed taxes for more than twenty-five years, I haven’t learned anything or changed my thinking about it. I don’t want to learn; I just want to accomplish the goal—to get them filed this year. I don’t know how to respond to mistakes; when the IRS sends me a terse and rather arrogant form letter, I usually have no idea what they’re on about or what to do to fix it.[7]
There is a solution, of course. A context-free rule to the rescue! Perhaps it’s something such as the following:
That’s simple and unambiguous.
The problem with recipes—with context-free rules—is that you can never specify everything fully. For instance, in the corn muffin recipe, it says to cook for “about 20 minutes.” When do I cook longer? Or shorter? How do I know when it’s done? You can set up more rules to explain, and then more rules to explain those, but there’s a practical limit to how much you can effectively specify without running into a Clinton-esque “It depends upon what the meaning of the word is is.” This phenomenon is known as infinite regression. At some point, you have to stop defining explicitly.
Rules can get you started, but they won’t carry you further.
Once past the hurdles of the novice, one begins to see the problems from the viewpoint of the advanced beginner. Advanced beginners can start to break away from the fixed rule set a little bit. They can try tasks on their own, but they still have difficulty troubleshooting.
They want information fast. For instance, you may feel like this when you’re learning a new language or API and you find yourself scanning through the documentation quickly looking for that one method signature or set of arguments. You don’t want to be bogged down with lengthy theory at this point or spoon-fed the basics yet again.
Advanced beginners don’t want the big picture.
Advanced beginners can start using advice in the correct context, based on similar situations they’ve experienced in the recent past but just barely. And although they can start formulating some overall principles, there is no “big picture.” They have no holistic understanding and really don’t want it yet. If you tried to force the larger context on an advanced beginner, they would probably dismiss it as irrelevant.
You might see this sort of reaction when the CEO calls an all-hands meeting and presents charts and figures showing sales projections and such. Many of the less experienced staff will tend to dismiss it as not being relevant to their individual job.
Of course, it is very relevant and can help determine whether you’ll still have a job with this company next year. But you won’t see the connection while you’re at the lower skill levels.
At the third stage, practitioners can now develop conceptual models of the problem domain and work with those models effectively. They can troubleshoot problems on their own and begin to figure out novel problems—ones they haven’t faced before. They can begin to seek out and apply advice from experts and use it effectively.
Competents can troubleshoot.
Instead of following the sort of knee-jerk response of the previous levels, the competent practitioner will seek out and solve problems; their work is based more on deliberate planning and past experience. Without more experience, they’ll still have trouble trying to determine which details to focus on when problem solving.
You might see folks at this level typically described as “having initiative” and being “resourceful.” They tend to be in a leadership role in the team (whether it’s formal or not).[8] These are great folks to have on your team. They can mentor the novices and don’t annoy the experts overly much.
In the field of software development, we’re getting there, but even at this level, practitioners can’t apply agile methods the way we would like—there isn’t yet enough ability for reflection and self-correction. For that, we need to make a breakthrough to the next level: proficient.
Proficient practitioners need the big picture. They will seek out and want to understand the larger conceptual framework around this skill. They will be very frustrated by oversimplified information.
For instance, someone at the proficient stage will not react well when they call the tech support hotline and are asked whether it’s plugged in. (Personally, I want to reach through the phone and remove the first vital organ that presents itself in these situations.)
Proficient practitioners make a major breakthrough on the Dreyfus model: they can correct previous poor task performance. They can reflect on how they’ve done and revise their approach to perform better the next time. Up until this stage, that sort of self-improvement is simply not available.
Proficient practitioners can self-correct.
Also, they can learn from the experience of others. As a proficient practitioner, you can read case studies, listen to water-cooler gossip of failed projects, see what others have done, and learn effectively from the story, even though you didn’t participate in it firsthand.
Along with the capacity to learn from others comes the ability to understand and apply maxims, which are proverbial, fundamental truths that can be applied to the situation at hand.[9] Maxims are not recipes; they have to be applied within a certain context.
For instance, a well-known maxim from the extreme programming methodology tells you to “test everything that can possibly break.”
To the novice, this is a recipe. What do I test? All the setter and getter methods? Simple print statements? They’ll end up testing irrelevant things.
But the proficient practitioner knows what can possibly break—or more correctly, what is likely to break. They have the experience and the judgment to understand what this maxim means in context. And context, as it turns out, is key to becoming an expert.
Proficient practitioners have enough experience that they know—from experience—what’s likely to happen next; and when it doesn’t work out that way, they know what needs to change. It becomes apparent to them which plans need to be discarded and what needs to be done instead.
Similarly, software Patterns (as espoused in Design Patterns: Elements of Reusable Object-Oriented Software [GHJV95], also known as the Gang of Four book) can be effectively applied by proficient-level practitioners (but not necessarily at lower skill levels; see the sidebar Misapplied Patterns and Fragile Methods).
Now we’re getting somewhere. Proficient practitioners can take full advantage of the reflection and feedback that is core to agile methods. This is a big leap from the earlier stages; someone at the proficient stage is much more like a junior expert than a really advanced competent.
Finally, at the fifth stage, we come to the end of the line: the expert.
Experts are the primary sources of knowledge and information in any field. They are the ones who continually look for better methods and better ways of doing things. They have a vast body of experience that they can tap into and apply in just the right context. These are the folks who write the books, write the articles, and do the lecture circuit. These are the modern wizards.
Statistically, there aren’t very many experts—probably something on the order of 1 to 5 percent of the population.[10]
Experts work from intuition, not from reason. This has some very interesting ramifications and raises some key questions—what is intuition, anyway? (We’ll delve more into the details of intuition throughout the book.)
Experts work from intuition.
Although experts can be amazingly intuitive—to the point that it looks like magic to the rest of us—they may be completely inarticulate as to how they arrived at a conclusion. They genuinely don’t know; it just “felt right.”
For instance, suppose a physician looks in at a patient. At a glance, the doctor says, “I think this patient has Blosen-Platt syndrome; better run these tests.” The staff runs the tests, and indeed, the doctor is correct. How did she know? Well, you could ask, but the doctor may well reply with “He didn’t look right.”
Indeed, the patient just didn’t look “right.” Somehow, in the vast array of experiences, distilled judgment, memories, and all the rest of the mental effluvia in the doctor’s brain, a particular combination of subtle clues in the patient came together and suggested a diagnosis. Maybe it was the skin pallor or the way the patient was slumped over—who knows?
The expert does. The expert knows the difference between irrelevant details and the very important details, perhaps not on a conscious level, but the expert knows which details to focus on and which details can be safely ignored. The expert is very good at targeted, focused pattern matching.