When you work as a professional programmer, chances are you’ll know
somebody (or you are somebody!) who’s going through this common development
horror story: “We started working on this project five years ago, and the
technology we were using/making was modern then, but it’s obsolete now.
Things keep getting more and more complex with this obsolete technology, so
it keeps getting less and less likely that we’ll ever finish the project.
But if we rewrite, we could be here for another five years!”
Another popular one is: “We can’t develop fast enough to keep up with
modern user needs.” Or, “While we were developing, Company X wrote a product
better than ours much more quickly than we did.”
We know now that the source of these problems is
complexity. You start out with a simple project that
can be completed in one month. Then you add complexity, and the task will
take three months. Then you take each piece of that and make it more
complex, and the task will take nine months.
Complexity builds on complexity—it’s not just a linear thing. That is,
you can’t make assumptions like: “We have 10 features, so adding 1 more will
only add 10 percent more time.” In fact, that one new feature will have to
be coordinated with all 10 of your existing features. So, if it takes 10
hours of coding time to implement the feature itself, it may well take
another 10 hours of coding time to make the 10 existing features all
interact properly with the new feature. The more features there are, the
higher the cost of adding a feature gets. You can minimize this problem by
having an excellent software design, but there will still always be some
slight extra cost for every new feature.
Some projects start out with such a complex set of requirements that
they never get a first version out. If you’re in this situation, you should
just trim features. Don’t shoot for the moon in your first release—get out
something that works and make it work better over time.
There are other ways to add complexity than just adding features, too.
The most common other ways are:
- Expanding the purpose of the software
Generally, just don’t ever do this. Your marketing department
might be drooling over the idea of making a single piece of software
that does your taxes and cooks dinner, but you should be screaming as
loud as you can whenever any suggestion like that comes near your
desk. Stick to the existing purpose of your software—it just has to do
what it does well, and you will succeed (as long
as your software helps people with something they actually need and
want help with).
- Adding programmers
Yes, that’s right—adding more people to the team does not make
things simpler; instead, it adds complexity. There’s a famous book
called The Mythical Man Month by Fred Brooks,
that points this out. If you have 10 programmers, adding an eleventh
means spending time to groove in that one programmer, plus time to
groove in the existing 10 programmers to the new person, plus the time
spent by the new person interacting with the existing 10 programmers,
and so on and so on. You are more likely to be successful with a small
group of expert programmers than a large group of inexpert
programmers.
- Changing things that don’t need to be changed
Any time you change something, you’re adding complexity. Whether
it’s a requirement, a design, or just a piece of code, you’re
introducing the possibility of bugs, as well as the time required to
decide upon the change, the time required to implement the change, the
time required to validate that the new change works with all the other
pieces of the software, the time required to track the change, and the
time required to test the change. Each change builds on the last in
terms of all this complexity, so the more you change, the more time
each new change is going to take. It’s still important to make certain
changes, but you should be making informed decisions about them, not
just making changes on a whim.
- Being locked into bad technologies
Basically, this is where you decide to use some technology, and
then are stuck with it for a long time because you’re so dependent on
it. A technology in this sense is “bad” if it locks you in (doesn’t
allow you to switch easily to some other technology in the future),
isn’t going to be flexible enough for your future needs, or just
doesn’t have the level of quality you need in order to design simple
software with it.
- Misunderstanding
Programmers who don’t fully understand their work tend to
develop complex systems. It can become a vicious cycle:
misunderstanding leads to complexity, which leads to further
misunderstanding, and so on. One of the best ways to improve your
design skills is to be sure that you fully understand the systems and
tools you are working with. The better you understand these, and the
more you know about software in general, the simpler your designs can
be.
- Poor design or no design
Basically, this just means “a failure to plan for change.”
Things are going to change, and design work is required to maintain
simplicity while the project grows. You have to design well at the
start and keep on designing well as the system expands—otherwise, you
can introduce massive complexity very fast, because with a poor
design, each new feature multiplies the
complexity of the code instead of just adding a
little bit to it.
- Reinventing the wheel
If, for example, you invent your own protocol when a perfectly
good one exists, you’re going to be spending a lot of time working on
the protocol, when you could just be working on your software. You
should almost never have any huge invented-in-house dependency, like a
web server, a protocol, or a major library, unless that
is your product. The only times it’s okay to
reinvent the wheel are when any of the following are true:
All of these factors are slowly and gradually
harmful to your project, not immediately destructive. Most of them only do
long-term damage—something you won’t see for a year or more—so when somebody
proposes them, often they sound harmless. And even when you start
implementing them, they may seem fine. But as time goes on—and particularly
as more and more of these stack up—the complexity becomes more apparent and
grows and grows and grows, until you’re another victim of that
ever-so-common horror story, The Never-Shipping
Product.
The basic purpose of any given system that you’re working on should
be pretty simple. That helps keep the system as a whole as simple as it
can realistically be. But if you start to add features that fulfill some
other purpose, things get very complex very quickly.
For example, the basic purpose of a word processor is to help you write
things. If we suddenly made it also able to read your email, it would get
ridiculously complicated. Can you imagine what the user interface would
look like? Where would you put all the buttons? We would say that this is
a violation of your word processor’s purpose. You
didn’t even expand its purpose; you just added features that have nothing
to do with it.
It’s also important to think about the user’s
purpose. Your user will be trying to do something. Ideally, the purpose of
a program should be very close (in the exact words you’d use to describe
it) to the user’s purpose. For example, let’s say the user’s purpose is to
do her taxes. She wants software whose purpose is to help people do their
taxes.
If your purpose and the user’s purpose don’t match up, you’re
probably making her life difficult. For example, if she wants to read her
email, but the primary purpose of the program she’s using is to show ads
to users, those purposes are not matched up.
Want to see your user get angry really fast? Make it difficult for
her to accomplish her purpose. Pop up windows in her face when she’s
trying to do something. Add so many features to your program that she
can’t find the right one. Use lots of strange icons that she doesn’t
understand. There are lots of ways to do it, but they all boil down to
interfering with the user’s purpose or violating the basic purpose of the
program itself.
Sometimes, marketers or managers have goals for a program that are
not really aligned with the basic purpose of the program, like “be cute,”
“have an edgy design,” “become popular with the news media,” “use the
latest technologies,” and so on. These people may be important to your
organization, but they are not the people who should be deciding what your
program does! As a software designer or technical manager, it’s
your job to see that the program stays on track and
never violates its basic purpose. Nobody else is going to hold that
responsibility. Sometimes you might really have to fight for it, but it’s
well worth it in the long run.
And it’s not as if you’d come to a marketing failure with that
philosophy. There are many, many products that have been extremely
successful by sticking to just one purpose. Soap’s purpose is just to
clean things. Salt just makes things salty. A light bulb just lights
things up. But all of these are products that have supported enormous
corporations for decades. You don’t have to have a complicated product to
have effective marketing—you just have to have knowledge and skill in
marketing, which is a completely separate field from software
design.
Really, there’s no need to get fancy and complex and try to do 500
things at once in a single program. Users are happiest with a focused,
simple product that never violates its basic purpose.
Often, if something is getting very complex, that means there is an
error in the design somewhere far below the level where the complexity
appears.
For example, it’s very difficult to make a car drive fast if it has
square wheels. Tuning the engine isn’t going to solve the problem—you need
to redesign the car so that its wheels are round.
Any time there’s an “unsolvable complexity” in your program, it’s
because there’s something fundamentally wrong with the design. If the
problem appears unsolvable at one level, back up and look at what might be
underlying the problem.
Programmers actually do this quite often. You may find yourself
saying, “I have this terribly messy code, and it’s really complex to add a
new feature!” Well, your fundamental problem there is that the code is
messy. Clean it up, make the already existing code simple, and you’ll find
that adding the new feature will be simple as well.
If somebody comes up to you and says something like, “How do I
make this pony fly to the moon?” the question you need to ask is, “What
problem are you trying to solve?” You may find out that what this person
really needs is to collect some gray rocks. Why he thought he had to fly
to the moon, and use a pony to do it, only he may
know. People do get confused like this. Ask them what
problem they’re trying to solve, though, and a
simple solution will start to present itself. For example, in this case,
once we understand the problem fully, the solution becomes simple and
obvious: he should just walk outside and find some gray rocks—no pony
required.
So, when things get complex, back up and take a look at the
problem you’re trying to solve. Take a really big
step back. You are allowed to question everything.
Maybe you thought that adding two and two was the only way to get four,
and you didn’t think about adding one and three instead, or skipping the
addition entirely and just putting four there. The problem is, “How do I
get the number four?” Any method of solving that
problem is acceptable, so what you need to do is figure out what the
best method would be for the situation that you’re in.
Discard your assumptions. Really look at the
problem you’re trying to solve. Make sure that you fully understand
every aspect of it, and then figure out the simplest way to solve it.
Don’t ask, “How do I solve this problem using my current code?” or “How
did Professor Anne solve this problem in her program?” No—just ask
yourself, “How, in general, in a perfect world, should this sort of
problem be solved?” From there, you might see how your code needs to be
reworked. Then you can rework your code. Then you
can solve the problem.
As a programmer, you will run into complexity. Other programmers
will write complex programs that you will have to fix. Hardware designers
and language designers will make your life difficult.
If some part of your system is too complex,
there is a specific way to fix it—redesign the individual pieces, in small
steps. Each fix should be as small as you can safely make it without
introducing further complexity. When you’re going through this process,
the greatest danger is that you could possibly introduce
more complexity with your fixes. This is why so many
redesigns or rewrites ultimately fail—they introduce more complexity than
they fix, or they end up being just as complex as the original system
was.
Each step could be as small as giving a single variable a better
name, or just adding a few comments to confusing code. But more often, the
steps involve splitting one complex piece into multiple simple
pieces.
For example, if you have one long file that contains all of your
code, start improving it by splitting off one tiny
piece into a separate file. Then improve the design of that tiny piece.
Then split off some other tiny piece of the system into a new file, and
improve its design. Continue like this, and eventually you’ll end up with
a reliable, understandable, and maintainable system.
If your system is very complex, this can take
quite a bit of work, so you must be patient. You must first conceive of a
system that is simpler than the one you have now—even
if just in a small way. Then you work toward that simpler system, step by
step. Once you reach that simpler system, you again conceive an even
simpler system, and work toward that. You don’t ever
have to conceive the “perfect” system, because there is no such thing. You
just have to continuously work toward a system that is
better than the one you have now, and eventually you
will reach a highly manageable level of simplicity.
It is important to note, however, that you cannot stop writing
features and spend a long time just redesigning. The Law of Change tells
us that the environment around your program will be continuously changing,
and thus your program’s functionality must adapt. If you fail to adapt and
improve from the user’s perspective for any significant length of time,
you risk the loss of your user base and the death of your project.
There are, thankfully, various ways to balance these two needs of
writing features and handling complexity. One of the best ways is to do
your redesigning purely with the goal of making some specific feature
easier to implement, and then implementing that feature. That way, you
switch regularly between redesign work and feature work. This also helps
your new design fit your needs well, because you’re creating it with a
real use in mind. Your system will slowly get less complex over time, and
you will still keep pace with your users’ needs. You can even do this for
bugs—if you see that some bug would be easier to fix with a different
design, redesign the code before fixing it.
The above is all well and good, but what do you actually
do to make one piece simpler? Well, this is where
all of the world’s existing knowledge about software design comes into
play. It helps a lot to study up on design patterns, methods of dealing
with legacy code, and all the tools of software engineering in general.
It can be particularly helpful to know multiple programming languages
and be familiar with many different libraries, because each involves
different ways of thinking about problems that could be applicable to
your situation, even if you’re not using those languages or
libraries.
Studying those materials will give you many options to choose from
when you are faced with a complexity. The laws of software design can
help you pick which options are good, and then your judgment and
experience can determine what to actually do with your specific problem.
Never robotically apply a tool purely because some authority has deemed
it best—always do what is right for the code you’re looking at and the
situation you’re in.
Sometimes, though, you may look at a piece of code and not know
any tools to use to simplify it. Or you may be new
to programming and not have the time to study up on all this information
immediately. In that case, you should just look at the complexity and
ask yourself, “How could this be easier to deal with or more
understandable?” That’s the key question behind every simplification.
Any true answer to it is a valid way of making your
code simpler; the tools and techniques of software design just help us
come up with better answers.
Some designers, when faced with a very complex system, throw it out
and start over again. However, rewriting a system from the ground up is
essentially an admission of failure as a designer. It is making the
statement, “We failed to design a maintainable system and so must start
over.”
Some people believe that all systems must
eventually be rewritten. This is not true. It is
possible to design a system that never needs to be thrown away. A software
designer saying “We’ll have to throw the whole thing away someday anyway”
would be much like a building architect saying “This skyscraper will fall
down someday anyway.” If the skyscraper were poorly designed and not
maintained well, then yes, someday it would fall down. But if it were
built right to start with and then properly maintained, why would it
collapse?
It is just as possible to build maintainable software systems as it
is to build sound skyscrapers.
Now, with all that said, there are situations in which rewriting is
acceptable. However, they are very rare. You should only rewrite if
all of the following are true:
If all of the above points are true, you may be in a situation where
it is acceptable to rewrite. Otherwise, the correct thing to do is to
handle the complexity of the existing system without
a rewrite, by improving the system’s design in a series of simple
steps.