Chapter 3
Managing the Design Process
Given that design (as a verb) is an important means of developing ‘solutions’ to complex ill-structured problems, an important question is how we manage this process, and what information we need to record as part of that task.
3.1Cognitive capacity
Back in 1956, George Miller, a psychologist from Harvard University, published a paper titled The Magical Number Seven, Plus or Minus Two: Some Limits on our Capacity for Processing Information (Miller 1956). Essentially, this very extensively-cited paper was about the limitations of human cognitive capacity for processing different forms of information. Strictly speaking, Miller's main focus was upon the ability to make judgements related to distinguishing between different stimuli of the same type, such as sound tones, or tastes, although he also discussed the quite distinct limitations upon immediate memory. However his use of the number seven as representing some form of limit on cognitive capacity has subsequently been interpreted (or mis-interpreted) very liberally.
The key point for our purposes here is to appreciate that the human brain has a limited capacity for recognising things on a ‘one-dimensional’ scale. Indeed, the exact number of things that can be distinguished is not too critical as far as we are concerned; what matters is that there is a limit on how much information about a design can be readily accommodated in the designer's head, and that the limit is set quite low.
In practical terms, what this means is that managing the design process requires that the designer needs to use some way of handling very many design elements that each have complex properties. It simply isn't possible to hold everything that needs to be considered in one's head while developing or modifying a design model, meaning that the designer needs tools to help them cope with these limitations upon cognitive capacity. (And of course, the tools themselves may involve cognitive limitations. As we will see, diagrammatical notations do seem to be harder to use when they have overly many symbols—a point that the designers of the UML do not seem to have grasped (Moody 2009).)
So the rest of this chapter looks at some of the ways that we can cope with these cognitive limitations, and what it is necessary for a designer to do.
3.2The power of abstraction
In order to create manageable models of the intended system and envisage how it will behave, we need to find ways to put aside the details that don't matter and to focus upon the factors that do matter.
The concept of abstraction is an essential one when formulating design ideas in any branch of engineering. The process of abstraction involves removing or omitting unneeded detail from a description of a model or plan, while still retaining a description of those properties that are relevant. Doing so reduces the cognitive load involved in understanding particular aspects of a design model.
Our earlier example of moving house illustrates this well—the two-dimensional plan provides a model of the house that retains only the information that is important to consider when positioning furniture. It describes the things we need to know for this purpose (where sockets are located, which ways the doors open etc.) but omits such information as whether windows are clear or opaque, what colours the walls have been painted, or where the lights are located etc. Of course, for other tasks we might need a quite different abstraction of the house. If we are planning to modify the power outlets and lights, we would need a very different sort of representation that relates to the different electrical circuits, and that probably omits much of the information considered appropriate for positioning furniture.
Abstraction is both an important concept and also one that novice software designers may well find difficult to employ effectively, as demonstrated in an early study by Adelson & Soloway (1985). Programmers are used to working with what in many ways is a wonderfully pliable medium, but the nature of programming requires that the programmer needs to anticipate all of the different ways that a chunk of software will need to work. It is only too tempting for the novice designer to focus upon the ‘comfort zone’ of detailed programming structures when thinking about design, since these are likely to be more familiar, whereas as a designer, they need to learn how to think about the properties of a system in an abstract way. Design models need to relate to events, entities, objects or whatever other key elements are involved in thinking about a particular aspect of the design, and to leave to a later phase issues of detail such as the choice of specific data structures, or the form of loop construct needed for a given algorithm etc.
Abstraction therefore plays a key role in this book, corresponding to its central role in designing systems. In particular, when we look at how the characteristics of a design can be represented we will find that we need to think about a number of different abstractions. The effective and appropriate use of abstraction is a key skill that any designer needs to learn and employ. We will be revisiting this in the next chapter and in many of the later chapters.
As a final thought about abstraction, and to provide a tangible illustration, we briefly revisit our first sketch related to the CCC.
3.3Modelling and making design choices
When describing what designing is about, we might say that our aim is to produce a ‘design solution’ that will meet a specific need. In computing, that need will typically be documented in some form of requirements specification. This may be little more than a few jotted notes or sketches, through to a very detailed formal document (the specification for the CCC system comes somewhere between those). It is then the task of the designer or design team to formulate a way of meeting that need by creating some form of design model for the proposed solution.
The act of designing something usually involves identifying suitable options for the key features of our ‘design solution’ and then evaluating the consequences of adopting each of them. For example, one of our first decisions may well be to choose an appropriate architectural style for our application. (We will discuss the idea of system architecture and architectural style more fully in later chapters; for the moment we can just consider the architecture of an application as being the types of software module we will employ.) So, for a particular application, we might consider organising it using an object-oriented form, based upon a set of classes and objects, or organising it around a central database. Both approaches will have their merits: using objects may mean that any future evolution of our application that is anticipated will be easier to achieve; while using a database may mean that the application can be delivered to the end-users more quickly, although future changes may require more work. And here we can see one aspect of the problem of choice, in that our reasons for preferring one design solution over another may well be based upon how we rank quite different criteria, with the options also incurring quite different degrees and forms of technical debt.
Making a choice between the different design options may also be based upon criteria that are relatively ill-defined. And as the early choices we have to make are going to be based upon a very abstract and incomplete model, making them will almost always involve making trade-offs between estimates of the likely impact of the different options upon both system functionality as well as such non-functional properties of the model as size, speed and ease of change. There may also be other more context-specific factors that might influence design choices: in that an application may need to conform to the way that an organisation operates; it may need to work with, and communicate with, other systems; and there will nearly always be the need to plan for likely future extensions. This last point, with its implications of different levels of technical debt, reminds us that designers need to think to the future, not just to the present.
Throughout this process though, the designer needs to keep in mind that the ultimate criterion has to be fitness for purpose. It is not that the other factors don't matter, they certainly do, but they need to be subordinate to the need to produce a system that does the required job. This goes for most artifacts of course—however much we might like the added “bells and whistles” of a particular model of car or of a specific hi-fi amplifier—pragmatism usually means that our ultimate choice will need to be made on the basis that it can be relied upon to do the job that we need it to do, and that it will do so at a reasonable cost.
Using these criteria to help evaluate possible options can be made easier if the designer produces some form of design model that can be used to help manage the cognitive processes involved. The nature of an ISP means that we need to postulate a design solution and then reflect on whether or not it will do the job we want, and how well it will do it. Having some form of abstract design model, possibly realised in the form of diagrams and text, can help with managing this process of reflection, perhaps by using it to evaluate different scenarios of use or change via some form of ‘design walk-through’.
And finally, the idea of fitness for purpose isn't some abstract concept, it needs to be given concrete values that can be used to assess the terms above. We need to have a clear picture of what we expect of something that ‘should work’ and of what we mean by ‘as well as possible’. If we are building something that is safety-critical (that is, lives will be at risk if it fails), then we should have some quite definite ideas about these terms. This is less so where the concept of ‘works’ might be less precise (such as an application that depends upon some form of image recognition), where we might be willing to accept a less than complete level of reliability, at least in early versions.
This is maybe where some reverse thinking may help. Rather than trying to define ‘should work’, we might try to consider what sort of shortfalls in functionality or performance might still mean that we consider that the system ‘works’, even if it is not doing so as well as we wish. For the example of image recognition, we might be willing to accept that our application may sometimes be unable to recognise an image, or fail to get the right match, particularly if such a failure can be easily identified by the user. On the other hand, an application where the outputs are occasionally badly in error in a way that cannot easily be recognised by the user is likely to be considered unacceptable, even if this occurs very infrequently.
3.4Recording design decisions
Something that is often overlooked when a software application is developed is the value that can be derived from noting why particular design choices were made. While the rationale for these may have been obvious to the designers of the original system, they may well be far less obvious to those who are later tasked with extending or modifying a system. However, while our design model will usually record what choices were made, the rationale behind those choices is not so readily included. And although the absence of knowledge about the original rationale can add significantly to technical debt, it often remains overlooked. (In fairness, this problem is not one that is confined to software engineering; it happens in other forms of engineering too.)
Beginning with the original design, the recording of rationale is more likely to be encouraged if the software development project includes some form of design audit. Such an audit may consist of a peer review of the designer's ideas, or it may be something rather more formal. Whatever form it takes, an audit may help with recording the reasons for particular design choices. And conducting such an audit requires that we have a design model that is recorded in a form that can be shared with others.
Arguably there is an equally good reason for the rationale for maintenance (design evolution) changes to be recorded. Software applications may have quite long lifetimes and undergo extensive changes over that period, often involving much greater developer time than the original development. Ideally, the maintenance team need to be able to recreate the original models used by the designers so that they can reliably decide how to implement their changes in a manner that preserves the structure of the application (Littman, Pinto, Letovsky & Soloway 1987). In turn, they too should record the reasons for their changes.
Unfortunately, while different software design approaches generally encourage the recording of decisions about how an application should be structured, they are apt to place less emphasis upon recording the rationale for those decisions. It is easy to understand why this is so. A design usually evolves through a set of different stages and forms, and recording the reasons for each step and/or change in thinking may well be seen as a chore by the design team. This issue was recognised many years ago by Parnas & Clements (1986). They observe that, even if the design process was not a rational one (which it usually isn't), the documentation that described it could still make it appear as though it were. In other words, the design documentation should be written so as to describe a consistent and rational process.
Parnas & Clements argue that the principal benefits of doing so are that new members of a design team (or those later charged with extending the design) will then be able to absorb knowledge about the project more easily. Indeed, they observe that even for scientific studies such as mathematical proofs, the form eventually published is rarely the form in which the proof was actually derived, because as understanding grows, various aspects can usually be simplified. In the same way, it is the actual structure of the system design, and the reasons for it, rather than the way that these emerged, that will be of importance to new team members.
An excellent example of the complexity of team development is that of many open source software (OSS) projects. Although various tools exist to help track a project through its many iterations, there still seems to be relatively limited scope (or perhaps motivation) to record design rationale, which in turn can provide a challenge to anyone joining a project (Steinmacher, Silva, Gerosa & Redmiles 2015). And for professional developers, recording decisions and rationale is still only likely to occur where there are some appropriate quality control processes in place.
3.5Communicating ideas about a design model
Once a designer has formulated a ‘solution’ intended to meet the need of their customer, the next step is to consider how their ideas about the form of their design model are to be conveyed to others, since to be useful, a design needs to be implemented in some way.
Just when the concept of the creation of an artifact requiring some form of ‘design activity’ first emerged can only be a matter for conjecture. Many of the major creations of antiquity such as the Egyptian pyramids must have required quite a significant element of planning, so civil engineering in the broadest sense, may well be where this first became recognised1. Later on, in the thirteenth century, King Edward I of England employed ‘Master James of St George’ to build some of his major castles, many of which still stand today. While he was originally described as a ‘master mason’, his heritage leaves little doubt that he was someone who possessed significant design skills. Certainly, the nature of civil engineering did require clear directions to be conveyed from designer to the ‘production team’.
When software engineering first emerged as a discipline, ideas about the form software development should take were strongly influenced by the practices used in existing branches of engineering. Hence, the notion of there being some form of distinct design step tended to be assumed in early thinking about software development, as illustrated by the idea of the ‘waterfall model’, shown in Figure 3.1. In this approach, the expectation is that the design model will be complete before being implemented. However, this was always regarded as rather idealised, and design activities can also be, and usually are, interleaved within the other steps of development. This is particularly useful where the details of an application are not fully known, so its design may need to evolve as understanding of its role emerges, leading to the evolution of more ‘agile’ approaches as we will see later.
Figure 3.1: A waterfall model of software development
So, while a design model may help the designer cope with the cognitive challenges of designing complex things, it also plays an important role in sharing ideas about the design with others. The model may be little more than a list (as in the example of moving house), or it may be much more detailed, as in the ‘blueprint’ traditionally produced for mechanical design. The level of detail will depend upon how much freedom can be given to those responsible for fabricating the design ideas. For mechanical engineering where end-products are not easily modified or adjusted, the degree of tolerance that can be permitted is important, so those responsible for fabrication activities will need to follow the design plan very closely. For house-movers and software engineers, whose processes can be modified rather more easily, the plan can be much less detailed, to the point that it can evolve while the implementation task is progressing. This is a point that we will return to, as it is one of the ways in which too close an adherence to traditional design thinking is often not appropriate for the software domain.
So we can conclude that communication with those who are responsible for fabricating the solution is likely to be an important part of a designer's role. In addition though, there is also a need for communication with those who are commissioning the design. Their requirements might be very tightly specified and the description of these may focus upon such factors as their market and other constraints (“a table for use in village halls, cheap to fabricate, durable, easily cleaned, being capable of being safely stacked, and being light enough for one person to move it easily”). Or they may be rather broad and sweeping (the medieval baron who wants “a castle bigger and grander than the one owned by their rival across the valley”).
So here we see another demand upon the designer and their skill-set. The designer needs to have a good understanding of the problem domain (we can assume that in the examples of the previous paragraph the designer would know about tables or castles) and also of the materials that they need to work with. The great Victorian engineers like Isambard Kingdom Brunel were prepared to design railways, ships, bridges, railway stations, and many other necessary items as part of their portfolio. However, since that time design activities have tended to become more specialised, partly because of this need for deeper knowledge about the context of a design. So an aeronautical engineer is unlikely to be employed to design roads.
One thing Brunel didn't design was software!
In some ways though, software designers find themselves more in the situation of needing to create a wide range of applications, more as the Victorian engineers did (but fortunately without the need to wear tall stove-pipe hats). The use of software is so ubiquitous and its form so adaptable that over-specialisation is rarely an option. Admittedly a specialist in (say) database systems might (or almost certainly should) hesitate to take on the task of designing a real-time command and control system, but in principle at least, one person could possess the necessary range of skills for both.
So the acquisition of domain knowledge plays a particularly important role for software design. Some of this may be obtained from the customer's specification, but as we will see later, this may well fail to identify key elements of the ‘tacit’ knowledge that are familiar to everyone working in the domain—but that may not necessarily be familiar to the software designers.
If we think about designing software rather than castles, and particularly about designing software applications that will be implemented by a large team of people, what sort of thing are they likely to want to know from the designer(s)? At the very least this is likely to include:
what the overall ‘architecture’ of the application is to be—what will be the major elements, what they will do and how they will interact (this will determine many things, including how the elements will behave);
the details of key data objects that the application will need to use (usually the ‘real world’ things that need to be modelled or represented);
any specific algorithms to be used to achieve the necessary functionality;
how the elements of the application will be packaged and organised;
how the elements of the application will interact, including any dependencies (coupling) and causal (event-driven) links;
how the application will interface to, and interact with, its environment (users, other software components such as browsers, etc.).
Mostly these are concerned with the form or structure of the application, but some are also concerned with its behaviour, both overall and in detail. So the outcomes of the design process (the design model) is likely to be concerned with the way that the elements are structured, the functions that the system will perform, and the way that different elements will behave, individually and collectively. Whew!
Key take-home points about the design process
In this chapter we have considered the outcomes of the process of designing software, and addressed some ideas about what we might consider to be a ‘good’ design.
Abstraction is an important design tool. A designer needs to focus on the core issues that are relevant to a solution, and to successfully manage their ideas about this, needs to be able to ‘discard’ consideration of other issues that are not relevant to this.
Fitness for purpose should be the ultimate criterion. While a good design solution may exhibit many qualities, the ultimate need is that it should work and do what is required as well as possible.
Design models are a key product of designing. Design as a process (verb) leads to design as a product (noun). In its most simple form, where a designer also implements their design, the design model may simply reside in the designer's head, but beyond that it needs to be more formally recorded in whatever manner is most appropriate.
The rationale for design decisions should be recorded. While this may not be of immediate value to the original application development team, it will help others to ensure that future modifications and extensions will preserve the integrity of the original design structures, and so reduce the technical debt during later evolution of the system.
Design solutions need to be shared with others. While communication is particularly important where manufacturing forms a part of development, software applications are often created by teams of developers, and they all need to be aware of, and conforming to, a common set of goals and models.
Designers need domain knowledge. As well as knowing what can be done with a particular medium (in our case, software) a designer also needs to know something about the environment in which the eventual application is to operate.