Chapter 17. Memories of TRW's Software Productivity Project: A Beautiful Team, Challenged to Change the Culture[18]

Barry Boehm

Maria H. Penedo

and

IN THE EARLY 1980S, I (MARIA) WAS LOOKING FOR A JOB AFTER FINISHING MY DOCTORATE IN computer science at UCLA. I was full of enthusiasm for the field I was in and a little concerned about the kinds of jobs I would find. I wanted a place where I could follow my passion and make contributions. I had met Barry at one of the conferences where I presented some of my research results, and thus went to interview with him at TRW. During the interview, Barry described the environment around the company and told me about this new project he was starting whose objective was to revolutionize the way the company developed software.

Barry told me that he had conducted a software productivity study [1] in the company, which performed an economic analysis to determine whether a significant level of investment into software productivity aids would be justified. The factors that led to the analysis were driven by an overall corporate focus on industry and international competitiveness, but also included increased demand for software, limited supply of software engineers, rising software engineer support expectations, and reduced hardware costs—do they all sound familiar even today? This study led to the project that he wanted me to work on, the Software Productivity Project (SPP) [2,3]. He told me it would not be easy, since we would be trying to change the culture. Well, it seemed the perfect challenge for my first job in industry, so I accepted the offer and went to work on the project, together with a fantastic team of TRW veterans and newbies. We were going to attempt to change the current ways of developing software by providing a software development support environment for the aerospace part of the company.

Before it was acquired by Northrop Grumman, TRW was a conglomerate which included a large auto parts company (Thompson Products in Cleveland) and a large aerospace company (Ramo-Wooldridge in Los Angeles). It included a number of software-intensive product lines, including TRW Credit Data, and divisions producing point-of-sale systems, telecom switches, and industrial process control systems. In the mid-1980s, TRW was ranked number two in world software sales in the annual Datamation 100 lists, well behind IBM.

In the 1960s, TRW's aerospace sector pioneered in going from the code-and-fix approach to software development to the requirements-driven waterfall model, as described in Winston Royce's classic 1970 paper [4]. In the 1970s, TRW developed a set of waterfall-oriented software development policies, standards, review procedures, training courses, and requirements-driven software tools. These were a good match to TRW's engineering, science, and real-time control systems of the time, and the culture became strongly waterfall-oriented. By the late 1970s, though, TRW's applications became much more people-interactive, and the waterfall model didn't work well with requirements that were emerging as the project progressed.

At that time, the environment for projects included managers coordinating the people and activities, secretaries who typed the project's documents, meetings to plan and produce the project's activities and tasks, and system engineers and developers producing the designs and the systems. Only developers used software development tools and they had to work in batch processing mode or go to special bays with limited numbers of terminals for interactive development, using mainframe computers. Well, in the SPP we developed a new work environment where all members of the project (managers, system engineers, software developers, secretaries, and controllers) had individual offices with workstations, and communicated electronically via a local area network (LAN). Its architecture was based on the Unix operating system and it included commercial off-the-shelf (COTS) and locally developed tools in support of the full life cycle (e.g., requirements definition, traceability, design and development, forms management, and office automation tools). For this new project, we had to cope with significant technical and stakeholder-conflict risks, and with emergent requirements that could not be specified in advance; thus, we had a great opportunity to apply an early version of the risk-driven, concurrently engineered spiral model developed by Barry [5]. Those changes created culture-clash challenges, especially for the waterfall-oriented veterans and interactive-oriented newbies on the SPP itself. We can still recall heated meetings where TRW veterans would say things like, "But the policies don't allow you to prototype this early! Prototyping is coding before you have passed your critical design review!"

The project was very successful (with significant productivity gains—a factor of two or more, depending on reuse), but it was not easy to institutionalize the changes and convince the personnel. We also learned that productivity gains require an integrated program of initiatives in several areas and an ongoing and sustained effort. Even though this project happened long ago, the stories we tell in this chapter describe points that are still valid today, in spite of all the advancement in tools and technologies, both in planning and in executing productivity activities. Changing cultures is difficult, and it is even harder to keep up with the fast pace of technology in large organizations.

Scene: Bob Williams' office, late 1979. Bob is the vice president/general manager of the 2,000-person Software and Information Systems Division, one of six divisions in the TRW Defense and Space Systems Group (DSSG). Barry is his chief engineer and advanced technology business area manager.

Bob: I've just come back from a DSSG General Managers off-site about improving productivity. Corporate in Cleveland is making a big push to get the auto parts division to be more competitive with the Japanese, and wants everybody in TRW to focus on improving their productivity. It looks like the company will put up money for productivity initiatives if there's a good business case for them. I think it's worth a try. Do you think you can put something together for us?

Barry: Definitely. This fits with a lot of improvements we've talked about but haven't found funding for. Our Constructive Cost Estimation Model (COCOMO) provides us with a good framework for a business case. It shows how much our productivity goes up or down as we change some of the cost drivers like tool support, turnaround time, reusing components, and people factors. This last driver would fit with your ideas about multiple career paths for our people. We could probably use some of our local area network technology to get everybody interactively working and communicating. And we could probably get added support from some of the Defense Department's Ada initiatives. Are they looking for a full-up proposal?

Bob: Well, if we were proposing to the government, that's what we would do. But since this is an internal company initiative, the sponsors want a clearer idea of their options, before they commit to spend a lot of money. So, we have a couple of months to put a white paper together. Why don't you do a part-time study with Ray Wolverton and a couple of the Ada guys and put a draft together? And, let's get everybody involved by doing a survey of what people think would best help them improve their productivity.

Barry: Great. We'll get right on it and give you a progress report in a couple of weeks.

The project's incremental versus total upfront commitment was the first opportunity to fully apply the spiral model of software development [4] to incrementally explore options, refine scope, and obtain higher levels of management and user buy-in at the end of each spiral cycle. After the first white-paper cycle ended with approval to proceed, the second cycle of the spiral involved visits to advanced technology centers such as Xerox PARC and IBM Santa Teresa, LAN and workstation architecture and market analysis, more detailed manager and developer needs surveys, demos, and prototypes. Three operational concept options were prepared at expenditure levels of $2K, $10K, and $50K per person, resulting in selection of the $10K/person alternative, although the Xerox PARC workstations were tempting.

Scene: Bob Williams's office, mid-1980.

Bob: Good news! We've been selected to develop our proposed Software Productivity System. You guys did a great job in the Round 1 white paper, which got us support to develop the operational concept options. With the help of the productivity data from IBM and AT&T's initiatives, they bought into our $10K-per-person option with the local area network, the lower-cost version of the IBM private offices, the Unix-based support system for not just programmers but everybody on the project, and the complementary management and career path initiatives. So, we're funded at $1 million per year to do this, but we'll need to pass a review based on a set of prototypes, specifications, and plans, and find an early-adopter project to work with you on building what they need. I think I can find a good pilot project. Do you have anything further you need to get started?

Barry: That's great! I'll need some help in working with the facilities people on reconfiguring and wiring the private office complex, and on letting the TRW LAN people know that we'll have to do a competitive analysis of their product's maturity and performance. Also, we'll need to hire some top technical people with extensive Unix experience. There are a couple of people at UCLA and UC Santa Barbara that I think will be very interested.

Bob: Fine. You can put together a presentation on this for the next staff meeting, and I'll identify people to work with you on those.

The productivity project was fortunate that its early-adopter partner project was a portion of a very large real-time application with an open-minded manager and many performers who, after some resistance, appreciated the ability to have dedicated interactive workstations and electronic communication. And the Software Productivity Project team had a good balance of experienced TRW software developers and new-hire Unix environment experts. The Unix key experts from UCSB (Art Pyster) and UCLA (Maria Penedo) were very good in coming up with creative and new ideas, engaging the team members, and showing how Unix capabilities could improve automation, support rapid prototyping/development for both tools and target software, and shortcut some of the frequent difficulties projects would encounter. The pipe and filter and other features of Unix made it possible to respond regularly to those kinds of customer requests; we could put together simple solutions to problems and respond quickly to customer requests because of the flexibility of the Unix environment.

Scene: An early Unix demo to the partner project people, mid-1981.

Skeptical old-timer: All of that is very interesting, but can it do anything useful for the project, like create a specialized telephone directory?

Art: Sure. We'll just do a "grep" on the names in the project roster and the names in the company phonebook, "pipe" it to a new file … and there it is.

As we mentioned before, a project was assigned to use the new environment/tools being developed. The LAN idea was very welcomed, but the operating system (we were introducing Unix into a DEC VMS-oriented culture) and some of the new tools were not welcomed by all. To make it harder, the partner pilot project was using the VMS operating system, which already came with a few good automated tools. Also, several personnel in the assigned project complained that Unix on the VAX ran much slower than did the VMS operating system. That was true, but they missed the fact that that was a development environment and that overall productivity was improved because the people were the more valuable resource at that point (they still had the mindset that machine time was more important than people time). Over the course of the project, some became converts, some never did.

What was interesting, however, was to observe how sometimes something very small can have a huge effect on user acceptance. We have two examples to tell, both involving showing the value added in the new automation. The first one was about a secretary who typed most of the documents for the project and dealt with quite a few of the project personnel; many of the documents she typed had many equations for which she had to leave space to type, even if she used an automated word processor. That was both cumbersome and slow for her and her customers. Once she found out that Unix had the "eqn" tool that automated equations, she became a convert and was able to convert many more in her circle of work, including managers and system engineers, who were by then also being converted to use word processors. The other example was a savvy administrative assistant/data manager, who figured out she could automate some of their forms needs using Unix "scripts" as well as providing configuration management with the new tools; her boss was so impressed with the results that he mandated all his staff to use the system/tools. She used her creativity to solve a problem and had an open-minded boss who also dared to mandate (sometimes that is the only way to get some people to change).

Every project has its lone wolf, who, for reasons unknown to most of the team, appears to not want to be part of the team, lacking bedside manners. We had one of those, an extremely smart and talented person who had no patience for the "normal" people. He was extremely fast and constantly surprised us with his innovation. We remember that he got the Forms Management package to speed up performance by a factor of 4 or 5 because he just dug around into Unix until he found a few obscure interface calls. All that was said to him was that the package was running too slowly for many. He grumbled about how they should appreciate what they had, walked away, and came back two days later, all smiles and with a new version that ran four or five times faster … and we didn't even know that he was working on it.

But his aloofness turned out to be very difficult for many. It bothered the team; we wanted to bring him in, we felt it would really improve team collaboration since he was one of the most knowledgeable in the group. We don't know exactly what caused the change, but one of the members of the team, a smart lady with degrees in both music and computer science and with great wit, became his friend and broke through his glass shell. We think after that, he started trusting and actually enjoying some members of the team. Life in the project became so much better after that. The lesson here is that if people care and take their time, ways can be found to include everyone.

What would we do without email these days? (Actually, the email of yesterday may equate to the Instant Messenger or social networks of today.) We institutionalized email across the projects with the new environment. But of course, that brought about issues, which still apply to date. Email has many benefits: it enables people to communicate or broadcast messages instantaneously to multiple recipients across a great distance, it allows one to leave messages when the recipient is not at his or her desk, and so on. However, it can also incur negative productivity in that, instead of picking up a phone to talk to another person to solve a problem or to discuss some ideas, one defaults automatically to writing an email message. That can waste time on both sides, taking longer for resolution of issues when a short phone call would satisfy. There were many frustrating moments of people not addressing issues posed in an email because they either didn't read their email carefully or they misunderstood the email.

Also, the problem of sending cryptic email can create hurt feelings and misunderstandings. Due to the non-interactive and non-personal nature of email, the "quality" of communication often suffers. Misunderstandings develop and sometimes are not clarified or remedied until significant cost in time and labor is incurred. A side effect of the SPP environment occurred when some people stopped stepping out of their private offices to talk to each other. They sent cryptic email messages rather than popping their head into their neighbor's office to ask a question or discuss an issue. That led to some team issues that had to be solved by an all-hands where management encouraged the team to "talk" to each other rather than defaulting to email as a means of communication.

Also, email is not efficient for assigning action items or important items. One should never assume that the email was read correctly on the other side, if read at all. And sometimes over-reliance on email multiplies the inefficiencies—for example, if "Reply All" is used indiscriminately or accidentally.

The reason the project started in the first place was to bring new automated tools into the organization for the purpose of enhancing software engineering productivity. However, we found out that the pace of technology changed very rapidly and had to create means to keep looking toward the future in order to fulfill the goals of the project, to continuously bring in and deploy new technologies (that was before IT took such central places in enterprises). It wasn't (and still isn't) trivial. In parallel with the program, a future-looking activity evolved, and from workstations we moved into bitmap displays, computers at the desktop, and so forth. As this project moved on into other improvement activities, we found out that in-house technology or tools development within the organization was generally short-lived and invariably overtaken by commercial products. Each one of the efforts started with high hopes, made some progress, and caused changes and improvements, but organizations need to be constantly reinventing themselves. Many of those efforts served an excellent purpose in introducing the concepts into the consciousness of the user community and in illustrating the need and the benefits of the technologies, but many of them ultimately lost momentum due to the Iron Law of Software Maintenance and the relatively small user base across which to amortize changes to the custom in-house tools. That is the reason why, in today's world and age, most tools in use come from the commercial world, and those by themselves carry their own challenges of robustness, cost, adaptation to user needs, and evolution.

The productivity project was fortunate to have a number of the features that distinguish successful from unsuccessful software projects: top management support, capable and enthusiastic team members, realistic budgets and schedules, concurrent requirements and solution development, and iterative development. However, many projects have had all of those factors, but have fallen short of having a beautiful team experience. In comparing the productivity project with some of these other projects, we would say that some of the key beautiful team enablers were:

Our thanks to the contributors and reviewers of this chapter, including Christine Shu, Frank Belz, Art Pyster, and others who participated and made this a beautiful team. We would also like to dedicate the chapter to the memory of Don Stuckle, the productivity project manager and a truly beautiful person.



[18] Editors' note: if you've worked on a software team in the past 20 years, you have been influenced by Barry Boehm. He was one of the first people to take a systematic approach to estimating and planning software projects. And many people (including us) believe that his pioneering Spiral Model is the direct predecessor to the modern idea of iterative development.