How to Castrate a Bull

Tom Mendoza shredded our initial sales strategy as soon as we hired him. To grow a business, you must find either more customers or bigger customers. Our original plan was to sell to start-ups, or to small engineering workgroups within large companies, and find lots of them. Instead of selling to ten small companies, Tom asked, why not sell ten times as much—or a hundred times—to one large company? Once we got in the door, we would be able to count on a regular and significant stream of orders rolling in automatically as the customer grew. “We’ll make millions,” Tom said, “as long as the fax machine doesn’t break.”

In retrospect, it’s not surprising that our first customers were engineering workgroups. James and I had spent our entire careers as engineers in Silicon Valley, and as I said in Chapter Six, we designed a product that we would like to use ourselves. This is a common pattern. Sun, in its early days, consisted mostly of hardware and software engineers, and they developed workstations for use by hardware and software engineers. Apple consisted of two home computer hobbyists, and their first product was—surprise!—a home computer for hobbyists. We have met the customers, and they are us. This is a great strategy because you automatically have a strong, intuitive understanding of your customers.

Tom’s point was that large engineering groups are very similar to small ones, but they need more storage and have more money. So our first big customer transition was from small engineering workgroups to higher-end technical and scientific computing. We didn’t have to change much to reach these new customers. They were still very technical, so the sales process was similar: we would explain how our box worked, why it was better than the competition, and customers would figure out themselves how it could solve their problems. At many companies, we already had a foot in the door. Our storage was cheap enough to fit within the budget of a first-line manager, so rogue engineering groups would buy our storage if they got tired of waiting for someone upstairs to solve their problem. When we started selling at the departmental level, we already had supporters who could act as internal references.

Our products worked for both small and large technical environments. At first the most common applications were software development and chip design. Later we added seismic processing for oil and gas companies. They take multi-terabyte “photographs” of the ground, analyze them like crazy, which produces even more terabytes, and then crunch all that data down to a single bit of information: drill or don’t drill. (To put the amount of data in context, one terabyte would hold about two million copies of this book.) The military intelligence community is similar, except that they take multi-terabyte satellite photographs and crunch them down to a different bit of information: bomb or don’t bomb. Hollywood studios used our equipment to store animated movies like Ratatouille and computerized special effects for movies like King Kong and Transformers.

Internet customers became especially important as the dot-com boom progressed. At first they were like other technical customers. System administrators from technology companies started many of the early Internet companies as a hobby, because they wanted to read newsgroups that weren’t allowed at work (not only porn, although that was a factor). They installed equipment in the garage and sold login privileges to friends to help cover the costs. They naturally bought equipment that was familiar to them: Sun computers, Cisco routers, and NetApp storage.

The requirements for Internet companies changed dramatically when they realized that shutting down service for thousands (later millions) of people would earn them front page headlines. Suddenly their applications were mission critical—they absolutely had to keep working 100 percent of the time. Often they’d fire their first CEO and hire one with experience making things very reliable, like someone from a telephone company. Even though they were start-ups, they wanted the reliability that was normally associated with mature corporate data centers and they were unhappy if they didn’t get it.

I remember a particularly painful meeting with Yahoo in the mid-nineties. Yahoo was one of our largest Internet customers, and they stored all of their e-mail on NetApp. They told us, “We built our entire infrastructure on your equipment. Our customer base is doubling every three months, and your NAS storage has allowed us to scale faster than anything else we can imagine. But if we could find an alternative to NetApp, we would throw you out in an instant.”

We could have told Yahoo, “We’re sorry that you chose to build a high-reliability data center out of workgroup equipment. It’s no surprise that it doesn’t meet your needs. This must be very difficult for you.” Yahoo was frustrated with us, but really it was a tough love message. They hoped NetApp would succeed, because they depended on us to enable their rapid growth. We could have turned them down, but they represented enormous opportunity. Not only was Yahoo a big customer, but we felt that other Internet companies would face the same issues. What’s more, we were getting similar complaints from Cisco, which was one of our largest tech customers. They loved our systems at first, but like Yahoo, they started running into reliability problems when they had installed hundreds of them. We decided to focus on Yahoo’s and Cisco’s needs.

James and I had mostly worked in small companies, and we were clueless about big companies with big data centers. We no longer had an intuitive understanding of our customers’ needs. I suppose it’s inevitable that the strategy of sticking to what you know breaks down. If you keep growing long enough, you will eventually start selling to customers who are less like yourself. This is when you need to hire people with the customer-centric thinking skills described in Chapter Five. At a start-up serving customers like yourself, you unconsciously do the right thing; to mature that start-up into a large company serving other large companies, you must learn to consciously study and understand their special needs.

As VP of engineering, I started a program called “Love Yahoo/Love Cisco.” I appointed a full-time chief love officer (CLO), with the job of doing whatever it took to make Yahoo and Cisco happy. The motto was Customize then Standardize. If he had to customize our products to meet their needs, then so be it, but I also wanted him to put those changes back into our standard products so that the rest of our customers would eventually get the benefit too. Many salespeople asked me to add their customers to the Love Program, but I always refused. My main goal wasn’t to make Yahoo and Cisco happy—although that was a nice side effect—but to learn from them. Yahoo was one of our largest and most demanding Internet customers; if we could make Yahoo happy, we could make any Internet company happy. Likewise, for Cisco and other tech companies.

Bad Hair Day

I was once on a sales call in New York, talking about NetApp’s commitment to product quality. I told a story about how I tried to motivate the engineering department by promising to dye my hair any crazy color they wanted if they could reduce our failure rate by a factor of ten.

The magenta, blue, and red had long since grown out, so the customer looked me over and said, “And they chose gray?”

The Love Program completely changed our product road map. We designed systems that worked in pairs—if one failed the other could take over. We wrote software that replicated all data to a remote location—no data loss even if a data center burned down. And lots more technical details I won’t get into. We had to cancel or delay many other projects, but Yahoo and Cisco represented so much opportunity that it was worth it.

Investing in the Love Program was a major turning point for the company. Yahoo and Cisco are both still big customers today. If we had not taken their pain seriously, we would have remained a low-end company, and I believe that our growth would have stalled. In essence, we were betting that tech and Internet companies could fuel our growth, and we were right. By the time we reached a billion dollars, 70 percent of our revenue came from those two categories.

In betting on Yahoo, we were betting on the Internet. Earlier I said that the Internet was a giant wave that we spotted and chose to ride. The wave eventually crashed, but the ride carried us far. Our improvements in reliability and data protection set the foundation for going after enterprise customers more broadly.

The term enterprise refers to large businesses solving large problems. When people talk about enterprise-class equipment, they mean equipment reliable enough to handle your most mission-critical applications: stuff that would really damage your business if it failed. Many engineering groups don’t qualify as enterprise. For a large group, failure could mean that hundreds of engineers sit idle. It’s a big waste of time and—if it happened too often—could hurt the company’s competitiveness. But most engineers work on products that won’t ship for months or years, so a day or two of downtime can be absorbed in the development schedule. Painful, but it probably doesn’t put the company at risk.

To understand enterprise, consider Southwest Airlines. If their storage fails, they can’t sell tickets. In this post-9/11 era, if they can’t access their passenger lists or cargo manifests, both stored on NetApp, it is illegal for their planes to take off. If the NetApp storage stays down for two hours, then every Southwest plane in the air must land immediately at the nearest airport. It may be painful for Cisco if their development engineers sit idle for a few hours, but that’s nothing compared to the pain that Southwest would feel. This is why enterprise customers are so, so conservative. Companies are most protective of the applications with which they collect money from customers, count their money, or provide services to their customers.

Yahoo and other Internet companies were sort of enterprise: they did rely on our storage to provide services to their customers, but being so young and so recently managed like tech start-ups, they didn’t have the justified paranoia of mature enterprise companies. For years, Tom Mendoza tried to get NetApp into Wall Street trading firms. I remember him describing one meeting with a chief information officer (CIO) where she started by saying, “Okay, I know your story. You can do everything EMC can do, and you can do it for a lot less, right?” (EMC was and is our biggest competitor.) Tom nodded, and she said, “With a story like that, you will never make a single sale on Wall Street. My company has plenty of money, and as long as everything keeps working, I’ll get promoted. Why would I change anything?”

For enterprise customers, change equals risk. Consider the cost of an airline grounding every plane or of a Wall Street firm not being able to execute trades. The downside is so painful that they need a very, very good reason to change. If they have plenty of money and a solution that works, there is almost no amount of savings that would justify a new vendor, and especially not an unknown newcomer. Remember Tom Mendoza’s saying: Customers only open their wallets when they are in pain.

Fortunately for us, the tech crash sent the economy south, and the recession was good for us. It created so much pain that conservative enterprise customers were forced to consider new solutions. The attitude changed from, I’ll be promoted if I keep things working to I’ll be fired if I don’t cut costs. The tech crash also gave us extra incentive to go after the enterprise companies, because our Internet and technology customers—70 percent of our revenue—pretty much stopped buying.

We had actually been going after enterprise customers for several years before the crash. Dan was paranoid—in a good way—and relying so much on tech and Internet companies scared him. In the late nineties, Dan decided to target five additional market areas. The first was financial services, which was why Tom kept trying so hard to get into Wall Street, and the others were telecommunications, oil and gas, major manufacturing, and the U.S. government. Today we’re a major vendor in all five areas, but early on it was tough going.

I got an early hint that selling to non-tech companies was different when I made a sales call on Georgia Pacific, the paper company. I gave my best presentation about WAFL and RAID and Snapshots—all the NetApp technologies—and speed and reliability, the same tech-laden pitch that worked wonders with engineers. When I finished, the IT manager leaned forward and talked slowly, with a deep southern drawl: “Son. You gotta understand, here at Georgia Pacific, we take trees, and we turn ’em into toilet paper. What you need to explain to me is, exactly how will your box help me to do that?” He was polite about it, but he clearly didn’t care at all about our wonderful technology. My reaction at the time was to wonder how a company could survive without a more inquisitive IT department. (We didn’t get that sale.)

I got another hint when I visited an insurance company in the Midwest. The IT manager told me, “My strategy here is not to hire smart people.” His whole staff was sitting around the table, smiling and nodding. I was stunned! He called his staff idiots, right to their faces, and it didn’t bother them. To me, smart was the ultimate compliment, but he didn’t see it that way. What he was trying to tell me, but I was not yet clever enough to understand, was that he didn’t hire people to understand all the technical details of his data center. His strategy was to buy from vendors who could send in people to handle those details for him. His folks were plenty competent, but their skill was in overseeing the work of technology companies, not in understanding the technology itself. I saw smart as a good thing, but to him it meant something more like troublesome propeller-heads that I’d rather rent than hire. (We didn’t get that sale either.)

Rob Salmon, our then VP of North American sales, described an experience that finally helped me understand what was going on. Rob was going to meet the CIO of one of our first banking customers on the East Coast. Rob checked with our customer support organization before his trip to see if there had been any problems. No failures, just some simple questions, so Rob expected an easy call.

The CIO began the conversation by saying, “Rob, I want you to know that your sales guys really piss me off!” Not what Rob expected, so he asked what we were doing wrong. The CIO said, “Here’s the problem. They come in and they tell me all about your products. They tell me how fast they are, they tell me how reliable they are, how they work, speeds and feeds—more technical detail about your products than I could possibly want to know. And then they sit back and smile, like they’re done.”

“I pay you guys enough money that I want you to figure out my problems. I don’t want to figure out what your products do. You come in and look around, and you tell me how your products will fix my problems. That’s what I want.” In retrospect, that was pretty much what the man from Georgia Pacific was trying to explain. If you listen carefully enough, customers will often tell you how to win their business.

NetApp now has many customers who spend millions of dollars a year with us. The largest ones spend around $100 million a year. I don’t know if you have ever written a check for $100 million—I personally have not—but if I ever do, I can tell you one thing for sure: My expectations will be quite high. This may be the most important lesson of enterprise selling.

Although I started the Love Program in order to improve our products—I was the VP of engineering, after all—it also uncovered many nonproduct issues that helped us with enterprise customers.

An example was the way we handled product failures when we were small. A customer once called late at night with a big problem. An engineer named Varun Mehta, who happened to be working late, took the call and determined that their system needed to be replaced. He couldn’t find anyone from customer service, so he went to the manufacturing floor, pulled a new box off the line, and got on a plane to Los Angeles. He rented a van, drove to the customer’s site, and worked all night to solve the problem. The next morning, the customer called and told us the story, but when we went to look for Varun to thank him, he wasn’t to be found. After hours of searching, we discovered that he had been asleep in the rental van in the customer’s parking lot. This small customer saw Varun as a hero, and I felt the same way. He exemplified our values by going beyond the call of duty to make a customer successful.

Large companies have a very different perspective on failure: Heroics scare them. The difference is that small customers, with one or two systems, will probably never see a failure. If something does break, they are thrilled if we can fix it quickly. Large customers buy so many systems that things will break periodically no matter how high the reliability. Part of our problem with Cisco and Yahoo was that we needed more reliable systems, but we also needed to offer a better failure experience.

Much of the failure experience comes from how people in the company behave. Are there repeatable processes to handle failure, staffed with full-time people who understand how customers think? Heroics scare large customers because heroes come and go. They want to know that you’ll handle the problem the same way, the right way, every time. They certainly don’t want Varun sleeping in a van in their parking lot.

This is a great example of how behaviors need to change even as values stay the same. I want our employees today to work just as hard as Varun did to ensure customer success, but I also hope that no engineer ever again sneaks into manufacturing at midnight and swipes a replacement system. We do it differently now.

Years later I got a lesson on failure experience from a support engineer in our call center. Walking by his cube, I noticed that he had ripped dozens of pages out of the manual and taped them on the wall. I figured he might have some good ideas for improving the product, so I gestured to the wall and asked, “What is most important when a customer calls?” He said, “The most important thing is that they don’t hear me flipping through a manual.”

Semper Wi Fi

We invited customers to a meeting to share their storage experiences, and several military officers came. The military uses a bunch of our equipment, some in Iraq bolted inside Humvees. Combat generates lots of data.

One officer mentioned the smoking pit scenario, and a civilian in the room asked, “What’s that?” The officer explained, “Our mission is to support the warfighter no matter what, even if a data center is bombed and becomes a smoking pit.”

Later a civilian described the work he was doing to create a secure data center, and the same officer said, “People like to think all strategic, but sometimes you’ve got to be more tactical. Like if you’re out in the field, and you need a secure data center, I say put up a tent and two marines in front with .45s. There is your secure data center.”

Our philosophy was to build products with appliance-like simplicity. But was simple always better? There is a tension between a simple appliance and a complex data center. If you make an appliance complicated enough to meet enterprise requirements, is it still an appliance?

I love using thought experiments to test ideas. On a business trip, I noticed a coffee machine in my hotel room, and I started asking myself questions. What if I were Mr. Hilton, and I had hundreds of hotels, each with hundreds of rooms, and each room had a Mr. Coffee? What would my problems be—aside from my granddaughter Paris—and how would I solve them?

With so many Mr. Coffees, there will always be some broken ones. How am I going to find them? I could wait for customers to complain, but that would make them unhappy. What if the Mr. Coffee could detect when it’s broken? Maybe it could send a signal to alert the maintenance people. That’s a good start, but I’ve got a hotel to run, so I don’t really want to be in the Mr. Coffee maintenance business. It’d be best if the Mr. Coffee Company would offer me a service where they detect the signal and they fix the broken machine.

If you look at the coffee maker itself, this updated Mr. Coffee is obviously much more complex than the original. From Mr. Hilton’s perspective, however, the new Mr. Coffee is simpler. It is still an appliance, but it’s an appliance optimized for the enterprise instead of the home.

In some cases, enterprise customers don’t just want you to fix what breaks, but to design, install, and manage the whole solution for them. This was the opposite of our early technical customers who loved learning about the capabilities of a new product. If a feature was interesting enough to catch their attention, then they would experiment until they figured out what it was good for. Better yet, they would often install it themselves, manage it, and even fix it if need be. As a start-up, it was easiest to deal with technical customers. But for success in the broad market, we had to provide a broad range of services. For many years, our professional services organization, which provided these services, was the fastest-growing part of our business.

Our very first storage system was so “simple” that there was no good way to make backups (copies of the data in case something bad happens to the original). It was hard to manage because it didn’t support the standard tools that people use to control network devices. We made the box itself as simple as possible, but ironically, that made things more complex for the system administrator who managed the box. To simplify life for the sysadmin, we needed to make our product more complex. In other words, we initially drew our circle of simplicity around the box itself, but we should have included the sysadmin inside the circle as well.

Oliver Wendell Holmes once said, “I wouldn’t give a fig for the simplicity on this side of complexity; I would give my right arm for the simplicity on the far side of complexity.” I think that his far side of complexity is when you draw the circle of simplicity around everything that matters. Draw too small a circle and your simplicity is naive.

We expanded the circle again when customers asked us to store data for Microsoft Windows. (Our first boxes supported only the UNIX operating system.) We briefly considered building a second appliance just for Windows. That would have been easier for us to implement, and probably easier for system administrators to understand. On the other hand, many of our customers told us they were planning to migrate from UNIX to Windows. They needed a storage system that would let users access their files before, during, and after the migration. UNIX and Windows have fundamentally different models of file access, so creating a single system that would let them share data seamlessly was extremely difficult for us—but it sure did simplify life for our customers. They loved it.

To satisfy large enterprise customers you must draw a very large circle of simplicity. They have dozens of data centers scattered across multiple continents, and they have hardware and software from many different vendors. This is the environment they are trying to simplify. Such environments are so complex that simplicity is unachievable. On the other hand, I believe that we can make them simpler than they are. Not simple, but simpler. That is our goal.

Enterprise customers demand that their vendors work together. At first I didn’t understand. Engineers look at what should work, and they often think in terms of industry standards. For instance, an engineer might say, “The NetApp storage speaks NFS over TCP/IP, two standard protocols, and Oracle lives on top of Sun Solaris, which handles those protocols, so that should work fine. And SAP lives on Oracle, which lives on Sun, which handles NFS, which talks to the storage. What could go wrong?” (Don’t worry—you aren’t expected to understand that. When I said it to customers, most of them didn’t either.)

Businesspeople think differently. They know that eventually something will break, and when it does, they worry that a bigger vendor will point at a smaller vendor and say, “That configuration isn’t certified. Call us back after you’ve replaced the storage from that company we’ve never heard of.” At that point, it doesn’t matter at all what should work. That’s why they insist that all the vendors involved in a solution shake hands and promise, “If something goes wrong, we will fix it together.” At first, this insistence on partnerships and certifications frustrated me, especially since we didn’t have them. In hindsight, they were right and I was wrong. Eventually things always do break, and it takes cooperating vendors to fix them.

Our experience with Oracle shows how these partnerships can evolve. When customers first started using NAS storage with Oracle databases, in the mid-nineties, both NetApp and Oracle recommended against it. Back then, even I didn’t think it made sense to run high-end applications like that over a network. The customers who did it were so happy, however, that we approached Oracle to ask if they would support it. Oracle resisted for years, but—ironically—their engineering department started using NetApp internally. The external message was that Oracle won’t work with NAS, but the reality was that the programmers developing Oracle’s software used NetApp NAS every day. I chatted recently with the CTO of VMware, a company much younger than NetApp, and he sees the same pattern: many software vendors publicly state that their application doesn’t work with VMware, even though their programmers use it all the time.

Eventually, Oracle marketing gave in and asked us to sponsor a cost-of-ownership study against our large competitor EMC. They told us which research firm to use and told us to pay for the study. (When the large vendor in a potential partnership asks the small vendor to jump, the correct answer is, “How high, Sir!”) The study found that NetApp NAS cost only a quarter as much to buy and operate as SAN storage from EMC, and that was the beginning of a beautiful relationship. Here’s how Oracle viewed the world: every customer has a certain amount of money to spend on information technology—some on Oracle and the rest on everything else. If Oracle could help customers save money on “everything else,” then hopefully they’d spend more with Oracle. Of course, it was after the dot-com crash that saving money really became important.

A few years ago, Oracle built a data center—one of the largest in the world—to run Oracle applications for their customers, and they chose NetApp NAS for all the storage. Our journey with Oracle symbolizes our entire history with enterprise customers. They went from not trusting us at all to using us internally, and from there to depending on us to provide critical services for their customers.

INTERLUDE

Shark Island—A Parable of Risk and Mass Media