12

Coinbase Crackup

Nathalie’s finger hovered over the “send” button. As Coinbase’s longtime HR maven, she had written the email weeks earlier, and desperately hoped it would stay in her draft folder forever. But the bomb threat Coinbase had received that morning was more chilling and more credible than any of the previous ones. She stared at the ominous all-caps email telling the entire staff to flee the building but stay calm. Should she hit send? She had to decide.

As she sat at her desk in Coinbase’s spacious open office high above San Francisco, Nathalie wondered just when the company had changed. Since joining the company as chief of staff at the ramshackle Bluxome Street apartment three years ago, she had risen to director and was on her way to becoming a vice president. The title was good, and the money was better. Yet she missed the early days when Coinbase felt less corporate and she could lead activities—hot-tub parties in Napa or fire-spinning classes in the city—with Brian and Olaf and a small crew who felt like family. Security mattered less back then, too. On Bluxome Street, you had to deal with the odd kook knocking at the door. Now, Coinbase was hiring former FBI agents and drafting emails to deal with emergency evacuations.

Nathalie wasn’t the only one on edge. Philip Martin, the security director, saw it as his job to be paranoid, and these days that wasn’t hard. “We had these weird packages that kept arriving at our P.O. box,” he recalls. Meanwhile, the bomb threats and other violent messages became near-weekly events. One recent incident had caused a squad of SFPD officers to swarm Market Street outside the Coinbase building. It turned out to be a false alarm but only added to the growing sense of unease within the company.

In response to this latest threat, Nathalie conferred again with the security team. She moved the email back to her drafts folder. A bomb threat was a real risk, but so was spreading panic through the workplace. She prayed she had made the right decision.

There were other things to worry about too. Mike Lempres, the company’s political fixer and a longtime Justice Department veteran, worried what would happen if organized crime set its sights on Coinbase. Control Risks, a security consulting firm, had recorded an average of two crypto-related kidnappings every quarter, with criminals choosing targets based on public reports of their wealth. “These guys’ ignorance is an issue,” says Lempres. “They think if they kidnap Brian, he’ll give them bitcoin. Silicon Valley is really ill-suited to deal with old-school thugs like the Russian or Italian mafias.”

Martin also worried that the growing publicity surrounding bitcoin, and therefore Coinbase, would attract crooks plotting physical robbery. That’s one reason why, by 2017, Brian and other top crypto executives rarely appeared in public without a retinue of bodyguards. They also became well versed in emergency tactics, such as using code words in the case of kidnapping or violence.

On top of these security concerns, the company faced threats from its own customers. The bull market of 2017 had put a strain on Coinbase’s capacities, leading to technical meltdowns like the June flash crash and to a growing backlog of support tickets. Customers seethed in emails and especially on online forums like Reddit, accusing the company of conspiratorial plots to steal their crypto. That wasn’t the case, of course. Coinbase was simply swamped and couldn’t keep up with the massive increase in transaction volume and flood of new customers. Like a brave dog paddling against a too-powerful current, Coinbase employees worked nights and weekends to keep the site running and clear the backlog. But the chaos unleashed by crypto mania just kept growing. And then came December.

• • •

On New Year’s Day 2017, investors had cheered bitcoin’s long-awaited return to $1,000. Eleven months later, the currency smashed through the $10,000 mark. Some of Wall Street’s finest thinkers offered elegant technical explanations for the incredible gain. Quants at Goldman Sachs employed something called the Elliott Wave Theory to suggest that the run-up represented an “impulse wave pattern” typical of mass market psychology. A financial technician named J. C. Parets said the upswing mirrored the Fibonacci sequence, a famous mathematical pattern that occurs in seashells, pine cones, and other elements of the natural world. Others called it a speculative mania. Or simply a bubble.

A week into December, bitcoin broke $16,000, fueled in part by massive trading on exchanges in Seoul and Tokyo. In the United States, taxi drivers and personal trainers joined hedge funds and day traders in bidding up the price still higher.

The frenzy also fueled an eye-popping run on Ethereum, which brushed $1,000 by December, and XRP, which had begun the year worth half a penny and now sold for $3. Altcoins, shitcoins, anything plausibly related to blockchain soared in value. As for Litecoin, Charlie Lee’s creation had popped after it was listed on Coinbase that summer, and by mid-December it cracked $350—up from $4 at the start of the year. In a moment of exquisite timing, Lee sold off his entire hoard right near the all-time high, reaping $20 million from his invention.

As prices climbed and climbed, Brian published a blog post in early December titled “Please Invest Responsibly” that dryly warned customers of the volatility associated with cryptocurrency investing. The market paid no heed whatsoever. Prices kept climbing.

Brian’s call for responsible crypto investing wasn’t just ineffective—it was hypocritical. Coinbase, after all, offered a service that made irresponsible investing easy: buying crypto with credit cards. While it was rash to invest in a market that screamed “bubble,” it was downright reckless to put the purchases on a Visa or Mastercard. Brian may have been worried about what he was witnessing, but he wasn’t above collecting a 4 percent service charge to people paying for their investments on high-interest plastic. JPMorgan Chase, Bank of America, and others that issued the credit cards became alarmed and would within weeks ban their use for crypto purchases—a sure sign that many buying crypto on credit were in tight financial straits.

The December insanity spurred other unintended and absurd consequences. The long-running civil war over bitcoin’s block size had never been resolved, meaning only one megabyte’s worth of transactions could be stuffed into every block, and a new block could still only be added to the blockchain every ten minutes. Now, as the number of users on the bitcoin network swelled by millions every week, what had been a minor nuisance swelled into an epic backlog. Take any choke point that’s already heavily congested—say New York City’s Lincoln Tunnel or the 405 in Los Angeles—and then add fifty times the traffic. That’s what happened to bitcoin’s blockchain. Its network ground to a virtual halt. This meant the only way to ensure a transaction made it onto the blockchain within a reasonable amount of time was to pay the bitcoin miners who maintained the ledger. With a captive, desperate customer base, those miners began demanding colossal premiums. Run-of-the-mill transactions became staggeringly expensive. On December 8, for example, a man named Kristian Freeman tweeted in dismay that sending $25 in bitcoin to a friend had triggered a $16 fee. Forty percent of his $41 transaction went to a service charge. Sure, a bitcoin user could refuse and offer a lowball fee. But this would mean waiting days for a transaction to clear—if it cleared at all.

• • •

Paradoxically, this moment of bitcoin’s biggest success, when it burst into the mainstream like never before, also showcased its biggest failure. Satoshi’s vision had promised a new and democratic form of internet-based money that could be used with few fees or constraints. The reality of bitcoin in December 2017, however, was a bloated and dysfunctional network that made Western Union wire transfers look cheap and efficient. Underscoring how impractical bitcoin had become, a major crypto conference in Miami that December declared it would not accept bitcoin as payment for entry.

By the time bitcoin crossed $15,000 in early December, the network was hopelessly clogged and transfer fees were astronomical. These facts did nothing to quell demand. The price continued to rise as much as $1,000 a day as frantic speculators bought more and more bitcoin. Everyone wanted to cash in, including companies that had nothing to do with crypto. An obscure beverage company called Long Island Iced Tea Corp. changed its name to the Long Blockchain Corp. The pivot boosted its share price 200 percent—and later, it brought an insider trading investigation from the SEC and a delisting from NASDAQ.

On December 17, the currency brushed against the unfathomable high point of $20,000. A single bitcoin was now worth the same as a pound of gold. On CNBC, the network had given over half of its airtime to the mania, sweeping aside staid coverage of stocks and bonds in favor of bringing on crypto pundits who of course predicted bitcoin would soar even higher.

• • •

Half the world, it seemed, was getting into crypto. And for many of those people, their first stop was Coinbase. In February 2014, the company counted one million customers and now, just under four years later, it had twenty million. On most days that December, more than one hundred thousand people signed up for their first Coinbase wallet.

Inside the company’s Market Street headquarters, Adam White recalls employees high-fiving as Coinbase notched a day with $4 billion worth of transactions. They whooped over reported daily revenue numbers. Meanwhile, Coinbase became the most downloaded app in iPhone’s App Store, a moment that was particularly sweet given how, not so long ago, Apple had turfed the company out of its App Store for offering crypto trading. Now Coinbase was more popular than Facebook or Twitter.

Coinbase was making waves with venture capitalists. It was also making money—lots of it. The company was processing millions of transactions of bitcoin, Ethereum, and Litecoin, and it took a cut of each one. The company’s margins were huge. While Coinbase had to spend a lot on engineers, the actual cost of performing a transaction—moving digital dust in and out of customers’ wallets—was almost zero. When a customer bought $100 of bitcoin, Coinbase could charge $2.99, and that was effectively pure profit.

“The first time I met him, Brian said, ‘I want to build a billion-dollar business,’” recalls Katie Haun, the former prosecutor turned Stanford crypto professor, who had recently joined the Coinbase board. Now, Brian had achieved that goal. The December surge in sign-ups meant Coinbase would book more than $1 billion in revenue in 2017 while, months earlier, it had taken its place as a unicorn—a startup valued at more than a billion dollars. And Coinbase wasn’t just any unicorn—a leak from board member Barry Schuler months later revealed it was worth $8 billion, making it one of the ten most valuable startups in the country. What Uber was to ride-hailing and what Airbnb was to home rentals, Coinbase was to crypto.

For Brian, all of this was vindication for the open secret he’d seized on at Y Combinator six years before. He had recognized that many more people would buy bitcoin if given an easy way to do it, and the success of Coinbase had proved him right. And now he had also realized a long-burning ambition—an ambition that had inflamed the tech visionaries who put their stamp on his hometown of San Jose and the famous valley that stretches north of it.

As the price of bitcoin reached its all-time high, Brian’s company had become a money-printing machine. The machine, however, had been overheating for a while. And amid the massive influx of customers during December, the money machine threatened to explode and take out Coinbase with it.

• • •

“We were all good software engineers, but none of us knew infrastructure,” says Coinbase’s second employee, Craig Hammell, explaining how the company had been built the Silicon Valley way—quickly and with whatever tools could help it add customers in a hurry. These included tools startups know well, like MongoDB to manage data and Heroku for apps. Such tools are fine for scaling a startup but not for processing millions of sensitive transactions. Coinbase was using West Coast coding to do the crypto equivalent of East Coast banking. Scaling up a dating app is one thing. Managing millions of people’s money is another. “This type of engineering was tough stuff. Things like MongoDB were OK for prototyping but not for a major financial operation,” says Charlie Lee.

In building Coinbase, it was as if Brian and the other engineers had constructed a finely designed California beach house and then plunked it down on the coast of Maine during a nor’easter. That house wouldn’t survive a battering from the howling wind and snow. The owners would rue the fact they hadn’t used better building materials as the house shook and creaked and, eventually, cracked. This was the state of Coinbase’s website in December 2017. Juan Suarez, the company’s longtime lawyer, recalls flying to Pittsburgh for a family Christmas visit only to land and receive an urgent message from Brian to turn around immediately: “The feeling was like, ‘Oh shit.’ It’s like we were on this bluff overlooking the ocean all by ourselves and all the crosswinds in the world were bearing down on us.”

Prior to December, customer trades had been getting delayed as parts of the website began to buckle. The influx of millions of new users in the days before Christmas, however, crashed the site, and it stayed down for hours at a time. Client orders ended up in technical purgatory. Angry users fumed on Reddit and Twitter.

Banking partners who took their deposits contributed to some of Coinbase’s technical snafus. Its biggest European partner, an Estonian bank called LHV, did not use APIs (application programming interfaces)—a standard way for computers to communicate with each other—but instead required Coinbase to manually upload transactions in a spreadsheet. Coinbase engineers wrote scripts to automatically fill the spreadsheets, only to discover they could upload just fifty items at a time. It was like trying to do calculus on an abacus.

Wherever the fault lay, disgruntled customers and ideological foes took out their frustrations on Coinbase, directing waves of malicious traffic at its website to knock it offline.

Underscoring the technical misery was a leadership vacuum. Even as the company’s business volume more than quadrupled, its executive ranks thinned. Fred’s leaving in January deprived Coinbase of the military-style efficiency he had helped create in its early days, while Olaf’s departure meant the loss of its chief internal diplomat. Olaf had the unusual quality—especially in ego-driven Silicon Valley—of being liked by absolutely everyone with whom he worked, a quality that made him invaluable in spotting problems and smoothing over office conflicts. By November, as trading volumes began to surge, Brian had only two longtime executives—Adam White and the general manager of Coinbase’s consumer division, Dan Romero—to help him avert a total meltdown. The lack of an executive suite, and the company’s mounting customer service problems, did not go unnoticed by Coinbase’s board, who began receiving emails from friends complaining about the disorder at the company.

Chris Dixon and other members of the board came up with what they hoped was a solution. Just as the eccentric Google founders had once needed what Silicon Valley calls “adult supervision”—and received it in the form of a veteran CEO, Eric Schmidt—the Coinbase board sent help in the form of Asiff Hirji, a veteran banking and telecom executive who had since taken a post at VC titan Andreessen Horowitz.

Hirji arrived in November during the speculative mania and just before the December insanity as Coinbase’s first chief operating officer. It was like starting a new job in the midst of a five-alarm fire. “They couldn’t cope with the scale,” he says. “The company had grown 40 times over that year. We didn’t know how much cash we had on hand, plus or minus $200 million, which is ridiculous.”

Hirji was also aghast at Coinbase’s financial infrastructure. He had seen his share of firms blow up as a result of relying on unconventional trading software and, when he first looked under the hood of Coinbase, he feared it would suffer the same fate. But in the trading frenzy of December, any major fixes would have to wait—it would be like trying to swap out a fighter jet’s engine midair. All Coinbase could do was hold on and hope.

• • •

A growing number of customers, however, were out of patience. More and more orders were getting delayed or vanishing—a maddening experience when the price of bitcoin was swinging by thousands of dollars in a day. Customers were stuck wondering if their order had been fulfilled at the rate of $16,000 or $19,000 per bitcoin, or if it had been fulfilled at all. Anger bred conspiracy theories; some believed Coinbase’s technical problems were just a ruse to steal their money—and they vowed revenge for this imagined crime. “We had these people saying, ‘We’re going to come to your office and blow it up and shoot everyone,’” Nathalie recalls.

Linda Xie, a longtime Coinbase product manager, shudders when she recalls anti-Coinbase zealots on Reddit posting sinister photos of employees and the company’s office. Their fury would boil over not only online but also on the streets of San Francisco. Soon, she stopped disclosing her identity at crypto meetups in the city, fed up with being accosted by random strangers angry at Coinbase. It took a toll.

“I was surprised how much it affected the leadership team,” says Linda. It made me realize how human people were—they were reading all the Reddit comments and taking it to heart, being saddened by what they read. People thought Coinbase was a black hole and that no one was listening.”

Coinbase’s ham-handed approach to customer service didn’t help. Even in the early days, the company had not been adept at addressing complaints—its first approach had involved Olaf fending off thousands of emails by writing an automated program in which a fictitious person, Roger, would respond to them. Years later, when Brian finally recognized the company needed a director of customer support, he didn’t turn to a newly minted MBA or a retail veteran. He turned to Reddit, using an online quiz to recruit candidates to fill the position.

The new director was, in Nathalie’s words, “a lovely human who likes to be introverted.” Hardly ideal for customer service. Additional members of the customer support team were added primarily because they loved bitcoin—a suitable quality for an online crypto company, of course, but not a quality necessarily helpful in calming legions of furious customers.

Typically, when situations reach crisis levels like this, Valley startups turn to PR firms that specialize in both the tech world and crisis communications to help fix, or at least contain, the problem. Brian, however, found media relations to be a waste of time and energy, preferring to focus on engineering matters. But as media demands became more insistent during 2017, he deputized a Coinbase engineer, David Farmer, to deal with it. Farmer, who knew nothing about communications, loathed the task and could only grit his teeth in annoyance as an avalanche of reporters’ emails demanded to know why the company appeared to be melting down.

On December 19, Bitcoin Cash—the bitcoin spin-off with the bigger blocks—became available for Coinbase customers to buy and sell. The new offering went haywire almost instantly as prices unexpectedly soared, in part because of a flood of “buy” orders. It turns out that many customers had ordered buys to be made no matter what the price was when it became available. Only four hours after launching Bitcoin Cash, Coinbase had to halt trading to sort out the mess—creating even more angry customers. Others had noticed an unusual price spike in Bitcoin Cash that had occurred hours before Coinbase announced trading for the currency. It was easy for conspiracy-minded fanatics to connect the dots. At the least, the activity looked suspicious and fanned further rage on social media. A typical tweet:

I don’t care how you slice it, this is INSIDER TRADING! Someone with alot of bitcoin knew @coinbase would add bitcoin Cash BCH and took one BIG chunk of profit from [it]. Whoever you are you are your making crypto look like Wall Street. Shame on you.

In response to the uproar, Brian declared Coinbase had a zero-tolerance policy for trading on inside information and announced an internal investigation. The company would find no evidence of wrongdoing. But behind the scenes, Coinbase executives quietly vaporized a channel on Slack—the company’s internal messaging system—called “Trading Strategies,” in which employees swapped ideas about how to make money from crypto trading.

In 2018, angry shareholders filed a class action lawsuit over the Bitcoin Cash debacle. The following year, a federal judge would declare Coinbase should stand trial for negligence.

• • •

By December 31, just two weeks after bitcoin touched the sun at $20,000, all cryptocurrencies were in a flat spin. Bitcoin had plunged 35 percent from its all-time high, and most expected it to continue its corrective plummet. A drop in trading volume had eased the problem of exorbitant transfer fees, though transactions remained slow and expensive. And Coinbase was still drowning in a morass of technical problems and customer rage.

As a short, gloomy day in San Francisco drew to a close, Brian made a new attempt to address the crises that had battered Coinbase. He went to the place he knew and where he felt at home: Reddit. His post began: “Coinbase CEO here—our support is very backed up. . . . Someone will respond to your support request, although it may take some time. Your coins are not ‘lost.’ Apologies for the delay, it is definitely not the experience we want to be providing to our customers.”