Other than the Ethernet port built into Xbox, the development of Xbox Live was primarily about conceiving a clear, coherent and achievable vision, understanding the technical challenges, and writing the code. Lots of code. It was also about preparing for what Allard, Ferroni and their team believed was the inevitable. The Future.
Xbox Live Purpose Statements
“You’ve got to know why you’re doing what you’re doing, what are the principles you’re going to work by, and then—so many projects don’t do this—we wrote down the metrics of what we want to achieve within five years in order to know that we succeeded.”
-Boyd Multerer
Multerer remembers a document from November 2000 that spelled out the Xbox Live purpose statement. Here are the main points of the purpose statement with commentary from Multerer:
Games Games Games
“And we had to talk about that. Because this is Microsoft, and are we really just about games? Are we going to put any other things in there? Are we building a service that’s about anything other than games? And we said, ‘No. We are just games.’”
Lead. Don’t Follow
“At the time… go back to 2000… there were no consoles that had a service. The closest was the Sega Dreamcast, which had a browser, and that was it. SegaNet wasn’t a really big service. And we clearly decided that we weren’t even going to look at anyone else. We were just going to design our own thing and go.”
It’s an Xbox community.
“We focused in on the word ‘community.’ In today’s parlance, we would say, ‘Oh. We were building a social network.’ Except we didn’t know to call it a social network. And community was the word we homed in on, and we ended up defining a lot of what would become primary characteristics of modern social networks, and that’s something I’m really proud of. You’ve got a list of people you communicate with. We’ve got good communication channels going through it. At the time, the closest things we were looking at was a little bit of Messenger, but we were trying to do something much deeper than just surface level communications.”
Profit generating… gamers are our assets.
“So the most important thing was gamers, but not just gamers… gamers who were going to be buying games. It wasn’t going to be a free service, and that was super controversial. Deciding to charge at all for this service… there were more yelling matches over that decision than almost any other decision. Because, you know, ‘Oh my gosh. On the PC everything’s free, and how can you possibly charge?’ ‘Well, if you charge for it, you can build a better service.’ If you charge for it you can actually invest in servers, especially in 2000. They were wildly expensive. And by setting that precedent, we were able to build a much bigger, more robust, more feature-filled service than we could have otherwise done.”
Be fiscally responsible.
“It was also a place where we could have dumped a huge amount of money. We ended up… at launch we had three data centers that we had built pretty much from scratch. And geographically located around the world. A big one in Seattle, one in London and one in Tokyo. And we had to be careful not to spend too much on it.”
Crawl, Walk, Run
“J and Cam loved that.”
Partner where appropriate
It’s a service, stupid, and it just works
“Those were the main principles that we were trying to live by.”
From these principal statements, they crafted the purpose statement for Box Live:
Build and manage the safe and profitable online gaming service for the global Xbox community and have fun doing it.
“And we debated every single word. And probably one of the ones that had the most debate was the word ‘the’. ‘Build and manage ‘the’. And ‘the’ meant exclusive. It meant, ‘Look. This is going to be the service for the console.’ We’re not going to have confusion. We’re not going to have different logons for different games. You’re going to log onto Xbox Live, and that’s going to be your logon, right? And I’d say that led to probably the two most controversial decisions we made in the industry, which now are no-brainers. They’re like, ‘Well, duh. Of course you do it that way.’”
Gamertags & Presence
The two controversial decisions Multerer is referring to were Gamertags and the concept of Presence. Initially, Multerer says that publishers hated Gamertags. “The reaction was sort of, ‘Wait a minute. What do you mean, you own my customer? They’re my customer. They’re playing my game, and they’re logging into my service, not your service.’” But the XBL team wasn’t having it. They explained that this was something that would benefit everyone. They explained how players could log into Xbox Live and build a reputation that crossed all the games they played… that XBL was about reputation and community, and if you split people up with different logons on a per game or per publisher basis, you fragment that community, and you’re not going to get any scale. “They didn’t like that because they wanted to control their customers. They saw it as a very competitive environment, so they didn’t want their competitor getting any benefit from work that they did.” Microsoft stuck to their guns, though. Gamertags stayed.
The idea of Presence might seem pretty obvious today, but back then the idea that you could log into an online service and see all your friends and what games they were playing (and they could see you and what games you were playing), that you could invite friends into the games you were playing, or they could invite you to theirs, was something new, and to the game publishers, simply a very bad idea. According to Multerer, “Their reaction was, ‘No. Wait a minute. First you said I don’t own my customer. And now you’re telling me that I have to advertise my competitor’s game in my game.’ I’m not kidding. This was basically the reaction.”
These two issues prevented some of the larger game companies from supporting Xbox Live for the first years, but eventually they came to realize that Xbox and Xbox Live weren’t going away, and they were missing out. Some larger companies did embrace the new ideas from the start, and Ubisoft fully implemented them when they released Splinter Cell on Xbox at the same time as the launch of Xbox Live in November 2002. “That was the game that kind of proved the point we were trying to make about Xbox Live. Until then, yeah, it was kind of cute and kind of nice, but Splinter Cell really used it and the game was clearly better because of it. And they built around it, and frankly, it helped make that company.”
One company that had played hardball all along, and continued to do so with XBL was Electronic Arts, and Multerer remembers an amusing anecdote about Don Mattrick, who, at the time they were pitching XBL to EA was president of worldwide studios, and who became president of the Entertainment Business Group at Microsoft in 2010, before leaving again to join Zynga.
A Don Mattrick Story
When Mattrick joined Microsoft, Todd Holmdahl brought him to meet Boyd Multerer. “Holmdahl introduces me to him. ‘Hi Boyd. I’d like you to meet Don Mattrick. And Don Mattrick, meet Boyd.’
He said, ‘Oh, it’s good to meet you.’
And I said, and this is true, ‘We’ve already met Don.’
‘Oh, we have?’
“So it was E3 2002. We haven’t shipped Xbox Live yet. Xbox is out. Xbox Live is not. I was in the booth, and we’re showing off a bunch of games. We had these little phone booths that were set up, and you could go into booths and you could play against people the other booths, just to prove that it worked. And I was manning the booth at the time, and I was up at the little desk at the front, and this guy walks up who I didn’t know, and I read his badge, and it said, ‘Don Mattrick Vice President EA’ And he’s asking a couple of questions about Live, and I’m like, ‘Well hey, it’s really good to meet you Don. I’m looking forward to EA games being on Xbox Live.’ And he says, ‘Ptahuogh’ and walks away. And I thought to myself, ‘Oh yeah? We’ll show you.’
“Anyway, so I told this story to Don… the first thing I said to him when he showed up at Microsoft, and he started laughing.”
Jon Grande, who worked at the Zone and on MMOs like Asheron’s Call, says that the real element of presence in Xbox Live was not fully implemented until the next generation console—Xbox 360. From observation, he knew that a lot of what players did in games was to socialize. It might be to talk smack or compare strategies after a mission, or just to get to know one another.
In MMOs, people cultivated friendships, and even romances. Grande remembers having heated arguments about socialization with people working on Xbox. Based on years of experience with the Zone, Asheron’s Call and Cyber Ace, “we knew that people spent more than half of their time—upwards of two-thirds of their time—socializing outside the context of playing the game.”
He says that there was some real concern that there would be pressure from other parts of Microsoft to add unwanted elements like a browser or forcing interoperability with other technologies that could dilute the purity of what Xbox was meant to be, and even detract from Xbox being taken seriously as a console. So, when Xbox Live initially launched, the idea of presence was only represented by seeing friends and playing in games with them.
Grande says, “It wasn’t really until Live shipped on 360 that it took really a full persona of its own. On one hand I think it was a necessary decision inside the company to avoid having to go back and rehash these arguments about integration with Windows or Internet Explorer or other teams, but I think it was a pretty significant missed opportunity where we could have driven a lot more social interaction in the platform earlier on.”
To Pay or Not to Pay
The decision to make Xbox Live a pay service was definitely controversial. At the time they were planning Xbox Live, people who used the internet had to pay ISPs for the service, but after that, they expected to get most other services for free. It’s still that way today. Gamers would pay for games, and in the early days of MMOs, they were willing to pay monthly subscriptions for the games they wanted to play. But the idea that a console system could come out and introduce the first-ever massively scaled console-based social service—and charge money for it—was anything but obvious to most people, but not to those who were designing the service, like Cam Ferroni, J Allard and Jon Thomason, who had come over to XBL after completing work on the Xbox OS. “Many felt that because PC gaming was free, we should be free,” says Ferroni. “But we believed then—and I think you see it carrying through now—that a high quality, consistent, easy experience, would be something that people would be willing to pay for, and would differentiate the Xbox from anyone else.” Ferroni notes that providing a friend list across all games, a single user login/ID, voice chat in all games, an achievement system (Gamertags) that provided a persistent identity as a gamer, as well as ways to play together were services such as they had never seen before on consoles, and worth a small subscription fee.
Thomason says, “The decision to charge for Xbox Live was one of the best decisions we ever made. The thing we knew, even at the time, was that if we didn’t charge for it from day one, we’d never be able to. And it was absolutely necessary to charge. The infrastructure cost so much that to have the level of service that people expect, you have to charge for it, unless you have some other way to monetize it, like ads or something else. This was before there was really a robust ad market for online stuff.” So they did a lot of math to come up with how much they could charge. Lots of analysis and figuring things out, and in the end, “we just picked a price point because it was the cost of a game. We could explain that.”
Despite the initial resistance from some publishers, Ferroni points out that those who did jump on board early were happy to have the extra sales and engagement, and players loved it. “They loved the fact that they could jump from game to game and find their friends easily—and jump right in. They realized our vision—that being in different cities and playing together could have all of the same excitement and quality of experience as playing next to each other in the arcade.”
Attacks and Cheating
No online service is immune from cyber attacks, and no online game is immune from attempts at cheating. But speaking about hackers and cheating, Xbox Live’s chief engineer Dinarte Morais says, “In terms of the service, I don’t think that was too high on the list for people.” On the other hand, he says that if players were looking to cheat, they would look for ways to modify their game locally. They might, for example, find a way to modify the game so that they always achieved perfect headshots.
Obviously, this kind of cheating, particularly in a competitive environment was unfair to other players and something to be discouraged. “So Xbox Live was putting in abilities to prevent spying or changing packets on the wire, but we also wanted, as much as possible, to keep the Xbox Live service’s reputation high. We never guaranteed that there would be no cheating, but we actively monitored the system and kicked boxes offline so that they can’t play with other people who don’t want to play with those who are potentially cheating. We also didn’t want developers to be the front line of that defense. They have enough work to do to build an excellent game; they shouldn’t need to be online security experts, as well.”
Morais says that they would look for operating systems that had been “modded” because a modded operating system would make it possible to mod the games, as well. “A lot of the data that was on the DVD was checked for authenticity. If you knew that the operating system was the original then you could, by extension, assume the operating system properly checked that the data it was using was authentic. So if you want to, say, edit a text file that controls how a car behaves in a racing game, and the game checked that the game itself was authentic, and the OS checked that the OS itself was authentic, then what you would have to do to break that change is to modify the OS to allow a slightly modified game to run.”
If they detected that the operating system running on your Xbox was not the official OS, they would ban your box. But typically, their integrity checks and attempts to secure the system from cheating started an escalating war between them and those who wanted to hack the system. “We didn’t read the DVD itself. We actually just looked at memory. Like I say, the system was dynamic. From the server side, we could change, at a moment’s notice, what we would look at. I think the original thing that we looked at was a checksum, adding up all the numbers to see if it matched the types of operating systems that should be on the box. If it didn’t, we didn’t let them connect. That was the first step in the escalating war. The counter response was to intercept the request and return the hash that the system was expecting so no one’s the wiser. Then that began ratcheting up the war.
“I implemented the security gateways, the first servers that you connect to when you log into Xbox Live. Like the key exchange that we did between two Xboxes on system link, we also did a key exchange with the Live service, so I wrote that portion of the front end. I also monitored the logs and came up with techniques for seeing how we were doing, if the detection system was returning valid responses, and then figuring out what the other responses should be, based on the results. The goal was to keep the Xbox Live service cheat free and with as high a reputation as possible. After we launched Xbox Live, I continued to work on that for another year.”
Directory Services
Multerer explains some additional challenges they faced, and perhaps some reasons why it was not ready to launch with Xbox. To begin with, they had to build an internet-scale directory service. The existing technology, Active Directory, wasn’t, in his words, “going to cut it at the time.” So they had to build something from scratch. “When I say directory service,” he says, “a lot of people would cringe because it’s so enterprise-y and kinda nerdy, right? But at the fundamental core of what Xbox Live is, it’s a place that hooks you up with other people to play games, and then furthermore, verify that they’re not cheating.”
It’s actually easy to create something “chintzy” that won’t scale to the level required, but it’s very difficult to create something that scales into the millions. And it’s not just a simple question of design. It gets more difficult when you consider the security issues. “Kerberos is a wonderful protocol that people have been using for a long time. Kerberos is the core standard authentication mechanism that you use every time you log into a Windows machine, and to most UNIX machines. It’s a great protocol. And it assumes that there is one user using one machine. I’ve got four people logged into one machine. You don’t mess with security protocols lightly. So ok, I’ve got four unique users that are using one machine and that needs to be expressed in Kerberos somehow, and have that plumb through the system in a way that you can’t DOS (Denial of Service) and take down right away. As soon as you start thinking about attackers, and we know that people wanted to bring the system down, you have to be really careful about what are your weaknesses, what are the bottlenecks in the system, how can you survive a DOS attack.” Multerer further points out that challenges such as these were new back then. Today, it wouldn’t take so long to design and implement a system like that. But back then, it did take time to do it and do it right.
Designing the directory services, he says, included matchmaking, leader-boards, a lot of value-added services, messaging systems, invitations systems, user interface design. In fact, he says that the high level vision of the project didn’t fully solidify until the end of 2000. “The vision starts to move into strategy.”
More and deeper technology information can be found in the Online Appendix under the headings, “Xbox Live Technology” and “A Technical Challenge”.