GAME OF X V.2

Even though there were good reasons to believe that the Talisman project would come to market too late to be competitive by it’s projected launch in May 1997, Bill Gates was still considering its viability, just as he had back in his previous email exchanges with St. John in February. On May 13, 1997, after reading a summary document about Talisman from Jim Allchin, he wrote a long email in which he states “I understand Talisman better today than before. I still believe it is a very important technology.”

The email was sent to Jay Torborg, but cc’d to a long list of highly placed individuals within Microsoft who might have some interest in the subject matter: Deborah Black, Moshe Dunie, Jim Allchin, Paul Maritz, Jim Veres, Nathan Myhrvold, Aaron Contorer, Mike Abrash, Carl Stork, Ed Fries, Robbie Bach, Alex St. John, John Ludwig, Todd Nielsen, Craig Mundie, Moshe Lichtman, Jim Kajiya, and Alvy Ray Smith.

However, Gates also expressed some concern, saying that there was “more uncertainty about Talisman’s role than ever before” because of what was going on in the industry with initiatives by Intel and the growth of PC graphics hardware. He continued, discussing the probability that SGI would soon introduce Intel based workstations. He saw this as a good thing because they would then endorse Windows NT, but also stated that Microsoft would offer them any of the extensions that Microsoft makes to their graphics APIs, “but we don’t want to give them away to Nintendo.” He also observed that SGI could ultimately be an ally or an enemy with regards to graphics APIs. Gates ended his email with a list of questions, some of them quite technical, and another list of action items, beginning with what amounted to an invitation for St. John to jump into the conversation:

“a. Make sure we are making the right choice by starting with the OGL code. We need to listen carefully to what Alex and the game developers are saying to make sure we can succeed with this approach.”

Gates expressed several concerns, including the report’s recommendation that they use OpenGL.

“It’s very disappointing to me to have to start with the OGL code since its based on a standard we do not control but I do understand why this is the recommended choice and that we can move it in that direction.”

And that’s where St. John jumped in, and his reply the next day at 6:44am was typically blunt.

“This is kind of screwy stuff you guys. I understand that you have a lot invested in this Talisman thing, but the strategy is bad. You’re going to get that special Microsoft Sound System kind of win out of this with a touch of At-Work and a dash of NT MIPS platform for all this effort. I’d really like to be able to help you here, but everybody seems to have a bad case of cotton ears. You don’t need to make a chip, or change the entire industries 3D paradigm to get the results you want.”

St. John then pointed out that support for D3D was growing, with six chips in development “specifically designed to accelerate our internal D3D data structures,” referring to both Intel and NVidia and pointing out that there’s no need for a “monolithic chip.” He writes, “We’re already on a winning course if we just recognize it.”

In the email, St. John suggested that Microsoft’s winning strategy is not in high level APIs or 3D hardware, but in the driver, “because the driver is the lowest level at which we can have control while staying cleanly and snuggly in the OS business.”

Having delivered that message, St. John continued, tackling the OpenGL vs. Direct3D issue and pointing out that “when it comes to multimedia 3D applications precious few of them will ever use OUR software rendering code base for anything.” Differentiating D3D from OGL, he says that he sees the former as a driver architecture for consumer hardware and the latter as an API that doesn’t come with a driver model, writing:

“I don’t care what code base the VESTIGIAL software emulation layer that no real-time application is likely to use comes from. It doesn’t matter, it’s not going to be used. I care that the driver architecture which is hugely problematic even after 2 years of work, evangelism, developer support, meltdowns, may get discarded casually in favor of a new one (MCD), and we’ll have to start all over again with a brand new 3D driver mess, because nobody understood that the issue had nothing to do with the high level API, or software rendering code, and therefore can’t be solved by changing it.”

He recommended that Direct3D Immediate Mode should be the consumer 3D driver model with OpenGL mounted on top of it, ending with,

“If you want to chuck D3D’s API look, and rendering code yeeehaw! Just don’t chuck the investment and leverage you’ve acquired with the driver architecture. Extend it, mutate it, but don’t lose it. You’ve already won if you just stick with it.”

Shifting focus, St. John addressed on the next “big fat hairy scary multimedia” issue, JavaMedia. He wrote,

“Trying to get anybody outside the game/multimedia community to add multimedia functionality to their serious productivity applications is like trying to lift a bowling ball with tweezers,”

and supports his assertions with a long list of reasons why JavaMedia was nothing to be concerned with.

At this point, St. John used the idea of “fat multimedia” to mention the secret project that Engstrom and Eisler were working on. He wrote, “Delivering “Fat Multimedia” is what the Chrome browser project is about.”

“• Kill Talisman asap, it’s a really bad idea, or have another strategy fast.

• You can have the 3d control you want without the risk or the dramatic change in course.

• It doesn’t matter what our software API for 3D is going to be, nobody that matters in consumer MM is going to use it. Just don’t screw up the drivers any more than they already are.

Let’s make them follow us somewhere for a change by having our own ideas about bringing interactive content to the net.

• Focus on your MM drivers, and driver architectures, they are the way to success.

• I’m a major reason you have any multimedia ISV’s to mess with. I hope you’re listening.

All of this is kind of moot if our next ‘Consumer OS’ is a great walrus of a beast that can’t guarantee ~20ms latencies to 1 real-time application in the foreground, and cough over control of memory resources. This functionality should be a ‘Feature’.”

Even though the initial thread that Gates had started was somewhat focused on Talisman, he was laying out some of Microsoft’s strategy in the 3D arena, which included considerations of using OpenGL as a base for a standard API. In response, John Ludwig wrote to Torborg asking for details about the OGL discussion that Gates’ referenced.

In his reply to Ludiwg, Torborg also attached two memos, one written by him and another written by Mike Abrash. (Abrash’s memo was written on April 4, 1997, which prompted St. John to call it a “smoking gun” that showed Abrash’s involvement in the OpenGL vs. Direct3D battles even while he was still at id, despite his assertions that he was not in any way concerned with the issue during his time there.)

“D3D is missing some key features which are necessary to replace OGL in the broader market—specifically printing and metafile support, and software fail-over.

“We evaluated the possibility of enhancing the D3D code base to add these features, as well as starting with the OGL code base and modifying to support better DirectX integration and D3D features. Frankly, D3D has no significant technical advantages over OGL, and OGL is more robust, better documented, and has the broad market features we need. The key D3D strength is driver support and our market momentum.”

Reading Abrash’s memo, it seems likely that it was the source of Torborg’s conclusions, or at least that he and Abrash had been communicating given that their assessments of OpenGL’s strengths and advantages and Direct3D’s weaknesses and deficiencies were identical. Abrash further states his belief that there should be one API to support games and consumer products, and high-end workstations, and he asserts that none of the software developers for the high end would be likely to switch to D3D, even if it were brought up to the same standards as OGL. He acknowledges that Microsoft is better off with a proprietary technology that they own and control over one that is an open standard that might not improve or innovate as quickly. “To put it bluntly, D3D has no compelling technical advantages and many disadvantages relative to OpenGL. If the both APIs were Microsoft-proprietary and a choice had to be made between the two, it would be no contest.”

He summarizes by writing, “…we have an API that does what we need but isn’t proprietary, and one that doesn’t do what we need but is proprietary. We can either spend a modest amount of time and resources making OpenGL proprietary, or a huge amount of time and resources making D3D adequate. Either way, we get a single proprietary API, but at very different costs.”

Torborg’s message prompted Ludwig to write privately to Brad Silverberg and Eric Engstrom the following:

“per your questions, yes they are dumping d3d, driver model, and api. I assume you knew all this eric.”

Meanwhile, the Talisman issue remained front and center for Alex St. John. On May 18th, St. John wrote and shared a document entitled, “The case for putting Talisman out of its misery.” Responding directly to Torborg’s “DirectX and Talisman Update” and attached documents, St. John spent 16 pages taking Talisman apart. At the top of the document, he summarizes some of his main considerations:

1. “The Talisman strategy takes very poor consideration of how consumer multimedia HW is made, adopted, or used by real titles. The entire strategy is painfully naïve about what it’s really like out there.

2. Talisman may be a very powerful cure for the disease that ails the industry, but the delivery mechanism is a coconut sized pill studded with razor blades. It will do more damage than it will cure.

3. Stable robust driver technology is the basis for all multimedia HW technology adoption and innovation. Talisman distracts our focus away from our most critical leverage point.

4. We’ve already blown our image and momentum to the market, tipped our hand to competitors, and positioned ourselves to disappointingly under deliver without having even shipped a beta technology.

5. Mutant Talisman designs made by the worst 3D chip makers in the industry will fundamentally damage our developer community and platform, not help it.”

Never known for an abundance of tact, St. John still could be relied upon to introduce some creative bon mots into his messages. Here are a couple of examples from his Talisman takedown:

“Let’s start with an explanation of the game business written by somebody who probably actually knows very little about it.”

“There’s a lot in these documents that suggests that the people who wrote them don’t actually have a much better idea of how games really work than the Talisman people do. When I see statements like the following in these reports it makes me feel like I might be witnessing a bunch of baboons speculating about how a toaster might work…” (The statement in question ignorantly seemed to presume that developers didn’t know how to make 3D applications, and that they would have to learn how if they were to adopt Talisman technology—and, in addition, presumed that the Talisman SDK would somehow fix the issue.)

On May 19th, Eric Engstrom also replied to Gates’ email with a very lengthy and detailed message, starting off by saying,

“Executive Summary: Adopt Mike Abrash’s proposal with the following caveat, kill MCD and use Direct3D IM as the blessed MCD (mini client driver) model for Direct3D Pro and OpenGL.”

Even though the email continued for several pages worth of explanation and detail, there was one statement, almost buried among the other statements, that changed the game.

While St. John had been fighting the internal battles, Eisler and Engstrom had been secretly working to hook D3D into Internet Explorer, just as they had previously hooked DirectX into Windows 95.

IE4 is now dependent upon D3D, and it is our plan that in the IE5 timeframe we incorporate even more mm functionality into IE -- that is build a greater dependency on D3D and its successors… the IE code-base is a longlived codebase unlike some games… a decision to change at a fundamental level our 3d api will have important engineering implications for IE.”

Engstrom did suggest a compromise plan that adopts much of what Abrash had proposed and supports that plan in considerable detail. He then ended his message commenting on John Carmack’s decision to back other software over DirectX saying,

“Should we listen to John’s complaints and try to address them, absolutely. Should we let him guide our multimedia strategy? I don’t think so.”

The significance of this email was huge for St. John in particular. It meant that he and his partners in disruption had won the battle for D3D dominance. OpenGL would continue to exist, but the only remnants of Talisman would soon be technical elements that ultimately got incorporated into D3D. Once again, the trio of Engstrom, Eisler, and St. John had outmaneuvered their internal rivals and prevailed.

“Even the compromise was engineered,” says St. John. “Like the Bunny-Gate maneuver, my job was to completely polarize the issue so Eric would sound reasonable and balanced proposing a compromise approach that I could then stun everybody by “grudgingly” approving of. In truth, getting the OGL guys to support D3D drivers with the OGL API is all we wanted all along. I just had to get them way out over their skis in order for Eric to tip them over.”

The irony about Talisman was that it never existed as anything more than a hardware specification that was NEVER BUILT. There was nothing to defeat other than the claims of Microsoft’s famous researchers about its theoretical performance if anybody ever built hardware that did it. We didn’t know about it when D3D was started and it defeated itself by never existing.

There were some test prototypes of the Talisman hardware, which, according to St. John, consisted of 5 custom ASICS (Application-Specific Integrated Circuits). Microsoft engineer Mikey Wetzel remembers his first and only look at Talisman in operation. “A handful of us, maybe three or four of us were in a room and the hardware guys brought it in and, ‘Well, before we officially kill it, thought you might want to see it.’”

The writing was on the wall, as one-by-one the outside partners who had signed up to implement Talisman dropped out. It had become clear that advances in what Wetzel called “brute force method, just raw polygons and things like that,” and the fact that many of the graphics card manufacturers were developing their own patented technologies, spelled the end for Talisman. However clever Talisman technology had been in 1994, it was not competitive in 1997. On the hardware side, it was already obsolete, while on the software side, the Microsoft evangelists, led by St. John, had found nobody in the game industry interested in developing for Talisman.

Even though the Talisman project ultimately died out, many of its engineers continued to work on 3D technology at Microsoft. St. John says, “Some of the engineers who came with the Talisman group like Chaz Boyd were really brilliant. They were a little caught in the middle over the Talisman nonsense but once it died they went on to do some great stuff for Direct3D and became very responsive to the developer community.” However, he does clarify that DirectX and Direct3D were successful before the Talisman developers came on. “Historically because it was such a battle and Talisman lost, I think they have tried to write themselves into the DirectX history book by pretending that it wasn’t really successful until they came along. That story is a HUGE stretch, but I can confirm that they and the former OpenGL guys like Otto Berkes who came to work on Direct3D did great stuff with it in their time.”

Colin McCartney, one of the original Rendermorphics engineers who would eventually play an important role in the early adoption stage of Xbox, was aware of Talisman and its battles with DirectX. At the time, he completely backed the direction taken by DirectX, and believed that Talisman was not the most practical solution at the time it was first promoted publicly. However today, he sees the project somewhat differently. “Talisman was being sold because they thought traditional approaches to 3D graphics were going to hit a wall, and couldn’t go further than that. They needed something better. From the D3D people, there was quite a bit of skepticism that wall was going to show up any time soon, and to be honest, I think that’s proved accurate. My opinion on the Talisman stuff has softened over the years, though. Microsoft Research continues to do interesting work on image-based rendering, and now I think one of the pieces of the Talisman stuff might have been just ahead of its time.”

Talisman was only one of many battles being waged in the halls of Microsoft in the late 1990s. “The real battles were not about 3D API’s,” says St. John, “they were about power and who controlled the API’s AFTER DirectX had succeeded! I was battling for control and to force these Microsoft platform groups to be accountable to developers… all the rest was just the shrapnel in the air from all the fighting. I was making it impossible for Microsoft platform groups to hide in their safe offices, vest their stock options and play all day at making technology with zero accountability to the thousands of external developers who were depending on us for their livelihoods, which these guys really hated. Actually making commercially usable technology is much harder, boring and thankless than making ‘cool’ technology with your friends and getting rich while Microsoft’s stock soars.”

On the next page is a Talisman postmortem by Jim Kajiya from Microsoft Research.

Jim Kajiya on Why Talisman Failed as a Product

“The main reason that the Talisman project failed is one of execution. First of all, Talisman was not intended to be a competitor with anybody’s graphics cards. It was intended to be a way of pushing the state of the art that anybody could use. They’d have to pay some nominal license to Microsoft, but Microsoft didn’t want get into graphics cards. They really wanted to make a technology demonstrator of the ideas, because obviously, if you just talk about ideas but don’t actually implement them, then you’re nowhere. So the idea was to implement something to have impressive graphics performance, but not to actually create a product that would compete with 3Dfx and all those other guys at the time.

“We picked a vendor to do the card because we wanted something real that people could see that these ideas actually worked. That was the whole point of that card. The problem with it is that we picked Cypress Semiconductor, which, unbeknownst to us, was going through some severe financial difficulties at the time. And so Cypress, they spun up a team to work on it, and I actually met some of the team members years later, and to this day they’re still believers, but apparently they were always budget limited because the company was not financially sound at the time. And eventually the project just got cancelled because the company decided it didn’t want to go in the graphics card direction. What they wanted to do was serve the telcom market. And so they cancelled the project. And our error was to give Cypress a one year exclusive access to the technology before we gave it to anybody else. And because they couldn’t spend enough money on it, it took them a long time to develop, and when they finally came out with it, it was just too late. The graphics card business is extremely competitive, and if you make one error in execution, you’re dead. And that’s what happened.

“It would have come out a year earlier if we had made the right decisions and had executed properly. Two years earlier it couldn’t have come out. We were still doing the basic designs and stuff.”