How we built a Super Nintendo out of a wireless keyboard

I wrote a guest article for Adafruit about the story behind the new Sifteo cubes:

In today’s world, video game consoles have become increasingly complex virtual worlds unto themselves. Shiny, high polygon count, immersive, but ultimately indirect. A video game controller is your gateway to the game’s world—but the gateway itself can be a constant reminder that you’re outside that world, looking in.

Likewise, the technology in these game consoles has become increasingly opaque. Decades ago, platforms like the Commodore 64 encouraged tinkerers and do-it-yourselfers of all kinds. You could buy commercial games, sure, but the manual that shipped with the C-64 also told you what you needed to know to make your own games, tools, or even robots. The manual included a full schematic, the components were in large through-hole packages, and most of them were commonly-available chips with published data sheets.

Fast forward three decades. Today’s video game consoles are as powerful and as complex as a personal computer, with elaborate security systems designed specifically to keep do-it-yourselfers out. They contain dozens of customized or special-purpose parts, and it takes some serious wizardry to do anything with them other than exactly what the manufacturer intended. These systems are discouragingly complicated. It’s so hard to see any common link between the circuits you can build at home, and the complex electrical engineering that goes into an Xbox 360 or Playstation 3.

We wanted to build something different. Our platform has no controller, no television. The system itself is the game world. To make this happen, we had to take our engineering back to basics too. This is a game platform built using parts that aren’t fundamentally different from the Arduino or Maple boards that tens of thousands of makers are using right now.

This is the story of how we built the hardware behind the new Sifteo Cubes, our second generation of a gaming platform that’s all about tactile sensation and real, physical objects.

Read the full article at Adafruit.

Announcing the VMware SVGA Device Developer Kit

Over the weekend, I finally had time to release another work-related open source project: the VMware SVGA Device Developer Kit. It’s a set of documentation and example code for the virtual graphics card that’s present in all VMware virtual machines. The examples run on the (virtual) bare metal, so it’s a really easy way to experiment with low-level graphics driver programming.

GPU Virtualization at WIOV ’08

I just got back from the first
USENIX Workshop on I/O Virtualization.

WIOV was an interesting workshop. It was really nice to see what I/O virtualization looks like from a wide range of different viewpoints. There was some good industry perspective from AMD, Intel, Microsoft, and Oracle. There were also a wide range of academic interests represented. Everyone brings not only their own terminology, but their own idea of which problems are particularly hard or useful to solve.

I did feel like a bit of an outsider, though. Nearly the entire workshop was focused on virtualizing storage and networking devices, which are quite a bit different from the devices I work with on a daily basis: mouse, keyboard, graphics, sound, and USB.

Part of this divide is along server vs. desktop lines, but I also feel like it’s in large part due to the wide difference in status of current virtualization approaches for desktop vs. server devices. Most of the current work on networking and storage devices is in improving manageability, or squeezing out that extra 10-20% of performance overhead. This is all already assuming that there are plenty of known virtualization approaches for those devices which are both correct and reasonably performant.

By contrast, current desktop device virtualization is largely neither correct nor performant. Even simple devices like mouse and keyboard must be fudged with heuristic after heuristic to bridge the impedance mismatch between a low-level mouse/keyboard and the high-level input stack abstraction you see at the windowing system or VNC layer. Sound devices are easy to emulate, but to build a truly correct sound device, you would need guarantees that can only be provided by a hard real-time operating system. USB seems on first glance like it could be simple, but there are huge impedance mismatches to be overcome, and even greater challenges in writing a CPU-efficient emulated USB controller.

I feel that graphics virtualization is the biggest unsolved problem in desktop device virtualization, which is why I enjoy my day job as much as I do. But it also means that graphics virtualization is really at a totally different stage than storage or networking virtualization. Nobody has an implementation that is complete or correct, let alone performant. My team at VMware is pushing the envelope, and the technology is already usable for many people- but there’s a huge amount of work left to be done.

I am very greatful to have had the opportunity to present a paper on graphics virtualization at WIOV ’08. Jeremy Sugerman and I co-authored a paper which describes a taxonomy of graphics virtualization approaches, and provides implementation details and analysis for the approach VMware is currently shipping as the DirectX 9.0 virtualization in VMware Fusion and VMware Workstation. This is currently the most detailed publicly available description of the work my team has been doing over the past 4 years. If you’re interested, I encourage you to read the paper closely. It’s really packed with as much information as we could squeeze into 8 pages.

The presentation followed the structure of the paper, but it was weighted heavily toward describing the motivation for GPU virtualization, the reasons why it’s hard, and the characteristics of the different approaches we outline in our taxonomy.

You can get the paper from the official WIOV ’08 site. You can also grab the paper and the presentation slides from my server:

The slides are also on SlideShare, embedded below.

Speaking at USENIX WIOV 2008

Well, this Monday I submitted the final copy of my paper, and yesterday everything was approved. Jeremy Sugerman and I wrote a paper for the USENIX Workshop on I/O Virtualization’s Industrial Practice session: GPU Virtualization on VMware’s Hosted I/O Architecture. We’re on the program for a 15-minute talk at the workshop in San Diego this December.

The paper is a detailed (or as detailed as will fit in 8 pages) description of the GPU virtualization work my team has been doing at VMware for the past couple years. This is the technology that makes it possible to run DirectX 9 applications and games inside your VMware Fusion VM. The paper includes a lot of background information about graphics virtualization, a detailed description of our virtual GPU architecture, and various benchmarks.

Thanks to everyone who helped me by reviewing drafts of the paper. Your feedback has been invaluable.

Update: The paper can now be downloaded in HTML or PDF, and presentation slides are available.

3D Graphics at VMware

Despite all the random posts about helicopters and embedded systems on here, I haven’t really mentioned what I spend most of my time on these days…

I work in the Interactive Devices group at VMware. For people who aren’t familiar with VMware’s products, we do virtualization: software that lets you run multiple virtual computers inside your real computer. It’s good for data center management and consolidation, but I’m most interested in the ways virtualization can be used on the desktop: It’s great for running Windows apps on your Linux box, or running Windows/Linux software on your Mac using Fusion.

And that’s where our team comes in. While much of the company is focused on doing higher-level tools (user interfaces, management software, APIs) or on infrastructure that primarily benefits server products, our team is really in the heart of the virtualization technology that makes VMware tick on the desktop. We virtualize all the random desktop peripherals like the mouse, keyboard, video card, and sound card- but our team’s two big projects are USB and 3D graphics.

I’ve been working at VMware for close to 3 years now. I spent about the first half of that on USB. I did much of the work in implementing USB 2.0, and I did the original port of our OS-specific USB code to Mac OS X prior to the first release of VMware Fusion. I also implemented Isochronous transfer support (for web cams and the like), and did a lot of neat internal testing infrastructure.

I found VMware’s USB project really interesting because it meshed well with my prior experience with embedded systems and creating USB hardware. It was also a good introduction to the challenges in creating virtual hardware devices for a VM, particularly for hardware with such an un-virtualization-friendly architecture.

I still keep tabs on USB at VMware, but lately I’ve been pretty absorbed with our 3D virtualization effort.

For some background: VMware products provide their virtual machines with a “VMware SVGA device”, a made-up video card which provides legacy emulation of VGA and VESA BIOS modes, plus a vendor-specific set of 2D and 3D acceleration features. We provide drivers for this card as part of VMware Tools, and they come with recent versions of Xorg.

Our most recent releases of VMware Workstation and VMware Fusion include 3D acceleration support which is roughly on par with Microsoft DirectX 8.1, but it’s buggy in a lot of ways and it’s missing key features. Needless to say, the 3D team hasn’t been twiddling its thumbs. Our legal department won’t let me disclose exactly how much progress we’ve made or what features we will support in future products, but I can say that I’m very excited about our current work.

3D graphics virtualization is a really complex and interesting problem. Fundamentally what we’re doing is a Direct3D to OpenGL API translator (much like what the Wine project has) but it’s complicated by the fact that we also have to behave like a video card, and we also have to get the data into and out of a Virtual Machine efficiently. All said and done, the actual 3D API translation issues are only part of the story. There are also a lot of interesting resource management and synchronization problems.

It’s a really interesting project and we’re making some great progress, but we also have one big problem:

Our team is tiny. We have a substantial amount of work to do- it’s like writing a major 3D graphics driver (think ATI or NVidia), writing a game engine, and writing a complex interprocess communication system all rolled into one. Right now we have 7 developers on the project, and it feels like we could use 70.

So, if you know anyone who has a passion for 3D graphics, who knows low-level systems programming inside and out, who isn’t afraid of single-stepping through machine code in the debugger: take a look at our job postings (search for OpenGL) or send a resume to micah at vmware dot com.