Temporal Hex Dump

After building some hardware to trace and inject data on the Nintendo DSi’s RAM bus, it became obvious pretty fast that there’s a lot of data there, and (as far as I know) no good tools for analyzing these sorts of logs.

The RAM tracer has already given us a lot of insight into how the DSi works by virtue of letting us inspect the boot process, the inter-processor communication, and most of the code that runs on the system. But all of that knowledge comes in an indirect way, from using the RAM tracer as a platform to run other experiments. I’ve been interested in figuring out whether there’s a way to use the RAM trace itself to help understand a system’s dynamic behaviour.

The RAM is on a packet-oriented bus, so it would make sense to have a tool that looks kind of like a packet-based protocol analyzer. Think Wireshark, but for memory.

But there are also a lot of complex patterns that show up over time. As the DS loads a file, or initializes itself, or renders frame after frame of a UI, there are obvious patterns that emerge. So it also might make sense to have a visual tool, like vusb-analyzer.

Unfortunately, both of these approaches ignore the spatial organization of memory. The bus is a stream of packets that say ‘read’ or ‘write’, but the contents of RAM as a whole is more like a file that’s changing over time. Like in a version control system.

So the tool I’ve been imagining is kind of a hybrid of these. It would have a graphical timeline that helps you visually navigate through large datasets and identify timing patterns. It would have a packet-by-packet listing of the reads and writes. And most importantly, it would be a hex dump tool. But instead of showing a hex dump of a static file, it would be a two-dimensional hex dump. The hex dump shows space, but you can also scrub forward or backward in time, and watch the hex dump change. The hex dump could be annotated with colors, to show which data is about to change, or which data recently changed. You could right click on a byte, and see hyperlinks to the memory transactions that are responsible for that byte’s previous and next values.

As far as I know, nobody’s written a tool like this. So I have no idea how useful it will actually be for reverse engineering or performance optimization, but it seems like a promising experiment at least. So far I’ve been working on an indexing and caching infrastructure to make it possible to interactively browse these huge memory dumps, and I’ve been working on the visual timeline widget. Here’s a quick screencast:

The top section shows read/write/zero activity binned by address, with each vertical pixel representing about 64 kB. The horizontal axis is time, with continuous zooming. The bottom section of the graph shows bandwidth, color-coded according to read/write/zero. Blue pixels are reads, reds are write, and orange is a write of a zero byte.

This log file is about a gigabyte of raw data, or about 2 minutes of wallclock time. It shows the Opera browser on the Nintendo DSi loading a very large web page, then crashing. You can see its heap growing, and you can watch the memory access patterns of code, data, and inter-processor communication.

There’s a lot of room for improvement, but I’m optimistic that this will be at least a useful tool for understanding the DSi, and maybe even a more generally applicable tool for reverse engineering and optimization.

As usual, the source is in svn if anyone’s interested. It’s implemented with C++, wxWidgets, sqlite3, and Boost. I’ve only tested it on Linux, but it “should” be portable.

Virtual USB Analyzer

From late 2005 to early 2007, I worked on the USB virtualization stack at VMware. We ran into all sorts of gnarly bugs, many of which were very hard to reproduce or which required access to esoteric or expensive hardware. To help with debugging problems both internally and with customers in the field, we added logging support to our virtual USB stack. Starting with VMware Workstation 5.5, if you set the right hidden config option we’d start dumping the contents of all USB packets to a log. It was a USB sniffer (like USB Snoopy), but built into the virtual hardware.

To make it easier to analyze the resulting logs, I started working on a GUI tool that could navigate through these giant log files. This tool proved to be really useful within our team at VMware, and we’d often ask customers on the beta forums to generate log files that we could analyze. I called this tool vusb-analyzer.

Well, it’s been a while, but I’m proud to now have the opportunity to release vusb-analyzer as open source software under the MIT license. This isn’t just a code dump- I removed the tool from our internal repository today, and all future development will occur in the open, in a Subversion repository on Source Forge.

Currently, vusb-analyzer is most useful for analyzing logs captured by VMware products. Indeed, this is a convenient way to debug USB drivers. You can attach your USB device to VMware Workstation, VMware Fusion, or the free-as-in-beer VMware Player, and it can transparently save a plaintext log of all USB packets that pass through. Debugging a driver in a VM is really convenient, and I find it quite useful, but I understand it’s not for everybody. If you already have a favorite USB sniffer tool, it’s easy to extend vusb-analyzer with a log format plugin so it can read other formats. We already did this once, to add support for the logs generated by our favorite hardware USB analyzers.

I hope vusb-analyzer turns out to be useful for the open source community, particularly to those who are working on Linux device drivers. To get started, visit the project’s web site:


Besides the usual introduction and download links, there is also a detailed tutorial with lots of shiny screenshots 🙂

Soft RFID tags and cuddly plushies

This week a new power tool arrived on my desk:

How did this happen? It was actually kind of complicated…

  1. Months ago, Scott and I dreamed of an automated kegerator. For the workplace, naturally. We’d need some way to hand out credits that coworkers could use in order to dispense beer, so we’d need a way to identify people or otherwise store these credits. We could use some kind of “credit card”, but that’s so last century. I wanted to just use the proximity ID badges we already had as employees.
  2. So, I got interested in RFID technology. It was also a decent excuse to brush up on analog electronics.
  3. I build an ultra-simple “software only” RFID reader that can read multiple card formats, including our ID badges.
  4. Hey, wait, I can read my prox card now… I can make an RFID garage door opener
  5. Uh, what now? Readers are easy, how about making an RFID tag from scratch?
  6. My RFID tag works fine, but it’s kind of ugly. What if I could just build a cloned badge directly into the sleeve of a hoodie? Now I could just wave my arm at the door to get into the office.
  7. This leads me to do some more research on the LilyPad “wearable e-textile” technology developed by Leah Buechley at my Alma Mater.
  8. I order some conductive thread, buy a sewing kit at Target, and try my hand at making some RFID antennas.
  9. Sewing accurate and tightly spaced spirals by hand using really finicky thread is not fun. I think about buying a sewing machine.
  10. I’m really tempted by the Singer Futura Computerized Embroidery Machine, but the $800 price tag scares me off.
  11. After only moderate fuss and only about an hour waiting in line, I get a basic mechanical sewing machine (a Singer Inspiration 4210) from my local Jo-Ann’s.

Whew. Ok, I have a sewing machine. So what do I do with it?

Cloth RFID tag prototypes

I did end up making a working cloth RFID tag. This is a 2×3 inch iron-on patch with an antenna coil made of conductive thread. I sewed the bulk of the antenna by machine (and it would have been thicker if my thread hadn’t broke) then I did the more intricate bits by hand. The antenna itself is actually conductive only on one side, since I had insulating thread in the bobbin. I ran the conductive thread back from the outside of the coil to the center of the tag by sewing on the opposite side, so as not to short-circuit the coil. The black blob is an ATtiny85 microcontroller in a SOIC-8 package.

In this tag, I attached the chip to the antenna using conductive epoxy. In my next prototype, I’ll probably use crimp beads as Leah Buechley suggests. The antenna should also really use more ‘turns’ of thread. This tag works with my Parallax RFID reader, but the range is only a few millimeters. On the right is a larger coil I was sewing for practice. This one would have been larger too, but I had problems with the thread breaking.

I plan to make more prototype RFID tags, both to practice my sewing and to try and increase the range.

So, what else can I do with this machine? I had some fun with it:


The red dude on the bottom is of course an anime smiley, which may or may not be from the Kingdom of Loathing universe. The “ears” and “legs” were totally unintentional, but I think it turned out pretty cute. Next I made the orange above it. It came out kinda lumpy, but it was good practice. I later practiced my vivisection skills by giving it the test-claw that I built while figuring out how to make my next critter- the cute yellow crab in the middle.

You may remember the crab from an earlier post of mine. It’s kind of a chibi version of Ikea’s KORALL KRABBA. They’re so cute together!


There are a few more pictures in my Sewing set on Flickr ^_^

New oscilloscope!

For a while now, I’ve been looking for a replacement for my ancient 50 MHz Hitachi analog scope. This scope has served me well for the 10+ years I’ve owned it, but it’s kind of a junker. Slow, out of calibration, flaky triggers… I’d could probably get a similar scope for about $15 on eBay.

As much as I love the instant response and tactile feel of analog scopes, I’ve been doing several projects lately that really could benefit from a digital scope and a logic analyzer. I’ve frustrated myself for hours trying to capture one-shot signals with my analog scope, when a digital scope would have made the job a piece of cake.

So, I finally bought a Bitscope BS100U. It’s a PC-based oscilloscope. I was initially really skeptical about PC-based scopes, but they do have a few big advantages: They don’t hog desk space, it’s easy to export data to your own tools or capture screenshots for the web, and they tend to be cheaper than a standalone DSO with similar features.

Why a Bitscope, instead of something by Pico or another of their competitors? The Bitscope is really hacker-friendly. The manual includes full schematics, their communications protocol is documented online, and there’s a free API for extending their software. Did I mention it’s cross-platform? (Windows and Linux)

Best of all, the Bitscope is a mixed-signal scope. The BS100U has two analog inputs and eight digital inputs. I’ll still need to pull out my homebrew “Saxoscope” for high-speed continuous logic capture, but the Bitscope’s MSO mode should be more than sufficient for day-to-day use.

So, I just got it in the mail today. I may post a full review once I’ve had more time to use it. My initial impressions:

Pros: Small, opto-isolated, mixed-signal, easy to set up, good build quality, fairly powerful software, open design, cheaper than a Tektronix digital scope.

Cons: Kind of slow, software UI is pretty odd, more expensive than some competing PC-based scopes ($595 without any probes/pods). Bitscope web site is very rarely updated.

(Screen shot of some mixed-signal capture from the 10 megabit SPI interface on my Propeller ethernet driver)