Tag Archives: graphing


Temporal Hex Dump

After building some hardware to trace and inject data on the Nintendo DSi’s RAM bus, it became obvious pretty fast that there’s a lot of data there, and (as far as I know) no good tools for analyzing these sorts of logs.

The RAM tracer has already given us a lot of insight into how the DSi works by virtue of letting us inspect the boot process, the inter-processor communication, and most of the code that runs on the system. But all of that knowledge comes in an indirect way, from using the RAM tracer as a platform to run other experiments. I’ve been interested in figuring out whether there’s a way to use the RAM trace itself to help understand a system’s dynamic behaviour.

The RAM is on a packet-oriented bus, so it would make sense to have a tool that looks kind of like a packet-based protocol analyzer. Think Wireshark, but for memory.

But there are also a lot of complex patterns that show up over time. As the DS loads a file, or initializes itself, or renders frame after frame of a UI, there are obvious patterns that emerge. So it also might make sense to have a visual tool, like vusb-analyzer.

Unfortunately, both of these approaches ignore the spatial organization of memory. The bus is a stream of packets that say ‘read’ or ‘write’, but the contents of RAM as a whole is more like a file that’s changing over time. Like in a version control system.

So the tool I’ve been imagining is kind of a hybrid of these. It would have a graphical timeline that helps you visually navigate through large datasets and identify timing patterns. It would have a packet-by-packet listing of the reads and writes. And most importantly, it would be a hex dump tool. But instead of showing a hex dump of a static file, it would be a two-dimensional hex dump. The hex dump shows space, but you can also scrub forward or backward in time, and watch the hex dump change. The hex dump could be annotated with colors, to show which data is about to change, or which data recently changed. You could right click on a byte, and see hyperlinks to the memory transactions that are responsible for that byte’s previous and next values.

As far as I know, nobody’s written a tool like this. So I have no idea how useful it will actually be for reverse engineering or performance optimization, but it seems like a promising experiment at least. So far I’ve been working on an indexing and caching infrastructure to make it possible to interactively browse these huge memory dumps, and I’ve been working on the visual timeline widget. Here’s a quick screencast:

The top section shows read/write/zero activity binned by address, with each vertical pixel representing about 64 kB. The horizontal axis is time, with continuous zooming. The bottom section of the graph shows bandwidth, color-coded according to read/write/zero. Blue pixels are reads, reds are write, and orange is a write of a zero byte.

This log file is about a gigabyte of raw data, or about 2 minutes of wallclock time. It shows the Opera browser on the Nintendo DSi loading a very large web page, then crashing. You can see its heap growing, and you can watch the memory access patterns of code, data, and inter-processor communication.

There’s a lot of room for improvement, but I’m optimistic that this will be at least a useful tool for understanding the DSi, and maybe even a more generally applicable tool for reverse engineering and optimization.

As usual, the source is in svn if anyone’s interested. It’s implemented with C++, wxWidgets, sqlite3, and Boost. I’ve only tested it on Linux, but it “should” be portable.


Wireless temperature picture frame mashup


This is the latest geeky addition to our home decor. It’s a Kodak W820 digital picture frame, showing a graph of real-time temperature data collected from around the house: upstairs and downstairs, garage, outdoors, and even inside the refrigerator.

More photos on Flickr, implementation details below…

Temperature Sensors

Most of my friends probably know that back in 2004, when I was still in college, I built a set of wireless temperature sensors. Each one is an Altoids tin containing a 9V battery, a Dallas 1-wire temperature sensor, PIC microcontroller, and a little 315 MHz AM radio transmitter. I built several of these indoor wireless sensors, plus one outdoor sensor which I built out of PVC pipe.

indoor_therm outdoor_mounted Dallas 1-wire Wireless Temperature Sensor

The PIC spends most of its time sleeping, but every 20 seconds or so it wakes up to read the current temperature from the 1-wire sensor. Most of the indoor sensors have the 1-wire temperature probe soldered to the side of the Altoids tin, for the best thermal conductivity. The refrigerator sensor has a length of 30-gauge wire that leads the 1-wire sensor itself into the fridge.

Every couple of minutes, each sensor transmits a burst of packets back to the base station. The base station is another piece of custom hardware: a matching AM radio receiver, some LEDs, and a TUSB3210 microcontroller to do protocol decoding and to interface back to a server machine over USB. In this case, the host machine is a little NSLU2 embedded Linux box which also acts as our home network’s file server.

NSLU2 and wireless temperature receiver

As usual, microcontroller firmware for the transmitter and receiver is open source.


The software running on the server machine has had a rough history. My original implementation was ridiculously complicated: The receiver unit connected to an NSLU2 embedded Linux box over USB. That machine ran a daemon, written in C, which listened for packets over USB and stored their contents in a remote MySQL database. The MySQL database was running on a more powerful server (a 1 GHz Athlon at the time) which had plenty of disk space. That server also ran a web application using mod_python, which pulled data from MySQL, generated graphs on demand using rrdtool, and generally looked really pretty.

My reasons were mostly historical. I wanted to save all of the temperature data (for another project I had at the time), so instead of sending data directly to a lossy RRD file, I just used rrdtool as a cache for the data I was about to graph. As you can imagine, if the cache ever had to be rebuilt, the server would come crashing to its knees. This implementation was plagued with performance and reliability issues.

So, I recently reimplemented a simpler server software solution. I hacked up the C daemon so that instead of writing to a MySQL database, it forks off an rrdtool process to store the data directly into an RRD database. Instead of generating graphs on-demand, I have a small update-graphs shell script which uses rrdtool to generate all of the graphs. This scripts is run from cron every 5 minutes. The resulting graphs aren’t nearly as intricate as the ones that my mod_python app used to generate, but anything looks pretty if you antialias it 😉

The digital picture frame

This frame is a Kodak W820, an amazingly featureful little gadget that I picked up at the local Fry’s for $180. It has an very pretty 8″ 800×480 display and a rather unique touch-based user interface. It also does a pretty ridiculously good job at playing video, considering it’s a picture frame.

But anyway, the real reason to buy this frame is for its Wi-fi connectivity. If you connect it to your network, you get a range of useful and “useful” services:

  • The frame runs its own HTTP server on port 80, with a web interface you can use for managing settings. You can’t upload pictures via this interface, but it does mean that you can configure the picture frame entirely without using Kodak’s silly Windows software.
  • It’s a uPnP media server and a client. The software that comes with it uses uPnP to browse an upload/download files on the frame’s internal memory or memory card, and the frame itself can use uPnP to stream images from your PC. This is a really interesting feature that could be exploited for more dynamic applications…
  • It has a Flickr client. Yes, the frame itself actually speaks the Flickr API.
  • There’s some fancy-pants Kodak online service, which looks like it’s probably little more than an RSS feed in disguise.
  • And best of all, you can give it custom RSS feeds. Like, say, real-time temperature graphs!

The only snag I hit in setting this up was in trying to keep the frame from caching old versions of the temperature graph. It seems to ignore HTTP cache control headers, and the cache seems to be able to hold quite a nontrivial amount of image data!

The best solution I’ve found so far: Use the RSS <ttl> tag to force the frame to reload the RSS feed itself pretty frequently. Then, in my update-graphs script, I dynamically generate an RSS file which includes a timestamp in the URL and GUID for each image. This seems to do the trick.

What next?

I’m really interested in graphing my home power usage, using a receiver circuit based on the one I built for The Kilowatt Clock. I’m currently waiting for some parts from Digi-key, but I’ll definitely blog about this when it’s done.

If you’re at all interested in displaying real-time stats around the home or office, I’d definitely encourage you to play with this picture frame. It has some great hacking potential, and I’m really excited to see what kinds of uses others come up with.