ds – scanlime

November Update

I guess I don’t have a complete blog entry to write on any one thing.. but there are several unrelated things that I feel like writing down. I recently joined Twitter, but I must suck at using it because none of these things ended up there.

Life

Tragic. Introspective. Complicated. This isn’t really a place where I would feel it appropriate to go into detail, but it would be unjust to omit given that the other things here are significantly less important.

Helicopters

I don’t have a huge amount of flying time on it yet, but I’ve been enjoying the Blade mSR so far. I’ve been flying it mostly in smallish spaces, so I haven’t had the guts to really see how maneuverable it is- but I’m impressed with its stability and responsiveness so far. Also, either I’m a totally awesome pilot, or (more likely) it’s actually pretty easy to fly.

DSi

I haven’t had much time to work on the DSi lately. I’ve let Bushing borrow my hardware, and he’s been working on producing a small quantity of additional RAM tracing/injecting setups. This isn’t the kind of thing we can make many of (I estimate it takes about 6 hours to solder one together) but it’d be helpful for the community if more than one of these rigs existed in the world.

What time I have spent on DSi-related stuff has been on Temporal Hex Dump. It’s actually coming along okay- I took a bit of a development detour to improve it’s Mac OS support after I bought a MacBook Pro… but the next step is to add the part of it that is actually a hex dump. So far, it has the timeline navigation widget and a correlated list of memory transactions:

Robot Odyssey

So, way back in April, I posted the first screenshots of Robot Odyssey on the Nintendo DS. This was I guess what I’d call a “binary port”. It runs the original Robot Odyssey binaries, via static binary translation, and with many modifications and UI enhancements to adapt the game for the DS console.

I never got around to posting pictures, but I actually did make significantly more progress on this project before getting distracted. I ended up implementing a pretty sophisticated system for patching the game and dynamically manipulating its world state, as well as “forking” additional copies of the running game very quickly for the purposes of having those forked-off copies render additional sprites. What is this good for? Well, I could have the game proper running on the DS’s top screen, while the bottom screen shows status icons for each robot, where the icons accurately depict the state of the robot’s bumpers and thrusters, and even what item the robot is holding at the time. I also had a mostly complete game load/save UI implemented, including a live preview that shows you where you saved while picking a game to load. My next step was going to be a UI for editing circuits inside your robots using the stylus.

Anyway, I know I’m being cruel by describing all these great features and not posting a video or a ROM image. Sorry. I promise I’ll make a video one of these days when I have a bit more time. As for a ROM image, right now the best I have is the source code for a tool which will generate a ROM given the original Robot Odyssey files. You can get the source from https://github.com/scanlime/robot-odyssey-ds.

To compile it, you’ll need devkitARM and Python. You’ll also need to put your original Robot Odyssey files (the DOS version, not Apple!) in the “original” directory in the source tree. You can leave them in a zip file, or strewn about the directory in random places. The translator knows what files it needs, and it will find them by SHA-1 hash.

I give no guarantees as to the current state of the source tree- I haven’t worked on it in a few months, and I think I last left it in a state where loading and saving were partially but not usably done. If you aren’t a developer, the source tree isn’t likely to be useful to you yet.

Homework assignment: If anyone in the audience wants to see this Robot Odyssey DS port turn into a real homebrew ROM image that anyone can download without compiling it themselves, we need to have redistribution approval from the game’s current copyright holders. Problem is, I have no idea who that is. I did email Warren Robinett about this project a while back, but he couldn’t tell me who currently owned the copyright. Following the trail of acquisitions, I think it might be owned by Houghton Mifflin Harcourt. Yeah, the textbook company. I did email them about the Robot Odyssey copyright multiple times, but I never got a reply. So, what now?

One approach is always to release it then look for the return address on the cease & desist letter. But, in all seriousness, I would like to be able to release a legal port of a classic game which the current copyright holder clearly has no commercial interest in. It seems like there should be some way to do this.

Temporal Hex Dump

After building some hardware to trace and inject data on the Nintendo DSi’s RAM bus, it became obvious pretty fast that there’s a lot of data there, and (as far as I know) no good tools for analyzing these sorts of logs.

The RAM tracer has already given us a lot of insight into how the DSi works by virtue of letting us inspect the boot process, the inter-processor communication, and most of the code that runs on the system. But all of that knowledge comes in an indirect way, from using the RAM tracer as a platform to run other experiments. I’ve been interested in figuring out whether there’s a way to use the RAM trace itself to help understand a system’s dynamic behaviour.

The RAM is on a packet-oriented bus, so it would make sense to have a tool that looks kind of like a packet-based protocol analyzer. Think Wireshark, but for memory.

But there are also a lot of complex patterns that show up over time. As the DS loads a file, or initializes itself, or renders frame after frame of a UI, there are obvious patterns that emerge. So it also might make sense to have a visual tool, like vusb-analyzer.

Unfortunately, both of these approaches ignore the spatial organization of memory. The bus is a stream of packets that say ‘read’ or ‘write’, but the contents of RAM as a whole is more like a file that’s changing over time. Like in a version control system.

So the tool I’ve been imagining is kind of a hybrid of these. It would have a graphical timeline that helps you visually navigate through large datasets and identify timing patterns. It would have a packet-by-packet listing of the reads and writes. And most importantly, it would be a hex dump tool. But instead of showing a hex dump of a static file, it would be a two-dimensional hex dump. The hex dump shows space, but you can also scrub forward or backward in time, and watch the hex dump change. The hex dump could be annotated with colors, to show which data is about to change, or which data recently changed. You could right click on a byte, and see hyperlinks to the memory transactions that are responsible for that byte’s previous and next values.

As far as I know, nobody’s written a tool like this. So I have no idea how useful it will actually be for reverse engineering or performance optimization, but it seems like a promising experiment at least. So far I’ve been working on an indexing and caching infrastructure to make it possible to interactively browse these huge memory dumps, and I’ve been working on the visual timeline widget. Here’s a quick screencast:

The top section shows read/write/zero activity binned by address, with each vertical pixel representing about 64 kB. The horizontal axis is time, with continuous zooming. The bottom section of the graph shows bandwidth, color-coded according to read/write/zero. Blue pixels are reads, reds are write, and orange is a write of a zero byte.

This log file is about a gigabyte of raw data, or about 2 minutes of wallclock time. It shows the Opera browser on the Nintendo DSi loading a very large web page, then crashing. You can see its heap growing, and you can watch the memory access patterns of code, data, and inter-processor communication.

There’s a lot of room for improvement, but I’m optimistic that this will be at least a useful tool for understanding the DSi, and maybe even a more generally applicable tool for reverse engineering and optimization.

As usual, the source is in svn if anyone’s interested. It’s implemented with C++, wxWidgets, sqlite3, and Boost. I’ve only tested it on Linux, but it “should” be portable.

DSi RAM tracing

It seems like a lot of people have been seeing my Flickr photostream and wondering what I must be up to- especially after my photos got linked on hackaday and reddit. Well, I’ve been meaning to write a detailed blog post explaining it all- but I keep running out of time. So I guess a “short” blog post is better than nothing.

The executive summary is that I’ve been working on making it possible to run homebrew games on the DSi. If you’ve used The Homebrew Channel on the Wii, you know what I’m talking about- we’d basically like to have the same thing for the DSi. And as it turns out, the DSi has a lot in common with the Wii.

I haven’t been working on this project alone. I’ve been collaborating with the excellent software and hardware engineers in Team Twiizers, the group responsible for The Homebrew Channel. Bushing has written a couple blog posts about our DSi efforts so far: one on the RAM tracing effort in general, and one with a sample trace.

My role in this project lately has been building hardware and software for a tool which is effectively a low-level CPU debugger for the DSi. It attaches to the RAM bus between the RAM and CPU, it’s capable of tracing all traffic in ‘real-time’ (after slowing down the system clock so the required bandwidth fits in USB 2.0), and just this week I got RAM injection working. The hardware interposes on one of the chip select signals between the CPU and RAM. If an address floats by that we want to patch, the hardware can quickly disable the real RAM chip and drive new data onto the bus as the CPU continues to read. It’s a very effective technique, and my hardware has a pretty capable system for storing and retrieving up to 16 kB of patch data now.

So why is all the complex hardware needed? Well, we hit two big dead-ends:

Several of us managed to run code from within a DSi-enhanced game using modified save files, but most of the interesting hardware on the DSi (like the SD card slot, and the internal flash memory) is disabled when these games are running. The earlier Team Twiizers Classic Word Games hack, WinterMute’s cookhack, and my own cookinject all suffer from this problem.
I’ve been able to read and write flash memory for a while. This isn’t as interesting as it may sound, since nearly all of the flash is encrypted. However, by running a lot of tests where we modified or compared the encrypted data, we were still able to gain a lot of knowledge about the layout of the flash memory, and some information about how the system boots.

What we’d really like to do is understand more about how the console’s main menu works, and how software is installed and loaded on the DSi. The best way to do this seemed to be RAM tracing. It required an FPGA and a lot of crazy soldering under a microscope, but now it’s working and we’re well on our way to unlocking the secrets of DSi homebrew.

If you’re interested in following our efforts to make homebrew possible on the DSi, definitely subscribe to Bushing’s blog at hackmii.com. He understands the Wii (and by extension, the DSi) better than I ever will, and he’s great at writing about it.

I have a Flickr set with lots more photos, but these are two of my favorite. The first is a photo of the entire setup as it’s currently configured. The second is a picture of the craziest of the microscopic solder joints in this project. This is where I separated the chip-select signal I mentioned above, giving the FPGA the ability to sit between the CPU and RAM. The trace I’m tapping into is buried under four other PCB layers. For a sense of scale, the orange enameled wire is 32 AWG, or about 0.2mm in diameter. Normal 30 AWG wire-wrapping wire is actually only the third smallest size of wire I’ve been using in this project 🙂

If you’re interested in the C and Verilog source code, it’s in Subversion as usual.

Fun with FPGAs: SPI EEPROM emulator

I found a fun excuse to practice my digital logic design over the past few days:

This is an FPGA emulating the SPI EEPROM (save game memory) in a modified Nintendo DS game cartridge. The emulated EEPROM is backed by 1 MB of SRAM, and there is a USB interface that you can use to simultaneously read or write the SRAM from a PC.

You can use it to really quickly modify or inspect the contents of a game’s save memory- for example, for developing save game exploits or reverse engineering save file formats. You can also use the large memory space as a fast shared memory buffer, for sending debugging information or memory dumps back to the PC, or for loading larger blocks of code from the PC.

The FPGA design is written in Verilog, and I’m running it on the popular Digilent Spartan 3 board. The upload/download tool that runs on the PC is written in Python, plus it can be used as a Python module for writing more complex tools that interact with the DS through this shared memory buffer. Source code is in Subversion, and released under an MIT-style license.

DSi hacking, BGA rework

I got a Nintendo DSi earlier this month. It’s a really cute little console, and a nice revision to the DS Lite. Screens are slightly larger, CPU is about twice as fast, 4x the RAM. And of course it has these camera thingies. Other folks have teardowns online with internal photos aplenty 🙂

So, I spent a week or so tooling around with the DSi doing normal consumer and/or homebrew things with it. Played some The World Ends With You, got an Acekard 2i and tried running some homebrew. Its DS compatibility mode seems to be pretty good. The only compatibility bug I bumped into was running my work-in-progress Robot Odyssey port. It mostly worked, but there was a little bit of crackling in the audio. Not surprising, I’m doing the audio in a somewhat unconventional way, and it doesn’t work properly in any DS emulator either 🙂

The DSi is pretty cute, but it would be nice if we could use its new features for homebrew software. It would be especially cool if you could run homebrew directly from its built-in SD card reader, without the need for any special-purpose hardware. That would really open the door to some cool indie games that nearly anyone could play.

Need to satisfy my curiosity, then. Out come the jewler’s screwdrivers, antistatic mat, and SMD rework tools…

There are a few other people working on hacking the DSi from other avenues. Team Twiizers just released a video of a save game exploit that they’re using to run code in DSi mode. This is pretty cool, and as far as I know it’s the first time a hobbyist has been able to run code in DSi mode. I’m not sure how much if any of the new hardware they’ve managed to reverse engineer through this hack.

There are also a handful of Wii hackers that have noticed that the DSi has pretty much the same file format for its downloadable games, so they’re trying to come up with a way to install homebrew DSiWare apps. Unfortunately, they need the system’s private key- and brute-forcing it doesn’t seem too promising. One way to get this private key would be to extract it from the DSi’s firmware.

The thing that interests me the most right now is figuring out how the DSi boots. There are plenty of other layers of software that you can exploit from the outside- but the bootloader and built-in firmware seems like the best starting point for any serious effort to reverse engineer the DSi’s hardware.

The DSi has upgradeable firmware, and there’s only one visible place that it could be stored: a 256 MB NAND flash chip. This is actually an embedded MMC (eMMC) chip from Samsung. It’s electrically pretty much the same as an MMC card, but in a tiny BGA form factor. It is wired directly up to the CPU, which would indicate that there is some kind of hardware or software bootloader that knows how to pull code off of the eMMC chip during early boot. This probably means there’s a mask ROM in the same package as the CPU. It would also make sense that this ROM would know how to write an initial firmware image to the eMMC chip during manufacturing, since there is no other way to access the eMMC chip after it’s soldered to the DSi’s main board.

I was hoping that the MMC data lines would be accessible via some of the many test points. None of them were obviously related to the NAND, though. I tried probing the SPI bus, where the original DS kept its firmware, but the eMMC chip was definitely on a separate bus. It didn’t help at all that the DSi refuses to boot unless every single one of its peripherals are attached, so this makes it rather hard to poke around the board’s test points while it’s running.

At this point I assumed that the traces running from eMMC chip to CPU were completely buried. My last recourse: desolder the eMMC chip, and hook it up to some kind of test jig that would let me read back its contents. Then perhaps I could copy it to a normal MMC card, and wire up an MMC socket on the DSi.

After practicing my BGA rework technique a bit, I desoldered the eMMC chip. If you power the DSi on without the eMMC chip, it displays a nice hexadecimal error code. This would seem to indicate that there is indeed a mask ROM in the CPU package- a hardware bootloader wouldn’t be smart enough to display a “nice” error like this.

There was a pleasant surprise underneath the eMMC chip: All of the data signals were in fact accessible on nearby components or vias. I didn’t find them earlier, since I couldn’t really poke around with the ‘scope and have the board access the eMMC at the same time. But, armed with this knowledge, I could switch to a simpler approach: Solder the eMMC chip back on, then build a passive sniffer that tells me exactly what reads and writes the DSi’s CPU makes.

By some miracle, I managed to get the eMMC chip soldered back on successfully. The DSi’s mainboard is a huge pain to do BGA rework on, at least with the tools I have. It has so many ground/power planes and fills that the board is really hard to keep hot with just a hot-air rework tool.

My poor DSi then is ‘mostly’ back in one piece. Now I’m working on the next step: Hooking up an FPGA that can dump all the memory reads/writes over USB 🙂

More stuff: There are a few more pictures on flickr, and I started a thread with a few more technical details on the gbadev forums.

Robot Odyssey DS: First screenshots

This is nowhere near ready for prime-time, but:

Yep, it’s Robot Odyssey for the Nintendo DS. I literally just got this working yesterday, so please don’t ask for any precompiled binaries. If you don’t already know where the source code is, you really don’t want to see it 🙂

Before you ask, this is not a general-purpose DOS emulator for the DS. It’s actually a static binary translator which does most of the work in porting a DOS game to the DS, but there’s still an awful lot of manual intervention required. I only really developed the translator with Robot Odyssey in mind, so there are sure to be features missing that you’d need in order to use it with other games.

My intent is to turn this into a real port of Robot Odyssey to the DS, not just an emulation. In particular, I’d like to re-do the load/save game UI to support timestamps and thumbnails, and I’d like a soldering mode that makes use of the DS’s touchscreen.