My friends at Tangible Interaction sent some electronic whiteboard hardware to take apart. In this video I’ll start to look at the transmitters, receivers, and communication protocol.
In today’s world, video game consoles have become increasingly complex virtual worlds unto themselves. Shiny, high polygon count, immersive, but ultimately indirect. A video game controller is your gateway to the game’s world—but the gateway itself can be a constant reminder that you’re outside that world, looking in.
Likewise, the technology in these game consoles has become increasingly opaque. Decades ago, platforms like the Commodore 64 encouraged tinkerers and do-it-yourselfers of all kinds. You could buy commercial games, sure, but the manual that shipped with the C-64 also told you what you needed to know to make your own games, tools, or even robots. The manual included a full schematic, the components were in large through-hole packages, and most of them were commonly-available chips with published data sheets.
Fast forward three decades. Today’s video game consoles are as powerful and as complex as a personal computer, with elaborate security systems designed specifically to keep do-it-yourselfers out. They contain dozens of customized or special-purpose parts, and it takes some serious wizardry to do anything with them other than exactly what the manufacturer intended. These systems are discouragingly complicated. It’s so hard to see any common link between the circuits you can build at home, and the complex electrical engineering that goes into an Xbox 360 or Playstation 3.
We wanted to build something different. Our platform has no controller, no television. The system itself is the game world. To make this happen, we had to take our engineering back to basics too. This is a game platform built using parts that aren’t fundamentally different from the Arduino or Maple boards that tens of thousands of makers are using right now.
This is the story of how we built the hardware behind the new Sifteo Cubes, our second generation of a gaming platform that’s all about tactile sensation and real, physical objects.
Making a Vibrator That Listens to Your Body
This is the opposite kind of project. A little bit of work and a little custom design to create something new and exciting that I can immediately use in my everyday life. It also happens to be a sex toy.
In other words, I wanted to hack something I actually use: my vagina.
I love feedback loops. Servo motors, thermostats, op-amps, DC-DC converters, social networks, flocking behavior. Our bodies. Massages, cuddling, sex– these are all ways of bringing a partner into your body’s most fun closed-loop systems.
To me, a good sex toy helps form feedback loops. It doesn’t get in the way. A good toy gives you simple ways of exchanging signals with a partner or with your own body. It acts as a conduit. A good sex toy is analog.
I was in the market for a remote-controlled vibrator recently, and I ended up with LELO’s Lyla vibe:
(Update: LELO was nice enough to send me their updated model, the Lyla 2, which also works with this remote. The Lyla, Lyla 2, and Tiani 2 are all known to use the same radio protocol.)
There are some things I really like about this toy. The vibrator itself is reasonably strong, rechargeable, waterproof, and quite comfortable. I was much less happy with the remote. The radio range was rather lackluster, and using the controls made me feel more like I was programming a VCR (remember those?) than having sex.
The optional accelerometer input on the remote was a good idea, but I feel like the execution leaves much to be desired. The controls are laggy, and controlling the vibrator by tilting the controller never really felt right to me.
So, naturally, I wanted to see if I could do better. I really had no idea where the project would go.
Choosing a Sensor
My early prototype used a simple knob attached to a variable resistor, and that already seemed like a big improvement over the original LELO remote. I wanted to build a simple proof-of-concept remote that would just demonstrate the improved radio range and responsiveness, without doing anything particularly fancy. After that, I planned to dabble in more esoteric input devices. Audio spectra, conductive fabric, capacitive sensors embedded in lingerie, and so on. The Arduino library I ended up writing for this project offers some great opportunities for further tinkering.
But I was rummaging through my sensor drawer, and thought: Why not sonar? A few quick tests, and it seemed more than responsive enough. And for myself, often lazy about mechanical things, a small sensor with no moving parts was much easier to build a robust prototype around.
At first, the benefits seemed pretty easy to understand. It was hands-free. This meant that, unlike the original remote, it doesn’t need a bright pink silicone jacket to stay clean. Well that sounds convenient. Then I start to play with it more, and I discover something really unique about this configuration.
Something about this toy really does become more than the sum of its parts. More than a simple remote control, it starts to feel a little like virtual reality. Haptic technology, it’s called. Interfacing with computers through our sense of touch.
This toy serves as a kind of analog bridge between two remote spaces: the column of ultrasonically-oscillating air in front of the remote, and whatever body part happens to be in contact with the vibrator. Touch that invisible space above the remote, and the vibrator touches you.
In fact, it does start to feel like there’s a palpable object in space above the remote’s sensors. Move your body close to it, and it reacts. Press into it lightly, or tease the edges. Flick your hand through it, or make graceful waves back and forth. You can use your whole body to touch it, almost like a big fuzzy vibrating cone floating in air.
If the sensor can see your body’s rhythms, it responds in kind, effortlessly synchronizing to its frequency. This is exactly the sort of closed-loop control I was after.
You can even use multiple vibrators. There’s no unique address or channel assigned to a particular vibrator, so any vibes that are turned on and within radio range will respond.
So, what does it look like?
The two black circles are ultrasonic transducers. One of them transmits short “chirps” at a frequency too high for humans to hear. The other listens for echoes. The 4-digit display gives another satisfying bit of feedback, in visceral high-contrast blue LED light. The external antenna gives it quite a bit more radio range than the original remote, and the exposed serial port on the left makes it easy to reprogram the remote using the Arduino IDE.
You can get an idea for how the prototype works in this short video:
That’s the end result, for now. The rest of this post will share the journey I took in building this toy. Perhaps it will inspire you to follow along, or to build something unique.
In order to replace the original remote control, first I had to understand it. My first stop was the FCC ID database, to see if they had any info that would help me know if it was even worthwhile to crack the remote open. I was in luck. The internal photos clearly showed an MSP430 microcontroller and CC2500 radio. Hackability was looking good so far.
The CC2500 is a really nice configurable 2.4 GHz radio and modem chip with an SPI interface. It’s part of a line of highly integrated radios made by Chipcon, now owned by Texas Instruments. They’re quite similar to the competing nRF24L01 radio made by Nordic Semiconductor. The CC2500 also has a popular sub-1 GHz sibling, the CC1100. This chip was featured in the ToorCon 14 badge, and the imminently hackable IM-ME toy.
I wouldn’t even bother with trying to read or reprogram the original microcontroller. By sniffing this SPI bus, I could reverse engineer the proper radio settings and protocol to use. Then I could wire up any CC2500 to any microcontroller I want, and control the vibrator over the air.
Unfortunately, opening the remote turned out to be somewhat messier. The pink silicone jacket is glued to the white plastic shell, and I failed to remove it without tearing the fragile silicone. I soon discovered that the shell itself was also glued shut, and it required quite a lot of cutting and prying to open. I was willing to sacrifice this remote for science, but I really wouldn’t advise ever opening one of these if you want it to stay nice and watertight.
Once I had the remote open and the circuit board extracted, I started making a test jig. I’ll often do this by using a short length of snappable 0.1″ headers as a mechanical anchor. I solder it to some big sturdy pads on the PCB. In this case, I used the pads for the battery contacts. Then, I break out the microscope and run thin AWG 32 magnet wire from the headers to whatever I want to probe. In this case, I wanted the SPI bus. I also replaced the original remote’s small vibrator motor with an LED, so I could see when it was on without it shaking my whole setup.
At this point there are all sorts of options for snooping on the communications between these radio and microprocessor. If you have room in your budget and toolbox for a special-purpose device, Total Phase makes a pretty sweet little SPI and I2C sniffer device. There are logic analyzers like the Saleae Logic, and open source tools like the Logic Shrimp.
Unfortunately, I had none of these handy. My Saleae Logic was far away, and my trusty Bitscope isn’t really that helpful for protocol reverse engineering once you get above the physical layer. So, I improvised. Often in this situation I’ll break out something like the Saxo board, with an FPGA and a thick USB 2.0 pipe. In this case, I was dealing with low enough data rates that I could do something even simpler. I plugged the remote into my Propeller Demo Board and wrote a quick program to capture the SPI traffic and send it back in ASCII over the serial port.
The full traces are available in Git. I was looking for two kinds of traffic: an initialization sequence for the radio, and SPI transfers which actually transmitted packets over the radio. I was also keeping my eye out for any clues as to how complex the protocol was. Do I need to pair with the vibrator? Exchange keys? Search for a radio channel to use?
When the remote turns “off”, it’s actually entering a low-power standby mode. It doesn’t fully shut down the radio in this mode, it just sends a standby command. As a result, waking up the remote doesn’t fully initialize the radio. To capture a complete init sequence, I would cut and reapply power, as if fresh batteries were just inserted. When I did this, I was greeted with a nice burst of configuration traffic:
300F SRES Strobe: Soft reset 0B0F 0A0F FSCTRL1 IF frequency of 253.9 kHz 0C0F 000F FSCTRL0 No frequency offset (default) 0D0F 5D0F FREQ2 FREQ = 0x5d13b1 = 2420 MHz 0E0F 130F FREQ1 0F0F B10F FREQ0 100F 2D0F MDMCFG4 CHANBW = 541.666 kHz 110F 3B0F MDMCFG3 DRATE = 249.94 kBaud 120F 730F MDMCFG2 MSK modulation, 30/32 sync word bits 130F 220F MDMCFG1 FEC disabled, 2 preamble bytes 140F F80F MDMCFG0 CHANSPC = 199.951 kHz ...
The hex numbers on the left come from the SPI sniffer. The text to the right is my own annotation, starting with the name of the radio register in question. Each line is one SPI transaction, and each group of four digits represents one byte going across the SPI bus. The first two digits are command data from the microcontroller (MOSI), the last two digits are the radio’s simultaneous response byte (MISO).
The log continues on like this a bit longer, but you can already see the most important radio parameters: A base frequency of 2.420 GHz, MSK modulation, 250 kBps data rate.
When I looked at the SPI trace for waking the remote up from sleep, I was greeted with a rather large red herring. It spends some time scanning ten different channels, numbered 0 through 9. This represents 2 MHz of spectrum. On each channel, it spends some time polling the Received Signal Strength Indication (RSSI) register. Is it listening to see if the channel is clear? Is it searching for other remotes? Listening for an initialization sequence from the vibrator?
As far as I can tell, none of these things are true. I’ve never seen the vibrator transmit or the remote receive, which rules out any kind of pairing sequence. To verify this, you can turn on a vibrator while the remote is already transmitting. The vibrator picks up the signal and starts moving, without any change to the remote’s routine. The original remote will even control multiple vibrators happily. There seems to be no pairing sequence at all: the address and radio channel are both hardcoded. Why scan through channels then? It feels like either a vestige of some library code the folks at LELO adopted. Or maybe it’s scaffolding for future functionality. Either way, it doesn’t seem to matter for the vibrators I have.
How about the actual packets? Well, they also seem to have a lot of vestigial content. It’s possible this is LELO being super clever and leaving room for future products to be protocol-compatible. Or it’s possible they just hacked together all the example code they could find until they had a working product. It’s hard to say.
The original remote transmits packets packets at a measly 9 Hz. This low update rate most definitely contributes to the laggy feeling and lacking radio range of the original remote. Why so slow? It was probably a battery life tradeoff. The original remote ran off of two AAAs, whereas my replacement is going to have significantly more power available. By transmitting about 10x as often, my remote can achieve much better responsiveness, plus it can tolerate more radio noise by virtue of having a lot more redundancy. Even if many packets are corrupted, quite a few packets are likely to make it through the air unharmed.
Each packet contains a motor strength update, as an 8-bit number which seems to have a usable range of 0 through 128. The packets themselves always have 9 bytes of payload provided by the microcontroller, plus a CRC and header which is generated by the CC2500 itself. Here’s the payload of a typical packet:
01 00 A5 28 28 00 00 00 05 \___/ Motor Strength
In this example, the motor strength is 0x28, or about 30% of full power. I’m not sure what the other bytes are for. They seem to stay constant, and simply replaying these packets back to the vibrator always seems to work. I’m also not sure why there are always two copies of the motor power byte. It seems most likely that this is for added redundancy, so that even if the radio is very noisy and there’s a CRC collision, the vibrator is unlikely to accept a corrupted motor strength byte.
At this point, I had enough information about the protocol to try and build my own emulation of the original remote. This was before I had ordered any CC2500 breakout boards, so I used the remote itself as a dumb CC2500 radio by holding the original MSP430 microcontroller in reset. This was the setup I used to develop an Arduino library that could configure the CC2500 radio correctly and send packets like the ones above.
Now that the yak was bald, I could get on to the really fun part of the project: Designing a better, stronger, faster remote using commonly available parts.
At this point, the project was seeming pretty straightforward: Off-the-shelf Arduino, CC2500 breakout board, sonar sensor, and LED display. But how would I power all of this? The sonar and LEDs are both pretty power-hungry, and I would be using the radio much more heavily than the original remote. I would need to budget nearly 100 mA, with 5v rails for the sonar and LED and 3.3v for the radio and Arduino.
I certainly could have used AA or AAA batteries. I wanted the mechanical design to be simple and compact, though. Designing my own battery holder would not have been simple, and an off-the-shelf plastic battery holder would have been bulky. I even thought about using a flashlight body as a battery holder, but I didn’t see an elegant way to attach my own mechanical parts to it. Rechargeable batteries come in much more friendly shapes. But now you need a charger.
This was getting complicated fast. Lithium polymer battery, a boost converter to raise the voltage to 5V for the sonar module, charging circuit, “fuel gauge” indicator. All of this work goes into every commercial product that runs on batteries, and we often take it for granted. As far as I’m aware, though, there isn’t a great equivalent for quick DIY prototyping. The Arduino Fio board is close to what I want: an Arduino with a built-in LiPo battery charger. But it doesn’t have the 5V boost converter or any way of monitoring the battery’s charge.
Without designing my own PCB, I’d need several separate components: battery, fuel gauge, charge/boost. All total, over $45 and a lot more bulk and complexity than I wanted. I was really hoping there was a better option.
It so happens that this sort of amalgamation of parts is already pretty commonplace in the form of portable cell-phone chargers. These devices are very little more than a boost converter, charger, lithium battery, and a very basic fuel gauge. Best of all, thanks to economy of scale, they’re really inexpensive. The 3200 mAH battery I used in this project was only $22, and it’s something I can reuse for multiple projects… or even to charge my phone.
Bill of Materials
This lists the exact parts I used in my prototype, but nearly everything here is commonly available from multiple manufacturers. In particular, many different vendors on eBay usually carry CC2500 breakout boards. Some of these have special features such as nicer antennas or power amplifiers. Keep an eye out for those.
- Arduino Pro Mini (3.3v)
- CC2500 radio module with SMA antenna (eBay)
- Anker SlimTalk external battery pack
- Parallax Ping ultrasonic distance sensor
- SparkFun serial 7-segment LED display
- Printed plastic enclosure
- Two M3x16 bolts and nuts (Update: The latest design in GitHub uses M3x25 bolts instead.)
- PCB-mount USB type A plug (These can be recycled from older USB thumb drives, or buy them new.)
- Thin hookup wire (I used Kynar-insulated AWG 30 wire-wrapping wire)
- Snappable 0.1″ headers
- 2-part epoxy or plastic adhesive
That’s it for the list of vitamins we’ll need to make the remote. There is no “main” PCB for this project. The small boards are all held in place by a custom-designed plastic enclosure.
At this point, I did a little bit of preparation on the Arduino by soldering the FTDI-compatible serial header, and disabling the pin-13 LED by desoldering its current limiting resistor. If it was still attached, its glow would be visible through the plastic enclosure. You may also want to remove the LED on the Ping module. I forgot to do this on my prototype, and its blinking is just barely visible through the plastic.
It’s easy to neglect the non-electronic parts of any electronics project. I’ve certainly built my share of prototypes that were little more than a bare circuit board. In this project, though, the plastic parts serve multiple purposes: providing a slot for the battery pack, holding the individual circuit boards in place, and providing a smooth exterior that’s a little less unfriendly to use in the bedroom.
This seemed like a perfect job for 3D printing. I could make a plastic enclosure that exactly fits each of the circuit boards, and presents the LEDs and sonar transducers nicely while hiding all of the sharp and uncuddly circuitry inside.
I chose to model the enclosure in Blender, a wonderfully powerful (if somewhat intimidating) open source tool. It takes some care to use Blender for CAD, but I enjoy having such diverse functionality integrated into one package, and the price is right.
To keep the model as parametric as possible, I modeled the shell and the negative space separately. First, I created meshes for each of the internal parts: The circuit boards, bolts, antenna hole, battery pack, USB connector. These meshes would become the negative space inside the enclosure. I separated them into two layers, for items that would mount to the top and to the bottom half of the case. On the top-half items, I extruded them downward, and the bottom-half items were extruded upward. This leaves the space toward the inside of the case hollow.
I modeled the outer shell using subdivision surfaces. A chain of boolean modifiers will non-destructively split the case into top and bottom, then carve out all holes. The downside to this technique is that all of the boolean operations can bog down Blender quite a bit. This can be mitigated by keeping your source meshes and your final meshes in different layers. With the final layers hidden, Blender won’t continuously recalculate them as you edit the source meshes.
The end result of all this 3D modeling was a set of polygonal meshes in STL format for the top and bottom halves of the case:
I used Slic3r to convert the polygon models into G-code, a simple language that defines the actual tool path used by the printer. This program slices the 3D model into many thin layers. The 3D printer will draw out each layer with a thin filament of molten plastic. After each layer, the nozzle moves up a little and the process repeats.
This is a visualization of the tool path for just one of these thin layers. The tight zig-zag pattern is a solid fill, but much of the interior is filled with a light-weight honeycomb pattern that saves material and time:
Over the course of several hours, my printer builds each part up from nothing.
A tiny bit of post-printing cleanup helps the parts assemble smoothly. A hobby knife makes quick work of the “brim” that is added along the bottom of each part to help it stick to the build surface. Since this will be a handheld project, it’s important to sand any sharp corners.
I also sanded the entire top face of each model, to eliminate any small bumps that would prevent the two halves from sitting flat against each other. This is a good time to test-fit the pieces with a pair of bolts and nuts. The bolts should go in easily, the nuts should be held in place by the hexagonal holes on the bottom half, and there should be only a very tiny gap between the two halves.
Each component sits in its own custom-fit hole, with some epoxy to hold it in place. All of the components on the bottom half need to be robustly anchored. The radio’s antenna connector, the battery pack USB plug, and the Arduino’s serial header will all be subject to some mechanical force during normal use.
I used a liberal dose of epoxy both under and above the USB plug. The Arduino and Radio boards both have flat backsides, so I put a dab of epoxy underneath both. On the radio, I added some additional dabs of epoxy at the corners near the SMA connector.
It’s probably a good idea to avoid getting any epoxy on the RF components on the front side of the radio, as it may affect the circuit’s impedances and lead to reduced radio range. Also try to keep it off of anything you’ll be soldering. Solder won’t stick to it or burn through it, you’ll just be left with a mess that you have to scrape off with a hobby knife.
I used the battery pack itself to help line up the USB connector while the epoxy sets. You’ll notice a little bit of wiggle when the connectors are mated. To make sure the battery pack plugs and unplugs smoothly, make sure that you seat the USB plug such that it’s pressed all the way to the back of the socket. This will ensure the plug isn’t stuck at a funny angle.
The top half assembles in a similar manner. The sonar and LED modules both press into their respective slots. When they’re all the way in, the front face of the LED module and the front of each sonar transducer should be approximately flush with the front of the enclosure.
Since these components are largely held in place by the shape of the enclosure, they only need a small amount of epoxy to keep them from sliding out of their holes. Note that the underside of the sonar module is exposed to the battery compartment. Make sure you go easy on the epoxy. Any blobby epoxy or messy wiring could get in the way of the battery pack.
After the epoxy set, I soldered everything up point-to-point style with wire-wrapping wire:
There will be six wires running between the top and bottom half. It’s okay to leave these a little bit long; there will be room to fold them up in the hollow space above the LED module.
Bolt the enclosure together, making sure the wires end up in the hollow area instead of pinched in the edges of the controller. Now it’s ready for some firmware! The Arduino sketch is in the project’s GitHub repository.
I designed the enclosure to work with common 3.3v FTDI cables. The programming slot is probably a bit too narrow for the FTDI Basic, but perhaps it works. I already had a Prop Plug handy, so I’ve been using that with a simple passive adaptor.
The firmware is currently pretty basic. It takes sonar measurements as fast as it can, feeding those into a median filter. Median filters are a little bit magical when it comes to discarding outliers from noisy-ish data. There’s a smidgen of state machinery to manage the “lock” mode. Finally, it scales the distance reading to a motor power level and sends packets to the radio and LED modules about 80 times a second.
The Next Thing
Where to go from here? Well, there is certainly room for improvement in the firmware. For different kinds of play, it may make sense to have different scaling algorithms for converting distance to intensity. I’m interested in making the firmware more versatile, but not at the expense of reducing its intuitive quality. The “Lock” mode is already way too unintuitive for my tastes. Perhaps an additional flavor of user interface, via an accelerometer or SoftPot would help.
But honestly, the thing I’m most excited about improving isn’t even technical. In some ways, this kind of toy feels like a musical instrument. It is a simple machine with very few inputs, but it interacts with your body in such a way that it opens up a broad array of techniques that can each be mastered. I’m looking forward to spending more time with it and learning how to play.
I’ve already been thinking about the vast array of other sensing technologies that may be applicable for sex toys that support your own feedback loops instead of obstructing them. What if you could use a Kinect camera to remotely detect even your body’s subtler rhythms? Imagine using a phase-locked loop to not just synchronize with your motion, but predict it. The PLL could compensate for all of the system’s lag, including the mechanical lag in the vibrator motor.
There could be a lot more to electronic sex toys than just a battery and a motor. I want the future to be full of toys that know how to play.
I’m not even sure why I built this at all. I guess I did always have a childhood fascination with making things that seemed professional in some way. A simulacrum of some expensive piece of A/V equipment, or simple computer games that came in a shoe box with construction-paper cover art.
This is another ancient project I dug up during my recent move to San Francisco: A rather gigantic “smart” remote control for X-10 home automation devices.
I’m only posting this because it seems cute in some ways… like finding one of your childhood finger-paintings. But it was really an awful project in pretty much every way. It was over-engineered, terrible at the job it was designed to do, and of course the battery life was pretty bad.
The extremely clunky user interface is based on a numeric keypad recycled from a very old television set, and a 16×2 alphanumeric LCD. The X-10 codes are transmitted via a reverse-engineered “Firecracker” board. A Basic Stamp 2 controls it all. This is one of my earlier microcontroller projects, from way back in 1999.
[flickr id=”5921077038″ thumbnail=”medium_640″ overlay=”false” size=”small” group=”” align=”none”]
[flickr id=”5920516143″ thumbnail=”medium_640″ overlay=”false” size=”small” group=”” align=”none”]
[flickr id=”5921079728″ thumbnail=”medium_640″ overlay=”false” size=”small” group=”” align=”none”]
[flickr id=”5921081670″ thumbnail=”medium_640″ overlay=”false” size=”small” group=”” align=”none”]
[flickr id=”5921082132″ thumbnail=”medium_640″ overlay=”false” size=”small” group=”” align=”none”]
[flickr id=”5921083104″ thumbnail=”medium_640″ overlay=”false” size=”small” group=”” align=”none”]
[flickr id=”5921082814″ thumbnail=”medium_640″ overlay=”false” size=”small” group=”” align=”none”]
A few years ago, I implemented an S/PDIF encoder object for the Parallax Propeller. When I first wrote this object, I wrote only a very terse blog post on the subject. I rather like the simplicity and effectiveness of this project, so I thought I’d write a more detailed explanation for anyone who’s curious about the gritty details.
The source code is open, under an MIT-style license. If you’re a fellow Propeller fan, it’s pretty easy to use this code to give your next sound project a digital output. If not, read on… perhaps you will be inspired to try exploring digital audio on a different microcontroller platform!
Digital Audio Primer
Starting from the very basics… what is S/PDIF, and why would we even want to generate it directly from a microcontroller?
Microcontroller audio projects are getting more and more popular, especially as legions of Arduino hackers build DIY drum machines, noise makers, 8-bit synthesizers, and so on. Many of these bit-bang low-fi audio in software. Some of them use external analog synthesizers, MP3 decoders, or other support ICs. If you wanted high-fidelity audio, though, your options get more limited. Some microcontrollers (like the Propeller) can perform PWM at a high enough frequency to produce reasonable audio quality. But this is still no match for an external DAC, much less a high-quality external DAC. So, if you’re really trying to produce higher quality audio without a lot of extra fuss or expense, it makes some sense to let someone else do the job.
If you have a hi-fi stereo receiver, you already have an external DAC and a good way to communicate with it. Nearly all consumer audio receivers now include digital audio inputs based on the Sony/Phillips Digital Interconnect Format (S/PDIF) standard. This consumer standard is actually a variant of the professional AES3 standard. Electrically, this is a high speed unidirectional serial link with a clock that runs at a high multiple of the audio sample rate. The physical transport can be a low-voltage signal over 75-ohm coax, or it could be optical. Optical interconnects (with TOSLINK connectors) are especially common, and to transmit these signals all you need is an LED.
For every audio sample. this digital signal transmits all of the bits in the sample, as well as some control information. Also, quite importantly in fact, it transmits the timing of these samples. The DAC synchronizes its conversion cycles to the time-of-arrival of each sample that comes over the digital bitstream. So, the analog timing characteristics inherent in this digital signal can also influence the resulting analog signal.
Sound nerds tend to get fabulously stressed out over jitter and wander and so on— names for different kinds of deviations from optimal bit timings. It’s good to keep in mind that, by nature, S/PDIF is a much more real-timey sort of signal than your average serial data link. But unless you’re the sort of hopeless audiophile who spends more on your amplifier power cables than I’d spend on a car, you probably shouldn’t get too bent out of shape over a few nanoseconds here or there.
S/PDIF Data Format
S/PDIF uses a serial signal clocked at 64x the audio sample rate. So, for 48 kHz audio, we need a serial signal at a whopping 3.072 megabits per second! However, the receiver doesn’t just need the bits, it also needs a clock. Since we only have one electrical or optical signal to work with, both the clock and data have to be recoverable from this one signal. S/PDIF does this using Biphase Mark encoding, which is a close relative of Manchester code. Because of this coding, we actually have to transmit on a clock rate which is 2x the bit rate. So, again for 48 kHz audio, we need a transmit clock of 6.144 MHz.
But wait, why 64x the sample rate? Even if we’re transmitting stereo audio at 24 bits per sample, that’s only 48 bits. Where do the other bits go?
Some of them are used by S/PDIF for synchronization purposes, and some are used for a low-frequency signaling channel which can transmit status words at a rate much slower than the audio sample rate, and some are effectively useless to us, reserved for obsolete or infrequently-used standards. The most primitive grouping of bits understood by S/PDIF is a 32-bit subframe which encodes one sample for one audio channel:
So many bits… what does it all mean???
- The preamble identifies the type of subframe, as we’ll see below. It is the only part of the stream which is not biphase-mark encoded. It is the only place where we’ll see a run of three clock cycles with no bit transitions, so this allows the receiver to uniquely identify the preamble within the received bitstream.
- Each subframe includes 24 bits of audio data, transmitted LSB-first and biphase-mark encoded. The low 4 bits of this stream may be used for other purposes, depending on which standard you’re reading.
- The Valid bit indicates that this subframe contains valid sample data, and it is okay to output. In practice this bit isn’t really useful, since with the advent of Dolby compressed audio over S/PDIF, the receiver has a lot more work to do in order to determine if the data is valid uncompressed audio.
- The User and Control bits are both part of a lower-bandwidth serial stream that we’ll see later.
- And finally, each subframe has a Parity bit to help detect single-bit errors.
The User and Control bits on each channel collectively form four low-bandwidth serial channels, each running at a rate of one bit per sample. In S/PDIF, the User bit is unused, and the Control bits on each channel transmit a 192-bit Channel Status word. This word is fully transmitted once per block where a block is defined as a group of 192 frames beginning with a Z preamble.
In the professional AES3 protocol, there is a lot of data packed into this status word. But S/PDIF uses it for very little. In fact, only 13 of these bits are used at all, and in practice there isn’t really anything useful in this word. AES3 encodes an exact sample rate here, but in S/PDIF the only indication of sample rate is the clock recovered from the S/PDIF bitstream itself.
Biphase Mark Code
Wow, so far this looks pretty easy. Well, except for the high bit rate, and the picky timing. But what about this biphase mark code?
There are a few different ways to think of biphase mark encoding. If you’re familiar with Frequency Shift Keying (FSK) modulation, it might make sense to think of BMC as a particular form of FSK. A string of ones would be encoded as a square wave at a frequency equal to the original bit rate. A string of zeros would be a square wave at half that frequency. Put another way, you can think of BMC in terms of bit transitions. A zero bit is encoded by a transition followed by a non-transition, whereas a one bit turns into two transitions.
This demystifies BMC a bit… but why do it at all? Well, like any protocol which has to travel over some kind of analog physical media, very low-frequency signals (down to and including DC) can be troublesome. Let’s say we’re using an optical TOSLINK cable to transmit S/PDIF, and we have two theoretical bitstreams. One of them always transmits “one” bits, the other always transmits “zero” bits. The first bitstream means the transmitter’s LED is always on, and the second means the LED is always off. How does the receiver tell these two streams apart?
At first it seems obvious. The “one” stream is brighter than the “zero” stream. But actually, this might not be true. Maybe the first stream has a very dim light or a long cable. Maybe the second stream has a light leak around the receiver. It’s unreliable to rely on any absolute amount of light to discriminate ones from zeros, and in fact it’s not that hard to imagine situations where one system’s zero is brighter than another system’s one.
Similar problems exist in many kinds of analog transmission problems. Radio receivers, for example, need to deal with a very wide range of signal strengths. Unbalanced coaxial cables, such as S/PDIF over copper, can face similar problems. The receiver circuit in each of these cases needs to employ some kind of automatic gain control (AGC). AGC circuits track the average power level of the received signal, and “center” the one/zero discrimination threshold around this value. It’s a simplification, but AGC circuits can also be thought of as high-pass filters, since they subtract the unknown DC bias in the received signal.
Since DC signals are removed by the receiver, we can’t use them to carry any useful data. Those hypothetical all-zero or all-one bitstreams would be a disaster, since the receiver would continuously detect a signal level equal to the average. Any tiny amount of electrical noise would be detected as a one or zero.
This is where BMC helps. We can use a simple SciPy simulation to plot unencoded and encoded bitstreams in the frequency domain:
Now it’s easy to see that BMC is in fact shifting the signal up in the frequency domain. It needs twice the bandwidth now, but the center frequency is now near the bit rate, and we no longer have any signal at DC. Hooray, no more grumpy receiver AGC.
Typically if you were generating an S/PDIF signal, it would be sane to use an FPGA or an ASIC. In silicon. But this article is about breaking the mold and doing it in pure software. Why? For fun, and maybe also to lower the barrier to entry on digital audio. There are a few challenges to overcome, though:
- Need to have enough CPU left over to generate the audio signal in the first place
- Very high bit rate for a software implementation
- Strict bit timing, at an unusual frequency
- Biphase mark encoding is not parallelizable
My platform of choice for this project was the multi-core Parallax Propeller, since it’s simple and hobbyist-friendly yet it also has features which directly address these challenges. The XMOS XCore, another parallel microcontroller, would also be a fabulous choice. It may also be possible to implement S/PDIF on a sufficiently fast single-core microcontroller. Unfortunately, an 8-bit µC like the AVR used in the popular Arduino board probably wouldn’t be fast enough.
Edit: Actually, perhaps it would be doable on the Arduino after all… you would just need a less common crystal frequency. To generate an S/PDIF signal with a 32 KHz sample rate, for example, you could run the AVR at 16.384 MHz. The encoded bitstream clock would need to be 4.096 MHz (32k * 64 * 2), and you can program the AVR’s SPI master to transmit at up to half the main oscillator frequency. So you could run the AVR at 8.192 MHz or 16.384 MHz. At the latter frequency, you would have 32 instructions for every 8 bits of encoded bitstream data. That should be enough to do the encoding in an ISR and have a little time left over for applications…
A multi-core microcontroller makes challenge (1) a piece of cake. On the Propeller, one of the eight CPU cores can be dedicated to S/PDIF encoding. The other seven are available for application code, sound streaming or synthesis, and for other I/O devices.
Challenges (2) and (3) can be mitigated if we have a little bit of help from hardware. If we were using traditional bit-banging, and toggling I/O pins in code, a very fast processor would be needed. Even if the encoding and output could be done in four instructions per bit, just the S/PDIF encoding would require a little over 24 MIPS of processing power. Annoyingly, the CPU clock would have to be run at a multiple of the audio bit rate. You would have no way to use a separate clock. But if we had some hardware to shift out bits at the right time, the CPU can spend that time doing other tasks. Many microcontrollers have an SPI port that may be able to do the job. The XCore actually has special-purpose shift register hardware just to help with high-speed I/O tasks like this. And the Propeller has something close enough— a “video generator” that can be configured as a latch and shift register. The Propeller’s video generator can be clocked by a PLL that we program to synthesize the audio bitstream clock.
Challenge (4) means we need at least a small amount of code which runs serially for every bit in the audio bitstream. The problem is similar to calculating parity. Every input bit affects all subsequent output bits. In my implementation, I just use the fastest unrolled loop I can to perform the biphase mark encoding in two instructions per bit. Here’s an excerpt from the meatiest part of the BMC implementation:
' Load the preamble. The preamble is not biphase encoded, ' but it is subject to being inverted if the previous cell ' was a 1. This step is omitted for the second half (second ' 32 cells) of a subframe. ' ' In biphase encoding, every bit unconditionally begins with ' one transition. We can add these transitions too, in the same ' operation. ' ' The masks below select all cells in the biphase register that are ' output after the bit we're currently encoding. Any time we ' XOR the biphase register with the mask, we're creating a ' transition on all future bits. The mask starts at the first ' odd numbered non-preamble bit. xor biphase, preamble ' To actually biphase encode our input data, we'll insert ' additional transitions every time there's a 1 bit in our input. ' For the first half of the subframe, we're processing 12 bits ' of subframe data. (16, minus the 4-bit preamble) ' ' The loop is unrolled, since this is very speed-critical. At ' 48 KHz, we have less than three instructions per bit! rcr subframe, #1 wc ' Extract the next LSB from the subframe if_nc xor biphase, mask_4 ' Insert a transition only for '1' bits. rcr subframe, #1 wc if_nc xor biphase, mask_5 rcr subframe, #1 wc if_nc xor biphase, mask_6 rcr subframe, #1 wc if_nc xor biphase, mask_7 rcr subframe, #1 wc if_nc xor biphase, mask_8 rcr subframe, #1 wc if_nc xor biphase, mask_9 rcr subframe, #1 wc if_nc xor biphase, mask_10 rcr subframe, #1 wc if_nc xor biphase, mask_11 rcr subframe, #1 wc if_nc xor biphase, mask_12 rcr subframe, #1 wc if_nc xor biphase, mask_13 rcr subframe, #1 wc if_nc xor biphase, mask_14 rcr subframe, #1 wc if_nc xor biphase, mask_15 waitvid palette, biphase ' Output the first half of this subframe ' <snip> ' S/PDIF preambles. These are ordered LSB-first, ready for loading into ' 'biphase' before encoding the rest of a subframe. ' ' These are the preamble encodings that occur after a '0' bit. After a '1' ' bit, these preambles are inverted. ' ' All odd-numbered unused bits must be '1', so we can insert the fixed ' transitions in the same operation. preamble_b long %010101010101010101010101_00010111 preamble_m long %010101010101010101010101_01000111 preamble_w long %010101010101010101010101_00100111 ' For speed, we precalculate all XOR masks. mask_0 long %11111111111111111111111111111110 mask_1 long %11111111111111111111111111111000 mask_2 long %11111111111111111111111111100000 mask_3 long %11111111111111111111111110000000 mask_4 long %11111111111111111111111000000000 mask_5 long %11111111111111111111100000000000 mask_6 long %11111111111111111110000000000000 mask_7 long %11111111111111111000000000000000 mask_8 long %11111111111111100000000000000000 mask_9 long %11111111111110000000000000000000 mask_10 long %11111111111000000000000000000000 mask_11 long %11111111100000000000000000000000 mask_12 long %11111110000000000000000000000000 mask_13 long %11111000000000000000000000000000 mask_14 long %11100000000000000000000000000000 mask_15 long %10000000000000000000000000000000
This code really does most of the work. The waitvid instruction waits until the video generator has buffer space available for another 32-bit word, but the video generator is actually clocking out data continuously, without any gaps. Since the biphase mark encoder’s output for one subframe is 64 bits, we split the subframe into two halves and process them each as above. The first half is special, though, since the preamble is not biphase mark coded.
Edit: I should mention that in most cases it probably makes more sense to use a 4-bit or 8-bit lookup table to do the BMC encoding. This approach seemed to make sense on the Propeller. However, for example, the S/PDIF library by XMOS uses a 4-bit table to do the conversion.
Using the SpdifOut Object
To use this SpdifOut object in your own Propeller project, you’ll need another cog to supply data to the S/PDIF cog. The object can receive sound samples one long at a time from hub memory, or you can set up a FIFO buffer for transferring data in more of a bursty fashion. In fact, transferring samples one-at-a-time is really the same thing as creating a one-entry FIFO buffer.
This is a complete example which plays uncompressed audio from an SD card:
CON _clkmode = xtal1 + pll16x _xinfreq = 5_000_000 SPDIF_PIN = 22 SD_CARD_PIN = 0 OBJ sd : "fsrw" spdif : "spdifOut" CON BUFFER_SIZE = 128 ' Must be a power of two VAR long bufA[BUFFER_SIZE] long bufB[BUFFER_SIZE] PUB main | f, c sd.mount(SD_CARD_PIN) sd.popen(string("audio.wav"), "r") spdif.setBuffer(@bufA, BUFFER_SIZE * 2) spdif.start(SPDIF_PIN) repeat ' Wait until the driver is using bufB, then read bufA repeat until spdif.getCount & BUFFER_SIZE sd.pread(@bufA, BUFFER_SIZE * 4) ' Now the opposite... repeat while spdif.getCount & BUFFER_SIZE sd.pread(@bufB, BUFFER_SIZE * 4)
Since we read from the SD card in large blocks, this code uses a double-buffering scheme. While we’re reading one block from the SD card, the other block is being played by the spdifOut module. To represent these two buffers as a FIFO for spdifOut, we just place them consecutively in memory. We can tell which buffer spdifOut is currently playing by looking at the low bits of its played-sample count.
A note about the WAV header: Any modern S/PDIF receiver will actually mute the received audio for a fraction of a second, while it detects whether the bitstream is using Dolby Digital compression. Older receivers without this feature would be in danger of damaging the speakers or amplifier if anyone mistakenly sent them a compressed bitstream they couldn’t handle. Assuming your receiver has this feature, there’s nothing to worry about. If you do have a receiver which starts playing the very first sample you get, you’ll need to be much more careful about the initial conditions. For example, you won’t want to start the S/PDIF cog until the buffer has some valid data in it.
If you’re writing an audio synthesizer, instead of producing big blocks of data, you’re probably producing samples one-at-a-time. This is a very simple sawtooth-wave synthesizer written in assembly. It uses a single long as its buffer, just enough to hold one signed 16-bit sample for each of the two stereo channels. A little bit of Spin code controls the synthesizer cog’s frequency in order to play a short riff:
CON _clkmode = xtal1 + pll16x _xinfreq = 5_000_000 SPDIF_PIN = 22 BPM = 120 ' Tempo ATTENUATION = 4 ' Power of two SAMPLE_FP = $17C6F ' Sample rate, fixed point ($100000000 / 44100) OBJ spdif : "spdifOut" VAR long buffer long countPtr long rate PUB main | songPtr count_addr := spdif.getCountAddr rate_addr := @rate spdif.setBuffer(buffer_addr := @buffer, 1) spdif.start(SPDIF_PIN) cognew(@synth, 0) repeat songPtr := @song repeat while rate := WORD[songPtr] * SAMPLE_FP songPtr += 2 waitcnt(cnt + clkfreq*60/BPM) DAT org 0 synth rdlong t1, count_addr ' Wait for the sample count to change cmp t1, spdif_count wz if_z jmp #synth mov spdif_count, t1 ' This is a sawtooth-wave synthesizer. "rate" determines the ' current tone frequency, and the high bits of "accumulator" ' are used to generate a signed 16-bit audio sample. rdlong t1, rate_addr ' Load wave rate add accumulator, t1 ' Update sawtooth wave mov t1, accumulator ' Chop off low bits... shr t1, #(16 + ATTENUATION) ' and decrease the output volume some sub t1, midpoint ' Convert unsigned to signed samples and t1, cFFFF ' Truncate to 16-bit signed mov t2, t1 shl t1, #16 ' Copy right channel to left or t2, t1 wrlong t2, buffer_addr ' Write the next sample now! jmp #synth count_addr long 0 buffer_addr long 0 rate_addr long 0 spdif_count long 0 accumulator long 0 midpoint long $8000 >> ATTENUATION ' Offset to sample midpoint cFFFF long $FFFF t1 res 1 t2 res 1 fit song word 440, 330, 392, 294, 330, 392, 330, 392, 0
The full source code is available on the Object Exchange or in my Subversion repository. I enjoyed giving one of my favorite microcontrollers a new kind of output device, and I’m looking forward to seeing what others come up with for the Propeller as well as for other microcontrollers.
Enjoy retro N64 games, but can’t stand the controller? That’s the situation I found myself in about 7 years ago, back in 2004. So I built an adaptor, to use Game Cube controllers on the N64.
The adaptor hardware is very simple- all you need is a PIC microcontroller. I originally designed the project to work with the very popular at the time PIC16F84A, or some smaller 8-pin chips. It bit-bangs both protocols, so you don’t need any more hardware than a tiny µC, a couple resistors, and optionally either a voltage booster or battery pack to run the “rumble” vibration motor.
I had a lot of fun building it, since it was an opportunity to reverse engineer a protocol that, as far as I could tell, had never been documented publicly. There were many web sites explaining the basics of the N64 and Game Cube protocols- enough to talk to the controllers with your own hardware. But there was significantly more to learn about emulating an N64 controller, since there are many features that you don’t really need to use the controller, but which games will use. The N64 also has the added complexity of having a memory card slot. The controller implements a protocol that tunnels SRAM reads/writes over the controller wire. Peripherals like the Rumble Pak pretend to be SRAM, but are actually memory-mapped I/O devices.
This is a very old project, but I thought I’d do a quick post with an update on it. I never really released the project, I just quietly posted the source code and the CIA feed. Since then, many others have found the project and built their own.
I recently heard from Jacques Gagnon, who went a step farther. He was frustrated by a few lingering bugs, most notably lack of support for the WaveBird wireless controllers. So he pulled the project into Google Code and has been hacking away! WaveBird controllers work now, and he’s added several other new features, such as the ability to store multiple sets of button mapping layouts in memory.
If you’re still interested in this classic gaming platform and you’d love to have your own Cube64 adaptor, I highly encourage you to check out Jacques’ work. The adaptors are easy to build, especially if you already have some experience with microcontroller programming.
The demo is quite impressive, and he uses some extremely clever tricks to fit it all on a microcontroller with 32kB of EEPROM and no hardware multiply or divide. Linus released the full source code after an exciting reverse engineering contest uncovered many of his techniques. They include decompressing code from EEPROM at runtime, a novel video bus which uses extra I/O pins as fast inter-processor communication, and a nice looking dithering scheme.
Turbulence earned a well-deserved first place in the console/wild category at Breakpoint 2009. Congrats to Linus for showing us what this chip is capable of!
Previously on the bloggy blog, I posted a few of my projects related to home data acquisition and to The Energy Detective (TED), a whole-house power measurement device. I made a set of homebrew wireless temperature sensors that display graphs on a digital picture frame, I reverse engineered the TED protocol, built a small self-contained open source TED receiver, and used that design to make a nifty clock.
So, I wanted to take the next step and set up real-time power usage graphing alongside the temperature graphs. The only problem: My sensor receivers are on a different power line phase than the TED. I’d need a TED receiver that can deal with all of the noise that switching power supplies put out, and receive weak TED signals that have been coupled across phases by the utility’s pole transformer. There were probably easier ways to solve this problem, but it was a good excuse to get more experience with analog electronics 🙂
The end result was this cute little doodad:
- Isolated power supplies for the host computer, receiver circuitry, and the power line signal itself. Separate voltage regulators for analog and digital.
- High-Q LC filter, centered at 125 kHz.
- Dual outputs: RS-232 serial and USB (Prolific USB-serial chip)
- Generates human-readable text at 9600 baud:
HC=245 KW=001.347 V=121.228 CNT=033
- Decodes packets with up to 1 watt / 1 millivolt precision!
- Last but not least, an oh-so-cute power/status LED which glows dimly to indicate power, and flashes brightly when a packet is received.
With all of this precision, the resulting graphs look pretty decent. Here are power and voltage graphs, for the same time period. You can certainly see the voltage dips when my refrigerator or heater fan kick on, but you can also see some loads on the voltage graph that don’t show up on the power graph. These are probably the neighbor’s appliances.
The circuit design is pretty straightforward, and it should be pretty easy to build for anyone an AVR programmer and some experience with hobbyist electronics. Inside the box the mains voltage is split into two 9V AC transformers: one for power, and one for data. The power transformer’s output is rectified, and the unregulated voltage is diverted to two 7805 regulators, one for the microcontroller and one for the amplifier. The amplifier is very sensitive to noise, and it helps to have some isolation from all the high frequency harmonics that a microcontroller will toss into its power supply.
I built the prototype on a piece of stripboard in a grounded aluminum box. From left to right: MCP6292 dual op-amp and all associated passives, power supply, ATtiny85 microcontroller, and some transistors to drive the LED brightness control circuit and the RS-232 output.
The analog front-end starts with a high-pass filter which cuts out nearly all of the 60 Hz AC power, leaving a manageable signal level. This goes through some clamping diodes, to protect the op-amp input from transients, then through a first amplification stage with a gain of about 68. It’s important that this pre-amplifier is non-inverting, to avoid oscillation. This then goes through an LC bandpass filter, and another amplification stage with a gain of about 31.
With these op-amps, you could achieve higher gains (around 80), but this is all I needed. Your mileage may vary. If you want variable gain control, try replacing the 3.3k resistor at the second op-amp’s inverting input with a multi-turn 5k trimmer pot. More ambitious readers may even want to design an AGC circuit. After the second gain stage, a final low-pass filter is very important in order to remove transients that can cause false transitions on the microcontroller’s input. The microcontroller finally gets a filtered and amplified signal. It proceeds to do the demodulation and decoding in software.
Over the weekend, I had a chance to finish up a project that I started (and immediately became distracted from) several weeks ago.
In our house, Paul and I have a game room. This is where the video games live, as well as other assorted geekery. We have Magic cards, D&D books, some manga.. it’s super nerdy 🙂
Best of all, Paul has a Lego city on display. We had been looking for an interesting way to add light to the city, so when I saw some RGB LED light strips for sale at Ikea, I knew I had to mod them. In their stock configuration, these light strips can do boring fully-saturated colors, and you switch between them with a boring push-button switch.
After ripping apart the Ikea light and rummaging through my junk drawers, I came up with this:
The Altoids tin has the modified driver circuit: It’s the original circuit board with the microcontroller removed, then a homemade Arduino clone to control it. The orange box is an old Cirque PS/2 touchpad, removed from its original case and covered in fabric.
The Arduino sketch (firmware) is a little C++ program that reads the touchpad and uses it to control Hue and Lightness in the HSL color space. The result is a pretty intuitive and unobtrusive control which makes it easy to both pick a color and desaturate it toward white or dim it toward black. You can easily get some really nice sunset and sky colors.
I measured the power consumption of the completed light at between 1 and 6 watts. With Bay Area electric rates, this means you’d pay about 7 cents a month to leave it plugged in with the lights fully off, twice that to constantly backlight your Lego city in a dim orange glow, and a maximum of 50 cents a month to run the light at full brightness continuously.
For many more pictures of the final installation and the build process, check out my Ikea DIODER set on Flickr.
My previous entry introduced a homebrew receiver for the powerline-based data protocol used by The Energy Detective. I just designed a second revision of that receiver. This one is self-contained: It gets power and modulated data from a 9V AC wall-wart transformer, and decoded data leaves via an RS-232 serial port at 9600 baud. Best of all the circuit is very simple: Just an 8-pin microcontroller and a single op-amp.
Major changes in this version:
- DC power for the circuit is now provided by the 9V AC input, instead of a separate power supply. Previously this would have caused unacceptable levels of harmonic distortion in the input signal. In the new design, this is mitigated by an inductor (which forms an LC low-pass filter), and by the lower power consumption of a single modern op-amp versus three ancient op-amps.
- By using a simpler filter design and a modern op-amp with a gain-bandwidth product of at least 10 MHz, the bandpass filter and amplifier can be built using only a single op-amp.
Note that the MAX475 op-amp I’m using has been discontinued by the manufacturer, and it’s now hard to find. I just used it because I had one handy in my junk drawer. I’ll verify this design with other op-amps as soon as I can, but it should work with just about any op-amp which can operate on a single-ended 5V supply, and which has a high enough GBW.