Tag Archives: TED


Open source extra-sensitive high resolution TED receiver

Previously on the bloggy blog, I posted a few of my projects related to home data acquisition and to The Energy Detective (TED), a whole-house power measurement device. I made a set of homebrew wireless temperature sensors that display graphs on a digital picture frame, I reverse engineered the TED protocol, built a small self-contained open source TED receiver, and used that design to make a nifty clock.

So, I wanted to take the next step and set up real-time power usage graphing alongside the temperature graphs. The only problem: My sensor receivers are on a different power line phase than the TED. I’d need a TED receiver that can deal with all of the noise that switching power supplies put out, and receive weak TED signals that have been coupled across phases by the utility’s pole transformer. There were probably easier ways to solve this problem, but it was a good excuse to get more experience with analog electronics 🙂

The end result was this cute little doodad:


  • Isolated power supplies for the host computer, receiver circuitry, and the power line signal itself. Separate voltage regulators for analog and digital.
  • High-Q LC filter, centered at 125 kHz.
  • Dual outputs: RS-232 serial and USB (Prolific USB-serial chip)
  • Generates human-readable text at 9600 baud:
    HC=245 KW=001.347 V=121.228 CNT=033
  • Decodes packets with up to 1 watt / 1 millivolt precision!
  • Last but not least, an oh-so-cute power/status LED which glows dimly to indicate power, and flashes brightly when a packet is received.

With all of this precision, the resulting graphs look pretty decent. Here are power and voltage graphs, for the same time period. You can certainly see the voltage dips when my refrigerator or heater fan kick on, but you can also see some loads on the voltage graph that don’t show up on the power graph. These are probably the neighbor’s appliances.

The circuit design is pretty straightforward, and it should be pretty easy to build for anyone an AVR programmer and some experience with hobbyist electronics. Inside the box the mains voltage is split into two 9V AC transformers: one for power, and one for data. The power transformer’s output is rectified, and the unregulated voltage is diverted to two 7805 regulators, one for the microcontroller and one for the amplifier. The amplifier is very sensitive to noise, and it helps to have some isolation from all the high frequency harmonics that a microcontroller will toss into its power supply.

I built the prototype on a piece of stripboard in a grounded aluminum box. From left to right: MCP6292 dual op-amp and all associated passives, power supply, ATtiny85 microcontroller, and some transistors to drive the LED brightness control circuit and the RS-232 output.

The analog front-end starts with a high-pass filter which cuts out nearly all of the 60 Hz AC power, leaving a manageable signal level. This goes through some clamping diodes, to protect the op-amp input from transients, then through a first amplification stage with a gain of about 68. It’s important that this pre-amplifier is non-inverting, to avoid oscillation. This then goes through an LC bandpass filter, and another amplification stage with a gain of about 31.

With these op-amps, you could achieve higher gains (around 80), but this is all I needed. Your mileage may vary. If you want variable gain control, try replacing the 3.3k resistor at the second op-amp’s inverting input with a multi-turn 5k trimmer pot. More ambitious readers may even want to design an AGC circuit. After the second gain stage, a final low-pass filter is very important in order to remove transients that can cause false transitions on the microcontroller’s input. The microcontroller finally gets a filtered and amplified signal. It proceeds to do the demodulation and decoding in software.

I apologize for the crude hand-drawn schematic. There are more pictures on Flickr, and software/firmware in Subversion. Happy hacking!


Self-contained TED receiver

My previous entry introduced a homebrew receiver for the powerline-based data protocol used by The Energy Detective. I just designed a second revision of that receiver. This one is self-contained: It gets power and modulated data from a 9V AC wall-wart transformer, and decoded data leaves via an RS-232 serial port at 9600 baud. Best of all the circuit is very simple: Just an 8-pin microcontroller and a single op-amp.

Major changes in this version:

  • DC power for the circuit is now provided by the 9V AC input, instead of a separate power supply. Previously this would have caused unacceptable levels of harmonic distortion in the input signal. In the new design, this is mitigated by an inductor (which forms an LC low-pass filter), and by the lower power consumption of a single modern op-amp versus three ancient op-amps.
  • By using a simpler filter design and a modern op-amp with a gain-bandwidth product of at least 10 MHz, the bandpass filter and amplifier can be built using only a single op-amp.

Note that the MAX475 op-amp I’m using has been discontinued by the manufacturer, and it’s now hard to find. I just used it because I had one handy in my junk drawer. I’ll verify this design with other op-amps as soon as I can, but it should work with just about any op-amp which can operate on a single-ended 5V supply, and which has a high enough GBW.

Firmware and schematics (PNG and EAGLE formats) are in Subversion and more info on the theory of operation was presented in my previous blog entry.


Interfacing with The Energy Detective

I recently bought The Energy Detective (TED), a pretty inexpensive and friendly way to keep tabs on your whole house’s electricity usage. It’s a lot like having a more featureful version of your utility company’s power meter, sitting on your kitchen counter. It can estimate your utility bill, and tell you how much electricity and money you’re using in real-time. The resolution is pretty good- 10 watts, 1 second.

As a product, I’ve been pretty happy with it. It does what it claims to, and the measurements seem to be fast and accurate. Of course, being the crazy hacker I am, I wanted to interface TED with other things. I don’t have a home automation system as such, but I did want to get real-time graphs of my electricity usage over various time periods. I also wanted the possibility to use TED as an information feed for various other display devices around the house. (But that’s a topic for another blog post…)

The stock TED package consists of two pieces: A measurement/transmit unit (MTU) and receive/display unit (RDU). The MTU has a pair of current transformers, and it installs entirely in your house’s breaker panel. It takes the power readings every second, and transmits them over the power lines to the RDU, which is just a little LCD panel that plugs into the wall. The RDU has a USB port, which you can use with TED’s “Footprints” software.

So, hoping that the USB port would do what I want, I bought the Footprints software for $45. The TED system itself is, in my opinion, a really good value. The Footprints software is not. As a consumer, I was disappointed by two main things:First of all- the UI lacks any polish whatsoever. It looks like a bad web page from the 90’s. Second of all, the data collection is not done in hardware, it’s implemented by a Windows service. This means you can’t collect data to graph unless you have a Windows PC running. Not exactly a power-efficient way to get detailed power usage graphs.

As a hobbyist, a few more things frustrated me about Footprints. The implementation was pretty amateurish. Their Windows service runs a minimal HTTP server which serves up data from an sqlite database in XML. The front end is actually just a Flash applet masquerading as a full application. Energy Inc, the company behind TED, has an API for Footprints: but you have to sign a legal agreement to get access to it, and I wasn’t able to get any details on what the API does and doesn’t include without signing the agreement. So, I opted not to. It would be much more fun to do a little reverse engineering…

So, I did. The end result is that I now have two ways of getting data out of my TED system.

Using the USB port

The TED RDU’s USB port is actually just a common FTDI usb-to-serial adapter. The RDU sends a binary packet every second, which includes all of the data accessible from the Footprints UI. This includes current power usage, current AC line voltage, utility rates, month-to-date totals, and anything else you’ve programmed into your RDU.

There has been some prior work on reverse engineering this protocol. The Misterhouse open source home automation project has a Perl module which can decode the TED messages.

Unfortunately, the Perl module in Misterhouse won’t work with more recent versions of the RDU like mine. The recent RDUs have a different packet length, and they require a polling command to be sent before they’ll reply with any data.

I found the correct polling command by snooping on Footprints’ serial traffic with Portmon. I also noticed a packet framing/escaping scheme, which explains some of the length variations that would have broken the module in Misterhouse.

The result is a Python module for receiving data from a TED RDU. It isn’t terribly featureful, but it should be pretty robust.

Direct from the wall socket

Now for the more exciting method: What about reading data directly from the power line, without using the TED receive/display unit at all? This could provide some exciting opportunities to embed a small and cheap TED receiver inside of other devices, and it would provide some insight on what exactly is being transmitted by the box in my breaker panel.

The TED RDU is pretty simple internally: A Dallas real-time clock, PIC18 microcontroller, chip-on-glass LCD, some buttons, and the TDA5051A, a single-chip home automation modem from Philips. This chip can receive and transmit ASK modulated signals at 1200 baud, with a carrier frequency of around 132 kHz.

Digi-key carries the TDA5051A, but I figured it would be more educational (and more hobbyist-friendly) to try and build a simpler receiver from scratch using only commonly available parts. The result is the following design, with an 8-pin AVR microcontroller and three op-amps:

Update: There is now a Revision 2 of the schematic, which uses only a single power supply and one op-amp.

  1. The power line is sampled via a 9V AC wall wart. With this design, it needs to be a separate isolated power supply. So far, I haven’t had any luck with using this same transformer to power the circuit. Any rectifier introduces high-frequency harmonics which drastically degrade the signal-to-noise ratio.
  2. The first stage is a 10x amplifier and 138kHz band-pass filter. This is from Texas Instrument’s “Filter Design in Thirty Seconds” cheat-sheet.
  3. The next stage is an amplifier and high-pass filter, to remove the last remnants of 60 Hz ripple.
  4. The third stage is just for amplification. In a design which used higher quality op-amps, this stage may not be necessary.
  5. After the third op-amp, the signal passes through an RC network which AC couples it, filters out high frequency noise which could cause glitches in the microcontroller’s I/O pin, and limits the current into the micro’s clamping diodes.
  6. At this point, we have an amplified digital signal which is still ASK-modulated:

The rest of the work happens in software. The ATtiny85 keeps itself quite busy:

  1. The AVR first applies a narrow digital band-pass filter, to strip out any ringing or other noise that remains after the analog band-pass filter.
  2. Next, a digital low-pass filter. This passes the 1200 baud serial data, but rejects higher frequency glitches.
  3. A threshold is applied, with hysteresis, to convert this data into a stream of ones and zeros.
  4. Next is a relatively typical software serial decoder, with majority-detect. The ASK modulated data has 8 data bits, 1 start bit, and 2 stop bits. Polarity is slightly different than typical RS-232. “1” is the presence of an ASK pulse, “0” is the absence of a pulse. Start bits are “1”, stop bits are both “0”.
  5. Using this serial engine, we receive an 11-byte packet.
  6. The packet is converted from TED’s raw format into a more human-readable (but still machine-friendly) serial format.
  7. The reformatted data is finally output via a software serial port at 9600 baud.

The end result, as viewed from a PC connected to the serial output pin:

HC=245 KW=001.324 V=121.138 CNT=023
HC=245 KW=001.335 V=121.135 CNT=024
HC=245 KW=001.348 V=121.072 CNT=025
HC=245 KW=001.345 V=121.021 CNT=026
HC=245 KW=001.345 V=121.044 CNT=027
HC=245 KW=001.324 V=121.152 CNT=028
HC=245 KW=001.314 V=121.280 CNT=029
HC=245 KW=001.314 V=121.306 CNT=030
HC=245 KW=001.311 V=121.297 CNT=031
HC=245 KW=001.349 V=121.232 CNT=032
HC=245 KW=001.347 V=121.228 CNT=033

The “KW” and “V” columns are self-explanatory. “HC” is my TED’s house code, and “CNT” is a packet counter. Normally in increments by one. If it skips any numbers, we missed a packet.

But wait, what’s wrong with this picture? The power and voltage readings have too much precision. The standard TED display unit will give you a resolution of 10 watts for power, and 0.1 volt. As my data above shows, the TED measurement unit is actually collecting data with far more precision. I can only guess why TED doesn’t give users more precision normally. I suspect they removed it because the extra precision may imply extra accuracy that may not exist.

So, what is TED actually sending? Once a second, it broadcasts an 11-byte packet over your power lines at 1200 baud:

  • Byte 0: Header (always 0x55)
  • Byte 1: House code
  • Byte 2: Packet counter
  • Byte 3: Raw power, bits 7-0
  • Byte 4: Raw power, bits 15-8
  • Byte 5: Raw power, bits 23-16
  • Byte 6: Raw voltage, bits 7-0
  • Byte 7: Raw voltage, bits 15-8
  • Byte 8: Raw voltage, bits 23-16
  • Byte 9: Unknown (Flags?)
  • Byte 10: Checksum

Pretty straightforward. I don’t know the actual A/D converter precision in the measurement/transmit unit, but both the power and voltage readings are sent on the wire as 24-bit raw numbers. I’m still not sure what byte 9 is. In my measurements, it hovered around 250, sometimes jumping up or down by one. It may be some kind of flag byte, or maybe it measures power line frequency? The checksum is dead simple: Add every byte in the packet (modulo 256), and the checksum byte ensures the result is zero.

To figure out what to do with these raw measurements, I collected data simultaneously with my circuit and with the TED RDU’s USB interface. Both sets of results went into a spreadsheet. After removing outliers, I did a linear regression. The resulting linear function is what you’ll find in the current firmware for my homebrew receiver. The following plots from the spreadsheet are a pretty striking illustration of the additional precision available via my raw interface:

The top graph shows power line voltage as recorded by the TED RDU. The second graph shows the raw values I’m receiving from the TED MTU. The bottom graph shows the correlation between the two, in blue, and my linear regression, in red.

I plan to keep improving the circuit. Hopefully there’s a way to get both data and power from a single supply, without dealing with any annoying high-voltage circuitry. If there is any interest, I might make a kit available. If you’re interested, post a comment and let me know what features you’d like. USB port? Serial port? Display?


Measuring energy usage

Lately I’ve been interested in measuring whole-house power consumption. I’d like to know where my electric bill is going, how often I forget to turn the lights off, how much the CFL bulbs are actually helping, etc. Besides, I really like measuring things in real-time and displaying them in interesting ways 😉

So, I’ve been pondering various ways to do this. Our house has a mechanical power meter out front, so I considered building an optical sensor that can sense the black line on the meter’s rotating disc. Apparently others have tried this, with limited success.

The other thought was to attach a clamp-on current meter in our breaker box. I was researching this idea when I stumbled upon a finished product that is pretty much exactly what I wanted to build- and for less than the price of a Fluke clamp-on attachment.

It’s called The Energy Detective. It has a pair of clamp-on probes combined with a power-line communications module that you install in your breaker box. Then there’s a display unit, which communicates over the power mains (like X10 or HomePNA) to make voltage and current readings from the sensor in the breaker box. It even has a USB interface. Their midrange model is $145, which is certainly less than I’d spend on the parts to build a similar unit from scratch.

Naturally, I had to order one. A review shall be forthcoming.