Chris Mullins

I recently moved from SmartThings to HomeAssistant. One of the things I didn’t have to think about too much with SmartThings was how to authenticate all of my connected devices (laptops, phones, tablets, etc.) with my HA platform. I wanted to find a good balance between security and convenience with HomeAssistant.

HomeAssistant makes it easy to secure your install with a password. Coupled with TLS, this is pretty solid. But there’s just something about the idea of a publically facing page that anyone on the Internet can get to, protected with nothing but a password that made me feel uneasy.

Client certificates are a very robust authentication mechanism that involves installing a digital certificate on each device you wish to grant access to. Each certificate is signed by the server certificate, which is how the server knows that the client is valid.

This feels nicer than HomeAssistant’s built-in security measures to me for a few reasons:

1. Individual client certificates can be revoked. You don’t have to configure authentication on every device you own if someone loses their phone.
2. While I highly doubt there are any issues with HomeAssistant, I feel more confident in nginx and openssl.
3. Unless you add a passphrase to the client certificates (I didn’t), the whole thing is passwordless and still manages to be pretty darn secure.
4. If I ever became truly paranoid, I could turn on HomeAssistant’s password protection and my HA dashboard would essentially need two authentication factors (the SSL cert + the password).

While I did find this approach more appealing, there are several drawbacks:

1. It’s way harder to set up. You need to run a bunch of openssl commands, and install a certificate on each device you want to grant access to.
2. The HomeAssistant web UI requires WebSockets, which seem to not play nicely in combination with client certificates on Safari or iOS devices. My household has iOS users, so this was something I needed to figure out.

I think I managed to get this working. The only disadvantage is that clients are granted access for an hour after successfully authenticating once. The basic approach is to tag authenticated browsers with an access token that’s good for a short period of time, long enough for them to establish a WebSocket connection. I’ll go through the steps in setting this up.

What you need

1. Install packages:  sudo apt-get install nginx nginx-extras lua5.1 liblua5.1-dev
2. openssl
3. luacrypto module, which exposes openssl bindings in lua.

luacrypto was kind of a pain to install. Here’s what I did to get it working with my nginx install. It involved patching configure.ac (thanks to this very helpful StackOverflow post for the tip):

Setting up a Certificate Authority

There are already good guides on doing this. I recommend this one. In this guide, I’m using the default_CA parameters pre-filled by openssl on my system.

Generate client certificates

I put a script in /usr/local/bin  to make this easier:

You then run this for each device you want to grant access to:

Make sure to supply an export password. Generated certificate files will be placed in /etc/ssl/ca/certs/users .

Get the certificates on the devices

The .p12 file is the one you want. Make sure to not compromise the certificates in the process.

I rsynced the files to my laptop and attached them to a LastPass note, which I could access on my devices. On most devices, you should be able to just open the .p12 file and it’ll do what you want.

On iOS devices, I needed to serve the certificates over HTTPS on a trusted network because they needed to be “opened” by Safari in order to be recognized.

Configure nginx

Here’s my nginx config. You’ll need to substitute your domain and SSL certificate parameters:

If this all worked, you should be able to access my-homeassistant-install.mydomain.com from a device with a client certificate installed, but not otherwise. Unfortunately if you’re using iOS or Safari, you’ll probably notice that the page loads, but you get a forever spinny wheel. If you look in the debugger console, you might see messages that look like this:

This is because the browser isn’t sending the client certificate information when trying to connect to the WebSocket and is therefore failing.

Fixing compatibility with iOS/Safari

Safari does actually send client cert info along with the initial request. Nginx has a really cool module that allows you to insert all sorts of fancy logic with lua scripts. I added one that tags browsers supplying a valid client certificate with a cookie granting access for about an hour. This worked really well. Since this is all over HTTPS, and the access tokens are short-lived, I felt pretty comfortable.

The easiest way I could think of to create a cookie that was valid for a limited time was to use an HMAC. Basically I “sign” a hash of the client’s certificate along with an expiry timestamp. The certificate hash, expiration timestamp, and HMAC are all stored in cookies. Nginx can then validate that the expiration timestamp is in the future, and that the HMAC signature matches what’s expected.

You’ll notice the commented-out line in the nginx config above. Uncomment it:

And add the script:

Update (August, 2019)

After upgrading to Ubuntu 18.04, the nginx LUA module stopped working, and every combination of fixes I tried didn’t work.

Fortunately, OpenResty worked out of the box. After adding their PPA and installing it, all of my existing scripts worked. I just copied over my entire nginx config.

My last post went over an ESP8266-based wifi gateway for Milight/LimitlessLED bulbs. This supports a few kinds of bulbs that have been around for a couple of years (e.g., this one).

About a year ago, newer bulbs and controllers started showing up that used a different 2.4 GHz protocol. This introduced some scrambling that made it difficult to emulate many devices. This was presumably done intentionally to prevent exactly the sort of thing that my last project accomplished (boo!).

The new bulbs actually have some really cool features that none of the old ones do, so there’s some incentive to figure this out. In particular, they support saturation, which allows for ~65k (2**16) colors with variable brightness instead of the 256 colors that the old one does. They also combine RGB and CCT (adjustable white temperature) in one bulb, which is super cool.

A few others have dug into this a little, but as far as I’ve been able to tell, no one has figured out (or at least shared) how to de-scramble the protocol. I think I’ve managed to do so. I should mention that I don’t have much experience doing this kind of thing, so it’s entirely possible the structure I’m imposing is a lot more complicated than what’s actually going on. But as far as I’ve been able to tell, it does work. I’ve tested with five devices – four remotes and one wifi box.

I’m going to start by detailing the structure, and I’ll follow up with some of the methodology I used to reverse the protocol.

Differences from old protocol

From a quick glance, there are a few superficial differences between the new and old protocols:

1. Listens on a different channelset (this was true of different bulb types among the old bulbs too). The new bulbs use channels 8, 39, and 70.
2. Different PL1167 syncword. It uses 0x7236 , 0x1809 .
3. Packets have 9 bytes instead of 7.
4. Packets are scrambled. The same command can look completely different.

The scrambling is the tricky part. As others who have stared at packet captures noticed, when the first byte of packets for the same command is held fixed, most of the other bytes stay fixed too. This suggests that the first byte is some kind of scramble key. Turns out this is the case.

Example packets for turning group 1 on with one of my remotes:

Notation

I’ll for a packet p, I’ll use p_i to refer to the 0-indexed ith byte in the packet. For example, p₀ refers to the 0th byte.

I’ll use p’ to refer to the scrambled packet for a packet p.

Structure

The 9 bytes of the packet are:

p0	p1	p2	p3	p4	p5	p6	p7	p8
Key	0x20	ID1	ID2	Command	Argument	Sequence	Group	Checksum

Packet scrambling

The designer of this protocol added in quite a few things to complicate reversing it. None of them are particularly hard on their own, but with them all added together it makes it pretty tough.

The scrambling algorithm is basically:

1. A scramble key k is computed from p₀
2. Each byte position i has a different set of four 1-byte integers A[i]. Integer A[i][j] is used when p₀ ≡ j mod 4.
3. A[i][j] is up-shifted by 0x80 when p₀ is in the range [0x54, 0xD3]. This does not apply to the checksum byte.
4. p’_i = ((p_i ⊕ k) + A[i][p₀ mod 4]) mod 256, where ⊕ is a bitwise exclusive or.

The algorithm to compute k is as follows (in ruby):

A values:

Position	0	1	2	3
p1	0x45	0x1F	0x14	0x5C
p2	0x2B	0xC9	0xE3	0x11
p3	0x6D	0x5F	0x8A	0x2B
p4	0xAF	0x03	0x1D	0xF3
p5	0x5A	0x22	0x30	0x11
p6	0x04	0xD8	0x71	0x42
p7	0xAF	0x04	0xDD	0x07
p8	0x61	0x13	0x38	0x64

There are probably actually several possible values for some of these. It really only matters that they line up in a particular way because of the checksum.

In addition to all of this, command arguments have different offsets from 0, and some commands (i.e., saturation and brightness) have the same p4 value with arguments spanning different ranges. For example, arguments for brightness start at 0x4F. Arguments for color start at 0x15.

Checksum

The checksum byte is calculated by summing all bytes in the unscrambled packet except for the first (scramble key) and last (checksum), and k + 2.

Code

Further detail is probably easier to communicate in code, so here is a ruby library that can encode/decode packets. The project on github also has a ton of packets I scraped from my devices.

Methodology

I scraped a ton of packets with this script.

Figuring this out was mostly making assumptions, pattern recognition, and trial and error. The most helpful assumption was that sequential values for arguments in the UDP protocol had sequential values in the 2.4 GHz protocol (this turned out to be true).

I noticed that packets that had p₀ values with the same remainder mod 4 followed a nearly sequential pattern. It often looked something like 0xC, 0xD, 0xA, 0xB, etc. A sequence follows this pattern when it’s xored with a constant. I wrote some cruddy ruby methods to brute-force search for constants that yielded the sequence [0, 1, …, N]. This also allowed me to find the values for the As.

Roughly the same process was repeated for each byte. Bytes that are constants were trickier because they didn’t follow a sequence. I instead brute-forced values for A given a sequence of xor keys.

Next Steps

I’ll be porting over the scrambling code to my ESP8266 milight hub to add support for the new bulbs.

UPDATE 2017-03-28: A few kind volunteers sent me packet captures from their devices, and the ID bytes were not staying fixed under decoding. Assuming my methodology is right, these should be the right values for all parameters, with the possible exception of p₁ and the checksum byte.

UPDATE 2017-03-20: I found that the wifi box I was testing with supported older protocols, which transmits the unscrambled device ID. There were several possible values for the ID byte offsets, and I chose a few of them arbitrarily. The decoded ID in the scrambled protocol was not matching the ID in the unscrambled protocol. Updating the additive offset values fixed this.