Chris Mullins http://blog.christophermullins.com I occasionally write about things. Usually these things are about computers. Mon, 25 Dec 2017 02:53:46 +0000 en-US hourly 1 https://wordpress.org/?v=4.9.5 Solar-Powered Outdoor Thermometer with Multiple DS18B20 Sensors http://blog.christophermullins.com/2017/12/24/solar-powered-outdoor-thermometer-with-multiple-ds18b20-sensors/ http://blog.christophermullins.com/2017/12/24/solar-powered-outdoor-thermometer-with-multiple-ds18b20-sensors/#comments Sun, 24 Dec 2017 21:44:05 +0000 http://blog.christophermullins.com/?p=304 Continue reading ]]> I’ve been pretty happy with how my ESP8266-powered outdoor thermometers turned out.  One of these has two sensors — one to measure ambient temperature, and one to measure the temperature of a hot tub.  It’s solar-powered (with an 18650 Li-Ion battery), uses a single GPIO pin, and never needs charging!

DS18B20 Temperature Sensor

DS18B20s* are great digital thermometers for DIY projects.  They’re about the same price as the more popular DHT11*, and while they don’t measure humidity, they have a really cool advantage: you can use multiple sensors on the same data pin!  They also support a parasitic power mode, which drops their standby power draw to close to 0.

Diagram showing how to set up DS18B20s in parasite power mode. Image from tweaking4all.

Firmware

I updated the ESP8266 firmware I’ve been using to support multiple sensors, and added a nice web UI:

This allows me to push readings from multiple sensors connected to the same GPIO pin to different MQTT topics.  You can also push readings to an HTTP server.

The finished setup

In deep sleep mode, this project is suitable for battery life.  My outdoor thermometer uses an 18650 cell with a 5v solar panel and battery protection circuit.  I’ve never needed to charge it.  It’s been running for months.

Tips on power efficiency:

  1. Use an efficient voltage regulator.  Many dev boards use an AMS1117, which has a very high quiescent current.  You’re probably best off with a buck converter*, but an HT7333 or similar would be a lot better too.
  2. Use parasitic power mode!  Just wire Gnd and Vin on the DS18B20 to Gnd, and add a 4.7 KΩ pullup resistor to the data line.
  3. Disable any LEDs that are on for long periods of time.  I just used a pair of pliers to break the power LED.

I use a 3.5mm aux cable soldered to the probe wires to connect the DS18B20 probe to the ESP8266 circuit:

This is nice because it’s easy to add length with an extension cable or split the line with a standard aux splitter:

Here is a sloppy circuit in Fritzing.

Links

  1. GitHub page for firmware
  2. DS18B20 datasheet

Components Used*

  1. 5V battery protection circuit for 18650 cells
  2. 5V 180mA solar panel (mine is actually 160mA, but can’t find the product link anymore)
  3. 18650 Li-Ion Battery (recommend getting Samsung or Panasonic)
  4. ESP8266-07 and Breakout Board (soldering required. Wemos D1 Mini works if you want to avoid soldering).
  5. Waterproof DS18B20 Temperature Sensor
  6. Buck Converter (not necessary if using Wemos D1 Mini)
  7. Optional: 3.5mm extension cable (cut in half, solder one end to ESP8266 board, other to DS18B20)

* Contains Amazon affiliate link

]]>
http://blog.christophermullins.com/2017/12/24/solar-powered-outdoor-thermometer-with-multiple-ds18b20-sensors/feed/ 3
Using a Milight remote with HomeAssistant http://blog.christophermullins.com/2017/07/08/using-a-milight-remote-with-homeassistant/ http://blog.christophermullins.com/2017/07/08/using-a-milight-remote-with-homeassistant/#comments Sat, 08 Jul 2017 18:20:30 +0000 http://blog.christophermullins.com/?p=275 Continue reading ]]> Milight sells 2.4 GHz remotes* for quite cheap (~$12) which can be used to control Milight bulbs*. They also make in-wall panels* that do the same thing. These work quite well, but if you’re using something like HASS, you’ll end up with stale state. For example, if you turn lights on with the remote, that change won’t be reflected in HASS.

I recently released v1.4.0 of esp8266_milight_hub which allows you to use Milight remotes in conjunction with HomeAssistant, or any other platform that works with MQTT. Here’s some footage of it in action:

The ESP8266 is passively listening for packets sent by the remote, and forwards the data to an MQTT topic. HASS reacts by sending commands to a different MQTT topic which instructs the ESP8266 to turn bulbs on. You could just as easily have HASS do something completely different, like control a non-Milight bulb.

If you’re interested in setting this up yourself, instructions are available GitHub project wiki.

* Contains Amazon affiliate link

]]>
http://blog.christophermullins.com/2017/07/08/using-a-milight-remote-with-homeassistant/feed/ 2
Securing HomeAssistant with client certificates (works with Safari/iOS) http://blog.christophermullins.com/2017/04/30/securing-homeassistant-with-client-certificates/ http://blog.christophermullins.com/2017/04/30/securing-homeassistant-with-client-certificates/#comments Sun, 30 Apr 2017 08:28:35 +0000 http://blog.christophermullins.com/?p=259 Continue reading ]]> I recently moved from SmartThings to HomeAssistant. One of the things I didn’t have to think about too much with SmartThings was how to authenticate all of my connected devices (laptops, phones, tablets, etc.) with my HA platform. I wanted to find a good balance between security and convenience with HomeAssistant.

HomeAssistant makes it easy to secure your install with a password. Coupled with TLS, this is pretty solid. But there’s just something about the idea of a publically facing page that anyone on the Internet can get to, protected with nothing but a password that made me feel uneasy.

Client certificates are a very robust authentication mechanism that involves installing a digital certificate on each device you wish to grant access to. Each certificate is signed by the server certificate, which is how the server knows that the client is valid.

This feels nicer than HomeAssistant’s built-in security measures to me for a few reasons:

  1. Individual client certificates can be revoked. You don’t have to configure authentication on every device you own if someone loses their phone.
  2. While I highly doubt there are any issues with HomeAssistant, I feel more confident in nginx and openssl.
  3. Unless you add a passphrase to the client certificates (I didn’t), the whole thing is passwordless and still manages to be pretty darn secure.
  4. If I ever became truly paranoid, I could turn on HomeAssistant’s password protection and my HA dashboard would essentially need two authentication factors (the SSL cert + the password).

While I did find this approach more appealing, there are several drawbacks:

  1. It’s way harder to set up. You need to run a bunch of openssl commands, and install a certificate on each device you want to grant access to.
  2. The HomeAssistant web UI requires WebSockets, which seem to not play nicely in combination with client certificates on Safari or iOS devices. My household has iOS users, so this was something I needed to figure out.

I think I managed to get this working. The only disadvantage is that clients are granted access for an hour after successfully authenticating once. The basic approach is to tag authenticated browsers with an access token that’s good for a short period of time, long enough for them to establish a WebSocket connection. I’ll go through the steps in setting this up.

What you need

  1. Install packages: 
    sudo apt-get install nginx nginx-extras lua5.1 liblua5.1-dev
  2. openssl
  3. luacrypto module, which exposes openssl bindings in lua.

luacrypto was kind of a pain to install. Here’s what I did to get it working with my nginx install. It involved patching

configure.ac
 (thanks to this very helpful StackOverflow post for the tip):

git clone https://github.com/mkottman/luacrypto /opt/luacrypto \
  && cd /opt/luacrypto \
  &&
     # Fix package names for compatibility with Ubuntu
     echo 'diff --git a/configure.ac b/configure.ac
index b6b9175..20ea20c 100644
--- a/configure.ac
+++ b/configure.ac
@@ -28,10 +28,10 @@ AC_CHECK_FUNCS([memset])

 # pkgconfig
 PKG_CHECK_MODULES([OPENSSL], [openssl])
-PKG_CHECK_MODULES([LUA], [lua])
+PKG_CHECK_MODULES([LUA], [lua5.1])

 # lua libdir
-LUALIBDIR="`$PKGCONFIG --variable=libdir lua`"
+LUALIBDIR="`$PKGCONFIG --variable=libdir lua5.1`"

 # dest of headers
 CRYPTOINC="${includedir}/${PACKAGE_NAME}"' \
  | git apply \
  && autoreconf -i \
  && ./configure \
  && make \
  && sudo mkdir -p /usr/local/lib/lua/5.1 \
  && sudo cp src/.libs/crypto.so /usr/local/lib/lua/5.1/crypto.so

Setting up a Certificate Authority

There are already good guides on doing this. I recommend this one. In this guide, I’m using the default_CA parameters pre-filled by openssl on my system.

Generate client certificates

I put a script in

/usr/local/bin
  to make this easier:

$ cat `which create-client-ssl-cert`
#!/bin/bash

function usage () {
  echo "$0 [CA section name] [username]"
  exit 1
}

if [ $# -ne 2 ]
then
  usage
fi

CA_NAME="$1"
USERNAME="$2"

SSL_DIR="/etc/ssl"
SSL_PRIVATE_DIR="$SSL_DIR/${CA_NAME}/private"
SSL_CERTS_DIR="$SSL_DIR/${CA_NAME}/certs"
USERS_DIR="${SSL_CERTS_DIR}/users"

mkdir -p ${USERS_DIR}

if [ -f "${USERS_DIR}/${USERNAME}.key" ]; then
  echo "Key for $USERNAME already exists! Delete it to continue."
  exit 1
fi

# Create the Client Key and CSR
openssl genrsa -des3 -out ${USERS_DIR}/${USERNAME}.key 1024
openssl req -new -key ${USERS_DIR}/${USERNAME}.key -out ${USERS_DIR}/${USERNAME}.csr

# Sign the client certificate with our CA cert.  Unlike signing our own server cert, this is what we want to do.
openssl x509 -req -days 1095 -in ${USERS_DIR}/${USERNAME}.csr -CA $SSL_CERTS_DIR/ca.crt -CAkey $SSL_PRIVATE_DIR/ca.key -CAserial $SSL_DIR/${CA_NAME}/serial -CAcreateserial -out ${USERS_DIR}/${USERNAME}.crt

echo "making p12 file"
#browsers need P12s (contain key and cert)
openssl pkcs12 -export -clcerts -in ${USERS_DIR}/${USERNAME}.crt -inkey ${USERS_DIR}/${USERNAME}.key -out ${USERS_DIR}/${USERNAME}.p12

echo "made ${USERS_DIR}/${USERNAME}.p12"

You then run this for each device you want to grant access to:

# create-client-ssl-cert ca chris_pixel

Make sure to supply an export password. Generated certificate files will be placed in

/etc/ssl/ca/certs/users
 .

Get the certificates on the devices

The .p12 file is the one you want. Make sure to not compromise the certificates in the process.

I rsynced the files to my laptop and attached them to a LastPass note, which I could access on my devices. On most devices, you should be able to just open the .p12 file and it’ll do what you want.

On iOS devices, I needed to serve the certificates over HTTPS on a trusted network because they needed to be “opened” by Safari in order to be recognized.

Configure nginx

Here’s my nginx config. You’ll need to substitute your domain and SSL certificate parameters:

map $http_upgrade $connection_upgrade {
    default upgrade;
    ''      close;
}

upstream hass_backend {
  server 127.0.0.1:8123;
}


server {
  listen 80;
  listen 81;
  server_name my-homeassistant-install.mydomain.com;
  return 301 https://$host$request_uri;
}

server {
  listen      443 ssl;
  server_name my-homeassistant-install.mydomain.com;

  error_log  /var/log/nginx/my-homeassistant-install.mydomain.com/error.log;
  access_log /var/log/nginx/my-homeassistant-install.mydomain.com/access.log;

  ssl_certificate     /etc/letsencrypt/live/my-homeassistant-install.mydomain.com/fullchain.pem;
  ssl_certificate_key /etc/letsencrypt/live/my-homeassistant-install.mydomain.com/privkey.pem;
  ssl_dhparam         /etc/nginx/ssl/dhparams.pem;
  ssl on;

  add_header Strict-Transport-Security "max-age=31536000; includeSubdomains";
  ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
  ssl_ciphers "EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH:!aNULL:!eNULL:!EXPORT:!DES:!MD5:!PSK:!RC4";
  ssl_prefer_server_ciphers on;
  ssl_session_cache shared:SSL:10m;

  ssl_client_certificate /etc/ssl/ca/certs/ca.crt;
  ssl_crl                /etc/ssl/ca/private/ca.crl;
  ssl_verify_client      optional;

  proxy_buffering off;
  proxy_redirect http:// https://;
  proxy_set_header        X-Real-IP $remote_addr;
  proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
  proxy_set_header        X-Forwarded-Proto $scheme;
  proxy_http_version 1.1;
  proxy_set_header Upgrade $http_upgrade;
  proxy_set_header Connection $connection_upgrade;
  proxy_set_header Host $host;

  location / {
    #access_by_lua_file /etc/nginx/scripts/hass_access.lua;
    proxy_pass  http://hass_backend;
  }
}

If this all worked, you should be able to access my-homeassistant-install.mydomain.com from a device with a client certificate installed, but not otherwise. Unfortunately if you’re using iOS or Safari, you’ll probably notice that the page loads, but you get a forever spinny wheel. If you look in the debugger console, you might see messages that look like this:

[Error] WebSocket connection to 'wss://my-homeassistant-install.mydomain.com/api/websocket' failed: Unexpected response code: 400 (x11)

This is because the browser isn’t sending the client certificate information when trying to connect to the WebSocket and is therefore failing.

Fixing compatibility with iOS/Safari

Safari does actually send client cert info along with the initial request. Nginx has a really cool module that allows you to insert all sorts of fancy logic with lua scripts. I added one that tags browsers supplying a valid client certificate with a cookie granting access for about an hour. This worked really well. Since this is all over HTTPS, and the access tokens are short-lived, I felt pretty comfortable.

The easiest way I could think of to create a cookie that was valid for a limited time was to use an HMAC. Basically I “sign” a hash of the client’s certificate along with an expiry timestamp. The certificate hash, expiration timestamp, and HMAC are all stored in cookies. Nginx can then validate that the expiration timestamp is in the future, and that the HMAC signature matches what’s expected.

You’ll notice the commented-out line in the nginx config above. Uncomment it:

access_by_lua_file /etc/nginx/scripts/hass_access.lua;

And add the script:

local HMAC_SECRET = "hunter2"
local crypto = require "crypto"

function ComputeHmac(msg, expires)
  return crypto.hmac.digest("sha256", string.format("%s%d", msg, expires), HMAC_SECRET)
end

verify_status = ngx.var.ssl_client_verify

if verify_status == "SUCCESS" then
  client = crypto.digest("sha256", ngx.var.ssl_client_cert)
  expires = ngx.time() + 3600

  ngx.header["Set-Cookie"] = {
    string.format("AccessToken=%s; path=/", ComputeHmac(client, expires)),
    string.format("ClientId=%s; path=/", client),
    string.format("AccessExpires=%d; path=/", expires)
  }
  return
elseif verify_status == "NONE" then
  client = ngx.var.cookie_ClientId
  client_hmac = ngx.var.cookie_AccessToken
  access_expires = ngx.var.cookie_AccessExpires

  if client ~= nil and client_hmac ~= nil and access_expires ~= nil then
    hmac = ComputeHmac(client, access_expires)

    if hmac ~= "" and hmac == client_hmac and tonumber(access_expires) > ngx.time() then
      return
    end
  end
end

ngx.exit(ngx.HTTP_FORBIDDEN)

 

]]>
http://blog.christophermullins.com/2017/04/30/securing-homeassistant-with-client-certificates/feed/ 10
Reverse engineering the new Milight/LimitlessLED 2.4 GHz Protocol http://blog.christophermullins.com/2017/03/18/reverse-engineering-the-new-milightlimitlessled-2-4-ghz-protocol/ http://blog.christophermullins.com/2017/03/18/reverse-engineering-the-new-milightlimitlessled-2-4-ghz-protocol/#comments Sat, 18 Mar 2017 08:20:31 +0000 http://blog.christophermullins.com/?p=238 Continue reading ]]> My last post went over an ESP8266-based wifi gateway for Milight/LimitlessLED bulbs. This supports a few kinds of bulbs that have been around for a couple of years (e.g., this one).

About a year ago, newer bulbs and controllers started showing up that used a different 2.4 GHz protocol. This introduced some scrambling that made it difficult to emulate many devices. This was presumably done intentionally to prevent exactly the sort of thing that my last project accomplished (boo!).

The new bulbs actually have some really cool features that none of the old ones do, so there’s some incentive to figure this out. In particular, they support saturation, which allows for ~65k (2**16) colors with variable brightness instead of the 256 colors that the old one does. They also combine RGB and CCT (adjustable white temperature) in one bulb, which is super cool.

A few others have dug into this a little, but as far as I’ve been able to tell, no one has figured out (or at least shared) how to de-scramble the protocol. I think I’ve managed to do so. I should mention that I don’t have much experience doing this kind of thing, so it’s entirely possible the structure I’m imposing is a lot more complicated than what’s actually going on. But as far as I’ve been able to tell, it does work. I’ve tested with five devices – four remotes and one wifi box.

I’m going to start by detailing the structure, and I’ll follow up with some of the methodology I used to reverse the protocol.

Differences from old protocol

From a quick glance, there are a few superficial differences between the new and old protocols:

  1. Listens on a different channelset (this was true of different bulb types among the old bulbs too). The new bulbs use channels 8, 39, and 70.
  2. Different PL1167 syncword. It uses
    0x7236
     ,
    0x1809
     .
  3. Packets have 9 bytes instead of 7.
  4. Packets are scrambled. The same command can look completely different.

The scrambling is the tricky part. As others who have stared at packet captures noticed, when the first byte of packets for the same command is held fixed, most of the other bytes stay fixed too. This suggests that the first byte is some kind of scramble key. Turns out this is the case.

Example packets for turning group 1 on with one of my remotes:

A7 45 E9 6F BB 99 9E CF 4C
E1 D6 4F 79 99 78 16 9A A7
23 D1 55 03 47 25 2E 5B E8
92 18 18 33 42 15 60 02 CE
CA E0 E0 EB 0A DD A1 CA 9F

Notation

I’ll for a packet p, I’ll use pi to refer to the 0-indexed ith byte in the packet. For example, p0 refers to the 0th byte.

I’ll use p’ to refer to the scrambled packet for a packet p.

Structure

The 9 bytes of the packet are:

p0 p1 p2 p3 p4 p5 p6 p7 p8
Key 0x20 ID1 ID2 Command Argument Sequence Group Checksum

Packet scrambling

The designer of this protocol added in quite a few things to complicate reversing it. None of them are particularly hard on their own, but with them all added together it makes it pretty tough.

The scrambling algorithm is basically:

  1. A scramble key k is computed from p0
  2. Each byte position i has a different set of four 1-byte integers A[i]. Integer A[i][j] is used when p0 ≡ j mod 4.
  3. A[i][j] is up-shifted by 0x80 when p0 is in the range [0x54, 0xD3]. This does not apply to the checksum byte.
  4. p’i = ((pik) + A[i][p0 mod 4]) mod 256, where ⊕ is a bitwise exclusive or.

The algorithm to compute k is as follows (in ruby):

def xor_key(p0)
  # Generate most significant nibble
  shift = (p0 & 0x0F) < 0x04 ? 0 : 1
  x = (((p0 & 0xF0) >> 4) + shift + 6) % 8
  msn = (((4 + x) ^ 1) & 0x0F) << 4

  # Generate least significant nibble
  lsn = ((((p0 & 0xF) + 4)^2) & 0x0F)

  msn | lsn
end

A values:

Position 0 1 2 3
p1 0x45 0x1F 0x14 0x5C
p2 0x2B 0xC9 0xE3 0x11
p3 0x6D 0x5F 0x8A 0x2B
p4 0xAF 0x03 0x1D 0xF3
p5 0x5A 0x22 0x30 0x11
p6 0x04 0xD8 0x71 0x42
p7 0xAF 0x04 0xDD 0x07
p8 0x61 0x13 0x38 0x64

There are probably actually several possible values for some of these. It really only matters that they line up in a particular way because of the checksum.

In addition to all of this, command arguments have different offsets from 0, and some commands (i.e., saturation and brightness) have the same p4 value with arguments spanning different ranges. For example, arguments for brightness start at 0x4F. Arguments for color start at 0x15.

Checksum

The checksum byte is calculated by summing all bytes in the unscrambled packet except for the first (scramble key) and last (checksum), and k + 2.

Code

Further detail is probably easier to communicate in code, so here is a ruby library that can encode/decode packets. The project on github also has a ton of packets I scraped from my devices.

Methodology

I scraped a ton of packets with this script.

Figuring this out was mostly making assumptions, pattern recognition, and trial and error. The most helpful assumption was that sequential values for arguments in the UDP protocol had sequential values in the 2.4 GHz protocol (this turned out to be true).

I noticed that packets that had p0 values with the same remainder mod 4 followed a nearly sequential pattern. It often looked something like 0xC, 0xD, 0xA, 0xB, etc. A sequence follows this pattern when it’s xored with a constant. I wrote some cruddy ruby methods to brute-force search for constants that yielded the sequence [0, 1, …, N]. This also allowed me to find the values for the As.

Roughly the same process was repeated for each byte. Bytes that are constants were trickier because they didn’t follow a sequence. I instead brute-forced values for A given a sequence of xor keys.

Next Steps

I’ll be porting over the scrambling code to my ESP8266 milight hub to add support for the new bulbs.

UPDATE 2017-03-28: A few kind volunteers sent me packet captures from their devices, and the ID bytes were not staying fixed under decoding. Assuming my methodology is right, these should be the right values for all parameters, with the possible exception of p1 and the checksum byte.

UPDATE 2017-03-20: I found that the wifi box I was testing with supported older protocols, which transmits the unscrambled device ID. There were several possible values for the ID byte offsets, and I chose a few of them arbitrarily. The decoded ID in the scrambled protocol was not matching the ID in the unscrambled protocol. Updating the additive offset values fixed this.

]]>
http://blog.christophermullins.com/2017/03/18/reverse-engineering-the-new-milightlimitlessled-2-4-ghz-protocol/feed/ 8
Milight WiFi Gateway Emulator on an ESP8266 http://blog.christophermullins.com/2017/02/11/milight-wifi-gateway-emulator-on-an-esp8266/ http://blog.christophermullins.com/2017/02/11/milight-wifi-gateway-emulator-on-an-esp8266/#comments Sun, 12 Feb 2017 05:45:20 +0000 http://blog.christophermullins.com/?p=207 Continue reading ]]> Milight bulbs* are cheap smart bulbs that are controllable with an undocumented 2.4 GHz protocol. In order to control them, you either need a remote* (~$13), which allows you to control them directly, or a WiFi gateway* (~$30), which allows you to control them with a mobile app or a UDP protocol.

A few days ago, I posted my Arduino code to emulate a Milight WiFi gateway on an ESP8266 (link). This allows you to use an NRF24L01+ 2.4 GHz tranceiver module* and an ESP8266* to emulate a WiFi gateway, which provides the following benefits:

  1. Virtually unlimited groups. The OTS gateways are limited to four groups.
  2. Exposes a nice REST API as opposed to the clunky UDP protocol.
  3. Secure the gateway with a username/password (note that the 2.4 GHz protocol used by the bulbs is inherently insecure, so this only does so much good).

I wanted to follow up with a blog post that details how to use this. I’m going to cover:

  1. How to setup the hardware.
  2. How to install and configure the firmware.
  3. How to use the web UI and REST API to pair/unpair and control bulbs.

Shopping List

This should run you approximately ~$10, depending on where you shop, and how long you’re willing to wait for shipping. Items from Chinese sellers on ebay usually come at significant discounts, but it often takes 3-4 weeks to receive items you order.

  1. An ESP8266 module that supports SPI. I highly recommend a NodeMCU v2*.
  2. An NRF24L01+ module. You can get a pack of 10* on Amazon for $11. You can also get one that supports an external antenna if range is a concern (link*).
  3. Dupont female-to-female jumper cables (at least 7). You’ll need these to connect the ESP8266 and the NRF24L01+.
  4. Micro USB cable.

If you get a bare ESP8266 module, you’ll need to figure out how to power it (you’ll likely need a voltage regulator), and you’ll probably have to be mildly handy with soldering.

Setting up the Hardware

The only thing to do here is to connect the ESP8266 to the NRF24L01+ using the jumper cables. I found this guide pretty handy, but I’ve included some primitive instructions and photos below.

NodeMCU Pinout

NRF24L01+ Pinout

 

NodeMCU Pin NRF24L01+ Pin
3V (NOT Vin) VCC
G GND
D0 CE
D5 (HSCLK) SCK
D6 (HMISO) MISO
D7 (HMOSI) MOSI
D8 (HCS) CSN

Installing drivers

There are a couple of different versions of NodeMCUs (I’m not convinced they’re all actually from the same manufacturer). Depending on which one you got, you’ll need to install the corresponding USB driver in order to flash its firmware.

The two versions I’m aware of are the v2 and the v3. The v2 is smaller and has a CP2102 USB to UART module. You can identify it as the small square chip near the micro USB port:

NodeMCU v2 with CP2102 circled

Install drivers for the v2 here.

The v3 is larger and has a CH34* UART module, which thin and rectangular:

NodeMCU v3 with CH34* circled

The CH34* drivers seem more community-supported. This blog post goes over different options.

I’ve been able to use both the v2 and v3 with OS X Yosemite.

Installing Firmware

If you’re comfortable with PlatformIO, you can check out the source from Github. You should be able to build and upload the project from the PlatformIO editor.

Update – Mar 26, 2017: I highly recommend using PlatformIO to install the firmware. The below instructions are finicky and unless you get the arguments exactly right, the filesystem on your ESP will not work correctly. Using PlatformIO is a more robust way to get a fresh ESP set up. Further instructions are in the README.

Update – Feb 26, 2017: if you’ve used your ESP for other things before, it’s probably a good idea to clear the flash with

esptool.py --port /dev/ttyUSB0 erase_flash
 . Thanks to Richard for pointing this out in the comments.

If not, you can download a pre-compiled firmware binary here. If you’re on Windows, the NodeMCU flasher tool is probably the easiest way to get it installed.

On OS X (maybe Linux?), following the NodeMCU guide, you should:

  1. Connect the NodeMCU to your computer using a micro USB cable.
  2. Install esptool
    git clone https://github.com/themadinventor/esptool.git \
      && cd esptool \
      && sudo python ./setup.py install
  3. Flash the firmware:
    python esptool.py --port /dev/cu.SLAB_USBtoUART \
      --baud 115200 write_flash -fm=dio -fs=4MB 0x00000 \
      /path/to/firmware/download/esp8266_milight_hub_d1_mini-1.5.0.bin

    Note that 
    /dev/cu.SLAB_USBtoUART
     should be substituted for 
    /dev/cu.wchusbserial1410
     if you’re using a v3 NodeMCU. Be sure to specify the real path to the firmware file.
  4. Restart the device. To be safe, just unplug it from USB and plug it back in.

Setup firmware

Note that you’ll have to do all of these things before you can use the UI, even if you used the pre-compiled firmware:

  1. Connect the device to your WiFi. Once it’s booted, you should be able to see a WiFi network named “ESPXXXXXX”, where XXXXXX is a random identifier. Connect to this network and follow the configuration wizard that should come up.  The password will be milightHub.
  2. Find the IP address of the device. There are a bunch of ways to do this. I usually just look in my router’s client list. It should be listening on port 80, so you could use
    nmap
      or something.

You should now be able to navigate to http://<ip_of_isp>.

Using the Web UI

The UI is useful for a couple of things.

If you have Milight bulbs already, you probably have them paired with an existing device. Rather than unpairing them and re-pairing with the ESP8266 gateway, you can just have the ESP8266 gateway spoof the ID of your existing gateway or remote. Just click on the “Start Sniffing” button near the bottom and push buttons in the app or on the remote. You should see packets start to appear:

The “Device ID” field shows the unique identifier assigned to that device. To have the ESP8266 gateway spoof it, scroll up to the top and enter it:

The controls should not work as expected. You can click on the “Save” button below if you want to save the identifier in the dropdown for next time.

The UI is also useful for pairing/unpairing bulbs. Just enter the gateway ID, click on the group corresponding to the bulb you wish to pair/unpair, screw in the bulb, and quickly (within ~3-5s) press the appropriate button. The bulb should flash on and off if it was successful.

Using the REST API

The UI is great for poking around and setting things up, but if you want to tie this into a home automation setup, you’ll probably want a programmatic interface. The API is fully documented in the Github readme, but here’s a quick example:

curl -vvv -X PUT \
  --data-binary '{"status": "on", "hue":0}' \
  http://esp-milight.sidoh.org/gateways/0xCD86/2

This will turn bulbs paired with device 0xCD86, group 2 on and set the color to red (hue = 0).

UPDATE – Feb 12, 2016

I realized this project would be a lot more immediately useful to people if it just supported the existing Milight UDP protocol. This would allow people to use the existing integrations others have built for OpenHab, Home Assistant, SmartThings, etc.

The Web UI has a section to manage gateway servers. Each server will need a device ID and a port.

* Amazon affiliate link.

]]>
http://blog.christophermullins.com/2017/02/11/milight-wifi-gateway-emulator-on-an-esp8266/feed/ 415
In-depth how-to: Integrating Dash Buttons with SmartThings http://blog.christophermullins.com/2016/12/07/my-amazon-dash-button-setup-smartthings/ http://blog.christophermullins.com/2016/12/07/my-amazon-dash-button-setup-smartthings/#respond Wed, 07 Dec 2016 19:20:55 +0000 http://blog.christophermullins.com/?p=165 Continue reading ]]> After quite a bit of iteration, I’m mostly happy with the way I’ve integrated Dash buttons into my home automation setup. Here’s a demo:

My goals were:

  1. Make the buttons as responsive as possible.
  2. Make it robust.
  3. Setup should survive reboots and power outages without manual intervention.
  4. Integrate with SmartThings.

In a previous post, I outlined two different approaches. I went with the approach that had the lowest latency (<1s). This one is quite a bit more work — mostly because it requires a dedicated wireless card.

Here’s the equipment I used:

  1. Raspberry Pi 2 (I used the CanaKit starter kit. If you’re buying now, you’d probably want the Pi 3 edition).
  2. Edimax EW-7811Un USB WiFi dongle.

Important! to use this approach, you need at least one WiFi dongle that supports monitor mode. The Edimax dongle I suggested doesn’t support monitor mode, but the one that comes with the CanaKit 2 does. Note that the Pi 3’s onboard WiFi device does not support monitor mode, so you’ll want to buy a dongle that does (you can buy the CanaKit dongle separately for $9).

Set up dash buttons

This approach will work with the normal setup process, but with a slight modification, you can ensure that the dash buttons don’t contact Amazon when pressed.

The only thing you need to do differently is set up the buttons on a network you can delete later. I have dd-wrt on my router, so I used a virtual interface. If your router supports a “guest network” or something to that effect, it’s the same thing.

Create the network, set up the dash buttons on it, delete the network. The buttons will still attempt to connect when pressed, but won’t be able to because it doesn’t exist.

Install required packages

sudo apt-get install iw tcpdump libpcap-dev wpasupplicant ifrename

Set up the network

If you’re using ethernet + a WiFi dongle, you shouldn’t need to do much of anything. If you’re using two WiFi devices, it’s a little trickier. In order for this to work consistently across reboots, you’ll have to:

  1. Make sure that the interfaces (wlan0, etc.) are named consistently. They seemed to randomly swap by default, which obviously caused some problems.
  2. Tell the OS which device should be connecting to the network.

(1) is easy enough with ifrename. There’s probably a way to do it with udev, but this is way easier. It allows you to assign names to interfaces based on hardware (MAC) addresses. Open up

/etc/iftab
  in your favorite editor (just create it if it doesn’t exist). Mine looks like this:

monitorwan mac 00:00:00:00:00:00
mainwan mac 00:00:00:00:00:01

After a reboot, you should see that the devices are named appropriately:

$ ifconfig | grep wan
mainwan   Link encap:Ethernet  HWaddr 00:00:00:00:00:01
monitorwan Link encap:Ethernet  HWaddr 00:00:00:00:00:00

Notice you can name the interfaces whatever you want. monitorwan and mainwan seemed more informative than wlan0 and wlan1. 🙂

(2) is also pretty straightforward. There might be an easier way to do this, but I just did it by editing

/etc/network/interfaces
 to my liking:

auto lo
iface lo inet loopback

auto eth0
allow-hotplug eth0
iface eth0 inet manual

allow-hotplug monitorwan
auto monitorwan

allow-hotplug mainwan
auto mainwan
iface mainwan inet dhcp
	wpa-ssid "MyNetworkName"
	wpa-psk 7d3102a8c6159bb6486d54951805a719e8b4a2878a8015b1942a5097a5f85195

The wpa-psk field is a pre-shared key generated from your network SSID and passphrase. You can generate it with the 

wpa_passphrase
 tool (from the wpasupplicant package):

$ wpa_passphrase MyNetworkName mysuperdupergoodpassword
network={
	ssid="MyNetworkName"
	#psk="mysuperdupergoodpassword"
	psk=7d3102a8c6159bb6486d54951805a719e8b4a2878a8015b1942a5097a5f85195
}

You can apply these settings with a 

sudo service networking restart
 . Probably good to reboot to make sure it works as expected.

Download ha_gateway

This setup uses ha_gateway, which is a small REST gateway I use to bridge a bunch of custom hackery with the rest of my home automation setup (mostly SmartThings). To install it, just check out the project from Github:

sudo mkdir -p /apps && \
sudo chown $USER /apps && \
cd /apps && \
git clone https://github.com/sidoh/ha_gateway.git && \
cd /apps/ha_gateway && \
bundle install

While I haven’t tested ha_gateway with anything but ruby 2.3.1, it probably works with 1.9+. If you’re getting errors when running

bundle install
 , post a comment and I’ll help debug.

Create the monitor interface

In order to use monitor mode, we create a virtual monitor interface. We can do this with the

iw
 tool, but I stuffed all of the setup into a script shipped with ha_gateway. It takes two arguments: the interface you’re using for monitor mode, and what you want to name the virtual interface

sudo /apps/ha_gateway/bin/create_monitor_interface monitorwan DashMonitor

This should create an interface called

DashMonitor
 :

$ ifconfig DashMonitor
DashMonitor Link encap:UNSPEC  HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:3219880 errors:0 dropped:5137 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:571027641 (544.5 MiB)  TX bytes:0 (0.0 B)

To test if it’s working, you can try a tcpdump:

$ sudo tcpdump -i DashMonitor -c 50
tcpdump: WARNING: DashMonitor: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on DashMonitor, link-type IEEE802_11_RADIO (802.11 plus radiotap header), capture size 65535 bytes
15:46:08.905143 2412 MHz 11g -17dB signal antenna 1 39.0 Mb/s MCS 4 20 MHz lon GI CF +QoS Data IV:1319 Pad 20 KeyID 0
15:46:08.905424 24.0 Mb/s 2412 MHz 11g -33dB signal antenna 1 Acknowledgment RA:00:00:00:00:00:00 (oui Unknown)
15:46:08.905540 24.0 Mb/s 2412 MHz 11g -17dB signal antenna 1
15:46:08.905767 24.0 Mb/s 2412 MHz 11g -33dB signal antenna 1 Acknowledgment RA:00:00:00:00:00:00 (oui Unknown)
15:46:08.905909 2412 MHz 11g -17dB signal antenna 1 39.0 Mb/s MCS 4 20 MHz lon GI CF +QoS Data IV:131a Pad 20 KeyID 0
5 packets captured
50 packets received by filter
14 packets dropped by kernel

If you have basically any WiFi traffic around you, you should see packets pretty much immediately. If you don’t, it either means the monitor device isn’t working, or you’re legitimately not seeing traffic on whatever channel the NIC is tuned to.

To make sure the monitor device survives reboots, you can invoke the same script from

/etc/rc.local
 :

#!/bin/sh -e
#
# rc.local
#
# This script is executed at the end of each multiuser runlevel.
# Make sure that the script will "exit 0" on success or any other
# value on error.
#
# In order to enable or disable this script just change the execution
# bits.
#
# By default this script does nothing.

# Print the IP address
_IP=$(hostname -I) || true
if [ "$_IP" ]; then
  printf "My IP address is %s\n" "$_IP"
fi

/apps/ha_gateway/bin/create_monitor_interface monitorwan DashMonitor || echo "Failed to create monitor interface"

exit 0

Figure out MAC addresses of your dash button(s)

The easiest way I’ve found to do this is to use the monitor mode NIC and search for packets associated with the network you set them up on. I set my dash buttons up on a network called CMDashButton:

$ sudo tcpdump -i DashMonitor -e | grep CMDashButton
tcpdump: WARNING: DashMonitor: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on DashMonitor, link-type IEEE802_11_RADIO (802.11 plus radiotap header), capture size 65535 bytes
16:24:28.680003 1.0 Mb/s 2412 MHz 11b -43dB signal antenna 1 BSSID:Broadcast DA:Broadcast SA:00:00:00:00:00:00 (oui Unknown) Probe Request (CMDashButton) [1.0* 2.0* 5.5* 6.0* 11.0* 12.0* 24.0* 9.0 Mbit]
[...clipped...]
2475 packets captured
2475 packets received by filter
0 packets dropped by kernel

The hardware address is shown after “SA:” prefix.

Configure ha_gateway

ha_gateway is configured using a central YAML config file. You can just copy from the example:

cd /apps/ha_gateway
cp config/ha_gateway.yml.example config/ha_gateway.yml

Ignore the stuff at the beginning and skip down to the 

listeners:
 key. You’ll create a listener for each dash button you want to use:

listeners:
  clorox_wipes_button:
    driver: tcpdump_monitor
    params:
      # Put the MAC address of your button here
      hw_addr: 00:00:00:00:00:00

      # Change this if you named your monitor interface something different.
      interface: DashMonitor

      # There will always be 5-10 packets each time you press the button. 
      # This prevents actions from repeating more than once every 5000ms
      # (5 seconds).
      dedup_threshold: 5000

      events:
        probe_received:
          http:
            method: PUT
            url: http://google.com/some/path
            params:
              some_param1: 'some_value'
              some_param2: 'some_other_value'

This will fire an HTTP PUT request to http://google.com/some/path with the specified params every time the button is pressed. We can worry about making it do something useful later. First, let’s verify the button presses are getting picked up.

Use the 

run_listeners.sh
 script to fire up the ha_gateway listener process. Note you’ll have to run it with sudo — it won’t be able to listen on the monitor interface otherwise:

cd /apps/ha_gateway
sudo bin/run_listeners.sh

After waiting 10-20 seconds, press your dash button. You should see a log message that looks like this:

D, [2016-12-04T16:51:45.906202 #4177] DEBUG -- : Read line from tcpdump: 16:51:45.842083 1.0 Mb/s 2412 MHz 11b -45dB signal antenna 1 Probe Request (CMDashButton) [1.0* 2.0* 5.5* 6.0* 11.0* 12.0* 24.0* 9.0 Mbit]

This means ha_gateway is successfully detecting dash button presses! Now let’s make it do something useful.

Integrating with Smart Things

ha_gateway integrates with SmartThings. We’ll be able to control your existing ST devices and routines with the dash button. Getting this working is a little complicated because SmartThings requires clients to oauth with it. Let’s get that out of the way first.

First, you’ll have to install ha_gateway’s SmartApp. Log into your ST account (https://graph.api.smartthings.com/) and click on “My Smart Apps”. Click on the green “New SmartApp” button on the right near the top. Click on the “From Code” tab and paste in this code:

This should take you to an editor page. Couple of things to do to finalize setup:

  1. Publish the newly created app – click on “Publish”, then “For Me”
  2. Click on the “App Settings” button, then click on the “OAuth” section.
  3. Click on the “Enable OAuth for this SmartApp” button.
  4. You should see two text fields containing a  “Client ID” and a “Client Secret”. Make note of ’em.
  5. Click on “Update” near the bottom. OAuth settings won’t persist if you skip this!

Copy the client ID and client secret into ha_gateway’s config YAML:

require_hmac_signatures: false

smartthings:
  client_id: 00000000-0000-0000-0000-000000000000
  client_secret: 00000000-0000-0000-0000-000000000000

site_location: 'http://ip-of-your-pi:8000'

Setting

site_location:
 is important so that the OAuth redirect ends up hitting the Pi again. For now, also make sure that
require_hmac_signatures:
  is set to false. It’ll make it easier to go through the OAuth process.

Now fire up the ha_gateway web server by running 

bin/run.sh
 . Now navigate to:

http://ip-of-your-pi:8000/smartthings/authorize

This should direct you to an OAuth page on ST’s site. Select a hub, check the switches you want to allow control of, and click “Authorize”. You’ll be redirected to and endpoint that outputs a JSON blob containing information about the devices you authorized, which might look something like this:

{"00000000-0000-0000-0000-000000000000":{"name":"Xmas Tree","status":"on"}}

You can control each device via RESTful PUT requests. For example:

$ curl -X PUT -d'command=toggle' http://ip-of-your-pi:8000/smartthings/switches/00000000-0000-0000-0000-000000000000

This would send the “toggle” command to “Xmas Tree”, which would turn it off since its previous status was on.

If you wanted to configure a dash button to switch on and off your Christmas Tree, you’d edit the listener config like so:

clorox_wipes_button:
    driver: tcpdump_monitor
    params:
      # Put the MAC address of your button here
      hw_addr: 00:00:00:00:00:00

      # Change this if you named your monitor interface something different.
      interface: DashMonitor

      # There will always be 5-10 packets each time you press the button. 
      # This prevents actions from repeating more than once every 5000ms
      # (5 seconds).
      dedup_threshold: 5000

      events:
        probe_received:
          http:
            method: PUT
            url: '/smartthings/switches/00000000-0000-0000-0000-000000000000'
            params:
              command: toggle

Notice we don’t need to provide the full URL, just the path. ha_gateway will assume we want to send the request to its REST server. It’ll fill in the URL specified in the

site_location:
  key.

You can start both the REST server and the listener process with the included start script. It’ll run the listener process as root, so make sure you’ve got an active sudo session (i.e., make sure it’s not prompting for a password):

sudo echo hi && \
cd /apps/ha_gateway && \
bin/start

Logs are in

logs/ha_gateway.log
  and
logs/listeners.log
 .

You can also run routines. You can access /smartthings/routines to get a list of routines. To run a routine, send a GET request to /smartthings/routines/<routine_name>. Normalize routine_name to be all lowercase, remove non-alphanumeric characters, and replace spaces with underscores (e.g., “Good Night!” -> “good_night”).

Starting ha_gateway at boot

Obviously we want the REST server and the listener process to survive a reboot. This is pretty easy. I use monit because I already had it set up, but it’s probably more straightforward to just add this line to

/etc/rc.local
 :

/apps/ha_gateway/bin/start || echo 'failed to start ha_gateway'

Make sure it appears above the

exit 0
  at the end of the script.

Securing it

If you don’t mind anyone on your network being able to access ha_gateway (and therefore turn off your Christmas Cheer), you can enable HMAC signatures. This will require anyone making a request to sign the request with a shared secret. Just edit the config file:

require_hmac_signatures: true
hmac_secret: <some random secret>

Conclusions

This works really well for me. It was way more work than I expected when I decided to look into hacking the dash buttons. I have five dash buttons for various uses, and they work very reliably. Adding new buttons is really straightforward.

]]>
http://blog.christophermullins.com/2016/12/07/my-amazon-dash-button-setup-smartthings/feed/ 0
Thoughts on Amazon Dash Button Hacks http://blog.christophermullins.com/2016/11/27/thoughts-on-amazon-dash-button-hacks/ http://blog.christophermullins.com/2016/11/27/thoughts-on-amazon-dash-button-hacks/#comments Sun, 27 Nov 2016 09:59:33 +0000 http://blog.christophermullins.com/?p=160 Continue reading ]]> Excited by the prospect of $5 hackable IoT buttons, I ordered a couple to toy around with and use in my home automation setup. I knew some folks had already figured out how to hack them, but hadn’t looked closely at how they were doing it until my buttons arrived.

Not surprisingly, the hacks are pretty hacky. There were two approaches I found:

  1. Listen for ARP packets sent from the dash button. Devices send ARP probes after associating with a network in order to determine whether their hardware (MAC) address is already in use. My casual experimentation showed these packets showed up 3-5 seconds after the button was pressed. This is the standard approach first outlined in Ted Benson’s medium post.
  2. Listen for probe request packets sent from the dash button. Devices wishing to connect with a particular network will send these out to kick off the process. These show up much sooner after pressing the button — usually within 1 second. Outlined by ridiculousfish here.

There are some not awesome things about both of these. Probably the worst is that you need to run something like tcpdump as root, and your network device has to be put in promiscuous mode (in the case of #1), or monitor mode (in case of #2). Setting promiscuous mode on a NIC instructs it to not ignore traffic that isn’t addressed to it (it normally does). Monitor mode is a different beast. It’s more like sniffing the waves out of the air. The card sits there without being associated with a network and can report any packets it hears on whatever channel it’s set to.

#2 has some particularly nasty downsides:

  • Need a dedicated wireless device that supports monitor mode. Since you’ll presumably need whatever box you’re working with connected, you’ll need another network device (second wireless device or ethernet).
  • Not all wireless devices support monitor mode. Fortunately, the dongle that came with my Raspberry Pi does. This post lists a few chipsets that support monitor mode.
  • Requires some networking setup. Creation of virtual monitor device feels kludgy and does not persist across restarts.
  • Much easier to spoof. As we’re listening to packets sent before the device is associated with a network, an attacker would only need the MAC address of your button. Probably not a good idea to use the buttons for much beyond switching your lights on and off. : )

Even with these, I’m pursuing #2 because:

  • Latency for #1 is just too high for most uses.
  • Since we’re listening to probe requests, we detect a button press before it associated with a network. This means we can set the buttons up on a fake network (I used a virtual interface in dd-wrt), and delete it when we’re done. Net effect is that buttons won’t be able to connect when they’re pressed, and won’t try to order Clorox Wipes when you try to turn on the kitchen lights.
  • This last benefit is mostly nice because Amazon was sending me a push notification every time I pressed a button to the effect of “Select an item to order to finish setup!”. Had to turn push notifications for Amazon off.

We’ll see how it turns out!

]]>
http://blog.christophermullins.com/2016/11/27/thoughts-on-amazon-dash-button-hacks/feed/ 1
Riddlegate: automating apartment intercoms http://blog.christophermullins.com/2016/06/18/riddlegate-automating-apartment-intercoms/ http://blog.christophermullins.com/2016/06/18/riddlegate-automating-apartment-intercoms/#respond Sat, 18 Jun 2016 19:12:20 +0000 http://blog.christophermullins.net/?p=142 Continue reading ]]> If you’ve ever lived in an apartment building or gated complex, you’ve probably seen one of these things:

Apartment Intercom

A person dials some code to reach you, you make sure it’s someone you’re expecting, and you press some digits on your phone to grant access.

However handy this is, there are situations where it’s not very helpful. I can’t count how many packages I’ve missed because I was away from my phone when the delivery driver arrived. I really wanted to be able to just give them a code that allowed them access.

Enter Twilio. It enables exactly this. You rent a phone number through them (very cheap — on the order of $1/month), which can be configured to call HTTP endpoints when it receives a call, SMS, etc.

I built a small application around this to automate my apartment’s intercom. The intercom is configured to call my Twilio number, which will interact with the application. I called it “Riddlegate” after that plotline in The Neverending Story with the sphinxes. It’s pretty simple and self-explanatory. Here’s a screenshot of the admin UI:

X30j6Vt

Now when I have a guest, I can give them instructions that don’t involve me being near my phone:

  1. Dial <number>
  2. Wait for tone
  3. Dial <passcode>

And Riddlegate will buzz them in!

A (hopefully-not-too-terse) setup guide is included in the Github README.

Security

Given that this controls access to your building/complex, security is an important concern. There are a few things Riddlegate does to improve security:

  1. Twilio signs all requests it sends with your API key. Riddlegate validates these signatures and denies access when it detects an invalid signature. This prevents a would-be attacker from brute-forcing your access code if they were to discover your endpoint URL.
  2. Admin area is password-protected. This is obviously only as secure as the password you choose. Also obviously better if you serve over HTTPS.
]]>
http://blog.christophermullins.com/2016/06/18/riddlegate-automating-apartment-intercoms/feed/ 0
Security cameras: automatically recording and uploading footage when a door is opened http://blog.christophermullins.com/2016/03/20/auto-trigger-security-camera-recording-and-upload/ http://blog.christophermullins.com/2016/03/20/auto-trigger-security-camera-recording-and-upload/#respond Sun, 20 Mar 2016 21:49:34 +0000 http://blog.christophermullins.net/?p=131 Continue reading ]]> My last post detailed how to integrate a cheap IP cam with SmartThings. I briefly mentioned a SmartApp that took a picture when the door opened. This was pretty straightforward. I wanted to take it a step further and trigger recording when my door was opened (but no one is home). Beyond the obvious, I had the following requirements:

  1. SmartThings needs to be able toggle recording. The SmartApp should notify my household when it begins recording.
  2. Tampering with the camera shouldn’t destroy already captured footage.
  3. Footage should start uploading somewhere offsite as soon as possible.
  4. Control over uploaded footage.

I already covered (1) in my last post. My integration with the IP camera enables a scheduled recording feature, and configures this feature to always record. Switching off recording clears the schedule.

Uploading footage to S3

The camera I’m using already has builtin support to upload footage to an FTP server, which leaves everything but uploading to offsite storage.

Since it’s easy and cheap, I decided to upload footage to Amazon S3. I then needed a tool that:

  1. Watched for newly created files to appear in the directory my internal FTP server is pointed at. When a new file is detected:
  2. Immediately begin a streaming upload to S3. Since the file size isn’t known ahead of time, I made use of the multipart upload API, which allows for the breaking down of uploads into smaller (5MB) chunks.
  3. When the file is done being written to, complete the upload (i.e., tell Amazon the file is done being uploaded). This makes the file available on S3.

This seemed like a good fit for inotify, which allows for the monitoring of filesystem events. It’s possible to set up notifications that are triggered when, for example, a new file within a directory is opened, closed, or modified.

I didn’t find a tool that did exactly what I wanted, so I made a ruby library that did. I called it “s3reamer“. My sincerest apologies for being an awful portmanteau-ist. I run this on my home server:

s3reamer /path/to/cam_data org-sidoh-mybucket --log-file "log/s3reamer.log"

This will automatically begin uploading to S3 when recording on the device is triggered. Here’s a snippet from the log file:

I, [2016-03-20T13:47:26.761571 #13905]  INFO -- : Setting up watch on: /path/to/cam/data
I, [2016-03-20T13:47:34.664618 #13905]  INFO -- : File opened: /path/to/cam/data/my/foscam/record/schedule_20160320_124834.mkv
I, [2016-03-20T13:47:46.218999 #13905]  INFO -- : File closed. Completing S3 upload: /path/to/cam/data/my/foscam/record/record/schedule_20160320_124834.mkv

Uploading starts within a few seconds of recording being switched on. Most of that delay is waiting for the camera to begin uploading.

SmartApp to trigger recording when door opens

This part was pretty straightforward. Code below. This also sends a push notification when recording is switched on so my household knows and can react accordingly.

]]>
http://blog.christophermullins.com/2016/03/20/auto-trigger-security-camera-recording-and-upload/feed/ 0
Integrating Foscam FI9821P with SmartThings http://blog.christophermullins.com/2016/03/07/integrating-foscam-fi9821p-with-smartthings/ http://blog.christophermullins.com/2016/03/07/integrating-foscam-fi9821p-with-smartthings/#comments Mon, 07 Mar 2016 09:24:47 +0000 http://blog.christophermullins.net/?p=125 Continue reading ]]> Motivated mostly by curiosity, I was recently in the market for a cheap IP camera. After a little bit of research, I settled on a Foscam FI9821P (I got mine for ~$45 as an Amazon Warehouse Deal). The app provided by Foscam is pretty nice, but I wanted to integrate it with my home automation setup as well. In particular, I wanted to accomplish the following:

  1. Secure access. Any communication with the camera should require some secure authentication mechanism.
  2. SmartThings integration. I wanted a device in SmartThings I could play around with.
  3. REST endpoint. Although I could probably get most of what I want done with SmartThings alone, I didn’t want to be bound to it.

SmartThings has a device type for cameras, so as long as there’s some way to access the camera within SmartThings, (2) is easy. In a previous post, I outlined a setup that uses HMAC to secure communication with smart home devices. I leveraged it in this project as well.

I should mention that I stumbled across some existing attempts at this, but nothing that would’ve given (1) and (3).

I put together this route for my home automation gateway, which accomplishes (1) and (3). With it, I can capture a snapshot and control some rudimentary functionalities of the camera. I can, for example, request a snapshot of what the camera is currently seeing simply by accessing this URL (with the appropriate security headers in place):

http://HA_GATWAY_URL/camera/foscam1/snapshot.jpg

You can see that there’s baked in support for multiple cameras (since the endpoint is scoped by a camera name). While I don’t anticipate buying more cameras, I figured adding support would make this project more generally useful.

To integrate with SmartThings, I created a virtual device (code embedded below). It allows me to request an image, shift the camera to one of three preset positions, and to start/stop recording. Here’s a demo of the interface:SmartThings Interface

This project was a lot of fun, and quite a bit easier than I was anticipating. My favorite thing this has enabled is a SmartThings SmartApp that signals the camera to take a picture when my front door opens. To avoid being too creepy, this only happens when no one is home. If I can muster the motivation, I’ll probably write a separate post about that.

]]>
http://blog.christophermullins.com/2016/03/07/integrating-foscam-fi9821p-with-smartthings/feed/ 5