Milight WiFi Gateway Emulator on an ESP8266

Milight bulbs* are cheap smart bulbs that are controllable with an undocumented 2.4 GHz protocol. In order to control them, you either need a remote* (~$13), which allows you to control them directly, or a WiFi gateway* (~$30), which allows you to control them with a mobile app or a UDP protocol.

A few days ago, I posted my Arduino code to emulate a Milight WiFi gateway on an ESP8266 (link). This allows you to use an NRF24L01+ 2.4 GHz tranceiver module* and an ESP8266* to emulate a WiFi gateway, which provides the following benefits:

  1. Virtually unlimited groups. The OTS gateways are limited to four groups.
  2. Exposes a nice REST API as opposed to the clunky UDP protocol.
  3. Secure the gateway with a username/password (note that the 2.4 GHz protocol used by the bulbs is inherently insecure, so this only does so much good).

I wanted to follow up with a blog post that details how to use this. I’m going to cover:

  1. How to setup the hardware.
  2. How to install and configure the firmware.
  3. How to use the web UI and REST API to pair/unpair and control bulbs.

Shopping List

This should run you approximately ~$10, depending on where you shop, and how long you’re willing to wait for shipping. Items from Chinese sellers on ebay usually come at significant discounts, but it often takes 3-4 weeks to receive items you order.

  1. An ESP8266 module that supports SPI. I highly recommend a NodeMCU v2*.
  2. An NRF24L01+ module. You can get a pack of 10* on Amazon for $11. You can also get one that supports an external antenna if range is a concern (link*).
  3. Dupont female-to-female jumper cables (at least 7). You’ll need these to connect the ESP8266 and the NRF24L01+.
  4. Micro USB cable.

If you get a bare ESP8266 module, you’ll need to figure out how to power it (you’ll likely need a voltage regulator), and you’ll probably have to be mildly handy with soldering.

Setting up the Hardware

The only thing to do here is to connect the ESP8266 to the NRF24L01+ using the jumper cables. I found this guide pretty handy, but I’ve included some primitive instructions and photos below.

NodeMCU Pinout

NRF24L01+ Pinout

 

NodeMCU Pin NRF24L01+ Pin
3V (NOT Vin) VCC
G GND
D0 CE
D5 (HSCLK) SCK
D6 (HMISO) MISO
D7 (HMOSI) MOSI
D8 (HCS) CSN

Installing drivers

There are a couple of different versions of NodeMCUs (I’m not convinced they’re all actually from the same manufacturer). Depending on which one you got, you’ll need to install the corresponding USB driver in order to flash its firmware.

The two versions I’m aware of are the v2 and the v3. The v2 is smaller and has a CP2102 USB to UART module. You can identify it as the small square chip near the micro USB port:

NodeMCU v2 with CP2102 circled

Install drivers for the v2 here.

The v3 is larger and has a CH34* UART module, which thin and rectangular:

NodeMCU v3 with CH34* circled

The CH34* drivers seem more community-supported. This blog post goes over different options.

I’ve been able to use both the v2 and v3 with OS X Yosemite.

Installing Firmware

If you’re comfortable with PlatformIO, you can check out the source from Github. You should be able to build and upload the project from the PlatformIO editor.

Update – Mar 26, 2017: I highly recommend using PlatformIO to install the firmware. The below instructions are finicky and unless you get the arguments exactly right, the filesystem on your ESP will not work correctly. Using PlatformIO is a more robust way to get a fresh ESP set up. Further instructions are in the README.

Update – Feb 26, 2017: if you’ve used your ESP for other things before, it’s probably a good idea to clear the flash with esptool.py --port /dev/ttyUSB0 erase_flash . Thanks to Richard for pointing this out in the comments.

If not, you can download a pre-compiled firmware binary here. If you’re on Windows, the NodeMCU flasher tool is probably the easiest way to get it installed.

On OS X (maybe Linux?), following the NodeMCU guide, you should:

  1. Connect the NodeMCU to your computer using a micro USB cable.
  2. Install esptool
  3. Flash the firmware:

    Note that  /dev/cu.SLAB_USBtoUART should be substituted for  /dev/cu.wchusbserial1410 if you’re using a v3 NodeMCU. Be sure to specify the real path to the firmware file.
  4. Restart the device. To be safe, just unplug it from USB and plug it back in.

Setup firmware

Note that you’ll have to do all of these things before you can use the UI, even if you used the pre-compiled firmware:

  1. Connect the device to your WiFi. Once it’s booted, you should be able to see a WiFi network named “ESPXXXXXX”, where XXXXXX is a random identifier. Connect to this network and follow the configuration wizard that should come up.  The password will be milightHub.
  2. Find the IP address of the device. There are a bunch of ways to do this. I usually just look in my router’s client list. It should be listening on port 80, so you could use nmap  or something.

You should now be able to navigate to http://<ip_of_isp>.

Using the Web UI

The UI is useful for a couple of things.

If you have Milight bulbs already, you probably have them paired with an existing device. Rather than unpairing them and re-pairing with the ESP8266 gateway, you can just have the ESP8266 gateway spoof the ID of your existing gateway or remote. Just click on the “Start Sniffing” button near the bottom and push buttons in the app or on the remote. You should see packets start to appear:

The “Device ID” field shows the unique identifier assigned to that device. To have the ESP8266 gateway spoof it, scroll up to the top and enter it:

The controls should not work as expected. You can click on the “Save” button below if you want to save the identifier in the dropdown for next time.

The UI is also useful for pairing/unpairing bulbs. Just enter the gateway ID, click on the group corresponding to the bulb you wish to pair/unpair, screw in the bulb, and quickly (within ~3-5s) press the appropriate button. The bulb should flash on and off if it was successful.

Using the REST API

The UI is great for poking around and setting things up, but if you want to tie this into a home automation setup, you’ll probably want a programmatic interface. The API is fully documented in the Github readme, but here’s a quick example:

This will turn bulbs paired with device 0xCD86, group 2 on and set the color to red (hue = 0).

UPDATE – Feb 12, 2016

I realized this project would be a lot more immediately useful to people if it just supported the existing Milight UDP protocol. This would allow people to use the existing integrations others have built for OpenHab, Home Assistant, SmartThings, etc.

The Web UI has a section to manage gateway servers. Each server will need a device ID and a port.

* Amazon affiliate link.

415 thoughts on “Milight WiFi Gateway Emulator on an ESP8266

  1. First of all, thanks for all the work you’ve done and the time you’ve spent on this. I’m very new (this is my first time) to the whole NRF24L01 + ESP8266 world but despite that I got them working up to a point but have been scratching my head with a little problem now. I can see the web interface and I can sniff packets from other devices (remote and android app), but when I try to spoof one of these deviceIDs, the bulbs don’t get updated with my choice of color or brightness. I might be missing something pretty obvious here, but can’t be sure. I was wondering also, is it better to spoof individual devices or is it better to spoof the official milight bridge and how exactly can I get the deviceID of the bridge?

    Any ideas or suggestion would be really helpful. thanks again!

     

    • The only ways to reset the admin password are:

      1. Do so through the UI / API. You need to know the current password to do this.

      2. Wipe the ESP and reflash the firmware (make sure to erase SPIFFS too).

  2. Hello Chris,

    thanks a lot for this really amazing project.

    Especially the MQTT implementation is a big step forward wrt. the possibilities to integrate this type of lightning system to any home automation solution.

    I’m currently about to switch towards using the Brigde’s MQTT interface for some older RGBW bulbs and a LED controller (Protocoll V5, all RGBW, Bridge is on version 1.7.0-dev3). This will replace the traditional UDP-way that I used for several years now to get a better sync with the usage of remotes (2×1-channel and 3×4-channel).

    After having played around with all of the stuff, there remain some questions and remarks, I kindly ask for help and further info:

    1. The bridge recognizes all of the 4-channel remotes I own without problem, but none of the two 1-channel versions. Is this a general limitation of the openmili-code or due to additional filtering on the bridge?

    2. In some cases, there seem to be a missing link between commands and result. Eg. if a bulb is “OFF” and you click on “Night”, this will lead to an “ON” state and a MQTT message containing both of the info “ON” and “night_mode”.
    When clicking “White” the behaviour is different, the state is not changed (also not for the web interface). From the logical side, one would expect three or more infos (“ON”, “white_mode”, hue-value set corresponding to “white” and at least brightness to a reasonable value (seems not to be fix for each type of bulb, but e.g. 80% would be a good starting point imo).

    3. The selections made for info to be provided in “Settings” seem only to have effect on the REST Api but not the MQTT-JSONs. Could this be changed? Especially having the “level” (and perhaps the RGB-value) also sent via MQTT would really be a nice to have.

    4. It’s not a big issue with MQTT to cope around the problem already mentionned in comment #2047 (also distributing commands on channel 0 to channels 1 to 4), but it seems this also leads to some inconsistent info provided on the web frontend and the REST Api(didn’t to much testing on that, might not be true).

    Once more: You’ve done a great job!

    Kind regards.

    • Hi,

      1. The bridge recognizes all of the 4-channel remotes I own without problem, but none of the two 1-channel versions. Is this a general limitation of the openmili-code or due to additional filtering on the bridge?

      The code at this point is only loosely based on Henryk’s original. There’s a bunch of additional reverse engineering that I and others have done to support additional protocols.

      If a device is not supported, it’s generally possible to add support. Would just need help getting some of the details.

      Do you have model numbers for these remotes?

      2. In some cases, there seem to be a missing link between commands and result. Eg. if a bulb is “OFF” and you click on “Night”, this will lead to an “ON” state and a MQTT message containing both of the info “ON” and “night_mode”.
      When clicking “White” the behaviour is different, the state is not changed (also not for the web interface). From the logical side, one would expect three or more infos (“ON”, “white_mode”, hue-value set corresponding to “white” and at least brightness to a reasonable value (seems not to be fix for each type of bulb, but e.g. 80% would be a good starting point imo).

      I think I’m not exactly following what’s going on. Could you provide some MQTT logs demonstrating the behavior you’re seeing, and then manually construct the messages you’d expect to see? Might help me understand.

      3. The selections made for info to be provided in “Settings” seem only to have effect on the REST Api but not the MQTT-JSONs. Could this be changed? Especially having the “level” (and perhaps the RGB-value) also sent via MQTT would really be a nice to have.

      MQTT supports all of the same parameters that the REST API does. It does not support anything outside of sending commands to bulbs, though (the /gateways/:device_id/:device_type/:group_id route).

      4. It’s not a big issue with MQTT to cope around the problem already mentionned in comment #2047 (also distributing commands on channel 0 to channels 1 to 4), but it seems this also leads to some inconsistent info provided on the web frontend and the REST Api(didn’t to much testing on that, might not be true).

      State handling for MQTT and REST are handled through the same codepath, so there shouldn’t be any inconsistency there.

      If you have logs or examples I’d be happy to look 🙂

    • Hi Chris,

      thanks a lot for your detailed response!

      Answering your questions in english is quite hard for me, but I try my very best!

      ad 1: “Do you have model numbers for these remotes?”

      No, there’s no label on them beside a short “2.4 G RF”. At milight.com, there’s a picture provided. They look like the ones showed under Milight Remote (the right, coloured 1-channel-model on the right). The working ones are sold there as Milight RGBW Remote – I more or less ought them all together.
      Would flashing an Arduino with Henryks Code and do some sniffing would help?

      ad 2:Please note: my observations are just what I see when using the RGBW type of bulbs I personally own. Might be different whith other harware.
      Using the web-ui provided by the hub and click on the buttons there gets me the following:
      Startet with “OFF”-State and Click on “Night” results in MQTT messages {“state”:”OFF”} and subsequent {“state”:”ON”,”command”:”night_mode”}. So different from my first observation, the night command always turns the bulbs off and then immediately on again (don’t know, if this is really sent out RF-wise). This seems to be a special behavior of the Night function. Then the bulb also is correctly indicated as beeing “ON”.

      If state is “OFF” when ckicking on “White”, the only message I get is {“command”:”white_mode”}, and the indicated state of the milight hub stays as “OFF”. In the real world the behaviour is quite different: The first click on “White” just turns the bulb on again, if it’s off. The colour state and also brightness would be reestablished at the settings the bulb itself had stored. So clicking on “White” when bulb is off should lead to just an “ON”-message.
      Only in case bulb is already turned “ON”, the “White” command leads to a result in reality that is something like {“command”:”white_mode”,”hue”:””}. Seems brightness just stays unchanged with the exeption coming from night mode, then brightness will be set to last level (but as there had not been any change when entering “night mode” this is ok, imo).
      Hope, you can follow my thoughts now, probably it’s best to just make some tests wrt, if this still is confusing.

      ad 3.:
      Might be some kind of misunderstanding. Sending all of the commands towards the esphub works as expected. The other way round is what I was talking about – the updates coming from the hub towards the broker e.g. in case the web interface or a remote is used. In the later case only a subset of the info seems to be sent to the broker (esp. not “level” and rgb values).

      My selections had been (excerpt from “ip/settings”):

      Group state fields:
      0 “state”
      1 “status”
      2 “brightness”
      3 “level”
      4 “hue”
      5 “color”
      6 “mode”
      7 “color_temp”
      8 “bulb_mode”
      9 “computed_color”

      Using the REST API for requesting them, I get all the values back, so they’re there…

      ad 4.: After having played around a little with my HA software, keeping track of group state changes to individual channels ist indeed no real issue. The rest of my irritation was most likely more related to the observation the state beeing not indicated correctly when using the “White” command as described in # 2.

      Kind regards

    • No, there’s no label on them beside a short “2.4 G RF”. At milight.com, there’s a picture provided. They look like the ones showed under Milight Remote (the right, coloured 1-channel-model on the right). The working ones are sold there as Milight RGBW Remote – I more or less ought them all together.
      Would flashing an Arduino with Henryks Code and do some sniffing would help?

      Gotcha. The product list on futlight.com does a better job of listing model numbers. Can you find it there?

      Henryk’s code at this point is a lot more limited than espMH. It uses a fixed set of RF channels, syncwords, and always expects packets of a certain length. If espMH isn’t seeing your remotes, I don’t think Henryk’s will either.

      For #2, it probably would be easiest if you could supply a script that reproduced your issue.

      What version are you using? I fixed some less-than-desirable state behavior in 1.7.

      Might be some kind of misunderstanding. Sending all of the commands towards the esphub works as expected. The other way round is what I was talking about – the updates coming from the hub towards the broker e.g. in case the web interface or a remote is used. In the later case only a subset of the info seems to be sent to the broker (esp. not “level” and rgb values).

      Which topic are you getting updates on? Note that you’ll want to be looking at the state topic, not the update topic. State is everything, while update only decodes the corresponding packet.

    • Thank you once more for your feedback!

      The remote looks like the one listed as part of FUT25.

      Used version is “1.7.0-dev3” (D1 mini), so this is quite up to date. Seems not all of the state issues are fixed, as already described.

      Thanks for pointing me to the fact the complete set of values being available under state, hope, I’ll get my client application to extract the relevant info from there.

      Still, it’s a little irritating what’s going on on the broker. E.g. setting level to 70 produces this traffic:
      Client mosqsub/XXXX received PUBLISH (d0, q0, r0, m0, ‘milight/0x63C2/rgbw/4’, … (14 bytes))
      {“level”:”70″}
      Client mosqsub/XXXX received PUBLISH (d0, q0, r0, m0, ‘milight/updates/0x63C2/rgbw/4’, … (18 bytes))
      {“brightness”:184}
      The later is the reaction of the hub. I changed the send code of my application to just send {“level”:70}. This seems to make a difference: Then also get the entire state will be updated as described now for hue – there the additional “” have no negative effects, and also the level command itself was also executed.

      Sending a hue command is slightly different (with or without quotes):
      Client mosqsub/XXXX received PUBLISH (d0, q0, r0, m0, ‘milight/0x63C2/rgbw/4’, … (13 bytes))
      {“hue”:”229″}
      Client mosqsub/XXXX received PUBLISH (d0, q0, r0, m0, ‘milight/updates/0x63C2/rgbw/4’, … (11 bytes))
      {“hue”:230}
      Client mosqsub/XXXX received PUBLISH (d0, q0, r0, m0, ‘milight/states/0x63C2/rgbw/4’, … (92 bytes))
      {“state”:”ON”,”status”:”ON”,”level”:72,”hue”:230,”color”:{“r”:0,”g”:42},”bulb_mode”:”color”}

      Additionally some traffic, when using mode commands:
      Client mosqsub/XXXX received PUBLISH (d0, q0, r0, m0, ‘milight/0x63C2/rgbw/4’, … (22 bytes))
      {“command”:night_mode}
      Client mosqsub/XXXX received PUBLISH (d0, q0, r0, m0, ‘milight/updates/0x63C2/rgbw/4’, … (15 bytes))
      {“state”:”OFF”}
      Client mosqsub/XXXX received PUBLISH (d0, q0, r0, m0, ‘milight/updates/0x63C2/rgbw/4’, … (37 bytes))
      {“state”:”ON”,”command”:”night_mode”}
      Client mosqsub/XXXX received PUBLISH (d0, q0, r0, m0, ‘milight/states/0x63C2/rgbw/4’, … (93 bytes))
      {“state”:”ON”,”status”:”ON”,”level”:72,”bulb_mode”:”night”,”color”:{“r”:255,”g”:255,”b”:255}}
      Client mosqsub/XXXX received PUBLISH (d0, q0, r0, m0, ‘milight/0x63C2/rgbw/4’, … (21 bytes))
      {“command”:set_white}
      Client mosqsub/XXXX received PUBLISH (d0, q0, r0, m0, ‘milight/updates/0x63C2/rgbw/4’, … (24 bytes))
      {“command”:”white_mode”}

      But still no “ON” reaction nor update to state, when directly switching from “OFF” to “white_mode” with “set_white”, initiated with {“command”:set_white}.

      Thank you for bringing some more light into my unserstanding, what’s going on.

    • Yeah I think it’s probably best to ignore the updates topic all together. It’s really only useful in a pretty rare set of circumstances. I think the update topic will always send “level” instead of brightness. The field configurations only apply to the state topic.

      That’s a known issue, and it’s unfortunately probably not going to get fixed unless it’s causing major issues. The problem here is that both hue and brightness are being stored in 7 bits, and converted to the appropriate range. This means there will be some collisions and imperfect conversions. The fix would be to store the values at their full resolution. But the underlying protocol uses fewer bits, so the added resolution is not really functional.

      I’ve noticed the same sorts of funny behavior with states for night mode. I’ll try to have that fixed in the 1.7 production release. The fundamental issue, I think, is that “night mode” is really a state in the same way that on/off are, but that’s not the way it’s represented currently. Probably too much work to refactor at this point, but it leaves a bunch of crappy corner cases.

    • Hi Chris,

      seems using the states data instead of udpates solves a lot of the issues I ran into.

      But still, imo, it’s not the night mode that is causing confusing wrt to the state the bulb really is in. Especially night mode seems to be handled correctly. It’s more white mode that causes confusion, as this some sort of multi functionality.

      Kind regards,

      J.

  3. It finally worked!!! I had a lot of problems making the NRFL2041, either Long Range or plain, to work. I simply soldered a 4.7uF 50v electrolitic capacitor between the ground and vin pins, and MAGIC!

Leave a Reply

Your email address will not be published. Required fields are marked *