Webpack with Arduino and PlatformIO

Web development for embedded devices sucks. More often than not, it involves editing concatenated strings in a header file. Even some of the most popular web projects like Tasmota, WiFiManager, and esphome work like this.

If you’ve done any web development recently, it’s probably involved build tooling like Webpack. This makes it super easy to include external libraries, and to pack all of your assets (javascript, images, stylesheets) into an easy-to-distribute bundle. There’s also a wealth of incredible development tools like hot reloading, and support for alternative languages like Typescript.

Why is the experience so different when developing web applications for embedded devices?

To serve assets, you have these options:

  1. Embed the assets in the source code. This is a pain in the butt, but is a very straightforward approach and results in a complete firmware image.
  2. Side-load assets on, e.g., SPIFFS. This introduces an out-of-band step to get your project running.
  3. Serve assets from an external source. This introduces a point of failure, and requires Internet access.

(2) and (3) make for a much nicer development experience, but make for a crappy user experience. (1) is the reverse. It makes sense that developers optimize for their users.

There’s good news: we can have our cake and eat it. It just requires some simple build tooling. We can use Webpack as we normally would, have it generate a messy header file that our compiler will happily pack into a firmware image for us.

Example

I put together a skeleton example on Github, available here:

https://github.com/sidoh/webpack_with_platformio

Hopefully this serves as a useful quickstart, but I’ll give a more detailed overview below.

Moving Pieces

There are three high-level parts:

  1. The main PlatformIO application (./platformio.ini, ./lib, ./src).
  2. The web app (./web)
  3. A Webpack plugin to cram the web assets into a header file and a PlatformIO build script to call it (./.build_web.py)

./web is a self-contained Webpack application. We can iterate on it using the standard builtin development server. When we’re done, we can call the standard pio run --target upload to run the PlatformIO build. It’ll rebuild the web assets, pack them into a header file, and recompile our application.

The webserver is configured to dynamically serve the assets from the header file.

Result

Now that the build process is set up, we can focus on writing a great web app. Here’s an example of something I built for one of my projects, epaper_templates (I’ll talk about the new version in a future post at some point).

epaper_templates bitmap editor

This is a simple bitmap editor with local image importing, undo/redo stack, resizing, etc. Writing this in raw Javascript would be a giant pain in the butt. Doing it with React and the full richness of the Javascript developer community was an absolute delight. It took me a couple of evenings.

Continuous Integration

It’s pretty straightforward to get a build tool like TravisCI to build and distribute pre-compiled firmware images, complete with web assets. My epaper_templates project does this for tagged releases. Its .travis.yml may serve as a useful reference.

Downsides

This is, of course, not entirely free. Let’s discuss some of the costs:

  1. We’re embedding all of our assets in source. We’re obviously using progmem, but we still only have a couple of MBs of flash to play with. The above example is around 500 KB when it’s said and done.
  2. It’s is clearly more work than just putting stuff in header strings. If what you’re trying to do is simple, this is probably not the right approach for you.
  3. It adds dependencies to your build (e.g., node, npm).

Conclusion

When we have powerful tools, we can build really cool stuff without too much effort. Hopefully this is a useful reference for anyone wanting to use modern web development tooling in their embedded projects.

Ready-Made MiLight Hub

h4nc over at the HomeAssistant community forums has put together a spectacular ready-made hardware set for my ESP8266 MiLight Hub software. This includes a PCB and a 3D-printed case.

These look super snazzy, and will remove a lot of the guesswork from the setup process (skip the jumper wire fiddling and head-scratching when you read the pinout wrong!)

Awesome job, h4nc!

h4nc is offering ready-made kits which include:

  • A NodeMCU pre-flashed with the latest version of ESP8266 MiLight Hub
  • An nRF24 module with antenna
  • 3D-printed case

To order one for yourself, please get in touch with him at h4nc.zigbee(a)gmail.com, or drop him a PM on the HomeAssistant community forum.

You can see (and leave) feedback on h4nc’s readymade kits on the aforementioned HomeAssistant forum post.

Open Source

h4nc graciously open-sourced his work. You can find the Gerber files on Github, and the 3D-printable STLs on Thingiverse.

If you do end up making one of these for yourself, please consider a donation to h4nc. Contact him at h4nc.zigbee(a)gmail.com.

Links

Custom HomeAssistant Auth Provider

Background

HomeAssistant has a builtin authentication system bundled with a couple of providers which allow you to control how users are authenticated. Although I have been impressed with how thorough and robust the implementation is, it’s not terribly easy to integrate with an external authentication provider. There is the command-line provider, but there you’re still limited to schemes that use username/passwords entered by the user in the UI.

I’ve been using client-side TLS certificates to authenticate with internal services for a couple of years, and have been really pleased with the result:

  • Done correctly, client certificates are very secure.
  • Putting aside a really expensive initial setup cost, client certificates provide a fantastic end-user experience.

My family likes using HomeAssistant, but I don’t want anyone to deal with another password. I’d rather just rely on the client certificate auth system I’ve got set up. I endeavored to accomplish this, and figured I’d share the process and the results.

The Problem

Despite the extensible provider architecture, HASS does not currently support custom providers. If you’re willing to don your hacker fedora, you can get this done, however. There are two basic problems that need solving:

  1. Create an auth provider that interacts with client certificates
  2. Find a way to plumb in a custom auth provider

In the following sections, I’ll detail the solution I found.

Custom Auth Provider

First, a bit of background on how I’ve got client certs set up. I wrote about an earlier version a while back, but it’s evolved since then. That said, it still works the same way using some nginx lua scripting:

  1. When requesting a protected resource, nginx validates a cookie (which is a signed JWT).
  2. If the cookie is valid, the user is let through. If not, they’re redirected to an auth endpoint.
  3. The auth endpoint validates a client certificate. On passing, it generates a JWT, writes it to a cookie, and redirects to (1).

Sidebar: This might seem convoluted, but I promise it makes some amount of sense. To avoid interrupting the flow of the writeup more than this sidebar already has, I’ll stuff my reasoning in an appendix.

Given this, it’d work nicely for the custom auth provider to operate by verifying the JWT in the cookie. The claims in the JWT contain a username we can use. Here’s a gist containing an auth provider that does this, and the relevant configuration changes. The rough flow here is:

  1. When the login flow is created, HASS puts the client’s cookies in the flow context.
  2. In a single go, we extract the relevant cookie from the flow context, parse the JWT it contains, verify the signature, and extract the appropriate claim from the payload to use as the username.
  3. Either create the username we found, or return the existing user with that username.

Note that this won’t work out of the box because HASS does not forward cookies from the client that initiated the login flow. This tiny patch of HASS internals fixes that, though (we’ll cover this in the next session).

Auth Provider Plumbing

Now we’ve got to get HASS to recognize our changes. This isn’t difficult to solve, but there’s not a way to accomplish any of it without getting into the guts of HomeAssistant. So if you’re looking for a clean, officially supported solution, you’re unfortunately not gonna find it here.

All we’ve got to do is put the auth provider from the previous section the source directory that HASS expects (homeassistant/auth/providers), and apply the patch for login_flow.py mentioned in the previous section. We’ll also want a way to ensure the changes stick around after system updates.

If you’re using docker, this reasonably straightforward. Simply put the auth provider and the patched version of login_flow.py in the HASS configuration directory, and then link them to the appropriate places within the docker container. I use a shell script to manage updates, so I just tossed the link commands at the end:

And that’s about it! When HomeAssistant reboots, the login page should look like this:

Clicking on the “Next” button should result in the user being logged in, provided they’ve got the JWT cookie set (and they have to, given that nginx will not let this page render otherwise).

Conclusion

This works beautifully. No password required when logging into HomeAssistant, and no meaningful security concessions.

I am, however, pretty disappointed with how much hacking around it took to accomplish this. Hopefully custom stuff like this gets easier in future releases.

Appendix: What’s all this JWT Cookie Rigamarole?

The client certificate SSO flow mentioned previously probably seems a little Goldbergian. The reasons that I landed on and stuck with it are:

  • Some websites just don’t work with client certificates. Particularly — and this may have improved over the last few years without me noticing — web socket requests do not play well. Sometimes, client certs are not sent along with requests.
  • Scoped authorization. Client certificates alone provide authentication, but nothing other than binary authorization. This system provides a natural place for more finely grained authorization.
  • Provides an escape hatch for systems where using client certificates is a pain in the butt. Curling an endpoint? No problem. Grab an access token, and put it in an Authorization header.
  • Similarly to the previous point, provides a way to grant temporary access without needing to deal with CRLs. Since JWTs can include expiration times, you can make access however temporary you like.