V2: WifiAP-only config, no reconfig

.gitignore

@@ -0,0 +1,3 @@
+.pio
+*.log
+*-

README.md

@@ -9,42 +9,115 @@ It almost exclusively uses MQTTS (MQTT over TLS) for communication.
 OTA updates are triggered over MQTTS but download the
 firmware using HTTP and then check the MD5 checksum to ensure no tampering occurred
 (but this does expose the firmware in the clear).
+Initial configuration is done using ESPAsyncWiFiManager, i.e., by running an access point and a web
+server.
 
 Disclaimer: I hesitated to call this firmware "secure" because I'm sure it can be hacked.
-It certainly hasn't been put under serious scrutiny (and there are open issues, see below).
-The goal here is to start with something reasonable as opposed to most esp8266 and esp32
-software out there that mostly uses unencrypted communication.
+Also, the initial configuration is not as bullet proof as I wish it were, it just gets painful real
+quick.
+The firmware also hasn't been put under serious scrutiny (and there are open issues, see below).
+So... why bother?
+The goal here is to start with something reasonable as opposed to defaulting to wide-open
+insecure unencrypted stuff.
+That being said, if you see a flaw, I'd love to hear about it (kindly) and I'd love even
+more to merge a fix!
 
-This base project does not include an HTTP(S) server simply because that's one less thing to secure
-with certificates and passwords. It means some more stuff needs to be done through MQTT than
-otherwise, but hopefully that's an acceptable tradeoff.
+Features
+--------
+
+- Initial configuration via web server over WiFi AP, includes configuration of MQTT server
+ and authentication (PSK)
+- Asynchronous MQTT client supporting QoS 0-2 and supporting PSK TLS cipher suites
+ (https://github.com/tve/async-mqtt-client)
+- OTA triggered by MQTT message, pulls firmware using HTTP and checks it using MD5
+ (uses https://github.com/tve/AsyncTCP)
 
 Open issues
 -----------
-The main open issue is the initial configuration. Right now the Wifi and MQTT server
-configs are hard-coded in the firmware. That's both inconvenient and insecure because
-OTA updates happen in the clear and thus an attacker could extract the Wifi and MQTT creds
-from an update as it happens or by downloading the firmware and inspecting it.
 
-The plan is to go through a 2-step process where the firmware contains some initial
-semi-public creds that allow it to connect to the MQTT server and from which it can then
-get a set of private creds which are stored on the device.
+- The initial configuration is not as secure as I'd like it to be: the AP is open and the web server
+ is on HTTP (unencrypted). I'd like to move to a secured AP with a default passwordi (which can be
+ changed in the first config run). Using the default password the communication can still be
+ cracked but the hurdle is much higher than currently.
+- The ESPAsyncWiFiManager does not seem to shut down the AP once Wifi is connected, this
+ should be forced.
+- The "reconfiguration" (i.e., starting up the AP even though the configured WiFi network can be
+ joined) is broken due to a bug in AsyncTCP.
+- I'd like to add a command line configuration so I don't need to mess with my laptop's WiFi to join
+ the AP and figure out how to cut&paste the MQTT PSK.
+- This base project does not deal with the physical security of the esp32. The chip contains
+ features to encrypt the firmware and make it tamper-proof but this base project does not make any
+ use of that and it may not work with the Arduino framework anyhow.
 
-For general use it would make sense to add an AP configuration mode with an HTTP server
-or WiFiManager.
-It would just have to be carefully crafted to be secure and not to use up a ton of resources.
+MQTT provisioning
+-------------------
 
-Another issue is that this base project does not deal with the physical security of the
-esp32. The chip contains features to encrypt the firmware and make it tamper-proof but
-this base project does not make any use of that. Maybe in the future...
+At boot, if there is no config saved in flash an mqtt identity is generated (it is the same as the
+Wifi access point SSID) and a random PSK is also generated. Both are stored in the new config and will
+persist, unless they're changed via the portal UI.
 
-Features
---------
+The idea is that the ident/psk can be entered into the MQTT server config, or custom ident/psk can
+be generated and configured using the portal.
+
+### Example
+
+Erasing the entire flash to get a clean slate starting point:
+```
+esptool.py --chip esp32 --port /dev/ttyUSB0 --baud 921600 --before default_reset erase_flash
+```
+Flashing and running the secure base example:
+```
+cd examples/basic
+pio run -e usb -t upload -t monitor
+```
+When it finally boots up, this is what happens:
+```
+===== ESP32 Secure Base Example =====
+E (55) SPIFFS: mount failed, -10025
+[E][SPIFFS.cpp:52] begin(): Mounting SPIFFS failed! Error: -1
+** Formatting fresh SPIFFS, takes ~20 seconds
+E (60) SPIFFS: mount failed, -10025
+Format took 19s
+No config file, initializing mqtt ident/psk
+MQTT ident=ESP-C44F330A9C35 psk=b9d4e59d1b55028d2d9ea49235dca5fa
+```
+The psk is not printed on subsequence boots, so it's important to capture it here.
+Alternatively, it is available through the portal UI.
+
+Wifi Observations
+-----------------
+
+The esp32-arduino WiFi class contains a number of reconnect features:
+- By default the STA config is saved to flash (this can be disabled with `WiFi.persistent(false)`)
+- By default the STA reconnects automatically after a disconnect (this can be disabled with
+ `WiFi.autoConnect(false)`), this happens in `WiFiGenericClass::_eventCallback`.
+- Need to verify that once disconnected the firmware keeps scanning to try to reconnect. (__TODO__)
+
+There are a couple of security issues that need to be considered when using the ESPAsyncWiFiManager:
+- The configuration portal must have a password otherwise an attacker can do a simple denial of
+ service attack on the AP, wait for the config portal to come up and then hack the system.
+ It's probably best to force setting the password using an extra parameter in the config.
+- The configuration portal should take some time to come up to make attacking the portal a tad
+ harder, this can be controlled using `setConnectTimeout`.
+
+The use of the ESPAsyncWiFiManager class falls into two main use-cases: systems that are not useful
+without Wifi where configuration can be blocking and where failure is best dealt with by reset, and
+systems that need to operate with or without Wifi where configuration should not be blocking and
+failure should not cause a reset.
+
+Systems that cannot meaningfully operate without Wifi should:
+- Set an upper bound to the configuration portal using
+`ESPAsyncWiFiManager.setConfigurationPortalTimeout` and reset the system if no connection can be
+established. This ensures the system doesn't get stuck. A value of 5-10 minutes
+should be a reasonable compromise between usability and ensuring that the system doesn't get stuck
+if unattended. If `ESPAsyncWiFiManager.autoConnect` returns false (not connected) then reset.
+
+Systems that must operate even without Wifi should turn on the STA using `WiFi.mode(WIFI_STA)` and
+then check periodically whether a connection has been established. If not, use
+`ESPAsyncWiFiManager.setupConfigPortalModeless` to start the configuration portal.
+Once connected, the portal should be shut down.
+
+An alternative for systems that must operate even without Wifi is to use the blocking
+`autoConnect` with an acceptable `setConnectTimeout` and accept the fact that the system won't
+operate for a few minutes if the AP cannot be contacted after a reset.
 
-- Asynchronous MQTT client supporting QoS 0-2 and supporting PSK TLS cipher suites
- (https://github.com/tve/async-mqtt-client)
-- SNTP time synchronization
-- OTA triggered by MQTT message, pulls firmware using HTTP and checks it using MD5
- (uses https://github.com/tve/AsyncTCP)
-- Simple MQTT based time zone offset
-- Periodic status info sent via MQTT

examples/basic/lib/ESPSecureBase

@@ -0,0 +1 @@
+../../../src

examples/basic/main.cpp

@@ -0,0 +1,90 @@
+// ESP32 Secure Base Basic Example
+// Copyright (c) 2019 Thorsten von Eicken, all rights reserved
+
+// Example application that prints Wifi info in loop().
+// When starting up, it will block in setup() until WiFi has connected to an access point.
+// If a configuration has been set-up, it tries to connect to the configured AP for a few seconds.
+// If that fails, it starts an AP and waits for a configuration to be set.
+// While in set-up mode, it resets every few minutes in case that fixes something.
+
+#include <Arduino.h>
+#include <WiFi.h>
+#include <ESPSecureBase.h>
+
+//===== I/O pins/devices
+
+#define BUTTON 0
+#ifndef LED
+#define LED 1
+#endif
+
+// configuration portal timing
+
+#define CONF_CONN_TIMEOUT 10 // seconds to try to connect to AP before setting up config portal
+#define CONF_PORTAL_TIMEOUT (5*60) // seconds before config portal stops and device resets
+
+// MQTT message handling
+
+void onMqttMessage(char* topic, char* payload, MqttProps properties,
+ size_t len, size_t index, size_t total)
+{
+ if (strcmp(topic, "system/offset") == 0) {
+ //onTimeZone(topic, payload, properties, len, index, total);
+ } else if (strlen(topic) == mqTopicLen+4 && len == total &&
+ strncmp(topic, mqTopic, mqTopicLen) == 0 &&
+ strcmp(topic+mqTopicLen, "/ota") == 0)
+ {
+ ESBOTA::begin(payload, len);
+ }
+}
+
+//===== Setup
+
+void setup() {
+ ESBConfig config;
+
+ Serial.begin(115200);
+ printf("\n===== ESP32 Secure Base Example =====\n");
+ pinMode(BUTTON, INPUT_PULLUP);
+
+#if 0
+ // wipe-out saved wifi settings for testing purposes
+ WiFi.begin("none", "no-no-no");
+ delay(1000);
+#endif
+
+ // Configure Wifi.
+ bool connected = config.setup(CONF_CONN_TIMEOUT, CONF_PORTAL_TIMEOUT);
+
+ // If config failed, reset device to start over.
+ if (!connected) {
+ printf("*** Resetting...\n");
+ delay(500); // time for the portal to send responses
+ ESP.restart();
+ while (true) delay(100);
+ }
+
+ printf("Wifi connected to %s\n", WiFi.SSID().c_str());
+
+ mqttSetup(config);
+
+ printf("\n===== Setup complete\n");
+}
+
+void reconfig() {
+ ESBConfig config;
+ bool saved = config.reconfig(CONF_CONN_TIMEOUT, CONF_PORTAL_TIMEOUT);
+ if (saved) {
+ mqttSetup(config);
+ }
+}
+
+void loop() {
+ //WiFi.printDiag(Serial);
+ printf("* Wifi:%s MQTT:%s\n", WiFi.isConnected()?"OK":"---", mqttClient.connected()?"OK":"---");
+ mqttLoop();
+
+ if (digitalRead(BUTTON) == LOW) reconfig();
+
+ delay(2000);
+}

examples/basic/platformio.ini

@@ -0,0 +1,37 @@
+[platformio]
+default_envs = usb
+src_dir = .
+
+[common]
+build_flags = -DASYNC_TCP_SSL_ENABLED
+lib_deps =
+ https://github.com/tve/AsyncTCP.git#mbed-tls-try2
+ https://github.com/me-no-dev/ESPAsyncWebServer.git
+ https://github.com/tve/ESPAsyncWiFiManager.git
+ https://github.com/tve/async-mqtt-client.git
+lib_ignore = ESPAsyncTCP
+
+[env:usb]
+#platform = https://github.com/platformio/platform-espressif32.git#feature/stage
+platform = espressif32
+framework = arduino
+board = nodemcu-32s
+build_flags = ${common.build_flags}
+# -DCORE_DEBUG_LEVEL=ARDUHAL_LOG_LEVEL_DEBUG
+lib_deps = ${common.lib_deps}
+lib_ignore = ${common.lib_ignore}
+lib_ldf_mode = chain+
+upload_port = /dev/ttyUSB0
+monitor_port = /dev/ttyUSB0
+monitor_speed = 115200
+
+[env:ota]
+platform = espressif32
+framework = arduino
+board = nodemcu-32s
+mqtt_device = esp32-c09b7ca4ae30
+build_flags = ${common.build_flags}
+lib_deps = ${common.lib_deps}
+lib_ldf_mode = chain+
+upload_protocol = custom
+extra_scripts = pre:../../publish_firmware.py

library.json

@@ -0,0 +1,16 @@
+{
+ "name":"ESPSecureBase",
+ "description":"Secure MQTT Base Project Libray for ESP32",
+ "keywords":"",
+ "repository":
+ {
+ "type": "git",
+ "url": "https://github.com/tve/esp32-secure-base.git"
+ },
+ "version": "0.1.0",
+ "frameworks": "arduino",
+ "platforms": "espressif32",
+ "build": {
+ "libCompatMode": "strict"
+ } 
+}

publish_firmware.py

@@ -1,15 +1,24 @@
 import requests
 import sys
 from os.path import basename, splitext
-from platformio import util
+#from platformio import util
 from datetime import date
 
 Import('env')
 #print env.Dump()
 
-project_config = util.load_project_config()
-ota_config = {k: v for k, v in project_config.items("mqtt_ota")}
-version = "2019-04-17" # project_config.get("common", "release_version")
+try:
+ import configparser
+except ImportError:
+ import ConfigParser as configparser
+
+config = configparser.ConfigParser()
+config.read("platformio.ini")
+
+#value1 = config.get("my_env", "custom_option1")
+#
+#ota_config = {k: v for k, v in config.get("mqtt_ota")}
+#version = "2019-04-17" # project_config.get("common", "release_version")
 
 #
 # Push new firmware to the OTA storage
@@ -30,11 +39,14 @@ def publish_firmware(source, target, env):
  ])
  print("URL: {0}".format(url))
 
+ #print(env.Dump())
+
  headers = {
  "Content-type": "application/octet-stream",
  }
- if ota_config["device"]:
- headers["mqtt_device"] = ota_config["device"]
+ mqtt_device = config.get("env:"+env['PIOENV'], "mqtt_device")
+ if mqtt_device:
+ headers["mqtt_device"] = mqtt_device
 
  r = None
  try:
@@ -58,7 +70,7 @@ def publish_firmware(source, target, env):
 
 
 # Custom upload command and program name
-print env["PROJECT_DIR"]
+print(env["PROJECT_DIR"])
 env.Replace(
  PROGNAME=basename(env["PROJECT_DIR"]),
  UPLOADCMD=publish_firmware

src/ESPSecureBase.cpp

@@ -0,0 +1,7 @@
+// The following are not used, but if they're not included here then platformio doesn't
+// find them when they're included in some other library. Sigh...
+
+#include <FS.h>
+#include <SPIFFS.h>
+#include <DNSServer.h>
+#include <ArduinoJson.h>