New Feature Blab: OTA and New Neighbors

[timmbogner]

There are two features which I'd like to propose as the next additions. I'm hoping to do some light discussion here, then jump to a pull request for each:

1. OTA updates to be initiated by a SystemPacket or other trigger: OTA firmware updating is tricky to add to a system that actively avoids WiFi. We'll need to have a way to trigger a separate mode for updating the firmware. My first proposal is to invoke this mode with a system command, but I think there should also be an alternative. We could allow the user to bring a pin high/low to drop into the new mode as well. The pin could be customized in config to any GPIO. I think we could just query that pin in the loop, doesn't need to be attached to an interrupt or anything.
2. Adding two more neighbors to make four total: This doesn't really add anything new, so I don't expect any surprises. It'll just be a lot of repetition. @aviateur17 let me know if you can think of anything within timekeeping that would be broken by this.

[Stips5]

What about nodes beeing connected to wifi all the time, related to OTA

[timmbogner]

I have thought about bringing MQTT and UART protocols to the node sketch, but it's been a while.
It doesn't hurt and adds some usefulness for nodes within WiFi range. There aren't a ton of drawbacks, except it does add some complication. With WiFi configs being added to the node, it would be a good time.

I'll keep this in mind!

[aviateur17]

1. OTA updates to be initiated by a SystemPacket or other trigger: OTA firmware updating is tricky to add to a system that actively avoids WiFi. We'll need to have a way to trigger a separate mode for updating the firmware. My first proposal is to invoke this mode with a system command, but I think there should also be an alternative. We could allow the user to bring a pin high/low to drop into the new mode as well. The pin could be customized in config to any GPIO. I think we could just query that pin in the loop, doesn't need to be attached to an interrupt or anything.

Shouldn't be too bad. The irrigation example shows that we can receive and process commands.
Is this primarily for nodes?
If ESP-NOW, disable and enable Access Point mode for WiFi?
If LoRa just enable AP mode otherwise normal operations?
Timeout of like 5 minutes if nothing happens and then go back to FDRS mode?
IP address all the same or based on Reading ID or something else?
Web interface to load firmware?

2. Adding two more neighbors to make four total:

Did you want to add two more ESP-NOW Neighbors and two more LoRa Neighbors?

let me know if you can think of anything within timekeeping that would be broken by this.

Shouldn't be too difficult. in fdrs_lora.h anywhere there is LoRa1 or LoRa2 we would need to add LoRa3 and LoRa4
Similar on the ESP-NOW side, add more events and actions. Shouldn't be too difficult.

[timmbogner]

Okay I just had a sudden change of thinking on the OTA thing... The easiest way to do this is to simply have a function like updateFDRS() which enters the the OTA update waiting mode. The user is then in charge of creating the trigger for this function if they want OTA updates. The dead-simplest way to trigger it remotely would be the adding it as a controller 'type'. So for example in the Irrigation controller, we would just add an option for type 2 (0. SET, 1. GET, 2. UPDATE) and run updateFDRS() from within it.

Now on gateways it's trickier. I think we'll still need to invoke it directly using a SystemPacket.

[aviateur17]

Okay I just had a sudden change of thinking on the OTA thing... The easiest way to do this is to simply have a function like updateFDRS() which enters the the OTA update waiting mode. The user is then in charge of creating the trigger for this function if they want OTA updates. The dead-simplest way to trigger it remotely would be the adding it as a controller 'type'. So for example in the Irrigation controller, we would just add an option for type 2 (0. SET, 1. GET, 2. UPDATE) and run updateFDRS() from within it.

Now on gateways it's trickier. I think we'll still need to invoke it directly using a SystemPacket.

If you don't mind, it'd be nice to implement the SystemPacket mechanism where it puts the controller in OTA mode. That way we can reuse that mechanism for other options in the future and it makes it easier for users as they wouldn't need to write their own code to use the updateFDRS function.

Regarding the controller showing up as a COM port in Arduino I think I'll have to look that up. I would assume it's the same as platform IO as all you do is type in the IP address and (optionally) password and Arduino tries to communicate via the standard OTA port to upload the sketch. If you have a link for more detail I'd be open to that otherwise I'll try doing a bit of searching in the future.

ESP-Home project has a nifty feature that will push firmware to ESP from web browser even when the device is connected to USB port on the same PC. Once the initial base firmware has been pushed you can then remove the USB connection and use WiFi OTA for all future updates.

[timmbogner]

If you don't mind, it'd be nice to implement the SystemPacket mechanism where it puts the controller in OTA mode.

That's totally fine, as both concepts can still exist together. I think the main take-away should be a user-accessible update function.

The reason for my change of thinking was that it's actually not that simple to send a SystemPacket directly to a node, because you don't know its address beforehand. I think my initial idea for the SP trigger was to send out a broadcast packet from an outside device, and that still may be possible. By moving the trigger to the "user level" I figured it would be easier to invoke OTA directly from the front-end.

The other option would be to tell a gateway to send out SPs to put all of its registered controllers into OTA mode. I don't dislike this, but we'll have to decide how to trigger it. Using the outside device to tell the gateway to tell the controllers to update? I think that would be reasonable actually. Still thinking...

Regarding the controller showing up as a COM port in Arduino

It sounds like Arduino IDE makes it even simpler and auto-detects the IP. Deff check it out.

ESP-Home project has a nifty feature that will push firmware to ESP from web browser even when the device is connected to USB port on the same PC. Once the initial base firmware has been pushed you can then remove the USB connection and use WiFi OTA for all future updates.

The first half is what I'd love for FDRS. Unfortunately I don't think we can keep it as simple for OTA because of the ESP-NOW /WiFi conflict.

[aviateur17]

What do you think of using a Reading ID to target a specific controller going into Updating mode?

[timmbogner]

I don't love it... In nodes with multiple IDs, such as Irrigation, it doesn't make sense to define singular READING_ID. Also, several controllers could be subscribed to the same ID. Or what if the controller wants to unsubscribe from one ID and sub to another? In general I'm trying to signal to our users that they aren't tied to just one ID and that they are free to implement more complex code that sends lots of data. Having a singular ID in config sends a mixed signal in that regard, which is why that config isn't actually tied to anything (except the one version of loadFDRS()).

That was sort of why I wanted to leave it to the user. We're in a tricky spot trying to talk to individual nodes without the registration stuff. I think it's easiest to just instruct a gateway to send OTA-trigger SystemPackets to every node listening/registered to it.

I know at first my idea was for an external device to send the nodes directly into OTA mode with a SP, but I think that's too convoluted upon further thought. I really think the trigger should come from a gateway, a user's code, or via a GPIO.