OTA LittleFS filesystem updates

[pietglas]

Dear @earlephilhower,

maybe a silly question, but have compressed (gzip -9) and non-compressed LittleFS filesystem updates using OTA been verified to work? The docu states that the core can do this, but it is a bit confusing as it mentions functionality from the esp8266 core (the eboot bootloader) which is not present in the pico core.

I am asking because i have been trying without success to update the LittleFS filesystem of my pico using a HTTP web server. These problems could well be my own fault; i took the HTTPUpdate, Updater and HTTPClient classes and made some modifications to get it to work with Ethernet instead of WiFi.

Updating the sketch works fine, both compressed and non-compressed. I got non-compressed filesystem updating to work, but had to make a modification for that. In the compressed case, I did define ATOMIC_FS_UPDATE. The file is downloaded and the (compressed) data written to flash, after rebooting i cannot read the filesystem anymore, which has then compressed size (FSInfo.usedBytes).

I could give more details, but first i wanted to make sure whether the problems i have are self-made or not :)

Thank you for this great core!

Best,
Piet

[earlephilhower]

I haven't tried it in a while, so there may be some bit rot, but I think this is the root cause:

... the (compressed) data written to flash ...

For ATOMIC updates (the only way to do a FS upgrade) the OTA code simply writes the data directly into flash as-received. Only the bootloader has gunzip support, not the main core (it's ~10K of code and ~40Kof RAM required so non-trivial). The core ATOMIC_xx OTA code just writes bytes as-received.

Can you try a FS upload uncompressed, instead?

[pietglas]

Thank your for the reply!

When i try the uncompressed uploading the program crashes, seemingly at UpdaterClass::_writeBuffer(), when calling

flash_range_erase((intptr_t)_currentAddress - (intptr_t)XIP_BASE, 4096) (Updater.cpp, line 295).

Printing _currentAddress shows that it is zero, presumably because it is initialized to 0 and set nowhere? If i set _currentAddress = _startAddress in UpdaterClass::begin(), uncompressed updates are working fine.

Regarding your comment:

For ATOMIC updates (the only way to do a FS upgrade) ...

ATOMIC_FS_UPDATES variable is referenced only once in the code, at HTTPUpdate.cpp, line 285, on ending the process:

#ifdef ATOMIC_FS_UPDATE
                if (_rebootOnUpdate) {
#else
                if (_rebootOnUpdate && !spiffs) {
#endif
                    rp2040.restart();
                }

It only has the effect of triggering a reboot in the case of updating the filesystem (spiffs true)? I can do an uncompressed fs update (with above modification) independent of defining ATOMIC_FS_UPDATE.

Upon rebooting, does the bootloader check if the filesystem is currently compressed?

[earlephilhower]

Looks like I should a) finish at least one coffee and b) read the code before commenting. 😆

Any ATOMIC_FS_UPDATE reference here should be deleted from the code, it's just not supported. On the 8266 there's a separate, unused area of flash for OTA binaries so with this flag you can tell the main app OTA code to stuff a FS image into this spot, instead of overwriting the actual FS region. Then, on the 8266 the eboot will decompress and write from the OTA area into the FS area.

On this core I chose to do something different w/OTA. Because most boards will be stuck w/2MB of flash I didn't want to waste so much area and limit FS and app size. So instead of the bootloader reading an unused region to overwrite flash, the bootloader can actually read and decompress a file from the LittleFS filesystem and into another region of flash.

So, it's impossible for the bootloader to overwrite the FS because it needs to read from it...you'd overwrite the file you were reading in that case! And so I should drop any ref to ATOMIC.

That means that you can update the FS, but you cannot use a compressed fs.gz file. You need to upload to OTA the uncompressed binary image.

Again, I suggest you try uploading an uncompressed FS image (and no ATMOIC_X).

[pietglas]

Coffee is essential :D Thank you for clarifying! I will stick to uploading uncompressed fs images.

[pietglas]

Please note my previous comment:

When i try the uncompressed uploading the program crashes, seemingly at UpdaterClass::_writeBuffer(), ...

Unfortunately i cannot test this with the original codebase, as i do not have a pico with wifi support. In case you encounter the same error in the future, the modification i described above fixed it in my case.

[earlephilhower]

Yup, I think you're 100% right on this from just code inspection. It used to be set in the main path, but now since we're just writing a file that variable isn't set or used in the normal case, but in the FS is looks like it's still needed. Thx for the reminder!

[pietglas]

Happy i could contribute!