Just a thought. Large http.Updates

[Hiddenvision]

Sorry if this is the wrong place for suggestions but delete as required.

To Precede.
I've been using the http Update process (v2.30) for a year or two. Very happy.
But I guess my code has always been smaller than 500k as I never had any failed to update.
Recently updated to V2.4 and Bam, updates fail. (Due to Sketch being too big).
V2.3 Binsize 489k
V2.4 Binsize 538k

Got me thinking.?

Is it possible to:
1: Download the update.Bin from http into FS,
2: Then flash a small Minimal Bin file from FS and reboot,
3: Then finally the minimal boot will simply mounts the FS and flash the main update.bin and reboot again.
4: On reboot delete the update bin if required.

Would that Present any issues? Apart from someone having to write the code.!

OR

Can the Flash allocation be changed to give 1M + 1M + 2MB
rather than the current 512 + 512 + 3MB

I actually use 64Mb memory so lots of space to play with.

Sorry again if this is posted in the wrong place.
Hv.

[Hiddenvision]

Ok So I tried a quick test and looks like Option 1 will work.
Just got to make a simple project that contains enough to Flash from FS
and also include a little rescue up-loader in case the downloaded file failed.

Be nice if Option 2 was possible though.
Any thoughts on that.. anyone.

[d-a-v]

For the generic board and 64MB we currently have at least 16MB(15MB spiffs) and 4M(3MB spiffs) (which you seem to use).
Would you need 16MB(14MB spiffs) / 4M(2MB spiffs) ?
You can try it yourself as per this comment and following.

I'm not an expert myself about OTA constraints, but beeing able to live-update 1MB sketchs would be I guess very welcome.

Please share your findings.

Out of topic, your original concern beeing binary size, you have now two ways of reducing it. Consider tools/board.txt.py --nofloatboards generator option, and Tools/DebugLevel/NoAssert-NDEBUG in menus.

[Hiddenvision]

Hi @d-a-v
Many thanks for the pointers.
My initial findings. Gonna need more coffee.

I am guessing that you refer to editing/recompiling the Source
rather than editing files on the local machine.? or am I being dense.

I am still running 2.3, as my projects when compiled on 2.4 have a few issues.
Least not the OTA size thing but qwerks with some of the other functions I used.
Not worth mentioning until I can document the issues and research solutions.
But too much for me to deal with all in one go right now.

Hv.

[d-a-v]

@Hiddenvision You need latest git master ("nightly" version after 2.4.0) for the mentioned points above. No need to edit source files.

When you have dealt with your other 2.4.0 issues, I guess core-2.4.1 will be available through the boards manager, which will include the mentioned options.

edited

[Hiddenvision]

I had a play with various things.
I got the idea from your previous pointers.
Looking thru various files, I added in the missing menu options in the boards.txt file.
Then I created a few more xxx.ld files.
Reprogramming the devices seems to have changed the flash allocation. Cool.

I have attached the .ld files and also made the edits to the boards.txt.py file
Happy for this to be included in the builds if desired.
Not sure if the numbers are right but I am sure someone with more info will correct if required.

On that subject of numbers what is the largest compiled sketch the ESP will run.
Is it only limited by what's allocated.?
Would seem logical to increase the max sketch size for the larger flash devices.

FLASH_ALLOCATION_TEST_sdk.zip

This was a very trim & Edited Boards.txt file
boards.txt.zip

So if you want to try,
DO NOT TRY THIS ON PRODUCTION FIRMWARE.!!
1: Backup your current boards.txt file,
2: Extract and use the one above, or edit a copy of your existing file, adding in the extra menu options.
3: You may also want to backup the ld folder.
4: Extract the extra .ld files from the other zip.

?? It may work if you put the boards.txt.py file in place but as it is missing from the install I guess not. ??

5: Restart the IDE.

if it all goes pear shaped, read this post backwards to reverse the process.
if your not sure where things are, then stop right there.

Hv.

[Hiddenvision]

Just to update on the findings.

I just updated to 2.4
Realised I forgot to copy the old boards.txt file
So, made all the edits from scratch again,
Programmed device using http update, (Serial Programming is so yesterday)
the headers are "pre-sketch-update" in case they look out of sequence.

x-ESP8266-free-space: 1605632
x-ESP8266-sketch-size: 489648 << Notice the size using 2.3
x-ESP8266-chip-size: 4194304
x-ESP8266-sdk-version: 1.5.3(aec24ac9)
x-ESP8266-mode: sketch
Worked fine.

Re compiled again Started update process again.

x-ESP8266-free-space: 1564672
x-ESP8266-sketch-size: 531536 << Notice the size using 2.4
x-ESP8266-chip-size: 4194304
x-ESP8266-sdk-version: 2.1.0(deb1901)
x-ESP8266-mode: sketch
PERFECT. Complete 100%

And after a reboot.

x-ESP-fw-ver: 1802012220
x-ESP-heap-free: 20384
x-ESP-SD-size: 1953282
x-ESP-SD-free: 1730645
x-ESP-SD-ver: 100
x-Voltage: 3055
x-ESP-free-space: 1564672
x-ESP-sketch-size: 531536 << Notice the size using 2.4
x-ESP-chip-size: 4194304
x-ESP-sdk-version: 2.1.0(deb1901)

just gotta work on that POST ARGS Error I get on some pages with 2.4 and were sorted.
Who's gonna be the first to do a 4 or 8meg update.?

Now just got to find a pretty way to optimise all the F() Strings.

Hv.

[d-a-v]

Happy for this to be included in the builds if desired

Why not making a pull request for these ?

On that subject of numbers what is the largest compiled sketch the ESP will run.

someone corrects me, I think 1MB is the max addressable program space by design.

[Hiddenvision]

Why not make a pull request.?

Ran out of coffee.
Plus I don't know how yet, and not sure if it is all correct, AND, so much more.
I tried to follow the guide to install from Git and was going to do some edits.
1+1 does not always equal 42. so another day.

But hey, I am pretty good at cooking a sponge cake.

I think 1MB is the max addressable program space by design.

Darn, 1MB limit, Who would'a thunk it.
I know it sounds horrible, but I hope your wrong. ;-)

Hv.

[Hiddenvision]

just gotta work on that POST ARGS Error.

Yep 2.4 handles things slightly different.
or is fussier about my code doing things wrong.
Either way , Sorted now.

[d-a-v]

Darn, 1MB limit, Who would'a thunk it.
I know it sounds horrible, but I hope your wrong. ;-)

I am wrong but that's another story.

[Hiddenvision]

This was the required edits for the new version boards.txt.py

That looked rubbish, Zip file below.

[Hiddenvision]

boards.txt.py.zip