section .text will not fit in region iram1_0_seg

[dalbert2]

Basic Infos

• This issue complies with the issue POLICY doc.
• I have read the documentation at readthedocs and the issue is not addressed there.
• I have tested that the issue is present in current master branch (aka latest git).
• I have searched the issue tracker for a similar issue.
• [ N/A ] If there is a stack dump, I have decoded it.
• I have filled out all fields below.

Platform

• Hardware: [ESP-12F]
• Core Version: [2.4.1 Release]
• Development Env: [Arduino IDE and VisualMicro (tried both)]
• Operating System: [Windows]

Settings in IDE

• Module: [Generic ESP8266 Module|Nodemcu|WifInfo (tried all 3)]
• Flash Mode: [qio]
• Flash Size: [4MB]
• lwip Variant: [v2 Lower Memory]
• Reset Method: [nodemcu]
• Flash Frequency: [40Mhz]
• CPU Frequency: [160MHz]
• Upload Using: [SERIAL]
• Upload Speed: [115200] (serial upload only)

Problem Description

Section full when building with 2.4.1 (works fine with 2.4.0)

Detailed problem description goes here.
Codebase that builds perfectly under 2.4.0 will not build with 2.4.1

Building 1.0.22 release using 2.4.0 core for WifInfo (ESP12 4M/1M):

Compiling 'Gateway2' for 'WifInfo'
Program size: 442,404 bytes (used 42% of a 1,044,464 byte maximum) (164.09 secs)
Minimum Memory Usage: 46580 bytes (57% of a 81920 byte maximum)

Building 1.0.22 release using 2.4.1 core for WifInfo (ESP12 4M/1M):

Compiling 'Gateway2' for 'WifInfo'
ld.exe: C:\Users\david\AppData\Local\Temp\VMBuilds\Gateway2\esp8266_wifinfo\Debug\Gateway2.ino.elf section .text will not fit in region iram1_0_seg
 

Error linking for board WifInfo
Build failed for project 'Gateway2'
collect2.exe*: error: ld returned 1 exit status

Building 1.0.22 release using 2.4.1 core for Generic ESP8266 module using Arduino IDE

Arduino: 1.8.5 (Windows 10), Board: "Generic ESP8266 Module, 160 MHz, nodemcu, 26 MHz, 40MHz, QIO, 4M (1M SPIFFS), 2, v2 Lower Memory, Disabled, None, Only Sketch, 115200"
...
Linking everything together...
"C:\Users\david\AppData\Local\Arduino15\packages\esp8266\tools\xtensa-lx106-elf-gcc\1.20.0-26-gb404fb9-2/bin/xtensa-lx106-elf-gcc" -g -w -Os -nostdlib -Wl,--no-check-sections -u call_user_start -u _printf_float -u _scanf_float -Wl,-static "-LC:\Users\david\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.4.1/tools/sdk/lib" "-LC:\Users\david\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.4.1/tools/sdk/ld" "-LC:\Users\david\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.4.1/tools/sdk/libc/xtensa-lx106-elf/lib" "-Teagle.flash.4m1m.ld" -Wl,--gc-sections -Wl,-wrap,system_restart_local -Wl,-wrap,spi_flash_read  -o "C:\Users\david\AppData\Local\Temp\arduino_build_376519/Gateway2.ino.elf" -Wl,--start-group "C:\Users\david\AppData\Local\Temp\arduino_build_376519\sketch\gw2icon.c.o" "C:\Users\david\AppData\Local\Temp\arduino_build_376519\sketch\DNSServer.cpp.o" "C:\Users\david\AppData\Local\Temp\arduino_build_376519\sketch\Gateway2.ino.cpp.o" "C:\Users\david\AppData\Local\Temp\arduino_build_376519\sketch\command.cpp.o" "C:\Users\david\AppData\Local\Temp\arduino_build_376519\sketch\commands.cpp.o" "C:\Users\david\AppData\Local\Temp\arduino_build_376519\sketch\config.cpp.o" "C:\Users\david\AppData\Local\Temp\arduino_build_376519\sketch\crc16.cpp.o" "C:\Users\david\AppData\Local\Temp\arduino_build_376519\sketch\crc32.cpp.o" "C:\Users\david\AppData\Local\Temp\arduino_build_376519\sketch\filesystem.cpp.o" "C:\Users\david\AppData\Local\Temp\arduino_build_376519\sketch\fw_version.cpp.o" "C:\Users\david\AppData\Local\Temp\arduino_build_376519\sketch\fwupdate.cpp.o" "C:\Users\david\AppData\Local\Temp\arduino_build_376519\sketch\led.cpp.o" "C:\Users\david\AppData\Local\Temp\arduino_build_376519\sketch\neighbors.cpp.o" "C:\Users\david\AppData\Local\Temp\arduino_build_376519\sketch\network.cpp.o" "C:\Users\david\AppData\Local\Temp\arduino_build_376519\sketch\protocol.cpp.o" "C:\Users\david\AppData\Local\Temp\arduino_build_376519\sketch\radio.cpp.o" "C:\Users\david\AppData\Local\Temp\arduino_build_376519\sketch\report.cpp.o" "C:\Users\david\AppData\Local\Temp\arduino_build_376519\sketch\semtech.cpp.o" "C:\Users\david\AppData\Local\Temp\arduino_build_376519\sketch\sensor.cpp.o" "C:\Users\david\AppData\Local\Temp\arduino_build_376519\sketch\switch.cpp.o" "C:\Users\david\AppData\Local\Temp\arduino_build_376519\sketch\telnet.cpp.o" "C:\Users\david\AppData\Local\Temp\arduino_build_376519\sketch\web_css.cpp.o" "C:\Users\david\AppData\Local\Temp\arduino_build_376519\sketch\web_html.cpp.o" "C:\Users\david\AppData\Local\Temp\arduino_build_376519\sketch\web_js.cpp.o" "C:\Users\david\AppData\Local\Temp\arduino_build_376519\sketch\webserver.cpp.o" "C:\Users\david\AppData\Local\Temp\arduino_build_376519\sketch\src\NtpClient\NTPClientLib.cpp.o" "C:\Users\david\AppData\Local\Temp\arduino_build_376519\sketch\src\Time\DateStrings.cpp.o" "C:\Users\david\AppData\Local\Temp\arduino_build_376519\sketch\src\Time\Time.cpp.o" "C:\Users\david\AppData\Local\Temp\arduino_build_376519\libraries\EEPROM\EEPROM.cpp.o" "C:\Users\david\AppData\Local\Temp\arduino_build_376519\libraries\ESP8266WiFi\ESP8266WiFi.cpp.o" "C:\Users\david\AppData\Local\Temp\arduino_build_376519\libraries\ESP8266WiFi\ESP8266WiFiAP.cpp.o" "C:\Users\david\AppData\Local\Temp\arduino_build_376519\libraries\ESP8266WiFi\ESP8266WiFiGeneric.cpp.o" "C:\Users\david\AppData\Local\Temp\arduino_build_376519\libraries\ESP8266WiFi\ESP8266WiFiMulti.cpp.o" "C:\Users\david\AppData\Local\Temp\arduino_build_376519\libraries\ESP8266WiFi\ESP8266WiFiSTA.cpp.o" "C:\Users\david\AppData\Local\Temp\arduino_build_376519\libraries\ESP8266WiFi\ESP8266WiFiScan.cpp.o" "C:\Users\david\AppData\Local\Temp\arduino_build_376519\libraries\ESP8266WiFi\WiFiClient.cpp.o" "C:\Users\david\AppData\Local\Temp\arduino_build_376519\libraries\ESP8266WiFi\WiFiClientSecure.cpp.o" "C:\Users\david\AppData\Local\Temp\arduino_build_376519\libraries\ESP8266WiFi\WiFiServer.cpp.o" "C:\Users\david\AppData\Local\Temp\arduino_build_376519\libraries\ESP8266WiFi\WiFiServerSecure.cpp.o" "C:\Users\david\AppData\Local\Temp\arduino_build_376519\libraries\ESP8266WiFi\WiFiUdp.cpp.o" "C:\Users\david\AppData\Local\Temp\arduino_build_376519\libraries\SPI\SPI.cpp.o" "C:\Users\david\AppData\Local\Temp\arduino_build_376519\libraries\ESP8266HTTPClient\ESP8266HTTPClient.cpp.o" "C:\Users\david\AppData\Local\Temp\arduino_build_376519\libraries\ESP8266httpUpdate\ESP8266httpUpdate.cpp.o" "C:\Users\david\AppData\Local\Temp\arduino_build_376519\libraries\Ticker\Ticker.cpp.o" "C:\Users\david\AppData\Local\Temp\arduino_build_376519\libraries\ESP8266mDNS\ESP8266mDNS.cpp.o" "C:\Users\david\AppData\Local\Temp\arduino_build_376519\libraries\ESP8266SSDP\ESP8266SSDP.cpp.o" "C:\Users\david\AppData\Local\Temp\arduino_build_376519\libraries\ESP8266WebServer\ESP8266WebServer.cpp.o" "C:\Users\david\AppData\Local\Temp\arduino_build_376519\libraries\ESP8266WebServer\ESP8266WebServerSecure.cpp.o" "C:\Users\david\AppData\Local\Temp\arduino_build_376519\libraries\ESP8266WebServer\Parsing.cpp.o" "C:\Users\david\AppData\Local\Temp\arduino_build_376519\libraries\ESP8266WebServer\detail\mimetable.cpp.o" "C:\Users\david\AppData\Local\Temp\arduino_build_376519/arduino.ar" -lhal -lphy -lpp -lnet80211 -llwip2 -lwpa -lcrypto -lmain -lwps -laxtls -lespnow -lsmartconfig -lairkiss -lwpa2 -lstdc++ -lm -lc -lgcc -Wl,--end-group  "-LC:\Users\david\AppData\Local\Temp\arduino_build_376519"
c:/users/david/appdata/local/arduino15/packages/esp8266/tools/xtensa-lx106-elf-gcc/1.20.0-26-gb404fb9-2/bin/../lib/gcc/xtensa-lx106-elf/4.8.2/../../../../xtensa-lx106-elf/bin/ld.exe: C:\Users\david\AppData\Local\Temp\arduino_build_376519/Gateway2.ino.elf section `.text' will not fit in region `iram1_0_seg'

collect2.exe: error: ld returned 1 exit status

Using library EEPROM at version 1.0 in folder: C:\Users\david\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.4.1\libraries\EEPROM
Using library ESP8266WiFi at version 1.0 in folder: C:\Users\david\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.4.1\libraries\ESP8266WiFi
Using library SPI at version 1.0 in folder: C:\Users\david\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.4.1\libraries\SPI
Using library ESP8266HTTPClient at version 1.1 in folder: C:\Users\david\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.4.1\libraries\ESP8266HTTPClient
Using library ESP8266httpUpdate at version 1.1 in folder: C:\Users\david\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.4.1\libraries\ESP8266httpUpdate
Using library Ticker at version 1.0 in folder: C:\Users\david\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.4.1\libraries\Ticker
Using library ESP8266mDNS in folder: C:\Users\david\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.4.1\libraries\ESP8266mDNS (legacy)
Using library ESP8266SSDP in folder: C:\Users\david\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.4.1\libraries\ESP8266SSDP (legacy)
Using library ESP8266WebServer at version 1.0 in folder: C:\Users\david\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.4.1\libraries\ESP8266WebServer
exit status 1
Error compiling for board Generic ESP8266 Module.

MCVE Sketch

It's a big proprietary codebase, I'm sorry, I can't post it.

Debug Messages

Debug messages go here

[Pablo2048]

Well, you can't post the code so how do you think one can help you? Do you use F(), PSTR, ICACHE_FLASH_ATTR...?

[dalbert2]

Pablo, I appreciate the issue with posting defects without code; unfortunately, this issue is with the final link step. There are many libraries and tools where addressing problems does not require public sharing of a complete source code base. Were this required to debug popular toolchains, they couldn't be used for anything closed-source.

I posted the issue for two reasons: 1) it seems like a serious problem with the 2.4.1 release and 2) sometimes the issue is obvious to an experienced developer who may be able to point me in the right direction quickly (I've already spent hours on this).

I am very experienced with bare-metal ARM/Cortex development using the gnu toolchain, but am relatively new to the xtensa tools and so far have been blissfully isolated from them thanks to the delightful Arduino/ESP8266 Core IDE/tools you have so graciously made possible. I'm going through the linker scripts and build products now to figure out what's going on and will post more on this as I dig deeper.

[Pablo2048]

@dalbert2 you have to test the issue with git version first (IMO this has been suggested many times by developers). IMO no one can tell you what is wrong with near to zero informations about your project. To be more specific - did you check that you use ICACHE_FLASH_ATTR decorators where possible as i suggest? That's all i can say...

[igrr]

I think we have recently moved some stuff (vtables?) into IRAM to free up some heap space. If your application used almost all available IRAM in the previous version it may fail to link with the new one. You can check this by running xtensa-lx106-elf-objdump -h your_sketch.elf with the old (2.4.0) version, and see how much of the IRAM space was used.

[dalbert2]

Thank you both for your help!

Pablo, yes I make extensive use if ICACHE_FLASH_ATTR for all code except functions that will be called from an interrupt context. I also make extensive use of PROGMEM for literals.

Ivan, thank you for jumping in! My .text section (built with 2.4.0) is 0x7f43 bytes, so yes, it's on the edge and apparently the change to 2.4.1 pushed it over the limit. The majority of my code (0x60a2d bytes) is in .irom0.text located in SPI flash, but it looks like I will need to find a way to reclaim some instruction RAM.

A request: the build output reports "Memory Usage" which I had assumed to be RAM, but I now understand is only data RAM (.data+.rodata+.bss). It also reports "Program Size" which is SPI Flash usage (.irom0.text+.text+.rodata+?). However, it doesn't let you know how close you are to the 0x8000 (32K) limit of instruction RAM which is actually the tightest memory constraint on the platform. It would be very nice if the build output included % IRAM used. This would help many Arduino users who won't be familiar with code segmentation at all and it will also help experienced embedded developers who are not familiar with the split regions for instruction and data RAM (since your very nice Arduino environment insulates us from all of that). For example, I knew some of my code would be located in RAM, but I didn't realize that instruction RAM was separate from data RAM, so when I saw I had plenty of free RAM, I thought I was OK.

Thank you both again for the quick responses and for helping me understand the issue!

For my reference and for anyone who comes across this later:
https://github.com/esp8266/esp8266-wiki/wiki/Memory-Map

[Adam5Wu]

I have experienced similar issue recently - if I enable debug serial I would get the error message.

Like @igrr suspected, my issue was related to the vtable placement.
It seems the base SDK + Arduino already consumed around 7300h iram;
And vtables in my code pushed the use to around 7fc0h;
There is not enough room for debug serial code anymore.

I had to remove some virtual destructors in the base classes to reduce vtable size.

[dalbert2]

@Adam5Wu I haven't had a chance to check yet, but if the base SDK + Arduino pushes IRAM to 0x7300 bytes, that is a very serious problem; it means there's less a smidge more than 3K instruction RAM left for the user.

[devyte]

Hi @dalbert2 , I still don't have plans to travel to Maryland, so that beer is still pending :)

One PR that could be relevant for you is #4384 , although I somehow doubt it will be enough.
As explained above, several things have been moved out of heap into iram, which makes things tighter. I'm considering whether to leave this issue open to track related effort, but a quick look at the current code declared with ICACHE_RAM_ATTR tells me there isn't that much that can really be moved here in the core. I'm going to take another overview over the weekend, and decide then whether to keep this open or not.

Are you using your own ISRs? If you are, and they contain significant amount of code, I have an idea that may help you out. Let me know if that's the case, and we can discuss details. Also, if it's viable, I could take a private look at your code, and provide some feedback as to what you could do, as I have done a couple of times for others.

[dalbert2]

Hi @devyte, the beer will be waiting when you have time to visit!

Thank you for the pointer to #4384 If the cont_* functions haven't been moved to SPI Flash in 2.4.1, I'll try making a build using git head (the instructions for that don't seem to work but I'll figure it out) or I will make the changes to 2.4.1 locally; thanks for the suggestion and for finding more IRAM space to reclaim!

My code has several ISRs; the application is a wireless gateway (proprietary telemetry network to WiFi) so there are ISRs to respond to radio events and a couple of front-panel push-buttons. My understanding is that any code that runs in the interrupt context must be located in IRAM. Now that I understand the problem (extremely limited space for ISR code), I can likely fix by reducing ISR functionality and shifting more of the work to the event loop. If I run into trouble, I'll happily take you up on your kind offer and if you have other approaches for freeing up IRAM, I'd love to hear them.

I still haven't checked base free IRAM, but if it really is ~3K as Adam reported, I think it's a significant issue that merits further examination. Because the ESP8266/Arduino is an embedded platform, anyone using it for non-trivial applications will likely have some ISR code. At the very least, I think the need for ICACHE_FLASH_ATTR and ICACHE_RAM_ATTR decorators should be front and center in the documentation, covered at least as well as the PROGMEM documentation since this is not an obvious thing for developers new to this platform and likely bites everyone as soon as they start writing non-trivial programs. I am not part of this project, but will happily contribute the documentation if permitted.

BTW, with respect to your comment in #797, IMO any developer who is writing interrupt handlers should be able to do so without the need for garbage collection and we'd benefit far more from better interrupt handling behavior and more available instruction RAM. Thanks again!

[Eszartek]

I hit against this error right after 2.4.1 was committed (I normally sync to HEAD every few days), so I just rolled back to 2.4.0, expecting this just to be some minor commit typo that would be fixed soon. I use no extra ISR's (but I was planning to soon), and I try hard to not to use iram for variables, so this change in iram usage may be a bigger issue for me than I first thought.

Does anyone have a list of advanced best practices to reduce iram usage other than the typical PROGMEM/PSTR()/F() wrapping?

My .text usage under 2.4.0 is 0x7480, not quite as tight as dalbert2's project, but clearly pushing the limits.

Backing out the vtables change from #4179 itself doesn't help , and all was fine leading up to 2.4.1 anyway, so I'm guessing something else is using more iram in 2.4.1 as well, but I can't pinpoint it yet.

It is commit: 170911a that triggers the blowout of iram for me. I'll look into why that is.

On the previous commit 0643d6e .text usage is at 0x74f8. So, 2.4.1 works for me, I got confused until I tested further.

[Eszartek]

So far, my first guess is that now that gdb_hooks.h is included explicitly in core_esp8266_main.cpp, it is pulling in extra code bits that use iram, even though it is supposed to be just a stub when not using gdb.

Update: I checked out HEAD, reverted core_esp8266_main.cpp to the version from the last working commit 0643d6e and my project builds fine. Nope, I messed up again, still working on it.

[devyte]

@dalbert2 Contributions are always welcome. If you propose a change, I'll take a look.
If you don't mind a bit of additional overhead in calling the code inside the ISR, one idea currently on the table is to schedule a function from the ISRs. The scheduled function then gets called once as though it were in the loop.
HOWEVER: currently there are two caveats with this:

1. the function that does the scheduling is currently not declared for IRAM (see Schedule.h)
2. the scheduled functions currently get called after the next loop. It is being discussed moving that to before the next loop => latency in reaching your code

The idea is that you move out all your ISR code out of IRAM, and move that single scheduling function into IRAM.

There's a PR that sort of goes along these lines, but for the Ticker instead of hw ISRs: #4209 . It can serve to explain the idea.

[hreintke]

@devyte I have also been thinking/working on de scheduled_interrupt addition to functionalinterrupt (#2745)
Can finish/clean that and create a PR for that.

[devyte]

@hreintke That sounds interesting, and I've been thinking of a rewrite of the scheduled function code in C++.
BTW, #2738 is also yours. Is that superseded by #4209 ?
And about functionalInterrupt, there's a mem leak in the current implementation. Can you think of a way to fix it? When I looked at it, I didn't see an obvious solution.
But let's discuss that elsewhere.

[Adam5Wu]

I hit the IRAM wall again when I am trying to make a moderate complexity project, with ArduinoJson, AsyncWebServer, AsyncMQTT, vFATFS.

The vtable size is 1750h and there no way it could fit into IRAM in the near term, even with all the improvement efforts mentioned in above discussions...

Is there an easy way to make vtable location configurable? I am not very familiar with the linker part of workflow, is it possible to have #ifdef in the eagle.app.v6.common.ld?