Stream File Problems 2.6.2

[laercionit]

##Platform

• Hardware: ESP-12
• Core Version: 2.6.2
• Development Env: Arduino
• Operating System: Windows

Settings in IDE

Problem Description

Webserver stream problems:

bool ServerFileReadFS(String Path) {
  Waiting();
  if (Path.endsWith(F("/"))) Path += F("home.htm");
  String contentType = ServerContentType(Path);
  String PathWithGz = Path + F(".gz");
  if (FilesFSExists(PathWithGz))  Path += F(".gz");
  if (FilesFSOpen(Path)) {
    DebugSerial.println(String(F("ServerFileReadFS: Load ")) + (Path));
    if (server.streamFile(FilesFS, contentType) == FilesFS.size()) {
      FilesFSClose();
      return 1;
    } //else DebugSerial.println(String(F("ServerLoadFile: FS Error stream File ")) + (Path));
  } //else DebugSerial.println(String(F("ServerLoadFile: FS File not Exists ")) + (Path));
  FilesFSClose();
  return 0;
}

Debug Messages

ServerFileReadFS: Load /home.htm.gz

User exception (panic/abort/assert)
Panic src/include/DataSource.h:93 const uint8_t* BufferedStreamDataSource<TStream>::get_buffer(size_t) [with TStream = Stream; uint8_t = unsigned char; size_t: Assertion 'cb == stream_rem' failed.

>>>stack>>>

ctx: cont
sp: 3ffffd30 end: 3fffffc0 offset: 0000
3ffffd30:  3fff6364 0000036f 3fff300c 40245e0d  
3ffffd40:  000000fe 00000000 00000000 00000000  
3ffffd50:  00000000 00000000 00000000 4028f974  
3ffffd60:  3fff1768 00000000 3fff4b0c 0000036f  
3ffffd70:  0000036f 00000000 3fff4b0c 402427c6  
3ffffd80:  00000000 00000000 4bc6a7f0 4024280e  
3ffffd90:  3fff5b5c 00000000 3fff4b0c 40234f6a  
3ffffda0:  00000000 3fff02c8 00000020 0000036f  
3ffffdb0:  00000000 0000036f 3fff5b5c 40235145  
3ffffdc0:  3fff5b5c 0000086a 0000001c 0000086a  
3ffffdd0:  00000001 3fff0bfc 3fff5b5c 40235206  
3ffffde0:  3fff5b5c 3fff0bfc 00000000 40235740  
3ffffdf0:  80fffe88 3ffffe10 3fff0bfc 0000086a  
3ffffe00:  3ffffe88 3fff02c8 3fff0bfc 4022f5e1  
3ffffe10:  3fff4b00 000c000f 80fffe4c 40240a51  
3ffffe20:  3fff2134 3ffffeb0 3ffffeb0 3fff1728  
3ffffe30:  00000000 00000001 3fff0bfc 4022f916  
3ffffe40:  3fff5f00 0023002f 80ff006d 3fff4b00  
3ffffe50:  0017001f 8028d524 3fff4b00 000c000f  
3ffffe60:  80000000 3fff2100 000c000f 80240c30  
3ffffe70:  3fff2100 000c000f 80ff006d 3fff4aec  
3ffffe80:  000c000f 00ff0308 74786574 6d74682f  
3ffffe90:  89ff006c 59524f54 80ff002f 40240e14  
3ffffea0:  00000000 3fff0308 3ffffed0 4022fab2  
3ffffeb0:  3fff2134 000c000f 09fe006d 4023f648  
3ffffec0:  00000000 3fff0308 3fff02c8 4022fc7a  
3ffffed0:  0000002f 00000019 81ff15f8 4023fcc8  
3ffffee0:  00000000 3fff0308 3fff02c8 401000e1  
3ffffef0:  00544500 3fff0308 3fff02c8 40230b3a  
3fffff00:  00000000 80000000 4bc6a700 00000000  
3fffff10:  8023f63c 3fff6000 0000007f 80005054  
3fffff20:  3fff02ec 3fff0308 00000001 40201974  
3fffff30:  00794235 0000000a 3fff15f8 4023fcc8  
3fffff40:  3fffdad0 00000000 3fff02c8 3fff1728  
3fffff50:  00000001 3fff02ec 3fff02c8 40231f74  
3fffff60:  4024d0b0 00000000 00001388 4022e934  
3fffff70:  00000000 3fff5b5c 80ff16e8 40203248  
3fffff80:  3fffdad0 00000000 3fff16e8 40232038  
3fffff90:  3fffdad0 00000000 3fff16e8 40232063  
3fffffa0:  feefeffe feefeffe feefeffe 402421a0  
3fffffb0:  feefeffe feefeffe 3ffe97a4 40101665  
<<<stack<<<

 ets Jan  8 2013,rst cause:2, boot mode:(3,6)

Decoding 28 results
0x40245e0d: spiffs_hydro_read at D:\SETUP\Programas\Desenvolvimento\ArduinoCoreTest\portable\packages\esp8266\hardware\esp8266\2.6.2\cores\esp8266\spiffs/spiffs_hydrogen.cpp line 418
:  (inlined by) SPIFFS_read at D:\SETUP\Programas\Desenvolvimento\ArduinoCoreTest\portable\packages\esp8266\hardware\esp8266\2.6.2\cores\esp8266\spiffs/spiffs_hydrogen.cpp line 429
0x4028f974: chip_v6_unset_chanfreq at ?? line ?
0x402427c6: raise_exception at D:\SETUP\Programas\Desenvolvimento\ArduinoCoreTest\portable\packages\esp8266\hardware\esp8266\2.6.2\cores\esp8266/core_esp8266_postmortem.cpp line 245
0x4024280e: __assert_func at ?? line ?
0x40234f6a: BufferedStreamDataSource ::get_buffer(unsigned int) at D:\SETUP\Programas\Desenvolvimento\ArduinoCoreTest\portable\packages\esp8266\hardware\esp8266\2.6.2\libraries\ESP8266WiFi\src/include/DataSource.h line 95
0x40235145: ClientContext::_write_some() at D:\SETUP\Programas\Desenvolvimento\ArduinoCoreTest\portable\packages\esp8266\hardware\esp8266\2.6.2\libraries\ESP8266WiFi\src/include/ClientContext.h line 495
0x40235206: ClientContext::_write_from_source(DataSource*) at D:\SETUP\Programas\Desenvolvimento\ArduinoCoreTest\portable\packages\esp8266\hardware\esp8266\2.6.2\libraries\ESP8266WiFi\src/include/ClientContext.h line 450
0x40235740: WiFiClient::write(Stream&) at D:\SETUP\Programas\Desenvolvimento\ArduinoCoreTest\portable\packages\esp8266\hardware\esp8266\2.6.2\libraries\ESP8266WiFi\src/include/ClientContext.h line 372
:  (inlined by) WiFiClient::write(Stream&) at D:\SETUP\Programas\Desenvolvimento\ArduinoCoreTest\portable\packages\esp8266\hardware\esp8266\2.6.2\libraries\ESP8266WiFi\src/WiFiClient.cpp line 242
0x4022f5e1: unsigned int esp8266webserver::ESP8266WebServerTemplate ::streamFile (fs::File&, String const&) at D:\SETUP\Programas\Desenvolvimento\ArduinoCoreTest\portable\packages\esp8266\hardware\esp8266\2.6.2\libraries\ESP8266WebServer\src/ESP8266WebServer.h line 157 (discriminator 1)
0x40240a51: String::invalidate() at D:\SETUP\Programas\Desenvolvimento\ArduinoCoreTest\portable\packages\esp8266\hardware\esp8266\2.6.2\cores\esp8266/WString.cpp line 140
0x4022f916: ServerFileReadFS(String) at C:\Users\Laercio Mendon�a\AppData\Local\Temp\arduino_build_763520\sketch/Server.h line 121
0x40240e14: String::operator=(String const&) at D:\SETUP\Programas\Desenvolvimento\ArduinoCoreTest\portable\packages\esp8266\hardware\esp8266\2.6.2\cores\esp8266/WString.cpp line 262
0x4022fab2: ServerLoadFile(String) at C:\Users\Laercio Mendon�a\AppData\Local\Temp\arduino_build_763520\sketch/Server.h line 134
0x4023f648: HardwareSerial::write(unsigned char const*, unsigned int) at D:\SETUP\Programas\Desenvolvimento\ArduinoCoreTest\portable\packages\esp8266\hardware\esp8266\2.6.2\cores\esp8266/HardwareSerial.h line 165
0x4022fc7a: operator() at C:\Users\Laercio Mendon�a\AppData\Local\Temp\arduino_build_763520\sketch/Server.h line 402
:  (inlined by) _M_invoke at d:\setup\programas\desenvolvimento\arduinocoretest\portable\packages\esp8266\tools\xtensa-lx106-elf-gcc\2.5.0-4-b40a506\xtensa-lx106-elf\include\c++\4.8.2/functional line 2071
0x4023fcc8: Print::println() at D:\SETUP\Programas\Desenvolvimento\ArduinoCoreTest\portable\packages\esp8266\hardware\esp8266\2.6.2\cores\esp8266/Print.cpp line 178
0x401000e1: std::function ::operator()() const at d:\setup\programas\desenvolvimento\arduinocoretest\portable\packages\esp8266\tools\xtensa-lx106-elf-gcc\2.5.0-4-b40a506\xtensa-lx106-elf\include\c++\4.8.2/functional line 2465
0x40230b3a: esp8266webserver::ESP8266WebServerTemplate ::_handleRequest() at D:\SETUP\Programas\Desenvolvimento\ArduinoCoreTest\portable\packages\esp8266\hardware\esp8266\2.6.2\libraries\ESP8266WebServer\src/ESP8266WebServer-impl.h line 673
0x40201974: esp8266webserver::FunctionRequestHandler ::canHandle(HTTPMethod, String) at D:\SETUP\Programas\Desenvolvimento\ArduinoCoreTest\portable\packages\esp8266\hardware\esp8266\2.6.2\libraries\ESP8266WebServer\src/detail/RequestHandlersImpl.h line 23
0x4023fcc8: Print::println() at D:\SETUP\Programas\Desenvolvimento\ArduinoCoreTest\portable\packages\esp8266\hardware\esp8266\2.6.2\cores\esp8266/Print.cpp line 178
0x40231f74: esp8266webserver::ESP8266WebServerTemplate ::handleClient() at D:\SETUP\Programas\Desenvolvimento\ArduinoCoreTest\portable\packages\esp8266\hardware\esp8266\2.6.2\libraries\ESP8266WebServer\src/ESP8266WebServer-impl.h line 335
0x4024d0b0: spiffs_cache_page_get_by_fd at ?? line ?
0x4022e934: SerialDebugMonitor() at C:\Users\Laercio Mendon�a\AppData\Local\Temp\arduino_build_763520\sketch/Serial.h line 20 (discriminator 1)
0x40203248: RTCMonitor() at C:\Users\Laercio Mendon�a\AppData\Local\Temp\arduino_build_763520\sketch/RTC.h line 227
0x40232038: Waiting at C:\Users\Laercio Mendon�a\AppData\Local\Temp\arduino_build_763520\sketch/Server.h line 410
:  (inlined by) ServerMonitor() at C:\Users\Laercio Mendon�a\AppData\Local\Temp\arduino_build_763520\sketch/Server.h line 411
0x40232063: loop at D:\Save\Aquarino\Projects\Aquarino-ControllerBeta/Aquarino-ControllerBeta.ino line 90
0x402421a0: loop_wrapper() at D:\SETUP\Programas\Desenvolvimento\ArduinoCoreTest\portable\packages\esp8266\hardware\esp8266\2.6.2\cores\esp8266/core_esp8266_main.cpp line 180
0x40101665: cont_wrapper at D:\SETUP\Programas\Desenvolvimento\ArduinoCoreTest\portable\packages\esp8266\hardware\esp8266\2.6.2\cores\esp8266/cont.S line 81

[d-a-v]

This is not an MCVE for us to be able to reproduce.
Can you also enable all debug options (including OOM)
or print getFreeHeap() around where your issue happens ?

[laercionit]

request: GET / HTTP/1.1
method: GET url: / search:
headerName: Host
headerValue: 192.168.1.110
headerName: Connection
headerValue: keep-alive
headerName: Upgrade-Insecure-Requests
headerValue: 1
headerName: User-Agent
headerValue: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36
headerName: Accept
headerValue: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,/;q=0.8,application/signed-exchange;v=b3
headerName: Accept-Encoding
headerValue: gzip, deflate
headerName: Accept-Language
headerValue: pt-BR,pt;q=0.9,en-US;q=0.8,en;q=0.7
:c0 1, 436
args:
args count: 0
args:
args count: 0
Request: /
Arguments:
final list of key/value pairs:
request handler not found
24248 HEAP FREE
ServerFileReadFS: Load /home.htm.gz
:wr 193 0
:wrc 193 193 0
:wr 2154 0
SPIFFS_read rc=-10014

User exception (panic/abort/assert)
Panic src/include/DataSource.h:93 const uint8_t* BufferedStreamDataSource::get_buffer(size_t) [with TStream = Stream; uint8_t = unsigned char; size_t: Assertion 'cb == stream_rem' failed.

[d-a-v]

SPIFFS_read rc=-10014

That seems to be the issue.
SPIFFS reports a file size,
Datasource tries to read that exact size

Arduino/libraries/ESP8266WiFi/src/include/DataSource.h

Lines 92 to 93 in 1d05283

	const size_t cb = _stream.readBytes(reinterpret_cast<char*>(_buffer.get() + stream_read), stream_rem);
	assert(cb == stream_rem);

but it is a short read. The following assert kills the esp.

The API is such that this function get_buffer() is not supposed to read anything shorter than the requested size.

@laercionit Can you verify your file is really unreadable ?

Anyway, this has to be fixed.
Marking it for v3.0.0

[laercionit]

OK, I just changed the code:

    if (server.streamFile (FilesFS, contentType) == FilesFS.size ()) {

for

server.streamFile (FilesFS, contentType);

and it worked correctly.

[laercionit]

Taking advantage of the subject, what are these new options for exeptions? What behavior for each of them?

[d-a-v]

First: in case of oom, new will return nullptr
Second: in case of oom, new will call abort()
Third: in case of oom, a real c++ exception is raised and is catchable (try..catch)

[array81]

I think I have a similar problem with 2.7.0. My webserver have a html page with same JS and CSS files.
When I try to load the page I get a similar problem.
Can you get the right code to read file from SPIFFS and send it to client and avoid this problem?

[MagicBugsBunny]

Also have the Problem when the web page load a CSV file from sd Card.

[schimkat]

Just upgraded to 2.7.1 .. and got the problem. It worked fine with the older version - but I cannot recall which version. Any workarounds out there? :-)

[d-a-v]

Can you have a look to this example which is able to stream files from FSes ?
https://github.com/esp8266/Arduino/tree/master/libraries/ESP8266WebServer/examples/FSBrowser
Don't forget to enable OOM debug option.

[MagicBugsBunny]

I've tested the example and it Looks ok.
Then i replace the files on sd Card with my "web page" files.
Same issue. As it load the CSV file the exception from post 1 appear.