LittleFS mklittlefs error (crashes in WebServer w/small files)

[mbm60]

Basic Infos

• This issue complies with the issue POLICY doc.
• I have read the documentation at readthedocs and the issue is not addressed there.
• I have tested that the issue is present in current master branch (aka latest git).
• I have searched the issue tracker for a similar issue.
• If there is a stack dump, I have decoded it.
• I have filled out all fields below.

Platform

• Hardware: [ESP-12]
• Core Version: [2.6.0-dev]
• Development Env: [Arduino IDE]
• Operating System: [Windows]

Settings in IDE

• Module: [Generic ESP8266 Module]
• Flash Mode: [qio]
• Flash Size: [4MB/1MB]
• lwip Variant: [v2 Lower Memory]
• Reset Method: [ck]
• Flash Frequency: [40Mhz]
• CPU Frequency: [80Mhz]
• Upload Using: [SERIAL]
• Upload Speed: [115200]
• Erase Flash: [ All Flash Contents ]
• Espressif Firmware: [ nonos-sdk 2.2.1]

Problem Description

I uploaded example FSBrowser with LittleFS filesystem. when i want to open device IP in browser, The device will be reset repeatedly until the browser displays the "The connection was reset" page.
But with SPIFFS filesystem, everything works fine.

Debug Messages

FS File: Station.html, size: 18.52KB
FS File: favicon.ico, size: 1.12KB
FS File: header.html, size: 2.39KB
FS File: home_lan.html, size: 22.89KB
FS File: index.htm, size: 657B
FS File: logo.gif, size: 1.94KB
FS File: logo.jpg, size: 5.50KB
FS File: navigation-basic.html, size: 5.00KB
FS File: style.css, size: 4.60KB
FS File: tc2func.js, size: 8.12KB

Connecting to G5_MBM
scandone
state: 0 -> 2 (b0)
state: 2 -> 3 (0)
state: 3 -> 5 (10)
add 0
aid 1
cnt 
state: 5 -> 2 (3c0)
rm 0
..reconnect
state: 2 -> 0 (0)
scandone
state: 0 -> 2 (b0)
state: 2 -> 3 (0)
state: 3 -> 5 (10)
add 0
aid 1
cnt 

connected with G5_MBM, channel 6
dhcp client start...
ip:192.168.43.106,mask:255.255.255.0,gw:192.168.43.1
.
Connected! IP address: 192.168.43.106
Open http://esp8266fs.local/edit to see the file browser
HTTP server started
handleFileRead: /

Panic C:\Users\MBM\Documents\Arduino\hardware\esp8266com\esp8266\libraries\ESP8266WiFi\src/include/DataSource.h:93 const uint8_t* BufferedStreamDataSource<TSt: Assertion 'cb == stream_rem' failed.

>>>stack>>>

ctx: cont
sp: 3ffffc40 end: 3fffffc0 offset: 01b0
3ffffdf0:  00000000 00000000 401004e4 40206b9e  
3ffffe00:  00000000 3ffee76c 3ffffe48 000003dc  
3ffffe10:  00000000 00000291 3ffefbcc 40206dc1  
3ffffe20:  3ffefbcc 4021bb0c 0000001c 00000291  
3ffffe30:  3ffefbcc 00000001 3ffefbcc 40206e82  
3ffffe40:  00000001 3ffffe70 00000000 4020735c  
3ffffe50:  03000001 3ffffe94 00000291 3ffe87d3  
3ffffe60:  00000001 3ffe84e8 00000291 4020545c  
3ffffe70:  40213ec0 00000000 000003e8 402105af  
3ffffe80:  3fff017c 3fff01b4 00000000 00000000  
3ffffe90:  3ffee8dc 00000000 00000000 0921006d  
3ffffea0:  3ffeffdc 000d000f 3ffffee0 74786574  
3ffffeb0:  6d74682f 89fe006c 3ffffee0 402107e4  
3ffffec0:  3ffee7b0 00000001 402114fc 3ffeea38  
3ffffed0:  00000000 3ffee7b0 3ffee76c 40205502  
3ffffee0:  646e692f 682e7865 8a006d74 00000000  
3ffffef0:  00000000 00000000 3ffee76c 401000a9  
3fffff00:  00000000 00000000 01db22d0 402083e4  
3fffff10:  00000000 03000000 00000000 00000000  
3fffff20:  0a005054 402161f8 401004e4 000006d8  
3fffff30:  3ffee7b0 00000001 3ffee770 40212f30  
3fffff40:  0007ea4c 00000000 00000000 00000019  
3fffff50:  00000000 3ffefbcc 3ffee76c 3ffeea38  
3fffff60:  00000001 3ffee794 3ffee76c 402085d4  
3fffff70:  40213b58 00000000 00001388 4020b7ca  
3fffff80:  00000000 3ffefbcc 3ffeea08 40209dfe  
3fffff90:  3fffdad0 00000000 3ffeea08 40205cc0  
3fffffa0:  3fffdad0 00000000 3ffeea08 4021151c  
3fffffb0:  feefeffe feefeffe 3ffe8518 40100369  
<<<stack<<<

 ets Jan  8 2013,rst cause:2, boot mode:(3,6)

load 0x4010f000, len 1384, room 16 
tail 8
chksum 0x2d
csum 0x2d
vd93c7e06
~ld

[earlephilhower]

Please post the decoded stack dump, too. The raw hex dump isn't useful outside of your own single compile.

[mbm60]

Decoding stack results

0x401004e4: millis() at C:\Users\MBM\Documents\Arduino\hardware\esp8266com\esp8266\cores\esp8266\core_esp8266_wiring.cpp line 186
0x40206b9e: BufferedStreamDataSource ::get_buffer(unsigned int) at C:\Users\MBM\Documents\Arduino\hardware\esp8266com\esp8266\libraries\ESP8266WiFi\src/include/DataSource.h line 95
0x40206dc1: ClientContext::_write_some() at C:\Users\MBM\Documents\Arduino\hardware\esp8266com\esp8266\libraries\ESP8266WiFi\src/include/ClientContext.h line 486
0x4021bb0c: operator new(unsigned int) at ../../../../../dl/gcc-xtensa/libstdc++-v3/libsupc++/new_op.cc line 52
0x40206e82: ClientContext::_write_from_source(DataSource*) at C:\Users\MBM\Documents\Arduino\hardware\esp8266com\esp8266\libraries\ESP8266WiFi\src/include/ClientContext.h line 445
0x4020735c: WiFiClient::write(Stream&) at C:\Users\MBM\Documents\Arduino\hardware\esp8266com\esp8266\libraries\ESP8266WiFi\src/include/ClientContext.h line 369
0x4020545c: handleFileRead(String) at C:\Users\MBM\Desktop\FSBrowser2/FSBrowser2.ino line 107
0x402105af: String::reserve(unsigned int) at C:\Users\MBM\Documents\Arduino\hardware\esp8266com\esp8266\cores\esp8266\WString.cpp line 147
0x402107e4: String::String(String const&) at C:\Users\MBM\Documents\Arduino\hardware\esp8266com\esp8266\cores\esp8266\WString.cpp line 41
0x40205502: std::_Function_handler ::_M_invoke(const std::_Any_data &) at C:\Users\MBM\Desktop\FSBrowser2/FSBrowser2.ino line 271
0x401000a9: std::function ::operator()() const at c:\users\mbm\appdata\local\arduino15\packages\esp8266\tools\xtensa-lx106-elf-gcc\2.5.0-3-20ed2b9\xtensa-lx106-elf\include\c++\4.8.2/functional line 2465
0x402083e4: ESP8266WebServer::_handleRequest() at C:\Users\MBM\Documents\Arduino\hardware\esp8266com\esp8266\libraries\ESP8266WebServer\src\ESP8266WebServer.cpp line 625
0x401004e4: millis() at C:\Users\MBM\Documents\Arduino\hardware\esp8266com\esp8266\cores\esp8266\core_esp8266_wiring.cpp line 186
0x40212f30: WiFiServer::available(unsigned char*) at C:\Users\MBM\Documents\Arduino\hardware\esp8266com\esp8266\libraries\ESP8266WiFi\src\WiFiServer.cpp line 116
0x402085d4: ESP8266WebServer::handleClient() at C:\Users\MBM\Documents\Arduino\hardware\esp8266com\esp8266\libraries\ESP8266WebServer\src\ESP8266WebServer.cpp line 318
0x4020b7ca: esp8266::MDNSImplementation::MDNSResponder::_process(bool) at C:\Users\MBM\Documents\Arduino\hardware\esp8266com\esp8266\libraries\ESP8266mDNS\src\LEAmDNS_Control.cpp line 84
0x40209dfe: esp8266::MDNSImplementation::MDNSResponder::update() at C:\Users\MBM\Documents\Arduino\hardware\esp8266com\esp8266\libraries\ESP8266mDNS\src\LEAmDNS.cpp line 1126
0x40205cc0: loop() at C:\Users\MBM\Desktop\FSBrowser2/FSBrowser2.ino line 293
0x4021151c: loop_wrapper() at C:\Users\MBM\Documents\Arduino\hardware\esp8266com\esp8266\cores\esp8266\core_esp8266_main.cpp line 132

[mbm60]

@earlephilhower I tested a lot of things and realized that, when the program wants to open a file with size less than 1KB (1024 bytes), the device will reset.

[earlephilhower]

Good debug.

It may just be an OOM issue with the example code, in which case there's not much that can be done.

Let me try and make a small test and see what's happening.

[earlephilhower]

LittleFS is fine by my testing. You're just running out of memory in the WebServer which is actually pretty common. A simple test like this runs forever and shows no problems w/<1KB files.

#include <LittleFS.h>

const char *str="Now is the time for all good men to come to the aid of their country.\nYa te vas para no volver.\nRien n'empeche.\n";

void setup() {
  Serial.begin(115200);
  LittleFS.begin();
  for (int i=0; i<10; i++) {
    char fname[32];
    sprintf(fname, "file%02d.txt", i);
    Serial.printf("Writing %s\n", fname);
    File f = LittleFS.open(fname, "w");
    f.write(fname, strlen(fname));
    f.write(str, strlen(str));
    f.close();
  }
}

void loop() {
  Serial.printf("Heap Free: %d\n", ESP.getFreeHeap());
  for (int i=0; i<10; i++) {
    char fname[32];
    char buff[512];
    sprintf(fname, "file%02d.txt", i);
    Serial.printf("Reading %s\n", fname);
    File f = LittleFS.open(fname, "r");
    int len = f.read((uint8_t*)buff, 512);
    buff[len] = 0;
    Serial.printf("Free: %d, Read: %d bytes, '%s'\n", ESP.getFreeHeap(), len, buff);
    f.close();
  }
}

[mbm60]

In the case you said, it is your right.
But this happens only in the examples of the webserver library.
For example, in the FSBrowser file, inside the "handleFileRead" function after calling "server.streamFile" on line 106 when the ( index.htm < 1KB ) reset occurs.

[mbm60]

I think the problem is from the "Arduino ESP8266 LittleFS Filesystem Uploader" plugin.
Because, when i write string like:

"<!DOCTYPE html>
<html>
<head>
  <meta http-equiv="Content-type" content="text/html; charset=utf-8">
  <title>ESP Monitor</title>
</head>
<body id="index" style="margin:0; padding:0;">
    <input type="button" id="start-button" value="Start"/>
    <input type="button" id="stop-button" value="Stop"/>
  </div>
</body>
</html>"

with file.write and read that with file.read, the result is correct.
But when write files in folder data with "LittleFS Filesystem Uploader", the result with file.read is:

<!DOCTYPE html>
<html>
<head>
  <meta http-equiv="Content-typ⸮e" content="text/html; charset=utf-8">
  <title>ESP Monitor</ti⸮tle>
</head>
<body id="index" style="margin:0; padding:0;">
 ⸮   <input type="button" id="start-button" value="Start"/>
    <⸮input type="button" id="stop-button" value="Stop"/>
  </div>
<⸮/body>
</html>

This only happens when the file size is less than 1024 bytes.

[earlephilhower]

That's very interesting. There might be a difference in version/options in mklittlefs.exe and the current core. I thought it was at a stable version, but will look at it and see if there's anything amiss...

[earlephilhower]

Reproduced the failure and am able to fix it by updating the mklittlefs.exe. There'll be a PR soon with the update.