Writing files with SPIFFS no longer working for ESP-01

[funtastic99]

Basic Infos

SPIFFS no longer works on a ESP-01.

Hardware

Hardware: ESP-01
Core Version: 2.3.0-rc2
also tried with stable

I reproduced this problem on several different ESP-01. So seems not to be hardware related

Description

Problem description
SPIFFS worked for a long time but all of a sudden it stoped working on a ESP-01.
Whenever i write a file its not existing or empty after next boot.
Using the same sketch on an Wemos D1 mini its works without problems. Actually im only facing this problem on a ESP-01.

Settings in IDE

Module: Generic ESP8266 Module
Flash Size: 1MB (64k SPIFFS), also tried with 512k (64k SPIFFS)
CPU Frequency: 80Mhz
Flash Mode: DIO
Flash Frequency: 40Mhz
Upload Using: SERIAL
Reset Method: ck

Sketch

#include <FS.h>

void setup()
{
Serial.begin(115200);

if (!SPIFFS.begin()) Serial.println("error while mounting filesystem!");

readFile();
writeFile();
readFile();

Serial.println("done");
}

void readFile()
{
Serial.println("reading");
File f = SPIFFS.open("/myFile.txt", "r");
if (!f) Serial.println("file not available");
else if (f.available()<=0) Serial.println("file exists but available <0");
else
{
String ssidString = f.readStringUntil('#');
Serial.print("read from file: ");
Serial.println(ssidString);
}
f.close();
}

void writeFile()
{
Serial.println("writing");
File f = SPIFFS.open("/myFile.txt", "w");
if (!f) Serial.println("File creation failed");
else
{
f.print("networkConfig");
f.print("#");
f.flush();
f.close();
}
}

void loop()
{
}

Debug Messages

messages here

• Output of first run:
  reading
  file not available
  writing
  reading
  read from file: <--- here we see the problem already
  done

• Output of second run:
  eading
  file not available
  writing
  reading
  read from file:
  done

• Output of third run:
  reading
  read from file:
  writing
  File creation failed <---- check this!
  reading
  read from file:
  done

Any ideas?
thank you!

[funtastic99]

I found out some additional information that might be helpful.
The working ESP-01 report as flash_id:
Manufacturer: e0
Device: 4014

and the new ones that are not working:
Manufacturer: 85
Device: 6014

May the new flash chip is not supported by the arduino esp plugin yet ?

[ondabeach]

I'm having the same problem.

Out of the first batch of 20 where the problem first showed up only 7 of them worked properly. All functions except for SPIFFS seem fine.

Out of the next batch of 20, 9 worked properly.

I just received another 2 batches of 20. One batch from a different seller on eBay and the other batch from a seller on AliExpress.

Out of the new batch from eBay only 2 work properly.

All 20 of the new batch from Aliexpress are faulty.

I checked the Chip ID's and I can confirm that Flash Chips with a manufacturer ID of 85 and Device ID of 6014 do not work. I can also confirm that the 2 that did work have the manufacturer ID of E0 and Chip ID of 4014.

I had a look at the flash chips with a magnifying glass and found the faulty ones were all branded FUYA and the 2 that have working SPIFFS are branded BergMicro.

I sure hope someone has a solution for this as it's a major problem for anyone who needs to store stuff like settings. EEPROM is an alternative but it's so clunky compared to SPIFFS and I really don't want to have to recode my sketches.

[funtastic99]

Its definitely a problem with writing SPIFFS. Reading works because i uploaded a flash image of a functional device with SPIFFS data present. But for reading the flash as anyway only mapped to memory while for writing a correct initializiation and timing is needed. I wonder why the booter itself has no problems writing the flash but SPIFFS has. I would have expected that finally the same functions are used to write the flash but it seems that the spiffs implementation is doing something strang and chip dependend.

[Pablo2048]

So it seems like problem in SPIFFS and/or write implementation. Right?

[funtastic99]

yes seems like

[ondabeach]

I found that the 'Upload Sketch Data' function of the IDE works. A sketch can then read the data but a sketch can't write to SPIFFS.

This will impact a lot of people, I've bought esp-01 modules witj these PUYA branded flash chips from 3 different suppliers now. Out of the 80 modules only 18 have worked as they had BergMicro flash chips onboard.

[weswitt]

I'm seeing a similar, but different problem. My sketch does not find any files in the fs after uploading the spiffs image. I're tried uploading with esptool and the Arduino IDE, but I get the same result. The processor is an ESP01 on a Sonoff device. I've tried both the 512k64k and 1m64k configurations and neither work.

Is the issue I'm seeing related to this report? Am I possibly doing something wrong? What is the best way to diagnose this? I've added some code to the setup function that lists all files and it finds none.

[ondabeach]

@weswitt is this a new board purchased recently?
Can you check the writing on the Flash chip, is it PUYA brand? Look for a number like P25Q80?.
Or you can load and run the following code to get the chip data. You're looking for the manufacturer ID of 85. If you have that flash chip then yes, you have the same problem as us.

//*************************************************
#include "FS.h"

void setup() {
Serial.begin(9600);

SPIFFS.begin();
FSInfo fs_info;
SPIFFS.info(fs_info);

float fileTotalKB = (float)fs_info.totalBytes / 1024.0;
float fileUsedKB = (float)fs_info.usedBytes / 1024.0;

float flashChipSize = (float)ESP.getFlashChipSize() / 1024.0 / 1024.0;
float realFlashChipSize = (float)ESP.getFlashChipRealSize() / 1024.0 / 1024.0;
float flashFreq = (float)ESP.getFlashChipSpeed() / 1000.0 / 1000.0;
FlashMode_t ideMode = ESP.getFlashChipMode();

Serial.printf("\n#####################\n");

Serial.printf("__________________________\n\n");
Serial.println("Firmware: ");
Serial.printf(" Chip Id: %08X\n", ESP.getChipId());
Serial.print(" Core version: "); Serial.println(ESP.getCoreVersion());
Serial.print(" SDK version: "); Serial.println(ESP.getSdkVersion());
Serial.print(" Boot version: "); Serial.println(ESP.getBootVersion());
Serial.print(" Boot mode: "); Serial.println(ESP.getBootMode());

Serial.printf("__________________________\n\n");

Serial.println("Flash chip information: ");
Serial.printf(" Flash chip Id: %08X (for example: Id=001640E0 Manuf=E0, Device=4016 (swap bytes))\n", ESP.getFlashChipId());
Serial.printf(" Sketch thinks Flash RAM is size: "); Serial.print(flashChipSize); Serial.println(" MB");
Serial.print(" Actual size based on chip Id: "); Serial.print(realFlashChipSize); Serial.println(" MB ... given by (2^( "Device" - 1) / 8 / 1024");
Serial.print(" Flash frequency: "); Serial.print(flashFreq); Serial.println(" MHz");
Serial.printf(" Flash write mode: %s\n", (ideMode == FM_QIO ? "QIO" : ideMode == FM_QOUT ? "QOUT" : ideMode == FM_DIO ? "DIO" : ideMode == FM_DOUT ? "DOUT" : "UNKNOWN"));

Serial.printf("__________________________\n\n");

Serial.println("File system (SPIFFS): ");
Serial.print(" Total KB: "); Serial.print(fileTotalKB); Serial.println(" KB");
Serial.print(" Used KB: "); Serial.print(fileUsedKB); Serial.println(" KB");
Serial.printf(" Block size: %lu\n", fs_info.blockSize);
Serial.printf(" Page size: %lu\n", fs_info.pageSize);
Serial.printf(" Maximum open files: %lu\n", fs_info.maxOpenFiles);
Serial.printf(" Maximum path length: %lu\n\n", fs_info.maxPathLength);

Dir dir = SPIFFS.openDir("/");
Serial.println("SPIFFS directory {/} :");
while (dir.next()) {
Serial.print(" "); Serial.println(dir.fileName());
Serial.println(" "); Serial.println(dir.fileSize());
}

Serial.printf("__________________________\n\n");

Serial.printf("CPU frequency: %u MHz\n\n", ESP.getCpuFreqMHz());
Serial.print("#####################");

// open file for reading
File f = SPIFFS.open("/login.txt", "r");
if (!f) {
Serial.println("file open failed");
} Serial.println("====== Reading from SPIFFS file =======");
// write 10 strings to file
for (int i=1; i<=10; i++){
String s=f.readStringUntil('\n');
Serial.print(i);
Serial.print(":");
Serial.println(s);
}
}

void loop() {
delay(2);
}
//********************************************

[devyte]

Same issue as #3616 , tracking there.
Closing this one.

[weswitt]

@ondabeach thanks for the assistance. below is the diagnostic output from the sketch you provided:

#####################

---

Firmware:
Chip Id: 0042A910
Core version: 2_4_0
SDK version: 2.1.0(deb1901)
Boot version: 31
Boot mode: 1

---

Flash chip information:
Flash chip Id: 001440E0 (for example: Id=001640E0 Manuf=E0, Device=4016 (swap bytes))
Sketch thinks Flash RAM is size: 0.50 MB
Actual size based on chip Id: 1.00 MB ... given by (2^( "Device" - 1) / 8 / 1024
Flash frequency: 40.00 MHz
Flash write mode: DIO

---

File system (SPIFFS):
Total KB: 51.72 KB
Used KB: 0.00 KB
Block size: 4096
Page size: 256
Maximum open files: 5
Maximum path length: 32

SPIFFS directory {/} :

---

CPU frequency: 80 MHz

#####################file open failed
====== Reading from SPIFFS file =======
1:
2:
3:
4:
5:
6:
7:
8:
9:
10:

[ondabeach]

@weswitt The last sketch was just to get the flash chip data, the printing of 1~10 at the bottom was left in by mistake.

The chip ID retrieved shows that you have a flash chip that should work, it does for the OP and myself.
FYI, you have flash size set to 512k 64k but your board actually has 1MB. That isn't an issue as the actual size is bigger than your setting.

Try the following code, it should work on your board.

//**********************************************************
#include "FS.h"

String realSize = String(ESP.getFlashChipRealSize());
String ideSize = String(ESP.getFlashChipSize());
bool flashCorrectlyConfigured = realSize.equals(ideSize);

void setup() {
Serial.begin(9600);

if(flashCorrectlyConfigured){
SPIFFS.begin();
Serial.println("flash correctly configured, SPIFFS started, IDE size: " + ideSize + ", real size: " + realSize);
}
else Serial.println("flash incorrectly configured, SPIFFS cannot start, IDE size: " + ideSize + ", real size: " + realSize);

Serial.println("\nVery basic Spiffs example, writing 10 lines to SPIFFS filesystem, and then read them back");
SPIFFS.begin();

if (!SPIFFS.exists("/formatComplete.txt")) {
Serial.println("Please wait 30 secs for SPIFFS to be formatted");
SPIFFS.format();
//delay(30000);
Serial.println("Spiffs formatted");

File f = SPIFFS.open("/formatComplete.txt", "w");
if (!f) {
    Serial.println("file open failed");
} else {
    f.println("Format Complete");
}

} else {
Serial.println("SPIFFS is formatted. Moving along...");
}
}

void loop() {
// open file for writing
File f = SPIFFS.open("/f.txt", "w");
if (!f) {
Serial.println("file open failed");
}
Serial.println("====== Writing to SPIFFS file =========");
// write 10 strings to file
for (int i=1; i<=10; i++){
f.print("Millis() : ");
f.println(millis());
Serial.println(millis());
}

f.close();

// open file for reading
f = SPIFFS.open("/f.txt", "r");
if (!f) {
Serial.println("file open failed");
} Serial.println("====== Reading from SPIFFS file =======");
// write 10 strings to file
for (int i=1; i<=10; i++){
String s=f.readStringUntil('\n');
Serial.print(i);
Serial.print(":");
Serial.println(s);
}
// wait a few seconds before doing it all over again
delay(3000);

}
//*************************************************************************

[weswitt]

Thanks @ondabeach. Running that code does confirm that writing to and reading from SPIFFS work fine on my device.

But this brings me back to my problem. What I'm seeing is that when I flash a SPIFFS to the device and then run a sketch the sketch sees zero files.

This happens if I use the Arduino IDE or esptool. This is the esptool command line that I'm using:

esptool.exe -vv -cd ck -cb 115200 -cp com3 -ca 0xEB000 -cf test-spiffs.bin

Does the tool used to flash the SPIFFS need to be up todate with the SPIFFS in the core? Maybe the tool is too old and is not writing a format that the core runtime understands?

[ondabeach]

No problem @weswitt.

I'm not sure if esptool needs to.be updated. Much quicker and simpler to use the IDE as you're already in there rather than using command line methods.

We know your hardware is ok and that sketch can write and read SPIFFS so to look at your uploading files to SPIFFS issue:

No idea of your experience level, I'm just a hacker myself so we'll start with the basics.

Use "Show Sketch Folder" under "Sketch" menu.
Create a folder there called "data".
Create/copy your files there. (For testing just create a text file there called "Test.txt" with contents as follows:
Line 1
Line 2
Line 3
Line 4
Line 5
Line 6
Line 7
Line 8
Line 9
Line 10

Make sure you carriage return after the "10" or that line won't be read.
Select "Sketch Data Upload" from Tools menu. (Make sure serial monitor is not open and board is in upload mode)

Here's a tidied up version of the Get Flash Data/SPIFFS Read sketch.

//*************************************************************
#include "FS.h"

void setup() {
Serial.begin(9600);

SPIFFS.begin();
FSInfo fs_info;
SPIFFS.info(fs_info);

float fileTotalKB = (float)fs_info.totalBytes / 1024.0;
float fileUsedKB = (float)fs_info.usedBytes / 1024.0;

float flashChipSize = (float)ESP.getFlashChipSize() / 1024.0 / 1024.0;
float realFlashChipSize = (float)ESP.getFlashChipRealSize() / 1024.0 / 1024.0;
float flashFreq = (float)ESP.getFlashChipSpeed() / 1000.0 / 1000.0;
FlashMode_t ideMode = ESP.getFlashChipMode();

Serial.println("==========================================================");
Serial.println("Firmware: ");
Serial.printf(" Chip Id: %08X\n", ESP.getChipId());
Serial.print(" Core version: "); Serial.println(ESP.getCoreVersion());
Serial.print(" SDK version: "); Serial.println(ESP.getSdkVersion());
Serial.print(" Boot version: "); Serial.println(ESP.getBootVersion());
Serial.print(" Boot mode: "); Serial.println(ESP.getBootMode());

Serial.printf("__________________________\n\n");

Serial.println("Flash chip information: ");
Serial.printf(" Flash chip Id: %08X (for example: Id=001640E0 Manuf=E0, Device=4016 (swap bytes))\n", ESP.getFlashChipId());
Serial.printf(" Sketch thinks Flash RAM is size: "); Serial.print(flashChipSize); Serial.println(" MB");
Serial.print(" Actual size based on chip Id: "); Serial.print(realFlashChipSize); Serial.println(" MB ... given by (2^( "Device" - 1) / 8 / 1024");
Serial.print(" Flash frequency: "); Serial.print(flashFreq); Serial.println(" MHz");
Serial.printf(" Flash write mode: %s\n", (ideMode == FM_QIO ? "QIO" : ideMode == FM_QOUT ? "QOUT" : ideMode == FM_DIO ? "DIO" : ideMode == FM_DOUT ? "DOUT" : "UNKNOWN"));
Serial.printf(" CPU frequency: %u MHz\n\n", ESP.getCpuFreqMHz());
Serial.printf("__________________________\n\n");

Serial.println("File system (SPIFFS): ");
Serial.print(" Total KB: "); Serial.print(fileTotalKB); Serial.println(" KB");
Serial.print(" Used KB: "); Serial.print(fileUsedKB); Serial.println(" KB");
Serial.printf(" Block size: %lu\n", fs_info.blockSize);
Serial.printf(" Page size: %lu\n", fs_info.pageSize);
Serial.printf(" Maximum open files: %lu\n", fs_info.maxOpenFiles);
Serial.printf(" Maximum path length: %lu\n\n", fs_info.maxPathLength);
Serial.printf("__________________________\n\n");

Dir dir = SPIFFS.openDir("/");
Serial.println("SPIFFS directory {/} :");
while (dir.next()) {
Serial.print("File Name "); Serial.println(dir.fileName());
Serial.print("File Length "); Serial.println(dir.fileSize());
Serial.print("====== First 10 lines of file ("); Serial.print(dir.fileName()); Serial.println(") =======");
// open file for reading
File f = SPIFFS.open("/Test.txt", "r");
if (!f) {
Serial.println("file open failed");
}
else{
// read 10 strings from file
for (int i=1; i<=10; i++){
String s=f.readStringUntil('\n');
Serial.print(i);
Serial.print(" : ");
Serial.println(s);
}
}
Serial.println("==========================================================");
}

}

void loop() {
delay(3000);
}
//****************************************************************