UART splits packets on every 0xFF byte.

[hitecSmartHome]

Board

ESP32-Wrover-E

Device Description

• PSRAM
• FLASH
• SPI

Hardware Configuration

Version

latest master (checkout manually)

IDE Name

VsCode

Operating System

Windows 10

Flash frequency

80

PSRAM enabled

yes

Upload speed

115200

Description

When the ESP32 receives a frame which has 0xFF in it, it will break the packet into multiple packets.
We measured with oscilloscope and there is no byte timeout, the bytes are continous.
So a single continous packet which has 3pcs 0xFF byte in it, it will be split into 3 different packets.

Sketch

#define MBUS_BAUD 115200
#define MBUS_RX 35
#define MBUS_TX 32
#define MBUS_RTS 33

#define MBUS_RX_TIMEOUT 1 // Can't set it higher if UART_MODE_RS485_HALF_DUPLEX
#define MAX_MBUS_DATA_LENGTH 256
#define MAX_RX_BUFFER_SIZE 256

void Modbus::init() {
    Serial1.setRxBufferSize(MAX_RX_BUFFER_SIZE);
    Serial1.setTxBufferSize(MAX_MBUS_DATA_LENGTH);
    Serial1.begin(MBUS_BAUD, SERIAL_8N1, MBUS_RX, MBUS_TX);
    Serial1.setPins(-1, -1, -1, MBUS_RTS);
    Serial1.setMode(UART_MODE_RS485_HALF_DUPLEX);
    Serial1.setRxTimeout(MBUS_RX_TIMEOUT);

    Serial1.onReceive(
        std::bind(&Modbus::handlePacket, this),
        PACKET_TRIGGER_ONLY_ON_TIMEOUT
    );
    Serial1.onReceiveError(
        std::bind(&Modbus::handleReceiveError, this, std::placeholders::_1)
    );
}

void Modbus::handlePacket() {
    int available = Serial1.available();

    uint8_t rawPacket[available] = {0};
    int read = Serial1.readBytes(rawPacket, available);

    utils.printRawPacket("UART packet: ", rawPacket, read);
}

void Utils::printRawPacket(const char* title, uint8_t* data, int length) {
    printf("%s: ", title);
    for (int i = 0; i < length; i++) {
        printf("0x%02x ", data[i]);
    }
    printf("\n");
}

Expected single packet:

0x04 0x03 0x40 0xff 0xff 0xff 0xff 0xff 0xff 0xff 0x00 0x00 0x00 0x1b 0x00 0x00 0x00 0x0b 0x00 0x01 0x00 0x1b 0x00 0x00 0x00 0x35 0x00 0x00 0x00 0x1c 0x00 0x00 0x00 0x1f 0x00 0x00 0x00 0x1f 0x00 0x00 0x00 0x27 0x00 0x00 0x00 0x25 0x00 0x00 0x00 0x83 0x00 0x00 0x00 0x58 0x00 0x00 0x94 0x20 0x00 0x04 0xff 0x3d 0xff 0xff 0xeb 0x35

Received packets instead.

0x04 0x03 0x40

0xff 0xff 0xff

0xff 0xff 0xff

0xff 0x00 0x00 0x00 0x1b 0x00 0x00 0x00 0x0b 0x00 0x01 0x00 0x1b 0x00 0x00 0x00 0x35 0x00 0x00 0x00 0x1c 0x00 0x00 0x00 0x1f 0x00 0x00 0x00 0x1f 0x00 0x00 0x00 0x27 0x00 0x00 0x00 0x25 0x00 0x00 0x00 0x83 0x00 0x00 0x00 0x58 0x00 0x00 0x94 0x20 0x00 0x04

0xff 0x3d 0xff 0xff 0xeb 0x35

I have checked existing issues, online documentation and the Troubleshooting Guide

• I confirm I have checked existing issues, online documentation and Troubleshooting guide.

[SuGlider]

@hitecSmartHome - a couple questions:

• What is the value of PACKET_TRIGGER_ONLY_ON_TIMEOUT?
• Could you please set CORE_DEBUG_LEVEL=5 to get a Verbose Log output. (-DCORE_DEBUG_LEVEL=5 to the compiler options)
• The issue sketch uses some external classes. Please make it just Arduino plain API code in order to allow us to test it.
• Confirm what Arduino Core version is being used for this report.

[hitecSmartHome]

Sorry for not being more precise. I'm using ESP-IDF v5.3.0.240821 and I don't know the exact arduino version for now. Will have to check this out.

PACKET_TRIGGER_ONLY_ON_TIMEOUT is true. I'm waiting for packets and not bytes.

#define MBUS_BAUD 115200
#define MBUS_RX 35
#define MBUS_TX 32
#define MBUS_RTS 33

#define PACKET_TRIGGER_ONLY_ON_TIMEOUT 1
#define MBUS_RX_TIMEOUT 1 // Can't set it higher if UART_MODE_RS485_HALF_DUPLEX
#define MAX_MBUS_DATA_LENGTH 256
#define MAX_RX_BUFFER_SIZE 256

void setup(){
    Serial1.setRxBufferSize(MAX_RX_BUFFER_SIZE);
    Serial1.setTxBufferSize(MAX_MBUS_DATA_LENGTH);
    Serial1.begin(MBUS_BAUD, SERIAL_8N1, MBUS_RX, MBUS_TX);
    Serial1.setPins(-1, -1, -1, MBUS_RTS);
    Serial1.setMode(UART_MODE_RS485_HALF_DUPLEX);
    Serial1.setRxTimeout(MBUS_RX_TIMEOUT);

    Serial1.onReceive([](){
        int available = Serial1.available();

        uint8_t rawPacket[available] = {0};
        int read = Serial1.readBytes(rawPacket, available);

        printf("Raw uart data: ");
        for (int i = 0; i < length; i++) {
            printf("0x%02x ", data[i]);
        }
        printf("\n");

    },PACKET_TRIGGER_ONLY_ON_TIMEOUT);
}

If I throw any frame ( valid or invalid modbus frame it does not matter ) and if it has any 0xff it will split it into two packets. But it will split only when there are more bytes after a 0xff byte. So for example 0x00 0x02 0xff will not be a separate packet but 0x00 0x02 0xff 0x33 0x21 will be a separate packet like 0x00 0x02 and 0xff 0x33 0x21 We measured it with oscilloscope directly on the esp32's receiver pin and the data does not have any break or timeout. If you need I can take a picture of the oscilloscope's display.

[SuGlider]

@hitecSmartHome - I have modified the original Sketch and I am able to reproduce the reported issue.

I have found out that when the UART is set using EVEN or ODD parity bit, it works correctly.
The issue seems related to the IDF UART driver of the ESP32 that for some reason gets a BREAK after a specific incoming sequence of bits.

[SuGlider]

It seems that 0xff + some other byte will trigger a BREAK. This seems to interfere with the IDF UART Driver and it causes the package to be broken into pieces when it is read using "onReceive()".

I'll try to figure out a way to create a work around for it..

[hitecSmartHome]

I did not see any correlation to the sequence of bits or bytes. I observed only this 0xFF break only. But thank you for the response, I have tried out some configuration options but nothing helps so far. Thank you very much for your time!

[SuGlider]

@hitecSmartHome - Not sure if this information may help within your project, but I also have found out that this 0xFF issue only happens with the ESP32. It doesn't happen with the ESP32-S2, S3, C3 and C6.

Therefore, I think that it may be an ESP32 internal hardware design problem that has been solved latter.
Using another SoC may solve this specific issue in a product.

[SuGlider]

Lastly, I also verified that when MBUS_RX_TIMEOUT is 2 or higher, the issue doesn't happen.
Therefore, it seem linked to the ESP32 SoC and the settings used. It won't be fixed with software change.

It seems that there is no way to fix it, unless using ODD/EVEN parity or a RX_TIMEOUT higher than 1.

[hitecSmartHome]

@hitecSmartHome - Not sure if this information may help within your project, but I also have found out that this 0xFF issue only happens with the ESP32. It doesn't happen with the ESP32-S2, S3, C3 and C6.

Therefore, I think that it may be an ESP32 internal hardware design problem that has been solved latter. Using another SoC may solve this specific issue in a product.

That is bad news for us for sure.

It seems that there is no way to fix it, unless using ODD/EVEN parity or a RX_TIMEOUT higher than 1.

How can we set RX_TIMEOUT higher than 1 with UART_MODE_RS485_HALF_DUPLEX mode?

[SuGlider]

How can we set RX_TIMEOUT higher than 1 with UART_MODE_RS485_HALF_DUPLEX mode?

E (32) uart: tout_thresh = 2 > maximum value = 1
[    35][W][HardwareSerial.cpp:196] onReceive(): OnReceive is set to Timeout only, thus FIFO Full is now 120 bytes.

Using #define MBUS_RX_TIMEOUT 2
First line is IDF error message. Seconds is Arduino Core warning message.

=======
But it seems that if I set the BaudRate to 500000, it goes fine.
Could you please check if this is correct on your end?

[hitecSmartHome]

I have tried to set it to 5 but I got an error message like that and it did not have any effect. I can try that again tomorrow.