HardwareSerial end() / begin() does not appear to cleanly reinitialize port

[rayozzie]

Hardware:

Board: Adafruit Huzzah32 Feather
Core Installation/update date: 19-Feb-2018
IDE name: Arduino 1.8.5
Flash Frequency: 80MHz
Upload Speed: 921600

Description:

I have been fortunate thus far that everything I've tried has functioned flawlessly, so thank you very much for that.

On the Adafruit module, I am using the UART (pins 16/17) on Serial1 via:
HardwareSerial Serial1(2);

I am initializing it to 9600bps, and all is good. It works well.

However, I actually have a GPIO-driven 4-port UART multiplexer connected to those RX/TX pins. I have used this mux design for years, and the general algorithm when switching is:

• Serial1.end() to deinit the driver
• digitalWrite to two GPIO pins to select the desired port
• Serial1.begin(newSpeed) to reinit the driver

At the current time, all devices connected to all ports just happen to be operating at 9600bps, so this switchover should be occurring in a flawless manner without any framing errors or any issues at all.

However, there appears to be a bug somewhere in the end()/begin() code path that is causing extremely bad behavior. The way that it manifests itself is that after the end/begin reinit,

a) a fragment of the tail end of the data that had been previously successfully transmitted prior to the end() is spuriously transmitted redundantly after the begin()

b) buffer pointers appear to be getting somewhat confused, because after the reinit certain sequences of bytes are transmitted out-of-order - i.e. the string:
{"req":"time"}
...is commonly transmitted as...
e"}{"req:"tim

c) received data is wrong or reordered in a way that I have not yet been able to determine.

After many hours of debugging, I finally tried simply removing the calls to Serial1.end() and Serial1.begin(), knowing that (for now) my devices are all operating at 9600bps. After doing this, I am having zero bugs or issues - which is why I'm filing this issue.

I regret that I am on my way to the airport and haven't had a chance to create a repro sketch for you, but I felt it best to file this while it is fresh in my mind rather than to wait two weeks. Perhaps whoever is familiar with the HardwareSerial driver will recognize the failure to fully clear and reinitialize the driver context on begin().

Good hunting, and thanks again.

[lbernstone]

I don't know anything about the internal workings of the esp-hal-uart, but for troubleshooting and perhaps a workaround, have you tried putting a flush in between the end and begin?

[rayozzie]

I did, and it caused incredibly unstable behavior. The flush call caused transmission of a large amount of garbage – consistent with the theory that the state of the buffer pointers is likely bad. From: lbernstone [mailto:notifications@github.com] Sent: Friday, February 23, 2018 11:39 AM To: espressif/arduino-esp32 Cc: Ray Ozzie; Author Subject: Re: [espressif/arduino-esp32] HardwareSerial end() / begin() does not appear to cleanly reinitialize port (#1145) I don't know anything about the internal workings of the esp-hal-uart, but for troubleshooting and perhaps a workaround, have you tried putting a flush in between the end and begin? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<#1145 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AAuhPVyc4k8nQfFnHA2WQ_0cyuH0KeYCks5tXumngaJpZM4SRBmf>.

[lbernstone]

Do you get the same behavior on HardwareSerial(1)? From the esp32 technical manual:

Three UART controllers share a 1024 x 8-bit RAM space. As illustrated in Figure 79, RAM is allocated in different
blocks. One block holds 128 x 8-bit data. Figure 79 illustrates the default RAM allocated to Tx_FIFO and
Rx_FIFO of the three UART controllers. Tx_FIFO of UARTn can be extended by setting UARTn_TX_SIZE, while
Rx_FIFO of UARTn can be extended by setting UARTn_RX_SIZE.
NOTICE: Extending the FIFO space of a UART controller may take up the FIFO space of another UART
controller.
If none of the UART controllers is active, setting UART_MEM_PD, UART1_MEM_PD, and UART2_MEM_PD can
prompt the RAM to enter low-power mode.
In UART0, bit UART_TXFIFO_RST and bit UART_RXFIFO_RST can be set to reset Tx_FIFO or Rx_FIFO,
respectively. In UART1, bit UART1_TXFIFO_RST and bit UART1_RXFIFO_RST can be set to reset Tx_FIFO or
Rx_FIFO, respectively.
Note:
UART2 doesn’t have any register to reset Tx_FIFO or Rx_FIFO, and the UART1_TXFIFO_RST and UART1_RXFIFO_RST
in UART1 may impact the functioning of UART2. Therefore, these 2 registers in UART1 should only be used when the
Tx_FIFO and Rx_FIFO in UART2 do not have any data.

[rayozzie]

Unfortunately, it’s not possible for me to try UART1 because the pins aren’t exposed on the Arduino Huzzah32. Not using it. I am using UART0 for output only because, in the Arduino IDE it’s wired to be used for the debug console. From: lbernstone <notifications@github.com> Reply-To: espressif/arduino-esp32 <reply@reply.github.com> Date: Friday, February 23, 2018 at 11:53 AM To: espressif/arduino-esp32 <arduino-esp32@noreply.github.com> Cc: Ray Ozzie <ray@ozzie.net>, Author <author@noreply.github.com> Subject: Re: [espressif/arduino-esp32] HardwareSerial end() / begin() does not appear to cleanly reinitialize port (#1145) Do you get the same behavior on HardwareSerial(1)? From the esp32 technical manual: Three UART controllers share a 1024 x 8-bit RAM space. As illustrated in Figure 79, RAM is allocated in different blocks. One block holds 128 x 8-bit data. Figure 79 illustrates the default RAM allocated to Tx_FIFO and Rx_FIFO of the three UART controllers. Tx_FIFO of UARTn can be extended by setting UARTn_TX_SIZE, while Rx_FIFO of UARTn can be extended by setting UARTn_RX_SIZE. NOTICE: Extending the FIFO space of a UART controller may take up the FIFO space of another UART controller. If none of the UART controllers is active, setting UART_MEM_PD, UART1_MEM_PD, and UART2_MEM_PD can prompt the RAM to enter low-power mode. In UART0, bit UART_TXFIFO_RST and bit UART_RXFIFO_RST can be set to reset Tx_FIFO or Rx_FIFO, respectively. In UART1, bit UART1_TXFIFO_RST and bit UART1_RXFIFO_RST can be set to reset Tx_FIFO or Rx_FIFO, respectively. Note: UART2 doesn’t have any register to reset Tx_FIFO or Rx_FIFO, and the UART1_TXFIFO_RST and UART1_RXFIFO_RST in UART1 may impact the functioning of UART2. Therefore, these 2 registers in UART1 should only be used when the Tx_FIFO and Rx_FIFO in UART2 do not have any data. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<#1145 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AAuhPVyyX0hcLiSq7xIV6qjKDk0hvrD4ks5tXu0UgaJpZM4SRBmf>.

[lbernstone]

You should be able to map that to whatever pins you like:

HardwareSerial Serial1(1);
Serial1.begin(9600, SERIAL_8N1, 16, 17);

[negativekelvin]

Arduino has not fixed uart fifo reset bug that was fixed in idf

[rayozzie]

This does feel somewhat similar to these: espressif/esp-idf#1202 #991 I am not familiar with the engineering processes and practices. If indeed there were fixes that resolved these bugs, and these or any other fixes have not made it into the Arduino driver, where does an issue need to be submitted to get some priority on resolution and delivery in that code base? Thx On Feb 24, 2018, at 4:13 AM, negativekelvin <notifications@github.com<mailto:notifications@github.com>> wrote: Arduino has not fixed uart fifo reset bug that was fixed in idf — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<#1145 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AAuhPT8bmgXyIg1C5JYWXWQ4t5PPWvc_ks5tX35ngaJpZM4SRBmf>.

[JacoFourie]

If I put a Flash before sending I am able to use all 3 the UARTS without a problem.

Serial1.flush();

I use
UART0 for debug
UART1 for RS-485 (MODBUS) (Set VSD values)
UART2 actuator control.

and it all works fine now.

[rayozzie]

You are suggesting this as a workaround for the end() followed by a begin() to reinitialize ports? I never even considered a flush before every write(), however I did try a flush() before the end() and after the begin(), and got quite unstable results. On Feb 27, 2018, at 6:28 AM, JacoFourie <notifications@github.com<mailto:notifications@github.com>> wrote: If I put a Flash before sending I am able to use all 3 the UARTS without a problem. Serial1.flush(); — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<#1145 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AAuhPdMo7gkAFtDMZIUvAYSdhfv4GlAiks5tY5KCgaJpZM4SRBmf>.

[JacoFourie]

I am using all UARTs all the time. Before I do a send on UART1 I do a flush.
UART 0 and 2 works fine. for me.

MyModbusSerial.flush();
MyModbusSerial.println("Testing");

[rayozzie]

To be clear, the problem I am encountering has nothing at all to do with uarts or flush. They are all working. The problem is that when a working Uart is uninitialized with end() and then reinitialized again with begin(), it puts the Uart into a messed up state in which there is misordering and reappearance of previously buffered contents. On Feb 27, 2018, at 8:57 AM, JacoFourie <notifications@github.com<mailto:notifications@github.com>> wrote: I am using all UARTs all the time. Before I do a send on UART1 I do a flush. UART 0 and 2 works fine. for me. MyModbusSerial.flush(); MyModbusSerial.println("Testing"); — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<#1145 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AAuhPS2YXRmEbw_w-fOV9DfXWxzeHDA3ks5tY7VfgaJpZM4SRBmf>.

[JacoFourie]

OK. Why do you need to do that ? If they are working then let them work.

[rayozzie]

My device has a UART multiplexer attached to RX/TX on one of the uarts (with a GPIO-driven selector), that fans out to four different serial devices. Each of the four can potentially be at a different baud rate, necessitating an end()/begin() to change the baud rare each time it is switched. On Feb 27, 2018, at 9:15 AM, JacoFourie <notifications@github.com<mailto:notifications@github.com>> wrote: OK. Why do you need to do that ? If they are working then let them work. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<#1145 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AAuhPfc-HuNF1Kj9gyL1tK2ufvcEPzlwks5tY7mdgaJpZM4SRBmf>.