Hardware USB JTAG/Serial creates lags when used.

[sblantipodi]

Board

Lolin ESP32-S3 mini

Device Description

Plain Lolin ESP32-S3 mini

Hardware Configuration

GPIO 16

Version

latest master (checkout manually)

IDE Name

CLION

Operating System

Windows 11

Flash frequency

80MHz

PSRAM enabled

yes

Upload speed

115200

Description

My firmware Luciferin is based on the NeoPixelBus library to control LEDs to act like an ambilight clone.

NeoPixelBus uses RMT to drive the LEDs.

When I set the board in TinyUSB mode I can drive the LEDs at 144 FPS without any flickering or lags.

When I set the board in USB JTAG/Serial I can see heavy flickering even at 60FPS.

The author of NeoPixelBus supposed that two things can cause this issue:

• ISR priority and saturation, their hardware may still fire interrupts (data ready) and it is written that causes tons of interrupts; which have overhead to save registers, copy data, restore registers. Too many is a waist just for ISR overhead. Then their ISR maybe at a higher priority starving the RMT ISR>
• ISR cycle consumption, their ISR takes way to long to do work, so when it releases and the RMT ISR triggers, it way past a PWM timing acceptance.
  TinyUSB maybe at the same priority as the RMT ISR.

Is there anything to fix in the RMT API that can cause this flickering?

Isn't USB JTAG/Serial an hardware method and as such supposed to work better than the software TinyUSB method?

Sketch

https://github.com/sblantipodi/glow_worm_luciferin

Debug Message

-

Other Steps to Reproduce

I have checked existing issues, online documentation and the Troubleshooting Guide

• I confirm I have checked existing issues, online documentation and Troubleshooting guide.

[SuGlider]

I've a few questions:

What Arduino Core is used for measuring the FPS?
The FPS difference is verified when the USB is connected or not connected or in both cases?

Could you describe how to measure the FPS in order to allow me replicate this issue?

[sblantipodi]

@SuGlider thanks for the answer.

I'm using Arduino v2.0.9.
My firmware Luciferin is able to measure how much FPS the ESP is delivering to the LED strip.

The FPS difference (between the PC pushing informations on Serial and the firmware delivering that infos to the led strip) happens when printing some info to serial with Serial.print() and no USB is connected, this cause a big lag.
This problem can be workarounded with this:
Serial.setTxTimeoutMs(0);

but the LEDs flickering when driving LEDs via USB with many Serial.read() remains...

Hardware CDC should be more stable but with the Espressif implementation the TinyUSB software method seems much more stable to me.

Do you have a LED strip to try it out? If yes, I can explain how to use Luciferin to reproduce the problem with easy.

[sblantipodi]

@SuGlider is there anything I can do to help you reproducing the issue?
is there a way to set ISR at the same priority of the RMT ISR?

[SuGlider]

@sblantipodi - I have looked into the Project. I found the void Globals::sendSerialInfo() function that seems to be used to display the FPS information in the Serial port.

Congratulations for your project! It looks great and very well done!
Arduino uses to run in a single Task for setup() and loop(). In a single Task, everything is processed in sequence and Printing/Reading too many messages consumes some time, degrading a bit the FPS.

Serial.setTimeOut(0) shall solve the issue with JTAG/CDC delay in writing to the port when it is plugged/unplugged, as you have already verified. It has no effect with the ESP32 or with the ESP32-S2 (those two have no CDC/JTAG interface.)

JTAG/CDC is good whenever your device needs to upload firmware using the WEB USB LaunchPad platform. Not sure if this is the best performing interface when compared to UART or OTG (TinyUSB).

The concept of ISR priority is not the right way to think an ESP32 application.
It shall use FreeRTOS task priority instead.

I suggest using Task resources from FreeRTOS in order to make it more efficient and Real Time.
Your project is all about Performance, the LED/RMT shall run on its own Task with High Priority, making sure that the LED processing is the most important and fastest activity/task in the code.

Does the Flickering change if CDC/TinyUSB is used instead of CDC/JTAG?
CDC/JTAG Reading has no TimeOut. I see that the project already uses 2048 bytes in the RX Buffer.

I'd suggest a few possible improvements:

• In mainLoop.cpp, I can see that Serial port is read for configuration parameters.
  Maybe it can be made more efficient by reading a block of data at once and processing it.
  A long set of testings and single byte reads may take too long for an application like in your project.

• Another possible architecture, would be creating a FreeROTS task for processing the LOG messages, with a Message Queue. This would run in a separated task and consume this Queue, message by message, printing it out. This may help unloading the Main Arduino Task, alowing it to process faster and, therefore, improve the FPS.

[SuGlider]

@sblantipodi - Thinking about how to read the CDC/JTAG, an option is using onEvent() to register a Callback function for any ARDUINO_HW_CDC_RX_EVENT receiving package, instead of quering the interface with Serial.available() in the mainLoop().

This is valid for both JTAG/CDC and OTG/CDC (TinyUSB). But each one has its own event names (ARDUINO_HW_CDC_RX_EVENT / ARDUINO_USB_CDC_RX_EVENT).

An example of how to use can be found at https://github.com/espressif/arduino-esp32/blob/master/libraries/USB/examples/USBSerial/USBSerial.ino

Using onEvent() will allow your application to use a callback function only whenever Data is received, processing it faster, at once. Maybe it can solve the Flicker issue.

[sblantipodi]

Hi @SuGlider thanks for the very detailed answer, I really, really appreciate it.
It took a bit to answer because I tried all the suggestions :)

Serial.setTimeOut(0) shall solve the issue with JTAG/CDC delay in writing to the port when it is plugged/unplugged, as you have already verified. It has no effect with the ESP32 or with the ESP32-S2 (those two have no CDC/JTAG interface.)

Correct, setting the timeout to 0 improves the lag.

I suggest using Task resources from FreeRTOS in order to make it more efficient and Real Time.

I created a dedicated high priority task, the strange thing is that it seems to create even more flickering.

Does the Flickering change if CDC/TinyUSB is used instead of CDC/JTAG?

If I use CDC/JTAG the flickering is crazy,
if I use CDC/TinyUSB is gone completely.

I think that the answer to this can be found in the official TinyUSB site.
TinyUSB is an open-source cross-platform USB Host/Device stack for embedded systems, designed to be memory-safe with no dynamic allocation and thread-safe with all interrupt events being deferred and then handled in the non-ISR task function.

• Maybe it can be made more efficient by reading a block of data at once and processing it.

This improved the flickering by a very small margin, but it is a good improvement, thanks for pointing it out.

• Another possible architecture, would be creating a FreeROTS task for processing the LOG messages, with a Message Queue.

this improved the flickering a little bit more...

Thinking about how to read the CDC/JTAG, an option is using onEvent() to register a Callback function for any ARDUINO_HW_CDC_RX_EVENT receiving package, instead of quering the interface with Serial.available() in the mainLoop().

I implemented all the suggestions, the flickering has been reduced by a little margin but it's there and it's severe.

The only things that solves the flickerin 100% is switching from CDC/JTAG to CDC/TinyUSB but this is more a workaround rather than a real solution to the problem.

[SuGlider]

@sblantipodi please try Luciferin using the Arduino Core 3.0.0 from the master branch. It has the PR #9275 that shall fix issues with timeout while writing the USB JTAG/Serial when it is unplugged.

Thanks!

[sblantipodi]

@SuGlider thanks for quoting me and for informing me about the progress on this issue. I appreciate it.
I will be able to test it on Friday and I'll report back my findings. Thanks.