SPI with the R4 is slower than with the R3

[RudolphRiedel]

The SPI with the R4 is really slow, way slower than with the R3 when running the same clock and the same code.

SPI_Arduino_Uno.zip

This has three files for use with Saleae Logic 2.

• SPI_Arduino_Uno_R3_SPI_Transfer.sal - single byte SPI.transfer()
• SPI_Arduino_Uno_R4_SPI_Transfer.sal - single byte SPI.transfer()
• SPI_Arduino_Uno_R4_SPI_Transfer_Buffer.sal - buffer transmit, for 4 bytes and for the whole large buffer

The "large" transfer has 3 bytes address + 228 bytes data.
R3: 524µs including all calculations necessary for that data
R4: 2.52ms including all calculations necessary for that data - so this takes almost five times longer
R4 buffer: 634µs without any calculations, only transferring the buffer

Is this something that is actively looked into so far?

The underlying FSP library from Renesas is rather slow.
It configures the SPI on every single transfer, even going as far as activating and deactivating the SPI thru the enable bit.

On the R3 I get <2µs pauses between two transfers.
On the R4 there are 11µs between two transfers - at three times the core clock.

I modified SPI.cpp to use a busy loop for with TX only for the buffer transfer:

uint8_t *buffer = (uint8_t *) buf;

_spi_ctrl.p_regs->SPCR = 0x4a; /* enable, no irq, master, tx only */

for(size_t index = 0; index < count; index++)
{
    _spi_ctrl.p_regs->SPDR_BY = buffer[index];
    while (0 == _spi_ctrl.p_regs->SPSR_b.SPTEF) {}
}
    while (_spi_ctrl.p_regs->SPSR_b.IDLNF) {}

_spi_ctrl.p_regs->SPCR = 0xb9; /* off, interrupts, master, full duplex */

This brought down the time to transfer the buffer to 310us with 400ns pauses.
Still not what the RA4M1 could do but at least it beats the AVR on the R3.
So this is entirely an software issue.

A couple of function calls into R_SPI_WriteRead() to check out what is going on there I found that
the buffer is send with an interrupt function.
So essentially it looks like the function is spending more time going into and out of that interrupt than it takes to transfer a byte.

And on top the RA4M1 has DMA, please make use of it.
Preferably take inspiration from the Teensy 4 SPI class that has DMA support baked in to allow this:

SPI.transfer(source, dest, count, event);
SPI.transfer( ((uint8_t *) &EVE_dma_buffer[0]) + 1U, NULL, (((EVE_dma_buffer_index) * 4U) - 1U), EVE_spi_event);

This "event" is for a callback function, a custom one that can be attached to the event and allows code to be executed when the DMA is done like setting CS to high again.
And setting "dest" to NULL makes the operation write only.

[aentinger]

Hi @RudolphRiedel ☕ 👋

Thank you for taking the time to take those measurements and report them 👍 .

The main motivation for using the FSP layer is to achieve maximum portability across various Renesas platforms. That being said, the performance penalty incurred simply sucks.

We'll be looking into ways to enhance SPI performance.

[RudolphRiedel]

I am afraid the this is not the only bottleneck that FSP presents but SPI is kind of "my thing" right now and in general something like this really is not unique to FSP.

So I sure will be having some fun going around the SPI class as I did before with a number of other controllers.
But while this works it really reduces the compatibility, playing nice with other SPI nodes is more difficult to achieve then.

On the Arduino side it really would be nice to have a common API that supports SPI transmits over DMA.

[aentinger]

On the Arduino side it really would be nice to have a common API that supports SPI transmits over DMA.

No comments on DMA but if you'd be willing to work on PR improving the performance for the SPI module I'd be happy to check and merge it. Certainly we - Arduino - don't want stay of valueable community contributions. If SPI is your thing, I kindly invite you to bring it on! 🚀

[RudolphRiedel]

I can try, I wanted to play with the RA4M1 anyways.
And even if my pull request gets rejected I still will have had some fun with it and code that I can use for myself. :-)
Don't hold your breath though. :-)

[aentinger]

I'm looking forward to see what you're coming up with 😉

[greiman]

I posted a comparison of Uno R3 vs R4 SdFat performance on the Arduino form. Also Saleae capture of the SPI signals here:

https://forum.arduino.cc/t/uno-r4-poor-spi-performance/1143935

Summary: the R3 is about eight times faster than the R4 for the current version of SdFat..

R4 single bytes transfer standard SPI library:
write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
82.98,7164,5160,6162
82.98,7176,5160,6163

R3 array with my custom AVR SPI functions:
write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
689.18,760,728,736
689.18,760,728,736

The clock rate for the R4 is 24 MHz but there is a huge gap of about 11 μs between bytes.

The Arduino SD.h library which is based on a modified 2009 version of SdFat is also very slow on the R4.

I looked at the Renesas FSP SPI implementation. I suspect there is little hope of decent performance at 24 MHz without DMA.

For fast SPI, it would be nice if Arduino added this member function to the SPI API. It is in many third party board support packages. The current void transfer(void *buf, size_t count) is difficult to use since it requires a temporary buffer in many cases.

void transfer(const void* txBuf, void* rxBuf, size_t count);

If txBuf is nullptr, 0XFF is sent. If rxBuf is nullptr receive data is discarded.

I looked at the RA4M1 SPI hardware. It has 32-bit RX and TX buffers so it may be possible to get close to 24 MHz with programmed I/O. The Hardware has a feature that prevents overruns by clock stretching. See my above post for cases where the clock is stopped in FSP and the eighth bit is delayed for about 2 μs. This is great to avoid overruns when using both transmit and receive buffering.

I have a mod for the Arduino SAMD21 core that gets close to full speed at 12 MHz using 8-bit SPI I/O. It is about three times faster than the Arduino core functions with SdFat.

Here is the small but tricky implementation that takes advantage of buffer registers in the SAMD21/SAMD51.

void SERCOM::transferDataSPI(const void *txBuf, void *rxBuf, size_t count) {
  const uint8_t *tx = reinterpret_cast<const uint8_t *>(txBuf);
  uint8_t *rx = reinterpret_cast<uint8_t *>(rxBuf);
  size_t ir = 0;
  size_t it = 0;
  if (rx) {
    while (it < 2 && it < count) {
      if (sercom->SPI.INTFLAG.bit.DRE) {
        sercom->SPI.DATA.reg = tx ? tx[it] : 0XFF;
        it++;
      }
    }
    while (it < count) {
      if (sercom->SPI.INTFLAG.bit.RXC) {
        rx[ir++] = sercom->SPI.DATA.reg;
        sercom->SPI.DATA.reg = tx ? tx[it] : 0XFF;
        it++;
      }
    }
    while (ir < count) {
      if (sercom->SPI.INTFLAG.bit.RXC) {
        rx[ir++] = sercom->SPI.DATA.reg;
      }
    }
  } else if (tx && count) {  // might hang if count == 0
    // Writing '0' to this bit will disable the SPI receiver immediately.
    // The receive buffer will be flushed, data from ongoing receptions
    // will be lost and STATUS.BUFOVF will be cleared.
    sercom->SPI.CTRLB.bit.RXEN = 0;
    while (it < count) {
      if (sercom->SPI.INTFLAG.bit.DRE) {
        sercom->SPI.DATA.reg = tx[it++];
      }
    }
    // wait till all data sent
    while (sercom->SPI.INTFLAG.bit.TXC == 0) {
    }
    // Writing '1' to CTRLB.RXEN when the SPI is enabled will set
    // SYNCBUSY.CTRLB, which will remain set until the receiver is
    // enabled, and CTRLB.RXEN will read back as '1'.
    sercom->SPI.CTRLB.bit.RXEN = 1;
    while (sercom->SPI.CTRLB.bit.RXEN == 0) {
    }
  } 
} 

Arduino Core performance:

write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
468.78,1111,1087,1089

With the above function:

write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
1359.80,17535,368,374

It would be nice to have something like this for Uno R4.

[RudolphRiedel]

I have a couple ideas that I need to try out and I will start with the standard API before venturing into non standard API options.
The challenge is of course to make this robust and non breaking as well. :-)
Well, I already sped up things significantly here.

I am actually quite familiar with the SERCOMs from the ATSAM but this will not be of much help here. :-)

[greiman]

RudolphRiedel

The point of the SAMD code was that the RA4M1 has similar RX/TX buffer registers so it should be possible to do array send/receive in a loop at fairly high speed.

Even more interesting is the fact that the RA4M1 has a 32 bit mode so you should be able to transfer the bulk of an array using 32-bit mode.

Looks like you are using 8MHz clock to send a 228 byte array in 634µs.

What rate do you get for 24MHz?

I tried the R4 unmodified array transfer at 24 MHz and get about 650µs. This is for both send and receive.

Here is my test program. I looked at it with Salese and the array takes 635µs but the print is 656µs. The clock is 24 MHz.

#define CS_PIN 10
#define SPI_CLOCK 24000000
void test() {
  uint8_t buf[228];
  SPI.beginTransaction(SPISettings(SPI_CLOCK, MSBFIRST, SPI_MODE0));
  uint32_t m = micros();
  SPI.transfer(buf, sizeof(buf));
  m = micros() - m;
  Serial.println(m);
}

It has the strange clock stretching with seven pulses at 24 MHz then the final pulse much later.

I ran it at 8MHz and the time is the same as at 24 MHz so it is totally CPU limited. The clock looks like your clock.

[WestfW]

It looks vaguely like things would get better if the fsp libraries were compiled to use DMA for SPI, even if they continue to block until the end of the transfer...

[greiman]

It looks vaguely like things would get better if the fsp libraries were compiled to use DMA for SPI, even if they continue to block until the end of the transfer...

DMA is the best answer. I did DMA for a number of MCUs for use in my Arduino SdFat library.

With my Due DMA function I get:

write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
4533.09,2313,110,111
4537.21,127,110,111

Due with the standard Arduino library using void transfer(void* buf, size_t count) array transfer:

write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
1598.98,336,317,318
1597.95,336,317,318

Due with the standard Arduino library single byte uint8_t transfer(uint8_t data) in a loop:

write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
693.39,754,735,736
693.19,754,735,736

The RA4M1 seems to support SPI reliably at 24MHz clock so I would hope to see something close to half the Due rate with SdFat.

This is the result for Uno R4 with the current Arduino library array transfer function at 24 MHz:

write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
430.33,2638,634,1181
429.85,2663,634,1183

I would hope to get over 2,000 KB/sec with DMA on Uno R4.

Even if DMA is not used, it should be possible to optimize array transfer using the buffering in the SPI controller better.

Here is the improvement I got in SAMD21 at 12 MHz:

MRK ZERO Arduino standard library array transfer:

write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
425.97,1220,1196,1199
425.89,1221,1196,1199

MRK ZERO with the optimized array transfer function in the above post:

write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
1365.37,396,368,372
1365.37,394,368,372

[greiman]

The comment by WestfW about blocking with the current SPI API, even if you are using DMA, convinced me DMA is necessary and an improved API is needed.

I ran the Uno R4 FreeRTOS-Blink example. It is wonderful to see Arduino support an RTOS on Uno R4. Next the libraries need to be RTOS friendly.

The SPI API need a clean way to implement non-blocking transfers.

Most RTOS APIs support a more general array transfer. This is becoming common, sometimes with a callback parameter:

void transfer(const void* txBuf, void* rxBuf, size_t count, callback_t callback = nullptr);

[greiman]

I decided to try RudolphRiedel's program at 24 MHz.

Looks like it is fairly good. About 109µs to transfer a 228 byte buffer. That's over 2000KB/sec. This is way faster than an Uno R3.

I am now going to try using 32-bit transfers for all except mod 4 of the count. That should allow fast combined write/read transfers.

[RudolphRiedel]

Hello,

I was fighting for the last couple of days to speed up single byte transfers, baby steps, understanding the unit before going further.
And it did not work at all.

Right now I have it up and running:

However, on the way I found out that the datasheet is not correct and that I need to set SPDCR to 0x040.

According to the datasheet though, b7 and b6 are "Reserved": "These bits are read as 0. The write value should be 0."

This is from R7FA4M1AB.h:
union
{
__IOM uint8_t SPDCR; /*!< (@ 0x0000000B) SPI Data Control Register */

struct
{
    __IOM uint8_t SPFC   : 2;  /*!< [1..0] Number of Frames Specification                     */
    __IOM uint8_t SLSEL  : 2;  /*!< [3..2] SSL Pin Output Select                                    */
    __IOM uint8_t SPRDTD : 1;  /*!< [4..4] SPI Receive/Transmit Data Selection             */
    __IOM uint8_t SPLW   : 1;  /*!< [5..5] SPI Word Access/Halfword Access Specification    */
    __IOM uint8_t SPBYT  : 1;  /*!< [6..6] SPI Byte Access Specification                        */
    uint8_t              : 1;
} SPDCR_b;

};

So SPFC, SLSEL and SPBYT are missing from the datasheet.
What the heck Renesas?

The question really is, what else is missing or wrong?
I am going on a fishing trip for more information now, perhaps there is a newer datasheet for a different controller of the RA family that is more accurate.

[greiman]

However, on the way I found out that the datasheet is not correct and that I need to set SPDCR to 0x040.

I also discovered the datasheet is incomplete and often wrong. There are even whole registers, not just bit fields that are not documented.

I am reduced to searching all of the FSP source for register names/addresses and bit fields.

I wrote a program to dump all the registers and use it to see what bits are used in the current SPI library.

I fought with 32-bit transfers because the bytes were not sent in the right order. I found an undocumented register, SPDCR2, that has an endian selection bit.

I also can find some definitions of fields by looking at datasheets for newer members of the RA family. UGH!

I now have array transfer working with 32-bit transfers. I can send a 228 byte array in 84µs, or about 2700 KB/Sec. Close enough to the 3000 KB/sec max for 24 MHz clock.

I should be able to do array send/receive since the time for one 32-bit transfer at 24MHz is 64 CPU cycles. Much better than 16 CPU cycle for 8-bit transfers.