Skip to content

SD vs ExFat - Raw Logging Tests #1

Closed
@adamgarbo

Description

@adamgarbo

Hi @PaulZC,

I thought I'd create an issue where I can share some of the results of my ongoing tests of the new u-blox library.

My first goal was to determine what kind of performance gains can be achieved using the ExFat classes of v2.0 of the SdFat library. Having read that 7 Hz appeared to be the upper limit for the SD library, I thought I'd start at 10 Hz. While the results below were only for a short period of time, they were quite telling!

Test Configuration:

  • All I2C pull-ups were disabled (ZED-F9P, MMDLCB, Artemis)
  • u-blox library example: DataLoggingExample5_Fast_RXM was used for both tests, with only the library, object instantiation and write command code being modified.
  • A micros() timer was added to the SD write commands to measure latency.

Components

  • Sparkfun Artemis Processor Board
  • SparkFun MicroMod Data Logging Carrier Board (MMDLCB)
  • SparkFun GPS-RTK2 ZED-F9P SMA
  • u-blox ANN-MB antenna

ZED-F9P Configuration:

  • Navigation rate: 10 Hz
  • Constellation: GPS only
  • Messages logged: RAWX/SFRBX

SD Test Configuration:

  • Arduino SD library
  • microSD card: SanDisk Industrial XI 32 GB
    • Formatted using FAT32

ExFat Test Configuration:

  • SdFat library (SdExFat/ExFile classes)
  • microSD card: SanDisk Extreme 64 GB
    • Formatted using ExFat

Results:

  • Figure 1 illustrates just how quickly the file buffer fills up at 10 Hz when using the SD library. It's clear that the 12 ms SD write latency is creating a bottleneck that causes the buffer to overflow.
  • The second SD test filled the buffer quicker likely because the receiver had a better fix at this time.
  • After maxing out the file buffer, eventually the entire system became non-responsible and could not be halted via serial input.
  • By comparison, the ExFat write latency of ~400 μs appears to be sufficient to allow the data to be written to the card and prevent the overflow issue.
  • I also left the ExFat tests run for longer periods of time and never saw the buffer exceed 5000 bytes.

max_buffer
Figure 1. Max buffer size over time

latency
Figure 2. SD write latencies

Conclusion & Next Steps:

  • The performance gains offered by the new SdFat library are significant!
  • The next step will be to see just how far I can push the library when using an ExFat formatted card.
  • I'll also start looking at data integrity after longer duration logging and see if I run into any of the same gremlins you and @nseidle had run into previously.

Cheers,
Adam

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions