SDMMC performance issues

[yiming564]

Created a seperated issue from #211 .

In short, esp_littlefs's sdmmc speed is pretty slow, and from my simple test the sequential writing speed at 40Mhz 4 line sdmmc mode is 48 KiB/s.

In conparison, FatFS's sequential writing speed is 13.76 MiB/s, hundreds of times of esp_littlefs's speed.

---

I do all these tests on a U3 (UHS Speed Class 3) SD Card so it's unlikely to be the bottleneck, the ASSSD Benchmark score is shown below:

---

LittleFS's configuration:

CONFIG_LITTLEFS_SDMMC_SUPPORT=y
CONFIG_LITTLEFS_MAX_PARTITIONS=3
CONFIG_LITTLEFS_PAGE_SIZE=512
CONFIG_LITTLEFS_OBJ_NAME_LEN=255
CONFIG_LITTLEFS_READ_SIZE=128
CONFIG_LITTLEFS_WRITE_SIZE=128
CONFIG_LITTLEFS_LOOKAHEAD_SIZE=512
CONFIG_LITTLEFS_CACHE_SIZE=512
CONFIG_LITTLEFS_BLOCK_CYCLES=512
CONFIG_LITTLEFS_USE_MTIME=y
# CONFIG_LITTLEFS_USE_ONLY_HASH is not set
CONFIG_LITTLEFS_HUMAN_READABLE=y
CONFIG_LITTLEFS_MTIME_USE_SECONDS=y
# CONFIG_LITTLEFS_MTIME_USE_NONCE is not set
# CONFIG_LITTLEFS_SPIFFS_COMPAT is not set
# CONFIG_LITTLEFS_FLUSH_FILE_EVERY_WRITE is not set
CONFIG_LITTLEFS_FCNTL_GET_PATH=y
CONFIG_LITTLEFS_FCNTL_F_GETPATH_VALUE=20
CONFIG_LITTLEFS_MULTIVERSION=y
CONFIG_LITTLEFS_DISK_VERSION_MOST_RECENT=y
# CONFIG_LITTLEFS_DISK_VERSION_2_1 is not set
# CONFIG_LITTLEFS_DISK_VERSION_2_0 is not set
# CONFIG_LITTLEFS_MALLOC_STRATEGY_DISABLE is not set
CONFIG_LITTLEFS_MALLOC_STRATEGY_DEFAULT=y
# CONFIG_LITTLEFS_MALLOC_STRATEGY_INTERNAL is not set
CONFIG_LITTLEFS_ASSERTS=y

[yiming564]

More info: LittleFS vs FATFS on spi flash:

I (00:00:03.215) app_manager: Start stress test /sys/BigFile.txt.
I (00:00:03.276) app_manager: Write 0/64 blk.
I (00:00:43.954) app_manager: Completed!
I (00:00:43.956) app_manager: File size check: 4194304 Bytes
I (00:00:43.956) app_manager: Start read test.
I (00:00:43.960) app_manager: Read 0/64 blk.
I (00:00:44.508) app_manager: Completed!
I (00:00:44.509) app_manager: File size check: 4194304 Bytes
I (00:00:44.509) app_manager: Start stress test /fat/BigFile.txt.
I (00:00:44.516) app_manager: Write 0/64 blk.
I (00:01:50.833) app_manager: Completed!
I (00:01:51.020) app_manager: File size check: 4194304 Bytes
I (00:01:51.020) app_manager: Start read test.
I (00:01:51.021) app_manager: Read 0/64 blk.
I (00:01:51.335) app_manager: Completed!
I (00:01:51.336) app_manager: File size check: 4194304 Bytes

	Read	Write
LittleFS	7.299 MiB/s	0.098 MiB/s
FATFS	12.73 MiB/s	0.086 MiB/s

spi flash's configuration is QIO 120 Mhz.

[BrianPugh]

That's an excellent datapoint, and about the performance difference when compared to FAT that I would expect. This strongly indicates that it's an sdmmc specific issue, probably related to read/write sizes.

[Josele]

I would like to share some benchmark results regarding this issue which I found interesting. I wanted to see how the transfer speed varies based on the type of SD cards.
Summary of the test (worst case scenarios) :
Netac 8GB HC (10):
Write: 0.070 MB/s
Read: 0.066 MB/s

OLEOO 16GB V10 A1-1 HC (10):
Write: 0.014 MB/s
Read: 0.309 MB/s

Note: This card good quite hot.

SanDisk Exteme Pro 128GB V30 A2-3 XC I
Write: 0.146 MB/s
Read: 0.207 MB/s

Results:

Netac 8GB HC (10):

--- Benchmarking: /sd/test.bin ---
File size: 65536 ---
Write Buffer size: 4096 ---
WRITE: 65536 bytes in 0.86s (0.073 MB/s)
READ:  65536 bytes in 0.60s (0.105 MB/s)

--- Benchmarking: /sd/test.bin ---
File size: 65536 ---
Write Buffer size: 8192 ---
WRITE: 65536 bytes in 0.86s (0.073 MB/s)
READ:  65536 bytes in 0.60s (0.105 MB/s)

--- Benchmarking: /sd/test.bin ---
File size: 65536 ---
Write Buffer size: 16384 ---
WRITE: 65536 bytes in 0.84s (0.074 MB/s)
READ:  65536 bytes in 0.60s (0.105 MB/s)

--- Benchmarking: /sd/test.bin ---
File size: 65536 ---
Write Buffer size: 65536 ---
WRITE: 65536 bytes in 0.85s (0.073 MB/s)
READ:  65536 bytes in 0.60s (0.105 MB/s)

--- Benchmarking: /sd/test.bin ---
File size: 1048576 ---
Write Buffer size: 4096 ---
WRITE: 1048576 bytes in 13.63s (0.073 MB/s)
READ:  1048576 bytes in 13.29s (0.075 MB/s)

--- Benchmarking: /sd/test.bin ---
File size: 1048576 ---
Write Buffer size: 8192 ---
WRITE: 1048576 bytes in 14.29s (0.070 MB/s)
READ:  1048576 bytes in 15.22s (0.066 MB/s)

--- Benchmarking: /sd/test.bin ---
File size: 1048576 ---
Write Buffer size: 16384 ---
WRITE: 1048576 bytes in 13.62s (0.073 MB/s)
READ:  1048576 bytes in 13.30s (0.075 MB/s)

--- Benchmarking: /sd/test.bin ---
File size: 1048576 ---
Write Buffer size: 32768 ---
WRITE: 1048576 bytes in 14.34s (0.070 MB/s)
READ:  1048576 bytes in 15.20s (0.066 MB/s)
Test finished

OLEOO 16GB V10 A1-1 HC (10):

--- Benchmarking: /sd/test.bin ---
File size: 65536 ---
Write Buffer size: 4096 ---
WRITE: 65536 bytes in 4.33s (0.014 MB/s)
READ:  65536 bytes in 0.15s (0.428 MB/s)

--- Benchmarking: /sd/test.bin ---
File size: 65536 ---
Write Buffer size: 8192 ---
WRITE: 65536 bytes in 4.32s (0.014 MB/s)
READ:  65536 bytes in 0.15s (0.431 MB/s)

--- Benchmarking: /sd/test.bin ---
File size: 65536 ---
Write Buffer size: 16384 ---
WRITE: 65536 bytes in 4.33s (0.014 MB/s)
READ:  65536 bytes in 0.15s (0.422 MB/s)

--- Benchmarking: /sd/test.bin ---
File size: 65536 ---
Write Buffer size: 65536 ---
WRITE: 65536 bytes in 4.33s (0.014 MB/s)
READ:  65536 bytes in 0.15s (0.425 MB/s)

--- Benchmarking: /sd/test.bin ---
File size: 1048576 ---
Write Buffer size: 4096 ---
WRITE: 1048576 bytes in 69.35s (0.014 MB/s)
READ:  1048576 bytes in 3.23s (0.310 MB/s)

--- Benchmarking: /sd/test.bin ---
File size: 1048576 ---
Write Buffer size: 8192 ---
WRITE: 1048576 bytes in 69.49s (0.014 MB/s)
READ:  1048576 bytes in 3.24s (0.309 MB/s)

--- Benchmarking: /sd/test.bin ---
File size: 1048576 ---
Write Buffer size: 16384 ---
WRITE: 1048576 bytes in 69.32s (0.014 MB/s)
READ:  1048576 bytes in 3.21s (0.311 MB/s)

--- Benchmarking: /sd/test.bin ---
File size: 1048576 ---
Write Buffer size: 32768 ---
WRITE: 1048576 bytes in 69.46s (0.014 MB/s)
READ:  1048576 bytes in 3.21s (0.311 MB/s)
Test finished

--- Benchmarking: /sd/test.bin ---
File size: 65536 ---
Write Buffer size: 4096 ---
WRITE: 1048576 bytes in 4.34s (0.014 MB/s)
READ:  1048576 bytes in 0.15s (0.425 MB/s)

SanDisk Exteme Pro 128GB V30 A2-3 XC I

--- Benchmarking: /sd/test.bin ---
File size: 65536 ---
Write Buffer size: 4096 ---
WRITE: 65536 bytes in 0.40s (0.157 MB/s)
READ:  65536 bytes in 0.22s (0.284 MB/s)

--- Benchmarking: /sd/test.bin ---
File size: 65536 ---
Write Buffer size: 8192 ---
WRITE: 65536 bytes in 0.40s (0.155 MB/s)
READ:  65536 bytes in 0.22s (0.282 MB/s)

--- Benchmarking: /sd/test.bin ---
File size: 65536 ---
Write Buffer size: 16384 ---
WRITE: 65536 bytes in 0.40s (0.155 MB/s)
READ:  65536 bytes in 0.21s (0.292 MB/s)

--- Benchmarking: /sd/test.bin ---
File size: 65536 ---
Write Buffer size: 65536 ---
WRITE: 65536 bytes in 0.40s (0.157 MB/s)
READ:  65536 bytes in 0.22s (0.288 MB/s)

--- Benchmarking: /sd/test.bin ---
File size: 1048576 ---
Write Buffer size: 4096 ---
WRITE: 1048576 bytes in 6.37s (0.157 MB/s)
READ:  1048576 bytes in 4.71s (0.212 MB/s)

--- Benchmarking: /sd/test.bin ---
File size: 1048576 ---
Write Buffer size: 8192 ---
WRITE: 1048576 bytes in 6.74s (0.148 MB/s)
READ:  1048576 bytes in 4.75s (0.210 MB/s)

--- Benchmarking: /sd/test.bin ---
File size: 1048576 ---
Write Buffer size: 16384 ---
WRITE: 1048576 bytes in 6.52s (0.153 MB/s)
READ:  1048576 bytes in 4.76s (0.210 MB/s)

--- Benchmarking: /sd/test.bin ---
File size: 1048576 ---
Write Buffer size: 32768 ---
WRITE: 1048576 bytes in 6.89s (0.145 MB/s)
READ:  1048576 bytes in 4.79s (0.209 MB/s)
Test finished

Snip of the code:

void runBenchmark(const char *path, size_t testFileSize, size_t heapUsed)
{
  uint8_t *buffer = (uint8_t *)heap_caps_malloc(heapUsed, MALLOC_CAP_INTERNAL | MALLOC_CAP_CACHE_ALIGNED | MALLOC_CAP_DMA);
  if (!buffer)
    return;
  memset(buffer, 0xAA, heapUsed);
  int fd;
  int flags = O_CREAT | O_TRUNC | O_RDWR;

  Serial.printf("\n--- Benchmarking: %s ---\n", path);
  Serial.printf("File size: %u ---\n", testFileSize);
  Serial.printf("Write Buffer size: %u ---\n", heapUsed);

  // --- WRITE TEST ---
  fd = open(path, flags, 0666);
  if (fd < 0)
  {
    Serial.println("Write open failed");
    free(buffer);
    return;
  }

  uint32_t start = millis();
  size_t bytesWritten = 0;
  while (bytesWritten < testFileSize)
  {
    bytesWritten += write(fd, buffer, heapUsed);
  }
  uint32_t end = millis();
  close(fd);

  float writeTime = (end - start) / 1000.0f;
  float writeSpeed = (testFileSize / 1024.0f / 1024.0f) / writeTime;
  Serial.printf("WRITE: %u bytes in %.2fs (%.3f MB/s)\n", testFileSize, writeTime, writeSpeed);

  // --- READ TEST ---
  flags = O_RDONLY;
  fd = open(path, flags);
  if (fd < 0)
  {
    Serial.println("Read open failed");
    free(buffer);
    return;
  }

  start = millis();
  size_t bytesRead = 0;
  while (bytesRead < testFileSize)
  {
    bytesRead += read(fd, buffer, heapUsed);
  }
  end = millis();
  close(fd);
  float readTime = (end - start) / 1000.0f;
  float readSpeed = (testFileSize / 1024.0f / 1024.0f) / readTime;
  Serial.printf("READ:  %u bytes in %.2fs (%.3f MB/s)\n",  testFileSize, readTime, readSpeed);

  free(buffer);
}

Some notes: For the size of the card, I have forced the sdmmc_card_t capacity to be less than 4GB to work properly with littlefs.
Edit: There was a bug in the formatting. I've retested and obtained really similar results.