Is it possible to use I2S audio and Camera at the sametime?

[takurot]

Hi

I want to do getting image from Camera and playing sounds via I2S both at the same time. I tried to with the sample, but it did not work. When the camera get image, I2S audio play no sound at all. I can not figure out if I use library wrong or if it is not even possible. If you have any experience on it, please advice.

I use ESP32-Wrover-dev board, and my test code is as bellow

Camera Definition

#define PWDN_GPIO_NUM    -1
#define RESET_GPIO_NUM   -1
#define XCLK_GPIO_NUM    21
#define SIOD_GPIO_NUM    26
#define SIOC_GPIO_NUM    27

#define Y9_GPIO_NUM      35
#define Y8_GPIO_NUM      34
#define Y7_GPIO_NUM      39
#define Y6_GPIO_NUM      36
#define Y5_GPIO_NUM      19
#define Y4_GPIO_NUM      18
#define Y3_GPIO_NUM       5
#define Y2_GPIO_NUM       4
#define VSYNC_GPIO_NUM   25
#define HREF_GPIO_NUM    23
#define PCLK_GPIO_NUM    22

Test Code. It can play sounds if I remove "esp_camera_fb_get();"

#include "esp_camera.h"
#include <HTTPClient.h>
#include <Preferences.h>
#include <WiFi.h>
#include "Audio.h"

/* I2S Audio Settings */
#define I2S_DOUT     15
#define I2S_BCLK     12
#define I2S_LRC      33

Audio audio();

void setup() {
  Serial.begin(115200);
  Serial.setDebugOutput(true);

  if(psramFound()){
    Serial.println("PSRAM Found");
  }

  camera_config_t config;
  config.ledc_channel = LEDC_CHANNEL_0;
  config.ledc_timer = LEDC_TIMER_0;
  config.pin_d0 = Y2_GPIO_NUM;
  config.pin_d1 = Y3_GPIO_NUM;
  config.pin_d2 = Y4_GPIO_NUM;
  config.pin_d3 = Y5_GPIO_NUM;
  config.pin_d4 = Y6_GPIO_NUM;
  config.pin_d5 = Y7_GPIO_NUM;
  config.pin_d6 = Y8_GPIO_NUM;
  config.pin_d7 = Y9_GPIO_NUM;
  config.pin_xclk = XCLK_GPIO_NUM;
  config.pin_pclk = PCLK_GPIO_NUM;
  config.pin_vsync = VSYNC_GPIO_NUM;
  config.pin_href = HREF_GPIO_NUM;
  config.pin_sscb_sda = SIOD_GPIO_NUM;
  config.pin_sscb_scl = SIOC_GPIO_NUM;
  config.pin_pwdn = PWDN_GPIO_NUM;
  config.pin_reset = RESET_GPIO_NUM;
  config.xclk_freq_hz = 20000000;
//  config.frame_size = FRAMESIZE_UXGA;
  config.frame_size = FRAMESIZE_VGA;
  config.pixel_format = PIXFORMAT_JPEG; // for streaming
  //config.pixel_format = PIXFORMAT_RGB565; // for face detection/recognition
  config.grab_mode = CAMERA_GRAB_WHEN_EMPTY;
  config.fb_location = CAMERA_FB_IN_PSRAM;
  config.jpeg_quality = 10;
  config.fb_count = 1;

  // camera init
  esp_err_t err = esp_camera_init(&config);
  if (err != ESP_OK) {
    Serial.printf("Camera init failed with error 0x%x", err);
    return;
  }

  WiFi.begin(ssid, password);
  WiFi.setSleep(false);

  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }
  Serial.println("");
  Serial.println("WiFi connected");

  delay(1500);

  audio.setPinout(I2S_BCLK, I2S_LRC, I2S_DOUT);
  audio.setVolume(10);
  audio.connecttohost("http://mp3.ffh.de/radioffh/hqlivestream.mp3");

}

void loop() {
  audio.loop();

// ---- get camera capture
  camera_fb_t *fb = NULL;
  fb = esp_camera_fb_get();
  
  if (!fb) {
    Serial.println("failure: Camera capture");
    return;
  }else{
    Serial.println("Success: Camera capture");
  }

  esp_camera_fb_return(fb);
}

[Claessens14]

were you able to resolve this?

[takurot]

Yes, I resolved it.

Changing the I2S port number in the Audio definition will work.

Audio audio(false, 3, 1);

With the library we used, the third argument in the above definition is the I2S port number (set to port 1). Default is port 0.

In the example sketch esp_camera.h, it seems that I2S port 0 is used to control the camera, and if I2S audio output is on port 0, there is a resource conflict. Therefore, it is necessary to change the port number of the I2S audio output. I think it is the same thing if you use another library. Most ESP32s have two I2S ports, but some devices may not. The device I used this time was ESP32-S3.

[Claessens14]

Okay, thank you for the response. would your resulting code look this?

/* I2S Audio Settings */
#define I2S_DOUT     15
#define I2S_BCLK     12
#define I2S_LRC      33

Audio audio(false, 3, 1);

    ...

audio.setPinout(I2S_BCLK, I2S_LRC, I2S_DOUT);

Also are you using the ESP32-S3-WROOM CAM, or a seed studio s3 board, or something else? did you change from the Wrover board?

Result

Your solution worked! Thank you very much!

I am curious to know how you discovered that is was port conflict and that setting i2sPort = 1 for the audio call would fix it?

[takurot]

I'm very happy that your board worked!

I developed a custom board based with ESP32-S3, but it's mostly based on M5 Timer camera F (https://docs.m5stack.com/en/unit/timercam_f).

I was looking into the ESP32 camera library document, and it says it's using I2S port... So I just imagined that I2S port may be conflicted. It was really time consuming to find out!