1- < img src = " arduino_ai_speech_assets/elato-alien.png " alt = " Elato Logo " width = " 100% " >
1+ ![ Elato Logo ] ( https://raw.githubusercontent.com/openai/openai-cookbook/refs/heads/main/examples/voice_solutions/ arduino_ai_speech_assets/elato-alien.png)
22
3- # 👾 ElatoAI: Running OpenAI Realtime API Speech on ESP32 on Arduino with Deno Edge Functions
3+ ## 👾 ElatoAI: Running OpenAI Realtime API Speech on ESP32 on Arduino with Deno Edge Functions
44
55This guide shows how to build a AI voice agent device with Realtime AI Speech powered by OpenAI Realtime API, ESP32, Secure WebSockets, and Deno Edge Functions for >10-minute uninterrupted global conversations.
66
77An active version of this README is available at [ ElatoAI] ( https://github.com/akdeb/ElatoAI ) .
88
99<div align =" center " >
10-
11- [ ![ Discord Follow] ( https://dcbadge.vercel.app/api/server/KJWxDPBRUj?style=flat )] ( https://discord.gg/KJWxDPBRUj )
12- [ ![ License: MIT] ( https://img.shields.io/badge/license-MIT-blue )] ( https://www.gnu.org/licenses/gpl-3.0.en.html ) &ensp ;&ensp ;&ensp ;
13- ![ Node.js] ( https://img.shields.io/badge/Node.js-22.13.0-yellow.svg )
14- ![ Next.js] ( https://img.shields.io/badge/Next.js-14.2.7-brightgreen.svg )
15- ![ React] ( https://img.shields.io/badge/React-18.2.0-blue.svg )
16-
17- </div >
18-
19- ## Demo Video
20-
21- https://github.com/user-attachments/assets/aa60e54c-5847-4a68-80b5-5d6b1a5b9328
22-
23- <div align =" center " >
24- <a href =" https://www.youtube.com/watch?v=o1eIAwVll5I " >
25- <img src =" https://img.shields.io/badge/Watch%20Demo-YouTube-red?style=for-the-badge&logo=youtube " alt =" Watch Demo on YouTube " >
26- </a >
10+ <a href="https://www.youtube.com/watch?v=o1eIAwVll5I" target="_blank">
11+ <img src="https://raw.githubusercontent.com/akdeb/ElatoAI/refs/heads/main/assets/thumbnail.png" alt="Elato AI Demo Video" width="100%" style="border-radius:10px" />
12+ </a >
2713</div >
2814
29- ## Hardware Design
15+ ## ⚡️ DIY Hardware Design
3016
3117The reference implementation uses an ESP32-S3 microcontroller with minimal additional components:
3218
33- <img src =" arduino_ai_speech_assets/pcb-design.png " alt =" Hardware Setup " width =" 100% " >
19+ <img src =" https://raw.githubusercontent.com/openai/openai-cookbook/refs/heads/main/examples/voice_solutions/ arduino_ai_speech_assets/pcb-design.png" alt =" Hardware Setup " width =" 100% " >
3420
3521** Required Components:**
3622- ESP32-S3 development board
@@ -40,13 +26,23 @@ The reference implementation uses an ESP32-S3 microcontroller with minimal addit
4026- RGB LED for visual feedback
4127- Optional: touch sensor for alternative control
4228
43- ** Optional hardware :**
29+ ** Hardware options :**
4430A fully assembled PCB and device is available in the [ ElatoAI store] ( https://www.elatoai.com/products ) .
4531
46- ## 🚀 Quick Start Guide
32+ ## 📱 App Design
33+
34+ Control your ESP32 AI device from your phone with your own webapp.
35+
36+ <img src =" https://raw.githubusercontent.com/openai/openai-cookbook/refs/heads/main/examples/voice_solutions/arduino_ai_speech_assets/mockups.png " alt =" App Screenshots " width =" 100% " >
37+
38+ | Select from a list of AI characters | Talk to your AI with real-time responses | Create personalized AI characters |
39+ | :--:| :--:| :--:|
40+
41+
42+ ## ✨ Quick Start Tutorial
4743
4844<a href =" https://www.youtube.com/watch?v=bXrNRpGOJWw " >
49- <img src =" https://img.shields.io/badge/Quickstart %20Tutorial-YouTube-yellow?style=for-the-badge&logo=youtube " alt =" Watch Demo on YouTube " >
45+ <img src =" https://img.shields.io/badge/Quick%20start %20Tutorial-YouTube-yellow?style=for-the-badge&logo=youtube " alt =" Watch Demo on YouTube " >
5046</a >
5147
52481 . ** Clone the repository**
@@ -192,35 +188,13 @@ We have a [Usecases.md](https://github.com/akdeb/ElatoAI/tree/main/Usecases.md)
192188
193189## 🗺️ High-Level Flow
194190
195- ``` mermaid
196- flowchart TD
197- User[User Speech] --> ESP32
198- ESP32[ESP32 Device] -->|WebSocket| Edge[Deno Edge Function]
199- Edge -->|OpenAI API| OpenAI[OpenAI Realtime API]
200- OpenAI --> Edge
201- Edge -->|WebSocket| ESP32
202- ESP32 --> User[AI Generated Speech]
203- ```
191+ <img src =" https://raw.githubusercontent.com/openai/openai-cookbook/refs/heads/main/examples/voice_solutions/arduino_ai_speech_assets/flowchart.png " alt =" App Screenshots " width =" 100% " >
204192
205193## Project Structure
206194
207- ``` mermaid
208- graph TD
209- repo[ElatoAI]
210- repo --> frontend[Frontend Vercel NextJS]
211- repo --> deno[Deno Edge Function]
212- repo --> esp32[ESP32 Arduino Client]
213- deno --> supabase[Supabase DB]
214-
215- frontend --> supabase
216- esp32 --> websockets[Secure WebSockets]
217- esp32 --> opus[Opus Codec]
218- esp32 --> audio_tools[arduino-audio-tools]
219- esp32 --> libopus[arduino-libopus]
220- esp32 --> ESPAsyncWebServer[ESPAsyncWebServer]
221- ```
195+ <img src =" https://raw.githubusercontent.com/openai/openai-cookbook/refs/heads/main/examples/voice_solutions/arduino_ai_speech_assets/structure.png " alt =" App Screenshots " width =" 100% " >
222196
223- ## ⚙️ PlatformIO Configuration
197+ ## ⚙️ PlatformIO Config
224198
225199``` ini
226200[env:esp32-s3-devkitc-1]
@@ -264,4 +238,4 @@ This project is licensed under the MIT License - see the [LICENSE](LICENSE) file
264238
265239---
266240
267- ** If you find this project interesting or useful, drop a GitHub ⭐️ at [ ElatoAI] ( https://github.com/akdeb/ElatoAI ) . It helps a lot !**
241+ ** This example is part of the [ OpenAI Cookbook ] ( https://github.com/openai/openai-cookbook ) . For the full project and latest updates, check out [ ElatoAI] ( https://github.com/akdeb/ElatoAI ) and consider giving it a ⭐️ if you find it useful !**
0 commit comments