🔍 OpenLLM Monitor – Real-Time Observability Dashboard for LLM APIs (OpenAI, Ollama, OpenRouter)

[prajeesh-chavan]

Hey everyone 👋

I just built and open-sourced OpenLLM Monitor, a plug-and-play dashboard that gives developers full visibility into how their LLM prompts are performing — cost, latency, token usage, retries, and more.

🔧 What It Does

• Logs every request to OpenAI, Ollama, OpenRouter (more coming)
• Tracks latency, retry count, and error rates
• Breaks down token usage (prompt vs. completion)
• Estimates cost per request in real-time
• Lets you replay prompts and switch providers for debugging
• Real-time log view with charts and filters

🌐 Built with

Node.js · Express · MongoDB · React · Tailwind · WebSockets

🧠 Why I Built It

I found myself debugging LLM completions with zero visibility — no logs, no token breakdown, no replay. This tool solves that pain, especially for indie devs and teams building GenAI apps.

🔗 GitHub:

github.com/prajeesh-chavan/openllm-monitor

---

Would love to hear:

• How you monitor your LLM usage today?
• What features you'd want in a tool like this?
• Is it useful enough to plug into your workflow?

Happy to collaborate if you're building something in this space!

– Prajeesh Chavan 🚀