This project represents monitoring loan retail system in near real time.

It uses:

• Clickhouse as main OLAP db
• Grafana with CH connector plugin for monitoring
• Kafka for reading messages
• dbt for transformations and version control
• mimesis lib on Python for generate synth data

For building project with every tool:

docker compose up -d

Dash in grafana with info about loan requests:

---

TODO:

1. Add Flink for ELT instead of Kafka Engine
2. Add command to Makefile
3. Add logging
4. Make params customizable (message frequency)