🧪 ELK Playground: Simulated App + Log Pipeline

Welcome to the ELK Playground — a hands-on environment for learning how to use the Elastic Stack (Elasticsearch, Logstash, Kibana) with a simulated Python application that generates logs and mimics production incidents.

---

🚀 What’s Inside

• Python Simulation App
  Generates structured logs with varying severity (INFO, WARN, ERROR, CRITICAL) to mimic real-world service behavior and failures.

• Logstash Pipeline
  Parses the app logs and ships them to Elasticsearch with field extraction and timestamp handling.

• Elasticsearch
  Stores and indexes logs for fast search and analysis.

• Kibana Dashboard
  Visualizes logs, error rates, service health, and incident timelines.

---

📦 Getting Started

1. Clone the Repo

git clone https://github.com/your-username/elk-playground.git
cd elk-playground

2. Run the Stack

Make sure Docker and Docker Compose are installed.

docker-compose up

This will start:

• Elasticsearch on localhost:9200
• Logstash on localhost:5044
• Kibana on localhost:5601

3. Generate Logs

Run the simulation app to produce logs:

python simulate_app.py

This writes logs to app_simulation.log, which Logstash ingests and forwards to Elasticsearch.

---

📊 Visualize in Kibana

1. Open http://localhost:5601
2. Go to Stack Management → Data Views
3. Create a data view for app-logs
4. Explore logs in Discover
5. Build dashboards to visualize:
   • CPU and memory usage
   • Error spikes
   • Service health
   • Incident timeline

---

🧠 Learn & Experiment

This repo is a sandbox — feel free to:

• Modify the simulation app to generate different log formats
• Add new Logstash filters
• Create custom Kibana visualizations
• Integrate alerts or Slack notifications

---

📁 File Structure

elk-playground/
├── docker-compose.yml
├── logstash.conf
├── app_simulation.log
├── simulate_app.py
└── README.md

---

🛠 Requirements

• Docker + Docker Compose
• Python 3.8+
• Kibana, Elasticsearch, Logstash (via Docker)