This is a collection of studies about monitoring, aka Prometheus + Grafana + Alertmanager.
There is a sample nodejs app getting application metrics.
usage:
docker-compose up -d
To monitor Linux hosts, install node_exports on target.
Download desired version:
wget https://github.com/prometheus/node_exporter/releases/download/v1.1.2/node_exporter-1.1.2.linux-amd64.tar.gz
Untar and Run
tar -xvfz node_exporter-1.1.2.linux-amd64.tar.gz
cd node_exporter-1.1.2.linux-amd64.tar.gz
./node_exporter &
Import the alert dashboard located in folder ./grafana-data/dashboards/
There is some fake Linux hosts on prometheus.yaml to generate service_down alerts.
- job_name: node
honor_timestamps: true
scrape_interval: 30s
scrape_timeout: 10s
metrics_path: /metrics
scheme: http
static_configs:
- targets:
- 192.168.0.18:9100
- 192.168.0.19:9100
- 192.168.0.20:9100
- 192.168.0.21:9100
When you add node_exporter to some server, just add in this section.
To setup Pagerdutty [https://pagerduty.com/] integration, you need to create a free tier account and generate an API Token.
In alertmanager.yaml, set a receiver to pagerduty:
receivers:
- name: 'team-X-pager'
pagerduty_configs:
- service_key: APIKEY
More information in Pagerduty Documentation :[https://www.pagerduty.com/docs/guides/prometheus-integration-guide/]