Trigger an alert when metric reaches predefined treshold

[ateska]

The user should be able to configure an alert on any metrics (counter?) so that if the metric reaches that threshold, alert is triggered.

Example of the alert definition from config:

[metric:alert:my-alert]
metric_name=web_requests_duration_min
metric_tags=
    method=GET
    status=200
    path=*
threshold=>0.5
title="Alert title"

The alert should be installed to the metrics at

asab/asab/metrics/service.py

Line 109 in d6007d4

metric._initialize_storage(

Questions:

• Will this work for dynamic tags?
• Alert is specific to a metrics type, so it is an internal logic of the each metric class how to implement threshold detection
• Alerts should be also "untriggered" (aka clear).