bzon/prometheus-msteams
Prometheus Alerts for Microsoft Teams
1M+
bzon/prometheus-msteams repository overview
Overview
A lightweight Go Web Server that receives POST alert messages from Prometheus Alert Manager and sends it to a Microsoft Teams Channel using an incoming webhook url.
Getting Started (Quickstart)
Installation
OPTION 1: Run using docker.
docker run -d -p 2000:2000 \
--name="promteams" \
-e TEAMS_INCOMING_WEBHOOK_URL="https://outlook.office.com/webhook/xxx" \
-e PROMTEAMS_DEBUG="true" \
docker.io/bzon/prometheus-msteams:latest
Setting up Prometheus Alert Manager
By default, prometheus-msteams creates a request uri handler /alertmanager.
route:
group_by: ['alertname']
group_interval: 30s
repeat_interval: 30s
group_wait: 30s
receiver: 'prometheus-msteams'
receivers:
- name: 'prometheus-msteams'
webhook_configs: # https://prometheus.io/docs/alerting/configuration/#webhook_config
- send_resolved: true
url: 'http://localhost:2000/alertmanager' # the prometheus-msteams proxy
If you don't have Prometheus running yet and you wan't to try how this works,
try stefanprodan's Prometheus in Docker to help you install a local Prometheus setup quickly in a single machine.
Simulating a Prometheus Alerts to Teams Channel
Create the following json data as prom-alert.json.
{
"version": "4",
"groupKey": "{}:{alertname=\"high_memory_load\"}",
"status": "firing",
"receiver": "teams_proxy",
"groupLabels": {
"alertname": "high_memory_load"
},
"commonLabels": {
"alertname": "high_memory_load",
"monitor": "master",
"severity": "warning"
},
"commonAnnotations": {
"summary": "Server High Memory usage"
},
"externalURL": "http://docker.for.mac.host.internal:9093",
"alerts": [
{
"labels": {
"alertname": "high_memory_load",
"instance": "10.80.40.11:9100",
"job": "docker_nodes",
"monitor": "master",
"severity": "warning"
},
"annotations": {
"description": "10.80.40.11 reported high memory usage with 23.28%.",
"summary": "Server High Memory usage"
},
"startsAt": "2018-03-07T06:33:21.873077559-05:00",
"endsAt": "0001-01-01T00:00:00Z"
}
]
}
curl -X POST -d @prom-alert.json http://localhost:2000/alertmanager
The teams channel should received a message.
Sending Alerts to Multiple Teams Channel
Creating the Configuration File
Create a prometheus-msteams config file.
connectors:
- high_priority_channel: "https://outlook.office.com/webhook/xxxx/aaa/bbb"
- low_priority_channel: "https://outlook.office.com/webhook/xxxx/aaa/ccc"
NOTE: high_priority_channel and low_priority_channel are example handler names.
When running as a docker container, mount the config file in the container and set the CONFIG_FILE environment variable.
docker run -d -p 2000:2000 \
--name="promteams" \
-v /tmp/config.yml:/tmp/config.yml \
-e CONFIG_FILE="/tmp/config.yml" \
-e PROMTEAMS_DEBUG="true" \
docker.io/bzon/prometheus-msteams:latest
When running as a binary, use the --config flag.
PROMTEAMS_DEBUG="true" prometheus-msteams server \
-l localhost \
-p 2000 \
--config /tmp/config.yml
This will create the request uri handlers /high_priority_channel and /low_priority_channel.
Setting up Prometheus Alert Manager
Considering the prometheus-msteams config file settings, your Alert Manager would have a configuration like the following.
route:
group_by: ['alertname']
group_interval: 30s
repeat_interval: 30s
group_wait: 30s
receiver: 'low_priority_receiver' # default/fallback request handler
routes:
- match:
severity: critical
receiver: high_priority_receiver
- match:
severity: warning
receiver: low_priority_receiver
receivers:
- name: 'high_priority_receiver'
webhook_configs:
- send_resolved: true
url: 'http://localhost:2000/high_priority_channel' # request handler 1
- name: 'low_priority_receiver'
webhook_configs:
- send_resolved: true
url: 'http://localhost:2000/low_priority_channel' # request handler 2
Debugging
For debugging purposes, set the following environment variable PROMTEAMS_DEBUG=true to see the JSON body received from Prometheus and the JSON body created to be sent to Microsoft Teams.
2018/03/19 11:21:23 Request received from Prometheus Alert Manager
{
"version": "4",
"groupKey": "{}:{alertname=\"high_memory_load\"}",
"status": "firing",
"receiver": "teams_proxy",
"groupLabels": {
"alertname": "high_memory_load"
},
"commonLabels": {
"alertname": "high_memory_load",
"monitor": "master",
"severity": "warning"
},
"commonAnnotations": {
"summary": "Server High Memory usage"
},
"externalURL": "http://docker.for.mac.host.internal:9093",
"alerts": [
{
"labels": {
"alertname": "high_memory_load",
"instance": "10.80.40.11:9100",
"job": "docker_nodes",
"monitor": "master",
"severity": "warning"
},
"annotations": {
"description": "10.80.40.11 reported high memory usage with 23.28%.",
"summary": "Server High Memory usage"
},
"startsAt": "2018-03-07T06:33:21.873077559-05:00",
"endsAt": "0001-01-01T00:00:00Z"
}
]
}
2018/03/19 11:21:23 Created a card for Microsoft Teams
{
"@type": "MessageCard",
"@context": "http://schema.org/extensions",
"themeColor": "8C1A1A",
"summary": "Server High Memory usage",
"title": "Prometheus Alert (firing)",
"sections": [
{
"activityTitle": "[10.80.40.11 reported high memory usage with 23.28%.](http://docker.for.mac.host.internal:9093)",
"facts": [
{
"name": "description",
"value": "10.80.40.11 reported high memory usage with 23.28%."
},
{
"name": "summary",
"value": "Server High Memory usage"
},
{
"name": "job",
"value": "docker_nodes"
},
{
"name": "monitor",
"value": "master"
},
{
"name": "severity",
"value": "warning"
},
{
"name": "alertname",
"value": "high_memory_load"
},
{
"name": "instance",
"value": "10.80.40.11:9100"
}
],
"markdown": false
}
]
}
Tag summary
Content type
Image
Digest
Size
13.2 MB
Last updated
over 6 years ago
docker pull bzon/prometheus-msteams