webdevops/azure-metrics-exporter

By webdevops

Updated 18 days ago

Azure Insights metrics exporter for Prometheus

Image
Integration & Delivery
Monitoring & Observability
Security

1M+

Azure Insights metrics exporter

licenseDockerDocker Build Status

Prometheus exporter for Azure Insights metrics (on demand). Supports metrics fetching from all resource with one scrape (automatic service discovery) and also supports dimensions.

Configuration (except Azure connection) of this exporter is made entirely in Prometheus instead of a seperate configuration file, see examples below.

Configuration

Normally no configuration is needed but can be customized using environment variables.

Usage:
  azure-metrics-exporter [OPTIONS]

Application Options:
      --debug                              debug mode [$DEBUG]
  -v, --verbose                            verbose mode [$VERBOSE]
      --log.json                           Switch log output to json format [$LOG_JSON]
      --azure-environment=                 Azure environment name (default: AZUREPUBLICCLOUD) [$AZURE_ENVIRONMENT]
      --concurrency.subscription=          Concurrent subscription fetches (default: 5) [$CONCURRENCY_SUBSCRIPTION]
      --concurrency.subscription.resource= Concurrent requests per resource (inside subscription requests) (default:
                                           10) [$CONCURRENCY_SUBSCRIPTION_RESOURCE]
      --enable-caching                     Enable internal caching [$ENABLE_CACHING]
      --bind=                              Server address (default: :8080) [$SERVER_BIND]

Help Options:
  -h, --help                               Show this help message

for Azure API authentication (using ENV vars) see https://github.com/Azure/azure-sdk-for-go#authentication

Metrics

MetricDescription
azurerm_stats_metric_collecttimeGeneral exporter stats
azurerm_stats_metric_requestsCounter of resource metric requests with result (error, success)
azurerm_resource_metricResource metrics exported by probes (can be changed using name parameter)
azurerm_loganalytics_query_resultLogAnalytics rows exported by probes

HTTP Endpoints

EndpointDescription
/metricsDefault prometheus golang metrics
/probe/metrics/resourceProbe metrics for one resource (see azurerm_resource_metric)
/probe/metrics/listProbe metrics for list of resources (see azurerm_resource_metric)
/probe/metrics/scrapeProbe metrics for list of resources and config on resource by tag name (see azurerm_resource_metric)
/probe/loganalytics/queryProbe metrics from LogAnalytics query (see azurerm_loganalytics_query_result)

/probe/metrics/resource parameters

GET parameterDefaultRequiredMultipleDescription
subscriptionyesyesAzure Subscription ID
targetyesyesAzure Resource URI
timespanPT1MnonoMetric timespan
intervalnonoMetric timespan
metricnoyesMetric name
aggregationnoyesMetric aggregation (minimum, maximum, average, total, count, multiple possible separated with ,)
nameazurerm_resource_metricnonoPrometheus metric name
metricFilternonoPrometheus metric filter (dimension support)
metricTopnonoPrometheus metric dimension count (dimension support)
metricOrderBynonoPrometheus metric order by (dimension support)
cache(same as timespan)nonoUse of internal metrics caching

Hint: Multiple values can be specified multiple times or with a comma in a single value.

/probe/metrics/list parameters

GET parameterDefaultRequiredMultipleDescription
subscriptionyesyesAzure Subscription ID (or multiple separate by comma)
filteryesnoAzure Resource filter (https://docs.microsoft.com/en-us/rest/api/resources/resources/list)
timespanPT1MnonoMetric timespan
intervalnonoMetric timespan
metricnoyesMetric name
aggregationnoyesMetric aggregation (minimum, maximum, average, total, count, multiple possible separated with ,)
nameazurerm_resource_metricnonoPrometheus metric name
metricFilternonoPrometheus metric filter (dimension support)
metricTopnonoPrometheus metric dimension count (dimension support)
metricOrderBynonoPrometheus metric order by (dimension support)
cache(same as timespan)nonoUse of internal metrics caching

Hint: Multiple values can be specified multiple times or with a comma in a single value.

/probe/metrics/scrape parameters

GET parameterDefaultRequiredMultipleDescription
subscriptionyesyesAzure Subscription ID (or multiple separate by comma)
filteryesnoAzure Resource filter (https://docs.microsoft.com/en-us/rest/api/resources/resources/list)
metricTagNameyesnoResource tag name for getting "metric" list
aggregationTagNameyesnoResource tag name for getting "aggregation" list
timespanPT1MnonoMetric timespan
intervalnonoMetric timespan
metricnoyesMetric name
aggregationnoyesMetric aggregation (minimum, maximum, average, total, multiple possible separated with ,)
nameazurerm_resource_metricnonoPrometheus metric name
metricFilternonoPrometheus metric filter (dimension support)
metricTopnonoPrometheus metric dimension count (integer, dimension support)
metricOrderBynonoPrometheus metric order by (dimension support)
cache(same as timespan)nonoUse of internal metrics caching

Hint: Multiple values can be specified multiple times or with a comma in a single value.

/probe/loganalytics/query parameters

GET parameterDefaultRequiredDescription
workspace yesAzure LogAnalytics workspace ID
queryyesLogAnalytics query
timespanyesQuery timespan

Prometheus configuration

Azure Redis metrics

- job_name: azure-metrics-redis
  scrape_interval: 1m
  metrics_path: /probe/metrics/list
  params:
    name: ["my_own_metric_name"]
    subscription: 
    - xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
    filter: ["resourceType eq 'Microsoft.Cache/Redis'"]
    metric:       
    - connectedclients
    - totalcommandsprocessed
    - cachehits
    - cachemisses
    - getcommands
    - setcommands
    - operationsPerSecond
    - evictedkeys
    - totalkeys
    - expiredkeys
    - usedmemory
    - usedmemorypercentage
    - usedmemoryRss
    - serverLoad
    - cacheWrite
    - cacheRead
    - percentProcessorTime
    - cacheLatency
    - errors
    interval: ["PT1M"]
    timespan: ["PT1M"]
    aggregation:  
    - average
    - total
  static_configs:
  - targets: ["azure-metrics:8080"]

Virtual Gateway metrics

- job_name: azure-metrics-virtualNetworkGateways
  scrape_interval: 1m
  metrics_path: /probe/metrics/list
  params:
    name: ["my_own_metric_name"]
    subscription: 
    - xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
    filter: ["resourceType eq 'Microsoft.Network/virtualNetworkGateways'"]
    metric:       
    - AverageBandwidth
    - P2SBandwidth
    - P2SConnectionCount
    - TunnelAverageBandwidth
    - TunnelEgressBytes
    - TunnelIngressBytes
    - TunnelEgressPackets
    - TunnelIngressPackets
    - TunnelEgressPacketDropTSMismatch
    - TunnelIngressPacketDropTSMismatch
    interval: ["PT5M"]
    timespan: ["PT5M"]
    aggregation:
    - average
    - total
  static_configs:
  - targets: ["azure-metrics:8080"]

Virtual Gateway connection metrics (dimension support)

- job_name: azure-metrics-virtualNetworkGateways-connections
  scrape_interval: 1m
  metrics_path: /probe/metrics/list
  params:
    name: ["my_own_metric_name"]
    subscription:
    - xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
    filter: ["resourceType eq 'Microsoft.Network/virtualNetworkGateways'"]
    metric:
    - TunnelAverageBandwidth
    - TunnelEgressBytes
    - TunnelIngressBytes
    - TunnelEgressPackets
    - TunnelIngressPackets
    - TunnelEgressPacketDropTSMismatch
    - TunnelIngressPacketDropTSMismatch
    interval: ["PT5M"]
    timespan: ["PT5M"]
    aggregation:  
    - average
    - total
    # by connection (dimension support)
    metricFilter: ["ConnectionName eq '*'"]
    metricTop: ["10"]
  static_configs:
  - targets: ["azure-metrics:8080"]

In these examples all metrics are published with metric name my_own_metric_name.

The List of supported metrics is available in the Microsoft Azure docs.

Docker Pull Command

docker pull webdevops/azure-metrics-exporter