48n6e/mongo-oplog-exporter

By 48n6e

•Updated about 1 month ago

Prometheus exporter for MongoDB oplog & currentOp — slow queries, active ops, write metrics.

Image

Databases & storage

801

Overview Tags

48n6e/mongo-oplog-exporter repository overview

⁠简介

GitHub: https://github.com/48N6E/mongo-oplog-exporter⁠

MongoDB oplog 与 currentOp 的 Prometheus 导出器，通过 GET /metrics 暴露指标。

慢查询：currentOp 采集 secs_running、command、planSummary
当前运行语句：周期性 currentOp，含 query、client、appName
写入热点 / 审计：实时 tail oplog，按 op、ns 统计写入
告警：指标 log_to_metric_mongo_currentop、log_to_metric_mongo_oplog

⁠What it solves

Need	How
Slow queries	`currentOp` → `secs_running`, `command`, `planSummary`
Active statements	Poll `currentOp` every N seconds → `query`, `client`, `appName`
Write audit / hot collections	Tail oplog → `op`, `ns`, write rate per collection
Alerts in Prometheus/Grafana	Counters `log_to_metric_mongo_currentop` / `log_to_metric_mongo_oplog`

⁠Prerequisites

MongoDB
Dedicated read-only MongoDB user:

Role	Database	Purpose
`clusterMonitor`	`admin`	Run `currentOp`
`read`	`local`	Read `local.oplog.rs`

use admin
db.createUser({
  user: "exporter",
  pwd: "change-me",
  roles: [
    { role: "clusterMonitor", db: "admin" },
    { role: "read", db: "local" }
  ]
})

Connection URI (authSource=admin required):

mongodb://exporter:[email protected]:27017/?authSource=admin

⁠Quick start

docker run -d \
  --name mongo-oplog-exporter \
  --restart unless-stopped \
  -p 7777:7777 \
  -e MONGO_URL="mongodb://exporter:[email protected]:27017/?authSource=admin" \
  -e MONGO_DIRECT_CONNECTION="true" \
  -e PORT="7777" \
  -e OPEN_OPLOG="true" \
  -e OPEN_CURRENTOP="true" \
  -e OPLOG_IGNORE_OP='["n"]' \
  -e OPLOG_IGNORE_FIELD='["lsid","ui","prevOpTime","stmtId","ts","txnNumber","wall","o2","o","postImageOpTime","h","fromMigrate"]' \
  -e OPENLOG_OPLOG="false" \
  -e OPLOG_METRICS_TYPE="counter" \
  -e OPLOG_INIT_LABEL='["op","ns","node_name","service_name","t","cluster","v","o2","lsid","ui","prevOpTime","stmtId","ts","txnNumber","wall","preImageOpTime","writeConflicts","transaction","fromMigrate"]' \
  -e OPLOG_METRICS_NAME="log_to_metric_mongo_oplog" \
  -e CURRENTOP_IGNORE_OP='["none"]' \
  -e CURRENTOP_IGNORE_FIELD='["lsid","waitingForFlowControl","flowControlStats","numYields","currentOpTime","clientMetadata","command.$clusterTime","locks","waitingForLock","lockStats","threadId"]' \
  -e OPENLOG_CURRENTOP="false" \
  -e CURRENTOP_METRICS_TYPE="counter" \
  -e CURRENTOP_INIT_LABEL='["command","opid","v","active","t","connectionId","desc","host","type","exporter_type","effectiveUsers","client","clientMetadata","secs_running","microsecs_running","waitingForLatch","appName","planSummary","cursor","preImageOpTime","writeConflicts","transaction","postImageOpTime","originatingCommand","query","fromMigrate"]' \
  -e CURRENTOP_METRICS_NAME="log_to_metric_mongo_currentop" \
  -e CURRENTOP_IGNORE_KEYVALUE='[{"command":"hello"},{"command":"currentOp"},{"command":"isMaster"},{"appName":"OplogFetcher"},{"ns":"admin"},{"ns":"local"},{"appName":"QAN"}]' \
  -e CURRENTOP_INTERVAL="20000" \
  48n6e/mongo-oplog-exporter:latest

Verify:

curl http://localhost:7777/metrics

⁠Production example

docker run -d \
  --name mongo-oplog-exporter \
  --restart unless-stopped \
  -p 7777:7777 \
  -e MONGO_URL="mongodb://exporter:[email protected]:27017/?authSource=admin" \
  -e MONGO_DIRECT_CONNECTION="true" \
  -e PORT="7777" \
  -e OPEN_OPLOG="true" \
  -e OPEN_CURRENTOP="true" \
  -e OPLOG_IGNORE_OP='["n"]' \
  -e OPLOG_IGNORE_FIELD='["lsid","ui","prevOpTime","stmtId","ts","txnNumber","wall","o2","o","postImageOpTime","h","fromMigrate"]' \
  -e OPENLOG_OPLOG="false" \
  -e OPLOG_METRICS_TYPE="counter" \
  -e OPLOG_INIT_LABEL='["op","ns","node_name","service_name","t","cluster","v","o2","lsid","ui","prevOpTime","stmtId","ts","txnNumber","wall","preImageOpTime","writeConflicts","transaction","fromMigrate"]' \
  -e OPLOG_METRICS_NAME="log_to_metric_mongo_oplog" \
  -e CURRENTOP_IGNORE_OP='["none"]' \
  -e CURRENTOP_IGNORE_FIELD='["lsid","waitingForFlowControl","flowControlStats","numYields","currentOpTime","clientMetadata","command.$clusterTime","locks","waitingForLock","lockStats","threadId"]' \
  -e OPENLOG_CURRENTOP="false" \
  -e CURRENTOP_METRICS_TYPE="counter" \
  -e CURRENTOP_INIT_LABEL='["command","opid","v","active","t","connectionId","desc","host","type","exporter_type","effectiveUsers","client","clientMetadata","secs_running","microsecs_running","waitingForLatch","appName","planSummary","cursor","preImageOpTime","writeConflicts","transaction","postImageOpTime","originatingCommand","query","fromMigrate"]' \
  -e CURRENTOP_METRICS_NAME="log_to_metric_mongo_currentop" \
  -e CURRENTOP_IGNORE_KEYVALUE='[{"command":"hello"},{"command":"currentOp"},{"command":"isMaster"},{"appName":"OplogFetcher"},{"ns":"admin"},{"ns":"local"},{"appName":"QAN"}]' \
  -e CURRENTOP_INTERVAL="20000" \
  --memory=1000m \
  --cpus=1 \
  48n6e/mongo-oplog-exporter:latest \
  node --trace-warnings --gc-interval=1000 --max-old-space-size=2048 app.js

⁠Common environment variables

Variable	Description	Default
`MONGO_URL`	MongoDB connection URI	`mongodb://localhost:27017`
`MONGO_DIRECT_CONNECTION`	Direct single-node connection	`true`
`PORT`	Metrics HTTP port	`7777`
`OPEN_OPLOG`	Enable oplog tailing	`true`
`OPEN_CURRENTOP`	Enable currentOp polling	`false`
`CURRENTOP_INTERVAL`	currentOp interval (ms)	`20000`
`OPLOG_IGNORE_OP`	Ignored oplog op types	`["n"]`
`OPLOG_IGNORE_FIELD`	Oplog fields stripped before labeling	see `docker run` example
`OPLOG_INIT_LABEL`	Pre-declared oplog counter labels	see `docker run` example
`OPLOG_METRICS_NAME`	Oplog metric name	`log_to_metric_mongo_oplog`
`CURRENTOP_IGNORE_OP`	Ignored currentOp op types	`["none"]`
`CURRENTOP_IGNORE_FIELD`	currentOp fields stripped before labeling	see `docker run` example
`CURRENTOP_IGNORE_KEYVALUE`	Filter currentOp by key-value pairs	see `docker run` example
`CURRENTOP_INIT_LABEL`	Pre-declared currentOp counter labels	see `docker run` example
`CURRENTOP_METRICS_NAME`	currentOp metric name	`log_to_metric_mongo_currentop`

Environment variables override config.json. JSON arrays (e.g. OPLOG_IGNORE_OP) use format '["n"]'.

⁠Metrics

Name	Type	Source
`log_to_metric_mongo_oplog`	counter	oplog document fields as labels
`log_to_metric_mongo_currentop`	counter	each in-flight operation as labels

The registry resets after each Prometheus scrape (window-counter behavior).

⁠Prometheus scrape config

scrape_configs:
  - job_name: mongo-oplog-exporter
    static_configs:
      - targets: ['localhost:7777']

⁠License

GPL-3.0-or-later

Tag summary

Recent tags

Content type

Image

Digest

sha256:56cec34c7…

Size

75.5 MB

Last updated

about 1 month ago

docker pull 48n6e/mongo-oplog-exporter