Sign inSign up

gusgh13900/api-discovery-agent

By gusgh13900

โ€ขUpdated about 1 month ago

Log-collecting agent for API Discovery for NGINX โ€” ships NGINX access logs to the dashboard

Image
Networking
Monitoring & observability
0

213

gusgh13900/api-discovery-agent repository overview

โ API Discovery for NGINX โ€” Agent

๐Ÿ‡ฐ๐Ÿ‡ท ํ•œ๊ตญ์–ด ๋ฌธ์„œ๋Š” ์ด ํŽ˜์ด์ง€ ์•„๋ž˜์ชฝ์— ์žˆ์Šต๋‹ˆ๋‹ค. ยท Korean documentation follows below.

The log collector for API Discovery for NGINX. It runs on your NGINX host, tails the access log (and optionally the NGINX Plus REST API), and ships records to the dashboard.

๐Ÿ”— Dashboard image: gusgh13900/api-discovery-nginxโ  โ€” collector server + web UI (:8080). Keep it on the same version tag as this image.

ImageRoleDownloadOn diskRuns on
api-discovery-agent (this image)NGINX log collector~21 MB~56 MBNGINX host
api-discovery-nginxCollector server + web dashboard (:8080)~288 MB~1 GBDashboard host
  • Source: open source under Apache-2.0 โ€” github.com/VEEP09/api-discovery-for-nginxโ 
  • Architecture: linux/amd64 (no ARM build)
  • Configuration: entirely through environment variables. Secrets are never baked into the image.
  • Footprint: distroless, no shell, single Python dependency. It reads your logs read-only and sends them onward โ€” nothing else.

โ Quick start

Start the dashboardโ  first, then run this on your NGINX host:

docker run -d --name api-discovery-agent \
  --network host \
  -e DASHBOARD_URL=http://<dashboard-host>:8080 \
  -e INGEST_TOKEN=<same token as the dashboard> \
  -e LOG_PATH=/var/log/nginx/api_access.log \
  -e CURSOR_FILE=/agent/state/agent_cursor.json \
  -e NGINX_PLUS_API=http://127.0.0.1:411/api/9 \
  -v /var/log/nginx:/var/log/nginx:ro \
  -v $PWD/state:/agent/state \
  gusgh13900/api-discovery-agent:1.2.1

INGEST_TOKEN must be identical on both sides. A mismatch is the usual reason an agent starts cleanly, reports HTTP 401 in its log, and never appears in the dashboard. Leaving it empty disables authentication entirely โ€” local use only.

Drop --network host if the dashboard is remote and you are not collecting NGINX Plus metrics.


โ Environment variables

VariableRequiredDefaultDescription
DASHBOARD_URLโœ…โ€”Dashboard address, e.g. http://10.0.0.5:8080
INGEST_TOKEN(empty)Must match the dashboard
LOG_PATH/var/log/nginx/api_access.logActive log file, as seen inside the container
LOG_ROTATE_DIR/var/log/nginx/oldlogrotate archive directory (empty to skip)
LOG_ROTATE_PATTERNapi_access.log-*.gzGlob for rotated files (must contain an 8-digit date)
CURSOR_FILE/agent/state/agent_cursor.jsonRead position, so restarts don't resend. Put this on a volume.
NGINX_PLUS_API(empty)NGINX Plus REST API URL (empty to skip)
POLL_INTERVAL30Seconds between polls
BATCH_SIZE5000Maximum rows per request
RETAIN_DAYS7Skip rotated logs older than this
SKIP_ERROR_RESPONSESfalseExclude 4xx/5xx responses
AGENT_IDhostnameIdentifier shown in the dashboard
  • Volumes: /var/log/nginx (read-only), /agent/state (cursor persistence)
  • Log format: JSON, one object per line. The log_format block to copy is on the dashboard image pageโ .

โ docker-compose

curl -O https://raw.githubusercontent.com/VEEP09/api-discovery-for-nginx/main/docker-compose.agent.yml
DASHBOARD_URL=http://<dashboard-host>:8080 INGEST_TOKEN=<token> \
  docker compose -f docker-compose.agent.yml up -d

Run this on your NGINX host, not on the dashboard host.


โ Versions

TagDescription
latestNewest stable release (currently 1.2.1)
1.2.1No source changes; released alongside the dashboard. Release notesโ 
1.2.0English log output and --help. Release notesโ 
1.1.0Base image refresh, no source changes. Release notesโ 
1.0.0Initial release

Pin a version tag in production rather than using latest, and keep the dashboard and agent on the same tag. Sizes above are compressed download sizes.



โ ํ•œ๊ตญ์–ด

โ API Discovery for NGINX โ€” ์—์ด์ „ํŠธ

API Discovery for NGINX ์˜ ๋กœ๊ทธ ์ˆ˜์ง‘ ์—์ด์ „ํŠธ์ž…๋‹ˆ๋‹ค. NGINX ์„œ๋ฒ„์—์„œ ์‹คํ–‰๋˜๋ฉฐ ์•ก์„ธ์Šค ๋กœ๊ทธ(๋ฐ ์„ ํƒ์ ์œผ๋กœ NGINX Plus REST API ์ง€ํ‘œ)๋ฅผ ์ฝ์–ด ๋Œ€์‹œ๋ณด๋“œ๋กœ ์ „์†กํ•ฉ๋‹ˆ๋‹ค.

๐Ÿ”— ๋Œ€์‹œ๋ณด๋“œ ์ด๋ฏธ์ง€: gusgh13900/api-discovery-nginxโ  โ€” ์ˆ˜์ง‘ ์„œ๋ฒ„ + ์›น UI(:8080). ์ด ์ด๋ฏธ์ง€์™€ ๋™์ผํ•œ ๋ฒ„์ „ ํƒœ๊ทธ๋กœ ๋งž์ถฐ ์‚ฌ์šฉํ•˜์„ธ์š”.

์ด๋ฏธ์ง€์—ญํ• ๋‹ค์šด๋กœ๋“œ๋””์Šคํฌ์‹คํ–‰ ์œ„์น˜
api-discovery-agent (์ด ์ด๋ฏธ์ง€)NGINX ๋กœ๊ทธ ์ˆ˜์ง‘ ์—์ด์ „ํŠธ~21 MB~56 MBNGINX ์„œ๋ฒ„
api-discovery-nginx์ˆ˜์ง‘ ์„œ๋ฒ„ + ์›น ๋Œ€์‹œ๋ณด๋“œ (:8080)~288 MB~1 GB๋Œ€์‹œ๋ณด๋“œ ์„œ๋ฒ„
  • ์†Œ์Šค: Apache-2.0 ์˜คํ”ˆ์†Œ์Šค โ€” github.com/VEEP09/api-discovery-for-nginxโ 
  • ์•„ํ‚คํ…์ฒ˜: linux/amd64 (ARM ๋ฏธ์ง€์›)
  • ์„ค์ •: ์ „๋ถ€ ํ™˜๊ฒฝ๋ณ€์ˆ˜๋กœ ์ฃผ์ž…ํ•ฉ๋‹ˆ๋‹ค. ๋น„๋ฐ€๊ฐ’์€ ์ด๋ฏธ์ง€์— ํฌํ•จ๋˜์ง€ ์•Š์Šต๋‹ˆ๋‹ค.
  • ๊ตฌ์„ฑ: distroless ์ด๋ฏธ์ง€๋กœ ์…ธ์ด ์—†๊ณ  Python ์˜์กด์„ฑ์€ ํ•˜๋‚˜๋ฟ์ž…๋‹ˆ๋‹ค. ๋กœ๊ทธ๋ฅผ ์ฝ๊ธฐ ์ „์šฉ์œผ๋กœ ์ฝ์–ด ์ „์†กํ•˜๋Š” ๊ฒƒ ์™ธ์—๋Š” ์•„๋ฌด๊ฒƒ๋„ ํ•˜์ง€ ์•Š์Šต๋‹ˆ๋‹ค.
โ ๋น ๋ฅธ ์‹œ์ž‘

๋จผ์ € ๋Œ€์‹œ๋ณด๋“œโ ๋ฅผ ๋„์šด ๋’ค, NGINX ์„œ๋ฒ„์—์„œ ์‹คํ–‰ํ•ฉ๋‹ˆ๋‹ค.

docker run -d --name api-discovery-agent \
  --network host \
  -e DASHBOARD_URL=http://<๋Œ€์‹œ๋ณด๋“œ-์„œ๋ฒ„>:8080 \
  -e INGEST_TOKEN=<๋Œ€์‹œ๋ณด๋“œ์™€ ๋™์ผํ•œ ํ† ํฐ> \
  -e LOG_PATH=/var/log/nginx/api_access.log \
  -e CURSOR_FILE=/agent/state/agent_cursor.json \
  -e NGINX_PLUS_API=http://127.0.0.1:411/api/9 \
  -v /var/log/nginx:/var/log/nginx:ro \
  -v $PWD/state:/agent/state \
  gusgh13900/api-discovery-agent:1.2.1

INGEST_TOKEN ์€ ์–‘์ชฝ์ด ๋ฐ˜๋“œ์‹œ ๋™์ผํ•ด์•ผ ํ•ฉ๋‹ˆ๋‹ค. ๊ฐ’์ด ๋‹ค๋ฅด๋ฉด ์—์ด์ „ํŠธ๋Š” ์ •์ƒ ๊ธฐ๋™ํ•˜๊ณ  ๋กœ๊ทธ์— HTTP 401 ๋งŒ ๋ฐ˜๋ณตํ•ด์„œ ์ฐ์œผ๋ฉฐ ๋Œ€์‹œ๋ณด๋“œ์—๋Š” ๋๋‚ด ๋‚˜ํƒ€๋‚˜์ง€ ์•Š์Šต๋‹ˆ๋‹ค. ๋น„์›Œ๋‘๋ฉด ์ธ์ฆ์ด ์™„์ „ํžˆ ๋น„ํ™œ์„ฑํ™”๋˜๋ฏ€๋กœ ๋กœ์ปฌ ์ „์šฉ์œผ๋กœ๋งŒ ์“ฐ์„ธ์š”.

๋Œ€์‹œ๋ณด๋“œ๊ฐ€ ์›๊ฒฉ์ด๊ณ  NGINX Plus ๋ฉ”ํŠธ๋ฆญ์„ ์ˆ˜์ง‘ํ•˜์ง€ ์•Š๋Š”๋‹ค๋ฉด --network host ๋Š” ๋นผ๋„ ๋ฉ๋‹ˆ๋‹ค.

โ ํ™˜๊ฒฝ๋ณ€์ˆ˜
๋ณ€์ˆ˜ํ•„์ˆ˜๊ธฐ๋ณธ๊ฐ’์„ค๋ช…
DASHBOARD_URLโœ…โ€”๋Œ€์‹œ๋ณด๋“œ ์ฃผ์†Œ (์˜ˆ: http://10.0.0.5:8080)
INGEST_TOKEN(๋นˆ ๊ฐ’)๋Œ€์‹œ๋ณด๋“œ์™€ ๋™์ผํ•ด์•ผ ํ•จ
LOG_PATH/var/log/nginx/api_access.logํ™œ์„ฑ ๋กœ๊ทธ ํŒŒ์ผ(์ปจํ…Œ์ด๋„ˆ ๋‚ด๋ถ€ ๊ฒฝ๋กœ)
LOG_ROTATE_DIR/var/log/nginx/oldlogrotate ๋ณด๊ด€ ๋””๋ ‰ํ† ๋ฆฌ (๋นˆ ๊ฐ’์ด๋ฉด ๋ฏธ์ˆ˜์ง‘)
LOG_ROTATE_PATTERNapi_access.log-*.gzrotated ํŒŒ์ผ glob (๋‚ ์งœ 8์ž๋ฆฌ ํฌํ•จ ํ•„์š”)
CURSOR_FILE/agent/state/agent_cursor.json์ฝ๊ธฐ ์œ„์น˜ ์ €์žฅ โ€” ์žฌ์‹œ์ž‘ ์‹œ ์ค‘๋ณต ์ „์†ก ๋ฐฉ์ง€. ๋ณผ๋ฅจ ๊ฒฝ๋กœ๋กœ ์ง€์ •ํ•˜์„ธ์š”.
NGINX_PLUS_API(๋นˆ ๊ฐ’)NGINX Plus REST API URL (๋นˆ ๊ฐ’์ด๋ฉด ๋ฏธ์ˆ˜์ง‘)
POLL_INTERVAL30ํด๋ง ์ฃผ๊ธฐ(์ดˆ)
BATCH_SIZE50001ํšŒ ์ „์†ก ์ตœ๋Œ€ ํ–‰ ์ˆ˜
RETAIN_DAYS7์ด ๊ธฐ๊ฐ„๋ณด๋‹ค ์˜ค๋ž˜๋œ rotated ๋กœ๊ทธ๋Š” ๊ฑด๋„ˆ๋œ€
SKIP_ERROR_RESPONSESfalse4xx/5xx ์‘๋‹ต ์ œ์™ธ ์—ฌ๋ถ€
AGENT_IDํ˜ธ์ŠคํŠธ๋ช…๋Œ€์‹œ๋ณด๋“œ์— ํ‘œ์‹œ๋  ์‹๋ณ„์ž
โ docker-compose
curl -O https://raw.githubusercontent.com/VEEP09/api-discovery-for-nginx/main/docker-compose.agent.yml
DASHBOARD_URL=http://<๋Œ€์‹œ๋ณด๋“œ-์„œ๋ฒ„>:8080 INGEST_TOKEN=<ํ† ํฐ> \
  docker compose -f docker-compose.agent.yml up -d

๋Œ€์‹œ๋ณด๋“œ ์„œ๋ฒ„๊ฐ€ ์•„๋‹ˆ๋ผ NGINX ์„œ๋ฒ„์—์„œ ์‹คํ–‰ํ•˜์„ธ์š”.

โ ๋ฒ„์ „
ํƒœ๊ทธ์„ค๋ช…
latest์ตœ์‹  ์•ˆ์ • ๋ฒ„์ „ (ํ˜„์žฌ 1.2.1)
1.2.1์†Œ์Šค ๋ณ€๊ฒฝ ์—†์Œ, ๋Œ€์‹œ๋ณด๋“œ์™€ ํ•จ๊ป˜ ๋ฐฐํฌ. ๋ฆด๋ฆฌ์Šค ๋…ธํŠธโ 
1.2.0๋กœ๊ทธ ์ถœ๋ ฅยท--help ์˜๋ฌธํ™”. ๋ฆด๋ฆฌ์Šค ๋…ธํŠธโ 
1.1.0๋ฒ ์ด์Šค ์ด๋ฏธ์ง€ ๊ฐฑ์‹ , ์†Œ์Šค ๋ณ€๊ฒฝ ์—†์Œ. ๋ฆด๋ฆฌ์Šค ๋…ธํŠธโ 
1.0.0์ตœ์ดˆ ๋ฆด๋ฆฌ์Šค

ํ”„๋กœ๋•์…˜์—์„œ๋Š” latest ๋Œ€์‹  ๊ณ ์ • ๋ฒ„์ „ ํƒœ๊ทธ๋ฅผ ์“ฐ๊ณ , ๋Œ€์‹œ๋ณด๋“œ์™€ ์—์ด์ „ํŠธ๋ฅผ ๊ฐ™์€ ํƒœ๊ทธ๋กœ ๋งž์ถ”์„ธ์š”. ์œ„ ํฌ๊ธฐ๋Š” ์••์ถ•๋œ ๋‹ค์šด๋กœ๋“œ ๊ธฐ์ค€์ž…๋‹ˆ๋‹ค.

โ ๋งํฌ

NGINXยฎ is a registered trademark of F5, Inc. This project is an independent tool and is not affiliated with, endorsed by, or sponsored by F5 or NGINX. "NGINX" is used here only to describe compatibility.

Tag summary

Content type

Image

Digest

sha256:ddc40f3c7โ€ฆ

Size

20.3 MB

Last updated

about 1 month ago

docker pull gusgh13900/api-discovery-agent