jonbirge/logpager repository overview

⁠log-pager

⁠Overview

Lightweight security log forensics and blacklisting web interface, intended to provide a dashboard for threats. Displays log events as heatmap using tile plot, allowing user to click on a given period to drill down into the log. Performs asynchronous geolocation and reverse DNS resolution.

⁠Threat intel and blocking

For each IP in the log, there is a button to pull intel about an IP, including port scans, whois, traceroute and ping graphs. There is also a button to add the IP to a local blocklist.

⁠Approach

The approach is to treat the log file itself as truth and run live UNIX tool commands on the host (within the Docker container) to extract data from the log file directly, essentially running the kinds of local unix forensic commands a sysadmin would. This approach is intended to minimize the impact on the server, with no resources being used except when the web interface is actively being queried.

⁠Potential blacklist service

If enough people adopt this, the next step will be to build a service that would collect blacklisted IPs (if you voluntarily configured your instance of log-pager to forward blacklist items) and publicly provide provide aggregated blacklists based on crowdsourced human judgement, rather than algorithms. If you think this would be interesting, please reach out via the github repo.

⁠Demo

A public demo of the current development branch may (or may not) be running at https://nyc.birgefuller.com/logs/⁠

⁠Screenshots

⁠Default log display

⁠Search display

⁠Intel page

⁠Docker image

This repo automatically builds a Docker image that can be pulled from the GitHub Container Registry. See the Packages tab in the GitHub repo for the latest version.

⁠General usage

Mount the log files of interest as /access.log and /auth.log in the Docker container. Connect to the container on HTTP port 80 and the default interface will be served. There is no security or SSL provided as this is primarily intended as an auxilary container to be integrated with other containers and hosted behind a reverse proxy, such as Traefik. Right now this only work with web and sshd log files.

You can quickly stand up a fully functional demo that uses Traefik for security using the docker-compose.yml file found in /test/stack in the Github repo. You can easily connect this to your auth.log file if you'd like monitor for sshd attacks.

⁠Use with docker-compose

Here is an example docker-compose.yml file showing how to integrate with a reverse proxy (Traefik) to access logs for all proxy traffic. (Obviously, you'll want to have other services as well which I haven't shown here.)

services:

  traefik:
    image: traefik
    restart: always
    command:
      - "--configFile=/etc/traefik.yml"
    ports:
      - "80:80"
      - "8080:8080"  # traefik admin
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - ./traefik.yml:/etc/traefik.yml:ro
      - ./traefik:/etc/traefik
      - ./logs/:/logs/:rw
    depends_on:
      - logpager
      - www

  logpager:
    image: logpager_test
    restart: always
    environment:
      SQL_HOST: db
      SQL_PASS: testpass
      SQL_USER: root
    labels:
      - "traefik.http.routers.logpagerdev.rule=PathPrefix(`/logs`)"
      - "traefik.http.middlewares.striplogdev.stripprefix.prefixes=/logs/"
      - "traefik.http.routers.logpagerdev.middlewares=striplogdev"
    volumes:
      - /var/logs/auth.log:/auth.log:ro
      - ./logs/access.log:/access.log:ro
    depends_on:
      - db

  db:
    image: mysql
    restart: always
    volumes:
      - dbdata:/var/lib/mysql
    environment:
      MYSQL_ROOT_PASSWORD: testpass

  www:
    image: nginx
    restart: always
    labels:
      - "traefik.http.routers.www.rule=PathPrefix(`/`)"
      - "traefik.http.routers.www.middlewares=blacklist@file"
    volumes:
      - ./www:/usr/share/nginx/html:rw
      - ./nginx.conf:/etc/nginx/nginx.conf:ro

  # Other services...