malte311/fly-scraper repository overview

⁠Fly-Scraper

A simple tool to track prices on Google flights.

⁠Installation (via Docker)

1. Download this repository:

git clone [email protected]:Malte311/Fly-Scraper.git

1. Adjust the docker-compose.yml file. Specify the correct server url and port. If you want to get notifications via email, you need a gmail account. Set the MAIL_USER variable to the sender email (has to be a gmail account) and MAIL_PW to the corresponding app password (not the "normal" password!). The receiving email address can use any email provider.

version: "2"

services:
  web:
    image: malte311/fly-scraper:latest
    container_name: fly-scraper
    ports:
      - "127.0.0.1:8080:80"
    volumes:
      - ./data/:/php-code/data/
    restart: always
    environment:
      - SERVERURL=https://example.com/fly-scraper
      - [email protected]
      - MAIL_PW=password
      - [email protected]

1. Get the newest Docker image and start a Docker container (inside of the project folder):

docker-compose pull
docker-compose up -d

1. Install a cronjob to run the scraper periodically.

chmod +x cron.sh
crontab -e -u root

A cronjob every 45 minutes could look like this:

*/45 * * * * /bin/sh /var/docker-compose/fly-scraper/cron.sh

Note that the scraper has a session limit of 2 connections per session in order to avoid too many requests at the same time. This limit can be changed by adjusting the parameter given to run_scraper.py in the file cron.sh.