samirettali/pastebin-scraper repository overview

⁠Pastebin scraper

This is a simple pastebin scraper.

It uses the paid scraping APIs⁠ to get the pastes and can use MongoDB or Postgres as databases for storage. It also uses Healthchecks⁠ to monitor it's status.

These environmental variables need to exist in the .env file and they will be used by docker-compose:

• STORAGE_TYPE: postgres or mongo

If you choose Mongo you have to set these environment variables:

• MONGO_URI
• MONGO_DB
• MONGO_COL

Alternatively, if you choose Postgres:

• POSTGRES_HOST
• POSTGRES_PORT
• POSTGRES_USER
• POSTGRES_PASSWORD
• POSTGRES_DBNAME

⁠Running

You can pull a docker image built for x86 from docker hub:

$ docker pull samirettali/pastebin-scraper

Or if you are running it on another architecture (arm for example):

$ git clone github.com/samirettali/pastebin-scraper
$ cd pastebin-scraper

Open Dockerfile and change the ENV variable according to your architecture, and then:

$ docker build -t pastebin-scraper .