Source: webmole/crawler-benchmark⁠ Built by travis: WebMole/crawler-benchmark⁠

Crawler Benchmark

A Reference Framework for the Automated Exploration of Web Applications. Provides some general web features to let you test crawlers in a well defined environment.

Usage

1. Clone repository

2. cd into the repository

3. Install Docker⁠

4. Install docker-compose⁠

5. Build and use the docker image with docker-compose⁠

   cd crawler-benchmark
   cp .env.example .env # then edit with desired credentials
   docker-compose up -d
   

   Copy

When it's done, you can visit the app running at localhost:8080⁠