bugspider is a commandline utility for domain gathering and automated vulnerability scanning.
ssllabs.com and queues them for further scanning using a beanstalk deamon. It saves the domains in a csv file for easy manual analysis.deduplication: test, if the domain was processed before and push it to further tubes, if not processed already
opengit: test the domain for a publicly available git repository. Read the writeup for additional information on this vunlerability.
There a two docker images provided:
a bundled image (Tag: bundle) including a local beanstalk daemon and an entrypoint for spawning multiple producers and workers. This image is ideal for a quick start launching only one container, but lacks the scaleability to spawn more producers/workers when needed.
a standalone image (Tag: latest) containing only the commandline tool as an entrypoint. This image is ideal in a deployment cluster like Docker Swarm or Kubernetes. The image needs an external beanstalk daemon.
./bugspider <command>
COMMANDS:
worker [tubes_to_watch...]
Starts a listener to check for duplicates and save the domain.
The worker runs until manually stopped and waits for new jobs
from beanstalk. Defaults to all tubes.
scraper (ssllabs|immuniweb)
Starts scraping one of the given providers. The scraped domains
will be sent to beanstalk for further processing. The scraper
runs until manually stopped and repeats the process every 10 seconds.
TUBES:
deduplication
Checks from a CSV Log File, if a host was already processed.
If not already processed, the Host will be pushed to other tubes
for further processing.
opengit
Checks a host for public accessable git repositories (domain.com/.git).
If a Host ist vulnerable, it will be logged, the accessable git config
will be stored in a separate 'config' dir.
To configure the tubes, which the worker should be listen to, you have to use command line arguments (see Usage).
To configure the beanstalk host (and port), you can set the environment Variable BENASTALK_HOST
Using bugspider as a bundled, ready to use, version:
docker run -d \
--name=bugspider \
-v $(pwd)/output:/app/output \
--net=container:openvpn robinmoser/bugspider:bundle
Using multiple bugspider instances with the standalone image:
# create a shared network
docker network create bugspider-net
# start a beanstalk daemon
docker run -d \
--network bugspider-web \
--name=beanstalk schickling/beanstalkd
# start multiple workers
docker run -d \
--network bugspider-web \
-e BEANSTALK_HOST=beanstalk:11300 \
bugspider:latest worker deduplication
docker run -d \
--network bugspider-web \
-e BEANSTALK_HOST=beanstalk:11300 \
bugspider:latest worker opengit
docker run -d \
--network bugspider-web \
-e BEANSTALK_HOST=beanstalk:11300 \
bugspider:latest worker opengit
# start both producers
docker run -d \
--network bugspider-web \
-e BEANSTALK_HOST=beanstalk:11300 \
bugspider:latest scraper ssllabs
docker run -d \
--network bugspider-web \
-e BEANSTALK_HOST=beanstalk:11300 \
bugspider:latest scraper immuniweb
Content type
Image
Digest
sha256:d9b9c3237…
Size
4.4 MB
Last updated
almost 2 years ago
docker pull robinmoser/bugspider