Sign inSign up

suezawa/autorecon

By suezawa

•Updated over 5 years ago

Forked version of AutoRecon

Image
1

456

suezawa/autorecon repository overview

⁠(Added by @rung) Forked version of AutoRecon

This is a forked version of AutoRecon for my personal use. It's intended to be used on Docker.(Docker Hub⁠)

# Confirming network connectivity using popular 30 ports.
docker run --network host -it --rm --entrypoint=nmap suezawa/autorecon --top-ports 30 -Pn -open <ip>

# Start scanning
docker run --network host -it -v $HOME/autorecon/:/results suezawa/autorecon -v <ip>

⁠AutoRecon

AutoRecon is a multi-threaded network reconnaissance tool which performs automated enumeration of services. It is intended as a time-saving tool for use in CTFs and other penetration testing environments (e.g. OSCP). It may also be useful in real-world engagements.

The tool works by firstly performing port scans / service detection scans. From those initial results, the tool will launch further enumeration scans of those services using a number of different tools. For example, if HTTP is found, nikto will be launched (as well as many others).

Everything in the tool is highly configurable. The default configuration performs no automated exploitation to keep the tool in line with OSCP exam rules. If you wish to add automatic exploit tools to the configuration, you do so at your own risk. The author will not be held responsible for negative actions that result from the mis-use of this tool.

⁠Origin

AutoRecon was inspired by three tools which the author used during the OSCP labs: Reconnoitre⁠, ReconScan⁠, and bscan⁠. While all three tools were useful, none of the three alone had the functionality desired. AutoRecon combines the best features of the aforementioned tools while also implementing many new features to help testers with enumeration of multiple targets.

⁠Features

  • Supports multiple targets in the form of IP addresses, IP ranges (CIDR notation), and resolvable hostnames.
  • Can scan targets concurrently, utilizing multiple processors if they are available.
  • Customizable port scanning profiles for flexibility in your initial scans.
  • Customizable service enumeration commands and suggested manual follow-up commands.
  • An intuitive directory structure for results gathering.
  • Full logging of commands that were run, along with errors if they fail.
  • Global and per-scan pattern matching so you can highlight/extract important information from the noise.

⁠Quick Install (Docker)

If you don't have a Kali instance, you can quickly install AutoRecon using the Dockerfile in the repository. Simply download the Dockerfile, and run the following command from the same directory:

sudo docker build -t tib3rius/autorecon .

⁠Requirements

  • Python 3
  • python3-pip
  • pipx (optional, but recommended)
⁠Python 3

If you don't have these installed, and are running Kali Linux, you can execute the following:

$ sudo apt install python3
$ sudo apt install python3-pip

Additionally, if you experience any issues with the stability of the python3-pip installation (as reported by a number of people installing pip3 via apt on the OSCP distribution of Kali), you can install it manually as follows:

$ curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
$ python3 get-pip.py

The pip3 command should now be usable.

⁠pipx

Further, it's recommended you use pipx to manage your python packages; this installs each python package in it's own virtualenv, and makes it available in the global context, which avoids conflicting package dependencies and the resulting instability. To summarise the installation instructions:

$ python3 -m pip install --user pipx
$ python3 -m pipx ensurepath

Note that if you want to elevate privileges to run a program installed with pipx, with sudo, you have two options:

  1. Append the appropriate path to your execution command, using one of the following examples (recommended):
$ sudo env "PATH=$PATH" autorecon [OPTIONS]
$ sudo $(which autorecon) [OPTIONS]

To make this easier, you could add the following alias to your ~/.profile (or equivalent):

alias sudo="sudo env \"PATH=$PATH\""
  1. Add the pipx binary path to the secure_path set in /etc/sudoers
sudo visudo /etc/sudoers

Update the secure_path directive as follows:

Defaults	secure_path="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/home/kali/.local/bin"

If you're not using Kali Linux, make sure to adjust the path to the relevant user. Further detail on the installation of pipx is available in their installation instructions available here⁠. Please refer to this for any issues you experience.

⁠Supporting packages

Several commands used in AutoRecon reference the SecLists project, in the directory /usr/share/seclists/. You can either manually download the SecLists project to this directory (https://github.com/danielmiessler/SecLists⁠), or if you are using Kali Linux (highly recommended) you can run the following:

$ sudo apt install seclists

AutoRecon will still run if you do not install SecLists, though several commands may fail, and some manual commands may not run either.

Additionally the following commands may need to be installed, depending on your OS:

curl
enum4linux
gobuster
nbtscan
nikto
nmap
onesixtyone
oscanner
smbclient
smbmap
smtp-user-enum
snmpwalk
sslscan
svwar
tnscmd10g
whatweb
wkhtmltoimage

On Kali Linux, you can ensure these are all installed using the following command:

$ sudo apt install seclists curl enum4linux gobuster nbtscan nikto nmap onesixtyone oscanner smbclient smbmap smtp-user-enum snmp sslscan sipvicious tnscmd10g whatweb wkhtmltopdf

⁠Installation

Ensure you have all of the requirements installed as per the previous section.

$ pipx install git+https://github.com/Tib3rius/AutoRecon.git
⁠Using pip
$ python3 -m pip install git+https://github.com/Tib3rius/AutoRecon.git
⁠Manual

If you'd prefer not to use pip or pipx, you can always still install and execute autorecon.py manually as a script. First install the dependencies:

$ python3 -m pip install -r requirements.txt

You will then be able to run the autorecon.py script (from <AUTORECON_ROOT_DIR>/src/autorecon):

$ python3 autorecon.py [OPTIONS] 127.0.0.1

See detailed usage options below.

⁠Usage

AutoRecon uses Python 3 specific functionality and does not support Python 2.

usage: autorecon    [-h] [-t TARGET_FILE] [-ct <number>] [-cs <number>]
                    [--profile PROFILE_NAME] [-o OUTPUT_DIR] [--single-target]
                    [--only-scans-dir] [--heartbeat HEARTBEAT]
                    [--nmap NMAP | --nmap-append NMAP_APPEND] [-v]
                    [--disable-sanity-checks]
                    [targets [targets ...]]

Network reconnaissance tool to port scan and automatically enumerate services
found on multiple targets.

positional arguments:
  targets               IP addresses (e.g. 10.0.0.1), CIDR notation (e.g.
                        10.0.0.1/24), or resolvable hostnames (e.g. foo.bar)
                        to scan.

optional arguments:
  -h, --help            show this help message and exit
  -t TARGET_FILE, --targets TARGET_FILE
                        Read targets from file.
  -ct <number>, --concurrent-targets <number>
                        The maximum number of target hosts to scan
                        concurrently. Default: 5
  -cs <number>, --concurrent-scans <number>
                        The maximum number of scans to perform per target
                        host. Default: 10
  --profile PROFILE_NAME
                        The port scanning profile to use (defined in port-
                        scan-profiles.toml). Default: default
  -o OUTPUT_DIR, --output OUTPUT_DIR
                        The output directory for results. Default: results
  --single-target       Only scan a single target. A directory named after the
                        target will not be created. Instead, the directory
                        structure will be created within the output directory.
                        Default: false
  --only-scans-dir      Only create the "scans" directory for results. Other
                        directories (e.g. exploit, loot, report) will not be
                        created. Default: false
  --heartbeat HEARTBEAT
                        Specifies the heartbeat interval (in seconds) for task
                        status messages. Default: 60
  --nmap NMAP           Override the {nmap_extra} variable in scans. Default:
                        -vv --reason -Pn
  --nmap-append NMAP_APPEND
                        Append to the default {nmap_extra} variable in scans.
  -v, --verbose         Enable verbose output. Repeat for more verbosity.
  --disable-sanity-checks
                        Disable sanity checks that would otherwise prevent the
                        scans from running. Default: false
⁠Examples

Scanning a single target:

$ autorecon 127.0.0.1
[*] Scanning target 127.0.0.1
[*] Running service detection nmap-full-tcp on 127.0.0.1
[*] Running service detection nmap-top-20-udp on 127.0.0.1
[*] Running service detection nmap-quick on 127.0.0.1
[*] Service detection nmap-quick on 127.0.0.1 finished successfully
[*] [127.0.0.1] ssh found on tcp/22
[*] [127.0.0.1] http found on tcp/80
[*] [127.0.0.1] rpcbind found on tcp/111
[*] [127.0.0.1] postgresql found on tcp/5432
[*] Running task tcp/22/nmap-ssh on 127.0.0.1
[*] Running task tcp/80/nmap-http on 127.0.0.1
[*] Running task tcp/80/curl-index on 127.0.0.1
[*] Running task tcp/80/curl-robots on 127.0.0.1
[*] Running task tcp/80/whatweb on 127.0.0.1
[*] Running task tcp/80/nikto on 127.0.0.1
[*] Running task tcp/111/nmap-nfs on 127.0.0.1
[*] Task tcp/80/curl-index on 127.0.0.1 finished successfully
[*] Task tcp/80/curl-robots on 127.0.0.1 finished successfully
[*] Task tcp/22/nmap-ssh on 127.0.0.1 finished successfully
[*] Task tcp/80/whatweb on 127.0.0.1 finished successfully
[*] Task tcp/111/nmap-nfs on 127.0.0.1 finished successfully
[*] Task tcp/80/nmap-http on 127.0.0.1 finished successfully
[*] Task tcp/80/nikto on 127.0.0.1 finished successfully
[*] Service detection nmap-top-20-udp on 127.0.0.1 finished successfully
[*] Service detection nmap-full-tcp on 127.0.0.1 finished successfully
[*] [127.0.0.1] http found on tcp/5984
[*] [127.0.0.1] rtsp found on tcp/5985
[*] Running task tcp/5984/nmap-http on 127.0.0.1
[*] Running task tcp/5984/curl-index on 127.0.0.1
[*] Running task tcp/5984/curl-robots on 127.0.0.1
[*] Running task tcp/5984/whatweb on 127.0.0.1
[*] Running task tcp/5984/nikto on 127.0.0.1
[*] Task tcp/5984/curl-index on 127.0.0.1 finished successfully
[*] Task tcp/5984/curl-robots on 127.0.0.1 finished successfully
[*] Task tcp/5984/whatweb on 127.0.0.1 finished successfully
[*] Task tcp/5984/nikto on 127.0.0.1 finished successfully
[*] Task tcp/5984/nmap-http on 127.0.0.1 finished successfully
[*] Finished scanning target 127.0.0.1

The default port scan profile first performs a full TCP port scan, a top 20 UDP port scan, and a top 1000 TCP port scan. You may ask why AutoRecon scans the top 1000 TCP ports at the same time as a full TCP port scan (which also scans those ports). The reason is simple: most open ports will generally be in the top 1000, and we want to start enumerating services quickly, rather than wait for Nmap to scan every single port. As you can see, all the service enumeration scans actually finish before the full TCP port scan is done. While there is a slight duplication of efforts, it pays off by getting actual enumeration results back to the tester quicker.

Note that the actual command line output will be colorized if your terminal supports it.

Scanning multiple targets

$ autorecon 192.168.1.100 192.168.1.1/30 localhost
[*] Scanning target 192.168.1.100
[*] Scanning target 192.168.1.1
[*] Scanning target 192.168.1.2
[*] Scanning target localhost
[*] Running service detection nmap-quick on 192.168.1.100
[*] Running service detection nmap-quick on localhost
[*] Running service detection nmap-top-20-udp on 192.168.1.100
[*] Running service detection nmap-quick on 192.168.1.1
[*] Running service detection nmap-quick on 192.168.1.2
[*] Running service detection nmap-top-20-udp on 192.168.1.1
[*] Running service detection nmap-full-tcp on 192.168.1.100
[*] Running service detection nmap-top-20-udp on localhost
[*] Running service detection nmap-top-20-udp on 192.168.1.2
[*] Running service detection nmap-full-tcp on localhost
[*] Running service detection nmap-full-tcp on 192.168.1.1
[*] Running service detection nmap-full-tcp on 192.168.1.2
...

AutoRecon supports multiple targets per scan, and will expand IP ranges provided in CIDR notation. By default, only 5 targets will be scanned at a time, with 10 scans per target.

Scanning multiple targets with advanced options

$ autorecon -ct 2 -cs 2 -vv -o outputdir 192.168.1.100 192.168.1.1/30 localhost
[*] Scanning target 192.168.1.100
[*] Scanning target 192.168.1.1
[*] Running service detection nmap-quick on 192.168.1.100 with nmap -vv --reason -Pn -sV -sC --version-all -oN "/root/outputdir/192.168.1.100/scans/_quick_tcp_nmap.txt" -oX "/root/outputdir/192.168.1.100/scans/_quick_tcp_nmap.xml" 192.168.1.100
[*] Running service detection nmap-quick on 192.168.1.1 with nmap -vv --reason -Pn -sV -sC --version-all -oN "/root/outputdir/192.168.1.1/scans/_quick_tcp_nmap.txt" -oX "/root/outputdir/192.168.1.1/scans/_quick_tcp_nmap.xml" 192.168.1.1
[*] Running service detection nmap-top-20-udp on 192.168.1.100 with nmap -vv --reason -Pn -sU -A --top-ports=20 --version-all -oN "/root/outputdir/192.168.1.100/scans/_top_20_udp_nmap.txt" -oX "/root/outputdir/192.168.1.100/scans/_top_20_udp_nmap.xml" 192.168.1.100
[*] Running service detection nmap-top-20-udp on 192.168.1.1 with nmap -vv --reason -Pn -sU -A --top-ports=20 --version-all -oN "/root/outputdir/192.168.1.1/scans/_top_20_udp_nmap.txt" -oX "/root/outputdir/192.168.1.1/scans/_top_20_udp_nmap.xml" 192.168.1.1
[-] [192.168.1.1 nmap-quick] Starting Nmap 7.70 ( https://nmap.org ) at 2019-03-01 17:25 EST
[-] [192.168.1.100 nmap-quick] Starting Nmap 7.70 ( https://nmap.org ) at 2019-03-01 17:25 EST
[-] [192.168.1.100 nmap-top-20-udp] Starting Nmap 7.70 ( https://nmap.org ) at 2019-03-01 17:25 EST
[-] [192.168.1.1 nmap-top-20-udp] Starting Nmap 7.70 ( https://nmap.org ) at 2019-03-01 17:25 EST
[-] [192.168.1.1 nmap-quick] NSE: Loaded 148 scripts for scanning.
[-] [192.168.1.1 nmap-quick] NSE: Script Pre-scanning.
[-] [192.168.1.1 nmap-quick] NSE: Starting runlevel 1 (of 2) scan.
[-] [192.168.1.1 nmap-quick] Initiating NSE at 17:25
[-] [192.168.1.1 nmap-quick] Completed NSE at 17:25, 0.00s elapsed
[-] [192.168.1.1 nmap-quick] NSE: Starting runlevel 2 (of 2) scan.
[-] [192.168.1.1 nmap-quick] Initiating NSE at 17:25
[-] [192.168.1.1 nmap-quick] Completed NSE at 17:25, 0.00s elapsed
[-] [192.168.1.1 nmap-quick] Initiating ARP Ping Scan at 17:25
[-] [192.168.1.100 nmap-quick] NSE: Loaded 148 scripts for scanning.
[-] [192.168.1.100 nmap-quick] NSE: Script Pre-scanning.
[-] [192.168.1.100 nmap-quick] NSE: Starting runlevel 1 (of 2) scan.
[-] [192.168.1.100 nmap-quick] Initiating NSE at 17:25
[-] [192.168.1.100 nmap-quick] Completed NSE at 17:25, 0.00s elapsed
[-] [192.168.1.100 nmap-quick] NSE: Starting runlevel 2 (of 2) scan.
[-] [192.168.1.100 nmap-quick] Initiating NSE at 17:25
[-] [192.168.1.100 nmap-quick] Completed NSE at 17:25, 0.00s elapsed
[-] [192.168.1.100 nmap-quick] Initiating ARP Ping Scan at 17:25
...

In this example, the -ct option limits the number of concurrent targets to 2, and the -cs option limits the number of concurrent scans per target to 2. The -vv option makes the output very verbose, showing the output of every scan being run. The -o option sets a custom output directory for scan results to be saved.

⁠Verbosity

AutoRecon supports three levels of verbosity:

  • (none) Minimal output. AutoRecon will announce when target scans start and finish, as well as which services were identified.
  • (-v) Verbose output. AutoRecon will additionally specify the exact commands which are being run, as well as highlighting any patterns which are matched in command output.
  • (-vv) Very verbose output. AutoRecon will output everything. Literally every line from all commands which are currently running. When scanning multiple targets concurrently, this can lead to a ridiculous amount of output. It is not advised to use -vv unless you absolutely need to see live output from commands.
⁠Results

By default, results will be stored in the ./results directory. A new sub directory is created for every target. The structure of this sub directory is:

.
├── exploit/
├── loot/
├── report/
│   ├── local.txt
│   ├── notes.txt
│   ├── proof.txt
│   └── screenshots/
└── scans/
    ├── _commands.log
    ├── _manual_commands.txt
    └── xml/

The exploit directory is intended to contain any exploit code you download / write for the target.

The loot directory is intended to contain any loot (e.g. hashes, interesting files) you find on the target.

The report directory contains some auto-generated files and directories that are useful for reporting:

  • local.txt can be used to store the local.txt flag found on targets.
  • notes.txt should contain a basic template where you can write notes for each service discovered.
  • proof.txt can be used to store the proof.txt flag found on targets.
  • The screenshots directory is intended to contain the screenshots you use to document the exploitation of the target.

The scans directory is where all results from scans performed by AutoRecon will go. This includes port scans / service detection scans, as well as any service enumeration scans. It also contains two other files:

  • _commands.log contains a list of every command AutoRecon ran against the target. This is useful if one of the commands fails and you want to run it again with modifications.
  • _manual_commands.txt contains any commands that are deemed "too dangerous" to run automatically, either because they are too intrusive, require modification based on human analysis, or just work better when there is a human monitoring them.

If a scan results in an error, a file called _errors.log will also appear in the scans directory with some details to alert the user.

If output matches a defined pattern, a file called _patterns.log will also appear in the scans directory with details about the matched output.

The scans/xml directory stores any XML output (e.g. from Nmap scans) separately from the main scan outputs, so that the scans directory itself does not get too cluttered.

⁠Port Scan profiles

The port-scan-profiles.toml file is where you can define the initial port scans / service detection commands. The configuration file uses the TOML format, which is explained here: https://github.com/toml-lang/toml⁠

Here is an example profile called "quick":

[quick]

    [quick.nmap-quick]

        [quick.nmap-quick.service-detection]
        command = 'nmap {nmap_extra} -sV --version-all -oN "{scandir}/_quick_tcp_nmap.txt" -oX "{scandir}/xml/_quick_tcp_nmap.xml" {address}'
        pattern = '^(?P<port>\d+)\/(?P<protocol>(tcp|udp))(.*)open(\s*)(?P<service>[\w\-\/]+)(\s*)(.*)$'

    [quick.nmap-top-20-udp]

        [quick.nmap-top-20-udp.service-detection]
        command = 'nmap {nmap_extra} -sU -A --top-ports=20 --version-all -oN "{scandir}/_top_20_udp_nmap.txt" -oX "{scandir}/xml/_top_20_udp_nmap.xml" {address}'
        pattern = '^(?P<port>\d+)\/(?P<protocol>(tcp|udp))(.*)open(\s*)(?P<service>[\w\-\/]+)(\s*)(.*)$'

Note that indentation is optional, it is used here purely for aesthetics. The "quick" profile defines a scan called "nmap-quick". This scan has a service-detection command which uses nmap to scan the top 1000 TCP ports. The command uses two references: {scandir} is the location of the scans directory for the target, and {address} is the address of the target.

A regex pattern is defined which matches three named groups (port, protocol, and service) in the output. Every service-detection command must have a corresponding pattern that matches all three of those groups. AutoRecon will attempt to do some checks and refuse to scan if any of these groups are missing.

An almost identical scan called "nmap-top-20-udp" is also defined. This scans the top 20 UDP ports.

Here is a more complicated example:

[udp]

    [udp.udp-top-20]

        [udp.udp-top-20.port-scan]
        command = 'unicornscan -mU -p 631,161,137,123,138,1434,445,135,67,53,139,500,68,520,1900,4500,514,49152,162,69 {address} 2>&1 | tee "{scandir}/_top_20_udp_unicornscan.txt"'
        pattern = '^UDP open\s*[\w-]+\[\s*(?P<port>\d+)\].*$'

        [udp.udp-top-20.service-detection]
        command = 'nmap {nmap_extra} -sU -A -p {ports} --version-all -oN "{scandir}/_top_20_udp_nmap.txt" -oX "{scandir}/xml/_top_20_udp_nmap.xml" {address}'
        pattern = '^(?P<port>\d+)\/(?P<protocol>(udp))(.*)open(\s*)(?P<service>[\w\-\/]+)(\s*)(.*)$'

In this example, a profile called "udp" defines a scan called "udp-top-20". This scan has two commands, one is a port-scan and the other is a service-detection. When a port-scan command is defined, it will always be run first. The corresponding pattern must match a named group "port" which extracts the port number from the output.

The service-detection will be run after the port-scan command has finished, and uses a new reference: {ports}. This reference is a comma-separated string of all the ports extracted by the port-scan command. Note that the same three named groups (port, protocol, and service) are defined in the service-detection pattern.

Both the port-scan and the service-detection commands use the {scandir} and {address} references.

Note that if a port-scan command is defined without a corresponding service-detection command, AutoRecon will refuse to scan.

This more complicated example is only really useful if you want to use unicornscan's speed in conjuction with nmap's service detection abilities. If you are content with using Nmap for both port scanning and service detection, you do not need to use this setup.

⁠Service Scans

The service-scans.toml file is where you can define service enumeration scans and other manual commands associated with certain services.

Here is an example of a simple configuration:

[ftp]

service-names = [
    '^ftp',
    '^ftp\-data'
]

    [[ftp.scan]]
    name = 'nmap-ftp'
    command = 'nmap {nmap_extra} -sV -p {port} --script="(ftp* or ssl*) and not (brute or broadcast or dos or external or fuzzer)" -oN "{scandir}/{protocol}_{port}_ftp_nmap.txt" -oX "{scandir}/xml/{protocol}_{port}_ftp_nmap.xml" {address}'

        [[ftp.scan.pattern]]
        description = 'Anonymous FTP Enabled!'
        pattern = 'Anonymous FTP login allowed'

    [[ftp.manual]]
    description = 'Bruteforce logins:'
    commands = [
        'hydra -L "{username_wordlist}" -P "{password_wordlist}" -e nsr -s {port} -o "{scandir}/{protocol}_{port}_ftp_hydra.txt" ftp://{address}',
        'medusa -U "{username_wordlist}" -P "{password_wordlist}" -e ns -n {port} -O "{scandir}/{protocol}_{port}_ftp_medusa.txt" -M ftp -h {address}'
    ]

Note that indentation is optional, it is used here purely for aesthetics. The service "ftp" is defined here. The service-names array contains regex strings which should match the service name from the service-detection scans. Regex is used to be as flexible as possible. The service-names array works on a whitelist basis; as long as one of the regex strings matches, the service will get scanned.

An optional ignore-service-names array can also be defined, if you want to blacklist certain regex strings from matching.

The ftp.scan section defines a single scan, named nmap-ftp. This scan defines a command which runs nmap with several ftp-related scripts. Several references are used here:

  • {nmap_extra} by default is set to "-vv --reason -Pn" but this can be overridden or appended to using the --nmap or --nmap-append command line options respectively. If the protocol is UDP, "-sU" will also be appended.
  • {port} is the port that the service is running on.
  • {scandir} is the location of the scans directory for the target.
  • {protocol} is the protocol being used (either tcp or udp).
  • {address} is the address of the target.

A pattern is defined for the nmap-ftp scan, which matches the simple pattern "Anonymous FTP login allowed". In the event that this pattern matches output of the nmap-ftp command, the pattern description ("Anonymous FTP Enabled!") will be saved to t

Tag summary

Content type

Image

Digest

Size

1.1 GB

Last updated

over 5 years ago

docker pull suezawa/autorecon