Modular Distributed Fingerprinting Engine – Scannerl

Introduction

Scannerl is a modular, distributed fingerprinting engine implemented in Erlang. It can scan very large number of targets on a single host, but also can be distributed across multiple hosts. This tool is tested on Linux (Ubuntu, Debian, Arch & Kali Linux), but should work on other Linux distros, too.

Scannerl: Modular Distributed Fingerprinting Engine

Scannerl is an open source distributed fingerprinting tool developed by Kudelski Security. There are other fingerprinting tools, but those tools come with different limitations/problems such as: scanning on a few hosts at the time (not suitable for large IP addresses ranges); higher probability of being blacklisted, if large IP addresses range is protected by IPS devices (incomplete information). Scannerl successfully fights these limitations, therefore you can fingerprint multiple hosts simultaneously.

What is ZMap to port scanning, Scannerl is to fingerprinting. So, if you are planning large-scale fingerprinting sessions, this tool is the right choice. You can use it on a single host, but you can also easily distribute work over several machines.

Features:

• Fast: With Scannerl you can spread the tasks across multiple hosts and increase the overall performance. Other fingerprinting tools are limited by the available resources of the host they run on (network card, CPU, RAM, network bandwidth, etc.). Cluster of virtual servers will be enough to perform large-scale scans, there is no need for high-end server.
• Distributed:  Master/Slave architecture enables workload across different hosts. Process is easy and transparent, you’ll only need to provide the hosts to use.
• Scalable: Thanks to Erlang’s small sized processes, the tool can execute a large number of tasks in parallel, on the same host. Since Scannerl has ability to distribute the work across different hosts, we can say that it’s high-functioning and easily scalable tool.
• Modular: You can easily add custom modules in order to fingerprint specific protocols and services in a few lines of code. In addition, it’s possible to add output modules to insert any results directly into a database technology of your choice.
• Stealth: Reduced chance of being blocked. When using a single host to fingerprint a large number of IPs, there are high possabilites that ISPs/Firewalls might block your probes. But with Scannerl you can distribute your scan among several IP addresses and reduce the chance to be blocked.
• Smart: Scannerl can retrieve specific information from a fingerprint session (a field in the header, the version, etc.).

Requirements:

• Erlang v18+

Modules

This fingerprinting tool is very modular, therefore it’s easy to add new modules at compile time or dynamically (external file). Available modules:

Install

To build from source and to use Scannerl, first you need to install Erlang/OTP:

• Debian

$ sudo apt install erlang erlang-src rebar

• Arch

$ sudo pacman -S erlang-nox rebar

Then clone Scannerl from the github repo, and build:

$ git clone https://github.com/kudelskisecurity/scannerl.git
$ cd scannerl
$ ./build.sh

• Kali

First install dependencies:

$ sudo apt install libssl-dev automake autoconf libncurses5-dev

Then install rebar (Erlang build tool for compiling and testing Erlang applications):

$ cd /tmp
$ git clone git://github.com/rebar/rebar.git; cd rebar
$ ./bootstrap
$ sudo cp rebar /usr/local/bin/rebar

Install kerl and Erlang/OTP 20.1

$ cd /tmp
$ curl -O https://raw.githubusercontent.com/kerl/kerl/master/kerl
$ chmod +x kerl
$ sudo cp kerl /usr/local/bin/kerl
$ kerl build 20.1 20.1
$ sudo mkdir /opt/kerl; sudo chown -R $USER /opt/kerl
$ kerl install 20.1 /opt/kerl/20.1

Then you’ll be able to build Scannerl:

$ source /opt/kerl/20.1/activate
$ git clone https://github.com/kudelskisecurity/scannerl.git
$ cd scannerl
$ ./build.sh

Basic Usage

$ ./scannerl -h
   ____   ____    _    _   _ _   _ _____ ____  _
  / ___| / ___|  / \  | \ | | \ | | ____|  _ \| |
  \___ \| |     / _ \ |  \| |  \| |  _| | |_) | |
   ___) | |___ / ___ \| |\  | |\  | |___|  _ <| |___
  |____/ \____/_/   \_\_| \_|_| \_|_____|_| \_\_____|

USAGE
  scannerl MODULE TARGETS [NODES] [OPTIONS]

  MODULE:
    -m <mod> --module <mod>
      mod: the fingerprinting module to use.
           arguments are separated with a colon.

  TARGETS:
    -f <target> --target <target>
      target: a list of target separated by a comma.
    -F <path> --target-file <path>
      path: the path of the file containing one target per line.
    -d <domain> --domain <domain>
      domain: a list of domains separated by a comma.
    -D <path> --domain-file <path>
      path: the path of the file containing one domain per line.

  NODES:
    -s <node> --slave <node>
      node: a list of node (hostnames not IPs) separated by a comma.
    -S <path> --slave-file <path>
      path: the path of the file containing one node per line.
            a node can also be supplied with a multiplier (<node>*<nb>).

  OPTIONS:
    -o <mod> --output <mod>     comma separated list of output module(s) to use.
    -p <port> --port <port>     the port to fingerprint.
    -t <sec> --timeout <sec>    the fingerprinting process timeout.
    -T <sec> --stimeout <sec>   slave connection timeout (default: 10).
    -j <nb> --max-pkt <nb>      max pkt to receive (int or "infinity").
    -r <nb> --retry <nb>        retry counter (default: 0).
    -c <cidr> --prefix <cidr>   sub-divide range with prefix > cidr (default: 24).
    -M <port> --message <port>  port to listen for message (default: 57005).
    -P <nb> --process <nb>      max simultaneous process per node (default: 28232).
    -Q <nb> --queue <nb>        max nb unprocessed results in queue (default: infinity).
    -C <path> --config <path>   read arguments from file, one per line.
    -O <mode> --outmode <mode>  0: on Master, 1: on slave, >1: on broker (default: 0).
    -v <val> --verbose <val>    be verbose (0 <= int <= 255).
    -K <opt> --socket <opt>     comma separated socket option (key[:value]).
    -l --list-modules           list available fp/out modules.
    -V --list-debug             list available debug options.
    -A --print-args             Output the args record.
    -X --priv-ports             use only source port between 1 and 1024.
    -N --nosafe                 keep going even if some slaves fail to start.
    -w --www                    DNS will try for www.<domain>.
    -b --progress               show progress.
    -x --dryrun                 dry run.

Distributed scan [setup & usage]

In order to perform distributed scan, you’ll need:

• Master node: to run Scannerl’s binary (needs installed Scannerl)
• Slave node(s): to connect Scannerl (need installed Erlang)

Requirements – all hosts:

• have the same version of Erlang installed
• are able to connect to each other using SSH public key
• names resolve (use /etc/hosts if no proper DNS is setup)
• have the same Erlang security cookie
• must allow connection to Erlang EPMD port (TCP/4369)
• have the following range of ports opened: TCP/11100 to TCP/11100 + number-of-slaves

To use, provide a list of slaves – example ( -s or -S switches):

$ ./scannerl -m httpbg -d example.com -s host1,host2,host3

$ ./scannerl -l

Standalone usage

You can use Scannerl on the local host without any other host, but the slave will be created anyway. So, you’ll need to fulfill same requirements described above. Make sure your host is able to resolve itself with the following:

$ grep -q "127.0.1.1\s*`hostname`" /etc/hosts || echo "127.0.1.1 `hostname`" | sudo tee -a /etc/hosts

Then create SSH key and add it to the authorized_keys. It’s assumed that you have SSH server running:

$ cat $HOME/.ssh/id_rsa.pub >> $HOME/.ssh/authorized_keys

Standalone scan example:

$ ./scannerl -m httpbg -d example.com

For further information and usage guide, click the “documentation” button below.