Machinae: Security Intelligence Collector

CyberPunk Information Gathering
Last Release: 05/09/2019     Last Commit: 05/18/2021
Machinae: Security Intelligence Collector

Introduction

Machinae is one of the best open-source tools used to gather forensic data from public sources. It was inspired by and designed to be an upgrade of the Automater, another collecting intelligence tool that has the ability to automate the OSINT framework of an IP address.

Like its predecessor, Machinae works by tapping into mainframes and collecting the needed Intel, this can range from URLs, IPs, domain names to file hashes. After collecting data it can also carry out an in-depth analysis of the gathered information.

Machinae: Security Intelligence Collector

As an intelligence-driven information gathering tool, this intelligence collector is at a whole new level when it comes to seeking hidden or readily available information. It does not matter in which section of the system the Intel is hidden, this tool can tweak any system into giving sensitive information.

It is the best when it comes to spotting and taking advantage of system vulnerabilities, with its intelligence threat analyses, malware and malleable security scans in cybersecurity.

Machinae has the capability to access, domain names, SSL fingerprints and email address. It has the ability to collect all the tiny bits of information on any network and correlate them to form a single block of useful intelligence data. Like any other interactive medium, it also allows you to intercept and inspect Intel before it reaches the intended receiver by modifying and replaying HTTP/1, HTTP/2 traffic, WebSockets and other SSL-protected protocols.

Features:

  • Ability to specify labels for single-line multimatch JSON outputs and relative time parameters
  • Upgraded to support Python 3
  • Support additional output types, including JSON, while making extraneous output optional
  • Ability to specify status codes to ignore per-API
  • Support simple paginated responses
  • Support url decoding values in results and url encoding ‘target’ in request URL
  • Prepend “http://” to URL targets when not starting with http:// or https://
  • “Short” output mode – simply output yes/no/error for each site
  • Support JSON parsing out-of-the-box without the need to write regular expressions
  • Automatically Defangs output and MISP Support (example: machinae.yml)
  • And much more…
Note: The output formats in Machinae are however limited to only JSON, normal, and normal with dot escaping.

Machinae: Out-of-the-Box Data Sources

It provides out of the box support for numerous sources, including:

  • IPVoid
  • URLVoid
  • URL Unshortener
  • Malc0de
  • SANS
  • FreeGeoIP (freegeoip.io)
  • Fortinet Category
  • VirusTotal pDNS (via web scrape – commented out)
  • VxVault
  • VirusTotal pDNS (via JSON API)
  • VirusTotal URL Report (via JSON API)
  • VirusTotal File Report (via JSON API)
  • Reputation Authority
  • ThreatExpert
  • ProjectHoneypot
  • McAfee Threat Intelligence
  • StopForumSpam
  • Cymru MHR
  • ICSI Certificate Notary
  • TotalHash (disabled by default)
  • DomainTools Prsed and Reverse Whois (Requires API key)
  • DomainTools Reputation
  • IP WHOIS (Using RIR REST interfaces)
  • Hacked IP
  • Metadefender Cloud (Requires API key)
  • GreyNoise (Requires API key)
  • IBM XForce (Required API key)

Supported Platforms:

  • Linux (Ubuntu, Debian, Linux Mint, etc.), Windows, OSX

Dependencies:

  • python3-dev (Debian-based)
  • libyaml-dev (Debian-based)
  • Latest config file.
Important: Check Configuration section for details and instructions on how to properly adjust config file.

Install Machinae

Clone/Download:

$ git clone https://github.com/r3vn/badKarma.git

Use pip3 to install:

$ pip3 install machinae

You can also install it directly:

$ pip3 install git+https://github.com/HurricaneLabs/machinae.git

Configuration

This intelligence tool is very easy to configure, with the availability of the merging system function, you can easily tune your system without interfering with the original machinae.yml. Depending on your local configuration, you can simply merge system file by locating system default configuration, /etc/machinae.yml, and merging it into local system configuration, /etc/machinae.local.yml. Local user configuration should be performed as a final step ~/.machinae .yml.

To perform basic HTTP authentications, simply create a YAML file and fill it with the required details. This may include your username and API key or password.

Usage

Use -h to list all available options. 

usage: machinae [-h] [-c CONFIG] [--nomerge] [-d DELAY] [-f FILE] [-i INFILE] [-v]
            [-o {D,J,N,S}] [-O {ipv4,ipv6,fqdn,email,sslfp,hash,url}] [-q]
            [-s SITES] [-a AUTH] [-H HTTP_PROXY]
            [--dump-config | --detect-otype]
            ...
  • -c/--config and --nomerge: additional details
  • -d/--delay:
  • -o: output format
    • N: normal, default output
    • D: default output but dot characters are replaced
    • J: Json output
  • -f/--file: write to file (default: stdout)
  • -O/--otype: overrides the target type auto-detection that is passed in
  • -q: disable verbose mode
  • -H/--http-proxy: passes HTTP proxy on the command line
  • all: runs through all services including those marked as “default: false“
  • …
Documentation Box
Download Box
CyberSecuritylinuxOS XOSINTPentestingPython