Netcap: Secure and Scalable Network Traffic Analysis Framework

Last Release: 11/12/2019    

Netcap: Secure and Scalable Network Traffic Analysis Framework

Introduction

Netcap (NETwork CAPture) is a command-line-based framework that is used to perform packet data analysis on network traffic, the tool has the ability to capture and convert streams of data packets within network traffic into audit records which the system can easily identify as protocols. The audit records generated can be kept on a separate drive or transferred from one end to the other through network servers.

Netcap: Secure and Scalable Network Traffic Analysis Framework Logo

It’s open source and written in Go which has improved its efficiency in terms of compilation speed and improved binary generation processes.

Netcap: Framework for Secure and Scalable Network Traffic Analysis

Netcap is very effective in collecting data packets over network traffic, packets can be gathered from input sources such as offline PCAP-NG or PCAP dumpfiles, other packets can be accessed and collected via a live interface.

Netcap uses Google’s Protocol Buffers to encode its output, which allows accessing it across a wide range of programming languages. Alternatively, output can be emitted as comma separated values, which is a common input format for data analysis tools and systems. (Netcap Docu)

To perform a capture on a packet Netcap uses its capture tool net.capture. The captured data is then transferred to a set of workers who split the packet into sections to come up with a protobuf audit record that is similar to the file it was generated from.

Audit records generated by Netcap are properly labeled with specific rulesets; this makes it easy for the system to tell if a given record is safe to process. For the data not to occupy too much space in the system Netcap compresses all its data in gzip.

Packet data that has already been decoded can also be converted into protocol buffers which can easily be used with the audit records. Netcap is equipped with a layer encoder and a custom encoder, the layer encoder is used to process gopacket layers while custom encoder works on any protocol that cannot be processed by gopacket. 

Netcap can view the available audit records by using the dump tool net.dump and later converts the audit records into supported file formats such as CSV and JSON. Log errors that result from malformed packets are recorded in the errors.log section.

All protocols that are unknown are placed in the unknown.pcap for further critical analysis. Due to its ability to analyze and decode all sets of network traffic it has access to, it is advisable to set Netcap on berkeley filter. 

Features:

  • PCAP and PCAP-NG Support
  • Supports USB Capture
  • Allows Live Capture
  • Uses a CLI (Command-line Interface)
  • Can collect network packets from distributed sources
  • Ideal for: monitoring honeypots, medical and industrial devices
  • Very useful for forensic data analysis

Components:

  • net.capture
  • net.dump
  • net.label
  • net.collect
  • net.agent
  • net.proxy
  • net.util
  • net.export

Supported Platforms

Windows, Linux, OS X 

Netcap Install

Using Go Get:

$ go get -u github.com/dreadl0ck/netcap/...

To install the command-line (using Development Build), run:

$ go build -o $(go env GOPATH)/bin/netcap -i github.com/dreadl0ck/netcap/cmd

Installation via Homebrew (OS X):

$ brew tap dreadl0ck/formulas
$ brew install netcap
Netcap uses Zeus Buildsystem, but it can be install without it. Shell scripts can be fount int the zeus/generated directory. For detailed instructions, click on the ‘documentation’ button below.

After installing Netcap it is advisable to perform a test run to check if all the units are functioning properly, from the project root launch:

$ go test -v -bench=. ./...

Netcap Usage

                       / |  
 _______    ______   _10 |_     _______   ______    ______
/     / \  /    / \ / 01/  |   /     / | /    / \  /    / \
0010100 /|/011010 /|101010/   /0101010/  001010  |/100110  |
01 |  00 |00    00 |  10 | __ 00 |       /    10 |00 |  01 |
10 |  01 |01001010/   00 |/  |01 \_____ /0101000 |00 |__10/|
10 |  00 |00/    / |  10  00/ 00/    / |00    00 |00/   00/
00/   10/  0101000/    0010/   0010010/  0010100/ 1010100/
                                                  00 |
Network Protocol Analysis Framework               00 |
created by Philipp Mieden, 2018                   00/
v0.3.5

 +---------------+--------+                                                                                                                                                                                                             
 |   Setting     | Value  |                                                                                                                                                                                                             
 +---------------+--------+                                                                                                                                                                                                             
 | Workers       | 1000   |                                                                                                                                                                                                             
 | MemBuffer     | true   |                                                                                                                                                                                                             
 | Compression   | true   |                                                                                                                                                                                                             
 | PacketBuffer  | 100    |                                                                                                                                                                                                             
 +---------------+--------+                                                                                                                                                                                                             
 spawned 1000 workers                                                                                                                                                                                                                   
 initialized 29 layer encoders | buffer size: 4096                                                                                                                                                                                      
 initialized 7 custom encoders | buffer size: 4096                                                                                                                                                                                      
 running since 15.535785474s, captured 13000 packets…                                                  

To view all the encoders that Netcap has, run –encoders, depending on the operation you want to execute you can use these flags to exclude or include an encoder -exclude, -include.

Examples:

Reading traffic from an interface that is live:

$ net.capture -iface eth0
* Use Ctrl-C (SIGINT) to stop the above operation.

Reading traffic from PCAP and PCAP-NG dumpfiles:

$ net.capture -r traffic.pcap

Reading a dumpfile and printing to stdout as CSV:

$ net.dump -r TCP.ncap.gz

Saving CSV output to a file:

$ net.dump -r TCP.ncap.gz -select Timestamp,SrcPort,DstPort > tcp.csv
Documentation Box
Download Box