Moloch is a system that provides tools to visually assess traffic flows and search for information related to network activity. The project was created in 2012 with the goal of creating an open replacement for a trading platform network packet processing that can scale to the level of AOL traffic volumes.
The introduction of the new system at AOL allowed them to achieve full control over the infrastructure by deploying them on their servers and significantly reducing costs.
Using Moloch to fully capture traffic on all AOL networks costs the same amount as when using a commercial solution that previously spent capturing traffic on a single network. The system can be scaled to handle traffic at speeds of tens of gigabits per second. The amount of data stored is limited only by the size of the available disk array. The session metadata is indexed in a cluster based on the Elasticsearch engine.
About Moloch
Moloch includes tools to capture and index traffic in the PCAP format normal, as well as for quick access to indexed data.
To analyze the accumulated information, a web interface is proposed that allows browsing, searching and exporting samples. Also an API is provided that allows you to transfer data about captured packets in PCAP format and analyzed sessions in JSON format to third-party applications. Using the PCAP format greatly simplifies integration with existing traffic analyzers such as Wireshark.
Access to Moloch is protected using HTTPS with strong passwords or by using an authenticating proxy server provided by the web server. All PCAPs are stored in the sensors and are only accessed via the Moloch interface or API. Moloch is not intended to replace an IDS, but works alongside them to store and index all network traffic in standard PCAP format, providing quick access.
Moloch It consists of three basic components:
- Traffic capture system: a multithreaded C language application to monitor traffic, write PCAP dumps to disk, analyze captured packets, and send metadata about sessions (SPI, stateful packet inspection) and protocols to the Elasticsearch cluster. PCAP files can be stored in encrypted form.
- A web interface based on the Node.js platform, that runs on each traffic capture server and processes requests related to accessing indexed data and transferring PCAP files through the Elasticsearch-based metadata repository and API.
- The web interface provides various display modesFrom general statistics, connection maps and visual graphs with data on changes in network activity to tools for studying individual sessions, analyzing activity by protocol and analyzing data from PCAP dumps.
The code is written in C language (Node.js / JavaScript interface) and is distributed under the Apache 2.0 license. Work on Linux and FreeBSD is supported. The ready-to-use packages are prepared for different versions of CentOS and Ubuntu.
How to install Moloch on Linux?
By default, packages built for Ubuntu and CentOS are offered, which we can obtain from the official website of the project.
In the case of those who use Ubuntu, they can obtain the package by typing any of the following commands.
For Ubuntu 16.04 LTS:
wget https://s3.amazonaws.com/files.molo.ch/builds/ubuntu-16.04/moloch_2.3.0-1_amd64.deb
For Ubuntu 18.04 LTS:
wget https://s3.amazonaws.com/files.molo.ch/builds/ubuntu-18.04/moloch_2.3.0-1_amd64.deb
To install, just type:
sudo apt install ./moloch*.deb
In the case of those who are CentOS users, the available packages can be obtained by typing.
6 CentOS
wget https://s3.amazonaws.com/files.molo.ch/builds/centos-6/moloch-2.3.0-1.x86_64.rpm
7 CentOS
wget https://s3.amazonaws.com/files.molo.ch/builds/centos-7/moloch-2.3.0-1.x86_64.rpm
8 CentOS
wget https://s3.amazonaws.com/files.molo.ch/builds/centos-8/moloch-2.3.0-1.x86_64.rpm
To install, just type:
sudo rpm install moloch*.rpm
For the case of other distributions compilation can be done by typing:
git clone https://github.com/aol/moloch ./easybutton-build.sh --install make config
Finally for the configuration, you can consult the wiki from the link below.