insights/content/tutorials/mikrotik-netflow-elk/index.md

---
title: "Traffic Analysis on MikroTik Routers: A Study Using Traffic Flow, Filebeat, Elasticsearch, and Kibana"
description: Configure Filebeat and MikroTik Traffic Flow to send NetFlow data to ElasticSearch for real-time analysis.
summary: Step-by-step guides on adjusting Filebeat configuration, enabling NetFlow modules, and setting up Kibana dashboards.
date: 2024-11-11T13:26:00+07:00
lastmod:
draft: false
noindex: false
nav_weight: 1000
categories:
    - TIL
tags:
    - MikroTik
    - NetFlow
    - ElasticSearch
    - Kibana
    - Filebeat
images:
authors:
    - ditatompel
---

In the mid-1990s, **Cisco** introduced the **NetFlow** feature on its routers.
This NetFlow feature provides the ability to collect information on incoming
and outgoing packets from a network interfaces. In general, the NetFlow setup
consists of these three main components:

-   **Flow Exporter**: Responsible for collecting network packets and sending
    them to the **Flow Collector**.
-   **Flow Collector**: Responsible for receiving and preprocessing data sent
    from the **Flow Exporter**.
-   **Analysis Application**: An application that is responsible for analyzing
    data received from the Flow Collector and usually visualizes it in the form
    of graphs.

This NetFlow feature was then implemented by many companies that produce
routers and switches with different names. For example, **Juniper Networks**
uses the term **J-Flow**, while **MikroTik** uses the name **Traffic Flow**.

I'm currently running a **MikroTik RB450G** router for my home network, which
is why I'm excited to share my experience with using the Traffic Flow feature
on one.

{{< youtube yHbH-oJX-Lg >}}

## Prerequisites

Before starting, you need to install and ensure that **Filebeat**,
**ElasticSearch**, and **Kibana** are running properly. Please follow the
installation and configuration process from the related documentation pages:

-   [ElasticSearch][elasticsearch-install]
-   [Kibana][kibana-install]
-   [Filebeat][filebeat-install]

As additional information related to this article, I installed Filebeat on a
Linux machine that is still connected to the same network as my MikroTik
router. For ElasticSearch and Kibana, I set them up on a Virtual Private Server
(VPS).

## Access Rights for Filebeat

I created a new user with special access rights that will be used by Filebeat
to send processed data from the router to ElasticSearch. To do this, you need
to:

1.  Go to the **"Management"** > **"Security"** > **"Roles"** page in the
    Kibana Dashboard.
2.  Create a new role and give it a name for the role. In this article, I named
    it `filebeat_setup`.

In the **"Cluster privileges"** section, I granted access to the following:

-   `monitor`
-   `manage_ilm`
-   `manage_index_templates`
-   `manage_collector`
-   `manage_ingest_pipelines`
-   `manage_logstash_pipelines`
-   `manage_ml`
-   `manage_pipeline`

And, in the **"Index privileges"** section, I granted **all** privileges for
all **indices**.

To complete the setup, follow these additional steps:

1.  Go to the **"Management"** > **"Security"** page.
2.  Create a new user and name it `custom_filebeat`.
3.  In the **"Privileges"** section, assign the `filebeat_setup` role that we
    created earlier.
4.  Additionally, assign the following roles:
    -   `kibana_admin`
    -   `ingest_admin`
    -   `editor`
    -   `monitoring_user`
    -   `kibana_system`

## Filebeat NetFlow Module

Login to the Linux machine that has Filebeat installed and adjust the
configuration in `/etc/filebeat/filebeat.yml`, especially in the `setup.kibana`
and `output.elasticsearch` sections. My current `filebeat.yml` configuration is
as follows:

```yml
setup.kibana:
    host: "https://kibana.ditatompel.com:443"

output.elasticsearch:
    hosts: ["https://elastic-ap-southeast1-ctb1.ditatompel.com:443"]
    username: "custom_filebeat"
    password: "MySuperSecretPasswordThatMayAppearsOnYourFreakinAICompletions"
    ssl:
        enabled: true
```

Then edit `/etc/filebeat/modules.d/netflow.yml.disabled` and change
`netflow_host` to `0.0.0.0` so that Filebeat can receive data from the MikroTik
router. My current `netflow` module configuration is as follows:

```yml
- module: netflow
  log:
      enabled: true
      var:
          netflow_host: 0.0.0.0
          netflow_port: 2055
          internal_networks:
              - private
```

After that, enable the `netflow` module by running `sudo filebeat modules
enable netflow` command. To see the available modules, both active and
inactive, use the command `sudo filebeat modules list`.

Then run the command `sudo filebeat setup -e` to set up the _index template_
and _dashboards_ on Kibana.

Finally, restart the Filebeat service by running the command `sudo systemctl
restart filebeat`.

## MikroTik Traffic Flow Configuration

Login to the MikroTik router, using either SSH or Winbox. In this article, I
use **Winbox** for easier configuration management.

Go to **"IP"** > **"Traffic Flow"**. Click on the **"Targets"** menu and add a
new target. Change the **"Dst. Address"** field to the IP address of your
Filebeat server. Set the **"Version"** to **"IPFIX"** and ensure the
**"Enabled" checkbox** is selected. Then, click the **"Ok"** button.

In the **"Traffic Flow Settings"** menu, select the interfaces you want to
process and verify that the **"Enabled" checkbox** remains checked.

## Kibana Dashboard

For processed NetFlow results, go to **"Analytics"** > **"Dashboards"**. You
will see a variety of automatically generated dashboards by Filebeat. Search
for keywords like _"netflow"_ and explore the dashboards.

[elasticsearch-install]: https://www.elastic.co/guide/en/elasticsearch/reference/current/install-elasticsearch.html
[kibana-install]: https://www.elastic.co/guide/en/kibana/current/install.html
[filebeat-install]: https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-installation-configuration.html