This system is spec'd to handle X flows/s with a data retention time of X days.
I've also chosen to run all 4 service on a single host, they could be installed on separate machines if you so desire.
| CPU Cores | 4 |
| Memory | 4-8GB |
| OS Storage | 8GB |
| Data Storage | 1TB |
| OS | Debian 10 |
apt install wget git htop curl openjdk-11-jdk unzip1. Download and add the signing keys for the Elastisearch repo
wget -qO - https://d3g5vo6xdbdb9a.cloudfront.net/GPG-KEY-opendistroforelasticsearch | apt-key add -2. Add the repo
echo "deb https://d3g5vo6xdbdb9a.cloudfront.net/apt stable main" | tee -a /etc/apt/sources.list.d/opendistroforelasticsearch.list3. Install the x86 variant of Elastisearch OSS.
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-oss-7.10.2-amd64.deb
dpkg -i elasticsearch-oss-7.10.2-amd64.deb4. Install Open Distro for Elastisearch
apt install opendistroforelasticsearch1. Open /etc/elasticsearch/elasticsearch.yml
vi /etc/elasticsearch/elasticsearch.ymlModify the following:
cluster.name: elastiflow-lab
node.name: ${HOSTNAME}
bootstrap.memory_lock: true
network.host: localhost # Optional - Set this to your server's IP2. Open /usr/lib/systemd/system/elasticsearch.service
vi /usr/lib/systemd/system/elasticsearch.serviceUnder the ‘[Service]’ category, add the following line:
LimitMEMLOCK=infinity3. Open /etc/elasticsearch/jvm.options
vi /etc/elasticsearch/jvm.optionsChange the -Xms and -Xmx values to the following
-Xms12g
-Xmx12g4. Reload systemd, enable, and start.
systemctl daemon-reload
systemctl start elasticsearch
systemctl enable elasticsearch1. Use wget or curl to download the .deb package.
wget https://elastiflow-packages.s3.amazonaws.com/flow-collector/flow-collector_5.1.1_linux_amd64.deb
curl https://elastiflow-packages.s3.amazonaws.com/flow-collector/flow-collector_5.1.1_linux_amd64.deb --output flow-collector_5.1.1_linux_amd64.debUse apt to install the package.
apt install ./flow-collector_5.1.1_linux_amd64.debStart the service.
systemctl enable flowcoll.service
systemctl start flowcoll.service