Install Elastic Stack: Elasticsearch, Logstash, Kibana

This guide walks you through installing the Elastic Stack—Elasticsearch, Logstash, and Kibana—on a single Linux server. You’ll end up with a working log ingestion pipeline: Logstash collects and processes data, Elasticsearch indexes and stores it, and Kibana provides a web UI for searching and visualizing. Before starting, ensure you have a Linux system (Ubuntu 20.04/22.04 or similar) with at least 4GB RAM, root or sudo access, and a stable internet connection.

What You’ll Need

• A Linux server (Ubuntu/Debian recommended) with sudo privileges
• Java 8, 11, or 17 (OpenJDK) installed (Elasticsearch 7.x requires Java 8+; 8.x requires Java 17)
• At least 4GB of RAM (more if you plan to index large volumes)
• curl and wget utilities
• Access to the internet to download packages

Step-by-Step

1. Install Java – Elasticsearch and Logstash need Java. Install OpenJDK 17 (for Elasticsearch 8.x):
   sudo apt update && sudo apt install openjdk-17-jdk -y
   Verify with java -version.
2. Add the Elastic GPG key and repository – Download and install the Elastic signing key:
   wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -
   Then add the repository:
   echo "deb https://artifacts.elastic.co/packages/8.x/apt stable main" | sudo tee /etc/apt/sources.list.d/elastic-8.x.list
3. Update and install Elasticsearch – Run sudo apt update then sudo apt install elasticsearch -y. This installs Elasticsearch 8.x.
4. Configure Elasticsearch – Edit /etc/elasticsearch/elasticsearch.yml. For a single-node setup, set:
   network.host: 0.0.0.0
discovery.type: single-node
   Also set xpack.security.enabled: true (default in 8.x) to enable security. Save and exit.
5. Start and enable Elasticsearch – Run:
   sudo systemctl daemon-reload
sudo systemctl enable elasticsearch
sudo systemctl start elasticsearch
   Check status with sudo systemctl status elasticsearch.
6. Set the built-in passwords – Elasticsearch 8.x auto-generates passwords for the elastic user during installation. Find them in the install output or run:
   sudo /usr/share/elasticsearch/bin/elasticsearch-setup-passwords auto
   Note the passwords for elastic and kibana_system.
7. Install Logstash – Run sudo apt install logstash -y.
8. Create a Logstash pipeline – Create a config file, e.g., /etc/logstash/conf.d/my-pipeline.conf, with a simple input, filter, and output. Example:

   input { stdin { } }
   output { elasticsearch { hosts => ["localhost:9200"] user => "elastic" password => "your_elastic_password" } }

9. Start Logstash – Enable and start the service:
   sudo systemctl enable logstash
sudo systemctl start logstash
   Logstash takes a few seconds to start; check with sudo systemctl status logstash.
10. Install Kibana – Run sudo apt install kibana -y.
11. Configure Kibana – Edit /etc/kibana/kibana.yml. Set:
    server.host: "0.0.0.0"
elasticsearch.hosts: ["http://localhost:9200"]
elasticsearch.username: "kibana_system"
elasticsearch.password: "your_kibana_password"
12. Start Kibana – Run:
    sudo systemctl enable kibana
sudo systemctl start kibana
13. Access Kibana – Open a browser and go to http://your-server-ip:5601. Log in with the elastic user and the password you set.

Common Mistakes

• Forgetting to set discovery.type: single-node – Without this, Elasticsearch tries to form a cluster and fails, leaving the node in a red state.
• Using the wrong Java version – Elasticsearch 8.x requires Java 17; using an older version causes startup failure.
• Not adjusting memory settings – By default, Elasticsearch uses 1GB heap. If your server has less RAM, it may crash. Set ES_JAVA_OPTS="-Xms512m -Xmx512m" in /etc/default/elasticsearch.
• Typing the wrong password in Logstash output – The password must match the elastic user’s password. A typo leads to authentication errors.
• Opening ports without firewall rules – Kibana runs on port 5601; ensure your firewall allows access or you won’t be able to reach it.

Tips

• Use a dedicated data directory for Elasticsearch (e.g., a separate disk) to avoid filling up the system partition.
• Set up index lifecycle management (ILM) to automatically roll over and delete old indices to save space.
• For production, increase the heap size to 50% of available RAM, but never exceed 32GB.
• Test your Logstash pipeline with sudo -u logstash /usr/share/logstash/bin/logstash --config.test_and_exit -f /etc/logstash/conf.d/my-pipeline.conf before starting the service.
• Use a reverse proxy (like Nginx) with HTTPS to secure Kibana access.

Troubleshooting

• Elasticsearch won’t start – Check logs at /var/log/elasticsearch/elasticsearch.log. Common issues: Java version mismatch, low disk space, or incorrect permissions on data directories. Run sudo journalctl -u elasticsearch -f for real-time logs.
• Kibana shows “Kibana server is not ready yet” – This usually means Kibana can’t connect to Elasticsearch. Verify Elasticsearch is running (curl localhost:9200), and check the credentials in kibana.yml.
• Logstash can’t connect to Elasticsearch – Ensure the hosts and credentials are correct. Test with curl -u elastic:password localhost:9200 from the Logstash server.
• Port 5601 is not accessible – Check firewall rules: sudo ufw allow 5601/tcp or sudo iptables -L.
• Out of memory errors – Reduce the heap size or add swap space. Check current usage with free -h.
• Elasticsearch status is yellow – This is normal for a single-node cluster (unassigned replicas). To fix, set index.number_of_replicas: 0 for your indices.

Frequently Asked Questions

Do I need a separate server for each component?

No, you can run all three on one server for testing or small deployments. For production, you might separate them for performance and scalability.

Can I use Docker instead of installing directly?

Yes, Elastic provides official Docker images for all components. The setup is similar but uses containers and docker-compose.

What is the default port for Elasticsearch?

Elasticsearch uses port 9200 for HTTP and 9300 for internal node communication. Kibana uses 5601.

How do I secure the Elastic Stack?

Elasticsearch 8.x enables security by default. You can also enable TLS encryption and use role-based access control. For Kibana, put it behind a reverse proxy with HTTPS.

Can I upgrade from an older version?

Yes, but you must follow the official upgrade path. For example, from 7.x to 8.x, you need to use the upgrade assistant in Kibana and ensure compatibility.

Why is my Logstash pipeline not processing any data?

Check the Logstash logs at /var/log/logstash/logstash-plain.log. Common issues: wrong input configuration, permissions on the input file, or errors in the filter plugins.

Related

• Gaming Merch Drops & Restocks: How They Work
• Galaxy S6 Boot Loop: How to Fix It
• Yandex Bot Traffic & Troll Farm Claim: Not Proven