502 Bad Gateway After Installing ELK Stack: Fix

A 502 Bad Gateway error after installing the ELK Stack (Elasticsearch, Logstash, Kibana) almost always means Kibana cannot reach Elasticsearch, or a reverse proxy (Nginx, Apache) cannot reach Kibana. The fix is to check each service’s status, verify the Elasticsearch URL and port in Kibana’s configuration, and confirm the reverse proxy’s upstream settings match the actual listening address.

What the ELK Stack Components Do

The ELK Stack is a trio of open-source tools used for searching, analyzing, and visualizing log data. Elasticsearch stores and indexes the data, Logstash ingests and transforms it before sending it to Elasticsearch, and Kibana provides a web interface for querying and graphing that data. A 502 error typically appears in the browser when Kibana itself is unreachable, or when Kibana’s backend request to Elasticsearch fails and it returns a bad gateway response.

Each component runs as a separate service, usually on different ports:

• Elasticsearch: default port 9200 (HTTP API)
• Kibana: default port 5601 (web UI)
• Logstash: default port 5044 (Beats input) or 5000 (syslog)

If you installed all three on the same machine, a 502 usually points to Kibana’s configuration, not Logstash. Logstash rarely causes a 502 because it is not in the browser request path.

Why a 502 Happens After Installation

A 502 Bad Gateway is an HTTP status code that means a server acting as a gateway or proxy received an invalid response from the upstream server. In the ELK context, there are two common scenarios:

1. Kibana cannot connect to Elasticsearch. Kibana’s elasticsearch.hosts setting points to the wrong URL, port, or protocol (http vs https). If Elasticsearch is not running or is bound to localhost only, Kibana will show a 502.
2. Nginx or Apache cannot connect to Kibana. If you put a reverse proxy in front of Kibana, the proxy’s proxy_pass directive might point to the wrong port or IP. If Kibana is bound to 127.0.0.1 and the proxy tries to reach it via a hostname, the connection fails.

Another subtle cause is Elasticsearch’s network.host setting. By default, Elasticsearch binds to localhost only. If Kibana runs on a different server, it cannot reach Elasticsearch, and Kibana returns a 502 to the browser.

Step-by-Step Troubleshooting

Start with the most direct checks, then move to configuration files.

1. Check service status. Run systemctl status elasticsearch, systemctl status kibana, and systemctl status nginx (or the equivalent for your OS). Look for “active (running)� or error messages. If any service is dead, start it and check logs.
2. Test Elasticsearch directly. From the Kibana server, run curl http://localhost:9200. You should see a JSON response with a cluster name. If not, Elasticsearch is not listening on that interface.
3. Test Kibana directly. Run curl http://localhost:5601/api/status. A 200 response means Kibana is up. If you get a connection refused, Kibana is not running or is bound to a different interface.
4. Check Kibana’s config file. In /etc/kibana/kibana.yml, verify server.host (usually 0.0.0.0 or localhost) and elasticsearch.hosts (usually http://localhost:9200). If you changed Elasticsearch’s port, update this.
5. Check the reverse proxy config. For Nginx, look at /etc/nginx/sites-available/default or your site config. The proxy_pass should point to http://127.0.0.1:5601 or http://localhost:5601. For Apache, check the ProxyPass directive in the virtual host.
6. Restart all services. After any config change, run systemctl restart elasticsearch kibana nginx (or the relevant service manager). Then test again.

If the error persists, check the logs:

• Kibana log: /var/log/kibana/kibana.log
• Elasticsearch log: /var/log/elasticsearch/elasticsearch.log
• Nginx error log: /var/log/nginx/error.log

Common Misconfigurations and Fixes

Several mistakes appear repeatedly in ELK installations. Here are the most frequent ones and how to correct them.

• Elasticsearch bound to localhost only. If your elasticsearch.yml has network.host: 127.0.0.1, only local connections work. Change it to 0.0.0.0 if Kibana is on another machine, but be aware this exposes the API to the network – use a firewall.
• Kibana’s elasticsearch.hosts uses https. If Elasticsearch is not configured for TLS, use http:// instead. Mixing protocols causes a 502.
• Reverse proxy tries to reach Kibana via a hostname that resolves to a different IP. Use 127.0.0.1 instead of the machine’s hostname in proxy_pass.
• Port conflict. Another process might be using port 5601 or 9200. Run ss -tlnp | grep 5601 to see what’s listening.
• Elasticsearch still starting. Elasticsearch can take 20-30 seconds to become ready. If you start Kibana immediately, it might fail initially. Wait and retry.

Reverse Proxy vs Direct Access

Many users install Nginx or Apache as a reverse proxy to provide HTTPS or a friendly URL for Kibana. This adds a layer where a 502 can occur. If you access Kibana directly at http://server-ip:5601 and it works, the problem is in the proxy configuration. If direct access also fails, the problem is between Kibana and Elasticsearch.

For Nginx, a typical working configuration looks like this:

location / {
    proxy_pass http://127.0.0.1:5601;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection 'upgrade';
    proxy_set_header Host $host;
    proxy_cache_bypass $http_upgrade;
}

For Apache, you would use ProxyPass / http://127.0.0.1:5601/ and ProxyPassReverse / http://127.0.0.1:5601/. Ensure the proxy modules are enabled (proxy, proxy_http).

When the 502 Is Actually a Kibana Error

Sometimes the browser shows a 502 but the real issue is a Kibana error page. If Kibana cannot connect to Elasticsearch, it might return a 503 or a generic error that a proxy translates to 502. Check Kibana’s logs for lines like “connect ECONNREFUSED 127.0.0.1:9200� or “license is not available�. The latter often happens when Elasticsearch is running but Kibana’s user lacks permissions. In that case, you need to create a Kibana user with the appropriate roles in Elasticsearch, or temporarily set xpack.security.enabled: false in elasticsearch.yml to test (not recommended for production).

Frequently Asked Questions

Why does my 502 only happen when I use the domain name, not the IP?

This usually points to a reverse proxy configuration issue. The proxy might be trying to reach Kibana via a hostname that resolves to a public IP, but Kibana is bound to localhost. Change the proxy_pass to use 127.0.0.1, or bind Kibana to 0.0.0.0 and restrict access with a firewall.

Can a firewall cause a 502 error?

Yes. If a firewall blocks port 9200 between Kibana and Elasticsearch, Kibana cannot fetch data and may return a 502. Similarly, if the reverse proxy cannot reach port 5601 due to firewall rules, you get a 502. Check your firewall rules (ufw, firewalld, iptables) to allow traffic on the necessary ports.

What if Elasticsearch is on a different server?

In that case, Kibana’s elasticsearch.hosts must point to the Elasticsearch server’s IP or hostname, e.g., http://192.168.1.10:9200. Elasticsearch must also be configured to listen on a network interface that is reachable (not just localhost). You may need to set network.host: 0.0.0.0 and configure discovery.seed_hosts appropriately.

How do I know if it’s a Kibana or Elasticsearch problem?

Run curl http://localhost:9200 from the Kibana server. If that returns JSON, Elasticsearch is fine. Then run curl http://localhost:5601/api/status. If that returns JSON with a status, Kibana is fine. If both are fine, the issue is in the reverse proxy.

Do I need to restart everything after changing a config file?

Yes. Most ELK components read their configuration only at startup. After editing kibana.yml, elasticsearch.yml, or Nginx config, restart the respective service. For Nginx, you can often use nginx -s reload instead of a full restart.

Can a 502 be caused by low memory?

Yes. Elasticsearch and Kibana are memory-hungry. If the system runs out of RAM, Elasticsearch might crash or stop responding, causing Kibana to return a 502. Check free -h and the Elasticsearch logs for “OutOfMemoryError�. Consider increasing memory or reducing the heap size in jvm.options if it is set too high.

Related

• FiveM Server Hosting for GTA V Enhanced: Key Facts
• Fix Apache AH00558: Could Not Determine Server’s Domain Name
• Google TV Default Launcher vs Lightweight Launcher: What Changes?