ELFK (elasticsearch, logstash, filebeat, kibana) Log analysis solution for nginx server log and application log
- First Install Elasticsearch, Logstash, Kibana, Filebeat
- Run Elasticsearch and Kibana
- Setup Logstash
- Run Logstash and Filebeat
6.3.5
Filebeat collects the following log files:
- Nginx:
- access.log
- error.log
- Application:
- appplication.log contains log level DEBUG, INFO, ERROR
- error.log only contains log level ERROR
Convert unstructured logs into structured log files.
- Use spring boot default log format(Unstructured log)
- Use JSON format application log(Structured log)
- Start the elk stack, navigate to the elk-configs folder
./filebeat -e -c filebeat.yml -d "publish"
- Start the logstash
./logstash -f ../config/logstash.conf
- Start the kibana
./kibana --allow-root
- Start the logging application
mvn spring-boot:run
- Login to kibana at https://localhost:5601
- Import indexes and other ELK config using elk-configs/elkSavedObjects.json