This project implemented a lambda architecture for analyzing domestic flight data in the US from 2009 to 2020. It used Apache Spark for batch processing, Spark Streaming for real-time analysis, and SVM models to predict flight cancellations and delays, with Docker for cluster management and Grafana for real-time visualization.
python
docker
kafka
spark
cassandra
apache-spark
influxdb
bigdata
jupyter-notebook
spark-streaming
portainer
lambda-architecture
spark-sql
spark-mllib
hadoop-hdfs
apache-hadoop
apache-mllib
roma-tre-university
flights-data
flights-delays
-
Updated
Jul 28, 2023 - Jupyter Notebook