Pinned Loading

1. aws_data_pipeline aws_data_pipeline Public

   Full data pipeline and data engineering project using AWS services (MSK, Kafka, Spark Stream/SQL, Elastic Stack, Iceberg, Glue, Athena, Streamlit, etc.)

   Python 1

2. alpha_lakehouse alpha_lakehouse Public

   Integrated end-to-end data project using lambda and data lakehouse architecture to compile financial data

   Python 3 1

3. databricks_cross_org_data_replication databricks_cross_org_data_replication Public

   Data replication and lineage management mini-project using Azure Databricks

   Python 3

4. stocks_udf stocks_udf Public

   Using Spark Vectorized UDFs and AI tools on stock price data

   Jupyter Notebook 2 1

5. databricks_streaming_iot databricks_streaming_iot Public

   Spark structured streaming mini-project for IoT devices using Databricks

   Python 1

6. data-lake_comparison data-lake_comparison Public

   Simple mini personal project comparing Delta Lake and Iceberg in terms of usage and performance when handling both high and low cardinality datasets

   Jupyter Notebook