Built an end-to-end cloud data pipeline (AWS S3, Snowflake, dbt, Airflow) processing 10M+ orders, 23M+ order items, and 300K customer reviews. Implemented keyless S3-to-Snowflake ingestion via IAM storage integration and a Medallion Architecture (Bronze/Silver/Gold) in dbt with incremental MERGE-based loading, cutting reprocessing time significantly. Added an LLM-powered enrichment pipeline (Gemini) converting unstructured reviews into structured sentiment/topic data, a RAG system for natural-language Q&A with cited sources, and a Text-to-SQL system with query safety validation. Orchestrated the full workflow with Dockerized Airflow and built Streamlit dashboards and AI chat apps for business reporting, backed by automated dbt data quality tests.
Built an end-to-end pipeline that automatically ingests, validates, and loads NYC TLC taxi trip data into BigQuery every month — with zero manual intervention, safe reruns, automatic failure alerts, and a live monitoring dashboard.
An end-to-end batch data pipeline that ingests raw e-commerce data from AWS S3, processes it through a Medallion Architecture (Bronze → Silver → Gold) using PySpark on Databricks, and models it into a Snowflake schema of analytics-ready fact and dimension tables — fully orchestrated with Databricks Jobs.