Production-ready PySpark ETL template for Databricks — medallion architecture, DABs, tests, DQX, CI/CD, and agentic development with Claude Code.

This Repository has like a goal, build generic solution for use on any projects

End-to-end SaaS subscription data platform using AWS, Databricks (Delta Lake), and Snowflake featuring streaming ingestion, CDC/SCD modeling, medallion architecture, and Iceberg-based analytics.

This project built an automated, end-to-end data pipeline from raw CSVs to real-time Databricks dashboards for Rideeasy Cab Aggregator.

This azure databricks project implements a modern data engineering pipeline on Azure using the Medallion Architecture (Bronze → Silver → Gold) to process NYC taxi trip data.

Managing and monitoring jobs in Databricks can be a daunting task, especially as the number of jobs grows. To streamline this process, we can automate the extraction of job metadata and store it in a centralized table for easy access and monitoring.

End-to-end B2B sales pipeline built on Databricks and dbt. Medallion architecture, Delta Lake, automated Workflows, 26/26 dbt tests, and a live AI/BI dashboard across 5,000 deals and 43,160 activities.

Medallion pipeline over the public NYC TLC taxi dataset on Databricks Free Edition.