Senior Databricks Solution Architect

We are seeking a Senior Databricks Architect to design, implement, and optimize enterprise-scale data platforms using the Databricks Lakehouse Platform . The ideal candidate will have deep expertise in Databricks architecture, Apache Spark, Delta Lake, data engineering, cloud platforms, and enterprise data solutions. Key Responsibilities • Design and architect Databricks Lakehouse solutions for large-scale enterprise data platforms. • Lead end-to-end Databricks implementation, including architecture, development, deployment, and optimization. • Define Databricks workspace architecture, cluster strategies, security models, and operational standards. • Design and develop scalable data pipelines using Databricks, Apache Spark, PySpark, SQL, and Delta Lake . • Implement data ingestion frameworks supporting batch and real-time data processing. • Design Delta Lake architectures including: • Delta tables • Medallion architecture (Bronze/Silver/Gold layers) • Data quality frameworks • Schema evolution and optimization • Perform Databricks performance tuning: • Cluster optimization • Spark job optimization • Query tuning • Cost optimization • Implement Databricks governance and security using: • Unity Catalog • Access controls • Data lineage • Audit capabilities • Lead migration projects from legacy data platforms to Databricks Lakehouse. • Define best practices for Databricks CI/CD, DevOps, and environment management. • Collaborate with data engineers, data scientists, analysts, and enterprise architects. • Provide technical guidance, architecture documentation, and design reviews. Skills & Experience • Strong experience in data engineering, big data, or solution architecture. • Strong hands-on experience with Databricks Platform . • Expert knowledge of: • Apache Spark • PySpark / Scala • Delta Lake • SQL • Data Lakehouse architecture • ETL/ELT development • Experience designing and delivering enterprise Databricks solutions. • Strong experience with cloud platforms: • Azure Databricks (preferred) • AWS Databricks • Google Cloud Databricks • Experience with: • Databricks Workflows • Jobs orchestration • Notebooks • Repos • Cluster management • Unity Catalog Preferred Skills • Databricks Certified Data Engineer Professional / Databricks Certified Architect. • Experience with: • MLflow • Databricks Machine Learning • Feature Store • Model deployment • MLOps • Knowledge of streaming technologies: • Structured Streaming • Kafka • Experience with data governance, compliance, and enterprise security frameworks. Key Competencies • Strong Databricks architecture and implementation expertise. • Ability to design scalable and cost-efficient Lakehouse solutions. • Strong troubleshooting and performance optimization skills. • Ability to lead technical discussions with enterprise stakeholders.