Data Lead

We are looking for a Data Lead – Data Engineering to drive technical diagnostics, optimization, and stabilization of enterprise data platforms supporting large-scale ETL and batch processing systems. Key Responsibilities • Strong expertise in: Azure Data Factory (ADF),Databricks (PySpark / Spark SQL preferred),SSIS and SQL Server • Hands-on experience with: Batch orchestration tools (Control-M / Autosys),ETL/ELT pipeline design and optimization • Strong experience in: SQL development and performance tuning, Troubleshooting and debugging complex data pipelines • Familiarity with: Monitoring and alerting tools (Azure Monitor, Log Analytics) • Exposure to large-scale data processing and batch system • Review and optimize ADF pipelines, SSIS packages, and Databricks workflows for performance, scalability, and • Analyze batch scheduling and orchestration frameworks (Control-M / Autosys), including dependencies, triggers, and SLA adherence • Investigate and resolve pipeline failures, job delays, and runtime issues, ensuring faster recovery and minimal business • Evaluate source-to-target integration flows, SQL logic, and transformation layers for efficiency and correctness • Define and improve error handling, retry mechanisms, restart ability, and reprocessing strategies across ETL and batch • Optimize runtime performance of data pipelines, including query tuning and batch execution improvements • Establish and enhance monitoring and alerting frameworks using tools like Azure Monitor, Control-M dashboards, and custom logging solutions • Identify automation opportunities to reduce manual intervention and improve operational efficiency • Ensure data quality, consistency, and reliability across ETL processes and reporting outputs • Support modernization initiatives, including migration from legacy ETL (SSIS) to cloud-based platforms like Databricks • Collaborate with data engineers, architects, business, and support teams to drive improvements and ensure stable production systems