Senior Data Engineer - #1630

Key Responsibilities • Design, build and operate scalable data architectures and pipelines for ingesting, transforming and serving data across diverse source systems and use cases. • Develop robust data models and reusable data capabilities for applications, analysts, data scientists and other data consumers. • Apply proven architectural and engineering practices to improve the reliability, security, observability, performance and maintainability of data systems. • Evaluate technologies and architectural approaches, make sound technical trade-offs, and contribute to the evolution of the Data Programmers architecture and engineering standards. • Champion modern software engineering practices (automated testing, code review, CI/CD, infrastructure-as-code) and help the team consistently meet these standards through review and coaching. • Work cross-functionally with engineers, Product Managers, Data Scientists, analysts and users, while providing technical leadership through design reviews, mentoring and knowledge sharing. What we are looking for: • Strong software engineering fundamentals and proficiency in Python and SQL, with hands-on experience building and operating complex production data systems. • Strong experience in enterprise data architecture and engineering, with the ability to apply established patterns and practices to new technical problems. • Experience designing data pipelines and data models, with a strong understanding of data warehouses, data lakes and lake house architectures. • Experience with cloud platforms, preferably AWS, and modern data warehouse or data platforms such as Redshift, Snowflake, Databricks, BigQuery or equivalent. • Experience with data orchestration, transformation and modelling using modern engineering approaches and tools. • Strong understanding of production engineering practices including testing, CI/CD, monitoring, troubleshooting and data quality. Good to have: • Hands-on experience with workflow orchestration tools such as Apache Airflow or equivalent. • Experience with transformation and analytics engineering frameworks such as dbt or equivalent. • Experience with distributed data processing technologies such as Apache Spark. • Deep experience with AWS data services and cloud infrastructure. • Familiarity with BI and analytics tools such as Tableau, Power BI or equivalent. • Experience with infrastructure-as-code, data observability, metadata, catalogue or lineage capabilities. • Experience working with sensitive or regulated data, and an interest in using technology and data for public good.