Data Engineer (Azure, Databrick)

Key Responsibilities Data Management & Transformation • Design, develop and maintain data pipelines that extract data from business applications, databases and external systems. • Implement data quality checks and validation processes to ensure data accuracy, consistency and completeness. • Perform data cleansing, aggregation, filtering, transformation and enrichment. • Establish and maintain governance frameworks for data and algorithms used in analytics, automated decision-making and analytical applications. • Develop and maintain logical and physical data models, including data structures, relationships and constraints. • Ensure data solutions comply with security, governance and organisational best practices. Data Platform Optimisation & Automation • Develop and maintain CI/CD pipelines for data deployments and cloud resource provisioning. • Optimise data pipelines and processing workflows for performance, scalability and efficiency. • Optimise data models and algorithms for data quality, security, governance, performance and scalability. • Monitor data warehouse and ETL platform capacity, including capacity planning and forecasting. • Monitor and tune data platforms, troubleshoot performance bottlenecks and implement optimisation techniques such as caching and indexing. • Monitor platform resource and credit consumption and perform regular housekeeping activities. • Support the deployment, integration and maintenance of AI/ML solutions within the data platform. Collaboration & Business Enablement • Partner with the Data Lead and business users to manage and maximise the value of data as a business asset. • Work with stakeholders to understand data requirements and translate them into scalable technical solutions. • Guide business users in the creation and maintenance of reports and dashboards. • Collaborate with technology, data and business teams to improve data quality, accessibility and usability. • Any other ad-hoc duties as assigned by supervisor Requirements • 5+ years of experience in data management or data engineering, covering areas such as data integration, data modelling, data quality and platform optimisation. • Hands-on experience designing and implementing data architectures using Azure and Databricks. • Strong programming skills in Python, PySpark and SQL . • Strong experience with relational and non-relational databases , including SQL and NoSQL databases. • Hands-on experience with Azure DevOps and Databricks LakeFlow Jobs , including version control, CI/CD and pipeline deployment. • Experience with data catalogue and governance tools such as Databricks Unity Catalog and Microsoft Purview . • Experience supporting AI/ML model deployment, feature engineering and production inference is an advantage. Interested applicants, please email your resume to Lisa (Lisha) Zhang Email: lisazhang@recruitexpress.com.sg CEI Reg No: R25157448 EA Licence No: 99C4599 Recruit Express Pte Ltd