Lead Data Engineer

SimplifyNext is a Singapore-headquartered digital engineering firm, with offices in Thailand and Malaysia, working across three converged areas: agentic AI, AI-enabled application modernisation, and intelligent automation. We bring strong business process consulting together with deep technology skills and modern delivery practices – product-centric, agile and AI-assisted. We build bespoke applications and AI systems, and we go deep on the platforms our clients run on, including Microsoft, AWS, ServiceNow, Databricks and UiPath. That combination of engineering depth and platform depth is what our clients come to us for. We build and modernise the systems people depend on daily. Our 300+ practitioners are multi-disciplinary by design – business consultants, software engineers, architects, AI engineers and designers working on the same delivery, not handing off between silos. We focus on delivering outcomes for clients, building strong careers for our people, and staying ahead of the technology curve. We hire because we are growing. Role Purpose The Data Engineer designs, builds and deploys scalable data lakehouse platforms for our clients - architecting end-to-end pipelines, establishing data quality frameworks, and delivering production-ready solutions that form the foundation of an organisation's data infrastructure. You will work across modern lakehouse stacks (Databricks, Spark, Microsoft Fabric or AWS) alongside architects, AI engineers and business analysts on the same delivery, not in a separate data silo. What You'll Own: Data Pipeline & Architecture Design • Design and implement scalable ETL/ELT pipelines on modern cloud data platforms - AWS (Glue, Step Functions, Lambda, S3), Databricks, Spark or Microsoft Fabric. • Architect and build data lakehouse solutions using open table formats such as Apache Iceberg, Delta Lake or S3 Tables, including schema evolution, partition evolution and ACID transactions. • Optimise pipelines for performance, cost and reliability at enterprise scale. Data Quality & Governance • Define, implement and maintain automated data quality validation frameworks, with metrics and monitoring for accuracy, completeness and consistency. • Enforce data governance standards, access controls, lineage and PII handling across the platform, appropriate to public sector and regulated client environments. Application Development & Deployment • Write production-quality code and deploy solutions on cloud infrastructure, including Government Commercial Cloud (GCC) environments. • Apply DevOps practices - CI/CD pipelines and infrastructure-as-code (Terraform, CloudFormation or equivalent) - for repeatable and auditable deployments. AI & Analytics Enablement • Prepare and serve data for AI and agentic workloads - feature pipelines, retrieval sources and knowledge bases. • Support analytics and BI teams with reliable, well-modelled datasets and clear semantics. Collaboration, Handover & Day 2 Operations • Work across cross-functional teams and communicate technical designs clearly to non-technical stakeholders. • Produce thorough technical documentation to support handover to Day 2 operations teams. • Own the pipelines you build in production - monitoring, incident response, root-cause fixes and cost tuning as data volumes grow. AI-Assisted Delivery We deliver with an in-house AI workbench that pairs practitioners and AI across the lifecycle. As a Data Engineer you'll use it to: • Accelerate pipeline scaffolding, transformation logic and test data generation. • Support schema mapping, profiling and technical documentation of source systems. • Assist with code review, data quality rule suggestion and runbook drafting. AI accelerates the work; you own the design decisions, the data correctness and the outcome. What You'll Bring Must-Have: • Degree in Computer Science, Data Engineering, Information Systems or a related field. • 4+ years of hands-on experience in data engineering, ETL/ELT development or data platform roles, building and operating production pipelines. • Strong hands-on experience with at least one modern data platform - Databricks, Apache Spark, Microsoft Fabric or AWS (Glue, Step Functions, Lambda, S3, Redshift). • Proficiency with open table formats and lakehouse architecture - Apache Iceberg, Delta Lake or S3 Tables - including schema evolution, partitioning and ACID transactions. • Strong SQL and proficiency in Python (or Scala/Java) for data processing and automation. • Experience designing data models and warehouse/lakehouse layers for analytics, reporting and AI workloads. • Experience defining and automating data quality validation, monitoring and reconciliation. • Working knowledge of DevOps for data - CI/CD, version control and infrastructure-as-code (Terraform, CloudFormation or equivalent). • Ability to work directly with business and technical stakeholders and communicate technical designs clearly to non-technical audiences. Good-to-Have: • Experience delivering data projects in the Singapore Public Sector, particularly in a Government Commercial Cloud (GCC / GCC+) environment. • Cloud or platform certification - AWS Certified Data Analytics, AWS Certified Solutions Architect, Databricks Data Engineer, or Azure/Fabric Data Engineer. • Exposure to AI/ML workloads - feature pipelines, vector stores, RAG data preparation or MLOps. • Experience with data migration from legacy systems, including reconciliation and cutover. • Familiarity with governance frameworks and PII handling (e.g. masking, tokenisation, Presidio). • Experience with BI and semantic layers (Power BI, Tableau, Looker) and enabling self-service analytics. This role is not a fit if: • You want to work only on ad-hoc analysis and dashboards rather than building and running pipelines. • You see data quality, testing and documentation as someone else's job. • You're uncomfortable supporting what you build once it's in production. • You prefer a sin…