Data Scientist/Engineer - #1639

Role Overview We are seeking a skilled and motivated Data Engineer to design, develop, and maintain scalable data platforms and pipelines that support the agency’s digital services, analytics, reporting, and data-driven decision-making. The successful candidate will work closely with data scientists, analysts, application teams, cybersecurity teams, and business stakeholders to ensure that data is accurate, reliable, secure, governed, and accessible for authorised use. The role will involve working with modern cloud and data technologies while adhering to Singapore public-sector requirements relating to data governance, cybersecurity, privacy, and information management. Key Responsibilities Data Engineering & Development • Design, develop, and maintain robust ETL/ELT data pipelines for structured and unstructured data. • Build and optimise data ingestion, transformation, integration, and processing workflows. • Develop scalable data platforms, data warehouses, data lakes, and/or lakehouse solutions. • Integrate data from APIs, databases, enterprise applications, files, and other source systems. • Implement automated data validation, quality checks, monitoring, and reconciliation processes. • Optimise data pipelines and queries for performance, scalability, and cost efficiency. • Develop reusable data engineering components and establish coding and development standards. Data Governance & Security • Ensure data pipelines and platforms comply with applicable government data governance, security, privacy, and information management requirements. • Implement appropriate access controls, encryption, audit logging, and data protection mechanisms. • Support data classification, lineage, metadata management, retention, and lifecycle management. • Work with data governance and cybersecurity teams to identify and mitigate data-related risks. • Maintain appropriate technical documentation for datasets, pipelines, interfaces, and data flows. Cloud & Technology • Develop and operate data solutions using cloud and/or on-premises technologies. • Work with technologies such as Python, SQL, Spark, Airflow, Kafka, Databricks, Snowflake, Azure, AWS, or Google Cloud, depending on the agency's technology environment. • Implement CI/CD, infrastructure automation, testing, and deployment practices for data solutions. • Monitor system performance and troubleshoot data pipeline and platform issues. Stakeholder Collaboration • Work with business users and domain experts to understand data requirements and translate them into technical solutions. • Collaborate with data analysts and data scientists to provide reliable and well-structured datasets. • Work with application and integration teams on APIs and upstream/downstream system dependencies. • Communicate technical concepts clearly to both technical and non-technical stakeholders. • Participate in technology architecture, solution design, and technical review sessions. Operational Excellence • Monitor production data pipelines and resolve incidents within agreed service levels. • Perform root-cause analysis and implement preventive measures for recurring issues. • Develop operational dashboards, alerts, and monitoring mechanisms. • Participate in disaster recovery, business continuity, and technology resilience activities where required. • Continuously evaluate and adopt appropriate data engineering practices and technologies. Requirements Essential • Degree or diploma in Computer Science, Information Technology, Data Engineering, Engineering, Mathematics, Statistics, or a related discipline. • 3–6 years of relevant experience in data engineering, data platform development, ETL/ELT, or a related technical role. • Strong programming skills in Python, Java, Scala, or a similar language. • Strong SQL skills and experience with relational databases. • Hands-on experience designing and implementing data pipelines and data integration solutions. • Experience with data warehousing, data lakes, or modern data platforms. • Experience with cloud technologies and/or enterprise data platforms. • Understanding of data security, data governance, and data quality principles. • Strong analytical, problem-solving, and troubleshooting skills. • Good written and verbal communication skills. Good to Have • Experience with Apache Spark, Databricks, Airflow, Kafka, dbt, Snowflake, or equivalent technologies. • Experience with AWS, Microsoft Azure, or Google Cloud. • Experience implementing CI/CD and DevOps practices for data platforms. • Experience with data modelling and dimensional modelling. • Knowledge of metadata management, data lineage, master data management, and data cataloguing. • Experience working in a regulated, public-sector, financial-services, or other highly governed environment. • Relevant professional certifications in cloud, data engineering, or related technologies.