Data Engineer

• Employment type - 1-year contract (Renewable) • Eligibility: Singaporean ONLY (require security clearance) • Location: Ang Mo Kio • Working Days/ Hours: Mon-Fri: 8.30am to 6pm Role Overview As a Data Engineer, you will design, build, and operate scalable, secure, and high-performance enterprise data platforms and ETL pipelines that enable analytics, reporting, and data-driven as well as AI-driven decision-making. You will play a key technical role in delivering end-to-end data pipelines, analytics platforms, integration solutions, and generative AI solutions across hybrid and cloud environments, including Government Commercial Cloud (GCC) and MINDEF Commercial Cloud (MCC). You will work closely with data and solution architects, business stakeholders, and infrastructure teams to translate requirements into robust, production-grade data solutions. Job Responsibilities 1. Data Engineering & Platform Design • Design and develop scalable and reliable data pipelines. • Build enterprise data platforms including data warehouses, data lakes, lakehouses, and AI-ready data platforms that support GenAI workloads (e.g., real-time ingestion, vector search, semantic retrieval). • Implement cloud, hybrid, and on-premises data architectures aligned with enterprise standards. 2. Data Integration & ETL/ELT • Design, develop, optimize, and maintain scalable ETL/ELT data pipelines using PySpark, Spark SQL, and modern data processing frameworks. Experience with Talend is preferred. • Build and support data pipelines that enable Generative AI, Large Language Models (LLMs), Retrieval-Augmented Generation (RAG), and AI analytics use cases. • Integrate structured, semi-structured, and unstructured data sources. • Implement and maintain data quality, validation, and error-handling frameworks. 3. Application Migration Testing, Validation & Test Automation • Analyze migrated applications, application flows, interfaces, APIs, databases, and integration dependencies to determine testing scope and coverage. • Develop key test assets, including functional, integration, regression, and production verification test cases, test data requirements, and Requirements Traceability Matrix (RTM). • Execute and automate testing where feasible, including smoke, functional, regression, integration, API, and batch validation testing. • Support application modernization initiatives by validating business functionality, interfaces, and system integrations post-migration. • Leverage AI-assisted tools to accelerate application discovery, test scenario generation, test automation, and coverage analysis. • Perform security fix verification and regression testing to validate remediation changes, ensure business continuity, and support defect management and closure. • Produce testing deliverables including Test Strategy, Test Suites, Test Execution Reports, Defect Reports, Coverage Reports, and UAT Readiness Reports. 4. Storage & Data Management • Design and manage data storage solutions with a strong emphasis on modern data platforms, data quality, and data governance. • Implement and maintain data lifecycle management and cost-optimization strategies. • Ensure data lineage, traceability, and auditability. 5. Analytics Enablement & BI Integration • Engineer analytics-ready data models for reporting and self-service analytics. • Enable BI Dashboards, Reports, and analytics through performant and governed data pipelines. 6. Security, Governance & Compliance • Ensure compliance with government security and privacy requirements. • Implement and maintain encryption, access controls, and secure data handling practices. • Implement and maintain security policies and procedures for the system. Job Requirements Education • Bachelor’s or Master’s degree in Computer Science, Engineering, Data Science, or a related field. Core Technical Skills • Strong hands-on data engineering experience. • Functional, regression, integration, and production validation testing. • API, interface, database, MQ (IBM MQ preferred), and batch dependency analysis. • Relational databases (Oracle, SQL Server, PostgreSQL, DB2). • CI/CD-integrated testing frameworks. • Expertise in ETL/ELT with Talend; hands-on experience preferred. • Experience with data warehouses, data lakehouses, data lakes, and object storage systems. Hands-on experience with Databricks preferred. • Strong SQL, Java, and Python (PySpark) skills. Experience & Security Requirements • Minimum 3+ years of experience in enterprise data engineering roles • Prior CAT 1 / G50 security clearance (mandatory) • Previous experience delivering MINDEF projects (mandatory)