Senior AI Engineer

Job Description: What will you do: 1. Experimentation & Evaluation • Understand the business problem, POC objectives, and evaluation metrics. • Design experiments to test different model configurations, prompts, or retrieval strategies. • Analyse Gen AI outputs for quality, accuracy, and alignment with requirements; identify common failure modes (hallucination, bias, irrelevant answers, factual errors). • Support SMEs in defining ground truth benchmarks for evaluation. 2. Data Preparation & Pipelines • Profile and clean sample datasets for experimentation (lightweight data prep). • Build and test simple pipelines for data ingestion, prompt construction, and output evaluation. 3. Gen AI & Agentic Techniques • Work with foundation models via AWS Bedrock, Google Vertex AI, or Azure AI Foundry depending on engagement cloud posture. • Apply working knowledge of China-origin models (DeepSeek, Qwen, GLM) as increasingly relevant, cost-effective alternatives. • Apply agentic orchestration frameworks such as AWS Strands, LangGraph, or equivalent, for designing and testing multi-step agent workflows. • Apply prompt strategies, prompt engineering patterns, and RAG design (chunking, embeddings, retrieval evaluation); support ingesting/vectorising content to knowledge bases. • Provide insights and recommendations to improve model performance in quick iterations, including fine-tuning approaches where applicable. 4. FDE & Development/Maintenance Coverage • During FDE engagements: rapidly test candidate models, prompts, and retrieval strategies, giving the team fast, evidence-based go/no-go signals. • During system development & maintenance engagements: support ongoing model/prompt tuning and monitoring as applications move toward production. 5. Collaboration • Collaborate with developers on integrating models into the POC workflow, and work closely with PM, devs, and SMEs to refine data and prompts. • Partner with the Data Scientist on evaluation methodology where classical statistical baselines are in play, and with the AI/LLM Specialist when an engagement moves toward production-grade evaluation. • Document experiments briefly but clearly (hypothesis → result → conclusion). Role Levels We Are Hiring For We are hiring at two levels for this role. All responsibilities above apply to both; the distinction is in scope of ownership, years of experience, and seniority of judgement expected. AI Engineer • 4–5 years of hands-on experience in AI/ML or Gen AI engineering. Runs experiments and prototypes independently within a defined POC/POV scope, under guidance from a Senior AI Engineer or AI/LLM Specialist. • Executes rapid experimentation cycles for one engagement at a time; escalates ambiguous evaluation calls to senior team members. Senior AI Engineer • 6+ years of hands-on experience, including prior ownership of experimentation strategy for complex or ambiguous problem statements. Sets the experimentation approach across multiple engagements and mentors junior AI Engineers. • Advises PMs and stakeholders directly on feasibility and experimentation trade-offs; represents technical experimentation findings in client conversations. Qualifications The ideal candidate should possess: • 4+ years hands-on experience in AI/ML or Gen AI engineering (see Role Levels for the split between AI Engineer and Senior AI Engineer). • Understanding of Gen AI concepts (tokenization, embeddings, RAG, prompting, evaluation). • Familiar with at least one major cloud AI service (AWS Bedrock, Google Vertex AI, or Azure AI Foundry); working knowledge of others a plus. • Working knowledge of the China AI model landscape (DeepSeek, Qwen, GLM) a strong plus. • Familiarity with agentic orchestration frameworks (AWS Strands, LangGraph, or equivalent), for designing and testing multi-step agent workflows. • Ability to do rapid experimentation rather than perfect models. • Basic proficiency in Python and Gen AI tools (e.g., model SDKs, vector DBs). • Analytical mindset: can quantify subjective output (accuracy, relevance, readability). • Good data wrangling skills to prepare small datasets quickly. Preferred Qualifications • Generative AI Leader or Machine Learning Engineer certification, or equivalent. • Exposure to LLMOps practices (model monitoring, versioning) for production transition. • Familiarity with model fine-tuning techniques. • Exposure to regulated government cloud environments. Tech Stack (Illustrative) • Languages: Python • LLM Runtime: AWS Bedrock, Google Vertex AI, Azure AI Foundry; DeepSeek/Qwen/GLM (China stack) • Agentic Frameworks: AWS Strands, LangGraph • Data & Eval: Model SDKs, vector DBs, pandas/Jupyter-style tooling for experimentation