Principal Site Reliability Engineer, Infrastructure & Platform

Responsibilities: Infrastructure Automation & Configuration Management • Author, maintain, and refactor Ansible playbooks and roles across a large-scale multi-datacenter inventory, covering the full lifecycle from bare-metal provisioning to application deployment • Develop and improve CI/CD pipelines (GitLab CI) for infrastructure automation, including linting, testing, and staged rollout across regions • Manage secrets lifecycle using HashiCorp Vault, including AppRole authentication, secret rotation, and PKI integration • Maintain CMDB/IPAM accuracy in NetBox as a source of truth for all infrastructure assets Compute & Virtualization • Deploy and manage Proxmox VE hypervisor clusters on bare-metal HPE hardware, including cluster formation, OVS networking, ZFS storage, and VM replication • Provision and lifecycle-manage virtual machines using cloud-init, QCOW2 images, and Proxmox API automation • Manage physical server provisioning end-to-end via HPE iLO (firmware updates, SPP deployment, OS installation via virtual media) Container & Kubernetes Platforms • Manage self-hosted Kubernetes clusters on-premises, including control plane operations, node provisioning, workload deployment, and upgrade management • Operate Docker-based workloads on infrastructure VMs using compose-driven deployments and container health monitoring • Maintain container image pipelines and registry infrastructure (Azure Container Registry or AWS ECR) Cloud Platforms • Engineer and maintain infrastructure on AWS and Azure, integrating cloud resources with on-premises systems (DNS, monitoring, identity, networking) • Apply cloud cost awareness, security best practices, and IaC principles (IAM, security groups, networking, storage) across AWS and Azure environments Networking & Core Services • Operate and troubleshoot core distributed services including authoritative DNS (BIND9), recursive DNS (Unbound), load balancing (HAProxy), and high-availability VIPs (Keepalived/VRRP) • Maintain directory services (OpenLDAP master-replica topology) and AAA infrastructure (FreeRADIUS) used for SSH, VPN, and network device authentication • Manage OVS-based network configurations, VLAN topologies, and bonded NIC arrangements across hypervisor fleets Observability & Security • Maintain and extend monitoring infrastructure (Prometheus, Observium) across a global fleet including SNMP polling, metrics collection, and alerting • Manage centralised log aggregation pipelines (Fluentbit) and ensure log delivery integrity across DCs • Operate runtime security tooling (Falco) and file integrity monitoring (AIDE) in production environments • Support PCI-DSS compliance activities including CIS hardening, audit logging (auditd), and participation in control reviews Reliability & Incident Response • Participate in a 24x7 on-call rotation, responding to and leading production incident resolution • Conduct blameless post-mortems and drive remediation of root causes through automation and system improvements • Define and track SLOs/SLIs for critical infrastructure services • Identify and address single points of failure; design and implement HA improvements Requirements: • Strong Linux systems administration skills (RHEL/CentOS preferred) including systemd, networking, storage, kernel tuning, and package management • Proficiency with Ansible (or similar tool) for large-scale configuration management, including role design, inventory management, and CI/CD integration • Hands-on experience with at least one hypervisor platform, preferably Proxmox VE, Harvester (Kubevirt) or similar (VMware vSphere, KVM) • Production experience operating on-premise Kubernetes clusters (rke2, k3s, etc) • Practical AWS or Azure experience including compute, networking (VPC/VNet, security groups, DNS), IAM, and managed services • Solid understanding of networking fundamentals: VLANs, bonding/LAG, routing, BGP concepts, DNS, load balancing, and firewall rule management • Experience with secrets management platforms (HashiCorp Vault or equivalent) • Familiarity with PCI-DSS requirements as they apply to infrastructure -- hardening standards (CIS benchmarks), audit logging, access control • Experience writing and maintaining CI/CD pipelines (GitLab CI, GitHub Actions, or equivalent) • Demonstrable on-call experience and comfort leading incident response in a global production environment • 7+ years of experience in a Site Reliability Engineering, DevOps, or Infrastructure Engineering role in a production environment