EngineeringHybrid

Member of Technical Staff (AI Infrastructure Engineer)

Perplexity · Hybrid

Apply

Last seen by MeritLog September 10, 2026Source: AshbySource version: ashby-public-job-posting-v1

MeritLog read this listing from Perplexity's Ashby job board and last checked it on September 10, 2026.

Source: the employer's Ashby job board. Open the original listing for current details.

Job details

Work model: Hybrid
Salary: Not listed by source
Location: London

What the role asks for

What you'd do

Design, deploy, and maintain scalable Kubernetes clusters for AI model inference and training workloads
Manage and optimize Slurm-based HPC environments for distributed training of large language models
Develop robust APIs and orchestration systems for both training pipelines and inference services
Implement resource scheduling and job management systems across heterogeneous compute environments
Benchmark system performance, diagnose bottlenecks, and implement improvements across both training and inference infrastructure
Build monitoring, alerting, and observability solutions tailored to ML workloads running on Kubernetes and Slurm
Respond swiftly to system outages and collaborate across teams to maintain high uptime for critical training runs and inference services
Optimize cluster utilization and implement autoscaling strategies for dynamic workload demands

What they're asking for

Strong expertise in Kubernetes administration, including custom resource definitions, operators, and cluster managementSkill
Hands-on experience with Slurm workload management, including job scheduling, resource allocation, and cluster optimizationSkill
Experience with deploying and managing distributed training systems at scaleSkill
Deep understanding of container orchestration and distributed systems architectureSkill
High level familiarity with LLM architecture and training processes (Multi-Head Attention, Multi/Grouped-Query, distributed training strategies)Skill
Experience managing GPU clusters and optimizing compute resource utilizationSkill
Expert-level Kubernetes administration and YAML configuration managementSkill
Proficiency with Slurm job scheduling, resource management, and cluster configurationSkill
Python and C++ programming with focus on systems and infrastructure automationSkill
Hands-on experience with ML frameworks such as PyTorch in distributed training contextsSkill
Strong understanding of networking, storage, and compute resource management for ML workloadsSkill
Experience developing APIs and managing distributed systems for both batch and real-time workloadsSkill
Solid debugging and monitoring skills with expertise in observability tools for containerized environmentsSkill
Experience with Kubernetes operators and custom controllers for ML workloadsSkillPreferred
Advanced Slurm administration including multi-cluster federation and advanced scheduling policiesSkillPreferred
Familiarity with GPU cluster management and CUDA optimizationSkillPreferred
Experience with other ML frameworks like TensorFlow or distributed training librariesSkillPreferred
Background in HPC environments, parallel computing, and high-performance networkingSkillPreferred
Knowledge of infrastructure as code (Terraform, Ansible) and GitOps practicesSkillPreferred
Experience with container registries, image optimization, and multi-stage builds for ML workloadsSkillPreferred
Demonstrated experience managing large-scale Kubernetes deployments in production environmentsSkillPreferred
Proven track record with Slurm cluster administration and HPC workload managementSkillPreferred

Parsed by MeritLog from the employer’s own posting. The full description follows below.

Job description

We are looking for an AI Infra engineer to join our growing team. We work with Kubernetes, Slurm, Python, C++, PyTorch, and primarily on AWS. As an AI Infrastructure Engineer, you will be partnering closely with our Inference and Research teams to build, deploy, and optimize our large-scale AI training and inference clusters. RESPONSIBILITIES - Design, deploy, and maintain scalable Kubernetes clusters for AI model inference and training workloads - Manage and optimize Slurm-based HPC environments for distributed training of large language models - Develop robust APIs and orchestration systems for both training pipelines and inference services - Implement resource scheduling and job management systems across heterogeneous compute environments - Benchmark system performance, diagnose bottlenecks, and implement improvements across both training and inference infrastructure - Build monitoring, alerting, and observability solutions tailored to ML workloads running on Kubernetes and Slurm - Respond swiftly to system outages and collaborate across teams to maintain high uptime for critical training runs and inference services - Optimize cluster utilization and implement autoscaling strategies for dynamic workload demands QUALIFICATIONS - Strong expertise in Kubernetes administration, including custom resource definitions, operators, and cluster management - Hands-on experience with Slurm workload management, including job scheduling, resource allocation, and cluster optimization - Experience with deploying and managing distributed training systems at scale - Deep understanding of container orchestration and distributed systems architecture - High level familiarity with LLM architecture and training processes (Multi-Head Attention, Multi/Grouped-Query, distributed training strategies) - Experience managing GPU clusters and optimizing compute resource utilization REQUIRED SKILLS - Expert-level Kubernetes administration and YAML configuration management - Proficiency with Slurm job scheduling, resource management, and cluster configuration - Python and C++ programming with focus on systems and infrastructure automation - Hands-on experience with ML frameworks such as PyTorch in distributed training contexts - Strong understanding of networking, storage, and compute resource management for ML workloads - Experience developing APIs and managing distributed systems for both batch and real-time workloads - Solid debugging and monitoring skills with expertise in observability tools for containerized environments PREFERRED SKILLS - Experience with Kubernetes operators and custom controllers for ML workloads - Advanced Slurm administration including multi-cluster federation and advanced scheduling policies - Familiarity with GPU cluster management and CUDA optimization - Experience with other ML frameworks like TensorFlow or distributed training libraries - Background in HPC environments, parallel computing, and high-performance networking - Knowledge of infrastructure as code (Terraform, Ansible) and GitOps practices - Experience with container registries, image optimization, and multi-stage builds for ML workloads REQUIRED EXPERIENCE - Demonstrated experience managing large-scale Kubernetes deployments in production environments - Proven track record with Slurm cluster administration and HPC workload management - Previous roles in SRE, DevOps, or Platform Engineering with focus on ML infrastructure - Experience supporting both long-running training jobs and high-availability inference services - Ideally, 3-5 years of relevant experience in ML systems deployment with specific focus on cluster orchestration and resource management

Keep exploring

More Engineering roles

Search all jobs