Staff Software Engineer, Inference at Anthropic

London, UK

on-site · full-time · Visa sponsorship

Apply for this role at Anthropic

Responsibilities:
- Architect, build and operate the inference stack that serves Claude across heterogeneous accelerators and clouds.
- Design intelligent request routing, batching, caching and load-balancing to maximise compute efficiency and lower latency.
- Lead fleet-wide orchestration and autoscaling to balance production, research and experimental workloads.
- Integrate new accelerator platforms and maintain compute-agnostic deployments and deployment pipelines.
- Collaborate with researchers to enable high-performance experiments and deliver production-grade features.

Requirements:
- Extensive software engineering experience building large-scale, high-performance distributed systems.
- Deep familiarity with orchestration, request routing, load balancing and autoscaling at fleet scale.
- Experience or strong understanding of LLM inference concerns (batching, caching, multi-accelerator strategies).
- Practical experience with Kubernetes and cloud providers (AWS/GCP) and proficiency in Python or Rust.
- Results-oriented, comfortable with ambiguous problems, and committed to technical excellence and AI safety.