Sr. Software Engineer, Inference at Anthropic

London, UK

on-site · full-time · Visa sponsorship

Apply for this role at Anthropic

Responsibilities:
- Develop and operate inference infrastructure to serve models reliably and efficiently across multiple accelerator types.
- Implement and tune request routing, batching, caching and autoscaling to meet latency and throughput targets.
- Support deployment pipelines, multi-region/fleet orchestration and integration of new hardware platforms.
- Use observability data to identify bottlenecks and drive performance and reliability improvements.
- Work closely with researchers and product teams to enable production features and research experiments.

Requirements:
- Strong software engineering experience with distributed systems and production services.
- Experience with load balancing, request routing and performance-oriented batching/caching strategies for LLMs.
- Familiarity with Kubernetes and cloud infrastructure (AWS/GCP); experience deploying ML systems at scale is a plus.
- Proficiency in Python or Rust and demonstrable performance tuning experience.
- Results-driven, adaptable, and interested in machine learning systems and their societal impact.