Data Scientist - AI Evaluation & Benchmarking Manager at PwC
United Kingdom
Posted
on-site · full time · Visa sponsorship
Responsibilities:
- Design and run end-to-end AI benchmarking and experimentation workflows aligned to client use cases.
- Build scalable evaluation frameworks, metrics, and automated pipelines for repeatable model assessment.
- Develop, maintain, and harden experimentation infrastructure with production-grade engineering (CI/CD, containerisation).
- Translate experimental results into clear, client-ready insights and support technical demos and deep dives.
- Monitor literature and best practices to continuously improve evaluation methodologies and tooling.
Requirements:
- Strong hands-on data science / ML background with practical experience in LLM experimentation and structured evaluation frameworks.
- Proficient in Python, including asynchronous programming and multithreading; ability to write maintainable, testable code.
- Experience deploying ML workloads on cloud platforms (Azure/AWS/GCP) and familiarity with CI/CD and Docker/Podman workflows.
- Solid understanding of statistics, experimental design, and benchmarking methodologies; able to turn findings into actionable recommendations.
- Able to manage fast-moving workstreams independently, communicate clearly to technical and non-technical stakeholders, and demonstrate emerging leadership.