AI Platform Engineer (m/f/x) at Scalable GmbH
Germany
Posted
on-site · full-time · Visa sponsorship
Apply for this role at Scalable GmbH
Responsibilities:
- Design, build and operate an internal AI provisioning platform to provide unified access to LLMs and MCP servers (e.g., LiteLLM).
- Implement observability: logging, tracing, metrics, and alerting for LLMs using Datadog, Langfuse or similar tools.
- Enforce model governance: security, data privacy, rate limiting, and API cost controls.
- Collaborate with AI Software Engineers to integrate agentic workflows and cloud-native architectures.
- Build CI/CD pipelines, containerized deployments, and infrastructure as code for scalable model hosting.
- Monitor production reliability and performance; diagnose incidents and optimize cost/performance trade-offs.
Requirements:
- Degree in Computer Science or related field and 2–5 years in Platform Engineering, DevOps, or SRE.
- Hands-on experience with AWS and Infrastructure as Code (Terraform).
- Proficiency in at least one language (Python or Go), Docker, and CI/CD tooling.
- Experience with observability/telemetry stacks and LLM-specific tooling (Langfuse, model logs, cost metrics).
- Strong understanding of security, data privacy, API rate limiting, and model governance practices.
- Good collaboration skills for working with data science and ML teams.