ML Data Engineer at Recraft

London, United Kingdom

on-site · full_time · Visa sponsorship

Apply for this role at Recraft

Responsibilities:
- Design, build and maintain high-throughput data ingestion pipelines for large-scale image (and occasional text/HTML) datasets.
- Own end-to-end pipeline stages: raw collection → quality/beauty/relevance filtering → deduplication/validation → train-ready artifacts.
- Operate and improve Kubernetes-based distributed data jobs (retries, monitoring, scaling, automation).
- Manage S3-style object storage layout, lifecycle, throughput and cost trade-offs.
- Implement observability around pipelines (progress, health metrics, alerts) to speed iteration and debugging.
- Partner closely with ML engineers to align datasets with training needs and experimentation cadence.

Requirements:
- Strong Python skills with clean, production-ready code practices.
- Hands-on Kubernetes experience for batch/distributed processing and containerized workloads.
- Proven experience handling unstructured image data at scale: loading, filtering, transforming.
- Experience building reliable data ingestion/parsing tools and handling real-world failure cases.
- Comfortable working with object storage (S3) and moving large volumes of data efficiently.
- Detail-oriented ownership mindset and pragmatic approach to reliability and performance.
- English proficiency (B2+).
- Nice-to-have: familiarity with ML training workflows (PyTorch) and dataset considerations for model quality.