Data Engineer, Data Quality & Provenance at Wayve
Germany
Posted
on-site · full_time · Visa sponsorship
Responsibilities:
- Design and operate scalable batch and streaming pipelines for multimodal fleet and simulation data.
- Build data models, catalogues, indexes and query capabilities to make sensor, vehicle-state, map and event data discoverable and usable.
- Produce versioned, reproducible datasets for training, evaluation, replay, scenario mining and safety analysis.
- Implement ingestion, synchronization, transformation, curation, labelling and quality-validation workflows.
- Define schemas, APIs and data contracts in partnership with autonomy, ML, simulation and safety teams.
- Improve reliability, performance and cost-efficiency of large-scale storage and compute workloads; add observability and incident response processes.
Requirements:
- Strong hands-on Python and SQL skills and solid production software-engineering fundamentals.
- Experience designing and operating large-scale distributed data systems (beyond small analytics pipelines).
- Production experience with distributed processing and workflow orchestration tools such as Spark, Flyte or Airflow.
- Experience building cloud-based data platforms using object storage; knowledge of data organization, versioning, querying, governance and cost management.
- Ownership of data quality, lineage, observability, reproducibility and incident response in production workflows.
- Ability to translate ambiguous requirements from ML, robotics or simulation teams into durable implementations.
- Familiarity with geospatial and multimodal datasets is a strong plus.