Senior Data Engineer — Lab Data Pipelines & Data Mesh (12 month FTC) at AstraZeneca

on-site · full time · Visa sponsorship

Responsibilities:
- Build resilient lab-to-cloud pipelines: ingest data from instruments, robotics and ELNs into Databricks/AWS with automated validation and provenance.
- Design and publish domain data products into a lakehouse-based data mesh (Delta Lake/Databricks) for self-service analytics and ML.
- Implement ETL/ELT orchestration, retries, lineage, monitoring and runbooks to improve reliability and turnaround times.
- Define standardized schemas and shaping for lab data types (chromatography, spectroscopy, assays, automation outputs).
- Collaborate with scientists, automation and informatics teams to enable LLM/agent-assisted extraction from ELNs and downstream data products.
- Embed security, auditability and role-based access controls to meet regulatory and compliance requirements.
- Provide APIs, curated datasets and documentation so scientists can discover and reuse trusted data without manual copying.

Requirements:
- BS/MS in Computer Science, Data Engineering, Bioinformatics or related discipline.
- 5+ years delivering production-grade data pipelines; life sciences/lab experience strongly preferred.
- Strong Python and SQL skills; experience with Databricks, Delta Lake and pipeline orchestration frameworks.
- Practical knowledge of cloud platforms (AWS services: S3, EC2, EKS, Glue or equivalent) and data mesh concepts.
- Experience with data quality, lineage, monitoring tools and implementing access controls for regulated data.
- Strong collaboration skills: ability to partner with scientists, robotics and IT to translate needs into production solutions.