Senior Data Engineer at Refractal
United Kingdom
Posted
on-site · full_time · Visa sponsorship
Apply for this role at Refractal
Responsibilities:
- Design and deliver an append-only, cryptographically signed evidence store with clear provenance and auditability.
- Build and operate streaming and batch data pipelines to turn ingested activity and detections into queryable datasets.
- Define schemas and work with integrations engineers to ensure consistent ingestion and transformation.
- Own data quality, retention, residency, deletion controls, and policy-driven enforcement for regulated customers.
- Produce training and evaluation datasets with data scientists and security engineers, ensuring separation and lineage.
- Implement reporting capabilities to quickly generate compliance artefacts for EU AI Act, NIST AI RMF, and ISO/IEC 42001.
Requirements:
- 6+ years owning production data engineering pipelines.
- Strong Python and SQL skills; experience with Postgres and a warehouse/analytics store (BigQuery, ClickHouse, or similar).
- Practical experience with streaming systems (Kafka, Pub/Sub) and orchestration tools (Airflow, Dagster).
- Data modelling expertise for audit trails, lineage, and multi-tenant isolation.
- Security-minded approach to handling sensitive data and compliance requirements.
- Nice to have: domain experience in regulated industries (banking, insurance), familiarity with UK GDPR and data residency controls.