Data Engineering Jobs in London

• London
• Hybrid · 3 days in office
• £500 – £600 pd
• Mid-level
• Contract

Designs, builds and maintains scalable data pipelines and ETL processes, working hands-on with Scala, Java or Python alongside Spark, Elastic, REST APIs and Tomcat. Builds reusable shared libraries and platform capabilities, with Azure Data Factory, Databricks and Elasticsearch exposure, collaborating with Technology, Product and Business stakeholders on data quality and reliability.

Spark · Apache Airflow · Azure +7

• London
• Hybrid
• Senior

Customer-facing engineering role building and productionising data and AI solutions on the Databricks platform, owning architecture and design decisions for end-to-end systems spanning data engineering, ML/AI and application development. Engineers embed with client teams, from technical ICs to executives, and contribute reusable frameworks back to the product. Travel to customer sites around 20% of the time.

Databricks · AWS · Azure +7

• London
• Hybrid · 2–3 days in office
• £745 pd
• Lead · manages people
• Contract

Lead data delivery squads supporting a Corporate & Investment Banking data platform, overseeing how banking data is integrated, processed, mapped and consumed across enterprise platforms. The role is delivery-focused rather than hands-on engineering, running Scrum ceremonies, managing stakeholders and challenging technical teams on data logic and modelling.

Jira · Databricks · Data Lakes +1

• London
• Hybrid
• Senior

Customer-facing, hands-on role building and productionizing data and AI solutions on the Databricks platform. You own architecture and design decisions, implement end-to-end systems spanning data engineering, ML/AI and application development, and embed with client teams from technical ICs to executives. Billable delivery work with around 20% travel to customer sites.

Databricks · AWS · Azure +4

• London
• On-site
• Senior
• Permanent

Build an internal data platform from the ground up, defining architecture for petabyte-scale robot telemetry pipelines, compute infrastructure and orchestration. Work spans backend services, data warehousing, BI dashboards and MLOps evaluation workflows, collaborating closely with machine learning teams on humanoid robot training data.

Kubernetes

• London
• Hybrid
• £75,500 pa
• Senior

Designs, builds and optimises scalable data pipelines and processing systems for large volumes of clinical and genomic data, covering acquisition, transformation, lineage, metadata and de-identification. Works cloud-first in AWS using Python, Prefect, Tableau, Semarchy, Aspera and Globus, within the Generate Research Data service supporting the National Genomic Research Library.

AWS · Prefect · Tableau +7

• London
• £80,000 – £115,000 pa
• Senior
• Permanent

Hands-on engineering role migrating a Futures algorithmic trading platform's market data integration from the legacy LSEG/Refinitiv RFA to the Real-Time SDK (RTSDK). Involves analysing existing RFA integrations, developing Java applications and shared frameworks, and working with Futures and Options market data across exchanges while maintaining backward compatibility and trading workflows.

LSEG · Refinitiv · RFA +2

• London
• Hybrid · 1 day in office
• £500 pd
• Senior
• Contract

Hands-on senior engineer maintaining and developing data pipelines within a Measurement Innovation Team, focused on automating campaign measurement. Work includes GCP and BigQuery pipelines, privacy-safe joins via BigQuery and Snowflake clean rooms, cloud migration and security, plus some AWS. Day-to-day use of AI coding agents such as Claude Code is expected, alongside direct collaboration with external partners' data teams and internal stakeholders.

GCP · BigQuery · Snowflake +2

• London
• Permanent

Builds and maintains scalable ETL/ELT pipelines and logical/physical data models for supplier mapping, performance and risk analysis, and a trusted supplier data repository. Partners with engineers, scientists and business teams to deliver self-service data products, with cross-timezone collaboration and evaluation of generative AI tooling for data workflows.

• London
• Hybrid
• Entry level
• Permanent

Design, build and automate distributed data pipelines, schemas, warehouses and SQL/NoSQL systems, and own metrics, reports and dashboards that drive business decisions. Interns work with a manager and mentor alongside engineers, analysts and data scientists, troubleshooting data issues and contributing to mission-critical projects.

SQL · NoSQL · MapReduce

• London
• Hybrid
• £80,000 – £115,000 pa
• Lead
• Permanent

As one of the first senior data hires, you'll own the company's data architecture, building scalable batch and real-time pipelines, ingestion and storage solutions, and vector search and ML data pipelines. You'll work with AI, backend and product teams on model training and inference, ensure reliability and governance, and mentor future data engineers.

Qdrant · Milvus · pgvector +7

• London
• £500 – £600 pd
• Mid-level

Hands-on builder role reporting to the Digital, Data & AI Infrastructure Lead, turning operational business challenges into rapid prototypes, automated data flows and AI-enabled dashboards. Day to day involves building AI assistants, R Shiny apps with natural-language co-pilots, ETL jobs via REST APIs and SQL, and supervised human-in-the-loop automation, iterating directly with non-technical users.

R Shiny · Shiny for Python · pandas +7

• London
• Hybrid
• £65,000 – £75,000 pa
• Senior
• Permanent

Designs, architects and scales enterprise-grade AI platforms, building data pipelines, cloud-native services and RESTful APIs that productionise ML, NLP and LLM models. Works alongside Data Scientists, Software Engineers and Product teams, mentoring engineers and shaping engineering standards. Remote-first with optional hybrid London working.

AWS · Azure · Docker +7

• London
• On-site
• £500 – £650 pd
• Mid-level
• Contract

Build and maintain document ingestion and RAG retrieval pipelines for AI-enabled defence planning tools, extracting clean text from PDF, DOCX and HTML and designing chunking, embedding and vector search strategies. Own the Postgres layer, build labelled ground-truth datasets to measure retrieval quality, and harden prototypes into production services with Docker and CI/CD.

FastAPI · PyTest · pgvector +7