Data Engineering Jobs in London

• London
• Senior · manages people

Owns the core write-path operations for Delta Lake, the open-source storage layer behind the Databricks Lakehouse, driving performance innovations like Low Shuffle Merge and Deletion Vectors. Designs and maintains high-QPS, low-latency backend systems at massive scale while providing technical leadership, mentoring engineers and leading complex cross-team projects.

Databricks · Delta Lake · Apache Iceberg +6

• London
• Senior

Founding backend engineer role building the Lakebase managed Postgres transactional layer, spanning control plane, proxy data path, and platform reliability. Work covers large-scale distributed systems, high-throughput connection handling, and AI-powered automation to meet SLIs and SLOs across AWS, GCS, and Azure storage.

AWS S3 · GCS · Azure Blob Storage +6

• London
• On-site
• Mid-level
• Permanent

Embedded within a hedge fund's investment team, building Python research platforms, simulation and backtesting frameworks, data pipelines and tooling for systematic trading strategies. Works daily with quantitative researchers and portfolio managers to productionise signals and models, with a focus on testing, CI/CD and scalable system design.

• London

Build a greenfield research and trading platform from scratch within a pod-like team at a prop trading firm, working on GPU-heavy distributed training and MapReduce-style systems. The role centres on Ray, Spark, Dask, Databricks/Anyscale and Kubernetes for ML clusters, with strong autonomy to shape direction early in a deep learning-first environment.

Ray · Spark · Dask +6

• London
• Hybrid
• £150,000 – £165,000 pa
• Director · manages people
• Permanent

Owns the enterprise data, analytics and AI strategy in a complex global environment, leading a multi-disciplinary function spanning analytics, data science and data engineering. Sets up governance frameworks, drives data-led decision making among senior stakeholders and delivers AI/ML use cases on scalable modern platforms.

• London
• Hybrid · 3 days in office
• £50,000 – £60,000 pa
• Mid-level
• Contract

Designs, develops and maintains BI solutions for a data consultancy's client projects, building ETL pipelines, optimising T-SQL queries and stored procedures, and creating semantic and tabular models with SSAS and DAX. Works with dimensional modelling and Kimball data warehousing practices, collaborating with analysts in a fast-paced client-facing environment where priorities shift.

SQL Server · SSAS · Azure

• London
• Remote
• £75,000 – £105,000 pa
• Senior

Design, build and deploy production-grade LLM and Generative AI applications, including RAG solutions, scalable data pipelines and AI-powered services for enterprise clients. The role blends software and data engineering with modern AI platforms, plus deployment, monitoring and optimisation of live systems, in a fully remote team of engineers, architects and AI specialists.

Databricks · MLflow · LangChain +5

• London
• Hybrid
• £50,000 – £60,000 pa

Builds and evolves a modern cloud-based data platform, designing scalable Snowflake pipelines and ETL/ELT processes in SQL and Python. Creates data models supporting reporting, analytics and emerging AI use cases, while optimising workloads for performance and cost. Works in a collaborative engineering team with AWS, Azure or GCP, integrating data from multiple source systems.

Snowflake · AWS · Azure +1

• London
• Hybrid · 2 days in office
• £35,000 – £50,000 pa
• Entry level
• Permanent

Hands-on role developing and maintaining software with Python, Go and TypeScript, processing and validating live and historical sports datasets, and supporting AWS cloud infrastructure and production systems. Works closely with experienced engineers on SQL databases, troubleshooting and technical projects within the sports and betting sector.

Python · Go · TypeScript +3

• London
• Hybrid
• Entry level

Works deep inside the data processing infrastructure behind Palantir Foundry, on how computations are represented, query plans optimised, operators executed, and data read and written efficiently. Builds on and extends Apache Spark, DataFusion and Iceberg, taking on production problems such as writing planning rules or speeding up large table scans through design, code review, testing and release.

Apache Spark · Apache DataFusion · Apache Iceberg +7

• London
• Hybrid · 4 days in office
• £60,000 pa
• Mid-level

Designs and maintains cloud-based data solutions on Google Cloud Platform, building ELT pipelines and BigQuery data models that integrate advertising, analytics and Salesforce data. Works within a specialist Data & Tech Solutions team at a media agency, supporting reporting, campaign activation and AI-driven initiatives with Looker or similar BI tools.

Google Cloud Platform · Google BigQuery · AWS +6

• London
• Hybrid · 2 days in office
• £80,000 – £100,000 pa
• Senior

Manage and secure AWS cloud infrastructure, build infrastructure as code and CI/CD pipelines, and maintain cloud-based data pipelines. Work closely with the CTO and software team on developer access, permissions, security monitoring and incident response, using AWS CDK, GitHub Actions, Lambda, ECS, Fargate and TypeScript.

AWS · AWS CDK · GitHub Actions +7

• London
• £500 – £600 pd
• Contract

Hands-on contract role building and improving data pipelines on Google Cloud Platform using Python, SQL and BigQuery. The work focuses on advertising measurement data, taking ownership of partially built pipelines and validating whether campaign activity drives measurable outcomes. AI coding agents such as Claude Code are a core part of the daily engineering workflow.

GCP · BigQuery · Python +3

• London
• Hybrid · 2–3 days in office
• £98,500 pa
• Senior
• Contract

Technical architect role within a dedicated design team, building automated and agentic production pipelines that scale paid media creative across EMEA. Involves designing agentic workflows with Anthropic Claude and OpenAI APIs, architecting MCP servers, and building agents that read design systems and output production-ready static and video assets.

Anthropic Claude · OpenAI · Model Context Protocol +5

• London
• Remote
• £50,000 – £70,000 pa
• Senior

Design, build and operate resilient AWS cloud infrastructure for public-sector workflow and AI orchestration platforms, working across platform and delivery teams. The role spans Terraform and Kubernetes, CI/CD pipelines, Kafka event-driven systems, databases, Keycloak IdAM and observability stacks, with a strong DevSecOps and secure-by-design focus. Fully remote with paid travel to client sites, and SC/DV clearance eligibility required.

AWS · Azure · GCP +7