Data Engineering Jobs in London

• London
• Hybrid · 2–3 days in office
• Lead · manages people
• Contract

Hands-on technical leadership role driving a newly established Quantexa capability for AML and financial crime initiatives across EMEA. Leads and mentors a small team of data engineers while designing and building scalable pipelines and ETL processes using Scala, Spark, Python and SQL, with cloud architecture on Azure and Databricks.

Quantexa · Scala · Spark +8

• London
• Hybrid · 2–3 days in office
• Contract

Hands-on data engineering role building scalable pipelines and entity resolution for a Quantexa-led AML and financial crime programme in a global bank. Day to day involves Scala, Spark, Python and SQL work across Azure, Databricks and Elasticsearch, plus reusable frameworks and automation, collaborating with technology, product and business stakeholders across EMEA.

Quantexa · Azure · Databricks +4

• London
• Senior

Serve as a trusted technical advisor guiding enterprise customers through the architecture, administration and security of their Databricks deployments on AWS, Azure or GCP. The role spans cloud networking, identity, IaC automation and platform administration, with live coding in Python and PySpark, and requires travel up to 30%.

Databricks · AWS · GCP +7

• London
• Senior

Drive large-scale enterprise data warehouse migrations to the Databricks Intelligence Platform across EMEA, acting as the tier-3 technical escalation point for complex, strategic accounts. Build assessment tooling, programmatic migration assets and guardrails, while mentoring the regional Solutions Architect community and feeding field insights back to Product and Engineering.

Databricks · Apache Spark · Teradata +7

• London
• Hybrid · 4 days in office
• £500 – £550 pd
• Mid-level
• Contract

Design, build and support scalable data solutions on Azure Databricks, developing SQL, PySpark and Python pipelines and maintaining ETL/ELT flows across Azure, SQL Server and Delta Lake. The role also covers Lakeflow pipelines, Unity Catalog, CI/CD automation and production troubleshooting within a growing data team.

Azure · Databricks · SQL Server +7

• London
• Remote
• £432 – £484 pd
• Senior
• Contract

Design and deploy production-grade Generative AI workflows and backend security guardrails for a global financial institution's AI Centre of Excellence. The role blends Python/Java engineering, AWS/Azure cloud pipelines with Airflow, and roughly 20-25% frontend/UI work for guardrail tooling, plus production support and ML evaluation metrics.

AWS · Azure · Apache Airflow

• London
• Hybrid · 2 days in office
• £90,000 – £95,000 pa
• Lead

Hands-on technical leadership role within a small data engineering squad, setting engineering standards while building scalable pipelines in Python, SQL and Snowflake. Involves data modelling, integrations, performance and reliability work on an Enterprise Data Platform, plus mentoring engineers and shaping architecture with AWS cloud technologies.

Snowflake · AWS · dbt +3

• London
• Hybrid · 3 days in office
• £50,000 – £60,000 pa

Builds and maintains scalable ETL/ELT pipelines and curated, business-ready datasets that power reporting, dashboards and self-service analytics. Works closely with BI Developers and stakeholders to optimise SQL, data models and pipeline performance, with a focus on cloud platforms such as AWS Redshift, Athena, S3 and Glue. Backend-focused rather than dashboard or reporting development.

AWS · Redshift · Athena +6

• London
• Hybrid · 1–2 days in office
• £60,000 – £70,000 pa
• Contract

Hands-on engineering role building a modern Azure data platform, designing and developing scalable automated pipelines with Data Factory, Databricks, Synapse and Logic Apps. Covers the full data lifecycle from ingestion to transformation and delivery, with work on performance, data quality and monitoring alongside data engineers, architects, product owners and BI teams in an agile setting.

Azure Data Factory · Databricks · Azure Synapse +7

• London
• Hybrid
• £60,000 – £80,000 pa
• Senior
• Permanent

Design and implement robust data pipelines and configure Quantexa components while leading the technical development of Quantexa-based solutions for established clients. Work with Spark, Scala and Python Databricks, with exposure to DevOps and CI/CD tooling such as Jenkins, GitHub Actions, Azure DevOps, Docker and Kubernetes. Hybrid role in London supporting client data engineering and integration engagements.

Quantexa · Spark · Scala +7

• London
• £100,000 – £150,000 pa
• Senior
• Permanent

Lead data architecture engagements for insurance clients, shaping strategies and recommendations aligned to business priorities. You'll direct discussions with senior business and technology stakeholders, guide delivery teams, and build client relationships while contributing to proposals and new engagements. Insurance sector knowledge and independent client-facing confidence are central.

• London
• Hybrid · 2 days in office
• £90,000 – £125,000 pa
• Lead · manages people
• Permanent

A principal-level role providing technical and people leadership across a major UK consumer business's data engineering function. You'll shape the architecture of a cloud-first platform built on Snowflake, AWS and dbt, set data modelling and engineering standards, and mentor Lead Data Engineers while owning technical risk and cost optimisation.

Snowflake · AWS · dbt +7

• London
• Hybrid
• £80,000 – £90,000 pa
• Senior

Designs and scales Azure Databricks data platforms, building pipelines with PySpark, Delta Lake and Airbyte while developing LLM and RAG-powered features in customer-facing products. Works closely with Product and Engineering stakeholders, optimising Azure environments for performance and cost, and mentoring colleagues in a growing data function.

Azure · Databricks · PySpark +4

• London
• Hybrid · 2 days in office
• £70,000 – £85,000 pa
• Senior
• Permanent

Designs and maintains data transformations, builds trusted datasets and scalable data products for business intelligence. Works with SQL, cloud platforms like Databricks, Snowflake or BigQuery, dbt and Python, delivering to Power BI while partnering with analysts and stakeholders to define requirements and champion analytics engineering best practices.

Databricks · Snowflake · BigQuery +3

• London
• Hybrid · 2 days in office
• £85,000 – £90,000 pa
• Senior

Builds and scales customer-facing data products on an Azure Databricks lakehouse, turning raw and third-party data into enriched, product-ready datasets delivered via APIs and internal tools. Applies LLMs, RAG, embeddings and semantic search to add classification, tagging and insight features, while owning platform performance, cost and governance. Works closely with product and commercial teams and mentors junior engineers.

Azure Databricks · Azure · Unity Catalog +7