Data Engineering Jobs

• London
• Mid-level
• Contract

Lead the shift from manual to automated processing across a Lloyd's insurer's Consortium and Facility business, designing scalable data pipelines that deliver bordereaux and feeds to follow markets. Hands-on role automating fee and profit commission calculations, including Consortia fees and Coverholder commissions, while maintaining Power BI reporting and working with underwriting, finance and operations teams.

Microsoft SQL Server · Power BI · Azure +2

• Belfast
• Hybrid · 3 days in office
• Lead

Hands-on lead role building and optimising large-scale Scala Spark reconciliation pipelines on Databricks, migrating legacy Cloudera Hadoop workloads to cloud-native AWS architectures using Delta Lake and Databricks Workflows. Involves refactoring over-engineered Spark jobs, translating architectural designs into reusable components, and collaborating daily with architects, platform and DevOps teams.

Databricks · AWS · Delta Lake +7

• Newcastle upon Tyne
• Hybrid · 2 days in office
• £84,000 pa
• Senior

Own and oversee a business-critical enterprise data platform built on Microsoft Azure, acting as technical lead for architecture, scalability, governance and operational excellence. Work with Infrastructure, Security and Architecture teams on CI/CD, automation, monitoring and Infrastructure-as-Code, supporting production and incident management while enabling Data Engineers and shaping platform strategy across Data and AI initiatives.

Microsoft Azure · Azure Data Factory · Microsoft Fabric +5

• London
• Hybrid · 1 day in office
• £150,000 – £180,000 pa
• Director · manages people

Executive role owning the data vision for a private equity-backed financial services firm, leading a large-scale transformation of its data and analytics function. Responsibilities span reshaping the operating model, building a modern cloud data platform with lake and warehousing capabilities, and driving data engineering, governance and self-service analytics. Also involves applying AI to engineering productivity and improving customer onboarding and personalisation.

Azure · Microsoft Data Ecosystem

• London
• Hybrid
• £55,000 – £65,000 pa
• Mid-level
• Permanent

Design, build and maintain ETL/data pipelines and secure cloud-based data platforms using Python and SQL, applying DataOps and DevOps practices. Work with architects, analysts and software engineers on high-impact projects supporting UK national security, covering data modelling, orchestration and governance.

AWS

• London
• On-site
• Mid-level
• Permanent

Embedded within a hedge fund's investment team, building Python research platforms, simulation and backtesting frameworks, data pipelines and tooling for systematic trading strategies. Works daily with quantitative researchers and portfolio managers to productionise signals and models, with a focus on testing, CI/CD and scalable system design.

• London
• Hybrid
• £80,000 – £90,000 pa
• Lead
• Permanent

Hands-on data engineering lead role at a private equity firm, designing and scaling modern data platforms across a portfolio of companies. Combines technical delivery, data architecture and senior stakeholder advisory, with direct C-suite exposure and a focus on making infrastructure AI-ready for advanced analytics and agentic AI use cases.