Latest Data Pipeline Engineer Jobs

Lead Python Developer

Our client, an innovative and rapidly growing business within the Exchange Traded Funds (ETF) sector, is seeking an experienced Lead Python Developer to play a key role in shaping the future of their technology and data platforms.This is a rare...

Infosec London, City And County Of the City Of London, United Kingdom £100,000 – £120,000 pa

Full Stack Developer

This role involves integrating hardware and cloud technology to deliver innovative services, including building data pipelines, managing cloud infrastructure, and creating digital applications. You will work on cutting-edge IoT and data-driven projects with a focus on cost-saving insights and predictive maintenance.

CMD Recruitment Melksham, SN12 6JL, United Kingdom £55,000 – £60,000 pa

On-site Permanent

DWIT Technical Architect

This role involves detailed analysis and documentation of legacy data feeds to support their migration to AWS and the decommissioning of outdated systems. The Technical Architect will work hands-on to discover dependencies, produce architecture artefacts, and support transformation planning using AI-enabled tools, while collaborating with engineers, architects, and stakeholders across a complex environment.

Hays Technology Telford, Shropshire, SY2 5TN, United Kingdom £700 pd

Hybrid Contract Clearance Required

Databricks Data Engineer

Databricks Data Engineer - Data Ingestion6 months - £550 a day (outside IR35)Location: UK - Hybrid/RemoteWe are looking for an experienced Databricks Data Engineer / Data Ingestion Engineer to join a large, global organisation.The role is focused on data ingestion...

TRIA London, United Kingdom £525 – £575 pd

Principal Data Engineer

Design and build a foundational data platform from the ground up for a large private real estate group, integrating siloed systems into a unified, trusted data source. Develop robust ETL/ELT pipelines, REST APIs, and data models while implementing production-grade LLM workflows with auditability and human-in-the-loop oversight. Deliver dashboards, automation, and internal tools that are actively used by principals and operating teams, with full autonomy over technology choices and architectural direction.

VIQU IT London, United Kingdom £1,000 pd

Azure Data Platform Engineer

Design and operate a secure, scalable Azure data platform using Databricks, Data Lake, and Data Factory. Build data ingestion and transformation pipelines while implementing governance, security, and automation best practices. Collaborate with architects to deliver data products and support the platform roadmap.

BrightBox Group Mansfield, Nottinghamshire, United Kingdom £400 – £450 pd

Hybrid Contract

Principal AI Data Engineer

This role involves developing and deploying AI, GenAI, and agentic AI systems using Azure AI Foundry, Databricks Mosaic AI, and LangChain-based frameworks. The engineer will build production-ready generative AI applications, focusing on RAG pipelines, agent design, and scalable workflows on Databricks. Responsibilities include prompt engineering, evaluation framework development, and applying DevOps practices to ensure robust, high-performing AI solutions for commercial trading teams.

Tatton Recruitment South Bank, London, SE1 9PZ, United Kingdom £750 – £800 pd

Senior Software Engineer

Designs and maintains high-performance, production-grade Python systems focused on scalable data processing and concurrency. Builds multithreaded and multiprocessing solutions for CPU, memory, and I/O-intensive workloads on local server infrastructure. Emphasises clean code, test-driven development, and performance optimisation in a non-modelling software engineering role.

Pure Resourcing Solutions Cambridge, United Kingdom £65,000 – £70,000 pa

Forward Deployed AI Engineer

As a Forward Deployed AI Engineer, you will work directly with clients to own and implement Gen AI strategies, building end-to-end workflows and taking them to production. You will collaborate in small teams to deliver high-stakes projects, solve real-world problems at scale, and contribute learnings back to Palantir's AIP product suite.

Palantir Technologies United States

Hybrid Permanent

Data Engineer

This role involves designing, developing, and maintaining ETL pipelines using Talend, building Oracle SQL and DDL artefacts, and contributing to CI/CD pipelines. You will work in a secure environment, supporting mission-critical data services and mentoring junior team members.

TXP Telford, Shropshire, SY2 5TN, United Kingdom £540 – £541 pd

Hybrid Contract Flexible Clearance Required

Senior Data Engineer

This role involves designing and building scalable data pipelines on Azure and Databricks, using Delta Lake and Medallion architecture. You'll own data models across multiple layers, implement CI/CD via Azure DevOps, and support integration with Microsoft Fabric and OneLake. The position emphasizes technical leadership, engineering best practices, and end-to-end ownership of a modern data platform.

Head Resourcing Glasgow, City Of Glasgow, G2 1AL, United Kingdom £65,000 – £75,000 pa

Data Engineer

This role involves designing and maintaining data pipelines in a high-pressure environment, with a focus on Python, SQL, and cloud-based data technologies. The engineer will work on Spark and Delta Lake implementations, support incident resolution within SLAs, and contribute to data architecture using Medallion principles. The position requires strong problem-solving skills and the ability to operate effectively across both legacy and modern data systems.

Hays Technology Milton Keynes, Buckinghamshire, United Kingdom £400 pd

Hybrid Contract

Data Engineer - PING Engineer

This role involves transforming business questions into trusted data insights by building and maintaining dashboards, reports, and data pipelines within a newly cloud-based environment. The individual will contribute to shaping an Azure Lakehouse rebuild from the ground up, working closely with modern cloud systems including Azure and cloud ERP/CRM platforms. The position bridges data analysis and engineering, focusing on robust data solutions that support business operations.

Interface Recruitment Gainsborough, Lincolnshire, United Kingdom

Senior Data Engineer

Job Title: Senior Data EngineerLocation: London, UKContract: Initial 6 months, with a strong likelihood of extensionOn-site Requirement: 3–5 days per week at the client's London officeOutside IR35Job Description:We are seeking a seasoned Senior Data Engineer with a strong background in...

BrightBox Group London, United Kingdom

Data Engineer

This role involves building and maintaining ETL/ELT pipelines, developing and optimising SQL databases, and integrating data from ERP, CRM, and telematics systems. The engineer will support Power BI reporting and ensure data quality, security, and performance within a modern cloud data environment.

Tenth Revolution Group Kent, United Kingdom £50,000 – £55,000 pa

Hybrid Permanent