Data Engineering Jobs

• London
• Hybrid
• US$10,500 pm
• Entry level
• Internship

Works deep inside the data processing infrastructure behind Foundry, covering query plan optimisation, execution operators, and efficient data reads and writes. Extends open-source engines and table formats such as Apache Spark, DataFusion, and Iceberg, with work spanning design, code review, testing, and release. An internship aimed at students graduating in 2028.

Apache Spark · Apache DataFusion · Apache Iceberg +7

• New York, United States
• Hybrid
• US$135,000 – US$200,000 pa
• Mid-level

Works deep inside the data processing infrastructure behind Palantir Foundry, designing query planning and optimization, extending execution engines, and improving operators, parallelism and memory management. Builds on and extends Apache Spark, DataFusion, Comet, Velox and Iceberg, owning changes from design through production with a high bar for correctness and performance.

Apache Spark · Apache DataFusion · Apache Iceberg +7

• London
• Lead · manages people

Hands-on technical leader who guides a small team of around five engineers while staying deep in the engineering work, driving architecture, design and code on the hardest problems. The team builds Lakebase, a managed Postgres transactional layer, across areas such as platform reliability, the network proxy data path, or distributed storage in Rust.

Databricks · Lakebase · Postgres +6

• London
• Hybrid
• Senior

Customer-facing, hands-on role building and productionizing data and AI solutions on the Databricks platform. You own architecture and design decisions, implement end-to-end systems spanning data engineering, ML/AI and application development, and embed with client teams from technical ICs to executives. Billable delivery work with around 20% travel to customer sites.

Databricks · AWS · Azure +4

• Edinburgh
• £400 – £450 pd
• Senior

Designs end-to-end AI, ML and data architectures on Google Cloud Platform, covering Generative AI, RAG, agentic AI and LLMOps. Works with engineering, data science and security teams to translate business requirements into secure, compliant cloud-native solutions, and leads architecture governance through review committees.

GCP · BigQuery · Cloud Run +7

• Birmingham
• Hybrid
• Lead
• Permanent · Flexible

Lead BI Developer role focused on designing and building data platforms during a major business and IT transformation, with data enhancement as a central focus. The business is transitioning to Microsoft Fabric, working with SQL, Python/PySpark, Data Factory and AI tools such as Microsoft Copilot. The role is expected to grow, requiring managerial and leadership experience to build out additional team members.

Microsoft Fabric · Data Factory · Microsoft Copilot

• London
• Remote
• £75,000 – £105,000 pa
• Senior

Design, build and deploy production-grade LLM and Generative AI applications, including RAG solutions, scalable data pipelines and AI-powered services for enterprise clients. The role blends software and data engineering with modern AI platforms, plus deployment, monitoring and optimisation of live systems, in a fully remote team of engineers, architects and AI specialists.

Databricks · MLflow · LangChain +5

• Birmingham
• Remote
• Lead · manages people

Lead BI Developer role focused on designing and building data platforms as the business transitions to Microsoft Fabric. Core technologies include SQL, Microsoft Fabric, Python/PySpark and Data Factory, with exposure to AI tools such as Microsoft Copilot. Managerial and leadership experience is beneficial as the project and team are expected to grow.

Microsoft Fabric · Python · PySpark +2

• United Kingdom
• Senior

Owns the end-to-end security telemetry pipeline, ingesting cloud, identity, endpoint and asset data into Elasticsearch and keeping the underlying Elastic Cloud on Kubernetes clusters healthy. Builds custom and Elastic integrations, manages ILM, CCS, capacity and shards, and uses Terraform, Helm and AI-assisted automation to reduce operational toil.

Elasticsearch · Elastic Cloud on Kubernetes · Kubernetes +7

• Welwyn Garden City
• Hybrid
• £500 – £600 pd
• Mid-level
• Contract

Hands-on engineering role building AI-powered applications, automation workflows and internal business tools. You will develop and deploy R Shiny apps, build data pipelines, integrate systems via REST APIs and work with LLMs and Generative AI to create AI assistants and decision-support tools, collaborating directly with business stakeholders from concept to production.

Shiny · Snowflake · LangChain +3

• Bromley
• Hybrid · 3 days in office
• £700 – £750 pd
• Mid-level
• Contract

Builds and supports enterprise data pipelines and OpenShift-based solutions for an investment banking client, using Scala, Apache Spark and containerisation. Works on batch and real-time processing, CI/CD and cloud-native deployment within a cross-functional Agile team on a 12-month contract.

Scala · Apache Spark · OpenShift +1

• Birmingham
• £45,000 – £55,000 pa
• Senior
• Permanent

Hands-on role building and maintaining Python applications, APIs and integrations with FastAPI, plus data pipelines, SQL models and ERP connections. You will also contribute to an internal React and TypeScript web platform and deliver automation and AI solutions, working across cloud environments such as Google Cloud Platform.

FastAPI · React · TypeScript +5

• London
• Hybrid · 2–3 days in office
• £500 – £650 pd
• Mid-level
• Contract

Hands-on contract role building, automating and securing cloud and data infrastructure in live production environments. You'll write Terraform IaC, maintain GitHub CI/CD pipelines, and support Azure modernisation, Snowflake operations, Zero Trust access and secure networking. Candidates may come from an Azure/Kubernetes or Snowflake DataOps background, provided IaC and automation experience is strong.

Azure · Terraform · Kubernetes +2

• Newcastle upon Tyne
• Hybrid
• £60,000 – £85,000 pa
• Mid-level

Design, build and support cloud-native integration solutions on Microsoft Azure, connecting critical business systems, data platforms and applications. Day to day involves Azure Logic Apps Standard, Azure Functions, Service Bus, Terraform IaC and API development, plus 2nd/3rd line support, documentation and stakeholder-facing requirements work in an international organisation.

Microsoft Azure · Azure Logic Apps · Azure Functions +2

• London
• Hybrid · 2–3 days in office
• £650 – £750 pd
• Senior
• Contract

Designs and delivers secure, scalable cloud and data platform architectures in an Azure environment, working closely with security and enterprise architecture teams. Translates business requirements into technical designs while staying hands-on through implementation and handover, applying Terraform, GitHub and Zero Trust principles. Snowflake and DataOps experience is desirable.

Azure · Snowflake · Terraform +1