Data Engineering Jobs

• Sydney, Australia
• Hybrid
• Entry level
• Internship

Forward Deployed Software Engineers embed with government customers to understand open-ended problems and build data-driven solutions, spanning architecture discussions, large-scale data wrangling, custom web apps and executive briefings. Interns work in small teams with minimal supervision and own end-to-end delivery of high-stakes projects, using languages such as Python, Java, C++ or TypeScript.

Palantir

• New York, United States
• Hybrid
• US$10,500 pm
• Entry level
• Internship

Works deep inside the data processing infrastructure behind Palantir Foundry, covering query plan optimization, operator execution, and efficient data reads and writes. Extends open-source engines and table formats such as Apache Spark, DataFusion, and Iceberg, shipping changes through design, code review, testing, and release. An internship for candidates graduating in 2028.

Apache Spark · Apache DataFusion · Apache Iceberg +7

• London
• Senior · manages people

Owns the core write-path operations for Delta Lake, the open-source storage layer behind the Databricks Lakehouse, driving performance innovations like Low Shuffle Merge and Deletion Vectors. Designs and maintains high-QPS, low-latency backend systems at massive scale while providing technical leadership, mentoring engineers and leading complex cross-team projects.

Databricks · Delta Lake · Apache Iceberg +6

• London
• Senior

Founding backend engineer role building the Lakebase managed Postgres transactional layer, spanning control plane, proxy data path, and platform reliability. Work covers large-scale distributed systems, high-throughput connection handling, and AI-powered automation to meet SLIs and SLOs across AWS, GCS, and Azure storage.

AWS S3 · GCS · Azure Blob Storage +6

• United Kingdom
• Senior

Join the Elasticsearch core team to build foundational infrastructure and shared functionality, including services, settings, scripting, and serverless orchestration. You'll design horizontally-scalable APIs, tackle concurrency and performance issues, and improve observability while leading cross-team projects in an open-source, distributed environment.

Elasticsearch · Painless · Buildkite +7

• United Kingdom
• Lead

Designs and operates core distributed systems for a serverless Elasticsearch offering, focusing on cluster coordination, shard allocation, and consistent indexing across nodes. Involves debugging concurrency and performance issues at petabyte scale, owning operational investigations end-to-end, and collaborating asynchronously across a distributed team.

Elasticsearch · Lucene · JDK +1

• Welwyn Garden City
• Hybrid · 2–3 days in office
• £500 – £600 pd
• Mid-level

Hands-on builder of AI solutions within a global pharma company's Business Digital, Data & AI Infrastructure team. You turn business needs into working tools: AI skills and agents, agentified R Shiny dashboards, process automations and scheduled data pipelines, working iteratively with non-technical users and producing validation evidence for regulated use cases.

pandas · Shiny · Snowflake +3

• Sheffield
• Remote
• £65,000 – £75,000 pa
• Senior

Owns the curated data layer within a Databricks Lakehouse, designing dimensional and semantic models and building complex SQL and PySpark transformation pipelines. Defines KPI logic, conformed dimensions and data quality standards, leads modelling design reviews and mentors engineers while working with architects, BI developers and business stakeholders. Remote with ad-hoc client site travel.

Databricks · Lakehouse

• London
• Hybrid
• Contract

Builds and maintains dashboards in Amazon Quick and develops robust data pipelines, bridging backend processing and frontend visualization. Handles complex SQL data wrangling and works with AWS services including S3, Athena, Redshift and Glue, plus Python for automation. Partners with business stakeholders to translate requirements into KPIs and technical specifications.

Amazon QuickSight · AWS · Amazon S3 +3

• Birmingham
• Remote
• Lead · manages people

Lead BI Developer role focused on designing and building data platforms as the business transitions to Microsoft Fabric. Day-to-day work spans SQL, Python/PySpark, Data Factory and AI tools like Microsoft Copilot, with growing managerial and leadership responsibilities as the team expands. Primarily home-based with flexible UK office locations.

Microsoft Fabric · Python · PySpark +2

• London
• Hybrid
• £85,000 – £115,000 pa
• Mid-level
• Permanent

Develops and maintains high-performance, data-driven applications using KDB+/q and Python, building and optimising data pipelines and time-series solutions. Works on query performance, data processing and system efficiency, troubleshooting issues and collaborating with technical teams. Financial services or capital markets background is preferred.

KDB+ · Python · SQL +2

• London
• Hybrid · 4 days in office
• £500 – £550 pd
• Mid-level
• Contract

Design, build and support scalable data solutions on Azure Databricks, developing SQL, PySpark and Python pipelines and maintaining ETL/ELT flows across Azure, SQL Server and Delta Lake. The role also covers Lakeflow pipelines, Unity Catalog, CI/CD automation and production troubleshooting within a growing data team.

Azure · Databricks · SQL Server +7

• Barrow-in-Furness
• Hybrid · 3 days in office
• £67 ph
• Mid-level
• Contract

Builds and maintains BI reports, dashboards and ETL pipelines using IBM Cognos Analytics and Tableau, working with stakeholders to gather reporting requirements and turn complex business data into KPI insights. Sits in a growing data and analytics team supporting operational and strategic decisions, with a focus on data quality and consistency across platforms.

IBM Cognos Analytics · Tableau · Power BI

• London
• Hybrid · 4 days in office
• £700 – £771 pd
• Senior
• Contract

Contract role rebuilding legacy SQL Server transformation logic (SSIS, SSRS, SSAS) into Snowflake, dbt and Python pipelines, with Power BI reporting on top, for a credit risk reporting function. The developer designs scalable Snowflake solutions, tunes query performance and cost, and provides technical leadership while engaging senior business stakeholders on regulatory reporting programmes.

Snowflake · dbt · Python +7

• Egham
• Hybrid · 2–3 days in office
• £85,000 – £95,000 pa
• Lead
• Permanent

Own and evolve a global Snowflake platform already live across multiple countries, combining hands-on engineering with platform strategy, standards and governance. Work with data teams in the UK, US and Europe on international rollouts, AWS and dbt adoption, and CI/CD practices, while managing third-party vendors and delivery partners.

Snowflake · AWS · dbt +2