Latest Big Data Jobs

Spotlight

Senior Data Engineer

This role involves designing, building, and operating scalable batch and streaming data pipelines that transform vast, unstructured exposed data into reliable, queryable intelligence. The Senior Data Engineer will own end-to-end data workflows, ensure data quality and model readiness, and collaborate closely with ML teams on feature engineering. The position is central to enabling analytics and AI capabilities on a platform that detects corporate data exposure across supply chains.

Lab 1 London, Greater London, United Kingdom £85,000 – £125,000 pa

Hybrid Permanent

AI Application/Big Data Engineer

This role involves designing and building AI-driven data pipelines and integrations to enable a QAS-powered response suggestion capability in Salesforce Service Cloud. Responsibilities include optimizing data processing, integrating LLMs, and ensuring data reliability and performance.

ARM London, United Kingdom

Hybrid Contract

Senior Data Engineer (SC Cleared)

This role involves designing and maintaining scalable data pipelines on Azure Databricks to support economic analysis, forecasting, and modelling. The engineer will work extensively with large financial datasets, implementing complex transformations using PySpark or Scala, and optimising pipeline performance and cost. The position is embedded in a security-cleared environment, indicating work with sensitive data.

LMA Recruitment London, United Kingdom £70,000 – £75,000 pa

On-site Permanent Clearance Required

Senior Data Engineer

This role involves designing and building modern data pipelines to support large-scale migration and regulatory data initiatives within a global banking environment. The engineer will collaborate with cross-functional teams to implement robust data quality checks, validate data processes, and ensure secure access controls. Work includes data migration, transformation, and integration across cloud platforms with a focus on scalability and compliance.

Fraser & Co. Talent Partners Limited Mount Pleasant, Greater London, London, United Kingdom £450 – £550 pd

Hybrid Contract Clearance Required

Data Engineer

The role involves building and maintaining scalable data pipelines using Apache Spark and Airflow within a cloud-based, containerised environment powered by Kubernetes. The engineer will support both batch and streaming workloads as part of a broader data platform modernisation initiative. The position blends hands-on development with integration across distributed systems in a hybrid working setup.

Huxley Associates London, City And County Of the City Of London, United Kingdom £500 ph

Hybrid Contract

Data Engineer

Design and build scalable data pipelines and ingestion processes using big data technologies like Apache Spark and Hadoop within secure government environments. Develop and optimise high-performance data integration solutions on AWS, leveraging tools such as Apache NiFi and programming languages like Python or Java. Work collaboratively in Agile teams to deliver reliable, secure data platforms supporting analytics and operational decision-making.

Copello Manchester, United Kingdom

Hybrid Permanent Clearance Required

Data Engineer

Design and build scalable data pipelines and high-performance data platforms within secure government environments. Work closely with Agile teams to integrate data from multiple sources using modern big data technologies. Focus on developing reliable, tested ETL processes and optimising data systems for analytics and operational use.

Copello Southampton, Hampshire, SO19 8NJ, United Kingdom

Hybrid Permanent Clearance Required

Data Engineer DV Cleared

Data Engineer Opportunity, DV-cleared onlyLondon Manchester BristolA progressive, leading-edge UK consulting and technology organisation is hiring Data Engineers to deliver mission-critical work across defence and security programmes, building modern data platforms and production-grade pipelines that enable better decisions at pace....

Datatech London, United Kingdom

DV Cleared Data Engineer

Design and maintain secure, scalable data pipelines and platforms in a classified government environment. Work with batch and streaming data using Python, SQL, and Elasticsearch, within Agile delivery frameworks. Support intelligence operations through robust data engineering and modern architecture on cloud and on-premise systems.

Experis Bath, Somerset, TA7 8PH, United Kingdom £400 – £500 pd

On-site Contract Clearance Required

Data Engineer, Prime Video Finance Analytics

Come build the future of entertainment with us. Are you interested in shaping the future of movies and television? Do you want to define the next generation of how and what Amazon customers are watching?Prime Video is a premium streaming...

Amazon London, United Kingdom

Permanent

Data Engineer, Strategic Account Services

What is the Amazon Marketplace?Amazon is the largest marketplace on earth. Millions of customers shop in Amazon’s marketplaces globally. Every day, customers browse, purchase, and review products sold by third-party (3P) sellers right alongside products sold by Amazon. Since 2000,...

Amazon London, United Kingdom

Permanent

Pre-sales Senior Technical Solutions Engineer (Data and AI) x4

This role involves partnering with sales teams to demonstrate Databricks' Data Intelligence Platform through technical evaluations, custom demos, and solution design. You'll engage directly with clients to understand data and AI challenges, delivering tailored technical narratives and reference architectures. The position emphasizes hands-on coding, cloud integration, and building trusted advisor relationships in sectors like media, finance, and public services.

Databricks London, United Kingdom

Hybrid Permanent

Pre-sales Senior Technical Solutions Engineer (Data and AI)

Act as a technical advisor to enterprise clients, delivering custom demonstrations and proof-of-concepts for the Databricks Data Intelligence Platform. Focus on solving complex data challenges in sectors like financial services, retail, and media by designing scalable data architectures and integrating with cloud ecosystems. Collaborate with sales teams to translate business needs into technical solutions while building trusted relationships with customers.

Databricks London, United Kingdom

Hybrid Permanent

Data Engineer

As a Data & AI Engineer, you will design and build scalable data pipelines, integrate data from multiple sources, and develop advanced analytics and AI solutions. You'll work closely with analysts and stakeholders to deliver high-impact, production-ready solutions, with a focus on data quality and performance.

Tenth Revolution Group Newcastle upon Tyne, United Kingdom £45,000 – £51,000 pa

Hybrid Permanent

Data Engineer

This role involves designing and building scalable data pipelines, developing AI and ML solutions, and integrating data from multiple sources. You'll work on large-scale transformation projects, collaborate with cross-functional teams, and ensure data quality and governance.

Tenth Revolution Group Manchester, United Kingdom £40,000 – £55,000 pa

Senior Software Engineer

As a Senior Software Engineer at Confluent, you will design, implement, and maintain Golang infrastructure services, primarily as Kubernetes operators, to support the Confluent Cloud Platform. You will work closely with the engineering team to ensure the availability, performance, and security of the platform, focusing on large-scale systems and cloud infrastructure.

Confluent London, London, City Of, United Kingdom

Remote Permanent