Latest streaming engineer Jobs

Senior Software Engineer - Java Streaming - Connectors

This role involves developing and maintaining high-performance data connectors and integrations for the ClickHouse real-time analytics platform, with a focus on JVM-based technologies like Java, Kafka, Flink, and Beam. The engineer will work on low-level database drivers, SDKs, and streaming connectors to ensure seamless integration with tools such as Spark, Grafana, and Power BI. Collaboration with open-source communities and enterprise users is central to improving scalability, reliability, and developer experience across global data ecosystems.

ClickHouse United Kingdom

Software Engineer - AI / ML

This role involves building and maintaining scalable data pipelines for AI/ML applications, focusing on data quality, reliability, and observability. The engineer will work closely with data science and product teams to design and implement data infrastructure on Google Cloud Platform using tools like Airflow, Dataflow, and Spark. Key responsibilities include managing batch and streaming pipelines, ensuring robust data tracking, and supporting cross-functional initiatives in a collaborative environment.

EF Recruitment Victoria, Greater London, London, SW1P 1BX, United Kingdom £350 pd

Hybrid Contract

Data Engineer

This role involves building scalable data pipelines and automating ETL processes to support a data-driven organisation. The engineer will work on streaming data ingestion, data modelling, and cost optimisation while promoting best practices in security and governance. Collaboration with data engineering teams and stakeholder engagement are key to delivering reusable, high-quality data solutions.

Third Nexus Group Limited London, United Kingdom £75,000 – £100,000 pa

Hybrid Permanent Clearance Required

Data Engineer

Design, build, and operate data infrastructure for high-level applications using open-source data integration and streaming technologies. Focus on scalability, reliability, and operability within distributed production systems. Work in a consultancy environment requiring UK security clearance.

Hexwired Recruitment Limited Reading, Berkshire, United Kingdom £40,000 – £55,000 pa

Senior Software Engineer II, Kora Compute

Design and develop the compute layer for Kora, a cloud-native event streaming platform built on Apache Kafka. Focus on building highly available, secure, and scalable multi-tenant systems in the cloud. Work closely with stakeholders to deliver end-to-end technical solutions for distributed systems challenges.

Confluent United Kingdom

Remote Permanent

Data Engineer

This role involves designing, building, and optimising scalable data pipelines using Python and Databricks within a Lakehouse architecture. The engineer will develop ETL/ELT workflows, ensure data quality and governance, and support analytics and machine learning initiatives. Collaboration with data scientists and analysts is key, along with mentoring junior engineers and implementing monitoring for data systems.

Adecco London, United Kingdom £600 – £700 pd

Hybrid Contract

Senior Software Engineer

As a Senior Software Engineer, you will work on the Tableflow project, developing and maintaining a multi-tenant, highly available storage infrastructure for Kora. Your responsibilities include designing and implementing strategies for table materialization and compaction at scale, ensuring high availability and scalability, and collaborating with cross-functional teams to integrate storage features.

Confluent United Kingdom

Remote Permanent

Data Engineer

This role involves building and maintaining scalable data pipelines on Google Cloud Platform, with a focus on BigQuery, Cloud Composer, and Airflow. The engineer will implement data quality checks, monitoring, and observability using tools like Monte Carlo and Looker, while collaborating closely with data science and product teams. The position emphasizes robust data infrastructure to support AI/ML workflows across the organization.

EF Recruitment London, United Kingdom £300 – £350 pd

Hybrid Contract

Lead Data Engineer

This role involves designing and building scalable data pipelines and distributed systems to support AI-driven operations. The engineer will implement real-time and batch processing, optimize vector search for AI features, and ensure data quality and governance. Close collaboration with backend and AI teams is required to support product development and model deployment.

Matchtech London, United Kingdom £80,000 – £110,000 pa

On-site Permanent

Senior Data Engineer (Databricks)

Design and build scalable data pipelines within a Databricks and Azure-based Lakehouse environment, focusing on high-performance batch and streaming solutions using PySpark and Delta Lake. Implement medallion architecture patterns and data quality frameworks while collaborating with data scientists and product teams. Influence platform evolution and engineering best practices in a cloud-first, technically rigorous team.

Tenth Revolution Group Surrey, United Kingdom £75,000 – £90,000 pa

Software Engineer

Develop and maintain backend services, APIs, and automation for a cyber big data platform, with a focus on security, cloud infrastructure, and DevOps excellence. Work extensively with Go, Python, Terraform, and Kong in AWS or Azure environments to support secure data engineering and governance. Contribute to CI/CD pipelines, infrastructure as code, and platform services that enhance control, auditability, and operational reliability.

Experis Sheffield, South Yorkshire, United Kingdom £400 – £460 pd

Hybrid Contract Clearance Required

DV Cleared Data Engineer

Job Title: DV Cleared Data EngineerLocation: BathDuration: 12 monthsRate: Up to £500 per dayMust be willing and eligible to go through the DV clearance processOur client, a reputable organisation supporting high-profile government programmes, is hiring a DV Cleared Data Engineer...

Experis Bath, Somerset, United Kingdom £400 – £500 pd

Databricks Data Engineer

Design and build scalable data pipelines and analytics solutions using Azure Databricks and Azure Data Factory within an HR technology environment. Work with complex, multi-format datasets to deliver production-ready, secure, and observable data products integrated across the employee lifecycle. Collaborate with engineering, product, and HR teams in an Agile environment with strong DevOps and CI/CD practices.

Coltech Recruitment London, United Kingdom £400 – £600 pa

AI Deployment Engineer

This role involves owning the data infrastructure for AI deployments, integrating AI solutions into existing business systems, and ensuring reliable and scalable production-grade pipelines. You will work closely with AI strategists to build and maintain the technical backbone that supports AI tooling, including data pipelines, APIs, and system integrations.

Emponics Almondsbury, Gloucestershire, BS32 4DS, United Kingdom £80,000 – £130,000 pa

Permanent

Senior Data Engineer

This role involves designing and maintaining scalable ETL/ELT pipelines for large-scale economic and financial datasets, using Azure Databricks, Spark, and Delta Lake. The engineer will implement data governance with Azure Purview, optimize cloud workloads, and collaborate with economists and data scientists to deliver reliable data infrastructure. Key responsibilities include automation, CI/CD integration, and mentoring within a hybrid working environment.

Velocity Talent London, United Kingdom £75,000 – £80,000 pa

Hybrid Permanent Clearance Required