Future Tech Jobs Future Tech Jobs

Big Data Engineer Jobs

Engineers who design, build, and optimise the infrastructure that processes and stores massive datasets. A critical role in the data engineering ecosystem, ensuring data is accessible and usable at scale.

Open roles
0
Hiring companies
1
Browse latest jobs → Get an alert

Big Data Engineers are the architects of the data infrastructure that powers modern businesses. They design, build, and optimise the systems that process, store, and manage vast amounts of data. This role is distinct from data science and machine learning engineering, focusing instead on the foundational layers that enable data-driven decision-making. Big Data Engineers work with technologies like Apache Hadoop, Apache Spark, and data lakes, ensuring that data is accessible, reliable, and scalable.

What the role does

Inside the role of a Big Data Engineer

A typical week for a Big Data Engineer is a mix of designing and implementing data pipelines, troubleshooting issues, and collaborating with cross-functional teams.

  1. 01
    Design and implement data pipelines using tools like Apache Spark and Hadoop.
  2. 02
    Optimise data storage and retrieval processes for efficiency and scalability.
  3. 03
    Collaborate with data scientists and analysts to understand data requirements.
  4. 04
    Troubleshoot and resolve issues in the data infrastructure.
  5. 05
    Document and maintain data architecture and processes.
  6. 06
    Stay updated with the latest big data technologies and best practices.
Career ladder

From Junior to Principal

A typical UK progression for big data engineers. Years are guidance — strong people move faster, and many senior folks sidestep into research, product or management.

  1. Level 1

    Junior Big Data Engineer

    0–2 yrs

    Assists in the design and implementation of data pipelines, focusing on learning and contributing to small-scale projects.

  2. Level 2

    Big Data Engineer

    2–5 yrs

    Takes ownership of designing and implementing data pipelines, ensuring they meet performance and scalability requirements.

  3. Level 3

    Senior Big Data Engineer

    5–8 yrs

    Leads the design and optimisation of complex data infrastructure, mentoring junior engineers and collaborating with cross-functional teams.

  4. Level 4

    Principal Big Data Engineer

    8+ yrs

    Strategises and oversees the development of enterprise-wide data infrastructure, driving innovation and best practices.

Pathway

How to become a Big Data Engineer

There's no single route, but most people follow some version of these steps.

  1. 1

    Learn the Basics

    Start by gaining a solid understanding of big data technologies and tools, such as Hadoop and Spark.

  2. 2

    Build Small Projects

    Gain hands-on experience by working on small-scale data pipeline projects, focusing on design and implementation.

  3. 3

    Optimise and Scale

    Learn to optimise data storage and processing for efficiency and scalability, handling larger datasets.

  4. 4

    Lead Projects

    Take on leadership roles in designing and implementing complex data infrastructure, mentoring junior engineers.

  5. 5

    Strategise and Innovate

    Develop strategic initiatives to drive innovation in data infrastructure, collaborating with cross-functional teams.

  6. 6

    Mentor and Advise

    Serve as a senior advisor, guiding the development of enterprise-wide data strategies and best practices.

No Big Data Engineer Jobs jobs right now

We don't have any matching roles at the moment, but new jobs are added daily.

Latest jobs

Source Global Research logo

AI & Data Engineer

This role involves maintaining and enhancing the company's data infrastructure while leading AI-driven improvements. You will design and deploy AI features, build vector databases, and transform ETL/ELT processes into AI-ready pipelines. Additionally, you will mentor the team on MLOps and AI best practices, ensuring data quality and system performance.

Source Global Research London, United Kingdom
Hybrid Permanent
Confluent logo

Senior Software Engineer II, Kora Compute

This role involves building and maintaining the compute layer of Kora, a cloud-native event streaming platform for Kafka. You will work on highly available, reliable, and scalable systems, collaborating closely with the team and stakeholders to deliver top-notch solutions. The role focuses on solving complex distributed systems and infrastructure challenges, with a strong emphasis on customer success and team collaboration.

Confluent United Kingdom
Remote Permanent
RSM logo

Senior Data Engineer

The Senior Data Engineer / Analytics Lead will establish and lead the data engineering and analytics capability within RSM's Forensic Technology team. Responsibilities include designing and building data pipelines, working directly with clients and regulators, mentoring junior staff, and setting technical standards. This greenfield role offers the opportunity to shape the function and deliver impactful forensic solutions.

RSM London, United Kingdom
On-site Permanent
Databricks logo

EMEA Energy Industry Go-To-Market (GTM) Leader

This role involves leading the go-to-market strategy for Databricks in the energy sector across EMEA. Responsibilities include setting AI vision with top energy accounts, driving pipeline growth, and aligning cross-functional teams. The position requires deep industry expertise and a proven track record in strategic consulting and technology commercialization.

Databricks United Kingdom
Hybrid Permanent
Genomics England logo

Bioinformatics Engineer (we have office locations in Cambridge, Leeds & London)

The Bioinformatics Engineer will develop and scale knowledge platforms that support the NHS Whole Genome Sequencing (WGS) Genomic Medicine Service. Key responsibilities include building APIs, data models, and pipelines for high-quality genomic data, collaborating with genomic data scientists and engineers, and ensuring clinical and biological accuracy in software. The role involves working in an agile team to deliver robust, testable, and clinically relevant systems.

Genomics England Cambridge, Cambridgeshire, United Kingdom £56,000 – £65,000 pa
On-site Permanent Clearance Required
Belcan logo

Technical Data Engineer

The role involves authoring, reviewing, and maintaining Aircraft Maintenance Manuals (AMM) and Troubleshooting Manuals (TSM) for Landing Gear systems. You will work within a transnational technical data environment, ensuring maintenance documentation accurately reflects aircraft configuration and meets regulatory requirements. Key responsibilities include managing source data, responding to customer queries, and driving continuous improvement in technical data processes.

Belcan Filton, Gloucestershire, United Kingdom £45 ph
Hybrid Contract Part-time Clearance Required
Browse all jobs →
Hiring locations

Where this role is hiring

The locations with the most live listings for this role today.

London

Location
1 open
FAQs

Common questions

  • Key skills include proficiency in big data technologies like Hadoop and Spark, strong programming skills, and a deep understanding of data storage and retrieval systems.

  • A Big Data Engineer focuses on building and optimising the infrastructure that processes and stores data, while a Data Scientist focuses on analysing and modelling data to derive insights.

  • Common tools include Apache Hadoop, Apache Spark, Apache Kafka, and data lakes like Amazon S3 and Google Cloud Storage.

  • Advance by gaining experience, leading projects, and staying updated with the latest technologies and best practices in data engineering.

  • Salaries vary based on experience and location. For specific salary ranges, please refer to the salary section on this page.

Hiring big data engineers?

Post your role in 90 seconds and reach the specialist audience that already reads this page.

Post a job → See pricing

🍪 Basic, anonymous analytics (aggregate counts only, never shared) are always on. Accept to also enable third-party tools — Google Analytics, Microsoft Clarity, Mixpanel, Leadfeeder — which help us match you with relevant Data Engineering roles. We never sell your data or use it for cross-site advertising. See our Cookie Policy.