Data Engineer - SC Cleared

Role: Data Engineer

Location: Edinburgh or Luton (hybrid/onsite as required)
Engagement: Contract - Inside IR35
Security Clearance: Active SC Clearance required

Rate: £500 - £600 per day - inside IR35

Project Description

We are seeking an experienced Data Engineer to design, build, deploy, and maintain robust data platforms and pipelines within a secure environment. You will be responsible for the end-to-end data engineering lifecycle, transforming raw data into high-quality, consumable datasets that support analytics, reporting, and advanced modelling.

You will own and optimise the data operations infrastructure, ensuring performance, reliability, scalability, and security as data volumes and processing demands grow. This role requires strong problem-solving skills, the ability to integrate data from multiple sources, and hands-on experience with modern data engineering tools and practices.

Key Responsibilities

Design, develop, deploy, and support scalable data infrastructure, pipelines, and architectures

Orchestrate ingestion and storage of raw data into structured and unstructured data solutions

Implement reliable, automated, and well-tested data ingestion and processing workflows

Build and maintain batch and real-time data processing systems

Manage and optimise performance, reliability, scalability, and security of data platforms

Support data governance, quality, and compliance requirements

Prepare data pipelines for descriptive, predictive, and prescriptive analytics

Collaborate closely with data scientists, architects, IT teams, and business stakeholders

Identify opportunities for new data acquisition and improved data utilisation

Monitor, manage, and enhance data quality and reliability through automated tooling

Skills and Experience Required

Active SC Clearance (mandatory)

Strong experience designing and maintaining data pipelines, data warehouses, and data platforms

Solid knowledge of DataOps practices, including CI/CD, containerisation, and workflow orchestration

Hands-on experience with ETL/ELT frameworks and big data tools (e.g. Spark, Airflow, Hive)

Proficiency in programming languages such as Python, Java, and SQL

Experience with SQL and NoSQL database design and optimisation

Strong understanding of batch and streaming data processing

Degree in a STEM-related field; Master's degree desirable

Data engineering certifications (e.g. IBM Certified Data Engineer) are advantageous