Data Engineer

Stott and May
Reading
1 month ago
Create job alert

Job Title: Senior Data Engineer

Location: UK (Hybrid, 2–3 days per week in-office)

Rate: £446/day (Inside IR35)

Contract Duration: 6 months

Additional Requirements: May require occasional travel to Dublin office

Overview

We are looking for an experienced Senior Data Engineer to join a Data & Analytics (DnA) team. You will design, build, and operate production-grade data products across customer, commercial, financial, sales, and broader data domains. This role is hands-on and heavily focused on Databricks-based engineering, data quality, governance, and DevOps-aligned delivery.

You will work closely with the Data Engineering Manager, Product Owner, Data Product Manager, Data Scientists, Head of Data & Analytics, and IT teams to transform business requirements into governed, decision-grade datasets embedded in business processes and trusted for reporting, analytics, and advanced use cases.

Responsibilities
  • Design, build, and maintain pipelines in Databricks using Delta Lake and Delta Live Tables.
  • Implement medallion architectures (Bronze/Silver/Gold) and deliver reusable, discoverable data products.
  • Ensure pipelines meet non-functional requirements such as freshness, latency, completeness, scalability, and cost-efficiency.
  • Own and operate Databricks assets including jobs, notebooks, SQL, and Unity Catalog objects.
  • Apply Git-based DevOps practices, CI/CD, and Databricks Asset Bundles to safely promote changes across environments.
  • Implement monitoring, alerting, runbooks, incident response, and root-cause analysis.
  • Enforce governance and security using Unity Catalog (lineage, classification, ACLs, row/column-level security).
  • Define and maintain data-quality rules, expectations, and SLOs within pipelines.
  • Support root-cause analysis of data anomalies and production issues.
  • Partner with Product Owner, Product Manager, and business stakeholders to translate requirements into functional and non-functional delivery scope.
  • Collaborate with IT platform teams to define data contracts, SLAs, and schema evolution strategies.
  • Produce clear technical documentation (data contracts, source-to-target mappings, release notes).
Qualifications
  • 6+ years in data engineering or advanced analytics engineering roles.
  • Strong hands-on expertise in Python and SQL.
  • Proven experience building production pipelines in Databricks.
  • Excellent attention to detail, with the ability to create effective documentation and process diagrams.
  • Solid understanding of data modelling, performance tuning, and cost optimisation.
Desirable Skills
  • Hands-on experience with Databricks Lakehouse, including Delta Lake and Delta Live Tables for batch/stream pipelines.
  • Knowledge of pipeline health monitoring, SLA/SLO management, and incident response.
  • Unity Catalog governance and security expertise, including lineage, table ACLs, and row/column-level security.
  • Familiarity with Databricks DevOps/DataOps practices (Git-based development, CI/CD, automated testing).
  • Performance and cost optimization strategies for Databricks (autoscaling, Photon/serverless, partitioning, Z-Ordering, OPTIMIZE/VACUUM).
  • Semantic layer and metrics engineering experience for consistent business metrics and self-service analytics.
  • Experience with cloud-native analytics platforms (preferably Azure) operating as enterprise-grade production services.


#J-18808-Ljbffr

Related Jobs

View all jobs

Data Engineer

Data Engineer

Data Engineer

Data Engineer

Data Engineer

Data Engineer

Subscribe to Future Tech Insights for the latest jobs & insights, direct to your inbox.

By subscribing, you agree to our privacy policy and terms of service.

Industry Insights

Discover insightful articles, industry insights, expert tips, and curated resources.

New Data Engineering Employers to Watch in 2026: UK and Global Companies Driving the Data Revolution

Data engineering is at the heart of the digital economy, transforming raw data into actionable insights, powering analytics, AI systems, and cloud infrastructure. As the UK and global markets continue to invest heavily in data platforms, pipelines, and real-time analytics, demand for skilled data engineers is growing rapidly. For professionals exploring opportunities on www.DataEngineeringJobs.co.uk , the critical question is: which companies are expanding, hiring, and shaping the future of data-driven business? This article highlights new data engineering employers to watch in 2026, including UK startups, scale-ups, and international firms expanding in the UK.

How Many Data Engineering Tools Do You Need to Know to Get a Data Engineering Job?

If you’re aiming for a career in data engineering, it can feel like you’re staring at a never-ending list of tools and technologies — SQL, Python, Spark, Kafka, Airflow, dbt, Snowflake, Redshift, Terraform, Kubernetes, and the list goes on. Scroll job boards and LinkedIn, and it’s easy to conclude that unless you have experience with every modern tool in the data stack, you won’t even get a callback. Here’s the honest truth most data engineering hiring managers will quietly agree with: 👉 They don’t hire you because you know every tool — they hire you because you can solve real data problems with the tools you know. Tools matter. But only in service of outcomes. Jobs are won by candidates who know why a technology is used, when to use it, and how to explain their decisions. So how many data engineering tools do you actually need to know to get a job? For most job seekers, the answer is far fewer than you think — but you do need them in the right combination and order. This article breaks down what employers really expect, which tools are core, which are role-specific, and how to focus your learning so you look capable and employable rather than overwhelmed.

What Hiring Managers Look for First in Data Engineering Job Applications (UK Guide)

If you’re applying for data engineering jobs in the UK, the first thing to understand is this: Hiring managers don’t read every word of your CV. They scan it. They look for signals of relevance, credibility, delivery and collaboration — and if they don’t see the right signals quickly, your application may never get a second look. In data engineering, hiring managers are especially focused on whether you can build and operate reliable, scalable data systems, handle real-world data challenges and work effectively with analytics, BI, data science and engineering teams. This guide breaks down exactly what they look at first in your application — and how to shape your CV, portfolio and cover letter so you stand out.