Data Engineering Lead

Data Engineering Lead

Certain Advantage are hiring for a Data Engineering Lead based in London.

This role is on an initial contract till the end of the year and is required to be onsite 3 days a week.

The successful candidate will lead the data acquisition function by defining ingestion strategy and building Azure-based data pipelines.

You will translate end-user requirements into scalable engineering solutions, act as the domain’s primary Data Manager for data governance, lineage, quality, and master data management, and lead a team of data engineers in delivering reliable, compliant, and high-value data solutions to Trading, Risk, Finance, and Operations.

Your responsibilities:

Own ingestion strategy and define scalable batch/streaming data patterns

Design and deliver Azure-based data pipelines and platform solutions

Standardize connectors, schemas, controls, CI/CD, IaC, and automated testing

Lead onboarding of enterprise and market data sources with SLAs and observability

Partner with Product Owners, Architects, and business stakeholders on data design and governance

Act as domain Data Manager for lineage, metadata, quality, access, and master data ownership

Ensure compliance with data lifecycle, audit, records management, and governance standards

Drive engineering best practices, operational excellence, and continuous improvement

Manage capacity planning, skills development, and vendor contributions

The individual:

Expertise in Azure data services (ADLS, ADF, Databricks, Synapse, Event Hub/Kafka), Python, SQL, Spark/PySpark.

Experience with data modeling, metadata and lineage tooling; performance & cost optimization.

Proven leadership in engineering teams; Agile/DevOps ways of working; CI/CD; test automation.

Exposure to energy trading systems (ETRM/Endur), market data (ICE, Platts, Bloomberg), SAP.

Degree in Computer Science/Engineering or equivalent experience; Azure Data Engineer or Databricks certifications a plus.

Does this sound like your next career move? Apply today.

Working with Certain Advantage

We go the extra mile to find the best people for the job. If you’re hunting for a role where you can make an impact and grow your career, we’ll work with you to find it.

We work with businesses across the UK to find the best people in Finance, Marketing, IT and Engineering. If this job isn’t for you, head to (url removed) and register for job alerts and career guidance tips