Senior Data Platform Engineer

Senior Data Platform Engineer - HYBRID

We are working with one of our largest customers who are looking for a talented and experienced Senior Data Platform Engineer

Overall Objectives of the Job

We are seeking a skilled and experienced Senior Data / Platform Engineer to join our Data & Analytics team. This hybrid role combines hands-on data engineering on Databricks and Azure Synapse with platform administration responsibilities across our cloud data estate. The role holder will design, build, and operate scalable data pipelines while also maintaining the underlying Azure platform — including infrastructure-as-code (Pulumi), CI/CD automation, monitoring, security, and Databricks workspace administration. The ideal candidate combines strong Python/PySpark engineering skills with deep Azure platform knowledge and a service-excellence mindset.

Duties and Responsibilities

• Lead solution design activities, collaborating with peers and mentoring junior colleagues to define and execute the team backlog.
• Develop, test, and document scalable ETL/ELT data pipelines and workflows using Databricks and Azure Synapse to ingest and transform data from a variety of sources.
• Administer and maintain Azure data platform components including Synapse, Databricks, ADLS Gen2, Key Vault, networking (VNets, NSGs, Managed Private Endpoints) and access control (RBAC, ACLs).
• Manage infrastructure-as-code across Dev, Staging, and Production environments using Pulumi (and equivalents such as Terraform / Bicep).
• Design and operate CI/CD pipelines using GitHub Actions (with OIDC federation) and/or Azure DevOps, supporting trunk-based development practices.

Must Have

• Databricks: hands-on experience building and optimizing pipelines, managing Delta Lake, and administering workspaces (cluster policies, Unity Catalog, Secret Scopes, Workflows).
• Python / PySpark: strong programming skills for data processing, automation, and scripting.
• Azure data stack: Synapse, Databricks, ADLS Gen2, Key Vault — including Linked Services, Managed Identity, and Spark Pool configuration.
• Azure platform fundamentals: compute, storage, networking (VNets, NSGs, Private Endpoints), identity and RBAC.
• CI/CD: GitHub Actions (with OIDC federation) and/or Azure DevOps for data and platform deployments.
• Infrastructure-as-Code: Pulumi (or Terraform / Bicep) across multiple environments.

Due to the volume of applications received for positions, it will not be possible to respond to all applications and only applicants who are considered suitable for interview will be contacted.

Proactive Appointments Limited operates as an employment agency and employment business and is an equal opportunities organisation

We take our obligations to protect your personal data very seriously. Any information provided to us will be processed as detailed in our Privacy Notice, a copy of which can be found on our website