Data Engineer

As a Data Engineer, you will be responsible for:

Data Engineering & Development

• Design, build, and maintain high-quality, scalable, and tested data pipelines.
  
  
• Develop and manage Databricks structured streaming pipelines.
  
  
• Build and optimize event-driven and real-time data processing solutions.
  
  
• Implement and maintain Unity Catalog-based Lakehouse architecture.
  
  
• Develop analytics-ready datasets to support business insights and reporting.
  
  Platform & Automation
  
  
• Build and manage CI/CD pipelines using Azure DevOps.
  
  
• Identify and implement automation opportunities across workflows.
  
  
• Ensure reliable and stable data platform operations.
  
  
• Apply governance, security, and documentation standards.
  
  Data Quality & Reliability
  
  
• Establish the Data Lakehouse as a trusted and reliable source of truth.
  
  
• Monitor, troubleshoot, and resolve data incidents.
  
  
• Support business users and technical teams with data-related queries.
  
  
• Continuously improve platform performance and reliability.
  
  Collaboration & Support
  
  
• Work closely with data science, analytics, platform, and business teams.
  
  
• Champion data engineering best practices.
  
  
• Provide technical guidance and mentorship where required.
  
  
• Contribute to a culture of learning, quality, and continuous improvement.
  
  Essential Skills
  
  
• Strong experience with Azure Databricks and cloud data platforms.
  
  
• Advanced proficiency in Python, PySpark, and SQL.
  
  
• Experience developing Spark/Databricks pipelines.
  
  
• Hands-on experience with structured streaming and event-driven systems.
  
  
• Strong understanding of Lakehouse architecture and best practices.
  
  
• Experience with Unity Catalog.
  
  
• Expertise in Azure DevOps and CI/CD pipelines.
  
  
• Knowledge of data modelling (dimensional/star schemas).
  
  
• Experience working in Agile environments.
  
  Desirable Skills
  
  
• Exposure to multiple data technology stacks.
  
  
• Experience in large-scale enterprise environments.
  
  
• Knowledge of security, governance, and compliance frameworks