Azure Data Engineer (Mid to Senior)

Industry: Banking and financial services · Level: Mid to Senior · Experience: Mid 4–6 years, Senior 7+ years

Employment type: Fixed-term contract of at least 12 months, with a strong possibility of extension or conversion to permanent · Rate: Competitive hourly rate

Summary

Our client, a leading financial services institution, is looking for an Azure Data Engineer to build and operate the cloud data platform behind regulatory reporting, risk, finance and customer analytics. You will design reliable, secure pipelines on Azure and Microsoft Fabric, and deliver governed data the business can trust.

Minimum qualifications

• Bachelor's degree or National Diploma (NQF level 6+) in Computer Science, Information Systems, Engineering, Mathematics or a related field

• 4+ years of data engineering experience for mid-level; 7+ years for senior

• At least 2 years of hands-on Azure data platform experience

• Experience in banking, insurance or another regulated financial environment

Certifications

• Microsoft Certified: Fabric Data Engineer Associate (DP-700) (strongly preferred)

• Microsoft Certified: Azure Data Engineer Associate (DP-203) (retired in 2025 but still valued)

• Microsoft Azure Data Fundamentals (DP-900) or Azure Fundamentals (AZ-900) (advantageous)

• Databricks Certified Data Engineer Associate or Professional (advantageous)

• Azure Solutions Architect Expert (AZ-305) or Azure Security Engineer (AZ-500) (advantageous for senior)

Duties

• Design, build and maintain batch and streaming data pipelines on Azure and Microsoft Fabric

• Develop ETL/ELT processes using Azure Data Factory, Synapse pipelines, Databricks and Fabric Data Factory

• Build and optimise data lakes and warehouses on ADLS Gen2, Synapse, Fabric lakehouses and Azure SQL

• Ingest data from core banking, payments, CRM and third-party sources, including real-time feeds via Event Hubs or Kafka

• Implement data quality rules, lineage, cataloguing and monitoring with Microsoft Purview

• Secure data using Entra ID, RBAC, Key Vault, encryption and masking for sensitive customer information

• Automate deployments with CI/CD and Infrastructure as Code

• Tune performance, troubleshoot failed jobs and manage cloud cost

• Work with analysts, data scientists and business teams to deliver trusted datasets for Power BI and reporting

• Senior level: define data architecture and standards, lead engineers and make platform design decisions

Requirements

Technical skills

• Python and advanced SQL; PySpark or Scala (advantageous)

• Azure services: Data Factory, Synapse Analytics, Databricks, Data Lake Storage Gen2, Azure SQL, Event Hubs, Stream Analytics, Key Vault, Purview, Azure Monitor

• Microsoft Fabric: lakehouse, warehouse, notebooks, pipelines and OneLake

• Data modelling: dimensional modelling, medallion architecture, data vault

• Infrastructure as Code: Bicep, ARM or Terraform

• Git, Azure DevOps or GitHub Actions, automated testing for data pipelines

• Power BI exposure (advantageous)

Financial services knowledge

• Banking data domains such as payments, accounts, lending, risk and finance

• Data governance, lineage and retention; POPIA, BCBS 239, PCI DSS and audit requirements

• Experience supporting regulatory and management reporting (advantageous)

Personal attributes

• Strong analytical mindset with a focus on data accuracy and reliability

• Able to explain technical trade-offs to non-technical stakeholders

• Works well in agile, cross-functional delivery teams

Skills

Azure, Python, SQL, PySpark, Scala, Azure Data Factory, Azure Synapse Analytics, Azure Databricks, Microsoft Fabric, OneLake, Data Lake Storage Gen2, Azure SQL, Event Hubs, Stream Analytics, Microsoft Purview, Key Vault, Entra ID, Azure Monitor, Power BI, ETL, ELT, Data Lakehouse, Medallion Architecture, Dimensional Modelling, Data Vault, Bicep, ARM Templates, Terraform, Azure DevOps, GitHub Actions, Git, CI/CD, Data Governance, Data Quality, BCBS 239, POPIA, PCI DSS, Banking