151 open roles
Data Engineers
Job description
Key Responsibility: You will be assigned a portfolio of client engagements where you will be expected to:
- Design and build scalable data platforms using modern cloud-native and Lakehouse architectures
- Develop and optimise data pipelines using Python, SQL, and tools such as Azure Data Factory, AWS Glue, Google Cloud Dataflow, Databricks, and dbt
- Modernise legacy data environments, migrating from on-premises solutions to cloud-native platforms such as Microsoft Fabric, Azure Synapse Analytics, AWS Redshift, Google Big. Query, or Databricks
- Engage with clients to conceptualize data solutions aligned to their business strategy
- Support our sales team with pre-sales activities, proof-of-concept deliveries, and technical proposals
- Provide technical guidance and mentorship to junior and intermediate consultants
- Lead technical reviews and contribute to consultants' growth plans
- Identify opportunities to automate manual processes, optimise data delivery, and improve infrastructure scalability
- Work with stakeholders, including executive, product, and analytics teams, to address data infrastructure needs
- Drive knowledge sharing through technical blogs, internal forums, and workshops
- Balance billable project work with team support responsibilities Requirements Data Engineer – Candidate Requirements Intermediate Level 3–5 years' experience 3–5 years of hands-on experience in data engineering. Strong proficiency in Python and/or SQL , including query optimisation. Experience working with both relational and non-relational databases. Experience designing and building data pipelines and data models. Understanding and practical experience with lakehouse architectures , including the medallion pattern. Practical experience with at least one major cloud platform, including: Microsoft Azure AWS Google Cloud Platform (GCP) Familiarity with: Databricks Snowflake Delta Lake Py. Spark Understanding of data transformation frameworks such as dbt . Experience with version control using Git . Understanding of CI/CD practices for data workflows. Strong analytical and problem-solving skills. Ability to perform root-cause analysis on complex data issues. Good communication and stakeholder engagement skills. Senior Level 6–8+ years' experience 6–8+ years of hands-on experience in data engineering. All intermediate-level technical requirements, together with demonstrable experience in: Leading end-to-end data platform delivery. Architecting enterprise-grade lakehouse environments. Implementing data mesh patterns. Infrastructure-as-code using tools such as Terraform, Bicep, AWS CDK or Pulumi. DevOps and CI/CD pipelines. Working effectively with cross-functional teams in a dynamic consulting environment. Mentoring junior engineers. Contributing to technical strategy and solution direction.
Qualifications
Computer Science Information Systems Information Technology or a related field. Master's degree in a relevant field is advantageous. Certifications One or more of the following certifications would be advantageous: Microsoft Fabric Data Engineer Associate Microsoft Azure Data Engineer Associate Databricks Certified Data Engineer Associate Google Professional Data Engineer AWS Certified Data Engineer – Associate Databricks Certified Data Engineer Professional Technology Experience Languages & Frameworks Python Py.
Spark SQL dbt Microsoft Fabric & Azure Microsoft Fabric Lakehouses Fabric Pipelines Fabric Semantic Models Direct Lake Azure Data Factory Azure Data Lake Storage Gen2 Azure Synapse Analytics Azure Databricks Azure Event Hubs Google Cloud Platform Big. Query Cloud Storage Dataflow Dataproc Pub/Sub Amazon Web Services Amazon S3 AWS Glue Amazon Redshift Amazon EMR Amazon Kinesis Databricks & Data Platforms Databricks Delta Lake Unity Catalog MLflow Databricks Workflows Databases Azure SQL Azure Cosmos DB PostgreSQL Snowflake Big.
Query Amazon Redshift DevOps & Infrastructure as Code Git Azure DevOps GitHub Actions Terraform Bicep AWS CDK CI/CD pipelines Streaming & Messaging Azure Event Hubs Azure Stream Analytics Apache Kafka Amazon Kinesis Google Pub/Sub Visualisation & Analytics Microsoft Power BI Microsoft Fabric Real-Time Dashboards Looker / Looker Studio Amazon Quick.
Sight
Description copied from BluePearl's careers page. Read the full posting before you apply.
More jobs at BluePearl
SAP Analytics Cloud (SAC) Planning Architect
BluePearl· Johannesburg, South AfricaOT Architect – Mining
BluePearl· Johannesburg, South AfricaJob Specification: Intermediate SAP Analytics Cloud (Sac) Planning Consultant
BluePearl· Johannesburg, South AfricaTechnical Support Professional – AMS
BluePearl· Cape Town, South AfricaL3 DC Operations Analyst-BCPDR
BluePearl· lagos, Nigeria
More jobs in Johannesburg
Managing Director - Orphoz
McKinsey & Company· Johannesburg, South AfricaSales Account Manager
Supermicro· South Africa - JohannesburgTechnology Enabled Finance Transformation Senior Manager
EY· Johannesburg, GP, ZAB2B SaaS Head of Marketing
RecruitMyMom· Sandton, Johannesburg, Gauteng, South AfricaMedical Sales Territory Manager – JHB Central / North
sabenzait· Johannesburg, Gauteng, South Africa