Overseas Contractor

Key Responsibilities

Design develop and maintain scalable data pipelines using Azure Databricks and Azure Data Factory Develop ETLELT solutions for ingesting transforming and loading data from multiple sources Build and optimize Py. Spark applications and Spark SQL transformations for largescale data processing Create and manage Databricks notebooks workflows jobs and clusters Implement Delta Lakebased data solutions ensuring data quality reliability and performance Develop data ingestion frameworks using Azure Data Lake Storage ADLS Gen2 Monitor troubleshoot and optimize data pipelines and Databricks workloads Implement CICD pipelines for Databricks artifacts using Azure DevOps or GitHub Collaborate with data architects business analysts and stakeholders to deliver enterprise data solutions Ensure security governance and compliance using Unity Catalog Azure Key Vault and related Azure services Mandatory Skills Strong programming experience in Python Expertise in Py.

Spark and Spark SQL Handson experience with Azure Databricks Experience with Azure Data Factory ADF Knowledge of Azure Data Lake Storage ADLS Gen2 Strong SQL and data modeling skills Experience in ETLELT development and optimization Exposure to Delta Lake Databricks Workflows and Notebook development Version control using Git.

Git. Hub Understanding of performance tuning and troubleshooting Spark jobs Goodto. Have Skills Azure Synapse Analytics Unity Catalog Azure DevOps CICD Azure Key Vault Snowflake Data Governance and Data Quality frameworks Monitoring and Logging solutions in Azure CDC and Realtime Data Processing Preferred Certifications Microsoft Azure Data Engineer Associate DP203 Databricks Data Engineer Associate