Senior Data Engineer

We are seeking an experienced Senior Data Engineer to join our team. The project entails the development and optimization of a new Data Platform to adjust our current data processing system. The Senior Data Engineer will be responsible for designing, implementing, and maintaining the platform to meet the business's data needs.

f you have a passion for Data Software Engineering to deliver impactful web applications, we would love to hear from you.

Responsibilities

Develop data processing pipelines using Azure Data Factory and Azure Functions Implement data ingestion and transformation processes using Databricks and Py. Spark Write efficient code in Python and SQL to extract, transform, and load data Collaborate with stakeholders to gather requirements and ensure the platform meets business needs Monitor and troubleshoot the platform's performance and address any issues Implement data quality assurance measures and process improvements Optimize data storage and retrieval operations Conduct data analysis and generate insights to guide decision-making Train and mentor junior Data Engineers Participate in peer code reviews and collaborative team activities Requirements 3+ years of hands-on experience in big-data field and expertise in Data Software Engineering Strong knowledge and experience with Azure Data Factory, Azure Functions, and Databricks Proficiency in Py.

Spark, Python, and SQL Understanding of batch processing and ETL principles in Data Warehouses Experience with data completeness signals and orchestration Knowledge of schema migrations and datasets evolution Familiarity with approaches for historical reprocessing and data correction Good problem-solving and communication skills Proficient in English with a B2+ level, strong speaking English Nice to have Familiarity with Microsoft Power BI Competency in Azure Blob Storage, Azure File Storage