Senior Data Engineer

Senior Data Engineer Location

• India (Coimbatore preferred) Work Mode
• WFO preferred, hybrid or remote will also be considered for the right candidate Experience - 5+ years in Data Engineering, including hands-on cloud data platform delivery Employment Type
• Full-Time ROLE OVERVIEW Vserve is looking for a Senior Data Engineer to design, build, and own scalable data platforms on Microsoft Azure ecosystems. This role will lead the development of Enterprise Data Warehousing, ETL/ELT pipelines, and analytics- ready data models that power reporting and decision-making for our retail, wholesale, and distribution clients. The ideal candidate is equally comfortable architecting cloud-native pipelines and partnering directly with business stakeholders to translate requirements into reliable, production-grade data solutions.

KEY RESPONSIBILITIES

• Design, build, and maintain scalable Enterprise Data Warehouse (EDW) solutions on Azure (ADF, Databricks, Azure SQL, Microsoft Fabric).
• Develop and orchestrate end-to-end ETL/ELT pipelines using Apache Airflow, Azure Data Factory, and Databricks, ensuring reliability and minimal manual intervention.
• Build and optimize Py. Spark transformations and Delta Lake pipelines to process large-scale datasets into curated, analytics-ready data products.
• Design dimensional data models (star/snowflake schemas), fact and dimension tables, and data marts to support sales, inventory, demand, and profitability reporting.
• Develop Power BI semantic models, datasets, and interactive dashboards enabling self-service analytics and KPI reporting for business stakeholders.
• Implement data quality and validation frameworks, including reconciliation checks, exception reporting, and automated alerting.
• Own data reconciliation between ERP/business systems (e.g., Microsoft Dynamics AX / D365 F&O, POS platforms) and the enterprise data warehouse.
• Implement complex business logic for margin calculations, cost attribution, and promotional/discount allocation to support accurate financial reporting.
• Build automated data ingestion frameworks using Python and REST APIs, and implement incremental loading strategies for efficient, near real-time reporting.
• Tune and optimize SQL/PL-SQL queries and data workflows for performance across high-volume, regulated, or compliance-sensitive datasets.
• Implement CI/CD for data pipelines (GitHub Actions, Git) to improve release consistency and reduce deployment effort.
• Partner with business users, analysts, and SMEs to gather requirements and translate them into scalable, well- documented data engineering solutions.
• Mentor junior data engineers and contribute to data architecture and engineering best practices across the team. CANDIDATE PROFILE — SKILLS & EXPERIENCE Must-Have
• 5+ years of hands-on Data Engineering experience, with demonstrated ownership of production data pipelines end to end.
• Strong experience with Microsoft Azure data services: Azure Data Factory (ADF), Databricks, Azure Data Lake, Azure SQL.
• Hands-on expertise in Apache Spark / Py. Spark and Delta Lake for large-scale data processing.
• Proven experience building and orchestrating pipelines with Apache Airflow or equivalent workflow orchestration tools.
• Advanced SQL skills and solid Python (Pandas, Num. Py) programming ability.
• Strong grounding in dimensional data modeling (star/snowflake schemas, normalization, ER-diagram design).
• Experience with Power BI (DAX, Power Query) and/or Microsoft Fabric for reporting and dashboard delivery.
• Experience implementing data quality, validation, and reconciliation frameworks in a production environment. Good-to-Have
• Exposure to Microsoft Dynamics AX / Dynamics 365 Finance & Operations (D365 F&O) or similar ERP/POS platforms (e.g., Aptos POS).
• Experience with Informatica Power. Center or other enterprise ETL tools.
• Familiarity with MongoDB, PostgreSQL, or DynamoDB alongside relational/warehouse platforms.
• Experience in retail, wholesale distribution, or financial services data domains.
• Working knowledge of CI/CD practices (GitHub Actions, Git) and Agile delivery methodologies.
• Prior experience mentoring engineers or leading a small data engineering pod.

Education

• Bachelor degree in Computer Science, Information Technology, or a related field is required.
• Master degree is a plus.

Requirements

Azure, ADF, Databricks, Py. Spark, Python, SQL, Airflow, Delta Lake, ETL/ELT, Data Warehousing, Data Modeling, Power BI, DAX, Data Quality, REST API, Git/CI-CD, D365 F&O