Job description
Data Engineer Location-New Delhi Exp-3-5 years
- Design, develop, and maintain robust data pipelines using Azure Data Factory (ADF).- Build ETL/ELT processes that ingest, transform, and load data from various sources into data lakes, warehouses, or databases.
- Implement both batch and near real-time data flows.
- Create, schedule, and monitor ADF pipelines.
- Implement complex data workflows, including dependency management, parameterization, and dynamic pipeline design.
- Integrate ADF with other Azure services (Blob Storage, Synapse, Databricks, Key Vault, etc.).
- Implement logging, alerting, and retry logic for production stability.
- Analyze large datasets to identify trends, issues, and transformation needs.
- Write performant SQL queries and transformations.
- Collaborate with data analysts to understand data requirements and structure transformations accordingly.
- Ensure data is accurate, consistent, and aligned with business logic.
- Quickly diagnose and resolve data quality issues, pipeline failures, or performance bottlenecks.
- Perform root cause analysis for pipeline errors and implement long-term fixes.
- Debug and optimize Spark or SQL transformations in environments like Databricks.
- Implement data validation and quality checks within pipelines.
- Perform reconciliation and sanity checks across data sources.
- Maintain data dictionaries and metadata repositories.
- Work closely with data architects, analysts, BI developers, and business users.
- Translate business data needs into technical implementations.
- Document pipeline logic, data flows, and troubleshooting procedures.
- Work closely with data architects, analysts, BI developers, and business users.
- Translate business data needs into technical implementations.
- Document pipeline logic, data flows, and troubleshooting procedures.
- Optimize ADF pipeline execution time and cost.
- Improve query performance using indexing, partitioning, and efficient transformation logic.
- Recommend improvements in data infrastructure and tooling.
Benefits
What a Consulting role at Thoucentric will offer you? Opportunity to define your career path and not as enforced by a manager A great consulting environment with a chance to work with Fortune 500 companies and startups alike. A dynamic but relaxed and supportive working environment that encourages personal development.
Be part of One Extended Family. We bond beyond work - sports, get-togethers, common interests etc. Work in a very enriching environment with Open Culture, Flat Organization and Excellent Peer Group. Be part of the exciting Growth Story of Thoucentric!