Job description
We are seeking an experienced Data Engineer to design, build, and optimize scalable data pipelines in a cloud-based environment. This role focuses on developing efficient data ingestion frameworks, ensuring data quality, and enabling seamless data access for analytics and business operations. The ideal candidate will have strong hands-on experience with modern data engineering tools and AWS services.
Roles & Responsibilities:
- Design and implement scalable data ingestion and processing pipelines using AWS services and modern data tools
- Perform data ingestion, validation, and transformation using existing data frameworks
- Develop and enhance data ingestion frameworks to support fast and reliable data onboarding
- Process structured and semi-structured data using batch processing and store data in S3
- Transform and load data into Redshift and manage data operations using DBT
- Build and manage workflow orchestration using Airflow or Step Functions
- Export and integrate processed data with external systems via S3
- Implement data validation, auditing, and monitoring to ensure data quality and reliability
- Create configurable data pipeline steps using JSON, tables, or similar methods for non-technical users
- Support containerized deployments using Docker and contribute to deployment processes Key Skills & Qualifications:
- 8–11 years of experience in Data Engineering or related roles
- Strong hands-on experience with AWS services such as Glue, Lambda, S3, and Step Functions
- Proficiency in Python, Py. Spark, and advanced SQL
- Experience with DBT for data transformation and modeling
- Expertise in workflow orchestration tools like Airflow or Step Functions
- Hands-on experience with cloud data warehouses such as Redshift, Snowflake, or Big. Query
- Experience handling structured and semi-structured data pipelines
- Knowledge of Shell scripting and containerization using Docker
- Basic understanding of data governance and DevOps deployment practices
- Familiarity with Terraform for infrastructure management (nice to have)
- Strong problem-solving skills and ability to work in fast-paced environments Apply by sending your CV to careers@cliqhr.co.in