Job description
We are looking for a highly skilled Lead Data Engineer / Architect to design, build, and optimize scalable data pipelines and modern data platforms. This role is critical in transforming raw data into meaningful insights and enabling enterprise-level analytics.
Key Responsibilities
Design, develop, and optimize ETL/ELT pipelines using Python, Py. Spark, Scala, and SQL Build and manage data workflows using Informatica, Databricks, AWS Glue, Snowflake Work across multi-cloud platforms (AWS, Azure, GCP, Microsoft Fabric) Manage and optimize databases such as Redshift, SQL Server, AWS RDS Orchestrate workflows using Apache Airflow Implement real-time streaming solutions using Apache Kafka Deploy and manage containerized applications using Kubernetes Automate processes using Unix shell scripting Implement CI/CD pipelines and ensure data governance Collaborate with BI teams for Tableau, Looker, Power BI Troubleshoot and optimize performance of pipelines and queries Maintain technical documentation Requirements Bachelor’s/Master’s degree in Computer Science or related field 7+ years of experience in Data Engineering Strong skills in Python, Py.
Spark, Scala, SQL Experience with Informatica, Databricks, AWS Glue Expertise in Snowflake & Redshift Experience in AWS / Azure / GCP Experience with Apache Airflow Knowledge of Apache Kafka & Kubernetes Experience with CI/CD pipelines Exposure to BI tools (Tableau, Looker, Power BI)