Job description
Design, build, and maintain scalable ETL/ELT data pipelines. Develop data processing workflows using Python and PySpark. Work with Databricks and Snowflake for data storage, transformation, and analytics. Integrate data from multiple sources while ensuring data quality and integrity. Optimize SQL queries and improve data pipeline performance.
Collaborate with Data Analysts, Data Scientists, and Software Engineers to deliver data solutions. Monitor, troubleshoot, and maintain data workflows. Document data models, pipelines, and technical processes.
Requirements
Bachelor's degree in Computer Science, Information Technology, Data Science, or a related field. Strong proficiency in Python, PySpark, and SQL. Hands-on experience with Databricks and Snowflake. Good understanding of ETL/ELT pipeline development. Familiarity with PyTorch is an added advantage. Knowledge of data warehousing concepts and cloud platforms is preferred.
Strong analytical and problem-solving skills. Excellent communication and teamwork abilities.