Databricks Engineer

We are looking for an experienced Databricks Engineer with 5-8 years of experience in designing, developing, and optimizing large-scale data solutions. The ideal candidate should have strong expertise in Databricks, PySpark, Spark, SQL, cloud platforms (AWS/Azure/GCP), and modern data engineering practices to build scalable and high-performance data pipelines.

Responsibilities

Roles & Responsibilities Design, develop, and maintain data pipelines using Databricks. Build ETL/ELT workflows for large-scale data processing. Develop and optimize data transformation logic using PySpark and Spark SQL. Implement and manage Delta Lake for reliable and efficient data storage. Work with batch and real-time data ingestion frameworks. Monitor and optimize data pipelines for performance and scalability. Collaborate with data analysts, data scientists, and business stakeholders. Ensure data quality, security, and governance standards are followed. Troubleshoot production issues and provide ongoing support.

Requirements

• 5-8 years of experience in Data Engineering and Big Data technologies. • Strong hands-on experience with Databricks, PySpark, and Spark SQL. • Experience with Delta Lake, Databricks Workflows, and Unity Catalog. • Proficiency in Python and SQL. • Experience with Azure/AWS/GCP cloud platforms. • Knowledge of Azure Data Factory (ADF), Apache Airflow, or similar orchestration tools. • Understanding of data warehousing concepts and ETL/ELT methodologies. • Experience with Kafka/Event Hub for streaming data processing is preferred. • Familiarity with Git, CI/CD pipelines, and DevOps practices. • Strong problem-solving, analytical, and communication skills.