Data Engineer - GCP

Key Responsibilities

 Design, develop, and maintain ETL/ELT pipelines using tools such as Dataflow, Dataproc(Apache Beam), Cloud Composer (Airflow) and DBT.  Build and optimize data warehouses and data lakes on BigQuery and Cloud Storage (GCS).  Work with structured and semi-structured data from multiple sources and ensure data quality, lineage, and governance.

 Exposure to CI/CD pipelines for data workflows using Cloud Build, Terraform & helm charts.  Implement & workw ith data transformation frameworks (DBT, SQL, Spark, or Python-based frameworks).  Optimize query performance and storage costs within BigQuery and GCS.  Collaborate with cross-functional teams to define data models, schemas, and partitioning strategies.

 Integrate with third-party APIs and streaming platforms to process real-time data.  Ensure security, compliance, and best practices in data architecture and access management.  Monitor, debug, and maintain existing pipelines to ensure data freshness and reliability.

Requirements

Required Qualifications  Bachelor’s or master’s degree in computer science, information systems, or a related field.  3-5 years of experience as a Data Engineer or in a similar data-focused role.  Strong experience with Google Cloud Platform (GCP) — particularly BigQuery, Cloud Storage, Dataflow, Composer, AlloyDB.

 Proficient in SQL, Python, Airflow, DBT.  Experience with data modelling, schema design, and partitioning strategies.  Working knowledge of version control (Git) and CI/CD workflows.  Familiarity with infrastructure-as-code tools such as Terraform or Terragrunt.  Strong analytical and problem-solving skills with attention to detail.

Preferred Qualifications

 Experience with streaming data pipelines using Pub/Sub, KMS and Cloud Spanner.  Exposure to machine learning pipelines (Vertex AI, AI Platform, or MLflow).  Knowledge of containerization and Kubernetes (GKE).  Familiarity with IAM, data governance, metadata management, and security best practices on GCP.  Experience in building data products for analytics or AI/ML use cases.