4,110 open roles
Senior Data Engineer (GCP)
Job description
We are seeking a Senior Data Engineer (Google Cloud Platform) . You will build reliable ETL and ELT pipelines, optimize SQL and orchestration in Apache Airflow and help teams turn raw data into trusted, ready-to-use datasets. You will partner with clients and delivery teams to improve quality, performance and operational stability across cloud data platforms.
Responsibilities
Design and build scalable ETL and ELT pipelines using Python, SQL,. Big Query and cloud-native services Develop and optimize complex SQL for transformation, validation, troubleshooting and performance Create and maintain data models and warehousing solutions that support quality, availability and scale Orchestrate workflows in Apache Airflow, including scheduling, monitoring and incident resolution Support pipeline delivery and platform operations across Google Cloud Platform and Amazon Web Services (AWS) Improve CI/CD, source control and automated deployment practices for data solution Investigate data, integration and infrastructure issues, driving root-cause analysis and fixes Collaborate with clients and cross-functional teams to clarify requirements, risks, dependencies and delivery plans Requirements Hands-on experience building ETL and ELT pipelines and supporting data warehousing Advanced SQL skill set including optimization, data validation and complex transformation Python proficiency for pipeline development and software engineering practices Experience with distributed processing using Py.
Spark or Apache Beam Practical cloud experience with Google Cloud Platform and or Amazon Web Services (AWS) for data workloads Workflow orchestration background with Apache Airflow or Google Cloud Composer Knowledge of data modeling data lifecycle management and data quality controls Clear communication with the ability to explain trade-offs, risks and solution options to stakeholders Nice to have Infrastructure-as-code exposure using Terraform and or Kubernetes Streaming familiarity with Kafka and event-driven patterns Data integration tooling experience such as Fivetran
Description copied from EPAM Systems's careers page. Read the full posting before you apply.
More jobs at EPAM Systems
Senior Cloud & Infrastructure Engineer (Azure & Terraform)
EPAM Systems· Remote (Canada)Physics Content Specialist
EPAM Systems· Remote (Kazakhstan; Kyrgyzstan)Math Content Specialist
EPAM Systems· Remote (Kazakhstan; Kyrgyzstan)Chemistry Content Specialist
EPAM Systems· Remote (Kazakhstan; Kyrgyzstan)Senior Cloudflare Network & Security Engineer
EPAM Systems· Remote (Turkiye)
More jobs in Kuala Lumpur
Business Support, Intern
Integral UK· Kuala Lumpur, MalaysiaTechnical Specialist, Emerging Technologies
PlanGrid· APAC - Malaysia - Kuala Lumpur - Kuala Lumpur, Avenue 10Technical Lead
Jones Lang LaSalle· Kuala Lumpur, MalaysiaStatutory Senior Analyst - NAM
Baker Hughes· MY-KUL-KUALA LUMPUR-21ST FLOOR MENARA TAN & TAN