Job description
Equal Opportunity Employer All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, age, national origin, protected veteran status, or disability status. EEO/Disabled/Vets Job Description : We are looking for a Senior Data Engineer with experience in excellent analytical abilities in GCP Big.
Query, ETL/BI, Python, Composer/Airfl ow, SQL, GCP Services - with the ability to operate in an agile environment and keen to detail while understanding the larger data warehouse. Skills & Competencies ➔ A minimum of 4.5+ years of working experience as a hands-on data engineering is a must ➔ Advanced Degree in the fi eld of Computer Science, Data Engineering, Data Analytics or equivalent discipline ➔ Strong programming experience with Python, SQL, ETL and effi ciently querying big datasets using SQL ➔ Prior experience with CircleCI CI/CD, Terraform and Github version control is a must ➔ Solid understanding of Data Collection, Data Ingestion, Data Integration, and Data Operations & Maintenance ➔ Implement data governance practices, including data quality, security, and compliance, in line with organizational standards.
➔ Domain knowledge on News, Media & Communications is an added advantage. R E S P O N S I B I L I T I E S ➔ Design, build, and maintain scalable ETL/ELT pipelines in Google Cloud, using tools like Cloud Composer (Airfl ow), Dataproc and Big. Query. ➔ Collaborate with cross-functional teams to defi ne data architecture and data warehouse requirements ➔ Automate data pipelines, ETL/ELT processes, and data integration workfl ows using infrastructure-as-code (IaC) and confi guration management tools ➔ Deep understanding of Big.
Query, including SQL, optimization techniques, partitioning, clustering, and performance tuning. ➔ Deliver production quality, modular, reproducible code in Python, SQL ➔ Implement continuous integration and continuous deployment (CI/CD) pipelines for data pipelines and data-related applications ➔ Keep abreast of the latest developments in the fi eld through continuous learning and proactively champion promising new methods relevant to the problems at hand.
Our Tech Stack ● Google Cloud Platform is IaaS ● BigQuery, Data. Proc, Python, Pyspark, SQL ● Composer / Airfl ow - orchestration tools ● Terraform, CircleCI, CI/CD, GIthub, Jira, Confl uence This job is posted with NTS Technology Services Pvt. Ltd. Job Category: